- Email: [email protected]
ARTIFICIAL HAND CONTROL PROBLEMS: HUMAN INTENTION RECOGNITION
ARTIFICIAL HAND CONTROL PROBLEMS: HUMAN INTENTION RECOGNITION Andrzej Wołczowski*, Krzysztof Krysztoforski** *
Institute of Informatics Automatics and Robotics, [email protected] ** Institute of Machine Design and Operations, [email protected] Wroclaw University of Technology, Wybrzeże Wyspiańskiego 27, 50-370 Wrocław, Poland
Abstract: The paper discusses the artificial hand movement control problem, treated as recognition of electromyography signals; it describes the measuring standard and the implementation aspects connected to EMG signals measurements. The methods used for reduction and classification of the EMG signals are presented together with the discussion of the factors affecting of movement decision recognition of human hand. Copyright © 2006 IFAC Keywords: Bio control, Pattern recognition, Signal processing, Neural networks, Artificial hand.
1. INTRODUCTION The dexterity and precision of the human hand come both from its nominal motorial abilities resulting from its kinematic structure, and, above all, from the process, realized by the central nervous system, controlling the structure. In the case of a artificial hand the most natural way to control its motion is to use the signals from the human organism, normally associated with the movement of the original hand. A type of biosignals are electrical potentials accompanying the activity of the skeleton muscles, called electromyography signals (EMG signals). The main advantage of EMG signals is the non-invasive character of their acquisition – since they can be measured directly on the surface of the skin (De Luca, 1997). On the basis of these signals, the intention of the hand movement can be recognized and then realized through executing, by the prosthesis joints, the trajectory – assigned off-line (at the stage of the control process synthesis) – of the recognized decision. Unfortunately, the control process can be performed only in open loop, since the transmission of the information dealing with the tactile image of a grabbed object in the opposite direction, that is from
the artificial hand to the central nervous system, is practically impossible. The artificial hand dexterity requires the concurrent control of many joints, which induces a huge number of classes to be recognized in the EMG signals (generated by many pairs of antagonistic muscles) (Wolczowski, et al., 2005). Moreover taking into account various inferences accompanying the EMG signals measurement, especially when the electrodes are on the surface of the skin, the decision recognition problem is difficult to solve. The research presented in this paper refers to the analysis of the factors influencing the reliability of the recognition of user’s movement decision on the basis of EMG signals collected from the user’s muscles. This work extends authors’ previous papers dealing with the algorithms for EMG signal recognition (Krysztoforski, et al., 2004; Wolczowski, et al., 2005). In sections 2 and 3 the measuring standard as well as the strategy for the EMG signals measurement are described. Section 4 introduces several methods we compared for the problem under investigation.. The analysis of the results we obtained and the conclusions are reported in sections 5 and 6.
2. EMG SIGNAL ACQUISITION TEST-BED For the EMG signal measurements a 8-channels measuring system was constructed (Krysztoforski and Wolczowski, 2004). The electrodes, collecting the analog signal from the surface of the skin, make the input of each channel, whereas the file with the digital record of the measured signal course makes the output. The general conception of the EMG signal acquisition test-bed is shown in figure 1. The test-bed consists of three main blocks: • PC, equipped with a card (Ni 4477, National Instruments); • Filtering, Amplifying and Separating Block (FAS) – seen as an additional box; • active measuring electrodes (in a differential configuration, with a third reference electrode) – fastened on the forearm of the subject.
Fig. 2. The layout of the electrodes on the right forearm. For the analysis of movement recognition ten types of movement were singled out: (1) flexion of the digits of hand, (2) extension of the digits of hand, (3) supination of the forearm, (4) pronation of the forearm, (5) flexion of the wrist, (6) extension of the wrist, (7) flexion of the thumb, (8) extension of the thumb, (9) flexion of the fingers II-V, (1) extension of the fingers II-V. The above mentioned movements correspond to ten recognizable classes of signals. Other parameters of the experiments: 1) each of the singled-out movements (singled-out class of movements) was registered in the period of seven seconds, comprising three time intervals: 1 second for the natural state, 5 seconds for movement (together with the transition from the natural to the active state), 1 second for the transition to the release condition; 2) the result of every seven-second registration is a fragment of a text file comprising 7000 x 6 samples of EMG signal (for 6 channels); 3) the period between successive classes measurements of movement amounted to 1 minute; 4) each series of measurements was preceded by a seven-second record of the release condition.
Fig. 1. A subject during an experiment.
A fragment of the EMG signal is presented in the Fig. 3.
The measurements were executed considering the following parameters: 1) the frequency of the EMG signal sampling: fs =1kHz; 2) the amplification of the FAS block: k =1000V/V; 3) the number of used measuring channels was limited to 6 (6 measuring electrodes); 4) the time of the measuring contact stabilization (after the last electrode has been fastened): ts = 5min. 3.
ORGANIZATION OF THE EXPERIMENTS
Six healthy subject, aged 23-25, marked with symbols P1-P6, participated at the experiments. The research was conducted both for the left (1) and the right (2) arms. The measurement results were indicated as Qij (i- person, j-limb). The effect of the isotonic and isometric muscular tone on the classification results was also examined. The electrodes, connected to the respective measuring channels, were put over the following forearm muscles: (1) – extensor muscle of the fingers, (2) – radial extensor of the wrist, short, (3) – superficial flexor muscle of the fingers, (4) – ulnar flexor muscle of the wrist, (5) – extensor muscle of the thumb, short and (6) – flexor muscle of thumb, long. The layout of the electrodes is shown in fig. 2.
Fig.3. EMG signal. Table 1 The recorded signal file structure channels: samples 1 2 3 . .
1,
2,
3, ...........
6
21194 23795 -21894 ….. 232945 27103 39457 -5343 ….. 260009 12429 14051 29861 …. 257260 …..……………………………… …………………………………..
The measured EMG signals are recorded in text files organized as matrixes. Each line (row of the matrices) contains 6 samples (each from a different channel) registered at the same discrete moment of time. Each column represents successive samples of one signal, measured at a frequency of 1kHz. The signal that was registered in such a way is preanalyzed with respect to the energy aggregated for all 6 components (signals from 6 channels). Thus 256ms frames are selected (256 samples windows) from the signal, in whose first successive 40ms of time the cumulative energy crossed the accepted threshold. The succeeding groups of these frames (preimages) are representative of succeeding movements – they correspond to the distinct class of movement. Depending on the type of movement and the value of the accepted threshold, it is possible to obtain from 20 to 27 frames per one class. 4. DESCRIPTION OF THE METHODS CONSIDERED The movement class recognition through the analysis of the measured EMG signals consists of three stages: features extraction stage, feature selection stage (and the tuning of the selected process parameters) and pattern recognition stage. EMG measurem. sys. (6 channels)
Feature set creation (extract.& reduction) Artif. hand actuator control
Subject’s arm
Recognition system (neural network & fuzzy logic) Recognized class of hand action
Fig. 4. Block diagram of the EMG signal recognition process. For every stage, literature counts several methods that can be exploited. The aim of our study was to single out the best methods assuring the optimisation of the EMG signal classification process. As a criterion of quality the recognition error and the calculation costs were considered. In the analysis the following methods were taken into account: • for the feature extraction: Fast Fourier Transform (FFT) and Discrete Wavelet Transform (DWT) methods (Mallat, 1998); • for the feature reduction: Interpolation (IP), Principal Component Analysis (PCA) and Sequential Backward Selection (SBS) methods (Englehart, 1998; Nishikawa, 2001); • for the feature recognition: k-Nearest Neighbour (kNN) and Learning Vector Quantization (LVQ) methods (Kurzynski, 1997; Kohonen, 1995).
In our research work two models of classifiers were used. The first one is the statistical kNN model. In this algorithm the effect of the variation of the number of neighbours on the recognition error was examined. This method does not require the learning stage and is well suited to the searching for the feature set - with the use of one of the reduction methods. As the second classifier the LVQ neural network proposed by Kohonen (1995) was applied. It is the modification of the self-organising network to that of the one with the supervisor. LVQ is a onelayer network that performs classification according to the nearest neighbour rule. The learning process is performed with the help of two algorithms called LVQ1 and LVQ3 (Kohonen, 1995). For these algorithms two constant learning coefficients are accepted (on the basis of experiments), respectively: α LVQ 1 (t ) = 0 .005 , α LVQ 3 (t ) = 0 .001 . Since it is difficult to analyze separately the methods mentioned above, the effect of different combinations of the methods on the recognition process was successively examined: • FFT+kNN+IP versus FFT+kNN+SBS At the first stage the effect of the feature selection on the classification efficiency was examined. For the feature extraction the Fourier transform was applied, and for the recognition the kNN statistical classifier was used. The two selection methods: Interpolation and Sequential Backward Selection were compared. For the further research the IP method was chosen.. • FFT+IP+kNN versus FFT+IP+LVQ At the second stage the effect of the classifiers was examined. The results obtained for the kNN statistical classifier and the results obtained with the use of the LVQ neural classifier were compared. • DWT+SBS+kNN versus DWT+PCA+kNN At the third stage the effect of the feature selection, obtained this time with the use of the Discrete Wavelet Transform, was examined. Similarly as at the first stage, for the recognition the kNN statistical classifier was used. Two selection methods: Sequential Backward Selection and Principal Component Analysis were compared. The SBS method was chosen. • DWT+SBS+kNN versus DWT+SBS+LVQ Once again the effect of the classifiers – the kNN statistical classifier and LVQ neural classifier – was compared. • FFT+IP+LVQ versus DWT+SBS+LVQ Finally a comparison was drawn between the two methods (the two combinations of extraction/reduction/classification methods) from the presented above, that provided the best recognition results. 5. COMPARISON OF THE FIFE METHODS The results of the comparison between the IP and SBS reduction methods are given in Fig. 5. The mean recognition error, presented in Fig. 5(a), equals 20%, that is an unsatisfactory result. Fig. 5(a) and 5(b) show that the SBS method reduces the recognition error by few points per cent, but it increases the
dimension of the features set more than tenfold. Therefore, in what follows the IP method is considered. izot.
45
FFT/kNN/SBS
35 Error [%]
izom.
FFT/kNN/IP
40
According to the previous remark, we tried to determine which classes are recognized incorrectly and which channels are responsible for the highest error. Then we evaluated, individually for each subject, the effect of the reduction of the classes on the recognition process. The results are presented in Table 2.
30 25
Table 2 Results of classes number and measuring channels reduction
20 15 10 5 0 Q11
Q12
Q21
Q31
Q41
Q52
Q62
Q11
izot.
700
No of features
600
Muscle contract.
Rejected channels
Rejected classes
Error [%]
Q11 Q12 Q21 Q31 Q41 Q52 Q62 Q11 Q21
Izot. Izot. Izot. Izot. Izot. Izot. Izot. Izom. Izom.
2,3 1 1 3,4 1,3 3
7,8,10 1,7,10 2,9 2,4,7 2,3,4,7,9 2,5,10 2,3 1,2,8
9.0 9.5 8.6 8.9 10.0 12.9 10.0 8.9 10.0
Q21
Tested person
a)
Tested subject
izom.
IP SBS
500 400 300 200 100 0 Q11
b))
Q12
Q21
Q31
Q41
Q52
Q62
Q11
Q21
Tested person
Fig. 5. a) The influence of the reduction method on the recognition error; b) The comparison of the features number for chosen reduction method.
The network comprises a minimal number of neurons that is equal to the number of the recognised classes and a maximal number of neurons that is equal to the number of frames per class multiplied by the number of classes. The upper and lower limits define the optimization area of the network structure. The classification results and the number of neurons for an individual class are shown in the Fig. 7. izot.
16 14
izom.
FFT/IP/LVQ
12 Error [%]
As mentioned in Section 3, the course of the EMG signal is divided into 256 sample windows, yielding successively about 20 frames per class. The analysis of the partial recognition errors (Fig. 6) shows that the frames from the beginnings and ends of each class are recognized incorrectly (on average 2 frames per class). Having obtained jointly (for all classes) about 200 frames, we obtain on average 20 frames classified incorrectly, which makes the recognition error equal to 10 per cent. That error (10 per cent) will be accepted as the upper admissible error for a classifier.
After the optimization of the feature sets, we have optimized the number of measuring channels and the number of recognizable classes in order to obtain the recognition error that amounted to about 10%. The parameters obtained in this way were then used for further investigations.
10 8 6 4 2 0 Q11
Q12
Q21
a)
It can be noticed that the 20% error presented in the Fig. 5 refers only to certain being recognised classes. By resigning from some specified movements, the reliability of the artificial hand can be significantly increased. On the basis of the physiological premises we are allowed to suppose that the EMG signals collected form different places on the forearm have different effects on the recognition of different movement classes.
Q41
Q52
Q62
Q11
Q21
Tested person izot.
No of neurons per class
16
Fig. 6. Incorrectly classified initial and final phases (frames) of the hand movements.
Q31
izom.
14 12 10 8 6 4 2 0 Q11
b)
Q12
Q21
Q31
Q41
Q52
Q62
Q11
Q21
Tested person
Fig. 7. a) LVQ classification results; b) Neuron number per individual class. In both cases similar recognition errors were obtained, but the required size of the knowledge base for the neural network classifier is twice as small as that of the statistical classifier (kNN classifier).
The analysis of the performances of the discreet wavelet transform (DWT) was the next step of our work. The feature extraction module for an individual measuring channel implements the fast wavelet algorithm of Mallat (1998). Fig. 8 shows how this algorithm works.
2
Syg. In
Hi Hi
2
2
Hi
2
2
Syg. Out6
Syg. Out1
Syg. Out0
Fig. 8. The diagram showing the calculation of the wavelet decomposition coefficients by means of Mallat algorithm. Among many base functions used in the wavelet transform, it has been decided upon to examine four families: Daubechies, Coiflets, Symmlets and Biorthogonal. Seven levels of Mallat decomposition (levels 0 ÷ 6) for an individual frame were accepted, which corresponds to the range of the analysed frequencies from 4-500Hz. The coefficients obtained at the successive levels of filtration make up other feature sets that after the concatenation can be jointly treated as the resultant feature vector. However, as the examinations results suggest so large a number of features is unfavourable. To reduce the total number of features, each set of features obtained at successive levels of filtration is substituted for by an individual feature through calculating the resultant feature energy (counted as the average of the module of consecutive responses of the filter). Thus, the resultant feature vector for an individual EMG signal consists of seven elements. The algorithm of the artificial hand control uses 6 EMG signals (6 measuring channels). Hence the feature extraction process gives a 42-element feature vector in every cycle.
Fig. 9. The adjustment of the sym7 wavelet to the EMG signal Then the feature reduction was executed by means of the PCA and SBS methods. The recognition results for the kNN classifier was examined. The results are presented in Fig. 10.
12
Wavelet sym7,coif3 sym2 db3, sym3 db2, sym2 db7 sym2
10 8 6 4 2 0 Q11 Q12 Q21 Q31 Q41 Q52 Q62 Q11 Q21 Tested person
a) 25
izot.
SBS
izom.
PCA 20 15 10 5 0 Q11
Q12
Q21
Q31
b)
Q41
Q52
Q62
Q11
Q21
Tested person
Fig. 10. a) The effect of the feature set reduction on the classification error; b) The number of features in a vector giving the least recognition error.
Error [%]
10 9 8 7 6 5 4 3 2 1 0
izot.
izom.
DWT/SBS/LVQ
Q11 Q12 Q21 Q31 Q41 Q52 Q62 Q11 Q21 Tested person
a)
Error [%] 10.9 10.0 5.9 10.5 13.8 10.0
izot.
14 No of neurons per class
Tested person P1 P2 P3 P4 P5 P6
izom.
DWT/kNN/SBS DWT/kNN/PCA
14
Every family of wavelets was examined. As the criterion in the process of the selection of the best wavelet the least recognition error has been decided upon, while the parameters resulting from Table 2 was used. The results for particular persons and wavelets are gathered in the Table 3. Table 3 The results of the optimisation of the base function
izot.
16
Error [%]
Lo
Lo
2
No of features
Lo
izom.
12 10 8 6 4 2 0 Q11
Figure 9 constitutes the illustration for Table 3. It shows that for the person P1 the adjustment of the sym7 wavelet is very close to the EMG signal course, generated by the person.
b)
Q12
Q21
Q31
Q41
Q52
Q62
Q11
Q21
Tested person
Fig. 11. a) The results of the LVQ classification; b) The number of neurons for an individual class.
Next the features obtained with the help of SBS method were chosen for the testing of the LVQ classifier. The features yielded the least classification error (although the feature number was in most cases larger than for the PCA method). The obtained results are presented in figure 11. 6. CONCLUSIONS The presented analysis shows that the recognition quality is affected by many factors as: the selection of the extraction method, the selection of the wavelet in the DWT extraction method, the selection of the feature reduction method, the selection of the classification method, the number of the neighbours in the kNN classifier, the number of the neurons in the LVQ classifier as well as the selection of the electrodes displacement, the number of channels and recognised classes with regard to the individual features of the examined persons. The effect of the factors was the subject of the research work that was carried out. In individual cases some sets of parameters bring about better recognition than others (for each subject we chose individual parameters according to Table 2.). However, it can be said that the wavelet representation is able to generate a set of most significant features. For the EMG signals in most cases the best correlation is found for the Symmlets and Daubechies families of wavelets. 14,7
15 13
DWT/SBS
Error[%]
11 9
12,9
FFT/IP 9,3
8,9
8,1 7,7 6
7
9,2 9,2
8,4
7,4
5,4
5 3 1 -1
P1
P2
P3
P4
P5
P6
Tested person
a)
84
No of features
90 80
FFT/IP
70
DWT/SBS
65
60 50 40
36
30 20
20 13
7
10
11
12 15
20 8
12
0 P1
b)
P2
P3
P4
P5
P6
Tested person
Fig. 12. a) The recognition error for the LVQ classifier with the application of the FFT/IP and DWT/SBS methods, b) The comparison of the feature number used in the recognition process. The experiments that have been carried out have shown that the best results are provided by the team DWT/SBS generating feature sets of the least size and least recognition error for the LVQ classifier.
The comparison of the methods is presented in Fig. 12. The optimal number k in the kNN classifier was equal to 3. The results of the classification obtained by means of LVQ network are comparable to the kNN classifier, however the neutral classifier gains advantage (about 50%) in the sphere of the amount of information remembered by the system. Eventually, the obtained mean recognition error equals to 7%. Another step in the research work will be the experiments that deal with the implementation of the obtained algorithms which perform recognition on the microcontroller. We will also analyse the implementations in the dedicated FPGA systems. It has been also noticed that the initial and final periods of the electromyographic activity, during the hand movements, are of the chaotic kind and thus are the cause of the increase in the recognition errors. The detection of such a situation was not the subject of the research work but it may lead to further improvement of the recognition efficiency. REFERENCES De Luca, C.J. (1997). The use of Surface Electromyography in Biomechanics. Journal of Applied Biomechanics, Vol. 13, No. 2. Englehart, K. (1998). Signal representation for classification of the transient myoelectric signal In: Ph.D. Thesis, University of New Brunswick, Fredericton, New Brunswick. Kohonen, T.K. (1995). Self-Organizing Maps. Springer, Berlin. Krysztoforski, K. and A. Wolczowski (2004). Laboratory Test-Bed for EMG Signal Measurement (in Polish). In: Postepy Robotyki (K. Tchon. (Ed)), 203-212. WKiL, Warszawa. Krysztoforski K., A. Wołczowski and S. Busz (2004) Hand finger posture recognition using EMG signals (in Polish). In: Postepy Robotyki (K. Tchon. (Ed)), 203-212. WKiL, Warszawa. Kurzynski, M.: Pattern Recognition. Statistical Methods, (in Polish), Oficyna Wydawnicza PWr, Wroclaw, 1997. Mallat, S. (1998). A wavelet tour of signal processing. Academic Press, San Diego. Nishikawa, D. (2001). Studies on Electromyogram to Motion Classifier, In: Ph.D. Thesis, Graduate school of engineering, Hokkaido University, Sapporo. Wołczowski A., P. Szecówka, K. Krysztoforski and M. Kowalski (2005). On hardware implementation of the artificial hand control algorithm. 11 IEEE International Conference on Methods and Models in Automation and Robotics, 665670, Międzyzdroje.
Institute of Informatics Automatics and Robotics, [email protected] ** Institute of Machine Design and Operations, [email protected] Wroclaw University of Technology, Wybrzeże Wyspiańskiego 27, 50-370 Wrocław, Poland
Abstract: The paper discusses the artificial hand movement control problem, treated as recognition of electromyography signals; it describes the measuring standard and the implementation aspects connected to EMG signals measurements. The methods used for reduction and classification of the EMG signals are presented together with the discussion of the factors affecting of movement decision recognition of human hand. Copyright © 2006 IFAC Keywords: Bio control, Pattern recognition, Signal processing, Neural networks, Artificial hand.
1. INTRODUCTION The dexterity and precision of the human hand come both from its nominal motorial abilities resulting from its kinematic structure, and, above all, from the process, realized by the central nervous system, controlling the structure. In the case of a artificial hand the most natural way to control its motion is to use the signals from the human organism, normally associated with the movement of the original hand. A type of biosignals are electrical potentials accompanying the activity of the skeleton muscles, called electromyography signals (EMG signals). The main advantage of EMG signals is the non-invasive character of their acquisition – since they can be measured directly on the surface of the skin (De Luca, 1997). On the basis of these signals, the intention of the hand movement can be recognized and then realized through executing, by the prosthesis joints, the trajectory – assigned off-line (at the stage of the control process synthesis) – of the recognized decision. Unfortunately, the control process can be performed only in open loop, since the transmission of the information dealing with the tactile image of a grabbed object in the opposite direction, that is from
the artificial hand to the central nervous system, is practically impossible. The artificial hand dexterity requires the concurrent control of many joints, which induces a huge number of classes to be recognized in the EMG signals (generated by many pairs of antagonistic muscles) (Wolczowski, et al., 2005). Moreover taking into account various inferences accompanying the EMG signals measurement, especially when the electrodes are on the surface of the skin, the decision recognition problem is difficult to solve. The research presented in this paper refers to the analysis of the factors influencing the reliability of the recognition of user’s movement decision on the basis of EMG signals collected from the user’s muscles. This work extends authors’ previous papers dealing with the algorithms for EMG signal recognition (Krysztoforski, et al., 2004; Wolczowski, et al., 2005). In sections 2 and 3 the measuring standard as well as the strategy for the EMG signals measurement are described. Section 4 introduces several methods we compared for the problem under investigation.. The analysis of the results we obtained and the conclusions are reported in sections 5 and 6.
2. EMG SIGNAL ACQUISITION TEST-BED For the EMG signal measurements a 8-channels measuring system was constructed (Krysztoforski and Wolczowski, 2004). The electrodes, collecting the analog signal from the surface of the skin, make the input of each channel, whereas the file with the digital record of the measured signal course makes the output. The general conception of the EMG signal acquisition test-bed is shown in figure 1. The test-bed consists of three main blocks: • PC, equipped with a card (Ni 4477, National Instruments); • Filtering, Amplifying and Separating Block (FAS) – seen as an additional box; • active measuring electrodes (in a differential configuration, with a third reference electrode) – fastened on the forearm of the subject.
Fig. 2. The layout of the electrodes on the right forearm. For the analysis of movement recognition ten types of movement were singled out: (1) flexion of the digits of hand, (2) extension of the digits of hand, (3) supination of the forearm, (4) pronation of the forearm, (5) flexion of the wrist, (6) extension of the wrist, (7) flexion of the thumb, (8) extension of the thumb, (9) flexion of the fingers II-V, (1) extension of the fingers II-V. The above mentioned movements correspond to ten recognizable classes of signals. Other parameters of the experiments: 1) each of the singled-out movements (singled-out class of movements) was registered in the period of seven seconds, comprising three time intervals: 1 second for the natural state, 5 seconds for movement (together with the transition from the natural to the active state), 1 second for the transition to the release condition; 2) the result of every seven-second registration is a fragment of a text file comprising 7000 x 6 samples of EMG signal (for 6 channels); 3) the period between successive classes measurements of movement amounted to 1 minute; 4) each series of measurements was preceded by a seven-second record of the release condition.
Fig. 1. A subject during an experiment.
A fragment of the EMG signal is presented in the Fig. 3.
The measurements were executed considering the following parameters: 1) the frequency of the EMG signal sampling: fs =1kHz; 2) the amplification of the FAS block: k =1000V/V; 3) the number of used measuring channels was limited to 6 (6 measuring electrodes); 4) the time of the measuring contact stabilization (after the last electrode has been fastened): ts = 5min. 3.
ORGANIZATION OF THE EXPERIMENTS
Six healthy subject, aged 23-25, marked with symbols P1-P6, participated at the experiments. The research was conducted both for the left (1) and the right (2) arms. The measurement results were indicated as Qij (i- person, j-limb). The effect of the isotonic and isometric muscular tone on the classification results was also examined. The electrodes, connected to the respective measuring channels, were put over the following forearm muscles: (1) – extensor muscle of the fingers, (2) – radial extensor of the wrist, short, (3) – superficial flexor muscle of the fingers, (4) – ulnar flexor muscle of the wrist, (5) – extensor muscle of the thumb, short and (6) – flexor muscle of thumb, long. The layout of the electrodes is shown in fig. 2.
Fig.3. EMG signal. Table 1 The recorded signal file structure channels: samples 1 2 3 . .
1,
2,
3, ...........
6
21194 23795 -21894 ….. 232945 27103 39457 -5343 ….. 260009 12429 14051 29861 …. 257260 …..……………………………… …………………………………..
The measured EMG signals are recorded in text files organized as matrixes. Each line (row of the matrices) contains 6 samples (each from a different channel) registered at the same discrete moment of time. Each column represents successive samples of one signal, measured at a frequency of 1kHz. The signal that was registered in such a way is preanalyzed with respect to the energy aggregated for all 6 components (signals from 6 channels). Thus 256ms frames are selected (256 samples windows) from the signal, in whose first successive 40ms of time the cumulative energy crossed the accepted threshold. The succeeding groups of these frames (preimages) are representative of succeeding movements – they correspond to the distinct class of movement. Depending on the type of movement and the value of the accepted threshold, it is possible to obtain from 20 to 27 frames per one class. 4. DESCRIPTION OF THE METHODS CONSIDERED The movement class recognition through the analysis of the measured EMG signals consists of three stages: features extraction stage, feature selection stage (and the tuning of the selected process parameters) and pattern recognition stage. EMG measurem. sys. (6 channels)
Feature set creation (extract.& reduction) Artif. hand actuator control
Subject’s arm
Recognition system (neural network & fuzzy logic) Recognized class of hand action
Fig. 4. Block diagram of the EMG signal recognition process. For every stage, literature counts several methods that can be exploited. The aim of our study was to single out the best methods assuring the optimisation of the EMG signal classification process. As a criterion of quality the recognition error and the calculation costs were considered. In the analysis the following methods were taken into account: • for the feature extraction: Fast Fourier Transform (FFT) and Discrete Wavelet Transform (DWT) methods (Mallat, 1998); • for the feature reduction: Interpolation (IP), Principal Component Analysis (PCA) and Sequential Backward Selection (SBS) methods (Englehart, 1998; Nishikawa, 2001); • for the feature recognition: k-Nearest Neighbour (kNN) and Learning Vector Quantization (LVQ) methods (Kurzynski, 1997; Kohonen, 1995).
In our research work two models of classifiers were used. The first one is the statistical kNN model. In this algorithm the effect of the variation of the number of neighbours on the recognition error was examined. This method does not require the learning stage and is well suited to the searching for the feature set - with the use of one of the reduction methods. As the second classifier the LVQ neural network proposed by Kohonen (1995) was applied. It is the modification of the self-organising network to that of the one with the supervisor. LVQ is a onelayer network that performs classification according to the nearest neighbour rule. The learning process is performed with the help of two algorithms called LVQ1 and LVQ3 (Kohonen, 1995). For these algorithms two constant learning coefficients are accepted (on the basis of experiments), respectively: α LVQ 1 (t ) = 0 .005 , α LVQ 3 (t ) = 0 .001 . Since it is difficult to analyze separately the methods mentioned above, the effect of different combinations of the methods on the recognition process was successively examined: • FFT+kNN+IP versus FFT+kNN+SBS At the first stage the effect of the feature selection on the classification efficiency was examined. For the feature extraction the Fourier transform was applied, and for the recognition the kNN statistical classifier was used. The two selection methods: Interpolation and Sequential Backward Selection were compared. For the further research the IP method was chosen.. • FFT+IP+kNN versus FFT+IP+LVQ At the second stage the effect of the classifiers was examined. The results obtained for the kNN statistical classifier and the results obtained with the use of the LVQ neural classifier were compared. • DWT+SBS+kNN versus DWT+PCA+kNN At the third stage the effect of the feature selection, obtained this time with the use of the Discrete Wavelet Transform, was examined. Similarly as at the first stage, for the recognition the kNN statistical classifier was used. Two selection methods: Sequential Backward Selection and Principal Component Analysis were compared. The SBS method was chosen. • DWT+SBS+kNN versus DWT+SBS+LVQ Once again the effect of the classifiers – the kNN statistical classifier and LVQ neural classifier – was compared. • FFT+IP+LVQ versus DWT+SBS+LVQ Finally a comparison was drawn between the two methods (the two combinations of extraction/reduction/classification methods) from the presented above, that provided the best recognition results. 5. COMPARISON OF THE FIFE METHODS The results of the comparison between the IP and SBS reduction methods are given in Fig. 5. The mean recognition error, presented in Fig. 5(a), equals 20%, that is an unsatisfactory result. Fig. 5(a) and 5(b) show that the SBS method reduces the recognition error by few points per cent, but it increases the
dimension of the features set more than tenfold. Therefore, in what follows the IP method is considered. izot.
45
FFT/kNN/SBS
35 Error [%]
izom.
FFT/kNN/IP
40
According to the previous remark, we tried to determine which classes are recognized incorrectly and which channels are responsible for the highest error. Then we evaluated, individually for each subject, the effect of the reduction of the classes on the recognition process. The results are presented in Table 2.
30 25
Table 2 Results of classes number and measuring channels reduction
20 15 10 5 0 Q11
Q12
Q21
Q31
Q41
Q52
Q62
Q11
izot.
700
No of features
600
Muscle contract.
Rejected channels
Rejected classes
Error [%]
Q11 Q12 Q21 Q31 Q41 Q52 Q62 Q11 Q21
Izot. Izot. Izot. Izot. Izot. Izot. Izot. Izom. Izom.
2,3 1 1 3,4 1,3 3
7,8,10 1,7,10 2,9 2,4,7 2,3,4,7,9 2,5,10 2,3 1,2,8
9.0 9.5 8.6 8.9 10.0 12.9 10.0 8.9 10.0
Q21
Tested person
a)
Tested subject
izom.
IP SBS
500 400 300 200 100 0 Q11
b))
Q12
Q21
Q31
Q41
Q52
Q62
Q11
Q21
Tested person
Fig. 5. a) The influence of the reduction method on the recognition error; b) The comparison of the features number for chosen reduction method.
The network comprises a minimal number of neurons that is equal to the number of the recognised classes and a maximal number of neurons that is equal to the number of frames per class multiplied by the number of classes. The upper and lower limits define the optimization area of the network structure. The classification results and the number of neurons for an individual class are shown in the Fig. 7. izot.
16 14
izom.
FFT/IP/LVQ
12 Error [%]
As mentioned in Section 3, the course of the EMG signal is divided into 256 sample windows, yielding successively about 20 frames per class. The analysis of the partial recognition errors (Fig. 6) shows that the frames from the beginnings and ends of each class are recognized incorrectly (on average 2 frames per class). Having obtained jointly (for all classes) about 200 frames, we obtain on average 20 frames classified incorrectly, which makes the recognition error equal to 10 per cent. That error (10 per cent) will be accepted as the upper admissible error for a classifier.
After the optimization of the feature sets, we have optimized the number of measuring channels and the number of recognizable classes in order to obtain the recognition error that amounted to about 10%. The parameters obtained in this way were then used for further investigations.
10 8 6 4 2 0 Q11
Q12
Q21
a)
It can be noticed that the 20% error presented in the Fig. 5 refers only to certain being recognised classes. By resigning from some specified movements, the reliability of the artificial hand can be significantly increased. On the basis of the physiological premises we are allowed to suppose that the EMG signals collected form different places on the forearm have different effects on the recognition of different movement classes.
Q41
Q52
Q62
Q11
Q21
Tested person izot.
No of neurons per class
16
Fig. 6. Incorrectly classified initial and final phases (frames) of the hand movements.
Q31
izom.
14 12 10 8 6 4 2 0 Q11
b)
Q12
Q21
Q31
Q41
Q52
Q62
Q11
Q21
Tested person
Fig. 7. a) LVQ classification results; b) Neuron number per individual class. In both cases similar recognition errors were obtained, but the required size of the knowledge base for the neural network classifier is twice as small as that of the statistical classifier (kNN classifier).
The analysis of the performances of the discreet wavelet transform (DWT) was the next step of our work. The feature extraction module for an individual measuring channel implements the fast wavelet algorithm of Mallat (1998). Fig. 8 shows how this algorithm works.
2
Syg. In
Hi Hi
2
2
Hi
2
2
Syg. Out6
Syg. Out1
Syg. Out0
Fig. 8. The diagram showing the calculation of the wavelet decomposition coefficients by means of Mallat algorithm. Among many base functions used in the wavelet transform, it has been decided upon to examine four families: Daubechies, Coiflets, Symmlets and Biorthogonal. Seven levels of Mallat decomposition (levels 0 ÷ 6) for an individual frame were accepted, which corresponds to the range of the analysed frequencies from 4-500Hz. The coefficients obtained at the successive levels of filtration make up other feature sets that after the concatenation can be jointly treated as the resultant feature vector. However, as the examinations results suggest so large a number of features is unfavourable. To reduce the total number of features, each set of features obtained at successive levels of filtration is substituted for by an individual feature through calculating the resultant feature energy (counted as the average of the module of consecutive responses of the filter). Thus, the resultant feature vector for an individual EMG signal consists of seven elements. The algorithm of the artificial hand control uses 6 EMG signals (6 measuring channels). Hence the feature extraction process gives a 42-element feature vector in every cycle.
Fig. 9. The adjustment of the sym7 wavelet to the EMG signal Then the feature reduction was executed by means of the PCA and SBS methods. The recognition results for the kNN classifier was examined. The results are presented in Fig. 10.
12
Wavelet sym7,coif3 sym2 db3, sym3 db2, sym2 db7 sym2
10 8 6 4 2 0 Q11 Q12 Q21 Q31 Q41 Q52 Q62 Q11 Q21 Tested person
a) 25
izot.
SBS
izom.
PCA 20 15 10 5 0 Q11
Q12
Q21
Q31
b)
Q41
Q52
Q62
Q11
Q21
Tested person
Fig. 10. a) The effect of the feature set reduction on the classification error; b) The number of features in a vector giving the least recognition error.
Error [%]
10 9 8 7 6 5 4 3 2 1 0
izot.
izom.
DWT/SBS/LVQ
Q11 Q12 Q21 Q31 Q41 Q52 Q62 Q11 Q21 Tested person
a)
Error [%] 10.9 10.0 5.9 10.5 13.8 10.0
izot.
14 No of neurons per class
Tested person P1 P2 P3 P4 P5 P6
izom.
DWT/kNN/SBS DWT/kNN/PCA
14
Every family of wavelets was examined. As the criterion in the process of the selection of the best wavelet the least recognition error has been decided upon, while the parameters resulting from Table 2 was used. The results for particular persons and wavelets are gathered in the Table 3. Table 3 The results of the optimisation of the base function
izot.
16
Error [%]
Lo
Lo
2
No of features
Lo
izom.
12 10 8 6 4 2 0 Q11
Figure 9 constitutes the illustration for Table 3. It shows that for the person P1 the adjustment of the sym7 wavelet is very close to the EMG signal course, generated by the person.
b)
Q12
Q21
Q31
Q41
Q52
Q62
Q11
Q21
Tested person
Fig. 11. a) The results of the LVQ classification; b) The number of neurons for an individual class.
Next the features obtained with the help of SBS method were chosen for the testing of the LVQ classifier. The features yielded the least classification error (although the feature number was in most cases larger than for the PCA method). The obtained results are presented in figure 11. 6. CONCLUSIONS The presented analysis shows that the recognition quality is affected by many factors as: the selection of the extraction method, the selection of the wavelet in the DWT extraction method, the selection of the feature reduction method, the selection of the classification method, the number of the neighbours in the kNN classifier, the number of the neurons in the LVQ classifier as well as the selection of the electrodes displacement, the number of channels and recognised classes with regard to the individual features of the examined persons. The effect of the factors was the subject of the research work that was carried out. In individual cases some sets of parameters bring about better recognition than others (for each subject we chose individual parameters according to Table 2.). However, it can be said that the wavelet representation is able to generate a set of most significant features. For the EMG signals in most cases the best correlation is found for the Symmlets and Daubechies families of wavelets. 14,7
15 13
DWT/SBS
Error[%]
11 9
12,9
FFT/IP 9,3
8,9
8,1 7,7 6
7
9,2 9,2
8,4
7,4
5,4
5 3 1 -1
P1
P2
P3
P4
P5
P6
Tested person
a)
84
No of features
90 80
FFT/IP
70
DWT/SBS
65
60 50 40
36
30 20
20 13
7
10
11
12 15
20 8
12
0 P1
b)
P2
P3
P4
P5
P6
Tested person
Fig. 12. a) The recognition error for the LVQ classifier with the application of the FFT/IP and DWT/SBS methods, b) The comparison of the feature number used in the recognition process. The experiments that have been carried out have shown that the best results are provided by the team DWT/SBS generating feature sets of the least size and least recognition error for the LVQ classifier.
The comparison of the methods is presented in Fig. 12. The optimal number k in the kNN classifier was equal to 3. The results of the classification obtained by means of LVQ network are comparable to the kNN classifier, however the neutral classifier gains advantage (about 50%) in the sphere of the amount of information remembered by the system. Eventually, the obtained mean recognition error equals to 7%. Another step in the research work will be the experiments that deal with the implementation of the obtained algorithms which perform recognition on the microcontroller. We will also analyse the implementations in the dedicated FPGA systems. It has been also noticed that the initial and final periods of the electromyographic activity, during the hand movements, are of the chaotic kind and thus are the cause of the increase in the recognition errors. The detection of such a situation was not the subject of the research work but it may lead to further improvement of the recognition efficiency. REFERENCES De Luca, C.J. (1997). The use of Surface Electromyography in Biomechanics. Journal of Applied Biomechanics, Vol. 13, No. 2. Englehart, K. (1998). Signal representation for classification of the transient myoelectric signal In: Ph.D. Thesis, University of New Brunswick, Fredericton, New Brunswick. Kohonen, T.K. (1995). Self-Organizing Maps. Springer, Berlin. Krysztoforski, K. and A. Wolczowski (2004). Laboratory Test-Bed for EMG Signal Measurement (in Polish). In: Postepy Robotyki (K. Tchon. (Ed)), 203-212. WKiL, Warszawa. Krysztoforski K., A. Wołczowski and S. Busz (2004) Hand finger posture recognition using EMG signals (in Polish). In: Postepy Robotyki (K. Tchon. (Ed)), 203-212. WKiL, Warszawa. Kurzynski, M.: Pattern Recognition. Statistical Methods, (in Polish), Oficyna Wydawnicza PWr, Wroclaw, 1997. Mallat, S. (1998). A wavelet tour of signal processing. Academic Press, San Diego. Nishikawa, D. (2001). Studies on Electromyogram to Motion Classifier, In: Ph.D. Thesis, Graduate school of engineering, Hokkaido University, Sapporo. Wołczowski A., P. Szecówka, K. Krysztoforski and M. Kowalski (2005). On hardware implementation of the artificial hand control algorithm. 11 IEEE International Conference on Methods and Models in Automation and Robotics, 665670, Międzyzdroje.