- Email: [email protected]
Fuzzy Classification of Hand’s Motion
Proceedings, 15th IFAC Conference on Proceedings, 15th IFAC and Conference on Systems Programmable Devices Embedded Proceedings, 15th 15th IFAC Conference Conference on Proceedings, IFAC on Programmable Devices and Embedded Systems Ostrava, Czech Republic, May 23-25, 2018 Available online at www.sciencedirect.com Programmable Devices and Embedded Systems Programmable Devices Embedded Systems Proceedings, 15th IFAC and Conference Ostrava, Czech Republic, May 23-25,on 2018 Ostrava, Czech Republic, May 23-25, 2018 Ostrava, Czech Devices Republic, May 23-25, 2018 Programmable and Embedded Systems Ostrava, Czech Republic, May 23-25, 2018
ScienceDirect
IFAC PapersOnLine 51-6 (2018) 354–359
Fuzzy Classification of Hand’s Motion Fuzzy Fuzzy Classification Classification of of Hand’s Hand’s Motion Motion ∗ ∗ Fuzzy Classification of Hand’s Motion Lukas Peter Filip Maryncak Antotnino Proto ∗∗ ∗ ∗
Lukas Peter ∗∗ Filip Maryncak Antotnino Proto ∗∗ ∗ ∗ ∗ Lukas Maryncak Antotnino Martin Cerny Lukas Peter Peter Filip Filip Maryncak Antotnino Proto Proto ∗ Martin Cerny ∗ ∗ ∗ Martin Cerny Lukas Peter ∗ Filip Maryncak Antotnino Proto ∗ Martin Cerny ∗ ∗ of Ostrava, Department of Cybernetics and Cerny ∗ VSB-Technical UniversityMartin ∗ VSB-Technical University of Ostrava, Department of Cybernetics and ∗ VSB-Technical University of Ostrava, Ostrava, Department of Republic, Cybernetics and and BiomedicalUniversity Engineering, OstravaDepartment 70800, Czech VSB-Technical of of Cybernetics Biomedical Engineering, Ostrava 70800, Czech Republic, ∗ Biomedical Engineering, Ostrava 70800, Czech Republic, (e-mail:[email protected]) VSB-Technical of Ostrava, Department of Cybernetics and BiomedicalUniversity Engineering, Ostrava 70800, Czech Republic, (e-mail:[email protected]) (e-mail:[email protected]) Biomedical Engineering, Ostrava 70800, Czech Republic, (e-mail:[email protected]) Abstract: The goal of this work (e-mail:[email protected]) was to create the measurement circuit that would be able to Abstract: The goal of of this of work was to create create the measurement circuit that that would be able able to Abstract: The goal this work was the circuit would measure and classify signal myopotentials to classify specific gestures hand.Realization of Abstract: The goal of this of work was to to create the measurement measurement circuit of that would be be able to to measure and classify signal myopotentials to classify specific gestures of hand.Realization of measure and classify signal of myopotentials tois classify classify specific gestures of hand.Realization of the system for classification hand’s gestures described in thisgestures paper. Hardware prototype Abstract: The goal of this work was to create the measurement circuit that would be able to measure and classify signal of myopotentials to specific of hand.Realization of the system for classification classification of hand’s hand’s gestures is described described in thisorder paper. Hardware prototype of the system for of gestures is in this paper. Hardware prototype of four measuring channels was created by combination of 2nd filters and right amount measure and signalwas of hand’s myopotentials tois classify specific gestures of hand.Realization of the system forclassify classification gestures described in thisorder paper. Hardware prototype four measuring channels created by combination of 2nd filters and right amount four measuring channels was created by combination of 2nd filters and amount amplification. For digitizing data, Arduino Nano microcontroller was right selected and the system for classification of the hand’s gestures is described thisorder paper. Hardware prototype of four measuring channels was created bythe combination of in 2nd order filters and right amount amplification. For digitizing the data, the Arduino Nano microcontroller was selected and amplification. For digitizing the data, the Arduino Nano microcontroller was selected and programmed using defined communication protocol. The computer software is programmed four measuring channels was created by combination of 2nd order filters and right amount amplification. For digitizing the data, the Arduino Nano microcontroller was selected and programmed using defined defined communication protocol. The computer computer software is programmed programmed using communication protocol. The software is programmed in C# programming language. Signal processing and drawing to user software interface is selected in real time. amplification. For digitizing the data, the protocol. Arduino Nano microcontroller was and programmed using defined communication The computer is in C#one programming language. Signal processing and drawing tofuzzy userlogic interface is programmed in real realsystem time. in C# programming language. Signal processing and drawing to user interface is in time. The of five possible gestures that user made is chosen using and designed programmed using defined communication protocol. The computer software is programmed in C# programming language. Signal processing and drawing to user interface is in real time. The one of of five possible possible gestures gestures that that user made made is chosen chosen using using fuzzy logic logic and and designed system system The of scaling. in C#one programming language. processing drawing userlogic interface is in realsystem time. The one of five five possible gesturesSignal that user user made is isand chosen usingtofuzzy fuzzy and designed designed of scaling. of scaling. The one of five possible gestures that user made is chosen using fuzzy logic and designed system of scaling. © 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. of scaling. electromyography, fuzzy, classiffication of myopotentials, hands gestures. Keywords: Keywords: electromyography, fuzzy, classiffication of myopotentials, hands gestures. Keywords: electromyography, fuzzy, fuzzy, classiffication classiffication of of myopotentials, myopotentials, hands hands gestures. gestures. Keywords: electromyography, Keywords: electromyography, fuzzy, classiffication of myopotentials, hands gestures. 1. INTRODUCTION gesture classification. The communication protocol was 1. INTRODUCTION INTRODUCTION gesture classification. TheNano, communication protocol was 1. gesture classification. The communication protocol was created between Arduino that used protocol for digitizing 1. INTRODUCTION gesture classification. The communication was created between Arduino Nano, that used for digitizing Nowadays the wearable technologies are trying to simplify gesture created between Arduino that for the signal and octet stuffing, and computer. the algo1. INTRODUCTION classification. TheNano, communication protocol was created between Arduino Nano, that used usedThat for digitizing digitizing Nowadays the wearable wearable technologies are trying to simplify simplify the signal and octet stuffing, and computer. computer. That the algoalgoNowadays the technologies are trying to everyday activities of their users are are trying to collect data created the signal and octet stuffing, and That the rithm for adaptive segmentation was designed. Adaptive Nowadays the wearable technologies trying to simplify between Arduino Nano, that used for digitizing the signal and octet segmentation stuffing, and computer. That Adaptive the algoeveryday activities of their users are trying to collect data rithm for adaptive was designed. everyday activities of their users are are trying to collect collect data the during theactivities daywearable and of night. Under the are continuous rithm for and adaptive segmentation was designed. designed. Adaptive segmentation allows to detect inThat both Nowadays the technologies trying tocollection simplify everyday their users trying to data signal octet segmentation stuffing, anddifferences computer. theamplialgorithm for adaptive was Adaptive during the daycan andimagine night. Under the watch, continuous collection segmentation allows toreal detect differences inchannels. both ampliduring the day and night. Under the continuous collection of data, you a smart that measure segmentation allows to detect differences in both amplitude and frequency into timedifferences forwas multiple Afeveryday activities of their users are trying to collect data during the day and night. Under the continuous collection rithm for adaptive segmentation designed. Adaptive segmentation allows detect in both ampliof data, you cancount imagine asteps smart watch, that measure tude and frequency insegmentation, real time time for for multiple multiple channels. Afof data, you can imagine a smart watch, that measure your heart rate, your and based on that data tude and frequency in real channels. After the right gesture the two phase gesture during the day and night. Under the continuous collection of data, you cancount imagine asteps smart watch, that measure segmentation allowsinsegmentation, toreal detect inphase both gesture amplitude and frequency timedifferences for multiple channels. Afyour heart rate, your and based on that data ter the right gesture the two your heart rate, count yourasteps steps and based on that that data tude evaluates your daily activity. But and what you that could control ter the right gesture segmentation, thecorrect two phase phase gesture classification is usedin to determine gesture (Beof data, you can imagine smart watch, measure your heart rate, count your based on data and frequency real time forthe multiple channels. After the right gesture segmentation, the two gesture evaluates your daily activity. But what youphone could control classification is used used to to determine the correct gesture (Beevaluates your daily activity. But what you could control applications in daily your computer orand mobile just by ter classification is determine the correct gesture (Benatti et right al., 2015). your heart rate, count your steps based on that data evaluates your activity. But what you could control the gesture segmentation, the two phase gesture classification is used to determine the correct gesture (Beapplications in your your computer or mobile mobile phone just by natti et al., 2015). applications in computer or phone just by making a gesture of your hand? This is exactly the just subject natti et et al., al., 2015). 2015). evaluates your daily activity. But what youphone could control applications in your computer or mobile by classification is used to determine the correct gesture (Benatti making a gesture of your hand? This is exactly the subject making a gesture gesture of your your hand?and This is exactly exactly the just subject matter of thisinpaper (Merletti Parker, 2004). applications your computer or mobile phone by making a of hand? This is the subject natti et al., 2015). 2. HARDWARE PROTOTYPE matter of this paper (Merletti and Parker, 2004). 2. HARDWARE HARDWARE PROTOTYPE PROTOTYPE matter of of gesture this paper paper (Merletti and Parker, 2004). making of your hand? ThisParker, is exactly the matter this (Merletti and 2004). 2. 2. HARDWARE PROTOTYPE First of aall, the genesis of myopotentials must be subject underFirst ofItofall, all, the genesis of myopotentials myopotentials must be underundermatter paper (Merletti andincludes Parker,both 2004). first major was to create prototypes with which it 2. goal HARDWARE PROTOTYPE First of the genesis of must be stood.of isthis a the complex system that neural and The First all, genesis of myopotentials must be underThe first major goal was to create create prototypes with with which which it stood. It is a complex system that includes both neural and The first major goal was to prototypes it could be possible to measure myopotentials noise stood. It is a complex system that includes both neural and muscular system. Thesystem theoretical schematic for neural measuring first major goal was to create prototypeswithout with which it First ofIt all, genesis of myopotentials must be understood. issystem. a the complex that includes both and The could be possible to measure myopotentials without noise muscular The theoretical schematic for measuring could be major possible to was measure myopotentials without noise and with right amplification (Chen et al., 2007). muscular system. The theoretical schematic for neural measuring myopotentials wasThe designed based on this both knowledge. The first goal to create prototypes with which it could be possible to measure myopotentials without noise stood. It issystem. a complex system that includes and The muscular theoretical schematic for measuring and with with right amplification amplification (Chen et al., al., 2007). 2007). myopotentials was designed based on this this knowledge. The and right (Chen et myopotentials was designed based on knowledge. The design was then used for creating PCB to measure myopocould be possible to measure myopotentials without noise and with right amplification (Chen et al., 2007). muscular system. The theoretical schematic for measuring myopotentials designed basedPCB on this knowledge. The design was thenwas used for creating togalvanic measure myopodesign was then used for creating PCB to measure myopotentialswas (Svecova etdesigned al., 2017). Also isolation myopotentials was based onthe this knowledge. The and with right amplification (Chen et al., 2007). design then used for creating PCB to measure myopotentials (Svecova et al., 2017). Also the galvanic isolation tentials (Svecova etthe al., 2017). Also the galvanic isolation between users and power source must be created bedesign was then used for creating PCB to measure myopotentials (Svecova et al., 2017). Also the galvanic isolation between users and and the power power source must be be created and bebetween users the source must created because of (Svecova usage of et active reference electrode (Merletti tentials al., 2017). Also the galvanic isolation between users and the power source must be created because of usage of active reference electrode (Merletti and cause usage of active active reference electrode (Merletti and Farina,of2016). between users and the power source must be created and be- Fig. 2. Block scheme of measuring circuit. Myopotencial cause usage of reference electrode (Merletti Farina,of2016). 2016). Farina, 2. Block scheme of measuring circuit. Myopotencial cause of2016). usage of active reference electrode (Merletti and Fig. Farina, Fig. signals 2. Block Blockarescheme scheme of measuring measuring circuit. Myopotencial Myopotencial measured and preprocessed by developed Fig. 2. of circuit. signals are measured and preprocessed by developed Farina, 2016). signals arescheme measured and preprocessed preprocessed by one developed four channel analog hardware. There used active Fig. signals 2. Block of measuring circuit. Myopotencial are measured and by developed four channel analog hardware. There used one active four channel analog hardware. There used one active electrode someasured galvanic isolation isThere needed. It was develsignals are and preprocessed by developed four channel analog hardware. used one active electrode so galvanic isolation is needed. It was develelectrode so galvanic isolation is needed. It was developed measuring software where was used fuzzy classifour channel analog hardware. used one classiactive electrode so galvanic isolation isThere needed. Itfuzzy was developed measuring software where was used oped measuring measuring software whereiswas was usedItfuzzy fuzzy classification forsorecognition of gesture of hand’s movement. electrode galvanic isolation needed. was developed software where used classification for recognition of gesture of hand’s movement. fication for recognition recognition of gesture gesture of hand’s hand’s movement. oped measuring software where was used fuzzy classification for of of movement. The fication myopotentials are measured from four channels. Each for recognition of gesture offour hand’s movement. The myopotentials are measured from channels. Each The myopotentials myopotentials are measured frominfour four channels.mode, Each channel consists ofare two electrodes differential The measured from channels. Each channel consists of two electrodes in differential mode, channel consists of of two electrodes infour differential mode, one instrumental amplifier, one high-pass filter, onemode, lowThe myopotentials are measured from channels. Each channel consists two electrodes in differential one instrumental amplifier, one high-pass filter, one lowone instrumental amplifier, one high-pass filter, one lowpass filter, one notch filter with adjustable quality and one channel consists of two electrodes in differential mode, one instrumental amplifier, oneadjustable high-pass filter, one lowpass filter, one notch filter with quality and one pass instrumental filter, one one notch filter with adjustable quality and one operational amplifier for final amplification (Slanina etlowal., one amplifier, one high-pass filter, one pass filter, notch filter with adjustable quality and one operational amplifier for final amplification (Slanina et al., operational amplifier for final amplification (Slanina et al., Fig. 1. Muscles of the forearm. For one movement are pass 2017). filter, one notch filter with adjustable quality and one operational amplifier for final amplification (Slanina et al., Fig. 1. Muscles of the forearm. For one movement are 2017). Fig. 1. Muscles of the forearm. For one movement are 2017). used two and more muscles. With right position of amplifier for final amplification (Slanina et al., Fig. used 1. Muscles of more the forearm. For oneright movement are 2017). two and and muscles. With position of operational used two more muscles. With right position of electrodes forofmeasuring EMCFor is possible toposition recognize Fig. used 1. Muscles the forearm. one movement are 2017). 2.1 Galvanic two and more muscles. With right of electrodes for measuring EMC is possible to recognize recognize 2.1 Galvanic Isolation Isolation electrodes for measuring EMC is possible to what two kindand of muscles areEMC usedWith forpossible each movement. 2.1 Galvanic Galvanic Isolation Isolation used more muscles. right position of 2.1 electrodes for measuring is to recognize what kind of muscles are used for each movement. what kind of muscles are used for each movement. for measuring possible tomyopotenrecognize 2.1 what of muscles areEMC used for each movement. The Galvanic whole prototype’s Withelectrodes fullykind functional prototype foris measuring Isolation board is powered by the Arduino The whole prototype’s board is powered by the Arduino With fully functional prototype for measuring myopotenwhat kind of muscles are used for each movement. The whole prototype’s board is is powered by the the5 Arduino Arduino With fully functional prototype for measuring myopotenNano. Therefore the supply voltage is USB V. The tials, it could now be possible to create algorithm for The whole prototype’s board powered by With fully functional prototype for measuring myopotenNano. Therefore the supply voltage is USB 5 V. The tials, it could now be possible to create algorithm for Nano. Therefore the supply voltage is USB 5 V. The The tials, it could now be possible to create algorithm for The whole prototype’s board is powered by the Arduino With fully functional prototype for measuring myopotentials, it could now be possible to create algorithm for Nano. Therefore the supply voltage is USB 5 V. Nano. Therefore the supply voltage is USB 5 V. The tials, it could now be possible to create algorithm for Copyright © 2018 IFAC 354 2405-8963 © 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Copyright © 2018 IFAC 354 Copyright 2018 IFAC 354 Peer review© of International Federation of Automatic Copyright ©under 2018 responsibility IFAC 354Control. 10.1016/j.ifacol.2018.07.179 Copyright © 2018 IFAC 354
2018 IFAC PDES Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
analogue part was necessary to be powered symmetrically, for this purpose was added a simple circuit to create ±2,5 V and virtual ground. The reference electrode is connected to the virtual ground, therefore the galvanic isolation must be present while using the device. It is for safety of the user and also to meet the requirements of IEC 60601-1 (Boyali et al., 2015). The most convenient way is to isolate the whole USB on its way from computer (power source) to Arduino. The isolator ADuM4160 was used for data lines D+ and D-. Because the ADuM4160 did not provide enough power to supply the rest of the board, the DC-DC converter was used to isolate the power lines. The DC-DC converter provides an isolated power source to supply the prototype board and Arduino while the ADuM4160 provides galvanic isolation of the data lines.
355
2.4 Low Pass Filter As mentioned before, the second order Sallen-Key topology was used for low-pass filter as well. The cut-off frequency was set close to 500 Hz (495 Hz). The difference is caused because of values of electric components (Phinyomark et al., 2012).
Fig. 5. Active low pass filter with cut-off frequency 500 Hz The notch filter for 50 Hz must be used to reduce noise from electrical network and surroundings. Notch filter with very narrow frequency characteristic was used to preserve the most of the precious biological signal. It is a combination of two operational amplifier and twin T connection. It has also adjustable quality (Q) with the trimmer.
Fig. 3. Galvanic isolation. 2.2 The Instrumental Amplifier The INA126 from Texas Instruments was used as instrumentation amplifier. It acts as a differential amplifier and has easily adjustable amplification. The amplification was set on 9. Effect of polarization of electrodes was appearing with higher value of amplification. Fig. 6. Notch filter with cut-off frequency 50 Hz
2.3 Filtration The myopotentials have a frequency spectrum between 20 Hz and 500 Hz so filtering the unneeded frequencies is in place. The topology of second order Sallen-Key was used for high-pass filter as well as for the low-pass filter.
2.5 Final Amplification The final amplification was used to amplify to signal to get most of the resolution from A/D converter. It was used a classic non-inverting wiring. The amplification was set to A = 341. So the total amplification is 350 with a combination with instrumental amplifier. The amplified biological signal goes to the Arduino Nano where is converted to a digital signal and sent to the computer via galvanic isolation (Tomczy´ nski et al., 2015). 3. SOFTWARE
Fig. 4. Active high pass filter with cut-off frequency 20 Hz The cut-off frequency was calculated thanks to equation 1. 1 √ (1) fc = 2π C1C2R3R6 355
The software part consists of communication protocol between Arduino Nano and the computer, plotting the measured signal, adaptive segmentation of the gestures, calculating the features and finally two phase classification of the gestures.
2018 IFAC PDES 356 Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
are divided into four channels as per connection on the prototype board. The measured signal is plotted in real time as shown in Fig. 10.
Fig. 8. Plotted signal from four channels of EMG measurement. Myopotencials were measured from four muscles. It can be seen that each movement of the hand activates different muscles. 3.2 Adaptive Segmentation One of the most crucial part was to know when and where did the gesture occurred in the signal. This was what adaptive segmentation was used for. Method of adaptive segmentation was dividing the signal into quasi-stationarity segments of variable length, depending on the occurrence of non-stationarities in the signal (diagram of the segmentation of signal can be seen on the figure 10). The key factors for choosing the right method of adaptive segmentation were: • Fast algorithm • High precision • Multiple channel segmentation
Based on the key factors, the algorithm using two connected windows and detecting differences of amplitude and frequency was used. Two windows are moving along the signal in each channel. In each window the differences are computed. Amplitude and frequency difference computing was based on equation 2 for Amplitude difference and on equation 3. WL |xi | (2) ADIF =
Fig. 7. Diagram of the data receiving. Table 1. Octet stuffing Unique values 0xFC 0xFD 0xFE
Octet stuffing 0xFE 0xDC 0xFE 0xDD 0xFE 0xDE
Meaning start of packet end of packet octet stuffing mark
i=1
3.1 Communication Protocol
F DIF =
The signal is digitized by 10-bit A/D converter which is included in Arduino Nano. The values are continually read from analogue pins, checked for unique values and then put into packets. Three forbidden values are used in packet to determine where packet starts, where packet ends and to mark the octet stuffing. The unique values can be situated in specified positions in the packed only. Based on this knowledge, it is possible to determine the start of the packet and the end of the packet when processed in the computer. When the COM port is successfully opened, the incoming measured data starts streaming into the PC. The octet stuffing is removed in the first place. Then the values 356
WL i=1
|xi − xi−1 |
(3)
Where W L is window length and it was set to 400 samples. Combining equation 2 and equation 3 the total difference was calculated: DIF = 1 · |ADIF1 − ADIF2 | + 7 · |F DIF1 − F DIF2 | (4) If the difference is higher than the calculated threshold, the segment border is marked on the place of local maximum of the difference. 1 · DIF (5) T HR = BL Where BL is size of incoming data or the number of values in current sample.
2018 IFAC PDES Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
357
Fig. 10. Plotted segmented signal. For the third gesture there is missing one of evaluated segment in third channel. In this case it is used algorithm based on comparison all of rest segments in same time to evaluate the beginning and the end of missing segment. • Line up values of the beginnings • Line up values of the ends • Choose the first value from the beginnings (the earliest beginning) • Choose the last value from the ends (the latest end) • Use these values as segment values for the missing segment for one gesture 3.4 Gestures and Electrode Placement Fig. 9. Diagram of the segmentation of the measured signal signal.
The algorithm is programmed to classify five gestures (Fig. 11).
3.3 Feature Extraction Three features were chosen to be extracted from each segment window and they are Root Mean Square (RMS), Logarithmic Band Power (LBP) and first derivation (DIFF). The RMS is calculated by: N 1 x2 (6) xRM S = N i=1 i
The LBP is calculated by:
xLBP = log(1 +
N 1 2 x ) N i=1 i
(7)
And first derivation is calculated by: xDIF F =
N 1 |xi−1 − xi | N i=1
(8)
These three features ale calculated for each channel. So there are twelve feature values for each detected segment. If there is not find segments based on described algorithm in some channel it is automatically compute as: • Compare segments from all of rest channels for one gesture • Evaluate the beginnings of the segments • Evaluate the ends of the segments 357
Fig. 11. Gesture demonstration. 1 - Fist; 2 - Right; 3 Left; 4 - Up; 5 - Down
2018 IFAC PDES 358 Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
Each gesture has different values as well as features in a different channel. Based on that, the algorithm using fuzzy logic was designed for gesture classification. 3.5 Fuzzy Set Design Because the first phase of classification is based on fuzzy logic, the fuzzy sets must be designed in the first place. For each gesture 102 instances were measured. All three features were calculated for each instance and from these features, the standard deviation and expected value is calculated. Using these two parameters for each gesture and feature in each channel, twelve fuzzy sets were designed (4 channel x 3 features). Each fuzzy set contains the member function of all five gestures. These member functions are based on expected value standard deviation. After the member functions are cast into the fuzzy set, the fuzzy set is adjusted so everywhere in the fuzzy set the sum of the member function at certain point equals 1.
Fig. 12. Adjusted fuzzy set. 3.6 Results The adaptive segmentation and plotting the graph runs smoothly as well as the classification when it comes to computing memory. It uses only a small amount of processor and RAM. The gesture classification can be described in diagram of gesture classification (Fig. 13).
Fig. 13. Diagram of the gesture classification.
Segment borders are plotted in real time directly on the measures signal. The classification is also done and shown in real time, right after the gesture is done. There is an option to save the raw values in.Csv format for further analysis. There is also an option to save just two features from finding segments (Fig. 14). ACKNOWLEDGEMENT The work and the contributions were supported by the project SV4506631/2101 ’Biomedic´ınsk´e inˇzen´ yrsk´e syst´emy XII’. 4. CONCLUSION The goal of this work was the creation of the fuzzy logic algorithm to classify gesture of hand movements. This was a completely experimental creation of a new classification algorithm. The basis for gesture recognition is segmentation, that is, the recognition that there was a gesture in the signal. For this purpose, the adaptive segmentation method was used using two connected windows. These two linked windows float after the signal and look for 358
Fig. 14. Record of real time segmentation and gesture classification. changes in the difference that is higher than the threshold. This algorithm works perfectly and uses only a very small percentage of the operating performance. The adaptive segmentation and plotting the graph runs smoothly as well as the classification when it comes to computing memory. It uses only a small amount of processor and RAM.
2018 IFAC PDES Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
Segment borders are plotted in real time directly on the measures signal. The classification is also done and shown in real time, right after the gesture is done. There is an option to save the raw values in.Csv format for further analysis. There is also an option to save just two features from finding segments. The algorithm was tested by counting the successful classification for each gesture. Each gesture was performed 40 times. The results shown, that the Fist has 100% successful rate of classification. Next most successful gesture is Left with 85% rate. Other gestures are around 60%. This can be caused by too much overlap in fuzzy sets. Overall success rate 73% is sufficient at this first experimental stage. There are some considerable options that can improve further success rate of classification. By using more electrodes, we get more values that can define the gesture. More values mean more data to work with but it also means higher demands on computing power. By classifying less gestures, the fuzzy set is going to overlap less, so it will definitely increase accuracy but in the cost of less gestures. This project was done only for one individual. The input data for fuzzy sets as well as the testing. If would be more data from more individuals collected, the accuracy of classification could improve. By using a different algorithm such as artificial neural network, fuzzy k-NN classification or Bayes theorem. The accuracy of the classification is depending more on the software part than the hardware part. So the future of this work should focus more improving the classification algorithm than the hardware prototype. REFERENCES Simone Benatti, Filippo Casamassima, Bojan Milosevic, Elisabetta Farella, Philipp Sch¨ onle, Schekeb Fateh, Thomas Burger, Qiuting Huang, and Luca Benini. A versatile embedded platform for emg acquisition and gesture recognition. IEEE transactions on biomedical circuits and systems, 9(5):620–630, 2015. Ali Boyali, Naohisa Hashimoto, and Osamu Matsumoto. Hand posture and gesture recognition using myo armband and spectral collaborative representation based classification. In Consumer Electronics (GCCE), 2015 IEEE 4th Global Conference on, pages 200–201. IEEE, 2015. Xiang Chen, Xu Zhang, Zhang-Yan Zhao, Ji-Hai Yang, Vuokko Lantz, and Kong-Qiao Wang. Multiple hand gesture recognition based on surface emg signal. In Bioinformatics and Biomedical Engineering, 2007. ICBBE 2007. The 1st International Conference on, pages 506–509. IEEE, 2007. Roberto Merletti and Dario Farina. Surface electromyography: physiology, engineering and applications. John Wiley & Sons, 2016. Roberto Merletti and Philip A Parker. Electromyography: physiology, engineering, and non-invasive applications, volume 11. John Wiley & Sons, 2004. 359
359
Angkoon Phinyomark, Sirinee Thongpanja, Huosheng Hu, Pornchai Phukpattaranont, and Chusak Limsakul. The usefulness of mean and median frequencies in electromyography analysis. In Computational intelligence in electromyography analysis-A perspective on current applications and future challenges. InTech, 2012. Zdenek Slanina, Sarka Mikolajkova, and David Vala. Human vehicle interaction. In AIP Conference Proceedings, volume 1836, page 020050. AIP Publishing, 2017. Lucie Svecova, David Vala, and Zdenek Slanina. Emg as objective method for revealed mistakes in sport shooting. In International Conference on Intelligent Information Technologies for Industry, pages 439–448. Springer, 2017. Jakub Tomczy´ nski, Piotr Kaczmarek, and Tomasz Ma´ nkowski. Hand gesture-based interface with multichannel semg band enabling unknown gesture discrimination. In Robot Motion and Control (RoMoCo), 2015 10th International Workshop on, pages 52–57. IEEE, 2015.
ScienceDirect
IFAC PapersOnLine 51-6 (2018) 354–359
Fuzzy Classification of Hand’s Motion Fuzzy Fuzzy Classification Classification of of Hand’s Hand’s Motion Motion ∗ ∗ Fuzzy Classification of Hand’s Motion Lukas Peter Filip Maryncak Antotnino Proto ∗∗ ∗ ∗
Lukas Peter ∗∗ Filip Maryncak Antotnino Proto ∗∗ ∗ ∗ ∗ Lukas Maryncak Antotnino Martin Cerny Lukas Peter Peter Filip Filip Maryncak Antotnino Proto Proto ∗ Martin Cerny ∗ ∗ ∗ Martin Cerny Lukas Peter ∗ Filip Maryncak Antotnino Proto ∗ Martin Cerny ∗ ∗ of Ostrava, Department of Cybernetics and Cerny ∗ VSB-Technical UniversityMartin ∗ VSB-Technical University of Ostrava, Department of Cybernetics and ∗ VSB-Technical University of Ostrava, Ostrava, Department of Republic, Cybernetics and and BiomedicalUniversity Engineering, OstravaDepartment 70800, Czech VSB-Technical of of Cybernetics Biomedical Engineering, Ostrava 70800, Czech Republic, ∗ Biomedical Engineering, Ostrava 70800, Czech Republic, (e-mail:[email protected]) VSB-Technical of Ostrava, Department of Cybernetics and BiomedicalUniversity Engineering, Ostrava 70800, Czech Republic, (e-mail:[email protected]) (e-mail:[email protected]) Biomedical Engineering, Ostrava 70800, Czech Republic, (e-mail:[email protected]) Abstract: The goal of this work (e-mail:[email protected]) was to create the measurement circuit that would be able to Abstract: The goal of of this of work was to create create the measurement circuit that that would be able able to Abstract: The goal this work was the circuit would measure and classify signal myopotentials to classify specific gestures hand.Realization of Abstract: The goal of this of work was to to create the measurement measurement circuit of that would be be able to to measure and classify signal myopotentials to classify specific gestures of hand.Realization of measure and classify signal of myopotentials tois classify classify specific gestures of hand.Realization of the system for classification hand’s gestures described in thisgestures paper. Hardware prototype Abstract: The goal of this work was to create the measurement circuit that would be able to measure and classify signal of myopotentials to specific of hand.Realization of the system for classification classification of hand’s hand’s gestures is described described in thisorder paper. Hardware prototype of the system for of gestures is in this paper. Hardware prototype of four measuring channels was created by combination of 2nd filters and right amount measure and signalwas of hand’s myopotentials tois classify specific gestures of hand.Realization of the system forclassify classification gestures described in thisorder paper. Hardware prototype four measuring channels created by combination of 2nd filters and right amount four measuring channels was created by combination of 2nd filters and amount amplification. For digitizing data, Arduino Nano microcontroller was right selected and the system for classification of the hand’s gestures is described thisorder paper. Hardware prototype of four measuring channels was created bythe combination of in 2nd order filters and right amount amplification. For digitizing the data, the Arduino Nano microcontroller was selected and amplification. For digitizing the data, the Arduino Nano microcontroller was selected and programmed using defined communication protocol. The computer software is programmed four measuring channels was created by combination of 2nd order filters and right amount amplification. For digitizing the data, the Arduino Nano microcontroller was selected and programmed using defined defined communication protocol. The computer computer software is programmed programmed using communication protocol. The software is programmed in C# programming language. Signal processing and drawing to user software interface is selected in real time. amplification. For digitizing the data, the protocol. Arduino Nano microcontroller was and programmed using defined communication The computer is in C#one programming language. Signal processing and drawing tofuzzy userlogic interface is programmed in real realsystem time. in C# programming language. Signal processing and drawing to user interface is in time. The of five possible gestures that user made is chosen using and designed programmed using defined communication protocol. The computer software is programmed in C# programming language. Signal processing and drawing to user interface is in real time. The one of of five possible possible gestures gestures that that user made made is chosen chosen using using fuzzy logic logic and and designed system system The of scaling. in C#one programming language. processing drawing userlogic interface is in realsystem time. The one of five five possible gesturesSignal that user user made is isand chosen usingtofuzzy fuzzy and designed designed of scaling. of scaling. The one of five possible gestures that user made is chosen using fuzzy logic and designed system of scaling. © 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. of scaling. electromyography, fuzzy, classiffication of myopotentials, hands gestures. Keywords: Keywords: electromyography, fuzzy, classiffication of myopotentials, hands gestures. Keywords: electromyography, fuzzy, fuzzy, classiffication classiffication of of myopotentials, myopotentials, hands hands gestures. gestures. Keywords: electromyography, Keywords: electromyography, fuzzy, classiffication of myopotentials, hands gestures. 1. INTRODUCTION gesture classification. The communication protocol was 1. INTRODUCTION INTRODUCTION gesture classification. TheNano, communication protocol was 1. gesture classification. The communication protocol was created between Arduino that used protocol for digitizing 1. INTRODUCTION gesture classification. The communication was created between Arduino Nano, that used for digitizing Nowadays the wearable technologies are trying to simplify gesture created between Arduino that for the signal and octet stuffing, and computer. the algo1. INTRODUCTION classification. TheNano, communication protocol was created between Arduino Nano, that used usedThat for digitizing digitizing Nowadays the wearable wearable technologies are trying to simplify simplify the signal and octet stuffing, and computer. computer. That the algoalgoNowadays the technologies are trying to everyday activities of their users are are trying to collect data created the signal and octet stuffing, and That the rithm for adaptive segmentation was designed. Adaptive Nowadays the wearable technologies trying to simplify between Arduino Nano, that used for digitizing the signal and octet segmentation stuffing, and computer. That Adaptive the algoeveryday activities of their users are trying to collect data rithm for adaptive was designed. everyday activities of their users are are trying to collect collect data the during theactivities daywearable and of night. Under the are continuous rithm for and adaptive segmentation was designed. designed. Adaptive segmentation allows to detect inThat both Nowadays the technologies trying tocollection simplify everyday their users trying to data signal octet segmentation stuffing, anddifferences computer. theamplialgorithm for adaptive was Adaptive during the daycan andimagine night. Under the watch, continuous collection segmentation allows toreal detect differences inchannels. both ampliduring the day and night. Under the continuous collection of data, you a smart that measure segmentation allows to detect differences in both amplitude and frequency into timedifferences forwas multiple Afeveryday activities of their users are trying to collect data during the day and night. Under the continuous collection rithm for adaptive segmentation designed. Adaptive segmentation allows detect in both ampliof data, you cancount imagine asteps smart watch, that measure tude and frequency insegmentation, real time time for for multiple multiple channels. Afof data, you can imagine a smart watch, that measure your heart rate, your and based on that data tude and frequency in real channels. After the right gesture the two phase gesture during the day and night. Under the continuous collection of data, you cancount imagine asteps smart watch, that measure segmentation allowsinsegmentation, toreal detect inphase both gesture amplitude and frequency timedifferences for multiple channels. Afyour heart rate, your and based on that data ter the right gesture the two your heart rate, count yourasteps steps and based on that that data tude evaluates your daily activity. But and what you that could control ter the right gesture segmentation, thecorrect two phase phase gesture classification is usedin to determine gesture (Beof data, you can imagine smart watch, measure your heart rate, count your based on data and frequency real time forthe multiple channels. After the right gesture segmentation, the two gesture evaluates your daily activity. But what youphone could control classification is used used to to determine the correct gesture (Beevaluates your daily activity. But what you could control applications in daily your computer orand mobile just by ter classification is determine the correct gesture (Benatti et right al., 2015). your heart rate, count your steps based on that data evaluates your activity. But what you could control the gesture segmentation, the two phase gesture classification is used to determine the correct gesture (Beapplications in your your computer or mobile mobile phone just by natti et al., 2015). applications in computer or phone just by making a gesture of your hand? This is exactly the just subject natti et et al., al., 2015). 2015). evaluates your daily activity. But what youphone could control applications in your computer or mobile by classification is used to determine the correct gesture (Benatti making a gesture of your hand? This is exactly the subject making a gesture gesture of your your hand?and This is exactly exactly the just subject matter of thisinpaper (Merletti Parker, 2004). applications your computer or mobile phone by making a of hand? This is the subject natti et al., 2015). 2. HARDWARE PROTOTYPE matter of this paper (Merletti and Parker, 2004). 2. HARDWARE HARDWARE PROTOTYPE PROTOTYPE matter of of gesture this paper paper (Merletti and Parker, 2004). making of your hand? ThisParker, is exactly the matter this (Merletti and 2004). 2. 2. HARDWARE PROTOTYPE First of aall, the genesis of myopotentials must be subject underFirst ofItofall, all, the genesis of myopotentials myopotentials must be underundermatter paper (Merletti andincludes Parker,both 2004). first major was to create prototypes with which it 2. goal HARDWARE PROTOTYPE First of the genesis of must be stood.of isthis a the complex system that neural and The First all, genesis of myopotentials must be underThe first major goal was to create create prototypes with with which which it stood. It is a complex system that includes both neural and The first major goal was to prototypes it could be possible to measure myopotentials noise stood. It is a complex system that includes both neural and muscular system. Thesystem theoretical schematic for neural measuring first major goal was to create prototypeswithout with which it First ofIt all, genesis of myopotentials must be understood. issystem. a the complex that includes both and The could be possible to measure myopotentials without noise muscular The theoretical schematic for measuring could be major possible to was measure myopotentials without noise and with right amplification (Chen et al., 2007). muscular system. The theoretical schematic for neural measuring myopotentials wasThe designed based on this both knowledge. The first goal to create prototypes with which it could be possible to measure myopotentials without noise stood. It issystem. a complex system that includes and The muscular theoretical schematic for measuring and with with right amplification amplification (Chen et al., al., 2007). 2007). myopotentials was designed based on this this knowledge. The and right (Chen et myopotentials was designed based on knowledge. The design was then used for creating PCB to measure myopocould be possible to measure myopotentials without noise and with right amplification (Chen et al., 2007). muscular system. The theoretical schematic for measuring myopotentials designed basedPCB on this knowledge. The design was thenwas used for creating togalvanic measure myopodesign was then used for creating PCB to measure myopotentialswas (Svecova etdesigned al., 2017). Also isolation myopotentials was based onthe this knowledge. The and with right amplification (Chen et al., 2007). design then used for creating PCB to measure myopotentials (Svecova et al., 2017). Also the galvanic isolation tentials (Svecova etthe al., 2017). Also the galvanic isolation between users and power source must be created bedesign was then used for creating PCB to measure myopotentials (Svecova et al., 2017). Also the galvanic isolation between users and and the power power source must be be created and bebetween users the source must created because of (Svecova usage of et active reference electrode (Merletti tentials al., 2017). Also the galvanic isolation between users and the power source must be created because of usage of active reference electrode (Merletti and cause usage of active active reference electrode (Merletti and Farina,of2016). between users and the power source must be created and be- Fig. 2. Block scheme of measuring circuit. Myopotencial cause usage of reference electrode (Merletti Farina,of2016). 2016). Farina, 2. Block scheme of measuring circuit. Myopotencial cause of2016). usage of active reference electrode (Merletti and Fig. Farina, Fig. signals 2. Block Blockarescheme scheme of measuring measuring circuit. Myopotencial Myopotencial measured and preprocessed by developed Fig. 2. of circuit. signals are measured and preprocessed by developed Farina, 2016). signals arescheme measured and preprocessed preprocessed by one developed four channel analog hardware. There used active Fig. signals 2. Block of measuring circuit. Myopotencial are measured and by developed four channel analog hardware. There used one active four channel analog hardware. There used one active electrode someasured galvanic isolation isThere needed. It was develsignals are and preprocessed by developed four channel analog hardware. used one active electrode so galvanic isolation is needed. It was develelectrode so galvanic isolation is needed. It was developed measuring software where was used fuzzy classifour channel analog hardware. used one classiactive electrode so galvanic isolation isThere needed. Itfuzzy was developed measuring software where was used oped measuring measuring software whereiswas was usedItfuzzy fuzzy classification forsorecognition of gesture of hand’s movement. electrode galvanic isolation needed. was developed software where used classification for recognition of gesture of hand’s movement. fication for recognition recognition of gesture gesture of hand’s hand’s movement. oped measuring software where was used fuzzy classification for of of movement. The fication myopotentials are measured from four channels. Each for recognition of gesture offour hand’s movement. The myopotentials are measured from channels. Each The myopotentials myopotentials are measured frominfour four channels.mode, Each channel consists ofare two electrodes differential The measured from channels. Each channel consists of two electrodes in differential mode, channel consists of of two electrodes infour differential mode, one instrumental amplifier, one high-pass filter, onemode, lowThe myopotentials are measured from channels. Each channel consists two electrodes in differential one instrumental amplifier, one high-pass filter, one lowone instrumental amplifier, one high-pass filter, one lowpass filter, one notch filter with adjustable quality and one channel consists of two electrodes in differential mode, one instrumental amplifier, oneadjustable high-pass filter, one lowpass filter, one notch filter with quality and one pass instrumental filter, one one notch filter with adjustable quality and one operational amplifier for final amplification (Slanina etlowal., one amplifier, one high-pass filter, one pass filter, notch filter with adjustable quality and one operational amplifier for final amplification (Slanina et al., operational amplifier for final amplification (Slanina et al., Fig. 1. Muscles of the forearm. For one movement are pass 2017). filter, one notch filter with adjustable quality and one operational amplifier for final amplification (Slanina et al., Fig. 1. Muscles of the forearm. For one movement are 2017). Fig. 1. Muscles of the forearm. For one movement are 2017). used two and more muscles. With right position of amplifier for final amplification (Slanina et al., Fig. used 1. Muscles of more the forearm. For oneright movement are 2017). two and and muscles. With position of operational used two more muscles. With right position of electrodes forofmeasuring EMCFor is possible toposition recognize Fig. used 1. Muscles the forearm. one movement are 2017). 2.1 Galvanic two and more muscles. With right of electrodes for measuring EMC is possible to recognize recognize 2.1 Galvanic Isolation Isolation electrodes for measuring EMC is possible to what two kindand of muscles areEMC usedWith forpossible each movement. 2.1 Galvanic Galvanic Isolation Isolation used more muscles. right position of 2.1 electrodes for measuring is to recognize what kind of muscles are used for each movement. what kind of muscles are used for each movement. for measuring possible tomyopotenrecognize 2.1 what of muscles areEMC used for each movement. The Galvanic whole prototype’s Withelectrodes fullykind functional prototype foris measuring Isolation board is powered by the Arduino The whole prototype’s board is powered by the Arduino With fully functional prototype for measuring myopotenwhat kind of muscles are used for each movement. The whole prototype’s board is is powered by the the5 Arduino Arduino With fully functional prototype for measuring myopotenNano. Therefore the supply voltage is USB V. The tials, it could now be possible to create algorithm for The whole prototype’s board powered by With fully functional prototype for measuring myopotenNano. Therefore the supply voltage is USB 5 V. The tials, it could now be possible to create algorithm for Nano. Therefore the supply voltage is USB 5 V. The The tials, it could now be possible to create algorithm for The whole prototype’s board is powered by the Arduino With fully functional prototype for measuring myopotentials, it could now be possible to create algorithm for Nano. Therefore the supply voltage is USB 5 V. Nano. Therefore the supply voltage is USB 5 V. The tials, it could now be possible to create algorithm for Copyright © 2018 IFAC 354 2405-8963 © 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Copyright © 2018 IFAC 354 Copyright 2018 IFAC 354 Peer review© of International Federation of Automatic Copyright ©under 2018 responsibility IFAC 354Control. 10.1016/j.ifacol.2018.07.179 Copyright © 2018 IFAC 354
2018 IFAC PDES Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
analogue part was necessary to be powered symmetrically, for this purpose was added a simple circuit to create ±2,5 V and virtual ground. The reference electrode is connected to the virtual ground, therefore the galvanic isolation must be present while using the device. It is for safety of the user and also to meet the requirements of IEC 60601-1 (Boyali et al., 2015). The most convenient way is to isolate the whole USB on its way from computer (power source) to Arduino. The isolator ADuM4160 was used for data lines D+ and D-. Because the ADuM4160 did not provide enough power to supply the rest of the board, the DC-DC converter was used to isolate the power lines. The DC-DC converter provides an isolated power source to supply the prototype board and Arduino while the ADuM4160 provides galvanic isolation of the data lines.
355
2.4 Low Pass Filter As mentioned before, the second order Sallen-Key topology was used for low-pass filter as well. The cut-off frequency was set close to 500 Hz (495 Hz). The difference is caused because of values of electric components (Phinyomark et al., 2012).
Fig. 5. Active low pass filter with cut-off frequency 500 Hz The notch filter for 50 Hz must be used to reduce noise from electrical network and surroundings. Notch filter with very narrow frequency characteristic was used to preserve the most of the precious biological signal. It is a combination of two operational amplifier and twin T connection. It has also adjustable quality (Q) with the trimmer.
Fig. 3. Galvanic isolation. 2.2 The Instrumental Amplifier The INA126 from Texas Instruments was used as instrumentation amplifier. It acts as a differential amplifier and has easily adjustable amplification. The amplification was set on 9. Effect of polarization of electrodes was appearing with higher value of amplification. Fig. 6. Notch filter with cut-off frequency 50 Hz
2.3 Filtration The myopotentials have a frequency spectrum between 20 Hz and 500 Hz so filtering the unneeded frequencies is in place. The topology of second order Sallen-Key was used for high-pass filter as well as for the low-pass filter.
2.5 Final Amplification The final amplification was used to amplify to signal to get most of the resolution from A/D converter. It was used a classic non-inverting wiring. The amplification was set to A = 341. So the total amplification is 350 with a combination with instrumental amplifier. The amplified biological signal goes to the Arduino Nano where is converted to a digital signal and sent to the computer via galvanic isolation (Tomczy´ nski et al., 2015). 3. SOFTWARE
Fig. 4. Active high pass filter with cut-off frequency 20 Hz The cut-off frequency was calculated thanks to equation 1. 1 √ (1) fc = 2π C1C2R3R6 355
The software part consists of communication protocol between Arduino Nano and the computer, plotting the measured signal, adaptive segmentation of the gestures, calculating the features and finally two phase classification of the gestures.
2018 IFAC PDES 356 Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
are divided into four channels as per connection on the prototype board. The measured signal is plotted in real time as shown in Fig. 10.
Fig. 8. Plotted signal from four channels of EMG measurement. Myopotencials were measured from four muscles. It can be seen that each movement of the hand activates different muscles. 3.2 Adaptive Segmentation One of the most crucial part was to know when and where did the gesture occurred in the signal. This was what adaptive segmentation was used for. Method of adaptive segmentation was dividing the signal into quasi-stationarity segments of variable length, depending on the occurrence of non-stationarities in the signal (diagram of the segmentation of signal can be seen on the figure 10). The key factors for choosing the right method of adaptive segmentation were: • Fast algorithm • High precision • Multiple channel segmentation
Based on the key factors, the algorithm using two connected windows and detecting differences of amplitude and frequency was used. Two windows are moving along the signal in each channel. In each window the differences are computed. Amplitude and frequency difference computing was based on equation 2 for Amplitude difference and on equation 3. WL |xi | (2) ADIF =
Fig. 7. Diagram of the data receiving. Table 1. Octet stuffing Unique values 0xFC 0xFD 0xFE
Octet stuffing 0xFE 0xDC 0xFE 0xDD 0xFE 0xDE
Meaning start of packet end of packet octet stuffing mark
i=1
3.1 Communication Protocol
F DIF =
The signal is digitized by 10-bit A/D converter which is included in Arduino Nano. The values are continually read from analogue pins, checked for unique values and then put into packets. Three forbidden values are used in packet to determine where packet starts, where packet ends and to mark the octet stuffing. The unique values can be situated in specified positions in the packed only. Based on this knowledge, it is possible to determine the start of the packet and the end of the packet when processed in the computer. When the COM port is successfully opened, the incoming measured data starts streaming into the PC. The octet stuffing is removed in the first place. Then the values 356
WL i=1
|xi − xi−1 |
(3)
Where W L is window length and it was set to 400 samples. Combining equation 2 and equation 3 the total difference was calculated: DIF = 1 · |ADIF1 − ADIF2 | + 7 · |F DIF1 − F DIF2 | (4) If the difference is higher than the calculated threshold, the segment border is marked on the place of local maximum of the difference. 1 · DIF (5) T HR = BL Where BL is size of incoming data or the number of values in current sample.
2018 IFAC PDES Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
357
Fig. 10. Plotted segmented signal. For the third gesture there is missing one of evaluated segment in third channel. In this case it is used algorithm based on comparison all of rest segments in same time to evaluate the beginning and the end of missing segment. • Line up values of the beginnings • Line up values of the ends • Choose the first value from the beginnings (the earliest beginning) • Choose the last value from the ends (the latest end) • Use these values as segment values for the missing segment for one gesture 3.4 Gestures and Electrode Placement Fig. 9. Diagram of the segmentation of the measured signal signal.
The algorithm is programmed to classify five gestures (Fig. 11).
3.3 Feature Extraction Three features were chosen to be extracted from each segment window and they are Root Mean Square (RMS), Logarithmic Band Power (LBP) and first derivation (DIFF). The RMS is calculated by: N 1 x2 (6) xRM S = N i=1 i
The LBP is calculated by:
xLBP = log(1 +
N 1 2 x ) N i=1 i
(7)
And first derivation is calculated by: xDIF F =
N 1 |xi−1 − xi | N i=1
(8)
These three features ale calculated for each channel. So there are twelve feature values for each detected segment. If there is not find segments based on described algorithm in some channel it is automatically compute as: • Compare segments from all of rest channels for one gesture • Evaluate the beginnings of the segments • Evaluate the ends of the segments 357
Fig. 11. Gesture demonstration. 1 - Fist; 2 - Right; 3 Left; 4 - Up; 5 - Down
2018 IFAC PDES 358 Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
Each gesture has different values as well as features in a different channel. Based on that, the algorithm using fuzzy logic was designed for gesture classification. 3.5 Fuzzy Set Design Because the first phase of classification is based on fuzzy logic, the fuzzy sets must be designed in the first place. For each gesture 102 instances were measured. All three features were calculated for each instance and from these features, the standard deviation and expected value is calculated. Using these two parameters for each gesture and feature in each channel, twelve fuzzy sets were designed (4 channel x 3 features). Each fuzzy set contains the member function of all five gestures. These member functions are based on expected value standard deviation. After the member functions are cast into the fuzzy set, the fuzzy set is adjusted so everywhere in the fuzzy set the sum of the member function at certain point equals 1.
Fig. 12. Adjusted fuzzy set. 3.6 Results The adaptive segmentation and plotting the graph runs smoothly as well as the classification when it comes to computing memory. It uses only a small amount of processor and RAM. The gesture classification can be described in diagram of gesture classification (Fig. 13).
Fig. 13. Diagram of the gesture classification.
Segment borders are plotted in real time directly on the measures signal. The classification is also done and shown in real time, right after the gesture is done. There is an option to save the raw values in.Csv format for further analysis. There is also an option to save just two features from finding segments (Fig. 14). ACKNOWLEDGEMENT The work and the contributions were supported by the project SV4506631/2101 ’Biomedic´ınsk´e inˇzen´ yrsk´e syst´emy XII’. 4. CONCLUSION The goal of this work was the creation of the fuzzy logic algorithm to classify gesture of hand movements. This was a completely experimental creation of a new classification algorithm. The basis for gesture recognition is segmentation, that is, the recognition that there was a gesture in the signal. For this purpose, the adaptive segmentation method was used using two connected windows. These two linked windows float after the signal and look for 358
Fig. 14. Record of real time segmentation and gesture classification. changes in the difference that is higher than the threshold. This algorithm works perfectly and uses only a very small percentage of the operating performance. The adaptive segmentation and plotting the graph runs smoothly as well as the classification when it comes to computing memory. It uses only a small amount of processor and RAM.
2018 IFAC PDES Ostrava, Czech Republic, May 23-25, 2018 Lukas Peter et al. / IFAC PapersOnLine 51-6 (2018) 354–359
Segment borders are plotted in real time directly on the measures signal. The classification is also done and shown in real time, right after the gesture is done. There is an option to save the raw values in.Csv format for further analysis. There is also an option to save just two features from finding segments. The algorithm was tested by counting the successful classification for each gesture. Each gesture was performed 40 times. The results shown, that the Fist has 100% successful rate of classification. Next most successful gesture is Left with 85% rate. Other gestures are around 60%. This can be caused by too much overlap in fuzzy sets. Overall success rate 73% is sufficient at this first experimental stage. There are some considerable options that can improve further success rate of classification. By using more electrodes, we get more values that can define the gesture. More values mean more data to work with but it also means higher demands on computing power. By classifying less gestures, the fuzzy set is going to overlap less, so it will definitely increase accuracy but in the cost of less gestures. This project was done only for one individual. The input data for fuzzy sets as well as the testing. If would be more data from more individuals collected, the accuracy of classification could improve. By using a different algorithm such as artificial neural network, fuzzy k-NN classification or Bayes theorem. The accuracy of the classification is depending more on the software part than the hardware part. So the future of this work should focus more improving the classification algorithm than the hardware prototype. REFERENCES Simone Benatti, Filippo Casamassima, Bojan Milosevic, Elisabetta Farella, Philipp Sch¨ onle, Schekeb Fateh, Thomas Burger, Qiuting Huang, and Luca Benini. A versatile embedded platform for emg acquisition and gesture recognition. IEEE transactions on biomedical circuits and systems, 9(5):620–630, 2015. Ali Boyali, Naohisa Hashimoto, and Osamu Matsumoto. Hand posture and gesture recognition using myo armband and spectral collaborative representation based classification. In Consumer Electronics (GCCE), 2015 IEEE 4th Global Conference on, pages 200–201. IEEE, 2015. Xiang Chen, Xu Zhang, Zhang-Yan Zhao, Ji-Hai Yang, Vuokko Lantz, and Kong-Qiao Wang. Multiple hand gesture recognition based on surface emg signal. In Bioinformatics and Biomedical Engineering, 2007. ICBBE 2007. The 1st International Conference on, pages 506–509. IEEE, 2007. Roberto Merletti and Dario Farina. Surface electromyography: physiology, engineering and applications. John Wiley & Sons, 2016. Roberto Merletti and Philip A Parker. Electromyography: physiology, engineering, and non-invasive applications, volume 11. John Wiley & Sons, 2004. 359
359
Angkoon Phinyomark, Sirinee Thongpanja, Huosheng Hu, Pornchai Phukpattaranont, and Chusak Limsakul. The usefulness of mean and median frequencies in electromyography analysis. In Computational intelligence in electromyography analysis-A perspective on current applications and future challenges. InTech, 2012. Zdenek Slanina, Sarka Mikolajkova, and David Vala. Human vehicle interaction. In AIP Conference Proceedings, volume 1836, page 020050. AIP Publishing, 2017. Lucie Svecova, David Vala, and Zdenek Slanina. Emg as objective method for revealed mistakes in sport shooting. In International Conference on Intelligent Information Technologies for Industry, pages 439–448. Springer, 2017. Jakub Tomczy´ nski, Piotr Kaczmarek, and Tomasz Ma´ nkowski. Hand gesture-based interface with multichannel semg band enabling unknown gesture discrimination. In Robot Motion and Control (RoMoCo), 2015 10th International Workshop on, pages 52–57. IEEE, 2015.