- Email: [email protected]
An expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System for digital modulation recognition
Expert Systems with Applications Expert Systems with Applications 33 (2007) 582–589 www.elsevier.com/locate/eswa
An expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System for digital modulation recognition Engin Avci *, Davut Hanbay, Asaf Varol Firat University, Department of Electronic and Computer Education, 23119 Elazig, Turkey
Abstract This paper presents a comparative study of implementation of feature extraction and classification algorithms based on discrete wavelet decompositions and Adaptive Network Based Fuzzy Inference System (ANFIS) for digital modulation recognition. Here, in first stage, 20 different feature extraction methods are generated by separately using Daubechies, Biorthogonal, Coiflets, Symlets wavelet families. In second stage, the performance comparison of these feature extraction methods is performed by using a new Expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System (EDWANFIS). The digital modulated signals used in this experimental study are ASK8, FSK8, PSK8, QASK8. EDWANFIS structure consists of two parts. The first part is Discrete Wavelet Transform (DWT)adaptive wavelet entropy and Adaptive Network Based Fuzzy Inference System for Automatic Digital Modulation Recognition (ADMR). The performance of this comparison system is evaluated by using total 800 digital modulated signals for each of these feature extraction methods. The performance comparison of these features extraction methods and the advantages and disadvantages of the methods are examined. 2006 Elsevier Ltd. All rights reserved. Keywords: Digital modulation; DWT; Wavelet Adaptive Network Based Fuzzy Inference System; ADMR; Feature extraction; Wavelet entropy; Expert system
1. Introduction Automatic modulation recognition (AMR) is an essentially interesting problem with a variety of regulatory and military communication applications (Wu, Ren, Wang, & Zhao, 2004). AMR is used for military communication intelligence studies such as spectrum surveillance, threat estimation, etc. In previous studies, modulation recognition was performed by using human operator’s interpretation of measured parameters to classify signals (Wu, Wang, Liu, & Ren, 2005). In these studies, modulated signal properties such as signal spectrum features, instantaneous amplitudes
* Corresponding author. Tel.: +90 4242370000 4257; fax: +90 4242367064. E-mail addresses: enginavci@firat.edu.tr, [email protected] (E. Avci), dhanbay@firat.edu.tr (D. Hanbay).
0957-4174/$ - see front matter 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2006.06.001
and instantaneous phase were evaluated for modulation recognition. Then, a lot of methods have been developed for Automatic Digital Modulation Recognition (ADMR). In these studies, the demodulator banks designed for each of special modulation types were used for ADMR (Wu et al., 2005). The demodulator bank method is called as semi-automatic because of a human operator is still required to listen outputs of these demodulator banks (Wu et al., 2005). In Liedtke (1984), an ADMR method is presented for some types of digital modulation. The types of digital modulation are ASK2, FSK2, PSK2, and PSK4, PSK8. In this study, universial demodulator technique is used for recognizer. For this reason, amplitude histogram, the frequency histogram, the phase difference histogram, amplitude variance, and the frequency variance key features are used. In DeSimio and Glenn (1988), an adaptive technique is used for ADMR. Some types of digital modulation are used in this study. These digital modulation types are ASK2,
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
PSK2, PSK4, and FSK2. In this study, the key features derived from the signal envelope, the signal spectra, the signal squared and the fourth power of the signal are used to decide type of modulation. In Polydoros and Kim (1990), a modulation recognizer which uses decision-theoretic approach is presented for PSK2 and PSK4. In this study, carrier frequency, the initial phase, the symbol rate and the signal-to-noise ratio are used as key features for ADMR. In Hsue and Soliman (1990), a modulation recognizer which uses zero-crossing characteristic of the modulated signals is presented for MPSK, MFSK. In Soliman and Hsue (1992), a modulation recognizer which uses the statistical moments of the modulated signal phase is introduced. Here, the even order moments of the modulated signal phase are used for evaluate the number of levels of MPSK signals. In Alssaleh, Farrell, and Mammone (1992), key features which are obtained from the averaged spectrum of the instantaneous frequency are used for PSK2, PSK4, FSK2, and FSK4 digital modulation types. In Nagy (1994a), a modulation recognition method is presented for multichannel systems. Here, the intercepted signal is divided into individual parts. Then, each of these individual parts is classified by using a single tone classifier for recognizing of ASK, PSK2, PSK4, and FSK2. In Beidas and Weber (1995), a modulation recognizer which uses the time-domain Higher-Order Correlations is presented for discriminate between the MFSK signals. In Azzouz and Nandi (1995), a modulation recognizer which uses the key features used are obtained from the instantaneous amplitude, the instantaneous phase and the instantaneous frequency of the intercepted signal is introduction for the ASK2, ASK4, PSK2, PSK4, FSK2, and FSK4 digital modulation types. A lot of studies have realized on the topic of automatic modulation recognition using ANNs approximations (Nandi & Azzouz, 1998; Kavalov, 2001; Kremer & Shiels, 1999). Nandi and Azzouz (1997) proposed a single hidden layer ANN structure for automatic modulation classification. This network has a 4-node input layer, a 25-node hidden layer and a 7-node output layer. Nevertheless a degradation of performance at higher signal noise ratios (SNR) will appear when the ANN is trained on signals with lower SNR. The generalization capability of ANNs must be increased for overcoming this shortcoming of ANNs classifiers. Therefore, a compact set of features, which capture all the major characteristics of the intercepted signals in a relatively small number of the components must be obtained from intercepted signal (Wu et al., 2004). Then, these features must be given to ANN inputs for modulations classification. For this reason, the wavelet transform is used for the extraction of key features at pattern recognition and classification areas (Wu et al., 2004). Nowadays, in many areas such as image processing, signal processing, especially image compression, speech processing, computer vision, the wavelet transform types are commonly used (Wu et al., 2004). The DWT have been used in automatic digital modulation recognition for communi-
583
cation signal processing (Wu et al., 2004); nevertheless, to date Discrete Wavelet Adaptive Network Based Fuzzy Inference System (DWANFIS) analysis for automatic digital modulation recognition using adaptive entropy approach is a relatively new approach. In this study presents a new method (EDWANFIS) in ADMR. The novelties presented in this study can be arranged in order as below: (1) The effectiveness of the Discrete Wavelet Transform (DWT) features is shown to be used for ADMR. (2) Expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System (EDWANFIS) on adaptive wavelet algorithm is used for increasing the effectiveness of the ADMR at various SNR rates and various parameters changing. (3) At this study, DWT and EDWANFIS method on ADMR field is firstly have been used for automatic digital modulation literature. More successfully results at various SNR rates and various parameters changing than preceding studies realized by using statistical pattern recognition, and decision-theoretic approach were obtained from this application. Here, the comparison of Daubechies, Biorthogonal, Coiflets, Symlets wavelet families for digital modulation recognition is performed by using a new Expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System (EDWANFIS) method. The digital modulated signals used in this experimental study are ASK8, FSK8, PSK8, QASK8. This paper is organized as follows: In Section 2, Discrete Wavelet Adaptive Network Based Fuzzy Inference System (DWANFIS), and DWT are briefly reviewed. In Section 3, digital modulated signals database, ADMR system based on EDWANFIS, and experimental results are explained respectively. In Section 4, conclusion and discussions are given. 2. Theory 2.1. DWT and Discrete Wavelet Adaptive Network Based Fuzzy Inference System Discrete Wavelet Transform decomposes non-station signals at different frequency intervals with various resolutions. DWT analyses a signal into rough approximation (a) and detail (d) components. Therefore, DWT is commonly used in feature extraction and classification stages of pattern recognition. Discrete wavelet transform performs two functions (Graps, 1995). These functions which can be shown as low-pass and high-pass filters are called scaling functions and wavelet functions. Wavelet coefficients of the DWT at 7 levels are shown as Fig. 1. The original modulated signal is passed through lowpass and high-pass filters, and approximation (a), and detail (d) coefficients signals are obtained. DWT uses the
584
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
Signal
a1
a2
a3
a4
a5
a6
a7
d1
d2
d3
d4
d5
d6
d7 Fig. 1. Wavelet coefficients of the DWT at 7 levels.
fact that it is possible to resolve high frequency components within a small time window, and only low frequency components need large time windows. This is because a low frequency component completes a cycle in a large time interval whereas a high frequency component completes a cycle in a much shorter interval. Therefore, slow varying components can only be identified over long time intervals but fast varying components can be identified over short time intervals. DWT can be regarded as a continuous time wavelet decomposition sampled at different frequencies at every level or stage. Both artificial neural network and fuzzy logic are used in ANFIS’s architecture. ANFIS is consisted of if-then rules and couples of input–output, for ANFIS training is used learning algorithms of neural network (Zadeh, 1987). Adaptive Network Based Fuzzy Inference Systems (ANFIS) are systems that are constructed to make use of some organizational principles resembling those of the human brain (Davis & Mermelstein, 1980). ANFIS represent the promising new generation of information processing systems. Adaptive network based fuzzy inference systems are good at tasks such as pattern matching and classification, function approximation, optimization and data clustering, while traditional computers, because of their architecture, are inefficient at these tasks, especially pattern-matching tasks (Davis & Mermelstein, 1980; Tufekci & Gowdy, 2000). As for DWANFIS try to com-
bine aspects of the discrete wavelet transformation for purpose of feature extraction and selection with the characteristic decision capabilities of adaptive network based fuzzy inference system approaches (Long & Datta, 1996). The Discrete Wavelet Adaptive Network Based Fuzzy Inference System (DWANFIS) is constructed based on the discrete wavelet transform theory (Ha, Tran, & Dissanayake, 2005; Pati & Krishnaprasad, 1993) and is an alternative to adaptive network based fuzzy inference systems (Zhang, Walter, Miao, & Lee, 1995). Discrete wavelet transform (Avci, Turkoglu, & Poyraz, 2005a) is a powerful tool for non-stationary signal analysis. Let x(t) be a piecewise continuous function. Discrete wavelet transform allows one to decompose x(t) using a wavelet function w:Rn ! R. Based on the wavelet decomposition, DWANFIS structure is defined by P wi fi ðw½Di ðx ti ÞÞ P yðxÞ ¼ i ð1Þ i wi here wi are weights of the DWANFIS inputs, Di are dilation vectors specifying the diagonal dilation matrices Di, ti are translation vectors, and fi are Sugeno output functions of the ANFIS. An algorithm of the hybrid type has been derived for adjusting the parameters of the DWANFIS (Davis & Mermelstein, 1980; Tufekci & Gowdy, 2000; Zadeh, 1987). Applications of wavelet adaptive network based fuzzy inference system in the medical field include for detection of electrocardiography changes in patients with partial epilepsy using feature extraction (Bishop, 1996; Wong & Nandi, 2004; Avci, Turkoglu, & Poyraz, 2005b), bearing fault diagnosis based on wavelet transform and fuzzy inference (Avci et al., 2005a; Bishop, 1996), for satellite image fusion (Zadeh, 1987); nevertheless, to date EDWANFIS analysis for automatic digital modulation recognition using adaptive entropy approach is a relatively new approach. 3. Methodology and experimental applications 3.1. Digital modulated signal database At the digital modulation systems, a modulated signal can be commonly represented as KðtÞ ¼ M n sD ðtÞ cosð2pfn ðtÞ þ hn ðtÞÞ;
ð2Þ
where Mn, fn, hn, and sD are the message amplitude, message frequency, message phase, and pulse shaping function, respectively (Wong & Nandi, 2004). The digital message signal used in this study can be shown as in Fig. 2. The four important types of digital modulation techniques are Amplitude Shift Keying-8 (ASK8), Frequency Shift Keying-8 (FSK8), Phase Shift Keying-8 (PSK8), and Quadrature Amplitude Shift Keying-8 (QASK8). In this study, the generating phases of the digital modulating signal database are given as below:
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589 Digital Modulated Signal
6
Norm Entropy1
3 2
Norm EntropyN
Adaptive Network Based Fuzzy Inference System
1 0
error 0
100
200
300 400 500 600 700 800 Sample numbers of message signal
900
1000
+
∑
-
desired output
Classification Block
Adjustable Parameters
4
Structure of EDWANFIS Automatic Digital Modulation Recognition System
DWT
5
Feature Extraction Block
Amplitude of message signal (M=8 Levels)
7
585
Fig. 2. The digital message signal used in this study. Fig. 3. The structure of DWNN for analog modulation classification. Table 1 The parameters of message signal used in this study Parameters
Values
Sample numbers of message signal Sampling frequency , fs Carrier frequency, fc Baud rate
1000 samples 25 Hz 10 Hz 300 baud
Here, the message signal is digitally simulated according to Equal (2) in MATLAB Communication Toolbox. First, this message signal is generated by using random integer up to M = 8 – level. Second, this message signal is re-sampled at baud rate (300 baud) for pulse shaping before passing through respective modulators. The message signal parameters used for digital modulation are given in Table 1. Third, 200 simulated signals of each of the digital modulation types of ASK8, FSK8, PSK8, and QASK8 have been generated for these experimental studies. In this reason, additive white Gaussian noise is added according the various signal-to-noise ratios (SNR), i.e. at between 5 and 60 dB. Hundred of these 200 digital modulated signals for each of these four types digital modulation (ASK8, FSK8, PSK8, and QASK8) are used for training stage of the EDWANFIS system. Others 100 of 200 digital modulated signals for each of these four types digital modulation (ASK8, FSK8, PSK8, and QASK8) are used for testing stage of the EDWANFIS system. Totally 800 digital modulated signals are used for the training and testing phases of EDWANFIS system developed in this study. 3.2. Using of EDWANFIS Automatic Digital Modulation Recognition System EDWANFIS structure developed in this study for recognition of digital modulated signal waveform patterns is showed in Fig. 3. EDWANFIS system is given in Fig. 3.
This system is separated two layers, which are feature extraction and classification layers. The feature extraction layer of EDWANFIS system is the most important component of designing the automatic system based on digital modulation recognition. The tree structure is used m = 7 as level for DWT of the digital modulated signals. Four various digital modulation types (ASK8, FSK8, PSK8, and QASK8) are used to obtaining the digital modulated signals. Two hundred simulated signals for each of these four types digital modulation are generated. Hundred of these 200 modulated signals for each of these four types digital modulation are generated for training layer of the EDWANFIS automatic digital modulation recognition system. Others 100 of 200 digital modulated signals for each of these four types digital modulation are generated for testing layer of the EDWANFIS automatic digital modulation recognition system. In this way, totally 800 digital modulated signals are generated for the training and testing layers of EDWANFIS automatic digital modulation recognition system developed in this study. 3.2.1. DWT phase The DWT is applied for each of these 800 digital modulated signals. The decomposition structure and reconstruction trees at level 7 as shown in Fig. 1 are used. db2, db3, db5, db8, db10, bior1.3, bior2.2, bior2.8, bior3.5, bior6.8, coif1, coif2, coif3, coif4, coif5, sym2, sym3, sym5, sym7, and sym8 wavelet filters for Daubechies, Biorthogonal, Coiflets, and Symlets are separately used for DWT phase of EDWANFIS automatic digital modulation recognition system respectively. Thus, performance comparison of these wavelet filters for automatic digital modulation recognition by using EDWANFIS is performed. In this way, for each of these wavelet filters, two types of coefficients: one approximation coefficients cA and seven – detail coefficients cDs are obtained from DWT decomposi-
586
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
tion of each of 800 digital modulated signals. The ASK8 digital modulated signal used in this study, a detail coefficient waveforms of DWT at ASK8 modulated signal by using coif2 wavelet filters can be shown as in Figs. 4 and 5, respectively. For each of these DWT coefficients, adaptive wavelet norm entropy values are calculated by using equation given in Eq. (3) (Avci et al., 2005b). P p j si j EðsÞ ¼ i ð3Þ N where the adaptive wavelet entropy E is a real number, s is the terminal node signal and (si) i the waveform of terminal node signals. In norm entropy, P is the power and must be such that 1 6 P < 2. For the EDWANFIS learning process, the P parameter is updated by using 0.1 increasing steps together with weights to minimize the error. Updating of this P parame1 Amplitude of digital modulated signal
0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Length of digital modulated signal
Fig. 4. An original ASK8 modulated signal used in this study.
0.15 0.1
Magnitude (dB)
0.05 0 -0.05 -0.1 -0.15 -0.2 0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Time in seconds
Fig. 5. A detail coefficient waveforms of DWT at ASK8 modulated signal by using coif2 wavelet filters.
ter is performed at the end of 1000 epoch numbers for training of Adaptive Network Based Fuzzy Inference System (ANFIS). The resultant entropy data are normalized by dividing with maximum norm entropy value. Thus, the feature vector is extracted by computing the 8-wavelet entropy values per digital modulated signal. In this way, size of the feature vector obtained by calculating adaptive wavelet norm entropy values for training layer of EDWANFIS is 4 · 100 · 8 = 400 · 8. This feature vector takes from number of digital modulation types used in this study x number of digital modulated signals used for each of digital modulation types x number of adaptive wavelet norm entropies calculated for DWT decomposition coefficients of a digital modulated signal. In the same way, a 4 · 100 · 8 = 400 · 8 feature vector is obtained from other 100 digital modulated signals for each of ASK8, FSK8, PSK8, and QASK8 digital modulation types in testing layer of EDWANFIS. In this study, for this feature vector was generated by separately using db2, db3, db5, db8, db10, bior1.3, bior2.2, bior2.8, bior3.5, bior6.8, coif1, coif2, coif3, coif4, coif5, sym2, sym3, sym5, sym7, and sym8 wavelet filters. The feature vectors generated by using these wavelet filters can be named as in below respectively: • The feature vector obtained by using db2 wavelet filter (FV-1): For this feature vector, db2 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using db3 wavelet filter (FV-2): For this feature vector, db3 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using db5 wavelet filter (FV-3): For this feature vector, db5 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using db8 wavelet filter (FV-4): For this feature vector, db8 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using db10 wavelet filter (FV-5): For this feature vector, db10 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior1.3 wavelet filter (FV-6): For this feature vector, bior1.3 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior2.2 wavelet filter (FV-7): For this feature vector, bior2.2 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior2.8 wavelet filter (FV-8): For this feature vector, bior2.8 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior3.5 wavelet filter (FV-9): For this feature vector, bior3.5 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior6.8 wavelet filter (FV-10): For this feature vector, bior6.8 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using coif1 wavelet filter (FV-11): For this feature vector, coif1 wavelet filter is used for DWT decomposition stage of EDWANFIS.
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
• The feature vector obtained by using coif2 wavelet filter (FV-12): For this feature vector, coif2 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using coif3 wavelet filter (FV-13): For this feature vector, coif3 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using coif4 wavelet filter (FV-14): For this feature vector, coif4 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using coif5 wavelet filter (FV-15): For this feature vector, coif5 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym2 wavelet filter (FV-16): For this feature vector, sym2 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym3 wavelet filter (FV-17): For this feature vector, sym3 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym5 wavelet filter (FV-18): For this feature vector, sym5 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym7 wavelet filter (FV-19): For this feature vector, sym7 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym8 wavelet filter (FV-20): For this feature vector, sym8 wavelet filter is used for DWT decomposition stage of EDWANFIS. 3.2.2. ANFIS phase Both artificial neural network and fuzzy logic are used in ANFIS’s architecture. ANFIS is consisted of if-then rules and couples of input–output, for ANFIS training is used learning algorithms of neural network (Davis & Mermelstein, 1980; Zadeh, 1987). Readers may find more information about ANFIS structure at (Davis & Mermelstein, 1980; Zadeh, 1987). This stage realizes the intelligent classification using features from discrete wavelet stage. The training parameters and the structure of the ANFIS used in this study are shown in Table 2. These parameters were selected for the best performance, after several different experiments, such as the number of input membership functions, the size of the ANFIS layers, value of the moment constant and learning rate, and type of the activation functions. Table 2 The training parameters and the structure of the ANFIS used in this study Number of inputs Number of outputs Rules number The number of layers Learning rule
Sum-squared error
8 4 256 5 Hybrid learning algorithm (back-propagation for nonlinear parameters (ai, ci) and Least squre errors for linear parameters (pi, qi,ri, si, ssi, ppi,rri, qqi, ui)) 0.00001
587
This ANFIS structure has eight inputs that are adaptive wavelet entropies, which are given in Eq. (3). These inputs are cA1, cD1, cD2, cD3, cD4, cD5, cD6, cD7 DWT coefficients. This architecture is formed by using five layers and 256 if-then rules. 4. Results of the experimental studies It was performed experiments using total 4 (number of digital modulation types used in this study) · 200 (number of digital modulated signals used for each of digital modulation types) = 800 digital modulated signals. 4 · 100 = 400 of these signals were used for training of EDWANFIS. Others 4 · 100 = 400 signals were used for testing of this EDWANFIS system. These training and testing processes were repeated for each of feature vectors which are FV-1, FV-2, FV-3, FV-4, FV-5, FV-6, FV-7, FV-8, FV-9, FV10, FV-11, FV-12, FV-13, FV-14, FV-15, FV-16, FV-17, FV-18, FV-19, and FV-20. In this way, the performance comparison of db2, db3, db5, db8, db10, bior1.3, bior2.2, bior2.8, bior3.5, bior6.8, coif1, coif2, coif3, coif4, coif5, sym2, sym3, sym5, sym7, and sym8 wavelet filters for automatic digital modulation recognition were performed. This performance comparison was realized for expended training time of ANFIS and correct recognition rate of digital modulation types in test stage for each of these feature vectors. The comparison results were given below. The performance comparison for expended training time of ANFIS and correct recognition rate of digital modulation types in test stage are tabulated in Table 3. As shown in Table 1, FV-2 feature extraction algorithm for EDWANFIS reached to desired error value in the shortest time (Training epoch numbers: 342). Therefore, training performance of this algorithm is the better than the others. Updating of P parameter of adaptive wavelet norm entropy is performed at the end of 1000 epoch numbers for training of ANFIS. Last value of P parameter of adaptive wavelet norm entropy is 1 for this FV-2 feature extraction algorithm. Moreover, FV-20 feature extraction algorithm for EDWANFIS reached to desired error value in the longest time (Training epoch numbers: 7062). Therefore, training performance of this algorithm is the worse than the others. Last value of P parameter of adaptive wavelet norm entropy is 1.7 for this algorithm. Namely, updating of P parameter of adaptive wavelet norm entropy has performed seven times (1.1, 1.2, 1.3, 1.4, 1.5, 1.6, and 1.7) by using FV-20 feature extraction algorithm for EDWANFIS. Because the P parameter is updated by using 0.1 increasing steps during the EDWANFIS learning process. The last column of Table 1 shows the correct recognition rate of digital modulation types by using feature extraction algorithms. As it is clear from the Table 1, FV-1 and FV-17 feature extraction algorithms for EDWANFIS have best correctness performance for recognition of digital modulation types. This state clearly indicated that the feature extraction methods realized by using wavelet families are very efficiently for automatic digital modulation recognition. Therefore, the feature
588
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
Table 3 Performance comparison for expended time at training of ANFIS and correct recognition rate of digital modulation types in test stage for each of these feature vectors The feature vectors
Training epoch numbers
Last value of P parameter of adaptive wavelet norm entropy
MSE (Mean Square Error) value
Correct recognition rate of digital modulation types (%)
FV-1 FV-2 FV-3 FV-4 FV-5 FV-6 FV-7 FV-8 FV-9 FV-10 FV-11 FV-12 FV-13 FV-14 FV-15 FV-16 FV-17 FV-18 FV-19 FV-20
986 343 1154 876 972 359 447 2059 468 1121 741 1560 642 882 1487 1347 4043 1137 2107 7062
1 1 1.1 1 1 1 1 1.2 1 1.1 1 1.1 1 1 1.1 1.1 1.4 1.1 1.2 1.7
0.000008296 0.000007254 0.000007388 0.000006130 0.000009899 0.000002270 0.000005620 0.000008426 0.000004070 0.000007210 0.000007505 0.000005873 0.000009712 0.000008975 0.000007580 0.000001301 0.000001084 0.000005780 0.000001304 0.000001528
98 93 91 80 75 94 91 80 86 85 93 93 89 84 86 91 98 88 88 80
extraction and recognition methods proposed in this study can be evaluated as alternative to previous digital modulation recognition studies such as statistical pattern recognition approaches, decision-theoretic approaches (Fabrizi, Lopes, & Lockhart, 1986; Chan & Gadbois, 1989; Nagy, 1994b; Jovanovic, Doroslovacki, & Dragosevic, 1994; Aljalili, 1995; Nandi & Azzouz, 1995; Azzouz & Nandi, 1997). The successfully modulation recognition ratios about % 80–90 were obtained from these studies (Fabrizi et al., 1986; Chan & Gadbois, 1989; Nagy, 1994b; Jovanovic et al., 1994; Al-jalili, 1995; Nandi & Azzouz, 1995), but these studies have many disadvantages such as requiring long signal durations and experienced modulation operators, complex algorithms, big size feature vectors and excessive computer storage requirements (Avci et al., 2005b), etc. For overcoming these disadvantages, in this study, both DWT and ANFIS were used for an efficiently feature extraction from four (ASK8, FSK8, PSK8, and QASK8) different digital modulated signals. Therefore, comparison of wavelet families for automatic digital modulation recognition by using this EDWANFIS method is performed in this study. 5. Conclusions The effective feature extraction is very important topic for pattern recognition applications. The DWT has been demonstrated as an effective tool for extracting information from the signals in many studies (Burrus, Gopinath, & Guo, 1998; Keeton & Schlindwein, 1997). For this reason, the DWT was used for feature extraction methods part in this study. Here, EDWANFIS system was developed for the explaining of the digital modulated signals using pattern
recognition methods. The digital modulation recognition performance of this method was demonstrated on the total 400 digital modulated signals. In this paper, EDWANFIS method is suggested for automatic digital modulation recognition and the comparison of db2, db3, db5, db8, db10, bior1.3, bior2.2, bior2.8, bior3.5, bior6.8, coif1, coif2, coif3, coif4, coif5, sym2, sym3, sym5, sym7, and sym8 wavelet filters for automatic digital modulation recognition is performed. A comparative study is given which uses 20 different feature extraction algorithms, which all of these methods based on DWT. In EDWANFIS system, adaptive wavelet entropies are calculated for very useful features for characterizing the digital modulated signals in this study. The information obtained from the adaptive wavelet entropy is related to the energy and amplitude of digital modulated signal. This new EDWANFIS automatic digital modulation recognition system obtains more successfully results than classic digital recognition approaches such as spectral, decisiontheoretic, and statistical pattern recognition approaches. Moreover, this study shows that the EDWANFIS automatic digital modulation recognition system has many advantages such as rapidity, easy algorithm.
References Al-jalili, Y. O. (1995). Identification algorithm for upper sideband and lower sideband SSB signals. Signal Processing, 42(2), 207–213. Alssaleh, K., Farrell, K., & Mammone, R. J. (1992). A new method of modulation classification for digitally modulated signals. In MELCOM 92, communication, fusing, command control, and intelligence (pp. 30.5.1–30.5.5).
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589 Avci, E., Turkoglu, I., & Poyraz, M. (2005a). Intelligent target recognition based on wavelet packet neural network. Experts Systems with Applications, 29(1). Avci, E., Turkoglu, I., & Poyraz, M. (2005b). In Intelligent target recognition based on wavelet adaptive network based fuzzy inference system. Lecture notes in computer science (Vols. 3522, pp. 594–601). Springer-Verlag, May. Azzouz, E. E., & Nandi, A. K. (1995). Automatic identification of digital modulations. Signal Processing, 47(1), 55–69. Azzouz, E. E., & Nandi, A. K. (1997). Automatic modulation recognition – I. Journal of Franklin Institute, 334B(2), 241–273. Beidas, B. F., & Weber, C. L. (1995). Higher-order correlation-based approach to modulation classification of digitally modulated signals. IEEE Journal of Selected Areas in Communications, 13(1). Bishop, C. M. (1996). Neural networks for pattern recognition. Oxford: Clarendon Press. Burrus, C. S., Gopinath, R. A., & Guo, H. (1998). Introduction to wavelet and wavelet transforms. NJ, USA: Prentice-Hall. Chan, Y. T., & Gadbois, L. G. (1989). Identification of the modulation type of a signal. Signal Processing, 16(2), 149–154. Davis, S. B., & Mermelstein, P. (1980). Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustic, Speech and Signal Processing, 28(4), 357–366. DeSimio, M. P., & Glenn, E. P. (1988). Adaptive generation of decision functions for classification of digitally modulated signals. In NAECON (pp. 1010–1014). Fabrizi, P. M., Lopes, L. B., & Lockhart, G. B. (1986). Receiver recognition of analogue modulation types. In IERE conference on radio receiver and associated systems, Bangor, Wales (pp. 135–140). Graps, A. (1995). An introduction to wavelets. Computational Science and Engineering, IEEE, 2(2), 50–61. Ha, Q. P., Tran, T. H., & Dissanayake, G. (2005). A wavelet and neural network based voice interface system for wheelchair control. International Journal of Intelligent Systems Technologies and Applications, 1(1/2). Hsue, Z. S., & Soliman, S. S. (1990). Automatic modulation classification using zero-crossing. lEE Proceedings, Part F Radar Signal Process, 137(6), 459–464. Jovanovic, S. D., Doroslovacki, M. I., Dragosevic, & M. V. (1994). Recognition of low modulation index AM signals in additive Gaussian noise. In European association for signal processing V conference, Edinburgh, Scotland (pp. 1923–1926). Kavalov, D. (2001). Improved noise characteristics of a saw artificial neural network RF signal processor for modulation recognition. In 2001 IEEE ultrasonics symposium (pp. 19–22). Keeton, P. I. J., & Schlindwein, F. S. (1997). Application of wavelets in Doppler ultrasound. Measurement, 17(1), 38–45.
589
Kremer, S. C., & Shiels, J. (1999). A testbed for automatic modulation recognition using artificial neural networks. In CCECE’97 (pp. 67–70). Liedtke, F. F. (1984). Computer simulation of an automatic classification procedure for digitally modulated communication signals with unknown parameters. Siynal Processing, 6(4), 311–323. Long, C. J., & Datta, S., 1996. Wavelet based feature extraction for phoneme recognition. In Proceedings of the 4th international conference of spoken language processing, Philadelphia, USA, Vol. 1 (pp. 264–267) (Oct. 1996). Nagy, P. A. J. (1994a). Analysis of a method for classification of analogue modulated radio signals. In European association for signal processing VII conference 94, Edinburgh, Scotland (pp. 1015–1018). Nagy, P. A. J. (1994b). A modulation classifier for multi-channel systems and multi-transmitter situations. In: MILCOM 1994 conference. Nandi, A. K., & Azzouz, E. E. (1995). Recognition of analogue modulations. Signal Processing, 46(2), 211–222. Nandi, A. K., & Azzouz, E. E. (1997). Modulation recognition using artificial neural networks. Signal Processing, 56, 165–175. Nandi, A. K., & Azzouz, E. E. (1998). Algorithms for automatic modulation recognition of communication signals. IEEE Transactions on Communications, 46, 431–436. Pati, Y. C., & Krishnaprasad, P. S. (1993). Analysis and synthesis of feedforward neural networks using discrete affine wavelet transform. IEEE Transactions on Neural Networks, 4(2), 73–85. Polydoros, A., & Kim, K. (1990). On the detection and classification of quadrature digital modulations in broad-band noise. IEEE Transactions on Communications, 38(8), 1199–1211. Soliman, S. S., & Hsue, Z. S. (1992). Signal classification using statistical moments. 1EEE Transactions on Communications, 40(5), 908–916. Tufekci, Z., & Gowdy, J. N. (2000). Feature extraction using discrete wavelet transform for speech recognition. Proceedings of IEEE Southeastcon, 116–123. Wong, M. L. D., & Nandi, A. K. (2004). Automatic digital modulation recognition using artificial neural network and genetic algorithm. Signal Processing, 84, 351–365. Wu, Z., Ren, G., Wang, X., & Zhao, Y. (2004). Automatic digital modulation recognition using wavelet transform and neural networks. ISNN 2004, LNCS 3173 (pp. 936–940). Wu, Z., Wang, X., Liu, C., & Ren, G. (2005). Automatic digital modulation recognition based on ART2A-DWNN. ISNN 2005, LNCS 3497 (pp. 381–386). Zadeh, L. A. (1987). Fuzzy sets and applications: selected papers by L.A. Zadeh. New York: John Wiley & Sons. Zhang, J., Walter, G. G., Miao, Y., & Lee, W. N. W. (1995). Wavelet neural networks for function learning. IEEE Transactions on Signal Processing, 43(6).
An expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System for digital modulation recognition Engin Avci *, Davut Hanbay, Asaf Varol Firat University, Department of Electronic and Computer Education, 23119 Elazig, Turkey
Abstract This paper presents a comparative study of implementation of feature extraction and classification algorithms based on discrete wavelet decompositions and Adaptive Network Based Fuzzy Inference System (ANFIS) for digital modulation recognition. Here, in first stage, 20 different feature extraction methods are generated by separately using Daubechies, Biorthogonal, Coiflets, Symlets wavelet families. In second stage, the performance comparison of these feature extraction methods is performed by using a new Expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System (EDWANFIS). The digital modulated signals used in this experimental study are ASK8, FSK8, PSK8, QASK8. EDWANFIS structure consists of two parts. The first part is Discrete Wavelet Transform (DWT)adaptive wavelet entropy and Adaptive Network Based Fuzzy Inference System for Automatic Digital Modulation Recognition (ADMR). The performance of this comparison system is evaluated by using total 800 digital modulated signals for each of these feature extraction methods. The performance comparison of these features extraction methods and the advantages and disadvantages of the methods are examined. 2006 Elsevier Ltd. All rights reserved. Keywords: Digital modulation; DWT; Wavelet Adaptive Network Based Fuzzy Inference System; ADMR; Feature extraction; Wavelet entropy; Expert system
1. Introduction Automatic modulation recognition (AMR) is an essentially interesting problem with a variety of regulatory and military communication applications (Wu, Ren, Wang, & Zhao, 2004). AMR is used for military communication intelligence studies such as spectrum surveillance, threat estimation, etc. In previous studies, modulation recognition was performed by using human operator’s interpretation of measured parameters to classify signals (Wu, Wang, Liu, & Ren, 2005). In these studies, modulated signal properties such as signal spectrum features, instantaneous amplitudes
* Corresponding author. Tel.: +90 4242370000 4257; fax: +90 4242367064. E-mail addresses: enginavci@firat.edu.tr, [email protected] (E. Avci), dhanbay@firat.edu.tr (D. Hanbay).
0957-4174/$ - see front matter 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2006.06.001
and instantaneous phase were evaluated for modulation recognition. Then, a lot of methods have been developed for Automatic Digital Modulation Recognition (ADMR). In these studies, the demodulator banks designed for each of special modulation types were used for ADMR (Wu et al., 2005). The demodulator bank method is called as semi-automatic because of a human operator is still required to listen outputs of these demodulator banks (Wu et al., 2005). In Liedtke (1984), an ADMR method is presented for some types of digital modulation. The types of digital modulation are ASK2, FSK2, PSK2, and PSK4, PSK8. In this study, universial demodulator technique is used for recognizer. For this reason, amplitude histogram, the frequency histogram, the phase difference histogram, amplitude variance, and the frequency variance key features are used. In DeSimio and Glenn (1988), an adaptive technique is used for ADMR. Some types of digital modulation are used in this study. These digital modulation types are ASK2,
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
PSK2, PSK4, and FSK2. In this study, the key features derived from the signal envelope, the signal spectra, the signal squared and the fourth power of the signal are used to decide type of modulation. In Polydoros and Kim (1990), a modulation recognizer which uses decision-theoretic approach is presented for PSK2 and PSK4. In this study, carrier frequency, the initial phase, the symbol rate and the signal-to-noise ratio are used as key features for ADMR. In Hsue and Soliman (1990), a modulation recognizer which uses zero-crossing characteristic of the modulated signals is presented for MPSK, MFSK. In Soliman and Hsue (1992), a modulation recognizer which uses the statistical moments of the modulated signal phase is introduced. Here, the even order moments of the modulated signal phase are used for evaluate the number of levels of MPSK signals. In Alssaleh, Farrell, and Mammone (1992), key features which are obtained from the averaged spectrum of the instantaneous frequency are used for PSK2, PSK4, FSK2, and FSK4 digital modulation types. In Nagy (1994a), a modulation recognition method is presented for multichannel systems. Here, the intercepted signal is divided into individual parts. Then, each of these individual parts is classified by using a single tone classifier for recognizing of ASK, PSK2, PSK4, and FSK2. In Beidas and Weber (1995), a modulation recognizer which uses the time-domain Higher-Order Correlations is presented for discriminate between the MFSK signals. In Azzouz and Nandi (1995), a modulation recognizer which uses the key features used are obtained from the instantaneous amplitude, the instantaneous phase and the instantaneous frequency of the intercepted signal is introduction for the ASK2, ASK4, PSK2, PSK4, FSK2, and FSK4 digital modulation types. A lot of studies have realized on the topic of automatic modulation recognition using ANNs approximations (Nandi & Azzouz, 1998; Kavalov, 2001; Kremer & Shiels, 1999). Nandi and Azzouz (1997) proposed a single hidden layer ANN structure for automatic modulation classification. This network has a 4-node input layer, a 25-node hidden layer and a 7-node output layer. Nevertheless a degradation of performance at higher signal noise ratios (SNR) will appear when the ANN is trained on signals with lower SNR. The generalization capability of ANNs must be increased for overcoming this shortcoming of ANNs classifiers. Therefore, a compact set of features, which capture all the major characteristics of the intercepted signals in a relatively small number of the components must be obtained from intercepted signal (Wu et al., 2004). Then, these features must be given to ANN inputs for modulations classification. For this reason, the wavelet transform is used for the extraction of key features at pattern recognition and classification areas (Wu et al., 2004). Nowadays, in many areas such as image processing, signal processing, especially image compression, speech processing, computer vision, the wavelet transform types are commonly used (Wu et al., 2004). The DWT have been used in automatic digital modulation recognition for communi-
583
cation signal processing (Wu et al., 2004); nevertheless, to date Discrete Wavelet Adaptive Network Based Fuzzy Inference System (DWANFIS) analysis for automatic digital modulation recognition using adaptive entropy approach is a relatively new approach. In this study presents a new method (EDWANFIS) in ADMR. The novelties presented in this study can be arranged in order as below: (1) The effectiveness of the Discrete Wavelet Transform (DWT) features is shown to be used for ADMR. (2) Expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System (EDWANFIS) on adaptive wavelet algorithm is used for increasing the effectiveness of the ADMR at various SNR rates and various parameters changing. (3) At this study, DWT and EDWANFIS method on ADMR field is firstly have been used for automatic digital modulation literature. More successfully results at various SNR rates and various parameters changing than preceding studies realized by using statistical pattern recognition, and decision-theoretic approach were obtained from this application. Here, the comparison of Daubechies, Biorthogonal, Coiflets, Symlets wavelet families for digital modulation recognition is performed by using a new Expert Discrete Wavelet Adaptive Network Based Fuzzy Inference System (EDWANFIS) method. The digital modulated signals used in this experimental study are ASK8, FSK8, PSK8, QASK8. This paper is organized as follows: In Section 2, Discrete Wavelet Adaptive Network Based Fuzzy Inference System (DWANFIS), and DWT are briefly reviewed. In Section 3, digital modulated signals database, ADMR system based on EDWANFIS, and experimental results are explained respectively. In Section 4, conclusion and discussions are given. 2. Theory 2.1. DWT and Discrete Wavelet Adaptive Network Based Fuzzy Inference System Discrete Wavelet Transform decomposes non-station signals at different frequency intervals with various resolutions. DWT analyses a signal into rough approximation (a) and detail (d) components. Therefore, DWT is commonly used in feature extraction and classification stages of pattern recognition. Discrete wavelet transform performs two functions (Graps, 1995). These functions which can be shown as low-pass and high-pass filters are called scaling functions and wavelet functions. Wavelet coefficients of the DWT at 7 levels are shown as Fig. 1. The original modulated signal is passed through lowpass and high-pass filters, and approximation (a), and detail (d) coefficients signals are obtained. DWT uses the
584
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
Signal
a1
a2
a3
a4
a5
a6
a7
d1
d2
d3
d4
d5
d6
d7 Fig. 1. Wavelet coefficients of the DWT at 7 levels.
fact that it is possible to resolve high frequency components within a small time window, and only low frequency components need large time windows. This is because a low frequency component completes a cycle in a large time interval whereas a high frequency component completes a cycle in a much shorter interval. Therefore, slow varying components can only be identified over long time intervals but fast varying components can be identified over short time intervals. DWT can be regarded as a continuous time wavelet decomposition sampled at different frequencies at every level or stage. Both artificial neural network and fuzzy logic are used in ANFIS’s architecture. ANFIS is consisted of if-then rules and couples of input–output, for ANFIS training is used learning algorithms of neural network (Zadeh, 1987). Adaptive Network Based Fuzzy Inference Systems (ANFIS) are systems that are constructed to make use of some organizational principles resembling those of the human brain (Davis & Mermelstein, 1980). ANFIS represent the promising new generation of information processing systems. Adaptive network based fuzzy inference systems are good at tasks such as pattern matching and classification, function approximation, optimization and data clustering, while traditional computers, because of their architecture, are inefficient at these tasks, especially pattern-matching tasks (Davis & Mermelstein, 1980; Tufekci & Gowdy, 2000). As for DWANFIS try to com-
bine aspects of the discrete wavelet transformation for purpose of feature extraction and selection with the characteristic decision capabilities of adaptive network based fuzzy inference system approaches (Long & Datta, 1996). The Discrete Wavelet Adaptive Network Based Fuzzy Inference System (DWANFIS) is constructed based on the discrete wavelet transform theory (Ha, Tran, & Dissanayake, 2005; Pati & Krishnaprasad, 1993) and is an alternative to adaptive network based fuzzy inference systems (Zhang, Walter, Miao, & Lee, 1995). Discrete wavelet transform (Avci, Turkoglu, & Poyraz, 2005a) is a powerful tool for non-stationary signal analysis. Let x(t) be a piecewise continuous function. Discrete wavelet transform allows one to decompose x(t) using a wavelet function w:Rn ! R. Based on the wavelet decomposition, DWANFIS structure is defined by P wi fi ðw½Di ðx ti ÞÞ P yðxÞ ¼ i ð1Þ i wi here wi are weights of the DWANFIS inputs, Di are dilation vectors specifying the diagonal dilation matrices Di, ti are translation vectors, and fi are Sugeno output functions of the ANFIS. An algorithm of the hybrid type has been derived for adjusting the parameters of the DWANFIS (Davis & Mermelstein, 1980; Tufekci & Gowdy, 2000; Zadeh, 1987). Applications of wavelet adaptive network based fuzzy inference system in the medical field include for detection of electrocardiography changes in patients with partial epilepsy using feature extraction (Bishop, 1996; Wong & Nandi, 2004; Avci, Turkoglu, & Poyraz, 2005b), bearing fault diagnosis based on wavelet transform and fuzzy inference (Avci et al., 2005a; Bishop, 1996), for satellite image fusion (Zadeh, 1987); nevertheless, to date EDWANFIS analysis for automatic digital modulation recognition using adaptive entropy approach is a relatively new approach. 3. Methodology and experimental applications 3.1. Digital modulated signal database At the digital modulation systems, a modulated signal can be commonly represented as KðtÞ ¼ M n sD ðtÞ cosð2pfn ðtÞ þ hn ðtÞÞ;
ð2Þ
where Mn, fn, hn, and sD are the message amplitude, message frequency, message phase, and pulse shaping function, respectively (Wong & Nandi, 2004). The digital message signal used in this study can be shown as in Fig. 2. The four important types of digital modulation techniques are Amplitude Shift Keying-8 (ASK8), Frequency Shift Keying-8 (FSK8), Phase Shift Keying-8 (PSK8), and Quadrature Amplitude Shift Keying-8 (QASK8). In this study, the generating phases of the digital modulating signal database are given as below:
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589 Digital Modulated Signal
6
Norm Entropy1
3 2
Norm EntropyN
Adaptive Network Based Fuzzy Inference System
1 0
error 0
100
200
300 400 500 600 700 800 Sample numbers of message signal
900
1000
+
∑
-
desired output
Classification Block
Adjustable Parameters
4
Structure of EDWANFIS Automatic Digital Modulation Recognition System
DWT
5
Feature Extraction Block
Amplitude of message signal (M=8 Levels)
7
585
Fig. 2. The digital message signal used in this study. Fig. 3. The structure of DWNN for analog modulation classification. Table 1 The parameters of message signal used in this study Parameters
Values
Sample numbers of message signal Sampling frequency , fs Carrier frequency, fc Baud rate
1000 samples 25 Hz 10 Hz 300 baud
Here, the message signal is digitally simulated according to Equal (2) in MATLAB Communication Toolbox. First, this message signal is generated by using random integer up to M = 8 – level. Second, this message signal is re-sampled at baud rate (300 baud) for pulse shaping before passing through respective modulators. The message signal parameters used for digital modulation are given in Table 1. Third, 200 simulated signals of each of the digital modulation types of ASK8, FSK8, PSK8, and QASK8 have been generated for these experimental studies. In this reason, additive white Gaussian noise is added according the various signal-to-noise ratios (SNR), i.e. at between 5 and 60 dB. Hundred of these 200 digital modulated signals for each of these four types digital modulation (ASK8, FSK8, PSK8, and QASK8) are used for training stage of the EDWANFIS system. Others 100 of 200 digital modulated signals for each of these four types digital modulation (ASK8, FSK8, PSK8, and QASK8) are used for testing stage of the EDWANFIS system. Totally 800 digital modulated signals are used for the training and testing phases of EDWANFIS system developed in this study. 3.2. Using of EDWANFIS Automatic Digital Modulation Recognition System EDWANFIS structure developed in this study for recognition of digital modulated signal waveform patterns is showed in Fig. 3. EDWANFIS system is given in Fig. 3.
This system is separated two layers, which are feature extraction and classification layers. The feature extraction layer of EDWANFIS system is the most important component of designing the automatic system based on digital modulation recognition. The tree structure is used m = 7 as level for DWT of the digital modulated signals. Four various digital modulation types (ASK8, FSK8, PSK8, and QASK8) are used to obtaining the digital modulated signals. Two hundred simulated signals for each of these four types digital modulation are generated. Hundred of these 200 modulated signals for each of these four types digital modulation are generated for training layer of the EDWANFIS automatic digital modulation recognition system. Others 100 of 200 digital modulated signals for each of these four types digital modulation are generated for testing layer of the EDWANFIS automatic digital modulation recognition system. In this way, totally 800 digital modulated signals are generated for the training and testing layers of EDWANFIS automatic digital modulation recognition system developed in this study. 3.2.1. DWT phase The DWT is applied for each of these 800 digital modulated signals. The decomposition structure and reconstruction trees at level 7 as shown in Fig. 1 are used. db2, db3, db5, db8, db10, bior1.3, bior2.2, bior2.8, bior3.5, bior6.8, coif1, coif2, coif3, coif4, coif5, sym2, sym3, sym5, sym7, and sym8 wavelet filters for Daubechies, Biorthogonal, Coiflets, and Symlets are separately used for DWT phase of EDWANFIS automatic digital modulation recognition system respectively. Thus, performance comparison of these wavelet filters for automatic digital modulation recognition by using EDWANFIS is performed. In this way, for each of these wavelet filters, two types of coefficients: one approximation coefficients cA and seven – detail coefficients cDs are obtained from DWT decomposi-
586
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
tion of each of 800 digital modulated signals. The ASK8 digital modulated signal used in this study, a detail coefficient waveforms of DWT at ASK8 modulated signal by using coif2 wavelet filters can be shown as in Figs. 4 and 5, respectively. For each of these DWT coefficients, adaptive wavelet norm entropy values are calculated by using equation given in Eq. (3) (Avci et al., 2005b). P p j si j EðsÞ ¼ i ð3Þ N where the adaptive wavelet entropy E is a real number, s is the terminal node signal and (si) i the waveform of terminal node signals. In norm entropy, P is the power and must be such that 1 6 P < 2. For the EDWANFIS learning process, the P parameter is updated by using 0.1 increasing steps together with weights to minimize the error. Updating of this P parame1 Amplitude of digital modulated signal
0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Length of digital modulated signal
Fig. 4. An original ASK8 modulated signal used in this study.
0.15 0.1
Magnitude (dB)
0.05 0 -0.05 -0.1 -0.15 -0.2 0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Time in seconds
Fig. 5. A detail coefficient waveforms of DWT at ASK8 modulated signal by using coif2 wavelet filters.
ter is performed at the end of 1000 epoch numbers for training of Adaptive Network Based Fuzzy Inference System (ANFIS). The resultant entropy data are normalized by dividing with maximum norm entropy value. Thus, the feature vector is extracted by computing the 8-wavelet entropy values per digital modulated signal. In this way, size of the feature vector obtained by calculating adaptive wavelet norm entropy values for training layer of EDWANFIS is 4 · 100 · 8 = 400 · 8. This feature vector takes from number of digital modulation types used in this study x number of digital modulated signals used for each of digital modulation types x number of adaptive wavelet norm entropies calculated for DWT decomposition coefficients of a digital modulated signal. In the same way, a 4 · 100 · 8 = 400 · 8 feature vector is obtained from other 100 digital modulated signals for each of ASK8, FSK8, PSK8, and QASK8 digital modulation types in testing layer of EDWANFIS. In this study, for this feature vector was generated by separately using db2, db3, db5, db8, db10, bior1.3, bior2.2, bior2.8, bior3.5, bior6.8, coif1, coif2, coif3, coif4, coif5, sym2, sym3, sym5, sym7, and sym8 wavelet filters. The feature vectors generated by using these wavelet filters can be named as in below respectively: • The feature vector obtained by using db2 wavelet filter (FV-1): For this feature vector, db2 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using db3 wavelet filter (FV-2): For this feature vector, db3 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using db5 wavelet filter (FV-3): For this feature vector, db5 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using db8 wavelet filter (FV-4): For this feature vector, db8 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using db10 wavelet filter (FV-5): For this feature vector, db10 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior1.3 wavelet filter (FV-6): For this feature vector, bior1.3 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior2.2 wavelet filter (FV-7): For this feature vector, bior2.2 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior2.8 wavelet filter (FV-8): For this feature vector, bior2.8 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior3.5 wavelet filter (FV-9): For this feature vector, bior3.5 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using bior6.8 wavelet filter (FV-10): For this feature vector, bior6.8 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using coif1 wavelet filter (FV-11): For this feature vector, coif1 wavelet filter is used for DWT decomposition stage of EDWANFIS.
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
• The feature vector obtained by using coif2 wavelet filter (FV-12): For this feature vector, coif2 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using coif3 wavelet filter (FV-13): For this feature vector, coif3 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using coif4 wavelet filter (FV-14): For this feature vector, coif4 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using coif5 wavelet filter (FV-15): For this feature vector, coif5 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym2 wavelet filter (FV-16): For this feature vector, sym2 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym3 wavelet filter (FV-17): For this feature vector, sym3 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym5 wavelet filter (FV-18): For this feature vector, sym5 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym7 wavelet filter (FV-19): For this feature vector, sym7 wavelet filter is used for DWT decomposition stage of EDWANFIS. • The feature vector obtained by using sym8 wavelet filter (FV-20): For this feature vector, sym8 wavelet filter is used for DWT decomposition stage of EDWANFIS. 3.2.2. ANFIS phase Both artificial neural network and fuzzy logic are used in ANFIS’s architecture. ANFIS is consisted of if-then rules and couples of input–output, for ANFIS training is used learning algorithms of neural network (Davis & Mermelstein, 1980; Zadeh, 1987). Readers may find more information about ANFIS structure at (Davis & Mermelstein, 1980; Zadeh, 1987). This stage realizes the intelligent classification using features from discrete wavelet stage. The training parameters and the structure of the ANFIS used in this study are shown in Table 2. These parameters were selected for the best performance, after several different experiments, such as the number of input membership functions, the size of the ANFIS layers, value of the moment constant and learning rate, and type of the activation functions. Table 2 The training parameters and the structure of the ANFIS used in this study Number of inputs Number of outputs Rules number The number of layers Learning rule
Sum-squared error
8 4 256 5 Hybrid learning algorithm (back-propagation for nonlinear parameters (ai, ci) and Least squre errors for linear parameters (pi, qi,ri, si, ssi, ppi,rri, qqi, ui)) 0.00001
587
This ANFIS structure has eight inputs that are adaptive wavelet entropies, which are given in Eq. (3). These inputs are cA1, cD1, cD2, cD3, cD4, cD5, cD6, cD7 DWT coefficients. This architecture is formed by using five layers and 256 if-then rules. 4. Results of the experimental studies It was performed experiments using total 4 (number of digital modulation types used in this study) · 200 (number of digital modulated signals used for each of digital modulation types) = 800 digital modulated signals. 4 · 100 = 400 of these signals were used for training of EDWANFIS. Others 4 · 100 = 400 signals were used for testing of this EDWANFIS system. These training and testing processes were repeated for each of feature vectors which are FV-1, FV-2, FV-3, FV-4, FV-5, FV-6, FV-7, FV-8, FV-9, FV10, FV-11, FV-12, FV-13, FV-14, FV-15, FV-16, FV-17, FV-18, FV-19, and FV-20. In this way, the performance comparison of db2, db3, db5, db8, db10, bior1.3, bior2.2, bior2.8, bior3.5, bior6.8, coif1, coif2, coif3, coif4, coif5, sym2, sym3, sym5, sym7, and sym8 wavelet filters for automatic digital modulation recognition were performed. This performance comparison was realized for expended training time of ANFIS and correct recognition rate of digital modulation types in test stage for each of these feature vectors. The comparison results were given below. The performance comparison for expended training time of ANFIS and correct recognition rate of digital modulation types in test stage are tabulated in Table 3. As shown in Table 1, FV-2 feature extraction algorithm for EDWANFIS reached to desired error value in the shortest time (Training epoch numbers: 342). Therefore, training performance of this algorithm is the better than the others. Updating of P parameter of adaptive wavelet norm entropy is performed at the end of 1000 epoch numbers for training of ANFIS. Last value of P parameter of adaptive wavelet norm entropy is 1 for this FV-2 feature extraction algorithm. Moreover, FV-20 feature extraction algorithm for EDWANFIS reached to desired error value in the longest time (Training epoch numbers: 7062). Therefore, training performance of this algorithm is the worse than the others. Last value of P parameter of adaptive wavelet norm entropy is 1.7 for this algorithm. Namely, updating of P parameter of adaptive wavelet norm entropy has performed seven times (1.1, 1.2, 1.3, 1.4, 1.5, 1.6, and 1.7) by using FV-20 feature extraction algorithm for EDWANFIS. Because the P parameter is updated by using 0.1 increasing steps during the EDWANFIS learning process. The last column of Table 1 shows the correct recognition rate of digital modulation types by using feature extraction algorithms. As it is clear from the Table 1, FV-1 and FV-17 feature extraction algorithms for EDWANFIS have best correctness performance for recognition of digital modulation types. This state clearly indicated that the feature extraction methods realized by using wavelet families are very efficiently for automatic digital modulation recognition. Therefore, the feature
588
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589
Table 3 Performance comparison for expended time at training of ANFIS and correct recognition rate of digital modulation types in test stage for each of these feature vectors The feature vectors
Training epoch numbers
Last value of P parameter of adaptive wavelet norm entropy
MSE (Mean Square Error) value
Correct recognition rate of digital modulation types (%)
FV-1 FV-2 FV-3 FV-4 FV-5 FV-6 FV-7 FV-8 FV-9 FV-10 FV-11 FV-12 FV-13 FV-14 FV-15 FV-16 FV-17 FV-18 FV-19 FV-20
986 343 1154 876 972 359 447 2059 468 1121 741 1560 642 882 1487 1347 4043 1137 2107 7062
1 1 1.1 1 1 1 1 1.2 1 1.1 1 1.1 1 1 1.1 1.1 1.4 1.1 1.2 1.7
0.000008296 0.000007254 0.000007388 0.000006130 0.000009899 0.000002270 0.000005620 0.000008426 0.000004070 0.000007210 0.000007505 0.000005873 0.000009712 0.000008975 0.000007580 0.000001301 0.000001084 0.000005780 0.000001304 0.000001528
98 93 91 80 75 94 91 80 86 85 93 93 89 84 86 91 98 88 88 80
extraction and recognition methods proposed in this study can be evaluated as alternative to previous digital modulation recognition studies such as statistical pattern recognition approaches, decision-theoretic approaches (Fabrizi, Lopes, & Lockhart, 1986; Chan & Gadbois, 1989; Nagy, 1994b; Jovanovic, Doroslovacki, & Dragosevic, 1994; Aljalili, 1995; Nandi & Azzouz, 1995; Azzouz & Nandi, 1997). The successfully modulation recognition ratios about % 80–90 were obtained from these studies (Fabrizi et al., 1986; Chan & Gadbois, 1989; Nagy, 1994b; Jovanovic et al., 1994; Al-jalili, 1995; Nandi & Azzouz, 1995), but these studies have many disadvantages such as requiring long signal durations and experienced modulation operators, complex algorithms, big size feature vectors and excessive computer storage requirements (Avci et al., 2005b), etc. For overcoming these disadvantages, in this study, both DWT and ANFIS were used for an efficiently feature extraction from four (ASK8, FSK8, PSK8, and QASK8) different digital modulated signals. Therefore, comparison of wavelet families for automatic digital modulation recognition by using this EDWANFIS method is performed in this study. 5. Conclusions The effective feature extraction is very important topic for pattern recognition applications. The DWT has been demonstrated as an effective tool for extracting information from the signals in many studies (Burrus, Gopinath, & Guo, 1998; Keeton & Schlindwein, 1997). For this reason, the DWT was used for feature extraction methods part in this study. Here, EDWANFIS system was developed for the explaining of the digital modulated signals using pattern
recognition methods. The digital modulation recognition performance of this method was demonstrated on the total 400 digital modulated signals. In this paper, EDWANFIS method is suggested for automatic digital modulation recognition and the comparison of db2, db3, db5, db8, db10, bior1.3, bior2.2, bior2.8, bior3.5, bior6.8, coif1, coif2, coif3, coif4, coif5, sym2, sym3, sym5, sym7, and sym8 wavelet filters for automatic digital modulation recognition is performed. A comparative study is given which uses 20 different feature extraction algorithms, which all of these methods based on DWT. In EDWANFIS system, adaptive wavelet entropies are calculated for very useful features for characterizing the digital modulated signals in this study. The information obtained from the adaptive wavelet entropy is related to the energy and amplitude of digital modulated signal. This new EDWANFIS automatic digital modulation recognition system obtains more successfully results than classic digital recognition approaches such as spectral, decisiontheoretic, and statistical pattern recognition approaches. Moreover, this study shows that the EDWANFIS automatic digital modulation recognition system has many advantages such as rapidity, easy algorithm.
References Al-jalili, Y. O. (1995). Identification algorithm for upper sideband and lower sideband SSB signals. Signal Processing, 42(2), 207–213. Alssaleh, K., Farrell, K., & Mammone, R. J. (1992). A new method of modulation classification for digitally modulated signals. In MELCOM 92, communication, fusing, command control, and intelligence (pp. 30.5.1–30.5.5).
E. Avci et al. / Expert Systems with Applications 33 (2007) 582–589 Avci, E., Turkoglu, I., & Poyraz, M. (2005a). Intelligent target recognition based on wavelet packet neural network. Experts Systems with Applications, 29(1). Avci, E., Turkoglu, I., & Poyraz, M. (2005b). In Intelligent target recognition based on wavelet adaptive network based fuzzy inference system. Lecture notes in computer science (Vols. 3522, pp. 594–601). Springer-Verlag, May. Azzouz, E. E., & Nandi, A. K. (1995). Automatic identification of digital modulations. Signal Processing, 47(1), 55–69. Azzouz, E. E., & Nandi, A. K. (1997). Automatic modulation recognition – I. Journal of Franklin Institute, 334B(2), 241–273. Beidas, B. F., & Weber, C. L. (1995). Higher-order correlation-based approach to modulation classification of digitally modulated signals. IEEE Journal of Selected Areas in Communications, 13(1). Bishop, C. M. (1996). Neural networks for pattern recognition. Oxford: Clarendon Press. Burrus, C. S., Gopinath, R. A., & Guo, H. (1998). Introduction to wavelet and wavelet transforms. NJ, USA: Prentice-Hall. Chan, Y. T., & Gadbois, L. G. (1989). Identification of the modulation type of a signal. Signal Processing, 16(2), 149–154. Davis, S. B., & Mermelstein, P. (1980). Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustic, Speech and Signal Processing, 28(4), 357–366. DeSimio, M. P., & Glenn, E. P. (1988). Adaptive generation of decision functions for classification of digitally modulated signals. In NAECON (pp. 1010–1014). Fabrizi, P. M., Lopes, L. B., & Lockhart, G. B. (1986). Receiver recognition of analogue modulation types. In IERE conference on radio receiver and associated systems, Bangor, Wales (pp. 135–140). Graps, A. (1995). An introduction to wavelets. Computational Science and Engineering, IEEE, 2(2), 50–61. Ha, Q. P., Tran, T. H., & Dissanayake, G. (2005). A wavelet and neural network based voice interface system for wheelchair control. International Journal of Intelligent Systems Technologies and Applications, 1(1/2). Hsue, Z. S., & Soliman, S. S. (1990). Automatic modulation classification using zero-crossing. lEE Proceedings, Part F Radar Signal Process, 137(6), 459–464. Jovanovic, S. D., Doroslovacki, M. I., Dragosevic, & M. V. (1994). Recognition of low modulation index AM signals in additive Gaussian noise. In European association for signal processing V conference, Edinburgh, Scotland (pp. 1923–1926). Kavalov, D. (2001). Improved noise characteristics of a saw artificial neural network RF signal processor for modulation recognition. In 2001 IEEE ultrasonics symposium (pp. 19–22). Keeton, P. I. J., & Schlindwein, F. S. (1997). Application of wavelets in Doppler ultrasound. Measurement, 17(1), 38–45.
589
Kremer, S. C., & Shiels, J. (1999). A testbed for automatic modulation recognition using artificial neural networks. In CCECE’97 (pp. 67–70). Liedtke, F. F. (1984). Computer simulation of an automatic classification procedure for digitally modulated communication signals with unknown parameters. Siynal Processing, 6(4), 311–323. Long, C. J., & Datta, S., 1996. Wavelet based feature extraction for phoneme recognition. In Proceedings of the 4th international conference of spoken language processing, Philadelphia, USA, Vol. 1 (pp. 264–267) (Oct. 1996). Nagy, P. A. J. (1994a). Analysis of a method for classification of analogue modulated radio signals. In European association for signal processing VII conference 94, Edinburgh, Scotland (pp. 1015–1018). Nagy, P. A. J. (1994b). A modulation classifier for multi-channel systems and multi-transmitter situations. In: MILCOM 1994 conference. Nandi, A. K., & Azzouz, E. E. (1995). Recognition of analogue modulations. Signal Processing, 46(2), 211–222. Nandi, A. K., & Azzouz, E. E. (1997). Modulation recognition using artificial neural networks. Signal Processing, 56, 165–175. Nandi, A. K., & Azzouz, E. E. (1998). Algorithms for automatic modulation recognition of communication signals. IEEE Transactions on Communications, 46, 431–436. Pati, Y. C., & Krishnaprasad, P. S. (1993). Analysis and synthesis of feedforward neural networks using discrete affine wavelet transform. IEEE Transactions on Neural Networks, 4(2), 73–85. Polydoros, A., & Kim, K. (1990). On the detection and classification of quadrature digital modulations in broad-band noise. IEEE Transactions on Communications, 38(8), 1199–1211. Soliman, S. S., & Hsue, Z. S. (1992). Signal classification using statistical moments. 1EEE Transactions on Communications, 40(5), 908–916. Tufekci, Z., & Gowdy, J. N. (2000). Feature extraction using discrete wavelet transform for speech recognition. Proceedings of IEEE Southeastcon, 116–123. Wong, M. L. D., & Nandi, A. K. (2004). Automatic digital modulation recognition using artificial neural network and genetic algorithm. Signal Processing, 84, 351–365. Wu, Z., Ren, G., Wang, X., & Zhao, Y. (2004). Automatic digital modulation recognition using wavelet transform and neural networks. ISNN 2004, LNCS 3173 (pp. 936–940). Wu, Z., Wang, X., Liu, C., & Ren, G. (2005). Automatic digital modulation recognition based on ART2A-DWNN. ISNN 2005, LNCS 3497 (pp. 381–386). Zadeh, L. A. (1987). Fuzzy sets and applications: selected papers by L.A. Zadeh. New York: John Wiley & Sons. Zhang, J., Walter, G. G., Miao, Y., & Lee, W. N. W. (1995). Wavelet neural networks for function learning. IEEE Transactions on Signal Processing, 43(6).