- Email: [email protected]
Research on WNN soft fault diagnosis for analog circuit based on adaptive UKF algorithm
Accepted Manuscript Title: Research on WNN Soft Fault Diagnosis for Analog Circuit Based on Adaptive UKF Algorithm Author: Gan Xu-sheng Gao Wen-ming Dai Zhe Liu Wei-dong PII: DOI: Reference:
S1568-4946(16)30581-6 http://dx.doi.org/doi:10.1016/j.asoc.2016.11.012 ASOC 3902
To appear in:
Applied Soft Computing
Received date: Revised date: Accepted date:
11-8-2015 17-7-2016 7-11-2016
Please cite this article as: Gan Xu-sheng, Gao Wen-ming, Dai Zhe, Liu Wei-dong, Research on WNN Soft Fault Diagnosis for Analog Circuit Based on Adaptive UKF Algorithm, Applied Soft Computing Journal http://dx.doi.org/10.1016/j.asoc.2016.11.012 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Research on WNN Soft Fault Diagnosis for Analog Circuit Based on Adaptive UKF Algorithm The correct order :
Author Names 1. Gan Xu-sheng 2. Gao Wen-ming 3. Dai Zhe 4. Liu Wei-dong
Affiliations XiJing College, Shaanxi, Xi’an 710123, China Air Traffic Control and Navigation College, Air Force Engineering University Air Traffic Control and Navigation College, Air Force Engineering University Air Traffic Control and Navigation College, Air Force Engineering University
Address of correspondence: Air Traffic Control and Navigation College, Air Force Engineering University, Shaanxi, Xi’an 710038, China The correspondence author： Gan Xu-sheng E-mail: [email protected]
Tele: 15934896556
Graphical abstract
Analog circuit Adaptive factor
R2 3k C1
R1
V+
+ V1
+
UKF algorithm
1k
Wavelet neural network
5n
C2 +
V in
Out
5n
2k
15Vdc
V 2 -15Vdc
R3 R4
V-
+
4k
Adaptive UKF
R5 0
WNN training based on adaptive UKF algorithm
Sample construction using multi-resolution transform
Soft fault samples
AUKF-WNN
Train
4k
Soft fault diagnosis model based on AUKF-WNN
Fault mode
R2-
R2+
R3-
R3+
C1-
C1+
C2-
C2+
Normal
Total
Sample number
32
30
28
27
29
29
32
32
31
270
C
27
24
23
23
25
24
27
29
27
229
R
84.4%
80.0%
82.1%
85.2%
86.2%
82.8%
84.4%
90.6%
87.1%
84.8%
BP-WNN
C
29
26
25
24
26
25
28
30
29
242
R
90.6%
86.7%
89.3%
88.9%
89.7%
86.2%
87.5%
93.8%
93.6%
89.6%
UKF-WNN
C
32
30
27
25
29
26
32
31
31
263
R
100%
100%
96.4%
92.6%
100%
89.7%
100%
96.9%
100%
97.4%
AUKF-WNN
Fault diagnosis results
0
Research Highlights 1. An improved UKF algorithm with an adaptive factor is proposed. 2. A WNN modeling method based on improved UKF algorithm, namely AUKF- WNN, is given. 3. a construction method of fault samples using Multi-resolution Transform is introduced in soft fault diagnosis for analog circuit. 4. AUKF-WNN is used to establish the diagnosis model of soft fault for analog circuit. 5. For aerodynamic modeling, AUKF-WNN model shows a good diagnosis capability of soft fault for analog circuit.
Abstract: To improve the diagnosis capability of soft fault for analog circuit, a WNN diagnosis model is proposed based on fault feature samples extracted, which is trained by a modified UKF algorithm. An adaptive factor is firstly introduced to enhance the accuracy of UKF algorithm. Then, the UKF algorithm with adaptive factor is used to optimize the parameters of WNN, establishing the soft fault diagnosis model for fault feature samples extracted by multi-resolution transform. Finally, each fault mode is diagnosed and determined by the model. The simulation experiment on Sallen-Key bandpass filter indicates that, the proposed method has a good convergence rate and diagnosis accuracy rate for all faults in analog circuit. The feasibility and effectiveness of this method is also validated. Keywords: Wavelet Neural Network; UKF Algorithm; Multi-resolution Transform; Analog Circuit; Fault Diagnosis
1 Introduction With the rapid development of science and technology, the electronic equipments are becoming more and more complex, leading to more difficult fault diagnosis. Currently, a focus of concern is on soft fault diagnosis of analog circuit. Soft faults caused by the tolerance etc. may easily deteriorate the performance of circuit and system. To ensure the reliability and integrity of electronic equipment, the soft fault in analog circuit need to be diagnosed and located accurately for protection and recovery. Therefore, soft fault diagnosis has become one of the most important research hotspots. Essentially, the soft fault diagnosis for analog circuit is a classification problem. That is to say, using a certain classification algorithm, the data measured from analog circuit are divided into two categories: normal data and fault data. Previous research has shown that, when a soft fault occurs in analog circuit, the deviations of parameter values exert different effects on the circuit and brings about a large number of circuit states, which increases calculation complexity for conventional methods [1,2]. In contrast, Neural Network (NN) completely converts the original modes of fault diagnosis for analog circuit [3,4]. In the past decades, NN has been a hot topic in fault diagnosis research for analog circuit, and is competent for fault diagnosis tasks even for nonlinear analog circuit without explicit model. Spina et al proposed a fault diagnosis method for linear analog circuit based on white noise generator and Back Propagation (BP) NN [5]. A white noise generator is firstly
taken as the excitation source, and then based on the excitation source BP-NN is used to conduct the response analysis of analog circuit. However, the proposed method does not take into account the more complicated nonlinear cases. EI-Gamal and Wang et al introduced a neuro-fuzzy fault diagnosis method for analog circuit [6][7]. This hybrid method combines the remarkable pattern recognition capabilities of NN with the ability of fuzzy logic to incorporate and interpret linguistic knowledge, so as to classify and isolate the faults in analog circuit. However, this method has two main shortcomings: (1) the selection of fuzzy membership function is lack of specific guiding principles; (2) the extraction of fuzzy rule is not easy to be implemented. Aminian et al developed an analog circuit fault diagnosis system based on Bayesian NN by using wavelet transform, normalization and principal component analysis as preprocessors [8]. Through these preprocessing techniques, the optimal fault features obtained from the output of an analog circuit are used to train and test a Bayesian NN for the identification of fault components. However, due to the assumption of mutually independent attributes in Bayesian optimization algorithm, when the number of attributes is large and the correlation between attributes is complex in analog circuit fault diagnosis, the diagnosis efficiency of Bayesian NN is very low. In addition, Chai et al presented a RBF NN fault diagnosis method for analog circuit, which is combined with wavelet transform and incremental projection algorithm [9]. Wavelet packet is introduced to extract the fault features to retrench the input of RBF NN, and then the incremental projection algorithm is used to optimize the parameters of RBF NN, improving the capability of fault diagnosis for analog circuit. However, it is usually very difficult for RBF NN diagnosis model, which is constructed by arbitrarily selecting the center from the data points, to get the satisfactory accuracy. The common ground of these studies is that, considering the difficulties to model the complex relationship between input and output of analog circuit, using fault features extracted by wavelet transform, the advantages of NN (such as simplicity and adaptive learning) are applied to build the diagnosis model to overcome the drawbacks of conventional diagnosis method. However, the diagnosis accuracy and efficiency gotten by these NNs cannot yet meet the diagnostic requirements on the practical applications, due to intrinsic reasons of NN optimization algorithm, limitation of fault features extraction, and difficulty brought by component tolerance in analog circuit. In spite of many improvements for it, the problem cannot be solved fundamentally. Wavelet Neural Network (WNN) is a novel NN derived by wavelet analysis theory [10][11]. It can overcome many disadvantages of conventional NN and is an ideal substitute for conventional NN, which provides the necessary condition for fault diagnosis in analog circuit. But there are still several problems to be solved in order to achieve good diagnosis effect such as, extraction of fault features, selection of WNN training algorithm, etc.
According to above analysis, for analog circuit, the multi-resolution wavelet transform is introduced to extract the soft fault features from analog circuit, which is different from conventional wavelet transform. For the extracted fault features, an adaptive UKF algorithm is proposed for WNN training [12], which can improve the soft fault diagnosis capability of WNN compared with conventional BP algorithm and can overcome the shortcoming of conventional NN. The rest of the paper is organized as follows. Section 1 gives a short introduction to WNN method. UKF algorithm and its adaptive improvement are described in Section 2. The realization steps of WNN trained by an adaptive UKF algorithm are given in Section 3. Section 4 elaborates the multi-resolution transform method of sample construction. The experiments and result analysis are presented in Section 5. The last section gives some concluding remarks.
2 Wavelet Neural Network WNN is a particular neural network derived on the basis of wavelet theory. It combines the time-frequency localization of wavelet transform and self-learning function of neural network, and uses the wavelet basis function as the activation function to replace Sigmoid function in conventional neural network [10]. Compared with conventional NN, WNN has better accuracy, faster convergence and higher fault tolerance for complex nonlinear, uncertain, unknown systems [11,13]. For the discrete wavelet transform, through the wavelet function
m
m,n (t ) 22 ( m2t n ，)m, n Z
(1)
a set of orthogonal basis of the space, namely, L2 ( R) Wm , can be generated. Using the m
multi-resolution analysis in L2 ( R) , a series of closed subspaces can be obtained as
V1 V2
V2 V1 V0
, where Vm span{2 m/2 (2 m t k )} , Vm1 Vm Wm , and (t ) is the scale function
of corresponding wavelet [14]. Then the function f (t ) in L2 ( R) can be decomposed into
f (t ) f , m, n m,n (t )
(2)
m,n
Furthermore, Eq.(2) can be expressed as N
N
m,n
m,n
m
fˆ wm,n m n, (t ) wm n ,2 2 (2 m t n)
(3)
I
where t j w ji xi , w ji is the connection weights from input layer to hidden layer, wkj is the i 1
connection weights from the hidden layer to the output layer, the activation function in hidden node
is the orthogonal wavelet function m,n (t ) . Eq. (3) can be described and realized by the topology structure of WNN as shown in Fig. 1.
Actually, the training process of WNN refers to the optimization process of parameter vector
k ( w ji , wkj , m j , n j ) of WNN. In general, using the different optimization algorithms, different training algorithms of WNN can be obtained. In state space model of WNN, the parameter vector k can be considered as the state variables, the output of network can be considered as the observation variables. Then the state space model of WNN can be expressed as
k k 1 k
(4)
yk h(k , uk ) k
(5)
where uk denotes the input of network. yk denotes the output of network. h(k , uk ) is the nonlinear function parameterized. k is the process noise that is the white Gaussian noise with mean 0 and variance Qk . k is the measure noise that is white Gaussian noise with mean 0 and variance Rk .
3 Adaptive UKF Algorithm The basic idea of Adaptive UKF (AUKF) algorithm is: an adaptive factor is introduced to adjust the contribution value of state equation on the filter estimation in real-time, so as to make the covariance matrix of state parameter predictions more reasonable for accuracy improvement of UKF [15][16]. Suppose that the state equation and observation equation are, respectively,
xk 1 f ( xk ) k
(6)
yk h( xk , uk ) k
(7)
where k is the process noise with the covariance matrix Qk , k is the measure noise with the covariance matrix Rk , k and k are the white Gaussian noise with mean 0, and k is irrelevant to k . Previous research shows that the adaptive factor has relation to the discrimination statistics. Accordingly, the residual error of the prediction can be taken as the discrimination statistics of the adaptive factor
k h( xk , uk ) yk
(8)
k k k 1
(9)
The construction function of the adaptive factor has many forms, such as three-piecewise function, two-piecewise function, exponential function. Here, two-piecewise function can be selected, namely,
k 0 k 0 0 exp( k ) k 0
(10)
where the initial value of the adaptive factor satisfies 0 0 1 . Then, the realization steps of AUKF algorithm are as follows. (1) Initialization xˆ0 E x0
(11)
P0 E ( x0 xˆ0 )( x0 xˆ0 )T
(12)
(2) Calculation of Sigma point and time update
k 1 [ xˆk 1 xˆk 1 ( (n ) Pk 1 )i xˆk 1 ( (n ) Pk 1 )i ] ，i 1, 2, , n
(13)
k |k 1 f ( k 1 )
(14)
2n
xˆk Wi i ,k |k 1
(15)
i 0
n 1
Pk Wi i ,k |k 1 xˆk i ,k |k 1 xˆk Qk
(16)
yk |k 1 h( k |k 1 , uk )
(17)
T
i 0
2n
yˆ k Wi yi ,k |k 1
(18)
i 0
(3) Measure update 2n
Pyˆ yˆ Wi yi ,k |k 1 yˆ k yi ,k |k 1 yˆ k k
k
T
2n
Pxˆ yˆ Wi i ,k |k 1 xˆk yi ,k |k 1 yˆ k k
k
(19)
i 0
T
(20)
i 0
Pyˆ yˆ ( Pyˆ yˆ Rk ) / k Rk k
k
k
k
K k Pxˆ yˆ Pyˆ1yˆ / k
(21)
xˆk xˆk Kk ( yk yˆk )
(22)
k
k
k
k
Pk Pk / k K k Pyˆ yˆ K kT k
(23)
k
where the scale parameter 2 (n ) n . is used to control the distance from Sigma point to the center point and 104 1 . The other scale parameter 3 n , where n is the dimension
number of random variables. AUKF algorithm is used to estimate the optimal parameter values of WNN. As the variance of parameter values tends to zero (at this point Kalman gain tends to zero), the algorithm converges absolutely.
4 WNN Training Based on Adaptive UKF Algorithm The flow chart of WNN training based on AUKF algorithm is shown in Fig. 2. Firstly, the parameter value and its variance are initialized. Then, the time update is conducted and Sigma points are generated through UT transform to enter the forward propagation of WNN. Next, the adaptive factor is introduced to calculate the variance, covariance and Kalman gain matrix. Finally, the measure update for parameter values and error variance is carried out. The process moves in circles until the convergence conditions are met [17].
5 Fault Feature Extraction Based on Multi-resolution Transform 5.1 Principle of Multi-resolution Transform
The concept of multi-resolution transform is proposed by Mallat [18,19], in which the orthogonal binary wavelet transform algorithm with the discrete form is also given. That is, any function
f (t ) L2 ( R) can be reconstructed in light of low frequency part (approximation par2t) of f under
2 N and high frequency part (detail part) of f under 2 j (1 j N ) . The relationship of the decomposition is f (t ) An Dn Dn1
D2 D1 , where
f (t )
represents the signal, A the low frequency part, D the high frequency part, n the number of decomposition layer. It can be found from Fig. 3 that, the multi-resolution transform only decomposes low-frequency space in depth without considering high frequency parts. The purpose is to construct an orthogonal wavelet basis that highly approximates L2 ( R) in frequency. These orthogonal wavelet bases with different resolution are equivalent to bandpass filters with different bandwidths. The multi-resolution transform in L2 ( R) refers to constructing the subspace sequences {V j } jZ in the space with the following properties. (1) Monotonicity:
V2 V1 V0 V1 V2
(2) Approximation: close{
V f } L2 ( R) ，
j
V f {0}
j
(3) Retractility: (t ) V j (2t ) V j 1 (4) Translational invariance: (t ) V j (t 2 j 1 k ) V j ， k Z (5) Existence of Riesz basis: the existence of (t ) V0 , make { (2 j t k )}kZ constituted Riesz basis of V j . Suppose that {V j } jZ is a multi-resolution transform in L2 ( R) . Then, there is a unique function
(t ) L2 ( R) , which makes j ,k 2 j /2 (2 j t k ) )，k Z
(24)
the standard orthogonal basis of V j , where (t ) is the scale function. The multi-resolution transform conducts the decomposition through a special space, and cleverly constructs the wavelet basis. It decomposes L2 ( R) into nested closed-subspace sequences {V j } jZ according to the resolution 2 j and into orthogonal wavelet subspace sequences {W j } jZ according to the pyramid decomposition as shown in Fig. 4. In light of multi-resolution transform principle, the closure of W j can cover all of space
L2 ( R) W j and meets V j W j V j 1 . Suppose f (t ) V0 , then V0 can be decomposed into j
N
V0 V1 W1 V2 W2 W1 VN W j j 1
(25)
where W j is the orthogonal complement of V j in V j 1 . Then, f (t ) can be decomposed into N
f (t ) f N t( ) g j t ( ) j 1
N
f , N ,k (t ) N ,k (t ) f , j ,k (t ) j ,k (t ) kZ
(26)
j 1 kZ
where N ,k (t ) 2 N /2 (2 N t k ) is the scale function and is the standard orthogonal basis of VN .
j ,k (t ) 2 j /2 (2 j t k ) is the wavelet function that is the standard orthogonal basis of W j and L2 ( R) . In Eq.(26),
f N (t ) is the approximation of f (t ) under the scale N , and describes the
components where the resolution of f (t ) does not exceed 2 N . g j (t ) describes the component
where the resolution of f (t ) is between 2 j and 2 j 1 . So the signal can be decomposed onto any time-frequency resolution by multi-resolution transform.
5.2 Multi-resolution Transform Method There are abundant frequency components in output signal of analog circuit. When a fault occurs, these frequency components may be affected to different extents. Some components are weakened, while the others are strengthened. So the variance of some frequency components can be extracted in output signal to characterize a fault, as the input variables of WNN [20]. When the trouble occurs in analog circuit, the variance of output signal is not obvious. For this case, appropriate decomposition scale need to be chosen for multi-resolution transform of output signal according to the wavelet basis function types and fault signal length. By approximation and detail coefficients of multi-resolution transform, the mutation characteristics of different components of output signal can be completely described. These characteristics can be reflected in decomposition coefficient sequence of each layer. Therefore, the decomposition coefficient sequence can be treated as fault features of each mode. The square sum of coefficient sequence in each decomposition layer of output signal is normalized and sorted in light of the scale order, so as to construct the feature vector. Through the above analysis, the steps of fault feature extraction based on multi-resolution transform are as follows. (1) n -size multi-resolution decomposition is performed for the fault sampling signal f (t ) of analog circuit. (2) The high-frequency coefficient sequence D1 , D2 ,
, Dj ,
, Dn in each layer and the
low-frequency sequence An in n -th layer are extracted. (3) Ei is the square sum of each components in high-frequency sequence Di , and is called the i -th layer high frequency energy, i 1,
,
j,
, n . In addition, E0 is the square sum of
low-frequency sequence An , and is called the n -th layer low-frequency energy. (4) The normalized vector {E0 , E1 ,
, Ej ,
, En } is integrated with expected output vector,
namely, to construct the fault features of analog circuit.
6 Experiment Simulation The experiment environments are: Pentium IV 2.4 GHz CPU, 2GB DDR RAM, 80GB+7200 RPM hard drive, Windows XP operating system. In the experiment, the fault samples of analog circuit can be collected using Capture CIS module in software ORCAD 10.5. All simulations for the algorithms involved were implemented in MATLAB programming environment. In addition, no heuristic algorithm was used in the experiments.
6.1 Sallen-Key Bandpass Filter Circuit and Its Fault Set In the experiment, Sallen-Key bandpass filter is selected as the research subject of fault diagnosis for analog circuit. The nominal value of each component in the circuit is shown in Fig. 5. The normal tolerance range of resistance is set as 5%. The normal tolerance range of capacitance is set as 10%. The output “Out” is only test node. To simplify the problem, here we only consider the case with a single soft fault of resistance and capacitance, without considering multiple-fault and hard fault [21]. The software ORCAD is used to simulate the circuit. Through AC small-signal sensitivity analysis, the sensitivity of the influence of each component on output waveform is compared. It can be known that the changes of R2, R3, C1, C2 exert the strongest impact on the output response waveform of the circuit. Therefore, it can be determined that the fault set contains 8 fault modes, namely, R2-, R2+, R3-, R3+, C1-, C1+, C2-, C2+, where the symbol ‘+’ and ‘-’ represents too large soft fault and too small soft fault, respectively. All nominal value of deviation error are set as ±50%. In this way, 8 fault modes together with normal mode add up to 9 modes. In addition, “0-l” method can be used to obtain the expected output vector of 8 output nodes in WNN, as shown in Table 1.
The diagnosis idea based on WNN is as follows. Firstly, the fault model of circuit component is set and Monte Carlo analysis is conducted to determine the fault set. Then, through multi-resolution transform the fault features are extracted from the signal under each mode in fault set, to construct the training sample set for WNN training. Finally, the signals to be diagnosed are processed uniformly to get the test sample set for validation of WNN model. According to the output result, the fault states of analog circuit can be diagnosed.
6.2 Extraction of Fault Feature Samples for Analog Circuit In Fig. 5, the input power source Vin is set as sine wave input excitation in the circuit, where the voltage amplitude is 4V, the voltage frequency is 1KHz, the start time of scan is 0.25ms, the end time
of scan is 1.5ms, the step size of scan is 0.5μs. According to the feature of each mode, the fault model is set for the fault component. Each component without the fault is in the range of normal tolerance. The characteristic analysis of the circuit is set as the transient analysis. Monte Carlo analysis is conducted through software ORCAD. In this way, the output waveform under each fault mode can be obtained as shown in Fig. 6 (a) ~ (h), where the dotted line denotes the voltage frequency response waveform of the output of normal signal. The simulation can generate 50 transient response waveforms under each mode, and the data obtained by waveform sampling is stored in output text file of software ORCAD.
The fault feature sampling data in the period from 0.25ms to 1.5ms of each waveform from *.Out files is read to get 450 signals. The db2 wavelet is selected as the wavelet basis function to conduct 5 layers multi-resolution decomposition for the collected data, obtaining 6 decomposition coefficient sequences (D1, D2, D3, D4, D5, D6, A). Afterwards, the energy of high and low frequency coefficient sequence is calculated and normalized, constituting 6 dimension vector as the corresponding input vector of each signal. The obtained input vector can be combined through the corresponding expected output vector to get 450 fault feature samples, of which 180 fault feature samples are randomly selected for WNN training and the other 270 samples are for testing WNN model.
6.3 Experiment Analysis To validate the effectiveness of the algorithm, AUKF-WNN is compared with BP-NN [5], FL-NN [6], BY-NN [8], RBF-NN [9], BP-WNN [10] and UKF-WNN [17] (WNN is not derived from multi-resolution wavelet analysis. This is a main difference of WNN from AUKF-WNN) in fault diagnosis test for the selected circuit. The number of hidden nodes of WNN can be determined as 20 by empirical formula J I K d ( d is a integer in [1,10] ) [21]. The initial values of the parameters w ji , wkj , m j , n j can be taken as the random number in [0,1] . Maximum number of training can be set as 500. The wavelet basis function can be selected as Morlet wavelet function
(t ) cos(1.75t )et /2 . In the experiment, if the absolute value of difference between expected output 2
and actual output of an output node in WNN is greater than the determining value 0.3, it can be considered that the diagnosis result is incorrect. For UKF algorithm, the value of covariance matrix
Rk of measure noise has little effect on the result, the diagonal components of Rk can be taken as 0.0001. The value of covariance matrix Pxˆk of state parameter and covariance matrix Qk of
process noise have influences on the calculation speed and accuracy of WNN, and the diagonal components of Pxˆk and Qk can be taken as 0.1. The initial value of the adaptive factor 0 =0.95. The modeling capabilities of BP-NN, FL-NN, BY-NN, RBF-NN, BP-WNN, UKF-WNN and AUKF-WNN are compared in Table 2 for the fault features extracted by wavelet transform (wavelet packet) and multi-resolution transform, respectively. The correct diagnosis results of 7 built models (15 hidden nodes) for each mode are shown in Table 3, where C represents the number of correct diagnosis and R represents the correct diagnosis rate. The training convergence curves of the 3 algorithms are given in Fig. 7, where MSE denotes the root mean square error of the network training. Fig. 8 shows the impact of hidden nodes number J on test performance of the 3 fault diagnosis models.
It can be seen from the experiment results that, for soft fault diagnosis of analog circuits, compared with the 4 NNs (BP-NN, FL-NN, BY-NN and RBF-NN), the 3 WNNs (BP-WNN, UKF-WNN and AUKF-WNN) have a more obvious diagnostic advantage. This is also consistent with the theory analysis that WNN is superior to NN in accuracy and convergence. In 3 WNNs, UKF-WNN and AUKF-WNN are better than BP-WNN in Training Misclassification Number (TrMN), Test Misclassification Number (TeMN) and Convergence Times (CT). This shows that UKF and AUKF algorithms have superiorities over conventional BP algorithms in training WNN. Compared with UKF-WNN, AUKF-WNN shows better training, test and convergence performance, and can get satisfactory accuracy only by less than 200 iterations. It can also be found that, the diagnosis correct rate of other modes is 100% except R3, R3+, C1+ and C2+, leading to a total correct rate up to 97.4%. As the number of hidden nodes is selected in the range from 10 to 18, TeMN can be kept below 10. It is shown that, WNN based on AUKF algorithm has many advantages such as model realization, performance improvement, nonlinear characteristic description, which can establish the predominance of AUKF-WNN in soft fault diagnosis for analog circuit. On the other hand, through the comparison in Table 2, compared with the use of fault features extracted by wavelet transform, the use of the fault features extracted by multi-resolution transform can achieve better effect of soft fault diagnosis for analog circuits, which is very important for the wider application of AUKF-WNN.
7 Conclusion In order to raise the correct diagnosis rate of soft fault for analog circuit, based on the fault features extracted by the multi-resolution transform, a WNN soft fault diagnosis method is proposed using the adaptive UKF algorithm to train. The simulation results on Sallen-Key bandpass filter indicates that, WNN model obtained by AUKF training algorithm has fast convergence rate and high diagnosis accuracy, greatly improving the diagnosis performance of soft fault, which provides a new approach for fault diagnosis for analog circuit.
References [1] Huang J, He Y G. The state-of-the-art of fault diagnosis of analog circuits and prospect. Microelectronics, 2004, 34(1): 21-25. [2] Xie T, He Y. Fault diagnosis of analog circuit based on high-order cumulants and information fusion. Journal of Electronic Testing, 2014, 30(5): 505-514. [3] Tan Y H, He Y G. Wavelet method for fault diagnosis of analogue circuits. Transactions of China Electro-technical Society, 2005, 20(8): 89-93. [4] Zhou J J, Cheng H H, An M, et a1. Research on analog circuit fault diagnosis based on neural network method. Modern Electronics Technique, 2015, 23: 47-50. [5] Spina R, Upadhyaya S J. Linear circuit fault diagnosis using neuromorphic analyzers. IEEE Transactions on Circuits System-II: Analog Digital Signal Processing, 1997, 44(3): 188-196. [6] Mohamed EI-Gamal, Samah EI-Tantawy. Fuzzy inference system and neuro-fuzzy systems for analog fault diagnosis. Electrical Engineering Research, 2013, 1(4): 116 -125. [7] Wang W Z, Jin D M. Neuro-fuzzy system with high-speed low-power analog blocks. Fuzzy Sets & Systems, 2006, 157(22): 2974-2982. [8] Aminian F, Aminian M. Fault diagnosis of analog circuits using Bayesian neural networks with wavelet transform as preprocessor. Journal of Electronic Testing, 2001, 17(1): 29-36. [9] Yu H Y, Xiao M P, Zhao X. Improved RBF network application in analog circuit fault isolation. Journal of Measurement Science and Instrumentation, 2012, 3(1): 70-74. [10] Zhang Q H, Benveniste A. Wavelet network. IEEE Transactions on Neural Networks, 1992, 3(6): 889-898. [11] Preseren P P, Stopar B. Wavelet neural network employment for continuous GNSS orbit function construction: application for the assisted-GNSS principle. Applied Soft Computing,
2013, 13(5): 2526-2536. [12] Jin Y, Chen G J, Liu H. Fault diagnosis of analog circuit based on wavelet neural network. Chinese Journal of Scientific Instrument, 2007, 28(9): 1600-1604. [13] Hsu C F. A self-evolving functional-linked wavelet neural network for control applications. Applied Soft Computing, 2013, 13(11): 4392-4402. [14] Mallat S. A theory for multriresolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1989, 11(7): 674-693. [15] Yang Y X, Xu T H. An adaptive kalman filter combining variance component estimation with covariance matrix estimation based on moving window. Geomatics and Information Science of Wuhan University, 2003, 28(6): 714-183. [16] Nie J L, Qin Y, Liu H. BP neural network based on adaptive UKF algorithm and its application in height fitting. Science of Surveying and Mapping, 2007, 32(6): 120-122. [17] Xue B W, Zhang Z F, He J Q, Gan X S, Wavelet neural network algorithm based on adaptive unscented kalman filter and Its Applications, Fire Control & Command Control, 2010, 35(12): 159-162. [18] Mallat S G. Multifrequency channel decompositions of images and wavelet models. IEEE Trans. on Acoustics, Speed and Signal Processing, 1989, 37: 12-14. [19] Daubechies I. Ten lectures on wavelets. SIAM Philadelphia PA, 1992, 16: 78-81. [20] Mallat S G. A theory for multiresolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1989, 11(7): 674-693. [21] Gao M H. The research of analog circuit soft fault diagnosis based on wavelet analysis and neural network. Master Degree Dissertation, 2009.
x1 x2
xI
1
w ji
(m j , n j )
1
wkj
y1
2
2
y2
I
J
yK
Fig. 1 Topology structure of WNN
k ⑥ Kalman gain matrix
k
Kk
Rk ⑦ measure update of parameter value
yk
k
Pk
① time update of parameter value
k
k
① time update of error variance
Pk 1
xˆk 1
k
④ calculation of covariance
k 1
② calculation of Sigma point
k 1 y k |k 1
k |k 1
yˆ k xˆk
③ WNN forward propagation
uk
y k | k 1 yˆ k
k |k 1 xˆk
Fig. 2 Flow of WNN training based on AUKF algorithm
④ calculation of variance
Qk
k
⑤ variance transform
⑦ measure update of error variance
xˆk
Pxˆ yˆ
Pyˆ yˆ
Pyˆ yˆ k
Pk
k
f (t)
A1
A2
A3
D1
D2
D3
Fig. 3 Tree structure of three-layer multi-resolution analysis
V0
W1
W2
V1
V2
Fig. 4 Multi-resolution analysis diagram
Wj
WN
Vj
VN
R2 3k C1
R1
V+
+
+ 1k
5n
C2 V in
+
Out
2k
15Vdc
V2 -15Vdc
R3
5n
V1
R4
V-
+
4k R5 0
Fig. 5 Sallen-Key bandpass filter
4k
0
(a) Output waveform of R2- mode
(b) Output waveform of R2+ mode
(c) Output waveform of R3- mode
(d) Output waveform of R3+ mode
(e) Output waveform of C1- mode
(f) Output waveform of C1+ mode
(g) Output waveform of C2- mode
(h) Output waveform of C2+ mode
Fig. 6 Output waveform of each fault mode
0.12
50
BP-W NN
0.1
40
MSE
测试错分数 TeMN
UKF-W NN 0.08
BP-W NN
0.06
UKF-W NN
0.04
30 20
AUKF-W NN
10
0.02
AUKF-W NN 0
0 0
100
200
300
400
500
Training times
Fig. 7 Training convergence curves
0
5
10
15
20
J
Fig. 8 Impact of hidden node number on model test performance
Table 1 Expected output feature of WNN-based fault diagnosis system Fault state
Expected output feature
R2-
R2+
R3-
R3+
C1-
C1+
C2-
C2+
正常
Y1
0
1
1
1
1
1
1
1
1
Y2
1
0
1
1
1
1
1
1
1
Y3
1
1
0
1
1
1
1
1
1
Y4
1
1
1
0
1
1
1
1
1
Y5
1
1
1
1
0
1
1
1
1
Y6
1
1
1
1
1
0
1
1
1
Y7
1
1
1
1
1
1
0
1
1
Y8
1
1
1
1
1
1
1
0
1
Table 2 Comparison of modeling capability based on 3 algorithms Hidden nodes number
10
15
20
Diagnosis model
Multi-resolution transform feature extraction
Wavelet transform feature extraction
TrMN
TeMN
CT
TrMN
TeMN
CT
BP-NN
37
52
500
43
66
500
FL-NN
41
45
500
34
44
500
BY-NN
32
36
500
35
41
500
RBF-NN
19
34
479
23
37
455
BP-WNN
21
28
478
28
39
443
UKF-WNN
15
19
212
25
27
175
AUKF-WNN
8
10
106
9
14
159
BP-NN
35
42
500
34
47
500
FL-NN
36
39
500
35
46
500
BY-NN
28
34
500
30
36
500
RBF-NN
20
37
435
25
38
467
BP-WNN
22
25
334
27
32
407
UKF-WNN
10
16
152
15
25
246
AUKF-WNN
6
7
89
8
12
184
BP-NN
45
64
500
38
55
500
FL-NN
35
43
500
40
47
500
BY-NN
30
42
500
33
50
500
RBF-NN
23
35
456
32
39
489
BP-WNN
25
31
428
28
36
461
UKF-WNN
20
24
250
18
29
280
AUKF-WNN
15
18
126
17
21
132
Table 3 Comparison of correct diagnosis result of the model Fault mode
R2-
R2+
R3-
R3+
C1-
C1+
C2-
C2+
Normal
Total
Sample number
32
30
28
27
29
29
32
32
31
270
C
27
24
23
23
25
24
27
29
27
229
R
84.4%
80.0%
82.1%
85.2%
86.2%
82.8%
84.4%
90.6%
87.1%
84.8%
C
29
26
25
24
26
25
28
30
29
242
R
90.6%
86.7%
89.3%
88.9%
89.7%
86.2%
87.5%
93.8%
93.6%
89.6%
C
32
30
27
25
29
26
32
31
31
263
R
100%
100%
96.4%
92.6%
100%
89.7%
100%
96.9%
100%
97.4%
BP-WNN
UKF-WNN
AUKF-WNN
S1568-4946(16)30581-6 http://dx.doi.org/doi:10.1016/j.asoc.2016.11.012 ASOC 3902
To appear in:
Applied Soft Computing
Received date: Revised date: Accepted date:
11-8-2015 17-7-2016 7-11-2016
Please cite this article as: Gan Xu-sheng, Gao Wen-ming, Dai Zhe, Liu Wei-dong, Research on WNN Soft Fault Diagnosis for Analog Circuit Based on Adaptive UKF Algorithm, Applied Soft Computing Journal http://dx.doi.org/10.1016/j.asoc.2016.11.012 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Research on WNN Soft Fault Diagnosis for Analog Circuit Based on Adaptive UKF Algorithm The correct order :
Author Names 1. Gan Xu-sheng 2. Gao Wen-ming 3. Dai Zhe 4. Liu Wei-dong
Affiliations XiJing College, Shaanxi, Xi’an 710123, China Air Traffic Control and Navigation College, Air Force Engineering University Air Traffic Control and Navigation College, Air Force Engineering University Air Traffic Control and Navigation College, Air Force Engineering University
Address of correspondence: Air Traffic Control and Navigation College, Air Force Engineering University, Shaanxi, Xi’an 710038, China The correspondence author： Gan Xu-sheng E-mail: [email protected]
Tele: 15934896556
Graphical abstract
Analog circuit Adaptive factor
R2 3k C1
R1
V+
+ V1
+
UKF algorithm
1k
Wavelet neural network
5n
C2 +
V in
Out
5n
2k
15Vdc
V 2 -15Vdc
R3 R4
V-
+
4k
Adaptive UKF
R5 0
WNN training based on adaptive UKF algorithm
Sample construction using multi-resolution transform
Soft fault samples
AUKF-WNN
Train
4k
Soft fault diagnosis model based on AUKF-WNN
Fault mode
R2-
R2+
R3-
R3+
C1-
C1+
C2-
C2+
Normal
Total
Sample number
32
30
28
27
29
29
32
32
31
270
C
27
24
23
23
25
24
27
29
27
229
R
84.4%
80.0%
82.1%
85.2%
86.2%
82.8%
84.4%
90.6%
87.1%
84.8%
BP-WNN
C
29
26
25
24
26
25
28
30
29
242
R
90.6%
86.7%
89.3%
88.9%
89.7%
86.2%
87.5%
93.8%
93.6%
89.6%
UKF-WNN
C
32
30
27
25
29
26
32
31
31
263
R
100%
100%
96.4%
92.6%
100%
89.7%
100%
96.9%
100%
97.4%
AUKF-WNN
Fault diagnosis results
0
Research Highlights 1. An improved UKF algorithm with an adaptive factor is proposed. 2. A WNN modeling method based on improved UKF algorithm, namely AUKF- WNN, is given. 3. a construction method of fault samples using Multi-resolution Transform is introduced in soft fault diagnosis for analog circuit. 4. AUKF-WNN is used to establish the diagnosis model of soft fault for analog circuit. 5. For aerodynamic modeling, AUKF-WNN model shows a good diagnosis capability of soft fault for analog circuit.
Abstract: To improve the diagnosis capability of soft fault for analog circuit, a WNN diagnosis model is proposed based on fault feature samples extracted, which is trained by a modified UKF algorithm. An adaptive factor is firstly introduced to enhance the accuracy of UKF algorithm. Then, the UKF algorithm with adaptive factor is used to optimize the parameters of WNN, establishing the soft fault diagnosis model for fault feature samples extracted by multi-resolution transform. Finally, each fault mode is diagnosed and determined by the model. The simulation experiment on Sallen-Key bandpass filter indicates that, the proposed method has a good convergence rate and diagnosis accuracy rate for all faults in analog circuit. The feasibility and effectiveness of this method is also validated. Keywords: Wavelet Neural Network; UKF Algorithm; Multi-resolution Transform; Analog Circuit; Fault Diagnosis
1 Introduction With the rapid development of science and technology, the electronic equipments are becoming more and more complex, leading to more difficult fault diagnosis. Currently, a focus of concern is on soft fault diagnosis of analog circuit. Soft faults caused by the tolerance etc. may easily deteriorate the performance of circuit and system. To ensure the reliability and integrity of electronic equipment, the soft fault in analog circuit need to be diagnosed and located accurately for protection and recovery. Therefore, soft fault diagnosis has become one of the most important research hotspots. Essentially, the soft fault diagnosis for analog circuit is a classification problem. That is to say, using a certain classification algorithm, the data measured from analog circuit are divided into two categories: normal data and fault data. Previous research has shown that, when a soft fault occurs in analog circuit, the deviations of parameter values exert different effects on the circuit and brings about a large number of circuit states, which increases calculation complexity for conventional methods [1,2]. In contrast, Neural Network (NN) completely converts the original modes of fault diagnosis for analog circuit [3,4]. In the past decades, NN has been a hot topic in fault diagnosis research for analog circuit, and is competent for fault diagnosis tasks even for nonlinear analog circuit without explicit model. Spina et al proposed a fault diagnosis method for linear analog circuit based on white noise generator and Back Propagation (BP) NN [5]. A white noise generator is firstly
taken as the excitation source, and then based on the excitation source BP-NN is used to conduct the response analysis of analog circuit. However, the proposed method does not take into account the more complicated nonlinear cases. EI-Gamal and Wang et al introduced a neuro-fuzzy fault diagnosis method for analog circuit [6][7]. This hybrid method combines the remarkable pattern recognition capabilities of NN with the ability of fuzzy logic to incorporate and interpret linguistic knowledge, so as to classify and isolate the faults in analog circuit. However, this method has two main shortcomings: (1) the selection of fuzzy membership function is lack of specific guiding principles; (2) the extraction of fuzzy rule is not easy to be implemented. Aminian et al developed an analog circuit fault diagnosis system based on Bayesian NN by using wavelet transform, normalization and principal component analysis as preprocessors [8]. Through these preprocessing techniques, the optimal fault features obtained from the output of an analog circuit are used to train and test a Bayesian NN for the identification of fault components. However, due to the assumption of mutually independent attributes in Bayesian optimization algorithm, when the number of attributes is large and the correlation between attributes is complex in analog circuit fault diagnosis, the diagnosis efficiency of Bayesian NN is very low. In addition, Chai et al presented a RBF NN fault diagnosis method for analog circuit, which is combined with wavelet transform and incremental projection algorithm [9]. Wavelet packet is introduced to extract the fault features to retrench the input of RBF NN, and then the incremental projection algorithm is used to optimize the parameters of RBF NN, improving the capability of fault diagnosis for analog circuit. However, it is usually very difficult for RBF NN diagnosis model, which is constructed by arbitrarily selecting the center from the data points, to get the satisfactory accuracy. The common ground of these studies is that, considering the difficulties to model the complex relationship between input and output of analog circuit, using fault features extracted by wavelet transform, the advantages of NN (such as simplicity and adaptive learning) are applied to build the diagnosis model to overcome the drawbacks of conventional diagnosis method. However, the diagnosis accuracy and efficiency gotten by these NNs cannot yet meet the diagnostic requirements on the practical applications, due to intrinsic reasons of NN optimization algorithm, limitation of fault features extraction, and difficulty brought by component tolerance in analog circuit. In spite of many improvements for it, the problem cannot be solved fundamentally. Wavelet Neural Network (WNN) is a novel NN derived by wavelet analysis theory [10][11]. It can overcome many disadvantages of conventional NN and is an ideal substitute for conventional NN, which provides the necessary condition for fault diagnosis in analog circuit. But there are still several problems to be solved in order to achieve good diagnosis effect such as, extraction of fault features, selection of WNN training algorithm, etc.
According to above analysis, for analog circuit, the multi-resolution wavelet transform is introduced to extract the soft fault features from analog circuit, which is different from conventional wavelet transform. For the extracted fault features, an adaptive UKF algorithm is proposed for WNN training [12], which can improve the soft fault diagnosis capability of WNN compared with conventional BP algorithm and can overcome the shortcoming of conventional NN. The rest of the paper is organized as follows. Section 1 gives a short introduction to WNN method. UKF algorithm and its adaptive improvement are described in Section 2. The realization steps of WNN trained by an adaptive UKF algorithm are given in Section 3. Section 4 elaborates the multi-resolution transform method of sample construction. The experiments and result analysis are presented in Section 5. The last section gives some concluding remarks.
2 Wavelet Neural Network WNN is a particular neural network derived on the basis of wavelet theory. It combines the time-frequency localization of wavelet transform and self-learning function of neural network, and uses the wavelet basis function as the activation function to replace Sigmoid function in conventional neural network [10]. Compared with conventional NN, WNN has better accuracy, faster convergence and higher fault tolerance for complex nonlinear, uncertain, unknown systems [11,13]. For the discrete wavelet transform, through the wavelet function
m
m,n (t ) 22 ( m2t n ，)m, n Z
(1)
a set of orthogonal basis of the space, namely, L2 ( R) Wm , can be generated. Using the m
multi-resolution analysis in L2 ( R) , a series of closed subspaces can be obtained as
V1 V2
V2 V1 V0
, where Vm span{2 m/2 (2 m t k )} , Vm1 Vm Wm , and (t ) is the scale function
of corresponding wavelet [14]. Then the function f (t ) in L2 ( R) can be decomposed into
f (t ) f , m, n m,n (t )
(2)
m,n
Furthermore, Eq.(2) can be expressed as N
N
m,n
m,n
m
fˆ wm,n m n, (t ) wm n ,2 2 (2 m t n)
(3)
I
where t j w ji xi , w ji is the connection weights from input layer to hidden layer, wkj is the i 1
connection weights from the hidden layer to the output layer, the activation function in hidden node
is the orthogonal wavelet function m,n (t ) . Eq. (3) can be described and realized by the topology structure of WNN as shown in Fig. 1.
Actually, the training process of WNN refers to the optimization process of parameter vector
k ( w ji , wkj , m j , n j ) of WNN. In general, using the different optimization algorithms, different training algorithms of WNN can be obtained. In state space model of WNN, the parameter vector k can be considered as the state variables, the output of network can be considered as the observation variables. Then the state space model of WNN can be expressed as
k k 1 k
(4)
yk h(k , uk ) k
(5)
where uk denotes the input of network. yk denotes the output of network. h(k , uk ) is the nonlinear function parameterized. k is the process noise that is the white Gaussian noise with mean 0 and variance Qk . k is the measure noise that is white Gaussian noise with mean 0 and variance Rk .
3 Adaptive UKF Algorithm The basic idea of Adaptive UKF (AUKF) algorithm is: an adaptive factor is introduced to adjust the contribution value of state equation on the filter estimation in real-time, so as to make the covariance matrix of state parameter predictions more reasonable for accuracy improvement of UKF [15][16]. Suppose that the state equation and observation equation are, respectively,
xk 1 f ( xk ) k
(6)
yk h( xk , uk ) k
(7)
where k is the process noise with the covariance matrix Qk , k is the measure noise with the covariance matrix Rk , k and k are the white Gaussian noise with mean 0, and k is irrelevant to k . Previous research shows that the adaptive factor has relation to the discrimination statistics. Accordingly, the residual error of the prediction can be taken as the discrimination statistics of the adaptive factor
k h( xk , uk ) yk
(8)
k k k 1
(9)
The construction function of the adaptive factor has many forms, such as three-piecewise function, two-piecewise function, exponential function. Here, two-piecewise function can be selected, namely,
k 0 k 0 0 exp( k ) k 0
(10)
where the initial value of the adaptive factor satisfies 0 0 1 . Then, the realization steps of AUKF algorithm are as follows. (1) Initialization xˆ0 E x0
(11)
P0 E ( x0 xˆ0 )( x0 xˆ0 )T
(12)
(2) Calculation of Sigma point and time update
k 1 [ xˆk 1 xˆk 1 ( (n ) Pk 1 )i xˆk 1 ( (n ) Pk 1 )i ] ，i 1, 2, , n
(13)
k |k 1 f ( k 1 )
(14)
2n
xˆk Wi i ,k |k 1
(15)
i 0
n 1
Pk Wi i ,k |k 1 xˆk i ,k |k 1 xˆk Qk
(16)
yk |k 1 h( k |k 1 , uk )
(17)
T
i 0
2n
yˆ k Wi yi ,k |k 1
(18)
i 0
(3) Measure update 2n
Pyˆ yˆ Wi yi ,k |k 1 yˆ k yi ,k |k 1 yˆ k k
k
T
2n
Pxˆ yˆ Wi i ,k |k 1 xˆk yi ,k |k 1 yˆ k k
k
(19)
i 0
T
(20)
i 0
Pyˆ yˆ ( Pyˆ yˆ Rk ) / k Rk k
k
k
k
K k Pxˆ yˆ Pyˆ1yˆ / k
(21)
xˆk xˆk Kk ( yk yˆk )
(22)
k
k
k
k
Pk Pk / k K k Pyˆ yˆ K kT k
(23)
k
where the scale parameter 2 (n ) n . is used to control the distance from Sigma point to the center point and 104 1 . The other scale parameter 3 n , where n is the dimension
number of random variables. AUKF algorithm is used to estimate the optimal parameter values of WNN. As the variance of parameter values tends to zero (at this point Kalman gain tends to zero), the algorithm converges absolutely.
4 WNN Training Based on Adaptive UKF Algorithm The flow chart of WNN training based on AUKF algorithm is shown in Fig. 2. Firstly, the parameter value and its variance are initialized. Then, the time update is conducted and Sigma points are generated through UT transform to enter the forward propagation of WNN. Next, the adaptive factor is introduced to calculate the variance, covariance and Kalman gain matrix. Finally, the measure update for parameter values and error variance is carried out. The process moves in circles until the convergence conditions are met [17].
5 Fault Feature Extraction Based on Multi-resolution Transform 5.1 Principle of Multi-resolution Transform
The concept of multi-resolution transform is proposed by Mallat [18,19], in which the orthogonal binary wavelet transform algorithm with the discrete form is also given. That is, any function
f (t ) L2 ( R) can be reconstructed in light of low frequency part (approximation par2t) of f under
2 N and high frequency part (detail part) of f under 2 j (1 j N ) . The relationship of the decomposition is f (t ) An Dn Dn1
D2 D1 , where
f (t )
represents the signal, A the low frequency part, D the high frequency part, n the number of decomposition layer. It can be found from Fig. 3 that, the multi-resolution transform only decomposes low-frequency space in depth without considering high frequency parts. The purpose is to construct an orthogonal wavelet basis that highly approximates L2 ( R) in frequency. These orthogonal wavelet bases with different resolution are equivalent to bandpass filters with different bandwidths. The multi-resolution transform in L2 ( R) refers to constructing the subspace sequences {V j } jZ in the space with the following properties. (1) Monotonicity:
V2 V1 V0 V1 V2
(2) Approximation: close{
V f } L2 ( R) ，
j
V f {0}
j
(3) Retractility: (t ) V j (2t ) V j 1 (4) Translational invariance: (t ) V j (t 2 j 1 k ) V j ， k Z (5) Existence of Riesz basis: the existence of (t ) V0 , make { (2 j t k )}kZ constituted Riesz basis of V j . Suppose that {V j } jZ is a multi-resolution transform in L2 ( R) . Then, there is a unique function
(t ) L2 ( R) , which makes j ,k 2 j /2 (2 j t k ) )，k Z
(24)
the standard orthogonal basis of V j , where (t ) is the scale function. The multi-resolution transform conducts the decomposition through a special space, and cleverly constructs the wavelet basis. It decomposes L2 ( R) into nested closed-subspace sequences {V j } jZ according to the resolution 2 j and into orthogonal wavelet subspace sequences {W j } jZ according to the pyramid decomposition as shown in Fig. 4. In light of multi-resolution transform principle, the closure of W j can cover all of space
L2 ( R) W j and meets V j W j V j 1 . Suppose f (t ) V0 , then V0 can be decomposed into j
N
V0 V1 W1 V2 W2 W1 VN W j j 1
(25)
where W j is the orthogonal complement of V j in V j 1 . Then, f (t ) can be decomposed into N
f (t ) f N t( ) g j t ( ) j 1
N
f , N ,k (t ) N ,k (t ) f , j ,k (t ) j ,k (t ) kZ
(26)
j 1 kZ
where N ,k (t ) 2 N /2 (2 N t k ) is the scale function and is the standard orthogonal basis of VN .
j ,k (t ) 2 j /2 (2 j t k ) is the wavelet function that is the standard orthogonal basis of W j and L2 ( R) . In Eq.(26),
f N (t ) is the approximation of f (t ) under the scale N , and describes the
components where the resolution of f (t ) does not exceed 2 N . g j (t ) describes the component
where the resolution of f (t ) is between 2 j and 2 j 1 . So the signal can be decomposed onto any time-frequency resolution by multi-resolution transform.
5.2 Multi-resolution Transform Method There are abundant frequency components in output signal of analog circuit. When a fault occurs, these frequency components may be affected to different extents. Some components are weakened, while the others are strengthened. So the variance of some frequency components can be extracted in output signal to characterize a fault, as the input variables of WNN [20]. When the trouble occurs in analog circuit, the variance of output signal is not obvious. For this case, appropriate decomposition scale need to be chosen for multi-resolution transform of output signal according to the wavelet basis function types and fault signal length. By approximation and detail coefficients of multi-resolution transform, the mutation characteristics of different components of output signal can be completely described. These characteristics can be reflected in decomposition coefficient sequence of each layer. Therefore, the decomposition coefficient sequence can be treated as fault features of each mode. The square sum of coefficient sequence in each decomposition layer of output signal is normalized and sorted in light of the scale order, so as to construct the feature vector. Through the above analysis, the steps of fault feature extraction based on multi-resolution transform are as follows. (1) n -size multi-resolution decomposition is performed for the fault sampling signal f (t ) of analog circuit. (2) The high-frequency coefficient sequence D1 , D2 ,
, Dj ,
, Dn in each layer and the
low-frequency sequence An in n -th layer are extracted. (3) Ei is the square sum of each components in high-frequency sequence Di , and is called the i -th layer high frequency energy, i 1,
,
j,
, n . In addition, E0 is the square sum of
low-frequency sequence An , and is called the n -th layer low-frequency energy. (4) The normalized vector {E0 , E1 ,
, Ej ,
, En } is integrated with expected output vector,
namely, to construct the fault features of analog circuit.
6 Experiment Simulation The experiment environments are: Pentium IV 2.4 GHz CPU, 2GB DDR RAM, 80GB+7200 RPM hard drive, Windows XP operating system. In the experiment, the fault samples of analog circuit can be collected using Capture CIS module in software ORCAD 10.5. All simulations for the algorithms involved were implemented in MATLAB programming environment. In addition, no heuristic algorithm was used in the experiments.
6.1 Sallen-Key Bandpass Filter Circuit and Its Fault Set In the experiment, Sallen-Key bandpass filter is selected as the research subject of fault diagnosis for analog circuit. The nominal value of each component in the circuit is shown in Fig. 5. The normal tolerance range of resistance is set as 5%. The normal tolerance range of capacitance is set as 10%. The output “Out” is only test node. To simplify the problem, here we only consider the case with a single soft fault of resistance and capacitance, without considering multiple-fault and hard fault [21]. The software ORCAD is used to simulate the circuit. Through AC small-signal sensitivity analysis, the sensitivity of the influence of each component on output waveform is compared. It can be known that the changes of R2, R3, C1, C2 exert the strongest impact on the output response waveform of the circuit. Therefore, it can be determined that the fault set contains 8 fault modes, namely, R2-, R2+, R3-, R3+, C1-, C1+, C2-, C2+, where the symbol ‘+’ and ‘-’ represents too large soft fault and too small soft fault, respectively. All nominal value of deviation error are set as ±50%. In this way, 8 fault modes together with normal mode add up to 9 modes. In addition, “0-l” method can be used to obtain the expected output vector of 8 output nodes in WNN, as shown in Table 1.
The diagnosis idea based on WNN is as follows. Firstly, the fault model of circuit component is set and Monte Carlo analysis is conducted to determine the fault set. Then, through multi-resolution transform the fault features are extracted from the signal under each mode in fault set, to construct the training sample set for WNN training. Finally, the signals to be diagnosed are processed uniformly to get the test sample set for validation of WNN model. According to the output result, the fault states of analog circuit can be diagnosed.
6.2 Extraction of Fault Feature Samples for Analog Circuit In Fig. 5, the input power source Vin is set as sine wave input excitation in the circuit, where the voltage amplitude is 4V, the voltage frequency is 1KHz, the start time of scan is 0.25ms, the end time
of scan is 1.5ms, the step size of scan is 0.5μs. According to the feature of each mode, the fault model is set for the fault component. Each component without the fault is in the range of normal tolerance. The characteristic analysis of the circuit is set as the transient analysis. Monte Carlo analysis is conducted through software ORCAD. In this way, the output waveform under each fault mode can be obtained as shown in Fig. 6 (a) ~ (h), where the dotted line denotes the voltage frequency response waveform of the output of normal signal. The simulation can generate 50 transient response waveforms under each mode, and the data obtained by waveform sampling is stored in output text file of software ORCAD.
The fault feature sampling data in the period from 0.25ms to 1.5ms of each waveform from *.Out files is read to get 450 signals. The db2 wavelet is selected as the wavelet basis function to conduct 5 layers multi-resolution decomposition for the collected data, obtaining 6 decomposition coefficient sequences (D1, D2, D3, D4, D5, D6, A). Afterwards, the energy of high and low frequency coefficient sequence is calculated and normalized, constituting 6 dimension vector as the corresponding input vector of each signal. The obtained input vector can be combined through the corresponding expected output vector to get 450 fault feature samples, of which 180 fault feature samples are randomly selected for WNN training and the other 270 samples are for testing WNN model.
6.3 Experiment Analysis To validate the effectiveness of the algorithm, AUKF-WNN is compared with BP-NN [5], FL-NN [6], BY-NN [8], RBF-NN [9], BP-WNN [10] and UKF-WNN [17] (WNN is not derived from multi-resolution wavelet analysis. This is a main difference of WNN from AUKF-WNN) in fault diagnosis test for the selected circuit. The number of hidden nodes of WNN can be determined as 20 by empirical formula J I K d ( d is a integer in [1,10] ) [21]. The initial values of the parameters w ji , wkj , m j , n j can be taken as the random number in [0,1] . Maximum number of training can be set as 500. The wavelet basis function can be selected as Morlet wavelet function
(t ) cos(1.75t )et /2 . In the experiment, if the absolute value of difference between expected output 2
and actual output of an output node in WNN is greater than the determining value 0.3, it can be considered that the diagnosis result is incorrect. For UKF algorithm, the value of covariance matrix
Rk of measure noise has little effect on the result, the diagonal components of Rk can be taken as 0.0001. The value of covariance matrix Pxˆk of state parameter and covariance matrix Qk of
process noise have influences on the calculation speed and accuracy of WNN, and the diagonal components of Pxˆk and Qk can be taken as 0.1. The initial value of the adaptive factor 0 =0.95. The modeling capabilities of BP-NN, FL-NN, BY-NN, RBF-NN, BP-WNN, UKF-WNN and AUKF-WNN are compared in Table 2 for the fault features extracted by wavelet transform (wavelet packet) and multi-resolution transform, respectively. The correct diagnosis results of 7 built models (15 hidden nodes) for each mode are shown in Table 3, where C represents the number of correct diagnosis and R represents the correct diagnosis rate. The training convergence curves of the 3 algorithms are given in Fig. 7, where MSE denotes the root mean square error of the network training. Fig. 8 shows the impact of hidden nodes number J on test performance of the 3 fault diagnosis models.
It can be seen from the experiment results that, for soft fault diagnosis of analog circuits, compared with the 4 NNs (BP-NN, FL-NN, BY-NN and RBF-NN), the 3 WNNs (BP-WNN, UKF-WNN and AUKF-WNN) have a more obvious diagnostic advantage. This is also consistent with the theory analysis that WNN is superior to NN in accuracy and convergence. In 3 WNNs, UKF-WNN and AUKF-WNN are better than BP-WNN in Training Misclassification Number (TrMN), Test Misclassification Number (TeMN) and Convergence Times (CT). This shows that UKF and AUKF algorithms have superiorities over conventional BP algorithms in training WNN. Compared with UKF-WNN, AUKF-WNN shows better training, test and convergence performance, and can get satisfactory accuracy only by less than 200 iterations. It can also be found that, the diagnosis correct rate of other modes is 100% except R3, R3+, C1+ and C2+, leading to a total correct rate up to 97.4%. As the number of hidden nodes is selected in the range from 10 to 18, TeMN can be kept below 10. It is shown that, WNN based on AUKF algorithm has many advantages such as model realization, performance improvement, nonlinear characteristic description, which can establish the predominance of AUKF-WNN in soft fault diagnosis for analog circuit. On the other hand, through the comparison in Table 2, compared with the use of fault features extracted by wavelet transform, the use of the fault features extracted by multi-resolution transform can achieve better effect of soft fault diagnosis for analog circuits, which is very important for the wider application of AUKF-WNN.
7 Conclusion In order to raise the correct diagnosis rate of soft fault for analog circuit, based on the fault features extracted by the multi-resolution transform, a WNN soft fault diagnosis method is proposed using the adaptive UKF algorithm to train. The simulation results on Sallen-Key bandpass filter indicates that, WNN model obtained by AUKF training algorithm has fast convergence rate and high diagnosis accuracy, greatly improving the diagnosis performance of soft fault, which provides a new approach for fault diagnosis for analog circuit.
References [1] Huang J, He Y G. The state-of-the-art of fault diagnosis of analog circuits and prospect. Microelectronics, 2004, 34(1): 21-25. [2] Xie T, He Y. Fault diagnosis of analog circuit based on high-order cumulants and information fusion. Journal of Electronic Testing, 2014, 30(5): 505-514. [3] Tan Y H, He Y G. Wavelet method for fault diagnosis of analogue circuits. Transactions of China Electro-technical Society, 2005, 20(8): 89-93. [4] Zhou J J, Cheng H H, An M, et a1. Research on analog circuit fault diagnosis based on neural network method. Modern Electronics Technique, 2015, 23: 47-50. [5] Spina R, Upadhyaya S J. Linear circuit fault diagnosis using neuromorphic analyzers. IEEE Transactions on Circuits System-II: Analog Digital Signal Processing, 1997, 44(3): 188-196. [6] Mohamed EI-Gamal, Samah EI-Tantawy. Fuzzy inference system and neuro-fuzzy systems for analog fault diagnosis. Electrical Engineering Research, 2013, 1(4): 116 -125. [7] Wang W Z, Jin D M. Neuro-fuzzy system with high-speed low-power analog blocks. Fuzzy Sets & Systems, 2006, 157(22): 2974-2982. [8] Aminian F, Aminian M. Fault diagnosis of analog circuits using Bayesian neural networks with wavelet transform as preprocessor. Journal of Electronic Testing, 2001, 17(1): 29-36. [9] Yu H Y, Xiao M P, Zhao X. Improved RBF network application in analog circuit fault isolation. Journal of Measurement Science and Instrumentation, 2012, 3(1): 70-74. [10] Zhang Q H, Benveniste A. Wavelet network. IEEE Transactions on Neural Networks, 1992, 3(6): 889-898. [11] Preseren P P, Stopar B. Wavelet neural network employment for continuous GNSS orbit function construction: application for the assisted-GNSS principle. Applied Soft Computing,
2013, 13(5): 2526-2536. [12] Jin Y, Chen G J, Liu H. Fault diagnosis of analog circuit based on wavelet neural network. Chinese Journal of Scientific Instrument, 2007, 28(9): 1600-1604. [13] Hsu C F. A self-evolving functional-linked wavelet neural network for control applications. Applied Soft Computing, 2013, 13(11): 4392-4402. [14] Mallat S. A theory for multriresolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1989, 11(7): 674-693. [15] Yang Y X, Xu T H. An adaptive kalman filter combining variance component estimation with covariance matrix estimation based on moving window. Geomatics and Information Science of Wuhan University, 2003, 28(6): 714-183. [16] Nie J L, Qin Y, Liu H. BP neural network based on adaptive UKF algorithm and its application in height fitting. Science of Surveying and Mapping, 2007, 32(6): 120-122. [17] Xue B W, Zhang Z F, He J Q, Gan X S, Wavelet neural network algorithm based on adaptive unscented kalman filter and Its Applications, Fire Control & Command Control, 2010, 35(12): 159-162. [18] Mallat S G. Multifrequency channel decompositions of images and wavelet models. IEEE Trans. on Acoustics, Speed and Signal Processing, 1989, 37: 12-14. [19] Daubechies I. Ten lectures on wavelets. SIAM Philadelphia PA, 1992, 16: 78-81. [20] Mallat S G. A theory for multiresolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1989, 11(7): 674-693. [21] Gao M H. The research of analog circuit soft fault diagnosis based on wavelet analysis and neural network. Master Degree Dissertation, 2009.
x1 x2
xI
1
w ji
(m j , n j )
1
wkj
y1
2
2
y2
I
J
yK
Fig. 1 Topology structure of WNN
k ⑥ Kalman gain matrix
k
Kk
Rk ⑦ measure update of parameter value
yk
k
Pk
① time update of parameter value
k
k
① time update of error variance
Pk 1
xˆk 1
k
④ calculation of covariance
k 1
② calculation of Sigma point
k 1 y k |k 1
k |k 1
yˆ k xˆk
③ WNN forward propagation
uk
y k | k 1 yˆ k
k |k 1 xˆk
Fig. 2 Flow of WNN training based on AUKF algorithm
④ calculation of variance
Qk
k
⑤ variance transform
⑦ measure update of error variance
xˆk
Pxˆ yˆ
Pyˆ yˆ
Pyˆ yˆ k
Pk
k
f (t)
A1
A2
A3
D1
D2
D3
Fig. 3 Tree structure of three-layer multi-resolution analysis
V0
W1
W2
V1
V2
Fig. 4 Multi-resolution analysis diagram
Wj
WN
Vj
VN
R2 3k C1
R1
V+
+
+ 1k
5n
C2 V in
+
Out
2k
15Vdc
V2 -15Vdc
R3
5n
V1
R4
V-
+
4k R5 0
Fig. 5 Sallen-Key bandpass filter
4k
0
(a) Output waveform of R2- mode
(b) Output waveform of R2+ mode
(c) Output waveform of R3- mode
(d) Output waveform of R3+ mode
(e) Output waveform of C1- mode
(f) Output waveform of C1+ mode
(g) Output waveform of C2- mode
(h) Output waveform of C2+ mode
Fig. 6 Output waveform of each fault mode
0.12
50
BP-W NN
0.1
40
MSE
测试错分数 TeMN
UKF-W NN 0.08
BP-W NN
0.06
UKF-W NN
0.04
30 20
AUKF-W NN
10
0.02
AUKF-W NN 0
0 0
100
200
300
400
500
Training times
Fig. 7 Training convergence curves
0
5
10
15
20
J
Fig. 8 Impact of hidden node number on model test performance
Table 1 Expected output feature of WNN-based fault diagnosis system Fault state
Expected output feature
R2-
R2+
R3-
R3+
C1-
C1+
C2-
C2+
正常
Y1
0
1
1
1
1
1
1
1
1
Y2
1
0
1
1
1
1
1
1
1
Y3
1
1
0
1
1
1
1
1
1
Y4
1
1
1
0
1
1
1
1
1
Y5
1
1
1
1
0
1
1
1
1
Y6
1
1
1
1
1
0
1
1
1
Y7
1
1
1
1
1
1
0
1
1
Y8
1
1
1
1
1
1
1
0
1
Table 2 Comparison of modeling capability based on 3 algorithms Hidden nodes number
10
15
20
Diagnosis model
Multi-resolution transform feature extraction
Wavelet transform feature extraction
TrMN
TeMN
CT
TrMN
TeMN
CT
BP-NN
37
52
500
43
66
500
FL-NN
41
45
500
34
44
500
BY-NN
32
36
500
35
41
500
RBF-NN
19
34
479
23
37
455
BP-WNN
21
28
478
28
39
443
UKF-WNN
15
19
212
25
27
175
AUKF-WNN
8
10
106
9
14
159
BP-NN
35
42
500
34
47
500
FL-NN
36
39
500
35
46
500
BY-NN
28
34
500
30
36
500
RBF-NN
20
37
435
25
38
467
BP-WNN
22
25
334
27
32
407
UKF-WNN
10
16
152
15
25
246
AUKF-WNN
6
7
89
8
12
184
BP-NN
45
64
500
38
55
500
FL-NN
35
43
500
40
47
500
BY-NN
30
42
500
33
50
500
RBF-NN
23
35
456
32
39
489
BP-WNN
25
31
428
28
36
461
UKF-WNN
20
24
250
18
29
280
AUKF-WNN
15
18
126
17
21
132
Table 3 Comparison of correct diagnosis result of the model Fault mode
R2-
R2+
R3-
R3+
C1-
C1+
C2-
C2+
Normal
Total
Sample number
32
30
28
27
29
29
32
32
31
270
C
27
24
23
23
25
24
27
29
27
229
R
84.4%
80.0%
82.1%
85.2%
86.2%
82.8%
84.4%
90.6%
87.1%
84.8%
C
29
26
25
24
26
25
28
30
29
242
R
90.6%
86.7%
89.3%
88.9%
89.7%
86.2%
87.5%
93.8%
93.6%
89.6%
C
32
30
27
25
29
26
32
31
31
263
R
100%
100%
96.4%
92.6%
100%
89.7%
100%
96.9%
100%
97.4%
BP-WNN
UKF-WNN
AUKF-WNN