A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants

ISA Transactions xxx (xxxx) xxx

Contents lists available at ScienceDirect

ISA Transactions journal homepage: www.elsevier.com/locate/isatrans

Research article

A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants ∗

Hang Wang a , , Min-jun Peng a , J. Wesley Hines b , Gang-yang Zheng a , Yong-kuo Liu a , Belle R. Upadhyaya b a b

Key Subject Laboratory of Nuclear Safety and Simulation Technology, Harbin Engineering University, Harbin, 150001, China Department of Nuclear Engineering, University of Tennessee at Knoxville, Knoxville, 37996, United States

highlights • • • •

The challenges of knowledge-based diagnosis are compensated by the proposed algorithm. Training data is simulated to resolve the issue of real data acquisition. Improved PSO with multiple search strategies is presented in this paper. The optimization of hyper-parameters of SVM by improved PSO is compared with others.

article

info

Article history: Received 4 August 2018 Received in revised form 12 May 2019 Accepted 17 May 2019 Available online xxxx Keywords: Process fault diagnosis Hybrid strategy Support vector machine Particle swarm optimization On-line simulation model

a b s t r a c t The safety and public health during nuclear power plant operation can be enhanced by accurately recognizing and diagnosing potential problems when a malfunction occurs. However, there are still obvious technological gaps in fault diagnosis applications, mainly because adopting a single fault diagnosis method may reduce fault diagnosis accuracy. In addition, some of the proposed solutions rely heavily on fault examples, which cannot fully cover future possible fault modes in nuclear plant operation. This paper presents the results of a research in hybrid fault diagnosis techniques that utilizes support vector machine (SVM) and improved particle swarm optimization (PSO) to perform further diagnosis on the basis of qualitative reasoning by knowledge-based preliminary diagnosis and sample data provided by an on-line simulation model. Further, SVM has relatively good classification ability with small samples compared to other machine learning methodologies. However, there are some challenges in the selection of hyper-parameters in SVM that warrants the adoption of intelligent optimization algorithms. Hence, the major contribution of this paper is to propose a hybrid fault diagnosis method with a comprehensive and reasonable design. Also, improved PSO combined with a variety of search strategies are achieved and compared with other current optimization algorithms. Simulation tests are used to verify the accuracy and interpretability of research findings presented in this paper, which would be beneficial for intelligent execution of nuclear power plant operation. © 2019 ISA. Published by Elsevier Ltd. All rights reserved.

1. Introduction 1.1. Need for improved faculty diagnosis The growing energy demand and the call for environmentally friendly and reliable energy solutions have spurred research and development efforts toward offshore oil and gas exploitation and other remote power supplies. Energy deficiency has also lead to an increased effort to develop modularized and compact nuclear

∗ Corresponding author. E-mail address: [email protected] (H. Wang).

power plants (NPP). To be economically viable, these modularized NPPs will have fewer operators and may be controlled autonomously. One of the ways to ensure the safety of the reactor is through plant-wide condition monitoring. This task has been relatively simplified with the introduction of digital instrumentation and control system to the reactor [1]. With more operating information acquired from digital I&C systems [2], data mining and analytics by machine learning algorithms could be carried out effectively. However, current plants lack the capacity to analyze very small malfunctions such as small reactor coolant pipe leakage, small variations in process variables due to degradation, among others. But, quick detection and resolution of the fault are important to the safe operation of the plant [3]. In the absence

https://doi.org/10.1016/j.isatra.2019.05.016 0019-0578/© 2019 ISA. Published by Elsevier Ltd. All rights reserved.

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

2

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

Fig. 1. The block diagram of a computerized operator support system.

of an effective fault diagnostic system, operators could only rely on their own experiences, which could be subjective [4,5]. Moreover, the response time would be long, which naturally leads to tremendous psychological pressure on operators [6]. To ensure prompt fault detection and diagnosis, researchers have studied online monitoring, computerized operation procedure, design and evaluation of human–machine interface [7]. A representative flowchart of the existing research findings is as shown in Fig. 1. However, there are unresolved issues with fault diagnosis techniques. For data-driven methods, one of these issues is in obtaining representative sample data for different faults. Hadad et al. adopted a back propagation neural network and wavelet transform to diagnose system abnormalities [8]. Wolbrecht applied Bayesian network model to diagnose the power failure of the equipment [9]. Zhang adopted particle filter model for fault detection and diagnosis of dynamic process system [10]. However, it is difficult to obtain sample data under different scenarios resulting in diagnostic results that are difficult to understand. In addition, for knowledge-based methodologies, Lind [11] illustrates the multi-layer flow model (MFM) for fault diagnosis, which could show the interpretive reasoning process. Kramer et al. proposed a method to analyze the fault propagation path by using the Symbol Directed Graph (SDG) [12]. However, it is difficult to acquire knowledge and develop complete rules with a limited scope of qualitative analysis. Therefore, a single diagnostic method cannot accurately and convincingly solve this problem [13]. To further address the problem, researchers have combined various methods to achieve more accurate fault diagnosis. Currently, the PRODIAG system developed by Argonne National Laboratories in the United States, and diagnostic system designed by OECD Halden Reactor Project have been successfully applied in the United States and some European countries [14]. In addition, Chu et al. [15] at Harbin Engineering University studied Bayesian networks and used the multi-flow model (MFM) to eliminate uncertainty in fault diagnosis methods. However, these methods are generally based on data-driven analysis, supplemented by qualitative empirical knowledge. Although some progress has been made, the diagnostic results still remain highly uncertain because they have no solution to the huge demand for operational data under malfunctions in actual NPPs. In order to solve these problems, a research project was initiated to make use of online simulation models to provide dynamic

tracking and accelerated simulation [16,17]. System-level distributed online monitoring is achieved by a real-time simulation model which is set as a dynamic platform. If there is a fault, a knowledge-based preliminary diagnosis is applied as it has a lower demand for measured data, takes less modeling efforts and diagnostic results could be interpreted easily. In addition, the simulation model will predict the trends of measurements according to the results from knowledge-based diagnosis 1.2. The importance of the proposed fault diagnosis system Knowledge-based methods are considered to be the most suitable way for fault diagnosis, but their results are generally uncertain for some sets of failures. In the event of multiple faults, the relevant characteristic parameters have the same qualitative trends. Hence, it is difficult to achieve accurate reasoning without referencing qualitative analysis, which eventually gives an inconclusive result [18]. Moreover, faults with small changes in parameters are difficult to detect with qualitative analysis [19]. In order to assist operators to clearly understand the state of NPP and obtain unique results, other methods are supplemented on the basis of knowledge-based diagnosis. A fault diagnostic system based solely on a data-driven method is limited, because it is difficult to acquire samples under malfunctions, and the results cannot be explained properly [20]. However, in combination with the knowledge-based method, the data-driven approach could be integrated with the thermalhydraulic simulation model troubleshoot and validate. In this way, several different methods are integrated into a hybrid system, with one complementing the other’s weaknesses. The hybrid solution not only improves the accuracy of fault diagnosis, but also increases the credibility and interpretability of results [21]. Further advantages of the hybrid system are: 1. Although there are similarities in the qualitative trends for different faults diagnosed using knowledge-based methods, the corresponding quantitative changes are totally different and detectable with data-driven analysis. 2. All possible failure modes are analyzed and trained by a single data-driven method which slows the computation speed and affects the accuracy. In the hybrid system, the failure mode is

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

3

reduced to a limited number, thus avoiding too many training modes [22].

relaxation factors to allow the existence of few false samples [28], so the constraint condition for SVM is:

3. By integrating the online simulation model, requisite samples will be calculated quickly for training and learning which will solve the problem of obtaining sample data. In order to provide on-line diagnostic results on time, the speed of the faster-than-real-time simulation and training time of data-driven methods need to improve. Consequently, machine learning algorithms that require a lot of samples such as neural networks and artificial immune network [23] will have serious defects, mainly because they are based on classical statistics which are established under asymptotic theories [24]. On the other hand, support vector machine (SVM) is based on structural risk minimization, with the capability to overcome the differences in the classical statistics between the empirical risk and expected risk [25]. In addition, the small data requirement of SVM for pattern classification tasks justifies its selection in this paper. A significant limitation of SVM is that there is no generally accepted method for selecting some hyper-parameters that could affect its classification accuracy and speed. An effective online diagnostic process not only requires accurate results but also has to be rapid and timely. Although scholars have studied a large number of parameter optimization methods such as genetic algorithm (GA), adaptive artificial bee colony algorithm (AABC), artificial fish algorithm (AFA) and their variants, these algorithms are generally inefficient, time-consuming, and have low accuracy, except particle swarm optimization algorithm which has been found to be effective for parameter optimization [26]. Hence, in this paper, we propose the use of PSO for parameter optimization. This paper is arranged as follows: The theories and methods are introduced in Section 2. The flow chart representation of the proposed hybrid fault diagnosis method is presented in Section 3. In Section 4, the analysis and comparison of the simulation results are presented. The methodology is summarized in Section 5 with concluding remarks.

yi (w T xi + b) ≥ 1 − ξi ,

i = 1, 2, . . . , n

(1)

where, w is the weight, b is bias coefficient, ξi is known as the relaxation variable. For preventing arbitrarily large relaxation variables, constraint conditions are introduced to minimize ξi . Hence, the classification equation in (1) is refined to: min

1 2

n ∑

∥w∥2 + C

ξi

i=1

s.t .yi (w T xi + b) = 1,

i = 1, 2˜, . . . , n

ξi ≥ 0, i = 1, 2, . . . , n (2)

Where, C is the penalty used to control the weight of maximum hyperplane and minimum deviation of the data points. Specifically, ξ need to be optimized by SVM while C should have prior value. Then, Lagrange multipliers are applied as in Eq. (3): L(w, b, ξ , λ, r) =

1 2

n ∑

∥w∥2 + C

ξi −

i=1

− 1 + ξi −

)

n ∑

n ∑

( ( ) λi yi wT xi + b

i=1

(3)

ri ξ i

i=1

Partial derivatives with respect to w , ξ and b are performed separately: n

∑ ∂L =0⇒w= λi yi xi ∂w

(4)

i=1

n

∑ ∂L =0⇒ λi y i = 0 ∂b

(5)

i=1

2. Theories of fault definite diagnosis

∂L = 0 ⇒ C − λi − ri = 0, ∂ξi

2.1. Support vector machine (SVM)

The maximum value of λ is found and w , ξ , and b are expressed in terms of λ. Thus, Eq. (2) is converted to Eq. (7) as

SVM is based on statistical learning theory and structural risk minimization principle, with the capability to improve the generalization ability and find the best balance between experience, risk and confidence range using limited data samples [27]. SVM algorithm can be used as a classifier and for regression analysis. As process fault diagnosis is a form of pattern classification task, we utilized SVM as a classifier. In practical application, signals from the NPP have strong nonlinear coupling. These non-linear characteristics exclude the use of linear classifiers for the analysis. That is, since these variables cannot be separated by linear classification, the nonlinear mapping will be utilized to transfer the original input variables (x1 , y1 ), (x2 , y2 ), . . . , (xn , yn ), x ∈ Rn , y ∈ {1, −1} into high dimensional space. SVM is trained in hyperplane which enables searching for non-linear functions in the original input space. At this point, the support vectors of SVM are a linear combination of nonlinear functions. Moreover, the data vectors are always as a form of inner products in SVM which can be replaced by appropriate kernel functions so that variables will be mapped to high-dimensional space implicitly. Moreover, in real signals, there are noises, inherently unusable variables and outliers. Since the support vectors themselves are only a few, the outliers will have a huge impact on the classification result. Therefore, in order to weigh the empirical risk and the confidence risk, Cortes and Vapnik introduced non-negative

max λ

n ∑

λi −

i=1

n 1∑

2

(6)

λi λj yi yj κ (xi , xj )

i,j=1

s.t .0 ≤ λi ≤ C , i = 1, 2, . . . , n n ∑

i = 1, 2, . . . , n

(7)

λi yi = 0

i=1

However, selecting an appropriate kernel function is based on experience, as there is no systematic theory for the selection. In the literature, the results are very similar under different kernel functions, so this paper adopts the most common approach— radial basis function (RBF), to establish the SVM model. Besides, if the width of the RBF is not reasonable, the classification of SVM will not be accurate. Therefore, it is necessary to optimize the width of RBF. Also, the penalty factor C needs to be optimized, as it plays a significant role in improving the performance of SVM. 2.2. Particle swarm optimization (PSO) based on a hybrid search strategy The elementary PSO model is first proposed by Kennedy and Eberhart to describe group behaviors [29]. The basic idea is to set the initial values random in a solution space, and then search for individual optimal values and global optimal values at a certain probability, a complex nonlinear search process [30]. Although

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

4

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

local optimal and long iteration issues have been addressed, there are still a few challenges in the utilization of PSO [31]. This paper proposes an enhanced PSO using the search strategy. Particles are divided into two groups: Optimal population and non-optimal population. First, all particles are initialized with logistic chaos to ensure the randomness of the initial population. If the dimension of the variables is D and z1 = [z11 , z12 , . . . , z1D ], and the range of values for each dimension is [0, 1], then, z2 , z3 , . . . , zN can be represented as: zn+1 = µzn (1 − zn ),

n = 1, 2, . . . , N ; 0 < zn < 1; u ∈ [0, 4] (8)

So, the initial positions of each particle are: xij = psop1j + (psop2j − psop1j )zij

(9)

Where, psop1i is the lowest limit of each dimension and psop2i represents the highest limit, i is the number of particle and j describes the dimension. For the optimal population, the velocity updating formula of these particles is shown below. gd vi (t + 1) = wk vi (t) + c1k r1 (xpd i (t) − xi (t)) + c2k r2 (xk +φk − xi (t)) (10)

Where, vi (t) is the velocity in time t and k represents time algebra, and φk means the neighborhood of the particle. In addipb gb tion, xi (t) and xk (t) means individual optimal value and global optimal value respectively, r is a random number between (0, 1) while the ‘‘cognition’’ and ‘‘social’’ learning factors are respectively expressed as c1k and c2k . For the setting of inertial weights wk , the nonlinear adjustment is adopted to avoid the incompatibility between the linear decreasing weight and actual searching process:

w = a × exp(bk2 ) × rand(0, 1) 1 wmax b= 2 In( ) K −1 wmin a = wmax exp(−b)

(11) (12) (13)

Here, K is the maximum iteration, wmax and wmin represent the maximum and minimum limit of w , with a value of 0.9 and 0.4 respectively [32]. These equations show the nonlinear rate of change in the inertial weight and attenuation speed. If a linear model is used, this algorithm will still converge although the right inertia weight cannot be quickly determined. For learning factors, they are set to a fixed value in the standard PSO which does not improve the convergence rate. Thus, asynchronous learning factors are adopted, similar to Eqs. (11)– (13). Initially, particles tend to learn from the individual optimal particle, so the initial value of the cognitive learning factor c1k is greater than the stop value. In the end, the particles tend to learn more from the global optimal value, so the initial value of the social learning factor c2k is less than the end value. In this way, the nonlinear asynchronous adjustment of learning factors would avoid local optimum and speed up the optimization [33]. In order to ensure global optimal is obtained, simulated annealing algorithm (SAA) based on the Metropolis criterion is utilized and a set of control parameters are adopted to obtain an optimal value in time [34]. In this approach, PSO is used for global search and SAA is utilized for neighborhood. This search optimization technique results in reduced computation time. In most applications, neighborhood search based on SAA is executed using Eq. (10). The steps involved are as follows: (1) Assume T0 is the initial temperature in SAA, represented by each particle’s classification accuracy of SVM; (2) Set a local perturbation range as: xij = xij + Rangej ∗ (2 ∗ rand (1, D) − 1) Where, Rangej is the range of local disturbances.

(14)

(3) The difference between the initial fitness value and the fitness after a disturbance in the neighborhood is defined as: ∆ = fitness(k1 ) − fitness(initial). If ∆ ≥ 0, the global optimal value is updated after the disturbance; If ∆ < 0, the global optimal value may be updated with probability p according to the Metropolis criterion, and the calculation of probability p is as follows: p = exp(

fitness(k1 ) − fitness(initial)

) (15) kT Where, T represents the thermodynamic temperature in SAA and k is the Boltzmann constant. (4) Annealing cooling is implemented for T , then step (3) is repeated until the termination of annealing temperature or the maximum iterations, and the optimal value will be considered as global optimal value. For the non-optimal population, these particles have relatively lower fitness from the whole group, but it does not mean that these particles are all worse than particles in the optimal population. Even nonlinear adjustment of inertia weight and asynchronous learning factors have been done, but due to the random search, it is possible for optimal population particles to have local optimal. Thus, the purpose of the non-optimal population is to make up for the deficiency of the optimal population and adopt a different searching strategy to find the global optimal values. Consequently, the velocities of the particles are updated as follows:

⎧ gd gd1 if fitness(xk ) > fitness(xl ) ⎪ ⎪ ⎪ ⎪ pd1 ⎪ ⎪ vi (t + 1) = wvi (t) + c11 r1 (xi (t) − xi (t)) ⎪ ⎪ ⎪ ⎨ + c r (xgd1 + φ − x (t)) + c r (xgd − x (t)) 21 2 k k i 22 2 k i gd gd1 ⎪ if fitness(x ) < fitness(x ) ⎪ k l ⎪ ⎪ ⎪ pd1 ⎪ ⎪ (t) − xi (t)) v (t + 1) = wv (t) + c r (x i i 11 1 i ⎪ ⎪ ⎩ gd1 + c21 r2 (xk + φk − xi (t))

(16)

gd1

Where, xl is the global optimal value of the non-optimal population. W is set as a constant value 0.729, c11 and c21 is the ‘‘cognitive’’ and ‘‘social’’ learning factors all with a constant value of 2 because the search strategy for non-optimal particles are mainly for expansion and they complement the optimal population. C22 represents the ‘‘social’’ learning factor inherited from the optimal population and it is introduced to assist the nonoptimal population. Moreover, in order to avoid the species from non-optimal space, adaptive high-frequency random variation based on the genetic algorithm is applied to the non-optimal population [35]: if rand ≥ 0.5 xi (t + 1) = xi (t) + 0.5 × rand1 × xi (t) if rand < 0.5 ⎪ ⎩ xi (t + 1) = xi (t) − 0.5 × rand1 × xi (t)

⎧ ⎪ ⎨

(17)

where rand and rand1 represent the random numbers between 0 to 1 respectively. As shown in Eq. (17), particles would change by a large margin to avoid falling into a local optimum. To guarantee the effective support for optimal population, elimination mechanism is introduced. Specifically, all the particles are sorted according to the corresponding fitness after each time step. And then, some of the particles in optimal population will be replaced by the particles in a non-optimal population who have higher fitness. The corresponding number of particles replaced in a population is derived using the equation below: Num = Pso(0.7 − (0.7 − 0.5) × k/K )

(18)

Num is the number of particles updated in the optimal population. PSO describes the total number of the optimal population. As

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

5

Fig. 2. Diagram of fault diagnosis based on hybrid methods.

shown in Eq. (18), 70% of the particles are updated at the early stage while only 50% of the total particles are updated in the end. The reason is in the early stage, more global search would be done. But in the end, a relatively small number of particles would be updated to enhance local searching. As for non-optimal population, some particles would be supplemented to maintain the total number of particles. For these new particles, they are initialized by Eqs. (8) and (9). Due to the application of hybrid search strategies, there is nearly no contingency during the search which is much better than basic PSO and standard PSO. Thus, if the global fitness of the whole particles is no less than 95% in 10 iterations, then hybrid PSO will be stopped in advance and directed into the classification of SVM.

simulation model separately. The simulation model will calculate faster-than-real-time on the basis of different conditions. After obtaining scenarios under a fault, the simulation model will be reset to the initial moment and continue to repeat the same process until getting all the samples of each fault. Then, the data is normalized and stored in the real-time database. After gathering the simulated samples, the proposed fault diagnosis system is implemented as shown in Fig. 3. SVM will be trained by improved PSO algorithms. During the on-line diagnosis, actual real-time data will be acquired from the real-time database and normalized as simulated data. Then, the confirmed result will be diagnosed by SVM and displayed in the human– machine interface, suitable for providing accurate information for the operators. 4. Simulation analysis

3. Complete scheme of hybrid process fault diagnosis The diagram of the proposed hybrid fault diagnosis process is shown in Fig. 2. The input of this architecture is the realtime data of NPP which could be acquired from the I&C system. The plant state is determined by the distributed on-line monitoring module [36]. If the NPP is in steady state, then the on-line monitoring module will continue the monitoring and the fault diagnosis module will not be activated;however, if an anomaly is detected, the fault diagnosis module will be activated instantly to locate and diagnose the malfunctions [37]. As knowledge-based methods do not require a complex model, the computational space is relatively low and the result could be interpreted, thus, it is suitable for the preliminary diagnosis. The diagnostic result from knowledge-based methods is always in a set. Even if a single fault is diagnosed, there is a need for verification. In fault state, the online thermal-hydraulic simulation model will be switched from online tracking mode to off-line fasterthan real-time mode and initialized to the moment when an anomaly is detected. Moreover, fault sets are transferred to the

In order to verify the accuracy and speed of the proposed fault definite diagnosis, some sensitive parameter settings are compared by utilizing simulated data through the simulation model. Then, improved PSO performance is compared with new optimization algorithms to demonstrate its advantages. Finally, a small leakage in the cold leg pipe of the Reactor Coolant System (RCS) is simulated using the full-scope simulator and the corresponding sample data for training is acquired from the simulation model. 4.1. The development environment of algorithms This methodology is still being researched and cannot be directly evaluated on a real NPP, so it is connected to a full-scope pressurized water reactor simulator instead. Data is acquired and stored in the real-time database for further calculation by the thermal-hydraulic simulation model. The simulated samples are transferred to fault diagnosis algorithms that were developed using MATLAB. The result is displayed in the human–machine interface developed by Visual Studio C#, and the diagram of each module is described in Fig. 4.

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

6

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx Table 1 Diagnosed accuracy under different kernel function. Kernel function

Best c

Best g

Best accuracy Average accuracy Time

Polynomial kernel 76.34 879.62 92.86% RBF 56.33 621 93.90% Sigmoid 83.1711 971.818 93.52%

92.21% 92.89% 92.65%

303 s 302 s 309 s

the on-line thermal-hydraulic simulator. Measurements in RCS, chemistry and volume control system are collected for further analysis. The simulated data is divided into the training part and test part and 5 fold cross validation for SVM is adopted for verification. By default, the data is normalized to (0, 1) and the noise intensity is 30dBW. Also, the RBF kernel function is selected and the sample size is 200 for each fault. In addition, the searching range of the kernel function width g and penalty factor c is (0.01, 1000) and (0.1, 100) respectively. The population number is set to 20 and the maximum evolutionary algebra is set to 200 for PSO. In the following experiments, the algorithm runs 20 times independently to obtain reasonable results. (1) The selection of kernel function First, the kernel functions in SVM are compared and the results are shown in Table 1. Under three typical nonlinear kernel functions, every index is nearly the same. However, the RBF kernel has the best accuracy from the experiments. For computation time, RBF and polynomial kernel function are both less timeconsuming. Therefore, RBF is selected as the kernel function for SVM. (2) Data preprocessing Due to the numbers of measurements in NPP, the accuracy of fault diagnosis will be seriously affected if the data is not normalized. Thus, different data preprocessing applied is shown in Table 2. As seen in the table, the best accuracy is obtained when the data is normalized, between (−1, 1) . (3) The number of samples As all the samples are provided by the simulation model separately, the number of samples has to be controlled due to time constraints. However, if the number is too small, the diagnostic accuracy will be affected, as shown in Table 3. With the increase in data quantity, best accuracy and average accuracy are all improving, which means the algorithm will become more accurate and stable with more samples. Since computation time is an important consideration, the number of samples is set as 200.

Fig. 3. Diagram of online training based on SVM and PSO.

4.2. Sensitivity analysis of critical parameters In the implementation of the proposed improved PSO and SVM for pattern recognition, it is important to determine some sensitive parameter settings that may directly affect the final accuracy. Therefore, in this section, some critical parameters are compared while keeping other parameters constant. A small leakage in 1# the RCS cold leg, a small leakage in 1# hot leg, a small tube rupture in 1# steam generator, 1# coolant pump shaft and inadvertent drop of a single control rod are simulated using

(4) The population of PSO During parameter optimization, the number of population will influence the accuracy to some extent as shown in Table 4. When the population is 5, its accuracy significantly declines. But, with the increase in population, the best accuracy and average accuracy are basically increasing. However, the computation time also increases with the increase in population. Therefore, after ensuring a relatively high accuracy, the number of population is selected as 10. x (5) Evolution algebra of PSO The evolution algebra also influences the SVM accuracy. In theory, the more the number of evolution algebra, the better the parameter optimization. However, the algebra also leads to high calculation time as shown in Table 5. When the evolution algebra is 2, the accuracy is relatively lower than others. For higher number of evolutionary algebras, there is no obvious improvement in accuracy. Therefore, the evolution algebra is set as 10 to speed up the calculation.

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

7

Table 2 Diagnosed accuracy under different data pre-processing. Interval

Normalized mode

Best accuracy

Average accuracy

Time

(−∞, +∞) (0, 1) (−1, 1) (0, 1) (0.1, 0.9)

No normalized (x − xmin )/ (xmax − xmin ) 2(x − xmin )/ (xmax − xmin ) − 1 x/ xmax 0.1 + (x − xmin )/(xmax − xmin )*0.8

19.15% 93.90% 95.71% 77.72% 91.33%

19.04% 92.89% 95.01% 77.04% 90.22%

389 302 295 358 336

s s s s s

Fig. 4. Development environment and interdependence among modules during fault diagnosis. Table 3 Diagnosed accuracy under different number of samples.

Table 6 Diagnosed accuracy under different PSO algorithms.

The number of samples

Best accuracy

Average accuracy

Time

Optimization algorithm

Best accuracy

Average accuracy

Time

100 200 300 400

93.90% 95.71% 97.29% 97.85%

92.89% 95.12% 96.66% 97.52%

28.7 s 55.4 s 85.7 s 105.4 s

GA AABC AFA Improved PSO

94.86% 90.61% 95.02% 95.71%

94.22% 89.78% 94.72% 95.12%

47.9 s 88.2 s 158.3 s 55.4 s

Table 4 Diagnosed accuracy under different population number. Population of PSO

Best accuracy

Average accuracy

Time

5 10 20 50 100 200

94.85% 95.71% 95.42% 95.52% 95.90% 96.72%

94.64% 95.30% 95.01% 95.06% 95.39% 95.17%

225.8 s 247.6 s 295 s 448 s 690 s 1149 s

Table 5 Diagnosed accuracy under different evolutionary algebra. Evolution algebra

Best accuracy

Average accuracy

Time

2 10 50 100 300 500

94.82% 95.71% 95.78% 96.38% 95.90% 95.80%

94.21% 95.12% 95.02% 95.42% 95.19% 95.3%

46.6 s 55.4 s 92.3 s 141.6 s 330.2 s 535.2 s

4.3. Comparison of typical algorithms

Beside PSO, many other algorithms based on swarm intelligence theory could also be utilized for parameter optimization. Thus, in this paper, we evaluated GA, AABC and AFA to compare with the proposed PSO [38].

Table 6 shows the accuracy and time for different algorithms after 20 independent calculations. Among them, the optimal accuracy and average accuracy by GA are studied in Fig. 6. Compared with that of improved PSO in Fig. 5, the average accuracy in inchoate generations are relatively high and tend to be optimal, leading to a serious loss of diversity. For AABC, it has the relative lowest accuracy as shown in Fig. 7. During the implementation, the average fitness gradually increases, influenced by the bees. Then, there are some fluctuations in average fitness caused by scouts. But overall, the diversity of populations is lower than the proposed PSO. For AFA, as shown in Fig. 8, the global optimum is better than AABC and GA when a group of forage behavior is considered. Although, the fish populations have some random behavior, but the average fitness is still nearly the same with the global optimum which make it much easier to tend toward local optimal. But as shown in Fig. 5, the diversity of species in improved PSO is much better than others. Especially chaos initialization, nonlinear changing of inertia weight and learning factors are adopted at the beginning which makes the fluctuation in average accuracy significantly large. Meanwhile, as the variation factor, updating and elimination in the optimal and non-optimal population are implemented, the population still remains diversified at the end of evolution algebra. In summary, the algorithm described in this paper will be better adapted in engineering applications although there may be more fault types.

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

8

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

Fig. 5. Accuracy of improved PSO.

Fig. 6. GA for parameter optimization.

4.4. Test case The tests above are all based on the data provided by the simulation model, but the simulation model is aimed to provide samples and the actual measurements should be obtained from the full scope simulator, similar to the real process. In this section, we describe the simulation of a 2 cm2 leakage hole in # 1 RCS cold leg in the full scope simulator. This small break will lead to a slow change in parameters and the reactor will not shut down because of the compensation from the I&C system. This compensatory action is mainly done by the automatic opening of the chemical and volume control charging valve and the electric heater in the pressurizer. Hence, the slow rate of change makes it difficult for operators to quickly identify and analyze the fault. The causal reasoning of alarms based on qualitative analysis is displayed in Fig. 9, where the yellow nodes represent the state

of reasoning analysis with no support of actual data, while the red nodes describe the measurements in the actual system. The specific cause and effect path is as follows: due to the break in RCS, the shielding function of the pressure boundary is disabled (Leak) which further causes the coolant flow to be reduced (L). Then, the water volume of RCS decreases (L) which lead to water negative fluctuation (L) from pressurizer, thus, the water level of pressurizer 008MN declines (L). After that, the pressure of pressurizer and pressure after the orifice plate in chemistry and volume control system are all decreasing (L). In order to make up for these changes, the electric heater comes on (H) and the charging flow is increased (H) by the I&C system. Synthesizing the above information, leakage in the cold leg, in the hot leg or in the pressurizer could be predicted, as the measurements from these faults are all in line with the causal path

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

9

Fig. 7. AABC for parameter optimization.

Fig. 8. AFA for parameter optimization.

as shown in Fig. 9. Thus, the fault cannot be clearly diagnosed by knowledge-based methodologies alone. Therefore, the algorithms described in Section 2 will be utilized for further quantitative analysis. However, due to the similar characteristics of these failures, the different fault degree inserted in the developed simulation model may influence the accuracy, so we utilized an approximate estimated of the fault degree according to safety analysis report. The plots of some parameters from the full scope simulator are shown in Figs. 10 and 11. From the figures, it is observed that the reactor is still in operation and the electric heater in pressurizer and charging valves are adjusted to compensate for the loss of coolant after the fault. Thus, leakage flow can be estimated using the equation

∫ t2 fleak =

t1

t1

Where, fleak is leakage flow, kg/s; t1 and t2 represents start and stop time, s; f1 , f2 and f3 are the flows measured by flow meters after steam generators in normal while f4 , f5 and f6 are the flows after faults, kg/s; fc is charging flow during faults, kg/s; △Lp and Ap is the water level and area of pressurizer respectively. Moreover, v , △h is the average specific heat capacity (m3 /kg) and latent heat of vaporization (kJ/kg), while Wp is the electric power of the heater, w . Combined with the changes in Figs. 10 and 11, the average leakage flow is estimated as 21.51 kg/s, 60 s after the fault. As the leakage of coolant is similar to the discharge of single-phase liquid in non-critical flow area which could be simplified as the Bernoulli equation in formula 20. Thus, the corresponding fracture area is 2.37 cm2 . G = ACd

t2 − t1

∫ t2 =

fleak dt

(19)

(f1 + f2 + f3 − f4 − f5 − f6 + fc ) dt + t2 − t1

A∆Lp

v

−

∫ t2

Wp dt t1 ∆ h

√

2ρ (Po − Pd )

(20) 2

Where, G, A — leakage flow, kg/s; fracture area, m P0, Pb — pressure of RCS and backpressure, Pa;

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

10

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

Fig. 9. Causal path based on qualitative Knowledge. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 10. Pressurizer water level and pressure.

Fig. 11. Heater power and charging flow.

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

11

Fig. 12. The time process of NPP, on-line monitoring and fault diagnosis. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 13. Comparison of pressurizer water level.

Fig. 14. Comparison of pressurizer temperature.

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

12

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

Fig. 15. Pressure in charging line.

Fig. 16. Diagnosis based on proposed algorithms.

ρ — the density of the coolant, kg/m3 ; Cd — leakage coefficient, when L/D≈0, Cd = 0.61. After estimating the fault degree, three faults are separately inserted in the simulation model. The simulation model calculations are 4 times faster than real-time to provide the samples. The time process of the NPP, the distributed on-line monitoring and fault diagnosis system are compared in Fig. 12. The black lines represent the real-time process and the green lines are the hyper real-time process. Although the same magnitudes of these faults are simulated, there are obvious quantitative differences in each parameter, as shown in Figs. 13–15. After gathering all the samples, the critical parameters verified in Section 4.2 will be adopted to optimize the width of RBF and the penalty factor. Using the 5 fold cross-validation, the best value of the penalty factor is 1.121 and that of width is 578.18, the diagnostic accuracy is 98.6154% during the training process. The accuracy is higher than the previous section because there is no noise in the simulated samples and the characteristic information after faults are

more precise. The final results are shown in Fig. 14, where 0–3 is the respective category labels. 0 means the NPP is in normal, 1 means leakage in 1# cold leg, 2 describes leakage in 1# hot leg while 3 means leakage in the pressurizer. From the results, most of the malfunctions are classified as normal operation because the change in characteristics is not obvious at the beginning. However, after 15 s, the diagnosis results are in accordance with the actual situation. Finally, the accuracy is 94.78%, which meets the requirements of fault diagnosis. In order to verify the accuracy of proposed SVM and improved PSO algorithm, some typical supervised methods such as adaptive fuzzy neural network (ANFIS) and back propagation neural networks (BPNN) are adopted to replace the proposed method [39, 40]. For ANFIS, it was generated using grid partitioning which had 27 rules, 3 inputs, 1 output, and 3 Gaussian membership functions per input. The faults could not be classified at the first few seconds which is similar to that in Fig. 16. And for BPNN, it has 15 hidden layers with Sigmoid activation function, 0.2 learning rate and elastic BP gradient descent algorithm. From the results of

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

13

Fig. 17. Diagnosis based on ANFIS.

Fig. 18. Diagnosis based on BPNN.

ANFIS and BPNN as shown in Figs. 17 and 18, there are misclassifications when fault deteriorates in BPNN with a better result than ANFIS. It is also observed that their overall performance is inferior to the proposed method in this paper. As for SVM and improved PSO, it could optimize the hyper parameters as much as possible and further seek for the optimal hyperplane between different categories. However, ANFIS and BPNN tend to overfit as training data is relatively limited because samples are acquired on-line by the mechanism simulation model. Moreover, the measured realtime data has some differences from the training samples which also make them less accurate than the proposed algorithm. 5. Concluding remarks Large faults that are not novel may be inferred by a qualitative model during fault diagnosis. In this paper, we investigate the diagnosis of small faults such as a small leakage in cold leg of the

reactor coolant system and adopts a hybrid fault diagnosis algorithm capable of higher detection accuracy with a small number of samples. This paper focuses mainly on fault diagnosis in the reactor coolant system of pressurized water reactor and presents relevant technical diagnostic route based on an online thermal-hydraulic simulation model, support vector machine, and improved particle swarm optimization. The originality and useful features of this study are summarized as below: 1. The defects in the qualitative analysis are compensated with a noise-reducing algorithm. The data-driven methodology is integrated with knowledge-based methodologies to troubleshoot and validate system faults. The knowledge-based method utilizes fewer sample data, while the data-driven approach is trained with the data from a plant simulator 2. For the selection of significant parameters such as kernel function width and punishment factor in SVM, a hybrid PSO algorithm

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.

14

H. Wang, M.-j. Peng, J.W. Hines et al. / ISA Transactions xxx (xxxx) xxx

is adopted, and it guarantees parameter optimization and reduced the training time. 3. The calculation of fault severity with the approximate analytical formulas reduces the workload of the thermal-hydraulic simulation model and further improves the classification accuracy. 4. Compared with other parameter optimization methods, the optimization of critical parameters with PSO results in more accurate and faster SVM classification. In conclusion, the proposed methodology gives a significant improvement on the accuracy of process fault diagnosis. More importantly, to maintain system safety and reliability, this method can be integrated into the existing computerized operator support system and can be extended to diagnose faults in complex thermal plants and other components of NPP. Our future work is to build an experimental platform, and then to acquire actual data to verify the proposed on-line simulation model and fault diagnosis algorithm. Acknowledgments This work is funded by the Chinese national research project ‘‘Research of Online Monitoring and Operator Support Technology’’. The authors are also grateful for the support from the Chinese national scholarship council (201706680057). Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper References [1] Idaho National Laboratory. Report from the light water reactor sustainability. In: Workshop on on-line monitoring technologies. INL/EXT-10-19500. 2010. [2] International Atomic Energy Agency. Advanced surveillance, diagnostic and prognostic techniques in monitoring structures, systems and components in nuclear power plants. IAEA Nuclear Energy Series No. NP-T-3.14, Vienna: IAEA; 2013. [3] International Atomic Energy Agency. On-line monitoring for improving performance of nuclear power plants part 1: instrument channel monitoring. IAEA Nuclear Energy Series. No. NP-T-1.1, Vienna: IAEA; 2008. [4] Peng CL, Chen GH, et al. Methodology for analyzing the dependencies between human operators in digital control system. Fuzzy Sets and Systems 2016;293:127–43. [5] Song JG, Jung WL, et al. An analysis of technical security control requirements for digital I & C systems in nuclear power plant. Nucl Eng Technol 2013;45(5):637–52. [6] Chung HY, Bien Z, Park JH, Seong PH. Incipient multiple fault diagnosis in real time with application to large-scale system. IEEE Trans Nucl Sci 1994;41(4):1692–703. [7] Qin SJ. Survey on data-driven industrial process monitoring and diagnosis. Annu Rev Control 36(2):220-234. [8] Hadad K, Pourahmadi M, Maraghi H. Fault diagnosis and classification based on wavelet transform and neural network. Prog Nucl Energy 2011;53(6):41–7. [9] Wolbrecht E, Ambrosio BD, Passch B. Monitoring and diagnosis of a multistage manufacturing process using Bayesian networks. Artif Intell Eng Des Manuf 2000;14(2):53–67. [10] Zhang Z, Chen J. Fault detection and diagnosis based on particle filters combined with interactive multiple-model estimation in dynamic process systems. ISA Trans. 2019;85:247–61. [11] Lind M. An introduction to multilevel flow modeling. J Nucl Saf Simul 2011;2(1):22–32. [12] Kramer MA, Palowitch BL. A rule-based approach to fault diagnosis using the signed directed graph. AIChE J 1987;33(7):1067–78.

[13] Wang H, Peng M, Wu P, Cheng S. Improved methods of online monitoring and prediction in condensate and feed water system of nuclear power plant. Ann. Nucl. Energy 2016;90:44–53. [14] Thomas U, Roger L, et al. A computerized operator support system prototype. INL/EXT-15-36788, Idaho National Laboratory; 2015. [15] Chu YY, Yang M, Yang F. Design of an operator support system for online maintenance at nuclear power plant. In: Proceedings of International Symposium on Future I & C for Nuclear Power Plants, ICI2011. Daejeon, Korea. August 21–25, 2011. [16] Peng M, Wang H, Chen S, et al. An intelligent hybrid methodology of online system-level fault diagnosis for nuclear power plant. Nucl Eng Technol 2018;50(3):396–410. [17] Wang H, Peng M, et al. An integrated data-driven methodology for early fault detection and diagnosis in nuclear power plant. Int J Nucl Saf Simul 2017;8(3):225–38. [18] Kramer MA, Palowitch BL. A rule-based approach to fault diagnosis using the signed directed graph. AIChE J 1987;33(7):1067–78. [19] Ramesha TS, Davisa JF, Schwenzer GM. Knowledge-based diagnostic systems for continuous process operations based upon the task framework. Comput Chem Eng 1992;16(2):109–27. [20] Wang H, Chai TY, Ding JL, et al. Data driven fault diagnosis and fault tolerant control: some advances and possible new directions. Acta Automat Sinica 2009;35(6):739–47. [21] Ranaee V, Ebrahimzadeh A, Ghaderi R. Application of the PSO–SVM model for recognition of control chart patterns. ISA Trans. 2010;49:577–86. [22] Kwon KC, Kim JH. Hidden Markov models-based real-time transient identifications in nuclear power plants. Int. J. Intell. Syst. 2002;17(8):791–811. [23] Park S, Park J, Heo G. Transient diagnosis and prognosis for secondary system in nuclear power plants. Nucl Eng Technol 2016;48(5):1184–91. [24] Ge ZQ. Review on data-driven modeling and monitoring for plant-wide industrial processes. Chemometr Intell Lab Syst 2017;171:16–25. [25] Hsu CW, Lin CJ. A comparison of methods for multi-class support vector machines. IEEE Trans Neural Netw 2002;13(2):415–25. [26] Wu PF, Guo LQ, Zhou DX, Li S. An improved particle swarm optimization algorithm for reliability-redundancy allocation problem with mixed redundancy strategy and heterogeneous components. Reliab Eng Syst Saf 2019;181:62–74. [27] Vapnik V, Chervoknenkis AY. The necessary and sufficient conditions for the uniform convergence of averages to their expected values. Teor Veroyatn Primen 1981;26(3). 543564. [28] Vapnik V. Statistical learning theory. second ed.. New York: J. Wiley; 1998, p. 175–203. [29] Kennedy J, Eberhart R. Particle swarm optimization. In: Proceeding of IEEE international conference on neural networks. Perth: 1995. p. 1942–48. [30] Messaoud RB. Observer for nonlinear systems using mean value theorem and particle swarm optimization algorithm. ISA Trans. 2019;85:226–36. [31] Riccardo P, James K, Tim B. Particle swarm optimization-an overview. Swarm Intell 2007;1(1):33–57. [32] Shi YH, Eberhart R. A modified particle swarm optimizer. In: Evolutionary computation proceedings, the IEEE international conference on computational intelligence. AK, USA: Anchorage; 1998, p. 69–73. [33] Mohammadia M, Lakestanib M, Mohamed MH. Intelligent parameter optimization of savonius rotor using artificial neural network and genetic algorithm. Energy 2018;143(15):56–68. [34] Angeline PJ. Using selection to improve particle swarm optimization. In: IEEE international conference on evolutionary computation. Anchorage; 1998, p. 84–9. [35] Ji Z, Liao HL, Wang YW, et al. A novel intelligent particle optimizer for global optimization of multimodal functions. In: IEEE congress on evolutionary computation. Singapore: 2007. [36] Peng M, Wang H, Yang X, et al. Real-time simulations to enhance distributed on-line monitoring and fault detection in pressurized water reactors. Ann. Nucl. Energy 2017;85:259–70. [37] Liu Z, Guo W, Hu J, Ma W. Hybrid intelligent multi-fault detection method for rotating machinery based on RSGWPT, KPCA and twin SVM. ISA Trans. 2017;66:249–61. [38] He Q, Hu X, Ren H, Zhang H. A novel artificial fish swarm algorithm for solving large-scale reliability–redundancy application problem. ISA Trans. 2015;59:105–13. [39] Atashi A, Abba E, Dor S. Breast cancer risk assessment using adaptive neuro-fuzzy inference system (ANFIS) and subtractive clustering algorithm. Multidiscip Cancer Investig 2017;1(2):20–6. [40] Liu YK, Peng MJ, Xie CL. Research and design of distributed fault diagnosis system in nuclear power plant. Prog. Nucl. Energy 2013;68:97–110.

Please cite this article as: H. Wang, M.-j. Peng, J.W. Hines et al., A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Transactions (2019), https://doi.org/10.1016/j.isatra.2019.05.016.