- Email: [email protected]
Process Monitoring Based on Recursive Probabilistic PCA for Multi-mode Process∗
9th International Symposium on Advanced Control of Chemical Processes 9th International Symposium on Advanced Control of Chemical Processes June 7-10, 2015. Whistler, British Columbia,Control Canadaof Chemical Processes 9th International International Symposium on Advanced Advanced Control 9th Symposium on June 7-10, 2015. Whistler, British Columbia, Canadaof Chemical Processes Available online at www.sciencedirect.com June Canada June 7-10, 7-10, 2015. 2015. Whistler, Whistler, British British Columbia, Columbia, Canada
ScienceDirect IFAC-PapersOnLine 48-8 (2015) 1294–1299
Process Monitoring Based on Recursive Process Process Monitoring Monitoring Based Based on on Recursive Recursive Probabilistic PCA for Multi-mode Process Probabilistic PCA for Multi-mode Probabilistic PCA for Multi-mode Process Process ∗ Zhengdao Zhang, Bican Peng, Linbo Xie, Li Peng ∗ , Zhengdao Zhang, Bican Peng, Linbo Xie, Li Peng ∗ , Zhengdao Zhang, Bican Peng, Linbo Xie, Li Peng Zhengdao Zhang, Bican Peng, Linbo Xie, Li Peng ∗ ,, ∗ ∗ Key Laboratory of Advanced Process Control for Light Industry Key Laboratory of Advanced Process Control for Light Industry ∗ ∗ Key Laboratory of Process Control for Industry (Ministry of Education), Jiangnan University, Wuxi, Jiangsu 214122 Key Laboratory of Advanced Advanced Process Control for Light Light Industry (Ministry of Education), Jiangnan University, Wuxi, Jiangsu 214122 (Ministry of Education), Jiangnan University, Wuxi, Jiangsu China (e-mail: [email protected], [email protected], (Ministry of Education), Jiangnan University, Wuxi, Jiangsu 214122 214122 China (e-mail: [email protected], [email protected], China (e-mail: (e-mail: [email protected], [email protected], [email protected], [email protected]) China [email protected], [email protected], [email protected], [email protected]) [email protected], [email protected], [email protected]) [email protected])
Abstract: A A recursive recursive probabilistic probabilistic principal principal component component analysis analysis (PPCA) (PPCA) based based data-driven data-driven Abstract: Abstract: A recursive probabilistic principal component analysis (PPCA) based data-driven fault identification method is proposed to handle the missing data samples and the mode trantranAbstract: A recursive probabilistic principal component analysis (PPCA) based data-driven fault identification method is proposed to handle the missing data samples and the mode fault identification method is proposed to handle the missing data samples and the mode transition in multi-mode process. This model is recursively obtained by using the increasing number fault identification method is proposed to handle the missing data samples and the mode transition in multi-mode process. This model is recursively obtained by using the increasing number sition in process. This is by using the of normal normal observations with partly partly missing data. First, First,obtained based on on the singular value of of number historic sition in multi-mode multi-mode process. This model model is recursively recursively obtained bythe using the increasing increasing number of observations with missing data. based singular value of normal observations with partly missing into data.different First, based based onmodes the singular singular valuetransitions. of historic historic data matrix, the whole process is divided steady and mode of normal observations with partly missing data. First, on the value of historic data matrix, the whole process is divided into different steady modes and mode transitions. data matrix, the process different steady and mode For steady steady modes, the conventional conventional PPCA into is used used to obtain obtain the modes principal components, and to to data matrix, the whole whole process is is divided divided into different steady modes and mode transitions. transitions. For modes, the PPCA is to the principal components, and For steady modes, the conventional PPCA is used to obtain the principal components, and to impute the missing data. When the mode is a mode transition, the proposed recursive PPCA is For steady modes, the conventional PPCA is used to obtain the principal components, and to impute the missing data. When the mode is aa mode transition, the proposed recursive PPCA is impute the missing data. When the mode is mode transition, the proposed recursive PPCA is applied, which can actually reveal the between-mode dynamics for process monitoring and fault impute the missing data. When thethe mode is a mode transition, the process proposed recursive and PPCA is applied, which can actually reveal dynamics for applied, which can actually reveal the between-mode between-mode dynamics for analysis process monitoring monitoring and fault fault detection. Aftercan that, in order order to identify identify the faults, faults, aa dynamics contribution method is is developed developed applied, which actually reveal the between-mode for process monitoring and fault detection. After that, in to the contribution analysis method detection. After that, in order to identify the faults, aa contribution analysis method is developed and used to identify the variables which make the major contributions to the occurrence of faults. detection. After that, in order to identify the faults, contribution analysis method is developed and used to identify the variables which make the major contributions to the occurrence of faults. and to make major to of The used effectiveness of the the variables proposed which approach is the demonstrated by the the Tennessee Tennessee Eastman chemical chemical and used to identify identify variables which make the major contributions contributions to the the occurrence occurrence of faults. faults. The effectiveness of proposed approach is demonstrated by Eastman The effectiveness of proposed approach is the process. The results results show that the presented approach can can by accurately detectEastman abnormalchemical events, The effectiveness of the the proposed approach is demonstrated demonstrated by the Tennessee Tennessee Eastman chemical process. The show that the presented approach accurately detect abnormal events, process. Thefaults, resultsand show that the presented approach can accurately accurately detect detect abnormal abnormal events, events, identify the the it is isthat alsothe robust to mode mode transitions. process. The results show presented approach can identify faults, and it also robust to transitions. identify identify the the faults, faults, and and it it is is also also robust robust to to mode mode transitions. transitions. © 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Multi-mode, Multi-mode, mode mode transition, transition, missing missing data, data, recursive recursive probabilistic probabilistic PCA, PCA, Fault Fault Keywords: Keywords:and Multi-mode, mode transition, transition, missing missing data, data, recursive recursive probabilistic probabilistic PCA, PCA, Fault Fault detection identification. Keywords: Multi-mode, mode detection detection and and identification. identification. detection and identification. 1. INTRODUCTION INTRODUCTION Therefore, PPCA is not adaptive to multi-mode process 1. Therefore, PPCA is not adaptive to multi-mode process 1. Therefore, PPCA is not adaptive to multi-mode process in general, since it can lead to the frequent false alarms 1. INTRODUCTION INTRODUCTION Therefore, PPCA is not adaptive to multi-mode process in general, since it can lead to the frequent false alarms general, since it can lead to the frequent false alarms in the case of the operation mode is transiting from one general, since itoperation can lead mode to theisfrequent false alarms Since 1990s, 1990s, multivariate multivariate statistical statistical methods, methods, such such as as prinprin- in the case of the transiting from one Since in the case of the operation mode is transiting from one mode to another. Since 1990s, multivariate statistical methods, such as prinin the case of the operation mode is transiting from one cipal component analysis (PCA), avoid of directly modSince 1990s, multivariate statistical methods, such as prinmode to another. cipal component component analysis analysis (PCA), (PCA), avoid avoid of of directly directly modmod- mode to another. cipal mode to another. elling of complex systems, have been successfully applied cipal component analysis (PCA), avoid of directly modthere are many literature reported on the condielling of of complex complex systems, systems, have have been been successfully successfully applied applied Recently, there many literature reported on condielling to the the ofmonitoring monitoring and fault faulthave diagnosis in many many industrial industrial elling complex systems, been successfully applied Recently, Recently, there are arefor many literatureprocess reportedwhich on the the conditional monitoring multi-mode consider to and diagnosis in Recently, there are many literature reported on the conditional monitoring for multi-mode process which consider to the monitoring and fault diagnosis in many industrial processes (Chiang et al. (2001)). However, often caused by to the monitoring and fault diagnosis in many industrial tional monitoring for multi-mode process which consider the case of changing setting parameters. In (Hwang and processes (Chiang et al. (2001)). However, often caused by tional monitoring for multi-mode process which consider case of changing setting parameters. In (Hwang and processes (Chiang al. often by a sudden sudden mechanical mechanical breakdown, sensor failure failure orcaused malfuncprocesses (Chiang et et breakdown, al. (2001)). (2001)). However, However, oftenor caused by the the case of changing setting parameters. In (Hwang and Han (1999)), a multi-level clustering PCA method was a sensor malfuncthe case of changing setting parameters. In (Hwang and Han (1999)), a multi-level clustering PCA method was aation sudden mechanical breakdown, sensor failure or malfuncoccurred in data acquisition system, partly missing sudden mechanical breakdown, sensor failure or malfuncHan (1999)), (1999)), a multi-level multi-level clustering PCA in method was proposed to solve the multi-mode problem Tennessee tion occurred in data acquisition system, partly missing Han a clustering PCA method was proposed to solve the multi-mode problem in Tennessee tion occurred in data acquisition system, partly missing data or irregularly sampled data is a common phenomenon tion occurred in data acquisition system, partly missing proposed to solve the multi-mode problem in Tennessee Eastman (TE) process. A Gaussian mixture model and data or irregularly sampled data is a common phenomenon proposed to solve the multi-mode problem in Tennessee (TE) process. A Gaussian mixture model and data or sampled aa common in industrial industrial practice. Up to todata now,is the missing phenomenon data sample sample Eastman data or irregularly irregularly sampled data isthe common phenomenon Eastman (TE) process. A Gaussian mixture and PCA method been which can in practice. Up now, missing data Eastman (TE)have process. Acombined Gaussian together, mixture model model and PCA method have been combined together, which can in industrial practice. Up to now, the missing data sample problem is being a major challenge of most of existing in industrial practice. Up to challenge now, the missing data sample improve PCA method have been combined together, which can the efficiency of modeling the whole process (Xu problem is being a major of most of existing PCA method have been combinedthe together, which (Xu can improve the efficiency of modeling whole process problem is being aa major challenge of most of existing monitoring approaches. The process monitoring and fault problem is being major challenge of most of existing improve the efficiency efficiency of modeling modeling theproposed whole process process (Xu et al. (2010)). After that, Xu also a mixture monitoring approaches. The process monitoring and fault improve the of the whole (Xu et al. (2010)). After that, Xu also proposed a mixture monitoring approaches. The process monitoring and fault detection techniques techniques with partly missing measurements monitoring approaches. Thepartly processmissing monitoring and fault et al. (2010)). After Xu also a PCA to capture normal of production detection with measurements et al. model (2010)). After that, that, Xu distribution also proposed proposed a mixture mixture PCA model to capture normal distribution of production detection techniques with partly missing measurements have not been well studied. detection techniques with partly missing measurements PCA model to capture normal distribution of production process (Xu et al. (2011)). These methods did not conhave not been well studied. PCA model to capture normal distribution of production process (Xu et al. (2011)). These methods did not conhave not been well studied. have not been well studied. process (Xu et al. (2011)). These methods did not consider the mode transition in multi-mode process. To solve process (Xu et transition al. (2011)). These methods did not conBased on on aa Gaussian Gaussian latent latent variable variable model, model, probabilisprobabilis- sider the mode in multi-mode process. To solve Based sider the mode transition in multi-mode process. To solve this problem, Zhao has proposed a PCA-based modeling Based on a Gaussian latent variable model, probabilissider the mode transition in multi-mode process. To solve tic principal component analysis (PPCA) method, whose Based on a Gaussian latent variable model, probabilisthis problem, Zhao has proposed a PCA-based modeling tic principal principal component component analysis analysis (PPCA) (PPCA) method, method, whose whose this problem, Zhao has aa PCA-based modeling and monitoring strategy for multi-mode processes with tic problem, Zhao has proposed proposed PCA-based modeling parameters can be determined determined by(PPCA) the eigenvalue eigenvalue decomtic principalcan component analysisby method,decomwhose this and monitoring strategy for multi-mode processes with parameters be the and monitoring strategy for multi-mode processes with between-mode transitions. A mode-common subspace for parameters can be determined by the eigenvalue decomand monitoring strategy for multi-mode processes with position of the measurement sample covariance matrix or parameters can be determined by the eigenvalue decombetween-mode transitions. A mode-common subspace for position of the measurement sample covariance matrix or between-mode transitions. A mode-common subspace for all modes data is separated at first. Then, the operating position of the measurement sample covariance matrix or between-mode transitions. Aatmode-common subspace for the expectation-maximization expectation-maximization (EM) covariance algorithm (Kim (Kim and position of the measurement sample matrixand or all modes data is separated first. Then, the operating the (EM) algorithm all modes data is separated at first. Then, the operating data for each mode is projected to the mode-common the expectation-maximization (EM) algorithm (Kim and all modes data is separated at first. Then, the operating Lee (2003)), has been regarded as an efficient method the expectation-maximization (EM) algorithm (Kim and data for each mode is projected to the mode-common Lee (2003)), (2003)), has has been been regarded regarded as as an an efficient method method data for each is to mode-common subspace mode-special remaining subspace. A mode Lee for and eachaa mode mode is projected projected to the the mode-common for handling handling missing data and forming forming mixturemethod model. data Lee (2003)), missing has been regarded as an aaefficient efficient subspace and mode-special remaining subspace. A mode for data and mixture model. subspace and a mode-special remaining subspace. A mode transition identification algorithm was designed to detect for handling missing data and forming aa mixture model. subspace and a mode-special remaining subspace. A mode However, PPCA often follows the unimodal distribution for handling missing data and forming mixture model. transition identification algorithm was designed to detect However, PPCA often follows the unimodal distribution transition identification algorithm was designed to detect the abnormal behaviors (Zhao et al. (2010)). Wang idenHowever, PPCA often follows the unimodal distribution transition identification algorithm was(2010)). designed to detect assumption of the operating data, which is similar to PCA. However, PPCA often follows the unimodal distribution the abnormal behaviors (Zhao et al. Wang idenassumption of of the the operating operating data, data, which which is is similar similar to to PCA. PCA. tified the abnormal behaviors (Zhao et al. (2010)). Wang identhe stable modes, mode transitions, and noise at assumption the abnormal behaviors (Zhao et al. (2010)). Wang idenIn practice, mode transition often takes place in some assumption of the operating data, which is similar to PCA. tified the stable modes, mode transitions, and noise at In practice, practice, mode mode transition transition often often takes takes place place in in some some tified the stable modes, mode transitions, and noise at first. Then, according to the data distribution, a proper In tified the stable modes, mode transitions, and noise at of industrial processes because of different manufacturIn practice, mode transition often takes place in some first. Then, according to the data distribution, a proper of industrial processes because of different manufacturfirst. Then, according to the data distribution, a proper multivariate statistical algorithm was chosen to detect the of industrial processes because of different manufacturfirst. Then, according to the data distribution, a proper ing strategies, various product specifications, and so on. of industrial processes because of different manufacturstatistical algorithm was chosen to detect the ing strategies, strategies, various product product specifications, and and so on. on. multivariate multivariate statistical algorithm was to detect for each mode (Wang et al. (2012)). Zhang ing multivariate statistical algorithm was chosen chosen toproposed detect the thea ing strategies, various various product specifications, specifications, and so so on. fault fault for each mode (Wang et al. (2012)). Zhang proposed a fault for for each each mode (Wang et et al.multi-modes (2012)). Zhang Zhang proposed a The work is supported by National Natural Science Foundation recursive PCA to separate the by taking into fault mode (Wang al. (2012)). proposed a The work is supported by National Natural Science Foundation recursive PCA to separate the multi-modes by taking into recursive PCA to separate the multi-modes by taking into account of the cross-mode correlations and extracting the of The China (No.61374047, No.61202473) and Fundamental Research work is supported by National Natural Science Foundation recursive PCA to separate the multi-modes by taking into work is supportedNo.61202473) by National Natural Science Foundation account of the cross-mode correlations and extracting the of The China (No.61374047, and Fundamental Research account of the correlations and the common between the different modes(Zhang Funds for (No.61374047, Central Universities (JUSRP51322B, JUSRP111A49). of China No.61202473) and Research account ofinformation the cross-mode cross-mode correlations and extracting extracting the of China No.61202473) and Fundamental Fundamental Research common information between the different modes(Zhang Funds for (No.61374047, Central Universities (JUSRP51322B, JUSRP111A49). common information between the different modes(Zhang Funds for Central Universities (JUSRP51322B, JUSRP111A49). common information between the different modes(Zhang Funds for Central Universities (JUSRP51322B, JUSRP111A49). Copyright 2015 IFAC 1295Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2015, 2015 IFAC 1295 Copyright 2015 IFAC 1295 Peer review© of International Federation of Automatic Copyright ©under 2015 responsibility IFAC 1295Control. 10.1016/j.ifacol.2015.09.147
IFAC ADCHEM 2015 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
1295
et al. (2012)). And then, to improve the monitoring performance of the between-mode, Zhang extracted the common subspace form the different modes and between-mode transition which was been applied to conditional monitoring for the electro-fused magnesia furnace(Zhang and Li (2013)). And then, this method has also been generalized to detect the fault of non-gaussian processes (Zhang et al. (2013)).
Rd , (i = 1, ..., k). If a new observation yk+1 is available, then the mean value is changed to k 1 µk+1 = µk + yk+1 (3) k+1 k+1
It should be noted that all of aforementioned methods are the improvements of PCA approaches, which implies that the performance and efficiency of those approaches are still susceptible to the influence of commonly occurred interference factors such as stochastic noise and missing data (Wang and Li (2012); Xia et al. (2013); Elshenawy et al. (2009)). To overcome monitoring problems both on data missing and mode transition in multi-mode process, this paper proposed a recursive PPCA method within a probabilistic framework. The historical data are divided into serval operation modes and the mode transitions between them based on the singular value recognition. Than, the PPCA method is used to model the steady modes and the recursive PPCA method is used to model the mode transition. The missing data can also be estimated in a probabilistic framework. After that, a Mahalanobis contribution analysis based on the recursive PPCA method is proposed for fault detection and diagnosis. The efficiency of our approach is illustrated through the TE chemical process, and the result shows that the presented approach can accurately detect abnormal events and identify the faults in the multi-mode system.
then, the covariance matrix Sk+1 can be calculated recursively as
2. MODELING OF MULTI-MODE SYSTEM
Define a variable as ∆µk+1 = µk+1 − µk
Sk+1 =
(5)
k 1 T Notice that Sk = k−1 i=1 (yi − µk ) (yi − µk ), (5) can be reformulated as k−1 Sk + ∆µk+1 ∆µTk+1 Sk+1 = k (6) 1 + (yk+1 − µk − ∆µk+1 )(yk+1 − µk − ∆µk+1 )T k Equation (6) indicates that Sk+1 is two rank-one modifications from Sk . Consequently, the above update procedure has to be performed twice to estimate the eigenvalues and eigenvectors matrix for the current covariance matrix. After that, the parameters of the proposed PPCA model can be estimated by the observations and the updated values. Following the prior distribution assumptions, we obtain (7) E(xi ) = (W T W + σ 2 I)−1 W T (yi − µ) and E(xi xTi ) = σ 2 (W T W + σ 2 I)−1 + E(xi )E(xi )T
2.1 PPCA and Recursive PPCA Algorithm Assume that the latent variable model relates a ddimensional observed variable y to a q-dimensional score vector of latent variable x , which implies: y = Wx + µ +
k+1 1 (yi − µk+1 )T (yi − µk+1 ) k i=1
(4)
(1)
(8)
where E(·) is the expectation operator, i = 1...k. Then, the corresponding log-likelihood of observed data under this model is defined as k log p(yi , xi ) (9) lk = i=1
where W ∈ Rd×q a linear transformation matrix which is composed of the eigenvectors of sample covariance matrix corresponding to the q(q < d) largest eigenvalues, µ is the mean of the data y, and is the noise term which is assumed to be Gaussian and isotropic, ∼ G(0, σ 2 I). Then, the conditional probability distribution y|x is also Gaussian, which is follow y|x ∼ G(W x + µ, σ 2 I)
(2)
Furthermore, by adopting a Gaussian prior distribution for the scores variables x, x ∼ G(0, I), then, the marginal distribution of y is Gaussian in the form of y ∼ G(µ, C) , where the covariance matrix is C = W W T + σ 2 I. Therefore, the PPCA method provides a way to constrain the model complexity via the selection of q, and the model parameters can be estimated by the maximum likelihood algorithm (Qu et al. (2009); Li et al. (2013)). When the process transit from one mode to another, the recursive PPCA method can update the model parameters in a recursive way. Suppose that the historical output data can be denoted as Yk = [y1 , y2 , ..., yk ]T , where yi ∈
where the joint probability density p(yi , xi ) is given by −xi 2 p(yi , xi ) = (2πσ 2 )−d/2 × exp{ } 2 2 −yi − W xi − µ ×exp{ }(2π)−q/2 2σ 2
(10)
Take the conditional expectation with respect to the distribution in Equation (10), we obtain k d 1 { log σk2 + tr(E(xi xTi )) E(lk ) = − 2 2 i=1 1 1 + 2 yi − µk 2 − 2 E(xi )T WkT (yi − µk ) 2σk σk 1 T + 2 tr(Wk Wk E(xi xTi ))} 2σk
(11)
Maximize the conditional expectation of(11) with respect to Wk and σk2 , then the parameters are updated as k k T T Wk = (yi − µk )E(xi ) E(xi xi ) (12)
1296
i=1
i=1
IFAC ADCHEM 2015 1296 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
k
1 { yi − µk 2 − 2E(xi )T WkT (yi − µk ) kd i=1 +tr E(xi xTi )WkT Wk }
σk2 =
2.2 Missing data Imputation Approach For an incomplete measurement yk , it can be divided into the observable part yk,o and the missing part yk,u . Therefore, (9) is rewritten as k
log p(yi,o , yi,u , xi )
(14)
i=1
Mode 2
Fig. 1. Global mode structure 3.1 Mode Detection and Process Modeling It is noted that the singular values of process data can be changed with the increasing samples in between-mode H (Wang and Li (2012)). Let Y = U ΣV Σ = p , where 2 2 diag(τ1 , τ2 , ..., τp , ), and define τ = τ = Y F , i=1 i then the mode detection index at k time instant can be defined as k 1 Tk (j) = Σi (j, j) (17) L i=k−L+1
Similar to the Maximum Likelihood Estimation in prior subsection, we obtain the estimation of yk y˜k (n) = θk Wk [Nk −1 Wk T (yk − µk )] + µk
between-mode trasition
Mode 1
Let the window width of original data is L, namely, the sample yk−L+1 will be replaced by the new observation yk+1 at k + 1 instant. Thus, we can update model with new parameters of W and σ 2 .
lk =
Global mode
(13)
(15)
which presents the average of parameter τ at k time instant, where Σk (j, j) is the jth diagonal values of Σk , and Σk means the matrix Σ at k time instant as well. Then, at k + 1 time instant, we have
where Nk = Wk T Wk +σk2 I. The value of θk can be obtained by minimizing ˜ yk,o − yk,o 2 .
Based on above results, at k + 1 time instant, an EM algorithm for PPCA modeling can be summarized as follows:
Let µk+1 (0) = µk , Wk+1 (0) = Wk ,σk+1 (0) = σk ,Nk+1 (0) = 2 Wk+1 (0)T Wk+1 (0) + σk+1 (0)I, then repeat the following steps: y˜k+1 (n + 1) = θk+1 (n + 1)Wk+1 (n)[Nk+1 (n)−1 Wk+1 (n)T (yk+1 (n) − µk+1 (n))] + µk+1 (n) min { ˜ yk+1,o (n + 1) − yk+1,o 2 } θk+1 (n+1)
k 1 µk + y˜k+1 k+1 k+1 ∆µk+1 (n + 1) = µk+1 (n + 1) − µk k (16) 1 T Sk+1 (n + 1) = (˜ yi (n) − µ)(˜ yi (n) − µ) k i=1 Wk+1 (n + 1) = Sk+1 (n)Wk+1 (n)(σk+1 (n)2 I) + Nk+1 (n)T Sk+1 (n)Wk+1 (n))−1 1 σk+1 (n + 1)2 = tr(Sk+1 (n) p − Sk+1 (n)Wk+1 (n)Nk+1 (n)−1 Wk+1 (n + 1)T )
Tk+1 (j) =
1 L
k+1
Σi (j, j)
(18)
i=k−L+2
Therefore, mode transition time can be recognized by the range ability of τ 2 , as: Tk+1 (j) − Tk (j) > γj (19) Tk (j) where γj is a predefined parameter in dynamic process, which is related to mode transitions and determined by human experience (Wang and Li (2012)). 3.2 Confidence Bound and Fault Diagnosis
µk+1 (n + 1) =
where the n is the number of iterations. Above setps are repeated iteratively until convergence and then the process of parameters learning is finished.
Based on probability theory, the β% confidence bound can be defined as (Chen and Sun (2009)): p(y)dy = β (20) y:p(y)>h
where p(y) is the probability distribution of y, and the squared Mahalanobis distance based equivalent confidence bound, which can be used for fault detection and diagnosis, is shown as follow: Mk2 = (yk − µk )T Sk−1 (yk − µk ) > χ2d (β)
where β is the fractile of the χ2 distribution with the degree of freedom d, and the value can be updated as −1 2 Mk+1 = (yk+1 − µk+1 )T Sk+1 (yk+1 − µk+1 ) > χ2d (β) (22)
3. ONLINE MONITORING AND FAULT DIAGNOSIS For simplification of the expression, we assume that the whole process has two stationary modes and a betweenmode, which is shown in Fig.1. Than, Mode 1 and Mode 2 are steady modes built with regular PPCA, while the between-mode transition will be modelled by the proposed recursive PPCA method.
(21)
Therefore, the recalculated monitoring statistic is given by −1 2 E(Mk+1 ) = tr(Sk+1 {(z − µk+1 )(z − µk+1 )T + Q}) (23)
where z and Q are related as y|y0 ∼ N (z, Q), whose details can be found in Chen and Sun (2009) and 2 He et al. (22). 2 Finally, we can calculate Mk+1 − E Mk+1 for fault
1297
IFAC ADCHEM 2015 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
diagnosis with 99% confidence bound, and the confidence 2 limit is Mk+1 − χ2d (β) (Chen and Sun (2009)). As a result, the process behaviour is considered faulty if the statistic of a new observation exceeds the control limit.
On-line fault diagnosis of recursive PPCA
Model development
Acquire normal samples Y(k)
Y(k)
3.3 Modeling and Monitoring for Multi-mode Process In this part, the step-by-step procedure of recursive PPCA model based multi-mode monitoring approach is given below, and the flowchart of this method on fault detection and diagnosis is shown in Fig. 3. Firstly, the procedure of model development is given, as:
Calculate
In this section, the TE process is used to evaluate the effectiveness of the proposed fault detection and identification approach based on recursive PPCA in multi-mode system. The TE process is an open-lop unstable plant-wide process control problem considered as a benchmark simulation for various process monitoring techniques. This process produces two products (G and H) from four reactants (A, C, D and E), and F is a byproduct. In addition, an inert component B also presents in C stream and in trace amount in the A feed stream. The process consists of five major units, which include an exothermic two-phase reactor, a flash separator, a recycle compressor, a reboiled stripper, and a product condenser(Liu and Chen (2010); Yin et al. (2012)). The TE process has total 11 input variables (without agitator speed) and 41 measurement variables. And the process measurements are sampled with an interval of 3 min. For simplicity, only 22 continuous measurements, listed in Table 1, are selected in this simulation.
Calculate
Calculate and its confidence limits
Calculate contributions of each variable
Calculate the upper confidence limits for contributions
Input new samples Y(k+1)
2 new , Wnew , new
Calculate and its confidence limits
M 2new
M2
Fault detection
Beyond the confidence limits?
No
Rcalculate
Exceed confidence limits?
In addition, the online fault diagnosis of recursive PPCA approach is also given below:
4. SIMULATION STUDY
, W , 2
(1) Acquire normal operating data and normalize the data using the mean and standard deviation of each variable. (2) Calculate parameters of µ, W and σ 2 . (3) Calculate squared Mahalanobis distance M 2 and its confidence limits. (4) Calculate E M 2 and M 2 − E M 2 to estimate contribution of each variable. (5) Calculate the upper confidence limits for contributions.
(1) Obtain new observation data, and then make model detection in multi-mode process. 2 (2) Recalculate new parameters of µk+1 , Wk+1 and σk+1 . 2 (3) Recalculate new squared Mahalanobis distance Mk+1 and its confidence limits. (4) If any statistic exceeds its corresponding confidence limit, calculate the contributions of each variable after fault detection. (5) Monitor the contribution of each variable and make fault diagnosis.
1297
No
Yes End
Get the faulty variables
Fig. 2. Procedures of recursive PPCA method have to be adjusted to meet the production specifications, which can cause various operation modes. Some of them are steady state modes, while others are between-mode transitions. In the simulation, we suppose that the whole process is divided into two modes and a between-mode, which are listed on Table 2. Mode transition is introduced
Since it is difficult to produce products consistently in industrial scale, the quality control in industry relies significantly on the consistency of process conditions. Because of process changing, the operating conditions 1298
Table 1. Continuously Measured Variables Variable’s ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Description A feed (stream 1) D feed (stream 2) E feed (stream 3) A and C feed (stream 4) Recycle flow (stream 8) Reactor feed rate (stream 6) Reactor pressure Reactor level Reactor temperature Purge rate (stream 9) Product separator temperature Product separator level Product separator pressure Product separator underflow (stream 10) Stripper level Stripper pressure Stripper underflow (stream 11) Stripper temperature Stripper stream flow Compressor work Reactor cooling water outlet temperature Condenser cooling water outlet temperature
IFAC ADCHEM 2015 1298 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
by decreasing the value of reactor temperature from 120 to 110. To test the ability of proposed method, we define mode 1, mode 2 and the mode transition between as normal operating modes, and let failures occur in mode 2. Both methods are performed on the 10% randomly missing data and the number of principal components is set to 5 based on the results of a 10-fold cross validation.
140
120
100
M2
80
60
Table 2. Operating Modes of Simulated Te Process Modes Normal Operating Mode 1 Between-mode transition Normal Operating Mode 2 Faulty Operating Mode
40
Samples 1st 300th 301st 600th 601st 700th 701st 900th
20
0
The fault detection result with missing data of the proposed method is shown in Fig. 3. The M 2 calculated from the recursively updated PPCA model as well as the 99% confidence. This figure shows the values of statistics is sharply increasing from 702th time instant,and it is outside the M 2 limit at 703th time instant, which implies that fault has been detected in mode 2 for the time of the 703th sample. For all other time instants from 1 to 700, statistics are within their confidence limits, which means the false alarm rate is significantly eliminated. 140
120
100
0
100
200
300
400 500 Sample Number
600
700
800
900
Fig. 4. Monitoring results by conventional PPCA Furthermore, the diagnosis results are shown in Fig. 5 and Fig. 6, which indicate that the recursive PPCA based and the conventional PPCA based multi-mode monitoring and fault diagnosis have different contribution results. It can be seen that the contributions of the 2nd and 3rd variable are much higher than those of other variables in Fig. 5, which means that the proposed method can isolate fault variables efficiently. However, it is very difficult to determine which variables are the dominant sources of fault in Fig. 6, since 7 variables have exceeded control limits. As a result, the contribution plots shows that the proposed method is capable of isolating variables as few as possible, but not essential, faulty variables to further fault diagnosis. 7
M2
80 6
60 5
Contribution
40
20
0
4
3
2
0
100
200
300
400 500 Sample Number
600
700
800
900 1
0 0
Fig. 3. Monitoring results by recursive PPCA
5
10
15
20
Variable
Based on the measurements of mode 1, the simulation results of PPCA with a 10% randomly missing rate is shown in Fig. 4. In the plot, the regular PPCA based calculated M 2 frequently exceed their respective confidence limits in the mode transition and the steady part of mode 2 where the process is operating normally. The similar results can be obtained if only the measurements of mode 2 are used for regular PPCA. Clearly, both regular PPCA and recursive PPCA can detect the fault correctly at the 703th sample. This may be explained by the fact that mode 2 is a stationary process which is modeling by regular PPCA method in multi-mode system, and only transition process is modeling by recursive PPCA. However, the false alarm rate of conventional PPCA is much higher than the false alarm rate of the proposed method, which means the recursive PPCA method is more effective to multi-mode system.
Fig. 5. Recursive PPCA based contribution plots 5. CONCLUSION In this study, a recursive PPCA model based process monitoring and fault identification approach is developed for the multi-mode processes with randomly missing data. Different from the conventional PPCA, the proposed method recursively obtain the estimation of the missing measurement data and update model parameters. Based on the model detection index, a global mode with steady-state modes and between-mode transitions is thus developed. The illustration results demonstrate that the proposed approach can effectively detect and identify fault with missing data for multi-mode process, and it will not trigger false alarms in between-mode transition.
1299
IFAC ADCHEM 2015 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
6
5
Contribution
4
3
2
1
0
0
5
10
15
20
Variable
Fig. 6. Conventional PPCA based contribution plots REFERENCES Chen, T. and Sun, Y. (2009). Probabilistic contribution analysis for statistical process monitoring: a missing variable approach. Control Engineering Practice, 17, 469–477. Chiang, L., Braatz, R., and Russell, E. (2001). Fault detection and diagnosis in industrial systems. Springer Verlag, London. Elshenawy, L., Yin, S., Naik, A., and Ding, S. (2009). Efficient recursive principal component analysis algorithms for process monitoring. Industrial & Engineering Chemistry Research, 49, 252–259. He, B., Yang, X., Chen, T., and Zhang, J. (22). Reconstruction-based multivariate contribution analysis for fault isolation: A branch and bound approach. Journal of Process Control, 2012, 1228–1236. Hwang, D. and Han, C. (1999). Real-time monitoring for a process with multiple operating modes. Control Engineering Practice, 7, 891–902. Kim, D. and Lee, I. (2003). Process monitoring based on probabilistic pca. Chemometrics and Intelligent Laboratory Systems, 67, 109–123. Li, L., Li, Y., and Li, Z. (2013). Efficient missing data imputing for traffic flow by considering temporal and spatial dependence. Transportation Rearch Part C, 34, 108–120. Liu, J. and Chen, D. (2010). Operational performance assessment and fault isolation for multimode processes. Industrial & Engineering Chemistry Research, 49, 3700– 3714. Qu, L., Zhang, Y., and Hu, J. (2009). Ppca-based missing data imputation for traffic flow volume: a systematical approach. IEEE Transaction on Intelligent Transportation Systems, 10, 512–522. Wang, F., Tan, S., Peng, J., and Chang, Y. (2012). Process monitoring based on mode identification for multi-mode process with transitions. Chemometrics and Intelligent Laboratory Systems, 110, 144–155. Wang, Q. and Li, H. (2012). A process singular value recognition based recursive pca approach to modetransition process monitoring. Journal of Chemical Industry and Engineering, 63, 2948–2952. Xia, L., Chu, J., and Geng, Z. (2013). Process monitoring based on improved recursive pca methods by adaptive extracting principal components. Transactions of the Institute of Measurement and Control, 35, 1024–1045.
1299
Xu, X., Xie, L., and Wang, S. (2010). A multi-mode process monitoring method based gaussian mixture models. Computers and Applied Chemistry, 27, 17–22. Xu, X., Xie, L., and Wang, S. (2011). Multi-mode process monitoring method based on pca mixture model. Journal of Chemical Industry and Engineering, 62, 743–752. Yin, S., Ding, S.X., Haghani, A., Hao, H., and Zhang, P. (2012). A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark tennessee eastman process. Journal of Process Control, 2, 1567–1581. Zhang, Y., An, J., and Ma, C. (2013). Fault detection of non-gaussian processes based on model migration. IEEE Transactions on Control Systems Technology, 21, 1517– 1525. Zhang, Y., Chai, T., Li, Z., and Yang, C. (2012). Modeling and monitoring of dynamic processes. IEEE Transactions on Neural Networks and Learning Systems, 23, 277–284. Zhang, Y. and Li, S. (2013). Modeling and monitoring between-mode transition of multimodes processes. IEEE Transactions on Industrial Informatics, 9, 2248–2255. Zhao, C., Yao, Y., Gao, F., and Wang, F. (2010). Statistical analysis and online monitoring for multimode processes with bbetween-mode transitions. Chemical Engineering Science, 65, 5961–5975.
1300
ScienceDirect IFAC-PapersOnLine 48-8 (2015) 1294–1299
Process Monitoring Based on Recursive Process Process Monitoring Monitoring Based Based on on Recursive Recursive Probabilistic PCA for Multi-mode Process Probabilistic PCA for Multi-mode Probabilistic PCA for Multi-mode Process Process ∗ Zhengdao Zhang, Bican Peng, Linbo Xie, Li Peng ∗ , Zhengdao Zhang, Bican Peng, Linbo Xie, Li Peng ∗ , Zhengdao Zhang, Bican Peng, Linbo Xie, Li Peng Zhengdao Zhang, Bican Peng, Linbo Xie, Li Peng ∗ ,, ∗ ∗ Key Laboratory of Advanced Process Control for Light Industry Key Laboratory of Advanced Process Control for Light Industry ∗ ∗ Key Laboratory of Process Control for Industry (Ministry of Education), Jiangnan University, Wuxi, Jiangsu 214122 Key Laboratory of Advanced Advanced Process Control for Light Light Industry (Ministry of Education), Jiangnan University, Wuxi, Jiangsu 214122 (Ministry of Education), Jiangnan University, Wuxi, Jiangsu China (e-mail: [email protected], [email protected], (Ministry of Education), Jiangnan University, Wuxi, Jiangsu 214122 214122 China (e-mail: [email protected], [email protected], China (e-mail: (e-mail: [email protected], [email protected], [email protected], [email protected]) China [email protected], [email protected], [email protected], [email protected]) [email protected], [email protected], [email protected]) [email protected])
Abstract: A A recursive recursive probabilistic probabilistic principal principal component component analysis analysis (PPCA) (PPCA) based based data-driven data-driven Abstract: Abstract: A recursive probabilistic principal component analysis (PPCA) based data-driven fault identification method is proposed to handle the missing data samples and the mode trantranAbstract: A recursive probabilistic principal component analysis (PPCA) based data-driven fault identification method is proposed to handle the missing data samples and the mode fault identification method is proposed to handle the missing data samples and the mode transition in multi-mode process. This model is recursively obtained by using the increasing number fault identification method is proposed to handle the missing data samples and the mode transition in multi-mode process. This model is recursively obtained by using the increasing number sition in process. This is by using the of normal normal observations with partly partly missing data. First, First,obtained based on on the singular value of of number historic sition in multi-mode multi-mode process. This model model is recursively recursively obtained bythe using the increasing increasing number of observations with missing data. based singular value of normal observations with partly missing into data.different First, based based onmodes the singular singular valuetransitions. of historic historic data matrix, the whole process is divided steady and mode of normal observations with partly missing data. First, on the value of historic data matrix, the whole process is divided into different steady modes and mode transitions. data matrix, the process different steady and mode For steady steady modes, the conventional conventional PPCA into is used used to obtain obtain the modes principal components, and to to data matrix, the whole whole process is is divided divided into different steady modes and mode transitions. transitions. For modes, the PPCA is to the principal components, and For steady modes, the conventional PPCA is used to obtain the principal components, and to impute the missing data. When the mode is a mode transition, the proposed recursive PPCA is For steady modes, the conventional PPCA is used to obtain the principal components, and to impute the missing data. When the mode is aa mode transition, the proposed recursive PPCA is impute the missing data. When the mode is mode transition, the proposed recursive PPCA is applied, which can actually reveal the between-mode dynamics for process monitoring and fault impute the missing data. When thethe mode is a mode transition, the process proposed recursive and PPCA is applied, which can actually reveal dynamics for applied, which can actually reveal the between-mode between-mode dynamics for analysis process monitoring monitoring and fault fault detection. Aftercan that, in order order to identify identify the faults, faults, aa dynamics contribution method is is developed developed applied, which actually reveal the between-mode for process monitoring and fault detection. After that, in to the contribution analysis method detection. After that, in order to identify the faults, aa contribution analysis method is developed and used to identify the variables which make the major contributions to the occurrence of faults. detection. After that, in order to identify the faults, contribution analysis method is developed and used to identify the variables which make the major contributions to the occurrence of faults. and to make major to of The used effectiveness of the the variables proposed which approach is the demonstrated by the the Tennessee Tennessee Eastman chemical chemical and used to identify identify variables which make the major contributions contributions to the the occurrence occurrence of faults. faults. The effectiveness of proposed approach is demonstrated by Eastman The effectiveness of proposed approach is the process. The results results show that the presented approach can can by accurately detectEastman abnormalchemical events, The effectiveness of the the proposed approach is demonstrated demonstrated by the Tennessee Tennessee Eastman chemical process. The show that the presented approach accurately detect abnormal events, process. Thefaults, resultsand show that the presented approach can accurately accurately detect detect abnormal abnormal events, events, identify the the it is isthat alsothe robust to mode mode transitions. process. The results show presented approach can identify faults, and it also robust to transitions. identify identify the the faults, faults, and and it it is is also also robust robust to to mode mode transitions. transitions. © 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Multi-mode, Multi-mode, mode mode transition, transition, missing missing data, data, recursive recursive probabilistic probabilistic PCA, PCA, Fault Fault Keywords: Keywords:and Multi-mode, mode transition, transition, missing missing data, data, recursive recursive probabilistic probabilistic PCA, PCA, Fault Fault detection identification. Keywords: Multi-mode, mode detection detection and and identification. identification. detection and identification. 1. INTRODUCTION INTRODUCTION Therefore, PPCA is not adaptive to multi-mode process 1. Therefore, PPCA is not adaptive to multi-mode process 1. Therefore, PPCA is not adaptive to multi-mode process in general, since it can lead to the frequent false alarms 1. INTRODUCTION INTRODUCTION Therefore, PPCA is not adaptive to multi-mode process in general, since it can lead to the frequent false alarms general, since it can lead to the frequent false alarms in the case of the operation mode is transiting from one general, since itoperation can lead mode to theisfrequent false alarms Since 1990s, 1990s, multivariate multivariate statistical statistical methods, methods, such such as as prinprin- in the case of the transiting from one Since in the case of the operation mode is transiting from one mode to another. Since 1990s, multivariate statistical methods, such as prinin the case of the operation mode is transiting from one cipal component analysis (PCA), avoid of directly modSince 1990s, multivariate statistical methods, such as prinmode to another. cipal component component analysis analysis (PCA), (PCA), avoid avoid of of directly directly modmod- mode to another. cipal mode to another. elling of complex systems, have been successfully applied cipal component analysis (PCA), avoid of directly modthere are many literature reported on the condielling of of complex complex systems, systems, have have been been successfully successfully applied applied Recently, there many literature reported on condielling to the the ofmonitoring monitoring and fault faulthave diagnosis in many many industrial industrial elling complex systems, been successfully applied Recently, Recently, there are arefor many literatureprocess reportedwhich on the the conditional monitoring multi-mode consider to and diagnosis in Recently, there are many literature reported on the conditional monitoring for multi-mode process which consider to the monitoring and fault diagnosis in many industrial processes (Chiang et al. (2001)). However, often caused by to the monitoring and fault diagnosis in many industrial tional monitoring for multi-mode process which consider the case of changing setting parameters. In (Hwang and processes (Chiang et al. (2001)). However, often caused by tional monitoring for multi-mode process which consider case of changing setting parameters. In (Hwang and processes (Chiang al. often by a sudden sudden mechanical mechanical breakdown, sensor failure failure orcaused malfuncprocesses (Chiang et et breakdown, al. (2001)). (2001)). However, However, oftenor caused by the the case of changing setting parameters. In (Hwang and Han (1999)), a multi-level clustering PCA method was a sensor malfuncthe case of changing setting parameters. In (Hwang and Han (1999)), a multi-level clustering PCA method was aation sudden mechanical breakdown, sensor failure or malfuncoccurred in data acquisition system, partly missing sudden mechanical breakdown, sensor failure or malfuncHan (1999)), (1999)), a multi-level multi-level clustering PCA in method was proposed to solve the multi-mode problem Tennessee tion occurred in data acquisition system, partly missing Han a clustering PCA method was proposed to solve the multi-mode problem in Tennessee tion occurred in data acquisition system, partly missing data or irregularly sampled data is a common phenomenon tion occurred in data acquisition system, partly missing proposed to solve the multi-mode problem in Tennessee Eastman (TE) process. A Gaussian mixture model and data or irregularly sampled data is a common phenomenon proposed to solve the multi-mode problem in Tennessee (TE) process. A Gaussian mixture model and data or sampled aa common in industrial industrial practice. Up to todata now,is the missing phenomenon data sample sample Eastman data or irregularly irregularly sampled data isthe common phenomenon Eastman (TE) process. A Gaussian mixture and PCA method been which can in practice. Up now, missing data Eastman (TE)have process. Acombined Gaussian together, mixture model model and PCA method have been combined together, which can in industrial practice. Up to now, the missing data sample problem is being a major challenge of most of existing in industrial practice. Up to challenge now, the missing data sample improve PCA method have been combined together, which can the efficiency of modeling the whole process (Xu problem is being a major of most of existing PCA method have been combinedthe together, which (Xu can improve the efficiency of modeling whole process problem is being aa major challenge of most of existing monitoring approaches. The process monitoring and fault problem is being major challenge of most of existing improve the efficiency efficiency of modeling modeling theproposed whole process process (Xu et al. (2010)). After that, Xu also a mixture monitoring approaches. The process monitoring and fault improve the of the whole (Xu et al. (2010)). After that, Xu also proposed a mixture monitoring approaches. The process monitoring and fault detection techniques techniques with partly missing measurements monitoring approaches. Thepartly processmissing monitoring and fault et al. (2010)). After Xu also a PCA to capture normal of production detection with measurements et al. model (2010)). After that, that, Xu distribution also proposed proposed a mixture mixture PCA model to capture normal distribution of production detection techniques with partly missing measurements have not been well studied. detection techniques with partly missing measurements PCA model to capture normal distribution of production process (Xu et al. (2011)). These methods did not conhave not been well studied. PCA model to capture normal distribution of production process (Xu et al. (2011)). These methods did not conhave not been well studied. have not been well studied. process (Xu et al. (2011)). These methods did not consider the mode transition in multi-mode process. To solve process (Xu et transition al. (2011)). These methods did not conBased on on aa Gaussian Gaussian latent latent variable variable model, model, probabilisprobabilis- sider the mode in multi-mode process. To solve Based sider the mode transition in multi-mode process. To solve this problem, Zhao has proposed a PCA-based modeling Based on a Gaussian latent variable model, probabilissider the mode transition in multi-mode process. To solve tic principal component analysis (PPCA) method, whose Based on a Gaussian latent variable model, probabilisthis problem, Zhao has proposed a PCA-based modeling tic principal principal component component analysis analysis (PPCA) (PPCA) method, method, whose whose this problem, Zhao has aa PCA-based modeling and monitoring strategy for multi-mode processes with tic problem, Zhao has proposed proposed PCA-based modeling parameters can be determined determined by(PPCA) the eigenvalue eigenvalue decomtic principalcan component analysisby method,decomwhose this and monitoring strategy for multi-mode processes with parameters be the and monitoring strategy for multi-mode processes with between-mode transitions. A mode-common subspace for parameters can be determined by the eigenvalue decomand monitoring strategy for multi-mode processes with position of the measurement sample covariance matrix or parameters can be determined by the eigenvalue decombetween-mode transitions. A mode-common subspace for position of the measurement sample covariance matrix or between-mode transitions. A mode-common subspace for all modes data is separated at first. Then, the operating position of the measurement sample covariance matrix or between-mode transitions. Aatmode-common subspace for the expectation-maximization expectation-maximization (EM) covariance algorithm (Kim (Kim and position of the measurement sample matrixand or all modes data is separated first. Then, the operating the (EM) algorithm all modes data is separated at first. Then, the operating data for each mode is projected to the mode-common the expectation-maximization (EM) algorithm (Kim and all modes data is separated at first. Then, the operating Lee (2003)), has been regarded as an efficient method the expectation-maximization (EM) algorithm (Kim and data for each mode is projected to the mode-common Lee (2003)), (2003)), has has been been regarded regarded as as an an efficient method method data for each is to mode-common subspace mode-special remaining subspace. A mode Lee for and eachaa mode mode is projected projected to the the mode-common for handling handling missing data and forming forming mixturemethod model. data Lee (2003)), missing has been regarded as an aaefficient efficient subspace and mode-special remaining subspace. A mode for data and mixture model. subspace and a mode-special remaining subspace. A mode transition identification algorithm was designed to detect for handling missing data and forming aa mixture model. subspace and a mode-special remaining subspace. A mode However, PPCA often follows the unimodal distribution for handling missing data and forming mixture model. transition identification algorithm was designed to detect However, PPCA often follows the unimodal distribution transition identification algorithm was designed to detect the abnormal behaviors (Zhao et al. (2010)). Wang idenHowever, PPCA often follows the unimodal distribution transition identification algorithm was(2010)). designed to detect assumption of the operating data, which is similar to PCA. However, PPCA often follows the unimodal distribution the abnormal behaviors (Zhao et al. Wang idenassumption of of the the operating operating data, data, which which is is similar similar to to PCA. PCA. tified the abnormal behaviors (Zhao et al. (2010)). Wang identhe stable modes, mode transitions, and noise at assumption the abnormal behaviors (Zhao et al. (2010)). Wang idenIn practice, mode transition often takes place in some assumption of the operating data, which is similar to PCA. tified the stable modes, mode transitions, and noise at In practice, practice, mode mode transition transition often often takes takes place place in in some some tified the stable modes, mode transitions, and noise at first. Then, according to the data distribution, a proper In tified the stable modes, mode transitions, and noise at of industrial processes because of different manufacturIn practice, mode transition often takes place in some first. Then, according to the data distribution, a proper of industrial processes because of different manufacturfirst. Then, according to the data distribution, a proper multivariate statistical algorithm was chosen to detect the of industrial processes because of different manufacturfirst. Then, according to the data distribution, a proper ing strategies, various product specifications, and so on. of industrial processes because of different manufacturstatistical algorithm was chosen to detect the ing strategies, strategies, various product product specifications, and and so on. on. multivariate multivariate statistical algorithm was to detect for each mode (Wang et al. (2012)). Zhang ing multivariate statistical algorithm was chosen chosen toproposed detect the thea ing strategies, various various product specifications, specifications, and so so on. fault fault for each mode (Wang et al. (2012)). Zhang proposed a fault for for each each mode (Wang et et al.multi-modes (2012)). Zhang Zhang proposed a The work is supported by National Natural Science Foundation recursive PCA to separate the by taking into fault mode (Wang al. (2012)). proposed a The work is supported by National Natural Science Foundation recursive PCA to separate the multi-modes by taking into recursive PCA to separate the multi-modes by taking into account of the cross-mode correlations and extracting the of The China (No.61374047, No.61202473) and Fundamental Research work is supported by National Natural Science Foundation recursive PCA to separate the multi-modes by taking into work is supportedNo.61202473) by National Natural Science Foundation account of the cross-mode correlations and extracting the of The China (No.61374047, and Fundamental Research account of the correlations and the common between the different modes(Zhang Funds for (No.61374047, Central Universities (JUSRP51322B, JUSRP111A49). of China No.61202473) and Research account ofinformation the cross-mode cross-mode correlations and extracting extracting the of China No.61202473) and Fundamental Fundamental Research common information between the different modes(Zhang Funds for (No.61374047, Central Universities (JUSRP51322B, JUSRP111A49). common information between the different modes(Zhang Funds for Central Universities (JUSRP51322B, JUSRP111A49). common information between the different modes(Zhang Funds for Central Universities (JUSRP51322B, JUSRP111A49). Copyright 2015 IFAC 1295Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2015, 2015 IFAC 1295 Copyright 2015 IFAC 1295 Peer review© of International Federation of Automatic Copyright ©under 2015 responsibility IFAC 1295Control. 10.1016/j.ifacol.2015.09.147
IFAC ADCHEM 2015 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
1295
et al. (2012)). And then, to improve the monitoring performance of the between-mode, Zhang extracted the common subspace form the different modes and between-mode transition which was been applied to conditional monitoring for the electro-fused magnesia furnace(Zhang and Li (2013)). And then, this method has also been generalized to detect the fault of non-gaussian processes (Zhang et al. (2013)).
Rd , (i = 1, ..., k). If a new observation yk+1 is available, then the mean value is changed to k 1 µk+1 = µk + yk+1 (3) k+1 k+1
It should be noted that all of aforementioned methods are the improvements of PCA approaches, which implies that the performance and efficiency of those approaches are still susceptible to the influence of commonly occurred interference factors such as stochastic noise and missing data (Wang and Li (2012); Xia et al. (2013); Elshenawy et al. (2009)). To overcome monitoring problems both on data missing and mode transition in multi-mode process, this paper proposed a recursive PPCA method within a probabilistic framework. The historical data are divided into serval operation modes and the mode transitions between them based on the singular value recognition. Than, the PPCA method is used to model the steady modes and the recursive PPCA method is used to model the mode transition. The missing data can also be estimated in a probabilistic framework. After that, a Mahalanobis contribution analysis based on the recursive PPCA method is proposed for fault detection and diagnosis. The efficiency of our approach is illustrated through the TE chemical process, and the result shows that the presented approach can accurately detect abnormal events and identify the faults in the multi-mode system.
then, the covariance matrix Sk+1 can be calculated recursively as
2. MODELING OF MULTI-MODE SYSTEM
Define a variable as ∆µk+1 = µk+1 − µk
Sk+1 =
(5)
k 1 T Notice that Sk = k−1 i=1 (yi − µk ) (yi − µk ), (5) can be reformulated as k−1 Sk + ∆µk+1 ∆µTk+1 Sk+1 = k (6) 1 + (yk+1 − µk − ∆µk+1 )(yk+1 − µk − ∆µk+1 )T k Equation (6) indicates that Sk+1 is two rank-one modifications from Sk . Consequently, the above update procedure has to be performed twice to estimate the eigenvalues and eigenvectors matrix for the current covariance matrix. After that, the parameters of the proposed PPCA model can be estimated by the observations and the updated values. Following the prior distribution assumptions, we obtain (7) E(xi ) = (W T W + σ 2 I)−1 W T (yi − µ) and E(xi xTi ) = σ 2 (W T W + σ 2 I)−1 + E(xi )E(xi )T
2.1 PPCA and Recursive PPCA Algorithm Assume that the latent variable model relates a ddimensional observed variable y to a q-dimensional score vector of latent variable x , which implies: y = Wx + µ +
k+1 1 (yi − µk+1 )T (yi − µk+1 ) k i=1
(4)
(1)
(8)
where E(·) is the expectation operator, i = 1...k. Then, the corresponding log-likelihood of observed data under this model is defined as k log p(yi , xi ) (9) lk = i=1
where W ∈ Rd×q a linear transformation matrix which is composed of the eigenvectors of sample covariance matrix corresponding to the q(q < d) largest eigenvalues, µ is the mean of the data y, and is the noise term which is assumed to be Gaussian and isotropic, ∼ G(0, σ 2 I). Then, the conditional probability distribution y|x is also Gaussian, which is follow y|x ∼ G(W x + µ, σ 2 I)
(2)
Furthermore, by adopting a Gaussian prior distribution for the scores variables x, x ∼ G(0, I), then, the marginal distribution of y is Gaussian in the form of y ∼ G(µ, C) , where the covariance matrix is C = W W T + σ 2 I. Therefore, the PPCA method provides a way to constrain the model complexity via the selection of q, and the model parameters can be estimated by the maximum likelihood algorithm (Qu et al. (2009); Li et al. (2013)). When the process transit from one mode to another, the recursive PPCA method can update the model parameters in a recursive way. Suppose that the historical output data can be denoted as Yk = [y1 , y2 , ..., yk ]T , where yi ∈
where the joint probability density p(yi , xi ) is given by −xi 2 p(yi , xi ) = (2πσ 2 )−d/2 × exp{ } 2 2 −yi − W xi − µ ×exp{ }(2π)−q/2 2σ 2
(10)
Take the conditional expectation with respect to the distribution in Equation (10), we obtain k d 1 { log σk2 + tr(E(xi xTi )) E(lk ) = − 2 2 i=1 1 1 + 2 yi − µk 2 − 2 E(xi )T WkT (yi − µk ) 2σk σk 1 T + 2 tr(Wk Wk E(xi xTi ))} 2σk
(11)
Maximize the conditional expectation of(11) with respect to Wk and σk2 , then the parameters are updated as k k T T Wk = (yi − µk )E(xi ) E(xi xi ) (12)
1296
i=1
i=1
IFAC ADCHEM 2015 1296 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
k
1 { yi − µk 2 − 2E(xi )T WkT (yi − µk ) kd i=1 +tr E(xi xTi )WkT Wk }
σk2 =
2.2 Missing data Imputation Approach For an incomplete measurement yk , it can be divided into the observable part yk,o and the missing part yk,u . Therefore, (9) is rewritten as k
log p(yi,o , yi,u , xi )
(14)
i=1
Mode 2
Fig. 1. Global mode structure 3.1 Mode Detection and Process Modeling It is noted that the singular values of process data can be changed with the increasing samples in between-mode H (Wang and Li (2012)). Let Y = U ΣV Σ = p , where 2 2 diag(τ1 , τ2 , ..., τp , ), and define τ = τ = Y F , i=1 i then the mode detection index at k time instant can be defined as k 1 Tk (j) = Σi (j, j) (17) L i=k−L+1
Similar to the Maximum Likelihood Estimation in prior subsection, we obtain the estimation of yk y˜k (n) = θk Wk [Nk −1 Wk T (yk − µk )] + µk
between-mode trasition
Mode 1
Let the window width of original data is L, namely, the sample yk−L+1 will be replaced by the new observation yk+1 at k + 1 instant. Thus, we can update model with new parameters of W and σ 2 .
lk =
Global mode
(13)
(15)
which presents the average of parameter τ at k time instant, where Σk (j, j) is the jth diagonal values of Σk , and Σk means the matrix Σ at k time instant as well. Then, at k + 1 time instant, we have
where Nk = Wk T Wk +σk2 I. The value of θk can be obtained by minimizing ˜ yk,o − yk,o 2 .
Based on above results, at k + 1 time instant, an EM algorithm for PPCA modeling can be summarized as follows:
Let µk+1 (0) = µk , Wk+1 (0) = Wk ,σk+1 (0) = σk ,Nk+1 (0) = 2 Wk+1 (0)T Wk+1 (0) + σk+1 (0)I, then repeat the following steps: y˜k+1 (n + 1) = θk+1 (n + 1)Wk+1 (n)[Nk+1 (n)−1 Wk+1 (n)T (yk+1 (n) − µk+1 (n))] + µk+1 (n) min { ˜ yk+1,o (n + 1) − yk+1,o 2 } θk+1 (n+1)
k 1 µk + y˜k+1 k+1 k+1 ∆µk+1 (n + 1) = µk+1 (n + 1) − µk k (16) 1 T Sk+1 (n + 1) = (˜ yi (n) − µ)(˜ yi (n) − µ) k i=1 Wk+1 (n + 1) = Sk+1 (n)Wk+1 (n)(σk+1 (n)2 I) + Nk+1 (n)T Sk+1 (n)Wk+1 (n))−1 1 σk+1 (n + 1)2 = tr(Sk+1 (n) p − Sk+1 (n)Wk+1 (n)Nk+1 (n)−1 Wk+1 (n + 1)T )
Tk+1 (j) =
1 L
k+1
Σi (j, j)
(18)
i=k−L+2
Therefore, mode transition time can be recognized by the range ability of τ 2 , as: Tk+1 (j) − Tk (j) > γj (19) Tk (j) where γj is a predefined parameter in dynamic process, which is related to mode transitions and determined by human experience (Wang and Li (2012)). 3.2 Confidence Bound and Fault Diagnosis
µk+1 (n + 1) =
where the n is the number of iterations. Above setps are repeated iteratively until convergence and then the process of parameters learning is finished.
Based on probability theory, the β% confidence bound can be defined as (Chen and Sun (2009)): p(y)dy = β (20) y:p(y)>h
where p(y) is the probability distribution of y, and the squared Mahalanobis distance based equivalent confidence bound, which can be used for fault detection and diagnosis, is shown as follow: Mk2 = (yk − µk )T Sk−1 (yk − µk ) > χ2d (β)
where β is the fractile of the χ2 distribution with the degree of freedom d, and the value can be updated as −1 2 Mk+1 = (yk+1 − µk+1 )T Sk+1 (yk+1 − µk+1 ) > χ2d (β) (22)
3. ONLINE MONITORING AND FAULT DIAGNOSIS For simplification of the expression, we assume that the whole process has two stationary modes and a betweenmode, which is shown in Fig.1. Than, Mode 1 and Mode 2 are steady modes built with regular PPCA, while the between-mode transition will be modelled by the proposed recursive PPCA method.
(21)
Therefore, the recalculated monitoring statistic is given by −1 2 E(Mk+1 ) = tr(Sk+1 {(z − µk+1 )(z − µk+1 )T + Q}) (23)
where z and Q are related as y|y0 ∼ N (z, Q), whose details can be found in Chen and Sun (2009) and 2 He et al. (22). 2 Finally, we can calculate Mk+1 − E Mk+1 for fault
1297
IFAC ADCHEM 2015 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
diagnosis with 99% confidence bound, and the confidence 2 limit is Mk+1 − χ2d (β) (Chen and Sun (2009)). As a result, the process behaviour is considered faulty if the statistic of a new observation exceeds the control limit.
On-line fault diagnosis of recursive PPCA
Model development
Acquire normal samples Y(k)
Y(k)
3.3 Modeling and Monitoring for Multi-mode Process In this part, the step-by-step procedure of recursive PPCA model based multi-mode monitoring approach is given below, and the flowchart of this method on fault detection and diagnosis is shown in Fig. 3. Firstly, the procedure of model development is given, as:
Calculate
In this section, the TE process is used to evaluate the effectiveness of the proposed fault detection and identification approach based on recursive PPCA in multi-mode system. The TE process is an open-lop unstable plant-wide process control problem considered as a benchmark simulation for various process monitoring techniques. This process produces two products (G and H) from four reactants (A, C, D and E), and F is a byproduct. In addition, an inert component B also presents in C stream and in trace amount in the A feed stream. The process consists of five major units, which include an exothermic two-phase reactor, a flash separator, a recycle compressor, a reboiled stripper, and a product condenser(Liu and Chen (2010); Yin et al. (2012)). The TE process has total 11 input variables (without agitator speed) and 41 measurement variables. And the process measurements are sampled with an interval of 3 min. For simplicity, only 22 continuous measurements, listed in Table 1, are selected in this simulation.
Calculate
Calculate and its confidence limits
Calculate contributions of each variable
Calculate the upper confidence limits for contributions
Input new samples Y(k+1)
2 new , Wnew , new
Calculate and its confidence limits
M 2new
M2
Fault detection
Beyond the confidence limits?
No
Rcalculate
Exceed confidence limits?
In addition, the online fault diagnosis of recursive PPCA approach is also given below:
4. SIMULATION STUDY
, W , 2
(1) Acquire normal operating data and normalize the data using the mean and standard deviation of each variable. (2) Calculate parameters of µ, W and σ 2 . (3) Calculate squared Mahalanobis distance M 2 and its confidence limits. (4) Calculate E M 2 and M 2 − E M 2 to estimate contribution of each variable. (5) Calculate the upper confidence limits for contributions.
(1) Obtain new observation data, and then make model detection in multi-mode process. 2 (2) Recalculate new parameters of µk+1 , Wk+1 and σk+1 . 2 (3) Recalculate new squared Mahalanobis distance Mk+1 and its confidence limits. (4) If any statistic exceeds its corresponding confidence limit, calculate the contributions of each variable after fault detection. (5) Monitor the contribution of each variable and make fault diagnosis.
1297
No
Yes End
Get the faulty variables
Fig. 2. Procedures of recursive PPCA method have to be adjusted to meet the production specifications, which can cause various operation modes. Some of them are steady state modes, while others are between-mode transitions. In the simulation, we suppose that the whole process is divided into two modes and a between-mode, which are listed on Table 2. Mode transition is introduced
Since it is difficult to produce products consistently in industrial scale, the quality control in industry relies significantly on the consistency of process conditions. Because of process changing, the operating conditions 1298
Table 1. Continuously Measured Variables Variable’s ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Description A feed (stream 1) D feed (stream 2) E feed (stream 3) A and C feed (stream 4) Recycle flow (stream 8) Reactor feed rate (stream 6) Reactor pressure Reactor level Reactor temperature Purge rate (stream 9) Product separator temperature Product separator level Product separator pressure Product separator underflow (stream 10) Stripper level Stripper pressure Stripper underflow (stream 11) Stripper temperature Stripper stream flow Compressor work Reactor cooling water outlet temperature Condenser cooling water outlet temperature
IFAC ADCHEM 2015 1298 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
by decreasing the value of reactor temperature from 120 to 110. To test the ability of proposed method, we define mode 1, mode 2 and the mode transition between as normal operating modes, and let failures occur in mode 2. Both methods are performed on the 10% randomly missing data and the number of principal components is set to 5 based on the results of a 10-fold cross validation.
140
120
100
M2
80
60
Table 2. Operating Modes of Simulated Te Process Modes Normal Operating Mode 1 Between-mode transition Normal Operating Mode 2 Faulty Operating Mode
40
Samples 1st 300th 301st 600th 601st 700th 701st 900th
20
0
The fault detection result with missing data of the proposed method is shown in Fig. 3. The M 2 calculated from the recursively updated PPCA model as well as the 99% confidence. This figure shows the values of statistics is sharply increasing from 702th time instant,and it is outside the M 2 limit at 703th time instant, which implies that fault has been detected in mode 2 for the time of the 703th sample. For all other time instants from 1 to 700, statistics are within their confidence limits, which means the false alarm rate is significantly eliminated. 140
120
100
0
100
200
300
400 500 Sample Number
600
700
800
900
Fig. 4. Monitoring results by conventional PPCA Furthermore, the diagnosis results are shown in Fig. 5 and Fig. 6, which indicate that the recursive PPCA based and the conventional PPCA based multi-mode monitoring and fault diagnosis have different contribution results. It can be seen that the contributions of the 2nd and 3rd variable are much higher than those of other variables in Fig. 5, which means that the proposed method can isolate fault variables efficiently. However, it is very difficult to determine which variables are the dominant sources of fault in Fig. 6, since 7 variables have exceeded control limits. As a result, the contribution plots shows that the proposed method is capable of isolating variables as few as possible, but not essential, faulty variables to further fault diagnosis. 7
M2
80 6
60 5
Contribution
40
20
0
4
3
2
0
100
200
300
400 500 Sample Number
600
700
800
900 1
0 0
Fig. 3. Monitoring results by recursive PPCA
5
10
15
20
Variable
Based on the measurements of mode 1, the simulation results of PPCA with a 10% randomly missing rate is shown in Fig. 4. In the plot, the regular PPCA based calculated M 2 frequently exceed their respective confidence limits in the mode transition and the steady part of mode 2 where the process is operating normally. The similar results can be obtained if only the measurements of mode 2 are used for regular PPCA. Clearly, both regular PPCA and recursive PPCA can detect the fault correctly at the 703th sample. This may be explained by the fact that mode 2 is a stationary process which is modeling by regular PPCA method in multi-mode system, and only transition process is modeling by recursive PPCA. However, the false alarm rate of conventional PPCA is much higher than the false alarm rate of the proposed method, which means the recursive PPCA method is more effective to multi-mode system.
Fig. 5. Recursive PPCA based contribution plots 5. CONCLUSION In this study, a recursive PPCA model based process monitoring and fault identification approach is developed for the multi-mode processes with randomly missing data. Different from the conventional PPCA, the proposed method recursively obtain the estimation of the missing measurement data and update model parameters. Based on the model detection index, a global mode with steady-state modes and between-mode transitions is thus developed. The illustration results demonstrate that the proposed approach can effectively detect and identify fault with missing data for multi-mode process, and it will not trigger false alarms in between-mode transition.
1299
IFAC ADCHEM 2015 June 7-10, 2015. Whistler, BC, Canada Zhengdao Zhang et al. / IFAC-PapersOnLine 48-8 (2015) 1294–1299
6
5
Contribution
4
3
2
1
0
0
5
10
15
20
Variable
Fig. 6. Conventional PPCA based contribution plots REFERENCES Chen, T. and Sun, Y. (2009). Probabilistic contribution analysis for statistical process monitoring: a missing variable approach. Control Engineering Practice, 17, 469–477. Chiang, L., Braatz, R., and Russell, E. (2001). Fault detection and diagnosis in industrial systems. Springer Verlag, London. Elshenawy, L., Yin, S., Naik, A., and Ding, S. (2009). Efficient recursive principal component analysis algorithms for process monitoring. Industrial & Engineering Chemistry Research, 49, 252–259. He, B., Yang, X., Chen, T., and Zhang, J. (22). Reconstruction-based multivariate contribution analysis for fault isolation: A branch and bound approach. Journal of Process Control, 2012, 1228–1236. Hwang, D. and Han, C. (1999). Real-time monitoring for a process with multiple operating modes. Control Engineering Practice, 7, 891–902. Kim, D. and Lee, I. (2003). Process monitoring based on probabilistic pca. Chemometrics and Intelligent Laboratory Systems, 67, 109–123. Li, L., Li, Y., and Li, Z. (2013). Efficient missing data imputing for traffic flow by considering temporal and spatial dependence. Transportation Rearch Part C, 34, 108–120. Liu, J. and Chen, D. (2010). Operational performance assessment and fault isolation for multimode processes. Industrial & Engineering Chemistry Research, 49, 3700– 3714. Qu, L., Zhang, Y., and Hu, J. (2009). Ppca-based missing data imputation for traffic flow volume: a systematical approach. IEEE Transaction on Intelligent Transportation Systems, 10, 512–522. Wang, F., Tan, S., Peng, J., and Chang, Y. (2012). Process monitoring based on mode identification for multi-mode process with transitions. Chemometrics and Intelligent Laboratory Systems, 110, 144–155. Wang, Q. and Li, H. (2012). A process singular value recognition based recursive pca approach to modetransition process monitoring. Journal of Chemical Industry and Engineering, 63, 2948–2952. Xia, L., Chu, J., and Geng, Z. (2013). Process monitoring based on improved recursive pca methods by adaptive extracting principal components. Transactions of the Institute of Measurement and Control, 35, 1024–1045.
1299
Xu, X., Xie, L., and Wang, S. (2010). A multi-mode process monitoring method based gaussian mixture models. Computers and Applied Chemistry, 27, 17–22. Xu, X., Xie, L., and Wang, S. (2011). Multi-mode process monitoring method based on pca mixture model. Journal of Chemical Industry and Engineering, 62, 743–752. Yin, S., Ding, S.X., Haghani, A., Hao, H., and Zhang, P. (2012). A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark tennessee eastman process. Journal of Process Control, 2, 1567–1581. Zhang, Y., An, J., and Ma, C. (2013). Fault detection of non-gaussian processes based on model migration. IEEE Transactions on Control Systems Technology, 21, 1517– 1525. Zhang, Y., Chai, T., Li, Z., and Yang, C. (2012). Modeling and monitoring of dynamic processes. IEEE Transactions on Neural Networks and Learning Systems, 23, 277–284. Zhang, Y. and Li, S. (2013). Modeling and monitoring between-mode transition of multimodes processes. IEEE Transactions on Industrial Informatics, 9, 2248–2255. Zhao, C., Yao, Y., Gao, F., and Wang, F. (2010). Statistical analysis and online monitoring for multimode processes with bbetween-mode transitions. Chemical Engineering Science, 65, 5961–5975.
1300