- Email: [email protected]
REGRESSION-BASED VARIABLE RECONSTRUCTION IN MULTIVARIATE SYSTEMS
REGRESSION-BASED VARIABLE RECONSTRUCTION IN MULTIVARIATE SYSTEMS Dirk Lieftucht
Uwe Kruger 1
George W. Irwin
Intelligent Systems and Control Research Group, Queen’s University Belfast, BT9 5AH, U.K.
Abstract: This paper analyses the potential of multivariate statistical process control in identifying and isolating process fault conditions. The analysis reveals that existing work suffers from inherent limitations if complex fault scenarios arise. Based on the assumption that the fault signature is deterministic while the monitored variables are stochastic in nature, a new regression-based fault reconstruction technique is introduced here to overcome these limitations. Copyright 2006 IFAC Keywords: Statistical process control, multivariate systems, fault identification, fault isolation, radial base function networks.
1. INTRODUCTION
related variables. The diagnosis, i.e. the isolation and identification of potential fault conditions responsible for this behaviour, includes the use of contribution charts (Miller et al., 1998) and variable reconstruction (Dunia and Qin, 1998).
Effective process monitoring must address the demand for safe, reliable and economic operation of industrial plants. This requirement, and particularly the detection and diagnosis of abnormal behaviour, has led to the evolution of a range of statistically based condition monitoring strategies, collectively referred to as multivariate statistical process control (MSPC) (MacGregor et al., 1991). Assuming steady-state process operation, MSPC commonly employs a linear principal component analysis (PCA) model for process monitoring. PCA decomposes the variable space into a model plane (describing significant variation of the recorded process variables) and a residual subspace (representing the mismatch between these variables and their PCA prediction).
This paper examines the usefulness of contribution charts and variable reconstruction if complex fault conditions are encountered. This investigation highlights that contribution charts can identify “broken relationships” between the recorded variables but may not produce a correct picture of the individual variables affected. Moreover, although variable reconstruction may offer a clearer picture of the contribution of individual variables to such behaviour, it suffers from the following limitations: (i) the maximum number of variables that can be reconstructed is equal to the number of retained principal components (PCs), (ii) each reconstructed variable reduces the dimension of the residual subspace and (iii) only linear fault paths can be reconstructed.
The detection of abnormal plant behaviour usually relies on univariate monitoring statistics or scatter diagrams (MacGregor et al., 1991), which are based on a reduced set of statistically uncor-
To address these practically very important problems, the paper develops a new approach for identifying and isolating complex fault signatures.
1
Corresponding author: Email: [email protected], Tel: +44(0)2890974059, Fax: +44(0)2890667023.
402
and scaled observations. The mismatch error, e(k), between the measured and predicted sensor readings is given by: £ ¤ e(k) = z(k) − Pt(k) = IN − PPT z(k). (2)
This technique is based on the work in (Nelson et al., 1996), which discussed the recovering of missing data in multivariate data sets. More precisely, the paper summarises a total of 3 different techniques to handle missing data. A detailed analysis in (Arteaga and Ferrer, 2002) showed that these methods can be characterised into two projectionbased and a third regression-based technique. Since conventional variable reconstruction originates from projection-based techniques, the new fault diagnosis method relates to the regressionbased technique.
Using Equations (1) and (2), the T2 and Q monitoring statistics can be defined as follows: T 2 (k) = tT (k)Λt(k),
with Λ being a diagonal matrix of the first n largest eigenvalues of SZZ stored in descending order. PCA projects the measured data points onto a model plane and a residual subspace. The model plane is spanned by the first n dominant eigenvectors of SZZ and describes the linear relationships between the variables. In contrast, the residual subspace is spanned by the remaining (N − n) eigenvectors of SZZ and represents the mismatch error of the PCA model prediction. Each of the above statistics can be plotted against time and the confidence limits of both statistics are calculated as shown in (Jackson, 1991).
The development of the new technique relies on the following assumptions: (i) any fault signature is deterministic, (ii) the fault signature is assumed to be superimposed onto the recorded process variables and (iii) the recorded variables follow a multivariate normal distribution. The new technique is based on estimating the fault signature that is manifested in the PCA score variables if a fault condition is detected. Next, the estimated fault signature is removed from the score variables, thus allowing separation of normal and abnormal variation from these and hence, the original process variables.
The variation of individual score variables can be presented by scatter plots (MacGregor et al., 1991; Kresta et al., 1991). These are bivariate plots in which the values of two score variables are plotted against each other. This is important for process monitoring, as the first few score variables represent most of the process variation. Therefore, if these variables indicate an out-of-statisticalcontrol situation, the process exhibits abnormally large variation that may require operator intervention. Confidence regions for scatter diagrams can be obtained as discussed by (Jackson, 1991).
The new approach utilises a Radial Basis Function Network (RBFN) (Moody and Darken, 1989; Renals and Rohwer, 1989; Yao and Zafiriou, 1990), which aims to describe a fault signature of each score variable using a least squares approach. Through a simulation example, the paper shows that the new method does not suffer from the limitations of contribution charts and conventional variable reconstruction, and can successfully isolate the fault signature from the score and the recorded process variables.
2.2 Fault isolation and identification The isolation and identification of the kind, location and size of fault conditions is typically carried out using contribution charts (Russell et al., 2000), which reveal broken relationships between the process variables. Applications of these charts may be found in (Martin and Morris, 1996; Wise and Gallagher, 1996; Kourti et al., 1996). Variable reconstruction, a competitive technique to contribution charts, was proposed in (Dunia and Qin, 1998). For sensor faults however, (Lieftucht et al., 2004) showed that complete fault isolation can only be achieved if the influence of a reconstructed sensor is incorporated into the covariance matrix. The reconstruction of faulty variables is ˆ, the prediction of z, using the PCA based on z model (Dunia and Qin, 1998):
2. PROCESS MONITORING PRELIMINARIES This section briefly reviews fault detection, isolation and identification using PCA.
2.1 Fault detection PCA utilises univariate monitoring statistics based on the determination of score variables: t(k) = PT z(k),
Q(k) = eT (k)e(k), (3)
(1)
where t(k) ∈ Rn is the vector of n retained PCs, z ∈ RN is the vector of N recorded process variables and P ∈ RN ×n is the loading matrix containing the first n < N dominant eigenvectors of the covariance matrix SZZ as col1 ZT Z ∈ umn vectors. The matrix SZZ = K−1 N ×N R £is calculated using ¤a reference data set ZT = z(1) z(2) . . . z(K) of K mean-centred
ˆ z(k) = Pt(k) = PPT z(k) = Cz(k),
(4)
where C = PPT . More precisely, the prediction e of the reconstructed vector e z(k), b z(k) is given by: b eP eTe e z(k) = P z(k),
403
(5)
4. REGRESSION-BASED VARIABLE RECONSTRUCTION
e represents the reconstructed loading where P matrix (Lieftucht et al., 2004).
This section introduces a new regression-based variable reconstruction technique that is based on the following assumptions:
3. PROBLEMS WITH CONVENTIONAL FAULT ISOLATION AND IDENTIFICATION
• The fault signatures are deterministic, that is the signature for a particular process variable is a function of the sample index k; • The fault is superimposed onto the process variables, i.e. it is superimposed to the PCA T M score variables: M C t(k) = C P z(k), C t(k) = T t(k) + f (k), where the subscript C refers C to the complete set of score variables, i.e. the N retained and discarded ones, and the superscripts M and T refer to “measured” (including fault) and “true” (without fault); • The recorded process variables are described by a multivariate normal distribution under normal operation.
This section shows potential problems with contribution charts and variable reconstruction in diagnosing complex fault conditions.
3.1 Problems with contribution charts. Problem 1. Contribution charts highlight broken relationships between the recorded variables. However, if a substantial number of variables are affected by a complex fault condition, contribution charts may produce a confusing picture that does not allow a detailed analysis into the nature of the detected fault condition. Examples are given in (Kruger and Sandoz, 2001; Lieftucht et al., 2006).
It should be noted that the third assumption forms the basis for PCA based process monitoring (Jackson, 1991). Based on the above assumptions, the fault signature can be described by a parametric curve that can be identified using various techniques, e.g. polynomials, principal curves, artificial neural networks etc. (Walczak and Massart, 1996). For simplicity, a radial basis function network (RBFN) is utilised in this paper.
3.2 Problems with variable reconstruction. Problem 2. The maximum number of variables to be reconstructed equals the number of retained PCs (Dunia and Qin, 1998). This limitation renders the technique difficult to apply for diagnosing process faults, which typically affect a larger number of variables;
The new technique estimates the fault signature f (k) and separates this signature from the computed score sequences M C t(k). It should be noted that the score variables are statistically independent, whereas the original process variables are correlated. Hence, the affect of the fault on one process variable may also manifest itself on other process variables. Consequently, separating the fault signature from the “true” process variation on the basis of the score variables circumvents the correlation issue.
Problem 3. Each reconstructed variable reduces the dimensions of the residual subspace. Consequently, if the number of reconstructed variables is equal to, or exceeds, the dimension of this space the Q statistic, which is based on the PCA residuals, is equal to zero. In such a situation, projection-based variable reconstruction may, in fact, lead to the Q statistic becoming zero, preventing an analysis of the effect of the reconstruction on the recorded variables; and
This separation is graphically illustrated in Figure 1 below. The first block produces the complete z(k) CP
Mt(k) C
J(k) RBFN
Tt(k)
f(k) a
C
+
Problem 4. The variable reconstruction technique relies on the linear relationships among the recorded process variables, described in the loading matrix, and therefore only faults that can be described by linear relationships between the reconstructed variables are fully reconstructible.
Fig. 1. Structure of the regression-based variable reconstruction technique. set of N score variables M C t(k) from the recorded process variables z(k). The score variable set is then used to model the fault signature f (k) which is subtracted from the computed score variables to produce TC t(k).
The impact of these problems is demonstrated in Section 5. To overcome the above limitations, this paper develops a new technique described in Section 4, which is then compared to the conventional techniques in Section 5.
The output from a RBFN network can be formulated as follows:
404
R X
f1 (k) =
L1
i=1
fN (k) =
R X
LN
¡M
C t1 (k), ci , r
.. . ¡M
¢
C tN (k), ci , r
a1i 40
, ¢
30
(6)
20 10
aN i
100
200
300
400
500
600
700
800
900
1000
100
200
300
400
500
600
700
800
900
1000
i=1 20
where R represents the number of network nodes, t (k)−c Lj = exp(−( j r i )2 ); 1 ≤ j ≤ N is the Gaussian basis function with the centres ci and the radius r and a1i , ..., aN R are the network weights. The separation of the fault signature can now be expressed as follows:
15 10 5 0
Data Point
15
M C t(k)
= J(k)a + TC t(k) = f (k) + TC t(k).
(7)
10
Here, J is a matrix storing the values for each network node for the kth sample, TC t(k) includes the isolated stochastic part of the measured score variables, which the RBFN is not able to model and a the vector of identified network parameters.
5
0
−5
−10
−15
5. SIMULATION EXAMPLE
−20
z1 z2 z3 z4 100
This section summarises the application of contribution charts, conventional variable reconstruction and the new technique to an example. This involves a total of 4 process variables constructed as follows. First, 2000 samples from two independent sequences x1 , x2 ∈ N {0, 1} were generated. The 4 process variables, z, were then constructed ¡ ¢T as linear combinations of x = x1 x2 , which were superimposed by identically and independently distributed noise sequences of N {0, 0.0025}, producing e, to represent measurement noise: z = Bx + e.
300
400
500
Data Point
600
700
800
900
1000
Fig. 2. Monitoring statistics and contribution charts describing fault condition of the synthetic example process tive to the fault, although the T2 statistic also showed an increase in the number of violations of its 99% confidence limit, indicating abnormal process behaviour. To identify the root cause of this behaviour the contribution chart of the Q statistic is shown in the third plot of Figure 2. The 1st process variable shows the most significant contribution to this event, followed by variables 2, 3 and 4. Consequently, the contribution chart could not be relied upon to correctly diagnose that variables 2, 3 and 4 are affected by the simulated fault condition.
(8)
Here, B is a parameter matrix representing the linear combinations. The first 1000 samples were used to identify a PCA model, for which the first 2 PCs were retained. The second data set was augmented by adding the following fault sequences to the last 600 samples of variables 2, 3 and 4: 0.004958q + 0.004916 q 2 − 1.973q + 0.9753 0.001608q + 0.01555 . f3 = 2 q − 1.873q + 0.9048 0.007373q + 0.007252 f4 = 2 q − 1.946q + 0.9512
200
Next, conventional projection-based variable reconstruction is used to diagnose this fault condition. To compare the impact of variable reconstruction upon the score variables, the scatter diagrams of the score variables for the fault condition are shown in Figure 3.
f2 =
Each process variable was reconstructed in turn and the one contributing most to the T2 and Q monitoring statistics (Variable 2) was selected. Variable 2 and each of the remaining variables, 1, 3 and 4, were then reconstructed in turn, which suggested that variables 2 and 4 were the most dominant contributors to this event. After the reconstruction of these variables, the residual subspace was of dimension zero and consequently no Q statistic could be established. Furthermore,
(9)
Here fi denotes the fault added to the ith process variable and q is the backshift operator. The two upper plots in Figure 2 show the univariate T2 and Q statistics generated for the second data set. The Q statistic was particularly sensi-
405
5
5
5
−5
Score 1
Score 1
12
0
0 −5
0
10 8
−5
6
−5
−4
−3
−2
Score 2
−1
0
−1
−0.5
0
Score 3
0.5
0
1
2
Score 4
3
4
0 −1
Score 2
Score 2
2
0 −1 −2 −3 −4 −5
100
200
300
400
100
200
300
400
500
600
700
800
900
1000
500
600
700
800
900
1000
−2 −3 −4
−1
−0.5
0
Score 3
−5
0.5
0
1
2
Score 4
0.6
3
0.5 0.4
Score 3
0.5
0.3
0
0.2
−0.5
0.1
−1 0
1
2
Score 4
3
Data Point
Fig. 3. Scatter diagrams of the fault. since 2 PCs were retained in the PCA model, only two variables could be reconstructed. The scatter diagrams after reconstruction are shown in Figure 4. These indicate that the fault could not be isolated completely from the recorded variables since a substantial number of violations of the 99% confidence regions remain. Consequently, conventional projection-based variable reconstruction also did not offer a correct diagnosis of this event.
5 4 3 2 1 0 −1 1000 800 600 400 200
5
Score 1
0 −5
−6
−4
−2
Score 2
0
0 −5
2
−0.5
0
0.5
Score 3
Variable
0
Fig. 5. Monitoring statistics and reconstruction charts describing fault condition of the synthetic example process after regression-based variable reconstruction
−5
1
2
−1.5
−1
−0.5
0
0.5
1
−1.5
−1
−0.5
0
0.5
1
−1.5
−1
−0.5
0
0.5
1
Score 4
2
0
Score 2
Score 2
2
5
Score 1
5
1.5
1
Data Point
−2 −4 −6
0 −2
It should be noted that although the first variable is not affected by the fault condition, the fault might be felt in each of the score variables since M C t = C Pz.
−4 −6
−0.5
0
0.5
Score 3
1
Score 4
1
Score 3
4
3.5
3
2.5
0.5 0 −0.5
5
0
Fig. 4. Scatter diagrams of the fault condition after reconstruction of variable 2 and 4.
−5
5
Score 1
5
Score 1
Score 4
0
−5 −4
−2
Score 2
0
2
−5 −0.5
0
0.5
Score 3
1
1.5
−0.5
0
−0.5
0
−0.5
0
0.5
1
1.5
2
0.5
1
1.5
2
0.5
1
1.5
2
Score 4
2
0
Score 2
Score 2
2
Finally, the application of the new regression based method is summarised in Figure 5. The two upper plots in Figure 5 show the monitoring statistics following the regression-based variable reconstruction process.
0
−2 −4
0 −2 −4
−0.5
0
0.5
Score 3
1
1.5
Score 4
Score 3
1.5
This figure shows that based on reconstructing each score variable both statistics are within their limits. The lower part of Figure 5 correctly shows that variables 2, 3 and 4 are affected and that variable 1 did not correspond to this simulated event. The path of this fault scenario can be investigated by plotting the fault sequences for score variables 1 to 4 on scatter diagrams, as shown in Figure 6.
1 0.5 0 −0.5
Score 4
Fig. 6. Scatter diagrams of the fault isolated by the regressive variable reconstruction method.
406
6. CONCLUSIONS
Kruger, U. and D. J. Sandoz (2001). Applications of pca for the detection and diagnosis of anomalous process behaviour. In: Proceedings of the IFAC conference on new technologies for computer control. Hong Kong, P.R. China. pp. 413 – 418. Lieftucht, D., U. Kruger and G. W. Irwin (2006). Improved diagnosis of abnormal process behaviour using multivariate statistics. Computers & Chemical Eng. 30(5), 901–912. Lieftucht, D., U. Kruger and G.W. Irwin (2004). Improved diagnosis of sensor faults using multivariate statistics. ACC pp. 4403–4407. MacGregor, J. F., T. E. Marlin, J. V. Kresta and B. Skagerberg (1991). Multivariate statistical methods in process analysis and control. In: Proc. of the 4th Int. Conf. on Chemical Process Control. AIChE Publication, No. P-67. New York. pp. 79–99. Martin, E. B. and A. J. Morris (1996). An overview of multivariate statistical process control in continuous and batch process performance monitoring. Trans. of the Inst. of Measurement and Control 18(1), 51–60. Miller, P., R. E. Swanson and C. F. Heckler (1998). Contribution plots: A missing link in multivariate quality control. Applied Mathematics and Computer Science 8(4), 775–792. Moody, J. and C. J. Darken (1989). Fast learning in networks of locally-tuned processing units. Neural Computation 1, 151–160. Nelson, P. R. C., P. A. Taylor and J. F. MacGregor (1996). Missing data methods in pca and pls: Score calculations with incomplete observations. Chem. and Int. Lab. Syst. 35(1), 45– 65. Renals, S. and R. Rohwer (1989). Phoneme classification experiments using radial basis functions. Vol. 461-467. Proc. Int. Joint Conference Neural Networks. Washington, DC. Russell, E. L., L. H. Chiang and R. D. Braatz (2000). Data-Driven Techniques for Fault Detection and Diagnosis in Chemical Processes. Adv. in Ind. Control. Springer. London. Walczak, B. and D. L. Massart (1996). The radial basis functions - partial least squares approach as a flexible non-linear regression technique. Analytica Chimica Acta 331, 177– 185. Wise, B. M. and N. B. Gallagher (1996). The process chemometrics approach to process monitoring and fault detection. JPC 6(6), 329– 348. Yao, S. C. and E. Zafiriou (1990). Control system sensor failure detection via networks of localized receptive fields. ACC.
This paper has studied the identification and isolation of abnormal events in multivariate processes using contribution charts and projectionbased variable reconstruction. This study showed that these PCA based techniques inherently suffer from the following limitations if complex process faults are to be diagnosed: (i) contribution charts may produce misleading pictures that do not allow a clear diagnosis of the root cause of such events; (ii) the maximum number of variables that can be reconstructed equals to number of retained PCs; (iii) the reconstruction process reduces the dimension of the residual subspace, i.e. its dimension may be equal to zero after reconstruction; and (iv) variable reconstruction relies on linear relationships between the analysed variables, i.e. faults that produce nonlinear relationships between these variables may not be isolated from the recorded variables. To address these deficiencies, the paper proposes a regression-based variable reconstruction technique under the assumption that the fault signature is deterministic, whilst the analysed process variables are stochastic in nature. This assumption allows describing the fault signature by a parametric model to be identified. For simplicity, the work presented relies on the use of RBF networks to model the deterministic fault condition, which is then removed from the data. Hence, this separation produces an in-statistical-control situation for the associated monitoring charts without the presence of the fault condition.
REFERENCES Arteaga, F. and A. Ferrer (2002). Dealing with missing data in mspc: several methods, different interpretations, some examples. J. of Chemometrics 16, 408–418. Dunia, R. and S. J. Qin (1998). A unified geometric approach to process and sensor fault identification and reconstruction: the unidimensional fault case. Computers & Chemical Eng. 22(7-8), 927–943. Jackson, J. E. (1991). A Users Guide to Principal Components. Wiley Series in Probability and Mathematical Statistics. John Wiley. New York. Kourti, T., J. Lee and J. F. MacGregor (1996). Experiences with industrial applications of projection methods for multivariate statistical process control. Computers & Chemical Eng. 20(971), 745–750. Kresta, J. V., J. F. MacGregor and T. E. Marlin (1991). Multivariate statistical monitoring of process operating performance. Can. J. of Chem. Eng. 69, 35–47.
407
Uwe Kruger 1
George W. Irwin
Intelligent Systems and Control Research Group, Queen’s University Belfast, BT9 5AH, U.K.
Abstract: This paper analyses the potential of multivariate statistical process control in identifying and isolating process fault conditions. The analysis reveals that existing work suffers from inherent limitations if complex fault scenarios arise. Based on the assumption that the fault signature is deterministic while the monitored variables are stochastic in nature, a new regression-based fault reconstruction technique is introduced here to overcome these limitations. Copyright 2006 IFAC Keywords: Statistical process control, multivariate systems, fault identification, fault isolation, radial base function networks.
1. INTRODUCTION
related variables. The diagnosis, i.e. the isolation and identification of potential fault conditions responsible for this behaviour, includes the use of contribution charts (Miller et al., 1998) and variable reconstruction (Dunia and Qin, 1998).
Effective process monitoring must address the demand for safe, reliable and economic operation of industrial plants. This requirement, and particularly the detection and diagnosis of abnormal behaviour, has led to the evolution of a range of statistically based condition monitoring strategies, collectively referred to as multivariate statistical process control (MSPC) (MacGregor et al., 1991). Assuming steady-state process operation, MSPC commonly employs a linear principal component analysis (PCA) model for process monitoring. PCA decomposes the variable space into a model plane (describing significant variation of the recorded process variables) and a residual subspace (representing the mismatch between these variables and their PCA prediction).
This paper examines the usefulness of contribution charts and variable reconstruction if complex fault conditions are encountered. This investigation highlights that contribution charts can identify “broken relationships” between the recorded variables but may not produce a correct picture of the individual variables affected. Moreover, although variable reconstruction may offer a clearer picture of the contribution of individual variables to such behaviour, it suffers from the following limitations: (i) the maximum number of variables that can be reconstructed is equal to the number of retained principal components (PCs), (ii) each reconstructed variable reduces the dimension of the residual subspace and (iii) only linear fault paths can be reconstructed.
The detection of abnormal plant behaviour usually relies on univariate monitoring statistics or scatter diagrams (MacGregor et al., 1991), which are based on a reduced set of statistically uncor-
To address these practically very important problems, the paper develops a new approach for identifying and isolating complex fault signatures.
1
Corresponding author: Email: [email protected], Tel: +44(0)2890974059, Fax: +44(0)2890667023.
402
and scaled observations. The mismatch error, e(k), between the measured and predicted sensor readings is given by: £ ¤ e(k) = z(k) − Pt(k) = IN − PPT z(k). (2)
This technique is based on the work in (Nelson et al., 1996), which discussed the recovering of missing data in multivariate data sets. More precisely, the paper summarises a total of 3 different techniques to handle missing data. A detailed analysis in (Arteaga and Ferrer, 2002) showed that these methods can be characterised into two projectionbased and a third regression-based technique. Since conventional variable reconstruction originates from projection-based techniques, the new fault diagnosis method relates to the regressionbased technique.
Using Equations (1) and (2), the T2 and Q monitoring statistics can be defined as follows: T 2 (k) = tT (k)Λt(k),
with Λ being a diagonal matrix of the first n largest eigenvalues of SZZ stored in descending order. PCA projects the measured data points onto a model plane and a residual subspace. The model plane is spanned by the first n dominant eigenvectors of SZZ and describes the linear relationships between the variables. In contrast, the residual subspace is spanned by the remaining (N − n) eigenvectors of SZZ and represents the mismatch error of the PCA model prediction. Each of the above statistics can be plotted against time and the confidence limits of both statistics are calculated as shown in (Jackson, 1991).
The development of the new technique relies on the following assumptions: (i) any fault signature is deterministic, (ii) the fault signature is assumed to be superimposed onto the recorded process variables and (iii) the recorded variables follow a multivariate normal distribution. The new technique is based on estimating the fault signature that is manifested in the PCA score variables if a fault condition is detected. Next, the estimated fault signature is removed from the score variables, thus allowing separation of normal and abnormal variation from these and hence, the original process variables.
The variation of individual score variables can be presented by scatter plots (MacGregor et al., 1991; Kresta et al., 1991). These are bivariate plots in which the values of two score variables are plotted against each other. This is important for process monitoring, as the first few score variables represent most of the process variation. Therefore, if these variables indicate an out-of-statisticalcontrol situation, the process exhibits abnormally large variation that may require operator intervention. Confidence regions for scatter diagrams can be obtained as discussed by (Jackson, 1991).
The new approach utilises a Radial Basis Function Network (RBFN) (Moody and Darken, 1989; Renals and Rohwer, 1989; Yao and Zafiriou, 1990), which aims to describe a fault signature of each score variable using a least squares approach. Through a simulation example, the paper shows that the new method does not suffer from the limitations of contribution charts and conventional variable reconstruction, and can successfully isolate the fault signature from the score and the recorded process variables.
2.2 Fault isolation and identification The isolation and identification of the kind, location and size of fault conditions is typically carried out using contribution charts (Russell et al., 2000), which reveal broken relationships between the process variables. Applications of these charts may be found in (Martin and Morris, 1996; Wise and Gallagher, 1996; Kourti et al., 1996). Variable reconstruction, a competitive technique to contribution charts, was proposed in (Dunia and Qin, 1998). For sensor faults however, (Lieftucht et al., 2004) showed that complete fault isolation can only be achieved if the influence of a reconstructed sensor is incorporated into the covariance matrix. The reconstruction of faulty variables is ˆ, the prediction of z, using the PCA based on z model (Dunia and Qin, 1998):
2. PROCESS MONITORING PRELIMINARIES This section briefly reviews fault detection, isolation and identification using PCA.
2.1 Fault detection PCA utilises univariate monitoring statistics based on the determination of score variables: t(k) = PT z(k),
Q(k) = eT (k)e(k), (3)
(1)
where t(k) ∈ Rn is the vector of n retained PCs, z ∈ RN is the vector of N recorded process variables and P ∈ RN ×n is the loading matrix containing the first n < N dominant eigenvectors of the covariance matrix SZZ as col1 ZT Z ∈ umn vectors. The matrix SZZ = K−1 N ×N R £is calculated using ¤a reference data set ZT = z(1) z(2) . . . z(K) of K mean-centred
ˆ z(k) = Pt(k) = PPT z(k) = Cz(k),
(4)
where C = PPT . More precisely, the prediction e of the reconstructed vector e z(k), b z(k) is given by: b eP eTe e z(k) = P z(k),
403
(5)
4. REGRESSION-BASED VARIABLE RECONSTRUCTION
e represents the reconstructed loading where P matrix (Lieftucht et al., 2004).
This section introduces a new regression-based variable reconstruction technique that is based on the following assumptions:
3. PROBLEMS WITH CONVENTIONAL FAULT ISOLATION AND IDENTIFICATION
• The fault signatures are deterministic, that is the signature for a particular process variable is a function of the sample index k; • The fault is superimposed onto the process variables, i.e. it is superimposed to the PCA T M score variables: M C t(k) = C P z(k), C t(k) = T t(k) + f (k), where the subscript C refers C to the complete set of score variables, i.e. the N retained and discarded ones, and the superscripts M and T refer to “measured” (including fault) and “true” (without fault); • The recorded process variables are described by a multivariate normal distribution under normal operation.
This section shows potential problems with contribution charts and variable reconstruction in diagnosing complex fault conditions.
3.1 Problems with contribution charts. Problem 1. Contribution charts highlight broken relationships between the recorded variables. However, if a substantial number of variables are affected by a complex fault condition, contribution charts may produce a confusing picture that does not allow a detailed analysis into the nature of the detected fault condition. Examples are given in (Kruger and Sandoz, 2001; Lieftucht et al., 2006).
It should be noted that the third assumption forms the basis for PCA based process monitoring (Jackson, 1991). Based on the above assumptions, the fault signature can be described by a parametric curve that can be identified using various techniques, e.g. polynomials, principal curves, artificial neural networks etc. (Walczak and Massart, 1996). For simplicity, a radial basis function network (RBFN) is utilised in this paper.
3.2 Problems with variable reconstruction. Problem 2. The maximum number of variables to be reconstructed equals the number of retained PCs (Dunia and Qin, 1998). This limitation renders the technique difficult to apply for diagnosing process faults, which typically affect a larger number of variables;
The new technique estimates the fault signature f (k) and separates this signature from the computed score sequences M C t(k). It should be noted that the score variables are statistically independent, whereas the original process variables are correlated. Hence, the affect of the fault on one process variable may also manifest itself on other process variables. Consequently, separating the fault signature from the “true” process variation on the basis of the score variables circumvents the correlation issue.
Problem 3. Each reconstructed variable reduces the dimensions of the residual subspace. Consequently, if the number of reconstructed variables is equal to, or exceeds, the dimension of this space the Q statistic, which is based on the PCA residuals, is equal to zero. In such a situation, projection-based variable reconstruction may, in fact, lead to the Q statistic becoming zero, preventing an analysis of the effect of the reconstruction on the recorded variables; and
This separation is graphically illustrated in Figure 1 below. The first block produces the complete z(k) CP
Mt(k) C
J(k) RBFN
Tt(k)
f(k) a
C
+
Problem 4. The variable reconstruction technique relies on the linear relationships among the recorded process variables, described in the loading matrix, and therefore only faults that can be described by linear relationships between the reconstructed variables are fully reconstructible.
Fig. 1. Structure of the regression-based variable reconstruction technique. set of N score variables M C t(k) from the recorded process variables z(k). The score variable set is then used to model the fault signature f (k) which is subtracted from the computed score variables to produce TC t(k).
The impact of these problems is demonstrated in Section 5. To overcome the above limitations, this paper develops a new technique described in Section 4, which is then compared to the conventional techniques in Section 5.
The output from a RBFN network can be formulated as follows:
404
R X
f1 (k) =
L1
i=1
fN (k) =
R X
LN
¡M
C t1 (k), ci , r
.. . ¡M
¢
C tN (k), ci , r
a1i 40
, ¢
30
(6)
20 10
aN i
100
200
300
400
500
600
700
800
900
1000
100
200
300
400
500
600
700
800
900
1000
i=1 20
where R represents the number of network nodes, t (k)−c Lj = exp(−( j r i )2 ); 1 ≤ j ≤ N is the Gaussian basis function with the centres ci and the radius r and a1i , ..., aN R are the network weights. The separation of the fault signature can now be expressed as follows:
15 10 5 0
Data Point
15
M C t(k)
= J(k)a + TC t(k) = f (k) + TC t(k).
(7)
10
Here, J is a matrix storing the values for each network node for the kth sample, TC t(k) includes the isolated stochastic part of the measured score variables, which the RBFN is not able to model and a the vector of identified network parameters.
5
0
−5
−10
−15
5. SIMULATION EXAMPLE
−20
z1 z2 z3 z4 100
This section summarises the application of contribution charts, conventional variable reconstruction and the new technique to an example. This involves a total of 4 process variables constructed as follows. First, 2000 samples from two independent sequences x1 , x2 ∈ N {0, 1} were generated. The 4 process variables, z, were then constructed ¡ ¢T as linear combinations of x = x1 x2 , which were superimposed by identically and independently distributed noise sequences of N {0, 0.0025}, producing e, to represent measurement noise: z = Bx + e.
300
400
500
Data Point
600
700
800
900
1000
Fig. 2. Monitoring statistics and contribution charts describing fault condition of the synthetic example process tive to the fault, although the T2 statistic also showed an increase in the number of violations of its 99% confidence limit, indicating abnormal process behaviour. To identify the root cause of this behaviour the contribution chart of the Q statistic is shown in the third plot of Figure 2. The 1st process variable shows the most significant contribution to this event, followed by variables 2, 3 and 4. Consequently, the contribution chart could not be relied upon to correctly diagnose that variables 2, 3 and 4 are affected by the simulated fault condition.
(8)
Here, B is a parameter matrix representing the linear combinations. The first 1000 samples were used to identify a PCA model, for which the first 2 PCs were retained. The second data set was augmented by adding the following fault sequences to the last 600 samples of variables 2, 3 and 4: 0.004958q + 0.004916 q 2 − 1.973q + 0.9753 0.001608q + 0.01555 . f3 = 2 q − 1.873q + 0.9048 0.007373q + 0.007252 f4 = 2 q − 1.946q + 0.9512
200
Next, conventional projection-based variable reconstruction is used to diagnose this fault condition. To compare the impact of variable reconstruction upon the score variables, the scatter diagrams of the score variables for the fault condition are shown in Figure 3.
f2 =
Each process variable was reconstructed in turn and the one contributing most to the T2 and Q monitoring statistics (Variable 2) was selected. Variable 2 and each of the remaining variables, 1, 3 and 4, were then reconstructed in turn, which suggested that variables 2 and 4 were the most dominant contributors to this event. After the reconstruction of these variables, the residual subspace was of dimension zero and consequently no Q statistic could be established. Furthermore,
(9)
Here fi denotes the fault added to the ith process variable and q is the backshift operator. The two upper plots in Figure 2 show the univariate T2 and Q statistics generated for the second data set. The Q statistic was particularly sensi-
405
5
5
5
−5
Score 1
Score 1
12
0
0 −5
0
10 8
−5
6
−5
−4
−3
−2
Score 2
−1
0
−1
−0.5
0
Score 3
0.5
0
1
2
Score 4
3
4
0 −1
Score 2
Score 2
2
0 −1 −2 −3 −4 −5
100
200
300
400
100
200
300
400
500
600
700
800
900
1000
500
600
700
800
900
1000
−2 −3 −4
−1
−0.5
0
Score 3
−5
0.5
0
1
2
Score 4
0.6
3
0.5 0.4
Score 3
0.5
0.3
0
0.2
−0.5
0.1
−1 0
1
2
Score 4
3
Data Point
Fig. 3. Scatter diagrams of the fault. since 2 PCs were retained in the PCA model, only two variables could be reconstructed. The scatter diagrams after reconstruction are shown in Figure 4. These indicate that the fault could not be isolated completely from the recorded variables since a substantial number of violations of the 99% confidence regions remain. Consequently, conventional projection-based variable reconstruction also did not offer a correct diagnosis of this event.
5 4 3 2 1 0 −1 1000 800 600 400 200
5
Score 1
0 −5
−6
−4
−2
Score 2
0
0 −5
2
−0.5
0
0.5
Score 3
Variable
0
Fig. 5. Monitoring statistics and reconstruction charts describing fault condition of the synthetic example process after regression-based variable reconstruction
−5
1
2
−1.5
−1
−0.5
0
0.5
1
−1.5
−1
−0.5
0
0.5
1
−1.5
−1
−0.5
0
0.5
1
Score 4
2
0
Score 2
Score 2
2
5
Score 1
5
1.5
1
Data Point
−2 −4 −6
0 −2
It should be noted that although the first variable is not affected by the fault condition, the fault might be felt in each of the score variables since M C t = C Pz.
−4 −6
−0.5
0
0.5
Score 3
1
Score 4
1
Score 3
4
3.5
3
2.5
0.5 0 −0.5
5
0
Fig. 4. Scatter diagrams of the fault condition after reconstruction of variable 2 and 4.
−5
5
Score 1
5
Score 1
Score 4
0
−5 −4
−2
Score 2
0
2
−5 −0.5
0
0.5
Score 3
1
1.5
−0.5
0
−0.5
0
−0.5
0
0.5
1
1.5
2
0.5
1
1.5
2
0.5
1
1.5
2
Score 4
2
0
Score 2
Score 2
2
Finally, the application of the new regression based method is summarised in Figure 5. The two upper plots in Figure 5 show the monitoring statistics following the regression-based variable reconstruction process.
0
−2 −4
0 −2 −4
−0.5
0
0.5
Score 3
1
1.5
Score 4
Score 3
1.5
This figure shows that based on reconstructing each score variable both statistics are within their limits. The lower part of Figure 5 correctly shows that variables 2, 3 and 4 are affected and that variable 1 did not correspond to this simulated event. The path of this fault scenario can be investigated by plotting the fault sequences for score variables 1 to 4 on scatter diagrams, as shown in Figure 6.
1 0.5 0 −0.5
Score 4
Fig. 6. Scatter diagrams of the fault isolated by the regressive variable reconstruction method.
406
6. CONCLUSIONS
Kruger, U. and D. J. Sandoz (2001). Applications of pca for the detection and diagnosis of anomalous process behaviour. In: Proceedings of the IFAC conference on new technologies for computer control. Hong Kong, P.R. China. pp. 413 – 418. Lieftucht, D., U. Kruger and G. W. Irwin (2006). Improved diagnosis of abnormal process behaviour using multivariate statistics. Computers & Chemical Eng. 30(5), 901–912. Lieftucht, D., U. Kruger and G.W. Irwin (2004). Improved diagnosis of sensor faults using multivariate statistics. ACC pp. 4403–4407. MacGregor, J. F., T. E. Marlin, J. V. Kresta and B. Skagerberg (1991). Multivariate statistical methods in process analysis and control. In: Proc. of the 4th Int. Conf. on Chemical Process Control. AIChE Publication, No. P-67. New York. pp. 79–99. Martin, E. B. and A. J. Morris (1996). An overview of multivariate statistical process control in continuous and batch process performance monitoring. Trans. of the Inst. of Measurement and Control 18(1), 51–60. Miller, P., R. E. Swanson and C. F. Heckler (1998). Contribution plots: A missing link in multivariate quality control. Applied Mathematics and Computer Science 8(4), 775–792. Moody, J. and C. J. Darken (1989). Fast learning in networks of locally-tuned processing units. Neural Computation 1, 151–160. Nelson, P. R. C., P. A. Taylor and J. F. MacGregor (1996). Missing data methods in pca and pls: Score calculations with incomplete observations. Chem. and Int. Lab. Syst. 35(1), 45– 65. Renals, S. and R. Rohwer (1989). Phoneme classification experiments using radial basis functions. Vol. 461-467. Proc. Int. Joint Conference Neural Networks. Washington, DC. Russell, E. L., L. H. Chiang and R. D. Braatz (2000). Data-Driven Techniques for Fault Detection and Diagnosis in Chemical Processes. Adv. in Ind. Control. Springer. London. Walczak, B. and D. L. Massart (1996). The radial basis functions - partial least squares approach as a flexible non-linear regression technique. Analytica Chimica Acta 331, 177– 185. Wise, B. M. and N. B. Gallagher (1996). The process chemometrics approach to process monitoring and fault detection. JPC 6(6), 329– 348. Yao, S. C. and E. Zafiriou (1990). Control system sensor failure detection via networks of localized receptive fields. ACC.
This paper has studied the identification and isolation of abnormal events in multivariate processes using contribution charts and projectionbased variable reconstruction. This study showed that these PCA based techniques inherently suffer from the following limitations if complex process faults are to be diagnosed: (i) contribution charts may produce misleading pictures that do not allow a clear diagnosis of the root cause of such events; (ii) the maximum number of variables that can be reconstructed equals to number of retained PCs; (iii) the reconstruction process reduces the dimension of the residual subspace, i.e. its dimension may be equal to zero after reconstruction; and (iv) variable reconstruction relies on linear relationships between the analysed variables, i.e. faults that produce nonlinear relationships between these variables may not be isolated from the recorded variables. To address these deficiencies, the paper proposes a regression-based variable reconstruction technique under the assumption that the fault signature is deterministic, whilst the analysed process variables are stochastic in nature. This assumption allows describing the fault signature by a parametric model to be identified. For simplicity, the work presented relies on the use of RBF networks to model the deterministic fault condition, which is then removed from the data. Hence, this separation produces an in-statistical-control situation for the associated monitoring charts without the presence of the fault condition.
REFERENCES Arteaga, F. and A. Ferrer (2002). Dealing with missing data in mspc: several methods, different interpretations, some examples. J. of Chemometrics 16, 408–418. Dunia, R. and S. J. Qin (1998). A unified geometric approach to process and sensor fault identification and reconstruction: the unidimensional fault case. Computers & Chemical Eng. 22(7-8), 927–943. Jackson, J. E. (1991). A Users Guide to Principal Components. Wiley Series in Probability and Mathematical Statistics. John Wiley. New York. Kourti, T., J. Lee and J. F. MacGregor (1996). Experiences with industrial applications of projection methods for multivariate statistical process control. Computers & Chemical Eng. 20(971), 745–750. Kresta, J. V., J. F. MacGregor and T. E. Marlin (1991). Multivariate statistical monitoring of process operating performance. Can. J. of Chem. Eng. 69, 35–47.
407