- Email: [email protected]
PLS-based EWMA fault detection strategy for process monitoring
Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
Contents lists available at ScienceDirect
Journal of Loss Prevention in the Process Industries journal homepage: www.elsevier.com/locate/jlp
PLS-based EWMA fault detection strategy for process monitoring Fouzi Harrou a, Mohamed N. Nounou a, *, Hazem N. Nounou b, Muddu Madakyaru a a b
Chemical Engineering Program, Texas A&M University at Qatar, Doha, Qatar Electrical and Computer Engineering Program, Texas A&M University at Qatar, Doha, Qatar
a r t i c l e i n f o
a b s t r a c t
Article history: Received 1 December 2014 Received in revised form 20 May 2015 Accepted 20 May 2015 Available online 22 May 2015
Fault detection (FD) and diagnosis in industrial processes is essential to ensure process safety and maintain product quality. Partial least squares (PLS) has been used successfully in process monitoring because it can effectively deal with highly correlated process variables. However, the conventional PLSbased detection metrics, such as the Hotelling's T2 and the Q statistics are ill suited to detect small faults because they only use information from the most recent observations. Other univariate statistical monitoring methods, such as the exponentially weighted moving average (EWMA) control scheme, has shown better abilities to detect small faults. However, EWMA can only be used to monitor single variables. Therefore, the main objective of this paper is to combine the advantages of the univariate EWMA and PLS methods to enhance their performances and widen their applicability in practice. The performance of the proposed PLS-based EWMA FD method was compared with that of the conventional PLS FD method through two simulated examples, one using synthetic data and the other using simulated distillation column data. The simulation results clearly show the effectiveness of the proposed method over the conventional PLS, especially in the presence of faults with small magnitudes. © 2015 Elsevier Ltd. All rights reserved.
Keywords: Partial least squares Statistical fault detection Small faults Data-based fault detection EWMA control scheme Distillation columns
1. Introduction 1.1. The state of the art Process safety and loss prevention demands are continuously increasing in modern industrial processes. With the higher expectations in the productivity and safety of processes, process monitoring is becoming more important than ever before. Ensuring safety of chemical processes and products is a fundamental requirement from societal, political, and environmental point of views. Towards this end, early fault detection can assist the operation engineers in an industrial plant to take the best course of actions to prevent critical situations that may otherwise lead to fatalities, productivity losses, or environmental issue. Therefore, fault detection is essential to product quality and operational safety (?). A survey carried out by Nimmo (1995) showed that the petrochemical industry in the USA could economize up to 10 billion $ per year if the anomalies that occur in the monitored process could be suitably detected and diagnosed. Of course, when an
* Corresponding author. E-mail addresses: [email protected] (F. Harrou), mohamed.nounou@ qatar.tamu.edu (M.N. Nounou), [email protected] (H.N. Nounou), [email protected] (M. Madakyaru). http://dx.doi.org/10.1016/j.jlp.2015.05.017 0950-4230/© 2015 Elsevier Ltd. All rights reserved.
anomaly or a fault occurs in a process, the monitoring technique must immediately detect the fault and assist in determining whether the process can operate normally or not. Once the anomaly is detected, its source should be identified and corrective actions should be taken before degrades the performance of the process (Isermann (2006)). Therefore, the ability to detect and isolate faults in complex industrial processes is primordial in order to fulfill dependability requirements. This work focuses on fault detection (FD). Over the past four decades, there has been resurgent interest in monitoring approaches for safer operation of systems or processes (Hwang et al. (2010); Isermann (2006); Qin (2012); Venkatasubramanian et al. (2003a); Qingsong (2004); Venkatasubramanian et al. (2003b)). Two main classes of monitoring approaches can generally be distinguished: data-based and model-based approaches (Hwang et al. (2010); Qin (2012); Venkatasubramanian et al. (2003a, b)). Model-based fault detection (FD) approaches rely on comparing the measured process variables with information obtained from an explicit mathematical model, which is usually derived from a basic understanding of the process under normal operating conditions (Isermann (2006); Gao and Dai (2013); Harrou et al. (2014)). The effectiveness of such methods depends on the accuracy of the models used, which can be complex and hard to derive in practice, thus making nonapplicable
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
in a wide range of processes. Data-based approaches, n the other hand, reply on the availability of process data (Qin (2012); Venkatasubramanian et al. (2003b); Chiang (2000); Harrou et al. (2013)). Such approaches utilize process data collected under normal operating conditions to build an empirical model representing the nominal behavior of the monitored process which is then used to detect faults in future data. Explicit models (which are often costly and time consuming to develop)s are not required, so data-based methods are of greater interest in many real-life applications. Nevertheless, the performance of data-based methodologies relies largely on the quantity and quality of the available data. This work is centered on FD using data-based approaches. Several data-based FD techniques are referenced in the literature, and they can be broadly divided into two main classes: univariate and multivariate techniques (Montgomery (2005); Bissell (1994)). Univariate methods, such as the exponentially weighted moving average (EWMA) and cumulative sum (CUSUM) schemes, are used to monitor only single process variables (Page (1954); Hawkins and Olwell (1998); Montgomery (2005); Lucas and Saccucci (1990)). However, modern industrial processes often present a large number of highly correlated process variables, and thus they can be unsuitable for monitoring multivariate processes. Moreover, multivariate FD methods (Hwang et al. (2010); Qin (2012)), have been effectively used to detect faults in multivariate processes with highly cross-correlated variables. A multivariate FD method take into account the correlation between the process variables while univariate FD methods do not. Multivariate statistical monitoring methods include the latent variable methods, e.g., partial least square (PLS) regression, principal component analysis (PCA), canonical variate analysis (CVA), independent component analysis (ICA), (Chaing et al. (2001); Venkatasubramanian et al. (2003b)), neural networks (Subbaraj and Kannapiran (2010)), Fuzzy systems (Dexter and Benouarets (1996)) as well as the pattern recognition methods (Mohammadi and Asgary (2005)). This paper presents a statistical FD scheme based on a PLS model. PLS also known as Projection to Latent Structures is one of the well known multivariate statistical techniques for dimensionality reduction of process data, which was originally proposed by Herman Wold and coworkers for econometrics and chemometrics (Wold et al. (1984)). The feature of a PLS model is its ability to deal with collinear variables in both the input (predictor) matrix X and output (response) matrix Y (Geladi and Kowalski (1986)). In its general form, PLS computes the latent variables that capture the largest variations in the data and maximize the cross-correlation among the predictor and the response variables (Geladi and Kowalski (1986)). Numerous extensions of the linear PLS models were also developed and used in process monitoring, which include the multiway PLS (Nomikos and MacGregor (1995)), multi-scale PLS (MSPLS) (H.W. Lee et al. (2009)), recursive PLS (Wang et al. (2003)), multi-block PLS s et al. (2012)), dynamic PLS (Ahn et al. (2008)) as (Servera-France well as multi-phase PLS (Lu et al. (2004)). The success of PLS-based monitoring methods in chemometrics has engendered a large number of applications in various areas, such as robotic (Muradore and Fiorini (2012)), automotive (Kembhavi et al. (2011), bioinformatics (Li et al. (2007)), health (Ramírez et al. (2010)), and many others. However, PLS-based monitoring statistics, such as the T2 and Q statistics, may not be suitable to detect changes resulting from small faults (Montgomery (2005)). Unlike PLS-based statistics, univariate statistics, such as EWMA (Lowry et al. (1992); Montgomery (2005); Bersimis et al. (2007); Lowry and Montgomery (1995)), have shown a greater ability to detect small faults in the process mean. Thus, the main goal of this paper is to extend the advantages of the univariate EWMA FD method to handle multivariate processes by developing a PLS-based EWMA fault detection method, which integrates the advantages of method techniques.
109
The rest of this paper is organized as follows. Section 2 presents a brief introduction to PLS and its utilization in fault detection. Then, the EWMA control scheme is described in Section 3. Next, the proposed PLS-based EWMA FD approach, that integrates PLS modeling and the EWMA control scheme, is presented in Section 4. Then, the performances of the various techniques are compared through simulated examples in Section 5. Finally, some concluding remarks and future research directions are presented in Section 7. 2. Process monitoring using PLS 2.1. Partial least square modeling Consider an input data matrix X 2 Rnm having n observations and m variables and an output data matrix Y 2 Rnp consisting of p response variables. The two matrices are generally pre-processed by centering and scaling to have zero mean and unit variance prior to PLS modeling. The key philosophy underpinning PLS is to build a model by relating the latent variables (LVs) associated with X and Y. In other words, PLS relies on linearly transforming both X and Y into their respective LVs, and then relate the two sets of transformed or latent variables. Thus, a PLS model can be represented by two sets of linear equations: the inner model and the outer model (Geladi and Kowalski (1986)). The outer models related the original variables to their respective LVs, and the inner model relates the two sets of LVs. The outer model, which relates the original variables with the LVs can be written as (Kourti and MacGregor (1995)):
8 l P > > b > ti PTi þ E ¼ TPT þ E < X¼ XþE¼ i¼1
l > > b þ F ¼ P u qT þ F ¼ UQ T þ F > :Y ¼ Y i i
(1)
i¼1
b and Y b represent the estimated input and output matrices where X using the retained LVs X and Y, respectively; the matrices T 2 Rnl and U 2 Rnq consist of the l retained LVs for the input matrix and the q retained LVs for the output matrix, respectively; the matrices E 2 Rnm and F 2 Rnp represent the approximation residuals of the input and output matrices, respectively; and the matrices P 2 Rml and Q 2 Rpq represent the input and output loading matrices, respectively. Typically, the number of LVs for the input data, l, is often relatively small compared with the number of the original input variables, and can be estimated using crossvalidation or some other technique (Li et al. (2002)), which will be discussed further in the next section. The depiction of the input (predictor) and output (response) matrices (given in equation (1)) is obtained by maximizing of the covariance between the predictor and response LVs, i.e., T and U. The linear inner model relating the predictor and response LVs can be written as:
U ¼ TB þ H;
(2)
where B represents a regression matrix which consists of the model parameters relating the predictor and response LVs, and H represents a residual matrix. The response Y can now be expressed as:
Y ¼ TBQ T þ F* :
(3)
In PLS regression, each pair of latent variables, tj and uj (j ¼ 1,…,l) is sequentially extracted through an iterative procedure. The most commonly used procedure is the non-linear iterative partial least squares (NIPALS) algorithm (Wold (1966); Geladi and Kowalski (1986); MacGregor and Kourti (1995)). This algorithm sequentially extracts each pair of corresponding latent variables as
110
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
a liner combination of the input and output variables prior to fitting the linear regression model (Wold (1966)). The first set of loading vectors p1 and q1 represent the dominant direction obtained by maximization of the covariance betwen X and Y. Projection of X data on p1 and Y data on q1 resulted in the first set of score vectors t1 and u1, hence establishes the outer relation. The matrices X and Y can now be related through their respective scores (inner model), b 1 ¼ t1 b1 Þ. The representing a linear regression between t1 and u1 ð u calculation of first two LVs is illustrated in Fig. 1. After the scores and loadings have been calculated for the first latent variable the residuals corresponding to the input and output variables are computed by the following equations.
E1 ¼ X t1 p1 F1 ¼ Y u1 q1 ¼ Y t1 b1 q1
(4)
The procedure for determining the LVs and loading vectors is repeated using the newly computed residuals until they are these residuals are small enough or do not contain any meaningful variations. In the PLS, the case of a single output is termed PLS1 and that of multiple outputs is termed PLS2 (Geladi and Kowalski (1986)). This work focuses on single output models. 2.2. Selection of the number of PLS components An important step task in developing a PLS model is choosing the number of retained LVs, l, that are needed to adequately capture most of the variations in the data. The goodness of the developed PLS model relies on selecting the appropriate number of LVs to be included, l. Overestimating the value of l may cause overfitting the data by retaining more noise, while underestimating l may leave out some important variations. Therefore, it is important to have a good estimate of the number of LVs l. Several techniques have been proposed to estimate the number of LVs, which include the Scree plots (Zhu and Ghodsi (2006)), cumulative percent variance, parallel analysis, sequential tests, resampling, profile likelihood (Zhu and Ghodsi (2006); Li et al. (2002)), and cross validation (CV) (Li et al. (2002)). In this work, cross-validation will be used to estimate l. In CV, the training data set is divided into two complementary subsets: one set is utilized to learn or train the model and the remainder is used to test or validate the model. The predicted residual sum of squares (PRESS) (Li et al. (2002)) is the most common statistical method utilized to compute the optimum number of LVs, and is given by (Li et al. (2002)):
PRESSl ¼
n X l 2 yi b yi ;
(5)
i¼1
where l represents the number of rtained LVs, n is the total number l of observations in the training data set, yi and b y i are the ith measured and predicted output samples, respectively. The optimal
number of LVs is selected by minimizing the PRESS criterion (Li et al. (2002)). 2.3. Conventional PLS monitoring statistics In PLS-based process monitoring, a PLS reference model is developed using fault-free data to represent the nominal process behavior. Thus, the monitoring system can compare the new process performance with a predefined behavior to ensure it remains under normal operating conditions. When a fault occurs, the process shifts away from the nominal operational zone, thus indicating that a drift in the process behavior has occurred. Of course, once a reference PLS model has been built, it can be used along with detection indices, such as the Hotelling's T2 and Q statistics, to detect faults. The Hotelling's T2 is a statistical metric that captures the behavior of the retained LVs. It measures the variation in the LVs at different times instances, and is defined as (Hotelling (1933)): T 1 b bt; T 2 ¼ bt L
(6) T
b T, b is the covariance matrix of the l retained LVs. If b ¼ 1 T where L n1 2 , given in the T2 statistic is smaller than the control limit, Tl;n;a (Hotelling (1933)), then it can be assumed that the process operates under abnormal conditions. The T2 statistic limits are calculated with the hypothesis that the data are normally distributed (Hotelling (1933)). The Q(X) statistic or squared prediction error (SPE), on the other hand, shows deviations from normal operating conditions based on variations in the predictor variables that are not described by the PLS model (Geladi and Kowalski (1986)). In other words, the Q(X) statistic quantifies the loss of fit from the PLS model developed and is defined as (Jackson and Mudholkar (1979)): 2
b jj2 ; Q ðXÞ ¼ jjx x
(7)
b represents the prediction of x by the PLS model. When a where x new data vector is available, the Q(X) statistic is calculated and ðXÞ compared with the control limit Qa given in (Jackson and Mudholkar (1979)). If the computed Q(X) statistics do not exceed the confidence limit, the monitored process is considered to be operating normally. To undertake monitoring on the basis of the indices above, control limits need to be computed using the statistics of the normal data (Jackson and Mudholkar (1979)). Unfortunately, the T2 and Q statistics take into account only the present information of the process, and thus they have a short memory. Consequently, these detection indices are relatively insensitive to small changes in the process variables, which may result in missed detections (Montgomery (2005)). These drawback of the T2 and Q statistics motivate the use of other alternatives in order to surmount these disadvantages. In this study, a fault detection technique which is based on linear a PLS model and the
Fig. 1. Linear PLS algorithm.
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
EWMA control scheme is developed in order to provide improved detection performance compared to the conventional PLS based FD method. A succinct introduction to the basic ideas behind the EWMA monitoring scheme is presented in the subsequent section. 3. EWMA statistical control scheme Statistical process control (SPC) was employed extensively in a variety of industrial processes. Control charts are one of the most frequently used techniques in SPC, and have been widely used as a monitoring tool in quality engineering to detect the existence of possible anomalies in the mean or variance of process measurements. Control charts play a crucial role in detecting whether a process is still working under normal operating conditions (usually termed in-control) or not (out-of-control). In other words, a control chart depicts a process over time and helps determining the state of the monitored process monitored (Montgomery (2005)). Numerous control charts, including the Shewhart charts, CUSUM charts (Page (1954); Montgomery (2005)), and EWMA charts (Hawkins and Olwell (1998); Hunter (1986); Lucas and Saccucci (1990)), have been created to monitor the mean of process variables over time. The primary utilization of control charts has focused on industrial quality control applications for many decades. Nowadays, the use of control charts has been broadened to many n (2009)), medicine (Biau et al. fields, including economics (Frise (2011)), informatics (Park (2005)), and others. Shewhart control charts are very popular in SPC and can be utilized to successfully detect large variations in the process mean (Human et al. (2010)). However, a key disadvantage is that Shewhart charts only use the most recent measurement and thus does not have a strong memory of the process history. Consequently, these charts possess low sensitivity to small drifts in the data (Montgomery (2005)), making Shewhart charts less useful in monitoring processes with small faults. To help detect smaller shifts, control charts with longer memory need to be used, such as the EWMA and CUSUM charts (Montgomery (2005)). However, since EWMA uses a weighted mean of the past and present observations, it is a lot less sensitive to violating normality assumptions than CUSUM charts (Hawkins and Olwell (1998); Cinar and Undey (1999)), making EWMA a more popular approach (Montgomery (2005)). The EWMA control scheme, also termed as geometric moving average (GMA)or exponential smoothing (Montgomery (2005)), was first introduced by Roberts (Cinar and Undey (1999)), and has been extensively used in time series analysis (Superville and Yorke (2012)). EWMA has been found to be a powerful and sensitive method that can be used as an effective process monitoring strategy (Montgomery (2005)). The properties of this control scheme and its use in process monitoring were studied in (Lucas and Saccucci (1990); Hunter (1986); Montgomery (2005)). The moving average in an EWMA control scheme is calculated by multiplying the historical observations by a weight that decays exponentially with time (Montgomery (2005)). Monitoring the process mean using the EWMA control scheme can be performed for individual observations as well as for averages of rational subgroups (Montgomery (2005)). For individual observations, the EWMA decision function, zt, can be calculated recursively as follows:
zt ¼ lxt þ ð1 lÞzt1 ;
(8)
where xt represents the value of the monitored variable at time t (t ¼ 1,…,n), and l is a smoothing parameter lying between 0 and 1 (i.e. 0 < l 1) that determines the temporal memory of the EWMA decision function. Larger l values result in shorter EWMA scheme memory (by giving more weight to more recent data samples and
111
less weight to older ones). When l ¼ 1 (which corresponds to no smoothing), then monitoring using EWMA reduces to the Shewhart control chart technique, and when l ¼ 0, no features in the data are retained. In other words, this coefficient l acts as a forgetting factor. Generally, in quality control, a smaller l value results in smaller shifts being detected quicker (Lucas and Saccucci (1990); Montgomery (2005)). Therefore, l should be adjusted to an appropriate value in accordance with the characteristics of the monitored process; l is generally established between 0.2 and 0.3 (Hunter (1986)). Over the past few years, the EWMA control scheme has attracted increasing attention in industrial quality monitoring by virtue of this flexibility. The EWMA control scheme declares a fault when the decision statistic, zt, exceeds the control limits expressed as the asymptotic standard deviation of zt. In other words, the strategy consists of depicting the EWMA statistic zt versus the sample number on the control chart together with the upper control limit (UCL) and the lower control limit (LCL). If the observations xt are uncorrelated, the control limits of the EWMA statistic, zt, can be computed as follows (Montgomery (2005)): s
zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi h i l 1 ð1 lÞ2t UCL\LCL ¼ m0 ±Ls0 ð2 lÞ
(9)
The values of the UCL and LCL are functions of time. However, after a long time, they converge to the following values (i.e. as t become larger the term [1 (1 l)2t] approaches unity): s
zfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflffl{ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi l UCL\LCL ¼ m0 ±Ls0 ð2 lÞ
(10)
where L represents the control limit width (which determines the confidence limits, usually defined as 3 for a number of false alarms qffiffiffiffiffiffiffiffiffiffiffi ffi l of 0.3%), s ¼ s0 nð2lÞ is the asymptotic standard deviation of the EWMA statistic zt, and s0 is the in-control standard deviation of x. More details about the properties of the EWMA control scheme can be found in (Lucas and Saccucci (1990)). Note that a slight modification of the two-sided EWMA scheme (which has upper and lower control limits) can lead to its one-sided version, which has only an upper control limit. To do that, one needs to replace the original statistic zt by its absolute value. In the next section, the one-sided EWMA control scheme is integrated with PLS to help enhance the ability of PLS in detecting faults with small amplitude affecting the mean of process or system measurements. 4. Proposed PLS-based EWMA FD strategy In this section, PLS is integrated with EWMA to develop a new FD scheme with greater sensitivity to small data faults. Towards this end, in this approach, PLS is used as modeling framework. Once developed, PLS models can be combined with EWMA control schemes to detect unusual process conditions, such as process and sensor faults. The residuals of the response variables from the PLS model are used are evaluated to determine the presence or absence of faults (Kinnaert (2003); Nyberg and Nyberg (1999)). Under normal operating conditions (no faults), the residuals of the monitored system are zero or close to zero in the presence of modeling measurement noise and uncertainties. However, in the presence of a fault, the residuals differ significantly from zero, showing the existence of a new state that can be clearly distinguished from the normal faultless working mode (Kinnaert (2003);
112
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
Nyberg and Nyberg (1999)). In this work, EWMA is used to enhance process monitoring by integration its with PLS modeling. Due to the capacity of the EWMA control scheme to detect drifts with low severity in the data, this technique is appropriate for improving the detection of small faults. Thus, this work exploits the advantages of the EWMA control scheme to improve fault detection in relation to conventional PLS-based methods. Of course, this approach can only provide detection (i.e., no isolation) of faults. The detailed procedure will be presented in the subsequent section. 4.1. PLS-based EWMA process monitoring algorithm In this approach, the EWMA control scheme is applied using the residuals of the response variables from the PLS model. As shown in Equation (1) of the outer model, the output vector y can be written as the sum of an approximated vector b y and a residual vector F, i.e.,
y¼b y þ F:
(11)
The difference between the observed value of the input variable, y, and the predicted value, b y , obtained from PLS model represent the residuals in the output variable, F ¼ [f1,…,ft,…,fn] which can be used as an indicator to detect a possible fault. The EWMA decision function based on the residuals of the response variable can be computed as follows:
zt ¼ lft þ ð1 lÞzt1 ;
t2½1; n:
(12)
Since the EWMA control scheme is applied on the residuals vector of the response variable, only one EWMA decision function will be computed to monitor system. However, this approach can only detect the presence of faults, i.e., it can not determine their locations. The PLS-based EWMA fault detection algorithm are summarized in Table 1. In the next section, the performance of the PLS-based EWMA fault detection method will be evaluated and compared to that of the conventional PLS fault detection scheme through two simulated example, one using synthetic data and the other using simulated distillation column data. 5. Simulated examples 5.1. Fault detection in synthetic data In this section, the performance of the proposed PLS-based EWMA fault detection algorithm is assessed through its utilization to detect faults in synthetic data sets which contained several different types of fault scenarios. We also conducted the same tests for the standard PLS
method and compared the results with each other. 5.1.1. Data generation The simulated data sets used in the simulated example consists of six input variables and one output. The input data set consists of six random variables, which are generated as follows:
8 x1 > > > > x > > < 2 x3 x4 > > > > > x5 > : x6
¼ u1 þ ε1 ; ¼ u1 þ ε2 ; ¼ u1 þ ε3 ; ¼ u2 þ ε4 ; ¼ 2u2 þ ε5 ; ¼ 2x1 þ 2x4 þ ε6 ;
(13)
where, εi, represent measurements errors, which follow a zeromean Gaussian distribution having a standard deviation of 0.095. The first two input variables u1 and u2 represent a quad-chirp signal (which is sinusoidal waveform with quadratically increasing frequency) and a mishmash signal (which oscillates at frequencies that increase with time), respectively. The other input variables are computed as linear combinations of the first two inputs, which means that the input matrix X ¼ [x1,…,x6] is of rank 2. Then, the output variable is obtained as linear combination of the input variables as follows:
y¼
6 X
ai xi ;
(14)
i¼1
where ai ¼ {1,2,1,1.5,0.5}, with i 2 [1,6]. 5.1.2. PLS model building As a first step of the modeling building process, a PLS model is constructed using a training data set, which defines the normal operating conditions. The training data consist of one thousand process measurements that are generated using the model described earlier. These data are scaled to be zero mean with a unit variance, and are then used to develop a PLS model. An important step in PLS model building is the selection of the number of latent variables. In this study, the cross validation technique has been used for this purpose. To compute the PRESS criterion, the training data set is divided into subsets, the model is fitted using the first subset, and the prediction errors are calculated using the second subset. The optimal number of the of latent variables has been found to be l ¼ 2. Now, the performances of the different FD techniques will be assessed. Three different cases of faults are considered. In the first case study, it is assumed that the testing data sets contains additive
Table 1 PLS-based EWMA fault detection algorithm. Step
Action
1.
Given: A training fault-free data set (X and y) that represents the normal process operations and a testing data set (possibly faulty data), The parameters of the EWMA control scheme: smoothing parameter l and the control limit width L, Data preprocessing Scale the data that is used for process model building, to zero mean and unit variance, Build the PLS model using the training fault-free data Select the number of latent variables with the smallest index PRESS, Express the data matrix as a sum of approximate and residual matrices as shown in equation (1), Test the new data Scale the new data with the mean and standard deviation obtained from the training data, Compute the residuals of the response variables, F, Compute the EWMA decision function, zt, Compute the EWMA control limits, Check for faults Declare a fault when the EWMA decision function, zt, exceeds the control limits.
2. 3.
4.
5.
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
bias faults, (case A). In the second case study, sensor precision degradation fault is considered by contaminating the first input variable x1 by additive random Gaussian noise (case B). In the third case study, it is assumed that the testing data set contains a drift sensor fault (case C). The three different types of fault cases are summarized in Table 2. 5.1.3. Case A: sensor bias fault In this case study, the problem of detecting sensor bias faults is considered. The testing data set, which is simulated using the same model given in Equations (13) and (14), consists of 500 data samples, which are completely independent from the training data. The testing data set is first scaled with the mean and standard deviation of the training fault free data. The PLS model constructed using the training fault-free data is used in this section to detect possible faults using unseen testing data. Herein, the performance of PLSbased EWMA fault detection method is illustrated and compared with that of conventional PLS. Two examples are presented here to illustrate the ability of PLS-based EWMA control scheme to detect bias fault. In the first example, an additive fault is introduced in the variable x1 between samples 200 and 300. This fault is represented by a constant bias of amplitude equal 50% of the total variation in x1, which is large enough for easy detection by both the conventional PLS and the PLS-based EWMA control scheme. The T2 and Q(X) statistics for this testing data with additive bias fault are shown in Figs. 2 and 3, respectively. As it can be observed, the T2 and Q(X) confidence limits have been exceeded, which means that the fault has been detected, but with some missed detections. The blue shaded area represents the zone where the fault is introduced in the testing data. The results of the PLS-based EWMA fault detection algorithm using l ¼ 0.25, which are plotted in Fig. 4, clearly show the violation of the confidence limits and thus the ability of the proposed PLS-based EWMA fault detection algorithm to correctly detect this bias type faults without any false alarms. In the second example, a small bias, which is 5% of the total variation in x1, is added to the input variable x1 between samples 200 and 300. The time evolution of T2 and Q(X) monitoring indices based on the testing data, are shown in Figs. 5 and 6, respectively. The dashed lines represent a 95% confidence limits used to identify the possible faults. As can be seen from Figs. 5 and 6, the conventional PLS was unable to detect this small bias fault. This is because the conventional PLS only takes into account the present information of the process, making it not powerful enough to detect small changes. On the other hand, the results of the PLS-based EWMA fault detection algorithm, with a smoothing parameter l ¼ 0.2, for this testing data are shown in Fig. 7. These results show that the proposed fault detection approach could satisfactorily detect the given bias fault without any false alarms. As can be noticed, this case study clearly shows the advantage of the PLS-based EWMA fault detection algorithm over the conventional PLS approach, especially in the presence of faults with small magnitudes. The satisfactory fault detection ability of the proposed PLS-based EWMA control scheme is maily due to the memory effect of the EWMA control scheme. 5.1.4. Case B: precision degradation fault In this case study, the ability of the proposed PLS-based EWMA fault detection method is assessed through its utilization to detect a
113
precision degradation type sensor fault. Towards this end, the faultfree input variable x1 is corrupted using random Gaussian noise with an enlarged variance of s2 ¼ 0.5 from sample number 300 till the end of the simulated testing data. The Q(X) and T2 monitoring statistics for this precision degradation fault case are shown in Figs. 8 and 9, respectively. The results show that the fault was detected using the Q(X) and T2 monitoring statistics, but with some missed detections. The PLS-based EWMA decision function for this case, which is plotted in Fig. 10, clearly shows the violation of the confidence limits and thus the ability of the proposed PLS-based EWMA fault detection method to detect this type of fault. Although the conventional PLS also detected these faults, the fault detection ability of the PLS-based EWMA FD method was clearly better than that of the conventional PLS. 5.1.5. Case C: drift sensor faults Slow drifting (or ramping) sensor fault usually indicates a slow degradation of sensor properties over a long period of time. Slow drifting usually can occurs when a sensor is aged or degraded. This case study is aimed to assess the ability of the proposed PLS-based EWMA fault detection method in detecting ‘slow drift sensor fault’. In this case study, a slow drifting sensor fault with the slope of 0.01 was added to the input variable x1 starting at sample number 300 of the simulated testing data. In other words, the input variable x1 was linearly increased from sample 300 till the end of testing data by adding 0.01(k 300) to the x1 value of each sample in this range, where k is the sample number. The results of the T2 and Q(X) statistics are shown in Figs. 12 and 11 respectively. Fig. 12, shows that Q(X) statistic could successfully detect this fault by exceeding the threshold value. The Q(X) statistic gradually increased as the drift sensor fault increases, and began to violate the confidence limits when the fault magnitude became sufficiently large enough to be detected by the given model. However, this slow drifting fault is undetectable by the Hotelling's T2 statistic as shown in Fig. 11. The T2 statistic as shown in Fig. 11 is constantly below the confidence limit signaling no abnormality in the testing data. Thus, the Q(X) statistic is more sensitive than Hotelling's T2 statistic in this case because small faults might not produce significant deviation in the PCs. Moreover, Fig. 13, which shows that the results of the proposed PLS-based EWMA fault detection results, shows that the EWMA statistic violates the threshold after sample 334 and the EWMA statistic keeps on increasing, signaling that a slow drifting fault had started around this sample number. Because the magnitude for this drifting fault is slowly increasing, the EWMA statistic is increased gradually until it exceeded the corresponding control limits. This result demonstrates the limitation of conventional PLS, that is for slow (gradual) sensor drifts, better results can be obtained by the PLS-based EWMA control scheme. This can be seen from Fig. 12 where the Q(X) statistic violates the confidence limit only around sample number 400, which shows that it is slow in reacting to small fault in the process mean. Thus the proposed PLS-based EWMA fault detection approach is able to detect the fault almost 60 time samples earlier than the conventional PLS based Q(X) statistic. Therefore, the proposed method is more able to deal with drifting faults which develop slowly with time. These results show the advantage of combining PLS modeling and the EWMA control scheme for detecting small faults.
Table 2 Fault types tested in the simulation study. Fault type
Description
Parameters
Bias Precision degradation Drift
x1(t) ¼ x1(t) þ b x1(t) ¼ x1(t) þ N(0,s2) x1(t) ¼ x1(t) þ s(k ks)
b: bias s2: variance s: slope, and ks: starting point of the drift fault
114
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
Fig. 2. The time evolution of the T2 statistic in the presence of a bias fault in x1 (Case A, first example).
Fig. 3. The time evolution of the Q statistic in the presence of a bias fault in x1 (Case A, first example).
Fig. 4. The time evolution of the EWMA statistic in the presence of a bias fault in x1 (Case A, first example).
Fig. 5. The time evolution of the T2 statistic in the presence of a bias fault in x1 (Case A, second example).
Fig. 6. The time evolution of the Q(x) statistic in the presence of a bias fault in x1 (Case A, second example).
Fig. 7. The time evolution of the EWMA statistic in the presence of a bias fault in x1 (Case A, second example).
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
115
Fig. 8. The time evolution of the T2 statistic in the presence of a precision degradation fault in x1 (Case B).
Fig. 9. The time evolution of the Q(x) statistic in the presence of a precision degradation fault in x1 (Case B).
Fig. 10. The time evolution of the EWMA statistic in the presence of a precision degradation fault in x1 (Case B).
Fig. 11. The evolution of the T2 statistic in the presence of drift fault in x1 with slope 0.01 (Case C, first example).
Fig. 12. The evolution of the Q statistic in the presence of Drift fault with slope 0.01 in x1 (Case C, first example).
6. Fault detection in simulated distillation column data
6.1. Process description and data generation
In this section, the effectiveness of the proposed PLS-based EWMA control scheme is illustrated through its application using a distillation column data set.
To illustrate the effectiveness of the proposed PLS-base EWMA monitoring method in practice, in this example, it is applied to detect faults in simulated distillation column data. Towards this
Fig. 13. The evolution of the EWMA statistic in the presence of drift fault with slope 0.01 in x1 (Case C, first example).
116
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119 Table 3 Steady state operating conditions of the distillation column Madakyaru et al. (2013). Process variable Feed F T P zF Reflux Drum D T Reflux P xD Reboiler Drum B Q T P xB
Value 1 kg mol/sec 322 K 1.7225 106 Pa 0.4 0.40206 kg mol/sec 325 K 62.6602 kg/sec 1.7022 106 Pa 0.979
(propane) in the distillate stream (i.e., xD). The generated in this simulated example are assumed to be noise-free, which are then contaminated with Gaussian noise with zero mean. To build a PLS model, the process data are divided into training and testing subsets (each of them containing 512 observations), the model is fitted using the training subset, and the prediction errors are calculated using the testing subset. Plots of these fault-free measurements data are shown in Figures (14a,14b,14c,14d), with signal-to-noise ratio of 10. After mean-centering and scaling all variables to unit standard deviation, a PLS model was constructed based on the fault-free training data. Based on cross-validation technique, the number of LVs required for the PLS model was found to be three. 6.2. Prediction quality of PLS model
0.5979 kg mol/sec 2.7385 107 Watts 366 K 1.72362 106 Pa 0.01
end, the dynamic operation of a distillation column consisting of 32 theoretical layers (including the reboiler and total condenser) is simulated using Aspen Tech 7.2. The feed stream enters the column at stage 16 as a saturated liquid having a flow rate of 1 kmol/s, a temperature of 322 K, and compositions of 40 mol% propane and 60 mol% isobutene. The nominal steady state operating conditions of the column are presented in Table 3. The process data, which consist of 1024 observations, are obtained by disturbing the feed flow rates and the reflux streams from their normal operational values. Firstly, step changes of amplitudes ±2% in the flow rate of the feed around its normal operational condition have been introduced, and in each case, the process is allowed to settle to a new steady state. After achieving the normal conditions again, similar step changes of amplitudes ±2% in the reflux flow rate around its normal condition are introduced. In this simulated process, the predictor (input) variables consisting of ten temperatures (Tc’s) at different trays of the column, the feed flow rate (F), and the reflux stream flow rate (R). The response (output) variable is the compositions of the light component
Once the number of LVs has been determined, testing the model prediction capability is a critical step for model acceptance. To illustrate the quality of the constructed PLS model, one common and simple approach is to plot the predicted versus observed values ~ eiro et al. (2008)). Fig. 15 shows a scatter plot of (or vice versa) (Pin the observed versus predicted values of the testing data set obtained from the constructed PLS model. As can be viewed from the Fig. 15, the points follow the diagonal line (predicted ¼ observed) quite well, and there is no indication of a curvature or other anomalies. To evaluate the goodness or the accuracy of the model in fitting the data, various statistics have been developed, which include the root mean-square-error (RMSE), R2 also known as the coefficient of determination, Bayesian information criterion (BIC), and others ~ eiro et al. (2008)). In this study, RMSE and R2 have been used to (Pin evaluate the goodness of the constructed PLS model, and gave values of R2 ¼ 0.96 and RMSE ¼ 0.003. A value of R2 ¼ 0.96 means that 96% of the total sum of squares in the testing set is described by the model, and that only 4% is in the residuals. Therefore, the constructed PLS model effectively quantifies the process behavior. 6.3. Results and discussion Now, the FD abilities of the conventional PLS and the proposed
Fig. 14. Distillation column example: dynamic inputeoutput data used for training and testing the model with noise SNR ¼ 10 (solid red line: noise-free data; blue dots: noisy data). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
117
small fault without false alarms. The smoothing parameter l and the control limit L used in this simulation are 0.2 and 3, respectively. This case clearly demonstrates the advantage of the PLSbased EWMA method over the conventional PLS method, especially in the case of small faults.
Fig. 15. Scatter plots of predicted and observed training data.
PLS-based EWMA FD algorithm are assessed using the distillation column data. Two case studies representing different types of sensor faults are considered for this purpose. In the first case study, it is assumed that the testing data set contains an additive bias fault, case (i), and in the second case study, it is assumed that the testing data set contains a drift sensor fault, case (ii). Case (i). Bias sensor fault In this case study, the performance of the PLS-based EWMA FD scheme is assessed when used to detect small additive bias fault in the distillation column data. To do that, a bias fault, with a magnitude of 2% of the total variation in temperature Tc3, was introduced to the temperature sensor measurements ‘Tc3’ between samples 150 and 250. The performances of the Q and T2 statistics are demonstrated in Figs. 16 and 17, respectively. The dashed lines represent the 95% confidence limits used to identify possible faults. The blue shaded area represents the zone where the fault is introduced in the testing data. These results show that the conventional PLS based methods (Q and T2) are completely unable to detect this small simulated fault. This is because these conventional PLS based fault detection metrics only take into account the information provided by the present data samples in the decision making process, which makes these metrics not very powerful in detecting small changes. The results of the PLS-based EWMA fault detection algorithm, however, which are illustrated in Fig. 18, clearly show the ability this proposed method in detecting this
Case (ii). Slow drift sensor fault This case aims to assess the performance of the proposed PLSbased EWMA FD scheme in detecting a slow drift fault. Towards this end, a slow drifting sensor fault with the slope of 0.01 was added to the temperature sensor Tc3 starting at sample 250 till the end of testing data. The Q(X) and T2 statistics for this case are plotted in Figs. 19 and 20, respectively. The Q(X) statistic begins to increase linearly from the 300th sample and exceeds the control limit at the 356th sample, while the T2 statistic is unable to detect it suitably. The results of PLS-based EWMA FD algorithm for the considered slow drift type sensor fault are shown in Fig. 21. From Fig. 21, one can easily notice that the EWMA statistic gradually increases as the fault slowly increases from the 250th sample and goes beyond the control limit at the 300th sample. Thus, the proposed method detects the fault 56 samples earlier than conventional PLS-based Q statistic. These results show the advantage of combining PLS modeling and EWMA control scheme for detecting small faults. In summary, the proposed PLS-based EWMA fault detection method showed satisfactory fault detection abilities for all fault types that are tested in this study compared with the conventional PLS method. The simulation results demonstrate an improved ability of the proposed PLS-based EWMA fault detection method over the conventional PLS-base metrics, such as Q and T2, especially for faults with small magnitudes. 7. Conclusion Fault detection is a vital problem with important impact on the safe and reliable operation of complex industrial processes. The necessity for having reliable system is more important than ever before. PLS-based fault detection indices, such as Q and T2, have been widely used in practice. Hoever, these metric do not perform well in the case of faults with small magnitudes. EWMA has been shown to possess a higher sensitivity to small faults. In this paper, a fault detection method that combines the advantages of PLS and EWMA is developed to provide improved fault detection abilities. The developed PLS-based EWMA method uses PLS as a modeling framework to compute the model residuals that are then evaluated
Fig. 16. The evolution of the T2 statistic in the presence of bias fault in x5,(Case (i)).
Fig. 17. The evolution of the Q(X) statistic in the presence of bias fault in x5, (Case (i)).
118
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
Fig. 18. The evolution of the EWMA statistic in the presence of bias fault in x5, (Case (i)).
Fig. 19. The evolution of the T2 statistic in the presence of bias fault in x5, (Case (ii)).
Fig. 20. The evolution of the Q(X) statistic in the presence of bias fault in x5, (Case (ii)).
Fig. 21. The evolution of the EWMA statistic in the presence of bias fault in x5, (Case (ii)).
using EWMA. The effectiveness of the developed method is demonstrated through its application using synthetic data and simulated distillation column data. The simulation results obtained in both examples show improved performance of the developed method over the PLS-based metrics, especially in detecting faults with small magnitudes. A limitation of the developed technique is that it might not be suitable when the data are multimodal and/or when strong nonlinear correlations are present. However, it could be extended to handle nonlinear processes (e.g., using kernel PLS). Acknowledgment This work was made possible by NPRP grant NPRP 7-1172-2-439 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors. References Ahn, S., Lee, C., Jung, Y., Han, C., Yoon, E., Lee, G., 2008. Fault diagnosis of the multistage flash desalination process based on signed digraph and dynamic partial least square. Desalination 228 (1), 68e83. Bersimis, S., Psarakis, S., Panaretos, J., 2007. Multivariate statistical process control charts: an overview. Qual. Reliab. Eng. Int. 23 (5), 517e543.
Biau, D., Dessolle, L., Porcher, R., 2011. Applications and experiences of quality control to surgical and interventional procedures. In: Ivanov, O. (Ed.), Applications and Experiences of Quality Control. Intech. Bissell, D., 1994. Statistical Methods for SPC and TQM, vol. 26. CRC Press. Chaing, L., Russel, E., Braatz, R., 2001. Fault Detection and Diagnosis in Industrial Systems. Springer, London. Chiang, L., 2000. Data-driven Methods for Fault Detection and Diagnosis in Chemical Processes. Cinar, A., Undey, C., 1999. Statistical process and controller performance monitoring. a tutorial on current methods and future directions. In: American Control Conference, 1999. Proceedings of the 1999, vol. 4. IEEE, pp. 2625e2639. Dexter, A., Benouarets, M., 1996. Generic approach to identifying faults in hvac plants. ASHRAE Trans. 102 (1), 550e556. n, M., 2009. Optimal sequential surveillance for finance, public health, and Frise other areas. Seq. Anal. 28 (3), 310e337. Gao, Z., Dai, X., 2013. From model, signal to knowledge: a data-driven perspective of fault detection and diagnosis. IEEE Trans. Ind. Inf. 9 (4), 2226e2238. Geladi, P., Kowalski, B., 1986. Partial least-squares regression: a tutorial. Anal. Chim. acta 185, 1e17. Harrou, F., Nounou, M., Nounou, H., Madakyaru, M., 2013. Statistical fault detection using pca-based glr hypothesis testing. J. Loss Prev. Process Ind. 26 (1), 129e139. Harrou, F., Fillatre, L., N, I., 2014. Anomaly detection/detectability for a linear model with a bounded nuisance parameter. Annu. Rev. Control 38 (1), 32e44. Hawkins, D.M., Olwell, D.H., 1998. Cumulative Sum Charts and Charting for Quality Improvement. Springer Verlag. Hotelling, H., 1933. Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24, 417e441. Human, S.W., Chakraborti, S., Smit, C.F., 2010. Shewhart-type control charts for variation in phase i data analysis. Comput. Stat. Data Anal. 54 (4), 863e874. Hunter, J.S., 1986. The exponentially weighted moving average. J. Qual. Technol. 18
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119 (4), 203e210. Hwang, I., Kim, S., Kim, Y., Seah, C., 2010. A survey of fault detection, isolation, and reconfiguration methods. IEEE Trans. Control Syst. Technol. 18 (3), 636e653. Isermann, R., 2006. Fault-diagnosis Systems: an in Troduction from Fault Detection to Fault Tolerance. Springer. Jackson, J., Mudholkar, G., 1979. Control procedures for residuals associated with principal component analysis. Technometrics 21, 341e349. Kembhavi, A., Harwood, D., Davis, L., 2011. Vehicle detection using partial least squares. Pattern Anal. Mach. Intell. IEEE Trans. 33 (6), 1250e1265. Kinnaert, M., 2003. Fault diagnosis based on analytical models for linear and nonlinear systems - a tutorial. In: Proceedings of the 15th International Workshop on Principles of Diagnosis, pp. 37e50. Kourti, T., MacGregor, J., 1995. Process analysis, monitoring and diagnosis using multivariate projection methods: a tutorial. Chemom. Intell. Lab. Syst. 28 (3), 3e21. Lee, H.W., L, M., Park, J., 2009. Multi-scale extension of PLS algorithm for advanced on-line process monitoring. Chemom. Intell. Lab. Syst. 98 (2), 201e212. Li, B., Morris, J., Martin, E., 2002. Model selection for partial least squares regression. Chemom. Intell. Lab. Syst. 64 (1), 79e89. Li, G., Zeng, X., Yang, J., Yang, M., 2007. Partial least squares based dimension reduction with gene selection for tumor classification. In: Bioinformatics and Bioengineering, 2007. BIBE 2007. Proceedings of the 7th IEEE International Conference on. IEEE, pp. 1439e1444. Lowry, C.A., Montgomery, D.C., 1995. A review of multivariate control charts. IIE Trans. 27 (6), 800e810. Lowry, C.A., Woodall, W.H., Champ, C.W., Rigdon, S.E., 1992. A multivariate exponentially weighted moving average control chart. Technometrics 34 (1), 46e53. Lu, N., Gao, F., Wang, F., 2004. Sub-pca modeling and on-line monitoring strategy for batch processes. AIChE J. 50 (1), 255e259. Lucas, J., Saccucci, M., 1990. Exponentially weighted moving average control schemes: properties and enhancements. Technometrics 32 (1), 1e12. MacGregor, J., Kourti, T., 1995. Statistical process control of multivariate processes. Control Eng. Pract. 3 (3). Madakyaru, M., Nounou, M., Nounou, H., 2013. Integrated multiscale latent variable regression and application to distillation columns. Model. Simul. Eng. 2013, 1e17. Mohammadi, K., Asgary, R., 2005. Pattern recognition and fault detection in mems. Comput. Recognit. Syst. Adv. Soft Comput. 30, 877e884. Montgomery, D.C., 2005. Introduction to Statistical Quality Control. John Wiley& Sons, New York. Muradore, R., Fiorini, P., 2012. A PLS-based statistical approach for fault detection and isolation of robotic manipulators. Ind. Electron. IEEE Trans. 59 (8), 3167e3175.
119
Nimmo, I., 1995. Adequately address abnormal operations. Chem. Eng. Prog. 91 (9). Nomikos, P., MacGregor, J., 1995. Multi-way partial least squares in monitoring batch processes. Chemom. Intell. lab. Syst. 30 (1), 97e108. Nyberg, M., Nyberg, C.M., 1999. Model Based Fault Diagnosis: Methods, Theory, and Automotive Engine Applications (PhD. thesis). Page, E.S., 1954. Continuous inspection schemes. Biometrika 100e115. Park, Y., 2005. A Statistical Process Control Approach for Network Intrusion Detection. ~ eiro, G., Perelman, S., Guerschman, J., Paruelo, J., 2008. How to evaluate models: Pin observed vs. predicted or predicted vs. observed? Ecol. Model. 216 (3), 316e322. Qin, S., 2012. Survey on data-driven industrial process monitoring and diagnosis. Annu. Rev. Control 36 (2), 220e234. Qingsong, Y., May 2004. Model-based and Data Driven Fault Diagnosis Methods with Applications to Process Monitoring (Phd. thesis). Case Western reserve University. rriz, J., Segovia, F., Chaves, R., Salas-Gonzalez, D., Lo pez, M., Alvarez, I., Ramírez, J., Go Padilla, P., 2010. Early alzheimer's disease diagnosis using partial least squares and random forests. In: Biomedical Imaging: from Nano to Macro, 2010 IEEE International Symposium on. IEEE, pp. 81e84. s, D., Arteaga-Moreno, F., Gil-Saura, I., Gallarza, M., 2012. Servera-France A multiblock pls-based algorithm applied to a causal model in marketing. Appl. ¸ e253. ^AS Stoch. Models Bus. Ind. 29, 241a Subbaraj, P., Kannapiran, B., 2010. Artificial neural network approach for fault detection in pneumatic valve in cooler water spray system. Int. J. Comput. Appl. 9 (7), 43e52. Superville, C., Yorke, G., 2012. Monitoring techniques for time series production processes. J. Int. Divers. 2012 (4). Venkatasubramanian, V., Rengaswamy, R., Kavuri, S., Yin, K., 2003a. A review of process fault detection and diagnosis part I: quantitative model-based methods. Comput. Chem. Eng. 27, 293e311. Venkatasubramanian, V., Rengaswamy, R., Kavuri, S., Yin, K., 2003b. A review of process fault detection and diagnosis part III: process history based methods. Comput. Chem. Eng. 27, 327e346. Wang, X., Kruger, U., Lennox, B., 2003. Recursive partial least squares algorithms for monitoring complex industrial processes. Control Eng. Pract. 11 (6), 613e632. Wold, H., 1966. Nonlinear estimation by iterative least squares procedures. Res. Pap. stat. 12 (5), 134e139. Wold, S., Ruhe, H., Wold III, H., D, W., 1984. The collinearity problem in linear regression. the partial least squares (PLS) approach to generalized inverses. SIAM J. Sci. Stat. Comput. 5 (3), 735e743. Zhu, M., Ghodsi, A., 2006. Automatic dimensionality selection from the scree plot via the use of profile likelihood. Comput. Stat. Data Anal. 51, 918e930.
Contents lists available at ScienceDirect
Journal of Loss Prevention in the Process Industries journal homepage: www.elsevier.com/locate/jlp
PLS-based EWMA fault detection strategy for process monitoring Fouzi Harrou a, Mohamed N. Nounou a, *, Hazem N. Nounou b, Muddu Madakyaru a a b
Chemical Engineering Program, Texas A&M University at Qatar, Doha, Qatar Electrical and Computer Engineering Program, Texas A&M University at Qatar, Doha, Qatar
a r t i c l e i n f o
a b s t r a c t
Article history: Received 1 December 2014 Received in revised form 20 May 2015 Accepted 20 May 2015 Available online 22 May 2015
Fault detection (FD) and diagnosis in industrial processes is essential to ensure process safety and maintain product quality. Partial least squares (PLS) has been used successfully in process monitoring because it can effectively deal with highly correlated process variables. However, the conventional PLSbased detection metrics, such as the Hotelling's T2 and the Q statistics are ill suited to detect small faults because they only use information from the most recent observations. Other univariate statistical monitoring methods, such as the exponentially weighted moving average (EWMA) control scheme, has shown better abilities to detect small faults. However, EWMA can only be used to monitor single variables. Therefore, the main objective of this paper is to combine the advantages of the univariate EWMA and PLS methods to enhance their performances and widen their applicability in practice. The performance of the proposed PLS-based EWMA FD method was compared with that of the conventional PLS FD method through two simulated examples, one using synthetic data and the other using simulated distillation column data. The simulation results clearly show the effectiveness of the proposed method over the conventional PLS, especially in the presence of faults with small magnitudes. © 2015 Elsevier Ltd. All rights reserved.
Keywords: Partial least squares Statistical fault detection Small faults Data-based fault detection EWMA control scheme Distillation columns
1. Introduction 1.1. The state of the art Process safety and loss prevention demands are continuously increasing in modern industrial processes. With the higher expectations in the productivity and safety of processes, process monitoring is becoming more important than ever before. Ensuring safety of chemical processes and products is a fundamental requirement from societal, political, and environmental point of views. Towards this end, early fault detection can assist the operation engineers in an industrial plant to take the best course of actions to prevent critical situations that may otherwise lead to fatalities, productivity losses, or environmental issue. Therefore, fault detection is essential to product quality and operational safety (?). A survey carried out by Nimmo (1995) showed that the petrochemical industry in the USA could economize up to 10 billion $ per year if the anomalies that occur in the monitored process could be suitably detected and diagnosed. Of course, when an
* Corresponding author. E-mail addresses: [email protected] (F. Harrou), mohamed.nounou@ qatar.tamu.edu (M.N. Nounou), [email protected] (H.N. Nounou), [email protected] (M. Madakyaru). http://dx.doi.org/10.1016/j.jlp.2015.05.017 0950-4230/© 2015 Elsevier Ltd. All rights reserved.
anomaly or a fault occurs in a process, the monitoring technique must immediately detect the fault and assist in determining whether the process can operate normally or not. Once the anomaly is detected, its source should be identified and corrective actions should be taken before degrades the performance of the process (Isermann (2006)). Therefore, the ability to detect and isolate faults in complex industrial processes is primordial in order to fulfill dependability requirements. This work focuses on fault detection (FD). Over the past four decades, there has been resurgent interest in monitoring approaches for safer operation of systems or processes (Hwang et al. (2010); Isermann (2006); Qin (2012); Venkatasubramanian et al. (2003a); Qingsong (2004); Venkatasubramanian et al. (2003b)). Two main classes of monitoring approaches can generally be distinguished: data-based and model-based approaches (Hwang et al. (2010); Qin (2012); Venkatasubramanian et al. (2003a, b)). Model-based fault detection (FD) approaches rely on comparing the measured process variables with information obtained from an explicit mathematical model, which is usually derived from a basic understanding of the process under normal operating conditions (Isermann (2006); Gao and Dai (2013); Harrou et al. (2014)). The effectiveness of such methods depends on the accuracy of the models used, which can be complex and hard to derive in practice, thus making nonapplicable
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
in a wide range of processes. Data-based approaches, n the other hand, reply on the availability of process data (Qin (2012); Venkatasubramanian et al. (2003b); Chiang (2000); Harrou et al. (2013)). Such approaches utilize process data collected under normal operating conditions to build an empirical model representing the nominal behavior of the monitored process which is then used to detect faults in future data. Explicit models (which are often costly and time consuming to develop)s are not required, so data-based methods are of greater interest in many real-life applications. Nevertheless, the performance of data-based methodologies relies largely on the quantity and quality of the available data. This work is centered on FD using data-based approaches. Several data-based FD techniques are referenced in the literature, and they can be broadly divided into two main classes: univariate and multivariate techniques (Montgomery (2005); Bissell (1994)). Univariate methods, such as the exponentially weighted moving average (EWMA) and cumulative sum (CUSUM) schemes, are used to monitor only single process variables (Page (1954); Hawkins and Olwell (1998); Montgomery (2005); Lucas and Saccucci (1990)). However, modern industrial processes often present a large number of highly correlated process variables, and thus they can be unsuitable for monitoring multivariate processes. Moreover, multivariate FD methods (Hwang et al. (2010); Qin (2012)), have been effectively used to detect faults in multivariate processes with highly cross-correlated variables. A multivariate FD method take into account the correlation between the process variables while univariate FD methods do not. Multivariate statistical monitoring methods include the latent variable methods, e.g., partial least square (PLS) regression, principal component analysis (PCA), canonical variate analysis (CVA), independent component analysis (ICA), (Chaing et al. (2001); Venkatasubramanian et al. (2003b)), neural networks (Subbaraj and Kannapiran (2010)), Fuzzy systems (Dexter and Benouarets (1996)) as well as the pattern recognition methods (Mohammadi and Asgary (2005)). This paper presents a statistical FD scheme based on a PLS model. PLS also known as Projection to Latent Structures is one of the well known multivariate statistical techniques for dimensionality reduction of process data, which was originally proposed by Herman Wold and coworkers for econometrics and chemometrics (Wold et al. (1984)). The feature of a PLS model is its ability to deal with collinear variables in both the input (predictor) matrix X and output (response) matrix Y (Geladi and Kowalski (1986)). In its general form, PLS computes the latent variables that capture the largest variations in the data and maximize the cross-correlation among the predictor and the response variables (Geladi and Kowalski (1986)). Numerous extensions of the linear PLS models were also developed and used in process monitoring, which include the multiway PLS (Nomikos and MacGregor (1995)), multi-scale PLS (MSPLS) (H.W. Lee et al. (2009)), recursive PLS (Wang et al. (2003)), multi-block PLS s et al. (2012)), dynamic PLS (Ahn et al. (2008)) as (Servera-France well as multi-phase PLS (Lu et al. (2004)). The success of PLS-based monitoring methods in chemometrics has engendered a large number of applications in various areas, such as robotic (Muradore and Fiorini (2012)), automotive (Kembhavi et al. (2011), bioinformatics (Li et al. (2007)), health (Ramírez et al. (2010)), and many others. However, PLS-based monitoring statistics, such as the T2 and Q statistics, may not be suitable to detect changes resulting from small faults (Montgomery (2005)). Unlike PLS-based statistics, univariate statistics, such as EWMA (Lowry et al. (1992); Montgomery (2005); Bersimis et al. (2007); Lowry and Montgomery (1995)), have shown a greater ability to detect small faults in the process mean. Thus, the main goal of this paper is to extend the advantages of the univariate EWMA FD method to handle multivariate processes by developing a PLS-based EWMA fault detection method, which integrates the advantages of method techniques.
109
The rest of this paper is organized as follows. Section 2 presents a brief introduction to PLS and its utilization in fault detection. Then, the EWMA control scheme is described in Section 3. Next, the proposed PLS-based EWMA FD approach, that integrates PLS modeling and the EWMA control scheme, is presented in Section 4. Then, the performances of the various techniques are compared through simulated examples in Section 5. Finally, some concluding remarks and future research directions are presented in Section 7. 2. Process monitoring using PLS 2.1. Partial least square modeling Consider an input data matrix X 2 Rnm having n observations and m variables and an output data matrix Y 2 Rnp consisting of p response variables. The two matrices are generally pre-processed by centering and scaling to have zero mean and unit variance prior to PLS modeling. The key philosophy underpinning PLS is to build a model by relating the latent variables (LVs) associated with X and Y. In other words, PLS relies on linearly transforming both X and Y into their respective LVs, and then relate the two sets of transformed or latent variables. Thus, a PLS model can be represented by two sets of linear equations: the inner model and the outer model (Geladi and Kowalski (1986)). The outer models related the original variables to their respective LVs, and the inner model relates the two sets of LVs. The outer model, which relates the original variables with the LVs can be written as (Kourti and MacGregor (1995)):
8 l P > > b > ti PTi þ E ¼ TPT þ E < X¼ XþE¼ i¼1
l > > b þ F ¼ P u qT þ F ¼ UQ T þ F > :Y ¼ Y i i
(1)
i¼1
b and Y b represent the estimated input and output matrices where X using the retained LVs X and Y, respectively; the matrices T 2 Rnl and U 2 Rnq consist of the l retained LVs for the input matrix and the q retained LVs for the output matrix, respectively; the matrices E 2 Rnm and F 2 Rnp represent the approximation residuals of the input and output matrices, respectively; and the matrices P 2 Rml and Q 2 Rpq represent the input and output loading matrices, respectively. Typically, the number of LVs for the input data, l, is often relatively small compared with the number of the original input variables, and can be estimated using crossvalidation or some other technique (Li et al. (2002)), which will be discussed further in the next section. The depiction of the input (predictor) and output (response) matrices (given in equation (1)) is obtained by maximizing of the covariance between the predictor and response LVs, i.e., T and U. The linear inner model relating the predictor and response LVs can be written as:
U ¼ TB þ H;
(2)
where B represents a regression matrix which consists of the model parameters relating the predictor and response LVs, and H represents a residual matrix. The response Y can now be expressed as:
Y ¼ TBQ T þ F* :
(3)
In PLS regression, each pair of latent variables, tj and uj (j ¼ 1,…,l) is sequentially extracted through an iterative procedure. The most commonly used procedure is the non-linear iterative partial least squares (NIPALS) algorithm (Wold (1966); Geladi and Kowalski (1986); MacGregor and Kourti (1995)). This algorithm sequentially extracts each pair of corresponding latent variables as
110
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
a liner combination of the input and output variables prior to fitting the linear regression model (Wold (1966)). The first set of loading vectors p1 and q1 represent the dominant direction obtained by maximization of the covariance betwen X and Y. Projection of X data on p1 and Y data on q1 resulted in the first set of score vectors t1 and u1, hence establishes the outer relation. The matrices X and Y can now be related through their respective scores (inner model), b 1 ¼ t1 b1 Þ. The representing a linear regression between t1 and u1 ð u calculation of first two LVs is illustrated in Fig. 1. After the scores and loadings have been calculated for the first latent variable the residuals corresponding to the input and output variables are computed by the following equations.
E1 ¼ X t1 p1 F1 ¼ Y u1 q1 ¼ Y t1 b1 q1
(4)
The procedure for determining the LVs and loading vectors is repeated using the newly computed residuals until they are these residuals are small enough or do not contain any meaningful variations. In the PLS, the case of a single output is termed PLS1 and that of multiple outputs is termed PLS2 (Geladi and Kowalski (1986)). This work focuses on single output models. 2.2. Selection of the number of PLS components An important step task in developing a PLS model is choosing the number of retained LVs, l, that are needed to adequately capture most of the variations in the data. The goodness of the developed PLS model relies on selecting the appropriate number of LVs to be included, l. Overestimating the value of l may cause overfitting the data by retaining more noise, while underestimating l may leave out some important variations. Therefore, it is important to have a good estimate of the number of LVs l. Several techniques have been proposed to estimate the number of LVs, which include the Scree plots (Zhu and Ghodsi (2006)), cumulative percent variance, parallel analysis, sequential tests, resampling, profile likelihood (Zhu and Ghodsi (2006); Li et al. (2002)), and cross validation (CV) (Li et al. (2002)). In this work, cross-validation will be used to estimate l. In CV, the training data set is divided into two complementary subsets: one set is utilized to learn or train the model and the remainder is used to test or validate the model. The predicted residual sum of squares (PRESS) (Li et al. (2002)) is the most common statistical method utilized to compute the optimum number of LVs, and is given by (Li et al. (2002)):
PRESSl ¼
n X l 2 yi b yi ;
(5)
i¼1
where l represents the number of rtained LVs, n is the total number l of observations in the training data set, yi and b y i are the ith measured and predicted output samples, respectively. The optimal
number of LVs is selected by minimizing the PRESS criterion (Li et al. (2002)). 2.3. Conventional PLS monitoring statistics In PLS-based process monitoring, a PLS reference model is developed using fault-free data to represent the nominal process behavior. Thus, the monitoring system can compare the new process performance with a predefined behavior to ensure it remains under normal operating conditions. When a fault occurs, the process shifts away from the nominal operational zone, thus indicating that a drift in the process behavior has occurred. Of course, once a reference PLS model has been built, it can be used along with detection indices, such as the Hotelling's T2 and Q statistics, to detect faults. The Hotelling's T2 is a statistical metric that captures the behavior of the retained LVs. It measures the variation in the LVs at different times instances, and is defined as (Hotelling (1933)): T 1 b bt; T 2 ¼ bt L
(6) T
b T, b is the covariance matrix of the l retained LVs. If b ¼ 1 T where L n1 2 , given in the T2 statistic is smaller than the control limit, Tl;n;a (Hotelling (1933)), then it can be assumed that the process operates under abnormal conditions. The T2 statistic limits are calculated with the hypothesis that the data are normally distributed (Hotelling (1933)). The Q(X) statistic or squared prediction error (SPE), on the other hand, shows deviations from normal operating conditions based on variations in the predictor variables that are not described by the PLS model (Geladi and Kowalski (1986)). In other words, the Q(X) statistic quantifies the loss of fit from the PLS model developed and is defined as (Jackson and Mudholkar (1979)): 2
b jj2 ; Q ðXÞ ¼ jjx x
(7)
b represents the prediction of x by the PLS model. When a where x new data vector is available, the Q(X) statistic is calculated and ðXÞ compared with the control limit Qa given in (Jackson and Mudholkar (1979)). If the computed Q(X) statistics do not exceed the confidence limit, the monitored process is considered to be operating normally. To undertake monitoring on the basis of the indices above, control limits need to be computed using the statistics of the normal data (Jackson and Mudholkar (1979)). Unfortunately, the T2 and Q statistics take into account only the present information of the process, and thus they have a short memory. Consequently, these detection indices are relatively insensitive to small changes in the process variables, which may result in missed detections (Montgomery (2005)). These drawback of the T2 and Q statistics motivate the use of other alternatives in order to surmount these disadvantages. In this study, a fault detection technique which is based on linear a PLS model and the
Fig. 1. Linear PLS algorithm.
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
EWMA control scheme is developed in order to provide improved detection performance compared to the conventional PLS based FD method. A succinct introduction to the basic ideas behind the EWMA monitoring scheme is presented in the subsequent section. 3. EWMA statistical control scheme Statistical process control (SPC) was employed extensively in a variety of industrial processes. Control charts are one of the most frequently used techniques in SPC, and have been widely used as a monitoring tool in quality engineering to detect the existence of possible anomalies in the mean or variance of process measurements. Control charts play a crucial role in detecting whether a process is still working under normal operating conditions (usually termed in-control) or not (out-of-control). In other words, a control chart depicts a process over time and helps determining the state of the monitored process monitored (Montgomery (2005)). Numerous control charts, including the Shewhart charts, CUSUM charts (Page (1954); Montgomery (2005)), and EWMA charts (Hawkins and Olwell (1998); Hunter (1986); Lucas and Saccucci (1990)), have been created to monitor the mean of process variables over time. The primary utilization of control charts has focused on industrial quality control applications for many decades. Nowadays, the use of control charts has been broadened to many n (2009)), medicine (Biau et al. fields, including economics (Frise (2011)), informatics (Park (2005)), and others. Shewhart control charts are very popular in SPC and can be utilized to successfully detect large variations in the process mean (Human et al. (2010)). However, a key disadvantage is that Shewhart charts only use the most recent measurement and thus does not have a strong memory of the process history. Consequently, these charts possess low sensitivity to small drifts in the data (Montgomery (2005)), making Shewhart charts less useful in monitoring processes with small faults. To help detect smaller shifts, control charts with longer memory need to be used, such as the EWMA and CUSUM charts (Montgomery (2005)). However, since EWMA uses a weighted mean of the past and present observations, it is a lot less sensitive to violating normality assumptions than CUSUM charts (Hawkins and Olwell (1998); Cinar and Undey (1999)), making EWMA a more popular approach (Montgomery (2005)). The EWMA control scheme, also termed as geometric moving average (GMA)or exponential smoothing (Montgomery (2005)), was first introduced by Roberts (Cinar and Undey (1999)), and has been extensively used in time series analysis (Superville and Yorke (2012)). EWMA has been found to be a powerful and sensitive method that can be used as an effective process monitoring strategy (Montgomery (2005)). The properties of this control scheme and its use in process monitoring were studied in (Lucas and Saccucci (1990); Hunter (1986); Montgomery (2005)). The moving average in an EWMA control scheme is calculated by multiplying the historical observations by a weight that decays exponentially with time (Montgomery (2005)). Monitoring the process mean using the EWMA control scheme can be performed for individual observations as well as for averages of rational subgroups (Montgomery (2005)). For individual observations, the EWMA decision function, zt, can be calculated recursively as follows:
zt ¼ lxt þ ð1 lÞzt1 ;
(8)
where xt represents the value of the monitored variable at time t (t ¼ 1,…,n), and l is a smoothing parameter lying between 0 and 1 (i.e. 0 < l 1) that determines the temporal memory of the EWMA decision function. Larger l values result in shorter EWMA scheme memory (by giving more weight to more recent data samples and
111
less weight to older ones). When l ¼ 1 (which corresponds to no smoothing), then monitoring using EWMA reduces to the Shewhart control chart technique, and when l ¼ 0, no features in the data are retained. In other words, this coefficient l acts as a forgetting factor. Generally, in quality control, a smaller l value results in smaller shifts being detected quicker (Lucas and Saccucci (1990); Montgomery (2005)). Therefore, l should be adjusted to an appropriate value in accordance with the characteristics of the monitored process; l is generally established between 0.2 and 0.3 (Hunter (1986)). Over the past few years, the EWMA control scheme has attracted increasing attention in industrial quality monitoring by virtue of this flexibility. The EWMA control scheme declares a fault when the decision statistic, zt, exceeds the control limits expressed as the asymptotic standard deviation of zt. In other words, the strategy consists of depicting the EWMA statistic zt versus the sample number on the control chart together with the upper control limit (UCL) and the lower control limit (LCL). If the observations xt are uncorrelated, the control limits of the EWMA statistic, zt, can be computed as follows (Montgomery (2005)): s
zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi h i l 1 ð1 lÞ2t UCL\LCL ¼ m0 ±Ls0 ð2 lÞ
(9)
The values of the UCL and LCL are functions of time. However, after a long time, they converge to the following values (i.e. as t become larger the term [1 (1 l)2t] approaches unity): s
zfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflffl{ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi l UCL\LCL ¼ m0 ±Ls0 ð2 lÞ
(10)
where L represents the control limit width (which determines the confidence limits, usually defined as 3 for a number of false alarms qffiffiffiffiffiffiffiffiffiffiffi ffi l of 0.3%), s ¼ s0 nð2lÞ is the asymptotic standard deviation of the EWMA statistic zt, and s0 is the in-control standard deviation of x. More details about the properties of the EWMA control scheme can be found in (Lucas and Saccucci (1990)). Note that a slight modification of the two-sided EWMA scheme (which has upper and lower control limits) can lead to its one-sided version, which has only an upper control limit. To do that, one needs to replace the original statistic zt by its absolute value. In the next section, the one-sided EWMA control scheme is integrated with PLS to help enhance the ability of PLS in detecting faults with small amplitude affecting the mean of process or system measurements. 4. Proposed PLS-based EWMA FD strategy In this section, PLS is integrated with EWMA to develop a new FD scheme with greater sensitivity to small data faults. Towards this end, in this approach, PLS is used as modeling framework. Once developed, PLS models can be combined with EWMA control schemes to detect unusual process conditions, such as process and sensor faults. The residuals of the response variables from the PLS model are used are evaluated to determine the presence or absence of faults (Kinnaert (2003); Nyberg and Nyberg (1999)). Under normal operating conditions (no faults), the residuals of the monitored system are zero or close to zero in the presence of modeling measurement noise and uncertainties. However, in the presence of a fault, the residuals differ significantly from zero, showing the existence of a new state that can be clearly distinguished from the normal faultless working mode (Kinnaert (2003);
112
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
Nyberg and Nyberg (1999)). In this work, EWMA is used to enhance process monitoring by integration its with PLS modeling. Due to the capacity of the EWMA control scheme to detect drifts with low severity in the data, this technique is appropriate for improving the detection of small faults. Thus, this work exploits the advantages of the EWMA control scheme to improve fault detection in relation to conventional PLS-based methods. Of course, this approach can only provide detection (i.e., no isolation) of faults. The detailed procedure will be presented in the subsequent section. 4.1. PLS-based EWMA process monitoring algorithm In this approach, the EWMA control scheme is applied using the residuals of the response variables from the PLS model. As shown in Equation (1) of the outer model, the output vector y can be written as the sum of an approximated vector b y and a residual vector F, i.e.,
y¼b y þ F:
(11)
The difference between the observed value of the input variable, y, and the predicted value, b y , obtained from PLS model represent the residuals in the output variable, F ¼ [f1,…,ft,…,fn] which can be used as an indicator to detect a possible fault. The EWMA decision function based on the residuals of the response variable can be computed as follows:
zt ¼ lft þ ð1 lÞzt1 ;
t2½1; n:
(12)
Since the EWMA control scheme is applied on the residuals vector of the response variable, only one EWMA decision function will be computed to monitor system. However, this approach can only detect the presence of faults, i.e., it can not determine their locations. The PLS-based EWMA fault detection algorithm are summarized in Table 1. In the next section, the performance of the PLS-based EWMA fault detection method will be evaluated and compared to that of the conventional PLS fault detection scheme through two simulated example, one using synthetic data and the other using simulated distillation column data. 5. Simulated examples 5.1. Fault detection in synthetic data In this section, the performance of the proposed PLS-based EWMA fault detection algorithm is assessed through its utilization to detect faults in synthetic data sets which contained several different types of fault scenarios. We also conducted the same tests for the standard PLS
method and compared the results with each other. 5.1.1. Data generation The simulated data sets used in the simulated example consists of six input variables and one output. The input data set consists of six random variables, which are generated as follows:
8 x1 > > > > x > > < 2 x3 x4 > > > > > x5 > : x6
¼ u1 þ ε1 ; ¼ u1 þ ε2 ; ¼ u1 þ ε3 ; ¼ u2 þ ε4 ; ¼ 2u2 þ ε5 ; ¼ 2x1 þ 2x4 þ ε6 ;
(13)
where, εi, represent measurements errors, which follow a zeromean Gaussian distribution having a standard deviation of 0.095. The first two input variables u1 and u2 represent a quad-chirp signal (which is sinusoidal waveform with quadratically increasing frequency) and a mishmash signal (which oscillates at frequencies that increase with time), respectively. The other input variables are computed as linear combinations of the first two inputs, which means that the input matrix X ¼ [x1,…,x6] is of rank 2. Then, the output variable is obtained as linear combination of the input variables as follows:
y¼
6 X
ai xi ;
(14)
i¼1
where ai ¼ {1,2,1,1.5,0.5}, with i 2 [1,6]. 5.1.2. PLS model building As a first step of the modeling building process, a PLS model is constructed using a training data set, which defines the normal operating conditions. The training data consist of one thousand process measurements that are generated using the model described earlier. These data are scaled to be zero mean with a unit variance, and are then used to develop a PLS model. An important step in PLS model building is the selection of the number of latent variables. In this study, the cross validation technique has been used for this purpose. To compute the PRESS criterion, the training data set is divided into subsets, the model is fitted using the first subset, and the prediction errors are calculated using the second subset. The optimal number of the of latent variables has been found to be l ¼ 2. Now, the performances of the different FD techniques will be assessed. Three different cases of faults are considered. In the first case study, it is assumed that the testing data sets contains additive
Table 1 PLS-based EWMA fault detection algorithm. Step
Action
1.
Given: A training fault-free data set (X and y) that represents the normal process operations and a testing data set (possibly faulty data), The parameters of the EWMA control scheme: smoothing parameter l and the control limit width L, Data preprocessing Scale the data that is used for process model building, to zero mean and unit variance, Build the PLS model using the training fault-free data Select the number of latent variables with the smallest index PRESS, Express the data matrix as a sum of approximate and residual matrices as shown in equation (1), Test the new data Scale the new data with the mean and standard deviation obtained from the training data, Compute the residuals of the response variables, F, Compute the EWMA decision function, zt, Compute the EWMA control limits, Check for faults Declare a fault when the EWMA decision function, zt, exceeds the control limits.
2. 3.
4.
5.
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
bias faults, (case A). In the second case study, sensor precision degradation fault is considered by contaminating the first input variable x1 by additive random Gaussian noise (case B). In the third case study, it is assumed that the testing data set contains a drift sensor fault (case C). The three different types of fault cases are summarized in Table 2. 5.1.3. Case A: sensor bias fault In this case study, the problem of detecting sensor bias faults is considered. The testing data set, which is simulated using the same model given in Equations (13) and (14), consists of 500 data samples, which are completely independent from the training data. The testing data set is first scaled with the mean and standard deviation of the training fault free data. The PLS model constructed using the training fault-free data is used in this section to detect possible faults using unseen testing data. Herein, the performance of PLSbased EWMA fault detection method is illustrated and compared with that of conventional PLS. Two examples are presented here to illustrate the ability of PLS-based EWMA control scheme to detect bias fault. In the first example, an additive fault is introduced in the variable x1 between samples 200 and 300. This fault is represented by a constant bias of amplitude equal 50% of the total variation in x1, which is large enough for easy detection by both the conventional PLS and the PLS-based EWMA control scheme. The T2 and Q(X) statistics for this testing data with additive bias fault are shown in Figs. 2 and 3, respectively. As it can be observed, the T2 and Q(X) confidence limits have been exceeded, which means that the fault has been detected, but with some missed detections. The blue shaded area represents the zone where the fault is introduced in the testing data. The results of the PLS-based EWMA fault detection algorithm using l ¼ 0.25, which are plotted in Fig. 4, clearly show the violation of the confidence limits and thus the ability of the proposed PLS-based EWMA fault detection algorithm to correctly detect this bias type faults without any false alarms. In the second example, a small bias, which is 5% of the total variation in x1, is added to the input variable x1 between samples 200 and 300. The time evolution of T2 and Q(X) monitoring indices based on the testing data, are shown in Figs. 5 and 6, respectively. The dashed lines represent a 95% confidence limits used to identify the possible faults. As can be seen from Figs. 5 and 6, the conventional PLS was unable to detect this small bias fault. This is because the conventional PLS only takes into account the present information of the process, making it not powerful enough to detect small changes. On the other hand, the results of the PLS-based EWMA fault detection algorithm, with a smoothing parameter l ¼ 0.2, for this testing data are shown in Fig. 7. These results show that the proposed fault detection approach could satisfactorily detect the given bias fault without any false alarms. As can be noticed, this case study clearly shows the advantage of the PLS-based EWMA fault detection algorithm over the conventional PLS approach, especially in the presence of faults with small magnitudes. The satisfactory fault detection ability of the proposed PLS-based EWMA control scheme is maily due to the memory effect of the EWMA control scheme. 5.1.4. Case B: precision degradation fault In this case study, the ability of the proposed PLS-based EWMA fault detection method is assessed through its utilization to detect a
113
precision degradation type sensor fault. Towards this end, the faultfree input variable x1 is corrupted using random Gaussian noise with an enlarged variance of s2 ¼ 0.5 from sample number 300 till the end of the simulated testing data. The Q(X) and T2 monitoring statistics for this precision degradation fault case are shown in Figs. 8 and 9, respectively. The results show that the fault was detected using the Q(X) and T2 monitoring statistics, but with some missed detections. The PLS-based EWMA decision function for this case, which is plotted in Fig. 10, clearly shows the violation of the confidence limits and thus the ability of the proposed PLS-based EWMA fault detection method to detect this type of fault. Although the conventional PLS also detected these faults, the fault detection ability of the PLS-based EWMA FD method was clearly better than that of the conventional PLS. 5.1.5. Case C: drift sensor faults Slow drifting (or ramping) sensor fault usually indicates a slow degradation of sensor properties over a long period of time. Slow drifting usually can occurs when a sensor is aged or degraded. This case study is aimed to assess the ability of the proposed PLS-based EWMA fault detection method in detecting ‘slow drift sensor fault’. In this case study, a slow drifting sensor fault with the slope of 0.01 was added to the input variable x1 starting at sample number 300 of the simulated testing data. In other words, the input variable x1 was linearly increased from sample 300 till the end of testing data by adding 0.01(k 300) to the x1 value of each sample in this range, where k is the sample number. The results of the T2 and Q(X) statistics are shown in Figs. 12 and 11 respectively. Fig. 12, shows that Q(X) statistic could successfully detect this fault by exceeding the threshold value. The Q(X) statistic gradually increased as the drift sensor fault increases, and began to violate the confidence limits when the fault magnitude became sufficiently large enough to be detected by the given model. However, this slow drifting fault is undetectable by the Hotelling's T2 statistic as shown in Fig. 11. The T2 statistic as shown in Fig. 11 is constantly below the confidence limit signaling no abnormality in the testing data. Thus, the Q(X) statistic is more sensitive than Hotelling's T2 statistic in this case because small faults might not produce significant deviation in the PCs. Moreover, Fig. 13, which shows that the results of the proposed PLS-based EWMA fault detection results, shows that the EWMA statistic violates the threshold after sample 334 and the EWMA statistic keeps on increasing, signaling that a slow drifting fault had started around this sample number. Because the magnitude for this drifting fault is slowly increasing, the EWMA statistic is increased gradually until it exceeded the corresponding control limits. This result demonstrates the limitation of conventional PLS, that is for slow (gradual) sensor drifts, better results can be obtained by the PLS-based EWMA control scheme. This can be seen from Fig. 12 where the Q(X) statistic violates the confidence limit only around sample number 400, which shows that it is slow in reacting to small fault in the process mean. Thus the proposed PLS-based EWMA fault detection approach is able to detect the fault almost 60 time samples earlier than the conventional PLS based Q(X) statistic. Therefore, the proposed method is more able to deal with drifting faults which develop slowly with time. These results show the advantage of combining PLS modeling and the EWMA control scheme for detecting small faults.
Table 2 Fault types tested in the simulation study. Fault type
Description
Parameters
Bias Precision degradation Drift
x1(t) ¼ x1(t) þ b x1(t) ¼ x1(t) þ N(0,s2) x1(t) ¼ x1(t) þ s(k ks)
b: bias s2: variance s: slope, and ks: starting point of the drift fault
114
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
Fig. 2. The time evolution of the T2 statistic in the presence of a bias fault in x1 (Case A, first example).
Fig. 3. The time evolution of the Q statistic in the presence of a bias fault in x1 (Case A, first example).
Fig. 4. The time evolution of the EWMA statistic in the presence of a bias fault in x1 (Case A, first example).
Fig. 5. The time evolution of the T2 statistic in the presence of a bias fault in x1 (Case A, second example).
Fig. 6. The time evolution of the Q(x) statistic in the presence of a bias fault in x1 (Case A, second example).
Fig. 7. The time evolution of the EWMA statistic in the presence of a bias fault in x1 (Case A, second example).
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
115
Fig. 8. The time evolution of the T2 statistic in the presence of a precision degradation fault in x1 (Case B).
Fig. 9. The time evolution of the Q(x) statistic in the presence of a precision degradation fault in x1 (Case B).
Fig. 10. The time evolution of the EWMA statistic in the presence of a precision degradation fault in x1 (Case B).
Fig. 11. The evolution of the T2 statistic in the presence of drift fault in x1 with slope 0.01 (Case C, first example).
Fig. 12. The evolution of the Q statistic in the presence of Drift fault with slope 0.01 in x1 (Case C, first example).
6. Fault detection in simulated distillation column data
6.1. Process description and data generation
In this section, the effectiveness of the proposed PLS-based EWMA control scheme is illustrated through its application using a distillation column data set.
To illustrate the effectiveness of the proposed PLS-base EWMA monitoring method in practice, in this example, it is applied to detect faults in simulated distillation column data. Towards this
Fig. 13. The evolution of the EWMA statistic in the presence of drift fault with slope 0.01 in x1 (Case C, first example).
116
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119 Table 3 Steady state operating conditions of the distillation column Madakyaru et al. (2013). Process variable Feed F T P zF Reflux Drum D T Reflux P xD Reboiler Drum B Q T P xB
Value 1 kg mol/sec 322 K 1.7225 106 Pa 0.4 0.40206 kg mol/sec 325 K 62.6602 kg/sec 1.7022 106 Pa 0.979
(propane) in the distillate stream (i.e., xD). The generated in this simulated example are assumed to be noise-free, which are then contaminated with Gaussian noise with zero mean. To build a PLS model, the process data are divided into training and testing subsets (each of them containing 512 observations), the model is fitted using the training subset, and the prediction errors are calculated using the testing subset. Plots of these fault-free measurements data are shown in Figures (14a,14b,14c,14d), with signal-to-noise ratio of 10. After mean-centering and scaling all variables to unit standard deviation, a PLS model was constructed based on the fault-free training data. Based on cross-validation technique, the number of LVs required for the PLS model was found to be three. 6.2. Prediction quality of PLS model
0.5979 kg mol/sec 2.7385 107 Watts 366 K 1.72362 106 Pa 0.01
end, the dynamic operation of a distillation column consisting of 32 theoretical layers (including the reboiler and total condenser) is simulated using Aspen Tech 7.2. The feed stream enters the column at stage 16 as a saturated liquid having a flow rate of 1 kmol/s, a temperature of 322 K, and compositions of 40 mol% propane and 60 mol% isobutene. The nominal steady state operating conditions of the column are presented in Table 3. The process data, which consist of 1024 observations, are obtained by disturbing the feed flow rates and the reflux streams from their normal operational values. Firstly, step changes of amplitudes ±2% in the flow rate of the feed around its normal operational condition have been introduced, and in each case, the process is allowed to settle to a new steady state. After achieving the normal conditions again, similar step changes of amplitudes ±2% in the reflux flow rate around its normal condition are introduced. In this simulated process, the predictor (input) variables consisting of ten temperatures (Tc’s) at different trays of the column, the feed flow rate (F), and the reflux stream flow rate (R). The response (output) variable is the compositions of the light component
Once the number of LVs has been determined, testing the model prediction capability is a critical step for model acceptance. To illustrate the quality of the constructed PLS model, one common and simple approach is to plot the predicted versus observed values ~ eiro et al. (2008)). Fig. 15 shows a scatter plot of (or vice versa) (Pin the observed versus predicted values of the testing data set obtained from the constructed PLS model. As can be viewed from the Fig. 15, the points follow the diagonal line (predicted ¼ observed) quite well, and there is no indication of a curvature or other anomalies. To evaluate the goodness or the accuracy of the model in fitting the data, various statistics have been developed, which include the root mean-square-error (RMSE), R2 also known as the coefficient of determination, Bayesian information criterion (BIC), and others ~ eiro et al. (2008)). In this study, RMSE and R2 have been used to (Pin evaluate the goodness of the constructed PLS model, and gave values of R2 ¼ 0.96 and RMSE ¼ 0.003. A value of R2 ¼ 0.96 means that 96% of the total sum of squares in the testing set is described by the model, and that only 4% is in the residuals. Therefore, the constructed PLS model effectively quantifies the process behavior. 6.3. Results and discussion Now, the FD abilities of the conventional PLS and the proposed
Fig. 14. Distillation column example: dynamic inputeoutput data used for training and testing the model with noise SNR ¼ 10 (solid red line: noise-free data; blue dots: noisy data). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
117
small fault without false alarms. The smoothing parameter l and the control limit L used in this simulation are 0.2 and 3, respectively. This case clearly demonstrates the advantage of the PLSbased EWMA method over the conventional PLS method, especially in the case of small faults.
Fig. 15. Scatter plots of predicted and observed training data.
PLS-based EWMA FD algorithm are assessed using the distillation column data. Two case studies representing different types of sensor faults are considered for this purpose. In the first case study, it is assumed that the testing data set contains an additive bias fault, case (i), and in the second case study, it is assumed that the testing data set contains a drift sensor fault, case (ii). Case (i). Bias sensor fault In this case study, the performance of the PLS-based EWMA FD scheme is assessed when used to detect small additive bias fault in the distillation column data. To do that, a bias fault, with a magnitude of 2% of the total variation in temperature Tc3, was introduced to the temperature sensor measurements ‘Tc3’ between samples 150 and 250. The performances of the Q and T2 statistics are demonstrated in Figs. 16 and 17, respectively. The dashed lines represent the 95% confidence limits used to identify possible faults. The blue shaded area represents the zone where the fault is introduced in the testing data. These results show that the conventional PLS based methods (Q and T2) are completely unable to detect this small simulated fault. This is because these conventional PLS based fault detection metrics only take into account the information provided by the present data samples in the decision making process, which makes these metrics not very powerful in detecting small changes. The results of the PLS-based EWMA fault detection algorithm, however, which are illustrated in Fig. 18, clearly show the ability this proposed method in detecting this
Case (ii). Slow drift sensor fault This case aims to assess the performance of the proposed PLSbased EWMA FD scheme in detecting a slow drift fault. Towards this end, a slow drifting sensor fault with the slope of 0.01 was added to the temperature sensor Tc3 starting at sample 250 till the end of testing data. The Q(X) and T2 statistics for this case are plotted in Figs. 19 and 20, respectively. The Q(X) statistic begins to increase linearly from the 300th sample and exceeds the control limit at the 356th sample, while the T2 statistic is unable to detect it suitably. The results of PLS-based EWMA FD algorithm for the considered slow drift type sensor fault are shown in Fig. 21. From Fig. 21, one can easily notice that the EWMA statistic gradually increases as the fault slowly increases from the 250th sample and goes beyond the control limit at the 300th sample. Thus, the proposed method detects the fault 56 samples earlier than conventional PLS-based Q statistic. These results show the advantage of combining PLS modeling and EWMA control scheme for detecting small faults. In summary, the proposed PLS-based EWMA fault detection method showed satisfactory fault detection abilities for all fault types that are tested in this study compared with the conventional PLS method. The simulation results demonstrate an improved ability of the proposed PLS-based EWMA fault detection method over the conventional PLS-base metrics, such as Q and T2, especially for faults with small magnitudes. 7. Conclusion Fault detection is a vital problem with important impact on the safe and reliable operation of complex industrial processes. The necessity for having reliable system is more important than ever before. PLS-based fault detection indices, such as Q and T2, have been widely used in practice. Hoever, these metric do not perform well in the case of faults with small magnitudes. EWMA has been shown to possess a higher sensitivity to small faults. In this paper, a fault detection method that combines the advantages of PLS and EWMA is developed to provide improved fault detection abilities. The developed PLS-based EWMA method uses PLS as a modeling framework to compute the model residuals that are then evaluated
Fig. 16. The evolution of the T2 statistic in the presence of bias fault in x5,(Case (i)).
Fig. 17. The evolution of the Q(X) statistic in the presence of bias fault in x5, (Case (i)).
118
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119
Fig. 18. The evolution of the EWMA statistic in the presence of bias fault in x5, (Case (i)).
Fig. 19. The evolution of the T2 statistic in the presence of bias fault in x5, (Case (ii)).
Fig. 20. The evolution of the Q(X) statistic in the presence of bias fault in x5, (Case (ii)).
Fig. 21. The evolution of the EWMA statistic in the presence of bias fault in x5, (Case (ii)).
using EWMA. The effectiveness of the developed method is demonstrated through its application using synthetic data and simulated distillation column data. The simulation results obtained in both examples show improved performance of the developed method over the PLS-based metrics, especially in detecting faults with small magnitudes. A limitation of the developed technique is that it might not be suitable when the data are multimodal and/or when strong nonlinear correlations are present. However, it could be extended to handle nonlinear processes (e.g., using kernel PLS). Acknowledgment This work was made possible by NPRP grant NPRP 7-1172-2-439 from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors. References Ahn, S., Lee, C., Jung, Y., Han, C., Yoon, E., Lee, G., 2008. Fault diagnosis of the multistage flash desalination process based on signed digraph and dynamic partial least square. Desalination 228 (1), 68e83. Bersimis, S., Psarakis, S., Panaretos, J., 2007. Multivariate statistical process control charts: an overview. Qual. Reliab. Eng. Int. 23 (5), 517e543.
Biau, D., Dessolle, L., Porcher, R., 2011. Applications and experiences of quality control to surgical and interventional procedures. In: Ivanov, O. (Ed.), Applications and Experiences of Quality Control. Intech. Bissell, D., 1994. Statistical Methods for SPC and TQM, vol. 26. CRC Press. Chaing, L., Russel, E., Braatz, R., 2001. Fault Detection and Diagnosis in Industrial Systems. Springer, London. Chiang, L., 2000. Data-driven Methods for Fault Detection and Diagnosis in Chemical Processes. Cinar, A., Undey, C., 1999. Statistical process and controller performance monitoring. a tutorial on current methods and future directions. In: American Control Conference, 1999. Proceedings of the 1999, vol. 4. IEEE, pp. 2625e2639. Dexter, A., Benouarets, M., 1996. Generic approach to identifying faults in hvac plants. ASHRAE Trans. 102 (1), 550e556. n, M., 2009. Optimal sequential surveillance for finance, public health, and Frise other areas. Seq. Anal. 28 (3), 310e337. Gao, Z., Dai, X., 2013. From model, signal to knowledge: a data-driven perspective of fault detection and diagnosis. IEEE Trans. Ind. Inf. 9 (4), 2226e2238. Geladi, P., Kowalski, B., 1986. Partial least-squares regression: a tutorial. Anal. Chim. acta 185, 1e17. Harrou, F., Nounou, M., Nounou, H., Madakyaru, M., 2013. Statistical fault detection using pca-based glr hypothesis testing. J. Loss Prev. Process Ind. 26 (1), 129e139. Harrou, F., Fillatre, L., N, I., 2014. Anomaly detection/detectability for a linear model with a bounded nuisance parameter. Annu. Rev. Control 38 (1), 32e44. Hawkins, D.M., Olwell, D.H., 1998. Cumulative Sum Charts and Charting for Quality Improvement. Springer Verlag. Hotelling, H., 1933. Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24, 417e441. Human, S.W., Chakraborti, S., Smit, C.F., 2010. Shewhart-type control charts for variation in phase i data analysis. Comput. Stat. Data Anal. 54 (4), 863e874. Hunter, J.S., 1986. The exponentially weighted moving average. J. Qual. Technol. 18
F. Harrou et al. / Journal of Loss Prevention in the Process Industries 36 (2015) 108e119 (4), 203e210. Hwang, I., Kim, S., Kim, Y., Seah, C., 2010. A survey of fault detection, isolation, and reconfiguration methods. IEEE Trans. Control Syst. Technol. 18 (3), 636e653. Isermann, R., 2006. Fault-diagnosis Systems: an in Troduction from Fault Detection to Fault Tolerance. Springer. Jackson, J., Mudholkar, G., 1979. Control procedures for residuals associated with principal component analysis. Technometrics 21, 341e349. Kembhavi, A., Harwood, D., Davis, L., 2011. Vehicle detection using partial least squares. Pattern Anal. Mach. Intell. IEEE Trans. 33 (6), 1250e1265. Kinnaert, M., 2003. Fault diagnosis based on analytical models for linear and nonlinear systems - a tutorial. In: Proceedings of the 15th International Workshop on Principles of Diagnosis, pp. 37e50. Kourti, T., MacGregor, J., 1995. Process analysis, monitoring and diagnosis using multivariate projection methods: a tutorial. Chemom. Intell. Lab. Syst. 28 (3), 3e21. Lee, H.W., L, M., Park, J., 2009. Multi-scale extension of PLS algorithm for advanced on-line process monitoring. Chemom. Intell. Lab. Syst. 98 (2), 201e212. Li, B., Morris, J., Martin, E., 2002. Model selection for partial least squares regression. Chemom. Intell. Lab. Syst. 64 (1), 79e89. Li, G., Zeng, X., Yang, J., Yang, M., 2007. Partial least squares based dimension reduction with gene selection for tumor classification. In: Bioinformatics and Bioengineering, 2007. BIBE 2007. Proceedings of the 7th IEEE International Conference on. IEEE, pp. 1439e1444. Lowry, C.A., Montgomery, D.C., 1995. A review of multivariate control charts. IIE Trans. 27 (6), 800e810. Lowry, C.A., Woodall, W.H., Champ, C.W., Rigdon, S.E., 1992. A multivariate exponentially weighted moving average control chart. Technometrics 34 (1), 46e53. Lu, N., Gao, F., Wang, F., 2004. Sub-pca modeling and on-line monitoring strategy for batch processes. AIChE J. 50 (1), 255e259. Lucas, J., Saccucci, M., 1990. Exponentially weighted moving average control schemes: properties and enhancements. Technometrics 32 (1), 1e12. MacGregor, J., Kourti, T., 1995. Statistical process control of multivariate processes. Control Eng. Pract. 3 (3). Madakyaru, M., Nounou, M., Nounou, H., 2013. Integrated multiscale latent variable regression and application to distillation columns. Model. Simul. Eng. 2013, 1e17. Mohammadi, K., Asgary, R., 2005. Pattern recognition and fault detection in mems. Comput. Recognit. Syst. Adv. Soft Comput. 30, 877e884. Montgomery, D.C., 2005. Introduction to Statistical Quality Control. John Wiley& Sons, New York. Muradore, R., Fiorini, P., 2012. A PLS-based statistical approach for fault detection and isolation of robotic manipulators. Ind. Electron. IEEE Trans. 59 (8), 3167e3175.
119
Nimmo, I., 1995. Adequately address abnormal operations. Chem. Eng. Prog. 91 (9). Nomikos, P., MacGregor, J., 1995. Multi-way partial least squares in monitoring batch processes. Chemom. Intell. lab. Syst. 30 (1), 97e108. Nyberg, M., Nyberg, C.M., 1999. Model Based Fault Diagnosis: Methods, Theory, and Automotive Engine Applications (PhD. thesis). Page, E.S., 1954. Continuous inspection schemes. Biometrika 100e115. Park, Y., 2005. A Statistical Process Control Approach for Network Intrusion Detection. ~ eiro, G., Perelman, S., Guerschman, J., Paruelo, J., 2008. How to evaluate models: Pin observed vs. predicted or predicted vs. observed? Ecol. Model. 216 (3), 316e322. Qin, S., 2012. Survey on data-driven industrial process monitoring and diagnosis. Annu. Rev. Control 36 (2), 220e234. Qingsong, Y., May 2004. Model-based and Data Driven Fault Diagnosis Methods with Applications to Process Monitoring (Phd. thesis). Case Western reserve University. rriz, J., Segovia, F., Chaves, R., Salas-Gonzalez, D., Lo pez, M., Alvarez, I., Ramírez, J., Go Padilla, P., 2010. Early alzheimer's disease diagnosis using partial least squares and random forests. In: Biomedical Imaging: from Nano to Macro, 2010 IEEE International Symposium on. IEEE, pp. 81e84. s, D., Arteaga-Moreno, F., Gil-Saura, I., Gallarza, M., 2012. Servera-France A multiblock pls-based algorithm applied to a causal model in marketing. Appl. ¸ e253. ^AS Stoch. Models Bus. Ind. 29, 241a Subbaraj, P., Kannapiran, B., 2010. Artificial neural network approach for fault detection in pneumatic valve in cooler water spray system. Int. J. Comput. Appl. 9 (7), 43e52. Superville, C., Yorke, G., 2012. Monitoring techniques for time series production processes. J. Int. Divers. 2012 (4). Venkatasubramanian, V., Rengaswamy, R., Kavuri, S., Yin, K., 2003a. A review of process fault detection and diagnosis part I: quantitative model-based methods. Comput. Chem. Eng. 27, 293e311. Venkatasubramanian, V., Rengaswamy, R., Kavuri, S., Yin, K., 2003b. A review of process fault detection and diagnosis part III: process history based methods. Comput. Chem. Eng. 27, 327e346. Wang, X., Kruger, U., Lennox, B., 2003. Recursive partial least squares algorithms for monitoring complex industrial processes. Control Eng. Pract. 11 (6), 613e632. Wold, H., 1966. Nonlinear estimation by iterative least squares procedures. Res. Pap. stat. 12 (5), 134e139. Wold, S., Ruhe, H., Wold III, H., D, W., 1984. The collinearity problem in linear regression. the partial least squares (PLS) approach to generalized inverses. SIAM J. Sci. Stat. Comput. 5 (3), 735e743. Zhu, M., Ghodsi, A., 2006. Automatic dimensionality selection from the scree plot via the use of profile likelihood. Comput. Stat. Data Anal. 51, 918e930.