- Email: [email protected]
A Data-driven Fault Detection Method Based on Dissipative Trajectories
11th IFAC Symposium on Dynamics and Control of 11th Symposium on and Process including Biosystems 11th IFAC IFACSystems, Symposium on Dynamics Dynamics and Control Control of of 11th IFAC Symposium on Dynamics and Control of Process including Biosystems June 6-8,Systems, 2016. NTNU, Trondheim, Norway Available online at www.sciencedirect.com Process Systems, including Biosystems Process Systems, including Biosystems June 6-8, 2016. NTNU, Trondheim, Norway June June 6-8, 6-8, 2016. 2016. NTNU, NTNU, Trondheim, Trondheim, Norway Norway
ScienceDirect
A A A
IFAC-PapersOnLine 49-7 (2016) 717–722
Data-driven Fault Detection Method Data-driven Fault Method Data-driven Fault Detection Detection Method Based on Dissipative Trajectories Based on Dissipative Trajectories Based ∗on Dissipative Trajectories ∗∗ ∗
Qingyang Lei ∗ Muhammad Tajammal Munir ∗∗ Jie Bao ∗ ∗∗ Jie Bao ∗ ∗∗ Qingyang Lei ∗∗ Muhammad Muhammad Tajammal Munir Qingyang Tajammal Munir Brent Young Qingyang Lei Lei Muhammad Tajammal Munir ∗∗ Jie Jie Bao Bao ∗ ∗∗ ∗∗ Brent Young ∗∗ Brent Young Brent Young ∗ University of New South Wales, Sydney, NSW, 2052, Australia ∗ ∗ University of New South Wales, Sydney, NSW, 2052, Australia ∗ University of New South Wales, (e-mail: University of New South [email protected]). Wales, Sydney, Sydney, NSW, NSW, 2052, 2052, Australia Australia ∗∗ (e-mail: [email protected]). (e-mail: [email protected]). University of Auckland, Auckland, 1010, New Zealand (e-mail: (e-mail: [email protected]). ∗∗ ∗∗ University of Auckland, Auckland, 1010, New Zealand (e-mail: ∗∗ University of Auckland, Auckland, 1010, New Zealand (e-mail: [email protected]). University of Auckland, Auckland, 1010, New Zealand (e-mail: [email protected]). [email protected]). [email protected]). Abstract: Fault detection is becoming increasingly important as the complexity of industrial Abstract: Fault detection becoming increasingly important as the complexity of industrial Abstract: Fault is becoming increasingly important as complexity of process develops. In this is paper, a data-driven fault detection method is proposed. The Abstract: Fault detection detection is becoming increasingly important as the the complexity of industrial industrial process develops. In this paper, a data-driven fault detection method is proposed. The process develops. In this paper, a data-driven fault detection method is proposed. The dissipativity theory is adopted to find the appropriate dissipativity properties for the process process develops. Inis this paper, a data-driven fault dissipativity detection method is proposed. The dissipativity to properties for dissipativity theory is adopted adopted to find find the the appropriate appropriate dissipativity properties for the the process process input outputtheory trajectory. The dissipativity properties can be viewed as an abstract energy dissipativity theory is adopted to find the appropriate dissipativity properties for the process input output trajectory. The dissipativity properties be viewed as an process abstract energy input output trajectory. The dissipativity properties can be as abstract energy property, and the dissipativity properties of input outputcan trajectories represent dynamic input output trajectory. The properties dissipativity properties can be viewed viewed as an an process abstractdynamic energy property, and the dissipativity of input output trajectories represent property, and the dissipativity properties of input output trajectories represent process dynamic features. As faults occur, the dissipative trajectories will change thus allow fault detection to be property, and the dissipativity properties of input output trajectories represent process dynamic features. As faults occur, the dissipative trajectories will change thus allow fault detection to be features. As faults occur, the dissipative trajectories will change thus allow fault detection to performed based on these dissipativity properties. A training algorithm is developed to search features. Asbased faultson occur, the dissipativeproperties. trajectoriesAwill changealgorithm thus allow fault detection to be be performed these dissipativity training is developed to search performed based on these dissipativity properties. A training algorithm is developed to search for the related properties using input output data.A A prior knowledge of developed the process can be performed based on these dissipativity properties. training algorithm is to search for the related properties using input output A prior of the process can be for the related properties using input output data. A prior knowledge of the process can be incorporated into the algorithm to facilitate thedata. training. Theknowledge proposed fault detection method for the related properties using input output data. A prior knowledge of the process can be incorporated into the facilitate the training. The proposed fault detection method incorporated into algorithm to facilitate The proposed fault detection is illustrated on casealgorithm study of ato mono-chlorobenzene plant simulated using VMGSim. incorporated intoaa the the algorithm tomono-chlorobenzene facilitate the the training. training. Thesimulated proposedusing fault VMGSim. detection method method is illustrated on case study of a plant is illustrated on a case study of a mono-chlorobenzene plant simulated using VMGSim. is illustrated on a case study of a mono-chlorobenzene plant simulated using VMGSim. © 2016, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Dissipativity Theory, Dissipative Trajectory, Fault Detection, Data-driven. Keywords: Keywords: Dissipativity Dissipativity Theory, Theory, Dissipative Dissipative Trajectory, Trajectory, Fault Fault Detection, Detection, Data-driven. Data-driven. Keywords: Dissipativity Theory, Dissipative Trajectory, Fault Detection, Data-driven. 1. INTRODUCTION First developed as a framework for analysing dynamical 1. First developed as for dynamical 1. INTRODUCTION INTRODUCTION First developed as aaa framework framework for analysing analysing dynamical systems, the dissipativity theory was initially proposed in 1. INTRODUCTION First developed as framework for analysing dynamical systems, the dissipativity theory was initially proposed in systems, the dissipativity theory was initially proposed in Willems (1972). The dissipativity properties represent the systems, the dissipativity theory was initiallyrepresent proposedthe in Willems (1972). dissipativity properties Willems (1972). The The dissipativity properties represent the process dynamics features, e.g. the gain, phase or their (1972). The dissipativity properties represent the The growing complexity of modern industrial processes Willems process dynamics features, e.g. the the As gain, phase orchange their process dynamics features, e.g. gain, phase their combination (Rojas et al. (2008)). such, the or The growing complexity of processes dynamics features, e.g. the As gain, phase orchange their The growing complexity of modern modern industrial processes present a serious challenge for real industrial time process mon- process The growing complexity of modern industrial processes combination (Rojas et al. (2008)). such, the combination (Rojas et al. (2008)). As such, the change of process dynamical behavior when a fault occurs may present a serious challenge for real time process moncombination (Rojas et al. (2008)). As such, the change present a serious challenge for real time process monitoring. Considering the plant economy, industrial propresent aConsidering serious challenge for economy, real timeindustrial process monof process dynamical behavior when aa fault occurs may of process dynamical behavior when fault occurs may be reflected by change in the dissipativity properties. It itoring. the plant proprocess dynamical behavior when a faultproperties. occurs may itoring. Considering the plant plant economy, industrial pro- of cesses are often operated near economy, their design constraints, itoring. Considering the industrial probe reflected by change in the dissipativity It be reflected by change in the dissipativity properties. It has been shown that dissipativity properties derived from cesses are often operated near their design constraints, reflected by change in the dissipativity properties. It cesses are often often operated near their design constraints, thus failure in fault detection may rapidly leadconstraints, to adverse be cesses are operated near their design has been shown that dissipativity properties derived has beenmodels shown can thatbe dissipativity properties derived from process used for fault detection (Leifrom and thus failure rapidly lead to been shown that dissipativity properties derived from thus failure in in fault fault detection may rapidly lead(2009), to adverse adverse consequences. It is detection reported may in Izadi et al. due has thus failure in fault detection may rapidly lead to adverse process models can be used for fault detection (Lei and process models can be used afor for fault detection detection (Lei and and Bao (2015)). In can this be paper, data-driven fault detection consequences. It in (2009), due models used fault (Lei consequences. It is is reported reported in Izadi Izadi et et al. al. (2009),alone due process to abnormal events, the US petrochemical industry consequences. It is reported in Izadi et al. (2009), due Bao (2015)). In this paper, aaondata-driven fault detection Bao (2015)). In this paper, data-driven fault detection approach is developed based the behavior of processes to abnormal events, the US petrochemical industry alone (2015)). In this paper, a data-driven fault detection to abnormal events, the the 20 USbillion petrochemical industry alone Bao suffered approximately USD in industry annual losses. to abnormal events, US petrochemical alone approach is based the process behavior of approach is developed developed based on on behaviorinput/output of processes processes captured by the dissipativity of the suffered approximately 20 billion in losses. is developed based on the behavior of processes suffered approximately 20 detection billion USD USD in annual annualsteadily losses. approach As may be expected, fault is becoming suffered approximately 20 billion USD in annual losses. captured by dissipativity of input/output captured by the the dissipativity of the the process process input/output trajectories, which are determined through a training As may be expected, fault detection is becoming steadily captured by the dissipativity of the process input/output As may be expected, fault detection is becoming steadily more crucial. As may be expected, fault detection is becoming steadily trajectories, which are determined training trajectories, which are data. determined through through a training procedure using process more trajectories, which are determined through aa training more crucial. crucial. more crucial. procedure using process data. Existing fault detection approaches can be virtually clas- procedure procedure using using process process data. data. Dissipativity of a dynamic process often captures limited Existing approaches can virtually clasExisting fault detection approaches can be be virtually clas- Dissipativity sified intofault twodetection categories, model based and data based Existing fault detection approaches can be virtually clasof dynamic often Dissipativity of aaa (e.g., dynamic process often captures captures limited process features the process information included limited in the sified into two categories, model based and data based of dynamic process often captures limited sified into two two categories, model based and data data based approaches. Considering it ismodel difficult to construct an based accu- Dissipativity sified into categories, based and process features (e.g., the information included in the process features (e.g., the information included in the traditional QSR dissipativity is limited). As such, QSRapproaches. Considering it is difficult to construct an accufeatures (e.g., the information included inQSRthe approaches. Considering it is is difficult difficult to construct construct an accuaccurate industrial process model, data based approaches have process approaches. Considering it to an traditional QSR dissipativity is limited). As such, traditional QSR dissipativity is limited). As such, QSRdissipativity can be too coarse for fault detection applicarate industrial process model, data based approaches have traditional QSR dissipativity is limited). As such, QSRrate industrial process model, model, data based approaches approaches have dissipativity can be too coarse for fault detection applicagained more attention from the industry. Various multirate industrial process data based have dissipativity can be be too coarse coarse for fault faultthe detection applications. It is shown in Chen et al. (2010), passivity condigained attention from the Various can too for detection applicagained more attention from the industry. industry. Various multivariate more statistical process monitoring methods havemultibeen dissipativity gained more attention from the industry. Various multitions. It is shown in Chen et al. (2010), the passivity conditions. It is shown in Chen et al. (2010), the passivity condition (a special case of dissipativity condition) was applied variate statistical process monitoring methods have been It is shown in of Chen et al. (2010), the passivity condivariate statistical process monitoring methodscomponents have been been tions. developed in the past few years, e.g., principal variate statistical process monitoring methods have tion (a case condition) was tion (a special special caseand of dissipativity dissipativity condition) was applied applied for fault detection the results can be very conservative. developed in the past few years, e.g., principal components tion (a special case of dissipativity condition) was applied developed in the past few years, e.g., principal components analysis (PCA) (Kresta et al. (1991)) , dynamic principal developed in the(Kresta past fewetyears, e.g., principal components for fault and results conservative. for fault detection detection and the thedissipativity, results can can be be very conservative. Moreover, in traditional thevery storage function analysis (PCA) dynamic principal fault detection and the results can be very conservative. analysis (PCA) (Kresta et al. al. (1991)) (1991)) dynamic principal for components analysis (DPCA) (Russell,,, et al. (2000)). analysis (PCA) (Kresta et al. (1991)) dynamic principal Moreover, in traditional dissipativity, the storage function Moreover, in traditional dissipativity, the storage function is defined on state variables. However state measurements components analysis (DPCA) (Russell et al. (2000)). Moreover, in traditional dissipativity, the storage function components analysis (DPCA) (Russell et al. (2000)). components analysis (DPCA) (Russell et al. (2000)). on state Another data-driven fault detection methods received at- is is defined on state state variables. variables. However stateofmeasurements measurements aredefined often unavailable, leadingHowever to the need a demanding is defined on state variables. However state measurements Another data-driven fault detection methods received atare often unavailable, leading to the need of aa demanding Another data-driven fault detection methods received attention isdata-driven support vector machine (SVM). Support vector are often unavailable, leading to the need of demanding task of state estimation. In this paper, storage functions Another fault detection methods received atare often unavailable, leading to the needstorage of a demanding tention vector (SVM). of estimation. In functions tention isissupport support vector machine machine (SVM). Support Support vector machineis a comparatively new machine learningvector algo- task task of state state estimation. In this this paper, paper, storage functions and supply rates are in Quadratic Difference Forms (QdF) tention is support vector machine (SVM). Support vector task of state estimation. In this paper, storage functions machine is machine learning supply rates are Difference (QdF) machine is a a comparatively comparatively new machine learning algorithm developed in Vapnik andnew Kotz (1982). The mainalgoidea and and supply rates are in in Quadratic Quadratic Difference Forms (QdF) (Willems and Trentelman (1998)) so that theyForms are function machine is a comparatively new machine learning algoand supply rates are in Quadratic Difference Forms (QdF) rithm developed Vapnik and Kotz The Trentelman (1998)) so they are rithm developed inan Vapnik andseparation Kotz (1982). (1982). The main main idea of SVM is to findin optimal boundary (inidea the (Willems (Willems and Trentelmanhistory. (1998)) QdF so that that they dissipativity are function function of processand input/output based rithm developed in Vapnik and Kotz (1982). The main idea (Willems and Trentelman (1998)) so that they are function of SVM is to find an optimal separation boundary (in the of process input/output history. QdF based dissipativity of SVM is to find an optimal separation boundary (in the form of a hyper-plane) in a way that the margin between of process input/output history. QdF based dissipativity may capture far more detailed process dynamic features of SVM is to find an optimal separation boundary (in the of process input/output history.process QdF based dissipativity form of in the between capture far detailed form of a a hyper-plane) hyper-plane) in a a way way that that the margin marginand between two sample classes is maximized (Mahadevan Shah may may capture far more more detailed (Tippett process dynamic dynamic features compared to QSR dissipativity and Bao features (2011)) form of a hyper-plane) in a way that the margin between may capture far more detailed process dynamic features two sample classes is maximized (Mahadevan and Shah compared to QSR dissipativity (Tippett and Bao (2011)) two sample classes is maximized (Mahadevan and Shah (2009)). There are reports of successful SVM application compared to QSR dissipativity (Tippett and Bao (2011)) and thus allows much better fault detection performance. two sample classes is maximized (Mahadevan and Shah compared to QSR dissipativity (Tippett and Bao (2011)) (2009)). of (2009)). There are reports of successful successful SVM application in variousThere cases,are e.g.reports in Mahadevan and SVM Shah application (2009) and and and thus thus allows allows much much better better fault fault detection detection performance. performance. (2009)). There are reports of successful SVM application and thus better fault detection performance. Similar toallows SVM,much the proposed method requires a training in various cases, e.g. in Mahadevan and Shah (2009) and in various cases, e.g. in Mahadevan and Shah (2009) and Widodo and Yang (2007). More details on SVM can be in various cases, e.g. in Mahadevan and Shah (2009) and Similar to SVM, the proposed method requires aa searches training Similar to SVM, the proposed method requires training procedure. In the off-line design stage, a algorithm Widodo and Yang (2007). More details on SVM can be to SVM, the proposed method requires a searches training Widodo and Yang (2007). More More details details on on SVM SVM can can be be Similar found in and Vapnik (2013). Widodo Yang (2007). procedure. In the off-line design stage, a algorithm procedure. In the off-line design stage, a algorithm searches found in Vapnik (2013). procedure. In the off-line design stage, a algorithm searches found in in Vapnik Vapnik (2013). (2013). found Copyright © 2016, 2016 IFAC 717 Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2016 IFAC 717 Copyright © 2016 IFAC 717 Peer review under responsibility of International Federation of Automatic Copyright © 2016 IFAC 717Control. 10.1016/j.ifacol.2016.07.266
IFAC DYCOPS-CAB, 2016 718 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
for the dissipativity properties of the process input output dissipative trajectories. The training is performed based on process input and output data. The on-line implementation involves only the evaluation of the dissipativity inequality which is computational tractable. A comparison study between SVM and proposed method is conducted, and the proposed method outperformed SVM. This paper is organised as follows. A brief introduction on dissipativity system theory and dissipative trajectory is given in Section 2. The techniques of proposed data-driven fault detection method are presented in Section 3, an illustrative example and comparison study is included in Section 3 as well. The proposed fault detection method is demonstrated using a case study on a mono-chlorobenzene plant simulated using VMGSim in Section 4, followed by the conclusion. 2. SYSTEM BEHAVIOUR AND DISSIPATIVE TRAJECTORY First developed as a framework for analysing dynamical systems, the dissipativity theory was initially proposed in Willems (1972). In this paper, the proposed datadriven fault detection approach is developed based on dissipativity inequality. A general process can be described by a state space equation as follows x(t + 1) = Ax(t) + Bu(t) (1) y(t) = Cx(t) + Du(t), where x ∈ X ⊂ Rn is state variable vector, u ∈ U ⊂ Rp is the input variable vector, and y ∈ Y ⊂ Rq is the output variable vector. If a function defined on process input output variables denoted as the supply rate s(u(t), y(t)), and a function defined on process states, denoted as the storage function V (x(t)) can be found, such that s(u(t), y(t)) ≥ V (x(t + 1)) − V (x(t)) (2) for all time steps (Willems (1972)), then the process in (1) is said to be dissipative with respect to that supply rate and storage function. A commonly used supply rate (QSR dissipativity) takes the following form s(u(t), y(t)) = y (t)Qy(t) + 2y (t)Su(t) + u (t)Ru(t). (3) However, the dissipativity properties of a system is generally difficult to determine from process input/output data as the dissipativity inequality (2) has to be satisfied for all input output trajectories (Tippett et al. (2013)). In this paper, a fault detection approach is developed based on the change of process behaviors which is manifested by input output trajectories, i.e., the dissipativity of the trajectories. The dissipative trajectory is defined as below. Definition 1 Dissipative Trajectory.(Tippett and Bao (2013)) An input/output trajectory of a process is said to be dissipative with respect to supply rate Qφc if, at all time instances k, the following inequality is satisfied: k Qφc (u(t), y(t)) ≥ 0, (4) Wk = t=0
718
where Qφc is the supply rate induced by Qc (ζ, η) Sc (ζ, η) . φc (ζ, η) = Sc (ζ, η) Rc (ζ, η)
(5)
The concept of a dissipative trajectory can be understood as a weaker condition of classical dissipativity, see Tippett and Bao (2013). The definition of dissipative trajectory does not explicitly require the dissipativity inequality to be satisfied for all possible input. The concept from behavioural approach (Willems (2007)) also motivates the idea of finding the dissipativity properties for the input output trajectory. The adoption of dissipative trajectory concept is justified as the process operating data are by and large a fraction of all possible trajectories. It is generally not feasible to acquire input output data for all possible input trajectories. However, using proposed method the dissipativity properties for certain dissipative input output trajectories can be found. Note that in this paper, both the supply rate and storage function are in quadratic difference form (QdF). The QdF supply rate QΦ (ˆ u(t), yˆ(t)) for discrete time system is initially proposed in Kaneko and Fujii (2000), and takes a following form ˜ S˜ yˆ(t) yˆ(t) Q QΦ (ˆ (6) u(t), yˆ(t)) = ˜ u ˆ(t) u ˆ(t) S˜ R with
u ˆ(t) = u (t) u (t + 1) . . . u (t + N ) yˆ(t) = y (t) y (t + 1) . . . y (t + N )
(7)
The QdF supply rate QΦ can be understood as an extension to the traditional QSR-dissipativity. As the QdF can be rewritten into a following form ˜ y (t) + 2ˆ ˜ u(t). QΦ (ˆ u(t), yˆ(t)) = yˆ (t)Qˆ y (t)S˜u ˆ(t) + u ˆ (t)Rˆ (8) In this work, storage functions are defined as QdFs of process input and output trajectory (denoted as QΨ (ˆ u(t), yˆ(t))). As such, the proposed method does not need state measurement or estimation, leading to less computational complexity. The dissipativity inequality in (2) can be easily written into a QdF form, based on extended input and output variables, as follows QΦ (ˆ u(t), yˆ(t)) ≥ QΨ (ˆ u(t + 1), yˆ(t + 1)) − QΨ (ˆ u(t), yˆ(t)). (9) A training algorithm is developed in the next section to search for the appropriate dissipativity properties (QΦ , QΨ ) based on process input output data. 3. DISSIPATIVITY TRAJECTORIES BASED FAULT DETECTION METHOD The basic idea of proposed method is to search for the dissipativity properties based on input and output data. The process input output data trace certain dissipative trajectories as the process is operated under nominal condition. When faults occur the trajectory is no longer dissipative, thus allow changes to be detected by checking the dissipativity properties.
IFAC DYCOPS-CAB, 2016 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
The decision making is based on the following dissipativity inequality. If the inequality holds, the data are tracing a dissipative trajectory, and the process is assumed to be operating normally. However, if the dissipativity condition is breached, the data are no longer tracing the dissipative trajectory and one can conclude there may be a fault in the process.
719
steamflow valve steam
condensate
u(t), yˆ(t)) ≥ QΨ (ˆ u(t + 1), yˆ(t + 1)) − QΨ (ˆ u(t), yˆ(t)) QΦ (ˆ (10) water
condensate valve
3.1 Off-line Training A recursive algorithm is developed to search for the appropriate dissipativity properties using process input output data. First, m training samples are collected into the extend input and output form. For example, set N = 2 in equation (7), this indicates the QdF order is 2. The order of QdF in this proposed method is chosen on a trial-and-error basis. One can increase the QdF order if low order can not deliver satisfactory performance. As more data been included into QdF, performance would be improved at the expense of computational efforts. Class label yi is also assigned to each sample when collecting the data, samples from normal operating condition are labelled as yi = 1 while samples from faulty condition are labelled as yi = −1.
Then the search starts with a initial guess of the properties (coefficients for QΦ , QΨ ), where prior knowledge on the process can be considered. If no prior knowledge is available, the initial guess can be a all-one matrix of proper dimension. The coefficient matrix for QΦ and QΨ , denoted as φ˜ and ψ˜ respectively, are in the following form ˜ S˜ ˜ Y˜ Q X φ˜ = ˜ ˜ , ψ˜ = ˜ ˜ (11) S R Y Z
The decision variables for the training algorithm are the ˜ S, ˜ R, ˜ X, ˜ Y˜ , Z. ˜ The dimension of matrices coefficients Q, these decision variables is a function of the QdF order N . The training algorithm updates the dissipativity properties in each iteration through minimization of a objective function θ. The objective function is used to evaluate if the input output data are tracking a dissipative trajectory. The objective function, denoted as θ, is simply the summation of all the scores αi , as follows m αi (12) θ= i=1
To calculate the objective function, one needs to check the dissipativity inequality for each training sample at each ˜ If the inequality (9) iteration (with temporary φ˜ and ψ). holds for data from nominal operating condition, assign a negative score αi = −a to that sample. If the inequality holds for data from faulty operating condition (which leads to a missed alarm), assign a positive score αi = b. Similarly, assign a negative score if the inequality does not hold for data from faulty condition (successful detection), positive score if the inequality does not hold for data from normal condition (false alarm). The scoring rules for each sample can be summarised as
719
Fig. 1. A Heat Exchanger Example (Isermann (2011)) αi = −a if yi = 1 and QΦ,i − ∆QΨ,i > 0 αi = b if yi = 1 and QΦ,i − ∆QΨ,i < 0 (13) αi = −a if yi = −1 and QΦ,i − ∆QΨ,i < 0 αi = b if yi = −1 and QΦ,i − ∆QΨ,i > 0 where QΦ,i − ∆QΨ,i denotes the dissipativity inequality in equation (9) for each sample i, and the values of a and b (both are positive real) are the tuning parameters of the training algorithm. The above scoring rules can be written into a more compact form as below (14) αi yi (QΦ,i − ∆QΨ,i ) < 0 In each iteration, the algorithm updates the decision variables. And the training can be organized as a minimization problem with a form given below m αi minimize θ = (15) i=1
subject to αi yi (QΦ,i − ∆QΨ,i ) < 0, i = 1, 2, . . . m In each iteration, the scores αi are calculated using the temporary dissipativity coefficients, and θ for that iteration is the summation of all αi in that iteration. Once the stopping criterion are satisfied (objective function θ hits certain value), the training algorithm is stopped and the dissipativity properties (QΦ , QΨ ) for a dissipative input output trajectory is found to be the properties in the last iteration. 3.2 On-line Implementation
At every sampling step t, evaluate the dissipativity inequality u(t), yˆ(t))−(QΨ (ˆ u(t+1), yˆ(t+1))−QΨ (ˆ u(t), yˆ(t))) ≥ 0 QΦ (ˆ (16) If (16) holds, then these process input output data are tracing a dissipative trajectory that determined in the offline training stage, and no fault is detected. Otherwise the trajectory is not dissipative and a fault has been detected. The on-line implementation of the proposed approach only involves the evaluation of a quadratic form of the processes inputs and outputs over a moving window. Thus the computational burden is reasonably tractable. 3.3 Illustrative Example A heat exchanger as shown in Fig. 1 is studied. In this process, steam mass flow ms is measured as input variable, and the fluid outlet temperature vo is measured as
IFAC DYCOPS-CAB, 2016 720 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
x1
Training Results
4
Maximum margin
2
1
0
−1
Φ
Ψ
Ψ
Q (t) − (Q (t+1) − Q (t))
3
−2
−3
Optimal hyper-plane 10
20
30
40
50
60
Samples
70
80
90
100
110
0
Fig. 2. Fault Detection Results
x2
Fig. 3. Example of classification
output variable, sampling period is 5 seconds. There is a frequently found fault for heat exchanger, i.e. air (inert gas) in steam space. Input output samples are obtained from simulation studies.
3.4 Support Vector Machine Comparison Study The proposed method requires a training procedure with a complexity similar to that of SVM. There are also many reported successful applications of SVM in fault detection. Thus a comparison study is done in this subsection. The basic concept of SVM is to find a separation boundary, such that the margin between two sample classes is maximized. For a typical two-class problem (i.e. classify normal and faulty condition), after training, the support vector machine can find a optimal hyper-plane that separates these two classes, see Abe (2005). As shown in Figure 3, the optimal plane is found such that the margin between two classes is maximized. The circles/squares with grey colour that are used in constructing the separation boundaries are called support vectors (SV). The SVs contain all information for constructing the separation plane. Thus a comparison study between SVM and proposed method is done. The training procedure of a SVM is performed using the same samples as previous subsection (input output data as in (7) with N = 2). And the SVM detection accuracy is 10% less than the proposed approach. 720
Y
The QdF order used for this example is 2, input output data are collected as in (7) with N = 2. To choose N = 2 is because with 2 extra step data, the proposed method is cable of delivering satisfactory detection results. Generally speaking, as more data been included into QdF, performance would be improved at the expense of computational efforts. While having large N benefits the performance, time needed for collecting the data would also increase, thus having N = 2 gives the balance between performance and efficiency. The training fault detection result can be seen in Fig. 2. The vertical axis represents the value for QΦ − ∆QΨ . According to (9), if QΦ − ∆QΨ is positive, the inequality holds and the process input output data are tracing a dissipative trajectory. When fault occurs (after the vertical dashed line), the data are no longer tracing the dissipative trajectory and QΦ −∆QΨ is negative. As can be seen from Fig. 2, the dissipativity properties are able to detect the change in operating condition.
3
Nominal Samples
3
Faulty Samples
3
2
2
2
1
1
1
0
0
0
Y
0
Y
−4
−1
−1
−1
−2
−2
−2
−3 −2
−1
0
U
1
2
−3 −2
−1
0
U
1
2
−3 −2
All Samples Faulty Samples Nominal Samples
−1
0
U
1
2
Fig. 4. Result using SVM (cross indicates misclassification) Table 1. Illustrative Example Result Summary Method SVM Dissipativity
Training Accuracy 91% 100%
Testing Accuracy 88% 98%
Results of SVM using input output at each step is shown in Fig. 4. The normal operating samples are marked as squares, whereas samples from faulty condition are marked as circles. The misclassified samples are marked by a cross in the centre. As shown in Figure 4, there are significant overlapping between the two classes, therefore it is difficult to acquire satisfactory result. This somehow explains why SVM is not performing well in this particular example. The fault detection performance of the dissipativity based and SVM based methods are summarised in Table 1. The dissipativity based method outperformed the SVM based method in terms of accuracy. 4. CASE STUDY ON MONO-CHLOROBENZENE PLANT A monochlorobenzene (MCB) plant is studied in this section. The plant is simulated using VMGSim, the process flowsheet is depicted in Fig. 5. A mixture of benzene, monochlorobenzene (MCB), and HCl is present in the feed of this MCB separation process. The vapour stream coming out of the flash tank is fed into the absorber (T1) where it is put into contact with recycled MCB. Most of the HCl product comes out of the absorber as vapour. The liquid product (L2) coming out of the absorber is mixed with liquid product (L1) from the flash vessel (F1) in a mixer (M1). The mixture coming out of the mixer (M1) is then fed into the distillation column (T2). In this column a fixed amount (1% of inlet feed to the
IFAC DYCOPS-CAB, 2016 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
721
Fig. 5. Mono-chlorobenzene plant under study
The feed flow rate is in constant oscillation, and the primary fault in this process takes place in the recycle valve (V-7). A change of flow rate through the valve is used to simulate a block in the valve. The initial fault in recycle valve changed the flow rate to around 90% of the set point. Due to the presence of other controllers, the system can recover from the change. Then the next fault in recycle valve changed the flow rate to around 40% of the nominal value. The system can not to recover from this change even with the help of other controllers. All other controllers was working fine except the recycle valve, which was too small to handle the required flow. We took 100 samples (50 nominal operating samples and 50 faulty operating samples) out of all the 2000 samples to do the training for both SVM and proposed method. The first fault occurred at around the 1000th samples (after the vertical dashed line in Fig. 6 and 7), while the second fault occurred at around the 1600th samples. Each sample contains the flow rate and composition from the input and output streams of the process (’Feed’, ’HCl’, ’Benzene’ and ’MCB’ in Fig. 5). It is assumed we do not have access to the measurements of the recycle stream where the fault had occurred. The SVM testing result is shown in Fig. 6, the dissipativity method result is shown in Fig. 7. The Y axis in Fig. 6 and 7 represents the class label, where +1 indicates nominal condition and -1 indicates faulty condition. A summary of these two methods is presented in Table 2. It is obvious that the proposed method outperformed the SVM based method, as the proposed method can detect a wider range of faulty operating condition. It is observed that both 721
Table 2. Case Study Result Summary Method SVM Dissipativity
Training Accuracy 100% 100%
Testing Accuracy 87.2% 93.6%
SVM Result
1.5
1
0.5
Class Label
plant) is purged from the process to avoid HCl build-up in the system. The distillate product (D) contains most of the benzene and bottom product (B) contains most of the MCB. A fraction of the bottom product stream is recycled back into the absorber.
0
−0.5
−1
−1.5
0
200
400
600
800
1000
Samples
1200
1400
1600
1800
Fig. 6. SVM Result methods did not perform well around the first 100 samples after the fault. This can be explained as, after the first stage of fault, the control system is able to rectify the changes in the process, for example, in Fig. 8 the benzene flow out of the system was temporarily restored to its nominal value. It also worth noting that the proposed method has better detection performance after the second fault, as in Fig. 6 one can observe a long series of missed alarms after the second fault around the 1600th samples 5. DISCUSSION AND CONCLUSION In this paper, an approach to train the dissipativty properties using process date is developed. The idea of finding the dissipativity properties for process input output trajectories leads to simplified fault detection approach. The efficacy of proposed method is demonstrated in a MCB plant case study, where the dissipativity motivated datadriven method outperformed SVM. The primary advantage of proposed approach is its simplicity, particularly in terms of on-line implementation.
IFAC DYCOPS-CAB, 2016 722 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
Dissipativity Result
1.5
1
Class Label
0.5
0
−0.5
−1
−1.5
0
200
400
600
800
1000
Samples
1200
1400
1600
1800
1600
1800
Fig. 7. Dissipativity Result Benzene − Molar Flow
25 24 23
kgmole/h
22 21 20 19 18 17 16 15
0
200
400
600
800
1000
Samples
1200
1400
Fig. 8. Benzene Flow Rate In this paper, once the dissipativity properties are found off-line, on-line evaluation of the dissipativity inequality can be readily carried out through simple calculations. The dissipativity properties also provide insights into the dynamic features of the processes, e.g. the gain, phase or their combination. Whilst SVM does not provide details of the physical features of that system. One possible extension to proposed method is data-driven fault diagnosis. The off-line training stage requires samples from both nominal and faulty operating condition, with modest alteration the proposed algorithm can be used to find dissipativity properties that are sensitivity to individual faults. Thus allow fault diagnosis to be performed based on process data. Another possible future extension is to develop an integrated fault detection and fault-tolerant control design structure. As reported in Bao et al. (2003), the dissipativity properties are used in fault-tolerant control design and showed promising results. There is also a general consensus in the process control community that fault detection and fault tolerant control should be integrated to offer the most efficient approach as stated in Mhaskar et al. (2012). REFERENCES Abe, S. (2005). Support vector machines for pattern classification, volume 2. Springer. Bao, J., Zhang, W.Z., and Lee, P. (2003). Decentralized fault-tolerant control system design for unstable processes. Chemical Engineering Science, 58(22), 5045– 5054. Chen, W., Ding, S., Khan, A., and Abid, M. (2010). Energy based fault detection for dissipative systems. In Control and Fault-Tolerant Systems (SysTol), 2010 Conference on, 517–521. IEEE. Isermann, R. (2011). Fault-diagnosis applications: modelbased condition monitoring: actuators, drives, ma722
chinery, plants, sensors, and fault-tolerant systems. Springer. Izadi, I., Shah, S.L., Shook, D.S., and Chen, T. (2009). An introduction to alarm analysis and design. In 7th IFAC International Symposium on Fault Detection, Supervision and Safety of Technical Systems, SAFEPROCESS’09, June 30, 2009 - July 3, 2009. Kaneko, O. and Fujii, T. (2000). Discrete-time average positivity and spectral factorization in a behavioral framework. Systems & control letters, 39(1), 31–44. Kresta, J.V., MacGregor, J.F., and Marlin, T.E. (1991). Multivariate statistical monitoring of process operating performance. The Canadian Journal of Chemical Engineering, 69(1), 35–47. Lei, Q. and Bao, J. (2015). Dissipativity based fault detection and diagnosis for linear systems. In 2015 IEEE Conference on Control Applications (CCA), Sydney, Australia. Mahadevan, S. and Shah, S.L. (2009). Fault detection and diagnosis in process data using one-class support vector machines. Journal of Process Control, 19(10), 1627–1639. Mhaskar, P., Liu, J., and Christofides, P.D. (2012). Fault-tolerant process control: methods and applications. Springer Science & Business Media. Rojas, O.J., Bao, J., and Lee, P.L. (2008). On dissipativity, passivity and dynamic operability of nonlinear processes. Journal of Process Control, 18(5), 515–526. Russell, E.L., Chiang, L.H., and Braatz, R.D. (2000). Fault detection in industrial processes using canonical variate analysis and dynamic principal component analysis. Chemometrics and Intelligent Laboratory Systems, 51(1), 81–93. Tippett, M.J. and Bao, J. (2011). Dissipativity based analysis using dynamic supply rates. In Proceedings of the 18th IFAC World Congress, volume 28. Tippett, M.J. and Bao, J. (2013). Distributed model predictive control based on dissipativity. AIChE Journal, 59(3), 787–804. doi:10.1002/aic.13868. Tippett, M.J., Hioe, D., and Bao, J. (2013). Integrated approach to identification and control of multivariable processes based on dissipativity. In Thermodynamic Foundations of Mathematical Systems Theory, volume 1, 54–59. Vapnik, V. (2013). The nature of statistical learning theory. Springer Science & Business Media. Vapnik, V.N. and Kotz, S. (1982). Estimation of dependences based on empirical data, volume 40. Springerverlag New York. Widodo, A. and Yang, B.S. (2007). Support vector machine in machine condition monitoring and fault diagnosis. Mechanical Systems and Signal Processing, 21(6), 2560–2574. Willems, J.C. and Trentelman, H.L. (1998). On quadratic differential forms. SIAM Journal on Control and Optimization, 36(5), 1703–1749. Willems, J.C. (1972). Dissipative dynamical systems part i: General theory. Archive for rational mechanics and analysis, 45(5), 321–351. Willems, J.C. (2007). The behavioral approach to open and interconnected systems. Control Systems, IEEE, 27(6), 46–99.
ScienceDirect
A A A
IFAC-PapersOnLine 49-7 (2016) 717–722
Data-driven Fault Detection Method Data-driven Fault Method Data-driven Fault Detection Detection Method Based on Dissipative Trajectories Based on Dissipative Trajectories Based ∗on Dissipative Trajectories ∗∗ ∗
Qingyang Lei ∗ Muhammad Tajammal Munir ∗∗ Jie Bao ∗ ∗∗ Jie Bao ∗ ∗∗ Qingyang Lei ∗∗ Muhammad Muhammad Tajammal Munir Qingyang Tajammal Munir Brent Young Qingyang Lei Lei Muhammad Tajammal Munir ∗∗ Jie Jie Bao Bao ∗ ∗∗ ∗∗ Brent Young ∗∗ Brent Young Brent Young ∗ University of New South Wales, Sydney, NSW, 2052, Australia ∗ ∗ University of New South Wales, Sydney, NSW, 2052, Australia ∗ University of New South Wales, (e-mail: University of New South [email protected]). Wales, Sydney, Sydney, NSW, NSW, 2052, 2052, Australia Australia ∗∗ (e-mail: [email protected]). (e-mail: [email protected]). University of Auckland, Auckland, 1010, New Zealand (e-mail: (e-mail: [email protected]). ∗∗ ∗∗ University of Auckland, Auckland, 1010, New Zealand (e-mail: ∗∗ University of Auckland, Auckland, 1010, New Zealand (e-mail: [email protected]). University of Auckland, Auckland, 1010, New Zealand (e-mail: [email protected]). [email protected]). [email protected]). Abstract: Fault detection is becoming increasingly important as the complexity of industrial Abstract: Fault detection becoming increasingly important as the complexity of industrial Abstract: Fault is becoming increasingly important as complexity of process develops. In this is paper, a data-driven fault detection method is proposed. The Abstract: Fault detection detection is becoming increasingly important as the the complexity of industrial industrial process develops. In this paper, a data-driven fault detection method is proposed. The process develops. In this paper, a data-driven fault detection method is proposed. The dissipativity theory is adopted to find the appropriate dissipativity properties for the process process develops. Inis this paper, a data-driven fault dissipativity detection method is proposed. The dissipativity to properties for dissipativity theory is adopted adopted to find find the the appropriate appropriate dissipativity properties for the the process process input outputtheory trajectory. The dissipativity properties can be viewed as an abstract energy dissipativity theory is adopted to find the appropriate dissipativity properties for the process input output trajectory. The dissipativity properties be viewed as an process abstract energy input output trajectory. The dissipativity properties can be as abstract energy property, and the dissipativity properties of input outputcan trajectories represent dynamic input output trajectory. The properties dissipativity properties can be viewed viewed as an an process abstractdynamic energy property, and the dissipativity of input output trajectories represent property, and the dissipativity properties of input output trajectories represent process dynamic features. As faults occur, the dissipative trajectories will change thus allow fault detection to be property, and the dissipativity properties of input output trajectories represent process dynamic features. As faults occur, the dissipative trajectories will change thus allow fault detection to be features. As faults occur, the dissipative trajectories will change thus allow fault detection to performed based on these dissipativity properties. A training algorithm is developed to search features. Asbased faultson occur, the dissipativeproperties. trajectoriesAwill changealgorithm thus allow fault detection to be be performed these dissipativity training is developed to search performed based on these dissipativity properties. A training algorithm is developed to search for the related properties using input output data.A A prior knowledge of developed the process can be performed based on these dissipativity properties. training algorithm is to search for the related properties using input output A prior of the process can be for the related properties using input output data. A prior knowledge of the process can be incorporated into the algorithm to facilitate thedata. training. Theknowledge proposed fault detection method for the related properties using input output data. A prior knowledge of the process can be incorporated into the facilitate the training. The proposed fault detection method incorporated into algorithm to facilitate The proposed fault detection is illustrated on casealgorithm study of ato mono-chlorobenzene plant simulated using VMGSim. incorporated intoaa the the algorithm tomono-chlorobenzene facilitate the the training. training. Thesimulated proposedusing fault VMGSim. detection method method is illustrated on case study of a plant is illustrated on a case study of a mono-chlorobenzene plant simulated using VMGSim. is illustrated on a case study of a mono-chlorobenzene plant simulated using VMGSim. © 2016, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Dissipativity Theory, Dissipative Trajectory, Fault Detection, Data-driven. Keywords: Keywords: Dissipativity Dissipativity Theory, Theory, Dissipative Dissipative Trajectory, Trajectory, Fault Fault Detection, Detection, Data-driven. Data-driven. Keywords: Dissipativity Theory, Dissipative Trajectory, Fault Detection, Data-driven. 1. INTRODUCTION First developed as a framework for analysing dynamical 1. First developed as for dynamical 1. INTRODUCTION INTRODUCTION First developed as aaa framework framework for analysing analysing dynamical systems, the dissipativity theory was initially proposed in 1. INTRODUCTION First developed as framework for analysing dynamical systems, the dissipativity theory was initially proposed in systems, the dissipativity theory was initially proposed in Willems (1972). The dissipativity properties represent the systems, the dissipativity theory was initiallyrepresent proposedthe in Willems (1972). dissipativity properties Willems (1972). The The dissipativity properties represent the process dynamics features, e.g. the gain, phase or their (1972). The dissipativity properties represent the The growing complexity of modern industrial processes Willems process dynamics features, e.g. the the As gain, phase orchange their process dynamics features, e.g. gain, phase their combination (Rojas et al. (2008)). such, the or The growing complexity of processes dynamics features, e.g. the As gain, phase orchange their The growing complexity of modern modern industrial processes present a serious challenge for real industrial time process mon- process The growing complexity of modern industrial processes combination (Rojas et al. (2008)). such, the combination (Rojas et al. (2008)). As such, the change of process dynamical behavior when a fault occurs may present a serious challenge for real time process moncombination (Rojas et al. (2008)). As such, the change present a serious challenge for real time process monitoring. Considering the plant economy, industrial propresent aConsidering serious challenge for economy, real timeindustrial process monof process dynamical behavior when aa fault occurs may of process dynamical behavior when fault occurs may be reflected by change in the dissipativity properties. It itoring. the plant proprocess dynamical behavior when a faultproperties. occurs may itoring. Considering the plant plant economy, industrial pro- of cesses are often operated near economy, their design constraints, itoring. Considering the industrial probe reflected by change in the dissipativity It be reflected by change in the dissipativity properties. It has been shown that dissipativity properties derived from cesses are often operated near their design constraints, reflected by change in the dissipativity properties. It cesses are often often operated near their design constraints, thus failure in fault detection may rapidly leadconstraints, to adverse be cesses are operated near their design has been shown that dissipativity properties derived has beenmodels shown can thatbe dissipativity properties derived from process used for fault detection (Leifrom and thus failure rapidly lead to been shown that dissipativity properties derived from thus failure in in fault fault detection may rapidly lead(2009), to adverse adverse consequences. It is detection reported may in Izadi et al. due has thus failure in fault detection may rapidly lead to adverse process models can be used for fault detection (Lei and process models can be used afor for fault detection detection (Lei and and Bao (2015)). In can this be paper, data-driven fault detection consequences. It in (2009), due models used fault (Lei consequences. It is is reported reported in Izadi Izadi et et al. al. (2009),alone due process to abnormal events, the US petrochemical industry consequences. It is reported in Izadi et al. (2009), due Bao (2015)). In this paper, aaondata-driven fault detection Bao (2015)). In this paper, data-driven fault detection approach is developed based the behavior of processes to abnormal events, the US petrochemical industry alone (2015)). In this paper, a data-driven fault detection to abnormal events, the the 20 USbillion petrochemical industry alone Bao suffered approximately USD in industry annual losses. to abnormal events, US petrochemical alone approach is based the process behavior of approach is developed developed based on on behaviorinput/output of processes processes captured by the dissipativity of the suffered approximately 20 billion in losses. is developed based on the behavior of processes suffered approximately 20 detection billion USD USD in annual annualsteadily losses. approach As may be expected, fault is becoming suffered approximately 20 billion USD in annual losses. captured by dissipativity of input/output captured by the the dissipativity of the the process process input/output trajectories, which are determined through a training As may be expected, fault detection is becoming steadily captured by the dissipativity of the process input/output As may be expected, fault detection is becoming steadily more crucial. As may be expected, fault detection is becoming steadily trajectories, which are determined training trajectories, which are data. determined through through a training procedure using process more trajectories, which are determined through aa training more crucial. crucial. more crucial. procedure using process data. Existing fault detection approaches can be virtually clas- procedure procedure using using process process data. data. Dissipativity of a dynamic process often captures limited Existing approaches can virtually clasExisting fault detection approaches can be be virtually clas- Dissipativity sified intofault twodetection categories, model based and data based Existing fault detection approaches can be virtually clasof dynamic often Dissipativity of aaa (e.g., dynamic process often captures captures limited process features the process information included limited in the sified into two categories, model based and data based of dynamic process often captures limited sified into two two categories, model based and data data based approaches. Considering it ismodel difficult to construct an based accu- Dissipativity sified into categories, based and process features (e.g., the information included in the process features (e.g., the information included in the traditional QSR dissipativity is limited). As such, QSRapproaches. Considering it is difficult to construct an accufeatures (e.g., the information included inQSRthe approaches. Considering it is is difficult difficult to construct construct an accuaccurate industrial process model, data based approaches have process approaches. Considering it to an traditional QSR dissipativity is limited). As such, traditional QSR dissipativity is limited). As such, QSRdissipativity can be too coarse for fault detection applicarate industrial process model, data based approaches have traditional QSR dissipativity is limited). As such, QSRrate industrial process model, model, data based approaches approaches have dissipativity can be too coarse for fault detection applicagained more attention from the industry. Various multirate industrial process data based have dissipativity can be be too coarse coarse for fault faultthe detection applications. It is shown in Chen et al. (2010), passivity condigained attention from the Various can too for detection applicagained more attention from the industry. industry. Various multivariate more statistical process monitoring methods havemultibeen dissipativity gained more attention from the industry. Various multitions. It is shown in Chen et al. (2010), the passivity conditions. It is shown in Chen et al. (2010), the passivity condition (a special case of dissipativity condition) was applied variate statistical process monitoring methods have been It is shown in of Chen et al. (2010), the passivity condivariate statistical process monitoring methodscomponents have been been tions. developed in the past few years, e.g., principal variate statistical process monitoring methods have tion (a case condition) was tion (a special special caseand of dissipativity dissipativity condition) was applied applied for fault detection the results can be very conservative. developed in the past few years, e.g., principal components tion (a special case of dissipativity condition) was applied developed in the past few years, e.g., principal components analysis (PCA) (Kresta et al. (1991)) , dynamic principal developed in the(Kresta past fewetyears, e.g., principal components for fault and results conservative. for fault detection detection and the thedissipativity, results can can be be very conservative. Moreover, in traditional thevery storage function analysis (PCA) dynamic principal fault detection and the results can be very conservative. analysis (PCA) (Kresta et al. al. (1991)) (1991)) dynamic principal for components analysis (DPCA) (Russell,,, et al. (2000)). analysis (PCA) (Kresta et al. (1991)) dynamic principal Moreover, in traditional dissipativity, the storage function Moreover, in traditional dissipativity, the storage function is defined on state variables. However state measurements components analysis (DPCA) (Russell et al. (2000)). Moreover, in traditional dissipativity, the storage function components analysis (DPCA) (Russell et al. (2000)). components analysis (DPCA) (Russell et al. (2000)). on state Another data-driven fault detection methods received at- is is defined on state state variables. variables. However stateofmeasurements measurements aredefined often unavailable, leadingHowever to the need a demanding is defined on state variables. However state measurements Another data-driven fault detection methods received atare often unavailable, leading to the need of aa demanding Another data-driven fault detection methods received attention isdata-driven support vector machine (SVM). Support vector are often unavailable, leading to the need of demanding task of state estimation. In this paper, storage functions Another fault detection methods received atare often unavailable, leading to the needstorage of a demanding tention vector (SVM). of estimation. In functions tention isissupport support vector machine machine (SVM). Support Support vector machineis a comparatively new machine learningvector algo- task task of state state estimation. In this this paper, paper, storage functions and supply rates are in Quadratic Difference Forms (QdF) tention is support vector machine (SVM). Support vector task of state estimation. In this paper, storage functions machine is machine learning supply rates are Difference (QdF) machine is a a comparatively comparatively new machine learning algorithm developed in Vapnik andnew Kotz (1982). The mainalgoidea and and supply rates are in in Quadratic Quadratic Difference Forms (QdF) (Willems and Trentelman (1998)) so that theyForms are function machine is a comparatively new machine learning algoand supply rates are in Quadratic Difference Forms (QdF) rithm developed Vapnik and Kotz The Trentelman (1998)) so they are rithm developed inan Vapnik andseparation Kotz (1982). (1982). The main main idea of SVM is to findin optimal boundary (inidea the (Willems (Willems and Trentelmanhistory. (1998)) QdF so that that they dissipativity are function function of processand input/output based rithm developed in Vapnik and Kotz (1982). The main idea (Willems and Trentelman (1998)) so that they are function of SVM is to find an optimal separation boundary (in the of process input/output history. QdF based dissipativity of SVM is to find an optimal separation boundary (in the form of a hyper-plane) in a way that the margin between of process input/output history. QdF based dissipativity may capture far more detailed process dynamic features of SVM is to find an optimal separation boundary (in the of process input/output history.process QdF based dissipativity form of in the between capture far detailed form of a a hyper-plane) hyper-plane) in a a way way that that the margin marginand between two sample classes is maximized (Mahadevan Shah may may capture far more more detailed (Tippett process dynamic dynamic features compared to QSR dissipativity and Bao features (2011)) form of a hyper-plane) in a way that the margin between may capture far more detailed process dynamic features two sample classes is maximized (Mahadevan and Shah compared to QSR dissipativity (Tippett and Bao (2011)) two sample classes is maximized (Mahadevan and Shah (2009)). There are reports of successful SVM application compared to QSR dissipativity (Tippett and Bao (2011)) and thus allows much better fault detection performance. two sample classes is maximized (Mahadevan and Shah compared to QSR dissipativity (Tippett and Bao (2011)) (2009)). of (2009)). There are reports of successful successful SVM application in variousThere cases,are e.g.reports in Mahadevan and SVM Shah application (2009) and and and thus thus allows allows much much better better fault fault detection detection performance. performance. (2009)). There are reports of successful SVM application and thus better fault detection performance. Similar toallows SVM,much the proposed method requires a training in various cases, e.g. in Mahadevan and Shah (2009) and in various cases, e.g. in Mahadevan and Shah (2009) and Widodo and Yang (2007). More details on SVM can be in various cases, e.g. in Mahadevan and Shah (2009) and Similar to SVM, the proposed method requires aa searches training Similar to SVM, the proposed method requires training procedure. In the off-line design stage, a algorithm Widodo and Yang (2007). More details on SVM can be to SVM, the proposed method requires a searches training Widodo and Yang (2007). More More details details on on SVM SVM can can be be Similar found in and Vapnik (2013). Widodo Yang (2007). procedure. In the off-line design stage, a algorithm procedure. In the off-line design stage, a algorithm searches found in Vapnik (2013). procedure. In the off-line design stage, a algorithm searches found in in Vapnik Vapnik (2013). (2013). found Copyright © 2016, 2016 IFAC 717 Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2016 IFAC 717 Copyright © 2016 IFAC 717 Peer review under responsibility of International Federation of Automatic Copyright © 2016 IFAC 717Control. 10.1016/j.ifacol.2016.07.266
IFAC DYCOPS-CAB, 2016 718 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
for the dissipativity properties of the process input output dissipative trajectories. The training is performed based on process input and output data. The on-line implementation involves only the evaluation of the dissipativity inequality which is computational tractable. A comparison study between SVM and proposed method is conducted, and the proposed method outperformed SVM. This paper is organised as follows. A brief introduction on dissipativity system theory and dissipative trajectory is given in Section 2. The techniques of proposed data-driven fault detection method are presented in Section 3, an illustrative example and comparison study is included in Section 3 as well. The proposed fault detection method is demonstrated using a case study on a mono-chlorobenzene plant simulated using VMGSim in Section 4, followed by the conclusion. 2. SYSTEM BEHAVIOUR AND DISSIPATIVE TRAJECTORY First developed as a framework for analysing dynamical systems, the dissipativity theory was initially proposed in Willems (1972). In this paper, the proposed datadriven fault detection approach is developed based on dissipativity inequality. A general process can be described by a state space equation as follows x(t + 1) = Ax(t) + Bu(t) (1) y(t) = Cx(t) + Du(t), where x ∈ X ⊂ Rn is state variable vector, u ∈ U ⊂ Rp is the input variable vector, and y ∈ Y ⊂ Rq is the output variable vector. If a function defined on process input output variables denoted as the supply rate s(u(t), y(t)), and a function defined on process states, denoted as the storage function V (x(t)) can be found, such that s(u(t), y(t)) ≥ V (x(t + 1)) − V (x(t)) (2) for all time steps (Willems (1972)), then the process in (1) is said to be dissipative with respect to that supply rate and storage function. A commonly used supply rate (QSR dissipativity) takes the following form s(u(t), y(t)) = y (t)Qy(t) + 2y (t)Su(t) + u (t)Ru(t). (3) However, the dissipativity properties of a system is generally difficult to determine from process input/output data as the dissipativity inequality (2) has to be satisfied for all input output trajectories (Tippett et al. (2013)). In this paper, a fault detection approach is developed based on the change of process behaviors which is manifested by input output trajectories, i.e., the dissipativity of the trajectories. The dissipative trajectory is defined as below. Definition 1 Dissipative Trajectory.(Tippett and Bao (2013)) An input/output trajectory of a process is said to be dissipative with respect to supply rate Qφc if, at all time instances k, the following inequality is satisfied: k Qφc (u(t), y(t)) ≥ 0, (4) Wk = t=0
718
where Qφc is the supply rate induced by Qc (ζ, η) Sc (ζ, η) . φc (ζ, η) = Sc (ζ, η) Rc (ζ, η)
(5)
The concept of a dissipative trajectory can be understood as a weaker condition of classical dissipativity, see Tippett and Bao (2013). The definition of dissipative trajectory does not explicitly require the dissipativity inequality to be satisfied for all possible input. The concept from behavioural approach (Willems (2007)) also motivates the idea of finding the dissipativity properties for the input output trajectory. The adoption of dissipative trajectory concept is justified as the process operating data are by and large a fraction of all possible trajectories. It is generally not feasible to acquire input output data for all possible input trajectories. However, using proposed method the dissipativity properties for certain dissipative input output trajectories can be found. Note that in this paper, both the supply rate and storage function are in quadratic difference form (QdF). The QdF supply rate QΦ (ˆ u(t), yˆ(t)) for discrete time system is initially proposed in Kaneko and Fujii (2000), and takes a following form ˜ S˜ yˆ(t) yˆ(t) Q QΦ (ˆ (6) u(t), yˆ(t)) = ˜ u ˆ(t) u ˆ(t) S˜ R with
u ˆ(t) = u (t) u (t + 1) . . . u (t + N ) yˆ(t) = y (t) y (t + 1) . . . y (t + N )
(7)
The QdF supply rate QΦ can be understood as an extension to the traditional QSR-dissipativity. As the QdF can be rewritten into a following form ˜ y (t) + 2ˆ ˜ u(t). QΦ (ˆ u(t), yˆ(t)) = yˆ (t)Qˆ y (t)S˜u ˆ(t) + u ˆ (t)Rˆ (8) In this work, storage functions are defined as QdFs of process input and output trajectory (denoted as QΨ (ˆ u(t), yˆ(t))). As such, the proposed method does not need state measurement or estimation, leading to less computational complexity. The dissipativity inequality in (2) can be easily written into a QdF form, based on extended input and output variables, as follows QΦ (ˆ u(t), yˆ(t)) ≥ QΨ (ˆ u(t + 1), yˆ(t + 1)) − QΨ (ˆ u(t), yˆ(t)). (9) A training algorithm is developed in the next section to search for the appropriate dissipativity properties (QΦ , QΨ ) based on process input output data. 3. DISSIPATIVITY TRAJECTORIES BASED FAULT DETECTION METHOD The basic idea of proposed method is to search for the dissipativity properties based on input and output data. The process input output data trace certain dissipative trajectories as the process is operated under nominal condition. When faults occur the trajectory is no longer dissipative, thus allow changes to be detected by checking the dissipativity properties.
IFAC DYCOPS-CAB, 2016 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
The decision making is based on the following dissipativity inequality. If the inequality holds, the data are tracing a dissipative trajectory, and the process is assumed to be operating normally. However, if the dissipativity condition is breached, the data are no longer tracing the dissipative trajectory and one can conclude there may be a fault in the process.
719
steamflow valve steam
condensate
u(t), yˆ(t)) ≥ QΨ (ˆ u(t + 1), yˆ(t + 1)) − QΨ (ˆ u(t), yˆ(t)) QΦ (ˆ (10) water
condensate valve
3.1 Off-line Training A recursive algorithm is developed to search for the appropriate dissipativity properties using process input output data. First, m training samples are collected into the extend input and output form. For example, set N = 2 in equation (7), this indicates the QdF order is 2. The order of QdF in this proposed method is chosen on a trial-and-error basis. One can increase the QdF order if low order can not deliver satisfactory performance. As more data been included into QdF, performance would be improved at the expense of computational efforts. Class label yi is also assigned to each sample when collecting the data, samples from normal operating condition are labelled as yi = 1 while samples from faulty condition are labelled as yi = −1.
Then the search starts with a initial guess of the properties (coefficients for QΦ , QΨ ), where prior knowledge on the process can be considered. If no prior knowledge is available, the initial guess can be a all-one matrix of proper dimension. The coefficient matrix for QΦ and QΨ , denoted as φ˜ and ψ˜ respectively, are in the following form ˜ S˜ ˜ Y˜ Q X φ˜ = ˜ ˜ , ψ˜ = ˜ ˜ (11) S R Y Z
The decision variables for the training algorithm are the ˜ S, ˜ R, ˜ X, ˜ Y˜ , Z. ˜ The dimension of matrices coefficients Q, these decision variables is a function of the QdF order N . The training algorithm updates the dissipativity properties in each iteration through minimization of a objective function θ. The objective function is used to evaluate if the input output data are tracking a dissipative trajectory. The objective function, denoted as θ, is simply the summation of all the scores αi , as follows m αi (12) θ= i=1
To calculate the objective function, one needs to check the dissipativity inequality for each training sample at each ˜ If the inequality (9) iteration (with temporary φ˜ and ψ). holds for data from nominal operating condition, assign a negative score αi = −a to that sample. If the inequality holds for data from faulty operating condition (which leads to a missed alarm), assign a positive score αi = b. Similarly, assign a negative score if the inequality does not hold for data from faulty condition (successful detection), positive score if the inequality does not hold for data from normal condition (false alarm). The scoring rules for each sample can be summarised as
719
Fig. 1. A Heat Exchanger Example (Isermann (2011)) αi = −a if yi = 1 and QΦ,i − ∆QΨ,i > 0 αi = b if yi = 1 and QΦ,i − ∆QΨ,i < 0 (13) αi = −a if yi = −1 and QΦ,i − ∆QΨ,i < 0 αi = b if yi = −1 and QΦ,i − ∆QΨ,i > 0 where QΦ,i − ∆QΨ,i denotes the dissipativity inequality in equation (9) for each sample i, and the values of a and b (both are positive real) are the tuning parameters of the training algorithm. The above scoring rules can be written into a more compact form as below (14) αi yi (QΦ,i − ∆QΨ,i ) < 0 In each iteration, the algorithm updates the decision variables. And the training can be organized as a minimization problem with a form given below m αi minimize θ = (15) i=1
subject to αi yi (QΦ,i − ∆QΨ,i ) < 0, i = 1, 2, . . . m In each iteration, the scores αi are calculated using the temporary dissipativity coefficients, and θ for that iteration is the summation of all αi in that iteration. Once the stopping criterion are satisfied (objective function θ hits certain value), the training algorithm is stopped and the dissipativity properties (QΦ , QΨ ) for a dissipative input output trajectory is found to be the properties in the last iteration. 3.2 On-line Implementation
At every sampling step t, evaluate the dissipativity inequality u(t), yˆ(t))−(QΨ (ˆ u(t+1), yˆ(t+1))−QΨ (ˆ u(t), yˆ(t))) ≥ 0 QΦ (ˆ (16) If (16) holds, then these process input output data are tracing a dissipative trajectory that determined in the offline training stage, and no fault is detected. Otherwise the trajectory is not dissipative and a fault has been detected. The on-line implementation of the proposed approach only involves the evaluation of a quadratic form of the processes inputs and outputs over a moving window. Thus the computational burden is reasonably tractable. 3.3 Illustrative Example A heat exchanger as shown in Fig. 1 is studied. In this process, steam mass flow ms is measured as input variable, and the fluid outlet temperature vo is measured as
IFAC DYCOPS-CAB, 2016 720 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
x1
Training Results
4
Maximum margin
2
1
0
−1
Φ
Ψ
Ψ
Q (t) − (Q (t+1) − Q (t))
3
−2
−3
Optimal hyper-plane 10
20
30
40
50
60
Samples
70
80
90
100
110
0
Fig. 2. Fault Detection Results
x2
Fig. 3. Example of classification
output variable, sampling period is 5 seconds. There is a frequently found fault for heat exchanger, i.e. air (inert gas) in steam space. Input output samples are obtained from simulation studies.
3.4 Support Vector Machine Comparison Study The proposed method requires a training procedure with a complexity similar to that of SVM. There are also many reported successful applications of SVM in fault detection. Thus a comparison study is done in this subsection. The basic concept of SVM is to find a separation boundary, such that the margin between two sample classes is maximized. For a typical two-class problem (i.e. classify normal and faulty condition), after training, the support vector machine can find a optimal hyper-plane that separates these two classes, see Abe (2005). As shown in Figure 3, the optimal plane is found such that the margin between two classes is maximized. The circles/squares with grey colour that are used in constructing the separation boundaries are called support vectors (SV). The SVs contain all information for constructing the separation plane. Thus a comparison study between SVM and proposed method is done. The training procedure of a SVM is performed using the same samples as previous subsection (input output data as in (7) with N = 2). And the SVM detection accuracy is 10% less than the proposed approach. 720
Y
The QdF order used for this example is 2, input output data are collected as in (7) with N = 2. To choose N = 2 is because with 2 extra step data, the proposed method is cable of delivering satisfactory detection results. Generally speaking, as more data been included into QdF, performance would be improved at the expense of computational efforts. While having large N benefits the performance, time needed for collecting the data would also increase, thus having N = 2 gives the balance between performance and efficiency. The training fault detection result can be seen in Fig. 2. The vertical axis represents the value for QΦ − ∆QΨ . According to (9), if QΦ − ∆QΨ is positive, the inequality holds and the process input output data are tracing a dissipative trajectory. When fault occurs (after the vertical dashed line), the data are no longer tracing the dissipative trajectory and QΦ −∆QΨ is negative. As can be seen from Fig. 2, the dissipativity properties are able to detect the change in operating condition.
3
Nominal Samples
3
Faulty Samples
3
2
2
2
1
1
1
0
0
0
Y
0
Y
−4
−1
−1
−1
−2
−2
−2
−3 −2
−1
0
U
1
2
−3 −2
−1
0
U
1
2
−3 −2
All Samples Faulty Samples Nominal Samples
−1
0
U
1
2
Fig. 4. Result using SVM (cross indicates misclassification) Table 1. Illustrative Example Result Summary Method SVM Dissipativity
Training Accuracy 91% 100%
Testing Accuracy 88% 98%
Results of SVM using input output at each step is shown in Fig. 4. The normal operating samples are marked as squares, whereas samples from faulty condition are marked as circles. The misclassified samples are marked by a cross in the centre. As shown in Figure 4, there are significant overlapping between the two classes, therefore it is difficult to acquire satisfactory result. This somehow explains why SVM is not performing well in this particular example. The fault detection performance of the dissipativity based and SVM based methods are summarised in Table 1. The dissipativity based method outperformed the SVM based method in terms of accuracy. 4. CASE STUDY ON MONO-CHLOROBENZENE PLANT A monochlorobenzene (MCB) plant is studied in this section. The plant is simulated using VMGSim, the process flowsheet is depicted in Fig. 5. A mixture of benzene, monochlorobenzene (MCB), and HCl is present in the feed of this MCB separation process. The vapour stream coming out of the flash tank is fed into the absorber (T1) where it is put into contact with recycled MCB. Most of the HCl product comes out of the absorber as vapour. The liquid product (L2) coming out of the absorber is mixed with liquid product (L1) from the flash vessel (F1) in a mixer (M1). The mixture coming out of the mixer (M1) is then fed into the distillation column (T2). In this column a fixed amount (1% of inlet feed to the
IFAC DYCOPS-CAB, 2016 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
721
Fig. 5. Mono-chlorobenzene plant under study
The feed flow rate is in constant oscillation, and the primary fault in this process takes place in the recycle valve (V-7). A change of flow rate through the valve is used to simulate a block in the valve. The initial fault in recycle valve changed the flow rate to around 90% of the set point. Due to the presence of other controllers, the system can recover from the change. Then the next fault in recycle valve changed the flow rate to around 40% of the nominal value. The system can not to recover from this change even with the help of other controllers. All other controllers was working fine except the recycle valve, which was too small to handle the required flow. We took 100 samples (50 nominal operating samples and 50 faulty operating samples) out of all the 2000 samples to do the training for both SVM and proposed method. The first fault occurred at around the 1000th samples (after the vertical dashed line in Fig. 6 and 7), while the second fault occurred at around the 1600th samples. Each sample contains the flow rate and composition from the input and output streams of the process (’Feed’, ’HCl’, ’Benzene’ and ’MCB’ in Fig. 5). It is assumed we do not have access to the measurements of the recycle stream where the fault had occurred. The SVM testing result is shown in Fig. 6, the dissipativity method result is shown in Fig. 7. The Y axis in Fig. 6 and 7 represents the class label, where +1 indicates nominal condition and -1 indicates faulty condition. A summary of these two methods is presented in Table 2. It is obvious that the proposed method outperformed the SVM based method, as the proposed method can detect a wider range of faulty operating condition. It is observed that both 721
Table 2. Case Study Result Summary Method SVM Dissipativity
Training Accuracy 100% 100%
Testing Accuracy 87.2% 93.6%
SVM Result
1.5
1
0.5
Class Label
plant) is purged from the process to avoid HCl build-up in the system. The distillate product (D) contains most of the benzene and bottom product (B) contains most of the MCB. A fraction of the bottom product stream is recycled back into the absorber.
0
−0.5
−1
−1.5
0
200
400
600
800
1000
Samples
1200
1400
1600
1800
Fig. 6. SVM Result methods did not perform well around the first 100 samples after the fault. This can be explained as, after the first stage of fault, the control system is able to rectify the changes in the process, for example, in Fig. 8 the benzene flow out of the system was temporarily restored to its nominal value. It also worth noting that the proposed method has better detection performance after the second fault, as in Fig. 6 one can observe a long series of missed alarms after the second fault around the 1600th samples 5. DISCUSSION AND CONCLUSION In this paper, an approach to train the dissipativty properties using process date is developed. The idea of finding the dissipativity properties for process input output trajectories leads to simplified fault detection approach. The efficacy of proposed method is demonstrated in a MCB plant case study, where the dissipativity motivated datadriven method outperformed SVM. The primary advantage of proposed approach is its simplicity, particularly in terms of on-line implementation.
IFAC DYCOPS-CAB, 2016 722 June 6-8, 2016. NTNU, Trondheim, Norway Qingyang Lei et al. / IFAC-PapersOnLine 49-7 (2016) 717–722
Dissipativity Result
1.5
1
Class Label
0.5
0
−0.5
−1
−1.5
0
200
400
600
800
1000
Samples
1200
1400
1600
1800
1600
1800
Fig. 7. Dissipativity Result Benzene − Molar Flow
25 24 23
kgmole/h
22 21 20 19 18 17 16 15
0
200
400
600
800
1000
Samples
1200
1400
Fig. 8. Benzene Flow Rate In this paper, once the dissipativity properties are found off-line, on-line evaluation of the dissipativity inequality can be readily carried out through simple calculations. The dissipativity properties also provide insights into the dynamic features of the processes, e.g. the gain, phase or their combination. Whilst SVM does not provide details of the physical features of that system. One possible extension to proposed method is data-driven fault diagnosis. The off-line training stage requires samples from both nominal and faulty operating condition, with modest alteration the proposed algorithm can be used to find dissipativity properties that are sensitivity to individual faults. Thus allow fault diagnosis to be performed based on process data. Another possible future extension is to develop an integrated fault detection and fault-tolerant control design structure. As reported in Bao et al. (2003), the dissipativity properties are used in fault-tolerant control design and showed promising results. There is also a general consensus in the process control community that fault detection and fault tolerant control should be integrated to offer the most efficient approach as stated in Mhaskar et al. (2012). REFERENCES Abe, S. (2005). Support vector machines for pattern classification, volume 2. Springer. Bao, J., Zhang, W.Z., and Lee, P. (2003). Decentralized fault-tolerant control system design for unstable processes. Chemical Engineering Science, 58(22), 5045– 5054. Chen, W., Ding, S., Khan, A., and Abid, M. (2010). Energy based fault detection for dissipative systems. In Control and Fault-Tolerant Systems (SysTol), 2010 Conference on, 517–521. IEEE. Isermann, R. (2011). Fault-diagnosis applications: modelbased condition monitoring: actuators, drives, ma722
chinery, plants, sensors, and fault-tolerant systems. Springer. Izadi, I., Shah, S.L., Shook, D.S., and Chen, T. (2009). An introduction to alarm analysis and design. In 7th IFAC International Symposium on Fault Detection, Supervision and Safety of Technical Systems, SAFEPROCESS’09, June 30, 2009 - July 3, 2009. Kaneko, O. and Fujii, T. (2000). Discrete-time average positivity and spectral factorization in a behavioral framework. Systems & control letters, 39(1), 31–44. Kresta, J.V., MacGregor, J.F., and Marlin, T.E. (1991). Multivariate statistical monitoring of process operating performance. The Canadian Journal of Chemical Engineering, 69(1), 35–47. Lei, Q. and Bao, J. (2015). Dissipativity based fault detection and diagnosis for linear systems. In 2015 IEEE Conference on Control Applications (CCA), Sydney, Australia. Mahadevan, S. and Shah, S.L. (2009). Fault detection and diagnosis in process data using one-class support vector machines. Journal of Process Control, 19(10), 1627–1639. Mhaskar, P., Liu, J., and Christofides, P.D. (2012). Fault-tolerant process control: methods and applications. Springer Science & Business Media. Rojas, O.J., Bao, J., and Lee, P.L. (2008). On dissipativity, passivity and dynamic operability of nonlinear processes. Journal of Process Control, 18(5), 515–526. Russell, E.L., Chiang, L.H., and Braatz, R.D. (2000). Fault detection in industrial processes using canonical variate analysis and dynamic principal component analysis. Chemometrics and Intelligent Laboratory Systems, 51(1), 81–93. Tippett, M.J. and Bao, J. (2011). Dissipativity based analysis using dynamic supply rates. In Proceedings of the 18th IFAC World Congress, volume 28. Tippett, M.J. and Bao, J. (2013). Distributed model predictive control based on dissipativity. AIChE Journal, 59(3), 787–804. doi:10.1002/aic.13868. Tippett, M.J., Hioe, D., and Bao, J. (2013). Integrated approach to identification and control of multivariable processes based on dissipativity. In Thermodynamic Foundations of Mathematical Systems Theory, volume 1, 54–59. Vapnik, V. (2013). The nature of statistical learning theory. Springer Science & Business Media. Vapnik, V.N. and Kotz, S. (1982). Estimation of dependences based on empirical data, volume 40. Springerverlag New York. Widodo, A. and Yang, B.S. (2007). Support vector machine in machine condition monitoring and fault diagnosis. Mechanical Systems and Signal Processing, 21(6), 2560–2574. Willems, J.C. and Trentelman, H.L. (1998). On quadratic differential forms. SIAM Journal on Control and Optimization, 36(5), 1703–1749. Willems, J.C. (1972). Dissipative dynamical systems part i: General theory. Archive for rational mechanics and analysis, 45(5), 321–351. Willems, J.C. (2007). The behavioral approach to open and interconnected systems. Control Systems, IEEE, 27(6), 46–99.