- Email: [email protected]
A Causality Capturing Method for Diagnosis Based on Transfer Entropy by Analyzing Trends of Time Series
9th IFAC Symposium on Fault Detection, Supervision and 9th IFAC on Fault Detection, Supervision and Safety of Symposium Technical Processes 9th IFAC on Fault Detection, Supervision and Available online at www.sciencedirect.com Safety of Symposium Technical Processes 9th IFAC on Fault Detection, Supervision September 2-4, 2015. Arts et Métiers ParisTech, Paris, and France Safety of Symposium Technical Processes September 2-4, 2015. Arts et Métiers ParisTech, Paris, France Safety of Technical Processes September 2-4, 2015. Arts et Métiers ParisTech, Paris, France September 2-4, 2015. Arts et Métiers ParisTech, Paris, France
ScienceDirect
IFAC-PapersOnLine 48-21 (2015) 778–783
A Causality Capturing Method for Diagnosis Based on A Causality Capturing Method for Diagnosis Based on A Causality Capturing Method for Diagnosis Based on Transfer Entropy by Analyzing Trends of Time Series A Causality Capturing Method for Diagnosis Based on Transfer Entropy by Analyzing Trends of Time Series Transfer Entropy by Analyzing Trends of Time Series Transfer EntropyCen byGuo, Analyzing Fan Yang, Trends Weijun Yuof Time Series
Cen Guo, Guo, Fan Fan Yang, Yang, Weijun Weijun Yu Yu Cen Cen Guo, Fan Yang, Weijun Yu Tsinghua Laboratory for Information Science and Technology, Tsinghua Laboratory for Information ScienceUniversity, and Technology, Department of Automation, Tsinghua Tsinghua Laboratory for Information Science and Tsinghua Laboratory for Information ScienceUniversity, and Technology, Technology, Department of(e-mails: Automation, Tsinghua Beijing 100084, China [email protected], Department of Automation, Tsinghua University, Department of Automation, Tsinghua University, Beijing 100084, China (e-mails: [email protected], [email protected], [email protected]) Beijing China Beijing 100084, 100084, China (e-mails: (e-mails: [email protected], [email protected], [email protected], [email protected], [email protected]) [email protected]) [email protected], [email protected]) Abstract: Since modern industrial processes become much larger and more complex, efficient and Abstract: Since much larger more complex, efficient effective causality detectionindustrial methods processes are neededbecome to capture the process topology, causesand of Abstract: Since modern modern industrial processes become much larger and and more diagnose complex, root efficient and Abstract: Since modern industrial processes become much larger and more complex, efficient and effective causality detection methods are needed to capture the process topology, diagnose root causes widespread or even plant-wide process malfunction, andthe further ensure the safety of root processes. A effective causality detection methods are needed to capture process topology, diagnose causes of of effective causality detection methods are needed to capture process topology, diagnose root causes of widespread or even plant-wide process malfunction, andthe further ensure the safety of which processes. A modified transfer entropy method, named trend transfer entropy, is proposed in this paper, focuses widespread or even plant-wide process malfunction, and further ensure the safety of processes. A widespread ortrends even malfunction, andseries further ensure the safety of processes. A modified transfer entropy method, named transfer entropy, isthemselves proposed inand this paper, which focuses on analyzing ofplant-wide time seriesprocess rather trend than the original thus, compared to the modified transfer entropy method, named trend transfer entropy, is proposed in this paper, which focuses modified transfer entropy method, named trend transfer entropy, is proposed in this paper, which focuses on analyzing trends of time series rather than the original series themselves and thus, compared to the traditional transfer proves to be more robust in conditions of data drifting and noise disturbance. on analyzing trendsentropy, of series rather than the series themselves and thus, compared to the on analyzing of time timeproves series rather thanrobust the original original series themselves and thus, compared to the the traditional transfer to be computational more inload conditions of datasaving drifting and noise Moreover, thetrends newentropy, method can reduce effectively, valuable timedisturbance. before traditional transfer entropy, proves to be more robust in conditions of data drifting and noise disturbance. traditional transfer entropy, proves to be more robust in conditions of data drifting and noise disturbance. Moreover, new methodSimulation can reducestudies computational load to effectively, saving valuable before occurrence the of an accident. are presented illustrate the procedure andtime features of the the Moreover, the new method can computational load effectively, saving valuable time before the Moreover,method. the new methodSimulation can reduce reducestudies computational load to effectively, saving valuable before occurrence of an accident. are presented illustrate the procedure andtime features of the proposed occurrence of occurrencemethod. of an an accident. accident. Simulation Simulation studies studies are are presented presented to to illustrate illustrate the the procedure procedure and and features features of of the the proposed proposed method. Keywords: Causality detection, Transfer entropy, Trend analysis, Fault diagnosis © 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. proposed method. Keywords: Keywords: Causality Causality detection, detection, Transfer Transfer entropy, entropy, Trend Trend analysis, analysis, Fault Fault diagnosis diagnosis Keywords: Causality detection, Transfer entropy, Trend analysis, Fault diagnosis
1. INTRODUCTION 1. INTRODUCTION 1. INTRODUCTION As modern industrial processes become increasingly large 1. INTRODUCTION As processes become increasingly large and modern complex,industrial the relationship within process variables is far As modern industrial processes become increasingly large As modern industrial processes become increasingly large and the within process variables far more complicated than before. This may result inis and complex, complex, the relationship relationship within process variables is the far and complex, the relationship within process variables is far more complicated than before. This may result in the occurrence of ‘alarmthan flood’, a phenomenon instantaneous more complicated before. This mayof result in the more complicated than before. This may result in the occurrence of ‘alarm flood’, a phenomenon of instantaneous emergence massive caused ofbyinstantaneous redundant occurrence ofof‘alarm flood’,alarms a phenomenon occurrence ofof‘alarm flood’, a phenomenon ofbyinstantaneous emergence alarms caused redundant configurations of massive alarms as well as fault propagation through emergence of massive alarms caused by redundant emergence of massive alarms caused by redundant configurations of alarms as well as fault propagation through the interconnected network of process components. In order configurations of alarms as well as fault propagation through configurations of alarms as well as fault propagation through the interconnected network of process components. In order to avoid undesirable consequences andcomponents. ensure the safety of the interconnected network of process In the interconnected network of process components. In order order to avoid undesirable consequences and ensure efficient the safety of process operation in the current situation, and to avoid undesirable consequences and ensure the safety of to avoid operation undesirable and ensure thetosafety of process inconsequences the current situation, efficient and effective causality detection methods are needed capture process operation in the current situation, efficient and process operation in the current situation, efficient and effective causality detection methods are needed to capture the process topology and thus diagnose root to causes of effective causality detection methods are needed capture effective causality methods are needed to(Yang capture the and diagnose root widespread evendetection plant-wide process malfunction et the process processortopology topology and thus thus diagnose root causes causes of of the processExisting topology and thus root causes of widespread or process malfunction (Yang et al, 2014). techniques for diagnose causality capture can be widespread or even even plant-wide plant-wide process malfunction (Yang et widespread or even plant-wide process malfunction (Yang et al, 2014). Existing techniques for causality capture can be generally divided into two types,for namely, process knowledge al, 2014). Existing techniques causality capture can be al, 2014). Existing techniques for causality capture can be generally divided into two types, namely, process knowledge based methods andinto process data based methods. The former generally divided two types, namely, process knowledge generally divided tworeachability types, namely, process knowledge based methods andinto process data based methods. The former includes adjacency and matrices (Jiang et al., based methods and process data based methods. The based methods and process data based methods. The former former includes adjacency andgraph reachability matrices (Jiang et 2009), signed directed based methods (Vedam et al., al., includes adjacency and reachability matrices (Jiang et al., includes adjacency andgraph reachability matrices (Jiang et al., 2009), signed directed based methods (Vedam 1997; Venkatasubramanian et al., 2000; Yang et al., 2012; 2009), signed directed graph based methods (Vedam et al., 2009), signed directed based methods (Vedam etweb al., 1997; Venkatasubramanian et 2000; et Yang et al., 2013), andgraph plant topology extraction 1997; Venkatasubramanian et al., al., 2000; Yang Yang et al., al.,by2012; 2012; 1997; Venkatasubramanian et al., 2000; Yang et al., 2012; Yang et al., 2013), and plant topology extraction by web language (Yim2013), et al.,and 2006; Thambirajah al., 2009), Yang et al., plant topology et extraction bywhile web Yang et al., 2013), and plant topology analysis, extraction bywhile web language (Yim et Thambirajah the latter contains cross-correlation Granger language (Yim et al., al., 2006; 2006; Thambirajah et et al., al., 2009), 2009), while language (Yim et al., 2006; Thambirajah et al., 2009), while the latter contains cross-correlation analysis, Granger causality analysis (Granger, 1969), Bayesian network the latter contains cross-correlation analysis, Granger the latter(Cowell contains cross-correlation analysis,(Schreiber, Granger causality analysis (Granger, 1969), entropy Bayesian network learning et al., 1999), transfer causality analysis (Granger, 1969), Bayesian network causality analysis (Granger, 1969), Bayesian network learning (Cowell et al., 1999), transfer entropy (Schreiber, 2000) (TE), and other entropy based methods (Doyle et al., learning (Cowell et al., 1999), transfer entropy (Schreiber, learning (Cowell et toal.,achieve 1999),better transfer entropy (Schreiber, 2000) (TE), and other entropy based methods (Doyle et al., 1993a, b). Besides, diagnosis performance, 2000) (TE), and other entropy based methods (Doyle et al., 2000) (TE), and other entropy based diagnosis methods (Doyle etwith al., 1993a, b). Besides, to achieve better performance, especially when abnormal situations are associated 1993a, b). Besides, to achieve better diagnosis performance, 1993a, b). Besides, to achieve better diagnosis performance, especially when abnormal situations are associated with unknown multiple situations faults, integrated methods with that especially faults when orabnormal are associated especially when abnormal situations are associated with unknown faults multiple faults, integrated methods that combine analysis process knowledge unknown process faults or or data multiple faults,with integrated methods that unknown faults or multiple faults, integrated methods that combine process data analysis with process knowledge extraction are proposed and demonstrated be quite combine process data analysis with processto knowledge combine process data analysis with process knowledge extraction are proposed and demonstrated to be quite effective (Chiang et al., 2003). extraction are and extraction are proposed proposed and demonstrated demonstrated to to be be quite quite effective (Chiang et al., 2003). effective (Chiang et al., 2003). Transfer entropy etisal.,an2003). information-theoretic method for effective (Chiang Transfer entropy information-theoretic for causality measurement, unlike most of themethod other dataTransfer entropy is is an anwhich, information-theoretic method for Transfer entropy is anwhich, information-theoretic method for causality measurement, unlike most of the other datacausality measurement, which, unlike most of the other datacausality measurement, which, unlike most of the other data-
based methods, is free from model assumptions and based methods, is situations. free fromTomodel applicable to various extend assumptions the applicationand of based methods, is free from model assumptions and based methods, is situations. free fromhave assumptions and applicable toimproved various Tomodel extend the application of TE, several methods been developed. Direct applicable to situations. To the of applicableentropy to various various Totoextend extend the application application of TE, methods been developed. transfer wassituations. proposedhave distinguish directDirect and TE, several several improved improved methods have been developed. Direct TE, several improved methods have been developed. Direct transfer entropy was proposed to distinguish direct and indirect among excluding transfer causal entropyrelationships was proposed to variables distinguishby direct and transfer entropy wasrest proposed to variables distinguish and indirect causal among excluding the effects fromrelationships the of the process (Duanby etdirect al., 2013; indirect causal relationships among variables by excluding indirect causal relationships among variables by excluding the effects from the rest of the process (Duan et al., 2013; Duan et al., 2015); symbolic transfer (Duan entropyet calculates the effects from the rest of process al., 2013; the effects from the rest of the the of process (Duan et calculates al., 2013; Duan et ofal., 2015); symbolic transfer entropy symbols time series instead original values to reduce Duan et al., 2015); symbolic transfer entropy calculates Duan et al., 2015); symbolic transfer entropy calculates symbols of time series instead of original values to reduce noise influence (Staniek et al., 2008); partialvalues transfer symbols of series instead of to entropy reduce symbols of time time(Staniek instead of original original reduce noise influence et al., 2008); partialvalues transfer entropy on rank vectors is,series to some extent, a combination ofto the two noise influence (Staniek et al., 2008); partial transfer entropy noise influence (Staniek et al., 2008); partial transfer entropy on rank vectors is, and to some extent, a have combination of the two methods above, proves to both advantages on rank is, to some extent, aa combination of the on rank vectors vectors is, and toHowever, some extent, combination offocus the two two methods proves to both (Kugiumtzis, 2013). these methods on methods above, above, and provesfew to ofhave have both advantages advantages methods above, and proves to have both advantages (Kugiumtzis, 2013). However, few of these methods focus alleviating the2013). computational is the bottleneck (Kugiumtzis, However, load, few ofwhich these methods focus on on (Kugiumtzis, 2013). However, load, few these methods focus on alleviating the computational which is bottleneck problem when TEoffor causality in alleviating the implementing computationalthe load, which is the thedetection bottleneck alleviating the cases, computational load, which thedetection bottleneck problem when implementing TE for in real industrial since it the requires a causality largeis amount of data problem when implementing the TE for causality detection in problem when implementing the TE for causality detection in real industrial cases, since it requires a large amount of data to improve thecases, accuracy. Init some cases, the amount pursuit of new real industrial since requires a large of data real industrial cases, since it requires a large amount data to improve the accuracy. In some cases, the pursuit of new functionalities requires more computing resources and to improve the In some cases, pursuit to improve time, the accuracy. accuracy. In low somediagnosis cases, the the pursuit of of new new functionalities requires more computing resources and processing leading to effectiveness and functionalities requires more computing resources and functionalities requires more computing resources and processing time, leadingtoto low diagnosis effectiveness unsatisfactory response process fault from the perspective processing time, leading to low diagnosis effectiveness and processing time, leading lowcomputational diagnosis effectiveness anda unsatisfactory response totoprocess fault from the perspective of process safety. Although the burden is not unsatisfactory response to process fault from the perspective unsatisfactory response to process fault from the perspective of process safety. Although the computational burden is not bigprocess issue safety. for theAlthough offline analysis of causality capture, of the computational burden is not itaa of process safety. Although the computational burden is not ita big issue for the offline analysis of causality capture, prevents from being usedofforcausality online diagnosis, big issuetheformethod the offline analysis capture, in it big issue for the offline analysis of causality capture, it prevents the method from being used for online diagnosis, in particularthe formethod novel fault diagnosis with varied causality, and prevents from being used for online diagnosis, in prevents the method from being used for online diagnosis, in particular for novel fault diagnosis with varied causality, and other applications suchdiagnosis as control loop causality, configuration particular for novel fault with and particular for fault with varied varied and other applications suchdiagnosis as control loop causality, configuration (Whiteman et novel al., 2014). other applications such as control loop configuration other applications such as control loop configuration (Whiteman et al., 2014). (Whiteman et 2014). In addition to all the improved approaches mentioned above, (Whiteman et al., al., 2014). In addition to all the improved approaches above, transfer entropy was also applied to binary mentioned alarm sequences, In addition to all the improved approaches mentioned above, In addition to all the improved approaches mentioned above, transfer entropy was also applied to binary alarm sequences, which aims at reducing computational burden and can be transfer entropy was also applied to binary alarm sequences, transfer entropy was also applied to binary alarm sequences, which aims at reducing computational burden and can be utilized in multivariate alarm analysis andburden design and (Duan et al., which aims at reducing computational can be which Yu aims at reducing computational can be utilized in et multivariate alarm analysis andburden design and (Duan et on al., 2014; al., 2014). Yet the technique depends heavily utilized in multivariate alarm analysis and design (Duan et al., utilized in multivariate alarm analysis and design (Duan et al., 2014; Yuof et al., 2014). Yet the heavily on settings alarm limits andtechnique loses a depends large amount of 2014; Yu et al., 2014). Yet technique depends heavily on 2014; Yuof et in al., 2014). Yet the the technique heavily on settings alarm limits and loses a depends large toamount of information process data. Thus it is sensitive noise and settings of alarm limits and loses aa large amount of settings of alarm limits and loses large amount of information in process data. Thus it is sensitive to noise and has a deteriorated performance. In general, better information in process data. Thus it is sensitive to noise and information in process data. Thus it is sensitive to noise and has a deteriorated performance. In general, better has a deteriorated performance. In general, better has a deteriorated performance. In general, better
2405-8963 © 2015, IFAC (International Federation of Automatic Control) Copyright 2015 IFAC 778Hosting by Elsevier Ltd. All rights reserved. Peer review©under of International Federation of Automatic Copyright 2015 responsibility IFAC 778Control. Copyright © 2015 IFAC 778 10.1016/j.ifacol.2015.09.621 Copyright © 2015 IFAC 778
SAFEPROCESS 2015 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
performance and wide applications of TE require a more comprehensive approach with feasible computational load.
couple of times using surrogate data generated by randomly disorganizing the sequence of time series for each time, and then their mean is compared with the TE value obtained from the calculation of the original series. If two values are almost equal, we can say that there is no significant causal relationship between two series, since the causality is destroyed in disordered data. Therefore, the average TE value can be used as a threshold after adding some standard deviations (Bauer et al., 2007). For the concern of convenience, TE value in this paper is the remains after subtraction of a threshold and 3 standard deviations.
A new method based on the modification of TE, named trend transfer entropy (TTE), is proposed in this paper. When computing TTE, the trends of time series rather than series themselves are considered. Accordingly, the TTE method presents the capability of reducing influence from noise and data drifting, and performs well when dealing with mass data from long time series. It should be noted that TTE is not a substitute of TE, but a supplement of the conventional method to reveal and reflect causal relationship among process variables from a novel perspective.
3. TREND TRANSFER ENTROPY
The remaining part of this paper is organized as follows: Section 2 describes TE and its significance test method; Section 3 introduces the proposed TTE method; Section 4 presents simulation studies to illustrate the effectiveness of the proposed approach; followed by conclusions in Section 5. 2. BASICS OF TRANSFER ENTROPY 2.1 Introduction to Transfer Entropy TE is used to test the causality between time series by measuring the reduction of information uncertainty. Its basic definition is given by (1): (𝑘𝑘)
(𝑙𝑙)
𝑇𝑇𝑇𝑇𝐽𝐽→𝐼𝐼 = ∑ 𝑝𝑝(𝑖𝑖𝑡𝑡+ℎ , 𝑖𝑖𝑡𝑡 , 𝑗𝑗𝑡𝑡 )log 2
(𝑘𝑘) (𝑙𝑙)
𝑝𝑝(𝑖𝑖𝑡𝑡+ℎ ,|𝑖𝑖𝑡𝑡 ,𝑗𝑗𝑡𝑡 ) (𝑘𝑘)
𝑝𝑝(𝑖𝑖𝑡𝑡+ℎ |𝑖𝑖𝑡𝑡 )
779
(1)
where I and J denote two time series; p means the complete or conditional probability density function (PDF); subscripts t and h represent the current time t and the prediction horizon h; superscripts k and l represent k and l continuous historical values before the subscript time. The estimation of PDFs can be obtained by statistical histograms using historical process data, or through kernel methods which are more accurate in specific cases. Although PDF estimation is surely nontrivial for the calculation of TE, it is not a prime focus in this paper. The equation indicates if there exists causality from J to I, it will be helpful to predict the value of I by additionally using the historical data of J. To be specific, the value of the numerator should be larger than that of the denominator, and the equation value should be larger than 0 if J affects I. On the contrary, the equation value should be close to 0, if no causal relationship exists from J to I.
In real cases of modern industrial processes, it is commonly known that values of variables can be influenced from the cause variable to the effect variable. Given that most variable values are approximately continuous and seldom mutated, their changes during a short period of time can be considered linear. Thus, if there exists causality between two variables, the linear value change of the cause one will quite possibly lead to a regulated value change of the other, no matter if the corresponding effects appear instantly or subsequently, or if the value of the effect variable reacts linearly or follows a certain pattern. Otherwise, they are considered independent. If changes of variable values which are regarded as trends can be properly used, we can reveal causal relationship from a novel perspective of trend causality, comparing to the traditional view of value causality. This is the main idea of the TTE method. There are generally three steps when computing TTE. The first is original series processing by means of piecewise linearization to extract trend information conveyed in slopes; the second is cloud classification to generate symbolic trend series; and the last is to calculate the TE value of the new symbolic trend series and test the significance. 3.1 Piecewise Linearization
2.2 Significance Test
We implement piecewise linearization on the original time series to extract trend information conveyed in slopes and meanwhile reduce the effects of noise and data drifting. In view of calculation convenience, feature-based piecewise linearization methods usually perform better than fittingbased approaches, e.g., the piecewise least squares method, within an acceptable range of errors. Thus, we prefer to adopt a feature-based method here. The term feature refers to the points that generally determine the shape of a series. These points include two different types: extreme points and turning points. Extreme points locate between two segments with slopes that have different signs; turning points separate segments that have the same sign but relatively obvious difference in their slopes. Examples of points from two types are shown in Fig. 1.
Considering the existence of noise, it is natural that the TE value of two time series does not equal to 0 exactly. Therefore, a threshold to distinguish real causality from false results caused by noise interference is of great necessity. Monte-Carlo examination was suggested to be a method of significance test (Kantz, 1997). TE values are calculated for a
Fig. 1 Segments of time series with extreme points in (1) (2) and turning points in (3)-(6).
Two conditions should be satisfied before utilizing TE: a sufficient length of series and the stationarity of series. The former indicates that the accuracy of TE relates positively with the size of data to a certain degree, while the latter reflects the limitation of TE, especially for the implementation in practical industrial processes.
779
SAFEPROCESS 2015 780 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
It is comparatively easier to figure out extreme points in time series than to select an appropriate number of turning points, because using different thresholds to determine the degree of obviousness leads to diverse collections of turning points. Thus, the extent to which the new series formed by feature points fits the original one can be adjusted via the threshold according to the knowledge of real process states.
reflects the fuzziness of the numerical range of concepts; 𝐻𝐻e , on the other hand, measures the uncertainty of the entropy, describing the cohesiveness of the drops in a cloud. A basic normal cloud generator was presented in (Russell et al., 2003). The algorithm is as follows:
Another problem which requires special attention is that, since time series obtained from industrial processes are usually affected by noise, feature points may retain that information without any filtration if simply keeping all of them intact. Therefore, the selection of feature points is necessary in order to reduce noise influence and reflect real changes of process variable values. A selection algorithm is presented below towards this issue:
Step 2: Generate a normal random number 𝑥𝑥 with 𝐸𝐸x as expectation and the absolute value of 𝐸𝐸n as standard deviation. 𝑥𝑥 is a cloud drop in the discourse domain.
Step 1: Generate a normal random number 𝐸𝐸n′ with 𝐸𝐸n as expectation and 𝐻𝐻e as standard deviation.
Step 3: Calculate
𝑦𝑦 = exp {−
(𝑥𝑥− 𝐸𝐸x )2 ′ )2 2( 𝐸𝐸n
} ,
where
y
is
membership of x belonging to a certain concept. Step 4: Repeat steps 1 to 3 until N number of cloud drops are generated.
Step 1: Search for feature points in the order from the starting point to the ending point of time series.
Two parameters should be especially concerned here – the length of each fundamental segment that can be represented by one symbol and the size of the symbol collection. Assuming the length of the original time series as L, we divide it into n isometric segments with length l, which satisfies the equation 𝐿𝐿 = 𝑛𝑛𝑛𝑛. Accordingly, one symbol in the trend series corresponds to l time intervals in the original series. In order that a maximum segment length can be effectively represented by a single symbol and the whole series data can be well compressed, l is set to be the approximate greatest common divisor of distances between each adjacent pair of feature points. As for the size of the symbol collection, it should meet the demand of analysis precision in the real processes. As shown in Figs. 3 and 4, the more subtly we classify those segments, the larger the size of the symbol collection is.
Step 2: If two adjacent feature points are within a predefined threshold, e.g., 10 time intervals, the latter one is considered invalid. Step 3: Go back to step 1 unless the whole series is processed. The main purpose of this algorithm is to guarantee necessary space between each pair of adjacent feature points, so that points caused by noise will mostly not be included in the final collection. In Fig. 2, the solid line is the original time series, while the dashed line is the reconstructed series based on selected feature points with the threshold of adjacency to be 5 time intervals.
Fig. 2 Original time series (solid line) and processed time series by piecewise linearization (dashed line). Fig. 3 Segments classification by descriptions of natural language.
3.2 Cloud Classification After the feature-based piecewise linearization on the original time series, we use a cloud model for symbol assignment towards the reconstructed series, because it can express the fuzziness and randomness in the classification by converting the continuous slopes to natural language expressions, such as ‘ascend’, ‘stable’, and ‘descend’. Define T as a linguistic value in the discourse domain of u. For mapping 𝐶𝐶𝑇𝑇 (𝑥𝑥): 𝑢𝑢 → [0,1] , ∀𝑥𝑥 ∈ 𝑢𝑢, 𝑥𝑥 → 𝐶𝐶𝑇𝑇 (𝑥𝑥). The distribution of 𝐶𝐶𝑇𝑇 (𝑥𝑥) in u is called a membership cloud of T. If 𝐶𝐶𝑇𝑇 (𝑥𝑥) follows a Gaussian distribution, the model is further named as a normal cloud (Dai et al., 2007). Three numerical characteristics form the basic features of the normal cloud model: expectation 𝐸𝐸x , entropy 𝐸𝐸n , and excess entropy 𝐻𝐻e . 𝐸𝐸x is the expectation of cloud drops in the discourse domain and the best representation of a qualitative concept; 𝐸𝐸n
Fig. 4 Normal cloud classification model with cloud classes 1-5 from left to right corresponding to ‘severely ascend’, ‘slowly ascend’, ‘stable’, ‘slowly descend’, and ‘severely descend’, respectively. In each cloud, a small range of membership values is assigned to a single slope angle, but only one value is generated in one realization. The final class 780
SAFEPROCESS 2015 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
of a slope depends on the cloud attribution of its largest membership value.
781
tube of input. Therefore, the level of Tank 1 affects that of Tank 2, but Tank 2 has no influence on Tank 1.
Once the length of every fundamental segment and the size of the symbol collection are properly set, we assign different symbols to all the classes in the normal cloud model. For example, we set the symbol of a segment with slope -60 degree to be 1 and the symbol for slope -30 degree to be 2 (Fig. 4). If there are segments that cover small pieces with more than one slope, the slope of the whole segment is given as 𝑘𝑘𝑖𝑖 by the probability of 𝑥𝑥𝑖𝑖 /𝑙𝑙, where 𝑥𝑥𝑖𝑖 is the length of the piece with the slope 𝑘𝑘𝑖𝑖 . When the symbol assignment completes, symbolic trend series are generated and the TE values based on new series are computed afterwards.
Fig. 6 Time series of the level of Tank 1 (a) and Tank 2 (b). By adjusting input flow and valves manually, we can generate data of tank levels under different simulation conditions.
3.3 Calculation of TTE and Significance Test Similar to the way of computing TE, the symbolic trend series instead of the original ones are used for the calculation of TTE. In view of computational load, a single symbol in TTE corresponds to a period of variable values in TE. Besides, the size of the symbol collection is adaptable as mentioned in the last subsection. These two features enable the TTE method to convert a long continuous series with length L into a comparably short discrete series with length n ( 𝑛𝑛 = 𝐿𝐿/𝑙𝑙 , where l is the length of isometric segments), leading to a remarkable reduction on the size of massive data that needs further computation. It should also be noted that this reduction is attributed to the different view of process data, but not suffering the loss of valid information. Therefore, the TTE method computes much faster than TE towards same series, and reflects trend causality among variables rather than value causality. Significance test method still works for the TTE method.
4.1 Stationary Series with Additional Noise We import Gaussian white noise with different amplitudes into the input and output data, and compare the average results of TTE and TE from 100 realizations.
Fig. 7 Causality strength of stationary series with additional Gaussian white noise calculated using TTE and TE. (The top line represents results of TTE, while the second represents results of TE. The other two lines that approach to zero show the calculation results from the opposite direction of causality Tank 2 to Tank 1.)
4. SIMULATIONS AND RESULTS TTE is evaluated on a double-tank system, a simple but common component of various industrial processes. Features of TTE are discussed in details and comparisons with TE further illustrate the robustness of the proposed method. The schematic of the double-tank system is shown in Fig. 5 and the level data of Tank 1 and Tank 2 in a period of time is shown in Fig. 6.
As shown in Fig. 7, by compressing the results of each method to [0, 1] respectively, we can see that the values of TTE decrease much slower than those of TE in situations of importing additional Gaussian white noise with increasing amplitude levels, which means TTE is more robust in the presence of noise. 4.2 Non-stationary Series The phenomenon of data drifting occurs commonly in real industrial processes. According to the second condition for TE calculation, it is inadequate for TE to detect causal relationship among variables in non-stationary cases. However, since piecewise linearization reduces the influence of data drifting, the TTE method performs effectively when dealing with the situation.
Fig. 5 Schematic of a double-tank system. Fi1 and Fi2 are inflows; Fo1 and Fo2 are outflows.
A stochastic trend 𝜔𝜔𝑡𝑡 = 𝜔𝜔𝑡𝑡−1 + 𝜀𝜀𝑡𝑡 is added to both of the tank levels, where 𝜀𝜀𝑡𝑡 is Gaussian white noise with unit variance. The time series of the level of Tank 1 and Tank 2 in one realization is shown in Fig. 8.
Tank 1 receives an external flow as the only source. A part of its output goes into Tank 2, while the rest flows outside the system. Tank 2 is similar with Tank 1 except for the second 781
SAFEPROCESS 2015 782 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
Trend transfer entropy performs well in situations of noise interference and data drifting, and proves to be helpful to reduce computational complexity. Thus, it is capable of dealing with practical problems in industrial processes and ensuring the process safety as a supplement of existing methods such as the conventional TE. Future work will be focused on analysis and discussion of parameters and properties of TTE. The proposed method will be tested on industrial processes. ACKNOWLEDGEMENTS
Fig. 8 Non-stationary time series of the level of Tank 1 (solid line) and Tank 2 (dashed line).
The authors would appreciate the financial support from the National High-tech Research and Development Program of China (2013AA040702) and the National Natural Science Foundation of China (61433001).
TTE and TE are tested with non-stationary time series in 100 realizations. The numbers of successful detections (the calculated value is larger than the threshold) and correct detections (the value of the direction Tank 1 to Tank 2 is larger than that of the direction Tank 2 to Tank 1) are shown in Table 1. According to the results, we can conclude that TTE performs much better with regard to non-stationary time series.
REFERENCES Bauer, M., Cox, J.W., Caveness, M.H., Downs, J.J., and Thornhill, N.F. (2007). Finding the direction of disturbance propagation in a chemical process using transfer entropy. IEEE Transactions on Control Systems Technology, 15(1), 12-21. Chiang, L.H. and Braatz, R.D. (2003). Process monitoring using causal map and multivariate statistics: fault detection and identification. Chemometrics and Intelligent Laboratory Systems, 65, 159-178. Cowell, R.G., Dawid, A.P., lauritzen, S.L., and Spiegelhalter, D.J. (1999). Probabilistic networks and expert systems. Springer-Verlag, New York. Dai, C., Zhu, Y., Chen, W., and Lin, J. (2007). Cloud model based genetic algorithm and its applications. Chinese Journal of Electronics, 35(7), 1419-1424 Doyle, R.J., Charest, L., Rouquette, N., Wyatt, J., and Robertson, C. (1993a). Causal modelling and eventdriven simulation for monitoring of continuous systems. Computers in Aerospace, 9, 395-405. Doyle, R.J., Chien, S.A., Fayyad, U.M., and Wyatt, E.J. (1993b). Focused real-time systems monitoring based on multiple anomaly models. 7th Int. Workshop on Qualitative Reasoning, Eastsound, Washington. Duan, P., Yang, F., Chen, T., and Shah, S.L. (2013). Direct causality detection via the transfer entropy approach. IEEE Transactions on Control Systems Technology, 21(6), 2052-2066. Duan, P., Chen, T., Shah, S.L., and Yang, F. (2014). Methods for root cause diagnosis of plant-wide oscillations. AIChE Journal, 60(6), 2019-2034. Duan, P., Yang, F., Shah, S.L., and Chen, T. (2015). Transfer zero-entropy and its application for capturing cause and effect relationship between variables. IEEE Transactions on Control Systems Technology, 23(3), 855-867. Granger, C.W.J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3), 424-438. Jiang, H.L., Patwardhan, R., and Shah, S.L. (2009). Root cause diagnosis of plant-wide oscillations using the concept of adjacency matrix. Journal of Process Control, 19, 1347–1354. Kantz, H. and Schreiber, T. (1997). Nonlinear Time Series Analysis. Cambridge University Press, Cambridge, UK.
Table 1 TTE and TE results in 100 realizations Methods TTE TE
Successful detections Tank 1 to Tank 2 to Tank 2 Tank 1 97 6 95 92
Correct detections 97 43
4.3 Computational Load of TTE As mentioned in Section 3, TTE can be regarded as a coarsegrained version of TE. Thus, by using TTE, the computational load is reduced significantly, and moreover, the compression scale is quite flexible by adjusting the length of each fundamental segment. However, the total length of the time series still affects the accuracy of TTE, which is the same for TE. As shown in Fig. 9, if we keep the segment length intact and change the length of the calculated series, TTE values become stable after a sufficient length of series is used. If the series are too long, the slight decline of the solid line implies that the accumulated noise effects will influence the performance of TTE.
Fig. 9 TTE value changes as the length of series increases until a certain length is reached. (The solid line represents TTE values in the causal direction Tank 1 to Tank 2; the dashed line corresponds to the opposite direction Tank 2 to Tank 1.) 5. CONCLUSIONS
782
SAFEPROCESS 2015 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
Kugiumtzis, D. (2013). Partial transfer entropy on rank vectors. Eur. Phys. J. Special Topics, 222, 401-420. Russell, S. and Norvig, P. (2003). Artificial intelligence: a modern approach. Prentice Hall, New Jersey. Schreiber, T. (2000). Measuring information transfer. Physical Review Letters, 85(2), 461. Staniek, M. and Lehnertz, K. (2008). Symbolic transfer entropy. Physical Review Letters, 100(15), 158101. Thambirajah, J., Benabbas, L., Bauer, M., and Thornhill, N.F. (2009). Cause-and-effect analysis in chemical processes utilizing XML, plant connectivity and quantitative process history. Computers & Chemical Engineering, 33(2), 503-512. Vedam, H. and Venkatasubramanian, V. (1997). Signed digraph based multiple fault diagnosis. Computers & Chemical Engineering, 21, 655-660. Venkatasubramanian, V., Zhao, J., and Viswanathan S. (2000). Intelligent system for HAZOP analysis of complex process plants. Computers & Chemical Engineering, 24, 2291-2302 Whiteman, Z., Tangirala, A., and Ogunnaike, B.A. (2014). On-line determination of appropriate control loop configuration using directed spectral decomposition of process data. 2014 AIChE Annual Meeting, Atlanta, GA. Yang, F., Shah, S.L., and Xiao, D. (2012). Signed directed graph modeling and validation of industrial processes by process knowledge and process data. International Journal of Applied Mathematics and Computer Science, 22(1), 41-53. Yang, F., Xiao, D., and Shah, S.L. (2013). Signed directed graph-based hierarchical modelling and fault propagation analysis for large-scale systems. IET Control Theory and Applications, 7(4), 537-550. Yang, F., Duan, P., Shah, S. L., and Chen, T. (2014). Capturing Connectivity and Causality in Complex Industrial Processes, Springer. Yu, W. and Yang, F. (2014). Detection of causality between industrial alarm data based on transfer entropy. Proceedings of the International Conference on Chemical Engineering, Dhaka, Bangladesh, 205-209. Yim, S.Y., Ananthakumar, H.G., Benabbas, L., Horch, A., and Drath, R., Thornhill, N.F. (2006). Using process topology in plant-wide control loop performance assessment. Computers & Chemical Engineering, 31(2), 86-99.
783
783
ScienceDirect
IFAC-PapersOnLine 48-21 (2015) 778–783
A Causality Capturing Method for Diagnosis Based on A Causality Capturing Method for Diagnosis Based on A Causality Capturing Method for Diagnosis Based on Transfer Entropy by Analyzing Trends of Time Series A Causality Capturing Method for Diagnosis Based on Transfer Entropy by Analyzing Trends of Time Series Transfer Entropy by Analyzing Trends of Time Series Transfer EntropyCen byGuo, Analyzing Fan Yang, Trends Weijun Yuof Time Series
Cen Guo, Guo, Fan Fan Yang, Yang, Weijun Weijun Yu Yu Cen Cen Guo, Fan Yang, Weijun Yu Tsinghua Laboratory for Information Science and Technology, Tsinghua Laboratory for Information ScienceUniversity, and Technology, Department of Automation, Tsinghua Tsinghua Laboratory for Information Science and Tsinghua Laboratory for Information ScienceUniversity, and Technology, Technology, Department of(e-mails: Automation, Tsinghua Beijing 100084, China [email protected], Department of Automation, Tsinghua University, Department of Automation, Tsinghua University, Beijing 100084, China (e-mails: [email protected], [email protected], [email protected]) Beijing China Beijing 100084, 100084, China (e-mails: (e-mails: [email protected], [email protected], [email protected], [email protected], [email protected]) [email protected]) [email protected], [email protected]) Abstract: Since modern industrial processes become much larger and more complex, efficient and Abstract: Since much larger more complex, efficient effective causality detectionindustrial methods processes are neededbecome to capture the process topology, causesand of Abstract: Since modern modern industrial processes become much larger and and more diagnose complex, root efficient and Abstract: Since modern industrial processes become much larger and more complex, efficient and effective causality detection methods are needed to capture the process topology, diagnose root causes widespread or even plant-wide process malfunction, andthe further ensure the safety of root processes. A effective causality detection methods are needed to capture process topology, diagnose causes of of effective causality detection methods are needed to capture process topology, diagnose root causes of widespread or even plant-wide process malfunction, andthe further ensure the safety of which processes. A modified transfer entropy method, named trend transfer entropy, is proposed in this paper, focuses widespread or even plant-wide process malfunction, and further ensure the safety of processes. A widespread ortrends even malfunction, andseries further ensure the safety of processes. A modified transfer entropy method, named transfer entropy, isthemselves proposed inand this paper, which focuses on analyzing ofplant-wide time seriesprocess rather trend than the original thus, compared to the modified transfer entropy method, named trend transfer entropy, is proposed in this paper, which focuses modified transfer entropy method, named trend transfer entropy, is proposed in this paper, which focuses on analyzing trends of time series rather than the original series themselves and thus, compared to the traditional transfer proves to be more robust in conditions of data drifting and noise disturbance. on analyzing trendsentropy, of series rather than the series themselves and thus, compared to the on analyzing of time timeproves series rather thanrobust the original original series themselves and thus, compared to the the traditional transfer to be computational more inload conditions of datasaving drifting and noise Moreover, thetrends newentropy, method can reduce effectively, valuable timedisturbance. before traditional transfer entropy, proves to be more robust in conditions of data drifting and noise disturbance. traditional transfer entropy, proves to be more robust in conditions of data drifting and noise disturbance. Moreover, new methodSimulation can reducestudies computational load to effectively, saving valuable before occurrence the of an accident. are presented illustrate the procedure andtime features of the the Moreover, the new method can computational load effectively, saving valuable time before the Moreover,method. the new methodSimulation can reduce reducestudies computational load to effectively, saving valuable before occurrence of an accident. are presented illustrate the procedure andtime features of the proposed occurrence of occurrencemethod. of an an accident. accident. Simulation Simulation studies studies are are presented presented to to illustrate illustrate the the procedure procedure and and features features of of the the proposed proposed method. Keywords: Causality detection, Transfer entropy, Trend analysis, Fault diagnosis © 2015, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. proposed method. Keywords: Keywords: Causality Causality detection, detection, Transfer Transfer entropy, entropy, Trend Trend analysis, analysis, Fault Fault diagnosis diagnosis Keywords: Causality detection, Transfer entropy, Trend analysis, Fault diagnosis
1. INTRODUCTION 1. INTRODUCTION 1. INTRODUCTION As modern industrial processes become increasingly large 1. INTRODUCTION As processes become increasingly large and modern complex,industrial the relationship within process variables is far As modern industrial processes become increasingly large As modern industrial processes become increasingly large and the within process variables far more complicated than before. This may result inis and complex, complex, the relationship relationship within process variables is the far and complex, the relationship within process variables is far more complicated than before. This may result in the occurrence of ‘alarmthan flood’, a phenomenon instantaneous more complicated before. This mayof result in the more complicated than before. This may result in the occurrence of ‘alarm flood’, a phenomenon of instantaneous emergence massive caused ofbyinstantaneous redundant occurrence ofof‘alarm flood’,alarms a phenomenon occurrence ofof‘alarm flood’, a phenomenon ofbyinstantaneous emergence alarms caused redundant configurations of massive alarms as well as fault propagation through emergence of massive alarms caused by redundant emergence of massive alarms caused by redundant configurations of alarms as well as fault propagation through the interconnected network of process components. In order configurations of alarms as well as fault propagation through configurations of alarms as well as fault propagation through the interconnected network of process components. In order to avoid undesirable consequences andcomponents. ensure the safety of the interconnected network of process In the interconnected network of process components. In order order to avoid undesirable consequences and ensure efficient the safety of process operation in the current situation, and to avoid undesirable consequences and ensure the safety of to avoid operation undesirable and ensure thetosafety of process inconsequences the current situation, efficient and effective causality detection methods are needed capture process operation in the current situation, efficient and process operation in the current situation, efficient and effective causality detection methods are needed to capture the process topology and thus diagnose root to causes of effective causality detection methods are needed capture effective causality methods are needed to(Yang capture the and diagnose root widespread evendetection plant-wide process malfunction et the process processortopology topology and thus thus diagnose root causes causes of of the processExisting topology and thus root causes of widespread or process malfunction (Yang et al, 2014). techniques for diagnose causality capture can be widespread or even even plant-wide plant-wide process malfunction (Yang et widespread or even plant-wide process malfunction (Yang et al, 2014). Existing techniques for causality capture can be generally divided into two types,for namely, process knowledge al, 2014). Existing techniques causality capture can be al, 2014). Existing techniques for causality capture can be generally divided into two types, namely, process knowledge based methods andinto process data based methods. The former generally divided two types, namely, process knowledge generally divided tworeachability types, namely, process knowledge based methods andinto process data based methods. The former includes adjacency and matrices (Jiang et al., based methods and process data based methods. The based methods and process data based methods. The former former includes adjacency andgraph reachability matrices (Jiang et 2009), signed directed based methods (Vedam et al., al., includes adjacency and reachability matrices (Jiang et al., includes adjacency andgraph reachability matrices (Jiang et al., 2009), signed directed based methods (Vedam 1997; Venkatasubramanian et al., 2000; Yang et al., 2012; 2009), signed directed graph based methods (Vedam et al., 2009), signed directed based methods (Vedam etweb al., 1997; Venkatasubramanian et 2000; et Yang et al., 2013), andgraph plant topology extraction 1997; Venkatasubramanian et al., al., 2000; Yang Yang et al., al.,by2012; 2012; 1997; Venkatasubramanian et al., 2000; Yang et al., 2012; Yang et al., 2013), and plant topology extraction by web language (Yim2013), et al.,and 2006; Thambirajah al., 2009), Yang et al., plant topology et extraction bywhile web Yang et al., 2013), and plant topology analysis, extraction bywhile web language (Yim et Thambirajah the latter contains cross-correlation Granger language (Yim et al., al., 2006; 2006; Thambirajah et et al., al., 2009), 2009), while language (Yim et al., 2006; Thambirajah et al., 2009), while the latter contains cross-correlation analysis, Granger causality analysis (Granger, 1969), Bayesian network the latter contains cross-correlation analysis, Granger the latter(Cowell contains cross-correlation analysis,(Schreiber, Granger causality analysis (Granger, 1969), entropy Bayesian network learning et al., 1999), transfer causality analysis (Granger, 1969), Bayesian network causality analysis (Granger, 1969), Bayesian network learning (Cowell et al., 1999), transfer entropy (Schreiber, 2000) (TE), and other entropy based methods (Doyle et al., learning (Cowell et al., 1999), transfer entropy (Schreiber, learning (Cowell et toal.,achieve 1999),better transfer entropy (Schreiber, 2000) (TE), and other entropy based methods (Doyle et al., 1993a, b). Besides, diagnosis performance, 2000) (TE), and other entropy based methods (Doyle et al., 2000) (TE), and other entropy based diagnosis methods (Doyle etwith al., 1993a, b). Besides, to achieve better performance, especially when abnormal situations are associated 1993a, b). Besides, to achieve better diagnosis performance, 1993a, b). Besides, to achieve better diagnosis performance, especially when abnormal situations are associated with unknown multiple situations faults, integrated methods with that especially faults when orabnormal are associated especially when abnormal situations are associated with unknown faults multiple faults, integrated methods that combine analysis process knowledge unknown process faults or or data multiple faults,with integrated methods that unknown faults or multiple faults, integrated methods that combine process data analysis with process knowledge extraction are proposed and demonstrated be quite combine process data analysis with processto knowledge combine process data analysis with process knowledge extraction are proposed and demonstrated to be quite effective (Chiang et al., 2003). extraction are and extraction are proposed proposed and demonstrated demonstrated to to be be quite quite effective (Chiang et al., 2003). effective (Chiang et al., 2003). Transfer entropy etisal.,an2003). information-theoretic method for effective (Chiang Transfer entropy information-theoretic for causality measurement, unlike most of themethod other dataTransfer entropy is is an anwhich, information-theoretic method for Transfer entropy is anwhich, information-theoretic method for causality measurement, unlike most of the other datacausality measurement, which, unlike most of the other datacausality measurement, which, unlike most of the other data-
based methods, is free from model assumptions and based methods, is situations. free fromTomodel applicable to various extend assumptions the applicationand of based methods, is free from model assumptions and based methods, is situations. free fromhave assumptions and applicable toimproved various Tomodel extend the application of TE, several methods been developed. Direct applicable to situations. To the of applicableentropy to various various Totoextend extend the application application of TE, methods been developed. transfer wassituations. proposedhave distinguish directDirect and TE, several several improved improved methods have been developed. Direct TE, several improved methods have been developed. Direct transfer entropy was proposed to distinguish direct and indirect among excluding transfer causal entropyrelationships was proposed to variables distinguishby direct and transfer entropy wasrest proposed to variables distinguish and indirect causal among excluding the effects fromrelationships the of the process (Duanby etdirect al., 2013; indirect causal relationships among variables by excluding indirect causal relationships among variables by excluding the effects from the rest of the process (Duan et al., 2013; Duan et al., 2015); symbolic transfer (Duan entropyet calculates the effects from the rest of process al., 2013; the effects from the rest of the the of process (Duan et calculates al., 2013; Duan et ofal., 2015); symbolic transfer entropy symbols time series instead original values to reduce Duan et al., 2015); symbolic transfer entropy calculates Duan et al., 2015); symbolic transfer entropy calculates symbols of time series instead of original values to reduce noise influence (Staniek et al., 2008); partialvalues transfer symbols of series instead of to entropy reduce symbols of time time(Staniek instead of original original reduce noise influence et al., 2008); partialvalues transfer entropy on rank vectors is,series to some extent, a combination ofto the two noise influence (Staniek et al., 2008); partial transfer entropy noise influence (Staniek et al., 2008); partial transfer entropy on rank vectors is, and to some extent, a have combination of the two methods above, proves to both advantages on rank is, to some extent, aa combination of the on rank vectors vectors is, and toHowever, some extent, combination offocus the two two methods proves to both (Kugiumtzis, 2013). these methods on methods above, above, and provesfew to ofhave have both advantages advantages methods above, and proves to have both advantages (Kugiumtzis, 2013). However, few of these methods focus alleviating the2013). computational is the bottleneck (Kugiumtzis, However, load, few ofwhich these methods focus on on (Kugiumtzis, 2013). However, load, few these methods focus on alleviating the computational which is bottleneck problem when TEoffor causality in alleviating the implementing computationalthe load, which is the thedetection bottleneck alleviating the cases, computational load, which thedetection bottleneck problem when implementing TE for in real industrial since it the requires a causality largeis amount of data problem when implementing the TE for causality detection in problem when implementing the TE for causality detection in real industrial cases, since it requires a large amount of data to improve thecases, accuracy. Init some cases, the amount pursuit of new real industrial since requires a large of data real industrial cases, since it requires a large amount data to improve the accuracy. In some cases, the pursuit of new functionalities requires more computing resources and to improve the In some cases, pursuit to improve time, the accuracy. accuracy. In low somediagnosis cases, the the pursuit of of new new functionalities requires more computing resources and processing leading to effectiveness and functionalities requires more computing resources and functionalities requires more computing resources and processing time, leadingtoto low diagnosis effectiveness unsatisfactory response process fault from the perspective processing time, leading to low diagnosis effectiveness and processing time, leading lowcomputational diagnosis effectiveness anda unsatisfactory response totoprocess fault from the perspective of process safety. Although the burden is not unsatisfactory response to process fault from the perspective unsatisfactory response to process fault from the perspective of process safety. Although the computational burden is not bigprocess issue safety. for theAlthough offline analysis of causality capture, of the computational burden is not itaa of process safety. Although the computational burden is not ita big issue for the offline analysis of causality capture, prevents from being usedofforcausality online diagnosis, big issuetheformethod the offline analysis capture, in it big issue for the offline analysis of causality capture, it prevents the method from being used for online diagnosis, in particularthe formethod novel fault diagnosis with varied causality, and prevents from being used for online diagnosis, in prevents the method from being used for online diagnosis, in particular for novel fault diagnosis with varied causality, and other applications suchdiagnosis as control loop causality, configuration particular for novel fault with and particular for fault with varied varied and other applications suchdiagnosis as control loop causality, configuration (Whiteman et novel al., 2014). other applications such as control loop configuration other applications such as control loop configuration (Whiteman et al., 2014). (Whiteman et 2014). In addition to all the improved approaches mentioned above, (Whiteman et al., al., 2014). In addition to all the improved approaches above, transfer entropy was also applied to binary mentioned alarm sequences, In addition to all the improved approaches mentioned above, In addition to all the improved approaches mentioned above, transfer entropy was also applied to binary alarm sequences, which aims at reducing computational burden and can be transfer entropy was also applied to binary alarm sequences, transfer entropy was also applied to binary alarm sequences, which aims at reducing computational burden and can be utilized in multivariate alarm analysis andburden design and (Duan et al., which aims at reducing computational can be which Yu aims at reducing computational can be utilized in et multivariate alarm analysis andburden design and (Duan et on al., 2014; al., 2014). Yet the technique depends heavily utilized in multivariate alarm analysis and design (Duan et al., utilized in multivariate alarm analysis and design (Duan et al., 2014; Yuof et al., 2014). Yet the heavily on settings alarm limits andtechnique loses a depends large amount of 2014; Yu et al., 2014). Yet technique depends heavily on 2014; Yuof et in al., 2014). Yet the the technique heavily on settings alarm limits and loses a depends large toamount of information process data. Thus it is sensitive noise and settings of alarm limits and loses aa large amount of settings of alarm limits and loses large amount of information in process data. Thus it is sensitive to noise and has a deteriorated performance. In general, better information in process data. Thus it is sensitive to noise and information in process data. Thus it is sensitive to noise and has a deteriorated performance. In general, better has a deteriorated performance. In general, better has a deteriorated performance. In general, better
2405-8963 © 2015, IFAC (International Federation of Automatic Control) Copyright 2015 IFAC 778Hosting by Elsevier Ltd. All rights reserved. Peer review©under of International Federation of Automatic Copyright 2015 responsibility IFAC 778Control. Copyright © 2015 IFAC 778 10.1016/j.ifacol.2015.09.621 Copyright © 2015 IFAC 778
SAFEPROCESS 2015 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
performance and wide applications of TE require a more comprehensive approach with feasible computational load.
couple of times using surrogate data generated by randomly disorganizing the sequence of time series for each time, and then their mean is compared with the TE value obtained from the calculation of the original series. If two values are almost equal, we can say that there is no significant causal relationship between two series, since the causality is destroyed in disordered data. Therefore, the average TE value can be used as a threshold after adding some standard deviations (Bauer et al., 2007). For the concern of convenience, TE value in this paper is the remains after subtraction of a threshold and 3 standard deviations.
A new method based on the modification of TE, named trend transfer entropy (TTE), is proposed in this paper. When computing TTE, the trends of time series rather than series themselves are considered. Accordingly, the TTE method presents the capability of reducing influence from noise and data drifting, and performs well when dealing with mass data from long time series. It should be noted that TTE is not a substitute of TE, but a supplement of the conventional method to reveal and reflect causal relationship among process variables from a novel perspective.
3. TREND TRANSFER ENTROPY
The remaining part of this paper is organized as follows: Section 2 describes TE and its significance test method; Section 3 introduces the proposed TTE method; Section 4 presents simulation studies to illustrate the effectiveness of the proposed approach; followed by conclusions in Section 5. 2. BASICS OF TRANSFER ENTROPY 2.1 Introduction to Transfer Entropy TE is used to test the causality between time series by measuring the reduction of information uncertainty. Its basic definition is given by (1): (𝑘𝑘)
(𝑙𝑙)
𝑇𝑇𝑇𝑇𝐽𝐽→𝐼𝐼 = ∑ 𝑝𝑝(𝑖𝑖𝑡𝑡+ℎ , 𝑖𝑖𝑡𝑡 , 𝑗𝑗𝑡𝑡 )log 2
(𝑘𝑘) (𝑙𝑙)
𝑝𝑝(𝑖𝑖𝑡𝑡+ℎ ,|𝑖𝑖𝑡𝑡 ,𝑗𝑗𝑡𝑡 ) (𝑘𝑘)
𝑝𝑝(𝑖𝑖𝑡𝑡+ℎ |𝑖𝑖𝑡𝑡 )
779
(1)
where I and J denote two time series; p means the complete or conditional probability density function (PDF); subscripts t and h represent the current time t and the prediction horizon h; superscripts k and l represent k and l continuous historical values before the subscript time. The estimation of PDFs can be obtained by statistical histograms using historical process data, or through kernel methods which are more accurate in specific cases. Although PDF estimation is surely nontrivial for the calculation of TE, it is not a prime focus in this paper. The equation indicates if there exists causality from J to I, it will be helpful to predict the value of I by additionally using the historical data of J. To be specific, the value of the numerator should be larger than that of the denominator, and the equation value should be larger than 0 if J affects I. On the contrary, the equation value should be close to 0, if no causal relationship exists from J to I.
In real cases of modern industrial processes, it is commonly known that values of variables can be influenced from the cause variable to the effect variable. Given that most variable values are approximately continuous and seldom mutated, their changes during a short period of time can be considered linear. Thus, if there exists causality between two variables, the linear value change of the cause one will quite possibly lead to a regulated value change of the other, no matter if the corresponding effects appear instantly or subsequently, or if the value of the effect variable reacts linearly or follows a certain pattern. Otherwise, they are considered independent. If changes of variable values which are regarded as trends can be properly used, we can reveal causal relationship from a novel perspective of trend causality, comparing to the traditional view of value causality. This is the main idea of the TTE method. There are generally three steps when computing TTE. The first is original series processing by means of piecewise linearization to extract trend information conveyed in slopes; the second is cloud classification to generate symbolic trend series; and the last is to calculate the TE value of the new symbolic trend series and test the significance. 3.1 Piecewise Linearization
2.2 Significance Test
We implement piecewise linearization on the original time series to extract trend information conveyed in slopes and meanwhile reduce the effects of noise and data drifting. In view of calculation convenience, feature-based piecewise linearization methods usually perform better than fittingbased approaches, e.g., the piecewise least squares method, within an acceptable range of errors. Thus, we prefer to adopt a feature-based method here. The term feature refers to the points that generally determine the shape of a series. These points include two different types: extreme points and turning points. Extreme points locate between two segments with slopes that have different signs; turning points separate segments that have the same sign but relatively obvious difference in their slopes. Examples of points from two types are shown in Fig. 1.
Considering the existence of noise, it is natural that the TE value of two time series does not equal to 0 exactly. Therefore, a threshold to distinguish real causality from false results caused by noise interference is of great necessity. Monte-Carlo examination was suggested to be a method of significance test (Kantz, 1997). TE values are calculated for a
Fig. 1 Segments of time series with extreme points in (1) (2) and turning points in (3)-(6).
Two conditions should be satisfied before utilizing TE: a sufficient length of series and the stationarity of series. The former indicates that the accuracy of TE relates positively with the size of data to a certain degree, while the latter reflects the limitation of TE, especially for the implementation in practical industrial processes.
779
SAFEPROCESS 2015 780 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
It is comparatively easier to figure out extreme points in time series than to select an appropriate number of turning points, because using different thresholds to determine the degree of obviousness leads to diverse collections of turning points. Thus, the extent to which the new series formed by feature points fits the original one can be adjusted via the threshold according to the knowledge of real process states.
reflects the fuzziness of the numerical range of concepts; 𝐻𝐻e , on the other hand, measures the uncertainty of the entropy, describing the cohesiveness of the drops in a cloud. A basic normal cloud generator was presented in (Russell et al., 2003). The algorithm is as follows:
Another problem which requires special attention is that, since time series obtained from industrial processes are usually affected by noise, feature points may retain that information without any filtration if simply keeping all of them intact. Therefore, the selection of feature points is necessary in order to reduce noise influence and reflect real changes of process variable values. A selection algorithm is presented below towards this issue:
Step 2: Generate a normal random number 𝑥𝑥 with 𝐸𝐸x as expectation and the absolute value of 𝐸𝐸n as standard deviation. 𝑥𝑥 is a cloud drop in the discourse domain.
Step 1: Generate a normal random number 𝐸𝐸n′ with 𝐸𝐸n as expectation and 𝐻𝐻e as standard deviation.
Step 3: Calculate
𝑦𝑦 = exp {−
(𝑥𝑥− 𝐸𝐸x )2 ′ )2 2( 𝐸𝐸n
} ,
where
y
is
membership of x belonging to a certain concept. Step 4: Repeat steps 1 to 3 until N number of cloud drops are generated.
Step 1: Search for feature points in the order from the starting point to the ending point of time series.
Two parameters should be especially concerned here – the length of each fundamental segment that can be represented by one symbol and the size of the symbol collection. Assuming the length of the original time series as L, we divide it into n isometric segments with length l, which satisfies the equation 𝐿𝐿 = 𝑛𝑛𝑛𝑛. Accordingly, one symbol in the trend series corresponds to l time intervals in the original series. In order that a maximum segment length can be effectively represented by a single symbol and the whole series data can be well compressed, l is set to be the approximate greatest common divisor of distances between each adjacent pair of feature points. As for the size of the symbol collection, it should meet the demand of analysis precision in the real processes. As shown in Figs. 3 and 4, the more subtly we classify those segments, the larger the size of the symbol collection is.
Step 2: If two adjacent feature points are within a predefined threshold, e.g., 10 time intervals, the latter one is considered invalid. Step 3: Go back to step 1 unless the whole series is processed. The main purpose of this algorithm is to guarantee necessary space between each pair of adjacent feature points, so that points caused by noise will mostly not be included in the final collection. In Fig. 2, the solid line is the original time series, while the dashed line is the reconstructed series based on selected feature points with the threshold of adjacency to be 5 time intervals.
Fig. 2 Original time series (solid line) and processed time series by piecewise linearization (dashed line). Fig. 3 Segments classification by descriptions of natural language.
3.2 Cloud Classification After the feature-based piecewise linearization on the original time series, we use a cloud model for symbol assignment towards the reconstructed series, because it can express the fuzziness and randomness in the classification by converting the continuous slopes to natural language expressions, such as ‘ascend’, ‘stable’, and ‘descend’. Define T as a linguistic value in the discourse domain of u. For mapping 𝐶𝐶𝑇𝑇 (𝑥𝑥): 𝑢𝑢 → [0,1] , ∀𝑥𝑥 ∈ 𝑢𝑢, 𝑥𝑥 → 𝐶𝐶𝑇𝑇 (𝑥𝑥). The distribution of 𝐶𝐶𝑇𝑇 (𝑥𝑥) in u is called a membership cloud of T. If 𝐶𝐶𝑇𝑇 (𝑥𝑥) follows a Gaussian distribution, the model is further named as a normal cloud (Dai et al., 2007). Three numerical characteristics form the basic features of the normal cloud model: expectation 𝐸𝐸x , entropy 𝐸𝐸n , and excess entropy 𝐻𝐻e . 𝐸𝐸x is the expectation of cloud drops in the discourse domain and the best representation of a qualitative concept; 𝐸𝐸n
Fig. 4 Normal cloud classification model with cloud classes 1-5 from left to right corresponding to ‘severely ascend’, ‘slowly ascend’, ‘stable’, ‘slowly descend’, and ‘severely descend’, respectively. In each cloud, a small range of membership values is assigned to a single slope angle, but only one value is generated in one realization. The final class 780
SAFEPROCESS 2015 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
of a slope depends on the cloud attribution of its largest membership value.
781
tube of input. Therefore, the level of Tank 1 affects that of Tank 2, but Tank 2 has no influence on Tank 1.
Once the length of every fundamental segment and the size of the symbol collection are properly set, we assign different symbols to all the classes in the normal cloud model. For example, we set the symbol of a segment with slope -60 degree to be 1 and the symbol for slope -30 degree to be 2 (Fig. 4). If there are segments that cover small pieces with more than one slope, the slope of the whole segment is given as 𝑘𝑘𝑖𝑖 by the probability of 𝑥𝑥𝑖𝑖 /𝑙𝑙, where 𝑥𝑥𝑖𝑖 is the length of the piece with the slope 𝑘𝑘𝑖𝑖 . When the symbol assignment completes, symbolic trend series are generated and the TE values based on new series are computed afterwards.
Fig. 6 Time series of the level of Tank 1 (a) and Tank 2 (b). By adjusting input flow and valves manually, we can generate data of tank levels under different simulation conditions.
3.3 Calculation of TTE and Significance Test Similar to the way of computing TE, the symbolic trend series instead of the original ones are used for the calculation of TTE. In view of computational load, a single symbol in TTE corresponds to a period of variable values in TE. Besides, the size of the symbol collection is adaptable as mentioned in the last subsection. These two features enable the TTE method to convert a long continuous series with length L into a comparably short discrete series with length n ( 𝑛𝑛 = 𝐿𝐿/𝑙𝑙 , where l is the length of isometric segments), leading to a remarkable reduction on the size of massive data that needs further computation. It should also be noted that this reduction is attributed to the different view of process data, but not suffering the loss of valid information. Therefore, the TTE method computes much faster than TE towards same series, and reflects trend causality among variables rather than value causality. Significance test method still works for the TTE method.
4.1 Stationary Series with Additional Noise We import Gaussian white noise with different amplitudes into the input and output data, and compare the average results of TTE and TE from 100 realizations.
Fig. 7 Causality strength of stationary series with additional Gaussian white noise calculated using TTE and TE. (The top line represents results of TTE, while the second represents results of TE. The other two lines that approach to zero show the calculation results from the opposite direction of causality Tank 2 to Tank 1.)
4. SIMULATIONS AND RESULTS TTE is evaluated on a double-tank system, a simple but common component of various industrial processes. Features of TTE are discussed in details and comparisons with TE further illustrate the robustness of the proposed method. The schematic of the double-tank system is shown in Fig. 5 and the level data of Tank 1 and Tank 2 in a period of time is shown in Fig. 6.
As shown in Fig. 7, by compressing the results of each method to [0, 1] respectively, we can see that the values of TTE decrease much slower than those of TE in situations of importing additional Gaussian white noise with increasing amplitude levels, which means TTE is more robust in the presence of noise. 4.2 Non-stationary Series The phenomenon of data drifting occurs commonly in real industrial processes. According to the second condition for TE calculation, it is inadequate for TE to detect causal relationship among variables in non-stationary cases. However, since piecewise linearization reduces the influence of data drifting, the TTE method performs effectively when dealing with the situation.
Fig. 5 Schematic of a double-tank system. Fi1 and Fi2 are inflows; Fo1 and Fo2 are outflows.
A stochastic trend 𝜔𝜔𝑡𝑡 = 𝜔𝜔𝑡𝑡−1 + 𝜀𝜀𝑡𝑡 is added to both of the tank levels, where 𝜀𝜀𝑡𝑡 is Gaussian white noise with unit variance. The time series of the level of Tank 1 and Tank 2 in one realization is shown in Fig. 8.
Tank 1 receives an external flow as the only source. A part of its output goes into Tank 2, while the rest flows outside the system. Tank 2 is similar with Tank 1 except for the second 781
SAFEPROCESS 2015 782 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
Trend transfer entropy performs well in situations of noise interference and data drifting, and proves to be helpful to reduce computational complexity. Thus, it is capable of dealing with practical problems in industrial processes and ensuring the process safety as a supplement of existing methods such as the conventional TE. Future work will be focused on analysis and discussion of parameters and properties of TTE. The proposed method will be tested on industrial processes. ACKNOWLEDGEMENTS
Fig. 8 Non-stationary time series of the level of Tank 1 (solid line) and Tank 2 (dashed line).
The authors would appreciate the financial support from the National High-tech Research and Development Program of China (2013AA040702) and the National Natural Science Foundation of China (61433001).
TTE and TE are tested with non-stationary time series in 100 realizations. The numbers of successful detections (the calculated value is larger than the threshold) and correct detections (the value of the direction Tank 1 to Tank 2 is larger than that of the direction Tank 2 to Tank 1) are shown in Table 1. According to the results, we can conclude that TTE performs much better with regard to non-stationary time series.
REFERENCES Bauer, M., Cox, J.W., Caveness, M.H., Downs, J.J., and Thornhill, N.F. (2007). Finding the direction of disturbance propagation in a chemical process using transfer entropy. IEEE Transactions on Control Systems Technology, 15(1), 12-21. Chiang, L.H. and Braatz, R.D. (2003). Process monitoring using causal map and multivariate statistics: fault detection and identification. Chemometrics and Intelligent Laboratory Systems, 65, 159-178. Cowell, R.G., Dawid, A.P., lauritzen, S.L., and Spiegelhalter, D.J. (1999). Probabilistic networks and expert systems. Springer-Verlag, New York. Dai, C., Zhu, Y., Chen, W., and Lin, J. (2007). Cloud model based genetic algorithm and its applications. Chinese Journal of Electronics, 35(7), 1419-1424 Doyle, R.J., Charest, L., Rouquette, N., Wyatt, J., and Robertson, C. (1993a). Causal modelling and eventdriven simulation for monitoring of continuous systems. Computers in Aerospace, 9, 395-405. Doyle, R.J., Chien, S.A., Fayyad, U.M., and Wyatt, E.J. (1993b). Focused real-time systems monitoring based on multiple anomaly models. 7th Int. Workshop on Qualitative Reasoning, Eastsound, Washington. Duan, P., Yang, F., Chen, T., and Shah, S.L. (2013). Direct causality detection via the transfer entropy approach. IEEE Transactions on Control Systems Technology, 21(6), 2052-2066. Duan, P., Chen, T., Shah, S.L., and Yang, F. (2014). Methods for root cause diagnosis of plant-wide oscillations. AIChE Journal, 60(6), 2019-2034. Duan, P., Yang, F., Shah, S.L., and Chen, T. (2015). Transfer zero-entropy and its application for capturing cause and effect relationship between variables. IEEE Transactions on Control Systems Technology, 23(3), 855-867. Granger, C.W.J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3), 424-438. Jiang, H.L., Patwardhan, R., and Shah, S.L. (2009). Root cause diagnosis of plant-wide oscillations using the concept of adjacency matrix. Journal of Process Control, 19, 1347–1354. Kantz, H. and Schreiber, T. (1997). Nonlinear Time Series Analysis. Cambridge University Press, Cambridge, UK.
Table 1 TTE and TE results in 100 realizations Methods TTE TE
Successful detections Tank 1 to Tank 2 to Tank 2 Tank 1 97 6 95 92
Correct detections 97 43
4.3 Computational Load of TTE As mentioned in Section 3, TTE can be regarded as a coarsegrained version of TE. Thus, by using TTE, the computational load is reduced significantly, and moreover, the compression scale is quite flexible by adjusting the length of each fundamental segment. However, the total length of the time series still affects the accuracy of TTE, which is the same for TE. As shown in Fig. 9, if we keep the segment length intact and change the length of the calculated series, TTE values become stable after a sufficient length of series is used. If the series are too long, the slight decline of the solid line implies that the accumulated noise effects will influence the performance of TTE.
Fig. 9 TTE value changes as the length of series increases until a certain length is reached. (The solid line represents TTE values in the causal direction Tank 1 to Tank 2; the dashed line corresponds to the opposite direction Tank 2 to Tank 1.) 5. CONCLUSIONS
782
SAFEPROCESS 2015 September 2-4, 2015. Paris, France
Cen Guo et al. / IFAC-PapersOnLine 48-21 (2015) 778–783
Kugiumtzis, D. (2013). Partial transfer entropy on rank vectors. Eur. Phys. J. Special Topics, 222, 401-420. Russell, S. and Norvig, P. (2003). Artificial intelligence: a modern approach. Prentice Hall, New Jersey. Schreiber, T. (2000). Measuring information transfer. Physical Review Letters, 85(2), 461. Staniek, M. and Lehnertz, K. (2008). Symbolic transfer entropy. Physical Review Letters, 100(15), 158101. Thambirajah, J., Benabbas, L., Bauer, M., and Thornhill, N.F. (2009). Cause-and-effect analysis in chemical processes utilizing XML, plant connectivity and quantitative process history. Computers & Chemical Engineering, 33(2), 503-512. Vedam, H. and Venkatasubramanian, V. (1997). Signed digraph based multiple fault diagnosis. Computers & Chemical Engineering, 21, 655-660. Venkatasubramanian, V., Zhao, J., and Viswanathan S. (2000). Intelligent system for HAZOP analysis of complex process plants. Computers & Chemical Engineering, 24, 2291-2302 Whiteman, Z., Tangirala, A., and Ogunnaike, B.A. (2014). On-line determination of appropriate control loop configuration using directed spectral decomposition of process data. 2014 AIChE Annual Meeting, Atlanta, GA. Yang, F., Shah, S.L., and Xiao, D. (2012). Signed directed graph modeling and validation of industrial processes by process knowledge and process data. International Journal of Applied Mathematics and Computer Science, 22(1), 41-53. Yang, F., Xiao, D., and Shah, S.L. (2013). Signed directed graph-based hierarchical modelling and fault propagation analysis for large-scale systems. IET Control Theory and Applications, 7(4), 537-550. Yang, F., Duan, P., Shah, S. L., and Chen, T. (2014). Capturing Connectivity and Causality in Complex Industrial Processes, Springer. Yu, W. and Yang, F. (2014). Detection of causality between industrial alarm data based on transfer entropy. Proceedings of the International Conference on Chemical Engineering, Dhaka, Bangladesh, 205-209. Yim, S.Y., Ananthakumar, H.G., Benabbas, L., Horch, A., and Drath, R., Thornhill, N.F. (2006). Using process topology in plant-wide control loop performance assessment. Computers & Chemical Engineering, 31(2), 86-99.
783
783