- Email: [email protected]
Anomaly Detection in Complex Systems†
Copyright Cl IFAC Fault Detection, Supervision and Safety of Technical Processes, Washington, D.e., USA, 2003
c:
IFAC
0
[>
Publications www.elsevier.comllocatelifac
Anomaly Detection In Complex Systemst Amit Surana
Asok Ray
Shin Chin
The Pennsylvania State University University Park, PA 16802
Abstract: This paper presents a novel concept of anomaly detection in complex systems using tools of Symbolic Dynamics and Pattern Discovery. Anomaly is defined as a deviation from the nominal behaviour and can be associated with parametric changes that may gradually evolve. The proposed methodology is based on two-time-scale analysis of observed asymptotic behaviour of the dynamical system. This concept of small change detection in dynamical systems is elucidated through an example of the forced Duffing equation with a slowly varying dissipation parameter. Copyright © 2003 IFA C
Keywords: Fault Diagnosis and Prognosis; Symbolic Dynamics; Pattern Discovery; Complexity Measures 1 Introduction Engineering and scientific theories of control. communication and computing have matured in recent decades facilitating creation of systems of bewildering complexity almost comparable to that of biological systems. This complexity is largely hidden and cryptic in the laboratory environments as well as during normal operations of large-scale systems; however. it becomes conspicuous acutely when contributing to rare cascading failures (Badii and Politi. 1997; West, 1999). Early detection of anomalies in complex systems is not only essential for prevention of cascading catastrophic failures but also for enhanced system performance and availability. However. solely based on the fundamental principles of physics. accurate and computationally tractable modelling of complex system dynamics is often infeasible. One may have to rely on semi-empirical models whose parameters can be identified using time series data generated from sensors and other sources of information (Abarbanel. 1996). Accordingly, decision and control systems may have to be fonnulated based on such semi-empirical models and real-time data obtained through monitoring the system. The objective of the research work presented in this paper is to quantitatively detect the qualitative features of anomalies in complex systems using symbolic dynamics and pattern discovery. The theme of anomaly detection is presented below. The proposed methodology of anomaly detection is built upon the concepts of Dynamical Systems Theory, in which Symbolic Dynamics provides a qualitative description of dynamical behaviour in terms of symbol sequences. Computational
Mechanics provides a means to find patterns in these symbol sequences through the epsilon machine (Emachine) reconstruction (Crutchfield. 1994). Anomaly detection is then viewed as identifying variations in these patterns as parametric or nonparametric changes evolve. In order to facilitate small change detection in dynamical system parameters, we propose to excite the system with a priori known stimuli and discover anomaly patterns, if any, from the resulting responses. As an example, this paper considers a class of non-linear nonautonomous electro-mechanicaI systems in which anomalies occur at a slow time-scale while the inferences are made based on the observation of the fast time-scale system dynamics. The proposed procedure of anomaly detection relies on the two time-scale analysis of the asymptotic response of the dynamical system. The paper is organized in six sections. Section 2 provides the motivation of using Symbolic Dynamics as a means of encoding nonlinear dynamical systems. Section 3 presents a brief overview of Computational Mechanics from the perspectives of pattern discovery along with the procedure for E -machine reconstruction. The Emachines represent the behaviour of dynamical systems based on the observed time series data. Section 4 presents the underlying approach for anomaly detection. Section 5 illustrates the anomaly detection methodology on the well-mown Duffing equation as a prototype of complex electromechanical systems. Section 6 summarizes and concludes the paper with recommendations for future research.
t This woo has been supported in part by the U.S. Army Research Office under Grant No. DAADI9-0I·I.0646 and NASA GleM Research Center under Grant No. NAG3·2448.
1119
2 Symbolic DyDllDKs This section briefly describes the concept of Symbolic Dynamics and the motivation for using this concept as a means of encoding nonlinear systems. 2.1 Modelling and Analysis of Dynamical SysteJm Continuously varying physical processes are often modelled as a finite-dimensional dynamical system in the setting of an initial value problem as:
'd;> =!(x(t).O); x(O)=XO where tE [0.-) is the time;
XE
(I)
bounded) set g C Rn within which the motion is circumscribed is identified with the phase space itself. The encoding of g is accomplished by ~ ='1 Bo.···. Bb-tl introducing a partition ConSlSUng of nb mutually exclusive, i.e.,
Bj
nBIc =0
nb-I
U Bj j=O
Vj ~ k. and exhaustive subsets, i.e.•
=X . The
dynamical system describes an
orbit O. (XO.xl···.xn ...) • which passes through or
9t n is the state
touches various elements of the partition ~. Let us denote the index of domain BE tjJ visited at the time instant i as the symbol Uj Er. The set
vector in the phas~ spact!; and IJ e 9t m is the (possibly slowly) varying parameter vector. The vector field !(x.lJ) is, in general, a (locally) Lipschtiz nonlinear operator that suffices to ensure a unique local solution to the differential equation (I) (Vidyasagar. 1993). Formally such a solution can be expressed as a continuous function of the initial value Xo as ..t(t) = fPl XO where fP I represents a one-
r
=(O.···.nb -I} of symbols labelling the partition elements is called the symbol alphabt!t. Each initial state
Xo
generates a sequence of
symbols defined by a mapping from the phase space to the symbol space as:
parameter family of maps of the phase space into itself. This evolution in time t can be viewed as a flow of points in the phase space. For a dissipative system, the flow may contract onto sets of lower dimension. known as attractors. which represent the asymptotic behaviour of the dynamical system, characterized by several phenomena (e.g., {lXt!d poilll, limit cyclt!. qUIJSi-pt!riodicily. clwos. !ractal bt!haviollr. and bijiucation) (Ott, 1993).
XO 1-+ Uo ul,··un ...
(2)
Such a mapping is called Symbolic Dynamics as it attributes a l~gal (i.e.• physically admissible) symbol sequence to the system dynamics starting from an initial state XO. Since the size of each cell is finite and the cardinality of the alphabet 1: is finite. any such symbol sequence represents. through iteration. a phase trajectory that has a compact support Q in the phase space. The partition of g represented by the symbol alphabet 1: is called "generating one" if a symbol sequence uO.ul.u2··· uniquely
Modelling of a physical process. having the mathematical structure of Eq. (I), may not always be feasible solely based on the fundamental principles of physics. A convenient way of learning the dynamical behaviour is to rely on the (sensor-based) observed behaviour. The need to extract relevant physical information about the observed dynamics when they are operating in a chaotic regime has lead to development of non/war timt! ~rit!s analysis (NTSA) techniques. Analysis of (potentially) chaotic systems using NTSA techniques is classified into two areas (Abarbanel. 1996): • Bt!haviour identification • Mod~lling and Prt!diction
detennines a specific initial condition XO. That is. the mapping from the phase space to the symbol space is invertible. In general. a dynamical system would allow only legal concatenations of symbols to occur as there are many ilJegaJ (i.e., physically inadmissible) sequences. A grammar (i.e.• a set of rules) that detennines legality of symbol strings in the alphabet 1: may change with the parameter vector IJ. Among the legal symbol sequences. some symbols may occur more frequently than others. Hence, a certain probability p(s N ) is attributed to each
The first area shows how chaotic systems may be separated from stochastic ones and. at the same time. provides estimates of the degrees of freedom and the complexity of the underlying chaotic system. Based on this information. the second area fonnulates a state-space model for prediction of anomalies and incipient failures.
observed sequence sN = uO,ul.U2 ···uN.
Thus.
the collection of all probabilities P(sN). N 0,1.2··· defines a stochastic process that is a
=
symbolic probabilistic description of the continuous system dynamics. Nevertheless. the symbolic stochastic process depends on the specific partitioning of the phase space and is nonMarkovian. in general. Even if a partitioning that makes the stochastic process a Markov chain exists. identifICation of such a partitioning is not always feasible because the individual members may have complicated fractal boundaries instead of being simple geometrical objects. In essence. there is a trade-off between selecting a simple partitioning leading to a complicated stochastic process. and a complicated partitioning leading to a simple stochastic process.
2.2 TIme Series Analysis of Observed Data An alternative tool for behaviour identification of nonlinear dynamical systems is based on the concept of Formal Languagt!s for transitions from smooth dynamics in Eq. (I) to a discrete symbolic description (Baddi and Politi. 1997). Continuous time is first discrctized based on an appropriate Poincare section P. The resulting phase space of the dynamical system (I) through P is divided into cells. so as to obtain a "coordinate grid" for the dynamics. For simplicity. a compact (i.e .• closed and
1120
Given the intricacy of a symbolic pattern of unknown origin (i.e., with an unknown WlderJying dynamics), the challenge is to regenerate the pattern with a model to be selected within an appropriate class. The pattern may be identified by: (i) the singk-~m approach, leading to Kolmogorov Chiatin (KC) complexity, or algorithmic complexity (Cover and Thomas, 1991) for exact regeneration; or (ii) the ensemble approach (i.e., regarding the pattern as one of many possible outcomes of an experiment) for estimated regeneration. In the latter case, the model applies to the source itself and describes the set of rules common to all patterns produced by it. The single-item approach is commonly adopted in coding theory and computer science, while the ensemble approach has physical and statistical relevance. This paper makes use of an ensemble approach known as Computational Mechanics for discovering patterns in the behaviour of the dynamical system based on nonlinear time series analysis of observed data as described in the next section.
3 Computational Mec:hanJcs and E-madllnes Computational mechanics like statistical mechanics is concerned with large systems consisting of many individual components. However, motivating questions of computational mechanics centres on how a system of many components processes information: How is information stored, transmitted and transfonned? Whereas statistical mechanics starts with a system's desaiption of its components local space-time behaviour and interactions, computational mechanics begins with the joint probability distribution over the state space trajectories. With the knowledge of this joint distribution, the information processing being perfonned by the system can be inferred by a procedure known as e -machine reconstruction [Cru94J. This is developed using the statistical mechanics of orbit ensembles, rather than focusing on the computational complexity of individual orbits. Given a time series. the (un knowable) exact states of an observed system are translated into sequence of symbols via a measurement channel. Two histories (i.e.. two series of past data) carry equivalent information if they lead to the same (conditional) probability distribution in the future (i.e., if it makes no difference whether one or the other data-series is observed). Under these circumstances. i.e.. the effects of the two series being indistinguishable, they can be lumped together. This procedure identifies causal states, and also identifies the structure of connections or succession in causal states, and creates what is known as an "epsilon-machine". These e -machines form a special class of Deterministic Finite State Automata (DFSA) with transitions labelled with conditional probabilities and hence can also be viewed as Markov chains. The features of an E -machine are summarized below ICrutchfield 1994]: (i) it provides a minimal description of the pattern or regUlarities in a system in the sense that the pattern is an algebraic structure determined by the causal states and their transitions; (ii) it reflects a balanced utilization of deterministic and random information processing; (iii) it is unique
and optimal in the sense of maximal pn:dictive power and minimum model size according to the principle of Occam Razor (i.e. causes should not be increased beyond necessity); and (iv) it is minimally stochastic. However, an e -machine can only approximately represent chaotic motions of a dynamical system unless the number of casual states is infinite. In that case, the DFSA should be replaced by the next most powerful model in the hierarchy of machines known as the casual hierarchy (Crutchfield, 1994), in analogy with the Chomsky hierarchy of forrnallanguages [Martin, 1997). E -machines can be used to define and calculate macroscopic or global properties such as entropy rate, excess entropy and statistical complexity (Feldman and Crutchfield, 1998), that reflect the characteristic average information processing capabilities of the system. The entropy rate indicates predictability of the dynamical system. On the other hand, excess entropy provides a measure of the memory requirements in a spatial configuration and represents difficulties in performance prediction. An E -machine provides a natural measure of the statistical complexity of a process. namely the amount of information needed to specify the state of the e -machine (i.e. the Shannon entropy). Statistical Complexity is distinct from and dual to information-theoretic entropies and dimension (Crutchfield and Young, 1989). Existence of chaos, observed from time series data, exhibits a rich variety of unpredictability that spans the two extreme behaviours: periodic and random. The chaos that can be viewed as an amalgam of periodic and stochastic processes, contains rich information content in terms of diversity of temporal patterns that can be gainfully used for detection of small anomalies at an early stage. Statistical complexity captures the computational effort required in modelling the complex behaviour that cannot be directly obtained from the dynamical entropies. It is minimal for both deterministic and random behaviour. The role of the E -machine in this paper is to represent the behaviour of dynamical systems based on the observed time series data. A procedure for construction of E -machines is outlined below.
At first a symbolic representation of the dynamical system is constructed by partitioning the compact region of the phase space. Each partition is labelled by symbols in the alphabet (i.e., eTj e 1:) so that a trajectory in the phase space can be represented by a string of symbols as described earlier in Section 2. This string of symbols is then translated into a parse tree. Using Crutchfield's notation (CrulChfield and Young, 1989) a tree T={n,t} consists of nodes n = {nj } and directed. labelled links t = {t;} connecting them in a hierarchical structure with no
closed paths. An L -level subtree T"L starts at the node n and contains all nodes that can be reached within L links. 1be next step determines the minimum set of subtrees that describe the entire tree. A probabilistic structure is added to each tree node.
1121
P~. (L) = nj(L)1 N(L)
where nj(L) is the number of
I
occurrences of the associated L -length sequences relative to the tocal number N(L) of L -length sequences. The topological structure is described by an (-directed graph G IV, E), which consists of a set of vertices V and labelled direct edges E connecting them. Each graph node is associated to one of the basic subtrees and is known as a casual state.
=
Having identified the casual slates, the transition probabilities
T;/r>
the
between
states
upon
r
occurrence of symbols (7 E can be detennined. The causal states along with the transition probabilities fonns the E -machine and the directed graph G is its state-machine representation. The Tij of the connection matrix components
T = rT(O') , which is a stochastic matrix, yield the
O'er
probability of transition from i 'h to Statistical complexity of the obtained as:
iJ" - -
E
j'h
casual state.
-machine can be
r" Pi log Pi
;=1
where n =/V I and (PI P2'" p,,] is the eigenvector of T normalized in probability.
left
4 Anomaly DetectJoD Methodology This section describes construction of the proposed anomaly detection methodology. The concept is based on the fact that bifurcation of nonlinear systems represents a significant change in the qualitative behaviour that may occur with small parametric or non-parametric. Some of these changes may take place on their own accord and account for the anomaly while the others can be altered in a controlled fashion. This paper considers a class of non-linear nonautonomous electromechanical systems, which exhibit phenomena at two time scales. Anomalies occur at the slow time scale while the observation of the dynamical behaviour, based on which inferences are made, takes place at the fast time scale. It is assumed that: (i) the system behaviour is stationary at the fast time scale; and (ii) any observable nonstationary behaviour is associated with changes evolving at the slow time scale. The goal is to make inferences about evolving anomalies based on the asymptotic behaviour derived from the sensor data. However, only sufficient changes in the slowly varying parameter from the nominal condition may lead to a detectable difference in the asymptotic behaviour. The need to detect such small changes and hence early detection of an anomaly motivate the proposed stimulus-response approach. In this approach, the dynamical system is penurbed with an appropriate known input excitation to observe the asymptotic behaviour at the fast time scale. As a result of the combination of the input stimulus and penurbed parametcr(s), it might be possible to
1122
observe a detectable change in the nature of asymptotic behaviour that would otherwise remain unperceivable over a long period of time. The connection matrix T (see Section 3) is treated as a vector representation of any possible anomalies in the dynamical system. The induced 2-norrn of the difference between the T - matrices under nominal and anomalous conditions has been used in this paper as a measure of anomaly. There can be many other different measures of anomaly. For example, a possible measure of anomaly (that has not been used in this paper) is the metric of the absolute value of the differences in any induced norm or statistical complexity measure of the two respective E -
machines. The proposed anomaly detection methodology is separated into two pans: (i) Forward problem; and (ii) Inverse problem. The objective in the forward problem is to learn how the grammar underlying the system dynamics changes as the system parameters change. Following steps are identified to solve the forward problem: I Selection of appropriate Input Stimuli.
2 Signal-noise separation, time interval selection, and phase-space reconstruction from time series data (Abarbanel, 1996). 3 Choice of a phase space partitioning to generate Symbolic Dynamics. 4 £ -machine reconstruction using generated symbol sequence(s) and determining the connection matrix. The forward problem is that of learning, where the value of a parameter is associated with an anomaly measure. In contrast, the inverse problem is that of inferring the system parameters based on the observed asymptotic behaviour. In general, the inverse problem could be ill-posed and have no unique inverse. That is, it may not be possible to attribute a unique value to the parameter based on the observed behaviour of the system. Nevenheless, the feasible range of parameter variation estimates can be narrowed down from the intersection of the information generated from responses under several stimuli chosen in the forward problem. Following steps are identified to solve tbe inverse problem: I Excitation with known input stimuli selected in the forward problem. 2 Generation of the asymptotic behaviour for each input stimulus as time series data. 3 Embedding the time series data in the phase space determined for the corresponding input stimuli in Step 2 of the forward problem. 4 Generation of the symbol sequence using the same phase-space partition as in Step 3 of the forward problem. 5 E -machine reconstruction using the symbol sequence and determining the anomaly measure. 6 Identification of the parameter deviation based on the anomaly measure. This paper concentrates on the forward problem and illustrates it with an example in next section.
Iablc 2 6DDJllab: MCMIIlCS !W:4l- 2,00
5 An Example or Anomaly Detection We illustrate: the small change detection concept in a dynamical system using the second-order nonautonomous, forced Duffing equation (1bompson and Stewart, 1988) that is given by: 2
d
X (t)
dr 2
t
+P(t$)tix(d ) + x(t)+ x 3 (t)
t
=Acos«(t)t)
Ta)
Ta)
0.0028
0
Till 0.0028 0 T e) 0.7105 0.7105
(3)
T4)
The dissipation parameter P(t$) varies with the slow
Te)
T')
0.7105 0.7105 0 1.7322
1.7326 1.7326 1.7326 0
Tit)
1.7326 1.7326
'$
and it is treated as a constant in the fast time time scale at which the system is ellcited. TIle goal is to detect, at an early stage, changes in P(t s) which are associated with the anomaly. In the forward problem, the first task is selcction of appropriate input stimuli. Keeping the amplitude A = 22 to be a constant, we vary (t) within the range 1.0 to 2.0. Figure I shows the system responses as phase-plane (i.e., x(t) versus x(t» plots for four
i"
~
·10
Ta) Tb) Te) T')
0 0.5922 0.5922 0.5758
Tb) 0.5922 0 0 0.0165
Te) 0.5922 0 0 0.0165
r(l) 0
oS
io
.
5
5
10~''''''':''.. 0 .•
C;
... . . ......
·10 .......... ~.
·5
.
..
11(1) 0 (I)
~
0
~~~~~~
-Iot~
5
= 1.67
(1)=2.00
Figure I Phase Plots with Different Stimuli Next we follow the procedure. described in Section 4. to construct £ -machines and obtain the connection matrix for each phase plot in Figure 1. The £ -machines were constructed with the symbol alphabet I = {O, I} by partitioning the phase plots into two halves. with each half associated with a symbol 0 or I. For (t) = 1.67 • each phase plot in the left column of Figure 1 was partitioned by the ;c =0 axis. while for (t) 2.00 each phase plot in the right column of Figure I was partitioned by the = 0 axis. It should be noted that corresponding to each input ellcitation the phase space partitioning remains the same although. for different excitations. the partitioning can be chosen differently. Asymptotic parts of the time series data of phase trajectories were scanned to generate the symbolic strings. The time interval was selected using the minimum average mutual infonnation (Abarbanel. 1996).
=
Ta)
·10
io
remaining three phase plots at P= 0.100, 0.120, and 0.170, respectively, which are also almost similar to each other. These observations reveal that the stimuli at the excitation frequencies of (t) = 1.67 and (t) 2.00 can be effectively used for detection of small changes in the range of 0.100 < P < 0.120 and 0.170 < P < 0. 190, respectively. Additional stimuli can be identified which lead to significant changes in the asymptotic behaviour for other ranges of p . For eumple. a stimulus at the excitation frequency of (t) = 5.0 (not shown in Figure 1) detects small changes in the range 0.310 < P< 0.320. The information generated from different excitations would yield a narrow range of P exhibiting the current status of the anomaly.
Difference
0
10
=
flJ -
0
10
different values of the parameter P at 0.100, 0.120, 0.170, and 0.190 under the stimuli with the excitation frequencies of (t) = 1.67 (first column) and (t) = 2.00 (second column). As seen in these plates, the initial transients in all cases are different as the (random) initial condition is not the same although it is located within a ball of radius 0.2 centred at origin. This is to emphasize the fact that the initial conditions can never be exactly determined and hence the analysis is based on the asymptotic part of the response. The asymptotic behaviour of the system response in the left column of Figure I for P= 0.100 is significantly different from those of the remaining three phase plots at P 0.120, 0.170, and 0.190, respectively, which are quite similar to each other. Similarly, the asymptotic behaviour of the system response in the right column of Figure 1 for P = 0.190 is significantly different from those of the
Table J Anomaly Measures for
~
I.
=
J.67
T4) 0.5758 0.0165 0.0165 0
x
Tables 1 and 2 list the anomaly measure T(e). Be {a.b.c,d} based on induced 2-nonns of the differences between connection matrices of E machines under nominal and anomalous conditions.
1123
as discussed in Section 4. The system is assumed to be at the nominal condition for the dissipation parameter P 0.100. Each entry in Tables 1 and 2 represents the relative measure of anomaly that the system has been subjected to. In the slow time scale, the gradual increase of P signifies growth of anomaly. For example. entrees in the row
=
corresponding
11 T(8) -
T(a)
112,
10
T(a)
where
in T(t1):
Table
are
Further research is recommended for application of the anomaly detection technique 10 complex dynamical systems: • Separation of information-bearing part of the signal from noise;
from 0.100 to 0.120 is reflected in the large values of anomaly measure in first row in Table I. Similarly, the entries in fourth row in Table 2 capture the change in P to 0.190.
• Identification of the minimal submanifold of the phase space from time series data, which contains relevant information for anomaly detection;
It should be noted that the individual connection matrices do not have significantly different induced 2-norms although their differences may have. For
• Choice of partitioning of the phase space or a submanifold of the phase space 10 generate the symbol alphabet;
example, the induced 2-norms of each T(O) , Be {a,h.c,d} is approximately 1.25 and 1.23 for the CtJ
uuction of small anomal;~s.
6.1 Recommendations for Future Research
Be (a,h,c,d},
denotes the connection matrix obtained for the Emachines representing the asymptotic behaviour in the plates l(a)-l(d), respectively. The change in P
stimuli at CtJ = 1.67 and Table 2, respectively.
same stimulus is the measure of anomaly that the This vector system has been subjected to. representation of anomalies is more powerful than a scalar measwe. 'The major conclusion is: Symbolic Dynamics along with thL stimulusrespotlM IMthodology and having a ,,~ctor representation of anomaly is ~Jfect;v~ for ~arly
• Order reduction/augmentation of the causal states of an E -machine lO match those of another E -machine so that their stochastic mattices are compatible and have similar physical significance.
= 2.00 in Table 1 and
For each stimulus and range of p, we develop such tables in the forward problem described in Section 4. These tables provide inputs for the inverse problem to detect anomaly based on the sensor data collected in real time. Following Steps 1-5 of the inverse problem, described in Section 4, we obtain a connection matrix for the asymptotic behaviour under a known stimulus. The a prior; information on the anomaly measure is then used 10 determine the possible range in which the parameter lies. The range of unknown parameter can be further narrowed down by repeating this step for other known stimuli at the fast time scale.
References Abarbanel, H.D.l, (1996), ThL Analysis of Ohserv~d Clwotic Data, Springer-Verlag, New York. Badii, R. and Politi. A. (1997), Complexity HierarcmCIJI structures and scaling in physics, Cambridge University Press. United Kingdom. Cover, T. M. and Thomas. J. A. (1991), ElelMnts of ilifonnation Theory. John Wiley, New York. Crutchfield, J. P. and Young. K. (1989). "Inferring Statistical Complexity". Physical Review Letters 63, pp.
105-1OS.
6. Summary and Conclusions
Crutchfiekl. J.P. (1994). "The Calculi of Emergence: Computation. Dynamics and Induction" Physica D, 75, pp. 11-54.
TIlis paper presents a novel concept of anomaly detection in complex systems based on a combination of Nonlinear Systems theory and Language theory, and maJces use of the tools of Symbolic Dynamics and Pattern Discovery. It is assumed that dynamical systems under consideration exhibit nonlinear dynamical behaviour on two time scales. Anomalies occur on a slow time scale that is several orders of magnitude larger than the fast time scale of the system dynamics. It is also assumed that the unforced dynamical system (i.e., in the absence of external stimUli) is stationary at the fast time scale and that any non-stationary behaviour is observable only on the slow time scale.
Feldman, D. P. and Crutcbfiekl. J . P. (1998), "Discovering Non-c:ritic:al Orpnization: Statistical Mechanical. Information Theoretic. and Computational Views of Pauems in One-Dimensional Spin Systems". Sanla Fe InstitUk Working Paper 98-04·026. Martin. J.C. (1997), Inlroduction to, Lang/Ulges and tM 1Mory of Computation, 2'" ed., McGraw Hill, New York. Ott, E. (1993). Choos in Dynomical Systems. Cambridge University Press, United Kingdom.
Shalizi. C. R. and Crutchfield. J. P. (2001). "Computational Mechanics: Paucrn and Prediction. Structure and Simplicity". Jounwl of Statistical Physics, 104, pp. 817-879.
TIlis concept of small change detection in dynamical systems is elucidated from the solutions of the forced Duffing equation with a slowly varying dissipation parameter. The time series data of asymptotic phase trajectories are scanned 10 create the respective symbolic dynamics (i.e., strings of symbols). The stochastic matrix, derived from an individual Emachine, is treated as the vector representation of a phase trajectory's asymptotic behaviour. The distance between any two such vectors under the
Thompson. J.M.T. and Stewart. H.P. (1988). Nonlinear Dynomics and Choos. John Wiley. OIicbester, U.K . .
Vidyasagar. M. (1993). Nonlinear Systems ANIlysis. Prentice Hall, Englewood Cliffs, NJ. West. B. (1999), Physiology, Promisquity and Prophecy at The Millenium: A Tale of Tails. World Scientific, Singapore.
1124
c:
IFAC
0
[>
Publications www.elsevier.comllocatelifac
Anomaly Detection In Complex Systemst Amit Surana
Asok Ray
Shin Chin
The Pennsylvania State University University Park, PA 16802
Abstract: This paper presents a novel concept of anomaly detection in complex systems using tools of Symbolic Dynamics and Pattern Discovery. Anomaly is defined as a deviation from the nominal behaviour and can be associated with parametric changes that may gradually evolve. The proposed methodology is based on two-time-scale analysis of observed asymptotic behaviour of the dynamical system. This concept of small change detection in dynamical systems is elucidated through an example of the forced Duffing equation with a slowly varying dissipation parameter. Copyright © 2003 IFA C
Keywords: Fault Diagnosis and Prognosis; Symbolic Dynamics; Pattern Discovery; Complexity Measures 1 Introduction Engineering and scientific theories of control. communication and computing have matured in recent decades facilitating creation of systems of bewildering complexity almost comparable to that of biological systems. This complexity is largely hidden and cryptic in the laboratory environments as well as during normal operations of large-scale systems; however. it becomes conspicuous acutely when contributing to rare cascading failures (Badii and Politi. 1997; West, 1999). Early detection of anomalies in complex systems is not only essential for prevention of cascading catastrophic failures but also for enhanced system performance and availability. However. solely based on the fundamental principles of physics. accurate and computationally tractable modelling of complex system dynamics is often infeasible. One may have to rely on semi-empirical models whose parameters can be identified using time series data generated from sensors and other sources of information (Abarbanel. 1996). Accordingly, decision and control systems may have to be fonnulated based on such semi-empirical models and real-time data obtained through monitoring the system. The objective of the research work presented in this paper is to quantitatively detect the qualitative features of anomalies in complex systems using symbolic dynamics and pattern discovery. The theme of anomaly detection is presented below. The proposed methodology of anomaly detection is built upon the concepts of Dynamical Systems Theory, in which Symbolic Dynamics provides a qualitative description of dynamical behaviour in terms of symbol sequences. Computational
Mechanics provides a means to find patterns in these symbol sequences through the epsilon machine (Emachine) reconstruction (Crutchfield. 1994). Anomaly detection is then viewed as identifying variations in these patterns as parametric or nonparametric changes evolve. In order to facilitate small change detection in dynamical system parameters, we propose to excite the system with a priori known stimuli and discover anomaly patterns, if any, from the resulting responses. As an example, this paper considers a class of non-linear nonautonomous electro-mechanicaI systems in which anomalies occur at a slow time-scale while the inferences are made based on the observation of the fast time-scale system dynamics. The proposed procedure of anomaly detection relies on the two time-scale analysis of the asymptotic response of the dynamical system. The paper is organized in six sections. Section 2 provides the motivation of using Symbolic Dynamics as a means of encoding nonlinear dynamical systems. Section 3 presents a brief overview of Computational Mechanics from the perspectives of pattern discovery along with the procedure for E -machine reconstruction. The Emachines represent the behaviour of dynamical systems based on the observed time series data. Section 4 presents the underlying approach for anomaly detection. Section 5 illustrates the anomaly detection methodology on the well-mown Duffing equation as a prototype of complex electromechanical systems. Section 6 summarizes and concludes the paper with recommendations for future research.
t This woo has been supported in part by the U.S. Army Research Office under Grant No. DAADI9-0I·I.0646 and NASA GleM Research Center under Grant No. NAG3·2448.
1119
2 Symbolic DyDllDKs This section briefly describes the concept of Symbolic Dynamics and the motivation for using this concept as a means of encoding nonlinear systems. 2.1 Modelling and Analysis of Dynamical SysteJm Continuously varying physical processes are often modelled as a finite-dimensional dynamical system in the setting of an initial value problem as:
'd;> =!(x(t).O); x(O)=XO where tE [0.-) is the time;
XE
(I)
bounded) set g C Rn within which the motion is circumscribed is identified with the phase space itself. The encoding of g is accomplished by ~ ='1 Bo.···. Bb-tl introducing a partition ConSlSUng of nb mutually exclusive, i.e.,
Bj
nBIc =0
nb-I
U Bj j=O
Vj ~ k. and exhaustive subsets, i.e.•
=X . The
dynamical system describes an
orbit O. (XO.xl···.xn ...) • which passes through or
9t n is the state
touches various elements of the partition ~. Let us denote the index of domain BE tjJ visited at the time instant i as the symbol Uj Er. The set
vector in the phas~ spact!; and IJ e 9t m is the (possibly slowly) varying parameter vector. The vector field !(x.lJ) is, in general, a (locally) Lipschtiz nonlinear operator that suffices to ensure a unique local solution to the differential equation (I) (Vidyasagar. 1993). Formally such a solution can be expressed as a continuous function of the initial value Xo as ..t(t) = fPl XO where fP I represents a one-
r
=(O.···.nb -I} of symbols labelling the partition elements is called the symbol alphabt!t. Each initial state
Xo
generates a sequence of
symbols defined by a mapping from the phase space to the symbol space as:
parameter family of maps of the phase space into itself. This evolution in time t can be viewed as a flow of points in the phase space. For a dissipative system, the flow may contract onto sets of lower dimension. known as attractors. which represent the asymptotic behaviour of the dynamical system, characterized by several phenomena (e.g., {lXt!d poilll, limit cyclt!. qUIJSi-pt!riodicily. clwos. !ractal bt!haviollr. and bijiucation) (Ott, 1993).
XO 1-+ Uo ul,··un ...
(2)
Such a mapping is called Symbolic Dynamics as it attributes a l~gal (i.e.• physically admissible) symbol sequence to the system dynamics starting from an initial state XO. Since the size of each cell is finite and the cardinality of the alphabet 1: is finite. any such symbol sequence represents. through iteration. a phase trajectory that has a compact support Q in the phase space. The partition of g represented by the symbol alphabet 1: is called "generating one" if a symbol sequence uO.ul.u2··· uniquely
Modelling of a physical process. having the mathematical structure of Eq. (I), may not always be feasible solely based on the fundamental principles of physics. A convenient way of learning the dynamical behaviour is to rely on the (sensor-based) observed behaviour. The need to extract relevant physical information about the observed dynamics when they are operating in a chaotic regime has lead to development of non/war timt! ~rit!s analysis (NTSA) techniques. Analysis of (potentially) chaotic systems using NTSA techniques is classified into two areas (Abarbanel. 1996): • Bt!haviour identification • Mod~lling and Prt!diction
detennines a specific initial condition XO. That is. the mapping from the phase space to the symbol space is invertible. In general. a dynamical system would allow only legal concatenations of symbols to occur as there are many ilJegaJ (i.e., physically inadmissible) sequences. A grammar (i.e.• a set of rules) that detennines legality of symbol strings in the alphabet 1: may change with the parameter vector IJ. Among the legal symbol sequences. some symbols may occur more frequently than others. Hence, a certain probability p(s N ) is attributed to each
The first area shows how chaotic systems may be separated from stochastic ones and. at the same time. provides estimates of the degrees of freedom and the complexity of the underlying chaotic system. Based on this information. the second area fonnulates a state-space model for prediction of anomalies and incipient failures.
observed sequence sN = uO,ul.U2 ···uN.
Thus.
the collection of all probabilities P(sN). N 0,1.2··· defines a stochastic process that is a
=
symbolic probabilistic description of the continuous system dynamics. Nevertheless. the symbolic stochastic process depends on the specific partitioning of the phase space and is nonMarkovian. in general. Even if a partitioning that makes the stochastic process a Markov chain exists. identifICation of such a partitioning is not always feasible because the individual members may have complicated fractal boundaries instead of being simple geometrical objects. In essence. there is a trade-off between selecting a simple partitioning leading to a complicated stochastic process. and a complicated partitioning leading to a simple stochastic process.
2.2 TIme Series Analysis of Observed Data An alternative tool for behaviour identification of nonlinear dynamical systems is based on the concept of Formal Languagt!s for transitions from smooth dynamics in Eq. (I) to a discrete symbolic description (Baddi and Politi. 1997). Continuous time is first discrctized based on an appropriate Poincare section P. The resulting phase space of the dynamical system (I) through P is divided into cells. so as to obtain a "coordinate grid" for the dynamics. For simplicity. a compact (i.e .• closed and
1120
Given the intricacy of a symbolic pattern of unknown origin (i.e., with an unknown WlderJying dynamics), the challenge is to regenerate the pattern with a model to be selected within an appropriate class. The pattern may be identified by: (i) the singk-~m approach, leading to Kolmogorov Chiatin (KC) complexity, or algorithmic complexity (Cover and Thomas, 1991) for exact regeneration; or (ii) the ensemble approach (i.e., regarding the pattern as one of many possible outcomes of an experiment) for estimated regeneration. In the latter case, the model applies to the source itself and describes the set of rules common to all patterns produced by it. The single-item approach is commonly adopted in coding theory and computer science, while the ensemble approach has physical and statistical relevance. This paper makes use of an ensemble approach known as Computational Mechanics for discovering patterns in the behaviour of the dynamical system based on nonlinear time series analysis of observed data as described in the next section.
3 Computational Mec:hanJcs and E-madllnes Computational mechanics like statistical mechanics is concerned with large systems consisting of many individual components. However, motivating questions of computational mechanics centres on how a system of many components processes information: How is information stored, transmitted and transfonned? Whereas statistical mechanics starts with a system's desaiption of its components local space-time behaviour and interactions, computational mechanics begins with the joint probability distribution over the state space trajectories. With the knowledge of this joint distribution, the information processing being perfonned by the system can be inferred by a procedure known as e -machine reconstruction [Cru94J. This is developed using the statistical mechanics of orbit ensembles, rather than focusing on the computational complexity of individual orbits. Given a time series. the (un knowable) exact states of an observed system are translated into sequence of symbols via a measurement channel. Two histories (i.e.. two series of past data) carry equivalent information if they lead to the same (conditional) probability distribution in the future (i.e., if it makes no difference whether one or the other data-series is observed). Under these circumstances. i.e.. the effects of the two series being indistinguishable, they can be lumped together. This procedure identifies causal states, and also identifies the structure of connections or succession in causal states, and creates what is known as an "epsilon-machine". These e -machines form a special class of Deterministic Finite State Automata (DFSA) with transitions labelled with conditional probabilities and hence can also be viewed as Markov chains. The features of an E -machine are summarized below ICrutchfield 1994]: (i) it provides a minimal description of the pattern or regUlarities in a system in the sense that the pattern is an algebraic structure determined by the causal states and their transitions; (ii) it reflects a balanced utilization of deterministic and random information processing; (iii) it is unique
and optimal in the sense of maximal pn:dictive power and minimum model size according to the principle of Occam Razor (i.e. causes should not be increased beyond necessity); and (iv) it is minimally stochastic. However, an e -machine can only approximately represent chaotic motions of a dynamical system unless the number of casual states is infinite. In that case, the DFSA should be replaced by the next most powerful model in the hierarchy of machines known as the casual hierarchy (Crutchfield, 1994), in analogy with the Chomsky hierarchy of forrnallanguages [Martin, 1997). E -machines can be used to define and calculate macroscopic or global properties such as entropy rate, excess entropy and statistical complexity (Feldman and Crutchfield, 1998), that reflect the characteristic average information processing capabilities of the system. The entropy rate indicates predictability of the dynamical system. On the other hand, excess entropy provides a measure of the memory requirements in a spatial configuration and represents difficulties in performance prediction. An E -machine provides a natural measure of the statistical complexity of a process. namely the amount of information needed to specify the state of the e -machine (i.e. the Shannon entropy). Statistical Complexity is distinct from and dual to information-theoretic entropies and dimension (Crutchfield and Young, 1989). Existence of chaos, observed from time series data, exhibits a rich variety of unpredictability that spans the two extreme behaviours: periodic and random. The chaos that can be viewed as an amalgam of periodic and stochastic processes, contains rich information content in terms of diversity of temporal patterns that can be gainfully used for detection of small anomalies at an early stage. Statistical complexity captures the computational effort required in modelling the complex behaviour that cannot be directly obtained from the dynamical entropies. It is minimal for both deterministic and random behaviour. The role of the E -machine in this paper is to represent the behaviour of dynamical systems based on the observed time series data. A procedure for construction of E -machines is outlined below.
At first a symbolic representation of the dynamical system is constructed by partitioning the compact region of the phase space. Each partition is labelled by symbols in the alphabet (i.e., eTj e 1:) so that a trajectory in the phase space can be represented by a string of symbols as described earlier in Section 2. This string of symbols is then translated into a parse tree. Using Crutchfield's notation (CrulChfield and Young, 1989) a tree T={n,t} consists of nodes n = {nj } and directed. labelled links t = {t;} connecting them in a hierarchical structure with no
closed paths. An L -level subtree T"L starts at the node n and contains all nodes that can be reached within L links. 1be next step determines the minimum set of subtrees that describe the entire tree. A probabilistic structure is added to each tree node.
1121
P~. (L) = nj(L)1 N(L)
where nj(L) is the number of
I
occurrences of the associated L -length sequences relative to the tocal number N(L) of L -length sequences. The topological structure is described by an (-directed graph G IV, E), which consists of a set of vertices V and labelled direct edges E connecting them. Each graph node is associated to one of the basic subtrees and is known as a casual state.
=
Having identified the casual slates, the transition probabilities
T;/r>
the
between
states
upon
r
occurrence of symbols (7 E can be detennined. The causal states along with the transition probabilities fonns the E -machine and the directed graph G is its state-machine representation. The Tij of the connection matrix components
T = rT(O') , which is a stochastic matrix, yield the
O'er
probability of transition from i 'h to Statistical complexity of the obtained as:
iJ" - -
E
j'h
casual state.
-machine can be
r" Pi log Pi
;=1
where n =/V I and (PI P2'" p,,] is the eigenvector of T normalized in probability.
left
4 Anomaly DetectJoD Methodology This section describes construction of the proposed anomaly detection methodology. The concept is based on the fact that bifurcation of nonlinear systems represents a significant change in the qualitative behaviour that may occur with small parametric or non-parametric. Some of these changes may take place on their own accord and account for the anomaly while the others can be altered in a controlled fashion. This paper considers a class of non-linear nonautonomous electromechanical systems, which exhibit phenomena at two time scales. Anomalies occur at the slow time scale while the observation of the dynamical behaviour, based on which inferences are made, takes place at the fast time scale. It is assumed that: (i) the system behaviour is stationary at the fast time scale; and (ii) any observable nonstationary behaviour is associated with changes evolving at the slow time scale. The goal is to make inferences about evolving anomalies based on the asymptotic behaviour derived from the sensor data. However, only sufficient changes in the slowly varying parameter from the nominal condition may lead to a detectable difference in the asymptotic behaviour. The need to detect such small changes and hence early detection of an anomaly motivate the proposed stimulus-response approach. In this approach, the dynamical system is penurbed with an appropriate known input excitation to observe the asymptotic behaviour at the fast time scale. As a result of the combination of the input stimulus and penurbed parametcr(s), it might be possible to
1122
observe a detectable change in the nature of asymptotic behaviour that would otherwise remain unperceivable over a long period of time. The connection matrix T (see Section 3) is treated as a vector representation of any possible anomalies in the dynamical system. The induced 2-norrn of the difference between the T - matrices under nominal and anomalous conditions has been used in this paper as a measure of anomaly. There can be many other different measures of anomaly. For example, a possible measure of anomaly (that has not been used in this paper) is the metric of the absolute value of the differences in any induced norm or statistical complexity measure of the two respective E -
machines. The proposed anomaly detection methodology is separated into two pans: (i) Forward problem; and (ii) Inverse problem. The objective in the forward problem is to learn how the grammar underlying the system dynamics changes as the system parameters change. Following steps are identified to solve the forward problem: I Selection of appropriate Input Stimuli.
2 Signal-noise separation, time interval selection, and phase-space reconstruction from time series data (Abarbanel, 1996). 3 Choice of a phase space partitioning to generate Symbolic Dynamics. 4 £ -machine reconstruction using generated symbol sequence(s) and determining the connection matrix. The forward problem is that of learning, where the value of a parameter is associated with an anomaly measure. In contrast, the inverse problem is that of inferring the system parameters based on the observed asymptotic behaviour. In general, the inverse problem could be ill-posed and have no unique inverse. That is, it may not be possible to attribute a unique value to the parameter based on the observed behaviour of the system. Nevenheless, the feasible range of parameter variation estimates can be narrowed down from the intersection of the information generated from responses under several stimuli chosen in the forward problem. Following steps are identified to solve tbe inverse problem: I Excitation with known input stimuli selected in the forward problem. 2 Generation of the asymptotic behaviour for each input stimulus as time series data. 3 Embedding the time series data in the phase space determined for the corresponding input stimuli in Step 2 of the forward problem. 4 Generation of the symbol sequence using the same phase-space partition as in Step 3 of the forward problem. 5 E -machine reconstruction using the symbol sequence and determining the anomaly measure. 6 Identification of the parameter deviation based on the anomaly measure. This paper concentrates on the forward problem and illustrates it with an example in next section.
Iablc 2 6DDJllab: MCMIIlCS !W:4l- 2,00
5 An Example or Anomaly Detection We illustrate: the small change detection concept in a dynamical system using the second-order nonautonomous, forced Duffing equation (1bompson and Stewart, 1988) that is given by: 2
d
X (t)
dr 2
t
+P(t$)tix(d ) + x(t)+ x 3 (t)
t
=Acos«(t)t)
Ta)
Ta)
0.0028
0
Till 0.0028 0 T e) 0.7105 0.7105
(3)
T4)
The dissipation parameter P(t$) varies with the slow
Te)
T')
0.7105 0.7105 0 1.7322
1.7326 1.7326 1.7326 0
Tit)
1.7326 1.7326
'$
and it is treated as a constant in the fast time time scale at which the system is ellcited. TIle goal is to detect, at an early stage, changes in P(t s) which are associated with the anomaly. In the forward problem, the first task is selcction of appropriate input stimuli. Keeping the amplitude A = 22 to be a constant, we vary (t) within the range 1.0 to 2.0. Figure I shows the system responses as phase-plane (i.e., x(t) versus x(t» plots for four
i"
~
·10
Ta) Tb) Te) T')
0 0.5922 0.5922 0.5758
Tb) 0.5922 0 0 0.0165
Te) 0.5922 0 0 0.0165
r(l) 0
oS
io
.
5
5
10~''''''':''.. 0 .•
C;
... . . ......
·10 .......... ~.
·5
.
..
11(1) 0 (I)
~
0
~~~~~~
-Iot~
5
= 1.67
(1)=2.00
Figure I Phase Plots with Different Stimuli Next we follow the procedure. described in Section 4. to construct £ -machines and obtain the connection matrix for each phase plot in Figure 1. The £ -machines were constructed with the symbol alphabet I = {O, I} by partitioning the phase plots into two halves. with each half associated with a symbol 0 or I. For (t) = 1.67 • each phase plot in the left column of Figure 1 was partitioned by the ;c =0 axis. while for (t) 2.00 each phase plot in the right column of Figure I was partitioned by the = 0 axis. It should be noted that corresponding to each input ellcitation the phase space partitioning remains the same although. for different excitations. the partitioning can be chosen differently. Asymptotic parts of the time series data of phase trajectories were scanned to generate the symbolic strings. The time interval was selected using the minimum average mutual infonnation (Abarbanel. 1996).
=
Ta)
·10
io
remaining three phase plots at P= 0.100, 0.120, and 0.170, respectively, which are also almost similar to each other. These observations reveal that the stimuli at the excitation frequencies of (t) = 1.67 and (t) 2.00 can be effectively used for detection of small changes in the range of 0.100 < P < 0.120 and 0.170 < P < 0. 190, respectively. Additional stimuli can be identified which lead to significant changes in the asymptotic behaviour for other ranges of p . For eumple. a stimulus at the excitation frequency of (t) = 5.0 (not shown in Figure 1) detects small changes in the range 0.310 < P< 0.320. The information generated from different excitations would yield a narrow range of P exhibiting the current status of the anomaly.
Difference
0
10
=
flJ -
0
10
different values of the parameter P at 0.100, 0.120, 0.170, and 0.190 under the stimuli with the excitation frequencies of (t) = 1.67 (first column) and (t) = 2.00 (second column). As seen in these plates, the initial transients in all cases are different as the (random) initial condition is not the same although it is located within a ball of radius 0.2 centred at origin. This is to emphasize the fact that the initial conditions can never be exactly determined and hence the analysis is based on the asymptotic part of the response. The asymptotic behaviour of the system response in the left column of Figure I for P= 0.100 is significantly different from those of the remaining three phase plots at P 0.120, 0.170, and 0.190, respectively, which are quite similar to each other. Similarly, the asymptotic behaviour of the system response in the right column of Figure 1 for P = 0.190 is significantly different from those of the
Table J Anomaly Measures for
~
I.
=
J.67
T4) 0.5758 0.0165 0.0165 0
x
Tables 1 and 2 list the anomaly measure T(e). Be {a.b.c,d} based on induced 2-nonns of the differences between connection matrices of E machines under nominal and anomalous conditions.
1123
as discussed in Section 4. The system is assumed to be at the nominal condition for the dissipation parameter P 0.100. Each entry in Tables 1 and 2 represents the relative measure of anomaly that the system has been subjected to. In the slow time scale, the gradual increase of P signifies growth of anomaly. For example. entrees in the row
=
corresponding
11 T(8) -
T(a)
112,
10
T(a)
where
in T(t1):
Table
are
Further research is recommended for application of the anomaly detection technique 10 complex dynamical systems: • Separation of information-bearing part of the signal from noise;
from 0.100 to 0.120 is reflected in the large values of anomaly measure in first row in Table I. Similarly, the entries in fourth row in Table 2 capture the change in P to 0.190.
• Identification of the minimal submanifold of the phase space from time series data, which contains relevant information for anomaly detection;
It should be noted that the individual connection matrices do not have significantly different induced 2-norms although their differences may have. For
• Choice of partitioning of the phase space or a submanifold of the phase space 10 generate the symbol alphabet;
example, the induced 2-norms of each T(O) , Be {a,h.c,d} is approximately 1.25 and 1.23 for the CtJ
uuction of small anomal;~s.
6.1 Recommendations for Future Research
Be (a,h,c,d},
denotes the connection matrix obtained for the Emachines representing the asymptotic behaviour in the plates l(a)-l(d), respectively. The change in P
stimuli at CtJ = 1.67 and Table 2, respectively.
same stimulus is the measure of anomaly that the This vector system has been subjected to. representation of anomalies is more powerful than a scalar measwe. 'The major conclusion is: Symbolic Dynamics along with thL stimulusrespotlM IMthodology and having a ,,~ctor representation of anomaly is ~Jfect;v~ for ~arly
• Order reduction/augmentation of the causal states of an E -machine lO match those of another E -machine so that their stochastic mattices are compatible and have similar physical significance.
= 2.00 in Table 1 and
For each stimulus and range of p, we develop such tables in the forward problem described in Section 4. These tables provide inputs for the inverse problem to detect anomaly based on the sensor data collected in real time. Following Steps 1-5 of the inverse problem, described in Section 4, we obtain a connection matrix for the asymptotic behaviour under a known stimulus. The a prior; information on the anomaly measure is then used 10 determine the possible range in which the parameter lies. The range of unknown parameter can be further narrowed down by repeating this step for other known stimuli at the fast time scale.
References Abarbanel, H.D.l, (1996), ThL Analysis of Ohserv~d Clwotic Data, Springer-Verlag, New York. Badii, R. and Politi. A. (1997), Complexity HierarcmCIJI structures and scaling in physics, Cambridge University Press. United Kingdom. Cover, T. M. and Thomas. J. A. (1991), ElelMnts of ilifonnation Theory. John Wiley, New York. Crutchfield, J. P. and Young. K. (1989). "Inferring Statistical Complexity". Physical Review Letters 63, pp.
105-1OS.
6. Summary and Conclusions
Crutchfiekl. J.P. (1994). "The Calculi of Emergence: Computation. Dynamics and Induction" Physica D, 75, pp. 11-54.
TIlis paper presents a novel concept of anomaly detection in complex systems based on a combination of Nonlinear Systems theory and Language theory, and maJces use of the tools of Symbolic Dynamics and Pattern Discovery. It is assumed that dynamical systems under consideration exhibit nonlinear dynamical behaviour on two time scales. Anomalies occur on a slow time scale that is several orders of magnitude larger than the fast time scale of the system dynamics. It is also assumed that the unforced dynamical system (i.e., in the absence of external stimUli) is stationary at the fast time scale and that any non-stationary behaviour is observable only on the slow time scale.
Feldman, D. P. and Crutcbfiekl. J . P. (1998), "Discovering Non-c:ritic:al Orpnization: Statistical Mechanical. Information Theoretic. and Computational Views of Pauems in One-Dimensional Spin Systems". Sanla Fe InstitUk Working Paper 98-04·026. Martin. J.C. (1997), Inlroduction to, Lang/Ulges and tM 1Mory of Computation, 2'" ed., McGraw Hill, New York. Ott, E. (1993). Choos in Dynomical Systems. Cambridge University Press, United Kingdom.
Shalizi. C. R. and Crutchfield. J. P. (2001). "Computational Mechanics: Paucrn and Prediction. Structure and Simplicity". Jounwl of Statistical Physics, 104, pp. 817-879.
TIlis concept of small change detection in dynamical systems is elucidated from the solutions of the forced Duffing equation with a slowly varying dissipation parameter. The time series data of asymptotic phase trajectories are scanned 10 create the respective symbolic dynamics (i.e., strings of symbols). The stochastic matrix, derived from an individual Emachine, is treated as the vector representation of a phase trajectory's asymptotic behaviour. The distance between any two such vectors under the
Thompson. J.M.T. and Stewart. H.P. (1988). Nonlinear Dynomics and Choos. John Wiley. OIicbester, U.K . .
Vidyasagar. M. (1993). Nonlinear Systems ANIlysis. Prentice Hall, Englewood Cliffs, NJ. West. B. (1999), Physiology, Promisquity and Prophecy at The Millenium: A Tale of Tails. World Scientific, Singapore.
1124