- Email: [email protected]
A cosine similarity-based negative selection algorithm for time series novelty detection
ARTICLE IN PRESS
Mechanical Systems and Signal Processing Mechanical Systems and Signal Processing 20 (2006) 1461–1472 www.elsevier.com/locate/jnlabr/ymssp
A cosine similarity-based negative selection algorithm for time series novelty detection Yonggui Dong, Zhaoyan Sun, Huibo Jia State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instruments and Mechanology, Tsinghua University, Room 3514, Beijing 100084, P.R. China Received 10 November 2003; received in revised form 7 December 2004; accepted 29 December 2004 Available online 5 March 2005
Abstract Detecting the new or anomalous signal sequences in the observed time series data is a problem of great practical interest for many applications. The bio-inspired negative selection algorithm, whose main idea is to discriminate the non-self pattern from self pattern, has drawn much attention because only normal information is needed for training. Most of the proposed algorithms are based on binary-valued string matching. A real-valued negative selection algorithm for novelty detection in vibration signal is implemented in this paper. The vector set for calculation is constructed by sampling the discrete time series from a moving time window. The matching affinity between two vectors is measured by cosine similarity. The calculated results show that the cosine similarity-based algorithm is more practical for potential applications in online signal monitoring. r 2005 Elsevier Ltd. All rights reserved.
1. Introduction In most industrial applications, the behaviour of a system is often characterised by observing and analysing a series of parameters over time. Detecting the anomalous signal sequences in the observed time series data is an important task for system monitoring and diagnosis. A common approach is to design a classifier, for which one or more of the classes are allocated to the Corresponding author.
E-mail address: [email protected] (Y. Dong). 0888-3270/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.ymssp.2004.12.006
ARTICLE IN PRESS 1462
Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
abnormal patterns corresponding to those faults to be identified. Most existing methods require prior knowledge about various novelty conditions [1]. Given the fact that one can never train a monitoring system on all possible cases whose data the monitored system is likely to encounter, novelty detection, whose object is to differentiate between known and unknown object information during monitoring, has become one important but extremely challenging task in practice. Novelty detection is an approach to the problem of fault detection, which only requires data from the normal class. It is commonly used to implement a damage detection strategy for structural health monitoring in aerospace, civil and mechanical engineering infrastructure [2,3]. There exist several models of novelty detection that have been shown to perform well on different data. Statistical pattern recognition techniques, including neural networks, are usually applied to the problems [4,5]. The neural network-based methods have been widely used for mechanical systems such as jet engines [6]. The k-nearest neighbour algorithm is another approach for novelty detection. One set for normal signal patterns is established. If the monitored signal pattern is at a greater distance from each element of the normal set, a novelty is declared. This method has been used for rotor fault detection [7], as well as industrial process monitoring [8]. The main problem with this technique is that usually large-sized datasets need to be maintained and correspondingly, a large number of computations have to be performed. Over the last few years, a bio-inspired negative selection algorithm has been proposed for novelty detection in time series data [9]. The main idea of the algorithm is based on ideas from the natural immune system. The natural immune system detects the foreign antigens in a way known as self–non-self discrimination. Inspired from such a mechanism, Forrest proposed an artificial immune system for computer network security [10]. It is a simplification of the complex chemistry of antibody/antigen recognition in natural immune systems. The parameters to be monitored are encoded to binary strings with definite length. The self–non-self discrimination is implemented by applying an r-contiguous-bits matching rule to the binary strings [11,12]. Dasgupta forwards the binary-based algorithm to industrial applications, especially tool breakage detection [13,14]. Unlike the case of binary-based computer networks, the binary coding performs a very low level representation for signal monitoring. Therefore, the real-valued negative selection algorithms are preferred and proposed [15–17]. Euclidean distance is usually used for discrimination calculation. Our approach is to implement a more effective real-valued algorithm for monitoring time series data. The vector set for calculation is constructed by sampling the discrete vibration signal from a moving time window. Cosine similarity is used for calculating the affinity between two vectors. The calculated results indicate that the cosine similarity-based algorithm is more practical for potential applications in online signal monitoring.
2. Real-valued negative selection algorithm Natural immune systems are able to detect the infection of any foreign cell or molecule (antigen). The detection is accomplished only when the invading antigen is ‘‘recognised’’ by a specific antibody or T-cell receptor. Therefore, the diversity of the T-cell receptors must be large enough to cover all possible antigens that the immune system may encounter. Moreover, the
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1463
binding between the receptors and the body’s own (self) cells must be avoided. The immune system implements such an immune detection in a way known as self–non-self discrimination. The T-cell receptors are created by a random genetic rearrangement process. Those cells that successfully bind with self-cells are destroyed in the thymus. Only those cells that fail to bind to self-cells are allowed to leave the thymus and become antibodies of the body’s immune system. Such an antibody-generation-selection process is called negative selection. The main purpose of novelty detection is to distinguish self, which corresponds to the normal states of the monitored system, from non-self, which corresponds to the novel or anomalous states. Similar to the negative selection process illustrated above, two sets are maintained for the artificial immune system, the self-set S for normal data series and the antibody set D for novelty detectors. The following definitions and parameters are utilised in this paper.
x(i) is utilised to denote the discrete time series sampled from the continuous signals, where i
~ x1 ¼ ðx1 ; x2 ; x3 ; x4 ; x5 Þ, ~ x2 ¼ ðx6 ; x7 ; x8 ; x9 ; x10 Þ, ~ x3 ¼ ðx11 ; x12 ; x13 ; x14 ; x15 Þ, .. .
l
x(i)
denotes sampling point number (time). Especially, xs(i) is used to denote the time series that are considered as normal for the system. l-dimensional vectors set UDRl is defined as the feature collection that to be monitored. The ldimensional vectors in U can be obtained from the time series x(i). As shown in Fig. 1, slide one window with length l along time (sampling points) axis, the x(i) is separated into equal-sized data segments. The segmenting window can be overlapped, or non-overlapped. For example, xðiÞ ¼ fx1 ; x2 ; . . . ; xn g; let the window length l ¼ 5; in non-overlapped cases, the l-dimensional vectors can be constructed as
Sampling points(i)
Fig. 1. Separate the time series into equal-sized data segments by a sliding window.
ARTICLE IN PRESS 1464
Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
Correspondingly, for overlapped cases, say, l ¼ 5 and the step of the sliding window step_s ¼ 1; the l-dimensional vectors are ~ x1 ¼ ðx1 ; x2 ; x3 ; x4 ; x5 Þ, ~ x2 ¼ ðx2 ; x3 ; x4 ; x5 ; x6 Þ, ~ x3 ¼ ðx3 ; x4 ; x5 ; x6 ; x7 Þ, .. .
By the method mentioned above, the l-dimensional vectors can be constructed from the time series corresponding to both the normal and the anomalous system states. The overall vectors collection is denoted as U. The vector constructed from the normal time series data is ~ xis 2 U: On the other hand, the i vector constructed from the monitored time series data is ~ xm 2 U: The vector in antibody set D is ~ xid 2 U: The subset for normal vectors is self-set SDU, and the subset for novelty data segments is non-self-set NDU. Since U is S the collection T of all the possible vectors, both for normal and anomalous cases, there exist S N ¼ U, S N ¼ F, where F denotes a blank set that contains no vectors.
It can be seen that the antibody set D is actually the subset of N, i.e. DDN. Let dð~ x; ~ yÞ represent l the distance function over space R , the negative selection algorithm for time series novelty detection can be implemented as follows. 2.1. Self-set S construction Assume that we have collected enough time series data xraw(i) from a normally operating system. Subtract the average value of xraw(i) to eliminate the influence of the offset value and get an xs(i) (the reason for the arrangement of such a preprocessing step will be described later). Then apply one sliding window with length l to xs(i); this results in the construction of self-set S. 2.2. Matching threshold for the self-set For each vector ~ xis 2 S; calculate the minimum distance between itself and other vectors within S to get lis ¼ j min dð~ xis ; ~ xjs Þ,
(1)
~ xs 2S;jai
* *
*
where function dðx ; y Þ denotes the distance between the two l-dimensional vectors ~ x and y : Then the maximum value of all lis ’s is obtained by ls ¼ maxðlis Þ: After that, determine a matching threshold rs for the self-set S according to ls. The threshold rs must be greater than ls in order that all vectors constructed from the normal time series data can be classified into the self-set. A reasonably greater rs would benefit in avoiding the ‘‘false positive’’ detection, in which case the normal data segment is classified as abnormal. On the other hand, the threshold rs should not be so great that the detection system becomes insensitive to real abnormal data. If so, the ‘‘false negative’’ detection would happen if the abnormal data are classified as normal.
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1465
2.3. Antibody set D generation Utilising a uniformly distributed pseudo-random generator, generate the candidate detector x 2 ½1:0; 1:0l : Establish the antibodyc set D with the negative selection algorithm shown in * Fig. 2. First, the candidate detector x is matched with all the vectors in the self-set S by xis Þ: If the minimum value of all the distances is less than the calculating the distancesc dð~ xc ; ~ * threshold ls, cthen the x is considered as representing a normal data segment and cancelled. If * not, i.e. the x represents an abnormal data segment, it is matched with the vectors in antibody set D to determine whether it has been covered by any of the vectors in D. Certainly, if this is the first candidate detector for D, it will become one new detector without a matching calculation. Unlike the algorithm proposed by Forrest [10] that all antibody elements have the same *i matching threshold, each detector x d in D has its own matching threshold rid in our approach, as xid Þ is less than the corresponding rid ; it can be determined shown in Fig. 3. Therefore, if any dð~ xc ; ~ *c *i *c that the x has been covered by the detector x d and should be cancelled. Otherwise, x becomes a c *i * new detector in D and the corresponding threshold is given as rcd ¼ min dðx ; x s Þ rs : S In such a way, any vector in the antibody set D has two features: (a). The vector itself is not in *i *j the area centered by any x s 2 S with the radius rs : (b). For any vector x ; if there exists one i j i * * * xj must be a non-self-vector and the corresponding data x d 2 D that fulfils dðx ; x d Þorid ; then ~ segment can be classified as abnormal. *c
2.4. Time series data novelty detection When monitoring the time series data,i preprocess the data in the same way as described in step * *i 1 to obtain the corresponding vector x m : Calculate the distances between x m and each vector
Random vector generator
Self-set S
ϖc
rs
x
min d S
ϖc ϖi (x , xs)
< rs ? Y
N
Antibody-set D
cancel
ϖ ϖ
min(d (xc, xid) - rdi ) < 0 ? D
Y
N new detector .. £ add
into
rdc =
cancel
antibody-set£ © ϖ ϖ
min d (xc, xsi ) - rs
Fig. 2. Antibody set generation.
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1466
x si
rs xdi
rdi
Fig. 3. Matching threshold of detector in antibody set D.
*j
*i
*j
within D. If there exists one x d that fulfils d ðx m ; x d Þorjd ; then the data segment corresponding to *i x m is novel or anomalous. It can be seen that the performance of the algorithm described above would be greatly * * * * influenced by the distance measure dðx ; y Þ of vectors x ¼ ðx1; x2 ; . . . ; xl Þ and y ¼ ðy1; y2 ; . . . ; yl Þ: Considering that the time series novelty usually performs as the corresponding data segment is ‘‘not similar to’’ the normal data segments, we choose the cosine similarity as the affinity function between two l-dimensional vectors. l P
xi yi simðx ; y Þ ¼ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi . l l P P x2i y2i * *
i¼1
i¼1
(2)
i¼1
This function gives similarity measure in the sense that the cosine value gets larger as the two vectors become more nearly parallel in the l-dimensional space. Or in other words, as the two data segments become more similar, their cosine similarity approaches 1.0 and their distance approaches 0.0. Therefore, the cosine similarity can be used as a distance metric in the following way: * *
* *
dðx ; y Þ ¼ 1 sim ðx ; y Þ.
(3)
Since the cosine similarity reflects the magnitude of the angle between two vectors in the ldimensional space, it is a many-to-one function. Compared with the other distance measures, like Euclidean distance, the cosine similarity ignores the magnitude difference between the two vectors, i.e. l P
Axi yi i¼1 s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi simðAx; yÞ ¼ ¼ simðx; yÞ. l l P 2P 2 ðAxi Þ yi i¼1
i¼1
Therefore, when the algorithm is used for time series monitoring, the variation range of data need not be predetermined like those algorithms proposed in current documents [14–17]. This would be a great benefit for cases such as online signal monitoring because the entire elimination of the amplitude shift is somewhat unpractical. Of course, since simðx þ c; yÞasimðx; yÞ; the data
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1467
average value needs to be subtracted as described in step 1. Fortunately, this is a classical modular practically utilised in almost all signal monitoring systems. Such a preprocessing step would not affect the practicability of our algorithm.
3. Calculation results and analysis Using the algorithm illustrated above, one simulated time series datum of 100 points is generated for calculation. As shown in Fig. 4, the normal discrete sine signal is p i . (4) xðiÞ ¼ sin 10 Within points 65–85, the signal is anomalous as p 1 þ randðiÞ sin i . (5) 2 10 Where rand(i) is a pseudo-random number uniformly distributed within interval [0.0,1.0]. We used the first 60 points (three periods) as the self-data (normal) to establish self-set S. The whole data series are used as monitored data for calculation. The window length is l ¼ 5: Sliding the window along the time series takes place in four different ways as follows: xðiÞ ¼
(a) Sliding the window in a non-overlapping way, i.e. sliding steps are both equal to l for self-data (step_s ¼ l) as well as monitored data (step_m ¼ l). (b) Sliding step for self-data is step_s ¼ l (non-overlapped), and sliding step for monitored data is step_m ¼ 1 (4 points overlapped). (c) Sliding step for self-data is step_s ¼ 1 (4 points overlapped), and sliding step for monitored data is step_m ¼ l (non-overlapped). (d) Sliding step for self-data is step_s ¼ 1 (4 points overlapped), and sliding step for monitored data is step_m ¼ 1 (4 points overlapped). *
*
Establish the corresponding self-sets S and monitored sets M. Let x s and x m denote the vectors *i in S and M, respectively, and calculate the minimum distance dist_m(i) between each vector x m in 2
x(i)
1 0 -1 -2
0
20
40 60 sample points
80
100
Fig. 4. Simulated time series data: anomalous within 65–85 points.
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1468
M and all the vectors in S *
*i
*
*i
dðx s ; x m Þ ¼ min ð1 simðx s ; x m ÞÞ. dist_mðiÞ ¼ min * * x s 2S
(6)
x s 2S
The calculation results are shown in Fig. 5a–d. It can be seen that the results of Fig. 5a, c and d are just the same as what we expected. dist_m(i) is zero with the first 60 points, indicating *i that there at least exists one vector in S which is the same as each x m constructed from the first 60 sample points. However, from the result of Fig. 5b, we would view most normal data as anomalous because most of the dist_m(i)’s are more than zero. The reason for such a ‘‘false positive’’ result is that, when non-overlapped windows are applied, the vector in the self-set is not enough to cover all the possible normal data segments. On the other hand, the reason for the normal result of Fig. 5a is that the beginning points of the two sets are the same. If we apply the window from the second point in the monitored data, a ‘‘false positive’’ result will also be obtained (data not shown in this paper). In view of the fact that the beginning sample point of a time series cannot be precisely controlled in practical application, the overlapped sliding window is preferred even though such a scheme would greatly increase the computational cost. Using the self-set constructed in Fig. 5d to establish one antibody set of 20 detectors by the negative selection algorithm shown in Fig. 2 (since the self-set is constructed from pure sine wave, ls is zero, we set the matching threshold for self-set as rs ¼ 0:01), the monitored set in Fig. 5d is detected. The novel points that matched with the detectors are marked by triangle symbols in Fig. 5d. Note that no normal data points are marked as novel, particularly for the normal points after the 60th point; the false positives are successfully avoided. Considering that the practical signals are often corrupted by noise, another simulated time series data of 200 points is calculated to confirm the practicability of our approach. As shown in
0.2
dist_m(i)
dist_m(i)
0.3
0.1 0 -0.1
0
20
80
100
0
20
60 40 sample points
80
100
0
20
40 60 sample points
80
100
(b)
0.3
0.3
0.2
0.2
0.1 0 -0.1
(c)
60 40 sample points
dist_m(i)
dist_m(i)
(a)
1.2 1 0.8 0.6 0.4 0.2 0 -0.2
0.1 0 -0.1
0
20
40 60 sample points
80
100
(d)
Fig. 5. Calculated distances for overlapped and non-overlapped windows.
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1469
2 0.001 Dist_s(i)
x(i)
1 0 -1 -2 0
50
Dist_m(i)
0.3 0.2 0.1 0 -0.1
0
200
0
(c)
100 sample points
0
50 sample points
0
100 sample points
(b)
Dist_D(i)
(a)
100 150 sample points
0.0005
200
(d)
100
0.3 0 -0.3
200
Fig. 6. Results of applying the algorithm to one simulated time series.
Fig. 6a, the first 100 points are a noisy sine wave p randðiÞ sin i , xðiÞ ¼ 1 þ 10 10
(7)
where rand(i) is a pseudo-random number as defined in Eq. (5). The latter 100 points are a noisy sawtooth wave p randðiÞ sawtooth i , (8) xðiÞ ¼ 1 þ 10 10 where the function sawtoothððp=10ÞiÞ generates a sawtooth wave with peaks of 1 to +1 instead of the sine wave in Eq. (7). The first 100 points are treated as normal time series and the latter 100 points are considered as abnormal. Just as we mentioned above, setting l ¼ 5 and step_s ¼ 1, the self-set S is established by the first 100 points and correspondingly the monitored set M is established by the whole 200 points data series. In such a way, each sampled point in the data series actually corresponds to one 5-dimensional vector. For each vector ~ xis 2 S; the corresponding Dist_sðiÞ ¼ lis is calculated as shown in Fig. 6b. It can be seen that the maximum value of the lis ’s is less than 0.001. Thus, we set the matching threshold rs for the self-set S as 0.001. As a comparison, the minimum distance dist_m(i) between each vector in M and those vectors in S is also calculated by Eq. (6) and the result is shown in Fig. 6c. It can be seen that the distance values increase significantly for the latter 100 points. If we set one distance value rp as the matching threshold, say, rp ¼ rs ; e.g. most of the latter 100 points can be considered as abnormal. *i One antibody set D of 10 detectors is established with rs ¼ 0:001: The affinity between the x m in M and the detectors within D is evaluated by *i
*j
Dist_DðiÞ ¼ min ½d ðx m ; x d Þ rjd . j *
x d 2D
*i
In such a way, for each x m ; if the Dist_DðiÞo0; the corresponding data segment is novel or anomalous. The calculated result is shown in Fig. 6d. For the first 100 points, no Dist_DðiÞ is
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1470
found to be negative, indicating that rs ¼ 0:001 is large enough to avoid the ‘‘false positive’’ detection. For the latter 100 points, negative values appear and the corresponding data segment can be classified as abnormal. Compared with the result in Fig. 6c, only several of the latter 100 points can be regarded as abnormal from Fig. 6d. This is because of the coverage of the antibody set. Since the detectors in D are randomly generated, it can only cover one part of the abnormal cases. On the other hand, less computations need to be performed because there are only 10 detectors in D. Increasing the number of detectors in D would surely benefit the novel detection, but a larger dataset needs to be maintained and more computations need to be performed correspondingly. As a practical application, a vibration signal series sampled from a rigidity rotor system (only one-channel signal is utilised) is detected. Fig. 7a is the signal for normal state and the signal in Fig. 7b is for the case that rubbing faulty occurred (average values are subtracted). We use the normal signal as self-series xs(i), the rubbing-signal as the series xm(i) to be monitored. Applying the overlapped window of l ¼ 5 and step_s ¼ 1 to xs(i) and xm(i), the self-set S and monitored set M are constructed, respectively. First, the dist_sðiÞ ¼ lis ’s are calculated and shown in Fig. 7c. The maximum value is 0.05. Then the minimum distances between each vector in M and all the vectors in S are calculated as dist_m(i) shown in Fig. 7d. The average of dist_m(i) is 0.04, indicating that most of the data segments in the monitored series perform normally compared with the normal series. Setting the matching threshold for self-set as rs ¼ 0.05 to ensure no false positive cases would be encountered, one antibody set of 30 detectors is established by the above-mentioned negative selection algorithm. When the monitored set M is detected by the antibody set, 58 data segments are reported novel as marked by triangle symbols in Fig. 7d. Note that even though the reported novelty number is quite less than that of dist_mðiÞ4rs ¼ 0:05; most data segments whose dist_m(i) are greatly more than rs ¼ 0:05 (indicating that the segments are significantly anomalous) are reported. In view of the fact that only danger alarms need to be generated for online signal monitoring, such a result could be accepted for most cases. 0.3
0.4
0.2 0.1 x (i) x(m i )
x (i) x(s i )
0.2 0
0 -0.1
-0.2
-0.2
-0.4 0
200
-0.3
1000
0
200
(b)
0.06 0.05 0.04 0.03 0.02 0.01 0 -0.01
400 600 Sample Points
800
1000
0.3 0.2 0.1 0 -0.1 0
(c)
800
dist_m(i)
dist_s(i)
(a)
400 600 Sample Points
200
400
600
Sample Points
800
1000
0
(d)
200
400
600
Sample Points
Fig. 7. Calculation results for the vibration signal of a rotor system.
800
1000
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1471
4. Conclusion In this paper, we proposed and demonstrated a real-valued negative selection algorithm for novelty detection in time series. Our attention is mainly focused on developing one efficient immunology-inspired algorithm for online system monitoring. Compared with the binary-valued negative selection algorithms, our approach accomplishes the calculation in real domain. This will benefit the subsequent time series processing, such as system faulty diagnosis, because the realvalued representation is much closer to the original problem. Moreover, the main advantage of our algorithm might be the utilisation of cosine similarity. It makes the algorithm insensitive to the absolute amplitude of the time series. Unlike other approaches like Euclidean distance-based algorithm, where the raw data need to be normalised first, only the average value need to be subtracted as preprocessing. Such a character is especially practical for online monitoring. The coverage of the antibody set might be the main problem to be overcome. The simplest way is to increase the number of detectors. However, such a method would increase the computational cost and therefore decrease the real-time property. Another way is to make each detector more efficient. Fortunately, some of these approaches had been reported in documents like [8,11]. Another practical problem is the determination of the matching threshold rs and the window length. A larger rs and a longer window may make the algorithm relatively robust to noise but may result in an insensitive antibody set for novelty detection. This is especially difficult when a low signal-to-noise ratio signal is monitored. We believe that the classical data processing techniques for online monitoring should be an inspiration source for a more efficient algorithm.
References [1] R. Kozma, M. Sakuma, Y. Yokoyama, et al., On the accuracy of mapping by neural networks trained by backpropagation with forgetting, Neurocomputing 13 (1996) 295–311. [2] K. Worden, Structural fault detection using a novelty measure, Journal of Sound and Vibration 201 (1) (1997) 85–101. [3] H. Sohn, C.R. Farrar, Damage diagnosis using time series analysis of vibration signals, Smart Materials and Structures 10 (2001) 446–451. [4] M. Markou, S. Singh, Novelty detection: a review—part 1: statistical approaches, Signal Processing 83 (2003) 2481–2497. [5] M. Markou, S. Singh, Novelty detection: a review—part 2: neural network based approaches, Signal Processing 83 (2003) 2499–2521. [6] L. Tarassenko, A. Nairac, N. Townsend, et al., Novelty detection for the identification of abnormalities, International Journal of Systems Science 31 (11) (2000) 1427–1439. [7] S.E. Guttormsson, R.J. Marks II, M.A. El-sharkawi, Elliptical novelty grouping for on-line short-turn detection of excited running rotors, IEEE Transactions on Energy Conversion 14 (1) (1999) 16–22. [8] J. Manuel, G. Illa, Nearest-neighbours for time series, Applied Intelligence 20 (2004) 21–35. [9] D. Dasgupta, S. Forrest, Novelty detection in time series data using ideas from immunology, Proceedings of the Fifth International Conference on Intelligent Systems, 1996. [10] S. Forrest, A.S. Perelson, L. Allen, Self-nonself discrimination in a computer, Proceedings of the IEEE Symposium on Research in Security and Privacy, Oakland, CA, 1994, pp. 202–212. [11] D. Dasgupta, F. Gonzalez, An immunity-based technique to characterize intrusions in computer networks, IEEE Transactions on Evolutionary Computation 6 (2002) 281–291.
ARTICLE IN PRESS 1472
Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
[12] P. D’Haeseleer, S. Forrest, P. Helman, Immunological approach to change detection: algorithms, analysis and implications, Proceedings of the IEEE Computer Society Symposium on Research in Security and Privacy, Oakland, CA, 1996, pp. 110–119. [13] D. Dasgupta, S. Forrest, Artificial immune systems in industrial applications, Proceedings of the Second International Conference on Intelligent Processing and Manufacturing Material (IPMM), Honolulu, 1999. [14] D. Dasgupta, F. Nino, Comparison of negative and positive selection algorithms in novel pattern detection, Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, vol. 1, Nashville, TN, 2000, pp. 125–130. [15] F. Gonzalez, D. Dasgupta, R. Kozma, Combining negative selection and classification techniques for anomaly detection, Proceedings of the Congress on Evolutionary Computation, Hawaii, 2002, pp. 705–710. [16] F. Nino, O. Beltran, A change detection software agent based on immune mixed selection, Proceedings of the Congress on Evolutionary Computation, Hawaii, 2002, pp. 693–698. [17] F. Gonzalez, D. Dasgupta, Neuro-immune and self-organizing map approaches to anomaly detection: a comparison, Proceedings of the First International Conference on Artificial Immune Systems, Canterbury, UK, 2002, pp. 203–211.
Mechanical Systems and Signal Processing Mechanical Systems and Signal Processing 20 (2006) 1461–1472 www.elsevier.com/locate/jnlabr/ymssp
A cosine similarity-based negative selection algorithm for time series novelty detection Yonggui Dong, Zhaoyan Sun, Huibo Jia State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instruments and Mechanology, Tsinghua University, Room 3514, Beijing 100084, P.R. China Received 10 November 2003; received in revised form 7 December 2004; accepted 29 December 2004 Available online 5 March 2005
Abstract Detecting the new or anomalous signal sequences in the observed time series data is a problem of great practical interest for many applications. The bio-inspired negative selection algorithm, whose main idea is to discriminate the non-self pattern from self pattern, has drawn much attention because only normal information is needed for training. Most of the proposed algorithms are based on binary-valued string matching. A real-valued negative selection algorithm for novelty detection in vibration signal is implemented in this paper. The vector set for calculation is constructed by sampling the discrete time series from a moving time window. The matching affinity between two vectors is measured by cosine similarity. The calculated results show that the cosine similarity-based algorithm is more practical for potential applications in online signal monitoring. r 2005 Elsevier Ltd. All rights reserved.
1. Introduction In most industrial applications, the behaviour of a system is often characterised by observing and analysing a series of parameters over time. Detecting the anomalous signal sequences in the observed time series data is an important task for system monitoring and diagnosis. A common approach is to design a classifier, for which one or more of the classes are allocated to the Corresponding author.
E-mail address: [email protected] (Y. Dong). 0888-3270/$ - see front matter r 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.ymssp.2004.12.006
ARTICLE IN PRESS 1462
Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
abnormal patterns corresponding to those faults to be identified. Most existing methods require prior knowledge about various novelty conditions [1]. Given the fact that one can never train a monitoring system on all possible cases whose data the monitored system is likely to encounter, novelty detection, whose object is to differentiate between known and unknown object information during monitoring, has become one important but extremely challenging task in practice. Novelty detection is an approach to the problem of fault detection, which only requires data from the normal class. It is commonly used to implement a damage detection strategy for structural health monitoring in aerospace, civil and mechanical engineering infrastructure [2,3]. There exist several models of novelty detection that have been shown to perform well on different data. Statistical pattern recognition techniques, including neural networks, are usually applied to the problems [4,5]. The neural network-based methods have been widely used for mechanical systems such as jet engines [6]. The k-nearest neighbour algorithm is another approach for novelty detection. One set for normal signal patterns is established. If the monitored signal pattern is at a greater distance from each element of the normal set, a novelty is declared. This method has been used for rotor fault detection [7], as well as industrial process monitoring [8]. The main problem with this technique is that usually large-sized datasets need to be maintained and correspondingly, a large number of computations have to be performed. Over the last few years, a bio-inspired negative selection algorithm has been proposed for novelty detection in time series data [9]. The main idea of the algorithm is based on ideas from the natural immune system. The natural immune system detects the foreign antigens in a way known as self–non-self discrimination. Inspired from such a mechanism, Forrest proposed an artificial immune system for computer network security [10]. It is a simplification of the complex chemistry of antibody/antigen recognition in natural immune systems. The parameters to be monitored are encoded to binary strings with definite length. The self–non-self discrimination is implemented by applying an r-contiguous-bits matching rule to the binary strings [11,12]. Dasgupta forwards the binary-based algorithm to industrial applications, especially tool breakage detection [13,14]. Unlike the case of binary-based computer networks, the binary coding performs a very low level representation for signal monitoring. Therefore, the real-valued negative selection algorithms are preferred and proposed [15–17]. Euclidean distance is usually used for discrimination calculation. Our approach is to implement a more effective real-valued algorithm for monitoring time series data. The vector set for calculation is constructed by sampling the discrete vibration signal from a moving time window. Cosine similarity is used for calculating the affinity between two vectors. The calculated results indicate that the cosine similarity-based algorithm is more practical for potential applications in online signal monitoring.
2. Real-valued negative selection algorithm Natural immune systems are able to detect the infection of any foreign cell or molecule (antigen). The detection is accomplished only when the invading antigen is ‘‘recognised’’ by a specific antibody or T-cell receptor. Therefore, the diversity of the T-cell receptors must be large enough to cover all possible antigens that the immune system may encounter. Moreover, the
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1463
binding between the receptors and the body’s own (self) cells must be avoided. The immune system implements such an immune detection in a way known as self–non-self discrimination. The T-cell receptors are created by a random genetic rearrangement process. Those cells that successfully bind with self-cells are destroyed in the thymus. Only those cells that fail to bind to self-cells are allowed to leave the thymus and become antibodies of the body’s immune system. Such an antibody-generation-selection process is called negative selection. The main purpose of novelty detection is to distinguish self, which corresponds to the normal states of the monitored system, from non-self, which corresponds to the novel or anomalous states. Similar to the negative selection process illustrated above, two sets are maintained for the artificial immune system, the self-set S for normal data series and the antibody set D for novelty detectors. The following definitions and parameters are utilised in this paper.
x(i) is utilised to denote the discrete time series sampled from the continuous signals, where i
~ x1 ¼ ðx1 ; x2 ; x3 ; x4 ; x5 Þ, ~ x2 ¼ ðx6 ; x7 ; x8 ; x9 ; x10 Þ, ~ x3 ¼ ðx11 ; x12 ; x13 ; x14 ; x15 Þ, .. .
l
x(i)
denotes sampling point number (time). Especially, xs(i) is used to denote the time series that are considered as normal for the system. l-dimensional vectors set UDRl is defined as the feature collection that to be monitored. The ldimensional vectors in U can be obtained from the time series x(i). As shown in Fig. 1, slide one window with length l along time (sampling points) axis, the x(i) is separated into equal-sized data segments. The segmenting window can be overlapped, or non-overlapped. For example, xðiÞ ¼ fx1 ; x2 ; . . . ; xn g; let the window length l ¼ 5; in non-overlapped cases, the l-dimensional vectors can be constructed as
Sampling points(i)
Fig. 1. Separate the time series into equal-sized data segments by a sliding window.
ARTICLE IN PRESS 1464
Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
Correspondingly, for overlapped cases, say, l ¼ 5 and the step of the sliding window step_s ¼ 1; the l-dimensional vectors are ~ x1 ¼ ðx1 ; x2 ; x3 ; x4 ; x5 Þ, ~ x2 ¼ ðx2 ; x3 ; x4 ; x5 ; x6 Þ, ~ x3 ¼ ðx3 ; x4 ; x5 ; x6 ; x7 Þ, .. .
By the method mentioned above, the l-dimensional vectors can be constructed from the time series corresponding to both the normal and the anomalous system states. The overall vectors collection is denoted as U. The vector constructed from the normal time series data is ~ xis 2 U: On the other hand, the i vector constructed from the monitored time series data is ~ xm 2 U: The vector in antibody set D is ~ xid 2 U: The subset for normal vectors is self-set SDU, and the subset for novelty data segments is non-self-set NDU. Since U is S the collection T of all the possible vectors, both for normal and anomalous cases, there exist S N ¼ U, S N ¼ F, where F denotes a blank set that contains no vectors.
It can be seen that the antibody set D is actually the subset of N, i.e. DDN. Let dð~ x; ~ yÞ represent l the distance function over space R , the negative selection algorithm for time series novelty detection can be implemented as follows. 2.1. Self-set S construction Assume that we have collected enough time series data xraw(i) from a normally operating system. Subtract the average value of xraw(i) to eliminate the influence of the offset value and get an xs(i) (the reason for the arrangement of such a preprocessing step will be described later). Then apply one sliding window with length l to xs(i); this results in the construction of self-set S. 2.2. Matching threshold for the self-set For each vector ~ xis 2 S; calculate the minimum distance between itself and other vectors within S to get lis ¼ j min dð~ xis ; ~ xjs Þ,
(1)
~ xs 2S;jai
* *
*
where function dðx ; y Þ denotes the distance between the two l-dimensional vectors ~ x and y : Then the maximum value of all lis ’s is obtained by ls ¼ maxðlis Þ: After that, determine a matching threshold rs for the self-set S according to ls. The threshold rs must be greater than ls in order that all vectors constructed from the normal time series data can be classified into the self-set. A reasonably greater rs would benefit in avoiding the ‘‘false positive’’ detection, in which case the normal data segment is classified as abnormal. On the other hand, the threshold rs should not be so great that the detection system becomes insensitive to real abnormal data. If so, the ‘‘false negative’’ detection would happen if the abnormal data are classified as normal.
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1465
2.3. Antibody set D generation Utilising a uniformly distributed pseudo-random generator, generate the candidate detector x 2 ½1:0; 1:0l : Establish the antibodyc set D with the negative selection algorithm shown in * Fig. 2. First, the candidate detector x is matched with all the vectors in the self-set S by xis Þ: If the minimum value of all the distances is less than the calculating the distancesc dð~ xc ; ~ * threshold ls, cthen the x is considered as representing a normal data segment and cancelled. If * not, i.e. the x represents an abnormal data segment, it is matched with the vectors in antibody set D to determine whether it has been covered by any of the vectors in D. Certainly, if this is the first candidate detector for D, it will become one new detector without a matching calculation. Unlike the algorithm proposed by Forrest [10] that all antibody elements have the same *i matching threshold, each detector x d in D has its own matching threshold rid in our approach, as xid Þ is less than the corresponding rid ; it can be determined shown in Fig. 3. Therefore, if any dð~ xc ; ~ *c *i *c that the x has been covered by the detector x d and should be cancelled. Otherwise, x becomes a c *i * new detector in D and the corresponding threshold is given as rcd ¼ min dðx ; x s Þ rs : S In such a way, any vector in the antibody set D has two features: (a). The vector itself is not in *i *j the area centered by any x s 2 S with the radius rs : (b). For any vector x ; if there exists one i j i * * * xj must be a non-self-vector and the corresponding data x d 2 D that fulfils dðx ; x d Þorid ; then ~ segment can be classified as abnormal. *c
2.4. Time series data novelty detection When monitoring the time series data,i preprocess the data in the same way as described in step * *i 1 to obtain the corresponding vector x m : Calculate the distances between x m and each vector
Random vector generator
Self-set S
ϖc
rs
x
min d S
ϖc ϖi (x , xs)
< rs ? Y
N
Antibody-set D
cancel
ϖ ϖ
min(d (xc, xid) - rdi ) < 0 ? D
Y
N new detector .. £ add
into
rdc =
cancel
antibody-set£ © ϖ ϖ
min d (xc, xsi ) - rs
Fig. 2. Antibody set generation.
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1466
x si
rs xdi
rdi
Fig. 3. Matching threshold of detector in antibody set D.
*j
*i
*j
within D. If there exists one x d that fulfils d ðx m ; x d Þorjd ; then the data segment corresponding to *i x m is novel or anomalous. It can be seen that the performance of the algorithm described above would be greatly * * * * influenced by the distance measure dðx ; y Þ of vectors x ¼ ðx1; x2 ; . . . ; xl Þ and y ¼ ðy1; y2 ; . . . ; yl Þ: Considering that the time series novelty usually performs as the corresponding data segment is ‘‘not similar to’’ the normal data segments, we choose the cosine similarity as the affinity function between two l-dimensional vectors. l P
xi yi simðx ; y Þ ¼ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi . l l P P x2i y2i * *
i¼1
i¼1
(2)
i¼1
This function gives similarity measure in the sense that the cosine value gets larger as the two vectors become more nearly parallel in the l-dimensional space. Or in other words, as the two data segments become more similar, their cosine similarity approaches 1.0 and their distance approaches 0.0. Therefore, the cosine similarity can be used as a distance metric in the following way: * *
* *
dðx ; y Þ ¼ 1 sim ðx ; y Þ.
(3)
Since the cosine similarity reflects the magnitude of the angle between two vectors in the ldimensional space, it is a many-to-one function. Compared with the other distance measures, like Euclidean distance, the cosine similarity ignores the magnitude difference between the two vectors, i.e. l P
Axi yi i¼1 s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi simðAx; yÞ ¼ ¼ simðx; yÞ. l l P 2P 2 ðAxi Þ yi i¼1
i¼1
Therefore, when the algorithm is used for time series monitoring, the variation range of data need not be predetermined like those algorithms proposed in current documents [14–17]. This would be a great benefit for cases such as online signal monitoring because the entire elimination of the amplitude shift is somewhat unpractical. Of course, since simðx þ c; yÞasimðx; yÞ; the data
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1467
average value needs to be subtracted as described in step 1. Fortunately, this is a classical modular practically utilised in almost all signal monitoring systems. Such a preprocessing step would not affect the practicability of our algorithm.
3. Calculation results and analysis Using the algorithm illustrated above, one simulated time series datum of 100 points is generated for calculation. As shown in Fig. 4, the normal discrete sine signal is p i . (4) xðiÞ ¼ sin 10 Within points 65–85, the signal is anomalous as p 1 þ randðiÞ sin i . (5) 2 10 Where rand(i) is a pseudo-random number uniformly distributed within interval [0.0,1.0]. We used the first 60 points (three periods) as the self-data (normal) to establish self-set S. The whole data series are used as monitored data for calculation. The window length is l ¼ 5: Sliding the window along the time series takes place in four different ways as follows: xðiÞ ¼
(a) Sliding the window in a non-overlapping way, i.e. sliding steps are both equal to l for self-data (step_s ¼ l) as well as monitored data (step_m ¼ l). (b) Sliding step for self-data is step_s ¼ l (non-overlapped), and sliding step for monitored data is step_m ¼ 1 (4 points overlapped). (c) Sliding step for self-data is step_s ¼ 1 (4 points overlapped), and sliding step for monitored data is step_m ¼ l (non-overlapped). (d) Sliding step for self-data is step_s ¼ 1 (4 points overlapped), and sliding step for monitored data is step_m ¼ 1 (4 points overlapped). *
*
Establish the corresponding self-sets S and monitored sets M. Let x s and x m denote the vectors *i in S and M, respectively, and calculate the minimum distance dist_m(i) between each vector x m in 2
x(i)
1 0 -1 -2
0
20
40 60 sample points
80
100
Fig. 4. Simulated time series data: anomalous within 65–85 points.
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1468
M and all the vectors in S *
*i
*
*i
dðx s ; x m Þ ¼ min ð1 simðx s ; x m ÞÞ. dist_mðiÞ ¼ min * * x s 2S
(6)
x s 2S
The calculation results are shown in Fig. 5a–d. It can be seen that the results of Fig. 5a, c and d are just the same as what we expected. dist_m(i) is zero with the first 60 points, indicating *i that there at least exists one vector in S which is the same as each x m constructed from the first 60 sample points. However, from the result of Fig. 5b, we would view most normal data as anomalous because most of the dist_m(i)’s are more than zero. The reason for such a ‘‘false positive’’ result is that, when non-overlapped windows are applied, the vector in the self-set is not enough to cover all the possible normal data segments. On the other hand, the reason for the normal result of Fig. 5a is that the beginning points of the two sets are the same. If we apply the window from the second point in the monitored data, a ‘‘false positive’’ result will also be obtained (data not shown in this paper). In view of the fact that the beginning sample point of a time series cannot be precisely controlled in practical application, the overlapped sliding window is preferred even though such a scheme would greatly increase the computational cost. Using the self-set constructed in Fig. 5d to establish one antibody set of 20 detectors by the negative selection algorithm shown in Fig. 2 (since the self-set is constructed from pure sine wave, ls is zero, we set the matching threshold for self-set as rs ¼ 0:01), the monitored set in Fig. 5d is detected. The novel points that matched with the detectors are marked by triangle symbols in Fig. 5d. Note that no normal data points are marked as novel, particularly for the normal points after the 60th point; the false positives are successfully avoided. Considering that the practical signals are often corrupted by noise, another simulated time series data of 200 points is calculated to confirm the practicability of our approach. As shown in
0.2
dist_m(i)
dist_m(i)
0.3
0.1 0 -0.1
0
20
80
100
0
20
60 40 sample points
80
100
0
20
40 60 sample points
80
100
(b)
0.3
0.3
0.2
0.2
0.1 0 -0.1
(c)
60 40 sample points
dist_m(i)
dist_m(i)
(a)
1.2 1 0.8 0.6 0.4 0.2 0 -0.2
0.1 0 -0.1
0
20
40 60 sample points
80
100
(d)
Fig. 5. Calculated distances for overlapped and non-overlapped windows.
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1469
2 0.001 Dist_s(i)
x(i)
1 0 -1 -2 0
50
Dist_m(i)
0.3 0.2 0.1 0 -0.1
0
200
0
(c)
100 sample points
0
50 sample points
0
100 sample points
(b)
Dist_D(i)
(a)
100 150 sample points
0.0005
200
(d)
100
0.3 0 -0.3
200
Fig. 6. Results of applying the algorithm to one simulated time series.
Fig. 6a, the first 100 points are a noisy sine wave p randðiÞ sin i , xðiÞ ¼ 1 þ 10 10
(7)
where rand(i) is a pseudo-random number as defined in Eq. (5). The latter 100 points are a noisy sawtooth wave p randðiÞ sawtooth i , (8) xðiÞ ¼ 1 þ 10 10 where the function sawtoothððp=10ÞiÞ generates a sawtooth wave with peaks of 1 to +1 instead of the sine wave in Eq. (7). The first 100 points are treated as normal time series and the latter 100 points are considered as abnormal. Just as we mentioned above, setting l ¼ 5 and step_s ¼ 1, the self-set S is established by the first 100 points and correspondingly the monitored set M is established by the whole 200 points data series. In such a way, each sampled point in the data series actually corresponds to one 5-dimensional vector. For each vector ~ xis 2 S; the corresponding Dist_sðiÞ ¼ lis is calculated as shown in Fig. 6b. It can be seen that the maximum value of the lis ’s is less than 0.001. Thus, we set the matching threshold rs for the self-set S as 0.001. As a comparison, the minimum distance dist_m(i) between each vector in M and those vectors in S is also calculated by Eq. (6) and the result is shown in Fig. 6c. It can be seen that the distance values increase significantly for the latter 100 points. If we set one distance value rp as the matching threshold, say, rp ¼ rs ; e.g. most of the latter 100 points can be considered as abnormal. *i One antibody set D of 10 detectors is established with rs ¼ 0:001: The affinity between the x m in M and the detectors within D is evaluated by *i
*j
Dist_DðiÞ ¼ min ½d ðx m ; x d Þ rjd . j *
x d 2D
*i
In such a way, for each x m ; if the Dist_DðiÞo0; the corresponding data segment is novel or anomalous. The calculated result is shown in Fig. 6d. For the first 100 points, no Dist_DðiÞ is
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1470
found to be negative, indicating that rs ¼ 0:001 is large enough to avoid the ‘‘false positive’’ detection. For the latter 100 points, negative values appear and the corresponding data segment can be classified as abnormal. Compared with the result in Fig. 6c, only several of the latter 100 points can be regarded as abnormal from Fig. 6d. This is because of the coverage of the antibody set. Since the detectors in D are randomly generated, it can only cover one part of the abnormal cases. On the other hand, less computations need to be performed because there are only 10 detectors in D. Increasing the number of detectors in D would surely benefit the novel detection, but a larger dataset needs to be maintained and more computations need to be performed correspondingly. As a practical application, a vibration signal series sampled from a rigidity rotor system (only one-channel signal is utilised) is detected. Fig. 7a is the signal for normal state and the signal in Fig. 7b is for the case that rubbing faulty occurred (average values are subtracted). We use the normal signal as self-series xs(i), the rubbing-signal as the series xm(i) to be monitored. Applying the overlapped window of l ¼ 5 and step_s ¼ 1 to xs(i) and xm(i), the self-set S and monitored set M are constructed, respectively. First, the dist_sðiÞ ¼ lis ’s are calculated and shown in Fig. 7c. The maximum value is 0.05. Then the minimum distances between each vector in M and all the vectors in S are calculated as dist_m(i) shown in Fig. 7d. The average of dist_m(i) is 0.04, indicating that most of the data segments in the monitored series perform normally compared with the normal series. Setting the matching threshold for self-set as rs ¼ 0.05 to ensure no false positive cases would be encountered, one antibody set of 30 detectors is established by the above-mentioned negative selection algorithm. When the monitored set M is detected by the antibody set, 58 data segments are reported novel as marked by triangle symbols in Fig. 7d. Note that even though the reported novelty number is quite less than that of dist_mðiÞ4rs ¼ 0:05; most data segments whose dist_m(i) are greatly more than rs ¼ 0:05 (indicating that the segments are significantly anomalous) are reported. In view of the fact that only danger alarms need to be generated for online signal monitoring, such a result could be accepted for most cases. 0.3
0.4
0.2 0.1 x (i) x(m i )
x (i) x(s i )
0.2 0
0 -0.1
-0.2
-0.2
-0.4 0
200
-0.3
1000
0
200
(b)
0.06 0.05 0.04 0.03 0.02 0.01 0 -0.01
400 600 Sample Points
800
1000
0.3 0.2 0.1 0 -0.1 0
(c)
800
dist_m(i)
dist_s(i)
(a)
400 600 Sample Points
200
400
600
Sample Points
800
1000
0
(d)
200
400
600
Sample Points
Fig. 7. Calculation results for the vibration signal of a rotor system.
800
1000
ARTICLE IN PRESS Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
1471
4. Conclusion In this paper, we proposed and demonstrated a real-valued negative selection algorithm for novelty detection in time series. Our attention is mainly focused on developing one efficient immunology-inspired algorithm for online system monitoring. Compared with the binary-valued negative selection algorithms, our approach accomplishes the calculation in real domain. This will benefit the subsequent time series processing, such as system faulty diagnosis, because the realvalued representation is much closer to the original problem. Moreover, the main advantage of our algorithm might be the utilisation of cosine similarity. It makes the algorithm insensitive to the absolute amplitude of the time series. Unlike other approaches like Euclidean distance-based algorithm, where the raw data need to be normalised first, only the average value need to be subtracted as preprocessing. Such a character is especially practical for online monitoring. The coverage of the antibody set might be the main problem to be overcome. The simplest way is to increase the number of detectors. However, such a method would increase the computational cost and therefore decrease the real-time property. Another way is to make each detector more efficient. Fortunately, some of these approaches had been reported in documents like [8,11]. Another practical problem is the determination of the matching threshold rs and the window length. A larger rs and a longer window may make the algorithm relatively robust to noise but may result in an insensitive antibody set for novelty detection. This is especially difficult when a low signal-to-noise ratio signal is monitored. We believe that the classical data processing techniques for online monitoring should be an inspiration source for a more efficient algorithm.
References [1] R. Kozma, M. Sakuma, Y. Yokoyama, et al., On the accuracy of mapping by neural networks trained by backpropagation with forgetting, Neurocomputing 13 (1996) 295–311. [2] K. Worden, Structural fault detection using a novelty measure, Journal of Sound and Vibration 201 (1) (1997) 85–101. [3] H. Sohn, C.R. Farrar, Damage diagnosis using time series analysis of vibration signals, Smart Materials and Structures 10 (2001) 446–451. [4] M. Markou, S. Singh, Novelty detection: a review—part 1: statistical approaches, Signal Processing 83 (2003) 2481–2497. [5] M. Markou, S. Singh, Novelty detection: a review—part 2: neural network based approaches, Signal Processing 83 (2003) 2499–2521. [6] L. Tarassenko, A. Nairac, N. Townsend, et al., Novelty detection for the identification of abnormalities, International Journal of Systems Science 31 (11) (2000) 1427–1439. [7] S.E. Guttormsson, R.J. Marks II, M.A. El-sharkawi, Elliptical novelty grouping for on-line short-turn detection of excited running rotors, IEEE Transactions on Energy Conversion 14 (1) (1999) 16–22. [8] J. Manuel, G. Illa, Nearest-neighbours for time series, Applied Intelligence 20 (2004) 21–35. [9] D. Dasgupta, S. Forrest, Novelty detection in time series data using ideas from immunology, Proceedings of the Fifth International Conference on Intelligent Systems, 1996. [10] S. Forrest, A.S. Perelson, L. Allen, Self-nonself discrimination in a computer, Proceedings of the IEEE Symposium on Research in Security and Privacy, Oakland, CA, 1994, pp. 202–212. [11] D. Dasgupta, F. Gonzalez, An immunity-based technique to characterize intrusions in computer networks, IEEE Transactions on Evolutionary Computation 6 (2002) 281–291.
ARTICLE IN PRESS 1472
Y. Dong et al. / Mechanical Systems and Signal Processing 20 (2006) 1461–1472
[12] P. D’Haeseleer, S. Forrest, P. Helman, Immunological approach to change detection: algorithms, analysis and implications, Proceedings of the IEEE Computer Society Symposium on Research in Security and Privacy, Oakland, CA, 1996, pp. 110–119. [13] D. Dasgupta, S. Forrest, Artificial immune systems in industrial applications, Proceedings of the Second International Conference on Intelligent Processing and Manufacturing Material (IPMM), Honolulu, 1999. [14] D. Dasgupta, F. Nino, Comparison of negative and positive selection algorithms in novel pattern detection, Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, vol. 1, Nashville, TN, 2000, pp. 125–130. [15] F. Gonzalez, D. Dasgupta, R. Kozma, Combining negative selection and classification techniques for anomaly detection, Proceedings of the Congress on Evolutionary Computation, Hawaii, 2002, pp. 705–710. [16] F. Nino, O. Beltran, A change detection software agent based on immune mixed selection, Proceedings of the Congress on Evolutionary Computation, Hawaii, 2002, pp. 693–698. [17] F. Gonzalez, D. Dasgupta, Neuro-immune and self-organizing map approaches to anomaly detection: a comparison, Proceedings of the First International Conference on Artificial Immune Systems, Canterbury, UK, 2002, pp. 203–211.