- Email: [email protected]
Sequential Detection of Changes in Stochastic Processes
Copyright © IFAC Identification and System Parameter Estimation. Budapest. Hungary 1991
SEQUENTIAL DETECTION OF CHANGES IN STOCHASTIC PROCESSES I. V. Nikiforov Institute o/Control Sciences, Pro/soyuznaya St., 65, Moscow, 117342, USSR
Abstract. This report is concerned with the problem of sequential (on-line) detection of abrupt changes in stochastic processes. Many different tasks of control theory, identification theory and signal processing can be solved by making use of such approach. This report presents some results of recent work on development and investigation of the sequential change detection algorithm based upon a likelihood ratio. Both simple (additive) and more complex (spectral) changes are introduced through this general approach. Some typical application of on-line change detection algorithm are also described. Keywords. Signal detection; Stochastic systems; change detection; monitoring. standard criterion of optimality for offline algorithm is to maximize a test power (3= 1- q ,where 0= P(H,,/H~). with a fixed probability of false alarm (level of the test) oL= P{H1IHo) . In asymptotic (non local) case 0(. and ¥" are replaced by exponential level and exponential error of the second type (see J.Deshayes and D.Piccard,1986).
1 . INTRODUCTION AND PROBLEM STATEMENT. This report is concerned with the problem of detection of abrupt changes in stochastic processes. Many different tasks of control theory, identification theory and signal processing can be solved by making use of such approach. There are three main classes of change detection problems,
1.3. A posteriori (off-line) estimation of a change time. A given finite size sample is (!lK)1£K~#' previously collected. Till the time 1;0-::1included, the vector 9~()fand then B=f), It is necessary to estimate an unknown time io : 1< -to ~/II. The criterion of optimality is to maximize the probability
1.1 Sequential (on-line) change detection. Let ('lit) 1~K~t; be a random sequence with the conditional distribution density P9 (Cjf; IIj $-1) ,where f) is a parameters vector. Till the time to-£ included , the vector (14. and then (under t"? to) B. Bz . The time moment io is assumed to be unknown. It is necessary to detect a situation when change occurs ( to!f t ) by means of observations (!lit. 11~K~e . The criterion of optimality for sequential algorithm is to minimize the mean time delay E{ta.-t o +1 Ita.">-- -to)
e
D
p(lto-to/~£) that estimator belong to a given confidence interval, or to minimize the first two moments of probability distribution of an estimation error. J.Deshayes and D.Picard (1986) suggested to use a relative estimation error
where ta. is a stopping time (time of detection), with an assigned mean time before false alarm
i o{II)
/01.. (N) -
or probability of a false alarm
Hi.:
. d proper t 'les an d studle
P8{YKIJ~-f)=Peo{/lKI//~-f)
to ~ K 01 Po (jjl( IJtf):: 'p()2 i
0
is an unknown time
>
tr -
?O
;.1-'> :;;0
f
This report is concerned with the first class (1.1) - sequential change detection. This problem was formulated in 50's - 60's for the independent gaussian random sequence. Early investigators (see Girshick and Rubin,
(!I~ /:11-')
« t o 1>N.
to(~)~oo , N-to(N)"""OO
""' ~ 0 N It is obviou~ that there exist some other possible statements of the change detection problem, for example, detection of several change points. But these three classes of change detection problem are basic in some sense.
~f:;o)
f S K U.-i. ,09 (jjK/!//-f) = ,o()~ (J'K IJ~-f)
where
(}o( (N) / - - 0 IV ..." 00
1.2 A posteriori (off-line) change detection. Let (Y~)~£~twbe a finite size sample, previously collected. It is a necessary test between: Ho : HIU.N
to
for a local asymptotic case
E(to..! tc.. «iD)
P(ta.
-
!-I
The
11
1952; Page, 1954; Shiryaev, 1961,1963) solved this problem using the Bayesian approach and Wald's sequential probability ratio test (SPRT) renewed from zero with the lower boundary equal to zero, which was given a name of a cumulative sum algorithm (CSA). Shiryaev (1961, 1963) proved the optimality change detection algorithm based upon the Bayesian approach, and compared this algorithm with the CSA for continuous time. Lorden (1971) proved the asymptotic optimal i ty of the CSA and discussed the change detection problem wi th an unknown Other very important results were obtained and presented in recent papers (see Pollak, 1987; Pollak and Siegmund, 1985; Dragalin, 1988). Dependent random sequences and vector case of the parameter were described in the papers (see Willsky,1976; Nikiforov,1978,1979,1980,Basseville and Benveniste 1983a,1983b;Bansal and Papantoni-Kazakos,1986)and books (Nikiforov, 1983; Basseville and Benveniste (Ed.), 1986). Some results of recent work can be found in the survey papers (Phillips, 1969; Gibra, 1975; Willsky, 1976; Mironovski, 1980 ; Kligene and Telksnys, 1982; Basseville, 1988; Frank, 1990) and in books (van Dobben De Bruyn, 1968; Shiryaev 1978; Nikiforov, 1983; Basseville and Benveniste (Ed.); 1986).
following condition is fulfilled (4 )
Sometimes (3) can be replaced by (3a)
7:°.=
E{ta..-t,,+1/ilt*to)
It is obvious that parameters 7: ('i;*or i- 0 ) and (T ) describe the statistical properties of the sequential algorithm for two values of parameters 8z and B1. only. Therefore, the spec ial function B) = E & ( la.)
i3r
L(
was introduced. This function was called an average run length (ARL). L(&) is a mean stopping time (from start of observation to the time ta..) under a constant value (J • The ARL function fully defines the properties of algorithm: LfB() = T.J L{Oz.) = Eez. etA) =T The role of this function in the change detection problem is as important as that of the power function in statistical hypothesis test. 3. INDEPENDENT SEQUENCE, SCALAR CASE. Let (YK)1~K£tbe a random independent sequence.
P%
2. CRITERIA
3.1 . Let be a priori known probability distribution of a random c hange time -to
We start with classification concerning the assumptions about an unknown change time
PJr
to
2.1. Let PJr be a priori known probability distribution of a random change time to(;#, where AI=(Q,1,2r ,,). In the on-line framework, the criterion of optimality is to minimize the mean time delay (Shiryaev,1961,1963)
E7[ (t(;.-
+'0+1. /-t:c...";;to)
({;o
='
I'l.. / 6o~o) =
(1)
.
/
or
where h is a threshold.
t
a .:: -to) >,.
T
1r:~ (
(5)
f
3.2. Let io be a non-random unknown value, or random value with unknown a priori probability distribution. Other conditions hold such as 3.1. Page (1954) proposed a cumulative sum algorithm based upon the SPRT : t:a. = Lhf it : ~f ~ ~ ].J !li = (9i--t+.t;r :~}~~~) +~ (x) t mQx(o,A:j 6) Lorden (1971), Moustakides (1986) proved optimal properties CSA from the "worst case" criterion point of view. There are many others well known suboptimal algorithms for solving problem 3.2. All this algorithms are based upon a very important concept in mathematical statistics, namely the likelihood ratio (LR). From the general view point the designing change detection algorithm includes two problems: - derivation of a sufficient statistics (SS) ; - designing of stopping rules.
(2a)
where cl.: 0 ~ '" < 1.. and Tare given constants. Other possible criterion in this case is to minimize the risk ~
J(~tA) = /}r (tt< <: to)+ C E;,-(tA-to+J..!tc.. Ho)&r(ta. ~~) where c >O is a given constant. 2.2. Let io be a non-random unknown value. In some applications, it is useful to consider that to is a random value with unknown a priori probability distribution, or to is a non-random unknown value. In this case it is very useful to have algorithms which are free from distribution of change time to . For this reason, the criterion of optimality is to minimize the "worst case" mean time delay(Lorden,19711
r~~ stLfJ
II-:t.
f-71f
ta=/../lht: 9t-:,-n} 7 ii: = tn.(a. + ejt--f)-~(f-a)fth
(2 )
/
ft.-a.) a..
Till the time to-1. included observations density is I}{jt;}and then (under t ~t,,) P2.{n). Shiryaev (1961,1963) proposed an optimal algorithm based upon calculation of the a posteriori probability Jr~ revealing a change time has occured . The stopping time ta.. has the form (a posteriori probability replaced by other function for simplicity it=t},..2!1:.....):
in the class of algorithms for which the following condition is fulfilled
EJT (t 0.
(3a)
-to~::t
e.z, .
r=
SI.<.fJ
/; --(
eJSs¥F(fa.-t.+1/tlA~toJ!I/)(3)
-to ~ 1-
in the class of algorithms for which the
12
ss:
LR is a
SI:
4.2 . Let &2- be an unknown vector. Other conditions hold such as 4.1. There are different possibly approaches to this problem. The GLR is the most general algorithm: . -to.. = i t : ~t ~ ~}, t /J Q tna:>e. S(..(.P L Ut . 10) d1: - {$K~ t IB.z-B.d ~ £ ,':K Pt;1(!ldY,!:; It is obvious that the GLR is generalization of the CSA under the unknown vector B.t.. Because of the double maximization, this algorithm is very time consuming. The number of each step calculations tends to infinity fast with t tend to infinity, because it is necessary to estimate Su.p under each possible change time k from 1 to t. For this reason, immediate use of GLR for practical implementation is not obvious. Let us first investigate some possible simplification of the GLR. There are two possible approachs: - eliminate first maximization
= f1
Let EJ.. and Ez denote expectations of random variables under the two distributions characterized by Bt(before change) and 82 (after canhge) respectively, then: E-f(SI:)LO ~
"I {
t"2(St)?O.
A decision function 9t of the Bayesian algorithm (5) based upon "weighting" a priori information about a change time and ss (S")'HI 0 and decision function is growing. The final cycle of the SPRT ends with reaching the upper boundary. For our further discussion it is very useful to rewrite CSA (6) in other form
it:
t -&.. 3b- /:::t~'':K
!zf5't) Ptf!/t/
fhax.
i_
(7)
t
/J
~ "Ut.
~.z (~,)
it =
(8)
e
4.1. Let (YK)(5K5f: be random sequence (scalar or vector) with a conditional density PO(!I.fIJ~-(), where (j is a vector: Be Till the time tg-i included (;=B1 and then 0=82 , Other conditions hold such as 3.2. In this case the generalized CSA has the form
QI: 11
= (s t
t-~ +i
(e.fJ
st(P.fJ ()L) = -&.
~
J) (:t;' / ~
A,
I(AJ
jr@
"
y,
'it - p U1
f{}z (!IdJt'-~;)rJ".
fJ&~ (!fdJt~~)
P&d!idyr;)
fBt(!fd!f,,-p,/
tJ ,~ fl:/J
(!( \
6. LOCAL APPROACH.
:-f)
Pth (~: / y/~-0 where
f..J, - -
(J2.))~ Pf}.z
(11 )
(17, /YL' -~)
p(}.f
w/:: t W(.·, ,_ (n (i~/1 ~ '..1) /J
«It. .
if ~
1.
, where ~ is maximum likelihood (ML) estimate of the vector B.z. , using a sliding time window [t-m+1;ml, m is a fixed length of this time window. It can be said that decision function (11) computes a distance between two models. The first model is characterized by B1~ and the other one is characterized by ~ . Basseville and Benveniste (1983b), Basseville (1986) proposed other possible distance. They recognized that LR can not be convenient for some application (for example, speech segmentation) ,because the behaviour of decision function (11) is not symmetric for the detection parameter jumps from Bf to {)2 and conversely. From the point of statistical decision theory this non-symmetric behaviour is normal, but it is disadvantageous for speech segmentation. For this reason, it is iseful to replace LR by:
From practical point of view, algorithm 3.1- 3.3 have restricted applications to control Dheory and identification theory because the random sequence (¥K)UI'!tis independent and is a scalar value.
(-i :
Id ~ f1:Jz.cY~·l'1f;)
(.' =t:-/11+i
4. MORE COMPLEX MODEL.
(.'/1/
)
Basseville and Benveniste (1983b) and also Appel and Brandt (1983) proposed this approach. The main idea of two-model approach is to replace (10) by: -ea. = ~·k.f ( " " ~~ ~
f~.t:~t: /r92.-Bd~f t:K Pe.dj/) where [(/')'7'0 in a convenient manner. It becomes obvious from comparison decision function (7) and (8), that the Lorden's algorithm is the generalization of the CSA for case of the unknown value &2.' Generalized likelihood ratio (GLR) algorithm (8) has been recently studied for a more complex model by Willsky(1976).
-to. ':
(
5. TWO-MODEL APPROACH.
Till the time to-i included {j:BLand then {;=&2., where (12 is unknown value. Other coditions hold such as 3.2. Lorden (1971) proposed an algorithm: i:(),= '-111ft.' It ... hJ ~ fhQX.tt.r'
/;
and replaced GLR by "two-model" approach; - eliminate the second maximization I"~~(/~C( ) by using "local" approach.
3.3. Let (~K).HKSi:: be independent random sequence. Assume the distribution density belonging to a Koopman- Darmois famil y Pe(~)= eXf (BT(1) - ~(B)).
91: =
PGi!ld!l.tP.J
If vector eland &.zare inaccurately known we are to consider the local hypotheses case / e-f- BL / ~ 0 • For example, this is a problem of signal detection with a very low signal noise ratio (SNR). It is obvious that for a fixed size jump of the parameter a local algorithm is not optimal, but this approach has two main advantages: - change detection algorithm is very simple (low computational cost) and
~
is an indicator of the event
fit is a counter of an observation
it.
number after the last nulling Bansal and Papantoni-Kazakos(1986) proved optimal properties under some model.
13
calculation ARL function is also more simple; - local hypotheses case is the "worst" change detection case and this is useful to optimize the algorithm for this purpose. The ideas of the effective increments, the local tests are very powerful and fruitful in statistical decision theory and estimation theory. This ideas were founded by Fisher, Le Cam, Roussas (1972), Ibragimov and Khasminskii (1972), Davies (1973) and other investigators. They derived very deep and precise mathematical results in this theory. Let us discuss the key point of this theory, asymptotic expansion of the LR. This expansion is possible, for example, for the p-dependent Markov sequence (AR-model) and for ARMA-model under Gaussian assumption. We write this expansion for p-dependent Markov sequence
S/((),
f)+
~
Sf; ~
n
i=1
ift.}:: tl"'>
If
']=(&))
t
L.
-to ,(t)=&~)
t ~i:DJ({}=Bf+r9).
9i: ~ A~-,
n1i
F
(.1.. z'
-1
T
t:
~ 7(fJ.,)lIt.1 Vt:
;d It ;/, ) ~
(14 )
vt-1 J {Jt_i7 Q }+Xi,
(15)
is a known matrix (m x n) , I'l'v>/'t) ~k().()= ~ is a known system output, Xt is a unknown system input, measurement noise ~c is a sequence of a Gaussian vector:
for t <: to for 1: ~-eo In this case the decision function and stopping rule of CSA algorithm (6),(9) has the form
~"':f{t ·
=
/J
u.,oi.
Ye- = HIt + Si- r I{t~t:O)" > where tt~R.h, Ytf:./{M, '5tGR~.1fR; H
e
= Cji-1 +::.c.[c)+ . 6.2. Let vector e~ before the change time to be known exactly, and after be described by the inequality (ez-&.ff 7{B.()(&2-B.()~/I/- .
cu.dO.z:
The analytical redundancy approach is well-known in the automatic control theory. Basically this method is widely used for failure detection and diagnosis. Willsky (1976), Mironovski (1980), Frank (1990) described this problem in detailes. The aim of this paragraph is to demonstrate and to compare the traditional analytical redundancy approach (see, Frank, 1990) and the unified statistical approach, based upon a likelihood ratio. Let us dis c ussed one simple example. Consider a linear system given by
eo
-ea. =
r
7. CHANGE DETECTION AND ANALYTICAL REDUNDANCY.
6.1. Let vectore be described by the equation B= 1" Jc.. ,where a vector C:lcl_~ determines the direction of the change of the vector ,CGR.'7.., and j) is the s c alar value:
fit
forf~to
where oFL(c,x) is a generalized hypergeometric function.
which is a Gaussian vector with the known covariance matrix, zero mean before the change time and non zero mean after the change time: N(:rt8)["& > 7'{9-))
to
l
Sic: == - i 2~l +
where .L3L· is a vector of effective increments, l!;<'fR'2., .r{B) is the Fisher information matrix (r x r). Let us consider again the change detection problem in a local case, Introduce the random vector of effective increments which are the main parts of the asymptotically sufficient statistics (ASS)
iNtO>
for t..(
J
r
~ 0 t-".?O
1:. ~""
00
t -1; (rt rtiJ'}:,l)
((;2.-iJ(J J/9.,)fG~-&.,):./I; /'Z -tilt .f I . where IT) T J I./ 1S noncentral ~Z-distriution with r degree of fredom, tAZis a noncentral paramet e r. Using this expression for the density of the ~2-distribution, the CSA decision function can be written in following way -t-a.:: /......! it .' 9i- ~ R} > It:: (Si.) +.1
Vector Ai and positive definite symmetric matrix rIB) can be written under some conditions in the following way
£{X-t} ~
jrJr (Pz{@)urS'l=i.) ,_j
7
Pp.,u/);
Asymptotically, under local condition
<~f
09
r
'l.
h where·lJ~.rg)is the weight function, Stz. is ellipsoid surface separating the indifference zone from the zone of a cceptance of the hypoteses "change occur". Using expansion (12), we arrive at a conclusion that in this case the ASS is assigned by t t . " f T 7-'1 J t IJ" = L X, fl:: ~ 4..{ vI {&.( ~1
df)I;)-;::/ "[dB,,- ; reIFt&)of)t > (12)
o Of, = -
f lJ?.J8-)Pe.(y/)dS't
p (It.
nS+)
=0
>
E
(';t-~{
) ". bZI.
Till the time -eo-1. included no change occurs (i.e . I[t-~t.j).1=OUnder t<:t.) and then change oc c urs (i. e. Ilf~ to} A =,1 underf~t-o). Model (15) is a typical model of a measurement system. The term I{O;io}1:J is needed for sensors failure modelling. It is necessary to detect a situation when sensors failure occurs by means of the observations (!lK)f~K~~'
(13 )
7.1. Traditional analytical redundancy approach. Failure detection systems are based on the use of analytical redundancy relations or parity checks: tt=TTY-t) (16)
In this case, the CSA synthesis requaires the use of the Wald's weight function method, i.e., instead of the LR sl we use its analog calculated by taking into account the "weighting" of all possible directions:
s,t
where T(m;«m-,.,;)is a special matrix: rTH
=0
j
'T' r". ::
I
.
It is obvious, that these conditions
14
X
eliminate unknown input signal t . In this case (16) can be rewritten as
Et =
TT5t
+- I{i->.-t:o}
so that
T
1/(O>b 2 I)
.;({EtJ=
/II(7''A.>~'I)
(
is possible to derive the final criterio n of optimality (Taylor, 1968). The criterion for CC is to maximize a total mean profit of the common operation of technological process and quality control system. From this point of view, (1) - (2) and (3) - (4) are auxiliry criteria.
Ll fo'!..
;lo'l.
tLbo
t~
'to
8.1.1. Example. Let(ff/tC)1~"-~tbe a Gaussian AR random sequence. Let mean mJ.. of the AR-model be a target value. It is necessary to detect situation when a mean change occurs. Usually there are two "taboo" reg ions: mz:= rn.-( ± 2. /( , where k is a reference value. Two-sided CSA (9) can be rewritten as follows -co-=t..kf{t: C3f,,:h)VC9t'd)], +
These artifical measurement are usually called "residuals". Residuals are designed to reflect a possible failure in analyzed system (15) . Change detection algorithm can be designed by means of these residuals. 7.2. Likelihood ratio approach. In this case LR can be rewritten as follows
S f
t
-&. fo, {Y.: / f.. (1l2 ))
1
=
,
fl
(
X
)
9" =[ji---f
=1
Xi
st_ x..-t 1 -,
T ( 7';1
,=1-
-
where -if), matrix
6"2.
•.)
t
TTy. i
_ r i Tl ) T~/i ., / ' - (1Ir
"'-
t
~
··, /1/-
where i!L!Ii, =!/t-i. is shift operator, 0(,1''''10I/, are autoregressive coefficients.
8.2. Early detection of failure. The continuous normal operation of the manufacturing process is a very important problem. An abnormal situation must be detected very fast in order to avoid industrial accident. It is important in this case to use information of different sensors (inputs and outputs of plants) together. Consider a regression model
17 )
rn-I/. are eigenvectors of the
p= I -
H(HTH)-iHT
!le
These eigenvectors correspond to unit y eigenvalues. It becomes obvious from comparison (16) and (17), the residuals (16) are a convenient statistics for algorithm from statistical point of view. There are different possible algorithms for solving this problem. It can be GLR or CSA. For example, in this case CSA (14) can be rewritten as follows
:i~~iY./ Jt'" (ft)+.1 III .en.01.F (fIZ-n , nt )j-x f) "2 + 24 '(18)
= - Y1
:Xi: ~
VeTVI,
' Vt
= vt-i. F[9t_1'ny -+ 1ft
= eTXi +l1.t,
where Xt is a known input, Yt-is a known output) X(;61<.7 8 is a parameters vector $eR.1!:. ,!Lt is a Gaussian whi te noise: E(n(;)",0, E(fll)=";n2. Normal operation of the manufactur ing process is characterized by the vector81 . It is necessary to detect a situation when a parameters vector jumps from Bito 82. The direction of th e jump is unknown. GLR or CSA (14) are convient in this case. For exa mple CSA (14) can be rewritten as follows
f:o.::t;' k.f[t
St
+(1-"'1'2 - ...-"'t~."){/Ir/11.,-K) '7 ,
it =[ ft-I - (1-"'fr- ... -olf~I)(!I~-1>I1+iOJ:
J
r8-( Yt: I X, ' (f}-f) where (/(= 0, B:z. = ;1 , (6;) is a least squares (LS) estimate of an unknown input signal )',' under assumption that no change occurs {@f= ()) or c hange occurs (et="')' It is possible to prove, that in this case LR can be written as follows L
.
= Vt--t I {9l_1>O) +Xt f t f t = :/t- B/Xt.J '.hCf):o :,.2. R)(x, /(Xy. ~ EO"Xl) . The computation of ta, Si: and;Xi hold Vi;
8. QUALITY CONTROL AND PERFORMANCE MONITORING.
1
such as (14). Another important problem is robust change detection algoritms with respect to nuisance parameters. Let(~/tC)(~JC,~be a random sequence with the conditional densi ty P(!J
Typical features of modern technological process and plants are complexity, high capacity, high risk for environment. Change detection a lgorithms can be used for performance monitoring of the technological process and for quality control.
e
e
8.1. Statistical quality contro l. In fact, it was first implementation of change detection algorithms, which were given the name of control charts (CC). Consider a technological process which has two possible conditions: "in control" (no c han ge occurs) and "out of control" (change occur). Let ('f/tC)1~JC~'t be a sequence of the observations of the performance index with the conditional distribution densi ty Pe (Y~Nt.,--f). These technological process conditions are characterized by parameters vector e~ (in contro l) and 9z (out of contro l). It is necessary to detect a situation when a process is out of control . In the simplest case a parameter 0 is a mean or variance. In quality contro l problem it
High reliability measurement systems are an essential part of the modern plants or movin g objects (ship, aircraft, ... ). Sensors failure detection is a very important problem for high d egree reliability in control data. The problem of the "soft" failures detection, which
15
are not detected by the built-in test equipment, can be solved by using redundancy.
arrival time includes two subproblems: - sequential (on-line) detection of P(S-) wave and preliminary estimation the time at which P- (S-) wave arrives; - a posteriori (off-line) estimation the time at which P- (S-) wave arrives and identification of the wave's type by using fixed size seismic data sample.
9.1. Inertial navigation system (INS) monitoring (Potter and Suman, 1986; Varavva, Kireichikov and Nikiforov, 1985, 1988; Kireichikov, Mangushev and Nikiforov, 1990). A simplified INS error model involves linear differential equations. In discrete time the dynamics of errors takes the form of a state space model
{ where
XI;
10.1 Sequential detection of P- (S-) wave. Early investigators derived the model for seismi c noise, P- wave and Swave. It was proved that AR-model is convenient for this purpose. We assume that the arrival of a seismic wave or a change of type of seismic wave results in changes in spectral and polarization properties of a s i gnal. For example, P-wave has a linear polarization in a source- receiver direction . For this reason the model of signal can be written in the following way
= F(i; tf, f)Xt-1 + ~i: +Itl;~tc} A
Yt .: !tU)lt
+VI;
(19)
,x"fR."- 'It fR."', S/;~R.~ Vi,fR."'; Si-.>
Vt
are independent Gaussian zero mean sequence of random vectors. The term J{~~~.~ A is needed for sensors failure modelling. INS output measurements involve useful signal and measurement noise :
at
r-t
LIt = Zt
Zi
;Vt =[
y"
+
Xt
Yt
$.. 1 [_o..(r./(:1 /. l
y{ . (l11.
+
Y
R-'.f",..] QOL
00,',
J
Ps At 'Ni-L' +IIf) Zi: (3t';{t-t' + ft,
r
(20)
L=1
where Zt is a three- component seismic signal, t f is a time at which the Pwave arrives, IVt is a three component seismic noise, A", .. ., Ap,. are seismic noise autoregressive matrices, :::et is a one- component P-wave, (31,··~jJPs are a Pwave autoregressive coefficients, H is a matrix (3 x 1) which depends on a sourcereceiver direction. The matrix H is a priori unknown. For this reason, immediate use of model (20) for practical implementation is not obvious. It is simpler to use one- component change detection algorithm. In this case we assume that the arrival of the P-wave results in a change in autoregressive coefficients and variance. The CSA decision function (13) can be rewritten in the following way
= Ilfl; - tiN = >It - Yzt: It can be proved (Willsky, 1976) that LR can be written in the following way 'f- 2:
+ I (t >, t: PJ H X t
t':{
It is possible to eliminate the useful signal by using a pair of these INS :
S}= "'
= Ifi PIr
,
/."=1
where oot. and If",' are the measurement innovations processes of Kalman filters under hypothesis ho(no change occur) and hi (change occur), Ri is a covarianc e matrix of the innovations. Willsky (1976, 1986) described GLR for this case. CSA (14) are convient for this case too. The computation of +'a.., ~~ and it hold such as (14), vector Vi and ma tr ix 7(6) can be wr i t ten
fa.: CIt/rt :!Jt>,h} "Jl=
!It = I~~ U'~-1 Et ) (f (!L -1 \ G'£ GE2 '/
9.2. Redundant inertial sensors unit monitoring (Potter and Suman, 1986; Kireichikov, Mangushev and Nikiforov, 1990). The primary goal of the redundant inertial sensors unit is tolerant with respect to sensors failures. The errors model (without failure and with failure) of the array of five gyros can be written by using equation (15) under n=3, m=5. There are two possible approaches: - to use robust ("worst case") estimate; - to use failure detection and fault diagnosis algorithms. The first subproblem of the second approach is the sequential change detection. The change dete c tion algorithms in this case was described in paragraph 7.
j
l/l _/'
($H
+!/[c)~
(~t-f)
>
(
21 )
{ Z i -I'
Ft = 2-t - a.rU~-i > a T= (oI.("'''J 011') j where oi1) .. .,oIl'are AR- model coefficients, zf is one compone nt of a threecomponent seismic signal, ~~ is a residual variance, C is a vector of the assumed direction of the parameter jumps, et eP+J.. Let us remark that ARmodel for CSA (21) is not convenient model for seismic noise or for P-wave. Roughly speaking, this is an intermediate model. On the other hand, parameters a,GlZ.and C defined the separated surface equation in the parametric space. The computation of the preliminary estimation the time at whi c h P-wave arrives is based upon a simplified maximum likelihood (SML) algorithm. Let(rJ(.)"$K~tc..be a fixed size sample (see problem statement 1.3). It is necessary to estimate an unknown time ~p . Assume that 91 and Bz known vectors. In this c ase likelihood function has form K-( I ' ~) t't " 1)
10. AUTOMATIC ON-LINE SEISMIC PROCESSING. One of the main tasks of the on-line digital seismic data processing is detection and estimation of the arrival time of the seismic waves (Nikiforo v and Tikhonov, 1986; Nikiforov, Tikhono v and Mikhailova, 1989). It is possible to estimate the source- receiver distance by using delay between P-wave and S-wave arrival times. The general problem of the detection and estimation of the
FK
:
L. ~ ~ (2dz,·_~ +? Pe/ZL .J2 , ·:P i=' ( l=K
under assumption that k is the time at which P-wave arrives. SML-estimate is
16
based upon maximization of the
generalize CSA for a complex signal: i f' rr -'( "') ,,I 0/,,1 + 2...
11: =
nt
Vt ~
""
Vt
s"Et
) Vt;= vi-,.1.(9f-170YC/t.
c-/:=':It -S[ti. ) ']=(I'n)~ CM. z'E(slST)
(22)
t:
where (A)~ is a complex conjugate with respect to A, (A)f~ {(A)"'£". The computation of t" and Si: hold such as (14) under r =
Let us return to CSA decision function, which can be rewritten in the following
2e.
(23 ) It follows from comparison (22),(23) and decision function (9), that SML- estimate can be computed by using standard CSA. It is possible to prove that algorithms (9) and (13) are equivalent in some sense (Nikiforov, 1983), for this reason A.
REFERENCES Appel U. and Brandt A.V. (1983). Adaptive sequential segmentation of piecewise stationary time series. Information Sciences, vol.29.
-ta.
-hia. + i. In practice, the seismic signal ARmodels before and after P-wave arrival are unknown and very nonstationary. For this reason precision of SML- estimate is not high, but sufficient for choosing a center of the fixed size time window. A posteriori estimation the time t f realized by using this time window. tpSML::'
Bansal R.K. and Papantoni-Kazakos P. (1986). An algorithm for detecting a change in a Stochastic Process. IEEE Trans. on Inform. Theory, vol.IT-32, no 2, pp.227-235. Basseville M. (1986). The two-model approach for the on-line detection of changes A.-R. processes. Im the book "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, p. 169215.
11. IMPROVEMENT OF THE TRACKING CAPABILITY OF THE ADAPTIVE ALGORITHM Improvement of the tracking capability of the adaptive algorithm is an importent problem in identifying timevarying linear system parameters. Assume that 6(1:) is a time- varying parameters vector is subjected to random jumps (Perriot- Mathona, 1984; Wahnon and Berman, 1990). This is an important problem for the adaptive LS algorithm in the adaptive arrays theory ( Monzingo and Miller, 1980). The complex random process is the natural domain for narrowband electromagnetic signals (radio and radar). Array signal processing based upon adaptive filter contains adjustable weight coefficients:
!it ==
..AT
m..(t-)
Sf +
ht)
Basseville M. (1988). Detecting changes in signals and systems - A surey. Automatica, vol.24, pp.309-326. Basseville M. and Benveniste A. (1983). Design and comparative study of some sequential jump detection algorithms for digital signals. IEEE Trans. Acoustics" Speech, Signal Processing, vol.ASSP-31, no 3, pp.521-535. Basseville M. and Benveniste A. (1983). Sequential detection of abrupt changes in spectral characteristics of digital signals. IEEE Trans. Information Theory, vol.IT-20, no 5, pp.709-723.
(24 )
1ft is an output complex signal, Y.t EC
If,'
=
~ t-~'
Basseville M. and Benveniste A., (Ed.). (1986). Detection of Abrupt Changes in Signals and Dynamical Systems. Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York. Basseville M., Benveniste A. and Moustakides G. (1986). The local method applied to the robust detection of changes in the poles of a pole-zero system. in Detection of Abrupt Changes in Signals and Dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, p. 259274.
) L': ".J I: )
where Il is a smoothing constant O.{.A,:: 1. . The "effective memory" N of this algorithm has the form )lz _-1_ 1- '}.
Let us consider one possible solution of this problem. Assume that ;l = Il opf ' where ~OHis an optimum value of the smoothing constant for "slow variation" of the true vector m(t). Detection of abrupt changes based upon CSA (14). In this case e~ is equal to adaptive LS estimate ~(t). Let us discuss a typical cycle. Till the stopping time -to. we use A=).opt. After the stopping time ia. it is necessary to reduce?: i\ ~ > N(). hta. and then to increase.?l step-by-step , from A"',1t to /lopt::. It is very simple to
J.,..,,,
Davies R.B. (1973). Asymptotic inference in stationary gaussian time series. Advances Applied Probability, vol.5, no 3, pp.469-497. Dragalin V. (1988). Asymptotic solutions in detecting a change in distribution under an unknown parameter. Statistical problems of control, Issue 83, Vilnius. pp. 45-52.
"',,,)=
17
Nikiforov I.V. (1983). Sequential detection of abrupt changes in time series properties. Naouka, Moscow. (in Russian).
Deshayes J.,Picard D.(1986). Off-line statistical analysis of change-point models using non parametrical and likelihood methods. in "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, pp. 216-258.
Nikiforov I.V. (1986). Sequential detection of changes in stochastic systems.in "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, pp. 216-258.
Frank P.M. (1990). Fault diagnosis in dynamic systems using analytical and knowledge based redundancy - A survey and new results. Automatica, vol.26, pp.459-474.
Nikiforov I.V. and Tikhonov I.N. (1986). Application of change detection theory to seismic signal processing. In "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, p. 355373.
Gibra I.N . (1975). Recent developments in control charts techniques. Jal of Quality Technology, vol.7, pp.183-192. Girshick M.A. and Rubin H. (1952). A Bayes approach to a quality control model. Annals Mathematical Statistics, vol.23,pp.114-125.
Nikiforov I.V., Tikhonov I.N. and Mikhailova T.G. (1989). Automatic on-line processing of seismic station data - Theory and applications. Far Eastern Dept. of Academy of Sciences, Vladivostok, USSR (in Russian).
Himmelblau D.M. (1978). Fault detection and diagnosis in chemical and petrochemical processes. Chemical Engineering Monographs, vol.8, Elsevier, Amsterdam.
Page E.S. (1954). Continuous inspection schemes. Biometrika, vol.41, pp.100-115.
Ibragimov I.A . and Khasminski R.Z. (1972) Asymptotic behavior of statistical estimators in the smooth case. I: study of the likelihood ratio. Theory of probab. and Allp., vol.17, no 3, pp. 445-462
Perriot-Mathona D.M. (1984) Improvements in the application of stochastic estimation algorithms- parameter jump detection. IEEE Trans. Autom. Control, vol. AC-29 p. 962- 969.
Kireichikov V., Mangushev V. and Nikiforov I. (1990). Investigation and application of CUSUM algorithms to monitoring of sensors. Third All-Union Seminar on Detection of Change in Random Processes. in book "Statistical problems of control", Vilnius, Issue 89, p. 124-130 (in Russian).
Phillips M.J. (1969). A survey of sampling procedures for continuous production. Jal Royal Statistical Society, vol.A-132, no 2, pp.205-228. Pollak M. (1987). Average run lengths of an optimal method of detecting a change in distribution. Annals Statistics, vol.15, pp.749-779.
Kligiene N. and Telksnys L. (1982). Methods of detecting instants of change of random process properties. Automation and Remote Control, vol.44, no 10, part.II, pp.1241-1283.
Pollak M. and Siegmund D. (1985). A diffusion process and ist application to detecting a change in the drift of a Brownian motion. Biometrika, vol.72, pp.267-280.
Lorden G. (1971). Procedures for reacting to a change in distribution. Annals Math. Statistics, vol.26, no 1, pp.7-22.
Potter J.E. and Suman M.C. (1986). Extension of the midvalue selection for redundancy management of inertial sensors. Journal Guidance, vol.9, no 1, p. 37-44.
Mironovski L.A. (1980). Functional diagnosis of dynamic sistems - A survey. Automation and Remote Control, vol.41, pp.1122-1143. Monzingo R.A. and Miller T.W. (1980) Introduction to adaptive arrays. New York: John Willey.
Roussas G.G. (1972). Contiguity of probability measures, some applications in statistics. Cambridge University Press, Mass.
Nikiforov I.V. (1978). A statistical method for detecting the time at which the sensor properties change: Preprint IMEKO Symp. on Application of Statist. Meth. in Measurement. Leningrad 1978, p.1-7.
Shiryaev A.N. (1961). The problem of the most rapid detection of a disturbance in a stationary process. Soviet Math. Dokl., no 2, pp.795-799. Shiryaev A.N. (1963). On optimum methods in quikest detection problems. Theory of Probab. and its Appl., vol.8, no.1 pp.22- 46.
Nikiforov I.V. (1979). Cumulative sums for detection of changes in random process characteristics. Automation and Remote Control, vol.40, no 2, part 1, pp.48-58 .
Shiryaev A.N. (1978). Optimal Stopping Rules. Springer-Verlag, New York.
Nikiforov I.V. (1980) . Modification and analysis of the cumulative sum procedure. Automation and Remote Control, vol.41, no 9, pp.74-80.
Taylor H.M. (1968) The economic design of cumulative sum control charts. Technometrics, vol.10, no 3, p. 479-488.
18
Van Dobben De Bruyn D.S. (1968). Cumulative sum tests: Theory and practice. Hafner Publishing Co., New York. Varavva V., Kireichikov V. and Nikiforov I. (1985) Detection of sensors failure as a problem of detecting the change in random sequence properties. Abstracts of Second All-Union Conference on Modern Methods of Planning and Experiments Analysis in Investigating Random Fields and Processes, Sevastopol, part 1, p. 18-20. Varavva V., Kireichikov V. and Nikiforov I. (1988). Appl ica t ion of change detection algorithms to monitoring of measurement systems of moving object. Second All-Union Seminar on Detection of Change in Random Processes. in book "Statistical problems of control", Vilnius, Issue 83, p. 169-174 (in Russian) . Wahnon E. and Berman N. (1990) Tracking algorithm designed by the local asymptotic approach. IEEE Trans. Autom. Control, vol. AC-35, no 4, p.440- 443. Willsky A.S. (1976). A survey of design methods for failure detection in dynamic systems. Automatica, vol.12, pp.601-611. Willsky A.S.(1986). Detection of abrupt changes in dynamic systems. in "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, pp. 27-49.
1.s . . . . . . .
-1.5.
o
200
400
600
800
1.000
.1200
1.400
Fig. l(a). Gaussian vector random sequence (XK).I~ J(~i ,X~&R5 Change time t. =1000. Signal/noise ratio SNR = 0.2
1.0.
l
8.
.~
6.
I· · !~
4. 2. 0.
L.J..i~."",,·l
0
290
490
Fig. l(b)
690
890
1.000
Desicion function of the CSA.
19
I
I
1.2'00
1.400
SEQUENTIAL DETECTION OF CHANGES IN STOCHASTIC PROCESSES I. V. Nikiforov Institute o/Control Sciences, Pro/soyuznaya St., 65, Moscow, 117342, USSR
Abstract. This report is concerned with the problem of sequential (on-line) detection of abrupt changes in stochastic processes. Many different tasks of control theory, identification theory and signal processing can be solved by making use of such approach. This report presents some results of recent work on development and investigation of the sequential change detection algorithm based upon a likelihood ratio. Both simple (additive) and more complex (spectral) changes are introduced through this general approach. Some typical application of on-line change detection algorithm are also described. Keywords. Signal detection; Stochastic systems; change detection; monitoring. standard criterion of optimality for offline algorithm is to maximize a test power (3= 1- q ,where 0= P(H,,/H~). with a fixed probability of false alarm (level of the test) oL= P{H1IHo) . In asymptotic (non local) case 0(. and ¥" are replaced by exponential level and exponential error of the second type (see J.Deshayes and D.Piccard,1986).
1 . INTRODUCTION AND PROBLEM STATEMENT. This report is concerned with the problem of detection of abrupt changes in stochastic processes. Many different tasks of control theory, identification theory and signal processing can be solved by making use of such approach. There are three main classes of change detection problems,
1.3. A posteriori (off-line) estimation of a change time. A given finite size sample is (!lK)1£K~#' previously collected. Till the time 1;0-::1included, the vector 9~()fand then B=f), It is necessary to estimate an unknown time io : 1< -to ~/II. The criterion of optimality is to maximize the probability
1.1 Sequential (on-line) change detection. Let ('lit) 1~K~t; be a random sequence with the conditional distribution density P9 (Cjf; IIj $-1) ,where f) is a parameters vector. Till the time to-£ included , the vector (14. and then (under t"? to) B. Bz . The time moment io is assumed to be unknown. It is necessary to detect a situation when change occurs ( to!f t ) by means of observations (!lit. 11~K~e . The criterion of optimality for sequential algorithm is to minimize the mean time delay E{ta.-t o +1 Ita.">-- -to)
e
D
p(lto-to/~£) that estimator belong to a given confidence interval, or to minimize the first two moments of probability distribution of an estimation error. J.Deshayes and D.Picard (1986) suggested to use a relative estimation error
where ta. is a stopping time (time of detection), with an assigned mean time before false alarm
i o{II)
/01.. (N) -
or probability of a false alarm
Hi.:
. d proper t 'les an d studle
P8{YKIJ~-f)=Peo{/lKI//~-f)
to ~ K 01 Po (jjl( IJtf):: 'p()2 i
0
is an unknown time
>
tr -
?O
;.1-'> :;;0
f
This report is concerned with the first class (1.1) - sequential change detection. This problem was formulated in 50's - 60's for the independent gaussian random sequence. Early investigators (see Girshick and Rubin,
(!I~ /:11-')
« t o 1>N.
to(~)~oo , N-to(N)"""OO
""' ~ 0 N It is obviou~ that there exist some other possible statements of the change detection problem, for example, detection of several change points. But these three classes of change detection problem are basic in some sense.
~f:;o)
f S K U.-i. ,09 (jjK/!//-f) = ,o()~ (J'K IJ~-f)
where
(}o( (N) / - - 0 IV ..." 00
1.2 A posteriori (off-line) change detection. Let (Y~)~£~twbe a finite size sample, previously collected. It is a necessary test between: Ho : HIU.N
to
for a local asymptotic case
E(to..! tc.. «iD)
P(ta.
-
!-I
The
11
1952; Page, 1954; Shiryaev, 1961,1963) solved this problem using the Bayesian approach and Wald's sequential probability ratio test (SPRT) renewed from zero with the lower boundary equal to zero, which was given a name of a cumulative sum algorithm (CSA). Shiryaev (1961, 1963) proved the optimality change detection algorithm based upon the Bayesian approach, and compared this algorithm with the CSA for continuous time. Lorden (1971) proved the asymptotic optimal i ty of the CSA and discussed the change detection problem wi th an unknown Other very important results were obtained and presented in recent papers (see Pollak, 1987; Pollak and Siegmund, 1985; Dragalin, 1988). Dependent random sequences and vector case of the parameter were described in the papers (see Willsky,1976; Nikiforov,1978,1979,1980,Basseville and Benveniste 1983a,1983b;Bansal and Papantoni-Kazakos,1986)and books (Nikiforov, 1983; Basseville and Benveniste (Ed.), 1986). Some results of recent work can be found in the survey papers (Phillips, 1969; Gibra, 1975; Willsky, 1976; Mironovski, 1980 ; Kligene and Telksnys, 1982; Basseville, 1988; Frank, 1990) and in books (van Dobben De Bruyn, 1968; Shiryaev 1978; Nikiforov, 1983; Basseville and Benveniste (Ed.); 1986).
following condition is fulfilled (4 )
Sometimes (3) can be replaced by (3a)
7:°.=
E{ta..-t,,+1/ilt*to)
It is obvious that parameters 7: ('i;*or i- 0 ) and (T ) describe the statistical properties of the sequential algorithm for two values of parameters 8z and B1. only. Therefore, the spec ial function B) = E & ( la.)
i3r
L(
was introduced. This function was called an average run length (ARL). L(&) is a mean stopping time (from start of observation to the time ta..) under a constant value (J • The ARL function fully defines the properties of algorithm: LfB() = T.J L{Oz.) = Eez. etA) =T The role of this function in the change detection problem is as important as that of the power function in statistical hypothesis test. 3. INDEPENDENT SEQUENCE, SCALAR CASE. Let (YK)1~K£tbe a random independent sequence.
P%
2. CRITERIA
3.1 . Let be a priori known probability distribution of a random c hange time -to
We start with classification concerning the assumptions about an unknown change time
PJr
to
2.1. Let PJr be a priori known probability distribution of a random change time to(;#, where AI=(Q,1,2r ,,). In the on-line framework, the criterion of optimality is to minimize the mean time delay (Shiryaev,1961,1963)
E7[ (t(;.-
+'0+1. /-t:c...";;to)
({;o
='
I'l.. / 6o~o) =
(1)
.
/
or
where h is a threshold.
t
a .:: -to) >,.
T
1r:~ (
(5)
f
3.2. Let io be a non-random unknown value, or random value with unknown a priori probability distribution. Other conditions hold such as 3.1. Page (1954) proposed a cumulative sum algorithm based upon the SPRT : t:a. = Lhf it : ~f ~ ~ ].J !li = (9i--t+.t;r :~}~~~) +~ (x) t mQx(o,A:j 6) Lorden (1971), Moustakides (1986) proved optimal properties CSA from the "worst case" criterion point of view. There are many others well known suboptimal algorithms for solving problem 3.2. All this algorithms are based upon a very important concept in mathematical statistics, namely the likelihood ratio (LR). From the general view point the designing change detection algorithm includes two problems: - derivation of a sufficient statistics (SS) ; - designing of stopping rules.
(2a)
where cl.: 0 ~ '" < 1.. and Tare given constants. Other possible criterion in this case is to minimize the risk ~
J(~tA) = /}r (tt< <: to)+ C E;,-(tA-to+J..!tc.. Ho)&r(ta. ~~) where c >O is a given constant. 2.2. Let io be a non-random unknown value. In some applications, it is useful to consider that to is a random value with unknown a priori probability distribution, or to is a non-random unknown value. In this case it is very useful to have algorithms which are free from distribution of change time to . For this reason, the criterion of optimality is to minimize the "worst case" mean time delay(Lorden,19711
r~~ stLfJ
II-:t.
f-71f
ta=/../lht: 9t-:,-n} 7 ii: = tn.(a. + ejt--f)-~(f-a)fth
(2 )
/
ft.-a.) a..
Till the time to-1. included observations density is I}{jt;}and then (under t ~t,,) P2.{n). Shiryaev (1961,1963) proposed an optimal algorithm based upon calculation of the a posteriori probability Jr~ revealing a change time has occured . The stopping time ta.. has the form (a posteriori probability replaced by other function for simplicity it=t},..2!1:.....):
in the class of algorithms for which the following condition is fulfilled
EJT (t 0.
(3a)
-to~::t
e.z, .
r=
SI.<.fJ
/; --(
eJSs¥F(fa.-t.+1/tlA~toJ!I/)(3)
-to ~ 1-
in the class of algorithms for which the
12
ss:
LR is a
SI:
4.2 . Let &2- be an unknown vector. Other conditions hold such as 4.1. There are different possibly approaches to this problem. The GLR is the most general algorithm: . -to.. = i t : ~t ~ ~}, t /J Q tna:>e. S(..(.P L Ut . 10) d1: - {$K~ t IB.z-B.d ~ £ ,':K Pt;1(!ldY,!:; It is obvious that the GLR is generalization of the CSA under the unknown vector B.t.. Because of the double maximization, this algorithm is very time consuming. The number of each step calculations tends to infinity fast with t tend to infinity, because it is necessary to estimate Su.p under each possible change time k from 1 to t. For this reason, immediate use of GLR for practical implementation is not obvious. Let us first investigate some possible simplification of the GLR. There are two possible approachs: - eliminate first maximization
= f1
Let EJ.. and Ez denote expectations of random variables under the two distributions characterized by Bt(before change) and 82 (after canhge) respectively, then: E-f(SI:)LO ~
"I {
t"2(St)?O.
A decision function 9t of the Bayesian algorithm (5) based upon "weighting" a priori information about a change time and ss (S")'HI 0 and decision function is growing. The final cycle of the SPRT ends with reaching the upper boundary. For our further discussion it is very useful to rewrite CSA (6) in other form
it:
t -&.. 3b- /:::t~'':K
!zf5't) Ptf!/t/
fhax.
i_
(7)
t
/J
~ "Ut.
~.z (~,)
it =
(8)
e
4.1. Let (YK)(5K5f: be random sequence (scalar or vector) with a conditional density PO(!I.fIJ~-(), where (j is a vector: Be Till the time tg-i included (;=B1 and then 0=82 , Other conditions hold such as 3.2. In this case the generalized CSA has the form
QI: 11
= (s t
t-~ +i
(e.fJ
st(P.fJ ()L) = -&.
~
J) (:t;' / ~
A,
I(AJ
jr@
"
y,
'it - p U1
f{}z (!IdJt'-~;)rJ".
fJ&~ (!fdJt~~)
P&d!idyr;)
fBt(!fd!f,,-p,/
tJ ,~ fl:/J
(!( \
6. LOCAL APPROACH.
:-f)
Pth (~: / y/~-0 where
f..J, - -
(J2.))~ Pf}.z
(11 )
(17, /YL' -~)
p(}.f
w/:: t W(.·, ,_ (n (i~/1 ~ '..1) /J
«It. .
if ~
1.
, where ~ is maximum likelihood (ML) estimate of the vector B.z. , using a sliding time window [t-m+1;ml, m is a fixed length of this time window. It can be said that decision function (11) computes a distance between two models. The first model is characterized by B1~ and the other one is characterized by ~ . Basseville and Benveniste (1983b), Basseville (1986) proposed other possible distance. They recognized that LR can not be convenient for some application (for example, speech segmentation) ,because the behaviour of decision function (11) is not symmetric for the detection parameter jumps from Bf to {)2 and conversely. From the point of statistical decision theory this non-symmetric behaviour is normal, but it is disadvantageous for speech segmentation. For this reason, it is iseful to replace LR by:
From practical point of view, algorithm 3.1- 3.3 have restricted applications to control Dheory and identification theory because the random sequence (¥K)UI'!tis independent and is a scalar value.
(-i :
Id ~ f1:Jz.cY~·l'1f;)
(.' =t:-/11+i
4. MORE COMPLEX MODEL.
(.'/1/
)
Basseville and Benveniste (1983b) and also Appel and Brandt (1983) proposed this approach. The main idea of two-model approach is to replace (10) by: -ea. = ~·k.f ( " " ~~ ~
f~.t:~t: /r92.-Bd~f t:K Pe.dj/) where [(/')'7'0 in a convenient manner. It becomes obvious from comparison decision function (7) and (8), that the Lorden's algorithm is the generalization of the CSA for case of the unknown value &2.' Generalized likelihood ratio (GLR) algorithm (8) has been recently studied for a more complex model by Willsky(1976).
-to. ':
(
5. TWO-MODEL APPROACH.
Till the time to-i included {j:BLand then {;=&2., where (12 is unknown value. Other coditions hold such as 3.2. Lorden (1971) proposed an algorithm: i:(),= '-111ft.' It ... hJ ~ fhQX.tt.r'
/;
and replaced GLR by "two-model" approach; - eliminate the second maximization I"~~(/~C( ) by using "local" approach.
3.3. Let (~K).HKSi:: be independent random sequence. Assume the distribution density belonging to a Koopman- Darmois famil y Pe(~)= eXf (BT(1) - ~(B)).
91: =
PGi!ld!l.tP.J
If vector eland &.zare inaccurately known we are to consider the local hypotheses case / e-f- BL / ~ 0 • For example, this is a problem of signal detection with a very low signal noise ratio (SNR). It is obvious that for a fixed size jump of the parameter a local algorithm is not optimal, but this approach has two main advantages: - change detection algorithm is very simple (low computational cost) and
~
is an indicator of the event
fit is a counter of an observation
it.
number after the last nulling Bansal and Papantoni-Kazakos(1986) proved optimal properties under some model.
13
calculation ARL function is also more simple; - local hypotheses case is the "worst" change detection case and this is useful to optimize the algorithm for this purpose. The ideas of the effective increments, the local tests are very powerful and fruitful in statistical decision theory and estimation theory. This ideas were founded by Fisher, Le Cam, Roussas (1972), Ibragimov and Khasminskii (1972), Davies (1973) and other investigators. They derived very deep and precise mathematical results in this theory. Let us discuss the key point of this theory, asymptotic expansion of the LR. This expansion is possible, for example, for the p-dependent Markov sequence (AR-model) and for ARMA-model under Gaussian assumption. We write this expansion for p-dependent Markov sequence
S/((),
f)+
~
Sf; ~
n
i=1
ift.}:: tl"'>
If
']=(&))
t
L.
-to ,(t)=&~)
t ~i:DJ({}=Bf+r9).
9i: ~ A~-,
n1i
F
(.1.. z'
-1
T
t:
~ 7(fJ.,)lIt.1 Vt:
;d It ;/, ) ~
(14 )
vt-1 J {Jt_i7 Q }+Xi,
(15)
is a known matrix (m x n) , I'l'v>/'t) ~k().()= ~ is a known system output, Xt is a unknown system input, measurement noise ~c is a sequence of a Gaussian vector:
for t <: to for 1: ~-eo In this case the decision function and stopping rule of CSA algorithm (6),(9) has the form
~"':f{t ·
=
/J
u.,oi.
Ye- = HIt + Si- r I{t~t:O)" > where tt~R.h, Ytf:./{M, '5tGR~.1fR; H
e
= Cji-1 +::.c.[c)+ . 6.2. Let vector e~ before the change time to be known exactly, and after be described by the inequality (ez-&.ff 7{B.()(&2-B.()~/I/- .
cu.dO.z:
The analytical redundancy approach is well-known in the automatic control theory. Basically this method is widely used for failure detection and diagnosis. Willsky (1976), Mironovski (1980), Frank (1990) described this problem in detailes. The aim of this paragraph is to demonstrate and to compare the traditional analytical redundancy approach (see, Frank, 1990) and the unified statistical approach, based upon a likelihood ratio. Let us dis c ussed one simple example. Consider a linear system given by
eo
-ea. =
r
7. CHANGE DETECTION AND ANALYTICAL REDUNDANCY.
6.1. Let vectore be described by the equation B= 1" Jc.. ,where a vector C:lcl_~ determines the direction of the change of the vector ,CGR.'7.., and j) is the s c alar value:
fit
forf~to
where oFL(c,x) is a generalized hypergeometric function.
which is a Gaussian vector with the known covariance matrix, zero mean before the change time and non zero mean after the change time: N(:rt8)["& > 7'{9-))
to
l
Sic: == - i 2~l +
where .L3L· is a vector of effective increments, l!;<'fR'2., .r{B) is the Fisher information matrix (r x r). Let us consider again the change detection problem in a local case, Introduce the random vector of effective increments which are the main parts of the asymptotically sufficient statistics (ASS)
iNtO>
for t..(
J
r
~ 0 t-".?O
1:. ~""
00
t -1; (rt rtiJ'}:,l)
((;2.-iJ(J J/9.,)fG~-&.,):./I; /'Z -tilt .f I . where IT) T J I./ 1S noncentral ~Z-distriution with r degree of fredom, tAZis a noncentral paramet e r. Using this expression for the density of the ~2-distribution, the CSA decision function can be written in following way -t-a.:: /......! it .' 9i- ~ R} > It:: (Si.) +.1
Vector Ai and positive definite symmetric matrix rIB) can be written under some conditions in the following way
£{X-t} ~
jrJr (Pz{@)urS'l=i.) ,_j
7
Pp.,u/);
Asymptotically, under local condition
<~f
09
r
'l.
h where·lJ~.rg)is the weight function, Stz. is ellipsoid surface separating the indifference zone from the zone of a cceptance of the hypoteses "change occur". Using expansion (12), we arrive at a conclusion that in this case the ASS is assigned by t t . " f T 7-'1 J t IJ" = L X, fl:: ~ 4..{ vI {&.( ~1
df)I;)-;::/ "[dB,,- ; reIFt&)of)t > (12)
o Of, = -
f lJ?.J8-)Pe.(y/)dS't
p (It.
nS+)
=0
>
E
(';t-~{
) ". bZI.
Till the time -eo-1. included no change occurs (i.e . I[t-~t.j).1=OUnder t<:t.) and then change oc c urs (i. e. Ilf~ to} A =,1 underf~t-o). Model (15) is a typical model of a measurement system. The term I{O;io}1:J is needed for sensors failure modelling. It is necessary to detect a situation when sensors failure occurs by means of the observations (!lK)f~K~~'
(13 )
7.1. Traditional analytical redundancy approach. Failure detection systems are based on the use of analytical redundancy relations or parity checks: tt=TTY-t) (16)
In this case, the CSA synthesis requaires the use of the Wald's weight function method, i.e., instead of the LR sl we use its analog calculated by taking into account the "weighting" of all possible directions:
s,t
where T(m;«m-,.,;)is a special matrix: rTH
=0
j
'T' r". ::
I
.
It is obvious, that these conditions
14
X
eliminate unknown input signal t . In this case (16) can be rewritten as
Et =
TT5t
+- I{i->.-t:o}
so that
T
1/(O>b 2 I)
.;({EtJ=
/II(7''A.>~'I)
(
is possible to derive the final criterio n of optimality (Taylor, 1968). The criterion for CC is to maximize a total mean profit of the common operation of technological process and quality control system. From this point of view, (1) - (2) and (3) - (4) are auxiliry criteria.
Ll fo'!..
;lo'l.
tLbo
t~
'to
8.1.1. Example. Let(ff/tC)1~"-~tbe a Gaussian AR random sequence. Let mean mJ.. of the AR-model be a target value. It is necessary to detect situation when a mean change occurs. Usually there are two "taboo" reg ions: mz:= rn.-( ± 2. /( , where k is a reference value. Two-sided CSA (9) can be rewritten as follows -co-=t..kf{t: C3f,,:h)VC9t'd)], +
These artifical measurement are usually called "residuals". Residuals are designed to reflect a possible failure in analyzed system (15) . Change detection algorithm can be designed by means of these residuals. 7.2. Likelihood ratio approach. In this case LR can be rewritten as follows
S f
t
-&. fo, {Y.: / f.. (1l2 ))
1
=
,
fl
(
X
)
9" =[ji---f
=1
Xi
st_ x..-t 1 -,
T ( 7';1
,=1-
-
where -if), matrix
6"2.
•.)
t
TTy. i
_ r i Tl ) T~/i ., / ' - (1Ir
"'-
t
~
··, /1/-
where i!L!Ii, =!/t-i. is shift operator, 0(,1''''10I/, are autoregressive coefficients.
8.2. Early detection of failure. The continuous normal operation of the manufacturing process is a very important problem. An abnormal situation must be detected very fast in order to avoid industrial accident. It is important in this case to use information of different sensors (inputs and outputs of plants) together. Consider a regression model
17 )
rn-I/. are eigenvectors of the
p= I -
H(HTH)-iHT
!le
These eigenvectors correspond to unit y eigenvalues. It becomes obvious from comparison (16) and (17), the residuals (16) are a convenient statistics for algorithm from statistical point of view. There are different possible algorithms for solving this problem. It can be GLR or CSA. For example, in this case CSA (14) can be rewritten as follows
:i~~iY./ Jt'" (ft)+.1 III .en.01.F (fIZ-n , nt )j-x f) "2 + 24 '(18)
= - Y1
:Xi: ~
VeTVI,
' Vt
= vt-i. F[9t_1'ny -+ 1ft
= eTXi +l1.t,
where Xt is a known input, Yt-is a known output) X(;61<.7 8 is a parameters vector $eR.1!:. ,!Lt is a Gaussian whi te noise: E(n(;)",0, E(fll)=";n2. Normal operation of the manufactur ing process is characterized by the vector81 . It is necessary to detect a situation when a parameters vector jumps from Bito 82. The direction of th e jump is unknown. GLR or CSA (14) are convient in this case. For exa mple CSA (14) can be rewritten as follows
f:o.::t;' k.f[t
St
+(1-"'1'2 - ...-"'t~."){/Ir/11.,-K) '7 ,
it =[ ft-I - (1-"'fr- ... -olf~I)(!I~-1>I1+iOJ:
J
r8-( Yt: I X, ' (f}-f) where (/(= 0, B:z. = ;1 , (6;) is a least squares (LS) estimate of an unknown input signal )',' under assumption that no change occurs {@f= ()) or c hange occurs (et="')' It is possible to prove, that in this case LR can be written as follows L
.
= Vt--t I {9l_1>O) +Xt f t f t = :/t- B/Xt.J '.hCf):o :,.2. R)(x, /(Xy. ~ EO"Xl) . The computation of ta, Si: and;Xi hold Vi;
8. QUALITY CONTROL AND PERFORMANCE MONITORING.
1
such as (14). Another important problem is robust change detection algoritms with respect to nuisance parameters. Let(~/tC)(~JC,~be a random sequence with the conditional densi ty P(!J
Typical features of modern technological process and plants are complexity, high capacity, high risk for environment. Change detection a lgorithms can be used for performance monitoring of the technological process and for quality control.
e
e
8.1. Statistical quality contro l. In fact, it was first implementation of change detection algorithms, which were given the name of control charts (CC). Consider a technological process which has two possible conditions: "in control" (no c han ge occurs) and "out of control" (change occur). Let ('f/tC)1~JC~'t be a sequence of the observations of the performance index with the conditional distribution densi ty Pe (Y~Nt.,--f). These technological process conditions are characterized by parameters vector e~ (in contro l) and 9z (out of contro l). It is necessary to detect a situation when a process is out of control . In the simplest case a parameter 0 is a mean or variance. In quality contro l problem it
High reliability measurement systems are an essential part of the modern plants or movin g objects (ship, aircraft, ... ). Sensors failure detection is a very important problem for high d egree reliability in control data. The problem of the "soft" failures detection, which
15
are not detected by the built-in test equipment, can be solved by using redundancy.
arrival time includes two subproblems: - sequential (on-line) detection of P(S-) wave and preliminary estimation the time at which P- (S-) wave arrives; - a posteriori (off-line) estimation the time at which P- (S-) wave arrives and identification of the wave's type by using fixed size seismic data sample.
9.1. Inertial navigation system (INS) monitoring (Potter and Suman, 1986; Varavva, Kireichikov and Nikiforov, 1985, 1988; Kireichikov, Mangushev and Nikiforov, 1990). A simplified INS error model involves linear differential equations. In discrete time the dynamics of errors takes the form of a state space model
{ where
XI;
10.1 Sequential detection of P- (S-) wave. Early investigators derived the model for seismi c noise, P- wave and Swave. It was proved that AR-model is convenient for this purpose. We assume that the arrival of a seismic wave or a change of type of seismic wave results in changes in spectral and polarization properties of a s i gnal. For example, P-wave has a linear polarization in a source- receiver direction . For this reason the model of signal can be written in the following way
= F(i; tf, f)Xt-1 + ~i: +Itl;~tc} A
Yt .: !tU)lt
+VI;
(19)
,x"fR."- 'It fR."', S/;~R.~ Vi,fR."'; Si-.>
Vt
are independent Gaussian zero mean sequence of random vectors. The term J{~~~.~ A is needed for sensors failure modelling. INS output measurements involve useful signal and measurement noise :
at
r-t
LIt = Zt
Zi
;Vt =[
y"
+
Xt
Yt
$.. 1 [_o..(r./(:1 /. l
y{ . (l11.
+
Y
R-'.f",..] QOL
00,',
J
Ps At 'Ni-L' +IIf) Zi: (3t';{t-t' + ft,
r
(20)
L=1
where Zt is a three- component seismic signal, t f is a time at which the Pwave arrives, IVt is a three component seismic noise, A", .. ., Ap,. are seismic noise autoregressive matrices, :::et is a one- component P-wave, (31,··~jJPs are a Pwave autoregressive coefficients, H is a matrix (3 x 1) which depends on a sourcereceiver direction. The matrix H is a priori unknown. For this reason, immediate use of model (20) for practical implementation is not obvious. It is simpler to use one- component change detection algorithm. In this case we assume that the arrival of the P-wave results in a change in autoregressive coefficients and variance. The CSA decision function (13) can be rewritten in the following way
= Ilfl; - tiN = >It - Yzt: It can be proved (Willsky, 1976) that LR can be written in the following way 'f- 2:
+ I (t >, t: PJ H X t
t':{
It is possible to eliminate the useful signal by using a pair of these INS :
S}= "'
= Ifi PIr
,
/."=1
where oot. and If",' are the measurement innovations processes of Kalman filters under hypothesis ho(no change occur) and hi (change occur), Ri is a covarianc e matrix of the innovations. Willsky (1976, 1986) described GLR for this case. CSA (14) are convient for this case too. The computation of +'a.., ~~ and it hold such as (14), vector Vi and ma tr ix 7(6) can be wr i t ten
fa.: CIt/rt :!Jt>,h} "Jl=
!It = I~~ U'~-1 Et ) (f (!L -1 \ G'£ GE2 '/
9.2. Redundant inertial sensors unit monitoring (Potter and Suman, 1986; Kireichikov, Mangushev and Nikiforov, 1990). The primary goal of the redundant inertial sensors unit is tolerant with respect to sensors failures. The errors model (without failure and with failure) of the array of five gyros can be written by using equation (15) under n=3, m=5. There are two possible approaches: - to use robust ("worst case") estimate; - to use failure detection and fault diagnosis algorithms. The first subproblem of the second approach is the sequential change detection. The change dete c tion algorithms in this case was described in paragraph 7.
j
l/l _/'
($H
+!/[c)~
(~t-f)
>
(
21 )
{ Z i -I'
Ft = 2-t - a.rU~-i > a T= (oI.("'''J 011') j where oi1) .. .,oIl'are AR- model coefficients, zf is one compone nt of a threecomponent seismic signal, ~~ is a residual variance, C is a vector of the assumed direction of the parameter jumps, et eP+J.. Let us remark that ARmodel for CSA (21) is not convenient model for seismic noise or for P-wave. Roughly speaking, this is an intermediate model. On the other hand, parameters a,GlZ.and C defined the separated surface equation in the parametric space. The computation of the preliminary estimation the time at whi c h P-wave arrives is based upon a simplified maximum likelihood (SML) algorithm. Let(rJ(.)"$K~tc..be a fixed size sample (see problem statement 1.3). It is necessary to estimate an unknown time ~p . Assume that 91 and Bz known vectors. In this c ase likelihood function has form K-( I ' ~) t't " 1)
10. AUTOMATIC ON-LINE SEISMIC PROCESSING. One of the main tasks of the on-line digital seismic data processing is detection and estimation of the arrival time of the seismic waves (Nikiforo v and Tikhonov, 1986; Nikiforov, Tikhono v and Mikhailova, 1989). It is possible to estimate the source- receiver distance by using delay between P-wave and S-wave arrival times. The general problem of the detection and estimation of the
FK
:
L. ~ ~ (2dz,·_~ +? Pe/ZL .J2 , ·:P i=' ( l=K
under assumption that k is the time at which P-wave arrives. SML-estimate is
16
based upon maximization of the
generalize CSA for a complex signal: i f' rr -'( "') ,,I 0/,,1 + 2...
11: =
nt
Vt ~
""
Vt
s"Et
) Vt;= vi-,.1.(9f-170YC/t.
c-/:=':It -S[ti. ) ']=(I'n)~ CM. z'E(slST)
(22)
t:
where (A)~ is a complex conjugate with respect to A, (A)f~ {(A)"'£". The computation of t" and Si: hold such as (14) under r =
Let us return to CSA decision function, which can be rewritten in the following
2e.
(23 ) It follows from comparison (22),(23) and decision function (9), that SML- estimate can be computed by using standard CSA. It is possible to prove that algorithms (9) and (13) are equivalent in some sense (Nikiforov, 1983), for this reason A.
REFERENCES Appel U. and Brandt A.V. (1983). Adaptive sequential segmentation of piecewise stationary time series. Information Sciences, vol.29.
-ta.
-hia. + i. In practice, the seismic signal ARmodels before and after P-wave arrival are unknown and very nonstationary. For this reason precision of SML- estimate is not high, but sufficient for choosing a center of the fixed size time window. A posteriori estimation the time t f realized by using this time window. tpSML::'
Bansal R.K. and Papantoni-Kazakos P. (1986). An algorithm for detecting a change in a Stochastic Process. IEEE Trans. on Inform. Theory, vol.IT-32, no 2, pp.227-235. Basseville M. (1986). The two-model approach for the on-line detection of changes A.-R. processes. Im the book "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, p. 169215.
11. IMPROVEMENT OF THE TRACKING CAPABILITY OF THE ADAPTIVE ALGORITHM Improvement of the tracking capability of the adaptive algorithm is an importent problem in identifying timevarying linear system parameters. Assume that 6(1:) is a time- varying parameters vector is subjected to random jumps (Perriot- Mathona, 1984; Wahnon and Berman, 1990). This is an important problem for the adaptive LS algorithm in the adaptive arrays theory ( Monzingo and Miller, 1980). The complex random process is the natural domain for narrowband electromagnetic signals (radio and radar). Array signal processing based upon adaptive filter contains adjustable weight coefficients:
!it ==
..AT
m..(t-)
Sf +
ht)
Basseville M. (1988). Detecting changes in signals and systems - A surey. Automatica, vol.24, pp.309-326. Basseville M. and Benveniste A. (1983). Design and comparative study of some sequential jump detection algorithms for digital signals. IEEE Trans. Acoustics" Speech, Signal Processing, vol.ASSP-31, no 3, pp.521-535. Basseville M. and Benveniste A. (1983). Sequential detection of abrupt changes in spectral characteristics of digital signals. IEEE Trans. Information Theory, vol.IT-20, no 5, pp.709-723.
(24 )
1ft is an output complex signal, Y.t EC
If,'
=
~ t-~'
Basseville M. and Benveniste A., (Ed.). (1986). Detection of Abrupt Changes in Signals and Dynamical Systems. Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York. Basseville M., Benveniste A. and Moustakides G. (1986). The local method applied to the robust detection of changes in the poles of a pole-zero system. in Detection of Abrupt Changes in Signals and Dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, p. 259274.
) L': ".J I: )
where Il is a smoothing constant O.{.A,:: 1. . The "effective memory" N of this algorithm has the form )lz _-1_ 1- '}.
Let us consider one possible solution of this problem. Assume that ;l = Il opf ' where ~OHis an optimum value of the smoothing constant for "slow variation" of the true vector m(t). Detection of abrupt changes based upon CSA (14). In this case e~ is equal to adaptive LS estimate ~(t). Let us discuss a typical cycle. Till the stopping time -to. we use A=).opt. After the stopping time ia. it is necessary to reduce?: i\ ~ > N(). hta. and then to increase.?l step-by-step , from A"',1t to /lopt::. It is very simple to
J.,..,,,
Davies R.B. (1973). Asymptotic inference in stationary gaussian time series. Advances Applied Probability, vol.5, no 3, pp.469-497. Dragalin V. (1988). Asymptotic solutions in detecting a change in distribution under an unknown parameter. Statistical problems of control, Issue 83, Vilnius. pp. 45-52.
"',,,)=
17
Nikiforov I.V. (1983). Sequential detection of abrupt changes in time series properties. Naouka, Moscow. (in Russian).
Deshayes J.,Picard D.(1986). Off-line statistical analysis of change-point models using non parametrical and likelihood methods. in "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, pp. 216-258.
Nikiforov I.V. (1986). Sequential detection of changes in stochastic systems.in "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, pp. 216-258.
Frank P.M. (1990). Fault diagnosis in dynamic systems using analytical and knowledge based redundancy - A survey and new results. Automatica, vol.26, pp.459-474.
Nikiforov I.V. and Tikhonov I.N. (1986). Application of change detection theory to seismic signal processing. In "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, p. 355373.
Gibra I.N . (1975). Recent developments in control charts techniques. Jal of Quality Technology, vol.7, pp.183-192. Girshick M.A. and Rubin H. (1952). A Bayes approach to a quality control model. Annals Mathematical Statistics, vol.23,pp.114-125.
Nikiforov I.V., Tikhonov I.N. and Mikhailova T.G. (1989). Automatic on-line processing of seismic station data - Theory and applications. Far Eastern Dept. of Academy of Sciences, Vladivostok, USSR (in Russian).
Himmelblau D.M. (1978). Fault detection and diagnosis in chemical and petrochemical processes. Chemical Engineering Monographs, vol.8, Elsevier, Amsterdam.
Page E.S. (1954). Continuous inspection schemes. Biometrika, vol.41, pp.100-115.
Ibragimov I.A . and Khasminski R.Z. (1972) Asymptotic behavior of statistical estimators in the smooth case. I: study of the likelihood ratio. Theory of probab. and Allp., vol.17, no 3, pp. 445-462
Perriot-Mathona D.M. (1984) Improvements in the application of stochastic estimation algorithms- parameter jump detection. IEEE Trans. Autom. Control, vol. AC-29 p. 962- 969.
Kireichikov V., Mangushev V. and Nikiforov I. (1990). Investigation and application of CUSUM algorithms to monitoring of sensors. Third All-Union Seminar on Detection of Change in Random Processes. in book "Statistical problems of control", Vilnius, Issue 89, p. 124-130 (in Russian).
Phillips M.J. (1969). A survey of sampling procedures for continuous production. Jal Royal Statistical Society, vol.A-132, no 2, pp.205-228. Pollak M. (1987). Average run lengths of an optimal method of detecting a change in distribution. Annals Statistics, vol.15, pp.749-779.
Kligiene N. and Telksnys L. (1982). Methods of detecting instants of change of random process properties. Automation and Remote Control, vol.44, no 10, part.II, pp.1241-1283.
Pollak M. and Siegmund D. (1985). A diffusion process and ist application to detecting a change in the drift of a Brownian motion. Biometrika, vol.72, pp.267-280.
Lorden G. (1971). Procedures for reacting to a change in distribution. Annals Math. Statistics, vol.26, no 1, pp.7-22.
Potter J.E. and Suman M.C. (1986). Extension of the midvalue selection for redundancy management of inertial sensors. Journal Guidance, vol.9, no 1, p. 37-44.
Mironovski L.A. (1980). Functional diagnosis of dynamic sistems - A survey. Automation and Remote Control, vol.41, pp.1122-1143. Monzingo R.A. and Miller T.W. (1980) Introduction to adaptive arrays. New York: John Willey.
Roussas G.G. (1972). Contiguity of probability measures, some applications in statistics. Cambridge University Press, Mass.
Nikiforov I.V. (1978). A statistical method for detecting the time at which the sensor properties change: Preprint IMEKO Symp. on Application of Statist. Meth. in Measurement. Leningrad 1978, p.1-7.
Shiryaev A.N. (1961). The problem of the most rapid detection of a disturbance in a stationary process. Soviet Math. Dokl., no 2, pp.795-799. Shiryaev A.N. (1963). On optimum methods in quikest detection problems. Theory of Probab. and its Appl., vol.8, no.1 pp.22- 46.
Nikiforov I.V. (1979). Cumulative sums for detection of changes in random process characteristics. Automation and Remote Control, vol.40, no 2, part 1, pp.48-58 .
Shiryaev A.N. (1978). Optimal Stopping Rules. Springer-Verlag, New York.
Nikiforov I.V. (1980) . Modification and analysis of the cumulative sum procedure. Automation and Remote Control, vol.41, no 9, pp.74-80.
Taylor H.M. (1968) The economic design of cumulative sum control charts. Technometrics, vol.10, no 3, p. 479-488.
18
Van Dobben De Bruyn D.S. (1968). Cumulative sum tests: Theory and practice. Hafner Publishing Co., New York. Varavva V., Kireichikov V. and Nikiforov I. (1985) Detection of sensors failure as a problem of detecting the change in random sequence properties. Abstracts of Second All-Union Conference on Modern Methods of Planning and Experiments Analysis in Investigating Random Fields and Processes, Sevastopol, part 1, p. 18-20. Varavva V., Kireichikov V. and Nikiforov I. (1988). Appl ica t ion of change detection algorithms to monitoring of measurement systems of moving object. Second All-Union Seminar on Detection of Change in Random Processes. in book "Statistical problems of control", Vilnius, Issue 83, p. 169-174 (in Russian) . Wahnon E. and Berman N. (1990) Tracking algorithm designed by the local asymptotic approach. IEEE Trans. Autom. Control, vol. AC-35, no 4, p.440- 443. Willsky A.S. (1976). A survey of design methods for failure detection in dynamic systems. Automatica, vol.12, pp.601-611. Willsky A.S.(1986). Detection of abrupt changes in dynamic systems. in "Detection of abrupt changes in signals and dynamical systems" (M.Basseville, A.Benveniste eds.) Lecture Notes in Control and Information Sciences, LNCIS 77, Springer-Verlag, New York, pp. 27-49.
1.s . . . . . . .
-1.5.
o
200
400
600
800
1.000
.1200
1.400
Fig. l(a). Gaussian vector random sequence (XK).I~ J(~i ,X~&R5 Change time t. =1000. Signal/noise ratio SNR = 0.2
1.0.
l
8.
.~
6.
I· · !~
4. 2. 0.
L.J..i~."",,·l
0
290
490
Fig. l(b)
690
890
1.000
Desicion function of the CSA.
19
I
I
1.2'00
1.400