Recognition of generalized markoff processes and application to electrocardiography

INFORMA

TION SCIENCES

51

14, 5 l-56 (1978)

Recognition of Generdizd Markoff l’mwses and Application to ElectrocardiographY* JOHN M. RICHARDSON,

VRUDHULA

K. MURTHY

and L. JULIAN HAYWOOD Los Angeles Gnu@-Uniwrsity of Southern CaIifomia Medical Center, 1200 North State Street, Los Angeles, California 90033

ABSTRACT The observed sequence of electrocardiographic R-R intervals have been analyzed by appropriate linear and nonlinear Markoff processes. Recent investigtions by the authors have shown that appropriate comprehensive descriptions of the models are provided only by generalized or multi-step Markoff processes. In this paper the problem of recofgnizing sequences of states in a generalized Markoff process model is discussed and solutions provided.

1. INTRODUCTION It is assumed by a number of investigators that different categories of anomalous ECGs have sequences of properties (e.g. R-R intervals) that are described by appropriate Markoff processes. Recent investigations [1, 21 have shown that adequate descriptions in many cases are provided only by generalized, or multi-step, Markoff processes. In the following discussion we treat the problem of recognizing sequences of states when each category is assumed to be characterized by a generalized Markoff process. 2.

STATEMENT AND GENERAL SOLUTION OF THE PROBLEM

Let us assume that we are given q categories of which the cth category is defined by the generalized Markoff process for the states x,,: ~“+,=f,(x,,...,x,-p~+*)+m,

(1.1)

*Requests for reprints: L. Julian Haywood, M.D., Box 305, 1200 North State Street, Los Angeles, California 90033. 0020-0255/78/0014-0051/$01.25

0

Elsevier North-Holland,

Inc., 1978

RICHARDSON ET AL.

52 where the r,, are Gaussian random variables with Er, = 0,

Er,,r,,,= c&,,,

(1.2)

where S,,, is the Kronecker S-function such that ~WPl=1 =o

for

n-m

otherwise.

To complete our description we assume that the a priori probability of the existence of category c is P(c). Letusnow assume that a certain sequence of numbers 2 ,, f 2,, . , ,Zmis actually observed. To which category should we assign the observed sequence? Using the familiar (0,l) loss function (a penalty of 0 for a correct decision and a penalty of 1 for an incorrect decision), we obtain the well-known decision rule: the best choice of c is the one for which 2,) is a maximum. Thus the remaining work is the calculation of P(CI&,..., the a posteriori probability P (cl&, . . . ,Z,), i.e., the probability of c conditioned by the requirement that lx, - &:,i< E, n = 1,. . . , N, where E is made arbitrarily small. The statement of our problem is still not complete. However, we have not yet defined the initial conditions for the generalized Markoff process (1.1). Before attacking this question directly, let us first consider a generalized Markoff process restated in terms of probability densities. Starting at the tune m, the probability density of the sequence of states terminating at a later time n (R > m) can be written

= P(xnlxn-

*,...,x,-p,,c)P(x,-,,...,x,jc).

(1.3)

The result on the first line comes, of course, from the definition of conditional probability densities. That on the second line is an expression of the fact that the probability density of x, given all past x’s actually depends, according to the assumed generalized Markoff process (1.1) for category c, only upon the previousp, x’s Clearly, we must require that n - m >pc. For the purpose of the present discussion we have chosen a notational convention according to which the x’s in the sequence of increasing time are ordered from right to left. Equation (1.3) is a recursion relation which on successive iteration yields the end result ptxZ8,*.*7xP?lIc)= fi p(xilxi-,,***,xj-p,,C)P(Xp c+m-*,..*,X,/C)* i=p,--nr

(1.4)

RECOGNITION

OF GENERALIZED

Now we are in a position number of alternatives;

to discuss initial

conditions,

of which there are a

(a) Use the first pC observed values _Cr,.. . ,i_c as the initial (1.1) with n >pC. In terms applicable to (1.4) we set

conditions

for

m= 1,

n=N,

f&,...,

53

MARKOFF PROCESSES

(1.5)

x,lc)= fi s(xi-fi), i=l

where S is the Dirac S-function defined as a generalized function. (b) Use an arbitrary distribution of pC successive values of the x’s occurring prior to observation interval (1, N). Specifically we assume: n=N,

m< -pc+l,

fyxm+,c-,,...,

x, 1c)

(14

arbitrary.

(c) In (b) let m-+ - 00.

3. SOLUTION FOR ALTERNATIVE

(a)

When we assume that the initial conditions for the generalized process (1.1) in Sec. 2 is x, = Zi, i = 1,. . . ,pc, we obtain for N >pc logP(x,

,..., Xp,+rl&...,

Markoff

a,,,) = - f(N-p,)log(2n$)

+[xp,+l-fc(~~~....1~1)]2). (2.‘) The a posteriori probability of the membership in category c is then given by logP(cl&

,..., I,)=logP&

of the observed signal &, . . . , 2,

,..., ZPC,,,&~ )...) &c)+logP(i&

-~(N-p,)log(2nu,2)+logP(~~~,...,~:,,,)+B,

,..., f,,c)+B

(2.2)

54

RICHARDSON ET AL.

where B is an additive constant (independent of c) to be determined by the normalization condition on c, and where P (2&. . . , I,,c) is the a priori joint probability density on &, . . . , 1, and probability on c. Writing

P&9...,

zI,c)=P(z&

,...) &,c)P(c),

(2.3)

it is clear that our remaining problem is the choice of P(.$,, . . . , i,lc), since P(c) is assumed to be known.

There are several courses of action open to us: First, we could assume that N is very large, in which case the choice of the set P (&,, . . . , Z11c), c = 1,. . . , q, does not matter. Second, we could assume that P (&,, . . . , .f,Ic) has evolved via the generalized Markoff process of category c from an initial probability density given at an earlier time. However, this case is the same as alternatives (a) and (b), to be discussed in the next section. The third possible course of action is to assume that the set P ($,, . . . ,Z,Ic), c = 1,. . . , q, is simply given without any further underlying assumptions. In our view, this is not satisfactory. In the first case (i.e., N large), it is appropriate to consider 1 -logP(cJ& N-p

,..., .Q=&N--

N-P, 2(N-F)

Wd

+ &,N,

(2.4)

where

(2.5)

9c,N= -

(2.6) where B is the normalizing factor for P(clZ,,,...,Z-,), and x hence has a component which depends linearly on the sample size N. The quantity ~7 is chosen to be some sort of average of the numbers pl,. . . ,p4. In determining the maximum of P(cl&,..., 5,) on c, we can ignore the term B/(N-p3 in &,N, since it is independent of c. As N+oc it is clear that +c,N+

-

1 lim 24 N-roe N-P,

L

5 [xi-f,(~i-,,...,%--p,)12 ;sp,+,

N-P,

2(N -P)

log27ru~+log27ru,z,

g+c,m,

(2.7)

(2.8)

REGOGNITION

OF GENERALIZED

55

MARKOFF PROCESSES

where C is a constant independent of c and, of course, N. Thus we obtain

(2.11)

An important practical question is how large N must be in order to obtain a satisfactory approximation to the infinite limit.

4. SOLUTION FOR ALTERNATIVES (b) and (c) Here we assume that the probability density is given for a sequence of states occurring prior ta the observation interval (1,N) as specified by Eq. (1.6). The probability density of the sequence x,, . . . , Xi for the case Of category C iS then given by P(XN,...,X,[C)=

pxo* -*JdX,p(XN,...,XmlC),

in which m < -pc + 1. The probab~ty density P(xv, . .-,x,&) (1.4). Let us assume the initial probability density

is given

P-1) by Eq.

where in the recursian relation

P,(x,,,+,, . . . ,xm,. rfc) and P,(x,,,+,,,- $,. . . ,.x&) have the same functional form (i.e., the first probability density is the same function of its arguments as the second is of its arguments). Let us further assume that the steady-state

56

RICHARDSON ET AL.

probability density is unique. In this case (1.1) reduces to

P(XN

,*..9

x,lc)=~dxo~~~~~x~-,+,P,(x, XT+,+,lc) ,a..>

if

N
if N=p,,

=P,(X,,...,X,JC)

xp,(~,?...J,lc)

if

N >pc.

(3.4)

The reader will note that the last expression of (3.3) is very similar [except for the absence of the factor P(c)] to Eq. (2.3). In fact the treatment of Sec. 3 could be employed to yield this expression if we had made the assumption that P(&..., Zllc) is steady-state. The probability of membership in category c of a particular observed sequence Zn,. . . , 17,is of course given P(C!l.&,...,

~,)=AP(&l)*‘.,

&Ic)P(c),

(3.5)

where A is a normalization constant independent of c, and where I,lc) is obtained from (3.4) by the substitution of &,...,?i for L

P(&...,

Xn,.a*, X,.

Alternative (c) involves assuming an arbitrary probability density of a sequence of pc successive states occurring in the infinitely remote past. If we require that the generalized Markoff process pertaining to each category has the property of an arbitrary probability density (belonging to a class of densities of reasonable behavior, which will not be defined here) which approaches a unique steady-state density by repeated apportion of the recursion relation of the type (3.3), then it follows that alternative (c) also yields the results given in (3.4). REFERENCES 1. L. J. Haywood, S. A. Saltzberg, V. K. Mm-thy, R. Huss, G. A. Harvey and Clinical use of R-R interval prediction for ECG monitoring: time series autoregressive models, J. Assoc. A&. Med. Zn#wn. 6 (2), 111-l 16 (Mar.-Apr. 2. J. Richardson, V. K. Murthy and L. J. Haywood, Nonlinear Markoff models cardiographic R-R interval analysis, M&a. Bioscj. (to be published). Received April 1977

R. JSalaba, analysis by 1972). for electro-