- Email: [email protected]
Stochastic Comparability and Dual Q-Functions
Journal of Mathematical Analysis and Applications 234, 482᎐499 Ž1999. Article ID jmaa.1999.6356, available online at http:rrwww.idealibrary.com on
Stochastic Comparability and Dual Q-Functions Hanjun Zhang Research Department, Changsha Railway Uni¨ ersity, Changsha, People’s Republic of China
and Anyue Chen School of Computing and Mathematical Sciences, Uni¨ ersity of Greenwich, London, United Kingdom Submitted by E. S. Lee Received October 5, 1998
In this paper we discuss the relationship and characterization of stochastic comparability, duality, and Feller᎐Reuter᎐Riley transition functions which are closely linked with each other for continuous time Markov chains. A necessary and sufficient condition for two Feller minimal transition functions to be stochastically comparable is given in terms of their density q-matrices only. Moreover, a necessary and sufficient condition under which a transition function is a dual for some stochastically monotone q-function is given in terms of, again, its density q-matrix. Finally, for a class of q-matrices, the necessary and sufficient condition for a transition function to be a Feller᎐Reuter᎐Riley transition function is also given. 䊚 1999 Academic Press
Key Words: stochastic comparability; duality; stochastic monotonicity; Feller᎐ Reuter᎐Riley transition functions; zero-exit; zero-entrance.
1. INTRODUCTION In this paper we discuss the characterization of stochastic comparability, dual q-functions, and the Feller᎐Reuter᎐Riley q-functions in terms of the q-matrices. It is shown that these key concepts have close links with each other. They are important concepts in the study of Markov processes, in particular, in the study of continuous time Markov chains ŽCTMC. and interacting particle systems. Good references on these topics are, among 482 0022-247Xr99 $30.00 Copyright 䊚 1999 by Academic Press All rights of reproduction in any form reserved.
STOCHASTIC COMPARABILITY
483
others, Chung Ž1967., Anderson Ž1991., Hou and Guo Ž1988., Wang and Yang Ž1992., and Yang Ž1990. for the former and Liggett Ž1985. and Chen Ž1992. for the latter. For convenience we shall follow the general terminology and notations of the particularly readable Anderson Ž1991., for example, the usage the terms of q-matrix and Q-matrix, q-functions and Q-functions, q-resolvent and Q-resolvent, etc. Note the difference of each pair in the above. For details see Anderson Ž1991.. For simplicity in this paper we only consider CTMCs on a linear ordering set. Namely, we assume that the state space E s Zqs 0, 1, 2, . . . 4 with the natural ordering. For such CTMCs, the stochastic comparability of the two transition functions P Ž1. Ž t . and P Ž2. Ž t . can be defined as follows: Ž . DEFINITION 1.1. Two standard transition functions P Ž1. Ž t . s Ž piŽ1. j t ; Ž . . i, j g E . and P Ž2. Ž t . s Ž piŽ2. t ; i, j g E are called stochastically comparaj ble if
Ý piŽ1.j Ž t . F Ý pmŽ2.j Ž t . whenever i F m, for all k g E and t G 0. Ž 1.1. jGk
jGk
A single transition function P Ž t . is said to be stochastically monotone if it is self-comparable, that is, if Ý jG k pi j Ž t . is a nondecreasing function of i for each fixed k g E and t G 0. Sometimes, we just simply refer to ‘‘comparable’’ and ‘‘monotone’’ if no confusion will be caused. A fundamental result, due to Siegmund Ž1976., is the following: PROPOSITION 1.1 ŽSiegmund’s theorem.. A transition function P Ž t . is stochastically monotone if and only if there exists a dual for P Ž t ., namely, if and only if there exists another Ž standard. transition function P˜Ž t . such that ⬁
j
Ý ˜pi k Ž t . s Ý pjk Ž t . ks0
Ž ᭙ i , j g E, t G 0 . .
Ž 1.2.
ksi
The importance of Proposition 1.1 lies in the fact that it reveals the close link between two dual processes. Indeed, a monotone q-function P Ž t . and its dual P˜Ž t . are totally determined with each other as ⬁
˜pi j Ž t . s Ý Ž pjk Ž t . y pjy1, k Ž t . . ksi
Ž ᭙ i , j g E, t G 0 .
Ž 1.3.
484
ZHANG AND CHEN
where pyl , k Ž t . ' 0 and j
pji Ž t . s
Ý Ž ˜pi k Ž t . y ˜piq1, k Ž t . .
Ž ᭙ i , j g E, t G 0 . . Ž 1.4.
ks0
The following relationship is also easy to obtain: pji Ž t . y pjy1, i Ž t . s ˜ pi j Ž t . y ˜ piq1, j Ž t . .
Ž 1.5.
It is worth pointing out that Proposition 1.1 can be stated in an equivalent form, i.e., using the ‘‘language’’ of its dual process. PROPOSITION 1.2. Suppose P˜Ž t . s ˜ pi j Ž t .; i, j g E4 is a Ž standard. transition function which satisfies the following two conditions j
Ž i.
Ý ˜pi k Ž t .
is a nonincreasing function in i for each
ks0
j g E and t G 0;
Ž ii .
lim ˜ pi j Ž t . s 0
iª⬁
Ž ᭙ j g E, ᭙ t G 0 . .
Ž 1.6. Ž 1.7.
Then there exists a stochastically monotone transition function P Ž t . satisfying Ž1.2. ᎐ Ž1.5.. Proposition 1.2 is just a ‘‘conjugation’’ of Proposition 1.1 and the proof, which is the same as Theorem 1.1, is omitted here. A q-function satisfying Ž1.7. in the Proposition 1.2 is called a Feller᎐Reuter᎐Riley function. It is well known that for any standard transition function P Ž t ., the limit lim Ž P Ž t . y I . rt s Q
tª0
Ž 1.8.
exists and this limit matrix Q s qi j ; i, j g E4 , called a q-matrix, satisfies the following conditions 0 F qi j - q⬁
Ý qi j F yqi i F q⬁
Ž i / j. Ž᭙i g E. .
Ž 1.9. Ž 1.10.
j/i
Let qi s yqii Ž i g E .. Note that qi s q⬁ is possible for some i g E Žor even all i g E .. However, this case does not occur when discussing stochastic monotonicity; see Anderson Ž1991. or Chen and Zhang Ž1998.. Indeed, in discussing stochastic monotonicity, the associated q-matrix is necessarily totally stable. Recall that a q-matrix Q is called totally stable if
STOCHASTIC COMPARABILITY
485
all qi Ž i g E . are finite. Moreover, if we have further
Ý qi j s qi - q⬁
Ž᭙i g E. ,
Ž 1.11.
j/i
then Q is called conservative. It is also well known that for a totally stable q-matrix Q, there always exists a standard transition function P Ž t ., called the Feller minimal Qfunction, such that Ž1.8. holds true. In nearly all the problems of interest, in particular, in the applications, what we know is not the transition function, but the q-matrix Q. Thus it has considerable significance, both in theory and application, to answer the following basic questions. Question 1. For a given totally stable q-matrix Q, what are the necessary and sufficient conditions for the minimal q-function to be the dual of some stochastically monotone q-function? Question 2. For two given totally stable q-matrices Q Ž1. and Q Ž2., under what conditions will F Ž1. Ž t . and F Ž2. Ž t ., the corresponding Feller minimal functions of Q Ž1. and Q Ž2., respectively, be stochastically comparable? Question 3. For a given totally stable q-matrix Q, what are the necessary and sufficient conditions for the Feller minimal Q-function to be a Feller᎐Reuter᎐Riley Q-function? Kirstein Ž1976. was the first to give the answer to Question 2 under the condition that both Q Ž1. and Q Ž2. are regular. Anderson Ž1991. improved Kirstein’s result by removing the regularity condition. Unfortunately, Anderson’s result is not exactly correct since the monotone convergence theorem was incorrectly used in his proof. Hence some amendments to Anderson’s results are necessary. Also, to our knowledge, Question 1 above remains open. As to Question 3, Reuter and Riley Ž1972. gave a sufficient condition for the minimal Q-function to be the Feller᎐ Reuter᎐Riley Q-function. In this paper, we shall systematically discuss the above three questions. For Questions 1 and 2, the complete answers are given in Theorems 4.6 and 3.1, respectively. As to Question 3, the discussion is concentrated on a class of important q-matrices and a necessary and sufficient condition is given for these q-matrices. It is worth pointing out that the Feller᎐Reuter᎐Riley q-function has an important application, in particular, in the study of strong ergodicity of q-functions. We shall discuss this in a subsequent paper.
486
ZHANG AND CHEN
2. PRELIMINARIES Let lq 1 s u s Ž u i . : u i G 0 Ž i g E . and
½
Ý ui - ⬁ igE
5
where E s 0, 1, 2, . . . 4
d
k k DEFINITION 2.1. Let u, ¨ g lq 1 . We say that u G ¨ if Ý is0 u i G Ý is0 ¨ i for all k G 0.
The following Proposition 2.2 and Lemma 2.3 are analogues to Proposition 7.3.1 and Lemma 7.3.3 of Anderson Ž1991., respectively, and so the proofs are omitted. PROPOSITION 2.2. Let a i , i G 0, and bi , i G 0 be non-negati¨ e ¨ ectors. The following statements are equi¨ alent. Ž1. a i G bj for all i F j Ž2. There exists a sequence c i , i G 0, c i G c iq1 for all i G 0 such that a i G c i G bi for all i G 0. Ž3. For all k G 0, Ý kis0 a i u i G Ý kis0 bi ¨ i for all u, ¨ g lq 1 . LEMMA 2.3. Let P Ž k . s piŽ jk . ; i, j g E4 Ž k s 1, 2. be two substochastic matrices, namely, piŽ jk . G 0 Ž᭙ i, j g E . and Ý j g E piŽ jk . F 1 Ž᭙ i g E . Ž k s 1, 2. and let Ž P Ž k . . n s piŽ jk . Ž n.; i, j g E4 be the nth power of P Ž k . Ž k s 1, 2.. Now if j
j
Ý ks0
piŽ1. k G
Ý
pmŽ2.k
for all j G 0 and i F m,
Ž 2.1.
ks0
then for any n G 0 j
Ý ks0
j
piŽ1. k Ž n. G
Ý
pmŽ2.k Ž n . for all j G 0 and i F m.
Ž 2.2.
ks0
The following proposition is of basic importance since it reveals the relationship between two minimal Q Ž k .-functions Ž k s 1, 2. in terms of their q-matrices. Ž2. 4 PROPOSITION 2.4. Suppose Q Ž1. s qiŽ1. s qiŽ2. j ; i, j g E and Q j ; i, j g Ž1. Ž1. Ž2. Ž . E4 are two q-matrices. Let F Ž t . s f i j Ž t .; i, j g E4 and F Ž t . s f iŽ2. j t ; i, j g E4 be their corresponding minimal q-functions. Then the following
487
STOCHASTIC COMPARABILITY
statements are equi¨ alent j
Ž i.
Ý
j
f iŽ1. k Ž t. G
ks0
Ý
f mŽ2.k Ž t . whene¨ er i F m and for all j G 0
ks0
Ž 2.3. j
Ž ii .
j
Ý qiŽ1.k G Ý qmŽ2.k whene¨ er i F m, and j is such that either ks0
ks0
j - i or j G m.
Ž 2.4.
Proof. Ži. « Žii. is easy. We just need to prove Žii. « Ži.. First suppose both Q Ž1. and Q Ž2. are uniformly bounded; then the proof can be easily given. Indeed, the proof of Theorem 7.3.4 in Anderson Ž1991., together with some obvious amendment, yields the conclusion. Now drop the uniformly bounded assumption and use Proposition 2.2.14 in Anderson Ž1991. to complete the proof. For the given two q-matrices Q Ž1. and Q Ž2. satisfying condition Žii., let N Q Ž r . s N qiŽ jr . ; i, j g E4 Ž r s 1, 2. denote the corresponding truncated q-matrices, defined as N
qiŽ jr . s qiŽ jr . ,
0 F i, j - N y 1
N
qiŽNr . s d iŽ r . q
Ý
qiŽ jr . ,
i-N
jGN N
qiŽ jr . s 0,
j ) N or i G N
for r s 1, 2, where d iŽ r . is the nonconservative quantity of Q Ž r . at i, i.e., d iŽ r . s y
⬁
for r s 1, 2.
Ý qiŽ jr . js1
Note that both of the truncated q-matrices are uniformly bounded and j Ž1. j Ž2. that Ý ks 0 N qi k G Ý ks0 N qm k whenever i F m and j is such that either j - i or j G m, which follows from the fact that j
Ý N qiŽkr . ks0
¡ q Ý ~ s ¢0, i
Žr. ik ,
jFNy1
ks0
jGN
488
ZHANG AND CHEN
j Žr. if i - N, and that Ý ks 0 N qi k s 0 if i G N. Hence, by what we have proved above, and using an obvious notation, we have
j
j Ž1. N fi k Ž t . G
Ý ks0
Ý N f mŽ2.k Ž t . for all i F m and
j G 0.
Ž 2.5.
ks0
Since Ž2.5. holds for every N, then by using Proposition 2.2.14 in Anderson Ž1991. we obtain Žr. N fi j
Ž t . ª f iŽjr . Ž t .
as N ª ⬁, for all i , j, t G 0 Ž r s 1, 2 . .
Thus j
Ý
j
f iŽ1. k Ž t. G
ks0
Ý
f mŽ2.k Ž t . for all j G 0 and i F m,
ks0
which gives the desired result.
QED
Remark 2.1. Note that the proof given here is similar to the one given in Theorem 7.3.4 of Anderson Ž1991.. However, there is an important difference: in our Ž2.5. both sides are of finite term sums only while in Ž7.3.10. of Anderson Ž1991. both sides are of infinite term sums. This is the place where Anderson Ž1991. incorrectly used the monotone convergence theorem and then caused an incomplete result. See the following Remark 3.1.
3. STOCHASTIC COMPARABILITY The purpose of this section is to give the characterization of stochastic comparability in terms of the q-matrices. First give the following definition: DEFINITION 3.1. Two q-matrices Q Ž r . s Ž qiŽ jr . ; i, j g E ., r s 1, 2, are said to be Žstochastically . comparable if
Ý qiŽ1.j F Ý qmŽ2.j , jGk
for all i F m, and k F i or k G m q 1. Ž 3.1.
jGk
Note that whether two q-matrices are Žstochastically . comparable is an easy-checking condition. We are now ready to state the following result which answers Question 2 in Section 1 completely.
STOCHASTIC COMPARABILITY
489
THEOREM 3.1. For any two totally stable q-matrices Q Ž1. and Q Ž2., the Ž2. Ž . Ž . 4 Ž . minimal q-functions F Ž t . Ž t . s f iŽ1. t s f iŽ2. j t ; i, j g E and F j t ; i, j g E4 are stochastically comparable if and only if the following two conditions are satisfied: Ži. Q Ž1. and Q Ž2. are comparable. Žii. Q Ž2. is zero-exit, i.e., for any ) 0, the equation
Ž I y QŽ2. . U s 0,
Ž 0 F U F 1.
has only the zero solution. Proof. We first prove the necessity of conditions Ži. and Žii.. Note that for any q-function which satisfies the Kolmogorov backward equations, in particular, for Feller minimal q-functions f iŽjr . Ž t .4 , we have the following simple fact: lim tª0
1 t
Ý
f iŽjr . Ž t . s
jgA_i
Ý
qiŽ jr .
᭙A ; E.
jgA_i
Now condition Ži. immediately follows. In order to obtain condition Žii., letting k s 0, i s 0 in Ž1.1. then yields ⬁
⬁
js0
js0
Ý f mŽ2.j Ž t . G Ý f 0Ž1.j Ž t . ) 0,
for all m G 0, t G 0
and thus ⬁
inf
Ý f mŽ2.j Ž t . ) 0.
m js0
It implies, due to a basic result in Hou and Guo Ž1988., that Q Ž2. is zero-exit. Conversely, suppose that conditions Ži. and Žii. are satisfied. By first taking ⌬ f E, and defining an order relation as ⌬ - 0 - 1 - 2 - ⭈⭈⭈ and, second, defining two Q-matrices in E⌬ as ⌬Q
where
s Ž ⌬ qiŽ jk . ; i , j g E⌬ . ,
Žk.
¡0
Žk. ⌬ qi j
s
~
Ž 3.2.
i s ⌬ , j g E⌬
d iŽ k .
i g E, j s ⌬
qiŽ jk .
i, j g E
¢
k s 1, 2
Ž k s 1, 2 .
Ž 3.3.
490
ZHANG AND CHEN
and d Ž k . s Ž d iŽ k ., i g E . is the nonconservative quantity of Q Ž k ., i.e., d iŽ k . s y
Ž i g E, k s 1, 2 . ,
Ý qiŽ jk .
Ž 3.4.
jgE
then it is easy to see that, by using condition Ži. and Ž3.4. j
Ý
j Ž1. ⌬ qi k
G
ks⌬
Ý
Ž2. ⌬ qm k
for all i F m, and j is such that either
ks⌬
j - i or j G m.
Ž 3.5.
Now by Proposition 2.2, we obtain j
Ý
j Ž1. ⌬ fi k Ž t . G
ks⌬
Ž2. ⌬ fm k
Ý
whenever i F m, for all j G ⌬ Ž 3.6.
Ž t.
ks⌬
Since ⌬ Q Ž2. is conservative and zero-exit, and thus is regular Žit is easy to prove that ⌬ Q Ž2. is zero-exit if and only if Q Ž2. is.. We know that the Feller minimal ⌬ Q Ž2.-function is honest, i.e., ⬁
Ý
Ž2. ⌬ fm k
Ž t. s 1
for all m G ⌬ , t G 0.
Ž 3.7.
ks⌬
Using Ž3.6. and Ž3.7. then yields
Ý ⌬ f iŽ1.j Ž t . F Ý ⌬ f iŽ2.j Ž t . jGk
for all i F m and k g E⌬ .
Ž 3.8.
jGk
But ⌬ is an absorbing state for each ⌬ Q Ž k . Žsee Ž3.3.. and thus Žr. ⌬ fi j
Ž t . s f iŽjr . Ž t . ,
i , j g E,
r s 1, 2,
t G 0.
Ž 3.9.
Ž . 4 Now combining Ž3.8. and Ž3.9. shows that F Ž1. Ž t . s f iŽ1. j t ; i, j g E and Ž2. Ž . Ž2. Ž . 4 F t s f i j t ; i, j g E are comparable. QED Remark 3.1. Note that condition Žii. is missing in Anderson Ž1991.. Unfortunately, without this condition, the theorem would fail. A counterexample can easily be given and shall be omitted here. Note also that our proof of the necessity of condition Ži. is much simpler than that of Anderson’s. The following corollary is then obvious. COROLLARY 3.2. For any two conser¨ ati¨ e q-matrices Q Ž1. and Q Ž2., the Ž . 4 and F Ž2. Ž t . s f iŽ2. Ž . minimal q-functions F Ž1. Ž t . s f iŽ1. j t ; i, j g E j t ;
STOCHASTIC COMPARABILITY
491
i, j g E4 are stochastically comparable if and only if the following two conditions are satisfied: Ži. Žii.
Q Ž1. and Q Ž2. are comparable. Q Ž2. is regular.
4. DUAL Q-FUNCTIONS We now turn to Question 1 and give the following two definitions first. DEFINITION 4.1. A transition function P Ž t . s Ž pi j Ž t .; i, j g E, t G 0. is called a Feller᎐Reuter᎐Riley transition function if for any t ) 0, pi j Ž t . ª 0
as i ª ⬁ for all j g E
Ž 4.1.
DEFINITION 4.2. A q-matrix Q s Ž qi j ; i, j g E . is called a Feller᎐ Reuter᎐Riley q-matrix if qi j ª 0
as i ª ⬁ for all j g E.
Ž 4.2.
Recall Proposition 1.1 that a transition function P Ž t . is monotone if and only if there exists a dual transition function P˜Ž t ., say, for P Ž t .. Further˜ Then Q˜ is a totally stable q-matrix more, let the q-matrix of P˜Ž t . be Q. ˜ and P˜Ž t . is the minimal Q-function. For details, see Anderson Ž1991. or Chen and Zhang Ž1998.. The following lemma was originally obtained by Zhang Ž1991.. For convenience, we repeat his proof here. LEMMA 4.1. Let Q be a totally stable q-matrix, and ⌽ Ž . s Ž i j Ž .; i, j g E, ) 0. be the minimal Q-resol¨ ent, i.e., the Laplace transform of the minimal Q-function. Suppose inf
igE
ˆ c Ž . ) 0; Ý i j Ž . s
Ž 4.3.
jgE
then for any row coordination family Ž ., we ha¨ e lim Ž . 1 - ⬁.
ª⬁
Ž 4.4.
Remark 4.1. We refer to Yang Ž1990. for the definition of the row coordination family and many important properties, in particular, for any row coordination family Ž ., Ž .1 is a nondecreasing function of , and thus the limit in Ž4.4. does exist.
492
ZHANG AND CHEN
Proof. Since Ž . is a row coordination family, we have
i Ž . s i Ž . q Ž y .
k Ž . k i Ž .
Ý
Ž i g E, , ) 0 .
kgE
Ž 4.5. which yields
Ž . 1 s Ž . 1 q Ž y . Ž . Ž Ž . 1 . .
Ž 4.6.
Suppose ) ; then by using Ž4.3., we obtain
Ž . 1 G Ž . 1 q Ž y . Ž . Ž c Ž . 1 .
Ž 4.7.
which shows that
Ž Ž . 1 . c Ž . F Ž . 1 q Ž Ž . 1 . c Ž . .
Ž 4.8.
Noting that 0 - cŽ . and that lim ª⬁ Ž .1 s 0 Žsee also Yang Ž1990.. and thus letting ª ⬁ in Ž4.8. yields Ž4.4.. QED A q-resolvent ⌽ Ž . s Ž i j Ž .; i, j g E, ) 0. is called a Feller᎐ Reuter᎐Riley q-resolvent if for any ) 0,
i j Ž . ª 0
as i ª ⬁ for all j g E.
It is well known that ⌽ Ž . is a Feller᎐Reuter᎐Riley q-resolvent if and only if its q-function is a Feller᎐Reuter᎐Riley q-function. See Reuter and Riley Ž1972.. LEMMA 4.2. Q-resol¨ ent. If Ži.
inf
igE
Let Q be a totally stable q-matrix and ⌽ Ž . be the minimal
ˆ cŽ . ) 0, and Ý i j Ž . s jgE
Žii. ⌽ Ž . is a Feller᎐Reuter᎐Riley Q-resol¨ ent, then Q is zero-entrance, i.e., for any ) 0, the equation Y Ž I y Q . s 0,
Y g lq 1
has no solution other than the tri¨ ial solution Y s 0. Proof. The proof is nearly the same as the one given in Reuter and Riley Ž1972.. The difference is that here we use condition Ži. and lim ª⬁ Ž .1 s c - q⬁ instead of ⌽ Ž . honest and Ž .1 ' c, respectively. The detail is omitted. QED
493
STOCHASTIC COMPARABILITY
LEMMA 4.3.
˜ s Ž q˜i j ; i, j g E . be a totally stable q-matrix satisfying Let Q j
j
q˜i k G
Ý ks0
Ž᭙j / i. .
Ý q˜iq1, k
Ž 4.9.
ks0
j Then for any fixed j g E, Ý ks ˜i k x for i G j q 1 and thus the limit 0q j lim iª⬁ Ý ks0 q˜i k exists. In particular,
lim q˜i j s ˆ ˜c j G 0.
Ž 4.10.
iª⬁
Furthermore, define j
q ji s
Ý Ž q˜i k y q˜iq1, k .
Ž᭙i, j g E.
Ž 4.11.
ks0
and j
hj s dj y
Ž᭙j g E.
Ý ˜c i
Ž 4.12.
is0
where ⬁
d j s y Ý q ji .
Ž 4.13.
is0
Then Q s Ž qi j ; i, j g E . is a totally stable q-matrix and thus d j G 0 Ž᭙ j g E . and, also, h j G 0, h jq1 F h j Ž᭙ j g E .. Proof. Easy and thus omitted.
QED
˜ be a q-matrix satisfying Ž4.9. and Q be defined as LEMMA 4.4. Let Q Ž4.11.. If for some ) 0, the equation yi s
⬁
Ý
y k qk i
Y s Ž y i ; i g E . g lq 1
Ž 4.14.
ks0
has a nonzero solution, then for any ) 0, the equation
x i s d˜i q
⬁
Ý q˜i k x k ,
0 F xi F 1
Ž i g E.
Ž 4.15.
ks0
has a solution satisfying sup i g E x i s 1 where d˜i Ž i g E . is the nonconser¨ a˜ ti¨ e quantity of Q. Proof. Fix ) 0; then for any nonzero solution Ž yi , i g E . of Eq. Ž4.14., we have ⬁
Ý h i yi - q⬁ is0
Ž 4.16.
494
ZHANG AND CHEN
where h i is given in Ž4.12. and hence h iq1 F h i , Ž i g E .. Let x0 s
⬁
1 c
xk s
Ý yi h i , is0
ky1
1
yn q x 0 ,
Ý
c
kG1
Ž 4.17.
ns0
where 0-cs
⬁
1
is0
Ý yi q
⬁
Ý yi h i - q⬁.
Ž 4.18.
is0
Using Ž4.14. yields ⬁
ny1
Ý
yi s
is0
Ý
ny1
yk
ks0
žÝ /
qk i .
Ž 4.19.
Ý q˜m i .
Ž 4.20.
is0
By using Ž4.10. ᎐ Ž4.13. we obtain my1
k
qk i s yh k y
Ý is0
is0
Hence
c Ž x n y x 0 . s
⬁
ny1
Ý
yk
ks0
s
⬁
qk i
is0
k
Ý
y k yh k y
ks0
sy
žÝ / ž Ý /
⬁
Ý
yk h k q
ks0
sy
q˜ni
is0
⬁
Ý
⬁ is0
yk h k q
ks0
⬁
Ý q˜ni ⬁
Ý is0
ž
y Ý yk ksi
/ iy1
q˜ni yc q cx 0 q
ž
Ý ks0
yk
/
and so cx n s Ý⬁is0 q˜ni Žyc q cx i .. That is, x n s d˜n q Ý⬁¨ s0 q˜n ¨ x ¨ . By Ž4.17. and Ž4.18., we see that sup x i s 1,
0 F x i F 1,
igE
igE
which gives the required results.
QED
Recall by Chen and Zhang Ž1998. that we have the following result. PROPOSITION 4.5. If P Ž t . is a stochastically monotone Q-function, then P Ž t . must satisfy the Kolmogoro¨ forward equations.
495
STOCHASTIC COMPARABILITY
We are now ready to give the following existence theorem of dual transition functions which answers Question 1 in Section 1 completely.
˜ the minimal Q-function ˜ THEOREM 4.6. For a gi¨ en q-matrix Q, is a dual transition function of some stochastically monotone q-function if and only if the following two conditions are satisfied: j
j
Ý q˜i k G Ý q˜iq1, k
Ž i.
ks0
Žii.
Ž᭙j / i. .
Ž 4.21.
ks0
At least one of the following two conditions holds true.
˜ is a Feller᎐Reuter᎐Riley q-matrix and also zero-entrance, i.e., Ža. Q the equation, V g lq 1 ,
˜. s 0, V Ž I y Q
Ž 4.22.
has only the zero solution. Žb. For some ) 0 Ž and hence for all ) 0. the equation
x i s d˜i q
⬁
Ý q˜i k x k ,
0 F x i F 1,
igE
Ž 4.23.
ks0
has a solution satisfying sup x i s 1
Ž 4.24.
igE
˜ where d˜s Ž d˜i , i g E . is the nonconser¨ ati¨ e quantity of Q. ˜ Proof. First prove the necessity. Let the minimal Q-function P˜Ž t . be the dual transition function of a stochastically monotone q-function P Ž t .. Then by Ž1.4. j
pji Ž t . s
Ý Ž ˜pi k Ž t . y ˜piq1, k Ž t . .
Ž ᭙ i , j g E, t G 0 .
Ž 4.25.
ks0
which immediately gives j
q ji s
Ý Ž q˜i k y q˜iq1, k .
Ž 4.26.
ks0
where Q s Ž qi j ; i, j g E . is the q-matrix of P Ž t .. Hence condition Ži. follows.
496
ZHANG AND CHEN
In order to get condition Žii. all we need to do is to prove that Ža. must hold true if Žb. does not. Assume that Žb. does not hold true; then by Hou and Guo Ž1988., Ž1 y Ý j g E ˜i j Ž ., i g E . is the maximal solution of the ˜ Ž . s Ž ˜i j Ž .; i, j g E . is the minimal Q-re˜ equation Ž4.15., where ⌽ solvent. Since Žb. does not hold true, we must have inf
igE
ˆ c Ž . ) 0. Ý ˜i j Ž . s
Ž 4.27.
jgE
˜ is zero-entrance since ⌽ Ž . is the Feller᎐ Thus by Lemma 4.2, Q ˜ ˜ is a Feller᎐ Reuter᎐Riley Q-resolvent. We now further prove that Q Reuter᎐Riley q-matrix. By Lemma 4.4 and Ž4.26., Q s Ž qi j ; i, j g E . is zero-entrance and then using Proposition 4.5 yields the fact that P Ž t . must be the minimal Q-function. Hence P Ž t . must satisfy the Kolmogorov backward equations. By Ž1.2. we then have ⬁
i
jsk
js0
Ý qi j s Ý q˜k j
Ž i, k g E. .
Ž 4.28.
Note that Ž4.28. holds true if and only if P Ž t . satisfies the Kolmogorov ˜ is a Feller᎐ backward equation; see Chen and Zhang Ž1998.. Thus Q Reuter᎐Riley q-matrix. We now prove the sufficiency. Using condition Ži. and Proposition 2.4 ˜., we have Žset Q Ž1. s Q Ž2. s Q k
k
Ý f˜i j Ž t . G Ý f˜iq1, j Ž t . js0
Ž i , k g E, t G 0 .
Ž 4.29.
js0
˜ where Ž f˜i j Ž t .; i, j g E, t G 0. is the minimal Q-function. If condition Ža. in ˜ Žii. holds, then f˜i j Ž t .4 is the Feller᎐Reuter᎐Riley Q-function; see Reuter and Riley Ž1972.. If condition Žb. in Žii. holds, then by Hou and Guo Ž1988., again, Ž1 y Ý j g E i j Ž .; i g E, ) 0. is the maximal solution of the equation Ž4.15., we then have inf
igE
Ý i j Ž . s 0
Ž ) 0. .
jgE
By Ž4.29. we get lim
iª⬁
Ý i j Ž . s 0
Ž ) 0. ,
jgE
which shows that lim i j Ž . s 0
iª⬁
Ž ) 0, j g E . .
Ž 4.30.
STOCHASTIC COMPARABILITY
497
˜ Hence, f˜i j Ž t .4 is the Feller᎐Reuter᎐Riley Q-function in both cases. This fact, together with Ž4.29. and Proposition 1.2, shows that f˜i j Ž t .4 is the dual transition function of some stochastically monotone transition function. This ends the proof. QED 5. FELLER᎐REUTER᎐RILEY q-FUNCTIONS Finally we discuss Question 3 announced in the introduction. Our main interest is such q-matrices Q s Ž qi j ; i, j g E . satisfying j
j
qi k G
Ý ks0
Ž j / i. .
Ý qiq1, k
Ž 5.1.
ks0
Note that many important q-matrices, such as birth and death q-matrices, do satisfy Ž5.1.. The main result is the following: THEOREM 5.1. Let Q s Ž qi j ; i, j g E . be a totally stable q-matrix satisfying Ž5.1.. Then the minimal Q-function is a Feller᎐Reuter᎐Riley q-function if and only if at least one of the following two conditions holds: Ži. Žii.
Q is both a Feller᎐Reuter᎐Riley q-matrix and zero-entrance. For any ) 0 the equation
¡ ~ x s d q Ý q x ¢0 F x F 1, i g E ⬁
i
i
ik
k
ks0
Ž 5.2.
i
has a solution satisfying sup i g E x i s 1. Proof. Since Ž5.1. holds, by Proposition 2.4 Žset Q Ž1. s Q Ž2. s Q ., we obtain that j
Ý ks0
j
fi k Ž t . G
Ý
f iq1, k Ž t .
Ž 5.3.
ks0
where F Ž t . s Ž f i j Ž t .; i, j g E, t G 0. is the minimal Q-function. By Ž5.3. and Propositions 1.1 and 1.2, F Ž t . is a Feller᎐Reuter᎐Riley q-function if and only if F Ž t . is the dual transition function of some stochastically monotone q-function. The result thus follows from Theorem 4.6. QED Remark 5.1. Reuter and Riley Ž1972. showed that for a given q-matrix Q, if Ži. holds true, then the minimal Q-function is the Feller᎐Reuter᎐Riley Q-function. For their case, Ž5.1. is not necessary.
498
ZHANG AND CHEN
By Hou and Guo Ž1988., we now have COROLLARY 5.2. If Q satisfies Ž5.1. and is conser¨ ati¨ e, then the minimal Q-function is a Feller᎐Reuter᎐Riley q-function if and only if at least one of the following two conditions holds true Ži. Žii.
Q is a Feller᎐Reuter᎐Riley q-matrix and zero-entrance. Q is nonzero-exit.
Remark 5.2. It is easy to see that Corollary 5.2 still holds true if one replaces the assumption that Q is conservative by the much more wider assumption that the nonconservative quantity d i 4 is bounded. 6. EXAMPLES We first provide an example to show the application of Theorem 3.1. Consider the well-known and important birth and death q-matrices. Let Q s Ž qi j ; i, j g E . be a birth and death q-matrix. That is, qi , iq1 s i ) 0,
i g E,
qi , iy1 s i ) 0,
qii s y Ž i q i . , i G 0,
qi j s 0
iG1 for all other i , j
and 0 G 0. EXAMPLE A. Let Q Ž r . s Ž qiŽ jr . ; i, j g E . Ž r s 1, 2. be two birth and death q-matrices. Then their minimal Q Ž r .-functions Ž r s 1, 2. are stochastically comparable if and only if the following two conditions hold Ž2. Ž1. i F i ,
Ž i. Ž ii .
R Ž2. s
⬁
Ý ns1
ž
1
Ž2. n
q
Ž2. Ž1. i F i
Ž2. n Ž2. Ž2. n ny1
q ⭈⭈⭈ q
Ž i g E. Ž2. Ž2. n ⭈⭈⭈ 2 Ž2. Ž2. Ž2. n ⭈⭈⭈ 2 1
/
s q⬁.
Indeed, it is easy to check that condition Ži. holds if and only if Q Ž1. and Q Ž2. are comparable whilst condition Žii. holds if and only if Q Ž2. is zero-exit. Theorem 3.1 thus gives the required result. The following example gives an application of Theorem 4.6. EXAMPLE B. Let Q s Ž qi j ; i, j g E . be a birth and death q-matrix; then the minimal Q-function is a dual transition function of some monotone transition function if and only if the following two conditions hold true: Ži. Žii.
0 s 0, S s ⬁ or R - ⬁,
499
STOCHASTIC COMPARABILITY
where Rs
⬁
Ý ns1
ž
1
n
q
n n ny1
q ⭈⭈⭈ q
n ⭈⭈⭈ 2 n ⭈⭈⭈ 2 1
/
and Ss
⬁
1
Ý
nq 1
ns1
ž
1q
n n
q
n ny1 n ny1
q ⭈⭈⭈ q
n ⭈⭈⭈ 2 1 n ⭈⭈⭈ 2 1
/
.
Proof. It is easy to check that for a birth and death q-matrix Ž4.21. holds if and only if 0 s 0. Also since any birth and death q-matrix is a Feller᎐Reuter᎐Riley q-matrix condition Žii. in Theorem 4.6 holds if and only if S s ⬁ or R - ⬁. Now the results follow from Theorem 4.6. QED EXAMPLE C. Let Q be a birth and death q-matrix and 0 s 0; then the minimal Q-function is a Feller᎐Reuter᎐Riley Q-function if and only if R - ⬁ or S s ⬁. This result follows immediately from Theorem 5.1 and Example B.
REFERENCES 1. W. J. Anderson, ‘‘Continuous᎐Time Markov Chains,’’ Springer Series in Statistics, Springer-Verlag, New York, 1991. 2. A. Y. Chen and H. J. Zhang, ‘‘Existence and Uniqueness of Stochastically Monotone Q-Processes,’’ to appear in SEAMS. Bull. Math., 1998. 3. M. F. Chen, ‘‘From Markov Chains to Non-Equilibrium Particle Systems,’’ World Scientific, Singapore, 1992. 4. K. L. Chung, ‘‘Markov Chains with Stationary Transition Probabilities,’’ Springer-Verlag, BerlinrNew York, 1967. 5. Z. T. Hou and F. Guo, Time᎐Homogeneous Markov Processes with Countable State Space, Springer-Verlag, BerlinrNew York, 1988. 6. B. M. Kirstein, Monotonicity and Comparability of Time᎐Homogeneous Markov Processes with Discrete State Space. Math. Operationsforsch. Statist. 7 Ž1976., 151᎐168. 7. T. M. Liggett, ‘‘Interacting Particle Systems,’’ Springer-Verlag, BerlinrNew York, 1985. 8. Reuter and Riley, The Feller property for Markov semigroups on a countable state space, J. London Math. Soc. Ž 2 . Ž1972., 267᎐275. 9. D. Siegmund, The equivalence of absorbing and reflecting barrier problems for stochastically monotone Markov process, Ann. Probab. 4, No. 6, 1976, 914᎐924. 10. Z. K. Wang and X. Q. Yang, ‘‘Birth and Death Processes and Markov Chains,’’ Springer-Verlag, BerlinrNew York, 1992. 11. X. Q. Yang, ‘‘The Construction Theory of Denumerable Markov Processes,’’ Wiley, New York, 1990. 12. H. J. Zhang, The H-Condition in Construction Theory of Q-Processes. J. Changsha Railway Uni¨ . 10, No. 1, 1991, 68᎐72.
Stochastic Comparability and Dual Q-Functions Hanjun Zhang Research Department, Changsha Railway Uni¨ ersity, Changsha, People’s Republic of China
and Anyue Chen School of Computing and Mathematical Sciences, Uni¨ ersity of Greenwich, London, United Kingdom Submitted by E. S. Lee Received October 5, 1998
In this paper we discuss the relationship and characterization of stochastic comparability, duality, and Feller᎐Reuter᎐Riley transition functions which are closely linked with each other for continuous time Markov chains. A necessary and sufficient condition for two Feller minimal transition functions to be stochastically comparable is given in terms of their density q-matrices only. Moreover, a necessary and sufficient condition under which a transition function is a dual for some stochastically monotone q-function is given in terms of, again, its density q-matrix. Finally, for a class of q-matrices, the necessary and sufficient condition for a transition function to be a Feller᎐Reuter᎐Riley transition function is also given. 䊚 1999 Academic Press
Key Words: stochastic comparability; duality; stochastic monotonicity; Feller᎐ Reuter᎐Riley transition functions; zero-exit; zero-entrance.
1. INTRODUCTION In this paper we discuss the characterization of stochastic comparability, dual q-functions, and the Feller᎐Reuter᎐Riley q-functions in terms of the q-matrices. It is shown that these key concepts have close links with each other. They are important concepts in the study of Markov processes, in particular, in the study of continuous time Markov chains ŽCTMC. and interacting particle systems. Good references on these topics are, among 482 0022-247Xr99 $30.00 Copyright 䊚 1999 by Academic Press All rights of reproduction in any form reserved.
STOCHASTIC COMPARABILITY
483
others, Chung Ž1967., Anderson Ž1991., Hou and Guo Ž1988., Wang and Yang Ž1992., and Yang Ž1990. for the former and Liggett Ž1985. and Chen Ž1992. for the latter. For convenience we shall follow the general terminology and notations of the particularly readable Anderson Ž1991., for example, the usage the terms of q-matrix and Q-matrix, q-functions and Q-functions, q-resolvent and Q-resolvent, etc. Note the difference of each pair in the above. For details see Anderson Ž1991.. For simplicity in this paper we only consider CTMCs on a linear ordering set. Namely, we assume that the state space E s Zqs 0, 1, 2, . . . 4 with the natural ordering. For such CTMCs, the stochastic comparability of the two transition functions P Ž1. Ž t . and P Ž2. Ž t . can be defined as follows: Ž . DEFINITION 1.1. Two standard transition functions P Ž1. Ž t . s Ž piŽ1. j t ; Ž . . i, j g E . and P Ž2. Ž t . s Ž piŽ2. t ; i, j g E are called stochastically comparaj ble if
Ý piŽ1.j Ž t . F Ý pmŽ2.j Ž t . whenever i F m, for all k g E and t G 0. Ž 1.1. jGk
jGk
A single transition function P Ž t . is said to be stochastically monotone if it is self-comparable, that is, if Ý jG k pi j Ž t . is a nondecreasing function of i for each fixed k g E and t G 0. Sometimes, we just simply refer to ‘‘comparable’’ and ‘‘monotone’’ if no confusion will be caused. A fundamental result, due to Siegmund Ž1976., is the following: PROPOSITION 1.1 ŽSiegmund’s theorem.. A transition function P Ž t . is stochastically monotone if and only if there exists a dual for P Ž t ., namely, if and only if there exists another Ž standard. transition function P˜Ž t . such that ⬁
j
Ý ˜pi k Ž t . s Ý pjk Ž t . ks0
Ž ᭙ i , j g E, t G 0 . .
Ž 1.2.
ksi
The importance of Proposition 1.1 lies in the fact that it reveals the close link between two dual processes. Indeed, a monotone q-function P Ž t . and its dual P˜Ž t . are totally determined with each other as ⬁
˜pi j Ž t . s Ý Ž pjk Ž t . y pjy1, k Ž t . . ksi
Ž ᭙ i , j g E, t G 0 .
Ž 1.3.
484
ZHANG AND CHEN
where pyl , k Ž t . ' 0 and j
pji Ž t . s
Ý Ž ˜pi k Ž t . y ˜piq1, k Ž t . .
Ž ᭙ i , j g E, t G 0 . . Ž 1.4.
ks0
The following relationship is also easy to obtain: pji Ž t . y pjy1, i Ž t . s ˜ pi j Ž t . y ˜ piq1, j Ž t . .
Ž 1.5.
It is worth pointing out that Proposition 1.1 can be stated in an equivalent form, i.e., using the ‘‘language’’ of its dual process. PROPOSITION 1.2. Suppose P˜Ž t . s ˜ pi j Ž t .; i, j g E4 is a Ž standard. transition function which satisfies the following two conditions j
Ž i.
Ý ˜pi k Ž t .
is a nonincreasing function in i for each
ks0
j g E and t G 0;
Ž ii .
lim ˜ pi j Ž t . s 0
iª⬁
Ž ᭙ j g E, ᭙ t G 0 . .
Ž 1.6. Ž 1.7.
Then there exists a stochastically monotone transition function P Ž t . satisfying Ž1.2. ᎐ Ž1.5.. Proposition 1.2 is just a ‘‘conjugation’’ of Proposition 1.1 and the proof, which is the same as Theorem 1.1, is omitted here. A q-function satisfying Ž1.7. in the Proposition 1.2 is called a Feller᎐Reuter᎐Riley function. It is well known that for any standard transition function P Ž t ., the limit lim Ž P Ž t . y I . rt s Q
tª0
Ž 1.8.
exists and this limit matrix Q s qi j ; i, j g E4 , called a q-matrix, satisfies the following conditions 0 F qi j - q⬁
Ý qi j F yqi i F q⬁
Ž i / j. Ž᭙i g E. .
Ž 1.9. Ž 1.10.
j/i
Let qi s yqii Ž i g E .. Note that qi s q⬁ is possible for some i g E Žor even all i g E .. However, this case does not occur when discussing stochastic monotonicity; see Anderson Ž1991. or Chen and Zhang Ž1998.. Indeed, in discussing stochastic monotonicity, the associated q-matrix is necessarily totally stable. Recall that a q-matrix Q is called totally stable if
STOCHASTIC COMPARABILITY
485
all qi Ž i g E . are finite. Moreover, if we have further
Ý qi j s qi - q⬁
Ž᭙i g E. ,
Ž 1.11.
j/i
then Q is called conservative. It is also well known that for a totally stable q-matrix Q, there always exists a standard transition function P Ž t ., called the Feller minimal Qfunction, such that Ž1.8. holds true. In nearly all the problems of interest, in particular, in the applications, what we know is not the transition function, but the q-matrix Q. Thus it has considerable significance, both in theory and application, to answer the following basic questions. Question 1. For a given totally stable q-matrix Q, what are the necessary and sufficient conditions for the minimal q-function to be the dual of some stochastically monotone q-function? Question 2. For two given totally stable q-matrices Q Ž1. and Q Ž2., under what conditions will F Ž1. Ž t . and F Ž2. Ž t ., the corresponding Feller minimal functions of Q Ž1. and Q Ž2., respectively, be stochastically comparable? Question 3. For a given totally stable q-matrix Q, what are the necessary and sufficient conditions for the Feller minimal Q-function to be a Feller᎐Reuter᎐Riley Q-function? Kirstein Ž1976. was the first to give the answer to Question 2 under the condition that both Q Ž1. and Q Ž2. are regular. Anderson Ž1991. improved Kirstein’s result by removing the regularity condition. Unfortunately, Anderson’s result is not exactly correct since the monotone convergence theorem was incorrectly used in his proof. Hence some amendments to Anderson’s results are necessary. Also, to our knowledge, Question 1 above remains open. As to Question 3, Reuter and Riley Ž1972. gave a sufficient condition for the minimal Q-function to be the Feller᎐ Reuter᎐Riley Q-function. In this paper, we shall systematically discuss the above three questions. For Questions 1 and 2, the complete answers are given in Theorems 4.6 and 3.1, respectively. As to Question 3, the discussion is concentrated on a class of important q-matrices and a necessary and sufficient condition is given for these q-matrices. It is worth pointing out that the Feller᎐Reuter᎐Riley q-function has an important application, in particular, in the study of strong ergodicity of q-functions. We shall discuss this in a subsequent paper.
486
ZHANG AND CHEN
2. PRELIMINARIES Let lq 1 s u s Ž u i . : u i G 0 Ž i g E . and
½
Ý ui - ⬁ igE
5
where E s 0, 1, 2, . . . 4
d
k k DEFINITION 2.1. Let u, ¨ g lq 1 . We say that u G ¨ if Ý is0 u i G Ý is0 ¨ i for all k G 0.
The following Proposition 2.2 and Lemma 2.3 are analogues to Proposition 7.3.1 and Lemma 7.3.3 of Anderson Ž1991., respectively, and so the proofs are omitted. PROPOSITION 2.2. Let a i , i G 0, and bi , i G 0 be non-negati¨ e ¨ ectors. The following statements are equi¨ alent. Ž1. a i G bj for all i F j Ž2. There exists a sequence c i , i G 0, c i G c iq1 for all i G 0 such that a i G c i G bi for all i G 0. Ž3. For all k G 0, Ý kis0 a i u i G Ý kis0 bi ¨ i for all u, ¨ g lq 1 . LEMMA 2.3. Let P Ž k . s piŽ jk . ; i, j g E4 Ž k s 1, 2. be two substochastic matrices, namely, piŽ jk . G 0 Ž᭙ i, j g E . and Ý j g E piŽ jk . F 1 Ž᭙ i g E . Ž k s 1, 2. and let Ž P Ž k . . n s piŽ jk . Ž n.; i, j g E4 be the nth power of P Ž k . Ž k s 1, 2.. Now if j
j
Ý ks0
piŽ1. k G
Ý
pmŽ2.k
for all j G 0 and i F m,
Ž 2.1.
ks0
then for any n G 0 j
Ý ks0
j
piŽ1. k Ž n. G
Ý
pmŽ2.k Ž n . for all j G 0 and i F m.
Ž 2.2.
ks0
The following proposition is of basic importance since it reveals the relationship between two minimal Q Ž k .-functions Ž k s 1, 2. in terms of their q-matrices. Ž2. 4 PROPOSITION 2.4. Suppose Q Ž1. s qiŽ1. s qiŽ2. j ; i, j g E and Q j ; i, j g Ž1. Ž1. Ž2. Ž . E4 are two q-matrices. Let F Ž t . s f i j Ž t .; i, j g E4 and F Ž t . s f iŽ2. j t ; i, j g E4 be their corresponding minimal q-functions. Then the following
487
STOCHASTIC COMPARABILITY
statements are equi¨ alent j
Ž i.
Ý
j
f iŽ1. k Ž t. G
ks0
Ý
f mŽ2.k Ž t . whene¨ er i F m and for all j G 0
ks0
Ž 2.3. j
Ž ii .
j
Ý qiŽ1.k G Ý qmŽ2.k whene¨ er i F m, and j is such that either ks0
ks0
j - i or j G m.
Ž 2.4.
Proof. Ži. « Žii. is easy. We just need to prove Žii. « Ži.. First suppose both Q Ž1. and Q Ž2. are uniformly bounded; then the proof can be easily given. Indeed, the proof of Theorem 7.3.4 in Anderson Ž1991., together with some obvious amendment, yields the conclusion. Now drop the uniformly bounded assumption and use Proposition 2.2.14 in Anderson Ž1991. to complete the proof. For the given two q-matrices Q Ž1. and Q Ž2. satisfying condition Žii., let N Q Ž r . s N qiŽ jr . ; i, j g E4 Ž r s 1, 2. denote the corresponding truncated q-matrices, defined as N
qiŽ jr . s qiŽ jr . ,
0 F i, j - N y 1
N
qiŽNr . s d iŽ r . q
Ý
qiŽ jr . ,
i-N
jGN N
qiŽ jr . s 0,
j ) N or i G N
for r s 1, 2, where d iŽ r . is the nonconservative quantity of Q Ž r . at i, i.e., d iŽ r . s y
⬁
for r s 1, 2.
Ý qiŽ jr . js1
Note that both of the truncated q-matrices are uniformly bounded and j Ž1. j Ž2. that Ý ks 0 N qi k G Ý ks0 N qm k whenever i F m and j is such that either j - i or j G m, which follows from the fact that j
Ý N qiŽkr . ks0
¡ q Ý ~ s ¢0, i
Žr. ik ,
jFNy1
ks0
jGN
488
ZHANG AND CHEN
j Žr. if i - N, and that Ý ks 0 N qi k s 0 if i G N. Hence, by what we have proved above, and using an obvious notation, we have
j
j Ž1. N fi k Ž t . G
Ý ks0
Ý N f mŽ2.k Ž t . for all i F m and
j G 0.
Ž 2.5.
ks0
Since Ž2.5. holds for every N, then by using Proposition 2.2.14 in Anderson Ž1991. we obtain Žr. N fi j
Ž t . ª f iŽjr . Ž t .
as N ª ⬁, for all i , j, t G 0 Ž r s 1, 2 . .
Thus j
Ý
j
f iŽ1. k Ž t. G
ks0
Ý
f mŽ2.k Ž t . for all j G 0 and i F m,
ks0
which gives the desired result.
QED
Remark 2.1. Note that the proof given here is similar to the one given in Theorem 7.3.4 of Anderson Ž1991.. However, there is an important difference: in our Ž2.5. both sides are of finite term sums only while in Ž7.3.10. of Anderson Ž1991. both sides are of infinite term sums. This is the place where Anderson Ž1991. incorrectly used the monotone convergence theorem and then caused an incomplete result. See the following Remark 3.1.
3. STOCHASTIC COMPARABILITY The purpose of this section is to give the characterization of stochastic comparability in terms of the q-matrices. First give the following definition: DEFINITION 3.1. Two q-matrices Q Ž r . s Ž qiŽ jr . ; i, j g E ., r s 1, 2, are said to be Žstochastically . comparable if
Ý qiŽ1.j F Ý qmŽ2.j , jGk
for all i F m, and k F i or k G m q 1. Ž 3.1.
jGk
Note that whether two q-matrices are Žstochastically . comparable is an easy-checking condition. We are now ready to state the following result which answers Question 2 in Section 1 completely.
STOCHASTIC COMPARABILITY
489
THEOREM 3.1. For any two totally stable q-matrices Q Ž1. and Q Ž2., the Ž2. Ž . Ž . 4 Ž . minimal q-functions F Ž t . Ž t . s f iŽ1. t s f iŽ2. j t ; i, j g E and F j t ; i, j g E4 are stochastically comparable if and only if the following two conditions are satisfied: Ži. Q Ž1. and Q Ž2. are comparable. Žii. Q Ž2. is zero-exit, i.e., for any ) 0, the equation
Ž I y QŽ2. . U s 0,
Ž 0 F U F 1.
has only the zero solution. Proof. We first prove the necessity of conditions Ži. and Žii.. Note that for any q-function which satisfies the Kolmogorov backward equations, in particular, for Feller minimal q-functions f iŽjr . Ž t .4 , we have the following simple fact: lim tª0
1 t
Ý
f iŽjr . Ž t . s
jgA_i
Ý
qiŽ jr .
᭙A ; E.
jgA_i
Now condition Ži. immediately follows. In order to obtain condition Žii., letting k s 0, i s 0 in Ž1.1. then yields ⬁
⬁
js0
js0
Ý f mŽ2.j Ž t . G Ý f 0Ž1.j Ž t . ) 0,
for all m G 0, t G 0
and thus ⬁
inf
Ý f mŽ2.j Ž t . ) 0.
m js0
It implies, due to a basic result in Hou and Guo Ž1988., that Q Ž2. is zero-exit. Conversely, suppose that conditions Ži. and Žii. are satisfied. By first taking ⌬ f E, and defining an order relation as ⌬ - 0 - 1 - 2 - ⭈⭈⭈ and, second, defining two Q-matrices in E⌬ as ⌬Q
where
s Ž ⌬ qiŽ jk . ; i , j g E⌬ . ,
Žk.
¡0
Žk. ⌬ qi j
s
~
Ž 3.2.
i s ⌬ , j g E⌬
d iŽ k .
i g E, j s ⌬
qiŽ jk .
i, j g E
¢
k s 1, 2
Ž k s 1, 2 .
Ž 3.3.
490
ZHANG AND CHEN
and d Ž k . s Ž d iŽ k ., i g E . is the nonconservative quantity of Q Ž k ., i.e., d iŽ k . s y
Ž i g E, k s 1, 2 . ,
Ý qiŽ jk .
Ž 3.4.
jgE
then it is easy to see that, by using condition Ži. and Ž3.4. j
Ý
j Ž1. ⌬ qi k
G
ks⌬
Ý
Ž2. ⌬ qm k
for all i F m, and j is such that either
ks⌬
j - i or j G m.
Ž 3.5.
Now by Proposition 2.2, we obtain j
Ý
j Ž1. ⌬ fi k Ž t . G
ks⌬
Ž2. ⌬ fm k
Ý
whenever i F m, for all j G ⌬ Ž 3.6.
Ž t.
ks⌬
Since ⌬ Q Ž2. is conservative and zero-exit, and thus is regular Žit is easy to prove that ⌬ Q Ž2. is zero-exit if and only if Q Ž2. is.. We know that the Feller minimal ⌬ Q Ž2.-function is honest, i.e., ⬁
Ý
Ž2. ⌬ fm k
Ž t. s 1
for all m G ⌬ , t G 0.
Ž 3.7.
ks⌬
Using Ž3.6. and Ž3.7. then yields
Ý ⌬ f iŽ1.j Ž t . F Ý ⌬ f iŽ2.j Ž t . jGk
for all i F m and k g E⌬ .
Ž 3.8.
jGk
But ⌬ is an absorbing state for each ⌬ Q Ž k . Žsee Ž3.3.. and thus Žr. ⌬ fi j
Ž t . s f iŽjr . Ž t . ,
i , j g E,
r s 1, 2,
t G 0.
Ž 3.9.
Ž . 4 Now combining Ž3.8. and Ž3.9. shows that F Ž1. Ž t . s f iŽ1. j t ; i, j g E and Ž2. Ž . Ž2. Ž . 4 F t s f i j t ; i, j g E are comparable. QED Remark 3.1. Note that condition Žii. is missing in Anderson Ž1991.. Unfortunately, without this condition, the theorem would fail. A counterexample can easily be given and shall be omitted here. Note also that our proof of the necessity of condition Ži. is much simpler than that of Anderson’s. The following corollary is then obvious. COROLLARY 3.2. For any two conser¨ ati¨ e q-matrices Q Ž1. and Q Ž2., the Ž . 4 and F Ž2. Ž t . s f iŽ2. Ž . minimal q-functions F Ž1. Ž t . s f iŽ1. j t ; i, j g E j t ;
STOCHASTIC COMPARABILITY
491
i, j g E4 are stochastically comparable if and only if the following two conditions are satisfied: Ži. Žii.
Q Ž1. and Q Ž2. are comparable. Q Ž2. is regular.
4. DUAL Q-FUNCTIONS We now turn to Question 1 and give the following two definitions first. DEFINITION 4.1. A transition function P Ž t . s Ž pi j Ž t .; i, j g E, t G 0. is called a Feller᎐Reuter᎐Riley transition function if for any t ) 0, pi j Ž t . ª 0
as i ª ⬁ for all j g E
Ž 4.1.
DEFINITION 4.2. A q-matrix Q s Ž qi j ; i, j g E . is called a Feller᎐ Reuter᎐Riley q-matrix if qi j ª 0
as i ª ⬁ for all j g E.
Ž 4.2.
Recall Proposition 1.1 that a transition function P Ž t . is monotone if and only if there exists a dual transition function P˜Ž t ., say, for P Ž t .. Further˜ Then Q˜ is a totally stable q-matrix more, let the q-matrix of P˜Ž t . be Q. ˜ and P˜Ž t . is the minimal Q-function. For details, see Anderson Ž1991. or Chen and Zhang Ž1998.. The following lemma was originally obtained by Zhang Ž1991.. For convenience, we repeat his proof here. LEMMA 4.1. Let Q be a totally stable q-matrix, and ⌽ Ž . s Ž i j Ž .; i, j g E, ) 0. be the minimal Q-resol¨ ent, i.e., the Laplace transform of the minimal Q-function. Suppose inf
igE
ˆ c Ž . ) 0; Ý i j Ž . s
Ž 4.3.
jgE
then for any row coordination family Ž ., we ha¨ e lim Ž . 1 - ⬁.
ª⬁
Ž 4.4.
Remark 4.1. We refer to Yang Ž1990. for the definition of the row coordination family and many important properties, in particular, for any row coordination family Ž ., Ž .1 is a nondecreasing function of , and thus the limit in Ž4.4. does exist.
492
ZHANG AND CHEN
Proof. Since Ž . is a row coordination family, we have
i Ž . s i Ž . q Ž y .
k Ž . k i Ž .
Ý
Ž i g E, , ) 0 .
kgE
Ž 4.5. which yields
Ž . 1 s Ž . 1 q Ž y . Ž . Ž Ž . 1 . .
Ž 4.6.
Suppose ) ; then by using Ž4.3., we obtain
Ž . 1 G Ž . 1 q Ž y . Ž . Ž c Ž . 1 .
Ž 4.7.
which shows that
Ž Ž . 1 . c Ž . F Ž . 1 q Ž Ž . 1 . c Ž . .
Ž 4.8.
Noting that 0 - cŽ . and that lim ª⬁ Ž .1 s 0 Žsee also Yang Ž1990.. and thus letting ª ⬁ in Ž4.8. yields Ž4.4.. QED A q-resolvent ⌽ Ž . s Ž i j Ž .; i, j g E, ) 0. is called a Feller᎐ Reuter᎐Riley q-resolvent if for any ) 0,
i j Ž . ª 0
as i ª ⬁ for all j g E.
It is well known that ⌽ Ž . is a Feller᎐Reuter᎐Riley q-resolvent if and only if its q-function is a Feller᎐Reuter᎐Riley q-function. See Reuter and Riley Ž1972.. LEMMA 4.2. Q-resol¨ ent. If Ži.
inf
igE
Let Q be a totally stable q-matrix and ⌽ Ž . be the minimal
ˆ cŽ . ) 0, and Ý i j Ž . s jgE
Žii. ⌽ Ž . is a Feller᎐Reuter᎐Riley Q-resol¨ ent, then Q is zero-entrance, i.e., for any ) 0, the equation Y Ž I y Q . s 0,
Y g lq 1
has no solution other than the tri¨ ial solution Y s 0. Proof. The proof is nearly the same as the one given in Reuter and Riley Ž1972.. The difference is that here we use condition Ži. and lim ª⬁ Ž .1 s c - q⬁ instead of ⌽ Ž . honest and Ž .1 ' c, respectively. The detail is omitted. QED
493
STOCHASTIC COMPARABILITY
LEMMA 4.3.
˜ s Ž q˜i j ; i, j g E . be a totally stable q-matrix satisfying Let Q j
j
q˜i k G
Ý ks0
Ž᭙j / i. .
Ý q˜iq1, k
Ž 4.9.
ks0
j Then for any fixed j g E, Ý ks ˜i k x for i G j q 1 and thus the limit 0q j lim iª⬁ Ý ks0 q˜i k exists. In particular,
lim q˜i j s ˆ ˜c j G 0.
Ž 4.10.
iª⬁
Furthermore, define j
q ji s
Ý Ž q˜i k y q˜iq1, k .
Ž᭙i, j g E.
Ž 4.11.
ks0
and j
hj s dj y
Ž᭙j g E.
Ý ˜c i
Ž 4.12.
is0
where ⬁
d j s y Ý q ji .
Ž 4.13.
is0
Then Q s Ž qi j ; i, j g E . is a totally stable q-matrix and thus d j G 0 Ž᭙ j g E . and, also, h j G 0, h jq1 F h j Ž᭙ j g E .. Proof. Easy and thus omitted.
QED
˜ be a q-matrix satisfying Ž4.9. and Q be defined as LEMMA 4.4. Let Q Ž4.11.. If for some ) 0, the equation yi s
⬁
Ý
y k qk i
Y s Ž y i ; i g E . g lq 1
Ž 4.14.
ks0
has a nonzero solution, then for any ) 0, the equation
x i s d˜i q
⬁
Ý q˜i k x k ,
0 F xi F 1
Ž i g E.
Ž 4.15.
ks0
has a solution satisfying sup i g E x i s 1 where d˜i Ž i g E . is the nonconser¨ a˜ ti¨ e quantity of Q. Proof. Fix ) 0; then for any nonzero solution Ž yi , i g E . of Eq. Ž4.14., we have ⬁
Ý h i yi - q⬁ is0
Ž 4.16.
494
ZHANG AND CHEN
where h i is given in Ž4.12. and hence h iq1 F h i , Ž i g E .. Let x0 s
⬁
1 c
xk s
Ý yi h i , is0
ky1
1
yn q x 0 ,
Ý
c
kG1
Ž 4.17.
ns0
where 0-cs
⬁
1
is0
Ý yi q
⬁
Ý yi h i - q⬁.
Ž 4.18.
is0
Using Ž4.14. yields ⬁
ny1
Ý
yi s
is0
Ý
ny1
yk
ks0
žÝ /
qk i .
Ž 4.19.
Ý q˜m i .
Ž 4.20.
is0
By using Ž4.10. ᎐ Ž4.13. we obtain my1
k
qk i s yh k y
Ý is0
is0
Hence
c Ž x n y x 0 . s
⬁
ny1
Ý
yk
ks0
s
⬁
qk i
is0
k
Ý
y k yh k y
ks0
sy
žÝ / ž Ý /
⬁
Ý
yk h k q
ks0
sy
q˜ni
is0
⬁
Ý
⬁ is0
yk h k q
ks0
⬁
Ý q˜ni ⬁
Ý is0
ž
y Ý yk ksi
/ iy1
q˜ni yc q cx 0 q
ž
Ý ks0
yk
/
and so cx n s Ý⬁is0 q˜ni Žyc q cx i .. That is, x n s d˜n q Ý⬁¨ s0 q˜n ¨ x ¨ . By Ž4.17. and Ž4.18., we see that sup x i s 1,
0 F x i F 1,
igE
igE
which gives the required results.
QED
Recall by Chen and Zhang Ž1998. that we have the following result. PROPOSITION 4.5. If P Ž t . is a stochastically monotone Q-function, then P Ž t . must satisfy the Kolmogoro¨ forward equations.
495
STOCHASTIC COMPARABILITY
We are now ready to give the following existence theorem of dual transition functions which answers Question 1 in Section 1 completely.
˜ the minimal Q-function ˜ THEOREM 4.6. For a gi¨ en q-matrix Q, is a dual transition function of some stochastically monotone q-function if and only if the following two conditions are satisfied: j
j
Ý q˜i k G Ý q˜iq1, k
Ž i.
ks0
Žii.
Ž᭙j / i. .
Ž 4.21.
ks0
At least one of the following two conditions holds true.
˜ is a Feller᎐Reuter᎐Riley q-matrix and also zero-entrance, i.e., Ža. Q the equation, V g lq 1 ,
˜. s 0, V Ž I y Q
Ž 4.22.
has only the zero solution. Žb. For some ) 0 Ž and hence for all ) 0. the equation
x i s d˜i q
⬁
Ý q˜i k x k ,
0 F x i F 1,
igE
Ž 4.23.
ks0
has a solution satisfying sup x i s 1
Ž 4.24.
igE
˜ where d˜s Ž d˜i , i g E . is the nonconser¨ ati¨ e quantity of Q. ˜ Proof. First prove the necessity. Let the minimal Q-function P˜Ž t . be the dual transition function of a stochastically monotone q-function P Ž t .. Then by Ž1.4. j
pji Ž t . s
Ý Ž ˜pi k Ž t . y ˜piq1, k Ž t . .
Ž ᭙ i , j g E, t G 0 .
Ž 4.25.
ks0
which immediately gives j
q ji s
Ý Ž q˜i k y q˜iq1, k .
Ž 4.26.
ks0
where Q s Ž qi j ; i, j g E . is the q-matrix of P Ž t .. Hence condition Ži. follows.
496
ZHANG AND CHEN
In order to get condition Žii. all we need to do is to prove that Ža. must hold true if Žb. does not. Assume that Žb. does not hold true; then by Hou and Guo Ž1988., Ž1 y Ý j g E ˜i j Ž ., i g E . is the maximal solution of the ˜ Ž . s Ž ˜i j Ž .; i, j g E . is the minimal Q-re˜ equation Ž4.15., where ⌽ solvent. Since Žb. does not hold true, we must have inf
igE
ˆ c Ž . ) 0. Ý ˜i j Ž . s
Ž 4.27.
jgE
˜ is zero-entrance since ⌽ Ž . is the Feller᎐ Thus by Lemma 4.2, Q ˜ ˜ is a Feller᎐ Reuter᎐Riley Q-resolvent. We now further prove that Q Reuter᎐Riley q-matrix. By Lemma 4.4 and Ž4.26., Q s Ž qi j ; i, j g E . is zero-entrance and then using Proposition 4.5 yields the fact that P Ž t . must be the minimal Q-function. Hence P Ž t . must satisfy the Kolmogorov backward equations. By Ž1.2. we then have ⬁
i
jsk
js0
Ý qi j s Ý q˜k j
Ž i, k g E. .
Ž 4.28.
Note that Ž4.28. holds true if and only if P Ž t . satisfies the Kolmogorov ˜ is a Feller᎐ backward equation; see Chen and Zhang Ž1998.. Thus Q Reuter᎐Riley q-matrix. We now prove the sufficiency. Using condition Ži. and Proposition 2.4 ˜., we have Žset Q Ž1. s Q Ž2. s Q k
k
Ý f˜i j Ž t . G Ý f˜iq1, j Ž t . js0
Ž i , k g E, t G 0 .
Ž 4.29.
js0
˜ where Ž f˜i j Ž t .; i, j g E, t G 0. is the minimal Q-function. If condition Ža. in ˜ Žii. holds, then f˜i j Ž t .4 is the Feller᎐Reuter᎐Riley Q-function; see Reuter and Riley Ž1972.. If condition Žb. in Žii. holds, then by Hou and Guo Ž1988., again, Ž1 y Ý j g E i j Ž .; i g E, ) 0. is the maximal solution of the equation Ž4.15., we then have inf
igE
Ý i j Ž . s 0
Ž ) 0. .
jgE
By Ž4.29. we get lim
iª⬁
Ý i j Ž . s 0
Ž ) 0. ,
jgE
which shows that lim i j Ž . s 0
iª⬁
Ž ) 0, j g E . .
Ž 4.30.
STOCHASTIC COMPARABILITY
497
˜ Hence, f˜i j Ž t .4 is the Feller᎐Reuter᎐Riley Q-function in both cases. This fact, together with Ž4.29. and Proposition 1.2, shows that f˜i j Ž t .4 is the dual transition function of some stochastically monotone transition function. This ends the proof. QED 5. FELLER᎐REUTER᎐RILEY q-FUNCTIONS Finally we discuss Question 3 announced in the introduction. Our main interest is such q-matrices Q s Ž qi j ; i, j g E . satisfying j
j
qi k G
Ý ks0
Ž j / i. .
Ý qiq1, k
Ž 5.1.
ks0
Note that many important q-matrices, such as birth and death q-matrices, do satisfy Ž5.1.. The main result is the following: THEOREM 5.1. Let Q s Ž qi j ; i, j g E . be a totally stable q-matrix satisfying Ž5.1.. Then the minimal Q-function is a Feller᎐Reuter᎐Riley q-function if and only if at least one of the following two conditions holds: Ži. Žii.
Q is both a Feller᎐Reuter᎐Riley q-matrix and zero-entrance. For any ) 0 the equation
¡ ~ x s d q Ý q x ¢0 F x F 1, i g E ⬁
i
i
ik
k
ks0
Ž 5.2.
i
has a solution satisfying sup i g E x i s 1. Proof. Since Ž5.1. holds, by Proposition 2.4 Žset Q Ž1. s Q Ž2. s Q ., we obtain that j
Ý ks0
j
fi k Ž t . G
Ý
f iq1, k Ž t .
Ž 5.3.
ks0
where F Ž t . s Ž f i j Ž t .; i, j g E, t G 0. is the minimal Q-function. By Ž5.3. and Propositions 1.1 and 1.2, F Ž t . is a Feller᎐Reuter᎐Riley q-function if and only if F Ž t . is the dual transition function of some stochastically monotone q-function. The result thus follows from Theorem 4.6. QED Remark 5.1. Reuter and Riley Ž1972. showed that for a given q-matrix Q, if Ži. holds true, then the minimal Q-function is the Feller᎐Reuter᎐Riley Q-function. For their case, Ž5.1. is not necessary.
498
ZHANG AND CHEN
By Hou and Guo Ž1988., we now have COROLLARY 5.2. If Q satisfies Ž5.1. and is conser¨ ati¨ e, then the minimal Q-function is a Feller᎐Reuter᎐Riley q-function if and only if at least one of the following two conditions holds true Ži. Žii.
Q is a Feller᎐Reuter᎐Riley q-matrix and zero-entrance. Q is nonzero-exit.
Remark 5.2. It is easy to see that Corollary 5.2 still holds true if one replaces the assumption that Q is conservative by the much more wider assumption that the nonconservative quantity d i 4 is bounded. 6. EXAMPLES We first provide an example to show the application of Theorem 3.1. Consider the well-known and important birth and death q-matrices. Let Q s Ž qi j ; i, j g E . be a birth and death q-matrix. That is, qi , iq1 s i ) 0,
i g E,
qi , iy1 s i ) 0,
qii s y Ž i q i . , i G 0,
qi j s 0
iG1 for all other i , j
and 0 G 0. EXAMPLE A. Let Q Ž r . s Ž qiŽ jr . ; i, j g E . Ž r s 1, 2. be two birth and death q-matrices. Then their minimal Q Ž r .-functions Ž r s 1, 2. are stochastically comparable if and only if the following two conditions hold Ž2. Ž1. i F i ,
Ž i. Ž ii .
R Ž2. s
⬁
Ý ns1
ž
1
Ž2. n
q
Ž2. Ž1. i F i
Ž2. n Ž2. Ž2. n ny1
q ⭈⭈⭈ q
Ž i g E. Ž2. Ž2. n ⭈⭈⭈ 2 Ž2. Ž2. Ž2. n ⭈⭈⭈ 2 1
/
s q⬁.
Indeed, it is easy to check that condition Ži. holds if and only if Q Ž1. and Q Ž2. are comparable whilst condition Žii. holds if and only if Q Ž2. is zero-exit. Theorem 3.1 thus gives the required result. The following example gives an application of Theorem 4.6. EXAMPLE B. Let Q s Ž qi j ; i, j g E . be a birth and death q-matrix; then the minimal Q-function is a dual transition function of some monotone transition function if and only if the following two conditions hold true: Ži. Žii.
0 s 0, S s ⬁ or R - ⬁,
499
STOCHASTIC COMPARABILITY
where Rs
⬁
Ý ns1
ž
1
n
q
n n ny1
q ⭈⭈⭈ q
n ⭈⭈⭈ 2 n ⭈⭈⭈ 2 1
/
and Ss
⬁
1
Ý
nq 1
ns1
ž
1q
n n
q
n ny1 n ny1
q ⭈⭈⭈ q
n ⭈⭈⭈ 2 1 n ⭈⭈⭈ 2 1
/
.
Proof. It is easy to check that for a birth and death q-matrix Ž4.21. holds if and only if 0 s 0. Also since any birth and death q-matrix is a Feller᎐Reuter᎐Riley q-matrix condition Žii. in Theorem 4.6 holds if and only if S s ⬁ or R - ⬁. Now the results follow from Theorem 4.6. QED EXAMPLE C. Let Q be a birth and death q-matrix and 0 s 0; then the minimal Q-function is a Feller᎐Reuter᎐Riley Q-function if and only if R - ⬁ or S s ⬁. This result follows immediately from Theorem 5.1 and Example B.
REFERENCES 1. W. J. Anderson, ‘‘Continuous᎐Time Markov Chains,’’ Springer Series in Statistics, Springer-Verlag, New York, 1991. 2. A. Y. Chen and H. J. Zhang, ‘‘Existence and Uniqueness of Stochastically Monotone Q-Processes,’’ to appear in SEAMS. Bull. Math., 1998. 3. M. F. Chen, ‘‘From Markov Chains to Non-Equilibrium Particle Systems,’’ World Scientific, Singapore, 1992. 4. K. L. Chung, ‘‘Markov Chains with Stationary Transition Probabilities,’’ Springer-Verlag, BerlinrNew York, 1967. 5. Z. T. Hou and F. Guo, Time᎐Homogeneous Markov Processes with Countable State Space, Springer-Verlag, BerlinrNew York, 1988. 6. B. M. Kirstein, Monotonicity and Comparability of Time᎐Homogeneous Markov Processes with Discrete State Space. Math. Operationsforsch. Statist. 7 Ž1976., 151᎐168. 7. T. M. Liggett, ‘‘Interacting Particle Systems,’’ Springer-Verlag, BerlinrNew York, 1985. 8. Reuter and Riley, The Feller property for Markov semigroups on a countable state space, J. London Math. Soc. Ž 2 . Ž1972., 267᎐275. 9. D. Siegmund, The equivalence of absorbing and reflecting barrier problems for stochastically monotone Markov process, Ann. Probab. 4, No. 6, 1976, 914᎐924. 10. Z. K. Wang and X. Q. Yang, ‘‘Birth and Death Processes and Markov Chains,’’ Springer-Verlag, BerlinrNew York, 1992. 11. X. Q. Yang, ‘‘The Construction Theory of Denumerable Markov Processes,’’ Wiley, New York, 1990. 12. H. J. Zhang, The H-Condition in Construction Theory of Q-Processes. J. Changsha Railway Uni¨ . 10, No. 1, 1991, 68᎐72.