- Email: [email protected]
Estimation of the stationary distribution of semi-Markov processes with Borel state space
ARTICLE IN PRESS
Statistics & Probability Letters 76 (2006) 1536–1542 www.elsevier.com/locate/stapro
Estimation of the stationary distribution of semi-Markov processes with Borel state space N. Limnios Laboratoire de Mathe´matiques Applique´es, Universite´ de Technologie de Compie´gne, B.P. 20529, 60205 Compie´gne Cedex, France Received 11 January 2005; received in revised form 22 February 2006; accepted 1 March 2006 Available online 18 April 2006
Abstract We present an empirical estimator of the stationary distribution of continuous time semi-Markov processes with Borel state space. It comes as a particular case of an estimator of a linear functional of the stationary distribution. We give asymptotic results for strong consistency, and the weak and strong invariance principles. r 2006 Elsevier B.V. All rights reserved. Keywords: Stationary distribution; Semi-Markov process; Markov renewal process; Strong consistency; Weak invariance principle; Strong invariance principle
1. Introduction Nonparametric statistical inference for semi-Markov kernels was performed in several works. Moore and Pyke (1968) studied empirical estimators for finite semi-Markov kernels; Lagakos et al. (1978) gave maximum likelihood estimators for nonergodic finite semi-Markov kernels; Gill (1980) studied Kaplan–Meier type estimators by using point process theory; Greenwood and Wefelmeyer (1996) studied efficiency of empirical estimators for linear functionals for a general state space; Ouhbi and Limnios (1999) and Limnios and Ouhbi (2003) studied empirical estimators, and non-linear functionals of semi-Markov kernels including Markov renewal matrices. In this paper we propose a nonparametric estimator of the stationary distribution of semi-Markov processes. Stationary distributions of processes, when they exist, are very important quantities in theory and applications. See Athreya and Majumdar (2003) and Henderson and Glynn (2000) for estimation of stationary distribution of Markov chains with general state space. We have already proposed a nonparametric estimator for the stationary distribution of a finite state space semi-Markov process, based on the separate estimation of the embedded Markov chain and of the sojourn times in the states of the process (Limnios et al., 2005). In the present paper, we propose a different kind of estimator for Borel-measurable state space semiMarkov processes. We construct an estimator of a linear functional of the stationary distribution. The estimator of the stationary distribution appears as a particular case. E-mail address: [email protected]. 0167-7152/$ - see front matter r 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.spl.2006.03.015
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1537
We obtain the weak invariance principle by a martingale approach in the setting of Liptser and Shiryaev (1989) for Markov processes (see also Koroliuk and Limnios, 2005). The strong invariance principle is also given by the corresponding principle for martingales given by Heyde and Scott (1973) and Serfozo’s approach for embedded processes (Serfozo, 1975). In Section 2, we present the basic semi-Markov process used here. In Section 3, we give the estimator and the weak invariance principle. In Section 4, we give the strong invariance principle. 2. The semi-Markov setting Let ðE; EÞ be a Borel-measurable space, and let Qðx; A GÞ, x 2 E; A 2 E; G 2 Bþ , be a semi-Markov kernel on ðE Rþ ; E Bþ Þ where Bþ is the Borel s-algebra of subsets of Rþ (see, e.g., Limnios and Opris- an, 2001). Let ðZðtÞ; t 2 Rþ Þ be an ðE; EÞ-valued progressively measurable time-homogeneous semi-Markov process with semi-Markov kernel Q, and ðJ n ; Sn ; n 2 NÞ be the embedded Markov renewal process, both defined on the complete probability space ðO; F; PÞ. Define also the point process ðNðtÞ; tX0Þ which counts the jumps of Z in the time interval ð0; t. Specifically: Qðx; A GÞ:¼PðJ nþ1 2 A; S nþ1 S n 2 G j J n ¼ xÞ, and ZðtÞ ¼ J NðtÞ
and
J n ¼ ZðSn Þ,
for x 2 E; A 2 E; G 2 Bþ , nX0; tX0. The process ðJ n Þ is a Markov chain with state space ðE; EÞ and transition probability kernel Pðx; dyÞ:¼ Qðx; dy ½0; 1ÞÞ. The process ðS n Þ represents the jump times of Z, 0pS 0 pS 1 p pS n pS nþ1 p , with ~ GÞ:¼PðZð0Þ 2 A; X 0 2 GÞ, A 2 E and X n :¼S n S n1 ; nX1, and X 0 ¼ S 0 , are the inter-jump times. Let mðA G 2 Bþ be the initial distribution of the process ðJ n ; X n ; nX0Þ. kth order moment of the sojourn time in state x 2 E, that Ris, mk ðxÞ:¼E½X knþ1 j J n ¼ x ¼ R Let mk ðxÞ be the k Rþ Qðx; E dsÞs , k ¼ 1; 2; . . . . Set mðxÞ:¼m1 ðxÞ. Here we put Qðx; E dsÞ:¼ E Qðx; dy dsÞ. Let F x ðtÞ:¼Qðx; E ½0; tÞ be the sojourn time distribution in state x 2 E, and let lðx; tÞ be its hazard rate function, that is, Z t ¯ lðx; uÞ du . F x ðtÞ ¼ 1 F x ðtÞ ¼ exp 0
Let us define the transition operator Q as follows: ZZ Qðx; dy dsÞ f ðy; sÞ, Qf ðxÞ:¼ ERþ
for f : E Rþ ! Rþ measurable. Let m be a measure on ðE; EÞ, and define the measure mQ on ðE Rþ ; E Bþ Þ as follows: Z mQðA GÞ:¼ mðdxÞQðx; A GÞ. E
R As usual, let mðgÞ ¼ mg ¼ E mðdxÞgðxÞ, for g a real-valued measurable function on E. Define also the following functional: ZZZ mQf :¼ mðdxÞQðx; dy dsÞ f ðy; sÞ. EERþ
Let also define the n-fold convolution of Q by itself, that is, ZZ Qðx; dy dsÞQðn1Þ ðy; B ðG sÞÞ; QðnÞ ðx; B GÞ ¼
nX2,
ERþ
Qð1Þ ðx; B GÞ ¼ Qðx; B GÞ, and Qð0Þ ðx; B GÞ ¼ 1B ðxÞ, with x 2 E, B 2 E, and G 2 Bþ .
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1538
Let us define Z mðdxÞPð j J 0 ¼ xÞ and Pm ðÞ ¼
Z ~ mðdx dsÞPð j J 0 ¼ x; X 0 ¼ sÞ,
Pm~ ðÞ ¼
E
ERþ
for probability measures m on ðE; EÞ and m~ on ðE Rþ ; E Bþ Þ, respectively. Let W ðtÞ; tX0, be the standard Brownian motion in R1 . In the sequel the symbol ) will stand for the weak convergence of random elements in the Skorokhod space D½0; 1Þ, (see, e.g., Jacod and Shiryaev, 1987). It is worth noticing that the progressive measurability property for the semi-Markov process is required for its integral functional to be measurable. For example, in the particular case of a Polish space it is sufficient to suppose that the semi-Markov process is cadlag. 3. Empirical estimator and the weak invariance principle Let us consider the following assumptions. Assumptions. A1 The semi-Markov process Z is regular, that is, for any x 2 E, and any tX0: Px ðNðtÞo1Þ ¼ 1. A2 The Markov chain J is Harris ergodic, with stationary distribution n. A3 The mean sojourn time is finite, that is, Z ~ 0om:¼ nðdxÞmðxÞo1, E R R1 and E nðdxÞrx o1, where rx :¼ 0 tF¯ x ðtÞ dt. Note that from A2, the following limit exists 1 Xn lim Pk f ¼ P1 f , k¼1 n!1 n in the sup-norm in the Banach space of bounded measurable functions f : E ! R. Let us consider for a fixed time t40, the r.v.s U t :¼t S NðtÞ
and
V t :¼SNðtÞþ1 t,
and consider the backward and forward ðE Rþ ; E Bþ Þ-valued processes ðZt ; U t ; tX0Þ, and ðZ t ; V t ; tX0Þ, respectively. Then the following result is well known (see, e.g., Limnios and Opris- an, 2001). Lemma 1. Both processes ðZ t ; U t ; tX0Þ, and ðZ t ; V t ; tX0Þ are Markov processes, with common stationary distribution Z Z 1 ~ GÞ:¼ nðdxÞ ½1 F x ðuÞ du, (1) pðA ~ A m G on ðE Rþ ; E Bþ Þ. Both processes are continuous from the right. The semigroup generated by these processes are strongly continuous. ~ Rþ Þ, on ðE; EÞ, is the stationary probability measure of ðZðtÞ; tX0Þ, which is The marginal law pðAÞ:¼pðA also the limit, as t ! 1, of its transition function, that is, Z 1 lim PðZðtÞ 2 A j Zð0Þ ¼ xÞ ¼ pðAÞ:¼ nðdyÞmðyÞ, t!1 ~ A m for any x 2 E.
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1539
~ then this process is stationary, Now, as for ergodic Markov processes, if the initial law of ðZt ; V t ; tX0Þ is p, ~ GÞ. Hence, for any tX0, ZðtÞ has distribution that is, for any fixed tX0, we have Pp~ ððZ t ; V t Þ 2 A GÞ ¼ pðA p under Pp~ , (see, e.g., Limnios and Opris- an, 2001). Let B be the space of all E Bþ -measurable bounded functions jðx; tÞ differentiable at t, with the sup norm, that is, kjk:¼supx;t jjðx; tÞj: Let P be the projection operator, acting on B as follows: Z ZZ 1 ~ Pjðx; uÞ ¼ nðdyÞF y ðsÞjðy; sÞ ds. (2) pðdy dsÞjðy; sÞ1ERþ ðx; uÞ ¼ ~ ERþ m ERþ The generator L of the Markov process ðZðtÞ; UðtÞ; tX0Þ is well known to be defined as follows (see, e.g., Gihman and Skorohod, 1974; Korolyuk and Turbin, 1993): Z q Ljðx; uÞ ¼ jðx; uÞ þ lðx; uÞ Pðx; dyÞjðy; 0Þ jðx; uÞ ; j 2 B. (3) qu E Let R0 be the potential operator of the Markov process ðZ t ; U t ; tX0Þ, that is, R0 ¼ ðL þ PÞ1 P. We have R0 P ¼ PR0 ¼ 0. Let R00 be the potential operator of the embedded Markov chain J, that is R00 ¼ ðI P þ P1 Þ1 P1 . Let us consider a measurable bounded function v : E ! R. We intend to estimate functionals of the form R v~:¼pðvÞ ¼ E pðdxÞvðxÞ. We suppose here that v belongs to the range RL of the infinitesimal generator L on L2 ðE; E; pÞ. We propose the following empirical estimator for v~: Z 1 T aT :¼ vðZðsÞÞ ds. (4) T 0 In the particular case, where v ¼ 1A , with A 2 E, we get the following empirical estimator for p: Z 1 T pT ðAÞ:¼ 1A ðZðsÞÞ ds. T 0
(5)
Of course, from the SLLN for ergodic Markov processes (see, e.g., Meyn and Tweedie, 1996), we get aT !~v;
T ! 1;
(6)
Px a:s:;
for any x 2 E. Hence the estimator aT is strongly consistent. Let us consider an observation of the process Z in the time interval ½0; T, where T40 is arbitrary but fixed time, or equivalently, ðJ 0 ; X 1 ; . . . ; J NðTÞ1 ; X NðTÞ ; J NðTÞ ; U T Þ. It is clear that estimators aT and pT are Pp~ -unbiased, meaning that Pp~ aT ¼ v~ and Pp~ pT ðAÞ ¼ pðAÞ;
T40; A 2 E.
Let us consider the family of processes aeT ; TX0; e40, Z T 1 aeT :¼ vðZðs=e2 ÞÞ ds e1 v~. eT 0 Of course, in case where v ¼ 1A , we have v~ ¼ pðAÞ. Theorem 1. Under Assumptions A1–A3, the following weak convergence holds: aet ¼)bW ðtÞ=t as e ! 0, where b2 ¼
1 ~ m
Z E
nðdxÞ 2m2 ðxÞv20 ðxÞ m2 ðxÞv20 ðxÞ 2mðxÞv0 ðxÞR00 v0 ðxÞmðxÞ .
(7)
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1540
Proof. It is well known that the following process (see, e.g., Liptser and Shiryaev, 1989): Z t LjðZðsÞ; UðsÞÞ ds; tX0, MðtÞ:¼jðZðtÞ; UðtÞÞ jðZð0Þ; Uð0ÞÞ
(8)
0
is an Ft -martingale, where Ft :¼sðZðsÞ; UðsÞ; sptÞ, tX0, and L is the generator of ðZðtÞ; UðtÞ; tX0Þ defined in (3). For a bounded test function jðx; tÞ; x 2 E; tX0, we have that P sup ejðZðt=e2 Þ; Uðt=e2 ÞÞ ! 0; e ! 0; T40. tpT
Now, from the martingale invariance principle, (see, e.g., Liptser and Shiryaev, 1989, Theorem 1, p. 685), we have eMðt=e2 Þ¼)bW ðtÞ, where ( Z b:¼ 2
)1=2 ~ pðdx dtÞv0 ðxÞR0 v0 ðxÞ
,
(9)
ERþ
and Ljðx; tÞ ¼ v0 ðxÞ:¼vðxÞ v~. This gives also, for t40, eMðt=e2 Þ=t¼)bW ðtÞ=t. Hence, we get Z 1 t LjðZðs=e2 ÞÞ ds¼)bW ðtÞ=t. et 0 In order to compute b, we have to give a solution of the following equation: Lj ¼ v0 ,
(10)
where the necessary balance condition Pv0 ¼ 0 is trivially fulfilled, since v0 ¼ v Pv. We look for a solution j of (10) satisfying: j ¼ R0 v0 . Now, let us consider the Eq. (10), that is, q jðx; tÞ þ lðx; tÞ½Pjðx; 0Þ jðx; tÞ ¼ v0 ðxÞ. qt We get the following solution of this equation: jðx; tÞ ¼ Sðx; tÞ þ Pjðx; 0Þ,
(11)
(12)
where ð2Þ
Sðx; tÞ ¼ v0 ðxÞF¯ x ðtÞ=F¯ x ðtÞ. From (12) and (13), we get, respectively, jðx; 0Þ ¼ Sðx; 0Þ þ Pjðx; 0Þ, and Sðx; 0Þ ¼ v0 ðxÞmðxÞ. Hence ðI PÞjðx; 0Þ ¼ Sðx; 0Þ. Consequently jðx; 0Þ ¼ R00 Sðx; 0Þ ¼ R00 v0 ðxÞmðxÞ.
(13)
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1541
From the last equation, applying the operator P on both sides, we get Pjðx; 0Þ ¼ PR00 v0 ðxÞmðxÞ ¼ ðR00 IÞv0 mðxÞ. Whence we can write solution (12) as follows: " # ð2Þ F¯ x ðtÞ jðx; tÞ ¼ v0 ðxÞ mðxÞ R00 v0 ðxÞmðxÞ. F¯ x ðtÞ Finally, introducing Eq. (14), into (9), we get the desired conclusion.
(14) &
4. The strong invariance principle Let v0 be defined as in the previous section, and let g be a measurable mapping defined on E Rþ by gðx; sÞ ¼ v0 ðxÞs. Of course, we have Pp v0 ðJ 0 Þ ¼ 0. Let us consider the process ðJ n ; X nþ1 ; nX0Þ for which we can easily prove the following result. e dy dsÞ ¼ Lemma 2. The process ðJ n ; X nþ1 Þ is a Markov renewal process, with semi-Markov kernel Qðx; Pðx; dyÞHðy; dsÞ, where P is the transition kernel of the embedded Markov chain ðJ n Þ, and Hðy; dsÞ ¼ Qðy; E dsÞ. The stationary distribution of ðJ n ; X nþ1 Þ is n~ :¼nH, that is, n~ ðdy dsÞ ¼ nðdyÞHðy; dsÞ. Let us consider the following additional assumptions: A4 The Markov chain ðJ n ; X nþ1 Þ is stationary, that is P ðJ 0 ; X 1 Þ1 ¼ n~ . A5 We have X eðnÞ Q g o1, 2
nX1
RR
1=2 on L2 ð~nÞ. for the norm kf kR2 :¼ ERþ f 2 d~n 1 ¯ A6 We have supx2E 0 tF x ðtÞ dto þ 1. Note that A6 imply A3. P Let us define s2 :¼limn!1 Pð nk¼1 v0 ðJ k1 ÞX k Þ2 =n, and let K be the subset of the space C ¼ C½0; 1 of continuous functions on ½0; 1, with the uniform topology, defined as follows (see, e.g., Gut, 1988): ( ) 2 Z 1 d xðtÞ dtp1 . K ¼ x 2 C : x is absolutely continuous; xð0Þ ¼ 0; dt 0 Let us define the sequence of processes an ðtÞ ant , nX1, that is, Z 1 nt v0 ðZðsÞÞ ds; tX0 an ðtÞ ¼ nt 0 R1 ð2Þ and F¯ x ðtÞ:¼ F¯ x ðsÞds. 0
Theorem 2. p Let us assume that A1, A2, A4–A6, hold true. If moreover s2 40, then the sequence ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi 1=2 ftn an ðtÞ=s 2 log log n; nX3g, viewed as a subset of C, is Pp~ -a.s. relative compact (in the uniform topology), and the set of its limit points is exactly K. Proof. Let us consider the process xðtÞ; tX0, with Z t xðtÞ:¼ v0 ðZðsÞÞ ds; tX0. 0
Let
zn :¼ supfxðsÞ xðS n Þ : Sn psoSnþ1 g,
and Bn :¼sð2n log log nÞ1=2 , for nX3.
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1542
We have, on the event fS n psoSnþ1 g, Z s xðsÞ xðS n Þ ¼ v0 ðZðuÞÞ du ¼ UðsÞv0 ðJ n Þ. Sn
Therefore, we get z½nt ¼ sup UðsÞv0 ðJ ½nt Þ Bn s
Bn ¼
X ½ntþ1 v0 ðJ ½nt Þ a:s: Bn
where the sup is taken on fS ½nt psoS½ntþ1 g. Now, since the sequence ðX n ; nX1Þ, verifies the LIL under Pm for any probability measure m on ðE; EÞ (see, Limnios and Opris-an, 2001, Theorem 3.14, p 79), we have that X ½ntþ1 =Bn ! 0, Pm -a.s., as n ! 1, and then, from boundedness of v0 , we get z½nt a:s: ! 0; Bn
n ! 1.
(15)
Under Assumptions A4 and A5, we get, from Heyde and Scott (1973) results (we use here the particular case in the formulation as given in Herkenrath et al., 2003) that the limit points of the sequence ðxðS ½nt Þ=Bn ; nX3Þ belong to the set K. This, together with the above convergence (15), implies, from Corollary 3.2 in Serfozo (1975), that ðxðntÞ=Bn ; nX3Þ has its limit points in K, which implies, finally, that ðntan ðtÞ=Bn ; nX3Þ has also its limit points in K. & References Athreya, K.B., Majumdar, M., 2003. Estimating the stationary distribution of a Markov chain. Econom. Theory 21 (2), 729–742. Gihman, I.I., Skorohod, A.V., 1974. Theory of Stochastic Processes, vol. 2. Springer, Berlin. Gill, R.D., 1980. Nonparametric estimation based on censored observations of Markov renewal process. Z. Wahrsch. Verw. Geb. 53, 97–116. Greenwood, P.E., Wefelmeyer, W., 1996. Empirical estimators for semi-Markov processes. Math. Methods Statist. 5 (3), 299–315. Gut, A., 1988. Stopped Random Walks. Limit Theorems and Applications. Springer-Verlag, New York. Henderson, S.G., Glynn, P.W., 2000. Computing densities for Markov chains via simulation. Math. Oper. Res. 26 (2), 375–400. Herkenrath, U., Iosifescu, M., Rudolph, A., 2003. Letter to the, editor. A note on invariance principles for iterated random functions. J. Appl. Probab. 40, 834–837. Heyde, C.C., Scott, D.J., 1973. Invariance principles for the law of iterated logarithm for martingales and processes with stationary increments. Ann. Probab. 1, 428–436. Jacod, J., Shiryaev, A.N., 1987. Limit Theorems for Stochastic Processes. Springer, Berlin. Koroliuk, V.S., Limnios, N., 2005. Stochastic Systems in Merging Phase Space. World Scientific, London. Korolyuk, V.S., Turbin, A.F., 1993. Mathematical Foundations of the State Lumping of Large Systems. Kluwer, Dordrecht. Lagakos, S.W., Sommer, C.J., Zelen, M., 1978. Semi-Markov models for partially censored data. Biometrika 65 (2), 311–317. Limnios, N., Opris-an, G., 2001. Semi-Markov Processes and Reliability. Birkha¨user, Boston. Limnios, N., Ouhbi, B., 2003. Empirical estimators of reliability and related functions for semi-Markov systems. In: Lindqvist, B., Doksum, K. (Eds.), Mathematical and Statistical Methods in Reliability. World Scientific, Singapore, p. 2005. Limnios, N., Ouhbi, B., Sadek, A., 2005. Empirical estimator of stationary distribution for semi-Markov processes. Comm. Stat. Theory Methods 34 (3), 987–995. Liptser, R.Sh., Shiryaev, A.N., 1989. Theory of Martingales. Kluwer Academic Publishers, Dordrecht The Netherlands. Meyn, S.P., Tweedie, R.L., 1996. Markov Chains and Stochastic Stability, 3rd printing, Springer-Verlag, New York. Moore, E.H., Pyke, R., 1968. Estimation of the transition distributions of a Markov renewal process. Ann. Inst. Statist. Math. 20, 411–424. Ouhbi, B., Limnios, N., 1999. Nonparametric estimation for semi-Markov processes based on its hazard rate functions. Statist. Inf. Stochast. Process. 2 (2), 151–173. Serfozo, R., 1975. Functional limit theorems for stochastic processes based on embedded processes. Adv. in Appl. Probab. 7, 123–139.
Statistics & Probability Letters 76 (2006) 1536–1542 www.elsevier.com/locate/stapro
Estimation of the stationary distribution of semi-Markov processes with Borel state space N. Limnios Laboratoire de Mathe´matiques Applique´es, Universite´ de Technologie de Compie´gne, B.P. 20529, 60205 Compie´gne Cedex, France Received 11 January 2005; received in revised form 22 February 2006; accepted 1 March 2006 Available online 18 April 2006
Abstract We present an empirical estimator of the stationary distribution of continuous time semi-Markov processes with Borel state space. It comes as a particular case of an estimator of a linear functional of the stationary distribution. We give asymptotic results for strong consistency, and the weak and strong invariance principles. r 2006 Elsevier B.V. All rights reserved. Keywords: Stationary distribution; Semi-Markov process; Markov renewal process; Strong consistency; Weak invariance principle; Strong invariance principle
1. Introduction Nonparametric statistical inference for semi-Markov kernels was performed in several works. Moore and Pyke (1968) studied empirical estimators for finite semi-Markov kernels; Lagakos et al. (1978) gave maximum likelihood estimators for nonergodic finite semi-Markov kernels; Gill (1980) studied Kaplan–Meier type estimators by using point process theory; Greenwood and Wefelmeyer (1996) studied efficiency of empirical estimators for linear functionals for a general state space; Ouhbi and Limnios (1999) and Limnios and Ouhbi (2003) studied empirical estimators, and non-linear functionals of semi-Markov kernels including Markov renewal matrices. In this paper we propose a nonparametric estimator of the stationary distribution of semi-Markov processes. Stationary distributions of processes, when they exist, are very important quantities in theory and applications. See Athreya and Majumdar (2003) and Henderson and Glynn (2000) for estimation of stationary distribution of Markov chains with general state space. We have already proposed a nonparametric estimator for the stationary distribution of a finite state space semi-Markov process, based on the separate estimation of the embedded Markov chain and of the sojourn times in the states of the process (Limnios et al., 2005). In the present paper, we propose a different kind of estimator for Borel-measurable state space semiMarkov processes. We construct an estimator of a linear functional of the stationary distribution. The estimator of the stationary distribution appears as a particular case. E-mail address: [email protected]. 0167-7152/$ - see front matter r 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.spl.2006.03.015
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1537
We obtain the weak invariance principle by a martingale approach in the setting of Liptser and Shiryaev (1989) for Markov processes (see also Koroliuk and Limnios, 2005). The strong invariance principle is also given by the corresponding principle for martingales given by Heyde and Scott (1973) and Serfozo’s approach for embedded processes (Serfozo, 1975). In Section 2, we present the basic semi-Markov process used here. In Section 3, we give the estimator and the weak invariance principle. In Section 4, we give the strong invariance principle. 2. The semi-Markov setting Let ðE; EÞ be a Borel-measurable space, and let Qðx; A GÞ, x 2 E; A 2 E; G 2 Bþ , be a semi-Markov kernel on ðE Rþ ; E Bþ Þ where Bþ is the Borel s-algebra of subsets of Rþ (see, e.g., Limnios and Opris- an, 2001). Let ðZðtÞ; t 2 Rþ Þ be an ðE; EÞ-valued progressively measurable time-homogeneous semi-Markov process with semi-Markov kernel Q, and ðJ n ; Sn ; n 2 NÞ be the embedded Markov renewal process, both defined on the complete probability space ðO; F; PÞ. Define also the point process ðNðtÞ; tX0Þ which counts the jumps of Z in the time interval ð0; t. Specifically: Qðx; A GÞ:¼PðJ nþ1 2 A; S nþ1 S n 2 G j J n ¼ xÞ, and ZðtÞ ¼ J NðtÞ
and
J n ¼ ZðSn Þ,
for x 2 E; A 2 E; G 2 Bþ , nX0; tX0. The process ðJ n Þ is a Markov chain with state space ðE; EÞ and transition probability kernel Pðx; dyÞ:¼ Qðx; dy ½0; 1ÞÞ. The process ðS n Þ represents the jump times of Z, 0pS 0 pS 1 p pS n pS nþ1 p , with ~ GÞ:¼PðZð0Þ 2 A; X 0 2 GÞ, A 2 E and X n :¼S n S n1 ; nX1, and X 0 ¼ S 0 , are the inter-jump times. Let mðA G 2 Bþ be the initial distribution of the process ðJ n ; X n ; nX0Þ. kth order moment of the sojourn time in state x 2 E, that Ris, mk ðxÞ:¼E½X knþ1 j J n ¼ x ¼ R Let mk ðxÞ be the k Rþ Qðx; E dsÞs , k ¼ 1; 2; . . . . Set mðxÞ:¼m1 ðxÞ. Here we put Qðx; E dsÞ:¼ E Qðx; dy dsÞ. Let F x ðtÞ:¼Qðx; E ½0; tÞ be the sojourn time distribution in state x 2 E, and let lðx; tÞ be its hazard rate function, that is, Z t ¯ lðx; uÞ du . F x ðtÞ ¼ 1 F x ðtÞ ¼ exp 0
Let us define the transition operator Q as follows: ZZ Qðx; dy dsÞ f ðy; sÞ, Qf ðxÞ:¼ ERþ
for f : E Rþ ! Rþ measurable. Let m be a measure on ðE; EÞ, and define the measure mQ on ðE Rþ ; E Bþ Þ as follows: Z mQðA GÞ:¼ mðdxÞQðx; A GÞ. E
R As usual, let mðgÞ ¼ mg ¼ E mðdxÞgðxÞ, for g a real-valued measurable function on E. Define also the following functional: ZZZ mQf :¼ mðdxÞQðx; dy dsÞ f ðy; sÞ. EERþ
Let also define the n-fold convolution of Q by itself, that is, ZZ Qðx; dy dsÞQðn1Þ ðy; B ðG sÞÞ; QðnÞ ðx; B GÞ ¼
nX2,
ERþ
Qð1Þ ðx; B GÞ ¼ Qðx; B GÞ, and Qð0Þ ðx; B GÞ ¼ 1B ðxÞ, with x 2 E, B 2 E, and G 2 Bþ .
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1538
Let us define Z mðdxÞPð j J 0 ¼ xÞ and Pm ðÞ ¼
Z ~ mðdx dsÞPð j J 0 ¼ x; X 0 ¼ sÞ,
Pm~ ðÞ ¼
E
ERþ
for probability measures m on ðE; EÞ and m~ on ðE Rþ ; E Bþ Þ, respectively. Let W ðtÞ; tX0, be the standard Brownian motion in R1 . In the sequel the symbol ) will stand for the weak convergence of random elements in the Skorokhod space D½0; 1Þ, (see, e.g., Jacod and Shiryaev, 1987). It is worth noticing that the progressive measurability property for the semi-Markov process is required for its integral functional to be measurable. For example, in the particular case of a Polish space it is sufficient to suppose that the semi-Markov process is cadlag. 3. Empirical estimator and the weak invariance principle Let us consider the following assumptions. Assumptions. A1 The semi-Markov process Z is regular, that is, for any x 2 E, and any tX0: Px ðNðtÞo1Þ ¼ 1. A2 The Markov chain J is Harris ergodic, with stationary distribution n. A3 The mean sojourn time is finite, that is, Z ~ 0om:¼ nðdxÞmðxÞo1, E R R1 and E nðdxÞrx o1, where rx :¼ 0 tF¯ x ðtÞ dt. Note that from A2, the following limit exists 1 Xn lim Pk f ¼ P1 f , k¼1 n!1 n in the sup-norm in the Banach space of bounded measurable functions f : E ! R. Let us consider for a fixed time t40, the r.v.s U t :¼t S NðtÞ
and
V t :¼SNðtÞþ1 t,
and consider the backward and forward ðE Rþ ; E Bþ Þ-valued processes ðZt ; U t ; tX0Þ, and ðZ t ; V t ; tX0Þ, respectively. Then the following result is well known (see, e.g., Limnios and Opris- an, 2001). Lemma 1. Both processes ðZ t ; U t ; tX0Þ, and ðZ t ; V t ; tX0Þ are Markov processes, with common stationary distribution Z Z 1 ~ GÞ:¼ nðdxÞ ½1 F x ðuÞ du, (1) pðA ~ A m G on ðE Rþ ; E Bþ Þ. Both processes are continuous from the right. The semigroup generated by these processes are strongly continuous. ~ Rþ Þ, on ðE; EÞ, is the stationary probability measure of ðZðtÞ; tX0Þ, which is The marginal law pðAÞ:¼pðA also the limit, as t ! 1, of its transition function, that is, Z 1 lim PðZðtÞ 2 A j Zð0Þ ¼ xÞ ¼ pðAÞ:¼ nðdyÞmðyÞ, t!1 ~ A m for any x 2 E.
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1539
~ then this process is stationary, Now, as for ergodic Markov processes, if the initial law of ðZt ; V t ; tX0Þ is p, ~ GÞ. Hence, for any tX0, ZðtÞ has distribution that is, for any fixed tX0, we have Pp~ ððZ t ; V t Þ 2 A GÞ ¼ pðA p under Pp~ , (see, e.g., Limnios and Opris- an, 2001). Let B be the space of all E Bþ -measurable bounded functions jðx; tÞ differentiable at t, with the sup norm, that is, kjk:¼supx;t jjðx; tÞj: Let P be the projection operator, acting on B as follows: Z ZZ 1 ~ Pjðx; uÞ ¼ nðdyÞF y ðsÞjðy; sÞ ds. (2) pðdy dsÞjðy; sÞ1ERþ ðx; uÞ ¼ ~ ERþ m ERþ The generator L of the Markov process ðZðtÞ; UðtÞ; tX0Þ is well known to be defined as follows (see, e.g., Gihman and Skorohod, 1974; Korolyuk and Turbin, 1993): Z q Ljðx; uÞ ¼ jðx; uÞ þ lðx; uÞ Pðx; dyÞjðy; 0Þ jðx; uÞ ; j 2 B. (3) qu E Let R0 be the potential operator of the Markov process ðZ t ; U t ; tX0Þ, that is, R0 ¼ ðL þ PÞ1 P. We have R0 P ¼ PR0 ¼ 0. Let R00 be the potential operator of the embedded Markov chain J, that is R00 ¼ ðI P þ P1 Þ1 P1 . Let us consider a measurable bounded function v : E ! R. We intend to estimate functionals of the form R v~:¼pðvÞ ¼ E pðdxÞvðxÞ. We suppose here that v belongs to the range RL of the infinitesimal generator L on L2 ðE; E; pÞ. We propose the following empirical estimator for v~: Z 1 T aT :¼ vðZðsÞÞ ds. (4) T 0 In the particular case, where v ¼ 1A , with A 2 E, we get the following empirical estimator for p: Z 1 T pT ðAÞ:¼ 1A ðZðsÞÞ ds. T 0
(5)
Of course, from the SLLN for ergodic Markov processes (see, e.g., Meyn and Tweedie, 1996), we get aT !~v;
T ! 1;
(6)
Px a:s:;
for any x 2 E. Hence the estimator aT is strongly consistent. Let us consider an observation of the process Z in the time interval ½0; T, where T40 is arbitrary but fixed time, or equivalently, ðJ 0 ; X 1 ; . . . ; J NðTÞ1 ; X NðTÞ ; J NðTÞ ; U T Þ. It is clear that estimators aT and pT are Pp~ -unbiased, meaning that Pp~ aT ¼ v~ and Pp~ pT ðAÞ ¼ pðAÞ;
T40; A 2 E.
Let us consider the family of processes aeT ; TX0; e40, Z T 1 aeT :¼ vðZðs=e2 ÞÞ ds e1 v~. eT 0 Of course, in case where v ¼ 1A , we have v~ ¼ pðAÞ. Theorem 1. Under Assumptions A1–A3, the following weak convergence holds: aet ¼)bW ðtÞ=t as e ! 0, where b2 ¼
1 ~ m
Z E
nðdxÞ 2m2 ðxÞv20 ðxÞ m2 ðxÞv20 ðxÞ 2mðxÞv0 ðxÞR00 v0 ðxÞmðxÞ .
(7)
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1540
Proof. It is well known that the following process (see, e.g., Liptser and Shiryaev, 1989): Z t LjðZðsÞ; UðsÞÞ ds; tX0, MðtÞ:¼jðZðtÞ; UðtÞÞ jðZð0Þ; Uð0ÞÞ
(8)
0
is an Ft -martingale, where Ft :¼sðZðsÞ; UðsÞ; sptÞ, tX0, and L is the generator of ðZðtÞ; UðtÞ; tX0Þ defined in (3). For a bounded test function jðx; tÞ; x 2 E; tX0, we have that P sup ejðZðt=e2 Þ; Uðt=e2 ÞÞ ! 0; e ! 0; T40. tpT
Now, from the martingale invariance principle, (see, e.g., Liptser and Shiryaev, 1989, Theorem 1, p. 685), we have eMðt=e2 Þ¼)bW ðtÞ, where ( Z b:¼ 2
)1=2 ~ pðdx dtÞv0 ðxÞR0 v0 ðxÞ
,
(9)
ERþ
and Ljðx; tÞ ¼ v0 ðxÞ:¼vðxÞ v~. This gives also, for t40, eMðt=e2 Þ=t¼)bW ðtÞ=t. Hence, we get Z 1 t LjðZðs=e2 ÞÞ ds¼)bW ðtÞ=t. et 0 In order to compute b, we have to give a solution of the following equation: Lj ¼ v0 ,
(10)
where the necessary balance condition Pv0 ¼ 0 is trivially fulfilled, since v0 ¼ v Pv. We look for a solution j of (10) satisfying: j ¼ R0 v0 . Now, let us consider the Eq. (10), that is, q jðx; tÞ þ lðx; tÞ½Pjðx; 0Þ jðx; tÞ ¼ v0 ðxÞ. qt We get the following solution of this equation: jðx; tÞ ¼ Sðx; tÞ þ Pjðx; 0Þ,
(11)
(12)
where ð2Þ
Sðx; tÞ ¼ v0 ðxÞF¯ x ðtÞ=F¯ x ðtÞ. From (12) and (13), we get, respectively, jðx; 0Þ ¼ Sðx; 0Þ þ Pjðx; 0Þ, and Sðx; 0Þ ¼ v0 ðxÞmðxÞ. Hence ðI PÞjðx; 0Þ ¼ Sðx; 0Þ. Consequently jðx; 0Þ ¼ R00 Sðx; 0Þ ¼ R00 v0 ðxÞmðxÞ.
(13)
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1541
From the last equation, applying the operator P on both sides, we get Pjðx; 0Þ ¼ PR00 v0 ðxÞmðxÞ ¼ ðR00 IÞv0 mðxÞ. Whence we can write solution (12) as follows: " # ð2Þ F¯ x ðtÞ jðx; tÞ ¼ v0 ðxÞ mðxÞ R00 v0 ðxÞmðxÞ. F¯ x ðtÞ Finally, introducing Eq. (14), into (9), we get the desired conclusion.
(14) &
4. The strong invariance principle Let v0 be defined as in the previous section, and let g be a measurable mapping defined on E Rþ by gðx; sÞ ¼ v0 ðxÞs. Of course, we have Pp v0 ðJ 0 Þ ¼ 0. Let us consider the process ðJ n ; X nþ1 ; nX0Þ for which we can easily prove the following result. e dy dsÞ ¼ Lemma 2. The process ðJ n ; X nþ1 Þ is a Markov renewal process, with semi-Markov kernel Qðx; Pðx; dyÞHðy; dsÞ, where P is the transition kernel of the embedded Markov chain ðJ n Þ, and Hðy; dsÞ ¼ Qðy; E dsÞ. The stationary distribution of ðJ n ; X nþ1 Þ is n~ :¼nH, that is, n~ ðdy dsÞ ¼ nðdyÞHðy; dsÞ. Let us consider the following additional assumptions: A4 The Markov chain ðJ n ; X nþ1 Þ is stationary, that is P ðJ 0 ; X 1 Þ1 ¼ n~ . A5 We have X eðnÞ Q g o1, 2
nX1
RR
1=2 on L2 ð~nÞ. for the norm kf kR2 :¼ ERþ f 2 d~n 1 ¯ A6 We have supx2E 0 tF x ðtÞ dto þ 1. Note that A6 imply A3. P Let us define s2 :¼limn!1 Pð nk¼1 v0 ðJ k1 ÞX k Þ2 =n, and let K be the subset of the space C ¼ C½0; 1 of continuous functions on ½0; 1, with the uniform topology, defined as follows (see, e.g., Gut, 1988): ( ) 2 Z 1 d xðtÞ dtp1 . K ¼ x 2 C : x is absolutely continuous; xð0Þ ¼ 0; dt 0 Let us define the sequence of processes an ðtÞ ant , nX1, that is, Z 1 nt v0 ðZðsÞÞ ds; tX0 an ðtÞ ¼ nt 0 R1 ð2Þ and F¯ x ðtÞ:¼ F¯ x ðsÞds. 0
Theorem 2. p Let us assume that A1, A2, A4–A6, hold true. If moreover s2 40, then the sequence ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi 1=2 ftn an ðtÞ=s 2 log log n; nX3g, viewed as a subset of C, is Pp~ -a.s. relative compact (in the uniform topology), and the set of its limit points is exactly K. Proof. Let us consider the process xðtÞ; tX0, with Z t xðtÞ:¼ v0 ðZðsÞÞ ds; tX0. 0
Let
zn :¼ supfxðsÞ xðS n Þ : Sn psoSnþ1 g,
and Bn :¼sð2n log log nÞ1=2 , for nX3.
ARTICLE IN PRESS N. Limnios / Statistics & Probability Letters 76 (2006) 1536–1542
1542
We have, on the event fS n psoSnþ1 g, Z s xðsÞ xðS n Þ ¼ v0 ðZðuÞÞ du ¼ UðsÞv0 ðJ n Þ. Sn
Therefore, we get z½nt ¼ sup UðsÞv0 ðJ ½nt Þ Bn s
Bn ¼
X ½ntþ1 v0 ðJ ½nt Þ a:s: Bn
where the sup is taken on fS ½nt psoS½ntþ1 g. Now, since the sequence ðX n ; nX1Þ, verifies the LIL under Pm for any probability measure m on ðE; EÞ (see, Limnios and Opris-an, 2001, Theorem 3.14, p 79), we have that X ½ntþ1 =Bn ! 0, Pm -a.s., as n ! 1, and then, from boundedness of v0 , we get z½nt a:s: ! 0; Bn
n ! 1.
(15)
Under Assumptions A4 and A5, we get, from Heyde and Scott (1973) results (we use here the particular case in the formulation as given in Herkenrath et al., 2003) that the limit points of the sequence ðxðS ½nt Þ=Bn ; nX3Þ belong to the set K. This, together with the above convergence (15), implies, from Corollary 3.2 in Serfozo (1975), that ðxðntÞ=Bn ; nX3Þ has its limit points in K, which implies, finally, that ðntan ðtÞ=Bn ; nX3Þ has also its limit points in K. & References Athreya, K.B., Majumdar, M., 2003. Estimating the stationary distribution of a Markov chain. Econom. Theory 21 (2), 729–742. Gihman, I.I., Skorohod, A.V., 1974. Theory of Stochastic Processes, vol. 2. Springer, Berlin. Gill, R.D., 1980. Nonparametric estimation based on censored observations of Markov renewal process. Z. Wahrsch. Verw. Geb. 53, 97–116. Greenwood, P.E., Wefelmeyer, W., 1996. Empirical estimators for semi-Markov processes. Math. Methods Statist. 5 (3), 299–315. Gut, A., 1988. Stopped Random Walks. Limit Theorems and Applications. Springer-Verlag, New York. Henderson, S.G., Glynn, P.W., 2000. Computing densities for Markov chains via simulation. Math. Oper. Res. 26 (2), 375–400. Herkenrath, U., Iosifescu, M., Rudolph, A., 2003. Letter to the, editor. A note on invariance principles for iterated random functions. J. Appl. Probab. 40, 834–837. Heyde, C.C., Scott, D.J., 1973. Invariance principles for the law of iterated logarithm for martingales and processes with stationary increments. Ann. Probab. 1, 428–436. Jacod, J., Shiryaev, A.N., 1987. Limit Theorems for Stochastic Processes. Springer, Berlin. Koroliuk, V.S., Limnios, N., 2005. Stochastic Systems in Merging Phase Space. World Scientific, London. Korolyuk, V.S., Turbin, A.F., 1993. Mathematical Foundations of the State Lumping of Large Systems. Kluwer, Dordrecht. Lagakos, S.W., Sommer, C.J., Zelen, M., 1978. Semi-Markov models for partially censored data. Biometrika 65 (2), 311–317. Limnios, N., Opris-an, G., 2001. Semi-Markov Processes and Reliability. Birkha¨user, Boston. Limnios, N., Ouhbi, B., 2003. Empirical estimators of reliability and related functions for semi-Markov systems. In: Lindqvist, B., Doksum, K. (Eds.), Mathematical and Statistical Methods in Reliability. World Scientific, Singapore, p. 2005. Limnios, N., Ouhbi, B., Sadek, A., 2005. Empirical estimator of stationary distribution for semi-Markov processes. Comm. Stat. Theory Methods 34 (3), 987–995. Liptser, R.Sh., Shiryaev, A.N., 1989. Theory of Martingales. Kluwer Academic Publishers, Dordrecht The Netherlands. Meyn, S.P., Tweedie, R.L., 1996. Markov Chains and Stochastic Stability, 3rd printing, Springer-Verlag, New York. Moore, E.H., Pyke, R., 1968. Estimation of the transition distributions of a Markov renewal process. Ann. Inst. Statist. Math. 20, 411–424. Ouhbi, B., Limnios, N., 1999. Nonparametric estimation for semi-Markov processes based on its hazard rate functions. Statist. Inf. Stochast. Process. 2 (2), 151–173. Serfozo, R., 1975. Functional limit theorems for stochastic processes based on embedded processes. Adv. in Appl. Probab. 7, 123–139.