- Email: [email protected]
A class of random deviation theorems and the approach of Laplace transform
Statistics and Probability Letters 79 (2009) 202–210
Contents lists available at ScienceDirect
Statistics and Probability Letters journal homepage: www.elsevier.com/locate/stapro
A class of random deviation theorems and the approach of Laplace transform Gaorong Li a,c,∗ , Shuang Chen b , Junhua Zhang c a
School of Finance and Statistics, East China Normal University, Shanghai 200241, PR China
b
School of Sciences, Hebei University of Technology, Tianjin 300130, PR China
c
Beijing University of Technology, Beijing 100022, PR China
article
info
a b s t r a c t
Article history: Received 3 December 2004 Received in revised form 30 September 2007 Accepted 23 July 2008 Available online 13 August 2008
In this paper, we use the notion of log likelihood ratio to obtain the limit properties of the sequences of dependent nonnegative integrable random variables. A kind of strong limit theorem represented by inequalities which we call the strong deviation theorem is obtained. The bounds given by these theorems depend on sample point. Crown Copyright © 2008 Published by Elsevier B.V. All rights reserved.
MSC: primary 60F15 secondary 60B12
1. Introduction Let {Xn , n ≥ 1} be a sequence of nonnegative integrable random variables on the probability space (Ω , F , P ). Assume that for any n, the vector (X1 , X2 , . . . , Xn ) has a joint distribution density function fn (x1 , . . . , xnQ ), where xk ≥ 0, 1 ≤ k ≤ n. n Let fk (xk ) stand for the density of the random variable Xk (k = 1, 2, . . .). Let πn (x1 , . . . , xn ) = k=1 fk (xk ) be the reference product density function. Let rn (ω) = ln [fn (X1 , . . . , Xn )/πn (x1 , . . . , xn )]
"
# Y n = ln fn (X1 , . . . , Xn ) fk (Xk ) ,
(1)
k=1
where ω is a sample point. In statistical terms, rn (ω) is called the likelihood ratio, which is of fundamental importance in the theory of testing the statistical hypotheses (cf. Laha and Rohatgi (1979, p. 388) and Billingsley (1986, p. 483)). Let r (ω) = lim sup n→∞
1 n
rn (ω),
(2)
with ln 0 = −∞.r (ω) is called asymptotic log-likelihood ratio. Although r (ω) is not a proper metric between probability measures, we Q nevertheless think of it as a measure of n ‘‘dissimilarity’’ between their joint distribution fn (x1 , . . . , xn ) and the product k=1 fk (xk ) of their marginals. Obviously,
∗
Corresponding address: Department of Statistics, School of Finance and Statistics, East China Normal University, Shanghai 200241, PR China. E-mail addresses: [email protected] (G. Li), [email protected] (S. Chen), [email protected] (J. Zhang).
0167-7152/$ – see front matter Crown Copyright © 2008 Published by Elsevier B.V. All rights reserved. doi:10.1016/j.spl.2008.07.048
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
203
r (ω) = 0 if and only if fn (x1 , . . . , xn ) =
n Y
fk (xk ),
n ≥ 1,
(3)
k=1
and it will be shown in (30) that r (ω) ≥ 0 a.s. in any case. Hence, r (ω) can be used as a random measure of the deviation between the true Q joint distribution density function fn (x1 , . . . , xn ) (n > 1) and the reference product density function πn (x1 , . . . , xn ) = nk=1 fk (xk ). Roughly speaking, this deviation may be regarded as the one between {Xn , n ≥ 1} and the independence case. The smaller r (ω) is, the smaller the deviation is. In this paper, we extend the analytic technique proposed by Liu (cf. Liu (1990), Liu and Wang (2002) and Liu (2003)) to the case of dependent nonnegative integrable random variables. Yang (2003) studied the limit properties for Markov chains indexed by a homogeneous tree through the analytic technique. The purpose of the paper is to establish a kind of strong limit theorem represented by inequalities with random bounds for the dependent random variables, that is, to study the expressions lim inf n→∞
n 1X
n k =1
(Xk − mk ),
lim sup n→∞
n 1X
n k=1
(Xk − mk ),
where mk = EXk is the mean of Xk (k = 1, 2, . . .). We provide a lower bound for the lim inf and an upper bound for the lim sup in terms of some functions of the Laplace transforms of the tails of Xn (n = 1, 2, . . .), the martingale convergence theorem and so called asymptotic log-likelihood ratio defined by (2). In the proof, the approach of applying the tool of Laplace transform to the study of strong limit theorem is proposed. Definition 1. Let {Xn , n ≥ 1} be a sequence of random variables, and fk (xk ), k = 1, 2, . . . , n be the marginal density functions of fn (x1 , . . . , xn ) (n = 1, 2, . . .), let Laplace transform and tail probability Laplace transform as follows: f˜k (s) =
+∞
Z
e−sxk fk (xk )dxk ,
(4)
0
and Qk (s) =
+∞
Z
e−sx qk (x)dx,
(5)
fk (xk )dxk ,
(6)
0
where qk (x) =
+∞
Z
x ≥ 0.
x
In this paper, we assume that there exists s0 ∈ (0, +∞), such that f˜k (s) < ∞,
s ∈ [−s0 , s0 ], k = 1, 2, . . . , n.
(7)
In order to prove our main results, we first give two lemmas, and they will be shown that it plays a central role in the proofs. Lemma 1. Let f˜k (s), Qk (s), qk (x) be defined by (4)–(6), respectively, then Qk (s) =
1 − f˜k (s) s
,
(8)
and Qk (0) = EXk = mk ,
k = 1, 2, . . . , n.
(9)
Proof. By (5) and (6), we have Qk (s) =
+∞
Z
e−sx qk (x)dx 0
+∞
Z
e−sx
=
Z
0
e−sx 0
fk (xk )dxk
dx
x
+∞
Z =
+∞
Z 1− 0
x
fk (xk )dxk
dx
204
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
+∞
Z
e−sx dx −
= 1 s
e−sx
+∞
Z
e−sx
−
x
Z
fk (xk )dxk
dx
0
0
0
=
+∞
Z
x
Z
fk (xk )dxk
dx.
(10)
0
0
Using the property of the Laplace transform of integral (cf. Joseph and Ward (1999)) x
Z
fk (xk )dxk
£
+∞
Z
e−sx
=
fk (xk )dxk
dx =
0
0
0
x
Z
By (10) and (11), (8) follows.
f˜k (s) s
.
(11)
Remark. £ denotes the Laplace transform of
Rx 0
fk (xk )dxk .
Lemma 2. Let fn (x1 , x2 , . . . , xn ), gn (x1 , x2 , . . . , xn ) be two probability density functions on (Ω , F , P ). Let tn (ω) =
gn (X1 , X2 , . . . , Xn )
,
(12)
ln tn (ω) ≤ 0 a.s.
(13)
fn (X1 , X2 , . . . , Xn )
then lim sup n
1 n
Proof. Apparently {tn , Fn , n ≥ 1} is a nonnegative martingale, and Etn = 1 (cf. Doob (1953)), we have by the Doob’s martingale convergence theorem, there exists a integral random variable t∞ (ω), such that tn (·) → t∞ (·) a.s. and (13) follows. 2. Main results Theorem 1. Let {Xn , n ≥ 1} be a sequence of nonnegative integrable random variables on the probability space (Ω , F , P ), r (ω), f˜k (s), Qk (s), qk (x) be given as above, and f˜k (s) is defined in [−s0 , s0 ], let Z +∞ mk = xk fk (xk )dxk < +∞, k = 1, 2, . . . , n, (14) 0
D = {ω : r (ω) < ∞} ,
P (D) = 1.
(15)
If there exists function q(x), x ≥ 0, such that qk (x) ≤ q(x),
x ≥ 0, k = 1, 2, . . . , n,
(16)
+∞
Z
q(x)dx = m < ∞.
(17)
0
Then lim inf n→∞
n 1X
n k=1
(Xk − mk ) ≥ α(r (ω)) a.s.
(18)
where
α(x) = sup {ϕ(s, x), 0 < s ≤ s0 } , x ≥ 0, n 1X x ϕ(s, x) = lim inf [Qk (s) − Qk (0)] − , n→∞
n k=1
s
(19) 0 < s ≤ s0 ,
(20)
and then
α(x) ≤ 0,
lim α(x) = α(0) = 0.
x→0+
(21)
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
205
Proof. For arbitrary s ∈ [−s0 , s0 ], let gk (s, xk ) = e−sxk fk (xk )/f˜k (s).
(22)
Then +∞
Z
gk (s, xk )dxk = 1.
(23)
0
Let qn (s, x1 , x2 , . . . , xn ) =
n Y
n h Y
gk (s, xk ) =
k=1
=
k=1
1 n Q
i
e−sXk fk (xk )/f˜k (s)
exp −s
f˜k (s)
n X
! ·
Xk
k=1
n Y
fk (xk ).
(24)
k=1
k=1
Therefore, qn (s, x1 , x2 , . . . , xn ) is an n multivariate probability density function, let tn (s, ω) =
qn (s, X1 , . . . , Xn ) fn (X1 , . . . , Xn )
.
(25)
It is easy to see that tn (s, ω) is a martingale. By Lemma 2, there exists A(s) ∈ F , P (A(s)) = 1, such that lim tn (s, ω) < ∞,
ω ∈ A(s).
n→∞
This implies that lim sup n→∞
1 n
ln tn (s, ω) ≤ 0,
ω ∈ A(s).
(26)
By (24)–(26), we obtain
lim sup n→∞
1 n
1 n Q
ln
n P
exp −s
f˜k (s)
n Q
·
Xk
k=1
k =1
fk (xk )
k=1
f n ( x1 , . . . , xn )
≤ 0,
ω ∈ A(s),
(27)
that is lim sup n→∞
1 n
( −
n X
ln f˜k (s) − s
n X
k=1
) Xk − rn (ω)
≤ 0,
ω ∈ A(s).
(28)
k=1
By (2) and (28), we have lim sup n→∞
n 1X
n k=1
Xk (−s) ≤ lim sup n→∞
n 1X
n k=1
ln f˜k (s) + r (ω),
ω ∈ A(s).
(29)
Letting s = 0 in (29), we obtain r (ω) ≥ 0,
ω ∈ A(0).
(30)
Let 0 < s ≤ s0 . Then, dividing the two sides of (29) by −s, we obtain lim inf n→∞
n 1X
n k =1
Xk ≥ lim inf n→∞
n 1X
n k=1
−
f˜k (s) s
! −
r (ω) s
,
ω ∈ A(s).
By (14) and (31), the property of the inferior limit lim inf(an − bn ) ≥ d ⇒ lim inf(an − cn ) ≥ lim inf(bn − cn ) + d n→∞
n→∞
n→∞
(31)
206
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
and the inequality ln x ≤ x − 1, (x > 0), and Lemma 1, for ω ∈ A(s), we have lim inf n→∞
n 1X
n k=1
(Xk − mk ) ≥ lim inf n→∞
≥ lim inf n→∞
= lim inf n→∞
n 1X
" −
n k=1 n 1X
"
n k=1
−
#
ln f˜k (s)
− mk −
s
r (ω) s
#
f˜k (s) − 1
r (ω)
− mk −
s
s
n 1X
r (ω)
n k=1
s
[Qk (s) − Qk (0)] −
.
(32)
Let Q + be the set of rational numbers in the interval (0, s0 ] and let A∗ = ∩s∈Q + A(s), then P (A∗ ) = 1. By (32), we have lim inf n→∞
n 1X
n k=1
(Xk − mk ) ≥ lim inf n→∞
n 1X
r (ω)
n k=1
s
[Qk (s) − Qk (0)] −
,
ω ∈ A∗ , ∀s ∈ Q + .
(33)
Let n 1X
g (s) = lim inf
n k =1
n→∞
[Qk (s) − Qk (0)],
0 < s ≤ s0 .
(34)
By (20) and (34), we have x
ϕ(s, x) = g (s) − , 0 < s ≤ s0 , x ≥ 0, s n o x α(x) = sup g (s) − , 0 < s ≤ s0 .
(35) (36)
s
By L’Hospital rule, we have lim Qk (s) = lim f˜k0 (s) = mk .
s→0
s→0
Obviously g (s) ≤ 0, ϕ(s, x) ≤ 0, hence α(x) ≤ 0. Let Q (s) denote the Laplace transform of q(x), (x ≥ 0) Q (s) =
Z
+∞
e−sx q(x)dx.
(37)
0
If 0 ≤ s − t < s ≤ s0 < ∞, by (34) and (5) and noticing that qk (x) ≤ q(x)(x ≥ 0, k = 1, 2, . . . , n), we have 0 < g ( s − t ) − g ( s)
= lim inf n→∞
= lim inf n→∞
n 1X
n k=1
n→∞
n 1X
n k=1
≤ lim sup n→∞
= lim sup n→∞
≤ lim sup n→∞
[Qk (s − t ) − Qk (0)] − lim inf
n 1X
n k=1
[Qk (s − t ) − Qk (0)] + lim sup n→∞
[Qk (s) − Qk (0)]
n 1X
n k=1
[Qk (0) − Qk (s)]
n 1X
n k =1
[Qk (s − t ) − Qk (s)]
n 1X
n k =1 1
+∞
Z
e−sx qk (x)dx
0
+∞
X Z
+∞
Z
0
n
n k =1
e−(s−t )x qk (x)dx − e−(s−t )x q(x)dx −
0
= Q (s − t ) − Q (s).
+∞
Z
e−sx q(x)dx 0
(38)
By (38), we know that g (s) is a continuous function with respect to s on the interval [0, s0 ]. Now it is easy to see that ϕ(s, x) is also a continuous function with respect to s on the interval [0, s0 ]. By (36), for each ω ∈ A∗ ∩ A(0) ∩ D, take sn (ω) ∈ Q + , (n = 1, 2, . . .), such that lim ϕ(sn (ω), r (ω)) = α(r (ω)).
n→∞
(39)
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
207
By (33)–(36), we have lim inf n→∞
n 1X
n k =1
(Xk − mk ) ≥ ϕ(sn (ω), r (ω)),
ω ∈ A∗ ∩ A(0) ∩ D,
n = 1, 2, . . . .
(40)
By (39) and (40), we have lim inf n→∞
n 1X
n k =1
(Xk − mk ) ≥ α(r (ω)),
ω ∈ A∗ ∩ A(0) ∩ D.
(41)
Since P (A∗ ∩ A(0) ∩ D) = 1, (18) holds by (41). When 0 < s ≤ s0 , from (16) and (37), we have n 1X
n k=1
[Qk (s) − Qk (0)] = = ≥ =
n 1X
+∞
Z
n k=1
e−sx qk (x)dx − +∞
X Z
1
n k=1
qk (x)dx
0
0
n
+∞
Z
qk (x)(e−sx − 1)dx
0
n Z 1X
n k=1
+∞
q(x)(e
−sx
− 1)dx , (e−sx − 1 ≤ 0, qk (x) ≤ q(x))
0
n 1X
+∞
Z
n k=1
e−sx q(x)dx − 0
+∞
Z
q(x)dx
0
= Q (s) − Q (0).
(42)
For x > 0, we have
√ √ √ √ x α(x) ≥ ϕ( x, x) = g ( x) − √ ≥ Q ( x) − Q (0) − x. x
(43)
While for x = 0, we have
r ! 1
α(0) ≥ g
n
r ! ≥Q
1 n
− Q (0),
n ≥ 1.
(44)
Noticing that α(x) ≤ 0, (x ≥ 0), (21) follows from (43) and (44). Corollary 1. Under the condition of the Theorem 1, then lim inf n→∞
n 1X
n k =1
(Xk − mk ) ≥ α∗ (r (ω)) a.s.
(45)
where
o n x α∗ (x) = sup Q (s) − Q (0) − , 0 < s ≤ s0 , s
x ≥ 0.
(46)
Let Q (s) be defined by (37), and then
α∗ (x) ≤ 0,
lim α∗ (x) = α∗ (0) = 0.
(47)
n→∞
Proof. By (42), since x
ϕ(s, x) ≥ Q (s) − Q (0) − , s
0 < s ≤ s0 ,
x ≥ 0.
It is easy to see that α∗ (x) ≤ α(x). Therefore, (45) holds by (18) and (47) can be obtained by imitating the proof of (21).
(48)
Theorem 2. Let {Xn , n ≥ 1} be a sequence of nonnegative integrable random variables on the probability space (Ω , F , P ), r (ω), f˜k (s), Qk (s), qk (s), mk be given as above. If (15) holds, there exists function q(x) (x ≥ 0) satisfying the following condition n 1X
n k=1
qk (x) ≤ q(x),
x ≥ 0,
n ≥ 1,
(49)
+∞
Z
q(x)dx < ∞. 0
(50)
208
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
Then lim inf n→∞
n 1X
n k=1
(Xk − mk ) ≥ α(r (ω)) a.s.
(51)
where
o n x α(x) = sup Q (s) − Q (0) − , 0 < s ≤ s0 , s Z +∞ e−sx q(x)dx, Q (s) =
x ≥ 0,
(52) (53)
0
and then
α(x) ≤ 0,
x ≥ 0,
lim α(x) = α(0) = 0.
(54)
x→0+
Theorem 3. Under the condition of the Theorem 1, then lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ β(r (ω)) a.s.
(55)
where
β(x) = inf {ψ(s, x), −s0 ≤ s < 0} , x ≥ 0, n 1X x ψ(s, x) = lim sup [Qk (s) − Qk (0)] − , n→∞
n k=1
(56) x ≥ 0,
s
(57)
and then
β(x) ≥ 0,
x ≥ 0,
lim β(x) = β(0) = 0.
(58)
x→0+
Proof. Let −s0 ≤ s < 0, dividing the two sides of (29) by −s, we obtain lim sup n→∞
n 1X
n k=1
Xk ≤ lim sup n→∞
n 1X
n k=1
−
f˜k (s)
! −
s
r (ω) s
,
ω ∈ A(s).
(59)
By (14) and (59), the property of the superior limit lim sup(an − bn ) ≤ d ⇒ lim sup(an − cn ) ≤ lim sup(bn − cn ) + d n→∞
n→∞
n→∞
and the inequality ln x ≤ x − 1, (x > 0), and Lemma 1, for ω ∈ A(s), we have lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ lim sup n→∞
n 1X
n k=1 n
≤ lim sup n→∞
= lim sup n→∞
1X n k=1 n 1X
n k=1
" − " −
ln f˜k (s) s
# − mk −
f˜k (s) − 1 s
r (ω) s
#
r (ω)
− mk −
[Qk (s) − Qk (0)] −
s
r (ω) s
.
(60)
Let Q − be the set of rational numbers in the interval [−s0 , 0), and let A∗ = ∩s∈Q − A(s), then P (A∗ ) = 1. By (60), we have lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ lim sup n→∞
n 1X
n k=1
[Qk (s) − Qk (0)] −
r (ω) s
,
ω ∈ A∗ , ∀s ∈ Q − .
(61)
Let g (s) = lim sup n→∞
n 1X
n k=1
[Qk (s) − Qk (0)],
−s0 ≤ s < 0.
(62)
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
209
By (57) and (62), we have x
ψ(s, x) = g (s) − , −s0 ≤ s < 0, x ≥ 0, s n o x β(x) = inf g (s) − , −s0 ≤ s < 0 .
(63) (64)
s
Obviously g (s) ≥ 0, ψ(s, x) ≥ 0, hence β(x) ≥ 0. By imitating (37) and (38), we can also know that g (s) is a continuous function with respect to s on the interval [−s0 , 0], then it is easy to see that ψ(s, x) is also a continuous function with respect to s on the interval [−s0 , 0]. By (56), for each ω ∈ A∗ ∩ A(0) ∩ D, take λn (ω) ∈ Q − , (n = 1, 2, . . .), such that lim ψ(λn (ω), r (ω)) = β(r (ω)).
(65)
n→∞
By (61)–(64), it can be obtained that lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ ψ(λn (ω), r (ω)),
ω ∈ A∗ ∩ A(0) ∩ D,
n = 1, 2, . . . .
(66)
By (65) and (66), we have lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ β(r (ω)),
ω ∈ A∗ ∩ A(0) ∩ D.
(67)
Since P (A∗ ∩ A(0) ∩ D) = 1, (55) holds by (67). When −s0 ≤ s < 0, we have like (42) n 1X
n k=1
[Qk (s) − Qk (0)] ≤ Q (s) − Q (0),
(e−sx − 1 ≥ 0, qk (x) ≤ q(x)).
(68)
For x > 0, we have
√ √ √ √ x β(x) ≤ ψ( x, x) = g ( x) − √ ≤ Q ( x) − Q (0) − x.
(69)
x
While for x = 0, we have
r ! 1
β(0) ≤ g
n
r ! 1
≤Q
− Q (0),
n
n ≥ 1.
(70)
Noticing that β(x) ≥ 0, (x ≥ 0), (58) follows from (69) and (70).
Theorem 4. Under the condition of the Theorem 2, then lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ β(r (ω)) a.s.
(71)
where
n o x β(x) = inf Q (s) − Q (0) − , −s0 ≤ s < 0 , s
x ≥ 0,
(72)
and then
β(x) ≥ 0,
x ≥ 0,
lim β(x) = β(0) = 0.
x→0+
(73)
Corollary 2. If {Xn , n ≥ 1} is a sequence of independent nonnegative integrable random variables, then lim
n→∞
n 1X
n k=1
(Xk − mk ) = 0 a.s.
Proof. In this case, fn (x1 , . . . , xn ) =
(74)
Qn
k=1 fk
(xk ), and r (ω) = 0 a.s. Hence (74) follows directly from (21) and (58).
210
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
Acknowledgements This research is supported by China Postdoctoral Science Foundation Funded Project (grant 20080430633), Shanghai Postdoctoral Scientific Program (grant 08R214121), the National Natural Science Foundation of China (grant 10671052, 10571008), the Natural Science Foundation of Beijing (grant 1072004) and the Basic Research and Frontier Technology Foundation of Henan (grant 072300410090). The authors would like to thank the editor and the referees for helpful comments, which helped to improve an earlier version of the paper. References Billingsley, P., 1986. Probability and Measure. Wiley, New York. Doob, J.L., 1953. Stochastic Processes. Wiley, New York. Joseph, A., Ward, W., 1999. Infinite-series representations of Laplace transforms of probability density functions for numerical inversion. J. Oper. Res. Soc. Japan 42 (3), 268–285. Laha, R.G., Rohatgi, V.K., 1979. Probability Theory. John Wiley & Sons, New York. Liu, W., 1990. Relative entropy densities and a class of limit theorems of sequence of m-valued random variables. Ann. Probab. 18 (2), 829–839. Liu, W., 2003. Strong Deviation Theorems and Analytic Method. Science Press, Beijing (in Chinese). Liu, W., Wang, Y.J., 2002. A strong limit theorem expressed by inequalities for the sequences of absolutely continuous random variables. Hiroshima Math. J. 32, 379–387. Yang, W.G., 2003. Some limit properties for Markov chains indexed by a homogeneous tree. Statist. Probab. Lett. 65, 241–250.
Contents lists available at ScienceDirect
Statistics and Probability Letters journal homepage: www.elsevier.com/locate/stapro
A class of random deviation theorems and the approach of Laplace transform Gaorong Li a,c,∗ , Shuang Chen b , Junhua Zhang c a
School of Finance and Statistics, East China Normal University, Shanghai 200241, PR China
b
School of Sciences, Hebei University of Technology, Tianjin 300130, PR China
c
Beijing University of Technology, Beijing 100022, PR China
article
info
a b s t r a c t
Article history: Received 3 December 2004 Received in revised form 30 September 2007 Accepted 23 July 2008 Available online 13 August 2008
In this paper, we use the notion of log likelihood ratio to obtain the limit properties of the sequences of dependent nonnegative integrable random variables. A kind of strong limit theorem represented by inequalities which we call the strong deviation theorem is obtained. The bounds given by these theorems depend on sample point. Crown Copyright © 2008 Published by Elsevier B.V. All rights reserved.
MSC: primary 60F15 secondary 60B12
1. Introduction Let {Xn , n ≥ 1} be a sequence of nonnegative integrable random variables on the probability space (Ω , F , P ). Assume that for any n, the vector (X1 , X2 , . . . , Xn ) has a joint distribution density function fn (x1 , . . . , xnQ ), where xk ≥ 0, 1 ≤ k ≤ n. n Let fk (xk ) stand for the density of the random variable Xk (k = 1, 2, . . .). Let πn (x1 , . . . , xn ) = k=1 fk (xk ) be the reference product density function. Let rn (ω) = ln [fn (X1 , . . . , Xn )/πn (x1 , . . . , xn )]
"
# Y n = ln fn (X1 , . . . , Xn ) fk (Xk ) ,
(1)
k=1
where ω is a sample point. In statistical terms, rn (ω) is called the likelihood ratio, which is of fundamental importance in the theory of testing the statistical hypotheses (cf. Laha and Rohatgi (1979, p. 388) and Billingsley (1986, p. 483)). Let r (ω) = lim sup n→∞
1 n
rn (ω),
(2)
with ln 0 = −∞.r (ω) is called asymptotic log-likelihood ratio. Although r (ω) is not a proper metric between probability measures, we Q nevertheless think of it as a measure of n ‘‘dissimilarity’’ between their joint distribution fn (x1 , . . . , xn ) and the product k=1 fk (xk ) of their marginals. Obviously,
∗
Corresponding address: Department of Statistics, School of Finance and Statistics, East China Normal University, Shanghai 200241, PR China. E-mail addresses: [email protected] (G. Li), [email protected] (S. Chen), [email protected] (J. Zhang).
0167-7152/$ – see front matter Crown Copyright © 2008 Published by Elsevier B.V. All rights reserved. doi:10.1016/j.spl.2008.07.048
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
203
r (ω) = 0 if and only if fn (x1 , . . . , xn ) =
n Y
fk (xk ),
n ≥ 1,
(3)
k=1
and it will be shown in (30) that r (ω) ≥ 0 a.s. in any case. Hence, r (ω) can be used as a random measure of the deviation between the true Q joint distribution density function fn (x1 , . . . , xn ) (n > 1) and the reference product density function πn (x1 , . . . , xn ) = nk=1 fk (xk ). Roughly speaking, this deviation may be regarded as the one between {Xn , n ≥ 1} and the independence case. The smaller r (ω) is, the smaller the deviation is. In this paper, we extend the analytic technique proposed by Liu (cf. Liu (1990), Liu and Wang (2002) and Liu (2003)) to the case of dependent nonnegative integrable random variables. Yang (2003) studied the limit properties for Markov chains indexed by a homogeneous tree through the analytic technique. The purpose of the paper is to establish a kind of strong limit theorem represented by inequalities with random bounds for the dependent random variables, that is, to study the expressions lim inf n→∞
n 1X
n k =1
(Xk − mk ),
lim sup n→∞
n 1X
n k=1
(Xk − mk ),
where mk = EXk is the mean of Xk (k = 1, 2, . . .). We provide a lower bound for the lim inf and an upper bound for the lim sup in terms of some functions of the Laplace transforms of the tails of Xn (n = 1, 2, . . .), the martingale convergence theorem and so called asymptotic log-likelihood ratio defined by (2). In the proof, the approach of applying the tool of Laplace transform to the study of strong limit theorem is proposed. Definition 1. Let {Xn , n ≥ 1} be a sequence of random variables, and fk (xk ), k = 1, 2, . . . , n be the marginal density functions of fn (x1 , . . . , xn ) (n = 1, 2, . . .), let Laplace transform and tail probability Laplace transform as follows: f˜k (s) =
+∞
Z
e−sxk fk (xk )dxk ,
(4)
0
and Qk (s) =
+∞
Z
e−sx qk (x)dx,
(5)
fk (xk )dxk ,
(6)
0
where qk (x) =
+∞
Z
x ≥ 0.
x
In this paper, we assume that there exists s0 ∈ (0, +∞), such that f˜k (s) < ∞,
s ∈ [−s0 , s0 ], k = 1, 2, . . . , n.
(7)
In order to prove our main results, we first give two lemmas, and they will be shown that it plays a central role in the proofs. Lemma 1. Let f˜k (s), Qk (s), qk (x) be defined by (4)–(6), respectively, then Qk (s) =
1 − f˜k (s) s
,
(8)
and Qk (0) = EXk = mk ,
k = 1, 2, . . . , n.
(9)
Proof. By (5) and (6), we have Qk (s) =
+∞
Z
e−sx qk (x)dx 0
+∞
Z
e−sx
=
Z
0
e−sx 0
fk (xk )dxk
dx
x
+∞
Z =
+∞
Z 1− 0
x
fk (xk )dxk
dx
204
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
+∞
Z
e−sx dx −
= 1 s
e−sx
+∞
Z
e−sx
−
x
Z
fk (xk )dxk
dx
0
0
0
=
+∞
Z
x
Z
fk (xk )dxk
dx.
(10)
0
0
Using the property of the Laplace transform of integral (cf. Joseph and Ward (1999)) x
Z
fk (xk )dxk
£
+∞
Z
e−sx
=
fk (xk )dxk
dx =
0
0
0
x
Z
By (10) and (11), (8) follows.
f˜k (s) s
.
(11)
Remark. £ denotes the Laplace transform of
Rx 0
fk (xk )dxk .
Lemma 2. Let fn (x1 , x2 , . . . , xn ), gn (x1 , x2 , . . . , xn ) be two probability density functions on (Ω , F , P ). Let tn (ω) =
gn (X1 , X2 , . . . , Xn )
,
(12)
ln tn (ω) ≤ 0 a.s.
(13)
fn (X1 , X2 , . . . , Xn )
then lim sup n
1 n
Proof. Apparently {tn , Fn , n ≥ 1} is a nonnegative martingale, and Etn = 1 (cf. Doob (1953)), we have by the Doob’s martingale convergence theorem, there exists a integral random variable t∞ (ω), such that tn (·) → t∞ (·) a.s. and (13) follows. 2. Main results Theorem 1. Let {Xn , n ≥ 1} be a sequence of nonnegative integrable random variables on the probability space (Ω , F , P ), r (ω), f˜k (s), Qk (s), qk (x) be given as above, and f˜k (s) is defined in [−s0 , s0 ], let Z +∞ mk = xk fk (xk )dxk < +∞, k = 1, 2, . . . , n, (14) 0
D = {ω : r (ω) < ∞} ,
P (D) = 1.
(15)
If there exists function q(x), x ≥ 0, such that qk (x) ≤ q(x),
x ≥ 0, k = 1, 2, . . . , n,
(16)
+∞
Z
q(x)dx = m < ∞.
(17)
0
Then lim inf n→∞
n 1X
n k=1
(Xk − mk ) ≥ α(r (ω)) a.s.
(18)
where
α(x) = sup {ϕ(s, x), 0 < s ≤ s0 } , x ≥ 0, n 1X x ϕ(s, x) = lim inf [Qk (s) − Qk (0)] − , n→∞
n k=1
s
(19) 0 < s ≤ s0 ,
(20)
and then
α(x) ≤ 0,
lim α(x) = α(0) = 0.
x→0+
(21)
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
205
Proof. For arbitrary s ∈ [−s0 , s0 ], let gk (s, xk ) = e−sxk fk (xk )/f˜k (s).
(22)
Then +∞
Z
gk (s, xk )dxk = 1.
(23)
0
Let qn (s, x1 , x2 , . . . , xn ) =
n Y
n h Y
gk (s, xk ) =
k=1
=
k=1
1 n Q
i
e−sXk fk (xk )/f˜k (s)
exp −s
f˜k (s)
n X
! ·
Xk
k=1
n Y
fk (xk ).
(24)
k=1
k=1
Therefore, qn (s, x1 , x2 , . . . , xn ) is an n multivariate probability density function, let tn (s, ω) =
qn (s, X1 , . . . , Xn ) fn (X1 , . . . , Xn )
.
(25)
It is easy to see that tn (s, ω) is a martingale. By Lemma 2, there exists A(s) ∈ F , P (A(s)) = 1, such that lim tn (s, ω) < ∞,
ω ∈ A(s).
n→∞
This implies that lim sup n→∞
1 n
ln tn (s, ω) ≤ 0,
ω ∈ A(s).
(26)
By (24)–(26), we obtain
lim sup n→∞
1 n
1 n Q
ln
n P
exp −s
f˜k (s)
n Q
·
Xk
k=1
k =1
fk (xk )
k=1
f n ( x1 , . . . , xn )
≤ 0,
ω ∈ A(s),
(27)
that is lim sup n→∞
1 n
( −
n X
ln f˜k (s) − s
n X
k=1
) Xk − rn (ω)
≤ 0,
ω ∈ A(s).
(28)
k=1
By (2) and (28), we have lim sup n→∞
n 1X
n k=1
Xk (−s) ≤ lim sup n→∞
n 1X
n k=1
ln f˜k (s) + r (ω),
ω ∈ A(s).
(29)
Letting s = 0 in (29), we obtain r (ω) ≥ 0,
ω ∈ A(0).
(30)
Let 0 < s ≤ s0 . Then, dividing the two sides of (29) by −s, we obtain lim inf n→∞
n 1X
n k =1
Xk ≥ lim inf n→∞
n 1X
n k=1
−
f˜k (s) s
! −
r (ω) s
,
ω ∈ A(s).
By (14) and (31), the property of the inferior limit lim inf(an − bn ) ≥ d ⇒ lim inf(an − cn ) ≥ lim inf(bn − cn ) + d n→∞
n→∞
n→∞
(31)
206
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
and the inequality ln x ≤ x − 1, (x > 0), and Lemma 1, for ω ∈ A(s), we have lim inf n→∞
n 1X
n k=1
(Xk − mk ) ≥ lim inf n→∞
≥ lim inf n→∞
= lim inf n→∞
n 1X
" −
n k=1 n 1X
"
n k=1
−
#
ln f˜k (s)
− mk −
s
r (ω) s
#
f˜k (s) − 1
r (ω)
− mk −
s
s
n 1X
r (ω)
n k=1
s
[Qk (s) − Qk (0)] −
.
(32)
Let Q + be the set of rational numbers in the interval (0, s0 ] and let A∗ = ∩s∈Q + A(s), then P (A∗ ) = 1. By (32), we have lim inf n→∞
n 1X
n k=1
(Xk − mk ) ≥ lim inf n→∞
n 1X
r (ω)
n k=1
s
[Qk (s) − Qk (0)] −
,
ω ∈ A∗ , ∀s ∈ Q + .
(33)
Let n 1X
g (s) = lim inf
n k =1
n→∞
[Qk (s) − Qk (0)],
0 < s ≤ s0 .
(34)
By (20) and (34), we have x
ϕ(s, x) = g (s) − , 0 < s ≤ s0 , x ≥ 0, s n o x α(x) = sup g (s) − , 0 < s ≤ s0 .
(35) (36)
s
By L’Hospital rule, we have lim Qk (s) = lim f˜k0 (s) = mk .
s→0
s→0
Obviously g (s) ≤ 0, ϕ(s, x) ≤ 0, hence α(x) ≤ 0. Let Q (s) denote the Laplace transform of q(x), (x ≥ 0) Q (s) =
Z
+∞
e−sx q(x)dx.
(37)
0
If 0 ≤ s − t < s ≤ s0 < ∞, by (34) and (5) and noticing that qk (x) ≤ q(x)(x ≥ 0, k = 1, 2, . . . , n), we have 0 < g ( s − t ) − g ( s)
= lim inf n→∞
= lim inf n→∞
n 1X
n k=1
n→∞
n 1X
n k=1
≤ lim sup n→∞
= lim sup n→∞
≤ lim sup n→∞
[Qk (s − t ) − Qk (0)] − lim inf
n 1X
n k=1
[Qk (s − t ) − Qk (0)] + lim sup n→∞
[Qk (s) − Qk (0)]
n 1X
n k=1
[Qk (0) − Qk (s)]
n 1X
n k =1
[Qk (s − t ) − Qk (s)]
n 1X
n k =1 1
+∞
Z
e−sx qk (x)dx
0
+∞
X Z
+∞
Z
0
n
n k =1
e−(s−t )x qk (x)dx − e−(s−t )x q(x)dx −
0
= Q (s − t ) − Q (s).
+∞
Z
e−sx q(x)dx 0
(38)
By (38), we know that g (s) is a continuous function with respect to s on the interval [0, s0 ]. Now it is easy to see that ϕ(s, x) is also a continuous function with respect to s on the interval [0, s0 ]. By (36), for each ω ∈ A∗ ∩ A(0) ∩ D, take sn (ω) ∈ Q + , (n = 1, 2, . . .), such that lim ϕ(sn (ω), r (ω)) = α(r (ω)).
n→∞
(39)
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
207
By (33)–(36), we have lim inf n→∞
n 1X
n k =1
(Xk − mk ) ≥ ϕ(sn (ω), r (ω)),
ω ∈ A∗ ∩ A(0) ∩ D,
n = 1, 2, . . . .
(40)
By (39) and (40), we have lim inf n→∞
n 1X
n k =1
(Xk − mk ) ≥ α(r (ω)),
ω ∈ A∗ ∩ A(0) ∩ D.
(41)
Since P (A∗ ∩ A(0) ∩ D) = 1, (18) holds by (41). When 0 < s ≤ s0 , from (16) and (37), we have n 1X
n k=1
[Qk (s) − Qk (0)] = = ≥ =
n 1X
+∞
Z
n k=1
e−sx qk (x)dx − +∞
X Z
1
n k=1
qk (x)dx
0
0
n
+∞
Z
qk (x)(e−sx − 1)dx
0
n Z 1X
n k=1
+∞
q(x)(e
−sx
− 1)dx , (e−sx − 1 ≤ 0, qk (x) ≤ q(x))
0
n 1X
+∞
Z
n k=1
e−sx q(x)dx − 0
+∞
Z
q(x)dx
0
= Q (s) − Q (0).
(42)
For x > 0, we have
√ √ √ √ x α(x) ≥ ϕ( x, x) = g ( x) − √ ≥ Q ( x) − Q (0) − x. x
(43)
While for x = 0, we have
r ! 1
α(0) ≥ g
n
r ! ≥Q
1 n
− Q (0),
n ≥ 1.
(44)
Noticing that α(x) ≤ 0, (x ≥ 0), (21) follows from (43) and (44). Corollary 1. Under the condition of the Theorem 1, then lim inf n→∞
n 1X
n k =1
(Xk − mk ) ≥ α∗ (r (ω)) a.s.
(45)
where
o n x α∗ (x) = sup Q (s) − Q (0) − , 0 < s ≤ s0 , s
x ≥ 0.
(46)
Let Q (s) be defined by (37), and then
α∗ (x) ≤ 0,
lim α∗ (x) = α∗ (0) = 0.
(47)
n→∞
Proof. By (42), since x
ϕ(s, x) ≥ Q (s) − Q (0) − , s
0 < s ≤ s0 ,
x ≥ 0.
It is easy to see that α∗ (x) ≤ α(x). Therefore, (45) holds by (18) and (47) can be obtained by imitating the proof of (21).
(48)
Theorem 2. Let {Xn , n ≥ 1} be a sequence of nonnegative integrable random variables on the probability space (Ω , F , P ), r (ω), f˜k (s), Qk (s), qk (s), mk be given as above. If (15) holds, there exists function q(x) (x ≥ 0) satisfying the following condition n 1X
n k=1
qk (x) ≤ q(x),
x ≥ 0,
n ≥ 1,
(49)
+∞
Z
q(x)dx < ∞. 0
(50)
208
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
Then lim inf n→∞
n 1X
n k=1
(Xk − mk ) ≥ α(r (ω)) a.s.
(51)
where
o n x α(x) = sup Q (s) − Q (0) − , 0 < s ≤ s0 , s Z +∞ e−sx q(x)dx, Q (s) =
x ≥ 0,
(52) (53)
0
and then
α(x) ≤ 0,
x ≥ 0,
lim α(x) = α(0) = 0.
(54)
x→0+
Theorem 3. Under the condition of the Theorem 1, then lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ β(r (ω)) a.s.
(55)
where
β(x) = inf {ψ(s, x), −s0 ≤ s < 0} , x ≥ 0, n 1X x ψ(s, x) = lim sup [Qk (s) − Qk (0)] − , n→∞
n k=1
(56) x ≥ 0,
s
(57)
and then
β(x) ≥ 0,
x ≥ 0,
lim β(x) = β(0) = 0.
(58)
x→0+
Proof. Let −s0 ≤ s < 0, dividing the two sides of (29) by −s, we obtain lim sup n→∞
n 1X
n k=1
Xk ≤ lim sup n→∞
n 1X
n k=1
−
f˜k (s)
! −
s
r (ω) s
,
ω ∈ A(s).
(59)
By (14) and (59), the property of the superior limit lim sup(an − bn ) ≤ d ⇒ lim sup(an − cn ) ≤ lim sup(bn − cn ) + d n→∞
n→∞
n→∞
and the inequality ln x ≤ x − 1, (x > 0), and Lemma 1, for ω ∈ A(s), we have lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ lim sup n→∞
n 1X
n k=1 n
≤ lim sup n→∞
= lim sup n→∞
1X n k=1 n 1X
n k=1
" − " −
ln f˜k (s) s
# − mk −
f˜k (s) − 1 s
r (ω) s
#
r (ω)
− mk −
[Qk (s) − Qk (0)] −
s
r (ω) s
.
(60)
Let Q − be the set of rational numbers in the interval [−s0 , 0), and let A∗ = ∩s∈Q − A(s), then P (A∗ ) = 1. By (60), we have lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ lim sup n→∞
n 1X
n k=1
[Qk (s) − Qk (0)] −
r (ω) s
,
ω ∈ A∗ , ∀s ∈ Q − .
(61)
Let g (s) = lim sup n→∞
n 1X
n k=1
[Qk (s) − Qk (0)],
−s0 ≤ s < 0.
(62)
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
209
By (57) and (62), we have x
ψ(s, x) = g (s) − , −s0 ≤ s < 0, x ≥ 0, s n o x β(x) = inf g (s) − , −s0 ≤ s < 0 .
(63) (64)
s
Obviously g (s) ≥ 0, ψ(s, x) ≥ 0, hence β(x) ≥ 0. By imitating (37) and (38), we can also know that g (s) is a continuous function with respect to s on the interval [−s0 , 0], then it is easy to see that ψ(s, x) is also a continuous function with respect to s on the interval [−s0 , 0]. By (56), for each ω ∈ A∗ ∩ A(0) ∩ D, take λn (ω) ∈ Q − , (n = 1, 2, . . .), such that lim ψ(λn (ω), r (ω)) = β(r (ω)).
(65)
n→∞
By (61)–(64), it can be obtained that lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ ψ(λn (ω), r (ω)),
ω ∈ A∗ ∩ A(0) ∩ D,
n = 1, 2, . . . .
(66)
By (65) and (66), we have lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ β(r (ω)),
ω ∈ A∗ ∩ A(0) ∩ D.
(67)
Since P (A∗ ∩ A(0) ∩ D) = 1, (55) holds by (67). When −s0 ≤ s < 0, we have like (42) n 1X
n k=1
[Qk (s) − Qk (0)] ≤ Q (s) − Q (0),
(e−sx − 1 ≥ 0, qk (x) ≤ q(x)).
(68)
For x > 0, we have
√ √ √ √ x β(x) ≤ ψ( x, x) = g ( x) − √ ≤ Q ( x) − Q (0) − x.
(69)
x
While for x = 0, we have
r ! 1
β(0) ≤ g
n
r ! 1
≤Q
− Q (0),
n
n ≥ 1.
(70)
Noticing that β(x) ≥ 0, (x ≥ 0), (58) follows from (69) and (70).
Theorem 4. Under the condition of the Theorem 2, then lim sup n→∞
n 1X
n k=1
(Xk − mk ) ≤ β(r (ω)) a.s.
(71)
where
n o x β(x) = inf Q (s) − Q (0) − , −s0 ≤ s < 0 , s
x ≥ 0,
(72)
and then
β(x) ≥ 0,
x ≥ 0,
lim β(x) = β(0) = 0.
x→0+
(73)
Corollary 2. If {Xn , n ≥ 1} is a sequence of independent nonnegative integrable random variables, then lim
n→∞
n 1X
n k=1
(Xk − mk ) = 0 a.s.
Proof. In this case, fn (x1 , . . . , xn ) =
(74)
Qn
k=1 fk
(xk ), and r (ω) = 0 a.s. Hence (74) follows directly from (21) and (58).
210
G. Li et al. / Statistics and Probability Letters 79 (2009) 202–210
Acknowledgements This research is supported by China Postdoctoral Science Foundation Funded Project (grant 20080430633), Shanghai Postdoctoral Scientific Program (grant 08R214121), the National Natural Science Foundation of China (grant 10671052, 10571008), the Natural Science Foundation of Beijing (grant 1072004) and the Basic Research and Frontier Technology Foundation of Henan (grant 072300410090). The authors would like to thank the editor and the referees for helpful comments, which helped to improve an earlier version of the paper. References Billingsley, P., 1986. Probability and Measure. Wiley, New York. Doob, J.L., 1953. Stochastic Processes. Wiley, New York. Joseph, A., Ward, W., 1999. Infinite-series representations of Laplace transforms of probability density functions for numerical inversion. J. Oper. Res. Soc. Japan 42 (3), 268–285. Laha, R.G., Rohatgi, V.K., 1979. Probability Theory. John Wiley & Sons, New York. Liu, W., 1990. Relative entropy densities and a class of limit theorems of sequence of m-valued random variables. Ann. Probab. 18 (2), 829–839. Liu, W., 2003. Strong Deviation Theorems and Analytic Method. Science Press, Beijing (in Chinese). Liu, W., Wang, Y.J., 2002. A strong limit theorem expressed by inequalities for the sequences of absolutely continuous random variables. Hiroshima Math. J. 32, 379–387. Yang, W.G., 2003. Some limit properties for Markov chains indexed by a homogeneous tree. Statist. Probab. Lett. 65, 241–250.