- Email: [email protected]
A Kolmogorov inequality for weighted U-statistics
Statistics and Probability Letters 78 (2008) 3294–3297
Contents lists available at ScienceDirect
Statistics and Probability Letters journal homepage: www.elsevier.com/locate/stapro
A Kolmogorov inequality for weighted U-statistics Petroula M. Mavrikiou ∗ Department of Business Administration, School of Economic Sciences and Administration, Frederick University Cyprus, P.O. Box 24729, 1303 Nicosia, Cyprus
article
info
a b s t r a c t
Article history: Received 18 June 2007 Received in revised form 28 April 2008 Accepted 2 June 2008 Available online 17 June 2008
In this paper a Kolmogorov probability inequality for weighted U-statistics based on Bernoulli kernels is presented. This inequality which extends the results of [Turner, D.W., Young, D.M., Seaman, J.W., 1995. A Kolmogorov inequality for the sum of independent Bernoulli random variables with unequal means. Statist. Probab. Lett. 23, 243–245] is a Hoeffding type exponential inequality without any assumptions or restrictions. © 2008 Elsevier B.V. All rights reserved.
1. Introduction Let X1 , X2 , . . . , Xn be independent random variables from a distribution F . Consider a parametric function θ = θ (F ) for which there is an unbiased estimator. That is,
θ = θ (F ) = Eh(X1 , . . . , Xm ), m ≤ n, for a function h = h(X1 , . . . , Xm ) which is assumed to be symmetric without any loss of generality. Let wi1 ,...,im be nonnegative numbers such that
X
wi1 ,...,im =
1≤i1 <···
n m
.
We define the weighted U-statistic for estimation of the parameter θ = θ (F ) as Un = where
P
n −1 m
X
wi1 ,...,im h(Xi1 , . . . , Xim ),
1≤i1 <···
1≤i1 <···
denotes summation over the
n m
combinations of (i1 , . . . , im ) from {1, . . . , n}.
Un is an unbiased estimator of θ . In the case where wi1 ,...,im = 1 for all m combinations of (i1 , . . . , im ) from {1, . . . , n} we have the usual unweighted U-statistic whose theory and applications can be found in many references, e.g. Serfling (1980). For many U-statistics of interest the kernel h is a Bernoulli random variable (e.g., indicator function). Improvements, extensions and results related to Kolmogorov inequalities can be found among others, in Young et al. (1987), Turner et al. (1995) and Mavrikiou (2007). Exponential and Kolmogorov inequalities have been constructed for U-statistics based on Bernoulli kernels in Christofides (1991, 1994). In this paper a Kolmogorov probability inequality for weighted U-statistics based on Bernoulli kernels is presented. This result generalizes the inequality found in Turner et al. (1995). n
2. Preliminaries For proving the result, a representation of a weighted U-statistic as an average of averages of independent random variables is used. The representation is a straightforward extension of the one introduced and utilized by Hoeffding (1963).
∗
Tel.: +357 22345159x132; fax: +357 22438234. E-mail address: [email protected].
0167-7152/$ – see front matter © 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.spl.2008.06.013
P.M. Mavrikiou / Statistics and Probability Letters 78 (2008) 3294–3297
3295
Lemma 2.1. Let k = [n/m] be the greatest integer less than or equal to n/m, and let Un be the weighted U-statistic introduced in Section 1. Then, Un =
1 X n!
Hi1 ,...,in
p
where 1
Hi1 ,...,in = and
P
p
k
[h(Xi1 , . . . , Xim )wi1 ,...,im + h(Xim+1 , . . . , Xi2m )wim+1 ,...,i2m + · · · · · · + h(Xikm−m+1 , . . . , Xikm )wikm−m+1 ,...,ikm ]
denotes summation over the n! permutations of (i1 , . . . , in ) from {1, . . . , n}.
Proof. We may write k
X
Hi1 ,...,in = k.m!(n − m)!
X
p
h(Xi1 , . . . , Xim )wi1 ,...,im
c
or
X
Hi1 ,...,in =
p
n! X n m
h(Xi1 , . . . , Xim )wi1 ,...,im
c
thus, Un =
1 X n!
Hi1 ,...,in .
p
In addition to the previous lemma we will use the following result which is due to Hoeffding (1963). Lemma 2.2. Let X be a random variable such that c ≤ X ≤ b and EX = µ. Then, 1 2 2 Eet (X −µ) ≤ e 8 t (b−c )
for t > 0.
3. Main result Throughout this section it will be assumed that X1 , X2 , . . . , Xn is a sequence of independent random variables and h(Xi1 , . . . , Xim ) is a Bernoulli kernel with Eh(Xi1 , . . . , Xim ) = pi1 ,...,im . In addition p¯ will denote the expected value of the weighted U-statistic based on the kernel h, i.e., p¯ = EUn =
n −1 m
X
wi1 ,...,im pi1 ,...,im .
1≤i1 <···
The following result is a Hoeffding type exponential inequality for weighted U-statistics without any assumptions on
wi1 ,...,im ’s and the pi1 ,...,im ’s.
Theorem 3.1. Let Un be a weighted U-statistic based on a Bernoulli kernel. For > 0 and k = [n/m] P (Un − p¯ ≥ ) ≤
k−1 1 X1X
n!
p
k j =0
!
−2k 2
exp
wi2mj+1 ,...,imj+m
.
Proof. Let s be an arbitrary positive number. Then, using Markov’s inequality and the representation of the weighted Ustatistic as an average of averages of independent random variables we have inequality (1): P (Un − p¯ ≥ ) = P [sUn ≥ s ( + p¯ )]
≤ e−s(+¯p) EesUn =e
−s(+¯p)
E exp s
" =e
−s(+¯p)
E exp s
1 X n!
! Hi1 ,...,in
p
k−1 1 X1X
n!
p
k j =0
# h Ximj+1 , . . . , Ximj+m wimj+1 ,...,imj+m
.
(1)
3296
P.M. Mavrikiou / Statistics and Probability Letters 78 (2008) 3294–3297
Clearly,
n −1 m
X
h(Xi1 , . . . , Xim )wi1 ,...,im =
1≤i1 <···
k−1 1 X1X
n!
p
k j=0
h(Ximj+1 , . . . , Ximj+m )wimj+1 ,...,imj+m
and therefore using expected value p¯ =
n −1 m
X
pi1 ,...,im wi1 ,...,im =
1≤i1 <···
k−1 1 X1X
n!
k j =0
p
pimj+1 ,...,imj+m wimj+1 ,...,imj+m .
Thus,
( P (Un − p¯ ≥ ) ≤ e
≤e
−s
−s
= e−s
E exp s
n!
n!
k j =0
p
(
1 X
E exp
p
k−1 1 XY
n!
p
)
k−1 1 X1X
[h(Ximj+1 , . . . , Ximj+m ) − pimj+1 ,...,imj+m ]wimj+1 ,...,imj+m
k−1 s X h(Ximj+1 , . . . , Ximj+m ) − pimj+1 ,...,imj+m wimj+1 ,...,imj+m k j =0
E exp
ns k
j =0
[h(Ximj+1 , . . . , Ximj+m ) − pimj+1 ,...,imj+m ]wimj+1 ,...,mj+m
=
k −1 ns o 1 X Y − s e k E exp [h(Ximj+1 , . . . , Ximj+m ) − pimj+1 ,...,imjm ]wimj+1 ,...,imjm n! p j=0 k
≤
2 k −1 1 X Y − s s 2 wi2 ,...,i e k .e 8k mj+1 mj+m n! p j=0
=
=
k −1 1 XY
n!
p
p
k
j=0
k −1 1 XY
n!
s
exp −
+
) (2)
o
(3)
(4)
s2
wi2mj+1 ,...,imj+m 2
8k
exp [−g (s)] .
(5)
j=0
Inequality (2) is due to the convexity of the exponential function, equality (3) follows from independence and finally, inequality (4) is due to Lemma 2.2. For notational simplicity let wimj+1 ,...,imj+m = w . Therefore, the function g (s) in Eq. (5) equals g (s) = sk − Maximizing g (s) we get smax = 4w2k and g (smax ) = 2w2 , implying that 2
P (Un − p¯ ≥ ) ≤
≤
≤
=
k −1 1 XY
n!
p
1 X n!
k−1 1X
k j =0
p
1 X 1 n!
p
kk
k−1
k
p
exp
k−1 X j =0
k−1 1 X1X
n!
wi2mj+1 ,...,imj+m
j=0
"
k j =0
.
!
−2 2
exp
1 s2 w 2 8 k2
exp
−2 2
!#k (6)
wi2mj+1 ,...,imj+m
exp
−2k 2
!
wi2mj+1 ,...,imj+m ! −2k 2 . 2
wimj+1 ,...,imj+m
Notice that inequality (6) is due to the arithmetic–geometric mean while the following one is due to the elementary inequality
Pn
i=1
xi
k
≤ nk−1
Pn
i =1
xki for k > 1. Thus, the proof of the theorem is now complete.
Remarks. (i) Although in Section 1 the weighted U-statistic is defined based on independent and identically distributed random variables, the result proved in this paper does not require that the observations are identically distributed. (ii) For the special case of the weighted U-statistic being the sample weighted average of independent Bernoulli random variables, Theorem 3.1 can be stated as:
P.M. Mavrikiou / Statistics and Probability Letters 78 (2008) 3294–3297
3297
Corollary 3.2. Let Y1P , . . . , Yn be independent Bernoulli random variables with E (Yi ) =Ppi for i = 1, . . . , n. Let Y¯ = Pn n n 1 1 1 ¯ Y w , p = i i i=1 i=1 pi wi , > 0 and wi , i = 1, . . . , n non-negative numbers with n i=1 wi = 1. Then, n n
2 P Y¯ − p¯ ≥ ≤ e−2n
> 0.
(7)
(iii) Inequality (7) extends the result of Turner et al. (1995) to the case of weighted Bernoulli random variables. Acknowledgment The author would like to thank the referee for helpful comments. References Christofides, T.C., 1991. Probability inequalities with exponential bounds for U-statistics. Statist. Probab. Lett. 12, 257–261. Christofides, T.C., 1994. A Kolmogorov inequality for U-statistics based on Bernoulli kernels. Statist. Probab. Lett. 21, 357–362. Hoeffding, W., 1963. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58, 13–30. Mavrikiou, P.M., 2007. Kolmogorov inequalities for the partial sum of independent Bernoulli random variables. Statist. Probab. Lett. 77, 1117–1122. doi:10.1016/j.spl.2007.02.001. Serfling, R.J., 1980. Approximation Theorems of Mathematical Statistics. Wiley, New York. Turner, D.W., Young, D.M., Seaman, J.W., 1995. A Kolmogorov inequality for the sum of independent Bernoulli random variables with unequal means. Statist. Probab. Lett. 23, 243–245. Young, D.M., Seaman, W.J., Marco, R.V., 1987. A note on a Kolmogorov inequality. Statist. Probab. Lett. 5, 217–218.
Contents lists available at ScienceDirect
Statistics and Probability Letters journal homepage: www.elsevier.com/locate/stapro
A Kolmogorov inequality for weighted U-statistics Petroula M. Mavrikiou ∗ Department of Business Administration, School of Economic Sciences and Administration, Frederick University Cyprus, P.O. Box 24729, 1303 Nicosia, Cyprus
article
info
a b s t r a c t
Article history: Received 18 June 2007 Received in revised form 28 April 2008 Accepted 2 June 2008 Available online 17 June 2008
In this paper a Kolmogorov probability inequality for weighted U-statistics based on Bernoulli kernels is presented. This inequality which extends the results of [Turner, D.W., Young, D.M., Seaman, J.W., 1995. A Kolmogorov inequality for the sum of independent Bernoulli random variables with unequal means. Statist. Probab. Lett. 23, 243–245] is a Hoeffding type exponential inequality without any assumptions or restrictions. © 2008 Elsevier B.V. All rights reserved.
1. Introduction Let X1 , X2 , . . . , Xn be independent random variables from a distribution F . Consider a parametric function θ = θ (F ) for which there is an unbiased estimator. That is,
θ = θ (F ) = Eh(X1 , . . . , Xm ), m ≤ n, for a function h = h(X1 , . . . , Xm ) which is assumed to be symmetric without any loss of generality. Let wi1 ,...,im be nonnegative numbers such that
X
wi1 ,...,im =
1≤i1 <···
n m
.
We define the weighted U-statistic for estimation of the parameter θ = θ (F ) as Un = where
P
n −1 m
X
wi1 ,...,im h(Xi1 , . . . , Xim ),
1≤i1 <···
1≤i1 <···
denotes summation over the
n m
combinations of (i1 , . . . , im ) from {1, . . . , n}.
Un is an unbiased estimator of θ . In the case where wi1 ,...,im = 1 for all m combinations of (i1 , . . . , im ) from {1, . . . , n} we have the usual unweighted U-statistic whose theory and applications can be found in many references, e.g. Serfling (1980). For many U-statistics of interest the kernel h is a Bernoulli random variable (e.g., indicator function). Improvements, extensions and results related to Kolmogorov inequalities can be found among others, in Young et al. (1987), Turner et al. (1995) and Mavrikiou (2007). Exponential and Kolmogorov inequalities have been constructed for U-statistics based on Bernoulli kernels in Christofides (1991, 1994). In this paper a Kolmogorov probability inequality for weighted U-statistics based on Bernoulli kernels is presented. This result generalizes the inequality found in Turner et al. (1995). n
2. Preliminaries For proving the result, a representation of a weighted U-statistic as an average of averages of independent random variables is used. The representation is a straightforward extension of the one introduced and utilized by Hoeffding (1963).
∗
Tel.: +357 22345159x132; fax: +357 22438234. E-mail address: [email protected].
0167-7152/$ – see front matter © 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.spl.2008.06.013
P.M. Mavrikiou / Statistics and Probability Letters 78 (2008) 3294–3297
3295
Lemma 2.1. Let k = [n/m] be the greatest integer less than or equal to n/m, and let Un be the weighted U-statistic introduced in Section 1. Then, Un =
1 X n!
Hi1 ,...,in
p
where 1
Hi1 ,...,in = and
P
p
k
[h(Xi1 , . . . , Xim )wi1 ,...,im + h(Xim+1 , . . . , Xi2m )wim+1 ,...,i2m + · · · · · · + h(Xikm−m+1 , . . . , Xikm )wikm−m+1 ,...,ikm ]
denotes summation over the n! permutations of (i1 , . . . , in ) from {1, . . . , n}.
Proof. We may write k
X
Hi1 ,...,in = k.m!(n − m)!
X
p
h(Xi1 , . . . , Xim )wi1 ,...,im
c
or
X
Hi1 ,...,in =
p
n! X n m
h(Xi1 , . . . , Xim )wi1 ,...,im
c
thus, Un =
1 X n!
Hi1 ,...,in .
p
In addition to the previous lemma we will use the following result which is due to Hoeffding (1963). Lemma 2.2. Let X be a random variable such that c ≤ X ≤ b and EX = µ. Then, 1 2 2 Eet (X −µ) ≤ e 8 t (b−c )
for t > 0.
3. Main result Throughout this section it will be assumed that X1 , X2 , . . . , Xn is a sequence of independent random variables and h(Xi1 , . . . , Xim ) is a Bernoulli kernel with Eh(Xi1 , . . . , Xim ) = pi1 ,...,im . In addition p¯ will denote the expected value of the weighted U-statistic based on the kernel h, i.e., p¯ = EUn =
n −1 m
X
wi1 ,...,im pi1 ,...,im .
1≤i1 <···
The following result is a Hoeffding type exponential inequality for weighted U-statistics without any assumptions on
wi1 ,...,im ’s and the pi1 ,...,im ’s.
Theorem 3.1. Let Un be a weighted U-statistic based on a Bernoulli kernel. For > 0 and k = [n/m] P (Un − p¯ ≥ ) ≤
k−1 1 X1X
n!
p
k j =0
!
−2k 2
exp
wi2mj+1 ,...,imj+m
.
Proof. Let s be an arbitrary positive number. Then, using Markov’s inequality and the representation of the weighted Ustatistic as an average of averages of independent random variables we have inequality (1): P (Un − p¯ ≥ ) = P [sUn ≥ s ( + p¯ )]
≤ e−s(+¯p) EesUn =e
−s(+¯p)
E exp s
" =e
−s(+¯p)
E exp s
1 X n!
! Hi1 ,...,in
p
k−1 1 X1X
n!
p
k j =0
# h Ximj+1 , . . . , Ximj+m wimj+1 ,...,imj+m
.
(1)
3296
P.M. Mavrikiou / Statistics and Probability Letters 78 (2008) 3294–3297
Clearly,
n −1 m
X
h(Xi1 , . . . , Xim )wi1 ,...,im =
1≤i1 <···
k−1 1 X1X
n!
p
k j=0
h(Ximj+1 , . . . , Ximj+m )wimj+1 ,...,imj+m
and therefore using expected value p¯ =
n −1 m
X
pi1 ,...,im wi1 ,...,im =
1≤i1 <···
k−1 1 X1X
n!
k j =0
p
pimj+1 ,...,imj+m wimj+1 ,...,imj+m .
Thus,
( P (Un − p¯ ≥ ) ≤ e
≤e
−s
−s
= e−s
E exp s
n!
n!
k j =0
p
(
1 X
E exp
p
k−1 1 XY
n!
p
)
k−1 1 X1X
[h(Ximj+1 , . . . , Ximj+m ) − pimj+1 ,...,imj+m ]wimj+1 ,...,imj+m
k−1 s X h(Ximj+1 , . . . , Ximj+m ) − pimj+1 ,...,imj+m wimj+1 ,...,imj+m k j =0
E exp
ns k
j =0
[h(Ximj+1 , . . . , Ximj+m ) − pimj+1 ,...,imj+m ]wimj+1 ,...,mj+m
=
k −1 ns o 1 X Y − s e k E exp [h(Ximj+1 , . . . , Ximj+m ) − pimj+1 ,...,imjm ]wimj+1 ,...,imjm n! p j=0 k
≤
2 k −1 1 X Y − s s 2 wi2 ,...,i e k .e 8k mj+1 mj+m n! p j=0
=
=
k −1 1 XY
n!
p
p
k
j=0
k −1 1 XY
n!
s
exp −
+
) (2)
o
(3)
(4)
s2
wi2mj+1 ,...,imj+m 2
8k
exp [−g (s)] .
(5)
j=0
Inequality (2) is due to the convexity of the exponential function, equality (3) follows from independence and finally, inequality (4) is due to Lemma 2.2. For notational simplicity let wimj+1 ,...,imj+m = w . Therefore, the function g (s) in Eq. (5) equals g (s) = sk − Maximizing g (s) we get smax = 4w2k and g (smax ) = 2w2 , implying that 2
P (Un − p¯ ≥ ) ≤
≤
≤
=
k −1 1 XY
n!
p
1 X n!
k−1 1X
k j =0
p
1 X 1 n!
p
kk
k−1
k
p
exp
k−1 X j =0
k−1 1 X1X
n!
wi2mj+1 ,...,imj+m
j=0
"
k j =0
.
!
−2 2
exp
1 s2 w 2 8 k2
exp
−2 2
!#k (6)
wi2mj+1 ,...,imj+m
exp
−2k 2
!
wi2mj+1 ,...,imj+m ! −2k 2 . 2
wimj+1 ,...,imj+m
Notice that inequality (6) is due to the arithmetic–geometric mean while the following one is due to the elementary inequality
Pn
i=1
xi
k
≤ nk−1
Pn
i =1
xki for k > 1. Thus, the proof of the theorem is now complete.
Remarks. (i) Although in Section 1 the weighted U-statistic is defined based on independent and identically distributed random variables, the result proved in this paper does not require that the observations are identically distributed. (ii) For the special case of the weighted U-statistic being the sample weighted average of independent Bernoulli random variables, Theorem 3.1 can be stated as:
P.M. Mavrikiou / Statistics and Probability Letters 78 (2008) 3294–3297
3297
Corollary 3.2. Let Y1P , . . . , Yn be independent Bernoulli random variables with E (Yi ) =Ppi for i = 1, . . . , n. Let Y¯ = Pn n n 1 1 1 ¯ Y w , p = i i i=1 i=1 pi wi , > 0 and wi , i = 1, . . . , n non-negative numbers with n i=1 wi = 1. Then, n n
2 P Y¯ − p¯ ≥ ≤ e−2n
> 0.
(7)
(iii) Inequality (7) extends the result of Turner et al. (1995) to the case of weighted Bernoulli random variables. Acknowledgment The author would like to thank the referee for helpful comments. References Christofides, T.C., 1991. Probability inequalities with exponential bounds for U-statistics. Statist. Probab. Lett. 12, 257–261. Christofides, T.C., 1994. A Kolmogorov inequality for U-statistics based on Bernoulli kernels. Statist. Probab. Lett. 21, 357–362. Hoeffding, W., 1963. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58, 13–30. Mavrikiou, P.M., 2007. Kolmogorov inequalities for the partial sum of independent Bernoulli random variables. Statist. Probab. Lett. 77, 1117–1122. doi:10.1016/j.spl.2007.02.001. Serfling, R.J., 1980. Approximation Theorems of Mathematical Statistics. Wiley, New York. Turner, D.W., Young, D.M., Seaman, J.W., 1995. A Kolmogorov inequality for the sum of independent Bernoulli random variables with unequal means. Statist. Probab. Lett. 23, 243–245. Young, D.M., Seaman, W.J., Marco, R.V., 1987. A note on a Kolmogorov inequality. Statist. Probab. Lett. 5, 217–218.