Laws of large numbers for bootstrappedU-statistics

Journal

of Statistical

Planning

and Inference

9 (1984) 185-194

185

North-Holland

LAWS OF LARGE U-STATISTICS Krishna

NUMBERS

B. ATHREYA’

Iowa State University,

Malay

Ames,

IA 50011,

USA

GHOSH’

University

Leone

of Florida,

Gainesville,

FL 32611,

USA

Dayton,

OH 45435,

USA

Y. LOW3

Wright State University,

Pranab

K. SEN4

University Received

of North 18 May

Recommended

Abstract:

Carolina,

Chapel Hill, NC 27514,

1982; revised

by N.L.

manuscript

of finiteness

mean,

under the finiteness

U-statistics

under

Key words: tingales;

Classification: Asymptotic

U-statistics;

25 February

1983

law of large

numbers

is obtained

under

the

for some r> 1, and a weak law of large numbers

of the first moment.

conditions.

Stochastic

variance of a bootstrapped U-statistic pivot and the bias of the bootstrapped Subject

a strong

of the rth moment,

is obtained

parallel

received

USA

Johnson

For the bootstrapped

assumption

AMS

FOR BOOTSTRAPPED

The results convergence

are then extended of the jackknifed

is proved. The asymptotic U-statistic are indicated.

normality

to bootstrapped estimator

of the

of the bootstrapped

6OF15, 62699.

theory;

Bootstrap;

Laws of large numbers;

Means;

Reversed

(sub-)mar-

von Mises’ functionals.

1. Introduction Hoeffding’s (1948) U-statistics play a very useful role in problems of estimation and hypothesis testing. The class of one-sample U-statistics includes the sample mean, variance, Gini’s mean difference, signed-rank statistics, Kendall’s tau statist Research ’ Research 3 Research

supported by the National Science Foundation, Contract supported by the Army Research Office, Durham, Grant done while the author was visiting Iowa State University.

4 Research

supported

by the National

Heart,

Lung

and Blood

MCS-8201456. DAAG29-82-K-0136.

Institute,

2243-L.

0378-3758/84/$3.00

0

1984, Elsevier

Science

Publishers

B.V. (North-Holland)

Contract

NIH-NHLBI-71-

K.B. Athreya et al. / Bootstrapped

186

U-statistics

tics and many others. Efron’s (1979) bootstrapping is a resampling scheme in which a statistician attempts to learn the sampling properties of a statistic by recomputing its value on the basis of a new sample realized from the original one (this will be made more precise in the later sections). Bickel and Freedman (1981) have dealt very extensively with the limit laws of many bootstrapped functionals including t-statistics, CT-statistics, von Mises’ functionals and sample quantiles. Singh (1981) has also obtained asymptotic results justifying the validity of bootstrap approximations in certain cases. While studying the laws of large numbers for one-sample U-statistics, in Section 2, we prove first a strong law for the bootstrapped mean under the assumption of finiteness of a moment of order bigger than 1 and a weak law under the finiteness of the first moment. Next, in Section 3, a strong law of large numbers is proved for bootstrapped U-statistics under the finiteness of a moment of order bigger than 1. Also, the stochastic convergence of the jackknifed estimator of the variance of a bootstrapped I/-statistic is proved in this section. This result along with a result of Bickel and Freedman (1981) yield the asymptotic normality of the bootstrapped pivot (defined in Section 3) based on U-statistics.

2. Laws of large numbers for bootstrapped

means

Suppose that X,, . . . , X,, are independent and identically distributed (i.i.d.) random variables (T.v.) with an (unknown) distribution function (d.f.) F, and let xl, . . . ,x, be the corresponding observed realizations. Let X, = (X,, . . . , X,) and x,,). Then, the bootstrapping procedure may be described as follows. x,=(x,,..., (i) Construct the sample (empirical) d.f. F, (from the x,). (ii) With F,, fixed, draw a simple random sample (with replacement) of size m from the set of indices (1, . . . , n). The corresponding sample, called the bootstrapped sample, is denoted by X,*, = (X,*,, . . . , X,*,>, where, conditional on X,=x,,, the X$ are i.i.d. each assuming one of the n possible values x1, . . . ,x, with equal probability n-‘. (iii) Use the bootstrapped sample to estimate the variability or other characteristics of the estimator from the original sample. Our first theorem relates to the weak law of large numbers for the bootstrapped mean under the minimal assumption of finite first moment. 1. For the bootstrapped sample X,*, from X,,, let X$, =m-’ Cy’, Xz andX,,=n-’ En=, Xi. A ssume that EX, = p is finite. Then, X,$, - X, converges in probability to 0 as min(n, m) --t 03. Theorem

Proof.

The method of proof employs the characteristic q&,(t)

= E[exr(it{xz,

function technique.

- -%>)I = E(E[exp(it{%,-%I)

= E{ b#$Wm)l”),

Let

1&I) (2.1)

K.B. Athreya et al. / Bootstrapped U-statistics

187

where @,*(0)=E[exp(iB{X,*, --Xn}) 1X,], f or every real 0. For proving the theorem, it suffices to show that e,,,(t) + 1 as min(n,m) + 03. Since I#,*(t/m)l I 1, using the dominated convergence theorem, it suffices to show that {@,*(t/m)}” 2 1, as min(n, m) -+ 03, for every real t. Writing F,(U) = n-r En=, I(& -X,, I u), where Z(A) stands for the indicator function of the set A, one has the representation @,*(t/m) = 1 + R,(t/m), where

(2.2)

cc R,(t/m)

=

[exp(itu/m)

- 1 -i&/m]

U,(u).

(2.3)

.I’-cc

In view of (2.2), in order to show that {@,*(t/m)}“‘~ 1 as min(n, m) -+ 00, it suffices to show that m IR,(t/m)J 5 0 as min(n, m) --f 00. Now, using standard inequalities for characteristic functions [see e.g. Feller (1966, p. 487)], m IR,(t/m)I

I +t*m-’

u*dF,(u)+2 s ju1 sm”3

5 +t2m-‘/3 +2 ItI n-l

It/

f IXi-znl

lu I fin(u) s Iu/ >m”3

Z((Xj-Xnl>m1’3).

(2.4)

i=l

Thus, it remains only to show that as min(n, m) -+ 00,

=E[JX,-~~IZ(IX1-~~J>m”3)J,0.

(2.5)

This follows directly from the uniform integrability that m1’3 + 00 as min(n, m) + 00. 0

of X1 -2,

along with the fact

The next theorem relates to a strong law of large numbers. Theorem

2. Assume

that EJX, 1’ “<

00 for some BE (0,l).

Then, for every E>O

and d>O, sup sup IX&-XnI>s n>N

-0

asN-roo.

msdn

(2.6)

Remark 1. Note that (2.6) implies in particular

that x$-x,, +O a.s. as n -+ 00. A similar a.s. convergence result is given in Athreya (1983) where conditions relating to the growth rate of m with n and the moments of X, are given to ensure the a.s. convergence of .Y:m -X,, to 0. The method of proof used by Athreya (1983) is entirely different from the present method. Proof of Theorem 2. First, we prove the inequality that for every m 5 dn (d > 0) and

r 11, there exists a positive, finite number C,(d), such that E[(X,*, - Xn)” 1A’,] 5 C,(d)n-‘sir

for every n L 2,

(2.7)

K.B.

188

Athreya

et al. / Bootstrapped

U-statistics

where s,’ = n-r Cl=, (Xi - X,J2. Towards this proof, we note that (XZm_X”)2’ = m-2r i;,***i& m ,fi, (XZ, -%I). *r

(2.8)

Note that if one of the suffixes, say, c, appears exactly once in a summand of (2.8), then, by virtue of the conditional independence of the X2, E(X$ -Xn 1X,) = 0 a-e., so that the conditional expectation of the corresponding summand is also 0 a.e. A general term appearing in the right hand side of (2.8) with all distinct suffixes having possibly non-zero expectation is of the form (X2, - Xn)kl . *. (X2” - XJku where kr + .-. +k,=2r, The conditional

expectation

5

k,z2,

I= l,..., ~(21).

of this typical term, given X,, is equal to

i (Xi -X”)’

‘-“.+

= nrWu$

for every U 1 1.

I

i=l

(2.9)

Also, the number of such terms in (2.8) is O(nP). Hence, from (2.8) through (2.9), we obtain that E&Y& - XJzr 1X,] I K(r)m_2’

( j,

nr-‘mu)

s,” (2.10)

5 C,(d)n-‘sir,

as mend, d >O. Note that K(r) and C,(d) do not depend on n. This proves (2.7). Let us define then W,,, =

n-’ f: IXi-~l'++d, i=l

where 6 is defined as in the theorem.

$5 n-l f (Xi-/4)2S i=l

Note that

max IXi-j.l’-d

I Isicn

5 { i ,xi-a,“d](‘-6)/(i’~)w~,, i=l

Therefore,

W,,, I

= n(1-6)/(1+6)Wnf/bl+6).

(2.,,)

by (2.7) and (2.11), we obtain that E[(X,*, - XJ2’ 1X,] I C,(d)n-2”‘o+6)

W,frb/(‘+‘),

(2.12)

for every m>dn, d>O, nz2. Next, write A N for the event within the braces of (2.6), and let BN = {sup,,,,, W,,, s g], where g (BE IX, -,u I’+‘) is a finite constant. Note that the W,,, form a backward martingale, so that by the Kolmogorov maxi-

K.B. Athreya et al. / Bootstrapped U-statistics

ma1 inequality,

189

we have

P(B,)=P

/X,-pI1+‘>g-E

sup W,,,-E

LIIZN

I

{g-E

-p,ll+a}-lE

IX,

+O

I&-PI’+’

1W,,-E

IX, -p,l’+‘l (2.13)

asN-+=,

where the last step follows from the L,-convergence using the Markov inequality and (2.12), we have P&B/V)5

c P

>&, wn,s rg

I mkdn

fl;rN

s,gN

sup /X;*-XJ

of the mean (Wn,s). Also,

I

E[I(W,,6~g)C,(d)n-26r’(‘+d)W,f’6(1+6$’~2r

~C,*(g,c,r,d)

c n-26r’(1+d)~C*(g,~,r,d)N1-26r’(‘+6),

(2.14)

n>N

where C,* and C* are finite positive constants. Choose r (> 1) such that 2&/( 1 + 6) = 1 + y, for some y>O. Then, noting that P(AN)5P(ANBN) +P@,), (2.6) follows from (2.13), (2.14) and the proper choice of r. q

3. Laws of large numbers for bootstrapped

U-statistics

Let X,, . . . , X, be i.i.d. r.v. with the d.f. F, and for a kernel $Jof degree r, the U-statistic based on X,, . . . , X,, (with n 2 r) is defined as u,= 0 The corresponding denoted by u,*, = For notational

nr -’ F @(X+...,X,); 9’ bootstrapped

C,,,=

U-statistic

{15i,<...
based

on X,*l, . . . , X,*, (m 1 r) is

-1 m 0 r

F @(X& WX,:;). m.r

(3.2)

simplicity, we consider only the case of r = 2. Also, we denote by

IIOIlp= max{E I@(XI,X~)Ip, E I@(X19X2)lp>, The following theorem is a direct generalization Theorem

P>O.

(3.3)

of Theorem 2 to U-statistics.

3. If Ij@lj,+~<00 for some 6 E (0, l), then, for every E> 0, d > 0,

sup sup IU$,nzN

Proof.

(3.1)

Define

madn

U,I>E

+O

as N-t-.

(3.4)

3

V, =nm2 CyS, I,“,, @(Xi, Xi>. Then,

under

the assumption

that

190

K. B. Athreya et al. / Boo&trapped U-statistics

/1@11 t c 00 (implied by (3.3) for p> 1), it follows that P{sup,,, 1f_J,- V, I> E} -+ 0 [see Ghosh and Sen (1970)]. Hence, in (3.4), it suffices to replace U,, by V,,. Next, we make use of the Hoeffding (1961) decomposition for U,*, under the conditional setup, given X,,. Then, we have i-J,*, -

v n = 2u”’“In + u(2) nmr

(3.5)

where

U$=m-'

i

,

n-'i @(X,:.,Xj)-v,

i=l

(3.6)

j=l

(3.7) Note that if we let

Z~~=#(Xn*iX$)-n-’ 2 @(X~*Xj)-n-'

i

@(Xi,X$)+Vn,

i=l

j=l

for i,j= I,..., m, then, given X,,, Zn:j and Zz& are conditionally orthogonal whenever (i,j)#(r,s) (i.e. at least one of the two indices is different) (given X,,), the same is true for more than two Zzj’s. As such, we may repeat the steps in (2.8) through (2.12) and claim that for every r-2 1, mzdn (d>O) and nz2, E[( U$)”

1X,,] 5 Cr(d)n-2r’(’

+‘I( W;J’(’

+ @,

(3.8)

where WnT6= n-* i i /@(Xi,Xj)l’+’ i=[ j=l =n

-I

I

n-1

$,

= n-‘w;‘j+.-‘(,_])w(*f

Since W,$,

n,61

P

we proceed as in (2.13)-(2.15) sup sup IU(*)l>c nm

medn

nzN

-+O

;)-’

F

I@(Xi,Xj)[‘+d n.2

say.

j = 1,2, are both reverse martingales

by ll@ll,+~<~)~ and d>O,

Therefore,

+y(

l@(xi,X,)ll+‘]

(3.9)

with finite expectations (ensured and conclude that for every E> 0

as N-00.

(3.10)

3

to prove (3.4), it suffices to show that for every e>O and d >O, SUP

supIU,$I>e

madn

nzN

-0

asN--+aJ.

(3.11)

V,, i= Now, given X,, Un(fn)=m-’ C!” , r=l ZniO where Znio=n-’ C!‘= I 1 4(X? nrt X,)1, -**, m, are conditionally independent r.v. and ZniO can only assume n values w,, with equal (conditional) probability n-l. (=n-’ C,“,, @(Xa,Xj)V,), 1 casn,

K. B. Athreya et al. / Bootstrapped

U-statistics

191

Note that C”, , w,; = 0 and

= 2,wvcl+a){~n

3)241+6),

(3.12)

say,

where U,,, is a von Mises functional with finite expectation and hence converges a.s. to a finite constant as n -+ 00. As such, we proceed as in (2.8) through (2.15) and conclude that (3.11) holds. This proves (3.4). 0 The final result of this section concerns the convergence in probability of the jackknifed estimator of the variance of a bootstrapped U-statistic under the minimal assumption of the finiteness of the second moment of the kernel. Towards this, we introduce the following. Let

and, for every i (= 1, . . . , m), let

b(‘) nm = (m

f

- 1)-t

@(X,*i,X$),

(3.14)

j=l(j+i)

s2nm = (m - 2)-24(m - 1)2 (m - l)-’ f (@A - U,*,)’ i=l [ =4(m-1)2(m-2)-2

(m-1)-$,

{b~~}2-_(111--1)-1,{U,*,}2

4.

If 1/@/j2<00, then, as min(m, n) + 03, s,‘, --t 46,

Proof.

1.

(3.15)

Then, we have the following. Theorem

1

in probability.

(3.16)

First, note that E[U,*, 1X,] = V,, and H(LSr, - VII21= Eb%vL 5 3m-‘E 5

3m-’

- v,12 1&II} ne2

ll@l12~

(3.17)

192

K. B. Athreya et al. / Bootstrapped U-statistics

where the penultimate step follows by using (3.5)-(3.7). Chebyshev inequality, under 11 @I/*< 03, for every E < 0, P{IU,*,-V,J>s}+O

Hence, by (3.17) and the

as m-m.

(3.18)

Again, under the assumption that /I@J II1< w (implied by 11#[12
in probability

as min(m, n) + 03.

(3.19)

Hence, it suffices to show that

m-l f, {bg}2-

4-I+

e2 in probability as min(m, n) + 00.

(3.20)

To this end, we write a,; = ZniO+ V,, where the ZniOare defined after (3.11). Then, noting that ani =E[@(X$,X,$) 1X,,Xz], we have

E[(b$h-Qa,i)2] = E{E[(btJj-C7a,i)2 1Xn]} 5(m-1)-‘~~@~~2+0

as m+o3.

(3.21)

Note that by (3.21), E

m-’

i

i=l

(b$,-aa,i)2

as m-t-,

=E[bA$-a,l]2-*0 I

so that m-’ ir, (b:i - Q,;)~ + 0

in probability

as m + co.

(3.22)

Let us now define

a,*i= @(X~,y)dF(y)=&(X~),

i=l,...,

m.

(3.23)

s

In view of (3.22) and Lemma 1 of Sen (1960), to prove (3.20), it suffices to show that in probability

as min(m, n) -+ 00,

(3.24)

and m-’ f,

a,*i?+ [, + ~9~ in probability

as min(m, n) + 03.

Note that (3.25) follows directly from Theorem 1 by noting that E&(X,)s To prove (3.24), it suffices to show that as min(m, n) + 00,

me’i=l i (a,i-a,*,)2 =E[u,~-u,*~]~-+~. I

(3.25) II@112.

(3.26)

K.B. Athreya et al. / Bootstrapped U-statistics

Now,

193

by definition, E(a,, -

a,*J2= E (n - l)(n - 2)

#l)

n2 n +2(n-l)Uc2)+n-1 7 n2 ”

1,

Ui3’ + ne2UJ4)

where

(3.27)

U(l) n = [n(n - l)(n - 2)]-’ x {wG,&) UA2’= [n(n - l)]-’

C

- @*(X;)},

(3.28)

{@Cxi,xi>- @lwi>>

I cifjcn

Ui3) = [n(n - 1)1-l

C

x ~@-cPl(xi)~9

(3.29)

{0(xi~xj)-4~(xi))2~

(3.30)

lri#jrn

and

VA’)= n-l J, {O(xi~xi)-OI(xi))2~

(3.31)

Symmetric versions of these kernels may be obtained by averaging over permutations. Since U$), r= 1, . . . , 4, are all U-statistics with EU,(‘)= 0 and E 1U$‘j < 00, for r= l,..., 4, using the backward martingale property of U-statistics, it follows that E IUn(‘)/‘0, as n-+ 03 [see Chow and Teicher (1978, p. 377)], so that by (3.27), 0 E(a,, - azI)2 + 0 as n + 00. This completes the proof of the theorem. < 03 for some 6> 0, then, expressing s& as a linear combinaRemark 2. If 1/@1(2+6 tion of bootstrapped U-statistics and using Theorem 2, one can show that for every e>O and d>O, sup sup js&--4~,l>c naN

madn

+O

asN-,=.

(3.32)

1

Remark 3. Although the unbootstrapped U-statistics U-statistics are typically biased estimators of 8 = EU,. vation that E[U,*, 1X, = V,] is a von Mises functional, unbiased estimator of 8. The bias is however of the

are unbiased, bootstrapped This follows from the obserand, in general, V, is not an order O(n-‘).

Remark 4. Although we have not spelled explicitly, by virtue of the backward martingale property of U,*, and the components of V,, the weak law of large numbers for U$, holds under the condition that (I@11,< 03. Remark 5. The bootstrappedpivot for U-statistics can be taken as m”2(U:m - U,)/ nm. From Theorem 3.1 of Bickel and Freedman (1981) [which can easily be S generalized to the case where m # n], it follows that 9 m”2(U,*m - U,)/(4[,) m, N(0, 1) as min(m,n)-

K.B. Athreya

194

when 114112< 03. Combining /1#1/2<~,

this

result

with

U-statistics

out

Theorem

4, we have

under

as min(m,n)+m,

z

rrP2(U,*m - UJ/s,, A similar

et al. / Bootstrapped

conclusion

also holds

N(O,1).

(3.33)

for bootstrapped

von Mises functionals.

References Athreya, Bickel,

K.B. (1983). P.J.

and

D.A.

A strong

law for the bootstrap.

Freedman

(1981).

Statist.

Some asymptotic

Probability

theory

Letters.

1, 147-150.

for the bootstrap.

Ann.

Statist.

9,

1196-1217. Chow,

Y.S. and H. Teicher

Springer,

(1978). Probability

Theory,

methods:

look at the jackknife.

Efron,

B. (1979).

Feller,

W. (1966). An Introduction

Bootstrap

Another fo Probability

York. Ghosh, M. and P.K. Sen (1970). On the almost functions. Hoeffding,

Calcutta

Statist.

Assoc.

Bull.

W. (1948). A class of statistics

19, 293-325. Hoeffding, W. (1961). Carolina,

Independence,

Interchangeability,

Martingale.

New York.

Mimeo

The strong

Ann.

Theory and Applications,

sure convergence

Sfatist.

7, l-26.

Vol. 2. John

of von Mises’ differentiable

Wiley,

New

statistical

19, 41-44. with asymptotically

law of large

numbers

normal for

distribution.

u-statistics.

Inst.

Ann. Math. Statist.

Univ.

Statist. North

Ser. No. 302.

Sen, P.K. (1960). On some convergence properties of u-statistics. Calcutta Statisf. Assoc. Bull. 10, 1-18. Singh, K. (1981). On the asymptotic accuracy of Efron’s bootstrap. Ann. Sfatist. 9, 1187-1195.