- Email: [email protected]
Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problem
STATISTII~ & ELSEVIER
Statistics & Probability Letters 36 (1997) 9-21
Studentized permutation tests for non-i.i.d, hypotheses and the generalized Behrens-Fisher problem Arnold Janssen* Mathematical lnstitut, University of Dfisseldorf Universitiitsstrasse1, D-40225 Diisseldorf Germany
Received 1 December 1995; received in revised form 1 December 1996
Abstract It is shown that permutation tests based on studentized statistics are asymptotically exact of size ~ also under certain extended non-i.i.d, null hypotheses. To demonstrate the principle the results are applied to the generalized two-sample Behrens-Fisher problem for testing equality of the means under general non-parametric heterogeneous error distributions. Within this setting we propose a permutation version of the Welch test which is an extension of Pitman's two-sample permutation test. These results are special cases of a conditional central limit theorem for studentized permutation statistics which also applies to asymptotic power functions. © 1997 Elsevier Science B.V. AMS classification: 62G 10; 62G09 Keywords: Behrens-Fisher problem; Permutation test; Permutation statistic; Conditional central limit theorem; Survival
test
1. Introduction and examples Throughout, we are concerned with a two-sample non-parametric testing problem given by an extended null hypothesis Ho which is strictly larger than the restricted null hypothesis of i.i.d, random variables. The restriction of Ho to the i.i.d, case is denoted by iZlo, i.e. 1-7Io~ Ho. A famous example is the extended Behrens-Fisher problem where equality of two means is tested for a model with different unknown variances of the error variables. For this type of examples the choice of critical values (under Ho) is a serious problem. As message of this paper we suggest to carry out the underlying tests as permutation tests for studentized statistics which includes a variance correction of the permutation distribution. It is shown that in this case the conditional critical values of the permutation distribution work well at least in the asymptotic setting. We follow the approach of Neuhaus (1993), who successfully applied studentized permutation tests for survival problems, see Example 1.2 below.
* Tel.: + 49 21181 12165; fax: + 49 211 81 13117; e-mail:[email protected]. 0167-7152/97/$17.00 © 1997 Elsevier Science B.V. All rights reserved PII S0 1 67-7 1 52(97)00043-6
10
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
To explain the idea suppose that T~ is a reasonable test statistic, for instance, the linear test statistic (1.2) below. By definition permutation tests, given by T,, are distribution free and of exact level ~ for the i.i.d, null hypothesis only. Even in the asymptotic setup they are in general no longer of size 0~for Ho. To overcome this difficulty, permutation tests are proposed for studentized statistics 2V, = T,/V)/2, where V, is a suitable variance estimator. It is essential that the permutation procedure takes the denominator of T, into account which gives the permutation distribution its right asymptotic variance. This is the common feature of the following examples. Example 1.1 (Extended Behrens-Fisher problem). Let (Z/)/~ denote a sequence of real i.i.d, error variables with E(Zi) = 0 and Var(Zi) = 1. Let al, a2 denote two arbitrary positive unknown standard deviations. Consider then the two-sample problem of total sample size n = nl + n2 X i = [,21 --~ t T i Z i ,
1 <~i <~n l ,
l l l U ff~,
and
Yi:~2-]-ff2Znl+i, l<~i<<,n2, #2eN. The testing problem may be given by the one-sided null hypothesis {#1 ~< #2} or the two-sided analogue {Pl = P2}, where trl and a2 are nuisance parameters. It is crucial that tests should approximately attain the level e at the non-i.i.d, null hypothesis Ho = {#, = #2}
(1.1)
(or its boundary Ho for the one-sided case). The restricted null hypothesis is given by 17Io-- {#1 = P2, al = a2}. Common tests are based on the means X = 1- ~ X,,
F=--I ~ y,
nl i=1
n2 i=1
and
~ _ ~ ) 1/2 =
(£-
?)
or
= Tn/V /2,
(1.2)
where
Vn = nln2 ($2/nl + $2/n2)
(1.3)
n
defined by nl
- -
nl
--
1
E ( x , - £)2, 1
i=l
= -
n2
E (Y-
n2-- l i=l
It is a serious problem how to carry out the test by an appropriate choice of critical values. Example 1.2 (Random censoring model). Consider two groups of survival times $1 . . . . . S., and S,, + 1. . . . . S. which are i.i.d, within each group and censored by censoring variables C1, ..., C,, and C~, + 1, ..., C. again i.i.d, for each group and independent of the S's. Within the random censorship model only X~ = min(S~, Ci),
At = l(Si ~< C~), 1 ~< i ~< n,
(1.4)
are observable. Under the null hypothesis of equal survival distributions Ho = { ~ e ( s , ) = ~(s~)}
(1.5)
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
11
it is, in practice, often not realistic to assume equal censoring distributions £°(C1) and Y'(C.), which can be viewed as nuisance parameters. Thus, the observed random variables (1.4) are no longer i.i.d, under Ho since censoring destroys the i.i.d, structure. The restricted null hypothesis is given by
rio = {¢-~9($1) = ~(Sn), ~9(C1) = ¢~(Cn)}. Let T, be a survival test statistic, for instance the log-rank statistic, see Anderson et al. (1993), and let V. be a sequence of associated variance estimators. A remarkable result of Neuhaus (1993) shows that permutation tests based on 7", = T,/V~./2 are asymptotically of exact level 5 under the extended null hypothesis (1.5). His Monte Carlo results support these tests at finite sample size. Since we do not like to overload the paper, results concerning conditional survival statistics based on Section 3 will be published in a forthcoming paper. We briefly recall the notion of permutation tests (1.7) below which are conditional tests given by the data. Let ~, = T.((X,i)i~.) be an arbitrary test statistic based on random variables X,i. For fixed values x.i of X,~ and uniformly P-distributed random permutations (a(i))~ ,<. of 1, ..., n, which are independent of the data, let 7, E [0, 1] and C,,p(5) be the solutions of P(Tn((Xn6(i))i ~n) > Cn,p(5)) q- 7nP(Tn((Xna(i))i <~n) = Cn,p(5)) = 5
(1.6)
for 5 e (0, 1). Then
@n, Perrn :
'Fn
= c.,v(5 ) <
(1.7)
is the T. one-sided permutation test. Two-sided versions can be obtained by the sum of two one-sided tests of level 5/2. Although one might have the impression that permutation tests only work for the restricted i.i.d, null hypothesis rio they are asymptotic tests of exact level 5 for both Examples 1.1 and 1.2 under Ho provided ~0,,pe,m is based on studentized statistics ~.. In contrast to other non-parametric testing procedures with estimated critical values, the permutation procedures have the advantage that they are distribution free and of exact level 5 under the i.i.d, null hypothesis rio for each finite sample size. For instance, the statisticians can bootstrap ~, and apply a test with bootstraped critical values. In general, they do not keep the desired level 5, not even under rio. The asymptotic results of this paper support the permutation tests for a non-i.i.d. hypothesis and the Monte Carlo results of Section 5 recommend them for finite sample size at least in some neighborhood of rio. On the other hand, recall that every rio-similar level 5 test is a permutation test whenever the order statistics are boundedly complete under the restricted null hypothesis. Thus, our generalization fits in that common approach. Permutation methods can be found in the monographs of Edgington (1995) and Good (1994).
2. Permutation tests and the generalized Behrens-Fisher problem In this section Example 1.1 is studied in detail. The null hypothesis (or its boundary) Ho = {#1 = //2} can be divided into four non-disjoint parts Ho = H1 w H2 w H3 w H4, where H1 = {//1 =//2, al = a2,
Zi
standard normal},
H2 = {//1 ---//2, gi standard normal}, H3 = {//1 = / / 2 , ffl = ° ' 2 }
and
H4=Ho\(HlwH2wH3),
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
12
where the restricted i.i.d, null hypothesis is just H3. For each sub-hypothesis Hi, j = 1, 2, 3, classical tests are available which have some optimality properties. Again, we restrict ourselves to one-sided tests since two-sided tests work similarly. Let ~O.,stuabe students exact two-samples t-test which is optimal for H1 against normal regression. If the permutation test (1.7) is based on the numerator Tn (1.2) only we arrive at Pitman's optimal two-sample test for Ha against location alternatives which is denoted by q~n,Pitm. The testing problem given by H2 is known to be the famous Behrens-Fisher problem; see Behrens (1929) and Fisher (1936). The research of the Russian school shows that there exists no reasonable strictly valid test of exact size a at finite sample size for H2; see Linnik (1968) for further references. There is a huge amount of literature and proposals concerning the Behrens-Fisher problem. For instance, Thomasse (1974) summarized practical recipes of Fisher, Banerjee, Pagurova, Wald, H~jek, Welch, Welch and Aspin who gave interesting contributions. One of the most interesting tests is the well-known Welch test, Welch (1937, 1947), based on (1.2), ~n,Weleh = {~
~n
>~<
tw,
(2.1)
where the critical value t~ is calculated by Welch's method. Pfanzagl (1974) pointed out that the Welch test is asymptotically optimal of second order in the class of all translation invariant tests. It seems to us that none of the three tests q)n,Stud, q)n.Pitmand q~n,Welehcan be used for the general null hypothesis H0. For instance, Romano (1990) showed that Pitman's test only works well for equal sample size n~ = nz or al = az. Beran (1988) and Hall and Martin (1988) applied bootstrap tests to the Behrens-Fisher problem. For the non-parametric situation Sen (1962) and Fligner and Policello (1981) proposed rank tests with estimated variances for instance. In this section let q~n,P~rmdenote the one-sided permutation version (1.7) of the Welch test given by the studentized statistic ~n (1.2). Theorem 2.1 shows that q~.,P~rmis asymptotically at exact level ~ under Ho and it has the same asymptotic behavior as the optimal tests for the considered subregions under contiguous alternatives. The proofs are presented in Section 4. Theorem 2.1. Under the assumption of Example 1.1 we have as min(nl, n2) --~ oo (a) lim.-~oo Ee(~O.,pCrm) = ct for each P e Ho. (b) P e H1
~
P e H2 ~
~o.,po,.~ - ~%,Stud ~ 0,
q)n,Perm-- ~0n,Welch ~ 0,
P ~ H3 :=> q)n,Perm-- q)n,Pitm ~ 0, in probability as n -~ oo.
The results of Section 3 apply to the asymptotic power function of ~o.,ve,munder shift alternatives. Theorem 2.2. Consider means #ns within the groups j = 1, 2 of Example 1.1. Suppose that #j = lim n_.o~/t.i exists for j = 1, 2 and min(nl, n2) ~ oo holds. (a) (Consistency) I f lim E(~on,perm)~ 1.
(#~1 - ~.2) ~ oo we have
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
(b) If n l / n ~ ~: and
13
(#.1 - #.2) ~ K we have
lim E(~0,,P~rm)~ 1 - ~ ( u l - , - K/((1 - x)a 2 + ~:a2)1/2), n~oo
where q~ denotes the standard normal distribution f u n c t i o n and ~ ( U x _ , ) = 1 - ~.
As extension of Theorem 2.2(b) one can compute the asymptotic power function for other local alternatives and the asymptotic relative efficiency ARE of ~0.,po~. Moreover, the results apply to other statistics where X~ can be substituted by g(Xi) with finite variance.
3. A central limit theorem for conditional permutation distributions In this section we will derive the asymptotic distribution of studentized permutation statistics. The results apply to fixed or local alternatives which can be used to compute asymptotic power functions, see Section 2. Some results are of own interest. Conditional central limit theorems for linear permutation statistics can be found in the early German text book of Witting and N611e (1970). Recently, Romano (1989, 1990) studied the asymptotics of randomization tests and Mason and Newton (1992) applied conditional limit theorems to the bootstrap for the mean. In connection with survival permutation tests we refer to Neuhaus (1988) and Janssen (1989) where the interaction between scores and regression coefficients and vice versa, used in the present proof of Lemma 3.4, was a basic tool. Most of the proofs given in the literature rely on the same conditioning argument so that H&jek-type central limit theorems for rank statistics can be applied. Neuhaus (1993) introduced first permutation tests for studentized survival for some non-i.i.d, null hypothesis. He extended some results of Janssen (1991). Non-parametric permutation statistics can be handled by new empirical process considerations also under alternatives; see Praestgaard (1995). Let X.1, ..., X,, :(~2, ~¢, P) -~ R denote arbitrary random variables (not necessarily independent). We will consider a linear statistic T . = T , ( X n l , ... , X . , ) :=
.L ~ c,iX, i
(3.1)
i=l
given by a scheme of regression coefficients (c.~)~. for each n e N. Moreover, let T, := T,/V~,/2
(3.2)
be a studentized version of (3.1), where V. = V.((X.i)~.) may be viewed as a variance estimator. Below we will establish asymptotic normality of the permutation statistic generated by 7~,. It is based on uniformly distributed permutations (a(i))i ~. of 1, ..., n given by a probability measure/~ independent of the X.i. To be more explicit the permutation statistic is defined by (a(i)), ~, ~ T , ( ( X . ~ , ) ) i ~,)
(3.3)
for fixed values (X.i(~))i ~,. We will answer the following question. When does there exist a constant ff > 0 such that (3.3) is asymptotically normal, i.e. when does sup(IP(~V.((X,,,))~,) ~< t l X , 1 , ... , X , , ) - ~(t/~)l) b'-' 0 teR
converge to zero in P-probability?
(3.4)
d. Janssen / Statistics & Probability Letters 36 (1997) 9-21
14
The results have interesting applications for permutation tests. Let c,,p(c¢) = c,,p(~, co) denote the (1 - :¢)quantil of the distribution of (3.3) for fixed ~o e f2. Then (3.4) implies
c.,~(~) ~ ~Ul _~
(3.5)
in probability as n ~ oo. We will make some general assumptions
~ Zc.i=l, i=l
~c,i=0
foreachn~N.
(3.6)
i=1
Let X, := 1/n ~7= 1 X,i denote the average and suppose that lim inf 1 ~, (X,, - )?.)2 > 0 ,-~ n i=1
P-a.s.
(3.7)
Assume also that there exists some # > 0 such that 1
(X.i
8,)2(V,((X,,~,))i<,,))-a_____~ ~2
n i=1
(3.8)
P®~
holds in P ® P-probability as n ~ ~. Let L 1(0, 1) (and L 2 (0, 1), respectively) be the set of integrable (square integrable) functions on (0, 1) w.r.t. the uniform distribution. Two types of results are offered where either stronger conditions (3.9) or weaker assumptions (3.15) concerning the regression coefficients are required. Theorem 3.1. Suppose that for each e > 0 there exists a constant d > 0 such that
c.i2 l[a,~) (Inl/Z c.i]) <<.e
(3.9)
i=1
holds for all n ~ ~. I f in addition to (3.6)-(3.8) the condition max In- 1/2X,i[ F ~ 0
(3.10)
i~n
holds then we have asymptotic normality (3.4) in probability with ~ given by (3.8). Remark. Condition (3.9) is just the uniform integrability of the sequence of step functions u~-*ncZ.~l +E.,1), u ~ (0, 1), in LI(0, 1) (given by the entire function []). Notice that under assumption (3.10) the array n-1/ZX,1, ..., n-1/2X,, is infinitesimal, i.e for each ~ > 0 max P([n-1/2X.i[ >>,~) ~ 0
(3.11)
l <~i<~n
holds as n ~ oo. In the case of independent random variables the condition (3.10) is closely related to the central limit theorem. According to Gnedenko and Kolmogorov (1968) our statement (3.10) is then necessary for the validity of an unconditional central limit theorem for the suitably centered sums n-1/2 ~7= i X,i - a, provided (3.11) holds. The next lemma summarizes some results of Gnedenko and Kolmogorov 1968, p. 126). Its part (a) is well known in extreme value theory. Lemma 3.2. Let (X.i)i <~, be row-wise independent for each n. (a) Condition (3.10) is equivalent to
~ P(lX,il >~na/2e)--*O i=1
foreache>0.
(3.12)
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
15
(b) Suppose that (3.11) holds and that S. := n-1/2 ~,7= 1 X n i __ a. is tight for some real numbers a,. Then (3.10) holds if all cluster points of S, with respect to convergence in distribution are normally distributed. In the case of the two-sample problem, it sometimes happens (for nx/n ~ 0) that condition (3.9) is violated for the scheme c,g. We have again asymptotic normality (3.4) under weaker assumptions (3.13) where now a stronger condition (3.14) for X.~ is needed. Theorem 3.3. Assume that in addition to (3.6)-(3.8) we have (3.13)
max Ic,i] ~ O as n ~ oo.
l <~i<~n
Suppose that 1 "__L lim s u p - ~ ( X , i - X , ) 2 l[a,o~)([X,i- X , [ ) ~ 0 n~
P-a.s.
(3.14)
n i= 1
as d --* oo. Then asymptotic normality (3.4) holds.
For completeness we summarize a Hfijek-type central limit theorem for rank statistics which can also be deduced from H~tjek and Sid~tk (1967, p. 193ff). In order to make our paper self-contained, we give an elementary proof of Lemma 3.4. Observe that the basic tool, the central limit theorem for L2(0, 1) covergent score functions of Hgjek and Sidhk (1967, Theorem V 1.5(a)), can nowadays be proved at the level of an elementary statistic course, see Witting and Miiller-Funk (1995, p. 638). Lemma 3.4. Consider regression coefficients c.i with (3.6) and (3.9). Let b.i be a further scheme such that (3.6) and (3.13) holds for d.i:= n-1/Z b.i. Then S n = ~ cnlbnali) i=l
is asymptotically standard normal under uniformly distributed permutations.
Proof. Since n
S, ~ ~ nl/Z c,~i)d.i =: S',
(3.15)
i=l
are equal in distribution we obtain a rank statistic S'. with new regression coefficients d,i and new scores a,(i) := n a/2 c,i. Without any restriction, we may assume that c,a <<,c,2 <~ ". <~ c,, holds. Let q~,(u):= a.(1 + [nu]) = nl/2c,~1+[,,]),
u~(0, 1)
be the associated increasing step function in LZ(O, 1). Then the functions {~o,: n ~ ~} are relatively compact in L2(O, 1). This proof is elementary and can be given by standard arguments involving increasing functions. Due to (3.6) one obtains a subsequence {m} such that tpm(u) converges for rational u's and then a.s. Uniform integrability (3.9) shows L2(0, 1) convergence. Theorem V 1.5(a) of Hfijek and Sid~tk (1967) now proves the central limit theorem first along L2(0, 1) convergent subsequences of (¢o,),. [] It is easy to see that Lemma 3.4 also holds for schemes of length k(n) --, ~ . Our Theorems 3.1 and 3.3 are closely related to the results of Mason and Newton (1992), Section 2, where a.s. convergence results of the numerator of(3.2) are treated. Lemma 3.4 now establishes in both cases a rapid
16
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
proof of the conditional central limit theorem for studentized statistics where the influence of Hhjek's ideas and the conditioning arguments is transparent. The proofs of Theorem 3.1 and 3.3. Throughout, we will establish P-a.s. convergence of (3.4) along certain subsequences of a given subsequence. For fixed X,~(og) write
T.((X.o(i)), ~ .) = ~ c.i (X.,,(1) - X.) =: T',((a(i), ~ ,). i=l In order to apply Lemma 3.4, set b.i = (X,i(o)) -)?.(~o))( 1/n Y~'=1 (X.~(o)) -Xdco))2) - x/2 Whenever (3.10) is P-a.s. convergent for a subsequence {k} c ~ we obtain convergence in distribution (with standard normal limit N(0, 1)) of the conditional distributions
( (1 k - 2xx-1/2'~'~ L# T'k -ki~, ( X k i - Xk) ) P)
-+ N(0, 1)
(3.16)
for fixed X,~(oJ) as k -+ oo with P-probability 1. Passing once more to a.s. convergent subsequences of(3.8) we arrive at the conditional asymptotic normality (3.4) under the conditions of Theorem 3.1. Lemma 3.4 applies to Theorem 3.3 in the same way. The choice b,~ := n ~12c,~ and
/
c.~ : = ( X ~ : . ( o )
- X.(~o
(X.~(~o) - J~.(~o) 2
-~/2
i for fixed ~o e Q proves asymptotic normality of (3.16) P-a.s. along suitable subsequences.
[]
It is easy to see that there exist symmetric two-point variables such that the central theorem holds for S,, see Lemma 3.2(b), but (3.14) is violated. However, condition (3.14) holds for our two-sample problem also when (3.9) is violated. Since T, is invariant under the same shift for all X,~ we can always assume the condition
~E(Xni ) i=l
=
0
for all n e N.
(3.17)
4. The proofs of Section 2 Throughout, let X.~ .... ,X.. denote the pooled sample of Example 1.1 given by X.i = X~ for i ~< nl and X.~ = Y~-.1 for i > nl. At first, we calculate the asymptotic permutation distribution under alternatives. Without restrictions we may assume that the means satisfy n~la.1 + nzt,tn2 = 0, cf. (3.17).
Lemma 4.1. Assume that min(nl,n2) -~ ~ and lim.-,oo nffn = x holds for some x ~ [0, 1]. Suppose that as in Theorem 2.1 the means #.j of the jth sample are convergent to #jfor j = 1, 2. Then the permutation distribution of the test statistic T. given by (1.2) is asymptotically normal in the sense of(3.4) with ~ = 1. Proof. Define regression coefficients for 1 ~< i ~< n by
= ( n a n z y / 2 ~ l/n1 c.,
\--g-]
( - 1/n2
for
i<~nx,
(4.1)
i > nl.
Then the statistic ~, has just the form of (3.2) and the conditions (3.6) and (3.13) hold. It suffices to check the assumptions of Theorem 3.3. We will see that the validity of (3.14) is an immediate consequence of the strong
A. Janssen / Statistics & ProbabilityLetters 36 (1997) 9-21
17
law of large numbers. Observe that X. ~ 0 a.s. since x#l + (1 - x)#2 = 0. To prove (3.14) it remains to show that 1
"
lira s u p - ~ X2.i ltd,oo)(IX.i[) ~ 0 ,~ n i=1
P-a.s.
(4.2)
as d ~ ~ . This statement follows from the strong law of large numbers since the means E(X2,1) are bounded. Next, the conditions (3.7) and (3.8) are checked. Routine calculations show that
n 1 ~(X,, ~/i=
X,) z + x ( a ~ + #~) + (1 - K)(a 2, + #~)
P-a.s.
(4.3)
To prove (3.8) let us divide V. = W,1 - W,22 - W.~ into three parts given by
Wnl = (nl
? , / 2 nl nl ~ 2 1)n Z X2, + - i=1 (n2 1)n i=.~+1 X.,,
// nly/2 'xl/2 1 ~
w.2 = t,i.;-i).)
,z51x""
w.,
=(
?/1712 ~1/21
+
x"'
For each part the asymptotic permutation distribution of W.j((X.,,~))i ~.)) can be computed separately. Notice that
E(W.E((X.~))~<.IX.1, ... ,X..)) = (n~ - 1)n
X. ~ 0 .
(4.4)
Its conditional variance is according to the variance formula of Hijek and Sid/~k (1967) equal to
h--n
e-.a.s.
i=1
Thus, W~2((X.~o)~ <.) converges to zero in P ® P-probability. Similarly, it is proved that
W.3((X.,,~i)), <~,) ~
0
(4.5)
converges in probability. Finally W.1 is treated. It can be shown that W,I((X.~i))~<,) ~
p ® p
x(a~ + #~) + (1 - x)(a z + #z) =: p -
_
(4.6)
is convergent in probability. For its proof introduce the regression coefficients
i~nl, i>nl.
d.i = )" nz((nl - 1) n)- 1 for [nl((n2 - 1)n) -1 Obviously, we have
W.1 = ~ d,i X2.i i=1
(4.7)
and
nd,:= ~, d . i ~ l, i=1
• i=1
(dni
-
a~n) 2 ~ 0.
(4.8)
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
18
The statistic (4.6) will be dealt with by a truncation argument. Notice, first that its conditional expectation is convergent E
d . g X .2o . ) I X . 1 . . . . . X . .
=
i
2~ P Z X ni
d., i=1
(4.9)
i=1
P-a.s. If Zi has finite fourth moment, we see that 2 lXnl , ... ,Xnn dniXna(i)
Var i=
)
=
(dni--dn)2-~ i=l
X.2i - -
l, --
X.j
"=
P-a.s. if (4.8) is taken into account. Thus, ~ d n i Xna(i) 2 ~
(4.10)
P"
i=1
If E ( Z ~ ) = ~ let us decompose X.i = X'.i + )?.i:= ( a l Z i 1E_r,Kj(Z~) + #.1) + alZ~ IE_K,~1o(Zi) for i ~< nl for positive K and similarly for i > nl and al and #.1 replaced by aa and #.2. Obviously, we have again ~ d n i Xna(i) ,2 ~ i=l
(4.11)
PK
in probability, where PK = x l i m . ~ E(X'.]) + (1 - K) llm._.~ E(X..),'2 and E
\i = 1
-2 d.iX.,,ti)
(4.12)
~ P - PK
which becomes arbitrary small as K ~ oo. Since all random variables are non-negative we may apply " 1 d.i X.~.) becomes uniformly small in P ® P-probability whenever K is large. Markov's equality and Zi= ~2 Since X2i = X'.2 + )~2 holds, the desired statement (4.6) follows from (4.11) and (4.12). [] Proof of Theorem 2.1. Under/~1 = #z the statistic 2r. is asymptotically standard normal. By Lemma 4.1 the permutation critical value c.,p(e) converges in probability to u l - . . This assertion can be proved (by the subsequence principle for convergence in probability) first along subsequences, where nl/n is convergent. Notice that all other critical values of q~.,~tud"" are convergent to the same constant. [] Proof of Theorem 2.2. Obviously, V. is distribution free under shift alternatives and V . ( ( X . i ) i < . ) - ~ ( 1 - K)a 2 + x a ~ .
P-a.s. if nx/n ~ to. In the next step write r.((x.,),~.)
=
Z z,i=1
=:
--
z,
+
(#.,~ - #.,~)
/'12 i = n a + l
Tnl + Tn2.
Notice that T.I/V~./2 is always asymptotically standard normal, Since c.,p(~)~ul-~ is convergent in probability, we have consistency if Tn2 ~ zt3, since T.2/V~./2 ---, ~ . Under T.,2 ---,K we obtain the asymptotic power function of Theorem 2.2(b). []
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
19
5. Monte Carlo results In this section, we compare the actual size of type I errors of the studentized permutation test q),,Perm of Section 2 with the Welch test ~0,,Wel¢hfor normal and skewed distributions. Various Monte Carlo studies can be found in the literature. We refer to Best and Rayner (1987) for a power study of different tests (including the Welch test) for normal distributions. They also discussed the size of type I errors and summarized the results of earlier Monte Carlo computations, see also Wang (1971). In the normal case the Welch test can be recommended. Mehta and Srinivasan (1970) compared the Welch test with other classical tests for the Behrens-Fisher problem. Fung (1979) considered the accuracy of studentized Wilcoxon tests for the generalized Behrens-Fisher problem (Example 1.1) with nonnormal errors. The paper contains a comparison with the Welch test also under alternatives. Below, Tables 1-3 show some type I error probabilities where the nominal level was e = 0.0498, 0.0485 and 0.05, respectively, for computational reasons. The computations were carried out for normalized error variables Z ( = Zi) with E(Z) = 0 and Var(Z) = 1, see Example 1.1. In particular, we choose Z to be normal, log-normal Z = exp (~), where ~ is normal with mean ( - log 2)/2 and variance log(2), a shifted standard exponential random errors Z, and a uniformly distributed Z on the interval [ - ~ , ,,/~]. The computations are based o n 106 Monte Carlo replications. Under the conditions of Table 1 the permutation test q~,,P,rm is always better than cP,,w~I~h- Especially, we see that the Welch test may be bad for skewed distributions. For different sample sizes n~ = 4 and/72 ---- 8 (Table 2) the results are more complicated. Of course, if the errors are known to be normal then the Welch
Table 1 Type I error probability, nl = 6, n2 = 6, ~1 =/~2, ~ = 0.0498 Distribution
a~ : azz
1.0 : 1.0
1.0: 1.1
1.0 : 1.2
Normal Log-normal
qS,. wo,ch qS,. Por,~ qS,, W~,~h
Exponential
~bn, Perm 4~., Welch
0.0460 0.0501 0.0251 0.0496 0.0291 0.0494 0.0489 0,0500
0.0458 0.0500 0.0254 0.0500 0.0297 0.0501 0.0487 0.0500
0.0459 0.0500 0.0261 0.0507 0.0302 0.0504 0.0490 0.0502
~b,, P ~ m
Uniform
~b., wel~h q~,. Porto
Table 2 Type 1 error probability, nl = 4, n 2 = 8, #1 = #a, c~ = 0.0485 Distribution
a12'.a22
1.2 : 1.0
1.1 : 1.0
1.0 : 1.0
1.0: 1.1
1.0 : 1.2
Normal
q~., welch
0.0509 0.0523 0.0453 0.0555 0.0518 0.0556 0.0663 0.0538
0.0495 0.0497 0.0416 0.0516 0.0478 0.0518 0.0646 0.0508
0.0497 0.0481 0.0379 0.0486 0.0442 0.0481 0.0627 0.0482
0.0488 0.0456 0.0346 0.0455 0.0402 0.0447 0.0606 0.0452
0.0478 0.0436 0.0316 0.0429 0.0377 0.0426 0.0589 0.0430
~b., P ~ m
Log-normal Exponential Uniform
~b..wolch ~bn. Perm qS.,W~ch q~n, Perm q~.,Wclch q}n,Perm
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
20
Table 3 Type I error probability, n~ = 8, n 2 = 8, #1 = g2, ~ = 0.0500 Distribution
a~:a~
1.0:1.0
1.0:1.1
1.0:1.2
1.0:1.5
1.0:2.0
Normal
~bn,Welch q~,.Perm ~b~.welch ~bn.pcrm ~bn.welch ~bn,Pe~m ~b~.Welch ~b~.P~m
0.0478 0.0506 0.0291 0.0504 0.0342 0.0507 0.0502 0.0506
0.0476 0.0503 0.0292 0.0504 0.0344 0.0504 0.0497 0.0502
0.0480 0.0508 0.0299 0.0515 0.0350 0.0513 0.0501 0.0505
0.0479 0.0510 0.0339 0.0555 0.0382 0.0541 0.0506 0.0514
0.0486 0.0525 0.0426 0.0634 0.0449 0.0599 0.0516 0.0528
Log-normal Exponential Uniform
test should be applied (first row of Table 2). In the other cases, the permutation test works better (apart from two exceptional cases). It seems to us that an unequal sample size produces a further bias of the error I probability. Even under al = a2 = 1 (third column of Table 2) the accuracy of the Welch test is not acceptable. Table 3 contains further simulations for larger differences of the variances. Again the new permutation is acceptable except of two cases for skewed distributions under ¢r~/a~ = 2 (last column of Table 3).
Conclusions The present Monte Carlo results support the studentized permutation tests for models with non-parametric errors and slightly different variances.
Acknowledgements I thank H. Hebben for his computational assistance.
References Andersen, P.K., Borgan, O., Gill, R., Keiding, N., 1993. Statistical models based on counting processes. Springer Series in Statistis. Springer, New York. Behrens, W.V., 1929. Ein Beitrag zur Fehlerberechnung bei wenigen Beobachtungen, Landwirtsch. Jahrbiicher 68, 807-837. Beran, R., 1988. Prepivoting test statistics: a bootstrap view of asymptotic refinements. J. Amer. Statist. Assoc. 83, 687-697. Best, D.J., Rayner, J.C.W., 1987. Welch's approximate solution for the Behrens-Fisher problem. Technometrics 29, 205-210. Edgington, E.S., 1995. Randomization Tests, 3rd ed. Marcel Dekker, New York. Fisher, R.A., 1936. The fiducial argument in statistical inference. Ann. Eugenics 6, 391-422. Fligner, M., Policello, G., 1981. Robust rank procedures for the Behrens-Fisher problem. J. Amer. Statist. Assoc. 76, 162-168, Fung, K.Y., 1979. A Monte Carlo study of the studentized Wilcoxon statistic for the Behrens-Fisher problem. J. Statist Comput. Simul. 10, 15-24. Good, P., 1994. Permutation tests, a practical guide to resampling methods for testing hypothesis. Springer Series in Statistics. Springer, Berlin. Gnedenko, B.V., Kolmogorov, A.N., 1968. Limit Distributions of Sums of Independent Random Variables. Addison-Wesley, Reading, MA. H~tjek, J.A., ~idfik, 1967. Theory of Rank Tests. Academic Press, New York.
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
21
Hall, P., Martin, M., 1988. On the bootstrap and two-sample problems. Austral. J. Statist. A30, 179-192. Janssen, A. 1989. Local asymptotic normality for randomly censored data with applications to rank tests. Statist. Neerlandica 43, 109-125. Janssen, A. 1991. Conditional rank tests for randomly censored data. Ann. Statist. 19, 1434~1456. Linnik, J.N., 1968. Statistical problems with nuisance parameters. Translation of Mathematical Monograph, vol. 20. American Mathematical Society, Providence, RI. Mason, D.M., Newton, M.A., 1992. A rank statistics approach to the consistency of a general bootstrap. Ann. Statist. 20, 1611-1624. Mehta, J.S., Srinivasan, R., 1970. On the Behrens-Fisher probem. Biometrika 57, 649-655. Neuhaus, G., 1988. Asymptotically optimal rank tests for the two-sample problem with randomly censored data. Comm. Statist. Theory Methods 17, 2037-2058. Neuhaus, G., 1993. Conditional rank test for the two-sample problem under random censorship. Ann. Statist. 21, 1760-1779. Pfanzagl, J., 1974. On the Behrens-Fisher problem. Biometrika 61, 39-47. Praestgaard, J.T., 1995. Permutation and bootstrap Kolmogorov-Smirnov tests for the equality of two distributions. Scand. J. Statist. 22, 305-322. Romano, J.P., 1989. Bootstrap and randomization tests of some nonparametric hypothesis. Ann. Statist. 17, 141-159. Romano, J.P., 1990. On the behaviour of randomization tests without a group invariance assumption. J. Amer. Statist. Assoc. 85, 686-692. Sen, P.K., 1962. On studentized non-parametric multi-sample location test. Ann. Math. Statist. 14, 119-131. Thomasse, A.H., 1974. Practical recipes to solve the Behrens-Fisher problem. Statistica Neerlandica 28, 127-138. Wang, Y.Y., 1971. Probabilities of the type I errors of the Welch tests for the Behrens-Fisher problem. J. Amer. Statist. Assoc. 66, 605-608. Welch, B.L., 1937. The significance of the difference between two means when the population variances are unequal. Biometrika 29, 350-362. Welch, B.L., 1947. The generalization of Student's problem when several different population variances are involved. Biometrika 34, 28-35. Witting, H., Miiller-Funk, U., 1995. Mathematische Statistik II. Teubner, Stuttgart. Witting, H., N~511e,G., 1970. Angewandte Mathematische Statistik. Teubner, Stuttgart.
Statistics & Probability Letters 36 (1997) 9-21
Studentized permutation tests for non-i.i.d, hypotheses and the generalized Behrens-Fisher problem Arnold Janssen* Mathematical lnstitut, University of Dfisseldorf Universitiitsstrasse1, D-40225 Diisseldorf Germany
Received 1 December 1995; received in revised form 1 December 1996
Abstract It is shown that permutation tests based on studentized statistics are asymptotically exact of size ~ also under certain extended non-i.i.d, null hypotheses. To demonstrate the principle the results are applied to the generalized two-sample Behrens-Fisher problem for testing equality of the means under general non-parametric heterogeneous error distributions. Within this setting we propose a permutation version of the Welch test which is an extension of Pitman's two-sample permutation test. These results are special cases of a conditional central limit theorem for studentized permutation statistics which also applies to asymptotic power functions. © 1997 Elsevier Science B.V. AMS classification: 62G 10; 62G09 Keywords: Behrens-Fisher problem; Permutation test; Permutation statistic; Conditional central limit theorem; Survival
test
1. Introduction and examples Throughout, we are concerned with a two-sample non-parametric testing problem given by an extended null hypothesis Ho which is strictly larger than the restricted null hypothesis of i.i.d, random variables. The restriction of Ho to the i.i.d, case is denoted by iZlo, i.e. 1-7Io~ Ho. A famous example is the extended Behrens-Fisher problem where equality of two means is tested for a model with different unknown variances of the error variables. For this type of examples the choice of critical values (under Ho) is a serious problem. As message of this paper we suggest to carry out the underlying tests as permutation tests for studentized statistics which includes a variance correction of the permutation distribution. It is shown that in this case the conditional critical values of the permutation distribution work well at least in the asymptotic setting. We follow the approach of Neuhaus (1993), who successfully applied studentized permutation tests for survival problems, see Example 1.2 below.
* Tel.: + 49 21181 12165; fax: + 49 211 81 13117; e-mail:[email protected]. 0167-7152/97/$17.00 © 1997 Elsevier Science B.V. All rights reserved PII S0 1 67-7 1 52(97)00043-6
10
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
To explain the idea suppose that T~ is a reasonable test statistic, for instance, the linear test statistic (1.2) below. By definition permutation tests, given by T,, are distribution free and of exact level ~ for the i.i.d, null hypothesis only. Even in the asymptotic setup they are in general no longer of size 0~for Ho. To overcome this difficulty, permutation tests are proposed for studentized statistics 2V, = T,/V)/2, where V, is a suitable variance estimator. It is essential that the permutation procedure takes the denominator of T, into account which gives the permutation distribution its right asymptotic variance. This is the common feature of the following examples. Example 1.1 (Extended Behrens-Fisher problem). Let (Z/)/~ denote a sequence of real i.i.d, error variables with E(Zi) = 0 and Var(Zi) = 1. Let al, a2 denote two arbitrary positive unknown standard deviations. Consider then the two-sample problem of total sample size n = nl + n2 X i = [,21 --~ t T i Z i ,
1 <~i <~n l ,
l l l U ff~,
and
Yi:~2-]-ff2Znl+i, l<~i<<,n2, #2eN. The testing problem may be given by the one-sided null hypothesis {#1 ~< #2} or the two-sided analogue {Pl = P2}, where trl and a2 are nuisance parameters. It is crucial that tests should approximately attain the level e at the non-i.i.d, null hypothesis Ho = {#, = #2}
(1.1)
(or its boundary Ho for the one-sided case). The restricted null hypothesis is given by 17Io-- {#1 = P2, al = a2}. Common tests are based on the means X = 1- ~ X,,
F=--I ~ y,
nl i=1
n2 i=1
and
~ _ ~ ) 1/2 =
(£-
?)
or
= Tn/V /2,
(1.2)
where
Vn = nln2 ($2/nl + $2/n2)
(1.3)
n
defined by nl
- -
nl
--
1
E ( x , - £)2, 1
i=l
= -
n2
E (Y-
n2-- l i=l
It is a serious problem how to carry out the test by an appropriate choice of critical values. Example 1.2 (Random censoring model). Consider two groups of survival times $1 . . . . . S., and S,, + 1. . . . . S. which are i.i.d, within each group and censored by censoring variables C1, ..., C,, and C~, + 1, ..., C. again i.i.d, for each group and independent of the S's. Within the random censorship model only X~ = min(S~, Ci),
At = l(Si ~< C~), 1 ~< i ~< n,
(1.4)
are observable. Under the null hypothesis of equal survival distributions Ho = { ~ e ( s , ) = ~(s~)}
(1.5)
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
11
it is, in practice, often not realistic to assume equal censoring distributions £°(C1) and Y'(C.), which can be viewed as nuisance parameters. Thus, the observed random variables (1.4) are no longer i.i.d, under Ho since censoring destroys the i.i.d, structure. The restricted null hypothesis is given by
rio = {¢-~9($1) = ~(Sn), ~9(C1) = ¢~(Cn)}. Let T, be a survival test statistic, for instance the log-rank statistic, see Anderson et al. (1993), and let V. be a sequence of associated variance estimators. A remarkable result of Neuhaus (1993) shows that permutation tests based on 7", = T,/V~./2 are asymptotically of exact level 5 under the extended null hypothesis (1.5). His Monte Carlo results support these tests at finite sample size. Since we do not like to overload the paper, results concerning conditional survival statistics based on Section 3 will be published in a forthcoming paper. We briefly recall the notion of permutation tests (1.7) below which are conditional tests given by the data. Let ~, = T.((X,i)i~.) be an arbitrary test statistic based on random variables X,i. For fixed values x.i of X,~ and uniformly P-distributed random permutations (a(i))~ ,<. of 1, ..., n, which are independent of the data, let 7, E [0, 1] and C,,p(5) be the solutions of P(Tn((Xn6(i))i ~n) > Cn,p(5)) q- 7nP(Tn((Xna(i))i <~n) = Cn,p(5)) = 5
(1.6)
for 5 e (0, 1). Then
@n, Perrn :
'Fn
= c.,v(5 ) <
(1.7)
is the T. one-sided permutation test. Two-sided versions can be obtained by the sum of two one-sided tests of level 5/2. Although one might have the impression that permutation tests only work for the restricted i.i.d, null hypothesis rio they are asymptotic tests of exact level 5 for both Examples 1.1 and 1.2 under Ho provided ~0,,pe,m is based on studentized statistics ~.. In contrast to other non-parametric testing procedures with estimated critical values, the permutation procedures have the advantage that they are distribution free and of exact level 5 under the i.i.d, null hypothesis rio for each finite sample size. For instance, the statisticians can bootstrap ~, and apply a test with bootstraped critical values. In general, they do not keep the desired level 5, not even under rio. The asymptotic results of this paper support the permutation tests for a non-i.i.d. hypothesis and the Monte Carlo results of Section 5 recommend them for finite sample size at least in some neighborhood of rio. On the other hand, recall that every rio-similar level 5 test is a permutation test whenever the order statistics are boundedly complete under the restricted null hypothesis. Thus, our generalization fits in that common approach. Permutation methods can be found in the monographs of Edgington (1995) and Good (1994).
2. Permutation tests and the generalized Behrens-Fisher problem In this section Example 1.1 is studied in detail. The null hypothesis (or its boundary) Ho = {#1 = //2} can be divided into four non-disjoint parts Ho = H1 w H2 w H3 w H4, where H1 = {//1 =//2, al = a2,
Zi
standard normal},
H2 = {//1 ---//2, gi standard normal}, H3 = {//1 = / / 2 , ffl = ° ' 2 }
and
H4=Ho\(HlwH2wH3),
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
12
where the restricted i.i.d, null hypothesis is just H3. For each sub-hypothesis Hi, j = 1, 2, 3, classical tests are available which have some optimality properties. Again, we restrict ourselves to one-sided tests since two-sided tests work similarly. Let ~O.,stuabe students exact two-samples t-test which is optimal for H1 against normal regression. If the permutation test (1.7) is based on the numerator Tn (1.2) only we arrive at Pitman's optimal two-sample test for Ha against location alternatives which is denoted by q~n,Pitm. The testing problem given by H2 is known to be the famous Behrens-Fisher problem; see Behrens (1929) and Fisher (1936). The research of the Russian school shows that there exists no reasonable strictly valid test of exact size a at finite sample size for H2; see Linnik (1968) for further references. There is a huge amount of literature and proposals concerning the Behrens-Fisher problem. For instance, Thomasse (1974) summarized practical recipes of Fisher, Banerjee, Pagurova, Wald, H~jek, Welch, Welch and Aspin who gave interesting contributions. One of the most interesting tests is the well-known Welch test, Welch (1937, 1947), based on (1.2), ~n,Weleh = {~
~n
>~<
tw,
(2.1)
where the critical value t~ is calculated by Welch's method. Pfanzagl (1974) pointed out that the Welch test is asymptotically optimal of second order in the class of all translation invariant tests. It seems to us that none of the three tests q)n,Stud, q)n.Pitmand q~n,Welehcan be used for the general null hypothesis H0. For instance, Romano (1990) showed that Pitman's test only works well for equal sample size n~ = nz or al = az. Beran (1988) and Hall and Martin (1988) applied bootstrap tests to the Behrens-Fisher problem. For the non-parametric situation Sen (1962) and Fligner and Policello (1981) proposed rank tests with estimated variances for instance. In this section let q~n,P~rmdenote the one-sided permutation version (1.7) of the Welch test given by the studentized statistic ~n (1.2). Theorem 2.1 shows that q~.,P~rmis asymptotically at exact level ~ under Ho and it has the same asymptotic behavior as the optimal tests for the considered subregions under contiguous alternatives. The proofs are presented in Section 4. Theorem 2.1. Under the assumption of Example 1.1 we have as min(nl, n2) --~ oo (a) lim.-~oo Ee(~O.,pCrm) = ct for each P e Ho. (b) P e H1
~
P e H2 ~
~o.,po,.~ - ~%,Stud ~ 0,
q)n,Perm-- ~0n,Welch ~ 0,
P ~ H3 :=> q)n,Perm-- q)n,Pitm ~ 0, in probability as n -~ oo.
The results of Section 3 apply to the asymptotic power function of ~o.,ve,munder shift alternatives. Theorem 2.2. Consider means #ns within the groups j = 1, 2 of Example 1.1. Suppose that #j = lim n_.o~/t.i exists for j = 1, 2 and min(nl, n2) ~ oo holds. (a) (Consistency) I f lim E(~on,perm)~ 1.
(#~1 - ~.2) ~ oo we have
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
(b) If n l / n ~ ~: and
13
(#.1 - #.2) ~ K we have
lim E(~0,,P~rm)~ 1 - ~ ( u l - , - K/((1 - x)a 2 + ~:a2)1/2), n~oo
where q~ denotes the standard normal distribution f u n c t i o n and ~ ( U x _ , ) = 1 - ~.
As extension of Theorem 2.2(b) one can compute the asymptotic power function for other local alternatives and the asymptotic relative efficiency ARE of ~0.,po~. Moreover, the results apply to other statistics where X~ can be substituted by g(Xi) with finite variance.
3. A central limit theorem for conditional permutation distributions In this section we will derive the asymptotic distribution of studentized permutation statistics. The results apply to fixed or local alternatives which can be used to compute asymptotic power functions, see Section 2. Some results are of own interest. Conditional central limit theorems for linear permutation statistics can be found in the early German text book of Witting and N611e (1970). Recently, Romano (1989, 1990) studied the asymptotics of randomization tests and Mason and Newton (1992) applied conditional limit theorems to the bootstrap for the mean. In connection with survival permutation tests we refer to Neuhaus (1988) and Janssen (1989) where the interaction between scores and regression coefficients and vice versa, used in the present proof of Lemma 3.4, was a basic tool. Most of the proofs given in the literature rely on the same conditioning argument so that H&jek-type central limit theorems for rank statistics can be applied. Neuhaus (1993) introduced first permutation tests for studentized survival for some non-i.i.d, null hypothesis. He extended some results of Janssen (1991). Non-parametric permutation statistics can be handled by new empirical process considerations also under alternatives; see Praestgaard (1995). Let X.1, ..., X,, :(~2, ~¢, P) -~ R denote arbitrary random variables (not necessarily independent). We will consider a linear statistic T . = T , ( X n l , ... , X . , ) :=
.L ~ c,iX, i
(3.1)
i=l
given by a scheme of regression coefficients (c.~)~. for each n e N. Moreover, let T, := T,/V~,/2
(3.2)
be a studentized version of (3.1), where V. = V.((X.i)~.) may be viewed as a variance estimator. Below we will establish asymptotic normality of the permutation statistic generated by 7~,. It is based on uniformly distributed permutations (a(i))i ~. of 1, ..., n given by a probability measure/~ independent of the X.i. To be more explicit the permutation statistic is defined by (a(i)), ~, ~ T , ( ( X . ~ , ) ) i ~,)
(3.3)
for fixed values (X.i(~))i ~,. We will answer the following question. When does there exist a constant ff > 0 such that (3.3) is asymptotically normal, i.e. when does sup(IP(~V.((X,,,))~,) ~< t l X , 1 , ... , X , , ) - ~(t/~)l) b'-' 0 teR
converge to zero in P-probability?
(3.4)
d. Janssen / Statistics & Probability Letters 36 (1997) 9-21
14
The results have interesting applications for permutation tests. Let c,,p(c¢) = c,,p(~, co) denote the (1 - :¢)quantil of the distribution of (3.3) for fixed ~o e f2. Then (3.4) implies
c.,~(~) ~ ~Ul _~
(3.5)
in probability as n ~ oo. We will make some general assumptions
~ Zc.i=l, i=l
~c,i=0
foreachn~N.
(3.6)
i=1
Let X, := 1/n ~7= 1 X,i denote the average and suppose that lim inf 1 ~, (X,, - )?.)2 > 0 ,-~ n i=1
P-a.s.
(3.7)
Assume also that there exists some # > 0 such that 1
(X.i
8,)2(V,((X,,~,))i<,,))-a_____~ ~2
n i=1
(3.8)
P®~
holds in P ® P-probability as n ~ ~. Let L 1(0, 1) (and L 2 (0, 1), respectively) be the set of integrable (square integrable) functions on (0, 1) w.r.t. the uniform distribution. Two types of results are offered where either stronger conditions (3.9) or weaker assumptions (3.15) concerning the regression coefficients are required. Theorem 3.1. Suppose that for each e > 0 there exists a constant d > 0 such that
c.i2 l[a,~) (Inl/Z c.i]) <<.e
(3.9)
i=1
holds for all n ~ ~. I f in addition to (3.6)-(3.8) the condition max In- 1/2X,i[ F ~ 0
(3.10)
i~n
holds then we have asymptotic normality (3.4) in probability with ~ given by (3.8). Remark. Condition (3.9) is just the uniform integrability of the sequence of step functions u~-*ncZ.~l +E.,1), u ~ (0, 1), in LI(0, 1) (given by the entire function []). Notice that under assumption (3.10) the array n-1/ZX,1, ..., n-1/2X,, is infinitesimal, i.e for each ~ > 0 max P([n-1/2X.i[ >>,~) ~ 0
(3.11)
l <~i<~n
holds as n ~ oo. In the case of independent random variables the condition (3.10) is closely related to the central limit theorem. According to Gnedenko and Kolmogorov (1968) our statement (3.10) is then necessary for the validity of an unconditional central limit theorem for the suitably centered sums n-1/2 ~7= i X,i - a, provided (3.11) holds. The next lemma summarizes some results of Gnedenko and Kolmogorov 1968, p. 126). Its part (a) is well known in extreme value theory. Lemma 3.2. Let (X.i)i <~, be row-wise independent for each n. (a) Condition (3.10) is equivalent to
~ P(lX,il >~na/2e)--*O i=1
foreache>0.
(3.12)
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
15
(b) Suppose that (3.11) holds and that S. := n-1/2 ~,7= 1 X n i __ a. is tight for some real numbers a,. Then (3.10) holds if all cluster points of S, with respect to convergence in distribution are normally distributed. In the case of the two-sample problem, it sometimes happens (for nx/n ~ 0) that condition (3.9) is violated for the scheme c,g. We have again asymptotic normality (3.4) under weaker assumptions (3.13) where now a stronger condition (3.14) for X.~ is needed. Theorem 3.3. Assume that in addition to (3.6)-(3.8) we have (3.13)
max Ic,i] ~ O as n ~ oo.
l <~i<~n
Suppose that 1 "__L lim s u p - ~ ( X , i - X , ) 2 l[a,o~)([X,i- X , [ ) ~ 0 n~
P-a.s.
(3.14)
n i= 1
as d --* oo. Then asymptotic normality (3.4) holds.
For completeness we summarize a Hfijek-type central limit theorem for rank statistics which can also be deduced from H~tjek and Sid~tk (1967, p. 193ff). In order to make our paper self-contained, we give an elementary proof of Lemma 3.4. Observe that the basic tool, the central limit theorem for L2(0, 1) covergent score functions of Hgjek and Sidhk (1967, Theorem V 1.5(a)), can nowadays be proved at the level of an elementary statistic course, see Witting and Miiller-Funk (1995, p. 638). Lemma 3.4. Consider regression coefficients c.i with (3.6) and (3.9). Let b.i be a further scheme such that (3.6) and (3.13) holds for d.i:= n-1/Z b.i. Then S n = ~ cnlbnali) i=l
is asymptotically standard normal under uniformly distributed permutations.
Proof. Since n
S, ~ ~ nl/Z c,~i)d.i =: S',
(3.15)
i=l
are equal in distribution we obtain a rank statistic S'. with new regression coefficients d,i and new scores a,(i) := n a/2 c,i. Without any restriction, we may assume that c,a <<,c,2 <~ ". <~ c,, holds. Let q~,(u):= a.(1 + [nu]) = nl/2c,~1+[,,]),
u~(0, 1)
be the associated increasing step function in LZ(O, 1). Then the functions {~o,: n ~ ~} are relatively compact in L2(O, 1). This proof is elementary and can be given by standard arguments involving increasing functions. Due to (3.6) one obtains a subsequence {m} such that tpm(u) converges for rational u's and then a.s. Uniform integrability (3.9) shows L2(0, 1) convergence. Theorem V 1.5(a) of Hfijek and Sid~tk (1967) now proves the central limit theorem first along L2(0, 1) convergent subsequences of (¢o,),. [] It is easy to see that Lemma 3.4 also holds for schemes of length k(n) --, ~ . Our Theorems 3.1 and 3.3 are closely related to the results of Mason and Newton (1992), Section 2, where a.s. convergence results of the numerator of(3.2) are treated. Lemma 3.4 now establishes in both cases a rapid
16
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
proof of the conditional central limit theorem for studentized statistics where the influence of Hhjek's ideas and the conditioning arguments is transparent. The proofs of Theorem 3.1 and 3.3. Throughout, we will establish P-a.s. convergence of (3.4) along certain subsequences of a given subsequence. For fixed X,~(og) write
T.((X.o(i)), ~ .) = ~ c.i (X.,,(1) - X.) =: T',((a(i), ~ ,). i=l In order to apply Lemma 3.4, set b.i = (X,i(o)) -)?.(~o))( 1/n Y~'=1 (X.~(o)) -Xdco))2) - x/2 Whenever (3.10) is P-a.s. convergent for a subsequence {k} c ~ we obtain convergence in distribution (with standard normal limit N(0, 1)) of the conditional distributions
( (1 k - 2xx-1/2'~'~ L# T'k -ki~, ( X k i - Xk) ) P)
-+ N(0, 1)
(3.16)
for fixed X,~(oJ) as k -+ oo with P-probability 1. Passing once more to a.s. convergent subsequences of(3.8) we arrive at the conditional asymptotic normality (3.4) under the conditions of Theorem 3.1. Lemma 3.4 applies to Theorem 3.3 in the same way. The choice b,~ := n ~12c,~ and
/
c.~ : = ( X ~ : . ( o )
- X.(~o
(X.~(~o) - J~.(~o) 2
-~/2
i for fixed ~o e Q proves asymptotic normality of (3.16) P-a.s. along suitable subsequences.
[]
It is easy to see that there exist symmetric two-point variables such that the central theorem holds for S,, see Lemma 3.2(b), but (3.14) is violated. However, condition (3.14) holds for our two-sample problem also when (3.9) is violated. Since T, is invariant under the same shift for all X,~ we can always assume the condition
~E(Xni ) i=l
=
0
for all n e N.
(3.17)
4. The proofs of Section 2 Throughout, let X.~ .... ,X.. denote the pooled sample of Example 1.1 given by X.i = X~ for i ~< nl and X.~ = Y~-.1 for i > nl. At first, we calculate the asymptotic permutation distribution under alternatives. Without restrictions we may assume that the means satisfy n~la.1 + nzt,tn2 = 0, cf. (3.17).
Lemma 4.1. Assume that min(nl,n2) -~ ~ and lim.-,oo nffn = x holds for some x ~ [0, 1]. Suppose that as in Theorem 2.1 the means #.j of the jth sample are convergent to #jfor j = 1, 2. Then the permutation distribution of the test statistic T. given by (1.2) is asymptotically normal in the sense of(3.4) with ~ = 1. Proof. Define regression coefficients for 1 ~< i ~< n by
= ( n a n z y / 2 ~ l/n1 c.,
\--g-]
( - 1/n2
for
i<~nx,
(4.1)
i > nl.
Then the statistic ~, has just the form of (3.2) and the conditions (3.6) and (3.13) hold. It suffices to check the assumptions of Theorem 3.3. We will see that the validity of (3.14) is an immediate consequence of the strong
A. Janssen / Statistics & ProbabilityLetters 36 (1997) 9-21
17
law of large numbers. Observe that X. ~ 0 a.s. since x#l + (1 - x)#2 = 0. To prove (3.14) it remains to show that 1
"
lira s u p - ~ X2.i ltd,oo)(IX.i[) ~ 0 ,~ n i=1
P-a.s.
(4.2)
as d ~ ~ . This statement follows from the strong law of large numbers since the means E(X2,1) are bounded. Next, the conditions (3.7) and (3.8) are checked. Routine calculations show that
n 1 ~(X,, ~/i=
X,) z + x ( a ~ + #~) + (1 - K)(a 2, + #~)
P-a.s.
(4.3)
To prove (3.8) let us divide V. = W,1 - W,22 - W.~ into three parts given by
Wnl = (nl
? , / 2 nl nl ~ 2 1)n Z X2, + - i=1 (n2 1)n i=.~+1 X.,,
// nly/2 'xl/2 1 ~
w.2 = t,i.;-i).)
,z51x""
w.,
=(
?/1712 ~1/21
+
x"'
For each part the asymptotic permutation distribution of W.j((X.,,~))i ~.)) can be computed separately. Notice that
E(W.E((X.~))~<.IX.1, ... ,X..)) = (n~ - 1)n
X. ~ 0 .
(4.4)
Its conditional variance is according to the variance formula of Hijek and Sid/~k (1967) equal to
h--n
e-.a.s.
i=1
Thus, W~2((X.~o)~ <.) converges to zero in P ® P-probability. Similarly, it is proved that
W.3((X.,,~i)), <~,) ~
0
(4.5)
converges in probability. Finally W.1 is treated. It can be shown that W,I((X.~i))~<,) ~
p ® p
x(a~ + #~) + (1 - x)(a z + #z) =: p -
_
(4.6)
is convergent in probability. For its proof introduce the regression coefficients
i~nl, i>nl.
d.i = )" nz((nl - 1) n)- 1 for [nl((n2 - 1)n) -1 Obviously, we have
W.1 = ~ d,i X2.i i=1
(4.7)
and
nd,:= ~, d . i ~ l, i=1
• i=1
(dni
-
a~n) 2 ~ 0.
(4.8)
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
18
The statistic (4.6) will be dealt with by a truncation argument. Notice, first that its conditional expectation is convergent E
d . g X .2o . ) I X . 1 . . . . . X . .
=
i
2~ P Z X ni
d., i=1
(4.9)
i=1
P-a.s. If Zi has finite fourth moment, we see that 2 lXnl , ... ,Xnn dniXna(i)
Var i=
)
=
(dni--dn)2-~ i=l
X.2i - -
l, --
X.j
"=
P-a.s. if (4.8) is taken into account. Thus, ~ d n i Xna(i) 2 ~
(4.10)
P"
i=1
If E ( Z ~ ) = ~ let us decompose X.i = X'.i + )?.i:= ( a l Z i 1E_r,Kj(Z~) + #.1) + alZ~ IE_K,~1o(Zi) for i ~< nl for positive K and similarly for i > nl and al and #.1 replaced by aa and #.2. Obviously, we have again ~ d n i Xna(i) ,2 ~ i=l
(4.11)
PK
in probability, where PK = x l i m . ~ E(X'.]) + (1 - K) llm._.~ E(X..),'2 and E
\i = 1
-2 d.iX.,,ti)
(4.12)
~ P - PK
which becomes arbitrary small as K ~ oo. Since all random variables are non-negative we may apply " 1 d.i X.~.) becomes uniformly small in P ® P-probability whenever K is large. Markov's equality and Zi= ~2 Since X2i = X'.2 + )~2 holds, the desired statement (4.6) follows from (4.11) and (4.12). [] Proof of Theorem 2.1. Under/~1 = #z the statistic 2r. is asymptotically standard normal. By Lemma 4.1 the permutation critical value c.,p(e) converges in probability to u l - . . This assertion can be proved (by the subsequence principle for convergence in probability) first along subsequences, where nl/n is convergent. Notice that all other critical values of q~.,~tud"" are convergent to the same constant. [] Proof of Theorem 2.2. Obviously, V. is distribution free under shift alternatives and V . ( ( X . i ) i < . ) - ~ ( 1 - K)a 2 + x a ~ .
P-a.s. if nx/n ~ to. In the next step write r.((x.,),~.)
=
Z z,i=1
=:
--
z,
+
(#.,~ - #.,~)
/'12 i = n a + l
Tnl + Tn2.
Notice that T.I/V~./2 is always asymptotically standard normal, Since c.,p(~)~ul-~ is convergent in probability, we have consistency if Tn2 ~ zt3, since T.2/V~./2 ---, ~ . Under T.,2 ---,K we obtain the asymptotic power function of Theorem 2.2(b). []
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
19
5. Monte Carlo results In this section, we compare the actual size of type I errors of the studentized permutation test q),,Perm of Section 2 with the Welch test ~0,,Wel¢hfor normal and skewed distributions. Various Monte Carlo studies can be found in the literature. We refer to Best and Rayner (1987) for a power study of different tests (including the Welch test) for normal distributions. They also discussed the size of type I errors and summarized the results of earlier Monte Carlo computations, see also Wang (1971). In the normal case the Welch test can be recommended. Mehta and Srinivasan (1970) compared the Welch test with other classical tests for the Behrens-Fisher problem. Fung (1979) considered the accuracy of studentized Wilcoxon tests for the generalized Behrens-Fisher problem (Example 1.1) with nonnormal errors. The paper contains a comparison with the Welch test also under alternatives. Below, Tables 1-3 show some type I error probabilities where the nominal level was e = 0.0498, 0.0485 and 0.05, respectively, for computational reasons. The computations were carried out for normalized error variables Z ( = Zi) with E(Z) = 0 and Var(Z) = 1, see Example 1.1. In particular, we choose Z to be normal, log-normal Z = exp (~), where ~ is normal with mean ( - log 2)/2 and variance log(2), a shifted standard exponential random errors Z, and a uniformly distributed Z on the interval [ - ~ , ,,/~]. The computations are based o n 106 Monte Carlo replications. Under the conditions of Table 1 the permutation test q~,,P,rm is always better than cP,,w~I~h- Especially, we see that the Welch test may be bad for skewed distributions. For different sample sizes n~ = 4 and/72 ---- 8 (Table 2) the results are more complicated. Of course, if the errors are known to be normal then the Welch
Table 1 Type I error probability, nl = 6, n2 = 6, ~1 =/~2, ~ = 0.0498 Distribution
a~ : azz
1.0 : 1.0
1.0: 1.1
1.0 : 1.2
Normal Log-normal
qS,. wo,ch qS,. Por,~ qS,, W~,~h
Exponential
~bn, Perm 4~., Welch
0.0460 0.0501 0.0251 0.0496 0.0291 0.0494 0.0489 0,0500
0.0458 0.0500 0.0254 0.0500 0.0297 0.0501 0.0487 0.0500
0.0459 0.0500 0.0261 0.0507 0.0302 0.0504 0.0490 0.0502
~b,, P ~ m
Uniform
~b., wel~h q~,. Porto
Table 2 Type 1 error probability, nl = 4, n 2 = 8, #1 = #a, c~ = 0.0485 Distribution
a12'.a22
1.2 : 1.0
1.1 : 1.0
1.0 : 1.0
1.0: 1.1
1.0 : 1.2
Normal
q~., welch
0.0509 0.0523 0.0453 0.0555 0.0518 0.0556 0.0663 0.0538
0.0495 0.0497 0.0416 0.0516 0.0478 0.0518 0.0646 0.0508
0.0497 0.0481 0.0379 0.0486 0.0442 0.0481 0.0627 0.0482
0.0488 0.0456 0.0346 0.0455 0.0402 0.0447 0.0606 0.0452
0.0478 0.0436 0.0316 0.0429 0.0377 0.0426 0.0589 0.0430
~b., P ~ m
Log-normal Exponential Uniform
~b..wolch ~bn. Perm qS.,W~ch q~n, Perm q~.,Wclch q}n,Perm
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
20
Table 3 Type I error probability, n~ = 8, n 2 = 8, #1 = g2, ~ = 0.0500 Distribution
a~:a~
1.0:1.0
1.0:1.1
1.0:1.2
1.0:1.5
1.0:2.0
Normal
~bn,Welch q~,.Perm ~b~.welch ~bn.pcrm ~bn.welch ~bn,Pe~m ~b~.Welch ~b~.P~m
0.0478 0.0506 0.0291 0.0504 0.0342 0.0507 0.0502 0.0506
0.0476 0.0503 0.0292 0.0504 0.0344 0.0504 0.0497 0.0502
0.0480 0.0508 0.0299 0.0515 0.0350 0.0513 0.0501 0.0505
0.0479 0.0510 0.0339 0.0555 0.0382 0.0541 0.0506 0.0514
0.0486 0.0525 0.0426 0.0634 0.0449 0.0599 0.0516 0.0528
Log-normal Exponential Uniform
test should be applied (first row of Table 2). In the other cases, the permutation test works better (apart from two exceptional cases). It seems to us that an unequal sample size produces a further bias of the error I probability. Even under al = a2 = 1 (third column of Table 2) the accuracy of the Welch test is not acceptable. Table 3 contains further simulations for larger differences of the variances. Again the new permutation is acceptable except of two cases for skewed distributions under ¢r~/a~ = 2 (last column of Table 3).
Conclusions The present Monte Carlo results support the studentized permutation tests for models with non-parametric errors and slightly different variances.
Acknowledgements I thank H. Hebben for his computational assistance.
References Andersen, P.K., Borgan, O., Gill, R., Keiding, N., 1993. Statistical models based on counting processes. Springer Series in Statistis. Springer, New York. Behrens, W.V., 1929. Ein Beitrag zur Fehlerberechnung bei wenigen Beobachtungen, Landwirtsch. Jahrbiicher 68, 807-837. Beran, R., 1988. Prepivoting test statistics: a bootstrap view of asymptotic refinements. J. Amer. Statist. Assoc. 83, 687-697. Best, D.J., Rayner, J.C.W., 1987. Welch's approximate solution for the Behrens-Fisher problem. Technometrics 29, 205-210. Edgington, E.S., 1995. Randomization Tests, 3rd ed. Marcel Dekker, New York. Fisher, R.A., 1936. The fiducial argument in statistical inference. Ann. Eugenics 6, 391-422. Fligner, M., Policello, G., 1981. Robust rank procedures for the Behrens-Fisher problem. J. Amer. Statist. Assoc. 76, 162-168, Fung, K.Y., 1979. A Monte Carlo study of the studentized Wilcoxon statistic for the Behrens-Fisher problem. J. Statist Comput. Simul. 10, 15-24. Good, P., 1994. Permutation tests, a practical guide to resampling methods for testing hypothesis. Springer Series in Statistics. Springer, Berlin. Gnedenko, B.V., Kolmogorov, A.N., 1968. Limit Distributions of Sums of Independent Random Variables. Addison-Wesley, Reading, MA. H~tjek, J.A., ~idfik, 1967. Theory of Rank Tests. Academic Press, New York.
A. Janssen / Statistics & Probability Letters 36 (1997) 9-21
21
Hall, P., Martin, M., 1988. On the bootstrap and two-sample problems. Austral. J. Statist. A30, 179-192. Janssen, A. 1989. Local asymptotic normality for randomly censored data with applications to rank tests. Statist. Neerlandica 43, 109-125. Janssen, A. 1991. Conditional rank tests for randomly censored data. Ann. Statist. 19, 1434~1456. Linnik, J.N., 1968. Statistical problems with nuisance parameters. Translation of Mathematical Monograph, vol. 20. American Mathematical Society, Providence, RI. Mason, D.M., Newton, M.A., 1992. A rank statistics approach to the consistency of a general bootstrap. Ann. Statist. 20, 1611-1624. Mehta, J.S., Srinivasan, R., 1970. On the Behrens-Fisher probem. Biometrika 57, 649-655. Neuhaus, G., 1988. Asymptotically optimal rank tests for the two-sample problem with randomly censored data. Comm. Statist. Theory Methods 17, 2037-2058. Neuhaus, G., 1993. Conditional rank test for the two-sample problem under random censorship. Ann. Statist. 21, 1760-1779. Pfanzagl, J., 1974. On the Behrens-Fisher problem. Biometrika 61, 39-47. Praestgaard, J.T., 1995. Permutation and bootstrap Kolmogorov-Smirnov tests for the equality of two distributions. Scand. J. Statist. 22, 305-322. Romano, J.P., 1989. Bootstrap and randomization tests of some nonparametric hypothesis. Ann. Statist. 17, 141-159. Romano, J.P., 1990. On the behaviour of randomization tests without a group invariance assumption. J. Amer. Statist. Assoc. 85, 686-692. Sen, P.K., 1962. On studentized non-parametric multi-sample location test. Ann. Math. Statist. 14, 119-131. Thomasse, A.H., 1974. Practical recipes to solve the Behrens-Fisher problem. Statistica Neerlandica 28, 127-138. Wang, Y.Y., 1971. Probabilities of the type I errors of the Welch tests for the Behrens-Fisher problem. J. Amer. Statist. Assoc. 66, 605-608. Welch, B.L., 1937. The significance of the difference between two means when the population variances are unequal. Biometrika 29, 350-362. Welch, B.L., 1947. The generalization of Student's problem when several different population variances are involved. Biometrika 34, 28-35. Witting, H., Miiller-Funk, U., 1995. Mathematische Statistik II. Teubner, Stuttgart. Witting, H., N~511e,G., 1970. Angewandte Mathematische Statistik. Teubner, Stuttgart.