A bi-aspect nonparametric test for the two-sample location problem

Computational Statistics & Data Analysis 44 (2004) 639 – 648 www.elsevier.com/locate/csda

A bi-aspect nonparametric test for the two-sample location problem Marco Marozzi Department of Statistics, University of Bologna, via delle Belle Arti n. 41, 40126 Bologna, Italy Received January 2002; received in revised form September 2002

Abstract Permutation methods are prized for their lack of assumptions concerning distributions of variables. A bi-aspect permutation test based on the Nonparametric Combination of Dependent Tests theory is developed for testing hypotheses of location shifts of two independent populations. The test is obtained by combining the traditional permutation test with a test that takes into account whether a sample observation is less than or equal to, or greater than the pooled sample median. The procedure to compute the proposed test is presented. The type-one error rate and power of the test are investigated for many distributions and sample-size settings via Monte Carlo simulations. These simulations show that the proposed test is remarkably more powerful than the traditional permutation test under heavy-tailed distributions like the Cauchy, the half-Cauchy, a 10% and a 30% outlier distribution. When sampling from the double exponential and the exponential distributions, the proposed test appears to be better on the whole than the traditional permutation test. Under normal, uniform, a chi-squared and a bimodal distribution, the bi-aspect test is practically as powerful as the traditional permutation test. Moreover, in these simulations the proposed test maintained its type-one error rate close to the nominal signi8cance level. c 2002 Elsevier B.V. All rights reserved.
Keywords: Nonparametric methods; Permutation tests; Nonparametric Combination of Dependent Tests; Type-one error rate; Power

1. Introduction The most commonly used method for comparing locations of two populations is the Student t-test. When the populations are distributed normally with a common variance,

E-mail address: [email protected] (M. Marozzi). c 2002 Elsevier B.V. All rights reserved. 0167-9473/$ - see front matter
PII: S 0 1 6 7 - 9 4 7 3 ( 0 2 ) 0 0 2 7 9 - 7

640

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

this test is the uniformly most powerful similar test for testing the null hypothesis of equal locations against the one-sided alternative hypothesis (Lehmann, 1986). In this context, the Student test is also exact, unbiased and consistent. When the populations are not distributed normally with a common variance, these properties are no longer satis8ed. Ludbrook and Dudley (1998) underline that the Student test is very often used even when it is not valid. A test that is valid for every distribution is the traditional permutation test (PT) for comparing two locations (Good, 2000). The permutation testing idea goes back to the origin of inferential statistics: the earliest contributions are of Fisher (1934, 1935) and Pitman (1937a, 1937b, 1938). More recently, Pesarin (2001) develops an interesting theory, the Nonparametric Combination of Dependent Tests (NPC). This theory yields good results for many complex multivariate and multiaspect problems, including problems that have not yet been solved within a parametric setting. Both the Student t-test and the traditional permutation test may be seen as uni-aspect tests because they consider only the numerical aspect Xi of each sample unit. In this paper, NPC theory is used to develop a bi-aspect nonparametric test for the two-sample location problem. We devised the test by combining the traditional permutation test with a simple test that takes into account if a sample observation is large or small. Through simulation experiments, we found that the consideration of this aspect is very important when samples comes from heavy-tailed distributions like the Cauchy, the half-Cauchy, and some mixtures of normal distributions with unequal variances. 2. The traditional permutation test for the two-sample location problem Let 1 X = (1 X1 ; : : : ; 1 Xn1 ) and 2 X = (2 X1 ; : : : ; 2 Xn2 ) be independent random samples from populations with continuous distribution functions that may diFer only in their locations. Let n1 + n2 = n. The objective is to test the hypothesis H0 : 1  = 2  against the location shift alternative H1 : 1  ¿ 2 , where h  denotes the location parameter of population h (h = 1; 2). Let # = 1  − 2 ; we can specify the system of hypothesis as H0 : # = 0 against H1 : # ¿ 0: The traditional permutation solution for testing H0 : # = 0 against H1 : # ¿ 0 is based on the 8rst sample total statistic whose observed value is n1
0 PT = 1 Xi : i=1

To compute the p-value LPT of the test, one should compute PT for each permutation of the pooled sample X = (1 X ; 2 X ). Since it is generally impractical to calculate all permutations, LPT is strong-consistently estimated by taking a random sample of size K of all permutations and computing the proportion of permutations that have test statistic greater than or equal to the observed statistic 0 PT . LPT is then estimated as recommended by Pesarin (2001) K 1
LˆPT (0 PT ) = I (K PT ¿ 0 PT ); K k=1

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

641

where K PT denotes the value of PT in the kth (k = 1; : : : ; K) permutation of X and I (:) denotes the indicator function: I (k PT ¿ 0 PT ) = 1 when k PT ¿ 0 PT and I (k PT ¿ 0 PT ) = 0 otherwise. Let 0 ¡  ¡ 1 be the nominal signi8cance level: if LˆPT 6  then H0 is rejected, otherwise is accepted.

3. The proposed bi-aspect test To test H0 : # = 0 against H1 : # ¿ 0, PT considers only the quantitative aspect 1 Xi of each 8rst sample element, that can be called the numerical aspect, and so does the Student t-test (the t statistic is permutationally equivalent to PT statistic). To construct our test, we considered another quantitative aspect of the sample units. This second aspect is related to the classi8cation of the sample units into two groups, one of the large units and the other of the small units. A way to formalize this classi8cation is to de8ne “large” as an observation that is greater than the median M˜ of X and “small” as an observation that is less than or equal to M˜ . M˜ is de8ned as M˜ = X((n+1)=2) if n is odd and as M˜ = 12 (X(n=2) + X((n=2)+1) ) if n is even, where X(i) is the ith order statistic in the pooled sample. We can call this second aspect the categorical aspect. Let Table 1 represent the outcome of this classi8cation. Under H0 , we expect n1g =n1 and n2g =n2 to be not very diFerent, while under H1 we expect that the larger # is, the larger the diFerence between n1g =n1 and n2g =n2 . Note that ng does not change after permuting X . A natural and convenient test statistic to study the second aspect is Tb = n1g =

n1


I (1 Xi ¿ M˜ );

i=1

which is a version of the Fisher’s exact probability test. Tb may be also seen as a permutationversion of the median test. To take into account the 8rst aspect, we use n1 Ta = PT = i=1 1 Xi . Both Ta and Tb are signi8cant for large values of test statistics and provide information on the testing problem: by means of NPC theory we can combine the information provided by the two tests in one combined test. The testing procedure for testing H0 vs. H1 is carried out in two successive steps. In the 8rst step, we de8ne the two partial systems of hypotheses as H0a : # = 0 vs. H1a : # ¿ 0 and H0b : P{1 X ¿ M˜ } = P{2 X ¿ M˜ } vs. H1b : P{1 X ¿ M˜ } ¿ P{2 X ¿ M˜ }. We would like to note that this decomposition Table 1 Arrangement of the sample units into the two considered groups

Small unit group Large unit group

First sample

Second sample

n1s n1g

n2s n2g

ns ng

n1

n2

n

642

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

of H0 is merely formal: H0 can be stated equivalently as H0 : F1 = F2 , when Fh is the distribution of population h (h = 1; 2), and both H0a and H0b are implied by H0 . Then, we permutationally estimate the p-values of the partial tests Ta and Tb as K K 1 1 + k=1 I (k Ta ¿ 0 Ta ) + k=1 I (k Tb ¿ 0 Tb ) LˆTa (0 Ta ) = 2 and LˆTb (0 Tb ) = 2 : K +1 K +1 K With respect to the standard LPT estimator LˆPT (0 PT ) = K1 k=1 I (K PT ¿ 0 PT ); 12 and 1 have been added, respectively, to the numerator and denominator of the fraction. This is done to obtain estimated p-values in the open interval ]0; 1[ and avoid computational problems which may arise in the second step of the procedure when adopting certain combining functions. However, since we use a large K, this alteration is practically irrelevant and LˆTa (0 Ta ) and LˆTb (0 Tb ) are unbiased and consistent estimates of LTa and LTb (see Pesarin, 2001, p. 144). In the second step, we construct the (global) test statistic for testing H0 : # = 0 vs. H1 : # ¿ 0 by combining the p-values associated with the partial tests Ta and Tb . Since the partial p-values are permutationally equivalent to the partial tests, we may combine the partial p-values rather than the partial tests without loss of generality (Pesarin, 2001). To combine LTa and LTb we use the Tippett combining function. Our bi-aspect test is then de8ned as Tab = max(1 − LTa ; 1 − LTb ): We used the Tippett combining function because under H1 it selects the best partial test p-value (namely the smallest p-value). This is important because we expect that our bi-aspect test will be particularly useful when sampling from some heavy-tailed distributions for which the usual test based on the sample total is not very sensitive to detect location shifts. It is known (Johnson and Kotz, 1970) that under Cauchy or half-Cauchy distributions, the sample mean (that is permutationally equivalent to the sample total) is not a proper statistic for estimating the location or for testing location shifts. Moreover, it is well known that the uniformly most powerful parametric test for testing H0 : # = 0 vs. H1 : # ¿ 0 does not exist for the Cauchy and half-Cauchy distributions (see e.g. Lehmann, 1986). In these (and similar) cases we expect that the test on the second aspect, Tb , will help Ta (that is to say PT) for detecting location shifts. Our aim is to propose a test that performs similarly to the usual permutation test PT when the sample total is a proper statistic to study the location and that may perform better than PT otherwise. We would like to propose a test with good overall performance that may be used without worrying about the distribution underlying the data that, of course, is usually unknown. The observed value of Tab is estimated as 0 Tˆ ab = max(1 − LˆTa (0 Ta ); 1 − LˆTb (0 Tb )). Its distribution is simulated through the same results and the same K permutations of the 8rst step. More precisely, the value of Tab in the 8rst permutation of X is computed as 1 Tab

= max(1 − LˆTa (1 Ta ); 1 − LˆTb (1 Tb )):

Then, by repeating the computation of the global test statistic for the other K −1 permutations of X , we obtain the vector simulating the distribution of Tab (1 Tab ; 2 Tab ; : : : ; K Tab ).

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

643

Large values of the observed test statistic are evidence against the null hypothesis. The p-value of Tab is strong-consistently estimated as K 1
I (k Tab ¿ 0 Tˆ ab ): LˆTab = K k=1

We reject H0 at signi8cance level 0 ¡  ¡ 1 if LˆTab 6 . We would like to note that Pesarin (2001, pp. 165–168) proved that a combined test is exact, unbiased and consistent when the partial tests are exact, unbiased and consistent. Since Ta and Tb are exact, unbiased and consistent tests, this result applies to Tab . The procedure for computing the p-value of Tab is summarized in the steps of the following algorithm: (i) pool 1 X and 2 X into X and compute M˜ , (ii) compute the observed values of the partial test statistics: 0 Ta as the sum of elements of the 8rst sample and 0 Tb as the number of elements of the 8rst sample that are greater than M˜ , (iii) obtain the 8rst permutation X ∗ of X , (iv) compute the 8rst permutation values of Ta and Tb : 1 Ta as the sum of the 8rst n1 elements of X ∗ and 1 Tb as the number of the 8rst n1 elements of X ∗ that are greater than M˜ , (v) carry out K − 1 independent repetitions of steps (iii) and (iv), (vi) compute the number of permutations that have test statistics Ta greater than or equal to 0 Ta , add 12 and divide by K + 1 to estimate the p-value LˆTa (0 Ta ) of Ta . Proceed in an analogous way to obtain LˆTb (0 Tb ), (vii) estimate the observed value of Tab as 0 Tˆ ab = max(1 − LˆTa (0 Ta ); 1 − LˆTb (0 Tb )), (viii) compute the number of permutations that have test statistic Ta greater than or equal to 1 Ta , add 12 and divide by K + 1 to obtain the 8rst permutation p-value LˆTa (1 Ta ) of Ta . Proceed in an analogous way to obtain LˆTb (1 Tb ), (ix) estimate the 8rst permutation value of Tab as 1 Tˆ ab = max(1 − LˆTa (1 Ta ); 1 − LˆTb (1 Tb )), (x) repeat steps (viii) and (ix) for the other K − 1 permutations (i.e. use k Ta and k Tb with k = 2; : : : ; K instead of 1 Ta and 1 Tb ), (xi) compute the proportion of permutations that have test statistic Tab greater than or equal to 0 Tab to obtain the p-value of the bi-aspect test. 4. Evaluation of the proposed bi-aspect test To get an insight into the performance of our bi-aspect test, Monte Carlo simulations were used to estimate the type-one error rate and the power of Tab , PT and the Student’s test for many con8gurations of sample-sizes and distribution functions. It is worth noting that we tackle the problem of studying the power behavior of Tab by simulation experiments because until now, according to Pesarin (2001), theoretical methods useful for comparing the power of tests based on NPC theory have not yet been proposed. We

644

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

considered the Student’s test because it is the most common method to test the system of hypotheses under study. Of course, the true counterpart of Tab is PT, which is valid even when the data are not normal. The comparison between Tab and the Student’s test is particularly interesting under normality, namely, when the most used method to test H0 : # = 0 vs. H1 : # ¿ 0 is also the best method. 4.1. Simulation conditions We considered ten distributions (let N (; 2 ) denote the normal distribution with mean  and variance 2 ): (i) a standard normal distribution N (0; 1), √ (ii) a uniform distribution between 0 and 2 3 that is a symmetric and moderate platykurtic distribution (its kurtosis is exactly 0.6 times that of the normal distribution), √ (iii) a double exponential distribution with scale parameter of 1= 2 that is a symmetric and quite leptokurtic distribution with kurtosis two times that of the normal one, (iv) a standard Cauchy distribution that has the density f(x) = (1= )1=(1 + x2 ); x ∈ R, (v) a chi-squared distribution with eight degrees of freedom and scale parameter of 0.25: this curve has a positive skewness level of one, (vi) an exponential distribution with scale parameter of one: this curve has a positive skewness level of two, (vii) a standard half-Cauchy distribution that has f(x) = (2= )1=(1 + x2 ); x ∈ R+ , (viii) a bimodal distribution with light-tails obtained as a mixture of a N (−1:5; 1) with probability 0.5 and a N (1:5; 1) with probability 0.5, (ix) a 10% outlier distribution obtained as a mixture of a N (0; 1) with probability 0.9 and a N (1; 100) with probability 0.1, (x) a 30% outlier distribution obtained as a mixture of a N (0; 1) with probability 0.7 and a N (1; 100) with probability 0.3. The programs for performing Tab and all simulations were coded in R language using the R 1.2.3. program. Random samples varying in sizes were generated from two independent normal, uniform, Cauchy, chi-squared and exponential distributions by means of the rnorm, runif, rcauchy, rchisq and rexp function respectively. The double exponential samples were generated as the exponential ones and then by randomly assigning a positive or negative sign to each term of the samples. The half-Cauchy samples were generated as the Cauchy ones and then by changing the sign for each negative term of the samples. The mixtures of two normal distributions were generated by selecting each observation from one of the two distributions according to the outcome of a proper Bernoulli trial. The null hypothesis tested was that the two populations had equal location parameters against the directional alternative hypothesis that the 8rst one had a larger location parameter. Three equal sample-size con8gurations that were considered: (n1 ; n2 ) = (10; 10); (20; 20) and (40; 40). We studied four couples of unequal sizes as well:

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

645

Table 2 Positive values of # used for power estimation

Distribution

Sample-size setting n1 n2

Normal Uniform Double exponential Cauchy Chi-squared Exponential Half-Cauchy Bimodal 10% outlier 30% outlier

10 10

20 20

40 40

10 20

10 40

20 10

40 10

0.77 0.77 0.73 3.1 0.74 0.7 2.5 1.4 1.45 3.7

0.53 0.53 0.52 3 0.52 0.5 2.5 0.96 1.4 2.75

0.37 0.37 0.37 3 0.37 0.36 2.5 0.67 1.1 2

0.65 0.65 0.63 3 0.65 0.62 3.3 1.19 1.6 3.3

0.59 0.59 0.58 3.7 0.61 0.59 4.7 1.07 1.65 3.1

0.65 0.65 0.63 3 0.63 0.6 2 1.19 1.6 3.3

0.59 0.59 0.58 3.7 0.56 0.54 1.7 1.07 1.65 3.1

(10; 20); (10; 40); (20; 10) and (40; 10). To estimate the size of the tests we considered # = 0, while to estimate the power we considered a positive value of the location shift # that was speci8ed so that the power of the traditional permutation test for each distribution and sample-size setting was near 50%. To this end, preliminary trials were done to determine appropriate # values for the considered settings (see Table 2). For each con8guration, we used 4000 Monte Carlo simulations and 1000 permutations. To obtain the desired location shift, the two samples were drawn independently from the same distribution and then # was added to the 8rst sample elements. 4.2. Simulation results As reported in Table 3, the type-one error rate estimates show that both Tab and PT maintain their size rather close to the nominal 5% level for every distribution. Size estimates of the t-test are not tabulated, but we noted that when sampling from the Cauchy and the half-Cauchy distributions, its size is very far from 5% (of course, this is not a surprising result). The power estimates of t, PT, Ta and Tab tests are shown in Table 4, while Table 5 reports the estimates of the relative power of Tab with respect to PT. We note that under normal distributions Tab is practically as powerful as both PT and as the t-test. Since the t-test is optimal in this case (and PT is a permutation version of the t-test), this result is quite interesting. Tab is practically as powerful as PT under the considered chi-squared distributions as well. When sampling from the uniform and bimodal distributions, Tab behaves quite similarly to PT. We note that under the double exponential and exponential distributions, Tab appears to be better than PT on the whole. The most interesting result of our simulation study is when sampling from the considered four heavy-tailed distributions. As shown in Tables 4 and 5, under the Cauchy, half-Cauchy, 10% outlier and 30% outlier distributions, Tab is remarkably more pow-

646

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

Table 3 Percent size estimates for  = 5%

n1 10 n2 10 Test

20 20

40 40

Normal PT 5.38 4.50 4.60 Tab 5.23 4.35 4.70 Uniform PT 4.70 4.93 4.70 Tab 4.78 4.90 4.60 Double exponential PT 5.15 5.05 4.80 Tab 5.08 4.95 4.73 Cauchy PT 5.08 4.68 4.43 Tab 5.25 4.55 4.65 Chi-squared PT 5.08 4.90 4.93 Tab 4.98 4.83 4.68

10 20

10 40

20 10

40 10

4.65 4.75

4.98 4.75

4.48 4.40

4.98 4.38

5.28 5.23

4.65 4.18

5.25 5.40

5.05 4.83

4.60 4.73

4.88 4.15

5.08 4.78

5.20 4.35

5.40 4.85

5.03 3.95

4.58 4.15

5.05 4.50

4.88 4.85

5.03 4.25

4.95 4.65

5.55 4.80

n1 10 n2 10 Test

20 20

Exponential PT 5.48 5.30 Tab 5.60 5.23 Half-Cauchy PT 4.48 4.88 Tab 4.60 5.08 Bimodal PT 5.18 4.68 Tab 5.08 4.50 10% outlier PT 4.88 5.43 Tab 5.13 5.60 30% outlier PT 4.80 5.10 Tab 4.80 5.10

40 40

10 20

10 40

20 10

40 10

4.88 4.70

4.58 4.38

4.98 4.23

4.85 4.55

4.90 4.13

5.30 5.35

4.25 4.05

4.75 4.35

4.55 4.70

5.30 4.45

4.98 4.73

4.85 4.60

5.50 4.65

4.88 4.85

4.95 4.43

4.78 4.45

4.93 4.78

5.05 4.35

4.50 4.35

4.75 3.68

4.90 4.48

5.23 5.25

5.05 3.88

5.70 5.55

5.13 4.20

erful than PT. At 8rst this result may appear to be rather surprising, but the reason is rather simple. It is well known that in such cases the (8rst) sample total statistic is not a proper statistic for studying the location and in fact the test based on it, PT, is not very sensitive to detect location shifts. Therefore, # has to be large (and the two populations very diFerent) so that PT may reach a power close to 50%. On the contrary, in these cases Tb is very sensitive to detect location shifts. So, why do not use Tb , rather than Tab , for our testing problem? The fact is that Tb performs very poorly under distributions like the normal, uniform, chi-squared, exponential and bimodal with light-tails distributions. In fact in these cases the 8rst sample total is either the best statistic for studying location and in any case, it is a reasonable statistic. Tab combines the sensitivity to detect location shifts of Ta (that is to say PT) in these cases with the sensitivity of Tb under the four considered heavy-tailed distributions. As we noted earlier, the combining function of Tippett is very useful in reaching this result. It is quite interesting to emphasize that Tab seems to be substantially better than either of its components under double exponential and exponential distributions. 5. Conclusion Our Monte Carlo study showed that the proposed bi-aspect test for the two-sample location problem is particularly useful under heavy-tailed distributions for which the 8rst sample total is not very sensitive in detecting location shifts. Moreover, under those distributions for which the 8rst sample total oFers good performance, our test behaves very similarly to this one. The researcher should take into consideration the use of

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

647

Table 4 Percent power estimates for  = 5%

n1 10 n2 10 Test

20 20

40 40

Normal t 49.7 50.5 50.2 PT 50.0 50.2 49.9 Tb 17.2 21.3 25.7 Tab 49.2 49.5 48.8 Uniform t 48.1 48.7 49.1 PT 48.0 48.9 49.0 Tb 10.3 13.5 16.5 Tab 47.3 47.9 46.7 Double exponential t 50.6 50.6 49.5 PT 50.8 50.1 49.4 Tb 30.3 41.7 52.6 Tab 53.8 55.9 59.9 Cauchy t 43.6 42.5 43.8 PT 50.9 49.8 52.5 Tb 72.9 98.1 100 Tab 78.3 97.8 100 Chi-squared t 48.7 48.8 50.0 PT 48.8 48.8 49.7 Tb 18.5 22.7 28.1 Tab 48.9 48.6 49.0

10 20

10 40

20 10

40 10

51.6 51.7 27.5 49.8

49.6 49.3 31.5 45.8

49.8 49.8 26.2 47.0

50.0 49.8 31.5 45.8

48.3 48.2 16.9 44.3

49.1 48.9 19.9 44.7

48.5 48.6 17.1 45.2

49.5 49.2 20.0 44.5

49.9 49.6 44.4 54.9

50.5 50.3 49.7 49.3

50.7 50.5 44.1 55.5

52.2 52.1 49.9 50.9

40.9 47.8 87.6 85.0

41.0 49.0 94.7 82.5

41.3 48.4 88.5 86.5

40.2 48.3 94.6 82.7

49.6 47.8 31.0 48.2

50.7 48.5 39.2 46.0

49.9 51.1 25.1 49.3

47.7 50.1 28.1 46.3

n1 10 n2 10 Test

20 20

Exponential t 51.3 49.7 PT 51.6 49.5 Tb 26.2 31.0 Tab 53.6 52.1 Half-Cauchy t 43.4 40.5 PT 52.4 49.2 Tb 66.3 91.7 Tab 73.2 91.7 Bimodal t 50.4 49.8 PT 50.2 49.8 Tb 10.7 12.5 Tab 49.3 48.6 10% outlier t 47.3 49.8 PT 50.9 50.2 Tb 49.3 83.8 Tab 66.0 85.5 30% outlier t 49.7 49.8 PT 50.3 49.6 Tb 79.3 96.9 Tab 81.2 96.6

40 40

10 20

10 40

20 10

40 10

50.9 50.7 36.6 53.1

49.6 47.1 45.3 55.2

48.6 44.2 51.4 52.9

50.1 52.7 29.4 52.4

51.4 56.1 32.2 53.4

38.7 47.1 99.8 99.7

50.2 49.8 99.3 98.8

54.7 50.5 100 100

29.6 48.9 58.5 64.5

18.4 50.2 55.0 54.9

50.2 50.4 14.7 47.6

49.7 49.4 16.3 46.3

49.2 48.9 20.5 44.8

51.0 50.6 17.3 46.8

50.8 51.0 18.8 46.1

49.3 49.2 92.8 92.9

49.2 51.1 80.3 81.9

48.5 48.7 89.1 74.7

48.7 50.5 80.4 80.6

50.7 50.2 90.4 75.8

50.4 50.3 99.6 99.5

50.0 50.2 89.0 85.7

48.6 48.9 88.8 72.0

49.2 49.1 89.3 85.9

50.4 49.9 89.2 73.6

40 40

10 20

10 40

20 10

40 10

1.05

1.17

1.20

0.99

0.95

2.12

1.99

1.98

1.32

1.09

0.94

0.94

0.91

0.93

0.90

1.89

1.60

1.53

1.60

1.51

1.98

1.71

1.47

1.75

1.47

Table 5 Estimate of the relative power of Tab with respect to PT

n1 10 n2 10

20 20

40 40

Normal 0.98 0.99 0.98 Uniform 0.98 0.98 0.95 Double exponential 1.06 1.11 1.21 Cauchy 1.54 1.97 1.90 Chi-squared 1.00 1.00 0.98

10 20

10 40

20 10

40 10

0.96

0.93

0.95

0.92

0.92

0.91

0.93

0.90

1.11

0.98

1.10

0.98

1.78

1.68

1.79

1.71

1.01

0.95

0.96

0.92

n1 n2

10 10

20 20

Exponential 1.04 1.05 Half-Cauchy 1.40 1.86 Bimodal 0.98 0.98 10% outlier 1.30 1.70 30% outlier 1.61 1.95

648

M. Marozzi / Computational Statistics & Data Analysis 44 (2004) 639 – 648

our bi-aspect test for tackling the two-sample problem; at worst, this test behaves very similarly to the traditional permutation test, and it might be much better than this one in some other situations. Finally, it should be cautioned that these results may directly apply only to the settings studied here. Acknowledgements The author would like to thank the two referees, who considerably helped to improve the manuscript. References Fisher, R.A., 1934. Statistical Methods for Research Workers. Oliver & Boyd, Edinburgh. Fisher, R.A., 1935. The Design of Experiments. Oliver & Boyd, Edinburgh. Good, P., 2000. Permutation Tests, a Practical Guide to Resampling Methods for Testing Hypotheses, 2nd Edition. Springer, New York. Johnson, N.L., Kotz, S., 1970. Continuous Univariate Distributions-1. Houghton MiUin, Boston. Lehmann, E.L., 1986. Testing Statistical Hypotheses, 2nd Edition. Wiley, New York. Ludbrook, J., Dudley, H., 1998. Why permutation tests are superior to t and F tests in biomedical research. Amer. Statist. 52, 127–132. Pesarin, F., 2001. Multivariate Permutation Tests with Applications in Biostatistics. Wiley, Chichester. Pitman, E.J.G., 1937a. Signi8cance tests which may be applied to samples from any population. J. Roy. Statist. Soc. Ser. B 4, 119–130. Pitman, E.J.G., 1937b. Signi8cance tests which may be applied to samples from any population. II. The correlation coeVcient. J. Roy. Statist. Soc. Ser. B 4, 225–232. Pitman, E.J.G., 1938. Signi8cance tests which may be applied to samples from any population. III. The analysis of variance test. Biometrika 29, 322–335.