- Email: [email protected]
Robustness of MML estimators based on censored samples and robust test statistics
Journal of Statistical Plan&g and Inference 4 (1980) 123-143. @I North-Holland Publisting Company
OF MNIJ ESTIMATURS BASED QN $iiidPLES A R0BUST TEST STAJTS~CS
Dzpr. of Mathemtuica~ Sciences, McMaster Cfniuersity,Hamilton, Canada
Received 4 September 1979; revised manuscript received IS April 1980 Recommerided by J. Srivastava Abstract: We investigate the efficiences of Tiku’s (1967) modified maximum likelihood estimators fiC and oC (based on symmetrically censored normal samples) for estimating the location and scale parameters fi and o of symmetric non-normal distributions. We show that g, and u, are jointly more efficient than 2 and s for long-tailed distributions (kurtosis pf= P~;c~E~4.2, flz*= 4.2 for the Logistic), and always more eficient than the trimmed mean c1-r and the matching sample estimate m-rof cr. We also show that cc, and aC are jointly at least as efficient as some of the more prominent ‘“robust” estimators (Gross, 1976). We show that the statistic tC- kC&/cxC, 119= n - 2r + 2r#I (r is the number of observations censored on each side of the salslple and fi is a constant), is robust and powerful for testing an assumed value of p. We define a stat%% T, (based on pC and a,) for testing that two symmetric distributions ace idcatical and show that T, is robust and generally more powerful than the well-known nonparametric bzatistics (Wilcoxan, normal-score, Koirnogorov-Smirnov), against the important location-shift alternatives. We generalize the statistic TCto test that k symmetric distributions are identical. ‘Ii:e asymptcJtic distributions of c, and TC zre normal, under some very general regularity conditions. For small samples, the upper (low;:r) percentage points oB tCand T, are shown to be closely approximated by Student’s r-distribu,:ons. Besides, the statistics cc, and cr, (and hence t, and TJ are explicit and simple funct’Dns of sample observaiions and are easy to compute.
Key words: Robust Estimation; Robust Tests; !Vonparametric Tests; Modified Maximum Likelihood Estimatols; Censored Samples; Qutliers.
We investigate the joint efficiencies of Tiku’s (1967, 2970) MML (modified ;naximum likelihood) estimators I_C,and O, based on symmetrically censored normal samples for estimating the lacati.on and scare pilrameters p and cp of symmetric nonnormal distributions, focussing on symmetric distributions with existent mean (location parameter) ~4. and existent standard deviation (scale parameter) 0: We show that pL,and cc are jointly more ef?kient than the pair of eStimatoxs #.+ and C+ (trimmed mean and t and c)r, (based on s IWO)). We a&o s and celebrated ‘“robust” estimators defined for complete samples (wave, Hampel I
123
and bisquare estimators (Gross, 1976)). As compared with the sample mearr and standard deviation Z and s, this 10% pair p, and cr, is shown to be considerably more efficient for long-tailed distributions \kurtosis 0: = bk4/& > 4.2), ‘t;*ttslightly less efficient for distributions with 3~& * ~4.2 and considerably less efficient for short-tailed distributions ;PF ( 3). We define the Studentized statistic I!,= ,u~&/o=~ and investigate its robustn,ess arrci power properties for testing an assumed value of CL.The null distribution of tC, unlike that of the Student’s t = I&/s statistic, is shown to be robust and more powerfui than the Studentiaed statistic based on pr and 0,. We define a statistic TC (based on the MML estimatars) for testing that two symmetric distributions are identical and show that the nul? distribrrtion of TC is robust; TC is also shown to be generally more powerful than the well-known nonparametric statistics (Wi coxon, normal-score, Kolmogrov-Smirnov) against the important location-shift alternatives, for longtailed distributions. The statistic ‘IC iv generalized to test that k symmetric distributions are identical, Note that CL,and o, are simple and explicit functions ot’ sample ohervations for both symmetrically and nonsymmetrically censored samples (Tiku, 1967, 1978). The estimators pC and o, based on samples censored only on one side (in the direction of the longer tail) seem to provide efficient and robust estimators of the mean p atid standard deviation u in the more difficult situation of skew distributions (fiC and a;, are correlated in case of skew distributions). This is under investigation at the present time and we hope to report our findings in a future paper. Note that the MML estimators can also be worked out from censored samples in regression (both simple and mtJltiple) and experimental design framework and these estimators are simple and explicit functions of sample observations: see Tiku (1973, 1978) and Tiku and Stewart i[1977). It may be remembered that the problem tackled in this paper is the estimatiorl of both location and scale, and not I\=cation alone (Huber (1972), Andrews et al, (1972), Bickel f 1976), Wegman and Carroll (1977) and Stigler (P977)), of sym;ne::ic distributions.
2. MML estimators for the normal distribution
tet
he a random sample from the normal N(JL, crj distribution. Let X+X,,,..
..,X tt __ r
(2.2)
be the rf’ylpeI1 symmetrically ceujsored sample of size n - 2r, obtained by arrangj in ~~~~~nding order of magnitude and censoring (trimming) the r sma2ie~ I’ 1 scrvations; the value of r wi later. Tiku’s ( 1Ye,i,
Robusmessoof MML
esti,;rlators
125
3978: modified maximum likelihood (MML) estimators of p and o, calculated from (2.21, are given by (for a formal definition see Tiku and Stewart (1977))
and o, = {B + &3*+ 4AC))/2+,(A
- l)},
G.4)
where m = n--2t+2&
A = H- 2r,
B = ra(xn-r -- &+,),
and C = Wxr(X, - p,)‘+ @{(X**’ lr+l
/AC)2 + (X,.-r - /A,)“}
n-r =
1
(2.5)
Xi’+rp(X~+,+Xf,._,)-mr;,2;
r+l
the coefRcients cy and @ are given by P :={gW -
g(MMh2
-
h I),
a =
guw-
h,P,
(2.6)
where h, and h2 are determined by the equations (4 = r/n)
QW,j = q + Jfq(1- qh), g(h) = fUM - FW”r, h F(h)
=
f(z) dz,
I-00
Q(b) = q - &Cl- q)ld
(2.7)
Q(h) = 1-F(h),
f(z) = (2?rp2 exp( - $2”).
For n z 10, however, CYand @ mzy be obtained from the simpler equations (Tiku, 1970, 1978)
Note that 0 < cly< 1 and 0 < fi c IL;see Tiku (1967, Table 1). For normal samples, the efficiencies and the distributions of p, and O, are investigated by Tiku (1978). Suficc it to say here that p, and u, provide an ideal pair of estimators of the mean ELand the standard deviation a of the N(p, TV) distribution, hosed on Type 11 censored samples; see also Tiku (1968, 1970, 1973) and Smith, Zeiss and Syler (1973). Note that for normal samples (2.2), (CL,&A/&Y and i,.h - l)o$~” are independently distributed as normal N(O, 1) and x2 with A - 1 d.o.f. (degrees 0’ freederrl), respectively, for large A - n - 2r; see (1978, I,Ciiilmas 1 and 2). ean .Z and do, reduce to the sa Note that for r ==0 (no ce the s;itn&- s:zndard deviati
The question is how efkient are pC and s, for estimating the mean and standard: deviation p and G of symmetric nqn-normal distributions as compared with say the sample mean and standard deviation f aud s of complete sa..p!es asrd the prominent pair of ‘robust’ estimators based on the censotio sample (Z.Z) given by
and i‘
-
+ (&I. A2 + r((xn--r- f&d2
v-r)3llJ(n a 2 - 1);
(2.10)
see Huber (1970), Gross (1976) and Stigler (4977). Note that a comparison between the efficiencies of the two pairs (p,, 0~1 and (pT, +) is also of interest since both of them are applicable in the robustness as well as in the censoring framework.
We focus on symmetric distributions (continuous) with existent mean g and existent standard deviation cr. Consider then the family of symmetric distributions (with mean p and standard deviation 0 which may without any loss of generality be taken as 0 and 1, respectively) given by f(P;x)=C(llcr){l+(x-~j2/kOZ}-p., k=2p-3, c = Qj/42rr(p
-QQ
(3.1)
pr2, -- 3/2)r(p - b/Z..)).
?-Me that 1”= Y&&(X- &/CT& has Student’s f-distribution with t, = 2p - 1 d.o.f. (degrees of freedom). The kurtosis of the family (3.1) is given by 6: = 3fp - 3/2)/(p - 5/2) for p ES5/2 which assumes values between 3 and 00. For p =I00, (3.1) reduces to the normal .N(p, o) distribution. Notice that she estimator eL,is of the same form as the Winsorized mean and, therefore, the asymptotic form (or influence curve) and the asymptotic variance of pc are exactly elf the same form as c,quation (bj in Lemma 2 and equation (5”) in Section 3 of Sh(orack (1974), and 1~~is asymptotically normally distributed; see also Stigler (19’74). The exact variance of JA,is given by the equation V(&) :=:(1’Vl)/m2, where K!,...,
vector with (19- 2r) P aided statistics n - Y.The variance V(pTj is given bIy (3.2) wit 1, P-
(3.2)
Rabrcstness@’MML estimators
127
For ~~20, the expected values and the variances and covariances of order statist& of ran&m sampks from the family (3.1) are given by Tiku and Xumra _(197&)W I$r-n~ 20,~thesevalues Imay ~beobtained from the asymptotic expressions f3i_iten~tjg;;D~~d~-(-~Si70i pi ~V6$)AUrthe t family &qx) given by (3-I), the; exact varian~a 06 p+ and El,‘are given b&ow, for n = 10 and 20 and q = r/n :=0.1, 0.2 and 0.3. It is clear thatkforp s 5 (fl$ ~4.2, /3f = 4.2 for the Logistic), both @Tand pc with a proportion of censored observations 4 = r//n less than or equal to 0.2 are more eficient than the sampk mean ZeNote that for the family f(p; x) the n/lVB for estimating lu is given by MVB(& = lI( - E(d2 log L/dp2)} = a2(p + l)(p - 3/2)/np(p - I/?,) and th?refore, the estimators @Tand p, have remarkably high efficiencies. Note that i, pT and p-Zare unbiased. To investigate the bias and the efi%iencie%of the above estimators of CJwe note that for the famil;t fip; X) with existent standard deviation a, the sample standard deviation s, and hence CT and 0;: which are devoid of the “disruptive” extreme order statistics, are asymptotically normally distributed (Cram&, lJ)46, pp. 353, 354 and 366) and, therefore, the MSE of s, WTand cr,, like the MSE of 2, pr- and ~IY~, will for large n be appropriate measures of their eficiencies. Unfortunately, it is difficult to derive mathematical expressions for the MSE of s, G,- ano a, and. therefore, wl;fresorted to simulations, using Box and Muller (1958) eq!latiol:$ for I’abie 1 Exact variances of ~.&r and &,/a .Y
_I__ oc
3.5 6
4.5 4.5
5” 4.2
10 3.4
0.0771 b,= 0.0653 0.0814
0.0894 0.0921
0.0943 0.0960
0.0958 0.0973
0.1013 0.1013
0.1053 0. 1043
q so.2
pT=0.05S3 0.0747 wc= 0.0578 0.0767
0.0904 0.0912
0.0970 0.0972
0.0991 0.0990
0.1071 0.1059
0. 1133 0. 1 113
q = 0.3
f&T=0.0559 0.0774 +L~ = 0.0563 0.0774
0.0955 0.0950
0.1035 0.1026
0.1060 0.1050
0.1158 0. I 143
0. 1238 0. 1218
pm,.= ‘I.0283 0.0372
p#;= 0,0314 0.0395
0.0439 0.0455
0.0466 0.0476
0.0474 0.0482
0.0505 0.0504
0.0530 0.0520
q = 0.2
pT= 0,0264 0.0363 pC= 0.0279 0.0374
0.0444 0.0448
0.0480 0.0479
0.0491 0.0488
0.0534 0.0524
0.0568 0.0552
q = 0.3
pT= 0.0268 0.0376 &kc=0.02?0 0.0375
0.0469 0.
0.0511 ?6
0.0525 51
0.0577 563
0.062 1 2
?I=10
p
2
a?
O”
2.5 OQ
* q ~0.1 ti.&).0599
3
II = 20
q = 0.1
“This distribution is almost identical with the Logistic (Tiku and Jones, 1971).
generating random normal tieviates. The simulated v~3ues (basedi. on lUQO0 j Monte-Carlo runs) are given below, for n = X0 and 20 a ,~d 9 = r/t2 = 10.2 and 0.2. ; It is clear from these values that cr,, besides having small/ :r bias than aIrTV has much smaller MSE tkur CT, for the family f(ls; x). N&e thpr for long-taikd disttibutians (p ~3.5 S..e.flz> 6), arc with 4 = r/n = OJ is more efficient than s, fncidentally, note that for large HI(n Z 20), the apprkmate ~o~I~u~
E(o,)=[E(B)+d{E(B2)+4AE(C)}]j2d{A(A - 1))
(3.3)
provides close approximations to E(tr,) ;
& z(J!:i-~)/U, E(B')~=2mE(.&,_,), E(B2) :=2r2a2E(Z5_,- i.;+l&,_r) iUl
d
E(C)=:mfrEiZP)+2rpE(Z2,_,)-mV(Cc,) d-f+1 Fer example for ct = 20 and 4 = 0.1 and Q.2, the velues of E(o,/cr) for the family f(p; x), obtained from the Eq. (3.3), are 0,732, 0,838, La.909, 0.936, 0,944 and 0.995, and 0.673, 0.790, 0.874,0.908,0.919 and 0.988, for p = 2, 2.5, 3.5,4.5, 5 and 00,respectively. These values are in close agreeinent with the srmulated values given above. A natural measure of the joint deticiency (efkkncy = l/deficiency) of the pair ( per q) is given by Def( per u’,) = MSE(& + MS&,), and similarly for the other two pairs (2. s) arjd (+r, CF&.Note that ?, @Tand pcRI= are unbiased (their %ISE are Table 2 Valuesof (I/a)
and (i/a21 MSE
medn
n=lO P
2
Mean
q
s
0.i)"0.87 0.I 0.66 0.I 0.71 0.2 0.54 0.2 G.64
@.r 0‘ (B'r C',
2.5 --_s-
0.92 0.75 0.81 0.63 0.75
3.5
flSJ20 4.5
0.95 0.81 0.88 0.70 0.83
5
bc’
2
2.5
3.5
0.96 0.96 0.98 0.92 0.97 0.9S 0.83
0.84
0.89
0,67
0.76
0.82
4.5
0.98
0.85 0.89 0.90 0.96 0.73 0.84 0.90 0.93 0.73 0.73 0.78 0.55 0.64 O.'?i0.74 0.85 0.86 0.93 0.67 0.80 O.t3!G 0.90
5
@
0.99 0.86 0.95 0.74 0.92
0.99 0.911 0.99 0.81 0.99
MSE O.!P 0.2520.1310.0940.0830.0760.956fI.194 0.0870.0480.0400.0370.026 (9-z 0.1 0.1800.1290.1030.0960.094X076 0.1440.0900.0650.0560.0550.040 @e 0.1 0.1590.1130.0940,0900.0880.0750.1150.0420.048Q.042MM0 0.034 cr, 0.2 0.2590.1% 0.1620.1510,1490.1280.2260.1550.1160.102G.tOZ0.073 QP< 0.2 0.1970.1490.1310.1250.1230.1090.1500.0850.060060570.3560.049 m-u -s !i
“hJP
Cj ‘2 (1, @7. = IL,
=
i
iWK.j
S7.
= 0;.
= S.
Robustness of AdhE estimators
129
Table3 Joint deficiencies,forthefamily f(p;x) n=lO
re=2@ "r.
P
2
2.5
3.5
4.5
s
Q)
2
2.5
3,s
4.5
5
Qo
9=“0 (3,d
0.3520.2310.1940.1830.1760,1560,2440.1370.0980.0900.0870.076
q=O.l (/A~,+)0.2400.2060.1920.1900.1900.1810.1720.1270.1090.10'4 0,1020.093 (p,,uJ 092240.1940.1860.1850.1790.1800.1460.1020.0940.0900.0880.086 q =0.2 (~;,,cr,, 0.3140.2700.2520.2480.2480.2410.2520.1910.1600.1500.1500.132 (~,,a,)0.2550.2260.2220.2220.2220.2200.1780.1220.1050.1050.1050.104
equal to their variances) and are uncorrelated with s, cr_~and 0,. For the family f(p; x), the deficiencies of (2, s), (k, v&and (EC,,a,) are given in Table 3. It is clear that the pair (p,, o,) is always more efficient than the pair (pT, c& As compared with the pair (2, s), (CL,,a,) with 4 = r/n = 0.1 is more efficient for p ~4.5 (@:>4.5) although slightly less efficient for p ~4.5, for the family f(p; x). To extend the scope of the above comparisons, we simulated (from 10000 Monte-Carlo runs) the deficiencies of the above three pairs of testimators for other symmetric distributions frequently encountered in practice, namely, (i) the shorttailed distributions (0: <3), (ii) the mixtures vf(p; x)+ (1 - rr)f(p; ax), and (iiij the outlier situations, that is, the sample corlsists of n - k observations from f(P; X) and k (small) observations from J@; ax);, 0 > I, and f has mean zero and belongs to the familjr (3.1). The pair (p,, (I‘,) emerges to be jointly more efficient than (cc=,v~), although slightly less efficient for a few &tirt-tailed distributions with 6;~ 1.8 (pf= 1.8 for the Uniform). As compared with the pair (X, s), (p,, cr$ with q = r/n = 0.1 emerges to be jointly more efhcient for the distributions (ii) and (iii) which have mostly @>4.2, but less efficient for the distributions (i) which have /3$< 3. For example for the Uniform, the Normal-mixtures, the Logistic-mixtures and the Normal-outliner situations, the simuislted values of the means of S, UT and a,, and the MSE of X, &kTand p, and s, oT and a, are given below; n = 20 and up= r/n =. 0,l. For samples with any proportion q = ~‘/t2of observations censored on either side of the sample (2.2), the pair (@T,@T)is considerably less efficient than the pair (CL,,a,). For complete samples, (2, s) as the most efkient pair for short-tailed distributions (a: 9: 3). For complete sample; from long-tailed distributions (0: > bservations on each s 3), one may deliberately censor ab ordered sample and calculate air Cfi,,
ML.
130
7-m
Table 4 Simulated
values of (I/o)
Mean md ( l/c?)
Mean Distribution”
s
cl-
1.01 0.97 0.96 0.97 0.96 0.97 0.97
1.03 0.81 0.75 0.76 0.72 0.80 0.75
MSE, ra = 20
MSE
MSE
=c 1.10 0.87 0.82 0.80 0.79 0.88 0.81
0.050 0.050 0.050 0.050 0.050 0.350 0.050
0.06’7 0.041 0.037 0.037 0.034 0.042 0,036
O,Orid 0.042 0.038 0.037 0.036 0.042 0.037
s
u-f
q
0.012 0.065 0.075 0.086 0.092 0.053 &OS9
0,023 0,065 0.089 0.088 O.JO9 0.065 0.084
0.039 0,049 0.068 0,088 0.085 0.043 0.062
“( 1) Unifform; (2) O.SSiV(O, 1) + O.OSN(O, 3); (3) 0.9ON(O, 1) f O.lON(O, 3); (4) O.%L(C, I) + O.OSL~O, 3); (5) 0.9OL(Y), l)+O.lOL(O, 3); (6) (n - l)N(O, 1) and lN(O,3): (7) (n -2)N(O, 1) and 2IV(O, 3).
efficient
than (i., s) for long-taiiled distributions with fifb4.2 and only slightly less But then the question is how efficient efficient for distributions with 3 5 &“~~4.2. I iis this 10% pair (p,, uC) as compared with the prominent “robust” estimators defined for complete samples, from long-tailed distributions; see Gross (1976) and all the references cited in that paper. Of alll the large number of “robust” c;-stimators, Gross ( 1976, pp. 415, 4 16) recommended the wave estimators ~24, bisc+lare estimators BS82 and Hampel estimators H22 as being generally the most efficient. These are given by the following equations; w23:
,&=
TO+ (k&) tan- * {U Jv sin z
uf,=
(kS,)J[nj~sin2z,/(~cos2,)2}], k = 2.4, and the summations include oniy those
k ‘I 8.2,
3
such that
zi = (x, -p T(J/( k&J, and
#(z) = z( I- z2dZ
IZlC 1,
(3.5)
Robustnessaf MA4Lestimators
131
the equations for these estimators are given by Gross (1976, p. 411) but arti too lengthy to be reproduc@ here;
(3.6)
and H22:
~CI=:median {xt}and So= median (14 - Toi}, i = P, 2, . , . , n. We simulated (from 10000 runs) the means and MSE of the estimators (3.4) to (3.6) for all the dist&utions considered above, for samples of size n = 10 and 20. The estimators H22 were found to be slightly less efficient than the estimators w24 and BS82; see also Gross (1976, p, 414). The 10% estimators I_L,and o,(q = r/n = 0.1) were found to be essentially jointly as efficient as the estimators w24 and BS82 (and generally more efficient than ~18, BS74, BS90. I-US, H22 and H25, Gross 1976). For example, we have the following simulated values of the mean and MSE. Note that all the above estimators of iu are unbiased and uncorrelated with the estimators of CR Table 5 Simulated values of f ljcr) Mean and ( lfcr2) MSE, n = 20 Mean Distributiona
o-,
(11 (2)
1.03 0.98 0.93 0.83 0.71 0.82 0.87 0.81
(31 (4) (5) (61 (7j (8)
o, 1.03 0.98 0.93 0.82 0.71 0.82 0.87 0.81
MSE @WI 0.060 0.052 0.048 0.037 0.028 0.038 0.042 0.037
MSE PB
0.060 0,052 0.048 0.037 0.027 0,038 0.042 0.037
%
Def of3
0.016 0.016 0.031 0.031 0.040 0.041 0.065 0.046 0.113 0.114 0,065 0.065 G.0411 0.044 0.062 0.062
w24 0.076 0.083 0.088 0.102 0.141 0.103 0.085 0.099
BS82 0.076 0.083 0,089 0.103 0.141 0.103 0.086 0.099
a(l) Uniform; (2) Normal; (3) f(a; X)with p = 5; (4) f(p; x) with p = 2.5; (5) f(p; x) with p= 2; (6) O.PON(O,l)+O.lOlv(O, 3); (7) (n - W(C), 1) and NO, 3), (8) (n - 2)N(O, 1) and 2N(O, 3).
FOP“disaster” situations, that is, when the sample contains a large Proportion (say more than 20%) of outliers or the sample comes from a distribution with nonexistent meun or standard deviation (CL and (T then being exclusively the location and scale parameters), the “robust” estimators ~24 and BS82 are considerably nlore efficient than the 10% Pair (pC, a,). opefuhy 1 SJch &f~hltions can be spotted easily (through plotting for exampl and the efficiency of PC and cr, can then be considerably enhanced by censoring a greater proportion q = ~/PI( = 0.2 or 0.3) of observations on either side of the ordered sample. For le for five distributions, (1) (n - 5)N(O, 1j and 5N(O, 3); (2) (Q -- 5)N(O, 1) ex y and (5) ~ormall~n~form, we have the t‘s t2; ( Em (09 10); ( following values (Table 6, based on IO@00 runs; whenever possible the random
MT.L rnku
132 Table 6
Simulated values of the Mean and MSE, n = 2Q(a = 1)
(11
(3
--.
MSE
k
(41
- (3
q = 0.2 q =0.3
1.23 I.32 1.21 1.15
1.39 1.38 1.41 1.26
1.37 1.36 1.24 1.E
1.80 1.79 1.70 I,41
2.53 2.51 2.38 2,09
q = 0.2 q = 0.3
0.098 0.097 0,094 0.092
‘3.105 0,104 0,150 0.113
OJ106 0,105 0.108 0.095
0.185 0.183 0.259 0.167
0,362 0,359 0.468 0.329
q = 0.2 q = 0.3
0.195 0.190 0.133 0.158
0.3u5 0,287 0.347 0.224
0.299 0.294 0.197 0.190
1.10 1.09 0.994 0.516
3.09 3.02 2.91 1.81
0.293 0.287 0.227 r):250
0.410 0.391 0.497 0.337
0.:385 0.979 0.305 0.285
1.28 1.25 1.25 0.683
3.45 3.38 3.18 2.14
CL, PB
(31 .-
numbers were divided by a suitable constant to make the variance of the distribution unity). It is clear that cc, and 0;. (particularly with 4 = 0.3) are more efficient than the celebrated estimators w24 and BS82 (Gross, 1976). Besides, cr, has smaller bias. Incidentally, the estimators c;dTand CT with the same value of q (= 0.2 or 0.3) were found to be considerably less efficient than pC and Q,. The above findings substantiate the genera: belief that non-normality essentially comes from the tails and once the extreme observations (representing the tails) are censored, there is hardly any difference between a normal sample and a non-nolrmal sample and in that situation whatever is good for normal (p, and a; for example are good estimators for the normal N(p, 0)) is automatically gooc,Ifor non-normal samples. The only issue one has to resolve is how many observations to censor; see also Tiku (1974, 1975, 1977). We recommend about 30% censoring on either \ide of the sampfc for “disaster” distributions and about 10% for all other long-tailed distributions (@$3); see also Section 6.
To tent the null hyphothesis HO: q = 0, the Studentized statistic te based on (y,, o,) is defined as tc = ~&G&&.. F0r r - 0, tCreduces to the we&known Student’s E+,tatistic
(4.1) t =Ed&.
For normal
samples, the asymptotic (A = n - 2t --3,QQ)null distribution of t, is normal N(O, 1) and for small bamples the null distribution -of f, is closely approximated by Student’s t-distribution having n - 2r - 1 d.o.f.; see Tiku (1978, Lemmas 1 and 2 and Table I). For large samples (A --) 00)from any non-normal distribution which saWes the very -gener&l&gulariy conditions listed by Shorack (1974, p. 662), the distribution of to, like the distributions of the Studentized statistics f~rtn= ,uTl/(n- ~P)/u~ Table 7
Values of the powerof the statistics t, w = tTrimand tc; sample size n = 10 0.3
0.5
1
1,s
Normal t 0.050 0.09 w 0.051 0.09 tc 0.051 0.09
0.22 0.21 0.22
0.42 0.48 0.42
0.90 0.88 0.89
0.99 0.99 0.99
bgistic t o.o$l 0.07 w 0.051 0.07 tc 0.052 0.05
0.14 0.13 0.14
0.22 0.22 0.23
0.50 0.50 0.51
0.9ON(O, 1) + O,lON(O, 3) t 0.045 0.07 0.18 w 0.050 0.07 0.19 t, 0.052 0.08 0.20
0.34 0.35 0.36
and lN(0, 10) 0.04 0.09 0.17 0.08 0.18 0.33 0.09 0.19 0.34
Oa
II,
(n - l)N(O, 1) t 0.024 w 0.049 t, 0.054
0.1
2
0”
0.1
0.3
0.5
1
1.5
2
1.00 0.010 0.02 1.00 0.010 0.20 1.00 0.010 0.02
0.06 0.06 0.06
0.17 0.16 0.17
0.64 0.59 0.61
0.95 0.90 0.91
1.00 1.00 1.00
0.78 0.78 0.79
0.94 0.010 0.02 0.94 0.011 0.02 0.94 0.011 0.02
0.04 0.03 0.04
0.07 0.06 0.07
0.22 0.21 0.23
0.48 0.46 0.48
0.74 0.71 0.73
0.76 0.77 0.80
0.93 0.95 0.97
0.98 0.009 0.02 0.99 0.010 0.02 0.99 0.011 0.02
0.05 0.05 0.06
0.12 0.13 0.13
0.48 0.48 0.49
0.80 0.81 0.84
0.92 0.94 0.96
0.44 0.77 0.78
0.63 0.97 0.97
0.73 0.003 0.004 0.02 1.00 0.010 0.02 0.04 1.00 0.010 0.02 0.05
0.04 0.11 0.12
0.19 0.44 0.46
O-37 0.51 (t.BO 0.96 0.81 0.97
Student’s t (d.o.f. 3) t 0,045 0.09 0.32 w 0.048 0.11 0.34 tc 0.052 0.12 0.36
0.57 0.62 0.64
0.92 0.96 0.96
0.98 0.99 0.99
0.99 0.007 0.02 1.00 0.010 0.02 1.00 0.010 0.03
0.11 0.12 0.13
0.28 0.31 0.33
0.78 0.83 0.84
0.93 0.98 0.98
0.97 0.99 0.99
Stud&s t Id.0.f. 2) t 0,041 0.06 0.14 w 0.045 0.07 0.15 t, 0.052 0.08 0.17
0.22 0.27 0.29
0.54 0.65 0.67
0.75 0.86 0.86
0,84 0.004 0.01 0.94 0.008 0.01 0.94 0.009 0.03
0.03 0.04 0.04
0.06 0.08 0.09
0.28 0.34 0.37
0.52 0.63 0.65
0.69 OAl 0.82
Cauchy t 0.029 0.04 w 0.032 0.05 0.043 0.06 tc
0.07 0.10 0.12
0.12 0.19 0.22
0.27 0.42 0.45
0.39 0.62 0.65
OS0 0.002 0.00 0.75 0.004 0.01 0.76 0.005 0.01
0.01 0.02 0.02
0.03 0.05 0.06
0.10 0.17 0.19
0.20 0.36 0.40
0.30 0.53 0.55
0.41 0;33 8.33
0.61 O.$l 0,SQ 0.66 O.50 0.68
0.93 0.82 0.85
1.00 0.0:3 O.r)? 0.15 0.98 0.014 O.Q6 O,1 I 0.99 0.014 8.06 0.12
0.22 0.20 O.20
0.45 0.30 0.31
0.65 0.44 0.45
il.95 0.76 0.79
Uniformh t 0.052 0.24 w 0.055 0.20 tc O&f:’ 0.19 -_ _ _.-_.
a?“hesevl.li;es twe based on 10000 Monte-G+ rio runs. q-we
p ‘I;0, 0.1, O.lJ, 0.2, 0.25 and 0.4.
134
ML.
Tiku
based on r:he pair (clip,o.&, t,,,= pW&&Wbased on the wave estimators ~24, and K*= ~&&B based on the hisquare estimators BS82, is normal; see Huber and Shorack (1974). The asymptotic variances of t*r,im,tw and tB a,re known to be l:, j;ee Huber (19”1Oj, pa 458) and also Gross (1976, pp. 410, all). TO obtain tht asymptotic variance of tc we note that as A -+ m, the expression B in (2.4) tends to a constant in which case V(o,‘t = E(Q~) .- (E(rr,)}2= 0 from Eq. (3.3). I?;sr large samples, therefore, cr, converges to its expected value; and rr(r,) = ~/i(V(t,))= J(mV(~,?)/E(q.). The values of o&J work out to be very close to 1 for all the distributions consildered in Section 3, for all n 2: 20. For example for the foamily (3.1) with p = 2, 2.5, 3.5, 4.5, S and 08,the values (calculated from the equz(tions (3.2) and (3.3)) of &.. for n = 20 are 1.06, 1.05, 1.03, 1.02, I.02 and 1.OI,, respectively. The corresponding values for fTrlm,t,.+,and tB are also close to 1. For small samples, the null distribution of tlHm is known to be: closely approximated by Student’s t-distribution, having n - 2r - 1 d.o.f.; see Shorack (1974, p. 664) and Huber (1970). Expecting a similar result for to we simulated (under Hca) the probabilities P&Z h), )t being the upper 10, 5 or 1 percent point of Student’s distribution having II- 2r - d d.o.f., and found them very close to 0.10, 0.05 and 0.0 1, respectively. for the symmetric distributions with existent mean and variance (and even for Student’s fi and Cauchy with non-r:xistent variance); see the v&es for ~1= 0 in Table 7, for example. Note that the probabilities P( t, 2 h) are more stable than the probabilities P(tnim 2 h), under NQ (Table 7). For large san@es, ,the nonnull distributions of the statistics t, fm,,,, t,, ts and tc r?re normal with noncentrality parameters s1 = &&, Sz = &(n - 2r)/E(~r~), 8, = &)‘E(o,). Q= pJj;/iE(~r,) and 6:; = &&(o,), respectively, and standard deviations almost 1 for all n 2 20. The values of S:,, a4 and 6, are substantially the same but a2 is generally smaller thsn 5,. For large n, therefore, cw,fB and tc have essentially the ‘3arne power but tTrirnis less powerful than tc (see also Table 7). For long-tailed distributions with 0: b4.2, S1 is generally much smaller than 6,; it is slightly larger ;.han S5 for distributions with 3 s px 54.2, but considerably larger than S5 for short-tailed distributions (@t< 3). The implic:a.tion of these &values an the power /properties of the statistics t and tc is obvious; we omit details for conciseness. For small srdlmples we simulated the power of fc and gfii,,,. These values are given in Table 7, for H = 10. It is clear that maimis always trailing behind tee,as e.Kpected. We also silmulated the percentage points and the power of fw and lB baTed on the bbrolm$’ estimators ~24 and BS82 for sample size n = 10 and compared them with the comsponding values for tC (with q = 9.1 for disjrlbutions with existent mean and standard deviation and 4 = 0.3 for “disaster’” distributions); tw and ta were found to be iess powerful than lcvon the whole. M/e, however, omit details for conc&ness. Note that tc and (r/l,, a,,) have the distinct advantage that they are e robustness as well as i 4 1970)
Let xll,
x12, . . .
I
xln,
and xzl,
x22, . . .
9
x2n2
be two independent
random samples
illol)gl((x - &/cq) and (lla2)g2((x - &/c& One wants to test the null hypothesis Ho: that gt and g2 are identical in all respects. Note that, under Ha, p1 = p2, o1 = a2 and g, and g2 have the same function& form (both may be Logistic, for example). Let from two symmetric distributions
and (5.2) be the two Type 11 symmetrically censored samples obtained by arranging the above two random samples in ascending order of magnitude and censoring ri smallest and r, largest observations; we take r, =z:[i + 0. 112~1, i = 1,2, unless stated otherwise. Let (tic,, a,), i = 1If 2, be the two pairs of MML estimators (given by Eqs 2.3 and 2.4) calculated from the two samples (5.1) and (5.2), respectively. Consider the statistic
where 0,” = {(A, - 1)u;, + (A, - ‘I,cr:_,}/(A 1+ AZ _-2); c Ai = n, -2ri,
n+ = n, -2ri + Zr,Pi,
i = 1,~.
The coefficients ai and & needed to calculate g,, and a,, are given by the Eqs (2.6) or (2.8’ with q and PI replaced by qi -1 ri/ni and ni, respectively. If one of the two means bi is smaller than the other, the numerator in (5.3) will tend to get large in absolute value. If one of’ the two variances ~2 is smaller than the other or if one of the two distributions a (i = 1,2j has longer taits than the other (see the values of E(a,) given in Section 3), the denominator in (5.3) will tend ta get small. Large valrles of IT& therefore, lead to the rejection of Ho. 0f considerable practical intersect, however, is the location-shift alternative
ol= ~2 and gl and g2 have the same functional form. Under &, d = 0. Like the statistic tc (and under the same regularity conditions), the asymptotic (Ai tends to a! null distribution of TI is normal with mean 0, and variance almost 1 (for all n, := 20). For small samples (ni 3 6, i = 1, Z), the probabilities P(Tc 2: h) are closely approximated (under Ho) by the corresponding probabilities of the Student’s t distribution having v = cfW1 ( rt, - 211;- 1) d.o.f,, for all the distributions risingly, also for skew distributions): see the considered in Section 3 (an values for $ = 0 in Table 8.
ML.
136
7Mu
Table 8 C,, W and Tc; sample sizes 61,. n,) = (8.8)
‘v’alues of power of the statistics D’, d -
00
0.5
1
1.5
2
25
3
0’
0.5
1
1.5
2
2.5
3
Noma! D+ 0.050 10.21 0.51 C, 0.050 0.24 0.58 w 0.052 0.24 0.59 'Tc 0.051 0.23 0.58
0.81 0.88 0.83 0.87
0.95 0.99 1.000.0100.07 0.24 0.54 (4.810.95 0.98 1.00 1.000.0100.08 0.28 0.63 0.88 0.98 0.98 1.00 1.00 0.010 0.08 0.29 0.63 0.86 0.98 0.Y8 1.00 1.000.0110.08 0.28 0.59 0.85 0.97
0.98 1.00 1.00 1.00
Logistic D’ 0.050 C, 0.050 W 0.052 T<,. 0.053
0.45 0.48 0.50 0.51
0.63 0.68 0.70 0.69
0.68 0.72 0.75
0.12 (1.13 0.14 0.15
0.26 0.29 0.29 0.31
Normal Mixture: 0.9ON(O, D’ 0.050 0.19 0.45 C, 0.050 0.21 0.48 W 0.052 0.21 0.51 T, 0.052 0.22 0.52
lJ+O.lON(O. 0.72 0.90 0.74 0.31 0.77 0,93 0.78 0.94
0.79 0.84 0.35 0.84
0.89 0.92 0.93 0.93
3) 0.98 0.98 3.98 0.98
0.99 0.010 0.05 0.99 0.010 0.07 0.99 0.010 0.07 0.99 0,010 0.07
0.010 0.010 0.010 0.010
0.03 0.04 0.04 0.04
0.08 0.10 0.11
0.18 0.22 0.22
0.34 0.38 0.40
0.53 0.57 0.59
0.11 0.22 0.38 0.59 0.75 0.20 0.22 0.23 0.23
0.44 0.44 0.48 0.48
0.69 0.69 0.75 0.77
0.87 0.82 O&9 0.91
0.96 0.80 0.9!j 0.97
0.06 0.U6 0.06 0.07
0.19 0.21 0.22 0.22
0.41 0.41 0.45 0.48
0.67 0.63 0.70 0.74
0.86 0.79 0.87 0.92
0.96 0.87 0.94 0.98
Dixon’s Outlier :Uodel:(n - 2).N(O. 1) and 2NIO.3) D’ 0.050 0.17 0.36 0.61 0.81 0.92 0.94 0.010 0.05 0.77 0.88 0.94 0.010 0.05 C, 0.050 0.17 0.37 0.X W 0.052 0.18 0.40 0.601 0.82 0.93 0.97 0.010 0.05 ‘I., 0.050 0.17 0.41 0.67 0.85 0.93 0.98 0.010 0.05
0.15 0.15 0.16 0.17
0.32 0.30 0.33 0.37
0.55 0.48 0.54 0.60
0.73 0.64 0.72 0.78
0.80 0.76 0.84 0.90
Dixon’s Ootlicr Model:(Jt D’ 0.050 0.19 0.42 C, 0.050 0.18 0.39 W 0.052 0.19 0.42 T, 0.051 0.20 0.46
Dixon’s Outlier Model:(rt D’ 0.050 0.19 0.43 C, 0.050 0.20 0.47 W 0.052 0 21 0.49 T,, 0.051 0 ?I !J.50
Tiku’s D‘ C, W T,
Outlier 0.039 0.025 0.034 0.051
l)N(O, 1) and lN(O,3) 0.71 0.89 8.98 1.00 0.72 0.88 0.96 0.99 0.76 0.92 0.98 1.00 0.79 0.93 0.99 1.00
liN~(!l, 1) and lN(0, 0.67 O.&S 0.95 0.591 0.74 0.84 0.66 0.82 0.93 0.73 0.91 0.98
ModeP: fk = 1, A = 0.18 0.44 0.73 0.12 0.33 0.6t 0.16 0.41 r.71 0.23 0 58 0.86
0.010 0.010 0.010 0.011
10) 0.99 0.87 0.97 1.00
O.t!lO 0.010 0.010 0.010
0.05 0.05 0.05 0.06
0.17 0.15 0.18 0.19
0.36 0.30 0.35 0.43
0.60 0.43 0.55 0.69
0.80 0.54 0.72 0.88
J.92 0 59 0.81 0.97
0.007 0.004 0.006 O.Oi2
2) 0.91 0.81 0.80 0.97
0.98 0.93 0.!1’8 I.O+J
1.00 0.98 1.00 1.00
0.05 0.02 0.03 0.08
0.17 0.08 0.10 0.27
0.40 0.20 0.25 0.58
0.67 0.38 0.46 0.85
0.89 0.59 0.68 0.97
0.97 0.78 0.85 1.00
Student’s r (d.0.f. 2) D + 0.(?50 0.17 C, 0.050 0.17 w 0.052 0.18 T, 0 051 0.17
0.38 0.36 0.39 0.40
0.61 0.54 0.59 0.62
0.78 0.74 0.79 0.78
0.90 0.83 0.88 0.90
0.95 0.010 0.05 0.90 0.010 004 0.94 0.010 005 0.95 0.008 0.05
0.16 0.14 0.15 0.17
0.34 0.27 0.31 0.34
0.55 0.45 0.53 0.55
0.71 0.57 0.67 0.73
0.63 0.67 0.713 0.84
Cauchv D’ A050 C..14 C, 0.050 Cl.12 W 0.052 0.13 T, 0.049 0.14 c 0.048 0.14
0.28 0.23 0.27 0.27 0.31
0.46 0.36 0.41 0.42 0.50
0.62 0.47 0.54 0.54 0.65
0.73 0.56 0.66 0.67 0.77
O&l 0.66 Ct.75 0.75 0.85
0.10 0.08 0.10 0.08 il.10
0.20 0.15 0.19 0. I8 0.22
0.34 0.23 0.28 0.29 0.36
0.47 0.30 0.38 0.44 0.50
0.514 0.36 O&I8 0.5;4 0.6’4
0.010 0.010 0.010 0.006 0 006
0.04 0.04 0.04 0.03 0.03
Table 8 (continued) Values of the power of the statistics Lb
0”
0.5
Q*, C,, W and T,; sampie sizes (n,, n,) = (8,8)
1
1.5
2
2.5
3
I&G&ma 9+ O.OSO-oA.2 C, 0,050 0.17 W 0.052 OS T‘. 0.049 0.14 Tee 0.050 0.16
0.25 0.38 0.35 0.29 0.35
0.44 0.62 0.59 0.51 0.62
0.66 0,82 0.79 0.72 0.84
0.83 0.94 0,93 0.89 O.%
0.95 0.99 0,98 0.98 1.00
Chi-square (d.o.f, 4) DC O,O!!!O0.17 C, 0.050 0.21 W 0.052 0.21 Tc 0.052 0.19
0.41 0.46 0.46 0.42
0.68 0.71 0.72 0.67
0.87 Lt.87 0.89 0.85
0.96 0.94 0.96 0.95
0.78 0.73 0.75 0.70
0.95 0.91 0.93 0.80
0.99 0.97 0.98 0.97
1.00 0.99 0.99 0.99
0”
OS
2.5
-y--
1
1.5
2
O.OLO 0.03 0.018 0.04 0.010 0.04 0.011 0+04 0.010 0.05
0.07 0.13 0.12 0.10 0.13
0.18 0.30 0.28 0.21 0.31
0.34 0.50 0.48 0.36 0.54
0.99 0.98 0.99 0.98
0.010 0.010 0.018 OX!11
0.05 0.06 0.06 0.05
0.17 0.19 0.20 0.17
0.38 0.42 0.43 0.37
0.64 0.85 0.92 0.63 0.78 0.87 0.67 0.82 0.92 0.62
0.80
0.91
1.00 1.00 1.00 1.00
0.010 O.OlO 0.010 0.007
0.15 0.16 0.16 0.11
0.50 0.44 0.48 0.42
0.80 0.69 0.75 0.72
0.94 0.83 0.89 0.89
0.99 0.90 0.96 0.96
1.00 0.95 0.98 0.99
0.v 0.76 0.73 0.59 0.79
0.76 0.91 0.89 0.79 0.94
Exponential
D+ 0.050 0.37 C, W T,
0.050 0,052 0.047
0.37 0.38 0.32
Chi-square (d.o.f. 1) D+ 0.050 0.45 0.80
0.93 0.98 0.99 1.00 O.OlO 0.21 054 0.76 0.90 0.95 0.98 C, O.OSO 0.37 0.55 0.82 Q.90 0.94 0.97 0.010 0.16 0.35 0.54 0.66 0.76 0.82 W T,
0.052 0.043
0.40 0.28
0.69 0.59
0.86 0.80
0.93 O.9O
0.97 0.95
0.98 0.98
Log-normal (CT= 1) D’ 0.050 0.27 0.65 C, 0.050 0.28 0.56
0.86 0.74
0.94 0.84
0.98 0.91
0.99 0.94
0.010 0.807
0.17 0.10
0.40 0.34
0.62 0.59
0.75 0.76
0.86 0.87
0.91 0.93
0.010 0.010 0.052 0.29 0.58 0.79 ~?.S;J; 0.94 0.97 0.010
0.10 0.09 0.10
0.35 0.27
0.61 0.45
0.80 0.57
0.90 0.67
0.94 0.84
W 0.30 0.51 0.67 0.78 0.8.5 Tc 0.048 0.23 0.49 0.70 0.83 0.!)20.95 LOG9 0.07 0.23 0.46 0.65 0.80 0.86 These probabilities are exact for C, and W, randomized and based on 4000 Monte-Carlo runs for D‘, and simulated from 4000 runs for T,. bin random samp le s of size n from each of the two normal N(0, 1) disrr?wtions, the smallest k and the largest k observations are multiplied by A. for the Cauchy and rI = r, = O for the Uniform. ‘Here rr = [h+ 0.3n,] and r2= [$+ 0.3n.J, ‘%ere d = 0, 0.1, 0.2, 0.3, 0.4, 0.5 and 0.6,
Note that for normal samples E(o,) = CTfor large Al and AZ (Tiku, 1978) and the p~welr.of 7YY is, say for equal fll and 1z2,given by (n, = 11~= n and r,/n = rz,’rt = 4 = 0.1) (5.4)
P{Z 2: Z8 - 0.9836}, since
is ;Ss
nor
(I- A)% point (S is *
138
ML
Valiues of the ymet _.-A 0 0.5 -___11_ t 0.0s 0.13 -L 0.05 0.12 t 0.01 0,03 +L 0.01 0.03
Tikle
of t and Tc 1
1.5
2
2.5
3
3.5
4
0.26 0.25 0.09 0,09
0.44 0.43 0.20 0.20
0.64 0.63 0.33 0.36
0.80 0.39 MT 0.55
0.91 Q.98 0.75 0.n
0.97 0.96 0.88 Q.87
0.99 0.99 0.95 O.95
Thle power of the two-sample Student’s t-statistic for large normal samples is given by (nl = n2==nj (5.5)
P(Z 1: &j - ik}.
The values of the power of t and TC(for large normal samples) can, therefore! be calculated from (5.5) and (54) and we given in Table 9. It is clear that for normal samples the loss of power resulting from the use of ?‘;:in place of the UMP Student’s; f-statistic is not too serious. Nonparametric statistics are often used to test the null hypotiiesis Ho* partScularly if one suspects the validity of the normality assumptilon, and these are known to be generally more powerful than t for long-tailed symmetric disttibutions. ‘the most ptGT:.+Ient nonparametric statistics are the Wilcoxon s:,‘atisttc (5.6)
The Terry-Hoeffding
normal-score statistic (5.7)
and the Smirnov-Kolmogorov-Massey D’=
max
1.2 . . . . . n
[n*(
l-
El)-
statistic y
fl,ti] I
(J.8)
see Milton (1970, pp. 23, 2S and 27). A vast literature discusses the distributions and power properties of nonparametric statistics; se for example Lehmann (!9?5), Hotz (1964), Ramsey (1971) and Gastwirth (1965). Note that these statistics have remarkably high power, even for small samples (Table 8); see also Lehmann (197% and ivflilton (1970). Tables of the cumulative probabilities of the nulli distributions of ‘Jv, C1 and D+ are given by Wilcoxon, Katti and Wilcox (19731, KBotz (1964) and Kim and Jennrich (1973), respectively. Using these tdfks, we simulated (from 4000 Monte-Carlo runs) the values of the power of W, lo against I., for sample sizes (n,, 81~2 = (8,
(10,8) and (10,lQ). The values for D” were obtained through randomizF!ion betwee% the two appropriate cumulative probabilities (Kim and Jenrich, 1973). The statistic ‘& was found to be generally more pow&‘ul than these nonparametric statistics, except for some very short-tailed symmetric distributions (Uniform,
for example) and som: extremely skewed disttibutions (xt and Log-normal, !!or example) in which cazie theXatistics C1 and D’, respectively, were found to be considerably more powerful than T,; see Table 8. It is interesting to rote that with Tiku’s (1975, 1977) outlier model preserving symmetry, that is, the k smallest and the k largest observations of a random sample from the N(O, a) distribution are multiplied by a positive constant A (or a positive constant h is subtracted from and added to the smallest k and the largest k observations, respectively, of a random sample from the [email protected])), the sign% cance levels of the statistics W, C1 and Di- (but not 7’=)get substantially distorted from their preassigned values; see Table 8. For large samples the powe:r of T= is, say for
n1 = n2= n, given
by
((4= r&t = r&2)
the assumption is that the common pdf g satisfies the very general regularity conditions listed by Shorack (1974, p. 664): A and 2 are as in (5.4). The corresponding large sample power of W is given by P{ Z
2
Zs - af*(O)A},
(5’0)
where f*(O) is the density at zero of the difference u = zI - z2 of tw63independent random variables, each having the pdf g(z), z = (X- p)/o; see Lehmann (1975, pa 72). For the family (3.1), (5.9) is easy to work out from the equation (3.Q since
z”f(z) dz, and i?;l_r and - 5+1 tend to 6 as n tends to infinity, where S!&f(z) dz =
1- q;
ku (1967). Again, f “(0)iseasy toworkout for the family (3.1), de:;pite the fact that g(u) cannot be expressed in a closed form. The Eq. (5.9) alwiays assumes values larger Than (5.10), even in case Df the normal distribution (p = QOin (3.1)). For the normah (p = m), (5.9) reduces to (5.4), and (5.10) reduces to P{Z 2 Z6 - 0.9776). Incidentally, note that for large normal samples, the power of the two-sample statistic fTti;,,based on the estimators PT and 0-r is given by P(Z2:Zs -0.97 “A}. shidn be genernlized to test that The statistic Tc can in a straightforwartd e generalized statistic is given by k( > 2) symmetric distributions are identical
see Harter and Moore (1966) and also
where
The statistic T’$ has pQwet properties similar to TQ.We, however, omit details flax conciseness, IVote that the asympcXic (4 + m) nult distributian of ?‘$ is chin square having R - 1 d.o.f., and for small samples (q ;;r5, i = ‘1,2, . . . , k) the upper percentage points of ‘I’$ ate closely approximated by the corresponding uppet percentage points of the F-distribution having (k - I, u) d,o.f., t, = EFmz(Ar - I); see also Tiku (1978). Example relief of observed the other
(Lehmann, 1975, p. 37). In a comparison of two drugs used for the postoperative pain, the following nuanbers of hours of relief were for 16 paticms, of which 8 had been assigned to a standard drug A and 8 to an experimental drug B, A: B:
6.8 3.l 5.8 4s 3.3 4.7 4.2 4.9 4.4 2.5 2.8 2.1 6.6 0.0 4.8 2.3.
One wants to test whether there are any differences at all between the two drugs. From the following rankings of A and B A: B:
6 1
7 2
8 3
10 4
I1 5
13 9
14 12
16 15,
Lehmann calculates the values of the two-sided Smitnov-Kolmogorov-Massey statistic D = 5/8, and P(D 2 S/S) = 0.087. Here its = ot2= n = 8, and since the data clearly do not belong to “disaster” distributions, we take rl = rz = r”= #+O. l(S)] = 1 in (5.1) and (5.2). We ignore the smallest and the largest observations, i.e. 3.1 and 6.8, in the sample A, and similarly for the sample B, and calculate TC from the Eq. (5.3). Note that = 0.7113 and & = & = fi := 6.8281 (from Eq. (2.6)). The value of TC @l =a2=a works out to be 1.84, and P(l’&l~ 1.84) = 0.094 (8.o.f. 10). This is close to the probability based on the statistic D. Note that to compute D, ali the sixteen observations in the two samples have to be ordered to determine their ranks, but to calculate T’: rrot even the eight observations in either of the two samples have to be completely ordered. Qne simply has to locate the largesi and crllallest observations in each of lthe two samples.
To use the estimators hC and gC in ihe robustness framework and to achieve high e%ciency, one has to know whether the population has a short-tailed distribution case one chooses g = r/n = 0, a long-tailed distribution with existent deviation in w ic’Rcase one c .I, or a ““disaster”
Robusmcss a/‘MML estimaators
141
distribution in which case one chooses q = 0.3. In practice, there might be sirtuations when this knowledge is not ak all available. In such situations, one might uw t’& adaptive estimators JA and q$ defined to be exactly the Same as p, and q., respectively, bvt. _tith the constant fraction q = r/o replaced by the random faction i* A‘P/n, where t* = 0 if nz”/n = 0, I* = [i-k Q.ln] if m *In s 0.1, and +0,3n] if tn*/n = 0.1; m* is the number of values of l%il=/JQ- median(x,
.48 medianlxi- median(x,)j
andi=l,2,.,., n, which exceeds 3.0. The adaptive estimators g: and crx are on the whole as efficient as the adaptive estimators w24 and BS82. For example for the distributions (1) Uniform; (2) Normal; (3) f(p; X) with p = 5; (4) f(p; X) with p = 2.5; (5) (n - 5)N(O, 1) and 5N(O, 3); (6) Student’s t2; (7) Cauchy; and (8) Normal/Uniform; we have the following values (based on 10 000 runs, see Table 10); n = 20 (o= 1): Table 10 (1)
(21
(3)
(4)
(5)
(6)
(7)
(8)
0.96
0.91
0.81
1.29
1.31
1.60
2.36
Mean
a,*
0.99
MSE
&! a:
0.053 0.051 0.049 0.040 0.109 0.12 1 0.222 0.013 0.035 0.048 0.078 0.193 0.278 0.880
0.444 2.72
Def:
(p:, cr?, BS82
0.066 0.086 0.097 0.118 0.302 0.399 1.10 0.036 0.053 0.089 0.103 0.257 0.379 1.25
3.16/5.328 3.38/5.647;
In Table la, 5.328 and 5.649 are the sums i?tfthe deficiencies of the two pairs (& 0,“) and BS82 for the eight distributions (1) to (8), respectively. Like the Stddentized statistics based on ~24 and BS82, however, the small sample ml1 distribution of the Studentized statistic ~T&?/G:, m* being the value of nt obtained by replacing r by t* in (2.6) to (2.8), is difficult to determine and Studznt’s t-approximation does not provide accurate values for the percentage points, especially for “disaster” situations; see also Gross (1976, Table 6). This is seen as a drawback in these adaptive estimators.
My sincere thanks are due to the referees for many helpful comments and suggestions. Thanks are also due to NS 6 of Canada and Research Board fur res script.
142
ML.
zxu
Andrews, D., P.J. Bickel, F. Hampel, P. Huber, W. Rogers and J. I cri ey, wzj. &b~~t E~tim~b~ of Lwatiori : Su~ey anfi Adtmces. Princeto), University Press, Princeton. Bickel, p.J. (1976). Another look at Robustness: a review of reviews and SOrIle new de&l~~m&. ~. 3 Stand. J. Statist. 3, 145-168. BOX, G.E.P. and M.E. Muller, {1958), A note on the generation of random normal deviates. Ann. Mid. Stat&. 29, 610-611. Cramer. I-I. (1945). Mathernarical Methods of Statistics. Princeton University Press, Princeton. David, H.A. (1970). Order Statistics. Wiley, New York. Gastwirth, J.L. (1965). Percentile modification of two sample rank tests. .I. Amer. Stat. Assoc. 60, 1127-l 141. Gross, A.M. (1976). Confidence interval robustness with long-taibd symmetric distributions. J. Amer. Statist. Assoc. 71, 409-416. Harter. H.L. and A.M. Moore (1966). Iterative maximum likelihoud estimation of the parameters of normal populations from singly and doubly censored samples Biometika 53, 205-213. Huber, P.J. (1970). Studentizing robust estimates. In: M.L. Puri, eds., Nonprrrametric Techniques in Statistical lnfercrtce. Cambridge LJniversity Press, Cambridge, 453-463. Huber, P.J. ( B972). Robust statistics: a review. Ann. Muth. Stat. 43, 1041-1067. Holland. P.W. and R.E. Welsch (1977). Robust regression using iteratively reweighted least-squares. Comntun. S&r. A6 (9). 813-827. Kim, P.J. and RI. Jennrich (1973). Tables of the exact sampling distribution of two-sample Kolmogorov-Smirnov criteron D”,,,(rrt d n). in: H.L. Harter and D.B. Gwen, eds., S&cred Tables in Mathematicd Statistics, 2. American Mathematical Society, Providence, 79-170. Klotz, J. (1964). On the normal scores two-sample test. J. Amer. Statist. Assoc. 59, 652-664. Lehmann, E.L. (1975). Nonparametrics: Statistical Methods Based on Ranks. McGraw Hill, New York. Milton, R.C. (1970). Rmk Order Probabilities. Wiley, New York. Ramsey, F.L. (1971). Small sample power function for nonparametric tests of location in the double exponential family. J. Amer. Statr.\i. Assoc. 66, 149-151. Shorack. G. R. (1974). Random means. Anti. Statist. 2, 661-635. Smith. W.B., C.D. Zeis and G.W. Syler (1973). Three parameter lognormal estimation from censored data. J. lndian Starts. Assoc. 11, 15-31. Stigler. S.M. ( 1974). Linear functions of order statistics with smooth weight functions. Ann. Statist. 2, 676-699. Stigler, S-M. (1977). Do robust estimators work with real data? Am. Statist. 5, 1005-1098. Tiku, .M.L. (1967). Estimating the mean and standard deviation from censored normal samples, Biometrika 54, 155-165. Tiku, M.L. (1968). Estimating the parameters of log-normal distribution lrom censored samples. J. Amer. Statist. Assoc. 63, 134-140. Tiku, M.L. (1970). Monte-Carlo study of some simple estimators in censored normal samples. Biometika 57, 207-211. Tiku, M.L. (1973). Testing group effects from Type II censored normal samples in experimental design. Biontetrics 29, 25-33. Xku, M.L. (1974a). A new statistic for testing for normality. Common tiutist. 3 (3), 223-232. Tiku, M.L. ( 1974b). Testing normality and exponentiality in multiqgsamplesituations. Cornmun. Statist. 3 (8). 777-794. Tiku, M.L. (1975). A new statistic for testing suspected outliers. C%mmun. Statist, 4 (8). 737-752. ‘T&u, M.L. (1077). Rejoirler: “Comment on ‘A new statistic for testing suspected outliers”‘. C~m*nun. :;tatist. A6 (14), 141?- 1422. Tiku, ML. (1978). Linear regression model with censored observations. Contmun. Stutist. A7 (13), 1219-1232. ‘Xku. M.L. and P.W. Jones (1971). Best linear unbiased estimators for a distribution similar to the logistic. In: C.S. Carter, T.D. Dwividi, aser, J.R. McGregor and D.A. Sprott, of the “Stfftistics 71 Cawda”, 412-419.
Robustness of A4MLestimators
143
Tiku, ML. and D. Stewart (1977). Rstimattig and testing group effects from Type I censored normal samples in experimental design. Commun. Statist. A6 (15), i46-1501. Tiku, ML. and S. Kumra (1978). Eq~cted values and variarcrs and covariances of order statistics for a far&y ,of symmetrical distributions. Selected Tables in Mathematical Statistics. Submitted. Wegman, EJ, and R.J, CarroE (1977). A Monte-Carlo study of robust estimators of location. Cornmutt. Statist. Ad (9), 795-812. Wilcoxon, F, S,K. Katti and R.A. Wilcox (1973). Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test. In: H.L. Master and D.B. Chven, eds., Sehed T&es in mattkamatical Statistics, I. American Mathematical Society, Providence, 171-259.
OF MNIJ ESTIMATURS BASED QN $iiidPLES A R0BUST TEST STAJTS~CS
Dzpr. of Mathemtuica~ Sciences, McMaster Cfniuersity,Hamilton, Canada
Received 4 September 1979; revised manuscript received IS April 1980 Recommerided by J. Srivastava Abstract: We investigate the efficiences of Tiku’s (1967) modified maximum likelihood estimators fiC and oC (based on symmetrically censored normal samples) for estimating the location and scale parameters fi and o of symmetric non-normal distributions. We show that g, and u, are jointly more efficient than 2 and s for long-tailed distributions (kurtosis pf= P~;c~E~4.2, flz*= 4.2 for the Logistic), and always more eficient than the trimmed mean c1-r and the matching sample estimate m-rof cr. We also show that cc, and aC are jointly at least as efficient as some of the more prominent ‘“robust” estimators (Gross, 1976). We show that the statistic tC- kC&/cxC, 119= n - 2r + 2r#I (r is the number of observations censored on each side of the salslple and fi is a constant), is robust and powerful for testing an assumed value of p. We define a stat%% T, (based on pC and a,) for testing that two symmetric distributions ace idcatical and show that T, is robust and generally more powerful than the well-known nonparametric bzatistics (Wilcoxan, normal-score, Koirnogorov-Smirnov), against the important location-shift alternatives. We generalize the statistic TCto test that k symmetric distributions are identical. ‘Ii:e asymptcJtic distributions of c, and TC zre normal, under some very general regularity conditions. For small samples, the upper (low;:r) percentage points oB tCand T, are shown to be closely approximated by Student’s r-distribu,:ons. Besides, the statistics cc, and cr, (and hence t, and TJ are explicit and simple funct’Dns of sample observaiions and are easy to compute.
Key words: Robust Estimation; Robust Tests; !Vonparametric Tests; Modified Maximum Likelihood Estimatols; Censored Samples; Qutliers.
We investigate the joint efficiencies of Tiku’s (1967, 2970) MML (modified ;naximum likelihood) estimators I_C,and O, based on symmetrically censored normal samples for estimating the lacati.on and scare pilrameters p and cp of symmetric nonnormal distributions, focussing on symmetric distributions with existent mean (location parameter) ~4. and existent standard deviation (scale parameter) 0: We show that pL,and cc are jointly more ef?kient than the pair of eStimatoxs #.+ and C+ (trimmed mean and t and c)r, (based on s IWO)). We a&o s and celebrated ‘“robust” estimators defined for complete samples (wave, Hampel I
123
and bisquare estimators (Gross, 1976)). As compared with the sample mearr and standard deviation Z and s, this 10% pair p, and cr, is shown to be considerably more efficient for long-tailed distributions \kurtosis 0: = bk4/& > 4.2), ‘t;*ttslightly less efficient for distributions with 3~& * ~4.2 and considerably less efficient for short-tailed distributions ;PF ( 3). We define the Studentized statistic I!,= ,u~&/o=~ and investigate its robustn,ess arrci power properties for testing an assumed value of CL.The null distribution of tC, unlike that of the Student’s t = I&/s statistic, is shown to be robust and more powerfui than the Studentiaed statistic based on pr and 0,. We define a statistic TC (based on the MML estimatars) for testing that two symmetric distributions are identical and show that the nul? distribrrtion of TC is robust; TC is also shown to be generally more powerful than the well-known nonparametric statistics (Wi coxon, normal-score, Kolmogrov-Smirnov) against the important location-shift alternatives, for longtailed distributions. The statistic ‘IC iv generalized to test that k symmetric distributions are identical, Note that CL,and o, are simple and explicit functions ot’ sample ohervations for both symmetrically and nonsymmetrically censored samples (Tiku, 1967, 1978). The estimators pC and o, based on samples censored only on one side (in the direction of the longer tail) seem to provide efficient and robust estimators of the mean p atid standard deviation u in the more difficult situation of skew distributions (fiC and a;, are correlated in case of skew distributions). This is under investigation at the present time and we hope to report our findings in a future paper. Note that the MML estimators can also be worked out from censored samples in regression (both simple and mtJltiple) and experimental design framework and these estimators are simple and explicit functions of sample observations: see Tiku (1973, 1978) and Tiku and Stewart i[1977). It may be remembered that the problem tackled in this paper is the estimatiorl of both location and scale, and not I\=cation alone (Huber (1972), Andrews et al, (1972), Bickel f 1976), Wegman and Carroll (1977) and Stigler (P977)), of sym;ne::ic distributions.
2. MML estimators for the normal distribution
tet
he a random sample from the normal N(JL, crj distribution. Let X+X,,,..
..,X tt __ r
(2.2)
be the rf’ylpeI1 symmetrically ceujsored sample of size n - 2r, obtained by arrangj in ~~~~~nding order of magnitude and censoring (trimming) the r sma2ie~ I’ 1 scrvations; the value of r wi later. Tiku’s ( 1Ye,i,
Robusmessoof MML
esti,;rlators
125
3978: modified maximum likelihood (MML) estimators of p and o, calculated from (2.21, are given by (for a formal definition see Tiku and Stewart (1977))
and o, = {B + &3*+ 4AC))/2+,(A
- l)},
G.4)
where m = n--2t+2&
A = H- 2r,
B = ra(xn-r -- &+,),
and C = Wxr(X, - p,)‘+ @{(X**’ lr+l
/AC)2 + (X,.-r - /A,)“}
n-r =
1
(2.5)
Xi’+rp(X~+,+Xf,._,)-mr;,2;
r+l
the coefRcients cy and @ are given by P :={gW -
g(MMh2
-
h I),
a =
guw-
h,P,
(2.6)
where h, and h2 are determined by the equations (4 = r/n)
QW,j = q + Jfq(1- qh), g(h) = fUM - FW”r, h F(h)
=
f(z) dz,
I-00
Q(b) = q - &Cl- q)ld
(2.7)
Q(h) = 1-F(h),
f(z) = (2?rp2 exp( - $2”).
For n z 10, however, CYand @ mzy be obtained from the simpler equations (Tiku, 1970, 1978)
Note that 0 < cly< 1 and 0 < fi c IL;see Tiku (1967, Table 1). For normal samples, the efficiencies and the distributions of p, and O, are investigated by Tiku (1978). Suficc it to say here that p, and u, provide an ideal pair of estimators of the mean ELand the standard deviation a of the N(p, TV) distribution, hosed on Type 11 censored samples; see also Tiku (1968, 1970, 1973) and Smith, Zeiss and Syler (1973). Note that for normal samples (2.2), (CL,&A/&Y and i,.h - l)o$~” are independently distributed as normal N(O, 1) and x2 with A - 1 d.o.f. (degrees 0’ freederrl), respectively, for large A - n - 2r; see (1978, I,Ciiilmas 1 and 2). ean .Z and do, reduce to the sa Note that for r ==0 (no ce the s;itn&- s:zndard deviati
The question is how efkient are pC and s, for estimating the mean and standard: deviation p and G of symmetric nqn-normal distributions as compared with say the sample mean and standard deviation f aud s of complete sa..p!es asrd the prominent pair of ‘robust’ estimators based on the censotio sample (Z.Z) given by
and i‘
-
+ (&I. A2 + r((xn--r- f&d2
v-r)3llJ(n a 2 - 1);
(2.10)
see Huber (1970), Gross (1976) and Stigler (4977). Note that a comparison between the efficiencies of the two pairs (p,, 0~1 and (pT, +) is also of interest since both of them are applicable in the robustness as well as in the censoring framework.
We focus on symmetric distributions (continuous) with existent mean g and existent standard deviation cr. Consider then the family of symmetric distributions (with mean p and standard deviation 0 which may without any loss of generality be taken as 0 and 1, respectively) given by f(P;x)=C(llcr){l+(x-~j2/kOZ}-p., k=2p-3, c = Qj/42rr(p
(3.1)
pr2, -- 3/2)r(p - b/Z..)).
?-Me that 1”= Y&&(X- &/CT& has Student’s f-distribution with t, = 2p - 1 d.o.f. (degrees of freedom). The kurtosis of the family (3.1) is given by 6: = 3fp - 3/2)/(p - 5/2) for p ES5/2 which assumes values between 3 and 00. For p =I00, (3.1) reduces to the normal .N(p, o) distribution. Notice that she estimator eL,is of the same form as the Winsorized mean and, therefore, the asymptotic form (or influence curve) and the asymptotic variance of pc are exactly elf the same form as c,quation (bj in Lemma 2 and equation (5”) in Section 3 of Sh(orack (1974), and 1~~is asymptotically normally distributed; see also Stigler (19’74). The exact variance of JA,is given by the equation V(&) :=:(1’Vl)/m2, where K!,...,
vector with (19- 2r) P aided statistics n - Y.The variance V(pTj is given bIy (3.2) wit 1, P-
(3.2)
Rabrcstness@’MML estimators
127
For ~~20, the expected values and the variances and covariances of order statist& of ran&m sampks from the family (3.1) are given by Tiku and Xumra _(197&)W I$r-n~ 20,~thesevalues Imay ~beobtained from the asymptotic expressions f3i_iten~tjg;;D~~d~-(-~Si70i pi ~V6$)AUrthe t family &qx) given by (3-I), the; exact varian~a 06 p+ and El,‘are given b&ow, for n = 10 and 20 and q = r/n :=0.1, 0.2 and 0.3. It is clear thatkforp s 5 (fl$ ~4.2, /3f = 4.2 for the Logistic), both @Tand pc with a proportion of censored observations 4 = r//n less than or equal to 0.2 are more eficient than the sampk mean ZeNote that for the family f(p; x) the n/lVB for estimating lu is given by MVB(& = lI( - E(d2 log L/dp2)} = a2(p + l)(p - 3/2)/np(p - I/?,) and th?refore, the estimators @Tand p, have remarkably high efficiencies. Note that i, pT and p-Zare unbiased. To investigate the bias and the efi%iencie%of the above estimators of CJwe note that for the famil;t fip; X) with existent standard deviation a, the sample standard deviation s, and hence CT and 0;: which are devoid of the “disruptive” extreme order statistics, are asymptotically normally distributed (Cram&, lJ)46, pp. 353, 354 and 366) and, therefore, the MSE of s, WTand cr,, like the MSE of 2, pr- and ~IY~, will for large n be appropriate measures of their eficiencies. Unfortunately, it is difficult to derive mathematical expressions for the MSE of s, G,- ano a, and. therefore, wl;fresorted to simulations, using Box and Muller (1958) eq!latiol:$ for I’abie 1 Exact variances of ~.&r and &,/a .Y
_I__ oc
3.5 6
4.5 4.5
5” 4.2
10 3.4
0.0771 b,= 0.0653 0.0814
0.0894 0.0921
0.0943 0.0960
0.0958 0.0973
0.1013 0.1013
0.1053 0. 1043
q so.2
pT=0.05S3 0.0747 wc= 0.0578 0.0767
0.0904 0.0912
0.0970 0.0972
0.0991 0.0990
0.1071 0.1059
0. 1133 0. 1 113
q = 0.3
f&T=0.0559 0.0774 +L~ = 0.0563 0.0774
0.0955 0.0950
0.1035 0.1026
0.1060 0.1050
0.1158 0. I 143
0. 1238 0. 1218
pm,.= ‘I.0283 0.0372
p#;= 0,0314 0.0395
0.0439 0.0455
0.0466 0.0476
0.0474 0.0482
0.0505 0.0504
0.0530 0.0520
q = 0.2
pT= 0,0264 0.0363 pC= 0.0279 0.0374
0.0444 0.0448
0.0480 0.0479
0.0491 0.0488
0.0534 0.0524
0.0568 0.0552
q = 0.3
pT= 0.0268 0.0376 &kc=0.02?0 0.0375
0.0469 0.
0.0511 ?6
0.0525 51
0.0577 563
0.062 1 2
?I=10
p
2
a?
O”
2.5 OQ
* q ~0.1 ti.&).0599
3
II = 20
q = 0.1
“This distribution is almost identical with the Logistic (Tiku and Jones, 1971).
generating random normal tieviates. The simulated v~3ues (basedi. on lUQO0 j Monte-Carlo runs) are given below, for n = X0 and 20 a ,~d 9 = r/t2 = 10.2 and 0.2. ; It is clear from these values that cr,, besides having small/ :r bias than aIrTV has much smaller MSE tkur CT, for the family f(ls; x). N&e thpr for long-taikd disttibutians (p ~3.5 S..e.flz> 6), arc with 4 = r/n = OJ is more efficient than s, fncidentally, note that for large HI(n Z 20), the apprkmate ~o~I~u~
E(o,)=[E(B)+d{E(B2)+4AE(C)}]j2d{A(A - 1))
(3.3)
provides close approximations to E(tr,) ;
& z(J!:i-~)/U, E(B')~=2mE(.&,_,), E(B2) :=2r2a2E(Z5_,- i.;+l&,_r) iUl
d
E(C)=:mfrEiZP)+2rpE(Z2,_,)-mV(Cc,) d-f+1 Fer example for ct = 20 and 4 = 0.1 and Q.2, the velues of E(o,/cr) for the family f(p; x), obtained from the Eq. (3.3), are 0,732, 0,838, La.909, 0.936, 0,944 and 0.995, and 0.673, 0.790, 0.874,0.908,0.919 and 0.988, for p = 2, 2.5, 3.5,4.5, 5 and 00,respectively. These values are in close agreeinent with the srmulated values given above. A natural measure of the joint deticiency (efkkncy = l/deficiency) of the pair ( per q) is given by Def( per u’,) = MSE(& + MS&,), and similarly for the other two pairs (2. s) arjd (+r, CF&.Note that ?, @Tand pcRI= are unbiased (their %ISE are Table 2 Valuesof (I/a)
and (i/a21 MSE
medn
n=lO P
2
Mean
q
s
0.i)"0.87 0.I 0.66 0.I 0.71 0.2 0.54 0.2 G.64
@.r 0‘ (B'r C',
2.5 --_s-
0.92 0.75 0.81 0.63 0.75
3.5
flSJ20 4.5
0.95 0.81 0.88 0.70 0.83
5
bc’
2
2.5
3.5
0.96 0.96 0.98 0.92 0.97 0.9S 0.83
0.84
0.89
0,67
0.76
0.82
4.5
0.98
0.85 0.89 0.90 0.96 0.73 0.84 0.90 0.93 0.73 0.73 0.78 0.55 0.64 O.'?i0.74 0.85 0.86 0.93 0.67 0.80 O.t3!G 0.90
5
@
0.99 0.86 0.95 0.74 0.92
0.99 0.911 0.99 0.81 0.99
MSE O.!P 0.2520.1310.0940.0830.0760.956fI.194 0.0870.0480.0400.0370.026 (9-z 0.1 0.1800.1290.1030.0960.094X076 0.1440.0900.0650.0560.0550.040 @e 0.1 0.1590.1130.0940,0900.0880.0750.1150.0420.048Q.042MM0 0.034 cr, 0.2 0.2590.1% 0.1620.1510,1490.1280.2260.1550.1160.102G.tOZ0.073 QP< 0.2 0.1970.1490.1310.1250.1230.1090.1500.0850.060060570.3560.049 m-u -s !i
“hJP
Cj ‘2 (1, @7. = IL,
=
i
iWK.j
S7.
= 0;.
= S.
Robustness of AdhE estimators
129
Table3 Joint deficiencies,forthefamily f(p;x) n=lO
re=2@ "r.
P
2
2.5
3.5
4.5
s
Q)
2
2.5
3,s
4.5
5
Qo
9=“0 (3,d
0.3520.2310.1940.1830.1760,1560,2440.1370.0980.0900.0870.076
q=O.l (/A~,+)0.2400.2060.1920.1900.1900.1810.1720.1270.1090.10'4 0,1020.093 (p,,uJ 092240.1940.1860.1850.1790.1800.1460.1020.0940.0900.0880.086 q =0.2 (~;,,cr,, 0.3140.2700.2520.2480.2480.2410.2520.1910.1600.1500.1500.132 (~,,a,)0.2550.2260.2220.2220.2220.2200.1780.1220.1050.1050.1050.104
equal to their variances) and are uncorrelated with s, cr_~and 0,. For the family f(p; x), the deficiencies of (2, s), (k, v&and (EC,,a,) are given in Table 3. It is clear that the pair (p,, o,) is always more efficient than the pair (pT, c& As compared with the pair (2, s), (CL,,a,) with 4 = r/n = 0.1 is more efficient for p ~4.5 (@:>4.5) although slightly less efficient for p ~4.5, for the family f(p; x). To extend the scope of the above comparisons, we simulated (from 10000 Monte-Carlo runs) the deficiencies of the above three pairs of testimators for other symmetric distributions frequently encountered in practice, namely, (i) the shorttailed distributions (0: <3), (ii) the mixtures vf(p; x)+ (1 - rr)f(p; ax), and (iiij the outlier situations, that is, the sample corlsists of n - k observations from f(P; X) and k (small) observations from J@; ax);, 0 > I, and f has mean zero and belongs to the familjr (3.1). The pair (p,, (I‘,) emerges to be jointly more efficient than (cc=,v~), although slightly less efficient for a few &tirt-tailed distributions with 6;~ 1.8 (pf= 1.8 for the Uniform). As compared with the pair (X, s), (p,, cr$ with q = r/n = 0.1 emerges to be jointly more efhcient for the distributions (ii) and (iii) which have mostly @>4.2, but less efficient for the distributions (i) which have /3$< 3. For example for the Uniform, the Normal-mixtures, the Logistic-mixtures and the Normal-outliner situations, the simuislted values of the means of S, UT and a,, and the MSE of X, &kTand p, and s, oT and a, are given below; n = 20 and up= r/n =. 0,l. For samples with any proportion q = ~‘/t2of observations censored on either side of the sample (2.2), the pair (@T,@T)is considerably less efficient than the pair (CL,,a,). For complete samples, (2, s) as the most efkient pair for short-tailed distributions (a: 9: 3). For complete sample; from long-tailed distributions (0: > bservations on each s 3), one may deliberately censor ab ordered sample and calculate air Cfi,,
ML.
130
7-m
Table 4 Simulated
values of (I/o)
Mean md ( l/c?)
Mean Distribution”
s
cl-
1.01 0.97 0.96 0.97 0.96 0.97 0.97
1.03 0.81 0.75 0.76 0.72 0.80 0.75
MSE, ra = 20
MSE
MSE
=c 1.10 0.87 0.82 0.80 0.79 0.88 0.81
0.050 0.050 0.050 0.050 0.050 0.350 0.050
0.06’7 0.041 0.037 0.037 0.034 0.042 0,036
O,Orid 0.042 0.038 0.037 0.036 0.042 0.037
s
u-f
q
0.012 0.065 0.075 0.086 0.092 0.053 &OS9
0,023 0,065 0.089 0.088 O.JO9 0.065 0.084
0.039 0,049 0.068 0,088 0.085 0.043 0.062
“( 1) Unifform; (2) O.SSiV(O, 1) + O.OSN(O, 3); (3) 0.9ON(O, 1) f O.lON(O, 3); (4) O.%L(C, I) + O.OSL~O, 3); (5) 0.9OL(Y), l)+O.lOL(O, 3); (6) (n - l)N(O, 1) and lN(O,3): (7) (n -2)N(O, 1) and 2IV(O, 3).
efficient
than (i., s) for long-taiiled distributions with fifb4.2 and only slightly less But then the question is how efficient efficient for distributions with 3 5 &“~~4.2. I iis this 10% pair (p,, uC) as compared with the prominent “robust” estimators defined for complete samples, from long-tailed distributions; see Gross (1976) and all the references cited in that paper. Of alll the large number of “robust” c;-stimators, Gross ( 1976, pp. 415, 4 16) recommended the wave estimators ~24, bisc+lare estimators BS82 and Hampel estimators H22 as being generally the most efficient. These are given by the following equations; w23:
,&=
TO+ (k&) tan- * {U Jv sin z
uf,=
(kS,)J[nj~sin2z,/(~cos2,)2}], k = 2.4, and the summations include oniy those
k ‘I 8.2,
3
such that
zi = (x, -p T(J/( k&J, and
#(z) = z( I- z2dZ
IZlC 1,
(3.5)
Robustnessaf MA4Lestimators
131
the equations for these estimators are given by Gross (1976, p. 411) but arti too lengthy to be reproduc@ here;
(3.6)
and H22:
~CI=:median {xt}and So= median (14 - Toi}, i = P, 2, . , . , n. We simulated (from 10000 runs) the means and MSE of the estimators (3.4) to (3.6) for all the dist&utions considered above, for samples of size n = 10 and 20. The estimators H22 were found to be slightly less efficient than the estimators w24 and BS82; see also Gross (1976, p, 414). The 10% estimators I_L,and o,(q = r/n = 0.1) were found to be essentially jointly as efficient as the estimators w24 and BS82 (and generally more efficient than ~18, BS74, BS90. I-US, H22 and H25, Gross 1976). For example, we have the following simulated values of the mean and MSE. Note that all the above estimators of iu are unbiased and uncorrelated with the estimators of CR Table 5 Simulated values of f ljcr) Mean and ( lfcr2) MSE, n = 20 Mean Distributiona
o-,
(11 (2)
1.03 0.98 0.93 0.83 0.71 0.82 0.87 0.81
(31 (4) (5) (61 (7j (8)
o, 1.03 0.98 0.93 0.82 0.71 0.82 0.87 0.81
MSE @WI 0.060 0.052 0.048 0.037 0.028 0.038 0.042 0.037
MSE PB
0.060 0,052 0.048 0.037 0.027 0,038 0.042 0.037
%
Def of3
0.016 0.016 0.031 0.031 0.040 0.041 0.065 0.046 0.113 0.114 0,065 0.065 G.0411 0.044 0.062 0.062
w24 0.076 0.083 0.088 0.102 0.141 0.103 0.085 0.099
BS82 0.076 0.083 0,089 0.103 0.141 0.103 0.086 0.099
a(l) Uniform; (2) Normal; (3) f(a; X)with p = 5; (4) f(p; x) with p = 2.5; (5) f(p; x) with p= 2; (6) O.PON(O,l)+O.lOlv(O, 3); (7) (n - W(C), 1) and NO, 3), (8) (n - 2)N(O, 1) and 2N(O, 3).
FOP“disaster” situations, that is, when the sample contains a large Proportion (say more than 20%) of outliers or the sample comes from a distribution with nonexistent meun or standard deviation (CL and (T then being exclusively the location and scale parameters), the “robust” estimators ~24 and BS82 are considerably nlore efficient than the 10% Pair (pC, a,). opefuhy 1 SJch &f~hltions can be spotted easily (through plotting for exampl and the efficiency of PC and cr, can then be considerably enhanced by censoring a greater proportion q = ~/PI( = 0.2 or 0.3) of observations on either side of the ordered sample. For le for five distributions, (1) (n - 5)N(O, 1j and 5N(O, 3); (2) (Q -- 5)N(O, 1) ex y and (5) ~ormall~n~form, we have the t‘s t2; ( Em (09 10); ( following values (Table 6, based on IO@00 runs; whenever possible the random
MT.L rnku
132 Table 6
Simulated values of the Mean and MSE, n = 2Q(a = 1)
(11
(3
--.
MSE
k
(41
- (3
q = 0.2 q =0.3
1.23 I.32 1.21 1.15
1.39 1.38 1.41 1.26
1.37 1.36 1.24 1.E
1.80 1.79 1.70 I,41
2.53 2.51 2.38 2,09
q = 0.2 q = 0.3
0.098 0.097 0,094 0.092
‘3.105 0,104 0,150 0.113
OJ106 0,105 0.108 0.095
0.185 0.183 0.259 0.167
0,362 0,359 0.468 0.329
q = 0.2 q = 0.3
0.195 0.190 0.133 0.158
0.3u5 0,287 0.347 0.224
0.299 0.294 0.197 0.190
1.10 1.09 0.994 0.516
3.09 3.02 2.91 1.81
0.293 0.287 0.227 r):250
0.410 0.391 0.497 0.337
0.:385 0.979 0.305 0.285
1.28 1.25 1.25 0.683
3.45 3.38 3.18 2.14
CL, PB
(31 .-
numbers were divided by a suitable constant to make the variance of the distribution unity). It is clear that cc, and 0;. (particularly with 4 = 0.3) are more efficient than the celebrated estimators w24 and BS82 (Gross, 1976). Besides, cr, has smaller bias. Incidentally, the estimators c;dTand CT with the same value of q (= 0.2 or 0.3) were found to be considerably less efficient than pC and Q,. The above findings substantiate the genera: belief that non-normality essentially comes from the tails and once the extreme observations (representing the tails) are censored, there is hardly any difference between a normal sample and a non-nolrmal sample and in that situation whatever is good for normal (p, and a; for example are good estimators for the normal N(p, 0)) is automatically gooc,Ifor non-normal samples. The only issue one has to resolve is how many observations to censor; see also Tiku (1974, 1975, 1977). We recommend about 30% censoring on either \ide of the sampfc for “disaster” distributions and about 10% for all other long-tailed distributions (@$3); see also Section 6.
To tent the null hyphothesis HO: q = 0, the Studentized statistic te based on (y,, o,) is defined as tc = ~&G&&.. F0r r - 0, tCreduces to the we&known Student’s E+,tatistic
(4.1) t =Ed&.
For normal
samples, the asymptotic (A = n - 2t --3,QQ)null distribution of t, is normal N(O, 1) and for small bamples the null distribution -of f, is closely approximated by Student’s t-distribution having n - 2r - 1 d.o.f.; see Tiku (1978, Lemmas 1 and 2 and Table I). For large samples (A --) 00)from any non-normal distribution which saWes the very -gener&l&gulariy conditions listed by Shorack (1974, p. 662), the distribution of to, like the distributions of the Studentized statistics f~rtn= ,uTl/(n- ~P)/u~ Table 7
Values of the powerof the statistics t, w = tTrimand tc; sample size n = 10 0.3
0.5
1
1,s
Normal t 0.050 0.09 w 0.051 0.09 tc 0.051 0.09
0.22 0.21 0.22
0.42 0.48 0.42
0.90 0.88 0.89
0.99 0.99 0.99
bgistic t o.o$l 0.07 w 0.051 0.07 tc 0.052 0.05
0.14 0.13 0.14
0.22 0.22 0.23
0.50 0.50 0.51
0.9ON(O, 1) + O,lON(O, 3) t 0.045 0.07 0.18 w 0.050 0.07 0.19 t, 0.052 0.08 0.20
0.34 0.35 0.36
and lN(0, 10) 0.04 0.09 0.17 0.08 0.18 0.33 0.09 0.19 0.34
Oa
II,
(n - l)N(O, 1) t 0.024 w 0.049 t, 0.054
0.1
2
0”
0.1
0.3
0.5
1
1.5
2
1.00 0.010 0.02 1.00 0.010 0.20 1.00 0.010 0.02
0.06 0.06 0.06
0.17 0.16 0.17
0.64 0.59 0.61
0.95 0.90 0.91
1.00 1.00 1.00
0.78 0.78 0.79
0.94 0.010 0.02 0.94 0.011 0.02 0.94 0.011 0.02
0.04 0.03 0.04
0.07 0.06 0.07
0.22 0.21 0.23
0.48 0.46 0.48
0.74 0.71 0.73
0.76 0.77 0.80
0.93 0.95 0.97
0.98 0.009 0.02 0.99 0.010 0.02 0.99 0.011 0.02
0.05 0.05 0.06
0.12 0.13 0.13
0.48 0.48 0.49
0.80 0.81 0.84
0.92 0.94 0.96
0.44 0.77 0.78
0.63 0.97 0.97
0.73 0.003 0.004 0.02 1.00 0.010 0.02 0.04 1.00 0.010 0.02 0.05
0.04 0.11 0.12
0.19 0.44 0.46
O-37 0.51 (t.BO 0.96 0.81 0.97
Student’s t (d.o.f. 3) t 0,045 0.09 0.32 w 0.048 0.11 0.34 tc 0.052 0.12 0.36
0.57 0.62 0.64
0.92 0.96 0.96
0.98 0.99 0.99
0.99 0.007 0.02 1.00 0.010 0.02 1.00 0.010 0.03
0.11 0.12 0.13
0.28 0.31 0.33
0.78 0.83 0.84
0.93 0.98 0.98
0.97 0.99 0.99
Stud&s t Id.0.f. 2) t 0,041 0.06 0.14 w 0.045 0.07 0.15 t, 0.052 0.08 0.17
0.22 0.27 0.29
0.54 0.65 0.67
0.75 0.86 0.86
0,84 0.004 0.01 0.94 0.008 0.01 0.94 0.009 0.03
0.03 0.04 0.04
0.06 0.08 0.09
0.28 0.34 0.37
0.52 0.63 0.65
0.69 OAl 0.82
Cauchy t 0.029 0.04 w 0.032 0.05 0.043 0.06 tc
0.07 0.10 0.12
0.12 0.19 0.22
0.27 0.42 0.45
0.39 0.62 0.65
OS0 0.002 0.00 0.75 0.004 0.01 0.76 0.005 0.01
0.01 0.02 0.02
0.03 0.05 0.06
0.10 0.17 0.19
0.20 0.36 0.40
0.30 0.53 0.55
0.41 0;33 8.33
0.61 O.$l 0,SQ 0.66 O.50 0.68
0.93 0.82 0.85
1.00 0.0:3 O.r)? 0.15 0.98 0.014 O.Q6 O,1 I 0.99 0.014 8.06 0.12
0.22 0.20 O.20
0.45 0.30 0.31
0.65 0.44 0.45
il.95 0.76 0.79
Uniformh t 0.052 0.24 w 0.055 0.20 tc O&f:’ 0.19 -_ _ _.-_.
a?“hesevl.li;es twe based on 10000 Monte-G+ rio runs. q-we
p ‘I;0, 0.1, O.lJ, 0.2, 0.25 and 0.4.
134
ML.
Tiku
based on r:he pair (clip,o.&, t,,,= pW&&Wbased on the wave estimators ~24, and K*= ~&&B based on the hisquare estimators BS82, is normal; see Huber and Shorack (1974). The asymptotic variances of t*r,im,tw and tB a,re known to be l:, j;ee Huber (19”1Oj, pa 458) and also Gross (1976, pp. 410, all). TO obtain tht asymptotic variance of tc we note that as A -+ m, the expression B in (2.4) tends to a constant in which case V(o,‘t = E(Q~) .- (E(rr,)}2= 0 from Eq. (3.3). I?;sr large samples, therefore, cr, converges to its expected value; and rr(r,) = ~/i(V(t,))= J(mV(~,?)/E(q.). The values of o&J work out to be very close to 1 for all the distributions consildered in Section 3, for all n 2: 20. For example for the foamily (3.1) with p = 2, 2.5, 3.5, 4.5, S and 08,the values (calculated from the equz(tions (3.2) and (3.3)) of &.. for n = 20 are 1.06, 1.05, 1.03, 1.02, I.02 and 1.OI,, respectively. The corresponding values for fTrlm,t,.+,and tB are also close to 1. For small samples, the null distribution of tlHm is known to be: closely approximated by Student’s t-distribution, having n - 2r - 1 d.o.f.; see Shorack (1974, p. 664) and Huber (1970). Expecting a similar result for to we simulated (under Hca) the probabilities P&Z h), )t being the upper 10, 5 or 1 percent point of Student’s distribution having II- 2r - d d.o.f., and found them very close to 0.10, 0.05 and 0.0 1, respectively. for the symmetric distributions with existent mean and variance (and even for Student’s fi and Cauchy with non-r:xistent variance); see the v&es for ~1= 0 in Table 7, for example. Note that the probabilities P( t, 2 h) are more stable than the probabilities P(tnim 2 h), under NQ (Table 7). For large san@es, ,the nonnull distributions of the statistics t, fm,,,, t,, ts and tc r?re normal with noncentrality parameters s1 = &&, Sz = &(n - 2r)/E(~r~), 8, = &)‘E(o,). Q= pJj;/iE(~r,) and 6:; = &&(o,), respectively, and standard deviations almost 1 for all n 2 20. The values of S:,, a4 and 6, are substantially the same but a2 is generally smaller thsn 5,. For large n, therefore, cw,fB and tc have essentially the ‘3arne power but tTrirnis less powerful than tc (see also Table 7). For long-tailed distributions with 0: b4.2, S1 is generally much smaller than 6,; it is slightly larger ;.han S5 for distributions with 3 s px 54.2, but considerably larger than S5 for short-tailed distributions (@t< 3). The implic:a.tion of these &values an the power /properties of the statistics t and tc is obvious; we omit details for conciseness. For small srdlmples we simulated the power of fc and gfii,,,. These values are given in Table 7, for H = 10. It is clear that maimis always trailing behind tee,as e.Kpected. We also silmulated the percentage points and the power of fw and lB baTed on the bbrolm$’ estimators ~24 and BS82 for sample size n = 10 and compared them with the comsponding values for tC (with q = 9.1 for disjrlbutions with existent mean and standard deviation and 4 = 0.3 for “disaster’” distributions); tw and ta were found to be iess powerful than lcvon the whole. M/e, however, omit details for conc&ness. Note that tc and (r/l,, a,,) have the distinct advantage that they are e robustness as well as i 4 1970)
Let xll,
x12, . . .
I
xln,
and xzl,
x22, . . .
9
x2n2
be two independent
random samples
illol)gl((x - &/cq) and (lla2)g2((x - &/c& One wants to test the null hypothesis Ho: that gt and g2 are identical in all respects. Note that, under Ha, p1 = p2, o1 = a2 and g, and g2 have the same function& form (both may be Logistic, for example). Let from two symmetric distributions
and (5.2) be the two Type 11 symmetrically censored samples obtained by arranging the above two random samples in ascending order of magnitude and censoring ri smallest and r, largest observations; we take r, =z:[i + 0. 112~1, i = 1,2, unless stated otherwise. Let (tic,, a,), i = 1If 2, be the two pairs of MML estimators (given by Eqs 2.3 and 2.4) calculated from the two samples (5.1) and (5.2), respectively. Consider the statistic
where 0,” = {(A, - 1)u;, + (A, - ‘I,cr:_,}/(A 1+ AZ _-2); c Ai = n, -2ri,
n+ = n, -2ri + Zr,Pi,
i = 1,~.
The coefficients ai and & needed to calculate g,, and a,, are given by the Eqs (2.6) or (2.8’ with q and PI replaced by qi -1 ri/ni and ni, respectively. If one of the two means bi is smaller than the other, the numerator in (5.3) will tend to get large in absolute value. If one of’ the two variances ~2 is smaller than the other or if one of the two distributions a (i = 1,2j has longer taits than the other (see the values of E(a,) given in Section 3), the denominator in (5.3) will tend ta get small. Large valrles of IT& therefore, lead to the rejection of Ho. 0f considerable practical intersect, however, is the location-shift alternative
ol= ~2 and gl and g2 have the same functional form. Under &, d = 0. Like the statistic tc (and under the same regularity conditions), the asymptotic (Ai tends to a! null distribution of TI is normal with mean 0, and variance almost 1 (for all n, := 20). For small samples (ni 3 6, i = 1, Z), the probabilities P(Tc 2: h) are closely approximated (under Ho) by the corresponding probabilities of the Student’s t distribution having v = cfW1 ( rt, - 211;- 1) d.o.f,, for all the distributions risingly, also for skew distributions): see the considered in Section 3 (an values for $ = 0 in Table 8.
ML.
136
7Mu
Table 8 C,, W and Tc; sample sizes 61,. n,) = (8.8)
‘v’alues of power of the statistics D’, d -
00
0.5
1
1.5
2
25
3
0’
0.5
1
1.5
2
2.5
3
Noma! D+ 0.050 10.21 0.51 C, 0.050 0.24 0.58 w 0.052 0.24 0.59 'Tc 0.051 0.23 0.58
0.81 0.88 0.83 0.87
0.95 0.99 1.000.0100.07 0.24 0.54 (4.810.95 0.98 1.00 1.000.0100.08 0.28 0.63 0.88 0.98 0.98 1.00 1.00 0.010 0.08 0.29 0.63 0.86 0.98 0.Y8 1.00 1.000.0110.08 0.28 0.59 0.85 0.97
0.98 1.00 1.00 1.00
Logistic D’ 0.050 C, 0.050 W 0.052 T<,. 0.053
0.45 0.48 0.50 0.51
0.63 0.68 0.70 0.69
0.68 0.72 0.75
0.12 (1.13 0.14 0.15
0.26 0.29 0.29 0.31
Normal Mixture: 0.9ON(O, D’ 0.050 0.19 0.45 C, 0.050 0.21 0.48 W 0.052 0.21 0.51 T, 0.052 0.22 0.52
lJ+O.lON(O. 0.72 0.90 0.74 0.31 0.77 0,93 0.78 0.94
0.79 0.84 0.35 0.84
0.89 0.92 0.93 0.93
3) 0.98 0.98 3.98 0.98
0.99 0.010 0.05 0.99 0.010 0.07 0.99 0.010 0.07 0.99 0,010 0.07
0.010 0.010 0.010 0.010
0.03 0.04 0.04 0.04
0.08 0.10 0.11
0.18 0.22 0.22
0.34 0.38 0.40
0.53 0.57 0.59
0.11 0.22 0.38 0.59 0.75 0.20 0.22 0.23 0.23
0.44 0.44 0.48 0.48
0.69 0.69 0.75 0.77
0.87 0.82 O&9 0.91
0.96 0.80 0.9!j 0.97
0.06 0.U6 0.06 0.07
0.19 0.21 0.22 0.22
0.41 0.41 0.45 0.48
0.67 0.63 0.70 0.74
0.86 0.79 0.87 0.92
0.96 0.87 0.94 0.98
Dixon’s Outlier :Uodel:(n - 2).N(O. 1) and 2NIO.3) D’ 0.050 0.17 0.36 0.61 0.81 0.92 0.94 0.010 0.05 0.77 0.88 0.94 0.010 0.05 C, 0.050 0.17 0.37 0.X W 0.052 0.18 0.40 0.601 0.82 0.93 0.97 0.010 0.05 ‘I., 0.050 0.17 0.41 0.67 0.85 0.93 0.98 0.010 0.05
0.15 0.15 0.16 0.17
0.32 0.30 0.33 0.37
0.55 0.48 0.54 0.60
0.73 0.64 0.72 0.78
0.80 0.76 0.84 0.90
Dixon’s Ootlicr Model:(Jt D’ 0.050 0.19 0.42 C, 0.050 0.18 0.39 W 0.052 0.19 0.42 T, 0.051 0.20 0.46
Dixon’s Outlier Model:(rt D’ 0.050 0.19 0.43 C, 0.050 0.20 0.47 W 0.052 0 21 0.49 T,, 0.051 0 ?I !J.50
Tiku’s D‘ C, W T,
Outlier 0.039 0.025 0.034 0.051
l)N(O, 1) and lN(O,3) 0.71 0.89 8.98 1.00 0.72 0.88 0.96 0.99 0.76 0.92 0.98 1.00 0.79 0.93 0.99 1.00
liN~(!l, 1) and lN(0, 0.67 O.&S 0.95 0.591 0.74 0.84 0.66 0.82 0.93 0.73 0.91 0.98
ModeP: fk = 1, A = 0.18 0.44 0.73 0.12 0.33 0.6t 0.16 0.41 r.71 0.23 0 58 0.86
0.010 0.010 0.010 0.011
10) 0.99 0.87 0.97 1.00
O.t!lO 0.010 0.010 0.010
0.05 0.05 0.05 0.06
0.17 0.15 0.18 0.19
0.36 0.30 0.35 0.43
0.60 0.43 0.55 0.69
0.80 0.54 0.72 0.88
J.92 0 59 0.81 0.97
0.007 0.004 0.006 O.Oi2
2) 0.91 0.81 0.80 0.97
0.98 0.93 0.!1’8 I.O+J
1.00 0.98 1.00 1.00
0.05 0.02 0.03 0.08
0.17 0.08 0.10 0.27
0.40 0.20 0.25 0.58
0.67 0.38 0.46 0.85
0.89 0.59 0.68 0.97
0.97 0.78 0.85 1.00
Student’s r (d.0.f. 2) D + 0.(?50 0.17 C, 0.050 0.17 w 0.052 0.18 T, 0 051 0.17
0.38 0.36 0.39 0.40
0.61 0.54 0.59 0.62
0.78 0.74 0.79 0.78
0.90 0.83 0.88 0.90
0.95 0.010 0.05 0.90 0.010 004 0.94 0.010 005 0.95 0.008 0.05
0.16 0.14 0.15 0.17
0.34 0.27 0.31 0.34
0.55 0.45 0.53 0.55
0.71 0.57 0.67 0.73
0.63 0.67 0.713 0.84
Cauchv D’ A050 C..14 C, 0.050 Cl.12 W 0.052 0.13 T, 0.049 0.14 c 0.048 0.14
0.28 0.23 0.27 0.27 0.31
0.46 0.36 0.41 0.42 0.50
0.62 0.47 0.54 0.54 0.65
0.73 0.56 0.66 0.67 0.77
O&l 0.66 Ct.75 0.75 0.85
0.10 0.08 0.10 0.08 il.10
0.20 0.15 0.19 0. I8 0.22
0.34 0.23 0.28 0.29 0.36
0.47 0.30 0.38 0.44 0.50
0.514 0.36 O&I8 0.5;4 0.6’4
0.010 0.010 0.010 0.006 0 006
0.04 0.04 0.04 0.03 0.03
Table 8 (continued) Values of the power of the statistics Lb
0”
0.5
Q*, C,, W and T,; sampie sizes (n,, n,) = (8,8)
1
1.5
2
2.5
3
I&G&ma 9+ O.OSO-oA.2 C, 0,050 0.17 W 0.052 OS T‘. 0.049 0.14 Tee 0.050 0.16
0.25 0.38 0.35 0.29 0.35
0.44 0.62 0.59 0.51 0.62
0.66 0,82 0.79 0.72 0.84
0.83 0.94 0,93 0.89 O.%
0.95 0.99 0,98 0.98 1.00
Chi-square (d.o.f, 4) DC O,O!!!O0.17 C, 0.050 0.21 W 0.052 0.21 Tc 0.052 0.19
0.41 0.46 0.46 0.42
0.68 0.71 0.72 0.67
0.87 Lt.87 0.89 0.85
0.96 0.94 0.96 0.95
0.78 0.73 0.75 0.70
0.95 0.91 0.93 0.80
0.99 0.97 0.98 0.97
1.00 0.99 0.99 0.99
0”
OS
2.5
-y--
1
1.5
2
O.OLO 0.03 0.018 0.04 0.010 0.04 0.011 0+04 0.010 0.05
0.07 0.13 0.12 0.10 0.13
0.18 0.30 0.28 0.21 0.31
0.34 0.50 0.48 0.36 0.54
0.99 0.98 0.99 0.98
0.010 0.010 0.018 OX!11
0.05 0.06 0.06 0.05
0.17 0.19 0.20 0.17
0.38 0.42 0.43 0.37
0.64 0.85 0.92 0.63 0.78 0.87 0.67 0.82 0.92 0.62
0.80
0.91
1.00 1.00 1.00 1.00
0.010 O.OlO 0.010 0.007
0.15 0.16 0.16 0.11
0.50 0.44 0.48 0.42
0.80 0.69 0.75 0.72
0.94 0.83 0.89 0.89
0.99 0.90 0.96 0.96
1.00 0.95 0.98 0.99
0.v 0.76 0.73 0.59 0.79
0.76 0.91 0.89 0.79 0.94
Exponential
D+ 0.050 0.37 C, W T,
0.050 0,052 0.047
0.37 0.38 0.32
Chi-square (d.o.f. 1) D+ 0.050 0.45 0.80
0.93 0.98 0.99 1.00 O.OlO 0.21 054 0.76 0.90 0.95 0.98 C, O.OSO 0.37 0.55 0.82 Q.90 0.94 0.97 0.010 0.16 0.35 0.54 0.66 0.76 0.82 W T,
0.052 0.043
0.40 0.28
0.69 0.59
0.86 0.80
0.93 O.9O
0.97 0.95
0.98 0.98
Log-normal (CT= 1) D’ 0.050 0.27 0.65 C, 0.050 0.28 0.56
0.86 0.74
0.94 0.84
0.98 0.91
0.99 0.94
0.010 0.807
0.17 0.10
0.40 0.34
0.62 0.59
0.75 0.76
0.86 0.87
0.91 0.93
0.010 0.010 0.052 0.29 0.58 0.79 ~?.S;J; 0.94 0.97 0.010
0.10 0.09 0.10
0.35 0.27
0.61 0.45
0.80 0.57
0.90 0.67
0.94 0.84
W 0.30 0.51 0.67 0.78 0.8.5 Tc 0.048 0.23 0.49 0.70 0.83 0.!)20.95 LOG9 0.07 0.23 0.46 0.65 0.80 0.86 These probabilities are exact for C, and W, randomized and based on 4000 Monte-Carlo runs for D‘, and simulated from 4000 runs for T,. bin random samp le s of size n from each of the two normal N(0, 1) disrr?wtions, the smallest k and the largest k observations are multiplied by A. for the Cauchy and rI = r, = O for the Uniform. ‘Here rr = [h+ 0.3n,] and r2= [$+ 0.3n.J, ‘%ere d = 0, 0.1, 0.2, 0.3, 0.4, 0.5 and 0.6,
Note that for normal samples E(o,) = CTfor large Al and AZ (Tiku, 1978) and the p~welr.of 7YY is, say for equal fll and 1z2,given by (n, = 11~= n and r,/n = rz,’rt = 4 = 0.1) (5.4)
P{Z 2: Z8 - 0.9836}, since
is ;Ss
nor
(I- A)% point (S is *
138
ML
Valiues of the ymet _.-A 0 0.5 -___11_ t 0.0s 0.13 -L 0.05 0.12 t 0.01 0,03 +L 0.01 0.03
Tikle
of t and Tc 1
1.5
2
2.5
3
3.5
4
0.26 0.25 0.09 0,09
0.44 0.43 0.20 0.20
0.64 0.63 0.33 0.36
0.80 0.39 MT 0.55
0.91 Q.98 0.75 0.n
0.97 0.96 0.88 Q.87
0.99 0.99 0.95 O.95
Thle power of the two-sample Student’s t-statistic for large normal samples is given by (nl = n2==nj (5.5)
P(Z 1: &j - ik}.
The values of the power of t and TC(for large normal samples) can, therefore! be calculated from (5.5) and (54) and we given in Table 9. It is clear that for normal samples the loss of power resulting from the use of ?‘;:in place of the UMP Student’s; f-statistic is not too serious. Nonparametric statistics are often used to test the null hypotiiesis Ho* partScularly if one suspects the validity of the normality assumptilon, and these are known to be generally more powerful than t for long-tailed symmetric disttibutions. ‘the most ptGT:.+Ient nonparametric statistics are the Wilcoxon s:,‘atisttc (5.6)
The Terry-Hoeffding
normal-score statistic (5.7)
and the Smirnov-Kolmogorov-Massey D’=
max
1.2 . . . . . n
[n*(
l-
El)-
statistic y
fl,ti] I
(J.8)
see Milton (1970, pp. 23, 2S and 27). A vast literature discusses the distributions and power properties of nonparametric statistics; se for example Lehmann (!9?5), Hotz (1964), Ramsey (1971) and Gastwirth (1965). Note that these statistics have remarkably high power, even for small samples (Table 8); see also Lehmann (197% and ivflilton (1970). Tables of the cumulative probabilities of the nulli distributions of ‘Jv, C1 and D+ are given by Wilcoxon, Katti and Wilcox (19731, KBotz (1964) and Kim and Jennrich (1973), respectively. Using these tdfks, we simulated (from 4000 Monte-Carlo runs) the values of the power of W, lo against I., for sample sizes (n,, 81~2 = (8,
(10,8) and (10,lQ). The values for D” were obtained through randomizF!ion betwee% the two appropriate cumulative probabilities (Kim and Jenrich, 1973). The statistic ‘& was found to be generally more pow&‘ul than these nonparametric statistics, except for some very short-tailed symmetric distributions (Uniform,
for example) and som: extremely skewed disttibutions (xt and Log-normal, !!or example) in which cazie theXatistics C1 and D’, respectively, were found to be considerably more powerful than T,; see Table 8. It is interesting to rote that with Tiku’s (1975, 1977) outlier model preserving symmetry, that is, the k smallest and the k largest observations of a random sample from the N(O, a) distribution are multiplied by a positive constant A (or a positive constant h is subtracted from and added to the smallest k and the largest k observations, respectively, of a random sample from the [email protected])), the sign% cance levels of the statistics W, C1 and Di- (but not 7’=)get substantially distorted from their preassigned values; see Table 8. For large samples the powe:r of T= is, say for
n1 = n2= n, given
by
((4= r&t = r&2)
the assumption is that the common pdf g satisfies the very general regularity conditions listed by Shorack (1974, p. 664): A and 2 are as in (5.4). The corresponding large sample power of W is given by P{ Z
2
Zs - af*(O)A},
(5’0)
where f*(O) is the density at zero of the difference u = zI - z2 of tw63independent random variables, each having the pdf g(z), z = (X- p)/o; see Lehmann (1975, pa 72). For the family (3.1), (5.9) is easy to work out from the equation (3.Q since
z”f(z) dz, and i?;l_r and - 5+1 tend to 6 as n tends to infinity, where S!&f(z) dz =
1- q;
ku (1967). Again, f “(0)iseasy toworkout for the family (3.1), de:;pite the fact that g(u) cannot be expressed in a closed form. The Eq. (5.9) alwiays assumes values larger Than (5.10), even in case Df the normal distribution (p = QOin (3.1)). For the normah (p = m), (5.9) reduces to (5.4), and (5.10) reduces to P{Z 2 Z6 - 0.9776). Incidentally, note that for large normal samples, the power of the two-sample statistic fTti;,,based on the estimators PT and 0-r is given by P(Z2:Zs -0.97 “A}. shidn be genernlized to test that The statistic Tc can in a straightforwartd e generalized statistic is given by k( > 2) symmetric distributions are identical
see Harter and Moore (1966) and also
where
The statistic T’$ has pQwet properties similar to TQ.We, however, omit details flax conciseness, IVote that the asympcXic (4 + m) nult distributian of ?‘$ is chin square having R - 1 d.o.f., and for small samples (q ;;r5, i = ‘1,2, . . . , k) the upper percentage points of ‘I’$ ate closely approximated by the corresponding uppet percentage points of the F-distribution having (k - I, u) d,o.f., t, = EFmz(Ar - I); see also Tiku (1978). Example relief of observed the other
(Lehmann, 1975, p. 37). In a comparison of two drugs used for the postoperative pain, the following nuanbers of hours of relief were for 16 paticms, of which 8 had been assigned to a standard drug A and 8 to an experimental drug B, A: B:
6.8 3.l 5.8 4s 3.3 4.7 4.2 4.9 4.4 2.5 2.8 2.1 6.6 0.0 4.8 2.3.
One wants to test whether there are any differences at all between the two drugs. From the following rankings of A and B A: B:
6 1
7 2
8 3
10 4
I1 5
13 9
14 12
16 15,
Lehmann calculates the values of the two-sided Smitnov-Kolmogorov-Massey statistic D = 5/8, and P(D 2 S/S) = 0.087. Here its = ot2= n = 8, and since the data clearly do not belong to “disaster” distributions, we take rl = rz = r”= #+O. l(S)] = 1 in (5.1) and (5.2). We ignore the smallest and the largest observations, i.e. 3.1 and 6.8, in the sample A, and similarly for the sample B, and calculate TC from the Eq. (5.3). Note that = 0.7113 and & = & = fi := 6.8281 (from Eq. (2.6)). The value of TC @l =a2=a works out to be 1.84, and P(l’&l~ 1.84) = 0.094 (8.o.f. 10). This is close to the probability based on the statistic D. Note that to compute D, ali the sixteen observations in the two samples have to be ordered to determine their ranks, but to calculate T’: rrot even the eight observations in either of the two samples have to be completely ordered. Qne simply has to locate the largesi and crllallest observations in each of lthe two samples.
To use the estimators hC and gC in ihe robustness framework and to achieve high e%ciency, one has to know whether the population has a short-tailed distribution case one chooses g = r/n = 0, a long-tailed distribution with existent deviation in w ic’Rcase one c .I, or a ““disaster”
Robusmcss a/‘MML estimaators
141
distribution in which case one chooses q = 0.3. In practice, there might be sirtuations when this knowledge is not ak all available. In such situations, one might uw t’& adaptive estimators JA and q$ defined to be exactly the Same as p, and q., respectively, bvt. _tith the constant fraction q = r/o replaced by the random faction i* A‘P/n, where t* = 0 if nz”/n = 0, I* = [i-k Q.ln] if m *In s 0.1, and +0,3n] if tn*/n = 0.1; m* is the number of values of l%il=/JQ- median(x,
.48 medianlxi- median(x,)j
andi=l,2,.,., n, which exceeds 3.0. The adaptive estimators g: and crx are on the whole as efficient as the adaptive estimators w24 and BS82. For example for the distributions (1) Uniform; (2) Normal; (3) f(p; X) with p = 5; (4) f(p; X) with p = 2.5; (5) (n - 5)N(O, 1) and 5N(O, 3); (6) Student’s t2; (7) Cauchy; and (8) Normal/Uniform; we have the following values (based on 10 000 runs, see Table 10); n = 20 (o= 1): Table 10 (1)
(21
(3)
(4)
(5)
(6)
(7)
(8)
0.96
0.91
0.81
1.29
1.31
1.60
2.36
Mean
a,*
0.99
MSE
&! a:
0.053 0.051 0.049 0.040 0.109 0.12 1 0.222 0.013 0.035 0.048 0.078 0.193 0.278 0.880
0.444 2.72
Def:
(p:, cr?, BS82
0.066 0.086 0.097 0.118 0.302 0.399 1.10 0.036 0.053 0.089 0.103 0.257 0.379 1.25
3.16/5.328 3.38/5.647;
In Table la, 5.328 and 5.649 are the sums i?tfthe deficiencies of the two pairs (& 0,“) and BS82 for the eight distributions (1) to (8), respectively. Like the Stddentized statistics based on ~24 and BS82, however, the small sample ml1 distribution of the Studentized statistic ~T&?/G:, m* being the value of nt obtained by replacing r by t* in (2.6) to (2.8), is difficult to determine and Studznt’s t-approximation does not provide accurate values for the percentage points, especially for “disaster” situations; see also Gross (1976, Table 6). This is seen as a drawback in these adaptive estimators.
My sincere thanks are due to the referees for many helpful comments and suggestions. Thanks are also due to NS 6 of Canada and Research Board fur res script.
142
ML.
zxu
Andrews, D., P.J. Bickel, F. Hampel, P. Huber, W. Rogers and J. I cri ey, wzj. &b~~t E~tim~b~ of Lwatiori : Su~ey anfi Adtmces. Princeto), University Press, Princeton. Bickel, p.J. (1976). Another look at Robustness: a review of reviews and SOrIle new de&l~~m&. ~. 3 Stand. J. Statist. 3, 145-168. BOX, G.E.P. and M.E. Muller, {1958), A note on the generation of random normal deviates. Ann. Mid. Stat&. 29, 610-611. Cramer. I-I. (1945). Mathernarical Methods of Statistics. Princeton University Press, Princeton. David, H.A. (1970). Order Statistics. Wiley, New York. Gastwirth, J.L. (1965). Percentile modification of two sample rank tests. .I. Amer. Stat. Assoc. 60, 1127-l 141. Gross, A.M. (1976). Confidence interval robustness with long-taibd symmetric distributions. J. Amer. Statist. Assoc. 71, 409-416. Harter. H.L. and A.M. Moore (1966). Iterative maximum likelihoud estimation of the parameters of normal populations from singly and doubly censored samples Biometika 53, 205-213. Huber, P.J. (1970). Studentizing robust estimates. In: M.L. Puri, eds., Nonprrrametric Techniques in Statistical lnfercrtce. Cambridge LJniversity Press, Cambridge, 453-463. Huber, P.J. ( B972). Robust statistics: a review. Ann. Muth. Stat. 43, 1041-1067. Holland. P.W. and R.E. Welsch (1977). Robust regression using iteratively reweighted least-squares. Comntun. S&r. A6 (9). 813-827. Kim, P.J. and RI. Jennrich (1973). Tables of the exact sampling distribution of two-sample Kolmogorov-Smirnov criteron D”,,,(rrt d n). in: H.L. Harter and D.B. Gwen, eds., S&cred Tables in Mathematicd Statistics, 2. American Mathematical Society, Providence, 79-170. Klotz, J. (1964). On the normal scores two-sample test. J. Amer. Statist. Assoc. 59, 652-664. Lehmann, E.L. (1975). Nonparametrics: Statistical Methods Based on Ranks. McGraw Hill, New York. Milton, R.C. (1970). Rmk Order Probabilities. Wiley, New York. Ramsey, F.L. (1971). Small sample power function for nonparametric tests of location in the double exponential family. J. Amer. Statr.\i. Assoc. 66, 149-151. Shorack. G. R. (1974). Random means. Anti. Statist. 2, 661-635. Smith. W.B., C.D. Zeis and G.W. Syler (1973). Three parameter lognormal estimation from censored data. J. lndian Starts. Assoc. 11, 15-31. Stigler. S.M. ( 1974). Linear functions of order statistics with smooth weight functions. Ann. Statist. 2, 676-699. Stigler, S-M. (1977). Do robust estimators work with real data? Am. Statist. 5, 1005-1098. Tiku, .M.L. (1967). Estimating the mean and standard deviation from censored normal samples, Biometrika 54, 155-165. Tiku, M.L. (1968). Estimating the parameters of log-normal distribution lrom censored samples. J. Amer. Statist. Assoc. 63, 134-140. Tiku, M.L. (1970). Monte-Carlo study of some simple estimators in censored normal samples. Biometika 57, 207-211. Tiku, M.L. (1973). Testing group effects from Type II censored normal samples in experimental design. Biontetrics 29, 25-33. Xku, M.L. (1974a). A new statistic for testing for normality. Common tiutist. 3 (3), 223-232. Tiku, M.L. ( 1974b). Testing normality and exponentiality in multiqgsamplesituations. Cornmun. Statist. 3 (8). 777-794. Tiku, M.L. (1975). A new statistic for testing suspected outliers. C%mmun. Statist, 4 (8). 737-752. ‘T&u, M.L. (1077). Rejoirler: “Comment on ‘A new statistic for testing suspected outliers”‘. C~m*nun. :;tatist. A6 (14), 141?- 1422. Tiku, ML. (1978). Linear regression model with censored observations. Contmun. Stutist. A7 (13), 1219-1232. ‘Xku. M.L. and P.W. Jones (1971). Best linear unbiased estimators for a distribution similar to the logistic. In: C.S. Carter, T.D. Dwividi, aser, J.R. McGregor and D.A. Sprott, of the “Stfftistics 71 Cawda”, 412-419.
Robustness of A4MLestimators
143
Tiku, ML. and D. Stewart (1977). Rstimattig and testing group effects from Type I censored normal samples in experimental design. Commun. Statist. A6 (15), i46-1501. Tiku, ML. and S. Kumra (1978). Eq~cted values and variarcrs and covariances of order statistics for a far&y ,of symmetrical distributions. Selected Tables in Mathematical Statistics. Submitted. Wegman, EJ, and R.J, CarroE (1977). A Monte-Carlo study of robust estimators of location. Cornmutt. Statist. Ad (9), 795-812. Wilcoxon, F, S,K. Katti and R.A. Wilcox (1973). Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test. In: H.L. Master and D.B. Chven, eds., Sehed T&es in mattkamatical Statistics, I. American Mathematical Society, Providence, 171-259.