- Email: [email protected]
Locally most powerful test for testing the equality of variances of two linear models with common regression parameters
Statistics & Probability North-Holland
February
Letters 11 (1991) 149-153
1991
Locally most powerful test for testing the equality of variances of two linear models with common regression parameters Manzoor
Ahmad
University of @_&bee, MontrCal, Quebec, Canada H3C 3P8
Yogendra Department
P. Chaubey of h-fathematm
and Statistics,
Concordia University,
7141 Sherbrooke
St West, Mont&al,
Quebec, Canada H4B lR6
Received April 1989 Revised February 1990
Abstract: In this paper, the problem of testing the equality of two homoscedastic normal linear models with common regression parameters is considered. A locally most powerful test which is invariant with respect to the group of location and scale transformations of the observations is derived. The test statistic when simplified reduces to the ASR test statistic proposed and studied by Chaubey (1981). The robustness of this test is further explored. Keywords:
Heteroscedasticity,
LMP test, elliptically
symmetric
distributions.
ASR test
1. Introduction Consider
two linear
models
x=X,/3+&,,
given by i=l,2,
(1.1)
vector, X, is an n, X k known non-stochastic matrix with r( X,) = k parameter, and E,, the n, X 1 disturbance vector, is normally distributed with zero mean and dispersion matrix g(~,) = u,~Z~,, and Ed and ~~ are independently distributed. Such models have been extensively studied under the paradigm of Bayesian analysis (see Box and Tiao (1973, Chap. 9)). We are interested in testing H, : uf = u2’. This framework was considered by Chaubey (1981) who proposed and stadied a test based on the ratio of sums of squares of ordinary least squares residuals. This test was called the ASR test. The test was shown to have a monotone power function for two special cases; (i) the case when X, = X2, and (ii) the case when X, is simply a column of ones, a case considered by Geisser (1965). The power functions were numerically compared for the latter case with some alternative tests and a suggestion was made that the ASR test seemed to be locally powerful. No optimal tests for this problem have been derived in the literature. The problem may be considered arising out of testing the heteroscedasticity of a linear model where the regression model is decomposed where
r; is an n, x 1 observation
(n, 2 k), ,8 is a k X 1 regression
This work was partially 0167-7152/91/$03.50
supported
by NSERC
Grants
Nos. A3661 and A3450
0 1991 - Elsevier Science Publishers
B.V. (North-Holland)
149
STATISTICS & PROBABILITY
Volume 11, Number 2
February 1991
LETTERS
into two linear models as in (1.1) according to some criterion. Harrison and McCabe (1979) have proposed a test which is similar to the ASR test using this approach for testing the heteroscedasticity of a linear model. The problem in this form has attracted a lot of research but the optimal solution is yet to be obtained. In this paper, however, we demonstrate the local optimality of the ASR test for the problem stated above. It may be remarked here that the UMP test does not exist here because of the non-existence of a complete sufficient statistic. In Section 2, the locally most powerful invariant test is derived and it is observed to be the ASR test. Section 3 explores its robustness properties.
2. Locally optimal test for H, We write model (1.1) in a compact
form as
(2.1)
Y=Xp+&, where X’=
Y’ = (Y,‘, r;>,
(X,‘,
X2/),
E’ = (E;,
E;),
with
Let e = QY, where Q = Z - X( X’X))‘X’, denote the ordinary least square residuals. The ordinary least square residual vectors for the two models are given as e, = Q,,F + QlzY2 and e2 = Q2,G + Q2*yZ, where Q,, (i, j = 1, 2) is obtained by partitioning Q such that Q,, is an n, X n, matrix. Specifically, Q;, = Z - P,,, where P,, = X,(X/X)-IX,‘. Now consider the group G = (SC.,) where g,.,(Y) = c(Y - Xy), with c a positive constant and y a k x 1 vector. The problem of testing H, is invariant under G. Now we are ready to state the following theorem. Theorem 2.1. The locally most powerful invariant aguinst H, : 6’ > 1 depends on the statistic T = eie,/(
test (under the group of location and scale changes)
e;el + e;e2),
which rejects H, in f&our
for Ha
(2.2)
of H, if it is small.
Proof. Since the distribution of any invariant test statistic depends only 8, the parameter maximal invariant, we may assume without loss of generality that p = 0, uf = 8, and cr,’ = 1. A left invariant theorem, the measure on the group G = lF4+X [wk is ck-i dy dc. Using Wijsman’s (1967) representation ratio R of the non-null distribution to the null distribution of a maximal invariant is given by zc
0-42
n+kpl
/I0
R=
19)( Y - Xy)/20}
=
Z
[
(;’
dy dc (2.3)
cntk-’ Rk
where
150
Y - Xy)‘D(
10 //0
D(e)
exp{ -c’(
2
I
eF n2
exp{ -c’(Y-Xy)‘(Y-Xy)/2}
dydc
Volume
11, Number
2
STATISTICS
& PROBABILITY
Note that for any vector b by usual least squares
LETTERS
February
1991
results we have
(Y-Xh)‘D(B)(Y-Xb)=U(B)-t(?-h)‘X’D(B)X(?-h),
(2.4)
where p=
(X’D(B)X)_‘X’D(B)Y
and
Hence, the exponential term in the numerator Now, using the standard multivariate normal r( n/2)2 The denominator I?n/2)2
V(O) = (Y-
Xq)‘D(B)(Y-
XT).
of R becomes (- c*U( 6) - c*( 7 - y)‘X’D( 6)X( 7 - y))/28. and gamma integrals we can write the numerator of (2.3) as
(n/2)-1(21T)~/2~(n2+k)/*1X~~(e)XI-”2(~(e))-n/2,
of (2.3) can similarily (“/2)w(271)w2)
be written
as
, ,y’x,-~/y&~“/*
and therefore any invariant test depends only on the ratio T, = U(O)/(e’e). It is clear from the above exposition that no UMPI test for this problem exists. To get the LB1 test we expand r, in the powers of (0 - 1) and consider the linear term as the test statistic. Towards this goal we write p as ~=(x~x+(e-1)x;x,)~‘(x’~+(e-1)~;~,) and use the fact that (A + qB)-’
=A-’
- qA-‘BA-’
+ o(q) to get
(P-P)=(e-1)(xX-‘~;~,+~(e-i), where p^= (X’X))‘X’Y.
(2.5)
Using (2.4) again we have
e’e+(e-l)e;e,=(Y-xp^)‘o(e)(Y-xp^)=U(e)+(P^-~)’x~o(e)x(P^-P), and hence using (2.5) we get e;e2 - (e - 1)2e;X2( X’X)p’X’D(r3)X(
u(B)=e’e+(8-1) =e’e+(&l)e;e,-(e-1)
X’.X)-‘X&,
+
.((e
- I)‘)
2e~X2(X’X)-‘X,‘e2+o((e-1)2).
Therefore, T,=l-t(8-1)
e;e,/e’e+o(O-l)=l+(e-l)T+0(e-1).
Since the ratio of densities is a monotonic 0 values of T and the result follows. Corollary
function
of T,-‘, the LB1 test for H, vs. H, rejects H, for small
2.2. The locally optimal test derived above is equivalent
Proof. The ASR test statistic is given by T, = e;e,/e;e,. the statement is obvious. q
3. Robustness
to the ASR
test.
Since T, = (1 + T,)-’
which is monotone
in T2,
of the ASR test
Now we shall briefly discuss the robustness of Y = (Y,‘, Y,‘)’ is given by an elliptically ~(KP,
of the above test when the probability symmetric density
e)=e-~~‘24(o;-X~)‘~(e)(Y-Xp)/e},
density
function
(pdf)
(3.1) 151
Volume
11. Number
STATISTICS
2
where q( .) satisfies
s R”
& PROBABILITY
February
LETTERS
1991
the condition
q(u’u)
du=l.
(3.2)
Note that q( .) need not be exponential so that the joint distribution of Y, and Y, is more general than normal. For testing H, : 0 = 1 vs. H, : 6 > 1, note that this problem is invariant under group G as has been pointed out above. Proceeding in the same manner as in the previous section, the ratio of non-null to null distributions of a maximal invariant under group G (taking p = 0 again without loss of generality) is given by ra
“1’2cn+k-‘q{c2(y-Xy)‘D(B)(y-X-y)/B}
/
lJ0
dy dc (3.3)
cc
cnfkp’q{c2(y-Xy)‘(y-Xy)} JJ0
R”
q( .) by using
Simplifying the argument of the function numerator of the above ratio as
where q*(x)
@/‘I X’D( 6) XI ~“2~mP’1/2c”-1q*
(c*U(
= lw”q( u’u + x) du. The above
expression
@n,+k)‘21 x’o(e)xl Analogously,
dydc
the denominator
(2.4) as in the previous
6’)/6’)
further
XY)E~“~)
(2)
T(aY)
= T(Y)
simplifies
to
du
of the ratio in (3.3) is given by dw.
The above shows that the LB1 test is the same as the test derived test statistic7J Y), say-satisfies the following properties: T((Y-
we can write the
dc,
~“2(u(e))-“~2Jo’q*(w~)w”~~
IX’X/~“2(e’e)~“/2~aq*(UZ)U”~’
(1)
section
= T(Y)
under
the normal
model.
Moreover,
this
V y E R and VE p.d.,
v’a > 0.
Hence, the test derived here is null robust (see Kariya (1981)). Also, since the above test does not depend on q(e), any null robust invariant test is non-null robust. Furthermore, the optimum test based on the statistic derived above is optimality robust since it is independent of q( .). The null robustness of this test makes it a proper candidate for testing heteroscedasticity as it was used in Harrison and McCabe (1979).
Acknowledgement
The authors are grateful to B.K. Sinha for his comments. helped to improve the presentation of the paper.
The comments
of an anonymous
referee
also
References Box, G.E.P. and G.C. Tim (1973), Bayemm Inference trccrl Analysrs (Addison-Wesley, Reading, MA). 152
m Statm
Chaubey, Y.P. (1981), Testing the equality of variances linear models, Canad. J. Statist. 9, 119-127.
of two
Volume
11. Number
2
STATISTICS
& PROBABILITY
Geisser, S. (1965), A Bayes approach for combining correlated estimates, J. Amer. Statist. Assoc. 60, 602. Harrison, M.J. and B.P.M. McCabe (1979). A test of heteroscedasticity on ordinary least square residuals, J. Amer. Statist. Assoc. 74, 494-499. Kariya, T. (1981), Robustness of multivariate tests, Ann. Statist. 9, 1267-1275.
LETTERS
Wijsman, R.A. (1967). Cross-sections cation to densities of maximal Berkeley Symp. on Mathematwal Vol. I (University of California 400.
February
1991
of orbits and their appliinvariants, in: Proc. 5th St&stirs and Probabrlq, Press, Berkeley) pp. 389-
153
February
Letters 11 (1991) 149-153
1991
Locally most powerful test for testing the equality of variances of two linear models with common regression parameters Manzoor
Ahmad
University of @_&bee, MontrCal, Quebec, Canada H3C 3P8
Yogendra Department
P. Chaubey of h-fathematm
and Statistics,
Concordia University,
7141 Sherbrooke
St West, Mont&al,
Quebec, Canada H4B lR6
Received April 1989 Revised February 1990
Abstract: In this paper, the problem of testing the equality of two homoscedastic normal linear models with common regression parameters is considered. A locally most powerful test which is invariant with respect to the group of location and scale transformations of the observations is derived. The test statistic when simplified reduces to the ASR test statistic proposed and studied by Chaubey (1981). The robustness of this test is further explored. Keywords:
Heteroscedasticity,
LMP test, elliptically
symmetric
distributions.
ASR test
1. Introduction Consider
two linear
models
x=X,/3+&,,
given by i=l,2,
(1.1)
vector, X, is an n, X k known non-stochastic matrix with r( X,) = k parameter, and E,, the n, X 1 disturbance vector, is normally distributed with zero mean and dispersion matrix g(~,) = u,~Z~,, and Ed and ~~ are independently distributed. Such models have been extensively studied under the paradigm of Bayesian analysis (see Box and Tiao (1973, Chap. 9)). We are interested in testing H, : uf = u2’. This framework was considered by Chaubey (1981) who proposed and stadied a test based on the ratio of sums of squares of ordinary least squares residuals. This test was called the ASR test. The test was shown to have a monotone power function for two special cases; (i) the case when X, = X2, and (ii) the case when X, is simply a column of ones, a case considered by Geisser (1965). The power functions were numerically compared for the latter case with some alternative tests and a suggestion was made that the ASR test seemed to be locally powerful. No optimal tests for this problem have been derived in the literature. The problem may be considered arising out of testing the heteroscedasticity of a linear model where the regression model is decomposed where
r; is an n, x 1 observation
(n, 2 k), ,8 is a k X 1 regression
This work was partially 0167-7152/91/$03.50
supported
by NSERC
Grants
Nos. A3661 and A3450
0 1991 - Elsevier Science Publishers
B.V. (North-Holland)
149
STATISTICS & PROBABILITY
Volume 11, Number 2
February 1991
LETTERS
into two linear models as in (1.1) according to some criterion. Harrison and McCabe (1979) have proposed a test which is similar to the ASR test using this approach for testing the heteroscedasticity of a linear model. The problem in this form has attracted a lot of research but the optimal solution is yet to be obtained. In this paper, however, we demonstrate the local optimality of the ASR test for the problem stated above. It may be remarked here that the UMP test does not exist here because of the non-existence of a complete sufficient statistic. In Section 2, the locally most powerful invariant test is derived and it is observed to be the ASR test. Section 3 explores its robustness properties.
2. Locally optimal test for H, We write model (1.1) in a compact
form as
(2.1)
Y=Xp+&, where X’=
Y’ = (Y,‘, r;>,
(X,‘,
X2/),
E’ = (E;,
E;),
with
Let e = QY, where Q = Z - X( X’X))‘X’, denote the ordinary least square residuals. The ordinary least square residual vectors for the two models are given as e, = Q,,F + QlzY2 and e2 = Q2,G + Q2*yZ, where Q,, (i, j = 1, 2) is obtained by partitioning Q such that Q,, is an n, X n, matrix. Specifically, Q;, = Z - P,,, where P,, = X,(X/X)-IX,‘. Now consider the group G = (SC.,) where g,.,(Y) = c(Y - Xy), with c a positive constant and y a k x 1 vector. The problem of testing H, is invariant under G. Now we are ready to state the following theorem. Theorem 2.1. The locally most powerful invariant aguinst H, : 6’ > 1 depends on the statistic T = eie,/(
test (under the group of location and scale changes)
e;el + e;e2),
which rejects H, in f&our
for Ha
(2.2)
of H, if it is small.
Proof. Since the distribution of any invariant test statistic depends only 8, the parameter maximal invariant, we may assume without loss of generality that p = 0, uf = 8, and cr,’ = 1. A left invariant theorem, the measure on the group G = lF4+X [wk is ck-i dy dc. Using Wijsman’s (1967) representation ratio R of the non-null distribution to the null distribution of a maximal invariant is given by zc
0-42
n+kpl
/I0
R=
19)( Y - Xy)/20}
=
Z
[
(;’
dy dc (2.3)
cntk-’ Rk
where
150
Y - Xy)‘D(
10 //0
D(e)
exp{ -c’(
2
I
eF n2
exp{ -c’(Y-Xy)‘(Y-Xy)/2}
dydc
Volume
11, Number
2
STATISTICS
& PROBABILITY
Note that for any vector b by usual least squares
LETTERS
February
1991
results we have
(Y-Xh)‘D(B)(Y-Xb)=U(B)-t(?-h)‘X’D(B)X(?-h),
(2.4)
where p=
(X’D(B)X)_‘X’D(B)Y
and
Hence, the exponential term in the numerator Now, using the standard multivariate normal r( n/2)2 The denominator I?n/2)2
V(O) = (Y-
Xq)‘D(B)(Y-
XT).
of R becomes (- c*U( 6) - c*( 7 - y)‘X’D( 6)X( 7 - y))/28. and gamma integrals we can write the numerator of (2.3) as
(n/2)-1(21T)~/2~(n2+k)/*1X~~(e)XI-”2(~(e))-n/2,
of (2.3) can similarily (“/2)w(271)w2)
be written
as
, ,y’x,-~/y&~“/*
and therefore any invariant test depends only on the ratio T, = U(O)/(e’e). It is clear from the above exposition that no UMPI test for this problem exists. To get the LB1 test we expand r, in the powers of (0 - 1) and consider the linear term as the test statistic. Towards this goal we write p as ~=(x~x+(e-1)x;x,)~‘(x’~+(e-1)~;~,) and use the fact that (A + qB)-’
=A-’
- qA-‘BA-’
+ o(q) to get
(P-P)=(e-1)(xX-‘~;~,+~(e-i), where p^= (X’X))‘X’Y.
(2.5)
Using (2.4) again we have
e’e+(e-l)e;e,=(Y-xp^)‘o(e)(Y-xp^)=U(e)+(P^-~)’x~o(e)x(P^-P), and hence using (2.5) we get e;e2 - (e - 1)2e;X2( X’X)p’X’D(r3)X(
u(B)=e’e+(8-1) =e’e+(&l)e;e,-(e-1)
X’.X)-‘X&,
+
.((e
- I)‘)
2e~X2(X’X)-‘X,‘e2+o((e-1)2).
Therefore, T,=l-t(8-1)
e;e,/e’e+o(O-l)=l+(e-l)T+0(e-1).
Since the ratio of densities is a monotonic 0 values of T and the result follows. Corollary
function
of T,-‘, the LB1 test for H, vs. H, rejects H, for small
2.2. The locally optimal test derived above is equivalent
Proof. The ASR test statistic is given by T, = e;e,/e;e,. the statement is obvious. q
3. Robustness
to the ASR
test.
Since T, = (1 + T,)-’
which is monotone
in T2,
of the ASR test
Now we shall briefly discuss the robustness of Y = (Y,‘, Y,‘)’ is given by an elliptically ~(KP,
of the above test when the probability symmetric density
e)=e-~~‘24(o;-X~)‘~(e)(Y-Xp)/e},
density
function
(pdf)
(3.1) 151
Volume
11. Number
STATISTICS
2
where q( .) satisfies
s R”
& PROBABILITY
February
LETTERS
1991
the condition
q(u’u)
du=l.
(3.2)
Note that q( .) need not be exponential so that the joint distribution of Y, and Y, is more general than normal. For testing H, : 0 = 1 vs. H, : 6 > 1, note that this problem is invariant under group G as has been pointed out above. Proceeding in the same manner as in the previous section, the ratio of non-null to null distributions of a maximal invariant under group G (taking p = 0 again without loss of generality) is given by ra
“1’2cn+k-‘q{c2(y-Xy)‘D(B)(y-X-y)/B}
/
lJ0
dy dc (3.3)
cc
cnfkp’q{c2(y-Xy)‘(y-Xy)} JJ0
R”
q( .) by using
Simplifying the argument of the function numerator of the above ratio as
where q*(x)
@/‘I X’D( 6) XI ~“2~mP’1/2c”-1q*
(c*U(
= lw”q( u’u + x) du. The above
expression
@n,+k)‘21 x’o(e)xl Analogously,
dydc
the denominator
(2.4) as in the previous
6’)/6’)
further
XY)E~“~)
(2)
T(aY)
= T(Y)
simplifies
to
du
of the ratio in (3.3) is given by dw.
The above shows that the LB1 test is the same as the test derived test statistic7J Y), say-satisfies the following properties: T((Y-
we can write the
dc,
~“2(u(e))-“~2Jo’q*(w~)w”~~
IX’X/~“2(e’e)~“/2~aq*(UZ)U”~’
(1)
section
= T(Y)
under
the normal
model.
Moreover,
this
V y E R and VE p.d.,
v’a > 0.
Hence, the test derived here is null robust (see Kariya (1981)). Also, since the above test does not depend on q(e), any null robust invariant test is non-null robust. Furthermore, the optimum test based on the statistic derived above is optimality robust since it is independent of q( .). The null robustness of this test makes it a proper candidate for testing heteroscedasticity as it was used in Harrison and McCabe (1979).
Acknowledgement
The authors are grateful to B.K. Sinha for his comments. helped to improve the presentation of the paper.
The comments
of an anonymous
referee
also
References Box, G.E.P. and G.C. Tim (1973), Bayemm Inference trccrl Analysrs (Addison-Wesley, Reading, MA). 152
m Statm
Chaubey, Y.P. (1981), Testing the equality of variances linear models, Canad. J. Statist. 9, 119-127.
of two
Volume
11. Number
2
STATISTICS
& PROBABILITY
Geisser, S. (1965), A Bayes approach for combining correlated estimates, J. Amer. Statist. Assoc. 60, 602. Harrison, M.J. and B.P.M. McCabe (1979). A test of heteroscedasticity on ordinary least square residuals, J. Amer. Statist. Assoc. 74, 494-499. Kariya, T. (1981), Robustness of multivariate tests, Ann. Statist. 9, 1267-1275.
LETTERS
Wijsman, R.A. (1967). Cross-sections cation to densities of maximal Berkeley Symp. on Mathematwal Vol. I (University of California 400.
February
1991
of orbits and their appliinvariants, in: Proc. 5th St&stirs and Probabrlq, Press, Berkeley) pp. 389-
153