- Email: [email protected]
Wald, likelihood ratio, and infinite induced test statistics for joint one-sided hypothesis
261
Economics Letters 21 (1986) 261-264 North-Holland
WALD, LIKELIHOOD RATIO, AND INFINITE INDUCED FOR JOINT ONE-SIDED HYPOTHESES
TEST STATISTICS
Alan J. ROGERS Indiana Uniuersity, Bloomington, IN 47405, USA Received
24 February
1986
The relationship between certain infinite induced test statistics and the Wald and likelihood ratio statistics for testing hypotheses with one-sided alternatives is considered. In addition, it is shown that one of these infinite induced statistics reduces to a simple finite induced test statistic when the Wald statistic is equal to zero.
1. Introduction In view of the frequency with which prior information on the signs of parameters in economic models is present, it is important that methods of testing hypotheses with one-sided alternatives be available. Examples include sign restrictions on the coefficients of linear models and non-negativity restrictions on variances in error components and random coefficients models. This paper has two purposes. The first is to show that the Wald (W) and likelihood ratio (LR) statistics for testing a joint one-sided hypothesis are equal to certain ‘infinite induced’ test statistics, where the latter are simple modifications of the statistic introduced by Scheffe (1953) for classical two-sided problems. [A good discussion of finite and infinite induced tests for two-sided problems appears in Savin (1984).] This result is helpful in interpreting the results on the W and LR statistic obtained by Gourieroux, Holly and Montfort (1980,1982) and others, and it also provides an interesting parallel to the equivalence between the For X2 statistics and Scheffe’s (1953) statistic for problems with two-sided alternatives. The second purpose is to show that one of the infinite induced test statistics considered can be used to construct a test of arbitrarily high size. In this respect this statistic differs from the W and LR statistics, since the latter are equal to zero with positive probability, and as a consequence they do not embody potentially useful information. This feature is of particular importance in cases where one is especially concerned that the power of the test be high. In addition, a rather remarkable feature of this infinite induced test statistic is that when it is not equal to the W and LR statistics it reduces to a simple finite induced test statistic. For ease of exposition we will consider the case of inequality constraints on linear functions of the coefficient vector of a normal linear regression model. This allows us to make use of some results due to Gourieroux, Holly and Montfort (1982). Asymptotic parallels of these results and those derived below can be obtained under suitable regularity conditions, see, e.g., Gourieroux, Holly and Montfort (1980), and also Rogers (n.d.). We suppose that the null hypothesis is H, : p = 0 and the alternative is H, : p > 0 where p = R/3, ,8 is the coefficient vector in the model y = Xp + U. The dimension of 1-1is q x 1, and p > 0 means 0165-1765/86/$3.50
0 1986, Elsevier Science Publishers
B.V. (North-Holland)
262
A.J. Rogers / Testing hypotheses with one-sided alternatiues
that all elements of p are non-negative and at least one is positive. We assume for simplicity that u is distributed as N(0, I), where I is the identity matrix. We let I; = Rp, where b is the least squares estimator of /3, and ii = Rp, where 6 is the constrained least squares estimator obtained subject to the set of inequality restrictions R/3 2 0. (Throughout we use 2 and s to denote weak element-byelement vector inequalities.) Under Ha, fi is distributed as N(0, A), where A = R( X’X)-‘R’. The W and LR statistics for testing H, against H, are both equal to
and, under H,, w is distributed as a mixture of qx2 distributions and a distribution unit mass to the point zero [see Gourieroux, Holly and Monfort (1982)].
which assigns
2. The W statistic and two infinite induced test statistics To see how the W statistic, w, can be motivated in terms of an infinite induced test, we note that H, specifies a’~ = 0 for all vectors a # 0. Since b 2 0, one intuitively appealing way to test H, against H, is to accept H, if a$~ c for all a which satisfy a suitable normalization rule, and reject H, if a’fi > c for any such a. (Here c is some critical value.) The normalization we use is a’Aa = 1, and the reason for adopting this will become clear below. The infinite induced test just described accepts H, if the statistic s, = max{a’p]a’Aa=l} * does not exceed c, and rejects otherwise. Necessary and sufficient conditions for maximizing a% subject to a;la = 1 are ,fi - AAa = 0 for some X 2 0. So, at the maximum, a’p = X, and it can easily be verified that A=(+-‘~)‘/2, a= (@-‘fi)-‘/2A-1fi. Since s, =X, we have w= (si)*. The argument here is the same as that involved in demonstrating the equivalence between Scheffe’s (1953) test and the Wald statistic for the problem of testing the hypothesis p = 0 against the two sided alternative p # 0. In the latter case the Wald statistic is p’A-‘j? and Scheffe’s infinite induced statistic is max,{(a’/?)2 ] aila = l} [see Scheffe (1953,1959), Savin (1984)], and for a given vector a satisfying a’Aa = 1, a’,6 is distributed as N(0, 1) under H,. In the case we are concerned with, however, the motivation for basing a test on si is not entirely clear from the foregoing because no consideration has been given to the distribution of each of the a$. Additional insight can be gained by examining the relationship between 6 and fi. Let the s,(j=l, . ..) 24) be diagonal selection matrices with diagonal elements equal to either zero or one. Associate with these the 2q pairs of matrices G, = (I - S,)[I - A(S,AS,)+], 4. = (S,AS,)+, where ( e)’ denotes the Moore-Penrose inverse and I is the identity matrix of order q. We now consider the 24 cones defined by C, = {x E Iwq : P,x 5 0, G,x 2 O}. It can be verified that these cones partition lRq, and in the appendix (available from the author on request) we prove the following: Lemma.
Given p E C,, (i) ,C= G,fi, and (ii) pA-‘,h
= P’G,‘(G,AG,‘) ‘G,fi.
Now suppose that fi lies in one of the cones, C,, and consider the quantities a’G,$ for those vectors a such that a’G,AG,!a = 1. These a’G,P are unconditionally distributed as N(0, 1) under H,, and this suggests using an infinite induced test based on the statistic s2 = max { a'G,$ 1u’G,AG;u = l} a
A.J. Rogers / Testing hypotheses with one-sided alternatives
263
At the maximum, G,l; - XG,AG,‘a = 0, X 2 0, a = [jl’G,‘(G,AG,‘)+G,,hp’/2(G,AG,‘)tG,F and s2 = in X = [jYG,‘(G,AG,‘)‘~,p]‘/2. Further, w112 = s1 = s2 by the lemma given above. The difficulty interpreting the statistic s2 stems from the fact that, for a given vector a, the distribution of a’G,F conditional on p E C, is not the same as its unconditional distribution. Accordingly, we now consider an infinite induced test statistic based on fi, without reference to p. 3. A third infinite induced test statistic We now consider a test based on the quantities a$, with the vectors a chosen in such a way that the one-sided nature of the alternative is taken into account. One way of writing H, is b’A-‘p = 0 for all b # 0, and we can write H, as b’A-‘p > 0 for some b > 0. This last point follows from the fact that if p > 0, then, since A is positive definite, A -‘p must have at least one positive element, and so there exists a vector b > 0 such that b’A_‘p > 0. We set y = Aplp, T = A-$, so for those b such that b’A-‘b = 1, b’? is distributed as N(0, 1) under H,. Therefore, we will consider the statistic s,=my{b’P/b>O,
b’A-‘b=l}.
Now, whenever T has at least one positive element, necessary and sufficient conditions for maximizing b’? subject to b > 0 and b’A_ ‘b = 1 are T - hA -‘b s 0, b’? - X = 0 for some h > 0. At the maximum, sj = X, and we can find a selection matrix, S’, of maximum rank such that S/b = 0. That is, the i th diagonal element of S, is equal to one if the i th element of b is equal to zero, and is equal to zero if this element of b is positive. For this S, we can write (Z-s,)b>O,
(Z-s,)y-x(Z-s,)A-‘b=O,
S,b = 0,
SJy - XS’A-‘b
2 0,
and so Xb = [(I- S,)A-‘(ZS,)]‘(ZS,)? = (I- S,)[Z - A(S,AS,)+]fi= G,,h 2 0. Using this last result we also obtain S,? - XS,A-‘b = (S,AS,)‘S,fi = Pjj2 5 0. Therefore, fi E C, (where C, is the cone associated with the selection matrix S,). In addition, s3 = h = [p’G,‘Ap’G,fi]1/2 and so from the lemma of the last section we see that s3 = w ‘I2 . Therefore, whenever one or more element of y is positive, the four statistics s,, s2, s 3 and w112 take on the same positive value. It follows that tests based on these will give the same results provided that each test employs the same non-negative critical value. When?sO,(i.e., A-‘fi~OandE7;=O)thenw=s,=s,= 0, but s3 5 0. The implication of this is that sg can be used in conjunction with a negative critical value to obtain a test of arbitrarily high size. [The size of tests based on w or si, s2 cannot exceed 1 - Pr(p 5 0 ]H,) if these tests accept H, whenever w = s, = s2 = 0. This is so because these tests are based on ,Z, and so do not incorporate potentially useful information in j2 when fi = 0.1 We will now show that s3 reduces to a simple finite induced test statistic when y s 0. In this case, at the maximum of b’y subject to b > 0, b’A_‘b = 1, there is a selection matrix, S’, with ith element equal to one if the ith element of b is positive and equal to zero if this element of b is equal to zero such that S,? - XS, A ~ ‘S’b = 0, where now h 5 0. For this S,, we have b= -[~‘(S,Ap1S,)‘~]p’/2(S,A-‘S,)+?, and X= -[~‘(S,A-1S,)+~]1/2, with (S,A ~ ‘S,)‘? s 0 in order that the inequality restrictions on the elements of b be satisfied. Since s3 = X, we can find s3 by determining the maximum of - [p’(S,Ap’S,)‘T]1/2 over those S, satisfying (S,A-IS,)+? 5 0. But over all S, # 0, max (-[?‘(s,A~‘s,)‘P]
= max{ -(?:/A”)}
= mm{(~~/A”)},
264
A.J. Rogers / Testing hypotheses with one-sided alternatives
where i; is the ith element of T and A” is the ith diagonal element of A ~ ‘. The last result follows from the fact that (S’,A -‘S”*)’ - (S’A-‘S,)+ is positive semi-definite whenever S,* - Sj is positive semi-definite. And since (S,A-‘S,)? 5 0 when S, has just one non-zero element (and T 5 0), it follows that s3 = A = max, { i;/( A”)1’2 } when X 5 0. But, f=
m,ax { yii/( A”)1’2)
is just a finite induced test statistic based on the elements of p = A-‘jIi rather than directly on the elements of fi. Once it is noted that p is just the vector of Lagrange multipliers associated with the problem of minimizing (y - Xp)‘(y - Xp) subject to R/3 = 0, the statistic s3 has a simple interpretation as a multiple one-sided Lagrange multiplier statistic. Similarly, considered separately, w and f are also modified Lagrange multiplier statistics. [These are discussed in Rogers (n.d.).] We can summarize the relationship between s3, w and f by s,=w’/~
if
w>O,
=
References Gourieroux, C., A. Holly and A. Montfort, 1980, Kuhn-Tucker, likelihood ratio and Wald tests for nonlinear models with inequality constraints on the parameters, Discussion paper 770 (Department of Economics, Harvard University, Cambridge, MA). Gourieroux, C., A. Holly and A. Montfort, 1982, Likelihood ratio test, Wald test and Kuhn-Tucker test in linear models with inequality constraints on the regression parameters, Econometrica 50, 63-80. Rogers, A.J., nd., Modified Lagrange multiplier tests for problems with one-sided alternatives, Journal of Econometrics, forthcoming. Savin, N.E., 1984, Multiple hypothesis testing, in: Z. Griliches and M.D. Intrihgator, eds., Handbook of econometrics, Vol. 2 (North-Holland, Amsterdam) 827-879. Scheffe, H., 1953, A method of judging all contrasts in the analysis of variance, Biometrika 40, 87-104. Scheffe, H., 1959, The analysis of variance (Wiley, New York).
Economics Letters 21 (1986) 261-264 North-Holland
WALD, LIKELIHOOD RATIO, AND INFINITE INDUCED FOR JOINT ONE-SIDED HYPOTHESES
TEST STATISTICS
Alan J. ROGERS Indiana Uniuersity, Bloomington, IN 47405, USA Received
24 February
1986
The relationship between certain infinite induced test statistics and the Wald and likelihood ratio statistics for testing hypotheses with one-sided alternatives is considered. In addition, it is shown that one of these infinite induced statistics reduces to a simple finite induced test statistic when the Wald statistic is equal to zero.
1. Introduction In view of the frequency with which prior information on the signs of parameters in economic models is present, it is important that methods of testing hypotheses with one-sided alternatives be available. Examples include sign restrictions on the coefficients of linear models and non-negativity restrictions on variances in error components and random coefficients models. This paper has two purposes. The first is to show that the Wald (W) and likelihood ratio (LR) statistics for testing a joint one-sided hypothesis are equal to certain ‘infinite induced’ test statistics, where the latter are simple modifications of the statistic introduced by Scheffe (1953) for classical two-sided problems. [A good discussion of finite and infinite induced tests for two-sided problems appears in Savin (1984).] This result is helpful in interpreting the results on the W and LR statistic obtained by Gourieroux, Holly and Montfort (1980,1982) and others, and it also provides an interesting parallel to the equivalence between the For X2 statistics and Scheffe’s (1953) statistic for problems with two-sided alternatives. The second purpose is to show that one of the infinite induced test statistics considered can be used to construct a test of arbitrarily high size. In this respect this statistic differs from the W and LR statistics, since the latter are equal to zero with positive probability, and as a consequence they do not embody potentially useful information. This feature is of particular importance in cases where one is especially concerned that the power of the test be high. In addition, a rather remarkable feature of this infinite induced test statistic is that when it is not equal to the W and LR statistics it reduces to a simple finite induced test statistic. For ease of exposition we will consider the case of inequality constraints on linear functions of the coefficient vector of a normal linear regression model. This allows us to make use of some results due to Gourieroux, Holly and Montfort (1982). Asymptotic parallels of these results and those derived below can be obtained under suitable regularity conditions, see, e.g., Gourieroux, Holly and Montfort (1980), and also Rogers (n.d.). We suppose that the null hypothesis is H, : p = 0 and the alternative is H, : p > 0 where p = R/3, ,8 is the coefficient vector in the model y = Xp + U. The dimension of 1-1is q x 1, and p > 0 means 0165-1765/86/$3.50
0 1986, Elsevier Science Publishers
B.V. (North-Holland)
262
A.J. Rogers / Testing hypotheses with one-sided alternatiues
that all elements of p are non-negative and at least one is positive. We assume for simplicity that u is distributed as N(0, I), where I is the identity matrix. We let I; = Rp, where b is the least squares estimator of /3, and ii = Rp, where 6 is the constrained least squares estimator obtained subject to the set of inequality restrictions R/3 2 0. (Throughout we use 2 and s to denote weak element-byelement vector inequalities.) Under Ha, fi is distributed as N(0, A), where A = R( X’X)-‘R’. The W and LR statistics for testing H, against H, are both equal to
and, under H,, w is distributed as a mixture of qx2 distributions and a distribution unit mass to the point zero [see Gourieroux, Holly and Monfort (1982)].
which assigns
2. The W statistic and two infinite induced test statistics To see how the W statistic, w, can be motivated in terms of an infinite induced test, we note that H, specifies a’~ = 0 for all vectors a # 0. Since b 2 0, one intuitively appealing way to test H, against H, is to accept H, if a$~ c for all a which satisfy a suitable normalization rule, and reject H, if a’fi > c for any such a. (Here c is some critical value.) The normalization we use is a’Aa = 1, and the reason for adopting this will become clear below. The infinite induced test just described accepts H, if the statistic s, = max{a’p]a’Aa=l} * does not exceed c, and rejects otherwise. Necessary and sufficient conditions for maximizing a% subject to a;la = 1 are ,fi - AAa = 0 for some X 2 0. So, at the maximum, a’p = X, and it can easily be verified that A=(+-‘~)‘/2, a= (@-‘fi)-‘/2A-1fi. Since s, =X, we have w= (si)*. The argument here is the same as that involved in demonstrating the equivalence between Scheffe’s (1953) test and the Wald statistic for the problem of testing the hypothesis p = 0 against the two sided alternative p # 0. In the latter case the Wald statistic is p’A-‘j? and Scheffe’s infinite induced statistic is max,{(a’/?)2 ] aila = l} [see Scheffe (1953,1959), Savin (1984)], and for a given vector a satisfying a’Aa = 1, a’,6 is distributed as N(0, 1) under H,. In the case we are concerned with, however, the motivation for basing a test on si is not entirely clear from the foregoing because no consideration has been given to the distribution of each of the a$. Additional insight can be gained by examining the relationship between 6 and fi. Let the s,(j=l, . ..) 24) be diagonal selection matrices with diagonal elements equal to either zero or one. Associate with these the 2q pairs of matrices G, = (I - S,)[I - A(S,AS,)+], 4. = (S,AS,)+, where ( e)’ denotes the Moore-Penrose inverse and I is the identity matrix of order q. We now consider the 24 cones defined by C, = {x E Iwq : P,x 5 0, G,x 2 O}. It can be verified that these cones partition lRq, and in the appendix (available from the author on request) we prove the following: Lemma.
Given p E C,, (i) ,C= G,fi, and (ii) pA-‘,h
= P’G,‘(G,AG,‘) ‘G,fi.
Now suppose that fi lies in one of the cones, C,, and consider the quantities a’G,$ for those vectors a such that a’G,AG,!a = 1. These a’G,P are unconditionally distributed as N(0, 1) under H,, and this suggests using an infinite induced test based on the statistic s2 = max { a'G,$ 1u’G,AG;u = l} a
A.J. Rogers / Testing hypotheses with one-sided alternatives
263
At the maximum, G,l; - XG,AG,‘a = 0, X 2 0, a = [jl’G,‘(G,AG,‘)+G,,hp’/2(G,AG,‘)tG,F and s2 = in X = [jYG,‘(G,AG,‘)‘~,p]‘/2. Further, w112 = s1 = s2 by the lemma given above. The difficulty interpreting the statistic s2 stems from the fact that, for a given vector a, the distribution of a’G,F conditional on p E C, is not the same as its unconditional distribution. Accordingly, we now consider an infinite induced test statistic based on fi, without reference to p. 3. A third infinite induced test statistic We now consider a test based on the quantities a$, with the vectors a chosen in such a way that the one-sided nature of the alternative is taken into account. One way of writing H, is b’A-‘p = 0 for all b # 0, and we can write H, as b’A-‘p > 0 for some b > 0. This last point follows from the fact that if p > 0, then, since A is positive definite, A -‘p must have at least one positive element, and so there exists a vector b > 0 such that b’A_‘p > 0. We set y = Aplp, T = A-$, so for those b such that b’A-‘b = 1, b’? is distributed as N(0, 1) under H,. Therefore, we will consider the statistic s,=my{b’P/b>O,
b’A-‘b=l}.
Now, whenever T has at least one positive element, necessary and sufficient conditions for maximizing b’? subject to b > 0 and b’A_ ‘b = 1 are T - hA -‘b s 0, b’? - X = 0 for some h > 0. At the maximum, sj = X, and we can find a selection matrix, S’, of maximum rank such that S/b = 0. That is, the i th diagonal element of S, is equal to one if the i th element of b is equal to zero, and is equal to zero if this element of b is positive. For this S, we can write (Z-s,)b>O,
(Z-s,)y-x(Z-s,)A-‘b=O,
S,b = 0,
SJy - XS’A-‘b
2 0,
and so Xb = [(I- S,)A-‘(ZS,)]‘(ZS,)? = (I- S,)[Z - A(S,AS,)+]fi= G,,h 2 0. Using this last result we also obtain S,? - XS,A-‘b = (S,AS,)‘S,fi = Pjj2 5 0. Therefore, fi E C, (where C, is the cone associated with the selection matrix S,). In addition, s3 = h = [p’G,‘Ap’G,fi]1/2 and so from the lemma of the last section we see that s3 = w ‘I2 . Therefore, whenever one or more element of y is positive, the four statistics s,, s2, s 3 and w112 take on the same positive value. It follows that tests based on these will give the same results provided that each test employs the same non-negative critical value. When?sO,(i.e., A-‘fi~OandE7;=O)thenw=s,=s,= 0, but s3 5 0. The implication of this is that sg can be used in conjunction with a negative critical value to obtain a test of arbitrarily high size. [The size of tests based on w or si, s2 cannot exceed 1 - Pr(p 5 0 ]H,) if these tests accept H, whenever w = s, = s2 = 0. This is so because these tests are based on ,Z, and so do not incorporate potentially useful information in j2 when fi = 0.1 We will now show that s3 reduces to a simple finite induced test statistic when y s 0. In this case, at the maximum of b’y subject to b > 0, b’A_‘b = 1, there is a selection matrix, S’, with ith element equal to one if the ith element of b is positive and equal to zero if this element of b is equal to zero such that S,? - XS, A ~ ‘S’b = 0, where now h 5 0. For this S,, we have b= -[~‘(S,Ap1S,)‘~]p’/2(S,A-‘S,)+?, and X= -[~‘(S,A-1S,)+~]1/2, with (S,A ~ ‘S,)‘? s 0 in order that the inequality restrictions on the elements of b be satisfied. Since s3 = X, we can find s3 by determining the maximum of - [p’(S,Ap’S,)‘T]1/2 over those S, satisfying (S,A-IS,)+? 5 0. But over all S, # 0, max (-[?‘(s,A~‘s,)‘P]
= max{ -(?:/A”)}
= mm{(~~/A”)},
264
A.J. Rogers / Testing hypotheses with one-sided alternatives
where i; is the ith element of T and A” is the ith diagonal element of A ~ ‘. The last result follows from the fact that (S’,A -‘S”*)’ - (S’A-‘S,)+ is positive semi-definite whenever S,* - Sj is positive semi-definite. And since (S,A-‘S,)? 5 0 when S, has just one non-zero element (and T 5 0), it follows that s3 = A = max, { i;/( A”)1’2 } when X 5 0. But, f=
m,ax { yii/( A”)1’2)
is just a finite induced test statistic based on the elements of p = A-‘jIi rather than directly on the elements of fi. Once it is noted that p is just the vector of Lagrange multipliers associated with the problem of minimizing (y - Xp)‘(y - Xp) subject to R/3 = 0, the statistic s3 has a simple interpretation as a multiple one-sided Lagrange multiplier statistic. Similarly, considered separately, w and f are also modified Lagrange multiplier statistics. [These are discussed in Rogers (n.d.).] We can summarize the relationship between s3, w and f by s,=w’/~
if
w>O,
=
References Gourieroux, C., A. Holly and A. Montfort, 1980, Kuhn-Tucker, likelihood ratio and Wald tests for nonlinear models with inequality constraints on the parameters, Discussion paper 770 (Department of Economics, Harvard University, Cambridge, MA). Gourieroux, C., A. Holly and A. Montfort, 1982, Likelihood ratio test, Wald test and Kuhn-Tucker test in linear models with inequality constraints on the regression parameters, Econometrica 50, 63-80. Rogers, A.J., nd., Modified Lagrange multiplier tests for problems with one-sided alternatives, Journal of Econometrics, forthcoming. Savin, N.E., 1984, Multiple hypothesis testing, in: Z. Griliches and M.D. Intrihgator, eds., Handbook of econometrics, Vol. 2 (North-Holland, Amsterdam) 827-879. Scheffe, H., 1953, A method of judging all contrasts in the analysis of variance, Biometrika 40, 87-104. Scheffe, H., 1959, The analysis of variance (Wiley, New York).