- Email: [email protected]
A distribution function of the F-ratio when the Stein-rule estimator is used in place of the OLS estimator
257
Economics Letters 21 (1986) 257-260 North-Holland
A DISTRIBUTION FUNCTION OF THE F-RATIO WHEN THE STEIN-RULE IS USED IN PLACE OF THE OLS ESTIMATOR
ESTIMATOR
Kazuhiro OHTANI Kobe University of Commerce,
Received
Kobe 655, Japan
7 April 1986
In this paper, we derive a distribution function of the improved F-ratio obtained by using the Stein-rule estimator in place of the OLS estimator. It is shown that the test given by the improved F-ratio for the null hypothesis that all regression coefficients are zero can be conducted based on an F distribution, and the power of the test given by the improved F-ratio is lower than that of the test given by the usual F-ratio.
1. Introduction It is well known that the Stein-rule estimator of regression coefficients dominates the ordinary least squares (OLS) estimator under the squared error loss criterion. [See, for example, Judge and Bock (1978).] However, interval estimation and hypothesis testing when the Stein-rule estimator is used have not been examined until recently. A few examples of work on this problem are Ullah, Carter and Srivastava (1984) and Chi and Judge (1985). As stated in Chi and Judge (1985) the preclusion of interval estimation and hypothesis testing may detract from the use of the Stein-rule estimator in many practical situations, Since the exact distribution of the Stein-rule estimator is very complicated, the asymptotic expansion has been used to examine the sampling performance of the Stein-rule estimator. [See, for example, Ullah (1982) and Ullah, Carter and Srivastava (1984).] Among others, Ullah, Carter and Srivastava (1984) derived the asymptotic expansions of the general shrinkage estimator which includes the Stein-rule estimator as a special case, and also derived the asymptotic expansion of the distribution function of the F-ratio which is obtained by using the shrinkage estimator in place of the OLS estimator. [This is called an improved F-ratio in their paper.] Although their expression is still not easy to use in practice, Ohtani (1985) showed that the improved F-ratio can be bounded by two F distributions if the ordinary ridge regression estimator is used in place of the OLS estimator in the F-ratio. In this paper, we examine the sampling performance of the improved F-ratio obtained by using the Stein-rule estimator in place of the OLS estimator. It is shown that the distribution of the improved F-ratio under the null hypothesis that all regression coefficients are zero can be evaluated by an F distribution, and the power of the test given by the improved F-ratio is lower than that of the test given by the usual F-ratio. 0165-1765/86/$3.50
Q 1986, Elsevier Science Publishers
B.V. (North-Holland)
258
K. Ohtani / Distribution function of F-ratio with Stein-rule estimator
2. The model and the improved
Consider
a linear regression
y=xp+q
E -
N(0,
F-ratio
model,
CT’&),
(1)
where y is an n x 1 vector of a dependent variable, X is an n X p matrix of rank p of non-stochastic independent variables, p is a p X 1 vector of coefficients and E is an n X 1 vector of normal error terms. The OLS estimator of /? is b =
( x’x)-lx’y,
and the Stein-rule
(2) estimator
proposed
by James and Stein (1961) is
B = [l - ke’e/b’X’Xb]b,
(3)
where e = y - Xb and k is a constant The usual F-ratio is
F, =
[(n -P)/P]
and the improved F, = [(fl -PI/PI
which satisfies 0 5 k 5 2( p - 2)/( n - p + 2) for p & 3.
[(b - P)‘X’X(b - b)/e’el y F-ratio
obtained
[(P - P)‘X’X(b
by using the Stein-rule - P>/e’el.
(4) estimator
in place of the OLS estimator
is (5)
The meaning of the improved F-ratio is discussed in Ullah, Carter and Srivastava (1984, p. 115). In the following, we concentrate on the test for the null hypothesis, H,: ,L?= 0, against the alternative H, : p # 0. Under the null hypothesis the improved F-ratio is written as F,=
[(n
-P)/Pl[PX’XB/e’el
= [(n - p)/p] Denoting parameter distributed and noting written as F,=
[ 1 - ke’e/b’X’Xb]‘(
b’X’Xb/e’e).
(6)
the non-central chi-square variate with degrees of freedom (d.f.) p and non-centrality 0 as X?(e) and the &i-square variate with d.f. n -p as X’,_,, we see that b’X’Xb/a2 is as X’,2_%(S), where 8 = j?‘X’XP/a 2, and e’e/a2 is distributed as Xi_,,. Using the notation that X’,_,(e) and x5_, are mutually independent, the improved F-ratio under H, can be
K~-P)/P][~
-k/(x;Z(B)/x~-,)]‘(x~(e)/x~-,)
= [I - nz/Fd.n~,(e)]2Fb,~-~(e),
(7)
where m = k(n -p)/p and Fd.n_,(0) denotes a non-central F variate with d.f. p and n -p and non-centrality parameter 0. From (7) the distribution function of the improved F-ratio is written as pr( 4 < c) = Pr((1 - m/F’)2F’
< c),
(8)
259
K. Ohtani / Distribution function of F-ratio with Stein-rule estimator
where Pr( .) denotes a probability, c is a constant and F’ = Fi,n_,(d). (1 - m/F’)‘F’ < c, we obtain the solution such that X, -C F’ < A,, where x, = [2m + c - { (4m + C)c}1’2]/2,
X,=
[2m+c+
{(4m+c)c}i’*]/2.
Solving
the inequality,
(9)
Thus we have Pr( F, -c c) = Pr( hi -C Fi,R-P( 8) < A*).
(10)
This expression shows that the distribution function of the improved F-ratio can be evaluated by an F distribution, and the level and power of the test given by F, for the null hypothesis, H, : j3 = 0, can be computed by Pr(F&,(e) 5 hi) + Pr(F;,,,-,(e) 2 h2). Some remarks are in order. First, since X, = X2 when c = 0, and h, = 0 and X2 --, cc as c + co [note that X, can be rewritten as X, = 4m2/2[(2m + c) + ((4m + c)c}‘/*] so that X, -+ 0 as c + 001, Pr(E; 5 0) = 0 and Pr(F, < 00) = 1. Second, since m = 0 if k = 0, Pr(F, -C c) = Pr( F;,,_,(B) -c c) = Pr( F, -c c). Third, since the test given by F, reduces to a two-sided test and the test given by F, is a uniformly most powerful test [see, e.g., Toro-Vizcarrondo and Wallace (1968)], the power of the test given by F, will be lower than that of the test given by F,. Power of these two tests will be computed numerically in the next section. Finally, if k is chosen such that the limit of nk tends to a finite constant as n tends to infinity, m = k(n -p)/p does not approach zero as n -+ 00. Thus, lim n+,Pr(Fs < c) # lim,,,Pr(F, < c).
3. Power of the improved F-ratio In this section, we numerically compare powers of the test given by the improved F-ratio and of the test given by the usual F-ratio. Since the weighted mean squared error E[( fi - /3)‘X’X( b - p)] is minimized when we take k = ( p - 2)/(n -p + 2) [see, e.g., Ullah (1982, p. 306)], we set k = ( p 2)/(n -p + 2) in comparison of powers. [Note that nk tends to a finite constant as n + cc in this case.] We have computed powers of the tests given by the & and F, for p = 3, 5, 8 and n = 10,20, 30,40 and for various values of the non-centrality parameter 8. The level of the tests was fixed at 0.05. The results for n = 20 and 40 are shown in table 1. The results for n = 10 and 30 are similar to those for n = 20 and 40. Note that the values for 0 = 0 in table 1 are levels of each test. In table 1, the first row for a specific combination of p and n (i.e., F,) represents the power of the test given by the usual F-ratio and the second row (i.e., F,) represents the power of the test given by the improved F-ratio when the same critical value as in the test given by F, is used. Since the level of the test given by F, when the above critical value is used is different from the 5% level pre-assigned in this numerical study, we have sought the critical values such that the level of the test given by Fs is 0.05. The third row (i.e., F,) represents the power of the test given by F, with the above level-adjusted critical values. Also, the column c shows the critical values in each test. As seen from table 1 and as expected from the third remark in section 2, the power of the test given by the improved F-ratio is lower than that of the test given by the usual F-ratio. Specifically, the difference between the powers of these two tests tends to increase as n gets larger. The results in this paper mean that although the Stein-rule estimator can give a more accurate point estimate than the OLS estimator in terms of the squared error loss criterion, it rather gives a lower power in the test for the hypothesis, H, : /3 = 0.
260
K. Ohtani / Distnbution
Table 1 Level and power of the tests given by the improved P
3
5
8
n
Test
c
finctwn
oJF-ratio with Stein-rule estimator
and the usual
F-ratios.
0 0.0
1.0
2.0
3.0
5.0
10.0
15.0
20
F;, FS F,
3.20 3.20 2.77
0.050 0.036 0.050
0.101 0.070 0.095
0.161 0.114 0.149
0.225 0.164 0.209
0.358 0.276 0.336
0.651 0.557 0.627
0.836 0.766 0.819
40
F, F, F,
2.86 2.86 2.43
0.050 0.033 0.050
0.109 0.068 0.098
0.176 0.114 0.158
0.250 0.169 0.226
0.400 0.295 0.368
0.711 0.604 0.681
0.884 0.815 0.865
20
F, F, F,
2.90 2.90 2.10
0.050 0.023 0.050
0.084 0.037 0.077
0.123 0.056 0.110
0.167 0.079 0.148
0.262 0.136 0.235
0.506 0.320 0.470
0.705 0.514 0.671
40
F, F, F,
2.49 2.49 1.67
0.050 0.018 0.050
0.091 0.029 0.079
0.139 0.046 0.117
0.193 0.069 0.163
0.311 0.131 0.268
0.598 0.345 0.545
0.799 0.572 0.759
20
4, F, F,
2.85 2.85 1.80
0.050 0.017 0.050
0.071 0.024 0.067
0.095 0.033 0.088
0.121 0.044 0.112
0.180 0.071 0.167
0.346 0.165 0.326
0.513 0.283 0.490
40
F, F, F,
2.24 2.24 1.20
0.050 0.009 0.050
0.079 0.012 0.069
0.113 0.018 0.094
0.152 0.027 0.125
0.239 0.051 0.199
0.477 0.158 0.420
0.684 0.312 0.630
References Chi, X.W. and G.C. Judge, 1985, On assessing the precision of Stein’s estimator, Economics Letters 18, 143-148. James, W. and C. Stein, 1961, Estimation with quadratic loss, Proceedings of the Fourth Berkeley Symposium, Vol. 1 (University of California Press, Berkeley, CA) 361-379. Judge, G.C. and M.E. Bock, 1978, The statistical implications of pre-test and Stein-rule estimators in econometrics (North-Holland, Amsterdam). Ohtani, K., 1985, Bounds of the F-ratio incorporating the ordinary ridge regression estimator, Economics Letters 18, 161-164. Toro-Vizcarrondo, C. and T.D. Wallace, 1968, A test of the mean square error criterion for restrictions in linear regression, Journal of the American Statistical Association 63, 558-572. Ullah, A., 1982, The approximate distribution function of the Stein-rule estimator, Economics Letters 10, 305-308. Ullah, A., R.A.L. Carter and V.K. Srivastava, 1984, The sampling distribution of shrinkage estimators and their F-ratios in the regression model, Journal of Econometrics 25, 109-122.
Economics Letters 21 (1986) 257-260 North-Holland
A DISTRIBUTION FUNCTION OF THE F-RATIO WHEN THE STEIN-RULE IS USED IN PLACE OF THE OLS ESTIMATOR
ESTIMATOR
Kazuhiro OHTANI Kobe University of Commerce,
Received
Kobe 655, Japan
7 April 1986
In this paper, we derive a distribution function of the improved F-ratio obtained by using the Stein-rule estimator in place of the OLS estimator. It is shown that the test given by the improved F-ratio for the null hypothesis that all regression coefficients are zero can be conducted based on an F distribution, and the power of the test given by the improved F-ratio is lower than that of the test given by the usual F-ratio.
1. Introduction It is well known that the Stein-rule estimator of regression coefficients dominates the ordinary least squares (OLS) estimator under the squared error loss criterion. [See, for example, Judge and Bock (1978).] However, interval estimation and hypothesis testing when the Stein-rule estimator is used have not been examined until recently. A few examples of work on this problem are Ullah, Carter and Srivastava (1984) and Chi and Judge (1985). As stated in Chi and Judge (1985) the preclusion of interval estimation and hypothesis testing may detract from the use of the Stein-rule estimator in many practical situations, Since the exact distribution of the Stein-rule estimator is very complicated, the asymptotic expansion has been used to examine the sampling performance of the Stein-rule estimator. [See, for example, Ullah (1982) and Ullah, Carter and Srivastava (1984).] Among others, Ullah, Carter and Srivastava (1984) derived the asymptotic expansions of the general shrinkage estimator which includes the Stein-rule estimator as a special case, and also derived the asymptotic expansion of the distribution function of the F-ratio which is obtained by using the shrinkage estimator in place of the OLS estimator. [This is called an improved F-ratio in their paper.] Although their expression is still not easy to use in practice, Ohtani (1985) showed that the improved F-ratio can be bounded by two F distributions if the ordinary ridge regression estimator is used in place of the OLS estimator in the F-ratio. In this paper, we examine the sampling performance of the improved F-ratio obtained by using the Stein-rule estimator in place of the OLS estimator. It is shown that the distribution of the improved F-ratio under the null hypothesis that all regression coefficients are zero can be evaluated by an F distribution, and the power of the test given by the improved F-ratio is lower than that of the test given by the usual F-ratio. 0165-1765/86/$3.50
Q 1986, Elsevier Science Publishers
B.V. (North-Holland)
258
K. Ohtani / Distribution function of F-ratio with Stein-rule estimator
2. The model and the improved
Consider
a linear regression
y=xp+q
E -
N(0,
F-ratio
model,
CT’&),
(1)
where y is an n x 1 vector of a dependent variable, X is an n X p matrix of rank p of non-stochastic independent variables, p is a p X 1 vector of coefficients and E is an n X 1 vector of normal error terms. The OLS estimator of /? is b =
( x’x)-lx’y,
and the Stein-rule
(2) estimator
proposed
by James and Stein (1961) is
B = [l - ke’e/b’X’Xb]b,
(3)
where e = y - Xb and k is a constant The usual F-ratio is
F, =
[(n -P)/P]
and the improved F, = [(fl -PI/PI
which satisfies 0 5 k 5 2( p - 2)/( n - p + 2) for p & 3.
[(b - P)‘X’X(b - b)/e’el y F-ratio
obtained
[(P - P)‘X’X(b
by using the Stein-rule - P>/e’el.
(4) estimator
in place of the OLS estimator
is (5)
The meaning of the improved F-ratio is discussed in Ullah, Carter and Srivastava (1984, p. 115). In the following, we concentrate on the test for the null hypothesis, H,: ,L?= 0, against the alternative H, : p # 0. Under the null hypothesis the improved F-ratio is written as F,=
[(n
-P)/Pl[PX’XB/e’el
= [(n - p)/p] Denoting parameter distributed and noting written as F,=
[ 1 - ke’e/b’X’Xb]‘(
b’X’Xb/e’e).
(6)
the non-central chi-square variate with degrees of freedom (d.f.) p and non-centrality 0 as X?(e) and the &i-square variate with d.f. n -p as X’,_,, we see that b’X’Xb/a2 is as X’,2_%(S), where 8 = j?‘X’XP/a 2, and e’e/a2 is distributed as Xi_,,. Using the notation that X’,_,(e) and x5_, are mutually independent, the improved F-ratio under H, can be
K~-P)/P][~
-k/(x;Z(B)/x~-,)]‘(x~(e)/x~-,)
= [I - nz/Fd.n~,(e)]2Fb,~-~(e),
(7)
where m = k(n -p)/p and Fd.n_,(0) denotes a non-central F variate with d.f. p and n -p and non-centrality parameter 0. From (7) the distribution function of the improved F-ratio is written as pr( 4 < c) = Pr((1 - m/F’)2F’
< c),
(8)
259
K. Ohtani / Distribution function of F-ratio with Stein-rule estimator
where Pr( .) denotes a probability, c is a constant and F’ = Fi,n_,(d). (1 - m/F’)‘F’ < c, we obtain the solution such that X, -C F’ < A,, where x, = [2m + c - { (4m + C)c}1’2]/2,
X,=
[2m+c+
{(4m+c)c}i’*]/2.
Solving
the inequality,
(9)
Thus we have Pr( F, -c c) = Pr( hi -C Fi,R-P( 8) < A*).
(10)
This expression shows that the distribution function of the improved F-ratio can be evaluated by an F distribution, and the level and power of the test given by F, for the null hypothesis, H, : j3 = 0, can be computed by Pr(F&,(e) 5 hi) + Pr(F;,,,-,(e) 2 h2). Some remarks are in order. First, since X, = X2 when c = 0, and h, = 0 and X2 --, cc as c + co [note that X, can be rewritten as X, = 4m2/2[(2m + c) + ((4m + c)c}‘/*] so that X, -+ 0 as c + 001, Pr(E; 5 0) = 0 and Pr(F, < 00) = 1. Second, since m = 0 if k = 0, Pr(F, -C c) = Pr( F;,,_,(B) -c c) = Pr( F, -c c). Third, since the test given by F, reduces to a two-sided test and the test given by F, is a uniformly most powerful test [see, e.g., Toro-Vizcarrondo and Wallace (1968)], the power of the test given by F, will be lower than that of the test given by F,. Power of these two tests will be computed numerically in the next section. Finally, if k is chosen such that the limit of nk tends to a finite constant as n tends to infinity, m = k(n -p)/p does not approach zero as n -+ 00. Thus, lim n+,Pr(Fs < c) # lim,,,Pr(F, < c).
3. Power of the improved F-ratio In this section, we numerically compare powers of the test given by the improved F-ratio and of the test given by the usual F-ratio. Since the weighted mean squared error E[( fi - /3)‘X’X( b - p)] is minimized when we take k = ( p - 2)/(n -p + 2) [see, e.g., Ullah (1982, p. 306)], we set k = ( p 2)/(n -p + 2) in comparison of powers. [Note that nk tends to a finite constant as n + cc in this case.] We have computed powers of the tests given by the & and F, for p = 3, 5, 8 and n = 10,20, 30,40 and for various values of the non-centrality parameter 8. The level of the tests was fixed at 0.05. The results for n = 20 and 40 are shown in table 1. The results for n = 10 and 30 are similar to those for n = 20 and 40. Note that the values for 0 = 0 in table 1 are levels of each test. In table 1, the first row for a specific combination of p and n (i.e., F,) represents the power of the test given by the usual F-ratio and the second row (i.e., F,) represents the power of the test given by the improved F-ratio when the same critical value as in the test given by F, is used. Since the level of the test given by F, when the above critical value is used is different from the 5% level pre-assigned in this numerical study, we have sought the critical values such that the level of the test given by Fs is 0.05. The third row (i.e., F,) represents the power of the test given by F, with the above level-adjusted critical values. Also, the column c shows the critical values in each test. As seen from table 1 and as expected from the third remark in section 2, the power of the test given by the improved F-ratio is lower than that of the test given by the usual F-ratio. Specifically, the difference between the powers of these two tests tends to increase as n gets larger. The results in this paper mean that although the Stein-rule estimator can give a more accurate point estimate than the OLS estimator in terms of the squared error loss criterion, it rather gives a lower power in the test for the hypothesis, H, : /3 = 0.
260
K. Ohtani / Distnbution
Table 1 Level and power of the tests given by the improved P
3
5
8
n
Test
c
finctwn
oJF-ratio with Stein-rule estimator
and the usual
F-ratios.
0 0.0
1.0
2.0
3.0
5.0
10.0
15.0
20
F;, FS F,
3.20 3.20 2.77
0.050 0.036 0.050
0.101 0.070 0.095
0.161 0.114 0.149
0.225 0.164 0.209
0.358 0.276 0.336
0.651 0.557 0.627
0.836 0.766 0.819
40
F, F, F,
2.86 2.86 2.43
0.050 0.033 0.050
0.109 0.068 0.098
0.176 0.114 0.158
0.250 0.169 0.226
0.400 0.295 0.368
0.711 0.604 0.681
0.884 0.815 0.865
20
F, F, F,
2.90 2.90 2.10
0.050 0.023 0.050
0.084 0.037 0.077
0.123 0.056 0.110
0.167 0.079 0.148
0.262 0.136 0.235
0.506 0.320 0.470
0.705 0.514 0.671
40
F, F, F,
2.49 2.49 1.67
0.050 0.018 0.050
0.091 0.029 0.079
0.139 0.046 0.117
0.193 0.069 0.163
0.311 0.131 0.268
0.598 0.345 0.545
0.799 0.572 0.759
20
4, F, F,
2.85 2.85 1.80
0.050 0.017 0.050
0.071 0.024 0.067
0.095 0.033 0.088
0.121 0.044 0.112
0.180 0.071 0.167
0.346 0.165 0.326
0.513 0.283 0.490
40
F, F, F,
2.24 2.24 1.20
0.050 0.009 0.050
0.079 0.012 0.069
0.113 0.018 0.094
0.152 0.027 0.125
0.239 0.051 0.199
0.477 0.158 0.420
0.684 0.312 0.630
References Chi, X.W. and G.C. Judge, 1985, On assessing the precision of Stein’s estimator, Economics Letters 18, 143-148. James, W. and C. Stein, 1961, Estimation with quadratic loss, Proceedings of the Fourth Berkeley Symposium, Vol. 1 (University of California Press, Berkeley, CA) 361-379. Judge, G.C. and M.E. Bock, 1978, The statistical implications of pre-test and Stein-rule estimators in econometrics (North-Holland, Amsterdam). Ohtani, K., 1985, Bounds of the F-ratio incorporating the ordinary ridge regression estimator, Economics Letters 18, 161-164. Toro-Vizcarrondo, C. and T.D. Wallace, 1968, A test of the mean square error criterion for restrictions in linear regression, Journal of the American Statistical Association 63, 558-572. Ullah, A., 1982, The approximate distribution function of the Stein-rule estimator, Economics Letters 10, 305-308. Ullah, A., R.A.L. Carter and V.K. Srivastava, 1984, The sampling distribution of shrinkage estimators and their F-ratios in the regression model, Journal of Econometrics 25, 109-122.