The exact Mean Squared Error of Stein-rule estimator in linear models

Journal

of Statistical

Planning

and Inference

18 (1988) 345-353

345

North-Holland

THE EXACT MEAN SQUARED ERROR OF STEIN-RULE ESTIMATOR IN LINEAR MODELS Shyamal Das PEDDADA Dept. of Mathematics, Central Michigan University, Mt. Pleasant, MI, and Dept. of Mathematics/Statistics, University of Nebraska, Lincoln, NE, USA

Parthasarathi

LAHIRI

Dept. of Mathematics/Statistics,

Received

16 January

Recommended

University of Nebraska, Lincoln, NE, USA

1987; revised

by M.L.

Abstract: The exact Mean Squared tion of the regression estimator

within

manuscript

received

23 April

1987

Puri

vector

Error

is obtained.

an ellipsoid

(MSE) of the Stein Rule estimator Further,

the inadmissibility

for a linear combina-

of the usual least squares

is investigated.

AMS Subject Classification: 62507. Key words and phrases: Ellipsoid;

fractional;

linear model;

non-normal;

operators.

1. Introduction

In the usual linear model Y=Xfl+ E, Y is an 12x 1 vector of observations, X is an n x m design matrix, p is an m x 1 vector of unknown parameters and E is an n x 1 random error vector. We shall assume that E has a multivariate normal distribution with mean vector 0 and covariance matrix 02Z, where o2 is unknown. Without loss of any generality one can assume that X’X=nZ. Under this model the usual least squares estimator for o2 is s2 which is proportional to Y’[Z- n-‘XX’] Y. A well known modification to the estimator, more commonly known as the Stein-rule estimator, is bi = (1 - as2/nb’b)b, where a is a scalar constant. It has been established that under squared error loss, 6i dominates b if 0
0

1988, Elsevier

Science Publishers

B.V. (North-Holland)

346

S.D. Peddada,

P. Lahiri / Stein-rule estimator in linear models

mean vector 6 in the linear model y = 0 + E. They state that it is not possible to improve yi (the i-th component of y) to estimate 0; for all i under squared error loss. Phillips (1984, 1985, 1987) in his ingenious works introduces a new mathematical technique for finding the exact probability density functions of some highly nonlinear statistics such as the Stein rule estimator and the SUR (seemingly unrelated) estimator by using fractional calculus. In the 1984 article he obtained the exact distribution and bias of h’bi. In this article, using Phillips (1984) we obtain the mean squared error (MSE) of h’bi and study the inadmissibility of h’b in an ellipsoid e,,,= {/3~ V’ 1p’N/31 cr2} w h ere N is some positive definite matrix. Such subsets of the parametric space have been studied earlier by authors such as Lautter (1975), Rao (1976), Hoffman (1977), Mathew (1983). One could use the methods developed by Baranchik (1973) to obtain MSE and other higher order moments for this type of problem. Such considerations have been made in Ullah (1974), Rao and Shinozaki (1978). Knight (1986) uses the fractional calculus technique developed in Phillips (1984) to derive the exact distribution of the Stein-rule estimator in the linear model under non-normal disturbances, wherein he discusses the usefulness of the development of a fractional calculus technique in obtaining moments of the Stein-rule estimator. The algebra developed in this paper will be useful in simplifying the density expression provided in Phillips (1984) which is in terms of operator calculus. In section 3, we use the results of Knight (1986) and the calculus developed in section 2 to study the inadmissibility of h’b under non-normal disturbance of the error. We shall use the notations similar to Phillips (1984) as follows: ax = a/ax, where x = (x,,...,x,)'. Ax = ax'ax = Cy!, (d2/dxf), the usual Laplacian derivative. Fractional operator: dia = (r(a))-’ {r exp(-rd,)ra-’ dr. (a),=a(a+l).,.(a+k-1) with (&=l. <, = (aa2/n)(h’axd;‘), where h = (h,, . . ..h.)‘. H(m,~)=~~~~u~/i!(m+l). 6H(m, 0) = H(m, u) - H(m + 1, u).

2. MSE due to h’b, Let us denote y1 = h’bl. Phillips (1984) we get: m ((n-w)=

c k=O

In this section

@/2)k /.,

[(-2&.)k(dz)k{h’/3+ x

Note that in the above expression, vanishes. Thus we have:

we shall obtain

exp(x’B+

the MSE of yl.

(02/n)h’x+

(~r~/2n)x’x)]~,~. z=o

for kr 3 each of the expressions

From

(a2/n)h’h} (2.1) in the above sum

347

S.D. Peddada. P. Lahiri / Stein-rule estimator in linear models

E(yf)

= (h’/?)2+ (oVn)h’h - 2(n - m)[<,(h’P+ (02/n>h’x) x exp(x’8 + (cJ~/~~)x’x)]~,~ + (n - m)(n - m + 2)[<,2exp(x’P + (~+~/2n)x’x)]~=~.

To evaluate

E(y:)

we therefore

need to compute

[&(h’P + (a2/n)h’x)exp(x’/3+

the following

(2.2)

two expressions:

(02/2n)x’x)]X=o

(2.3)

and [<,’ exp(x’P + (02/2n)x’x)]X,o. For computing results provide

(2.4)

(2.3) and (2.4) we need some results us with the algebra needed.

on C&and Ai. The following

Result 1. For I> 1, qr 1 we have

d’w(h’w)(w’wyJ=

where h = (hl, . . . , h,)’

Proof.

now follows

Proof.

See equation

if q=l+url,

1

we see that

by induction.

Result 2. Let B= (n/2a2)/l’P, &exp(x’p+

if q
22’(~+1)I(~++m+l),(h’w)(w’w)U

and w = (wl, . . . , w,)‘.

First let I= 1. Then

The result

0

w =x+ r@/a2. Then

(a2/2n)x’x) (16) in Phillips

= (aa2/n) exp(-8)h’awd;’

exp((02/2n)w’w).

(1984).

Result 3.

Proof.

h’dwd;’

exp((a2/2n)w’w)

See equation

(20) in Phillips

= (h’w/2)H(~m;(02/2n)w’w). (1984).

348

S.D. Peddada,

P. Lahiri / Stein-rule estimator in linear models

Result 4. h’f3wd;‘(h’w)exp((a2/2n)w’w)

= (n/2a2)h’hH(+rz;(a2/2n)w’w) ++(h’w)2H(+m+

1;(&2n)w*w).

Proof. Using the definition of the operator 0;’ we see that the left-hand side of the above result is equal to

I--exp(-rA,)(h’w)

h’aw

exp((02/2n)w’w) dr

JO

= h'aw

Using Result 1 we get 22’(u+ l)[(u++m

+ l)[(h’w)(w’~)~dr,

which after some algebra reduces to

s

u, .

0 u=o

{(h’h)(w’w)U+2u(h’w)2(w’w)U-‘}

x ,io (-2ro 2/n)’ NOW for 0 < r< n/2a2 (h’h)

(u++m+l)ldx /!

*

and using the argument given in Phillips (1984) we get

r ~~o((~2/2n)w’w)U$(1

+2r0~/n)-~‘~-~-

dr

I +2(h’w)2(02/2n)

; j,

((a2/2n)w~w)“-’ 1 ~ x(u-l)!

Now using elementary quired result.

(1 + 2ra2/n)-m’2-u-

calculus and some straightforward

’ dr.

algebra we get the re-

Result 5. [h’axn;‘(h’x)exp(x’P+ =

(o~/~Tz)x’x)]~,~

(n/2a2)exp(-O)[(h’h)H(+m;

13)- (n/a2)i5H($m;8)(h’P)*].

Proof. h’dxd;‘(h’x)

exp(x’P + (02/2n)xfx)

= exp(-B)h’awd;1(h’w-(n/~2)h’P)exp((02/2n)wfw) = exp(-8)[h’i3w&,‘h’wexp((02/2n)w’w)

- (n/a2)(h’P)h’awd;1exp((a2/2n)w’~)].

S.D. Peddada, P. Lahiri / Stein-rule estimator in linear models

Using Results 3 and 4 and w = @/a2

349

we get the required result.

Now we get (2.3) in the following result. Result 6. [<#‘/I

+ a2h’x/n)exp(x’P+

o~x’x/~~)],=~

= +aexp(-B){(h’p)2H(+z+ Proof.

l;e)+(a2/n)h’hH(3m;8)}.

Using Results 2, 3 and 5, we get the left-hand side equals (/I’D)& exp(x’fl+ CI~X’X/~~)],=~ + (02/n)Q’x

exp(x’P + ff 2x’x/2n) ( x=o

= (h’P)(aa2/n)exp(-O)[(h’aw)d;’

exp(a2w’w/2n)],=,~,~z a2x’x/2n)lXE0

+ (aa4/n2)[h’8xd;‘h’xexp(x’/?+

= h’P(aa2/n)exp(-B)[(~h’w)N(3m;a2w’w/2n)],,,p/,2 + (aa4/n2)(~/202)exp(-8)[(h’h)H(+z;

e)- (n/cr2)~H(+m; 13)(h’b)~].

After some algebra, we get the result. To get (2.4) we have the following result. Result 7.

&?exp(x’P+ 02x’x/2n) 1xzo =

a2a2

Texp(-O)[h’hsH(+m

- 1 ;e)

+n(h’p)26H(~m;e)/02].

Proof. <, exp(x’P + 02x’x/2n) = l,[& exp(x’P + a2x’x/2n)]

Now using the definition of &’ and some algebra we get: exp(-rd,)h’w(w’w)‘dr.

350

S.D. Peddada, P. Lahiri / Stein-rule estimator in linear models

Using Result 1 and simplifying we get co

+(ao’/n)exp(-0) x E

f

I

h’aw 0 (-r)‘(02/2rz)u+1221(u + l)/(u + +r + l)lh’w(w’w)U dr (2.4 + I)! (+?I + u + 1)

u=o l=O = +(ao2/n)2exp(-@

1 .g, [h’h(h’w)” + 2u(h’w)2(w’w)“-1] s m x ,& (0~/2n)~~‘(-r)~2~‘(u + l),(u+ $m+ l)/ dr

=+(ao2/n)zexp(-0)

1 ug, [h’h(w’w)“+ 2u(h’w)‘(w’w)“-‘1 s (&2n)” x (U++m)u!

cm (-‘n’“‘>’ I=0

(u +/y),

dr.

Now using the negative binomial series with 0 < r< n/2a2 and using the argument given in Phillips (1984), we get +(aa2/n)2exp(-0)

u$O [h’h( wtw)U+2u(h’w)2(w’w)U-1] (1 + 2r~~/n)-~

which upon some straightforward $a202

m’2dr,

algebra reduces to

exp(-0) uto [h’h(w’wY +2u(h’w)2(w’w)U-

Using the identity l/x(x- 1) = 1/(xperforming some algebra, ~a2~2exp(-8)[h’hsH()m-

‘1

(a2/2n)” (24++n)(u++m-

l)u! .

1) - l/x we get, after setting w = np/02 and 1;e)+(n(h’p)2/02}6H(3m;8)].

Thus using (2.2), (2.3), (2.4), Results 6, 7 and the identity MSE(y,)=Var(yr) +(Bias)’ we get the required MSE(y,). Hence, MSE(y,) = 02h’h/n + klh’h + k2(h’p)2, here k, = -02(n - m)a exp(-fl)H(+n;@/n + 02(n - m)(n -m + 2)a2 exp(--O)dH(+m - 1;8)/4n and k, = (n-m)aexp(-8)6H(3m;8)+Sa2(n-m)(n-m+2)exp(-B)GH(Sm;B).

S.D. Peddada, P. Lahiri / Stein-rule estimator in linear models

351

In the following we shall show that h’b is inadmissible in the ellipsoid 0,, where N= {2n(l +c)(m - 2)/(m-

2(1 + c))(m + 2))1,

for any positive real number c. First of all we should note that MSE(h’bi ) 5 MSE(h’b) iff 4[h’hH(3m;8)05a5

Letm>2(1+~)andflE8~ we have MSE(h’bi) 5 MSE(h’b).

Thenforallasuch

Theorem2.1.

Proof.

{n(h’p)2/a2}6H(~m;8)]

(n-m+2)[h’hsN(3m-l;e)+{n(h’P)2/a2}6H(3m;8)]

From the Cauchy-Schwartz

(2.5)

thatO
inequality we see that (h’p)‘/h’h ~/?‘,0. Hence,

VW2 ~ < (m - 2( 1 + c))(m + 2) h’h 2n (m - 2)(1+ c) =) (h’B2 < _o2 (+m - (1 + c) + i)(‘m + 1 + 1) for i = 0, 1,2,. . , h’h -n (+m-l+i)(i+c) *

h’h

l[

which implies (h’h)H(+m;B) - {n(h’P)2/a2}6H(+m;0) (h’h)GH(+m - I;@ + {n(h’P)2/a2}6H(+m;8)

1 ‘*

So we see that whenever O2(1 +c), then the MSE of h’bl is less than that of the estimator h’b, thus concluding the proof. If we choose c=$ in the above theorem, we see that for mr3,01a~l/(n-m+2), the estimator h’bl makes h’b inadmissible within B,.,.

3. Inadmissibility

under non-normality

The model under consideration y=xp+?f+u

is (3.1)

where Y, X and fl are as defined earlier; q=(~,,...,q~)’ and u=(u~,...,u~)‘. Assume (u+ I?) 1q-N@, a21) and the vi’s are independent with mean 0 and variance and third and higher order cumulants are same as those of the Ui’s. Let 1=q’A@/2a2, p*=p+x’q/n and P=@*‘p*/202 where M=I-n-‘XX’. Knight (1986) obtained the exact distribution of h’b, under the model (3.1). Using equation (12) and (13) of Knight (1986), Result 6 and 7 of Section 2 and with a little algebra, one can show that

352

S.D. Peddada, P. Lahiri / Stein-rule estimator in linear models

MSE(h’b,)

= o’h’h/n

+ K,*h’h + K,*(h’/3*)”

(3.2)

where K;” = -ao’n-‘(n-m+2,I)exp(-8*)H(3m;f?*) a202 +- 4n {(n-m)(n-m+2)+4A(n-m+l+A)}

x exp(-B*)6H(+z;O*)

(3.3)

K,* = a(n - m + 2A)exp(-8*)6H(+m;8*) +$a2((,--,)(n-,+2)+4A(n-m+l+A)}

x exp(-e*)6H(+n;e*).

(3.4)

For given q, one may be interested in studying the inadmissibility that h’b is inadmissible with respect to h’bl if and only if KI*h’h+K,*(h’/3*)2

of h’b. We see

5 0

(3.5)

which implies Olal

4(n-m+21) [(n-m)(n-m+2)+4A(n-m+

1 +A)] (3.6)

Following the argument 50~) and m>2(1 +c), Osac we have MSE(h’br)

given in Theorem then, for all

2.1, we see that if /I* E Oj!,= (p* 1p*‘Nb*

4(n - m + 2A)c [(n - m)(n -in + 2) + 4A(n -m

+ 1 + A)] ’

5 MSE(h’b).

Acknowledgement The authors would like to acknowledge the referee for his useful comments. We also wish to thank Ms. Roxann Roggenkamp for excellent typing of this manuscript.

References Baranchik, problems

A.J. (1973). Inadmissibility of maximum likelihood estimators in some multiple with three or more independent variables. Ann. Statist. 312-321.

regression

1,

K. (1977). Admissibility of linear estimators with respect to restricted parameter sets. Math. Operationsforsch. Statist. 8, 425-438. Judge, G.G. and M.E. Bock (1978). The Statistical Implications of Pre-test and Stein-rule Estimators in Econometrics. North-Holland, Amsterdam.

Hoffman,

S.D. Peddada, P. Lahiri / Stein-rule estimator in linear models Knight,

J.L. (1986). The distribution

of Stein-Rule

Estimator

in a model with non-normal

353 disturbances.

Econometric Theory 2, 202-219. Lautter,

H. (1975). A minimax

linear estimator

for linear parameters

qualities. Math. Operationsforsch. Statist. 6, 689-695. Matthew, T. (1983). Admissible linear estimation in singular

linear

under models

restrictions with respect

parameter set. Tech. Report. 8317, Indian Statistical Institute, New Delhi. Phillips, P.C.B. (1984). The exact distribution of the Stein-rule estimator.

in form of ineto a restricted

J. of Econometrics 25,

123-131. Phillips, Phillips,

P.C.B. P.C.B.

(1985). The exact distribution of the SUR estimator. Econometrica 53, 745-756. (1987). Fractional Matrix Calculus and the distribution of Multivariate tests. Cowles

Foundation Paper 664. Yale University, Rao,

C.R.

(1976). Estimation

Rao,

C.R.

and N. Shinozaki

parameters. Ullah,

CT 06520.

of parameters (1978).

Precision

in a linear

model.

of individual

Ann. Statist. 4, 1023-1027.

estimators

in simultaneous

estimation

A. (1974). On the sampling

distribution

of improved

estimators

for coefficients

in linear regres-

sion, J. of Econometrics 2, 143-150. Vinod,

of

Biometrika 65, 23-30.

H. and A. Ullah

(1982). Recent Advances in Regression. Marcel

Dekker,

New York.