Estimation of linear regression model with rank deficient observations matrix under linear restrictions

Microelectron. Reliab., Vol. 36, No. 1, pp. 109-110, 1996 Copyright © 1995 ElsevierScienceLtd Printed in Great Britain.All rights reserved 0026-2714/96 $9.50+.00 0026-2714(95)00018-6

Pergamon

TECHNICAL NOTE ESTIMATION OF LINEAR REGRESSION MODEL WITH RANK DEFICIENT OBSERVATIONS MATRIX U N D E R LINEAR RESTRICTIONS A. K. Srivastava Department of Statistics, Lucknow University, Lucknow, India

(Received for publication 14 December 1994) Abstract--The linear regression model with rank deficient observation matrix is postulated and Schmidt's estimator for coefficient vector is considered. An alternative to Schmidt's estimator is proposed and a natural generalization is suggested which helps in simplification of Schmidt's estimator.

1. INTRODUCTION It is well known that the least-squares method for estimating the coefficients in a linear regression model breaks down if the rank of observation matrix falls below the number of regression coefficients to be estimated. They, however, become estimable if some prior restrictions binding the coefficients are available, the number of which is equal to the number by which the rank falls short. This situation was considered by Schmidt [1] who derived an unbiased estimator. An alternative estimator is presented here and a natural generalization is forwarded. This generalization, however, turns out to be futile and does not serve any useful purpose but it brings out an interesting observation pertaining to the simplification of Schmidt's estimator.

where q is a (p - r) × 1 vector while Q i s a (p - r) x p matrix both having known elements. Under this set-up, Schmidt [1] presented the following estimator for/~:

\ Q / \ q } = [(X'X) 2 + Q ' Q ] - t [ X ' X . X ' y + Q'q],

(3)

where ( + ) denotes the Moore-Penrose inverse matrix operator. The estimator /~¢1~ is unbiased with variancecovariance matrix as

E(fi(1) - fl)(fi~1) - fly

=

O'2[(X'X) 2 q- Q ' Q ] - l ( x ' x ) 3 × [(X,X)2 + Q , Q ] - I

(4)

Stemming from Schmidt's proposition, an alternative estimator can be defined 2. LINEAR REGRESSION MODEL AND THE ESTIMATOR

]~lo)= [X'X + Q ' Q ] - I [ X ' y + Q'q]

Consider the linear regression model y = Xp + u,

(1)

where y is a T z 1 vector of T observations on the variable to be explained, X is a T × p matrix of T observations on p explanatory variables, p is the vector of regression coefficients and u is a T × 1 vector of disturbances with E(u) = 0 and E(uu') = a21T. If the observation matrix X is not of full column rank, the least-squares method for estimating/I breaks down. However, the regression coefficients become estimable if some linear restrictions binding them are available. Suppose the rank of matrix X is r ( < p ) and there are (p - r) linear restrictions given by

q = Qff,

(2)

(5)

which resembles the mixed regression estimator taking linear restrictions to be stochastic and assuming the disturbances associated with them to have mean vector O and the variance-covariance matrix lp_,. It is easy to observe that/~lo) is also unbiased with variance-covariance matrix E(]~0) - ~/)(/~co) - fl)' = cr2[X'X + Q ' Q ] - t x ' x × [ x ' x + Q ' Q ] - 1.

(6)

In fact, we can envisage the following class of estimators as a natural generalization: ]~lg~= [(X'X) g+l + Q'Q]-t[(X'X)gX'Y + Q'q],

(7)

where g is any non-negative integer specifying the estimator. 109

110

Technical Note

The estimators /~(g) are unbiased with variancecovariance matrix as

E(/~.)- ,q)(~(g)-/3)'

Using it, we observe that Q[X'X + Q ' Q ] - ~X"XlX'X + Q , Q ] - 1Q, = Q[X'X + Q ' Q ] - l(X'X + Q'Q - Q'Q)

= o . 2 [ - ( X , X ) g + 1 _it_ Q , Q ] - l ( X t X ) 2 g +

× [X'X + Q ' Q ] - 1Q,

1

(8)

x [(X'X) g+ ~ + Q ' Q ] - 1.

= Q[X'X + Q ' Q ] - ~ Q ' - Q[X'X + Q ' Q ] - I x Q"Q[X'X + Q,Q]-IQ,

The estimator D(0) reduces to the power generalization of ridge estimator if we take q = 0 and Q'Q to be a constant times an identity matrix, and in this way provides a kind of further generalization of power generalization of ridge estimator.

= [p-r

-- Ip-r

= 0

(13)

whence it follows that Q[X'X + Q ' Q ] - 1X' = 0. Similarly, we have

3. SOME REMARKS

In order to examine the relationship among the members of class defined by equation (7), we observe that the rank of (X'X + Q , Q ) - I . Q , Q is ( p - r) so that (p - r) characteristic roots of it will be non-zero and the remaining r will be zero. Suppose the non-zero characteristic roots are 01, 0 2 , . . . , 0p r with characteristic vectors as Z1, Z2 . . . . , Zp r, respectively. Thus we have [X'X+Q'Q]

1Q'QZi=0iZ~

(i:

(14)

1,2 . . . . . p - r )

X[X'X + Q ' Q ] - 1 X ' X = X[X'X + Q ' Q ] - l ( X ' X + Q ' Q - Q'Q) = x - x [ x ' x + Q ' Q ] - 1Q,Q = X,

(15)

where use has been made of equation (14). Now it is easy to see that I-(X,X)g +1 + Q , Q ] . I-X'X + Q ' Q ] - 1X'

= ( x ' x ) g x ' x [ X'X + Q ' Q ] - 1 x '

whence

+ Q'Q[X'X + Q'Q] (1-0i)Q'QZi=0iX'XZ

i

( i = 1,2 . . . . . p - r )

iX'

(9) = (X'X)gX '

but rank of X is r so that XZ~=0

(16)

I-(X'X)g+l + Q ' Q ] . [X'X + Q ' Q ] - 1 Q ,

( i = 1,2 . . . . . p - r ) .

(10)

Substituting equation (10) into equation (9), we lind

= ( x ' x ) g x ' . x I - x ' x + Q ' Q ] - ~Q' + Q'Q[X'X + Q'Q]-~Q' = Q',

(17)

where we have utilized equations (12), (14) and (15). From equations (16) and (17), we get

(1 - 01)Q'QZi = 0 or

[(X'X) g+~ + Q ' Q ] - ~ . [ ( X ' X ) g X ' + Q'] (1 - 0,)[X'X + Q ' Q ] - ~Q'QZ~ = 0

(i = 1, 2 ..... p - r). = [ x ' x + Q ' Q ] - l(x' + Q')

(18)

(11) Since QZi # 0 (i = 1, 2 , . . . , p - r), it follows from equation (11) that 01, 0 2 . . . . , Op_ r are all equal to unity. Thus the matrix [X'X + Q'Q] 1Q,Q has r roots 0 and (p - r) roots 1. Now observing that the (p - r) × (p - r) symmetric matrix Q[X'X + Q'Q]

1Q,

will have the same characteristic roots as

from where it follows that the estimators/3(o) and/)¢g) are identical. The above result clearly brings out that the generalization is not fruitful at all. It also suggests that /~(o) and /~m are identical, and therefore the estimator/~(ol should be used in practice for it has a simpler form in comparison with Schmidt's estimator. It will perhaps be interesting to consider the case when the number of linear restrictions on coefficient exceeds (p - r) and to develop appropriate estimators for/~.

[X'X + Q ' Q ] - IQ,Q, REFERENCES

it is obvious that Q[X'X + Q ' Q ] - ' Q ' = I,_,.

(12)

1. P. Schmidt, Econometrics. Marcel Dekker, New York (1976).