On the consistency of M-estimate in a linear model obtained through an estimating equation

Statistics & Probability North-Holland

Letters

14 (1992) 79-84

4 May 1992

On the consistency of M-estimate in a linear model obtained through an estimating equation C. Radhakrishna

Rao *

Center for M&variate

LC. Zhao

Analysis, Pennsylvania State University, University Park, PA, USA

*’

Center for Multivariate Analysis, Pennsylvania State University, University Park, PA, USA, and Department of Mathematics, University of Science and Technology of China, Hefei, China Received May 1991 Revised August 1991

Abstract: We consider the linear model yi = x:p + e,, i = 1,...,n, and an estimating + ‘. + $(yn - xAfi)x, = 0 and prove the consistency of the estimator p under some imposed are much weaker than those assumed in earlier work by other authors.

equation of the form $(y, - x;p)x, mild conditions on I), The conditions

AMS 1980 Subject Classifications: 62505, 62F10, 62F12. Keywords: Consistency,

estimating

equation,

Gauss-Markov

model.

1. Introduction Consider

value

the linear

yi=xL#+ei,

i=l,...,

model

m~i~14y.-X:P)=

Iz,

Correspondence to: C.R. Rao, Center for Multivariate Analysis, Pennsylvania State University, University Park, PA 16802, USA. * Research sponsored by the Air Force Office of Scientific Research under Grant AFOSR-89-0279 and the U.S. Army Research Office under Grant DAAL03-89-K0139. t Research partially supported by the National Natural Science Foundation of China. 0 1992 - Elsevier

Science

Publishers

kP(YimxjP^) i=l

(1.1)

where x, is a known p-vector, p is a p-vector of unknown parameters and {eJ is a sequence of i.i.d. random errors with a common distribution function F. An M-estimate of p is defined as any

0167-7152/92/$05.00

p^ such that

for a suitable function p, or any solution the estimating equation ijll$(Yi-*:8)xi=o

(1.4

for p of

(l-3)

for a suitable function I,!J.There is considerable literature on the asymptotic theory of M-estimation starting with the seminal work of Huber. For detailed references on this subject, the reader is referred to recent papers by Rao (1988) and Bai, Rao and Wu (1991). Recently, Zhao, Rao and Chen (1990) considered the method of estimation mentioned in (1.2)

B.V. All rights reserved

79

Volume 14, Number 1

STATISTICS

& PROBABILITY

with some mild restrictions on p and established the weak consistency of the M-estimate of p when p is fixed under the condition S;‘-0

as rz-+a

with S,=

~x~x;.

(1.4)

i=l

max xlS;‘x,

+ 0

as n + 03

4 May 1992

(A21 Er,Q(e,>= 0 and there exist positive constants cO, ci and A such that IErCl(e,+u)I~c,lu/

Et$‘(e,+u)
(1.5)

was used. In this paper we work on this problem when p is fixed and an estimating function sych as (1.3) is used for computing an M-estimate p of p. In this direction, Yohai and Maronna (1979) proved the weak consistency of p^ under the condition (1.4), but the additional condition (A,) in their paper, imposed on the function $ and the error distribution F are somewhat severe and they exclude some important cases such as least absolute deviations (LAD) estimate. Motivated by their work, in the present paper we get rid of this condition and prove the consistency of p^ under the condition (1.4) and other mild conditions, where p^ satisfies the more general condition

for lul
(2.1)

for lul
(2.2)

and

It may be noted that in most of the earlier papers, the stronger condition ldidn

LETTERS

(A3) S,‘+O

as n+w.

(2.3)

An example of $ and F satisfying the assumptions (Al) and (A21 is given by 4(u) = sign(u) and any F such that F(O) = 4 and F’(O) > 0. We prove the following theorem in Section 3. Theorem 2.1. Assume that (Al) - (A3) are satisfied and let p^ be an M-estimate of /3 which satisfies the condition (1.6). Then j? is weakly consistent for P.

3. Proof of the main result

In this section, we denote by Z(A) and #(A) the indicator function and cardinal number of a set A, respectively. Assume that SnO> 0 for some integer no and that n 2 03. Write p, = SA/‘p,

xin = S;l12xi,

i = 1,. ..,n.

(3.1)

The model (1.1) can be rewritten as

(1.6) with 11.II denoting the Euclidean norm of a vector. Note that there may exist p^ satisfying (1.6) even if the equation (1.3) has no solution at all in some circumstances. It may be noted that, when p^ is the least squares estimate of /3 in a Gauss-Markov model, the condition (1.4) is also necessary for the consistency of p^ (see for instance Drygas (19761, Chen (1979) and Chen, Chen, Wu and Zhao (1985)). The problem of necessity of (1.4) in the general case we are considering is an open one.

2. The main result We make the following assumptions on $, F and S,: (Al) CcIis nondecreasing. 80

y,=xL!np,+ei,

i=l,...,

n,

(3.2)

with ix&

=I,,

(3.3)

i=l

where I, is the identity matrix of order p. It is easily seen that bn = SA/‘p^ is an M-estimate of p,, in the model (3.2), which satisfies the following condition: 1/2 +(

Yi px:,B,)

11 =

‘p(l)

as n+w.

(3.4)

i=l

Without loss of generality, we assume that the true parameter p = 0 in model (1.0, i.e., p, = 0 in (3.2). In order to prove Theorem 2.1, we need only to prove that & = G,(I) in view of the condition S;’ + 0.

(3.5)

Volume

14, Number

STATISTICS

1

Denote by U the unit sphere 1) in Rp. Define D(Y,

L) = i

Hei

& PROBABILITY

{p E Rp: II/3 II=

L>O,yERP.

(3.6)

By (Al), D(y, L) is nonincreasing in L for any fixed y. By (3.4), for any given E > 0 and u > 0, there exists an integer izi 2 rza such that

llplnll)~~o) -+ for

nan1,

the integer

m=[pF'],

part of pa-*.

Further

take (Y> 0 such that

P{$(e,

+a)


Finally


(3.17)

take

that O
(28mc~/2)-1K,)


P(I~B~~~~L]
(3.18) (3.8)

by the monotonicity of D(y, . 1. In the sequel always assume that n > n,. For 6 > 0, &I > 0 and n E (0, l), define J={j:

Il~,~ll>~},

l
J”= {l,...,n} S( 6) = c

x1,x;,

we

(3.9)

-J,

and

The proof is divided into two parts, and we consider the case of y E AC in the first part. By (Al) we have c

rCr(e,-L,x:,Y)xl,y

i

n={yEU:y’S(G)y~&i},

l
Ixl!,yl>n}, (3.12)

Z,“={l,...,n}

+

C irJ-I,

IIcr(ei)xLyI

C ieJc

cCl(ei)xl,Y.

(3.20)

By (3.11) and (3.14), +VY’s(~)Y=

c

By (Al) and (A2), there exists a constant such that

K >0

c

(&r)‘>

that +.

< m = [ ~6~~1, by (3.12) and (3.18) we

Since #(J) have

E > 0, take

and divide U into M parts ti,, . . . , o,, the diameter of each part is less than take

which implies

irJ

-K
-“*E~‘~K,

(x;,r)2,

iEJc

-I,.

0 < Ed < 2-‘( c,p)

+

n==u-A, (3.11)

Given

L,) =z

ieI,n.l

(3.10)

$(-m)<

(3.19)

L, >“/77.

D(Y,

II Xjn II Ga),

= &&!,I(

it7

z,={i:

(3.16)

-K} <~/(32m).

P{$(e,-a)> (3.7)

which implies

and divide U into N parts p,, . . . , I&, such that the diameter of each part is less than i&3. Let

L,= 26(clpN)1'2(cos1"1/2)~1, 0 ~6
-Lx:,Y)x;,Y,

r=l

+(B,/IIs,,lI?

4 May 1992

LETTERS

(3.13) such that Ed. Then

: <

c

(&r)*+

c

iEJnI,

(xlM2

iEJ-I,

G EI

Gl,r)‘+

T2m

1

0
(214Mc,)-1K2~),

0 < &3 < min($,

E,/(~P)),

(3.14)

<

c isJnI,

Ix:,rl+

f.

81

Volume

14, Number

which implies c

& PROBABILITY

that

Ix:,rl>

idnl,

STATISTICS

1

LETTERS

4 May 1992

By (2.21, (3.11) and (3.14), (3.21)

+.

Let A, = ($(e, $(e,

+ a) > K and -a)

< -K,

By (3.17) and #(J)

for ~EJ}.

(3.22)

G m,

< 21”Ke2 2

~11 c

P(AC,) < A&.

(3.23) Ml

Since i E Z,, implies (Al) and (3.231, P

sup c i ysAc isIyn.l G p

i

$(e,

1X~,J I> 7,

that

by

(3.20, j=l

- L,xj,y)~~!~y

< 2’°K-2A4c

+ fK a 0 1

,“t”wc iE~nJG+c

- ~,~l,Y)XltlY

P

KI x;,y 1 a 0

(3.27)

sup c +!t(ei)x,!ny > &K < $. > 1 yGAc iEJ’

From that

1 < &&.

F < h16’

11

By (3.26) and (3.27),

Y +


$(ej)xinr,1/2

iEJc

j=l

(3.28)

(3.201, (3.24), (3.25) and (3.281, it follows

(3.24)

By (3.181, using the fact #(J) for i E I,“, we get

< m and I xl,r

I< 77

P

sup D(y, YEAC

L,)

> -$K

)

< f~.

(3.29)

Now we consider the case of y E A in the second part of the proof. By (3.6) and (Al),

i=l

G

16~K-1mc~/2

< $s.

(3.25) i=l

Let the non-empty sets among o1 fl A’,. . . , CM n AC be U,, . . . , MM,. Take yj E Z_$,j = 1,. . . , M,. By (2.2) and (3.131, -

C (+(eJ - @(ei-L24,))xI,Y. itJC

(3.30)

< (25s,K-‘)2c,p 82

< &E.

(3.26)

Let the non-empty sets among I’, n A,. . . , v, n A of L21/; = be I/ ,,..., A,,,,. Since the diameter (L,y: y E I/;., 1s . 1ess than iLZ~3, it can be covered by a p-dimensional closed convex set Tj with a diameter less than L2~3, j = 1,. . . , Nl. Take a point y, E J$ j = 1,. . . , Nl. Fix a Tj. If x6/3 (p E

Volume

14. Number

STATISTICS

1

& PROBABILITY

?;) keep the same sign, we take pij E q. such that 1x[,pij

4 May 1992

LETTERS

By (2.2), (3.35), (3.15) an the fact that L,(l + 8J < $L2, we get

IIB,, II2<

1= inf{ 1x:,p I: p E q}.

If xi,/3 > 0 for some /3 E 7;. and x~!~P < 0 for some p E q, then there exists a pij E q. such that xl!,pij = 0. Write V(ei,

U) =$(e,+u)

-+(ei)

-G(u),

N,

G

(3.31)

<

=E$(e,

By (Al)

+u),

C q(e,, -xL~,pij)x:,pij >

1

c26(c”Lz,E,)-2

X

of pi,,

C

E(+(ei)

-4(ei-xhBij))2(xInPil)2

iEJc

G

C

28(cOL~s~)-2c~

C

i&fN, . .

illc(4Cei)

-4(ei-xn:jnPij))XInPij

inf c ( -xLrnPij)G( 1
a

( x:nPij)2

icJc

j

>

$CoL;&,

(3.32)

u E R’.

and the selection

iI ic.l’

j=l

where G(u)

CP

< 29N(C,L2E,)-2C,p

(3.37)

= &.

By (2.2) and (3.15) we have

-xl,pij)

(3.33) Since

11pij - L,-yj 11< L,e3,

by (3.14) we have <

1 C

(x,nPij)2

-

C

632

By (3.30), (3.33), (3.36)-(3.38), P (3.34)

< 3L2,&,P < iL&. By the selection

j=

l,...,


(3.35)

By (2.0, (3.101, (3.11) and (3.34), and noting y, E A, we get i:f

C

(-xj,Pij)G(

that

c,i:f

we get

supD(y, i YE‘4

L2) > -&)L,E,

)

< $.

(3.39)

p{ IIp*, II a L} < E for L > max(L,, L2) and 2.1 is proved. 0

n >nl,

and

Theorem

-XlnPij)

iEJc

>

(3.38)

At last, we take (T = min(iK, ~c~L,E,) in (3.7) and (3.8). From (3.8), (3.29), (3.39) and the monotonicity of sup, E UD(y, . ), it follows that

of pij, (3.9) and (3.15),

<1L,X:l,Yjl

for iEJ”,

c,p < i&.

x,, II2

+&,)&II

(-XI,PljI

-2

(L2xlnYj)21

iclc

iEJc

26(CoL2&,)

References C

(

x,,Pij)2

iGJc

Bai,

C

(~l,-y~)~ - t&i

iEJc

(3.36)

Z.D., C.R. Rao and Y. Wu (1991), M-estimation of multivariate linear regression parameters under a convex discrepancy function, Stathtica Sinica (in press). Chen, X.R. (1979), Consistency of least squares estimates in linear models, Scientia Sinica (Chinese Edition) 22 (Special Issue), 162-176. 83

Volume

14, Number

1

STATISTICS

& PROBABILITY

Chen, X.R., G.J. Chen, Q.G. Wu and L.C. Zhao (19851, Theory of estimation of parameters in linear models (Science Press, Beijing). Drygas, H. (1976), Weak and strong consistency of the least squares estimates in regression models, Z. Wahrsch. Vera. Gebiete 34, 119-127. Rao, C.R. (1988), Methodology based on the L,-norm in statistical inference, SankhyZ 50, 289-313.

LETTERS

4 May 1992

Yohai, V.J. and R.A. Maronna (19791, Asymptotic behavior of M-estimators for the linear model, Ann. Statist. 7,248-268. Zhao, L.C., CR. Rao and X.R. Chen (1990), A note on the consistency of M-estimates in linear models, Technical Rept. No. 90-42, Center for Multivariate Analysis, PennSylvania State Univ. (Univerity Park, PA).