The laws of the iterated logarithm of some estimates in partly linear models

Statistics & Probability Letters 25 (1995) 153-162

ELSEVIER

The laws of the iterated logarithm of some estimates in partly linear models Jiti Gao Department of Statistics, University of Auckland, Private Bag 92019, Auckland, New Zealand

Received November 1993; revised September 1994

Abstract

Consider the regression model Yi = x;fl + g(q) + e i, 1 ~ i ~ n, where x i = (x~l,x~z . . . . . xip)' and t i ( t i t [0, 1]) are known and nonrandom design points, fl = (ill . . . . . ~p)' (p ~> 1) is an unknown parameter, g(-) is an unknown function, and e~ are i.i.d, random errors. Based on g estimated by nonparametric kernel estimation, the laws of the iterated logarithm of the least-square estimator of fl and an estimator of a 2 = Ee~ < ~ are investigated. Keywords: The law of iterated logarithm; Least-square estimator; Partly linear model

1. Introduction

Consider the model given by Yi = x;fl + g(ti) + e~,

1 <<.i <~ n.

(1)

The model defined in (1) belongs to the class of partly linear regression models, which was first discussed by Ansley and Wecker (1983) using a state space approach. Other related work is that of Heckman (1986), Rice (1986), Chen (1988), Chen and Shiau (1991), Speckman (1988), Robinson (1988), Kohn and Ansley (1989), Eubank and Whitney (1989), Eubank and Speckman (1990), Hong (1991), and Gao and Zhao (1993). Recently, Gao (1992) investigated the asymptotic normality of the weighted least-square estimator of/~ under the case where the (x~, ti) are nonrandom design points and g(-) is estimated by nonparametric kernel estimation. In this paper, we discuss the laws of the iterated logarithm of the least-square estimator of j6 and an estimate of 0 . 2 = E e l .

Research supported partially by National Natural Science Foundation of China and by a grant from the University of Auckland. 0167-7152/95/$9.50 © 1995 ElsevierScienceB.V~All rights reserved SSDI 0 1 6 7 - 7 1 5 2 ( 9 4 ) 0 0 2 1 7 - 7

J. Gao / Statistics & Probability Letters 25 (1995) 153 162

154

2. Main results If fl is known to be the true parameter, then the natural estimator of g(') can be defined by

.~n(t) = ~n(t, fl) = ~ Wn,(t)(Yi - x[fl),

(2)

i=l

where Wni(t)= Wni(t; tl, t2, ..: ,tn) are probability weight functions depending on the design points t l , ... , t n.

Now, based on the model Yi = x'fl + ~n(ti) + e, the least-square estimator/~n of fl can be defined by ~

(Yi

-

xifln ' ^ - ~n(ti,/~n)) 2 = min!

(3)

i=1

By (3), we obtain

/~n = (-~')~)-1~,~,

(4)

where

xi = x i -

~ Wnj (ti)x j,

Yi = Y i -

j=l

~ Wnj(ti) Yj, j=l

-~ = (~1 . . . . . ~n)', and ~ = (Y1 . . . . . yn),. Further, by Lemma 2 below we have lim 1 . ~ , ~ = B,

(5)

n~oo n

where B is positive definite. To state the main results of this paper, we introduce the following assumptions. Assumption 1. There exist some functions hj(t) over [0, 1] such that for all 1 ~< i ~< n and 1 ~
Xij = hj(ti) + Uij ,

and the real sequences u~j satisfy lim -

uiu" = B

(6b)

n-'*~ n i = 1

and lim sup an 1 max n~o~

l <~m<~n

uj,

< oo

(6c)

i=1

for any permutation 01,J2, ... ,in) of the integers (1, 2, ..., n), where ui = (Un, ..., Uip)', B is a positive definite matrix with order p x p, an = n 1/2 log n, and II • II denotes the Euclidean norm. Moreover, max IIui Ir ~< C < oo.

l <<.i<~n

Assumption 2. g and hj satisfy Lipschitz condition of order 1 over [0, 1].

(6d)

J. Gao / Statistics & Probability Letters 25 (1995) 153-162

155

Assumption 3. Assume that W.~(') satisfy (i) max1 . c.) = O(d.), where b. = o(n ~/2 (log n) 2)- 1 and c. and d. satisfy lim sup.~ ~ nc 4 log n < oo and lim s u p . . ~ nd 4 log n < 00. R e m a r k 1. The discussion of the relationship between Assumptions 1-3 and Assumptions of Speckman (1988) is omitted here for they are lengthy. The details are available from the author. N o w we give the main results of this paper. Theorem 1. (i) Assume that Assumptions 1-3 hold. Let Eel = 0 and E l e l I2+c < ~ for some c > O. Then lim sup (n/2 log log n) 1/21/~.j - flsl = (0.2bJJ) 1/2 .~0C3

a.s.,

(7)

where {/?.j} and {flj} denote the jth components of ft. and fl, respectively, and {b/k} denotes the jth row element and kth rank element of B -1 (ii) Assume that Assumptions 1-3 with b. = n - 3 / 4 (logn) 1 hold. Let Eel = 0 and Ee 4 < ~ . Then limsup(n/21oglogn)X/~10. ^2. _ 0.21 = (Var(e2)l/2 w h e r e 0.n A2 = /1-

1

~ i =. 1

(Yi

--

a.s.,

(8)

)~[fln) 2.

R e m a r k 2. U n d e r the case where the ( x , t~) are i.i.d, random variables, the above Assumptions 1 and 3 should be replaced by Assumption 1 °. E Hxl II2 < ~ , supo ~, ~ 1 E(xEj] t) < ~ , and B = cov(xl - E(xl It1)) is positive definite. Assumption 3 °. Assumption 3 holds with probability 1. N o w we give the result for the case where ( x , ti) are i.i.d, r a n d o m variables. Theorem 1 °. (i) Assume that Assumptions 1°, 2, and 3 ° hold. Let (xi, ti) and ei be independent and Eel = 0 and Ee 2 < oo. Then (7) holds. (ii) Assume that Assumptions l°, 2, and 3 ° hold. Let (xi, ti) and el be independent and Eel = 0, Ee~ < oo, and gllxl II4 < ~ . Let b, = n 3/4 ( l o g n ) - i in Assumption 3. Then (8) holds. R e m a r k 3. Theorems 1 and 1° not only construct the exact convergence rates of/~, and oA2, n but also provide some statistics for large sample tests Ho: fl = 0 and H6:0-2 = 0.2 (known). The proofs of Theorems 1 and 1° will be given in the next section.

3. Proofs 3.1.

F o r convenience and simplicity, let C (0 < C < oo) denote some constant which may have different values at each appearance throughout this paper. F o r proving the results, we introduce the following lemmas.

J. Gao / Statistics & Probability Letters 25 (1995) 1 5 3 - 1 6 2

156

Lemma 1. (i) Assume that Assumptions 2 and 3(iii) hold, then max

1 <~k<<_n

G.kj = max

1 <<.k~n

Gj(tk)-- 2~ W.i(tk)G~(ti) i=1

= O(c.) + O(d,),

(9)

for all 0 ~
(10)

l <~i<~n

where hni(t) = ~ = 1 W.i(t) xii. Proof. Trivial.

[]

L e m m a 2. Assume that Assumptions 1-3 hold, then lim 1 )?,j? = B,

(11)

tl~oo n

where B is defined by (6b). Proof. Follows from the proof of L e m m a 1 of Speckman (1988).

[]

L e m m a 3. Assume that Assumption 3 holds. Let Eel = 0 and Ee 4 < oo. Then for n large enou#h max

~ W.i(tk) ei = O(n-l/4(logn) -1/2)

a.s.

(12a)

a.s.

(12b)

l~
max

~, W.k(ti)ei = O(n-lt4(logn) -1/2

l <~k<~n i = 1

Proof. This lemma is a special case of L e m m a 5.2 of Muller and Stadtmuller (1987). The proof follows easily by applying Bennett's inequality (See Bennett, 1962). [] L e m m a 4. Assume that Assumption 3 with b, = n-3/4 (log n)-1 holds. Let Eel = 0 and Ee41 < oo. Then

I. = o(n 1/2)

a.s.,

(13)

where I. = Z[=I ~ , ¢ i W.k(ti) (el, - Ee;,) (el - Eel) and e[ = e i I ( leil <<.ill4). Proof. First, for any given C1 > 0, let j. = [C~(n/log n) 1/z] ([a] denotes the largest integer of a satisfying [a] ~< a). For 1 ~
(14)

J. Gao / Statistics & Probability L e t t e r s 25 (1995) 1 5 3 - 1 6 2

157

Then we have

~ Wn,(tl ) (e~ -

Jn = n-1/2 ~ i=1

E e l , ) ( e : - Ee:)

k#i

jn

E

~n

j=l

E

W.,(ti) (el, - Ee:,) (e; - Eel)

i ~ B ( j ) k e B i j , i) Jn

j=l

E

E

W.,(ti) (e:, - Eel,)(el - Eel)

ieB(j) keC(j)

=n-l/2 ~ Uni+n-l/2~ Vni i=1

i=1

= J~. + Jz.

(say).

(15)

Let again

U.j=

~

p.ij(e~-Ee[)=

i • B(j)

V.s=

~

~

u.ii,

(say),

(16)

v.ii,

(say),

(17)

i e B(j)

q.is(e:-Ee:)=

i e B(j)

~ i ~ B(j)

where P.ij = ~,k En,~.i) W.k(ti) (e'k -- Ee;,) and q.is = r,k ~ CtS) W.k(ti) (el, -- Eel,). Now, we note that (v.ij, i e B (j)) are conditionally independent r a n d o m variables under D.j = {ek, k e C(j)} given, E(v.ijlD.j) = 0, and

E(v~ijlD.j)<~a 2 max Iq.ijl 2 = tr 2q.ja 1 ,~i~n

for i e B(j),

(18)

where q.s = maxl < i <. I q.ij 1. Next, by the same reason as (14), we get I

I

q. = max Io.~l

= max

max

~,

W~dti)(e/,-Ee~)

l <~j
= o(n-t/4(logn)-l/2)

a.s.

(19)

Thus, by applying Bennett's inequality and using # B(j) <~ n/j,, we have

P( I 1/.il >1 dnl/Ej~ l [D. i) ~< 2 exp( - dgnj2 2/(2a2q.2 # B(j) + 4dnl/2jn lnl/4qn)) ~< 2 exp( - C(n log n)t/2j. 1) <~ 2 exp( - C C ; 1 log n)

(20)

for q. <~ CI n-*/4(log n)-*/2, where d = C~ is defined as in j. and # B ( j ) denotes the n u m b e r of elements in B(j) and C > 0 is some absolute constant.

J. Gao / Statistics & Probability Letters 25 (1995) 153-162

158

Thus, as k large enough

P ( U ( I d2nl >~C~' q" < C'n- '/*(l°g /1)- '/2)

(

<~ ~ E I(q. <~Cln-1/a(logn)-l/2)l n>~k

\

))

~ [r.jl >~Cln 1/2 \j=l

j.

<<.~

~ E(I(qn <~Cln-~/*(logn)-I/2)P([ V~j[ >~C1/11/2j; I [Dnj))

n>~k j= 1

< C/k.

(21)

Therefore, for sufficiently small C~ > 0

+ P(~2)>~k(q,>Cln-1/4(log/1)-l/2))-~0

as k ~ oe.

(22)

On the other hand, we note that (el, 1 ~< i ~
P(tJl~r >1 c,) <~(/1c~)-~ ~ EU~ j=l

<~n-lC; 2 ~

E

~

j = l ieB(j)

+ n-'C? 2 ~

(

~

,y

' Eek) W,k(ti)(ek--

keB(j,i)

~

~.

j = l ieB(j) k~B(j,i)

s e B(j,i)

E(e'i--Ee'i) 2

(

E (e~- Ee~)E(e;- ee;)

p ~ B(j,k) Jn

= n- 1C1-2 ~ j=l

~

~

W/,k(ti)2E(ei'-- Ee;) 2 E(e; -- Eel) 2

iEB(j) k~B(j,i)

A +n-1C12

~ j=l

t

~ ~ Wnk(ti) W n i ( t k ) E ( e i - - E e i ) i~B(j) k~B(j,i)

t

2

r

E(ei--Ee~) 2

<~C(nC2) - lj.( # Bt,jjj:,,2b2,, <<.CC~ 2j~ 1n- 1/2(log n)- 2 ~< C 2 / 1 - 1(log n)- 3/2

(23)

J. Gao / Statistics & Probability Letters 25 (1995) 153-162

159

(since j, >t Cl(n/log n) 1/2) for sufficiently small C1 > 0 and some C2 = 2CC~ 1. Hence, by Borel-Cantelli lemma, we have for j = 1, 2 Jj. = o(1)

a.s..

(24)

Therefore, we finally complete the proof of (13).

[]

3.2. Proofs of theorems Here we only prove Theorem 1, the proof of Theorem 1° follows by the similar reason. (i) In the following, we begin to prove (7). By (4), we have

(

)(

where

Oj = ~ W.,(ti)e,

and

O.j = O(t~)- ~ W.,(tj)O(t3.

i=1

i=1

Firstly, by the similar reason as the proof of Lemma 1, we have for all j

i=1

i=1

i=1

k=l

i=1

= O(nX/2(logn) - 1/2) = o(nl/2),

(26)

where h.k~ = hj(tk) -- 27= 1 W.i(tk) hj(ti). Secondly, by (4) and (6a) we have 2ij~ i =

i=1

A

i=1

W.i(tk) el

k

= J(J)~, + J(J)2, + J(J)3.

(say).

(27)

Thirdly, by the similar reason as the proof of Lemma 3 and use Assumptions l(6d) and 3(i) and (ii), we obtain for all j and k = 1, 2, 3

J(j)k. = o(n 1/2) a.s.

(28)

Thus, noting (25)-(28), in order to prove (7), it suffices to show that lim sup ](B- 1~,e)j L(2nlog log n)- 1 = (t72bJJ)l/2

a.s.,

n~oo

where {(B- 1,~'e)j} denotes the jth component of (B-l~"e) and e = (el, e2, ..., e.)'.

(29)

J. Gao / Statistics & Probability Letters 25 (1995) 153-162

160

Similar to (27), we also have

n-1/2(B-12'e)J= (~X s kbJk( - h k (Dkn+n-l/2 t s ) ) e s ) k = l s~= l

= ~ bJkDk"+n-'/2 ~=1 ~ ( ~k=t b~kusk)

(30)

where Dkn = n - l / 2 £ n = 1 (hk(ti) --~sn=l Wns(ti)Xsk)ei, 1 ~ k <~p. Thus, by the similar reason as Lemma 3, we get for all k,

IDk"l<~n-1/21~h"'keii=,

~=1W.s(ti)U~k)ei = o ( 1 ) a . s . ,

÷n

(31)

where h.ki = hj(tk) -- E7=1 w.,ttk) hj(ti). Lastly, by applying Corollary 5.2.3 of Stout (1974), we have

limsup i=, ~ ( ~k=X bJkuik)

a.s.

(32)

Up to now, we have completed the proof of (29). (ii). In the following, we will prove (8). By the definition of ~ , we get ^2

1 f , (I

2 ( g ' X ) - 12 ') I?

n

1

1

n

n

1

= - e'e - - e'2(2'2)-

12'e + - &(I

- 2 1 d'2(2'X)-

12'e + 2 1 CJ'e

n

n

- 2(2'2)-

'2')

~,

n

(33)

= n-1/2(I1. - I2. + Is. - 214. + 215.), where d = (O(q) - O.(q) ..... O(t.) - O.(t.))', e = (ex . . . . . e.)', and 0. is defined as in (2). Firstly, by Lemmas 1(9) and 4(12a), we have 113.1 ~< n -1/2

~ (O(ti) - O.(ti))z i=l

2n x/z max

l ,~k ,~n

= o(1)

O(tk) -

+

W.k(ti)ek

i=1

a.s.

(34)

Secondly, by (31)-(34) and use Lemma 2, we obtain for k = 2, 4

Ik. = o(1) a.s.

(35)

J. Gao / Statistics & Probability Letters 25 (1995) 153 162

161

Thirdly, we have nl/2lsn = ~ (g(ti)-- ~

i=1

Wnk(ti)g(tk))ei

k=l

- ~ W.i(ti)ez - ~ ~ W,k(ti)eke, = I6n- 17n- 18n i=1

(say).

(36)

i=1 k~i

Now, by the similar reason as (12) and use Lemma l(i), for n large enough n - 1/216n = o(1)

a.s.

(37)

Next, by Assumption 3(ii) again, we obtain Ilv, I ~< O(b,) ~ (e 2) = o(n 1/2) a.s. i=1

(38)

Recall that the definition of I. (see Lemma 4) and note that (e~' = el - el) 118, - I,I ~< ~ ~ i=l k#i

W.k(t,)e,(e['-- Eel')

+ ~ ~ W,k(ti)(el; -- Ee;,')re;- Eel) i=1 k~i

~(le['l-t-Ele~'l)i=l

~<(maxl<<.i<~n i=l~'~(W"i(tk)el)) + max

~ (W,k(t,)(e;-- Ee;)) ~

l<~k<~n i=1

i=1

((le;'l + Ele;'l))

= I9. + I10 n (say),

in order to prove 18,

I.=o(n 1/2)

a.s.

=

(39)

o(n 1/2) a.s., it suffices to show that and

lk,=O(1)

a.s.

fork=9,10.

(40)

The first term of (40) follows from Lemma 4 and the second term of (40) follows by the similar reason as Lemma 3. Thus, noting (33)-(40), in order to prove (8), it suffices to show that limsup Ill, - o'21 = (Var(e2)) 1/2 a.s.,

(41)

which follows from Hartman-Wintner law of the iterated logarithm. Up to now, we have finished the proof of Theorem 1. (iii) Here we omit the detail of the proof of Theorem 1°, which follows by the similar reason as Theorem 1. To get the proof of Theorem 1°, one needs only to modify above Lemmas 1-4.

Acknowledgements The author wishes to give his thanks to the Editor Professor R.A. Johnson and an anonymous referee for many constructive suggestions and comments which improved this paper.

162

J. Gao

Statistics & Probability Letters 25 (1995) 153-162

References Ansley, C.F. and W.E. Wecker (1983), Extensions and examples of the signal extraction approach to regression, In: A. Zeller, ed., Applied Time Series Analysis of Economic Data (Bureau of the Census, Washington) 181-192. Bennett, G. (1962), Probability inequalities for sums of independent random variables, J. Amer. Statist. Assoc. 57, 33-45. Chen, H. (1988), Convergence rates for parametric components in a partly linear model, Ann. Statist. 16, 136-146. Chen, H. and J.G. Shiau (1991), A two-stage spline smoothing method for partially linear models, J. Statist. Plann. Inference 25, 187-207. Eubank, R.L. and P. Speckman (1990), Curve fitting by polynomial-trigonometric regression, Biometrika, 77, 1-9. Eubank, R.L. and P. Whitney (1989), Convergence rates for estimation in certain partially linear models, J. Statist. Plann. Inference 23, 33 43. Gao, J.T. (1992), A large sample theory in semiparametric regression models, Ph.D. Thesis, Graduate School of the University of Science and Technology of China, P.R. China. Gao, J.T. and L.C. Zhao (1993), Adaptive estimation in partly linear models, Sci. China Ser. A 1, 14-27. Heckman, N. (1986), Spline smoothing in a partly linear model, J. Roy. Statist. Soc. Ser. B 48, 244-248. Hong, S.Y. (1991), Estimating theory in a class of semiparametric regression models, Sci. China Set. A 12, 1258-1272. Kohn, R. and Ansley, C.F. (1989), Discussion of "linear smoothers and additive models" by Buja et al., Ann. Statist. 17, 535-540. Muller, H.G. and Stadtmuller, U. (1987), Estimation of heteroscedasticity in regression analysis, Ann. Statist. 15, 610-625. Rice, J. (1986), Convergence rates for partially splined models, Statist. Probab. Lett. 4, 203-208. Robinson, P. (1988), Root-N-consistent semiparametric regression, Econometrica 56, 931-954. Speckman, P. (1988), Kernel smoothing in partial linear models, J. Roy. Statist. Soc. Set. B 50, 413-436. Stout, W. (1974), Almost Sure Convergence (Academic Press, New York).