- Email: [email protected]
Nonparametric curve estimation with time series errors
Journal
of Statistical
Planning
and Inference
167
28 (1991) 167-183
North-Holland
Nonparametric errors*
curve estimation
with time
Young K. Truong Department Received
of Biostatistics,
University
11 May 1989; revised
Recommended
Abstract:
a model
a parametric estimators
for the flexibility
Key
time series error.
Under
regression
part of a stationary
process
then these parameters
Subject words
average;
Chapel Hill, NC 27514, USA
21 June
1990
and the serial correlation is a smooth
appropriate
can be chosen to achieve the pointwise
processes), AMS
received
in which the mean function
by Stone in nonparametric forms
manuscript
Carolina,
by M. Hallin
To account
we propose
of North
Classification: and phrases:
optimal
Natve
regularity
function
estimation.
can be estimated
kernel
rate of convergence;
62605;
conditions,
or the uniform
with a finite number
Primary
in fitting
estimator;
optimal
Moreover,
of parameters
measurement
function a sequence
data,
and the noise is of local average
rate of convergence
as defined
if the time series error
structure
(e.g.,
with the usual root-n
secondary
repeated
nonparametric
finite order autoregressive
rate of convergence.
62E20.
nonparametric
time series; autoregressive
regression;
curve
estimation;
local
process.
1. Introduction In analysis
of human
longitudinal
x,,=ti(t;)+z;,n,
growth
curves,
data are often
modeled
as
t;=i/n,
where 19(. ) is a smooth function on [0, l] and Z,, are random errors with mean 0 and variance cr*, i=O , 1, . . . , n. See, for example, Gasser et al. (1984), Seber and Wild (1989) and the references given therein for many interesting applications of curve estimation. In the above model, the function f?(. ) is said to be parametric if it is defined in terms of a finite number of unknown parameters. Otherwise, it is called nonparametric. An advantage of the parametric approach is its interpretation of the model. However, as indicated in Gasser et al. (1984), parametric modelings are often not very flexible in fitting data. They provided an example on human growth * Research the University
was supported of North
0378-3758/91/$03.50
by Junior
Carolina
Faculty
at Chapel
@? 1991-Elsevier
Development
Award
and a Research
Hill.
Science
Publishers
B.V. (North-Holland)
Council
grant
from
168
Y.K. Truong / Semiparametric repeated measurement models
in which many parametric growth spurt (MS).
analyses
fail to account
for a phenomenon
known
as mid-
When data are collected sequentially over time, the errors Z; often have substantial correlation. This happens because measurements are being taken over time from the same subject or there may be other environmental factor such as sickness (Seber and Wild (1989, p. 272)). To model this correlation in practice, the error process is often
assumed
to be part of a stationary
process.
Followings
are two examples:
Example 1.1. AR(p) errors. In many applications, the correlation may expect to be decreasing as the delay k between Zj and Zi+k increases. The simplest case is Corr(Z,,Zi+,)=&, This is the covariance
lel cl.
structure
Z;=@Z;_,+&;,
of the AR(l)
process
e;‘~qo,c2).
Here (~~),~z is an innovation process. In general, let p denote a positive integer and (b,), 5 ;_ be constants called parameters. The process Z; is said to be a p-th order autoregressive time series if it satisfies Z;=b,Z;_,
i.i.d. E; (0,a2).
+ . ..+b.Z;_,+E;,
1.2. ARMA(p, q) errors. Let p and q be positive integers. Also, denote the The process Zj is said to be an autoparameters by (bi)i
In the above
examples,
l-b,z-...
suppose
-b/#O,
the following 1z1 51,
condition:
ZEC,
and there exists a sequence of real numbers holds. Then (Z,)iCL is stationary (~7~)~~~ such that (see Theorem 3.1.1 of Brockwell and Davis (1987)) Zj= C auEimu, u
C la,1
For example, the AR(l) process Zj=~Zi_i+ei (1~1
0(. ) is a smooth
c, i.d(o,02),
+ C a,.&,_., U function
with
bounded
t,=i/n, first
i=O,l,...,
derivative
on
n,
(1.1)
(0,l)
and
Y. K. Truong / Semiparametric
(a,.),.z
are unknown
constants.
repeated measurement
Note that this model
models
has the flexibility
169
to fit a
wide variety of data sets because of the nonparametric function 19(.) and it has the ability to account for serial correlation in the measurements. A similar model was also considered by Hardle and Pham (1986) in time series trend estimation. The nonparametric estimator to be considered in this paper is the naive kernel Under some appropriate conditions, this estimator based on local averages. estimator will be shown to possess the optimal rate of convergence nm”3 both pointwise and in the L2 norm; and the optimal rate of convergence (K’ log n)“3 in the L” norm. These results generalize partially the corresponding results established by Stone (1980, 1982) to include correlated observations. Moreover, they can also be used to fit the error process. For example, if the error process is an AR(p), then the parameters can be estimated with nm”2 rate of convergence. In fact, this result extends the corresponding ones in time series regression in which 0(. ) is assumed to be a parametric mean function. See Fuller (1976) and Hannan (1970). The rest of the paper is organized as follows: Section 2 describes the nonparametric estimator of Q( .) and optimal rates of convergence. The fitting of the error process is given in Section 3. Section 4 presents two numerical examples. Some concluding remarks are given in Section 5 and proofs are given in Section 6. We conclude this section with a brief review of some related work. For uncorrelated error models, besides the previously mentioned Gasser et al. (1984), Stone (1980, 1982) and the references given therein, Wong (1983) considered the problem on bandwidth selection. For correlated errors, Roussas (1988), Truong and Stone (1988) established rates of convergence for nonparametric estimators of the mean function based on the a-mixing, which is a nonparametric description of the dependence structure. One difficulty in this approach is the achievability of the L” rates for unbounded correlated observations, although optimal rates for bounded series are obtained by Truong and Stone (1988). Muller and Stadtmuller (1987, of correlation based on m1988) considered the L” rate and the estimation dependent error processes. Hart and Wehrly (1986), Hart (1989) and Chiu (1989) considered the problem on optimal bandwidth selection in the L2 sense. Hardle and Pham (1986) obtained pointwise central limit results for a class of robust smoothers. Except the work by Muller and Stadtmiiller (1988) on fitting finite order moving average processes, none of the above papers addressed the problem of nonparametric estimation of the mean function Q( .) and the fitting of error process.
2. Statement
of results
The kernel estimators of the function 0( .) will now be described. Let the observations y be given by (1.1). Let (a,,), k 1 be a sequence of positive numbers tending to zero. Given t E [0, 11, set I,(t)={i:Osisn
and
It;-tl
sS,l
and
N,,(t)=
#I,(t).
Y. K.
170
Truong / Semiparametric repeated measurement models
Also set &(t)=N,(t)-’
c I$, I!,(l)
which is a kernel estimator of e(t) based on local averages. Note that although this is a simple estimator, the local mean estimator has the advantage to provide justification
of the usual root-n
Section 3. The rates of convergence the following conditions. Condition
2.1.
about
of the nonparametric
There is a positive
constant
the parametric estimator
error process.
treated
here depend
See on
M, such that
for s, tE [0, 11.
~e(S)-ee
inference
2.2. sup c l%,ul cm. ” U
Condition
2.3. i.i.d.
hi -
N(0,~2),
u=o,
&l, +2 )....
The normality in Condition 2.3 is imposed only to avoid technical details. It can be weakened by using, for example, Assumption 2.6.3 of Brillinger (1981) which assumes basically that the innovation process (~;)~~z has moments (or cumulants) of any order so that its characteristic function admits a Taylor expansion. Given positive numbers a, and b,, n L 1, let a, - b, mean that a,/b, is bounded away from zero and infinity. Given random variables I$, n ~1, let V, = OJb,) mean that the random variables 6,’ V,, n L 1 are bounded in probability or, equivalently,
that lim
lim sup P( 1V,) > cb,)
Y. K.
Truong
/ Semiparametric
Let g( . ) be a real-valued
Theorem
2.2.
function
repeated
measurement
models
171
on [0, 11. Set
Suppose 6, - n ~’ and that Conditions 2.1 and 2.2 hold. Then
IIe,(.)-e(.>II,=O,(n~“>.
2.3. Suppose Conditions 2.1-2.3 hold. If 6, -(n-l ists a positive constant c such that
Theorem
lim P(lle,( n Proofs
.) - 19(.)I/mlc[n-’
of these theorems
log n)‘, then there ex-
log(n)]‘) =O.
will be given in Section
6.
Remark 2.1. The above theorems generalize results of Stone (1982) to cover dependent observations Y. Furthermore, Theorem 2.3 does not require interpolation in order to achieve the L” rate of convergence. Remark 2.2. Note that model (1.1) is a semiparametric model. An advantage of the present approach is the feasibility to carry out statistical inference about the correlation structure. Details are given in the next section. A completely nonparametric formulation of the curve estimation is given in Truong and Stone (1988) where the t, are allowed to be random. For this approach, the o-mixing condition is used in place of Condition 2.2 of the present paper.
3. Fitting of error processes In practice, the serially correlated error process is unknown and it may be of interest to estimate it. In this section, this process will be treated as a p-th order autogressive model AR(p). Specifically, let y=e(t;)+Z,,
(3.1)
Z;=b,Z;~,+~~~+b,Z;~,+E;,
(3.2)
- b,zP #O for all z E C such that (z( 5 1 and E; are i.i.d. N(0, a’), {Z;} is observable, then the AR(p) model can be as usually done in ordinary time series fitted by regressing Zj on Z,_,, . . ..Zj_. analysis. Since the actual observations are Yo, . . . , &, the parameters will have to be estimated from the residuals. To this end, denote the residuals by
where
1-b,z-...
i=O, 19*.., n. If the error process
p;=
r;-
Q&j) =
z;-
(&(t;) - d(f;)),
172
Y.K.
Truong
/ Semiparametric
repeated
and b :=(b,, . . . . 6,)‘. Also, let b^ be the estimator Pi-t, ...,Z;_P.
measurement
model5
of b obtained
by regressing
Zj on
Theorem 3.1. Suppose Condition 2.1, (3.1) and (3.2) hold. If gfl( .) is constructed based on 6, -(IT’ log n)“, then I/(6-b)
= N(0, 02r,-‘),
where the p XP matrix r, := [COV(Z,, Z,)]fj= The proof
of this theorem
1.
will be given in Section
6.
Remark 3.1. In classical time series analysis, the conclusion of the above theorem holds by fitting a parametric model to the mean function 0(. ). See Fuller (1976, Chapter 9) and Hannan (1970, Chapter 8). The above theorem shows that such a parametric assumption is not necessary. A practical implication of this is that the model has greater flexibility in fitting data. Remark 3.2. Let e2 and e2 denote respectively the estimator of cr2 based on Z ,, . . . , Z, and Z,, . . . , 2,. Then the above theorem shows b2+ o2 in probability. Furthermore, it also shows that fi(d2a’) and fi(82-o2) have the same limiting distributions. Remark 3.3. Note that the covariance matrix of b^ is identical to that of the estimator obtained by regressing Zj on Zj_ t, . . . , Z,_,. This implies that there is no loss of efficiency in parametric estimation under the reconstruction of the error process through smoothing.
4. Numerical
results
This section illustrates the methodology discussed in Sections 2 and 3 by giving two examples: The first is a simulation study and the second is based on actual data. Example 4.1.
The observations
y=sin(2rt,)+Z;,
were generated
Z,=0.5Z,-,+E;,
from 1.i.d. E, - N(O,O.S’),
i=O, 1) . ...200.
Figure 1 shows the scatterplot for these observations. A local average estimate of 0(t)=sin(2at) is obtained by choosing a smoothing parameter 6,=0.17. This is also presented in Figure 1. (In fact, a sequence of local average estimates were constructed with bandwidths ranging from 0.12 to 0.20. 6, = 0.17 was chosen because it provided the most suitable smooth estimate. One may also try to select the bandwidth via the method considered by Chiu (1989), which, however, requires the knowledge of time series component.) After the data were smoothed, the residuals y-@Jr,) (Figure 2) were used to
Y.K. Truong / Semiparametric repeated measurement models
0.2
0.0
Fig. 1. Scatter
plot of Y, =
0.4
0.6
0.8
sin(a?rt,)+Z,, i=O, 1, . . . . 200, where Z,=O.~Z,_I
The solid line is the estimate
based on local mean with bandwidth
173
=0.17.
1.0
and E,“G N(O,0.5’).
+E,
The broken
line is the curve
Q(t) = sin(2at).
0.2
0.0
0.4
Fig. 2. Residual
provide
parametric
estimates
0.6
t
plot of y-
0.8
1 .o
f?,,(f,), i=O, 1, . . . . 200.
for the time series Z,. The estimated
by
model
is given
1.l.d.
Z; = 0.463052;_,
+ E;,
E; -
N(O,O.23 136).
Indeed, this was obtained by fitting four autoregressive models (AR(p), p = 1, . . . ,4) to the residuals using Brillinger’s DRBLIB. These results are given in Table 1. Here, HQ(p) is defined by HQ(p) = log@*) + 2prC ’ log log(n) (Hannan and Quinn, 1979) with s* denoting the residual variance. The order of the fitted process is chosen to be the value of p at which HQ(p) is minimum. In this case, the minimum value is HQ(l). (The Hannan-Quinn criterion is preferred over other model selection criteria because (1) it is a consistent rule, and (2) in a simulation study, Liitkepohl (1985) has shown that this criterion leads most often to correct estimates for model order and has the smallest prediction error.) Table
1
Parametric
time series model
fitting
for the error
6, = 0.46305
1
6, = 0.43035,
2 4
s2
6
P
3
structure
6, ~0.43124, 6,=0.43051,
& = 0.07063
&,=0.07605,
&2=0.08041,
&=0.01260
&=0.01212,
&,=0.05733
HQ(P)
0.23136
~ 1.4304
0.23021
- 1.4187
0.23017
- 1.4022
0.22942
~ 1.3888
174
Y.K.
Truong
/ Semiparametric
repeated
measurement
models
Remark 4.1. Different values of 6, were used to study the effect of bandwidth on fitting parametric model to the residuals. The value of b^, changed from 0.44784 to 0.47061 when the bandwidth 6, was chosen from 0.15 to 0.18. Here any 6, E (0.15,O. 18) would yield a reasonable 0.20 produced under and oversmooth HQ was consistent-it 6,
smooth
estimates,
estimate
of 19(. ), as
respectively. picked first
the
0.12 hand,
AR
for
(0.12,0.20). 4.2.
random of males, 35-59, selected parlowering with center in ment Biostatistics University North (LRC-CPPT, In study, cholesterol were over period 7.6 so effectiveness cholestyramine drug lowering can evaluated. set measurements also by (1989) cancer Figure shows bimonthly of participant, ted the interval. observations then with bandwidth 0.10. was chosen eyes.) examine correlated the described Section was to residual (Figure by AR(l), . , AR(4). Results are summarized in Table 2, which indicates that serial correlation is present in the measurements. In fact, based on the Hannan-Quinn criterion, Table 2 suggests that the residuals may be best described by an AR(2) model. a
0.4
0.0
Fig. 3. Scatter
0.0
plot of 46 cholesterol
measurements.
0.2
Fig. 4. Residuals
0.4
plot of r;-
0.8
The data were smoothed
0.6
B,(t;), i=O, 1,
with bandwidth
0.8
,45.
1.0
= 0.10.
1.0
175
Y. K. Truong / Semiparametric repeated measurement models Table 2 Parametric
residuals
analysis
of cholesterol
measurements
6
P
b^, = ~ 0.28356
1
b^, = -0.41485,
2
t;z = -0.46303
6, = -0.50007, &= -0.53938, &= -0.18405 6, = -0.52286, 62= -0.60615, & = -0.24595, l& = PO.12379
3 4
s2
HQ(P)
312.71
5.8620
245.67
5.6791
237.35
5.7030
233.71
5.7459
5. Conclusion In this paper, a semiparametric model is proposed for analyzing repeated measurement data. The mean function is nonparametric so that it has greater flexibility in fitting various data sets. A parametric error process is imposed in response to the fact that not only serially correlated data arise quite often in growth curve analysis, it is also an important issue to estimate the correlation structure. Under some reasonable conditions, the proposed procedures for estimating the parametric and nonparametric components are shown to possess the desired optimal properties. In particular, the limiting distribution of the parametric estimators-parametric time series inference-may be useful for examining correlated structures in repeated measurement analyses. However, for analyses based on finite samples, one has to provide a bandwidth for the local average estimator. This can be accomplished by (1) trying a few bandwidths and select the ‘best’ one by eyes; (2) using a method proposed by Chiu (1989). An interesting question is: If a sequence of local average estimators is constructed with bandwidth chosen by Chiu’s method, do the optimal properties described in Section 2 still hold? (Chiu (1989) viewed the nonparametric mean function as a nuisance parameter and hence did not address rates of convergence in nonparametric estimation.)
6. Proofs
&, la,l*. For tE[O, 11, set K,(t)=
In the
following
an+ will be denoted ll,t,_,,56,,l, i=O, l,...,
Lemma
2.2 holds.
6.1. Suppose
Var
Proof.
discussion,
Condition
simply by a,,. Set Ilall*:= n. Then C, K,(t)=O(n&,).
Then
C K,(t)[ I$- t9(t,)] = O(n6,). > c I
Set U,= I’-O(t;)=
C, ai-u~,.
By Condition
2.2,
176
Y. K. Truong / Semiparametric repeated measurement models
Var
C K.(t)U-
(LL
+ 2 C C COV(Kj(t)ui,Kj+j(t)u,,j)
= C E[K;(t)U;12 = O(nd,)a2
+2 C C rO(nfi,)6: =
ij
llaj12
C C ai-ua;+~-uCOV(~u~~~)
Ki(t>K;+j(t)
,,a,,2 +2a2
c i;(J, i
c
lajl c
J
u
laJ
O(n&)
as desired. Lemma 6.2. Suppose that Conditions 2.2 and 2.3 hold. Then there is a positive constant c, such that
P
C K,(t)[I:-B(t,)] (I i
52exp(-c2c,ndi),
c>O.
The proof
k random cumulant,
depends on the following notion. Let (Z,, . . . , Z,) denote a vector of variables with EIZJlk
. . . , Z,) = c (-1y-l(p-
l)!E
n (;G”,
zJ)‘*-E(Jiipq)
where the sum extends over all partitions { vl, . . . , vp}, p = 1,2, . . . , k, of (1, . . . , k}. of iks, “..sk in the Taylor Equivalently, cum(Z,, . . . , Z,) is given by the coefficient series expansion of log E[exp(i C: ZjSJ)] about the origin (Brillinger, 1981). An important special case of this definition occurs when ZJ = Z, j = 1, . . . , k. This then gives the cumulant of order k of a univariate random variable. Examples: (a) cum(Z,) = EZ,. (b) cum(Z,, Z,) = Cov(Z,, Z,) = E(Z, Z,) - EZ,EZ,. (c) cum(Z1, Z,) = Var(Z,). - EZ,Z,EZ, - EZ,Z,EZ, + (d) cum(Zt, Z2, Z,) = EZt Z2 Z, - EZ,Z,EZ, 2EZ, EZ, EZ, . (e) For Gaussian stationary processes, cumk =0, for k23. See Brillinger (1981) for additional methods on computing cumulants. Proof of Lemma 6.2. As in Lemma 6.1, Ui := q- B(t,), i= 1, . . . , n. By Condition 2.3 and Theorem 2.3.l(iv) of Brillinger (1981), the absolute value of the k-th cumulant of C, K,(f)CI, has the form
Y.K.
= Therefore
Truong
/ Semiparametric
K,(r)Li,)j
6.1, there is a positive E [ exp ( sCK(t)U ; ,
Set s=cS,/c,.
Then P
=.s”Var(+ constant
models
117
there is a positive
7 K,(i)U,).
c,, such that
,)] =exp(s’Var
C K,(t)U,rcndfj
(
measurement
:
logE[exp(cC By Lemma
repeated
constant
(+ F K,(t)L’,)) c1 such that
i
~exp(&s2n~,).
7 Ki(r)U;jl
> 5 exp( - scnf3t, + +s2c@,,) 5 exp( - c2c,n8:).
Similarly, P
C K,(t)U,s
-cnf_St,
i
(
Iexp(-c2crn8i). )
Proof of Theorem Condition 2.1,
2.1.
je(t;)-e(t)l
t e [0, 11, set Z,,= I,,(t) and N, =N,,(t).
Given
sA4()Bn,
According
to
iEI,.
Thus N,’
F [@t;)-@(t)] ,I
Note that there is a positive Chebyshev’s inequality,
=O(K’).
constant
rVar
(6.1)
c2 such that N,, 1 c2nJn. By Lemma
(cn~‘n~,)2=0(1)
6.1 and
as c-w.
i (Note that 6, - n -‘.) Therefore N,’
;
[I’-Qt;)]
=0&P).
(6.2)
II The conclusion
of the theorem
follows
from (6.1) and (6.2).
Y. K. Truong / Semiparametric
178
Proof of Theorem
2.2.
le(tj)-e(t)l
By Condition SM,
If;-tl
repeated measurement
models
2.1, IMa&,
iEZ,@),
tE[O, I].
Thus N,(t))’ Set z,(t)
c [W;)-e(t)] I,,(0
= CiE,n(f’ [I:-e(ti)]. E[Zi(t)]
IM()6,,
By Lemma
= O(nd,)
te [O, 11.
ieZ,(t),
(6.3)
6.1, over t e [O, 11.
uniformly
Consequently, dr=O(n&). for t E [0, 11. Hence
Recall that N,(t)rc&,
(6.4)
by (6.4) and Markov’s
inequality
P
(6.5) It follows
from (6.3) and (6.5) that lim limP(I~8,-el~2~c(~n+(n-‘6,1)“2))=0. c+rn n
The conclusion or equivalently,
of the theorem 6, = n Pr.
Proof of Theorem 2.3. L, 1 [,;@+r’ log n]. Let whose coordinates is of [0, I] can be written as and all of its vertices in w such that t E Q,v. Let
now follows by choosing
6, so that 6, = (nm’s,‘)“2,
Let T be a positive constant such that 0< t< 1 and set W, be the collection of (L, + 1) points in [O, l] each of the form j/L,, for some integer j such that 01j~ L,. Then the union of L, subintervals, each having length A,, = L,’ W,. For each t E [0, l] there is a subinterval Q, with center C, denote the collection of centers of these subintervals.
Then P (
Izbpl, 1$(t)-e(t)1 >
=P ( It follows large)
ZC(H-'
i0gn)'
>
~~7 ,“~“t,p&(t)-e(t)1 n n
from An- 6:+‘/log le(t)-e(bq
n = o(6,)
rM,lt-wl
Therefore, to prove the theorem, tant c such that
IC(K'
>
and Condition
244,6,, it is sufficient
.
i0gn)r
tEQw,
2.1 that (for n sufficiently WEC,.
to show that there is a positive
cons-
Y. K. Truong / Semiparametric
repeated measurement
FEa; I”EU~,10n(r)-O(w)l 1imP n n ,* ( Set r,=In(w)= 8(w)=ave(
179
models
=O.
2c(n-‘logn)’
(6.6)
>
{i: 1
Na=Nn(w)= from
I$: iota},
#f,(w)
=0
and
(6.7)
i and ~,“c” /On(w)-d(w)l limp n 1, (1
)
To verify (6.7) and (6.8), set yn=iJIfl(w)= are positive constants cs and c, such that N~(w)-~,,(w)~cs~,~+~ By Lemma
and
6.2, there is a positive
(6.8)
=0.
~c(n-~logn)'
constant
#{i:
It,- WI 16,-A,}.
Nn(w)~c4ndn
Then
there
for all WEC,.
cs such that
P (I Note that there is a positive
constant
K
such that
#c, 5 nK. Hence
P
5 2n” exp( - c2c,na i). Since nS3, -log P Consequently,
n, we conclude NR(w)-’
that,
for c sufficiently
C [ x-O(t,)] i,(w)
large, 52nKexp(-c2
log n).
for c2 > K, (6.9)
Observe
that (6.8) follows
from (6.3) and (6.9).
Given t E [0, 11, set N, = N,(t) y,,
Thus
and Z, = I,(t)
and choose
w such that t E Q,,. Then
Y.K. Truong / Semiparametric
180
repeated measurement
models
and hence
Since
# C, I nK, it follows
from Chebyshev-type
5
inequality
P
involving
max max ( WEC,, I s,sn
=O(l)n’+KE/Y,I for k sufficiently theorem.
large.
Hence
(6.7) is valid.
This
E j Y, 1k that
2CsS;‘+r k
>
‘Lk (log n)k = O(l)
completes
the proof
of the
for all zE@ such that 1~15 1, Proof of Theorem 3.1. Since 1-6rz... -b,zp#O it follows from Theorem 3.1 .l of Brockwell and Davis (1987) that there exists a sequence (ai)ick such that C, Ia,1 < 00 and z;=
c a,_&,,. U
Thus by Theorem
2.3,
II&C.) -
et.)lI, = qm-’
(6.10)
log(n)I
Note that z;=
Y--&Jr;)
=z;-
(e,(t,) - &t;)).
Thus -~i+h~8n(~i)-~(~;)l-~;~B,(f,+~)--(~,+,)l
4Z+h=Z,Zi+h
+]e,(~,)-~(t,)l]&(~;+,)-O(t;+h)l, By Condition
(Proof
h=O,l,...,p-1.
2.1, (3.1) and (3.2),
n-’
c z;+,[&(t,) - fw,)]
.-’
c z;[&(t;+,)-W;+/J]
=
of (6.12) will be given shortly.)
O,((n~’ log n)lr), =0&KI
(6.12)
1ogn)“r).
Set
n-h
c
&(h)=(n-h)_’
h C .Z?,.Z!i+h, i=l
II
Z;Z;+h
and
i=l
9,(h)=(n-h))’
h=O,l,_.., p-l. Then
(6.11)
by (6.10)-(6.12),
n-h p);,(h) n
n-h = ~~~(h)+0,((n-110gn)2r), n
h=O,l,...,p-1.
Y. K. Truong / Semiparametric
That
repeated measurement
181
models
is, 8,(h)-~,(h)=o,(n~“2),
h=O,l,...,p-1.
If 6 denote the estimator of b obtained Yule-Walker estimator of b. Then
by regressing
Z, on Zi_,, . . ..Z._,,
i.e. the
b^-6=o,(n-“2). Consequently,
by Proposition
8.10.1 of Brockwell
and Davis (1987),
fi(b-b)d(O,o’T,‘). Proof of (6.12). Note that
W,.hN
n-l c Z;[&(t;+,)-
+n-’
c z,
N
n
(i;.,,, k jxt _) te(fk)- Wi+/?>l. E ,I / ii
(6.13)
Let m = m, = InS,,]. Then 2m-l5N,(t;)l2m+l, Put y(i)=E(Z,Z,+j),
i=O,l,...,
n.
(i=O, i: l,... ). By Theorem
qn-1p(
l
2.3.2 of Brillinger
(1981),
Zkj1_
c
~n@i+h)kEI,tU,+h)
+
Idi-0
W-A
=$ (,j;n(l-nml Ii11 lY(i)l)2=O($), since
Ci ly(i)l < 03. By m = [nS,] - log(n)(n-’
‘-’
7 “(&(:i+i,)
k&+/J
‘k
>
log(n)))2’
and Markov’s
inequality,
Y.K.
182
=
Truong
/ Semiparametric
O,((K’ log n)2’),
repeated
measurement
models
h = 0, 1, . . . ,p - 1.
(6.14)
This takes care of the first term of (6.13). The second term will now be considered. According to Condition 2.1,
M;Jt,
c EZ;+A4;6;
c ;
c lE(Z;ZJ J
= O(ns&
(6.15)
as 1; C, lE(ZfZj)l 5 Ci C, laiPul Cj laj_ulo*Sn ll~ll’o*. It now follows from (6.13)~(6.15) and 6, - (n-l log n)’ that (6.12) holds. This completes the proof of the theorem.
Acknowledgements The author is grateful to thank Chuck Stone,
to David Brillinger for the use of DRBLIB. Also, he wishes Dennis Cox, Editors and referees for helpful comments.
References Brillinger,
D.R.
Brockwell,
P.J.
(1981).
Time Series: Data Analysis
and R.A.
Davis (1987).
Chiu, S.T. (1989). Bandwidth
selection
Time Series:
and Theory. Theory
for kernel estimate
Holden-Day,
and Methods.
with correlated
San Francisco,
CA.
Springer,
New York.
noise. Statist.
Probab.
Lett.
8, 347-354. Gasser, T., H.G. Miiller, W. Kchler, L. Molinari and A. Prader analysis of growth curve. Ann. Statist. 12, 210-229. Fuller,
W.A.
(1976). Introduction
to Statistical
Time Series Analysis.
Hannan,
E.T.
(1970). Mulfiple
Time Series. Wiley,
Hannan,
E.T.
and B.G. Quinn
(1979). The determination
Statist. Sot. Ser. B 41, 190-195. HBrdle, W. and D.T. Pham (1986). Some theory
(1984).
Nonparametric
Wiley,
New York.
New York. of the order
on M-smoothing
of an autoregression.
Stark-r. Assoc.
measurements
7,
data.
J.
81, 1080-1088.
Kritchevsky, S. (1989). Lipid changes ment of Epidemiology, University
preceding the diagnosis of cancer. Doctoral of North Carolina, Chapel Hill, NC.
LRC-CPPT
Clinics
(1984).
J. Roy.
of time series. J. Time Ser. Anal.
191-204. Hart, J.D. (1989). Kernel regression with time series errors. Preprint. Hart, J.D. and T.E. Wehrly (1986). Kernel regression estimation using repeated Amer.
regression
Lipid
Research
Program.
Lipid
research
clinics
Dissertation,
primary
Depart-
prevention
trial
results: I. reduction in incidence of coronary heart disease. J. Amer. Medical Assoc. 251, 351-364. Liitkepohl, H. (1985). Comparison of criteria for estimating the order of a vector autoregressive process. J. Time Ser. Anal.
6, 35-52.
Y.K.
Miiller,
H.G.
Statist. Miiller,
Truong
and U. Stadtmiiller
/ Semiparametric
(1987). Estimation
measurement
of heteroscedasticity
183
models
in regression
analysis.
Ann.
15, 610-625. H.G.
and
Biometrika
U.
Stadtmiiller
(1988).
Detecting
C.J.
dependencies
in smooth
regression
models.
75, 639-650.
Roussas, G.G. (1988). Nonparametric regression Seber, G.A.F. and D.J. Wild (1989). Nonlinear Stone,
repeated
(1980).
Optimal
rates
estimation Regression.
of convergence
for
under mixing conditions. Wiley, New York.
nonparametric
estimators.
Preprint. Ann.
Statist.
8,
Ann.
Staiist.
10,
1348-1360. Stone,
C.J. (1982). Optimal
global
rates of convergence
for nonparametric
regression.
1040-1053. Truong,
Y.K. and C.J.
pear in Ann. Wong,
W.H.
Stat&r.
Stone (1988). Nonparametric
function
estimation
involving
time series. To ap-
Statist. (1983). On the consistency
11, 1036-1041.
of cross-validation
in kernel nonparametric
regression.
Ann.
of Statistical
Planning
and Inference
167
28 (1991) 167-183
North-Holland
Nonparametric errors*
curve estimation
with time
Young K. Truong Department Received
of Biostatistics,
University
11 May 1989; revised
Recommended
Abstract:
a model
a parametric estimators
for the flexibility
Key
time series error.
Under
regression
part of a stationary
process
then these parameters
Subject words
average;
Chapel Hill, NC 27514, USA
21 June
1990
and the serial correlation is a smooth
appropriate
can be chosen to achieve the pointwise
processes), AMS
received
in which the mean function
by Stone in nonparametric forms
manuscript
Carolina,
by M. Hallin
To account
we propose
of North
Classification: and phrases:
optimal
Natve
regularity
function
estimation.
can be estimated
kernel
rate of convergence;
62605;
conditions,
or the uniform
with a finite number
Primary
in fitting
estimator;
optimal
Moreover,
of parameters
measurement
function a sequence
data,
and the noise is of local average
rate of convergence
as defined
if the time series error
structure
(e.g.,
with the usual root-n
secondary
repeated
nonparametric
finite order autoregressive
rate of convergence.
62E20.
nonparametric
time series; autoregressive
regression;
curve
estimation;
local
process.
1. Introduction In analysis
of human
longitudinal
x,,=ti(t;)+z;,n,
growth
curves,
data are often
modeled
as
t;=i/n,
where 19(. ) is a smooth function on [0, l] and Z,, are random errors with mean 0 and variance cr*, i=O , 1, . . . , n. See, for example, Gasser et al. (1984), Seber and Wild (1989) and the references given therein for many interesting applications of curve estimation. In the above model, the function f?(. ) is said to be parametric if it is defined in terms of a finite number of unknown parameters. Otherwise, it is called nonparametric. An advantage of the parametric approach is its interpretation of the model. However, as indicated in Gasser et al. (1984), parametric modelings are often not very flexible in fitting data. They provided an example on human growth * Research the University
was supported of North
0378-3758/91/$03.50
by Junior
Carolina
Faculty
at Chapel
@? 1991-Elsevier
Development
Award
and a Research
Hill.
Science
Publishers
B.V. (North-Holland)
Council
grant
from
168
Y.K. Truong / Semiparametric repeated measurement models
in which many parametric growth spurt (MS).
analyses
fail to account
for a phenomenon
known
as mid-
When data are collected sequentially over time, the errors Z; often have substantial correlation. This happens because measurements are being taken over time from the same subject or there may be other environmental factor such as sickness (Seber and Wild (1989, p. 272)). To model this correlation in practice, the error process is often
assumed
to be part of a stationary
process.
Followings
are two examples:
Example 1.1. AR(p) errors. In many applications, the correlation may expect to be decreasing as the delay k between Zj and Zi+k increases. The simplest case is Corr(Z,,Zi+,)=&, This is the covariance
lel cl.
structure
Z;=@Z;_,+&;,
of the AR(l)
process
e;‘~qo,c2).
Here (~~),~z is an innovation process. In general, let p denote a positive integer and (b,), 5 ;_ be constants called parameters. The process Z; is said to be a p-th order autoregressive time series if it satisfies Z;=b,Z;_,
i.i.d. E; (0,a2).
+ . ..+b.Z;_,+E;,
1.2. ARMA(p, q) errors. Let p and q be positive integers. Also, denote the The process Zj is said to be an autoparameters by (bi)i
In the above
examples,
l-b,z-...
suppose
-b/#O,
the following 1z1 51,
condition:
ZEC,
and there exists a sequence of real numbers holds. Then (Z,)iCL is stationary (~7~)~~~ such that (see Theorem 3.1.1 of Brockwell and Davis (1987)) Zj= C auEimu, u
C la,1
For example, the AR(l) process Zj=~Zi_i+ei (1~1
0(. ) is a smooth
c, i.d(o,02),
+ C a,.&,_., U function
with
bounded
t,=i/n, first
i=O,l,...,
derivative
on
n,
(1.1)
(0,l)
and
Y. K. Truong / Semiparametric
(a,.),.z
are unknown
constants.
repeated measurement
Note that this model
models
has the flexibility
169
to fit a
wide variety of data sets because of the nonparametric function 19(.) and it has the ability to account for serial correlation in the measurements. A similar model was also considered by Hardle and Pham (1986) in time series trend estimation. The nonparametric estimator to be considered in this paper is the naive kernel Under some appropriate conditions, this estimator based on local averages. estimator will be shown to possess the optimal rate of convergence nm”3 both pointwise and in the L2 norm; and the optimal rate of convergence (K’ log n)“3 in the L” norm. These results generalize partially the corresponding results established by Stone (1980, 1982) to include correlated observations. Moreover, they can also be used to fit the error process. For example, if the error process is an AR(p), then the parameters can be estimated with nm”2 rate of convergence. In fact, this result extends the corresponding ones in time series regression in which 0(. ) is assumed to be a parametric mean function. See Fuller (1976) and Hannan (1970). The rest of the paper is organized as follows: Section 2 describes the nonparametric estimator of Q( .) and optimal rates of convergence. The fitting of the error process is given in Section 3. Section 4 presents two numerical examples. Some concluding remarks are given in Section 5 and proofs are given in Section 6. We conclude this section with a brief review of some related work. For uncorrelated error models, besides the previously mentioned Gasser et al. (1984), Stone (1980, 1982) and the references given therein, Wong (1983) considered the problem on bandwidth selection. For correlated errors, Roussas (1988), Truong and Stone (1988) established rates of convergence for nonparametric estimators of the mean function based on the a-mixing, which is a nonparametric description of the dependence structure. One difficulty in this approach is the achievability of the L” rates for unbounded correlated observations, although optimal rates for bounded series are obtained by Truong and Stone (1988). Muller and Stadtmuller (1987, of correlation based on m1988) considered the L” rate and the estimation dependent error processes. Hart and Wehrly (1986), Hart (1989) and Chiu (1989) considered the problem on optimal bandwidth selection in the L2 sense. Hardle and Pham (1986) obtained pointwise central limit results for a class of robust smoothers. Except the work by Muller and Stadtmiiller (1988) on fitting finite order moving average processes, none of the above papers addressed the problem of nonparametric estimation of the mean function Q( .) and the fitting of error process.
2. Statement
of results
The kernel estimators of the function 0( .) will now be described. Let the observations y be given by (1.1). Let (a,,), k 1 be a sequence of positive numbers tending to zero. Given t E [0, 11, set I,(t)={i:Osisn
and
It;-tl
sS,l
and
N,,(t)=
#I,(t).
Y. K.
170
Truong / Semiparametric repeated measurement models
Also set &(t)=N,(t)-’
c I$, I!,(l)
which is a kernel estimator of e(t) based on local averages. Note that although this is a simple estimator, the local mean estimator has the advantage to provide justification
of the usual root-n
Section 3. The rates of convergence the following conditions. Condition
2.1.
about
of the nonparametric
There is a positive
constant
the parametric estimator
error process.
treated
here depend
See on
M, such that
for s, tE [0, 11.
~e(S)-ee
inference
2.2. sup c l%,ul cm. ” U
Condition
2.3. i.i.d.
hi -
N(0,~2),
u=o,
&l, +2 )....
The normality in Condition 2.3 is imposed only to avoid technical details. It can be weakened by using, for example, Assumption 2.6.3 of Brillinger (1981) which assumes basically that the innovation process (~;)~~z has moments (or cumulants) of any order so that its characteristic function admits a Taylor expansion. Given positive numbers a, and b,, n L 1, let a, - b, mean that a,/b, is bounded away from zero and infinity. Given random variables I$, n ~1, let V, = OJb,) mean that the random variables 6,’ V,, n L 1 are bounded in probability or, equivalently,
that lim
lim sup P( 1V,) > cb,)
Y. K.
Truong
/ Semiparametric
Let g( . ) be a real-valued
Theorem
2.2.
function
repeated
measurement
models
171
on [0, 11. Set
Suppose 6, - n ~’ and that Conditions 2.1 and 2.2 hold. Then
IIe,(.)-e(.>II,=O,(n~“>.
2.3. Suppose Conditions 2.1-2.3 hold. If 6, -(n-l ists a positive constant c such that
Theorem
lim P(lle,( n Proofs
.) - 19(.)I/mlc[n-’
of these theorems
log n)‘, then there ex-
log(n)]‘) =O.
will be given in Section
6.
Remark 2.1. The above theorems generalize results of Stone (1982) to cover dependent observations Y. Furthermore, Theorem 2.3 does not require interpolation in order to achieve the L” rate of convergence. Remark 2.2. Note that model (1.1) is a semiparametric model. An advantage of the present approach is the feasibility to carry out statistical inference about the correlation structure. Details are given in the next section. A completely nonparametric formulation of the curve estimation is given in Truong and Stone (1988) where the t, are allowed to be random. For this approach, the o-mixing condition is used in place of Condition 2.2 of the present paper.
3. Fitting of error processes In practice, the serially correlated error process is unknown and it may be of interest to estimate it. In this section, this process will be treated as a p-th order autogressive model AR(p). Specifically, let y=e(t;)+Z,,
(3.1)
Z;=b,Z;~,+~~~+b,Z;~,+E;,
(3.2)
- b,zP #O for all z E C such that (z( 5 1 and E; are i.i.d. N(0, a’), {Z;} is observable, then the AR(p) model can be as usually done in ordinary time series fitted by regressing Zj on Z,_,, . . ..Zj_. analysis. Since the actual observations are Yo, . . . , &, the parameters will have to be estimated from the residuals. To this end, denote the residuals by
where
1-b,z-...
i=O, 19*.., n. If the error process
p;=
r;-
Q&j) =
z;-
(&(t;) - d(f;)),
172
Y.K.
Truong
/ Semiparametric
repeated
and b :=(b,, . . . . 6,)‘. Also, let b^ be the estimator Pi-t, ...,Z;_P.
measurement
model5
of b obtained
by regressing
Zj on
Theorem 3.1. Suppose Condition 2.1, (3.1) and (3.2) hold. If gfl( .) is constructed based on 6, -(IT’ log n)“, then I/(6-b)
= N(0, 02r,-‘),
where the p XP matrix r, := [COV(Z,, Z,)]fj= The proof
of this theorem
1.
will be given in Section
6.
Remark 3.1. In classical time series analysis, the conclusion of the above theorem holds by fitting a parametric model to the mean function 0(. ). See Fuller (1976, Chapter 9) and Hannan (1970, Chapter 8). The above theorem shows that such a parametric assumption is not necessary. A practical implication of this is that the model has greater flexibility in fitting data. Remark 3.2. Let e2 and e2 denote respectively the estimator of cr2 based on Z ,, . . . , Z, and Z,, . . . , 2,. Then the above theorem shows b2+ o2 in probability. Furthermore, it also shows that fi(d2a’) and fi(82-o2) have the same limiting distributions. Remark 3.3. Note that the covariance matrix of b^ is identical to that of the estimator obtained by regressing Zj on Zj_ t, . . . , Z,_,. This implies that there is no loss of efficiency in parametric estimation under the reconstruction of the error process through smoothing.
4. Numerical
results
This section illustrates the methodology discussed in Sections 2 and 3 by giving two examples: The first is a simulation study and the second is based on actual data. Example 4.1.
The observations
y=sin(2rt,)+Z;,
were generated
Z,=0.5Z,-,+E;,
from 1.i.d. E, - N(O,O.S’),
i=O, 1) . ...200.
Figure 1 shows the scatterplot for these observations. A local average estimate of 0(t)=sin(2at) is obtained by choosing a smoothing parameter 6,=0.17. This is also presented in Figure 1. (In fact, a sequence of local average estimates were constructed with bandwidths ranging from 0.12 to 0.20. 6, = 0.17 was chosen because it provided the most suitable smooth estimate. One may also try to select the bandwidth via the method considered by Chiu (1989), which, however, requires the knowledge of time series component.) After the data were smoothed, the residuals y-@Jr,) (Figure 2) were used to
Y.K. Truong / Semiparametric repeated measurement models
0.2
0.0
Fig. 1. Scatter
plot of Y, =
0.4
0.6
0.8
sin(a?rt,)+Z,, i=O, 1, . . . . 200, where Z,=O.~Z,_I
The solid line is the estimate
based on local mean with bandwidth
173
=0.17.
1.0
and E,“G N(O,0.5’).
+E,
The broken
line is the curve
Q(t) = sin(2at).
0.2
0.0
0.4
Fig. 2. Residual
provide
parametric
estimates
0.6
t
plot of y-
0.8
1 .o
f?,,(f,), i=O, 1, . . . . 200.
for the time series Z,. The estimated
by
model
is given
1.l.d.
Z; = 0.463052;_,
+ E;,
E; -
N(O,O.23 136).
Indeed, this was obtained by fitting four autoregressive models (AR(p), p = 1, . . . ,4) to the residuals using Brillinger’s DRBLIB. These results are given in Table 1. Here, HQ(p) is defined by HQ(p) = log@*) + 2prC ’ log log(n) (Hannan and Quinn, 1979) with s* denoting the residual variance. The order of the fitted process is chosen to be the value of p at which HQ(p) is minimum. In this case, the minimum value is HQ(l). (The Hannan-Quinn criterion is preferred over other model selection criteria because (1) it is a consistent rule, and (2) in a simulation study, Liitkepohl (1985) has shown that this criterion leads most often to correct estimates for model order and has the smallest prediction error.) Table
1
Parametric
time series model
fitting
for the error
6, = 0.46305
1
6, = 0.43035,
2 4
s2
6
P
3
structure
6, ~0.43124, 6,=0.43051,
& = 0.07063
&,=0.07605,
&2=0.08041,
&=0.01260
&=0.01212,
&,=0.05733
HQ(P)
0.23136
~ 1.4304
0.23021
- 1.4187
0.23017
- 1.4022
0.22942
~ 1.3888
174
Y.K.
Truong
/ Semiparametric
repeated
measurement
models
Remark 4.1. Different values of 6, were used to study the effect of bandwidth on fitting parametric model to the residuals. The value of b^, changed from 0.44784 to 0.47061 when the bandwidth 6, was chosen from 0.15 to 0.18. Here any 6, E (0.15,O. 18) would yield a reasonable 0.20 produced under and oversmooth HQ was consistent-it 6,
smooth
estimates,
estimate
of 19(. ), as
respectively. picked first
the
0.12 hand,
AR
for
(0.12,0.20). 4.2.
random of males, 35-59, selected parlowering with center in ment Biostatistics University North (LRC-CPPT, In study, cholesterol were over period 7.6 so effectiveness cholestyramine drug lowering can evaluated. set measurements also by (1989) cancer Figure shows bimonthly of participant, ted the interval. observations then with bandwidth 0.10. was chosen eyes.) examine correlated the described Section was to residual (Figure by AR(l), . , AR(4). Results are summarized in Table 2, which indicates that serial correlation is present in the measurements. In fact, based on the Hannan-Quinn criterion, Table 2 suggests that the residuals may be best described by an AR(2) model. a
0.4
0.0
Fig. 3. Scatter
0.0
plot of 46 cholesterol
measurements.
0.2
Fig. 4. Residuals
0.4
plot of r;-
0.8
The data were smoothed
0.6
B,(t;), i=O, 1,
with bandwidth
0.8
,45.
1.0
= 0.10.
1.0
175
Y. K. Truong / Semiparametric repeated measurement models Table 2 Parametric
residuals
analysis
of cholesterol
measurements
6
P
b^, = ~ 0.28356
1
b^, = -0.41485,
2
t;z = -0.46303
6, = -0.50007, &= -0.53938, &= -0.18405 6, = -0.52286, 62= -0.60615, & = -0.24595, l& = PO.12379
3 4
s2
HQ(P)
312.71
5.8620
245.67
5.6791
237.35
5.7030
233.71
5.7459
5. Conclusion In this paper, a semiparametric model is proposed for analyzing repeated measurement data. The mean function is nonparametric so that it has greater flexibility in fitting various data sets. A parametric error process is imposed in response to the fact that not only serially correlated data arise quite often in growth curve analysis, it is also an important issue to estimate the correlation structure. Under some reasonable conditions, the proposed procedures for estimating the parametric and nonparametric components are shown to possess the desired optimal properties. In particular, the limiting distribution of the parametric estimators-parametric time series inference-may be useful for examining correlated structures in repeated measurement analyses. However, for analyses based on finite samples, one has to provide a bandwidth for the local average estimator. This can be accomplished by (1) trying a few bandwidths and select the ‘best’ one by eyes; (2) using a method proposed by Chiu (1989). An interesting question is: If a sequence of local average estimators is constructed with bandwidth chosen by Chiu’s method, do the optimal properties described in Section 2 still hold? (Chiu (1989) viewed the nonparametric mean function as a nuisance parameter and hence did not address rates of convergence in nonparametric estimation.)
6. Proofs
&, la,l*. For tE[O, 11, set K,(t)=
In the
following
an+ will be denoted ll,t,_,,56,,l, i=O, l,...,
Lemma
2.2 holds.
6.1. Suppose
Var
Proof.
discussion,
Condition
simply by a,,. Set Ilall*:= n. Then C, K,(t)=O(n&,).
Then
C K,(t)[ I$- t9(t,)] = O(n6,). > c I
Set U,= I’-O(t;)=
C, ai-u~,.
By Condition
2.2,
176
Y. K. Truong / Semiparametric repeated measurement models
Var
C K.(t)U-
(LL
+ 2 C C COV(Kj(t)ui,Kj+j(t)u,,j)
= C E[K;(t)U;12 = O(nd,)a2
+2 C C rO(nfi,)6: =
ij
llaj12
C C ai-ua;+~-uCOV(~u~~~)
Ki(t>K;+j(t)
,,a,,2 +2a2
c i;(J, i
c
lajl c
J
u
laJ
O(n&)
as desired. Lemma 6.2. Suppose that Conditions 2.2 and 2.3 hold. Then there is a positive constant c, such that
P
C K,(t)[I:-B(t,)] (I i
52exp(-c2c,ndi),
c>O.
The proof
k random cumulant,
depends on the following notion. Let (Z,, . . . , Z,) denote a vector of variables with EIZJlk
. . . , Z,) = c (-1y-l(p-
l)!E
n (;G”,
zJ)‘*-E(Jiipq)
where the sum extends over all partitions { vl, . . . , vp}, p = 1,2, . . . , k, of (1, . . . , k}. of iks, “..sk in the Taylor Equivalently, cum(Z,, . . . , Z,) is given by the coefficient series expansion of log E[exp(i C: ZjSJ)] about the origin (Brillinger, 1981). An important special case of this definition occurs when ZJ = Z, j = 1, . . . , k. This then gives the cumulant of order k of a univariate random variable. Examples: (a) cum(Z,) = EZ,. (b) cum(Z,, Z,) = Cov(Z,, Z,) = E(Z, Z,) - EZ,EZ,. (c) cum(Z1, Z,) = Var(Z,). - EZ,Z,EZ, - EZ,Z,EZ, + (d) cum(Zt, Z2, Z,) = EZt Z2 Z, - EZ,Z,EZ, 2EZ, EZ, EZ, . (e) For Gaussian stationary processes, cumk =0, for k23. See Brillinger (1981) for additional methods on computing cumulants. Proof of Lemma 6.2. As in Lemma 6.1, Ui := q- B(t,), i= 1, . . . , n. By Condition 2.3 and Theorem 2.3.l(iv) of Brillinger (1981), the absolute value of the k-th cumulant of C, K,(f)CI, has the form
Y.K.
= Therefore
Truong
/ Semiparametric
K,(r)Li,)j
6.1, there is a positive E [ exp ( sCK(t)U ; ,
Set s=cS,/c,.
Then P
=.s”Var(+ constant
models
117
there is a positive
7 K,(i)U,).
c,, such that
,)] =exp(s’Var
C K,(t)U,rcndfj
(
measurement
:
logE[exp(cC By Lemma
repeated
constant
(+ F K,(t)L’,)) c1 such that
i
~exp(&s2n~,).
7 Ki(r)U;jl
> 5 exp( - scnf3t, + +s2c@,,) 5 exp( - c2c,n8:).
Similarly, P
C K,(t)U,s
-cnf_St,
i
(
Iexp(-c2crn8i). )
Proof of Theorem Condition 2.1,
2.1.
je(t;)-e(t)l
t e [0, 11, set Z,,= I,,(t) and N, =N,,(t).
Given
sA4()Bn,
According
to
iEI,.
Thus N,’
F [@t;)-@(t)] ,I
Note that there is a positive Chebyshev’s inequality,
=O(K’).
constant
rVar
(6.1)
c2 such that N,, 1 c2nJn. By Lemma
(cn~‘n~,)2=0(1)
6.1 and
as c-w.
i (Note that 6, - n -‘.) Therefore N,’
;
[I’-Qt;)]
=0&P).
(6.2)
II The conclusion
of the theorem
follows
from (6.1) and (6.2).
Y. K. Truong / Semiparametric
178
Proof of Theorem
2.2.
le(tj)-e(t)l
By Condition SM,
If;-tl
repeated measurement
models
2.1, IMa&,
iEZ,@),
tE[O, I].
Thus N,(t))’ Set z,(t)
c [W;)-e(t)] I,,(0
= CiE,n(f’ [I:-e(ti)]. E[Zi(t)]
IM()6,,
By Lemma
= O(nd,)
te [O, 11.
ieZ,(t),
(6.3)
6.1, over t e [O, 11.
uniformly
Consequently, dr=O(n&). for t E [0, 11. Hence
Recall that N,(t)rc&,
(6.4)
by (6.4) and Markov’s
inequality
P
(6.5) It follows
from (6.3) and (6.5) that lim limP(I~8,-el~2~c(~n+(n-‘6,1)“2))=0. c+rn n
The conclusion or equivalently,
of the theorem 6, = n Pr.
Proof of Theorem 2.3. L, 1 [,;@+r’ log n]. Let whose coordinates is of [0, I] can be written as and all of its vertices in w such that t E Q,v. Let
now follows by choosing
6, so that 6, = (nm’s,‘)“2,
Let T be a positive constant such that 0< t< 1 and set W, be the collection of (L, + 1) points in [O, l] each of the form j/L,, for some integer j such that 01j~ L,. Then the union of L, subintervals, each having length A,, = L,’ W,. For each t E [0, l] there is a subinterval Q, with center C, denote the collection of centers of these subintervals.
Then P (
Izbpl, 1$(t)-e(t)1 >
=P ( It follows large)
ZC(H-'
i0gn)'
>
~~7 ,“~“t,p&(t)-e(t)1 n n
from An- 6:+‘/log le(t)-e(bq
n = o(6,)
rM,lt-wl
Therefore, to prove the theorem, tant c such that
IC(K'
>
and Condition
244,6,, it is sufficient
.
i0gn)r
tEQw,
2.1 that (for n sufficiently WEC,.
to show that there is a positive
cons-
Y. K. Truong / Semiparametric
repeated measurement
FEa; I”EU~,10n(r)-O(w)l 1imP n n ,* ( Set r,=In(w)= 8(w)=ave(
179
models
=O.
2c(n-‘logn)’
(6.6)
>
{i: 1
Na=Nn(w)= from
I$: iota},
#f,(w)
=0
and
(6.7)
i and ~,“c” /On(w)-d(w)l limp n 1, (1
)
To verify (6.7) and (6.8), set yn=iJIfl(w)= are positive constants cs and c, such that N~(w)-~,,(w)~cs~,~+~ By Lemma
and
6.2, there is a positive
(6.8)
=0.
~c(n-~logn)'
constant
#{i:
It,- WI 16,-A,}.
Nn(w)~c4ndn
Then
there
for all WEC,.
cs such that
P (I Note that there is a positive
constant
K
such that
#c, 5 nK. Hence
P
5 2n” exp( - c2c,na i). Since nS3, -log P Consequently,
n, we conclude NR(w)-’
that,
for c sufficiently
C [ x-O(t,)] i,(w)
large, 52nKexp(-c2
log n).
for c2 > K, (6.9)
Observe
that (6.8) follows
from (6.3) and (6.9).
Given t E [0, 11, set N, = N,(t) y,,
Thus
and Z, = I,(t)
and choose
w such that t E Q,,. Then
Y.K. Truong / Semiparametric
180
repeated measurement
models
and hence
Since
# C, I nK, it follows
from Chebyshev-type
5
inequality
P
involving
max max ( WEC,, I s,sn
=O(l)n’+KE/Y,I for k sufficiently theorem.
large.
Hence
(6.7) is valid.
This
E j Y, 1k that
2CsS;‘+r k
>
‘Lk (log n)k = O(l)
completes
the proof
of the
for all zE@ such that 1~15 1, Proof of Theorem 3.1. Since 1-6rz... -b,zp#O it follows from Theorem 3.1 .l of Brockwell and Davis (1987) that there exists a sequence (ai)ick such that C, Ia,1 < 00 and z;=
c a,_&,,. U
Thus by Theorem
2.3,
II&C.) -
et.)lI, = qm-’
(6.10)
log(n)I
Note that z;=
Y--&Jr;)
=z;-
(e,(t,) - &t;)).
Thus -~i+h~8n(~i)-~(~;)l-~;~B,(f,+~)--(~,+,)l
4Z+h=Z,Zi+h
+]e,(~,)-~(t,)l]&(~;+,)-O(t;+h)l, By Condition
(Proof
h=O,l,...,p-1.
2.1, (3.1) and (3.2),
n-’
c z;+,[&(t,) - fw,)]
.-’
c z;[&(t;+,)-W;+/J]
=
of (6.12) will be given shortly.)
O,((n~’ log n)lr), =0&KI
(6.12)
1ogn)“r).
Set
n-h
c
&(h)=(n-h)_’
h C .Z?,.Z!i+h, i=l
II
Z;Z;+h
and
i=l
9,(h)=(n-h))’
h=O,l,_.., p-l. Then
(6.11)
by (6.10)-(6.12),
n-h p);,(h) n
n-h = ~~~(h)+0,((n-110gn)2r), n
h=O,l,...,p-1.
Y. K. Truong / Semiparametric
That
repeated measurement
181
models
is, 8,(h)-~,(h)=o,(n~“2),
h=O,l,...,p-1.
If 6 denote the estimator of b obtained Yule-Walker estimator of b. Then
by regressing
Z, on Zi_,, . . ..Z._,,
i.e. the
b^-6=o,(n-“2). Consequently,
by Proposition
8.10.1 of Brockwell
and Davis (1987),
fi(b-b)d(O,o’T,‘). Proof of (6.12). Note that
W,.hN
n-l c Z;[&(t;+,)-
+n-’
c z,
N
n
(i;.,,, k jxt _) te(fk)- Wi+/?>l. E ,I / ii
(6.13)
Let m = m, = InS,,]. Then 2m-l5N,(t;)l2m+l, Put y(i)=E(Z,Z,+j),
i=O,l,...,
n.
(i=O, i: l,... ). By Theorem
qn-1p(
l
2.3.2 of Brillinger
(1981),
Zkj1_
c
~n@i+h)kEI,tU,+h)
+
Idi-0
W-A
=$ (,j;n(l-nml Ii11 lY(i)l)2=O($), since
Ci ly(i)l < 03. By m = [nS,] - log(n)(n-’
‘-’
7 “(&(:i+i,)
k&+/J
‘k
>
log(n)))2’
and Markov’s
inequality,
Y.K.
182
=
Truong
/ Semiparametric
O,((K’ log n)2’),
repeated
measurement
models
h = 0, 1, . . . ,p - 1.
(6.14)
This takes care of the first term of (6.13). The second term will now be considered. According to Condition 2.1,
M;Jt,
c EZ;+A4;6;
c ;
c lE(Z;ZJ J
= O(ns&
(6.15)
as 1; C, lE(ZfZj)l 5 Ci C, laiPul Cj laj_ulo*Sn ll~ll’o*. It now follows from (6.13)~(6.15) and 6, - (n-l log n)’ that (6.12) holds. This completes the proof of the theorem.
Acknowledgements The author is grateful to thank Chuck Stone,
to David Brillinger for the use of DRBLIB. Also, he wishes Dennis Cox, Editors and referees for helpful comments.
References Brillinger,
D.R.
Brockwell,
P.J.
(1981).
Time Series: Data Analysis
and R.A.
Davis (1987).
Chiu, S.T. (1989). Bandwidth
selection
Time Series:
and Theory. Theory
for kernel estimate
Holden-Day,
and Methods.
with correlated
San Francisco,
CA.
Springer,
New York.
noise. Statist.
Probab.
Lett.
8, 347-354. Gasser, T., H.G. Miiller, W. Kchler, L. Molinari and A. Prader analysis of growth curve. Ann. Statist. 12, 210-229. Fuller,
W.A.
(1976). Introduction
to Statistical
Time Series Analysis.
Hannan,
E.T.
(1970). Mulfiple
Time Series. Wiley,
Hannan,
E.T.
and B.G. Quinn
(1979). The determination
Statist. Sot. Ser. B 41, 190-195. HBrdle, W. and D.T. Pham (1986). Some theory
(1984).
Nonparametric
Wiley,
New York.
New York. of the order
on M-smoothing
of an autoregression.
Stark-r. Assoc.
measurements
7,
data.
J.
81, 1080-1088.
Kritchevsky, S. (1989). Lipid changes ment of Epidemiology, University
preceding the diagnosis of cancer. Doctoral of North Carolina, Chapel Hill, NC.
LRC-CPPT
Clinics
(1984).
J. Roy.
of time series. J. Time Ser. Anal.
191-204. Hart, J.D. (1989). Kernel regression with time series errors. Preprint. Hart, J.D. and T.E. Wehrly (1986). Kernel regression estimation using repeated Amer.
regression
Lipid
Research
Program.
Lipid
research
clinics
Dissertation,
primary
Depart-
prevention
trial
results: I. reduction in incidence of coronary heart disease. J. Amer. Medical Assoc. 251, 351-364. Liitkepohl, H. (1985). Comparison of criteria for estimating the order of a vector autoregressive process. J. Time Ser. Anal.
6, 35-52.
Y.K.
Miiller,
H.G.
Statist. Miiller,
Truong
and U. Stadtmiiller
/ Semiparametric
(1987). Estimation
measurement
of heteroscedasticity
183
models
in regression
analysis.
Ann.
15, 610-625. H.G.
and
Biometrika
U.
Stadtmiiller
(1988).
Detecting
C.J.
dependencies
in smooth
regression
models.
75, 639-650.
Roussas, G.G. (1988). Nonparametric regression Seber, G.A.F. and D.J. Wild (1989). Nonlinear Stone,
repeated
(1980).
Optimal
rates
estimation Regression.
of convergence
for
under mixing conditions. Wiley, New York.
nonparametric
estimators.
Preprint. Ann.
Statist.
8,
Ann.
Staiist.
10,
1348-1360. Stone,
C.J. (1982). Optimal
global
rates of convergence
for nonparametric
regression.
1040-1053. Truong,
Y.K. and C.J.
pear in Ann. Wong,
W.H.
Stat&r.
Stone (1988). Nonparametric
function
estimation
involving
time series. To ap-
Statist. (1983). On the consistency
11, 1036-1041.
of cross-validation
in kernel nonparametric
regression.
Ann.