Statistics & Probability Letters 5 (1987) 279-285 North-Holland
NOTE
ON THE UNIFORM
June 1987
CONVERGENCE
OF DENSITY
ESTIMATES
F O R M I X I N G R A N D O M VARIABLES
D. I O A N N I D E S
University of Patras, Patras, Greece G.G. ROUSSAS
University of California, Davis, CA 95616, USA Received April 1986 Revised October 1986
Abstract: On the basis of the random variables X1..... Xn drawn from the (strictly) stationary and %-mixing (for some i = 1,... ,4) stochastic process { X. }, n >/1, a uniformly strongly consistent estimate of the (common) probability density function of the X's is constructed. For the case that the underlying process is also Markovian, uniformly strongly consistent estimates are constructed for the initial, the (X1, X2)-joint and the transition probability density functions of the process. AMS 1970 Subject Classifications: Primary 62G05; Secondary 62M05. Keywords: kernel estimates, strongly consistent estimates, mixing random variables, Markov processes.
1. Introduction Let f be the ( c o m m o n ) p r o b a b i l i t y d e n s i t y function (p.d.f.) of the X ' s a n d let f ~ ( x ) be the usual kernel estimate of f ( x ) defined b y j~(x) = ~
~2 K j=l
,
(1.0)
where 0 < h = h , $0 a n d K is a b o u n d e d p.d.f. satisfying certain a d d i t i o n a l conditions to be stated below. T h e m a i n result of this note is that f 2 ( x ) converges almost surely (a.s.) to f ( x ) u n i f o r m l y in x ~ R'. Below, the a s s u m p t i o n s are given u n d e r which this result is derived.
Assumptions (A1) The r.v.'s X1, X 2 . . . .
take values in Rt,
t >~ 1, a n d form a strictly s t a t i o n a r y sequence, {Xj}, j>~ 1. (A2) Let h = n 0, 0 > 0 , a n d let K be a b o u n d e d (by M ) p.d.f, defined on R'. ( E l ) Let c~ = a ( n ) be a positive integer a n d let /~ = / x ( n ) be the largest positive integer such that 2a/x ~< n. (E2) (i) U n d e r ~,-mixing, set q~* = ~o~ j=l ~i(J) a n d suppose that q~* < oo for i = 1 . . . . . 4, respectively. (E3a) U n d e r ~i-mixing, i = 1, 2, s u p p o s e that the integers a a n d ~t in ( E l ) also satisfy the requirement lim sup[1 + 2 el/2¢i ( a ) ] ~ < ~ . (E3b) U n d e r q~;mixing, i = 3, 4, s u p p o s e that the integers a a n d /~ in ( E l ) also satisfy the requirement lim sup[a + 6 el/2@i/("+ 1)(ot)]~ < ~ .
This work was partially supported by a grant from the Office of Research and Technology of Greece.
(Mi) Same 1,2,3.
0167-7152/87/$3.50 "C 1987, Elsevier Science Publishers B.V. (North-Holland)
as (E2)(i)
under
q~;mixing,
i=
279
Volume 5, Number 4
STATISTICS & PROBABILITY LETTERS
June 1987
Under the assumptions and provisos in Theorem 1.2, the convergence in (1.2) holds, provided 3' > (3 + Ot)/r.
and
The p r o o f of Propositions 2.1 and 2.2 is comm o n to a large extent and goes as follows. Write kn instead of n v, to simplify somewhat the notation, and observe that
Indeed, by letting 3' > 3, and using (F1), one has
2.2.
Proposition
sup Ilxll >k,, ~<
~ O.
sup n ~ f ( x ) = tl > k .
rl-~
n lL(x)-f(x)l
sup Ilxll >k. +
sup n~f(x) IIx tl >/,,,
n~j~(x)+
(2.4)
sup
nSf(x)<~
nVf(x)
sup It ~ II> n~
IPx It > , ~
sup Ilxllf(x)--'O. IIx II> n~
sup nSSf~(x) Ilxll >k.
Then, for all sufficiently large n, (2.2) becomes by means of L e m m a 2.1,
sup Ilxll >k.
also, under (A1)-(A2), (UF) and (UK1), the last term on the right-hand side above is b o u n d e d by Cln ~-° ( L e m m a 3.1 in Roussas, 1986a) which is < e / 2 (for some t > 0) for all sufficiently large n, provided 3 < 0. Then, for all such n 's,
P{ Ilxll>k.SUpn ~ ] ~ ( x ) - f ( x ) [ > ~ e ]
P[NxH>k°sup n S [ £ ( x ) - - f ( x ) l > e ] P
sup n S ~ ( x ) > ~ IIxll>k,
For simplicity, set Kj for fine Aj by
Aj=A.j(x)= Ilxll >k~
j=l
{
.
(2.5)
K ( ( x - Xy)/h)
flXj(,
and de-
ko}
)-xll < 5 - '
. . . . . n, x ~ R ' .
Then where, of course, the second term on the right-hand side above is either 0 or 1. Actually, the former happens because
P IIx H> ,~,
L e m m a 2.1. Under (A1)-(A2), (UF), (F1), (UK1), 0 ~< 8 < min(3', 0), one has
and for
][x]] >k,,
sup n~o~f~(x)-~O. Itxll >kn
(2.6) llx]l > k .
Proof. F r o m (1.0) and by means of (A1)-(A2), ( U F ) and (UK1), one has sup Ilxll >k. ~<
j=l
The terms on the right-hand side of (2.6) behave asymptotically as in the following lemmas.
nS~)~(x) With h as specified in and for all suffieiently large n, L e m m a 2.2.
sup n S [ K ( z ) l f ( x - h z ) - f ( x ) l Flxll>kn "tR +
j=l
sup nSf(x) IIxll >k.
<~Cln ~-° +
sup n~f(x), IIx II> k.
dz
P[
sup HxH>k,,
(A2),
under
(K),
n S ~ KjlA,,>~enht/8]=O, j=l
provided 7 > 8 + ( t -
1)0 (6 >~ 0). 281
Volume 5, Number 4
STATISTICS& PROBABILITYLETTERS
(M4) Under ~4-mixing, suppose that Y~j~-I (J + 1) t/2 l~b4(J) < ~ for some suitable (real number) l > 2 (see relation (1.4)). (UF) For any X l = ( X u . . . . . Xlt ) and x 2 = (x21 . . . . . x2, ) in R' and for some positive constant C 1, it is assumed that
1, 2, 3, the convergence in (1.1) holds for any 3, > 0, provided (Mi) holds, respectively, for i = 1, 2, 3, and also provided the first relation in (1.2) holds. Under eo4-mixing, the convergence in (1.1) holds, provided (M4) holds and also (6+0t)+
t
I f ( x a ) - f ( x 2 ) [ ~< C1 E I x 1 , - x2,1i--1
(F1) [ I x l l f ( x ) - - ' 0 as Ilxll ---' ~ , where the norm of x = (x 1. . . . . x,), II x 11, is defined by IIx II = max(lXll . . . . . Ix, I). (F2) fn' I l x l l r f ( x ) d x = g r < oo, for some 0 < r (~<1). (UK1) fn' II x II K ( x ) d x < ~ . (UK2) Let xl, x 2 and C2 he as in (UF). Then t
June 1987
l-2 0~<7< T ,
t [ 8 + O ( t + l)] < l_-_ 2 l 2l
3't l'
•>2.
(1.4)
In particular, if qSa(n) $ 0 exponentially fast, then (1.4) may be replaced by a+0t<½,
3'>0.
(1.5)
The purpose of this note is to prove the following
I K ( x l ) - K ( x 2 ) I < C2 E [Xli-Xzil. i=1
(K) IlxllK(x)~Oas [I x [I ---' ~ . The assumptions listed above are the same as those in Sections 2 and 3 in Roussas (1986a) except for assumptions (F1), (F2) and (K). In the reference just cited, the following results (see Theorems 3.1 and 3.2) were established.
Theorem 1.1. Under the assumptions and provisos in Proposition 1.1, and also the additional assumptions (F1)-(F2), (K), it holds that
sup[n~lf~(x)-f(x)l;x~Rt]--*O
a.s.
(1.6)
Corollary 1.1. By dropping assumption (F1), the convergence in (1.6) becomes
Proposition 1.1. Let assumptions (A1)-(A2), (UF), (UK1)-(UK2) hold. Then, for any 3' >~O,
sup[l.~(x)-f(x)l;
sup[n~lf.(x)-f(x)l;
Theorem 1.2. Under the assumptions and provisos in Proposition 1.2, and also the assumptions (F1)-(F2), (K), the convergence in (1.6) holds true.
Ilxll ~n~]--,0
0~<8<0,
a.s.,
(1.1)
provided: 8+0t<½
and
(1.3)
Proposition 1.2. Let assumptions (A1)-(A2), (UF), (UK1)-(UK2) hold. Then, under q~i-mixing, i=
280
(1.7)
(1.2)
for any determination of a satisfying (El), (E2)(i) and (E3a), under epi-mixing, i = 1, 2; or (El), (E2)(i) and (E3b), under ~i-mixing, i = 3, 4, respectively. In particular, if a = [nP/2] (where [x] is the integer part of x), then the boundedness in (1.2) is fulfilled, provided O
a.s.
Corollary 1.2. By dropping assumption (F1), the convergence in (1.6) becomes as that in (1.7).
{na+°t/a}
is bounded away from 0
x~R']--,o
2. Proof of main results
In view of Propositions 1.1 and 1.2, in order tc establish Theorems 1.1 and 1.2, it suffices to prove respectively, Proposition 2.1. Under the assumptions and pro. visos in Theorem 1.1, it holds that
sup[n~lf~(x)-f(x)l; provided Y > (8 + Ot)/r.
Ilxll > n Y] - , 0
a.s., (2.1)
Volume 5, Number 4
STATISTICS& PROBABILITYLETTERS
any number >/2. For 8 + Ot < ½, which implies that 8 + Ot < (1 - 2)/2l for some l > 2, and hence l(½ - 8 - Ot) > 1, the Borel-Cantelli lemma gives the desired result again. Similarly under ~4-mixing and the conditions of Theorem 1.2. [] Proof of Propositions 2.1, 2.2 (and hence of Theorems 1.1, 1.2). It is an immediate consequence of
June 1987
are estimated by/3 n and qn, respectively, where /3n(x) = ---71 nh i~=1K ~
qn( ) = nh 2, j = l _
inequalities (2.2), (2.5), (2.6) and Lemmas 2.2 and 2.3. [3
1
,
x ~ R t,
--T---
(3.1)
,,+l)
° /x-L) EKl--g~
rlh 2t j = l
(3.z) Proof of Corollaries 1.1 and 1.2. The only point
where (F1) was used was in proving (2.4). Thus, one has to set ~ = 0 and establish (2.4) without the help of (F1). Clearly, supll~ll > k , f ( x ) = a, $ a > 0. If a > 0, then for every n, there would exist x, with II x, II > k, such that f ( x , ) > a/2. Let 8 = a / 4 t C v Then for x's with IIx II > kn and 11x x, I1 ~<& one has I f ( x ) - f ( x , ) I <~a / 4 (by (UF)). Hence, for all such x 's, f ( x ) > a / 4 and therefore (by (F2)),
> £11 x II rf(x)
dx
>/f II x II rf(x) d x J( IIxll>k. and IIx-x, II<6) a • (volumeof Sn= ( x ~ R ' ;
Ilxll >k~
and I I x - x ~ l l < 8 ) ) - , a contradiction.
~,
[]
;c(x, x ' ) ~ R
2', x ~ R ' ,
) ( , = ( X a , Xj+l), j > l , so that
b,(x)= fnfi,(x, x')dx',
x ~ R t.
The underlying Markov process, may be assumed to satisfy condition @ for some 1 ~ p ~< (see Rosenblatt, 1970). Then so does the process {Xj}, j > l , andboth, (Xj} and { ~ } , j > l , are ~3-mixing with mixing coefficient q53(n)= CO", some C > 0 and 0 < 0 < 1. (See Rosenblatt, 1971; or Corollary 3.1 in Roussas and Ioannides, 1987). Remark 3.1. At this point, observe that, for x,
x ' ~ R t and 2 = (x, x'): 112 II = max( II x II, II x' II) < II x II + II x' II,
LI211K*(2) ~ g ( x ' ) [ Ilx[l g ( x ) ] + K ( x ) [ ]1x'll K ( x ' ) ] ,
3. Some applications to Markov processes Suppose that the underlying process ( Xj }, j >/ 1, in addition to being (strictly) stationary and q~s-mixing, for some i = 1. . . . . 4, is also Markovian, and set J(j = ( X j , Xj+ 1). Then the process {)(j }, j >~ 1, is also (strictly) stationary, Markovian and q~,-mixing (with the same mixing coefficients). Let p and q be the p.d.f.'s of the r.v.'s X 1 and )(1, respectively, and let t(. [.) be the transition p.d.f. of the pair (X 1, X2). Thus, if 2 - - ( x , x ' ) ~ R 2' with x ~ N', then t ( x ' l x ) = q(x, x ' ) / p ( x ) . Let h and K be as in (A2), and let K satisfy conditions (UK1)-(UK2), (K); set K * ( 2 ) = K * ( x , x')= K ( x ) K ( x ' ) , 2 ~ R2t, x ~ R t. The p.d.f.'s p and q
f. ll2lIK*( ) d 2 ~ 2f.
llxllK(x) dx
and
K*(x2)l K(Xl) I K(x~) - K ( x ; ) I
+ K(x~)IK(Xl) -
K(x2) ].
From these relations, it clearly follows that, if K satisfies conditions (UK1)-(UK2) and (K), then so does K* The objective here is to establish uniform strong consistency for the estimates/3,(x) and 0n(x, x ' ) of the p.d.f.'s p ( x ) and q(x, x'), respectively, and 283
June 198"
STATISTICS & PROBABILITY LETTERS
Volume 5, Number 4
L e m m a 2.3. Under the assumptions of either Theorem 1.1 or Theorem 1.2, it holds
+ P na ~" SI~lqx~l I > k,,/2)>~ ~enh' ] , [
(2.7)
1= 1
where n = 1
II x IP > k,,
j=
! n
s,,= Eg°,
provided 7 > (~ + O t ) / r (~ >1 0). Proof of Lemma
j=l
2.2. F o r x ~ R t a n d j = 1 . . . . .
n,
c o n s i d e r AC,,j(x); on this event, one has
1
k, _ n v+e > 2h 2
gn j = 7 [ I( ll X, ll > k./2) -- g'l( ll X, II > k,,/2, ] (so that ~ g , j = 0 and tion (F2),
independent of oa ~ A~j(x),
j= l,...,n,
and
I g,)l < h - ' ) . By assump-
x ~ R t.
I[~ll
Therefore, for any x ~ R t, j = 1 . . . . . n, and any oa ~ A ~ j ( x ) , assumption (K) implies, for all n >/ some (generic) n o independent of x ~ R', j = 1 . . . . . n, and oa ~ A ,Cj ( x ) .
n a ~ ~I(Nx, ii>k./2) =ha Y'. P
I1( - x,)/h, K((x- X,)/h)<
and
Hence, for x, j, oa and n as just specified,
F o r such n's, relation (2.7) becomes
n K(x-X, ) ht
n;,-(t-1)----------~
1 ~ ] sup - Y'. IA).>~3 = 0 , Ilxll>k,, n j = l
since supll~ll >k, (1/n)Y'-~=l 1.4;~ < 1.
[]
P r o o f of L e m m a 2.3. For any x E R ' with II x II > k , , and j = 1 . . . . . n, one has, on the event A , j ( x ) : II Xj II >~ II x II - k J 2 > k , / 2 . Therefore sup Ilxll >k,,
~
j=l
sup Ilxll > / .
nh']
ne~
<~2%,n- nS/n vr
Ilxll >k.
I4> 8M]
j=l
~< P n ~ Y'~ I( IIx~ II > k./:) >/ 8M ] j=l
EY/
(2.8'
Now, under the assumptions m a d e in T h e o r e n 1.1, it follows that (see T h e o r e m s 2.1 and 3.1 ir Roussas and Ioannides (1986b) applied with k, appearing there taken equal to n) P n a ]S"I > n ~
-%
(2.91
where M a, m 2 are > 0 constants independent oJ n and q = 1 - 2(6 + Ot). The l e m m a then follow~ on account of (2.8), (2.9) and the B o r e l - C a n t e l l lemma. The l e m m a also follows in the sam~ manner, under the assumptions made in Theorem 1.2. In fact, under those assumptions and undeJ q~i-mixing, i = 1, 2, 3, it is known (see Roussas 1986b) that
p nsIS~h> [
j=l >e/16M).
(for y > 8 + (t - 1)0). Therefore, the probability on the left-hand side in the l e m m a is b o u n d e d by [
>
\
<-24
P
j--I
this last expression is
n 0, provided 7 > (8 + Ot)/r
<
\~-~----. < h, l l ( x _ X j ) / h l [
j=l
<
M3n-l-tS~l<
n
M 4 / n l(O/2)a-°t), where M 4 is a constant independent of n and l i~
282
Volume 5, Number 4
STATISTICS & PROBABILITY LETTERS
Rosenblatt, M. (1971), Markov Processes, Structure and Asymptotic Behavior (Springer-Verlag, Berlin). Roussas, G.G. and D. loannides (1986), Probability bounds for sums in triangular arrays of random variables under mixing conditions, Technical Report No. 85, University of California, Davis, Division of Statistics. Roussas, G.G. and D. Ioannides (1987), Moment inequalities for mixing sequences of random variables, Stochastic Anal AppL 5 (1). Roussas, G.G. (1986a), Nonparametric estimation in mixing sequences of random variables, Technical Report No. 60 (revised), University of California, Davis, Division of Statistics.
June 1987
Roussas, G.G. (1986b), A moment inequality of Sn.k, for triangular arrays of random variables under mixing conditions, with applications, Technical Report No. 84, University of California, Davis, Division of Statistics. Rfischendorf, L. (1977), Consistency of estimates for multivariate density functions and for the mode, Sankhy6 A 39, 243-250. Yakowitz, S. (1985), Nonparametric density estimation and prediction for Markov sequences, J. Amer. Statist. Assoc. 80, 215-221.
285