On unbiased density estimation for ergodic diffusion

$1'Al1$11C8 & PROBAIIILITY LETTERS ELSEVIER

Statistics & Probability Letters 34 (19971 133-14(I

On unbiased density estimation for ergodic diffusion Yu.A. Kutoyants Laboratoire de Statistique et Processus, Unit'ersitk du Maine, B.P. 535, 72017 Le Mans cedex, France Received April 1996; revised September 1996

Abstract

Two classes of unbiased estimators of the density function of ergodic distribution for the diffusion process of observations are proposed. The estimators are square-root consistent and asymptotically normal. This curious situation is entirely different from the case of discrete-time models (Davis 1977) where the unbiased estimator rarely exists and usually the estimators are not square-root consistent. Keywords: Diffusion process; Nonparametric estimation; Density function estimation; Unbiased estimator; Asymptotic normality

I. Introduction

We consider the problem of density estimation by the observations of diffusion process in the situation when it is possible to have an unbiased square-root consistent estimator. It is known that in discrete-time case the density estimators usually have bias and the choice of optimal rate of convergence can be found as a balance of the random term (variance) and the term concerning the bias (Devroy and Gy6rfi, 1985). The situation in continuous-time models of observation is different. The problem of density estimation for ergodic diffusion process was studied by Banon (1978) and N'Guyen (1979). Castellana and Leadbetter (1986) showed that if the observed stochastic process { Yt, 0 <~t <~T} is stationary and the condition ! f . , ( u , v ) - f ( u ) f ( v ) l < ~g(r) E Ll(O, cx~)

(1)

is fulfilled then the wide class of estimators (including kernel-type estimators) are square-root consistent and asymptotically normal. Here f ( - ) and fr(., .) are one- and two-dimensional density functions which are supposed to be continuous at the point x (the function f~(u,v) is continuous at the point u = v = x for r > 0). Inequality (1) contains two conditions. The integrability in the vicinity z = 0 which corresponds to irregularity o[ the trajectories (the sampling collects a whole continuum of"somewhat independent" random variables, see Castellana and Leadbetter, 1986, p. 189) and the mixing-type condition for the large t. The rate v/T is possible for the kernel-type estimators if the trajectories Yt, t>~0 are not differentiable. Bosq (1996) showed that if the observed process belongs to a class of processes containing stationary Gaussian processes and has differentiable trajectories then this rate of convergence for the kernel-type estimators is unattainable. Condition (11 for ergodic diffusion processes were studied by Leblanc (1995) and Veretennikov (1996). 0167-7152/97/$17.00 @ 1997 Elsevier Science B.V. All rights reserved PIIS0167-7152(96)00174-5

Yu.A. Kutoyants I Statistics & Probability Letters 34 (1997) 133-140

134

Leblanc using a local version of inequality (1) (the majoration (1) is valid for the u,v from bounded sets) and wavelets, constructed an estimator having the same rate of convergence in L2-norm. Then Kutoyants (1995) proposed a lower minimax bound on the risks of all density estimators and the asymptotic normality of the kernel-type f r ( x ) and an unbiased f ~ ( x ) (see below (4)) estimators were established. For example,

£:{v~(fr(x) -

f ( x ) ) } ~ .,~:(0,I, ~),

where I . (playing the role of Fisher information) is given below. Note that in Kutoyants (1995) condition (1) was not checked and all conditions (as well as limit variances) are given in terms of ergodic (one-dimensional) distribution. The existence of x/T-consistent estimators make this problem of nonparametric estimation in certain sense similar to the problems of parameter estimation. This problem can be called following (Bickel, 1993) semiparametric because we estimate the value of the density at one point, i.e., one-dimensional parameter 0 = f ( x ) , having a nonparametric uncertainty. Recall that, if the trend coefficient is known to the observer function depending on some unknown parameter 79, i.e., S(x) = S(O,x) in (2) below then under regularity conditions ~,~{v/-f(0r - 0)} ~ J ( 0 , 1 ( 0 ) -~ ), where 0 r is a maximum likelihood estimator and I(0) is a Fisher information of the problem (Kutoyants, 1984, Theorem 3.4.4). In parametric problems there are a lot of unbiased estimators (see, e.g., Voinov and Nikulin, 1993) as well as in certain nonparametric estimation problems like distribution function estimation. In density estimation problem the situation is quite different. The unbiased estimator f * ( x ) of a density function f ( x ) in i.i.d, case was studied by Ibragimov and Khasminskii (1982). Assuming that the characteristic functions has a finite support, they constructed a lower bound on the risk of all estimators and showed that the unbiased estimator attains this bound. Moreover, it was shown that this estimator is asymptotically normal with the parametric

rate: 2'{x/-n(f*(x) - f ( x ) ) } ~ ./V(O,R(x)). Recall also that Watson and Leadbetter (1963) noted that this rate of convergence for the kernel-type estimators is possible if and only if the characteristic function has a compact support. In the present work we propose two classes of unbiased estimators of the density function f ( x ) of stationary distribution of ergodic diffusion process. The regularity conditions are less restrictive, say, the continuity of the function f ( x ) is sufficient for our purposes, but of course, this result is proved for another model and has another nature. The estimators constructed are asymptotically normal with the usual parametric rate v ~ . Let the observed continuous-time process {Xt, O<~t<~T} be diffusion

dXt = S(Xt)dt + a(Xt)dWt,

Xo, O<<,t<~T,

(2)

where Wt, t>~O is standard Wiener process, the initial value X0 is independent of Wt, t~>0 random variable, a(.) is known positive function and the trend coefficient S(.) is unknown to the observer. We suppose that the measurable functions S(.) and a(.) satisfy the conditions of existence and uniqueness of the solution of Eq. (2) (see, for example, Liptser and Shiryayev, 1977, Theorem 4.6 or any other book on stochastic differential equation). We will also assume that R0. The functional

G(S) =

~c a(y) -2 exp 2

~

av~ dy < ec,

(3)

Yu.A. Kutoyants I Statistics & Probability Letters 34 (1997) 133 140

135

or(y) 2 > 0 for all y C ~ and moreover, • as

v

" S(v)

~dt'~

-~c

--~ q ( .

This condition provides the existence of stationary distribution function a(y)-2exp

F(x) = G(S) -I

2

dv

dy

for the diffusion process (2) (see Mandl, 1968). We denote as f(x) its density function. In the following, we are interested in the problem of nonparametric (S(.) is unknown) estimation of this density function by observations {X,, 0 4 t < ~ T } as T--+ x . The Fisher information I, is I. =

a(~)f(~)

4J'(x)2 E \

where the random variable ~ has F(.) as distribution function. 2. Unbiased and consistent estimators

Below we suppose that the initial value X0 = ~ so the stochastic process {Xt, t>~0} is stationary. The unbiased estimator of the density function for the model (2) (proposed by Kutoyants, 1995) has the following form: ./.)*(X) - - T o . 2 (x) 2

fz

Z{x,<~ }

dX,.

(4)

if we suppose that EIS(~) ] < .'~ and Ecr(~)2 < vc, then its mathematical expectation is Elr*(x) = 2E(z{~,-x } S(c~))

2

~ S(y) exp 2

G(S)a(x) 2 _~ a(y) 2

- G ( S ) a' ( x ) 2 . _F. . ~ e x p { 2 f0' ~s d' )v

a(v) 2

J

dy

} d ( 2. £ ' s ~' ' d' v

)

S(y)x -

,r(x)2

-~,:

=

(5)

f(x).

Therefore this estimator is unbiased. Under certain additional conditions it was shown that this estimator is asymptotically normal, i.e., E~

{,,v<*(x)-

(see Kutoyants, 1995, Theorem 3).

I)

Yu.A. Kutoyants / Statistics & Probability Letters 34 (1997) 133-140

136

The two classes of unbiased estimators 3 f proposed below are described by two (almost) arbitrary measurable functions 91(') and 92('). Let us define the functions hi(.) and h2(.) by the equalities h~(y)=

-~dv, fooy aOi(v)

y E ~, i = 1,2

and suppose that R1. The function gl(') is such that the integral hi(y) exists for all y E R and

EIo1(¢)I <

lim hi(y) a(y)2f(y) = 0,

lyl~oo

oc,

E(hl(~) 2 + X{¢
y--..~

E(7.N
E(h2(~)a(~)Z{~
fr(l)(x) _ fr(2)(x) -

2 r + tr(x) 2 hl(Xt)] dXt + ~,/0 T01(Xt) dt, Ta-(x) 2 fo [Z{x'
Ttr(x )2h2(x )

{ 2 f o r X{X,
XlX,
•

The main result of this note is Theorem 1. Let conditions R0, Ri be fulfilled, then the estimator fT(i)(x) is unbiased and consistent, i = 1,2. Proof. For the first estimator we have (see (5))

E fr(l)(x) = f ( x ) + 2Eh1(¢)S(¢) + Egt(~), because by condition R I E

[X{X,
and therefore (see Liptser and Shiryayev, 1977), E

[X{x,<~/+ e(x)2h~(Xt)lo(X,)d~ = O.

Integrating by parts provides

2eh,(e)s(e)=2o(s)_, f_ = G ( S ) -l

h,(y)S(y)tr(y) 2 hl(y)d

oc

e x p ( 2 Jo[y a-~av~S(v)" ) dy

exp 2

_t-72~-,~2dv

Yu.A. Kutoyants / StatL~tics& Probability Letters 34 (1997) 133-140 = hl(y)rr(y)2f(y)._~[~_ -

/5

137

h'l(y)a(y)2 f ( y ) d y

OG

= -Eq~(~.). Therefore, E.li~.ll(x) = f(x). For the second estimator, in a similar way, we have Efz~.2~(x) = a(x)-2h2(x) -I {2E~{~
2Ez{~
L '

,

"

h2(y)S(Y)exp

~


{

2

dv .

}

dv

= {h2(x)a(x)2j(x)- .Lr h~(Y) er(y)2 f ( y ) d y } = h2(x)a(x)2f(x) - Ez{~
fT(X) = -~

Rx(Xt)dX, + ~

Qx(Xt)dt

if r[Rx(Xt)S(Xt) + Qx(Xt)]dt + -~if'

= -~

Rx(Xt)rr(Xt)dWt

(6)

with corresponding functions Rx(.) and Qx('). We have checked already that

E[Rx(X,) S(Xt) + Qx(Xt) ]

=

f(x).

Hence, by large numbers law [13] with probability 1,

rlira ~ T1 fo 7 [R.,(Xt ) S(Xt) + Qx(Xt )] dt = f(x). For the stochastic integral we have with probability 1 (Liptser and Shiryayev, 1977, Lemma 17.4), P-

lim T1. /a y Rx(X,) rr(Xt)dWr = O. r-:,c

From these two limits the strong consistency of both estimators follows.

3. Asymptotic normality To study the normed difference x / 7 ( f r ( x ) - f ( x ) ) we shall use the following central limit theorem for two integrals: ordinary and stochastic. Let the vector function m ( y ) = ( h ( y ) , g ( y ) ) where the measurable functions h(-) and g(.) be such that EIh(#)l

< oc,

Eh(~) = O,

Eg(~) 2 < oo

138

Yu.A. Kutoyants I Statistics & Probability Letters 34 (1997) 133-140

and the integrals 4G(S) f _ ~ h(Y) oo

h(z) f ( z ) p(s) f ( y ) d z d s d y - D(h,h) < oc, oc

g(z) f ( z ) p(s) f ( y ) d z d s d y <

4G(S) f_~c O(2

converge absolutely. The class of such functions m(.) we denote as ~¢. Introduce also the quantity Dh, o = 2Eh(~)

fo ~~g(s) ds

and put Dg, y = Eg(~) 2.

Lemma 1. Let condition R0 be fulfilled and the function m(.) = (h(.), g(')) C J[, then the vector {T-l/2foTh(xt)dt,

T-X/2forg(xt)dWtt }

is asymptotically normal with the limit covariance matrix D2 = ( D(h,h) Dh, y

Dh, o ) D o,g "

Proof. See Kutoyants (1995, Lemma 4). By this lemma, the sum

~r = T-1/2

/0

h(Xt)dt + T -1/2

/0

g(Xt)d~

under the same conditions is asymptotically normal, Ae(r/T) ::> .~(0,

D2.),

with the limit variance D~ = D(h,h) + 2Dh,.q + Do, o. Below,

hx(y) = Rx(y)S(y) + Qx(y) - f(x),

gx(y) = Rx(y)a(y),

where the functions Rx(.) and Qx(') are taken from the representation (6). Theorem 2. Let the conditions R0-R3 be fulfilled and the function rex(.) = (h,(.),gx(.)) 6 ~¢ then the unbiased estimator Jr(x) is asymptotically normal: .Lf{v/T(J~r(x)- f(x))} ~ JV(0,D}),

(7)

where D} = D(hx, hx) + 2Dh,.o, + Egx(~) 2.

(8)

Proof. According to (6), we can write

v/-T(fr(x) - f ( x ) ) = r -1/2

/0

hx(Xt)dt + T -1/2

9x(Xt)dWt. dO

Recall that Ehx(~) = 0. Therefore, the convergence (7) follows directly from the Lemma 1.

Yu.A. Kutoyants I Statistics & Probability Letters 34 (1997) 133-140

139

Put a = 1 and b = 0 then we have the asymptotic normality .~q{ x/T(fr(1)(x) - f ( x ) ) } ~ ~P(0,D~). The expression for the limit variance D 2 is quite cumbersome. It is calculated by formula (8) with 2S(y) hx(y) = gl(Y) - f ( x ) + a - - ~ Z l y < x } + 2 h l ( y ) S ( y ) ,

gx(y)-

2a(y) a(x) 2 7,(y
and, say, 4

Dq, o - a(x)4 E(ZI~
O(hx, hx ) = EG(~)2a(~) 2, where 2 G(y) - a(y)2 f(y)ET, N
1 and S ( y ) sgn(y) < - y where -? is some positive constant. Then condition R0 is fulfilled and we can take gl(') and (or) g2(') as any polynomial (assuming of course that h2(x) 7~ 0). Conditions RI, R2 will be fulfilled too. Put g l ( y ) = g2(y) = y then the estimators Example. Let a ( y ) =

,/0

f ( l ) ( x ) = "T

fr(2)(x) =

2

[2Z{x,
r

)~{X,
St dt,

Z{X ,
dt

will be unbiased, consistent and asymptotically normal (the conditions of Theorem 2 are fulfilled).

Remark. The following question naturally arises. Is it possible, to improve the estimator f r ( x ) by choosing the appropriate functions gl('), g2(') and constants a, b? As it was shown by Kutoyants (1996), the estimator f ~ ( x ) as well as the kernel-type estimators are asymptotically efficient.

Acknowledgements

The author would like to thank the referee for useful comments. References

Banon, G. (1978), Non parametric identification for diffusion processes, SIAM J. Control Optirn. 16, 380-395. Bickel, P.J. (1993), Estimation in semiparametric models, in: C.R. Rao, ed., Multivariate Analysis, Future Directions (Elsevier, Amsterdam).

140

Yu. A. Kutoyants I Statistics & Probability Letters 34 (1997) 133-140

Bosq, D. (1996), Nonparametric statistics for stochastic processes, Lecture Notes in Statistics, i10, Springer, New York. Castellana, J.V. and M.R. Leadbetter (1986), On smoothed density estimation for stationary processes, Stochastic Proc. Appl. 21, 179-193. Davis, K.B. (1977), Mean integrated square error properties of density estimates, Ann. StatLst. 5, 530-535. Devroy, L. and L. GySrfi (1985), Nonparametric Density Estimation. The LI View (Wiley, New York). Ibragimov, I.A. and R.Z. Khasminskii (1982), Estimation of distribution density belonging to a class of entire functions, Theory Probab. Appl. 27, 551-562. Kutoyants, Yu.A. (1984), Parameter Estimation for Stochastic Processes (Heldermann, Berlin). Kutoyants, Yu.A. (1995), On density estimation by the observations of ergodic diffusion process, Universit6 du Maine, Preprint N 8, January, 1995. Kutoyants, Yu.A. (1996), Some problems of nonparametric estimation by observations of ergodic diffusion process, Statist. Probab. Lett. Leblanc, F. (1995), Wavelet density estimation of a continuous time process and application to diffusion process, C.R. Acad Sci. Paris, t. 312, Sirie 1, 345-350. Liptser, R.S and A.N. Shiryayev (1977), Statistics of Random Processes, 1 (Springer, New York). Mahdi, P. (1968), Analytical Treatment of One-Dimensional Markot~ Processes (Academia, Prague; Springer, Berlin). N'Guyen, H.T. (1979), Density estimation in a continuous-time Markov processes, Ann. Statist. 7, 341-348. Veretennikov, A.Yu, On Castellana-Leadbetter's condition for the diffusion density estimation, submitted. Voinov, V.G. and M.S. Nikulin (1993), Unbi~ed Estimators and their Applications, Vol. 1 (Kluwer, Dordrecht). Watson, G.S. and M.R. Leadbetter (1963), On the estimation of the probability density, I, Ann. Math. Statist. 34, 480-491.