Uniform concentration inequality for ergodic diffusion processes

Stochastic Processes and their Applications 117 (2007) 830–839 www.elsevier.com/locate/spa

Uniform concentration inequality for ergodic diffusion processesI L. Galtchouk a,∗ , S. Pergamenshchikov b a IRMA, D´epartement de Math´ematiques, Universit´e de Strasbourg, 7 rue Ren´e Descartes, 67084, Strasbourg Cedex,

France b Laboratoire de Math´ematiques Raphael Salem, Site Colbert, UFR Sciences, Universit´e de Rouen, F76 821 Saint

Etienne, Cedex, France Received 17 February 2005; received in revised form 20 February 2006; accepted 10 October 2006 Available online 7 November 2006

Abstract We consider the deviation function in the ergodic theorem for an ergodic diffusion process (yt ) Z T ∆T (ϕ) = T −1/2 (ϕ(yt ) − m(ϕ))dt, 0

where ϕ is some function, m(ϕ) is the integral of ϕ with respect to the ergodic distribution of (yt ). We prove a concentration inequality for ∆T (ϕ) which is uniform with respect to ϕ and T ≥ 1. c 2006 Elsevier B.V. All rights reserved.

MSC: 60F10 Keywords: Ergodic diffusion processes; Tail distribution; Upper exponential bound; Concentration inequality

1. Introduction We consider the process (yt )t≥0 governed by the stochastic differential equation dyt = S(yt )dt + dwt ,

(1.1)

I This research was carried out in Department of Mathematics, Strasbourg University, France. The second author is partially supported by the RFFI-Grant 04-01-00855. ∗ Corresponding author. Tel.: +33 0390240193; fax: +33 0390240328. E-mail addresses: [email protected] (L. Galtchouk), [email protected] (S. Pergamenshchikov).

c 2006 Elsevier B.V. All rights reserved. 0304-4149/$ - see front matter doi:10.1016/j.spa.2006.10.008

L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

831

where (wt , Ft )t≥0 is a standard Wiener process and the function S satisfies the conditions under which this process is ergodic with ergodic density q. The ergodic theorem says that Z Z 1 T ϕ(y)q(y)dy a.s., ϕ(yt )dt = m(ϕ) = lim T →∞ T 0 R for any function ϕ for which the right-hand term is well defined (see e.g. [7]). Moreover, the central limit theorem for ergodic processes claims that Z T 1 ∆T (ϕ) = √ (ϕ(yt ) − m(ϕ))dt ⇒ N (0, σ 2 ) T 0 as T → ∞, where ⇒ means the convergence in law, σ 2 is a functional of Eq. (1.1) coefficients (see e.g. [1]). In this paper we prove a uniform concentration inequality for ∆T (ϕ): our main result says that sup sup

sup

T ≥1 ϕ∈Vν,l S∈Σ M,γ ,n

2

P S (|∆T (ϕ)| ≥ λ) ≤ 4e−κλ ,

where Σ M,γ ,n , Vν,l are functional classes defined in (2.1) and (3.3), the positive constant κ depends only on these classes. This upper bound possesses some advantages. Firstly, it is non-asymptotical, so it is true for any finite T . Secondly, the upper bound is uniform for functions ϕ and S from the given classes. The necessity of this type of upper bound arises, in particular, in problems of non-parametric estimation of the unknown drift S(·) in the Eq. (1.1) (see e.g. [5,8]). A widely used estimator of S(x0 ) at a fixed point x0 based on observations (yt , t ≤ T ), is a kernel estimator Sˆ T (x0 ) defined as RT h −1 0 Q((yt − x0 )h −1 )dyt Sˆ T (x0 ) = , (1.2) RT h −1 0 Q((yt − x0 )h −1 )dt where Q(·) is a kernel, hR = h(T ) is a bandwidth. T In the functional h −1 0 Q((yt − x0 )h −1 )dt the function ϕ(·) = h −1 Q((· − x0 )h −1 ) depends on the sample size T through the bandwidth h. So to study the asymptotic behaviour of this functional, one needs the given upper bound. In Section 3 we consider this application in detail. Concentration inequalities bound tail probabilities of some random variables. There are a number of papers devoted to concentration inequalities for functions of independent random variables (we refer the reader to [2] and references therein), for functions of dependent random variables (see [3,4,11]). Some applications of concentration inequalities to statistics are presented in [10]. Large deviation inequalities for diffusion processes are given in [13]. The paper is organized as follows. In the next section the statement of the problem is presented. In Section 3 we formulate the main results. In Section 4 we prove some moment inequality for solution of Eq. (1.1). The proofs of the main results are given in Section 5. The Appendix contains the proofs of some auxiliary results. 2. Statement of the problem Let (Ω , F, (Ft )t≥0 , P) be a filtered probability space satisfying the usual conditions and (wt , Ft )t≥0 be a standard Wiener process.

832

L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

Suppose that the observation process (yt )t≥0 is governed by the stochastic differential equation (1.1), where the unknown function S(·) satisfies the Lipschitz condition, the initial y value y0 is a constant. The Eq. (1.1) admits a strong solution. We denote by (Ft )t≥0 the natural filtration of the process (yt )t≥0 and by E S the expectation with respect to the distribution law P S of the process (yt )t≥0 under the drift S. Let M > 0, n > 0 and 0 < γ < 1. We define the following function class: ( ) ˙ Σ M,γ ,n = S : sup |S(z)| ≤ M, −1/γ ≤ S(x) ≤ −γ , |x| ≥ n . (2.1) |z|≤n

If S ∈ Σ M,γ ,n , then the process (yt )t≥0 possesses the ergodic density  Rx exp 2 0 S(z)dz q(x) = q S (x) = R +∞  Ry −∞ exp 2 0 S(z)dz dy

(2.2)

(see, e.g., [7], Ch. 4, 18, Theorem 2). It is easy to see that this density is uniformly bounded in the class (2.1), i.e. q ∗ = sup

q S (x) < +∞.

sup

(2.3)

x∈R S∈Σ M,γ ,n

Let now ϕ be some function such that ϕ ∗ = sup |ϕ(y)| < +∞

Z and

kϕk1 = R

y∈R

|ϕ(y)|dy < +∞.

(2.4)

We set Z m(ϕ) = R

ϕ(y)q(y)dy

(2.5)

and 1 ∆T (ϕ) = √ T

T

Z

(ϕ(yt ) − m(ϕ))dt.

(2.6)

0

The problem is to study the tail distribution of ∆T (ϕ) uniformly in ϕ. 3. Main results First, we consider functions ϕ from C(R) and set Vν,c = {ϕ ∈ C(R) : kϕk1 ≤ ν}.

(3.1)

Theorem 3.1. There is a positive constant κ = κ(M, γ , n, ν) such that sup sup

sup

T ≥1 ϕ∈Vν,c S∈Σ M,γ ,n

E S eκ ∆T (ϕ) ≤ 4. 2

(3.2)

Remark 3.1. Notice that this inequality was obtained in [12] but only for the process (yt )t≥0 from (1.1) with decreasing function S. This condition is very strong for applications.

L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

833

Now we set Vν,l = {ϕ ∈ Ł1 (R) : ϕ ∗ < +∞, kϕk1 ≤ ν}.

(3.3)

Theorem 3.2. There is a positive constant κ = κ(M, γ , n, ν) such that, for any λ > 0, sup sup

2

sup

T ≥1 ϕ∈Vν,l S∈Σ M,γ ,n

P S (|∆T (ϕ)| ≥ λ) ≤ 4e−κλ .

(3.4)

Application. Now we show how to apply this inequality to calculate an asymptotical minimax risk for the kernel estimator (1.2). Usually the minimax risk for estimators Sˆ T (x0 ) of S(x0 ) is defined as inf Sˆ T RT ( Sˆ T (x0 )), where RT ( Sˆ T (x0 )) =

sup

E S | Sˆ T (x0 ) − S(x0 )|.

S∈Σ M,γ ,n

Thus to calculate the asymptotical risk for the estimator (1.2) as T → ∞, we need to study RT the asymptotic properties of the functional h −1 0 Q((yt − x0 )h −1 )dt uniformly in S, where h = h(T ) is a positive function such that h(T ) → 0 and T h(T ) → ∞, as T → ∞. As it is established in [6] in some cases the optimal kernel is the indicator Q(z) = 1|z|≤1 . Therefore putting in this case   y − x0 1 ϕ(y) = ϕh (y) = Q h h we obtain that Z

1

dz = 2,

kϕh k1 = −1

i.e. this function belongs to Vν,l for ν ≥ 2. Moreover Z 1 q(x0 + zh)dz = 2q(x0 ) + ς (h), m(ϕh ) = −1

where sup

|ς (h)| ≤ ς ∗ h 2 .

S∈Σ M,γ ,n

Therefore by Theorem 3.2 we can calculate the denominator in the kernel estimator (1.2), i.e. for any δ > 0    Z T  1 yt − x0 dt − 2q(x0 ) > δ = 0. lim sup P S Q T →∞ S∈Σ M,γ ,n Th 0 h It means that we can replace the denominator by 2q(x0 ) in the minimax risk for the estimator (1.2) as T → ∞. 4. Moment inequality for process (1.1) In this section we prove the following result. Denote by N the set of positive integers.

834

L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

Proposition 4.1. For any m ∈ N, sup

sup E S yt2m ≤ lm∗ ,

S∈Σ M,γ ,n t≥0

where lm∗ = (1 + 4/γ )(2m + 1)!!r m

(4.1)

with some positive constant r = r (M, γ , n). Proof. Set z t (m) := E S yt2m . Applying the Ito formula to function x 2m yields Z t Z t z t (m) = z 0 (m) + 2m E S (ys )2m−1 S(ys )ds + m(2m − 1) z s (m − 1)ds, 0

where z 0 (m) =

y02m .

0

Therefore

z˙ t (m) = 2mE S (yt )2m−1 S(yt ) + m(2m − 1)z t (m − 1).

(4.2)

Notice now that, for any fixed K > n, x 2m−1 S(x) ≤ K 2m−1 S(x)1{|x|≤K } + x 2m−1 S(x)1{|x|>K } S(x) ≤ K 2m−1 sup |S(x)| + x 2m 1{|x|>K } . x |x|≤K Taking into account the definition (2.1) and the finite increments formula we obtain that for K ≥x >n ˙ (x))(x − n)| |S(x)| = |S(n) + S(θ ˙ (x))||x − n| ≤ |S(n)| + | S(θ ≤ M + K /γ .

(4.3)

Similarly we obtain the same bound for −K ≤ x < −n. Therefore sup |S(x)| ≤ M + K /γ .

(4.4)

|x|≤K

Moreover, for |x| > K one gets  M n S(x) ≤ −γ 1− x K K and putting K = K 0 = 2(M + nγ )/γ yields, for |x| ≥ K 0 , S(x) ≤ −γ /2. x Therefore inequalities (4.3)–(4.5) imply that γ x 2m−1 S(x) ≤ K 02m−1 K 1 − x 2m 1{|x|>K 0 } 2 γ 2m γ 2m 2m−1 K1 + K0 − x ≤ K0 2 2 γ 2m 2m ≤ 2K 1 − x , 2

(4.5)

L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

835

where K 1 = M + K 0 /γ . Taking into account this inequality, we obtain from (4.2) that z˙ t (m) ≤ 4m K 12m − mγ z t (m) + m(2m − 1)z t (m − 1). We can rewrite this inequality as follows z˙ t (m) = −mγ z t (m) + m(2m − 1)z t (m − 1) + ψt with ψt ≤ 4m K 12m . Therefore Z t Z t e−γ m(t−s) ψs ds z t (m) = z 0 (m)e−γ mt + m(2m − 1) e−γ m(t−s) z s (m − 1)ds + 0 0 Z t −γ m(t−s) e z s (m − 1)ds + βm , ≤ m(2m − 1) 0

where βm =

y02m

+ 4 K 12m /γ .

sup z t (m) ≤ t≥0

Resolving this inequality by recurrence, we find that

m (2m − 1)!! X βj m γ j=0

(2m − 1)!! (m + 1)βm γm ≤ (1 + 4/γ )(2m − 1)!!(m + 1)r m ,

≤

where r = max(y02 , K 12 )/γ . From here we obtain the desired result.



5. Proofs of main results Proof of Theorem 3.1. First, to represent ∆T (ϕ) as a continuous martingale plus a negligible term we need to find some bounded solution of the equation v˙ + 2Sv = 2ϕ, ˜

(5.1)

where ϕ˜ = ϕ − m(ϕ). To this end we define the following function Z +∞ Ry 2 u S(z)dz ϕ(y)e ˜ v(u) = −2 dy.

(5.2)

u

In Appendix we show that there exists some positive constant % = %(M, n, ν, γ ) such that sup

sup sup |v(y)| ≤ %.

(5.3)

S∈Σ M,γ ,n ϕ∈Vν,c y∈R

It is easy to check thatR the function v satisfies the Eq. (5.1). Therefore applying the Ito formula y to the function V (y) = 0 v(z)dz yields V (yT ) = V (y0 ) +

T

Z 0

ϕ(y ˜ t )dt +

T

Z

v(yt )dwt .

0

From here and (5.3) it follows the following upper bound: for any T ≥ 1, Z 1 T |∆T | ≤ %|yT | + %|y0 | + √ v(yt )dwt . T 0

(5.4)

836

L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

Further we need the following moment inequality for stochastic integrals: Z T 2m ES v(yt )dwt ≤ (2m − 1)!!%2m T m , 0

for any m ∈ N (see Lemma 4.11 in [9]). Thus, for any m ∈ N, E S (∆T (ϕ))

2m

%

2m−1

≤3

2m

E S (yT )

2m

+%

2m

(y0 )

2m

 + ES

1 √ T

T

Z

v(yt )dwt

2m !

0

≤ 32m−1 %2m (lm∗ + y02m + (2m − 1)!!) ≤ (3%)2m ((2m + 1)!!r m + y02m + (2m − 1)!!). It means that for κ = 1/(54%2 (r + y02 + 1)) we obtain E S eκ ∆T (ϕ) = 1 + 2

≤ 1+ ≤ 1+

+∞ m X κ E S (∆T (ϕ))2m m! m=1 +∞ m X κ (3%)2m ((2m + 1)!!r m + y02m + (2m − 1)!!) m! m=1 +∞ X

κ m (3%)2m ((3r )m + y02m + 2m )

m=1 +∞ X

(1/2)m = 4.

≤ 1+3

m=1

We get the desired inequality.



Proof of Theorem 3.2. Let ϕ ∈ Vν,l . For this function we consider the following sequence of continuous functions   Z 1 +∞ z−y η ϕ (y) = ϕ(z)g dz, (5.5) η −∞ η where g(z) is an infinitely differentiable non-negative even function which is equal to 0 for R1 |z| ≥ 1 and −1 g(z)dz = 1. The inequalities Z 1 Z +∞ Z +∞ η η kϕ k1 = |ϕ (y)|dy ≤ g(θ ) |ϕ(y + θ η)|dydθ = kϕk1 ≤ ν −∞

−1

−∞

show that ϕ η ∈ Vν,c . Therefore applying Theorem 3.1 to the continuous function ϕ η yields P S {|∆T (ϕ η )| > λ} ≤ 4e−κλ , 2

for any η > 0 and λ > 0. Moreover, the definition (5.5) implies that Z lim sup |ϕ η (y)|dy = 0. N →+∞ η>0 |y|≥N

(5.6)

L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

837

Therefore Z lim

+∞

η→0 −∞

|ϕ η (y) − ϕ(y)|dy = 0.

(5.7)

Let us show now that, for T ≥ 1, S ∈ Σ M,n,γ , lim E S |∆T (ϕ η ) − ∆T (ϕ)| = 0.

η→0

Indeed, since for each t > 0 the random variable yt has the bounded density (see, for example, [7], p. 91), we get that lim E S |ϕ η (yt ) − ϕ(yt )| = 0.

η→0

Notice that the function ϕ η is bounded, i.e.   Z z−y 1 +∞ η |ϕ(z)|g sup |ϕ (y)| ≤ sup dz ≤ ϕ ∗ < +∞. η η −∞ y∈R y∈R Therefore by the dominated convergence theorem Z T lim E S |ϕ η (yt ) − ϕ(yt )|dt = 0. η→0 0

Moreover, from (5.7) and (2.3) it follows limη→0 m(ϕ η ) = m(ϕ). Hence Z T 1 E|ϕ η (yt ) − ϕ(yt )|dt E S |∆T (ϕ η ) − ∆T (ϕ)| ≤ √ T 0 √ + T |m(ϕ η ) − m(ϕ)| → 0 as η → 0. Finally, taking into account (5.6) we find the following bound P S {|∆T (ϕ)| > λ} ≤ P S {|∆T (ϕ η )| > λ/2} + P S {|∆T (ϕ η ) − ∆T (ϕ)| > λ/2} 2 2 ≤ 4e−κλ /4 + E S |∆T (ϕ η ) − ∆T (ϕ)|. λ Limiting here η → 0 yields the inequality (3.4). Hence Theorem 3.2.  Appendix Proof of (5.3). If 0 ≤ u ≤ n then by definition (5.2) we have Z +∞ Ry |v(u)| ≤ 2 (|ϕ(y)| + |m(ϕ)|)e2 u S(z)dz dy u Z n Z +∞ Ry Ry 2 u S(z)dz (|ϕ(y)| + |m(ϕ)|)e (|ϕ(y)| + |m(ϕ)|)e2 u S(z)dz dy ≤2 dy + 2 u n Z n Z +∞ Ry ≤ 2e2Mn (|ϕ(y)| + |m(ϕ)|)dy + 2e2Mn (|ϕ(y)| + |m(ϕ)|)e2 n S(z)dz dy. u

n

Taking into account (2.3), it is easy to obtain that |m(ϕ)| ≤ q ∗ kϕk1 = q ∗ ν.

(A.1)

838

L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

Therefore, for 0 ≤ u ≤ n, |v(u)| ≤ 2e2Mn (ν + q ∗ νn) + 2e2Mn

Z

+∞

(|ϕ(y)| + |m(ϕ)|)e2

Ry n

S(z)dz

dy.

(A.2)

n

By the finite increments formula for z ≥ n and by taking into account that S ∈ Σ M,γ ,n we obtain that ˙ (z))(z − n) ≤ M − γ (z − n). S(z) = S(n) + S(θ Thus for any y ≥ n Z y 2 S(z)dz ≤ 2M(y − n) − γ (y − n)2 . n

Now we can estimate the integral in (A.2) as follows Z +∞ Ry (|ϕ(y)| + |m(ϕ)|)e2 n S(z)dz dy ≤ (c∗ + c1∗ q ∗ )ν,

(A.3)

n

where ∗

c = sup e

2M|z|−γ z 2

z∈R

,

c1∗

Z =

2

e2M|z|−γ z dz.

R

Therefore sup |v(u)| ≤ 2e2Mn ν(1 + q ∗ (n + c1∗ ) + c∗ ).

sup

s∈Σ M,γ ,n 0≤u≤n

Similar to the inequality (A.3) we obtain that, for u ≥ n, Z +∞ Ry |v(u)| ≤ 2 (|ϕ(y)| + |m(ϕ)|)e2 u S(z)dz dy ≤ 2(c∗ + c1∗ q ∗ )ν. u

It means that sup

sup |v(u)| ≤ 2e2Mn ν(1 + q ∗ (n + c1∗ ) + c∗ ) := %.

(A.4)

s∈Σ M,γ ,n u≥0

To obtain an upper bound for v(u) as u ≤ 0, we make use of the following equality Z Ry 2 0 S(z)dz ϕ(y)e ˜ dy = 0 R

from which it follows Z Z +∞ Ry 2 u S(z)dz ϕ(y)e ˜ v(u) = −2 dy = 2 u +∞

Z

2 ϕ(y)e ˜

Ry u

S(z)dz

dy

−∞ −2 ϕ(−y)e ˜

=2

u

Ry −u

S(−z)dz

dy.

−u

Now by the same way we can obtain the inequality (A.4) for negative u, i.e. sup

sup |v(u)| ≤ 2e2Mn ν(1 + q ∗ (n + c1∗ ) + c∗ ) := %.

s∈Σ M,γ ,n u≤0

From (A.4) and the last inequality, the inequality (5.3) follows.



L. Galtchouk, S. Pergamenshchikov / Stochastic Processes and their Applications 117 (2007) 830–839

839

References [1] R.N. Bhattacharya, On the functional central limit theorem and the law of the iterated logarithm for Markov process, Z. Wahrsch. Verw. Gebiete 60 (1982) 185–221. [2] S. Boucheron, G. Lugosi, P. Massart, Concentration inequalities using the entropy method, Ann. Probab. 31 (2003) 1583–1614. [3] J. Dedecker, P. Doukhan, A new covariance inequality and applications, Stochastic Process Appl. 106 (2003) 63–80. [4] J. Dedecker, C. Prieur, New dependence coefficients. Examples and applications to statistics, Probab. Theory Related Fields 132 (2005) 203–236. [5] L. Galtchouk, S. Pergamenshchikov, Sequential nonparametric adaptive estimation of the drift coefficient in diffusion processes, Math. Methods Statist. 10 (2001) 316–330. [6] L. Galtchouk, S. Pergamenshchikov, Asymptotically efficient sequential kernel estimates of the drift coefficient in ergodic diffusion processes, Stat. Inferences Stoch. Process. 9 (2006) 1–16. [7] I.I. Gihman, A.V. Skorohod, Stochastic Differential Equations, Springer, New York, 1972. [8] Yu.A. Kutoyants, Statistical Inferences for Ergodic Diffusion Processes, Springer, New York, 2003. [9] R.Sh. Liptser, A.N. Shiryaev, Statistics of Random Processes, I, Springer, New York, 1977. [10] P. Massart, Some applications of concentration inequalities to statistics, Ann. Fac. Sci. Toulouse Math. 9 (2000) 245–303. [11] V. Maume-Deschamps, Concentration inequalities and estimation of conditional probabilities, in: Lect. Notes in Statistics, vol. 187, Springer, New York, 2006. [12] S.M. Pergamenshchikov, On large deviation probabilities in ergodic theorem for singularly perturbed stochastic systems in: Weierstrass Institut f¨ur Angewandte Analysis und Stochastik, Berlin, 1998. Preprint n. 414. [13] A.Yu. Veretennikov, On large deviations for diffusion processes with measurable coefficients, Uspekhi Mat. Nauk 50 (1995) 135–146.