A note on recursive estimator of the density function which is not necessarily continuous

Microelectron. Reliab., Vol. 32, No. 1/2, pp. 79-87, 1992. Printed in Great Britain.

0026-2714/92/$5.00 + .00 Pergamon Press plc

A NOTE ON RECURSIVE ESTIMATOR OF THE DENSITY FUNCTION WHICH IS NOT NECESSARILY CONTINUOUS M.M. EL-FAHHAM Department of Mathematics, Faculty of Science, Cairo University, Giza, Egypt (Received for publication 30 October 1990)

Summary:

Consider a sequence XI'X2 '''''Xn of independent, identically distributed random variables with unknown density function f, which with its first derivative are not necessarily continuous, and let .

fnCXl =

n

x-X

I-1/2 Z h l/2KCh-- l j=l

3

be the recursive kernel estimator of f. It will be shown, under certain additional regularity conditions on K and lh(n)], that, * ] = 0 (n-4/5 ) if f and f' are continuous, where as MISE [fn(X) MISE

[fnl(x) ] = 0(n -3/4) if f is continuous and f' is not and

MISE [f*2"(X)]: 0(n -I/2) if f and f' are not continuous.

Introduction: Consider a sequence XI,X2,...,X n of n independent, identically distributed random variables

(i.i.d.) with unknown density function

(d.f.) f(x). A class of density estimators, known as Kernel estimators, which has been widely studied by Parzen [i] and Rosenblatt [21 may be defined as follows: ^ n x-X --/)f (x) = (n h) -1 Z K( ,--7 n n j=l

,

Parzen has shown that if the Kernel function K(.) satisfies certain conditions,

including

I

K(y)dy = 1 and h = h(n) is a bounded and ^

nonincreasing function of n such that h(n) = 0(n-l), then f(x) is a mean squared error consistnet estimator of f(x) at all continuity points x of f. E.J. Wegman and H.I.Davies [2J considered a modified estimator of the form , (n2hn) _i/2 fn(X) -MI~ 32/I-2--F

n (hj).i/2 X-Xh~ ' ~ K( ). j=l j 79

(1)

80

M . M . EL-FAHHAM

They

established

tors

as w e l l

sure

invariance

as

In this form are

f

(x)

n

not

laws

the

logarithm

distribution

for d e n s i t y

results

using

estima-

the a l m o s t

principle. article

we s h a l l

for an u n k n o w n

use

iterated

the a s y m p t o t i c

necessarily

authors

of

discuss

d.f.

f(x)

continuous.

the m e a n

classes

which

As a

integrated

square

,(=ll

with

its

measure

of

error

given

~o

==== If

of e s t i m a t o r s first

of the

derivative

improvement,

many

by

2

: = :

[fi(=,

- f(-I

dx

(2)

-.
=in==

2

° = :

I=i(=l

-co

-

d= :

(,,

where B{f(x))

=

E

[fn(X)]

- f(x).

Hence 2

['fnC=,I = ~l~S~ [f~(=I] -,- (l-n>= I~(=I/

~s~

Consider baised

on

function

a recursive

the r a n d o m satisfying

Kernel

sample.

estimator

Assuming

the f o l l o w i n g

t sup IK(=~l <= X

K(.)

of

the

form

is a s o m e

(i) of

suitable

f Kernel

conditions:

,

lira IxK(×ll

= 0,

X --*+o~

(A) co

cc

-fc~ IK(x) Idx < o0 a n d h(n) to z e r o

is a s e q u e n c e as n tends

As s y m b o l s , are

f-co x 2 K ( x ) d x

of n o n i n c r e a s i n g

to

we s h a l l

and

positive

< co , constants

~ .

f~(x) as an estimator for f when o if f is c o n t i n u o u s b u t f' is not, then we

continuous,

as an e s t i m a t o r

use

for

converging

f and

f

(x) as an e s t i m a t o r n2

for

f when

f and take

f' f

(x) n1

f and

are not continuous.

Main

results: Theorem

!-

Assum

conditions

(A) a n d

nh

continuous

and

f"

is

n

that ~

K(.)

oo

square

as

is a B o r e l n ~

integrable,

~

function

, if

f(x)

satisfies , f'(x)

are

then

o

[If(xllF +~(nhl

-i

O
llg(t) ll = ; g ~ ( t ) d t

2

n

where

,

and Yn

=

n Z j=l

hl/2 3

n

[=h ~.lltK1/2¢tl/ j=l

] 2

4

tf"(=l/

2

+0(h;l]} ,

f'

Recursive estimator of the density function

~ h .~ If (n2h n) - i/2~n---~ B < ~ and (nhn)-I j[l

---+

0

as

81

n

---~ ~

,

then

and hence MIISE

~

In particular

MIISE

0

as

n ---~ ~

e

if h. = 0 (n-V), j=l,2,...,n; 3 [fn(X)] = 0(n Y-I) ÷ 0(n -4~)

X > 0, then

, 1

and hence the asymptotic optimum value Yo of y is -~ and then get minimum MIISE

Theorem 2.

[fn (x)~ = 0(n-4/5). o Suppose K(.) and h

If f(x) is continuous,

n

satisfy conditions of theorem i.

f' has finite points of discontinuity

(bl,b 2 .... ,b£) and f" is square integrable,

MIISE [fnl * Ix~] = C2{MllSE [fno * (x)]

then

+

4~( n 0 y 2 (]_ n -- ~ ha.)f [ f (y-t)g(t)dt] dy+O ~ ha.)}, n j=l 3 _~ _co n j=l 3 £ 0 < C2 < 1 , A = Z A2 , j=l 3

where

Aj

If

we

=

f' (bj)- f' (b;)

i_ n~ k3...~ n j=l

0

Tbe e s t i m a t i o n f n l ( x )

as

, j=l,2,...,n.

n ---~ ~ , then as

is c o n s i s t e n t

n ~

~ ,

in MIISE s e n s e , s p e c i a l l y

h3 = 0(n-3') ' j=l,2 ..... n, "!( > 0, then MIISE

if

[fnl(x)]=0(n'f-1)+0(n-3~()

and we get the asymptotic optimum value 70 of ~ is %, and hence minimum MIISE[f~I(X)~

Theorem 3.

= 0(n-3/4).

Let K(.) and h

n

satisfy conditions of theorem i.

If f has k(k > 0) points of discontinuity, -~ = a O < a I < a 2 < ... <

a k < ak+ 1 = co , and for each

i=l,...,k+l,

f' has Zi(£i > 0) points of d i s c o n t i n u i ~

ai_ 1 = b.lo < bil < bi2 < -.- < bi£ ' < b.1 = a.1 ' also i £i+i co co 2

I -co

If'tx) Idx < co

I

If"Ix) l dx < co

-co

then

Ml~sE [fn ~x~] < c 2 MiIS~ If x~] +

82

M . M . EL-FAHHAM o~ oo 2 n ,2 n + "-~'( ~ h.)f fy K(x)dx I dy 4 0( 1 ]:lhj)= n j=l 3 o k 6 = : 62 , 6 = f(a-) - f(a +) P i i i=l I i

where

k+l

£ +i l

i=l

j=l

i = i, .... k+l

If

n -1 ~ n j=l

h. ~ ]

CODSiste~t

0

and m i n i m u m

MIISE

Proof: theorem

as

in M I I S E

> 0, then M I I S E

1 and

and

~ij = f'(b:j)

and

j = i, .... £.+i l

13

n ~

~ , then

sense.

In

the

= 0(n ~-I)

[f~2(x)]

= 0(n-i/2).

It is s u f f i c i e n t

- f' (bij + ) ,

.

estimation

particularif

[f~2(x)J

,

f

n2

hj = 0(n -~)

(x)

is

, j=l,2,...,n,

+ 0(n -~) and also we get ~o = ~

to prove

theorem

3 which

implies

2.

We can see

that co

~:s~ [fn21x)]-~/-o~ [f:ixl - f<×l:2 dx co ~- ~ / [(n2hn)-i/2 Z h -~ j=l

x-X.

n i:j2"(

oo (h2hn)-i = f [ -oo +2 ( n 2 h ) -i n

2 )

f(x)]

dx

3

n -i z x-X i Z h. EK (h ) + j=l ] i n

l=i
(b.h.)i ]

I '2 x-X. x-X / EK(~-~)K(h-~) i "j

-

X-X. -2f(x) ( n 2 h ) -I/ 2 n~ ( h ) -I/2EK( h ' ~ ) + f 2 ( x ) 3 d x . ~l j=l 3 3

(3)

First oo f

x-X. h-IEK2( h - ~ ) d x

= f

co co f

oo co = f / K2(t) f ( ~ - t h . ) d t d x -oo -co ]

-*

h:iK2(~.)f(y)dydx cc = f ¥2(t)dt+0(1) -co

•

(4)

Second co

x-X. co co h-!f(x)EK( .--r--])dx = f f(x)f K(t)f(x-thj)dtdx. ] n.3 -oo --co

/

(5)

At last n

z

co

x-X .

×-X

/ Chih I-1/2EK<@IEKC h----~Idx

i= J < j _co J 1 3 n co co = Z (hihj)I/2{f f ~ fo~ K(t)g(s) [f(x-thi)-f(x) ] . i=i < j -co [f
+ f(x-shj)]dt

dt

d s dx + f f f K(t)g(slf(x)[f
(6)

Recursive estimator of the density function

83

It is easy to see that the second term in the last equation is equal to n

co

oo

~n ~ hl/2 $

f (x) $ . . . .

j--I 3

K(t) f(x-th )dt dx ; 3

n hl/2 j=l 3

7j =

From (3) , (4) , (5) and (6) we get oo MISE [f~2(x)] = (n hn )-] )- i

f_ooK2(t) dt

n

+ 2(n~ hn

l=i < j -i

+ 2•n(n2h n)

1/2

oo

f-oo J(x,hi)J(x,hj)dx

(hih j)

n ~ oo ~ h I/2 I f K(t) f(x)f(x-thj)dt dx j=l 3 _~ _~

(n2hn)-l 7n2 f._~f2(x)d x - 2(n2hn)-l/2 j=lhl/2 j J'~ f(x)J(x,hjldx

_

- 2(n2hn )-1/2 7n f

f2(xldx + f

f2(x)dx+O((nhn I-l) ,

co where

J(x,h) = $ K(t) [f(x-th) - f(x)] dt . -co oe * MISE [fn2(X) ] = (nhn) -i I_~o K2(tldt

Then

n 0o + 2(n2 h )-i ~ (hihj)-i/2 f j(x,hi)J(x,hj)dx n l=i
2~n~hn~-l/2[1 - "%ln~h2-1/21 + [i-

~ h~./2 S fCx~JIx,hj~dx

j=l 3

_~

(n2hn)-I/27n]2 f~o f2 (x) dx + 0((n hn)-l) --co

By using HOlder's inequality we get MISE

2 2 [f~2(x)] ~ (n hn)-lltK(t) I + [l-(n2hn )-1/2~n~~2 I If(x) II n

+ 2(nZh )-i Z (hihj)+i/2 n l=i
J(x,hi) H llJ(x,hj)ll+0((n hn)-i ). (7)

By using Talylor series-like expansions for f satisfying conditions of our theorem a)

For

biJl_ 1 < x < b.131, bij2_ ] < x-th < biJ2, 1 -< Jl < li+l'

1 < J2 < li+l ' i=l,2,...,k+l f(x-th)-f(x)+th

t f'(x)-h 2 f (t~s)f"(x-sh)ds-o

0

'

Jl = J2

'

Jl < 92

,

Jl > J2

j-I =

~

7=Jl

(b.

17

- x + th)

~ i7

J~-l(biT- x + th) Z~iy 7=32

84

M . M . EL-FAmtAM b)

For b.1131_1. < x < bil]l. ~

bi2J2_l < x-th < b.1232.,I -< Jl -< lil+l'

i ~ J2 E ii2+i'I ~ i I ! k+l, i ~ i 2 ~ k+l, il~ i 2 ,

f(x-th)-f(x)+th

t f' (x)-h 2 / (t-s)f"(x-sh)ds o

=

i. il+l i2-i (i)

1£+i

Z

(b. -x+th) A. + Z Y= Jl ilY i]y y=il+l

~

92-1 i2-i ~- Z (bi2 ~- x +th)Ai2~/ Z. 6 ~(=J2 U=i 2

(ii)

o<

i. i2+i

i

(ii) -

Z Y=J2

(bi2y-x+th)Ai23.

-

Z

,-x+th)Zl/~

if

iI < i 2

il-]

I£+I

Z Y=i2+l

Z -x+th)A ~/'=i (by,f,

Jl -I -

(b

y'=l

,

il-i (bil Y

-

x

+

th) Ail Y

+

Z

y=l

6u

if i'l >

i2

u=i2

Let oo t = h 2 f K(t) f (t-s)f"(x-sh)dsdt , -oo < x < co , -oo o x-b. i .+i _ iy i Hiji(×,h)= Z A. f h x + th)K(t)dt + Y =j I~ -o~ (bi~ x-b k+1 i£+i YY' h + ~ ~ A., / (b , - x + th)K(t)dt ~(= i+l y'=l -~ G(x,h)

j-i Z Ai% ' f (bi3-x+th)K(t)dt y=l x-b. iy h i-I i£+i oo ~ ~ A , f (b , - x + th)K(t), y=l y'=l 77 x-b Y7' h

Hij2(x,h)=

-

bij_l

-

< x < bij ; j=l,2,...,li+ 1 , i=l,2,...,k+l, i-I

Vil (x,h)= U =i

Vi2(x,h) Where

= -

W(y) =

Further

6u{l-w(X-a~)} h

k x-a ~ • (5U W ( h ~ ) U=I

Y f -oc

V.(x,h) i

< x < a i-i

,

ai_ 1

= Hijl(X,h)

= Vil (×,h) + Vi2 (×,h) "

.,k+l, ""

< x < a. , i=l,...,k. i

, i=l,...,k+l

+ Hij2(x,h)

i=l, i'

K(t) dt , -~ < y < co

let , for j=l,...,li+l

Hij(x,h)

, ~

and

!

Recursive estimator of the density function Lena

1. lira 1 I G(x,h)dx = %(f t2K(t)dt) 2 • I (f"(x) 2dx. h÷0 h ~ -~o -co -~

Lemma 2. b..

x3

2

2

0

y

lira 1 f Hij(x'h)dx = (~ij b+0 h 3 bij_l 1 ~ j ~ 1 i+l ' i=l ..... k+l.

Lemma 3. a. lim -~ f V2(x,h)dx = ( ~ + 6~ l)f (I-W(y))2dy, i=]...... k+l. z h-~0 ai_ 1 o Lemma 4. b.. a. x3 1 i G(x,h)V.(x,h)dx = lira 1 I G(x,h)Hij(x,h)dx= l i m - ~ f 1 b+0 h 3 bij_l h~0 ai_ 1 b..

=

1 13 lira -~ I Hij(x,h)Vi(x,h)dx = 0,

h~0

bij_l

1 < j < li= 1 , i=l .... ,k+l Proofs of these lemmas given by Eden, C.V. [4] . We can see that co

co

j2 (x,h)dx = f {f K(t) [f(x-th)-f(x)Idt}2dx -co -co =

k+l Zi+l bij ~ ~ f {G (x, h) +Hij (x,h) +Vi (x, h) }2dx J=l j = 1 bij_l

=

k+l £i+i bij k+l £i+i bij ~ ~ f G2(x,h) dx + 7~ ~ f H2. (x,h) dx 13 i=l j = 1 bij_l i=l j = 1 bij_l

+

k÷l £i+I 7 Z i=l j = i

+ 2

bij k+l Zi+l bij f V2. (x,h) dx+2 Z Z f G (x,h) Hij (x,h) dx bij_l i J=l j = i bij_l

k÷l £i+I bij k+l £i+i b . ~ Z f G (x,h)Vi(x,h) dx+2 7~ Z fagHij (x'h)Vi (x'h) dx i=l j = 1 bij_ l i=l j = 1 bij_l

~o k+l li+l bij = I G2(x,h)dx + Z Z f H2..(x,h)dx -o~ i=l j = 1 bij_l 13

+

k+l £i+i bi 3 k+l a i ~ I V2 (x,h) dx+2 Z Z f C (x,h) Hij (x,h) dx i=l ai_ 1 i=l i=l bij_l

+ 2

k+l £i+i bij k+l a 1 ~ f G (x,h) Vi (x,h) dx+2 ~ ~ f Hij (x,h)Vi(x,h) dx. i=l ai_ 1 i=l j=l bij_1

Then from lemmas (i), (2), (3) and (4), we get co 4 ce I J2(x,h)dx = ~ ( f t2Klt)dt) 2 f (f"(x))~x + 0(h4 )

85

86

M . M . EL-FAHHAM

+ h3

1 .+I k+l i 0 y Z Z ( ~ . + ,~2 i=l j = i J ij-l)I-co{ I_~o(y-t)K(t)dt~2dy k+l

+ 0(h ~) + h i=]

(6~ + 6 2 )/ (l-W(y))2dy i-i o

But from the fact that 1 .+I k+l 1 Z ~2 : i=l j 1 ij-i

1 .+i i

k+l Z

=

+ 0(h)

Z

i:l

j : 1

A2 • = ~ t i3

and k+l Z

i:l

k+l 62 i-i

=

Z

i:l

6 2. i

6

=

,

We have co

IIf"(xlIl

.; J2(.,h.)d. : ~, h t I l t K'~(t) II _(~

]

+ 0(h~) 3

3

0 y + 2h 3 ZI f (f (y-t)K(t)dt) 2 dy+0(h3.)

oo + 2hj 6 / (l-W(y))2 o From

(7) and

dy + 0(hj).

(8)

(8) we get

.~sE [f~2(xl] _< Inhnl lllK(t~ll~+0(nh n) 11 + [~-

(n~hn)-l/2~,o]~

Irf(.lll~

+ 2(nh )-l[~Irt~It)ll~Ilf"II

~(

~ hS.) + 0(hS..) + 3 j=l ]

n o y + 24( Z h~)/ {I (y-t)K(t)dt}2dy j=l 3 _co -co

o completes

the

+

+

n

Which

2

+

j=l J

proof of the theorem.

REFERENCES

[i]

Parzen,

E.

(1962).

mode.

[2]

Rosenblatt,

M.

Ann. Math.

(1956).

sequences tsch.,

[3]

Wegman,

On estimation

of a probability

Statist.

The central

density

and

33, 1065-1076.

limit

of random variables.

theorem

for mixing

Z. Wahrscheinl,

chkei-

12, 155-171.

E. J. and J. I. Davies sire estimators Statist.,

Vol.7,

(1979).

"Remarks

of a probability 316-327.

on some recur-

density".

Ann. Math.

Recursive estimator of the density function

E43

Van Eeden C.

87

(1984). Mean integrated squared error of Kernel

estimators when the density and its derivative necessarily

continuous.

Roppart de Recherche$

D~partement de mathematigues ersit6 de Montr6al.

are not no.82,

et de Statistique,

Univ-