- Email: [email protected]
A note on random densities via wavelets
5'~ " -
,<
STATIS"I'ICS& PR~LITY LETTERS ELSEVIER
Statistics & Probability Letters 26 (1996) 315-321
A note on random densities via wavelets Brani V i d a k o v i c 1 ISDS, Duke University, P.O. Box 90251, Durham NC 27708-0251, USA Received February 1994
Abstract It is a well-known fact that any orthonormal basis in L2 can produce a "random density". If {~bn} is an orthonormal basis and {a,} is a sequence of random variables such that ~aZ, = 1 a.s., then f(x) = I ~ a , qS,(x)]2 is a random density. In this note we define a random density via orthogonal bases of wavelets and explore some of its basic properties.
Keywords: Wavelets; Parseval's identity; Random density
1. Introduction A wavelet is a function ff whose dilations and translations {~kjk(x) = U/Z~(ZJx -- k), ( j , k ) E Z × Z}
(1)
form a basis in L 2. For a fixed mother wavelet any L 2 function f can be represented through an expansion:
f ( x ) = ~ djk~k(x), jk
(2)
where ~k = ( f , ~'k) are the wavelet coefficients. We restrict our attention only to compactly supported wavelets producing orthonormal bases. One such family was discovered and described by Daubechies (1988). The nth member of Daubechies family, DAUB n, is supported on [0,2n - 1] and belongs to the C~[0,2n - 1] class, for ~ equal to approximately to 0.2 n + O(log n/n). The most simple o f all wavelets, the Haar wavelet, is the DAUB 1 wavelet. Though the Haar wavelet is very inconvenient for practical purposes (it does not sparsely represent functions and data), it is theoretically interesting; many wavelet properties are most apparent in the Haar wavelet basis case. For more on basics o f wavelets we direct the reader to Daubechies (1992), Meyer (1993), and Chui (1992). Defining a random density in L 2 via an orthonormal system of functions is not a new idea. Chen and Rubin (1986) discuss properties o f random densities introduced by a variety o f orthonormal bases: Fourier, Jacobi, Hermite, and Laguerre. They explore the analicity, tail behavior, and number o f zeroes of the random densities. t Research supported by NSF Grant DMS-9404151 at Duke University. 0167-7152/96/$12.00 (~) 1996 Elsevier Science B.V. All rights reserved SSDI 0 1 6 7 - 7 1 5 2 ( 9 5 ) 0 0 0 2 6 - 7
B. Vidakovic/Statistics & Probability Letters 26 (1996) 315-321
316
Using wavelets to build a random density is beneficial in several ways. We mention the good localization (in time-frequency) property and the possibility o f developing fast algorithms for wavelet transformations. For some other optimality properties as well as discussion on general use of wavelets in statistics, we refer the reader to the work o f Donoho, Johnstone et al. One o f the pros for using wavelets as bases is that they provide unconditional bases for many functional spaces. That is we are able to determine if the function belongs to a particular functional space just by looking at absolute values o f the wavelet coefficients. Wavelets provide unconditional bases for L p, p > 1, Sobolev Spaces W s, and H61der Spaces c~, under very mild conditions on the mother wavelet (See Daubechies, 1992, Chapter 9). There are other ways o f defining a random density; some are given in Thorburn (1986). Why does one need random densities? There are several places random densities have proven useful. For example, the comparison o f various density estimators requires a sequence o f random densities. The performance o f Bayes and especially F-minimax rules may be explored when the prior is elicited at random. In this note we define only compactly supported random densities. The method given in Section 2 can be extended with slight modifications to cover random densities on R. The key result that enables the definition of random density is the Parseval Identity. We state it in terms o f wavelets:
Result. If the family {~k} forms an orthonormal basis in L 2 then for any function f E L 2 jk
I(f,~k)l 2 = Ilfll 2.
(3)
Defining the coefficients djk = ( f , @k) to satisfy the condition ~ j k d2 = 1 implies that f f 2 ( x ) d x = 1. Remark. To define a random density in L 2 one does not need an orthonormal basis. It is sufficient to have a tight f r a m e )
2. Main result In this section we give a definition o f a wavelet-based random density, and prove that it integrates to one almost surely. The most important part o f the section is the constructive description of random coefficients Without loss o f generality we assume that the support o f the random density we want to generate coincides with the support o f the chosen mother wavelet. By rescaling and translating, any preassigned compact domain can be matched. All random densities in the examples we will give later are rescaled to the interval [0, 1]. An algorithm o f constructing a random density via wavelets is given. It is a generalization o f the RubinChen algorithm; see Chen and Rubin (1986). In what follows, we generate random wavelet coefficients. We start with the function coo~oo(x) + doo~boo(x) + dlo~klo(x) + d l l ~ l l ( x ) + d20~20(x)... We assume that the coefficient o f the "scaling" part, coo is zero. There is no substantial change in the algorithm if the coefficient 1A family of functions 9, in L2 is a frame if
V f E L2 AIIf]I2 <~~ ](f,9,)I2 <~BIIf]I2 n
for some constants 0 < A ~
B. Vidakovicl Statistics & Probability Letters 26 (1996) 315-321
/\ /\
/\ /\
/\
/\ /\
/\
317
/\ /\
/\
/\
Fig. 1. Tree of random variables ujk.
coo is fixed or random (between 0 or 1). All other coefficients are constructed using random numbers. The construction of the coefficients is such that they are subject to the constraint (3). The fact that compactly supported wavelets form an unconditional basis of L2 is used. We enlarge the class of possible random densities by assigning random signs to generated djks. Let Xjk be a family of i.i.d. Bernoulli(p) random variables, and let rs(p) be a random sign defined as rsjk(p) = 2Xjk-
1.
We suppose 0 ~
doo = rsoo( P ) V 5 - S uoo,
~/1 -
djk = r s j k ( P )
Ujk
2J
1-I Uj'k" j'k' E path(j,k )
(4)
(5)
This is not the only way of defining a random density via wavelets. The other ways are appealing from a nonparametric Bayes standpoint. The work in progress (Lavine, M., Mfiller, P., and Vidakovic, B.) connects wavelets and Polya trees and introduces random densities in a way suitable for Bayesian inference. The above definition gives a calculationally efficient way of constructing a random density. Any coefficient (5) needs only one new random sign and one new random number u. The previously generated random numbers from the path are used in the construction.
318
B. Vidakovie/Statistics & Probability Letters 26 (1996) 315 321
The name random density is justified by the following theorem:
Theorem 2.1. f f(x)dx=
l,
a.s.
(6)
Proof. From the definition of djk it follows: d2o q- d~o "4- d2l if- d~o -'f- d~l q- d~2 q- d23 q- . . . .
1 -- Uo0 q- 1 UO0( 1 _ UlO) q- ~1 U o 0 ( 1 - Ull )
l
1
+¼ uoou,o(1 - u2o) + ~ uooum(1 - u21 ) q- I uooull(1 - u22) q- ~ uOOUll(] - u23) @ ' "
(7)
The theorem will be true if the sum (7) is 1 a.s. It suffices to prove the following lemma.
Lemma 2.1. Let {u~} is a sequence of i . i d random variables on [0,1] such that P(ul = 1) < 1. Then 1~ Ui ----+0, i=1
a.s.
(8)
as n ---~c<~. n
Proof. Easy, by observing that I-Ii=l ui is a nonnegative supermartingale. It converges a.s. to 0 since it converges to 0 in probability (Chebyshev's Inequality). []
3. Properties and examples of random densities In this section we discuss expected values and smoothness of random densities. 1 Theorem 3.1. For the Haar wavelet and the "random sign parameter" p = ~,
(9)
E f ( x ) = l(x C [0,1]). In other words, the uniform [0,1] distribution is the expected value of the random density f .
Proof. We assume that the ujk have finite expectation 2. Also, the fact that the random sign is independent of ujks and that E(rs(1/2))=O will be used. The interchange of the sum and expectation that follows is justified by Fubini's theorem:
E
2 2 = E E djk ~jk(x) + 2E E E djkdj,k,@k(x)@k,(x) jk
=
E E 2d)k~:k(x) 2 jk
jk j~k ~
=
E jk
2 2k(x) Ed)k
=
E( 1 jk
-
2) J 2Jq2(2Jx k) -
ZJ
= ~ (1 - 2)2) y~ ~b2(2Jx - k) = ~ (1 - 2))J = 1. j k j
[]
B. VidakoviclStatistics & Probability Letters 26 (1996) 315 321
319
The choice of random variables {ujk} affects the global smoothness of random densities. Let c¢~([0, 1]) be the H61der space of functions, defined as follows: ~e~([0,1]) =
f E L°~([0, 1]); 1f(x + h ) - f ( x ) l SUPx, h Ihl ~
1j,
~c'([0, 1]) = { f C L~([0, 1]) U cn([0, 1 ] ) ; f (n) E c~c"([0, 1]), ~ = n + ~',0 < ~' < 1}. Theorem 3.2. Let the random density f be 9enerated by a sequence of i.i.d, random variables {ujk} such that Eujk = 2. Then f E cg~([0, 1]),~ = 1 log 2 ½, with probability 1. We implicitly assume that the mother
wavelet ~ belongs to the class c¢~, for ~ >~~. Remark. To generate smooth random densities one should choose the sequence {ujk} so that Eujk < ¼.
Proof of Theorem 3.2. It is known that the H61der space, cg~ can be characterized by wavelets in the following way: Let ~-'~j,kdjkq~k be the wavelet representation of the function g, then g C~
iff Vk Idjk[~
By SLLN Idjkl - E l ~ k l ~ 0, j ~ o~,
a.s.
Jensen's inequality gives
, / 2 J ( ! U 2) _ C . 2 -(l/2+=)j 2J
El~kl~< E ~ j2k= V
for ~ = ½ log z I and C = v/1 - 2. 9(x) = Y']~j,kdjk~jk(x) belongs to ~ with probability 1. The theorem follows from the fact that if 9 E c£~ then f = 92 E cg~. [] Figures 2-5 depict generated densities for different values of 2 using the DAUB 4 or DAUB 10 wavelet.
.2
.
.
Fig. 2. Random density with DAUB 10 and 2 = 0.01.
B. Vidakovic / Statistics & Probability Letters 26 (1996) 315-321
320
2
0.2
04
O~
O~
1
Fig. 3. Random density with D A U B 4 and 2 = 0.1.
0.2
0,4
0.6
0.~
Fig. 4. Random density with D A U B 4 and 2 = 0.25.
1
321
B. VidakoviclStatisties & Probability Letters 26 (1996) 315-321
j
ih '
'
'
'
o.~
0.4
"
"
"
o16
. . . .
oi~
Fig. 5. Random density with DAUB 4 and 2 = 0.5.
References Chen, J. and H. Rubin (1986), Some stochastic processes related to random density functions, Technical Report #86-40, Department of Statistics, Purdue University. Chui, C. (1992), An Introduction to Wavelets (Academic Press, New York). Daubechies, 1. (1988), Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math. XLI, 909-996. Daubechies, 1. (1992), Ten Lectures on Wavelets (SIAM, Philadelphia, PA). Donoho, D., I. Johnstone et al. (1992, 1993), Bank of papers at playfair.stanford.edu. Thorbum, D. (1986), A Bayesian Approach to density estimation. Biometrika 73, 65-75. Vidakovic, B. and P. Mfiller (1994), Wavelets for kids, Discussion Paper 94-13, ISDS, Duke Univ. Zeidler, E. (1985), Nonlinear Functional Analysis and its Applications, Vol. I (Springer, Berlin).
,<
STATIS"I'ICS& PR~LITY LETTERS ELSEVIER
Statistics & Probability Letters 26 (1996) 315-321
A note on random densities via wavelets Brani V i d a k o v i c 1 ISDS, Duke University, P.O. Box 90251, Durham NC 27708-0251, USA Received February 1994
Abstract It is a well-known fact that any orthonormal basis in L2 can produce a "random density". If {~bn} is an orthonormal basis and {a,} is a sequence of random variables such that ~aZ, = 1 a.s., then f(x) = I ~ a , qS,(x)]2 is a random density. In this note we define a random density via orthogonal bases of wavelets and explore some of its basic properties.
Keywords: Wavelets; Parseval's identity; Random density
1. Introduction A wavelet is a function ff whose dilations and translations {~kjk(x) = U/Z~(ZJx -- k), ( j , k ) E Z × Z}
(1)
form a basis in L 2. For a fixed mother wavelet any L 2 function f can be represented through an expansion:
f ( x ) = ~ djk~k(x), jk
(2)
where ~k = ( f , ~'k) are the wavelet coefficients. We restrict our attention only to compactly supported wavelets producing orthonormal bases. One such family was discovered and described by Daubechies (1988). The nth member of Daubechies family, DAUB n, is supported on [0,2n - 1] and belongs to the C~[0,2n - 1] class, for ~ equal to approximately to 0.2 n + O(log n/n). The most simple o f all wavelets, the Haar wavelet, is the DAUB 1 wavelet. Though the Haar wavelet is very inconvenient for practical purposes (it does not sparsely represent functions and data), it is theoretically interesting; many wavelet properties are most apparent in the Haar wavelet basis case. For more on basics o f wavelets we direct the reader to Daubechies (1992), Meyer (1993), and Chui (1992). Defining a random density in L 2 via an orthonormal system of functions is not a new idea. Chen and Rubin (1986) discuss properties o f random densities introduced by a variety o f orthonormal bases: Fourier, Jacobi, Hermite, and Laguerre. They explore the analicity, tail behavior, and number o f zeroes of the random densities. t Research supported by NSF Grant DMS-9404151 at Duke University. 0167-7152/96/$12.00 (~) 1996 Elsevier Science B.V. All rights reserved SSDI 0 1 6 7 - 7 1 5 2 ( 9 5 ) 0 0 0 2 6 - 7
B. Vidakovic/Statistics & Probability Letters 26 (1996) 315-321
316
Using wavelets to build a random density is beneficial in several ways. We mention the good localization (in time-frequency) property and the possibility o f developing fast algorithms for wavelet transformations. For some other optimality properties as well as discussion on general use of wavelets in statistics, we refer the reader to the work o f Donoho, Johnstone et al. One o f the pros for using wavelets as bases is that they provide unconditional bases for many functional spaces. That is we are able to determine if the function belongs to a particular functional space just by looking at absolute values o f the wavelet coefficients. Wavelets provide unconditional bases for L p, p > 1, Sobolev Spaces W s, and H61der Spaces c~, under very mild conditions on the mother wavelet (See Daubechies, 1992, Chapter 9). There are other ways o f defining a random density; some are given in Thorburn (1986). Why does one need random densities? There are several places random densities have proven useful. For example, the comparison o f various density estimators requires a sequence o f random densities. The performance o f Bayes and especially F-minimax rules may be explored when the prior is elicited at random. In this note we define only compactly supported random densities. The method given in Section 2 can be extended with slight modifications to cover random densities on R. The key result that enables the definition of random density is the Parseval Identity. We state it in terms o f wavelets:
Result. If the family {~k} forms an orthonormal basis in L 2 then for any function f E L 2 jk
I(f,~k)l 2 = Ilfll 2.
(3)
Defining the coefficients djk = ( f , @k) to satisfy the condition ~ j k d2 = 1 implies that f f 2 ( x ) d x = 1. Remark. To define a random density in L 2 one does not need an orthonormal basis. It is sufficient to have a tight f r a m e )
2. Main result In this section we give a definition o f a wavelet-based random density, and prove that it integrates to one almost surely. The most important part o f the section is the constructive description of random coefficients Without loss o f generality we assume that the support o f the random density we want to generate coincides with the support o f the chosen mother wavelet. By rescaling and translating, any preassigned compact domain can be matched. All random densities in the examples we will give later are rescaled to the interval [0, 1]. An algorithm o f constructing a random density via wavelets is given. It is a generalization o f the RubinChen algorithm; see Chen and Rubin (1986). In what follows, we generate random wavelet coefficients. We start with the function coo~oo(x) + doo~boo(x) + dlo~klo(x) + d l l ~ l l ( x ) + d20~20(x)... We assume that the coefficient o f the "scaling" part, coo is zero. There is no substantial change in the algorithm if the coefficient 1A family of functions 9, in L2 is a frame if
V f E L2 AIIf]I2 <~~ ](f,9,)I2 <~BIIf]I2 n
for some constants 0 < A ~
B. Vidakovicl Statistics & Probability Letters 26 (1996) 315-321
/\ /\
/\ /\
/\
/\ /\
/\
317
/\ /\
/\
/\
Fig. 1. Tree of random variables ujk.
coo is fixed or random (between 0 or 1). All other coefficients are constructed using random numbers. The construction of the coefficients is such that they are subject to the constraint (3). The fact that compactly supported wavelets form an unconditional basis of L2 is used. We enlarge the class of possible random densities by assigning random signs to generated djks. Let Xjk be a family of i.i.d. Bernoulli(p) random variables, and let rs(p) be a random sign defined as rsjk(p) = 2Xjk-
1.
We suppose 0 ~
doo = rsoo( P ) V 5 - S uoo,
~/1 -
djk = r s j k ( P )
Ujk
2J
1-I Uj'k" j'k' E path(j,k )
(4)
(5)
This is not the only way of defining a random density via wavelets. The other ways are appealing from a nonparametric Bayes standpoint. The work in progress (Lavine, M., Mfiller, P., and Vidakovic, B.) connects wavelets and Polya trees and introduces random densities in a way suitable for Bayesian inference. The above definition gives a calculationally efficient way of constructing a random density. Any coefficient (5) needs only one new random sign and one new random number u. The previously generated random numbers from the path are used in the construction.
318
B. Vidakovie/Statistics & Probability Letters 26 (1996) 315 321
The name random density is justified by the following theorem:
Theorem 2.1. f f(x)dx=
l,
a.s.
(6)
Proof. From the definition of djk it follows: d2o q- d~o "4- d2l if- d~o -'f- d~l q- d~2 q- d23 q- . . . .
1 -- Uo0 q- 1 UO0( 1 _ UlO) q- ~1 U o 0 ( 1 - Ull )
l
1
+¼ uoou,o(1 - u2o) + ~ uooum(1 - u21 ) q- I uooull(1 - u22) q- ~ uOOUll(] - u23) @ ' "
(7)
The theorem will be true if the sum (7) is 1 a.s. It suffices to prove the following lemma.
Lemma 2.1. Let {u~} is a sequence of i . i d random variables on [0,1] such that P(ul = 1) < 1. Then 1~ Ui ----+0, i=1
a.s.
(8)
as n ---~c<~. n
Proof. Easy, by observing that I-Ii=l ui is a nonnegative supermartingale. It converges a.s. to 0 since it converges to 0 in probability (Chebyshev's Inequality). []
3. Properties and examples of random densities In this section we discuss expected values and smoothness of random densities. 1 Theorem 3.1. For the Haar wavelet and the "random sign parameter" p = ~,
(9)
E f ( x ) = l(x C [0,1]). In other words, the uniform [0,1] distribution is the expected value of the random density f .
Proof. We assume that the ujk have finite expectation 2. Also, the fact that the random sign is independent of ujks and that E(rs(1/2))=O will be used. The interchange of the sum and expectation that follows is justified by Fubini's theorem:
E
2 2 = E E djk ~jk(x) + 2E E E djkdj,k,@k(x)@k,(x) jk
=
E E 2d)k~:k(x) 2 jk
jk j~k ~
=
E jk
2 2k(x) Ed)k
=
E( 1 jk
-
2) J 2Jq2(2Jx k) -
ZJ
= ~ (1 - 2)2) y~ ~b2(2Jx - k) = ~ (1 - 2))J = 1. j k j
[]
B. VidakoviclStatistics & Probability Letters 26 (1996) 315 321
319
The choice of random variables {ujk} affects the global smoothness of random densities. Let c¢~([0, 1]) be the H61der space of functions, defined as follows: ~e~([0,1]) =
f E L°~([0, 1]); 1f(x + h ) - f ( x ) l SUPx, h Ihl ~
1j,
~c'([0, 1]) = { f C L~([0, 1]) U cn([0, 1 ] ) ; f (n) E c~c"([0, 1]), ~ = n + ~',0 < ~' < 1}. Theorem 3.2. Let the random density f be 9enerated by a sequence of i.i.d, random variables {ujk} such that Eujk = 2. Then f E cg~([0, 1]),~ = 1 log 2 ½, with probability 1. We implicitly assume that the mother
wavelet ~ belongs to the class c¢~, for ~ >~~. Remark. To generate smooth random densities one should choose the sequence {ujk} so that Eujk < ¼.
Proof of Theorem 3.2. It is known that the H61der space, cg~ can be characterized by wavelets in the following way: Let ~-'~j,kdjkq~k be the wavelet representation of the function g, then g C~
iff Vk Idjk[~
By SLLN Idjkl - E l ~ k l ~ 0, j ~ o~,
a.s.
Jensen's inequality gives
, / 2 J ( ! U 2) _ C . 2 -(l/2+=)j 2J
El~kl~< E ~ j2k= V
for ~ = ½ log z I and C = v/1 - 2. 9(x) = Y']~j,kdjk~jk(x) belongs to ~ with probability 1. The theorem follows from the fact that if 9 E c£~ then f = 92 E cg~. [] Figures 2-5 depict generated densities for different values of 2 using the DAUB 4 or DAUB 10 wavelet.
.2
.
.
Fig. 2. Random density with DAUB 10 and 2 = 0.01.
B. Vidakovic / Statistics & Probability Letters 26 (1996) 315-321
320
2
0.2
04
O~
O~
1
Fig. 3. Random density with D A U B 4 and 2 = 0.1.
0.2
0,4
0.6
0.~
Fig. 4. Random density with D A U B 4 and 2 = 0.25.
1
321
B. VidakoviclStatisties & Probability Letters 26 (1996) 315-321
j
ih '
'
'
'
o.~
0.4
"
"
"
o16
. . . .
oi~
Fig. 5. Random density with DAUB 4 and 2 = 0.5.
References Chen, J. and H. Rubin (1986), Some stochastic processes related to random density functions, Technical Report #86-40, Department of Statistics, Purdue University. Chui, C. (1992), An Introduction to Wavelets (Academic Press, New York). Daubechies, 1. (1988), Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math. XLI, 909-996. Daubechies, 1. (1992), Ten Lectures on Wavelets (SIAM, Philadelphia, PA). Donoho, D., I. Johnstone et al. (1992, 1993), Bank of papers at playfair.stanford.edu. Thorbum, D. (1986), A Bayesian Approach to density estimation. Biometrika 73, 65-75. Vidakovic, B. and P. Mfiller (1994), Wavelets for kids, Discussion Paper 94-13, ISDS, Duke Univ. Zeidler, E. (1985), Nonlinear Functional Analysis and its Applications, Vol. I (Springer, Berlin).