Wavelet variations on the Shannon sampling theorem

BioSystems 34 (1995) 249-257

Wavelet variations on the Shannon sampling theorem Hubert

Bray’, Kent McCormick2,

R.O. Wells, Jr.*,3, Xiao-dong

Zhou

Rice University, Houston, TX 77251. USA

Abstract

The Shannon sampling theorem asserts that a continuous square-integrable function on the real line which has a compactly supported Fourier transform is uniquely determined by its restriction to a uniform lattice of points whose density is determined by the support of the Fourier transform. This result can be extended to the wavelet representation of functions in two ways. First, under the same type of conditions as for the Shannon theorem, the scaling coefficients of a wavelet expansion will determine uniquely the given square-integrable function. Secondly, for a more general function, there is a unique extension from a given set of scaling coefficients to a full wavelet expansion which minimizes the local obstructions to translation invariance in a variational sense. Key words: Wavelets; Shannon

sampling

Foreword I would like to dedicate this paper to Hans Bremermann on the occasion of this special volume of papers published in his honour. I have known Hans Bremermann since I first got my PhD in several complex variables in 1965 and met Hans at that time in a very memorable meeting in Oberwolfach. Bremermann was extremely warm and cordial to me at the time, and we became

* Corresponding author, Computational Mathematics Laboratory, Dept of Mathematics, Rice University, Houston, TX 77251, USA. ’Present address: Dept. of Mathematics, Stanford University, Stanford, CA 94305, USA. ’ Present address: Dept. of Physics, University of California, Berkeley, CA 94720, USA. 3 Present address: Computational Mathematics Laboratory, Dept of Mathematics, Rice University, Houston, TX 77251, USA. 0303-2647/9_5/$09.50 0 1995 Elsevier SSDI 0303-2647(94)01457-I

Science Ireland

friends and met over the years in many places. My wife was also from Bremen, and she and he grew up a few blocks from each other, and we had occasion to see each other at random times in Bremen, and at least once ran into each other on the streets of Bremen, not knowing that the other was even in Europe at the time. Maribel and he were good friends to us over the years, and I visited Berkeley in the summer of 1967 at his urging, and that was an important development in my own career. I too have wandered from my original field of several complex variables, having worked in algebraic geometry, mathematical physics, and more recently mathematics education and applied mathematics and its interaction with engineering. This paper here is a collaboration between a number of people: Hubert Bray and Kent McCormick were two undergraduates at Rice at the time of the work (Bray is in graduate school at Stanford in mathematics, and McCormick is in

Ltd. All rights

reserved

250

H. Bray et al. 1 BioSystems

graduate school at Berkeley in physics), and Xiaodong Zhou is a Research Associate here at Rice with whom I have worked for several years now since he got his PhD at Rice with Bob Hardt several years ago. It is, in its own way, symbolic of the interaction which Hans Bremermann exemplified with his work and his spirit. R. 0. Wells, Jr. 1. Introduction The Shannon sampling theorem expresses the fact that a given continuous L2 function on the real line is determined by the restriction of the function to a specific lattice of points provided that the function is band-limited, i.e. that it has a compactly supported Fourier transform with suitable support. From another point of view, one could say that a suitable function defined on the lattice is extendable in a unique fashion to a L2 function among all such extensions continuous by requiring the band-limited condition on all potential extensions. In this paper we want to look at two different variations of this result in the wavelet setting. We assume that we have a fixed wavelet basis for L’(R) consisting of compactly supported wavelets defined by a scaling equation of the form 4(x) = c +&(2x

- k),

where {aO, . . . , a,,_, } are satisfying c

k

k + 21 =

(1.1) scaling

260,1~

coefficients

(1.2)

EIku=2,

(1.3)

where g is an invariant of the wavelet system called the genus. The wavelet function is defined by $(x) ‘=I

bkr6(2x

(1.4)

-k),

where 6, : = ( - l)ka,, _ , _k. These are all discussed in the book by Daubechies (1992). In particular, any function fe L2(R) has an expansion at a particular scale in terms of both scaling functions and wavelets of the form: f(x)

= k;z

ck+k(x)

+

c jsZ+,leZ

&+jdx)>

(1.5)

34 (1995) 249-257

where (b,(X) :=

4(X

-

k),

t+bj,(x)

:=

2j’2~(2’~ - k),

( 1.6)

are the translates and resealings of the scaling and wavelet function which appear in this expansion (here Z + denotes the non-negative integers). We will call the {ck} the scaling expansion coefficients, and {dj,> the wavelet expansion coefficients of the function f. The variation of the Shannon sampling theorem comes from asking the following question. Given the scaling expansion coefficients {ck} in the expansion of Eq. 1.5, under what conditions does this determine the function_/‘? In a different manner of speaking, under what conditions do the scaling expansion coefficients determine the wavelet expansion coefficients? In the context of the Shannon sampling theorem we see that if we identify a sampling of the given function f with its scaling expansion coefficients at a fixed scale (which is done regularly in practical applications of wavelets), then the Shannon sampling theorem tells us that, under suitable hypotheses, knowing the sampling implies that we know the functions at all points, and hence we know all of its coefficients in any orthonormal expansion. A simple case of an extension would be the trivial extension, i.e. setting djI = 0, for j E Z+, 1 E Z. When we identify a sampling of a function with its scaling function expansion at a given level, then we are replacing the orginal function with this trivial extension, and there is, in principle, some loss of information (see, for example, Wells and Zhou (1994) for an error analyis of such an interpolation). In this paper we give two new extensions which are different from this trivial extension. The first makes the hypothesis that the resulting extension function satisfies a band-limited condition, and an interpolation formula similar to the Shannon interpolation formula is given. Namely, given a specific choice of scaling expansion coefficients there is a unique choice of an extension f e L2(R) such that supp 1~ ( -4,;). This is discussed in Section 3. The second extension theorem relates to the fact that the scaling and wavelet expansion coefficients are not translation invariant. A varia-

251

H. Bray et al. 1 BioSystems 34 (1995) 249-257

tional principle is formulated which minimizes the changes in wavelet expansion coefficients under translations for a given extension of the scaling expansion coefficients {ck}. It is shown that there is a unique extension which minimizes this variational integral, and recursive formulas are given for the wavelet expansion coefficients in terms of the scaling expansion coefficients. This second extension always exists, and makes no a priori hypothesis about the extended function. It is not clear at this time how these two unique extensions of a given set of scaling expansion coefficients are related to each other. In Section 2 is a discussion of the classical Shannon sampling theorem, and it is shown to be a simple consequence of the classical Poisson summation formula from Fourier analysis. This point of view is utilized to formulate and prove a wavelet variation of this same result in Section 3. In Section 4 we find a formulation of the variational principle for extensions of scaling expansion coefficients discussed above, and the existence of a minimizing extension is formulated, the proof of which depends on some delicate estimates which are given in Section 5. Finally, in Section 6 recursive formulas for the wavelet expansion coefficients are derived. We note in conclusion that other authors have considered wavelet variations of the Shannon sampling theorem, and we refer the reader to Frazier and Torres ( 1993) and Walter ( 1992) and the papers cited therein. 2. The Shannon sampling theorem We have the following version of the Shannon sampling theorem (Shannon, 1949) which is convenient for our purposes. We let 6, be the Dirac delta measure at the point y E R. We use the convention of the Fourier tranform defined by

F(f)(<) z=_?(t) :=

s

ec2xixef(x) dx,

(2.7)

R

and we extend this in a natural sures and distributions, as given, Stein and Weiss (1971). Let Z = cific interval of unit length which in the calculations below.

manner to meafor instance, by [ -&] be a spewill play a role

Theorem 2.1. Let fe ous square-integrable f^c (-$,i). Then g(x) =f(x)

F k=-s

determines

L*(R) n C(R) be a continufunction. Suppose supp

(2.8)

6,

_K

Proof Let g be the measure given by the Eq. 2.8, and take the Fourier transform of both sides, using the fact that

F

(

k;z6k

1

(2.9)

=kTz6*,

which is the Poisson summation formula expressed in this language (see Stein and Weiss (1971)). We obtain %?)(5)

= S(f)(5)

*

( ,Jm

Sk 1

Hence we obtain, letting x1 be the characteristic function of the interval Z, =U-)

= P(g)

XI

Finally we see, by taking the inverse Fourier transform, that f is a function of the scaling expansion coefficients, as given by

f =g * F-‘(x,). This last formula is thus a formula for f in terms of g, which is the sampling data, i.e. the values of fat the integer lattice. In particular, the (inverse) Fourier transform of x1 is the sine function, which gives the classical Shannon formula. 3. A wavelet variation of the Shannon sampling theorem In this section we want to show a variation the classical theorem of Shannon.

on

Theorem 3.1. Let f E L2(R) n C(R) be a continuous square-integrable function. Suppose S := suppfc Z : = ( -$$, then the scaling expansion coefficients off determine f. Proof First we note that the scaling expansion coefficients can be expressed in the form

H. Bray et al. / BioSystems 34 (1995) 249-257

252

(f* 6)(k)=

sm sm

the scale off corresponds to the scale of $, then we do not expect f to have significant energies in the higher frequency parts of the wavelet spectrum. That is,

f(O&k - 6 dt,

-cc

f(&Xt - 4 dt = U IA >,

=

-cc

where we use the standard convention and (. 1.) is the I(X) : = f( -x), product. Hence, we define

of letting L2 inner

(3.10)

g=(f*&

Taking the Fourier transform of Eq. 3.10 and using the Poisson summation formula (Eq. 2.9) as before, we find y(g)

4) * -f

= F(f*

k=

6,

--m

p(J)) * .f

=(9(f)’

k=

Since supp I

wd

and thus we obtain fcg*

9-1

cV

I *jk)2

to be small for choices of the translation parameter c (f”(x): =f(x - E)). Hence we want the reconstructed function, 1 to have the minimum amount of small scale information. So we choose _?so that

’da C

cf”

I $jk )'

s0

xs

is a minimum. If we consider span{ $#,} then

that

&

is small. Here, and in what follows, the sum is over all non-negative resealings j E Z, j 2 0, and over all translates k E Z. In addition, since the scale off is independent of its translation, we expect

#(_?I:=

= @-“(d

= md

1 $jk >'

-CL

3~ S c I,

which implies

F(f)

6k.

ccf

any variation,

g, in

3

that

(xs ) F:(4) .

for f a minimum, But

where g =

CBjk$jk and g” =

cpj@jk.

(3.11) 0

w3+ erg)

s I

Remark: Note that the Fourier transform of the scaling function Y( +) will vanish at the boundary of Z for many wavelet systems, so it is necessary for the support off to be somewhat smaller to ensure the convergence of Eq. 3.11.

=

0

(4.12)

dEC~+agEI~jk)(~jkI$f(XgE)

and hence

W(f+ @>ldJ

;

4. A variational principle: existence of the implicit coefficients We assume that for some given function f a set of ‘sampled data points’ is given by {ck} where ck = (fl&). Th e p ro bl em we consider is to reconstruct the function f from these scaling expansion coefficients. Let f, : = &&. Clearly, f, has the same scaling expansion coefficients as J but f, need not be equal to f at all and may not be a suitable ‘reconstruction’ off. If we assume that

=~~~.,i'd&~(*L.li,)(i,l/5)=0.

this gives the conSince the B,,, are independent, ditions on J” we are looking for: I d& 1

<$Rn

I $jk)($jk

IFE>=O,

s0 for all m E Z+, n E Z.

(4.13)

H. Bray et al. 1 BioSystems

253

34 (1995) 249-257

Moreover, if we have two such minima, 7 and 2 satisfying Eq. 4.13, then, since the functional X(y) is quadratic inI the minimizing points and critical points coincide. Hence if f and g are minima, then f + g is also a minima. Furthermore, if x d are minima and

to the constraint

UI6k>=(Zl&J~

L = C Ckbk + 1 bjk$jk

Then f

-

g is a minima

and

U-dI&>=O. But 0 is also a minima

satisfying

(014,)

= 0, so

1

9(3-i) which

=

implies

ds C @ - 2” 1a,hjk)* = 0

s

then Proof

VkEZ, is uniformly

llf;llL2

bounded.

Let

be a minimizing sequence, where the {ck) are fixed as part of the initial data, and the wavelet expansion coefficients bjk(i) depend on the sequence, and we will write bjk for simplicity, letting the dependence on the sequence index i be implicit. Now set

s0 that

We now formulate the following existence theorem. First recall that a wavelet system {$k}, {$jk} has regularity of order N if x$(x)

(f+#Q=ck

dx = 0,

I= 0,

and moreover,

we see that

IK IIt2 = 1 e: + 1 bJ2k, which we can rewrite

in the following

form:

. . . , IV.

R

Daubechies (1988) showed that there exist wavelet systems of regularity N = 0, . . . , g - 1, where g is the genus of the wavelet system. Theorem 4.1. Let {c$~}, {+jk} be a wavelet system of regularity N at least 2. Let {ck} be a given set of scaling expansion coefficients, then there exists a unique f~ L*(R) which has (f4k) = c,, and which is a minimum of the functional 2. The proof of this theorem is given in the next section. We do not know if the regularity hypothesis in Theorem 4.1 is necessary or not. 5. Estimates for the minimization extension theorem The proof of Theorem 4.1 is given by showing that a minimizing sequence for the functional S(f) converges weakly in L*(R). We start with the following lemma.

Ilr;.l:z=~eZ+

o’.x(ft,s)ds s

+

’G’f’WWf’UiAd~

[S 0

1

where 2(.A,

s) ‘=I

cf;(x

- s) I $jk >‘.

~‘oA?(~, c)de so we may Now we are minimizing assume that this integral is bounded by I, which implies that Ilr;.IliZ II

c: + l- +

'W"(LJ9

-X"(h,4)

s0 Now *CL,

0) - ~(.tY~ ~1 = 1 blfk - 1 (f’; I ll/,k )* - 1 Cpi 1 *jk >*

sequence Lemma 5.1. If V;} is an approximating of functions for the minimization of X(f) subject

d&.

254

H. Bray et al. / BioSysiems 34 (199.5) 249-257

because Pi

Now if we could

PE, + PE2, so

=

Ilr; Iliz I c c; + r +

’MC

bj:, - C Cf’; 1$j!t >‘>

that

1’ de C bi:, - 1 (P; 1tjjk )’ I CIC b$, Jo

JO

dE1

+

prove

where CL< 1 then we would (Pi

have

1 $jk)*

s

Ix Cpi

+ 2

1 *jk)Cpt

I $jk)I.

J

+2@&--ij+CX~b;;.

But CcPi

I*jk)CpE2

=(Pi

I P$)-C

Let q * = cbTk then

I tijk)

(Pi

I

I 4, xp; I h>,

(1 - cc)+ < c c: + 1- + 2Jcc:. which implies

and (P;IP;)=o,

. Yf

that

(I-r)q’-2~lj-(~c:+T)
SO

and thus we see that q is bounded.

Lemma 5.2. Let f

By the Cauchy

1

if and only if IF* 4 Iliz s @z IlrlIt2

.(,(PiIbi)‘)(~(P;/m,)2)

5 lip; Ilt+;

~bjkI#!fjk. Then

inequality

I4O 12

J$ cpi

=

Iit2 = (1

where c:)(I

h;),

IIf* 4 IIt2=

j~121&12 d5.

SO

Proof jtZ~kbZ
By the assumption

I

I*jk)CpE21*jk)

Sl

de

s0

b&)1’*,

s(;c$;'(~

on L we have

Cb;k

-C

cf", $jkj2

(

)

=i’d’(~“~~:‘-[~“:‘-Zcf:s,,‘])

which

implies

that

=6’+VkW2)

s c [S I

+ S’ dE(x 0

b& - 1 (PE2 I $jk

)2)*

zz

0

de k

1

j-(x - E)+(X - k) dx *

R

H. Bray et al. 1 BioSystem

+ k -E)+(X) dx -k+l

=sm dW* dz

=

-k

4k

f(x

LS

- z)~(x)

dx

R

1 1

I(Z

*

255

34 (1995) 249-257

+a(1

m v((5)12dr s -m

-r))

since CI< 1.

< 1

We now have the following Lemma which will complete the proof of the boundedness of the minimizing sequence.

’

4>(d12

Lemma 5.3. Suppose

= ,,G 4 IIL = IF4 /IL = IImtz

(5.14)

0 We know

for the wavelets

of interest

that

then, there exists an E > positive CY< 1, such that

0 such that

there

is a

161’< 1 and,

since these are positive

Proof

lf^l*g d5,

v1’1$1’ d5 I s

quantities, From

Ineq.

5.14 it follows

s

where that

g is a positive

real-valued

function

such

f=-$Fbjk2-ji2eXp

=7 2- “?,(;); bjk

,,,[FJ,

g(x) 2 I$(x)I’. Now let u be a C” function ( - 2,2), u 2 0 everywhere

with support in and ~(4) E 1 on

( - 191). Suppose that for some E > 0

and hence, li‘cx)I’ =r.f”=

j;2 .

2(-“2)‘j~+~2~~(~&)

’ kxk bjtk,bizkz exp(i($ Ii 2 where CL<

1 (this is shown

But on (E, co) u( - co, -E) < l,sotake g=r+(l-r)p

bel_ow in Lemma

5.3).

[$I’ is bounded

by T

We may assume j, introduce a constant, integral E given by

- $))2n<

2 j, since this will only so we have to estimate the

X ; )

0

then on (E, co) u( - co, -E), g 2 z 2 [$I’ and on ( - E,E) ~(x/E) = 1, and therefore, on this interval, g = 1 2 \$I’. Hence T + (1 - r)u(c) is an upper bound for I$/‘. But

s

that

-1 q((i’) l*g(5) dt = (i:

li’
5

0

x p - d5. & Consider

the factor

exp( kxk bjlklbizkz 1. * and thus we obtain do _-3t v(012g(5) d5 s

i($

- $))2n:,

(5.15)

and let fi = k, - 2j2-Ak,, then 5.15 becomes

(5.15)

256

H. Bray ei al. / BioSystems

34 (1995) 249-257

+y/.d ;

h(t)c’“-’

So, by substituting for E, we obtain

Eq. 5.16 into

the expression

+ 2N(2N - 1)~ ; 0

5 h’(t)2N-’

+ 2Np

0

0 h(t)<2N-2

and thus r”(k) is uniformly bounded on ( - 2c,2~) since every l/c2 has a 5’ and every l/& has a 4 factor. Thus y”(k) is bounded by some constant C which is also independent off;. This is important because

Now since G(O) = 0 of order N by the moment vanishing property of the wavelet and 4 is an analytic function we have

i($o) =&N for some Finally

C” function let

and hence we have the estimates

h(5)t2” h(k).

N(B)= C bj2,b+2J2-JlkI bj,,k, kl

then the integral E

=

1

E becomes

2(-‘/2)(i1

+jd

NW = C bj2,0+2~2m~lkIbjl,k,

.i,,i2 .i2 2J.l

kl 2E

(S0

so by the Cauchy-Schwarz

5

a B

-2E

AN

h(t)t2

%

x exp(y)

dE)N(P).

Let r(k) = ~(41s)

h(c)c’”

then and hence we obtain IN(

+ 2Np

;

h(<)t2”-’

5

=‘$

Consequently,

0

-IN( < c b$ 1’ pD2’ B P’

and

1

1 r “ = 1& p ” f 0

h’(t)t’”

f

s h’(5)(5)2N+p

+i,’ 0

0

h”(t)t*”

and thus we are left with

inequality

we have

257

H. Bray et al. / BioSystems 34 (1995) 249-257

(

1/2)(11+A).

-

2-W

+i2W

2%

Lx,.b;)

km =

s’ 0

.c,.~
+

Thus we obtain j;:,

i&)l’p(+.

C,. xb$,

A,,,

0

as required.

=

so’

1 bj/c dE5: (‘h&n14/)(b, I $jk >. may may

be be

o’da<@ 1d%,,>(bn 14”) s

by the scaling equation ing equation again, A m,n=

Proof of Theorum 4.1: Let V‘EL2(C):~,$lk)

I

Once the integrals are evaluated, {bjk} found by diagonalization. The integrals reduced to linear combinations of

and we can choose E = 1/2C, and obtain

Vc:=

dE1 (II/L, I4~)<4,I4)

= i,,iZ,i3,i4

(Eq.

1.1). Using

the scal-

ai, a,,%,%, 8

= C,,kEZ} + A2m+rz~i,-1,2n+i~-i4-11

Then

Choosing then

vk}

to be the minimization

sequence,

which allows one to compute the coefficients explicitly. This is similar to the calculation of connection coefficients given by Latto et al. (in press) and Beylkin (1992), and we omit any further details here. References

Hence

there

exists

a subsequence

f EL2, such that fk,+f weakly in Fatou’s lemma implies f E V, and

{fk,)p, and L2, as 1 +

co.

-Wf> =,i$ -%d. < 6. Computing the wavelet expansion coefficients in terms of the scaling expansion coefficients In this section we will show that the unique choice of extension described in the previous two sections can be given explicity by an iterative algorithm. The minimization condition requires ’ dE

C

(II/L

I $jk

><$jk

IX

> ~0.

(6.18)

s0 From linearity we may choose {ck} so that c, = 6,,,. sof= 6. Expanding the condition of Eq. 6.18 above we obtain 0 = <$mn 1i> and hence

o’dE T
14, >(d’, 1di >>

Beylkin, G., 1992, On the representation of operators in bases of compactly supported wavelets. SIAM .I. Numer. Anal. 6, 1716-1740. Daubechies, I., 1988, Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math. 41, 906-966,. Daubechies, I., 1992, Ten Lectures on Wavelets (SIAM, Philadelphia). Frazier, M. and Torres, R., 1993, The sampling theorem, $-transform, and Shannon wavelets for R, 2, T, and Z,, in: Wavelets: Mathematics and Applications, Benedetto and M. Frazier (eds.) (CRC Press, Boca Raton FL) pp. 221-245. Latto, A., Resnikoff, H.L. and Tenenbaum, E., 1994, The evaluation of connection coefficients of compactly supported wavelets, in: Proceedings of the French-USA Workshop on Wavelets and Turbulence, June 1991, Y. Maday (ed.) (Springer-Verlag, New York). Shannon, C.E., 1949, Communication in the presence of noise, Proc. IRE 37, 10-21, Stein E.M. and Weiss, G., 1971, Introduction to Fourier Analysis on Euclidean Spaces (Princeton University Press). Walter, G.G., 1992, A sampling theorem for wavelet subspaces. IEEE Trans. Inf. Theory, 38, 881-884. Wells, R.O. and Zhou, X., 1994, Wavelet interpolation and approximate solutions of elliptic partial differential equations, in: Noncompact Lie Groups. Proceedings of NATO Advanced Research Workshop. R. Wilson and E.A. Tanner (eds.) (Kluwer).