- Email: [email protected]
Gabor's signal expansion and the Gabor transform on a non-separable time–frequency lattice
Journal of the Franklin Institute 337 (2000) 291}301
Gabor's signal expansion and the Gabor transform on a non-separable time}frequency lattice Arno J. van Leest*, Martin J. Bastiaans Technische Universiteit Eindhoven, Faculteit Elektrotechniek, EH 3.21, Postbus 513, 5600 MB, Eindhoven, Netherlands
Abstract Gabor's signal expansion and the Gabor transform are formulated on a general, nonseparable time}frequency lattice instead of on the traditional rectangular lattice. The representation of the general lattice is based on the rectangular lattice via a shear operation, which corresponds to a description of the general lattice by means of a lattice generator matrix that has the Hermite normal form. The shear operation on the lattice is associated with simple operations on the signal, on the synthesis and the analysis window, and on Gabor's expansion coe$cients; these operations consist of multiplications by quadratic phase terms. Following this procedure, the well-known bi-orthogonality condition for the window functions in the rectangular sampling geometry, can be directly translated to the general case. In the same way, a modi"ed Zak transform can be de"ned for the non-separable case, with the help of which Gabor's signal expansion and the Gabor transform can be brought into product forms that are identical to the ones that are well known for the rectangular sampling geometry. 2000 The Franklin Institute. Published by Elsevier Science Ltd. All rights reserved.
1. Introduction Recently, a new sampling lattice*the quincunx lattice*has been introduced [1] as a sampling geometry in the Gabor scheme, which geometry is di!erent from the traditional rectangular sampling geometry [2]. While the basic operations in Gabor's
* Corresponding author. Tel.: #31-40-247-5720; fax: #31-40-244-8375. E-mail address: [email protected] (A.J. van Leest). 0016-0032/00/$20.00 2000 The Franklin Institute. Published by Elsevier Science Ltd. All rights reserved. PII: S 0 0 1 6 - 0 0 3 2 ( 0 0 ) 0 0 0 4 0 - 5
292
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
signal expansion and the Gabor transform*time shifting and modulation*are independent operations in the case of a rectangular time}frequency lattice, these operations are no longer independent in the case of non-separable time}frequency lattices, of which the quincunx lattice forms a particular example. In order to have a higher packing density in the time}frequency plane, the non-separable lattice can be adapted to the shape of the windowed Fourier transformed analysis window. The windowed Fourier transform of a Gaussian function, for instance, yields circular contour lines; in this case, a quincunx (or hexagonal) lattice yields the highest packing density. Some numerical comparisons between a rectangular lattice, a quincunx lattice and a mismatched lattice for speech signals can be found in [3]. In this paper we show how results that hold for rectangular sampling (see, for instance, [4}6]) can be transformed to the general, non-separable case. After a short introduction of Gabor's signal expansion and the Gabor transform on the well-known rectangular lattice, we de"ne time shifting and modulation in the Gabor scheme on a general, non-separable lattice. The general lattice is described by a lattice generator matrix, and we pay special attention to the case in which the lattice generator matrix takes the Hermite normal form [7]. We continue with demonstrating how a general, non-separable lattice can be derived from a rectangular lattice by means of a simple shear operation, once the lattice generator matrix is represented in its Hermite normal form. Moreover, we show how this shear operation on the rectangular lattice corresponds to multiplication of the signal, the window functions, and the expansion coe$cients by quadratic phase terms. We thus arrive at a simple procedure to transform results that are well known for the rectangular sampling geometry, to the general, non-separable case. As an example we show how the bi-orthogonality condition for the window functions reads in the case of a non-separable time}frequency lattice. As a second example we show how the well-known product forms [4,5] for Gabor's signal expansion and the Gabor transform in terms of the Fourier transform of the expansion coe$cients and the Zak transforms of the signal and the window functions, can be formulated in the non-separable case.
2. Gabor:s signal expansion on a rectangular lattice We start with the usual Gabor expansion [2,4,6] on a rectangular time}frequency lattice, in which case a signal u(t) can be expressed as a linear combination of properly shifted and modulated versions g (t) of a synthesis window g(t): KI u(t)" a g (t), KI KI K\ I\
(1)
g (t)"g(t!ma¹)e I@R. KI
(2)
with
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
293
The time step a¹ and the frequency step b) satisfy the relationships )¹"2p and ab)1; note that the factor 1/ab represents the degree of oversampling, and that in his original paper [2] Gabor considered the case of critical sampling, i.e. ab"1. The expansion coe$cients a follow from sampling the windowed Fourier transform with KI analysis window w(t),
u(t)wH(t!q)e\ SR dt, \ on the rectangular lattice (q"ma¹, u"kb)):
u(t)wH (t) dt"1u, w 2, (3) KI KI \ where the asterisk (H) denotes complex conjugation. The latter relationship is known as the Gabor transform. The synthesis window g(t) and the analysis window w(t) are related to each other in such a way that their shifted and modulated versions constitute two sets of functions that are biorthogonal: a " KI
g(t !ma¹)wH(t !ma¹)e I@R \R "d(t !t ). (4) K\ I\ The bi-orthogonality relation (4) leads immediately to the equivalent but simpler expression
¹ k a¹ "d , (5) g(t!ma¹)wH t! m# I ab b K\ where d is the Kronecker delta. In the case of critical sampling, i.e., ab"1, Eq. (5) I reduces to a¹ g(t!ma¹)wH(t![m#k]a¹)"d . I K\ 3. Gabor:s signal expansion on a non-separable lattice The rectangular (or separable) lattice that we considered in the previous section can be obtained by integer combinations of two orthogonal vectors v "[a¹, 0]2 and v "[0, b)]2. We thus express the lattice " in the form ""+n v #n v " n , n 39,. (6) We now consider Gabor's signal expansion on a time}frequency lattice that is no longer separable. We call a time}frequency lattice non-separable, if the time-shifts and
294
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
the modulations in the shifted and modulated windows g (t) and w (t) are not KI KI independent operations anymore. Such a lattice is obtained by integer combinations of two linearly independent, but no longer orthogonal vectors, which we express in the forms Dv "[aa¹D, cb)]2 and
Dv "[ba¹D, db)]2,
(7)
with a, b, c and d integers, a, b31>, D"ad!bc, and )¹"2p again. The "rst component in the vectors v and v corresponds to a time-shift aa¹ and ba¹, respectively, while the second component corresponds to a modulation by a frequency cb)/D and db)/D, respectively. Each point k3" in the time}frequency plane can be obtained by a matrix}vector product [cf. Eqs. (6) and (7)]
1 a¹D 0 ∀k n 9 k"ULn, with U" Z Z D 0 b)
and L"
a b c
d
.
The column vectors v and v are the columns of the lattice generator matrix UL. Note, moreover, that D is equal to the determinant of the matrix L. We assume that the integers a and b have no common divisors, and that the same holds for the integers c and d; hence gcd(a, b)"1 and gcd(c, d)"1. A possible common divisor can be uni"ed with a and b. The area of a cell (a parallelogram) in the time}frequency plane is equal to the determinant of the lattice generator matrix UL, which determinant is equal to ab)¹"ab2p. It is well known that the set of shifted and modulated versions of the window is not complete in the case that ab'1. The equality ab"1 corresponds to critical sampling, and ab(1 corresponds to oversampling. Note that we only consider those lattice generator matrices that can be decomposed into the two matrices U and L. The corresponding lattices have samples on the time- and frequency-axes and are therefore suitable for a discrete-time approach, as well. It is clear that for a given matrix U, there are a lot of matrices L that generate the same lattice ". As an example, we have plotted in Fig. 1a and b two possible combinations of lattice generators in the case of a quincunx lattice. One form, the
Fig. 1. The quincunx lattice and two possible combinations of generator vectors.
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
295
Hermite normal form [7], see Fig. 1b, is very interesting; it reads
L "
1
0
!r D
,
where the integer !r equals h c#h d, with the integers h and h such that h a#h b"1. Note that these integers h and h exist, since gcd(a, b)"1, and that they can be obtained by the Euclidean algorithm (see, for instance, [8]). The columns of the matrix UL are equal to [a¹,!rb)/D]2 and [0, b)]2, respectively. Consequently, the shifted and modulated versions g (t) of the synthesis window g(t) on the KI lattice " take the form g (t)"g(t!ma¹)e\ KP@R "e I@R. (8) KI The shifted and modulated versions w (t) of the analysis window w(t) are de"ned KI similarly. With this modi"ed de"nition (8) of the set of shifted and modulated window functions, the original expressions for Gabor's signal expansion (1) and the Gabor transform (3) remain valid in the non-separable case. Note that in the case of a rectangular lattice, i.e., D"1 and r"0, Eq. (8) indeed reduces to Eq. (2).
4. From the separable to the non-separable case A lot of algorithms to calculate the Gabor transform and Gabor's signal expansion for the rectangular case in an e$cient way have been developed in the last two decades. Therefore, it would be very interesting if these algorithms could be reused for the non-separable case. We will show that this is possible by rewriting Eq. (2) as follows: g (t)"g(t!ma¹)e P@R\K?2 ?2"e\ P@R ?2"e\ p?@PK "e I@R. KI The same is done for the shifted and modulated versions w (t) of the analysis window KI w(t). Substitution of this into the Gabor expansion yields u (t)" a g (t), 1u , w 2g (t)" KI KI KI KI K\ I\ K\ I\ where u (t)"u(t)e P@R ?2", g (t)"g(t)e P@R ?2", w (t)"w(t)e P@R ?2"
(9)
a "a e\ p?@PK ". KI KI
(10)
and
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
296
The shifted and modulated versions of the synthesis window g (t) and the analysis window w (t) are now on a rectangular lattice [cf. Eq. (2)], i.e., g (t)"g (t!ma¹)e I@R and w (t)"w (t!ma¹)e I@R. KI KI Consequently, algorithms for the rectangular lattice can be reused, but the signal u(t), the windows g(t) and w(t) have to be pre-multiplied by a quadratic phase term. After calculating the Gabor coe$cients a , these Gabor coe$cients a have to be KI KI post-multiplied by a quadratic phase term to obtain the Gabor coe$cients a for the KI non-separable lattice. A similar procedure holds for the Gabor expansion. In this case, after reconstructing the signal u (t) by using an e$cient algorithm for a rectangular lattice, the signal u (t) has to be post-multiplied by a quadratic phase term to obtain the original signal u(t). As an example, we take the Gabor expansion on a quincunx lattice. Here we have r"!1 and D"2. This quincunx lattice is depicted in Fig. 1b. The lattice generator matrix UL has the form
a¹
0
b) b)
.
The columns of the matrix UL are the generator vectors of the quincunx lattice. These vectors are depicted in Fig. 1b, as well. Substituting r"!1 and D"2 into Eq. (9) yields u (t)"u(t)e\ @R ?2, g (t)"g(t)e\ @R ?2, w (t)"w(t)e\ @R ?2 and substituting r"!1 and D"2 into Eq. (10) yields a "a e p?@K. KI KI Following the procedure outlined above and substituting the &primed' window functions into the bi-orthogonality relation [cf. Eq. (5)], it is not di$cult to show that, in the case of a quincunx sampling geometry, the bi-orthogonality relation takes the form
¹ k g(t!ma¹)wH t! m# a¹ (!1)KI"d , I b ab K\ while in the general non-separable case this relation reads
¹ k g(t!ma¹)wH t! m# a¹ e\ pP "KI"d . I b ab K\
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
297
5. The Fourier transform and the Zak transform in the case of a non-separable lattice It is well known (see, for instance [4,6]) that in the case of critical sampling, ab"1, Gabor's signal expansion (1) and the Gabor transform (3) can be transformed into product form. We therefore need the Fourier transform a (t/¹, u/)) of the twodimensional array of Gabor coe$cients a , de"ned by KI a (x, y)" a e\ pKW\IV, (11) KI K\ I\ and the Zak transforms u (xa¹, y)/a; a¹), g (xat, y)/a; a¹), and w (xa¹, y)/a; a¹) of the signal u(t) and the window functions g(t) and w(t), respectively, where the Zak transform fI (t, u; q) of a function f(t) is de"ned as (see, for instance [4,6,9,10]) fI (t, u; q)" f (t#nq)e\ LOS. (12) L\ Upon substituting from the Fourier transform (11) and the Zak transforms [cf. Eq. (12)] into Eqs. (1) and (3), it is not too di$cult to show that Gabor's signal expansion (1) can be transformed into the product form
) ) u xa¹, y ; a¹ "a (x, y)g xa¹, y ; a¹ , a a while the Gabor transform (3) can be transformed into the product form
) ) a (x, y)"a¹u xa¹, y ; a¹ w H xa¹, y ; a¹ . a a In the case of oversampling by a rational factor, ab"q/p)1, with p and q positive integers, p*q*1, use of the Fourier transform (11) and the Zak transform (12) leads to the sum-of-products forms [5]
ap¹ ) 1 N\ r ap¹ r ) , y ; ap¹ " a x, y# g (x#s) , y# ; a¹ q a p p q p a P
u (x#s)
(13)
and r a x, y# p
ap¹ O\ ap¹ ) ap¹ r ) " , y ; ap¹ w H (x#s) , y# ; a¹ . u (x#s) q q a q p a Q We now combine the p functions
r a (x, y)"a x, y# P p
(r"0, 1,2, p!1)
(14)
298
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
into the p-dimensional column vector of functions a"[a (x, y), a (x, y),2, a (x, y)]2 N\ and, likewise, the q functions
ap¹ ) u (x, y)"u (x#s) , y ; ap¹ Q q a
(s"0, 1,2, q!1)
into the q-dimensional column vector of functions
"[u (x, y), u (x, y),2, u (x, y)]2. O\ Moreover, we combine the q;p functions
ap¹ r ) , y# ; a¹ g "g (x#s) QP q p a and
ap¹ r ) w "w (x#s) , y# ; a¹ , QP q p a where r"0, 1,2, p!1 and s"0, 1,2, q!1, into the (q;p)-dimensional matrices of functions
G"
g (x, y) g (x, y) .
g (x, y) g (x, y) .
2
g
2
g
2
N\ .
.
.
2
.
N\
(x, y) (x, y)
g (x, y) g (x, y) 2 g (x, y) O\ O\ O\ N\
and
W"
w (x, y) w (x, y) .
w (x, y) w (x, y) .
2
.
.
2
w
O\
(x, y) w
O\
(x, y)
2 2
w (x, y) N\ w (x, y) N\ . , .
2 wO\ N\ (x, y)
respectively. With the help of these vectors and matrices, the sum-of-product forms (13) and (14) can be expressed in the elegant matrix}vector products 1
" Ga p
(15)
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
299
and ap¹ a" WH , q
(16)
respectively, where, as usual, the asterisk (H) denotes conjugation and transposition. The matrix}vector products (15) and (16) hold in the case of the rationally oversampled, separable-lattice geometry. In the non-separable case they hold as well, but then for the `primeda vectors a and and the `primeda matrices G and W , derived from the `primeda functions (9) and (10). If we substitute from these `primeda functions into the Fourier transform (12) and the Zak transform (11), the `primeda Fourier transform takes the form a (x, y)" a e\ pKW\IV KI K\ I\ " a e\ ?@pP "Ke\ pKW\IV, KI K\ I\ while the `primeda Zak transform takes the form fI (t, u; q)" f (t#nq)e\ LOS L\ " f (t#nq)e P@R>LO ?2"e\ LOS L\ "e @ ?pP "R 2 f (t#nq)e @ ?pP "LO 2e\ LOS\P@R ?2". L\ 6. Conclusions We have shown how the Gabor scheme for the rectangular (or separable) lattice can be extended to the general, non-separable lattice in a structured way; this was achieved by describing the non-separable lattice by means of a lattice generator matrix. We have written the generator matrix in the Hermite normal form to obtain a shear representation on the shifted and modulated windows, which shear representation then led to a modi"cation of the rectangular Gabor scheme and resulted in the Gabor scheme on a non-separable lattice. E$cient algorithms that are well known for the rectangular case, could thus easily be applied to the non-separable case: we have shown that the shear operation merely involves pre- and post-multiplications by quadratic phase terms. As examples we have formulated the bi-orthogonality condition for the analysis and the synthesis window in the non-separable sampling geometry explicitly, and we have formulated product forms for Gabor's signal expansion and the Gabor
300
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
transform in terms of the (modi"ed) Fourier transform of Gabor's expansion coe$cients and the (modi"ed) Zak transforms of the signal and the window functions. Finally we note that, in general, other transformations can be used to transform the results in the rectangular Gabor scheme to the non-separable case. In particular we want to mention the fractional Fourier transform [11] in this context [12]. The fractional Fourier transform is a generalization of the classical Fourier transform and can be seen as a rotation by an angle h in the time}frequency plane. The non-separable lattice is now obtained by a rotation instead of a shear.
References [1] P. Prinz, Calculating the dual Gabor window for general sampling sets, IEEE Trans. Signal Process. 44 (8) (1996) 2078}2082. [2] D. Gabor, Theory of communication, J. Inst. Elec. Eng. 93 (III) (1946) 429}457. [3] W. Kozek, H.G. Feichtinger, Time}frequency structured decorrelation of speech signals via nonseparable Gabor frames, Proc. ICASSP-97, Munich, 1997, Vol. 2, pp. 1439}1442. [4] M.J. Bastiaans, Gabor's signal expansion and its relation to sampling of the sliding-window spectrum, in: R.J. Marks II (Ed.), Advanced Topics in Shannon Sampling and Interpolation Theory, Springer, New York, 1993, pp. 1}35. [5] M.J. Bastiaans, Gabor's expansion and the Zak transform for continuous-time and discrete-time signals: critical sampling and rational oversampling, Eindhoven University of Technology Research Report 95-E-295, Eindhoven University of Technology, Eindhoven, Netherlands, 1995. [6] H.G. Feichtinger, T. Strohmer (Eds.), Gabor Analysis and Algorithms: Theory and Applications, BirkhaK user, Berlin, 1998. [7] C. Hermite, Sur l'introduction des variables continues dans la theH orie des nombres, J. Reine Angew. 41 (1851) 191}216. [8] J.R. Durbin, Modern Algebra: An Introduction, 3rd Edition, Wiley, New York, 1992. [9] J. Zak, The kq-representation in the dynamics of electrons in solids, Solid State Phys. 27 (1972) 1}62. [10] A.J.E.M. Janssen, The Zak transform: a signal transform for sampled time-continuous signals, Philips J. Res. 43 (1988) 23}69. [11] L.B. Almeida, The fractional Fourier transform and time}frequency representations, IEEE Trans. Signal Process. 42 (11) (1994) 3084}3091. [12] M.J. Bastiaans, A.J. van Leest, From the rectangular to the quincunx Gabor lattice via fractional Fourier transformation, IEEE Signal Process. Lett. 5 (8) (1998) 203}205.
Arno J. van Leest received the M.Sc. degree in Electrical Engineering from the Eindhoven University of Technology, Eindhoven, the Netherlands, in 1996. Since 1996, he has been a Ph.D. student with the Department of Electrical Engineering, Technische Universiteit Eindhoven, in the Signal Processing Systems Group. His research includes e$cient algorithms for the implementation of the Gabor transform. Martin J. Bastiaans received the M.Sc. degree in Electrical Engineering (with honors) and the Ph.D. degree in Technical Sciences from the Technische Universiteit Eindhoven (Eindhoven University of Technology), Eindhoven, the Netherlands, in 1969 and 1983, respectively. Since 1969 he has been Assistant Professor and since 1985 Associate Professor with the Department of Electrical Engineering, Technische Universiteit Eindhoven, in the Signal Processing Systems Group, where he teaches electrical circuit theory, digital signal processing, signal theory and modern signal transformations, and Fourier optics and holography. His
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
301
research covers di!erent aspects in the general "eld of signal and system theory, and includes a signaltheoretical approach of all kinds of problems that arise in Fourier optics, such as partial coherence, computer holography, optical signal and image processing, and optical computing. His main current research interest is in describing signals by means of a local frequency spectrum (for instance, the Wigner distribution function, the sliding-window-spectrum, Gabor's signal expansion, etc.). Dr. Bastiaans is a Fellow of the Optical Society of America*`in recognition of distinguished service in the advancement of optics, particularly for pioneering contributions to the description of optical signals using local frequency spectrum conceptsa*and a senior member of the Institute of Electrical and Electronics Engineers. He has published about 90 papers in international scienti"c journals and proceedings of scienti"c conferences.
Gabor's signal expansion and the Gabor transform on a non-separable time}frequency lattice Arno J. van Leest*, Martin J. Bastiaans Technische Universiteit Eindhoven, Faculteit Elektrotechniek, EH 3.21, Postbus 513, 5600 MB, Eindhoven, Netherlands
Abstract Gabor's signal expansion and the Gabor transform are formulated on a general, nonseparable time}frequency lattice instead of on the traditional rectangular lattice. The representation of the general lattice is based on the rectangular lattice via a shear operation, which corresponds to a description of the general lattice by means of a lattice generator matrix that has the Hermite normal form. The shear operation on the lattice is associated with simple operations on the signal, on the synthesis and the analysis window, and on Gabor's expansion coe$cients; these operations consist of multiplications by quadratic phase terms. Following this procedure, the well-known bi-orthogonality condition for the window functions in the rectangular sampling geometry, can be directly translated to the general case. In the same way, a modi"ed Zak transform can be de"ned for the non-separable case, with the help of which Gabor's signal expansion and the Gabor transform can be brought into product forms that are identical to the ones that are well known for the rectangular sampling geometry. 2000 The Franklin Institute. Published by Elsevier Science Ltd. All rights reserved.
1. Introduction Recently, a new sampling lattice*the quincunx lattice*has been introduced [1] as a sampling geometry in the Gabor scheme, which geometry is di!erent from the traditional rectangular sampling geometry [2]. While the basic operations in Gabor's
* Corresponding author. Tel.: #31-40-247-5720; fax: #31-40-244-8375. E-mail address: [email protected] (A.J. van Leest). 0016-0032/00/$20.00 2000 The Franklin Institute. Published by Elsevier Science Ltd. All rights reserved. PII: S 0 0 1 6 - 0 0 3 2 ( 0 0 ) 0 0 0 4 0 - 5
292
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
signal expansion and the Gabor transform*time shifting and modulation*are independent operations in the case of a rectangular time}frequency lattice, these operations are no longer independent in the case of non-separable time}frequency lattices, of which the quincunx lattice forms a particular example. In order to have a higher packing density in the time}frequency plane, the non-separable lattice can be adapted to the shape of the windowed Fourier transformed analysis window. The windowed Fourier transform of a Gaussian function, for instance, yields circular contour lines; in this case, a quincunx (or hexagonal) lattice yields the highest packing density. Some numerical comparisons between a rectangular lattice, a quincunx lattice and a mismatched lattice for speech signals can be found in [3]. In this paper we show how results that hold for rectangular sampling (see, for instance, [4}6]) can be transformed to the general, non-separable case. After a short introduction of Gabor's signal expansion and the Gabor transform on the well-known rectangular lattice, we de"ne time shifting and modulation in the Gabor scheme on a general, non-separable lattice. The general lattice is described by a lattice generator matrix, and we pay special attention to the case in which the lattice generator matrix takes the Hermite normal form [7]. We continue with demonstrating how a general, non-separable lattice can be derived from a rectangular lattice by means of a simple shear operation, once the lattice generator matrix is represented in its Hermite normal form. Moreover, we show how this shear operation on the rectangular lattice corresponds to multiplication of the signal, the window functions, and the expansion coe$cients by quadratic phase terms. We thus arrive at a simple procedure to transform results that are well known for the rectangular sampling geometry, to the general, non-separable case. As an example we show how the bi-orthogonality condition for the window functions reads in the case of a non-separable time}frequency lattice. As a second example we show how the well-known product forms [4,5] for Gabor's signal expansion and the Gabor transform in terms of the Fourier transform of the expansion coe$cients and the Zak transforms of the signal and the window functions, can be formulated in the non-separable case.
2. Gabor:s signal expansion on a rectangular lattice We start with the usual Gabor expansion [2,4,6] on a rectangular time}frequency lattice, in which case a signal u(t) can be expressed as a linear combination of properly shifted and modulated versions g (t) of a synthesis window g(t): KI u(t)" a g (t), KI KI K\ I\
(1)
g (t)"g(t!ma¹)e I@R. KI
(2)
with
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
293
The time step a¹ and the frequency step b) satisfy the relationships )¹"2p and ab)1; note that the factor 1/ab represents the degree of oversampling, and that in his original paper [2] Gabor considered the case of critical sampling, i.e. ab"1. The expansion coe$cients a follow from sampling the windowed Fourier transform with KI analysis window w(t),
u(t)wH(t!q)e\ SR dt, \ on the rectangular lattice (q"ma¹, u"kb)):
u(t)wH (t) dt"1u, w 2, (3) KI KI \ where the asterisk (H) denotes complex conjugation. The latter relationship is known as the Gabor transform. The synthesis window g(t) and the analysis window w(t) are related to each other in such a way that their shifted and modulated versions constitute two sets of functions that are biorthogonal: a " KI
g(t !ma¹)wH(t !ma¹)e I@R \R "d(t !t ). (4) K\ I\ The bi-orthogonality relation (4) leads immediately to the equivalent but simpler expression
¹ k a¹ "d , (5) g(t!ma¹)wH t! m# I ab b K\ where d is the Kronecker delta. In the case of critical sampling, i.e., ab"1, Eq. (5) I reduces to a¹ g(t!ma¹)wH(t![m#k]a¹)"d . I K\ 3. Gabor:s signal expansion on a non-separable lattice The rectangular (or separable) lattice that we considered in the previous section can be obtained by integer combinations of two orthogonal vectors v "[a¹, 0]2 and v "[0, b)]2. We thus express the lattice " in the form ""+n v #n v " n , n 39,. (6) We now consider Gabor's signal expansion on a time}frequency lattice that is no longer separable. We call a time}frequency lattice non-separable, if the time-shifts and
294
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
the modulations in the shifted and modulated windows g (t) and w (t) are not KI KI independent operations anymore. Such a lattice is obtained by integer combinations of two linearly independent, but no longer orthogonal vectors, which we express in the forms Dv "[aa¹D, cb)]2 and
Dv "[ba¹D, db)]2,
(7)
with a, b, c and d integers, a, b31>, D"ad!bc, and )¹"2p again. The "rst component in the vectors v and v corresponds to a time-shift aa¹ and ba¹, respectively, while the second component corresponds to a modulation by a frequency cb)/D and db)/D, respectively. Each point k3" in the time}frequency plane can be obtained by a matrix}vector product [cf. Eqs. (6) and (7)]
1 a¹D 0 ∀k n 9 k"ULn, with U" Z Z D 0 b)
and L"
a b c
d
.
The column vectors v and v are the columns of the lattice generator matrix UL. Note, moreover, that D is equal to the determinant of the matrix L. We assume that the integers a and b have no common divisors, and that the same holds for the integers c and d; hence gcd(a, b)"1 and gcd(c, d)"1. A possible common divisor can be uni"ed with a and b. The area of a cell (a parallelogram) in the time}frequency plane is equal to the determinant of the lattice generator matrix UL, which determinant is equal to ab)¹"ab2p. It is well known that the set of shifted and modulated versions of the window is not complete in the case that ab'1. The equality ab"1 corresponds to critical sampling, and ab(1 corresponds to oversampling. Note that we only consider those lattice generator matrices that can be decomposed into the two matrices U and L. The corresponding lattices have samples on the time- and frequency-axes and are therefore suitable for a discrete-time approach, as well. It is clear that for a given matrix U, there are a lot of matrices L that generate the same lattice ". As an example, we have plotted in Fig. 1a and b two possible combinations of lattice generators in the case of a quincunx lattice. One form, the
Fig. 1. The quincunx lattice and two possible combinations of generator vectors.
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
295
Hermite normal form [7], see Fig. 1b, is very interesting; it reads
L "
1
0
!r D
,
where the integer !r equals h c#h d, with the integers h and h such that h a#h b"1. Note that these integers h and h exist, since gcd(a, b)"1, and that they can be obtained by the Euclidean algorithm (see, for instance, [8]). The columns of the matrix UL are equal to [a¹,!rb)/D]2 and [0, b)]2, respectively. Consequently, the shifted and modulated versions g (t) of the synthesis window g(t) on the KI lattice " take the form g (t)"g(t!ma¹)e\ KP@R "e I@R. (8) KI The shifted and modulated versions w (t) of the analysis window w(t) are de"ned KI similarly. With this modi"ed de"nition (8) of the set of shifted and modulated window functions, the original expressions for Gabor's signal expansion (1) and the Gabor transform (3) remain valid in the non-separable case. Note that in the case of a rectangular lattice, i.e., D"1 and r"0, Eq. (8) indeed reduces to Eq. (2).
4. From the separable to the non-separable case A lot of algorithms to calculate the Gabor transform and Gabor's signal expansion for the rectangular case in an e$cient way have been developed in the last two decades. Therefore, it would be very interesting if these algorithms could be reused for the non-separable case. We will show that this is possible by rewriting Eq. (2) as follows: g (t)"g(t!ma¹)e P@R\K?2 ?2"e\ P@R ?2"e\ p?@PK "e I@R. KI The same is done for the shifted and modulated versions w (t) of the analysis window KI w(t). Substitution of this into the Gabor expansion yields u (t)" a g (t), 1u , w 2g (t)" KI KI KI KI K\ I\ K\ I\ where u (t)"u(t)e P@R ?2", g (t)"g(t)e P@R ?2", w (t)"w(t)e P@R ?2"
(9)
a "a e\ p?@PK ". KI KI
(10)
and
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
296
The shifted and modulated versions of the synthesis window g (t) and the analysis window w (t) are now on a rectangular lattice [cf. Eq. (2)], i.e., g (t)"g (t!ma¹)e I@R and w (t)"w (t!ma¹)e I@R. KI KI Consequently, algorithms for the rectangular lattice can be reused, but the signal u(t), the windows g(t) and w(t) have to be pre-multiplied by a quadratic phase term. After calculating the Gabor coe$cients a , these Gabor coe$cients a have to be KI KI post-multiplied by a quadratic phase term to obtain the Gabor coe$cients a for the KI non-separable lattice. A similar procedure holds for the Gabor expansion. In this case, after reconstructing the signal u (t) by using an e$cient algorithm for a rectangular lattice, the signal u (t) has to be post-multiplied by a quadratic phase term to obtain the original signal u(t). As an example, we take the Gabor expansion on a quincunx lattice. Here we have r"!1 and D"2. This quincunx lattice is depicted in Fig. 1b. The lattice generator matrix UL has the form
a¹
0
b) b)
.
The columns of the matrix UL are the generator vectors of the quincunx lattice. These vectors are depicted in Fig. 1b, as well. Substituting r"!1 and D"2 into Eq. (9) yields u (t)"u(t)e\ @R ?2, g (t)"g(t)e\ @R ?2, w (t)"w(t)e\ @R ?2 and substituting r"!1 and D"2 into Eq. (10) yields a "a e p?@K. KI KI Following the procedure outlined above and substituting the &primed' window functions into the bi-orthogonality relation [cf. Eq. (5)], it is not di$cult to show that, in the case of a quincunx sampling geometry, the bi-orthogonality relation takes the form
¹ k g(t!ma¹)wH t! m# a¹ (!1)KI"d , I b ab K\ while in the general non-separable case this relation reads
¹ k g(t!ma¹)wH t! m# a¹ e\ pP "KI"d . I b ab K\
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
297
5. The Fourier transform and the Zak transform in the case of a non-separable lattice It is well known (see, for instance [4,6]) that in the case of critical sampling, ab"1, Gabor's signal expansion (1) and the Gabor transform (3) can be transformed into product form. We therefore need the Fourier transform a (t/¹, u/)) of the twodimensional array of Gabor coe$cients a , de"ned by KI a (x, y)" a e\ pKW\IV, (11) KI K\ I\ and the Zak transforms u (xa¹, y)/a; a¹), g (xat, y)/a; a¹), and w (xa¹, y)/a; a¹) of the signal u(t) and the window functions g(t) and w(t), respectively, where the Zak transform fI (t, u; q) of a function f(t) is de"ned as (see, for instance [4,6,9,10]) fI (t, u; q)" f (t#nq)e\ LOS. (12) L\ Upon substituting from the Fourier transform (11) and the Zak transforms [cf. Eq. (12)] into Eqs. (1) and (3), it is not too di$cult to show that Gabor's signal expansion (1) can be transformed into the product form
) ) u xa¹, y ; a¹ "a (x, y)g xa¹, y ; a¹ , a a while the Gabor transform (3) can be transformed into the product form
) ) a (x, y)"a¹u xa¹, y ; a¹ w H xa¹, y ; a¹ . a a In the case of oversampling by a rational factor, ab"q/p)1, with p and q positive integers, p*q*1, use of the Fourier transform (11) and the Zak transform (12) leads to the sum-of-products forms [5]
ap¹ ) 1 N\ r ap¹ r ) , y ; ap¹ " a x, y# g (x#s) , y# ; a¹ q a p p q p a P
u (x#s)
(13)
and r a x, y# p
ap¹ O\ ap¹ ) ap¹ r ) " , y ; ap¹ w H (x#s) , y# ; a¹ . u (x#s) q q a q p a Q We now combine the p functions
r a (x, y)"a x, y# P p
(r"0, 1,2, p!1)
(14)
298
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
into the p-dimensional column vector of functions a"[a (x, y), a (x, y),2, a (x, y)]2 N\ and, likewise, the q functions
ap¹ ) u (x, y)"u (x#s) , y ; ap¹ Q q a
(s"0, 1,2, q!1)
into the q-dimensional column vector of functions
"[u (x, y), u (x, y),2, u (x, y)]2. O\ Moreover, we combine the q;p functions
ap¹ r ) , y# ; a¹ g "g (x#s) QP q p a and
ap¹ r ) w "w (x#s) , y# ; a¹ , QP q p a where r"0, 1,2, p!1 and s"0, 1,2, q!1, into the (q;p)-dimensional matrices of functions
G"
g (x, y) g (x, y) .
g (x, y) g (x, y) .
2
g
2
g
2
N\ .
.
.
2
.
N\
(x, y) (x, y)
g (x, y) g (x, y) 2 g (x, y) O\ O\ O\ N\
and
W"
w (x, y) w (x, y) .
w (x, y) w (x, y) .
2
.
.
2
w
O\
(x, y) w
O\
(x, y)
2 2
w (x, y) N\ w (x, y) N\ . , .
2 wO\ N\ (x, y)
respectively. With the help of these vectors and matrices, the sum-of-product forms (13) and (14) can be expressed in the elegant matrix}vector products 1
" Ga p
(15)
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
299
and ap¹ a" WH , q
(16)
respectively, where, as usual, the asterisk (H) denotes conjugation and transposition. The matrix}vector products (15) and (16) hold in the case of the rationally oversampled, separable-lattice geometry. In the non-separable case they hold as well, but then for the `primeda vectors a and and the `primeda matrices G and W , derived from the `primeda functions (9) and (10). If we substitute from these `primeda functions into the Fourier transform (12) and the Zak transform (11), the `primeda Fourier transform takes the form a (x, y)" a e\ pKW\IV KI K\ I\ " a e\ ?@pP "Ke\ pKW\IV, KI K\ I\ while the `primeda Zak transform takes the form fI (t, u; q)" f (t#nq)e\ LOS L\ " f (t#nq)e P@R>LO ?2"e\ LOS L\ "e @ ?pP "R 2 f (t#nq)e @ ?pP "LO 2e\ LOS\P@R ?2". L\ 6. Conclusions We have shown how the Gabor scheme for the rectangular (or separable) lattice can be extended to the general, non-separable lattice in a structured way; this was achieved by describing the non-separable lattice by means of a lattice generator matrix. We have written the generator matrix in the Hermite normal form to obtain a shear representation on the shifted and modulated windows, which shear representation then led to a modi"cation of the rectangular Gabor scheme and resulted in the Gabor scheme on a non-separable lattice. E$cient algorithms that are well known for the rectangular case, could thus easily be applied to the non-separable case: we have shown that the shear operation merely involves pre- and post-multiplications by quadratic phase terms. As examples we have formulated the bi-orthogonality condition for the analysis and the synthesis window in the non-separable sampling geometry explicitly, and we have formulated product forms for Gabor's signal expansion and the Gabor
300
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
transform in terms of the (modi"ed) Fourier transform of Gabor's expansion coe$cients and the (modi"ed) Zak transforms of the signal and the window functions. Finally we note that, in general, other transformations can be used to transform the results in the rectangular Gabor scheme to the non-separable case. In particular we want to mention the fractional Fourier transform [11] in this context [12]. The fractional Fourier transform is a generalization of the classical Fourier transform and can be seen as a rotation by an angle h in the time}frequency plane. The non-separable lattice is now obtained by a rotation instead of a shear.
References [1] P. Prinz, Calculating the dual Gabor window for general sampling sets, IEEE Trans. Signal Process. 44 (8) (1996) 2078}2082. [2] D. Gabor, Theory of communication, J. Inst. Elec. Eng. 93 (III) (1946) 429}457. [3] W. Kozek, H.G. Feichtinger, Time}frequency structured decorrelation of speech signals via nonseparable Gabor frames, Proc. ICASSP-97, Munich, 1997, Vol. 2, pp. 1439}1442. [4] M.J. Bastiaans, Gabor's signal expansion and its relation to sampling of the sliding-window spectrum, in: R.J. Marks II (Ed.), Advanced Topics in Shannon Sampling and Interpolation Theory, Springer, New York, 1993, pp. 1}35. [5] M.J. Bastiaans, Gabor's expansion and the Zak transform for continuous-time and discrete-time signals: critical sampling and rational oversampling, Eindhoven University of Technology Research Report 95-E-295, Eindhoven University of Technology, Eindhoven, Netherlands, 1995. [6] H.G. Feichtinger, T. Strohmer (Eds.), Gabor Analysis and Algorithms: Theory and Applications, BirkhaK user, Berlin, 1998. [7] C. Hermite, Sur l'introduction des variables continues dans la theH orie des nombres, J. Reine Angew. 41 (1851) 191}216. [8] J.R. Durbin, Modern Algebra: An Introduction, 3rd Edition, Wiley, New York, 1992. [9] J. Zak, The kq-representation in the dynamics of electrons in solids, Solid State Phys. 27 (1972) 1}62. [10] A.J.E.M. Janssen, The Zak transform: a signal transform for sampled time-continuous signals, Philips J. Res. 43 (1988) 23}69. [11] L.B. Almeida, The fractional Fourier transform and time}frequency representations, IEEE Trans. Signal Process. 42 (11) (1994) 3084}3091. [12] M.J. Bastiaans, A.J. van Leest, From the rectangular to the quincunx Gabor lattice via fractional Fourier transformation, IEEE Signal Process. Lett. 5 (8) (1998) 203}205.
Arno J. van Leest received the M.Sc. degree in Electrical Engineering from the Eindhoven University of Technology, Eindhoven, the Netherlands, in 1996. Since 1996, he has been a Ph.D. student with the Department of Electrical Engineering, Technische Universiteit Eindhoven, in the Signal Processing Systems Group. His research includes e$cient algorithms for the implementation of the Gabor transform. Martin J. Bastiaans received the M.Sc. degree in Electrical Engineering (with honors) and the Ph.D. degree in Technical Sciences from the Technische Universiteit Eindhoven (Eindhoven University of Technology), Eindhoven, the Netherlands, in 1969 and 1983, respectively. Since 1969 he has been Assistant Professor and since 1985 Associate Professor with the Department of Electrical Engineering, Technische Universiteit Eindhoven, in the Signal Processing Systems Group, where he teaches electrical circuit theory, digital signal processing, signal theory and modern signal transformations, and Fourier optics and holography. His
A.J. van Leest, M.J. Bastiaans / Journal of the Franklin Institute 337 (2000) 291}301
301
research covers di!erent aspects in the general "eld of signal and system theory, and includes a signaltheoretical approach of all kinds of problems that arise in Fourier optics, such as partial coherence, computer holography, optical signal and image processing, and optical computing. His main current research interest is in describing signals by means of a local frequency spectrum (for instance, the Wigner distribution function, the sliding-window-spectrum, Gabor's signal expansion, etc.). Dr. Bastiaans is a Fellow of the Optical Society of America*`in recognition of distinguished service in the advancement of optics, particularly for pioneering contributions to the description of optical signals using local frequency spectrum conceptsa*and a senior member of the Institute of Electrical and Electronics Engineers. He has published about 90 papers in international scienti"c journals and proceedings of scienti"c conferences.