- Email: [email protected]
Wavelets — something for analytical chemistry?
451
trends in analytical chemistry, vol. 16, no. 8, 1997
Wavelets - something for analytical chemistry? B. Walczak*
D.L. Massart
as signal decomposition tions, whereas in order offered by wavelets, a preferred. For practical signal processing is the
ChemoAC, Pharmaceutical Institute, Vrije Unversiteit Brussel, Laarbeeklaan 103, B- 1090 Brussels, Belgium
2. Wavelets
Institute of Chemistry, Silesian University, 9 Szkolna Street, 40-006 Katowice, Poland
Wavelets have shown great applicability in many diverse fields of science, and are now becoming of interest in analytical chemistry. This paper is intended as a first reading, introducing fundamentals of wavelets and wavelet transforms, and some applications thereof, such as signal compression and denoising, image processing, data set compression and the modeling of multivariate data sets. 1. Introduction This paper is addressed to all those who prefer to get insight into an analytical system via the signal given in Fig. la, rather than that in Fig. lb. We will present a story of the small waves called wavelets, which have recently made a revolutionary breakthrough in signal and image analysis. Although they offer numerous advantages over traditional signal processing methods, their penetration into chemistry and its problems can be described as retarded and hesitant. The theory of wavelets can be viewed as a common framework for the techniques that have been developed in various fields independently. Different origins of this theory lead to different languages, which additionally vary when shifting from continuous to discrete-time signals analysis. This diversity is helpful when describing different aspects of wavelet analysis. For certain applications it is convenient to consider wavelet analysis
*Corresponding author. Fax: +48 (32) 599 978. 01659936/97/$17.00
HISO165-9936(97)00065-4
onto a set of basis functo explain the advantages multiresolution analysis is reasons, the language of most suitable.
as a basis
The instrumental signals that we usually deal with are discrete-time signals, presented as a finite sequence of numbers. For instance, let us consider the following sequence of 4 numbers: (2, 3, 1, 7) (Scheme 1). It can be thought of as a vector in a 4-dimensional space and uniquely presented as a linear combination of the vectors in its basis: ( 1, 0, 0,0),(0, 1, 0,0),(0,0, 1,O) and (0,0, 0, 1). This basis, known as the canonical basis, has a valuable property: its vectors are linearly independent (i.e. orthogonal) and as their lengths equal 1, they are also orthonormal. In matrix notation this is: f=Sb where S is a matrix containing basis vectors in columns and b is a vector representing a set of coefficients. The canonical basis is one of many possible bases. Signal f can uniquely be presented as a linear combination of another set of basis vectors (see Fig. 2): f = Snew bnew where S”“” is a matrix containing new basis vectors in columns and b”“” is a vector representing a new set of b coefficients. By changing basis vectors we change signal representation. In a new basis, the signal is represented by a set of the new b coefficients, bnew . If the basis is orthonormal, the set of b coefficients can simply be Copyright
0 1997 Elsevier Science B.V. All rights reserved.
452
trends in analytical chemistry, vol. 16, no. 8, 1997
b)
a)
Fig. 1. The NMR signal [ 1 ] (a) after denoising and ( b) before denoising. calculated as: b new = (cpw)T
f
If the original signal contains N data points, so does the signal in the new basis. No information is gained or lost. So why would we want to change the basis? The answer is simple. The canonical basis is not always the best one. To represent even a very common constant signal ( 1, 1, 1,l) in the canonical basis, we still need 4 basis vectors. Would it not be nice to have a basis that includes a constant vector? Would it not be nice to have a basis that provides an accurate vector reproduction with a few vectors only? Here wavelets enter our story. Wavelets can be introduced as a new family of bases with special properties. Presentation of a signal in the new basis is generally called transformation (or a transform). For instance, projection of a signal onto the sine and cosine waves is known as a Fourier Transform, whereas projection of a signal onto the wavelet basis is called a Wavelet Transform. In each case the aim of the transform is to calculate a new set of b coefficients. In the case of the Fourier Transform they are called Fourier coefficients, while in the case of the wavelet transform they are called wavelet coefficients. New presentation of a signal can make a number of things easier. The goal of any transform is to reveal certain characteristics that are not apparent in the original signal, thus facilitating both analysis and processing of the transformation. Here we ought to mention that wavelet transform
Scheme 1.
can operate in continuous time (on functions) and in discrete time (on vectors). For functions and infinite signals the wavelet basis is infinite. For finite length vectors with N components, there will be N basis vectors and N transform coefficients. In this case wavelets are ordinary vectors (columns of matrix, S). Wavelets can be considered a new family of basis functions (vectors).
3. What makes wavelets special? Although there are endless possible bases for function space, almost all of them are uninteresting. To benefit from signal representation in another domain, basis functions ought to have some special properties. So what is special about wavelets? As their name suggests, wavelets are small waves (Fig. 3 ). Wavelets differ from zero in a limited domain only, called the support. We therefore see them as functions that are well localized in the time domain. But this is not the end of the story. Wavelet support can vary, thus allowing a different degree of wavelet localization. Moreover, wavelets are orthogonal (or even orthonormal). To achieve orthogonality with the local support was believed impossible until Daubechies’ work [ 21. Construction of the wavelet basis functions is quite intricate. Wavelets are obtained from a single prototype function by dilations and contractions (scaling), as well as by shifts (translations) (see Fig. 4). Shifts of the localized function are needed to cover the whole signal range, whereas scaling allows a multiresolution analysis of signals. Because all of the basis functions are derived from one prototype, they are self-similar (i.e. they have a fractal structure). Wavelets are obtained from a single prototype function by scaling, as well as by translation.
trends in analytical chemistry
al
453
vol. 16, no. 8, 1997
DanaLl
-1
.i::ii:iiiiiiiiiiiii*i:rr:i: ..J:‘r.:.‘...... . .
[=WI
=~1~ :!!!lii~iiifii::::::::::::i’
iiIi(i.:::::i:“t;::I:::;:..: . . . ..r...‘:.;:‘..rr::::.:::
S
ii:: ii
ii
:i:t,:.._.:::;;;;i..::iiiii
::::iii::i:L!iiIriI..:r!..i.
.iiliiliIIIiIiiiii;i:iiiijtI: ::::.“....I::::::::!i:.... a;!$;!i!i;iI,:
::::::::‘!iiiiii!i:::i::;...
=
...‘..I.
.::::t;;:::....:,::i!.;,..:. .t:::i._:::‘:::..i;;;~..::.:
b
=
b
::
ii
.iiiil;‘!!“iii’Lii.iijiii;I
f
.
::
S-
b-
b-
Fig. 2. a: Signal f can uniquely be presented as a linear combination of the different sets of basis vectors. b: By changing basis vectors, we change signal representation. In each basis the same signal is represented by a different set of b coefficients.
4. Multiresolution
analysis
The basic idea of wavelet analysis is that of multiresolution. This concept of the multiresolu-
-
Fig. 3. Wavelets.
tion approximation of functions was introduced by Meyer and Mallat [ 3,4], and provides a powerful framework to understand wavelet decomposition. The general concept of multiresolution can be
454
trends in analytical chemistry, vol. 16, no. 8, 1997
translation
scaling
Fig. 4. Illustration of the idea of translation and scaling: (a) examples of translated waveforms; (b) examples of scaled waveforms.
explained by a simple example taken from optics. Let us consider using binoculars, or a microscope. In both cases we apply the same tactics: if we intend to contemplate an object as a whole, we have to adjust an optical device to a lower resolution (thus disregarding perception of details). If, on the other hand, we are interested in the details of an observed object, we have to adjust the same optical device to a higher resolution. Similarly, to obtain a global view of any signal, we must look at it with low resolution. In order to see increasingly more details, we have to increase the resolution. The simultaneous appearance of a signal on multiple scales is known as multiresolution. The goal of multiresolution is decomposition of a function space into a nested set of subspaces: v”c
v’c
v*c
... WC
intrinsic property of these spaces is that as j increases, the approximation of the whole function space improves. As j decreases, these spaces contain gradually less information, because some ‘variations’ or ‘details’ are peeled off. To quantify the difference of information content in the two consecutive approximation spaces, we need to construct a complementary subspace, called the wavelet subspace. Thus the wavelet space IV-’ is the orthogonal component of Vj-’ in Vj (Scheme 2). approximatlon space at level j-1 $-l
VJC ...
which represent its successive approximations. The symbol C shows that the subspace Vi-’ is contained in the subspace Vj. The most important
wavelet space at level j-1
Scheme 2.
455
trends in analytical chemistry, vol. 16, no. 8, 1997
The symbol @ stands for the orthogonal sum in the sense that any signal in Vj can be uniquely written as the sum of a signal in Vj-’ and an orthogonal signal in I@-‘. The final requirement for multiresolution concerns a basis for each space. The basis functions for the approximation spaces are known as scaling functions. The basis functions for the wavelet spaces are known as wavelets. The nice part of this story is that all basis functions of the above described spaces can be generated from a single function, called the scaling function a,. We begin with a subspace V”, which we assume has an orthonormal basis consisting of the translations of the ‘localized’ scaling function. If the same function is compressed by a factor of 2 and then translated along the signal, it creates the basis for the vector space V’. In an analogous way we can construct the basis for the wavelet subspaces (see Fig. 5). We assume that IV0 has an orthonormal basis consisting of the translations of the wavelet functions, whereas the same function compressed by a factor of 2 and then translated creates the basis for the space IV’. Since V’CV’, it is possible to express all basis functions in V” in terms of the basis functions in V' ,
al
bl
subspace
VU
I
I
I
II
A
I
I
I subspace
W”
or more generally, since Vj-’ C Vj, it is possible to express basis functions in Vj-’ in terms of basis functions in Vj. Similarly, since Wj-’ CVj, the basis functions of the wavelet space Wj-‘, i.e. wavelets (‘I’), can also be derived from the scaling function. In the case when each VJ space has a finite basis, we can use matrix notation to express the fact that the scaling functions and wavelets at the levelj- 1 can be constructed as a linear combination of the scaling functions of finer scaling functions at the level j. U-‘(x) = @(x)Lj ‘I’+1 (x) = &(x)Hj where matrices Lj and Hj are the matrices of constants. The columns of L J are the shifted versions of one another, as are the columns of Hj, and therefore one column characterizes each matrix. To construct these columns, we need to know the sequence of numbers (CO,cl, .... CN), known as the wavelet filter coefficients. Matrices Lj and Hj are the matrices needed for transformation of any signal from the space Vj to the subspaces Vj-’ and Wj-‘, Matrices Lj and Hj are the matrices needed for
subspace
V’
subspace
W’
Fig. 5. a: Haar bases for the approximation subspaces V” and V’. b: Haar bases for the detail subspaces V” and V’. Basis functions for a given subspace are the translations of each other, while bases for the two consecutive subspaces differ due to scaling.
456
trends in analytical chemistry, vol. 16, no. 8, 1997
transformation of any signal from the space Vj to the subspaces Vj-’ and Wj-l.
scales of resolution, we apply the same transform. This is recursion - the same transform at a new scale.
5. Magic numbers
6. Discrete wavelet transform There exist many sets (families) of wavelets. They differ due to the compactness of the wavelets’ supports and their smoothness. We limit ourselves to the families of orthogonal wavelets, the Daubechies family [ 21, being the most popular among them. A particular wavelet within each family is uniquely specified by the set of coefficients CO,cl, ...) CN-
Each wavelet is uniquely characterized by the set of coefficients co, cl, .... cN. These coefficients can be found in the existing libraries of the wavelet filter coefficients. For instance, the first member of the Daubechies family (the eldest and the simplest wavelet, known as the Haar wavelet) is characterized by the two coefficients COand cl. The next member [ 21 is characterized by four coefficients CO,cl, ~2, c3 etc. These coefficients are used to transform a signal from the original domain to the wavelet basis, i.e. to construct the transform matrix ST (Scheme 3). It is helpful to think of the coefficients {CO, cl, .... cN } as a filter [ 5 1. Depending on the pattern in which these coefficients are ordered, we call them the low-pass and the high-pass filter, respectively. The low-pass filter, constructed with the coefficients {CO, cl, .... cN}, acts as a smoothing filter (in a way like a moving average), whereas the highpass filter, with coefficients { cN, -cN_l, .... cl, -CO}, can be considered a difference filter. In the field of signal processing this pair of filters is known as the quadrature mirror filter pair (QMF) [ 5 1. The output of the low-pass filter is called approximation, whereas the output of the highpass filter is called detail. In order to calculate the approximation and detail of the signal on different
There is an efficient hierarchical algorithm to perform wavelet decomposition that was proposed by Mallat (also known as a pyramid algorithm) [4]. It can be applied to any discrete-time finite signal, its length being equal to the integer power of 2. In the fast wavelet transform algorithm, the sampled signal is passed through the low-pass and the high-pass filters (see Fig. 6). Outputs of both filters are downsampled by a factor of 2. The high-pass filtered data set comprises the wavelet transform coefficients (details) at that scale of transform. The low-pass filtered data set comprises the approximation coefficients at that level of scale. Approximation coefficients can now be used as the sampled data input for another pair of wavelet filters (identical with the first pair), generating another set of the detail and approximation coefficients at the next-lower level of the scale. This process can continue until the limit for the unit interval is reached. The lower the level of the scale, the less information is contained in the approximation of the signal (in other words, more ‘variations’ are removed from the signal). In fact, these variations, peeled off level by level, are represented by the detail coefficients. In other words, the difference of information between the two consecutive approximations is contained in the detail coefficients at the corresponding level. Finally, the signal can be represented in the wavelet domain as a set of coefficients describing the roughest approximation and all details. An important feature of this discrete algorithm is its relatively low complexity. Independent of the depth of the tree, the complexity is linear in the number of input samples, with a constant factor that depends on the length of the filter.
7. From waves to wavelets
Scheme 3.
The best way to explain the advantages offered by wavelets is to compare them with waves, i.e. with sine and cosine, which are the basis functions used in the Fourier Transform (FT) [ 6 1. These basis functions differ in their frequencies, and FT
457
trends in analytical chemistry, vol. 76, no. 8, 1997
Original
lerel2
signal
a2
leuel1
I
level 0
d’
d2
I
I
Fig. 6. Scheme of Mallat’s pyramid algorithm for Discrete Wavelet Decomposition.
intends to analyze the frequency contents of the signals. Each coefficient of the FT corresponds to a single frequency only and provides information about the contribution by this frequency to the sig-
bl
al f
nal. It means that the FT decomposes the signal into its frequency components, with no time localization (see Fig. 7). Due to the global character (i.e. support) of the
Itime
&
time -
t t t c-
time
I time *
Fig. 7. Time-frequency representation: (a) of an original signal in time domain; (b) for Fourier decomposition; (c) for windowed Fourier decomposition; (d) for wavelet decomposition.
458
trends in analytical chemistry, vol. 76, no. 8, 1997
sine and cosine basis functions, the FT is not used in the case of non-stationary signals. Discontinuity occurring within the time domain affects the weights of all basis functions and therefore it is spread out across the whole frequency domain. A straightforward solution is to multiply the signal by a window function, in order to delimit the signal in time. A windowed FT can be used to give information about signals simultaneously in the time and frequency domains [ 71. A windowed FT with a fixed window size decomposes the time-frequency space into sections of equal dimensions. Although the windowed Fourier analysis is an improvement over the Fourier analysis, it is still inconvenient since the fixed window size makes the representation inadequate for approximation of singularities and for abrupt changes in the signal, and can result in the generation of fictitious features in a smoothed signal. As signals can contain contributions with varying degrees of localization in both time and requires frequency, an efficient representation basis functions that are localized in both the time and frequency domains. While sines and cosines provide complete frequency localization, wavelets are well localized in both the time and frequency domains. Due to this unique property of the wavelets, the Wavelet Transform can provide a much better time resolution for the high-frequency signals because it alters scales as the frequency increases. The higher the frequency, the shorter is the time window that it analyzes. Specific local properties of the wavelets can be particularly useful when describing signals with sharp spikes or discontinuities. Wavelets have a zoom-in and zoom-out property.
original signal
Fig. 8. Scheme sition.
of the full Wavelet
Packet decompo-
J Fig. 9. Examples
of possible
orthonormal
bases.
8. Where does this story end? So far we have limited our presentation to a Discrete Wavelet Transform (DWT), applicable to discrete-time finite length signals. Its extension is known as the Wavelet Packet Transform [ 8-101, which usually leads to a more efficient signal compression and is being used increasingly. 8.1. Wavelet packet transform In the DWT the detail branches are not used for further decomposition. If both filtering operations are applied to the approximation and to the detail coefficients, then the resulting subspaces form the full binary tree (see Fig. 8). Representation of the original signal in the wavelet packet domain is overcomplete, i.e. there are many possible orthonormal bases. Namely, any disjoint cover of the original data set is an orthonormal basis (see Fig. 9). The whole set of orthonormal bases is called the ‘wavelet packet library’ or ‘wavelet packet dictionary’. Using different criteria, one can select the socalled best-basis (where the term ‘best’ is defined by a user). Usually we are interested in such a basis in which a signal has a sparse representation, i.e.
trends in analytical chemistry
vol. 16, no. 8, 1997
there are only few coefficients with high amplitudes, whereas the remaining ones approach zero. Coefficients with small amplitudes can, without any significant loss of information, be omitted and the signal can be efficiently compressed. In order to find the best-basis with the desirable distribution of coefficients Coifmann and Wickerhauser proposed using the entropy criterion [ lo]. 8.2. Overcomplete
dictionaries
of waveforms
Wavelets and Wavelets Packets are merely the best known examples of the currently available signal representation schemes (dictionaries of vectors, also called waveforms or atoms) [ 111. Most of them are overcomplete, containing waveforms which have representation in terms of other waveforms, leading to a nonunique signal decomposition. Such nonuniqueness offers a possibility of adaptation (the most efficient representation of a signal can be chosen), but finding the optimal representation is not a trivial problem at all. The Coifman and Wickerhauser approach to the best-basis selection [ lo] can be applied to the Wavelet Packet and to the Cosine Packet only, because - although these are overcomplete dictionaries - there are special subcollections of their elements, yielding orthogonal bases for signal representation. In the case of the other overcomplete dictionaries, one has to choose from among many representations the one which is most suited for the decomposed signal, using such techniques as the Method of Frames, the Matching Pursuit or the Basis Pursuit (their sophistication and computational complexity increase in the order listed above) [ 111.
9. Applications The versatility of the wavelet transforms means that there are few branches of scientific research that cannot make use of them. Anything that involves the analysis of a time series is a good candidate for the wavelet treatment. An insight into chemical systems or phenomena via measured instrumental signals can be significantly improved by signal denoising and extraction of its relevant features.
459
9.1. Signal compression
and denoising
Perhaps the greatest potential of the wavelets has been claimed for signal compression and denoising (i.e. [ 12-141). The primary objective of any data compression is to transform the data to a form that requires the smallest possible amount of storage space, while retaining all the relevant information. Efficient compression, storage, and recovery of historical data is essential for many engineering tasks, as well as for further data processing and manipulation. Sparse representation of a signal in the wavelet domain allows for its significant compression. All wavelet coefficients with very small amplitudes can be eliminated without any loss of the relevant information. Compression is inherently associated with signal denoising, because small coefficients are assumed to represent the noise component of the signal. Their elimination ( thresholding [ 15 1) from the noisy signals is equivalent to data denoising. The thresholded signal is then inverse transformed, yielding a much clearer signal in which peaks can be distinguished (see Fig. 10). Moreover, wavelet decomposition of a signal allows characterization of the nature of its noise. Comparing the distribution of the detail coefficients along the first level of signal decomposition (associated with high frequencies), we can distinguish between the homo- and heteroscedastic noise types. Comparing the distribution of the detail coefficients among the first few levels of wavelet decomposition allows identification of a correlated noise type [ 16 1. Depending on the type of noise identified, one can vary the thresholding policy in order to optimize the denoising procedure. The power of wavelet transforms in signal compression and denoising is associated with the fact that - unlike the FT, which utilizes just a single set of basis functions - wavelet transforms have an infinite set of possible basis functions, among them the one that fits the type of signal analyzed. So far we have covered off-line processing of data only, but it is possible to apply waveletbased approaches to on-line compression and denoising as well. This opens possibilities of wavelet applications for on-line process control and in the analysis of process trends [ 17,181.
460
trends in analytical chemistry, vol. 16, no. 8, 1997
b)
4 60
20(
500
500 time domain
1000
wavelet domain d)
c)
wavelet domain
time domain
Fig. 10. a: Original signal. b: The same signal in the wavelet domain. c: The same signal in the wavelet domain after thresholding. d: Reconstructed signal.
9.2. Wave/et image processing
The multiresolution analysis of functions for a single input variable can easily be extended to the multi-input case. There are two ways to generalize the one-dimensional wavelet transform to two dimensions, known as the standard and nonstandard approaches [ 19,201. Standard decomposition of an image is appealing because it simply requires performing one-dimensional transforms on all rows, and then on all columns. The resulting values are all detail coefficients, except for a single overall average coefficient. The second type of the twodimensional wavelet transform, i.e. the nonstandard decomposition, alternates between operations on rows and columns. The choice of an approach depends on the application at hand. An example of an application of the standard two-dimensional Discrete Wavelet Transform to denoising of secondary ion mass spectroscopy (SIMS) images [ 211 gives evidence of the superiority of wavelets compared with most traditional linear smoothing methods, particularly in cases of
a high spatial variability of the original data [ 211. As reported, in all wavelet reconstructions the noise is efficiently suppressed, while sharp edges and corners in the original images are well preserved after the processing. 9.3. Data set compression Wavelet transforms can also be applied to compress a set of signals, i.e. to find the joint (common) best-basis, allowing their uniform representations in the wavelet domain. Although the well-known Principal Component Analysis (PCA) is the most efficient method of data compression, the constructed PCs have a global character (they are linear combinations of the original variables), whereas data compression in the wavelet domain, although less efficient, furnishes local features for further data processing. This can be particularly useful in solving calibration and pattern recognition problems. An effective approach to data compression was introduced by Wickerhauser [ 8 1. It aims to estab-
461
trends in analytical chemistry, vol. 16, no. 8, 1997
lish the joint best-basis based on the variance tree, elements of which are constructed in the following way: all N signals from the data set are decomposed and represented as the N sets of wavelet coefficients in the wavelet packet bases. Then the coefficients of the N signal trees are summed into two accumulator trees: the tree of means and the tree of squares. The tree of squares minus the squared tree of means leads to the binary tree of variance which can be searched for the best-basis, thereby maximizing the transform coding gain. Of course, data compression and denoising can be followed by selection of features (univariate, stepwise, or multivariate), or alternatively feature selection can be inherently built into the wavelet decomposition scheme. The latter approach, proposed by Walczak and Massart [ 22 1, is based on the WPT of the ‘relevant spectrum’, its definition being problem dependent. If the joint best-basis ought to preserve data variance, then the ‘relevant spectrum’ is equivalent to the ‘variance spectrum’, calculated for the data set of interest. In pattern recognition problems, the ‘relevant spectrum’ can be defined as the ‘Fisher spectrum’, representing the ratio of the between-groups to the withingroups variances, whereas for calibration problems it can be constructed as the ‘correlation spectrum’. In this approach the joint compressed best-basis is looked for in the ‘relevant spectrum’ only, and then all signals are decomposed into the best-basis and compressed. A small number of features containing relevant information for the problem at hand can significantly speed up and improve their further processing. Data set compression in the wavelet domain can be of great interest to speed up PCA of the large data sets [ 8,221. PCA already has a well established position in analytical chemistry as a visualization and reduction technique of multivariate data sets. Presentation of multivariate data into a few-dimensional PC space, describing the majority of the data variance, makes it possible to find the natural data patterns, to identify data outliers, to establish similarities of objects and variables, and to estimate the importance of the original variables. Calculation of the eigenvalue structure of matrices of the size (m X n), where min( m,n) exceeds 103, is terribly time consuming (if not impossible with personal computers). In such cases fast wavelet decomposition, leading to sparse representation of the data, and followed by data compression, can be performed as a preliminary step. PCA of the compressed data is then incomparably faster.
9.4. Other applications Modelling of multivariate data sets (for instance, calibration and classification of NIR data) can be significantly obscured not only by noisy features (which are usually very small), but mainly by irrelevant features. Elimination of both irrelevant and noisy features can be accomplished in the wavelet domain [ 23 1. This approach starts with preprocessing of the block of independent variables (signals), and namely with calculation of the joint best-basis for the set of signals and with identification of the significant and insignificant (noisy) coefficients. The term ‘significant coefficients’ means ‘coefficients important for reconstruction of the set of signals’, but it does not of course mean ‘coefficients relevant for data modelling’. In order to distinguish relevant from irrelevant coefficients in a set of significant coefficients, one can take the stability of the model regression coefficients, associated with noisy coefficients, as the basis. Stability of regression coefficients can be estimated, using the leaveone-out cross-validation procedure of modelling. Based on the stability of the regression coefficients associated with noisy wavelet coefficients, the threshold value can be calculated and used to eliminate all features with lower stability than the threshold found. Elimination of both noisy and irrelevant features can drastically increase the model’s predictive ability and its stability. Other applications that we would like to mention are: standardization of NIR spectra in the wavelet domain [ 241, removal of baseline effects, and multiscale regression [ 25 1, wavelet transforms for data preprocessing in pattern recognition problems [ 26,271, and wavelet transform for evaluation of peak intensities in flow-injection analysis [ 28 1. The local properties of the wavelet basis functions seem promising for quantum chemistry also [ 2931]. We hope that we have whetted your appetite sufficiently for you to further explore the fascinating field of wavelets, and we encourage you to make your own reconnaissance of the relevant literature via Internet (e.g., [ 321).
References [ 1 ] WaveLab, available from: ford.edu l-wavelab I [ 21 I. Daubechies, Orthonormal
http: //playfair.stanbases of compactly
462
supported wavelets, Comm. Pure Appl. Math. 41 (1988) 909-996. [ 3 ] Y. Meyer, Ondelettes et operateurs, Tome 1, Ondelettes, Herrmann, Paris, 1990. [ 41 S. Mallat, A theory for multiresolution signal decomposition, IEEE Trans. Pattern Anal. Machine Intell. 11 ( 1989) 674-693. [ 5 ] G. Strang, T. Nguyen, Wavelets and Filter Banks, Wellesley-Cambridge Press, Wellesley, MA, 1996. [ 61 J.B.J. Fourier, Theorie Analytique de la Chaleur in Oeuvres de Fourier, Tome 1, G. Darboux, Editor, Gauthiers-Villars, Paris, 1888. [ 7 ] D. Gabor, Theory of communication, J. IEE 96 (1946) 429-457. [ 8 ] M.V. Wickerhauser, Adapted Wavelet Analysis from Theory to Software, A.K. Peters, 1994. [ 91 M.A. Cody, The wavelet packet transform, Dr. Dobb’s J. 17 (1994) 16-28. [ lo] R.R. Coifman, M.V. Wickerhauser, Entropybased algorithms for best basis selection, IEEE Trans. Inform. Theory 38 (1992) 713-719. [ 111 S.S. Chen, D.L. Donoho, M.A. Saunders, Atomic Decomposition by Basis Pursuit, available from: http://[email protected] [ 121 B. Walczak, D.L. Massart, Noise suppression and signal compression using wavelet packet transform, Chemometr. Intell. Lab. Systems 36 (1997) 81-94. ]13 C.R. Mittermayr, S.G. Nikolov, H. Hutter, M. Grosserbauer, Wavelet denoising of Gaussian peaks: a comparative study, Chemometr. Intellig. Lab. Systems 34 ( 1996) 187-202. F.T. Chau, T.M. Shih, J.B. Gao, C.K. Chan, Appli114 cation of the fast wavelet transform method to compress UV-visible spectra, Appl. Spectrosc. 50 (1996) 339-348. D.L. Donoho, Wavelet shrinkage and W.V.D.: a 115 lo-minute tour, in: Progress in Wavelet Analysis and Applications, Y. Mayer and S. Roques, Editors, Editions Frontiers, France, 1993, pp. 109128. [ 16 1 I.M. Johnstone, B.V. Silverman, Wavelet threshold estimators for data with correlated noise, available from: http://playfair.stanford.edu [ 171 B.R. Bakshi, G. Stephanopoulos, Reasoning in time: modeling, analysis, and pattern recognition of temporal process trends, Adv. Chem. Eng. 22 ( 1995 ) 485-548. [ 181 B.R. Bakshi, G. Stephanopoulos, Representation of process trends. III. Multiscale extraction of trends from process data, Computers Chem. Eng. 18 ( 1994) 267-302. G. Beylkin, R. Coifman, V. Rokhlin, Fast wavelet 119 transforms and numerical algorithms, part 1, Comm. Pure Appl. Math. 44 (1991) 141-183. 120 R. DeVore, B. Jawerth, B. Lucier, Image compres-
trends in analytical chemistry, vol. 16, no. 8, 1997
sion through wavelet transform coding, IEEE Trans. Inf. Theory 38 (1992) 719-746. [ 211 S.G. Nikolov, H. Hutter, M. Grasserbauer, Denoising of SIMS images via wavelet shrinkage, Chemometr. Intell. Lab. Systems 34 (1996) 263-273. [ 221 B. Walczak, D.L. Massart, Wavelet Packet Transform applied to a set of signals: a new approach to the best-basis selection, Chemometr. Intell. Lab. Systems (in press). [ 23 ] D. Jouan-Rimbaud, B. Walczak, R.J. Poppei, D.L. Massart, Application of wavelet transform to extract the relevant component from spectral data for multivariate calibration, Anal. Chem. in press. [ 241 B. Walczak, E. Bouvresse, D.L. Massart, Standardization of near-infrared spectra in the wavelet domain, Chemometr. Intell. Lab. Systems 36 (1997) 41-51. [ 25 ] B.K. Alsberg, A.M. Woodward, D.B. Kell, An introduction to wavelet transform for chemometricians, Chemometr. Intell. Lab. Systems (in press). [ 261 M. Bos, J.A.M. Vrielink, The wavelet transform for preprocessing IR spectra in the identification of mono- and di-substituted benzenes, Chemometr. Intell. Lab. Systems 23 ( 1994) 115-122. [ 271 B. Walczak, B. van den Bogaert, D.L. Massart, Application of wavelet packet transform in pattern recognition of NIR data, Anal. Chem. 86 ( 1996) 1742-1747. [ 28 ] M. Bos, E. Hoogendam, Wavelet transform for the evaluation of peak intensities in flow-injection analysis, Anal. Chim. Acta 267 ( 1992) 73-80. ] S. Wei, M.Y. Chou, Wavelets in self-consistent ~29 electronic structure calculations, Phys. Rev. Lett. 76 (1996) 2650-2653. 130 ] J.P. Modisette, P. Nordlander, J.L. Kinsey, B.R. Johnson, Wavelet bases in eigenvalue problems in quantum mechanics, Chem. Phys. Lett. 250 (1996) 485-494. [ 311 J.L. Calais, Wavelets - something for quantum chemistry?, Int. J. Quantum Chem. 58 (1996) 541-548. [ 321 http://playfair.stanford.edu Doctor B. Walczak is a senior research worker of the Institute of Chemistry at the Silesian University, Katowice, Poland, engaged in the development of chemometrics and its implementation in different branches of analytical chemistry. She took her MSc and PhD degrees from the Silesian University and got wide research experience in several renowned academic centers in Europe (Hungarian Academy of Science, University of Orleans, Technical University of Graz and Vrve Universiteit Brussel). She has authored and co-authored over 50 papers and over 70 conference papers.
trends in analytical chemistry, vol. 16, no. 8, 1997
Professor D. L. Massart teaches analytical chemistry at the Vrije Universiteit Brussel. His main interest is in chemometrics. He is the first author of many articles and two books on the subject (the latest one being D.L. Massart,
463
B.G.M. Vandeginste, S.N. Deming, Y. Michotte, L. Kaufman, Chemometrics; A Textbook, Elsevier, Amsterdam, 1988). He also is editor-in-chief of Chemometrics and Intelligent Laboratory Systems.
Combination of solid-phase extraction procedures with gas chromatographic hyphenated techniques for chlorophenol determination in drinking water I. Rodriguez, R. Cela* Departamento de Qui’mica Analitica, Nutricidn y Bromatologia, Facultad de Quimica, Universidad de Santiago de Compostela, E- 15706 Santiago de Compostela, Spain This article deals with the determination of chlorophenols in natural and drinking water samples. Sample concentration by means of solid-phase extraction (SPE) on different sorbents is considered. Analyte derivatisation - pre or post concentration -and subsequent determination using several different gas chromatography-hyphenated techniques (atomic emission spectrometry, Fourier transform infrared spectrometry, mass spectrometry and tandem mass spectrometry) are shown.
1. Introduction Chlorophenols are present as pollutants in the aquatic environment as a result of the degradation of pesticides and insecticides [ 11; also, they can be formed from non-chlorinated phenols during chlorination of water. The US Environmental Protection Agency (EPA) [ 2,3 ] has compiled a list of several phenol compounds ( 11 in method 604 and 21 in method 804 1 ), considered to be priority pollutants in the *Corresponding author. 0165-9936/97/$17.00 PllSOl65-9936(
aquatic medium; among them, chlorophenols are especially toxic and potentially carcinogenic. In 1982, the European Community issued another pollutant list [ 41 that included many polychlorophenols and established their maximum allowable concentration in drinking waters (0.5 pg/l). Although high performance liquid chromatography (HPLC ) [ 5,6 ] and capillary electrophoresis (CE) [ 7-9 ] appear as general trends [ 10 ] to overcome the derivatisation of phenols, when dealing with drinking water samples, gas chromatography is most often employed on account of its high sensitivity and resolving power. However, the high polarity of free phenols hinders their correct chromatographic resolution because they produce broad, tailed peaks; this shortcoming can be circumvented by derivatising free phenols to less polar compounds with better chromatographic properties [ 1 l-141. On the other hand, the need to determine phenols at concentrations below 0.5 pg/l entails preconcentrating samples prior to injection into the gas or liquid chromatograph. Solid-phase extraction (SPE) on cartridges and membranes is gradually superseding traditional liquid-liquid extraction procedures for this purpose [ 10,l 1,15 1. Historically, C 18-type sorbents were first applied for concentrating phenols in water samples [ 16,17 ] providing acceptable recoveries for most phenols, albeit at highly variable breakthrough volumes, depending on the polarity of the compounds concerned [ 18 1. Converting chlorophenols into their acetylated derivatives prior to preconcentration decreases polarity differences between them and Copyright
97)00060-5
0 1997 Elsevier Science B.V. All rights reserved.
trends in analytical chemistry, vol. 16, no. 8, 1997
Wavelets - something for analytical chemistry? B. Walczak*
D.L. Massart
as signal decomposition tions, whereas in order offered by wavelets, a preferred. For practical signal processing is the
ChemoAC, Pharmaceutical Institute, Vrije Unversiteit Brussel, Laarbeeklaan 103, B- 1090 Brussels, Belgium
2. Wavelets
Institute of Chemistry, Silesian University, 9 Szkolna Street, 40-006 Katowice, Poland
Wavelets have shown great applicability in many diverse fields of science, and are now becoming of interest in analytical chemistry. This paper is intended as a first reading, introducing fundamentals of wavelets and wavelet transforms, and some applications thereof, such as signal compression and denoising, image processing, data set compression and the modeling of multivariate data sets. 1. Introduction This paper is addressed to all those who prefer to get insight into an analytical system via the signal given in Fig. la, rather than that in Fig. lb. We will present a story of the small waves called wavelets, which have recently made a revolutionary breakthrough in signal and image analysis. Although they offer numerous advantages over traditional signal processing methods, their penetration into chemistry and its problems can be described as retarded and hesitant. The theory of wavelets can be viewed as a common framework for the techniques that have been developed in various fields independently. Different origins of this theory lead to different languages, which additionally vary when shifting from continuous to discrete-time signals analysis. This diversity is helpful when describing different aspects of wavelet analysis. For certain applications it is convenient to consider wavelet analysis
*Corresponding author. Fax: +48 (32) 599 978. 01659936/97/$17.00
HISO165-9936(97)00065-4
onto a set of basis functo explain the advantages multiresolution analysis is reasons, the language of most suitable.
as a basis
The instrumental signals that we usually deal with are discrete-time signals, presented as a finite sequence of numbers. For instance, let us consider the following sequence of 4 numbers: (2, 3, 1, 7) (Scheme 1). It can be thought of as a vector in a 4-dimensional space and uniquely presented as a linear combination of the vectors in its basis: ( 1, 0, 0,0),(0, 1, 0,0),(0,0, 1,O) and (0,0, 0, 1). This basis, known as the canonical basis, has a valuable property: its vectors are linearly independent (i.e. orthogonal) and as their lengths equal 1, they are also orthonormal. In matrix notation this is: f=Sb where S is a matrix containing basis vectors in columns and b is a vector representing a set of coefficients. The canonical basis is one of many possible bases. Signal f can uniquely be presented as a linear combination of another set of basis vectors (see Fig. 2): f = Snew bnew where S”“” is a matrix containing new basis vectors in columns and b”“” is a vector representing a new set of b coefficients. By changing basis vectors we change signal representation. In a new basis, the signal is represented by a set of the new b coefficients, bnew . If the basis is orthonormal, the set of b coefficients can simply be Copyright
0 1997 Elsevier Science B.V. All rights reserved.
452
trends in analytical chemistry, vol. 16, no. 8, 1997
b)
a)
Fig. 1. The NMR signal [ 1 ] (a) after denoising and ( b) before denoising. calculated as: b new = (cpw)T
f
If the original signal contains N data points, so does the signal in the new basis. No information is gained or lost. So why would we want to change the basis? The answer is simple. The canonical basis is not always the best one. To represent even a very common constant signal ( 1, 1, 1,l) in the canonical basis, we still need 4 basis vectors. Would it not be nice to have a basis that includes a constant vector? Would it not be nice to have a basis that provides an accurate vector reproduction with a few vectors only? Here wavelets enter our story. Wavelets can be introduced as a new family of bases with special properties. Presentation of a signal in the new basis is generally called transformation (or a transform). For instance, projection of a signal onto the sine and cosine waves is known as a Fourier Transform, whereas projection of a signal onto the wavelet basis is called a Wavelet Transform. In each case the aim of the transform is to calculate a new set of b coefficients. In the case of the Fourier Transform they are called Fourier coefficients, while in the case of the wavelet transform they are called wavelet coefficients. New presentation of a signal can make a number of things easier. The goal of any transform is to reveal certain characteristics that are not apparent in the original signal, thus facilitating both analysis and processing of the transformation. Here we ought to mention that wavelet transform
Scheme 1.
can operate in continuous time (on functions) and in discrete time (on vectors). For functions and infinite signals the wavelet basis is infinite. For finite length vectors with N components, there will be N basis vectors and N transform coefficients. In this case wavelets are ordinary vectors (columns of matrix, S). Wavelets can be considered a new family of basis functions (vectors).
3. What makes wavelets special? Although there are endless possible bases for function space, almost all of them are uninteresting. To benefit from signal representation in another domain, basis functions ought to have some special properties. So what is special about wavelets? As their name suggests, wavelets are small waves (Fig. 3 ). Wavelets differ from zero in a limited domain only, called the support. We therefore see them as functions that are well localized in the time domain. But this is not the end of the story. Wavelet support can vary, thus allowing a different degree of wavelet localization. Moreover, wavelets are orthogonal (or even orthonormal). To achieve orthogonality with the local support was believed impossible until Daubechies’ work [ 21. Construction of the wavelet basis functions is quite intricate. Wavelets are obtained from a single prototype function by dilations and contractions (scaling), as well as by shifts (translations) (see Fig. 4). Shifts of the localized function are needed to cover the whole signal range, whereas scaling allows a multiresolution analysis of signals. Because all of the basis functions are derived from one prototype, they are self-similar (i.e. they have a fractal structure). Wavelets are obtained from a single prototype function by scaling, as well as by translation.
trends in analytical chemistry
al
453
vol. 16, no. 8, 1997
DanaLl
-1
.i::ii:iiiiiiiiiiiii*i:rr:i: ..J:‘r.:.‘...... . .
[=WI
=~1~ :!!!lii~iiifii::::::::::::i’
iiIi(i.:::::i:“t;::I:::;:..: . . . ..r...‘:.;:‘..rr::::.:::
S
ii:: ii
ii
:i:t,:.._.:::;;;;i..::iiiii
::::iii::i:L!iiIriI..:r!..i.
.iiliiliIIIiIiiiii;i:iiiijtI: ::::.“....I::::::::!i:.... a;!$;!i!i;iI,:
::::::::‘!iiiiii!i:::i::;...
=
...‘..I.
.::::t;;:::....:,::i!.;,..:. .t:::i._:::‘:::..i;;;~..::.:
b
=
b
::
ii
.iiiil;‘!!“iii’Lii.iijiii;I
f
.
::
S-
b-
b-
Fig. 2. a: Signal f can uniquely be presented as a linear combination of the different sets of basis vectors. b: By changing basis vectors, we change signal representation. In each basis the same signal is represented by a different set of b coefficients.
4. Multiresolution
analysis
The basic idea of wavelet analysis is that of multiresolution. This concept of the multiresolu-
-
Fig. 3. Wavelets.
tion approximation of functions was introduced by Meyer and Mallat [ 3,4], and provides a powerful framework to understand wavelet decomposition. The general concept of multiresolution can be
454
trends in analytical chemistry, vol. 16, no. 8, 1997
translation
scaling
Fig. 4. Illustration of the idea of translation and scaling: (a) examples of translated waveforms; (b) examples of scaled waveforms.
explained by a simple example taken from optics. Let us consider using binoculars, or a microscope. In both cases we apply the same tactics: if we intend to contemplate an object as a whole, we have to adjust an optical device to a lower resolution (thus disregarding perception of details). If, on the other hand, we are interested in the details of an observed object, we have to adjust the same optical device to a higher resolution. Similarly, to obtain a global view of any signal, we must look at it with low resolution. In order to see increasingly more details, we have to increase the resolution. The simultaneous appearance of a signal on multiple scales is known as multiresolution. The goal of multiresolution is decomposition of a function space into a nested set of subspaces: v”c
v’c
v*c
... WC
intrinsic property of these spaces is that as j increases, the approximation of the whole function space improves. As j decreases, these spaces contain gradually less information, because some ‘variations’ or ‘details’ are peeled off. To quantify the difference of information content in the two consecutive approximation spaces, we need to construct a complementary subspace, called the wavelet subspace. Thus the wavelet space IV-’ is the orthogonal component of Vj-’ in Vj (Scheme 2). approximatlon space at level j-1 $-l
VJC ...
which represent its successive approximations. The symbol C shows that the subspace Vi-’ is contained in the subspace Vj. The most important
wavelet space at level j-1
Scheme 2.
455
trends in analytical chemistry, vol. 16, no. 8, 1997
The symbol @ stands for the orthogonal sum in the sense that any signal in Vj can be uniquely written as the sum of a signal in Vj-’ and an orthogonal signal in I@-‘. The final requirement for multiresolution concerns a basis for each space. The basis functions for the approximation spaces are known as scaling functions. The basis functions for the wavelet spaces are known as wavelets. The nice part of this story is that all basis functions of the above described spaces can be generated from a single function, called the scaling function a,. We begin with a subspace V”, which we assume has an orthonormal basis consisting of the translations of the ‘localized’ scaling function. If the same function is compressed by a factor of 2 and then translated along the signal, it creates the basis for the vector space V’. In an analogous way we can construct the basis for the wavelet subspaces (see Fig. 5). We assume that IV0 has an orthonormal basis consisting of the translations of the wavelet functions, whereas the same function compressed by a factor of 2 and then translated creates the basis for the space IV’. Since V’CV’, it is possible to express all basis functions in V” in terms of the basis functions in V' ,
al
bl
subspace
VU
I
I
I
II
A
I
I
I subspace
W”
or more generally, since Vj-’ C Vj, it is possible to express basis functions in Vj-’ in terms of basis functions in Vj. Similarly, since Wj-’ CVj, the basis functions of the wavelet space Wj-‘, i.e. wavelets (‘I’), can also be derived from the scaling function. In the case when each VJ space has a finite basis, we can use matrix notation to express the fact that the scaling functions and wavelets at the levelj- 1 can be constructed as a linear combination of the scaling functions of finer scaling functions at the level j. U-‘(x) = @(x)Lj ‘I’+1 (x) = &(x)Hj where matrices Lj and Hj are the matrices of constants. The columns of L J are the shifted versions of one another, as are the columns of Hj, and therefore one column characterizes each matrix. To construct these columns, we need to know the sequence of numbers (CO,cl, .... CN), known as the wavelet filter coefficients. Matrices Lj and Hj are the matrices needed for transformation of any signal from the space Vj to the subspaces Vj-’ and Wj-‘, Matrices Lj and Hj are the matrices needed for
subspace
V’
subspace
W’
Fig. 5. a: Haar bases for the approximation subspaces V” and V’. b: Haar bases for the detail subspaces V” and V’. Basis functions for a given subspace are the translations of each other, while bases for the two consecutive subspaces differ due to scaling.
456
trends in analytical chemistry, vol. 16, no. 8, 1997
transformation of any signal from the space Vj to the subspaces Vj-’ and Wj-l.
scales of resolution, we apply the same transform. This is recursion - the same transform at a new scale.
5. Magic numbers
6. Discrete wavelet transform There exist many sets (families) of wavelets. They differ due to the compactness of the wavelets’ supports and their smoothness. We limit ourselves to the families of orthogonal wavelets, the Daubechies family [ 21, being the most popular among them. A particular wavelet within each family is uniquely specified by the set of coefficients CO,cl, ...) CN-
Each wavelet is uniquely characterized by the set of coefficients co, cl, .... cN. These coefficients can be found in the existing libraries of the wavelet filter coefficients. For instance, the first member of the Daubechies family (the eldest and the simplest wavelet, known as the Haar wavelet) is characterized by the two coefficients COand cl. The next member [ 21 is characterized by four coefficients CO,cl, ~2, c3 etc. These coefficients are used to transform a signal from the original domain to the wavelet basis, i.e. to construct the transform matrix ST (Scheme 3). It is helpful to think of the coefficients {CO, cl, .... cN } as a filter [ 5 1. Depending on the pattern in which these coefficients are ordered, we call them the low-pass and the high-pass filter, respectively. The low-pass filter, constructed with the coefficients {CO, cl, .... cN}, acts as a smoothing filter (in a way like a moving average), whereas the highpass filter, with coefficients { cN, -cN_l, .... cl, -CO}, can be considered a difference filter. In the field of signal processing this pair of filters is known as the quadrature mirror filter pair (QMF) [ 5 1. The output of the low-pass filter is called approximation, whereas the output of the highpass filter is called detail. In order to calculate the approximation and detail of the signal on different
There is an efficient hierarchical algorithm to perform wavelet decomposition that was proposed by Mallat (also known as a pyramid algorithm) [4]. It can be applied to any discrete-time finite signal, its length being equal to the integer power of 2. In the fast wavelet transform algorithm, the sampled signal is passed through the low-pass and the high-pass filters (see Fig. 6). Outputs of both filters are downsampled by a factor of 2. The high-pass filtered data set comprises the wavelet transform coefficients (details) at that scale of transform. The low-pass filtered data set comprises the approximation coefficients at that level of scale. Approximation coefficients can now be used as the sampled data input for another pair of wavelet filters (identical with the first pair), generating another set of the detail and approximation coefficients at the next-lower level of the scale. This process can continue until the limit for the unit interval is reached. The lower the level of the scale, the less information is contained in the approximation of the signal (in other words, more ‘variations’ are removed from the signal). In fact, these variations, peeled off level by level, are represented by the detail coefficients. In other words, the difference of information between the two consecutive approximations is contained in the detail coefficients at the corresponding level. Finally, the signal can be represented in the wavelet domain as a set of coefficients describing the roughest approximation and all details. An important feature of this discrete algorithm is its relatively low complexity. Independent of the depth of the tree, the complexity is linear in the number of input samples, with a constant factor that depends on the length of the filter.
7. From waves to wavelets
Scheme 3.
The best way to explain the advantages offered by wavelets is to compare them with waves, i.e. with sine and cosine, which are the basis functions used in the Fourier Transform (FT) [ 6 1. These basis functions differ in their frequencies, and FT
457
trends in analytical chemistry, vol. 76, no. 8, 1997
Original
lerel2
signal
a2
leuel1
I
level 0
d’
d2
I
I
Fig. 6. Scheme of Mallat’s pyramid algorithm for Discrete Wavelet Decomposition.
intends to analyze the frequency contents of the signals. Each coefficient of the FT corresponds to a single frequency only and provides information about the contribution by this frequency to the sig-
bl
al f
nal. It means that the FT decomposes the signal into its frequency components, with no time localization (see Fig. 7). Due to the global character (i.e. support) of the
Itime
&
time -
t t t c-
time
I time *
Fig. 7. Time-frequency representation: (a) of an original signal in time domain; (b) for Fourier decomposition; (c) for windowed Fourier decomposition; (d) for wavelet decomposition.
458
trends in analytical chemistry, vol. 76, no. 8, 1997
sine and cosine basis functions, the FT is not used in the case of non-stationary signals. Discontinuity occurring within the time domain affects the weights of all basis functions and therefore it is spread out across the whole frequency domain. A straightforward solution is to multiply the signal by a window function, in order to delimit the signal in time. A windowed FT can be used to give information about signals simultaneously in the time and frequency domains [ 71. A windowed FT with a fixed window size decomposes the time-frequency space into sections of equal dimensions. Although the windowed Fourier analysis is an improvement over the Fourier analysis, it is still inconvenient since the fixed window size makes the representation inadequate for approximation of singularities and for abrupt changes in the signal, and can result in the generation of fictitious features in a smoothed signal. As signals can contain contributions with varying degrees of localization in both time and requires frequency, an efficient representation basis functions that are localized in both the time and frequency domains. While sines and cosines provide complete frequency localization, wavelets are well localized in both the time and frequency domains. Due to this unique property of the wavelets, the Wavelet Transform can provide a much better time resolution for the high-frequency signals because it alters scales as the frequency increases. The higher the frequency, the shorter is the time window that it analyzes. Specific local properties of the wavelets can be particularly useful when describing signals with sharp spikes or discontinuities. Wavelets have a zoom-in and zoom-out property.
original signal
Fig. 8. Scheme sition.
of the full Wavelet
Packet decompo-
J Fig. 9. Examples
of possible
orthonormal
bases.
8. Where does this story end? So far we have limited our presentation to a Discrete Wavelet Transform (DWT), applicable to discrete-time finite length signals. Its extension is known as the Wavelet Packet Transform [ 8-101, which usually leads to a more efficient signal compression and is being used increasingly. 8.1. Wavelet packet transform In the DWT the detail branches are not used for further decomposition. If both filtering operations are applied to the approximation and to the detail coefficients, then the resulting subspaces form the full binary tree (see Fig. 8). Representation of the original signal in the wavelet packet domain is overcomplete, i.e. there are many possible orthonormal bases. Namely, any disjoint cover of the original data set is an orthonormal basis (see Fig. 9). The whole set of orthonormal bases is called the ‘wavelet packet library’ or ‘wavelet packet dictionary’. Using different criteria, one can select the socalled best-basis (where the term ‘best’ is defined by a user). Usually we are interested in such a basis in which a signal has a sparse representation, i.e.
trends in analytical chemistry
vol. 16, no. 8, 1997
there are only few coefficients with high amplitudes, whereas the remaining ones approach zero. Coefficients with small amplitudes can, without any significant loss of information, be omitted and the signal can be efficiently compressed. In order to find the best-basis with the desirable distribution of coefficients Coifmann and Wickerhauser proposed using the entropy criterion [ lo]. 8.2. Overcomplete
dictionaries
of waveforms
Wavelets and Wavelets Packets are merely the best known examples of the currently available signal representation schemes (dictionaries of vectors, also called waveforms or atoms) [ 111. Most of them are overcomplete, containing waveforms which have representation in terms of other waveforms, leading to a nonunique signal decomposition. Such nonuniqueness offers a possibility of adaptation (the most efficient representation of a signal can be chosen), but finding the optimal representation is not a trivial problem at all. The Coifman and Wickerhauser approach to the best-basis selection [ lo] can be applied to the Wavelet Packet and to the Cosine Packet only, because - although these are overcomplete dictionaries - there are special subcollections of their elements, yielding orthogonal bases for signal representation. In the case of the other overcomplete dictionaries, one has to choose from among many representations the one which is most suited for the decomposed signal, using such techniques as the Method of Frames, the Matching Pursuit or the Basis Pursuit (their sophistication and computational complexity increase in the order listed above) [ 111.
9. Applications The versatility of the wavelet transforms means that there are few branches of scientific research that cannot make use of them. Anything that involves the analysis of a time series is a good candidate for the wavelet treatment. An insight into chemical systems or phenomena via measured instrumental signals can be significantly improved by signal denoising and extraction of its relevant features.
459
9.1. Signal compression
and denoising
Perhaps the greatest potential of the wavelets has been claimed for signal compression and denoising (i.e. [ 12-141). The primary objective of any data compression is to transform the data to a form that requires the smallest possible amount of storage space, while retaining all the relevant information. Efficient compression, storage, and recovery of historical data is essential for many engineering tasks, as well as for further data processing and manipulation. Sparse representation of a signal in the wavelet domain allows for its significant compression. All wavelet coefficients with very small amplitudes can be eliminated without any loss of the relevant information. Compression is inherently associated with signal denoising, because small coefficients are assumed to represent the noise component of the signal. Their elimination ( thresholding [ 15 1) from the noisy signals is equivalent to data denoising. The thresholded signal is then inverse transformed, yielding a much clearer signal in which peaks can be distinguished (see Fig. 10). Moreover, wavelet decomposition of a signal allows characterization of the nature of its noise. Comparing the distribution of the detail coefficients along the first level of signal decomposition (associated with high frequencies), we can distinguish between the homo- and heteroscedastic noise types. Comparing the distribution of the detail coefficients among the first few levels of wavelet decomposition allows identification of a correlated noise type [ 16 1. Depending on the type of noise identified, one can vary the thresholding policy in order to optimize the denoising procedure. The power of wavelet transforms in signal compression and denoising is associated with the fact that - unlike the FT, which utilizes just a single set of basis functions - wavelet transforms have an infinite set of possible basis functions, among them the one that fits the type of signal analyzed. So far we have covered off-line processing of data only, but it is possible to apply waveletbased approaches to on-line compression and denoising as well. This opens possibilities of wavelet applications for on-line process control and in the analysis of process trends [ 17,181.
460
trends in analytical chemistry, vol. 16, no. 8, 1997
b)
4 60
20(
500
500 time domain
1000
wavelet domain d)
c)
wavelet domain
time domain
Fig. 10. a: Original signal. b: The same signal in the wavelet domain. c: The same signal in the wavelet domain after thresholding. d: Reconstructed signal.
9.2. Wave/et image processing
The multiresolution analysis of functions for a single input variable can easily be extended to the multi-input case. There are two ways to generalize the one-dimensional wavelet transform to two dimensions, known as the standard and nonstandard approaches [ 19,201. Standard decomposition of an image is appealing because it simply requires performing one-dimensional transforms on all rows, and then on all columns. The resulting values are all detail coefficients, except for a single overall average coefficient. The second type of the twodimensional wavelet transform, i.e. the nonstandard decomposition, alternates between operations on rows and columns. The choice of an approach depends on the application at hand. An example of an application of the standard two-dimensional Discrete Wavelet Transform to denoising of secondary ion mass spectroscopy (SIMS) images [ 211 gives evidence of the superiority of wavelets compared with most traditional linear smoothing methods, particularly in cases of
a high spatial variability of the original data [ 211. As reported, in all wavelet reconstructions the noise is efficiently suppressed, while sharp edges and corners in the original images are well preserved after the processing. 9.3. Data set compression Wavelet transforms can also be applied to compress a set of signals, i.e. to find the joint (common) best-basis, allowing their uniform representations in the wavelet domain. Although the well-known Principal Component Analysis (PCA) is the most efficient method of data compression, the constructed PCs have a global character (they are linear combinations of the original variables), whereas data compression in the wavelet domain, although less efficient, furnishes local features for further data processing. This can be particularly useful in solving calibration and pattern recognition problems. An effective approach to data compression was introduced by Wickerhauser [ 8 1. It aims to estab-
461
trends in analytical chemistry, vol. 16, no. 8, 1997
lish the joint best-basis based on the variance tree, elements of which are constructed in the following way: all N signals from the data set are decomposed and represented as the N sets of wavelet coefficients in the wavelet packet bases. Then the coefficients of the N signal trees are summed into two accumulator trees: the tree of means and the tree of squares. The tree of squares minus the squared tree of means leads to the binary tree of variance which can be searched for the best-basis, thereby maximizing the transform coding gain. Of course, data compression and denoising can be followed by selection of features (univariate, stepwise, or multivariate), or alternatively feature selection can be inherently built into the wavelet decomposition scheme. The latter approach, proposed by Walczak and Massart [ 22 1, is based on the WPT of the ‘relevant spectrum’, its definition being problem dependent. If the joint best-basis ought to preserve data variance, then the ‘relevant spectrum’ is equivalent to the ‘variance spectrum’, calculated for the data set of interest. In pattern recognition problems, the ‘relevant spectrum’ can be defined as the ‘Fisher spectrum’, representing the ratio of the between-groups to the withingroups variances, whereas for calibration problems it can be constructed as the ‘correlation spectrum’. In this approach the joint compressed best-basis is looked for in the ‘relevant spectrum’ only, and then all signals are decomposed into the best-basis and compressed. A small number of features containing relevant information for the problem at hand can significantly speed up and improve their further processing. Data set compression in the wavelet domain can be of great interest to speed up PCA of the large data sets [ 8,221. PCA already has a well established position in analytical chemistry as a visualization and reduction technique of multivariate data sets. Presentation of multivariate data into a few-dimensional PC space, describing the majority of the data variance, makes it possible to find the natural data patterns, to identify data outliers, to establish similarities of objects and variables, and to estimate the importance of the original variables. Calculation of the eigenvalue structure of matrices of the size (m X n), where min( m,n) exceeds 103, is terribly time consuming (if not impossible with personal computers). In such cases fast wavelet decomposition, leading to sparse representation of the data, and followed by data compression, can be performed as a preliminary step. PCA of the compressed data is then incomparably faster.
9.4. Other applications Modelling of multivariate data sets (for instance, calibration and classification of NIR data) can be significantly obscured not only by noisy features (which are usually very small), but mainly by irrelevant features. Elimination of both irrelevant and noisy features can be accomplished in the wavelet domain [ 23 1. This approach starts with preprocessing of the block of independent variables (signals), and namely with calculation of the joint best-basis for the set of signals and with identification of the significant and insignificant (noisy) coefficients. The term ‘significant coefficients’ means ‘coefficients important for reconstruction of the set of signals’, but it does not of course mean ‘coefficients relevant for data modelling’. In order to distinguish relevant from irrelevant coefficients in a set of significant coefficients, one can take the stability of the model regression coefficients, associated with noisy coefficients, as the basis. Stability of regression coefficients can be estimated, using the leaveone-out cross-validation procedure of modelling. Based on the stability of the regression coefficients associated with noisy wavelet coefficients, the threshold value can be calculated and used to eliminate all features with lower stability than the threshold found. Elimination of both noisy and irrelevant features can drastically increase the model’s predictive ability and its stability. Other applications that we would like to mention are: standardization of NIR spectra in the wavelet domain [ 241, removal of baseline effects, and multiscale regression [ 25 1, wavelet transforms for data preprocessing in pattern recognition problems [ 26,271, and wavelet transform for evaluation of peak intensities in flow-injection analysis [ 28 1. The local properties of the wavelet basis functions seem promising for quantum chemistry also [ 2931]. We hope that we have whetted your appetite sufficiently for you to further explore the fascinating field of wavelets, and we encourage you to make your own reconnaissance of the relevant literature via Internet (e.g., [ 321).
References [ 1 ] WaveLab, available from: ford.edu l-wavelab I [ 21 I. Daubechies, Orthonormal
http: //playfair.stanbases of compactly
462
supported wavelets, Comm. Pure Appl. Math. 41 (1988) 909-996. [ 3 ] Y. Meyer, Ondelettes et operateurs, Tome 1, Ondelettes, Herrmann, Paris, 1990. [ 41 S. Mallat, A theory for multiresolution signal decomposition, IEEE Trans. Pattern Anal. Machine Intell. 11 ( 1989) 674-693. [ 5 ] G. Strang, T. Nguyen, Wavelets and Filter Banks, Wellesley-Cambridge Press, Wellesley, MA, 1996. [ 61 J.B.J. Fourier, Theorie Analytique de la Chaleur in Oeuvres de Fourier, Tome 1, G. Darboux, Editor, Gauthiers-Villars, Paris, 1888. [ 7 ] D. Gabor, Theory of communication, J. IEE 96 (1946) 429-457. [ 8 ] M.V. Wickerhauser, Adapted Wavelet Analysis from Theory to Software, A.K. Peters, 1994. [ 91 M.A. Cody, The wavelet packet transform, Dr. Dobb’s J. 17 (1994) 16-28. [ lo] R.R. Coifman, M.V. Wickerhauser, Entropybased algorithms for best basis selection, IEEE Trans. Inform. Theory 38 (1992) 713-719. [ 111 S.S. Chen, D.L. Donoho, M.A. Saunders, Atomic Decomposition by Basis Pursuit, available from: http://[email protected] [ 121 B. Walczak, D.L. Massart, Noise suppression and signal compression using wavelet packet transform, Chemometr. Intell. Lab. Systems 36 (1997) 81-94. ]13 C.R. Mittermayr, S.G. Nikolov, H. Hutter, M. Grosserbauer, Wavelet denoising of Gaussian peaks: a comparative study, Chemometr. Intellig. Lab. Systems 34 ( 1996) 187-202. F.T. Chau, T.M. Shih, J.B. Gao, C.K. Chan, Appli114 cation of the fast wavelet transform method to compress UV-visible spectra, Appl. Spectrosc. 50 (1996) 339-348. D.L. Donoho, Wavelet shrinkage and W.V.D.: a 115 lo-minute tour, in: Progress in Wavelet Analysis and Applications, Y. Mayer and S. Roques, Editors, Editions Frontiers, France, 1993, pp. 109128. [ 16 1 I.M. Johnstone, B.V. Silverman, Wavelet threshold estimators for data with correlated noise, available from: http://playfair.stanford.edu [ 171 B.R. Bakshi, G. Stephanopoulos, Reasoning in time: modeling, analysis, and pattern recognition of temporal process trends, Adv. Chem. Eng. 22 ( 1995 ) 485-548. [ 181 B.R. Bakshi, G. Stephanopoulos, Representation of process trends. III. Multiscale extraction of trends from process data, Computers Chem. Eng. 18 ( 1994) 267-302. G. Beylkin, R. Coifman, V. Rokhlin, Fast wavelet 119 transforms and numerical algorithms, part 1, Comm. Pure Appl. Math. 44 (1991) 141-183. 120 R. DeVore, B. Jawerth, B. Lucier, Image compres-
trends in analytical chemistry, vol. 16, no. 8, 1997
sion through wavelet transform coding, IEEE Trans. Inf. Theory 38 (1992) 719-746. [ 211 S.G. Nikolov, H. Hutter, M. Grasserbauer, Denoising of SIMS images via wavelet shrinkage, Chemometr. Intell. Lab. Systems 34 (1996) 263-273. [ 221 B. Walczak, D.L. Massart, Wavelet Packet Transform applied to a set of signals: a new approach to the best-basis selection, Chemometr. Intell. Lab. Systems (in press). [ 23 ] D. Jouan-Rimbaud, B. Walczak, R.J. Poppei, D.L. Massart, Application of wavelet transform to extract the relevant component from spectral data for multivariate calibration, Anal. Chem. in press. [ 241 B. Walczak, E. Bouvresse, D.L. Massart, Standardization of near-infrared spectra in the wavelet domain, Chemometr. Intell. Lab. Systems 36 (1997) 41-51. [ 25 ] B.K. Alsberg, A.M. Woodward, D.B. Kell, An introduction to wavelet transform for chemometricians, Chemometr. Intell. Lab. Systems (in press). [ 261 M. Bos, J.A.M. Vrielink, The wavelet transform for preprocessing IR spectra in the identification of mono- and di-substituted benzenes, Chemometr. Intell. Lab. Systems 23 ( 1994) 115-122. [ 271 B. Walczak, B. van den Bogaert, D.L. Massart, Application of wavelet packet transform in pattern recognition of NIR data, Anal. Chem. 86 ( 1996) 1742-1747. [ 28 ] M. Bos, E. Hoogendam, Wavelet transform for the evaluation of peak intensities in flow-injection analysis, Anal. Chim. Acta 267 ( 1992) 73-80. ] S. Wei, M.Y. Chou, Wavelets in self-consistent ~29 electronic structure calculations, Phys. Rev. Lett. 76 (1996) 2650-2653. 130 ] J.P. Modisette, P. Nordlander, J.L. Kinsey, B.R. Johnson, Wavelet bases in eigenvalue problems in quantum mechanics, Chem. Phys. Lett. 250 (1996) 485-494. [ 311 J.L. Calais, Wavelets - something for quantum chemistry?, Int. J. Quantum Chem. 58 (1996) 541-548. [ 321 http://playfair.stanford.edu Doctor B. Walczak is a senior research worker of the Institute of Chemistry at the Silesian University, Katowice, Poland, engaged in the development of chemometrics and its implementation in different branches of analytical chemistry. She took her MSc and PhD degrees from the Silesian University and got wide research experience in several renowned academic centers in Europe (Hungarian Academy of Science, University of Orleans, Technical University of Graz and Vrve Universiteit Brussel). She has authored and co-authored over 50 papers and over 70 conference papers.
trends in analytical chemistry, vol. 16, no. 8, 1997
Professor D. L. Massart teaches analytical chemistry at the Vrije Universiteit Brussel. His main interest is in chemometrics. He is the first author of many articles and two books on the subject (the latest one being D.L. Massart,
463
B.G.M. Vandeginste, S.N. Deming, Y. Michotte, L. Kaufman, Chemometrics; A Textbook, Elsevier, Amsterdam, 1988). He also is editor-in-chief of Chemometrics and Intelligent Laboratory Systems.
Combination of solid-phase extraction procedures with gas chromatographic hyphenated techniques for chlorophenol determination in drinking water I. Rodriguez, R. Cela* Departamento de Qui’mica Analitica, Nutricidn y Bromatologia, Facultad de Quimica, Universidad de Santiago de Compostela, E- 15706 Santiago de Compostela, Spain This article deals with the determination of chlorophenols in natural and drinking water samples. Sample concentration by means of solid-phase extraction (SPE) on different sorbents is considered. Analyte derivatisation - pre or post concentration -and subsequent determination using several different gas chromatography-hyphenated techniques (atomic emission spectrometry, Fourier transform infrared spectrometry, mass spectrometry and tandem mass spectrometry) are shown.
1. Introduction Chlorophenols are present as pollutants in the aquatic environment as a result of the degradation of pesticides and insecticides [ 11; also, they can be formed from non-chlorinated phenols during chlorination of water. The US Environmental Protection Agency (EPA) [ 2,3 ] has compiled a list of several phenol compounds ( 11 in method 604 and 21 in method 804 1 ), considered to be priority pollutants in the *Corresponding author. 0165-9936/97/$17.00 PllSOl65-9936(
aquatic medium; among them, chlorophenols are especially toxic and potentially carcinogenic. In 1982, the European Community issued another pollutant list [ 41 that included many polychlorophenols and established their maximum allowable concentration in drinking waters (0.5 pg/l). Although high performance liquid chromatography (HPLC ) [ 5,6 ] and capillary electrophoresis (CE) [ 7-9 ] appear as general trends [ 10 ] to overcome the derivatisation of phenols, when dealing with drinking water samples, gas chromatography is most often employed on account of its high sensitivity and resolving power. However, the high polarity of free phenols hinders their correct chromatographic resolution because they produce broad, tailed peaks; this shortcoming can be circumvented by derivatising free phenols to less polar compounds with better chromatographic properties [ 1 l-141. On the other hand, the need to determine phenols at concentrations below 0.5 pg/l entails preconcentrating samples prior to injection into the gas or liquid chromatograph. Solid-phase extraction (SPE) on cartridges and membranes is gradually superseding traditional liquid-liquid extraction procedures for this purpose [ 10,l 1,15 1. Historically, C 18-type sorbents were first applied for concentrating phenols in water samples [ 16,17 ] providing acceptable recoveries for most phenols, albeit at highly variable breakthrough volumes, depending on the polarity of the compounds concerned [ 18 1. Converting chlorophenols into their acetylated derivatives prior to preconcentration decreases polarity differences between them and Copyright
97)00060-5
0 1997 Elsevier Science B.V. All rights reserved.