- Email: [email protected]
Wavelet decomposition tree selection for palm and face authentication
Available online at www.sciencedirect.com
Pattern Recognition Letters 29 (2008) 343–353 www.elsevier.com/locate/patrec
Wavelet decomposition tree selection for palm and face authentication Loris Nanni *, Alessandra Lumini DEIS, IEIIT-CNR, Universita` di Bologna, Viale Risorgimento 2, 40136 Bologna, Italy Received 8 March 2007; received in revised form 3 September 2007 Available online 24 October 2007 Communicated by F.Y. Shih
Abstract In this work, we study the usefulness of multi-resolution analysis for the face and palm authentication problems. The images are decomposed into frequency subbands with different levels of decomposition using different wavelets. We adopt as features for the authentication problem, the wavelet coefficients extracted from some ‘‘selected’’ subbands of several wavelet families. We propose to use a multi-matcher where each matcher is trained using a single subband, the matchers are combined using the ‘‘Max Rule’’. The band selection is performed by running Sequential Forward Floating Selection (SFFS). Moreover, several linear subspace projection techniques have been tested and compared. Experiments carried out on several biometric datasets show that the application of Laplacian EigenMaps (LEM) on a little subset of wavelet subbands (chosen by SFFS) permits to obtain a low Equal Error Rate. 2007 Elsevier B.V. All rights reserved. Keywords: Biometric verification; Wavelet decomposition; Face authentication; Palm authentication; Floating search
1. Introduction Advancing technological systems, such as the internet and cellular phones, have increased the need of personal identification and have made the use of a password, a secret code, or a personal identification numbers (PINs) unsafe and not user-friendly since they can easily be forgotten, compromised, shared, or observed. Biometrics, as the science of measuring and compiling distinguishing physical or biological features about an individual, such as facial structure, fingerprints or iris, is a solution for the growing need of security. The main advantage of biometrics is that it bases recognition on an intrinsic aspect of a human being and requires the person to be physically present at the point of the authentication. Palmprint serves as a reliable human identifier because the print patterns are not duplicated in other people, even in monozygotic twins. Related work concerning palmprint *
Corresponding author. Fax: +39 0547 338890. E-mail addresses: [email protected] (L. Nanni), alumini@deis. unibo.it (A. Lumini). 0167-8655/$ - see front matter 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2007.10.010
recognition consists of several statistical based methods including eigenpalm (Lu et al., 2003), fisherpalms (Wu et al., 2003), Gabor filters (Kong et al., 2003), Fourier Transform (Li et al., 2003), and local texture energy (You et al., 2003). A structural approach is presented in (Wu et al., 2002), based of the extraction from the palm image of structural information, like principal lines and creases. Face recognition problem has become one of the most relevant research areas in pattern recognition, mainly due to its wide potential application areas, ranging from human computer interaction to authentication and surveillance. Most of the appearance-based face recognition methods perform some kind of subspace analysis in the image space to extract the relevant feature vectors. The most widely used subspace analysis tools are Principal Component Analysis (Duda et al., 2000), Linear Discriminant Analysis (Belhumeur et al., 1997), Laplacian EigenMaps (He et al., 2005) and a blind source separation technique, called Independent Component Analysis (Baek et al., 2002). Two of the most recent papers based on wavelet are (Ekenel and Sankur, 2005; Connie et al., 2005). In the first
344
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
a multi-matcher is trained for Face Recognition based on all the subbands obtained by Daubechies wavelet decomposition (except the horizontal, vertical, diagonal details of the first level of decomposition1); this approach gains a performance improvement with respect to the Independent Component Analysis (Baek et al., 2002) only in presence of strong illumination variations. The second work shows that retaining the low-frequency subband of the wavelet transform and then reducing this subband using a linear projection permits to improve the performance with respect to that obtained using the original image projected onto a lower dimensional space by a linear projection. The main contribution of the paper is a method for searching the most discriminative set of wavelet channels, and the proposal of a multi-matcher for performing biometric authentication based on a fusion technique. In this study, we use several wavelet families2 in order to extract multiple subband images. These subband images contain coarse approximations of the image as well as horizontal, vertical and diagonal details at various scales. We present a new method which uses multiple subbands for building a set of classifiers, and we extract Laplacian EigenMaps features from these subbands. Best subbands are chosen by running Sequential Forward Floating Selection (SFFS) (Pudil et al., 1994; Nanni and Lumini, 2007a). A different classifier is trained for each subband, finally we exploit these multiple feature spaces by fusing their information with the ‘‘Max Rule’’ (Kittler et al., 1998). Our results are interesting: we show that combining k subbands (k = 6) from different wavelet families it is possible to obtain a significant reduction of the Equal Error Rate (EER) with respect to that obtained by Laplacian EigenMaps using the grey values of the original images. 2. Overview of system architecture Starting from the idea that different subbands of different wavelet families bring different information, we study the effects of the combination among different subbands. We argue that a feature selection method (where each feature is a given subband) may be very useful due to the different characteristics of these subbands. Therefore, we have selected the features to be combined by running Sequential Forward Floating Selection (SFFS)3 (Pudil et al., 1994). In the SFFS method, features are selected by adding the features, which provides the highest incremental of the objective function, to existing subset. As reported in Section 6, the subbands selected by SFFS belong to different wavelet functions, hence our results confirm that the subbands obtained applying different wavelet functions to the image are complementary, and, their complementarity can be exploited by fusion rules (Kittler et al., 1998).
For the performance evaluation we adopt the Equal Error Rate (see Section 6). The Equal Error Rate represents the intrinsic error of the system, and it is usually considered a good indicator of the performance of an identification system (Maio et al., 2003). Moreover, several unsupervised linear subspace projection techniques (Principal Component Analysis, Independent Component Analysis, Laplacian EigenMaps) have been tested and compared. We use Laplacian EigenMaps (which in our tests outperforms all the other methods, see Table 2) to project each subband onto a lower 100-dimensional space and then we use this space to train a 1-Nearest Neighbor classifier4 (Duda et al., 2000) (1-NN). Finally, the classifiers trained on the selected subbands are combined by the ‘‘Max Rule’’ (MAX) (Kittler et al., 1998), the max rule selects as final score the maximum score of the pool of k classifiers. The Max Rule works well when at least one classifier works well for a given pattern (Kuncheva, 2005). Please note that the max rule selection is performed independently for each pattern. For example, if in a face image there is a strong variation in illumination conditions the most useful feature set is the horizontal details subband, while if there is not a strong variation in illumination conditions the most useful feature set is the low-pass information subband (see Section 6 for more details). In Fig. 1 our system is detailed. 3. Multi-resolution analysis The wavelet transform, as compared to the traditional Fourier analysis, has a better space-frequency localization. Thus, it is suited for analyzing images where most of the informative content is represented by components localized in space – such as edges and borders – and by information at different scales or resolutions, with large and small features. In this work several wavelet families5 have been used, trusting in their capability to localize different information in time and frequency. In this paper we have tested the following wavelet functions: • • • • •
Haar (Haar). Daubechies order 4 (Daub). Symmlet order 2 (Sym). Coiflets order 2 (Coif). Biorthogonal order for reconstruction 2 and for decomposition 2 (Bio). • Reverse Biorthogonal order for reconstruction 2 and for decomposition 2 (RBio). • ‘‘Discrete’’ Meyer (Mey). Multi-resolution analysis (Ekenel and Sankur, 2005; Daubechies, 1992; Nanni and Lumini, 2007c) of the images is performed by reiterating the wavelet decomposition an
1
The high frequency subbands. We use the dyadic wavelet decomposition. 3 Implemented as in PrTools 3.1.7 (ftp://ftp.ph.tn.tudelft.nl/pub/bob/ prtools/prtools3.1.7). 2
4
Based on the Euclidean distance. We use the function dwt2.m (Symmetric-padding (half-point) (Strang and Nguyen, 1996)) of the Matlab 7.0 Wavelet Toolbox. 5
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Image processing
Feature extraction and Selection
Feature transformation
Classification and Fusion First matcher
2D wavelet + SFFS
Face\Palm Image
345
Laplacian Eigenmaps
Image PreProcessing
Nearest Neighbor classifier
Max Rule
ith matcher Laplacian Eigenmaps
Nearest Neighbor classifier
kth matcher Fig. 1. System proposed.
Fig. 2. Decomposition at level 2 of a given image.
arbitrary number of times on the low frequency part. At the first level, the original image is decomposed in four subbands leading to: the scaling component containing global low-pass information, and three wavelet components corresponding, respectively, to the horizontal, vertical and diagonal details. In Fig. 2, an example (palm image) of two level wavelet decomposition is reported. It is interesting to note (see Section 6) that the low-pass information on the wavelet transform is the most useful feature set for our verification problem; but where a strong variation in illumination conditions is present the most useful feature set is the horizontal details subband, which is less sensitive to light changing. 4. Feature transform Several linear subspace projection techniques have been tested and compared; experimental results (reported in Table 2) show that application of Laplacian EigenMaps outperforms Principal Component Analysis, Orthogonal Neighbourhood Preserving Projections and Independent Component Analysis:
Principal Component Analysis (PCA)3 (Duda et al., 2000): We use PCA to project each subband onto a lower 100-dimensional space. Moreover, in order to de-correlate the features and reduce computational complexity, PCA is also applied to project the data into a subspace where the preserved variance is 1 before the application of the other linear subspace projection techniques (presented next). Laplacian EigenMaps (LEM)6 (He et al., 2005): We use LEM to project the subband onto a lower 100-dimensional space. Independent Component Analysis (ICA)7 (Baek et al., 2002): In this paper, the representation coefficients of the patterns are assumed to be statistically independent. Therefore, in this architecture, the lexicographically ordered images constitute the columns of the observation matrix. We use ICA to project the subband onto a lower 100dimensional space.
6
Matlab code shared by Mikhail Belkin and Partha Niyogi. We use the FastIca toolbox whose Matlab code is publicly available at: www.cis.hut.fi/projects/ica/fastica/. 7
346
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Orthogonal Neighbourhood Preserving Projections (ONPP) (Kokiopoulou and Saad, 2005): ONPP is used to project the subband onto a lower 100-dimensional space.
6. Experiments We test our verification method on two biometric characteristics (palm and face), in order to evaluate how the performance depends on the dataset:
5. Feature selection We use SFFS as a feature selection method, where each ‘‘feature’’ is a classifier trained using the features extracted from a given wavelet subband, to find the most useful subbands for the recognition. Therefore we select (one for all) the classifiers to be combined by running Sequential Forward Floating Selection (SFFS) with objective function the minimization of the Equal Error Rate. In SFFS starting from a set of features F and an empty subset S0, the best feature subset of size k is constructed by adding to the subset the single feature f (f 2 F, f 62 Sk1) that gives the best performance for the new subset, this step is named forward optimization. Then a backward optimization is performed selecting as the subset of k 1 elements of Sk that has the best performance (Nanni and Lumini, 2007b). The pseudo-code of SFFS (from Nanni and Lumini, 2007b) is the following: Initialize an empty subset E0 = 100; n = 0; while n < k To search the classifier Cj so that the Equal Error Rate obtained combining the classifiers that belong to Sn1 and Cj is minimum; Sn is constructed by adding to the subset Sn1 the classifier Cj; En = Equal Error Rate obtained combing by sum rule the classifiers that belong to Sn; n = n+1; while n > 2 To test each combination of (n 1) classifiers that belong to Sn, we denote Be the combination that obtains the lowest Equal Error Rate and we denote its Equal Error Rate with EBe; if EBe < En1 En1 = Be; n = n 1; else break end end end We run two different methods based on SFFS: INTRA; INTER. In INTRA the fitness of SFFS is the Equal Error Rate obtained separately on each dataset (the subbands are selected separately on each dataset), in INTER the fitness is the average EER on the three datasets (the subbands are selected one for all on the three datasets).
• FACES: the tests have been conducted on the ORL (Samaria and Harter, 1994) and Yale-B (Belhumeur et al., 1997) datasets, which are two of the most used benchmarks in this field. ORL: It consists of 400 different images related to 40 individuals. In order to emphasize the ‘‘importance’’ of the shape of the face, we enlarge the image canvas from 112 · 92 to 142 · 122, by adding new rows and columns obtained simply smoothing to black the last row/column (as in Lumini and Nanni, 2005). Yale-B: It contains 5760 single light source images of 10 subjects each seen under 576 viewing conditions (9 poses · 64 illumination conditions). For every subject in a particular pose, an image with ambient (background) illumination was also captured. We use only the frontal poses (108 images for each individual). • PALMPRINTS: The experiments utilize inkless hand images obtained from digital Camera. The database (PALM) here used contains 720 right-hand images, 10 samples from each user, for 72 users. The palm image is extracted from the hand using a technique similar to that used in (Li et al., 2004). For all the datasets, to minimize the possible misleading results caused by the training data, the results have been averaged over five experiments, all conducted using the same parameters. For each experiment we randomly resampled the learning and the test sets (containing, respectively, half of the patterns). Before the wavelet decomposition the images are resized to 128 · 128. For the performance evaluation we adopt the Equal Error Rate (EER). EER is the error rate where the frequency of fraudulent accesses (FMR) and the frequency of rejections of people who should be correctly verified (FNMR) assume the same value; it can be adopted as a unique measure for characterizing the security level of a biometric system (Maio et al., 2003). If the trend of FMR and FNMR are plotted as a function of a threshold t indicating the degree of tolerance of the system which is a quantity in inverse relation to the degree of security (t = 0 and 1 indicate the maximum and minimum security lever, respectively). The point of intersection of the two curves denotes the EER (see Fig. 3). The EER represents the intrinsic error of the system, and it is usually considered a good indicator of the performance of an identification system. In real applications the security level is usually fixed at a value higher than EER in order to limit the number of fraudulent accesses (Maio et al., 2003). The pre-processing stage is very important to normalize the images in order to smoothen the noise and lighting
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
347
Table 2 Comparison among the linear subspace projection techniques PCA,LEM, ONPP and ICA Classifier
PALM Method
ORL Method Fig. 3. We plot the curves FMR and FNMR.
YALE-B Method
Table 1 EER obtained adopting different pre-processing procedures Dataset
Methods NO CONNIE ADAPT
PALM
ORL
YALE-B
10 4 4.1
3 3 3.5
2.2 1.7 1.3
1-NN
LCOS
PCA ONPP LEM ICA
4 5.2 3.1 3.3
3.7 5 3.7 3.7
PCA ONPP LEM ICA
3 4.1 2.7 4.1
4.8 4.5 4.6 7.6
PCA ONPP LEM ICA
1.7 1.6 2 0.5
3.5 2.2 4.5 2
3 2.5 PALM ORL YALE-B
2
effect. Our results have been obtained adopting the following pre-processing methods:
1.5 1
• NO: we do not use any pre-processing procedure. • CONNIE: we use the method proposed in (Connie et al., 2005). • ADAPT: we perform contrast-limited adaptive histogram equalization8 on the results of CONNIE. The first test, reported in Table 1, is aimed to prove the advantage of adopting a pre-processing procedure, in these tests we report the EER (in percentage) obtained projecting the original image onto a lower 100-dimensional space by PCA and then using a 1-NN classifier. These results demonstrate that, especially in the YALE-B and PALM datasets, the pre-processing procedure is important,9 even if it is interesting to note that in the YALE-B dataset ADAPT outperforms CONNIE, but in the ORL dataset CONNIE outperforms ADAPT. Since CONNIE proved the best preprocessing method in all the following experiments we adopt this pre-processing procedure. In Table 2 we compare, using the EER in percentage, the linear subspace projection techniques detailed in Section 4 (PCA, LEM, ICA and ONPP) using a 1-NN classifiers (1-NN) or the cosine distance (LCOS) for calculating the scores. 8
Implemented as in adapthisteq.m of the Matlab 7.0 Image Processing Toolbox. 9 In these dataset the performance of CONNIE was statistically superior, considering the 95% confidence intervals (computed using t distribution (Spiegel, 1992)), to NO.
0.5 1
2
3
4
5
6
7
8
9
10
Fig. 4. EER as a function of the number k of subbands selected by SFFS on PALM, ORL and YALE-B datasets.
Since (see Table 2) in our test, LEM outperforms all the other methods, we use LEM to project each subband in a reduced space.10 In Fig. 4, we plot the EER obtained combining the results gained by the k best subbands selected (the selected subbands are reported in Table 5) by SFFS (SFFS is run separately on each dataset). By the analysis of this plot we argue that k = 6 is a good compromise between performance and complexity, even if the fusion of two or three subbands reach good results if compared with a single subband. In Table 3, we report some experiments where we have reduced the dimension using a subspace where the dimension is: (number of pixel/100); (number of pixel/50); (number of pixel/25). In this way the dimension of the projection subspace is related to the size of the subband. In our opinion each different subband bring a different ‘‘quantity’’ of information and are differently affected by the noise, hence the different of the projection should be 10 We have run LEM, PCA, ONPP and ICA also in each subband, also in these tests LEM outperforms others Feature Transforms.
348
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Table 3 We report the results of the method INTER
Table 5 Subbands selected by SFFS on the three datasets (INTRA method) and performing a global optimization (INTER method)
Dataset PALM Method (Number of pixel/100) (Number of pixel/50) (Number of pixel/25) Dimension fixed to 100
ORL
PALM 2.83 2.87 2.81 2.68
2.14 2.28 2.22 1.99
1.2 1.36 1.2 1.19
different in each subbands. The main problem is that it is difficult to find an automatic method to find the dimension of the projected space. In our tests a value of 100 permits to obtain good results in all the dataset (we have three dataset and two biometric characteristics). The tests reported in Table 4 are aimed to compare the system verification performance gained by the following systems: • The related work proposed in (Ekenel and Sankur, 2005). • NOWAVELET, a system obtained training a 1-NN as classifier projecting the original image onto a lower 100-dimensional space by LEM. • BEST, the EER obtained by the best subband (best matcher, that is k = 1 and no fusion). • INTRA, the complete system described in Section 2 where the features are obtained selecting the best k = 6 subbands separately on each dataset. In INTRA the fitness of SFFS is the EER obtained in a given dataset. • INTER as the previous one, but selecting the six best subbands one for all on the three datasets. In INTER the fitness of SFFS is the average EER on the three datasets. The selected subbands are reported in Table 5, with GLOBAL we denote the subbands selected by the INTER method (which are the same for the three datasets), PALM, ORL and YALE-B denote the subbands selected by the INTRA method for the three datasets consider, respectively. In Table 5, we show which subbands are selected by SFFS. In each cell there are three values, the first indicate the wavelet family, the second is the subband, the third is
Table 4 Performance obtained for the verification problem adopting different methods (the * indicates the presence of feature normalization mean 0 and standard deviation 1) Dataset
Method Ekenel and Sankur (2005) Ekenel and Sankur (2005)* NOWAVELET BEST INTRA INTER
Dataset
YALE-B
PALM
ORL
YALE-B
10.3 3.7 3.1 2.9 2.68 2.68
14 3.8 2.7 2.27 1.54 1.99
1.62 3.6 2 1.73 0.87 1.19
ORL
YALE-B
GLOBAL
Selected subbands 1 B-I-3 M-I-2 M-II-3 M-I-2 2 M-I-3 H-II-1 D-I-2 M-II-3 3 H-I-2 R-III-3 H-II-3 R-III-3 4 R-III-3 R-II-3 H-II-2 R-II-3 5 H-IV-2 H-III-3 S-II-2 S-I-2 6 H-II-3 M-I-1 H-III-3 H-II-2 H Haar D Daubechies order 4 S Symmlet order 2 C Coiflets order 2 B Biorthogonal order for reconstruction 2 and for decomposition 2 R Reverse Biorthogonal order for reconstruction 2 and for decomposition 2 M ‘‘Discrete’’ Meyer I II III IV
Low-pass information Horizontal details Vertical details Diagonal details
the level of decomposition. In Fig. 5, we show the grey level image and the best four wavelet subbands extracted from that image. The experimental results show that: • Our approach (INTER) significantly improves the performance of the base method (NOWAVELET and BEST), the EER is very low not only when we select the best subbands on each dataset, but also when the optimization is performed on all the three datasets (same subbands used for all the three datasets). • It is very interesting to note that the subbands selected by SFFS belong to different wavelet functions, this fact provides evidence that the multi-resolution decompositions obtained by different wavelet functions are complementary, and, their complementarity can be exploited by fusion rules. • From the analysis of Table 4, we can observe that the low-pass information on the wavelet transform is, as expected, the most useful feature set for our verification problem; it is interesting to note that in the YALE-B dataset, where a strong variation in illumination conditions is present, the first feature vector selected is a horizontal details subband, which is less sensitive to light changing. In order to evaluate the benefit of the our method we perform a further experiment. The DET curve (Martin, 1997) is a two-dimensional measure of classification performance that plots the probability of false acceptation against the rate of false rejection. In Figs. 6–8, we plot the DET-curve on the tested dataset. The black line is the DET-curve obtained by INTRA,
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
349
Fig. 5. Grey level image and the best four wavelet subbands.
Fig. 6. DET-curve on the ORL dataset.
Fig. 8. DET-curve on the PALM dataset.
Table 6 EER obtained using only the best subband for each dataset Classifier (1-NN)
Method PCA ONPP LEM ICA
PALM
ORL
YALE-B
4 5 2.9 3.1
3.1 3.5 2.27 3.5
1.8 1.8 1.73 0.9
the red11 line is the DET-curve obtained by NOWAVELET. Also in these tests it is clear that INTRA outperforms NOWAVELET. As further experiment, we have ran LEM, PCA, ONPP and ICA in each wavelet subband, also in these tests LEM outperforms others Feature Transforms. In Table 6, we report the performance (EER) obtained using only the best subband for each dataset.
Fig. 7. DET-curve on the YALE-B dataset.
11 For interpretation of color in Figs. 6–8, the reader is referred to the web version of this article.
350
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
As final tests we report two examples of how max rule works (see Fig. 9): • In the first example we match two faces where in one face there is a strong variation in illumination conditions. • In the second example we match two faces where in both the faces there is NOT a variation in illumination conditions. In the first example the max rule selects the score obtained by the horizontal details subband (which is less sensitive to light changing) while in the latter example the max rule selects the score obtained by the low-pass information subband. In Fig. 9 with SC1, we name the score obtained by the low-pass information subband, while with SC2 we name the score obtained by the horizontal details subband.
Fig. 9. Examples of how max rule works.
6.1. Experiments on Q-statistic
Table 7 Characteristics of the wavelets tested in this work Wavelet
Orthogonal
Symmetry
Haar Daubechies Symmlet Coiflets Biorthogonal Reverse Biorthogonal Discrete Meyer
Yes Yes Yes Yes No (it is biothogonal) No (it is biothogonal) Yes
No No Near Near Yes Yes Yes
As a further experiment we investigated the relationship (i.e. the error independence) between the matchers trained using the different wavelet subbands. The average Q-statistic among the matchers is 0.4. This value is low and it is well known in the literature that combining ‘‘independent’’ classifiers (Nanni and Lumini, 2005; Nanni, 2005; Nanni and Lumini, 2007c) permits to dramatically reduce the error rate obtained by a ‘‘stand-alone’’ classifier.
Fig. 10. Wavelets used in this work.
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Error independence is an important property in combining classifiers. The independence of classifiers is calculated using Yule’s Q-statistic (Kuncheva and Whitaker, 2003). For two classifier Di and Dk the Q-statistic is Qi;k ¼
ad bc ad þ bc
where a is the probability of both classifiers being correct, d the probability of both classifiers being incorrect, b the probability first classifier is correct and second is incorrect, c is the probability second classifier is correct and first is incorrect. For statistically independent classifiers, Qi,k =
351
0. Q varies between 1 and 1. Classifiers that tend to recognize the same patterns correctly will have Q > 0, and those which commit errors on different patterns will have Q < 0. To better motivate the different behavior of the wavelet we report: • Main mathematical characteristics (see Table 7). • The plot of the wavelet functions tested in this work (see Fig. 10). • Multi-resolution of two images with different illumination conditions (see Figs. 11 and 12).
Fig. 11. Multi-resolution analysis of a face without variation in the illumination conditions.
352
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Fig. 12. Multi-resolution analysis of a face with variation in the illumination conditions.
The two most complementary wavelet selected by SFFS in INTER are Discrete Meyer and Reverse Biorthogonal, it is interesting to note that Discrete Meyer is orthogonal while Reverse Biorthogonal is not orthogonal. Probably this mathematical difference determines the complementarities of these two wavelets. From Fig. 10 it is clear that the pool of tested wavelet have different wavelet functions and this permit to obtain (slightly) different subband images. From the Figs. 11 and 12 some behaviors of the tested wavelet can be extracted:
• If in the images there are NOT illuminations problems the Haar’s horizontal details subband works better than the Meyer’s horizontal details subband (see Fig. 11), for this reason SFFS in the ORL dataset (illuminations variations are not present) selects the Haar’s horizontal details but not the Meyer’s horizontal details subband. • In Fig. 12, the previous conclusion is not more valid, the Meyer’s horizontal details subband works better probably this is due to the fact that the Meyer wavelet is less sensitive to the edge of the image.
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
• RBIO has a similar behaviour in both the images, probably for this reason the fusion among the Meyer subbands and the Reverse Biorthogonal subbands permits to obtain a robust system (please note that the first four subbands selected by SFFS in INTER belong to MEY or to RBIO). 6.1.1. Conclusions We investigated fusion of classifiers trained using different frequency subbands obtained by wavelet transform at different levels of decomposition. It is reported in the literature that classifier sets that enforce diversity far better than ones that do not. To enforce diversity we combine classifiers based on different feature extractions. The experimental results obtained on three different datasets are encouraging: we show that using a feature selection approach (SFFS) it is possible to choose few subbands to be exploited for combination and the EER gained by our multi-classifier is lower than that obtained by the best stand-alone matcher on the grey values of the original image. Our method can be easily combined with supervised learning (e.g. using the Fisher Transform to project the data) or to bio-inspired methods as (Tan and Chen, 2005). E.g. we can use our method to extract the features from given subwindows of the image and than the different contributions made by different parts of the image are emphasized (in a ‘‘weighted’’ fusion). As future work we want to study supervised methods (Gutschoven and Verlinde, 2000) to combine the scores of the pool of matchers. Acknowledgement This work has been supported by European Commission IST-2002-507634 Biosecure NoE projects. References Baek, K., Draper, B., Beveridge, R., She, K., 2002. PCA vs. ICA: A comparison on the FERET data set. In: Joint Conference on Information Sciences, Durham, NC, March 2002, pp. 824–827. Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J., 1997. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 711–720. Connie, T., Jin, A.T.B., Ong, M.G.K., Ling, D.N.C., 2005. An automated palmprint recognition system. Image and Vision Computing 23, 501–515. Daubechies, I., 1992. Ten Lectures on Wavelets. Capital City Press, Vermont. Duda, R.O., Hart, P.E., Stork, G., 2000. Pattern Classification, second ed. Wiley, New York. Ekenel, H.K., Sankur, B., 2005. Multiresolution face recognition. Image and Vision Computing 23, 469–477. Gutschoven, B., Verlinde, P., 2000. Multi-modal identity verification using support vector machines. In: Proceedings of the Third International
353
Conference on Information Fusion, Paris, France, 10–13 July 2000, pp. 3–8. He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H-J, 2005. Face Recognition using Laplacian faces. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (3), 328–340. Kittler, J., Hatef, M., Duin, R., Matas, J., 1998. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (3), 226–239. Kokiopoulou, E., Saad, Y., 2005. Orthogonal neighborhood preserving projections. In: IEEE International Conference on Data Mining, New Orleans, Lousiana, USA, 27–30 November, 2005. Kong, W.K., Zhang, D., Li, W., 2003. Palmprint feature extraction using 2-D Gabor filters. Pattern Recognition 36 (10), 2339–2347. Kuncheva, L.I., 2005. Diversity in multiple classifier systems. Information Fusion 6 (1), 3–4. Kuncheva, L.I., Whitaker, C.J., 2003. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51, 181–207. Li, W., Zhang, D., Xu, Z., 2003. Palmprint identification by Fourier Transform. International Journal of Pattern Recognition and Artificial Intelligence 16 (4), 417–432. Li, Q., Qiu, Z., Sun, D., Wu, J., 2004. Personal identification using Knuckleprint. Sinobiometrics, 680–689. Lu, G., Zhang, D., Wang, K., 2003. Palmprint recognition using eigenpalms features. Pattern Recognition Letters 24 (9–10), 1473–1477. Lumini, A., Nanni, L., 2005. Combining classifiers to obtain a reliable method for face recognition. Multimedia Cyberscape Journal 3 (3), 47– 53. Maio, D., Maltoni, D., Jain, A.K., Prabhakar, S., 2003. Handbook of Fingerprint Recognition. Springer, New York. Martin, A., et al., 1997. The DET curve in assessment of decision task performance. In: Proceedings of the EuroSpeech, pp. 1895–1898. Nanni, L., 2005. Comparison among feature extraction methods for HIV1 Protease Cleavage Site Prediction. Pattern Recognition, Pattern Recognition 39 (4), 711–713. Nanni, L., Lumini, A., 2005. Ensemble of parzen window classifiers for on-line signature verification. NeuroComputing 68 (6), 217–224. Nanni, L., Lumini, A., 2007a. A hybrid wavelet-based fingerprint matcher. Pattern Recognition 40 (11), 3146–3151. Nanni, L., Lumini, A., 2007b. RegionBoost learning for 2D + 3D based face recognition. Pattern Recognition Letters 28 (15), 2063– 2070. Nanni, L., Lumini, A., 2007c. A multi-expert approach for wavelet-based face detection. Pattern Recognition Letters 28 (12), 1541–1547. Pudil, P., Novovicova, J., Kittler, J., 1994. Floating search methods in feature selection. Pattern Recognition Letters 15 (11), 1119–1125. Samaria, F., Harter, A., 1994. Parameterisation of a stochastic model for human face identification. In: Proceedings of the Second IEEE Workshop on Applications of Computer Vision, Florida. http:// www.uk.research.att.com/facedatabase.html. Spiegel, M.R., 1992. Theory and Problems of Probability and Statistics. McGraw-Hill, New York, pp. 116–117. Strang, G., Nguyen, T., 1996. Wavelets and filter banks. Cambridge Press, Wellesley. Tan, K., Chen, S., 2005. Adaptively weighted sub-pattern PCA for face recognition. Neurocomputing 64, 505–511. Wu, X., Wang, K., Zhang, D., 2002. Fuzzy directional element energy feature (FDEEF) based palmprint identification. In: International Conference on Pattern Recognition, pp. 95–98. Wu, X., Zhang, D., Wang, K., 2003. Fisherpalms based palmprint recognition. Pattern Recognition Letters 24, 2829–2838. You, J., Li, W., Zhang, D., 2003. Hierarchical palmprint identification via multiple feature extraction. Pattern Recognition 35, 847–859.
Pattern Recognition Letters 29 (2008) 343–353 www.elsevier.com/locate/patrec
Wavelet decomposition tree selection for palm and face authentication Loris Nanni *, Alessandra Lumini DEIS, IEIIT-CNR, Universita` di Bologna, Viale Risorgimento 2, 40136 Bologna, Italy Received 8 March 2007; received in revised form 3 September 2007 Available online 24 October 2007 Communicated by F.Y. Shih
Abstract In this work, we study the usefulness of multi-resolution analysis for the face and palm authentication problems. The images are decomposed into frequency subbands with different levels of decomposition using different wavelets. We adopt as features for the authentication problem, the wavelet coefficients extracted from some ‘‘selected’’ subbands of several wavelet families. We propose to use a multi-matcher where each matcher is trained using a single subband, the matchers are combined using the ‘‘Max Rule’’. The band selection is performed by running Sequential Forward Floating Selection (SFFS). Moreover, several linear subspace projection techniques have been tested and compared. Experiments carried out on several biometric datasets show that the application of Laplacian EigenMaps (LEM) on a little subset of wavelet subbands (chosen by SFFS) permits to obtain a low Equal Error Rate. 2007 Elsevier B.V. All rights reserved. Keywords: Biometric verification; Wavelet decomposition; Face authentication; Palm authentication; Floating search
1. Introduction Advancing technological systems, such as the internet and cellular phones, have increased the need of personal identification and have made the use of a password, a secret code, or a personal identification numbers (PINs) unsafe and not user-friendly since they can easily be forgotten, compromised, shared, or observed. Biometrics, as the science of measuring and compiling distinguishing physical or biological features about an individual, such as facial structure, fingerprints or iris, is a solution for the growing need of security. The main advantage of biometrics is that it bases recognition on an intrinsic aspect of a human being and requires the person to be physically present at the point of the authentication. Palmprint serves as a reliable human identifier because the print patterns are not duplicated in other people, even in monozygotic twins. Related work concerning palmprint *
Corresponding author. Fax: +39 0547 338890. E-mail addresses: [email protected] (L. Nanni), alumini@deis. unibo.it (A. Lumini). 0167-8655/$ - see front matter 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2007.10.010
recognition consists of several statistical based methods including eigenpalm (Lu et al., 2003), fisherpalms (Wu et al., 2003), Gabor filters (Kong et al., 2003), Fourier Transform (Li et al., 2003), and local texture energy (You et al., 2003). A structural approach is presented in (Wu et al., 2002), based of the extraction from the palm image of structural information, like principal lines and creases. Face recognition problem has become one of the most relevant research areas in pattern recognition, mainly due to its wide potential application areas, ranging from human computer interaction to authentication and surveillance. Most of the appearance-based face recognition methods perform some kind of subspace analysis in the image space to extract the relevant feature vectors. The most widely used subspace analysis tools are Principal Component Analysis (Duda et al., 2000), Linear Discriminant Analysis (Belhumeur et al., 1997), Laplacian EigenMaps (He et al., 2005) and a blind source separation technique, called Independent Component Analysis (Baek et al., 2002). Two of the most recent papers based on wavelet are (Ekenel and Sankur, 2005; Connie et al., 2005). In the first
344
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
a multi-matcher is trained for Face Recognition based on all the subbands obtained by Daubechies wavelet decomposition (except the horizontal, vertical, diagonal details of the first level of decomposition1); this approach gains a performance improvement with respect to the Independent Component Analysis (Baek et al., 2002) only in presence of strong illumination variations. The second work shows that retaining the low-frequency subband of the wavelet transform and then reducing this subband using a linear projection permits to improve the performance with respect to that obtained using the original image projected onto a lower dimensional space by a linear projection. The main contribution of the paper is a method for searching the most discriminative set of wavelet channels, and the proposal of a multi-matcher for performing biometric authentication based on a fusion technique. In this study, we use several wavelet families2 in order to extract multiple subband images. These subband images contain coarse approximations of the image as well as horizontal, vertical and diagonal details at various scales. We present a new method which uses multiple subbands for building a set of classifiers, and we extract Laplacian EigenMaps features from these subbands. Best subbands are chosen by running Sequential Forward Floating Selection (SFFS) (Pudil et al., 1994; Nanni and Lumini, 2007a). A different classifier is trained for each subband, finally we exploit these multiple feature spaces by fusing their information with the ‘‘Max Rule’’ (Kittler et al., 1998). Our results are interesting: we show that combining k subbands (k = 6) from different wavelet families it is possible to obtain a significant reduction of the Equal Error Rate (EER) with respect to that obtained by Laplacian EigenMaps using the grey values of the original images. 2. Overview of system architecture Starting from the idea that different subbands of different wavelet families bring different information, we study the effects of the combination among different subbands. We argue that a feature selection method (where each feature is a given subband) may be very useful due to the different characteristics of these subbands. Therefore, we have selected the features to be combined by running Sequential Forward Floating Selection (SFFS)3 (Pudil et al., 1994). In the SFFS method, features are selected by adding the features, which provides the highest incremental of the objective function, to existing subset. As reported in Section 6, the subbands selected by SFFS belong to different wavelet functions, hence our results confirm that the subbands obtained applying different wavelet functions to the image are complementary, and, their complementarity can be exploited by fusion rules (Kittler et al., 1998).
For the performance evaluation we adopt the Equal Error Rate (see Section 6). The Equal Error Rate represents the intrinsic error of the system, and it is usually considered a good indicator of the performance of an identification system (Maio et al., 2003). Moreover, several unsupervised linear subspace projection techniques (Principal Component Analysis, Independent Component Analysis, Laplacian EigenMaps) have been tested and compared. We use Laplacian EigenMaps (which in our tests outperforms all the other methods, see Table 2) to project each subband onto a lower 100-dimensional space and then we use this space to train a 1-Nearest Neighbor classifier4 (Duda et al., 2000) (1-NN). Finally, the classifiers trained on the selected subbands are combined by the ‘‘Max Rule’’ (MAX) (Kittler et al., 1998), the max rule selects as final score the maximum score of the pool of k classifiers. The Max Rule works well when at least one classifier works well for a given pattern (Kuncheva, 2005). Please note that the max rule selection is performed independently for each pattern. For example, if in a face image there is a strong variation in illumination conditions the most useful feature set is the horizontal details subband, while if there is not a strong variation in illumination conditions the most useful feature set is the low-pass information subband (see Section 6 for more details). In Fig. 1 our system is detailed. 3. Multi-resolution analysis The wavelet transform, as compared to the traditional Fourier analysis, has a better space-frequency localization. Thus, it is suited for analyzing images where most of the informative content is represented by components localized in space – such as edges and borders – and by information at different scales or resolutions, with large and small features. In this work several wavelet families5 have been used, trusting in their capability to localize different information in time and frequency. In this paper we have tested the following wavelet functions: • • • • •
Haar (Haar). Daubechies order 4 (Daub). Symmlet order 2 (Sym). Coiflets order 2 (Coif). Biorthogonal order for reconstruction 2 and for decomposition 2 (Bio). • Reverse Biorthogonal order for reconstruction 2 and for decomposition 2 (RBio). • ‘‘Discrete’’ Meyer (Mey). Multi-resolution analysis (Ekenel and Sankur, 2005; Daubechies, 1992; Nanni and Lumini, 2007c) of the images is performed by reiterating the wavelet decomposition an
1
The high frequency subbands. We use the dyadic wavelet decomposition. 3 Implemented as in PrTools 3.1.7 (ftp://ftp.ph.tn.tudelft.nl/pub/bob/ prtools/prtools3.1.7). 2
4
Based on the Euclidean distance. We use the function dwt2.m (Symmetric-padding (half-point) (Strang and Nguyen, 1996)) of the Matlab 7.0 Wavelet Toolbox. 5
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Image processing
Feature extraction and Selection
Feature transformation
Classification and Fusion First matcher
2D wavelet + SFFS
Face\Palm Image
345
Laplacian Eigenmaps
Image PreProcessing
Nearest Neighbor classifier
Max Rule
ith matcher Laplacian Eigenmaps
Nearest Neighbor classifier
kth matcher Fig. 1. System proposed.
Fig. 2. Decomposition at level 2 of a given image.
arbitrary number of times on the low frequency part. At the first level, the original image is decomposed in four subbands leading to: the scaling component containing global low-pass information, and three wavelet components corresponding, respectively, to the horizontal, vertical and diagonal details. In Fig. 2, an example (palm image) of two level wavelet decomposition is reported. It is interesting to note (see Section 6) that the low-pass information on the wavelet transform is the most useful feature set for our verification problem; but where a strong variation in illumination conditions is present the most useful feature set is the horizontal details subband, which is less sensitive to light changing. 4. Feature transform Several linear subspace projection techniques have been tested and compared; experimental results (reported in Table 2) show that application of Laplacian EigenMaps outperforms Principal Component Analysis, Orthogonal Neighbourhood Preserving Projections and Independent Component Analysis:
Principal Component Analysis (PCA)3 (Duda et al., 2000): We use PCA to project each subband onto a lower 100-dimensional space. Moreover, in order to de-correlate the features and reduce computational complexity, PCA is also applied to project the data into a subspace where the preserved variance is 1 before the application of the other linear subspace projection techniques (presented next). Laplacian EigenMaps (LEM)6 (He et al., 2005): We use LEM to project the subband onto a lower 100-dimensional space. Independent Component Analysis (ICA)7 (Baek et al., 2002): In this paper, the representation coefficients of the patterns are assumed to be statistically independent. Therefore, in this architecture, the lexicographically ordered images constitute the columns of the observation matrix. We use ICA to project the subband onto a lower 100dimensional space.
6
Matlab code shared by Mikhail Belkin and Partha Niyogi. We use the FastIca toolbox whose Matlab code is publicly available at: www.cis.hut.fi/projects/ica/fastica/. 7
346
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Orthogonal Neighbourhood Preserving Projections (ONPP) (Kokiopoulou and Saad, 2005): ONPP is used to project the subband onto a lower 100-dimensional space.
6. Experiments We test our verification method on two biometric characteristics (palm and face), in order to evaluate how the performance depends on the dataset:
5. Feature selection We use SFFS as a feature selection method, where each ‘‘feature’’ is a classifier trained using the features extracted from a given wavelet subband, to find the most useful subbands for the recognition. Therefore we select (one for all) the classifiers to be combined by running Sequential Forward Floating Selection (SFFS) with objective function the minimization of the Equal Error Rate. In SFFS starting from a set of features F and an empty subset S0, the best feature subset of size k is constructed by adding to the subset the single feature f (f 2 F, f 62 Sk1) that gives the best performance for the new subset, this step is named forward optimization. Then a backward optimization is performed selecting as the subset of k 1 elements of Sk that has the best performance (Nanni and Lumini, 2007b). The pseudo-code of SFFS (from Nanni and Lumini, 2007b) is the following: Initialize an empty subset E0 = 100; n = 0; while n < k To search the classifier Cj so that the Equal Error Rate obtained combining the classifiers that belong to Sn1 and Cj is minimum; Sn is constructed by adding to the subset Sn1 the classifier Cj; En = Equal Error Rate obtained combing by sum rule the classifiers that belong to Sn; n = n+1; while n > 2 To test each combination of (n 1) classifiers that belong to Sn, we denote Be the combination that obtains the lowest Equal Error Rate and we denote its Equal Error Rate with EBe; if EBe < En1 En1 = Be; n = n 1; else break end end end We run two different methods based on SFFS: INTRA; INTER. In INTRA the fitness of SFFS is the Equal Error Rate obtained separately on each dataset (the subbands are selected separately on each dataset), in INTER the fitness is the average EER on the three datasets (the subbands are selected one for all on the three datasets).
• FACES: the tests have been conducted on the ORL (Samaria and Harter, 1994) and Yale-B (Belhumeur et al., 1997) datasets, which are two of the most used benchmarks in this field. ORL: It consists of 400 different images related to 40 individuals. In order to emphasize the ‘‘importance’’ of the shape of the face, we enlarge the image canvas from 112 · 92 to 142 · 122, by adding new rows and columns obtained simply smoothing to black the last row/column (as in Lumini and Nanni, 2005). Yale-B: It contains 5760 single light source images of 10 subjects each seen under 576 viewing conditions (9 poses · 64 illumination conditions). For every subject in a particular pose, an image with ambient (background) illumination was also captured. We use only the frontal poses (108 images for each individual). • PALMPRINTS: The experiments utilize inkless hand images obtained from digital Camera. The database (PALM) here used contains 720 right-hand images, 10 samples from each user, for 72 users. The palm image is extracted from the hand using a technique similar to that used in (Li et al., 2004). For all the datasets, to minimize the possible misleading results caused by the training data, the results have been averaged over five experiments, all conducted using the same parameters. For each experiment we randomly resampled the learning and the test sets (containing, respectively, half of the patterns). Before the wavelet decomposition the images are resized to 128 · 128. For the performance evaluation we adopt the Equal Error Rate (EER). EER is the error rate where the frequency of fraudulent accesses (FMR) and the frequency of rejections of people who should be correctly verified (FNMR) assume the same value; it can be adopted as a unique measure for characterizing the security level of a biometric system (Maio et al., 2003). If the trend of FMR and FNMR are plotted as a function of a threshold t indicating the degree of tolerance of the system which is a quantity in inverse relation to the degree of security (t = 0 and 1 indicate the maximum and minimum security lever, respectively). The point of intersection of the two curves denotes the EER (see Fig. 3). The EER represents the intrinsic error of the system, and it is usually considered a good indicator of the performance of an identification system. In real applications the security level is usually fixed at a value higher than EER in order to limit the number of fraudulent accesses (Maio et al., 2003). The pre-processing stage is very important to normalize the images in order to smoothen the noise and lighting
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
347
Table 2 Comparison among the linear subspace projection techniques PCA,LEM, ONPP and ICA Classifier
PALM Method
ORL Method Fig. 3. We plot the curves FMR and FNMR.
YALE-B Method
Table 1 EER obtained adopting different pre-processing procedures Dataset
Methods NO CONNIE ADAPT
PALM
ORL
YALE-B
10 4 4.1
3 3 3.5
2.2 1.7 1.3
1-NN
LCOS
PCA ONPP LEM ICA
4 5.2 3.1 3.3
3.7 5 3.7 3.7
PCA ONPP LEM ICA
3 4.1 2.7 4.1
4.8 4.5 4.6 7.6
PCA ONPP LEM ICA
1.7 1.6 2 0.5
3.5 2.2 4.5 2
3 2.5 PALM ORL YALE-B
2
effect. Our results have been obtained adopting the following pre-processing methods:
1.5 1
• NO: we do not use any pre-processing procedure. • CONNIE: we use the method proposed in (Connie et al., 2005). • ADAPT: we perform contrast-limited adaptive histogram equalization8 on the results of CONNIE. The first test, reported in Table 1, is aimed to prove the advantage of adopting a pre-processing procedure, in these tests we report the EER (in percentage) obtained projecting the original image onto a lower 100-dimensional space by PCA and then using a 1-NN classifier. These results demonstrate that, especially in the YALE-B and PALM datasets, the pre-processing procedure is important,9 even if it is interesting to note that in the YALE-B dataset ADAPT outperforms CONNIE, but in the ORL dataset CONNIE outperforms ADAPT. Since CONNIE proved the best preprocessing method in all the following experiments we adopt this pre-processing procedure. In Table 2 we compare, using the EER in percentage, the linear subspace projection techniques detailed in Section 4 (PCA, LEM, ICA and ONPP) using a 1-NN classifiers (1-NN) or the cosine distance (LCOS) for calculating the scores. 8
Implemented as in adapthisteq.m of the Matlab 7.0 Image Processing Toolbox. 9 In these dataset the performance of CONNIE was statistically superior, considering the 95% confidence intervals (computed using t distribution (Spiegel, 1992)), to NO.
0.5 1
2
3
4
5
6
7
8
9
10
Fig. 4. EER as a function of the number k of subbands selected by SFFS on PALM, ORL and YALE-B datasets.
Since (see Table 2) in our test, LEM outperforms all the other methods, we use LEM to project each subband in a reduced space.10 In Fig. 4, we plot the EER obtained combining the results gained by the k best subbands selected (the selected subbands are reported in Table 5) by SFFS (SFFS is run separately on each dataset). By the analysis of this plot we argue that k = 6 is a good compromise between performance and complexity, even if the fusion of two or three subbands reach good results if compared with a single subband. In Table 3, we report some experiments where we have reduced the dimension using a subspace where the dimension is: (number of pixel/100); (number of pixel/50); (number of pixel/25). In this way the dimension of the projection subspace is related to the size of the subband. In our opinion each different subband bring a different ‘‘quantity’’ of information and are differently affected by the noise, hence the different of the projection should be 10 We have run LEM, PCA, ONPP and ICA also in each subband, also in these tests LEM outperforms others Feature Transforms.
348
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Table 3 We report the results of the method INTER
Table 5 Subbands selected by SFFS on the three datasets (INTRA method) and performing a global optimization (INTER method)
Dataset PALM Method (Number of pixel/100) (Number of pixel/50) (Number of pixel/25) Dimension fixed to 100
ORL
PALM 2.83 2.87 2.81 2.68
2.14 2.28 2.22 1.99
1.2 1.36 1.2 1.19
different in each subbands. The main problem is that it is difficult to find an automatic method to find the dimension of the projected space. In our tests a value of 100 permits to obtain good results in all the dataset (we have three dataset and two biometric characteristics). The tests reported in Table 4 are aimed to compare the system verification performance gained by the following systems: • The related work proposed in (Ekenel and Sankur, 2005). • NOWAVELET, a system obtained training a 1-NN as classifier projecting the original image onto a lower 100-dimensional space by LEM. • BEST, the EER obtained by the best subband (best matcher, that is k = 1 and no fusion). • INTRA, the complete system described in Section 2 where the features are obtained selecting the best k = 6 subbands separately on each dataset. In INTRA the fitness of SFFS is the EER obtained in a given dataset. • INTER as the previous one, but selecting the six best subbands one for all on the three datasets. In INTER the fitness of SFFS is the average EER on the three datasets. The selected subbands are reported in Table 5, with GLOBAL we denote the subbands selected by the INTER method (which are the same for the three datasets), PALM, ORL and YALE-B denote the subbands selected by the INTRA method for the three datasets consider, respectively. In Table 5, we show which subbands are selected by SFFS. In each cell there are three values, the first indicate the wavelet family, the second is the subband, the third is
Table 4 Performance obtained for the verification problem adopting different methods (the * indicates the presence of feature normalization mean 0 and standard deviation 1) Dataset
Method Ekenel and Sankur (2005) Ekenel and Sankur (2005)* NOWAVELET BEST INTRA INTER
Dataset
YALE-B
PALM
ORL
YALE-B
10.3 3.7 3.1 2.9 2.68 2.68
14 3.8 2.7 2.27 1.54 1.99
1.62 3.6 2 1.73 0.87 1.19
ORL
YALE-B
GLOBAL
Selected subbands 1 B-I-3 M-I-2 M-II-3 M-I-2 2 M-I-3 H-II-1 D-I-2 M-II-3 3 H-I-2 R-III-3 H-II-3 R-III-3 4 R-III-3 R-II-3 H-II-2 R-II-3 5 H-IV-2 H-III-3 S-II-2 S-I-2 6 H-II-3 M-I-1 H-III-3 H-II-2 H Haar D Daubechies order 4 S Symmlet order 2 C Coiflets order 2 B Biorthogonal order for reconstruction 2 and for decomposition 2 R Reverse Biorthogonal order for reconstruction 2 and for decomposition 2 M ‘‘Discrete’’ Meyer I II III IV
Low-pass information Horizontal details Vertical details Diagonal details
the level of decomposition. In Fig. 5, we show the grey level image and the best four wavelet subbands extracted from that image. The experimental results show that: • Our approach (INTER) significantly improves the performance of the base method (NOWAVELET and BEST), the EER is very low not only when we select the best subbands on each dataset, but also when the optimization is performed on all the three datasets (same subbands used for all the three datasets). • It is very interesting to note that the subbands selected by SFFS belong to different wavelet functions, this fact provides evidence that the multi-resolution decompositions obtained by different wavelet functions are complementary, and, their complementarity can be exploited by fusion rules. • From the analysis of Table 4, we can observe that the low-pass information on the wavelet transform is, as expected, the most useful feature set for our verification problem; it is interesting to note that in the YALE-B dataset, where a strong variation in illumination conditions is present, the first feature vector selected is a horizontal details subband, which is less sensitive to light changing. In order to evaluate the benefit of the our method we perform a further experiment. The DET curve (Martin, 1997) is a two-dimensional measure of classification performance that plots the probability of false acceptation against the rate of false rejection. In Figs. 6–8, we plot the DET-curve on the tested dataset. The black line is the DET-curve obtained by INTRA,
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
349
Fig. 5. Grey level image and the best four wavelet subbands.
Fig. 6. DET-curve on the ORL dataset.
Fig. 8. DET-curve on the PALM dataset.
Table 6 EER obtained using only the best subband for each dataset Classifier (1-NN)
Method PCA ONPP LEM ICA
PALM
ORL
YALE-B
4 5 2.9 3.1
3.1 3.5 2.27 3.5
1.8 1.8 1.73 0.9
the red11 line is the DET-curve obtained by NOWAVELET. Also in these tests it is clear that INTRA outperforms NOWAVELET. As further experiment, we have ran LEM, PCA, ONPP and ICA in each wavelet subband, also in these tests LEM outperforms others Feature Transforms. In Table 6, we report the performance (EER) obtained using only the best subband for each dataset.
Fig. 7. DET-curve on the YALE-B dataset.
11 For interpretation of color in Figs. 6–8, the reader is referred to the web version of this article.
350
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
As final tests we report two examples of how max rule works (see Fig. 9): • In the first example we match two faces where in one face there is a strong variation in illumination conditions. • In the second example we match two faces where in both the faces there is NOT a variation in illumination conditions. In the first example the max rule selects the score obtained by the horizontal details subband (which is less sensitive to light changing) while in the latter example the max rule selects the score obtained by the low-pass information subband. In Fig. 9 with SC1, we name the score obtained by the low-pass information subband, while with SC2 we name the score obtained by the horizontal details subband.
Fig. 9. Examples of how max rule works.
6.1. Experiments on Q-statistic
Table 7 Characteristics of the wavelets tested in this work Wavelet
Orthogonal
Symmetry
Haar Daubechies Symmlet Coiflets Biorthogonal Reverse Biorthogonal Discrete Meyer
Yes Yes Yes Yes No (it is biothogonal) No (it is biothogonal) Yes
No No Near Near Yes Yes Yes
As a further experiment we investigated the relationship (i.e. the error independence) between the matchers trained using the different wavelet subbands. The average Q-statistic among the matchers is 0.4. This value is low and it is well known in the literature that combining ‘‘independent’’ classifiers (Nanni and Lumini, 2005; Nanni, 2005; Nanni and Lumini, 2007c) permits to dramatically reduce the error rate obtained by a ‘‘stand-alone’’ classifier.
Fig. 10. Wavelets used in this work.
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Error independence is an important property in combining classifiers. The independence of classifiers is calculated using Yule’s Q-statistic (Kuncheva and Whitaker, 2003). For two classifier Di and Dk the Q-statistic is Qi;k ¼
ad bc ad þ bc
where a is the probability of both classifiers being correct, d the probability of both classifiers being incorrect, b the probability first classifier is correct and second is incorrect, c is the probability second classifier is correct and first is incorrect. For statistically independent classifiers, Qi,k =
351
0. Q varies between 1 and 1. Classifiers that tend to recognize the same patterns correctly will have Q > 0, and those which commit errors on different patterns will have Q < 0. To better motivate the different behavior of the wavelet we report: • Main mathematical characteristics (see Table 7). • The plot of the wavelet functions tested in this work (see Fig. 10). • Multi-resolution of two images with different illumination conditions (see Figs. 11 and 12).
Fig. 11. Multi-resolution analysis of a face without variation in the illumination conditions.
352
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
Fig. 12. Multi-resolution analysis of a face with variation in the illumination conditions.
The two most complementary wavelet selected by SFFS in INTER are Discrete Meyer and Reverse Biorthogonal, it is interesting to note that Discrete Meyer is orthogonal while Reverse Biorthogonal is not orthogonal. Probably this mathematical difference determines the complementarities of these two wavelets. From Fig. 10 it is clear that the pool of tested wavelet have different wavelet functions and this permit to obtain (slightly) different subband images. From the Figs. 11 and 12 some behaviors of the tested wavelet can be extracted:
• If in the images there are NOT illuminations problems the Haar’s horizontal details subband works better than the Meyer’s horizontal details subband (see Fig. 11), for this reason SFFS in the ORL dataset (illuminations variations are not present) selects the Haar’s horizontal details but not the Meyer’s horizontal details subband. • In Fig. 12, the previous conclusion is not more valid, the Meyer’s horizontal details subband works better probably this is due to the fact that the Meyer wavelet is less sensitive to the edge of the image.
L. Nanni, A. Lumini / Pattern Recognition Letters 29 (2008) 343–353
• RBIO has a similar behaviour in both the images, probably for this reason the fusion among the Meyer subbands and the Reverse Biorthogonal subbands permits to obtain a robust system (please note that the first four subbands selected by SFFS in INTER belong to MEY or to RBIO). 6.1.1. Conclusions We investigated fusion of classifiers trained using different frequency subbands obtained by wavelet transform at different levels of decomposition. It is reported in the literature that classifier sets that enforce diversity far better than ones that do not. To enforce diversity we combine classifiers based on different feature extractions. The experimental results obtained on three different datasets are encouraging: we show that using a feature selection approach (SFFS) it is possible to choose few subbands to be exploited for combination and the EER gained by our multi-classifier is lower than that obtained by the best stand-alone matcher on the grey values of the original image. Our method can be easily combined with supervised learning (e.g. using the Fisher Transform to project the data) or to bio-inspired methods as (Tan and Chen, 2005). E.g. we can use our method to extract the features from given subwindows of the image and than the different contributions made by different parts of the image are emphasized (in a ‘‘weighted’’ fusion). As future work we want to study supervised methods (Gutschoven and Verlinde, 2000) to combine the scores of the pool of matchers. Acknowledgement This work has been supported by European Commission IST-2002-507634 Biosecure NoE projects. References Baek, K., Draper, B., Beveridge, R., She, K., 2002. PCA vs. ICA: A comparison on the FERET data set. In: Joint Conference on Information Sciences, Durham, NC, March 2002, pp. 824–827. Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J., 1997. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 711–720. Connie, T., Jin, A.T.B., Ong, M.G.K., Ling, D.N.C., 2005. An automated palmprint recognition system. Image and Vision Computing 23, 501–515. Daubechies, I., 1992. Ten Lectures on Wavelets. Capital City Press, Vermont. Duda, R.O., Hart, P.E., Stork, G., 2000. Pattern Classification, second ed. Wiley, New York. Ekenel, H.K., Sankur, B., 2005. Multiresolution face recognition. Image and Vision Computing 23, 469–477. Gutschoven, B., Verlinde, P., 2000. Multi-modal identity verification using support vector machines. In: Proceedings of the Third International
353
Conference on Information Fusion, Paris, France, 10–13 July 2000, pp. 3–8. He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H-J, 2005. Face Recognition using Laplacian faces. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (3), 328–340. Kittler, J., Hatef, M., Duin, R., Matas, J., 1998. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (3), 226–239. Kokiopoulou, E., Saad, Y., 2005. Orthogonal neighborhood preserving projections. In: IEEE International Conference on Data Mining, New Orleans, Lousiana, USA, 27–30 November, 2005. Kong, W.K., Zhang, D., Li, W., 2003. Palmprint feature extraction using 2-D Gabor filters. Pattern Recognition 36 (10), 2339–2347. Kuncheva, L.I., 2005. Diversity in multiple classifier systems. Information Fusion 6 (1), 3–4. Kuncheva, L.I., Whitaker, C.J., 2003. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51, 181–207. Li, W., Zhang, D., Xu, Z., 2003. Palmprint identification by Fourier Transform. International Journal of Pattern Recognition and Artificial Intelligence 16 (4), 417–432. Li, Q., Qiu, Z., Sun, D., Wu, J., 2004. Personal identification using Knuckleprint. Sinobiometrics, 680–689. Lu, G., Zhang, D., Wang, K., 2003. Palmprint recognition using eigenpalms features. Pattern Recognition Letters 24 (9–10), 1473–1477. Lumini, A., Nanni, L., 2005. Combining classifiers to obtain a reliable method for face recognition. Multimedia Cyberscape Journal 3 (3), 47– 53. Maio, D., Maltoni, D., Jain, A.K., Prabhakar, S., 2003. Handbook of Fingerprint Recognition. Springer, New York. Martin, A., et al., 1997. The DET curve in assessment of decision task performance. In: Proceedings of the EuroSpeech, pp. 1895–1898. Nanni, L., 2005. Comparison among feature extraction methods for HIV1 Protease Cleavage Site Prediction. Pattern Recognition, Pattern Recognition 39 (4), 711–713. Nanni, L., Lumini, A., 2005. Ensemble of parzen window classifiers for on-line signature verification. NeuroComputing 68 (6), 217–224. Nanni, L., Lumini, A., 2007a. A hybrid wavelet-based fingerprint matcher. Pattern Recognition 40 (11), 3146–3151. Nanni, L., Lumini, A., 2007b. RegionBoost learning for 2D + 3D based face recognition. Pattern Recognition Letters 28 (15), 2063– 2070. Nanni, L., Lumini, A., 2007c. A multi-expert approach for wavelet-based face detection. Pattern Recognition Letters 28 (12), 1541–1547. Pudil, P., Novovicova, J., Kittler, J., 1994. Floating search methods in feature selection. Pattern Recognition Letters 15 (11), 1119–1125. Samaria, F., Harter, A., 1994. Parameterisation of a stochastic model for human face identification. In: Proceedings of the Second IEEE Workshop on Applications of Computer Vision, Florida. http:// www.uk.research.att.com/facedatabase.html. Spiegel, M.R., 1992. Theory and Problems of Probability and Statistics. McGraw-Hill, New York, pp. 116–117. Strang, G., Nguyen, T., 1996. Wavelets and filter banks. Cambridge Press, Wellesley. Tan, K., Chen, S., 2005. Adaptively weighted sub-pattern PCA for face recognition. Neurocomputing 64, 505–511. Wu, X., Wang, K., Zhang, D., 2002. Fuzzy directional element energy feature (FDEEF) based palmprint identification. In: International Conference on Pattern Recognition, pp. 95–98. Wu, X., Zhang, D., Wang, K., 2003. Fisherpalms based palmprint recognition. Pattern Recognition Letters 24, 2829–2838. You, J., Li, W., Zhang, D., 2003. Hierarchical palmprint identification via multiple feature extraction. Pattern Recognition 35, 847–859.