- Email: [email protected]
Palmprint authentication using a symbolic representation of images
Image and Vision Computing 28 (2010) 343–351
Contents lists available at ScienceDirect
Image and Vision Computing journal homepage: www.elsevier.com/locate/imavis
Palmprint authentication using a symbolic representation of images Jiansheng Chen a,*, Yiu-Sang Moon b, Ming-Fai Wong c, Guangda Su a a b c
Department of Electronic Engineering, Tsinghua University, Haidian, Beijing 100084, PR China Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong Department of Computer Science, University of Toronto, 10 King’s College Road, Toronto, Canada M5S 3G4
a r t i c l e
i n f o
Article history: Received 4 December 2007 Received in revised form 30 March 2009 Accepted 2 June 2009
Keywords: Biometrics Palmprint authentication Symbolic representation Time series
a b s t r a c t A new branch of biometrics, palmprint authentication, has attracted increasing amount of attention because palmprints are abundant of line features so that low resolution images can be used. In this paper, we propose a new texture based approach for palmprint feature extraction, template representation and matching. An extension of the SAX (Symbolic Aggregate approXimation), a time series technology, to 2D data is the key to make this new approach effective, simple, flexible and reliable. Experiments show that by adopting the simple feature of grayscale information only, this approach can achieve an equal error rate of 0.3%, and a rank one identification accuracy of 99.9% on a 7752 palmprint public database. This new approach has very low computational complexity so that it can be efficiently implemented on slow mobile embedded platforms. The proposed approach does not rely on any parameter training process and therefore is fully reproducible. What is more, besides the palmprint authentication, the proposed 2D extension of SAX may also be applied to other problems of pattern recognition and data mining for 2D images. Ó 2009 Elsevier B.V. All rights reserved.
1. Introduction Automated personal authentication using biometric features has been widely studied during the last two decades. Previous research efforts have made it possible to apply biometric systems to practical applications for security or commercial purposes. Biometric systems based on fingerprint recognition [1], face recognition [2], and iris recognition [3] have already been developed to a quite mature stage so that they can be applied to critical security applications such as the immigration control and the crime investigation. Recently, a novel hand-based biometric feature, palmprint, has attracted an increasing amount of attention. Like any other biometric identifiers, palmprints are believed to have the critical properties of universality, uniqueness, permanence and collectability for personal authentication [4]. Palmprints have several advantages over other hand-based biometrics, such as fingerprint and hand geometry. Compared to fingertips, palms are larger in size and therefore are more robust to injuries and dirt. Also, low-resolution imaging can be employed in the palmprint recognition based on creases and palm lines, making it possible to perform real time preprocessing and feature extraction; and the cost of the capturing device can also be well controlled. Palmprint authentication is believed to be able to achieve the accuracy comparable to that of * Corresponding author. Tel.: +852 31634251; fax: +852 26035024. E-mail address: [email protected] (J. Chen). 0262-8856/$ - see front matter Ó 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2009.06.004
other hand-based biometric authentication technologies, including fingerprint [1] and hand geometry [5,6]. Texture and palm lines are the most clearly observable palmprint features in low resolution (such as 100 dpi) images [7], and thus have attracted most research efforts. In texture based palmprint authentication approaches [7–14], signal processing based texture analysis methods [15] are usually adopted. Typically, texture features are extracted by filtering the palmprint images using filters such as the Gabor filter [7,11–13], the ordinal filter [10], or the wavelet [8]. The image filtering may be performed in either the spatial domain or the frequency domain [14]. Recently, a lot of automated palmprint authentication methods [16–19] have focused on the palm line features, since they are more appealing than the texture for the human vision. In the offline method proposed in [16], the geometric shapes of the palm lines are extracted and approximated by straight-line segments. The slope, intercept and inclination of each segment are used as features for palmprint matching. C.C. Han et al. investigate the magnitude of palm lines in palmprint matching [17]. The latest related research reveals that the orientations of palm lines also contain strong discriminative power. Based on palm line orientations, a Competitive Code is designed for palmprint representation in [18]; and Y. Han et al. use local orientation histograms for describing palmprints [19]. Similar to the texture based methods; the palm line based methods usually employ image filtering for line feature extraction, leading to a high computational complexity. For example, in the Competitive Code method [18], six Gabor filters are
344
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
applied to each palmprint ROI (Region Of Interest) for generating the corresponding orientation map. Suppose that the Palmprint ROI is 128 128 pixels and the Gabor filters are 35 35 in size, the overall MADD (Multiplication + Addition) operations required for one palmprint is around 120 million, leading to a very long processing time especially on slow mobile platforms. Experiments show that extracting the Competitive Code for one palmprint takes more than eight seconds on a state of the art PDA [20]. This is far too slow for a real-time biometric system. Besides the computational complexity, selecting appropriate filter parameters is also nontrivial in filtering based palmprint authentication methods. It has been demonstrated in [4,7] that the authentication accuracy varies a lot when using different Gabor filter parameters, which need to be tuned in a try and error manner, indicating that the authentication performance will depend a lot on the training set used for parameter selection. This may account for the significant performance variations of different filtering based palmprint authentication methods on different databases [19]. In this paper, we propose a texture based approach for palmprint authentication, in which palmprint image grayscale information are directly adopted as features. The computational complexity of the feature extraction process is much lower than previous filtering based approaches, and thus can be implemented efficiently for even slow mobile embedded platforms. By extending the idea of SAX (Symbolic Aggregate approXimation) [21] in time series research to 2D images for palmprint representation and matching, the proposed method can achieve the authentication performance, in terms of EER (Equal Error Rate), comparable to the state of the art palmprint authentication methods. The rest of this paper is organized as the follows. Section 2 explains the 2D extension of SAX for images. Section 3 describes the feature extraction and matching processes of the proposed approach. Experiments and results are elaborated in Section 4. The last section is a conclusion of our work. 2. 2D SAX As a symbolic representation of sequential data, SAX (Symbolic Aggregate approXimation) [21] has been verified as a simple but effective tool for solving most time series data mining problems such as clustering, classification, indexing, anomaly detection, and motif finding [22]. For a real valued data sequence, its SAX representation is obtained by first transforming it into the PAA (Piecewise Aggregate Approximation) representation; then predetermined breakpoints are used to discretize the PAA coefficients into SAX symbols. Basically, SAX represents a real valued data sequence using a string of discrete symbols or characters, as is shown in Fig. 1. Dramatic dimensionality reduction can be achieved by converting a time series into its SAX representation. In order to preserve the discriminative power of the data as complete as possible, Keogh et al. proposed a discretization technique which can produce symbols with equiprobability by using discretization breakpoints following a Gaussian distribution [21]. This specific
Table 1 The lookup table for calculating dist() function when SAX_Level equals four [21].
‘a’ ‘b’ ‘c’ ‘d’
‘a’
‘b’
‘c’
‘d’
0 0 0.67 1.34
0 0 0 0.67
0.67 0 0 0
1.34 0.67 0 0
property of SAX has made it suitable for the feature representation in biometric authentication where the discriminative power of the representation is crucial. From Fig. 1, we can see that the degree of dimensionality reduction is decided by two parameters: the length of the resultant symbol string, or the SAX_Length; and the number of symbols used, or the SAX_Level. As an example, the SAX_Length equals eight and the SAX_Level equals four in Fig. 1. The smaller these two parameters are, the more intensive the dimensionality reduction is. A distance measurement called MINDIST, which strictly lower bounds the corresponding Euclidean distance of the original time series [23], is employed to measure the similarity between two SAX b ¼q ^2 ; . . . ; q ^w and ^1 ; q strings [21]. Given two SAX strings Q b ¼ ^c1 ; ^c2 ; . . . ; ^cw , their MINDIST can be calculated using Eq. (1), C where n is the length of the original time series, and dist() can be implemented using table lookup as is illustrated in Table 1 [21].
b; C bÞ ¼ MINDISTð Q
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiu w X nu t ^i ; ^ci Þ2 distðq w i¼1
ð1Þ
For high dimensional data, such as a 2D or 3D image, one feasible way of applying the SAX conversion is to firstly decompose it into a 1D sequence. For example, 3D fMRI (functional Magnetic Resonance Imaging) images are decomposed into 1D sequences using the Hilbert space filling curve for clustering and motif finding in [24]. In our previous work [25], palmprint ROIs are decomposed using spiral curves before SAX conversion. Nevertheless, the neighborhood information, which plays an important role in similarity measurement for high dimensional data, will inevitably be lost during the 1D decomposition no matter what space filling curve is used. For instance, in a 2D image, each pixel, except those on the boundary, has eight eight-neighbor pixels. However, after decomposition, each pixel will only have two neighbors in the resultant 1D sequence. Such a phenomenon is illustrated in Fig. 2, where (I) and (II) shows the up-left corners of two images. Without losing generality, pixels are represented by cells inside which the values stand for the grayscale level in the hexadecimal format. SI and SII ðS0I and S0II Þ are the 1D decomposition results using progressive scan (Hilbert curve scan). It is not difficult to observe that in their 2D forms, image (I) and image (II) are probably identical except for a one pixel shift in both the vertical and the horizontal directions. This apparent similarity, however, cannot be easily discovered from their corresponding 1D sequences. The fundamental reason for this problem is the serious loss of neighbor-
Fig. 1. Conversion of a time series to its SAX representation. (a) The original real valued time series. (b) PAA representation. (c) SAX representation.
345
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
real valued 2D data using a 2D matrix of symbols. Assume that the size of the input data Q is m n. First, Q is normalized to have a zero mean and a unit standard deviation. Then, similar to the PAA representation, Q can be represented by a matrix Q of the size w1 w2 . The ðith ; jth Þ element of Q can be calculated by Eq. (2). Simply stated, the original data Q is divided into w1 w2 equal size blocks, and the mean value of the data inside each block is calculated to form the dimensionality reduced representation: Q . m
1 Qði; jÞ ¼ w1 w2
hood information. In Fig. 2, ‘5’ and ‘A’, which are neighbors in both (I) and (II), are no longer adjacent in any of the 1D sequences; and they are separated in such a irregular way that existing time series alignment techniques, such as sequence shifting, DTW (Dynamic Time Warping) alignment [26] and LCSS (Longest Common Subsequence) alignment [27], cannot be consistently applied to recover the lost neighborhood information. This phenomenon accounts for the unsatisfactory verification accuracy we have achieved in our previous attempt of applying SAX to palmprint authentication [25]. To facilitate a more effective SAX conversion for 2D data, we propose the 2D SAX, an extension of the SAX, which represents a
n
j
w2 X
Q ðx; yÞ
ð2Þ
x¼wm ði1Þþ1 y¼wn ðj1Þþ1 1
Fig. 2. 1D decomposition of 2D images using different space filling curves.
i
w1 X
2
Finally, breakpoints identical to those used in 1D SAX converb , or the 2D sion are applied to convert Q into a symbol matrix Q SAX representation. The definitions and properties of these breakpoints can be found in [21]. These breakpoints are selected to equally divide the area under the probability distribution function curve of a standard normal distribution due to the Gaussianity nature of real life data [21]. Fig. 3 illustrates the process of a sample 2D SAX conversion. In particular, the 2D SAX_Length is now defined as w1 w2 ; and the definition of the 2D SAX_Level is identical to that of the 1D case. The total operations required for such a conversion is no more than mn additions, plus mn=w1 w2 divisions, plus ðSAX LevelÞ mn=w1 w2 comparisons. Also, the MINDIST for 2D SAX is now redefined as Eq. (3). Actually, such a definition can be further extended to 3D or even higher dimensionality.
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiffiffiffiffiffiffiffiffiuX w2 u w1 X mn b; C bÞ ¼ t ^i;j ; ^ci;j Þ2 MINDISTð Q distðq w1 w2 i¼1 j¼1
ð3Þ
The 2D SAX conversion can be directly applied to images, which are special cases of two-dimensional data. For color images, the 2D SAX conversion can be applied to each color channel, respectively,
Fig. 3. Conversion of a 2D data into its 2D SAX representation. (a) The original real valued 2D data of size 64 64. (b) The block wise average representation and the 2D SAX conversion. (c) The resultant 2D SAX representation.
Fig. 4. Conversion of a 2D image into its 2D SAX representation. (a) The original noisy image. (b) Noise suppression using Gaussian filtering. (c) The resultant 2D SAX representation. (d) Blowup of the lower-left corner of image (c).
346
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
and the MINDIST can simply be defined as the summation of the MINDIST values of all the channels. In this paper, only grayscale images are considered as the palmprint images [28,29] to be used are all intensity images. For a 2D image I, image filtering can be applied prior to the 2D SAX conversion. Suppose G is a digital filter, we denote the filtering process as Eq. (4). This enhances the flexibility of the 2D SAX representation, since image filtering is a fundamental tool in most image processing techniques. Fig. 4 illustrates the process of converting a noisy Lenna into its 32 32 2D SAX representation, in which the SAX_Level equals four. The original image occupies 262K bytes and the 2D SAX only needs 256 bytes to store. Tremendous dimensionality and numerosity reduction is thus achieved.
Q ¼IG
ð4Þ
3. 2D SAX conversion for palmprints Two public palmprint databases are used for the experiments in this work [28,29]. The PolyU Palmprint Database [28] contains 7752 grayscale palmprint images (384 284 pixels, 96 dpi) corresponding to 386 different palms. For each palm, there are around 20 samples collected in two sessions, where around 10 samples were captured in the first session and the second session, respectively. The average interval between the two sessions was 2 months. Palmprints in this database were captured using a patented palmprint capturing device [30] in which pegs are used to limit the movement and the distortion of the user’s palm. The illumination of the devices is tuned to facilitate the capture of palm
line details. As a result, palmprints in the PolyU Palmprint Database are believed to have relatively high image qualities. The other database used is the CASIA Palmprint Image Database [29] which contains 5239 grayscale palmprint images (640 480 pixels, 72 dpi) collected from 301 individuals. All palmprints were collected in one single session. Comparing to the PolyU capturing device, the device used for constructing the CASIA database imposes less physical constrains on the user’s palm, leading to more obvious palm movements, distortions and blurring in the captured palmprint images. Fig. 5 compares sample palmprint images from the two databases. Fig. 5(c) and (d) belong to the CASIA database and are captured from the same palm. However, intra-class variation caused by shifting and distortion can be easily observed. Moreover, Fig. 5(d) is obviously blurred probably because of the palm motion or the camera defocusing. Before the 2D SAX conversion, the input palmprint images should be aligned to a predefined universal coordinate to facilitate consistent ROI extraction. We adopt a palm coordinate system similar to the second square-based palm coordinate system proposed in [4]. The following steps are performed for the palmprint alignment. A sample palmprint form the PolyU database is used to illustrate this procedure in Fig. 6. (1) A fixed threshold is applied to convert the original image into a binary image (Fig. 6b). Isolated pixels, spurs and leaks are removed using morphological operations. (2) The palm boundary (Fig. 6c) is obtained using a boundary tracking algorithm [31].
Fig. 5. Sample palmprint images. Image (a) and image (b) belong to the PolyU database. They are captured from the same palm. Image (c) and image (d) belong to the CASIA database. They are also captured from the same palm.
Fig. 6. Palmprint image preprocessing: alignment and ROI extraction.
347
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
(3) The maximum curvature points P1 and P2 between the fingers (Fig. 6d) are located using a curvature maxima finding algorithm [32]. (4) P1 and P2 are connected by a line L1. Draw another line L2 passing through the middle point P3 of L1 perpendicularly (Fig. 6e). A point P4 is found on L2 so that the length between P3 and P4 equals a predefined value. (5) A square area S (Fig. 6f/g) of fixed size (135 135 pixels) is extracted with P4 as its center. S is defined as the ROI for 2D SAX conversion. (6) Non-uniform illumination correction is performed to enhance the palmprint texture (Fig. 6h). Fig. 7 shows the ROI extraction result for a palmprint from the CASIA database. The CASIA palmprints are first resized to 380 284 pixels before the ROI extraction. The extracted palmprint ROI are then converted to the 2D SAX form for palmprint representation and matching. Several sample PolyU palmprint ROIs are shown in Fig. 8, in which each two ROIs in the same column are from the same palm; and the ROIs in the first and second rows are from session one and session two, respectively. Visually, the ROIs are extracted consistently. An average filter G is employed to filter the extracted ROIs so as to suppress noises caused by factors such as dirt, injuries and camera noises. To fulfill our objective of low computational complexity, the filter G is decomposed into two 1D average filters, so that the filtering process can be equivalently implemented as the combination of two 1D filtering processes as is illustrated in Eqs. (5) and (6). And the 1D average filtering can be efficiently implemented such that each pixel requires only one addition plus one subtraction. As a result, the overall complexity of the average filtering process is around 4N, where N is the total number of pixels in the image. The 128 128 pixels central part of the filtered ROI is cropped
for the 2D SAX conversion. Fig. 9 shows the corresponding 2D SAX representations for the palmprint ROIs shown in Fig. 8, where w1 and w2 equal 32 and the SAX_Level is set to 4. These 2D SAX representations are used as palmprint templates. And the storage size of each template is 256 bytes.
2 3 1 617 7 61 1 ... 1 7 6 7 6 G¼6 7 ¼ 6 .. 7 7 ½1 1 ... 1 ¼ U V 4... ... ... ...5 6 4. 5 1 1 ... 1 1
ð5Þ
I G ¼ I ðU VÞ ¼ ðI UÞ V
ð6Þ
2
1
1
... 1
3
The MINDIST between two templates is adopted as the similarity measurement of the corresponding two palmprints. The smaller the MINDIST is, the more similar the two palmprints are. As human palm is a particularly flexible organ, distortions are unavoidable during palmprint capture, leading to the fact that no alignment scheme is able to extract the ROI perfectly [4]. We need to vertically and horizontally translate one of the templates to find the best match, or the smallest MINDIST value. The definition of the MINDIST after translation is identical to Eq. (3), except that the values of w1 and w2 should be modified according to the size of the overlapping template area. The range of the vertical and the horizontal translations are defined from 2 to 2. In the next section, we will show the discriminative power of the SAX representation and the MINDIST for the palmprint authentication through verification and identification experiments. 4. Experiments A complete one to one matching experiment is performed on the PolyU palmprint database [28] to test the effectiveness of the proposed method. There are altogether 74068 genuine matching
Fig. 7. A CASIA palmprint and the corresponding ROI extraction.
Fig. 8. Sample palmprint ROIs.
348
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
Fig. 9. 2D SAX representations for the sample palmprint ROIs in Fig. 8.
and 29968808 imposter matching. The experimental settings are as follows: G is a 7 7 average filter; SAX_Length = 32 32 = 1024, and SAX_Level equals 4. Fig. 10 shows the score distributions of the genuine matching and the imposter matching, as well as the ROC (Receiver Operating Characteristic) curve. The EER of the proposed method is 0.3% with the corresponding threshold at 46.2. We compare the performance of the proposed method to that of three widely referenced palmprint authentication approaches: PalmCode [7], FusionCode [11,12] and Competitive Code [18]. The comparison results are shown in Table 2. To avoid any uncertainty in the implementation, the accuracies of the three reference approaches are quoted from the original papers. The verification results for the PalmCode [7], the FusionCode [11], and the Competitive Code [18] were also generated from one to one matching among 7752 palmprints in the PolyU database. This ensures the fairness of the performance comparison. It can be observed from Table 2 that in general the proposed method over performs the FusionCode and the PalmCode. In terms of EER, our method is comparable to the Competitive Code method. It is true that for small FAR values, the Competitive Code method performs better. However, the outstanding performance of our methods for relatively large FAR values can be regarded as a trade off. Also, our method does not rely on any training of the parameters, and has very low computational complexity and requires relatively small storage size. Table 3 lists the estimated computational complexity of the different feature extraction methods. We only focus on the feature extraction process since the
Table 2 Verification accuracies of different methods on the PolyU database. False acceptance rate
106 105 104 103 102 101 Equal error rate
False rejection rate 2D SAX (%)
Competitive Code (%)
FusionCode (%)
PalmCode (%)
5.9 4.1 1.9 0.5 0.1 <0.01
1.1 0.9 0.6 0.5 0.3 0.2
6.7 4.0 2.0 1.1 0.4 0.2
8.5 5.6 3.0 1.4 0.5 0.1
0.3%
0.56%
0.6%
0.3%
palmprint preprocessing stage, which contains palmprint alignment and ROI cropping, is common to all the four methods. We assume that the palmprint ROI is 128 128 pixels and the Gabor filters used by the PlamCode, FusionCode and Competitive Code are of the size 35 35 [4]. The complexity of the proposed method is orders of magnitude lower than the other three methods, leading to the possibility of real time implementations for slow mobile embedded systems. The applicability of using low resolution cameras equipped on PDAs or cellular phones for palmprint capturing has been studied in [20]. We implement all the four methods on an embedded platform equipped with an Intel XScale-PXA255 400 MHz embedded processor, 64 MB SDRAM and 32 MB Flash memory. For conve-
Fig. 10. One to one verification results. (a) Genuine and imposter matching score distributions. (b) The receiver operating characteristic curve.
349
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351 Table 3 Comparisons of computational and space complexity.
2D SAX Competitive Code FusionCode PalmCode
Table 5 Identification accuracy of the proposed method.
Computational complexity
Template size (bytes)
82K Addition + 1K Division + 4K Comparison 120,000K MADD (Addition + Multiplication) 160,000K MADD (Addition + Multiplication) 40,000K MADD (Addition + Multiplication)
256
Rank
1 (%)
2 (%)
3 (%)
4 (%)
5 (%)
Accuracy
99.90
99.97
99.97
99.99
100
384 256 256
nience, the preprocessing module is reused in all the four methods. A fixed-point arithmetic system [33] is adopted to optimize the implementations. Altogether 120 PolyU palmprints are used for the speed testing, and Table 4 lists the average time consumption per palmprint for different methods. On average, the feature extraction process of our method can be finished in less than 90 ms, which is much faster than the other three methods. This is not surprising considering the extremely low computational complexity of the proposed method. The total time for processing a palmprint image, which is 1689 ms, is also comparable to the best known result reported in [20]. It can be noticed that in the proposed method, the time consumption (1605 ms) of the preprocessing is significant compared to the feature extraction time. However, in a real system, this problem might be effectively solved. For example, we have observed that the curvature maxima finding process consumes more than 70% of the total preprocessing time. In a real system, such as the one used for building the PolyU database [30], the prior knowledge about the positions of the pegs, can be directly utilized to narrow down the search area from the whole palm to the finger valleys only, leading to a considerable reduction of the preprocessing time. The template matching process of the proposed method can also be implemented efficiently. According to Eq. (3), the value of MINDIST is calculated using table lookup, a very fast operation for computers. The average time for one 2D SAX template matching is 1.2 ms on the embedded platform described above, indicating a capability of identifying more than 800 users per second. On a desktop PC equipped with a Pentium D 3.4 GHz processor, the identification capacity is no less than 12700 users per second. We also perform the identification experiment on the PolyU database. For each palm, we choose two palmprints, one from each session, to form the identification gallery. To ensure the reproducibility, we simply select palmprints with indexes equal to 1 (NN = 1) as gallery images [28]. For each palm, the average image of the two selected palmprints is converted to 2D SAX representation, which is used as the master template. The remaining 7752386*2 = 6980 palmprints form the probe set. Each palmprint in the probe set is identified against the 386 master templates in the gallery. The identification accuracy is shown in Table 5. Among all the 6980 identification attempts, there are only 7 failure cases at rank one. And all palmprints can be correctly identified at rank five.
To further test the reliability of the proposed method, a complete one to one matching experiment is also performed on the CASIA palmprint database [29]. There are altogether 19740 genuine matching and 13701201 imposter matching. Without loosing generality, we only compare the verification accuracy of the proposed method to that of the Competitive Code approach. Considering the observation that the palmprint images in the CASIA database suffers more from palm movements and distortions, we extend the range of the vertical and the horizontal translations in the template matching to 3 to 3 for both the proposed method and the Competitive Code method. All the other parameters are kept unchanged. The verification accuracies of the two methods are listed in Table 6, from which we can find that the CASIA database is more challenging than the PolyU database in terms of the palmprint verification. The EER degrades significantly for both methods. In general, the proposed method is comparable to the Competitive Code method even for small FAR scenarios. Observation on the experimental data indicates that most failure cases of both methods are caused by the out-of-plane palm movement, the nonlinear palm distortion, and the image blurring. Both the proposed method and the Competitive Code method adopt rigid template matching, therefore they have similar tolerance towards out-of-plane palm movement and the nonlinear palm distortion. Further analysis on the experimental results shows that comparing to the Competitive Code method, the proposed method demonstrates higher robustness towards image blurring. This is justifiable considering that the Competitive Code method assumes that each pixel in the palmprint lies on a palm line with certain direction, which determines the value of the Competitive Code at this pixel [18]. Nevertheless, when the palmprint image is blurred, some relatively weak palm lines are obscured so that the extraction of the Competitive Code of those pixels becomes sensitive to noises and illumination changes. This will eventually degrades the discriminative power of the extracted Competitive Code. While in the proposed method, the blurring effect can be somehow tolerated by the PAA conversion, which can be considered as a low pass filtering process. We use four genuine matching cases as examples to further illustrate this phenomenon. In Fig. 11, palmprints in the same column were captured from the same palm. However, certain variation can be observed between each two palmprints in the same column. In Fig. 11(a), a projective distortion can be observed between two palmprints probably because of a palm rotation around the wrist. Comparing to Fig. 11(b1), the palmprint in Fig. 11(b2) was captured when the palm is closer to the camera. Nonlinear distortion due to palm
Table 6 Verification accuracies on the CASIA database. False acceptance rate
Table 4 Comparisons of time consumptions on the mobile embedded platform.
7
Time consumption per palmprint image (ms)
2D SAX Competitive Code FusionCode PalmCode
Preprocessing
Feature extraction
Total
1605
84 9086 11296 2991
1689 10691 12901 4596
10 106 105 104 103 102 101 Equal Error Rate
False rejection rate 2D SAX (%)
Competitive Code (%)
25.9 16.6 6.4 3.4 2.0 0.8 0.2
33.9 18.2 3.6 2.0 1.5 0.9 0.5
0.9
0.9
350
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
Fig. 11. Authentication failure cases in the CASIA database.
Table 7 Effectiveness ( FRR(T) ) of different methods on difficult genuine matching cases.
Competitive Code 2D SAX
(a1) vs (a2) (%)
(b1) vs (b2) (%)
(c1) vs (c2) (%)
(d1) vs (d2) (%)
0.9
0.3
0.5
0.6
0.6
0.2
0.6
2.6
relaxation can be observed in Fig. 11(c2). The image blurring in Fig. 11(d) is probably caused by the relatively low illumination. Suppose that the score of a genuine matching case is T. Then the value of FRR(T) can be used to evaluate the effectiveness of a authentication method on this matching case. The higher the FRR(T) is, the more robust the method is on this specific matching case. As is shown in Table 6, the Competitive Code method and our proposed method have similar FRR distributions. Therefore, we adopt the value of FRR(T) directly for comparing the effectiveness of the two methods on the four difficult genuine matching cases shown in Fig. 11. The results are listed in Table 7. The values of FRR(T) of the two methods are very close for the first three matching cases: (a1) vs (a2), (b1) vs (b2) and (c1) vs (c2). As we have discussed, this reflects the similar tolerance towards out-of-plane palm movement and the nonlinear palm distortion of the two methods. However, for the image blurring case (d1) vs (d2), the proposed method demonstrates higher robustness than the Competitive Code method. This blurring robust property of our method may help to explain the relative performance advantage of our method on the CASIA database. Generally speaking, the palmprint images in the PolyU database are sharper than the images in the CASIA database. We estimated the sharpness of the palmprints in the two databases using the ‘Tenengrad’ function, which is calculated based on the magnitude of the image gradient [34]. Larger ‘Tenengrad’ function values generally indicate higher image sharpness. This function is widely adopted in auto focusing systems in digital imaging devices. For the PolyU database, the mean sharpness for all the 7752 palmprints is 0.181 and the standard deviation is 0.055. While for the CASIA database, the mean sharpness for all the 5239 palmprint is only 0.033 and the standard deviation is 0.012. 5. Conclusions The SAX conversion has been widely used in solving data mining problems for 1D time series because it is computationally effi-
cient, is easy to use, and is able to achieve a satisfactory balance between dimensionality reduction and discriminative power retaining. In this paper, we propose a natural extension of the SAX representation, 2D SAX, for two-dimensional data such as 2D images. We apply this new representation to the problem of texture based palmprint authentication for testing its effectiveness. Compared to previous palmprint authentication approach, our method mainly has two advantages. Firstly, it is simple to implement and the computational complexity of feature extraction and template matching is much lower than most previous methods, so that it can be efficiently implemented for slow mobile embedded systems. Also, experimental data show that our method is relatively more robust to image blurring. This property is probably useful for mobile palmprint authentication system where images are captured using low resolution/quality embedded cameras and the motion blur are usually inevitable due to the difficulty in user palm fixation. Secondly, our method does not require any training of parameters so that its performance does not rely on the selection of the training dataset and can be easily reproduced. Experimental results show that in term of the authentication accuracy, our method over performs most previous texture based methods, and is comparable to the state of the art palm line bases methods. In the PolyU database, which contains 7752 palmprints, our method can achieve an EER of 0.3% for a one to one verification experiment, and a rank one identification accuracy of 99.90%. On the more challenging CASIA database, an EER of 0.9% can be achieved. It should be emphasized that our method does not impose any assumptions on the palmprint texture and only the grayscale information is used. This indicates that our method can actually be applied to any other texture based image recognition problems. In addition, our method provides a lot of flexibility for practical applications. By choosing different values for SAX_Length and SAX_Level, different level of dimensionality and numerosity reduction can be achieved. Introducing the filtering bank G also provides the possibility of utilizing more complex texture features. This might be a possible way for further improving the accuracy of our method. Of course, as a trade off, the computational complexity will usually increase when more complicated filters are used. For example, if the G is a Gabor filter and SAX_Level is set to 2, the generated palmprint template in our method is identical to the PalmCode. What is more, it is quite probable that some effective 1D SAX data mining techniques [22], such as motif finding, indexing, abnormity locating and classification, can also be extended to 2D SAX, leading to many potential applications in image data mining. All these are worthy of our future efforts.
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
Acknowledgments The work described in this paper was substantially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. 415205). This work was also partially supported by a grant from Tsinghua University, China (Project No. 053207002). Portions of the research in this paper use the CASIA Palmprint Image Database collected by the Chinese Academy of Sciences’ Institute of Automation (CASIA). References [1] D. Maltoni, D. Maio, A.K. Jain, S. Prabhakar, Handbook of Fingerprint Recognition, Springer, 2003. [2] S.Z. Li, A.K. Jain, Handbook of Face Recognition, Springer, 2004. [3] J. Daugman, Biometric Personal Identification System Based on Iris Analysis, U.S. Patent No. 5,291,560, 1994. [4] D. Zhang, Palmprint Authentication, Norwell, Mass, Kluwer Academic Publishers, 2004. [5] R. Sanchez-Reillo, C. Sanchez-Avila, A. Gonzalez-Marcos, Biometric identification through hand geometry measurements, IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (10) (2000) 1168–1171. [6] C.C. Han, A hand-based personal authentication using a coarse-to-fine strategy, Image and Vision Computing 22 (11) (2004) 909–918. [7] D. Zhang, A. Kong, J. You, M. Wong, Online palmprint identification, IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (9) (2003) 1041– 1050. [8] X.Q. Wu, K.Q. Wang, D. Zhang, Wavelet based palmprint recognition, Proceedings of the International Conference on Machine Learning and Cybernetics 3 (2002) 1253–1257. [9] W. Li, J. You, D. Zhang, Texture-based palmprint retrieval using a layered search scheme for personal identification, IEEE Transactions on Multimedia 7 (5) (2005) 891–898. [10] Z. Sun, T. Tan, Y. Wang, S.Z. Li, Ordinal palmprint representation for personal identification, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 1 (2005) 279–284. [11] A. Kong, D. Zhang, Feature-level fusion for effective palmprint authentication, Proceedings of International Conference on Biometric Authentication (2004) 761–767. [12] A. Kong, D. Zhang, M. Kamel, Palmprint identification using feature-level fusion, Pattern Recognition 39 (2006) 478–487. [13] X. Pan, Q.Q. Ruan, Palmprint recognition with improved two-dimensional locality preserving projections, Image and Vision Computing 26 (9) (2008) 1261–1268. [14] W. Li, D. Zhang, Z. Xu, Palmprint identification by Fourier transform, International Journal on Pattern Recognition and Artificial Intelligence 16 (4) (2002) 417–432. [15] C.H. Chen, P.S.P. Wang, Handbook of Pattern Recognition and Computer Vision, World Scientific Publishing, 2005.
351
[16] D. Zhang, W. Shu, Two novel characteristics in palmprint verification: datum point invariance and line feature matching, Pattern Recognition 32 (1999) 691–702. [17] C.C. Han, H.L. Cheng, K.C. Fan, C.L. Lin, Personal authentication using palmprint features, Pattern Recognition 36 (2) (2003) 371–381. [18] A. Kong, D. Zhang, Competitive coding scheme for palmprint verification, Proceedings of International Conference on Pattern Recognition 1 (2004) 23– 26. [19] Y. Han, T. Tan, Z. Sun, Palmprint recognition based on directional features and graph matching, Proceedings of International Conference on Biometrics (2007) 1164–1173. [20] Y. Han, T. Tan, Z. Sun, Y. Hao, Embedded palmprint recognition system on mobile devices, Proceedings of International Conference on Biometrics (2007) 1184–1193. [21] J. Lin, E. Keogh, S. Lonardi, B. Chiu, A symbolic representation of time series, with implications for streaming algorithms, Proceedings of the ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery (2003) 2–11. [22] E. Keogh’s SAX Homepage:. [23] E. Keogh, K. Chakrabarti, M. Pazzani, S. Mehrotra, Locally adaptive dimensionality reduction for indexing large time series databases, Proceedings of ACM SIGMOD Conference on Management of Data (2001) 151–162. [24] Q. Wang, D. Kontos, L. Guo, V. Megalooikonomou, Application of time series techniques to data mining and analysis of spatial patterns in 3D images, Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing 3 (2004) 525–528. [25] J.S. Chen, Y.S. Moon, H.W. Yeung, Palmprint authentication using time series, Proceedigs of International Conference on Audio- and Video-Based Biometric Person Authentication (2005) 376–385. [26] C.S. Myers, L.R. Rabiner, A comparative study of several dynamic time-warping algorithms for connected word recognition, The Bell System Technical Journal 60 (7) (1981) 1389–1409. [27] L. Bergroth, H. Hakonen, T. Raita, A survey of longest common subsequence algorithms, Proceedings of International Symposium on String Processing and Information Retrieval (2000) 39–48. [28] PolyU Palmprint Database. Available from: . [30] D. Zhang et al., Apparatus for Capturing a Palmprint Image, United States Patent Application No. 10/253, September 2002. [31] A. Rosenfeld, A.C. Kak, Digital Picture Processing, Academic Press, San Diego, 1982. [32] M.H. Han, D. Jang, The use of maximum curvature points for the recognition of partially occluded objects, Pattern Recognition 23 (1990) 21–23. [33] K.C. Chan, Y.S. Moon, P.S. Cheng, Fast fingerprint verification using sub-regions of fingerprint images, IEEE Transactions on Circuits and Systems for Video Technology 14 (1) (2004) 95–101. [34] J.M. Tenenbaum, Accommodation in Computer Vision, Ph.D. Thesis, Stanford University, November 1970.
Contents lists available at ScienceDirect
Image and Vision Computing journal homepage: www.elsevier.com/locate/imavis
Palmprint authentication using a symbolic representation of images Jiansheng Chen a,*, Yiu-Sang Moon b, Ming-Fai Wong c, Guangda Su a a b c
Department of Electronic Engineering, Tsinghua University, Haidian, Beijing 100084, PR China Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong Department of Computer Science, University of Toronto, 10 King’s College Road, Toronto, Canada M5S 3G4
a r t i c l e
i n f o
Article history: Received 4 December 2007 Received in revised form 30 March 2009 Accepted 2 June 2009
Keywords: Biometrics Palmprint authentication Symbolic representation Time series
a b s t r a c t A new branch of biometrics, palmprint authentication, has attracted increasing amount of attention because palmprints are abundant of line features so that low resolution images can be used. In this paper, we propose a new texture based approach for palmprint feature extraction, template representation and matching. An extension of the SAX (Symbolic Aggregate approXimation), a time series technology, to 2D data is the key to make this new approach effective, simple, flexible and reliable. Experiments show that by adopting the simple feature of grayscale information only, this approach can achieve an equal error rate of 0.3%, and a rank one identification accuracy of 99.9% on a 7752 palmprint public database. This new approach has very low computational complexity so that it can be efficiently implemented on slow mobile embedded platforms. The proposed approach does not rely on any parameter training process and therefore is fully reproducible. What is more, besides the palmprint authentication, the proposed 2D extension of SAX may also be applied to other problems of pattern recognition and data mining for 2D images. Ó 2009 Elsevier B.V. All rights reserved.
1. Introduction Automated personal authentication using biometric features has been widely studied during the last two decades. Previous research efforts have made it possible to apply biometric systems to practical applications for security or commercial purposes. Biometric systems based on fingerprint recognition [1], face recognition [2], and iris recognition [3] have already been developed to a quite mature stage so that they can be applied to critical security applications such as the immigration control and the crime investigation. Recently, a novel hand-based biometric feature, palmprint, has attracted an increasing amount of attention. Like any other biometric identifiers, palmprints are believed to have the critical properties of universality, uniqueness, permanence and collectability for personal authentication [4]. Palmprints have several advantages over other hand-based biometrics, such as fingerprint and hand geometry. Compared to fingertips, palms are larger in size and therefore are more robust to injuries and dirt. Also, low-resolution imaging can be employed in the palmprint recognition based on creases and palm lines, making it possible to perform real time preprocessing and feature extraction; and the cost of the capturing device can also be well controlled. Palmprint authentication is believed to be able to achieve the accuracy comparable to that of * Corresponding author. Tel.: +852 31634251; fax: +852 26035024. E-mail address: [email protected] (J. Chen). 0262-8856/$ - see front matter Ó 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2009.06.004
other hand-based biometric authentication technologies, including fingerprint [1] and hand geometry [5,6]. Texture and palm lines are the most clearly observable palmprint features in low resolution (such as 100 dpi) images [7], and thus have attracted most research efforts. In texture based palmprint authentication approaches [7–14], signal processing based texture analysis methods [15] are usually adopted. Typically, texture features are extracted by filtering the palmprint images using filters such as the Gabor filter [7,11–13], the ordinal filter [10], or the wavelet [8]. The image filtering may be performed in either the spatial domain or the frequency domain [14]. Recently, a lot of automated palmprint authentication methods [16–19] have focused on the palm line features, since they are more appealing than the texture for the human vision. In the offline method proposed in [16], the geometric shapes of the palm lines are extracted and approximated by straight-line segments. The slope, intercept and inclination of each segment are used as features for palmprint matching. C.C. Han et al. investigate the magnitude of palm lines in palmprint matching [17]. The latest related research reveals that the orientations of palm lines also contain strong discriminative power. Based on palm line orientations, a Competitive Code is designed for palmprint representation in [18]; and Y. Han et al. use local orientation histograms for describing palmprints [19]. Similar to the texture based methods; the palm line based methods usually employ image filtering for line feature extraction, leading to a high computational complexity. For example, in the Competitive Code method [18], six Gabor filters are
344
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
applied to each palmprint ROI (Region Of Interest) for generating the corresponding orientation map. Suppose that the Palmprint ROI is 128 128 pixels and the Gabor filters are 35 35 in size, the overall MADD (Multiplication + Addition) operations required for one palmprint is around 120 million, leading to a very long processing time especially on slow mobile platforms. Experiments show that extracting the Competitive Code for one palmprint takes more than eight seconds on a state of the art PDA [20]. This is far too slow for a real-time biometric system. Besides the computational complexity, selecting appropriate filter parameters is also nontrivial in filtering based palmprint authentication methods. It has been demonstrated in [4,7] that the authentication accuracy varies a lot when using different Gabor filter parameters, which need to be tuned in a try and error manner, indicating that the authentication performance will depend a lot on the training set used for parameter selection. This may account for the significant performance variations of different filtering based palmprint authentication methods on different databases [19]. In this paper, we propose a texture based approach for palmprint authentication, in which palmprint image grayscale information are directly adopted as features. The computational complexity of the feature extraction process is much lower than previous filtering based approaches, and thus can be implemented efficiently for even slow mobile embedded platforms. By extending the idea of SAX (Symbolic Aggregate approXimation) [21] in time series research to 2D images for palmprint representation and matching, the proposed method can achieve the authentication performance, in terms of EER (Equal Error Rate), comparable to the state of the art palmprint authentication methods. The rest of this paper is organized as the follows. Section 2 explains the 2D extension of SAX for images. Section 3 describes the feature extraction and matching processes of the proposed approach. Experiments and results are elaborated in Section 4. The last section is a conclusion of our work. 2. 2D SAX As a symbolic representation of sequential data, SAX (Symbolic Aggregate approXimation) [21] has been verified as a simple but effective tool for solving most time series data mining problems such as clustering, classification, indexing, anomaly detection, and motif finding [22]. For a real valued data sequence, its SAX representation is obtained by first transforming it into the PAA (Piecewise Aggregate Approximation) representation; then predetermined breakpoints are used to discretize the PAA coefficients into SAX symbols. Basically, SAX represents a real valued data sequence using a string of discrete symbols or characters, as is shown in Fig. 1. Dramatic dimensionality reduction can be achieved by converting a time series into its SAX representation. In order to preserve the discriminative power of the data as complete as possible, Keogh et al. proposed a discretization technique which can produce symbols with equiprobability by using discretization breakpoints following a Gaussian distribution [21]. This specific
Table 1 The lookup table for calculating dist() function when SAX_Level equals four [21].
‘a’ ‘b’ ‘c’ ‘d’
‘a’
‘b’
‘c’
‘d’
0 0 0.67 1.34
0 0 0 0.67
0.67 0 0 0
1.34 0.67 0 0
property of SAX has made it suitable for the feature representation in biometric authentication where the discriminative power of the representation is crucial. From Fig. 1, we can see that the degree of dimensionality reduction is decided by two parameters: the length of the resultant symbol string, or the SAX_Length; and the number of symbols used, or the SAX_Level. As an example, the SAX_Length equals eight and the SAX_Level equals four in Fig. 1. The smaller these two parameters are, the more intensive the dimensionality reduction is. A distance measurement called MINDIST, which strictly lower bounds the corresponding Euclidean distance of the original time series [23], is employed to measure the similarity between two SAX b ¼q ^2 ; . . . ; q ^w and ^1 ; q strings [21]. Given two SAX strings Q b ¼ ^c1 ; ^c2 ; . . . ; ^cw , their MINDIST can be calculated using Eq. (1), C where n is the length of the original time series, and dist() can be implemented using table lookup as is illustrated in Table 1 [21].
b; C bÞ ¼ MINDISTð Q
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiu w X nu t ^i ; ^ci Þ2 distðq w i¼1
ð1Þ
For high dimensional data, such as a 2D or 3D image, one feasible way of applying the SAX conversion is to firstly decompose it into a 1D sequence. For example, 3D fMRI (functional Magnetic Resonance Imaging) images are decomposed into 1D sequences using the Hilbert space filling curve for clustering and motif finding in [24]. In our previous work [25], palmprint ROIs are decomposed using spiral curves before SAX conversion. Nevertheless, the neighborhood information, which plays an important role in similarity measurement for high dimensional data, will inevitably be lost during the 1D decomposition no matter what space filling curve is used. For instance, in a 2D image, each pixel, except those on the boundary, has eight eight-neighbor pixels. However, after decomposition, each pixel will only have two neighbors in the resultant 1D sequence. Such a phenomenon is illustrated in Fig. 2, where (I) and (II) shows the up-left corners of two images. Without losing generality, pixels are represented by cells inside which the values stand for the grayscale level in the hexadecimal format. SI and SII ðS0I and S0II Þ are the 1D decomposition results using progressive scan (Hilbert curve scan). It is not difficult to observe that in their 2D forms, image (I) and image (II) are probably identical except for a one pixel shift in both the vertical and the horizontal directions. This apparent similarity, however, cannot be easily discovered from their corresponding 1D sequences. The fundamental reason for this problem is the serious loss of neighbor-
Fig. 1. Conversion of a time series to its SAX representation. (a) The original real valued time series. (b) PAA representation. (c) SAX representation.
345
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
real valued 2D data using a 2D matrix of symbols. Assume that the size of the input data Q is m n. First, Q is normalized to have a zero mean and a unit standard deviation. Then, similar to the PAA representation, Q can be represented by a matrix Q of the size w1 w2 . The ðith ; jth Þ element of Q can be calculated by Eq. (2). Simply stated, the original data Q is divided into w1 w2 equal size blocks, and the mean value of the data inside each block is calculated to form the dimensionality reduced representation: Q . m
1 Qði; jÞ ¼ w1 w2
hood information. In Fig. 2, ‘5’ and ‘A’, which are neighbors in both (I) and (II), are no longer adjacent in any of the 1D sequences; and they are separated in such a irregular way that existing time series alignment techniques, such as sequence shifting, DTW (Dynamic Time Warping) alignment [26] and LCSS (Longest Common Subsequence) alignment [27], cannot be consistently applied to recover the lost neighborhood information. This phenomenon accounts for the unsatisfactory verification accuracy we have achieved in our previous attempt of applying SAX to palmprint authentication [25]. To facilitate a more effective SAX conversion for 2D data, we propose the 2D SAX, an extension of the SAX, which represents a
n
j
w2 X
Q ðx; yÞ
ð2Þ
x¼wm ði1Þþ1 y¼wn ðj1Þþ1 1
Fig. 2. 1D decomposition of 2D images using different space filling curves.
i
w1 X
2
Finally, breakpoints identical to those used in 1D SAX converb , or the 2D sion are applied to convert Q into a symbol matrix Q SAX representation. The definitions and properties of these breakpoints can be found in [21]. These breakpoints are selected to equally divide the area under the probability distribution function curve of a standard normal distribution due to the Gaussianity nature of real life data [21]. Fig. 3 illustrates the process of a sample 2D SAX conversion. In particular, the 2D SAX_Length is now defined as w1 w2 ; and the definition of the 2D SAX_Level is identical to that of the 1D case. The total operations required for such a conversion is no more than mn additions, plus mn=w1 w2 divisions, plus ðSAX LevelÞ mn=w1 w2 comparisons. Also, the MINDIST for 2D SAX is now redefined as Eq. (3). Actually, such a definition can be further extended to 3D or even higher dimensionality.
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiffiffiffiffiffiffiffiffiuX w2 u w1 X mn b; C bÞ ¼ t ^i;j ; ^ci;j Þ2 MINDISTð Q distðq w1 w2 i¼1 j¼1
ð3Þ
The 2D SAX conversion can be directly applied to images, which are special cases of two-dimensional data. For color images, the 2D SAX conversion can be applied to each color channel, respectively,
Fig. 3. Conversion of a 2D data into its 2D SAX representation. (a) The original real valued 2D data of size 64 64. (b) The block wise average representation and the 2D SAX conversion. (c) The resultant 2D SAX representation.
Fig. 4. Conversion of a 2D image into its 2D SAX representation. (a) The original noisy image. (b) Noise suppression using Gaussian filtering. (c) The resultant 2D SAX representation. (d) Blowup of the lower-left corner of image (c).
346
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
and the MINDIST can simply be defined as the summation of the MINDIST values of all the channels. In this paper, only grayscale images are considered as the palmprint images [28,29] to be used are all intensity images. For a 2D image I, image filtering can be applied prior to the 2D SAX conversion. Suppose G is a digital filter, we denote the filtering process as Eq. (4). This enhances the flexibility of the 2D SAX representation, since image filtering is a fundamental tool in most image processing techniques. Fig. 4 illustrates the process of converting a noisy Lenna into its 32 32 2D SAX representation, in which the SAX_Level equals four. The original image occupies 262K bytes and the 2D SAX only needs 256 bytes to store. Tremendous dimensionality and numerosity reduction is thus achieved.
Q ¼IG
ð4Þ
3. 2D SAX conversion for palmprints Two public palmprint databases are used for the experiments in this work [28,29]. The PolyU Palmprint Database [28] contains 7752 grayscale palmprint images (384 284 pixels, 96 dpi) corresponding to 386 different palms. For each palm, there are around 20 samples collected in two sessions, where around 10 samples were captured in the first session and the second session, respectively. The average interval between the two sessions was 2 months. Palmprints in this database were captured using a patented palmprint capturing device [30] in which pegs are used to limit the movement and the distortion of the user’s palm. The illumination of the devices is tuned to facilitate the capture of palm
line details. As a result, palmprints in the PolyU Palmprint Database are believed to have relatively high image qualities. The other database used is the CASIA Palmprint Image Database [29] which contains 5239 grayscale palmprint images (640 480 pixels, 72 dpi) collected from 301 individuals. All palmprints were collected in one single session. Comparing to the PolyU capturing device, the device used for constructing the CASIA database imposes less physical constrains on the user’s palm, leading to more obvious palm movements, distortions and blurring in the captured palmprint images. Fig. 5 compares sample palmprint images from the two databases. Fig. 5(c) and (d) belong to the CASIA database and are captured from the same palm. However, intra-class variation caused by shifting and distortion can be easily observed. Moreover, Fig. 5(d) is obviously blurred probably because of the palm motion or the camera defocusing. Before the 2D SAX conversion, the input palmprint images should be aligned to a predefined universal coordinate to facilitate consistent ROI extraction. We adopt a palm coordinate system similar to the second square-based palm coordinate system proposed in [4]. The following steps are performed for the palmprint alignment. A sample palmprint form the PolyU database is used to illustrate this procedure in Fig. 6. (1) A fixed threshold is applied to convert the original image into a binary image (Fig. 6b). Isolated pixels, spurs and leaks are removed using morphological operations. (2) The palm boundary (Fig. 6c) is obtained using a boundary tracking algorithm [31].
Fig. 5. Sample palmprint images. Image (a) and image (b) belong to the PolyU database. They are captured from the same palm. Image (c) and image (d) belong to the CASIA database. They are also captured from the same palm.
Fig. 6. Palmprint image preprocessing: alignment and ROI extraction.
347
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
(3) The maximum curvature points P1 and P2 between the fingers (Fig. 6d) are located using a curvature maxima finding algorithm [32]. (4) P1 and P2 are connected by a line L1. Draw another line L2 passing through the middle point P3 of L1 perpendicularly (Fig. 6e). A point P4 is found on L2 so that the length between P3 and P4 equals a predefined value. (5) A square area S (Fig. 6f/g) of fixed size (135 135 pixels) is extracted with P4 as its center. S is defined as the ROI for 2D SAX conversion. (6) Non-uniform illumination correction is performed to enhance the palmprint texture (Fig. 6h). Fig. 7 shows the ROI extraction result for a palmprint from the CASIA database. The CASIA palmprints are first resized to 380 284 pixels before the ROI extraction. The extracted palmprint ROI are then converted to the 2D SAX form for palmprint representation and matching. Several sample PolyU palmprint ROIs are shown in Fig. 8, in which each two ROIs in the same column are from the same palm; and the ROIs in the first and second rows are from session one and session two, respectively. Visually, the ROIs are extracted consistently. An average filter G is employed to filter the extracted ROIs so as to suppress noises caused by factors such as dirt, injuries and camera noises. To fulfill our objective of low computational complexity, the filter G is decomposed into two 1D average filters, so that the filtering process can be equivalently implemented as the combination of two 1D filtering processes as is illustrated in Eqs. (5) and (6). And the 1D average filtering can be efficiently implemented such that each pixel requires only one addition plus one subtraction. As a result, the overall complexity of the average filtering process is around 4N, where N is the total number of pixels in the image. The 128 128 pixels central part of the filtered ROI is cropped
for the 2D SAX conversion. Fig. 9 shows the corresponding 2D SAX representations for the palmprint ROIs shown in Fig. 8, where w1 and w2 equal 32 and the SAX_Level is set to 4. These 2D SAX representations are used as palmprint templates. And the storage size of each template is 256 bytes.
2 3 1 617 7 61 1 ... 1 7 6 7 6 G¼6 7 ¼ 6 .. 7 7 ½1 1 ... 1 ¼ U V 4... ... ... ...5 6 4. 5 1 1 ... 1 1
ð5Þ
I G ¼ I ðU VÞ ¼ ðI UÞ V
ð6Þ
2
1
1
... 1
3
The MINDIST between two templates is adopted as the similarity measurement of the corresponding two palmprints. The smaller the MINDIST is, the more similar the two palmprints are. As human palm is a particularly flexible organ, distortions are unavoidable during palmprint capture, leading to the fact that no alignment scheme is able to extract the ROI perfectly [4]. We need to vertically and horizontally translate one of the templates to find the best match, or the smallest MINDIST value. The definition of the MINDIST after translation is identical to Eq. (3), except that the values of w1 and w2 should be modified according to the size of the overlapping template area. The range of the vertical and the horizontal translations are defined from 2 to 2. In the next section, we will show the discriminative power of the SAX representation and the MINDIST for the palmprint authentication through verification and identification experiments. 4. Experiments A complete one to one matching experiment is performed on the PolyU palmprint database [28] to test the effectiveness of the proposed method. There are altogether 74068 genuine matching
Fig. 7. A CASIA palmprint and the corresponding ROI extraction.
Fig. 8. Sample palmprint ROIs.
348
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
Fig. 9. 2D SAX representations for the sample palmprint ROIs in Fig. 8.
and 29968808 imposter matching. The experimental settings are as follows: G is a 7 7 average filter; SAX_Length = 32 32 = 1024, and SAX_Level equals 4. Fig. 10 shows the score distributions of the genuine matching and the imposter matching, as well as the ROC (Receiver Operating Characteristic) curve. The EER of the proposed method is 0.3% with the corresponding threshold at 46.2. We compare the performance of the proposed method to that of three widely referenced palmprint authentication approaches: PalmCode [7], FusionCode [11,12] and Competitive Code [18]. The comparison results are shown in Table 2. To avoid any uncertainty in the implementation, the accuracies of the three reference approaches are quoted from the original papers. The verification results for the PalmCode [7], the FusionCode [11], and the Competitive Code [18] were also generated from one to one matching among 7752 palmprints in the PolyU database. This ensures the fairness of the performance comparison. It can be observed from Table 2 that in general the proposed method over performs the FusionCode and the PalmCode. In terms of EER, our method is comparable to the Competitive Code method. It is true that for small FAR values, the Competitive Code method performs better. However, the outstanding performance of our methods for relatively large FAR values can be regarded as a trade off. Also, our method does not rely on any training of the parameters, and has very low computational complexity and requires relatively small storage size. Table 3 lists the estimated computational complexity of the different feature extraction methods. We only focus on the feature extraction process since the
Table 2 Verification accuracies of different methods on the PolyU database. False acceptance rate
106 105 104 103 102 101 Equal error rate
False rejection rate 2D SAX (%)
Competitive Code (%)
FusionCode (%)
PalmCode (%)
5.9 4.1 1.9 0.5 0.1 <0.01
1.1 0.9 0.6 0.5 0.3 0.2
6.7 4.0 2.0 1.1 0.4 0.2
8.5 5.6 3.0 1.4 0.5 0.1
0.3%
0.56%
0.6%
0.3%
palmprint preprocessing stage, which contains palmprint alignment and ROI cropping, is common to all the four methods. We assume that the palmprint ROI is 128 128 pixels and the Gabor filters used by the PlamCode, FusionCode and Competitive Code are of the size 35 35 [4]. The complexity of the proposed method is orders of magnitude lower than the other three methods, leading to the possibility of real time implementations for slow mobile embedded systems. The applicability of using low resolution cameras equipped on PDAs or cellular phones for palmprint capturing has been studied in [20]. We implement all the four methods on an embedded platform equipped with an Intel XScale-PXA255 400 MHz embedded processor, 64 MB SDRAM and 32 MB Flash memory. For conve-
Fig. 10. One to one verification results. (a) Genuine and imposter matching score distributions. (b) The receiver operating characteristic curve.
349
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351 Table 3 Comparisons of computational and space complexity.
2D SAX Competitive Code FusionCode PalmCode
Table 5 Identification accuracy of the proposed method.
Computational complexity
Template size (bytes)
82K Addition + 1K Division + 4K Comparison 120,000K MADD (Addition + Multiplication) 160,000K MADD (Addition + Multiplication) 40,000K MADD (Addition + Multiplication)
256
Rank
1 (%)
2 (%)
3 (%)
4 (%)
5 (%)
Accuracy
99.90
99.97
99.97
99.99
100
384 256 256
nience, the preprocessing module is reused in all the four methods. A fixed-point arithmetic system [33] is adopted to optimize the implementations. Altogether 120 PolyU palmprints are used for the speed testing, and Table 4 lists the average time consumption per palmprint for different methods. On average, the feature extraction process of our method can be finished in less than 90 ms, which is much faster than the other three methods. This is not surprising considering the extremely low computational complexity of the proposed method. The total time for processing a palmprint image, which is 1689 ms, is also comparable to the best known result reported in [20]. It can be noticed that in the proposed method, the time consumption (1605 ms) of the preprocessing is significant compared to the feature extraction time. However, in a real system, this problem might be effectively solved. For example, we have observed that the curvature maxima finding process consumes more than 70% of the total preprocessing time. In a real system, such as the one used for building the PolyU database [30], the prior knowledge about the positions of the pegs, can be directly utilized to narrow down the search area from the whole palm to the finger valleys only, leading to a considerable reduction of the preprocessing time. The template matching process of the proposed method can also be implemented efficiently. According to Eq. (3), the value of MINDIST is calculated using table lookup, a very fast operation for computers. The average time for one 2D SAX template matching is 1.2 ms on the embedded platform described above, indicating a capability of identifying more than 800 users per second. On a desktop PC equipped with a Pentium D 3.4 GHz processor, the identification capacity is no less than 12700 users per second. We also perform the identification experiment on the PolyU database. For each palm, we choose two palmprints, one from each session, to form the identification gallery. To ensure the reproducibility, we simply select palmprints with indexes equal to 1 (NN = 1) as gallery images [28]. For each palm, the average image of the two selected palmprints is converted to 2D SAX representation, which is used as the master template. The remaining 7752386*2 = 6980 palmprints form the probe set. Each palmprint in the probe set is identified against the 386 master templates in the gallery. The identification accuracy is shown in Table 5. Among all the 6980 identification attempts, there are only 7 failure cases at rank one. And all palmprints can be correctly identified at rank five.
To further test the reliability of the proposed method, a complete one to one matching experiment is also performed on the CASIA palmprint database [29]. There are altogether 19740 genuine matching and 13701201 imposter matching. Without loosing generality, we only compare the verification accuracy of the proposed method to that of the Competitive Code approach. Considering the observation that the palmprint images in the CASIA database suffers more from palm movements and distortions, we extend the range of the vertical and the horizontal translations in the template matching to 3 to 3 for both the proposed method and the Competitive Code method. All the other parameters are kept unchanged. The verification accuracies of the two methods are listed in Table 6, from which we can find that the CASIA database is more challenging than the PolyU database in terms of the palmprint verification. The EER degrades significantly for both methods. In general, the proposed method is comparable to the Competitive Code method even for small FAR scenarios. Observation on the experimental data indicates that most failure cases of both methods are caused by the out-of-plane palm movement, the nonlinear palm distortion, and the image blurring. Both the proposed method and the Competitive Code method adopt rigid template matching, therefore they have similar tolerance towards out-of-plane palm movement and the nonlinear palm distortion. Further analysis on the experimental results shows that comparing to the Competitive Code method, the proposed method demonstrates higher robustness towards image blurring. This is justifiable considering that the Competitive Code method assumes that each pixel in the palmprint lies on a palm line with certain direction, which determines the value of the Competitive Code at this pixel [18]. Nevertheless, when the palmprint image is blurred, some relatively weak palm lines are obscured so that the extraction of the Competitive Code of those pixels becomes sensitive to noises and illumination changes. This will eventually degrades the discriminative power of the extracted Competitive Code. While in the proposed method, the blurring effect can be somehow tolerated by the PAA conversion, which can be considered as a low pass filtering process. We use four genuine matching cases as examples to further illustrate this phenomenon. In Fig. 11, palmprints in the same column were captured from the same palm. However, certain variation can be observed between each two palmprints in the same column. In Fig. 11(a), a projective distortion can be observed between two palmprints probably because of a palm rotation around the wrist. Comparing to Fig. 11(b1), the palmprint in Fig. 11(b2) was captured when the palm is closer to the camera. Nonlinear distortion due to palm
Table 6 Verification accuracies on the CASIA database. False acceptance rate
Table 4 Comparisons of time consumptions on the mobile embedded platform.
7
Time consumption per palmprint image (ms)
2D SAX Competitive Code FusionCode PalmCode
Preprocessing
Feature extraction
Total
1605
84 9086 11296 2991
1689 10691 12901 4596
10 106 105 104 103 102 101 Equal Error Rate
False rejection rate 2D SAX (%)
Competitive Code (%)
25.9 16.6 6.4 3.4 2.0 0.8 0.2
33.9 18.2 3.6 2.0 1.5 0.9 0.5
0.9
0.9
350
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
Fig. 11. Authentication failure cases in the CASIA database.
Table 7 Effectiveness ( FRR(T) ) of different methods on difficult genuine matching cases.
Competitive Code 2D SAX
(a1) vs (a2) (%)
(b1) vs (b2) (%)
(c1) vs (c2) (%)
(d1) vs (d2) (%)
0.9
0.3
0.5
0.6
0.6
0.2
0.6
2.6
relaxation can be observed in Fig. 11(c2). The image blurring in Fig. 11(d) is probably caused by the relatively low illumination. Suppose that the score of a genuine matching case is T. Then the value of FRR(T) can be used to evaluate the effectiveness of a authentication method on this matching case. The higher the FRR(T) is, the more robust the method is on this specific matching case. As is shown in Table 6, the Competitive Code method and our proposed method have similar FRR distributions. Therefore, we adopt the value of FRR(T) directly for comparing the effectiveness of the two methods on the four difficult genuine matching cases shown in Fig. 11. The results are listed in Table 7. The values of FRR(T) of the two methods are very close for the first three matching cases: (a1) vs (a2), (b1) vs (b2) and (c1) vs (c2). As we have discussed, this reflects the similar tolerance towards out-of-plane palm movement and the nonlinear palm distortion of the two methods. However, for the image blurring case (d1) vs (d2), the proposed method demonstrates higher robustness than the Competitive Code method. This blurring robust property of our method may help to explain the relative performance advantage of our method on the CASIA database. Generally speaking, the palmprint images in the PolyU database are sharper than the images in the CASIA database. We estimated the sharpness of the palmprints in the two databases using the ‘Tenengrad’ function, which is calculated based on the magnitude of the image gradient [34]. Larger ‘Tenengrad’ function values generally indicate higher image sharpness. This function is widely adopted in auto focusing systems in digital imaging devices. For the PolyU database, the mean sharpness for all the 7752 palmprints is 0.181 and the standard deviation is 0.055. While for the CASIA database, the mean sharpness for all the 5239 palmprint is only 0.033 and the standard deviation is 0.012. 5. Conclusions The SAX conversion has been widely used in solving data mining problems for 1D time series because it is computationally effi-
cient, is easy to use, and is able to achieve a satisfactory balance between dimensionality reduction and discriminative power retaining. In this paper, we propose a natural extension of the SAX representation, 2D SAX, for two-dimensional data such as 2D images. We apply this new representation to the problem of texture based palmprint authentication for testing its effectiveness. Compared to previous palmprint authentication approach, our method mainly has two advantages. Firstly, it is simple to implement and the computational complexity of feature extraction and template matching is much lower than most previous methods, so that it can be efficiently implemented for slow mobile embedded systems. Also, experimental data show that our method is relatively more robust to image blurring. This property is probably useful for mobile palmprint authentication system where images are captured using low resolution/quality embedded cameras and the motion blur are usually inevitable due to the difficulty in user palm fixation. Secondly, our method does not require any training of parameters so that its performance does not rely on the selection of the training dataset and can be easily reproduced. Experimental results show that in term of the authentication accuracy, our method over performs most previous texture based methods, and is comparable to the state of the art palm line bases methods. In the PolyU database, which contains 7752 palmprints, our method can achieve an EER of 0.3% for a one to one verification experiment, and a rank one identification accuracy of 99.90%. On the more challenging CASIA database, an EER of 0.9% can be achieved. It should be emphasized that our method does not impose any assumptions on the palmprint texture and only the grayscale information is used. This indicates that our method can actually be applied to any other texture based image recognition problems. In addition, our method provides a lot of flexibility for practical applications. By choosing different values for SAX_Length and SAX_Level, different level of dimensionality and numerosity reduction can be achieved. Introducing the filtering bank G also provides the possibility of utilizing more complex texture features. This might be a possible way for further improving the accuracy of our method. Of course, as a trade off, the computational complexity will usually increase when more complicated filters are used. For example, if the G is a Gabor filter and SAX_Level is set to 2, the generated palmprint template in our method is identical to the PalmCode. What is more, it is quite probable that some effective 1D SAX data mining techniques [22], such as motif finding, indexing, abnormity locating and classification, can also be extended to 2D SAX, leading to many potential applications in image data mining. All these are worthy of our future efforts.
J. Chen et al. / Image and Vision Computing 28 (2010) 343–351
Acknowledgments The work described in this paper was substantially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. 415205). This work was also partially supported by a grant from Tsinghua University, China (Project No. 053207002). Portions of the research in this paper use the CASIA Palmprint Image Database collected by the Chinese Academy of Sciences’ Institute of Automation (CASIA). References [1] D. Maltoni, D. Maio, A.K. Jain, S. Prabhakar, Handbook of Fingerprint Recognition, Springer, 2003. [2] S.Z. Li, A.K. Jain, Handbook of Face Recognition, Springer, 2004. [3] J. Daugman, Biometric Personal Identification System Based on Iris Analysis, U.S. Patent No. 5,291,560, 1994. [4] D. Zhang, Palmprint Authentication, Norwell, Mass, Kluwer Academic Publishers, 2004. [5] R. Sanchez-Reillo, C. Sanchez-Avila, A. Gonzalez-Marcos, Biometric identification through hand geometry measurements, IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (10) (2000) 1168–1171. [6] C.C. Han, A hand-based personal authentication using a coarse-to-fine strategy, Image and Vision Computing 22 (11) (2004) 909–918. [7] D. Zhang, A. Kong, J. You, M. Wong, Online palmprint identification, IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (9) (2003) 1041– 1050. [8] X.Q. Wu, K.Q. Wang, D. Zhang, Wavelet based palmprint recognition, Proceedings of the International Conference on Machine Learning and Cybernetics 3 (2002) 1253–1257. [9] W. Li, J. You, D. Zhang, Texture-based palmprint retrieval using a layered search scheme for personal identification, IEEE Transactions on Multimedia 7 (5) (2005) 891–898. [10] Z. Sun, T. Tan, Y. Wang, S.Z. Li, Ordinal palmprint representation for personal identification, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 1 (2005) 279–284. [11] A. Kong, D. Zhang, Feature-level fusion for effective palmprint authentication, Proceedings of International Conference on Biometric Authentication (2004) 761–767. [12] A. Kong, D. Zhang, M. Kamel, Palmprint identification using feature-level fusion, Pattern Recognition 39 (2006) 478–487. [13] X. Pan, Q.Q. Ruan, Palmprint recognition with improved two-dimensional locality preserving projections, Image and Vision Computing 26 (9) (2008) 1261–1268. [14] W. Li, D. Zhang, Z. Xu, Palmprint identification by Fourier transform, International Journal on Pattern Recognition and Artificial Intelligence 16 (4) (2002) 417–432. [15] C.H. Chen, P.S.P. Wang, Handbook of Pattern Recognition and Computer Vision, World Scientific Publishing, 2005.
351
[16] D. Zhang, W. Shu, Two novel characteristics in palmprint verification: datum point invariance and line feature matching, Pattern Recognition 32 (1999) 691–702. [17] C.C. Han, H.L. Cheng, K.C. Fan, C.L. Lin, Personal authentication using palmprint features, Pattern Recognition 36 (2) (2003) 371–381. [18] A. Kong, D. Zhang, Competitive coding scheme for palmprint verification, Proceedings of International Conference on Pattern Recognition 1 (2004) 23– 26. [19] Y. Han, T. Tan, Z. Sun, Palmprint recognition based on directional features and graph matching, Proceedings of International Conference on Biometrics (2007) 1164–1173. [20] Y. Han, T. Tan, Z. Sun, Y. Hao, Embedded palmprint recognition system on mobile devices, Proceedings of International Conference on Biometrics (2007) 1184–1193. [21] J. Lin, E. Keogh, S. Lonardi, B. Chiu, A symbolic representation of time series, with implications for streaming algorithms, Proceedings of the ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery (2003) 2–11. [22] E. Keogh’s SAX Homepage: