identification based on hands natural layout

Available online at www.sciencedirect.com

Image and Vision Computing 26 (2008) 451–465 www.elsevier.com/locate/imavis

Biometric verification/identification based on hands natural layout Miguel Ada´n *, Antonio Ada´n, Andre´s S. Va´zquez, Roberto Torres Escuela Superior de Informa´tica (UCLM), Paseo de la Universidad, 4, 13071 Ciudad Real, Spain Received 21 September 2005; received in revised form 16 February 2007; accepted 9 August 2007

Abstract In this paper, a hand biometric system for verification and recognition purposes is presented. The method is based on three keys. Firstly, the system is based on using a Natural Reference System (NRS) defined on the hand’s natural layout. Consequently, neither hand-pose training nor a pre-fixed position is required in the registration process. Secondly, the hand’s features are obtained through the polar representation of the hand’s contour. This implies minimum image processing and low computational cost. Thirdly, instead of common methods that use one hand, we use right and left hands. This allows us to consider distance measures for direct (R/R, L/L) and crossed (R/L, L/R) hands obtaining improvements in FAR/FRR and identification ratios. The paper shows details about the experimentation and presents the results of the method applied on 5640 images belonging to 470 users. The results are good enough to consider this biometric system for future security/control applications.  2007 Elsevier B.V. All rights reserved. Keywords: Biometric systems; Security; Hand geometry; Invariant features; Similarity

Contents 1. 2. 3. 4.

5.

Hand-based biometrics review and comparison. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Natural reference system and feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Similarity measure and verification process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Experimentation of the method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1. Prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. Verification test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3. Recognition test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4. Performance of the normalized similarity measure using typical geometry features . . . . . . . . . . . . . . . . . . . . . . . . 4.5. Comparison with other methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusions and future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

*

452 453 454 457 457 457 459 461 461 464 465 465

Corresponding author. E-mail addresses: [email protected] (M. Ada´n), [email protected] (A. Ada´n), [email protected] (A.S. Va´zquez), Roberto. [email protected] (R. Torres). 0262-8856/$ - see front matter  2007 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2007.08.010

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

452

hand geometry [1,2,16–18], palmprints [3–7], finger geometry [8,11,18,21], hand contours [9,16,14], knuckleprint [19], 3D reconstructions [13], etc. Registration is usually taken from devices with pegs and mirrors which control the hand position and obtain the up side of the hand [1,10–12,15]. In general, the main drawbacks of the hand-based biometrics are:

1. Hand-based biometrics review and comparison Hand-based biometric technologies are getting very popular for control purposes, such as access to buildings, airports, nuclear plants and stadiums. They are suitable for massive use because of their low processing time and real time response. Unlike biometric systems based on fingerprints and iris pattern, users are not reluctant to hand biometric ones. Most features related to a human hand are relatively invariant and peculiar, although not unique, to each individual. That is why these systems are commercially used more for verification than for recognition. These systems make use of only one hand, from which the features are obtained by different methods, such as

(a) The required upkeep due to damage, spoiling and dirtiness of the platform and mirrors. (b) The required training and cooperation of the individuals to place the hand in specific positions. (c) The requirements imposed on the users for removing their rings, bracelets, clothes, etc.

Table 1 Hand-based biometrics review Reference

1

2

3

4

5

6

7

8

Sa´nchez-Reillo et al. [1]

Hand and fingers geometry

Feature vector, #25

M

Yes

CCD

V/I

#20

¨ den et al. [2] O

Hand and fingers geometry Palmprint

Feature vector, #16

M

No

CCD

V/I

Template, image, 32 · 32

H

No

Scanner

384 bytes image

H H

No No

image

H

472 bytes

Hang et al. [3]

9

10

11

#200

FAR < 10

97

#28

#840

– FAR = 1

95

V/I

#50

#1500

No

FAR < 9

91

CCD Print + scanner

V V

#193 #20

#7752 ?

Yes No

FRR < 9 FAR = 1 ?

– –

No

Print + scanner

I

#100

#200

No

–

95

S

Yes

CCD

V

#206

#1030

No

–

Zhang et al. [4] Zhang and Shu [5] You et al. [6]

Palmprint Palmprint

Joshi et al. [8]

Middle/ring geometry

Template, Template, 400 · 400 Template, 232 · 232 Template,

Jain and Duta [9]

Hand’s contour

Template, #120 to 350

S

Yes

Scanner

V

#53

#353

No

FAR = 0.1 FRR = 0.1 FAR = 2.0

Jain and Ross [11] Han [12]

Hand and fingers geometry Hand/fingers geometry + palmprint

Feature vector, #16

M

Yes

CCD

V

#50

#500

?

FRR = 3.5 ?

–

Feature vector, #10 Template #500

H

Yes

CCD

V

#50

#1500

No

FAR = 3.7

–

Gonza´lez et al. [16]

Hand/fingers geometry + hand’s contour Hand/fingers geometry + palmprint

Feature vector #17 Template: #50 to 512 Feature vector #22 Template: # ?

M

No

Scanner

I

#50

#500

No

FRR = 5.3 –

97

M

No

Scanner

V

#50

#600

No

FAR = 1.0

–

Wong and Shi [18]

Fingers geometry

Feature vector #16 Template: # 50 to 80

S

No

Scanner

V

#22

#288

No

FRR = 1.3 FAR = 2.2

–

Li et al. [19] Kumar et al. [20]

Knuckprint Hand/fingers geometry + palmprint

Template: #100 Feature vector #(144 + 16)

H H

No No

CCD CCD

I V

#73 #100

#1423 #1000

No No

FRR = 11 – FAR = 5.08

97.6 –

Xiong et al. [21]

Fingers geometry

Feature vector # (37 for each finger)

M

No

CCD

V

#108

#540

?

FRR = 2.25 FAR = 2.41

–

Ada´n

Left/right hand’s contours

Feature vector #(14 + 14)

S

No

2 CCD

V/I

#470

#5640

Yes

FRR = 2.41 FAR = 1.3

97.6

Ong et al. [17]

Palmprint

–

FRR = 1.3 (1) Kind of technique, (2) features size, (3) hard/middle/soft (H/M/S) image processing, (4) pegs or special demands, (5) device, (6) verification/ identification, (7) number of users tested, (8) image database, (9) off-line/on-line system, (10) FAR/FRR (%), (11) identification rate (%).

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

(d) The heavy image processing required to extract hand features and the slowness of the system. It is really difficult to make a comparison between the different techniques published in the last years. Depending on each specific application and environment, several factors should be taken into account before choosing the most appropriate. In Table 1, we summarize essential characteristics of the latest systems and methods which have appeared in journals and conferences. In order to compare our system too, we have introduced it in the last row. In the cases where the authors did not introduce explicit numerical estimations or details concerning these parameters, the symbol ‘‘?’’ appears. In other cases, we had to simplify the results computing average values (i.e. FAR and FRR) and we also deduced or estimated some characteristic based on the information included in each paper (i.e. on-line/off-line system and H/M/S image processing). In this table, it can be seen that Joshi’s method yields excellent results even better than those reported in this paper. This method uses the line profiles of the ring and middle fingers which are the plots of the grey value along a line in the image. To obtain such good results a strict positioning control and illumination stability must be necessary. In order that the grey values do not change from one sample to another, no variation about the texture of the hands should also be mandatory in this experiment. This implies controlling several aspects such as short time between samples as well as temperature stability. Therefore, the implementation and application of this technique seem to be very complex. As we will present in Section 5, we have tested this technique under our experimental conditions and we have obtained the worst results. After analysing Table 1 we can state point by point the following comments concerning our technique compared to the others: (1) Original technique. Our technique uses both hands. As far as we are concerned, systems that make use of right and left hands do not exist. On the other hand, it is a new handbased biometric system based on the hand’s natural layout. The main goal of this idea is to increase the disparity between hands using the Natural Reference System (see Section 2). (2) The features are obtained on the polar representation of the hand’s contour, whereas the rest of the methods define them on the image. In spite of using two hands, a low number of features (14 + 14) are considered. (3) Easy and soft image processing. We just have to extract the contour of the hand. Consequently, we avoid hard image processing (i.e. segmentation) and highly reduce the computational cost. Besides, we think that this system is more robust than systems which are very

453

dependent on the image processing results. These circumstances may compensate for the cost of using two hands instead of one. (4) It is pegs-free and neither a pre-fixed pose nor training phase is required by the system. The individuals just have to extend their hands completely. Furthermore people do not have to take anything off because the system ignores the lower part of the hand. (5) We use two CCD cameras. So the cost of the system is higher than in the mono-camera cases. (6) Versatility: the method can be used for both verification and identification purposes. (7 and 8) The number of users is by far the highest (470) and the database is higher than the majority. Moreover the experimentation was not carried out in a lab, as for most cases, but in a real environment (secondary school). Both circumstances give more consistency to our method. (9) The system can be considered ‘on-line’ for attendance control applications in environments with a medium time request (1–3 seg). Except Zhang and Shu [5] the rest of the authors do not provide speed or time information. In Table 1, we have assumed that scanner devices are off-line systems. (10 and 11) Good performance. FAR/FRR and identification ratios are good enough compared to the rest of the works. In [14], we gave a brief description of our prototype. In this paper, we present a reviewed and detailed version which includes experimentation for recognition purposes. In Section 2, we deal with feature extraction, whereas in Section 3 we define the similarity measure through direct and crossed hands comparisons. Sections 4.1–4.4 are devoted to presenting the prototype and to show in the verification and recognition results. Section 4.5 shows an experimental comparison with other methods. Finally, Section 5 presents conclusions, further work and improvements. 2. Natural reference system and feature extraction In this work, we have assumed that when the hand is completely extended (tensed) the position of the fingers is invariant and, therefore, it is a personal property. Taking into account the experimental results presented in Section 4, we can claim that this hypothesis has been corroborated. Consequently, we have defined the hand Natural Reference System (NRS) taking into account the middle and thumb positions. Fig. 2(a) illustrates five different samples for several individuals in our database. Note that, the relative layout of the fingers remains invariant where curious and particular positions for couples of fingers can be seen.

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

454

a

after studying the correlation on a wider set of features in which features with high correlation have been rejected. Thus, we have taken the minimum number of features avoiding redundant information. The set S splits in four subsets as follows:

b O’ x0

X

O 4

3

X’

2 1

r

Y 5

y0

r

X’’ O’’

Y’ 0

0

00

ð2Þ

S 2 ¼ flj : l5 ¼ P 5 P 6 ; l6 ¼ P 7 P 8 ; l7 ¼ P 9 P 10 ; l8 ¼ P 11 P 13 g

ð3Þ

S 3 ¼ flj : l9 ¼ P 1 P 13 ; l10 ¼ P 1 P 4 ; l11 ¼ P 4 P 11 g ð4Þ S 4 ¼ flj : l12 ¼ \ðP 3 ;O00;P 2 ;l13 ¼ \ðP 2 ;O00;P 1 ;l14 ¼ \ðP 4 ;O00;P 3 g

Y’’ 0

S 1 ¼ flj : l1 ¼ P 1 P 11 ; l2 ¼ P 2 P 12 ; l3 ¼ P 3 P 12 ; l4 ¼ P 4 P 13 g

ð5Þ

00

Fig. 1. (a) {O, X, Y} and r; (b) {O , X , Y } and {O , X , Y00 } on the hand.

Let {O, X, Y} be the image coordinate system O being the centre of the image and assuming an image containing an extended hand D. Let 1, 2, 3, 4, and 5 be the labels of pinkie, ring finger, middle finger, index finger, and thumb respectively. The Natural Reference System of D is defined after the following steps (see Fig. 1(a) and (b)): (I) find the straight line r fitted to the skeleton of 3. (II) Set {O 0 , X 0 , Y 0 } with O 0 = O, Y 0 being parallel to r and passing by O 0 and X 0 being normal to Y 0 by O 0 . (III) Find P on 5 in the system {O 0 , X 0 , Y 0 }, P = (x0, y0), x0 = Min{x 0 : (x 0 , y 0 ) 2 D}. (IV) Move {O 0 , X 0 , Y 0 } to O00 = (0, y0) obtaining NRS {O00 , X00 , Y00 }. Using the natural hand layout has two advantages: neither hand-pose training nor pre-fixed position is required and it increases the dissimilarity between hands. Thus, even though two hands look apparently similar, they become very distinct in their respective NRSs. In this case, the new images translated to a common coordinate system would be very distant as well. Fig. 2(b) shows two hands that have similar geometric features where the origins for both reference systems are marked. Below, a superimposed representation verifies that when both hands are compared taking their own respective NRS the dissimilarity is evident. So, similar hands could turn into dissimilar hands. Taking into account NRS the contour of the hand is obtained and a part of it is used to define the feature vector. Let I = {Pi+1, Pi+2, . . . , Pi+n} be a series of pixels belonging to the contour of the hand D referenced to its NRS, where P i ¼ ðxi ; 0Þ;

P iþn ¼ ðxiþn ; 0Þ;

xi ¼ Minfx : ðx; yÞ 2 Dg;

xiþn ¼ Maxfx : ðx; yÞ 2 D; y ¼ 0g

ð1Þ

The series I goes on fingers 2, 3, and 4, and partially on fingers 1 and 5 (see Fig. 2(c)). In order to handle a normalized representation, I is normalized to a fixed number of pixel obtaining In (in our case n = 1000). Note that In is not ambiguous at all in the image and it is invariant to transformations (rotation and translation) as well. Finally, the polar representation of In yields the radius function f in which we have marked the following keypoints: Relative Maximum Points: {P1, P2, PP 3, P4}, Middle n

f ðP iþj Þ

Points: {P5, P6, . . . ,P10}, (where f ðP k Þ ¼ j¼1n ;k ¼ 5; 6; . . . ; 10) and Relative Minimum Points: {P11, P12, P13}. After fixing the set of key-points we define 14 features S = {l1, l2, . . . ,l14}. Feature vector S was established

Fig. 3(a) shows the position of the key-points on f, whereas Fig. 3(b) plots their location on the hand’s contour In. It is important to remark that this location does not coincide with typical hand geometrical points, i.e. finger crest, maximum curvature points, etc. For instance P1 is a Relative Maximum Point for f but it does not have to coincide with the curvature maximum point of the index finger. Consequently, the set S do not have an explicit geometrical meaning and do not correspond to typical geometrical measurements (i.e. finger and phalange widths). Fig. 3(c) illustrates the features inside the hand. 3. Similarity measure and verification process The similarity measure is based on a set of distances between right hands (R/R), left hands (L/L), right–left hands (R/L), and left–right hands (L/R). Let M, M 0 , M, M 0 2 {R, L} the hands belonging to two individuals. Taking into account the feature vector S = {l1, l2, . . . ,l14}, 14 distances di, i = 1, 2 . . . , 14 are defined as: d i ðM; M 0 Þ ¼ jli ðMÞ  li ðM 0 Þj;

i ¼ 1; 2; . . . ; 14

ð6Þ

Thus, the distance between two hands can be expressed as the distances vector ~ dðM; M 0 Þ ¼ fd 1 ; d 2 ; . . . d 14 g. Fig. 4 shows the correlation map of ~ dðM; M 0 Þ for right and left hands. This analysis has been carried out on 5640 images that have been registered in our experimental prototype. As we expected, there are several pairs of features that present medium and high correlation values. For instance in the case {l1, l2}, the correlation coefficient is 0.84 but this is a logical result because a hand that has a big index finger has a big middle finger as well. Anyway the coefficients are not high enough to ensure that a perfect functional relation exists between any pair of features. Table 2 presents more results of the correlation analysis. In this case, mean correlation values for the sets: S, S1, S2, S3, S4, and {S1, S2, S3} are shown. In general, a verification process compares the input sample with the hand template (prototype) stored in a database. In our case, we store a reduced number of the latest samples taken of each individual (in our case 5 samples) as prototypes and use all of them for comparison to a new input. If after comparing, there is at least one success, the verification result is evaluated as positive. This approach involves increasing the computation cost but the effectiveness of the system

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

improves. In practice, we have found that this strategy yields the best results. Our biometric system stores a hands database B consisting of h samples R/L for r individuals. B ¼ fðRjq ; Ljq Þ; j ¼ 1; . . . ; h; q ¼ 1; . . . ; rg. Suppose that an input sample Rq/Lq belonging to the qth individual is introduced for verification. The method considers all possible comparison pairs fðRq ; Rjk Þ; ðRq ; Ljk Þ; ðLq ; Rjk Þ; ðLq ; Ljk Þ; j ¼ 1; . . . ; h; k ¼

455

1; . . . ; rg. For each pair, which we will call (Mq,Mk), M 2 {R, L}, the Normalized Similarity Measure is defined as: Gq;k  meank fGq;k g ; GðM q ; M k Þ ¼ stdk fGq;k g 14 X Ai;q;k  Bi;q Gq;k ¼ ; ð7Þ C i;q  Bi;q i¼1

Fig. 2. (a) Invariance of the natural position of the fingers. (b) Reference systems defined in hands apparently similar. (c) Corresponding contours plotted on a common coordinate system.

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

456

Fig. 2 (continued)

Fig. 3. (a) Location of the key-points in f. (b) Visualization of the probable location of the points on the contour IN. (c) Visualization of features S1, S2, S3, and S4.

Table 2 Mean correlation values for different features sets

S S1 S2 S3 S4 {S1, S2, S3}

qR

qL

0.3754 0.8151 0.7225 0.7575 0.3348 0.6091

0.3600 0.8093 0.7538 0.7583 0.3740 0.5992

where Gq,k is the sum of 14 normalized values between [0, 1], Ai;q;k ¼ minj fd i ðM q ; M jk Þg (i-th minimum distance for the individual k) and Bi,q = mink {Ai,q,k}, Ci, q = maxk{Ai,q,k} (ith minimum and maximum distances for database B). The minor G(Mq, Mk) is the most similar two hands are. After comparing two hands, the verification decision depends on whether the similarity measures pass or not a specific threshold l which is empirically pre-fixed. There-

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

14

0.9 0.8

12

0.7 0.6

10

457

14 0.8 12 0.6

10

0.5 8

0.4 0.3

6

0.2 0.1

4

8

0.4

6

0.2

4

0

0 2

-0.1 2

4

6

8

10

12

14

2 2

4

6

8

10

12

14

-0.2

Fig. 4. Mean correlation maps of ~ dðM; M 0 Þ for right and left hands.

fore, the threshold value depends on the database and must be computed and updated when it changes. The next phase labels each matching (Mq, Mk)as suitable (1) or unsuitable (0) as follows:  1 if GðM q ; M k Þ < l pl ðM q ; M k Þ ¼ ð8Þ 0 if GðM q ; M k Þ P l: Finally, the verification decision (acceptance or rejection) or the hypothesis ‘‘the sample q belongs to the individual k’’ is established if at least one matching (Mq, Mk) is suitable. Formally: X pl ðM q ; M k Þ P 1: ð9Þ M q  M k () M2fR;LÞ

Next, we argue how this strategy makes a fusion between left and right information. In our context, biometric may have three levels for information fusion: (i) fusion at representation level, where features of left and right hands are concatenated, (ii) fusion at decision level, where the decision scores of left and right hand are combined to generate the final decision, (iii) fusion at abstract level, where a multiple decision considering left and right hands is computed. In our case, features of the left and right hands are concatenated for computing the expressions of the Normalized Similarity Measure (Eq. (7)). Note that in this, M 2 {R, L} and therefore cross NSMs – G(Rq, Lk) and G(Lq, Rk) – must also be computed. On the other hand, we apply fusion at decision level because the final verification decision (Eqs. (8) and (9)) is based on decision for couples {R, R}, {R, L}, {L, R}, {L, L} and consequently, we combine them. 4. Experimentation of the method 4.1. Prototype A mobile experimental prototype with 2 CCD cameras has been built. The acquired images have a resolution of 640*480 pixels. The hands are placed onto a robust platform that includes its own illumination system with diffuse light. This system allows the users to place their hands in a comfortable way on the top of the glass platform situated above the cameras (Fig. 5(a) and (b)). Each

camera’s field of view takes up around 65% of the platform’s area so that a small part of the other hand may appear in each image (see detail in Fig. 5(c)). The image processing consists of binarization, labelling of connected components and edge extraction of the biggest component. Neither hand-pose training nor pre-fixed position is required in the registration process. Anyway we asked people to follow this simple protocol: (1) extend the hands completely (tensed) (2) put them on the glass, (3) do not move them while the capture is made. The tension is required to avoid movements between fingers and therefore, changes in the angles between fingers, especially between the thumb and forefinger. So, there are no special requirements for the users and they do not have to worry about watches, clothes, bracelets, etc., because the system works with the upper part of the hand. 4.2. Verification test In order to prove the performance of the system we have carried out a verification test on 470 individuals corresponding to men and women from 15 to 18 years old. This experiment was developed in a secondary school with attendance control objectives. Therefore, the test was accomplished on a homogeneous sampling where sometimes teenagers were not motivated enough to make the register phase conveniently. Note that when an experimental test is accomplished on a set of individuals with similar characteristics (age and job) the verification becomes more difficult and the results are worse than in the case of taking a set of heterogeneous people. This circumstance is frequently absent in most of the publications. In this sense, we can say that our experimentation has a difficulty added. We collected the images after a three months interval where the registers of the same subject were collected during one week in two/three sessions. So far 6000 samples have been recorded, taking 6–8 acquisitions per user and for both hands. We have taken all possible five-sample combination as database and the remaining as testsamples. So, for each combination, the database B and the test-samples (or input samples) B 0 = {(Rq, Lq):q = 1,

458

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

Fig. 5. (a) Experimental setup. (b) Hands natural layout. (c) Software tool developed.

2, . . . ,470} are established. A total of 2820 verification tests have been carried out. The verification process is accomplished to each qth sample of B 0 . As a result of that, we obtain a set of potential users H(q) which verify the hypothesis (in other words, it verifies Mk ” Mq). Obviously the best result is achieved when Cardinal(H(q)) = 1. False Acceptance Ratio (FAR) and False Rejection Ratio (FRR) parameters are formally defined as follows:

P

k CardðFAk Þ

 100; rðr  1Þ P k CardðFRk Þ FRR ¼  100 r FAR ¼

ð10Þ

where sets FAk and FRk are computed following these rules: If q 2 HðkÞ; q 6¼ k ) q 2 FAk ;if k 62 HðkÞ ) k 2 FRk :

ð11Þ

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

459

Table 3 Verification results Test register

1st

2nd

3rd

4th

5th

6th

Mean

SD

FAR FRR l

3.192 1.702 1.475

0.730 1.914 1.6

0.846 0.425 1.6

0.912 1.063 1.6

0.869 1.063 1.6

1.274 1.489 1.58

1.304 1.276

0.943 0.538

As we said before, each one of the six samples taken per user has been tried as a query sample. Table 3 shows the verification results. Note that the best values arise when the third acquisition is taken as a query sample. In this case, we obtain ratios FAR = 0.84698%, FRR = 0.42553% (l = 1.600) and the cross-over point FA = FR = 0.7406 % with l = 1.6125. The average value for all cases is FAR = 1.3044 %, FRR = 1.2766% and the average crossover point is FA = FR = 1.426% for l = 1.56. Curiously, when the test-sample corresponds to the first and last registration the results are quite worse. This may be explained as follows. At first registration the users do not have any experience and probably move or do not extend their hands completely. On the other hand, after several registers and sessions, the teenager’s cooperation decreases. At this point, it is important to remark that in real performance conditions, where verification is carried out daily, the system updates the database B as verification is made. Thus for each subject, a FIFO structure contains the feature vectors corresponding to its five last successful verifications. On the other hand, if the system is used for practical – non-experimental-attendance control a good user cooperation is expected. A performance analysis of the method for each kind of matching R/R, L/R, R/L, L/L, is illustrated in Fig. 6(a) for the best case. In this, FRR and FAR curves have been plotted for all kind of matching. Black curves (3) correspond to our method, magenta and red curves (2) correspond to R/R and L/L matching and finally green and blue curves (1)

a

correspond to R/L and L/R matching. Looking at this figure it is evident that our method (3) based on multiple comparisons obtains better results than the methods based on single comparisons (1 and 2). Fig. 6(b) shows the Receiver Operating Characteristic (ROC) curves for previous cases (1)–(3). ROC curve represents FAR versus 100% – FRR. It can be seen that our method yields the best results. If low FAR is required, the system can work at FAR = 0.4486% and 96.5956% genuine rejection rate (l = 1.65), whereas if low FRR is required, the system can work at 99.3971% genuine rejection rate and FAR = 3.6303% (l = 1.45). In conclusion, we can say that these results are comparable to previous approaches [1–4,9,12,17,18,21]. 4.3. Recognition test We have carried out a recognition test on B 0 = {(Rq, Lq): q = 1, 2, . . . ,470}. Now, for each user q 2 B 0 , we find the best candidate after computing the Normalized Similarity Measure G(Mq, Mk) for each kind of association R/R, R/ L, L/L, L/R. Table 4 presents the recognition ratios from 1 to 10 best candidates and for different recognition strategies, whereas Fig. 7(a) plots their corresponding curves, X-axis being the number of candidates considered. The best result appears when we consider the mean Normalized Similarity Measure (NSM) for R/R and L/L cases (row (g)). Rows (a), (b), (c), and (d) show details of the identification percentages for each isolated association R/R, R/L,

b

100

10

80

(1)

(1) R/L, L/R (2) R/R, L/L (3) Multiple Matching

2

(1) R/L, L/R (2) R/R, L/L (3) Multiple matching (3)

(2)

100-FRR

FAR/FRR

(2)

60

FRR (3)

40 FAR

(1) (3)

(1,2)

20

0 -2

-1.5

-1 Threshold

-0.5

10

-2

10

-1

10 FAR

Fig. 6. (a) FAR and FRR curves. Comparison simple/matching strategies. (b) ROC curves.

0

10

1

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

460 Table 4 Identification results for several strategies Strategy

(a) (b) (c) (d) (e) (f) (g)

Candidates

R/L L/R R/R L/L Minimum Average R/R, L/L

1

1–2

1–3

1–4

1–5

1–6

1–7

1–8

1–9

1–10

27 29 92 90 91.8 94.8 97.6

38 38 96 94 96.3 97.8 98.7

44 44 97 96 97.4 98.7 99.0

49 49 98 97 98.3 99.0 99.3

52 53 98 97 98.7 99.3 99.5

55 56 98 97 99.0 99.4 99.5

59 58 98 98 99.1 99.5 99.5

61 60 98 98 99.2 99.5 99.6

64 62 98 98 99.2 99.5 99.6

65 64 99 98 99.3 99.5 99.6

Fig. 7. (a) Recognition percentages. (b) Place of true candidate inside of the list of candidates.

L/L, and L/R. Note that in these cases worse percentages are achieved especially when only the first candidate is taken. Row (e) corresponds to the strategy of minimum. In this, after computing NSM for all associations, we choose the association that yields the minimum NSM value. Finally, in the average strategy (row (f)) we use the average of the NSM for all associations. In Table 5, we analyse the performance of the system for the strategy (g) by taking each one of the six acquisitions as

input in the recognition test. Note that similar comments to the verification case can be made. The higher ratios are achieved for 2nd, 3rd, and 4th acquisitions and the average ratio is 97.58% which is also comparable to works [1– 3,6,16,19] referenced in Section 1. To complete the analysis of the recognition test, Fig. 7(b) marks the users that were not identified in the first place inside the list of candidates that the system provides. Although most users are well identified (first candidate),

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

461

Table 5 Identification results versus the input sample

Table 6 Verification results

Candidates

Verification

Method-1

Method-2

FAR (Mean) (%) FRR (Mean) (%)

1.304 1.276

1.36 2.58

1 1–2 1–3 1–4 1–5 1–6 1–7 1–8 1–9 1–10

Input sample 1st

2nd

3rd

4th

5th

6th

Mean

93.8 96.6 97.7 98.3 98.9 98.9 99.1 99.1 99.1 99.1

98.3 99.1 99.4 99.4 99.4 99.4 99.4 99.4 99.4 99.4

98.7 98.9 98.9 99.4 99.6 99.8 99.8 99.8 99.8 99.8

98.5 99.6 99.6 99.6 99.6 99.6 99.6 99.8 99.8 99.8

98.7 99.4 99.8 99.8 99.8 99.8 99.8 99.8 99.8 99.8

97.4 98.3 98.9 99.4 99.6 99.6 99.6 99.6 99.6 99.6

97.6 98.7 99.0 99.3 99.5 99.5 99.5 99.6 99.6 99.6

there are a few atypical cases that are not identified. Thus, individuals 54, 80, 244, 316, 354, and 453 were recognized in the places 5, 6, 4, 2, 11, and 4, respectively. These errors could be due to erroneous positions of the fingers or hands not completely extended. This prototype is currently implemented using MATLAB on a personal computer. Consequently, this version is still a bit slow for marketing purposes. The execution time for the pre-processing, feature extraction and verification are 0.92, 1.98, and 0.05 s. Therefore, the total execution time is currently about 3 s. As far as identification, the total time is around 14 s for our database of 470 individuals, which implies around 2.7 s for a database of 100 persons. 4.4. Performance of the normalized similarity measure using typical geometry features In order to complete the experimentation of our strategy, we present in this section how our verification algorithm works with features that do not depend on the hand natural layout and what are obtained on the hand image. On the other hand, we are encouraged to find out what the computational cost is in this case. We have considered a set of typical geometric features which require segmentation task on the hand image and we have run the verification and identification algorithms based on the Normalized Similarity Measure which was presented in Section 3. A set S 0 of 27 measurements was finally extracted from the image of the hand. Fig. 8 show the features extracted. Note that the hand and the fingers must be segmented to compute the hand size, the finger size and the finger length. Then, the crestpoints of the fingers must be calculated to extract the rest of the features.

Table 7 Identification results Identification

Method-1

Method-2

Candidates Candidates Candidates Candidates

97.6 99.0 99.5 99.6

95.0 98.0 98.7 99.1

1–1 (%) 1–3 (%) 1–6 (%) 1–10 (%)

Table 8 Computing times

Pre-processing time (s) Feature extraction time (s) Verification decision time (s) Identification decision time (s) Verification total time (s) Identification total time (s)

Method-1

Method-2

0.92 1.98 0.05 11.8 2.95 14.7

0.80 14.17 0.02 11.3 14.99 26.27

We have labelled ‘Method-1’ the method presented in this paper with the feature vector S (Eq. (2)–(5)) and ‘Method-2’ the same method but using the feature vector S 0 . Tables 6 and 7 show the performance results for verification and identification in both methods, whereas Table 8 gives the computing time corresponding to the principal steps of the process (preprocessing, feature extraction and verification/identification decision). Finally, the last two rows show the total time for verification and identification. As can be seen Method-2 yields worse verification and identification results than Method-1. Concerning the processing time, Method-1 is by far the fastest due mainly to the low image processing. In Section 5, we can state that the method based on Normalized Similarity Measure with S 0 can yield results comparable to ours but with much higher computational cost. Consequently, it offers a good performance/time balance.

4.5. Comparison with other methods In this section, we have carried out an experimental comparison between the proposed method and several

Fig. 8. Set S 0 of typical geometric features used for testing the performance of NSM.

462

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

methods referenced in Section 1. Of course, we cannot reproduce the systems, hand images and environments of other authors exactly. Nevertheless, we are encouraged to study the performance of different approaches under our experimental setup and conditions. However, the results of this work must be carefully analysed. In order to make a convincing comparison we selected those methods that we were able to reproduce with a guarantee. For this we imposed two requirements: to be able to extract all features of the method and to implement their original algorithms. Consequently, we rejected the methods in which the features are highly dependent on the own biometric system, on the hand position in it as well as those where the features are not clearly defined by the authors in their papers. Fig. 9 presents the whole set of features {F1, F2, . . . ,F32} and templates {v0, v1, . . . ,v8} chosen for test #A to #G. Features 1–5 correspond to lengths of the five fingers, F6–F17 are the widths of the fingers (except the thumb) at fold zones, F18, F19, and F20 are distances defined in the palm; F21–F26 correspond again to widths (except the thumb) at the middle point and at the one-five of the finger length, F27 is the radius of the maximum circle inside the hand, whereas F28 and F29 are the dimensions of the maximum rectangle inside the hand. Finally, F30, F31, and F32 are hand global dimensions (area, perimeter, and length). With regard to the templates v0 and v1, we compute the mean grey level of the profile along the middle and ring fingers. Template v2 corresponds to the pixels of the hand contour. Taking the centre of the circle defined for F27, templates v3 and v4 correspond to the vectors of distances and phases of the hand contour. For the index, middle and ring fingers, the top one-five portion contours correspond to the templates v5, v6, and v7. Finally, in template

v8 we store three widths {w1, w2, w3} per profile, 32 profiles per finger for index, middle, ring, and little fingers. We have carried out eight verification/identification strategies (tests #A to #G) corresponding to works of Jain et al., Xiong et al., Gonzalez et al., Kumar et al., Joshi et al., Wong et al., and we have also included our approach with identical conditions (test #H). Since this paper is mainly focused on presenting our system, we cannot present a detailed report of each one of the test. It is certain that this matter could be the topic of a second paper. For the moment a brief analysis of the results is presented. The same experiment was set for all methods including that which is proposed in this paper. The comparison test was accomplished on a new database of students 15–17 years old which was built during January 2007. A total of 205 individuals were registered taking three samples per student in two different sessions. After that, the verification and recognition was carried out for a single sample of each individual in a third session. For each test, from #A to #H, we computed ratios FAR, FRR, and ROC as well as the recognition ratio from 1 to 10 candidates. Table 9 shows the approaches tested as well as additional information concerning features, the kind of technique and algorithm used. As we expected the results obtained for all cases were worse than the original results included in Table 1. This is due to the fact that for all methods the original conditions have certainly changed in several aspects. Next we will analyse a set of key-points which must be taken into account in this experiment. Firstly, the size of the database, the number of hands in the training phase, the size of the templates and feature vectors for each case may vary with respect to the original works. A key-point is that this experience is carried out

Fig. 9. Key-points marked on the hand image (left-bottom) and definitions of the hand features and templates (left-above and right).

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

463

Table 9 Verification/identification strategies used Test

Method

Kind of technique

Features

Algorithm

#A #B

Jain and Duta [9] Joshi et al. [8]

Template v2: #500 Templates v0, v1: 220#

ICP between contours Cross-correlation for templates

#C

Xiong et al. [21]

Template v8: feature matrix: 4 · 32 · 3#

Euclidean distance

#D

Gonza´lez et al. [16]

Hand’s contour Middle and ring finger profiles: (mean grey level) Feature vectors for index, middle, ring and little finger Hands finger geometry

Feature vector: FD = [1, 2, 3, 4, 5, 18, 19, 20, 30, 31]

#E

Gonza´lez et al. [16]

#F

Kumar et al. [20]

Hands contour: (distance and phase from the origin) Hand/finger geometry

Nearby Neighbor Classifier (KNN) Nearby Neighbor Classifier (KNN) Normalized correlation

#G

Wong and Shi [18]

Finger geometry

#H

Ada´n

L/R hand’s contour

Templates v3, v4: #1000 Feature vector: FF = [2, 3, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 28, 29, 30, 32] Feature vector: FG = [1, 2, 3, 4, 5, 21, 22, 23, 24, 25, 26] Templates v5, v6 and v7: 100# Feature vector: S = {l1, l2, . . . ,l14} (Eqs. (2)–(5))

a

b

100

95

Threshold minimum distance classifier and ICP Normalized Similarity Measure (Eq. (7))

H E

A

9 cross-over FAR cross-over FRR

90

100-FRR

8

cross FAR/FRR

7 6 5

D

F

G C

85

80

B

4

75 3

70

2 1

65

0 A

Test FAR FRR

B

#A 4,8 3,2

C

#B 6,6 7,8

D E # TEST

#C 6,8 5,4

#D 4,0 4,5

F

#E 0,9 1,7

G

#F 4,0 4,6

#G 3,9 3,2

H

A B C D E F G H

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.5 9 9.5 10

FAR

#H 1,1 1,0

Fig. 10. (a) Table and figure with the cross-over values for FAR and FRR. (b) Verification results: comparative of performance for tests #A to #I by means of ROC curves.

on a highly homogeneous hand database which corresponds to teenagers from 15 to 17 years old. In the referenced works, the homogeneity of the hands was never included in their respective documents and we do not know this data. Obviously, experiments with a heterogeneous database would yield much better results than shown in Table 9. On the other hand, note that the experimental prototype is different. Consequently, those features which are very dependent on the hand image quality and on the illumination of the scene may vary a little. For instance, we have found that features 7, 8, 10, 11, 13, 14, 16, and 17, which are defined on the finger folds, may change a little from one sample to another (specially for test #F). In fact,

there are hands where the finger folds are hardly marked on the image. In the test #B the grey level of the profiles may also vary because of its sensitivity to the illumination. Another interesting factor to consider is that, in our experience, the samples were taken on different days and we did not have special illumination and positioning control as used in [8]. On the other hand, we have also found that there exist features which may vary as a result of the image processing itself. For instance the features 19, 20, 27, 28, 29, and 32 are highly dependant on the key-points (see Fig. 9 left). Fig. 10 shows the verification results through the ROC curves and presents the FAR/FRR cross-over values. It

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

464 100

100 90 80

Recognition percentage

Recognition Percentage

95

90

A B C D E F G H

85

80

75

1

1-2

1-3

1-4

1-5

1-6

1-7

1-8

1-9

70 60 50 #A #B #C #D #E #F #G #H

40 30 20 10 0

1-10

1

Candidates

1-2

1-3

1-4

1-5

Number of candidates Fig. 11. Identification results for test #A to #I.

Table 10 Identification results. Comparative of performance for tests #A to #I by means of the recognition percentage from 1 to 10 candidates Test

#A #B #C #D #E #F #G #H

Candidates 1

1–2

1–3

1–4

1–5

1–6

1–7

1–8

1–9

1–10

90.0 75.4 82.3 85.3 93.9 87.1 85.4 95.6

90.6 76.1 84.3 87.4 94.7 88.0 86.2 95.7

91.9 79.6 86.2 88.5 95.4 89.8 88.4 96.4

92.2 81.3 87.0 89.1 95.8 90.2 89.1 96.6

94.4 85.6 89.9 92.0 96.7 92.9 91.7 97.4

95.9 89.6 92.8 93.9 97.6 94.8 93.9 98.1

97.0 92.6 94.8 95.8 98.3 96.2 95.5 98.6

97.9 94.8 96.6 97.0 98.8 97.5 97.0 99.0

98.5 96.3 97.5 98.1 99.2 98.2 98.0 99.3

99.1 97.6 98.3 98.6 99.4 98.7 98.6 99.5

can be seen that tests #E and #H yield the best results with FAR/FRR cross-over ratios next to 1–2%, whereas test #C and, specially, #B give the worst results with FAR/FRR cross-over ratios around 7%. We think that the reason of such bad results lies in that this technique [8], which deals with the grey level of the profiles, is highly dependent on the illumination and perhaps we were unable to maintain it during the experimentation period. For the rest of the tests the cross-over ratios FAR/FRR are around 3–4% and for FAR = 3% the percent genuine rejection is higher than 90%. With regard to the identification results (Fig. 11), all the tests, except #B and #C, have success percentages above 85% for the first and second candidates. The best recognition ratios corresponds to the test #H (95,6%) and #E (93,9%) (Table 10). Making a reflection about this experience and analysing the results compared with ours, we conclude that such geometric features and templates are more changeable and less robust than ours due to the dependency on previous image processing. This circumstance may have a serious influence on the effectiveness of these systems. On the contrary, our approach avoids this problem; it just extracts the hand’s contour, finds the radius function and obtains the features in it. This makes the feature measurements more stable.

5. Conclusions and future work In this paper, we present a new hand biometric system for verification/recognition goals based on the natural layout of both right and left hands. The main contributions and key-points of this method are as follows: I. This strategy increases the disparity (distance) between hands in relation to those that use only one hand. This is due to the fact that, in our context biometric, we integrate two levels: fusion at representation level where features of left and right hands are concatenated and fusion at decision level, where the decision scores of left and right hand are combined to generate the final decision. II. It avoids pre-fixed poses and training phases since the individuals just have to extend their hands over a glass platform. Additionally, they do not have to remove their wristwatch, bracelets, or similar items because the system ignores the lower part of the hand. III. The computational cost is really low due to the fact that the image processing is minimum (it is only necessary to extract the hand contour). This makes the system suitable for on-line biometric applications. Obviously, other characteristics such as palmprint or knuckleprint

M. Ada´n et al. / Image and Vision Computing 26 (2008) 451–465

could be taken into account to obtain a more robust system but the system would become complicated and slow. IV. It is an original method based on the hand’s natural layout. V. It has been successfully tested under real conditions. The whole experimentation work has been carried out in a high school. We have carried out a wide experimentation in three directions. Firstly, verification and identification tests of the method have been carried out in real environments on 470 individuals yielding acceptable results. Secondly, we have run the algorithm based on the Normalized Similarity Measure using some typical geometric features obtained after an image processing. Thirdly, we have compared our method with others by carrying out a new experimentation on a more reduced set of individuals. The results are good enough to consider this biometric system for future security/control applications Our future work is addressed in three ways. Firstly, we are encouraged by the decrease in FAR and FRR values by improving the prototype, the registration phase, and the software that control it. Secondly, in order to make our system suitable for real verification environments we must optimize the program code and implement it on suitable hardware and software. Finally, we want to increase the database and check the performance of method daily for a longer period of time. Acknowledgements This research has been supported by the JJCC CastillaLa Mancha Spanish project PBI-02-008. Thanks to Sta Marı´a de Alarcos High School for your collaboration. References [1] R. Sa´nchez-Reillo, C. Sa´nchez-Avila, A. Gonza´lez-Marcos, Biometric identification through hand geometry measurements, IEEE Transactions on PAMI 22 (2000) 1168–1171. ¨ den, A. ErC [2] C. O ¸ il, V. Taylan, H. Kirmizitas, B. Bu¨ke, Hand recognition using implicit polynomials and geometric features, in: Audio and Video-Based Biometric Person Authentication AVBPA, Springer, 2001, pp. 336–341. [3] C. Han, H. Cheng, C. Lin, K. Fan, Personal authentication using palm-print features, Pattern Recognition 36 (2003) 371–381.

465

[4] D. Zhang, W.K. Kong, J. You, M. Wong, Online palmprint identification, IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (2003) 1041–1050. [5] D. Zhang, W. Shu, Two novel characteristic in palmprint verification: datum point invariance and line feature marching, Pattern Recognition 32 (1999) 691–702. [6] J. You, W. Li, D. Zhan, Hierarchical palmprint identification via multiple feature extraction, Pattern Recognition 35 (2002) 847–859. [7] X. Wu, D. Zhang, K. Wang, B. Huang, Palmprint classification using principal lines, Pattern Recognition 37 (2004) 1987–1998. [8] D.G. Joshi, Y.V. Rao, S. Kar, V. Kumar, R. Kumar, Computervision-based approach to personal identification using finger crease pattern, Pattern Recognition 31 (1998) 15–22. [9] A.K. Jain, N. Duta, Deformable matching of hand for verification, in: Proceedings of IEEE International Conference on Image Processing, Kobe, JP, 1999, pp. 857–861. [10] R.L. Zunkel, Hand geometry based verification, in: A. K Jain, R. Bole, S. Pankanti (Eds.), Biometrics Personal Identification in Networked Society, Kluwer Academic Publisher, 1999, pp. 87–101. [11] A.K. Jain, A. Ross, S. Pankanti, A prototype hand geometry-based verification system, in: Proceedings of Second International Conference on Audio- and Video-based Biometric Person Authentication, Washinton, DC, USA, 1999, pp. 166–171. [12] C.C. Han. A hand-based personal autentication using a coarse-to fine strategy, Image and Vision Computing 22 (2004) 909–918. [13] D. Sidlauskas, A new concept in biometric identification 3-dimensional hand geometry, Institute of Nuclear Materials Management, v.16., 28th Annual Meeting, 1987, pp. 442–447. [14] M. Ada´n, A. Ada´n, R. Torres, A.S. Va´zquez, G. Bueno. Personal authentication based on hands natural layout, in: Conference on Machine Vision Applications, Tsukuba Science City, Japan, 2005, pp. 160–163. [15] M. Golfareille, D. Maio, D. Davide Maltoni, On the error-reject trade-off in biometric verification systems, IEEE Transactions on PAMI 19 (1997) 786–796. [16] S. Gonzalez, C.M. Travieso, J.B. Alonso, M.A. Ferrer, ‘‘Automatic biometric identification system by hand geometry’’, in: IEEE International Carnahan Conference, 2003, pp. 281–284. [17] Michael Goh Kah Ong, Tee Connie, Andrew Teoh Beng Jin, David Ngo Chek Ling, ‘‘A Single-sensor Hand Geometry and Palmprint Verification System’’, WBMA’03, Berkeley, California, USA, 2003. [18] A. Wong, P. Shi, Peg-free hand geometry recognition using hierarchical geometry and shape matching, in: IAPR Workshop on Machine Vision Applications (MVA02), 2002. [19] Qiang Li, Zhengding Qiu, Dongmei Sun, Jie Wu, S.Z. Li, et al., Personal identification using knuckleprint, Sinobiometrics (2004) 680–689. [20] A. Kumar, D.C.M. Wong, H.C. Shen, A.K. Jain, Personal verification using palmprint and hand geometry biometric, in: J. Kittler, M.S. Nixon (Eds.), Audio- and Video-based Biometric Person Authentication, Guilford, UK, 2003, pp. 668–678. [21] W. Xiong, K-A. Toh, W-Y Yau, X. Jiang, Model-guided deformable hand shape recognition without positioning aids, Pattern Recognition 38 (2005) 1651–1664.