- Email: [email protected]
Image Specific Cross Cohort Normalization for Face Pair matching
ScienceDirect
Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 00 (2018) 000–000
Available online at www.sciencedirect.com
www.elsevier.com/locate/procedia
Procedia Computer Science 00 (2018) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 132 (2018) 1060–1067
International Conference on Computational Intelligence and Data Science (ICCIDS 2018) International Conference on Cohort Computational Intelligence and for DataFace SciencePair (ICCIDS 2018) Image Specific Cross Normalization matching b d Jogendra Garaina,Cross Ravi Kant KumarNormalization , Dipak Kumarc Dakshina Ranjan Kisku , and Image Specific Cohort for Face Pair matching e
Goutam Sanyal c Jogendra Garaina, Ravi KantDepartment KumarofbComputer , DipakScience Kumar Dakshina Ranjan Kiskud, and and Engineering e NationalGoutam Institute of Technology SanyalDurgapur a,b,c,d,e
Abstract
Durgapur – 713209, West Bengal, India a,b,c,d,e Department of Computer Science and Engineering {jogs.cse, vit.ravikant, dipakcsi, drkisku, nitgsanyal}@gmail.com National Institute of Technology Durgapur Durgapur – 713209, West Bengal, India {jogs.cse, vit.ravikant, dipakcsi, drkisku, nitgsanyal}@gmail.com
An image matching or face pair matching is purely different aspect with respect to the other problems of computer vision and Abstract pattern recognition. This is a very active and challenging topic due to the unavailability of any prior information to the matching expert about the input to matching be matched. Therefore an additional set of images this problem in somevision extent.and In An image matching or images face pair is purely different aspect with respect to can the resolve other problems of computer this context a cohort based face pair matching system is proposed. Initially the cohort set is common to all images but finally a pattern recognition. This is a very active and challenging topic due to the unavailability of any prior information to the matching subset of cohort images, specific to each of the paired images, are selected. Here Max-Min-Centroid-Cluster (MMCC) is applied expert about the input images to be matched. Therefore an additional set of images can resolve this problem in some extent. In which is capable enough choose relevant cohorts corresponding to target images. The raw similarity score but between this context a cohort basedtoface pairvery matching system is proposed. Initially the cohort set is common to all images finallythea input is normalized with to these to are obtain two normalized matching score. Afterwards the is closeness subsetimages of cohort images, specific eachsetofofthecohort pairedscores images, selected. Here Max-Min-Centroid-Cluster (MMCC) applied between images is measured by cross cohort normalization. The absolute difference of The theseraw twosimilarity crossly normalized scorethe is which is the capable enough to choose very relevant cohorts corresponding to target images. score between calculated and compared with a threshold value to decide the belonging of the input images to the same person or different input images is normalized with these set of cohort scores to obtain two normalized matching score. Afterwards the closeness person. The has beenby conducted on ORL face database the results foundofmake of the proposedscore system between the experiment images is measured cross cohort normalization. Theand absolute difference theseevidence two crossly normalized is to be efficient. calculated and compared with a threshold value to decide the belonging of the input images to the same person or different person. The experiment has been conducted on ORL face database and the results found make evidence of the proposed system © be 2018 The Authors. Published by Elsevier B.V. to efficient. Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and © 2018 2018 The Authors. Published by Elsevier Elsevier B.V. Ltd. Data Science (ICCIDSPublished 2018). by © The Authors. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Data Data Science Science (ICCIDS (ICCIDS 2018). 2018). Corresponding author mail: [email protected] 1877-0509 © 2018 Themail: Authors. Published by Elsevier B.V. Corresponding author [email protected]
Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Data Science (ICCIDS 2018). 1877-0509 © 2018 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Data Science (ICCIDS 2018).
1877-0509 © 2018 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Data Science (ICCIDS 2018). 10.1016/j.procs.2018.05.021
2
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. al. / Procedia Computer Science 00 (2018) 000–000
1061
Keywords: Face pair matching ; Cohort set; Cohort Selection; Cross Cohort Normalization; Max-Min-Centroid-Cluster.
1. Introduction The attention of modern research is being attracted towards a new aspect of biometric system i.e. image matching where two images of a person or different person are given as input and the system has to ascertain that the images belong to the same person or not. This is a very interesting problem because the system decides only on the basis of the input images with no prior knowledge about those images. The traditional purpose of a biometric system implies the identification problem and verification problem where the database contains the information about each and every identity. But in case of face pair matching, the system has only two images to process and finalize the decision. Neither any template nor any identity is provided. This kind of problems fall in the ‘under sampled’ category. In the literature it is found established that application of cohort set gives an encouraging result for this kind of problems which motivates the researcher to utilize the cohort set against this kind of problems. A cohort set is an additional set of images provided to be processed with the test images but it is disjoint to the test image set. Cohort score normalization is a well-known technique especially in the area of biometric research. But the proposed cross cohort normalization for face pair matching is a novel one and interesting too. In this method the first image prepares a set of cohort scores being matched with second cohort subset and the second input image prepares the same with the first cohort subset. Then the closeness between these two crossly normalized score states about the matching proximity of the input image pair. The paper is structured as follows. Section 2 states about literature survey related to the proposed work. Section 3 displays and explains the proposed architecture. The subsections of the proposed work are detailed in Section 4. The discussion about the database setup is in Section 5. The experiment and results analysis is in Section 6 and Section 7 concludes. 2. Literature Survey The concept of cohort selection begins to be applied with speaker verification [1] Then very shortly, it comes to the use of other biometric traits [2] and also in the format of multimodal biometric [3]. With the growing improvement of the verification and identification accuracy, the research community realizes that the performance of the image matching can also be enhanced with the application of cohort information [4]. Schroff F. et. al. [5] propose a face pair matching system to handle various challenges with input images which are represented with a signature made of a list of selected and ordered cohorts. M. Yang et. al. [6] claim that Sparse Variation Dictionary Learning (SVDL) can deal with the problems which have single sample per person for training. The system proposed by W. Deng et. al. [7] can also resolve this kind of under sampled problems as well as the problems with improper face alignment. A face recognition system proposed by Q. Yin et al. [8] performs well but it requires a large set of training data. The system selects the top matched cohorts from an available generic set of images to predict the appearance of an input face image. Tistarelli et. al. [9] use polynomial regression to extract the coefficients from a selected and sorted set of cohort images and report the improvement in a picture matching system. Kumar et. al. [10] proposed two different methods - Attribute classifier and simile classifiers for real world public face verification. Berg et. al. [11] used a reference set of faces to align first the "identity-preserving" and later a "Tom-Vs-Pete" classier is used for face verification. Li et. al. [12] proposed a method for unfamiliar face matching invariant to pose which shows an encouraging accuracy. 3. Architecture of the proposed Method The graphical view of the main architecture of the proposed work is shown in Fig. 1. Two face images (f1, f2) are given as input to the system with the target to decide that the images belong to same person or different persons. The face images are represented with Speeded Up Robust Feature (SURF) [13]. Then both the images are matched with
1062
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. at. / Procedia Computer Science 00 (2018) 000–000
3
a fixed set of cohort images to produce two set of cohort score {S1 } and S 2 which are passed through (MMCC) cohort selection method [14] to generate two relevant cohort subsets {C S1 } and {C S2 } corresponding to f1 and f2. Now f1 produces a set of cohort score S C1 being matched with {C S2 } and f2 produces another set of cohort score
{S C2 } being matched with {C S1 } . With the help of these score set the raw matching score (ρ) between f1 and f2 is
normalized and the closeness (Cn) is determined by taking absolute difference of them. Finally the closeness is compared with a decision threshold (θ).
Fig. 1. Architecture of the Proposed Method
4. PROPOSED METHOD The whole work is divided into five subsections- feature extraction, determination of matching score, cohort selection, closeness measure using cross cohort normalization and classification which are sequentially described below. 4.1. Feature Extraction All the images are featured by detected SURF points [13]. All the points are localized as shown in Fig. 2. SURF is a scale invariant, rotation invariant and (partially) illumination in-variant features. Therefore this feature can handle well with challenges available in ORL face database [15] on which the proposed system is investigated.
4
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. al. / Procedia Computer Science 00 (2018) 000–000
1063
Fig. 2. SURF point detection and localization
4.2. Determination of Matching Score After successfully completion of feature extraction, the matching score (ρ) as well as the cohort scores are determined using Best-fit algorithm [16]. For each input face pair, two sets of cohort score are produced which are further used for cohort subset selection. Samples of matching for both type of face pair, genuine as well as imposter, are shown in Fig. 3.
(a)
(b)
Fig. 3. SURF point matching (a) True pair (b) False pair
4.3. Cohort Selection The previous subsection outputs two sets of cohort score for a face pair. These sets of score are put to the (MMCC) cohort selection method [14] one by one for proper cohort subset selection because all cohorts are not relevant for every input image [17]. Therefore to avoid the unnecessary computation, a subset from the initial cohort pool is constructed corresponding to each input image. The cohort scores { δ1, δ2,…., δn-1} are clustered with kmeans clustering algorithm. After that the clusters with minimum and maximum centroid value are determined. To form the final cohort subset (Cf ), these two clusters are combined using union operation because most of the cohort images of these two clusters only carry distinguishable features. In the final experimental evaluation, k is considered as 5 after repetitive execution and analysis of its outcomes with different values of k. It is observed that for k=5, the cohort selection algorithm applied here selects a moderate number of cohorts for all images which is very essential to improve the matching accuracy of the presented system. Fig. 4 draws the flow diagram of the cohort subset selection using MMCC method based on k-means clustering. Later the raw similarity score (ρ) is normalized by the scores of this final cohort subset (Cf ). 4.4. Closeness Measure using Cross Cohort Normalization 1 2 1 The previous subsection provides two cohort subset, C f and C f for an input face pair (f1, f2) where C f 2 corresponds to f1 and C f corresponds to f2. With the help of these two subsets the raw matching score (ρ) between 2 f1 and f2 is crossly normalized with the help of T-norm score normalization [18]. f1 is matched with C f and produce
Jogendra et al. / Procedia Computer 132000–000 (2018) 1060–1067 J. Garain Garain et. at. / Procedia Computer ScienceScience 00 (2018)
1064
5
s a set of cohort scores whereas f2 is matched with C 1f and produces another set of cohort scores. The matching score (ρ) is normalized with these two sets of score. Therefore two crossly normalized scores η1 and η2 are obtained. Finally the closeness ( C n ) between f1 and f2 is measured using Equation (1).
C n 1 2
s.t. 1
1
2
2 and 2
1
1
1
(1)
where C n represents the cross normalized distance, ρ is raw matching score between f1 and f2, the mean of the scores obtained from ( C 1f ) is 1 and mean of the scores obtained from ( C 2f ) is 2 respectively. 1 and 2 are the standard deviation of all cohort scores from cohort subsets C 1f and C 2f respectively. The value of C n annunciates the possibility of the input images to be from a single person. The possibility is high when C n becomes low.
Fig. 4. Flowchart of Cohort selection using (MMCC ) method.
4.5. Classification The target of the system is to classify the input face pair as same person or distinct person. To do this tasks, the value of C n is compared with a threshold value (θ) and the decision is taken as per the expression shown in Equation (2).
C n , Face pair belongs to same person
, Face pair does not belong to same person
(2)
5. Database Setup The ORL (AT & T) face database is a collection of gray scale face images of size 92 112 pixels each. The images are captured at AT & T laboratory of Cambridge University during 1992 to 1994 with a homogenous
6
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. al. / Procedia Computer Science 00 (2018) 000–000
1065
background. A few instances are placed in Fig. 5. The database has total 400 images. There are 40 persons and each one has 10 instances which includes several kind of variations viz. open eye/close eye, smiling/neutral and with glasses/without glasses. The simulations are conducted in two phases. The first phase is for non cohort (baseline) system and the second phase is for the proposed system. Same protocol is followed in both the phases. Two instances per subjects are randomly chosen and stored in the initial cohort set. So the cohort pool contains 80 images in total. The remaining 8 samples per person are used for testing purpose. There is no requirement of training phase in image matching system. In testing two instances per subject are picked up randomly and formed an input face pair. So for genuine pair total test pair is 40. But in case of false pair, every subject is paired with all other subjects and in the same way two instances are chosen randomly to give as input. Therefore total 40 39=1560 test pairs are tested in case of imposter matching.
Fig.5. ORL database samples showing all ten instances
6. Experiment and Result Analysis To assess the performance of the proposed system, the parameters i.e. FMR (False Match Rate), TMR (True Match Rate), TNMR (True Non Match Rate) and FNMR (False Non Match Rate) defined in Equation 3, Equation 4, Equation 5 and Equation 6 are used. The matching accuracy can be calculated as per the Equation 7.
Number of genuine pair accepted as matched 100 % Total numberof genuine pair tested Number of false pair accepted as matched FMR 100 % Total numberof false pair tested Number of false pair rejected 100 % TNMR Total numberof false pair tested
TMR
FNMR
Numberof genuine pair rejected 100 % Total numberof genuine pair tested
FMR FNMR 100 % 2 OR T MR T NMR 100 % 2
(3) (4) (5) (6)
Accuracy 100
(7)
The ROC curve is depicted in Fig. 6 where X axis represents False Match Rate (FMR) and Y axis represents True Match Rate (TMR). The line of the curve indicates a better system when it has a higher value of Y with a lesser
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. at. / Procedia Computer Science 00 (2018) 000–000
1066
7
value of X. With that respect it is clearly noticeable in Fig. 6 that the proposed cohort based method representing a better system than the traditional non cohort system. The key objective of this proposed work is to highlight the superiority of the cohort based system over non-cohort system and it is shown successfully in this paper. The experimental findings of FMR, FNMR, EER (Equal Error Rate) TMR and Accuracy are shown in Table 1. EER is calculated as the average of FMR and FNMR. The proposed system is showing almost 5% better accuracy than the traditional system which is not using cohort.
Fig. 6. ROC Curve comparing the proposed method with baseline system Table 1. Performance Evaluation and Comparison.
System
FMR (%)
FNMR (%)
EER (%)
TMR (%)
Accuracy (%)
Baseline
2.69
37.50
20.10
62.50
79.90
Proposed
13.02
17.50
15.26
82.50
84.74
7. Conclusion This paper proposes a face pair matching system where two additional concepts are applied- cohort selection and cross cohort normalization. MMCC method and T-norm technique are applied for cohort selection and score normalization respectively. Speeded Up Robust Feature (SURF) is applied to represent the face images to be processed in the system. The accuracy achieved is quite good compared to non cohort system. It is understood by the investigations that cohort information help to improve the accuracy of the system but the technique to choose cohort subsets must be robust enough. In addition, the concept of normalization by crossly using the cohort information is interesting and very effective too. The performance of the system may be affected by some non face part in the images because no preprocessing or image enhancement technique is applied in the experiments. So the performance of the proposed system can be enhanced further if the images are preprocessed in an appropriate manner. This proposed method is not limited to only the feature extraction and cohort selection techniques applied here. It can be investigated with some other techniques also. References 1.
Auckenthaler, Roland, Michael Carey, and Harvey Lloyd-Thomas. (2000) "Score normalization for text-independent speaker verification systems." Digital Signal Processing 10.1-3: 42-54.
8
2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18.
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. al. / Procedia Computer Science 00 (2018) 000–000
Garain, Jogendra, Ravi Kant Kumar, Dakshina Ranjan Kisku, and Goutam Sanyal. (2016) "Selection of user-dependent cohorts using bezier curve for person identification." In International Conference Image Analysis and Recognition, Springer, Cham: 566-572. Merati, Amin, Norman Poh, and Josef Kittler. (2012) "User-specific cohort selection and score normalization for biometric systems." IEEE Transactions on Information Forensics and Security 7.4: 1270-1277. Sun, Yunlian, Massimo Tistarelli, and Norman Poh. (2013) "Picture-specific cohort score normalization for face pair matching." Biometrics: Theory, Applications and Systems (BTAS), 2013 IEEE Sixth International Conference on. IEEE: 1-8. Schroff, Florian, Tali Treibitz, David Kriegman, and Serge Belongie. (2011) "Pose, illumination and expression invariant pairwise face-similarity measure via doppelgänger list comparison." Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE: 2494-2501. Yang, Meng, Luc Van Gool, and Lei Zhang. (2013) "Sparse variation dictionary learning for face recognition with a single training sample per person." Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE: 689-696. Deng, Weihong, Jiani Hu, and Jun Guo. (2012) "Extended SRC: Undersampled face recognition via intraclass variant dictionary." IEEE Transactions on Pattern Analysis and Machine Intelligence 34.9: 1864-1870. Yin, Qi, Xiaoou Tang, and Jian Sun. (2011) "An associate-predict model for face recognition." Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE: 497-504. Tistarelli, Massimo, Yunlian Sun, and Norman Poh.(2014) "On the use of discriminative cohort score normalization for unconstrained face recognition." IEEE Transactions on information forensics and security 9.12: 2063-2075. Kumar, Neeraj, Alexander C. Berg, Peter N. Belhumeur, and Shree K. Nayar. "Attribute and simile classifiers for face verification." In Computer Vision, IEEE 12th International Conference on IEEE: 365-372. Berg, Thomas, and Peter N. Belhumeur. (2012, September). Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification. In BMVC (Vol. 2, p. 7). Li, Haoxiang, and Gang Hua. (2018) "Probabilistic elastic part model: a pose-invariant representation for real-world face verification." IEEE transactions on pattern analysis and machine intelligence 40.4: 918-930. Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. (2006) "Surf: Speeded up robust features." European conference on computer vision. Springer, Berlin, Heidelberg: 404-417. Garain, Jogendra, Ravi Kant Kumar, Goutam Sanyal, and Dakshina Ranjan Kisku.(2015) "Cohort selection of specific user using max-min-centroid-cluster (MMCC) method to enhance the performance of a biometric system." Int. J. Secur. Appl 9, no. 6 : 263-270. Samaria, Ferdinando S., and Andy C. Harter. (1994) "Parameterisation of a stochastic model for human face identification." Applications of Computer Vision, Proceedings of the Second IEEE Workshop on. IEEE:138-142. Garain, Jogendra, Adarsh Shah, Ravi Kant Kumar, Dakshina Ranjan Kisku, and Goutam Sanyal. (2016) "BCP-BCS: Best-Fit Cascaded Matching Paradigm with Cohort Selection Using Bezier Curve for Individual Recognition." Asian Conference on Computer Vision. Springer, Cham: 377-390. Merati, Amin, Norman Poh, and Josef Kittler. (2010) "Extracting discriminative information from cohort models." Biometrics: Theory Applications and Systems (BTAS), Fourth IEEE International Conference on. IEEE: 1-6. Tulyakov, Sergey, Zhi Zhang, and Venu Govindaraju. (2008) "Comparison of combination methods utilizing tnormalization and second best score model." In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW'08. IEEE Computer Society Conference on IEEE: 1-5.
1067
Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 00 (2018) 000–000
Available online at www.sciencedirect.com
www.elsevier.com/locate/procedia
Procedia Computer Science 00 (2018) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 132 (2018) 1060–1067
International Conference on Computational Intelligence and Data Science (ICCIDS 2018) International Conference on Cohort Computational Intelligence and for DataFace SciencePair (ICCIDS 2018) Image Specific Cross Normalization matching b d Jogendra Garaina,Cross Ravi Kant KumarNormalization , Dipak Kumarc Dakshina Ranjan Kisku , and Image Specific Cohort for Face Pair matching e
Goutam Sanyal c Jogendra Garaina, Ravi KantDepartment KumarofbComputer , DipakScience Kumar Dakshina Ranjan Kiskud, and and Engineering e NationalGoutam Institute of Technology SanyalDurgapur a,b,c,d,e
Abstract
Durgapur – 713209, West Bengal, India a,b,c,d,e Department of Computer Science and Engineering {jogs.cse, vit.ravikant, dipakcsi, drkisku, nitgsanyal}@gmail.com National Institute of Technology Durgapur Durgapur – 713209, West Bengal, India {jogs.cse, vit.ravikant, dipakcsi, drkisku, nitgsanyal}@gmail.com
An image matching or face pair matching is purely different aspect with respect to the other problems of computer vision and Abstract pattern recognition. This is a very active and challenging topic due to the unavailability of any prior information to the matching expert about the input to matching be matched. Therefore an additional set of images this problem in somevision extent.and In An image matching or images face pair is purely different aspect with respect to can the resolve other problems of computer this context a cohort based face pair matching system is proposed. Initially the cohort set is common to all images but finally a pattern recognition. This is a very active and challenging topic due to the unavailability of any prior information to the matching subset of cohort images, specific to each of the paired images, are selected. Here Max-Min-Centroid-Cluster (MMCC) is applied expert about the input images to be matched. Therefore an additional set of images can resolve this problem in some extent. In which is capable enough choose relevant cohorts corresponding to target images. The raw similarity score but between this context a cohort basedtoface pairvery matching system is proposed. Initially the cohort set is common to all images finallythea input is normalized with to these to are obtain two normalized matching score. Afterwards the is closeness subsetimages of cohort images, specific eachsetofofthecohort pairedscores images, selected. Here Max-Min-Centroid-Cluster (MMCC) applied between images is measured by cross cohort normalization. The absolute difference of The theseraw twosimilarity crossly normalized scorethe is which is the capable enough to choose very relevant cohorts corresponding to target images. score between calculated and compared with a threshold value to decide the belonging of the input images to the same person or different input images is normalized with these set of cohort scores to obtain two normalized matching score. Afterwards the closeness person. The has beenby conducted on ORL face database the results foundofmake of the proposedscore system between the experiment images is measured cross cohort normalization. Theand absolute difference theseevidence two crossly normalized is to be efficient. calculated and compared with a threshold value to decide the belonging of the input images to the same person or different person. The experiment has been conducted on ORL face database and the results found make evidence of the proposed system © be 2018 The Authors. Published by Elsevier B.V. to efficient. Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and © 2018 2018 The Authors. Published by Elsevier Elsevier B.V. Ltd. Data Science (ICCIDSPublished 2018). by © The Authors. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Data Data Science Science (ICCIDS (ICCIDS 2018). 2018). Corresponding author mail: [email protected] 1877-0509 © 2018 Themail: Authors. Published by Elsevier B.V. Corresponding author [email protected]
Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Data Science (ICCIDS 2018). 1877-0509 © 2018 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Data Science (ICCIDS 2018).
1877-0509 © 2018 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the International Conference on Computational Intelligence and Data Science (ICCIDS 2018). 10.1016/j.procs.2018.05.021
2
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. al. / Procedia Computer Science 00 (2018) 000–000
1061
Keywords: Face pair matching ; Cohort set; Cohort Selection; Cross Cohort Normalization; Max-Min-Centroid-Cluster.
1. Introduction The attention of modern research is being attracted towards a new aspect of biometric system i.e. image matching where two images of a person or different person are given as input and the system has to ascertain that the images belong to the same person or not. This is a very interesting problem because the system decides only on the basis of the input images with no prior knowledge about those images. The traditional purpose of a biometric system implies the identification problem and verification problem where the database contains the information about each and every identity. But in case of face pair matching, the system has only two images to process and finalize the decision. Neither any template nor any identity is provided. This kind of problems fall in the ‘under sampled’ category. In the literature it is found established that application of cohort set gives an encouraging result for this kind of problems which motivates the researcher to utilize the cohort set against this kind of problems. A cohort set is an additional set of images provided to be processed with the test images but it is disjoint to the test image set. Cohort score normalization is a well-known technique especially in the area of biometric research. But the proposed cross cohort normalization for face pair matching is a novel one and interesting too. In this method the first image prepares a set of cohort scores being matched with second cohort subset and the second input image prepares the same with the first cohort subset. Then the closeness between these two crossly normalized score states about the matching proximity of the input image pair. The paper is structured as follows. Section 2 states about literature survey related to the proposed work. Section 3 displays and explains the proposed architecture. The subsections of the proposed work are detailed in Section 4. The discussion about the database setup is in Section 5. The experiment and results analysis is in Section 6 and Section 7 concludes. 2. Literature Survey The concept of cohort selection begins to be applied with speaker verification [1] Then very shortly, it comes to the use of other biometric traits [2] and also in the format of multimodal biometric [3]. With the growing improvement of the verification and identification accuracy, the research community realizes that the performance of the image matching can also be enhanced with the application of cohort information [4]. Schroff F. et. al. [5] propose a face pair matching system to handle various challenges with input images which are represented with a signature made of a list of selected and ordered cohorts. M. Yang et. al. [6] claim that Sparse Variation Dictionary Learning (SVDL) can deal with the problems which have single sample per person for training. The system proposed by W. Deng et. al. [7] can also resolve this kind of under sampled problems as well as the problems with improper face alignment. A face recognition system proposed by Q. Yin et al. [8] performs well but it requires a large set of training data. The system selects the top matched cohorts from an available generic set of images to predict the appearance of an input face image. Tistarelli et. al. [9] use polynomial regression to extract the coefficients from a selected and sorted set of cohort images and report the improvement in a picture matching system. Kumar et. al. [10] proposed two different methods - Attribute classifier and simile classifiers for real world public face verification. Berg et. al. [11] used a reference set of faces to align first the "identity-preserving" and later a "Tom-Vs-Pete" classier is used for face verification. Li et. al. [12] proposed a method for unfamiliar face matching invariant to pose which shows an encouraging accuracy. 3. Architecture of the proposed Method The graphical view of the main architecture of the proposed work is shown in Fig. 1. Two face images (f1, f2) are given as input to the system with the target to decide that the images belong to same person or different persons. The face images are represented with Speeded Up Robust Feature (SURF) [13]. Then both the images are matched with
1062
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. at. / Procedia Computer Science 00 (2018) 000–000
3
a fixed set of cohort images to produce two set of cohort score {S1 } and S 2 which are passed through (MMCC) cohort selection method [14] to generate two relevant cohort subsets {C S1 } and {C S2 } corresponding to f1 and f2. Now f1 produces a set of cohort score S C1 being matched with {C S2 } and f2 produces another set of cohort score
{S C2 } being matched with {C S1 } . With the help of these score set the raw matching score (ρ) between f1 and f2 is
normalized and the closeness (Cn) is determined by taking absolute difference of them. Finally the closeness is compared with a decision threshold (θ).
Fig. 1. Architecture of the Proposed Method
4. PROPOSED METHOD The whole work is divided into five subsections- feature extraction, determination of matching score, cohort selection, closeness measure using cross cohort normalization and classification which are sequentially described below. 4.1. Feature Extraction All the images are featured by detected SURF points [13]. All the points are localized as shown in Fig. 2. SURF is a scale invariant, rotation invariant and (partially) illumination in-variant features. Therefore this feature can handle well with challenges available in ORL face database [15] on which the proposed system is investigated.
4
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. al. / Procedia Computer Science 00 (2018) 000–000
1063
Fig. 2. SURF point detection and localization
4.2. Determination of Matching Score After successfully completion of feature extraction, the matching score (ρ) as well as the cohort scores are determined using Best-fit algorithm [16]. For each input face pair, two sets of cohort score are produced which are further used for cohort subset selection. Samples of matching for both type of face pair, genuine as well as imposter, are shown in Fig. 3.
(a)
(b)
Fig. 3. SURF point matching (a) True pair (b) False pair
4.3. Cohort Selection The previous subsection outputs two sets of cohort score for a face pair. These sets of score are put to the (MMCC) cohort selection method [14] one by one for proper cohort subset selection because all cohorts are not relevant for every input image [17]. Therefore to avoid the unnecessary computation, a subset from the initial cohort pool is constructed corresponding to each input image. The cohort scores { δ1, δ2,…., δn-1} are clustered with kmeans clustering algorithm. After that the clusters with minimum and maximum centroid value are determined. To form the final cohort subset (Cf ), these two clusters are combined using union operation because most of the cohort images of these two clusters only carry distinguishable features. In the final experimental evaluation, k is considered as 5 after repetitive execution and analysis of its outcomes with different values of k. It is observed that for k=5, the cohort selection algorithm applied here selects a moderate number of cohorts for all images which is very essential to improve the matching accuracy of the presented system. Fig. 4 draws the flow diagram of the cohort subset selection using MMCC method based on k-means clustering. Later the raw similarity score (ρ) is normalized by the scores of this final cohort subset (Cf ). 4.4. Closeness Measure using Cross Cohort Normalization 1 2 1 The previous subsection provides two cohort subset, C f and C f for an input face pair (f1, f2) where C f 2 corresponds to f1 and C f corresponds to f2. With the help of these two subsets the raw matching score (ρ) between 2 f1 and f2 is crossly normalized with the help of T-norm score normalization [18]. f1 is matched with C f and produce
Jogendra et al. / Procedia Computer 132000–000 (2018) 1060–1067 J. Garain Garain et. at. / Procedia Computer ScienceScience 00 (2018)
1064
5
s a set of cohort scores whereas f2 is matched with C 1f and produces another set of cohort scores. The matching score (ρ) is normalized with these two sets of score. Therefore two crossly normalized scores η1 and η2 are obtained. Finally the closeness ( C n ) between f1 and f2 is measured using Equation (1).
C n 1 2
s.t. 1
1
2
2 and 2
1
1
1
(1)
where C n represents the cross normalized distance, ρ is raw matching score between f1 and f2, the mean of the scores obtained from ( C 1f ) is 1 and mean of the scores obtained from ( C 2f ) is 2 respectively. 1 and 2 are the standard deviation of all cohort scores from cohort subsets C 1f and C 2f respectively. The value of C n annunciates the possibility of the input images to be from a single person. The possibility is high when C n becomes low.
Fig. 4. Flowchart of Cohort selection using (MMCC ) method.
4.5. Classification The target of the system is to classify the input face pair as same person or distinct person. To do this tasks, the value of C n is compared with a threshold value (θ) and the decision is taken as per the expression shown in Equation (2).
C n , Face pair belongs to same person
, Face pair does not belong to same person
(2)
5. Database Setup The ORL (AT & T) face database is a collection of gray scale face images of size 92 112 pixels each. The images are captured at AT & T laboratory of Cambridge University during 1992 to 1994 with a homogenous
6
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. al. / Procedia Computer Science 00 (2018) 000–000
1065
background. A few instances are placed in Fig. 5. The database has total 400 images. There are 40 persons and each one has 10 instances which includes several kind of variations viz. open eye/close eye, smiling/neutral and with glasses/without glasses. The simulations are conducted in two phases. The first phase is for non cohort (baseline) system and the second phase is for the proposed system. Same protocol is followed in both the phases. Two instances per subjects are randomly chosen and stored in the initial cohort set. So the cohort pool contains 80 images in total. The remaining 8 samples per person are used for testing purpose. There is no requirement of training phase in image matching system. In testing two instances per subject are picked up randomly and formed an input face pair. So for genuine pair total test pair is 40. But in case of false pair, every subject is paired with all other subjects and in the same way two instances are chosen randomly to give as input. Therefore total 40 39=1560 test pairs are tested in case of imposter matching.
Fig.5. ORL database samples showing all ten instances
6. Experiment and Result Analysis To assess the performance of the proposed system, the parameters i.e. FMR (False Match Rate), TMR (True Match Rate), TNMR (True Non Match Rate) and FNMR (False Non Match Rate) defined in Equation 3, Equation 4, Equation 5 and Equation 6 are used. The matching accuracy can be calculated as per the Equation 7.
Number of genuine pair accepted as matched 100 % Total numberof genuine pair tested Number of false pair accepted as matched FMR 100 % Total numberof false pair tested Number of false pair rejected 100 % TNMR Total numberof false pair tested
TMR
FNMR
Numberof genuine pair rejected 100 % Total numberof genuine pair tested
FMR FNMR 100 % 2 OR T MR T NMR 100 % 2
(3) (4) (5) (6)
Accuracy 100
(7)
The ROC curve is depicted in Fig. 6 where X axis represents False Match Rate (FMR) and Y axis represents True Match Rate (TMR). The line of the curve indicates a better system when it has a higher value of Y with a lesser
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. at. / Procedia Computer Science 00 (2018) 000–000
1066
7
value of X. With that respect it is clearly noticeable in Fig. 6 that the proposed cohort based method representing a better system than the traditional non cohort system. The key objective of this proposed work is to highlight the superiority of the cohort based system over non-cohort system and it is shown successfully in this paper. The experimental findings of FMR, FNMR, EER (Equal Error Rate) TMR and Accuracy are shown in Table 1. EER is calculated as the average of FMR and FNMR. The proposed system is showing almost 5% better accuracy than the traditional system which is not using cohort.
Fig. 6. ROC Curve comparing the proposed method with baseline system Table 1. Performance Evaluation and Comparison.
System
FMR (%)
FNMR (%)
EER (%)
TMR (%)
Accuracy (%)
Baseline
2.69
37.50
20.10
62.50
79.90
Proposed
13.02
17.50
15.26
82.50
84.74
7. Conclusion This paper proposes a face pair matching system where two additional concepts are applied- cohort selection and cross cohort normalization. MMCC method and T-norm technique are applied for cohort selection and score normalization respectively. Speeded Up Robust Feature (SURF) is applied to represent the face images to be processed in the system. The accuracy achieved is quite good compared to non cohort system. It is understood by the investigations that cohort information help to improve the accuracy of the system but the technique to choose cohort subsets must be robust enough. In addition, the concept of normalization by crossly using the cohort information is interesting and very effective too. The performance of the system may be affected by some non face part in the images because no preprocessing or image enhancement technique is applied in the experiments. So the performance of the proposed system can be enhanced further if the images are preprocessed in an appropriate manner. This proposed method is not limited to only the feature extraction and cohort selection techniques applied here. It can be investigated with some other techniques also. References 1.
Auckenthaler, Roland, Michael Carey, and Harvey Lloyd-Thomas. (2000) "Score normalization for text-independent speaker verification systems." Digital Signal Processing 10.1-3: 42-54.
8
2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18.
Jogendra Garain et al. / Procedia Computer Science 132 (2018) 1060–1067 J. Garain et. al. / Procedia Computer Science 00 (2018) 000–000
Garain, Jogendra, Ravi Kant Kumar, Dakshina Ranjan Kisku, and Goutam Sanyal. (2016) "Selection of user-dependent cohorts using bezier curve for person identification." In International Conference Image Analysis and Recognition, Springer, Cham: 566-572. Merati, Amin, Norman Poh, and Josef Kittler. (2012) "User-specific cohort selection and score normalization for biometric systems." IEEE Transactions on Information Forensics and Security 7.4: 1270-1277. Sun, Yunlian, Massimo Tistarelli, and Norman Poh. (2013) "Picture-specific cohort score normalization for face pair matching." Biometrics: Theory, Applications and Systems (BTAS), 2013 IEEE Sixth International Conference on. IEEE: 1-8. Schroff, Florian, Tali Treibitz, David Kriegman, and Serge Belongie. (2011) "Pose, illumination and expression invariant pairwise face-similarity measure via doppelgänger list comparison." Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE: 2494-2501. Yang, Meng, Luc Van Gool, and Lei Zhang. (2013) "Sparse variation dictionary learning for face recognition with a single training sample per person." Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE: 689-696. Deng, Weihong, Jiani Hu, and Jun Guo. (2012) "Extended SRC: Undersampled face recognition via intraclass variant dictionary." IEEE Transactions on Pattern Analysis and Machine Intelligence 34.9: 1864-1870. Yin, Qi, Xiaoou Tang, and Jian Sun. (2011) "An associate-predict model for face recognition." Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE: 497-504. Tistarelli, Massimo, Yunlian Sun, and Norman Poh.(2014) "On the use of discriminative cohort score normalization for unconstrained face recognition." IEEE Transactions on information forensics and security 9.12: 2063-2075. Kumar, Neeraj, Alexander C. Berg, Peter N. Belhumeur, and Shree K. Nayar. "Attribute and simile classifiers for face verification." In Computer Vision, IEEE 12th International Conference on IEEE: 365-372. Berg, Thomas, and Peter N. Belhumeur. (2012, September). Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification. In BMVC (Vol. 2, p. 7). Li, Haoxiang, and Gang Hua. (2018) "Probabilistic elastic part model: a pose-invariant representation for real-world face verification." IEEE transactions on pattern analysis and machine intelligence 40.4: 918-930. Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. (2006) "Surf: Speeded up robust features." European conference on computer vision. Springer, Berlin, Heidelberg: 404-417. Garain, Jogendra, Ravi Kant Kumar, Goutam Sanyal, and Dakshina Ranjan Kisku.(2015) "Cohort selection of specific user using max-min-centroid-cluster (MMCC) method to enhance the performance of a biometric system." Int. J. Secur. Appl 9, no. 6 : 263-270. Samaria, Ferdinando S., and Andy C. Harter. (1994) "Parameterisation of a stochastic model for human face identification." Applications of Computer Vision, Proceedings of the Second IEEE Workshop on. IEEE:138-142. Garain, Jogendra, Adarsh Shah, Ravi Kant Kumar, Dakshina Ranjan Kisku, and Goutam Sanyal. (2016) "BCP-BCS: Best-Fit Cascaded Matching Paradigm with Cohort Selection Using Bezier Curve for Individual Recognition." Asian Conference on Computer Vision. Springer, Cham: 377-390. Merati, Amin, Norman Poh, and Josef Kittler. (2010) "Extracting discriminative information from cohort models." Biometrics: Theory Applications and Systems (BTAS), Fourth IEEE International Conference on. IEEE: 1-6. Tulyakov, Sergey, Zhi Zhang, and Venu Govindaraju. (2008) "Comparison of combination methods utilizing tnormalization and second best score model." In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW'08. IEEE Computer Society Conference on IEEE: 1-5.
1067