- Email: [email protected]
right shadows; facial symmetry
Pattern Recognition 39 (2006) 1542 – 1545 www.elsevier.com/locate/patcog
Rapid and brief communication
Face recognition robust to left/right shadows; facial symmetry Young-Jun Song, Young-Gil Kim∗ , Un-Dong Chang, Heak Bong Kwon Department of Computer and Communication Engineering, Chungbuk National University, 12 Gaesin-dong, Heungduk-gu Cheongju, Chungbuk 361-763, Korea Received 12 October 2005; accepted 15 February 2006
Abstract This paper has proposed an efficient shaded-face pre-processing technique using front-face symmetry. The existing face recognition PCA technique has a shortcoming of making illumination variation lower the recognition performance of a shaded face. The study has aimed to improve the performance by using the symmetry of the left and right face. In order to evaluate the performance of the proposed face recognition method, the study experimented with the Yale face database with left/right shadows. The experimental methods for this are as following: the existing PCA, PCA with first three eigenfaces excluded, histogram equalization and the proposed method. As the result, it was shown that the proposed method has a rather excellent recognition performance (98.9%). 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Face recognition; PCA; Facial symmetry; Illumination
1. Introduction Face recognition has been being studied as the ultimate certification field of biometric recognition. Particularly, it can be said that face recognition in security system is an essential technology for crime prevention and user certification. Also, high-qualified and low-priced computers and cameras are promoting popularization of the face recognition system. Of the existing face recognition methods, the PCA method is obtaining eigenvalue and eigenvector by using the dispersion of the whole training image, then eigenvalues arranged in descending order corresponding eigenvectors [1]. First, Turk, etc. [2] extracted noncorrelational features between objects by PCA, and applied the neighborhood algorithm classification method to face recognition. Recognition has a process of comparing the feature vector of training image and the feature vector of test image stored in database. However, great illumination variation can ∗ Corresponding author. Tel.: +82 43 261 2483; fax: +82 43 271 8085.
E-mail addresses: [email protected] (Y.-J. Song), [email protected] (Y.-G. Kim), [email protected] (U.-D. Chang), [email protected] (H.B. Kwon).
not facilitate division of classes as feature vectors obtained from face images. That is because illumination variation is centered on first three eigenfaces with the most information in time of comparison of eigenvalues. Therefore, except for the first 3 eigenfaces, dividing the classes of feature vectors becomes a little easier [3]. And histogram equalization technique, when luminance value distribution is not uniform, standardizes it by regulating values artificially. Early work in illumination invariant face recognition focused on image representations that are mostly insensitive to changes in illumination. Shashua and Riklin-Raviv [4] proposed a different illumination invariant image representation, the quotient image. The study has tried to raise the PCA face recognition performance by mirror image made out of luminance difference between the left and right against the front face shaded by illumination. So as to evaluate the recognition performance of the proposed method, the study compared PCA, PCA with first 3 eigenfaces excluded, histogram equalization, and the proposed method by using the Yale face database. The structure of the study is as following: Section 2 explains the proposed method using symmetry; Section 3 is on the face database used in the experiment, the
0031-3203/$30.00 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2006.02.018
Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542 – 1545
experimental method, and the results; finally, Section 4 comes to a conclusion. 2. The proposed method 2.1. System architecture The study has aimed to improve misrecognition of a face image shaded by illumination. In order to reduce the effect of illumination, the pre-processing of face recognition produces mirror image from shaded test image. That is to say, based on the face center of test image, first the luminance difference should be obtained between the right and left face image, then the brighter face of the two be mirrored, and its mirror image be produced. The mirror image produced by the preprocessing is used for the input image of face recognition system which the existing PCA method has been applied to. The whole architecture of the face recognition system including the proposed pre-processing system is as in Fig. 1. Above all, on the assumption that as for input image the front face has symmetry, the left and right face were divided on the horizontal-axis center of input image. Shade variation by illumination requires the block luminance difference between the left and right face; when it is
Input image
Make left/right half face by symmetry
Luminance difference between left and right face > Threshold
Yes
Make mirror image
No
1543
not bigger than threshold value (experimental value = 100), the PCA method must be applied to face recognition; however, when it is bigger, the mirror image be produced and PCA be applied to it. Also, by using Euclidean distance and comparing the similarities, face with the biggest similarity gets recognized as that of the same person. 2.2. Comparison of the left and right luminance value The left and right face, in case of little variation, has symmetry. So if illumination is uniformly applied to a front face, the left and right, with the nose a center, has almost the same luminance value. But when there is some pose variation, the luminance value difference of a specific part like the eyes and the mouth can be wrongly recognized as that of the whole face. As a method overcoming some pose variation, the average luminance value of a block (3 pixels × 3 pixels) can be used for comparison of the left and right face luminance value. When the luminance values of the left IL and the right face part IR are compared, the binary face image IB , newly composed by giving 1 to the big part and 0 to the small part in case of more than 30 (experiment value) of difference between the average block values, is recomposed by Eq. (1). IB = 1, |IL − IR | 30, (1) IB = 30 otherwise. When, of the recomposed left and right face, more than 100 is the number of image (about 38×31) 1 recomposed by 3×3 blocks, that is regarded as an image with big luminance difference in the left and right of face image. It means the case when shade by illumination occupies about more than 10% of the whole face. The shade comparison image (recomposed by 1178 pixels) are composed of 0 and 1: 0 means little shade difference between the left and right; 1 means some shade difference. As the number of 1 can tell what the left–right shade difference is, about 10% of 1178 pixels (100) is used as threshold value producing mirror image. Big luminance difference between the left and right face deteriorates recognition performance when PCA is used. That is because, by using face symmetry, the bright face is mirrored on the part of the dark face and a new face image is produced which compensates for the effect of shade.
PCA
3. Experimental results and analysis
Euclidean distance measure
Face recognition
Fig. 1. System block diagram.
The study made the Yale face database with much shade difference into regularized gray image of 112 × 92 size; divided training image and test image by the hold-and-out mode. Whether to recognize training image from test image could measure similarity between feature vectors by using Euclidean distance. Simulation experimented PCA, PCA with first 3 eigenfaces excluded, histogram equalization, and the proposed method. Fig. 2 shows the change of recognition performance according to increasing PCA dimension-level in each of all the methods.
Recognition rate (%)
1544
100 90 80 70 60 50 40 30 20 10 0
Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542 – 1545
PCA PCA with 1st 3 eigenfaces excluded Histogram equlization proposed method 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 PCA (dimension) Fig. 2. The simulation result.
The result of application of PCA to original image keeps 92.2% in the 10th dimension; the result of PCA with first 3 eigenfaces excluded shows high recognition performance of 95.6% in the 18th dimension; also, the result of PCA application after histogram equalization was recognition performance of 95.6% in the 17th dimension, the same as that of PCA with first 3 eigenfaces excluded, in the 18th. However, it showed low recognition performance under the 10th dimension. The result of PCA application to test image, recomposed of mirror image according to luminance difference (the proposed method), has kept high recognition performance of 98.9% in and over the 10th dimension. Besides, it showed higher performance even under the 10th dimension than any other method. 4. Conclusion The study has proposed an efficient pre-processing method of a shaded face by the symmetry of the front face. In order to reduce the effect of illumination, the study used the luminance difference between the left and right face, with test image as an object and the center-line of a face as a boundary. The study judged the existence of shade from this; when there was much shade difference between the two, it produced mirror image and applied PCA to it. When there was great shade by the effect of the left and right illumination, the proposed method had very efficient performance; has about 4% higher performance than the existing PCA, PCA with first 3 eigenfaces excluded, and histogram equalization. The future tasks are how much variation can be overcome by applying the proposed method to image with pose variation, and that the proposed method should be applied to another database (with partial shade) under illumination from various angles. 5. Summary This paper has proposed an efficient shaded-face preprocessing technique using front-face symmetry. The
existing face recognition PCA technique has a shortcoming of making illumination variation lower the recognition performance of a shaded face. The study has aimed to improve the performance by using the symmetry of the left and right face. In order to reduce the effect of illumination, the pre-processing of face recognition produces mirror image from shaded test image. That is to say, based on the face center of test image, first the luminance difference should be obtained between the right and left face image, then the brighter face of the two be mirrored, and its mirror image be produced. The mirror image produced by the pre-processing is used for the input image of face recognition system which the existing PCA method has been applied to. The proposed method is tested on the Yale face database with left/right shadows. The experimental methods for this are as following: the existing PCA, PCA with first three eigenfaces excluded, histogram equalization and the proposed method. The result of application of PCA to original image keeps 92.2% in the 10th dimension; the result of PCA with first three eigenfaces excluded shows high recognition performance of 95.6% in the 18th dimension; also, the result of PCA application after histogram equalization was recognition performance of 95.6% in the 17th dimension, the same as that of PCA with first three eigenfaces excluded, in the 18th. However, it showed low recognition performance under the 10th dimension. The result of PCA application to test image, recomposed of mirror image according to luminance difference (the proposed method), has kept high recognition performance of 98.9% in and over the 10th dimension. Besides, it showed higher performance even under the 10th dimension than any other method. As the result, it was shown that the proposed method has a rather excellent recognition performance.
Acknowledgments This work was supported by the Regional Research Centers Program of the Ministry of Education & Human Resources Development in Korea.
References [1] A.M. Martinez, A.C. Kak, PCA versus LDA, IEEE Trans. Pattern Anal. Mach. Intell. 29 (2) (2001) 228–233. [2] M. Turk, A. Pentland, Eigenfaces for recognition, J. Cognitive Neurosci. 3 (1) (1991) 71–86. [3] P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs. fisherfaces: recognition using class specific linear projection, in: European Conference on Computer Vision, 1996, pp. 45–58. [4] A. Shashua, T. Riklin-Raviv, The quotient image: class-based rerendering and recognition with varying illumination conditions, IEEE Trans. Pattern Anal. Mach. Intell. 23 (2) (2001).
Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542 – 1545
1545
About the Author—YOUNG-JUN SONG received the M.S. degree in computer and communication engineering from Chungbuk National University in 1996. He received the Ph.D. degree on face and image recognition. He currently works at Chungbuk National University, South Korea. His research interests also include face recognition, computer vision, image processing. About the Author—YOUNG-GIL KIM received the M.S. degree in computer and communication engineering from Chungbuk National University, Korea in 2001. He is currently working towards Ph.D. degree on face recognition and image segmentation. His research interests also include pattern recognition, computer vision, nonparametric analysis. About the Author—UN-DONG CHANG received the M.S. degree in computer and communication engineering from Chungbuk National University, Korea in 2002. He is currently working towards Ph.D. degree on pattern recognition, face recognition. About the Author—HEAK-BONG KWON received the M.S. degree in computer and communication engineering from Hoseo University, Korea in 1992. He received the Ph.D. degree on face and image recognition from Chungbuk National University, Korea in 2001. He currently works at Kimpo College, South Korea. His research interests also include image processing, computer vision, digital signal processing.
Rapid and brief communication
Face recognition robust to left/right shadows; facial symmetry Young-Jun Song, Young-Gil Kim∗ , Un-Dong Chang, Heak Bong Kwon Department of Computer and Communication Engineering, Chungbuk National University, 12 Gaesin-dong, Heungduk-gu Cheongju, Chungbuk 361-763, Korea Received 12 October 2005; accepted 15 February 2006
Abstract This paper has proposed an efficient shaded-face pre-processing technique using front-face symmetry. The existing face recognition PCA technique has a shortcoming of making illumination variation lower the recognition performance of a shaded face. The study has aimed to improve the performance by using the symmetry of the left and right face. In order to evaluate the performance of the proposed face recognition method, the study experimented with the Yale face database with left/right shadows. The experimental methods for this are as following: the existing PCA, PCA with first three eigenfaces excluded, histogram equalization and the proposed method. As the result, it was shown that the proposed method has a rather excellent recognition performance (98.9%). 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Face recognition; PCA; Facial symmetry; Illumination
1. Introduction Face recognition has been being studied as the ultimate certification field of biometric recognition. Particularly, it can be said that face recognition in security system is an essential technology for crime prevention and user certification. Also, high-qualified and low-priced computers and cameras are promoting popularization of the face recognition system. Of the existing face recognition methods, the PCA method is obtaining eigenvalue and eigenvector by using the dispersion of the whole training image, then eigenvalues arranged in descending order corresponding eigenvectors [1]. First, Turk, etc. [2] extracted noncorrelational features between objects by PCA, and applied the neighborhood algorithm classification method to face recognition. Recognition has a process of comparing the feature vector of training image and the feature vector of test image stored in database. However, great illumination variation can ∗ Corresponding author. Tel.: +82 43 261 2483; fax: +82 43 271 8085.
E-mail addresses: [email protected] (Y.-J. Song), [email protected] (Y.-G. Kim), [email protected] (U.-D. Chang), [email protected] (H.B. Kwon).
not facilitate division of classes as feature vectors obtained from face images. That is because illumination variation is centered on first three eigenfaces with the most information in time of comparison of eigenvalues. Therefore, except for the first 3 eigenfaces, dividing the classes of feature vectors becomes a little easier [3]. And histogram equalization technique, when luminance value distribution is not uniform, standardizes it by regulating values artificially. Early work in illumination invariant face recognition focused on image representations that are mostly insensitive to changes in illumination. Shashua and Riklin-Raviv [4] proposed a different illumination invariant image representation, the quotient image. The study has tried to raise the PCA face recognition performance by mirror image made out of luminance difference between the left and right against the front face shaded by illumination. So as to evaluate the recognition performance of the proposed method, the study compared PCA, PCA with first 3 eigenfaces excluded, histogram equalization, and the proposed method by using the Yale face database. The structure of the study is as following: Section 2 explains the proposed method using symmetry; Section 3 is on the face database used in the experiment, the
0031-3203/$30.00 䉷 2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2006.02.018
Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542 – 1545
experimental method, and the results; finally, Section 4 comes to a conclusion. 2. The proposed method 2.1. System architecture The study has aimed to improve misrecognition of a face image shaded by illumination. In order to reduce the effect of illumination, the pre-processing of face recognition produces mirror image from shaded test image. That is to say, based on the face center of test image, first the luminance difference should be obtained between the right and left face image, then the brighter face of the two be mirrored, and its mirror image be produced. The mirror image produced by the preprocessing is used for the input image of face recognition system which the existing PCA method has been applied to. The whole architecture of the face recognition system including the proposed pre-processing system is as in Fig. 1. Above all, on the assumption that as for input image the front face has symmetry, the left and right face were divided on the horizontal-axis center of input image. Shade variation by illumination requires the block luminance difference between the left and right face; when it is
Input image
Make left/right half face by symmetry
Luminance difference between left and right face > Threshold
Yes
Make mirror image
No
1543
not bigger than threshold value (experimental value = 100), the PCA method must be applied to face recognition; however, when it is bigger, the mirror image be produced and PCA be applied to it. Also, by using Euclidean distance and comparing the similarities, face with the biggest similarity gets recognized as that of the same person. 2.2. Comparison of the left and right luminance value The left and right face, in case of little variation, has symmetry. So if illumination is uniformly applied to a front face, the left and right, with the nose a center, has almost the same luminance value. But when there is some pose variation, the luminance value difference of a specific part like the eyes and the mouth can be wrongly recognized as that of the whole face. As a method overcoming some pose variation, the average luminance value of a block (3 pixels × 3 pixels) can be used for comparison of the left and right face luminance value. When the luminance values of the left IL and the right face part IR are compared, the binary face image IB , newly composed by giving 1 to the big part and 0 to the small part in case of more than 30 (experiment value) of difference between the average block values, is recomposed by Eq. (1). IB = 1, |IL − IR | 30, (1) IB = 30 otherwise. When, of the recomposed left and right face, more than 100 is the number of image (about 38×31) 1 recomposed by 3×3 blocks, that is regarded as an image with big luminance difference in the left and right of face image. It means the case when shade by illumination occupies about more than 10% of the whole face. The shade comparison image (recomposed by 1178 pixels) are composed of 0 and 1: 0 means little shade difference between the left and right; 1 means some shade difference. As the number of 1 can tell what the left–right shade difference is, about 10% of 1178 pixels (100) is used as threshold value producing mirror image. Big luminance difference between the left and right face deteriorates recognition performance when PCA is used. That is because, by using face symmetry, the bright face is mirrored on the part of the dark face and a new face image is produced which compensates for the effect of shade.
PCA
3. Experimental results and analysis
Euclidean distance measure
Face recognition
Fig. 1. System block diagram.
The study made the Yale face database with much shade difference into regularized gray image of 112 × 92 size; divided training image and test image by the hold-and-out mode. Whether to recognize training image from test image could measure similarity between feature vectors by using Euclidean distance. Simulation experimented PCA, PCA with first 3 eigenfaces excluded, histogram equalization, and the proposed method. Fig. 2 shows the change of recognition performance according to increasing PCA dimension-level in each of all the methods.
Recognition rate (%)
1544
100 90 80 70 60 50 40 30 20 10 0
Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542 – 1545
PCA PCA with 1st 3 eigenfaces excluded Histogram equlization proposed method 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 PCA (dimension) Fig. 2. The simulation result.
The result of application of PCA to original image keeps 92.2% in the 10th dimension; the result of PCA with first 3 eigenfaces excluded shows high recognition performance of 95.6% in the 18th dimension; also, the result of PCA application after histogram equalization was recognition performance of 95.6% in the 17th dimension, the same as that of PCA with first 3 eigenfaces excluded, in the 18th. However, it showed low recognition performance under the 10th dimension. The result of PCA application to test image, recomposed of mirror image according to luminance difference (the proposed method), has kept high recognition performance of 98.9% in and over the 10th dimension. Besides, it showed higher performance even under the 10th dimension than any other method. 4. Conclusion The study has proposed an efficient pre-processing method of a shaded face by the symmetry of the front face. In order to reduce the effect of illumination, the study used the luminance difference between the left and right face, with test image as an object and the center-line of a face as a boundary. The study judged the existence of shade from this; when there was much shade difference between the two, it produced mirror image and applied PCA to it. When there was great shade by the effect of the left and right illumination, the proposed method had very efficient performance; has about 4% higher performance than the existing PCA, PCA with first 3 eigenfaces excluded, and histogram equalization. The future tasks are how much variation can be overcome by applying the proposed method to image with pose variation, and that the proposed method should be applied to another database (with partial shade) under illumination from various angles. 5. Summary This paper has proposed an efficient shaded-face preprocessing technique using front-face symmetry. The
existing face recognition PCA technique has a shortcoming of making illumination variation lower the recognition performance of a shaded face. The study has aimed to improve the performance by using the symmetry of the left and right face. In order to reduce the effect of illumination, the pre-processing of face recognition produces mirror image from shaded test image. That is to say, based on the face center of test image, first the luminance difference should be obtained between the right and left face image, then the brighter face of the two be mirrored, and its mirror image be produced. The mirror image produced by the pre-processing is used for the input image of face recognition system which the existing PCA method has been applied to. The proposed method is tested on the Yale face database with left/right shadows. The experimental methods for this are as following: the existing PCA, PCA with first three eigenfaces excluded, histogram equalization and the proposed method. The result of application of PCA to original image keeps 92.2% in the 10th dimension; the result of PCA with first three eigenfaces excluded shows high recognition performance of 95.6% in the 18th dimension; also, the result of PCA application after histogram equalization was recognition performance of 95.6% in the 17th dimension, the same as that of PCA with first three eigenfaces excluded, in the 18th. However, it showed low recognition performance under the 10th dimension. The result of PCA application to test image, recomposed of mirror image according to luminance difference (the proposed method), has kept high recognition performance of 98.9% in and over the 10th dimension. Besides, it showed higher performance even under the 10th dimension than any other method. As the result, it was shown that the proposed method has a rather excellent recognition performance.
Acknowledgments This work was supported by the Regional Research Centers Program of the Ministry of Education & Human Resources Development in Korea.
References [1] A.M. Martinez, A.C. Kak, PCA versus LDA, IEEE Trans. Pattern Anal. Mach. Intell. 29 (2) (2001) 228–233. [2] M. Turk, A. Pentland, Eigenfaces for recognition, J. Cognitive Neurosci. 3 (1) (1991) 71–86. [3] P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs. fisherfaces: recognition using class specific linear projection, in: European Conference on Computer Vision, 1996, pp. 45–58. [4] A. Shashua, T. Riklin-Raviv, The quotient image: class-based rerendering and recognition with varying illumination conditions, IEEE Trans. Pattern Anal. Mach. Intell. 23 (2) (2001).
Y.-J. Song et al. / Pattern Recognition 39 (2006) 1542 – 1545
1545
About the Author—YOUNG-JUN SONG received the M.S. degree in computer and communication engineering from Chungbuk National University in 1996. He received the Ph.D. degree on face and image recognition. He currently works at Chungbuk National University, South Korea. His research interests also include face recognition, computer vision, image processing. About the Author—YOUNG-GIL KIM received the M.S. degree in computer and communication engineering from Chungbuk National University, Korea in 2001. He is currently working towards Ph.D. degree on face recognition and image segmentation. His research interests also include pattern recognition, computer vision, nonparametric analysis. About the Author—UN-DONG CHANG received the M.S. degree in computer and communication engineering from Chungbuk National University, Korea in 2002. He is currently working towards Ph.D. degree on pattern recognition, face recognition. About the Author—HEAK-BONG KWON received the M.S. degree in computer and communication engineering from Hoseo University, Korea in 1992. He received the Ph.D. degree on face and image recognition from Chungbuk National University, Korea in 2001. He currently works at Kimpo College, South Korea. His research interests also include image processing, computer vision, digital signal processing.