- Email: [email protected]
3D fingerprint imaging system based on full-field fringe projection profilometry
Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
Contents lists available at SciVerse ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
3D fingerprint imaging system based on full-field fringe projection profilometry Shujun Huang, Zonghua Zhang n, Yan Zhao, Jie Dai, Chao Chen, Yongjia Xu, E. Zhang, Lili Xie School of Mechanical Engineering, Hebei University of Technology, Tianjin 300136, China
art ic l e i nf o
a b s t r a c t
Article history: Received 23 April 2013 Received in revised form 13 June 2013 Accepted 1 July 2013
As an unique, unchangeable and easily acquired biometrics, fingerprint has been widely studied in academics and applied in many fields over the years. The traditional fingerprint recognition methods are based on the obtained 2D feature of fingerprint. However, fingerprint is a 3D biological characteristic. The mapping from 3D to 2D loses 1D information and causes nonlinear distortion of the captured fingerprint. Therefore, it is becoming more and more important to obtain 3D fingerprint information for recognition. In this paper, a novel 3D fingerprint imaging system is presented based on fringe projection technique to obtain 3D features and the corresponding color texture information. A series of color sinusoidal fringe patterns with optimum three-fringe numbers are projected onto a finger surface. From another viewpoint, the fringe patterns are deformed by the finger surface and captured by a CCD camera. 3D shape data of the finger can be obtained from the captured fringe pattern images. This paper studies the prototype of the 3D fingerprint imaging system, including principle of 3D fingerprint acquisition, hardware design of the 3D imaging system, 3D calibration of the system, and software development. Some experiments are carried out by acquiring several 3D fingerprint data. The experimental results demonstrate the feasibility of the proposed 3D fingerprint imaging system. & 2013 Elsevier Ltd. All rights reserved.
Keywords: 3D fingerprint 3D biometrics Fringe projection profilometry Optimum three-fringe numbers selection
1. Introduction Biometrics means recognizing (verifying or identifying) a person by using the inherent physiological or behavioral characteristics of human body [1,2]. Physiological characteristics refer to data directly measured from the human body parts and some representative physiological features include fingerprints, hand shape, palmprint, face, iris, retina, and so on; while behavior characteristic is a measure of personal habitual action indirect from human body characteristics, such as voice, keystroke habits and signature. Human recognition based on biometrics is widely applied in many fields like government, army, bank, social welfare safeguard, e-commerce, security defense, and to name a few. Fingerprint, as one of the most important biometrics, has been widely studied and applied to personal recognition in both forensics and civilian. It is a unique and unchangeable physiological characteristic of human being during the whole life. The traditional 2D fingerprint has been captured by using an ink-based offline method or an imaging device, for example a CCD camera, to get an image for recognition. These methods need the subject to press (roll) his/her fingers against a surface to obtain 2D fingerprint image, so that the captured 2D fingerprint images are often
n
Corresponding author. Tel.: +86 22 265 824 03; fax: +86 22 602 041 89. E-mail addresses: [email protected], [email protected] (Z. Zhang).
distorted in a nonlinear way. Such nonlinear distortion increases the intra-class variations among the fingerprint images of the same finger and introduces matching errors. The existing 2D methods are also greatly affected by the brightness and contrast of environmental light, the dirty things on finger surface, so the matching results are inaccurate. Due to 3D to 2D mapping, the captured fingerprint features by using 2D methods have unavoidable distortion and illumination shading [3,4]. Therefore, identification and verification are based on the captured inaccurate 2D fingerprint biometrics. Jain [1] pointed out that “Efforts are afoot to design better biometric sensors/readers, to improve algorithms to extract features from raw biometric data and to match two biometric samples quickly and accurately”. In fact, fingerprint is a 3D biometric feature and has many advantages in comparison to 2D biometrics. Firstly, due to non-contact operation, there is no elastic deformation of the obtained features; secondly, 3D fingerprint can obtain the characteristic distribution patterns of fingerprint without distortion; thirdly, they give local space coordinate geometry and azimuth of fingerprint in details; finally, they are insensitive to brightness, contrast of environmental light, and dirty things on finger surface. With the development of CCD cameras and DLP (Digital Light Processing) projectors, 3D biometrics have been widely studied in recent years. 3D face recognition [5] is in a period of rapid expansion and 3D ear recognition [6,7] is in the very early stage. A few researchers have attempted 3D fingerprint as biometrics.
0143-8166/$ - see front matter & 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.optlaseng.2013.07.001
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
2
Parziale et al. [8] reported one surround imager by using multicamera system to acquire different finger views, so that 3D representation of fingerprint is obtained by using shape-fromsilhouette method. Recently, a new type of touchless 3D fingerprint identification access control system is developed by TBS in Switzerland (www.tbs-biometrics.com). It greatly improved the accuracy of access control and attendance system. Wang et al. [9] employed a noncontact 3D method based on phase measuring profilometry (PMP) to acquire 3D ridge-depth information of human finger. Gabai et al. [10] introduced a dual-channel interferometric imaging system to obtain phase imaging of fingerprints. Due to the speckles in interferometry, the obtained phase and then shape information are inaccurate. This paper presents a new imaging system to get 3D fingerprint data based on the sinusoidal fringe projection technique [11–14]. The system includes a color CCD camera, a small DLP projector and a laptop computer. The straight sinusoidal fringe patterns are generated by software in the computer and projected onto the finger surface by the DLP projector. From another viewpoint, the fringe patterns appear deformed with regard to the finger surface shape. The modulated fringe patterns are recorded by the CCD camera and saved into the computer for post-processing. Combining four-step phase-shifting algorithm [15] and optimum threefringe number selection [16–17] can independently calculate absolute phase at each pixel position. An automatic 3D calibration method builds up the relationship between absolute phase map and 3D shape, so that 3D fingerprint data can be obtained. The following section introduces the principle of 3D fingerprint imaging system based on absolute phase measurement. Some experiments on capturing 3D fingerprint are demonstrated in Section 3. Section 4 gives the conclusions and future directions.
sinusoidal fringe patterns by software DLP projector finger surface CCD camera deformed fringe patterns four-step phase shifting three wrapped phase maps optimum 3 number selection absolute unwrapped phase map 3D calibration of system x,y,z data of finger surface 3D representation by shading Fig. 1. Flowchart of 3D fingerprint data acquisition and processing.
2. Principle of 3D fingerprint system Phase calculation-based full-field sinusoidal fringe projection technique is used to obtain 3D shape of fingerprint. Three wrapped maps are calculated by using four-step phase-shifting algorithm and optimum three-fringe number selection method is applied to independently calculate the absolute phase information pixel by pixel. After 3D calibrating the system, the absolute phase at each pixel position can be converted into x, y, and z coordinates. Fig. 1 shows the flowchart of 3D fingerprint data acquisition and processing. 2.1. Components of the 3D imaging system The proposed 3D fingerprint imaging system is based on the full-field sinusoidal fringe projection technique, as demonstrated in Fig. 2. It consists of a DLP projector, a color CCD camera and a laptop computer. The DLP projector and the CCD camera satisfy a conventional triangulation arrangement. Sinusoidal fringe patterns are generated by software in the computer and projected onto a measured finger surface by the DLP projector. From another different viewpoint, the fringe patterns are deformed with respect to the finger surface and ridges. The CCD camera captures the deformed fringe patterns and saves them into the computer for post-processing. 2.2. Four-step phase-shifting algorithm Multiple-step phase-shifting algorithms have been widely used to calculate phase information because of their high accuracy. Among them, four-step phase-shifting algorithm is one of the most used phase calculation method in fringe pattern processing and has gained many achievements in research and industrial
Fig. 2. System schematic diagram. (1) Computer, (2) CCD Camera, (3) DLP projector, and (4) finger surface.
fields [15]. There are π=2 shift in between for the four fringe patterns. In order to make this paper self-contained, the mathematical model and distribution characteristics of fringe patterns are briefly described. The intensity distribution of a captured fringe pattern can be expressed by the following mathematical expression: Iðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φðm; nÞ þ I n ðm; nÞ
ð1Þ
where m; n are the index of a pixel in the captured image along vertical and horizontal direction, respectively; Iðm; nÞ is the measured intensity at (m, n) pixel position; I d ðm; nÞ and I m ðm; nÞ are the intensity of background and the fringe modulation, respectively; φðm; nÞ is the phase corresponding to the measured object shape; I n ðm; nÞ is he additive random noise during capturing process. Four-step phase-shifting algorithm means four fringe patterns have π=2 phase shift in between are successively projected into the
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
measuring volume and they can be expressed as I 1 ðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φ½ðm; nÞ þ 0 þ I n ðm; nÞ
ð2:AÞ
I 2 ðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φ½ðm; nÞ þ π=2 þ I n ðm; nÞ
ð2:BÞ
I 3 ðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φ½ðm; nÞ þ π þ I n ðm; nÞ
ð2:CÞ
I 4 ðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φ½ðm; nÞ þ 3π=2 þ I n ðm; nÞ
ð2:DÞ
3
compensated, there are still uncorrected distortions from the projecting lens of the DLP projector. Transverse calibration relates to depth information obtained from the projected fringe patterns. Therefore, at each pixel position the following two polynomial equations are used to compensate for the radical distortion to give a high accurate relationship: ( xr ¼ a0 ðm; nÞz2r þ b0 ðm; nÞzr þ c0 ð5Þ yr ¼ a1 ðm; nÞz2r þ b1 ðm; nÞzr þ c1
Assuming the four captured fringe patterns in the same optical field have the same gray value of background, modulation and noise, the phase field φðm; nÞ can be accurately calculated by the following equation: I 4 ðm; nÞI 2 ðm; nÞ ð3Þ ϕðm; nÞ ¼ arctan I 3 ðm; nÞI 1 ðm; nÞ
where a0 ; b0 ; c0 ; a1 ; b1 ; c1 are the coefficient set of the system parameters and xr ; yr ; zr are the coordinates in the reference coordinate system. All the coefficient sets in Eqs. (4) and (5) can be calibrated by a white plate having known discrete markers on surface.
Because of the property of trigonometric function, the obtained phase is in the range of π=2–π=2 in mathematics and then been changed in the range of π–π, which needs to be unwrapped to retrieve the continuous phase map for 3D shape measurement.
3. System and experiments
2.3. Optimum three-fringe number selection The traditional spatial phase unwrapping methods easily accumulate errors and cannot give correct fringe order for discontinuous objects. There are many ridges and valleys on finger surface, so the normal spatial phase unwrapping methods are not suitable to measure 3D fingerprint shape. The optimum three-fringe number selection method [16,17] can solve this problem since it calculates absolute phase pixel by pixel. The optimum three-fringe number selection method defines pffiffiffiffi the numbers of projected fringes to be N; N1; N N . For example, if N ¼ 25, the other two fringe sets have fringe numbers of 24 and 20. This method resolves fringe order ambiguity as the beat obtained between N and N1 which is a single fringe over the full field of view. The reliability of the obtained fringe order is maximized as the fringe order calculation is performed through a geometric series of beat fringes with 1, 5 and 25 fringes. The obtained absolute phase data are unwrapped pixel by pixel, so that this method can measure objects having discontinuities and/or isolated surfaces.
Based on the theoretical analysis, a 3D fingerprint imaging system was designed and developed in the following subsections. The system was 3D calibrated by an accurately manufactured white plate and the performance of calibration was evaluated by Table 1 Parameters of the DLP Pico projector. Nominal brightness Standard resolution Light source Frame rate Shape dimension
10 lm 640 480 Solid-State 3 LED 60 Hz for NTSC 68 mm 45 mm 14 mm
2.4. System 3D calibration The obtained absolute phase map corresponds to shape of the measured finger surface. In order to get shape information, the system needs to be calibrated to build up the relationship not only between depth and absolute phase, but x, y coordinates and pixel positions, which is called the depth calibration and transverse calibration, respectively [18,19]. Phase to depth conversion can be represented by the following polynomial equation at each pixel position [20]: K
zðm; nÞ ¼ ∑ ak ðm; nÞφðm; nÞk k¼0
Fig. 3. Hardware setup of the 3D fingerprint imaging system including a laptop computer, a Pico projector and a color CCD camera.
ð4Þ
where ak ; ak1 ; :::a2 ; a1 are a coefficient set containing the systematic parameters. Depth calibration means to determine the coefficient set of the polynomial equation. Because the coefficient set of Eq. (4) depends on pixel position (m, n) and will have different values, a Look-Up Table (LUT) needs to be used at each pixel position to save the coefficient set. Except for depth data, 3D shape needs the transverse x and y coordinates as well, which can be realized by a procedure of transverse calibration. The relationship is linear for an ideal optical imaging system. However, for an actual 3D imaging system, the relationship is nonlinear because of the distortion of optical imaging and projecting lens. Although the imaging distortion can
Fig. 4. The partial enlarged view of the Pico projector and the color CCD camera.
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
4
using a translating stage. Some experiments on finger surfaces were carried out to test the validity of the developed system.
3.1. Hardware setup The hardware consists of off-the-shelf products, including a DLP projector, a CCD camera and a laptop computer. According to the size of most people’s finger, a measuring field of 30 mm 20 mm was determined and a DLP Pico projector from Texas Instruments was chosen. Its main parameters are listed in Table 1. The structure of the projector is compact and the projecting screen size conforms to the requirements of measuring field. The CCD camera is from SVS-VISTEK in Germany with the model of ECO415CVGE. It has a lens interface of CS-mount, a compact size of 38 mm 38 mm 33 mm, a resolution of 780 580, a frame rate of 86 Hz at full resolution, and an interface of GigE. According to the measuring field, a Computar lens having fixed focus-length of 25 mm was chosen. The Computar lens has an interface of C-mount. Due to CS-mount of the chosen CCD camera, a tube with length of 5 mm needs to be added between the lens and camera. A laptop computer with the model ThinkPad X1 was used to control the DLP projector and the CCD camera, to capture and lively display the deformed fringe patterns on finger surface, as demonstrated in Fig. 3. The projector and the CCD camera were assembled together to get a mobile and portable 3D fingerprint imaging system, as the partial enlarged view illustrated in Fig. 4.
3.2. Software system In order to control the 3D fingerprint system, a software system was developed in Microsoft Visual C++6.0 environment based on the provided development program of the CCD camera. The acquisition function includes the following parts: parameter controlling of the CCD camera, parameter controlling of the DLP projector, lively displaying the captured image, and saving the images. All the functions were designed as a friendly Graphic User Interface (GUI), as illustrated in Fig. 5.
Fig. 7. Profiles of the three color sinusoidal fringe patterns in Fig. 6 along the middle row direction.
Fig. 5. GUI of the 3D fingerprint imaging capturing system.
Fig. 6. Color fringe patterns projected on the finger surface. (a) Red fringe pattern, (b) green fringe pattern and (c) blue fringe pattern. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
On the left of the GUI, it is a lively displaying area to show the captured fringe pattern image on finger surface, as illustrated in Fig. 5. There is a white cross in the middle of the generated fringe pattern during living display to locate the measured finger. During the procedure of acquisition, the projected fringe patterns will not contain the white cross to avoid its effect on the following phase calculation. The right parts of the GUI contain the adjustable
5
parameters to control the CCD camera and the projector, to show a living image or profile of the captured fringe pattern, to start capturing and to save the captured images.
Table 4 The average value, absolute error and standard deviation of the distance between all the neighboring markers in y direction (Units mm). Position Actual distance
Measured distance
Absolute error
Standard deviation
4.5 2.5 2.5 4.5
1.5004 1.4998 1.5014 1.4995
0.0004 0.0002 0.0014 0.0005
0.0126 0.0099 0.0170 0.0172
1.5 1.5 1.5 1.5
Fig. 8. The used checkerboard and white plate. (a) Photo of the checkerboard having black and white square with size of 1.5 mm 1.5 mm, and (b) photo of the white plate having 9 12 green hollow rings on it with neighboring separation of 1.5 mm along horizontal and vertical direction. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Table 2 Data evaluation of depth measurement for the positions 4.5 mm, 2.5 mm, 2.5 mm and 4.5 mm (Units mm). Position
Measured depth
Absolute error
Standard deviation
4.5 2.5 2.5 4.5
4.5025 2.5146 2.4758 4.4699
0.0025 0.0146 0.0242 0.0301
0.0012 0.0002 0.0005 0.0013
Table 3 The average value, absolute error and standard deviation of the distance between all the neighboring markers in x direction (Units mm). Position Actual distance
Measured distance
Absolute error
Standard deviation
4.5 2.5 2.5 4.5
1.5000 1.5005 1.4994 1.4997
0.0000 0.0005 0.0006 0.0003
0.0130 0.0145 0.0173 0.0172
1.5 1.5 1.5 1.5
Fig. 9. Green fringe pattern images on the measured index finger of the first author by projecting the optimum fringe numbers of 25, 24 and 20. (a) 25, (b) 24 and (c) 20. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
6
3.3. Color channel for fringe projection The interaction of biological tissue and different wavelength light is of great significant research in biological recognition application fields. Certain elements in human tissues, for example, fingerprint surface, have the selective absorption effect on a specific wavelength of light, which means the skin has different absorption characteristics to different wavelengths of light. When light strikes skin tissues, parts of photons on the skin surface are reflected and other parts into skin tissues. In order to test which color channel is suitable to project fringe pattern onto finger surface, some verification experiments were carried out. Three fringe patterns having the same fringe numbers were projected separately onto the first author’s finger surface through red, green and blue channels of the projector, respectively. Three deformed fringe patterns on the finger surface were captured by the color CCD camera from a different viewpoint, as illustrated in Fig. 6. In order to compare modulation of the captured fringe patterns from the three color channels, their profiles along the middle row direction are illustrated in Fig. 7. The dot dash, solid and dash lines correspond to red, green and blue color channels, respectively. It indicates that fringe pattern in the green channel has largest modulation, which can give much more accurate phase data than the other two color channels. Therefore, green fringe pattern will be used in the proposed 3D fingerprint imaging system.
3.4. 3D calibration and performance evaluation In order to calibrate the proposed 3D fingerprint imaging system, two plates were designed and manufactured by Ti-Times (http://www.ti-times.com). One is a checkerboard having 9 12 white and black checkers, as illustrated in Fig. 8(a). Each checker has a square size of 1.5 mm 1.5 mm. The checkerboard is for calibrating the internal parameters of the CCD camera. The other is a white plate having 9 12 discrete hollow ring markers, as illustrated in Fig. 8(b). The separation of neighboring markers along row and column direction has the same value of 1.5 mm as the checker size. The plate was randomly positioned nearly perpendicular to the imaging axis several times in the measuring volume for calibrating the 3D fingerprint imaging system.
The same white plate was placed on an accurate translating stage with resolution of 1 μm for evaluating the calibrated 3D fingerprint imaging system. The plate was positioned at 4.5 mm, 2.5mm, 2.5 mm and 4.5 mm with respect to a chosen reference plane. The measured average value, absolute error (absolute difference between the measured average distance and the distance by the stage) and standard deviation value along the middle row for the four positions were listed in Table 2. By using the calibrated depth data, the measured distance between neighboring ring markers, absolute error and standard deviation along x and y direction can be calculated by Eq. (5), as shown in Tables 3 and 4. The experimental results on the four plate positions show that the proposed 3D calibration method accurately converts not only absolute phase into depth data but also pixel positions into transverse x and y coordinates. 3.5. Experimental results The right index finger of the first author was tested by the developed 3D fingerprint imaging system. Twelve fringe patterns images having the optimum fringe numbers of 25, 24 and 20 were generated in the computer and projected by the DLP projector onto the finger surface. The CCD camera captured the deformed fringe pattern images from another view of point. Three captured fringe pattern images having the projected fringe numbers of 25, 24 and 20 are illustrated in Fig. 9. Applying four-step phase-shifting algorithm to the captured fringe pattern images obtained three wrapped phase maps, as illustrated in Fig. 10(a)–(c) corresponding to the projected fringe numbers of 25, 24 and 20, respectively. Fig. 10(d) demonstrates the absolute phase map by using the optimum three-frequency selection method. The absolute phase map is converted into actual 3D shape data by using the calibrated coefficients. Fig. 11 displays 3D representations of the finger by gradient shading and texturemapping from two different viewpoints. Using the proposed 3D sensor system, four fingerprints were captured and processed to obtain the 3D shape and color texture information, as illustrated in Fig. 12. Left and right are gradient shading and texture-mapping respectively in each panel. The experimental results clearly show that the 3D fingerprint imaging system based on the fringe projection technique correctly obtains the 3D shape data of fingerprints of human being.
Fig. 10. Calculated wrapped phase maps by four-step phase-shifting algorithm and the unwrapped phase map. The three images correspond to the projected fringe numbers of 25, 24 and 20. (a) 25, (b) 24, (c) 20 and (d) unwrapped phase map.
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
7
Fig. 11. 3D representation of the captured fingerprint. (a) Gradient shading, (b) 3D texture-mapping from one viewpoint. (c) Gradient shading, (d) 3D texture-mapping from another viewpoint.
Fig. 12. Shading and texture mapping of four different 3D fingerprints. "(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)".
4. Conclusions In this paper, a novel 3D fingerprint imaging system has been presented to acquire 3D shape of fingerprint by projecting sinusoidal fringe pattern onto finger surface. The principle of 3D fingerprint acquisition is introduced to obtain phase data corresponding to 3D shape of fingerprint. The optimum three-fringe number selection method has been utilized to independently calculate absolute phase pixel by pixel to avoid the effects of ridge and valley of fingerprint on determining the absolute fringe order. A compact CCD camera from SVS-VISTEK with model of ECO415CVGE and a suitable lens having a fixed focus-length of 25 mm have been chosen as the imaging system. A small size DLP
Pico projector from TI has been used as the fringe projecting system. Therefore, the 3D fingerprint imaging system has a compact size and light weight. The corresponding GUI software has been developed in VC++ environment by using the provided development program of the CCD camera to capture, lively display, save and process the fringe patterns. After comparing the fringe contrast in red, green and blue channels on finger surface, green channel has been chosen to project sinusoidal fringe pattern. The experimental data on some index fingers surface show that the proposed imaging system correctly obtains 3D accurate shape of fingerprint. The capturing time for a 3D fingerprint is about 0.5 s because the CCD camera and the DLP projector have not been completely
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
8
synchronized. Future work will be done in this area to reduce the capturing time to about 0.2 s. Acknowledgments The authors would like to thank the National Natural Science Foundation of China (61171048, 61311130138), Program for New Century Excellent Talents in University (No. NECT-11–0932), the Key Project of Chinese Ministry of Education (No. 211016), Specialized Research Fund for the Doctoral Program of Higher Education (“SRFDP”) (No. 20111317120002), the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry (NO. 20101561), Research Project supported by Hebei Education Department (No. ZD2010121). References [1] Jain AK. Biometric recognition. Nature 2007;449:38–40. [2] Jain AK, Ross A, Salil P. An introduction to biometric recognition. IEEE Trans Circ Syst Video Technol 2004;14:4–21. [3] Chen X, Tian J, Yang X, Zhang Y. An algorithm for distorted fingerprint matching based on local triangle feature set. IEEE Trans Inf Forensic Secur 2006;1:169–77. [4] Ramotowski RS. Composition of latent print residue. In: Lee HC, Gaensslen RE, editors. Advances in fingerprint technology. Boca Raton, FL: CRC Press; 2001. p. 63–104. [5] Bowyer K, Chang K, Flynn P. A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition. Comput Vis Image Und 2006;101:1–15. [6] Yan P, Bowyer K. Biometric recognition using 3D ear shape. IEEE Trans Pattern Anal 2007;29:1297–308.
[7] Chen H, Bhanu B. Human ear recognition in 3D. IEEE Trans Pattern Anal 2007;29:718–36. [8] Parziale G, Diaz-Santana E, Hauke R. A multi-camera touchless device to acquire 3D rolled-equivalent fingerprints. Int Conf Biom 2006:244–50. [9] Wang YC, Hassebrook LG, Lau DL. Data acquisition and processing of 3-D fingerprints. IEEE Trans Inf. Forensic Secur 2010;5:1–9. [10] Gabai H, Shaked NT. Dual-channel low-coherence interferometry and its application to quantitative phase imaging of fingerprints. Opt Express 2012;20:26906–12. [11] Zhang S. Recent progresses on real-time 3D shape measurement using digital fringe projection techniques. Opt Lasers Eng 2010;48:149–58. [12] Zhang ZH. Review of single-shot 3D shape measurement by phase calculationbased fringe projection techniques. Opt Lasers Eng 2012;50:1097–106. [13] Pan B, Kemao LHQ, Asundi A. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Opt Lett 2009;34:416–8. [14] Sansoni G, Carocci M, Rodella R. Calibration and performance evaluation of a 3-D imaging sensor based on the projection of structured light. IEEE Trans Instrum Meas 2000;49:628–36. [15] Creath K. Phase measurement interferometry techniques. In: Wolf E, editor. Progress in optics XXVI. Amsterdam: North Holland Publ; 1988. p. 339–93. [16] Zhang ZH, Towers CE, Towers DP. Time efficient colour fringe projection system for 3-D shape and colour using optimum 3-frequency interferometry. Opt Express 2006;14:6444–55. [17] Towers CE, Towers DP, Jones JDC. Absolute fringe order calculation using optimised multi-frequency selection in full-field porfilometry. Opt Lasers Eng 2005;43:788–800. [18] Yin YK, Peng X, Li A, Liu X, Gao Bruce Z. Calibration of fringe projection profilometry with bundle adjustment strategy. Opt Lett 2012;37:542–4. [19] Jia P, Kofman J, English C. Comparison of linear and nonlinear calibration methods for phase-measuring profilometry. Opt Eng 2007;46:043601. [20] Zhang ZH, Ma HY, Guo T, Zhang SX, Chen JP. Simple, flexible calibration of phase calculation-based three-dimensional imaging system. Opt Lett 2011;36:1257–9.
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
Contents lists available at SciVerse ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
3D fingerprint imaging system based on full-field fringe projection profilometry Shujun Huang, Zonghua Zhang n, Yan Zhao, Jie Dai, Chao Chen, Yongjia Xu, E. Zhang, Lili Xie School of Mechanical Engineering, Hebei University of Technology, Tianjin 300136, China
art ic l e i nf o
a b s t r a c t
Article history: Received 23 April 2013 Received in revised form 13 June 2013 Accepted 1 July 2013
As an unique, unchangeable and easily acquired biometrics, fingerprint has been widely studied in academics and applied in many fields over the years. The traditional fingerprint recognition methods are based on the obtained 2D feature of fingerprint. However, fingerprint is a 3D biological characteristic. The mapping from 3D to 2D loses 1D information and causes nonlinear distortion of the captured fingerprint. Therefore, it is becoming more and more important to obtain 3D fingerprint information for recognition. In this paper, a novel 3D fingerprint imaging system is presented based on fringe projection technique to obtain 3D features and the corresponding color texture information. A series of color sinusoidal fringe patterns with optimum three-fringe numbers are projected onto a finger surface. From another viewpoint, the fringe patterns are deformed by the finger surface and captured by a CCD camera. 3D shape data of the finger can be obtained from the captured fringe pattern images. This paper studies the prototype of the 3D fingerprint imaging system, including principle of 3D fingerprint acquisition, hardware design of the 3D imaging system, 3D calibration of the system, and software development. Some experiments are carried out by acquiring several 3D fingerprint data. The experimental results demonstrate the feasibility of the proposed 3D fingerprint imaging system. & 2013 Elsevier Ltd. All rights reserved.
Keywords: 3D fingerprint 3D biometrics Fringe projection profilometry Optimum three-fringe numbers selection
1. Introduction Biometrics means recognizing (verifying or identifying) a person by using the inherent physiological or behavioral characteristics of human body [1,2]. Physiological characteristics refer to data directly measured from the human body parts and some representative physiological features include fingerprints, hand shape, palmprint, face, iris, retina, and so on; while behavior characteristic is a measure of personal habitual action indirect from human body characteristics, such as voice, keystroke habits and signature. Human recognition based on biometrics is widely applied in many fields like government, army, bank, social welfare safeguard, e-commerce, security defense, and to name a few. Fingerprint, as one of the most important biometrics, has been widely studied and applied to personal recognition in both forensics and civilian. It is a unique and unchangeable physiological characteristic of human being during the whole life. The traditional 2D fingerprint has been captured by using an ink-based offline method or an imaging device, for example a CCD camera, to get an image for recognition. These methods need the subject to press (roll) his/her fingers against a surface to obtain 2D fingerprint image, so that the captured 2D fingerprint images are often
n
Corresponding author. Tel.: +86 22 265 824 03; fax: +86 22 602 041 89. E-mail addresses: [email protected], [email protected] (Z. Zhang).
distorted in a nonlinear way. Such nonlinear distortion increases the intra-class variations among the fingerprint images of the same finger and introduces matching errors. The existing 2D methods are also greatly affected by the brightness and contrast of environmental light, the dirty things on finger surface, so the matching results are inaccurate. Due to 3D to 2D mapping, the captured fingerprint features by using 2D methods have unavoidable distortion and illumination shading [3,4]. Therefore, identification and verification are based on the captured inaccurate 2D fingerprint biometrics. Jain [1] pointed out that “Efforts are afoot to design better biometric sensors/readers, to improve algorithms to extract features from raw biometric data and to match two biometric samples quickly and accurately”. In fact, fingerprint is a 3D biometric feature and has many advantages in comparison to 2D biometrics. Firstly, due to non-contact operation, there is no elastic deformation of the obtained features; secondly, 3D fingerprint can obtain the characteristic distribution patterns of fingerprint without distortion; thirdly, they give local space coordinate geometry and azimuth of fingerprint in details; finally, they are insensitive to brightness, contrast of environmental light, and dirty things on finger surface. With the development of CCD cameras and DLP (Digital Light Processing) projectors, 3D biometrics have been widely studied in recent years. 3D face recognition [5] is in a period of rapid expansion and 3D ear recognition [6,7] is in the very early stage. A few researchers have attempted 3D fingerprint as biometrics.
0143-8166/$ - see front matter & 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.optlaseng.2013.07.001
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
2
Parziale et al. [8] reported one surround imager by using multicamera system to acquire different finger views, so that 3D representation of fingerprint is obtained by using shape-fromsilhouette method. Recently, a new type of touchless 3D fingerprint identification access control system is developed by TBS in Switzerland (www.tbs-biometrics.com). It greatly improved the accuracy of access control and attendance system. Wang et al. [9] employed a noncontact 3D method based on phase measuring profilometry (PMP) to acquire 3D ridge-depth information of human finger. Gabai et al. [10] introduced a dual-channel interferometric imaging system to obtain phase imaging of fingerprints. Due to the speckles in interferometry, the obtained phase and then shape information are inaccurate. This paper presents a new imaging system to get 3D fingerprint data based on the sinusoidal fringe projection technique [11–14]. The system includes a color CCD camera, a small DLP projector and a laptop computer. The straight sinusoidal fringe patterns are generated by software in the computer and projected onto the finger surface by the DLP projector. From another viewpoint, the fringe patterns appear deformed with regard to the finger surface shape. The modulated fringe patterns are recorded by the CCD camera and saved into the computer for post-processing. Combining four-step phase-shifting algorithm [15] and optimum threefringe number selection [16–17] can independently calculate absolute phase at each pixel position. An automatic 3D calibration method builds up the relationship between absolute phase map and 3D shape, so that 3D fingerprint data can be obtained. The following section introduces the principle of 3D fingerprint imaging system based on absolute phase measurement. Some experiments on capturing 3D fingerprint are demonstrated in Section 3. Section 4 gives the conclusions and future directions.
sinusoidal fringe patterns by software DLP projector finger surface CCD camera deformed fringe patterns four-step phase shifting three wrapped phase maps optimum 3 number selection absolute unwrapped phase map 3D calibration of system x,y,z data of finger surface 3D representation by shading Fig. 1. Flowchart of 3D fingerprint data acquisition and processing.
2. Principle of 3D fingerprint system Phase calculation-based full-field sinusoidal fringe projection technique is used to obtain 3D shape of fingerprint. Three wrapped maps are calculated by using four-step phase-shifting algorithm and optimum three-fringe number selection method is applied to independently calculate the absolute phase information pixel by pixel. After 3D calibrating the system, the absolute phase at each pixel position can be converted into x, y, and z coordinates. Fig. 1 shows the flowchart of 3D fingerprint data acquisition and processing. 2.1. Components of the 3D imaging system The proposed 3D fingerprint imaging system is based on the full-field sinusoidal fringe projection technique, as demonstrated in Fig. 2. It consists of a DLP projector, a color CCD camera and a laptop computer. The DLP projector and the CCD camera satisfy a conventional triangulation arrangement. Sinusoidal fringe patterns are generated by software in the computer and projected onto a measured finger surface by the DLP projector. From another different viewpoint, the fringe patterns are deformed with respect to the finger surface and ridges. The CCD camera captures the deformed fringe patterns and saves them into the computer for post-processing. 2.2. Four-step phase-shifting algorithm Multiple-step phase-shifting algorithms have been widely used to calculate phase information because of their high accuracy. Among them, four-step phase-shifting algorithm is one of the most used phase calculation method in fringe pattern processing and has gained many achievements in research and industrial
Fig. 2. System schematic diagram. (1) Computer, (2) CCD Camera, (3) DLP projector, and (4) finger surface.
fields [15]. There are π=2 shift in between for the four fringe patterns. In order to make this paper self-contained, the mathematical model and distribution characteristics of fringe patterns are briefly described. The intensity distribution of a captured fringe pattern can be expressed by the following mathematical expression: Iðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φðm; nÞ þ I n ðm; nÞ
ð1Þ
where m; n are the index of a pixel in the captured image along vertical and horizontal direction, respectively; Iðm; nÞ is the measured intensity at (m, n) pixel position; I d ðm; nÞ and I m ðm; nÞ are the intensity of background and the fringe modulation, respectively; φðm; nÞ is the phase corresponding to the measured object shape; I n ðm; nÞ is he additive random noise during capturing process. Four-step phase-shifting algorithm means four fringe patterns have π=2 phase shift in between are successively projected into the
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
measuring volume and they can be expressed as I 1 ðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φ½ðm; nÞ þ 0 þ I n ðm; nÞ
ð2:AÞ
I 2 ðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φ½ðm; nÞ þ π=2 þ I n ðm; nÞ
ð2:BÞ
I 3 ðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φ½ðm; nÞ þ π þ I n ðm; nÞ
ð2:CÞ
I 4 ðm; nÞ ¼ I d ðm; nÞ þ I m ðm; nÞ cos φ½ðm; nÞ þ 3π=2 þ I n ðm; nÞ
ð2:DÞ
3
compensated, there are still uncorrected distortions from the projecting lens of the DLP projector. Transverse calibration relates to depth information obtained from the projected fringe patterns. Therefore, at each pixel position the following two polynomial equations are used to compensate for the radical distortion to give a high accurate relationship: ( xr ¼ a0 ðm; nÞz2r þ b0 ðm; nÞzr þ c0 ð5Þ yr ¼ a1 ðm; nÞz2r þ b1 ðm; nÞzr þ c1
Assuming the four captured fringe patterns in the same optical field have the same gray value of background, modulation and noise, the phase field φðm; nÞ can be accurately calculated by the following equation: I 4 ðm; nÞI 2 ðm; nÞ ð3Þ ϕðm; nÞ ¼ arctan I 3 ðm; nÞI 1 ðm; nÞ
where a0 ; b0 ; c0 ; a1 ; b1 ; c1 are the coefficient set of the system parameters and xr ; yr ; zr are the coordinates in the reference coordinate system. All the coefficient sets in Eqs. (4) and (5) can be calibrated by a white plate having known discrete markers on surface.
Because of the property of trigonometric function, the obtained phase is in the range of π=2–π=2 in mathematics and then been changed in the range of π–π, which needs to be unwrapped to retrieve the continuous phase map for 3D shape measurement.
3. System and experiments
2.3. Optimum three-fringe number selection The traditional spatial phase unwrapping methods easily accumulate errors and cannot give correct fringe order for discontinuous objects. There are many ridges and valleys on finger surface, so the normal spatial phase unwrapping methods are not suitable to measure 3D fingerprint shape. The optimum three-fringe number selection method [16,17] can solve this problem since it calculates absolute phase pixel by pixel. The optimum three-fringe number selection method defines pffiffiffiffi the numbers of projected fringes to be N; N1; N N . For example, if N ¼ 25, the other two fringe sets have fringe numbers of 24 and 20. This method resolves fringe order ambiguity as the beat obtained between N and N1 which is a single fringe over the full field of view. The reliability of the obtained fringe order is maximized as the fringe order calculation is performed through a geometric series of beat fringes with 1, 5 and 25 fringes. The obtained absolute phase data are unwrapped pixel by pixel, so that this method can measure objects having discontinuities and/or isolated surfaces.
Based on the theoretical analysis, a 3D fingerprint imaging system was designed and developed in the following subsections. The system was 3D calibrated by an accurately manufactured white plate and the performance of calibration was evaluated by Table 1 Parameters of the DLP Pico projector. Nominal brightness Standard resolution Light source Frame rate Shape dimension
10 lm 640 480 Solid-State 3 LED 60 Hz for NTSC 68 mm 45 mm 14 mm
2.4. System 3D calibration The obtained absolute phase map corresponds to shape of the measured finger surface. In order to get shape information, the system needs to be calibrated to build up the relationship not only between depth and absolute phase, but x, y coordinates and pixel positions, which is called the depth calibration and transverse calibration, respectively [18,19]. Phase to depth conversion can be represented by the following polynomial equation at each pixel position [20]: K
zðm; nÞ ¼ ∑ ak ðm; nÞφðm; nÞk k¼0
Fig. 3. Hardware setup of the 3D fingerprint imaging system including a laptop computer, a Pico projector and a color CCD camera.
ð4Þ
where ak ; ak1 ; :::a2 ; a1 are a coefficient set containing the systematic parameters. Depth calibration means to determine the coefficient set of the polynomial equation. Because the coefficient set of Eq. (4) depends on pixel position (m, n) and will have different values, a Look-Up Table (LUT) needs to be used at each pixel position to save the coefficient set. Except for depth data, 3D shape needs the transverse x and y coordinates as well, which can be realized by a procedure of transverse calibration. The relationship is linear for an ideal optical imaging system. However, for an actual 3D imaging system, the relationship is nonlinear because of the distortion of optical imaging and projecting lens. Although the imaging distortion can
Fig. 4. The partial enlarged view of the Pico projector and the color CCD camera.
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
4
using a translating stage. Some experiments on finger surfaces were carried out to test the validity of the developed system.
3.1. Hardware setup The hardware consists of off-the-shelf products, including a DLP projector, a CCD camera and a laptop computer. According to the size of most people’s finger, a measuring field of 30 mm 20 mm was determined and a DLP Pico projector from Texas Instruments was chosen. Its main parameters are listed in Table 1. The structure of the projector is compact and the projecting screen size conforms to the requirements of measuring field. The CCD camera is from SVS-VISTEK in Germany with the model of ECO415CVGE. It has a lens interface of CS-mount, a compact size of 38 mm 38 mm 33 mm, a resolution of 780 580, a frame rate of 86 Hz at full resolution, and an interface of GigE. According to the measuring field, a Computar lens having fixed focus-length of 25 mm was chosen. The Computar lens has an interface of C-mount. Due to CS-mount of the chosen CCD camera, a tube with length of 5 mm needs to be added between the lens and camera. A laptop computer with the model ThinkPad X1 was used to control the DLP projector and the CCD camera, to capture and lively display the deformed fringe patterns on finger surface, as demonstrated in Fig. 3. The projector and the CCD camera were assembled together to get a mobile and portable 3D fingerprint imaging system, as the partial enlarged view illustrated in Fig. 4.
3.2. Software system In order to control the 3D fingerprint system, a software system was developed in Microsoft Visual C++6.0 environment based on the provided development program of the CCD camera. The acquisition function includes the following parts: parameter controlling of the CCD camera, parameter controlling of the DLP projector, lively displaying the captured image, and saving the images. All the functions were designed as a friendly Graphic User Interface (GUI), as illustrated in Fig. 5.
Fig. 7. Profiles of the three color sinusoidal fringe patterns in Fig. 6 along the middle row direction.
Fig. 5. GUI of the 3D fingerprint imaging capturing system.
Fig. 6. Color fringe patterns projected on the finger surface. (a) Red fringe pattern, (b) green fringe pattern and (c) blue fringe pattern. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
On the left of the GUI, it is a lively displaying area to show the captured fringe pattern image on finger surface, as illustrated in Fig. 5. There is a white cross in the middle of the generated fringe pattern during living display to locate the measured finger. During the procedure of acquisition, the projected fringe patterns will not contain the white cross to avoid its effect on the following phase calculation. The right parts of the GUI contain the adjustable
5
parameters to control the CCD camera and the projector, to show a living image or profile of the captured fringe pattern, to start capturing and to save the captured images.
Table 4 The average value, absolute error and standard deviation of the distance between all the neighboring markers in y direction (Units mm). Position Actual distance
Measured distance
Absolute error
Standard deviation
4.5 2.5 2.5 4.5
1.5004 1.4998 1.5014 1.4995
0.0004 0.0002 0.0014 0.0005
0.0126 0.0099 0.0170 0.0172
1.5 1.5 1.5 1.5
Fig. 8. The used checkerboard and white plate. (a) Photo of the checkerboard having black and white square with size of 1.5 mm 1.5 mm, and (b) photo of the white plate having 9 12 green hollow rings on it with neighboring separation of 1.5 mm along horizontal and vertical direction. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Table 2 Data evaluation of depth measurement for the positions 4.5 mm, 2.5 mm, 2.5 mm and 4.5 mm (Units mm). Position
Measured depth
Absolute error
Standard deviation
4.5 2.5 2.5 4.5
4.5025 2.5146 2.4758 4.4699
0.0025 0.0146 0.0242 0.0301
0.0012 0.0002 0.0005 0.0013
Table 3 The average value, absolute error and standard deviation of the distance between all the neighboring markers in x direction (Units mm). Position Actual distance
Measured distance
Absolute error
Standard deviation
4.5 2.5 2.5 4.5
1.5000 1.5005 1.4994 1.4997
0.0000 0.0005 0.0006 0.0003
0.0130 0.0145 0.0173 0.0172
1.5 1.5 1.5 1.5
Fig. 9. Green fringe pattern images on the measured index finger of the first author by projecting the optimum fringe numbers of 25, 24 and 20. (a) 25, (b) 24 and (c) 20. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
6
3.3. Color channel for fringe projection The interaction of biological tissue and different wavelength light is of great significant research in biological recognition application fields. Certain elements in human tissues, for example, fingerprint surface, have the selective absorption effect on a specific wavelength of light, which means the skin has different absorption characteristics to different wavelengths of light. When light strikes skin tissues, parts of photons on the skin surface are reflected and other parts into skin tissues. In order to test which color channel is suitable to project fringe pattern onto finger surface, some verification experiments were carried out. Three fringe patterns having the same fringe numbers were projected separately onto the first author’s finger surface through red, green and blue channels of the projector, respectively. Three deformed fringe patterns on the finger surface were captured by the color CCD camera from a different viewpoint, as illustrated in Fig. 6. In order to compare modulation of the captured fringe patterns from the three color channels, their profiles along the middle row direction are illustrated in Fig. 7. The dot dash, solid and dash lines correspond to red, green and blue color channels, respectively. It indicates that fringe pattern in the green channel has largest modulation, which can give much more accurate phase data than the other two color channels. Therefore, green fringe pattern will be used in the proposed 3D fingerprint imaging system.
3.4. 3D calibration and performance evaluation In order to calibrate the proposed 3D fingerprint imaging system, two plates were designed and manufactured by Ti-Times (http://www.ti-times.com). One is a checkerboard having 9 12 white and black checkers, as illustrated in Fig. 8(a). Each checker has a square size of 1.5 mm 1.5 mm. The checkerboard is for calibrating the internal parameters of the CCD camera. The other is a white plate having 9 12 discrete hollow ring markers, as illustrated in Fig. 8(b). The separation of neighboring markers along row and column direction has the same value of 1.5 mm as the checker size. The plate was randomly positioned nearly perpendicular to the imaging axis several times in the measuring volume for calibrating the 3D fingerprint imaging system.
The same white plate was placed on an accurate translating stage with resolution of 1 μm for evaluating the calibrated 3D fingerprint imaging system. The plate was positioned at 4.5 mm, 2.5mm, 2.5 mm and 4.5 mm with respect to a chosen reference plane. The measured average value, absolute error (absolute difference between the measured average distance and the distance by the stage) and standard deviation value along the middle row for the four positions were listed in Table 2. By using the calibrated depth data, the measured distance between neighboring ring markers, absolute error and standard deviation along x and y direction can be calculated by Eq. (5), as shown in Tables 3 and 4. The experimental results on the four plate positions show that the proposed 3D calibration method accurately converts not only absolute phase into depth data but also pixel positions into transverse x and y coordinates. 3.5. Experimental results The right index finger of the first author was tested by the developed 3D fingerprint imaging system. Twelve fringe patterns images having the optimum fringe numbers of 25, 24 and 20 were generated in the computer and projected by the DLP projector onto the finger surface. The CCD camera captured the deformed fringe pattern images from another view of point. Three captured fringe pattern images having the projected fringe numbers of 25, 24 and 20 are illustrated in Fig. 9. Applying four-step phase-shifting algorithm to the captured fringe pattern images obtained three wrapped phase maps, as illustrated in Fig. 10(a)–(c) corresponding to the projected fringe numbers of 25, 24 and 20, respectively. Fig. 10(d) demonstrates the absolute phase map by using the optimum three-frequency selection method. The absolute phase map is converted into actual 3D shape data by using the calibrated coefficients. Fig. 11 displays 3D representations of the finger by gradient shading and texturemapping from two different viewpoints. Using the proposed 3D sensor system, four fingerprints were captured and processed to obtain the 3D shape and color texture information, as illustrated in Fig. 12. Left and right are gradient shading and texture-mapping respectively in each panel. The experimental results clearly show that the 3D fingerprint imaging system based on the fringe projection technique correctly obtains the 3D shape data of fingerprints of human being.
Fig. 10. Calculated wrapped phase maps by four-step phase-shifting algorithm and the unwrapped phase map. The three images correspond to the projected fringe numbers of 25, 24 and 20. (a) 25, (b) 24, (c) 20 and (d) unwrapped phase map.
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
7
Fig. 11. 3D representation of the captured fingerprint. (a) Gradient shading, (b) 3D texture-mapping from one viewpoint. (c) Gradient shading, (d) 3D texture-mapping from another viewpoint.
Fig. 12. Shading and texture mapping of four different 3D fingerprints. "(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)".
4. Conclusions In this paper, a novel 3D fingerprint imaging system has been presented to acquire 3D shape of fingerprint by projecting sinusoidal fringe pattern onto finger surface. The principle of 3D fingerprint acquisition is introduced to obtain phase data corresponding to 3D shape of fingerprint. The optimum three-fringe number selection method has been utilized to independently calculate absolute phase pixel by pixel to avoid the effects of ridge and valley of fingerprint on determining the absolute fringe order. A compact CCD camera from SVS-VISTEK with model of ECO415CVGE and a suitable lens having a fixed focus-length of 25 mm have been chosen as the imaging system. A small size DLP
Pico projector from TI has been used as the fringe projecting system. Therefore, the 3D fingerprint imaging system has a compact size and light weight. The corresponding GUI software has been developed in VC++ environment by using the provided development program of the CCD camera to capture, lively display, save and process the fringe patterns. After comparing the fringe contrast in red, green and blue channels on finger surface, green channel has been chosen to project sinusoidal fringe pattern. The experimental data on some index fingers surface show that the proposed imaging system correctly obtains 3D accurate shape of fingerprint. The capturing time for a 3D fingerprint is about 0.5 s because the CCD camera and the DLP projector have not been completely
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i
S. Huang et al. / Optics and Lasers in Engineering ∎ (∎∎∎∎) ∎∎∎–∎∎∎
8
synchronized. Future work will be done in this area to reduce the capturing time to about 0.2 s. Acknowledgments The authors would like to thank the National Natural Science Foundation of China (61171048, 61311130138), Program for New Century Excellent Talents in University (No. NECT-11–0932), the Key Project of Chinese Ministry of Education (No. 211016), Specialized Research Fund for the Doctoral Program of Higher Education (“SRFDP”) (No. 20111317120002), the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry (NO. 20101561), Research Project supported by Hebei Education Department (No. ZD2010121). References [1] Jain AK. Biometric recognition. Nature 2007;449:38–40. [2] Jain AK, Ross A, Salil P. An introduction to biometric recognition. IEEE Trans Circ Syst Video Technol 2004;14:4–21. [3] Chen X, Tian J, Yang X, Zhang Y. An algorithm for distorted fingerprint matching based on local triangle feature set. IEEE Trans Inf Forensic Secur 2006;1:169–77. [4] Ramotowski RS. Composition of latent print residue. In: Lee HC, Gaensslen RE, editors. Advances in fingerprint technology. Boca Raton, FL: CRC Press; 2001. p. 63–104. [5] Bowyer K, Chang K, Flynn P. A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition. Comput Vis Image Und 2006;101:1–15. [6] Yan P, Bowyer K. Biometric recognition using 3D ear shape. IEEE Trans Pattern Anal 2007;29:1297–308.
[7] Chen H, Bhanu B. Human ear recognition in 3D. IEEE Trans Pattern Anal 2007;29:718–36. [8] Parziale G, Diaz-Santana E, Hauke R. A multi-camera touchless device to acquire 3D rolled-equivalent fingerprints. Int Conf Biom 2006:244–50. [9] Wang YC, Hassebrook LG, Lau DL. Data acquisition and processing of 3-D fingerprints. IEEE Trans Inf. Forensic Secur 2010;5:1–9. [10] Gabai H, Shaked NT. Dual-channel low-coherence interferometry and its application to quantitative phase imaging of fingerprints. Opt Express 2012;20:26906–12. [11] Zhang S. Recent progresses on real-time 3D shape measurement using digital fringe projection techniques. Opt Lasers Eng 2010;48:149–58. [12] Zhang ZH. Review of single-shot 3D shape measurement by phase calculationbased fringe projection techniques. Opt Lasers Eng 2012;50:1097–106. [13] Pan B, Kemao LHQ, Asundi A. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Opt Lett 2009;34:416–8. [14] Sansoni G, Carocci M, Rodella R. Calibration and performance evaluation of a 3-D imaging sensor based on the projection of structured light. IEEE Trans Instrum Meas 2000;49:628–36. [15] Creath K. Phase measurement interferometry techniques. In: Wolf E, editor. Progress in optics XXVI. Amsterdam: North Holland Publ; 1988. p. 339–93. [16] Zhang ZH, Towers CE, Towers DP. Time efficient colour fringe projection system for 3-D shape and colour using optimum 3-frequency interferometry. Opt Express 2006;14:6444–55. [17] Towers CE, Towers DP, Jones JDC. Absolute fringe order calculation using optimised multi-frequency selection in full-field porfilometry. Opt Lasers Eng 2005;43:788–800. [18] Yin YK, Peng X, Li A, Liu X, Gao Bruce Z. Calibration of fringe projection profilometry with bundle adjustment strategy. Opt Lett 2012;37:542–4. [19] Jia P, Kofman J, English C. Comparison of linear and nonlinear calibration methods for phase-measuring profilometry. Opt Eng 2007;46:043601. [20] Zhang ZH, Ma HY, Guo T, Zhang SX, Chen JP. Simple, flexible calibration of phase calculation-based three-dimensional imaging system. Opt Lett 2011;36:1257–9.
Please cite this article as: Huang S, et al. 3D fingerprint imaging system based on full-field fringe projection profilometry. Opt Laser Eng (2013), http://dx.doi.org/10.1016/j.optlaseng.2013.07.001i