- Email: [email protected]
3D shape from phase errors by using binary fringe with multi-step phase-shift technique
Optics and Lasers in Engineering 74 (2015) 22–27
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
3D shape from phase errors by using binary fringe with multi-step phase-shift technique Yuankun Liu n, Qican Zhang, Xianyu Su Opto-Electronic Department, Sichuan University, Chengdu 610064, China
art ic l e i nf o
a b s t r a c t
Article history: Received 1 December 2014 Received in revised form 1 April 2015 Accepted 23 April 2015 Available online 27 May 2015
A three-dimensional shape measurement method is presented, which is a uniaxial measurement by measuring phase errors instead of the well-known phase, modulation or contrast. A sequence of exposures are captured by using a multi-step phase-shift technique with the binary fringes. Then the high-accuracy phases can be obtained by using all the exposures, meanwhile, a set of low-accuracy phases can be calculated by dividing those exposures into a set of four-step phase-shift measurements. For each pixel there will be a set of phase errors by subtracting low-accuracy phases from the highaccuracy ones. And the weighted phase error of every pixel can be calculated. Meanwhile the phase error caused by the improperly defocused binary fringes has a unique relationship with the depth z. Therefore, the 3D information of every pixel can be obtained by analyzing the phase errors. It will be promising for a uniaxial measurement, such as deep holes. & 2015 Elsevier Ltd. All rights reserved.
Keywords: Fringe analysis Uniaxial Binary defocusing Three-dimensional sensing Phase error
1. Introduction Three-dimensional shape measurement based on sinusoidal fringe projection techniques has been widely used. Principally there are two categories of 3D shape measurement methods, one is to form a triangle for depth recovery [1], therefore the depth accuracy is based on the angle between the projection line and the camera imaging line as well as the phase accuracy, and it is not easy to measure the holes with the influence of occlusions. The other is to measure the steep object by setting the angle to be zero, which is called the uniaxial measurement by measuring modulation or contrast [2–4]. Because the modulation or contrast has a unique relationship with the depth z. Therefore by z-direction scanning, the modulation or contrast curve will give the depth. Recently Xu and Zhang [5] introduced a novel method, which is to use the phase errors as the signals to retrieve the 3D information. Because the phase error caused by the improperly defocused binary fringes has a unique relationship with the depth z. As a matter of fact the phase errors come from the high-order harmonics, whose magnitudes decrease with the increase of the defocused degree. In Ying's method, they used the three-step phaseshift technique to obtain the low accuracy phases from the binary
n
Corresponding author. E-mail address: [email protected] (Y. Liu).
http://dx.doi.org/10.1016/j.optlaseng.2015.04.014 0143-8166/& 2015 Elsevier Ltd. All rights reserved.
fringes and the ideal phases from the perfect sinusoidal fringes. Then the phase error can be calculated by the two measurements. There are six waves in the phase error map. They found the locations of the peaks and valleys of the six waves, then the 3D information of those peaks and valleys can be restored from the relationship between the phase errors and depth z. However, by using the three-step phase-shift technique, the highest magnitude high-order harmonic, i.e. the third-order harmonic was eliminated. Only those certain pixels, whose ideal phases are around 7 (2nk þ1)π/12, k¼1,...,5, can get the 3D information. The perfect sinusoidal fringes need the projector to cast the gray level patterns, which may reduce the ability of high speed and need to calibrate the systematic gamma. In this paper, we employ the multi-step phase-shift technique with projecting binary fringes. The perfect phases can be obtained by all exposures with the multi-step phase-shift technique. In order to use the third-order harmonic, we choose four-step phaseshift technique to calculate the low accuracy phases. The multistep phase-shift exposures will form a set of four-step phase-shift measurements, then a set of low accuracy phases will be obtained, i.e., for each pixel, there will be a set of phase errors, therefore we can calculate a phase error for every pixel, finally, the 3D information can be restored pixel by pixel with the known relationship between phase errors and depth z. The paper is organized as follows. In Section 2, the principle of the proposed technique is introduced. The experimental results
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
23
2. Principle
For the binary structured pattern, in the Fourier cosine series, the amounts of the third and fifth harmonics are 1/3 and 1/5 of the fundamental component. Therefore, the phase errors will become larger by using the four-step phase-shift technique. From Eq. 3 by using four-step technique we get
2.1. Multi-step phase-shift technique with binary fringes
ϕ0 ðx; yÞ ¼ arctan P3n ¼ 0
of the setup are presented in Section 3. Summary is shown in Section 4.
"P
Iðx; yÞ ¼ Aðx; yÞ þ Bðx; yÞ
C j cos ðk2π f 0 x þ ϕk0 Þ
ð1Þ
k ¼ 2m 1
where A(x,y) is the background intensity, B(x,y) is the fringe modulation, Cj is the magnitude of every component. Noted that there are only odd-order harmonics. Let's assume ϕ ¼ 2π f 0 x þ ϕ0 , then the each particular measurement intensity In can be rewritten by [7] I n ðx; yÞ ¼ a0 þ
1 X
ak cos ½kðϕðx; yÞ þ δn Þ
fa0 þ
n ¼ 0 fa0 þ
In phase-shifting technique, when projecting the binary fringes, the solution to achieve high accuracy phases is to increase the amount of exposures, which could not only reduce the random noise but also reduce the systematic errors including the highorder harmonics. In extreme case, one-pixel phase-shift can be used. The deformed binary fringes can be expressed as a Fourier cosine series by [6] 1 X
3
ð2Þ
k ¼ 2m 1
P1
k ¼ 2m 1 ak P1 k ¼ 2m 1 ak
cos ½kðϕðx; yÞ þ δn Þg sin ð2π n=4Þ
#
cos ½kðϕðx; yÞ þ δn Þg cos ð2π n=4Þ
ð4Þ Then the phase error will be
Δϕ ¼ ϕ0 ðx; yÞ ϕðx; yÞ
¼ arctan
ða3 a5 Þ sin ½4ϕðx; yÞ þ ða7 a9 Þ sin ½8ϕðx; yÞ þ ⋯ a1 þ ða3 þ a5 Þ cos ½4ϕðx; yÞ þða7 þ a9 Þ cos ½8ϕðx; yÞ þ ⋯
ð5Þ Because the amounts of the third and fifth harmonics are much larger than the rest harmonics, therefore four waves are expected within the phase errors. The phase errors with the three-step technique will demonstrate six waves in the phase errors, because the lowest harmonic is the fifth one. The results are shown in Fig. 1 by using the three-step and four-step phase-shift techniques with the ideal binary structured patterns. The curve is the phase errors versus the wrapped ideal phases. Obviously the four-step phaseshift technique contains more phase errors. 2.3. Phase error determination pixel by pixel
It is known that a multi-step phase-shift technique will reduce almost all high-order harmonics, e.g. the period of the grating is 36 pixels, then a 36-step phase-shift technique will be used to decipher the perfect phase. With N-phase-shifts, the algorithm for the perfect phase ϕðx; yÞ deciphering has a form of "P # N 1 I sin ð2π n=NÞ ϕðx; yÞ ¼ arctan PNn ¼ 10 n ð3Þ n ¼ 0 I n cos ð2π n=NÞ 2.2. Four-step phase-shift technique to decipher the phase errors
In Fig. 1, it shows that there are some pixels in the phase map, whose phase errors are at peaks or valleys. In Ying's work they used these pixels to restore the depth z by using the relationships between phase errors, ideal phases and depth z. Obviously there are always some pixels, whose phase errors are always small or even zero. Then it is difficult to get the 3D information of these pixels. As a matter of fact, with the multi-step exposures, a set of Table 1 Sensitivity of different phase-shifting algorithms to harmonics. Number of steps
A defocusing optical system can act as a low-pass filter system, which can dump the higher harmonic components. But if the projector is not properly defocused, there are still binary structures on the resultant fringe patterns, and the phase error is significant. It is known that there are only the 3n2m 71th order harmonics in three-step phase-shift technique and only the 5n2 m 7 1th order harmonics in five-step method, but all harmonics remain by using the four-step method, as shown in Table 1.
Harmonics 2
3 4 5 6 7 8
3
4
5
6
7
8
Fig. 1. The phase errors versus wrapped phase, (a) three-step phase-shift, and (b) four-step phase-shift.
9
10
11
12
13
14
15
24
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
four step phase-shift measurements can be formed, the process is shown in Table 2. And then we can calculate one phase error for each four-step phase-shift measurement. Therefore there will be a set of phase errors. The phase values for each four-step phase-shift measurement can be expressed by "P # 3 I sin ½2π ðnN s þ m 1Þ=N ϕm ðx; yÞ ¼ arctan P3n ¼ 0 nNs þ m n ¼ 0 I nN s þ m cos ½2π ðnN s þm 1Þ=NÞ m ¼ 1; :::; N s Ns ¼ INTðN=4Þ
ð6Þ
From Eq. (5) and Eq. (6), we can get Ns phase errors for each pixel, which can be expressed by
Δϕm ðx; yÞ ¼ ϕðx; yÞ ϕm ðx; yÞ " ¼ arctan
# ða3 a5 Þ sin ½4ϕðx; yÞ þ 42πNm þ ða7 a9 Þ sin ½8ϕðx; yÞ þ 82πNm þ ⋯ a1 þ ða3 þ a5 Þ cos ½4ϕðx; yÞ þ 42πNm þ ða7 þ a9 Þ cos ½8ϕðx; yÞ þ 82πNm þ ⋯
ð7Þ For the 36 phase-shift application, all 36 exposures can form nine four-step phase-shift measurements, and the phase interval between two neighboring measurements is 2π/9. Then we can get nine phase errors for every pixel. Fig. 2a shows that there are nine curves totally, and each of them is shifted by a certain phase 2π/9. Fig. 2b shows the nine phase errors of one pixel. The nine phase errors can form a wave for a single pixel as shown in Fig. 2b. For a real projecting and imaging system, the high-order harmonics will be suppressed more or less. Let's assume that the wave is a sinusoidal function, then the modulation calculation of the well-known N-shift technique can be used to find the weighted value of the phase errors. Then the formula is
described as vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi #2 " #2 u" NX u NX s 1 s 1 2 t ~ Δϕ ¼ Δϕ sin ð2π k=N s Þ þ Δϕ cos ð2π k=N s Þ Ns k¼0 k¼0 ð8Þ Noted that with respect to the sampling theory, at least three phase errors for each pixel are needed to calculate the final phase error by Eq. (8). Then the minimum amount of the projected patterns is 3 4 ¼12, no matter what the period of the binary pattern is. But the phase-shift is pixel based, therefore the valid amount of the projected patterns is 12, 18 or 36 when the period of the binary pattern is 36. And more patterns will produce the more accurate phase value and more phase-errors, then 36 patterns are used in this paper.
2.4. Phase error and depth z mapping It is clear that the phase errors will decrease with the increase of the defocus degree [8,9], and the degree of defocusing has a unique relationship with the depth distance (z) [5]. Therefore the depth information can then be retrieved from the degree of defocusing. In order words, the depth (z) of the object points can be written as functions of the phase error, ~ ðx; yÞÞ z ¼ f ð Δϕ
ð9Þ
This error function can be determined by calibration. Once this function is known, the depth information of all pixels can be measured.
Table 2 A set of four-step phase-shift measurements by pixel–pixel phase-shifting with a period of grating 36 pixels. Four-step phase-shift
Exposures 1
1 2 3 4 ... 9
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
...
32
33
34
35
36
... ... ... ... ...
Fig. 2. Phase errors for pixels, (a) nine phase-error curves, and (b) nine phase errors for one single pixel.
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
3. Experimental results The setup is shown in Fig. 3, where the projector sits below the camera and projects horizontal fringes to make sure that the
Fig. 3. The system setup.
25
optical axis of the projector and that of the camera are approximately in parallel. An IDS uEye camera is used, its resolution is 1280 1024 pixels. The pixel size is 5 5 μm. The focal length is 16 mm. A LCD projector is used to cast the binary fringes with the period 36 pixels. Its resolution is 1024 768 pixels. The camera and projected image are in focus at a plane, which is set as z ¼0. Then we move the plane towards the system with an increment of Δz ¼5 mm ( The bigger increment can reduce the amount of the required images, but it will also reduce the mapping coefficient accuracy.). In this research, we used the PI M-406.62S Metric long travel linear translation stage. This stage is 150 mm long with a traveling accuracy of 0.1 μm. The calibrated depth z is 100 mm. For each plane, we recorded 36 exposures with the multi-step phaseshift technique. Then there will be one high accurate phase values by using all exposures and nine low accurate ones by selecting a set of four exposures for every pixel. Fig. 4a shows one of the binary fringe images. The high accurate phases and one of the low accurate phases are shown in Fig. 4b and c. Fig. 4d shows the whole 600th column of the first four-step phase errors. There are nine phase errors for one pixel. Fig. 4e
Fig. 4. Phase error determination, (a) one frame binary fringe, (b)wrapped phase from all exposures, (c) wrapped phase from four-step phase-shift technique, (d) phase errors at the 600th row, (e) nine phase errors of every pixel (only 33 pixels demonstrated), and (f) phase errors (the first four-step result) vs. wrapped phase with different depth z.
Fig. 5. (a) Phase error vs. depth z of the 600th row, (b) phase errors vs. depth z of the 600th column, and (c) phase errors vs. depth z of one pixel.
26
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
shows the nine phase errors of the 600th column with a period of the captured binary fringe (about 33 pixels). The phase errors of the first four-step results versus the high accurate phases at different depths z are shown in Fig. 4f. Here the wrapped phases are resampled in [ π, π] with a 2π/1024 rad phase interval. Within each interval, the phase error is determined by averaging all points fall within the interval. Obviously there are four waves. By using Eq. (8), we can get the calculated phase error for every pixel. Fig. 5a and b shows the phase errors versus depth z of the 600th column and 600th row separately. From Fig. 5a we can find out that the amount of phase errors of every pixel at one row are almost same. It means that the defocusing degrees are same at a single row. On the other side, the defocusing degrees are not same at a single column as shown in Fig. 5b. Because the optical axis of the projector's lens is tilted upwards, then the pixels of one column of the LCD will get different defocusing degrees. The larger defocusing degrees will bring the smaller phase errors. The relationship (only one pixel) between phase errors and depth
z is shown in Fig. 5c. To reduce the influence of the noise, a 7 7 Gaussian filter is used to smooth the phase errors of every plane. Finally the 4th-order polynomials are used to fit the functions, and we will get the fit functions for every pixel. We measured a plane in the measuring volume to evaluate the error. Fig. 6 shows a profile of the plane, whose position is at 73 mm. The calculated mean is 73.581 mm and the STD is 0.960 mm. A double-slope object is measured to verify the proposed method as shown in Fig. 7a. One binary fringe image is shown in Fig. 7b. Fig. 7c shows the averaging phase errors for all pixels. Obviously, the phase errors are smaller in the upper part of the object, where the defocusing degree becomes larger due to the tilted axis of the projector. The reconstructed 3D shape is shown in Fig. 7d. Fig. 7e and f shows two cross-sections at the 300th and 750th lines separately (in red in Fig. 7a). We can conclude the height-phase-error mapping can well construct the shape. There are gaps between the object and the reference plane. The
Fig. 6. Profile of a plane with position of 73 mm.
Fig. 8. Two profiles of the two measurements with a shift of 20 mm.
Fig. 7. (a) The tested object, (b) one binary fringe image, (c) phase error distribution, (d) three-dimensional shape, (e) a profile of the 300th row, and (f) a profile of the 750th row (For interpretation of the references to color in this figure, the reader is referred to the web version of this article.).
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
magnitude of the discontinuities is 7 mm. Therefore the small jumps at the feet of the slopes are demonstrated. To evaluate the system, we shift the tested object backwards by 20 mm as shown in Fig. 8. It is clear that the two cross-sections of the two measurements are coincided.
27
information and still keeps the advantage of the high-speed projection ability of DLP projector by using the binary patterns. Therefore, it will be promising for industrial applications.
Acknowledgments 4. Summaries As a uniaxial 3D measurement method, there is no unwrapping process. It can measure the deep holes and is less influenced by surface reflectivity. Compared with the current technique, our method has the following advantages: ● The larger phase errors can be produced by the four-step phase-shift technique, that will produce the larger Signals. ● For each pixel, there will be a set of phase errors, which are all calculated by four-step phase-shift algorithm from a sequence of exposures. And a weighted phase error will be obtained for each pixel. That makes it possible to implement a full-field measurement. ● The sequence of exposures will also be used to produce the perfect phase. Noted that all projected fringes are binary structured patterns, which are not suffered from the nonlinearity and can be easily projected. By using a uniaxial setup, the accuracy in the proposed method is lower than the triangulation-based phase measuring methods. But our method has a better performance compared with the latter while measuring the steep objects such as deep holes. Moreover this new technique can give us the full-field depth
The authors acknowledge the support by National Key Scientific Apparatus Development project (2013YQ49087901) and the National Natural Science Foundation of China (NSFC) (61177010). References [1] Chen F, Brown GM, Song M. Overview of three-dimensional shape measurement using optical methods. Opt Eng 2000;39(1):10–22. [2] Dou YF, Su XY, et al. A flexible fast 3D profilometry based on modulation measurement. Opt Lasers Eng 2011;49(3):376–83. [3] Häusler G, Vogel M, Yang Z, Kessel A Faber C. SIM and deflectometry: new tools to acquire beautiful, SEM-like 3D images. In: Proceedings of the optical society of America meeting on imaging systems. Toronto; 2011. [4] Otani, Y, Kobayashi, F, Mizutani, Y, Watanabe, S, Harada, M, Yoshizawa, T. Uniaxial measurement of three-dimensional surface profile by liquid crystal digital shifter. In: Proceedings of the SPIE photonics and photonics 7790. San Diego, CA; 2010 p. 77900A. [5] Xu Y, Zhang S. Uniaxial three-dimensional shape measurement with projector defocusing. Op Eng 2012;51(2):023604-1–6. [6] Su XY, Zhou WS, et al. Automated phase-measuring profilometry using defocused projection of a Ronchi grating. Opt Commun 1992;94(6):561–73. [7] Pan B, Kemao Q, et al. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Opt Lett 2009;34(4):416–8. [8] Xu Y, Ekstrand L, Dai J, Zhang S. Phase error compensation for 3-d shape measurement with projector defocusing. Appl. Opt. 2011;50(17):2572–81. [9] Li Z, Li Y. Gamma-distorted fringe image modeling and accurate gamma correction for fast phase measuring profilometry. Opt Lett 2011;36(2):154–6.
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
3D shape from phase errors by using binary fringe with multi-step phase-shift technique Yuankun Liu n, Qican Zhang, Xianyu Su Opto-Electronic Department, Sichuan University, Chengdu 610064, China
art ic l e i nf o
a b s t r a c t
Article history: Received 1 December 2014 Received in revised form 1 April 2015 Accepted 23 April 2015 Available online 27 May 2015
A three-dimensional shape measurement method is presented, which is a uniaxial measurement by measuring phase errors instead of the well-known phase, modulation or contrast. A sequence of exposures are captured by using a multi-step phase-shift technique with the binary fringes. Then the high-accuracy phases can be obtained by using all the exposures, meanwhile, a set of low-accuracy phases can be calculated by dividing those exposures into a set of four-step phase-shift measurements. For each pixel there will be a set of phase errors by subtracting low-accuracy phases from the highaccuracy ones. And the weighted phase error of every pixel can be calculated. Meanwhile the phase error caused by the improperly defocused binary fringes has a unique relationship with the depth z. Therefore, the 3D information of every pixel can be obtained by analyzing the phase errors. It will be promising for a uniaxial measurement, such as deep holes. & 2015 Elsevier Ltd. All rights reserved.
Keywords: Fringe analysis Uniaxial Binary defocusing Three-dimensional sensing Phase error
1. Introduction Three-dimensional shape measurement based on sinusoidal fringe projection techniques has been widely used. Principally there are two categories of 3D shape measurement methods, one is to form a triangle for depth recovery [1], therefore the depth accuracy is based on the angle between the projection line and the camera imaging line as well as the phase accuracy, and it is not easy to measure the holes with the influence of occlusions. The other is to measure the steep object by setting the angle to be zero, which is called the uniaxial measurement by measuring modulation or contrast [2–4]. Because the modulation or contrast has a unique relationship with the depth z. Therefore by z-direction scanning, the modulation or contrast curve will give the depth. Recently Xu and Zhang [5] introduced a novel method, which is to use the phase errors as the signals to retrieve the 3D information. Because the phase error caused by the improperly defocused binary fringes has a unique relationship with the depth z. As a matter of fact the phase errors come from the high-order harmonics, whose magnitudes decrease with the increase of the defocused degree. In Ying's method, they used the three-step phaseshift technique to obtain the low accuracy phases from the binary
n
Corresponding author. E-mail address: [email protected] (Y. Liu).
http://dx.doi.org/10.1016/j.optlaseng.2015.04.014 0143-8166/& 2015 Elsevier Ltd. All rights reserved.
fringes and the ideal phases from the perfect sinusoidal fringes. Then the phase error can be calculated by the two measurements. There are six waves in the phase error map. They found the locations of the peaks and valleys of the six waves, then the 3D information of those peaks and valleys can be restored from the relationship between the phase errors and depth z. However, by using the three-step phase-shift technique, the highest magnitude high-order harmonic, i.e. the third-order harmonic was eliminated. Only those certain pixels, whose ideal phases are around 7 (2nk þ1)π/12, k¼1,...,5, can get the 3D information. The perfect sinusoidal fringes need the projector to cast the gray level patterns, which may reduce the ability of high speed and need to calibrate the systematic gamma. In this paper, we employ the multi-step phase-shift technique with projecting binary fringes. The perfect phases can be obtained by all exposures with the multi-step phase-shift technique. In order to use the third-order harmonic, we choose four-step phaseshift technique to calculate the low accuracy phases. The multistep phase-shift exposures will form a set of four-step phase-shift measurements, then a set of low accuracy phases will be obtained, i.e., for each pixel, there will be a set of phase errors, therefore we can calculate a phase error for every pixel, finally, the 3D information can be restored pixel by pixel with the known relationship between phase errors and depth z. The paper is organized as follows. In Section 2, the principle of the proposed technique is introduced. The experimental results
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
23
2. Principle
For the binary structured pattern, in the Fourier cosine series, the amounts of the third and fifth harmonics are 1/3 and 1/5 of the fundamental component. Therefore, the phase errors will become larger by using the four-step phase-shift technique. From Eq. 3 by using four-step technique we get
2.1. Multi-step phase-shift technique with binary fringes
ϕ0 ðx; yÞ ¼ arctan P3n ¼ 0
of the setup are presented in Section 3. Summary is shown in Section 4.
"P
Iðx; yÞ ¼ Aðx; yÞ þ Bðx; yÞ
C j cos ðk2π f 0 x þ ϕk0 Þ
ð1Þ
k ¼ 2m 1
where A(x,y) is the background intensity, B(x,y) is the fringe modulation, Cj is the magnitude of every component. Noted that there are only odd-order harmonics. Let's assume ϕ ¼ 2π f 0 x þ ϕ0 , then the each particular measurement intensity In can be rewritten by [7] I n ðx; yÞ ¼ a0 þ
1 X
ak cos ½kðϕðx; yÞ þ δn Þ
fa0 þ
n ¼ 0 fa0 þ
In phase-shifting technique, when projecting the binary fringes, the solution to achieve high accuracy phases is to increase the amount of exposures, which could not only reduce the random noise but also reduce the systematic errors including the highorder harmonics. In extreme case, one-pixel phase-shift can be used. The deformed binary fringes can be expressed as a Fourier cosine series by [6] 1 X
3
ð2Þ
k ¼ 2m 1
P1
k ¼ 2m 1 ak P1 k ¼ 2m 1 ak
cos ½kðϕðx; yÞ þ δn Þg sin ð2π n=4Þ
#
cos ½kðϕðx; yÞ þ δn Þg cos ð2π n=4Þ
ð4Þ Then the phase error will be
Δϕ ¼ ϕ0 ðx; yÞ ϕðx; yÞ
¼ arctan
ða3 a5 Þ sin ½4ϕðx; yÞ þ ða7 a9 Þ sin ½8ϕðx; yÞ þ ⋯ a1 þ ða3 þ a5 Þ cos ½4ϕðx; yÞ þða7 þ a9 Þ cos ½8ϕðx; yÞ þ ⋯
ð5Þ Because the amounts of the third and fifth harmonics are much larger than the rest harmonics, therefore four waves are expected within the phase errors. The phase errors with the three-step technique will demonstrate six waves in the phase errors, because the lowest harmonic is the fifth one. The results are shown in Fig. 1 by using the three-step and four-step phase-shift techniques with the ideal binary structured patterns. The curve is the phase errors versus the wrapped ideal phases. Obviously the four-step phaseshift technique contains more phase errors. 2.3. Phase error determination pixel by pixel
It is known that a multi-step phase-shift technique will reduce almost all high-order harmonics, e.g. the period of the grating is 36 pixels, then a 36-step phase-shift technique will be used to decipher the perfect phase. With N-phase-shifts, the algorithm for the perfect phase ϕðx; yÞ deciphering has a form of "P # N 1 I sin ð2π n=NÞ ϕðx; yÞ ¼ arctan PNn ¼ 10 n ð3Þ n ¼ 0 I n cos ð2π n=NÞ 2.2. Four-step phase-shift technique to decipher the phase errors
In Fig. 1, it shows that there are some pixels in the phase map, whose phase errors are at peaks or valleys. In Ying's work they used these pixels to restore the depth z by using the relationships between phase errors, ideal phases and depth z. Obviously there are always some pixels, whose phase errors are always small or even zero. Then it is difficult to get the 3D information of these pixels. As a matter of fact, with the multi-step exposures, a set of Table 1 Sensitivity of different phase-shifting algorithms to harmonics. Number of steps
A defocusing optical system can act as a low-pass filter system, which can dump the higher harmonic components. But if the projector is not properly defocused, there are still binary structures on the resultant fringe patterns, and the phase error is significant. It is known that there are only the 3n2m 71th order harmonics in three-step phase-shift technique and only the 5n2 m 7 1th order harmonics in five-step method, but all harmonics remain by using the four-step method, as shown in Table 1.
Harmonics 2
3 4 5 6 7 8
3
4
5
6
7
8
Fig. 1. The phase errors versus wrapped phase, (a) three-step phase-shift, and (b) four-step phase-shift.
9
10
11
12
13
14
15
24
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
four step phase-shift measurements can be formed, the process is shown in Table 2. And then we can calculate one phase error for each four-step phase-shift measurement. Therefore there will be a set of phase errors. The phase values for each four-step phase-shift measurement can be expressed by "P # 3 I sin ½2π ðnN s þ m 1Þ=N ϕm ðx; yÞ ¼ arctan P3n ¼ 0 nNs þ m n ¼ 0 I nN s þ m cos ½2π ðnN s þm 1Þ=NÞ m ¼ 1; :::; N s Ns ¼ INTðN=4Þ
ð6Þ
From Eq. (5) and Eq. (6), we can get Ns phase errors for each pixel, which can be expressed by
Δϕm ðx; yÞ ¼ ϕðx; yÞ ϕm ðx; yÞ " ¼ arctan
# ða3 a5 Þ sin ½4ϕðx; yÞ þ 42πNm þ ða7 a9 Þ sin ½8ϕðx; yÞ þ 82πNm þ ⋯ a1 þ ða3 þ a5 Þ cos ½4ϕðx; yÞ þ 42πNm þ ða7 þ a9 Þ cos ½8ϕðx; yÞ þ 82πNm þ ⋯
ð7Þ For the 36 phase-shift application, all 36 exposures can form nine four-step phase-shift measurements, and the phase interval between two neighboring measurements is 2π/9. Then we can get nine phase errors for every pixel. Fig. 2a shows that there are nine curves totally, and each of them is shifted by a certain phase 2π/9. Fig. 2b shows the nine phase errors of one pixel. The nine phase errors can form a wave for a single pixel as shown in Fig. 2b. For a real projecting and imaging system, the high-order harmonics will be suppressed more or less. Let's assume that the wave is a sinusoidal function, then the modulation calculation of the well-known N-shift technique can be used to find the weighted value of the phase errors. Then the formula is
described as vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi #2 " #2 u" NX u NX s 1 s 1 2 t ~ Δϕ ¼ Δϕ sin ð2π k=N s Þ þ Δϕ cos ð2π k=N s Þ Ns k¼0 k¼0 ð8Þ Noted that with respect to the sampling theory, at least three phase errors for each pixel are needed to calculate the final phase error by Eq. (8). Then the minimum amount of the projected patterns is 3 4 ¼12, no matter what the period of the binary pattern is. But the phase-shift is pixel based, therefore the valid amount of the projected patterns is 12, 18 or 36 when the period of the binary pattern is 36. And more patterns will produce the more accurate phase value and more phase-errors, then 36 patterns are used in this paper.
2.4. Phase error and depth z mapping It is clear that the phase errors will decrease with the increase of the defocus degree [8,9], and the degree of defocusing has a unique relationship with the depth distance (z) [5]. Therefore the depth information can then be retrieved from the degree of defocusing. In order words, the depth (z) of the object points can be written as functions of the phase error, ~ ðx; yÞÞ z ¼ f ð Δϕ
ð9Þ
This error function can be determined by calibration. Once this function is known, the depth information of all pixels can be measured.
Table 2 A set of four-step phase-shift measurements by pixel–pixel phase-shifting with a period of grating 36 pixels. Four-step phase-shift
Exposures 1
1 2 3 4 ... 9
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
...
32
33
34
35
36
... ... ... ... ...
Fig. 2. Phase errors for pixels, (a) nine phase-error curves, and (b) nine phase errors for one single pixel.
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
3. Experimental results The setup is shown in Fig. 3, where the projector sits below the camera and projects horizontal fringes to make sure that the
Fig. 3. The system setup.
25
optical axis of the projector and that of the camera are approximately in parallel. An IDS uEye camera is used, its resolution is 1280 1024 pixels. The pixel size is 5 5 μm. The focal length is 16 mm. A LCD projector is used to cast the binary fringes with the period 36 pixels. Its resolution is 1024 768 pixels. The camera and projected image are in focus at a plane, which is set as z ¼0. Then we move the plane towards the system with an increment of Δz ¼5 mm ( The bigger increment can reduce the amount of the required images, but it will also reduce the mapping coefficient accuracy.). In this research, we used the PI M-406.62S Metric long travel linear translation stage. This stage is 150 mm long with a traveling accuracy of 0.1 μm. The calibrated depth z is 100 mm. For each plane, we recorded 36 exposures with the multi-step phaseshift technique. Then there will be one high accurate phase values by using all exposures and nine low accurate ones by selecting a set of four exposures for every pixel. Fig. 4a shows one of the binary fringe images. The high accurate phases and one of the low accurate phases are shown in Fig. 4b and c. Fig. 4d shows the whole 600th column of the first four-step phase errors. There are nine phase errors for one pixel. Fig. 4e
Fig. 4. Phase error determination, (a) one frame binary fringe, (b)wrapped phase from all exposures, (c) wrapped phase from four-step phase-shift technique, (d) phase errors at the 600th row, (e) nine phase errors of every pixel (only 33 pixels demonstrated), and (f) phase errors (the first four-step result) vs. wrapped phase with different depth z.
Fig. 5. (a) Phase error vs. depth z of the 600th row, (b) phase errors vs. depth z of the 600th column, and (c) phase errors vs. depth z of one pixel.
26
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
shows the nine phase errors of the 600th column with a period of the captured binary fringe (about 33 pixels). The phase errors of the first four-step results versus the high accurate phases at different depths z are shown in Fig. 4f. Here the wrapped phases are resampled in [ π, π] with a 2π/1024 rad phase interval. Within each interval, the phase error is determined by averaging all points fall within the interval. Obviously there are four waves. By using Eq. (8), we can get the calculated phase error for every pixel. Fig. 5a and b shows the phase errors versus depth z of the 600th column and 600th row separately. From Fig. 5a we can find out that the amount of phase errors of every pixel at one row are almost same. It means that the defocusing degrees are same at a single row. On the other side, the defocusing degrees are not same at a single column as shown in Fig. 5b. Because the optical axis of the projector's lens is tilted upwards, then the pixels of one column of the LCD will get different defocusing degrees. The larger defocusing degrees will bring the smaller phase errors. The relationship (only one pixel) between phase errors and depth
z is shown in Fig. 5c. To reduce the influence of the noise, a 7 7 Gaussian filter is used to smooth the phase errors of every plane. Finally the 4th-order polynomials are used to fit the functions, and we will get the fit functions for every pixel. We measured a plane in the measuring volume to evaluate the error. Fig. 6 shows a profile of the plane, whose position is at 73 mm. The calculated mean is 73.581 mm and the STD is 0.960 mm. A double-slope object is measured to verify the proposed method as shown in Fig. 7a. One binary fringe image is shown in Fig. 7b. Fig. 7c shows the averaging phase errors for all pixels. Obviously, the phase errors are smaller in the upper part of the object, where the defocusing degree becomes larger due to the tilted axis of the projector. The reconstructed 3D shape is shown in Fig. 7d. Fig. 7e and f shows two cross-sections at the 300th and 750th lines separately (in red in Fig. 7a). We can conclude the height-phase-error mapping can well construct the shape. There are gaps between the object and the reference plane. The
Fig. 6. Profile of a plane with position of 73 mm.
Fig. 8. Two profiles of the two measurements with a shift of 20 mm.
Fig. 7. (a) The tested object, (b) one binary fringe image, (c) phase error distribution, (d) three-dimensional shape, (e) a profile of the 300th row, and (f) a profile of the 750th row (For interpretation of the references to color in this figure, the reader is referred to the web version of this article.).
Y. Liu et al. / Optics and Lasers in Engineering 74 (2015) 22–27
magnitude of the discontinuities is 7 mm. Therefore the small jumps at the feet of the slopes are demonstrated. To evaluate the system, we shift the tested object backwards by 20 mm as shown in Fig. 8. It is clear that the two cross-sections of the two measurements are coincided.
27
information and still keeps the advantage of the high-speed projection ability of DLP projector by using the binary patterns. Therefore, it will be promising for industrial applications.
Acknowledgments 4. Summaries As a uniaxial 3D measurement method, there is no unwrapping process. It can measure the deep holes and is less influenced by surface reflectivity. Compared with the current technique, our method has the following advantages: ● The larger phase errors can be produced by the four-step phase-shift technique, that will produce the larger Signals. ● For each pixel, there will be a set of phase errors, which are all calculated by four-step phase-shift algorithm from a sequence of exposures. And a weighted phase error will be obtained for each pixel. That makes it possible to implement a full-field measurement. ● The sequence of exposures will also be used to produce the perfect phase. Noted that all projected fringes are binary structured patterns, which are not suffered from the nonlinearity and can be easily projected. By using a uniaxial setup, the accuracy in the proposed method is lower than the triangulation-based phase measuring methods. But our method has a better performance compared with the latter while measuring the steep objects such as deep holes. Moreover this new technique can give us the full-field depth
The authors acknowledge the support by National Key Scientific Apparatus Development project (2013YQ49087901) and the National Natural Science Foundation of China (NSFC) (61177010). References [1] Chen F, Brown GM, Song M. Overview of three-dimensional shape measurement using optical methods. Opt Eng 2000;39(1):10–22. [2] Dou YF, Su XY, et al. A flexible fast 3D profilometry based on modulation measurement. Opt Lasers Eng 2011;49(3):376–83. [3] Häusler G, Vogel M, Yang Z, Kessel A Faber C. SIM and deflectometry: new tools to acquire beautiful, SEM-like 3D images. In: Proceedings of the optical society of America meeting on imaging systems. Toronto; 2011. [4] Otani, Y, Kobayashi, F, Mizutani, Y, Watanabe, S, Harada, M, Yoshizawa, T. Uniaxial measurement of three-dimensional surface profile by liquid crystal digital shifter. In: Proceedings of the SPIE photonics and photonics 7790. San Diego, CA; 2010 p. 77900A. [5] Xu Y, Zhang S. Uniaxial three-dimensional shape measurement with projector defocusing. Op Eng 2012;51(2):023604-1–6. [6] Su XY, Zhou WS, et al. Automated phase-measuring profilometry using defocused projection of a Ronchi grating. Opt Commun 1992;94(6):561–73. [7] Pan B, Kemao Q, et al. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Opt Lett 2009;34(4):416–8. [8] Xu Y, Ekstrand L, Dai J, Zhang S. Phase error compensation for 3-d shape measurement with projector defocusing. Appl. Opt. 2011;50(17):2572–81. [9] Li Z, Li Y. Gamma-distorted fringe image modeling and accurate gamma correction for fast phase measuring profilometry. Opt Lett 2011;36(2):154–6.