- Email: [email protected]
A multi-frequency inverse-phase error compensation method for projector nonlinear in 3D shape measurement
Optics Communications 419 (2018) 75–82
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
A multi-frequency inverse-phase error compensation method for projector nonlinear in 3D shape measurement Cuili Mao a,b , Rongsheng Lu a, *, Zhijian Liu a a b
School of Instrument Science and Opto-Electronics Engineering, Hefei University of Technology, Hefei 230009, China School of Mechanical & Automotive Engineering, Nanyang Institute of Technology, Nanyang 473004, China
ARTICLE
INFO
Keywords: Phase shifting profilometry Non-linear phase error Inverse-error compensation Multi-frequency method
ABSTRACT In fringe projection profilometry, the phase errors caused by the nonlinear intensity response of digital projectors needs to be correctly compensated. In this paper, a multi-frequency inverse-phase method is proposed. The theoretical model of periodical phase errors is analyzed. The periodical phase errors can be adaptively compensated in the wrapped maps by using a set of fringe patterns. The compensated phase is then unwrapped with multi-frequency method. Compared with conventional methods, the proposed method can greatly reduce the periodical phase error without calibrating measurement system. Some simulation and experimental results are presented to demonstrate the validity of the proposed approach.
1. Introduction Fringe projection profilometry (FPP) is one of the most widely used techniques for full-field three-dimensional (3D) shape measurement due to its qualities of non-contact, low-cost, easy availability, high resolution, etc. [1,2]. The phase-shifting fringe (PSF) is a popular branch of the FPP. It acquires 3D information from two-dimensional images by analyzing several deformed patterns projected by a projector and captured by CCD cameras [3–5]. The typical projection patterns include sinusoidal and binary fringe patterns, such as used in Fourier transform profilometry (FTP) [6], wavelet transform profilometry (WTP). Sinusoidal fringe patterns have been widely used in PSP for they have the ability to accomplish high accurate 3D shape measurement. However, The projector’s nonlinear response (i.e. gamma distortion) produces high-order harmonics, which deviates from its ideal sinusoidal distribution. Then phase errors inevitably occur in FPP [6]. To achieve accurate 3D shape measurement result, the phase error must be reduced. In recent years, a lot of theories and methods of alleviating the phase errors have been proposed with the configuration of digital video projectors (DVP) and CCD cameras. They can be mainly summarized into three categories: passively compensating phase error after the fringe patterns are captured [7–11], actively modifying the fringe patterns before their projection [12–17] and inverse-phase error compensation method [18–20]. For passively phase error compensation, one or more gamma coefficients should be firstly estimated from the captured fringe patterns using different algorithms. The distribution of phase error, *
which may be formed a look-up-table (LUT), is obtained by theoretical analysis and experiments. Then the phase error compensation can be applied to the deformed fringe patterns captured by the cameras with the calibration data. Zhang [8] proposed a full-field phase error detection and compensation method with the aid of the phase error distribution table. A mathematical model was developed for predicting the effects of non-unitary gamma [10]. The experimental results using simulated and real data have confirmed that the passive method of phase error compensation can achieve good effects under the conditions of constant measurement environment (i.e., constant ambient light illumination and surface reflection of the measured object, etc.) and stable system parameters (i.e., the parameters of the projector and camera). But the effectiveness of the passive method is diminished if the measurement conditions deviates from the calibration ones. Otherwise, the re-calibration should be carried out. At the same time the measurement precision and speed are closely related to the complexity of the error compensation algorithm. For achieving high precise sinusoidal fringe from projector, the actively phase error correction methods reencode the projector’s input fringe patterns according to the gamma value or the error distribution having been calibrated. The effectiveness of the active method is closely related to the calibration accuracy of the phase error and the accuracy of the re-coding fringe. Once the accurate coding stripes are produced, the measurement speed is relatively fast and no more phase error correction is required. But the re-calibrating process has to be done as the conditions, such as the system parameters
Corresponding author. E-mail address: [email protected] (R. Lu).
https://doi.org/10.1016/j.optcom.2018.03.006 Received 8 November 2017; Received in revised form 26 January 2018; Accepted 4 March 2018 0030-4018/© 2018 Elsevier B.V. All rights reserved.
C. Mao et al.
Optics Communications 419 (2018) 75–82
and/or the measurement environment, are changed. Inverse-phase error compensation methods are accomplished by projecting another set of fringes that have the opposite phase error with the original fringes. Phase error can be automatically eliminated by averaging the two sets of projection fringes. Huang [18] first proposed the embryonic form of this method. The multi-frequency inverse-phase error method was proposed by Lei [19] and has good effective results. It needs to unwrap the two sets of wrapped phases, which is time-consuming. In addition, the phase unwrapping should be firstly processed in spatial space, i.e., in the maps of the unit-frequency and the initial phase of pi/3. The process may cause the unwrapping phase error before the following phase compensation. For getting high measurement speed and accuracy, Cai [20] proposed an efficient method of phase error compensation by using Hilbert Transform (HT). Nevertheless, the measured object should not show discontinuities. For improving the measurement efficiency and reducing phase error, Su [21] proposed a binary fringe defocus method. But a low SNR and small measurement range is followed [22]. In order to compensate the phase error of the binary fringe patterns with projector defocusing, D. Zheng [23] compared the effect of double three-step, Hilbert three-step, five-step with theoretical analysis and experiments, and the conclusion was that the double three-step has the better overall effect. S. Zhang [24] compared the active and passive projector nonlinear gamma compensation methods, and finds that the active method tends to provide relatively more consistent high-quality result. He further found that the nonlinear phase error does not precisely follow a simple model with single gamma. Both active and passive phase error compensation methods need calibrate the distribution of phase error, then the error correction can be carried out, and the accuracy phase can be obtained under a certain condition. The calibration process must be implemented again with the change of the measurement conditions. Furthermore, the passive phase error compensation is applied to every pixel. It is obvious that the measurement speed is slow. However, no calibration is needed for inverse-phase error method, and the error compensation action is not necessary to apply to every pixel. In this paper, a new multi-frequency inverse-phase method with three-step phase shifting was proposed to compensate for nonlinear phase errors for higher accurate unwrapped phase map. The characteristics of the nonlinear phase error were analyzed. Then related experiments were performed to make the more comprehensive performance comparison with the commonly used active gamma compensation methods, the recently proposed HT method with 3-step phase shifting algorithm against the multi-frequency inverse-phase error compensation method proposed in this paper. The investigations show that our proposed method tends to provide more consistent high-quality unwrapping phase map. The rest of this paper is organized as follows. Section 2 gives the theoretical analysis of the FPP using sinusoidal patterns and the phase error model is showed, then the new phase error correction method is proposed. Section 3 gives the simulations of this method. Section 4 illustrates the experiment and the results. Section 5 summaries this paper.
be concise, coordinates (𝑥, 𝑦) are omitted hereafter, and 𝜑(𝑥, 𝑦) is the ideal phase. The gamma-distorted fringe patterns projected by a projector and captured by a camera, according to the power-law response, can be described by [20] 𝐼𝑛𝐶 = 𝑎(𝐼𝑖 )𝛾 + 𝑏 = 𝐵0 +
∞ ∑
(2)
[𝐵𝑘 cos(𝑘𝜑𝑛 )]
𝑘=1
where 𝐼𝑛𝐶 is the output grayscale value for a given input intensity value 𝐼𝑖 , 𝑎 and 𝑏 are constants, 𝛾 is the gamma factor of the used projector which is normally greater than 1, 𝐵0 is the average intensity (direct component), 𝐵𝑘 is the magnitude of the 𝑘th harmonic, 𝜑𝑛 = 𝜑 + 𝛿𝑛 represents the modulated phase coupled with the phase shift. Based on the least square algorithms (LSA) and the power-law response, the phase information can be retrieved from the captured images [5,7–9]. ∑𝑁−1 𝐶 𝐼 sin(𝛿𝑛 ) 𝜑𝐶 = − arctan ∑ 𝑛=0 𝑛 𝑁−1 𝐶 𝐼 𝑛=0 𝑛 cos(𝛿𝑛 ) (3) ] ∑𝑁−1 ∑∞ [ 𝑛=0 𝑘=1 𝐵𝑘 cos(𝑘𝜑𝑛 ) sin 𝛿𝑛 = − arctan ∑ ] 𝑁−1 ∑∞ [ 𝑛=0 𝑘=1 𝐵𝑘 cos(𝑘𝜑𝑛 ) cos 𝛿𝑛 where 𝜑𝐶 is the distorted phase due to the power-law response. Ideally, if 𝛾 is equals 1, 𝑘 equals 1. According to the trigonometric formula tan(𝛼 − 𝛽) =
(tan 𝛼 − tan 𝛽) (1 − tan 𝛼 tan 𝛽)
(4)
the phase error of N -step can be obtained [5] 𝛥𝜑𝑁 = 𝜑𝐶 − 𝜑 { = arctan
∑𝑁=1 ∑∞ [
]
𝑘=2 𝐵𝑘+1 − 𝐵𝑘−1 sin(𝑘𝜑𝑛 ) ] ∑𝑁=1 ∑∞ [ 𝑁𝐵1 + 𝑛=0 𝑘=2 𝐵𝑘+1 − 𝐵𝑘−1 cos(𝑘𝜑𝑛 ) 𝑛=0
}
(5)
Considering that 𝜑𝑛 = 𝜑 + 𝛿𝑛 = 𝜑 + 2𝜋𝑛∕𝑁, it can be proved easily { 0, 𝑘 ≠ 𝑚𝑁 𝑚 ∈ 𝑍+ 𝑁 sin(𝑚𝑁𝜑), 𝑘 = 𝑚𝑁, 𝑛=0 { 𝑁−1 ∑[ ] 0, 𝑘 ≠ 𝑚𝑁 cos(𝑘𝜑𝑛 ) = 𝑚 ∈ 𝑍+ 𝑁 cos(𝑚𝑁𝜑), 𝑘 = 𝑚𝑁, 𝑛=0 𝑁−1 ∑
[ ] sin(𝑘𝜑𝑛 ) =
(6)
So Eq. (5) can be simplified as } { ∑∞ 𝑚=0 (G𝑚𝑁+1 − G𝑚𝑁−1 ) sin(𝑚𝑁𝜑𝑛 ) (7) 𝛥𝜑𝑁 = arctan ∑ 1+ ∞ 𝑚=0 (G𝑚𝑁+1 − G𝑚𝑁−1 ) cos(𝑚𝑁𝜑𝑛 ) ] ∏ [ where 𝐺𝑆 = 𝐵𝑆 ∕𝐵1 = 𝑆𝑖=2 (𝛾 − 𝑖 + 1)∕(𝛾 + 𝑖) . As |(𝛾 − 𝑖 + 1)∕(𝛾 + 𝑖)| < 1 and is obviously decreases with the increase of harmonic order s, 𝐺𝑆 is far less than 1, so Eq. (6) can be further simplified as {∞ } ∑[ ] 𝛥𝜑𝑁 ≈ arctan (G𝑚𝑁+1 − G𝑚𝑁−1 ) sin(𝑚𝑁𝜑𝑛 ) 𝑚=0 (8) ∞ ∑[ ] ≅ 𝑐𝑚 sin(𝑚𝑁𝜑) 𝑚=1
where 𝑐𝑚 is constant. The harmonic coefficient of sin(𝑚𝑁𝜑) will drastically decrease as 𝑚increases, so we usually only focus on the low-order harmonic terms, then phase error is followed with different step of phase shifting.
2. Mathematical model of phase error 2.1. Phase analysis In FPP, a series of standard sinusoidal fringe patterns generated with computer, can be described by 𝐼𝑛𝑃 (𝑥, 𝑦) = 𝐴𝑃 (𝑥, 𝑦) + 𝐵 𝑃 (𝑥, 𝑦) cos[𝜑(𝑥, 𝑦) + 𝛿𝑛 ]
𝛥𝜑3 ≅ 𝑐1 sin(3𝜑) + 𝑐2 sin(6𝜑) ≈ 𝑐1 sin(3𝜑) 𝛥𝜑4 ≅ 𝑐1 sin(4𝜑) + 𝑐2 sin(8𝜑) ≈ 𝑐1 sin(4𝜑) 𝛥𝜑5 ≅ 𝑐1 sin(5𝜑) + 𝑐2 sin(10𝜑) ≈ 𝑐1 sin(5𝜑)
(1)
(9)
where 𝑐1 , 𝑐2 , 𝑐3 are constants and may have different value in Eq. (9). The phase error can be approximated as a periodic sinusoidal function and its period is related to the phase-shifting steps and the pitch of the fringe pattern. That is the reason why is called periodic system error. According to the phase error formulas, the periodic system error can
where 𝑛 = 0, 1, … , 𝑁 − 1 and 𝑁 denotes the number of phase steps in the phase shifting algorithm, 𝐼𝑛𝑃 is the intensity of the projected patterns, 𝐴𝑃 and 𝐵 𝑃 are the background intensity and intensity modulation amplitude respectively, 𝛿𝑛 = 2𝜋𝑛∕𝑁 is the phase-shifting amount. To 76
C. Mao et al.
Optics Communications 419 (2018) 75–82
reconstruct the 3D surface shape, the process of phase unwrapping must be carried out in order to remove the discontinuities from their principal values for obtaining the true continuous phase map. The basic unwrapping phase algorithm is as follows 𝛷(𝑥, 𝑦) = 𝜑(𝑥, 𝑦) + 2𝑘(𝑥, 𝑦)𝜋
(13)
where 𝜑(𝑥, 𝑦) is the wrapped phase obtained from Eq. (3), 𝛷(𝑥, 𝑦) is the unwrapped phase obtained from Eq. (12), and 𝑘(𝑥, 𝑦) is the integer representing fringe order. The key to a phase unwrapping algorithm is to obtain 𝑘(𝑥, 𝑦) accurately for each pixel, and the accuracy of phase unwrapping depends on the precision of 𝜑(𝑥, 𝑦). In order to correctly measure a discontinuity object with holes, steps and gaps, etc., unambiguous and exact absolute phase should be obtained. Multi-frequency phase-unwrapping algorithm has the ability to measure the 3D shape of discontinuous objects without complex phaseunwrapping operation. Three temporal phase unwrapping methods commonly used are analyzed and discussed in detail by Chao Zuo [25]. In order to improve accuracy and reliability of the unwrapped absolute phase, multi-frequency (hierarchical) temporal phase unwrapping method (MFPU) is selected in this paper according to comprehensive comparison. The coarse fringe pattern is used to get the fringe orders of fine fringe pattern, and the fine one to improve the accuracy of phase for obtaining object details. The principle of MFPU is { 𝛷ℎ = 𝜑ℎ + 2𝑘ℎ 𝜋 (14) 𝛷𝑙 = 𝜑𝑙 + 2𝑘𝑙 𝜋 where the 𝑘ℎ and 𝑘𝑙 are the respective integer fringe orders of the fine and coarse fringe patterns, 𝛷ℎ , 𝛷𝑙 , 𝜑ℎ , 𝜑𝑙 are unwrapping and wrapping phases of fine and coarse fringe patterns respectively. In two-frequency temporal phase unwrapping, the low-resolution phase distribution is retrieved by using a set of unit-frequency patterns, and thus no phase unwrapping is required for 𝜑𝑙 , that is, 𝛷ℎ = 𝜑𝑙 , so the fringe order 𝑘𝑙 = 0. Then the fringe order 𝑘ℎ for each pixel can be determined easily ] [ (𝑓ℎ ∕𝑓𝑙 )𝜑𝑙 − 𝜑ℎ (15) 𝑘ℎ = 𝑅𝑜𝑢𝑛𝑑 2𝜋
Fig. 1. Error distribution of before and after phase shifting. (a) 3-step, (4) 4-step.
be eliminated using the half cycle elimination error method, i.e., the reverse error compensation method. If two patterns are projected with half-cycle translation, their phase errors have equal absolute error values and opposite in signs. Then the periodic error can be automatically eliminated by averaging the two phases with inverse error for every pixel. For example, when 𝑁 = 3 is used, it is not difficult to find that the dominating phase error is periodic with a period of 2𝜋∕3. The another 3-step phase shifts have an initial phase offset of 𝜋∕3, i.e., half of the error period according to the analysis above. Similarly, the initial phase offset should be 𝜋∕4 and 𝜋∕5 for 𝑁 = 4 and 𝑁 = 5 respectively, then the other phase error is inferred as follows 𝛥𝜑′3 ≅ 𝑐1 sin[3(𝜑 + 𝜋∕3)] + 𝑐2 sin[6(𝜑 + 𝜋∕3)] + 𝑐3 sin[9(𝜑 + 𝜋∕3)] = 𝑐1 sin(3𝜑 + 𝜋) + 𝑐2 sin(6𝜑 + 2𝜋) + 𝑐3 sin(9𝜑 + 3𝜋) = −𝑐1 sin(3𝜑) + 𝑐2 sin(6𝜑) − 𝑐3 sin(9𝜑)
(10)
𝛥𝜑′4 ≅ 𝑐1 sin[4(𝜑 + 𝜋∕4)] = −𝑐1 sin(4𝜑)
(11)
𝛥𝜑′5
(12)
≅ 𝑐1 sin[5(𝜑 + 𝜋∕5)] = −𝑐1 sin(5𝜑)
where Round() represents a function of rounded integer of a decimal number, 𝑓𝑙 and 𝑓ℎ are the frequency of coarse and fine fringe respectively. Therefore, the accurate absolute unwrapped phase of high frequency fringe patterns can be obtained with Eq. (14). 2.3. Inverse-phase error compensation in wrapped phase (IECW) Previous inverse-phase error compensation method is done with unwrapping phase map, which should be obtained firstly. However, the mistake unwrapped phase will take place with fringe accompanied by large noise, especially when the frequency is a unit-frequency with a phase shifting. If the phase error compensation is done in the wrapped phase, this problem can be solved immediately without phase unwrapping. The wrapped phases of two sets of PSF which are with 𝜋∕3 phaseshifting are 𝜑′1 , 𝜑′2 respectively, it can be obtained according to above analysis 𝜑′1 = 𝜑 + 𝛥𝜑3
It can be concluded that the opposite phase error occurs if the initial phase offset is 𝜋∕3, 𝜋∕4, 𝜋∕5 for 𝑁 = 3, 4, 5 respectively. According to (8) and (9), it is clear that the odd harmonic error of (2m-1)N may be eliminated when phase shifting step is odd, that is 𝑁 = 3, 5, … , (2𝑘 + 1). The phase errors of step 𝑁 = 3, 4 are shown in Fig. 1, where the ideal phase is obtained with 20-step phase shifting method. The phase errors are opposite each other and the frequency ratio of phase errors is 9∕12 = 3∕4 at 600–700 pixels by looking at the axes of Fig. 1, which is consistent with the above reasoning.
𝜑′2 = 𝜑 + 𝜋∕3 − 𝛥𝜑3
(16)
The real wrapped phase can be obtained with the proposed IECW. { ′ (𝜑1 + 𝜑′2 − 𝜋∕3)∕2, 𝜑′1 < 𝜑′2 𝜑(𝑥, 𝑦) = (17) ′ ′ ′ ′ (𝜑 + 𝜑 − 𝜋∕3 + 2𝜋)∕2, 𝜑1 < 𝜑2 1
2
According to Eqs. (9)–(12), the residual error of the 3-step, 4-step and 5-step PSF can be obtained 𝛥3 = = 𝛥4 = = 𝛥5 = =
2.2. Principle of multi-frequency method The phase distribution obtained according to Eq. (2) is wrapped into the range (−𝜋, 𝜋]. Hence the result is given modulo 2𝜋 and discontinuities occur with values near to 2𝜋 in the phase distribution. To correctly 77
(𝜑′1 + 𝜑′2 − 𝜋∕3)∕2 − 𝜑 (𝛥𝜑3 + 𝛥𝜑′3 )∕2 ≈ 𝑐2 sin(6𝜑) (𝜑′1 + 𝜑′2 − 𝜋∕4)∕2 − 𝜑 (𝛥𝜑4 + 𝛥𝜑′4 )∕2 ≈ 𝑐2 sin(8𝜑) (𝜑′1 + 𝜑′2 − 𝜋∕5)∕2 − 𝜑 (𝛥𝜑4 + 𝛥𝜑′4 )∕2 ≈ 𝑐2 sin(10𝜑)
(18)
C. Mao et al.
Optics Communications 419 (2018) 75–82
Fig. 2. Combination flowchart for the IECW algorithm. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Fig. 3. The 50th line of the processing procedure (𝛾 = 2) (a) IECW and (b) MFPU.
When a limit error is set as 𝜀, if |𝛥𝜑| > 𝜀, the invalid point with large phase error can be detected straightforwardly using the proposed method. It can be seen from Eq. (15) that if the 𝜑ℎ or 𝜑𝑙 has large error, the fringe order 𝑘ℎ may be wrong and further the wrong unwrapped phase may be produced from Eq. (13). Compared with the popular modulation-based invalid point detection method, our detection technique has unique feature that we can classify valid and invalid points by phase error, which is more meaningful for objects with very high or very low reflectivity.
As we all know, the harmonic coefficient will decrease drastically with the increase of harmonic frequency, so 𝑐2 is much less than 𝑐1 , and can be ignored. 2.4. Invalid point detection According to Eq. (16), the phase error from the two sets of IECW can be achieved 𝛥𝜑 = (𝜑′1 − 𝜑′2 − 𝜋∕3)∕2
(19) 78
C. Mao et al.
Optics Communications 419 (2018) 75–82
Fig. 4. The processing of the 50th line (𝛾 = 2), noise amplitude Am=40 (a) IECW (b) MFPU without IECW (c) MFPU with IECW.
3. Simulations According to the above analysis, the phase error is decreased with the increase of phase shifting step, but the efficiency is reduced with increase of phase shifting step, so this paper uses the minimal 3-step for verifying the effectiveness of the proposed method. The projector’s gamma nonlinear effect can be simulated by an exponential model according to Eq. (2). There is no effect on fringe images when 𝛾 equals 1, here the common approximations 𝛾 = 2 is adopted. A 3-step FPP method with different frequencies is used to explain the combination flowchart for the MFPU with IECW algorithm, as is shown in Fig. 2 with two sets of 0 and 𝜋∕3 initial phase respectively. The 50th line of the processing procedure with MFPU and IECW is drawn in Fig. 3. In order to test the anti-noise ability of this method, a large amplitude noise of the 15% signal amplitude is added, and the simulation results is shown in Fig. 4. Not only the nonlinear gamma error reduce, the influence of noise has also been a certain degree of suppression as seen from Fig. 4(b) and (c), which respectively represents the unwrapping result without correction and with IECW. It can be seen from the results that the combination of IECW and MFPU can increase phase accuracy and also have the ability of detecting invalid points.
Fig. 5. Phase error distribution of the FPP with different phase error correction method. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
79
C. Mao et al.
Optics Communications 419 (2018) 75–82
Fig. 6. Unwrapped phase map (a) MFPU without IECW (b) MFPU with IECW (c) HT.
Fig. 7. Experimental results for measuring a ceramic bowl (a) photograph of the ceramic bowl (b) wrapped phase (c) phase distributions of the 800th row (d) MFPU without IECW (e) MFPU with IECW (f) HT.
4. Experiments and comparison with others
without any compensation, is three times the signal frequency (black PS fringe), and the residual phase error (indicated by blue solid line) is six times the signal frequency. The experimental results are consistent with the above analysis. The results showed in Fig. 5 and Fig. 6 made it clear that the proposed method, MFPU with IECW, has the optimum performance for nonlinear phase error compensation. The above experiments verified not only our previous theoretical analysis and simulations, but also the effectiveness of the proposed method without complicated calibrations. It is worth mentioned that the idea proposed in this work is inspired by the Lei’s paper [19]. The principle is very similar to the one proposed in the Lei’s paper. In our proposed method phase compensation is performed in the wrapped phase map, but it is done with the unwrapped phase map in the Lei’s method [19]. The main differences between them are: (1) Our method may achieve higher efficiency because the required number of captured images is less than the ones of Lei’s method. This is because that the compensation in our method may be applied only to the highest frequency, but to all the frequencies in the Lei’s paper. Here a single example will suffice. If four frequencies are used, the number differences of captured images in the two methods are listed in Table 1: Even if the Lei’s compensation is applied only to the highest frequency in the unwrapping maps, our method process is also quicker. This is because that our method requires less once than that of the Lei’s method for phase unwrapping. (2) Another advantage of our method is that, if the phase compensation is carried out in the wrapped map, such as our method does,
In this section, several classical phase error correction algorithms were compared with a flat plane. Then a ceramic bowl with a smooth surface and different steps was used to examine the effect of the proposed method. 4.1. System setup The experimental FPP setup consists of a DLP projector (LightCrafter4500, resolution:1024 × 768), a CCD camera (PointGrey FL3-U3-13E4C-C, resolution:1280 × 1024), and a computer with Intel(R) core(TM) i3-4150 CPU @3.50 GHZ and 4.00 GB RAM. The computer produces ideal sinusoidal fringes, which are projected to an object, then the camera capture the deformed fringes modulated by the surface of the object. 4.2. Comparison of algorithms Several classical phase error compensation algorithms, here HT 3-step, 3-step with MFPU and IECW, classical gamma correction algorithms and the classical 3-step without phase error correction are compared with a flat plane target in the experiments. The ideal unwrapping phase was obtained with 20-step phase shifting patterns. The unwrapping phase errors are shown in Fig. 5 and the unwrapped phase in Fig. 6(b) and Fig. 6(c) using the proposed method and HT method. It is obvious from Fig. 5 that the largest phase error (the red dashed line), 80
C. Mao et al.
Optics Communications 419 (2018) 75–82
Fig. 8. Experimental results for measuring a group of gauges (a) MFPU without IECW (b) MFPU with IECW (c) phase distributions of a row (d) phase distributions of a column. Table 1 Comparison of the two methods.
5. Conclusions
N-step
Method
Number of unwrapping
Number of images
3-step
Z.K. Lei’s The proposed
4×2=8 4
3 × 4 × 2 = 24 3 × (4 + 1) = 15
4-step
Z.K. Lei’s The proposed
4×2=8 4
4 × 4 × 2 = 32 4 × (4 + 1) = 20
In this paper, the phase error model of off-the-shelf digital video projectors is analyzed for the classical 3-, 4- and 5-step FPP. A new phase error correction method, MFPU with IECW, was proposed for projector’s nonlinear response. After presenting the comprehensive theoretical derivation and simulation of the nonlinear phase error for the proposed method with 3-step FPP, several commonly used phase error compensation methods are compared. According to the experimental results, it can be seen that the proposed method has the minimal residual phase error and can realize the 3D shape measurement with discontinuous and complicated objects and has the high phase measurement accuracy. If high efficiency is desired, and the object does not have discontinuities, then the HT three-step FPP is a good choice. Higher efficiency can be achieved if the IECW is used only in the highest frequency, as is seen in Fig. 2 when the procedures are removed in the right red box.
for unit-frequency phase map, i.e. the fringe frequency of 𝑓 = 1, the subsequent unwrapping process are not needed. However, in the Lei’s method, the unwrapping process for unit-frequency may not be omitted. (3) Our method may achieve higher accuracy. If the phase errors are left to the unwrapping map for compensation. The phase error may cause the fringe orders change at higher frequencies, which cannot be compensated by any inverse-compensation method. 4.3. Unwrapping of different object
Acknowledgment
In the remainder of this section, the experiments of a ceramic bowl with a smooth surface and several steps are accomplished. As is seen in Fig. 7(c), the periodic error is obvious with the red line, but both of the proposed and HT method are much smaller with the blue and green line respectively. The unwrapped phase point cloud data are showed in Fig. 7(d), (e), (f). Similarly, Fig. 8 shows a gauge block with four steps. A row and a column of the unwrapped phase are plotted in Fig. 8(c) and (d). The uncorrected phases, MFPU without IECW, are shown in Fig. 8(a), and (c), (d) with dash-dotted line. At the same time, the corrected phases with the proposed method are showed in Fig. 8(b) and (c), (d) with solid line. It can be seen from Fig. 7(c), (d), (e) and Fig. 8. The proposed method of MFPU with IECW can work effectively for different complicated surface and can significantly improve the object’s smoothness.
This work was supported by the National Major Scientific Instruments and Equipment Development Project of the Ministry of Science and Technology of China (2013YQ220749). References [1] S. Zhang, S.T. Yau, High-resolution real-time 3D absolute coordinate measurement based on a phase-shifting method, Opt. Express 14 (2006) 2644–2649. [2] Q. Hu, P.S. Huang, Calibration of a 3-D shape measurement system, Opt Eng. 42 (2003) 487–493. [3] J. Geng, Structured-light 3D surface imaging: A tutorial, Adv. Opt. Photon. 3 (2011) 128–160. [4] L.D. Yu, W. Zhang, W.S. Li, C.L. Pan, H.J. Xia, Simplification of high order polynomial calibration model for fringe projection profilometry, Meas. Sci. Technol. 27 (2016) 105202.
81
C. Mao et al.
Optics Communications 419 (2018) 75–82 [16] C. Zuo, Q. Chen, G.H. Gu, J.L. Ren, X.B. Sui, Y.Z. Zhang, Optimized three-step phaseshifting profilometry using the third harmonic injection, Opt. Appl. 2 (2013) 393– 408. [17] M. Dai, F. Yang, X. He, Single-shot color fringe projection for three dimensional shape measurement of objects with discontinuities, Appl. Opt. 51 (2012) 2062–2069. [18] P.S. Huang, Q.Y. Hu, F.P. Chiang, Double three-step phase-shifting algorithm, Appl. Optim. 41 (2002) 4503–4509. [19] Z.K. Lei, C.L. Wang, C.L. Zhou, Multi-frequency inverse-phase fringe projection profilometry for nonlinear phase error compensation, Opt. Lasers Eng. 66 (2015) 249–257. [20] Z.W. Cai, X.L. Liu, H. Jiang, D. He, X. Peng, S.J. Huang, Z.H. Zhang, Flexible phase error compensation based on Hilbert transform in phase shifting profilometry, Opt. Express 23 (2015) 25171–25181. [21] X.Y. Su, W.S. Zhou, G. Von Bally, D. Vukicevic, Automated phase-measuring profilometry using defocused projection of a Ronchi grating, Opt. Commun. 94 (1992) 561–573. [22] S. Lei, S. Zhang, Flexible 3-D shape measurement using projector defocusing, Opt. Lett. 34 (2009) 3080–3082. [23] D.L. Zheng, F.P. Da, K.M. Qian, H.S. Seah, Phase error analysis and compensation for phase shifting profilometry with projector defocusing, Appl. Optim. 55 (2016) 5721–5728. [24] S. Zhang, Comparative study on passive and active projector nonlinear gamma calibration, Appl. Optim. 54 (2015) 3834–3841. [25] C. Zuo, L. Huang, M.L. Zhang, Q. Chen, A. Asundi, Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review, Opt. Lasers Eng. 85 (2016) 84–103.
[5] K. Liu, Real-Time 3-D Reconstruction by Means of Structured Light Illumination (Ph.D.), University of Kentucky, Kentucky, United State, 2010. [6] S. Zhang, Handbook of 3D Machine Vision: Optical Metrology and Imaging, first ed., CRC Press, 2013. [7] B. Pan, K.M. Qian, H. Lei, A. Asundi, Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry, Opt. Lett. 34 (2009) 416–418. [8] C.W. Zhang, Hong Zhao, L. Zhang, X. Wang, Full-field phase error detection and compensation method for digital phase-shifting fringe, Meas. Sci. Technol. 26 (2015) 035201. [9] C. Xiong, J. Yao, J.B. Chen, H. Miao, A convenient look-up-table based method for the compensation of non-linear error in digital fringe projection, Theoret. Appl. Mech. Lett. 6 (2016) 49–53. [10] K. Liu, Y.C. Wang, D.L. Lau, Q. Hao, L.G. Hassebrook, Gamma model and its analysis for phase measuring profilometry, J. Opt. Soc. Amer. A. 27 (2010) 553–562. [11] K. Yatabe, K. Ishikawa, Y. Oikawa, Compensation of fringe distortion for phase shifting three-dimensional shape measurement by inverse map estimation, Appl. Optim. 55 (2016) 6017–6024. [12] D. Zheng, F. Da, Gamma correction for two step phase shifting fringe projection profilometry, Optik 124 (2013) 1392–1397. [13] S. Ma, C. Quan, R. Zhu, L. Chen, B. Li, C.J. Tay, A fast and accurate gamma correction based on Fourier spectrum analysis for digital fringe projection profilometry, Opt. Commun. 285 (2012) 533–538. [14] Z.W. Li, Y.F. Li, Gamma-distorted fringe image modeling and accurate gamma correction for fast phase measuring profilometry, Opt. Lett. 36 (2011) 154–156. [15] Z.X. Xu, Y.H. Chan, Removing harmonic distortion of measurements of a defocusing three-step phase-shifting digital fringe projection system, Opt. Lasers Eng. 90 (2017) 139–145.
82
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
A multi-frequency inverse-phase error compensation method for projector nonlinear in 3D shape measurement Cuili Mao a,b , Rongsheng Lu a, *, Zhijian Liu a a b
School of Instrument Science and Opto-Electronics Engineering, Hefei University of Technology, Hefei 230009, China School of Mechanical & Automotive Engineering, Nanyang Institute of Technology, Nanyang 473004, China
ARTICLE
INFO
Keywords: Phase shifting profilometry Non-linear phase error Inverse-error compensation Multi-frequency method
ABSTRACT In fringe projection profilometry, the phase errors caused by the nonlinear intensity response of digital projectors needs to be correctly compensated. In this paper, a multi-frequency inverse-phase method is proposed. The theoretical model of periodical phase errors is analyzed. The periodical phase errors can be adaptively compensated in the wrapped maps by using a set of fringe patterns. The compensated phase is then unwrapped with multi-frequency method. Compared with conventional methods, the proposed method can greatly reduce the periodical phase error without calibrating measurement system. Some simulation and experimental results are presented to demonstrate the validity of the proposed approach.
1. Introduction Fringe projection profilometry (FPP) is one of the most widely used techniques for full-field three-dimensional (3D) shape measurement due to its qualities of non-contact, low-cost, easy availability, high resolution, etc. [1,2]. The phase-shifting fringe (PSF) is a popular branch of the FPP. It acquires 3D information from two-dimensional images by analyzing several deformed patterns projected by a projector and captured by CCD cameras [3–5]. The typical projection patterns include sinusoidal and binary fringe patterns, such as used in Fourier transform profilometry (FTP) [6], wavelet transform profilometry (WTP). Sinusoidal fringe patterns have been widely used in PSP for they have the ability to accomplish high accurate 3D shape measurement. However, The projector’s nonlinear response (i.e. gamma distortion) produces high-order harmonics, which deviates from its ideal sinusoidal distribution. Then phase errors inevitably occur in FPP [6]. To achieve accurate 3D shape measurement result, the phase error must be reduced. In recent years, a lot of theories and methods of alleviating the phase errors have been proposed with the configuration of digital video projectors (DVP) and CCD cameras. They can be mainly summarized into three categories: passively compensating phase error after the fringe patterns are captured [7–11], actively modifying the fringe patterns before their projection [12–17] and inverse-phase error compensation method [18–20]. For passively phase error compensation, one or more gamma coefficients should be firstly estimated from the captured fringe patterns using different algorithms. The distribution of phase error, *
which may be formed a look-up-table (LUT), is obtained by theoretical analysis and experiments. Then the phase error compensation can be applied to the deformed fringe patterns captured by the cameras with the calibration data. Zhang [8] proposed a full-field phase error detection and compensation method with the aid of the phase error distribution table. A mathematical model was developed for predicting the effects of non-unitary gamma [10]. The experimental results using simulated and real data have confirmed that the passive method of phase error compensation can achieve good effects under the conditions of constant measurement environment (i.e., constant ambient light illumination and surface reflection of the measured object, etc.) and stable system parameters (i.e., the parameters of the projector and camera). But the effectiveness of the passive method is diminished if the measurement conditions deviates from the calibration ones. Otherwise, the re-calibration should be carried out. At the same time the measurement precision and speed are closely related to the complexity of the error compensation algorithm. For achieving high precise sinusoidal fringe from projector, the actively phase error correction methods reencode the projector’s input fringe patterns according to the gamma value or the error distribution having been calibrated. The effectiveness of the active method is closely related to the calibration accuracy of the phase error and the accuracy of the re-coding fringe. Once the accurate coding stripes are produced, the measurement speed is relatively fast and no more phase error correction is required. But the re-calibrating process has to be done as the conditions, such as the system parameters
Corresponding author. E-mail address: [email protected] (R. Lu).
https://doi.org/10.1016/j.optcom.2018.03.006 Received 8 November 2017; Received in revised form 26 January 2018; Accepted 4 March 2018 0030-4018/© 2018 Elsevier B.V. All rights reserved.
C. Mao et al.
Optics Communications 419 (2018) 75–82
and/or the measurement environment, are changed. Inverse-phase error compensation methods are accomplished by projecting another set of fringes that have the opposite phase error with the original fringes. Phase error can be automatically eliminated by averaging the two sets of projection fringes. Huang [18] first proposed the embryonic form of this method. The multi-frequency inverse-phase error method was proposed by Lei [19] and has good effective results. It needs to unwrap the two sets of wrapped phases, which is time-consuming. In addition, the phase unwrapping should be firstly processed in spatial space, i.e., in the maps of the unit-frequency and the initial phase of pi/3. The process may cause the unwrapping phase error before the following phase compensation. For getting high measurement speed and accuracy, Cai [20] proposed an efficient method of phase error compensation by using Hilbert Transform (HT). Nevertheless, the measured object should not show discontinuities. For improving the measurement efficiency and reducing phase error, Su [21] proposed a binary fringe defocus method. But a low SNR and small measurement range is followed [22]. In order to compensate the phase error of the binary fringe patterns with projector defocusing, D. Zheng [23] compared the effect of double three-step, Hilbert three-step, five-step with theoretical analysis and experiments, and the conclusion was that the double three-step has the better overall effect. S. Zhang [24] compared the active and passive projector nonlinear gamma compensation methods, and finds that the active method tends to provide relatively more consistent high-quality result. He further found that the nonlinear phase error does not precisely follow a simple model with single gamma. Both active and passive phase error compensation methods need calibrate the distribution of phase error, then the error correction can be carried out, and the accuracy phase can be obtained under a certain condition. The calibration process must be implemented again with the change of the measurement conditions. Furthermore, the passive phase error compensation is applied to every pixel. It is obvious that the measurement speed is slow. However, no calibration is needed for inverse-phase error method, and the error compensation action is not necessary to apply to every pixel. In this paper, a new multi-frequency inverse-phase method with three-step phase shifting was proposed to compensate for nonlinear phase errors for higher accurate unwrapped phase map. The characteristics of the nonlinear phase error were analyzed. Then related experiments were performed to make the more comprehensive performance comparison with the commonly used active gamma compensation methods, the recently proposed HT method with 3-step phase shifting algorithm against the multi-frequency inverse-phase error compensation method proposed in this paper. The investigations show that our proposed method tends to provide more consistent high-quality unwrapping phase map. The rest of this paper is organized as follows. Section 2 gives the theoretical analysis of the FPP using sinusoidal patterns and the phase error model is showed, then the new phase error correction method is proposed. Section 3 gives the simulations of this method. Section 4 illustrates the experiment and the results. Section 5 summaries this paper.
be concise, coordinates (𝑥, 𝑦) are omitted hereafter, and 𝜑(𝑥, 𝑦) is the ideal phase. The gamma-distorted fringe patterns projected by a projector and captured by a camera, according to the power-law response, can be described by [20] 𝐼𝑛𝐶 = 𝑎(𝐼𝑖 )𝛾 + 𝑏 = 𝐵0 +
∞ ∑
(2)
[𝐵𝑘 cos(𝑘𝜑𝑛 )]
𝑘=1
where 𝐼𝑛𝐶 is the output grayscale value for a given input intensity value 𝐼𝑖 , 𝑎 and 𝑏 are constants, 𝛾 is the gamma factor of the used projector which is normally greater than 1, 𝐵0 is the average intensity (direct component), 𝐵𝑘 is the magnitude of the 𝑘th harmonic, 𝜑𝑛 = 𝜑 + 𝛿𝑛 represents the modulated phase coupled with the phase shift. Based on the least square algorithms (LSA) and the power-law response, the phase information can be retrieved from the captured images [5,7–9]. ∑𝑁−1 𝐶 𝐼 sin(𝛿𝑛 ) 𝜑𝐶 = − arctan ∑ 𝑛=0 𝑛 𝑁−1 𝐶 𝐼 𝑛=0 𝑛 cos(𝛿𝑛 ) (3) ] ∑𝑁−1 ∑∞ [ 𝑛=0 𝑘=1 𝐵𝑘 cos(𝑘𝜑𝑛 ) sin 𝛿𝑛 = − arctan ∑ ] 𝑁−1 ∑∞ [ 𝑛=0 𝑘=1 𝐵𝑘 cos(𝑘𝜑𝑛 ) cos 𝛿𝑛 where 𝜑𝐶 is the distorted phase due to the power-law response. Ideally, if 𝛾 is equals 1, 𝑘 equals 1. According to the trigonometric formula tan(𝛼 − 𝛽) =
(tan 𝛼 − tan 𝛽) (1 − tan 𝛼 tan 𝛽)
(4)
the phase error of N -step can be obtained [5] 𝛥𝜑𝑁 = 𝜑𝐶 − 𝜑 { = arctan
∑𝑁=1 ∑∞ [
]
𝑘=2 𝐵𝑘+1 − 𝐵𝑘−1 sin(𝑘𝜑𝑛 ) ] ∑𝑁=1 ∑∞ [ 𝑁𝐵1 + 𝑛=0 𝑘=2 𝐵𝑘+1 − 𝐵𝑘−1 cos(𝑘𝜑𝑛 ) 𝑛=0
}
(5)
Considering that 𝜑𝑛 = 𝜑 + 𝛿𝑛 = 𝜑 + 2𝜋𝑛∕𝑁, it can be proved easily { 0, 𝑘 ≠ 𝑚𝑁 𝑚 ∈ 𝑍+ 𝑁 sin(𝑚𝑁𝜑), 𝑘 = 𝑚𝑁, 𝑛=0 { 𝑁−1 ∑[ ] 0, 𝑘 ≠ 𝑚𝑁 cos(𝑘𝜑𝑛 ) = 𝑚 ∈ 𝑍+ 𝑁 cos(𝑚𝑁𝜑), 𝑘 = 𝑚𝑁, 𝑛=0 𝑁−1 ∑
[ ] sin(𝑘𝜑𝑛 ) =
(6)
So Eq. (5) can be simplified as } { ∑∞ 𝑚=0 (G𝑚𝑁+1 − G𝑚𝑁−1 ) sin(𝑚𝑁𝜑𝑛 ) (7) 𝛥𝜑𝑁 = arctan ∑ 1+ ∞ 𝑚=0 (G𝑚𝑁+1 − G𝑚𝑁−1 ) cos(𝑚𝑁𝜑𝑛 ) ] ∏ [ where 𝐺𝑆 = 𝐵𝑆 ∕𝐵1 = 𝑆𝑖=2 (𝛾 − 𝑖 + 1)∕(𝛾 + 𝑖) . As |(𝛾 − 𝑖 + 1)∕(𝛾 + 𝑖)| < 1 and is obviously decreases with the increase of harmonic order s, 𝐺𝑆 is far less than 1, so Eq. (6) can be further simplified as {∞ } ∑[ ] 𝛥𝜑𝑁 ≈ arctan (G𝑚𝑁+1 − G𝑚𝑁−1 ) sin(𝑚𝑁𝜑𝑛 ) 𝑚=0 (8) ∞ ∑[ ] ≅ 𝑐𝑚 sin(𝑚𝑁𝜑) 𝑚=1
where 𝑐𝑚 is constant. The harmonic coefficient of sin(𝑚𝑁𝜑) will drastically decrease as 𝑚increases, so we usually only focus on the low-order harmonic terms, then phase error is followed with different step of phase shifting.
2. Mathematical model of phase error 2.1. Phase analysis In FPP, a series of standard sinusoidal fringe patterns generated with computer, can be described by 𝐼𝑛𝑃 (𝑥, 𝑦) = 𝐴𝑃 (𝑥, 𝑦) + 𝐵 𝑃 (𝑥, 𝑦) cos[𝜑(𝑥, 𝑦) + 𝛿𝑛 ]
𝛥𝜑3 ≅ 𝑐1 sin(3𝜑) + 𝑐2 sin(6𝜑) ≈ 𝑐1 sin(3𝜑) 𝛥𝜑4 ≅ 𝑐1 sin(4𝜑) + 𝑐2 sin(8𝜑) ≈ 𝑐1 sin(4𝜑) 𝛥𝜑5 ≅ 𝑐1 sin(5𝜑) + 𝑐2 sin(10𝜑) ≈ 𝑐1 sin(5𝜑)
(1)
(9)
where 𝑐1 , 𝑐2 , 𝑐3 are constants and may have different value in Eq. (9). The phase error can be approximated as a periodic sinusoidal function and its period is related to the phase-shifting steps and the pitch of the fringe pattern. That is the reason why is called periodic system error. According to the phase error formulas, the periodic system error can
where 𝑛 = 0, 1, … , 𝑁 − 1 and 𝑁 denotes the number of phase steps in the phase shifting algorithm, 𝐼𝑛𝑃 is the intensity of the projected patterns, 𝐴𝑃 and 𝐵 𝑃 are the background intensity and intensity modulation amplitude respectively, 𝛿𝑛 = 2𝜋𝑛∕𝑁 is the phase-shifting amount. To 76
C. Mao et al.
Optics Communications 419 (2018) 75–82
reconstruct the 3D surface shape, the process of phase unwrapping must be carried out in order to remove the discontinuities from their principal values for obtaining the true continuous phase map. The basic unwrapping phase algorithm is as follows 𝛷(𝑥, 𝑦) = 𝜑(𝑥, 𝑦) + 2𝑘(𝑥, 𝑦)𝜋
(13)
where 𝜑(𝑥, 𝑦) is the wrapped phase obtained from Eq. (3), 𝛷(𝑥, 𝑦) is the unwrapped phase obtained from Eq. (12), and 𝑘(𝑥, 𝑦) is the integer representing fringe order. The key to a phase unwrapping algorithm is to obtain 𝑘(𝑥, 𝑦) accurately for each pixel, and the accuracy of phase unwrapping depends on the precision of 𝜑(𝑥, 𝑦). In order to correctly measure a discontinuity object with holes, steps and gaps, etc., unambiguous and exact absolute phase should be obtained. Multi-frequency phase-unwrapping algorithm has the ability to measure the 3D shape of discontinuous objects without complex phaseunwrapping operation. Three temporal phase unwrapping methods commonly used are analyzed and discussed in detail by Chao Zuo [25]. In order to improve accuracy and reliability of the unwrapped absolute phase, multi-frequency (hierarchical) temporal phase unwrapping method (MFPU) is selected in this paper according to comprehensive comparison. The coarse fringe pattern is used to get the fringe orders of fine fringe pattern, and the fine one to improve the accuracy of phase for obtaining object details. The principle of MFPU is { 𝛷ℎ = 𝜑ℎ + 2𝑘ℎ 𝜋 (14) 𝛷𝑙 = 𝜑𝑙 + 2𝑘𝑙 𝜋 where the 𝑘ℎ and 𝑘𝑙 are the respective integer fringe orders of the fine and coarse fringe patterns, 𝛷ℎ , 𝛷𝑙 , 𝜑ℎ , 𝜑𝑙 are unwrapping and wrapping phases of fine and coarse fringe patterns respectively. In two-frequency temporal phase unwrapping, the low-resolution phase distribution is retrieved by using a set of unit-frequency patterns, and thus no phase unwrapping is required for 𝜑𝑙 , that is, 𝛷ℎ = 𝜑𝑙 , so the fringe order 𝑘𝑙 = 0. Then the fringe order 𝑘ℎ for each pixel can be determined easily ] [ (𝑓ℎ ∕𝑓𝑙 )𝜑𝑙 − 𝜑ℎ (15) 𝑘ℎ = 𝑅𝑜𝑢𝑛𝑑 2𝜋
Fig. 1. Error distribution of before and after phase shifting. (a) 3-step, (4) 4-step.
be eliminated using the half cycle elimination error method, i.e., the reverse error compensation method. If two patterns are projected with half-cycle translation, their phase errors have equal absolute error values and opposite in signs. Then the periodic error can be automatically eliminated by averaging the two phases with inverse error for every pixel. For example, when 𝑁 = 3 is used, it is not difficult to find that the dominating phase error is periodic with a period of 2𝜋∕3. The another 3-step phase shifts have an initial phase offset of 𝜋∕3, i.e., half of the error period according to the analysis above. Similarly, the initial phase offset should be 𝜋∕4 and 𝜋∕5 for 𝑁 = 4 and 𝑁 = 5 respectively, then the other phase error is inferred as follows 𝛥𝜑′3 ≅ 𝑐1 sin[3(𝜑 + 𝜋∕3)] + 𝑐2 sin[6(𝜑 + 𝜋∕3)] + 𝑐3 sin[9(𝜑 + 𝜋∕3)] = 𝑐1 sin(3𝜑 + 𝜋) + 𝑐2 sin(6𝜑 + 2𝜋) + 𝑐3 sin(9𝜑 + 3𝜋) = −𝑐1 sin(3𝜑) + 𝑐2 sin(6𝜑) − 𝑐3 sin(9𝜑)
(10)
𝛥𝜑′4 ≅ 𝑐1 sin[4(𝜑 + 𝜋∕4)] = −𝑐1 sin(4𝜑)
(11)
𝛥𝜑′5
(12)
≅ 𝑐1 sin[5(𝜑 + 𝜋∕5)] = −𝑐1 sin(5𝜑)
where Round() represents a function of rounded integer of a decimal number, 𝑓𝑙 and 𝑓ℎ are the frequency of coarse and fine fringe respectively. Therefore, the accurate absolute unwrapped phase of high frequency fringe patterns can be obtained with Eq. (14). 2.3. Inverse-phase error compensation in wrapped phase (IECW) Previous inverse-phase error compensation method is done with unwrapping phase map, which should be obtained firstly. However, the mistake unwrapped phase will take place with fringe accompanied by large noise, especially when the frequency is a unit-frequency with a phase shifting. If the phase error compensation is done in the wrapped phase, this problem can be solved immediately without phase unwrapping. The wrapped phases of two sets of PSF which are with 𝜋∕3 phaseshifting are 𝜑′1 , 𝜑′2 respectively, it can be obtained according to above analysis 𝜑′1 = 𝜑 + 𝛥𝜑3
It can be concluded that the opposite phase error occurs if the initial phase offset is 𝜋∕3, 𝜋∕4, 𝜋∕5 for 𝑁 = 3, 4, 5 respectively. According to (8) and (9), it is clear that the odd harmonic error of (2m-1)N may be eliminated when phase shifting step is odd, that is 𝑁 = 3, 5, … , (2𝑘 + 1). The phase errors of step 𝑁 = 3, 4 are shown in Fig. 1, where the ideal phase is obtained with 20-step phase shifting method. The phase errors are opposite each other and the frequency ratio of phase errors is 9∕12 = 3∕4 at 600–700 pixels by looking at the axes of Fig. 1, which is consistent with the above reasoning.
𝜑′2 = 𝜑 + 𝜋∕3 − 𝛥𝜑3
(16)
The real wrapped phase can be obtained with the proposed IECW. { ′ (𝜑1 + 𝜑′2 − 𝜋∕3)∕2, 𝜑′1 < 𝜑′2 𝜑(𝑥, 𝑦) = (17) ′ ′ ′ ′ (𝜑 + 𝜑 − 𝜋∕3 + 2𝜋)∕2, 𝜑1 < 𝜑2 1
2
According to Eqs. (9)–(12), the residual error of the 3-step, 4-step and 5-step PSF can be obtained 𝛥3 = = 𝛥4 = = 𝛥5 = =
2.2. Principle of multi-frequency method The phase distribution obtained according to Eq. (2) is wrapped into the range (−𝜋, 𝜋]. Hence the result is given modulo 2𝜋 and discontinuities occur with values near to 2𝜋 in the phase distribution. To correctly 77
(𝜑′1 + 𝜑′2 − 𝜋∕3)∕2 − 𝜑 (𝛥𝜑3 + 𝛥𝜑′3 )∕2 ≈ 𝑐2 sin(6𝜑) (𝜑′1 + 𝜑′2 − 𝜋∕4)∕2 − 𝜑 (𝛥𝜑4 + 𝛥𝜑′4 )∕2 ≈ 𝑐2 sin(8𝜑) (𝜑′1 + 𝜑′2 − 𝜋∕5)∕2 − 𝜑 (𝛥𝜑4 + 𝛥𝜑′4 )∕2 ≈ 𝑐2 sin(10𝜑)
(18)
C. Mao et al.
Optics Communications 419 (2018) 75–82
Fig. 2. Combination flowchart for the IECW algorithm. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Fig. 3. The 50th line of the processing procedure (𝛾 = 2) (a) IECW and (b) MFPU.
When a limit error is set as 𝜀, if |𝛥𝜑| > 𝜀, the invalid point with large phase error can be detected straightforwardly using the proposed method. It can be seen from Eq. (15) that if the 𝜑ℎ or 𝜑𝑙 has large error, the fringe order 𝑘ℎ may be wrong and further the wrong unwrapped phase may be produced from Eq. (13). Compared with the popular modulation-based invalid point detection method, our detection technique has unique feature that we can classify valid and invalid points by phase error, which is more meaningful for objects with very high or very low reflectivity.
As we all know, the harmonic coefficient will decrease drastically with the increase of harmonic frequency, so 𝑐2 is much less than 𝑐1 , and can be ignored. 2.4. Invalid point detection According to Eq. (16), the phase error from the two sets of IECW can be achieved 𝛥𝜑 = (𝜑′1 − 𝜑′2 − 𝜋∕3)∕2
(19) 78
C. Mao et al.
Optics Communications 419 (2018) 75–82
Fig. 4. The processing of the 50th line (𝛾 = 2), noise amplitude Am=40 (a) IECW (b) MFPU without IECW (c) MFPU with IECW.
3. Simulations According to the above analysis, the phase error is decreased with the increase of phase shifting step, but the efficiency is reduced with increase of phase shifting step, so this paper uses the minimal 3-step for verifying the effectiveness of the proposed method. The projector’s gamma nonlinear effect can be simulated by an exponential model according to Eq. (2). There is no effect on fringe images when 𝛾 equals 1, here the common approximations 𝛾 = 2 is adopted. A 3-step FPP method with different frequencies is used to explain the combination flowchart for the MFPU with IECW algorithm, as is shown in Fig. 2 with two sets of 0 and 𝜋∕3 initial phase respectively. The 50th line of the processing procedure with MFPU and IECW is drawn in Fig. 3. In order to test the anti-noise ability of this method, a large amplitude noise of the 15% signal amplitude is added, and the simulation results is shown in Fig. 4. Not only the nonlinear gamma error reduce, the influence of noise has also been a certain degree of suppression as seen from Fig. 4(b) and (c), which respectively represents the unwrapping result without correction and with IECW. It can be seen from the results that the combination of IECW and MFPU can increase phase accuracy and also have the ability of detecting invalid points.
Fig. 5. Phase error distribution of the FPP with different phase error correction method. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
79
C. Mao et al.
Optics Communications 419 (2018) 75–82
Fig. 6. Unwrapped phase map (a) MFPU without IECW (b) MFPU with IECW (c) HT.
Fig. 7. Experimental results for measuring a ceramic bowl (a) photograph of the ceramic bowl (b) wrapped phase (c) phase distributions of the 800th row (d) MFPU without IECW (e) MFPU with IECW (f) HT.
4. Experiments and comparison with others
without any compensation, is three times the signal frequency (black PS fringe), and the residual phase error (indicated by blue solid line) is six times the signal frequency. The experimental results are consistent with the above analysis. The results showed in Fig. 5 and Fig. 6 made it clear that the proposed method, MFPU with IECW, has the optimum performance for nonlinear phase error compensation. The above experiments verified not only our previous theoretical analysis and simulations, but also the effectiveness of the proposed method without complicated calibrations. It is worth mentioned that the idea proposed in this work is inspired by the Lei’s paper [19]. The principle is very similar to the one proposed in the Lei’s paper. In our proposed method phase compensation is performed in the wrapped phase map, but it is done with the unwrapped phase map in the Lei’s method [19]. The main differences between them are: (1) Our method may achieve higher efficiency because the required number of captured images is less than the ones of Lei’s method. This is because that the compensation in our method may be applied only to the highest frequency, but to all the frequencies in the Lei’s paper. Here a single example will suffice. If four frequencies are used, the number differences of captured images in the two methods are listed in Table 1: Even if the Lei’s compensation is applied only to the highest frequency in the unwrapping maps, our method process is also quicker. This is because that our method requires less once than that of the Lei’s method for phase unwrapping. (2) Another advantage of our method is that, if the phase compensation is carried out in the wrapped map, such as our method does,
In this section, several classical phase error correction algorithms were compared with a flat plane. Then a ceramic bowl with a smooth surface and different steps was used to examine the effect of the proposed method. 4.1. System setup The experimental FPP setup consists of a DLP projector (LightCrafter4500, resolution:1024 × 768), a CCD camera (PointGrey FL3-U3-13E4C-C, resolution:1280 × 1024), and a computer with Intel(R) core(TM) i3-4150 CPU @3.50 GHZ and 4.00 GB RAM. The computer produces ideal sinusoidal fringes, which are projected to an object, then the camera capture the deformed fringes modulated by the surface of the object. 4.2. Comparison of algorithms Several classical phase error compensation algorithms, here HT 3-step, 3-step with MFPU and IECW, classical gamma correction algorithms and the classical 3-step without phase error correction are compared with a flat plane target in the experiments. The ideal unwrapping phase was obtained with 20-step phase shifting patterns. The unwrapping phase errors are shown in Fig. 5 and the unwrapped phase in Fig. 6(b) and Fig. 6(c) using the proposed method and HT method. It is obvious from Fig. 5 that the largest phase error (the red dashed line), 80
C. Mao et al.
Optics Communications 419 (2018) 75–82
Fig. 8. Experimental results for measuring a group of gauges (a) MFPU without IECW (b) MFPU with IECW (c) phase distributions of a row (d) phase distributions of a column. Table 1 Comparison of the two methods.
5. Conclusions
N-step
Method
Number of unwrapping
Number of images
3-step
Z.K. Lei’s The proposed
4×2=8 4
3 × 4 × 2 = 24 3 × (4 + 1) = 15
4-step
Z.K. Lei’s The proposed
4×2=8 4
4 × 4 × 2 = 32 4 × (4 + 1) = 20
In this paper, the phase error model of off-the-shelf digital video projectors is analyzed for the classical 3-, 4- and 5-step FPP. A new phase error correction method, MFPU with IECW, was proposed for projector’s nonlinear response. After presenting the comprehensive theoretical derivation and simulation of the nonlinear phase error for the proposed method with 3-step FPP, several commonly used phase error compensation methods are compared. According to the experimental results, it can be seen that the proposed method has the minimal residual phase error and can realize the 3D shape measurement with discontinuous and complicated objects and has the high phase measurement accuracy. If high efficiency is desired, and the object does not have discontinuities, then the HT three-step FPP is a good choice. Higher efficiency can be achieved if the IECW is used only in the highest frequency, as is seen in Fig. 2 when the procedures are removed in the right red box.
for unit-frequency phase map, i.e. the fringe frequency of 𝑓 = 1, the subsequent unwrapping process are not needed. However, in the Lei’s method, the unwrapping process for unit-frequency may not be omitted. (3) Our method may achieve higher accuracy. If the phase errors are left to the unwrapping map for compensation. The phase error may cause the fringe orders change at higher frequencies, which cannot be compensated by any inverse-compensation method. 4.3. Unwrapping of different object
Acknowledgment
In the remainder of this section, the experiments of a ceramic bowl with a smooth surface and several steps are accomplished. As is seen in Fig. 7(c), the periodic error is obvious with the red line, but both of the proposed and HT method are much smaller with the blue and green line respectively. The unwrapped phase point cloud data are showed in Fig. 7(d), (e), (f). Similarly, Fig. 8 shows a gauge block with four steps. A row and a column of the unwrapped phase are plotted in Fig. 8(c) and (d). The uncorrected phases, MFPU without IECW, are shown in Fig. 8(a), and (c), (d) with dash-dotted line. At the same time, the corrected phases with the proposed method are showed in Fig. 8(b) and (c), (d) with solid line. It can be seen from Fig. 7(c), (d), (e) and Fig. 8. The proposed method of MFPU with IECW can work effectively for different complicated surface and can significantly improve the object’s smoothness.
This work was supported by the National Major Scientific Instruments and Equipment Development Project of the Ministry of Science and Technology of China (2013YQ220749). References [1] S. Zhang, S.T. Yau, High-resolution real-time 3D absolute coordinate measurement based on a phase-shifting method, Opt. Express 14 (2006) 2644–2649. [2] Q. Hu, P.S. Huang, Calibration of a 3-D shape measurement system, Opt Eng. 42 (2003) 487–493. [3] J. Geng, Structured-light 3D surface imaging: A tutorial, Adv. Opt. Photon. 3 (2011) 128–160. [4] L.D. Yu, W. Zhang, W.S. Li, C.L. Pan, H.J. Xia, Simplification of high order polynomial calibration model for fringe projection profilometry, Meas. Sci. Technol. 27 (2016) 105202.
81
C. Mao et al.
Optics Communications 419 (2018) 75–82 [16] C. Zuo, Q. Chen, G.H. Gu, J.L. Ren, X.B. Sui, Y.Z. Zhang, Optimized three-step phaseshifting profilometry using the third harmonic injection, Opt. Appl. 2 (2013) 393– 408. [17] M. Dai, F. Yang, X. He, Single-shot color fringe projection for three dimensional shape measurement of objects with discontinuities, Appl. Opt. 51 (2012) 2062–2069. [18] P.S. Huang, Q.Y. Hu, F.P. Chiang, Double three-step phase-shifting algorithm, Appl. Optim. 41 (2002) 4503–4509. [19] Z.K. Lei, C.L. Wang, C.L. Zhou, Multi-frequency inverse-phase fringe projection profilometry for nonlinear phase error compensation, Opt. Lasers Eng. 66 (2015) 249–257. [20] Z.W. Cai, X.L. Liu, H. Jiang, D. He, X. Peng, S.J. Huang, Z.H. Zhang, Flexible phase error compensation based on Hilbert transform in phase shifting profilometry, Opt. Express 23 (2015) 25171–25181. [21] X.Y. Su, W.S. Zhou, G. Von Bally, D. Vukicevic, Automated phase-measuring profilometry using defocused projection of a Ronchi grating, Opt. Commun. 94 (1992) 561–573. [22] S. Lei, S. Zhang, Flexible 3-D shape measurement using projector defocusing, Opt. Lett. 34 (2009) 3080–3082. [23] D.L. Zheng, F.P. Da, K.M. Qian, H.S. Seah, Phase error analysis and compensation for phase shifting profilometry with projector defocusing, Appl. Optim. 55 (2016) 5721–5728. [24] S. Zhang, Comparative study on passive and active projector nonlinear gamma calibration, Appl. Optim. 54 (2015) 3834–3841. [25] C. Zuo, L. Huang, M.L. Zhang, Q. Chen, A. Asundi, Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review, Opt. Lasers Eng. 85 (2016) 84–103.
[5] K. Liu, Real-Time 3-D Reconstruction by Means of Structured Light Illumination (Ph.D.), University of Kentucky, Kentucky, United State, 2010. [6] S. Zhang, Handbook of 3D Machine Vision: Optical Metrology and Imaging, first ed., CRC Press, 2013. [7] B. Pan, K.M. Qian, H. Lei, A. Asundi, Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry, Opt. Lett. 34 (2009) 416–418. [8] C.W. Zhang, Hong Zhao, L. Zhang, X. Wang, Full-field phase error detection and compensation method for digital phase-shifting fringe, Meas. Sci. Technol. 26 (2015) 035201. [9] C. Xiong, J. Yao, J.B. Chen, H. Miao, A convenient look-up-table based method for the compensation of non-linear error in digital fringe projection, Theoret. Appl. Mech. Lett. 6 (2016) 49–53. [10] K. Liu, Y.C. Wang, D.L. Lau, Q. Hao, L.G. Hassebrook, Gamma model and its analysis for phase measuring profilometry, J. Opt. Soc. Amer. A. 27 (2010) 553–562. [11] K. Yatabe, K. Ishikawa, Y. Oikawa, Compensation of fringe distortion for phase shifting three-dimensional shape measurement by inverse map estimation, Appl. Optim. 55 (2016) 6017–6024. [12] D. Zheng, F. Da, Gamma correction for two step phase shifting fringe projection profilometry, Optik 124 (2013) 1392–1397. [13] S. Ma, C. Quan, R. Zhu, L. Chen, B. Li, C.J. Tay, A fast and accurate gamma correction based on Fourier spectrum analysis for digital fringe projection profilometry, Opt. Commun. 285 (2012) 533–538. [14] Z.W. Li, Y.F. Li, Gamma-distorted fringe image modeling and accurate gamma correction for fast phase measuring profilometry, Opt. Lett. 36 (2011) 154–156. [15] Z.X. Xu, Y.H. Chan, Removing harmonic distortion of measurements of a defocusing three-step phase-shifting digital fringe projection system, Opt. Lasers Eng. 90 (2017) 139–145.
82