Single-shot phase measuring profilometry based on color binary grating with intervals

Optics Communications 451 (2019) 268–275

Contents lists available at ScienceDirect

Optics Communications journal homepage: www.elsevier.com/locate/optcom

Single-shot phase measuring profilometry based on color binary grating with intervals Yapin Wang, Yiping Cao ∗, Guangkai Fu, Lu Wang, Yingying Wan, Chengmeng Li Department of Optical Electronics Sichuan University, Chengdu Sichuan 610064, PR China

ARTICLE

INFO

Keywords: Three-dimensional measurement Phase measuring profilometry Color binary grating Color crosstalk

ABSTRACT A single-shot phase measuring profilometry (PMP) based on color binary grating with intervals is proposed. In traditional PMP based on binary grating, the width for non-zero transmittance of the binary grating is always 1/2 periods rigorously. In our proposed method, a color binary grating is introduced in which three binary gratings with the same period and the same width for non-zero transmittance less than 1/3 periods are misaligned 1/3 periods one another and programmed in red (R), green (G) and blue (B) channels respectively. Because the widths for non-zero transmittance in R, G and B channels of the introduced color binary grating are all less than 1/3 periods and misaligned 1/3 periods one another, the color overlap can be avoided from the source. When measuring the object, only a single-shot deformed pattern needs to be captured and three nearly unbroken sinusoidal deformed patterns with an equivalent phase-shifting of 2𝜋∕3 can be extracted from R, G and B components of the captured color deformed pattern. So the 3D shape of the measured object can be successfully reconstructed with three-step PMP. Experiment results show the feasibility of the proposed method. Due to its single-shot feature, it is promising to measure the dynamic objects.

1. Introduction The optical three-dimensional (3D) measurement based on structured light projection has been widely used in many fields such as computer vision [1], face recognition [2], industry inspection [3] and so on due to its good performances of non-contact feature, full-field, high precision and easy to information processing [4]. The Fourier transform profilometry (FTP) [5] and phase measuring profilometry (PMP) [6] are the two most popular methods to realize 3D shape reconstruction. The FTP can be applied into real-time or dynamic 3D measurement because it can reconstruct 3D shape of the object from one deformed pattern [7], but its measuring accuracy is somehow limited due to the filtering operation in spatial frequency spectrum domain [8]. Compared with FTP, the PMP can achieve higher measuring accuracy due to its point to point calculation, but it needs three or more phase-shifting deformed patterns [9,10] so the real-time or dynamic 3D measurement may be limited if PMP is directly applied. In order to achieve real-time PMP, the color fringe projection profilometry based on PMP has been presented [11,12]. In this method, a color fringe whose red(R), green(G) and blue(B) components encoded by three sinusoidal gratings with an equivalent phase-shifting of 2𝜋∕3 is projected onto the measured object [13], only one color deformed pattern caused by the profile of the object will be captured by a color charge coupled device (CCD) camera, then the corresponding phaseshifting deformed patterns can be separated simply from the R, G and B ∗

components of the captured color deformed pattern. So the 3D shape of the object to be measured can be reconstructed successfully. However, the color crosstalk problem [14,15] caused by the color overlap among R, G and B channels may introduce measuring errors. Many methods were proposed to solve the problem. Cao et al. proposed an improved RGB tricolor based on fast phase measuring profilometry in which the chroma transfer function (CTF) was introduced to calibrate the color crosstalk and the grayscale imbalance among the three channels as soon as possible [16]. Huang et al. proposed a look-up table (LUT) method to compensate for the color coupling and grayscale imbalance errors [14], but the crosstalk cannot be solved completely. Pan et al. proposed a hardware-based method to compensate for the color coupling and grayscale imbalance errors [17] by designing a threeCCD camera system and a three color filters detect system, but the measuring system may be more complicated due to the additional multiple CCD cameras and color filters [18]. Hu et al. proposed a blind color isolation (BCI) method to adaptively determine the demixing matrix among the three channels in which no extra images are introduced for color calibration [19]. This method did reduce the errors caused by the color crosstalk and grayscale imbalance problems, but computational process of this method is complicated and costs lots of time. Flores et al. proposed a color-fringe pattern profilometry by using a generalized phase-shifting algorithm [20] to reduce the influence of the color crosstalk effectively, but measuring accuracy

Corresponding author. E-mail address: [email protected] (Y. Cao).

https://doi.org/10.1016/j.optcom.2019.06.062 Received 12 April 2019; Received in revised form 6 June 2019; Accepted 24 June 2019 Available online 2 July 2019 0030-4018/© 2019 Elsevier B.V. All rights reserved.

Y. Wang, Y. Cao, G. Fu et al.

Optics Communications 451 (2019) 268–275

spectrum domain. The region of interest (ROI) frequency components in the dotted box of Fig. 2(b) can be remained by a proper low-pass filter, as shown in Fig. 2(c). At last, by inverse Fourier transform (IFT), the unbroken sinusoidal fringe pattern can be retrieved successfully as shown in Fig. 2(d). 2.1. The color binary grating encoding principle Fig. 1. The traditional binary grating and its cutaway view: (a) Traditional binary grating, (b) Cutaway view of (a).

Because the projection system does not exactly match the CCD camera’s magnification ratio, one pixel of the DLP may not be one pixel of CCD camera; Even though the projection system exactly matches the CCD camera’s magnification ratio, there will be chromatic aberration caused by optical lens. So the color crosstalk must be considered in PMP based color fringe pattern projection. In order to solve this problem and realize single-shot PMP, a color binary grating with intervals is proposed in which three binary gratings with the same period and the same duty cycle less than 1/3 are misaligned 1/3 periods one another and programmed in R, G and B channels respectively as shown in Fig. 3(a). Its partial enlarged pattern is shown in Fig. 3(b). It can be seen that there are dark intervals among the R, G and B components of the proposed color binary grating. As long as the dark intervals are wide enough, this proposed color binary grating can effectively avoid the color crosstalk problem from the source.

of this method may be effected by the nonlinearity caused by the projector when projecting color sinusoidal fringe and the refresh rate of the projector is limited to 120 Hz [21]. Furthermore, the chromatic aberration among the color channels existed in the traditional refractive lens of the projection system will also introduce measuring errors. The chromatic aberration problem is analyzed and a chromatic aberration compensation algorithm is proposed by Zhang et al. [22], but this method is only applied into the optimum-fringe selection method [23]. Just known for us, the digital light projector (DLP) is widely used as the structured light projector with which the binary grating has a higher frame refresh rate than the sinusoidal grating. So the binary grating is promising to be applied in dynamic 3D measurement. Ekstrand et al. build a 3D measurement system with nearly focused binary patterns in which the conventional sinusoidal fringe is approximated by the nonlinear error suppression [24]. But nine or more phase shifting steps algorithm must be applied and the measuring accuracy may be limited because the period of the used binary grating must be wider than 36 pixels. Li et al. proposed a 3D shape measurement based on binary color fringe defocused projection which can eliminate the nonlinear gamma caused by using the projected fringe compounded by sinusoidal fringes [25]. But this method needs a specific defocusing device and the color crosstalk problem still unavoidable. More advanced design of pulse width modulation (PWM) proposed by Zuo et al. can provide much better fringe profile [26,27]. The waveform can be optimized to resist certain harmonics and thus provides better performance under defocusing. But this method still needs a specific defocusing device. And the duty cycle of the binary fringe applied in the above three methods is always 1/2. Recently Fu et al. in our research group proposed a PMP based on binary fringe with duty cycle of 1/3 [28]. In order to realize real-time or dynamic 3D measurement and avoid color crosstalk problem, a single-shot PMP based on color binary grating with intervals is proposed. And it must be pointed out that when measuring the objects with colorful texture, the accuracy of the proposed method may somewhat be limited due to the color fringe projection

2.2. The sinusoidal fringe pattern extracting principle The corresponding mathematical model of the proposed color binary grating with intervals 𝐼𝐶 (𝑥, 𝑦) can be expressed as: ⃖⃖⃗ + 𝐼𝐺 (𝑥, 𝑦)𝐺 ⃖⃖⃗ + 𝐼𝐵 (𝑥, 𝑦)𝐵 ⃖⃖⃗ 𝐼𝐶 (𝑥, 𝑦) = 𝐼𝑅 (𝑥, 𝑦)𝑅

(1)

Where the 𝐼𝑅 (𝑥, 𝑦), 𝐼𝐺 (𝑥, 𝑦) and 𝐼𝐵 (𝑥, 𝑦) represent the grayscale distributions of R, G and B components in the color binary fringe, they can be expressed as: ⎧𝐼 (𝑥, 𝑦) = 𝐴 rect( 𝑥 ) ∗ comb( 𝑥 ) 0 ⎪ 𝑅 𝑤1 𝑇0 ⎪ ⎪𝐼 (𝑥, 𝑦) = 𝐴 rect( 𝑥 ) ∗ comb( 𝑥 − 𝑇0 ∕3 ) 0 ⎨ 𝐺 𝑤1 𝑇0 ⎪ 𝑥 − 2𝑇0 ∕3 𝑥 ⎪ ) ⎪𝐼𝐵 (𝑥, 𝑦) = 𝐴0 rect( 𝑤 ) ∗ comb( 𝑇0 1 ⎩

(2)

Without loss of generality, the R component 𝐼𝑅 (𝑥, 𝑦) is analyzed in detail. The Fourier spectrum 𝐺𝑅 (𝑓𝑥 , 𝑓𝑦 ) of the 𝐼𝑅 (𝑥, 𝑦) can be expressed as: 𝐺𝑅 (𝑓𝑥 , 𝑓𝑦 ) = 𝐴0 𝑤1

𝑗=∞ ∑

sinc(𝑤1 𝑓𝑥 ) ⋅ 𝛿(𝑓𝑥 − 𝑗𝑓0 )

(3)

𝑗=−∞

2. The proposed color binary grating encoding principle

Where, 𝑓0 = 1∕𝑇0 . It represents the fundamental frequency of the Fourier spectrum. By introducing a proper low-pass filter, the zero frequency spectrum, the positive fundamental frequency spectrum and the negative fundamental frequency spectrum can be filtered out as 𝐺𝐹 𝑅 (𝑓𝑥 , 𝑓𝑦 ):

The traditional binary grating in PMP is shown in Fig. 1(a). As can be seen in Fig. 1(b), the width of non-zero transmittance 𝑤1 is the same as that of zero transmittance 𝑤0 in one period 𝑇0 , that is the duty cycle 𝑤1 ∕𝑇0 maintains 1/2. By proper defocusing the projected binary grating, the corresponding sinusoidal fringe pattern can be approximated. But the binary defocusing method requires careful adjustment of the projector’s lens [29]. Fu et al. in our research group have proposed real-time 3D measuring method with a 1/3 duty cycle binary grating [28]. Just by proper filtering operation in spatial frequency domain, the unbroken sinusoidal fringe information can be extracted efficiently from the captured deformed pattern [28]. Thus, no defocusing projection is needed. A brief description of the whole process of extracting sinusoidal fringe pattern from the encoded binary grating with Fu’s method is as follows: A fringe pattern captured by the CCD camera is shown in Fig. 2(a). The spectrum information of the captured fringe pattern can be obtained by fast Fourier transform (FFT). As can be seen form Fig. 2(b), there are many higher harmonic frequencies in the spatial frequency

𝐺𝐹 𝑅 (𝑓𝑥 , 𝑓𝑦 ) = 𝐴0 𝑤1 [𝛿(𝑓𝑥 ) + sinc(

𝑤1 𝑤 )𝛿(𝑓𝑥 + 𝑓0 ) + sinc( 1 )𝛿(𝑓𝑥 − 𝑓0 )] (4) 𝑇0 𝑇0

By IFT, the light intensity distribution 𝐼𝐹 𝑅 (𝑥, 𝑦) can be expressed as: 𝐼𝐹 𝑅 (𝑥, 𝑦) = 𝐴0 𝑤1 + 2𝐴0 𝑤1 sinc(

𝑤1 ) cos(2𝜋𝑓0 𝑥) 𝑇0

(5)

It can be seen from Eqs. (4) and (5) that even though the 𝑤1 of the proposed color binary grating is less than that of 𝑤0 , the Fourier spectrum 𝐺𝐹 𝑅 (𝑓𝑥 , 𝑓𝑦 ) still contains the sinusoidal fringe information. 𝐼𝐹 𝑅 (𝑥, 𝑦) shows the sinusoidal fringe pattern feature and serves as the extracted nearly unbroken sinusoidal fringe pattern. It can be simplified as: 𝐼𝐹 𝑅 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓0 𝑥) 269

(6)

Y. Wang, Y. Cao, G. Fu et al.

Optics Communications 451 (2019) 268–275

Fig. 2. Sinusoidal fringe pattern extracting process: (a) Captured fringe pattern, (b) Fourier spectrum, (c) Remained spectrum by proper filtering, (d) Extracted sinusoidal fringe pattern.

Fig. 3. The encoded color binary grating principle: (a) The encoded color binary grating, (b) The partial enlarged pattern.

Fig. 4. The process of extracting sinusoidal pattern from the proposed color fringe pattern: (a) the simulated diagram of the whole process of extracting sinusoidal patterns, (b) the cutaway views of the corresponding three sinusoidal patterns with 2𝜋∕3.

Where 𝐴0 𝑤1 simplified as 𝐴(𝑥, 𝑦) represents the background light intensity, 2𝐴0 𝑤1 sinc(𝑤1 ∕𝑇0 ) simplified as𝐵(𝑥, 𝑦) reflects the contrast of the fringe pattern. In the same way, the extracted nearly unbroken sinusoidal fringe patterns 𝐼𝐹 𝐺 (𝑥, 𝑦) and 𝐼𝐹 𝐵 (𝑥, 𝑦) can also be extracted from G and B components, they can be represented respectively as: 𝐼𝐹 𝐺 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓0 𝑥 + 2𝜋∕3)

(7)

𝐼𝐹 𝐵 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓0 𝑥 + 4𝜋∕3)

(8)

Fig. 5. The RMS of different periods pixels.

It can be seen that the extracted three nearly unbroken sinusoidal fringe patterns have an equivalent phase-shifting of 2𝜋∕3 from Eqs. (6), (7) and (8). According to the method proposed in Fig. 2, the three nearly unbroken sinusoidal fringe patterns with an equivalent phase-shifting of 2𝜋∕3 one another can be respectively extracted from R, G and B channels (RC, GC and BC) of the proposed color binary grating and the simulated diagram of the whole process is shown in Fig. 4(a). After normalization, their cutaway views are shown in Fig. 4(b), it can be seen that the extracted three nearly unbroken sinusoidal fringe patterns have an equivalent phase-shifting of 2𝜋∕3 one another, so they can be used to reconstruct the 3D shape of the measured object with three-step PMP. In order to show the relationship between the period of the fringe and the quality of the fringe pattern, a series of experiments to measure the reference plane are made using different periods. By comparing the extracted cutaway views with the corresponding fitted sine curve which is approximate the true value, the quality of the extracted fringe pattern can be evaluated. The root of mean square error (RMS) between the

extracted cutaway views and the corresponding fitted sine curve can be calculated. Fig. 5 shows the RMS of different periods. It can be seen from Fig. 5 that the RMS has a fluctuations in a small range from 0.0116 to 0.0942 and does not present monotonicity according to the period. The RMS of period with 21 pixels is the smallest, it is 0.0116. 2.3. The PMP principle based on the proposed color binary grating When the proposed color binary grating is projected onto the reference plane by the DLP, the corresponding color fringe pattern 𝐼𝐶𝑟𝑒𝑓 (𝑥, 𝑦) can be captured by a color CCD camera expressed as: ⃖⃖⃗ + 𝐼𝐺𝑟𝑒𝑓 (𝑥, 𝑦)𝐺 ⃖⃖⃗ + 𝐼𝐵𝑟𝑒𝑓 (𝑥, 𝑦)𝐵 ⃖⃖⃗ 𝐼𝐶𝑟𝑒𝑓 (𝑥, 𝑦) = 𝐼𝑅𝑟𝑒𝑓 (𝑥, 𝑦)𝑅

(9)

Where 𝐼𝑅𝑟𝑒𝑓 (𝑥, 𝑦), 𝐼𝐺𝑟𝑒𝑓 (𝑥, 𝑦) and 𝐼𝐵𝑟𝑒𝑓 (𝑥, 𝑦) represent the grayscale of R, G and B components in the captured color fringe pattern respectively. They can be separated from 𝐼𝐶𝑟𝑒𝑓 (𝑥, 𝑦) easily by a color 270

Y. Wang, Y. Cao, G. Fu et al.

Optics Communications 451 (2019) 268–275

separating technique. Just as discussed in the above process in Section 2.2, the corresponding three sinusoidal fringe patterns with an equivalent phase-shifting of 2𝜋∕3 which can be extracted from the separated R, G and B components is represented as: ⎧𝐼𝐹 𝑅𝑟𝑒𝑓 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓 𝑥 + 𝜙𝑟 (𝑥, 𝑦)) ⎪ ⎨𝐼𝐹 𝐺𝑟𝑒𝑓 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓 𝑥 + 𝜙𝑟 (𝑥, 𝑦) + 2𝜋∕3) ⎪ ⎩𝐼𝐹 𝐵𝑟𝑒𝑓 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓 𝑥 + 𝜙𝑟 (𝑥, 𝑦) + 4𝜋∕3)

(10)

Where 𝑓 (𝑓 = 1∕𝑇 , 𝑇 denotes the pitch of the captured fringe pattern) represents the frequency of the captured fringe pattern, 𝜙𝑟 (𝑥, 𝑦) is the phase caused by the reference plane. It can be expressed as: √ 3[𝐼𝐹 𝑅𝑟𝑒𝑓 (𝑥, 𝑦) − 𝐼𝐹 𝐵𝑟𝑒𝑓 (𝑥, 𝑦)] 𝜙𝑟 (𝑥, 𝑦) = arctan{ } (11) 2𝐼𝐹 𝐺𝑟𝑒𝑓 (𝑥, 𝑦) − 𝐼𝐹 𝐺𝑟𝑒𝑓 (𝑥, 𝑦) − 𝐼𝐹 𝐵𝑟𝑒𝑓 (𝑥, 𝑦) Because the phase value of 𝜙𝑟 (𝑥, 𝑦) is wrapped in (−𝜋, 𝜋] due to the arctan calculation, it should be unwrapped to be a continuous phase 𝜙𝑢𝑤 𝑟 (𝑥, 𝑦) by a phase unwrapping algorithm [30]. And when measuring the objects with large or abrupt depth variations, temporal phase unwrapping will be used to remove the phase ambiguity [31]. While measuring, the same proposed color binary grating is projected onto the object and the corresponding deformed color fringe pattern is captured by the color CCD camera. It can be expressed as: ⃖⃖⃗ + 𝐼𝐺𝑑𝑒𝑓 (𝑥, 𝑦)𝐺 ⃖⃖⃗ + 𝐼𝐵𝑑𝑒𝑓 (𝑥, 𝑦)𝐵 ⃖⃖⃗ 𝐼𝐶𝑑𝑒𝑓 (𝑥, 𝑦) = 𝐼𝑅𝑑𝑒𝑓 (𝑥, 𝑦)𝑅

Fig. 6. The schematic diagram of the proposed method.

(12)

Where 𝐼𝑅𝑑𝑒𝑓 (𝑥, 𝑦), 𝐼𝐺𝑑𝑒𝑓 (𝑥, 𝑦) and 𝐼𝐵𝑑𝑒𝑓 (𝑥, 𝑦) represent the grayscale distributions of the R, G and B components in captured deformed color pattern respectively. They can be separated from 𝐼𝐶𝑑𝑒𝑓 (𝑥, 𝑦) easily. In the same way above, the corresponding extracted three nearly unbroken sinusoidal deformed patterns can be expressed as: ⎧𝐼 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓 𝑥 + 𝜙𝑑 (𝑥, 𝑦)) ⎪ 𝐹 𝑅𝑑𝑒𝑓 ⎨𝐼𝐹 𝐺𝑑𝑒𝑓 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓 𝑥 + 𝜙𝑑 (𝑥, 𝑦) + 2𝜋∕3) ⎪𝐼𝐹 𝐵𝑑𝑒𝑓 (𝑥, 𝑦) = 𝐴(𝑥, 𝑦) + 𝐵(𝑥, 𝑦) cos(2𝜋𝑓 𝑥 + 𝜙𝑑 (𝑥, 𝑦) + 4𝜋∕3) ⎩

(13)

Fig. 7. The experimental system of the proposed method.

Where 𝜙𝑑 (𝑥, 𝑦) is the wrapped phase caused by the measured object. It also can be expressed as: √ 3[𝐼𝐹 𝑅𝑑𝑒𝑓 (𝑥, 𝑦) − 𝐼𝐹 𝐵𝑑𝑒𝑓 (𝑥, 𝑦)] 𝜙𝑑 (𝑥, 𝑦) = arctan{ } (14) 2𝐼𝐹 𝐺𝑟𝑒𝑓 (𝑥, 𝑦) − 𝐼𝐹 𝐺𝑑𝑒𝑓 (𝑥, 𝑦) − 𝐼𝐹 𝐵𝑑𝑒𝑓 (𝑥, 𝑦)

3. Results and discussion The experimental system of the proposed method is set up as shown in Fig. 7. It mainly consists of a DLP (Light Crafter 4500 with a resolution of 912 pixel×1140 pixel) and a color CCD camera (DFK 23U274 with a resolution of 800 pixel×600 pixel at 26 fps). In the experiment, the R, G and B components of encoded color binary grating share the same period with 21 pixels and the same width of nonzero transmittance with 5 pixels, it means that the duty cycle is 5/21 less than 1/3. In this condition, the interval is 2pixels. The 3D shape information of measured object is processed in the computer with the configuration of Intel(R) core(TM) i5-4590 CPU @ 3.30 GHz and 1.86 GHz 8 GB physical memory.

It should also be unwrapped to be a continuous phase 𝜙𝑢𝑤 (𝑥, 𝑦). The 𝑑 phase 𝜑ℎ (𝑥, 𝑦) caused by the height of measured object can be obtained by the difference between 𝜙𝑢𝑤 (𝑥, 𝑦) and 𝜙𝑢𝑤 𝑟 (𝑥, 𝑦): 𝑑 𝑢𝑤 𝜑ℎ (𝑥, 𝑦) = 𝜙𝑢𝑤 𝑑 (𝑥, 𝑦) − 𝜙𝑟 (𝑥, 𝑦)

(15)

So the 3D shape of the measured object is reconstructed by the phase-to-height mapping relationship [32]: 1 1 1 = 𝑎(𝑥, 𝑦) + 𝑏(𝑥, 𝑦) + 𝑐(𝑥, 𝑦) ℎ(𝑥, 𝑦) 𝜑ℎ (𝑥, 𝑦) 𝜑2 (𝑥, 𝑦)

(16)

ℎ

Where the system constants 𝑎(𝑥, 𝑦), 𝑏(𝑥, 𝑦) and 𝑐(𝑥, 𝑦) can be calibrated by several planes with known heights. The schematic diagram of the proposed method is shown in Fig. 6. When the proposed color binary fringe is projected onto the measured object by the DLP, only one color deformed pattern is needed to be captured by the color CCD camera. Then three deformed patterns can be separated simply from the R, G and B components of the captured color deformed pattern. The three separated deformed patterns are processed respectively according to the process mentioned above. The corresponding three nearly unbroken sinusoidal deformed patterns with an equivalent phase-shifting can be extracted effectively. They are used to reconstruct the 3D shape of the measured object combined three-step phase-shifting algorithm. Due to the single-shot feature of the encoded color binary grating, the proposed method can be applied into dynamic 3D shape measurement.

3.1. The 3D shape reconstruction experiment In order to verify the feasibility of the proposed method, several measuring static objects experiments are conducted, one of them is to measuring a face model as shown in Fig. 8(a). The captured color deformed pattern is shown in Fig. 8(b) and its R, G and B components are separated as shown in Fig. 8(c)–(e). The corresponding three extracted nearly unbroken sinusoidal deformed patterns are shown in Fig. 8(f)–(h). Fig. 9(a) and (b) show the corresponding wrapped phase and the reconstructed 3D shape of the measured face model respectively. It can be seen that the 3D shape surface and most details of the measured face model can be well reconstructed. So the experiment result effectively verifies that the proposed method is feasible for 3D shape measurement. 271

Y. Wang, Y. Cao, G. Fu et al.

Optics Communications 451 (2019) 268–275

Fig. 8. The reconstruction process of measuring face model: (a) The measured face model, (b) Captured color deformed pattern, (d)-(e) Separated R, G and B components, (f)-(h) Corresponding extracted sinusoidal deformed patterns.

Fig. 9. The reconstruction result of the measured face model: (a) The wrapped phase distribution, (b) The reconstructed 3D shape.

Fig. 11. Cutaway views of the reconstruction result in 237th column using the three methods. Table 1 Measuring results for several planes at given known heights (mm).

3.2. Accuracy analysis In order to analyze the measuring accuracy of the proposed method, a series of planes with known heights are measured. Table 1 shows the measuring results of six planes with known heights in 2.880 mm, 5.500 mm, 10.000 mm, 12.500 mm, 18.000 mm, 25.500 mm and 28.000 mm: Where h is the known height of the tested plane and ℎ𝑎𝑣𝑔 represents the mean height in measurement. The MAE denotes the mean absolute error and RMS denotes the root of mean square error. It can be seen that the MAE and RMS are smaller than 0.076 mm and 0.049 mm respectively. So the proposed method can achieve higher measuring accuracy.

h

2.880

5.500

10.000

12.500

18.000

25.500

28.000

h𝑎𝑣𝑔 MAE RMS

2.852 0.075 0.045

5.485 0.073 0.042

9.975 0.076 0.048

12.489 0.063 0.040

18.014 0.060 0.043

25.483 0.065 0.049

27.986 0.068 0.041

Furthermore, several compared experiments are conducted between Li’s method [25] (Li) and the proposed method (Prop) by measuring the planes with known heights in 5.500 mm, 12.500 mm and 18.000 mm as shown in Table 2. It shows that the MAE and RMS of the proposed method are smaller than those of Li. The measuring average heights of

Fig. 10. Compared reconstruction results: (a) The heart model measured object (b) Captured color deformed pattern with Li, (c) Reconstruction results with Li, (d) Partial enlarges of (c), (e) Captured color deformed pattern with Prop, (f) Reconstruction results with Prop, (g) Partial enlarges of (f).

272

Y. Wang, Y. Cao, G. Fu et al.

Optics Communications 451 (2019) 268–275

Fig. 12. Compared reconstruction results: (a) The Mickey Mouse’s head model (b) Captured color deformed pattern with Prop𝑓 , (c) Captured binary deformed pattern with Fu𝑓 , (d) Captured binary deformed pattern with FTP3𝑓 , (e) Reconstruction results with Prop𝑓 , (f) Reconstruction results with Fu𝑓 , (g) Reconstruction results with FTP3𝑓 .

Fig. 13. Cutaway views of the reconstruction result using different methods: (a) Cutaway views of the reconstruction result in 150th raw, (b) The partial enlargement of the dotted box in Fig. 13(a).

Fig. 14. Reconstruction results of real-time swing heart model: (a) The measured hand, (b)–(d) Captured color deformed patterns in three states (they are come from Video 1), (e)–(g) Corresponding 3D shape reconstruction results in three states (they are come from Video 2).

the proposed method approach their true heights more accurately than those of Li. So the proposed method has higher measuring accuracy.

a heart model as shown in Fig. 10(a). Fig. 10(b) shows the captured color deformed pattern with Li’s method. The corresponding 3D shape reconstruction result of the measured heart model with Li’s method is shown in Fig. 10(c) and the region of dotted box in Fig. 10(c) is enlarged, as shown in Fig. 10(d). Fig. 10(e) shows the captured color deformed pattern with the proposed method. The corresponding 3D shape reconstruction result of the measured heart model with the

3.2.1. Comparison with Li’s method In order to further verify the validity and practicality of the proposed method, several compared experiments are conducted between Li’s method and the proposed method. One of the measured objects is 273

Y. Wang, Y. Cao, G. Fu et al.

Optics Communications 451 (2019) 268–275

Table 2 Comparisons with methods for reconstruction of known height planes (mm). h

5.500

Method h𝑎𝑣𝑔 MAE RMS

Li 5.478 0.085 0.057

12.500 Prop 5.485 0.073 0.042

Li 12.523 0.078 0.049

be captured at 26fps by the color CCD camera. The corresponding 3D shapes of the changing hand are reconstructed frame by frame. Fig. 14(b)–(d) shows three color deformed patterns of the dynamic changing measured hand at three different states. Their corresponding 3D shape reconstruction results are shown in Fig. 14(e)–(g). It can be seen that the 3D shapes of dynamic changing hand can be well reconstructed. So the proposed method is effective for measuring the 3D shapes of the measured dynamic changing objects at 26 fps.

18.000 Prop 12.489 0.063 0.040

Li 17.975 0.070 0.051

Prop 18.014 0.060 0.043

proposed method is shown in Fig. 10(f) and the region of dotted box in Fig. 10(f) is enlarged, as shown in Fig. 10(g). Comparing Fig. 10(c) with Fig. 10(f), we can see that the proposed method has a better reconstruction result. It is well known that the PMP with larger number-step phaseshifting algorithm may show much more accurate by the constraint of multiple frames deformed patterns [25], but the use of more fringe patterns does not necessarily enhance measurement quality [23]. Here the measuring result of PMP with 16-step phase-shifting algorithm (PMP-16) is regarded as the reference, it is much more approximate the true value. As can be seen from Fig. 11, some ripples appear in the result of Li’s method and a little bit more deviates from the result of PMP-16 but nearly no ripple appears in the result of the proposed method and the result of the proposed method much more matches with the result of PMP-16. We think the main reason should be attributed to the color crosstalk inhibiting ability of the proposed method.

4. Conclusions A single-shot phase measuring profilometry based on color binary grating with intervals is proposed. In the proposed method, only one color binary grating with intervals is needed to be encoded, it can effectively avoid the color crosstalk problem and improve chromatic aberration problem. When it is projected onto the measured object, the corresponding 3D shape of the measured object can be reconstructed from the single-shot color deformed pattern. Experimental results verify the feasibility and the validity of the proposed method. Due to the single fringe projection feature and the single-shot feature of the encoded color binary fringe, the proposed method can be used to realize dynamic 3D shape measurement. Acknowledgments This work was supported by the 863 National Plan Foundation of China under Grant No. 2007AA01Z333 and Special Grand National Project of China under Grant No. 2009ZX02204-008.

3.2.2. Comparison with Fu’s method and FTP Several compared experiments are also conducted among the proposed method, Fu’s method and FTP. For the proposed method and Fu’s method, the period of encoded color binary grating is 21 pixels and suppose its frequency is f. For the FTP method, the period of corresponding black and white binary pattern is 7 pixels, thus, the frequency is three times as that of the proposed method and Fu’s method (3f ). One of the measured objects is a model of Mickey Mouse’s head as shown in Fig. 12(a). Fig. 12(b), (c) and (d) are the captured deformed patterns with proposed method (Prop𝑓 ), Fu’s method (Fu𝑓 ) and FTP method (FTP3𝑓 ) respectively. Fig. 12(e), (f) and (g) are the reconstruction results of Prop𝑓 , Fu𝑓 and FTP3𝑓 . It can be seen that the shape of the measured object can be reconstructed by all the three methods. The cutaway views of the reconstruction results in 150th row are shown in Fig. 13(a). The measuring result of PMP with 16-step phaseshifting algorithm (PMP-16) is also regarded as the reference, it is much more approximate the true value. Fig. 13(b) is the partial enlargement of the dotted box in Fig. 13(a). It can be seen from Fig. 13 that the cutaway views of the proposed method and Fu’s method agree well with that of PMP-16, while the cutaway view of FTP3𝑓 has a quite large deviation from that of PMP-16 in some details of the measured object. Thus, it can be concluded that the accuracy of the proposed method is as good as that of Fu’s method and better than that of FTP3𝑓 . But Fu’s method has high requirements for both the projection and the camera. It needs high frame rate projection and the camera. The results of comparing experiments with Fu’s method and FTP3𝑓 can further verify the validity and practicality of the proposed method.

Appendix A. Supplementary data Supplementary material related to this article can be found online at https://doi.org/10.1016/j.optcom.2019.06.062. References [1] H. Lee, Y.K. Min, J.I. Moon, Three-dimensional sensing methodology combining stereo vision and phase-measuring profilometry based on dynamic programming, Opt. Eng. 56 (12) (2017) 124107. [2] M.A. Vázquez, F.J. Cuevas, A 3D Facial Recognition System Using Structured Light Projection, in: International Conference on HAIS, 2014, pp. 241-253. [3] J. Xu, N. Xi, C. Zhang, et al., Rapid 3D surface profile measurement of industrial parts using two-level structured light patterns, Opt. Lasers Eng. 49 (7) (2011) 907–914. [4] C.F. Jiang, B. Lim, S. Zhang, Three-dimensional shape measurement using a structured light system with dual projectors, Appl. Opt. 57 (14) (2018) 3983–3990. [5] M. Takeda, K. Mutoh, Fourier transform profilometry for the automatic measurement of 3-D object shapes, Appl. Opt. 22 (24) (1983) 3977–3982. [6] V. Srinivasan, H.C. Liu, M. Halioua, Automated phase-measuring profilometry of 3-d diffuse objects, Appl. Opt. 23 (18) (1984) 3105–3108. [7] X.Y. Su, W.J. Chen, Q.C. Zhang, Y.P. Cao, Dynamic 3-d shape measurement method based on FTP, Opt. Lasers Eng. 36 (2001) 49–64. [8] Y.Y. Wan, Y.P. Cao, C. Chen, et al., Single-shot real-time three dimensional measurement based on hue-height mapping, Opt. Commun. 416 (2018) 10–18. [9] L. Zhu, Y.P. Cao, D.W. He, et al., Real-time tricolor phase measuring profiometry based on CCD sensitivity calibration, J. Mod. Opt. 376 (2016) 72–80. [10] C. Zuo, S.J. Feng, L. Huang, et al., Phase shifting algorithms for fringe projection profilometry: A review, Opt. Lasers Eng. 109 (2018) 23–59. [11] P.S. Huang, Q. Hu, F. Jin, F.P. Chiang, Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring, Opt. Eng. 38 (6) (1999) 1065–1071. [12] Z.H. Zhang, C.E. Towers, D.P. Towers, Robost color and shape measurement of full color artifacts by RGB fringe projection, Opt. Eng. 51 (2) (2012) 021109. [13] P.S. Huang, C.P. Zhang, F.P. Chiang, High-speed 3-D shape measurement based on digital fringe projection, Opt. Eng. 42 (1) (2003) 163–168. [14] J.H. Pan, P.S. Huang, F.P. Chiang, Color-encoded digital fringe projection technique for high-speed 3D shape measurement: color coupling and imbalance compensation, Proc. SPIE-Int. Soc. Opt. Eng. 5265 (2004) 43–51. [15] Z.H. Zhang, C.E. Towers, D.P. Towers, Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection, Opt. Express 14 (14) (2006) 6444–6455.

3.3. Dynamic 3D shape reconstruction experiments In order to verify the effectiveness of the proposed method for realizing dynamic 3D shape measurement, several corresponding experiments for measuring the 3D shapes of dynamic objects are conducted. One of them is to measuring a dynamic changing hand, its experimental result is shown in Fig. 14. When the proposed color binary grating is projected onto the dynamic changing hand as shown in Fig. 14(a) steadily by the DLP, the corresponding deformed patterns contained the 3D shape information of the changing hand at different states can 274

Y. Wang, Y. Cao, G. Fu et al.

Optics Communications 451 (2019) 268–275 [25] B. Li, Y. Fu, Z. Wang, J. Zhang, High-speed, high-accuracy 3D shape measurement based on binary color fringe defocused projection, J. Eur. Opt. Soc.-Rapid 10 (2015) 15308. [26] C. Zuo, Q. Chen, S.J. Feng, et al., Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing, Appl. Opt. 51 (19) (2010) 4477–4490. [27] C. Zuo, Q. Chen, G.H. Gu, et al., High-speed three-dimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection, Opt. Lasers Eng. 51 (8) (2013) 953–960. [28] Fu Guangkai, Cao Yiping, Wang Yapin, et al., Dynamic phase measuring profilometry based on tricolor binary fringe encoding combined time-division multiplexing, Appl. Sci. 9 (4) (2019) 813. [29] S. Zhang, High-speed 3D shape measurement with structured light methods: A review, Opt. Lasers Eng. 106 (2018) 119–131. [30] K. Chen, J.T. Xi, Y.G. Yu, Quality-guided spatial phase unwrapping algorithm for fast three-dimensional measurement, Opt. Commun. 294 (2013) 139–147. [31] C. Zuo, L. Huang, M.L. Zhang, et al., Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review, Opt. Lasers Eng. 85 (2016) 84–103. [32] X.Y. Su, W.Z. Song, Y.P. Cao, L.Q. Xiang, Both phase height mapping and coordinates calibration in PMP, Proc. SPIE-Int. Soc. Opt. Eng. 4829 (2002) 874–875.

[16] Y.P. Cao, X.Y. Su, RGB Tricolor based fast phase measuring profilometry, Proc. SPIE. 4919 (1) (2002) 528–535. [17] J.H. Pan, P.S. Huang, F.P. Chiang, Color phase-shifting technique for three-dimensional shape measurement, Opt. Eng. 45 (1) (2006) 013602. [18] L. Zhu, Y.P. Cao, D. Wu, C. Chen, Grayscale imbalance correction in real-time phase measuring profiometry, Opt. Commun. 376 (2016) 72–80. [19] Y.S. Hu, J.T. Xi, J. Chicharo, Z.K. Yang, Blind color isolation for color-channelbased fringe pattern profiometry using digital projection, J. Opt. Soc. Am. A 24 (8) (2007) 2372–2382. [20] J.L. Flores, J.A. Ferrari, G.G. Torales, R.L. Saenz, A. Silva, Color-fringe pattern profilometry using a generalized phase-shifting algorithm, Appl. Opt. 54 (30) (2015) 8827–8834. [21] S. Zhang, D.V.D. Weide, J. Oliver, Superfast phase-shifting method for 3-D shape measurement, Opt. Express 18 (2010) 9684. [22] Z.H. Zhang, C.E. Towers, D.P. Towers, Compensating lateral chromatic aberration of a colour fringe projection system for shape metrology, Opt. Lasers Eng. 48 (2010) 159–165. [23] Z.H. Zhang, Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques, Opt. Lasers Eng. 50 (2012) 1097–1106. [24] L. Ekstrand, S. Zhang, Three-dimensional profilometry with nearly focused binary phase-shifting algorithms, Opt. Lett. 36 (23) (2011) 4518–4520.

275