- Email: [email protected]
Separation method of superimposed gratings in double-projector structured-light vision 3D measurement system
Optics Communications 456 (2020) 124676
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Separation method of superimposed gratings in double-projector structured-light vision 3D measurement system Li Yiming, Qu Xinghua, Zhang Fumin 1 , ∗, Zhang Yuanjun State Key Laboratory of Precision Measuring Technology and Instruments, Tianjin University, 92 Weijin Road, Tianjin, 300072, China
ARTICLE
INFO
Keywords: Structured-light vision 3D measurement Double-projector system Phase-shifted superimposed gratings Separation method
ABSTRACT In structured-light vision (SLV) 3D measurement technology, double-projector SLV is often used to solve the defects in single projector SLV, such as the shadow caused by the self-occlusion of the measured object’s surface profile and the limitation of the projector’s visual angle. Usually, to avoid gratings’ superimposition, multiple projectors cannot work at the same time. In this paper, we design a new separation method to separate phase-shifted superimposed gratings by controlling the sequence in double-projector SLV. Without changing the original measurement system, the four-step phase-shifted method and dual-frequency method are combined to solve the unwrapped phase. Based on the proposed separation method, the wrapped phases of left and right projections are obtained only by six phase-shifted gratings, whose efficiency is improved by 25%. The separation method includes the process of averaging phase patterns, which improves the phase accuracy of the measured object, comparing with the single projector measurement. Experiments demonstrate that this method can separate phase-shifted superimposed gratings simply, quickly, effectively and accurately without additional hardware devices. This method is practical and general.
1. Introduction SLV 3D measurement has a series of advantages, such as non-contact characteristics, non-destructive features, fast measurement, high precision and high automation [1–4]. It has become one of the most rapidly developed, widely used and potential methods in 3D data acquisition in recent years. It is the key technology of modern scientific research and engineering application [5,6]. However, nowadays, the mature 3D measurement technology is still limited to the single projector structure. There are still some key issues that need to be addressed, such as how to solve the shadow phenomenon caused by the sophisticated surface profile of measured objects, how to relief the limitation of the single projector’s visual angle, and how to obtain the complete 3D information of measured objects quickly and effectively. Shadow is a common phenomenon in SLV measurements. If an area on the surface of an object cannot be illuminated by projected light or whose reflected light cannot be observed by the camera, the area becomes an unmeasurable shadow area. The description of the shadow part in SLV 3D measurement is shown in Fig. 1. For some complicated measured surfaces, data loss is often caused by the raised area obscuring the camera’s line of sight and projecting light. The visual angle of the object is limited in a single projector system, so it is necessary to use multiple projectors to obtain the complete 3D information. Because of the fixed position
and non-contact feature of most measured objects, multi-angle measurement technology has been widely used. However, SLV technology is not easy to use in the aforementioned situations, because if multiple projectors work simultaneously, the gratings would be overlapped. So, the effective phase information cannot be recovered, affecting the 3D measurement results. To avoid the disadvantages of superimposed gratings in measurement, two projectors do not work simultaneously. Comparing with two single-projector systems, the measurement time has not been reduced. Therefore, the separation problem of superimposed gratings limits the development of multi-projector measurement technology. Nowadays multi-projector SLV measurement technology is still in its infancy [7]. To solve the shadow problem, Roland LPX250 laser scanning and measuring machine made in Japan used hardware processing and calculation to complete the measurement of rotary and shifting scanning. To solve the occlusion problem, this measuring method needed to rotate additional mechanical devices from multiple angles, which had a complex structure, high cost and long time. Literature [8] proposed the method of shadow profile to acquire the surface of an object and then analyzed the grayscale distribution of the object image in different lighting conditions. However, the 3D surface information of the sunken areas is difficult to obtain by the method of shadow profile. Literature [9] put forward that, in stereo vision measurement, 3D profile
∗ Correspondence to: 92 Weijin Road, Tianjin University, Tianjin, 300072, China. E-mail address: [email protected] (F. Zhang). 1 Zhang Fumin is willing to handle correspondence at all stages of refereeing and publication, also post-publication.
https://doi.org/10.1016/j.optcom.2019.124676 Received 3 August 2019; Received in revised form 25 September 2019; Accepted 30 September 2019 Available online 3 October 2019 0030-4018/© 2019 Elsevier B.V. All rights reserved.
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
Fig. 2. Geometric model of optical measurement system. Description: Fig. 2 is a classical geometric model for 3D measurement of the grating projection.
Fig. 1. Shadow schematic. Description: Fig. 1 shows the description of the shadow part in SLV 3D measurement. If an area on the surface of an object cannot be illuminated by projected light or whose reflected light cannot be observed by the camera, the area becomes an unmeasurable shadow area.
this paper. Section 4 introduces the experiments of double-projector 3D measurement based on the six-graph separation algorithm and analyzes the experimental results. Section 5 summarizes the paper and prospects the future research directions.
data of objects were acquired by the method of shadow profile, and then surface details such as dents, edges and corners were measured by stereo vision matching technology. The two methods have achieved certain results in 3D measurement of complex profile objects, but the algorithm and the experiment are sophisticated. As for the separation of superimposed signals, Maimone [10] et al. used several motion sensors to make the signals from different coding modes blurred until they could be removed. Griesseretal [11] et al. shielded the structured light signal to establish a system that could eliminate the relative signal of the projection direction. Both the algorithms and structures of this method were complex. This method not only need high hardware performance but also greatly increased the complexity of the measurement system. Changsoo Je [12] et al. proposed a method to solve the partial derivation of left projection in the vertical direction of right projection, and adopted the mode of color fringe displacement to eliminate the interference from the chromatic region through analysis, to achieve the separation of signals. However, this method had high requirements for illumination. Yan et al. [13] achieved signal separation by designing multi-layer signals that do not interfere with each other. Nevertheless, this method relied on the design of the coding signal, whose encoding and decoding methods were complicated. In this research, we propose a separation method called the sixgraph separation method to separate the superimposed gratings in double-projector SLV systems. It designs the projection sequences of two projectors and deals with the superimposed gratings by arctangent function. In detail, it encodes the projection gratings in sinusoidal fringes by the four-step phase-shifted method. In the traditional doubleprojector system, it needs four projection gratings in a single projector to get the wrapped phases, which means that it needs eight gratings as for two projectors. Using the six-graph method, it only needs six gratings in the situation that two projectors work simultaneously, reducing the measurable time. What is more, there is a process of averaging the phase, reducing the noise interference and the system error. Then we can obtain the wrapped phases of both sides from the superimposed gratings. Then we combine with the dual-frequency method to get the unwrapped phases. Calibration and registration are used to reconstruct 3D geometry based on the wrapped phases. Experiments demonstrated the success of the proposed method. The rest of this paper is organized as follows. Section 2 describes related basic principles of the 3D measurement system. Section 3 introduces two principles of separating superimposed gratings proposed in
2. Basic principle 2.1. System calibration
The key process of SLV 3D measurement is to project digital phaseshifted gratings to the measured objects. The grating fringes are deformed after modulated by the height of the measured objects, then the camera collects the image of the deformed fringes to obtain (u, v, 𝜃) information about the object. Where (u, v) is the pixel coordinate, and 𝜃 is the phase value. Finally, it transfers phase information of the object based on the system calibration into the world coordinate values (𝑋𝑤 , 𝑌𝑤 , 𝑍𝑤 ) [14]. Fig. 2 is a classical geometric model for 3D measurement of the grating projection. The measured object is placed on a reference plane. P is the exit pupil center of the DLP projector and I is the entry pupil center of the camera. The distance between the two points is d, the distance from P and I to the reference plane is equal to L. The optical axis of the camera is perpendicular to the reference plane, and O is the projection point of I on the reference plane. The intersection of the optical axes of the projector and the camera is used as the origin to establish the coordinate system. Referring to the system model, we can obtain the relationship between height and phase in Eq. (1). ℎ=
𝛥𝜑 ⋅ 𝐿 2𝜋𝑓 𝑑 + 𝛥𝜑
(1)
𝛥𝜑 is the phase difference caused by the height modulation of the measured object’s surface, where f is the frequency of the sinusoidal fringe. In this paper, the camera calibration parameters are obtained by the Zhang Zhengyou calibration method [15,16] and the camera calibration toolbox of MATLAB. So we can get the pixel coordinates (𝜇, 𝜈) of the image, to solve the world coordinate (𝑋𝑤 , 𝑌𝑤 ). The phase-height conversion model refers to the equal coordinate phase method [17–19]. ( ) Finally, the complete coordinate 𝑋𝑤 , 𝑌𝑤 , 𝑍𝑤 of the measured object is obtained. 2
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
( ) grating order of the right projector is 𝑅1 (𝜑0 = −𝜋), 𝑅3 𝜑0 = 0 , 𝑅1 (𝜑0 = −𝜋). Finally, the superimposed gratings 𝑁1 = 𝐿1 + 𝑅1 , 𝑁2 = 𝐿1 + 𝑅3 , 𝑁3 = 𝐿3 + 𝑅1 are obtained.
2.2. Four-step phase-shift method Four-step phase-shifted algorithm is widely used in high precision and fast structured light 3D measurement technology [20–23]. The expressions of the four-step phase-shift method with initial phases −𝜋, 𝜋/2, 0 and 𝜋/2 are as follows. 𝐼1 = 𝑎 + 𝑏 cos(𝜃(𝑥, 𝑦) − 𝜋)
(2)
𝐼2 = 𝑎 + 𝑏 cos(𝜃(𝑥, 𝑦) − 𝜋∕2)
(3)
𝐼3 = 𝑎 + 𝑏 cos(𝜃(𝑥, 𝑦))
(4)
𝐼4 = 𝑎 + 𝑏 cos(𝜃(𝑥, 𝑦) + 𝜋∕2)
(5)
𝐿 + 𝑅1 + 𝐿1 + 𝑅3 𝑁1 + 𝑁2 = 1 = 𝐿1 + 𝑎 2 2 𝐿 + 𝑅1 + 𝐿3 + 𝑅1 𝑁1 + 𝑁3 = 1 = 𝑅1 + 𝑎 2 2
(12)
Based on the method of phase complementarity, Eqs. (11)–(12) indicate that the effective phase information of a single projection is obtained. Because the four-step phase-shifted method can eliminate the background light intensity, the separated signal can be directly used to solve the wrapped phase. According to the method of phase complementarity, the superimposed gratings can be effectively separated.
The phase distribution can be obtained by using the arctangent function: 𝐼 (𝑥, 𝑦) − 𝐼2 (𝑥, 𝑦) 𝜃(𝑥, 𝑦) = 𝑎𝑟𝑐𝑡𝑎𝑛 4 (6) 𝐼1 (𝑥, 𝑦) − 𝐼3 (𝑥, 𝑦)
3.2. Six-graph separation method As for the method of phase complementarity, it still needs 8 patterns to obtain the wrapped phase. Therefore, we propose a new faster method for separating superimposed gratings. The initial phases of 𝐿1 ∕𝑅1 , 𝐿2 ∕𝑅2 , 𝐿3 ∕𝑅3 , 𝐿4 ∕𝑅4 are −𝜋, −𝜋∕2, 0 and 𝜋∕2 respectively. The left and right projectors work simultaneously, and the left projector projected 6 images in the order of 𝐿1 , 𝐿1 , 𝐿2 , 𝐿3 , 𝐿4 , and 𝐿4 . The right projector projected 6 images in the order of 𝑅1 , 𝑅3 , 𝑅2 , 𝑅1 , 𝑅2 , and 𝑅4 . Eq. (13) represents six superimposed gratings images captured by the camera.
In the Eqs. (2)–(6), 𝐼 denotes the uneven reflectivity of the surface of the measured object. 𝑎 is the background intensity. 𝑏 denotes the contrast of grating fringes, and 𝜃(𝑥, 𝑦) is the wrapped phase value. 2.3. Dual-frequency method Referring to the literature [24], this paper uses the dual-frequency method to calculate the unwrapped phase of the measured object. In this paper, k is the ratio of frequency of the high-frequency grating to the low-frequency grating frequency. Where 𝜃𝐻 (𝑥, 𝑦), 𝜃𝐿 (𝑥, 𝑦), 𝜑𝐻 (𝑥, 𝑦) and 𝜑𝐿 (𝑥, 𝑦) are high-frequency wrapped phase, low-frequency wrapped phase, high-frequency unwrapped phase, and low-frequency unwrapped phase respectively. The unwrapping results of the highfrequency fringes are as follows: ] [ 𝑘 ∗ 𝜑𝐿 (𝑥, 𝑦) − 𝜃𝐻 (𝑥, 𝑦) (7) 𝑛𝑢𝑚 (𝑥, 𝑦) = 𝑖𝑛𝑡 2𝜋 𝜑𝐻 (𝑥, 𝑦) = 2𝜋 ∗ 𝑛𝑢𝑚(𝑥, 𝑦) + 𝜃𝐻 (𝑥, 𝑦)
(11)
𝐻1 = 𝐿1 + 𝑅1 , 𝐻2 = 𝐿1 + 𝑅3 , 𝐻3 = 𝐿2 + 𝑅2
𝐻4 = 𝐿3 + 𝑅1 , 𝐻5 = 𝐿4 + 𝑅2 , 𝐻6 = 𝐿4 + 𝑅4
(13)
Referring to Eq. (5):
(8)
where num is a level value and int is an integer operation. For the left projection separated from the superimposed grating, we solved the wrapped phase 𝜃𝐻 (𝑥, 𝑦) and 𝜃𝐿 (𝑥, 𝑦) of high frequency and low frequency respectively. 𝜃𝐿 (𝑥, 𝑦), whose frequency is 1, is the same as the unwrapped phase 𝜑𝐿 (𝑥, 𝑦). The low-frequency grating acts as a ruler, and 𝜑𝐿 (𝑥, 𝑦) is used to obtain the unwrapped high-frequency phase 𝜑𝐻 (𝑥, 𝑦).
𝜃1 = 𝑡𝑎𝑛−1
𝐻5 − 𝐻3 𝐿 + 𝑅2 − (𝐿2 + 𝑅2 ) 𝐿 − 𝐿2 = 𝑡𝑎𝑛−1 4 = 𝑡𝑎𝑛−1 4 𝐻1 − 𝐻4 𝐿1 + 𝑅1 − (𝐿3 + 𝑅1 ) 𝐿1 − 𝐿3
𝜃2 = 𝑡𝑎𝑛−1
𝐻6 − 𝐻5 𝑅 − 𝑅2 𝐿 + 𝑅4 − (𝐿4 + 𝑅2 ) = 𝑡𝑎𝑛−1 4 = 𝑡𝑎𝑛−1 4 𝐻1 − 𝐻2 𝐿1 + 𝑅1 − (𝐿1 + 𝑅3 ) 𝑅1 − 𝑅3
(14)
In the Eq. (14), 𝜃1 and 𝜃2 are the wrapped phase value of left projection and right projection respectively, which proves that the superimposed gratings based on the six-graph separation method can be effectively separated. Assume that 𝜃1 and 𝜃2 are the separated low-frequency wrapped phases of the left and right projectors respectively. 𝜃3 and 𝜃4 are the separated high-frequency wrapped phases of the left and right projectors respectively. 𝜃1 and 𝜃3 are processed by the dual-frequency method to obtain high-frequency unwrapped phase 𝜑𝑙𝑒𝑓 𝑡 of the left projector. 𝜃2 and 𝜃4 are processed by dual-frequency method to obtain high-frequency unwrapped phase 𝜑𝑟𝑖𝑔ℎ𝑡 of right projector. For the dual projector single camera system, considering the superposition of light patterns, the two projectors cannot work at the same time. It takes time and workload of 8 images to solve the low-frequency wrapped phase as for the left and right projectors, and the same is true for the highfrequency wrapped phase. To solve the unwrapped phase of two sides, the left and right projectors need to project 16 images successively. In this way, the experimental process is long and tedious. Based on the separation algorithm, 6 images are projected in parallel when two projectors worked at the same time to solve the low-frequency wrapped phase as for the left and right projectors, and the same is true for the high-frequency wrapped phase. For the unwrapped phase, the dual-frequency method requires 12 sets of parallel images for simultaneous projection. Therefore, this separation method improves the measurement speed and reduces the measurement workload.
3. Separation method of superimposed gratings 3.1. Phase complementarity method Referring to the properties of sinusoidal phase information, we add Eqs. (3) and (5) to average the background light intensity of projection gratings. Similarly, using Eqs. (2) and (4), the background light intensity of projection gratings is also obtained. It can separate the phase information of the superimposed gratings. 𝑎 + 𝑏 cos(2𝜋𝑓 𝑥 + 𝜋2 ) + 𝑎 + 𝑏 cos(2𝜋𝑓 𝑥 − 𝜋2 ) 𝐼4 (𝑥, 𝑦) + 𝐼2 (𝑥, 𝑦) = = 𝑎 (9) 2 2 𝐼3 (𝑥, 𝑦) + 𝐼1 (𝑥, 𝑦) 𝑎 + 𝑏 cos(2𝜋𝑓 𝑥) + 𝑎 + 𝑏 cos(2𝜋𝑓 𝑥 − 𝜋) = =𝑎 (10) 2 2 The theoretical results are validated by taking 𝐼1 and 𝐼3 as examples. The initial phase is equal to −𝜋, −𝜋/2, 0 and 𝜋/2. Eqs. (9)–(10) indicate that the background light intensity of the phase-shifted gratings can be obtained from the superimposed gratings by the method of phase complementarity. The left and right projectors simultaneously project to the plane, where 𝜑0 represents phase. ( )the initial ( ) The grating order of the left projector is 𝐿1 𝜑0 = −𝜋 , 𝐿1 𝜑0 = −𝜋 , 𝐿3 (𝜑0 = 0). At the same time, the 3
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
4. Experiments and analysis In this paper, we have designed a double-projector measurement system based on the separation method of superimposed gratings to measure various real objects, like Agrippa face and standard sphere. Fig. 3 is the experimental device diagram. Two DLP Light Crafter 4500 1140*912 projectors and a Point Gray GS3-U3-41C6M-C 2048*2048 CMOS camera were used. Left and right projectors projected sinusoidal gratings with different phases and different sequences to the measured object, and the camera collected images with phase information. The 9*10 checkerboard was used for calibrating the camera. The length of the small checkerboard is 20 mm, and an accuracy of ±5 μm. Fig. 4 shows the overall structure design of the double-projector 3D measurement experiment. Firstly, we accomplished the system calibration. Secondly, based on the contents of 3.2, the double projectors worked simultaneously, and the camera collected superimposed gratings with phase information. Thirdly, we used the six-graph separation algorithm to separate the superimposed gratings and obtained the wrapped phase information of the left and right projections separately. Fourthly, unwrapped phases were obtained by the dual-frequency method, which was mentioned in Section 2.3. Fifthly, according to the principle of 2.1, the 3D point clouds from two perspectives of the measured object were obtained. Finally, a 3D point cloud with a complete surface profile of the object was obtained based on registration [25]. We have designed four sets of experiments to verify the three advantages of the separation algorithm proposed in this paper: great efficiency, speed, and accuracy. Experiment 1, based on the phase complementarity method, verified the feasibility of this method on separating superimposed gratings. Experiment 2, according to the six-graph separation method, proved that this algorithm can separate superimposed gratings effectively. The third experiment was the 3D measurement experiment of the Agrippa face, verifying the great efficiency, speed of the algorithm. The fourth experiment was a 3D measurement experiment
Fig. 3. Double-projector measurement system device. Description: Fig. 3 is the experimental device diagram.
of the standard sphere, which verified that the separation algorithm has higher accuracy than single projection. Finally, in experiment 1 and 2, the horizontal fringes were used in the left projector, and the vertical fringes were used in the right projector. In experiment 3 and 4, the vertical fringes were used in both the left and right projectors. 4.1. Experiment of phase complementarity method Referring to Section 3.1, we designed the experiment. The theoretical results were validated by taking 𝐼1 and 𝐼3 for example. Fig. 5 is superimposed grating images captured by the camera. (a) is the
Fig. 4. The overall structure double-projector 3D measurement. Description: Fig. 4 shows the overall structure design of the double-projector 3D measurement experiment.
Fig. 5. (a) 𝑁1 (b) 𝑁2 (c) 𝑁3 . Description: Fig. 5 is superimposed grating images captured by the camera. (a) is the superposition signal 𝑁1 , where the phases of the left projection grating and the right projection grating are both −𝜋. (b) is the superposition signal 𝑁2 , where the phases of left and right projections are −𝜋 and 0, respectively. (c) is the superposition signal 𝑁3 , where the phases of left and right projections are 0 and −𝜋, respectively;. 4
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
and obtain the unwrapped phase value faster. Moreover, there is a process of averaging projection patterns in the separation method, which reduces noise and system error. 4.3. 3D measurement of the Agrippa face To verify the completeness and validity of the double-projector 3D measurement system based on the six-graph separation algorithm, Agrippa face was selected as the measured object as shown in Fig. 9. The point cloud data of the measured object was simplified and processed by related algorithms. Besides, we combined the point clouds of left and right projection separated from the superimposed gratings to synthesize complete Agrippa face. Fig. 9 is the result of 3D measurement of the Agrippa face. (a) is the original image of an Agrippa face projected simultaneously by two projectors. (b) is the unwrapped phase image of the Agrippa face from the left projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (c) is the unwrapped phase image of the Agrippa face from the right projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (d) is the 3D measurement point cloud of the Agrippa face from the left projector. (e) is the result of the triangular meshing of point cloud in (d). (f) is the 3D measurement point cloud of the Agrippa face from the right projector. (g) is the result of the triangular meshing of point cloud in (f). (h) is the complete point cloud of Agrippa face after registration of point cloud in images (d) and (f). (i) is the complete point cloud (upward view) after registration. Fig. 9 shows that the 3D point clouds obtained by superimposed gratings are neat and the details are clear, which further proves that the six-graph separation algorithm can effectively separate the superimposed gratings. From (d) and (e), we can see that point clouds on the right side of the nose and right face are missing. From (f) and (g), it can be seen that the point clouds on the left side of the nose and left face are missing, too. (h) (i) are obtained after registration of point clouds on the left and right sides. The measurement incompleteness caused by the shadow on the nose and face of the Agrippa statue is effectively solved. The experimental results demonstrate that the six-graph separation method can effectively separate the superimposed gratings so that the double-projector system can solve the shadow problem caused by the complicated surface of the measured object when both left and right projectors are projected simultaneously. It expands the measurement range and obtain the complete phase information of the measured object through registration.
Fig. 6. Separation results of superimposed gratings based on phase complementarity principle. (a) the separated right projection; (b) the separated left projection; Description: Fig. 6 shows the separation results of superimposed gratings 𝑁1 , 𝑁2 and 𝑁3 in Fig. 5 based on Eqs. (9)–(12). (a) is the separation result of right projector from superimposed grating 𝑁1 and 𝑁2 . (b) is the separation result of left projector from superimposed gratings 𝑁1 and 𝑁3 .
superimposed signal 𝑁1 , where the phases of the left projection grating and the right projection grating are −𝜋. (b) is the superimposed signal 𝑁2 , where the phases of left and right projections are −𝜋 and 0, respectively. (c) is the superimposed signal 𝑁3 , where the phases of left and right projections are 0 and −𝜋, respectively. Fig. 6 shows the separation results of superimposed gratings 𝑁1 , 𝑁2 and 𝑁3 . (a) is the separation result of the right projector from superimposed gratings 𝑁1 and 𝑁2 . (b) is the separation result of the left projector from superimposed gratings 𝑁1 and 𝑁3 . The experimental results demonstrate that the separated gratings obtained by the phase complementary method has one more background intensity than that obtained by the single projector method. The background light intensity can be eliminated by the four-step phase-shifted method. So, this method is effective for the separation of superimposed gratings. 4.2. Experiment of six-graph separation method Fig. 7 shows 12 superimposed gratings with different phase combinations, which are divided into two groups according to the frequency of gratings. Fig. (a–f) represent the high-frequency gratings, whose frequency is 6. Fig. (g–l) represent the low-frequency gratings, whose frequency is 1. Referring to Sections 2 and 3, we designed simple projection gratings and separation algorithms. For high-frequency superimposed gratings, the left and right projectors projected six phaseshifted gratings to the measured object simultaneously, and then the camera collected superimposed gratings 𝐻1 , 𝐻2 , 𝐻3 , 𝐻4 , 𝐻5 , and 𝐻6 . (a) is the superimposed grating 𝐻1 , the combination of 𝐿1 and 𝑅1 . (b) is the superimposed grating 𝐻2 , the combination of 𝐿1 and 𝑅3 . (c) is the superimposed grating 𝐻3 , the combination of 𝐿2 and 𝑅2 . (d) is the superimposed grating 𝐻4 , the combination of 𝐿3 and 𝑅1 . (e) is the superimposed grating 𝐻5 , the combination of 𝐿4 and 𝑅2 . (f) is the superimposed grating 𝐻6 , the combination of 𝐿4 and 𝑅4 . For low-frequency superimposed gratings 𝑆1 , 𝑆2 , 𝑆3 , 𝑆4 , 𝑆5 and 𝑆6 , the phase distribution and combination sequence are consistent with those of high-frequency gratings. The superimposed gratings were separated by Eq. (14). Fig. 8 is the separation results of the superimposed gratings. (a) is the Agrippa face’s low-frequency wrapped phase of the right projector. (b) is the Agrippa face’s high-frequency wrapped phase of the right projector. (c) is the Agrippa face’s unwrapped phase of the right projector computed by the dual-frequency method. (d) is the Agrippa face’s low-frequency wrapped phase of the left projector. (e) is the Agrippa face’s highfrequency wrapped phase of the left projector. (f) is the Agrippa face’s unwrapped phase of the left projector computed by the dual-frequency method. From the separation results, we can see that the separation method proposed in this paper can separate the superimposed gratings effectively, eliminate the interference signal of the overlapped part,
4.4. 3D measurement experiment of the standard sphere To prove the influence of the separation algorithm on the accuracy, besides the double-projector experiment of the standard sphere, we added a group of experiments that the single projector projected alone. The diameter of the standard sphere measured by CMM is 40.056 mm. The diameter was measured by two methods, a single projector projecting alone and double projectors projecting simultaneously, ensuring the experimental conditions and the processing algorithm were consistent. The 3D point cloud data of the standard sphere were shown in the software NX Imageware. Fig. 10 shows the measurement results of the standard sphere diameter via different methods. (a) is the original image of a standard sphere projected simultaneously by two projectors. (b) is the unwrapped phase of the standard sphere from the left projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (c) is the unwrapped phase of the standard sphere from the right projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (d) is the 3D point cloud of the standard sphere from the left projector. (e) is the 3D 5
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
Fig. 7. High and low frequency superimposed gratings (a) 𝐻1 ; (b) 𝐻2 ; (c) 𝐻3 ; (d) 𝐻4 ; (e) 𝐻5 ; (f) 𝐻6 ; (g) 𝑆1 ; (h) 𝑆2 ; (i) 𝑆3 ; (j) 𝑆4 ; (k) 𝑆5 ; (l) 𝑆6 ; Description: Fig. 7 shows 12 superimposed gratings with different phase combinations. Fig. (a–f) represent the high-frequency gratings, whose frequency is 6. Fig. (g–l) represent the low-frequency gratings, whose frequency is 1. (a) is 𝐻1 , the superposition of 𝐿1 and 𝑅1 . (b) is 𝐻2 , the superposition of 𝐿1 and 𝑅3 . (c) is 𝐻3 , the superposition of 𝐿2 and 𝑅2 . (d) is 𝐻4 , the superposition of 𝐿3 and 𝑅1 . (e) is 𝐻5 , the superposition of 𝐿4 and 𝑅2 . (f) is 𝐻6 , the superposition of 𝐿4 and 𝑅4 . For low-frequency superimposed gratings, the phase distribution and combination sequence are consistent with those of high-frequency gratings. (g) is 𝑆1 . (h) is 𝑆2 . (i) is 𝑆3 . (j) is 𝑆4 . (k) is 𝑆5 . (l) is 𝑆6 .
Fig. 8. Separated results of superimposed gratings (a) low-frequency wrapped phase of right projector; (b) high-frequency wrapped phase of right projector; (c) unwrapped phase of right projector; (d) low-frequency wrapped phase of left projector; (e) high-frequency wrapped phase of left projector; (f) unwrapped phase of left projector; Description: Fig. 8 is the separation result of the superimposed gratings. (a) is the Agrippa face’s low-frequency wrapped phase of the right projector separated from the superimposed gratings. (b) is the Agrippa face’s high-frequency wrapped phase of the right projector separated from the superimposed gratings. (c) is the Agrippa face’s unwrapped phase of the right projector computed by the dual-frequency method. (d) is the Agrippa face’s low-frequency wrapped phase of the left projector separated from the superimposed gratings. (e) is the Agrippa face’s high-frequency wrapped phase of the left projector separated from the superimposed gratings. (f) is the Agrippa face’s unwrapped phase of the left projector computed by the dual-frequency method.
point cloud of the standard sphere from the right projector. (f) is the fitting result of the sphere based on (d). (g) is the fitting result of the sphere based on (e). The fitting results are shown in Table 1, including the diameter and the average error. D represents the condition projected by a single projector. DD represents the condition of simultaneous projection, which is based on the six-graph separation method. Table 1 shows the measurement results of the standard sphere via different measurement methods. For the left projector, the average
diameter of standard sphere is 39.976 mm and the mean error is 80 μm based on DD. The average diameter is 39.956 mm and the mean error was 100 μm in experiments projected based on D. For the right projector, the average diameter of the standard sphere is 39.966 mm and the mean error is 90 μm based on DD. The average diameter is 39.944 mm and the mean error is 112 μm based on D. The experimental results indicate that the 3D measurement results based on the six-graph separation method were closer to the true values than the results based on a single projector. In the separation algorithm, the gratings undergo 6
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
Fig. 9. The results of 3D measurement of Agrippa face (a) original map; (b) unwrapped phase of separated left projection; (c) unwrapped phase of separated right projection; (d) 3D point cloud based on (b); (e) triangular meshing of point cloud on (d); (f) 3D point cloud based on (c); (g) triangular meshing of point cloud based on (f); (h) complete point clouds of face profile; (i) complete point clouds of face profile (upward view); Description: Fig. 9 is the result of 3D measurement of Agrippa face. (a) is the original image of an Agrippa face projected simultaneously by two projectors; (b) is the unwrapped phase image of the Agrippa face from the left projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method; (c) is the unwrapped phase image of the Agrippa face from the right projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method; (d) is the 3D measurement point cloud of the Agrippa face from the left projector; (e) is the result of triangular meshing of point cloud in (d); (f) is the 3D measurement point cloud of the Agrippa face from the right projector; (g) is the result of triangular meshing of point cloud in (f); (h) is the complete point clouds of Agrippa face after registration of point clouds in images (d) and (f); (i) is the complete point clouds after registration (upward view). Table 1 Measurement results of standard sphere diameter under different measurement methods. Description: Table 1 shows the measurement results of standard sphere under different measurement methods, including the diameter and the average error. D represents the condition that projected by single projector. DD represents the condition of simultaneous projection, which is based on the six-graph separation method. Left projector
D DD
double-projector measurement SLV is a fast, accurate and effective 3D measurement method. 5. Conclusion and prospect Double-projector measurement technology can effectively solve the shadow phenomenon, where the efficient separation of superimposed gratings is the key question. We proposed a method to solve the aforementioned problem, called the six-graph separation method, by changing the sequence of gratings with different phase and calculating them. In this paper, the superimposed gratings are obtained by controlling the sequence of left and right projection with the doubleprojector system device. The separation algorithm is used to obtain the unwrapped phase of the measured object by combining the four-step phase-shifted method with the dual-frequency method. The phaseheight conversion model based on the equal-phase coordinate method is used to measure the object. Finally, three-dimensional measurement is completed. The practicability and generalization of this separation algorithm are verified by experiments. In conclusion, the practical significances in detail are as follows:
Right projector
Diameter (mm)
Mean error (mm)
Diameter (mm)
Mean error (mm)
39.956 39.976
0.100 0.080
39.944 39.966
0.112 0.090
the process of superposition and separation, whose phase information is averaged, effectively reducing the noise and random error. Thus, the measurement accuracy of superimposed parts is higher than projecting alone. The measurement results of standard sphere diameter are close for projectors on both sides. The experimental results manifest that the six-graph separation algorithm method is effective, fast and accurate in separating superimposed gratings. It further proves that the 7
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
Fig. 10. Measurement results of standard sphere diameter under different methods (a) original map; (b) unwrapped phase of separated left projection; (c) unwrapped phase of separated right projection; (d) 3D point cloud based on Fig. 9. (b); (e) 3D point cloud based on (c); (f) fitting result based on (d); (g) fitting result based on (e); Description: Fig. 10 shows the measurement results of standard sphere diameter under different methods. (a) is the original image of a standard sphere projected simultaneously by two projectors. (b) is the unwrapped phase pattern of the standard sphere from the left projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (c) is the unwrapped phase pattern of the standard sphere from the right projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (d) is the 3D point cloud of the standard sphere from the left projector. (e) is the 3D point cloud of the standard sphere from the right projector. (f) is the fitting result of sphere based on (d). (g) is the fitting result of sphere based on (e).
References
1. It solves the separation problem of superimposed gratings and promotes the development of double-projectorSLV 3D measurement technology. 2. Compared with single projector SLV, the point cloud obtained by this method has higher accuracy and can provide more perfect 3D information of objects. 3. Compared with the situation that multiple projectors project at different times, it improves the measurement speed. Due to the application and development of SLV 3D measurement technology development, the separation of superimposed gratings has gradually become a key technical problem that cannot be ignored and urgently to be solved. This paper separates the superimposed gratings in the double-projector system and improves the measurement speed and accuracy. However, there is still much room for improvement in the separation technology with higher efficiency, speed, and accuracy, and it will also be a research hotspot in the field of SLV 3D measurement.
[1] T. Bakirman, M.U. Gumusay, H.C. Reis, M.O. Selbesoglu, S. Yosmaoglu, M.C. Yaras, D.Z. Seker, B. Bayram, Comparison of low cost 3D structured light scanners for face modeling, Appl. Opt. (2017) http://dx.doi.org/10.1364/ao.56. 000985. [2] H. Nguyen, D. Nguyen, Z. Wang, H. Kieu, M. Le, Real-time, high-accuracy 3D imaging and shape measurement, Appl. Opt. (2015) http://dx.doi.org/10.1364/ ao.54.0000a9. [3] J. Geng, Structured-light 3D surface imaging: a tutorial, Adv. Opt. Photonics (2011) http://dx.doi.org/10.1364/aop.3.000128. [4] S. Van der Jeught, J.J.J. Dirckx, Real-time structured light profilometry: A review, Opt. Lasers Eng. (2015) http://dx.doi.org/10.1016/j.optlaseng.2016.01. 011. [5] H. Schwenke, U. Neuschaefer-Rube, T. Pfeifer, H. Kunzmann, Optical methods for dimensional metrology in production engineering, CIRP Ann. - Manuf. Technol. (2002) http://dx.doi.org/10.1016/S0007-8506(07)61707-7. [6] C.H. Bradley, Free-form surface reconstruction for machine vision rapid prototyping, Opt. Eng. (1993) http://dx.doi.org/10.1117/12.145064. [7] C. Je, K.H. Lee, S.W. Lee, Multi-projector color structured-light vision, Signal Process., Image Commun. (2013) http://dx.doi.org/10.1016/j.image.2013.05. 005. [8] S. Savarese, H. Rushmeier, F. Bernardini, P. Perona, Implementation of a shadow carving system for shape capture, in: Proc. - 1st Int. Symp. 3D Data Process. Vis. Transm. 3DPVT 2002, 2002. http://dx.doi.org/10.1109/TDPVT.2002.1024034. [9] G.K.M. Cheung, S. Baker, T. Kanade, Visual hull alignment and refinement across time: a 3D reconstruction algorithm combining shape-from-silhouette with stereo, in: 2003. http://dx.doi.org/10.1109/cvpr.2003.1211493. [10] A. Maimone, H. Fuchs, Reducing interference between multiple structured light depth sensors using motion, in: Proc. - IEEE Virtual Real. 2012. http://dx.doi. org/10.1109/VR.2012.6180879. [11] A. Griesser, T.P. Koninckx, L. Van Gool, Adaptive real-time 3D acquisition and contour tracking within a multiple structured light system, in: Proc. Pacific Conf. Comput. Graph. Appl. 2004. http://dx.doi.org/10.1109/PCCGA. 2004.1348367. [12] C. Je, S.W. Lee, R.H. Park, Colour-stripe permutation pattern for rapid structuredlight range imaging, Opt. Commun. (2012) http://dx.doi.org/10.1016/j.optcom. 2012.01.025. [13] Z. Yan, L. Yu, Y. Yang, Q. Liu, Beyond the interference problem: hierarchical patterns for multiple-projector structured light system, Appl. Opt. (2014) http: //dx.doi.org/10.1364/ao.53.003621.
Acknowledgments The author would like to thank the editor and reviewers for their valuable comments and suggestions. I am especially grateful to the graduated senior fellow apprentice Bai Jingxiang. During school, he provided me with a lot of help in research, which enabled me to get started and guided me during the experiment. This work was done at the State key laboratory of precision measurement and instrumentation. This research was supported by the National Natural Science Foundation of China [grant numbers: 51775379] and by the National Key R&D program of China [grant numbers: 2018YFB2003501]. Formatting of funding sources This work was supported by the National Natural Science Foundation of China [grant numbers: 51775379] and by the National key R&D program of China [grant numbers: 2018YFB2003501]. 8
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676 [20] M. Servin, J.C. Estrada, J.A. Quiroga, The general theory of phase shifting algorithms, Opt. Express (2009) http://dx.doi.org/10.1364/oe.17.021867. [21] C. Zuo, Q. Chen, G. Gu, S. Feng, F. Feng, High-speed three-dimensional profilometry for multiple objects with complex shapes, Opt. Express (2012) http://dx.doi.org/10.1364/oe.20.019493. [22] S. Zhang, S.-T. Yau, High dynamic range scanning technique, in: Two- ThreeDimensional Methods Insp. Metrol. VI, 2008. http://dx.doi.org/10.1117/12. 791265. [23] H. Jiang, H. Zhao, X. Li, High dynamic range fringe acquisition: A novel 3-D scanning technique for high-reflective surfaces, Opt. Lasers Eng. (2012) http://dx.doi.org/10.1016/j.optlaseng.2011.11.021. [24] C. Zuo, Q. Chen, G. Gu, S. Feng, F. Feng, R. Li, G. Shen, High-speed threedimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection, Opt. Lasers Eng. (2013) http://dx.doi. org/10.1016/j.optlaseng.2013.02.012. [25] P.J. Besl, N.D. McKay, A method for registration of 3-D shapes, IEEE Trans. Pattern Anal. Mach. Intell. (1992) http://dx.doi.org/10.1109/34.121791.
[14] P.S. Huang, Novel method for structured light system calibration, Opt. Eng. (2006) http://dx.doi.org/10.1117/1.2336196. [15] Z. Zhang, A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intell. (2000) http://dx.doi.org/10.1109/34.888718. [16] Y. Zhang, Z. Xiong, F. Wu, Unambiguous 3D measurement from speckleembedded fringe, Appl. Opt. (2013) http://dx.doi.org/10.1364/ao.52.007797. [17] Z. Xiaoling, L. Yuchi, Z. Meirong, N. Xiaobing, H. Yinguo, Calibration of a fringe projection profilometry system using virtual phase calibrating model planes, J. Opt. Pure Appl. Opt. (2005) http://dx.doi.org/10.1088/1464-4258/7/4/007. [18] C. D’Argenio, G. Di Leo, C. Liguori, A. Paolillo, A simplified procedure for the calibration of a fringe pattern profilometer, in: 2009 IEEE Intrumentation Meas. Technol. Conf. I2MTC 2009, 2009. http://dx.doi.org/10.1109/IMTC.2009. 5168531. [19] S. Zhang, Recent progresses on real-time 3D shape measurement using digital fringe projection techniques, Opt. Lasers Eng. (2010) http://dx.doi.org/10.1016/ j.optlaseng.2009.03.008.
9
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Separation method of superimposed gratings in double-projector structured-light vision 3D measurement system Li Yiming, Qu Xinghua, Zhang Fumin 1 , ∗, Zhang Yuanjun State Key Laboratory of Precision Measuring Technology and Instruments, Tianjin University, 92 Weijin Road, Tianjin, 300072, China
ARTICLE
INFO
Keywords: Structured-light vision 3D measurement Double-projector system Phase-shifted superimposed gratings Separation method
ABSTRACT In structured-light vision (SLV) 3D measurement technology, double-projector SLV is often used to solve the defects in single projector SLV, such as the shadow caused by the self-occlusion of the measured object’s surface profile and the limitation of the projector’s visual angle. Usually, to avoid gratings’ superimposition, multiple projectors cannot work at the same time. In this paper, we design a new separation method to separate phase-shifted superimposed gratings by controlling the sequence in double-projector SLV. Without changing the original measurement system, the four-step phase-shifted method and dual-frequency method are combined to solve the unwrapped phase. Based on the proposed separation method, the wrapped phases of left and right projections are obtained only by six phase-shifted gratings, whose efficiency is improved by 25%. The separation method includes the process of averaging phase patterns, which improves the phase accuracy of the measured object, comparing with the single projector measurement. Experiments demonstrate that this method can separate phase-shifted superimposed gratings simply, quickly, effectively and accurately without additional hardware devices. This method is practical and general.
1. Introduction SLV 3D measurement has a series of advantages, such as non-contact characteristics, non-destructive features, fast measurement, high precision and high automation [1–4]. It has become one of the most rapidly developed, widely used and potential methods in 3D data acquisition in recent years. It is the key technology of modern scientific research and engineering application [5,6]. However, nowadays, the mature 3D measurement technology is still limited to the single projector structure. There are still some key issues that need to be addressed, such as how to solve the shadow phenomenon caused by the sophisticated surface profile of measured objects, how to relief the limitation of the single projector’s visual angle, and how to obtain the complete 3D information of measured objects quickly and effectively. Shadow is a common phenomenon in SLV measurements. If an area on the surface of an object cannot be illuminated by projected light or whose reflected light cannot be observed by the camera, the area becomes an unmeasurable shadow area. The description of the shadow part in SLV 3D measurement is shown in Fig. 1. For some complicated measured surfaces, data loss is often caused by the raised area obscuring the camera’s line of sight and projecting light. The visual angle of the object is limited in a single projector system, so it is necessary to use multiple projectors to obtain the complete 3D information. Because of the fixed position
and non-contact feature of most measured objects, multi-angle measurement technology has been widely used. However, SLV technology is not easy to use in the aforementioned situations, because if multiple projectors work simultaneously, the gratings would be overlapped. So, the effective phase information cannot be recovered, affecting the 3D measurement results. To avoid the disadvantages of superimposed gratings in measurement, two projectors do not work simultaneously. Comparing with two single-projector systems, the measurement time has not been reduced. Therefore, the separation problem of superimposed gratings limits the development of multi-projector measurement technology. Nowadays multi-projector SLV measurement technology is still in its infancy [7]. To solve the shadow problem, Roland LPX250 laser scanning and measuring machine made in Japan used hardware processing and calculation to complete the measurement of rotary and shifting scanning. To solve the occlusion problem, this measuring method needed to rotate additional mechanical devices from multiple angles, which had a complex structure, high cost and long time. Literature [8] proposed the method of shadow profile to acquire the surface of an object and then analyzed the grayscale distribution of the object image in different lighting conditions. However, the 3D surface information of the sunken areas is difficult to obtain by the method of shadow profile. Literature [9] put forward that, in stereo vision measurement, 3D profile
∗ Correspondence to: 92 Weijin Road, Tianjin University, Tianjin, 300072, China. E-mail address: [email protected] (F. Zhang). 1 Zhang Fumin is willing to handle correspondence at all stages of refereeing and publication, also post-publication.
https://doi.org/10.1016/j.optcom.2019.124676 Received 3 August 2019; Received in revised form 25 September 2019; Accepted 30 September 2019 Available online 3 October 2019 0030-4018/© 2019 Elsevier B.V. All rights reserved.
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
Fig. 2. Geometric model of optical measurement system. Description: Fig. 2 is a classical geometric model for 3D measurement of the grating projection.
Fig. 1. Shadow schematic. Description: Fig. 1 shows the description of the shadow part in SLV 3D measurement. If an area on the surface of an object cannot be illuminated by projected light or whose reflected light cannot be observed by the camera, the area becomes an unmeasurable shadow area.
this paper. Section 4 introduces the experiments of double-projector 3D measurement based on the six-graph separation algorithm and analyzes the experimental results. Section 5 summarizes the paper and prospects the future research directions.
data of objects were acquired by the method of shadow profile, and then surface details such as dents, edges and corners were measured by stereo vision matching technology. The two methods have achieved certain results in 3D measurement of complex profile objects, but the algorithm and the experiment are sophisticated. As for the separation of superimposed signals, Maimone [10] et al. used several motion sensors to make the signals from different coding modes blurred until they could be removed. Griesseretal [11] et al. shielded the structured light signal to establish a system that could eliminate the relative signal of the projection direction. Both the algorithms and structures of this method were complex. This method not only need high hardware performance but also greatly increased the complexity of the measurement system. Changsoo Je [12] et al. proposed a method to solve the partial derivation of left projection in the vertical direction of right projection, and adopted the mode of color fringe displacement to eliminate the interference from the chromatic region through analysis, to achieve the separation of signals. However, this method had high requirements for illumination. Yan et al. [13] achieved signal separation by designing multi-layer signals that do not interfere with each other. Nevertheless, this method relied on the design of the coding signal, whose encoding and decoding methods were complicated. In this research, we propose a separation method called the sixgraph separation method to separate the superimposed gratings in double-projector SLV systems. It designs the projection sequences of two projectors and deals with the superimposed gratings by arctangent function. In detail, it encodes the projection gratings in sinusoidal fringes by the four-step phase-shifted method. In the traditional doubleprojector system, it needs four projection gratings in a single projector to get the wrapped phases, which means that it needs eight gratings as for two projectors. Using the six-graph method, it only needs six gratings in the situation that two projectors work simultaneously, reducing the measurable time. What is more, there is a process of averaging the phase, reducing the noise interference and the system error. Then we can obtain the wrapped phases of both sides from the superimposed gratings. Then we combine with the dual-frequency method to get the unwrapped phases. Calibration and registration are used to reconstruct 3D geometry based on the wrapped phases. Experiments demonstrated the success of the proposed method. The rest of this paper is organized as follows. Section 2 describes related basic principles of the 3D measurement system. Section 3 introduces two principles of separating superimposed gratings proposed in
2. Basic principle 2.1. System calibration
The key process of SLV 3D measurement is to project digital phaseshifted gratings to the measured objects. The grating fringes are deformed after modulated by the height of the measured objects, then the camera collects the image of the deformed fringes to obtain (u, v, 𝜃) information about the object. Where (u, v) is the pixel coordinate, and 𝜃 is the phase value. Finally, it transfers phase information of the object based on the system calibration into the world coordinate values (𝑋𝑤 , 𝑌𝑤 , 𝑍𝑤 ) [14]. Fig. 2 is a classical geometric model for 3D measurement of the grating projection. The measured object is placed on a reference plane. P is the exit pupil center of the DLP projector and I is the entry pupil center of the camera. The distance between the two points is d, the distance from P and I to the reference plane is equal to L. The optical axis of the camera is perpendicular to the reference plane, and O is the projection point of I on the reference plane. The intersection of the optical axes of the projector and the camera is used as the origin to establish the coordinate system. Referring to the system model, we can obtain the relationship between height and phase in Eq. (1). ℎ=
𝛥𝜑 ⋅ 𝐿 2𝜋𝑓 𝑑 + 𝛥𝜑
(1)
𝛥𝜑 is the phase difference caused by the height modulation of the measured object’s surface, where f is the frequency of the sinusoidal fringe. In this paper, the camera calibration parameters are obtained by the Zhang Zhengyou calibration method [15,16] and the camera calibration toolbox of MATLAB. So we can get the pixel coordinates (𝜇, 𝜈) of the image, to solve the world coordinate (𝑋𝑤 , 𝑌𝑤 ). The phase-height conversion model refers to the equal coordinate phase method [17–19]. ( ) Finally, the complete coordinate 𝑋𝑤 , 𝑌𝑤 , 𝑍𝑤 of the measured object is obtained. 2
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
( ) grating order of the right projector is 𝑅1 (𝜑0 = −𝜋), 𝑅3 𝜑0 = 0 , 𝑅1 (𝜑0 = −𝜋). Finally, the superimposed gratings 𝑁1 = 𝐿1 + 𝑅1 , 𝑁2 = 𝐿1 + 𝑅3 , 𝑁3 = 𝐿3 + 𝑅1 are obtained.
2.2. Four-step phase-shift method Four-step phase-shifted algorithm is widely used in high precision and fast structured light 3D measurement technology [20–23]. The expressions of the four-step phase-shift method with initial phases −𝜋, 𝜋/2, 0 and 𝜋/2 are as follows. 𝐼1 = 𝑎 + 𝑏 cos(𝜃(𝑥, 𝑦) − 𝜋)
(2)
𝐼2 = 𝑎 + 𝑏 cos(𝜃(𝑥, 𝑦) − 𝜋∕2)
(3)
𝐼3 = 𝑎 + 𝑏 cos(𝜃(𝑥, 𝑦))
(4)
𝐼4 = 𝑎 + 𝑏 cos(𝜃(𝑥, 𝑦) + 𝜋∕2)
(5)
𝐿 + 𝑅1 + 𝐿1 + 𝑅3 𝑁1 + 𝑁2 = 1 = 𝐿1 + 𝑎 2 2 𝐿 + 𝑅1 + 𝐿3 + 𝑅1 𝑁1 + 𝑁3 = 1 = 𝑅1 + 𝑎 2 2
(12)
Based on the method of phase complementarity, Eqs. (11)–(12) indicate that the effective phase information of a single projection is obtained. Because the four-step phase-shifted method can eliminate the background light intensity, the separated signal can be directly used to solve the wrapped phase. According to the method of phase complementarity, the superimposed gratings can be effectively separated.
The phase distribution can be obtained by using the arctangent function: 𝐼 (𝑥, 𝑦) − 𝐼2 (𝑥, 𝑦) 𝜃(𝑥, 𝑦) = 𝑎𝑟𝑐𝑡𝑎𝑛 4 (6) 𝐼1 (𝑥, 𝑦) − 𝐼3 (𝑥, 𝑦)
3.2. Six-graph separation method As for the method of phase complementarity, it still needs 8 patterns to obtain the wrapped phase. Therefore, we propose a new faster method for separating superimposed gratings. The initial phases of 𝐿1 ∕𝑅1 , 𝐿2 ∕𝑅2 , 𝐿3 ∕𝑅3 , 𝐿4 ∕𝑅4 are −𝜋, −𝜋∕2, 0 and 𝜋∕2 respectively. The left and right projectors work simultaneously, and the left projector projected 6 images in the order of 𝐿1 , 𝐿1 , 𝐿2 , 𝐿3 , 𝐿4 , and 𝐿4 . The right projector projected 6 images in the order of 𝑅1 , 𝑅3 , 𝑅2 , 𝑅1 , 𝑅2 , and 𝑅4 . Eq. (13) represents six superimposed gratings images captured by the camera.
In the Eqs. (2)–(6), 𝐼 denotes the uneven reflectivity of the surface of the measured object. 𝑎 is the background intensity. 𝑏 denotes the contrast of grating fringes, and 𝜃(𝑥, 𝑦) is the wrapped phase value. 2.3. Dual-frequency method Referring to the literature [24], this paper uses the dual-frequency method to calculate the unwrapped phase of the measured object. In this paper, k is the ratio of frequency of the high-frequency grating to the low-frequency grating frequency. Where 𝜃𝐻 (𝑥, 𝑦), 𝜃𝐿 (𝑥, 𝑦), 𝜑𝐻 (𝑥, 𝑦) and 𝜑𝐿 (𝑥, 𝑦) are high-frequency wrapped phase, low-frequency wrapped phase, high-frequency unwrapped phase, and low-frequency unwrapped phase respectively. The unwrapping results of the highfrequency fringes are as follows: ] [ 𝑘 ∗ 𝜑𝐿 (𝑥, 𝑦) − 𝜃𝐻 (𝑥, 𝑦) (7) 𝑛𝑢𝑚 (𝑥, 𝑦) = 𝑖𝑛𝑡 2𝜋 𝜑𝐻 (𝑥, 𝑦) = 2𝜋 ∗ 𝑛𝑢𝑚(𝑥, 𝑦) + 𝜃𝐻 (𝑥, 𝑦)
(11)
𝐻1 = 𝐿1 + 𝑅1 , 𝐻2 = 𝐿1 + 𝑅3 , 𝐻3 = 𝐿2 + 𝑅2
𝐻4 = 𝐿3 + 𝑅1 , 𝐻5 = 𝐿4 + 𝑅2 , 𝐻6 = 𝐿4 + 𝑅4
(13)
Referring to Eq. (5):
(8)
where num is a level value and int is an integer operation. For the left projection separated from the superimposed grating, we solved the wrapped phase 𝜃𝐻 (𝑥, 𝑦) and 𝜃𝐿 (𝑥, 𝑦) of high frequency and low frequency respectively. 𝜃𝐿 (𝑥, 𝑦), whose frequency is 1, is the same as the unwrapped phase 𝜑𝐿 (𝑥, 𝑦). The low-frequency grating acts as a ruler, and 𝜑𝐿 (𝑥, 𝑦) is used to obtain the unwrapped high-frequency phase 𝜑𝐻 (𝑥, 𝑦).
𝜃1 = 𝑡𝑎𝑛−1
𝐻5 − 𝐻3 𝐿 + 𝑅2 − (𝐿2 + 𝑅2 ) 𝐿 − 𝐿2 = 𝑡𝑎𝑛−1 4 = 𝑡𝑎𝑛−1 4 𝐻1 − 𝐻4 𝐿1 + 𝑅1 − (𝐿3 + 𝑅1 ) 𝐿1 − 𝐿3
𝜃2 = 𝑡𝑎𝑛−1
𝐻6 − 𝐻5 𝑅 − 𝑅2 𝐿 + 𝑅4 − (𝐿4 + 𝑅2 ) = 𝑡𝑎𝑛−1 4 = 𝑡𝑎𝑛−1 4 𝐻1 − 𝐻2 𝐿1 + 𝑅1 − (𝐿1 + 𝑅3 ) 𝑅1 − 𝑅3
(14)
In the Eq. (14), 𝜃1 and 𝜃2 are the wrapped phase value of left projection and right projection respectively, which proves that the superimposed gratings based on the six-graph separation method can be effectively separated. Assume that 𝜃1 and 𝜃2 are the separated low-frequency wrapped phases of the left and right projectors respectively. 𝜃3 and 𝜃4 are the separated high-frequency wrapped phases of the left and right projectors respectively. 𝜃1 and 𝜃3 are processed by the dual-frequency method to obtain high-frequency unwrapped phase 𝜑𝑙𝑒𝑓 𝑡 of the left projector. 𝜃2 and 𝜃4 are processed by dual-frequency method to obtain high-frequency unwrapped phase 𝜑𝑟𝑖𝑔ℎ𝑡 of right projector. For the dual projector single camera system, considering the superposition of light patterns, the two projectors cannot work at the same time. It takes time and workload of 8 images to solve the low-frequency wrapped phase as for the left and right projectors, and the same is true for the highfrequency wrapped phase. To solve the unwrapped phase of two sides, the left and right projectors need to project 16 images successively. In this way, the experimental process is long and tedious. Based on the separation algorithm, 6 images are projected in parallel when two projectors worked at the same time to solve the low-frequency wrapped phase as for the left and right projectors, and the same is true for the high-frequency wrapped phase. For the unwrapped phase, the dual-frequency method requires 12 sets of parallel images for simultaneous projection. Therefore, this separation method improves the measurement speed and reduces the measurement workload.
3. Separation method of superimposed gratings 3.1. Phase complementarity method Referring to the properties of sinusoidal phase information, we add Eqs. (3) and (5) to average the background light intensity of projection gratings. Similarly, using Eqs. (2) and (4), the background light intensity of projection gratings is also obtained. It can separate the phase information of the superimposed gratings. 𝑎 + 𝑏 cos(2𝜋𝑓 𝑥 + 𝜋2 ) + 𝑎 + 𝑏 cos(2𝜋𝑓 𝑥 − 𝜋2 ) 𝐼4 (𝑥, 𝑦) + 𝐼2 (𝑥, 𝑦) = = 𝑎 (9) 2 2 𝐼3 (𝑥, 𝑦) + 𝐼1 (𝑥, 𝑦) 𝑎 + 𝑏 cos(2𝜋𝑓 𝑥) + 𝑎 + 𝑏 cos(2𝜋𝑓 𝑥 − 𝜋) = =𝑎 (10) 2 2 The theoretical results are validated by taking 𝐼1 and 𝐼3 as examples. The initial phase is equal to −𝜋, −𝜋/2, 0 and 𝜋/2. Eqs. (9)–(10) indicate that the background light intensity of the phase-shifted gratings can be obtained from the superimposed gratings by the method of phase complementarity. The left and right projectors simultaneously project to the plane, where 𝜑0 represents phase. ( )the initial ( ) The grating order of the left projector is 𝐿1 𝜑0 = −𝜋 , 𝐿1 𝜑0 = −𝜋 , 𝐿3 (𝜑0 = 0). At the same time, the 3
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
4. Experiments and analysis In this paper, we have designed a double-projector measurement system based on the separation method of superimposed gratings to measure various real objects, like Agrippa face and standard sphere. Fig. 3 is the experimental device diagram. Two DLP Light Crafter 4500 1140*912 projectors and a Point Gray GS3-U3-41C6M-C 2048*2048 CMOS camera were used. Left and right projectors projected sinusoidal gratings with different phases and different sequences to the measured object, and the camera collected images with phase information. The 9*10 checkerboard was used for calibrating the camera. The length of the small checkerboard is 20 mm, and an accuracy of ±5 μm. Fig. 4 shows the overall structure design of the double-projector 3D measurement experiment. Firstly, we accomplished the system calibration. Secondly, based on the contents of 3.2, the double projectors worked simultaneously, and the camera collected superimposed gratings with phase information. Thirdly, we used the six-graph separation algorithm to separate the superimposed gratings and obtained the wrapped phase information of the left and right projections separately. Fourthly, unwrapped phases were obtained by the dual-frequency method, which was mentioned in Section 2.3. Fifthly, according to the principle of 2.1, the 3D point clouds from two perspectives of the measured object were obtained. Finally, a 3D point cloud with a complete surface profile of the object was obtained based on registration [25]. We have designed four sets of experiments to verify the three advantages of the separation algorithm proposed in this paper: great efficiency, speed, and accuracy. Experiment 1, based on the phase complementarity method, verified the feasibility of this method on separating superimposed gratings. Experiment 2, according to the six-graph separation method, proved that this algorithm can separate superimposed gratings effectively. The third experiment was the 3D measurement experiment of the Agrippa face, verifying the great efficiency, speed of the algorithm. The fourth experiment was a 3D measurement experiment
Fig. 3. Double-projector measurement system device. Description: Fig. 3 is the experimental device diagram.
of the standard sphere, which verified that the separation algorithm has higher accuracy than single projection. Finally, in experiment 1 and 2, the horizontal fringes were used in the left projector, and the vertical fringes were used in the right projector. In experiment 3 and 4, the vertical fringes were used in both the left and right projectors. 4.1. Experiment of phase complementarity method Referring to Section 3.1, we designed the experiment. The theoretical results were validated by taking 𝐼1 and 𝐼3 for example. Fig. 5 is superimposed grating images captured by the camera. (a) is the
Fig. 4. The overall structure double-projector 3D measurement. Description: Fig. 4 shows the overall structure design of the double-projector 3D measurement experiment.
Fig. 5. (a) 𝑁1 (b) 𝑁2 (c) 𝑁3 . Description: Fig. 5 is superimposed grating images captured by the camera. (a) is the superposition signal 𝑁1 , where the phases of the left projection grating and the right projection grating are both −𝜋. (b) is the superposition signal 𝑁2 , where the phases of left and right projections are −𝜋 and 0, respectively. (c) is the superposition signal 𝑁3 , where the phases of left and right projections are 0 and −𝜋, respectively;. 4
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
and obtain the unwrapped phase value faster. Moreover, there is a process of averaging projection patterns in the separation method, which reduces noise and system error. 4.3. 3D measurement of the Agrippa face To verify the completeness and validity of the double-projector 3D measurement system based on the six-graph separation algorithm, Agrippa face was selected as the measured object as shown in Fig. 9. The point cloud data of the measured object was simplified and processed by related algorithms. Besides, we combined the point clouds of left and right projection separated from the superimposed gratings to synthesize complete Agrippa face. Fig. 9 is the result of 3D measurement of the Agrippa face. (a) is the original image of an Agrippa face projected simultaneously by two projectors. (b) is the unwrapped phase image of the Agrippa face from the left projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (c) is the unwrapped phase image of the Agrippa face from the right projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (d) is the 3D measurement point cloud of the Agrippa face from the left projector. (e) is the result of the triangular meshing of point cloud in (d). (f) is the 3D measurement point cloud of the Agrippa face from the right projector. (g) is the result of the triangular meshing of point cloud in (f). (h) is the complete point cloud of Agrippa face after registration of point cloud in images (d) and (f). (i) is the complete point cloud (upward view) after registration. Fig. 9 shows that the 3D point clouds obtained by superimposed gratings are neat and the details are clear, which further proves that the six-graph separation algorithm can effectively separate the superimposed gratings. From (d) and (e), we can see that point clouds on the right side of the nose and right face are missing. From (f) and (g), it can be seen that the point clouds on the left side of the nose and left face are missing, too. (h) (i) are obtained after registration of point clouds on the left and right sides. The measurement incompleteness caused by the shadow on the nose and face of the Agrippa statue is effectively solved. The experimental results demonstrate that the six-graph separation method can effectively separate the superimposed gratings so that the double-projector system can solve the shadow problem caused by the complicated surface of the measured object when both left and right projectors are projected simultaneously. It expands the measurement range and obtain the complete phase information of the measured object through registration.
Fig. 6. Separation results of superimposed gratings based on phase complementarity principle. (a) the separated right projection; (b) the separated left projection; Description: Fig. 6 shows the separation results of superimposed gratings 𝑁1 , 𝑁2 and 𝑁3 in Fig. 5 based on Eqs. (9)–(12). (a) is the separation result of right projector from superimposed grating 𝑁1 and 𝑁2 . (b) is the separation result of left projector from superimposed gratings 𝑁1 and 𝑁3 .
superimposed signal 𝑁1 , where the phases of the left projection grating and the right projection grating are −𝜋. (b) is the superimposed signal 𝑁2 , where the phases of left and right projections are −𝜋 and 0, respectively. (c) is the superimposed signal 𝑁3 , where the phases of left and right projections are 0 and −𝜋, respectively. Fig. 6 shows the separation results of superimposed gratings 𝑁1 , 𝑁2 and 𝑁3 . (a) is the separation result of the right projector from superimposed gratings 𝑁1 and 𝑁2 . (b) is the separation result of the left projector from superimposed gratings 𝑁1 and 𝑁3 . The experimental results demonstrate that the separated gratings obtained by the phase complementary method has one more background intensity than that obtained by the single projector method. The background light intensity can be eliminated by the four-step phase-shifted method. So, this method is effective for the separation of superimposed gratings. 4.2. Experiment of six-graph separation method Fig. 7 shows 12 superimposed gratings with different phase combinations, which are divided into two groups according to the frequency of gratings. Fig. (a–f) represent the high-frequency gratings, whose frequency is 6. Fig. (g–l) represent the low-frequency gratings, whose frequency is 1. Referring to Sections 2 and 3, we designed simple projection gratings and separation algorithms. For high-frequency superimposed gratings, the left and right projectors projected six phaseshifted gratings to the measured object simultaneously, and then the camera collected superimposed gratings 𝐻1 , 𝐻2 , 𝐻3 , 𝐻4 , 𝐻5 , and 𝐻6 . (a) is the superimposed grating 𝐻1 , the combination of 𝐿1 and 𝑅1 . (b) is the superimposed grating 𝐻2 , the combination of 𝐿1 and 𝑅3 . (c) is the superimposed grating 𝐻3 , the combination of 𝐿2 and 𝑅2 . (d) is the superimposed grating 𝐻4 , the combination of 𝐿3 and 𝑅1 . (e) is the superimposed grating 𝐻5 , the combination of 𝐿4 and 𝑅2 . (f) is the superimposed grating 𝐻6 , the combination of 𝐿4 and 𝑅4 . For low-frequency superimposed gratings 𝑆1 , 𝑆2 , 𝑆3 , 𝑆4 , 𝑆5 and 𝑆6 , the phase distribution and combination sequence are consistent with those of high-frequency gratings. The superimposed gratings were separated by Eq. (14). Fig. 8 is the separation results of the superimposed gratings. (a) is the Agrippa face’s low-frequency wrapped phase of the right projector. (b) is the Agrippa face’s high-frequency wrapped phase of the right projector. (c) is the Agrippa face’s unwrapped phase of the right projector computed by the dual-frequency method. (d) is the Agrippa face’s low-frequency wrapped phase of the left projector. (e) is the Agrippa face’s highfrequency wrapped phase of the left projector. (f) is the Agrippa face’s unwrapped phase of the left projector computed by the dual-frequency method. From the separation results, we can see that the separation method proposed in this paper can separate the superimposed gratings effectively, eliminate the interference signal of the overlapped part,
4.4. 3D measurement experiment of the standard sphere To prove the influence of the separation algorithm on the accuracy, besides the double-projector experiment of the standard sphere, we added a group of experiments that the single projector projected alone. The diameter of the standard sphere measured by CMM is 40.056 mm. The diameter was measured by two methods, a single projector projecting alone and double projectors projecting simultaneously, ensuring the experimental conditions and the processing algorithm were consistent. The 3D point cloud data of the standard sphere were shown in the software NX Imageware. Fig. 10 shows the measurement results of the standard sphere diameter via different methods. (a) is the original image of a standard sphere projected simultaneously by two projectors. (b) is the unwrapped phase of the standard sphere from the left projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (c) is the unwrapped phase of the standard sphere from the right projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (d) is the 3D point cloud of the standard sphere from the left projector. (e) is the 3D 5
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
Fig. 7. High and low frequency superimposed gratings (a) 𝐻1 ; (b) 𝐻2 ; (c) 𝐻3 ; (d) 𝐻4 ; (e) 𝐻5 ; (f) 𝐻6 ; (g) 𝑆1 ; (h) 𝑆2 ; (i) 𝑆3 ; (j) 𝑆4 ; (k) 𝑆5 ; (l) 𝑆6 ; Description: Fig. 7 shows 12 superimposed gratings with different phase combinations. Fig. (a–f) represent the high-frequency gratings, whose frequency is 6. Fig. (g–l) represent the low-frequency gratings, whose frequency is 1. (a) is 𝐻1 , the superposition of 𝐿1 and 𝑅1 . (b) is 𝐻2 , the superposition of 𝐿1 and 𝑅3 . (c) is 𝐻3 , the superposition of 𝐿2 and 𝑅2 . (d) is 𝐻4 , the superposition of 𝐿3 and 𝑅1 . (e) is 𝐻5 , the superposition of 𝐿4 and 𝑅2 . (f) is 𝐻6 , the superposition of 𝐿4 and 𝑅4 . For low-frequency superimposed gratings, the phase distribution and combination sequence are consistent with those of high-frequency gratings. (g) is 𝑆1 . (h) is 𝑆2 . (i) is 𝑆3 . (j) is 𝑆4 . (k) is 𝑆5 . (l) is 𝑆6 .
Fig. 8. Separated results of superimposed gratings (a) low-frequency wrapped phase of right projector; (b) high-frequency wrapped phase of right projector; (c) unwrapped phase of right projector; (d) low-frequency wrapped phase of left projector; (e) high-frequency wrapped phase of left projector; (f) unwrapped phase of left projector; Description: Fig. 8 is the separation result of the superimposed gratings. (a) is the Agrippa face’s low-frequency wrapped phase of the right projector separated from the superimposed gratings. (b) is the Agrippa face’s high-frequency wrapped phase of the right projector separated from the superimposed gratings. (c) is the Agrippa face’s unwrapped phase of the right projector computed by the dual-frequency method. (d) is the Agrippa face’s low-frequency wrapped phase of the left projector separated from the superimposed gratings. (e) is the Agrippa face’s high-frequency wrapped phase of the left projector separated from the superimposed gratings. (f) is the Agrippa face’s unwrapped phase of the left projector computed by the dual-frequency method.
point cloud of the standard sphere from the right projector. (f) is the fitting result of the sphere based on (d). (g) is the fitting result of the sphere based on (e). The fitting results are shown in Table 1, including the diameter and the average error. D represents the condition projected by a single projector. DD represents the condition of simultaneous projection, which is based on the six-graph separation method. Table 1 shows the measurement results of the standard sphere via different measurement methods. For the left projector, the average
diameter of standard sphere is 39.976 mm and the mean error is 80 μm based on DD. The average diameter is 39.956 mm and the mean error was 100 μm in experiments projected based on D. For the right projector, the average diameter of the standard sphere is 39.966 mm and the mean error is 90 μm based on DD. The average diameter is 39.944 mm and the mean error is 112 μm based on D. The experimental results indicate that the 3D measurement results based on the six-graph separation method were closer to the true values than the results based on a single projector. In the separation algorithm, the gratings undergo 6
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
Fig. 9. The results of 3D measurement of Agrippa face (a) original map; (b) unwrapped phase of separated left projection; (c) unwrapped phase of separated right projection; (d) 3D point cloud based on (b); (e) triangular meshing of point cloud on (d); (f) 3D point cloud based on (c); (g) triangular meshing of point cloud based on (f); (h) complete point clouds of face profile; (i) complete point clouds of face profile (upward view); Description: Fig. 9 is the result of 3D measurement of Agrippa face. (a) is the original image of an Agrippa face projected simultaneously by two projectors; (b) is the unwrapped phase image of the Agrippa face from the left projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method; (c) is the unwrapped phase image of the Agrippa face from the right projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method; (d) is the 3D measurement point cloud of the Agrippa face from the left projector; (e) is the result of triangular meshing of point cloud in (d); (f) is the 3D measurement point cloud of the Agrippa face from the right projector; (g) is the result of triangular meshing of point cloud in (f); (h) is the complete point clouds of Agrippa face after registration of point clouds in images (d) and (f); (i) is the complete point clouds after registration (upward view). Table 1 Measurement results of standard sphere diameter under different measurement methods. Description: Table 1 shows the measurement results of standard sphere under different measurement methods, including the diameter and the average error. D represents the condition that projected by single projector. DD represents the condition of simultaneous projection, which is based on the six-graph separation method. Left projector
D DD
double-projector measurement SLV is a fast, accurate and effective 3D measurement method. 5. Conclusion and prospect Double-projector measurement technology can effectively solve the shadow phenomenon, where the efficient separation of superimposed gratings is the key question. We proposed a method to solve the aforementioned problem, called the six-graph separation method, by changing the sequence of gratings with different phase and calculating them. In this paper, the superimposed gratings are obtained by controlling the sequence of left and right projection with the doubleprojector system device. The separation algorithm is used to obtain the unwrapped phase of the measured object by combining the four-step phase-shifted method with the dual-frequency method. The phaseheight conversion model based on the equal-phase coordinate method is used to measure the object. Finally, three-dimensional measurement is completed. The practicability and generalization of this separation algorithm are verified by experiments. In conclusion, the practical significances in detail are as follows:
Right projector
Diameter (mm)
Mean error (mm)
Diameter (mm)
Mean error (mm)
39.956 39.976
0.100 0.080
39.944 39.966
0.112 0.090
the process of superposition and separation, whose phase information is averaged, effectively reducing the noise and random error. Thus, the measurement accuracy of superimposed parts is higher than projecting alone. The measurement results of standard sphere diameter are close for projectors on both sides. The experimental results manifest that the six-graph separation algorithm method is effective, fast and accurate in separating superimposed gratings. It further proves that the 7
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676
Fig. 10. Measurement results of standard sphere diameter under different methods (a) original map; (b) unwrapped phase of separated left projection; (c) unwrapped phase of separated right projection; (d) 3D point cloud based on Fig. 9. (b); (e) 3D point cloud based on (c); (f) fitting result based on (d); (g) fitting result based on (e); Description: Fig. 10 shows the measurement results of standard sphere diameter under different methods. (a) is the original image of a standard sphere projected simultaneously by two projectors. (b) is the unwrapped phase pattern of the standard sphere from the left projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (c) is the unwrapped phase pattern of the standard sphere from the right projector, which is separated from the superimposed gratings based on the six-graph separation algorithm and computed by the dual-frequency method. (d) is the 3D point cloud of the standard sphere from the left projector. (e) is the 3D point cloud of the standard sphere from the right projector. (f) is the fitting result of sphere based on (d). (g) is the fitting result of sphere based on (e).
References
1. It solves the separation problem of superimposed gratings and promotes the development of double-projectorSLV 3D measurement technology. 2. Compared with single projector SLV, the point cloud obtained by this method has higher accuracy and can provide more perfect 3D information of objects. 3. Compared with the situation that multiple projectors project at different times, it improves the measurement speed. Due to the application and development of SLV 3D measurement technology development, the separation of superimposed gratings has gradually become a key technical problem that cannot be ignored and urgently to be solved. This paper separates the superimposed gratings in the double-projector system and improves the measurement speed and accuracy. However, there is still much room for improvement in the separation technology with higher efficiency, speed, and accuracy, and it will also be a research hotspot in the field of SLV 3D measurement.
[1] T. Bakirman, M.U. Gumusay, H.C. Reis, M.O. Selbesoglu, S. Yosmaoglu, M.C. Yaras, D.Z. Seker, B. Bayram, Comparison of low cost 3D structured light scanners for face modeling, Appl. Opt. (2017) http://dx.doi.org/10.1364/ao.56. 000985. [2] H. Nguyen, D. Nguyen, Z. Wang, H. Kieu, M. Le, Real-time, high-accuracy 3D imaging and shape measurement, Appl. Opt. (2015) http://dx.doi.org/10.1364/ ao.54.0000a9. [3] J. Geng, Structured-light 3D surface imaging: a tutorial, Adv. Opt. Photonics (2011) http://dx.doi.org/10.1364/aop.3.000128. [4] S. Van der Jeught, J.J.J. Dirckx, Real-time structured light profilometry: A review, Opt. Lasers Eng. (2015) http://dx.doi.org/10.1016/j.optlaseng.2016.01. 011. [5] H. Schwenke, U. Neuschaefer-Rube, T. Pfeifer, H. Kunzmann, Optical methods for dimensional metrology in production engineering, CIRP Ann. - Manuf. Technol. (2002) http://dx.doi.org/10.1016/S0007-8506(07)61707-7. [6] C.H. Bradley, Free-form surface reconstruction for machine vision rapid prototyping, Opt. Eng. (1993) http://dx.doi.org/10.1117/12.145064. [7] C. Je, K.H. Lee, S.W. Lee, Multi-projector color structured-light vision, Signal Process., Image Commun. (2013) http://dx.doi.org/10.1016/j.image.2013.05. 005. [8] S. Savarese, H. Rushmeier, F. Bernardini, P. Perona, Implementation of a shadow carving system for shape capture, in: Proc. - 1st Int. Symp. 3D Data Process. Vis. Transm. 3DPVT 2002, 2002. http://dx.doi.org/10.1109/TDPVT.2002.1024034. [9] G.K.M. Cheung, S. Baker, T. Kanade, Visual hull alignment and refinement across time: a 3D reconstruction algorithm combining shape-from-silhouette with stereo, in: 2003. http://dx.doi.org/10.1109/cvpr.2003.1211493. [10] A. Maimone, H. Fuchs, Reducing interference between multiple structured light depth sensors using motion, in: Proc. - IEEE Virtual Real. 2012. http://dx.doi. org/10.1109/VR.2012.6180879. [11] A. Griesser, T.P. Koninckx, L. Van Gool, Adaptive real-time 3D acquisition and contour tracking within a multiple structured light system, in: Proc. Pacific Conf. Comput. Graph. Appl. 2004. http://dx.doi.org/10.1109/PCCGA. 2004.1348367. [12] C. Je, S.W. Lee, R.H. Park, Colour-stripe permutation pattern for rapid structuredlight range imaging, Opt. Commun. (2012) http://dx.doi.org/10.1016/j.optcom. 2012.01.025. [13] Z. Yan, L. Yu, Y. Yang, Q. Liu, Beyond the interference problem: hierarchical patterns for multiple-projector structured light system, Appl. Opt. (2014) http: //dx.doi.org/10.1364/ao.53.003621.
Acknowledgments The author would like to thank the editor and reviewers for their valuable comments and suggestions. I am especially grateful to the graduated senior fellow apprentice Bai Jingxiang. During school, he provided me with a lot of help in research, which enabled me to get started and guided me during the experiment. This work was done at the State key laboratory of precision measurement and instrumentation. This research was supported by the National Natural Science Foundation of China [grant numbers: 51775379] and by the National Key R&D program of China [grant numbers: 2018YFB2003501]. Formatting of funding sources This work was supported by the National Natural Science Foundation of China [grant numbers: 51775379] and by the National key R&D program of China [grant numbers: 2018YFB2003501]. 8
Y. Li, X. Qu, F. Zhang et al.
Optics Communications 456 (2020) 124676 [20] M. Servin, J.C. Estrada, J.A. Quiroga, The general theory of phase shifting algorithms, Opt. Express (2009) http://dx.doi.org/10.1364/oe.17.021867. [21] C. Zuo, Q. Chen, G. Gu, S. Feng, F. Feng, High-speed three-dimensional profilometry for multiple objects with complex shapes, Opt. Express (2012) http://dx.doi.org/10.1364/oe.20.019493. [22] S. Zhang, S.-T. Yau, High dynamic range scanning technique, in: Two- ThreeDimensional Methods Insp. Metrol. VI, 2008. http://dx.doi.org/10.1117/12. 791265. [23] H. Jiang, H. Zhao, X. Li, High dynamic range fringe acquisition: A novel 3-D scanning technique for high-reflective surfaces, Opt. Lasers Eng. (2012) http://dx.doi.org/10.1016/j.optlaseng.2011.11.021. [24] C. Zuo, Q. Chen, G. Gu, S. Feng, F. Feng, R. Li, G. Shen, High-speed threedimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection, Opt. Lasers Eng. (2013) http://dx.doi. org/10.1016/j.optlaseng.2013.02.012. [25] P.J. Besl, N.D. McKay, A method for registration of 3-D shapes, IEEE Trans. Pattern Anal. Mach. Intell. (1992) http://dx.doi.org/10.1109/34.121791.
[14] P.S. Huang, Novel method for structured light system calibration, Opt. Eng. (2006) http://dx.doi.org/10.1117/1.2336196. [15] Z. Zhang, A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intell. (2000) http://dx.doi.org/10.1109/34.888718. [16] Y. Zhang, Z. Xiong, F. Wu, Unambiguous 3D measurement from speckleembedded fringe, Appl. Opt. (2013) http://dx.doi.org/10.1364/ao.52.007797. [17] Z. Xiaoling, L. Yuchi, Z. Meirong, N. Xiaobing, H. Yinguo, Calibration of a fringe projection profilometry system using virtual phase calibrating model planes, J. Opt. Pure Appl. Opt. (2005) http://dx.doi.org/10.1088/1464-4258/7/4/007. [18] C. D’Argenio, G. Di Leo, C. Liguori, A. Paolillo, A simplified procedure for the calibration of a fringe pattern profilometer, in: 2009 IEEE Intrumentation Meas. Technol. Conf. I2MTC 2009, 2009. http://dx.doi.org/10.1109/IMTC.2009. 5168531. [19] S. Zhang, Recent progresses on real-time 3D shape measurement using digital fringe projection techniques, Opt. Lasers Eng. (2010) http://dx.doi.org/10.1016/ j.optlaseng.2009.03.008.
9