- Email: [email protected]
Analysis of imaging characteristics of parallax images in double diffraction grating imaging
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.com/locate/ijleo
Original research article
Analysis of imaging characteristics of parallax images in double diffraction grating imaging Hoon Yooa, Jung-Un Jangb,*, Jae-Young Jangb,* a b
Department of Electronics Engineering, Sangmyung University, 20 Hongjimoon-2gil, Jongno-gu, Seoul, South Korea Department of Optometry, Eulji University, 553, Sanseong-daero, Sujeong-gu, Seongnam-si, Gyonggi-do, South Korea
A R T IC LE I N F O
ABS TRA CT
Keywords: Diffraction grating imaging Parallax image array Three-dimensional imaging
The relation between the imaging characteristics of parallax images (PIs) and the system parameters of the double diffraction grating (DDG) imaging system is analyzed. DDG imaging method may produce parallax images (PIs) efficiently. However, a geometrical analysis is required to improve the applicability of the imaging system. The system parameters considered include the wavelength of light source, the spatial resolution of the diffraction grating, and positions of diffraction gratings and an imaging lens. The imaging characteristics of the DDG imaging defined and described in terms of positions and parallax angles of PIs. The ray-optical analysis is performed for the explanation of the imaging characteristics.
1. Introduction A parallax image array (PIA) has played an important role in the fields of three-dimensional (3-D) display, 3-D data processing, 3D profiling, and so on [1–11]. Thus, acquiring PIA data is an essential part of 3-D imaging technology. Recently, a method of using a single diffraction grating (SDG) for generating a PIA was proposed [12]. Compared with a conventional lens array and a camera array, this method has the advantages of simple optical systems, low aberration, and no lens distortion. Thus, it can be one of the promising techniques in 3-D imaging. However, it has drawbacks such that the numbers of parallax images (PIs) and viewpoints are limited by the physical size and higher-order diffraction efficiency of the diffraction grating (DG) in use. Accordingly, a method of using double diffraction gratings (DDG) has been studied as a way to increase the number of PIs and viewpoints while taking advantage of the DG in PIA generation [13]. In this method, the feasibility of DDG for generating a PIA is confirmed by wave-optical analysis and optical experiments. The total number of PIs by DDG may be increased by the square of the total number of PIs by the SDG. However, the lack of analysis of system parameters and imaging characteristics of PIs, especially PIs with non-sequential parallax angles (PAs), reduces the applicability of DDG imaging systems. Therefore, geometrical analysis and explanation of the imaging characteristics for the generated PIs are needed in order to improve the applicability of the DDG imaging system. In this letter, we analyze the imaging characteristics of the PIs according to the system parameters of the DDG imaging system. By analyzing imaging characteristics, we intend to improve the understanding and applicability of DDG imaging. The system parameters considered include the wavelength of light source, a spatial resolution of diffraction, and positions of diffraction gratings and an imaging lens. In order to investigate the imaging characteristics, we first derive the equations for imaging positions of the individual PIs generated by DDG. Next, the parallax angles corresponding to the individual PIs are derived in consideration of the geometrical relationship between the imaging positions of the individual PIs and a camera (imaging lens). The imaging characteristics are
⁎
Corresponding authors. E-mail addresses: [email protected] (J.-U. Jang), [email protected] (J.-Y. Jang).
https://doi.org/10.1016/j.ijleo.2019.163826 Received 5 September 2019; Accepted 19 November 2019 0030-4026/ © 2019 Elsevier GmbH. All rights reserved.
Please cite this article as: Hoon Yoo, Jung-Un Jang and Jae-Young Jang, Optik - International Journal for Light and Electron Optics, https://doi.org/10.1016/j.ijleo.2019.163826
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 1. Geometrical relationships of a point object, PIs, and DDG.
analyzed based on the derived equations and the validity of the analysis is verified through optical experiments. Since the DDG imaging system can be easily expanded from one-dimensional (1-D) to two-dimensional (2-D) in the PIA generation, the analysis is performed only for the 1-D PIA generation. 2. Geometrical analysis of parallax images in diffraction grating imaging The scattered light from an object is diffracted by DDG located on the optical path and then generates virtual images on the back of the DDG where the object is located. These virtual images are PIs having different viewpoints with respect to the object and are stored as a PIA by a capturing device such as a camera. The imaging depth and the size of the PIs are the same as the depth and size of the object. Fig. 1 describes the geometrical relationship between a point object and its PIs in DDG imaging. Here, let us assume that the first diffraction grating DG1 is located along the x-axis and the point object is located at (x0th, zO) in the x-z plane. Therefore, the zcoordinates of all PIs are zO. The second diffraction grating DG2 and imaging lens are separated from the origin by d2 and d1 + d2, respectively. For the convenience of explanation, the minus orders of PIs are omitted in Fig. 1. In Fig. 1, the 1st (red line) and 3rd (orange line) order PIs are generated due to the 1st order diffraction of the point object by DG1 and DG2, respectively. The point object x0th serves as the 0th order PI. The diffraction angles θ is given by θ = sin−1(λ/a) for both DG1 and DG2, where λ is the wavelength of a light source, and a is aperture width of DGs. Taking the diffraction order into account, x-coordinate of 1st order PI is given by
λ x1st = x 0th + |z O| tan ⎛sin−1 ⎛ ⎞ ⎞. ⎝ a ⎠⎠ ⎝ ⎜
⎟
(1)
The x-coordinate of 3rd order PI generated by DG2 is given by
λ x3rd = x 0th + (|z O| + d2) tan ⎛sin−1 ⎛ ⎞ ⎞. ⎝ a ⎠⎠ ⎝ ⎜
⎟
(2)
The 1st order diffracted ray (orange line) by DG2 is diffracted once more by DG1. As a result, the 2nd and 4th orders PIs are generated. The 2nd (green line) and 4th (cyan line) order PIs are obtained from -1st and 1st order diffractions of the 3rd order PI caused by DG1, respectively. Here, the 3rd order PI serves as a virtual object. The x-coordinate of 2nd order PI is given by
λ λ x2nd = x G2G1 + |z O| tan ⎛sin−1 ⎛ ⎞ − sin−1 ⎛ ⎞ ⎞, ⎝a⎠ ⎝ a ⎠⎠ ⎝ ⎜
⎟
(3)
and the x-coordinate of 4th order PI is given by
λ λ x 4th = x G2G1 + |z O| tan ⎛sin−1 ⎛ ⎞ + sin−1 ⎛ ⎞ ⎞, ⎝a⎠ ⎝ a ⎠⎠ ⎝ ⎜
⎟
(4)
where xG2G1 in Eqs. (3) and (4) is x-coordinate of the intersection of the orange line and DG1 and it is given by
λ x G2G1 = x 0th + d2 tan ⎛sin−1 ⎛ ⎞ ⎞. ⎝ a ⎠⎠ ⎝ ⎜
⎟
(5)
DDG generates PIs in the space where the object is located. In Fig. 1, although the rays that reach the imaging plane seem to come from PIs, only the rays that emanate from the point object are real. Thus, the parallax angle (PA) of the object corresponding to each 2
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 2. PAs of a point object, Ø1, and Ø3, corresponds to the 1st and 3rd order PIs, respectively.
PI can be explained by analyzing the relationship between the virtual ray from the PI to the optical center of the capturing device and the light ray from the object. Figs. 2 and 3 show the geometric relationship between the positions of the PIs generated by the DDG, the chief ray path of the point object and the virtual ray path of the PIs to explain the PA of each PI. The virtual rays from the 1st and 3rd order PIs to the optical center of the imaging lens meet the DG1 at point G1st and the DG2 at point G2nd, respectively, as shown in Fig. 2. At the points G1st and G2nd, the paths of the real rays from the point object change and are directed to the optical center of the imaging lens. Accordingly, the PA Ø1 of the point object corresponding to the 1st order PI is given by
ϕ1 = cos−1
⎞ ⎛ −z O , ⎜ (x − G )2 + z 2 ⎟ 0th 1st O ⎠ ⎝
(6)
where G1st in Eq. (6) is given by
d1 + d2 ⎞ x1st . G1st = ⎛ ⎝ d1 + d2 − z O ⎠ ⎜
The PA Ø
3
⎟
(7)
of the point object corresponding to the 3rd order PI is given by
d2 − z O ⎛ ϕ3 = cos−1 ⎜ 2 2 ( G − x 3 rd 0th ) + (d2 − z O ) ⎝
⎞ ⎟, ⎠
(8)
where G3rd in Eq. (8) is given by
d1 ⎞ x3rd . G3rd = ⎛ d d + 2 − zO ⎠ ⎝ 1 ⎜
⎟
(9)
Fig. 3. PAs of a point object, Ø2, and Ø4, corresponds to the 2nd and 4th order PIs, respectively. 3
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
As we can see from Figs. 2 and 3, the 1st and 3rd PIs in Fig. 2 contain the parallax information based on the real object (0th), while the 2nd and 4th PIs in Fig. 3 contain the parallax information based on the virtual object (3rd). As described in Fig. 3, the virtual rays from the 2nd and 4th order PIs to the optical center of the imaging lens meet the DG1 at the points G2nd and G4th, respectively. At the points G2nd and G4th, the paths of the real rays from the virtual object change and are directed to the optical center of the imaging lens. Therefore, the PA Ø 2 of the point object corresponding to the 2nd order PI is given by
ϕ2 = cos−1
⎛ −x3rd (G2nd − x3rd ) − z O (d1 + d2 − z O ) ⎜ (G − x )2 + z 2 x 2 + (d + d − z )2 2nd 3rd 1 2 O O 3rd ⎝
⎞ , ⎟ ⎠
(10)
where G2nd in Eq. (10) is given by
d1 + d2 ⎞ x2nd . G2nd = ⎛ ⎝ d1 + d2 − z O ⎠ ⎜
The PA Ø
4
⎟
(11)
of the point object corresponding to the 4th order PI is given by
ϕ4 = cos−1
⎛ −x3rd (G4th − x3rd ) − z O (d1 + d2 − z O ) ⎜ (G − x )2 + z 2 x 2 + (d + d − z )2 4th 3rd 1 2 O O 3rd ⎝
⎞ , ⎟ ⎠
(12)
where G4th in Eq. (12) is given by
d1 + d2 ⎞ x 4th . G4th = ⎛ ⎝ d1 + d2 − z O ⎠ ⎜
⎟
(13)
The parallax information of the 2nd and 4th order PIs is formed on the basis of the 3rd order PI. Meanwhile, the parallax information of the 3rd order PI is formed based on the point object (0th). Therefore, the viewing angles of the 2nd and 4th order PIs based on the point object can be obtained by subtracting Ø 2 from the Ø 3, Ø 3 - Ø 2, and by adding Ø 4 from Ø 3, Ø 3 + Ø 4, respectively. Fig. 4 shows the graph illustrating the change of the x-coordinates of PIs according to the distance (d2) between DG1 and DG2. The x-coordinate of each PI is calculated based on Eqs. (1)–(5). In the calculation, the spatial resolution of DG1 and DG2 are both equal to 500 lines/mm. The position of the point object and the imaging lens are fixed at (0, –100 mm) and (0, 300 mm) along the x and z-axes, respectively. The wavelength of the light source is 532 nm. The position of DG1 is fixed at the origin, and the position of DG2 (d2) is changed from 0 mm to 300 mm. In Fig. 4, the x-coordinates of the 0th and ± 1st PIs are constant because the position of DG1 is fixed with respect to the position of the object. In contrast, the x-coordinates of the ± 2nd, ± 3rd, and ± 4th order PIs generated by DG2 increase as DG2 moves away from DG1. Where the distance between the diffraction gratings (DGs) is zero, the ± 2nd and ± 3rd PIs are overlapped on the 0th and ± 1st PIs, respectively. The ± 2nd PIs overlap the ± 1st PIs once more at a distance of 100 mm between the DGs. As the distance between DGs increases, the PIs are divided into three groups as shown in Fig. 4. Fig. 5 is the graph showing the change of the PAs of PIs according to the distance between the DG1 and DG2. The PA of each PI is calculated based on Eqs. (6)–(13). The values of the optical variables used in the calculation are the same as those used in Fig. 4. Fig. 5(a) and (b) show the variation of the PA for all orders of PIs and PIs in each group, respectively. In Fig. 5, since DG1 is fixed at the origin, the PAs of PIs in the 0th group are unchanged. On the other hand, since PIs in the ± 1st group are generated by DG2, PAs decrease in absolute value as the distance between DGs increases. However, in fact, the ± 2nd PAs in the ± 1st group shows that the absolute value of the parallax angle increases as the distance between DG1 and DG2 increases. In Fig. 5, the plus and minus PA indicate the observation on the right and left sides of the object, respectively. In Fig. 5(a), the PAs of the nine PIs generated by DDG imaging are changed into the rules that are not related to the order of PIs. Because considering the positions and PAs of the PIs belonging to each group in Figs. 4 and 5, each group has the same characteristics as those generated from the independent SDG imaging system at different points of view of the object. More clearly, the PIs of each group have the PAs observed from the left and right sides of the PI centered on the group. This means that the PI located at the center
Fig. 4. x-coordinates of PIs according to the distance between DG1 and DG2. 4
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 5. (a) PAs of PIs according to the distance between DG1 and DG2, (b) Variation of PAs by a group.
of each PI group acts as a virtual object. Therefore, the PAs of the PIs generated by DDG cannot be sequential. These characteristics of PIs in DDG imaging are expected to be controlled through the addition and analysis of new optical elements. However, one realistic alternative is to physically recombine PIs acquired under certain conditions. 3. Experiment To verify the theoretical analysis for the positions of PIs, we perform optical experiments to observe the variation of the imaging characteristics of the parallax image according to the distance change between DG1 and DG2. In the experiment, DG1 is 100 mm away from the object and DG2 (d2) moves from the location of DG1 to 400 mm away from the object in order to maintain the same conditions as theoretical analysis. A dice is used as a 3-D test object, and the dimension of the object is 11 mm × 11 mm × 11 mm. Diode laser (λ = 532 nm) is used for illuminating the object. Fig. 6(a) shows the experimental setup. Fig. 6(b) shows PIs generated by a 1-D DDG imaging system according to the distance (d2) between DG1 and DG2. In the experiment, the size of the diffraction grating is act as an aperture to limit the field of view of the camera, so the ± 4th PIs gradually appear as DG2 approaches the camera. The numbers on the left in Fig. 6(b) represent the distance between DG1 and DG2. As in the analysis in Fig. 4, seven PIs from the 1st-order to the 3rd-order are overlapped at 100 mm and become five PIs. Here, as in the analysis in Fig. 5, the PAs of overlapping PIs are different from each other. However, due to differences in intensity of PIs, the effect of overlapping images on the PIs is minimal. As the analysis in Fig. 4, the PIs in Fig. 6 are divided into three groups and can be identified from the 150 mm due to the overlap of PIs. To verify the theoretical analysis for the PAs of PIs, experiments were performed to compare the images of the object photographed at the theoretically derived angle with the PIs generated by DDG imaging. In the experiments, each pair of DG1 and DG2 is crossed at the right angles for generation of 2-D PIA. ‘A postman pushing a cart’ is used as a 3-D test object, and the dimension of the object is 15 mm × 20 mm × 25 mm. From the analysis in Figs. 4 and 5, it can be seen that the PIs do not overlap each other and the parallax angles are distributed more or less evenly at the distances (d2) of 175 mm–200 mm between DG1 and DG2. Based on this, DG1 and DG2 were set to be 100 mm and 280 mm away from the object, respectively. The 3-D test object is placed on the center of the rotating stage and the PIA is acquired using the DDG imaging. Then, in order to compare the PAs of the PIs with that of the real object, the images of the 3-D object are captured by rotating the stage without DGs. Fig. 7(a)–(c) show the 3 × 9 PIs captured by DDG imaging method, the PIs of the area indicated by the yellow dotted lined in Fig. 7(a) and its theoretical PAs, and the images of the 3-D test object captured by a camera at the theoretical PAs, respectively. Corresponding to the PAs, the PIs in Fig. 7(b) and the object images in Fig. 7(c) are the 5
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 6. (a) Experimental setup, (b) PIs generated by 1-D DDG imaging according to the distance (d2) between DG1 and DG2.
Fig. 7. Comparison of PAs between PIs and object images. (a) 3 × 9 PIA captured by the 2-D DDG imaging system. (b) PIs and its theoretical PAs. (c) Object images and its PAs.
6
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 8. PI recombination. (a) Original PIs generated by the 1-D DDG imaging, (b) recombined PIs.
same or almost similar. As described above, it can be seen that the PIs of each group have PAs observed from the left and right sides of the PI positioned at the center of the group. Considering this from the point of view of an SDG imaging, the PI located at the center of each group acts as a virtual object. The experimental result verifies the theoretically derived PAs. As mentioned above, one of the realistic alternatives to the generation of PIs with sequential PAs in DDG imaging is to physically recombine PIs acquired under certain conditions. In order to confirm this possibility, PI recombination experiment was performed based on the theoretical analysis and the experimental result in Fig. 7. In this experiment, DG1 and DG2 were set to be 100 mm and 285 mm (d2 = 185 mm) away from the object, respectively. Fig. 8(a) shows the original PIs generated by 1-D DDG imaging, the order of PIs and the theoretical PAs. As discussed, the PAs do not coincide with the order of the PIs. The recombination of the PI reorders the original PIs according to the order of the theoretical PA while maintaining the 0th and ± 4th positions. Fig. 8(b) shows the recombined PIs based on theoretically derived PAs. As can be seen from the results in Fig. 8(b), the physical recombination of the PIs can be applied as an alternative to the sequential PAs of the PIs. 4. Conclusion In conclusion, we have geometrically analyzed the imaging characteristics of a DDG imaging system such as the position of PI and PA of each PI. The distance between PIs and the PA of each PI are related to the distance between DGs. The PI of DDG imaging can be divided into three groups considering the position and PA of PIs. The PAs of the PIs belonging to each group are formed based on the PIs located at the center of the group. The reason why the PA of PIs cannot be sequential in the DDG imaging is analyzed and explained. The possibility of PIs recombination has been confirmed as a practical method for generating PIs with sequential PAs. These analyses can be used to optimize and design system parameters for DDG imaging system. Acknowledgment This research was supported by the National Research Foundation of Korea (NRF-2019R1F1A1042989). References [1] J.-H. Park, J. Kim, Y. Kim, B. Lee, Resolution-enhanced three-dimension/two-dimension convertible display based on integral imaging, Opt. Express 13 (2005) 1875–1884. [2] R. Martínez-Cuenca, G. Saavedra, A. Pons, B. Javidi, M. Martínez-Corral, Facet braiding: a fundamental problem in integral imaging, Opt. Lett. 32 (2007) 1078–1080. [3] D.-H. Shin, H. Yoo, Image quality enhancement in 3D computational integral imaging by use of interpolation methods, Opt. Express 15 (2007) 12039–12049. [4] D.-H. Shin, B.-G. Lee, J.-J. Lee, Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging, Opt. Express 16 (2008) 16294–16304. [5] Y. Piao, D.-H. Shin, E.-S. Kim, Robust image encryption by combined use of integral imaging and pixel scrambling techniques, Opt. Lasers Eng. 47 (11) (2009) 1273–1281. [6] J.-Y. Jang, H.-S. Lee, S. Cha, S.-H. Shin, Viewing angle enhanced integral imaging display by using a high refractive index medium, Appl. Opt. 50 (7) (2011) B71–B76. [7] X. Xiao, B. Javidi, 3D photon counting integral imaging with unknown sensor positions, J. Opt. Soc. Am. A 29 (2012) 767–771. [8] X. Xiao, B. Javidi, Manuel Martinez-Corral, Adrian Stern, Advances in three-dimensional integral imaging: sensing, display, and applications [Invited], Appl. Opt. 52 (2013) 546–560. [9] J.-Y. Jang, D. Shin, E.-S. Kim, Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-
7
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
imaging, Opt. Express 22 (2) (2014) 1533–1550. [10] J.-Y. Jang, M. Cho, Orthoscopic real image reconstruction in integral imaging by rotating an elemental image based on the reference point of object space, Appl. Opt. 54 (18) (2015) 5877–5881. [11] H. Yoo, J.-Y. Jang, Intermediate elemental image reconstruction for refocused three-dimensional images in integral imaging by convolution with δ-function sequences, Opt. Lasers Eng. 97 (2017) 93–99. [12] J.-I. Ser, J.-Y. Jang, S. Cha, S.-H. Shin, Applicability of diffraction grating to parallax image array generation in integral imaging, Appl. Opt. 49 (13) (2010) 2429–2433. [13] J.-Y. Jang, J.-I. Ser, E.-S. Kim, Wave-optical analysis of parallax-image generation based on multiple diffraction gratings, Opt. Lett. 38 (11) (2013) 1835–1837.
8
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.com/locate/ijleo
Original research article
Analysis of imaging characteristics of parallax images in double diffraction grating imaging Hoon Yooa, Jung-Un Jangb,*, Jae-Young Jangb,* a b
Department of Electronics Engineering, Sangmyung University, 20 Hongjimoon-2gil, Jongno-gu, Seoul, South Korea Department of Optometry, Eulji University, 553, Sanseong-daero, Sujeong-gu, Seongnam-si, Gyonggi-do, South Korea
A R T IC LE I N F O
ABS TRA CT
Keywords: Diffraction grating imaging Parallax image array Three-dimensional imaging
The relation between the imaging characteristics of parallax images (PIs) and the system parameters of the double diffraction grating (DDG) imaging system is analyzed. DDG imaging method may produce parallax images (PIs) efficiently. However, a geometrical analysis is required to improve the applicability of the imaging system. The system parameters considered include the wavelength of light source, the spatial resolution of the diffraction grating, and positions of diffraction gratings and an imaging lens. The imaging characteristics of the DDG imaging defined and described in terms of positions and parallax angles of PIs. The ray-optical analysis is performed for the explanation of the imaging characteristics.
1. Introduction A parallax image array (PIA) has played an important role in the fields of three-dimensional (3-D) display, 3-D data processing, 3D profiling, and so on [1–11]. Thus, acquiring PIA data is an essential part of 3-D imaging technology. Recently, a method of using a single diffraction grating (SDG) for generating a PIA was proposed [12]. Compared with a conventional lens array and a camera array, this method has the advantages of simple optical systems, low aberration, and no lens distortion. Thus, it can be one of the promising techniques in 3-D imaging. However, it has drawbacks such that the numbers of parallax images (PIs) and viewpoints are limited by the physical size and higher-order diffraction efficiency of the diffraction grating (DG) in use. Accordingly, a method of using double diffraction gratings (DDG) has been studied as a way to increase the number of PIs and viewpoints while taking advantage of the DG in PIA generation [13]. In this method, the feasibility of DDG for generating a PIA is confirmed by wave-optical analysis and optical experiments. The total number of PIs by DDG may be increased by the square of the total number of PIs by the SDG. However, the lack of analysis of system parameters and imaging characteristics of PIs, especially PIs with non-sequential parallax angles (PAs), reduces the applicability of DDG imaging systems. Therefore, geometrical analysis and explanation of the imaging characteristics for the generated PIs are needed in order to improve the applicability of the DDG imaging system. In this letter, we analyze the imaging characteristics of the PIs according to the system parameters of the DDG imaging system. By analyzing imaging characteristics, we intend to improve the understanding and applicability of DDG imaging. The system parameters considered include the wavelength of light source, a spatial resolution of diffraction, and positions of diffraction gratings and an imaging lens. In order to investigate the imaging characteristics, we first derive the equations for imaging positions of the individual PIs generated by DDG. Next, the parallax angles corresponding to the individual PIs are derived in consideration of the geometrical relationship between the imaging positions of the individual PIs and a camera (imaging lens). The imaging characteristics are
⁎
Corresponding authors. E-mail addresses: [email protected] (J.-U. Jang), [email protected] (J.-Y. Jang).
https://doi.org/10.1016/j.ijleo.2019.163826 Received 5 September 2019; Accepted 19 November 2019 0030-4026/ © 2019 Elsevier GmbH. All rights reserved.
Please cite this article as: Hoon Yoo, Jung-Un Jang and Jae-Young Jang, Optik - International Journal for Light and Electron Optics, https://doi.org/10.1016/j.ijleo.2019.163826
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 1. Geometrical relationships of a point object, PIs, and DDG.
analyzed based on the derived equations and the validity of the analysis is verified through optical experiments. Since the DDG imaging system can be easily expanded from one-dimensional (1-D) to two-dimensional (2-D) in the PIA generation, the analysis is performed only for the 1-D PIA generation. 2. Geometrical analysis of parallax images in diffraction grating imaging The scattered light from an object is diffracted by DDG located on the optical path and then generates virtual images on the back of the DDG where the object is located. These virtual images are PIs having different viewpoints with respect to the object and are stored as a PIA by a capturing device such as a camera. The imaging depth and the size of the PIs are the same as the depth and size of the object. Fig. 1 describes the geometrical relationship between a point object and its PIs in DDG imaging. Here, let us assume that the first diffraction grating DG1 is located along the x-axis and the point object is located at (x0th, zO) in the x-z plane. Therefore, the zcoordinates of all PIs are zO. The second diffraction grating DG2 and imaging lens are separated from the origin by d2 and d1 + d2, respectively. For the convenience of explanation, the minus orders of PIs are omitted in Fig. 1. In Fig. 1, the 1st (red line) and 3rd (orange line) order PIs are generated due to the 1st order diffraction of the point object by DG1 and DG2, respectively. The point object x0th serves as the 0th order PI. The diffraction angles θ is given by θ = sin−1(λ/a) for both DG1 and DG2, where λ is the wavelength of a light source, and a is aperture width of DGs. Taking the diffraction order into account, x-coordinate of 1st order PI is given by
λ x1st = x 0th + |z O| tan ⎛sin−1 ⎛ ⎞ ⎞. ⎝ a ⎠⎠ ⎝ ⎜
⎟
(1)
The x-coordinate of 3rd order PI generated by DG2 is given by
λ x3rd = x 0th + (|z O| + d2) tan ⎛sin−1 ⎛ ⎞ ⎞. ⎝ a ⎠⎠ ⎝ ⎜
⎟
(2)
The 1st order diffracted ray (orange line) by DG2 is diffracted once more by DG1. As a result, the 2nd and 4th orders PIs are generated. The 2nd (green line) and 4th (cyan line) order PIs are obtained from -1st and 1st order diffractions of the 3rd order PI caused by DG1, respectively. Here, the 3rd order PI serves as a virtual object. The x-coordinate of 2nd order PI is given by
λ λ x2nd = x G2G1 + |z O| tan ⎛sin−1 ⎛ ⎞ − sin−1 ⎛ ⎞ ⎞, ⎝a⎠ ⎝ a ⎠⎠ ⎝ ⎜
⎟
(3)
and the x-coordinate of 4th order PI is given by
λ λ x 4th = x G2G1 + |z O| tan ⎛sin−1 ⎛ ⎞ + sin−1 ⎛ ⎞ ⎞, ⎝a⎠ ⎝ a ⎠⎠ ⎝ ⎜
⎟
(4)
where xG2G1 in Eqs. (3) and (4) is x-coordinate of the intersection of the orange line and DG1 and it is given by
λ x G2G1 = x 0th + d2 tan ⎛sin−1 ⎛ ⎞ ⎞. ⎝ a ⎠⎠ ⎝ ⎜
⎟
(5)
DDG generates PIs in the space where the object is located. In Fig. 1, although the rays that reach the imaging plane seem to come from PIs, only the rays that emanate from the point object are real. Thus, the parallax angle (PA) of the object corresponding to each 2
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 2. PAs of a point object, Ø1, and Ø3, corresponds to the 1st and 3rd order PIs, respectively.
PI can be explained by analyzing the relationship between the virtual ray from the PI to the optical center of the capturing device and the light ray from the object. Figs. 2 and 3 show the geometric relationship between the positions of the PIs generated by the DDG, the chief ray path of the point object and the virtual ray path of the PIs to explain the PA of each PI. The virtual rays from the 1st and 3rd order PIs to the optical center of the imaging lens meet the DG1 at point G1st and the DG2 at point G2nd, respectively, as shown in Fig. 2. At the points G1st and G2nd, the paths of the real rays from the point object change and are directed to the optical center of the imaging lens. Accordingly, the PA Ø1 of the point object corresponding to the 1st order PI is given by
ϕ1 = cos−1
⎞ ⎛ −z O , ⎜ (x − G )2 + z 2 ⎟ 0th 1st O ⎠ ⎝
(6)
where G1st in Eq. (6) is given by
d1 + d2 ⎞ x1st . G1st = ⎛ ⎝ d1 + d2 − z O ⎠ ⎜
The PA Ø
3
⎟
(7)
of the point object corresponding to the 3rd order PI is given by
d2 − z O ⎛ ϕ3 = cos−1 ⎜ 2 2 ( G − x 3 rd 0th ) + (d2 − z O ) ⎝
⎞ ⎟, ⎠
(8)
where G3rd in Eq. (8) is given by
d1 ⎞ x3rd . G3rd = ⎛ d d + 2 − zO ⎠ ⎝ 1 ⎜
⎟
(9)
Fig. 3. PAs of a point object, Ø2, and Ø4, corresponds to the 2nd and 4th order PIs, respectively. 3
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
As we can see from Figs. 2 and 3, the 1st and 3rd PIs in Fig. 2 contain the parallax information based on the real object (0th), while the 2nd and 4th PIs in Fig. 3 contain the parallax information based on the virtual object (3rd). As described in Fig. 3, the virtual rays from the 2nd and 4th order PIs to the optical center of the imaging lens meet the DG1 at the points G2nd and G4th, respectively. At the points G2nd and G4th, the paths of the real rays from the virtual object change and are directed to the optical center of the imaging lens. Therefore, the PA Ø 2 of the point object corresponding to the 2nd order PI is given by
ϕ2 = cos−1
⎛ −x3rd (G2nd − x3rd ) − z O (d1 + d2 − z O ) ⎜ (G − x )2 + z 2 x 2 + (d + d − z )2 2nd 3rd 1 2 O O 3rd ⎝
⎞ , ⎟ ⎠
(10)
where G2nd in Eq. (10) is given by
d1 + d2 ⎞ x2nd . G2nd = ⎛ ⎝ d1 + d2 − z O ⎠ ⎜
The PA Ø
4
⎟
(11)
of the point object corresponding to the 4th order PI is given by
ϕ4 = cos−1
⎛ −x3rd (G4th − x3rd ) − z O (d1 + d2 − z O ) ⎜ (G − x )2 + z 2 x 2 + (d + d − z )2 4th 3rd 1 2 O O 3rd ⎝
⎞ , ⎟ ⎠
(12)
where G4th in Eq. (12) is given by
d1 + d2 ⎞ x 4th . G4th = ⎛ ⎝ d1 + d2 − z O ⎠ ⎜
⎟
(13)
The parallax information of the 2nd and 4th order PIs is formed on the basis of the 3rd order PI. Meanwhile, the parallax information of the 3rd order PI is formed based on the point object (0th). Therefore, the viewing angles of the 2nd and 4th order PIs based on the point object can be obtained by subtracting Ø 2 from the Ø 3, Ø 3 - Ø 2, and by adding Ø 4 from Ø 3, Ø 3 + Ø 4, respectively. Fig. 4 shows the graph illustrating the change of the x-coordinates of PIs according to the distance (d2) between DG1 and DG2. The x-coordinate of each PI is calculated based on Eqs. (1)–(5). In the calculation, the spatial resolution of DG1 and DG2 are both equal to 500 lines/mm. The position of the point object and the imaging lens are fixed at (0, –100 mm) and (0, 300 mm) along the x and z-axes, respectively. The wavelength of the light source is 532 nm. The position of DG1 is fixed at the origin, and the position of DG2 (d2) is changed from 0 mm to 300 mm. In Fig. 4, the x-coordinates of the 0th and ± 1st PIs are constant because the position of DG1 is fixed with respect to the position of the object. In contrast, the x-coordinates of the ± 2nd, ± 3rd, and ± 4th order PIs generated by DG2 increase as DG2 moves away from DG1. Where the distance between the diffraction gratings (DGs) is zero, the ± 2nd and ± 3rd PIs are overlapped on the 0th and ± 1st PIs, respectively. The ± 2nd PIs overlap the ± 1st PIs once more at a distance of 100 mm between the DGs. As the distance between DGs increases, the PIs are divided into three groups as shown in Fig. 4. Fig. 5 is the graph showing the change of the PAs of PIs according to the distance between the DG1 and DG2. The PA of each PI is calculated based on Eqs. (6)–(13). The values of the optical variables used in the calculation are the same as those used in Fig. 4. Fig. 5(a) and (b) show the variation of the PA for all orders of PIs and PIs in each group, respectively. In Fig. 5, since DG1 is fixed at the origin, the PAs of PIs in the 0th group are unchanged. On the other hand, since PIs in the ± 1st group are generated by DG2, PAs decrease in absolute value as the distance between DGs increases. However, in fact, the ± 2nd PAs in the ± 1st group shows that the absolute value of the parallax angle increases as the distance between DG1 and DG2 increases. In Fig. 5, the plus and minus PA indicate the observation on the right and left sides of the object, respectively. In Fig. 5(a), the PAs of the nine PIs generated by DDG imaging are changed into the rules that are not related to the order of PIs. Because considering the positions and PAs of the PIs belonging to each group in Figs. 4 and 5, each group has the same characteristics as those generated from the independent SDG imaging system at different points of view of the object. More clearly, the PIs of each group have the PAs observed from the left and right sides of the PI centered on the group. This means that the PI located at the center
Fig. 4. x-coordinates of PIs according to the distance between DG1 and DG2. 4
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 5. (a) PAs of PIs according to the distance between DG1 and DG2, (b) Variation of PAs by a group.
of each PI group acts as a virtual object. Therefore, the PAs of the PIs generated by DDG cannot be sequential. These characteristics of PIs in DDG imaging are expected to be controlled through the addition and analysis of new optical elements. However, one realistic alternative is to physically recombine PIs acquired under certain conditions. 3. Experiment To verify the theoretical analysis for the positions of PIs, we perform optical experiments to observe the variation of the imaging characteristics of the parallax image according to the distance change between DG1 and DG2. In the experiment, DG1 is 100 mm away from the object and DG2 (d2) moves from the location of DG1 to 400 mm away from the object in order to maintain the same conditions as theoretical analysis. A dice is used as a 3-D test object, and the dimension of the object is 11 mm × 11 mm × 11 mm. Diode laser (λ = 532 nm) is used for illuminating the object. Fig. 6(a) shows the experimental setup. Fig. 6(b) shows PIs generated by a 1-D DDG imaging system according to the distance (d2) between DG1 and DG2. In the experiment, the size of the diffraction grating is act as an aperture to limit the field of view of the camera, so the ± 4th PIs gradually appear as DG2 approaches the camera. The numbers on the left in Fig. 6(b) represent the distance between DG1 and DG2. As in the analysis in Fig. 4, seven PIs from the 1st-order to the 3rd-order are overlapped at 100 mm and become five PIs. Here, as in the analysis in Fig. 5, the PAs of overlapping PIs are different from each other. However, due to differences in intensity of PIs, the effect of overlapping images on the PIs is minimal. As the analysis in Fig. 4, the PIs in Fig. 6 are divided into three groups and can be identified from the 150 mm due to the overlap of PIs. To verify the theoretical analysis for the PAs of PIs, experiments were performed to compare the images of the object photographed at the theoretically derived angle with the PIs generated by DDG imaging. In the experiments, each pair of DG1 and DG2 is crossed at the right angles for generation of 2-D PIA. ‘A postman pushing a cart’ is used as a 3-D test object, and the dimension of the object is 15 mm × 20 mm × 25 mm. From the analysis in Figs. 4 and 5, it can be seen that the PIs do not overlap each other and the parallax angles are distributed more or less evenly at the distances (d2) of 175 mm–200 mm between DG1 and DG2. Based on this, DG1 and DG2 were set to be 100 mm and 280 mm away from the object, respectively. The 3-D test object is placed on the center of the rotating stage and the PIA is acquired using the DDG imaging. Then, in order to compare the PAs of the PIs with that of the real object, the images of the 3-D object are captured by rotating the stage without DGs. Fig. 7(a)–(c) show the 3 × 9 PIs captured by DDG imaging method, the PIs of the area indicated by the yellow dotted lined in Fig. 7(a) and its theoretical PAs, and the images of the 3-D test object captured by a camera at the theoretical PAs, respectively. Corresponding to the PAs, the PIs in Fig. 7(b) and the object images in Fig. 7(c) are the 5
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 6. (a) Experimental setup, (b) PIs generated by 1-D DDG imaging according to the distance (d2) between DG1 and DG2.
Fig. 7. Comparison of PAs between PIs and object images. (a) 3 × 9 PIA captured by the 2-D DDG imaging system. (b) PIs and its theoretical PAs. (c) Object images and its PAs.
6
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
Fig. 8. PI recombination. (a) Original PIs generated by the 1-D DDG imaging, (b) recombined PIs.
same or almost similar. As described above, it can be seen that the PIs of each group have PAs observed from the left and right sides of the PI positioned at the center of the group. Considering this from the point of view of an SDG imaging, the PI located at the center of each group acts as a virtual object. The experimental result verifies the theoretically derived PAs. As mentioned above, one of the realistic alternatives to the generation of PIs with sequential PAs in DDG imaging is to physically recombine PIs acquired under certain conditions. In order to confirm this possibility, PI recombination experiment was performed based on the theoretical analysis and the experimental result in Fig. 7. In this experiment, DG1 and DG2 were set to be 100 mm and 285 mm (d2 = 185 mm) away from the object, respectively. Fig. 8(a) shows the original PIs generated by 1-D DDG imaging, the order of PIs and the theoretical PAs. As discussed, the PAs do not coincide with the order of the PIs. The recombination of the PI reorders the original PIs according to the order of the theoretical PA while maintaining the 0th and ± 4th positions. Fig. 8(b) shows the recombined PIs based on theoretically derived PAs. As can be seen from the results in Fig. 8(b), the physical recombination of the PIs can be applied as an alternative to the sequential PAs of the PIs. 4. Conclusion In conclusion, we have geometrically analyzed the imaging characteristics of a DDG imaging system such as the position of PI and PA of each PI. The distance between PIs and the PA of each PI are related to the distance between DGs. The PI of DDG imaging can be divided into three groups considering the position and PA of PIs. The PAs of the PIs belonging to each group are formed based on the PIs located at the center of the group. The reason why the PA of PIs cannot be sequential in the DDG imaging is analyzed and explained. The possibility of PIs recombination has been confirmed as a practical method for generating PIs with sequential PAs. These analyses can be used to optimize and design system parameters for DDG imaging system. Acknowledgment This research was supported by the National Research Foundation of Korea (NRF-2019R1F1A1042989). References [1] J.-H. Park, J. Kim, Y. Kim, B. Lee, Resolution-enhanced three-dimension/two-dimension convertible display based on integral imaging, Opt. Express 13 (2005) 1875–1884. [2] R. Martínez-Cuenca, G. Saavedra, A. Pons, B. Javidi, M. Martínez-Corral, Facet braiding: a fundamental problem in integral imaging, Opt. Lett. 32 (2007) 1078–1080. [3] D.-H. Shin, H. Yoo, Image quality enhancement in 3D computational integral imaging by use of interpolation methods, Opt. Express 15 (2007) 12039–12049. [4] D.-H. Shin, B.-G. Lee, J.-J. Lee, Occlusion removal method of partially occluded 3D object using sub-image block matching in computational integral imaging, Opt. Express 16 (2008) 16294–16304. [5] Y. Piao, D.-H. Shin, E.-S. Kim, Robust image encryption by combined use of integral imaging and pixel scrambling techniques, Opt. Lasers Eng. 47 (11) (2009) 1273–1281. [6] J.-Y. Jang, H.-S. Lee, S. Cha, S.-H. Shin, Viewing angle enhanced integral imaging display by using a high refractive index medium, Appl. Opt. 50 (7) (2011) B71–B76. [7] X. Xiao, B. Javidi, 3D photon counting integral imaging with unknown sensor positions, J. Opt. Soc. Am. A 29 (2012) 767–771. [8] X. Xiao, B. Javidi, Manuel Martinez-Corral, Adrian Stern, Advances in three-dimensional integral imaging: sensing, display, and applications [Invited], Appl. Opt. 52 (2013) 546–560. [9] J.-Y. Jang, D. Shin, E.-S. Kim, Optical three-dimensional refocusing from elemental images based on a sifting property of the periodic δ-function array in integral-
7
Optik - International Journal for Light and Electron Optics xxx (xxxx) xxxx
H. Yoo, et al.
imaging, Opt. Express 22 (2) (2014) 1533–1550. [10] J.-Y. Jang, M. Cho, Orthoscopic real image reconstruction in integral imaging by rotating an elemental image based on the reference point of object space, Appl. Opt. 54 (18) (2015) 5877–5881. [11] H. Yoo, J.-Y. Jang, Intermediate elemental image reconstruction for refocused three-dimensional images in integral imaging by convolution with δ-function sequences, Opt. Lasers Eng. 97 (2017) 93–99. [12] J.-I. Ser, J.-Y. Jang, S. Cha, S.-H. Shin, Applicability of diffraction grating to parallax image array generation in integral imaging, Appl. Opt. 49 (13) (2010) 2429–2433. [13] J.-Y. Jang, J.-I. Ser, E.-S. Kim, Wave-optical analysis of parallax-image generation based on multiple diffraction gratings, Opt. Lett. 38 (11) (2013) 1835–1837.
8