- Email: [email protected]
An accelerated hologram calculation using the wavefront recording plane method and wavelet transform
Optics Communications 393 (2017) 107–112
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
An accelerated hologram calculation using the wavefront recording plane method and wavelet transform
MARK
⁎
Daisuke Araia, Tomoyoshi Shimobabaa, , Takashi Nishitsujia, Takashi Kakuea, Nobuyuki Masudab, Tomoyoshi Itoa a b
Chiba University, Graduate School of Engineering, 1–33 Yayoi–cho, Inage–ku, Chiba 263-8522, Japan Department of Applied Electronics, Tokyo University of Science, 6–3–1 Niijuku, Katsushika–ku, Tokyo 125-8585, Japan
A R T I C L E I N F O
A BS T RAC T
Keywords: Computer-generated hologram 3D display Electroholography Hologram Holography
Fast hologram calculation methods are critical in real-time holography applications such as three-dimensional (3D) displays. We recently proposed a wavelet transform-based hologram calculation called WASABI. Even though WASABI can decrease the calculation time of a hologram from a point cloud, it increases the calculation time with increasing propagation distance. We also proposed a wavefront recoding plane (WRP) method. This is a two-step fast hologram calculation in which the first step calculates the superposition of light waves emitted from a point cloud in a virtual plane, and the second step performs a diffraction calculation from the virtual plane to the hologram plane. A drawback of the WRP method is in the first step when the point cloud has a large number of object points and/or a long distribution in the depth direction. In this paper, we propose a method combining WASABI and the WRP method in which the drawbacks of each can be complementarily solved. Using a consumer CPU, the proposed method succeeded in performing a hologram calculation with 2048 × 2048 pixels from a 3D object with one million points in approximately 0.4 s.
1. Introduction Due to the heavy calculation amounts of holograms, fast hologram calculation methods are essential in real-time holography applications such as three-dimensional (3D) displays [1]. Multiple algorithms have been proposed: point cloud methods [2–8], polygon methods [9–11], holographic stereograms [12–15], and RGB-D [16,17]. The details of these methods can be found in Refs. [18,19]. In point cloud methods, a 3D object is expressed as the aggregation of object points, and the light waves emitted from these object points are superposed in the hologram plane. The superposition is the most time-consuming part of the hologram calculation. To accelerate the superposition, we recently proposed a wavelet transform-based hologram calculation, which is referred to as the WAvelet ShrinkAge-Based superpositIon (WASABI) [20]. WASABI accelerates a hologram calculation by representing the object light with a few wavelet coefficients. Even though the WASABI can considerably decrease the calculation time of a hologram, it increases the calculation time with increasing propagation distance. We also proposed a wavefront recoding plane (WRP) method [21]. Subsequently, various improved methods for the WRP method have been proposed [22–35]. The WRP method is a two-step fast hologram
⁎
calculation in which the first step calculates the superposition of light waves emitted from object points in a virtual plane, and the second step performs diffraction calculation from the virtual plane to the hologram plane. A drawback of the WRP method is in the first step when the point cloud has a large number of object points and/or a long distribution in the depth direction. Several methods to alleviate this problem have been proposed. For example, multiple WRP methods introduce multiple virtual planes to split the long distribution [25,26,28,30], and the placement of the virtual planes can be optimized [35]. The tilted WRP method [27] decreases the calculation cost of the first step by using a tilted diffraction calculation [9], and the stretched WRP method [31] decreases the calculation cost using a non-uniform sampled diffraction calculation [36]. More recently, a WRP method using the sparse fast Fourier transform (FFT) has been proposed to accelerate the second step in the WRP method [33]. In this paper, we propose a method combining WASABI and the WRP method in which the drawbacks of each can be complementarily solved. That is, the calculation cost of the first step in the WRP method is accelerated using WASABI, resulting in a decrease in the total calculation time. Using a consumer CPU, the proposed method succeeded in completing a hologram calculation with 2048 × 2048
Corresponding author. E-mail address: [email protected] (T. Shimobaba).
http://dx.doi.org/10.1016/j.optcom.2017.02.038 Received 19 January 2017; Received in revised form 13 February 2017; Accepted 15 February 2017 0030-4018/ © 2017 Elsevier B.V. All rights reserved.
Optics Communications 393 (2017) 107–112
D. Arai et al.
Fig. 1. Hologram calculation for (a) a conventional hologram calculation and (b) the proposed method.
Fig. 2. Reconstructed images of a dinosaur composed of 11,646 object points at the two different distances of 0.1 m and 0.8 m.
object points. The light waves emitted from these object points are superimposed in the hologram plane using
pixels from a 3D object with one million points in approximately 0.4 s Section 2 describes the proposed method, and Section 3 shows the effectiveness of the proposed method. Section 4 concludes this study.
N
u (xh , yh) =
2. Proposed method
j
The proposed method combines WASABI and the WRP method to further accelerate the hologram calculation at long propagation distances. Fig. 1(a) shows a conventional hologram calculation from the 3D
⎛
⎞ 2π ⎟ rhj ⎟ = ⎝ λ ⎠
∑ aj exp ⎜⎜i
N
∑ aj u zj (xh − xj , yh − yj ), j
(1)
where i is the imaginary unit number, N is the number of object points, (xh , yh) are the coordinates of the hologram plane, (xj , yj , zj ) and aj are the coordinates and amplitude of the j-th object point, respectively, λ is the wavelength, and u zj is the point spread function (PSF) at a distance zj. 108
Optics Communications 393 (2017) 107–112
D. Arai et al.
Fig. 5. Calculation times for the merry-go-round image (95,949 object points) using the conventional method, WASABI, and WASABI+WRP at the two different distances of 0.1 m and 0.8 m.
Fig. 3. Calculation times for the dinosaur image (11,646 object points) using the conventional method, WASABI, and WASABI+WRP at the two different distances of 0.1 m and 0.8 m.
To decrease the computational complexity, look-up table methods, such as the novel look-up table (N-LUT) method [3], can decrease the calculation time. However, even when using look-up table methods, the computational complexity remains O (NW 2 ). Recently, we proposed the WASABI method [20] to decrease the computational complexity of Eq. (1). WASABI accelerates hologram calculations using the following three steps.
The computational complexity of Eq. (1) is decided by the number of object points and the average radius W of the PSFs. The average radius W is determined by
W =
1 N
N
∑ Wj = j
1 N
N
∑ zj tan θ, j
(2) 1. The PSFs u zj (xh , yh) for zj are pre-computed and then the spacedomain PSFs are transformed into wavelet-domain PSFs. The number of wavelet coefficients is reduced by selecting Nr large wavelet coefficients. 2. The reduced wavelet coefficients are superposed in the wavelet domain. 3. The inverse wavelet transform is performed to convert the superposed result in the wavelet domain to that in the space domain.
where the maximum divergence angle θ of an object point is θ = sin−1(λ /(2p )), where p is the sampling pitch of the hologram. The computational complexity of Eq. (1) is expressed as
O (NW 2 ).
(3)
It is clear that the computational complexity is increased as zj increases. The heaviest computation in Eq. (1) is the superposition.
Fig. 4. Reconstructed images of a merry-go-round composed of 95,949 object points at the two different distances of 0.1 m and 0.8 m.
109
Optics Communications 393 (2017) 107–112
D. Arai et al.
Fig. 6. Reconstructed images of a fountain composed of 978,416 object points at the two different distances of 0.1 m and 0.8 m.
O (rNW 2 ) + O (Nh2 ).
Even though WASABI can drastically decrease the computational complexity of the hologram calculation, it increases the calculation time with increasing propagation distance because the computational complexity depends on W , including zj. To further accelerate the hologram calculation at long distances, we propose a combination of WASABI and the WRP method (the WASABI +WRP method). As shown in Fig. 1(b), the WRP method introduces a virtual plane between the 3D object and the hologram plane. In the conventional WRP method [21,22], this virtual plane is calculated using Eq. (1) and the computational complexity is shown in Eq. (3). The computational cost becomes low when the virtual plane is placed near the 3D object. However, when the 3D object has a large number of object points and/or a long distribution in the depth direction, the computational complexity of the virtual plane becomes large due to Eq. (3). Instead of Eq. (1), we use WASABI to accelerate the calculation of the virtual plane; therefore, the computational complexity can be suppressed to that of Eq. (6). Subsequently, we calculate the numerical diffraction from the virtual plane to the hologram plane. For this diffraction calculation, in this paper, we used band-limited double-step Fresnel diffraction [17] because its complexity is lower than that of the conventional Fresnel diffraction and angular spectrum method [17,19]. The computational complexity of the diffraction calculation is O (Nh2 log Nh ). Therefore, the total computational complexity of the proposed method is
Fig. 7. Calculation times for the fountain image (978,416 object points) using the conventional method, WASABI, and WASABI+WRP at the two different distances of 0.1 m and 0.8 m.
In this study, the number of large wavelet coefficients, Nr, is defined as
Nr = r × 2πW 2,
(4)
where r is the selection rate of the larger wavelet coefficients. Therefore, in the second step of WASABI, the computational complexity of the superposition can be reduced to
O (NNr ) =
O (rNW 2 ).
(6)
O (rNW 2 ) + O (Nh2 ) + O (Nh2 log Nh ).
(5)
(7)
For comparison, the computational complexity of the conventional WRP method [21,22] is
If we use r = 1%, compared to the conventional method of Eq. (1), we can reduce the computational complexity of the superposition to approximately one hundredth. Subsequently, WASABI transforms the superposed result in the wavelet domain to that in the space domain using the inverse wavelet transform, whose computational complexity is only O (Nh2 ). Therefore, the total computational complexity of WASABI is
O (NW 2 ) + O (Nh2 log Nh ).
(8)
Even though the proposed method has an additional computational complexity, O (Nh2 ), for the inverse wavelet transform, the computational complexity of the proposed method is lower than that of the conventional WRP method. 110
Optics Communications 393 (2017) 107–112
D. Arai et al.
0.8 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 850 s, 27 s and 0.43 s, respectively. In the case of one million object points, WASABI+WRP shows further acceleration. Fig. 6 shows the reconstructed images of a fountain composed of 978,416 points. The reconstructed images in the center and right columns of Fig. 6 were obtained using WASABI and WASABI+WRP, respectively. Fig. 7 shows the calculation times. At a distance of 0.1 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 223 s, 3.9 s and 0.43 s, respectively. At a distance of 0.8 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 8800 s, 274 s and 0.43 s, respectively. In this case, WASABI+WRP can accelerate the hologram calculation by 20,000 times compared to the conventional method. A highly important point for WASABI+WRP is that it is not dependent on the propagation distance, unlike the conventional and WASABI methods. Even when calculating a greater number of object points, such as a fountain with approximately one million object points, on a normal CPU, WASABI+WRP successfully calculated a hologram with 2048 × 2048 pixels in approximately 0.4 s, that is, we can obtain a frame rate of 2.5.
3. Results In this section, we compare the computational performance and the quality of the reconstructed images obtained for three methods: the conventional method [3], the WASABI method [20], and the WASABI +WRP method. The calculation conditions included a wavelength of 633 nm, a hologram sampling interval of 10 μm , a hologram resolution of 2048×2048 pixels, and a distance of 0.01 m in the first step of the WASABI+WRP method. We empirically picked a selection ratio of r = 1% for the WASABI method according to the calculation time and the quality of the reconstructed images [20]. We used a computer with an Intel Core i7 6700 CPU, the Microsoft Windows 8.1 operating system, and the Microsoft Visual Studio C++2013 compiler. All the calculations were parallelized using four CPU threads and were performed using our wave optics library, CWO++ [37]. Fig. 2 shows the reconstructed images of a dinosaur composed of 11,646 object points at two different distances of 0.1 m and 0.8 m. The images in the left column of Fig. 2 were obtained using the conventional method. The reconstructed images in the center column of Fig. 2 were obtained using WASABI. We used structural similarities (SSIM) as the metrics of the image quality [38]. When SSIM is approximately 0.95, the image can be seen fairly well. And, when SSIM is over 0.98, the image quality is nearly the same as the original image. The SSIM between two images A and B is calculated by
SSIM =
(2μA μB + c1)(2σAB + c2 ) (μA2 + μB2 + c1)(σA2 + σB2 + c2 )
4. Conclusions We proposed a method combining the WRP method and WASABI to further accelerate hologram calculations. The drawbacks of the WRP method and WASABI were complementarily solved using the proposed method. Using a consumer CPU, the proposed method completed a hologram calculation with 2048 × 2048 pixels from a 3D object with one million points in approximately 0.4 s. In this paper, we used bandlimited Fresnel diffraction [17] in the second step of the proposed method. Even though this diffraction calculation was faster than the conventional Fresnel diffraction and angular spectrum method, the calculation cost was still considerable. The calculation cost of the second step in the proposed method will be improved using a sparse Fourier transform-based diffraction calculation [33], which will result in a further decrease in the total computational complexity. In addition, we will investigate a proper distance between 3D objects and the WRP because the WASABI+WRP method in Figs. 3 and 5 was not faster than the WASABI method at a distance of 0.1 m due to the additional calculation cost of diffraction calculation. We may be able to find the proper distance using the same method as in Ref. [35].
, (9)
where μa and μB are the averages of the image A and image B, σA and σB are the variances of the two images, σAB is the covariance of the two images, respectively. c1 and c2 are calculated by c1 = (k1 L )2 and c2 = (k2 L )2 where L is the dynamic range of the two images and k1=0.01 and k2=0.03, respectively. The SSIMs of the reconstructed images are 0.93 at a distance of 0.1 m, and 0.96 at a distance of 0.8 m. The reconstructed images in the right column of Fig. 2 were obtained using WASABI+WRP. The SSIMs of the reconstructed images are 0.95 and 0.96 at distances of 0.1 m and 0.8 m, respectively. Fig. 3 shows the calculation times for these three methods. At a distance of 0.1 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 2.8 s, 0.14 s and 0.39 s, respectively. For short distances, WASABI is better than WASABI+WRP due to the additional calculation cost of the second step in WASABI+WRP. By contrast, at a distance of 0.8 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 102 s, 22 s and 0.41 s, respectively. The conventional method and the WASABI method increase the calculation time with increasing propagation distance, whereas the calculation time of the WASABI +WRP method is constant and does not depend on the propagation distance. We show the performance of the proposed method in a case with a hundred thousand object points. Fig. 4 shows the reconstructed images of a merry-go-round composed of 95,949 points. The reconstructed images in the center column of Fig. 4 were obtained using the WASABI method. The SSIMs of the reconstructed images are 0.92 and 0.97 at distances of 0.1 m and 0.8 m, respectively. The reconstructed images in the right column of Fig. 4 were obtained using WASABI+WRP. The SSIM of the reconstructed images are 0.93 and 0.95 at distances of 0.1 m and 0.8 m, respectively. Fig. 5 shows the calculation times for the merry-go-round images. At a distance of 0.1 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 22 s, 0.44 s and 0.42 s, respectively. At a distance of
Acknowledgements This work was partially supported by JSPS KAKENHI Grant Numbers 16K00151 and 25240015. We thank Dr. Hirotaka Nakayama for providing 3D models. References [1] Ting-Chung Poon, Digital Holography and Three-dimensional Display: Principles and Applications, Springer Science & Business Media, 2006. [2] Mark E. Lucente, Interactive computation of holograms using a look-up table, J. Electron. Imag. 2 (1) (1993) 28–34. [3] Seung-Cheol Kim, Eun-Soo Kim, Effective generation of digital holograms of threedimensional objects using a novel look-up table method, Appl. Opt. 47 (19) (2008) D55–D62. [4] Kyoji Matsushima, Masahiro Takai, Recurrence formulas for fast creation of synthetic three-dimensional holograms, Appl. Opt. 39 (35) (2000) 6587–6594. [5] Hiroshi Yoshikawa, Fast computation of fresnel holograms employing difference, Opt. Rev. 8 (5) (2001) 331–335. [6] Tomoyoshi Shimobaba, Tomoyoshi Ito, An efficient computational method suitable for hardware of computer-generated hologram with phase computation by addition, Comput. Phys. Commun. 138 (1) (2001) 44–52. [7] Takeshi Yamaguchi, Hiroshi Yoshikawa, Computer-generated image hologram, Chin. Opt. Lett. 9 (12) (2011) 120006.
111
Optics Communications 393 (2017) 107–112
D. Arai et al.
[25] Tomoyoshi Shimobaba, Naohisa Okada, Takashi Kakue, Nobuyuki Masuda, Yasuyuki, Ichihashi, Ryutaro, Oi, Kenji Yamamoto, Tomoyoshi Ito. Computer holography using wavefront recording method, in: Digital Holography and ThreeDimensional Imaging, page DTu1A.2. Optical Society of America, 2013. [26] Anh-Hoang Phan, Mei lan Piao, Sang-Keun Gil, Nam Kim, Generation speed and reconstructed image quality enhancement of a long-depth object using double wavefront recording planes and a gpu, Appl. Opt. 53 (22) (2014) 4817–4824. [27] Daisuke Arai, Tomoyoshi Shimobaba, Koki Murano, Yutaka Endo, Ryuji Hirayama, Daisuke Hiyama, Takashi Kakue, Tomoyoshi Ito, Acceleration of computergenerated holograms using tilted wavefront recording plane method, Opt. Express 23 (2) (2015) 1740–1747. [28] N. Okada, T. Shimobaba, Y. Ichihashi, R. Oi, K. Yamamoto, T. Kakue, and T. Ito. Fast calculation of computer-generated hologram for RGB and depth images using wavefront recording plane method. Photon. Lett. Poland, 6: 2014, pp. 90–92. [29] P.W.M. Tsang, T.-C. Poon, Fast generation of digital holograms based on warping of the wavefront recording plane, Opt. Express 23 (6) (2015) 7667–7673. [30] Athanasia Symeonidou, David Blinder, Adrian Munteanu, Peter Schelkens, Computer-generated holograms by multiple wavefront recording plane method with occlusion culling, Opt. Express 23 (17) (2015) 22149–22161. [31] Chenliang Chang, Jun Wu, Yijun Qi, Caojin Yuan, Shouping Nie, Jun Xia, Simple calculation of a computer-generated hologram for lensless holographic 3d projection using a nonuniform sampled wavefront recording plane, Appl. Opt. 55 (28) (2016) 7988–7996. [32] Chen-Xiao Shi, Chang-Won Shin, Jong-Rae Jeong, Nam Kim.Acceleration method for computer generated hologram using random-sampled wavefront recording plane., in: Imaging and Applied Optics 2016, Optical Society of America, 6, 2016, p. DW5I. [33] Hak Gu Kim, Hyunwook Jeong, Yong Man Ro, Acceleration of the calculation speed of computer-generated holograms using the sparsity of the holographic fringe pattern for a 3d object, Opt. Express 24 (22) (2016) 25317–25328. [34] Chenliang Chang, Yijun Qi, Jun Wu, Caojin Yuan, Shouping Nie, Jun Xia, Numerical study for the calculation of computer-generated hologram in color holographic 3d projection enabled by modified wavefront recording plane method, Opt. Commun. 387 (2017) 267–274. [35] Naotaka Hasegawa, Tomoyoshi Shimobaba, Takashi Kakue, Tomoyoshi Ito, Acceleration of hologram generation by optimizing the arrangement of wavefront recording planes, Appl. Opt. 56 (1) (2017) A97–A103. [36] Tomoyoshi Shimobaba, Takashi Kakue, Minoru Oikawa, Naohisa Okada, Yutaka Endo, Ryuji Hirayama, Tomoyoshi Ito, Nonuniform sampled scalar diffraction calculation using nonuniform fast fourier transform, Opt. Lett. 38 (23) (2013) 5130–5133. [37] Tomoyoshi Shimobaba, Jiantong Weng, Takahiro Sakurai, Naohisa Okada, Takashi Nishitsuji, Naoki Takada, Atsushi Shiraki, Nobuyuki Masuda, Tomoyoshi Ito, Computational wave optics library for C++: cwo++ library, Comput. Phys. Commun. 183 (5) (2012) 1124–1138. [38] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, Eero P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process. 13 (4) (2004) 600–612.
[8] Takuya Sugawara, Yuki Ogihara, Yuji Sakamoto, Fast point-based method of a computer-generated hologram for a triangle-patch model by using a graphics processing unit, Appl. Opt. 55 (3) (2016) A160–A166. [9] Kyoji Matsushima, Computer-generated holograms for three-dimensional surface objects with shade and texture, Appl. Opt. 44 (22) (2005) 4607–4614. [10] Kenji Yamamoto, Yasuyuki Ichihashi, Takanori Senoh, Ryutaro Oi, Taiichiro Kurita, Calculating the fresnel diffraction of light from a shifted and tilted plane, Opt. Express 20 (12) (2012) 12949–12958. [11] Lukas Ahrenberg, Philip Benzie, Marcus Magnor, John Watson.Computer, generated holograms from three dimensional meshes using an analytic light transport model, Appl. Opt. 47 (10) (2008) 1567–1574. [12] Toyohiko Yatagai, Stereoscopic approach to 3-d display using computer-generated holograms, Appl. Opt. 15 (11) (1976) 2722–2729. [13] Masahiro Yamaguchi, Hideshi Hoshino, Toshio Honda, Nagaaki Ohyama. Phaseadded stereogram: calculation of hologram using computer graphics technique, in: Proceedings SPIE, International Society for Optics and Photonics, volume 1914 Sep. 1993. pp. 25–31. [14] Hoonjong Kang, Takeshi Yamaguchi, Hiroshi Yoshikawa, Accurate phase-added stereogram to improve the coherent stereogram, Appl. Opt. 47 (19) (2008) D44–D54. [15] Koki Wakunami, Masahiro Yamaguchi, Calculation for computer generated hologram using ray-sampling plane, Opt. Express 19 (10) (2011) 9086–9101. [16] Ryutaro Oi, Kenji Yamamoto, Makoto Okui, Electronic generation of holograms by using depth maps of real scenes, in: Proceedings SPIE, International Society for Optics and Photonics, volume 6912, Jan. 2008, page 69120M. [17] Naohisa Okada, Tomoyoshi Shimobaba, Yasuyuki Ichihashi, Ryutaro Oi, Kenji Yamamoto, Minoru Oikawa, Takashi Kakue, Nobuyuki Masuda, Tomoyoshi Ito, Band-limited double-step fresnel diffraction and its application to computer-generated holograms, Opt. Express 21 (7) (2013) 9192–9197. [18] P.W.M. Tsang, J.P. Liu, K.W.K. Cheung, T.-C. Poon, Modern methods for fast generation of digital holograms, 3D Res. 1 (2) (2010) 11–18. [19] T. Shimobaba, T. Kakue, T. Ito, Review of fast algorithms and hardware implementations on computer holography, IEEE Trans. Ind. Inform. 12 (4) (2016) 1611–1622. [20] Tomoyoshi Shimobaba, Tomoyoshi Ito, Fast generation of computer-generated holograms using wavelet shrinkage, Opt. Express 25 (1) (2017) 77–87. [21] Tomoyoshi Shimobaba, Nobuyuki Masuda, Tomoyoshi Ito, Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane, Opt. Lett. 34 (20) (2009) 3133–3135. [22] Tomoyoshi Shimobaba, Hirotaka Nakayama, Nobuyuki Masuda, Tomoyoshi Ito, Rapid calculation algorithm of fresnel computer-generated-hologram using look-up table and wavefront-recording plane methods for three-dimensional display, Opt. Express 18 (19) (2010) 19504–19509. [23] Peter Tsang, W.-K. Cheung, T.-C. Poon, C. Zhou, Holographic video at 40 frames per second for 4-million object points, Opt. Express 19 (16) (2011) 15205–15211. [24] Jiantong Weng, Tomoyoshi Shimobaba, Naohisa Okada, Hirotaka Nakayama, Minoru Oikawa, Nobuyuki Masuda, Tomoyoshi Ito, Generation of real-time large computer generated hologram using wavefront recording method, Opt. Express 20 (4) (2012) 4018–4023.
112
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
An accelerated hologram calculation using the wavefront recording plane method and wavelet transform
MARK
⁎
Daisuke Araia, Tomoyoshi Shimobabaa, , Takashi Nishitsujia, Takashi Kakuea, Nobuyuki Masudab, Tomoyoshi Itoa a b
Chiba University, Graduate School of Engineering, 1–33 Yayoi–cho, Inage–ku, Chiba 263-8522, Japan Department of Applied Electronics, Tokyo University of Science, 6–3–1 Niijuku, Katsushika–ku, Tokyo 125-8585, Japan
A R T I C L E I N F O
A BS T RAC T
Keywords: Computer-generated hologram 3D display Electroholography Hologram Holography
Fast hologram calculation methods are critical in real-time holography applications such as three-dimensional (3D) displays. We recently proposed a wavelet transform-based hologram calculation called WASABI. Even though WASABI can decrease the calculation time of a hologram from a point cloud, it increases the calculation time with increasing propagation distance. We also proposed a wavefront recoding plane (WRP) method. This is a two-step fast hologram calculation in which the first step calculates the superposition of light waves emitted from a point cloud in a virtual plane, and the second step performs a diffraction calculation from the virtual plane to the hologram plane. A drawback of the WRP method is in the first step when the point cloud has a large number of object points and/or a long distribution in the depth direction. In this paper, we propose a method combining WASABI and the WRP method in which the drawbacks of each can be complementarily solved. Using a consumer CPU, the proposed method succeeded in performing a hologram calculation with 2048 × 2048 pixels from a 3D object with one million points in approximately 0.4 s.
1. Introduction Due to the heavy calculation amounts of holograms, fast hologram calculation methods are essential in real-time holography applications such as three-dimensional (3D) displays [1]. Multiple algorithms have been proposed: point cloud methods [2–8], polygon methods [9–11], holographic stereograms [12–15], and RGB-D [16,17]. The details of these methods can be found in Refs. [18,19]. In point cloud methods, a 3D object is expressed as the aggregation of object points, and the light waves emitted from these object points are superposed in the hologram plane. The superposition is the most time-consuming part of the hologram calculation. To accelerate the superposition, we recently proposed a wavelet transform-based hologram calculation, which is referred to as the WAvelet ShrinkAge-Based superpositIon (WASABI) [20]. WASABI accelerates a hologram calculation by representing the object light with a few wavelet coefficients. Even though the WASABI can considerably decrease the calculation time of a hologram, it increases the calculation time with increasing propagation distance. We also proposed a wavefront recoding plane (WRP) method [21]. Subsequently, various improved methods for the WRP method have been proposed [22–35]. The WRP method is a two-step fast hologram
⁎
calculation in which the first step calculates the superposition of light waves emitted from object points in a virtual plane, and the second step performs diffraction calculation from the virtual plane to the hologram plane. A drawback of the WRP method is in the first step when the point cloud has a large number of object points and/or a long distribution in the depth direction. Several methods to alleviate this problem have been proposed. For example, multiple WRP methods introduce multiple virtual planes to split the long distribution [25,26,28,30], and the placement of the virtual planes can be optimized [35]. The tilted WRP method [27] decreases the calculation cost of the first step by using a tilted diffraction calculation [9], and the stretched WRP method [31] decreases the calculation cost using a non-uniform sampled diffraction calculation [36]. More recently, a WRP method using the sparse fast Fourier transform (FFT) has been proposed to accelerate the second step in the WRP method [33]. In this paper, we propose a method combining WASABI and the WRP method in which the drawbacks of each can be complementarily solved. That is, the calculation cost of the first step in the WRP method is accelerated using WASABI, resulting in a decrease in the total calculation time. Using a consumer CPU, the proposed method succeeded in completing a hologram calculation with 2048 × 2048
Corresponding author. E-mail address: [email protected] (T. Shimobaba).
http://dx.doi.org/10.1016/j.optcom.2017.02.038 Received 19 January 2017; Received in revised form 13 February 2017; Accepted 15 February 2017 0030-4018/ © 2017 Elsevier B.V. All rights reserved.
Optics Communications 393 (2017) 107–112
D. Arai et al.
Fig. 1. Hologram calculation for (a) a conventional hologram calculation and (b) the proposed method.
Fig. 2. Reconstructed images of a dinosaur composed of 11,646 object points at the two different distances of 0.1 m and 0.8 m.
object points. The light waves emitted from these object points are superimposed in the hologram plane using
pixels from a 3D object with one million points in approximately 0.4 s Section 2 describes the proposed method, and Section 3 shows the effectiveness of the proposed method. Section 4 concludes this study.
N
u (xh , yh) =
2. Proposed method
j
The proposed method combines WASABI and the WRP method to further accelerate the hologram calculation at long propagation distances. Fig. 1(a) shows a conventional hologram calculation from the 3D
⎛
⎞ 2π ⎟ rhj ⎟ = ⎝ λ ⎠
∑ aj exp ⎜⎜i
N
∑ aj u zj (xh − xj , yh − yj ), j
(1)
where i is the imaginary unit number, N is the number of object points, (xh , yh) are the coordinates of the hologram plane, (xj , yj , zj ) and aj are the coordinates and amplitude of the j-th object point, respectively, λ is the wavelength, and u zj is the point spread function (PSF) at a distance zj. 108
Optics Communications 393 (2017) 107–112
D. Arai et al.
Fig. 5. Calculation times for the merry-go-round image (95,949 object points) using the conventional method, WASABI, and WASABI+WRP at the two different distances of 0.1 m and 0.8 m.
Fig. 3. Calculation times for the dinosaur image (11,646 object points) using the conventional method, WASABI, and WASABI+WRP at the two different distances of 0.1 m and 0.8 m.
To decrease the computational complexity, look-up table methods, such as the novel look-up table (N-LUT) method [3], can decrease the calculation time. However, even when using look-up table methods, the computational complexity remains O (NW 2 ). Recently, we proposed the WASABI method [20] to decrease the computational complexity of Eq. (1). WASABI accelerates hologram calculations using the following three steps.
The computational complexity of Eq. (1) is decided by the number of object points and the average radius W of the PSFs. The average radius W is determined by
W =
1 N
N
∑ Wj = j
1 N
N
∑ zj tan θ, j
(2) 1. The PSFs u zj (xh , yh) for zj are pre-computed and then the spacedomain PSFs are transformed into wavelet-domain PSFs. The number of wavelet coefficients is reduced by selecting Nr large wavelet coefficients. 2. The reduced wavelet coefficients are superposed in the wavelet domain. 3. The inverse wavelet transform is performed to convert the superposed result in the wavelet domain to that in the space domain.
where the maximum divergence angle θ of an object point is θ = sin−1(λ /(2p )), where p is the sampling pitch of the hologram. The computational complexity of Eq. (1) is expressed as
O (NW 2 ).
(3)
It is clear that the computational complexity is increased as zj increases. The heaviest computation in Eq. (1) is the superposition.
Fig. 4. Reconstructed images of a merry-go-round composed of 95,949 object points at the two different distances of 0.1 m and 0.8 m.
109
Optics Communications 393 (2017) 107–112
D. Arai et al.
Fig. 6. Reconstructed images of a fountain composed of 978,416 object points at the two different distances of 0.1 m and 0.8 m.
O (rNW 2 ) + O (Nh2 ).
Even though WASABI can drastically decrease the computational complexity of the hologram calculation, it increases the calculation time with increasing propagation distance because the computational complexity depends on W , including zj. To further accelerate the hologram calculation at long distances, we propose a combination of WASABI and the WRP method (the WASABI +WRP method). As shown in Fig. 1(b), the WRP method introduces a virtual plane between the 3D object and the hologram plane. In the conventional WRP method [21,22], this virtual plane is calculated using Eq. (1) and the computational complexity is shown in Eq. (3). The computational cost becomes low when the virtual plane is placed near the 3D object. However, when the 3D object has a large number of object points and/or a long distribution in the depth direction, the computational complexity of the virtual plane becomes large due to Eq. (3). Instead of Eq. (1), we use WASABI to accelerate the calculation of the virtual plane; therefore, the computational complexity can be suppressed to that of Eq. (6). Subsequently, we calculate the numerical diffraction from the virtual plane to the hologram plane. For this diffraction calculation, in this paper, we used band-limited double-step Fresnel diffraction [17] because its complexity is lower than that of the conventional Fresnel diffraction and angular spectrum method [17,19]. The computational complexity of the diffraction calculation is O (Nh2 log Nh ). Therefore, the total computational complexity of the proposed method is
Fig. 7. Calculation times for the fountain image (978,416 object points) using the conventional method, WASABI, and WASABI+WRP at the two different distances of 0.1 m and 0.8 m.
In this study, the number of large wavelet coefficients, Nr, is defined as
Nr = r × 2πW 2,
(4)
where r is the selection rate of the larger wavelet coefficients. Therefore, in the second step of WASABI, the computational complexity of the superposition can be reduced to
O (NNr ) =
O (rNW 2 ).
(6)
O (rNW 2 ) + O (Nh2 ) + O (Nh2 log Nh ).
(5)
(7)
For comparison, the computational complexity of the conventional WRP method [21,22] is
If we use r = 1%, compared to the conventional method of Eq. (1), we can reduce the computational complexity of the superposition to approximately one hundredth. Subsequently, WASABI transforms the superposed result in the wavelet domain to that in the space domain using the inverse wavelet transform, whose computational complexity is only O (Nh2 ). Therefore, the total computational complexity of WASABI is
O (NW 2 ) + O (Nh2 log Nh ).
(8)
Even though the proposed method has an additional computational complexity, O (Nh2 ), for the inverse wavelet transform, the computational complexity of the proposed method is lower than that of the conventional WRP method. 110
Optics Communications 393 (2017) 107–112
D. Arai et al.
0.8 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 850 s, 27 s and 0.43 s, respectively. In the case of one million object points, WASABI+WRP shows further acceleration. Fig. 6 shows the reconstructed images of a fountain composed of 978,416 points. The reconstructed images in the center and right columns of Fig. 6 were obtained using WASABI and WASABI+WRP, respectively. Fig. 7 shows the calculation times. At a distance of 0.1 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 223 s, 3.9 s and 0.43 s, respectively. At a distance of 0.8 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 8800 s, 274 s and 0.43 s, respectively. In this case, WASABI+WRP can accelerate the hologram calculation by 20,000 times compared to the conventional method. A highly important point for WASABI+WRP is that it is not dependent on the propagation distance, unlike the conventional and WASABI methods. Even when calculating a greater number of object points, such as a fountain with approximately one million object points, on a normal CPU, WASABI+WRP successfully calculated a hologram with 2048 × 2048 pixels in approximately 0.4 s, that is, we can obtain a frame rate of 2.5.
3. Results In this section, we compare the computational performance and the quality of the reconstructed images obtained for three methods: the conventional method [3], the WASABI method [20], and the WASABI +WRP method. The calculation conditions included a wavelength of 633 nm, a hologram sampling interval of 10 μm , a hologram resolution of 2048×2048 pixels, and a distance of 0.01 m in the first step of the WASABI+WRP method. We empirically picked a selection ratio of r = 1% for the WASABI method according to the calculation time and the quality of the reconstructed images [20]. We used a computer with an Intel Core i7 6700 CPU, the Microsoft Windows 8.1 operating system, and the Microsoft Visual Studio C++2013 compiler. All the calculations were parallelized using four CPU threads and were performed using our wave optics library, CWO++ [37]. Fig. 2 shows the reconstructed images of a dinosaur composed of 11,646 object points at two different distances of 0.1 m and 0.8 m. The images in the left column of Fig. 2 were obtained using the conventional method. The reconstructed images in the center column of Fig. 2 were obtained using WASABI. We used structural similarities (SSIM) as the metrics of the image quality [38]. When SSIM is approximately 0.95, the image can be seen fairly well. And, when SSIM is over 0.98, the image quality is nearly the same as the original image. The SSIM between two images A and B is calculated by
SSIM =
(2μA μB + c1)(2σAB + c2 ) (μA2 + μB2 + c1)(σA2 + σB2 + c2 )
4. Conclusions We proposed a method combining the WRP method and WASABI to further accelerate hologram calculations. The drawbacks of the WRP method and WASABI were complementarily solved using the proposed method. Using a consumer CPU, the proposed method completed a hologram calculation with 2048 × 2048 pixels from a 3D object with one million points in approximately 0.4 s. In this paper, we used bandlimited Fresnel diffraction [17] in the second step of the proposed method. Even though this diffraction calculation was faster than the conventional Fresnel diffraction and angular spectrum method, the calculation cost was still considerable. The calculation cost of the second step in the proposed method will be improved using a sparse Fourier transform-based diffraction calculation [33], which will result in a further decrease in the total computational complexity. In addition, we will investigate a proper distance between 3D objects and the WRP because the WASABI+WRP method in Figs. 3 and 5 was not faster than the WASABI method at a distance of 0.1 m due to the additional calculation cost of diffraction calculation. We may be able to find the proper distance using the same method as in Ref. [35].
, (9)
where μa and μB are the averages of the image A and image B, σA and σB are the variances of the two images, σAB is the covariance of the two images, respectively. c1 and c2 are calculated by c1 = (k1 L )2 and c2 = (k2 L )2 where L is the dynamic range of the two images and k1=0.01 and k2=0.03, respectively. The SSIMs of the reconstructed images are 0.93 at a distance of 0.1 m, and 0.96 at a distance of 0.8 m. The reconstructed images in the right column of Fig. 2 were obtained using WASABI+WRP. The SSIMs of the reconstructed images are 0.95 and 0.96 at distances of 0.1 m and 0.8 m, respectively. Fig. 3 shows the calculation times for these three methods. At a distance of 0.1 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 2.8 s, 0.14 s and 0.39 s, respectively. For short distances, WASABI is better than WASABI+WRP due to the additional calculation cost of the second step in WASABI+WRP. By contrast, at a distance of 0.8 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 102 s, 22 s and 0.41 s, respectively. The conventional method and the WASABI method increase the calculation time with increasing propagation distance, whereas the calculation time of the WASABI +WRP method is constant and does not depend on the propagation distance. We show the performance of the proposed method in a case with a hundred thousand object points. Fig. 4 shows the reconstructed images of a merry-go-round composed of 95,949 points. The reconstructed images in the center column of Fig. 4 were obtained using the WASABI method. The SSIMs of the reconstructed images are 0.92 and 0.97 at distances of 0.1 m and 0.8 m, respectively. The reconstructed images in the right column of Fig. 4 were obtained using WASABI+WRP. The SSIM of the reconstructed images are 0.93 and 0.95 at distances of 0.1 m and 0.8 m, respectively. Fig. 5 shows the calculation times for the merry-go-round images. At a distance of 0.1 m, the calculation times for the conventional method, the WASABI method, and the WASABI+WRP method are approximately 22 s, 0.44 s and 0.42 s, respectively. At a distance of
Acknowledgements This work was partially supported by JSPS KAKENHI Grant Numbers 16K00151 and 25240015. We thank Dr. Hirotaka Nakayama for providing 3D models. References [1] Ting-Chung Poon, Digital Holography and Three-dimensional Display: Principles and Applications, Springer Science & Business Media, 2006. [2] Mark E. Lucente, Interactive computation of holograms using a look-up table, J. Electron. Imag. 2 (1) (1993) 28–34. [3] Seung-Cheol Kim, Eun-Soo Kim, Effective generation of digital holograms of threedimensional objects using a novel look-up table method, Appl. Opt. 47 (19) (2008) D55–D62. [4] Kyoji Matsushima, Masahiro Takai, Recurrence formulas for fast creation of synthetic three-dimensional holograms, Appl. Opt. 39 (35) (2000) 6587–6594. [5] Hiroshi Yoshikawa, Fast computation of fresnel holograms employing difference, Opt. Rev. 8 (5) (2001) 331–335. [6] Tomoyoshi Shimobaba, Tomoyoshi Ito, An efficient computational method suitable for hardware of computer-generated hologram with phase computation by addition, Comput. Phys. Commun. 138 (1) (2001) 44–52. [7] Takeshi Yamaguchi, Hiroshi Yoshikawa, Computer-generated image hologram, Chin. Opt. Lett. 9 (12) (2011) 120006.
111
Optics Communications 393 (2017) 107–112
D. Arai et al.
[25] Tomoyoshi Shimobaba, Naohisa Okada, Takashi Kakue, Nobuyuki Masuda, Yasuyuki, Ichihashi, Ryutaro, Oi, Kenji Yamamoto, Tomoyoshi Ito. Computer holography using wavefront recording method, in: Digital Holography and ThreeDimensional Imaging, page DTu1A.2. Optical Society of America, 2013. [26] Anh-Hoang Phan, Mei lan Piao, Sang-Keun Gil, Nam Kim, Generation speed and reconstructed image quality enhancement of a long-depth object using double wavefront recording planes and a gpu, Appl. Opt. 53 (22) (2014) 4817–4824. [27] Daisuke Arai, Tomoyoshi Shimobaba, Koki Murano, Yutaka Endo, Ryuji Hirayama, Daisuke Hiyama, Takashi Kakue, Tomoyoshi Ito, Acceleration of computergenerated holograms using tilted wavefront recording plane method, Opt. Express 23 (2) (2015) 1740–1747. [28] N. Okada, T. Shimobaba, Y. Ichihashi, R. Oi, K. Yamamoto, T. Kakue, and T. Ito. Fast calculation of computer-generated hologram for RGB and depth images using wavefront recording plane method. Photon. Lett. Poland, 6: 2014, pp. 90–92. [29] P.W.M. Tsang, T.-C. Poon, Fast generation of digital holograms based on warping of the wavefront recording plane, Opt. Express 23 (6) (2015) 7667–7673. [30] Athanasia Symeonidou, David Blinder, Adrian Munteanu, Peter Schelkens, Computer-generated holograms by multiple wavefront recording plane method with occlusion culling, Opt. Express 23 (17) (2015) 22149–22161. [31] Chenliang Chang, Jun Wu, Yijun Qi, Caojin Yuan, Shouping Nie, Jun Xia, Simple calculation of a computer-generated hologram for lensless holographic 3d projection using a nonuniform sampled wavefront recording plane, Appl. Opt. 55 (28) (2016) 7988–7996. [32] Chen-Xiao Shi, Chang-Won Shin, Jong-Rae Jeong, Nam Kim.Acceleration method for computer generated hologram using random-sampled wavefront recording plane., in: Imaging and Applied Optics 2016, Optical Society of America, 6, 2016, p. DW5I. [33] Hak Gu Kim, Hyunwook Jeong, Yong Man Ro, Acceleration of the calculation speed of computer-generated holograms using the sparsity of the holographic fringe pattern for a 3d object, Opt. Express 24 (22) (2016) 25317–25328. [34] Chenliang Chang, Yijun Qi, Jun Wu, Caojin Yuan, Shouping Nie, Jun Xia, Numerical study for the calculation of computer-generated hologram in color holographic 3d projection enabled by modified wavefront recording plane method, Opt. Commun. 387 (2017) 267–274. [35] Naotaka Hasegawa, Tomoyoshi Shimobaba, Takashi Kakue, Tomoyoshi Ito, Acceleration of hologram generation by optimizing the arrangement of wavefront recording planes, Appl. Opt. 56 (1) (2017) A97–A103. [36] Tomoyoshi Shimobaba, Takashi Kakue, Minoru Oikawa, Naohisa Okada, Yutaka Endo, Ryuji Hirayama, Tomoyoshi Ito, Nonuniform sampled scalar diffraction calculation using nonuniform fast fourier transform, Opt. Lett. 38 (23) (2013) 5130–5133. [37] Tomoyoshi Shimobaba, Jiantong Weng, Takahiro Sakurai, Naohisa Okada, Takashi Nishitsuji, Naoki Takada, Atsushi Shiraki, Nobuyuki Masuda, Tomoyoshi Ito, Computational wave optics library for C++: cwo++ library, Comput. Phys. Commun. 183 (5) (2012) 1124–1138. [38] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, Eero P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process. 13 (4) (2004) 600–612.
[8] Takuya Sugawara, Yuki Ogihara, Yuji Sakamoto, Fast point-based method of a computer-generated hologram for a triangle-patch model by using a graphics processing unit, Appl. Opt. 55 (3) (2016) A160–A166. [9] Kyoji Matsushima, Computer-generated holograms for three-dimensional surface objects with shade and texture, Appl. Opt. 44 (22) (2005) 4607–4614. [10] Kenji Yamamoto, Yasuyuki Ichihashi, Takanori Senoh, Ryutaro Oi, Taiichiro Kurita, Calculating the fresnel diffraction of light from a shifted and tilted plane, Opt. Express 20 (12) (2012) 12949–12958. [11] Lukas Ahrenberg, Philip Benzie, Marcus Magnor, John Watson.Computer, generated holograms from three dimensional meshes using an analytic light transport model, Appl. Opt. 47 (10) (2008) 1567–1574. [12] Toyohiko Yatagai, Stereoscopic approach to 3-d display using computer-generated holograms, Appl. Opt. 15 (11) (1976) 2722–2729. [13] Masahiro Yamaguchi, Hideshi Hoshino, Toshio Honda, Nagaaki Ohyama. Phaseadded stereogram: calculation of hologram using computer graphics technique, in: Proceedings SPIE, International Society for Optics and Photonics, volume 1914 Sep. 1993. pp. 25–31. [14] Hoonjong Kang, Takeshi Yamaguchi, Hiroshi Yoshikawa, Accurate phase-added stereogram to improve the coherent stereogram, Appl. Opt. 47 (19) (2008) D44–D54. [15] Koki Wakunami, Masahiro Yamaguchi, Calculation for computer generated hologram using ray-sampling plane, Opt. Express 19 (10) (2011) 9086–9101. [16] Ryutaro Oi, Kenji Yamamoto, Makoto Okui, Electronic generation of holograms by using depth maps of real scenes, in: Proceedings SPIE, International Society for Optics and Photonics, volume 6912, Jan. 2008, page 69120M. [17] Naohisa Okada, Tomoyoshi Shimobaba, Yasuyuki Ichihashi, Ryutaro Oi, Kenji Yamamoto, Minoru Oikawa, Takashi Kakue, Nobuyuki Masuda, Tomoyoshi Ito, Band-limited double-step fresnel diffraction and its application to computer-generated holograms, Opt. Express 21 (7) (2013) 9192–9197. [18] P.W.M. Tsang, J.P. Liu, K.W.K. Cheung, T.-C. Poon, Modern methods for fast generation of digital holograms, 3D Res. 1 (2) (2010) 11–18. [19] T. Shimobaba, T. Kakue, T. Ito, Review of fast algorithms and hardware implementations on computer holography, IEEE Trans. Ind. Inform. 12 (4) (2016) 1611–1622. [20] Tomoyoshi Shimobaba, Tomoyoshi Ito, Fast generation of computer-generated holograms using wavelet shrinkage, Opt. Express 25 (1) (2017) 77–87. [21] Tomoyoshi Shimobaba, Nobuyuki Masuda, Tomoyoshi Ito, Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane, Opt. Lett. 34 (20) (2009) 3133–3135. [22] Tomoyoshi Shimobaba, Hirotaka Nakayama, Nobuyuki Masuda, Tomoyoshi Ito, Rapid calculation algorithm of fresnel computer-generated-hologram using look-up table and wavefront-recording plane methods for three-dimensional display, Opt. Express 18 (19) (2010) 19504–19509. [23] Peter Tsang, W.-K. Cheung, T.-C. Poon, C. Zhou, Holographic video at 40 frames per second for 4-million object points, Opt. Express 19 (16) (2011) 15205–15211. [24] Jiantong Weng, Tomoyoshi Shimobaba, Naohisa Okada, Hirotaka Nakayama, Minoru Oikawa, Nobuyuki Masuda, Tomoyoshi Ito, Generation of real-time large computer generated hologram using wavefront recording method, Opt. Express 20 (4) (2012) 4018–4023.
112