- Email: [email protected]
Visualization of partially occluded 3D object using wedge prism-based axially distributed sensing
Optics Communications 313 (2014) 204–209
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Visualization of partially occluded 3D object using wedge prism-based axially distributed sensing Miao Zhang a, Yongri Piao b, Joon-Jae Lee a, Donghak Shin c, Byung-Gook Lee c,n a
Department of Game Mobil Contents, Keimyung University, Daegu 704701, Republic of Korea School of Information and Communication Engineering, Dalian University of Technology, Dalian 116024, China c Institute of Ambient Intelligence, Dongseo University, Busan 617716, Republic of Korea b
art ic l e i nf o
a b s t r a c t
Article history: Received 1 May 2013 Received in revised form 12 September 2013 Accepted 27 September 2013 Available online 12 October 2013
Recently, an axially distributed sensing system (ADS) was proposed for three-dimensional imaging and visualization. In ADS system, the 3D information cannot be collected when the coordinates of the object are close to the optical axis. In this paper, we present a wedge-prism based axially distributed sensing for improving the visual quality of 3D reconstructed images. In the proposed method, a wedge prism is placed in front of a camera and the parallax information is collected through the wedge prism by translating the wedge prism and the image sensor together along the optical axis. Accordingly, the 3D object is recorded as the improved multiple 2D perspective images within the full area of the image sensor. The volumetric images are generated from the recorded elemental images using a computational reconstruction algorithm based on ray back-projection. The proposed method is applied to partially occluded 3D object visualization. Preliminary experiments are performed to verify the approach. & 2013 Elsevier B.V. All rights reserved.
Keywords: 3D imaging Computational reconstruction Elemental images Object visualization
1. Introduction Since the development of the first three-dimensional (3D) technique ‘Stereoscope’, several kinds of 3D imaging and display techniques have been proposed for various applications [1–5]. Among them, integral-imaging has been known as one of the most promising 3D imaging and display techniques, because it has the ability to provide a full-parallax and continuous-viewing 3-D image with incoherent light. It also has wide applications such as 3D imaging, pattern recognition, object tracking and so on [3– 31]. One of the most interesting applications is visualization of partially occluded objects [6,7,10,15,21,22,28–31]. In fact, most 3D scenes might be viewed by a mixture of various object images having different depths. This means that a target could be partially occluded by foreground objects. Accordingly, visualization of 3D objects has been considered as one of the most challenging drawbacks in the fields of 3D object detection and recognition. Integral imaging technique for high-resolution visualization of partially occluded objects can be obtained by using the multiple cameras or moving cameras to acquire depth information. Even though it can provide high-resolution depth information, the system structure of integral imaging is complex because of 2D grid array structure of many cameras or moving a camera along both horizontal and vertical direction. Recently, an axially distributed image sensing (ADS) method was proposed for visualization of partially occluded objects [23–29].
n
Corresponding author. E-mail address: [email protected] (B.-G. Lee).
0030-4018/$ - see front matter & 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.optcom.2013.09.060
This method is performed with a simple architecture, where longitudinal perspective information of a 3D object is obtained by translating a camera along its optical axis. The capacity of this method to collect 3D information is directly ties into how far the object is located from the optical axis. The 3D information cannot be collected when the coordinates of the object are close to the optical axis. This implies that there is a lower capacity for 3D information collection for objects located close to the optical axis. Thus, this method due to the center of the CCD lies on the optical axis of the camera could suffer the limitation of the undesirable pickup zone and inefficient use in CCD sensor. In this paper, to overcome the limitation of the ADS method, we propose a new 3D imaging method using a wedge prism. This is called the wedge prism-based axially distributed sensing (WPADS) method, where a wedge prism is placed in front of a camera and the parallax information is collected through the wedge prism by translating the wedge prism and the camera together along the camera's optical axis. Due to the wedge prism involved in the proposed imaging system, it is able to redirect the outgoing light for the field of view (FOV) of the camera into a different angle. This means that the 3D object of interest can appear in the center of the field of view. As a result, we can record the complete multiple 2D perspective images of the 3D object using the full area of an image sensor in the proposed method.
2. Axially distributed sensing with wedge prism Fig. 1 shows the proposed system using the wedge prism. The camera with the wedge prism is translated along its longitudinal
M. Zhang et al. / Optics Communications 313 (2014) 204–209
205
Fig. 1. (a) Wedge prism-based axially distributed sensing (WPADS) and (b) ray trace through the wedge prism.
Fig. 2. Pinhole approximation of WPADS system.
axis and records multiple 2D perspective images of the 3D object. The wedge prism is mounted in front of the lens of the camera as shown in Fig. 1(a). In order to clearly explain the principle of ray transmission through the wedge prism as shown in Fig. 1(b), we assume that the angle of light from the vertical surface side as θi, and the angle of light from the incline surface side as θd, which is approximately given by θd ðnw 1Þα þ θi
ð1Þ
where nw is the index of refraction of prism material, and α is the angle between the prism surfaces. Note the proposed WPADS system can unify the conventional ADS system by adjusting the angle α¼ 01. Now, we explain the principle of 3D imaging using the proposed WPADS system. To do so, we consider the pinhole approximation of the proposed system as shown in Fig. 2. A set of intermittent images, called elemental images, are recorded by this camera being translated along its longitudinal axis. For simplicity, the ray approximation of geometric optics is considered
for the camera lens in order to ignore the optical effect. Each elemental image shares a common image center along the optical axis of the camera lens aligned with the coordinate z axis. The distance between the pinhole and the sensor is g. We assume that a 3D object is located at the longitudinal distance z0 away from the first camera. A total of K elemental images are captured. The elemental image furthest from the object is indexed as k ¼1 and the elemental image closest from the object is as k ¼K. The Kth elemental image is located at a distance of zk ¼z0 (k 1)Δz from the object, where Δz is the axial separation between two successive elemental images. From Fig. 2, a ray from an object point passing through the wedge prism and the pinhole is deflected by the angle θd and the optical path is no longer follow the original one. As elemental images are captured at different distances from the object, the objects will appear in elemental images at different sizes, reflecting different levels of magnification. In the WPADS system as shown in Fig. 3, a wedge prism is placed in front of a camera and the parallax information is collected
206
M. Zhang et al. / Optics Communications 313 (2014) 204–209
Fig. 3. FOV analysis of the WPADS.
Fig. 4. Ray diagram for computational reconstruction using kth elemental image recorded from the WPADS system.
Fig. 5. Ray diagram for calculation of minimum depth resolution in the WPADS system.
M. Zhang et al. / Optics Communications 313 (2014) 204–209
through the wedge prism by translating the wedge prism and the camera together along the camera's optical axis. Due to the wedge prism involved in the proposed imaging system, it is able to redirect the outgoing light for the FOV of the camera into a different angle. Accordingly, the off-axis object of interest can appear in the center of the FOV, and 3D information of the 3D object is able to be recorded in the center of the FOV using the full area of image sensor in the proposed method. As a result, resolution of the reconstructed 3D images may be improved with the full area of the image sensor. The computational reconstruction is shown in Fig. 4 which illustrates the 3D object reconstruction process of elemental image recorded in the WPADS system. The computational reconstruction algorithm is based on the ray propagation of elemental images. Let us suppose that the computational reconstruction of the 3D image on a reconstruction plane is at distance zk, and the wedge prism deflects light rays in x axis. For a fixed distance zk, the kth elemental image Ek(x, y) with N N pixels is propagated through the corresponding virtual pinhole and the wedge prism. By using the Eq. (1), we can obtain the deflected angle θdk(x) as below θdk ðxÞ ¼ ðnw 1Þα þ θik ðxÞ ¼ ðnw 1Þα þ arctan
cx g
ð2Þ
where c is the pixel size of the elemental image, and g is the distance between elemental image and pinhole array. Finally, we can obtain the reconstructed image at the distance z. For the faster calculation, we normalize the reconstruction process using the magnification ration Mk ¼zk/g. Then, the reconstructed image R(x', y', z) is the
207
summation of all the elemental image Ek and its given by 1 K ∑ R ðx′; y′Þ Kk¼1 k 1 K 1 ∑ Ek hk ðxÞ; y ¼ Kk¼1 cM k
Rðx′; y′; zÞ ¼
ð3Þ
where the distance between the optical axis and the reconstruction image point is hk(x) ¼tan θdk(x)zk.
3. Analysis of depth resolution For the WPADS system, we can calculate the minimum depth resolution in the reconstructed object space. We assume that two point sources at two different ranges or depths in the object space as shown in Fig. 5. Two different longitudinal depths are projected to the elemental image plane. Then, the minimum depth resolutionδ for the two depths can be obtained when they produce a shift of one pixel in the elemental image, which is obtained from the nearest elemental image from the object. If the pixel size of the recording sensor is c and the distance between the optical axis and two depths is h, the minimum depth resolution at the distance between the object and the nearest elemental image zmin is given by δ ¼ h tan θd ðx2Þ zmin
ð4Þ
Here, we can calculate tan θd(x2)¼tan[(nw 1)αþθi(x2)] by using Eq. (4), where tan θi ðx2Þ ¼ tan θi ðx1Þ c=g, tan θi ðx1Þ ¼ tan ½θd ðx1Þ ðnw 1Þαand tan θd ðx1Þ ¼ h=zðminÞ , respectively. When we use a camera with a focal length of 50 mm and pixel size of 8.2 μm, the depth changes as function of the longitudinal distance of the 3D object is shown in Fig. 6. From the results of Fig. 6, the minimum depth resolution of WPADS system is better than the conventional ADS system (α¼ 01) at the angles α ¼101 and α¼ 201, respectively. Additionally, we can easily find the minimum depth resolution of the WPADS system improves as α and z increase as shown in Fig. 6.
4. Experiments and results
Fig. 6. Minimum depth resolution according to the distance between objects and nearest EI.
To demonstrate the proposed method, we performed the preliminary experiments for visualization of partially occluded 3D objects. In the experiments, a wedge prism is placed in front of a camera as shown in Fig. 7. The 3D object is a toy car which is positioned at approximately 450 mm away from the first camera. The heavy occlusion is pine tree branches located at approximately 300 mm from the sensors (see Fig. 8). The camera has an image sensor with 2400 1600 pixels, and pixel size is 8.2 μm. The camera lens has the focal length of f¼ 50 mm, the angle α between the wedge prism surfaces is 101, and the index of refraction nw of
Fig. 7. Wedge prism based axially distributed sensing system.
208
M. Zhang et al. / Optics Communications 313 (2014) 204–209
Fig. 8. Example of elemental image in the optical experiment: (a) first elemental image (b) 41st elemental image.
Fig. 9. Reconstructed images for (a) z ¼ 350 mm, (b) z ¼450 mm, (c) z¼ 700 mm.
the wedge prism (BK7 glass) is 1.52. For the computational reconstruction, the pinhole gap g is approximately at 50 mm. The camera is translated at Δz ¼5 mm increments for a total of K ¼41 elemental images and total displacement distance of 200 mm. The first and the 41st elemental images are shown in
Fig. 8. It is seen that 3D object was recorded within the entire elemental images. This is because the proposed method uses tilting of the optical axis by wedge prism. This implies that there is a higher capacity for 3D information collection for object located closed to the sensor. Thus, the WPADS system not only can
M. Zhang et al. / Optics Communications 313 (2014) 204–209
improve the resolution of the elemental images, but also can shorten the pickup distance to collect more object information compared with the ADS system. To reconstruct the partially occluded 3D object, all 41 recorded elemental images are used in the computational reconstruction algorithm as shown in Fig. 4. The 2D slice images of the 3D scene were obtained according to the reconstruction distances. Some reconstructed images are shown in Fig. 9. The reconstructed slice image shown in Fig. 9(b) is focused on the ‘car’ object. Here the distance of the reconstruction plane was 450 mm from the sensor where the ‘car’ object is originally located. From the results of Fig. 9, the proposed method was demonstrated successfully for 3D image reconstruction of partially occluded objects. 5. Conclusions In conclusion, we have presented the proposed WPADS system for 3D imaging and visualization of partially occluded objects. In this system, a wedge prism was placed in front of a camera, and the parallax information was collected through the wedge prism and the camera by translating them together along the camera's optical axis. Due to the wedge prism involved in the proposed imaging system, the 3D object can appear in the center of the field of view and improve the visual quality of the reconstructed images with the full area of the image sensor. The experimental results show that the WPADS system was effective to be used for 3D visualization of partially occluded objects. There are broad applications of this approach to object classification, automatic target recognition and 3D microscopy such as conventional ADS system [23–27]. Acknowledgments This work was supported by research fund from the Dongseo University, Dongseo Frontier Project Research Fund of 2011.
209
References [1] S.A. Benton (Ed.), Selected Papers on Three-Dimensional Displays, SPIE Optical Engineering Press, Bellingham, WA, 2001. [2] T. Okoshi, Proceedings of the IEEE 68 (5) (1980) 548. [3] G. Lippmann, Journal of Physics 7 (1908) 821. [4] F. Okano, H. Hoshino, J. Arai, I. Yuyama, Applied Optics 36 (1997) 1598. [5] R. Martinez-Cuenca, A. Pons, G. Saavedra, M. Martinez-Corral, B. Javidi, Optics Express 14 (2006) 9657. [6] B. Javidi, R. Ponce-Díaz, S.-H. Hong, Optics Letters 31 (2006) 1106. [7] B. Tavakoli, B. Javidi, E. Watson, Optics Express 16 (2008) 4426. [8] D. Shin, M. Cho, K.-C. Park, E.-S. Kim, ETRI Journal 27 (2005) 708. [9] J.-S. Park, D.-C. Hwang, D. Shin, E.-S. Kim, Optics Engineering 45 (2006) 117004. [10] I. Moon, B. Javidi, Optics Express 16 (2008) 13080. [11] B. Javidi, Y.-S. Hwang, Journal of Display Technology 4 (2008) 3. [12] R. Schulein, B. Javidi, Journal of Display Technology 4 (2008) 351. [13] G. Li, K.-C. Kwon, G.-H. Shin, J.-S. Jeong, K.-H. Yoo, N. Kim, Journal of the Optical Society of Korea 16 (2012) 381. [14] D.-C. Hwang, D. Shin, S.-C. Kim, E.-S. Kim, Applied Optics 47 (2008) D128. [15] D. Shin, B.-G. Lee, J.-J. Lee, Optics Express 16 (2008) 16294. [16] Y. Piao, D. Shin, E.-S. Kim, Optics and Lasers in Engineering 47 (2009) 1273. [17] D. Shin, H. Yoo, Optics Express 16 (2008) 8855. [18] B. Javidi, I. Moon, S. Yeom, Optics Express 14 (2006) 12096. [19] C.M. Do, R. Martínez-Cuenca, B. Javidi, Journal of the Optical Society of America 26 (2009) 245. [20] G. Baasantseren, J.-H. Park, N. Kim, Japanese Journal of Applied Physics 48 (2009) 020216. [21] Y. Piao, M. Zhang, E.-S. Kim, Optics and Lasers in Engineering 51 (2012) 1602. [22] M. Zhang, Y. Piao, E.-S. Kim, Applied Optics 50 (2011) 5369. [23] R. Schulein, M. DaneshPanah, B. Javidi, Optics Letters 34 (2009) 2012. [24] D. Shin, M. Cho, B. Javidi, Optics Letters 35 (2010) 3646. [25] D. Shin, B. Javidi, Journal of Display Technology 8 (2012) 317. [26] X. Xiao, B. Javidi, Optics Letters 36 (2011) 1086. [27] C.-W. Chen, M. Cho, Y.-P. Huang, B. Javidi, Optics Letters 37 (2012) 4125. [28] X. Xiao, B. Javidi, M. Martinez-Corral, Adrian Stern, Applied Optics 52 (2012) 546. [29] M. Cho, M. Daneshpanah, I. Moon, B. Javidi, Proceedings of the IEEE Journal 99 (2011) 556. [30] R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, B. Javidi, Proceedings of the IEEE Journal 97 (2009) 1067. [31] Y. Rivenson, A. Rot, S. Balber, A. Stern, J. Rosen, Optics Letters 37 (2012) 1757.
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Visualization of partially occluded 3D object using wedge prism-based axially distributed sensing Miao Zhang a, Yongri Piao b, Joon-Jae Lee a, Donghak Shin c, Byung-Gook Lee c,n a
Department of Game Mobil Contents, Keimyung University, Daegu 704701, Republic of Korea School of Information and Communication Engineering, Dalian University of Technology, Dalian 116024, China c Institute of Ambient Intelligence, Dongseo University, Busan 617716, Republic of Korea b
art ic l e i nf o
a b s t r a c t
Article history: Received 1 May 2013 Received in revised form 12 September 2013 Accepted 27 September 2013 Available online 12 October 2013
Recently, an axially distributed sensing system (ADS) was proposed for three-dimensional imaging and visualization. In ADS system, the 3D information cannot be collected when the coordinates of the object are close to the optical axis. In this paper, we present a wedge-prism based axially distributed sensing for improving the visual quality of 3D reconstructed images. In the proposed method, a wedge prism is placed in front of a camera and the parallax information is collected through the wedge prism by translating the wedge prism and the image sensor together along the optical axis. Accordingly, the 3D object is recorded as the improved multiple 2D perspective images within the full area of the image sensor. The volumetric images are generated from the recorded elemental images using a computational reconstruction algorithm based on ray back-projection. The proposed method is applied to partially occluded 3D object visualization. Preliminary experiments are performed to verify the approach. & 2013 Elsevier B.V. All rights reserved.
Keywords: 3D imaging Computational reconstruction Elemental images Object visualization
1. Introduction Since the development of the first three-dimensional (3D) technique ‘Stereoscope’, several kinds of 3D imaging and display techniques have been proposed for various applications [1–5]. Among them, integral-imaging has been known as one of the most promising 3D imaging and display techniques, because it has the ability to provide a full-parallax and continuous-viewing 3-D image with incoherent light. It also has wide applications such as 3D imaging, pattern recognition, object tracking and so on [3– 31]. One of the most interesting applications is visualization of partially occluded objects [6,7,10,15,21,22,28–31]. In fact, most 3D scenes might be viewed by a mixture of various object images having different depths. This means that a target could be partially occluded by foreground objects. Accordingly, visualization of 3D objects has been considered as one of the most challenging drawbacks in the fields of 3D object detection and recognition. Integral imaging technique for high-resolution visualization of partially occluded objects can be obtained by using the multiple cameras or moving cameras to acquire depth information. Even though it can provide high-resolution depth information, the system structure of integral imaging is complex because of 2D grid array structure of many cameras or moving a camera along both horizontal and vertical direction. Recently, an axially distributed image sensing (ADS) method was proposed for visualization of partially occluded objects [23–29].
n
Corresponding author. E-mail address: [email protected] (B.-G. Lee).
0030-4018/$ - see front matter & 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.optcom.2013.09.060
This method is performed with a simple architecture, where longitudinal perspective information of a 3D object is obtained by translating a camera along its optical axis. The capacity of this method to collect 3D information is directly ties into how far the object is located from the optical axis. The 3D information cannot be collected when the coordinates of the object are close to the optical axis. This implies that there is a lower capacity for 3D information collection for objects located close to the optical axis. Thus, this method due to the center of the CCD lies on the optical axis of the camera could suffer the limitation of the undesirable pickup zone and inefficient use in CCD sensor. In this paper, to overcome the limitation of the ADS method, we propose a new 3D imaging method using a wedge prism. This is called the wedge prism-based axially distributed sensing (WPADS) method, where a wedge prism is placed in front of a camera and the parallax information is collected through the wedge prism by translating the wedge prism and the camera together along the camera's optical axis. Due to the wedge prism involved in the proposed imaging system, it is able to redirect the outgoing light for the field of view (FOV) of the camera into a different angle. This means that the 3D object of interest can appear in the center of the field of view. As a result, we can record the complete multiple 2D perspective images of the 3D object using the full area of an image sensor in the proposed method.
2. Axially distributed sensing with wedge prism Fig. 1 shows the proposed system using the wedge prism. The camera with the wedge prism is translated along its longitudinal
M. Zhang et al. / Optics Communications 313 (2014) 204–209
205
Fig. 1. (a) Wedge prism-based axially distributed sensing (WPADS) and (b) ray trace through the wedge prism.
Fig. 2. Pinhole approximation of WPADS system.
axis and records multiple 2D perspective images of the 3D object. The wedge prism is mounted in front of the lens of the camera as shown in Fig. 1(a). In order to clearly explain the principle of ray transmission through the wedge prism as shown in Fig. 1(b), we assume that the angle of light from the vertical surface side as θi, and the angle of light from the incline surface side as θd, which is approximately given by θd ðnw 1Þα þ θi
ð1Þ
where nw is the index of refraction of prism material, and α is the angle between the prism surfaces. Note the proposed WPADS system can unify the conventional ADS system by adjusting the angle α¼ 01. Now, we explain the principle of 3D imaging using the proposed WPADS system. To do so, we consider the pinhole approximation of the proposed system as shown in Fig. 2. A set of intermittent images, called elemental images, are recorded by this camera being translated along its longitudinal axis. For simplicity, the ray approximation of geometric optics is considered
for the camera lens in order to ignore the optical effect. Each elemental image shares a common image center along the optical axis of the camera lens aligned with the coordinate z axis. The distance between the pinhole and the sensor is g. We assume that a 3D object is located at the longitudinal distance z0 away from the first camera. A total of K elemental images are captured. The elemental image furthest from the object is indexed as k ¼1 and the elemental image closest from the object is as k ¼K. The Kth elemental image is located at a distance of zk ¼z0 (k 1)Δz from the object, where Δz is the axial separation between two successive elemental images. From Fig. 2, a ray from an object point passing through the wedge prism and the pinhole is deflected by the angle θd and the optical path is no longer follow the original one. As elemental images are captured at different distances from the object, the objects will appear in elemental images at different sizes, reflecting different levels of magnification. In the WPADS system as shown in Fig. 3, a wedge prism is placed in front of a camera and the parallax information is collected
206
M. Zhang et al. / Optics Communications 313 (2014) 204–209
Fig. 3. FOV analysis of the WPADS.
Fig. 4. Ray diagram for computational reconstruction using kth elemental image recorded from the WPADS system.
Fig. 5. Ray diagram for calculation of minimum depth resolution in the WPADS system.
M. Zhang et al. / Optics Communications 313 (2014) 204–209
through the wedge prism by translating the wedge prism and the camera together along the camera's optical axis. Due to the wedge prism involved in the proposed imaging system, it is able to redirect the outgoing light for the FOV of the camera into a different angle. Accordingly, the off-axis object of interest can appear in the center of the FOV, and 3D information of the 3D object is able to be recorded in the center of the FOV using the full area of image sensor in the proposed method. As a result, resolution of the reconstructed 3D images may be improved with the full area of the image sensor. The computational reconstruction is shown in Fig. 4 which illustrates the 3D object reconstruction process of elemental image recorded in the WPADS system. The computational reconstruction algorithm is based on the ray propagation of elemental images. Let us suppose that the computational reconstruction of the 3D image on a reconstruction plane is at distance zk, and the wedge prism deflects light rays in x axis. For a fixed distance zk, the kth elemental image Ek(x, y) with N N pixels is propagated through the corresponding virtual pinhole and the wedge prism. By using the Eq. (1), we can obtain the deflected angle θdk(x) as below θdk ðxÞ ¼ ðnw 1Þα þ θik ðxÞ ¼ ðnw 1Þα þ arctan
cx g
ð2Þ
where c is the pixel size of the elemental image, and g is the distance between elemental image and pinhole array. Finally, we can obtain the reconstructed image at the distance z. For the faster calculation, we normalize the reconstruction process using the magnification ration Mk ¼zk/g. Then, the reconstructed image R(x', y', z) is the
207
summation of all the elemental image Ek and its given by 1 K ∑ R ðx′; y′Þ Kk¼1 k 1 K 1 ∑ Ek hk ðxÞ; y ¼ Kk¼1 cM k
Rðx′; y′; zÞ ¼
ð3Þ
where the distance between the optical axis and the reconstruction image point is hk(x) ¼tan θdk(x)zk.
3. Analysis of depth resolution For the WPADS system, we can calculate the minimum depth resolution in the reconstructed object space. We assume that two point sources at two different ranges or depths in the object space as shown in Fig. 5. Two different longitudinal depths are projected to the elemental image plane. Then, the minimum depth resolutionδ for the two depths can be obtained when they produce a shift of one pixel in the elemental image, which is obtained from the nearest elemental image from the object. If the pixel size of the recording sensor is c and the distance between the optical axis and two depths is h, the minimum depth resolution at the distance between the object and the nearest elemental image zmin is given by δ ¼ h tan θd ðx2Þ zmin
ð4Þ
Here, we can calculate tan θd(x2)¼tan[(nw 1)αþθi(x2)] by using Eq. (4), where tan θi ðx2Þ ¼ tan θi ðx1Þ c=g, tan θi ðx1Þ ¼ tan ½θd ðx1Þ ðnw 1Þαand tan θd ðx1Þ ¼ h=zðminÞ , respectively. When we use a camera with a focal length of 50 mm and pixel size of 8.2 μm, the depth changes as function of the longitudinal distance of the 3D object is shown in Fig. 6. From the results of Fig. 6, the minimum depth resolution of WPADS system is better than the conventional ADS system (α¼ 01) at the angles α ¼101 and α¼ 201, respectively. Additionally, we can easily find the minimum depth resolution of the WPADS system improves as α and z increase as shown in Fig. 6.
4. Experiments and results
Fig. 6. Minimum depth resolution according to the distance between objects and nearest EI.
To demonstrate the proposed method, we performed the preliminary experiments for visualization of partially occluded 3D objects. In the experiments, a wedge prism is placed in front of a camera as shown in Fig. 7. The 3D object is a toy car which is positioned at approximately 450 mm away from the first camera. The heavy occlusion is pine tree branches located at approximately 300 mm from the sensors (see Fig. 8). The camera has an image sensor with 2400 1600 pixels, and pixel size is 8.2 μm. The camera lens has the focal length of f¼ 50 mm, the angle α between the wedge prism surfaces is 101, and the index of refraction nw of
Fig. 7. Wedge prism based axially distributed sensing system.
208
M. Zhang et al. / Optics Communications 313 (2014) 204–209
Fig. 8. Example of elemental image in the optical experiment: (a) first elemental image (b) 41st elemental image.
Fig. 9. Reconstructed images for (a) z ¼ 350 mm, (b) z ¼450 mm, (c) z¼ 700 mm.
the wedge prism (BK7 glass) is 1.52. For the computational reconstruction, the pinhole gap g is approximately at 50 mm. The camera is translated at Δz ¼5 mm increments for a total of K ¼41 elemental images and total displacement distance of 200 mm. The first and the 41st elemental images are shown in
Fig. 8. It is seen that 3D object was recorded within the entire elemental images. This is because the proposed method uses tilting of the optical axis by wedge prism. This implies that there is a higher capacity for 3D information collection for object located closed to the sensor. Thus, the WPADS system not only can
M. Zhang et al. / Optics Communications 313 (2014) 204–209
improve the resolution of the elemental images, but also can shorten the pickup distance to collect more object information compared with the ADS system. To reconstruct the partially occluded 3D object, all 41 recorded elemental images are used in the computational reconstruction algorithm as shown in Fig. 4. The 2D slice images of the 3D scene were obtained according to the reconstruction distances. Some reconstructed images are shown in Fig. 9. The reconstructed slice image shown in Fig. 9(b) is focused on the ‘car’ object. Here the distance of the reconstruction plane was 450 mm from the sensor where the ‘car’ object is originally located. From the results of Fig. 9, the proposed method was demonstrated successfully for 3D image reconstruction of partially occluded objects. 5. Conclusions In conclusion, we have presented the proposed WPADS system for 3D imaging and visualization of partially occluded objects. In this system, a wedge prism was placed in front of a camera, and the parallax information was collected through the wedge prism and the camera by translating them together along the camera's optical axis. Due to the wedge prism involved in the proposed imaging system, the 3D object can appear in the center of the field of view and improve the visual quality of the reconstructed images with the full area of the image sensor. The experimental results show that the WPADS system was effective to be used for 3D visualization of partially occluded objects. There are broad applications of this approach to object classification, automatic target recognition and 3D microscopy such as conventional ADS system [23–27]. Acknowledgments This work was supported by research fund from the Dongseo University, Dongseo Frontier Project Research Fund of 2011.
209
References [1] S.A. Benton (Ed.), Selected Papers on Three-Dimensional Displays, SPIE Optical Engineering Press, Bellingham, WA, 2001. [2] T. Okoshi, Proceedings of the IEEE 68 (5) (1980) 548. [3] G. Lippmann, Journal of Physics 7 (1908) 821. [4] F. Okano, H. Hoshino, J. Arai, I. Yuyama, Applied Optics 36 (1997) 1598. [5] R. Martinez-Cuenca, A. Pons, G. Saavedra, M. Martinez-Corral, B. Javidi, Optics Express 14 (2006) 9657. [6] B. Javidi, R. Ponce-Díaz, S.-H. Hong, Optics Letters 31 (2006) 1106. [7] B. Tavakoli, B. Javidi, E. Watson, Optics Express 16 (2008) 4426. [8] D. Shin, M. Cho, K.-C. Park, E.-S. Kim, ETRI Journal 27 (2005) 708. [9] J.-S. Park, D.-C. Hwang, D. Shin, E.-S. Kim, Optics Engineering 45 (2006) 117004. [10] I. Moon, B. Javidi, Optics Express 16 (2008) 13080. [11] B. Javidi, Y.-S. Hwang, Journal of Display Technology 4 (2008) 3. [12] R. Schulein, B. Javidi, Journal of Display Technology 4 (2008) 351. [13] G. Li, K.-C. Kwon, G.-H. Shin, J.-S. Jeong, K.-H. Yoo, N. Kim, Journal of the Optical Society of Korea 16 (2012) 381. [14] D.-C. Hwang, D. Shin, S.-C. Kim, E.-S. Kim, Applied Optics 47 (2008) D128. [15] D. Shin, B.-G. Lee, J.-J. Lee, Optics Express 16 (2008) 16294. [16] Y. Piao, D. Shin, E.-S. Kim, Optics and Lasers in Engineering 47 (2009) 1273. [17] D. Shin, H. Yoo, Optics Express 16 (2008) 8855. [18] B. Javidi, I. Moon, S. Yeom, Optics Express 14 (2006) 12096. [19] C.M. Do, R. Martínez-Cuenca, B. Javidi, Journal of the Optical Society of America 26 (2009) 245. [20] G. Baasantseren, J.-H. Park, N. Kim, Japanese Journal of Applied Physics 48 (2009) 020216. [21] Y. Piao, M. Zhang, E.-S. Kim, Optics and Lasers in Engineering 51 (2012) 1602. [22] M. Zhang, Y. Piao, E.-S. Kim, Applied Optics 50 (2011) 5369. [23] R. Schulein, M. DaneshPanah, B. Javidi, Optics Letters 34 (2009) 2012. [24] D. Shin, M. Cho, B. Javidi, Optics Letters 35 (2010) 3646. [25] D. Shin, B. Javidi, Journal of Display Technology 8 (2012) 317. [26] X. Xiao, B. Javidi, Optics Letters 36 (2011) 1086. [27] C.-W. Chen, M. Cho, Y.-P. Huang, B. Javidi, Optics Letters 37 (2012) 4125. [28] X. Xiao, B. Javidi, M. Martinez-Corral, Adrian Stern, Applied Optics 52 (2012) 546. [29] M. Cho, M. Daneshpanah, I. Moon, B. Javidi, Proceedings of the IEEE Journal 99 (2011) 556. [30] R. Martinez-Cuenca, G. Saavedra, M. Martinez-Corral, B. Javidi, Proceedings of the IEEE Journal 97 (2009) 1067. [31] Y. Rivenson, A. Rot, S. Balber, A. Stern, J. Rosen, Optics Letters 37 (2012) 1757.