- Email: [email protected]
Comparative study on light modulation characteristic between hexagonal and rectangular arranged macro lens array for integral imaging based light field display
Optics Communications 466 (2020) 125613
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Comparative study on light modulation characteristic between hexagonal and rectangular arranged macro lens array for integral imaging based light field display Jun Wen a ,1 , Xingpeng Yan a ,∗,1 , Xiaoyu Jiang a , Zhan Yan a , Ziqiang Wang a , Song Chen a , Min Lin b a b
Department of Information Communication, Army Academy of Armored Forces, Beijing 100072, China Department of Basic Education, Academy of Army Armored Forces, Beijing 100072, China
ARTICLE Keywords: Light field display Integral imaging Viewing resolution
INFO
ABSTRACT The trade-off relationship of the angular reconstruction rate and spatial reconstruction rate of the light field display system is further analyzed by taking the limited optical field of view of the lens into consideration. The comparison of the number of the perspectives and resolution of each perspective of the rectangular arranged lens array and the hexagonal arranged lens array is made, showing that the latter lens arrangement can enhance the resolution of the perspective images without notable shrinking of the number of the perspectives by improving the effectiveness of the projected pixels. The resolution enhancement is analyzed quantitatively in the frequency domain. The viewing zone shaping mechanism of the elemental image’s geometry is also presented. Numerical simulation illustrated the pixel effectiveness improvement, and optical experiment demonstrated the enhancement and superiority in the resolution of the 3D display using hexagonal arranged macro circular lens array compared with the traditional rectangular arranged lens array.
1. Introduction Three-dimensional (3D) display can bring images that we perceive in our daily life with as few alterations as possible, and it is thought to be the trend of the next generation display for it can reconstruct the depth information of the scene [1–3]. The light field display achieves this goal by reconstructing the intensity and the direction of the light from the original scene simultaneously, and it can provide full color and full parallax 3D display using commercial two-dimensional (2D) display devices and a light modulation unit, thus becoming one of the most promising display technology for the future. Integral imaging is a technique that includes two aspects of both light field recording and reconstruction, as a display technology, its imaging quality is not so ideal due to the limited bandwidth of currently available display devices and the inherent trade-off among the viewing parameters such as the resolution, viewing angle, depth range and etc. [3–6]. But the interests and endeavors never fade away to make it be a practical and promising candidate for 3D display. The newly aroused InIm based light field display system [7–10], which is usually made by substituting the micro-lens array with macrolens array and putting a holographic diffusor in a certain place in front of lens array, inherits the manifold merits of integral imaging display and can provide better 3D visual experience by reconstructing
continuous light field. However, the introducing of macro-lens array has also brought some new issues, one of the most obvious one is the effectiveness of the light modulation because of the limited field of view of the lens. Traditionally in an InIm display system, the recorded elemental images (EIs) are loaded into a 2D display panel covered by a microlens array. Generally, in most of the previous researches of the InIm based 3D display, the lens is regarded as a pinhole for simplicity. This consideration is reasonable because the lens array used in InIm for the process of both capturing and displaying the light field is relatively small, with a lens pitch less than 1 or 2 mm, some even on a level of several hundreds of microns [11,12]. The principal ray of the lens is enough for the approximation of its imaging characteristics. However, when a macro lens array is used as light modulation unit, the elemental images and the macro-lens used in the InIm based light field display system are relatively large in size, therefore the principal ray of the lens can only stand for parts of its imaging property. The distortion and aberration resulted from the non-ideal lens cannot be neglected if we want to get ideal imaging performance from the InIm based light field display system [13,14]. The lens array used in these systems is manually or mechanically assembled from ready-made high-quality macro lenses, and it is flexible in lens arrangement and cheap for large-format lens array fabrication compared to the microlens array manufactured by an expensive metal mold.
∗ Corresponding author. E-mail address: [email protected] (X. Yan). 1 These authors contributed equally.
https://doi.org/10.1016/j.optcom.2020.125613 Received 21 October 2019; Received in revised form 17 February 2020; Accepted 23 February 2020 Available online 7 March 2020 0030-4018/© 2020 Elsevier B.V. All rights reserved.
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
The basic lens geometric types (circular, hexagonal and square) in their basic arrangements (square or hexagonal grid) are used in various acquisitions and display setups. The usage of hexagonal arranged lens array for InIm is not a new topic. Actually, there are two kinds of hexagonal lens array, i.e. the hexagonal arranged circular lens array [15] as well as the hexagonal arranged hexagonal lens array [16,17]. Generally speaking, a hexagonal grid provides higher packing density and gives a more accurate approximation of circular regions compared with the rectangular grid. Besides, the sampling points of the hexagonal grid are uniformly connected in the sense that each sampling point is located at a fixed same distance to all the other six adjacent pixels, also, circularly band-limited signals are sampled more efficiently by hexagonal grids than by rectangular grids [18,19]. Previous researches in hexagonal lens array focused on the enhancement of spatial-angular light ray sampling and the influence on the resolution in integral-imaging-based hologram [20], but its impacts on light field reconstruction are not fully analyzed yet. In this paper, we further the current study on the light modulation characteristics of macro lens array in the InIm based light field display by treating the elemental macro lens as a paraxial imaging optics. The limited field of view (FOV) of the lens is taken into consideration for the optimum design of the LFD, and we propose to arrange the circular macro lenses in a hexagonal way to maximize the effective pixels covered by the lens for better display with enhanced resolution. The shape of EI is also hexagonal for seamless coverage of the display. The effectiveness of pixel projection is illustrated by numerical simulation, the resolution enhancement of reconstructed images is analyzed quantitatively in terms of angular frequency, and the viewing zone shaping mechanism of the EI’s geometry is also presented as a collateral effect. Optical experiments are conducted and the reconstructed images verify the validity and superiority of the proposed method.
the bandwidth of the display panel. The greater-than sign means that there are some ineffective pixels in the display system because of the improper system configuration or the limited FOV of the lens. Since the lens is a paraxial imaging optics, ineffective pixels are common in the light field display with large EI. The parallax between two adjacent perspectives can be expressed as difference ratio 𝐷𝑟 in the following way for simplification. 𝑝𝑝 (2) 𝐷𝑟 = 𝑔 where 𝑝𝑝 is the pixel pitch of the 2D display panel, and 𝑔 is the gap between the display panel and the LA. A reasonable 𝐷𝑟 is necessary to produce fluent motion parallax and acceptable 3D resolution. With the increasing demand for larger viewing angle, EI with larger size and lens with bigger pitch are inevitable. However, this arrangement will greatly sacrifice the spatial resolution, causing sparsely reconstructed light field in space even using the holographic diffusor. Research in light field acquisition reveals that the optimal sampling rate of the 3D scene is closely related to its complexity [20]. One can intuitionally know that the insufficient spatial reconstruction rate will be harmful to the reproduction of the high-frequency information of the scene. We analyze the phenomenon in terms of the effectiveness of the pixel projection and the characteristics of the reconstructed images in the frequency domain. 2.2. The resolution improvement of perspective images with a hexagonal arranged LA in term of pixel effectiveness When the pinhole model is taken for the lens, one cannot increase the number of the perspectives and the resolution of the perspective images simultaneously with a given display panel. However, when the lens is taken as a paraxial imaging optics with finite aperture, one may be able to improve the resolution of perspective images without distinct shrinking of the number of perspectives by replacing the traditionally rectangular arranged LA into a hexagonal arranged way, which can turn part of the ineffective pixels (originally out of the FOV of the lens) into effective pixels (within the FOV of the lens). This is especially true when the FOV of the lens is limited by a small F-number and the number of the ineffective pixels is large, which is often the case for most commercial lens. The effective imaging area covered by the circular lens in the display panel is also circular because of the circular symmetry of lens optical geometry. This circular area is closely related to the FOV of the lens and the gap between the LA and the DP, specifically, the radius of the effective area can be calculated as follows:
2. Principle 2.1. Spatio-angular resolution tradeoff in InIm based LFD The basic structure of the InIm based LFD is depicted as Fig. 1(a), it is quite similar to the traditional InIm display system, where the elemental image array (EIA) is loaded on a 2D display panel (DP) covered by a lens-array (LA). Besides, a holographic diffusor with a proper diffusion angle located at a certain distance in front of the LA is used to re-modulate the light distribution and get a continuous light field. The perspective or the viewpoint of InIm based LFD is defined as a set of rays with same propagating direction (as depicted in Fig. 1(b)). The interval of the two adjacent perspectives, 𝜃, is considered as the angular reconstruction interval, and the number of perspectives or the viewpoints is understood as angular resolution. The reconstructed angular range is closely related to the number of the pixels covered by each lens and the gap between the display panel and LA. The spacing between two adjacent lenses, 𝐿p , is considered as the spatial reconstruction interval, and the imaging resolution of each reconstructed perspective is recognized as the spatial resolution of the LFD, from the point of view of the observer. It obvious that there is an inherent tradeoff relation between the spatial and the angular resolutions, i.e. the angular resolution is increased at the cost of the reduction of spatial resolution under a certain bandwidth of the display, and vice versa. This tradeoff relationship is expressed as the contradictory between the number of the perspectives (related to the number of pixels of each EI) and the resolution of each perspective (related to the number of the lens) in terms of the hardware. Considering the total pixel number of the display panel is 𝑇 , the number of perspectives is 𝑁v and the pixel number of each perspective is 𝑅v , the following relationship is easy to see: 𝑇 ≥ 𝑁 v × 𝑅v
𝑟 = 𝑔 × tan 𝛽
(3)
where 𝛽 is the half of the FOV of the lens, and g is the gap between the LA and the DP. For most of the LFD based on InIm with LA consisted of single lenses, when the gap between the LA and DP equals to the focal length of the lens, i.e. the display system is working in focusing mode, and the effectively covered area on the DP cannot over the circle with a diameter of the lens itself (i.e. the lens aperture). However, the pitch of the lens cannot be smaller than the diameter of the lens because of the physical constraint regardless of the arrangement of the lens. That means part of the pixels are ineffective for the display and are wasted. Fig. 2 illustrates the effectively covered pixels in DP with rectangular lens array (RLA) and hexagonal lens array (HLA) when the radius of the effective area is smaller than the pitch of the lens. It is supposed that the whole DP is seamlessly covered by EIA and some parts of the EI are incomplete when they are located at the boundary of the DP. Note that when the lens is arranged in a regular grid, the shape of EI is rectangle, referred as REI (see Fig. 2(a)), while the lens is arranged in a hexagonal way, the shape of EI is also hexagon, referred as HEI (see Fig. 2(b)). Generally, the defined lens pitch for hexagonal arranged lens array is not equal horizontally and vertically, which is shown in Fig. 2(b) as 𝑝ℎ
(1)
It implies that one cannot increase both the number of the perspectives and the resolution of the perspective images without improving 2
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 1. (a) Sketch map of the system structure of the InIm based LFD, (b) spatial-angular light rays’ distribution of the reconstructed light field without considering the holographic diffusor.
Fig. 2. Effectively covered area using (a) RLA and (b) HLA.
and 𝑝𝑣 respectively. In order to give a comparable standard of the two kinds of the lens array, the lens pitch of RLA and HLA in this paper are both defined as the diameter of the inscribed circle of the EI, regardless of its shape. The geometric characteristics of the previous mentioned RLA are that the elementary squares are oriented along the side of the square, while in the HLA, the elementary hexagons are oriented along of the axis passing through the farthest vertices of the hexagon, which can
also be referred as the ‘‘horizontal’’ HLA as well. There are other variations for the layout of macro lens based on different geometric shapes, at least one obvious layout exists for each shape, i.e. the squares oriented along the diagonal of the square and the hexagons oriented orthogonally to their longest axis. However, these layouts can be obtained by rotating the current setups in an appropriate angle, and it makes no difference to the analysis in the paper in terms of pixel effectiveness in this subsection and frequency enhancement in the latter 3
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
clearly distinguish that in the RLA and HLA with the same pitch and effective area’s radius, the effectiveness of the pixels of the latter is not smaller than the former. That means in most of the case, arranging the circular lens in a hexagonal way can get higher pixel effectiveness. In fact, by substituting the REI with HEI, the number of ineffective pixels can be significantly decreased, also the number of perspectives (𝑁v ) is increased without a notable margin of the pixel number of the perspectives images (𝑅v ), which is meaningful for high-quality 3D image reconstruction.
subsection because of the symmetry of the effective area. While other setups, like non-square RLA with some aspect ratio, is out of the scope of this paper. Actually, the effectiveness of the pixels is closely relevant to the relationship between the radius of the effective area and the lens pitch, which is shown in Fig. 3. For both the RLA and the HLA case, there are three situations for the relative relationship between the effective area and the lens pitch, i.e. the effective area is totally within the EI, the EI is totally within the effective area, and neither the effective area nor the EI is totally within the other. Which is illustrated as the left, right and middle case of Fig. 3(a) and (b). The effectiveness of the pixel of the RLA can be expressed as:
𝑒𝑓 𝑓RLA
⎧ 𝜋𝑟2 ⎪ 𝑝2 , ⎪ = ⎨ 𝜋𝑟2 −4𝑠1 , ⎪ 𝑝2 ⎪1, ⎩
2.3. Reconstruction enhancement in terms of angular frequency
𝑟 ≤ 𝑅RI 𝑅RI < 𝑟 < 𝑅RC
Fig. 6 shows the image reconstruction characteristics in the spatial domain, the location of the lens is expressed by the optical center of it. The horizontal and vertical interval of reconstruction, referred as 𝛥𝑥 and 𝛥𝑦 in Fig. 6(a) and (b), is defined as the distances between two adjacent columns and two adjacent rows respectively. Consider the original light field distribution of the object in the lens array plane is 𝑂(𝑥, 𝑦), here, the representation of the light field covers only the spatial dimensions, the angular dimensions are considered implicitly for simplicity. Since the resolution of each orthographic view of the reconstructed light field is the same, the spatial reconstruction of the light field is angularly invariant. The reconstructed light field can be given by the following equations: ∑ 𝑂̄ RLA (𝑥, 𝑦) = 𝑂(𝑥, 𝑦)𝛿(𝑥 − 𝑚𝛥𝑥)𝛿(𝑦 − 𝑛𝛥𝑦) (6)
(4)
𝑟 ≥ 𝑅RC
where 𝑟 refers to the radius of the effective area, 𝑝 refers to the lens pitch (𝑝 = 𝑝𝑣 = 𝑝ℎ ), and 𝑠1 means the area out of the corresponding REI region. 𝑅RI is the radius of the inscribed circle of the REI, and 𝑅RC is the radius of the circumcircle of the REI. Similarly, the effectiveness of the pixel of the HLA can be expressed as:
𝑒𝑓 𝑓HLA
⎧ 2𝜋𝑟2 ⎪ √3𝑝2 , ⎪ = ⎨ 2(𝜋𝑟√2 −6𝑠2 ) , ⎪ 3𝑝2 ⎪1, ⎩
𝑟 ≤ 𝑅HI 𝑅HI < 𝑟 < 𝑅HC
(5)
𝑟 ≥ 𝑅HC
𝑚,𝑛
𝑂̄ HLA (𝑥, 𝑦) =
where 𝑠2 denotes the covered area √ out of the corresponding HEI, 𝑝 is the side length of HEI (𝑝 = 𝑝𝑣 ∕ 3 = 2𝑝ℎ ∕3). 𝑅HI is the radius of the inscribed circle of the HEI, and 𝑅HC is the radius of the circumcircle of the HEI. Both 𝑠1 and 𝑠2 can be easily obtained by simple calculation of an integral operation. To give a quantitative idea of the above analysis and further verify it, a numerical simulation is made. Taking a commercial 4K (3840 × 2160) 12.5-inch 2D DP into consideration, the basic parameters of the DP are as following, the pixel pitch is 72 μm, the effective display width is 277 mm horizontally and 156 mm vertically. The resolution of the perspective images is defined as the total number of pixels for convenience because the pixels’ distribution in the HLA perspective images is not in the regular grid horizontally and vertically. First, when the effective area is large enough or the limited FOV of the lens is not considered, the number of the perspectives and the resolution of the perspective images are only related to the pitch of the lens. Fig. 4 shows the variation trend of the number of perspectives and the resolution of each perspective when the lens is arranged in a regular grid and in a hexagonal way respectively. From the general trend of Fig. 4 we can see that the resolution of the perspective is inversely related to the pitch of the lens generally, and the number of the perspectives is positively related to it, regardless of the arrangement manner of the LA. That is the tradeoff phenomenon between the angular and spatial resolution revealed in the last subsection. When the limited FOV of the lens is taken into consideration, the increase of resolution for each perspective can be achieved without distinguishing shrinking of the effective number of the perspectives by turning the original ineffective pixels into effective, this can be shown by the effectiveness of pixels under HLA and RLA with same lens pitch and same FOV. Since the effectiveness of the pixels is a combined action of the FOV of the lens 𝛽 and the gap g between the LA and the DP, the radius of the effective area (see Eq. (3)) is used to measure the combined effect on it. Fig. 5 depicted the effectiveness of the pixels in DP with various lens pitch for both RLA and HLA. Generally speaking, when the radius of the effective area becomes large enough, the effectiveness of the pixel reaches its maximum, but one can
∑
𝑂 (𝑥, 𝑦) 𝛿 (𝑥 − 𝑚𝛥𝑥) 𝛿 [𝑦 − (𝑚 + 2𝑛) 𝛥𝑦]
(7)
𝑚,𝑛
where 𝑚 and 𝑛 stand for the sequential number of the lens in the horizontal and vertical direction respectively. 𝛿(⋅) represents the Dirac delta function. The reconstructed light field of the image by RLA and HLA are shown in Eqs. (6) and (7). According to the lens pitch defined in the previous subsection, the reconstruction interval has the following relationship. For RLA, there is 𝛥𝑥 = 𝛥𝑦 = 𝑝. As for HLA, there are √ 𝛥𝑥 = 3𝑝∕2 and 𝛥𝑦 = 𝑝∕2. The comparison of the reconstructed light field in the frequency domain can be obtained by Fourier transform of the above equations, which is: ) ( ) ( ∑ [ ] 𝑛 𝑚 𝛿 𝑓𝑦 − (8) ℱ 𝑂̄ 𝑅𝐿𝐴 (𝑥, 𝑦) = ℱ [𝑂(𝑥, 𝑦)] ⊗ 𝛿 𝑓𝑥 − 𝑝 𝑝 𝑚,𝑛 ( ) ( ) ∑ [ ] 𝑚 𝑚 + 2𝑛 ℱ 𝑂̄ 𝑅𝐿𝐴 (𝑥, 𝑦) = ℱ [𝑂(𝑥, 𝑦)] ⊗ 𝛿 𝑓𝑥 − √ (9) 𝛿 𝑓𝑦 − 𝑝 𝑚,𝑛 3𝑝 Here, ⊗ means the operation of convolution. ℱ stands for the Fourier transform operation. The above two equations indicate that the spectrum of the object field is duplicated with different manner related to the arrangement of the lens array. A more specific standard to measure the performance in the frequency domain can be indicated by the maximum radial spatial frequency that can be reconstructed without aliasing, which can be represented by: √ 1 𝑓𝜌,RLA = 𝑓𝑥2 + 𝑓𝑦2 ≤ (10) 2𝑝 √ 1 𝑓𝜌,HLA = 𝑓𝑥2 + 𝑓𝑦2 ≤ √ (11) 3𝑝 The above two equations reveal that the maximum spatial frequency √ is 𝑓𝜌,RLA = 1∕(2𝑝) for RLA and 𝑓𝜌,RLA = 1∕( 3𝑝) for HLA accordingly. So theoretically speaking, compared with the RLA case, the resolution √ enhancement of a ratio of 2∕ 3 is a reasonable expectation for the HLA based light field display. We verified this resolution enhancement later in our optical experiment visually. 4
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 3. The relationship between the effectively covered pixels and the EI, (a) for the RLA case, (b) for the HLA case.
Fig. 4. The general variation trend of the perspective number and its corresponding resolution for different lens pitch without considering the FOV of the lens and the effectiveness of the pixels.
Fig. 5. The effectiveness of pixels in DP with consideration of the lens’ FOV, (a) for RLA, (b) for HLA.
Fig. 6. The image reconstruction in the spatial domain for (a) RLA, (b) HLA.
5
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 7. The viewing zone shaped by the (a) REI, (b) HEI, and the detailed characteristics of view volume using (c) REI and (d) HEI.
Fig. 8. (a) General system structure of the fabricated prototype, (b) the fabricated RLA, (c) the fabricated HLA.
𝛥𝑡 vertically respectively. The horizontal and vertical angular interval of the adjacent view can be noted as 𝛥𝜃 and 𝛥𝜙 respectively and the maximum horizontal and vertical angular range can be signed as 𝜃max and 𝜙max accordingly for both the REI (see Fig. 7(c)) and the HEI (see Fig. 7(d)) case. It is obvious that when the FOV of the lens is large enough, the expressible angular range for the HEI case is larger than the REI case. When the diameter of the lens is 10 mm and the distance between the LA and the DP is set to be 10 mm, the maximin expressible angular range can be approximately 53◦ for the RLA case and 60◦ for the HLA case, supposing that the optical FOV of the lens is large enough and the DP is seamlessly covered with REI and HEI respectively, so there is a distinct viewing angle improvement by arranging the lens in a hexagonal way.
2.4. Viewing zone shaping mechanism of the EI The above analysis focuses on the number of the perspective and the resolution of each perspective, but one cannot omit the changing of the EI’s geometry and its’ effect on viewing zone shaping. Although the light ray emanated by each single pixel is modulated to be conical because of the shape of the lens. The shape of the view volume of each single EI is determined by the shape of the EI, i.e. a rectangular pyramid for REI (see Fig. 7(a)) and a hexagonal pyramid for HEI (see Fig. 7(b)), with their summits in touch at the center of the lens. The directions of perspective are represented as the principal ray of an individual lens. Take one EI for instance, suppose that the EI image is in the s-t plane, the pixel pitch can be expressed as 𝛥𝑠 horizontally and 6
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 9. The original 3D scene for RLA and HLA system, (a) the letter scene, (b) the play card scene.
Fig. 10. The front view of the ‘‘LAB’’ scene from (a) the HLA display system, (b) the RLA display system.
Fig. 11. The front view of the play cards scene from (a) the HLA display system, (b) the RLA display system.
The viewing zone of the whole system is defined as the common areas of the imbricated view volume, which will also be a rectangular pyramid and a hexagonal pyramid for REI and HEI respectively. Despite the fact that the HLA can bring in resolution enhancement and maximum expressible angular range improvement, there is no final conclusion on whether a hexagonal viewing zone is superior to a rectangular viewing zone, but we cannot omit this phenomenon as a side effect of the arranging the LA in a hexagonal way.
is higher than the RLA, when the covered display area is equivalent, the number of lens in the HLA is also larger than the RLA. Two scenes are used in the experiments. First a simple scene without much high-frequency information, which is made up of three letters, a red ‘L’, a green ‘A’ and a blue ‘B’, as shown in Fig. 9(a), and they are set as 𝑑𝐿 = 140 mm, 𝑑𝐴 = 150 mm, and 𝑑𝐵 = 160 mm away from the lens array respectively. As a contrast, three play cards, i.e. a heart, spade, diamond ‘k’, which we think is full of high frequency information compared with the former scene, is put in the same location, as depicted in Fig. 9(b). The EIA is generated using the ray tracing algorithm according to the lens array parameters used in the display process. The display results of the scenes are demonstrated in Fig. 10 and Fig. 11 for the ‘‘LAB’’ scene and play card scene respectively. From Fig. 10, we can clearly see that although the scene is relatively simple. The boundary blur for the RLA display system is quite distinct compared with the HLA display system. Similar result can also see in the display result of the play cards scene, see Fig. 11, the enhancement of resolution for the display is quite significant. The above outcomes are display results from the quasi-3D scene, which means the scene is made up of 2D plane images located in different depth, the main merit of such scenes is that one can clearly distinguish the resolution enhancement between the two systems. To further verify the effectiveness of the proposed method, a 3D scene made up of three dices with different pose are used, the scene is relatively simple without much high frequency information. The display results are shown in Fig. 12, from the front view of the display, we can
3. Experiments and discussions To verify the resolution improvement of the HLA, two prototypes were fabricated, the overall structure of the display system is shown in Fig. 8(a), A compound lens array, a high-resolution DP, and a holographic diffusor with a glass substrate. The size of the DP is 12.5 inch in diagonal, with a resolution of 3840 × 2160, the pixel pitch of the DP is approximately 72 μm. The gap between the DP and the optical center of the lens array is approximately 10.7 mm, the holographic diffusor with glass substrate is put parallelly 150 mm away from the lens array. The compound lens array is arranged in the following two ways, first, the traditional rectangular arranged one, which includes 22 lenses in column and 12 lenses in row, as shown in Fig. 8(b). The display system with this lens array is referred as the RLA system. Second, the updated hexagonal arrange one, with a total of 312 (12 × 26) lenses, as shown in Fig. 8(c). The display system with this lens array is referred as the HLA system. Since the filling rate of lens in the HLA 7
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 12. The front view of the dices scene from (a) the HLA display system, (b) the RLA display system.
Fig. 13. The front view of the tank scene from (a) the HLA display system, (b) the RLA display system.
Funding
distinguish obvious resolution enhancement in the HLA display system. Another 3D scene used in the experiment is a tank full of high frequency information, as depicted in Fig. 13, the HLA system is superior in the homogeneity of the displayed image and in detail presentation. The tail part of the tank is blurred in the RLA display system while it is clearly visible in the HLA display system. The previous analysis in √ the frequency domain reveals a resolution enhancement of a ratio of 2∕ 3, however, we can only verify this quantitative improvement in terms of visual quality, which we believe that the resolution enhancement of the HLA system is fully verified by the experimental results presented.
The National Key Research and Development Program of China (2017YFB1104500), National Natural Science Foundation of China (61775240), Foundation for the Author of National Excellent Doctoral Dissertation of the People’s Republic of China (FANEDD) (201432). References [1] J.-Y. Son, H. Lee, B.-R. Lee, K.-H. Lee, Holographic and light field imaging as future 3-D displays, Proc. IEEE 105 (5) (2017) 789–804. [2] H. Zhang, Y. Zhao, L. Cao, G. Jin, Three-dimensional display technologies in wave and ray optics: a review (Invited Paper), Chin. Opt. Lett. 12 (6) (2014) 060002–060007. [3] C.G. Luo, X. Xiao, M. Martinez-Corral, C.W. Chen, B. Javidi, Q.H. Wang, Analysis of the depth of field of integral imaging displays based on wave optics, Opt. Express 21 (25) (2013) 31263–31273. [4] R.R. Tamboli, B. Appina, S. Channappayya, S. Jana, Super-multiview content with high angular resolution: 3D quality assessment on horizontal-parallax lightfield display, Signal Process Image 47 (2016) 42–55. [5] A.J. Woods, H.-S. Kim, N.S. Holliman, H.-E. Kim, K.-M. Jeong, G.E. Favalora, S.-I. Hong, J.-H. Park, Analysis on ray reconstruction characteristics of multiview and integral imaging display, in: Stereoscopic Displays and Applications, Vol. XXIII, 2012. [6] A.J. Woods, A. Said, N.S. Holliman, E.-V. Talvala, J.O. Merritt, Spatial-angular analysis of displays for reproduction of light fields, in: Proc. SPIE 7237, Stereoscopic Displays and Applications, Vol. XX, 2009, p. 723707. [7] Y. Zhu, X. Sang, X. Yu, P. Wang, S. Xing, D. Chen, B. Yan, K. Wang, C. Yu, Wide field of view tabletop light field display based on piece-wise tracking and off-axis pickup, Opt. Commun. 402 (2017) 41–46. [8] S. Yang, X. Sang, X. Yu, X. Gao, L. Liu, B. Liu, L. Yang, 162-inch 3D light field display based on aspheric lens array and holographic functional screen, Opt. Express 26 (25) (2018) 33013–33021. [9] X. Sang, F.C. Fan, C.C. Jiang, S. Choi, W. Dou, C. Yu, D. Xu, Demonstration of a large-size real-time full-color three-dimensional display, Opt. Lett. 34 (24) (2009) 3803–3805. [10] Z. Yan, X. Yan, X. Jiang, H. Gao, J. Wen, Integral imaging based light field display with enhanced viewing resolution using holographic diffuser, Opt. Commun. 402 (2017) 437–441. [11] J. Arai, H. Kawai, F. Okano, Microlens arrays for integral imaging system, Appl. Opt. 45 (36) (2006). [12] R.F. Stevens, N. Davies, Lens arrays and photography, J. Photogr. Sci. 39 (5) (1991) 199–208. [13] W. Zhang, X. Sang, X. Gao, X. Yu, B. Yan, C. Yu, Wavefront aberration correction for integral imaging with the pre-filtering function array, Opt. Express 26 (21) (2018) 27064–27075. [14] X. Gao, X. Sang, X. Yu, W. Zhang, B. Yan, C. Yu, Aberration improvement of the floating 3D display system based on Tessar array and directional diffuser screen, Opt. Rev. 25 (4) (2018) 500–508.
4. Conclusion We present a comparative study on the resolution enhancement of the lens array arrangement. The most obvious and intuitive merit of such setup is the improvement of the filling rate. Besides, the tradeoff relationship between the angular reconstruction rate and spatial reconstruction rate inherent in the light field display is further revealed by taking the lens as a paraxial imaging optic. Numerical simulation reveals that the proposed method can effectively improve the validity of the covered pixels when the limited FOV of the lens is taking into consideration. The resolution enhancement is also measured in the frequency √ domain, which reveals a resolution enhancement of a ratio of 2∕ 3. Optical experiments show distinct resolution refinement of the proposed method, which means it is superior in detail presentation and display homogeneity compared with the tradition RLA display system. This method is especially useful when the scene reconstructed is complicated and full of the high frequency information, which is common in our daily scene. We think our report will help to the extension and promotion of the practical use of the InIm based LFD. CRediT authorship contribution statement Jun Wen: Methodology, Investigation, Validation, Writing - original draft. Xingpeng Yan: Conceptualization, Validation, Formal analysis, Visualization. Xiaoyu Jiang: Validation, Formal analysis, Visualization. Zhan Yan: Resources, Writing - review & editing, Supervision. Ziqiang Wang: Resources, Writing - review & editing, Supervision. Song Chen: Writing - review & editing. Min Lin: Writing - review & editing. Acknowledgments Special thanks should give to Ping Zhong for the helpful discussion on the construction of the model and the rendering of the images. 8
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613 [18] A. Karimzadeh, Analysis of the depth of field in hexagonal array integral imaging systems based on modulation transfer function and Strehl ratio, Appl. Opt. 55 (11) (2016) 3045–3050. [19] N. Chen, J. Yeom, J.H. Jung, J.H. Park, B. Lee, Resolution comparison between integral-imaging-based hologram synthesis methods using rectangular and hexagonal lens arrays, Opt. Express 19 (27) (2011) 26917–26927. [20] Y. Sheng, J.-H. Park, C. Yu, D. Han, N. Kim, L. Chen, Capture of the threedimensional information based on integral imaging and its sampling analysis, in: Holography, Diffractive Optics, and Applications, Vol. IV, 2010.
[15] S. Manolache, A. Aggoun, M. McCormick, N. Davies, S.Y. Kung, Analytical model of a three-dimensional integral image recording system that uses circular- and hexagonal-based spherical surface microlenses, J. Opt. Soc. Amer. A 18 (8) (2001) 1814–1821. [16] A.J. Woods, K. Yanaka, N.S. Holliman, J.O. Merritt, Integral photography using hexagonal fly’s eye lens and fractional view, in: Stereoscopic Displays and Applications, Vol. XIX, 2008. [17] S.S. Athineos, Photorealistic integral photography using a ray-traced model of capturing optics, J. Electron. Imaging 15 (4) (2006) 0430071–0430078.
9
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Comparative study on light modulation characteristic between hexagonal and rectangular arranged macro lens array for integral imaging based light field display Jun Wen a ,1 , Xingpeng Yan a ,∗,1 , Xiaoyu Jiang a , Zhan Yan a , Ziqiang Wang a , Song Chen a , Min Lin b a b
Department of Information Communication, Army Academy of Armored Forces, Beijing 100072, China Department of Basic Education, Academy of Army Armored Forces, Beijing 100072, China
ARTICLE Keywords: Light field display Integral imaging Viewing resolution
INFO
ABSTRACT The trade-off relationship of the angular reconstruction rate and spatial reconstruction rate of the light field display system is further analyzed by taking the limited optical field of view of the lens into consideration. The comparison of the number of the perspectives and resolution of each perspective of the rectangular arranged lens array and the hexagonal arranged lens array is made, showing that the latter lens arrangement can enhance the resolution of the perspective images without notable shrinking of the number of the perspectives by improving the effectiveness of the projected pixels. The resolution enhancement is analyzed quantitatively in the frequency domain. The viewing zone shaping mechanism of the elemental image’s geometry is also presented. Numerical simulation illustrated the pixel effectiveness improvement, and optical experiment demonstrated the enhancement and superiority in the resolution of the 3D display using hexagonal arranged macro circular lens array compared with the traditional rectangular arranged lens array.
1. Introduction Three-dimensional (3D) display can bring images that we perceive in our daily life with as few alterations as possible, and it is thought to be the trend of the next generation display for it can reconstruct the depth information of the scene [1–3]. The light field display achieves this goal by reconstructing the intensity and the direction of the light from the original scene simultaneously, and it can provide full color and full parallax 3D display using commercial two-dimensional (2D) display devices and a light modulation unit, thus becoming one of the most promising display technology for the future. Integral imaging is a technique that includes two aspects of both light field recording and reconstruction, as a display technology, its imaging quality is not so ideal due to the limited bandwidth of currently available display devices and the inherent trade-off among the viewing parameters such as the resolution, viewing angle, depth range and etc. [3–6]. But the interests and endeavors never fade away to make it be a practical and promising candidate for 3D display. The newly aroused InIm based light field display system [7–10], which is usually made by substituting the micro-lens array with macrolens array and putting a holographic diffusor in a certain place in front of lens array, inherits the manifold merits of integral imaging display and can provide better 3D visual experience by reconstructing
continuous light field. However, the introducing of macro-lens array has also brought some new issues, one of the most obvious one is the effectiveness of the light modulation because of the limited field of view of the lens. Traditionally in an InIm display system, the recorded elemental images (EIs) are loaded into a 2D display panel covered by a microlens array. Generally, in most of the previous researches of the InIm based 3D display, the lens is regarded as a pinhole for simplicity. This consideration is reasonable because the lens array used in InIm for the process of both capturing and displaying the light field is relatively small, with a lens pitch less than 1 or 2 mm, some even on a level of several hundreds of microns [11,12]. The principal ray of the lens is enough for the approximation of its imaging characteristics. However, when a macro lens array is used as light modulation unit, the elemental images and the macro-lens used in the InIm based light field display system are relatively large in size, therefore the principal ray of the lens can only stand for parts of its imaging property. The distortion and aberration resulted from the non-ideal lens cannot be neglected if we want to get ideal imaging performance from the InIm based light field display system [13,14]. The lens array used in these systems is manually or mechanically assembled from ready-made high-quality macro lenses, and it is flexible in lens arrangement and cheap for large-format lens array fabrication compared to the microlens array manufactured by an expensive metal mold.
∗ Corresponding author. E-mail address: [email protected] (X. Yan). 1 These authors contributed equally.
https://doi.org/10.1016/j.optcom.2020.125613 Received 21 October 2019; Received in revised form 17 February 2020; Accepted 23 February 2020 Available online 7 March 2020 0030-4018/© 2020 Elsevier B.V. All rights reserved.
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
The basic lens geometric types (circular, hexagonal and square) in their basic arrangements (square or hexagonal grid) are used in various acquisitions and display setups. The usage of hexagonal arranged lens array for InIm is not a new topic. Actually, there are two kinds of hexagonal lens array, i.e. the hexagonal arranged circular lens array [15] as well as the hexagonal arranged hexagonal lens array [16,17]. Generally speaking, a hexagonal grid provides higher packing density and gives a more accurate approximation of circular regions compared with the rectangular grid. Besides, the sampling points of the hexagonal grid are uniformly connected in the sense that each sampling point is located at a fixed same distance to all the other six adjacent pixels, also, circularly band-limited signals are sampled more efficiently by hexagonal grids than by rectangular grids [18,19]. Previous researches in hexagonal lens array focused on the enhancement of spatial-angular light ray sampling and the influence on the resolution in integral-imaging-based hologram [20], but its impacts on light field reconstruction are not fully analyzed yet. In this paper, we further the current study on the light modulation characteristics of macro lens array in the InIm based light field display by treating the elemental macro lens as a paraxial imaging optics. The limited field of view (FOV) of the lens is taken into consideration for the optimum design of the LFD, and we propose to arrange the circular macro lenses in a hexagonal way to maximize the effective pixels covered by the lens for better display with enhanced resolution. The shape of EI is also hexagonal for seamless coverage of the display. The effectiveness of pixel projection is illustrated by numerical simulation, the resolution enhancement of reconstructed images is analyzed quantitatively in terms of angular frequency, and the viewing zone shaping mechanism of the EI’s geometry is also presented as a collateral effect. Optical experiments are conducted and the reconstructed images verify the validity and superiority of the proposed method.
the bandwidth of the display panel. The greater-than sign means that there are some ineffective pixels in the display system because of the improper system configuration or the limited FOV of the lens. Since the lens is a paraxial imaging optics, ineffective pixels are common in the light field display with large EI. The parallax between two adjacent perspectives can be expressed as difference ratio 𝐷𝑟 in the following way for simplification. 𝑝𝑝 (2) 𝐷𝑟 = 𝑔 where 𝑝𝑝 is the pixel pitch of the 2D display panel, and 𝑔 is the gap between the display panel and the LA. A reasonable 𝐷𝑟 is necessary to produce fluent motion parallax and acceptable 3D resolution. With the increasing demand for larger viewing angle, EI with larger size and lens with bigger pitch are inevitable. However, this arrangement will greatly sacrifice the spatial resolution, causing sparsely reconstructed light field in space even using the holographic diffusor. Research in light field acquisition reveals that the optimal sampling rate of the 3D scene is closely related to its complexity [20]. One can intuitionally know that the insufficient spatial reconstruction rate will be harmful to the reproduction of the high-frequency information of the scene. We analyze the phenomenon in terms of the effectiveness of the pixel projection and the characteristics of the reconstructed images in the frequency domain. 2.2. The resolution improvement of perspective images with a hexagonal arranged LA in term of pixel effectiveness When the pinhole model is taken for the lens, one cannot increase the number of the perspectives and the resolution of the perspective images simultaneously with a given display panel. However, when the lens is taken as a paraxial imaging optics with finite aperture, one may be able to improve the resolution of perspective images without distinct shrinking of the number of perspectives by replacing the traditionally rectangular arranged LA into a hexagonal arranged way, which can turn part of the ineffective pixels (originally out of the FOV of the lens) into effective pixels (within the FOV of the lens). This is especially true when the FOV of the lens is limited by a small F-number and the number of the ineffective pixels is large, which is often the case for most commercial lens. The effective imaging area covered by the circular lens in the display panel is also circular because of the circular symmetry of lens optical geometry. This circular area is closely related to the FOV of the lens and the gap between the LA and the DP, specifically, the radius of the effective area can be calculated as follows:
2. Principle 2.1. Spatio-angular resolution tradeoff in InIm based LFD The basic structure of the InIm based LFD is depicted as Fig. 1(a), it is quite similar to the traditional InIm display system, where the elemental image array (EIA) is loaded on a 2D display panel (DP) covered by a lens-array (LA). Besides, a holographic diffusor with a proper diffusion angle located at a certain distance in front of the LA is used to re-modulate the light distribution and get a continuous light field. The perspective or the viewpoint of InIm based LFD is defined as a set of rays with same propagating direction (as depicted in Fig. 1(b)). The interval of the two adjacent perspectives, 𝜃, is considered as the angular reconstruction interval, and the number of perspectives or the viewpoints is understood as angular resolution. The reconstructed angular range is closely related to the number of the pixels covered by each lens and the gap between the display panel and LA. The spacing between two adjacent lenses, 𝐿p , is considered as the spatial reconstruction interval, and the imaging resolution of each reconstructed perspective is recognized as the spatial resolution of the LFD, from the point of view of the observer. It obvious that there is an inherent tradeoff relation between the spatial and the angular resolutions, i.e. the angular resolution is increased at the cost of the reduction of spatial resolution under a certain bandwidth of the display, and vice versa. This tradeoff relationship is expressed as the contradictory between the number of the perspectives (related to the number of pixels of each EI) and the resolution of each perspective (related to the number of the lens) in terms of the hardware. Considering the total pixel number of the display panel is 𝑇 , the number of perspectives is 𝑁v and the pixel number of each perspective is 𝑅v , the following relationship is easy to see: 𝑇 ≥ 𝑁 v × 𝑅v
𝑟 = 𝑔 × tan 𝛽
(3)
where 𝛽 is the half of the FOV of the lens, and g is the gap between the LA and the DP. For most of the LFD based on InIm with LA consisted of single lenses, when the gap between the LA and DP equals to the focal length of the lens, i.e. the display system is working in focusing mode, and the effectively covered area on the DP cannot over the circle with a diameter of the lens itself (i.e. the lens aperture). However, the pitch of the lens cannot be smaller than the diameter of the lens because of the physical constraint regardless of the arrangement of the lens. That means part of the pixels are ineffective for the display and are wasted. Fig. 2 illustrates the effectively covered pixels in DP with rectangular lens array (RLA) and hexagonal lens array (HLA) when the radius of the effective area is smaller than the pitch of the lens. It is supposed that the whole DP is seamlessly covered by EIA and some parts of the EI are incomplete when they are located at the boundary of the DP. Note that when the lens is arranged in a regular grid, the shape of EI is rectangle, referred as REI (see Fig. 2(a)), while the lens is arranged in a hexagonal way, the shape of EI is also hexagon, referred as HEI (see Fig. 2(b)). Generally, the defined lens pitch for hexagonal arranged lens array is not equal horizontally and vertically, which is shown in Fig. 2(b) as 𝑝ℎ
(1)
It implies that one cannot increase both the number of the perspectives and the resolution of the perspective images without improving 2
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 1. (a) Sketch map of the system structure of the InIm based LFD, (b) spatial-angular light rays’ distribution of the reconstructed light field without considering the holographic diffusor.
Fig. 2. Effectively covered area using (a) RLA and (b) HLA.
and 𝑝𝑣 respectively. In order to give a comparable standard of the two kinds of the lens array, the lens pitch of RLA and HLA in this paper are both defined as the diameter of the inscribed circle of the EI, regardless of its shape. The geometric characteristics of the previous mentioned RLA are that the elementary squares are oriented along the side of the square, while in the HLA, the elementary hexagons are oriented along of the axis passing through the farthest vertices of the hexagon, which can
also be referred as the ‘‘horizontal’’ HLA as well. There are other variations for the layout of macro lens based on different geometric shapes, at least one obvious layout exists for each shape, i.e. the squares oriented along the diagonal of the square and the hexagons oriented orthogonally to their longest axis. However, these layouts can be obtained by rotating the current setups in an appropriate angle, and it makes no difference to the analysis in the paper in terms of pixel effectiveness in this subsection and frequency enhancement in the latter 3
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
clearly distinguish that in the RLA and HLA with the same pitch and effective area’s radius, the effectiveness of the pixels of the latter is not smaller than the former. That means in most of the case, arranging the circular lens in a hexagonal way can get higher pixel effectiveness. In fact, by substituting the REI with HEI, the number of ineffective pixels can be significantly decreased, also the number of perspectives (𝑁v ) is increased without a notable margin of the pixel number of the perspectives images (𝑅v ), which is meaningful for high-quality 3D image reconstruction.
subsection because of the symmetry of the effective area. While other setups, like non-square RLA with some aspect ratio, is out of the scope of this paper. Actually, the effectiveness of the pixels is closely relevant to the relationship between the radius of the effective area and the lens pitch, which is shown in Fig. 3. For both the RLA and the HLA case, there are three situations for the relative relationship between the effective area and the lens pitch, i.e. the effective area is totally within the EI, the EI is totally within the effective area, and neither the effective area nor the EI is totally within the other. Which is illustrated as the left, right and middle case of Fig. 3(a) and (b). The effectiveness of the pixel of the RLA can be expressed as:
𝑒𝑓 𝑓RLA
⎧ 𝜋𝑟2 ⎪ 𝑝2 , ⎪ = ⎨ 𝜋𝑟2 −4𝑠1 , ⎪ 𝑝2 ⎪1, ⎩
2.3. Reconstruction enhancement in terms of angular frequency
𝑟 ≤ 𝑅RI 𝑅RI < 𝑟 < 𝑅RC
Fig. 6 shows the image reconstruction characteristics in the spatial domain, the location of the lens is expressed by the optical center of it. The horizontal and vertical interval of reconstruction, referred as 𝛥𝑥 and 𝛥𝑦 in Fig. 6(a) and (b), is defined as the distances between two adjacent columns and two adjacent rows respectively. Consider the original light field distribution of the object in the lens array plane is 𝑂(𝑥, 𝑦), here, the representation of the light field covers only the spatial dimensions, the angular dimensions are considered implicitly for simplicity. Since the resolution of each orthographic view of the reconstructed light field is the same, the spatial reconstruction of the light field is angularly invariant. The reconstructed light field can be given by the following equations: ∑ 𝑂̄ RLA (𝑥, 𝑦) = 𝑂(𝑥, 𝑦)𝛿(𝑥 − 𝑚𝛥𝑥)𝛿(𝑦 − 𝑛𝛥𝑦) (6)
(4)
𝑟 ≥ 𝑅RC
where 𝑟 refers to the radius of the effective area, 𝑝 refers to the lens pitch (𝑝 = 𝑝𝑣 = 𝑝ℎ ), and 𝑠1 means the area out of the corresponding REI region. 𝑅RI is the radius of the inscribed circle of the REI, and 𝑅RC is the radius of the circumcircle of the REI. Similarly, the effectiveness of the pixel of the HLA can be expressed as:
𝑒𝑓 𝑓HLA
⎧ 2𝜋𝑟2 ⎪ √3𝑝2 , ⎪ = ⎨ 2(𝜋𝑟√2 −6𝑠2 ) , ⎪ 3𝑝2 ⎪1, ⎩
𝑟 ≤ 𝑅HI 𝑅HI < 𝑟 < 𝑅HC
(5)
𝑟 ≥ 𝑅HC
𝑚,𝑛
𝑂̄ HLA (𝑥, 𝑦) =
where 𝑠2 denotes the covered area √ out of the corresponding HEI, 𝑝 is the side length of HEI (𝑝 = 𝑝𝑣 ∕ 3 = 2𝑝ℎ ∕3). 𝑅HI is the radius of the inscribed circle of the HEI, and 𝑅HC is the radius of the circumcircle of the HEI. Both 𝑠1 and 𝑠2 can be easily obtained by simple calculation of an integral operation. To give a quantitative idea of the above analysis and further verify it, a numerical simulation is made. Taking a commercial 4K (3840 × 2160) 12.5-inch 2D DP into consideration, the basic parameters of the DP are as following, the pixel pitch is 72 μm, the effective display width is 277 mm horizontally and 156 mm vertically. The resolution of the perspective images is defined as the total number of pixels for convenience because the pixels’ distribution in the HLA perspective images is not in the regular grid horizontally and vertically. First, when the effective area is large enough or the limited FOV of the lens is not considered, the number of the perspectives and the resolution of the perspective images are only related to the pitch of the lens. Fig. 4 shows the variation trend of the number of perspectives and the resolution of each perspective when the lens is arranged in a regular grid and in a hexagonal way respectively. From the general trend of Fig. 4 we can see that the resolution of the perspective is inversely related to the pitch of the lens generally, and the number of the perspectives is positively related to it, regardless of the arrangement manner of the LA. That is the tradeoff phenomenon between the angular and spatial resolution revealed in the last subsection. When the limited FOV of the lens is taken into consideration, the increase of resolution for each perspective can be achieved without distinguishing shrinking of the effective number of the perspectives by turning the original ineffective pixels into effective, this can be shown by the effectiveness of pixels under HLA and RLA with same lens pitch and same FOV. Since the effectiveness of the pixels is a combined action of the FOV of the lens 𝛽 and the gap g between the LA and the DP, the radius of the effective area (see Eq. (3)) is used to measure the combined effect on it. Fig. 5 depicted the effectiveness of the pixels in DP with various lens pitch for both RLA and HLA. Generally speaking, when the radius of the effective area becomes large enough, the effectiveness of the pixel reaches its maximum, but one can
∑
𝑂 (𝑥, 𝑦) 𝛿 (𝑥 − 𝑚𝛥𝑥) 𝛿 [𝑦 − (𝑚 + 2𝑛) 𝛥𝑦]
(7)
𝑚,𝑛
where 𝑚 and 𝑛 stand for the sequential number of the lens in the horizontal and vertical direction respectively. 𝛿(⋅) represents the Dirac delta function. The reconstructed light field of the image by RLA and HLA are shown in Eqs. (6) and (7). According to the lens pitch defined in the previous subsection, the reconstruction interval has the following relationship. For RLA, there is 𝛥𝑥 = 𝛥𝑦 = 𝑝. As for HLA, there are √ 𝛥𝑥 = 3𝑝∕2 and 𝛥𝑦 = 𝑝∕2. The comparison of the reconstructed light field in the frequency domain can be obtained by Fourier transform of the above equations, which is: ) ( ) ( ∑ [ ] 𝑛 𝑚 𝛿 𝑓𝑦 − (8) ℱ 𝑂̄ 𝑅𝐿𝐴 (𝑥, 𝑦) = ℱ [𝑂(𝑥, 𝑦)] ⊗ 𝛿 𝑓𝑥 − 𝑝 𝑝 𝑚,𝑛 ( ) ( ) ∑ [ ] 𝑚 𝑚 + 2𝑛 ℱ 𝑂̄ 𝑅𝐿𝐴 (𝑥, 𝑦) = ℱ [𝑂(𝑥, 𝑦)] ⊗ 𝛿 𝑓𝑥 − √ (9) 𝛿 𝑓𝑦 − 𝑝 𝑚,𝑛 3𝑝 Here, ⊗ means the operation of convolution. ℱ stands for the Fourier transform operation. The above two equations indicate that the spectrum of the object field is duplicated with different manner related to the arrangement of the lens array. A more specific standard to measure the performance in the frequency domain can be indicated by the maximum radial spatial frequency that can be reconstructed without aliasing, which can be represented by: √ 1 𝑓𝜌,RLA = 𝑓𝑥2 + 𝑓𝑦2 ≤ (10) 2𝑝 √ 1 𝑓𝜌,HLA = 𝑓𝑥2 + 𝑓𝑦2 ≤ √ (11) 3𝑝 The above two equations reveal that the maximum spatial frequency √ is 𝑓𝜌,RLA = 1∕(2𝑝) for RLA and 𝑓𝜌,RLA = 1∕( 3𝑝) for HLA accordingly. So theoretically speaking, compared with the RLA case, the resolution √ enhancement of a ratio of 2∕ 3 is a reasonable expectation for the HLA based light field display. We verified this resolution enhancement later in our optical experiment visually. 4
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 3. The relationship between the effectively covered pixels and the EI, (a) for the RLA case, (b) for the HLA case.
Fig. 4. The general variation trend of the perspective number and its corresponding resolution for different lens pitch without considering the FOV of the lens and the effectiveness of the pixels.
Fig. 5. The effectiveness of pixels in DP with consideration of the lens’ FOV, (a) for RLA, (b) for HLA.
Fig. 6. The image reconstruction in the spatial domain for (a) RLA, (b) HLA.
5
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 7. The viewing zone shaped by the (a) REI, (b) HEI, and the detailed characteristics of view volume using (c) REI and (d) HEI.
Fig. 8. (a) General system structure of the fabricated prototype, (b) the fabricated RLA, (c) the fabricated HLA.
𝛥𝑡 vertically respectively. The horizontal and vertical angular interval of the adjacent view can be noted as 𝛥𝜃 and 𝛥𝜙 respectively and the maximum horizontal and vertical angular range can be signed as 𝜃max and 𝜙max accordingly for both the REI (see Fig. 7(c)) and the HEI (see Fig. 7(d)) case. It is obvious that when the FOV of the lens is large enough, the expressible angular range for the HEI case is larger than the REI case. When the diameter of the lens is 10 mm and the distance between the LA and the DP is set to be 10 mm, the maximin expressible angular range can be approximately 53◦ for the RLA case and 60◦ for the HLA case, supposing that the optical FOV of the lens is large enough and the DP is seamlessly covered with REI and HEI respectively, so there is a distinct viewing angle improvement by arranging the lens in a hexagonal way.
2.4. Viewing zone shaping mechanism of the EI The above analysis focuses on the number of the perspective and the resolution of each perspective, but one cannot omit the changing of the EI’s geometry and its’ effect on viewing zone shaping. Although the light ray emanated by each single pixel is modulated to be conical because of the shape of the lens. The shape of the view volume of each single EI is determined by the shape of the EI, i.e. a rectangular pyramid for REI (see Fig. 7(a)) and a hexagonal pyramid for HEI (see Fig. 7(b)), with their summits in touch at the center of the lens. The directions of perspective are represented as the principal ray of an individual lens. Take one EI for instance, suppose that the EI image is in the s-t plane, the pixel pitch can be expressed as 𝛥𝑠 horizontally and 6
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 9. The original 3D scene for RLA and HLA system, (a) the letter scene, (b) the play card scene.
Fig. 10. The front view of the ‘‘LAB’’ scene from (a) the HLA display system, (b) the RLA display system.
Fig. 11. The front view of the play cards scene from (a) the HLA display system, (b) the RLA display system.
The viewing zone of the whole system is defined as the common areas of the imbricated view volume, which will also be a rectangular pyramid and a hexagonal pyramid for REI and HEI respectively. Despite the fact that the HLA can bring in resolution enhancement and maximum expressible angular range improvement, there is no final conclusion on whether a hexagonal viewing zone is superior to a rectangular viewing zone, but we cannot omit this phenomenon as a side effect of the arranging the LA in a hexagonal way.
is higher than the RLA, when the covered display area is equivalent, the number of lens in the HLA is also larger than the RLA. Two scenes are used in the experiments. First a simple scene without much high-frequency information, which is made up of three letters, a red ‘L’, a green ‘A’ and a blue ‘B’, as shown in Fig. 9(a), and they are set as 𝑑𝐿 = 140 mm, 𝑑𝐴 = 150 mm, and 𝑑𝐵 = 160 mm away from the lens array respectively. As a contrast, three play cards, i.e. a heart, spade, diamond ‘k’, which we think is full of high frequency information compared with the former scene, is put in the same location, as depicted in Fig. 9(b). The EIA is generated using the ray tracing algorithm according to the lens array parameters used in the display process. The display results of the scenes are demonstrated in Fig. 10 and Fig. 11 for the ‘‘LAB’’ scene and play card scene respectively. From Fig. 10, we can clearly see that although the scene is relatively simple. The boundary blur for the RLA display system is quite distinct compared with the HLA display system. Similar result can also see in the display result of the play cards scene, see Fig. 11, the enhancement of resolution for the display is quite significant. The above outcomes are display results from the quasi-3D scene, which means the scene is made up of 2D plane images located in different depth, the main merit of such scenes is that one can clearly distinguish the resolution enhancement between the two systems. To further verify the effectiveness of the proposed method, a 3D scene made up of three dices with different pose are used, the scene is relatively simple without much high frequency information. The display results are shown in Fig. 12, from the front view of the display, we can
3. Experiments and discussions To verify the resolution improvement of the HLA, two prototypes were fabricated, the overall structure of the display system is shown in Fig. 8(a), A compound lens array, a high-resolution DP, and a holographic diffusor with a glass substrate. The size of the DP is 12.5 inch in diagonal, with a resolution of 3840 × 2160, the pixel pitch of the DP is approximately 72 μm. The gap between the DP and the optical center of the lens array is approximately 10.7 mm, the holographic diffusor with glass substrate is put parallelly 150 mm away from the lens array. The compound lens array is arranged in the following two ways, first, the traditional rectangular arranged one, which includes 22 lenses in column and 12 lenses in row, as shown in Fig. 8(b). The display system with this lens array is referred as the RLA system. Second, the updated hexagonal arrange one, with a total of 312 (12 × 26) lenses, as shown in Fig. 8(c). The display system with this lens array is referred as the HLA system. Since the filling rate of lens in the HLA 7
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613
Fig. 12. The front view of the dices scene from (a) the HLA display system, (b) the RLA display system.
Fig. 13. The front view of the tank scene from (a) the HLA display system, (b) the RLA display system.
Funding
distinguish obvious resolution enhancement in the HLA display system. Another 3D scene used in the experiment is a tank full of high frequency information, as depicted in Fig. 13, the HLA system is superior in the homogeneity of the displayed image and in detail presentation. The tail part of the tank is blurred in the RLA display system while it is clearly visible in the HLA display system. The previous analysis in √ the frequency domain reveals a resolution enhancement of a ratio of 2∕ 3, however, we can only verify this quantitative improvement in terms of visual quality, which we believe that the resolution enhancement of the HLA system is fully verified by the experimental results presented.
The National Key Research and Development Program of China (2017YFB1104500), National Natural Science Foundation of China (61775240), Foundation for the Author of National Excellent Doctoral Dissertation of the People’s Republic of China (FANEDD) (201432). References [1] J.-Y. Son, H. Lee, B.-R. Lee, K.-H. Lee, Holographic and light field imaging as future 3-D displays, Proc. IEEE 105 (5) (2017) 789–804. [2] H. Zhang, Y. Zhao, L. Cao, G. Jin, Three-dimensional display technologies in wave and ray optics: a review (Invited Paper), Chin. Opt. Lett. 12 (6) (2014) 060002–060007. [3] C.G. Luo, X. Xiao, M. Martinez-Corral, C.W. Chen, B. Javidi, Q.H. Wang, Analysis of the depth of field of integral imaging displays based on wave optics, Opt. Express 21 (25) (2013) 31263–31273. [4] R.R. Tamboli, B. Appina, S. Channappayya, S. Jana, Super-multiview content with high angular resolution: 3D quality assessment on horizontal-parallax lightfield display, Signal Process Image 47 (2016) 42–55. [5] A.J. Woods, H.-S. Kim, N.S. Holliman, H.-E. Kim, K.-M. Jeong, G.E. Favalora, S.-I. Hong, J.-H. Park, Analysis on ray reconstruction characteristics of multiview and integral imaging display, in: Stereoscopic Displays and Applications, Vol. XXIII, 2012. [6] A.J. Woods, A. Said, N.S. Holliman, E.-V. Talvala, J.O. Merritt, Spatial-angular analysis of displays for reproduction of light fields, in: Proc. SPIE 7237, Stereoscopic Displays and Applications, Vol. XX, 2009, p. 723707. [7] Y. Zhu, X. Sang, X. Yu, P. Wang, S. Xing, D. Chen, B. Yan, K. Wang, C. Yu, Wide field of view tabletop light field display based on piece-wise tracking and off-axis pickup, Opt. Commun. 402 (2017) 41–46. [8] S. Yang, X. Sang, X. Yu, X. Gao, L. Liu, B. Liu, L. Yang, 162-inch 3D light field display based on aspheric lens array and holographic functional screen, Opt. Express 26 (25) (2018) 33013–33021. [9] X. Sang, F.C. Fan, C.C. Jiang, S. Choi, W. Dou, C. Yu, D. Xu, Demonstration of a large-size real-time full-color three-dimensional display, Opt. Lett. 34 (24) (2009) 3803–3805. [10] Z. Yan, X. Yan, X. Jiang, H. Gao, J. Wen, Integral imaging based light field display with enhanced viewing resolution using holographic diffuser, Opt. Commun. 402 (2017) 437–441. [11] J. Arai, H. Kawai, F. Okano, Microlens arrays for integral imaging system, Appl. Opt. 45 (36) (2006). [12] R.F. Stevens, N. Davies, Lens arrays and photography, J. Photogr. Sci. 39 (5) (1991) 199–208. [13] W. Zhang, X. Sang, X. Gao, X. Yu, B. Yan, C. Yu, Wavefront aberration correction for integral imaging with the pre-filtering function array, Opt. Express 26 (21) (2018) 27064–27075. [14] X. Gao, X. Sang, X. Yu, W. Zhang, B. Yan, C. Yu, Aberration improvement of the floating 3D display system based on Tessar array and directional diffuser screen, Opt. Rev. 25 (4) (2018) 500–508.
4. Conclusion We present a comparative study on the resolution enhancement of the lens array arrangement. The most obvious and intuitive merit of such setup is the improvement of the filling rate. Besides, the tradeoff relationship between the angular reconstruction rate and spatial reconstruction rate inherent in the light field display is further revealed by taking the lens as a paraxial imaging optic. Numerical simulation reveals that the proposed method can effectively improve the validity of the covered pixels when the limited FOV of the lens is taking into consideration. The resolution enhancement is also measured in the frequency √ domain, which reveals a resolution enhancement of a ratio of 2∕ 3. Optical experiments show distinct resolution refinement of the proposed method, which means it is superior in detail presentation and display homogeneity compared with the tradition RLA display system. This method is especially useful when the scene reconstructed is complicated and full of the high frequency information, which is common in our daily scene. We think our report will help to the extension and promotion of the practical use of the InIm based LFD. CRediT authorship contribution statement Jun Wen: Methodology, Investigation, Validation, Writing - original draft. Xingpeng Yan: Conceptualization, Validation, Formal analysis, Visualization. Xiaoyu Jiang: Validation, Formal analysis, Visualization. Zhan Yan: Resources, Writing - review & editing, Supervision. Ziqiang Wang: Resources, Writing - review & editing, Supervision. Song Chen: Writing - review & editing. Min Lin: Writing - review & editing. Acknowledgments Special thanks should give to Ping Zhong for the helpful discussion on the construction of the model and the rendering of the images. 8
J. Wen, X. Yan, X. Jiang et al.
Optics Communications 466 (2020) 125613 [18] A. Karimzadeh, Analysis of the depth of field in hexagonal array integral imaging systems based on modulation transfer function and Strehl ratio, Appl. Opt. 55 (11) (2016) 3045–3050. [19] N. Chen, J. Yeom, J.H. Jung, J.H. Park, B. Lee, Resolution comparison between integral-imaging-based hologram synthesis methods using rectangular and hexagonal lens arrays, Opt. Express 19 (27) (2011) 26917–26927. [20] Y. Sheng, J.-H. Park, C. Yu, D. Han, N. Kim, L. Chen, Capture of the threedimensional information based on integral imaging and its sampling analysis, in: Holography, Diffractive Optics, and Applications, Vol. IV, 2010.
[15] S. Manolache, A. Aggoun, M. McCormick, N. Davies, S.Y. Kung, Analytical model of a three-dimensional integral image recording system that uses circular- and hexagonal-based spherical surface microlenses, J. Opt. Soc. Amer. A 18 (8) (2001) 1814–1821. [16] A.J. Woods, K. Yanaka, N.S. Holliman, J.O. Merritt, Integral photography using hexagonal fly’s eye lens and fractional view, in: Stereoscopic Displays and Applications, Vol. XIX, 2008. [17] S.S. Athineos, Photorealistic integral photography using a ray-traced model of capturing optics, J. Electron. Imaging 15 (4) (2006) 0430071–0430078.
9