- Email: [email protected]
Super resolution ghost imaging based on Fourier spectrum acquisition
ARTICLE IN PRESS
JID: OLEN
[m5GeSdc;November 21, 2020;22:54]
Optics and Lasers in Engineering xxx (xxxx) xxx
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Super resolution ghost imaging based on Fourier spectrum acquisition Le Wang a,∗, Shengmei Zhao a,b a
Institute of Signal Processing and Transmission, Nanjing University of Posts and Telecommunications(NUPT), Nanjing 210003, China Key Lab of Broadband Wireless Communication and Sensor Network Technology, Nanjing University of Posts and Telecommunications, Ministry of Education, Nanjing 210023, China
b
a r t i c l e
i n f o
a b s t r a c t
Keywords: Ghost imaging Single pixel imaging Super resolution Fourier spectrum
In this paper, we propose a novel super-resolution ghost imaging based on Fourier spectrum acquisition. In the scheme, we use a set of sinusoidal speckle patterns with specific phase shift to illuminate the object, and then the corresponding total intensity of reflective light is measured by a bucket detector. By performing the inverse Fourier transformation on the bucket detection results, a set of low-resolution images with subpixel shift could be reconstructed. Subsequently, a super-resolution image could be obtained by combining all the reconstructed subpixel shifted images. The experimental results demonstrate that our proposed scheme can capture a superresolution image with both direct and indirect line of sight to the object. The proposed scheme has the advantage of achieving super-resolution ghost imaging beyond the limitation of the pixel resolution of the illuminated speckle patterns and without the use of any high-precision motorized translation stages for shifting the speckle patterns.
1. Introduction
tial resolution of the speckle patterns [17]. In addition, the computing time for the reconstruction of the image significantly grows with the increase of the spatial resolution of the speckle patterns. Therefore, both the SNR and the computing time of the reconstructed images limit the improvement of imaging resolution. To overcome the limits of the resolution of the ghost imaging, the subpixel shift method is introduced into GI [17–21], which can obtain a higher resolution image of the object by illuminating a series of subpixel shifted speckle patterns with a low resolution. Hence, this method has the potential to realize the super resolution imaging. However, the realization of subpixel shift of speckle patterns relies on the high-precision mechanical shift of speckle patterns by the aid of a high-precision motorized translation stage [18]. Therefore, the cost for constructing such ghost imaging system will dramatically increase. In addition, one can achieve subpixel shift of speckle patterns by loading patterns with subpixel shifts on DMD instead of mechanical scanning [19–21], however, this way still cannot really exceed the limitation of the resolution of DMD. In this paper, we propose a novel super-resolution ghost imaging based on Fourier spectrum acquisition. In the scheme, we use a set of sinusoidal speckle patterns with specific phase shift to illuminate the object, and then the corresponding total intensity of reflective light is measured by a bucket detector. By performing the Fourier transformation on the bucket detection results, a set of low-resolution images with subpixel shift could be reconstructed. Subsequently, a super-
Ghost imaging (GI), also named single-pixel imaging (SPI), is an intriguing optical imaging technique [1,2]. Ghost imaging can acquire an image of object by correlating the speckle patterns and corresponding bucket detection results, where the bucket detection results are detected by a bucket detector without any spatial resolution and the speckle patterns are measured by a spatially resolving detector or computing offline. GI offers great promise for its better robustness against harsh environment [3,4] and multiple types of information processing [5,6]. Therefore, GI has been drawn great attention and considerable number of approaches based on GI have been proposed [7–14]. For example, the iterative phase retrieval algorithm is used in GI to realize the highquality reconstruction [22]; The total sampling time of GI is reduced by using multiple-input-single-output technique [15]; Edge of the unknown object can be obtained by GI without the imaging at advance [16]. However, the pixel resolution of the reconstructed image by ghost imaging is usually low. On the one hand, the pixel resolution of the reconstructed image by ghost imaging is determined by the spatial resolution of the speckle patterns [17,18]. The speckle patterns are usually generated by the spatial light modulator (SLM) and digital micro-mirror device (DMD), but the resolution of these devices is limited, which results in the generation of limited spatial resolution of the speckle patterns. On the other hand, the signal-to-noise ratio (SNR) of the reconstructed image by ghost imaging decreases with the increase of the spa-
∗
Corresponding author. E-mail addresses: [email protected] (L. Wang), [email protected] (S. Zhao).
https://doi.org/10.1016/j.optlaseng.2020.106473 Received 19 June 2020; Received in revised form 10 October 2020; Accepted 10 November 2020 Available online xxx 0143-8166/© 2020 Elsevier Ltd. All rights reserved.
Please cite this article as: L. Wang and S. Zhao, Super resolution ghost imaging based on Fourier spectrum acquisition, Optics and Lasers in Engineering, https://doi.org/10.1016/j.optlaseng.2020.106473
ARTICLE IN PRESS
JID: OLEN
[m5GeSdc;November 21, 2020;22:54]
L. Wang and S. Zhao
Optics and Lasers in Engineering xxx (xxxx) xxx
Fig. 1. A schematic diagram of the proposed super resolution ghost imaging scheme based on Fourier spectrum acquisition.
resolution image could be obtained by combining all the reconstructed subpixel shifted images. Furthermore, the proposed scheme can capture a super-resolution image without a direct line of sight to the object. The advantage of the proposed scheme is that it could achieve superresolution ghost imaging beyond the limitation of the pixel resolution of the speckle patterns and DMD and without the use of any high-precision motorized translation stages for shifting the speckle patterns. The organization of the paper is as follows. In Section 2, our proposed scheme is presented. In Section 3, the performance of our proposed scheme is discussed by experiments. Finally, Section 4 concludes the paper.
where 𝜂 is the bucket detector responsivity, 𝐴 is the area illuminated by sinusoidal speckle patterns, 𝑇 (𝑥, 𝑦) is the distribution function of the object, and 𝑛 represents the environmental illumination. By substituting Eq. 1 and Eq. 2 to Eq. 3, Eq. 3 can be expressed as 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) = 𝑎𝜂
∫𝐴
𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑏𝜂
∫𝐴
cos(2𝜋𝑓𝑥 (𝑥 + 𝑥0 )
+2𝜋𝑓𝑦 (𝑦 + 𝑦0 ) + 𝜑0 )𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑛 = 𝑎𝜂
∫𝐴
𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑏𝜂 cos(𝜑0 )
+2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦
2. Scheme Description
−𝑏𝜂 sin(𝜑0 ) The schematic diagram of the proposed scheme is shown in Fig. 1. A digital light projector (DLP) is used to produce a series of sinusoidal speckle patterns with specific phase shift 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) with the pixel resolution 𝑁𝑥 × 𝑁𝑦 , 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) = 𝑎 + 𝑏 cos(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦 + 𝜑𝑥 + 𝜑𝑦 + 𝜑0 ),
(1)
where (𝑓𝑥 , 𝑓𝑦 ) is the spatial frequency and (𝑥, 𝑦) are Cartesian coordinates in the space domain where −𝑋 ≤ 𝑥 ≤ 𝑋 and −𝑌 ≤ 𝑦 ≤ 𝑌 . Hence, each pixel has a spatial size 𝛿𝑥 = 2𝑋∕𝑁𝑥 and 𝛿𝑦 = 2𝑌 ∕𝑁𝑦 . 𝑎 is the direct current (DC) component, which is equal to the average intensity of the sinusoidal speckle patterns, and 𝑏 represents the contrast. We can adjust the parameters 𝑎 and 𝑏 to make sure that the DLP can display all sinusoidal speckle patterns brightly. 𝜑0 is the initial phase. 𝜑𝑥 and 𝜑𝑦 are the phase shift along the x-axis and y-axis, respectively, 𝜑𝑥 = 2𝜋𝑓𝑥 𝑥0 ,
∫𝐴
∫𝐴
cos(2𝜋𝑓𝑥 (𝑥 + 𝑥0 )
sin(2𝜋𝑓𝑥 (𝑥 + 𝑥0 ) + 2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑛.
(4)
By using the four-step phase-shift approach [14,16], one can acquire a complex Fourier coefficient of the object 𝐶(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 ) by using four bucket detection results corresponding to four sinusoidal speckle patterns where these four sinusoidal speckle patterns have the same frequency (𝑓𝑥 and 𝑓𝑦 ) and the same phase shift along the x-axis and y-axis (𝜑𝑥 and 𝜑𝑦 ) but have a different initial phase 𝜑 (𝜑 are 0, 𝜋∕2, 𝜋 and 3𝜋∕2, respectively.). 𝐶(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 ) =
1 [(𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 0) − 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋)) 2𝑏𝜂 +𝑗(𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋∕2) − 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 3𝜋∕2))], (5)
where [𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 0) − 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋)]
𝜑𝑦 = 2𝜋𝑓𝑦 𝑦0 ,
(2)
+𝑗[𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋∕2) − 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 3𝜋∕2)]
where 𝑥0 and 𝑦0 are the spatial shift along the x-axis and y-axis, respectively. Then the sinusoidal speckle pattern 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) interacts with an object and then is measured by a bucket detector to generate a bucket detection result 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ),
= 2𝑏𝜂
𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) = 𝜂
= 2𝑏𝜂 {𝑇 (𝑥 − 𝑥0 , 𝑦 − 𝑦0 )},
∫𝐴
𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 )𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑛,
(3)
∫𝐴
−𝑗2𝑏𝜂 = 2𝑏𝜂
∫𝐴
cos(2𝜋𝑓𝑥 (𝑥 + 𝑥0 ) + 2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 ∫𝐴
sin(2𝜋𝑓𝑥 (𝑥 + 𝑥0 ) + 2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦
exp[−(2𝜋𝑓𝑥 (𝑥 + 𝑥0 ) + 2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))]𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 (6)
JID: OLEN
ARTICLE IN PRESS
[m5GeSdc;November 21, 2020;22:54]
L. Wang and S. Zhao
Optics and Lasers in Engineering xxx (xxxx) xxx
Fig. 2. A schematic diagram of the experimental system of the proposed super resolution ghost imaging scheme. DLP: digital light projector, BF: bandpass filter, BD: bucket detector, S: Aluminum laser protective screen, and ADC: analogue-to-digital converter.
Fig. 3. The experimental results of the USAF resolution chart by using the proposed scheme, where (a) is the result by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝑦0 = 0; (b) is the result by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕2, 𝑦0 = 0, 𝛿𝑦∕2; (c) is the result by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕4, 𝛿𝑥∕2, 3𝛿𝑥∕4, 𝑦0 = 0, 𝛿𝑦∕4, 𝛿𝑦∕2, 3𝛿𝑦∕4; (d) is the result by using sinusoidal speckle patterns with the pixel resolution 128 × 128 and 𝑥0 = 0, 𝑦0 = 0; (e) is the result by using sinusoidal speckle patterns with the pixel resolution 128 × 128 and 𝑥0 = 0, 𝛿𝑥∕2, 𝑦0 = 0, 𝛿𝑦∕2; (f) is the result by using sinusoidal speckle patterns with the pixel resolution 128 × 128 and 𝑥0 = 0, 𝛿𝑥∕4, 𝛿𝑥∕2, 3𝛿𝑥∕4, 𝑦0 = 0, 𝛿𝑦∕4, 𝛿𝑦∕2, 3𝛿𝑦∕4. The insets show the enlarged details of images.
and {⋅} represents the Fourier transformation. By using inverse Fourier transformation on Eq. 6, the reconstructed image of the object with the spatial shift along the x-axis, 𝑥0 , and y-axis, 𝑦0 , is able to obtain,
and 𝑦0 , 𝑇̂ (𝑥′ , 𝑦′ ) =
1 −1 𝑇̂ (𝑥 − 𝑥0 , 𝑦 − 𝑦0 ) = {[𝐵𝑖 (𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 0) − 𝐵𝑖 (𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋)] 2𝑏𝜂 +𝑗[𝐵𝑖 (𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋∕2) − 𝐵𝑖 (𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 3𝜋∕2)]}, (7)
3. Results discussion
where −1 {⋅} represents the inverse Fourier transformation. If 𝑥0 and 𝑦0 of sinusoidal speckle patterns 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) are set as 𝑥0 < 𝛿𝑥 and 𝑦0 < 𝛿𝑦, we can reconstruct the subpixel-shifted image of the object. Hence, 𝑥0 and 𝑦0 can be set as 𝑥0 = 𝛿𝑥𝐾𝑥 and 𝑦0 = 𝛿𝑦𝐾𝑦 where both 𝐾𝑥 and 𝐾𝑦 are fractional numbers less than 1, and then we can reconstruct the 𝐾𝑥 -pixel shifted image along the x-axis and the 𝐾𝑦 -pixel shifted image along the y-axis. Lastly, the high resolution image 𝑇̂ (𝑥′ , 𝑦′ ) can be reconstructed by combining all reconstructed subpixel-shifted images with different 𝑥0
The experimental system of the proposed super resolution ghost imaging scheme is shown in Fig. 2. A series of sinusoidal speckle patterns with specific phase shift 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) are generated by DLP (TI Digital lightCrafter 4500) controlled by a computer and then are projected onto an object, the USAF resolution test chart. The reflective light is detected by a bucket detector (Thorlabs PMM02) after a bandpass filter (633𝑛𝑚) and then a computer records a series of bucket detection results 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) via an analogue-to-digital converter
∑ 𝑥0 ,𝑦0
𝑇̂ (𝑥 − 𝑥0 , 𝑦 − 𝑦0 ).
(8)
JID: OLEN
ARTICLE IN PRESS
L. Wang and S. Zhao
[m5GeSdc;November 21, 2020;22:54] Optics and Lasers in Engineering xxx (xxxx) xxx
Fig. 4. The experimental results of the USAF resolution chart by using the proposed scheme with different density distributions of Fourier spectrum acquired from the bucket detector, where (a1)-(a6) are the results by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕2, 𝑦0 = 0, 𝛿𝑦∕2; (b1)-(b6) are the results by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕4, 𝛿𝑥∕2, 3𝛿𝑥∕4, 𝑦0 = 0, 𝛿𝑦∕4, 𝛿𝑦∕2, 3𝛿𝑦∕4; (c1)-(c6) are the density distributions of Fourier spectrum acquired from the bucket detector. The insets show the enlarged details of images.
(NI USB-6341). Subsequently, the subpixel-shifted images with different 𝑥0 and 𝑦0 can be reconstructed. Finally, the high resolution image of the object 𝑇̂ (𝑥′ , 𝑦′ ) can be obtained by combining all reconstructed subpixel-shifted images with different 𝑥0 and 𝑦0 . In addition, to verify the performance of imaging without a direct line of sight to the object for the proposed scheme, an Aluminum laser protective screen having a homogeneously diffusive surface is utilized as a scatterer and the bucket detector placed with back to object is used to detect the signals diffused from the Aluminum laser protective screen. Hence, we substitute the apparatus highlighted by a dashed blue square in Fig. 2 for that highlighted by a dashed green square in Fig. 2 in this experiment.
In order to compare the quality of the reconstructed image quantitatively, the signal-to-noise ratio (SNR) [21], the peak signal-to-noise (PSNR) and the structural similarity (SSIM) index [11,23] of the reconstructed high resolution image are used as an objective evaluation, which are defined as
𝑆𝑁𝑅 =
⟨𝐼𝑓 ⟩ − ⟨𝐼𝑏 ⟩ 1 (𝜎 2 𝑓
+ 𝜎𝑏 )
𝑃 𝑆𝑁𝑅 = 10 log10
,
max 𝑉 𝑎𝑙2 , 𝑀𝑆𝐸
(9)
(10)
ARTICLE IN PRESS
JID: OLEN L. Wang and S. Zhao
Optics and Lasers in Engineering xxx (xxxx) xxx
and 𝑆 𝑆 𝐼𝑀 =
(2𝑢𝑥 𝑢𝑦 + 𝐶1 )(2𝜎𝑥𝑦 + 𝐶2 ) (𝑢2𝑥 + 𝑢2𝑦 + 𝐶1 )(𝜎𝑥2 + 𝜎𝑦2 + 𝐶2 )
[m5GeSdc;November 21, 2020;22:54]
,
(11)
where ⟨𝐼𝑓 ⟩ and ⟨𝐼𝑏 ⟩ denote the average intensity of the feature (here calculated from the data highlighted by a solid red square in Fig. 3) and background (here calculated from the data highlighted by the dashed red square in Fig. 3), respectively. 𝜎𝑓 and 𝜎𝑏 are the standard deviations of the intensities in the feature and the background respectively. max 𝑉 𝑎𝑙 is the maximum possible pixel value of the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ). 𝑀𝑆𝐸 is the mean square error between the original image 𝑇 (𝑥′ , 𝑦′ ) and the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ). 𝑢𝑥 and 𝑢𝑦 are the average value of the original image 𝑇 (𝑥′ , 𝑦′ ) and the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ), respectively. 𝜎𝑥 and 𝜎𝑦 are the variance of the original image 𝑇 (𝑥′ , 𝑦′ ) and the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ), respectively. 𝜎𝑥𝑦 is the covariance of the original image 𝑇 (𝑥′ , 𝑦′ ) and the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ). 𝐶1 and 𝐶2 are parameters to avoid instability. To verify the feasibility of the proposed scheme, we perform the experiments and the results are shown in Fig. 3. Here we use 64 × 64-pixels and 128 × 128-pixels sinusoidal speckle patterns to illuminate the object. We show the reconstructed images in Fig. 3(b) and Fig. 3(e) obtained by the proposed scheme to combine four reconstructed subpixel-shifted images. Fig. 3(c) and Fig. 3(f) are the reconstructed images by the proposed scheme to combine 16 reconstructed subpixel-shifted images. Fig. 3(a) and Fig. 3(d) are the reconstructed images without pixel shifting. Comparing the results of Fig. 3(b) and Fig. 3(e) with those of Fig. 3(c) and Fig. 3(f), it is found that the reconstructed high-resolution images obtained by the proposed scheme to combine 16 reconstructed subpixelshifted images are clearer and corresponding SNRs are higher than those of combining four reconstructed subpixel-shifted images. From the enlarged details of images shown in the insets, we can further find that the resolution of the former is higher than the latter, because the smallest pixel shifting of reconstructed subpixel-shifted images to be combined of the former is half of that of the latter and thus the former has more sub-pixel information than the latter. In addition, in order to verify the ability of super-resolution imaging with the aid of the USAF resolution test chart, we compare the results
of the proposed scheme to the results by using traditional ghost imaging based on sinusoidal speckle patterns without subpixel shifting shown in Fig. 3(a) and Fig. 3(d). The USAF resolution test chart consists of multiple patterns of three bar targets with dimensions from big to small. Here, all patterns of three bar targets are divided into four groups (The groups are numbered 0, 1, 2, and 3.) and each group consists of six elements where each element has three horizontal bars and three vertical bars (The elements are numbered 1, 2, 3, 4, 5 and 6.). By reading off the group and element number of the first element which cannot be resolved, the limiting resolution may be determined by inspection and the resolution is defined as 𝑅𝑒𝑠 = 2𝐺+
𝑒−1 6
,
(12)
where 𝐺 and 𝑒 are the group number and the element number, respectively. From the enlarged details of images shown in Fig. 3(a)-(c) with 𝐺 = 1 and 𝑒 = 1, the limiting resolution by the proposed scheme (Fig. 3(b) and Fig. 3(c)) is larger than 2, i.e. 𝑅𝑒𝑠 ≥ 2, however, the limiting resolution by the traditional ghost imaging scheme (Fig. 3(a)) is smaller than 2, i.e. 𝑅𝑒𝑠 < 2. Similarly, since the group number and the element number of the enlarged details of images shown in Fig. 3(d)(f) are 𝐺 = 2 and 𝑒 = 2, the limiting resolution by the proposed scheme (Fig. 3(e) and Fig. 3(f)) is larger than 213∕6 but the traditional ghost imaging scheme (Fig. 3(d)) has the limiting resolution 𝑅𝑒𝑠 < 213∕6 . Therefore, the results show that the proposed scheme can acquire a higher resolution image comparing to traditional ghost imaging based on sinusoidal speckle patterns when the sinusoidal speckle patterns with the same pixel resolution are used. Additionally, SNRs of reconstructed images by using the proposed schemes are significantly higher than those of traditional ghost imaging. Moreover, we perform the experiments and obtain the results by using the proposed scheme with different density distributions of Fourier spectrum acquired from the bucket detector, which are shown in Fig. 4. It is seen that with the increase of the density distributions of Fourier spectrum, the image recovered by the proposed scheme becomes clearer as well as the corresponding SNR gradually increases and its resolution is gradually improved. In addition, we compare the results of Fig. 4(a) with those of Fig. 4(b), we can find that the reconstructed high-resolution
Fig. 5. The experimental results of the USAF resolution chart by using the proposed scheme with or without a direct line of sight to the object, where (a) and (c) are the results by using the proposed scheme with a direct line of sight to the object; (b) and (d) are the results by using the proposed scheme without a direct line of sight to the object; (a) and (b) are the results by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕2, 𝑦0 = 0, 𝛿𝑦∕2; (c) and (d) are the results by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕4, 𝛿𝑥∕2, 3𝛿𝑥∕4, 𝑦0 = 0, 𝛿𝑦∕4, 𝛿𝑦∕2, 3𝛿𝑦∕4; (a1), (b1), (c1) and (d1) are the results with the density distributions of Fourier spectrum coverage 100%; (a2), (b2), (c2) and (d2) are the results with the density distributions of Fourier spectrum coverage 80%. The insets show the enlarged details of images.
JID: OLEN
ARTICLE IN PRESS
L. Wang and S. Zhao
images obtained by combining 16 reconstructed subpixel-shifted images are clearer and corresponding SNRs and resolution are higher than those of combining four reconstructed subpixel-shifted images under the same condition of density distributions of Fourier spectrum. Lastly, we perform the experiments of the proposed scheme without a direct line of sight to the object, and the results are compared to those results with a direct line of sight to the object, which are shown in Fig. 5. The experimental results show that the proposed scheme can capture a super-resolution image without a direct line of sight to the object and the imaging performance is similar to that with a direct line of sight to the object. The reconstructed images obtained by combining 16 reconstructed subpixel-shifted images are clearer and corresponding SNRs and resolution are higher than those of combining four reconstructed subpixel-shifted images and the reconstructed images become clearer as well as the corresponding SNR is higher and its resolution is gradually improved with the increase of the density distributions of Fourier spectrum. However, the experimental results show that the proposed scheme without a direct line of sight to the object will have a slightly lower SNR than the proposed scheme with a direct line of sight to the object. 4. Conclusion We have proposed a novel super-resolution ghost imaging by using Fourier spectrum acquisition in the paper. We have performed the experiments to verify that the proposed scheme can achieve superresolution ghost imaging beyond the limitation of the pixel resolution of the speckle patterns. In addition, the proposed scheme has the ability of the super-resolution imaging without a direct line of sight to the object. Comparing to existing super-resolution ghost imaging schemes, the advantage of the proposed scheme is that it could capture a superresolution image without the use of any high-precision motorized translation stages for shifting the speckle patterns. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. CRediT authorship contribution statement Le Wang: Conceptualization, Methodology, Software, Formal analysis, Writing - original draft. Shengmei Zhao: Conceptualization, Supervision, Project administration, Writing - original draft. Acknowledgment The paper is supported by the National Natural Science Foundation of China (NSFC) (61871234), the Natural Science Foundation of Jiangsu Province (BK20180755), the Open research fund of Key Lab of
[m5GeSdc;November 21, 2020;22:54] Optics and Lasers in Engineering xxx (xxxx) xxx
Broadband Wireless Communication and Sensor Network Technology (JZNY201910), and the NUPTSF (NY218098, NY220004).
References [1] Strekalov DV, Sergienko AV, Klyshko DN, Shih YH. Observation of two-photon “ghost” interference and diffraction,. Phys Rev Lett 1995;74:3600–3. [2] Bennink RS, Bentley SJ, Boyd RW. “two-photon” coincidence imaging with a classical source,. Phys Rev Lett 2002;89:113601. [3] Zhao SM, Yang H, Li YQ, Cao F, Sheng YB, Cheng WW, Gong LY. The influence of atmospheric turbulence on holographic ghost imaging using orbital angular momentum entanglement: Simulation and experimental studies,. Opt Commun 2013;294:223–8. [4] Yin MQ, Wang L, Zhao SM. Experimental demonstration of influence of underwater turbulence on ghost imaging,. Chin Phys B 2019;28:094201. [5] Wang L, Zhao S, Cheng W, Gong L, Chen H. Optical image hiding based on computational ghost imaging,. Opt Commun 2016;366:314–20. [6] Jiao S, Zhou C, Shi Y, Zou W, Li X. Review on optical image hiding and watermarking techniques,. Opt Laser Technol 2019;109:370–80. [7] Gong W, Zhao C, Yu H, Chen M, Xu W, Han S. Three-dimensional ghost imaging lidar via sparsity constraint,. Sci Rep 2016;6:26133. [8] Cao DZ, Xu BL, Zhang SH, Wang KG. Color ghost imaging with pseudo-white-thermal light,. Chin Phys Lett 2015;32:114208. [9] Nie L, Bai Y, Fu X. Ghost telescope imaging system from the perspective of coherent-mode representation,. Opt Commun 2016;358:88–91. [10] Wang L, Zhao S. Fast reconstructed and high-quality ghost imaging with fast walsh-hadamard transform,. Photon Res 2016;4:240–4. [11] Zhao SM, Wang L, Liang W, Cheng W, Gong L. High performance optical encryption based on computational ghost imaging with QR code and compressive sensing technique,. Opt Commun 2015;353:90–5. [12] Wu H, Wang R, Huang Z, Xiao H, Liang J, Wang D, Tian X, Wang T, Cheng L. Online adaptive computational ghost imaging,. Opt Laser Eng 2020;128:106028. [13] Yu WK, Liu XF, Yao XR, Wang C, Zhai Y, Zhai GJ. Complementary compressive imaging for the telescopic system,. Sci Rep 2014;4:5834. [14] Zhang Z, Ma X, Zhong JS. Single-pixel imaging by means of fourier spectrum acquisition,. Nat Commun 2015;6:6225. [15] Wang L, Zhao SM. Multiple-input single-output ghost imaging,. IEEE Photon J 2020;12:6901113. [16] Ren HD, Zhao SM, Gruska J. Edge detection based on single-pixel imaging,. Opt Express 2018;26:5501–11. [17] Sun MJ, Edgar MP, Phillips DB, Gibson GM, Padgett MJ. Improving the signal– to-noise ratio of single-pixel imaging using digital microscanning,. Opt Express 2016;24:10476–85. [18] Tetsuno S, Shibuya X, Iwata T. Subpixel-shift cyclic-hadamard microscopic imaging using a pseudo-inverse-matrix procedure,. Opt Express, 2017;25:3420– 3432. [19] Zhao Y, Chen Q, Sui X, Gao H. Super resolution imaging based on a dynamic single pixel camera,. IEEE Photon J 2017;9:7802111. [20] Wang L, Zou L, Zhao S. Edge detection based on subpixel-speckle-shifting ghost imaging,. Opt Commun 2018;407:181–5. [21] Sun MJ, Wang HY, Huang JY. Improving the performance of computational ghost imaging by using a quadrant detector and digital micro-scanning,. Sci Rep 2019;9:4105. [22] Sui L, Cheng Y, Wang Z, Tian A, Anand KA. Single-pixel correlated imaging with high-quality reconstruction using iterative phase retrieval algorithm,. Opt Laser Eng 2018;111:108–13. [23] Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity,. IEEE Trans Image Process 2004;13:600– 612.
JID: OLEN
[m5GeSdc;November 21, 2020;22:54]
Optics and Lasers in Engineering xxx (xxxx) xxx
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Super resolution ghost imaging based on Fourier spectrum acquisition Le Wang a,∗, Shengmei Zhao a,b a
Institute of Signal Processing and Transmission, Nanjing University of Posts and Telecommunications(NUPT), Nanjing 210003, China Key Lab of Broadband Wireless Communication and Sensor Network Technology, Nanjing University of Posts and Telecommunications, Ministry of Education, Nanjing 210023, China
b
a r t i c l e
i n f o
a b s t r a c t
Keywords: Ghost imaging Single pixel imaging Super resolution Fourier spectrum
In this paper, we propose a novel super-resolution ghost imaging based on Fourier spectrum acquisition. In the scheme, we use a set of sinusoidal speckle patterns with specific phase shift to illuminate the object, and then the corresponding total intensity of reflective light is measured by a bucket detector. By performing the inverse Fourier transformation on the bucket detection results, a set of low-resolution images with subpixel shift could be reconstructed. Subsequently, a super-resolution image could be obtained by combining all the reconstructed subpixel shifted images. The experimental results demonstrate that our proposed scheme can capture a superresolution image with both direct and indirect line of sight to the object. The proposed scheme has the advantage of achieving super-resolution ghost imaging beyond the limitation of the pixel resolution of the illuminated speckle patterns and without the use of any high-precision motorized translation stages for shifting the speckle patterns.
1. Introduction
tial resolution of the speckle patterns [17]. In addition, the computing time for the reconstruction of the image significantly grows with the increase of the spatial resolution of the speckle patterns. Therefore, both the SNR and the computing time of the reconstructed images limit the improvement of imaging resolution. To overcome the limits of the resolution of the ghost imaging, the subpixel shift method is introduced into GI [17–21], which can obtain a higher resolution image of the object by illuminating a series of subpixel shifted speckle patterns with a low resolution. Hence, this method has the potential to realize the super resolution imaging. However, the realization of subpixel shift of speckle patterns relies on the high-precision mechanical shift of speckle patterns by the aid of a high-precision motorized translation stage [18]. Therefore, the cost for constructing such ghost imaging system will dramatically increase. In addition, one can achieve subpixel shift of speckle patterns by loading patterns with subpixel shifts on DMD instead of mechanical scanning [19–21], however, this way still cannot really exceed the limitation of the resolution of DMD. In this paper, we propose a novel super-resolution ghost imaging based on Fourier spectrum acquisition. In the scheme, we use a set of sinusoidal speckle patterns with specific phase shift to illuminate the object, and then the corresponding total intensity of reflective light is measured by a bucket detector. By performing the Fourier transformation on the bucket detection results, a set of low-resolution images with subpixel shift could be reconstructed. Subsequently, a super-
Ghost imaging (GI), also named single-pixel imaging (SPI), is an intriguing optical imaging technique [1,2]. Ghost imaging can acquire an image of object by correlating the speckle patterns and corresponding bucket detection results, where the bucket detection results are detected by a bucket detector without any spatial resolution and the speckle patterns are measured by a spatially resolving detector or computing offline. GI offers great promise for its better robustness against harsh environment [3,4] and multiple types of information processing [5,6]. Therefore, GI has been drawn great attention and considerable number of approaches based on GI have been proposed [7–14]. For example, the iterative phase retrieval algorithm is used in GI to realize the highquality reconstruction [22]; The total sampling time of GI is reduced by using multiple-input-single-output technique [15]; Edge of the unknown object can be obtained by GI without the imaging at advance [16]. However, the pixel resolution of the reconstructed image by ghost imaging is usually low. On the one hand, the pixel resolution of the reconstructed image by ghost imaging is determined by the spatial resolution of the speckle patterns [17,18]. The speckle patterns are usually generated by the spatial light modulator (SLM) and digital micro-mirror device (DMD), but the resolution of these devices is limited, which results in the generation of limited spatial resolution of the speckle patterns. On the other hand, the signal-to-noise ratio (SNR) of the reconstructed image by ghost imaging decreases with the increase of the spa-
∗
Corresponding author. E-mail addresses: [email protected] (L. Wang), [email protected] (S. Zhao).
https://doi.org/10.1016/j.optlaseng.2020.106473 Received 19 June 2020; Received in revised form 10 October 2020; Accepted 10 November 2020 Available online xxx 0143-8166/© 2020 Elsevier Ltd. All rights reserved.
Please cite this article as: L. Wang and S. Zhao, Super resolution ghost imaging based on Fourier spectrum acquisition, Optics and Lasers in Engineering, https://doi.org/10.1016/j.optlaseng.2020.106473
ARTICLE IN PRESS
JID: OLEN
[m5GeSdc;November 21, 2020;22:54]
L. Wang and S. Zhao
Optics and Lasers in Engineering xxx (xxxx) xxx
Fig. 1. A schematic diagram of the proposed super resolution ghost imaging scheme based on Fourier spectrum acquisition.
resolution image could be obtained by combining all the reconstructed subpixel shifted images. Furthermore, the proposed scheme can capture a super-resolution image without a direct line of sight to the object. The advantage of the proposed scheme is that it could achieve superresolution ghost imaging beyond the limitation of the pixel resolution of the speckle patterns and DMD and without the use of any high-precision motorized translation stages for shifting the speckle patterns. The organization of the paper is as follows. In Section 2, our proposed scheme is presented. In Section 3, the performance of our proposed scheme is discussed by experiments. Finally, Section 4 concludes the paper.
where 𝜂 is the bucket detector responsivity, 𝐴 is the area illuminated by sinusoidal speckle patterns, 𝑇 (𝑥, 𝑦) is the distribution function of the object, and 𝑛 represents the environmental illumination. By substituting Eq. 1 and Eq. 2 to Eq. 3, Eq. 3 can be expressed as 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) = 𝑎𝜂
∫𝐴
𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑏𝜂
∫𝐴
cos(2𝜋𝑓𝑥 (𝑥 + 𝑥0 )
+2𝜋𝑓𝑦 (𝑦 + 𝑦0 ) + 𝜑0 )𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑛 = 𝑎𝜂
∫𝐴
𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑏𝜂 cos(𝜑0 )
+2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦
2. Scheme Description
−𝑏𝜂 sin(𝜑0 ) The schematic diagram of the proposed scheme is shown in Fig. 1. A digital light projector (DLP) is used to produce a series of sinusoidal speckle patterns with specific phase shift 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) with the pixel resolution 𝑁𝑥 × 𝑁𝑦 , 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) = 𝑎 + 𝑏 cos(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦 + 𝜑𝑥 + 𝜑𝑦 + 𝜑0 ),
(1)
where (𝑓𝑥 , 𝑓𝑦 ) is the spatial frequency and (𝑥, 𝑦) are Cartesian coordinates in the space domain where −𝑋 ≤ 𝑥 ≤ 𝑋 and −𝑌 ≤ 𝑦 ≤ 𝑌 . Hence, each pixel has a spatial size 𝛿𝑥 = 2𝑋∕𝑁𝑥 and 𝛿𝑦 = 2𝑌 ∕𝑁𝑦 . 𝑎 is the direct current (DC) component, which is equal to the average intensity of the sinusoidal speckle patterns, and 𝑏 represents the contrast. We can adjust the parameters 𝑎 and 𝑏 to make sure that the DLP can display all sinusoidal speckle patterns brightly. 𝜑0 is the initial phase. 𝜑𝑥 and 𝜑𝑦 are the phase shift along the x-axis and y-axis, respectively, 𝜑𝑥 = 2𝜋𝑓𝑥 𝑥0 ,
∫𝐴
∫𝐴
cos(2𝜋𝑓𝑥 (𝑥 + 𝑥0 )
sin(2𝜋𝑓𝑥 (𝑥 + 𝑥0 ) + 2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑛.
(4)
By using the four-step phase-shift approach [14,16], one can acquire a complex Fourier coefficient of the object 𝐶(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 ) by using four bucket detection results corresponding to four sinusoidal speckle patterns where these four sinusoidal speckle patterns have the same frequency (𝑓𝑥 and 𝑓𝑦 ) and the same phase shift along the x-axis and y-axis (𝜑𝑥 and 𝜑𝑦 ) but have a different initial phase 𝜑 (𝜑 are 0, 𝜋∕2, 𝜋 and 3𝜋∕2, respectively.). 𝐶(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 ) =
1 [(𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 0) − 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋)) 2𝑏𝜂 +𝑗(𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋∕2) − 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 3𝜋∕2))], (5)
where [𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 0) − 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋)]
𝜑𝑦 = 2𝜋𝑓𝑦 𝑦0 ,
(2)
+𝑗[𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋∕2) − 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 3𝜋∕2)]
where 𝑥0 and 𝑦0 are the spatial shift along the x-axis and y-axis, respectively. Then the sinusoidal speckle pattern 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) interacts with an object and then is measured by a bucket detector to generate a bucket detection result 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ),
= 2𝑏𝜂
𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) = 𝜂
= 2𝑏𝜂 {𝑇 (𝑥 − 𝑥0 , 𝑦 − 𝑦0 )},
∫𝐴
𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 )𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 + 𝑛,
(3)
∫𝐴
−𝑗2𝑏𝜂 = 2𝑏𝜂
∫𝐴
cos(2𝜋𝑓𝑥 (𝑥 + 𝑥0 ) + 2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 ∫𝐴
sin(2𝜋𝑓𝑥 (𝑥 + 𝑥0 ) + 2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦
exp[−(2𝜋𝑓𝑥 (𝑥 + 𝑥0 ) + 2𝜋𝑓𝑦 (𝑦 + 𝑦0 ))]𝑇 (𝑥, 𝑦)𝑑 𝑥𝑑 𝑦 (6)
JID: OLEN
ARTICLE IN PRESS
[m5GeSdc;November 21, 2020;22:54]
L. Wang and S. Zhao
Optics and Lasers in Engineering xxx (xxxx) xxx
Fig. 2. A schematic diagram of the experimental system of the proposed super resolution ghost imaging scheme. DLP: digital light projector, BF: bandpass filter, BD: bucket detector, S: Aluminum laser protective screen, and ADC: analogue-to-digital converter.
Fig. 3. The experimental results of the USAF resolution chart by using the proposed scheme, where (a) is the result by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝑦0 = 0; (b) is the result by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕2, 𝑦0 = 0, 𝛿𝑦∕2; (c) is the result by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕4, 𝛿𝑥∕2, 3𝛿𝑥∕4, 𝑦0 = 0, 𝛿𝑦∕4, 𝛿𝑦∕2, 3𝛿𝑦∕4; (d) is the result by using sinusoidal speckle patterns with the pixel resolution 128 × 128 and 𝑥0 = 0, 𝑦0 = 0; (e) is the result by using sinusoidal speckle patterns with the pixel resolution 128 × 128 and 𝑥0 = 0, 𝛿𝑥∕2, 𝑦0 = 0, 𝛿𝑦∕2; (f) is the result by using sinusoidal speckle patterns with the pixel resolution 128 × 128 and 𝑥0 = 0, 𝛿𝑥∕4, 𝛿𝑥∕2, 3𝛿𝑥∕4, 𝑦0 = 0, 𝛿𝑦∕4, 𝛿𝑦∕2, 3𝛿𝑦∕4. The insets show the enlarged details of images.
and {⋅} represents the Fourier transformation. By using inverse Fourier transformation on Eq. 6, the reconstructed image of the object with the spatial shift along the x-axis, 𝑥0 , and y-axis, 𝑦0 , is able to obtain,
and 𝑦0 , 𝑇̂ (𝑥′ , 𝑦′ ) =
1 −1 𝑇̂ (𝑥 − 𝑥0 , 𝑦 − 𝑦0 ) = {[𝐵𝑖 (𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 0) − 𝐵𝑖 (𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋)] 2𝑏𝜂 +𝑗[𝐵𝑖 (𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜋∕2) − 𝐵𝑖 (𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 3𝜋∕2)]}, (7)
3. Results discussion
where −1 {⋅} represents the inverse Fourier transformation. If 𝑥0 and 𝑦0 of sinusoidal speckle patterns 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) are set as 𝑥0 < 𝛿𝑥 and 𝑦0 < 𝛿𝑦, we can reconstruct the subpixel-shifted image of the object. Hence, 𝑥0 and 𝑦0 can be set as 𝑥0 = 𝛿𝑥𝐾𝑥 and 𝑦0 = 𝛿𝑦𝐾𝑦 where both 𝐾𝑥 and 𝐾𝑦 are fractional numbers less than 1, and then we can reconstruct the 𝐾𝑥 -pixel shifted image along the x-axis and the 𝐾𝑦 -pixel shifted image along the y-axis. Lastly, the high resolution image 𝑇̂ (𝑥′ , 𝑦′ ) can be reconstructed by combining all reconstructed subpixel-shifted images with different 𝑥0
The experimental system of the proposed super resolution ghost imaging scheme is shown in Fig. 2. A series of sinusoidal speckle patterns with specific phase shift 𝐼(𝑥, 𝑦, 𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) are generated by DLP (TI Digital lightCrafter 4500) controlled by a computer and then are projected onto an object, the USAF resolution test chart. The reflective light is detected by a bucket detector (Thorlabs PMM02) after a bandpass filter (633𝑛𝑚) and then a computer records a series of bucket detection results 𝐵(𝑓𝑥 , 𝑓𝑦 , 𝜑𝑥 , 𝜑𝑦 , 𝜑0 ) via an analogue-to-digital converter
∑ 𝑥0 ,𝑦0
𝑇̂ (𝑥 − 𝑥0 , 𝑦 − 𝑦0 ).
(8)
JID: OLEN
ARTICLE IN PRESS
L. Wang and S. Zhao
[m5GeSdc;November 21, 2020;22:54] Optics and Lasers in Engineering xxx (xxxx) xxx
Fig. 4. The experimental results of the USAF resolution chart by using the proposed scheme with different density distributions of Fourier spectrum acquired from the bucket detector, where (a1)-(a6) are the results by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕2, 𝑦0 = 0, 𝛿𝑦∕2; (b1)-(b6) are the results by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕4, 𝛿𝑥∕2, 3𝛿𝑥∕4, 𝑦0 = 0, 𝛿𝑦∕4, 𝛿𝑦∕2, 3𝛿𝑦∕4; (c1)-(c6) are the density distributions of Fourier spectrum acquired from the bucket detector. The insets show the enlarged details of images.
(NI USB-6341). Subsequently, the subpixel-shifted images with different 𝑥0 and 𝑦0 can be reconstructed. Finally, the high resolution image of the object 𝑇̂ (𝑥′ , 𝑦′ ) can be obtained by combining all reconstructed subpixel-shifted images with different 𝑥0 and 𝑦0 . In addition, to verify the performance of imaging without a direct line of sight to the object for the proposed scheme, an Aluminum laser protective screen having a homogeneously diffusive surface is utilized as a scatterer and the bucket detector placed with back to object is used to detect the signals diffused from the Aluminum laser protective screen. Hence, we substitute the apparatus highlighted by a dashed blue square in Fig. 2 for that highlighted by a dashed green square in Fig. 2 in this experiment.
In order to compare the quality of the reconstructed image quantitatively, the signal-to-noise ratio (SNR) [21], the peak signal-to-noise (PSNR) and the structural similarity (SSIM) index [11,23] of the reconstructed high resolution image are used as an objective evaluation, which are defined as
𝑆𝑁𝑅 =
⟨𝐼𝑓 ⟩ − ⟨𝐼𝑏 ⟩ 1 (𝜎 2 𝑓
+ 𝜎𝑏 )
𝑃 𝑆𝑁𝑅 = 10 log10
,
max 𝑉 𝑎𝑙2 , 𝑀𝑆𝐸
(9)
(10)
ARTICLE IN PRESS
JID: OLEN L. Wang and S. Zhao
Optics and Lasers in Engineering xxx (xxxx) xxx
and 𝑆 𝑆 𝐼𝑀 =
(2𝑢𝑥 𝑢𝑦 + 𝐶1 )(2𝜎𝑥𝑦 + 𝐶2 ) (𝑢2𝑥 + 𝑢2𝑦 + 𝐶1 )(𝜎𝑥2 + 𝜎𝑦2 + 𝐶2 )
[m5GeSdc;November 21, 2020;22:54]
,
(11)
where ⟨𝐼𝑓 ⟩ and ⟨𝐼𝑏 ⟩ denote the average intensity of the feature (here calculated from the data highlighted by a solid red square in Fig. 3) and background (here calculated from the data highlighted by the dashed red square in Fig. 3), respectively. 𝜎𝑓 and 𝜎𝑏 are the standard deviations of the intensities in the feature and the background respectively. max 𝑉 𝑎𝑙 is the maximum possible pixel value of the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ). 𝑀𝑆𝐸 is the mean square error between the original image 𝑇 (𝑥′ , 𝑦′ ) and the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ). 𝑢𝑥 and 𝑢𝑦 are the average value of the original image 𝑇 (𝑥′ , 𝑦′ ) and the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ), respectively. 𝜎𝑥 and 𝜎𝑦 are the variance of the original image 𝑇 (𝑥′ , 𝑦′ ) and the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ), respectively. 𝜎𝑥𝑦 is the covariance of the original image 𝑇 (𝑥′ , 𝑦′ ) and the reconstructed image 𝑇̂ (𝑥′ , 𝑦′ ). 𝐶1 and 𝐶2 are parameters to avoid instability. To verify the feasibility of the proposed scheme, we perform the experiments and the results are shown in Fig. 3. Here we use 64 × 64-pixels and 128 × 128-pixels sinusoidal speckle patterns to illuminate the object. We show the reconstructed images in Fig. 3(b) and Fig. 3(e) obtained by the proposed scheme to combine four reconstructed subpixel-shifted images. Fig. 3(c) and Fig. 3(f) are the reconstructed images by the proposed scheme to combine 16 reconstructed subpixel-shifted images. Fig. 3(a) and Fig. 3(d) are the reconstructed images without pixel shifting. Comparing the results of Fig. 3(b) and Fig. 3(e) with those of Fig. 3(c) and Fig. 3(f), it is found that the reconstructed high-resolution images obtained by the proposed scheme to combine 16 reconstructed subpixelshifted images are clearer and corresponding SNRs are higher than those of combining four reconstructed subpixel-shifted images. From the enlarged details of images shown in the insets, we can further find that the resolution of the former is higher than the latter, because the smallest pixel shifting of reconstructed subpixel-shifted images to be combined of the former is half of that of the latter and thus the former has more sub-pixel information than the latter. In addition, in order to verify the ability of super-resolution imaging with the aid of the USAF resolution test chart, we compare the results
of the proposed scheme to the results by using traditional ghost imaging based on sinusoidal speckle patterns without subpixel shifting shown in Fig. 3(a) and Fig. 3(d). The USAF resolution test chart consists of multiple patterns of three bar targets with dimensions from big to small. Here, all patterns of three bar targets are divided into four groups (The groups are numbered 0, 1, 2, and 3.) and each group consists of six elements where each element has three horizontal bars and three vertical bars (The elements are numbered 1, 2, 3, 4, 5 and 6.). By reading off the group and element number of the first element which cannot be resolved, the limiting resolution may be determined by inspection and the resolution is defined as 𝑅𝑒𝑠 = 2𝐺+
𝑒−1 6
,
(12)
where 𝐺 and 𝑒 are the group number and the element number, respectively. From the enlarged details of images shown in Fig. 3(a)-(c) with 𝐺 = 1 and 𝑒 = 1, the limiting resolution by the proposed scheme (Fig. 3(b) and Fig. 3(c)) is larger than 2, i.e. 𝑅𝑒𝑠 ≥ 2, however, the limiting resolution by the traditional ghost imaging scheme (Fig. 3(a)) is smaller than 2, i.e. 𝑅𝑒𝑠 < 2. Similarly, since the group number and the element number of the enlarged details of images shown in Fig. 3(d)(f) are 𝐺 = 2 and 𝑒 = 2, the limiting resolution by the proposed scheme (Fig. 3(e) and Fig. 3(f)) is larger than 213∕6 but the traditional ghost imaging scheme (Fig. 3(d)) has the limiting resolution 𝑅𝑒𝑠 < 213∕6 . Therefore, the results show that the proposed scheme can acquire a higher resolution image comparing to traditional ghost imaging based on sinusoidal speckle patterns when the sinusoidal speckle patterns with the same pixel resolution are used. Additionally, SNRs of reconstructed images by using the proposed schemes are significantly higher than those of traditional ghost imaging. Moreover, we perform the experiments and obtain the results by using the proposed scheme with different density distributions of Fourier spectrum acquired from the bucket detector, which are shown in Fig. 4. It is seen that with the increase of the density distributions of Fourier spectrum, the image recovered by the proposed scheme becomes clearer as well as the corresponding SNR gradually increases and its resolution is gradually improved. In addition, we compare the results of Fig. 4(a) with those of Fig. 4(b), we can find that the reconstructed high-resolution
Fig. 5. The experimental results of the USAF resolution chart by using the proposed scheme with or without a direct line of sight to the object, where (a) and (c) are the results by using the proposed scheme with a direct line of sight to the object; (b) and (d) are the results by using the proposed scheme without a direct line of sight to the object; (a) and (b) are the results by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕2, 𝑦0 = 0, 𝛿𝑦∕2; (c) and (d) are the results by using sinusoidal speckle patterns with the pixel resolution 64 × 64 and 𝑥0 = 0, 𝛿𝑥∕4, 𝛿𝑥∕2, 3𝛿𝑥∕4, 𝑦0 = 0, 𝛿𝑦∕4, 𝛿𝑦∕2, 3𝛿𝑦∕4; (a1), (b1), (c1) and (d1) are the results with the density distributions of Fourier spectrum coverage 100%; (a2), (b2), (c2) and (d2) are the results with the density distributions of Fourier spectrum coverage 80%. The insets show the enlarged details of images.
JID: OLEN
ARTICLE IN PRESS
L. Wang and S. Zhao
images obtained by combining 16 reconstructed subpixel-shifted images are clearer and corresponding SNRs and resolution are higher than those of combining four reconstructed subpixel-shifted images under the same condition of density distributions of Fourier spectrum. Lastly, we perform the experiments of the proposed scheme without a direct line of sight to the object, and the results are compared to those results with a direct line of sight to the object, which are shown in Fig. 5. The experimental results show that the proposed scheme can capture a super-resolution image without a direct line of sight to the object and the imaging performance is similar to that with a direct line of sight to the object. The reconstructed images obtained by combining 16 reconstructed subpixel-shifted images are clearer and corresponding SNRs and resolution are higher than those of combining four reconstructed subpixel-shifted images and the reconstructed images become clearer as well as the corresponding SNR is higher and its resolution is gradually improved with the increase of the density distributions of Fourier spectrum. However, the experimental results show that the proposed scheme without a direct line of sight to the object will have a slightly lower SNR than the proposed scheme with a direct line of sight to the object. 4. Conclusion We have proposed a novel super-resolution ghost imaging by using Fourier spectrum acquisition in the paper. We have performed the experiments to verify that the proposed scheme can achieve superresolution ghost imaging beyond the limitation of the pixel resolution of the speckle patterns. In addition, the proposed scheme has the ability of the super-resolution imaging without a direct line of sight to the object. Comparing to existing super-resolution ghost imaging schemes, the advantage of the proposed scheme is that it could capture a superresolution image without the use of any high-precision motorized translation stages for shifting the speckle patterns. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. CRediT authorship contribution statement Le Wang: Conceptualization, Methodology, Software, Formal analysis, Writing - original draft. Shengmei Zhao: Conceptualization, Supervision, Project administration, Writing - original draft. Acknowledgment The paper is supported by the National Natural Science Foundation of China (NSFC) (61871234), the Natural Science Foundation of Jiangsu Province (BK20180755), the Open research fund of Key Lab of
[m5GeSdc;November 21, 2020;22:54] Optics and Lasers in Engineering xxx (xxxx) xxx
Broadband Wireless Communication and Sensor Network Technology (JZNY201910), and the NUPTSF (NY218098, NY220004).
References [1] Strekalov DV, Sergienko AV, Klyshko DN, Shih YH. Observation of two-photon “ghost” interference and diffraction,. Phys Rev Lett 1995;74:3600–3. [2] Bennink RS, Bentley SJ, Boyd RW. “two-photon” coincidence imaging with a classical source,. Phys Rev Lett 2002;89:113601. [3] Zhao SM, Yang H, Li YQ, Cao F, Sheng YB, Cheng WW, Gong LY. The influence of atmospheric turbulence on holographic ghost imaging using orbital angular momentum entanglement: Simulation and experimental studies,. Opt Commun 2013;294:223–8. [4] Yin MQ, Wang L, Zhao SM. Experimental demonstration of influence of underwater turbulence on ghost imaging,. Chin Phys B 2019;28:094201. [5] Wang L, Zhao S, Cheng W, Gong L, Chen H. Optical image hiding based on computational ghost imaging,. Opt Commun 2016;366:314–20. [6] Jiao S, Zhou C, Shi Y, Zou W, Li X. Review on optical image hiding and watermarking techniques,. Opt Laser Technol 2019;109:370–80. [7] Gong W, Zhao C, Yu H, Chen M, Xu W, Han S. Three-dimensional ghost imaging lidar via sparsity constraint,. Sci Rep 2016;6:26133. [8] Cao DZ, Xu BL, Zhang SH, Wang KG. Color ghost imaging with pseudo-white-thermal light,. Chin Phys Lett 2015;32:114208. [9] Nie L, Bai Y, Fu X. Ghost telescope imaging system from the perspective of coherent-mode representation,. Opt Commun 2016;358:88–91. [10] Wang L, Zhao S. Fast reconstructed and high-quality ghost imaging with fast walsh-hadamard transform,. Photon Res 2016;4:240–4. [11] Zhao SM, Wang L, Liang W, Cheng W, Gong L. High performance optical encryption based on computational ghost imaging with QR code and compressive sensing technique,. Opt Commun 2015;353:90–5. [12] Wu H, Wang R, Huang Z, Xiao H, Liang J, Wang D, Tian X, Wang T, Cheng L. Online adaptive computational ghost imaging,. Opt Laser Eng 2020;128:106028. [13] Yu WK, Liu XF, Yao XR, Wang C, Zhai Y, Zhai GJ. Complementary compressive imaging for the telescopic system,. Sci Rep 2014;4:5834. [14] Zhang Z, Ma X, Zhong JS. Single-pixel imaging by means of fourier spectrum acquisition,. Nat Commun 2015;6:6225. [15] Wang L, Zhao SM. Multiple-input single-output ghost imaging,. IEEE Photon J 2020;12:6901113. [16] Ren HD, Zhao SM, Gruska J. Edge detection based on single-pixel imaging,. Opt Express 2018;26:5501–11. [17] Sun MJ, Edgar MP, Phillips DB, Gibson GM, Padgett MJ. Improving the signal– to-noise ratio of single-pixel imaging using digital microscanning,. Opt Express 2016;24:10476–85. [18] Tetsuno S, Shibuya X, Iwata T. Subpixel-shift cyclic-hadamard microscopic imaging using a pseudo-inverse-matrix procedure,. Opt Express, 2017;25:3420– 3432. [19] Zhao Y, Chen Q, Sui X, Gao H. Super resolution imaging based on a dynamic single pixel camera,. IEEE Photon J 2017;9:7802111. [20] Wang L, Zou L, Zhao S. Edge detection based on subpixel-speckle-shifting ghost imaging,. Opt Commun 2018;407:181–5. [21] Sun MJ, Wang HY, Huang JY. Improving the performance of computational ghost imaging by using a quadrant detector and digital micro-scanning,. Sci Rep 2019;9:4105. [22] Sui L, Cheng Y, Wang Z, Tian A, Anand KA. Single-pixel correlated imaging with high-quality reconstruction using iterative phase retrieval algorithm,. Opt Laser Eng 2018;111:108–13. [23] Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity,. IEEE Trans Image Process 2004;13:600– 612.