- Email: [email protected]
Collective noise model for focal plane modulated single-pixel imaging
Optics and Lasers in Engineering 100 (2018) 18–22
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Collective noise model for focal plane modulated single-pixel imaging Ming-Jie Sun a,∗, Zi-Hao Xu a, Ling-An Wu b a b
School of Instrementation Science and Opto-electronic Engineering, Beihang University, Beijing, 100191, China Quantum Engineering Research Center, Beijing Institute of Aerospace Control Devices, China Aerospace, Beijing, 100094, China
a r t i c l e
i n f o
Keywords: Single-pixel imaging Collective noise model Computational imaging
a b s t r a c t Single-pixel imaging, also known as computational ghost imaging, provides an alternative method to perform imaging in various applications which are difficult for conventional cameras with pixelated detectors, such as multi-wavelength imaging, three-dimensional imaging, and imaging through turbulence. In recent years, many improvements have successfully increased the signal-to-noise ratio of single-pixel imaging systems, showing promise for the engineering feasibility of this technique. However, many of these improvements are based on empirical findings. In this work we perform an investigation of the noise from each system component that affects the quality of the reconstructed image in a single-pixel imaging system based on focal plane modulation. A collective noise model is built to describe the resultant influence of these different noise sources, and numerical simulations are performed to quantify the effect. Experiments have been conducted to verify the model, and the results agree well with the simulations. This work provides a simple yet accurate method for evaluating the performance of a single-pixel imaging system, without having to carry out actual experimental tests. © 2017 Elsevier Ltd. All rights reserved.
1. Introduction Since a major part of the information processed in our daily life is in graphic form, imaging technology is one of the most important tools in the development of society. Conventional digital imaging uses a lens to map the spatial information from a scene onto the focal plane, where a pixelated array records the light intensity. Over the past two decades, an alternative way to perform imaging known as ghost imaging [1–3] or single-pixel imaging [4–6] has aroused much interest in the scientific community. Single-pixel imaging (SPI) reconstructs an image by measuring the correlations between the scene and a series of masks. It enables various applications, such as multi-wavelength imaging [7,8], three-dimensional imaging [9,10] and imaging through turbulence under certain circumstances [11–13], all of which pose difficulties for conventional imaging. Besides environmental effects, internal noise of the imaging system is an important factor that determines the image quality for both conventional and SPI approaches. In conventional digital imaging systems, the noise originates from the electronic readout of the pixel array, which has a direct additive effect on the corresponding image. In SPI, however, several different components contribute to the noise, such as the light source and the bucket detector, and their effects on image quality are less straightforward due to the image reconstruction mechanism. As a matter of fact, the signal-to-noise ratio (SNR) issue is one of the major obstacles facing widespread application of SPI. ∗
Corresponding author. E-mail address: [email protected] (M.-J. Sun).
http://dx.doi.org/10.1016/j.optlaseng.2017.07.005 Received 19 April 2017; Received in revised form 5 July 2017; Accepted 7 July 2017 0143-8166/© 2017 Elsevier Ltd. All rights reserved.
Many promising schemes have been proposed to improve the SNR in the last decade. Computational ghost imaging [14] uses a spatial light modulator (SLM) to replace the reference arm in the original secondorder intensity correlation imaging systems, simplifying the setup and improving the performance. Differential ghost imaging [15] normalizes the total intensity of each measurement, minimizing the effect of intensity instabilities of the source. Compressive sensing [16,17] takes advantage of sparsity in the scene and improves the reconstructed image by minimizing a certain measure of the sparsity. High-order ghost imaging [18,19] exploits higher-order correlation to increase the SNR. Positive–negative ghost imaging [20,21] utilizes the symmetry of noise in positive and negative fluctuations, partially canceling the noise in the system. Single-pixel imaging based on different orthonormal bases [5–7,22,23] maintains the orthogonality within a series of sampling masks, leading to theoretically perfect reconstruction and a much smaller number of measurements. Digital microscanning [24] applies a super-resolution technique [25, 26] to achieve a better SNR and higher resolution. However, many of these improvements are obtained after the measurements have been performed, and the noise evaluations were based on the specific system configurations, which might pose certain difficulties for starting researchers not familiar with the area to understand the roles of the different noise sources and to decide what devices to choose in order to set up their own SPI systems with the desired performance.
M.-J. Sun et al.
Optics and Lasers in Engineering 100 (2018) 18–22
by driving current fluctuations. The characterization of laser noise has been widely investigated and is quite complicated if all aspects were considered [27,28]. For our purposes, we measured the output of a typical continuous wave diode laser with a low noise PIN detector (Thorlabs DET10A), and from 100,000 measured points determined that the random fluctuations exhibit an approximate Gaussian dependence on current. Modeling the light intensity as 𝐼0 + ▵ 𝐼𝑖 , from Eqs. (1) and (2) the reconstructed image OR, Light can be expressed as ) ( 𝑁 ∑ ∑ (( ∑ ) ) 𝑶𝑅,𝐿𝑖𝑔ℎ𝑡 = 𝑴𝑖 ⋅ (3) 𝐼0 + ▵ 𝐼𝑖 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 .
Fig. 1. Schematic diagram of an SPI system based on focal plane modulation.
▵ 𝑶𝑅,𝐿𝑖𝑔ℎ𝑡 =
𝑚
𝑖=1
𝑛
Subtracting Eq. (2) from Eq. (5), the noise introduced into the image is ▵ 𝑶𝑅,𝑆𝐿𝑀 =
𝑁 ∑
(
𝑖=1
𝐼0 ⋅ 𝑴 𝑖 ⋅
∑∑( 𝑚
𝑛
▵ 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛
) )
.
(6)
If the error caused by each pixel is due to imperfect manufacturing of the device, then ▵Mi can be viewed as a constant ▵M during N measurements. In this case, ▵OR, SLM adds only a constant to the reconstructed images, which can be reduced by normalization and so is inconsequential. If the error arises from the instability of the device voltage, then ▵M follows a similar Gaussian distribution to that of the device current fluctuations. 2.3. Detector noise There are two kinds of noise in a bucket detector. One is dark current ▵Di , which exists in the readouts of the detector when there is no incident light and can be considered to have a Gaussian distribution with a mean value of D [29]. The other type of noise ▵Si is induced by incident light, and its level is proportional to the detected signal Si . The measured signal is 𝑆𝑖 + ▵ 𝑆𝑖 + ▵ 𝐷𝑖 when light is present, and the reconstructed image OR, Det can be expressed as ( ) 𝑁 ∑ ∑∑( ) 𝐼0 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 + ▵ 𝑆𝑖 + ▵ 𝐷𝑖 . 𝑶𝑅,𝐷𝑒𝑡 = (𝑴 𝑖 ⋅ (7)
𝑛
𝑚
(4)
𝑖=1
The SLM noise arises from aberrations in surface curvature, comprising low order Zernike polynomials if the SLM is liquid crystal based, or from fluctuations in the tilt angles of the micromirrors if it is a digital micromirror device (DMD). In both cases, the masks generated by the SLM are 𝑴 𝑖 + ▵ 𝑴 𝑖 , where ▵Mi is an error function of i, m and n. The reconstructed image OR, SLM can be expressed as ( ) 𝑁 ∑∑( ( ∑ ) ) 𝑶𝑅,𝑆𝐿𝑀 = 𝑴𝑖 ⋅ (5) 𝐼0 𝑀𝑖,𝑚𝑛 + ▵ 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 .
where m and n indicate the horizontal and vertical coordinates of the mask, respectively. If the masks Mi are orthonormal such that their transpose is the same as their inverse, by performing N independent SLM mask exposures, the image OR can be perfectly reconstructed as [24] ) ( 𝑁 𝑁 ∑ ∑∑( ( ) ∑ ) 𝑴 𝑖 ⋅ 𝑆𝑖 = 𝐼0 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 . 𝑶𝑅 = 𝑴𝑖 ⋅ (2) 𝑖=1
𝑁 ∑ ( ) ▵ 𝐼𝑖 ⋅ 𝑴 𝑖 ⋅ 𝑆𝑖 ∕𝐼0 .
2.2. Spatial light modulator noise
In an SPI system based on focal plane modulation, as illustrated in Fig. 1, light of intensity I0 emitted from a source illuminates a scene O. The spatial information of the scene is imaged by an imaging lens onto its focal plane, where an SLM generates intensity modulation masks Mi , for up to i = 1, 2, …, N measurements. The transmitted or reflected light is then collected, and its total intensity signal Si measured by a bucket detector is ∑∑( ) 𝐼0 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 , 𝑆𝑖 = (1)
𝑖=1
𝑛
Subtracting Eq. (2) from Eq. (3), the noise introduced into the resulting image is
2. Collective noise model
𝑚
𝑚
𝑖=1
In two SPI schemes which are essentially the same, one scheme uses structured light illumination [5, 14], and the other uses focal plane modulation [4,24]. In this work we investigate the noise transmitted from the system components to the resulting reconstructed image for an SPI setup based on focal plane modulation. Digitization electronics is a common source of noise and is a topic that has been widely discussed. The main understanding is that the higher the digitization resolution, the less noise there will be, and the improvement approaches its limit as the resolution increases. However, higher resolution means slower sampling rates and larger amount of data transfer and computing, therefore a more practical question for setting up an SPI system is which digitizer would be best for the application. For example, a high resolution digitizer would be preferable for static imaging, and a low resolution but fast sampling rate digitizer for real-time motion detection, but this issue will not be discussed here. In this work, noises from the light source, SLM and detector are analyzed separately, their overall effect on image quality is investigated, and a collective noise model is built. Numerical simulations at different noise levels are performed based on this model to visualize the effect of each noise component on the reconstructed image. Measurements on an experimental setup agree well with the numerical simulation, demonstrating that our model is effective in predicting the performance of SPI systems. Our work provides a simple yet accurate noise model for researchers who are interested in SPI to understand and evaluate the system performance of their own SPI configuration before having to actually set up the system.
𝑚
𝑖=1
𝑛
Subtracting Eq. (2) from Eq. (7), the noise introduced is
𝑛
▵ 𝑶𝑅,𝐷𝑒𝑡 =
During the imaging process, if the focusing lens has zero aberration, we only have to consider the noise due to the light source, the SLM, and the bucket detector (digitization electronics is not included in the scope of this work). The contribution from each of these components will now be analyzed below to formulate a collective noise model.
𝑁 ∑ ( ) (▵ 𝑆𝑖 + ▵ 𝐷𝑖 ⋅ 𝑴 𝑖 ).
(8)
𝑖=1
We can further separate the detector induced noise into the signalrelated noise ▵OR, Det, S and signal-unrelated noise ▵OR, Det, D , as follows:
2.1. Light source noise
▵ 𝑶𝑅,𝐷𝑒𝑡,𝑆 =
For a laser operating well above threshold we can ignore its quantum fluctuations, so the chief source of noise in its emission is caused
▵ 𝑶𝑅,𝐷𝑒𝑡,𝐷 = 19
𝑁 ∑ ( ) ▵ 𝑆𝑖 ⋅ 𝑴 𝑖 ,
(9)
𝑖=1
𝑁 ∑ ( 𝑖=1
) ▵ 𝐷𝑖 ⋅ 𝑴 𝑖 .
(10)
M.-J. Sun et al.
Optics and Lasers in Engineering 100 (2018) 18–22
2.4. Collective noise model Following the analysis above, we can express the reconstructed image OR, Noise under the overall influence of noise from the light source, SLM and detector as 𝑶𝑅,𝑁𝑜𝑖𝑠𝑒 = =
𝑁 ∑ 𝑖=1
( (𝑴 𝑖
𝑁 ∑ ( 𝑖=1
+
𝑚
𝑛
)
𝑁 ∑
𝑴 𝑖 𝑆𝑖 +
𝑁 ∑ 𝑖=1
+
∑∑ ( )( ) (( 𝐼0 + ▵ 𝐼𝑖 𝑀𝑖,𝑚𝑛 + ▵ 𝑀𝑚𝑛 ⋅ 𝑂𝑚𝑛 )+ ▵ 𝑆𝑖 + ▵ 𝐷𝑖 𝑁 ∑ 𝑖=1
𝑴 𝑖 𝐼0
∑∑( ▵ 𝑀𝑖, 𝑚
𝑛
𝑚𝑛
⋅ 𝑂𝑚𝑛
)
∑∑( ) 𝑴 𝑖 ▵ 𝐼𝑖 ▵ 𝑀𝑚𝑛 𝑂𝑚𝑛 𝑚
𝑁 ∑ ( 𝑖=1
𝑖=1
(▵ 𝐼 𝑖 𝑴 𝑖 𝑆 𝑖 ∕𝐼 0 ) +
)
𝑛
)
▵ 𝑆𝑖 ⋅ 𝑴 𝑖 +
𝑁 ∑ ( 𝑖=1
▵ 𝐷𝑖 ⋅ 𝑴 𝑖
)
= 𝑶𝑅 + ▵ 𝑶𝑅,𝐿𝑖𝑔ℎ𝑡 + ▵ 𝑶𝑅,𝑆𝐿𝑀 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝑆 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝐷 𝑁
+
∑ 𝑖=1
𝑴 𝑖 ▵ 𝐼𝑖 ⋅
∑∑( 𝑚
𝑛
) ▵ 𝑀𝑚𝑛 𝑂𝑚𝑛 .
Fig. 2. Numerical simulation of the reconstructed image RMSE as a function of the RMS of each individual noise source and the collective noise. Each point is the ensemble average over 100 simulations.
(11)
If all the non-noise variables are normalized in Eq. (11), then all the noise contributions will be fractional. The last term of Eq. (11) contains two noise terms (▵Ii and ▵Mmn ), the product of which is one order smaller than the other terms so it can be neglected, and the reconstructed image and its collective noise become, respectively, 𝑶𝑅,Noise = 𝑶𝑅 + ▵ 𝑶𝑅,Light + ▵ 𝑶𝑅,𝑆𝐿𝑀 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝑆 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝐷
•
We can therefore separate the noises in an SPI system into two kinds, signal related and signal unrelated noise. The former has a smaller and more linear effect on the quality of reconstructed images compared to the latter. Keeping the signal unrelated noise as low as possible is crucial for a high performance SPI system, preferably below 0.2 RMS. In other words, a high precision SLM and a low dark current detector are more important to the performance of an SPI system than a stable illumination source. Besides having a high modulation rate, one of the reasons that a DMD is a popular choice for many recent SPI works is that it is in general much flatter than liquid crystal based SLMs, and the variation in its mirror tilt and reflection coefficient is minimal. We further investigated the effect of the collective noise by using Eq. (13) to increase the RMS values of all the noise sources simultaneously through a 0.05 increment, as shown by the light blue curve in Fig. 2. As we can see, the collective RMSE is not equivalent to the sum of each independent RMSE, because the noise transmission is not a linear process in SPI.
(12)
and ▵ 𝑶𝑅,Noise =▵ 𝑶𝑅,Light + ▵ 𝑶𝑅,𝑆𝐿𝑀 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝑆 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝐷 .
(13)
3. Results and discussion 3.1. Numerical simulation To visualize the effect of the total noise on a focal plane modulation based SPI system, numerical simulations were carried out in Labview using the collective noise model, i.e., Eqs. (12) and (13). The SLM sampling masks were 64 × 64 pixels in size and generated from a 4096 × 4096 Hadamard matrix [30,31], which is a typical choice for single-pixel imaging [7,8,10,23,24] and provides perfect reconstruction when full sampling is performed [22,32]. Non-orthogonal random sampling can also be used, but the noise it introduces is not related to any device and can be reduced statistically by increasing the sampling number, so its analysis will not be included here. To compare the influence of different noise sources on the reconstructed image, all physical entities are normalized, i. e., we assume 𝐼0 = 1, the matirx elements of Mi take the values of ‘−0.5′ and ‘0.5′, O has a reflectivity between 0 and 1, and the maximum intensity at the detector is normalized to 1. The noise level is represented by its rootmean-square (RMS) value. The influence of the noise is represented by the root-mean-square error (RMSE) between O and OR, X , i.e., the RMS of ▵OR, X , where X refers to a certain kind of noise. We first investigated the effect of each noise source independently by increasing its RMS value by a 0.05 increment for each reconstruction. The red, indigo, green and orange curves in Fig. 2 show the RMSE as a function of the light source noise ▵Ii , SLM noise ▵Mi , signal related detector noise ▵Si , and detector dark current ▵Di , respectively. Interestingly, the effects of ▵Ii and ▵Si almost overlap, while the effects of ▵Mi and ▵Di are very similar. The reasons behind this phenomenon are: •
•
mean due to the random nature of ▵Mi , and ▵OR, SLM is also unrelated to Si , so ▵OR, Det, D and ▵OR, SLM behave similarly. Because all physical entities in the simulation are normalized to 1, the numerical effects of noises with the same characteristics can be directly compared.
3.2. Experiment To validate the accuracy of our collective model, experiments following the procedure in Fig. 1 were performed. The beam from a 635 nm laser (PicoQuant PDL 800-B, laser head LDH-P-C-635M, 4 mW) was expanded by a diffuser to illuminate a white circle printed on a square piece of black paper. This object was then imaged by a camera lens (Nikon AF Nikkor EFL 50 mm, f/1.8D) onto the DMD (Texas Instruments Discovery 4100, 1024 × 768 pixels, wavelength 350 − 2500 nm) which sequentially generated 64 × 64 pixel masks formed from a 4096 × 4096 Hadamard matrix. The image was modulated by the masks and the total intensity of the reflected light was detected by a bucket detector (Thorlabs PDA100A-EC Si amplified detector with an ACL25416U-A lens). A high dynamic range analogue-to-digital converter, (PicoScope 6404D, sampling rate set to 2.5 Giga Samples/s, ADC resolution set to 12 bits, both to ensure that the digitization influence is minimized and negligible) that was synchronized with the DMD, acquired and transferred the intensity data to a laptop computer to perform image reconstruction using Eq. (11). The purpose of using a white circle on a black background as object was to accurately calculate the RMSE between the experimental result and the original object, since the edges of the black square could be precisely matched with the image frame.
▵OR, Laser is related to the measured signal Si , while ▵OR, Det, S is related to ▵Si which in turn is proportional to Si , therefore ▵OR, SLM and ▵OR, Det, S have the same trend. ▵OR, Det, D is related to ▵Di , which has a constant mean D and is not related to the signal Si . Since ▵OR, SLM depends on the sum of the terms in ▵Mi · O, which is a Gaussian distribution with a constant 20
M.-J. Sun et al.
Optics and Lasers in Engineering 100 (2018) 18–22
4. Conclusion Though not every noise factor related to SPI has been included, we believe that the collective noise model proposed here covers the most significant noise sources in the system. For laser illuminated conventional imaging, speckle can severely affect image quality [33]. However, because of the temporal changing of the patterns or modulation of the SLM and the cumulative measurements of SPI, laser speckle can be dramatically suppressed [34,35]. Experimental results demonstrate that laser speckle does not have a severe effect on SPI image quality. To summarize, we have investigated the individual effects of the noise from each system component on the resulting image in focal plane modulation based SPI. Through numerical simulation, we find that the noise can be catalogued into two kinds, signal related and signal unrelated noise. Their combined effect can be described by a collective noise model, which we have also simulated. Experimental results show good agreement with our simulation. By investigating the effects of different noises, we have verified that signal unrelated noise has a worse effect on image quality and therefore should be given greater attention when setting up an SPI system. Measures to decrease the signal unrelated noise include using a high precision SLM and a low dark current detector. Although there has been some rigorous analysis on the influence of different noise sources on image reconstruction [36,37], the model we present here provides a simple yet accurate method to evaluate performance without having to actually carry out experimental tests. This should be of interest to those who are interested in setting up an SPI system as efficiently as possible. Fig. 3. (a) Numerically simulated RMSE of the reconstructed image (light blue circular dots and curve) and experimental values (red sqaure dots) as a function of the RMS of collective system noise. Each experimental dot is the ensemble average over the RMSE of 20 reconstructed images. (b) Reconstructed images of a white circle printed on black paper as an object; the corresponding RMSE and RMS noise value (NRMS) are given below each image. (c) Reconstructed images of an actual doll, with the same NMRS as the circular object above.
Funding information This work was supported by the National Natural Science Foundation of China [grant number 61675016, 61307021], and Beijing Natural Science Foundation [grant number 4172039]. References
For ease of comparison, the light blue curve of Fig. 2 showing the numerically simulated RMSE is replicated in Fig. 3(a). The first red dot, representing the experimental RMSE, was calculated and plotted from the highest quality image (first item of Fig. 3(b)), i.e. the experimental system is calibrated against its minimum noise level, which is ∼0.027 of the RMS noise value. Starting from this point, we are able to introduce artificial noises into the system as follows. •
•
•
[1] Pittman TB, Shih YH, Strekalov DV, Sergienko AV. Optical imaging by means of two-photon quantum entanglement. Phys Rev A 1995;52:R3429. [2] Bennink RS, Bentley SJ, Boyd RW. ’Two-photon’ coincidence imaging with a classical source. Phys Rev Lett 2002;89:113601. [3] Yang C, Wang C, Guan J, Zhang C, Guo S, Gong W, et al. Scalar-matrix-structured ghost imaging. Photonics Res 2016;4:281–5. [4] Duarte MF, Davenport MA, Takhar D, Laska JN, Sun T, Kelly K, et al. Single-pixel imaging via compressive sampling. IEEE Signal Process Mag 2008;25:83–91. [5] Bromberg Y, Katz O, Silberberg S. Ghost imaging with a single detector. Phys Rev A 2009;79:053840. [6] Zhang Z, Ma X, Zhong J. Single-pixel imaging by means of Fourier spectrum acquisition. Nat Commun 2015;6:6225–30. [7] Radwell N, Mitchell KJ, Gibson GM, Edgar MP, Bowman R, Padgett MJ. Single-pixel infrared and visible microscope. Optica 2014;1:285–9. [8] Edgar MP, Gibson GM, Bowman RW, Sun B, Radwell N, Mitchell KJ, et al. Simultaneous real-time visible and infrared video with single-pixel detectors. Sci Reports 2015;5:10669. [9] Sun B, Edgar MP, Bowman RW, Vittert LE, Welsh SS, Bowman A, et al. 3D computational imaging with single-pixel detectors. Science 2013;340:844–7. [10] Sun M-J, Edgar MP, Gibson GM, Sun B, Radwell N, Padgett MJ. Single-pixel three-dimensional imaging with a time-based depth resolution. Nat Commun 2016;7:12010. [11] Meyers RE, Deacon KS, Shih Y. Turbulence-free ghost imaging. Appl Phys Lett 2011;98:111115. [12] Zhao C, Gong W, Chen M, Li E, Wang H, Xu W, et al. Ghost imaging lidar via sparsity constraints. Appl Phys Lett 2012;101:141123. [13] Shi D, Fan C, Zhang P, Shen H, Zhang J, Qiao C, et al. Two-wavelength ghost imaging through atmospheric turbulence. Opt Express 2013;21:2050–64. [14] Shapiro JH. Computational ghost imaging. Phys Rev A 2008;78:061802. [15] Ferri F, Magatii D, Lugiato LA, Gatti A. Differential ghost imaging. Phys Rev A 2010;104:253603. [16] Candes EJ. Compressive sampling. In: Proceedings of the international congress of mathematicians. International Mathematical Union; 2006. p. 1433–52. [17] Donoho DL. Compressed sensing. IEEE Trans Inform Theory 2006;52:1289–306. [18] Chen X-H, Agafonov IN, Luo K-H, Liu Q, Xian R, Chekhova MV, et al. High-visibility, high-order lensless ghost imaging with thermal light. Opt Lett 2010;35:1166–8. [19] Cao B, Zhang C. Third-order lensless ghost diffraction with classical fully incoherent light. Opt Lett 2010;35:2091–3. [20] Sun M-J, Li M-F, Wu L-A. Nonlocal imaging of a reflective object using positive and negative correlations. Appl Opt 2015;54:7494–9.
The laser power was varied by changing its driving current to simulate severe illumination fluctuations. Greyscale masks were generated on the DMD using its 8 bit greyscale mode to simulate micromirror jitter. Since the noise level of the detector we used in our experiment was not adjustable, the noise of the detector was first measured with no incident light; this was found to exhibit a fair Gaussian distribution. Then, artificial random noises of different levels were generated to mimic detectors with different noise levels. This simulated either the signal related ▵OR, Det, S or the signal unrelated ▵OR, Det, D , to imitate different noise types of the detector.
The equivalent increment in RMS noise was 0.05 for each individual noise configuration. Fig. 3(a) shows that the experimental results are in good agreement with the numerical simulation, demonstrating that our collective noise model is effective in predicting the performance of focal plane modulation based SPI systems. The reconstructed image was quite noisy when the RMS reached 0.377 (the fourth image of Fig. 3(b)) and became beyond recognition after the noise reached 0.527, hence no further experiment was performed. In a different experiment, a plastic doll was put on the table as a more realistic object. Similar quality images were obtained (Fig. 3(c)), with the same noise levels, showing that the SPI system and collective noise model work just as well with actual objects. 21
M.-J. Sun et al.
Optics and Lasers in Engineering 100 (2018) 18–22
[21] Sun M-J, He X-D, Li M-F, Wu L-A. Thermal light subwavelength diffraction using positive and negative correlations. Chin Opt Lett 2016;14:040301. [22] Sankaranarayanan AC, Studer C, Baraniuk RG. Cs-muvi: Video compressive sensing for spatial multiplexing cameras. In: Proceedings of IEEE international conference on computational photography. IEEE; 2012. p. 1–10. [23] Le W, Zhao S. Fast reconstructed and high-quality ghost imaging with fast Walsh–Hadamard transform. Photonics Res 2016;4:240–4. [24] Sun M-J, Edgar MP, Phillips DB, Gibson GM, Padgett MJ. Improving the signal– to-noise ratio of single-pixel imaging using digital microscanning. Opt Express 2016;24:10476–85. [25] Nasrollahi K, Moeslund TB. Super-resolution: a comprehensive survey. Mach Vision Appl 2014;25:1423–68. [26] Sun M, Yu K. A sur-pixel scan method for super-resolution reconstruction. Optik 2013;124:6905–9. [27] Petermann K. Laser diode modulation and noise. Berlin: Springer Science & Business Media; 2012. p. 152–208. Vol. 3. [28] Ralph TC, Huntington EH, Harb CC, Buchler BC, Lam PK, McClelland DE, et al. Understanding and controlling laser intensity noise. Optical and quantum electronics 1999;31:583–98.
[29] Srinivasan M, Vilnrotter V. Symbol-error probabilities for pulse-position modulation signaling with an avalanche photodiode receiver and Gaussian thermal noise. In: The telecommunications and mission operations progress report 42-134; 1998. p. 1–11. April–June 1998. [30] Pratt WK, Kane J, Andrews HC. Hadamard transform image coding. In: Proceedings of the IEEE, 57; 1969. p. 58–68. [31] Sloane NJ, Harwit M. Masks for Hadamard transform optics, and weighing designs. Appl Opt 1976;15:107–14. [32] Schechner YY, Nayar SK, Belhumeur PN. Multiplexing for optimal lighting. IEEE Trans Pattern Anal Mach Intell 2007;29:1339–54. [33] Goodman JW. Laser speckle and related phenomena. Springer-Verlag; 1984. [34] Trisnadi JI. Hadamard speckle contrast reduction. Opt Lett 2004;29:11–13. [35] Sun M, Lu Z. Speckle suppression with a rotating light pipe. Opt Eng 2010;49:024202. [36] Erkmen BI, Shapiro JH. Signal-to-noise ratio of Gaussian-state ghost imaging. Phys Rev A 2009;79:023833. [37] Hardy ND, Shapiro JH. Ghost imaging in reflection: resolution, contrast, and signal– to-noise ratio. Proc SPIE 2010;7815:78150L.
22
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Collective noise model for focal plane modulated single-pixel imaging Ming-Jie Sun a,∗, Zi-Hao Xu a, Ling-An Wu b a b
School of Instrementation Science and Opto-electronic Engineering, Beihang University, Beijing, 100191, China Quantum Engineering Research Center, Beijing Institute of Aerospace Control Devices, China Aerospace, Beijing, 100094, China
a r t i c l e
i n f o
Keywords: Single-pixel imaging Collective noise model Computational imaging
a b s t r a c t Single-pixel imaging, also known as computational ghost imaging, provides an alternative method to perform imaging in various applications which are difficult for conventional cameras with pixelated detectors, such as multi-wavelength imaging, three-dimensional imaging, and imaging through turbulence. In recent years, many improvements have successfully increased the signal-to-noise ratio of single-pixel imaging systems, showing promise for the engineering feasibility of this technique. However, many of these improvements are based on empirical findings. In this work we perform an investigation of the noise from each system component that affects the quality of the reconstructed image in a single-pixel imaging system based on focal plane modulation. A collective noise model is built to describe the resultant influence of these different noise sources, and numerical simulations are performed to quantify the effect. Experiments have been conducted to verify the model, and the results agree well with the simulations. This work provides a simple yet accurate method for evaluating the performance of a single-pixel imaging system, without having to carry out actual experimental tests. © 2017 Elsevier Ltd. All rights reserved.
1. Introduction Since a major part of the information processed in our daily life is in graphic form, imaging technology is one of the most important tools in the development of society. Conventional digital imaging uses a lens to map the spatial information from a scene onto the focal plane, where a pixelated array records the light intensity. Over the past two decades, an alternative way to perform imaging known as ghost imaging [1–3] or single-pixel imaging [4–6] has aroused much interest in the scientific community. Single-pixel imaging (SPI) reconstructs an image by measuring the correlations between the scene and a series of masks. It enables various applications, such as multi-wavelength imaging [7,8], three-dimensional imaging [9,10] and imaging through turbulence under certain circumstances [11–13], all of which pose difficulties for conventional imaging. Besides environmental effects, internal noise of the imaging system is an important factor that determines the image quality for both conventional and SPI approaches. In conventional digital imaging systems, the noise originates from the electronic readout of the pixel array, which has a direct additive effect on the corresponding image. In SPI, however, several different components contribute to the noise, such as the light source and the bucket detector, and their effects on image quality are less straightforward due to the image reconstruction mechanism. As a matter of fact, the signal-to-noise ratio (SNR) issue is one of the major obstacles facing widespread application of SPI. ∗
Corresponding author. E-mail address: [email protected] (M.-J. Sun).
http://dx.doi.org/10.1016/j.optlaseng.2017.07.005 Received 19 April 2017; Received in revised form 5 July 2017; Accepted 7 July 2017 0143-8166/© 2017 Elsevier Ltd. All rights reserved.
Many promising schemes have been proposed to improve the SNR in the last decade. Computational ghost imaging [14] uses a spatial light modulator (SLM) to replace the reference arm in the original secondorder intensity correlation imaging systems, simplifying the setup and improving the performance. Differential ghost imaging [15] normalizes the total intensity of each measurement, minimizing the effect of intensity instabilities of the source. Compressive sensing [16,17] takes advantage of sparsity in the scene and improves the reconstructed image by minimizing a certain measure of the sparsity. High-order ghost imaging [18,19] exploits higher-order correlation to increase the SNR. Positive–negative ghost imaging [20,21] utilizes the symmetry of noise in positive and negative fluctuations, partially canceling the noise in the system. Single-pixel imaging based on different orthonormal bases [5–7,22,23] maintains the orthogonality within a series of sampling masks, leading to theoretically perfect reconstruction and a much smaller number of measurements. Digital microscanning [24] applies a super-resolution technique [25, 26] to achieve a better SNR and higher resolution. However, many of these improvements are obtained after the measurements have been performed, and the noise evaluations were based on the specific system configurations, which might pose certain difficulties for starting researchers not familiar with the area to understand the roles of the different noise sources and to decide what devices to choose in order to set up their own SPI systems with the desired performance.
M.-J. Sun et al.
Optics and Lasers in Engineering 100 (2018) 18–22
by driving current fluctuations. The characterization of laser noise has been widely investigated and is quite complicated if all aspects were considered [27,28]. For our purposes, we measured the output of a typical continuous wave diode laser with a low noise PIN detector (Thorlabs DET10A), and from 100,000 measured points determined that the random fluctuations exhibit an approximate Gaussian dependence on current. Modeling the light intensity as 𝐼0 + ▵ 𝐼𝑖 , from Eqs. (1) and (2) the reconstructed image OR, Light can be expressed as ) ( 𝑁 ∑ ∑ (( ∑ ) ) 𝑶𝑅,𝐿𝑖𝑔ℎ𝑡 = 𝑴𝑖 ⋅ (3) 𝐼0 + ▵ 𝐼𝑖 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 .
Fig. 1. Schematic diagram of an SPI system based on focal plane modulation.
▵ 𝑶𝑅,𝐿𝑖𝑔ℎ𝑡 =
𝑚
𝑖=1
𝑛
Subtracting Eq. (2) from Eq. (5), the noise introduced into the image is ▵ 𝑶𝑅,𝑆𝐿𝑀 =
𝑁 ∑
(
𝑖=1
𝐼0 ⋅ 𝑴 𝑖 ⋅
∑∑( 𝑚
𝑛
▵ 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛
) )
.
(6)
If the error caused by each pixel is due to imperfect manufacturing of the device, then ▵Mi can be viewed as a constant ▵M during N measurements. In this case, ▵OR, SLM adds only a constant to the reconstructed images, which can be reduced by normalization and so is inconsequential. If the error arises from the instability of the device voltage, then ▵M follows a similar Gaussian distribution to that of the device current fluctuations. 2.3. Detector noise There are two kinds of noise in a bucket detector. One is dark current ▵Di , which exists in the readouts of the detector when there is no incident light and can be considered to have a Gaussian distribution with a mean value of D [29]. The other type of noise ▵Si is induced by incident light, and its level is proportional to the detected signal Si . The measured signal is 𝑆𝑖 + ▵ 𝑆𝑖 + ▵ 𝐷𝑖 when light is present, and the reconstructed image OR, Det can be expressed as ( ) 𝑁 ∑ ∑∑( ) 𝐼0 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 + ▵ 𝑆𝑖 + ▵ 𝐷𝑖 . 𝑶𝑅,𝐷𝑒𝑡 = (𝑴 𝑖 ⋅ (7)
𝑛
𝑚
(4)
𝑖=1
The SLM noise arises from aberrations in surface curvature, comprising low order Zernike polynomials if the SLM is liquid crystal based, or from fluctuations in the tilt angles of the micromirrors if it is a digital micromirror device (DMD). In both cases, the masks generated by the SLM are 𝑴 𝑖 + ▵ 𝑴 𝑖 , where ▵Mi is an error function of i, m and n. The reconstructed image OR, SLM can be expressed as ( ) 𝑁 ∑∑( ( ∑ ) ) 𝑶𝑅,𝑆𝐿𝑀 = 𝑴𝑖 ⋅ (5) 𝐼0 𝑀𝑖,𝑚𝑛 + ▵ 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 .
where m and n indicate the horizontal and vertical coordinates of the mask, respectively. If the masks Mi are orthonormal such that their transpose is the same as their inverse, by performing N independent SLM mask exposures, the image OR can be perfectly reconstructed as [24] ) ( 𝑁 𝑁 ∑ ∑∑( ( ) ∑ ) 𝑴 𝑖 ⋅ 𝑆𝑖 = 𝐼0 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 . 𝑶𝑅 = 𝑴𝑖 ⋅ (2) 𝑖=1
𝑁 ∑ ( ) ▵ 𝐼𝑖 ⋅ 𝑴 𝑖 ⋅ 𝑆𝑖 ∕𝐼0 .
2.2. Spatial light modulator noise
In an SPI system based on focal plane modulation, as illustrated in Fig. 1, light of intensity I0 emitted from a source illuminates a scene O. The spatial information of the scene is imaged by an imaging lens onto its focal plane, where an SLM generates intensity modulation masks Mi , for up to i = 1, 2, …, N measurements. The transmitted or reflected light is then collected, and its total intensity signal Si measured by a bucket detector is ∑∑( ) 𝐼0 𝑀𝑖,𝑚𝑛 ⋅ 𝑂𝑚𝑛 , 𝑆𝑖 = (1)
𝑖=1
𝑛
Subtracting Eq. (2) from Eq. (3), the noise introduced into the resulting image is
2. Collective noise model
𝑚
𝑚
𝑖=1
In two SPI schemes which are essentially the same, one scheme uses structured light illumination [5, 14], and the other uses focal plane modulation [4,24]. In this work we investigate the noise transmitted from the system components to the resulting reconstructed image for an SPI setup based on focal plane modulation. Digitization electronics is a common source of noise and is a topic that has been widely discussed. The main understanding is that the higher the digitization resolution, the less noise there will be, and the improvement approaches its limit as the resolution increases. However, higher resolution means slower sampling rates and larger amount of data transfer and computing, therefore a more practical question for setting up an SPI system is which digitizer would be best for the application. For example, a high resolution digitizer would be preferable for static imaging, and a low resolution but fast sampling rate digitizer for real-time motion detection, but this issue will not be discussed here. In this work, noises from the light source, SLM and detector are analyzed separately, their overall effect on image quality is investigated, and a collective noise model is built. Numerical simulations at different noise levels are performed based on this model to visualize the effect of each noise component on the reconstructed image. Measurements on an experimental setup agree well with the numerical simulation, demonstrating that our model is effective in predicting the performance of SPI systems. Our work provides a simple yet accurate noise model for researchers who are interested in SPI to understand and evaluate the system performance of their own SPI configuration before having to actually set up the system.
𝑚
𝑖=1
𝑛
Subtracting Eq. (2) from Eq. (7), the noise introduced is
𝑛
▵ 𝑶𝑅,𝐷𝑒𝑡 =
During the imaging process, if the focusing lens has zero aberration, we only have to consider the noise due to the light source, the SLM, and the bucket detector (digitization electronics is not included in the scope of this work). The contribution from each of these components will now be analyzed below to formulate a collective noise model.
𝑁 ∑ ( ) (▵ 𝑆𝑖 + ▵ 𝐷𝑖 ⋅ 𝑴 𝑖 ).
(8)
𝑖=1
We can further separate the detector induced noise into the signalrelated noise ▵OR, Det, S and signal-unrelated noise ▵OR, Det, D , as follows:
2.1. Light source noise
▵ 𝑶𝑅,𝐷𝑒𝑡,𝑆 =
For a laser operating well above threshold we can ignore its quantum fluctuations, so the chief source of noise in its emission is caused
▵ 𝑶𝑅,𝐷𝑒𝑡,𝐷 = 19
𝑁 ∑ ( ) ▵ 𝑆𝑖 ⋅ 𝑴 𝑖 ,
(9)
𝑖=1
𝑁 ∑ ( 𝑖=1
) ▵ 𝐷𝑖 ⋅ 𝑴 𝑖 .
(10)
M.-J. Sun et al.
Optics and Lasers in Engineering 100 (2018) 18–22
2.4. Collective noise model Following the analysis above, we can express the reconstructed image OR, Noise under the overall influence of noise from the light source, SLM and detector as 𝑶𝑅,𝑁𝑜𝑖𝑠𝑒 = =
𝑁 ∑ 𝑖=1
( (𝑴 𝑖
𝑁 ∑ ( 𝑖=1
+
𝑚
𝑛
)
𝑁 ∑
𝑴 𝑖 𝑆𝑖 +
𝑁 ∑ 𝑖=1
+
∑∑ ( )( ) (( 𝐼0 + ▵ 𝐼𝑖 𝑀𝑖,𝑚𝑛 + ▵ 𝑀𝑚𝑛 ⋅ 𝑂𝑚𝑛 )+ ▵ 𝑆𝑖 + ▵ 𝐷𝑖 𝑁 ∑ 𝑖=1
𝑴 𝑖 𝐼0
∑∑( ▵ 𝑀𝑖, 𝑚
𝑛
𝑚𝑛
⋅ 𝑂𝑚𝑛
)
∑∑( ) 𝑴 𝑖 ▵ 𝐼𝑖 ▵ 𝑀𝑚𝑛 𝑂𝑚𝑛 𝑚
𝑁 ∑ ( 𝑖=1
𝑖=1
(▵ 𝐼 𝑖 𝑴 𝑖 𝑆 𝑖 ∕𝐼 0 ) +
)
𝑛
)
▵ 𝑆𝑖 ⋅ 𝑴 𝑖 +
𝑁 ∑ ( 𝑖=1
▵ 𝐷𝑖 ⋅ 𝑴 𝑖
)
= 𝑶𝑅 + ▵ 𝑶𝑅,𝐿𝑖𝑔ℎ𝑡 + ▵ 𝑶𝑅,𝑆𝐿𝑀 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝑆 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝐷 𝑁
+
∑ 𝑖=1
𝑴 𝑖 ▵ 𝐼𝑖 ⋅
∑∑( 𝑚
𝑛
) ▵ 𝑀𝑚𝑛 𝑂𝑚𝑛 .
Fig. 2. Numerical simulation of the reconstructed image RMSE as a function of the RMS of each individual noise source and the collective noise. Each point is the ensemble average over 100 simulations.
(11)
If all the non-noise variables are normalized in Eq. (11), then all the noise contributions will be fractional. The last term of Eq. (11) contains two noise terms (▵Ii and ▵Mmn ), the product of which is one order smaller than the other terms so it can be neglected, and the reconstructed image and its collective noise become, respectively, 𝑶𝑅,Noise = 𝑶𝑅 + ▵ 𝑶𝑅,Light + ▵ 𝑶𝑅,𝑆𝐿𝑀 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝑆 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝐷
•
We can therefore separate the noises in an SPI system into two kinds, signal related and signal unrelated noise. The former has a smaller and more linear effect on the quality of reconstructed images compared to the latter. Keeping the signal unrelated noise as low as possible is crucial for a high performance SPI system, preferably below 0.2 RMS. In other words, a high precision SLM and a low dark current detector are more important to the performance of an SPI system than a stable illumination source. Besides having a high modulation rate, one of the reasons that a DMD is a popular choice for many recent SPI works is that it is in general much flatter than liquid crystal based SLMs, and the variation in its mirror tilt and reflection coefficient is minimal. We further investigated the effect of the collective noise by using Eq. (13) to increase the RMS values of all the noise sources simultaneously through a 0.05 increment, as shown by the light blue curve in Fig. 2. As we can see, the collective RMSE is not equivalent to the sum of each independent RMSE, because the noise transmission is not a linear process in SPI.
(12)
and ▵ 𝑶𝑅,Noise =▵ 𝑶𝑅,Light + ▵ 𝑶𝑅,𝑆𝐿𝑀 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝑆 + ▵ 𝑶𝑅,𝐷𝑒𝑡,𝐷 .
(13)
3. Results and discussion 3.1. Numerical simulation To visualize the effect of the total noise on a focal plane modulation based SPI system, numerical simulations were carried out in Labview using the collective noise model, i.e., Eqs. (12) and (13). The SLM sampling masks were 64 × 64 pixels in size and generated from a 4096 × 4096 Hadamard matrix [30,31], which is a typical choice for single-pixel imaging [7,8,10,23,24] and provides perfect reconstruction when full sampling is performed [22,32]. Non-orthogonal random sampling can also be used, but the noise it introduces is not related to any device and can be reduced statistically by increasing the sampling number, so its analysis will not be included here. To compare the influence of different noise sources on the reconstructed image, all physical entities are normalized, i. e., we assume 𝐼0 = 1, the matirx elements of Mi take the values of ‘−0.5′ and ‘0.5′, O has a reflectivity between 0 and 1, and the maximum intensity at the detector is normalized to 1. The noise level is represented by its rootmean-square (RMS) value. The influence of the noise is represented by the root-mean-square error (RMSE) between O and OR, X , i.e., the RMS of ▵OR, X , where X refers to a certain kind of noise. We first investigated the effect of each noise source independently by increasing its RMS value by a 0.05 increment for each reconstruction. The red, indigo, green and orange curves in Fig. 2 show the RMSE as a function of the light source noise ▵Ii , SLM noise ▵Mi , signal related detector noise ▵Si , and detector dark current ▵Di , respectively. Interestingly, the effects of ▵Ii and ▵Si almost overlap, while the effects of ▵Mi and ▵Di are very similar. The reasons behind this phenomenon are: •
•
mean due to the random nature of ▵Mi , and ▵OR, SLM is also unrelated to Si , so ▵OR, Det, D and ▵OR, SLM behave similarly. Because all physical entities in the simulation are normalized to 1, the numerical effects of noises with the same characteristics can be directly compared.
3.2. Experiment To validate the accuracy of our collective model, experiments following the procedure in Fig. 1 were performed. The beam from a 635 nm laser (PicoQuant PDL 800-B, laser head LDH-P-C-635M, 4 mW) was expanded by a diffuser to illuminate a white circle printed on a square piece of black paper. This object was then imaged by a camera lens (Nikon AF Nikkor EFL 50 mm, f/1.8D) onto the DMD (Texas Instruments Discovery 4100, 1024 × 768 pixels, wavelength 350 − 2500 nm) which sequentially generated 64 × 64 pixel masks formed from a 4096 × 4096 Hadamard matrix. The image was modulated by the masks and the total intensity of the reflected light was detected by a bucket detector (Thorlabs PDA100A-EC Si amplified detector with an ACL25416U-A lens). A high dynamic range analogue-to-digital converter, (PicoScope 6404D, sampling rate set to 2.5 Giga Samples/s, ADC resolution set to 12 bits, both to ensure that the digitization influence is minimized and negligible) that was synchronized with the DMD, acquired and transferred the intensity data to a laptop computer to perform image reconstruction using Eq. (11). The purpose of using a white circle on a black background as object was to accurately calculate the RMSE between the experimental result and the original object, since the edges of the black square could be precisely matched with the image frame.
▵OR, Laser is related to the measured signal Si , while ▵OR, Det, S is related to ▵Si which in turn is proportional to Si , therefore ▵OR, SLM and ▵OR, Det, S have the same trend. ▵OR, Det, D is related to ▵Di , which has a constant mean D and is not related to the signal Si . Since ▵OR, SLM depends on the sum of the terms in ▵Mi · O, which is a Gaussian distribution with a constant 20
M.-J. Sun et al.
Optics and Lasers in Engineering 100 (2018) 18–22
4. Conclusion Though not every noise factor related to SPI has been included, we believe that the collective noise model proposed here covers the most significant noise sources in the system. For laser illuminated conventional imaging, speckle can severely affect image quality [33]. However, because of the temporal changing of the patterns or modulation of the SLM and the cumulative measurements of SPI, laser speckle can be dramatically suppressed [34,35]. Experimental results demonstrate that laser speckle does not have a severe effect on SPI image quality. To summarize, we have investigated the individual effects of the noise from each system component on the resulting image in focal plane modulation based SPI. Through numerical simulation, we find that the noise can be catalogued into two kinds, signal related and signal unrelated noise. Their combined effect can be described by a collective noise model, which we have also simulated. Experimental results show good agreement with our simulation. By investigating the effects of different noises, we have verified that signal unrelated noise has a worse effect on image quality and therefore should be given greater attention when setting up an SPI system. Measures to decrease the signal unrelated noise include using a high precision SLM and a low dark current detector. Although there has been some rigorous analysis on the influence of different noise sources on image reconstruction [36,37], the model we present here provides a simple yet accurate method to evaluate performance without having to actually carry out experimental tests. This should be of interest to those who are interested in setting up an SPI system as efficiently as possible. Fig. 3. (a) Numerically simulated RMSE of the reconstructed image (light blue circular dots and curve) and experimental values (red sqaure dots) as a function of the RMS of collective system noise. Each experimental dot is the ensemble average over the RMSE of 20 reconstructed images. (b) Reconstructed images of a white circle printed on black paper as an object; the corresponding RMSE and RMS noise value (NRMS) are given below each image. (c) Reconstructed images of an actual doll, with the same NMRS as the circular object above.
Funding information This work was supported by the National Natural Science Foundation of China [grant number 61675016, 61307021], and Beijing Natural Science Foundation [grant number 4172039]. References
For ease of comparison, the light blue curve of Fig. 2 showing the numerically simulated RMSE is replicated in Fig. 3(a). The first red dot, representing the experimental RMSE, was calculated and plotted from the highest quality image (first item of Fig. 3(b)), i.e. the experimental system is calibrated against its minimum noise level, which is ∼0.027 of the RMS noise value. Starting from this point, we are able to introduce artificial noises into the system as follows. •
•
•
[1] Pittman TB, Shih YH, Strekalov DV, Sergienko AV. Optical imaging by means of two-photon quantum entanglement. Phys Rev A 1995;52:R3429. [2] Bennink RS, Bentley SJ, Boyd RW. ’Two-photon’ coincidence imaging with a classical source. Phys Rev Lett 2002;89:113601. [3] Yang C, Wang C, Guan J, Zhang C, Guo S, Gong W, et al. Scalar-matrix-structured ghost imaging. Photonics Res 2016;4:281–5. [4] Duarte MF, Davenport MA, Takhar D, Laska JN, Sun T, Kelly K, et al. Single-pixel imaging via compressive sampling. IEEE Signal Process Mag 2008;25:83–91. [5] Bromberg Y, Katz O, Silberberg S. Ghost imaging with a single detector. Phys Rev A 2009;79:053840. [6] Zhang Z, Ma X, Zhong J. Single-pixel imaging by means of Fourier spectrum acquisition. Nat Commun 2015;6:6225–30. [7] Radwell N, Mitchell KJ, Gibson GM, Edgar MP, Bowman R, Padgett MJ. Single-pixel infrared and visible microscope. Optica 2014;1:285–9. [8] Edgar MP, Gibson GM, Bowman RW, Sun B, Radwell N, Mitchell KJ, et al. Simultaneous real-time visible and infrared video with single-pixel detectors. Sci Reports 2015;5:10669. [9] Sun B, Edgar MP, Bowman RW, Vittert LE, Welsh SS, Bowman A, et al. 3D computational imaging with single-pixel detectors. Science 2013;340:844–7. [10] Sun M-J, Edgar MP, Gibson GM, Sun B, Radwell N, Padgett MJ. Single-pixel three-dimensional imaging with a time-based depth resolution. Nat Commun 2016;7:12010. [11] Meyers RE, Deacon KS, Shih Y. Turbulence-free ghost imaging. Appl Phys Lett 2011;98:111115. [12] Zhao C, Gong W, Chen M, Li E, Wang H, Xu W, et al. Ghost imaging lidar via sparsity constraints. Appl Phys Lett 2012;101:141123. [13] Shi D, Fan C, Zhang P, Shen H, Zhang J, Qiao C, et al. Two-wavelength ghost imaging through atmospheric turbulence. Opt Express 2013;21:2050–64. [14] Shapiro JH. Computational ghost imaging. Phys Rev A 2008;78:061802. [15] Ferri F, Magatii D, Lugiato LA, Gatti A. Differential ghost imaging. Phys Rev A 2010;104:253603. [16] Candes EJ. Compressive sampling. In: Proceedings of the international congress of mathematicians. International Mathematical Union; 2006. p. 1433–52. [17] Donoho DL. Compressed sensing. IEEE Trans Inform Theory 2006;52:1289–306. [18] Chen X-H, Agafonov IN, Luo K-H, Liu Q, Xian R, Chekhova MV, et al. High-visibility, high-order lensless ghost imaging with thermal light. Opt Lett 2010;35:1166–8. [19] Cao B, Zhang C. Third-order lensless ghost diffraction with classical fully incoherent light. Opt Lett 2010;35:2091–3. [20] Sun M-J, Li M-F, Wu L-A. Nonlocal imaging of a reflective object using positive and negative correlations. Appl Opt 2015;54:7494–9.
The laser power was varied by changing its driving current to simulate severe illumination fluctuations. Greyscale masks were generated on the DMD using its 8 bit greyscale mode to simulate micromirror jitter. Since the noise level of the detector we used in our experiment was not adjustable, the noise of the detector was first measured with no incident light; this was found to exhibit a fair Gaussian distribution. Then, artificial random noises of different levels were generated to mimic detectors with different noise levels. This simulated either the signal related ▵OR, Det, S or the signal unrelated ▵OR, Det, D , to imitate different noise types of the detector.
The equivalent increment in RMS noise was 0.05 for each individual noise configuration. Fig. 3(a) shows that the experimental results are in good agreement with the numerical simulation, demonstrating that our collective noise model is effective in predicting the performance of focal plane modulation based SPI systems. The reconstructed image was quite noisy when the RMS reached 0.377 (the fourth image of Fig. 3(b)) and became beyond recognition after the noise reached 0.527, hence no further experiment was performed. In a different experiment, a plastic doll was put on the table as a more realistic object. Similar quality images were obtained (Fig. 3(c)), with the same noise levels, showing that the SPI system and collective noise model work just as well with actual objects. 21
M.-J. Sun et al.
Optics and Lasers in Engineering 100 (2018) 18–22
[21] Sun M-J, He X-D, Li M-F, Wu L-A. Thermal light subwavelength diffraction using positive and negative correlations. Chin Opt Lett 2016;14:040301. [22] Sankaranarayanan AC, Studer C, Baraniuk RG. Cs-muvi: Video compressive sensing for spatial multiplexing cameras. In: Proceedings of IEEE international conference on computational photography. IEEE; 2012. p. 1–10. [23] Le W, Zhao S. Fast reconstructed and high-quality ghost imaging with fast Walsh–Hadamard transform. Photonics Res 2016;4:240–4. [24] Sun M-J, Edgar MP, Phillips DB, Gibson GM, Padgett MJ. Improving the signal– to-noise ratio of single-pixel imaging using digital microscanning. Opt Express 2016;24:10476–85. [25] Nasrollahi K, Moeslund TB. Super-resolution: a comprehensive survey. Mach Vision Appl 2014;25:1423–68. [26] Sun M, Yu K. A sur-pixel scan method for super-resolution reconstruction. Optik 2013;124:6905–9. [27] Petermann K. Laser diode modulation and noise. Berlin: Springer Science & Business Media; 2012. p. 152–208. Vol. 3. [28] Ralph TC, Huntington EH, Harb CC, Buchler BC, Lam PK, McClelland DE, et al. Understanding and controlling laser intensity noise. Optical and quantum electronics 1999;31:583–98.
[29] Srinivasan M, Vilnrotter V. Symbol-error probabilities for pulse-position modulation signaling with an avalanche photodiode receiver and Gaussian thermal noise. In: The telecommunications and mission operations progress report 42-134; 1998. p. 1–11. April–June 1998. [30] Pratt WK, Kane J, Andrews HC. Hadamard transform image coding. In: Proceedings of the IEEE, 57; 1969. p. 58–68. [31] Sloane NJ, Harwit M. Masks for Hadamard transform optics, and weighing designs. Appl Opt 1976;15:107–14. [32] Schechner YY, Nayar SK, Belhumeur PN. Multiplexing for optimal lighting. IEEE Trans Pattern Anal Mach Intell 2007;29:1339–54. [33] Goodman JW. Laser speckle and related phenomena. Springer-Verlag; 1984. [34] Trisnadi JI. Hadamard speckle contrast reduction. Opt Lett 2004;29:11–13. [35] Sun M, Lu Z. Speckle suppression with a rotating light pipe. Opt Eng 2010;49:024202. [36] Erkmen BI, Shapiro JH. Signal-to-noise ratio of Gaussian-state ghost imaging. Phys Rev A 2009;79:023833. [37] Hardy ND, Shapiro JH. Ghost imaging in reflection: resolution, contrast, and signal– to-noise ratio. Proc SPIE 2010;7815:78150L.
22