- Email: [email protected]
Measurement dimensions compressed spectral imaging with a single point detector
Optics Communications 365 (2016) 173–179
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Invited Paper
Measurement dimensions compressed spectral imaging with a single point detector Xue-Feng Liu a,n, Wen-Kai Yu a,b, Xu-Ri Yao a,b, Bin Dai a,b, Long-Zhen Li a,b, Chao Wang a, Guang-Jie Zhai a a Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China b University of Chinese Academy of Sciences, Beijing 100190, China
art ic l e i nf o
a b s t r a c t
Article history: Received 20 October 2015 Received in revised form 1 December 2015 Accepted 11 December 2015
An experimental demonstration of spectral imaging with measurement dimensions compressed has been performed. With the method of dual compressed sensing (CS) we derive, the spectral image of a colored object can be obtained with only a single point detector, and sub-sampling is achieved in both spatial and spectral domains. The performances of dual CS spectral imaging are analyzed, including the effects of dual modulation numbers and measurement noise on the imaging quality. Our scheme provides a stable, high-flux measurement approach of spectral imaging. & 2015 Elsevier B.V. All rights reserved.
Keywords: Spectral imaging Compressed sensing Imaging quality Sub-sampling
1. Introduction Spectral imaging, which can capture both the spatial information and spectral information of an object, is of great importance in physics and biology as it can give rich evidences in the diagnoses of matter component and structure. For a spectral image, there are three dimensions of information to be measured, twodimensional spatial information and one-dimensional spectral information. It is obviously not possible to obtain three-dimensional information simultaneously with current detectors, which have two-dimensional resolution at most. As an alternative approach, the detection of spatial image or spectrum should be performed by scanning, which will reduce the system stability and consistency because of the mechanical movement and non-simultaneous measurements of different spatial and spectral components. In addition, the three-dimensional information measurement generates large amount of data, causing difficulties in data storage and transmission. In recent years, a sampling theory called compressed sensing (CS) was derived and has attracted wide interests in many fields [1–3]. With this theory, we can measure signals with sampling number far less than Nyquist–Shannon theorem demands [4–7]. n
Corresponding author. E-mail addresses: [email protected] (X.-F. Liu), [email protected] (G.-J. Zhai). http://dx.doi.org/10.1016/j.optcom.2015.12.020 0030-4018/& 2015 Elsevier B.V. All rights reserved.
Based on CS theory, Baraniuk et al. proposed an imaging approach named single pixel camera, in which a single pixel detector is used to image a two-dimensional object with measurement number less than image pixels [8–10]. Although in single pixel camera the sampling time is sacrificed as modulation and detection to the object should be performed repeatedly, the use of CS and single pixel detector provides many benefits to practical applications. In general array detector with pixels having the same performances with high quality point detector is difficult to be manufactured. For example, in many applications such as laser radar and fluorescence lifetime imaging high time-resolved single photon detection is needed, which is difficult or very expensive to achieve array detection, while CS imaging can provide an easily implemented solution with a Geiger-mode avalanche photodiode (APD) or hybrid photomultiplier tube (PMT) [1]. The sub-sampling ability can reduce the amount of stored data and thus memory requirements, which is valuable in the aerospace remote sensing and other fields having limitations in data storage and transmission [11]. Another important advantage is that the light from object is concentratedly measured by the point detector instead of being assigned on many pixels, improving the measurement signal-to-noise ratio (SNR) [12]. Especially for some detectors which can respond only if the light intensity reaches a threshold, the high-flux measurement in CS imaging is able to increase the imaging sensitivity [13]. As single pixel camera shows, CS theory permits the detection dimension reduction, which is very helpful in spectral imaging. Many applications of CS theory in spectral imaging have been
174
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
reported [14,15]. A straightforward method is replacing the point detector in single pixel camera with a spectrometer [16,17]. Arce et al. proposed coded aperture snapshot spectral imaging which obtains spectral images with an array detector needless of scanning based on CS theory [18–21]. In the above systems, only measurement in spatial domain is compressed and thus the compression ratio does not reach the optimum. To make the spatial and spectral measurement be both modulated an approach of dual-coded compressive spectral imaging is derived, while the final signal should be detected by an array detector to realize single exposure imaging [22]. In Ref. [23], spectral imaging with separable modulations and sub-sampling in both the spatial and spectral domains is proposed and demonstrated by simulations, in which the spectral modulation is performed in parallel and signal measured with a linear detector. To realize high SNR spectral imaging to an object in weak illumination, high-flux measurement with a high performance point detector is meaningful. In this paper, it is experimentally demonstrated that by implementing dual compressed sensing spectral imaging can be realized with a position-fixed single pixel detector. The detection dimensions and sampling numbers in both the spatial and spectral domains are compressed to provide higher ability in the sub-sampling. The light signal is concentratedly detected, avoiding the intensity assigned in both spatial and spectral dimensions and achieving much higher measurement SNR compared with array or linear detection. This paper is organized as follows. The theory and experimental results are presented in Sections 2 and 3, respectively. In Section 4 the performances and development prospects of the system are discussed. Finally the paper is concluded in Section 5.
2. Theory of dual compressed sensing Compressed sensing is a mathematical theory in which sampling can be performed with fewer measurements than the number of signal elements [4–7]. For a signal t having one dimensional information, CS makes linear measurement in this dimension:
I = A(r1)t (r1),
(1)
where A is the measurement matrix, I is the measurement result and r1 denotes the dimension of signal and measurement matrix. If the signal has two dimensions, it is straightforward to perform the linear measurement in the two dimensions as
I = A(r1, r2)t (r1, r2),
⎧ 1 2 I − A2 (r2)S(r2) + τ1 S(r2) ⎪ ⎪ S(r2) = min 2 1 S 2 ⎨ 1 2 ⎪ t (r , r ) = min S(r2) − A1(r1)t (r1, r2) + τ2 t (r1, r2) ⎪ 2 ⎩ 1 2 t 2
(5)
1
where ⋯ r stands for lr norm, defined as ( x r )r = ∑i = 1 x i , and τ1, τ2 are constant parameters weighting the relative strength of the two terms. As the measurements and reconstructions are performed in the two dimensions successively using CS theory twice, we call this sampling process as dual CS. For random measurement matrix the signal can be reconstructed accurately through m ≥ Ck log(n/k ) measurements, in which n is the number of elements, k is the sparsity of signal, and C is a constant coefficient [6,7], while for separate matrices the total measurement number should be increased by a factor of 1 log10(n1n2) in the dual CS sampling [23,24]. 2 N
r
3. Experiment and results As spectral imaging has information in two dimensions to be obtained, it can be realized with a single point detector by dual CS. The experimental apparatus is given in Fig. 1. Light from a halogen lamp illuminates the object, which is then imaged onto the Digital micromirror device (DMD) by a lens L1. The DMD (Texas Instruments DLP7000) consists of 1024 768 micromirrors each of size 13.68 μm × 13.68 μm , which can reflect the light to two directions individually. Controlled by the random matrix loaded into DMD1, it can reflect the modulated image to the direction of lens L2. If we measure the total intensity of light collected by lens L2, the grey image of object can be reconstructed by CS algorithm with the modulated matrix on DMD1. To acquire the spectral image, the spectrum information of the whole light must be measured. Therefore, we focus the light by lens L2 to a pinhole, which is then collimated by the lens L3 and illuminates on a blazed grating. The spectral line will emerge on the focal plane of L4, on which another DMD2 locates for utilizing CS for the second time. Similar to DMD1, the DMD2 randomly reflects different spectrum components to the direction of lens L5, which collects the light to a PMT. Ignoring the loss on the optical elements, 25% total intensity of whole spatial and spectral components from the object is detected by the PMT, which will then output the total light intensity of both spatial and spectral modulated images with 12-bit analog to digital conversion.
(2)
where t has information in dimensions of r1 and r2 with element numbers of n1 and n2, respectively. However, in practice it is usually difficult to make measurements in two dimensions simultaneously, and the amount of calculation in the signal reconstruction may be intolerable as the whole signal elements have a large number of n1n2. Alternatively, we can perform compressive measurements and reconstructions in the two dimensions successively,
⎧ S(r2) = A1(r1)t (r1, r2) ⎨ ⎩ I = A2 (r2)S(r2) ⎪
(3)
where S (r2) is the intermediate result of measurement in r1 dimension. This process can be expressed as the separate or Kronecker CS [24,25],
I = A(r1, r2)t (r1, r2) = A2 (r2) ⊗ A1(r1)t (r1, r2)
(4)
where ⊗ denotes the Kronecker product. The reconstruction of t (r1, r2) is also performed in the two dimensions successively,
Fig. 1. Experimental setup of spectral imaging with dual compressed sensing. Obj, object. G, blazed grating. L1–L5, lens. DMD1, DMD2, digital micromirror device. PMT, photomultiplier tube. The DMD can reflect light to two directions. Left bottom, the colored object to be imaged. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
175
magnification factor, the system spatial resolution on the object will be 80 μm . We use a grating with blaze wavelength of 500 nm and line number Mg ¼1800 grooves/mm as spectral dispersive element, the spectrum line ranging from 520 nm to 620 nm dispersed by which occupies 1 1024 pixels of DMD2, and the spectrum resolution is about
Δλ =
Fig. 2. Working time sequences of the DMDs and PMT.
The working time sequences of the DMDs and PMT are shown in Fig. 2. m1 random matrixes A(x, y ) are loaded into DMD1 sequentially, modulating the spatial information of the object. When DMD1 states on each modulation the corresponding spectrum should be obtained, so m2 random matrixes B(λ ) are loaded into DMD2 sequentially to modulate the spectral information of the object. During each modulation of DMD2, the PMT detects the total intensity of object under the spatial modulation A(x, y ) and spectral modulation B(λ ). In the experiment the DMDs and PMT all work in external trigger modes, and the working sequences are controlled by a circuit providing synchronous signals. The spectral image of the object can be expressed by T (λ )(x, y ), in which λ denotes the wavelength and (x, y ) denotes the spatial coordinate. The modulation process of DMD1 can be described mathematically as
∑ A(x, y)T (λ)(x, y) = S(λ), x, y
(6)
where S (λ ) is the spectrum all over spatial areas of image which are reflected to the collecting direction, which is also the image on DMD2. The modulation process of DMD2 can be described as
∑ B(λ)S(λ) = I, λ
(7)
where I is the total intensity detected by the PMT. Based on CS algorithm, the spectrum S (λ ) can be recovered with B(λ ) and I, and then the spectral image T (λ )(x, y ) can be reconstructed with S (λ ) and A(x, y ). We use TVAL3 algorithm to recover the spectrum and image, which gives a sparsest solution in the gradient basis [26]. For the sparse basis related to the frequency transform such as wavelet or Fourier basis, the neglect of small high-frequency coefficients in the solving process may decrease the resolution, while with gradient basis it will not be influenced by the CS reconstruction. In our experiment, the object to be imaged is a film with total size of 3.2 mm 3.2 mm printed a green “C” and red “S” on it, which is shown in the left bottom of Fig. 1. The object is imaged onto 128 128 pixels of DMD1 with magnification factor of 0.55, while for increasing the reconstruction speed we combine 2 2 pixels as one pixel in the random modulated matrix, so the valid resolution of the imaging is 64 64. To ensure the overlapping of image and mirrors, in the calibration of optical setup a camera (AVT Manta G-145B) with lens is used to observe the mirror plane. Light from DMD1 is focused by a lens L2 with focal length of f2 ¼50.8 mm to the pinhole with diameter of d ¼1 mm, which restricts the spatial resolution on the DMD1 to be
Δx = 1.22
λf2 d
= 35 μm
(8)
for λ = 570 nm . As the size of DMD1 modulation pixel is 27.36 μm , the final spatial resolution on DMD1 plane is the convolution of the two blurring effects, which is 44.4 μm . Considering the
d cos θi = 2.4 nm Mgf3
(9)
where θi = 50° is the incident angle on the grating, and the focal length of collimation lens L3 is f3 = 150 mm . This value in Eq. (9) is much larger than the spectral range on one pixel of DMD2 which is about 0.1 nm, and therefore gives the spectral resolution of the system. The calibration of spectrum is performed with a commercial spectrometer (Avantes, ULS3648) collecting light from DMD2. With DMD2 set to work in scanning mode, the spectrometer can give the spectrum range of each column in DMD2. As the spectrum is a onedimensional quantity, DMD2 modulates only in the 1 1024 pixels by setting the other region keep reflecting light to the uncollected direction. In the spectral imaging of the object, the modulation numbers of DMD1 and DMD2 are m1 = 1600 and m2 = 400, respectively. Therefore, the measurements of both spatial and spectral information achieve sub-sampling, and the total compression ratio is (1600 × 400) /(4096 × 1024) = 15.3% . The working frequencies of DMD1 and DMD2 are 1 Hz and 450 Hz, respectively, so the spectral imaging costs 1600 s to take the entire modulation sequence. The imaging results are shown in Fig. 3. Based on TVAL3 algorithm, during each modulation A(x, y ) on DMD1 we can obtain a spectrum line according to the m2 = 400 modulations on DMD2 and correspondent intensities I detected by the PMT, which is shown as a row in Fig. 3(a). The average spectrum is also plotted, the two peaks around 540 nm and 600 nm are from “C” and “S”, respectively. Corresponding to spatial modulations on DMD1, m1 = 1600 spectrum lines are recovered successively. For clear vision, we only show the first 200 spectrum lines in Fig. 3(a). Then each column of Fig. 3(a) indicates the intensity fluctuation of light from spatial modulated image with the same wavelength. Combined the spatial modulations on DMD1 and intensity fluctuation of each wavelength, we can reconstruct images of different wavelengths with TVAL3 algorithm for the second time. In Fig. 3(b1) and (b2), the intensity fluctuations of light with λ = 530 nm and λ ¼610 nm are shown, which is transmitted from the green “C” and red “S”, respectively. As the spatial modulations on images of various wavelengths are different, the variation trends of the two curves in Fig. 3(b1) and (b2) are not the same, although they correspond to the same modulation matrices loaded into DMD1. The images under the two wavelengths are then reconstructed, giving the green “C” and red “S”, which are shown in Fig. 3(c1) and (c2). From the imaging results, we see the images of various wavelengths can be obtained without affected by other spectral components using our spectral imaging system with dual compressed sensing. If we integrate the intensities in each row of Fig. 3(a) over the whole spectrum and take the CS reconstruction, the spectral information will vanish and the image without spectrum can be obtained, similar to the conventional CS imaging. The intensity fluctuation of the whole spectrum is plotted in Fig. 3(b3) and the corresponding imaging result is shown in Fig. 3(c3), in which a clear image of “CS” emerges.
4. Discussions on the performances of dual CS spectral imaging 4.1. Effect of dual modulation numbers on the imaging quality The scheme of spectral imaging with dual compressed sensing
176
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
Fig. 3. Experimental results of spectral imaging. (a) Spectrum lines under different spatial modulations and the average spectrum curve. (b) Intensity fluctuations of different wavelengths, (b1) λ¼ 530 nm, (b2) λ ¼ 610 nm, and (b3) all-spectrum. (c) Imaging results of different wavelengths, (c1) λ ¼530 nm, (c2) λ¼ 610 nm, and (c3) all-spectrum. The reconstruction time is about 120 s.
can realize “3D” (two-dimensional image and one-dimensional spectrum) imaging with only a single point detector, while the imaging time will be expended. Compared with previous spectral imaging with array or linear detector, the sampling time of dual CS is increased because of the modulations on both spatial and spectral domains. Therefore, there is a tradeoff between the detection dimensions and the imaging time, which is decided by the product of numbers of spatial modulations m1 on DMD1 and spectral modulations m2 on DMD2. To study the impact of sampling numbers on the imaging quality, the numerical simulation is performed. The spectral object still includes a green “C” and red “S”, the center transmission wavelengths of which are 500 nm and 600 nm, respectively, with spectral widths of both 40 nm. The spatial imaging resolution is 64 64, and the spectrum range is 400–700 nm with resolution of 1 1024. In the simulation 10% Gaussian random noise is added in the measurement results. As the imaging qualities of different wavelengths are similar ignoring the specific spatial features, in the simulation we only reconstruct the green “C” with spectrum range of 480–520 nm.
Fig. 4 presents the reconstructions for various numbers of spatial modulations m1 and spectral modulations m2. In the three rows, the number of spectral modulations is 200, 400, and 600, respectively, and in the three columns, the number of spatial modulations is 800, 1600, and 2400, respectively. It is obvious that the increases of both m1 and m2 can improve the imaging quality, while the particular impacted aspects are different. In Fig. 4(a1– a3), the sampling number in spectral information m2 is only about 20% of the spectrum pixels, causing inaccuracy in the reconstruction of spectrum. Therefore, “S” emerges in the reconstruction results as the information of other wavelengths is contained in the intensity fluctuations of light used to recover “C”. When the spectral sampling number increases, the spectrum can be reconstructed accurately and the red “S” will not be imaged, as shown in Fig. 4(b) and (c). From the left column to the right column, the increase of spatial sampling number m1 can dramatically decrease the noise in the images, while it is unrelated with the spectrum reconstruction accuracy. In Fig. 4(a), with the increased spatial modulations, the images of “S” even tend to be more clear.
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
177
Fig. 4. Numerical simulation results of dual CS spectral imaging for various numbers of spatial modulations m1 and spectral modulations m2. The SNRs of real image “C” and ghost image “S” are marked under the imaged letters.
As the intensity fluctuation of the reconstructed spectrum has contained the information of “S”, the increase of spatial sampling can improve the quality of the ghost image which should not appear in the reconstructed result. To quantitatively evaluate the influence of modulation numbers on the imaging quality, the SNR of reconstructed real image and ghost image is calculated respectively, which is defined as
SNR =
max(U ) std( U − U0 )
(10)
where U0 is the original image, U is the reconstructed one and std denotes standard deviation. The SNR of real image is calculated with left half of original and reconstructed images containing “C” and SNR of ghost image is obtained with right half containing “S”, which are both marked in Fig. 4 to quantitatively support the above discussions. Fig. 5(a) plots the SNRs as a function of spatial modulation number m1. The spectral modulation number is only m2 = 100, so the ghost image exists. Along with spatial modulation increased, the SNRs of real and ghost images both become higher. Fig. 5(b) shows the influence of spectral modulation number m2 on the imaging quality with spatial modulation number of m1 = 2000. We can see that the quality of real image is slightly influenced by the spectral modulation number, while the SNR of ghost image decreases with m2 increased, indicating that the ghost image is disappearing. The simulations show that the spectral modulation number m2 decides the accuracy of spectrum reconstruction, and the spatial modulation number m1 affects the noise in the image. This
Fig. 5. (a) SNR of dual CS spectral imaging as a function of spatial modulation number m1 while m2 ¼ 100. (b) SNR of dual CS spectral imaging as a function of spectral modulation number m2 while m1 ¼2000. Circles and crosses are simulation results of real and ghost images, respectively.
178
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
conclusion is meaningful in the design of experimental process. In Fig. 4(a3) and (c1), the sampling numbers decided by m1 × m2 are the same, while the spectral imaging results are quite different. In Fig. 4(a3), the large number of spatial modulations makes the imaging noise be very low. However, the lack of enough spectral sampling causes ghost component in the spectral image, which is fatal in most spectral imaging applications. In Fig. 4(c1), the ghost image is eliminated by enough spectral modulations. Although there is a relatively high noise level, it can be distinguished artificially according to experience in most situations. Therefore, in spectral imaging, the proper choice of spatial modulations m1 and spectral modulations m2 is important for a finite sampling time. To obtain correct information about the spectral image, enough number of spectral sampling should be guaranteed. 4.2. Effect of measurement noise on the imaging quality We simulate the effect of dual CS spectral imaging with complex object, which is part of a circuit board with continuous spectrum from 400 nm to 700 nm and pixels of 60 100. With dual CS, we image the red (R), green (G) and blue (B) components of the object with spectra centered at 650 nm, 550 nm, and 450 nm respectively and bandwidths of 60 nm, which are shown in Fig. 6(a–c). The dual modulation numbers are m1 = 2000 and m2 = 500 respectively, realizing total compression ratio of 16.3%. By combining the three RGB components colored images are obtained in Fig. 6(d), showing no color distortion. The dependence of spectral imaging quality on the measurement noise is also studied with Gaussian random noise of different levels added in the total measurement intensity. The SNRs of RGB images are calculated, and the average value gives the SNR of the colored image. In the condition of low noise level dual CS spectral imaging can give high
quality image without color distortion, while with the increase of noise the SNR of image gradually decreases. For comparison in Fig. 6(e) we show the colored images obtained by traditional CS imaging. This can be realized in experiment by locating different spectral filters in front of the detector to perform CS imaging of RGB images and combining together. In the simulation, 2000 modulations are used to image the red, green, and blue components of object with CS algorithm respectively, and the color image is obtained by combining the reconstructed three RGB images. We can see that the SNR of traditional CS imaging reduces slower with increased noise in measurement. This is because in dual CS imaging the reconstructed errors in the cascade CS processes will be overlying, making it more sensitive to the measurement noise. However, Fig. 6(e) actually indicates a kind of scanning spectral imaging, and only images of several discrete wavelengths can be obtained. 4.3. Improvements and prospects Our spectral imaging system still has some limitations to be solved. First, as a demonstration of the dual CS spectral imaging method, in our experiment the spatial resolution is only 64 64, while the largest spatial resolution can reach that of the spatial modulation device, which is 1024 768 for a DMD. Similarly, the spectral pixels are 1 1024 decided by the spectral modulation, while the spectral resolution can reach at least less than 1 nm with careful adjustment of the grating system as the spectral bandwidth on each pixel is narrow enough (0.1 nm). Besides, combined with the prior knowledge of optical system, CS algorithm can further improve the resolution [27]. Second, in the CS reconstruction we utilize the sparse property in the spatial and spectral domains separately. However, larger sparsity may be
Fig. 6. Dependence of dual CS spectral imaging quality on the measurement noise. (a–c) Reconstructed images of red (R), green (G) and blue (B) components. (d) RGB combined dual CS spectral images. (e) Traditional CS imaging results. The noise level is defined as the proportion of standard deviation of random noise and measurement result.
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
obtained if the spatial and spectral information is jointly considered. For example, the images of adjacent wavelengths are usually similar. Therefore, the sampling number can be further decreased to improve the imaging speed. Finally, the imaging time is relatively long to realize modulations in both domains. However, the high-flux measurement makes the SNR on point detector be much higher than array detector measurement, and thus the exposure time of one sampling can be very short. The imaging time is basically restricted by the speed of modulation, i.e. the DMD frame rate in our experiment. Nevertheless, the spectral imaging scheme with dual CS and single point detector brings specific benefits and is valuable in real applications. The data compression ability is improved compared with traditional CS imaging as the sampling in both domains is compressed. The use of point detector relieves the requirement of spectral imaging on high performance array detector, which is generally hard or expensive to be obtained. This is especially important in the infrared band as in this wave range manufacture of high performance array detector such as array APD is extremely difficult. Because of the high-flux measurement, it is much easier to satisfy the intensity demand of detector in the weak light situation as the signals of whole spatial and spectral components are accumulated. Therefore, this system is very suitable for spectral imaging of static weak signal target, which is common in biology and astronomy.
5. Conclusion In conclusion, we experimentally demonstrate a spectral imaging scheme with dual compressed sensing. We successfully obtain the image of a spectral object with only a point detector, with spatial and spectral sampling numbers both compressed. We analyze the performances of our method, including the effects of dual modulation numbers and measurement noise on the imaging quality. Through simulation, we find that the dual CS imaging demands enough spectral modulation number to avoid false ghost image emergency, and in certain extent is more sensitive to measurement noise than traditional CS imaging. Our method can provide spectral image with enough spatial and spectral resolutions with an easily achievable high performance single point detector. As our spectral imaging scheme does not need mechanical movement and each measurement is a global sampling, it has satisfying stability and consistency. Moreover, the high-flux measurement provides great convenience in the detection of weak signal. Therefore, we hope that it will have wide applications in biology, astronomy and other fields in which high performance spectral imaging is interested.
Acknowledgments We thank Ruo-Ming Lan for helpful discussions. This work was supported by the National Major Scientific Instruments Development Project of China (Grant no. 2013YQ030595), the National High Technology Research and Development Program of China (Grant no. 2013AA122902), and the National Natural Science
179
Foundation of China (Grant nos. 11275024, 61575207).
References [1] G.A. Howland, P.B. Dixon, J.C. Howell, Photon-counting compressive sensing laser radar for 3D imaging, Appl. Opt. 50 (2011) 5917–5920. [2] C.Q. Zhao, W.L. Gong, M.L. Chen, E.R. Li, H. Wang, W.D. Xu, S.S. Han, Ghost imaging lidar via sparsity constraints, Appl. Phys. Lett. 101 (2012) 141123. [3] W.K. Yu, S. Li, X.R. Yao, X.F. Liu, L.A. Wu, G.J. Zhai, Protocol based on compressed sensing for high-speed authentication and cryptographic key distribution over a multiparty optical network, Appl. Opt. 52 (2013) 7882–7888. [4] E.J. Candès, T. Tao, Decoding by linear programming, IEEE Trans. Inf. Theory 51 (2005) 4203–4215. [5] E.J. Candès, J. Romberg, T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inf. Theory 52 (2006) 489–509. [6] E.J. Candès, T. Tao, Near optimal signal recovery from random projections: Universal encoding strategies?, IEEE Trans. Inf. Theory 52 (2006) 5406–5425. [7] D. Donoho, Compressed sensing, IEEE Trans. Inf. Theory 52 (2006) 1289–1306. [8] M.F. Duarte, M.A. Davenport, D. Takhar, J.N. Laska, T. Sun, K.F. Kelly, R. G. Baraniuk, Single-pixel imaging via compressive sampling, IEEE Signal Process. Mag. 25 (2008) 83–91. [9] R.G. Baraniuk, Compressive sensing, IEEE Signal Process. Mag. 24 (2007) 118–121. [10] D. Takhar, J.N. Laska, M.B. Wakin, M.F. Duarte, D. Baron, S. Sarvotham, K. F. Kelly, R.G. Baraniuk, A new compressive imaging camera architecture using optical-domain compression, Proc. SPIE 6065 (2006) 43–52. [11] J.W. Ma, Single-pixel remote sensing, IEEE Geosci. Remote Sens. Lett. 2 (2009) 199–203. [12] W.K. Yu, X.F. Liu, X.R. Yao, C. Wang, G.J. Zhai, Q. Zhao, Single-photon compressive imaging with some performance benefits over raster scanning, Phys. Lett. A 378 (2014) 3406–3411. [13] Y.Y. Schechner, S.K. Nayar, P.N. Belhumeur, A theory of multiplexed illumination, in: Proceedings of IEEE International Conference on Computer Vision, Nice, France, IEEE, 2003, pp. 808–815. [14] R.M. Willett, M.F. Duarte, M.A. Davenport, R.G. Baraniuk, Sparsity and structure in hyperspectral imaging, IEEE Signal Process. Mag. 31 (2014) 116–126. [15] M.F. Duarte, R.G. Baraniuk, Spectral compressive sensing, Appl. Comput. Harmon. Anal. 35 (2013) 111–129. [16] T. Sun, K. Kelly, Compressive sensing hyperspectral imager, in: Computational Optical Sensing and Imaging, San Jose, USA, OSA Technical Digest (CD), Optical Society of America, 2009, paper CTuA5. [17] F. Soldevila, E. Irles, V. Duràn, P. Clemente, M. Fernàndez-Alonso, E. Tajahuerce, J. Lancis, Single-pixel polarimetric imaging spectrometer by compressive sensing, Appl. Phys. B 113 (2013) 551–558. [18] H. Arguello, G. Arce, Code aperture design for compressive spectral imaging, Presented at the 18th European Signal Processing Conference, Aalborg, Denmark, 23–27 August 2010. [19] H. Arguello, G. Arce, Code aperture optimization for spectrally agile compressive imaging, J. Opt. Soc. Am. A 28 (2011) 2400–2413. [20] G.R. Arce, D.J. Brady, L. Carin, H. Arguello, D.S. Kittle, Compressive coded aperture spectral imaging, IEEE Signal Process. Mag. 31 (2014) 105–115. [21] Y.H. Wu, I.O. Mirza, G.R. Arce, D.W. Prather, Development of a digital-micromirror-device-based multishot snapshot spectral imaging system, Opt. Lett. 36 (2011) 2692–2694. [22] X. Lin, G. Wetzstein, Y.B. Liu, Q.H. Dai, Dual-coded compressive hyperspectral imaging, Opt. Lett. 39 (2014) 2044–2047. [23] Y. August, C. Vachman, Y. Rivenson, A. Stern, Compressive hyperspectral imaging by random separable projections in both the spatial and the spectral domains, Appl. Opt. 52 (2013) D46–D54. [24] Y. Rivenson, A. Stern, Compressed imaging with a separable sensing operator, IEEE Signal Process. Lett. 16 (2009) 449–452. [25] M.F. Duarte, R.G. Baraniuk, Kronecker compressive sensing, IEEE Trans. Image Process. 21 (2012) 494–504. [26] C.B. Li, An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing (Master thesis), Rice University, 2009. [27] J. Chen, W.L. Gong, andS.S. Han, Sub-Rayleigh ghost imaging via sparsity constraints based on a digital micro-mirror device, Phys. Lett. A 377 (2013) 31–33.
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Invited Paper
Measurement dimensions compressed spectral imaging with a single point detector Xue-Feng Liu a,n, Wen-Kai Yu a,b, Xu-Ri Yao a,b, Bin Dai a,b, Long-Zhen Li a,b, Chao Wang a, Guang-Jie Zhai a a Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China b University of Chinese Academy of Sciences, Beijing 100190, China
art ic l e i nf o
a b s t r a c t
Article history: Received 20 October 2015 Received in revised form 1 December 2015 Accepted 11 December 2015
An experimental demonstration of spectral imaging with measurement dimensions compressed has been performed. With the method of dual compressed sensing (CS) we derive, the spectral image of a colored object can be obtained with only a single point detector, and sub-sampling is achieved in both spatial and spectral domains. The performances of dual CS spectral imaging are analyzed, including the effects of dual modulation numbers and measurement noise on the imaging quality. Our scheme provides a stable, high-flux measurement approach of spectral imaging. & 2015 Elsevier B.V. All rights reserved.
Keywords: Spectral imaging Compressed sensing Imaging quality Sub-sampling
1. Introduction Spectral imaging, which can capture both the spatial information and spectral information of an object, is of great importance in physics and biology as it can give rich evidences in the diagnoses of matter component and structure. For a spectral image, there are three dimensions of information to be measured, twodimensional spatial information and one-dimensional spectral information. It is obviously not possible to obtain three-dimensional information simultaneously with current detectors, which have two-dimensional resolution at most. As an alternative approach, the detection of spatial image or spectrum should be performed by scanning, which will reduce the system stability and consistency because of the mechanical movement and non-simultaneous measurements of different spatial and spectral components. In addition, the three-dimensional information measurement generates large amount of data, causing difficulties in data storage and transmission. In recent years, a sampling theory called compressed sensing (CS) was derived and has attracted wide interests in many fields [1–3]. With this theory, we can measure signals with sampling number far less than Nyquist–Shannon theorem demands [4–7]. n
Corresponding author. E-mail addresses: [email protected] (X.-F. Liu), [email protected] (G.-J. Zhai). http://dx.doi.org/10.1016/j.optcom.2015.12.020 0030-4018/& 2015 Elsevier B.V. All rights reserved.
Based on CS theory, Baraniuk et al. proposed an imaging approach named single pixel camera, in which a single pixel detector is used to image a two-dimensional object with measurement number less than image pixels [8–10]. Although in single pixel camera the sampling time is sacrificed as modulation and detection to the object should be performed repeatedly, the use of CS and single pixel detector provides many benefits to practical applications. In general array detector with pixels having the same performances with high quality point detector is difficult to be manufactured. For example, in many applications such as laser radar and fluorescence lifetime imaging high time-resolved single photon detection is needed, which is difficult or very expensive to achieve array detection, while CS imaging can provide an easily implemented solution with a Geiger-mode avalanche photodiode (APD) or hybrid photomultiplier tube (PMT) [1]. The sub-sampling ability can reduce the amount of stored data and thus memory requirements, which is valuable in the aerospace remote sensing and other fields having limitations in data storage and transmission [11]. Another important advantage is that the light from object is concentratedly measured by the point detector instead of being assigned on many pixels, improving the measurement signal-to-noise ratio (SNR) [12]. Especially for some detectors which can respond only if the light intensity reaches a threshold, the high-flux measurement in CS imaging is able to increase the imaging sensitivity [13]. As single pixel camera shows, CS theory permits the detection dimension reduction, which is very helpful in spectral imaging. Many applications of CS theory in spectral imaging have been
174
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
reported [14,15]. A straightforward method is replacing the point detector in single pixel camera with a spectrometer [16,17]. Arce et al. proposed coded aperture snapshot spectral imaging which obtains spectral images with an array detector needless of scanning based on CS theory [18–21]. In the above systems, only measurement in spatial domain is compressed and thus the compression ratio does not reach the optimum. To make the spatial and spectral measurement be both modulated an approach of dual-coded compressive spectral imaging is derived, while the final signal should be detected by an array detector to realize single exposure imaging [22]. In Ref. [23], spectral imaging with separable modulations and sub-sampling in both the spatial and spectral domains is proposed and demonstrated by simulations, in which the spectral modulation is performed in parallel and signal measured with a linear detector. To realize high SNR spectral imaging to an object in weak illumination, high-flux measurement with a high performance point detector is meaningful. In this paper, it is experimentally demonstrated that by implementing dual compressed sensing spectral imaging can be realized with a position-fixed single pixel detector. The detection dimensions and sampling numbers in both the spatial and spectral domains are compressed to provide higher ability in the sub-sampling. The light signal is concentratedly detected, avoiding the intensity assigned in both spatial and spectral dimensions and achieving much higher measurement SNR compared with array or linear detection. This paper is organized as follows. The theory and experimental results are presented in Sections 2 and 3, respectively. In Section 4 the performances and development prospects of the system are discussed. Finally the paper is concluded in Section 5.
2. Theory of dual compressed sensing Compressed sensing is a mathematical theory in which sampling can be performed with fewer measurements than the number of signal elements [4–7]. For a signal t having one dimensional information, CS makes linear measurement in this dimension:
I = A(r1)t (r1),
(1)
where A is the measurement matrix, I is the measurement result and r1 denotes the dimension of signal and measurement matrix. If the signal has two dimensions, it is straightforward to perform the linear measurement in the two dimensions as
I = A(r1, r2)t (r1, r2),
⎧ 1 2 I − A2 (r2)S(r2) + τ1 S(r2) ⎪ ⎪ S(r2) = min 2 1 S 2 ⎨ 1 2 ⎪ t (r , r ) = min S(r2) − A1(r1)t (r1, r2) + τ2 t (r1, r2) ⎪ 2 ⎩ 1 2 t 2
(5)
1
where ⋯ r stands for lr norm, defined as ( x r )r = ∑i = 1 x i , and τ1, τ2 are constant parameters weighting the relative strength of the two terms. As the measurements and reconstructions are performed in the two dimensions successively using CS theory twice, we call this sampling process as dual CS. For random measurement matrix the signal can be reconstructed accurately through m ≥ Ck log(n/k ) measurements, in which n is the number of elements, k is the sparsity of signal, and C is a constant coefficient [6,7], while for separate matrices the total measurement number should be increased by a factor of 1 log10(n1n2) in the dual CS sampling [23,24]. 2 N
r
3. Experiment and results As spectral imaging has information in two dimensions to be obtained, it can be realized with a single point detector by dual CS. The experimental apparatus is given in Fig. 1. Light from a halogen lamp illuminates the object, which is then imaged onto the Digital micromirror device (DMD) by a lens L1. The DMD (Texas Instruments DLP7000) consists of 1024 768 micromirrors each of size 13.68 μm × 13.68 μm , which can reflect the light to two directions individually. Controlled by the random matrix loaded into DMD1, it can reflect the modulated image to the direction of lens L2. If we measure the total intensity of light collected by lens L2, the grey image of object can be reconstructed by CS algorithm with the modulated matrix on DMD1. To acquire the spectral image, the spectrum information of the whole light must be measured. Therefore, we focus the light by lens L2 to a pinhole, which is then collimated by the lens L3 and illuminates on a blazed grating. The spectral line will emerge on the focal plane of L4, on which another DMD2 locates for utilizing CS for the second time. Similar to DMD1, the DMD2 randomly reflects different spectrum components to the direction of lens L5, which collects the light to a PMT. Ignoring the loss on the optical elements, 25% total intensity of whole spatial and spectral components from the object is detected by the PMT, which will then output the total light intensity of both spatial and spectral modulated images with 12-bit analog to digital conversion.
(2)
where t has information in dimensions of r1 and r2 with element numbers of n1 and n2, respectively. However, in practice it is usually difficult to make measurements in two dimensions simultaneously, and the amount of calculation in the signal reconstruction may be intolerable as the whole signal elements have a large number of n1n2. Alternatively, we can perform compressive measurements and reconstructions in the two dimensions successively,
⎧ S(r2) = A1(r1)t (r1, r2) ⎨ ⎩ I = A2 (r2)S(r2) ⎪
(3)
where S (r2) is the intermediate result of measurement in r1 dimension. This process can be expressed as the separate or Kronecker CS [24,25],
I = A(r1, r2)t (r1, r2) = A2 (r2) ⊗ A1(r1)t (r1, r2)
(4)
where ⊗ denotes the Kronecker product. The reconstruction of t (r1, r2) is also performed in the two dimensions successively,
Fig. 1. Experimental setup of spectral imaging with dual compressed sensing. Obj, object. G, blazed grating. L1–L5, lens. DMD1, DMD2, digital micromirror device. PMT, photomultiplier tube. The DMD can reflect light to two directions. Left bottom, the colored object to be imaged. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
175
magnification factor, the system spatial resolution on the object will be 80 μm . We use a grating with blaze wavelength of 500 nm and line number Mg ¼1800 grooves/mm as spectral dispersive element, the spectrum line ranging from 520 nm to 620 nm dispersed by which occupies 1 1024 pixels of DMD2, and the spectrum resolution is about
Δλ =
Fig. 2. Working time sequences of the DMDs and PMT.
The working time sequences of the DMDs and PMT are shown in Fig. 2. m1 random matrixes A(x, y ) are loaded into DMD1 sequentially, modulating the spatial information of the object. When DMD1 states on each modulation the corresponding spectrum should be obtained, so m2 random matrixes B(λ ) are loaded into DMD2 sequentially to modulate the spectral information of the object. During each modulation of DMD2, the PMT detects the total intensity of object under the spatial modulation A(x, y ) and spectral modulation B(λ ). In the experiment the DMDs and PMT all work in external trigger modes, and the working sequences are controlled by a circuit providing synchronous signals. The spectral image of the object can be expressed by T (λ )(x, y ), in which λ denotes the wavelength and (x, y ) denotes the spatial coordinate. The modulation process of DMD1 can be described mathematically as
∑ A(x, y)T (λ)(x, y) = S(λ), x, y
(6)
where S (λ ) is the spectrum all over spatial areas of image which are reflected to the collecting direction, which is also the image on DMD2. The modulation process of DMD2 can be described as
∑ B(λ)S(λ) = I, λ
(7)
where I is the total intensity detected by the PMT. Based on CS algorithm, the spectrum S (λ ) can be recovered with B(λ ) and I, and then the spectral image T (λ )(x, y ) can be reconstructed with S (λ ) and A(x, y ). We use TVAL3 algorithm to recover the spectrum and image, which gives a sparsest solution in the gradient basis [26]. For the sparse basis related to the frequency transform such as wavelet or Fourier basis, the neglect of small high-frequency coefficients in the solving process may decrease the resolution, while with gradient basis it will not be influenced by the CS reconstruction. In our experiment, the object to be imaged is a film with total size of 3.2 mm 3.2 mm printed a green “C” and red “S” on it, which is shown in the left bottom of Fig. 1. The object is imaged onto 128 128 pixels of DMD1 with magnification factor of 0.55, while for increasing the reconstruction speed we combine 2 2 pixels as one pixel in the random modulated matrix, so the valid resolution of the imaging is 64 64. To ensure the overlapping of image and mirrors, in the calibration of optical setup a camera (AVT Manta G-145B) with lens is used to observe the mirror plane. Light from DMD1 is focused by a lens L2 with focal length of f2 ¼50.8 mm to the pinhole with diameter of d ¼1 mm, which restricts the spatial resolution on the DMD1 to be
Δx = 1.22
λf2 d
= 35 μm
(8)
for λ = 570 nm . As the size of DMD1 modulation pixel is 27.36 μm , the final spatial resolution on DMD1 plane is the convolution of the two blurring effects, which is 44.4 μm . Considering the
d cos θi = 2.4 nm Mgf3
(9)
where θi = 50° is the incident angle on the grating, and the focal length of collimation lens L3 is f3 = 150 mm . This value in Eq. (9) is much larger than the spectral range on one pixel of DMD2 which is about 0.1 nm, and therefore gives the spectral resolution of the system. The calibration of spectrum is performed with a commercial spectrometer (Avantes, ULS3648) collecting light from DMD2. With DMD2 set to work in scanning mode, the spectrometer can give the spectrum range of each column in DMD2. As the spectrum is a onedimensional quantity, DMD2 modulates only in the 1 1024 pixels by setting the other region keep reflecting light to the uncollected direction. In the spectral imaging of the object, the modulation numbers of DMD1 and DMD2 are m1 = 1600 and m2 = 400, respectively. Therefore, the measurements of both spatial and spectral information achieve sub-sampling, and the total compression ratio is (1600 × 400) /(4096 × 1024) = 15.3% . The working frequencies of DMD1 and DMD2 are 1 Hz and 450 Hz, respectively, so the spectral imaging costs 1600 s to take the entire modulation sequence. The imaging results are shown in Fig. 3. Based on TVAL3 algorithm, during each modulation A(x, y ) on DMD1 we can obtain a spectrum line according to the m2 = 400 modulations on DMD2 and correspondent intensities I detected by the PMT, which is shown as a row in Fig. 3(a). The average spectrum is also plotted, the two peaks around 540 nm and 600 nm are from “C” and “S”, respectively. Corresponding to spatial modulations on DMD1, m1 = 1600 spectrum lines are recovered successively. For clear vision, we only show the first 200 spectrum lines in Fig. 3(a). Then each column of Fig. 3(a) indicates the intensity fluctuation of light from spatial modulated image with the same wavelength. Combined the spatial modulations on DMD1 and intensity fluctuation of each wavelength, we can reconstruct images of different wavelengths with TVAL3 algorithm for the second time. In Fig. 3(b1) and (b2), the intensity fluctuations of light with λ = 530 nm and λ ¼610 nm are shown, which is transmitted from the green “C” and red “S”, respectively. As the spatial modulations on images of various wavelengths are different, the variation trends of the two curves in Fig. 3(b1) and (b2) are not the same, although they correspond to the same modulation matrices loaded into DMD1. The images under the two wavelengths are then reconstructed, giving the green “C” and red “S”, which are shown in Fig. 3(c1) and (c2). From the imaging results, we see the images of various wavelengths can be obtained without affected by other spectral components using our spectral imaging system with dual compressed sensing. If we integrate the intensities in each row of Fig. 3(a) over the whole spectrum and take the CS reconstruction, the spectral information will vanish and the image without spectrum can be obtained, similar to the conventional CS imaging. The intensity fluctuation of the whole spectrum is plotted in Fig. 3(b3) and the corresponding imaging result is shown in Fig. 3(c3), in which a clear image of “CS” emerges.
4. Discussions on the performances of dual CS spectral imaging 4.1. Effect of dual modulation numbers on the imaging quality The scheme of spectral imaging with dual compressed sensing
176
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
Fig. 3. Experimental results of spectral imaging. (a) Spectrum lines under different spatial modulations and the average spectrum curve. (b) Intensity fluctuations of different wavelengths, (b1) λ¼ 530 nm, (b2) λ ¼ 610 nm, and (b3) all-spectrum. (c) Imaging results of different wavelengths, (c1) λ ¼530 nm, (c2) λ¼ 610 nm, and (c3) all-spectrum. The reconstruction time is about 120 s.
can realize “3D” (two-dimensional image and one-dimensional spectrum) imaging with only a single point detector, while the imaging time will be expended. Compared with previous spectral imaging with array or linear detector, the sampling time of dual CS is increased because of the modulations on both spatial and spectral domains. Therefore, there is a tradeoff between the detection dimensions and the imaging time, which is decided by the product of numbers of spatial modulations m1 on DMD1 and spectral modulations m2 on DMD2. To study the impact of sampling numbers on the imaging quality, the numerical simulation is performed. The spectral object still includes a green “C” and red “S”, the center transmission wavelengths of which are 500 nm and 600 nm, respectively, with spectral widths of both 40 nm. The spatial imaging resolution is 64 64, and the spectrum range is 400–700 nm with resolution of 1 1024. In the simulation 10% Gaussian random noise is added in the measurement results. As the imaging qualities of different wavelengths are similar ignoring the specific spatial features, in the simulation we only reconstruct the green “C” with spectrum range of 480–520 nm.
Fig. 4 presents the reconstructions for various numbers of spatial modulations m1 and spectral modulations m2. In the three rows, the number of spectral modulations is 200, 400, and 600, respectively, and in the three columns, the number of spatial modulations is 800, 1600, and 2400, respectively. It is obvious that the increases of both m1 and m2 can improve the imaging quality, while the particular impacted aspects are different. In Fig. 4(a1– a3), the sampling number in spectral information m2 is only about 20% of the spectrum pixels, causing inaccuracy in the reconstruction of spectrum. Therefore, “S” emerges in the reconstruction results as the information of other wavelengths is contained in the intensity fluctuations of light used to recover “C”. When the spectral sampling number increases, the spectrum can be reconstructed accurately and the red “S” will not be imaged, as shown in Fig. 4(b) and (c). From the left column to the right column, the increase of spatial sampling number m1 can dramatically decrease the noise in the images, while it is unrelated with the spectrum reconstruction accuracy. In Fig. 4(a), with the increased spatial modulations, the images of “S” even tend to be more clear.
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
177
Fig. 4. Numerical simulation results of dual CS spectral imaging for various numbers of spatial modulations m1 and spectral modulations m2. The SNRs of real image “C” and ghost image “S” are marked under the imaged letters.
As the intensity fluctuation of the reconstructed spectrum has contained the information of “S”, the increase of spatial sampling can improve the quality of the ghost image which should not appear in the reconstructed result. To quantitatively evaluate the influence of modulation numbers on the imaging quality, the SNR of reconstructed real image and ghost image is calculated respectively, which is defined as
SNR =
max(U ) std( U − U0 )
(10)
where U0 is the original image, U is the reconstructed one and std denotes standard deviation. The SNR of real image is calculated with left half of original and reconstructed images containing “C” and SNR of ghost image is obtained with right half containing “S”, which are both marked in Fig. 4 to quantitatively support the above discussions. Fig. 5(a) plots the SNRs as a function of spatial modulation number m1. The spectral modulation number is only m2 = 100, so the ghost image exists. Along with spatial modulation increased, the SNRs of real and ghost images both become higher. Fig. 5(b) shows the influence of spectral modulation number m2 on the imaging quality with spatial modulation number of m1 = 2000. We can see that the quality of real image is slightly influenced by the spectral modulation number, while the SNR of ghost image decreases with m2 increased, indicating that the ghost image is disappearing. The simulations show that the spectral modulation number m2 decides the accuracy of spectrum reconstruction, and the spatial modulation number m1 affects the noise in the image. This
Fig. 5. (a) SNR of dual CS spectral imaging as a function of spatial modulation number m1 while m2 ¼ 100. (b) SNR of dual CS spectral imaging as a function of spectral modulation number m2 while m1 ¼2000. Circles and crosses are simulation results of real and ghost images, respectively.
178
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
conclusion is meaningful in the design of experimental process. In Fig. 4(a3) and (c1), the sampling numbers decided by m1 × m2 are the same, while the spectral imaging results are quite different. In Fig. 4(a3), the large number of spatial modulations makes the imaging noise be very low. However, the lack of enough spectral sampling causes ghost component in the spectral image, which is fatal in most spectral imaging applications. In Fig. 4(c1), the ghost image is eliminated by enough spectral modulations. Although there is a relatively high noise level, it can be distinguished artificially according to experience in most situations. Therefore, in spectral imaging, the proper choice of spatial modulations m1 and spectral modulations m2 is important for a finite sampling time. To obtain correct information about the spectral image, enough number of spectral sampling should be guaranteed. 4.2. Effect of measurement noise on the imaging quality We simulate the effect of dual CS spectral imaging with complex object, which is part of a circuit board with continuous spectrum from 400 nm to 700 nm and pixels of 60 100. With dual CS, we image the red (R), green (G) and blue (B) components of the object with spectra centered at 650 nm, 550 nm, and 450 nm respectively and bandwidths of 60 nm, which are shown in Fig. 6(a–c). The dual modulation numbers are m1 = 2000 and m2 = 500 respectively, realizing total compression ratio of 16.3%. By combining the three RGB components colored images are obtained in Fig. 6(d), showing no color distortion. The dependence of spectral imaging quality on the measurement noise is also studied with Gaussian random noise of different levels added in the total measurement intensity. The SNRs of RGB images are calculated, and the average value gives the SNR of the colored image. In the condition of low noise level dual CS spectral imaging can give high
quality image without color distortion, while with the increase of noise the SNR of image gradually decreases. For comparison in Fig. 6(e) we show the colored images obtained by traditional CS imaging. This can be realized in experiment by locating different spectral filters in front of the detector to perform CS imaging of RGB images and combining together. In the simulation, 2000 modulations are used to image the red, green, and blue components of object with CS algorithm respectively, and the color image is obtained by combining the reconstructed three RGB images. We can see that the SNR of traditional CS imaging reduces slower with increased noise in measurement. This is because in dual CS imaging the reconstructed errors in the cascade CS processes will be overlying, making it more sensitive to the measurement noise. However, Fig. 6(e) actually indicates a kind of scanning spectral imaging, and only images of several discrete wavelengths can be obtained. 4.3. Improvements and prospects Our spectral imaging system still has some limitations to be solved. First, as a demonstration of the dual CS spectral imaging method, in our experiment the spatial resolution is only 64 64, while the largest spatial resolution can reach that of the spatial modulation device, which is 1024 768 for a DMD. Similarly, the spectral pixels are 1 1024 decided by the spectral modulation, while the spectral resolution can reach at least less than 1 nm with careful adjustment of the grating system as the spectral bandwidth on each pixel is narrow enough (0.1 nm). Besides, combined with the prior knowledge of optical system, CS algorithm can further improve the resolution [27]. Second, in the CS reconstruction we utilize the sparse property in the spatial and spectral domains separately. However, larger sparsity may be
Fig. 6. Dependence of dual CS spectral imaging quality on the measurement noise. (a–c) Reconstructed images of red (R), green (G) and blue (B) components. (d) RGB combined dual CS spectral images. (e) Traditional CS imaging results. The noise level is defined as the proportion of standard deviation of random noise and measurement result.
X.-F. Liu et al. / Optics Communications 365 (2016) 173–179
obtained if the spatial and spectral information is jointly considered. For example, the images of adjacent wavelengths are usually similar. Therefore, the sampling number can be further decreased to improve the imaging speed. Finally, the imaging time is relatively long to realize modulations in both domains. However, the high-flux measurement makes the SNR on point detector be much higher than array detector measurement, and thus the exposure time of one sampling can be very short. The imaging time is basically restricted by the speed of modulation, i.e. the DMD frame rate in our experiment. Nevertheless, the spectral imaging scheme with dual CS and single point detector brings specific benefits and is valuable in real applications. The data compression ability is improved compared with traditional CS imaging as the sampling in both domains is compressed. The use of point detector relieves the requirement of spectral imaging on high performance array detector, which is generally hard or expensive to be obtained. This is especially important in the infrared band as in this wave range manufacture of high performance array detector such as array APD is extremely difficult. Because of the high-flux measurement, it is much easier to satisfy the intensity demand of detector in the weak light situation as the signals of whole spatial and spectral components are accumulated. Therefore, this system is very suitable for spectral imaging of static weak signal target, which is common in biology and astronomy.
5. Conclusion In conclusion, we experimentally demonstrate a spectral imaging scheme with dual compressed sensing. We successfully obtain the image of a spectral object with only a point detector, with spatial and spectral sampling numbers both compressed. We analyze the performances of our method, including the effects of dual modulation numbers and measurement noise on the imaging quality. Through simulation, we find that the dual CS imaging demands enough spectral modulation number to avoid false ghost image emergency, and in certain extent is more sensitive to measurement noise than traditional CS imaging. Our method can provide spectral image with enough spatial and spectral resolutions with an easily achievable high performance single point detector. As our spectral imaging scheme does not need mechanical movement and each measurement is a global sampling, it has satisfying stability and consistency. Moreover, the high-flux measurement provides great convenience in the detection of weak signal. Therefore, we hope that it will have wide applications in biology, astronomy and other fields in which high performance spectral imaging is interested.
Acknowledgments We thank Ruo-Ming Lan for helpful discussions. This work was supported by the National Major Scientific Instruments Development Project of China (Grant no. 2013YQ030595), the National High Technology Research and Development Program of China (Grant no. 2013AA122902), and the National Natural Science
179
Foundation of China (Grant nos. 11275024, 61575207).
References [1] G.A. Howland, P.B. Dixon, J.C. Howell, Photon-counting compressive sensing laser radar for 3D imaging, Appl. Opt. 50 (2011) 5917–5920. [2] C.Q. Zhao, W.L. Gong, M.L. Chen, E.R. Li, H. Wang, W.D. Xu, S.S. Han, Ghost imaging lidar via sparsity constraints, Appl. Phys. Lett. 101 (2012) 141123. [3] W.K. Yu, S. Li, X.R. Yao, X.F. Liu, L.A. Wu, G.J. Zhai, Protocol based on compressed sensing for high-speed authentication and cryptographic key distribution over a multiparty optical network, Appl. Opt. 52 (2013) 7882–7888. [4] E.J. Candès, T. Tao, Decoding by linear programming, IEEE Trans. Inf. Theory 51 (2005) 4203–4215. [5] E.J. Candès, J. Romberg, T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inf. Theory 52 (2006) 489–509. [6] E.J. Candès, T. Tao, Near optimal signal recovery from random projections: Universal encoding strategies?, IEEE Trans. Inf. Theory 52 (2006) 5406–5425. [7] D. Donoho, Compressed sensing, IEEE Trans. Inf. Theory 52 (2006) 1289–1306. [8] M.F. Duarte, M.A. Davenport, D. Takhar, J.N. Laska, T. Sun, K.F. Kelly, R. G. Baraniuk, Single-pixel imaging via compressive sampling, IEEE Signal Process. Mag. 25 (2008) 83–91. [9] R.G. Baraniuk, Compressive sensing, IEEE Signal Process. Mag. 24 (2007) 118–121. [10] D. Takhar, J.N. Laska, M.B. Wakin, M.F. Duarte, D. Baron, S. Sarvotham, K. F. Kelly, R.G. Baraniuk, A new compressive imaging camera architecture using optical-domain compression, Proc. SPIE 6065 (2006) 43–52. [11] J.W. Ma, Single-pixel remote sensing, IEEE Geosci. Remote Sens. Lett. 2 (2009) 199–203. [12] W.K. Yu, X.F. Liu, X.R. Yao, C. Wang, G.J. Zhai, Q. Zhao, Single-photon compressive imaging with some performance benefits over raster scanning, Phys. Lett. A 378 (2014) 3406–3411. [13] Y.Y. Schechner, S.K. Nayar, P.N. Belhumeur, A theory of multiplexed illumination, in: Proceedings of IEEE International Conference on Computer Vision, Nice, France, IEEE, 2003, pp. 808–815. [14] R.M. Willett, M.F. Duarte, M.A. Davenport, R.G. Baraniuk, Sparsity and structure in hyperspectral imaging, IEEE Signal Process. Mag. 31 (2014) 116–126. [15] M.F. Duarte, R.G. Baraniuk, Spectral compressive sensing, Appl. Comput. Harmon. Anal. 35 (2013) 111–129. [16] T. Sun, K. Kelly, Compressive sensing hyperspectral imager, in: Computational Optical Sensing and Imaging, San Jose, USA, OSA Technical Digest (CD), Optical Society of America, 2009, paper CTuA5. [17] F. Soldevila, E. Irles, V. Duràn, P. Clemente, M. Fernàndez-Alonso, E. Tajahuerce, J. Lancis, Single-pixel polarimetric imaging spectrometer by compressive sensing, Appl. Phys. B 113 (2013) 551–558. [18] H. Arguello, G. Arce, Code aperture design for compressive spectral imaging, Presented at the 18th European Signal Processing Conference, Aalborg, Denmark, 23–27 August 2010. [19] H. Arguello, G. Arce, Code aperture optimization for spectrally agile compressive imaging, J. Opt. Soc. Am. A 28 (2011) 2400–2413. [20] G.R. Arce, D.J. Brady, L. Carin, H. Arguello, D.S. Kittle, Compressive coded aperture spectral imaging, IEEE Signal Process. Mag. 31 (2014) 105–115. [21] Y.H. Wu, I.O. Mirza, G.R. Arce, D.W. Prather, Development of a digital-micromirror-device-based multishot snapshot spectral imaging system, Opt. Lett. 36 (2011) 2692–2694. [22] X. Lin, G. Wetzstein, Y.B. Liu, Q.H. Dai, Dual-coded compressive hyperspectral imaging, Opt. Lett. 39 (2014) 2044–2047. [23] Y. August, C. Vachman, Y. Rivenson, A. Stern, Compressive hyperspectral imaging by random separable projections in both the spatial and the spectral domains, Appl. Opt. 52 (2013) D46–D54. [24] Y. Rivenson, A. Stern, Compressed imaging with a separable sensing operator, IEEE Signal Process. Lett. 16 (2009) 449–452. [25] M.F. Duarte, R.G. Baraniuk, Kronecker compressive sensing, IEEE Trans. Image Process. 21 (2012) 494–504. [26] C.B. Li, An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing (Master thesis), Rice University, 2009. [27] J. Chen, W.L. Gong, andS.S. Han, Sub-Rayleigh ghost imaging via sparsity constraints based on a digital micro-mirror device, Phys. Lett. A 377 (2013) 31–33.