High throughput dual-wavelength temperature distribution imaging via compressive imaging

Optics Communications 410 (2018) 287–291

Contents lists available at ScienceDirect

Optics Communications journal homepage: www.elsevier.com/locate/optcom

High throughput dual-wavelength temperature distribution imaging via compressive imaging Xu-Ri Yao a, *, Ruo-Ming Lan b , Xue-Feng Liu a , Ge Zhu c , Fu Zheng a , Wen-Kai Yu c , Guang-Jie Zhai a, * a b c

Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China School of Physics and Electronics, Shandong Normal University, Jinan 250014, China School of Physics, Beijing Institute of Technology, Beijing 100081, China

a r t i c l e

i n f o

Keywords: Thermal imaging Computational imaging Single-pixel camera

a b s t r a c t Thermal imaging is an essential tool in a wide variety of research areas. In this work we demonstrate highthroughput double-wavelength temperature distribution imaging using a modified single-pixel camera without the requirement of a beam splitter (BS). A digital micro-mirror device (DMD) is utilized to display binary masks and split the incident radiation, which eliminates the necessity of a BS. Because the spatial resolution is dictated by the DMD, this thermal imaging system has the advantage of perfect spatial registration between the two images, which limits the need for the pixel registration and fine adjustments. Two bucket detectors, which measures the total light intensity reflected from the DMD, are employed in this system and yield an improvement in the detection efficiency of the narrow-band radiation. A compressive imaging algorithm is utilized to achieve under-sampling recovery. A proof-of-principle experiment was presented to demonstrate the feasibility of this structure. © 2017 Elsevier B.V. All rights reserved.

1. Introduction Optical radiation pyrometry, which eliminates the fragility concerns that currently limit thermocouple measurements, provides an effective and practical approach to temperature measurements in a high-speed or noncontact situation. Noncontact radiometric temperature measurements are based on blackbody emission theory [1,2]. In general, to deduce the temperature from the measurement of emitted radiation, the value of the surface emissivity 𝜖, must be known. Therefore, in conditions that do not allow the independent measurement of target emissivity, the true temperature cannot be measured by the conventional radiometric methods. Ratio pyrometry (also called two-color pyrometry or dualwavelength pyrometry) [3,4], which measures the ratio of energy collected at two adjacent wavelengths, is widely used for high-temperature measurement in various fields [5–10]. Wavelength ranges are chosen to be as close as possible, such that the effect of material-specific peculiarities (reflectance, emissivity) and optical obstructions (e.g., smoke, dust, or dirty lenses) from the target are nearly-identical for both wavelength ranges. The influences on measurements can be corrected by calculating * Corresponding authors.

E-mail addresses: [email protected] (X.-R. Yao), [email protected] (G.-J. Zhai). https://doi.org/10.1016/j.optcom.2017.10.028 Received 10 June 2017; Received in revised form 11 October 2017; Accepted 14 October 2017 Available online 5 November 2017 0030-4018/© 2017 Elsevier B.V. All rights reserved.

the ratio. The availability of infrared (IR) array sensors conveniently provides two-dimensional temperature distribution measurements that overcome the shortcomings of traditional point pyrometers. There are several possible procedures that can be used to complete two-color temperature distribution imaging with IR cameras. (1) In a singlecamera system, different filters mounted on a rotating filter wheel in front of the detector are employed to split the measured radiation. However, the results may lose accuracy in the case of a fast-moving or non-stationary target because the measurement alternates between the two channels [10,11]. (2) Two synchronized cameras fitted with filters capture the radiation divided by a beam splitter (BS). However, the price of the cameras used in such applications is high, which makes the two-camera solution unreasonable for practical applications [9,11]. (3) A double detector (sandwich design) fitted with filters can be used, but this device is challenging to employ when wavelengths vary [10,12]. Specialized optical image splitters have also been constructed for dualwavelength imaging, these are able to project two identical images side by side on the camera with complex optical structures [11,6]. The limited usage of the double wavelength thermal imaging arises from the complex dual light path design and the cost of IR array sensors. In

X.-R. Yao et al.

Optics Communications 410 (2018) 287–291

addition to the dual light path design, the detection of weak radiation after the narrow-banded filters is another obstacle in the two-color temperature imaging system. The ‘single-pixel camera’ [13,14] technique, which provides an approach to imaging in scenarios in which multi-pixel sensors are not available due to cost or technological constraints, exhibits superiority in many optical schemes [15–18], especially in some spectral imaging configurations [19–21]. Compressed sensing (CS) algorithms [22,23], which utilize the sparsity of a target in a certain domain, enables singlepixel cameras to recover an image with far fewer measurements than the Nyquist limit. In this work we demonstrate thermal imaging from a modified single-pixel camera for double wavelengths, which enables high-quality thermal imaging while lowering the complexity in the optical design of a device and exploiting high throughput detection. The advantages of the proposed method are (1) A digital micro-mirror device (DMD) is utilized to display binary masks and split the incident radiation, which eliminates the need for a BS. Pixel registration and fine adjustments of the two images are unnecessary because of this reflected structure. (2) The high throughput bucket detector measures many pixels simultaneously and so results in an improvement of the detection efficiency. This is helpful, because the radiation in a narrow-band is often very difficult to detect. (3) Compression algorithm is utilized for accurate reconstruction with measurements less than that of the pixels to be resolved. (4) Changing wavelengths is simple using this method. The manuscript consists of four sections. In the second section, the method is presented. The experimental procedure is discussed in the third section. An experimental thermal image with a resolution of 256 × 256 pixels is acquired to show the feasibility of this structure. A proof-of-principle experiment was performed to show the superiority of high throughput detection.

From the above equation, 𝑇𝑟 =

1 1 𝑇

+

ln(𝜀(𝜆1 )∕𝜀(𝜆2 ))

.

(6)

𝐶2 ( 𝜆1 − 𝜆1 ) 1 2

For the dual wavelength method, an a priori relation for the emissivity variation, but not the emissivity value, is needed to infer the target temperature. Supposing the target is a gray-body (𝜀 is independent of wavelength) or has equal emissivity at 𝜆1 and 𝜆2 , then 𝜀(𝜆1 ) = 𝜀(𝜆2 ), and 𝑇𝑟 = 𝑇 . The accuracy of the dual-wavelength method is crucially dependent on the selection of the two wavelengths 𝜆1 and 𝜆2 . The expected temperature of the target and the spectral sensitivities of the detectors influence the choice of wavelengths. According to Wien’s law, the wavelength 𝜆𝑟𝑚𝑎𝑥 for which the radiance is maximal is given by 𝜆𝑟𝑚𝑎𝑥 𝑇 = 2898 μm K, the higher the temperature, the shorter the radiation wavelength 𝜆𝑟𝑚𝑎𝑥 . Another factor that affects the accuracy is the interval between the two wavelengths, which should be sufficiently small to validate the gray-body hypothesis but large enough to ensure a sufficiently precise measurement. 2.2. Compressive imaging

2. Method

The CS reconstruction algorithm employs optimization to detect a sparse 𝑛 dimensional signal with 𝑚 < 𝑛 measurements by pursuing the sparse solution of the minimum 𝑙1-norm in the optimization program. If we denote the object as an 𝑛-dimensional vector 𝑂(𝑥), we assume that there exists a transformation matrix 𝛹 to the sparse basis such that 𝑂(𝑥) = 𝛹 ⋅ 𝑂′ (𝑥′ ), where 𝑂′ (𝑥′ ) is sparse. The a priori knowledge that the object can be sparsely expressed in a known basis is general, because many natural objects are indeed sparse in the appropriate basis. In CS, the measurement process can be formulated as

2.1. Double-wavelength temperature measurement

𝑦 = 𝐴𝛹 𝑂′ (𝑥′ ) + 𝑒,

The relationship between radiation energy and the true temperature of a target in a monochromatic wavelength 𝜆 is obtained by using Planck’s radiation law

where 𝐴 is an 𝑚×𝑛 measurement matrix (𝑚 < 𝑛) and 𝑒 is the noise. In this paper, a complementary measurement method is adopted [18]. Thus, 𝐴 is a −1/1 binary matrix with zero mean. Because 𝑚 < 𝑛, the observation vector 𝑦 does not specify a unique 𝑂(𝑥). In CS, the 𝑂′ (𝑥′ ) of the minimum 𝑙1-norm that yields a good agreement with the measurements is pursued through { } ′ ′ ‖2 ‖ ′ ′‖ 𝑂′ (𝑥′ ) = arg min ‖ ‖𝐴 ⋅ 𝛹 ⋅ 𝑂 (𝑥 ) − 𝑦‖2 − 𝜏 ‖𝑂 (𝑥 )‖1 , (8) 𝑂(𝑥) = 𝛹 ⋅ 𝑂̂ ′ (𝑥′ )

𝐶1

𝜀(𝜆) , (1) 𝑒𝐶2 ∕𝜆𝑇 − 1 where 𝑀 is the energy of radiation, 𝜆 is the wavelength, 𝑇 is the true temperature, 𝐶1 = 3.7403 [J∕m3 ] is the 1st radiation constant, 𝐶2 = 14387.69 [μm K] is the 2st radiation constant, 𝜀(𝜆) is the emissivity of the target at wavelength 𝜆. When 𝑇 < 3000 K, using Wien’s approximate formula, Eq. (1) can be simplified to

𝑀(𝜆, 𝜀, 𝑇 ) =

𝜆5

⋅

̂ where 𝑂(𝑥) is the reconstructed image and 𝜏 is a constant scalar that weights the relative strength of the two terms. In compressive imaging methods, the binary matrixes are generally realized via spatial structure illumination, and the observation vector 𝑦 corresponds to the signal from a bucket detector that collects all of the modulated light. Various iterative algorithms [24–26] have been developed to solve the optimization problem specified by Eq. (8). In dual-wavelength temperature distribution imaging, two images taken at different wavelengths are necessary. We employ a modified single-pixel camera system to capture the two images 𝐼𝜆1 (𝑥) and 𝐼𝜆2 (𝑥). The temperature distribution of the target is obtained from the ratio of two images,

𝐶1

⋅ 𝜀(𝜆)𝑒−𝐶2 ∕𝜆𝑇 . (2) 𝜆5 The two-color thermometry constructed operates at two wavelengths 𝜆1 and 𝜆2 . Ignoring the bandwidths of the two wavelengths, Eq. (2) yields [ ] ( ) 𝑀(𝜆1 , 𝜀(𝜆1 ), 𝑇 ) 𝜀(𝜆1 ) 𝜆2 5 𝐶 1 1 𝑅(𝑇 ) = = exp 2 ( − ) , (3) 𝑀(𝜆2 , 𝜀(𝜆2 ), 𝑇 ) 𝜀(𝜆2 ) 𝜆1 𝑇 𝜆1 𝜆2 𝑀(𝜆, 𝜀, 𝑇 ) =

where 𝑅(𝑇 ) is the ratio between the energies of the two wavelengths. Then, the temperature ratio 𝑇𝑟 is [ ] 𝐶2 (1∕𝜆1 ) − (1∕𝜆2 ) 𝑇𝑟 = . (4) ln 𝑅(𝑇 ) − 5 ln(𝜆2 ∕𝜆1 ) − ln(𝜀(𝜆1 )∕𝜀(𝜆2 )) From Eq. (2), the relation between the ratio temperature 𝑇𝑟 and the true temperature 𝑇 can be derived 𝐶1 𝜆1 5 𝐶1 𝜆2 5

⋅ 𝑒−𝐶2 ∕𝜆1 𝑇𝑟 = ⋅ 𝑒−𝐶2 ∕𝜆2 𝑇𝑟 =

𝐶1 𝜆1 5 𝐶1 𝜆2 5

(7)

𝑅(𝑥) =

𝜎1 𝐼𝜆1 (𝑥) 𝜎2 𝐼𝜆2 (𝑥)

,

(9)

where 𝜎1 and 𝜎2 characterize the optical efficiencies and detector sensitivities of the real system at the two wavelengths, respectively, and can be obtained by calibrating the instrument using a blackbody source at a known temperature.

⋅ 𝜀(𝜆1 )𝑒−𝐶2 ∕𝜆1 𝑇 (5) ⋅ 𝜀(𝜆2 )𝑒−𝐶2 ∕𝜆2 𝑇 . 288

X.-R. Yao et al.

Optics Communications 410 (2018) 287–291

Fig. 3. Reconstructed images of the filament at (a) 640 and (b) 810 nm.

Fig. 1. Schematic of double wavelength thermal imaging via compressive sensing. Imaging Lens is an achromatic lens with a focal length 50.4 mm, Collecting lens 1 and 2 are quartz lenses with a focal length of 30 mm. Filter 1 is a narrowband dielectric filter centered at 640 nm, and filter 2 is a narrowband dielectric filter centered at 810 nm.

Fig. 4. Temperature distribution images of the target recovered from (a) the image of the filament at 810 nm and (b) the two-color method. In (b), 𝑚 = 10000, sampling rate=15.3%.

were chosen after considering the target temperature and the quantum efficiency of the detector. A plot of the blackbody radiant flux densities (flux) as a function of temperature for 640 and 810 nm is shown in Fig. 2. The intensity ratio varies almost linearly with the temperature in the range of 1500 K–4000 K, which provides sufficient temperature sensitivity. The −1/1 binary patterns with zero mean of a size of 256 × 256 pixels were encoded on the DMD. Since only non-negative matrices can be physically displayed by DMD, the −1/1 binary matrix is realized by displaying 0/1 pattern followed immediately by its inverse. The number of the measurements was 10 000, the sampling rate was 15.3%, and the TVAL3 algorithm [26] was employed to solve Eq. (8) in the experiment. Two signals from the two CCDs were combined with binary patterns to reveal the double wavelengths images of the target. The intensity images of the target retrieved at 640 and 810 nm are shown in Fig. 3(a) and (b), respectively. Because the spatial resolution is dictated by the DMD, this dual-path single-pixel imaging system has the advantage of perfect spatial registration between the two images, which may be a problem when two cameras are used with a BS. The optical efficiencies and detector sensitivities of the two wavelengths 𝜎1 and 𝜎2 were calibrated using a stabilized tungsten-halogen light source (Thorlabs SLS201). The reconstructed temperature distribution image of the target is shown in Fig. 4(b). To provide a comparison, the temperature distribution recovered from the image of the filament at 810 nm according to the Planck’s radiation law is shown in Fig. 4(a). Because the emissivity 0 < 𝜖 < 1, the temperature in Fig. 4(a) is lower than the actual because of the influence of the emissivity. In CS, the number of measurements can be less than the number of the pixels to be resolved. A benefit of CS is that it is progressive, and increasing the number of measurements improves the quality of the reconstruction. We reduced the measurements used in the calculation by steps of 2500. Fig. 5 presents reconstructions of the temperature distribution of the target for various numbers of measurements. In this proof-of-principle experiment, the sampling frequency was 20 Hz, which was limited by the frame rate of the CCD and cannot meet the needs of practical applications. The highest frame frequency of the DMD can be more than 20 kHz. The CCDs can be substituted by point

Fig. 2. Spectral intensity of a blackbody at 640 (red curve) and 810 nm (blue curve), and the ratio of the intensities (green curve) as a function of temperature. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

3. Experiment Our experimental setup to demonstrate temperature distribution imaging via dual wavelength measurements is schematically illustrated in Fig. 1. The filament of a lamp with a temperature of approximately 2800 K acts as the object, and it is imaged onto the DMD using an achromatic imaging lens (𝑓 = 50.4 mm). The DMD consists of a 1024×768 array of individually addressable mirrors and the size of each mirror is 13.68 × 13.68 μm. Each mirror can be tilted in two different directions, +12◦ (1) or −12◦ (0) with respect to the surface of the DMD. The incident light from the target is reflected into the two directions depending on the pattern encoded on the DMD, and focused onto two chargecoupled devices (CCDs, IMPERX 1620M) by the two quartz lens. In this experiment, the CCDs are utilized as bucket detectors, which only just measure the total intensity. A narrowband filter with a central wavelength of 640 nm and a full-width at half-maximum (FWHM) of 10 nm and another narrowband filter with a central wavelength of 810 nm and a FWHM of 10 nm are positioned in front of the CCDs. As mentioned above, the accuracy of the dual wavelength method is crucially dependent on the selection of the two wavelengths. For this proof-of-principle experiment, the wavelengths 640 and 810 nm 289

X.-R. Yao et al.

Optics Communications 410 (2018) 287–291

Fig. 5. Reconstructions of temperature distribution of the target for various numbers of measurements: (a) 𝑚 = 7500, sampling rate=11.4%; (b) 𝑚 = 5000, sampling rate=7.6%; and (c) 𝑚 = 2500, sampling rate=3.8%.

several attenuation filters were placed after the filament to reduce the incident light, and as shown in Fig. 7(b), the image on CCD 1 vanished. Then 1800 images of the filament were recorded by CCD 1, and the average result of these images is shown in Fig. 7(c), the image of the filament still cannot be distinguished. In the second part, we encoded 1800 −1/1 binary patterns of 64 × 64 pixels onto the DMD. In 1 direction, as the light emerging from the filament was focused by the collecting lens onto a small area of the CCD camera, this focused spot recorded on CCD 2 was remained after attenuating (Fig. 7(d)). A part of the total intensities recorded by CCD 2 are shown in Fig. 7(e), and we utilized these feeble intensity fluctuations to reconstruct the image of the filament. The image retrieved from the CS algorithm is shown in Fig. 7(f), the filament is resolved. From this comparative experiment, our results indicate that with the same detector, the imaging system sensitivity can be promoted though high throughput detection. It is common in two-color temperature imaging systems to use narrowbanded filters to band limit the incident light, and the weak radiation detecting after the filters is an obstacle. This high sensitivity detecting configuration offers a very effective approach to promote the detection efficiency of two-color temperature imaging systems.

Fig. 6. Schematic of high throughput imaging. Compared to the experimental apparatuses shown in Fig. 1, collecting lens 1 is replaced with an imaging lens, and Filter 1 and 2 are the same narrowband dielectric filters centered at 640 nm.

detectors with fast timing, such as avalanche photodiodes (APDs), to achieve real-time thermal videos. A experiment has been performed to show the detection efficiency improvement benefits from the high-throughput detection. As shown in Fig. 6, compared to the experimental apparatuses shown in Fig. 1, collecting Lens 1 was replaced with an imaging lens, and the same narrowband filters with a central wavelength of 640 nm were inserted before the two CCDs in this experiment. In the first part, we constructed a conventional imaging system as reference. We fixed all the mirrors on the DMD tilted into the 0 direction and all the light were reflected into the imaging system. The image recorded form CCD 1 with a shutter of 50ms is shown in Fig. 7(a). To simulate a ultra weak light condition,

4. Conclusion We developed a double-wavelength temperature distribution imaging configuration that uses single-pixel imaging techniques to produce images simultaneously at two wavelengths. A BS is not required in this system, and the optical reflected structure of the DMD provides perfect pixel registration between the two wavelengths images. The bucket detector simultaneously measures many pixels, which results in an improvement of detection efficiency. Moreover, in this method, it is easy to change wavelengths. A thermal image recovered from undersampling measurements was obtained in the experiment to demonstrate

Fig. 7. (a) The image recorded form CCD 1. (b) The image recorded form CCD 1 after attenuating. (c) The average result of 1800 images recorded by CCD 1. (d) The image recorded by CCD 2 after attenuating. (e) A part of total intensity values recorded from CCD 2. (f) The image retrieved from the CS algorithm.

290

X.-R. Yao et al.

Optics Communications 410 (2018) 287–291

the feasibility of this structure. Additional improvements could be made by using imaging sensors whose response extends into the near-IR. Concurrent single-pixel imaging structures, in which mirrors of the DMD are divided into several groups and these groups work concurrently, can be employed to increase the frame rates of this thermal camera.

[7] M.D. Tarasov, A.I. Tolshmyakov, F.O. Kuznetsov, O.N. Petrushin, V.S. Petushkov, . Yu. A. Savel’Ev, M.Y. Tarakanov, V.A. Til’Kunov, Proc. SPIE 2869 (1997) 894–899. [8] J.E. Craig, D. Lee, R.A. Parker, F.S. Biancaniello, S.D. Ridder, Proceedings of the 18th ASM Heat Treating Society Conference, ASM International, 1999. [9] J.M. Densmore, B.E. Homan, M.M. Biss, K.L. McNesby, Appl. Opt. 50 (2011) 6267– 6271. [10] Hiroyuki. Usui, Kenji. Mitsui, 27th International Congress on High-Speed Photography and Photonics, in: Proc. of SPIE, vol. 6279, 2007, p. 627918. [11] A. Hijazia, V. Madhavanb, Multispectral image acquisition and processing, Proc. SPIE 7494 (2009) 749403. [12] Y. Shang, X. Ye, L. Cao, P. Song, J. Feng, Sci. Rep. 6 (2016) 25993. [13] M.F. Duarte, M.A. Davenport, D. Takhar, J.N. Laska, T. Sun, K.F. Kelly, R.G. Baraniuk, IEEE Signal Process. Mag. 25 (2) (2008) 83–91. [14] J. Romberg, IEEE Signal Process. Mag. 25 (2008) 14–20. [15] G.A. Howland, P.B. Dixon, J.C. Howell, Appl. Opt. 50 (2011) 5917–5920. [16] P. Clemente, V. Duràn, E. Tajahuerce, V. Torres-Company, J. Lancis, Phys. Rev. A 86 (2012) 041803. [17] J. Du, W.L. Gong, S.S. Han, Opt. Lett. 37 (2012) 1067–1069. [18] W.K. Yu, X.F. Liu, X.R. Yao, C. Wang, Y. Zhai, G.J. Zhai, Sci. Rep. 4 (2014) 5834. [19] N. Radwell, K.J. Mitchell, G.M. Gibson, M.P. Edgar, R. Bowan, M.J. Padgett, Optica 1 (2014) 285–289. [20] M.P. Edgar, G.M. Gibson, R.W. Bowman, B. Sun, N. Radwell, K.J. Mitchell, S.S. Welsh, M.J. Padgett, Sci. Rep. 5 (2015) 10669. [21] R.M. Lan, X.F. Liu, X.R. Yao, W.K. Yu, G.J. Zhai, Opt. Commun. 366 (2016) 349–353. [22] D.L. Donoho, IEEE trans, Inf. Theory 52 (2006) 1289–1306. [23] E.J. Candès, M.B. Wakin, IEEE Signal Process. Mag. 25 (2) (2008) 21–30. [24] C.Y. Pati, R. Rezaiifar, P.S. Krishnaprasad, Proceedings of the 27th Asilomar Conference, Pacific Grove, California, 1993, Vol. 1, IEEE, 1993, pp. 40–44. [25] I. Daubechies, M. Defrise, C. De Mol, Commun. Pur. Appl. Math. 57 (2004) 1413– 1457. [26] C.B. Li, Masters of Science Thesis, Rice University, 2010.

Funding information National Natural Science Foundation of China (61605218, 61601442), National Defense Science and Technology Innovation Foundation of Chinese Academy of Sciences (CXJJ-16S047), National Major Scientific Instruments Development Project of China (2013YQ030595) and the Shandong Province High College Technology Project of China (J14LN07). References [1] T.R. Harrison, Radiation Pyrometry and its Underlying Principles of Radiant Heat Transfer, Wiley, 1960. [2] H.J. Kostkowski, R.D. Lee, in: NBS Monograph, vol. 41, US Department of Commerce, National Bureau of Standards, 1962. [3] J.D. Fehribach, R.B. Johnson, Opt. Eng. 28 (12) (1989) 1255–1259. [4] B. Kateb, V. Yamamoto, C. Yu, W. Grundfest, J. Gruen, NeuroImage 47 (2009) T154– T162. [5] Z.Q. Li, J.L. Tang, X.c. Han, M. Zhang, SPIE Fiber Optic Laser Sensors 2070 (1993) 508–513. [6] D. Bizzak, M. Chyu, Rev. Sci. Instr. 65 (1994) 102–107.

291