- Email: [email protected]
Rapid quantitative interferometric microscopy using fast Fourier transform and differential–integral based phase retrieval algorithm (FFT-DI-PRA)
Optics Communications 456 (2020) 124613
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Rapid quantitative interferometric microscopy using fast Fourier transform and differential–integral based phase retrieval algorithm (FFT-DI-PRA) Qi Wei a , Mingyuan Zhang a , Miao Yu a , Liang Xue b , Cheng Liu a,c , Javier Vargas d , Fei Liu e , Shouyu Wang a,e ,∗ a
Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu 214122, China College of Electronics and Information Engineering, Shanghai University of Electric Power, Shanghai 200090, China c Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China d Faculty of Medicine, Department of Anatomy and Cell Biology, McGill University, Montreal, Quebec H3A0C7, Canada e Single Molecule Nanometry Laboratory (Sinmolab), Nanjing Agricultural University, Nanjing, Jiangsu, 210095, China b
ARTICLE
INFO
Keywords: Quantitative interferometric microscopy Rapid phase retrieval Cell phase imaging
ABSTRACT Phase retrieval in quantitative interferometric microscopy is a time-consuming and computationally expensive process, thus limiting the applications on cell dynamic detections. Here, we propose Fast Fourier Transform and Differential–Integral based Phase Retrieval Algorithm (FFT-DI-PRA) focusing on quantitative cell phase imaging to significantly accelerate the phase retrieval process. The proposed method first obtains the wrapped phase from a single-shot interferogram using the FFT based phase extraction method; then, its phase derivatives are computed along two orthogonal directions; finally, the unwrapped specimen phase is reconstructed by phase integration from corrected phase derivatives. Since the FFT based phase extraction method can effectively suppress the noise, the phase wraps can be easily recognized in phase derivatives avoiding the noise influences, thus supporting the rapid and accurate phase wrapping. As a perfect combination of the FFT based phase extraction and the differential–integral based phase wrapping, FFT-DI-PRA only takes ∼0.03 second for phase imaging from one single-shot interferogram of 512 × 512 pixels using a typical desktop; moreover, it provides high-accurate specimen phase distributions as shown here by both simulations and experiments, suggesting that FFT-DI-PRA is an effective tool used in rapid quantitative live cell phase imaging and display.
1. Introduction Phase imaging can provide high-contrast specimen details, especially for label free cells. Phase contrast and differential interference contrast microscopy are widely used phase imaging techniques [1], however, most of them only provide qualitative images. In order to expand the application scope of these optical microscopy techniques, different quantitative phase microscopy methods have been proposed [2], including approaches such as transport of intensity phase microscopy [3–10] and ptychography [11–16] among others. Though transport of intensity phase microscopy only needs simple operations, and ptychography can reach extremely high resolution and large field of view, these techniques require multiple captures for phase retrieval, then, they are not well suited for real time live cell observations. Different from these approaches, quantitative interferometric microscopy [17] can extract specimen phase distribution even from a single-shot interferogram, thus this approach represents an appropriate tool for quantitative cell imaging and analysis as well as display. Based on quantitative interferometric microscopy, various applications
have been reported: Popescu group [18,19] and Shaked group [20,21] provided cell screening methods; Park group [22,23] and Ito group [24] reported various three-dimensional imaging tools; Wang group designed quantitative interferometric microscopic cytometers [25–28]; and Ferraro group proposed phase imaging flow cytometry [29,30] as well as noise reduction display techniques [31,32]. However, the specimen phase can only be extracted from fringes patterns through phase extraction and unwrapping methods. Various single-shot phase extraction methods have been proposed [33–40], such as fast Fourier transform (FFT) based method [33,34] and Hilbert transform based method [35–37], and most of them have ultra-fast processing capabilities. Unfortunately, phase unwrapping is still a time-consuming and computationally expensive process required to reconstruct the continuous specimen phase from the wrapped phase provided by the phase extraction approach [41–43]. In order to further accelerate the phase retrieval speed, fast unwrapping or unwrapping free approaches have been proposed [44–54]. However, most of these methods require multiple interferograms/wavelengths with complicated experimental
∗ Corresponding author at: Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu 214122, China. E-mail address: [email protected] (S. Wang).
https://doi.org/10.1016/j.optcom.2019.124613 Received 19 June 2019; Received in revised form 13 September 2019; Accepted 22 September 2019 Available online 24 September 2019 0030-4018/© 2019 Elsevier B.V. All rights reserved.
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613
Fig. 1. Flowchart of FFT-DI-PRA. Step 1: Extract the wrapped phase from the single-shot interferogram using FFT based method according to Eqs. (1)–(6). Step 2: Compute the phase derivatives along two orthogonal directions from the wrapped phase according to Eqs. (7) and (8). Step 3: Compensate the rather large phase derivative values according to Eq. (9). Step 4: Reconstruct the unwrapped phase from phase discontinuity compensated phase derivatives using 2-D phase integration according to Eq. (10). Step 5: Remove the background in the specimen phase distribution.
setups [44–46] or GPU processing relying on extra hardware and complicated programming [47,48]; some of them accelerate the processing speed by pixel reduction [49,50]; besides, they often suffer from different restrictions, such as the phase discontinuities should not appear within the specimen region [51]. In this Letter, we propose an approach based on the Fast Fourier Transform and Differential–Integral based Phase Retrieval Algorithm (FFT-DI-PRA), which can obtain the specimen absolute phase from a single-shot interferogram extremely fast. FFT-DI-PRA first obtains the wrapped phase from a single-shot interferogram using the FFT based phase extraction method; then, its phase derivatives are computed along two orthogonal directions; finally, the unwrapped specimen phase is reconstructed by phase integration from corrected phase derivatives. As a perfect combination of the FFT based phase extraction and DI based phase wrapping, FFT based phase extraction method can effectively suppress the noise, the phase wraps can be easily recognized in phase derivatives avoiding the noise influences, supporting the rapid and accurate phase wrapping. Moreover, even if the phase discontinuities occur within the specimen region, FFT-DI-PRA still works well. Considering FFT-DI-PRA can retrieve the specimen phase at fast speed and high accuracy, it is an ideal tool for rapid quantitative phase imaging and display [18–32].
𝑏(𝑥, 𝑦)𝑒𝑥𝑝(−𝑖𝜑(𝑥, 𝑦)). 𝐼(𝑥, 𝑦) = 𝑎(𝑥, 𝑦) + 𝑏(𝑥, 𝑦) exp(𝑖𝜑(𝑥, 𝑦)) exp[𝑖(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦)] +𝑏(𝑥, 𝑦) exp(−𝑖𝜑(𝑥, 𝑦)) exp[−𝑖(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦)] 𝐼(𝑥, 𝑦) = 𝑎(𝑥, 𝑦) + 𝑐(𝑥, 𝑦) exp[𝑖(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦)] + 𝑐 ′ (𝑥, 𝑦) exp[−𝑖(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦)]
(3)
When using FFT based phase extraction method, FFT (F ) is functioned on 𝐼(𝑥, 𝑦) as shown in Eq. (4), in which A(𝜉, 𝜂), C(𝜉 − 𝑓𝑥 , 𝜂 − 𝑓𝑦 ) and 𝐶 ′ (𝜉 + 𝑓𝑥 , 𝜂 + 𝑓𝑦 ) are the FFT terms corresponding to those in the right side of Eq. (3). F [𝐼(𝑥, 𝑦)] = 𝐴(𝜉, 𝜂) + 𝐶(𝜉 − 𝑓𝑥 , 𝜂 − 𝑓𝑦 ) + 𝐶 ′ (𝜉 + 𝑓𝑥 , 𝜂 + 𝑓𝑦 )
(4)
Since A(𝜉,𝜂), C(𝜉 − 𝑓𝑥 , 𝜂 − 𝑓𝑦 ) and 𝐶 ′ (𝜉 + 𝑓𝑥 , 𝜂 + 𝑓𝑦 ) are separated in the frequency domain, C(𝜉 − 𝑓𝑥 , 𝜂 − 𝑓𝑦 ) (or 𝐶 ′ (𝜉 + 𝑓𝑥 , 𝜂 + 𝑓𝑦 )) can be simply filtered and moved into the center of the frequency domain. Therefore, 𝑐(𝑥, 𝑦) can be computed using inversed FFT (F −1 ) as shown in Eq. (5). 𝑐(𝑥, 𝑦) = F −1 [𝐶(𝜉, 𝜂)]
(5)
Finally, the specimen phase can be extracted according to Eq. (6), in which Im and Re are imaginary and real parts. 𝜑′ (𝑥, 𝑦) = arctan
Im[𝑐(𝑥, 𝑦)] Re[𝑐(𝑥, 𝑦)]
(6)
Unfortunately, the extracted phases 𝜑′ (𝑥, 𝑦) are often wrapped as show in Fig. 2. In order to unwrap the phase, DI based phase wrapping is implemented. First, the phase derivatives 𝛥𝜑′𝑥 (𝑥, 𝑦) and 𝛥𝜑′𝑦 (𝑥, 𝑦) along two orthogonal directions are computed from the wrapped phase (Step 2 and Eqs. (7)–(8)).
2. Principle The flowchart of FFT-DI-PRA is shown in Fig. 1. First, after singleshot interferogram recording, its wrapped phase is extracted using FFT based phase extraction method (Step 1 and Eqs. (1)–(6)) [33,34]. The intensity distribution of the interferogram 𝐼(𝑥, 𝑦) can be described by Eq. (1), in which 𝑎(𝑥, 𝑦) is the DC term, 𝑏(𝑥, 𝑦) is the amplitude of the interference, 2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦 indicates the tilting phase introduced by off-axis interference, and 𝜑(x,y) is the specimen phase distribution. 𝐼(𝑥, 𝑦) = 𝑎(𝑥, 𝑦) + 𝑏(𝑥, 𝑦) cos[2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦 + 𝜑(𝑥, 𝑦)]
(2)
𝛥𝜑𝑥 ′ (𝑥, 𝑦) = 𝜑′ (𝑥 + 1, 𝑦) − 𝜑′ (𝑥, 𝑦) ′
′
′
𝛥𝜑𝑦 (𝑥, 𝑦) = 𝜑 (𝑥, 𝑦 + 1) − 𝜑 (𝑥, 𝑦)
(7) (8)
It is known that FFT based phase extraction method can effectively suppress the noise, moreover, as this method mainly focuses on the cells, there are often no large phase changes in specimens, therefore, in the obtained phase derivatives, the rather high values (close to 2𝜋 or −2𝜋) are mostly caused by the phase discontinuities. Then, these
(1)
Eq. (1) can be derived into Eq. (2) as an exponential form, and then rewritten into Eq. (3), in which 𝑐(𝑥, 𝑦) = 𝑏(𝑥, 𝑦)𝑒𝑥𝑝(𝑖𝜑(𝑥, 𝑦)) and 𝑐 ′ (𝑥, 𝑦) = 2
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613
Fig. 2. FFT-DI-PRA verification using numerical simulation. (A) Setting specimen phase; (B) off-axis interferogram; (C) first order information selection in the frequency domain; (D) wrapped phase; (E) and (F) phase derivatives along the horizontal and vertical axes; (G) and (H) corrected phase derivatives; (I) unwrapped phase; (J) FFT-DI-PRA retrieved specimen phase; (K) traditional method retrieved specimen phase; (L) comparisons on central sectional phases in (A), (J) and (K). The black arrows in (E)–(H) indicate the differentiation directions. The unit of the color bar is rad and the white bar in (A) indicates 10 μm. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
rather large phase derivative values can be compensated according to Eq. (9) as Step 3 in Fig. 1.
𝛥𝜑𝑥∕𝑦
′
⎧𝛥𝜑 ′ (𝑥, 𝑦) − 2𝜋 ⎪ 𝑥∕𝑦 (𝑥, 𝑦) = ⎨𝛥𝜑𝑥∕𝑦 ′ (𝑥, 𝑦) + 2𝜋 ⎪𝛥𝜑𝑥∕𝑦 ′ (𝑥, 𝑦) ⎩
𝛥𝜑𝑥∕𝑦 ′ (𝑥, 𝑦) ∼ 2𝜋 𝛥𝜑𝑥∕𝑦 ′ (𝑥, 𝑦) ∼ −2𝜋 𝑜𝑡ℎ𝑒𝑟𝑠
therefore, FFT-DI-PRA can determine and compensate the relative large phase derivatives mostly due to the phase discontinuities. Additionally, the reported rapid phase retrieval methods including both phase extraction and unwrapping methods [33–54] are often proposed separately. Different methods have their own features, such as FFT based phase extraction method can effectively suppress the noise but sacrifices the specimen details [33,34], while Hilbert transform and derivative based phase extraction method have high precision but still keep the noise in the interferograms [35–37]; simplified network programming based phase unwrapping method can robustly deal with the low signal to noise ratio condition but suffer from long time processing [43], and pixel shifted phase unwrapping method is fast but easily influenced by noise [54]. Differently, combining with the advantage of the noise suppression of the FFT based phase extraction method and the fast processing speed of the DI based phase unwrapping method, we propose the FFT-DI-PRA which provides the whole procedures from fringes to continuous sample phases. Moreover, since the FFT based phase extraction method can suppress the noise, thus the phase discontinuity can be easily recognized in phase derivative distribution, obviously simplifying the phase unwrapping procedures. As the perfect combination of the FFT based phase extraction method and the DI based phase unwrapping method, FFT-DI-PRA can retrieve the specimen phase in rather high accuracy and fast speed. In the following section, both simulations and experiments are proposed to verify FFT-DI-PRA.
(9)
Next, the unwrapped phase after phase discontinuity compensation can be reconstructed from the phase derivatives using 2-D phase integration (Step 4 and Eq. (10)). 𝜑(𝑥, 𝑦) =
∬
𝛥𝜑𝑥 ′ (𝑥, 𝑦)𝑑𝑥 + 𝛥𝜑𝑦 ′ (𝑥, 𝑦)𝑑𝑦
(10)
Since the existed background phase influences the target observations and analysis, the specimen phase can be finally extracted by removing the background (Step 5) [55–57]. Here, our previously proposed background removing technique is adopted [57]. First, the sample corresponding pixels can be recognized according to the phase values, and the background pixels can be determined. Then, the background phase distribution composed of different orders of the Zernike polynomials can be fitted from the determined background pixels. Finally, the sample phase can be retrieved by subtracting the fitted background phase distribution from the unwrapped phase. It is noted that Steps 2 to 4 are actually simplified network programming based phase unwrapping method [43], as the proposed approach skips many steps in classical network programming algorithm. The skipped steps are helpful to distinguish whether the relatively large values in derivative phase distribution are caused by phase discontinuities, large phase changes or noises. However, when using FFT based phase extraction method, the noise can be significantly reduced; moreover, most of cells often do not have relatively large phase gradient,
3. Simulations and experiments In order to test the FFT-DI-PRA performance, numerical simulations were first implemented. The specimens are standard red blood cell 3
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613
Fig. 3. FFT-DI-PRA verification using experiments. (A) Optical system, L: He–Ne laser, BS: beam splitting prism, BE: micro-objective-pinhole-lens based beam expander, MO: micro-objective, S: RBC smear, CCD: CCD camera; (B) captured off-axis interferogram; (C) first order information selection in the frequency domain; (D) wrapped phase; (E) and (F) phase derivatives along the horizontal and vertical axes; (G) and (H) corrected phase derivatives; (I) unwrapped phase; (J) FFT-DI-PRA retrieved RBC phase; (K) traditional method retrieved RBC phase; subplot between (J) and (K) shows the comparisons on central sectional phases in (J) and (K). The black arrows in (E)–(H) indicate the differentiation directions. The unit of the color bar is rad and the white bar in (J) indicates 10 μm. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
(RBC) models [58] with the special biconcave configuration shown in Fig. 2(A). Fig. 2(B) is the numerically generated off-axis interferogram with a signal to noise ratio of 20 dB, which is close to experiments. By selecting the first order information in the frequency domain shown in Fig. 2(C) according to the FFT based phase extraction method, the wrapped phase is computed and shown in Fig. 2(D). Fig. 2(E) and (F) are the phase derivatives along the horizontal and vertical axes computed from Fig. 2(D). After correcting the phase by compensating the extremely high phase derivatives caused by the phase discontinuities shown in Fig. 2(G) and (H), the unwrapped phase is reconstructed in Fig. 2(I) using 2-D integration. Finally, the specimen phase is retrieved by removing the background. This specimen phase is shown in Fig. 2(J). The reconstructed biconcave structure with close
phase values compared to setting ones prove that FFT-DI-PRA can successfully retrieve the quantitative cell phase distribution even the phase discontinuities occur within the specimen region. Additionally, Fig. 2(K) shows the retrieved specimen phase from Fig. 2(B) using the classical phase retrieval approach (FFT based phase extraction method [33,34] and network programming based phase unwrapping method [43]). In order to quantitatively evaluate the accuracy of FFTDI-PRA, Fig. 2(L) lists the comparisons on central sectional phases in Fig. 2(A), (J) and (K), indicating that both the retrieved phases fit well with the actual one. Besides, the correlation coefficient between phases in Fig. 2(A) and (J) is 0.9891, and that between phases in Fig. 2(A) and (K) is 0.9906, both close to 1. The coincident central sectional phases 4
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613
Fig. 4. Time consuming of FFT-DI-PRA and traditional method with respect to interferogram pixel size.
pixel reduction. The extremely fast phase retrieval speed indicate that FFT-DI-PRA is a potential tool for real-time quantitative live cell phase imaging and display.
and high correlation coefficients prove that FFT-DI-PRA can provide specimen phases with high accuracy. Finally, FFT-DI-PRA was tested with experimental data using a microscopic system based on a Mach–Zehnder interferometer shown in Fig. 3(A). The optical system was composed by a He–Ne laser (Thorlabs, US) with a wavelength of 632.8 nm, a micro-objective (Daheng Optics, China) for specimen magnification and a CCD camera (AVT Pike 421B, Germany) with the pixel size of 7.4 μm. Fig. 3(B) is the captured offaxis interferogram when the RBC smear was used as the specimen. Using the FFT based method, the wrapped phase was first extracted as Fig. 3(D). Then, the phase derivatives along the horizontal and vertical axes were computed and are shown in Fig. 3(E) and (F), respectively. After phase correction was done by phase discontinuity compensation and the respective phases are shown in Fig. 3(G) and (H). Afterwards, the unwrapped phase was reconstructed (Fig. 3(I)) using 2-D phase integration. Finally, the specimen phase was retrieved by removing background and it is shown in Fig. 3(J). The retrieved RBC phases have the special biconcave configuration, which is similar to the RBC model in Fig. 2(A). Moreover, the retrieved phase by FFT-DIPRA was also compared with that in Fig. 3(K) obtained from classical method (FFT based phase extraction method [33,34] and network programming based phase unwrapping method [43]), the coincident central sectional phases proving the proposed method could achieve rather high accuracy. Since the time-consuming and massive-computational phase unwrapping is simplified, the phase retrieval operation is significantly accelerated. To demonstrate the efficiency, a desktop computer equipped with an Intel Core i5 CPU and a 16 GB RAM was used for phase retrieval, in addition, the FFT-DI-PRA was implemented in MATLAB 2019a. The obtained processing times when using the proposed FFTDI-PRA approach and the traditional phase retrieval method combining FFT based phase extraction and network programming based phase unwrapping are listed in Fig. 4. According to the linearly fitting results, FFT-DI-PRA requires ∼ 9.95×10−8 s/pixel in phase retrieval, while traditional method required ∼ 2.35×10−5 s/pixel. Especially, for the interferogram with 512 × 512 pixels, only 0.0317 s is required for specimen phase retrieval using FFT-DI-PRA, 216 times faster than traditional method relying on the classical network programming algorithm [43]. In other words, FFT-DI-PRA can reach the frame rate of 30 fps in quantitative phase imaging without using GPU parallel computation or
4. Conclusion In conclusion, FFT-DI-PRA which composes of FFT based phase extraction method and simplified network programming based phase unwrapping method is designed for rapid quantitative live cell phase imaging and display. Since the noise can be significantly reduced when using the FFT based phase extraction method, and most of cells often do not have relatively large phase gradient, therefore, the relative large phase derivatives mostly caused by the phase discontinuities can be determined and compensated though simplified network programming based phase unwrapping method. FFT-DI-PRA only requires one singleshot interferogram for phase retrieval, and for an interferogram of 512 × 512 pixels, it only needs ∼0.03 s using a typical desktop, supporting the frame rate of 30 fps in quantitative phase imaging without using GPU parallel computation or pixel reduction. Additionally, proved by simulation and experiments, FFT-DI-PRA can reach rather high accuracy in quantitative phase imaging. Considering its high accuracy and extremely fast speed, it is believed FFT-DI-PRA can be potentially adopted for real-time live cell observations and measurements. Acknowledgments Authors thank Dr. Lingyu Ai in Jiangnan University for providing the RBC smear. The work is supported National Natural Science Foundation of China (61705092, U1730132, 31870154) and Natural Science Foundation of Jiangsu Province of China (BK20170194, BE2018709). References [1] D.J. Stephens, V.J. Allan, Light microscopy techniques for live cell imaging, Science 300 (2003) 82–86. [2] Y. Park, C. Depeursinge, G. Popescu, Quantitative phase imaging in biomedicine, Nat. Photonics 12 (2018) 578–589. [3] M. Teague, Deterministic phase retrieval: a Green’s function solution, J. Opt. Soc. Amer. 73 (1983) 1434–1441. 5
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613 [31] P. Memmolo, V. Bianco, M. Paturzo, B. Javidi, P.A. Netti, P. Ferraro, Encoding multiple holograms for speckle-noise reduction in optical display, Opt. Express 22 (2014) 25768–25775. [32] V. Bianco, P. Memmolo, M. Paturzo, A. Finizio, B. Javidi, P. Ferraro, Quasi noise-free digital holography, Light-Sci. Appl. 5 (2016) e16142. [33] M. Takeda, H. Ina, S. Kobayashi, Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry, J. Opt. Soc. Amer. 72 (1982) 156–160. [34] G. Popescu, T. Ikeda, R. Dasari, M. Feld, Diffraction phase microscopy for quantifying cell structure and dynamics, Opt. Lett. 31 (2006) 775–777. [35] T. Ikeda, G. Popescu, R.R. Dasari, M.S. Feld, Hilbert phase microscopy for investigating fast dynamics in transparent systems, Opt. Lett. 30 (2005) 1165–1167. [36] S. Wang, K. Yan, L. Xue, Quantitative interferometric microscopy with two dimensional Hilbert transform based phase retrieval method, Opt. Commun. 383 (2017) 537–544. [37] S. Wang, K. Yan, N. Sun, X. Tian, W. Yu, Y. Li, A. Sun, L. Xue, Unwrapping free Hilbert transform based phase retrieval method in quantitative interferometric microscopy, Opt. Eng. 55 (2016) 113105. [38] B. Bhaduri, G. Popescu, Derivative method for phase retrieval in off-axis quantitative phase imaging, Opt. Lett. 37 (2012) 1868–1870. [39] S.K. Debnath, Y.K. Park, Real-time quantitative phase imaging with a spatial phase-shifting algorithm, Opt. Lett. 36 (2011) 4677–4679. [40] Q. Wei, Y. Li, J. Vargas, J. Wang, Q. Gong, Y. Kong, Z. Jiang, L. Xue, C. Liu, F. Liu, S. Wang, Principle component analysis based quantitative differential interference contrast microscopy, Opt. Lett. 44 (2019) 45–48. [41] D.C. Ghiglia, L.A. Romero, Robust two-dimensional weighted and unweighted phase unwrapping that uses fast transforms and iterative methods, J. Opt. Soc. Am. A 11 (1994) 107–117. [42] R.M. Goldstein, H.A. Zebker, C.L. Werner, Satellite radar interferometry: Two-dimensional phase unwrapping, Radio Sci. 23 (1988) 713–720. [43] Mario Costantini, A novel phase unwrapping method based on network programming, IEEE Trans. Geosci. Remote Sens. 36 (1998) 813–821. [44] J. Gass, A. Dakoff, M.K. Kim, Phase imaging without 2𝜋 ambiguity by multiwavelength digital holography, Opt. Lett. 28 (2003) 1141–1143. [45] N.A. Turko, N.T. Shaked, Simultaneous two-wavelength phase unwrapping using an external module for multiplexing off-axis holography, Opt. Lett. 42 (2017) 73–76. [46] N.A. Turko, P.J. Eravuchira, I. Barnea, N.T. Shaked, Simultaneous threewavelength unwrapping using external digital holographic multiplexing module, Opt. Lett. 43 (2018) 1943–1946. [47] T. Wang, L. Kai, Q. Kemao, Real-time reference-based dynamic phase retrieval algorithm for optical measurement, Appl. Opt. 56 (2017) 7726–7733. [48] O. Backoach, S. Kariv, P. Girshovitz, N.T. Shaked, Fast phase processing in offaxis holography by CUDA including parallel phase unwrapping, Opt. Express 24 (2016) 3177–3188. [49] P. Girshovitz, N.T. Shaked, Fast phase processing in off-axis holography using multiplexing with complex encoding and live-cell fluctuation map calculation in real-time, Opt. Express 23 (2015) 8773–8787. [50] P. Girshovitz, N.T. Shaked, Real-time quantitative phase reconstruction in off-axis digital holography using multiplexing, Opt. Lett. 39 (2014) 2262–2265. [51] H.V. Pham, C. Edwards, L.L. Goddard, G. Popescu, Fast phase reconstruction in white light diffraction phase microscopy, Appl. Opt. 52 (2013) A97–A101. [52] S. Wang, K. Yan, L. Xue, High speed moire based phase retrieval method for quantitative phase imaging of thin objects without phase unwrapping or aberration compensation, Opt. Commun. 359 (2016) 272–278. [53] M.A. Herráez, D.R. Burton, M.J. Lalor, M.A. Gdeisat, Fast two-dimensional phaseunwrapping algorithm based on sorting by reliability following a noncontinuous path, Appl. Opt. 41 (2002) 7437–7444. [54] L. Xue, S. Wang, K. Yan, N. Sun, Z. Li, F. Liu, Fast pixel shifting phase unwrapping algorithm in quantitative interferometric microscopy, Chin. Opt. Lett. 12 (2014) 071801. [55] L. Miccio, D. Alfieri, S. Grilli, P. Ferraro, Direct full compensation of the aberrations in quantitative phase microscopy of thin objects by a single digital hologram, Appl. Phys. Lett. 90 (2007) 041104. [56] K.W. Seo, Y.S. Choi, E.S. Seo, S.J. Lee, Aberration compensation for objective phase curvature in phase holographic microscopy, Opt. Lett. 37 (2012) 4976–4978. [57] L. Xue, S. Wang, K. Yan, N. Sun, Z. Li, F. Liu, Quantitative interferometric microscopy with improved full-field phase aberration compensation, Opt. Eng. 53 (2014) 113105. [58] S.V. Tsinopoulos, D. Polyzos, Scattering of He–Ne laser light by an average-sized red blood cell, Appl. Opt. 38 (1999) 5499–5510.
[4] Y. Shan, Q. Gong, J. Wang, J. Xu, Q. Wei, C. Liu, L. Xue, S. Wang, F. Liu, Measurements on ATP induced cellular fluctuations using real-time dual view transport of intensity phase microscopy, Biomed. Opt. Express 10 (2019) 2337–2354. [5] X. Meng, H. Huang, K. Yan, X. Tian, W. Yu, H. Cui, Y. Kong, L. Xue, C. Liu, S. Wang, Smartphone based hand-held quantitative phase microscope using transport of intensity equation method, Lab Chip 17 (2017) 104–109. [6] W. Yu, X. Tian, X. He, X. Song, L. Xue, C. Liu, S. Wang, Real time quantitative phase microscopy based on single-shot transport of intensity equation (ssTIE) method, Appl. Phys. Lett. 109 (2016) 071112. [7] X. Tian, W. Yu, X. Meng, A. Sun, L. Xue, C. Liu, S. Wang, Real-time quantitative phase imaging based on transport of intensity equation with dual simultaneously recorded field of view, Opt. Lett. 41 (2016) 1427–1430. [8] S.S. Kou, L. Waller, G. Barbastathis, C.J.R. Sheppard, Transport-of-intensity approach to differential interference contrast (TI-DIC) microscopy for quantitative phase imaging, Opt. Lett. 35 (2010) 447–449. [9] S. Zheng, B. Xue, W. Xue, X. Bai, F. Zhou, Transport of intensity phase imaging from multiple noisy intensities measured in unequally-spaced planes, Opt. Express 19 (2011) 972–985. [10] J. Zhong, R.A. Claus, J. Dauwels, L. Tian, L. Waller, Transport of intensity phase imaging by intensity spectrum fitting of exponentially spaced defocus planes, Opt. Express 22 (2014) 10661–10670. [11] G. Zheng, R. Horstmeyer, C. Yang, Wide-field, high-resolution Fourier ptychographic microscopy, Nat. Photonics 7 (2013) 739–745. [12] J.M. Rodenburg, H.M. Faulkner, A phase retrieval algorithm for shifting illumination, Appl. Phys. Lett. 85 (2004) 4795–4797. [13] A. Sun, X. He, Y. Kong, H. Cui, X. Song, L. Xue, S. Wang, C. Liu, Ultra-high speed digital micromirror device based ptychographic iterative engine method, Biomed. Opt. Express 8 (2017) 3155–3162. [14] L. Tian, Z. Liu, L.H. Yeh, M. Chen, J. Zhong, L. Waller, Computational illumination for high-speed in vitro Fourier ptychographic microscopy, Optica 2 (2015) 904–911. [15] S. Jiang, K. Guo, J. Liao, G. Zheng, Solving Fourier ptychographic imaging problems via neural network modeling and Tensorflow, Biomed. Opt. Express 9 (2018) 3306–3319. [16] B.K. Chen, P. Sidorenko, O. Lahav, O. Peleg, O. Cohen, Multiplexed single-shot ptychography, Opt. Lett. 43 (2018) 5379–5382. [17] M. Mir, B. Bhaduri, R. Wang, R. Zhu G. Popescu, Quantitative phase imaging, Prog. Opt. 57 (2012) 133–217. [18] M. Mir, H. Ding, Z. Wang, K. Tangella, G. Popescu, Blood screening using diffraction phase cytometry, J. Biomed. Opt. 15 (2010) 027016. [19] M. Mir, Z. Wang, K. Tangella, G. Popescu, Diffraction phase cytometry: blood on a CD-ROM, Opt. Express 17 (2009) 2579–2585. [20] P. Girshovitz, N.T. Shaked, Fast phase processing in off-axis holography using multiplexing with complex encoding and live-cell fluctuation map calculation in real-time, Opt. Express 23 (2015) 8773–8787. [21] G. Dardikman, N.T. Shaked, Review on methods of solving the refractive index– thickness coupling problem in digital holographic microscopy of biological cells, Opt. Commun. 422 (2018) 8–16. [22] K. Kim, Y.K. Park, Tomographic active optical trapping of arbitrarily shaped objects by exploiting 3D refractive index maps, Nature Commun. 8 (2017) 15340. [23] G. Kim, Y.J. Jo, H. Cho, H. Min, Y.K. Park, Learning-based screening of hematologic disorders using quantitative phase imaging of individual red blood cells, Biosens. Bioelectron. 123 (2019) 69–76. [24] T. Kakue, Y. Endo, T. Nishitsuji, T. Shimobaba, N. Masuda, T. Ito, Digital holographic high-speed 3D imaging for the vibrometry of fast-occurring phenomena, Sci. Rep. 7 (2017) 10413. [25] S. Wang, L. Xue, H. Li, J. Lai, Y. Song, Z. Li, Quantitative phase detection with expanded principal component analysis method on interferometric microscopic cytometer, Appl. Phys. B 116 (2014) 235–239. [26] L. Xue, S. Wang, K. Yan, N. Sun, P. Ferraro, Z. Li, F. Liu, Gravity driven high throughput phase detecting cytometer based on quantitative interferometric microscopy, Opt. Commun. 316 (2014) 5–9. [27] L. Xue, J. Vargas, S. Wang, Z. Li, F. Liu, Quantitative interferometric microscopy cytometer based on regularized optical flow algorithm, Opt. Commun. 350 (2015) 222–229. [28] K. Yan, L. Xue, S. Wang, Field of view scanning based quantitative interferometric microscopic cytometers for cellular imaging and analysis, Micros. Res. Tech. 81 (2018) 397–407. [29] F. Merola, P. Memmolo, L. Miccio, R. Savoia, M. Mugnano, A. Fontana, G. D’Ippolito, A. Sardo, A. Iolascon, A. Gambale, P. Ferraro, Tomographic flow cytometry by digital holography, Light-Sci. Appl. 6 (2017) e16241. [30] V. Bianco, M. Paturzo, V. Marchesano, I. Gallotta, E. Di Schiavib, P. Ferraro, Optofluidic holographic microscopy with custom field of view (FoV) using a linear array detector, Lab Chip 15 (2015) 2117–2124.
6
Contents lists available at ScienceDirect
Optics Communications journal homepage: www.elsevier.com/locate/optcom
Rapid quantitative interferometric microscopy using fast Fourier transform and differential–integral based phase retrieval algorithm (FFT-DI-PRA) Qi Wei a , Mingyuan Zhang a , Miao Yu a , Liang Xue b , Cheng Liu a,c , Javier Vargas d , Fei Liu e , Shouyu Wang a,e ,∗ a
Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu 214122, China College of Electronics and Information Engineering, Shanghai University of Electric Power, Shanghai 200090, China c Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China d Faculty of Medicine, Department of Anatomy and Cell Biology, McGill University, Montreal, Quebec H3A0C7, Canada e Single Molecule Nanometry Laboratory (Sinmolab), Nanjing Agricultural University, Nanjing, Jiangsu, 210095, China b
ARTICLE
INFO
Keywords: Quantitative interferometric microscopy Rapid phase retrieval Cell phase imaging
ABSTRACT Phase retrieval in quantitative interferometric microscopy is a time-consuming and computationally expensive process, thus limiting the applications on cell dynamic detections. Here, we propose Fast Fourier Transform and Differential–Integral based Phase Retrieval Algorithm (FFT-DI-PRA) focusing on quantitative cell phase imaging to significantly accelerate the phase retrieval process. The proposed method first obtains the wrapped phase from a single-shot interferogram using the FFT based phase extraction method; then, its phase derivatives are computed along two orthogonal directions; finally, the unwrapped specimen phase is reconstructed by phase integration from corrected phase derivatives. Since the FFT based phase extraction method can effectively suppress the noise, the phase wraps can be easily recognized in phase derivatives avoiding the noise influences, thus supporting the rapid and accurate phase wrapping. As a perfect combination of the FFT based phase extraction and the differential–integral based phase wrapping, FFT-DI-PRA only takes ∼0.03 second for phase imaging from one single-shot interferogram of 512 × 512 pixels using a typical desktop; moreover, it provides high-accurate specimen phase distributions as shown here by both simulations and experiments, suggesting that FFT-DI-PRA is an effective tool used in rapid quantitative live cell phase imaging and display.
1. Introduction Phase imaging can provide high-contrast specimen details, especially for label free cells. Phase contrast and differential interference contrast microscopy are widely used phase imaging techniques [1], however, most of them only provide qualitative images. In order to expand the application scope of these optical microscopy techniques, different quantitative phase microscopy methods have been proposed [2], including approaches such as transport of intensity phase microscopy [3–10] and ptychography [11–16] among others. Though transport of intensity phase microscopy only needs simple operations, and ptychography can reach extremely high resolution and large field of view, these techniques require multiple captures for phase retrieval, then, they are not well suited for real time live cell observations. Different from these approaches, quantitative interferometric microscopy [17] can extract specimen phase distribution even from a single-shot interferogram, thus this approach represents an appropriate tool for quantitative cell imaging and analysis as well as display. Based on quantitative interferometric microscopy, various applications
have been reported: Popescu group [18,19] and Shaked group [20,21] provided cell screening methods; Park group [22,23] and Ito group [24] reported various three-dimensional imaging tools; Wang group designed quantitative interferometric microscopic cytometers [25–28]; and Ferraro group proposed phase imaging flow cytometry [29,30] as well as noise reduction display techniques [31,32]. However, the specimen phase can only be extracted from fringes patterns through phase extraction and unwrapping methods. Various single-shot phase extraction methods have been proposed [33–40], such as fast Fourier transform (FFT) based method [33,34] and Hilbert transform based method [35–37], and most of them have ultra-fast processing capabilities. Unfortunately, phase unwrapping is still a time-consuming and computationally expensive process required to reconstruct the continuous specimen phase from the wrapped phase provided by the phase extraction approach [41–43]. In order to further accelerate the phase retrieval speed, fast unwrapping or unwrapping free approaches have been proposed [44–54]. However, most of these methods require multiple interferograms/wavelengths with complicated experimental
∗ Corresponding author at: Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu 214122, China. E-mail address: [email protected] (S. Wang).
https://doi.org/10.1016/j.optcom.2019.124613 Received 19 June 2019; Received in revised form 13 September 2019; Accepted 22 September 2019 Available online 24 September 2019 0030-4018/© 2019 Elsevier B.V. All rights reserved.
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613
Fig. 1. Flowchart of FFT-DI-PRA. Step 1: Extract the wrapped phase from the single-shot interferogram using FFT based method according to Eqs. (1)–(6). Step 2: Compute the phase derivatives along two orthogonal directions from the wrapped phase according to Eqs. (7) and (8). Step 3: Compensate the rather large phase derivative values according to Eq. (9). Step 4: Reconstruct the unwrapped phase from phase discontinuity compensated phase derivatives using 2-D phase integration according to Eq. (10). Step 5: Remove the background in the specimen phase distribution.
setups [44–46] or GPU processing relying on extra hardware and complicated programming [47,48]; some of them accelerate the processing speed by pixel reduction [49,50]; besides, they often suffer from different restrictions, such as the phase discontinuities should not appear within the specimen region [51]. In this Letter, we propose an approach based on the Fast Fourier Transform and Differential–Integral based Phase Retrieval Algorithm (FFT-DI-PRA), which can obtain the specimen absolute phase from a single-shot interferogram extremely fast. FFT-DI-PRA first obtains the wrapped phase from a single-shot interferogram using the FFT based phase extraction method; then, its phase derivatives are computed along two orthogonal directions; finally, the unwrapped specimen phase is reconstructed by phase integration from corrected phase derivatives. As a perfect combination of the FFT based phase extraction and DI based phase wrapping, FFT based phase extraction method can effectively suppress the noise, the phase wraps can be easily recognized in phase derivatives avoiding the noise influences, supporting the rapid and accurate phase wrapping. Moreover, even if the phase discontinuities occur within the specimen region, FFT-DI-PRA still works well. Considering FFT-DI-PRA can retrieve the specimen phase at fast speed and high accuracy, it is an ideal tool for rapid quantitative phase imaging and display [18–32].
𝑏(𝑥, 𝑦)𝑒𝑥𝑝(−𝑖𝜑(𝑥, 𝑦)). 𝐼(𝑥, 𝑦) = 𝑎(𝑥, 𝑦) + 𝑏(𝑥, 𝑦) exp(𝑖𝜑(𝑥, 𝑦)) exp[𝑖(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦)] +𝑏(𝑥, 𝑦) exp(−𝑖𝜑(𝑥, 𝑦)) exp[−𝑖(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦)] 𝐼(𝑥, 𝑦) = 𝑎(𝑥, 𝑦) + 𝑐(𝑥, 𝑦) exp[𝑖(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦)] + 𝑐 ′ (𝑥, 𝑦) exp[−𝑖(2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦)]
(3)
When using FFT based phase extraction method, FFT (F ) is functioned on 𝐼(𝑥, 𝑦) as shown in Eq. (4), in which A(𝜉, 𝜂), C(𝜉 − 𝑓𝑥 , 𝜂 − 𝑓𝑦 ) and 𝐶 ′ (𝜉 + 𝑓𝑥 , 𝜂 + 𝑓𝑦 ) are the FFT terms corresponding to those in the right side of Eq. (3). F [𝐼(𝑥, 𝑦)] = 𝐴(𝜉, 𝜂) + 𝐶(𝜉 − 𝑓𝑥 , 𝜂 − 𝑓𝑦 ) + 𝐶 ′ (𝜉 + 𝑓𝑥 , 𝜂 + 𝑓𝑦 )
(4)
Since A(𝜉,𝜂), C(𝜉 − 𝑓𝑥 , 𝜂 − 𝑓𝑦 ) and 𝐶 ′ (𝜉 + 𝑓𝑥 , 𝜂 + 𝑓𝑦 ) are separated in the frequency domain, C(𝜉 − 𝑓𝑥 , 𝜂 − 𝑓𝑦 ) (or 𝐶 ′ (𝜉 + 𝑓𝑥 , 𝜂 + 𝑓𝑦 )) can be simply filtered and moved into the center of the frequency domain. Therefore, 𝑐(𝑥, 𝑦) can be computed using inversed FFT (F −1 ) as shown in Eq. (5). 𝑐(𝑥, 𝑦) = F −1 [𝐶(𝜉, 𝜂)]
(5)
Finally, the specimen phase can be extracted according to Eq. (6), in which Im and Re are imaginary and real parts. 𝜑′ (𝑥, 𝑦) = arctan
Im[𝑐(𝑥, 𝑦)] Re[𝑐(𝑥, 𝑦)]
(6)
Unfortunately, the extracted phases 𝜑′ (𝑥, 𝑦) are often wrapped as show in Fig. 2. In order to unwrap the phase, DI based phase wrapping is implemented. First, the phase derivatives 𝛥𝜑′𝑥 (𝑥, 𝑦) and 𝛥𝜑′𝑦 (𝑥, 𝑦) along two orthogonal directions are computed from the wrapped phase (Step 2 and Eqs. (7)–(8)).
2. Principle The flowchart of FFT-DI-PRA is shown in Fig. 1. First, after singleshot interferogram recording, its wrapped phase is extracted using FFT based phase extraction method (Step 1 and Eqs. (1)–(6)) [33,34]. The intensity distribution of the interferogram 𝐼(𝑥, 𝑦) can be described by Eq. (1), in which 𝑎(𝑥, 𝑦) is the DC term, 𝑏(𝑥, 𝑦) is the amplitude of the interference, 2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦 indicates the tilting phase introduced by off-axis interference, and 𝜑(x,y) is the specimen phase distribution. 𝐼(𝑥, 𝑦) = 𝑎(𝑥, 𝑦) + 𝑏(𝑥, 𝑦) cos[2𝜋𝑓𝑥 𝑥 + 2𝜋𝑓𝑦 𝑦 + 𝜑(𝑥, 𝑦)]
(2)
𝛥𝜑𝑥 ′ (𝑥, 𝑦) = 𝜑′ (𝑥 + 1, 𝑦) − 𝜑′ (𝑥, 𝑦) ′
′
′
𝛥𝜑𝑦 (𝑥, 𝑦) = 𝜑 (𝑥, 𝑦 + 1) − 𝜑 (𝑥, 𝑦)
(7) (8)
It is known that FFT based phase extraction method can effectively suppress the noise, moreover, as this method mainly focuses on the cells, there are often no large phase changes in specimens, therefore, in the obtained phase derivatives, the rather high values (close to 2𝜋 or −2𝜋) are mostly caused by the phase discontinuities. Then, these
(1)
Eq. (1) can be derived into Eq. (2) as an exponential form, and then rewritten into Eq. (3), in which 𝑐(𝑥, 𝑦) = 𝑏(𝑥, 𝑦)𝑒𝑥𝑝(𝑖𝜑(𝑥, 𝑦)) and 𝑐 ′ (𝑥, 𝑦) = 2
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613
Fig. 2. FFT-DI-PRA verification using numerical simulation. (A) Setting specimen phase; (B) off-axis interferogram; (C) first order information selection in the frequency domain; (D) wrapped phase; (E) and (F) phase derivatives along the horizontal and vertical axes; (G) and (H) corrected phase derivatives; (I) unwrapped phase; (J) FFT-DI-PRA retrieved specimen phase; (K) traditional method retrieved specimen phase; (L) comparisons on central sectional phases in (A), (J) and (K). The black arrows in (E)–(H) indicate the differentiation directions. The unit of the color bar is rad and the white bar in (A) indicates 10 μm. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
rather large phase derivative values can be compensated according to Eq. (9) as Step 3 in Fig. 1.
𝛥𝜑𝑥∕𝑦
′
⎧𝛥𝜑 ′ (𝑥, 𝑦) − 2𝜋 ⎪ 𝑥∕𝑦 (𝑥, 𝑦) = ⎨𝛥𝜑𝑥∕𝑦 ′ (𝑥, 𝑦) + 2𝜋 ⎪𝛥𝜑𝑥∕𝑦 ′ (𝑥, 𝑦) ⎩
𝛥𝜑𝑥∕𝑦 ′ (𝑥, 𝑦) ∼ 2𝜋 𝛥𝜑𝑥∕𝑦 ′ (𝑥, 𝑦) ∼ −2𝜋 𝑜𝑡ℎ𝑒𝑟𝑠
therefore, FFT-DI-PRA can determine and compensate the relative large phase derivatives mostly due to the phase discontinuities. Additionally, the reported rapid phase retrieval methods including both phase extraction and unwrapping methods [33–54] are often proposed separately. Different methods have their own features, such as FFT based phase extraction method can effectively suppress the noise but sacrifices the specimen details [33,34], while Hilbert transform and derivative based phase extraction method have high precision but still keep the noise in the interferograms [35–37]; simplified network programming based phase unwrapping method can robustly deal with the low signal to noise ratio condition but suffer from long time processing [43], and pixel shifted phase unwrapping method is fast but easily influenced by noise [54]. Differently, combining with the advantage of the noise suppression of the FFT based phase extraction method and the fast processing speed of the DI based phase unwrapping method, we propose the FFT-DI-PRA which provides the whole procedures from fringes to continuous sample phases. Moreover, since the FFT based phase extraction method can suppress the noise, thus the phase discontinuity can be easily recognized in phase derivative distribution, obviously simplifying the phase unwrapping procedures. As the perfect combination of the FFT based phase extraction method and the DI based phase unwrapping method, FFT-DI-PRA can retrieve the specimen phase in rather high accuracy and fast speed. In the following section, both simulations and experiments are proposed to verify FFT-DI-PRA.
(9)
Next, the unwrapped phase after phase discontinuity compensation can be reconstructed from the phase derivatives using 2-D phase integration (Step 4 and Eq. (10)). 𝜑(𝑥, 𝑦) =
∬
𝛥𝜑𝑥 ′ (𝑥, 𝑦)𝑑𝑥 + 𝛥𝜑𝑦 ′ (𝑥, 𝑦)𝑑𝑦
(10)
Since the existed background phase influences the target observations and analysis, the specimen phase can be finally extracted by removing the background (Step 5) [55–57]. Here, our previously proposed background removing technique is adopted [57]. First, the sample corresponding pixels can be recognized according to the phase values, and the background pixels can be determined. Then, the background phase distribution composed of different orders of the Zernike polynomials can be fitted from the determined background pixels. Finally, the sample phase can be retrieved by subtracting the fitted background phase distribution from the unwrapped phase. It is noted that Steps 2 to 4 are actually simplified network programming based phase unwrapping method [43], as the proposed approach skips many steps in classical network programming algorithm. The skipped steps are helpful to distinguish whether the relatively large values in derivative phase distribution are caused by phase discontinuities, large phase changes or noises. However, when using FFT based phase extraction method, the noise can be significantly reduced; moreover, most of cells often do not have relatively large phase gradient,
3. Simulations and experiments In order to test the FFT-DI-PRA performance, numerical simulations were first implemented. The specimens are standard red blood cell 3
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613
Fig. 3. FFT-DI-PRA verification using experiments. (A) Optical system, L: He–Ne laser, BS: beam splitting prism, BE: micro-objective-pinhole-lens based beam expander, MO: micro-objective, S: RBC smear, CCD: CCD camera; (B) captured off-axis interferogram; (C) first order information selection in the frequency domain; (D) wrapped phase; (E) and (F) phase derivatives along the horizontal and vertical axes; (G) and (H) corrected phase derivatives; (I) unwrapped phase; (J) FFT-DI-PRA retrieved RBC phase; (K) traditional method retrieved RBC phase; subplot between (J) and (K) shows the comparisons on central sectional phases in (J) and (K). The black arrows in (E)–(H) indicate the differentiation directions. The unit of the color bar is rad and the white bar in (J) indicates 10 μm. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
(RBC) models [58] with the special biconcave configuration shown in Fig. 2(A). Fig. 2(B) is the numerically generated off-axis interferogram with a signal to noise ratio of 20 dB, which is close to experiments. By selecting the first order information in the frequency domain shown in Fig. 2(C) according to the FFT based phase extraction method, the wrapped phase is computed and shown in Fig. 2(D). Fig. 2(E) and (F) are the phase derivatives along the horizontal and vertical axes computed from Fig. 2(D). After correcting the phase by compensating the extremely high phase derivatives caused by the phase discontinuities shown in Fig. 2(G) and (H), the unwrapped phase is reconstructed in Fig. 2(I) using 2-D integration. Finally, the specimen phase is retrieved by removing the background. This specimen phase is shown in Fig. 2(J). The reconstructed biconcave structure with close
phase values compared to setting ones prove that FFT-DI-PRA can successfully retrieve the quantitative cell phase distribution even the phase discontinuities occur within the specimen region. Additionally, Fig. 2(K) shows the retrieved specimen phase from Fig. 2(B) using the classical phase retrieval approach (FFT based phase extraction method [33,34] and network programming based phase unwrapping method [43]). In order to quantitatively evaluate the accuracy of FFTDI-PRA, Fig. 2(L) lists the comparisons on central sectional phases in Fig. 2(A), (J) and (K), indicating that both the retrieved phases fit well with the actual one. Besides, the correlation coefficient between phases in Fig. 2(A) and (J) is 0.9891, and that between phases in Fig. 2(A) and (K) is 0.9906, both close to 1. The coincident central sectional phases 4
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613
Fig. 4. Time consuming of FFT-DI-PRA and traditional method with respect to interferogram pixel size.
pixel reduction. The extremely fast phase retrieval speed indicate that FFT-DI-PRA is a potential tool for real-time quantitative live cell phase imaging and display.
and high correlation coefficients prove that FFT-DI-PRA can provide specimen phases with high accuracy. Finally, FFT-DI-PRA was tested with experimental data using a microscopic system based on a Mach–Zehnder interferometer shown in Fig. 3(A). The optical system was composed by a He–Ne laser (Thorlabs, US) with a wavelength of 632.8 nm, a micro-objective (Daheng Optics, China) for specimen magnification and a CCD camera (AVT Pike 421B, Germany) with the pixel size of 7.4 μm. Fig. 3(B) is the captured offaxis interferogram when the RBC smear was used as the specimen. Using the FFT based method, the wrapped phase was first extracted as Fig. 3(D). Then, the phase derivatives along the horizontal and vertical axes were computed and are shown in Fig. 3(E) and (F), respectively. After phase correction was done by phase discontinuity compensation and the respective phases are shown in Fig. 3(G) and (H). Afterwards, the unwrapped phase was reconstructed (Fig. 3(I)) using 2-D phase integration. Finally, the specimen phase was retrieved by removing background and it is shown in Fig. 3(J). The retrieved RBC phases have the special biconcave configuration, which is similar to the RBC model in Fig. 2(A). Moreover, the retrieved phase by FFT-DIPRA was also compared with that in Fig. 3(K) obtained from classical method (FFT based phase extraction method [33,34] and network programming based phase unwrapping method [43]), the coincident central sectional phases proving the proposed method could achieve rather high accuracy. Since the time-consuming and massive-computational phase unwrapping is simplified, the phase retrieval operation is significantly accelerated. To demonstrate the efficiency, a desktop computer equipped with an Intel Core i5 CPU and a 16 GB RAM was used for phase retrieval, in addition, the FFT-DI-PRA was implemented in MATLAB 2019a. The obtained processing times when using the proposed FFTDI-PRA approach and the traditional phase retrieval method combining FFT based phase extraction and network programming based phase unwrapping are listed in Fig. 4. According to the linearly fitting results, FFT-DI-PRA requires ∼ 9.95×10−8 s/pixel in phase retrieval, while traditional method required ∼ 2.35×10−5 s/pixel. Especially, for the interferogram with 512 × 512 pixels, only 0.0317 s is required for specimen phase retrieval using FFT-DI-PRA, 216 times faster than traditional method relying on the classical network programming algorithm [43]. In other words, FFT-DI-PRA can reach the frame rate of 30 fps in quantitative phase imaging without using GPU parallel computation or
4. Conclusion In conclusion, FFT-DI-PRA which composes of FFT based phase extraction method and simplified network programming based phase unwrapping method is designed for rapid quantitative live cell phase imaging and display. Since the noise can be significantly reduced when using the FFT based phase extraction method, and most of cells often do not have relatively large phase gradient, therefore, the relative large phase derivatives mostly caused by the phase discontinuities can be determined and compensated though simplified network programming based phase unwrapping method. FFT-DI-PRA only requires one singleshot interferogram for phase retrieval, and for an interferogram of 512 × 512 pixels, it only needs ∼0.03 s using a typical desktop, supporting the frame rate of 30 fps in quantitative phase imaging without using GPU parallel computation or pixel reduction. Additionally, proved by simulation and experiments, FFT-DI-PRA can reach rather high accuracy in quantitative phase imaging. Considering its high accuracy and extremely fast speed, it is believed FFT-DI-PRA can be potentially adopted for real-time live cell observations and measurements. Acknowledgments Authors thank Dr. Lingyu Ai in Jiangnan University for providing the RBC smear. The work is supported National Natural Science Foundation of China (61705092, U1730132, 31870154) and Natural Science Foundation of Jiangsu Province of China (BK20170194, BE2018709). References [1] D.J. Stephens, V.J. Allan, Light microscopy techniques for live cell imaging, Science 300 (2003) 82–86. [2] Y. Park, C. Depeursinge, G. Popescu, Quantitative phase imaging in biomedicine, Nat. Photonics 12 (2018) 578–589. [3] M. Teague, Deterministic phase retrieval: a Green’s function solution, J. Opt. Soc. Amer. 73 (1983) 1434–1441. 5
Q. Wei, M. Zhang, M. Yu et al.
Optics Communications 456 (2020) 124613 [31] P. Memmolo, V. Bianco, M. Paturzo, B. Javidi, P.A. Netti, P. Ferraro, Encoding multiple holograms for speckle-noise reduction in optical display, Opt. Express 22 (2014) 25768–25775. [32] V. Bianco, P. Memmolo, M. Paturzo, A. Finizio, B. Javidi, P. Ferraro, Quasi noise-free digital holography, Light-Sci. Appl. 5 (2016) e16142. [33] M. Takeda, H. Ina, S. Kobayashi, Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry, J. Opt. Soc. Amer. 72 (1982) 156–160. [34] G. Popescu, T. Ikeda, R. Dasari, M. Feld, Diffraction phase microscopy for quantifying cell structure and dynamics, Opt. Lett. 31 (2006) 775–777. [35] T. Ikeda, G. Popescu, R.R. Dasari, M.S. Feld, Hilbert phase microscopy for investigating fast dynamics in transparent systems, Opt. Lett. 30 (2005) 1165–1167. [36] S. Wang, K. Yan, L. Xue, Quantitative interferometric microscopy with two dimensional Hilbert transform based phase retrieval method, Opt. Commun. 383 (2017) 537–544. [37] S. Wang, K. Yan, N. Sun, X. Tian, W. Yu, Y. Li, A. Sun, L. Xue, Unwrapping free Hilbert transform based phase retrieval method in quantitative interferometric microscopy, Opt. Eng. 55 (2016) 113105. [38] B. Bhaduri, G. Popescu, Derivative method for phase retrieval in off-axis quantitative phase imaging, Opt. Lett. 37 (2012) 1868–1870. [39] S.K. Debnath, Y.K. Park, Real-time quantitative phase imaging with a spatial phase-shifting algorithm, Opt. Lett. 36 (2011) 4677–4679. [40] Q. Wei, Y. Li, J. Vargas, J. Wang, Q. Gong, Y. Kong, Z. Jiang, L. Xue, C. Liu, F. Liu, S. Wang, Principle component analysis based quantitative differential interference contrast microscopy, Opt. Lett. 44 (2019) 45–48. [41] D.C. Ghiglia, L.A. Romero, Robust two-dimensional weighted and unweighted phase unwrapping that uses fast transforms and iterative methods, J. Opt. Soc. Am. A 11 (1994) 107–117. [42] R.M. Goldstein, H.A. Zebker, C.L. Werner, Satellite radar interferometry: Two-dimensional phase unwrapping, Radio Sci. 23 (1988) 713–720. [43] Mario Costantini, A novel phase unwrapping method based on network programming, IEEE Trans. Geosci. Remote Sens. 36 (1998) 813–821. [44] J. Gass, A. Dakoff, M.K. Kim, Phase imaging without 2𝜋 ambiguity by multiwavelength digital holography, Opt. Lett. 28 (2003) 1141–1143. [45] N.A. Turko, N.T. Shaked, Simultaneous two-wavelength phase unwrapping using an external module for multiplexing off-axis holography, Opt. Lett. 42 (2017) 73–76. [46] N.A. Turko, P.J. Eravuchira, I. Barnea, N.T. Shaked, Simultaneous threewavelength unwrapping using external digital holographic multiplexing module, Opt. Lett. 43 (2018) 1943–1946. [47] T. Wang, L. Kai, Q. Kemao, Real-time reference-based dynamic phase retrieval algorithm for optical measurement, Appl. Opt. 56 (2017) 7726–7733. [48] O. Backoach, S. Kariv, P. Girshovitz, N.T. Shaked, Fast phase processing in offaxis holography by CUDA including parallel phase unwrapping, Opt. Express 24 (2016) 3177–3188. [49] P. Girshovitz, N.T. Shaked, Fast phase processing in off-axis holography using multiplexing with complex encoding and live-cell fluctuation map calculation in real-time, Opt. Express 23 (2015) 8773–8787. [50] P. Girshovitz, N.T. Shaked, Real-time quantitative phase reconstruction in off-axis digital holography using multiplexing, Opt. Lett. 39 (2014) 2262–2265. [51] H.V. Pham, C. Edwards, L.L. Goddard, G. Popescu, Fast phase reconstruction in white light diffraction phase microscopy, Appl. Opt. 52 (2013) A97–A101. [52] S. Wang, K. Yan, L. Xue, High speed moire based phase retrieval method for quantitative phase imaging of thin objects without phase unwrapping or aberration compensation, Opt. Commun. 359 (2016) 272–278. [53] M.A. Herráez, D.R. Burton, M.J. Lalor, M.A. Gdeisat, Fast two-dimensional phaseunwrapping algorithm based on sorting by reliability following a noncontinuous path, Appl. Opt. 41 (2002) 7437–7444. [54] L. Xue, S. Wang, K. Yan, N. Sun, Z. Li, F. Liu, Fast pixel shifting phase unwrapping algorithm in quantitative interferometric microscopy, Chin. Opt. Lett. 12 (2014) 071801. [55] L. Miccio, D. Alfieri, S. Grilli, P. Ferraro, Direct full compensation of the aberrations in quantitative phase microscopy of thin objects by a single digital hologram, Appl. Phys. Lett. 90 (2007) 041104. [56] K.W. Seo, Y.S. Choi, E.S. Seo, S.J. Lee, Aberration compensation for objective phase curvature in phase holographic microscopy, Opt. Lett. 37 (2012) 4976–4978. [57] L. Xue, S. Wang, K. Yan, N. Sun, Z. Li, F. Liu, Quantitative interferometric microscopy with improved full-field phase aberration compensation, Opt. Eng. 53 (2014) 113105. [58] S.V. Tsinopoulos, D. Polyzos, Scattering of He–Ne laser light by an average-sized red blood cell, Appl. Opt. 38 (1999) 5499–5510.
[4] Y. Shan, Q. Gong, J. Wang, J. Xu, Q. Wei, C. Liu, L. Xue, S. Wang, F. Liu, Measurements on ATP induced cellular fluctuations using real-time dual view transport of intensity phase microscopy, Biomed. Opt. Express 10 (2019) 2337–2354. [5] X. Meng, H. Huang, K. Yan, X. Tian, W. Yu, H. Cui, Y. Kong, L. Xue, C. Liu, S. Wang, Smartphone based hand-held quantitative phase microscope using transport of intensity equation method, Lab Chip 17 (2017) 104–109. [6] W. Yu, X. Tian, X. He, X. Song, L. Xue, C. Liu, S. Wang, Real time quantitative phase microscopy based on single-shot transport of intensity equation (ssTIE) method, Appl. Phys. Lett. 109 (2016) 071112. [7] X. Tian, W. Yu, X. Meng, A. Sun, L. Xue, C. Liu, S. Wang, Real-time quantitative phase imaging based on transport of intensity equation with dual simultaneously recorded field of view, Opt. Lett. 41 (2016) 1427–1430. [8] S.S. Kou, L. Waller, G. Barbastathis, C.J.R. Sheppard, Transport-of-intensity approach to differential interference contrast (TI-DIC) microscopy for quantitative phase imaging, Opt. Lett. 35 (2010) 447–449. [9] S. Zheng, B. Xue, W. Xue, X. Bai, F. Zhou, Transport of intensity phase imaging from multiple noisy intensities measured in unequally-spaced planes, Opt. Express 19 (2011) 972–985. [10] J. Zhong, R.A. Claus, J. Dauwels, L. Tian, L. Waller, Transport of intensity phase imaging by intensity spectrum fitting of exponentially spaced defocus planes, Opt. Express 22 (2014) 10661–10670. [11] G. Zheng, R. Horstmeyer, C. Yang, Wide-field, high-resolution Fourier ptychographic microscopy, Nat. Photonics 7 (2013) 739–745. [12] J.M. Rodenburg, H.M. Faulkner, A phase retrieval algorithm for shifting illumination, Appl. Phys. Lett. 85 (2004) 4795–4797. [13] A. Sun, X. He, Y. Kong, H. Cui, X. Song, L. Xue, S. Wang, C. Liu, Ultra-high speed digital micromirror device based ptychographic iterative engine method, Biomed. Opt. Express 8 (2017) 3155–3162. [14] L. Tian, Z. Liu, L.H. Yeh, M. Chen, J. Zhong, L. Waller, Computational illumination for high-speed in vitro Fourier ptychographic microscopy, Optica 2 (2015) 904–911. [15] S. Jiang, K. Guo, J. Liao, G. Zheng, Solving Fourier ptychographic imaging problems via neural network modeling and Tensorflow, Biomed. Opt. Express 9 (2018) 3306–3319. [16] B.K. Chen, P. Sidorenko, O. Lahav, O. Peleg, O. Cohen, Multiplexed single-shot ptychography, Opt. Lett. 43 (2018) 5379–5382. [17] M. Mir, B. Bhaduri, R. Wang, R. Zhu G. Popescu, Quantitative phase imaging, Prog. Opt. 57 (2012) 133–217. [18] M. Mir, H. Ding, Z. Wang, K. Tangella, G. Popescu, Blood screening using diffraction phase cytometry, J. Biomed. Opt. 15 (2010) 027016. [19] M. Mir, Z. Wang, K. Tangella, G. Popescu, Diffraction phase cytometry: blood on a CD-ROM, Opt. Express 17 (2009) 2579–2585. [20] P. Girshovitz, N.T. Shaked, Fast phase processing in off-axis holography using multiplexing with complex encoding and live-cell fluctuation map calculation in real-time, Opt. Express 23 (2015) 8773–8787. [21] G. Dardikman, N.T. Shaked, Review on methods of solving the refractive index– thickness coupling problem in digital holographic microscopy of biological cells, Opt. Commun. 422 (2018) 8–16. [22] K. Kim, Y.K. Park, Tomographic active optical trapping of arbitrarily shaped objects by exploiting 3D refractive index maps, Nature Commun. 8 (2017) 15340. [23] G. Kim, Y.J. Jo, H. Cho, H. Min, Y.K. Park, Learning-based screening of hematologic disorders using quantitative phase imaging of individual red blood cells, Biosens. Bioelectron. 123 (2019) 69–76. [24] T. Kakue, Y. Endo, T. Nishitsuji, T. Shimobaba, N. Masuda, T. Ito, Digital holographic high-speed 3D imaging for the vibrometry of fast-occurring phenomena, Sci. Rep. 7 (2017) 10413. [25] S. Wang, L. Xue, H. Li, J. Lai, Y. Song, Z. Li, Quantitative phase detection with expanded principal component analysis method on interferometric microscopic cytometer, Appl. Phys. B 116 (2014) 235–239. [26] L. Xue, S. Wang, K. Yan, N. Sun, P. Ferraro, Z. Li, F. Liu, Gravity driven high throughput phase detecting cytometer based on quantitative interferometric microscopy, Opt. Commun. 316 (2014) 5–9. [27] L. Xue, J. Vargas, S. Wang, Z. Li, F. Liu, Quantitative interferometric microscopy cytometer based on regularized optical flow algorithm, Opt. Commun. 350 (2015) 222–229. [28] K. Yan, L. Xue, S. Wang, Field of view scanning based quantitative interferometric microscopic cytometers for cellular imaging and analysis, Micros. Res. Tech. 81 (2018) 397–407. [29] F. Merola, P. Memmolo, L. Miccio, R. Savoia, M. Mugnano, A. Fontana, G. D’Ippolito, A. Sardo, A. Iolascon, A. Gambale, P. Ferraro, Tomographic flow cytometry by digital holography, Light-Sci. Appl. 6 (2017) e16241. [30] V. Bianco, M. Paturzo, V. Marchesano, I. Gallotta, E. Di Schiavib, P. Ferraro, Optofluidic holographic microscopy with custom field of view (FoV) using a linear array detector, Lab Chip 15 (2015) 2117–2124.
6