- Email: [email protected]
Complex amplitude reconstruction by iterative amplitude-phase retrieval algorithm with reference
Optics and Lasers in Engineering 105 (2018) 54–59
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Complex amplitude reconstruction by iterative amplitude-phase retrieval algorithm with reference Cheng Shen a, Cheng Guo a, Jiubin Tan a, Shutian Liu b, Zhengjun Liu a,∗ a b
Department of Automatic Test and Control, Harbin Institute of Technology, Harbin 150001 China Department of Physics, Harbin Institute of Technology, Harbin 150001, China
a r t i c l e
i n f o
Keywords: Phase retrieval Image reconstruction techniques Computational imaging
a b s t r a c t Multi-image iterative phase retrieval methods have been successfully applied in plenty of research fields due to their simple but efficient implementation. However, there is a mismatch between the measurement of the first long imaging distance and the sequential interval. In this paper, an amplitude-phase retrieval algorithm with reference is put forward without additional measurements or priori knowledge. It gets rid of measuring the first imaging distance. With a designed update formula, it significantly raises the convergence speed and the reconstruction fidelity, especially in phase retrieval. Its superiority over the original amplitude-phase retrieval (APR) method is validated by numerical analysis and experiments. Furthermore, it provides a conceptual design of a compact holographic image sensor, which can achieve numerical refocusing easily. © 2018 Elsevier Ltd. All rights reserved.
1. Introduction Direct measurement of the wavefront has not yet been achieved due to the insufficient response rate of even state-of-the-art detectors, compared to the electromagnetic frequency of light. However, the phase information plays a significant role in numerous fields, including biological imaging [1–4], X-ray crystallography [5–7], digital refocusing [8] and optical metrology [9–10]. Thus so far, there have been plenty of solutions to the classical issue, so-called phase problem. Among them, the iterative phase retrieval method has been successfully applied because of its less demanding experimental implementation in contrast to interferometry [11] and holography [12] as well as having higher resolution than Shack-Hartmann wavefront sensor [13]. As the commencement of it, the Gerchberg–Saxton (GS) algorithm [14] initially needed the known amplitude distribution at both the object plane and the image plane. Then, Fienup [15] proved that the GS algorithm mathematically is an Error Reduction (ER) algorithm in nature. The requirement of a known object amplitude map can be removed by applying a support constraint. Also, to avoid stagnation in the ER algorithm, he put forward the hybrid input–output (HIO) algorithm by introducing the feedback. Recently, the priori knowledge of a tight support was waived by the shrinkwrap algorithm [16], which updates the support region during iteration from a support estimate with autocorrelation and has been verified in many X-ray single-shot coherent diffractive imaging experiments [17–18].
∗
Multi-image phase retrieval algorithms were proposed [19–22] afterwards. According to the strategy of generating multiple measurements, they can be categorized into two groups: lateral and axial scanning. Ptychography [19,23] is the representative of lateral scanning techniques. As for axial scanning, the single-beam multiple-intensity reconstruction (SBMIR) algorithm [20] and the multi-stage algorithm [21] both serially update the complex amplitude estimates during the iteration. Differently, the APR algorithm [22] copes with the update in a parallel way and employs the average operator, which enhances the noise robustness. Till now, the APR algorithm has been successfully applied in encryption [24,25] and coherent diffraction imaging [26,27]. Besides, the effect of position measurement error [28], experimental noise [29] and tilt illumination [27] has been fully discussed. To conclude, all multi-image phase retrieval algorithms are reference free, which means they do not demand the known object amplitude distribution. But a reference could effectively avoid the local minimum problem and accelerate the convergence speed. Thus, we consider to take the pattern at the first measuring plane as the ‘object’. Based on the idea, the known ‘object’ amplitude intrinsically included in the multi-image dataset can be utilized to design a new algorithm, which is named after APR with reference (APRr). As is shown below, the known ‘object’ amplitude can actually accelerate the convergence speed and helps acquire reconstruction with higher fidelity, especially for phase retrieval. Furthermore, taking the experimental Poisson noise into account, a weighted estimation formula is elaborated to suppress the noise
Corresponding author. E-mail address: [email protected] (Z. Liu).
https://doi.org/10.1016/j.optlaseng.2018.01.004 Received 30 November 2017; Received in revised form 31 December 2017; Accepted 8 January 2018 0143-8166/© 2018 Elsevier Ltd. All rights reserved.
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
Fig. 1. Schematic diagram of experimental setup and the corresponding simulation model (in the inset box).
Fig. 2. Numerical analysis of APR and APRr. (a) Simulated complex object; (b) Noise free: error distribution of the final amplitude (upper row) and phase (lower row) reconstructions of two algorithms and their corresponding convergence curves; (c) Poisson noise (Rnoise = 10%): the final amplitude (upper row) and phase (lower row) reconstructions of two algorithms and their corresponding convergence curves.
effect. Last but not least, the new designed algorithm can realize a prototype of a compact holographic image sensor.
intensity patterns I1 –IN are recorded. Accordingly, the distance between the sample I0 and the first measuring plane I1 is denoted by d0 with an internal of Δz in sequential measurements, shown in the inset box of Fig. 1. After the multi-image dataset is acquired, it will be fed into our designed algorithm as follows:
2. Reconstruction scheme The optical system is illustrated in Fig. 1, which is a basic coherent diffraction imaging (CDI) system. The working wavelength is 532 nm (MW-SGX, Changchun Laser Optoelectronics Technology). Shaped by an aperture, it generates the plane illumination on the sample. Then, a CCD (GS3-U3-41S4M, Point Grey Research) mounted on the translation stage (M-403, Physik Instrumente) moves along the optical axis and several
(i) Initialize a random phase guess 𝜑1 at the first measuring plane and combine it with the square root of the first intensity pattern to obtain the complex amplitude estimate √ ( ) 𝐸1 = 𝐼1 exp 𝑖𝜑1 ; (1) 55
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
Fig. 3. The binary sample experiment. (a) 4 representative intensity patterns; (b) Comparison of amplitude and phase retrieval from APR and APRr, with the scale bar in the first picture corresponding to 0.19 mm; (c) Amplitude and phase retrieval convergence curves of two algorithms.
(ii) Forward propagate it to get the sequential N−1 complex amplitude estimates ( ) 𝐸𝑛 = 𝐴′𝑛 exp 𝑖𝜑𝑛 , 𝑛 = 2, ⋯ 𝑁; (2)
There are plenty of cost functions used in the simulation and experiments, like mean square error (MSE) function [26], SG function and feature similarity index (FSIM) [30]. They can also act as the evaluation metrics of the final reconstruction. In this paper, logarithm of MSE (LMSE) and SG are adopted to monitor the convergence and evaluate the reconstruction when the original object function (ground truth) is known and unknown respectively. Also, SG can function as the metric for the auto-focus procedure.
(iii) Update their amplitude with the measured N−1 intensity patterns √ ( ) 𝐸𝑛 = 𝐼𝑛 exp 𝑖𝜑𝑛 , 𝑛 = 2, ⋯ 𝑁; (3) (iv) Backward propagate the updated estimates to get N−1 complex amplitude estimates at the first measuring plane ( ) 𝐸1,𝑛 = 𝐴1,𝑛 exp 𝑖𝜑1,𝑛 , 𝑛 = 2, ⋯ 𝑁; (4)
3. Numerical analysis
(v) Update the ‘object’ function using the elaborated formula [ √ ] ( ) 𝐸1 = 𝛽 𝐼1 + (1 − 𝛽)𝐴′ 1 exp 𝑖𝜑1 , 𝐴′ 1 =
1 𝑁 −1
𝑁 ∑ 𝑛=2
𝐴1,𝑛 ,𝜑1 =
1 𝑁 −1
𝑁 ∑ 𝑛=2
Firstly, the numerical analysis is given. The simulated sample consists of the cameraman image in MATLAB as amplitude and the 1951 USAF resolution test chart image [31] as phase, shown in Fig. 2(a). It is sampled as 256 × 256 pixels. d0 is taken as 50 mm and Δz is 20 mm with N = 5. Then, the generated intensity patterns are input into APR and APRr. The retrieval results after 1000 iterations under the noise-free condition are displayed in Fig. 2(b). From the error distribution (difference between the ground truth and the reconstruction), it can be clearly seen that the average error of APRr in both amplitude and phase is several orders of magnitude smaller than that of APR. From the view of LMSE convergence curves, APRr jumps out the stagnation stage much earlier and converges faster to a smaller LMSE value than APR. The results prove that APRr holds faster convergence speed and higher retrieval fidelity. In the experiment, Poisson noise (or shot noise) is evitable and will become dominant when adopting the dark illumination to fully exploit the dynamic range of detectors and avoid the overexposure. Thus, it is necessary to test two algorithms under Poisson noise. Here, the noise
(5) 𝜑1,𝑛 ;
(vi) Perform the steps (ii)–(v) until the convergence condition is satisfied; (vii) Backward propagate E1 to get the real object function if d0 is precisely known, otherwise the autofocus algorithm using metrics like squared-gradient (SG) function [29] would be incorporated. Here, 𝛽 is set as 0.9 empirically and the field is propagated between two planes using the angular spectrum approach. The convergence condition could be that the predefined threshold of some cost function or some fixed iteration number is reached. To sum up, compared with APR algorithm only adopting an average operation, APRr utilizes the known amplitude distribution as the real space constraint and a synthesized estimate is designed as Eq. (5) to suppress the Poisson noise. As is shown in the following, the modification significantly raises the convergence speed and the noise robustness, especially under the laser illumination. 56
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
Fig. 4. The biological sample experiment. (a) A3 representative intensity patterns. (b)–(e) Comparison of amplitude and phase retrieval from APR and APRr, with the scale bar in (b) corresponding to 0.31 mm. (d1 )–(e1): Enlarged details of phase reconstruction in the inset box of (d)–(e).
level is indicated by ) ( √ || | |√ || sum || 𝐼𝑛 + 𝑃 | − | 𝐼𝑛 || || | | || 𝑛 𝑅noise = (6) , 𝑛 = 1, ⋯ 𝑁, (√ ) | | sum | 𝐼𝑛 | | | where P is a 2D Poisson random variables and sum() represents the summation of elements in the matrix. Rnoise is the average of 𝑅𝑛noise . Fig. 2(c) shows the performance of APR and APRr when Rnoise = 10%, representing a high shot noise level. As we can see, APRr always obtains the results with a smaller LMSE from both amplitude and phase reconstruction. Notable is that APRr significantly enhances the ability of phase retrieval and raises the robustness to Poisson noise. It says that APRr is more advantageous in retrieving phase structures. This makes it quite promising in label-free or stain-free optical imaging of biological samples in vitro considering they mostly only contain phase information associated with the absorption variation, for example, blood smears and carcinoma cells [32].
wavefront of coherent plane illumination, much better than APR phase reconstruction. To quantify the convergence process, the SG function is employed and defined by { ( ( 2) 2 )} SG(𝜓 (𝑛) ) = avg grad2𝑥 |𝜓 (𝑛) | + grad2𝑦 |𝜓 (𝑛) | ,
(7)
where 𝜓 (n) is the object amplitude or phase function at the nth iteration and avg{} represents calculating the mean pixel value while grad() calculating the gradient distribution along the column or row direction. Due to the fact that the binary sample only contains the edge information, it is appropriate to utilize SG as the metric here. The larger SG is, the shaper edges are, thus the reconstruction is closer to the ground truth. Shown in Fig. 3(c), APRr is superior in both convergence speed and retrieval accuracy, which is in accordance with the visual perception in Fig. 3(b). Then, an ant sample is tested. Here, d0 is 24.00 mm and Δz is set as 1.00 mm with N = 5. Fig. 4(a) displays the first three ones among five diffraction intensity patterns and (b)–(e) show the results from APR and APRr after 1000 iterations. In the amplitude comparison, it is obvious that the ripple artifacts in the APR reconstruction have been suppressed well in the APRr counterpart. To emphasize details, the inset boxes show the enlarged ant legs in phase retrieval. As we can see, the phase retrieved by APRr can resolve the adjacent legs but they are distorted in the APR phase reconstruction. To sum up, the experimental results of both binary sample and biological sample demonstrate that APRr can acquire the reconstruction with enhanced contrast and higher fidelity, especially in phase retrieval. The conclusion complies with the numerical analysis above.
4. Experiment To further demonstrate the validity of APRr, two experiments are conducted. First, a binary sample is tested. It is a piece of chromium plated glass with transparent number arrays made by laser direct writing photolithography and resembles the USAF resolution test chart. Its size is indicated by the scale bar. Here, d0 is 144.41 mm and Δz is set as 1.00 mm with N = 10. Several intensity patterns recorded by the detector during the experiment are shown in Fig. 3(a). The retrieval results from two algorithms after 1000 iterations are displayed in Fig. 3(b). From amplitude, it can be seen that the reconstruction of APRr has sharper edges with less artifacts. More obvious is the flat phase in the transparent number region of APRr reconstruction, corresponding to the constant 57
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
Fig. 5. Schematic diagram of the APRr camera. (a) Simulated imaging of two ‘objects’ with a distance of 30 mm; (b) Auto-focus algorithm using SG; (c)–(f) Numerical refocusing.
5. Application
phase retrieval algorithms, it raises the convergence speed and the reconstruction fidelity without additional measurements or extra priori knowledge, which is validated by numerical analysis and experiments. Most impressive is its superior ability of phase retrieval. It is meaningful to biomedical imaging, where many transparent samples only contain phase information. At length, APRr provides a prototype of a compact holographic image sensor, which could realize numerical refocusing easily and does not require the calibration, compared to the newest holographic image sensor [8]. This could be our further work.
Another promising application of APRr is that it provides a conceptual design of a compact holographic image sensor. As is shown in Fig. 5(a), the camera is designed according to APRr and can realize the numerical refocusing after capture. The image at the front is 10 mm to the sensor, which moves 3 times along the optical axis with an interval of 10 mm. Then, the intensity patterns will be fed to APRr. To fully exploit the detector pixels, relaying lenses should be added before the sensor, which is not shown in Fig. 5. Above all, APRr gets rid of measuring the distance between the object plane and the first measuring plane, compared with other multiimage phase retrieval methods. The distance is usually large and hard to precisely measure, though it can be achieved by relatively demanding techniques like time of flight (TOF) [33]. Alternatively, auto-focus algorithms [34] can be combined with APRr to locate the object plane automatically, illustrated by Fig. 5(b)-(f). Here, SG function is just a simple representative of auto-focus algorithms. Two local maximum of the SG curve correspond to the location of objects at the front and the rear respectively. The essence of realizing the APRr camera lies on a known interval between the measuring planes, which can be easily achieved by controlling the sensor with a step motor. Moreover, APRr is compatible with other optical transformations, like APR in gyrator transforms [22], which can be achieved by the focus-tunable lens [35]. Another potential obstacle to application is the coherent assumption. Experiments of APR under LED illumination have shown that partially coherent illumination can still achieve the satisfactory reconstruction, which is similar with other computational phase retrieval techniques, including transport-ofintensity equation (TIE) [36,37] and Fourier ptychographic microscopy (FPM) [38,39].
Acknowledgments This work was supported by the National Natural Science Foundation of China (Nos. 61377016, 61575055 and 61575053), the Program for New Century Excellent Talents in University (No. NCET-12-0148), the China Postdoctoral Science Foundation (Nos. 2013M540278 and 2015T80340), the Fundamental Research Funds for the Central Universities (No. HIT.BRETIII.201406), the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry, China. References [1] Popescu G, Ikeda T, Dasari RR, Feld MS. Diffraction phase microscopy for quantifying cell structure and dynamics. Opt Lett 2006;31:775–7. [2] Waller L, Tsang M, Ponda S, Yang S, Barbastathis G. Phase and amplitude imaging from noisy images by Kalman filtering. Opt Express 2011;19:2805–15. [3] Zheng G, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy. Nat Photonics 2013;7:739–45. [4] Shechtman Y, Eldar YC, Cohen O, Chapman HN, Miao J, Segev M. Phase retrieval with application to optical imaging: a contemporary overview. IEEE Signal Process Mag 2015;32:87–109. [5] Millane RP. Phase retrieval in crystallography and optics. J Opt Soc Am A 1990;7:394–411. [6] Miao J, Charalambous P, Kirz J, Sayre D. Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens. Nature 1999;400:342–4. [7] Thibault P, Dierolf M, Menzel A, Bunk O, David C, Pfeiffer F. High-resolution scanning X-ray diffraction microscopy. Science 2008;321:379–82. [8] Lee K, Park Y. Exploiting the speckle-correlation scattering matrix for a compact reference-free holographic image sensor. Nat Commun 2016;7:13359. [9] Anand A, Chhaniwal VK, Almoro P, Pedrini G, Osten W. Shape and deformation measurements of 3D objects using volume speckle field and phase retrieval. Opt Lett 2009;34:1522–4.
6. Conclusion In conclusion, an augmented iterative amplitude-phase retrieval algorithm, namely APRr, is put forward. It takes the first intensity pattern as the object and a special update formula is designed to suppress the speckle noise effect. Compared with the existing multi-image 58
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
[10] Zuo C, Chen Q, Qu W, Asundi A. Noninterferometric single-shot quantitative phase microscopy. Opt Lett 2013;38:3538–41. [11] Hariharan P, Oreb BF, Eiju T. Digital phase-shifting interferometry: a simple error– compensating phase calculation algorithm. Appl Opt 1987;26:2504–6. [12] Gabor D. A new microscopic principle. Nature 1948;161:777–8. [13] Shack RV, Platt BC. Production and use of a lenticular Hartmann screen. J Opt Soc Am 1971;61:656–60. [14] Gerchberg RW, Saxton WO. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 1972;35:237–46. [15] Fienup JR. Phase retrieval algorithms: A comparison. Appl Opt 1982;21:2758–69. [16] Marchesini S, He H, Chapman HN, Hauriege SP, Noy A, Howells MR, Weierstall U, Spence JCH. X-ray image reconstruction from a diffraction pattern alone. Phys Rev B 2003;68:399–404. [17] Chao W, Harteneck BD, Liddle JA, Anderson EH, Attwood DT. Soft x-ray microscopy at a spatial resolution better than 15 nm. Nature 2005;435:1210–13. [18] Pfeifer MA, Williams GJ, Vartanyants IA, Harder R, Robinson IK. Three-dimensional mapping of a deformation field inside a nanocrystal. Nature 2006;442:63–6. [19] Rodenburg JM, Faulkner HML. A phase retrieval algorithm for shifting illumination. Appl Phys Lett 2004;85:4795–7. [20] Pedrini G, Osten W, Zhang Y. Wave-front reconstruction from a sequence of interferograms recorded at different planes. Opt Lett 2005;30:833–5. [21] Rodrigo JA, Duadi H, Alieva T, Zalevsky Z. Multi-stage phase retrieval algorithm based upon the gyrator transform. Opt Express 2010;18:1510–20. [22] Liu Z, Guo C, Tan J, Wu Q, Pan L, Liu S. Iterative phase amplitude retrieval from multiple images in gyrator domains. J Opt 2015;17:025701. [23] Putkunz CT, D’Alfonso AJ, Morgan AJ, Weyland M, Dwyer C, Bourgeois L, Etheridge J, Roberts A, Scholten RE, Nugent KA, Allen LJ. Atom-scale ptychographic electron diffractive imaging of boron nitride cones. Phys Rev Lett 2012;108:073901. [24] Liu Z, Shen C, Tan J, Liu S. A recovery method of double random phase encoding system with a parallel phase retrieval. IEEE Photonics J 2016;8:7801807. [25] Guo C, Wei C, Tan J, Chen K, Liu S, Wu Q, Liu Z. A review of iterative phase retrieval for measurement and encryption. Opt Lasers Eng 2017;89:2–12.
[26] Shen C, Tan J, Wei C, Liu Z. Coherent diffraction imaging by moving a lens. Opt Express 2016;24:16520–9. [27] Guo C, Li Q, Wei C, Tan J, Liu S, Liu Z. Axial multi-image phase retrieval under tilt illumination. Sci Rep 2017;7:7562. [28] Guo C, Tan J, Liu Z. Precision influence of a phase retrieval algorithm in fractional Fourier domains from position measurement error. Appl Opt 2015;54:6940–7. [29] Shen C, Bao X, Tan J, Liu S, Liu Z. Two noise-robust axial scanning multi-image phase retrieval algorithms based on Pauta criterion and smoothness constraint. Opt Express 2017;25:16235–49. [30] Zhang L, Zhang L, Mou X, Zhang D. FSIM: a feature similarity sndex for image quality assessment. IEEE Trans. Image Process 2011;20:2378–86. [31] https://www.thorlabs.com/thorproduct.cfm?partnumber=R3L3S1N. [32] Greenbaum A, Zhang Y, Feizi A, Chung P, Luo W, Kandukuri SR, Ozcan A. Wide-field computational imaging of pathology slides using lens-free on-chip microscopy. Sci Transl Med 2014;6 267ra175. [33] Delpy DT, Cope M, van der Zee P, Arridge S, Wray S, Wyatt J. Estimation of optical pathlength through tissue from direct time of flight measurement. Phys Med Biol 1988;33:1433–42. [34] He J, Zhou R, Hong Z. Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera. IEEE Trans Consum Electr 2003;49:257–62. [35] Mosso F, Peters E, Pérez DG. Complex wavefront reconstruction from multiple-image planes produced by a focus tunable lens. Opt Lett 2015;40:4623–6. [36] Paganin D, Nugent KA. Noninterferometric phase imaging with partially coherent light. Phys Rev Lett 1998;80:2586–9. [37] Zuo C, Chen Q, Yu Y, Asundi AK. Transport-of-intensity phase imaging using Savitzky-Golay differentiation filter: theory and applications. Opt Express 2013;21:5346–62. [38] Tian L, Waller L. 3D intensity and phase imaging from light field measurements in an LED array microscope. Optica 2015;2:104–11. [39] Dong S, Nanda P, Guo K, Liao J, Zheng G. Incoherent Fourier ptychographic photography using structured light. Photon Res 2015;3:19–23.
59
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Complex amplitude reconstruction by iterative amplitude-phase retrieval algorithm with reference Cheng Shen a, Cheng Guo a, Jiubin Tan a, Shutian Liu b, Zhengjun Liu a,∗ a b
Department of Automatic Test and Control, Harbin Institute of Technology, Harbin 150001 China Department of Physics, Harbin Institute of Technology, Harbin 150001, China
a r t i c l e
i n f o
Keywords: Phase retrieval Image reconstruction techniques Computational imaging
a b s t r a c t Multi-image iterative phase retrieval methods have been successfully applied in plenty of research fields due to their simple but efficient implementation. However, there is a mismatch between the measurement of the first long imaging distance and the sequential interval. In this paper, an amplitude-phase retrieval algorithm with reference is put forward without additional measurements or priori knowledge. It gets rid of measuring the first imaging distance. With a designed update formula, it significantly raises the convergence speed and the reconstruction fidelity, especially in phase retrieval. Its superiority over the original amplitude-phase retrieval (APR) method is validated by numerical analysis and experiments. Furthermore, it provides a conceptual design of a compact holographic image sensor, which can achieve numerical refocusing easily. © 2018 Elsevier Ltd. All rights reserved.
1. Introduction Direct measurement of the wavefront has not yet been achieved due to the insufficient response rate of even state-of-the-art detectors, compared to the electromagnetic frequency of light. However, the phase information plays a significant role in numerous fields, including biological imaging [1–4], X-ray crystallography [5–7], digital refocusing [8] and optical metrology [9–10]. Thus so far, there have been plenty of solutions to the classical issue, so-called phase problem. Among them, the iterative phase retrieval method has been successfully applied because of its less demanding experimental implementation in contrast to interferometry [11] and holography [12] as well as having higher resolution than Shack-Hartmann wavefront sensor [13]. As the commencement of it, the Gerchberg–Saxton (GS) algorithm [14] initially needed the known amplitude distribution at both the object plane and the image plane. Then, Fienup [15] proved that the GS algorithm mathematically is an Error Reduction (ER) algorithm in nature. The requirement of a known object amplitude map can be removed by applying a support constraint. Also, to avoid stagnation in the ER algorithm, he put forward the hybrid input–output (HIO) algorithm by introducing the feedback. Recently, the priori knowledge of a tight support was waived by the shrinkwrap algorithm [16], which updates the support region during iteration from a support estimate with autocorrelation and has been verified in many X-ray single-shot coherent diffractive imaging experiments [17–18].
∗
Multi-image phase retrieval algorithms were proposed [19–22] afterwards. According to the strategy of generating multiple measurements, they can be categorized into two groups: lateral and axial scanning. Ptychography [19,23] is the representative of lateral scanning techniques. As for axial scanning, the single-beam multiple-intensity reconstruction (SBMIR) algorithm [20] and the multi-stage algorithm [21] both serially update the complex amplitude estimates during the iteration. Differently, the APR algorithm [22] copes with the update in a parallel way and employs the average operator, which enhances the noise robustness. Till now, the APR algorithm has been successfully applied in encryption [24,25] and coherent diffraction imaging [26,27]. Besides, the effect of position measurement error [28], experimental noise [29] and tilt illumination [27] has been fully discussed. To conclude, all multi-image phase retrieval algorithms are reference free, which means they do not demand the known object amplitude distribution. But a reference could effectively avoid the local minimum problem and accelerate the convergence speed. Thus, we consider to take the pattern at the first measuring plane as the ‘object’. Based on the idea, the known ‘object’ amplitude intrinsically included in the multi-image dataset can be utilized to design a new algorithm, which is named after APR with reference (APRr). As is shown below, the known ‘object’ amplitude can actually accelerate the convergence speed and helps acquire reconstruction with higher fidelity, especially for phase retrieval. Furthermore, taking the experimental Poisson noise into account, a weighted estimation formula is elaborated to suppress the noise
Corresponding author. E-mail address: [email protected] (Z. Liu).
https://doi.org/10.1016/j.optlaseng.2018.01.004 Received 30 November 2017; Received in revised form 31 December 2017; Accepted 8 January 2018 0143-8166/© 2018 Elsevier Ltd. All rights reserved.
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
Fig. 1. Schematic diagram of experimental setup and the corresponding simulation model (in the inset box).
Fig. 2. Numerical analysis of APR and APRr. (a) Simulated complex object; (b) Noise free: error distribution of the final amplitude (upper row) and phase (lower row) reconstructions of two algorithms and their corresponding convergence curves; (c) Poisson noise (Rnoise = 10%): the final amplitude (upper row) and phase (lower row) reconstructions of two algorithms and their corresponding convergence curves.
effect. Last but not least, the new designed algorithm can realize a prototype of a compact holographic image sensor.
intensity patterns I1 –IN are recorded. Accordingly, the distance between the sample I0 and the first measuring plane I1 is denoted by d0 with an internal of Δz in sequential measurements, shown in the inset box of Fig. 1. After the multi-image dataset is acquired, it will be fed into our designed algorithm as follows:
2. Reconstruction scheme The optical system is illustrated in Fig. 1, which is a basic coherent diffraction imaging (CDI) system. The working wavelength is 532 nm (MW-SGX, Changchun Laser Optoelectronics Technology). Shaped by an aperture, it generates the plane illumination on the sample. Then, a CCD (GS3-U3-41S4M, Point Grey Research) mounted on the translation stage (M-403, Physik Instrumente) moves along the optical axis and several
(i) Initialize a random phase guess 𝜑1 at the first measuring plane and combine it with the square root of the first intensity pattern to obtain the complex amplitude estimate √ ( ) 𝐸1 = 𝐼1 exp 𝑖𝜑1 ; (1) 55
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
Fig. 3. The binary sample experiment. (a) 4 representative intensity patterns; (b) Comparison of amplitude and phase retrieval from APR and APRr, with the scale bar in the first picture corresponding to 0.19 mm; (c) Amplitude and phase retrieval convergence curves of two algorithms.
(ii) Forward propagate it to get the sequential N−1 complex amplitude estimates ( ) 𝐸𝑛 = 𝐴′𝑛 exp 𝑖𝜑𝑛 , 𝑛 = 2, ⋯ 𝑁; (2)
There are plenty of cost functions used in the simulation and experiments, like mean square error (MSE) function [26], SG function and feature similarity index (FSIM) [30]. They can also act as the evaluation metrics of the final reconstruction. In this paper, logarithm of MSE (LMSE) and SG are adopted to monitor the convergence and evaluate the reconstruction when the original object function (ground truth) is known and unknown respectively. Also, SG can function as the metric for the auto-focus procedure.
(iii) Update their amplitude with the measured N−1 intensity patterns √ ( ) 𝐸𝑛 = 𝐼𝑛 exp 𝑖𝜑𝑛 , 𝑛 = 2, ⋯ 𝑁; (3) (iv) Backward propagate the updated estimates to get N−1 complex amplitude estimates at the first measuring plane ( ) 𝐸1,𝑛 = 𝐴1,𝑛 exp 𝑖𝜑1,𝑛 , 𝑛 = 2, ⋯ 𝑁; (4)
3. Numerical analysis
(v) Update the ‘object’ function using the elaborated formula [ √ ] ( ) 𝐸1 = 𝛽 𝐼1 + (1 − 𝛽)𝐴′ 1 exp 𝑖𝜑1 , 𝐴′ 1 =
1 𝑁 −1
𝑁 ∑ 𝑛=2
𝐴1,𝑛 ,𝜑1 =
1 𝑁 −1
𝑁 ∑ 𝑛=2
Firstly, the numerical analysis is given. The simulated sample consists of the cameraman image in MATLAB as amplitude and the 1951 USAF resolution test chart image [31] as phase, shown in Fig. 2(a). It is sampled as 256 × 256 pixels. d0 is taken as 50 mm and Δz is 20 mm with N = 5. Then, the generated intensity patterns are input into APR and APRr. The retrieval results after 1000 iterations under the noise-free condition are displayed in Fig. 2(b). From the error distribution (difference between the ground truth and the reconstruction), it can be clearly seen that the average error of APRr in both amplitude and phase is several orders of magnitude smaller than that of APR. From the view of LMSE convergence curves, APRr jumps out the stagnation stage much earlier and converges faster to a smaller LMSE value than APR. The results prove that APRr holds faster convergence speed and higher retrieval fidelity. In the experiment, Poisson noise (or shot noise) is evitable and will become dominant when adopting the dark illumination to fully exploit the dynamic range of detectors and avoid the overexposure. Thus, it is necessary to test two algorithms under Poisson noise. Here, the noise
(5) 𝜑1,𝑛 ;
(vi) Perform the steps (ii)–(v) until the convergence condition is satisfied; (vii) Backward propagate E1 to get the real object function if d0 is precisely known, otherwise the autofocus algorithm using metrics like squared-gradient (SG) function [29] would be incorporated. Here, 𝛽 is set as 0.9 empirically and the field is propagated between two planes using the angular spectrum approach. The convergence condition could be that the predefined threshold of some cost function or some fixed iteration number is reached. To sum up, compared with APR algorithm only adopting an average operation, APRr utilizes the known amplitude distribution as the real space constraint and a synthesized estimate is designed as Eq. (5) to suppress the Poisson noise. As is shown in the following, the modification significantly raises the convergence speed and the noise robustness, especially under the laser illumination. 56
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
Fig. 4. The biological sample experiment. (a) A3 representative intensity patterns. (b)–(e) Comparison of amplitude and phase retrieval from APR and APRr, with the scale bar in (b) corresponding to 0.31 mm. (d1 )–(e1): Enlarged details of phase reconstruction in the inset box of (d)–(e).
level is indicated by ) ( √ || | |√ || sum || 𝐼𝑛 + 𝑃 | − | 𝐼𝑛 || || | | || 𝑛 𝑅noise = (6) , 𝑛 = 1, ⋯ 𝑁, (√ ) | | sum | 𝐼𝑛 | | | where P is a 2D Poisson random variables and sum() represents the summation of elements in the matrix. Rnoise is the average of 𝑅𝑛noise . Fig. 2(c) shows the performance of APR and APRr when Rnoise = 10%, representing a high shot noise level. As we can see, APRr always obtains the results with a smaller LMSE from both amplitude and phase reconstruction. Notable is that APRr significantly enhances the ability of phase retrieval and raises the robustness to Poisson noise. It says that APRr is more advantageous in retrieving phase structures. This makes it quite promising in label-free or stain-free optical imaging of biological samples in vitro considering they mostly only contain phase information associated with the absorption variation, for example, blood smears and carcinoma cells [32].
wavefront of coherent plane illumination, much better than APR phase reconstruction. To quantify the convergence process, the SG function is employed and defined by { ( ( 2) 2 )} SG(𝜓 (𝑛) ) = avg grad2𝑥 |𝜓 (𝑛) | + grad2𝑦 |𝜓 (𝑛) | ,
(7)
where 𝜓 (n) is the object amplitude or phase function at the nth iteration and avg{} represents calculating the mean pixel value while grad() calculating the gradient distribution along the column or row direction. Due to the fact that the binary sample only contains the edge information, it is appropriate to utilize SG as the metric here. The larger SG is, the shaper edges are, thus the reconstruction is closer to the ground truth. Shown in Fig. 3(c), APRr is superior in both convergence speed and retrieval accuracy, which is in accordance with the visual perception in Fig. 3(b). Then, an ant sample is tested. Here, d0 is 24.00 mm and Δz is set as 1.00 mm with N = 5. Fig. 4(a) displays the first three ones among five diffraction intensity patterns and (b)–(e) show the results from APR and APRr after 1000 iterations. In the amplitude comparison, it is obvious that the ripple artifacts in the APR reconstruction have been suppressed well in the APRr counterpart. To emphasize details, the inset boxes show the enlarged ant legs in phase retrieval. As we can see, the phase retrieved by APRr can resolve the adjacent legs but they are distorted in the APR phase reconstruction. To sum up, the experimental results of both binary sample and biological sample demonstrate that APRr can acquire the reconstruction with enhanced contrast and higher fidelity, especially in phase retrieval. The conclusion complies with the numerical analysis above.
4. Experiment To further demonstrate the validity of APRr, two experiments are conducted. First, a binary sample is tested. It is a piece of chromium plated glass with transparent number arrays made by laser direct writing photolithography and resembles the USAF resolution test chart. Its size is indicated by the scale bar. Here, d0 is 144.41 mm and Δz is set as 1.00 mm with N = 10. Several intensity patterns recorded by the detector during the experiment are shown in Fig. 3(a). The retrieval results from two algorithms after 1000 iterations are displayed in Fig. 3(b). From amplitude, it can be seen that the reconstruction of APRr has sharper edges with less artifacts. More obvious is the flat phase in the transparent number region of APRr reconstruction, corresponding to the constant 57
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
Fig. 5. Schematic diagram of the APRr camera. (a) Simulated imaging of two ‘objects’ with a distance of 30 mm; (b) Auto-focus algorithm using SG; (c)–(f) Numerical refocusing.
5. Application
phase retrieval algorithms, it raises the convergence speed and the reconstruction fidelity without additional measurements or extra priori knowledge, which is validated by numerical analysis and experiments. Most impressive is its superior ability of phase retrieval. It is meaningful to biomedical imaging, where many transparent samples only contain phase information. At length, APRr provides a prototype of a compact holographic image sensor, which could realize numerical refocusing easily and does not require the calibration, compared to the newest holographic image sensor [8]. This could be our further work.
Another promising application of APRr is that it provides a conceptual design of a compact holographic image sensor. As is shown in Fig. 5(a), the camera is designed according to APRr and can realize the numerical refocusing after capture. The image at the front is 10 mm to the sensor, which moves 3 times along the optical axis with an interval of 10 mm. Then, the intensity patterns will be fed to APRr. To fully exploit the detector pixels, relaying lenses should be added before the sensor, which is not shown in Fig. 5. Above all, APRr gets rid of measuring the distance between the object plane and the first measuring plane, compared with other multiimage phase retrieval methods. The distance is usually large and hard to precisely measure, though it can be achieved by relatively demanding techniques like time of flight (TOF) [33]. Alternatively, auto-focus algorithms [34] can be combined with APRr to locate the object plane automatically, illustrated by Fig. 5(b)-(f). Here, SG function is just a simple representative of auto-focus algorithms. Two local maximum of the SG curve correspond to the location of objects at the front and the rear respectively. The essence of realizing the APRr camera lies on a known interval between the measuring planes, which can be easily achieved by controlling the sensor with a step motor. Moreover, APRr is compatible with other optical transformations, like APR in gyrator transforms [22], which can be achieved by the focus-tunable lens [35]. Another potential obstacle to application is the coherent assumption. Experiments of APR under LED illumination have shown that partially coherent illumination can still achieve the satisfactory reconstruction, which is similar with other computational phase retrieval techniques, including transport-ofintensity equation (TIE) [36,37] and Fourier ptychographic microscopy (FPM) [38,39].
Acknowledgments This work was supported by the National Natural Science Foundation of China (Nos. 61377016, 61575055 and 61575053), the Program for New Century Excellent Talents in University (No. NCET-12-0148), the China Postdoctoral Science Foundation (Nos. 2013M540278 and 2015T80340), the Fundamental Research Funds for the Central Universities (No. HIT.BRETIII.201406), the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry, China. References [1] Popescu G, Ikeda T, Dasari RR, Feld MS. Diffraction phase microscopy for quantifying cell structure and dynamics. Opt Lett 2006;31:775–7. [2] Waller L, Tsang M, Ponda S, Yang S, Barbastathis G. Phase and amplitude imaging from noisy images by Kalman filtering. Opt Express 2011;19:2805–15. [3] Zheng G, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy. Nat Photonics 2013;7:739–45. [4] Shechtman Y, Eldar YC, Cohen O, Chapman HN, Miao J, Segev M. Phase retrieval with application to optical imaging: a contemporary overview. IEEE Signal Process Mag 2015;32:87–109. [5] Millane RP. Phase retrieval in crystallography and optics. J Opt Soc Am A 1990;7:394–411. [6] Miao J, Charalambous P, Kirz J, Sayre D. Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens. Nature 1999;400:342–4. [7] Thibault P, Dierolf M, Menzel A, Bunk O, David C, Pfeiffer F. High-resolution scanning X-ray diffraction microscopy. Science 2008;321:379–82. [8] Lee K, Park Y. Exploiting the speckle-correlation scattering matrix for a compact reference-free holographic image sensor. Nat Commun 2016;7:13359. [9] Anand A, Chhaniwal VK, Almoro P, Pedrini G, Osten W. Shape and deformation measurements of 3D objects using volume speckle field and phase retrieval. Opt Lett 2009;34:1522–4.
6. Conclusion In conclusion, an augmented iterative amplitude-phase retrieval algorithm, namely APRr, is put forward. It takes the first intensity pattern as the object and a special update formula is designed to suppress the speckle noise effect. Compared with the existing multi-image 58
C. Shen et al.
Optics and Lasers in Engineering 105 (2018) 54–59
[10] Zuo C, Chen Q, Qu W, Asundi A. Noninterferometric single-shot quantitative phase microscopy. Opt Lett 2013;38:3538–41. [11] Hariharan P, Oreb BF, Eiju T. Digital phase-shifting interferometry: a simple error– compensating phase calculation algorithm. Appl Opt 1987;26:2504–6. [12] Gabor D. A new microscopic principle. Nature 1948;161:777–8. [13] Shack RV, Platt BC. Production and use of a lenticular Hartmann screen. J Opt Soc Am 1971;61:656–60. [14] Gerchberg RW, Saxton WO. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 1972;35:237–46. [15] Fienup JR. Phase retrieval algorithms: A comparison. Appl Opt 1982;21:2758–69. [16] Marchesini S, He H, Chapman HN, Hauriege SP, Noy A, Howells MR, Weierstall U, Spence JCH. X-ray image reconstruction from a diffraction pattern alone. Phys Rev B 2003;68:399–404. [17] Chao W, Harteneck BD, Liddle JA, Anderson EH, Attwood DT. Soft x-ray microscopy at a spatial resolution better than 15 nm. Nature 2005;435:1210–13. [18] Pfeifer MA, Williams GJ, Vartanyants IA, Harder R, Robinson IK. Three-dimensional mapping of a deformation field inside a nanocrystal. Nature 2006;442:63–6. [19] Rodenburg JM, Faulkner HML. A phase retrieval algorithm for shifting illumination. Appl Phys Lett 2004;85:4795–7. [20] Pedrini G, Osten W, Zhang Y. Wave-front reconstruction from a sequence of interferograms recorded at different planes. Opt Lett 2005;30:833–5. [21] Rodrigo JA, Duadi H, Alieva T, Zalevsky Z. Multi-stage phase retrieval algorithm based upon the gyrator transform. Opt Express 2010;18:1510–20. [22] Liu Z, Guo C, Tan J, Wu Q, Pan L, Liu S. Iterative phase amplitude retrieval from multiple images in gyrator domains. J Opt 2015;17:025701. [23] Putkunz CT, D’Alfonso AJ, Morgan AJ, Weyland M, Dwyer C, Bourgeois L, Etheridge J, Roberts A, Scholten RE, Nugent KA, Allen LJ. Atom-scale ptychographic electron diffractive imaging of boron nitride cones. Phys Rev Lett 2012;108:073901. [24] Liu Z, Shen C, Tan J, Liu S. A recovery method of double random phase encoding system with a parallel phase retrieval. IEEE Photonics J 2016;8:7801807. [25] Guo C, Wei C, Tan J, Chen K, Liu S, Wu Q, Liu Z. A review of iterative phase retrieval for measurement and encryption. Opt Lasers Eng 2017;89:2–12.
[26] Shen C, Tan J, Wei C, Liu Z. Coherent diffraction imaging by moving a lens. Opt Express 2016;24:16520–9. [27] Guo C, Li Q, Wei C, Tan J, Liu S, Liu Z. Axial multi-image phase retrieval under tilt illumination. Sci Rep 2017;7:7562. [28] Guo C, Tan J, Liu Z. Precision influence of a phase retrieval algorithm in fractional Fourier domains from position measurement error. Appl Opt 2015;54:6940–7. [29] Shen C, Bao X, Tan J, Liu S, Liu Z. Two noise-robust axial scanning multi-image phase retrieval algorithms based on Pauta criterion and smoothness constraint. Opt Express 2017;25:16235–49. [30] Zhang L, Zhang L, Mou X, Zhang D. FSIM: a feature similarity sndex for image quality assessment. IEEE Trans. Image Process 2011;20:2378–86. [31] https://www.thorlabs.com/thorproduct.cfm?partnumber=R3L3S1N. [32] Greenbaum A, Zhang Y, Feizi A, Chung P, Luo W, Kandukuri SR, Ozcan A. Wide-field computational imaging of pathology slides using lens-free on-chip microscopy. Sci Transl Med 2014;6 267ra175. [33] Delpy DT, Cope M, van der Zee P, Arridge S, Wray S, Wyatt J. Estimation of optical pathlength through tissue from direct time of flight measurement. Phys Med Biol 1988;33:1433–42. [34] He J, Zhou R, Hong Z. Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera. IEEE Trans Consum Electr 2003;49:257–62. [35] Mosso F, Peters E, Pérez DG. Complex wavefront reconstruction from multiple-image planes produced by a focus tunable lens. Opt Lett 2015;40:4623–6. [36] Paganin D, Nugent KA. Noninterferometric phase imaging with partially coherent light. Phys Rev Lett 1998;80:2586–9. [37] Zuo C, Chen Q, Yu Y, Asundi AK. Transport-of-intensity phase imaging using Savitzky-Golay differentiation filter: theory and applications. Opt Express 2013;21:5346–62. [38] Tian L, Waller L. 3D intensity and phase imaging from light field measurements in an LED array microscope. Optica 2015;2:104–11. [39] Dong S, Nanda P, Guo K, Liao J, Zheng G. Incoherent Fourier ptychographic photography using structured light. Photon Res 2015;3:19–23.
59