- Email: [email protected]
A novel sampling moiré method and its application for distortion calibration in scanning electron microscope
Optics and Lasers in Engineering 127 (2020) 105990
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
A novel sampling moiré method and its application for distortion calibration in scanning electron microscope Q. Zhang, H. Xie∗, W. Shi, B. Fan AML, School of Aerospace Engineering, Tsinghua University, Beijing 100084, China
a r t i c l e
i n f o
Keywords: Moiré technique Fringe analysis Grating distortion analysis Scanning electron microscopy Distortion calibration
a b s t r a c t High-resolution scanning electron microscope (SEM) is widely used for microscale material and structure characterization. However, the distortion in SEM imaging causes serious error for the quantitative measurement of optical methods. In previous work, the distortion calibration requires to set up a mathematic model of SEM imaging and then to determine the model parameters by calculation. The distortion filed is hard to directly extract from the SEM image by classic methods. In order to tackle this problem, we developed a direct sampling moiré method (DSM) for simple and fast analysis of grating distortion. In DSM, the distortion fields can be directly presented visually through processing a single grating image. The measurement accuracy of DSM was verified by numerical simulation to be better than the classic sampling moiré. In the experiment, a standard grating with pitch of 6.5 𝜇m was used in DSM to calibrate the distortion fields at different magnifications (50 × , 100 × , 150 × , 300 × ) in SEM. The fitting results of the calibrated distortion field agreed well with the existing distortion model, which demonstrated the feasibility of DSM for distortion calibration in SEM.
1. Introduction In recent years, with the development of micro-machining technology, the characterization of material and structure in microscale becomes increasingly important. Combining the scanning electron microscope (SEM) and optical methods [1–9], such as geometric phase analysis (GPA) [1–3], digital image correlation (DIC) [4–6] and electron microscope moiré methods [7–9], non-contact and full-filed microscale deformation measurement can be achieved. However, with point-wise scanning procedure of SEM imaging, position error of scanning point will occur due to a variety of environmental and systems factors, which limits the measurement reliability of optical methods [10–11]. The SEM imaging error can be divided into two categories, drift distortion and spatial distortion [12], the former is time dependent since it results from environmental vibration, thermal expansion, creep of sample stage and magnetic drift, while the latter is related to spatial scale and location since it results from magnetic lens distortion [13]. Image post processing is a convenient and effective way for SEM image correction [12–16]. Sutton et al. [12] used multiple, time-spaced images and in-plane rigid body motions to extract the drift and spatial distortions of SEM at the same time. The method shows great robustness, but requires complex experimental procedure and iteration computation. In Li et al. ’s work [13], drift and distortion correction are conducted separately. For drift correction, a series of images at the same ∗
location are captured for performing image correlation between the first image and the consequent images, and then the drift–time relationship is obtained by interpolating. Distortion correction employs two images sharing with the same location under different imaging fields to a SEM imaging model for solving the distortion parameters. Both corrections are easy to implement, but the calculation process is still complicated. From the previous work, SEM image correction requires a mathematic model of SEM imaging and a calibration process for determining the model parameters, such as drift velocity and distortion coefficients. There is no method for directly obtaining the distortion field from the SEM image. The reason is that the calibration object is random speckles or natural texture on the surface of the sample, which cannot present a certain morphology with no distortion. Accurate grid and grating targets are appropriate calibration objects [17], but the previous non-parametric distortion calibration methods are difficult to realize at the microscale for SEM [12]. Therefore, a direct and efficient distortion calibration method capable of being applied in micro-scale is essential for SEM image correction. Sampling moiré method is a novel digital processing moiré metrology [18–20]. In this method, the specimen grating images before and after deformation is sampled and interpolated to generate the corresponding phase-shifted moiré patterns, then the deformation filed of specimen grating can be determined through moiré phase analysis. Recent researches shows that this method has advantage of simple
Corresponding author. E-mail address: [email protected] (H. Xie).
https://doi.org/10.1016/j.optlaseng.2019.105990 Received 18 September 2019; Received in revised form 13 November 2019; Accepted 12 December 2019 0143-8166/© 2019 Published by Elsevier Ltd.
Q. Zhang, H. Xie and W. Shi et al.
operation, wide measurement range, low requirement for specimen grating frequency, which has an extensive expecting of being applied to micro-scale measurement [21–23]. However, the classic sampling moiré method is still hard to be used for direct distortion calibration since the grating image without distortion (i.e. reference grating image) cannot be obtained. And thus, there is urgent need for developing new methods for simple and fast analyze of grating distortion in SEM. To meet the above mentioned requirement, the authors developed a direct sampling moiré method (DSM) for analyzing the grating distortion. Based on this method, the distortion filed can be directly extracted from a single SEM image of specimen grating. In the second part, the principle and process of classic sampling moiré method (SM) and DSM is introduced. In the third part, the feasibility of method is verified and the accuracy of DSM is compared with SM through numerical simulation. In the fourth part, DSM is applied to studying the characters and variation of distortion field at different magnifications in SEM. In the fourth part, the theoretical accuracy of selecting sampling pitch in DSM is discussed. 2. Method 2.1. Classic sampling moiré method (SM) The principle of sampling moiré method has been introduced in previous work [23]. The process of sampling moiré method is shown in Fig. 1. The image of specimen grating with pitch of pg is firstly recorded, as shown in Fig. 1(a). The grating intensity can be presented as the following expression: ( ) ( ) 2𝜋𝑥 𝐼 (𝑥) = 𝐼𝑏 + 𝐼𝑎 cos + 𝜙0 = 𝐼𝑏 + 𝐼𝑎 𝑐𝑜𝑠 𝜙𝑔 (1) 𝑝𝑔 where x is the pixel coordinate of the grating; Ib and Ia represent the background intensity and the amplitude of the grating intensity, respectively; 𝜙0 is the initial phase and 𝜙g is the grating phase Then the grating image is sampled in the principal direction of grating with a constant pitch (ps ), called sampling pitch. The sampling points is shown in Fig. 1(b), the blue points and green points represent two
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 2. Phase field of reference moiré fringe pattern.
cases of the sampling results. The difference between them is the position of sampling point, and it is one-pixel difference between the adjacent cases. In order to make the size of the sampled points equal to the grating image, it is extended through spline interpolation, as shown in Fig. 1(c). Different sampling points will generate different moiré patterns, as shown in Fig. 1(d). The generated sampling moiré patterns are actually phase-shifted moiré patterns, and the phase shifting step is 2𝜋/ps . The intensity of the i th phase-shifted sampling moiré patterns can be expressed as: ( ) 𝐼𝑚 (𝑥, 𝑖) = 𝐼𝑏 + 𝐼𝑎 cos 2𝜋𝑥 𝑝1 + 𝜙0 + 𝑖 2𝑝𝜋 𝑚 𝑠 ( ) = 𝐼𝑏 + 𝐼𝑎 cos 𝜙𝑚 + 𝑖 2𝑝𝜋 𝑠 ) ( 𝑖 = 0, 1, 2 ⋯ ⋯ 𝑝𝑠 − 1
(2)
where pm represents the moiré spacing; 𝜙m is the phase of the initial sampling moiré pattern. According to the phase-shifting principle, the phase field of initial sampling moiré pattern can be calculated using the above phase-shifted Fig. 1. flow chart of sampling moiré method.
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 3. Process of classic sampling moiré method (SM) and direct sampling moiré method (DSM).
moiré patterns:
( ) 𝐼𝑚 (𝑥, 𝑖) sin 𝑖 2𝑝𝜋 𝑖 =0 𝑠 𝜙𝑚 (𝑥) = −tan−1 ∑ ( ) 𝑝𝑠 −1 2𝜋 𝑖 𝐼 𝑥, 𝑖 cos ( ) 𝑚 𝑖=0 𝑝 ∑𝑝𝑠 −1
(3)
𝑠
The displacement value is proportional to the grating phase difference before and after deformation, and it is demonstrated that the grating phase difference is equal to the moiré phase difference before and after deformation. Thus the displacement field of specimen grating can be obtained through the moiré phase difference: 𝑢 = 𝑝𝑔
Δ𝜙𝑚 2𝜋
(4)
2.2. Direct sampling moiré method (DSM) In the classic sampling moiré method, the phase field of reference moiré pattern can be presented as: ( ) 1 1 𝜙𝑟 (𝑥) = 2𝜋𝑥 − (5) 𝑝𝑔 𝑝𝑠 As shown in Fig. 2, in case of the sampling pitch being the same as the grating pitch before deformation, the reference moiré phase field will be 𝜙𝑟 = 0, which means the deformed moiré phase field directly reflects the displacement field of specimen grating: 𝑢 = 𝑝𝑔
𝜙𝑟 − 𝜙𝑑 𝜙 = −𝑝𝑔 𝑑 2𝜋 2𝜋
where 𝜙d represents the deformed moiré phase field.
(6)
In the direct sampling moiré method (DSM), the grating pitch before deformation is firstly determined, then only a single deformed grating image is required to calculate the displacement field. From Fig. 3, it can be seen that DSM does not need to solve the mismatch problem of image coordinates before and after deformation, which will improve the efficiency and accuracy. The reference grating pitch can be simply determined according to the actual reference grating period and the pixel scale of image. Besides, it can also be calculated through Fourier frequency analysis [24–25] or sampling moiré phase analysis [26] on the reference grating image. The equation to determine the reference grating pitch through sampling moiré phase field is as following: ( ) 𝜕 𝜙𝑟 (𝑥) 1 1 = 2𝜋 − ≈ 𝜙𝑟 (𝑥 + 1) − 𝜙𝑟 (𝑥) (7) 𝜕𝑥 𝑝𝑔 𝑝𝑠 The determined grating pitch is usually sub-pixel, but the sampling pitch is integer-pixel in classic sampling moiré method. Thus a sub-pixel sampling method for DSM is developed in this study. In the sub-pixel sampling moiré method, the integer pixel closest to the reference grating pitch is firstly determined, then the grating image is zoomed in or zoomed out according to the ratio of integer pixel to the reference grating pitch, the remaining steps are the same as the flow chart in Fig. 1. 3. Simulation verification This section is to verify the measurement accuracy of DSM with subpixel sampling for measuring homogeneous and inhomogeneous deformation. Classic SM and DSM are simultaneously used to determine the Fig. 4. Loading diagram and applied different displacement fields in the simulation.
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 5. Simulation results of DSM and SM in the cases of homogeneous deformation and inhomogeneous deformation.
displacement field. The measurement accuracy of SM is demonstrated to be similar with the Fourier spectrum analysis method in our previous work. The grating image size is 151 × 351 pixels and the grating pitch in x-axis direction is 8.1 pixels. The loading diagram and applied displacement field in the simulation is illustrated in Fig. 4. A numerical uniaxial tension displacement filed is applied as the homogeneous deformation, while half of numerical three-point bending displacement filed is applied as the inhomogeneous deformation. A random Gaussian noise with σ = 0.01 is added to the grating images before and after deformation. The simulation results under a series of loading values are illustrated in Fig. 5(a) and (b), the applied strain of homogeneous deformation filed ranges from 100 to 4000 𝜇ɛ, while the applied inhomogeneous displacement is from equal to U to nine times of U (U is the displacement field in Fig. 4(b)). It can be seen that for homogeneous deformation, the root mean square errors (RMSE) of DSM and SM both remain stable with the applied strain increasing, while for inhomogeneous deformation, the RMSE of DSM and SM both rise slightly with the applied displacement increasing. The RMSE of DSM is less than SM regardless of homogeneous deformation and inhomogeneous deformation, since the calculation of phase difference before and after deformation is not needed in DSM. In order to study the effect of grating pitch on the measurement accuracy, a series of grating images with different grating pitches (8.1 to 36.1 pixels with interval of 4 pixels) are set to be the reference images. From Fig. 5(c) and (d), the RMSE increase significantly with the grating pitch increasing. The reason is that the number of sampling points is reduced with the increasing of grating pitch, which making less effective information in one sampling moiré fringe pattern. Thus it is suggested to select a smaller grating pitch in the optional range of sampling moiré method.
lected as the calibration target, shown in Fig. 6(a). The unidirectional grating with pitch of 6.5 𝜇m was fabricated by UV photolithography technique. The grating image in SEM and grating microstructure in AFM are shown in Fig. 6(b) and (c), respectively. The cross section of grating is rectangular and the depth is about 120 nm, making the SEM grating image with high contrast. For the e-beam, the accelerating voltage is 5 kV, spot size is 3 nm, and dwell time is 10 𝜇s. The SEM image size is 1024 × 884 pixels. The distortion field at a series of magnifications (50 × , 100 × , 150 × , 300 × ) were calibrated in this study.
4. Experiment 4.1. Specimen preparation and experiment setup In the experiment, the SEM imaging was performed in a FEI Quanta450 SEM. A silicon wafer with standard grating on the surface was se-
Fig. 6. (a) Silicon wafer with fabricated grating on the surface. (b) Grating image in SEM at magnification of 500 × . (c) 3D micro-structure of the grating in AFM.
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 7. Distortion calibration process based on sampling moiré method.
Fig. 8. Grating images in two orthogonal directions at different magnifications.
4.2. Distortion calibration process The distortion calibration for SEM image is based on the sampling moiré method. The process is shown in Fig. 7. The grating image is firstly processed by classis SM method to generate sampling moiré patterns and the corresponding phase field. Then the center region in the phase field, with one-tenth of the whole image size (i.e. 100 × 88 pixels), is used to determine the reference grating pitch, because the center region in SEM image is demonstrated to be the region with the smallest distortion. After determining the reference grating pitch, the grating image can be processed by DSM method to generate different moiré patterns and phase field. The phase field obtained by DSM represents the distortion of the grating image relative to reference grating, called distorted phase. Finally, the distorted phase can be converted into distorted filed in pixel according to Eq. (6). 4.3. Calibration results The grating images in two orthogonal directions at different magnifications are shown in Fig. 8. The reference grating pitch of each pair of images are determined by classic SM to be 3.7, 7.4, 11.0, 22.0 pixels, respectively. Then the distorted moiré phase fields and distortion fields can be obtained by DSM, which are illustrated in Figs. 9 and 10, respectively. It can be seen that the distortion reduces with the increasing magnification, and the distribution is small in the middle and large around the sides.
For better comparing the above distortion fields, a quantitative index Δd is applied to describe the magnitude of the distortion, which is as following: ∑𝑁 Δ𝑑 =
𝑖
(𝑑𝑖 − 𝑁
∑𝑁
𝑖=1 𝑑𝑖
𝑁
) (8)
where di is the value of i th point in distortion field, N is the number of the total points. It can be found that Δd is actually the root mean square error of the distortion field. The values of Δd in different cases are shown in Table 1. It is observed that distortion in U field is always less than V field. Besides, with magnification increasing, the variation of Δd becomes smaller, which is more aligned with the variation of spatial distortion, but opposite to the variation of drift distortion. It can be concluded that the spatial distortion contributes more than the drift distortion in this experiment. In SEM imaging process, spatial distortion mainly results from the magnetic lens aberrations. According to theoretical analysis of SEM imaging, a third-order polynomial function can be used to describe the
Table 1 The magnitude of the distortion index in different cases. Magnification
50 ×
100 ×
150 ×
300 ×
Δd in U filed (pixel) Δd in V filed (pixel)
1.27 1.42
0.77 0.95
0.54 0.67
0.42 0.53
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 9. Moiré phase in two orthogonal directions at different magnifications.
Fig. 10. Distortion fields in two orthogonal directions at different magnifications.
Fig. 11. Polynomial fitting results of the distortion fields at magnification of 50 × .
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
In DSM, the condition of sampling pitch being equal to the reference grating pitch was hard to satisfy. Thus a range of sampling pitches neighboring the determined reference grating pitch were used to obtain different moiré phase fields, and then the phase field with the smallest average absolute value is selected to be the best result of DSM. The displacement error influenced by selecting reference grating pitch was discussed through theoretical analysis, which was less than 0.026 pixel in this experiment. Declaration of Competing Interest
Fig. 12. Theoretical displacement error at different values of 𝛼 and pg .
spatial distortion [27–28]. Then third-order polynomial fitting was conducted on the calibrated distortion filed. The fitting result is shown in Fig. 11, which illuminates that the fitting results of the calibrated distortion field agrees well with the proposed distortion model. 5. Discussion The condition of sampling pitch being equal to the reference grating pitch is hard to satisfy in DSM. In this study, after determining the reference grating pitch, a range of neighboring sampling pitches are used to obtain different sampling moiré phase fields. Among these, the phase field with the smallest average absolute value is selected to be the best result of DSM. However, the range of sampling pitches is discrete, which means the sampling pitch can only be approach to the reference grating pitch. If the sampling pitch is not equal to the real reference grating pitch, then the displacement error for selecting reference grating pitch will be influenced by the minimum interval in the range of sampling pitches and the value of reference grating pitch. The equation to calculate the theoretical displacement error is as following: ) ( 𝑝𝑔 1 1 𝛿 = 𝑝𝑔 − =1− (9) 𝑝𝑔 𝑝𝑔 + 𝛼 𝑝𝑔 + 𝛼 where 𝛼 represents the minimum interval in the range of sampling pitches, and pg is the reference grating pitch. The theoretical displacement error at different values of 𝛼 and pg is shown in Fig. 12. It can be seen that the displacement error influenced by selecting reference grating pitch becomes more smaller in the case of smaller 𝛼 and larger pg . In our experiment, 𝛼 is 0.1 pixel and the smallest reference grating pitch is 3.7 pixel, then the theoretical displacement error is less than 0.026 pixel. 6. Conclusion In this study, a novel sampling moiré method, called DSM, was developed for simple and fast analysis of grating distortion. The method is capable of directly extracting the distortion field from a single grating image. In comparison to classic SM, DSM does not need to the calculate the phase difference before and after deformation, which improves the efficiency and accuracy. The measurement accuracy of DSM was verified by numerical simulation in cases of homogeneous and inhomogeneous deformation and the accuracy of DSM was illustrated to be better than classic SM. In the experiment, DSM was applied to study the characters and variation of distortion field at different magnifications in SEM. Through processing the SEM grating images, the distortion fields were directly presented visually by DSM, no longer in need of indirect decoupling calculation. The calibration results showed that with magnification increasing, the distortion reduced and the variation of distortion becomes smaller, which indicated that the spatial distortion contributes more than the drift distortion in the distortion filed. The fitting results of calibrated distortion field agreed well with the existing spatial distortion model.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper The authors declare the following financial interests/personal relationships which may be considered as potential competing interests Acknowledgment This research is financially supported by the National Key Research and Development Program of China with grant no. 2017YFB1103900, National Science and Technology Major Project (2017-VI-0003-0073) and the National Natural Science Foundation of China (grant nos. 11672153 and 11972209). References [1] Hÿtch MJ, Putaux JL, Pénisson JM. Measurement of the displacement field of dislocations to 0.03 Å by electron microscopy. Nature 2003;423(6937):270. [2] Wang Q, Kishimoto S, Xie H, et al. In situ high temperature creep deformation of micro-structure with metal film wire on flexible membrane using geometric phase analysis. Microelectron Reliab 2013;53(4):652–7. [3] Zhang H, Liu Z, Wen H, et al. Subset geometric phase analysis method for deformation evaluation of HRTEM images. Ultramicroscopy 2016;171:34–42. [4] Li N, Sutton MA, Li X, et al. Full-field thermal deformation measurements in a scanning electron microscope by 2D digital image correlation. Exp Mech 2008;48(5):635–46. [5] Zhou Z, Chen P, Huang F, et al. Experimental study on the micromechanical behavior of a PBX simulant using SEM and digital image correlation method. Opt Lasers Eng 2011;49(3):366–70. [6] Pan B, Wang Q. Single-camera microscopic stereo digital image correlation using a diffraction grating. Opt Express 2013;21(21):25056–68. [7] Kishimoto S, Wang Q, Xie H, et al. Study of the surface structure of butterfly wings using the scanning electron microscopic moiré method. Appl Opt 2007;46(28):7026–34. [8] Li Y, Xie H, Tang M, et al. The study on microscopic mechanical property of polycrystalline with SEM moiré method. Opt Lasers Eng 2012;50(12):1757–64. [9] Li C, Liu Z, Xie H, et al. Novel 3D SEM Moiré method for micro height measurement. Opt Express 2013;21(13):15734–46. [10] Sutton MA, Li N, Garcia D, et al. Metrology in a scanning electron microscope: theoretical developments and experimental validation. Meas Sci Technol 2006;17(10):2613. [11] Sutton MA, Li N, Joy DC, et al. Scanning electron microscopy for quantitative small and large deformation measurements part I: sem imaging at magnifications from 200 to 10,000. Exp Mech 2007;47(6):775–87. [12] Sutton MA, Li N, Garcia D, et al. Scanning electron microscopy for quantitative small and large deformation measurements part II: experimental validation for magnifications from 200 to 10,000. Exp Mech 2007;47(6):789–804. [13] Jin P, Li X. Correction of image drift and distortion in a scanning electron microscopy. J Microsc 2015;260(3):268–80. [14] Chen X, Han H, Xie QW, et al. Correction of SEM image distortion based on non-equidistant GM (1, 1) model[C]. In: 2012 International conference on wavelet analysis and pattern recognition. IEEE; 2012. p. 20–3. [15] Guery A, Latourte F, Hild F, et al. Characterization of SEM speckle pattern marking and imaging distortion by digital image correlation. Meas Sci Technol 2013;25(1):015401. [16] Marturi N, Dembélé S, Piat N. Fast image drift compensation in scanning electron microscope using image registration[C]. In: 2013 IEEE International Conference on Automation Science and Engineering (CASE). IEEE; 2013. p. 807–12. [17] Peuchot B. Camera virtual equivalent model 0.01 pixel detectors. Comput Med Imag Graph 1993;17(4–5):289–94. [18] Morimoto Y, Fujigaki M. Accuracy of sampling moiré method[c]. Int Soc Opt Photon 2009;7375:737526. [19] Ri S, Fujigaki M, Morimoto Y. Sampling moiré method for accurate small deformation distribution measurement. Exp Mech 2010;50(4):501–8. [20] Wang Q, Ri S, Tsuda H, et al. Optical full-field strain measurement method from wrapped sampling moiré phase to minimize the influence of defects and its applications. Opt Lasers Eng 2018;110:155–62.
Q. Zhang, H. Xie and W. Shi et al. [21] Wang Q, Ri S, Tsuda H. Digital sampling Moiré as a substitute for microscope scanning Moiré for high-sensitivity and full-field deformation measurement at micron/nano scales. Appl Opt 2016;55(25):6858–65. [22] Zhang Q, Xie H, Liu Z, et al. Sampling moiré method and its application to determine modulus of thermal barrier coatings under scanning electron microscope. Opt Lasers Eng 2018;107:315–24. [23] Wang Q, Ri S, Tsuda H, et al. Two-dimensional Moiré phase analysis for accurate strain distribution measurement and application in crack prediction. Opt Express 2017;25(12):13465–80. [24] Kemao Q. Two-dimensional windowed Fourier transform for fringe pattern analysis: principles, applications and implementations. Opt Lasers Eng 2007;45(2):304–17.
Optics and Lasers in Engineering 127 (2020) 105990 [25] Agarwal N, Kemao Q. Windowed Fourier ridges as a spatial carrier phase-shifting algorithm. Opt Eng 2017;56(8). [26] Ri S, et al. Dynamic thermal deformation measurement of large-scale, high-temperature piping in thermal power plants utilizing the sampling moiré method and grating magnets. Exp Mech 2013;53(9):1635–46. [27] Gnieser D, Frase CG, Bosse H, et al. Model-based correction of image distortion in scanning electron microscopy[C]. In: Key Engineering Materials, 437. Trans Tech Publications; 2010. p. 40–4. [28] Tamura K, Okayama S, Shimizu R. Third-order spherical aberration correction using multistage self-aligned quadrupole correction-lens systems. J Electron Microsc (Tokyo) 2010;59(3):197–206.
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
A novel sampling moiré method and its application for distortion calibration in scanning electron microscope Q. Zhang, H. Xie∗, W. Shi, B. Fan AML, School of Aerospace Engineering, Tsinghua University, Beijing 100084, China
a r t i c l e
i n f o
Keywords: Moiré technique Fringe analysis Grating distortion analysis Scanning electron microscopy Distortion calibration
a b s t r a c t High-resolution scanning electron microscope (SEM) is widely used for microscale material and structure characterization. However, the distortion in SEM imaging causes serious error for the quantitative measurement of optical methods. In previous work, the distortion calibration requires to set up a mathematic model of SEM imaging and then to determine the model parameters by calculation. The distortion filed is hard to directly extract from the SEM image by classic methods. In order to tackle this problem, we developed a direct sampling moiré method (DSM) for simple and fast analysis of grating distortion. In DSM, the distortion fields can be directly presented visually through processing a single grating image. The measurement accuracy of DSM was verified by numerical simulation to be better than the classic sampling moiré. In the experiment, a standard grating with pitch of 6.5 𝜇m was used in DSM to calibrate the distortion fields at different magnifications (50 × , 100 × , 150 × , 300 × ) in SEM. The fitting results of the calibrated distortion field agreed well with the existing distortion model, which demonstrated the feasibility of DSM for distortion calibration in SEM.
1. Introduction In recent years, with the development of micro-machining technology, the characterization of material and structure in microscale becomes increasingly important. Combining the scanning electron microscope (SEM) and optical methods [1–9], such as geometric phase analysis (GPA) [1–3], digital image correlation (DIC) [4–6] and electron microscope moiré methods [7–9], non-contact and full-filed microscale deformation measurement can be achieved. However, with point-wise scanning procedure of SEM imaging, position error of scanning point will occur due to a variety of environmental and systems factors, which limits the measurement reliability of optical methods [10–11]. The SEM imaging error can be divided into two categories, drift distortion and spatial distortion [12], the former is time dependent since it results from environmental vibration, thermal expansion, creep of sample stage and magnetic drift, while the latter is related to spatial scale and location since it results from magnetic lens distortion [13]. Image post processing is a convenient and effective way for SEM image correction [12–16]. Sutton et al. [12] used multiple, time-spaced images and in-plane rigid body motions to extract the drift and spatial distortions of SEM at the same time. The method shows great robustness, but requires complex experimental procedure and iteration computation. In Li et al. ’s work [13], drift and distortion correction are conducted separately. For drift correction, a series of images at the same ∗
location are captured for performing image correlation between the first image and the consequent images, and then the drift–time relationship is obtained by interpolating. Distortion correction employs two images sharing with the same location under different imaging fields to a SEM imaging model for solving the distortion parameters. Both corrections are easy to implement, but the calculation process is still complicated. From the previous work, SEM image correction requires a mathematic model of SEM imaging and a calibration process for determining the model parameters, such as drift velocity and distortion coefficients. There is no method for directly obtaining the distortion field from the SEM image. The reason is that the calibration object is random speckles or natural texture on the surface of the sample, which cannot present a certain morphology with no distortion. Accurate grid and grating targets are appropriate calibration objects [17], but the previous non-parametric distortion calibration methods are difficult to realize at the microscale for SEM [12]. Therefore, a direct and efficient distortion calibration method capable of being applied in micro-scale is essential for SEM image correction. Sampling moiré method is a novel digital processing moiré metrology [18–20]. In this method, the specimen grating images before and after deformation is sampled and interpolated to generate the corresponding phase-shifted moiré patterns, then the deformation filed of specimen grating can be determined through moiré phase analysis. Recent researches shows that this method has advantage of simple
Corresponding author. E-mail address: [email protected] (H. Xie).
https://doi.org/10.1016/j.optlaseng.2019.105990 Received 18 September 2019; Received in revised form 13 November 2019; Accepted 12 December 2019 0143-8166/© 2019 Published by Elsevier Ltd.
Q. Zhang, H. Xie and W. Shi et al.
operation, wide measurement range, low requirement for specimen grating frequency, which has an extensive expecting of being applied to micro-scale measurement [21–23]. However, the classic sampling moiré method is still hard to be used for direct distortion calibration since the grating image without distortion (i.e. reference grating image) cannot be obtained. And thus, there is urgent need for developing new methods for simple and fast analyze of grating distortion in SEM. To meet the above mentioned requirement, the authors developed a direct sampling moiré method (DSM) for analyzing the grating distortion. Based on this method, the distortion filed can be directly extracted from a single SEM image of specimen grating. In the second part, the principle and process of classic sampling moiré method (SM) and DSM is introduced. In the third part, the feasibility of method is verified and the accuracy of DSM is compared with SM through numerical simulation. In the fourth part, DSM is applied to studying the characters and variation of distortion field at different magnifications in SEM. In the fourth part, the theoretical accuracy of selecting sampling pitch in DSM is discussed. 2. Method 2.1. Classic sampling moiré method (SM) The principle of sampling moiré method has been introduced in previous work [23]. The process of sampling moiré method is shown in Fig. 1. The image of specimen grating with pitch of pg is firstly recorded, as shown in Fig. 1(a). The grating intensity can be presented as the following expression: ( ) ( ) 2𝜋𝑥 𝐼 (𝑥) = 𝐼𝑏 + 𝐼𝑎 cos + 𝜙0 = 𝐼𝑏 + 𝐼𝑎 𝑐𝑜𝑠 𝜙𝑔 (1) 𝑝𝑔 where x is the pixel coordinate of the grating; Ib and Ia represent the background intensity and the amplitude of the grating intensity, respectively; 𝜙0 is the initial phase and 𝜙g is the grating phase Then the grating image is sampled in the principal direction of grating with a constant pitch (ps ), called sampling pitch. The sampling points is shown in Fig. 1(b), the blue points and green points represent two
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 2. Phase field of reference moiré fringe pattern.
cases of the sampling results. The difference between them is the position of sampling point, and it is one-pixel difference between the adjacent cases. In order to make the size of the sampled points equal to the grating image, it is extended through spline interpolation, as shown in Fig. 1(c). Different sampling points will generate different moiré patterns, as shown in Fig. 1(d). The generated sampling moiré patterns are actually phase-shifted moiré patterns, and the phase shifting step is 2𝜋/ps . The intensity of the i th phase-shifted sampling moiré patterns can be expressed as: ( ) 𝐼𝑚 (𝑥, 𝑖) = 𝐼𝑏 + 𝐼𝑎 cos 2𝜋𝑥 𝑝1 + 𝜙0 + 𝑖 2𝑝𝜋 𝑚 𝑠 ( ) = 𝐼𝑏 + 𝐼𝑎 cos 𝜙𝑚 + 𝑖 2𝑝𝜋 𝑠 ) ( 𝑖 = 0, 1, 2 ⋯ ⋯ 𝑝𝑠 − 1
(2)
where pm represents the moiré spacing; 𝜙m is the phase of the initial sampling moiré pattern. According to the phase-shifting principle, the phase field of initial sampling moiré pattern can be calculated using the above phase-shifted Fig. 1. flow chart of sampling moiré method.
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 3. Process of classic sampling moiré method (SM) and direct sampling moiré method (DSM).
moiré patterns:
( ) 𝐼𝑚 (𝑥, 𝑖) sin 𝑖 2𝑝𝜋 𝑖 =0 𝑠 𝜙𝑚 (𝑥) = −tan−1 ∑ ( ) 𝑝𝑠 −1 2𝜋 𝑖 𝐼 𝑥, 𝑖 cos ( ) 𝑚 𝑖=0 𝑝 ∑𝑝𝑠 −1
(3)
𝑠
The displacement value is proportional to the grating phase difference before and after deformation, and it is demonstrated that the grating phase difference is equal to the moiré phase difference before and after deformation. Thus the displacement field of specimen grating can be obtained through the moiré phase difference: 𝑢 = 𝑝𝑔
Δ𝜙𝑚 2𝜋
(4)
2.2. Direct sampling moiré method (DSM) In the classic sampling moiré method, the phase field of reference moiré pattern can be presented as: ( ) 1 1 𝜙𝑟 (𝑥) = 2𝜋𝑥 − (5) 𝑝𝑔 𝑝𝑠 As shown in Fig. 2, in case of the sampling pitch being the same as the grating pitch before deformation, the reference moiré phase field will be 𝜙𝑟 = 0, which means the deformed moiré phase field directly reflects the displacement field of specimen grating: 𝑢 = 𝑝𝑔
𝜙𝑟 − 𝜙𝑑 𝜙 = −𝑝𝑔 𝑑 2𝜋 2𝜋
where 𝜙d represents the deformed moiré phase field.
(6)
In the direct sampling moiré method (DSM), the grating pitch before deformation is firstly determined, then only a single deformed grating image is required to calculate the displacement field. From Fig. 3, it can be seen that DSM does not need to solve the mismatch problem of image coordinates before and after deformation, which will improve the efficiency and accuracy. The reference grating pitch can be simply determined according to the actual reference grating period and the pixel scale of image. Besides, it can also be calculated through Fourier frequency analysis [24–25] or sampling moiré phase analysis [26] on the reference grating image. The equation to determine the reference grating pitch through sampling moiré phase field is as following: ( ) 𝜕 𝜙𝑟 (𝑥) 1 1 = 2𝜋 − ≈ 𝜙𝑟 (𝑥 + 1) − 𝜙𝑟 (𝑥) (7) 𝜕𝑥 𝑝𝑔 𝑝𝑠 The determined grating pitch is usually sub-pixel, but the sampling pitch is integer-pixel in classic sampling moiré method. Thus a sub-pixel sampling method for DSM is developed in this study. In the sub-pixel sampling moiré method, the integer pixel closest to the reference grating pitch is firstly determined, then the grating image is zoomed in or zoomed out according to the ratio of integer pixel to the reference grating pitch, the remaining steps are the same as the flow chart in Fig. 1. 3. Simulation verification This section is to verify the measurement accuracy of DSM with subpixel sampling for measuring homogeneous and inhomogeneous deformation. Classic SM and DSM are simultaneously used to determine the Fig. 4. Loading diagram and applied different displacement fields in the simulation.
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 5. Simulation results of DSM and SM in the cases of homogeneous deformation and inhomogeneous deformation.
displacement field. The measurement accuracy of SM is demonstrated to be similar with the Fourier spectrum analysis method in our previous work. The grating image size is 151 × 351 pixels and the grating pitch in x-axis direction is 8.1 pixels. The loading diagram and applied displacement field in the simulation is illustrated in Fig. 4. A numerical uniaxial tension displacement filed is applied as the homogeneous deformation, while half of numerical three-point bending displacement filed is applied as the inhomogeneous deformation. A random Gaussian noise with σ = 0.01 is added to the grating images before and after deformation. The simulation results under a series of loading values are illustrated in Fig. 5(a) and (b), the applied strain of homogeneous deformation filed ranges from 100 to 4000 𝜇ɛ, while the applied inhomogeneous displacement is from equal to U to nine times of U (U is the displacement field in Fig. 4(b)). It can be seen that for homogeneous deformation, the root mean square errors (RMSE) of DSM and SM both remain stable with the applied strain increasing, while for inhomogeneous deformation, the RMSE of DSM and SM both rise slightly with the applied displacement increasing. The RMSE of DSM is less than SM regardless of homogeneous deformation and inhomogeneous deformation, since the calculation of phase difference before and after deformation is not needed in DSM. In order to study the effect of grating pitch on the measurement accuracy, a series of grating images with different grating pitches (8.1 to 36.1 pixels with interval of 4 pixels) are set to be the reference images. From Fig. 5(c) and (d), the RMSE increase significantly with the grating pitch increasing. The reason is that the number of sampling points is reduced with the increasing of grating pitch, which making less effective information in one sampling moiré fringe pattern. Thus it is suggested to select a smaller grating pitch in the optional range of sampling moiré method.
lected as the calibration target, shown in Fig. 6(a). The unidirectional grating with pitch of 6.5 𝜇m was fabricated by UV photolithography technique. The grating image in SEM and grating microstructure in AFM are shown in Fig. 6(b) and (c), respectively. The cross section of grating is rectangular and the depth is about 120 nm, making the SEM grating image with high contrast. For the e-beam, the accelerating voltage is 5 kV, spot size is 3 nm, and dwell time is 10 𝜇s. The SEM image size is 1024 × 884 pixels. The distortion field at a series of magnifications (50 × , 100 × , 150 × , 300 × ) were calibrated in this study.
4. Experiment 4.1. Specimen preparation and experiment setup In the experiment, the SEM imaging was performed in a FEI Quanta450 SEM. A silicon wafer with standard grating on the surface was se-
Fig. 6. (a) Silicon wafer with fabricated grating on the surface. (b) Grating image in SEM at magnification of 500 × . (c) 3D micro-structure of the grating in AFM.
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 7. Distortion calibration process based on sampling moiré method.
Fig. 8. Grating images in two orthogonal directions at different magnifications.
4.2. Distortion calibration process The distortion calibration for SEM image is based on the sampling moiré method. The process is shown in Fig. 7. The grating image is firstly processed by classis SM method to generate sampling moiré patterns and the corresponding phase field. Then the center region in the phase field, with one-tenth of the whole image size (i.e. 100 × 88 pixels), is used to determine the reference grating pitch, because the center region in SEM image is demonstrated to be the region with the smallest distortion. After determining the reference grating pitch, the grating image can be processed by DSM method to generate different moiré patterns and phase field. The phase field obtained by DSM represents the distortion of the grating image relative to reference grating, called distorted phase. Finally, the distorted phase can be converted into distorted filed in pixel according to Eq. (6). 4.3. Calibration results The grating images in two orthogonal directions at different magnifications are shown in Fig. 8. The reference grating pitch of each pair of images are determined by classic SM to be 3.7, 7.4, 11.0, 22.0 pixels, respectively. Then the distorted moiré phase fields and distortion fields can be obtained by DSM, which are illustrated in Figs. 9 and 10, respectively. It can be seen that the distortion reduces with the increasing magnification, and the distribution is small in the middle and large around the sides.
For better comparing the above distortion fields, a quantitative index Δd is applied to describe the magnitude of the distortion, which is as following: ∑𝑁 Δ𝑑 =
𝑖
(𝑑𝑖 − 𝑁
∑𝑁
𝑖=1 𝑑𝑖
𝑁
) (8)
where di is the value of i th point in distortion field, N is the number of the total points. It can be found that Δd is actually the root mean square error of the distortion field. The values of Δd in different cases are shown in Table 1. It is observed that distortion in U field is always less than V field. Besides, with magnification increasing, the variation of Δd becomes smaller, which is more aligned with the variation of spatial distortion, but opposite to the variation of drift distortion. It can be concluded that the spatial distortion contributes more than the drift distortion in this experiment. In SEM imaging process, spatial distortion mainly results from the magnetic lens aberrations. According to theoretical analysis of SEM imaging, a third-order polynomial function can be used to describe the
Table 1 The magnitude of the distortion index in different cases. Magnification
50 ×
100 ×
150 ×
300 ×
Δd in U filed (pixel) Δd in V filed (pixel)
1.27 1.42
0.77 0.95
0.54 0.67
0.42 0.53
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
Fig. 9. Moiré phase in two orthogonal directions at different magnifications.
Fig. 10. Distortion fields in two orthogonal directions at different magnifications.
Fig. 11. Polynomial fitting results of the distortion fields at magnification of 50 × .
Q. Zhang, H. Xie and W. Shi et al.
Optics and Lasers in Engineering 127 (2020) 105990
In DSM, the condition of sampling pitch being equal to the reference grating pitch was hard to satisfy. Thus a range of sampling pitches neighboring the determined reference grating pitch were used to obtain different moiré phase fields, and then the phase field with the smallest average absolute value is selected to be the best result of DSM. The displacement error influenced by selecting reference grating pitch was discussed through theoretical analysis, which was less than 0.026 pixel in this experiment. Declaration of Competing Interest
Fig. 12. Theoretical displacement error at different values of 𝛼 and pg .
spatial distortion [27–28]. Then third-order polynomial fitting was conducted on the calibrated distortion filed. The fitting result is shown in Fig. 11, which illuminates that the fitting results of the calibrated distortion field agrees well with the proposed distortion model. 5. Discussion The condition of sampling pitch being equal to the reference grating pitch is hard to satisfy in DSM. In this study, after determining the reference grating pitch, a range of neighboring sampling pitches are used to obtain different sampling moiré phase fields. Among these, the phase field with the smallest average absolute value is selected to be the best result of DSM. However, the range of sampling pitches is discrete, which means the sampling pitch can only be approach to the reference grating pitch. If the sampling pitch is not equal to the real reference grating pitch, then the displacement error for selecting reference grating pitch will be influenced by the minimum interval in the range of sampling pitches and the value of reference grating pitch. The equation to calculate the theoretical displacement error is as following: ) ( 𝑝𝑔 1 1 𝛿 = 𝑝𝑔 − =1− (9) 𝑝𝑔 𝑝𝑔 + 𝛼 𝑝𝑔 + 𝛼 where 𝛼 represents the minimum interval in the range of sampling pitches, and pg is the reference grating pitch. The theoretical displacement error at different values of 𝛼 and pg is shown in Fig. 12. It can be seen that the displacement error influenced by selecting reference grating pitch becomes more smaller in the case of smaller 𝛼 and larger pg . In our experiment, 𝛼 is 0.1 pixel and the smallest reference grating pitch is 3.7 pixel, then the theoretical displacement error is less than 0.026 pixel. 6. Conclusion In this study, a novel sampling moiré method, called DSM, was developed for simple and fast analysis of grating distortion. The method is capable of directly extracting the distortion field from a single grating image. In comparison to classic SM, DSM does not need to the calculate the phase difference before and after deformation, which improves the efficiency and accuracy. The measurement accuracy of DSM was verified by numerical simulation in cases of homogeneous and inhomogeneous deformation and the accuracy of DSM was illustrated to be better than classic SM. In the experiment, DSM was applied to study the characters and variation of distortion field at different magnifications in SEM. Through processing the SEM grating images, the distortion fields were directly presented visually by DSM, no longer in need of indirect decoupling calculation. The calibration results showed that with magnification increasing, the distortion reduced and the variation of distortion becomes smaller, which indicated that the spatial distortion contributes more than the drift distortion in the distortion filed. The fitting results of calibrated distortion field agreed well with the existing spatial distortion model.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper The authors declare the following financial interests/personal relationships which may be considered as potential competing interests Acknowledgment This research is financially supported by the National Key Research and Development Program of China with grant no. 2017YFB1103900, National Science and Technology Major Project (2017-VI-0003-0073) and the National Natural Science Foundation of China (grant nos. 11672153 and 11972209). References [1] Hÿtch MJ, Putaux JL, Pénisson JM. Measurement of the displacement field of dislocations to 0.03 Å by electron microscopy. Nature 2003;423(6937):270. [2] Wang Q, Kishimoto S, Xie H, et al. In situ high temperature creep deformation of micro-structure with metal film wire on flexible membrane using geometric phase analysis. Microelectron Reliab 2013;53(4):652–7. [3] Zhang H, Liu Z, Wen H, et al. Subset geometric phase analysis method for deformation evaluation of HRTEM images. Ultramicroscopy 2016;171:34–42. [4] Li N, Sutton MA, Li X, et al. Full-field thermal deformation measurements in a scanning electron microscope by 2D digital image correlation. Exp Mech 2008;48(5):635–46. [5] Zhou Z, Chen P, Huang F, et al. Experimental study on the micromechanical behavior of a PBX simulant using SEM and digital image correlation method. Opt Lasers Eng 2011;49(3):366–70. [6] Pan B, Wang Q. Single-camera microscopic stereo digital image correlation using a diffraction grating. Opt Express 2013;21(21):25056–68. [7] Kishimoto S, Wang Q, Xie H, et al. Study of the surface structure of butterfly wings using the scanning electron microscopic moiré method. Appl Opt 2007;46(28):7026–34. [8] Li Y, Xie H, Tang M, et al. The study on microscopic mechanical property of polycrystalline with SEM moiré method. Opt Lasers Eng 2012;50(12):1757–64. [9] Li C, Liu Z, Xie H, et al. Novel 3D SEM Moiré method for micro height measurement. Opt Express 2013;21(13):15734–46. [10] Sutton MA, Li N, Garcia D, et al. Metrology in a scanning electron microscope: theoretical developments and experimental validation. Meas Sci Technol 2006;17(10):2613. [11] Sutton MA, Li N, Joy DC, et al. Scanning electron microscopy for quantitative small and large deformation measurements part I: sem imaging at magnifications from 200 to 10,000. Exp Mech 2007;47(6):775–87. [12] Sutton MA, Li N, Garcia D, et al. Scanning electron microscopy for quantitative small and large deformation measurements part II: experimental validation for magnifications from 200 to 10,000. Exp Mech 2007;47(6):789–804. [13] Jin P, Li X. Correction of image drift and distortion in a scanning electron microscopy. J Microsc 2015;260(3):268–80. [14] Chen X, Han H, Xie QW, et al. Correction of SEM image distortion based on non-equidistant GM (1, 1) model[C]. In: 2012 International conference on wavelet analysis and pattern recognition. IEEE; 2012. p. 20–3. [15] Guery A, Latourte F, Hild F, et al. Characterization of SEM speckle pattern marking and imaging distortion by digital image correlation. Meas Sci Technol 2013;25(1):015401. [16] Marturi N, Dembélé S, Piat N. Fast image drift compensation in scanning electron microscope using image registration[C]. In: 2013 IEEE International Conference on Automation Science and Engineering (CASE). IEEE; 2013. p. 807–12. [17] Peuchot B. Camera virtual equivalent model 0.01 pixel detectors. Comput Med Imag Graph 1993;17(4–5):289–94. [18] Morimoto Y, Fujigaki M. Accuracy of sampling moiré method[c]. Int Soc Opt Photon 2009;7375:737526. [19] Ri S, Fujigaki M, Morimoto Y. Sampling moiré method for accurate small deformation distribution measurement. Exp Mech 2010;50(4):501–8. [20] Wang Q, Ri S, Tsuda H, et al. Optical full-field strain measurement method from wrapped sampling moiré phase to minimize the influence of defects and its applications. Opt Lasers Eng 2018;110:155–62.
Q. Zhang, H. Xie and W. Shi et al. [21] Wang Q, Ri S, Tsuda H. Digital sampling Moiré as a substitute for microscope scanning Moiré for high-sensitivity and full-field deformation measurement at micron/nano scales. Appl Opt 2016;55(25):6858–65. [22] Zhang Q, Xie H, Liu Z, et al. Sampling moiré method and its application to determine modulus of thermal barrier coatings under scanning electron microscope. Opt Lasers Eng 2018;107:315–24. [23] Wang Q, Ri S, Tsuda H, et al. Two-dimensional Moiré phase analysis for accurate strain distribution measurement and application in crack prediction. Opt Express 2017;25(12):13465–80. [24] Kemao Q. Two-dimensional windowed Fourier transform for fringe pattern analysis: principles, applications and implementations. Opt Lasers Eng 2007;45(2):304–17.
Optics and Lasers in Engineering 127 (2020) 105990 [25] Agarwal N, Kemao Q. Windowed Fourier ridges as a spatial carrier phase-shifting algorithm. Opt Eng 2017;56(8). [26] Ri S, et al. Dynamic thermal deformation measurement of large-scale, high-temperature piping in thermal power plants utilizing the sampling moiré method and grating magnets. Exp Mech 2013;53(9):1635–46. [27] Gnieser D, Frase CG, Bosse H, et al. Model-based correction of image distortion in scanning electron microscopy[C]. In: Key Engineering Materials, 437. Trans Tech Publications; 2010. p. 40–4. [28] Tamura K, Okayama S, Shimizu R. Third-order spherical aberration correction using multistage self-aligned quadrupole correction-lens systems. J Electron Microsc (Tokyo) 2010;59(3):197–206.