- Email: [email protected]
Improved high resolution image processing of bright field electron micrographs
Ultramicroscopy North-Holland,
87
17 (1985) 87-104 Amsterdam
IMPROVED HIGH RESOLUTION MICROGRAPHS II. Experiment Earl J. KIRKLAND
IMAGE
PROCESSING
OF BRIGHT
FIELD ELECTRON
and B.M. SIEGEL
School of Applied and Engineering
Physics, Cornell Unioersity, Ithaca, New York 14853, USA
and N. UYEDA Institute
and Y. FUJIYOSHI
for Chemical Research, Kyoto Uniuerslty, r/jr, Kyoto- Fu 611, Japan
Received
1 March
1985
Two new methods of nonlinear image processing are applied to high resolution experimental micrographs of chlorinated copper phthalocyanine taken on the Kyoto 500 kV electron microscope. With these new methods of image processing the phase and amplitude of the specimen transmission function are reconstructed from a defocus series of conventional transmission electron micrographs (bright field). Strong scattering, partial coherence and statistical noise have been included. Both of these new methods are based on the MAP (maximum a posteriori) criterion generalized to include reconstruction from multiple input images. In a companion paper (the first part of this two-part report) the theory behind these methods was presented and in this paper it is tested on actual experimental micrographs. A significant increase in resolution has been obtained with computer image processing. The point-to-point resolution obtained here with computer image processing of 500 kV electron micrographs is of the order of 1.2-1.4 A which represents a 30-50s increase in resolution.
1. Introduction In Part I [l] of this report two new methods of nonlinear image processing were presented. In this paper these method will be applied to actual experimental micrographs of chlorinated copper phthalocyanine taken on the Kyoto 500 kV electron microscope [2,3j. In these new methods of image processing the phase and amplitude of the specimen transmission function may be reconstructed from a defocus series of aberration-limited conventional transmission electron micrographs with a significant increase in resolution: Although these methods of image processing will be applied only to periodic specimens in this report, they are completely general with regard to specimen periodicity and will work with amorphous specimens as well as crystalline specimens. This specimen is a relatively radiation-damage-resistant crystalline 0304-3991/85/$03,30 0 Elsevier Science Publishers (North-Holland Physics Publishing Division)
specimen [4-61 and with simple signal averaging (photographic superposition of 10 unit cells) can be imaged in an essentially aberration-limited mode. The other important mode of image formation from thin specimens, namely radiationdamage-limited imaging [7], will not be discussed in detail here. Please refer to the background discussion and references in refs. [1,8,9], and to the more recent reviews by Reimer [lo] on image formation in the electron microscope and Herrmann [ll] on high resolution instrumentation. Electron microscope image formation is inherently nonlinear. It is only under certain restricted conditions that the electron micrograph image intensity (optical density of electron plate or film) is approximately linear in the phase and/or amplitude of the electron wavefunction (for bright field conventional transmission electron microscopy). To work with thicker specimens and/or at higher B.V.
88
E.J. Kirkland
et al. /
Improved
resolutions it is necessary to consider nonlinear image formation and hence nonlinear image processing. Electron lenses also have rather large aberrations relative to the electron wavelength (of principal concern here is spherical aberration) which severely limit their resolving power. A series of micrographs taken at different defocus (a defocus series) as a whole contains more information than is easily interpretable in any single micrograph. The iterative nonlinear image processing methods presented in Part I [l] of this report and applied to experimental micrographs here reconstruct the complex electron transmission function of the specimen (considered as a real and imaginary image or as an amplitude and phase image) from a defocus series of aberration-limited images. The reconstructed image has all of the information contained in the whole defocus series in a single image that is easily interpreted visually. These methods recover both the amplitude and phase (or real and imaginary parts) of the specimen transmission function and alleviate enough of the effects of the aberrations to yield a significant increase in resolution. This has obvious advantages in atomic structure determination. Both previously presented methods [l] are based on the MAP [12,13] criterion which seeks to maximize the a-posteriori conditional probability of the reconstructed image given a measurement of the degraded image. The MAP method has been generalized to include a reconstruction of a single image from multiple input images [l]. More specifically, both methods involve a nonlinear reconstruction from a defocus series. One method uses an approximate form of partial coherence that allows the FFT (fast Fourier transform) algorithm to be exploited whereas the other uses a more exact formulation of partial coherence [14-161. The FFT approach requires much less computer time and would be appropriate for nearly coherent imaging of thin specimens. As will be shown below, both methods yield consistent results for the particular specimen studied here. 2. Experimental
micrographs
A schematic of the specimen model is shown in fig, 1. The atomic structure was estimated from
high resolutron
imqe
proce.wng.
II
electron diffraction data [17] and standard tables for bond lengths and angles for metal phthalocyanines. The structure of this specimen is thought to be primarily two-dimensional. It consists of planar molecules arranged in parallel planes as shown in fig. 1 (top or bottom view, c-axis perpendicular to plane of paper). These planes of molecules are stacked one plane on top of the other. When the c-axis of the crystal is aligned with the optical axis of the microscope, corresponding atoms of different molecules in different planes line up parallel to the optical axis of the microscope. The resulting images are the superposition of the images of all of the atomic layers (planes of molecules) of the crystal at slightly different defocus values and, if the specimen is thin enough, reproduce the two-dimensional structure of the planar molecules as shown in fig. 1. This is a particularly nice specimen to test image reconstruction methods because it is relatively well characterized and has a known structure with a variety of different spacings (both large and small) which can be used to gauge resolution. As described previously [3,9,18], the specimen was prepared by vacuum evaporation onto a cleaved surface of KCl. The thickness of the specimen was constrained to be less than approximately 50 A (corresponding to a thickness of 15 atomic layers). The specimen was supported on a thin carbon film at an angle of 26.5” to the optic axis of the microscope (c-axis of crystal parallel to optic axis). This causes a defocus difference of aproximately 8.8 A from one unit cell to the next, which is thought to be roughly negligible for the purposes cf this report. Using essentially the same procedure as described previously [3,9,18], a defocus series of micrographs was recorded with an electron optical magnification of 200,000 and a total radiation dose substantially less than the critical dose for this specimen (i.e. much less than 50 C/cm2 ). To increase the signal-to-noise ratio of the images each micrograph was translationally averaged (photographically) over 10 unit cells (increasing S/N by 3.2). The images were translated perpendicular to the direction of tilt so that only images of exactly the same defocus were summed. No symmetry averaging was done. This averaging
89
E.J. Kirkland et al. / Improved high resolution image processing. II
a= 19.62ii
b = 26.04 A c=
3.76 A
p=116.5O
I
-b-
asinP=
I
17.56A
Fig. 1. Specimen model: Projection of hexadecachlorophthalocyanine copper II (CuCl,,PC). optical axis to align the c-axis of the crystal with the optical axis of the microscope.
also effectively attenuates the image of the carbon support film (i.e., the carbon is random and uncorrelated with the specimen lattice constant). A subset of five micrographs out of the total series was chosen. One of these images is the “best focus” or “Scherzer focus” image, and the other four have increasing defocus. Each of these five micrographs was printed onto large (4 X 5 inch) sheet film with accurate alignment with respect to the film edges. A 256 X 384 pixel area of these transparencies corresponding to one unit cell was then digitized with an Optronics P-1700 scanning microdensitometer/filmwriter with a 100 X 100 pm aperture and reduced to 128 x 128 pixels (i.e. demagnified 2 x 3). Because the unit cell dimensions are within 1% of being in a ratio of 2/3, this allows the non-square unit cell to be scanned with a square aperture (as used by the Optronics P1700). The resulting digitized images are shown in fig. 2. Each 128 x 128 image was displayed as 256 x 384 pixels (i.e. magnified 2 X 3) on the P1700 filmwriter (with a 50 X 50 pm aperture). This peculiar scanning method is necessary due to the so-called wrap-around effect of discrete Fourier transforms (i.e. the digital image should be exactly one unit cell).
The specimen
is tilted
26.5’
to the
The electron optical parameters describing this series (fig. 2) are given in table 1. The measurement of the parameters other than defocus and alignment was described previously [9,18]. The relative defocus (defocus difference between micrographs) and alignment between micrographs were refined as part of the image reconstruction procedure. The defocus differences inferred from the tilt of the crystal were used as a starting point. The absolute defocus was determined by comparison to multislice image simulation. The average frequency dependence of the defocus series was described previously [9]. The sum of the magnitudes of the Fourier transforms of all images in the series was azimuthally averaged and Table 1 Electron-optical
parameters
Parameter
Value
Wavelength (A) Spherical aberration (mm) Chromatic aberration (mm) Pixel size (magnification (A)
0.0 142 f 0.0001 1.06 f 0.09 1.6+10%
Defocus spread (i\) Illumination semi-angle
(mrad)
(x)0.1373*1& (.~)0.2060+1% lOOi 0.6 + 0.2
90
E.J. Kirkland et al. / Improved high resolution image proce.wng.
II
Fig. 2. Original defocus series: These experimental micrographs were taken on the Kyoto 500 kV electron microscope. Each micrograph has been photographically averaged over 10 unit cells to Increase the signal-to-noise ratio (S/N). One unit cell is displayed: (a) is the best focus (or Scherzer focus) image. The digitized numerical ranges of the experimental images are: (a) 13 (black) to 207 (white), (b) 12 to 210, (c) 73 to 172, (d) 12 to 211, (e) 73 to 220 (in units of ADC levels).
plotted versus spatial frequency. The average signal-to-noise ratio of the series was estimated to be S/N = 20, and rhe power spectrum was found to have a kp2 frequency dependence, with information extending to a resolution of approximately 1 A. It is one of the aims of computer image processing to recover this high resolution information in a usable form such as an image. 3. Computer system The image processing algorithms presented Part I [l] of this report have been implemented
in on
a Digital Equipment Corp. (DEC) PDP 11/34A computer with floating point hardware and DEC RA80 (121 Mbytes) disk drive. The DEC RSX 11M V4.OB operating system and the DEC FORTRAN 77 V4.1 compiler were also used. All applications programming was done in FORTRAN and I/O device control for non-DEC peripherals was done in machine language via system level device drivers. As a measure of the computational speed of this system, a 128 ~128 complex-to-complex disk-data-buffered FFT (fast Fourier transform) takes approximately 2.5 min. An Optronics P-1700 scanning microdensitometer/filmwriter was used
91
E.J. Kirkland et al. / Improved high resolution image processing. II
for high quality image input and output. A Grinnell GMR-27 (512 x 512 pixels of 8 bits each) refreshed CRT display (frame buffer) was used for high speed intermediate image viewing. This computer system is described in more detail in ref. [19] (note that the Hazeltine 2000, RK03’S, and the PC05 have been eliminated; an RA80 disk and Tektronix 4109 terminal have been added; the software has been updated; and the VT105 has been upgraded to be a VT125).
4. Comparison with multislice
simulation
The previously described [8,9] multislice calculation with the more exact nonlinear partial coherence model [1,14-161, has been fit to the experimental defocus series for comparison. A five-parameter least-squares-type fit was performed using Powell’s method of minimization without derivatives [20,21]. The total squared error-fitting function appears to be a relatively wellbehaved function, so that other types of minimization algorithms might also work well. Two scal-
Fig. 3. Multislice comparison: The best fit of a multislice simulation to the original defocus series, fig. 2; (a) is the best focus (or Scherzer focus) image. The numerical range of the fit is (a) - 19 (black) to 166 (white), (b) - 14 to 197, (c) 5 to 139, (d) - 29 to 168, (e) -30 to 185 (in units of ADC levels). The best fit defocus values are, (a) 439 A, (b) 572 A, (c) 847 A, (d) 1056 A, (e) 1114 A.
92
E.J. Kirkland
et al. / Improved high resolutron imqe
ing constants (one multiplicative and one additive), two translational alignment parameters (one x and one y, neglecting rotation, i.e. x. y translation can be treated as a simple phase factor in Fourier space) as well as defocus were fit (i.e. defocus is a relatively small effect and the other larger effects must be properly dealt with to accurately find defocus). Note that the Scherzer focus position is relatively insensitive to the exact value of defocus so that the Scherzer or best-focus position should not be used to derive other quantitative defocus values if possible. The starting defocus values were 461, 717, 903, 1056, 1191 A, and the final best-fit values were 439, 572, 847, 1056, 1114 A. A maximum aperture of 10 mrad (1.42 A real space resolution) was allowed. The best-fit multislice series is shown in fig. 3. To obtain good agreement of theory and experiment an excessively large temperature factor (B = 32 A) was used in the final convolution with the cross transmission function (not in the actual multislice calculation). This presumably reflects an additional high frequency attenuation as might be due to the MTF of the recording film [22,23] or random misalignments during the photographic translational averaging. In light of the fact that the multislice calculation (fig. 3) ignores the tilt of the specimen and treats the constituent atoms as being free and unbound, the agreement between theory (fig. 3) and experiment (fig. 2) is quite good. For a similar fit using the partial comparison, coherence-FFT approximation image model yields defocus values of 439, 575, 853,1061,1121 A.
5. Image processing Both of the two methods presented in Part I [l] are tested below. The first method uses an approximate form of partial coherence that allows the FFT algorithm to be used and hence runs much faster. The other method uses a more exact formulation of partial coherence [14-161 but takes much more computer time. Each of these methods is tested on the same set of micrographs (fig. 2) using the same starting point. The below results will test the validity of the partial coherence approximation. Furthermore, the previous unrelated
processmg.
II
method of image processing presented in refs. [8,9] was also tested on this same set of experimental micrographs with the same initial starting point (image), so that all three methods may be readily compared. 5.1. Reconstruction upproximation
with the partial
coherence-FFT
The image reconstruction sequence using the partial coherence-FFT approximation method (section 5 of Part I [1]) is shown in figs. 4-7. All five original experimental micrographs (fig. 2) were used. The “best focus” experimental micrograph (fig. 2a) was used as the initial starting point for the image reconstruction algorithm. The starting point image was prepared as in section 4.1,2 of ref. [1]) with a Butterworth low pass filter (e.g. ref. [24]) width of 4.9 mrad and a sharp cutoff at 8 mrad (1.8 A). The optical parameters in table 1 and an S/N of 20 (as measured in ref. [9]) were used. Figs. 4a and 4b are the real and imaginary parts, respectively, of the initial starting point calculated from fig. 2a as described in ref. [l]. Figs. 4c and 4d are the real and imaginary parts, respectively, of the image reconstructed after 20 iterations during which the starting point was also used as the a-priori mean (for the reconstruction algorithm). After 20 iterations the a-priori mean was updated to be the current result and the algorithm was iterated further for a total of 60 iterations. A maximum aperture of 14 mrad (1.01 A) was allowed. The final rms relative error figure of merit was 0.1490 (eq. (25) in Part I [l], revised as in section 5.3). This result is shown in fig. 4e and 4f. The third input micrograph (fig. 2c) was used as the defocus reference pix (i.e. the best fit to the multislice simulation) with a value of 853 A consistent with this image model. The first input micrograph (fig. 2a) was used as the alignment reference. The reference values were held constant and all other defocus values and translational alignment offsets were treated as free parameters and relaxed by the reconstruction algorithm. The converged defocus values were 349, 598, 853, 1110, and 1186 A, and the converged alignment offsets were of the order of one pixel. All 60 iterations
E.J. Kirkland et al. / Improved high resolution image processing. II
took a total of approximately 5 days to calculate. Fig. 5 is the same as fig. 4 except that each image is displayed as an amplitude and phase part. The residue or rms error of the partial coherence-FFT approximation method is shown in fig. 6. This is calculated by reaberrating the reconstructed image (figs. 4e, 4f, 5e and 5f), subtracting the result from the original micrographs (fig. 2) and taking the absolute value. The residue should be essentially the random noise content of the original micrographs. The random nature of
93
the residue indicates an acceptable reconstructed to fig. 2 it image. For an accurate comparison should be realized that the grey scale of fig. 6 has been normalized to fill the available numerical range, which is roughly one third that of fig. 2. If the residues were displayed on the same scale as fig. 2 they would be light grey (fig. 10 in the next section illustrates this effect). The reconstructed phase (fig. 5f) is shown in fig. 7 with some simple image enhancements. Fig. 7a has grey scale histogram equalization (contrast
Fig. 4. Reconstruction sequence (real and imaginary parts) using the partial coherence-FFT approximation method: (a, b) the real and imaginary parts, respectively, of the initial starting point image obtained from the best focus image (fig. 2a); (c. d) real and imaginary parts of the image reconstructed from all five experimental micrographs (figs. 2aa2e) after 20 iterations; (e. f) same as (c, d) but after 60 iterations. The numerical ranges are (a) 5.008 (black) to 12.58 (white), (b) -9.368 to -1.799. (c) 1.874 to 14.01, (d) -10.67 to 0.068, (e) 0.639 to 14.23, (f) -11.63 to 1.369.
/
Improved
high resolution image processing.
II
Fig. 5. Reconstruction sequence (amplitude and phase parts) using the partial coherence-FFT approximation method: This is the same as fig. 4 except the images are displayed as amplitude and phase. The numerical ranges are (a) 10.17 (black) to 12.71 (white), (b) - 1.080 (black) to -0.142 rad (white), (c) 4.189 to 14.13, (d) - 1.349 to 0.006 rad, (e) 2.825 to 14.383, (f) - 1.503 to 0.119 rad.
stretching). Fig. 7b has a high pass Butterworth filter (e.g. ref. [24]) (low spatial frequencies attenuated) and greyscale histogram equalization, and fig. 7c is the reconstructed phase with symmetry averaging (2 mirror planes), a high pass filter, and greyscale histogram equalization. The reconstructed phase image can easily be calibrated in absolute physical units knowing only the incident beam energy (i.e. the phase is a dimensionless quantity) under the thick phase grating approximation (which is probably roughly valid for this specimen, but not extremely accurate). The only required calibration of the recording film is that it be linear in electron intensity. The full scale
range of fig. 5f is 2558 V A (black) to -208 V A (white), which for 15 atomic layers would be 171 V W to - 14 V A per atomic layer. It is also important to point out that the interpretation of the real and imaginary images for heavy/light atom discrimination presented previously [18] may not be applicable here because this particular specimen is too thick (see also the discussion below). 5.2. Reconstruction
with more exact partial
coherence
The image reconstruction sequence using the more exact partial coherence method (section 6 of Part I [l]) is shown in figs. 8-11. This calculation
E.J. Kirkland
et al, / Improved high resolution image processing. II
95
Fig. 6. Residue using the partial coherence-FFT approximation method: This is calculated by reaberrating the reconstructed image (figs. 4e, 4f, 5e and 5f), subtracting the result from the original micrographs (fig. 2) and taking the absolute value; (a) through (e) correspond to (a) through (e) of fig. 2. The numerical ranges are (a) 0.005 (white) to 49.29 (black). (b) 0.671 to 47.58, (c) 0.708 to 72.45, (d) 0.937 to 54.88, (e) 1.188 to 64.04. Note the random nature of the residue indicating an acceptable reconstructed image (i.e., the residue should be the random noise content of the original micrographs).
was carried out under the same conditions as the partial coherence FFT approximation calculation (section 5.1) to accurately compare the two approaches. Figs. S-11 correspond to figs. 4-7 respectively. Figs. 8a and 8b are the initial starting point (same as figs. 4a and 4b), figs. 8c and 8d are the complex image reconstructed after 60 iterations, and figs. 8e and 8f are the final result after 140 iterations. As before, the a-priori mean was the starting point for the first 20 iterations and was
then updated to the current result after 20 iterations. Unlike section 5.1 this update was also performed after 60 iterations. This calculation appears to take an amount of computer time roughly proportional to the fourth power of the maximum aperture. Therefore the first 20 iterations were done with a maximum aperture of 10 mrad (1.41 A), the next 10 were done with a maximum aperture of 12 mrad (1.18 A), and the final 110 iterations were done with a maximum aperture of 14 mrad (1.01 A) to minimize the required computer
E.J. Kirkland
et al
’ Improved high resolution image processing. II
Fig. 7. Simple enhancement of the partial coherence-FFT approximatation method result (phase part, fig. 5f): (a) reconstructed phase with greyscale histogram equalization: (b) reconstructed phase with a high pass filter and greyscale histogram equalization; (c) reconstructed phase with symmetry averaging (two mirror planes), high pass filter and greyscale histogram equalization. Note the outline of the carbon atoms in the benzene ring.
time (i.e. the first few iterations are not really very accurate anyway). A total of 140 iterations were required to obtain an rms relative error figure of merit (0.1496, eq. (25) in Part I [l], and revised in section 5.3) roughly the same as that for the partial coherence-FFT approximation method. The defocus reference image (fig. 2c) was held at a defocus value of 847 A consistent with this image model. Fig. 2a was again used as the alignment
reference. The converged defocus values were 350, 601, 847, 1095, and 1188 A, and the converged alignment offsets were in agreement with the alignment offsets from the partial coherence-FFT method to approximately f0.05 A. Six months were required for this calculation compared to five days with the FFT approximation (i.e., 36 times slower). This dramatically illustrates the power of the FFT. These images are displayed in fig. 9 as amplitude and phase components. The fullscale range of the reconstructed phase image (fig. 9f) is 2723 V A (black) to -472 V A (white), which would be 182 V A to -31 V A per atomic layer, again assuming 15 atomic layers. This is in good agreement with the FFT-partial coherence method in section 5.1. Fig. 10 (compare to fig. 6) is the residue or rms error of the partial coherence-FFT approximation method. Acceptable convergence of the image reconstruction algorithm is again indicated by the random nature of the residue. The grey scales of figs. 10a through 10e have been normalized to fill the whole range of numerical values in the image (which is roughly one third that of the original input image fig. 2). Fig. 1Of is fig. 10a displayed on the same scale as fig. 2a (original micrograph), and more accurately illustrates the actual size of the residue errors (i.e., the grey scale of fig. 2a and fig. 10f are normalized the same way). As was done in fig. 7 the reconstructed phase (fig. 9f) is shown in fig. 11 with some simple image enhancements. Grey scale histogram equalization (contrast stretching), high pass Butterworth filtering (e.g. ref. [24]) (low spatial frequencies attenuated) and symmetry averaging (two mirror planes) were again used. 5.3. Convergence The rms relative error is a quantitative measure of the accuracy of the reconstructed image (figs. 4, 5, 8 and 9). The reconstructed image is simply reaberrated in a manner that should yield back the original defocus series and the rms relative error is the integrated rms error between this reaberrated series and the original defocus series. The rms relative error is graphed in fig. 12 for each method versus the total number of iterations of the respec-
/ Improved high resolution image processing. II
97
Fig. 8. Reconstruction sequence (real and imaginary parts) using the (more) exact partial coherence method: (a, b) the real and imaginary parts, respectively, of the initial starting point image obtained from the best focus image (fig. 2a); (c, d) real and imaginary parts of the image reconstructed from all five experimental micrographs (figs. 2a-2e) after 60 iterations; (e, f) same as (c, d) but after 140 iterations. The numerical ranges are (a) 5.008 (black) to 12.58 (white), (b) - 9.368 to - 1.799, (c) 0.817 to 13.76, (d) - 10.25 to 2.017, (e) -0.226 to 13.95, (f) -10.85 to 3.095.
tive image reconstruction alogorithms. Note that Part I [l], eq. (25) defining the rms relative error, contains a mistake. The right-hand side of the equation should be enclosed by a square root sign (with the apologies of the author), consistent with the definition given in ref. [8]. Both of these curves tend toward a constant value indicating convergence. The final limiting value should be of order of the square root of the ratio of the noise power to the signal plus the noise power. With S/N = 20 the limiting value should be of order 0.218, which it is. In practice a value slightly smaller may be
reached because the noise outside the appropriate aperture is identically zeroed. The value reached here (0.149) is significantly better than the corresponding value obtained with the previously published method [8,9] (0.181). This indicates that the MAP criteria [1,12,13] is superior to the Wiener filter-Newton’s method approach used previously [8,9]. Figs. 13a-13c show the quantitative behavior of various imaging parameters that are relaxed (treated as free parameters) as part of the image reconstruction calculation, versus the total number
E.J. Kirkland
et al. / Improved
high resolution
image processing.
II
Fig. 9. Reconstruction sequence (amplitude and phase parts) using the (more) exact partial coherence method: This is the same as fig. 8 except that the images are displayed as amplitude and phase. The numerical ranges are (a) 10.17 (black) to 12.71 (white). (b) ~ 1.0X0 (black) to -0.142 rad (white), (c) 3.550 to 13.78. (d) - 1.459 to 0.186 rad, (e) 2.486 to 13.98. (f) - 1.600 to 0.277 rad.
of iterations. The solid curves labeled “FFT” are for the partial coherence-FFT approximation method and the dashed curves labeled “no FFT” are for the more exact partial coherence method. Fig. 13a shows the translational alignment offsets for the third micrograph (fig. 2~). Because this offset is done in Fourier space it can be continuous and does not have to be on pixel boundaries (0.1373 x 0.2060 A). This is equivalent to a Fourier interpolation in real space. Fig. 13b shows the defocus value for first micrograph (fig. 2a) which is also the best focus micrograph. For brevity only one example of each of these parameters is shown.
There are actually four pairs of translational offsets and four defocus values (a total of 12 parameters) that are relaxed. The other parameters showed a similar behavior. As discussed in Part I [l], one defocus value (of fig. 2c) and one pair of offsets (fig. 2a) cannot be relaxed and must be held fixed. Fig. 13c shows the relaxed background constant c,, as a function of the total number of iterations. All sets of curves tend to reach a constant value indicating convergence. The defocus and alignment values are consistent between both methods, however the background constant is slightly different. This may be due to the different way the
E.J. Kirkland
et al. / Improved high resolution Image processing. II
99
f Fig. 10. Residule using the (more) exact partial coherence method: This is calculated by reaberrating the reconstructed image (figs. 8e. gf, 9e and 9f), subtracting the result from the original micrographs (fig. 2) and taking the absolute value: (a) through (e) correspond to (a) through (e) of fig. 2. The numerical ranges are (a) 0.002 (white) to 53.16 (black). (b) 0.001 to 49.77, (c) 0.001 to 71.07. (d) 0.767 to 55.87. (e) 0.001 to 60.85. Note the random nature of the residue, indicating an acceptable reconstructed image (i.e.. the residue should be the random noise content of the original micrographs). For comparison (a) has been displayed in (f) with the same numerical range as the average of the original micrographs (fig. 2) of 0 (white) to 191 (black) giving a practical estimate of the absolute rms error of the reconstructed image (figs. 8, 9e and 9f).
overall image amplitude (not phase) is treated with regard to the partial coherence attenuation envelopes.
5.4. Discussion Both of the two new nonlinear methods of image reconstruction proposed in Part I [l] appear to work, as evidenced above. The reconstructed
complex specimen transmission function contains higher resolution information than the original best focus image. In particular, the carbon atoms in the benzene rings are clearly evident as a fuzzy ring in the reconstructed image (figs. 7 and 11) but not in the original defocus series (figs. 2a-2e). The point-to-point resolution in the reconstructed phase image (figs. 7 and 11) is estimated to be 1.2-1.4 A compared to an estimated resolution of 1.8-2 A in
100
E.J. Kirkland
et al. / Improued high resolution mage processmg.
II
E” 0.20 ix 0.10
0.00 1
10 Number
100 of
Iterations
Fig. 12. Quantitative convergence tests: The rms error (i.e.. the normalized integrated intensity of figs. 6 and 10) is plotted versus the number of iterations (see text) for both image reconstruction methods shown in figs. 4411. The solid curve labeled “FFT” is for the partial coherence-FFT approximation method and the dashed curve labeled “no FFT” is for the more exact partial coherence method. Note that each curve aproaches a constant value. indicating convergence. Both methods were iterated until the same approximate value was reached (0.14896 for the partial coherence-FFT approximation method and 0.14964 for the more exact partial coherence method)
Fig. 11. Simple enhancement of the (more) exact partial coherence method result (phase part, fig. 9f): (a) reconstructed phase with greyscale histogram equalization; (b) reconstructed phase with a high pass filter and greyscale histogram equalization: (c) reconstructed phase with symmetry averaging (two mirror planes), high pass filter and greyscale histogram equalization. Note the outline of the benzene ring.
the original best focus or Sherzer focus micrograph fig. 2a (Sherzer resolution is 1.64 A for this microscope). This extra high resolution information comes from the other micrographs in the defocus series. A spherical aberration coefficient of 0.4 mm or less would be required to get the same resolution without image processing. It should be noted that both methods yield a similar result although they are based on different image models. This
indicates that the approximations required to use the FFT algorithm are valid at least for this specimen. A major proportion of the difference between the two image models is a common attenuation envelope which should be roughly carried through to the final result and approximately offset “after the fact” with a simple high pass filter (i.e., fig. 11 has a stronger high pass filter than fig. 7). The method used in refs. [8,9] used the same FFT approximation in the image model but did not yield as good a result. This can now be attributed to the improved performance of the MAP criterion and not to the FFT approximations. Also the actual phase of the specimen transmission function is reconstructed. The phase can be calibrated in absolute V-A units and is easily related to the projected atomic potential of thin specimens. The reconstructed image is not necessarily the exact, unique image of the specimen but is a higher resolution image than the unaided original best focus image. Given a finite signal-tonoise ratio it should be impossible in general to reconstruct the exact image because the noise in
E.J. Kwkland
et al. / Improued high resolution image processing. II
100
10 Number
of
iterations
500 lb
,:
I
300 1
10 Number
100 of
Iterations __----
_ e
_.---
/ I
/
//
//’
‘no FFT
1
10 Number
100 of
Iterations
101
each pixel as well as the signal are unknown, leaving more unknowns than equations. However, a better image may be estimated in a statistical sense. This particular specimen is unfortunately tilted and the image reconstruction methods do not treat a specimen tilt properly. Schiske [25] has proposed some methods for treating tilt, but these ideas have not yet been incorporated. There are thus some image artifacts that may be due to the specimen tilt. The top and bottom parts of the image are at slightly different defocus values (+ 4.4 A). This defocus difference is roughly manifested in the image as a simple defocus effect (the dicrete Fourier transform wraparound effect adds other complications). The top and bottom carbon-chlorine rings therefore may contain artifacts. The left and right rings (in the center of the image) should be more realistic. The image of the heavy copper atom has developed into a donut shape and nearly disappeared in the imaginary part of the reconstructed specimen transmission function (figs. 4e, 4f, 8e and 8f), but reappears in the phase image (figs. 5e, 5f, 9e and 9f). As discussed in refs. [8,9], the specimen is so thick that the normalized projected atomic potential at the center of the copper atoms is large enough to cause a total phase shift of the transmitted electron wave of greater than 77/2 radians. The imaginary part of the specimen transmission function is roughly proportional to the sine of the projected atomic potential of the specimen and the real part is roughly proportional to the cosine. The projected atomic potential increases with both atomic number and specimen thickness. Therefore, when the specimen thickness is such that the normalized projected atomic potential is greater than a/2 radians at the center of the copper atoms (largest projected atomic potential), the contrast in the imaginary part of the reconstructed image is decreasing while the contrast in the real part is still
;
Fig. 13. Image parameter evolution (versus the total number of iterations). (a) The x and y translational alignment offsets for the third micrograph (fig. 2~). The corresponding values for the first micrograph were held constant (i.e. zero), and all four other pairs of offsets were relaxed. The other three pairs behaved similarly but are not shown. (b) The relaxed defocus value for the first micrograph (fig. 2a). The defocus value for
the third micrograph (fig. 2c) was held constant and all four other defocus values were relaxed. The other three values behaved similarity but are not shown. (c) The background constant. In (a)-(c) the solid curves labeled “FFT” correspond to the partial coherence-FFT approximation method (figs. 4-7) and the dashed curves labeled “no FFT” refer to the more exact partial coherence method (figs. 8-11).
102
E.J. Kirkland et (11./ Improoed high resolution imuge processing. II
increasing (as a function of the projected atomic potential). The donut shape results because the projected atomic potential is strongly peaked at the center of the atom. This effect is removed by conversion to an amplitude and phase image (up to a thickness of 277, where the conversion becomes undefined). Because of its simple relationship to the projected atomic potential (for sufficiently thin specimens), the phase of the specimen transmission function is very important. Any method based on a simple linear image reconstruction technique such as the method previously proposed by us [lg] or the dark field/bright field subtraction method proposed by Saxton [26] will not be able to recover the phase image. The so-called (linear) bright field term is linearly degenerate at the origin in reciprocal space and the imaginary part of the DC level of the image can never be found. There is thus an arbitrary background constant that prevents finding the actual phase of the image or specimen transmission function. The image reconstruction methods demonstrated here are based on more exact nonlinear image models in which this problem is overcome. Although it is not as rigorous with regard to the the high resolution image components. the method demonstrated in refs. [8,9] also treats this problem correctly to obtain a phase image. Furthermore, the bright field/dark field subtraction method subtracts two already noisy signals to obtain a third noisier signal and for thick specimens throws away the dominant dark field term. The methods demonstrated above use the information contained in the dark field term. Also it is not always practical or possible (as is the case here) to obtain bright field/dark field image pairs (a defocus series of such pairs) that are aligned. The method demonstrated here does not require any special out-of-the-ordinary operation of the microscope (only a single bright field through focus series is required, which is commonly taken anyway) and is capable of refining the translational alignment by itself. 6. Conclusions In this paper two new nonlinear methods of image processing of aberration-limited conven-
tional transmission electron micrographs have been applied to experimental high resolution micrographs taken on the Kyoto 500 kV electron microscope [2.3]. In these new methods of nonlinear image processing both the phase and amplitude of the complex specimen transmission function may be reconstructed from a series of micrographs taken at different defocus (a defocus series). In a companion paper (the theoretical basis of this report, Part I [l]) these new methods of image processing were presented. They have now been shown to work in practice. These methods appear to take a lot of computer time; however, it should be pointed out that the computer used here is relatively small by today’s standards (the forthcoming generation of microprocessors should outperform this computer), and computers keep improving at an alarming rate. Note also that these methods will always scale as a percentage of the unaided instrumental resolution of the microscope so that newer microscopes will also be able to take advantage of the increase in resolution available with computer image processing. Although this method has only been applied to periodic specimens, the periodicity of the specimen is not used by the method. This new method of image processing is completely general in regard to the specimen periodicity and will work with amorphous as well as periodic specimens. Using this new method of image processing the resolution may be effectively increased by combining several diverse sources of information (several micrographs taken at different defocus and an accurate knowledge of the microscope aberration function and the statistical properties of the noise content of the images) into a form (an image) that is easily interpreted visually. This increase in resolution has obvious advantages in atomic structure determination.
Appendix: Corrections to Part I With the apologies of the author there are three known errors in Part I [l] that are corrected below. The right-hand side of eq. (25) should be enclosed
E.J. Kirkland et al. / Improved high resolution image procesmg.
within
a square root sign as:
The small phase correction factor arising from the partial coherence was displayed with the oposite sign. Eq. (5e) should be: Tp,c(k’. k’-
k) = (u)-“*
exp -rr*azIo]’ [ + rr”a,“A;( 0. k)’
,2A$W
u
4u ia’A
k
u
&,(k’, P X exp (-i[
k’-k)=T,(k’,
k’-k)
vXw - 7;h~A~Xwlkl*/u]
(72a)
+2~*n~v~k+2n4a~A’0v~k~k~*/u).
Eqs. (5e) and (72a) are now corrected to agree with Ishizuka [16] (ref. [47] in Part I [l]).
Acknowledgements This work was supported by the National Science Foundation, US4 (Grant No. ECS-8205894) and the Ministry of Education, Science and Culture, Japan (special equipment and Grants-in-Aid No. 843003 and No. 347012).
References [l] E.J. Kirkland, Ultramicroscopy 15 (1984) 151. [2] K. Kobayashi, E. Suito, N. Uyeda, M. Watanabe, T. Yanaka. T. Etoh, H. Watanabe and M. Moriguci. in: 8th
103
Intern. Congr. on Electron Microscopy, Vol. 1. Eds. J. Sanders and D. Goodchild (Australian Acad. Sci.. Canberra. 1974) p. 30. [31 N. Uyeda. T. Kobayashi, K. Ishizuka and Y. Fujiyoshi. Chem. Scripta 14 (1978-1979) 47. T. Taoke, M. Watanabe, M. Ohara. T. [41 Y. Harada, Kobayashi and N. Uyeda, in: Proc. 28th Annual EMSA Meeting, Ed. C. Arceneaux (Claitor’s. Baton Rouge, LA, 1970) p. 524. and L. Reimer. Bull. Inst. Chem. Res., [51 T. Kobayashi Kyoto Univ. 53 (1975) 105. [cl N. Uyeda. T. Kobnyashi, M. Ohara, M. Watanabe, T. Taoka and Y. Harada, in: Proc. 5th European Congr. on Electron Microscopy, Manchester. 1972, p. 566. [71 E.J. Kirkland, W.T. Freeman. M. Ohtsuki, MS. Isaacson and B.M. Siegel. Ultramicroscopy 6 (1981) 367. 9 (1982) 45. PI E.J. Kirkland, Ultramicroscopy I91 E.J. Kirkland, B.M. Siegel. N. Uyeda and Y. Fujiyoshi. Ultramicroscopy 9 (1982) 65. Electron Microscopy. Physics of UOI L. Reimer. Transmission Image Formation and Microanalysis. Springer Series in Optical Sciences. Vol. 36 (Springer, Berlin. 1984). PII K.H. Herrmann, J. Microscopy 131 (1983) 67. WA B.R. Hunt. IEEE Trans. Computers C-26 (1977) 219. u31 H.J. Trussell and B.R. Hunt, IEEE Trans. Computers C-27 (1979) 57. u41 M. Born and E. Wolf. Principles of Optics. 5th ed. (Pergamon, Oxford, 1975) p. 530. u51 M.A. O’Keefe. in: Proc. 37th Annual EMSA Meeting, Ed. G.W. Bailey (Claitor’s, Baton Rouge, LA. 1979) p. 556. 5 (1980) 55. [161 K. Ishizuka, Ultramicroscopy v71 N. Uyeda. T. Kobayashi and E. Suito, J. Appl. Phys. 43 (1972) 5181. (181 E.J. Kirkland, B.M. Siegel, N. Uyeda and Y. Ftqiyoshi. Ultramicroscopy 5 (1980) 479. 8 (1982) 271. u91 E. Kirkland, Ultramicroscopy WI M.J.D. Powell, Computer J. 7 (1964) 155. WI F.S. Acton. Numerical Methods that Work (Harper and Row, New York, 1970). P21 D.A. Grano and K.H. Downing, in: Proc. 38th Annual EMSA Meeting, Ed. G.W. Bailey (Claitor’s, Baton Rouge. LA, 1980) p. 228. 7 (1982) v31 K.H. Downing and D.A. Grano, Ultramicroscopy 381. ~241 R.C. Gonzalez and P. Wintz, Digital Image Processing (Addison-Wesley, 1977). 9 (1982) 17. ~251 P. Schiske. Ultramicroscopy WI W.O. Saxton. J. Microsc. Spectrosc. Electron. 5 (1980) 661.
1; (54
and eq. (72a) should be:
II
87
17 (1985) 87-104 Amsterdam
IMPROVED HIGH RESOLUTION MICROGRAPHS II. Experiment Earl J. KIRKLAND
IMAGE
PROCESSING
OF BRIGHT
FIELD ELECTRON
and B.M. SIEGEL
School of Applied and Engineering
Physics, Cornell Unioersity, Ithaca, New York 14853, USA
and N. UYEDA Institute
and Y. FUJIYOSHI
for Chemical Research, Kyoto Uniuerslty, r/jr, Kyoto- Fu 611, Japan
Received
1 March
1985
Two new methods of nonlinear image processing are applied to high resolution experimental micrographs of chlorinated copper phthalocyanine taken on the Kyoto 500 kV electron microscope. With these new methods of image processing the phase and amplitude of the specimen transmission function are reconstructed from a defocus series of conventional transmission electron micrographs (bright field). Strong scattering, partial coherence and statistical noise have been included. Both of these new methods are based on the MAP (maximum a posteriori) criterion generalized to include reconstruction from multiple input images. In a companion paper (the first part of this two-part report) the theory behind these methods was presented and in this paper it is tested on actual experimental micrographs. A significant increase in resolution has been obtained with computer image processing. The point-to-point resolution obtained here with computer image processing of 500 kV electron micrographs is of the order of 1.2-1.4 A which represents a 30-50s increase in resolution.
1. Introduction In Part I [l] of this report two new methods of nonlinear image processing were presented. In this paper these method will be applied to actual experimental micrographs of chlorinated copper phthalocyanine taken on the Kyoto 500 kV electron microscope [2,3j. In these new methods of image processing the phase and amplitude of the specimen transmission function may be reconstructed from a defocus series of aberration-limited conventional transmission electron micrographs with a significant increase in resolution: Although these methods of image processing will be applied only to periodic specimens in this report, they are completely general with regard to specimen periodicity and will work with amorphous specimens as well as crystalline specimens. This specimen is a relatively radiation-damage-resistant crystalline 0304-3991/85/$03,30 0 Elsevier Science Publishers (North-Holland Physics Publishing Division)
specimen [4-61 and with simple signal averaging (photographic superposition of 10 unit cells) can be imaged in an essentially aberration-limited mode. The other important mode of image formation from thin specimens, namely radiationdamage-limited imaging [7], will not be discussed in detail here. Please refer to the background discussion and references in refs. [1,8,9], and to the more recent reviews by Reimer [lo] on image formation in the electron microscope and Herrmann [ll] on high resolution instrumentation. Electron microscope image formation is inherently nonlinear. It is only under certain restricted conditions that the electron micrograph image intensity (optical density of electron plate or film) is approximately linear in the phase and/or amplitude of the electron wavefunction (for bright field conventional transmission electron microscopy). To work with thicker specimens and/or at higher B.V.
88
E.J. Kirkland
et al. /
Improved
resolutions it is necessary to consider nonlinear image formation and hence nonlinear image processing. Electron lenses also have rather large aberrations relative to the electron wavelength (of principal concern here is spherical aberration) which severely limit their resolving power. A series of micrographs taken at different defocus (a defocus series) as a whole contains more information than is easily interpretable in any single micrograph. The iterative nonlinear image processing methods presented in Part I [l] of this report and applied to experimental micrographs here reconstruct the complex electron transmission function of the specimen (considered as a real and imaginary image or as an amplitude and phase image) from a defocus series of aberration-limited images. The reconstructed image has all of the information contained in the whole defocus series in a single image that is easily interpreted visually. These methods recover both the amplitude and phase (or real and imaginary parts) of the specimen transmission function and alleviate enough of the effects of the aberrations to yield a significant increase in resolution. This has obvious advantages in atomic structure determination. Both previously presented methods [l] are based on the MAP [12,13] criterion which seeks to maximize the a-posteriori conditional probability of the reconstructed image given a measurement of the degraded image. The MAP method has been generalized to include a reconstruction of a single image from multiple input images [l]. More specifically, both methods involve a nonlinear reconstruction from a defocus series. One method uses an approximate form of partial coherence that allows the FFT (fast Fourier transform) algorithm to be exploited whereas the other uses a more exact formulation of partial coherence [14-161. The FFT approach requires much less computer time and would be appropriate for nearly coherent imaging of thin specimens. As will be shown below, both methods yield consistent results for the particular specimen studied here. 2. Experimental
micrographs
A schematic of the specimen model is shown in fig, 1. The atomic structure was estimated from
high resolutron
imqe
proce.wng.
II
electron diffraction data [17] and standard tables for bond lengths and angles for metal phthalocyanines. The structure of this specimen is thought to be primarily two-dimensional. It consists of planar molecules arranged in parallel planes as shown in fig. 1 (top or bottom view, c-axis perpendicular to plane of paper). These planes of molecules are stacked one plane on top of the other. When the c-axis of the crystal is aligned with the optical axis of the microscope, corresponding atoms of different molecules in different planes line up parallel to the optical axis of the microscope. The resulting images are the superposition of the images of all of the atomic layers (planes of molecules) of the crystal at slightly different defocus values and, if the specimen is thin enough, reproduce the two-dimensional structure of the planar molecules as shown in fig. 1. This is a particularly nice specimen to test image reconstruction methods because it is relatively well characterized and has a known structure with a variety of different spacings (both large and small) which can be used to gauge resolution. As described previously [3,9,18], the specimen was prepared by vacuum evaporation onto a cleaved surface of KCl. The thickness of the specimen was constrained to be less than approximately 50 A (corresponding to a thickness of 15 atomic layers). The specimen was supported on a thin carbon film at an angle of 26.5” to the optic axis of the microscope (c-axis of crystal parallel to optic axis). This causes a defocus difference of aproximately 8.8 A from one unit cell to the next, which is thought to be roughly negligible for the purposes cf this report. Using essentially the same procedure as described previously [3,9,18], a defocus series of micrographs was recorded with an electron optical magnification of 200,000 and a total radiation dose substantially less than the critical dose for this specimen (i.e. much less than 50 C/cm2 ). To increase the signal-to-noise ratio of the images each micrograph was translationally averaged (photographically) over 10 unit cells (increasing S/N by 3.2). The images were translated perpendicular to the direction of tilt so that only images of exactly the same defocus were summed. No symmetry averaging was done. This averaging
89
E.J. Kirkland et al. / Improved high resolution image processing. II
a= 19.62ii
b = 26.04 A c=
3.76 A
p=116.5O
I
-b-
asinP=
I
17.56A
Fig. 1. Specimen model: Projection of hexadecachlorophthalocyanine copper II (CuCl,,PC). optical axis to align the c-axis of the crystal with the optical axis of the microscope.
also effectively attenuates the image of the carbon support film (i.e., the carbon is random and uncorrelated with the specimen lattice constant). A subset of five micrographs out of the total series was chosen. One of these images is the “best focus” or “Scherzer focus” image, and the other four have increasing defocus. Each of these five micrographs was printed onto large (4 X 5 inch) sheet film with accurate alignment with respect to the film edges. A 256 X 384 pixel area of these transparencies corresponding to one unit cell was then digitized with an Optronics P-1700 scanning microdensitometer/filmwriter with a 100 X 100 pm aperture and reduced to 128 x 128 pixels (i.e. demagnified 2 x 3). Because the unit cell dimensions are within 1% of being in a ratio of 2/3, this allows the non-square unit cell to be scanned with a square aperture (as used by the Optronics P1700). The resulting digitized images are shown in fig. 2. Each 128 x 128 image was displayed as 256 x 384 pixels (i.e. magnified 2 X 3) on the P1700 filmwriter (with a 50 X 50 pm aperture). This peculiar scanning method is necessary due to the so-called wrap-around effect of discrete Fourier transforms (i.e. the digital image should be exactly one unit cell).
The specimen
is tilted
26.5’
to the
The electron optical parameters describing this series (fig. 2) are given in table 1. The measurement of the parameters other than defocus and alignment was described previously [9,18]. The relative defocus (defocus difference between micrographs) and alignment between micrographs were refined as part of the image reconstruction procedure. The defocus differences inferred from the tilt of the crystal were used as a starting point. The absolute defocus was determined by comparison to multislice image simulation. The average frequency dependence of the defocus series was described previously [9]. The sum of the magnitudes of the Fourier transforms of all images in the series was azimuthally averaged and Table 1 Electron-optical
parameters
Parameter
Value
Wavelength (A) Spherical aberration (mm) Chromatic aberration (mm) Pixel size (magnification (A)
0.0 142 f 0.0001 1.06 f 0.09 1.6+10%
Defocus spread (i\) Illumination semi-angle
(mrad)
(x)0.1373*1& (.~)0.2060+1% lOOi 0.6 + 0.2
90
E.J. Kirkland et al. / Improved high resolution image proce.wng.
II
Fig. 2. Original defocus series: These experimental micrographs were taken on the Kyoto 500 kV electron microscope. Each micrograph has been photographically averaged over 10 unit cells to Increase the signal-to-noise ratio (S/N). One unit cell is displayed: (a) is the best focus (or Scherzer focus) image. The digitized numerical ranges of the experimental images are: (a) 13 (black) to 207 (white), (b) 12 to 210, (c) 73 to 172, (d) 12 to 211, (e) 73 to 220 (in units of ADC levels).
plotted versus spatial frequency. The average signal-to-noise ratio of the series was estimated to be S/N = 20, and rhe power spectrum was found to have a kp2 frequency dependence, with information extending to a resolution of approximately 1 A. It is one of the aims of computer image processing to recover this high resolution information in a usable form such as an image. 3. Computer system The image processing algorithms presented Part I [l] of this report have been implemented
in on
a Digital Equipment Corp. (DEC) PDP 11/34A computer with floating point hardware and DEC RA80 (121 Mbytes) disk drive. The DEC RSX 11M V4.OB operating system and the DEC FORTRAN 77 V4.1 compiler were also used. All applications programming was done in FORTRAN and I/O device control for non-DEC peripherals was done in machine language via system level device drivers. As a measure of the computational speed of this system, a 128 ~128 complex-to-complex disk-data-buffered FFT (fast Fourier transform) takes approximately 2.5 min. An Optronics P-1700 scanning microdensitometer/filmwriter was used
91
E.J. Kirkland et al. / Improved high resolution image processing. II
for high quality image input and output. A Grinnell GMR-27 (512 x 512 pixels of 8 bits each) refreshed CRT display (frame buffer) was used for high speed intermediate image viewing. This computer system is described in more detail in ref. [19] (note that the Hazeltine 2000, RK03’S, and the PC05 have been eliminated; an RA80 disk and Tektronix 4109 terminal have been added; the software has been updated; and the VT105 has been upgraded to be a VT125).
4. Comparison with multislice
simulation
The previously described [8,9] multislice calculation with the more exact nonlinear partial coherence model [1,14-161, has been fit to the experimental defocus series for comparison. A five-parameter least-squares-type fit was performed using Powell’s method of minimization without derivatives [20,21]. The total squared error-fitting function appears to be a relatively wellbehaved function, so that other types of minimization algorithms might also work well. Two scal-
Fig. 3. Multislice comparison: The best fit of a multislice simulation to the original defocus series, fig. 2; (a) is the best focus (or Scherzer focus) image. The numerical range of the fit is (a) - 19 (black) to 166 (white), (b) - 14 to 197, (c) 5 to 139, (d) - 29 to 168, (e) -30 to 185 (in units of ADC levels). The best fit defocus values are, (a) 439 A, (b) 572 A, (c) 847 A, (d) 1056 A, (e) 1114 A.
92
E.J. Kirkland
et al. / Improved high resolutron imqe
ing constants (one multiplicative and one additive), two translational alignment parameters (one x and one y, neglecting rotation, i.e. x. y translation can be treated as a simple phase factor in Fourier space) as well as defocus were fit (i.e. defocus is a relatively small effect and the other larger effects must be properly dealt with to accurately find defocus). Note that the Scherzer focus position is relatively insensitive to the exact value of defocus so that the Scherzer or best-focus position should not be used to derive other quantitative defocus values if possible. The starting defocus values were 461, 717, 903, 1056, 1191 A, and the final best-fit values were 439, 572, 847, 1056, 1114 A. A maximum aperture of 10 mrad (1.42 A real space resolution) was allowed. The best-fit multislice series is shown in fig. 3. To obtain good agreement of theory and experiment an excessively large temperature factor (B = 32 A) was used in the final convolution with the cross transmission function (not in the actual multislice calculation). This presumably reflects an additional high frequency attenuation as might be due to the MTF of the recording film [22,23] or random misalignments during the photographic translational averaging. In light of the fact that the multislice calculation (fig. 3) ignores the tilt of the specimen and treats the constituent atoms as being free and unbound, the agreement between theory (fig. 3) and experiment (fig. 2) is quite good. For a similar fit using the partial comparison, coherence-FFT approximation image model yields defocus values of 439, 575, 853,1061,1121 A.
5. Image processing Both of the two methods presented in Part I [l] are tested below. The first method uses an approximate form of partial coherence that allows the FFT algorithm to be used and hence runs much faster. The other method uses a more exact formulation of partial coherence [14-161 but takes much more computer time. Each of these methods is tested on the same set of micrographs (fig. 2) using the same starting point. The below results will test the validity of the partial coherence approximation. Furthermore, the previous unrelated
processmg.
II
method of image processing presented in refs. [8,9] was also tested on this same set of experimental micrographs with the same initial starting point (image), so that all three methods may be readily compared. 5.1. Reconstruction upproximation
with the partial
coherence-FFT
The image reconstruction sequence using the partial coherence-FFT approximation method (section 5 of Part I [1]) is shown in figs. 4-7. All five original experimental micrographs (fig. 2) were used. The “best focus” experimental micrograph (fig. 2a) was used as the initial starting point for the image reconstruction algorithm. The starting point image was prepared as in section 4.1,2 of ref. [1]) with a Butterworth low pass filter (e.g. ref. [24]) width of 4.9 mrad and a sharp cutoff at 8 mrad (1.8 A). The optical parameters in table 1 and an S/N of 20 (as measured in ref. [9]) were used. Figs. 4a and 4b are the real and imaginary parts, respectively, of the initial starting point calculated from fig. 2a as described in ref. [l]. Figs. 4c and 4d are the real and imaginary parts, respectively, of the image reconstructed after 20 iterations during which the starting point was also used as the a-priori mean (for the reconstruction algorithm). After 20 iterations the a-priori mean was updated to be the current result and the algorithm was iterated further for a total of 60 iterations. A maximum aperture of 14 mrad (1.01 A) was allowed. The final rms relative error figure of merit was 0.1490 (eq. (25) in Part I [l], revised as in section 5.3). This result is shown in fig. 4e and 4f. The third input micrograph (fig. 2c) was used as the defocus reference pix (i.e. the best fit to the multislice simulation) with a value of 853 A consistent with this image model. The first input micrograph (fig. 2a) was used as the alignment reference. The reference values were held constant and all other defocus values and translational alignment offsets were treated as free parameters and relaxed by the reconstruction algorithm. The converged defocus values were 349, 598, 853, 1110, and 1186 A, and the converged alignment offsets were of the order of one pixel. All 60 iterations
E.J. Kirkland et al. / Improved high resolution image processing. II
took a total of approximately 5 days to calculate. Fig. 5 is the same as fig. 4 except that each image is displayed as an amplitude and phase part. The residue or rms error of the partial coherence-FFT approximation method is shown in fig. 6. This is calculated by reaberrating the reconstructed image (figs. 4e, 4f, 5e and 5f), subtracting the result from the original micrographs (fig. 2) and taking the absolute value. The residue should be essentially the random noise content of the original micrographs. The random nature of
93
the residue indicates an acceptable reconstructed to fig. 2 it image. For an accurate comparison should be realized that the grey scale of fig. 6 has been normalized to fill the available numerical range, which is roughly one third that of fig. 2. If the residues were displayed on the same scale as fig. 2 they would be light grey (fig. 10 in the next section illustrates this effect). The reconstructed phase (fig. 5f) is shown in fig. 7 with some simple image enhancements. Fig. 7a has grey scale histogram equalization (contrast
Fig. 4. Reconstruction sequence (real and imaginary parts) using the partial coherence-FFT approximation method: (a, b) the real and imaginary parts, respectively, of the initial starting point image obtained from the best focus image (fig. 2a); (c. d) real and imaginary parts of the image reconstructed from all five experimental micrographs (figs. 2aa2e) after 20 iterations; (e. f) same as (c, d) but after 60 iterations. The numerical ranges are (a) 5.008 (black) to 12.58 (white), (b) -9.368 to -1.799. (c) 1.874 to 14.01, (d) -10.67 to 0.068, (e) 0.639 to 14.23, (f) -11.63 to 1.369.
/
Improved
high resolution image processing.
II
Fig. 5. Reconstruction sequence (amplitude and phase parts) using the partial coherence-FFT approximation method: This is the same as fig. 4 except the images are displayed as amplitude and phase. The numerical ranges are (a) 10.17 (black) to 12.71 (white), (b) - 1.080 (black) to -0.142 rad (white), (c) 4.189 to 14.13, (d) - 1.349 to 0.006 rad, (e) 2.825 to 14.383, (f) - 1.503 to 0.119 rad.
stretching). Fig. 7b has a high pass Butterworth filter (e.g. ref. [24]) (low spatial frequencies attenuated) and greyscale histogram equalization, and fig. 7c is the reconstructed phase with symmetry averaging (2 mirror planes), a high pass filter, and greyscale histogram equalization. The reconstructed phase image can easily be calibrated in absolute physical units knowing only the incident beam energy (i.e. the phase is a dimensionless quantity) under the thick phase grating approximation (which is probably roughly valid for this specimen, but not extremely accurate). The only required calibration of the recording film is that it be linear in electron intensity. The full scale
range of fig. 5f is 2558 V A (black) to -208 V A (white), which for 15 atomic layers would be 171 V W to - 14 V A per atomic layer. It is also important to point out that the interpretation of the real and imaginary images for heavy/light atom discrimination presented previously [18] may not be applicable here because this particular specimen is too thick (see also the discussion below). 5.2. Reconstruction
with more exact partial
coherence
The image reconstruction sequence using the more exact partial coherence method (section 6 of Part I [l]) is shown in figs. 8-11. This calculation
E.J. Kirkland
et al, / Improved high resolution image processing. II
95
Fig. 6. Residue using the partial coherence-FFT approximation method: This is calculated by reaberrating the reconstructed image (figs. 4e, 4f, 5e and 5f), subtracting the result from the original micrographs (fig. 2) and taking the absolute value; (a) through (e) correspond to (a) through (e) of fig. 2. The numerical ranges are (a) 0.005 (white) to 49.29 (black). (b) 0.671 to 47.58, (c) 0.708 to 72.45, (d) 0.937 to 54.88, (e) 1.188 to 64.04. Note the random nature of the residue indicating an acceptable reconstructed image (i.e., the residue should be the random noise content of the original micrographs).
was carried out under the same conditions as the partial coherence FFT approximation calculation (section 5.1) to accurately compare the two approaches. Figs. S-11 correspond to figs. 4-7 respectively. Figs. 8a and 8b are the initial starting point (same as figs. 4a and 4b), figs. 8c and 8d are the complex image reconstructed after 60 iterations, and figs. 8e and 8f are the final result after 140 iterations. As before, the a-priori mean was the starting point for the first 20 iterations and was
then updated to the current result after 20 iterations. Unlike section 5.1 this update was also performed after 60 iterations. This calculation appears to take an amount of computer time roughly proportional to the fourth power of the maximum aperture. Therefore the first 20 iterations were done with a maximum aperture of 10 mrad (1.41 A), the next 10 were done with a maximum aperture of 12 mrad (1.18 A), and the final 110 iterations were done with a maximum aperture of 14 mrad (1.01 A) to minimize the required computer
E.J. Kirkland
et al
’ Improved high resolution image processing. II
Fig. 7. Simple enhancement of the partial coherence-FFT approximatation method result (phase part, fig. 5f): (a) reconstructed phase with greyscale histogram equalization: (b) reconstructed phase with a high pass filter and greyscale histogram equalization; (c) reconstructed phase with symmetry averaging (two mirror planes), high pass filter and greyscale histogram equalization. Note the outline of the carbon atoms in the benzene ring.
time (i.e. the first few iterations are not really very accurate anyway). A total of 140 iterations were required to obtain an rms relative error figure of merit (0.1496, eq. (25) in Part I [l], and revised in section 5.3) roughly the same as that for the partial coherence-FFT approximation method. The defocus reference image (fig. 2c) was held at a defocus value of 847 A consistent with this image model. Fig. 2a was again used as the alignment
reference. The converged defocus values were 350, 601, 847, 1095, and 1188 A, and the converged alignment offsets were in agreement with the alignment offsets from the partial coherence-FFT method to approximately f0.05 A. Six months were required for this calculation compared to five days with the FFT approximation (i.e., 36 times slower). This dramatically illustrates the power of the FFT. These images are displayed in fig. 9 as amplitude and phase components. The fullscale range of the reconstructed phase image (fig. 9f) is 2723 V A (black) to -472 V A (white), which would be 182 V A to -31 V A per atomic layer, again assuming 15 atomic layers. This is in good agreement with the FFT-partial coherence method in section 5.1. Fig. 10 (compare to fig. 6) is the residue or rms error of the partial coherence-FFT approximation method. Acceptable convergence of the image reconstruction algorithm is again indicated by the random nature of the residue. The grey scales of figs. 10a through 10e have been normalized to fill the whole range of numerical values in the image (which is roughly one third that of the original input image fig. 2). Fig. 1Of is fig. 10a displayed on the same scale as fig. 2a (original micrograph), and more accurately illustrates the actual size of the residue errors (i.e., the grey scale of fig. 2a and fig. 10f are normalized the same way). As was done in fig. 7 the reconstructed phase (fig. 9f) is shown in fig. 11 with some simple image enhancements. Grey scale histogram equalization (contrast stretching), high pass Butterworth filtering (e.g. ref. [24]) (low spatial frequencies attenuated) and symmetry averaging (two mirror planes) were again used. 5.3. Convergence The rms relative error is a quantitative measure of the accuracy of the reconstructed image (figs. 4, 5, 8 and 9). The reconstructed image is simply reaberrated in a manner that should yield back the original defocus series and the rms relative error is the integrated rms error between this reaberrated series and the original defocus series. The rms relative error is graphed in fig. 12 for each method versus the total number of iterations of the respec-
/ Improved high resolution image processing. II
97
Fig. 8. Reconstruction sequence (real and imaginary parts) using the (more) exact partial coherence method: (a, b) the real and imaginary parts, respectively, of the initial starting point image obtained from the best focus image (fig. 2a); (c, d) real and imaginary parts of the image reconstructed from all five experimental micrographs (figs. 2a-2e) after 60 iterations; (e, f) same as (c, d) but after 140 iterations. The numerical ranges are (a) 5.008 (black) to 12.58 (white), (b) - 9.368 to - 1.799, (c) 0.817 to 13.76, (d) - 10.25 to 2.017, (e) -0.226 to 13.95, (f) -10.85 to 3.095.
tive image reconstruction alogorithms. Note that Part I [l], eq. (25) defining the rms relative error, contains a mistake. The right-hand side of the equation should be enclosed by a square root sign (with the apologies of the author), consistent with the definition given in ref. [8]. Both of these curves tend toward a constant value indicating convergence. The final limiting value should be of order of the square root of the ratio of the noise power to the signal plus the noise power. With S/N = 20 the limiting value should be of order 0.218, which it is. In practice a value slightly smaller may be
reached because the noise outside the appropriate aperture is identically zeroed. The value reached here (0.149) is significantly better than the corresponding value obtained with the previously published method [8,9] (0.181). This indicates that the MAP criteria [1,12,13] is superior to the Wiener filter-Newton’s method approach used previously [8,9]. Figs. 13a-13c show the quantitative behavior of various imaging parameters that are relaxed (treated as free parameters) as part of the image reconstruction calculation, versus the total number
E.J. Kirkland
et al. / Improved
high resolution
image processing.
II
Fig. 9. Reconstruction sequence (amplitude and phase parts) using the (more) exact partial coherence method: This is the same as fig. 8 except that the images are displayed as amplitude and phase. The numerical ranges are (a) 10.17 (black) to 12.71 (white). (b) ~ 1.0X0 (black) to -0.142 rad (white), (c) 3.550 to 13.78. (d) - 1.459 to 0.186 rad, (e) 2.486 to 13.98. (f) - 1.600 to 0.277 rad.
of iterations. The solid curves labeled “FFT” are for the partial coherence-FFT approximation method and the dashed curves labeled “no FFT” are for the more exact partial coherence method. Fig. 13a shows the translational alignment offsets for the third micrograph (fig. 2~). Because this offset is done in Fourier space it can be continuous and does not have to be on pixel boundaries (0.1373 x 0.2060 A). This is equivalent to a Fourier interpolation in real space. Fig. 13b shows the defocus value for first micrograph (fig. 2a) which is also the best focus micrograph. For brevity only one example of each of these parameters is shown.
There are actually four pairs of translational offsets and four defocus values (a total of 12 parameters) that are relaxed. The other parameters showed a similar behavior. As discussed in Part I [l], one defocus value (of fig. 2c) and one pair of offsets (fig. 2a) cannot be relaxed and must be held fixed. Fig. 13c shows the relaxed background constant c,, as a function of the total number of iterations. All sets of curves tend to reach a constant value indicating convergence. The defocus and alignment values are consistent between both methods, however the background constant is slightly different. This may be due to the different way the
E.J. Kirkland
et al. / Improved high resolution Image processing. II
99
f Fig. 10. Residule using the (more) exact partial coherence method: This is calculated by reaberrating the reconstructed image (figs. 8e. gf, 9e and 9f), subtracting the result from the original micrographs (fig. 2) and taking the absolute value: (a) through (e) correspond to (a) through (e) of fig. 2. The numerical ranges are (a) 0.002 (white) to 53.16 (black). (b) 0.001 to 49.77, (c) 0.001 to 71.07. (d) 0.767 to 55.87. (e) 0.001 to 60.85. Note the random nature of the residue, indicating an acceptable reconstructed image (i.e.. the residue should be the random noise content of the original micrographs). For comparison (a) has been displayed in (f) with the same numerical range as the average of the original micrographs (fig. 2) of 0 (white) to 191 (black) giving a practical estimate of the absolute rms error of the reconstructed image (figs. 8, 9e and 9f).
overall image amplitude (not phase) is treated with regard to the partial coherence attenuation envelopes.
5.4. Discussion Both of the two new nonlinear methods of image reconstruction proposed in Part I [l] appear to work, as evidenced above. The reconstructed
complex specimen transmission function contains higher resolution information than the original best focus image. In particular, the carbon atoms in the benzene rings are clearly evident as a fuzzy ring in the reconstructed image (figs. 7 and 11) but not in the original defocus series (figs. 2a-2e). The point-to-point resolution in the reconstructed phase image (figs. 7 and 11) is estimated to be 1.2-1.4 A compared to an estimated resolution of 1.8-2 A in
100
E.J. Kirkland
et al. / Improued high resolution mage processmg.
II
E” 0.20 ix 0.10
0.00 1
10 Number
100 of
Iterations
Fig. 12. Quantitative convergence tests: The rms error (i.e.. the normalized integrated intensity of figs. 6 and 10) is plotted versus the number of iterations (see text) for both image reconstruction methods shown in figs. 4411. The solid curve labeled “FFT” is for the partial coherence-FFT approximation method and the dashed curve labeled “no FFT” is for the more exact partial coherence method. Note that each curve aproaches a constant value. indicating convergence. Both methods were iterated until the same approximate value was reached (0.14896 for the partial coherence-FFT approximation method and 0.14964 for the more exact partial coherence method)
Fig. 11. Simple enhancement of the (more) exact partial coherence method result (phase part, fig. 9f): (a) reconstructed phase with greyscale histogram equalization; (b) reconstructed phase with a high pass filter and greyscale histogram equalization: (c) reconstructed phase with symmetry averaging (two mirror planes), high pass filter and greyscale histogram equalization. Note the outline of the benzene ring.
the original best focus or Sherzer focus micrograph fig. 2a (Sherzer resolution is 1.64 A for this microscope). This extra high resolution information comes from the other micrographs in the defocus series. A spherical aberration coefficient of 0.4 mm or less would be required to get the same resolution without image processing. It should be noted that both methods yield a similar result although they are based on different image models. This
indicates that the approximations required to use the FFT algorithm are valid at least for this specimen. A major proportion of the difference between the two image models is a common attenuation envelope which should be roughly carried through to the final result and approximately offset “after the fact” with a simple high pass filter (i.e., fig. 11 has a stronger high pass filter than fig. 7). The method used in refs. [8,9] used the same FFT approximation in the image model but did not yield as good a result. This can now be attributed to the improved performance of the MAP criterion and not to the FFT approximations. Also the actual phase of the specimen transmission function is reconstructed. The phase can be calibrated in absolute V-A units and is easily related to the projected atomic potential of thin specimens. The reconstructed image is not necessarily the exact, unique image of the specimen but is a higher resolution image than the unaided original best focus image. Given a finite signal-tonoise ratio it should be impossible in general to reconstruct the exact image because the noise in
E.J. Kwkland
et al. / Improued high resolution image processing. II
100
10 Number
of
iterations
500 lb
,:
I
300 1
10 Number
100 of
Iterations __----
_ e
_.---
/ I
/
//
//’
‘no FFT
1
10 Number
100 of
Iterations
101
each pixel as well as the signal are unknown, leaving more unknowns than equations. However, a better image may be estimated in a statistical sense. This particular specimen is unfortunately tilted and the image reconstruction methods do not treat a specimen tilt properly. Schiske [25] has proposed some methods for treating tilt, but these ideas have not yet been incorporated. There are thus some image artifacts that may be due to the specimen tilt. The top and bottom parts of the image are at slightly different defocus values (+ 4.4 A). This defocus difference is roughly manifested in the image as a simple defocus effect (the dicrete Fourier transform wraparound effect adds other complications). The top and bottom carbon-chlorine rings therefore may contain artifacts. The left and right rings (in the center of the image) should be more realistic. The image of the heavy copper atom has developed into a donut shape and nearly disappeared in the imaginary part of the reconstructed specimen transmission function (figs. 4e, 4f, 8e and 8f), but reappears in the phase image (figs. 5e, 5f, 9e and 9f). As discussed in refs. [8,9], the specimen is so thick that the normalized projected atomic potential at the center of the copper atoms is large enough to cause a total phase shift of the transmitted electron wave of greater than 77/2 radians. The imaginary part of the specimen transmission function is roughly proportional to the sine of the projected atomic potential of the specimen and the real part is roughly proportional to the cosine. The projected atomic potential increases with both atomic number and specimen thickness. Therefore, when the specimen thickness is such that the normalized projected atomic potential is greater than a/2 radians at the center of the copper atoms (largest projected atomic potential), the contrast in the imaginary part of the reconstructed image is decreasing while the contrast in the real part is still
;
Fig. 13. Image parameter evolution (versus the total number of iterations). (a) The x and y translational alignment offsets for the third micrograph (fig. 2~). The corresponding values for the first micrograph were held constant (i.e. zero), and all four other pairs of offsets were relaxed. The other three pairs behaved similarly but are not shown. (b) The relaxed defocus value for the first micrograph (fig. 2a). The defocus value for
the third micrograph (fig. 2c) was held constant and all four other defocus values were relaxed. The other three values behaved similarity but are not shown. (c) The background constant. In (a)-(c) the solid curves labeled “FFT” correspond to the partial coherence-FFT approximation method (figs. 4-7) and the dashed curves labeled “no FFT” refer to the more exact partial coherence method (figs. 8-11).
102
E.J. Kirkland et (11./ Improoed high resolution imuge processing. II
increasing (as a function of the projected atomic potential). The donut shape results because the projected atomic potential is strongly peaked at the center of the atom. This effect is removed by conversion to an amplitude and phase image (up to a thickness of 277, where the conversion becomes undefined). Because of its simple relationship to the projected atomic potential (for sufficiently thin specimens), the phase of the specimen transmission function is very important. Any method based on a simple linear image reconstruction technique such as the method previously proposed by us [lg] or the dark field/bright field subtraction method proposed by Saxton [26] will not be able to recover the phase image. The so-called (linear) bright field term is linearly degenerate at the origin in reciprocal space and the imaginary part of the DC level of the image can never be found. There is thus an arbitrary background constant that prevents finding the actual phase of the image or specimen transmission function. The image reconstruction methods demonstrated here are based on more exact nonlinear image models in which this problem is overcome. Although it is not as rigorous with regard to the the high resolution image components. the method demonstrated in refs. [8,9] also treats this problem correctly to obtain a phase image. Furthermore, the bright field/dark field subtraction method subtracts two already noisy signals to obtain a third noisier signal and for thick specimens throws away the dominant dark field term. The methods demonstrated above use the information contained in the dark field term. Also it is not always practical or possible (as is the case here) to obtain bright field/dark field image pairs (a defocus series of such pairs) that are aligned. The method demonstrated here does not require any special out-of-the-ordinary operation of the microscope (only a single bright field through focus series is required, which is commonly taken anyway) and is capable of refining the translational alignment by itself. 6. Conclusions In this paper two new nonlinear methods of image processing of aberration-limited conven-
tional transmission electron micrographs have been applied to experimental high resolution micrographs taken on the Kyoto 500 kV electron microscope [2.3]. In these new methods of nonlinear image processing both the phase and amplitude of the complex specimen transmission function may be reconstructed from a series of micrographs taken at different defocus (a defocus series). In a companion paper (the theoretical basis of this report, Part I [l]) these new methods of image processing were presented. They have now been shown to work in practice. These methods appear to take a lot of computer time; however, it should be pointed out that the computer used here is relatively small by today’s standards (the forthcoming generation of microprocessors should outperform this computer), and computers keep improving at an alarming rate. Note also that these methods will always scale as a percentage of the unaided instrumental resolution of the microscope so that newer microscopes will also be able to take advantage of the increase in resolution available with computer image processing. Although this method has only been applied to periodic specimens, the periodicity of the specimen is not used by the method. This new method of image processing is completely general in regard to the specimen periodicity and will work with amorphous as well as periodic specimens. Using this new method of image processing the resolution may be effectively increased by combining several diverse sources of information (several micrographs taken at different defocus and an accurate knowledge of the microscope aberration function and the statistical properties of the noise content of the images) into a form (an image) that is easily interpreted visually. This increase in resolution has obvious advantages in atomic structure determination.
Appendix: Corrections to Part I With the apologies of the author there are three known errors in Part I [l] that are corrected below. The right-hand side of eq. (25) should be enclosed
E.J. Kirkland et al. / Improved high resolution image procesmg.
within
a square root sign as:
The small phase correction factor arising from the partial coherence was displayed with the oposite sign. Eq. (5e) should be: Tp,c(k’. k’-
k) = (u)-“*
exp -rr*azIo]’ [ + rr”a,“A;( 0. k)’
,2A$W
u
4u ia’A
k
u
&,(k’, P X exp (-i[
k’-k)=T,(k’,
k’-k)
vXw - 7;h~A~Xwlkl*/u]
(72a)
+2~*n~v~k+2n4a~A’0v~k~k~*/u).
Eqs. (5e) and (72a) are now corrected to agree with Ishizuka [16] (ref. [47] in Part I [l]).
Acknowledgements This work was supported by the National Science Foundation, US4 (Grant No. ECS-8205894) and the Ministry of Education, Science and Culture, Japan (special equipment and Grants-in-Aid No. 843003 and No. 347012).
References [l] E.J. Kirkland, Ultramicroscopy 15 (1984) 151. [2] K. Kobayashi, E. Suito, N. Uyeda, M. Watanabe, T. Yanaka. T. Etoh, H. Watanabe and M. Moriguci. in: 8th
103
Intern. Congr. on Electron Microscopy, Vol. 1. Eds. J. Sanders and D. Goodchild (Australian Acad. Sci.. Canberra. 1974) p. 30. [31 N. Uyeda. T. Kobayashi, K. Ishizuka and Y. Fujiyoshi. Chem. Scripta 14 (1978-1979) 47. T. Taoke, M. Watanabe, M. Ohara. T. [41 Y. Harada, Kobayashi and N. Uyeda, in: Proc. 28th Annual EMSA Meeting, Ed. C. Arceneaux (Claitor’s. Baton Rouge, LA, 1970) p. 524. and L. Reimer. Bull. Inst. Chem. Res., [51 T. Kobayashi Kyoto Univ. 53 (1975) 105. [cl N. Uyeda. T. Kobnyashi, M. Ohara, M. Watanabe, T. Taoka and Y. Harada, in: Proc. 5th European Congr. on Electron Microscopy, Manchester. 1972, p. 566. [71 E.J. Kirkland, W.T. Freeman. M. Ohtsuki, MS. Isaacson and B.M. Siegel. Ultramicroscopy 6 (1981) 367. 9 (1982) 45. PI E.J. Kirkland, Ultramicroscopy I91 E.J. Kirkland, B.M. Siegel. N. Uyeda and Y. Fujiyoshi. Ultramicroscopy 9 (1982) 65. Electron Microscopy. Physics of UOI L. Reimer. Transmission Image Formation and Microanalysis. Springer Series in Optical Sciences. Vol. 36 (Springer, Berlin. 1984). PII K.H. Herrmann, J. Microscopy 131 (1983) 67. WA B.R. Hunt. IEEE Trans. Computers C-26 (1977) 219. u31 H.J. Trussell and B.R. Hunt, IEEE Trans. Computers C-27 (1979) 57. u41 M. Born and E. Wolf. Principles of Optics. 5th ed. (Pergamon, Oxford, 1975) p. 530. u51 M.A. O’Keefe. in: Proc. 37th Annual EMSA Meeting, Ed. G.W. Bailey (Claitor’s, Baton Rouge, LA. 1979) p. 556. 5 (1980) 55. [161 K. Ishizuka, Ultramicroscopy v71 N. Uyeda. T. Kobayashi and E. Suito, J. Appl. Phys. 43 (1972) 5181. (181 E.J. Kirkland, B.M. Siegel, N. Uyeda and Y. Ftqiyoshi. Ultramicroscopy 5 (1980) 479. 8 (1982) 271. u91 E. Kirkland, Ultramicroscopy WI M.J.D. Powell, Computer J. 7 (1964) 155. WI F.S. Acton. Numerical Methods that Work (Harper and Row, New York, 1970). P21 D.A. Grano and K.H. Downing, in: Proc. 38th Annual EMSA Meeting, Ed. G.W. Bailey (Claitor’s, Baton Rouge. LA, 1980) p. 228. 7 (1982) v31 K.H. Downing and D.A. Grano, Ultramicroscopy 381. ~241 R.C. Gonzalez and P. Wintz, Digital Image Processing (Addison-Wesley, 1977). 9 (1982) 17. ~251 P. Schiske. Ultramicroscopy WI W.O. Saxton. J. Microsc. Spectrosc. Electron. 5 (1980) 661.
1; (54
and eq. (72a) should be:
II