Local regularization of tilt projections reduces artifacts in electron tomography

Journal of Structural Biology 186 (2014) 28–37

Contents lists available at ScienceDirect

Journal of Structural Biology journal homepage: www.elsevier.com/locate/yjsbi

Local regularization of tilt projections reduces artifacts in electron tomography Mauro Maiorca a,b,⇑, Coralie Millet a,b, Eric Hanssen a,b,d, Brian Abbey b,c, Edmund Kazmierczak e,1, Leann Tilley a,b,⇑,1 a

Department of Biochemistry and Molecular Biology, Bio21 Molecular Science and Biotechnology Institute, The University of Melbourne, Vic. 3010, Australia Australian Research Council Centre of Excellence for Coherent X-ray Science, Australia Department of Physics, La Trobe University, Vic. 3071, Australia d Electron Microscopy Unit, Bio21 Molecular Science and Biotechnology Institute, The University of Melbourne, Melbourne, Vic. 3010, Australia e Department of Computer Science and Software Engineering, The University of Melbourne, Vic. 3010, Australia b c

a r t i c l e

i n f o

Article history: Received 17 December 2013 Received in revised form 5 March 2014 Accepted 10 March 2014 Available online 14 March 2014 Keywords: Non-linear diffusion Electron tomography Gold artifact reduction Plasmodium Image reconstruction

a b s t r a c t Electron tomography produces very high resolution 3D image volumes useful for investigating the structure and function of cellular components. Unfortunately, unavoidable discontinuities and physical constraints in the acquisition geometry lead to a range of artifacts that can affect the reconstructed image. In particular, highly electron dense regions, such as gold nanoparticles, can hide proximal biological structures and degrade the overall quality of the reconstructed tomograms. In this work we introduce a pre-reconstruction non-conservative non-linear isotropic diffusion (NID) filter that automatically identifies and reduces local irregularities in the tilt projections. We illustrate the improvement in quality obtained using this approach for reconstructed tomograms generated from samples of malaria parasite-infected red blood cells. A quantitative and qualitative evaluation for our approach on both simulated and real data is provided. Ó 2014 Elsevier Inc. All rights reserved.

1. Introduction Electron tomography (ET) is an important technique for the high-resolution imaging of 3D structure for a large range of life and materials science samples. In particular, the application of ET to elucidate the 3D architecture of biological samples provides new insights into their function (Downing et al., 2007; Leis et al., 2009; Milne and Subramaniam, 2009; Vanhecke et al., 2011). The method involves generating sections of tissues or cells (usually 100–400 nm) and preparing the sections for electron microscopy (EM). In many cases the samples are processed using chemical fixation, plastic embedding and staining with heavy metals (Frey et al., 2006; Gan and Jensen, 2012; McIntosh et al., 2005). Alternatively samples can be snap-frozen, sectioned if required, and examined using a cryo-EM stage (Dubochet et al., 1988; McDonald and Abbreviations: NID, non-linear isotropic diffusion; ET, electron tomography; FBP, filtered back projection; SIRT, simultaneous iterative reconstruction techniques. ⇑ Corresponding authors at: Department of Biochemistry and Molecular Biology, Bio21 Molecular Science and Biotechnology Institute, The University of Melbourne, Vic. 3010, Australia. E-mail addresses: [email protected] (M. Maiorca), ltilley@unimelb. edu.au (L. Tilley). 1 These authors contributed equally. http://dx.doi.org/10.1016/j.jsb.2014.03.009 1047-8477/Ó 2014 Elsevier Inc. All rights reserved.

Auer, 2006; Pierson et al., 2011). In some cases labeling of specific subcellular compartments or proteins is achieved using immunogold-tagged antibodies or protein A. This can be performed either before or after fixation and processing of the samples (Hanssen et al., 2010b). Once the biological sample sections have been prepared they are examined in an electron microscope with a tiltable stage operating at accelerating voltages of 120–300 kV. Images of the samples are collected at different projection angles over a tilt range of at least 120°. The imaging dose is kept to a minimum, and for cryo-stabilized samples should be less than 10,000 e/nm2, in an effort to prevent beam damage and thus preserve the biological architecture. After tilt series acquisition, the images are aligned, often making use of colloidal gold fiducial particles deposited onto the sample for this purpose (Kremer et al., 1996; Mastronarde, 1997; Penczek et al., 1995). The final 3D sample volume is obtained using volumetric reconstruction techniques (Herman, 2009) and processed using different segmentation tools to identify specific features (Ali et al., 2012; Kremer et al., 1996; Mumcuoglu et al., 2012; Nguyen and Ji, 2008). The requirement for low electron doses leading to short exposure times means that noise-related artifacts are an important issue in ET and can severely degrade the quality of the

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37

reconstruction (Arslan et al., 2006; Baumeister et al., 1999; Cao et al., 2010; Fernández and Martínez, 2010; Frangakis and Hegerl, 2001; van der Heide et al., 2007). The development of filtering algorithms can help to improve the quality of the reconstructed images and if coupled with further processing such as segmentation and rendering can help to identify important biological structures. Improving the quality of reconstructed tomograms for further processing is consequently a topic of key importance. The physical structure of the microscope stage where samples are mounted limits the angles at which the sample can be tilted, from about 70° to +70°. The limited range of angles in turn means that the sequence of tomographic tilt projections have a ‘‘wedge’’ of missing data in the Fourier domain (Saghi and Midgley, 2012). The result is that there is ‘‘streaking’’ of features where the signal is poor in the direction of the missing wedge (Radermacher, 1992). A particular problem arises where regions of high electron absorption (or scattering) are located proximal to regions of low absorption. Gold nanoparticles embedded in biological tissues for immunolabelling purposes or as fiducial markers for aligning tilt projections represent such a case. The strongly scattering gold particles can produce exponential edge-gradient effects that obscure features from surrounding regions. Tomograms reconstructed from such projection images exhibit streak artifacts around the gold particles such that the tomograms in these regions cannot be used for obtaining reliable quantitative information. For cryo-ET, where the biological features under investigation have particularly low contrast, such artifacts pose significant challenges when trying to interpret the reconstructed data. One way of reducing streak artifacts is by interpolation of data in the tilt projections. For example a family of partial diffusion equations has been used to create interpolated missing regions in pre-reconstruction data (Köstler et al., 2006), and feature-driven adaptive interpolation has been used to reduce streak artifacts in ET volumes (Cao et al., 2010). However these processes, and the method of interpolation itself, can lead to the introduction of new artifacts. In the case of ET of biological samples the high noise levels and low contrast of the observed biological structures mean that the usefulness of adaptive interpolation methods is limited. Methods for removing the contribution from regions of high electron density prior to reconstruction have been developed using interpolation in-painting or simply by replacing the regions of high signal with pixels of an average gray value (Cope et al., 2011). These techniques can be successful when the high-signal regions are distal from the structures of interest. Unfortunately, these approaches have limited usefulness for regularization of the artifacts produced by the gold particles used for immunolabelling in ET of biological samples. This is because the gold particles are, by design, proximal to the structures of interest (Jimenez and Post, 2012; Koster and Klumperman, 2003; Moritz et al., 1995). Modifying the acquisition geometry, for example, by collecting dual tilt series (Guesdon et al., 2013) has been shown to reduce the problem, but some distortion of the image still persists. A third approach to solving the problem of streaking due to the presence of the gold particles is via deconvolution of the reconstructed image with an estimated Point Spread Function (PSF) (Tchelidze et al., 2006). Unfortunately an accurate estimate for the PSF is not straightforward in ET as it depends on the specific acquisition geometry and settings as well as the type of reconstruction algorithm used, which may vary substantially between different studies. Our group recently introduced a method for improving the quality of ET reconstructions based on a local electron density redistribution using a conservative diffusive approach (Maiorca et al., 2012). This type of approach is well-established for conventional image processing and filtering (Weickert, 1996a). This method, when applied to ET reduces the degradation of reconstructed

29

volumes, whilst minimizing reconstruction artifacts and facilitating the supervized segmentation of the data (McMillan et al., 2013). However, a conservative approach is not sufficient when attempting to recover the signal from regions affected by severe local inhomogeneities in electron absorption. In the present paper we propose a non-conservative approach that specifically targets the highly non-regular regions surrounding the gold nanoparticles without affecting the rest of the image. 2. Materials and methods 2.1. Sample preparation and ET Plasmodium falciparum (3D7 strain) was cultured in human red blood cells using established protocols (Spycher et al., 2008). Trophozoite-infected red blood cells were harvested, fixed, embedded in epoxy resin, and sectioned (100–250 nm) as described previously (Hanssen et al., 2010a). In some cases infected red blood cells were permeabilized with Equinatoxin II (EqtII) (Hanssen et al., 2008), before labeling with primary antibodies and gold-labeled protein A (6 nm Aurion). The ET sections were stained and layered with gold fiducials (10 nm). Grids were mounted on a single tile holder and imaged using a Tecnai G2 F30 transmission electron microscope operating at 300 kV (Advanced Microscopy Facility, Bio21 Institute, Melbourne). Tilt projections were recorded between about 66° and +66° at 2° intervals and aligned with IMOD (http://bio3d.colorado.edu/imod/) (Kremer et al., 1996; Mastronarde, 1997), as described previously (Abu Bakar et al., 2010; Hanssen et al., 2008). Volumes were reconstructed using Filtered Back Projection (FBP, IMOD (Kremer et al., 1996)) or Simultaneous Iterative Reconstruction Techniques (SIRT, tomo3d (Agulleiro and Fernandez, 2011)). Erasing gold particles using a fiducial model (Cope et al., 2011) was performed using the Ccderaser program in IMOD. 2.2. Simulation A synthetic volume of 256  256  64 voxels was created, using a simulated pixel spacing of 1 nm. Pixel intensity values were uniformly distributed within 295 grayscale levels, with zero mean for each tilt projection. The volume contained three normally distributed classes of pixel intensities values with interclass distances of 242.5 and 23 grayscale levels, full width at half maximum (FWHM) of 50, 35 and 15 grayscale levels, designed to represent respectively gold nanoparticles, a double membrane, and the background. The gold nanoparticles were modeled as hard spheres with a diameter of 10 pixels, whilst the double membrane was simulated using two 3D-sinc functions of the form:

sin

ðxÞ ; x 2 R3 x

ð1Þ

The sample structure was rendered using IMOD and is presented in Fig. 1. To create a simulated dataset, projections of the synthetic volume were generated using Xmipp (Sorzano et al., 2004). The final volumes were reconstructed using both FBP (Kremer et al., 1996) and SIRT (Agulleiro and Fernandez, 2011). 2.3. Local regularization In tilt projection images, an important source of intensity irregularity arises from local spatial differences in the electron density within the sample. Scale-space theory allows the representation of such multi-scale signals as a series of smoothed images parameterized by a smoothing kernel known as the ‘scale parameter’. Within this theory local spatial inhomogeneities can be determined by convolution with a Gaussian characterized by the

30

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37

Fig.1. Simulated double membrane shaped as a ‘‘Mexican hat’’ (3D-sinc) with random gold particles near the membrane. (a) 3D rendering of the simulated double membrane, with each z coordinate represented with a different color. (b–f) Simulated tilt projections at different angles, respectively 90°, 70°, 0°, 70°, and 90°. The simulated gold particles are evident as back dots. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

appropriate scale parameter, which is typically the variance r. Since the Gaussian is a smoothly varying function, the derivative of the convolved image is well-defined. This enables the definition of features as differential invariants; that is features that are invariant under a variety of differential operators. These basic image processing concepts underpin the following analysis. If I(x) is the intensity measured for the imaged specimen represented as a grayscale image, the convolution of I(x) with a Gaussian smoothing kernel results in a blurred image Ir(x):

Ir ðxÞ ¼ ðI  Gr ÞðxÞ

ð2Þ

Where ⁄ is the convolution operator, and Gr denotes the twodimensional Gaussian of variance r > 0 given by:

1 jxj2 Gr ðxÞ ¼ pffiffiffiffiffiffiffiffiffiffi exp  2r 2pr

! ð3Þ

Differential invariants occurring for a specific value of r are indicative of sample inhomogeneities and may be enhanced by subtracting the corresponding smoothed image from the original. In the present case for a given projection I(x):

Cr ðxÞ ¼ ðI  Ir ÞðxÞ

ð4Þ

However, the quality of the experimental data in biological ET is often very poor and subject to noise. This can manifest as the appearance of residual small isolated peaks in Cr(x) which may be suppressed by further convolution with a narrow Gaussian Ga of variance ra = ar where a > 0; for the present dataset values of a from 0.2 to 0.5 were used:

Cr;a ðxÞ ¼ ðCr  Ga ÞðxÞ

ð5Þ

In line with scale-space theory the convolution of the image with a Gaussian kernel of increasing variance a will cause blurring (suppression) of features at larger and larger length scales. Hence the more connected the noise within the images is, the larger the value of a required to suppress that noise. To identify differential invariants characteristic of spatial inhomogeneities we compute the square of the magnitude of |rCr,a (x)|. Then to further suppress the isolated small peaks which can be due to noise in the original image we carry out a second convolution of this result with Ga(x), as follows,





Wr;a ðxÞ ¼ jr  Cr;a j2  Ga ðxÞ   ¼ ðð@ x Cr;a Þ2 þ ð@ y Cr;a Þ2 Þ  Ga ðxÞ

ð6Þ

Local regions of high electron density will generally result in the steepest gradients in Cr,a(x) and therefore the largest values of Wr,a(x); to enhance these areas we calculate the difference with  r;a ðxÞ, normalized by the total range: the mean value, W

W0r;a ðxÞ ¼

 r;a jðxÞ jWr;a  W maxðWr;a Þ  minðWr;a Þ

ð7Þ

where W0r;a ðxÞ lies in the interval [0, 1] and max(Wr,a), min(Wr,a) refer respectively to the maximum and the minimum value of Wr,a(x). Since we wish to only process the image in areas corresponding to the highly absorbing gold particles we here define a threshold, characterized by the parameter s above which the electron density is considered to be anomalously high. This threshold defines the discriminatory function ga,r(x, s):

( g ða;rÞ ðx; sÞ ¼

1 if ðW0r;a ðxÞ > sÞ and ðIðxÞ < Ir ðxÞÞ 0

otherwise

ð8Þ

where s lies the interval [0, 1] and the additional constraint that (I(x) < Ir(x)) arises from the fact that the image intensity, in the present case, is negative in areas of strong electron absorption. The larger the value of s the smaller the region of deviation in the local electron density that is selected out; for the present dataset values of s from 0.1 to 0.2 were used. Regularization of the binary function ga,r(x, s) is achieved via convolution with a Gaussian smoothing kernel Gs(x) with a variance rs in the interval [0, 1]:

g s;a;r ðx; sÞ ¼ ðg a;r  Gs Þðx; sÞ

ð9Þ

By making the analogy between concentration and the grayscale levels that make up the image intensity, Fick’s law of diffusion may be used to interpret any image intensity inhomogeneities. The image is taken to be the initial concentration at t = 0, i.e. u(x, 0) = I(x) and the subsequent ‘evolution’ of the image in time is described by:

@u ¼ div ðD  ruÞ @t

ð10Þ

where D is the diffusion tensor, a positive definite symmetric matrix (Weickert, 1996b). If D can be assumed to be constant over the entire image domain, the diffusion process is described as homogeneous (Koenderink, 1984). In that case, all the pixels that make up the image are treated equally. In addition, the case when ru is parallel to Dru is called isotropic; D can then be replaced by a positive scalar-valued diffusivity function. If, however, the diffusion tensor

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37

is a function of the differential structure of the time-evolving image, then the diffusion process is inhomogeneous. In this situation the diffusion depends on the image evolution and the resulting diffusion filters are non-linear. Diffusion that is independent of t is described as linear. The prototype non-linear diffusion paradigm (Perona and Malik, 1990) employed a filter based on the equation ou/ot = div(g(|ru|2) ru) , with diffusivities such as g(|ru|2) = (1 + |ru|2/K)1 where K is a constant, positive parameter. Since this filter utilizes a scalar diffusivity rather than a diffusion tensor we define it here as being isotropic (Weickert, 1996a). This filter has the property of blurring regions with differential invariants that appear throughout the entire image (e.g. noise) whilst preserving regions with large local irregularities (i.e. edges). In the present case we only wish to blur regions subject to severe local distortions due to high electron absorption (i.e. gold particles) whilst leaving the rest of the image unaltered, thus we propose to replace the diffusion tensor with the function gs,a,r(x, s), given in Eq. (9). This leads to the following modification of the Perona–Malik filter:

@u ¼ div ðg s;a;r  rus Þðx; sÞ @t

ð11Þ

where us(x, 0) = Is(x) is the image I(x) convolved with a Gaussian, Gb of user-defined variance rb P 0if the local image values have been classified as being due to the gold particles according the criterion in Eq. (12). If the image values do not contain any gold particles the image is left unaltered, i.e.:

( Is ðxÞ ¼

ðI  Gb ÞðxÞ if ðW0 > sÞ and ðI < Ir Þ IðxÞ

otherwise

ð12Þ

As rb is increased sample features at larger length scales will be smoothed. In fact we note that letting rb ? 1 produces the equivalent method to that described by (Cope et al., 2011). In this method, the gold particles are replaced with pixels of average grayscale value. This effectively eliminates shadowing artifacts when the tomogram is reconstructed, but also eliminates the positions of the gold particles, which provide key information regarding the locations of proteins of interest. Isotropic diffusion allows the filtered regions to be uniformly influenced by the surrounding area which partially compensates for the loss of information due to the high electron absorption. The time-evolution of the image u(x, t) in Eq. (11) is approximated via an iterative procedure using an explicit time discretization approach. Each iteration corresponds to an additional small ‘step’ in time and will result in increased diffusion within the filtered regions. The process is repeated, with each new time-evolved u(x, t) acting as the input for the next step until a satisfactory result for the image is obtained (the supervized stop condition) (Weickert, 1996a,b; Frangakis and Hegerl, 2001; Mendrik, 2009). Eventually further time evolution of u(x, t) will lead to the boundary of the gold particles becoming blurred. 3. Results We present the pre-NID algorithm as an approach to regularize regions surrounding gold nanoparticles in electron tomograms without affecting the rest of the image. Implementing the preNID algorithm requires setting five parameters, namely: a, s, r, rb and the number of iterations. The parameters a, s and r control the behavior of the discriminatory function, enabling the automatic selection of gold particles. The parameters rb and the number of iterations control the degree of diffusion, and thus the level of smoothing. We start by choosing a value for r. Smaller values of r will better preserve the shape of the gold particles, but may generate false positives, while larger values of r will ensure fewer false

31

positives but will be associated with some distortion of the shape of the gold particles. We suggest using the radius of the gold particles (in pixels) as the initial r value. The parameter a is usually within the interval [0, 1] and provides further control for the selection of gold particles taking into account the noise in the image. Increasing the value of a will cause blurring (suppression) of features at larger length scales; if the regions of noise within the image are more connected, a larger value of a is required to suppress that noise. However, the price to pay for this suppression is that the original shape of the gold will not be preserved. Finally we use the value of s to discriminate gold particles from the other features. The typical value of s is in the range [0, 1]; increasing the value of s has the effect of locally increasing sensitivity at the price of decreased specificity in the selection of gold particles. Smaller values of s have the opposite effect. We initially tested the pre-NID filter using both simulated data and real ET data from biological samples. For the simulated data (Fig. 1) we randomly added synthetic gold beads to the synthetic double membrane described by two sinc functions in Eq. (1) (Maiorca et al., 2012). A control ground truth was generated by making projections from the synthetic volume at angles from 90° to +90° in 0.05° increments using two tilt axes, perpendicular to each other. Data were reconstructed using IMOD (Kremer et al., 1996) with the following parameters used for the pre-NID: r = 4.0, a = 0.3, s = 0.2, rb = 8.0, and 15 iterations (see Section 2.3). The same parameters were used for the second tilt axis, which was rotated horizontally with respect to the first axis by 90°. Reconstructions of the simulated data are shown in Fig. 2 as yx (top row) and yz (bottom row) views. For the control image Fig. 2a the data covers the full range of angles (i.e. with no missing wedge); hence, no artifacts are apparent. Reconstructions from a single tilt simulated dataset from 70° to +70° in 2° increments (i.e. with a missing wedge) are presented without (Fig. 2b) and with (Fig. 2c) pre-NID processing. The missing information due to the lack of high angle tilt projections is evident in the xy views of both processed and unprocessed images; e.g. the double sinc features intersecting the yz plane representing the membrane are not reconstructed as full circles. Moreover, the streaks around the gold particles are clearly visible in both the xy and yz views of the unprocessed reconstruction. In the pre-NID treated data, the positions of the gold particles are retained, but the streak artifacts are substantively reduced, as revealed in both the xy and yz views. This is particularly evident in the region adjacent to the gold particle (indicated by the arrow). The simulated dual tilt reconstructions (Fig. 2d and e) show a partial recovery from the missing wedge artifacts, but the data without pre-NID processing (Fig. 2d) retain significant streak artifacts that would affect segmentation of the structures (e.g. see arrowed region). These artifacts are significantly reduced in the pre-NID processed data (Fig. 2e). For the first biological test sample (Fig. 3), we used P. falciparum-infected red blood cells that had been harvested, purified and permeabilized to release hemoglobin, then fixed, embedded in resin, sectioned (100 nm) and stained with uranyl acetate and lead citrate. The sections were examined, data collected at a defocus of 2.5 lm, and a region of interest in the host red blood cell cytoplasm was chosen that includes an electron-dense vesicle. The properties of these 80 nm vesicles have been previously reported (Hanssen et al., 2008). Tilt projection images of the section were recorded between 65° and +65° at 2° intervals and the datatsets were aligned using programs available through IMOD. As a ground truth control (Fig. 3a) we first collected a tilt series from the section prior to decoration with fiducial markers. The sample was then layered with colloidal gold particles (10 nm) and the same region was re-imaged under the same conditions and the image data was processed without (Fig. 3b) or with (Fig. 3c) the pre-NID filter using r = 3.5, a = 0.2, s = 0.1, rb = 4.5

32

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37

Fig.2. Reconstruction of synthetic double membrane with random spherical gold particles. For the control data (a) FBP reconstructions were obtained from projections over 180° (i.e. from 90° to +90°) with 0.05° increments. For simulation of single tilt ET data (b, c) FBP reconstructions were generated using projections from 70° to +70° with 2° increments. For simulation of dual tilt data (d, e) the FBP reconstructions were obtained as for (b, c) plus a second acquisition with the tilt axis at relative to the first data set. The two datasets were combined using the IMOD package (Kremer et al., 1996; Mastronarde, 1997). Tomogram slices from tilt projections subjected to pre-NID processing are shown in (c) and (e); each tilt stack has been iterated 15 times, r = 4.0, a = 0.3, rb = 8.0, s = 0.2.

Fig.3. Pre-NID treatment of tilt projections of a biological sample prepared with and without gold fiducials. Tilt projections were collected from a 100 nm section through a P. falciparum-infected red blood cell, in a region with an 80 nm electron dense vesicle, before (a) and after (b, c) deposition of 10 nm gold fiducials onto the surface. The datasets were aligned using the IMOD package. The tilt projections at 33° are displayed. Pre-NID processing was applied to (c) using a = 0.2, s = 0.1, rb = 4.5, r = 3.5, and 20 iterations. Intensity profiles of two lines (L1, L2) intersecting gold particles and the red blood cell membrane (RBCM) are shown in (d) and (e). The intensity profiles show that the IMOD 32-bit grayscale values are decreased in the regions of gold particles but remain unaltered in the rest of the image.

and 20 iterations. Intensity profiles through individual tilt projection images (at +33°) illustrate the effect of the pre-NID processing (Fig. 3d and e). In data collected without fiducials (Fig. 3a) a clear difference in signal is apparent between the weakly stained host cell cytoplasm and the electron-dense vesicle. In the sample containing the gold particles (Fig. 3b) the scattering from these structures is so strong that they occupy most of the dynamic range of the image (Fig. 3d and e). Upon pre-NID processing (Fig. 3c), the intensity of the signal from the gold particles is reduced substantively, permitting better analysis of the signal from the vesicle. The intensity profiles confirm that the image is not altered in regions outside the areas of the gold particles (Fig. 3d and e). We

have also compared the local pre-NID filter to the Perona and Malik (P-M) filter (Suppl. Fig. 1) for a simulated 1D line profile through one of the gold particles. The data clearly illustrate how the preNID filter greatly reduces the contribution to the electron absorption by the gold whilst preserving the integrity of the surrounding image. This is in contrast to the P–M filter which results in the entire profile being smoothed. The control, processed and unprocessed test datasets were reconstructed using Filtered Back Projection (FBP) and Simultaneous Iterative Reconstruction Technique (SIRT). The reconstructed test volumes were manually rigidly aligned to the control dataset, and the overlapping area between the test and

33

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37

Fig.4. Pre-NID treatment of an electron tomogram of a biological sample prepared with and without gold fiducials. A tomogram was generated using FBP from the sample described in Fig. 3. Scale bars, 40 nm. Images have been processed using a = 0.2, s = 0.1, rb = 4.5, r = 3.5 and 20 iterations. xy (a, c) and yz (b, d) views. (Left) Reconstruction of ground truth sample before addition of gold particles. (Middle) Reconstruction generated after addition of 10 nm gold particles. (Right) Reconstruction of processed projections. The filtered image presents less obvious metal shadow artifact (MA) around gold particles (GP) than the unfiltered version. Furthermore, the biological structures are more easily identified, including the red blood cell membrane (RBCM), an electron-dense vesicle (EDV), membrane features such as a Maurer’s cleft (MC) and the parasitophorous vacuole membrane (PVM). (e) Analysis of the grayscale intensity values in the region indicated by the red rectangle. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

control datasets was selected as the region of interest. Also, the intensity values of processed and unprocessed test datasets were linearly stretched in order to match the intensity levels of the control dataset. A visual assessment of FBP reconstructed data is presented in Fig. 4. Row (a) shows a virtual section (in xy view) near the surface of the reconstructed volume. These data reveal the red blood cell membrane (RBCM), the parasitophorous vacuole membrane (PVM) surrounding the intracellular parasite, and in the red blood cell, an electron-dense vesicle (EDV) and another cisternal membrane features known as a Maurer’s cleft (MC). In the unprocessed test sample it is clear that the gold fiducials introduce streak artifacts that may obscure relevant details (a and b). This is again evident in the higher magnification image shown in (c and d). In particular, the red blood cell membrane is partially obscured in the xy view and almost completely hidden in the yz view. These artifacts are ameliorated in the pre-NID processed images (right hand column). The retrieval of the membrane structure in the

region close to the gold particle (indicated with a red rectangle) is illustrated in the analysis of the intensity values (e). The membrane feature is recognized in the processed data (asterisk) but lost in the unprocessed data. Edges of the gold particles in the processed image appear slightly blurred, an effect that is evident after a high number of iterations (i.e. >20) of the pre-NID algorithm; fewer iterations reduces the blur around the gold, but at the cost of reduced artifact reduction. In an effort to quantitatively assess the improvement in the preNID treated image compared to the untreated image in regions close to and further from the gold particles, we compared the sum of squares difference (SSD) between unprocessed or processed data and the control data (i.e. before addition of the gold particles):

X jIU ðxÞ  IC ðxÞj2 SSDU x SSD ratio ¼ ¼X SSDP jIP ðxÞ  IC ðxÞj2 x

8x 2 M

ð13Þ

34

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37

Table 1 Ratios of the sum of squares differences (SSD) as defined in Eq. (13). The raw image data, and first and second directional derivatives of the image, were analyzed in the x, y and z axes. Values greater than one indicate an improvement in the processed image compared with the unprocessed image. Reconstruction type

FBP

Mask type

Gold proximal

Gold distal

Gold proximal

SIRT Gold distal

Image 1st Derivative x direction 1st Derivative y direction 1st Derivative z direction 2nd Derivative x direction 2nd Derivative y direction 2nd Derivative z direction

4.26 2.64 3.37 4.56 1.90 1.66 2.63

1.19 1.04 1.06 1.09 1.03 1.03 1.07

3.38 3.38 2.58 3.42 3.93 1.54 1.61

1.10 1.10 1.03 1.06 1.06 1.02 1.02

where IU and IP are, respectively, the unprocessed and processed reconstructed test images, and IC is the control image. The ratio is computed in regions of the image delineated by a mask M. Two masks were created, one containing values outside, but proximal to, the gold particles (in a radius of 5 voxels), and the other in regions distal to the gold particles (with a radius above 5 voxels). The results are shown in Table 1. The SSD ratios are high in the region close to the gold particles indicating that NID processing provides significant improvement in the data in this region. The ratios are slightly greater than one in the distal regions of the tomogram, indicating a small overall improvement in the processed data. Improvements are noted in the raw data and in first and second directional derivatives (corresponding to the identification of edges and changes in gradation, Table 1) suggesting that the processed data is more suitable for automatic segmentation and feature analysis.

Fig.5. Pre-NID treatment of an electron tomogram with immunolabelling, processed using different reconstruction methods. Tilt projections were collected from a 250 nm section of a P. falciparum-infected red blood cell labeled with protein A-gold. Reconstructions performed using both FBP and SIRT. Scale bar, 50 nm. (a, b, e, g, i and k) Unprocessed. (c, f and j) Processed FBP. (d, h and l) Processed SIRT. Images were processed using a = 0.3, s = 0.1, r = 3.5, rb = 2.5, and 3 iterations. The filtered images present less obvious metal shadow artifacts (MA) around the gold particles (GP) than the unfiltered versions.

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37

35

Fig.6. Cryo electron tomogram. Virtual section through a cryo electron tomogram of microtubules in a P. falciparum gametocyte. Scale bars = 50 nm. Panel (a) shows an unprocessed view of a longitudinal section through the tomogram, revealing microtubules (MT), ribosomes (R), the red blood cell membrane (RBCM), and a colloidal gold particle exhibiting streak artifacts. Panel (b) shows the processed tomogram with reduced artifacts around the gold. The image was processed using a = 0.45, s = 0.12, r = 5, rb = 2, and 10 iterations. Panels (c) and (d) show xy and xz details of unprocessed and processed data. The artifact around the gold is significantly reduced, while the rest of the image is preserved.

We also examined the ability of pre-NID processing to improve the quality of immunogold-labeled samples. For this preparation, P. falciparum-infected red blood cells were harvested and Equinatoxin II-permeabilized, then labeled with an antibody recognizing the virulence protein, PfEMP1, followed by protein A-gold (6 nm), prior to fixation, resin-embedding, sectioning (250 nm) and staining. Tilt projections were recorded between 66° and +66° at 2° intervals and the datatsets aligned and reconstructed using either the FBP or SIRT algorithms (Fig. 5). Previous studies have suggested that SIRT offers better reconstructions for such limited datasets (Gilbert, 1972) but we observed streak artifacts in tomograms reconstructed using either algorithm in the xy views (Fig. 5a, b, e and g) and in the yz view (Fig. 5i and k). The pre-NID processing, however, substantively reduced artifacts when used before either FBP- or SIRT-based reconstructions, as demonstrated in Fig. 5c, d, f, h, j and l. We compared the results of pre-NID processing with an existing ‘inpainting’ method that replaces the regions of high signal with pixels of an average gray value (Cope et al., 2011). Our pre-NID filter permits automatic identification of the gold particles. For example for this study we analyzed a total of 226 gold particles. The efficiency of the automatic identification of gold particle was assessed by printing Eq. (8) as an image volume to a file and overlaying it onto the original data. Manual verification revealed that the gold particles were correctly selected in >90% of cases. Manual selection of the gold particles will also be available upon implementation in IMOD. Importantly the positions of the gold particles are still clearly evident in the tomograms after pre-NID processing. By contrast the inpainting method requires manual definition of each of the gold particles and assignment of an average particle diameter. As shown in Suppl. Fig. 2a, the inpainting method gives a similar improvement in the retrieval of information in the regions around the gold shadows. However this method is not applicable in the case of immunogold labels, where the information about the location of the gold particles is important. Cryo electron tomography permits 3D visualization of cryo-sections or thin regions of whole cells (Cyrklaff et al., 2007; Kudryashev et al., 2010), but is particularly susceptible to streaking artifacts. We examined the ability of pre-NID processing to improve the quality of P. falciparum samples prepared for cryo electron tomography. For this sample sexual stage P. falciparum

gametocytes were mounted on carbon-coated grids, plunge frozen in liquid ethane then transferred to liquid nitrogen for imaging on a cryo stage. Tilt series were collected every 2° between 70° and +70°. The image data was processed without (Fig. 6a) or with (Fig. 6b) the pre-NID filter using a = 0.45, s = 0.12, r = 5, rb = 2, and 10 iterations. In the unprocessed tomograms streak artifacts are evident around the gold particle in the field of view. These artifacts are substantively reduced in the pre-NID-processed tomograms (Fig. 6 a, b, red arrows, and panels c and d). By contrast other features such as microtubules (MT) and ribosomes (R) that do not have the sharp gradient differential characteristic of gold particles, are not affected by the pre-NID processing.

4. Discussion Tomography is a widely used method in electron microscopy to recreate 3D volumes from a series of 2D images acquired at different tilt angles. Unfortunately, the physical constraints of the sample stage often lead to acquisition geometry irregularities and a paucity of data, as the tilt angles are typically restricted to around 70° to +70°. Under those conditions, the problem of tomographic reconstruction using standard techniques such as FBP is considered to be ‘‘ill-posed’’. As a consequence, reconstruction algorithms generally produce visible artifacts in the reconstructed volume. In addition, as the results in this paper show, artifacts can arise from the gold particles used for immunolabelling or as fiducial particles in ET which might hide proximal biological structures. For instance, in our biological test sample, several sections of the red blood cell membrane are obscured by distortions around the gold fiducials, thus making it impossible to ascertain the continuity of the membrane (which is clear in the data collected before addition of fiducials). As ET is specifically employed to resolve such fine ultrastructural information, the need to address the issue of artifacts is evident. In this paper we provide a scale space based approach to analyzing and accounting for the grayscale irregularities in tilt projection data caused by gold nanoparticles. We also describe a two-step non-linear isotropic diffusion filter for reducing those irregularities prior to reconstruction. We show that the processed tilt projections produce reconstructed volumes with fewer artifacts,

36

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37

allowing us to pick out more structural details in the regions proximal to the gold. When our pre-NID filter was employed, we observed improvements in the reconstructed tomograms, especially in regions proximal to the gold particles and in the yz views. The improvements were evident for both resin-embedded and cryostabilized biological samples, for gold particles used as fiducials on the sample surface as well as immunogold labels embedded within the sample, and when either FBP or SIRT were used for the reconstruction process. The locally smoothed regions in tilt projections remained smooth in the reconstructed volumes and information obscured by streak artifacts in our biological test sample was partially recovered after pre-NID processing. We anticipate that pre-NID processing will prove very useful for facilitating feature segmentation, especially for samples with low intrinsic contrast (e.g. cryo samples or samples of weakly contrasting polymer) or with high densities of gold particles. Our methods will be made available by distributing the source code (in IMOD) and our simulated and test images (CCDB). There are five parameters that need to be set in the pre-NID approach, namely: a, s, r, rb and the number of iterations. These parameters enable the automatic selection of gold particles and control the level of smoothing. For the data analyzed in this study we used r values close to the radius of the gold particles (in pixels), a values from 0.2 to 0.5, s values from 0.1 to 0.2, rb values from 2.0 to 8.0 and numbers of iterations from 3 to 20. This parameter range will be suitable for many studies, but additional work may be required for a comprehensive characterization of the parameter space. We also note that the method could be implemented in a parallel manner, since each projection can be treated independently using pre-NID filtering. Acknowledgments The authors acknowledge support from the Australian Research Council and the Australian National Health and Medical Research Council. We thank David Mastronarde, Boulder Laboratory for 3D EM of Cells, University of Colorado, USA, for useful discussions and input. We would also to acknowledge Dr Benedicta Arhatari (La Trobe University, Australia), Nico Persch and Prof. Joachim Weickert (Mathematical Image Analysis Group, Saarland University, Germany), Dr Adriënne Mendrik (Image Sciences Institute, Utrecht University, the Netherlands); and Dr Marek Cyrklaff (University of Heidelberg, Germany) for helpful discussions. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jsb.2014.03.009. References Abu Bakar, N.A., Klonis, N., Hanssen, E., Chan, C., Tilley, L., 2010. Digestive-vacuole genesis and endocytic processes in the early intraerythrocytic stages of Plasmodium falciparum. J. Cell Sci. 123, 441–450. Agulleiro, J.I., Fernandez, J.J., 2011. Fast tomographic reconstruction on multicore computers. Bioinformatics 27, 582–583. Ali, R.A., Landsberg, M.J., Knauth, E., Morgan, G.P., Marsh, B.J., et al., 2012. A 3D image filter for parameter-free segmentation of macromolecular structures from electron tomograms. PLoS One 7, e33697. Arslan, I., Tong, J.R., Midgley, P.A., 2006. Reducing the missing wedge: highresolution dual axis tomography of inorganic materials. Ultramicroscopy 106, 994–1000. Baumeister, W., Grimm, R., Walz, J., 1999. Electron tomography of molecules and cells. Trends Cell Biol. 9, 81–85. Cao, M., Zhang, H.B., Lu, Y., Nishi, R., Takaoka, A., 2010. Formation and reduction of streak artefacts in electron tomography. J. Microsc. 239, 66–71.

Cope, J., Heumann, J., Hoenger, A., 2011. Cryo-electron tomography for structural characterization of macromolecular complexes. Curr. Protoc. Protein Sci., 13 (Chapter 17, Unit17). Cyrklaff, M., Kudryashev, M., Leis, A., Leonard, K., Baumeister, W., et al., 2007. Cryoelectron tomography reveals periodic material at the inner side of subpellicular microtubules in apicomplexan parasites. J. Exp. Med. 204, 1281– 1287. Downing, K.H., Sui, H., Auer, M., 2007. Electron tomography: a 3D view of the subcellular world. Anal. Chem. 79, 7949–7957. Dubochet, J., Adrian, M., Chang, J.J., Homo, J.C., Lepault, J., et al., 1988. Cryo-electron microscopy of vitrified specimens. Q. Rev. Biophys. 21, 129–228. Fernández, J.J., Martínez, J.A., 2010. Three-dimensional feature-preserving noise reduction for real-time electron tomography. Digit. Signal Process. 20, 1162– 1172. Frangakis, A.S., Hegerl, R., 2001. Noise reduction in electron tomographic reconstructions using nonlinear anisotropic diffusion. J. Struct. Biol. 135, 239– 250. Frey, T.G., Perkins, G.A., Ellisman, M.H., 2006. Electron tomography of membranebound cellular organelles. Annu. Rev. Biophys. Biomol. Struct. 35, 199–224. Gan, L., Jensen, G.J., 2012. Electron tomography of cells. Q. Rev. Biophys. 45, 27–56. Gilbert, P., 1972. Iterative methods for the three-dimensional reconstruction of an object from projections. J. Theor. Biol. 36, 105–117. Guesdon, A., Blestel, S., Kervrann, C., Chretien, D., 2013. Single versus dual-axis cryoelectron tomography of microtubules assembled in vitro: limits and perspectives. J. Struct. Biol. 181, 169–178. Hanssen, E., Sougrat, R., Frankland, S., Deed, S., Klonis, N., et al., 2008. Electron tomography of the Maurer’s cleft organelles of Plasmodium falciparum-infected erythrocytes reveals novel structural features. Mol. Microbiol. 67, 703–718. Hanssen, E., McMillan, P.J., Tilley, L., 2010a. Cellular architecture of Plasmodium falciparum-infected erythrocytes. Int. J. Parasitol. 40, 1127–1135. Hanssen, E., Goldie, K.N., Tilley, L., 2010b. Ultrastructure of the asexual blood stages of Plasmodium falciparum. Methods Cell Biol. 96, 93–116. Herman, G.T., 2009. Fundamentals of Computerized Tomography: Image Reconstruction from Projections, second ed. Springer Publishing Company, Incorporated. Jimenez, N., Post, J.A., 2012. A novel approach for intracellular 3D immuno-labeling for electron tomography. Traffic 13, 926–933. Koenderink, J., 1984. The structure of images. Biol. Cybern. 50, 363–370. Koster, A.J., Klumperman, J., 2003. Electron microscopy in cell biology: integrating structure and function. Nat. Rev. Mol. Cell Biol. (Suppl.), 6–10. Köstler, H., Prümmer, M., Rüde, U., Hornegger, J., 2006. Adaptive variational sinogram interpolation of sparsely sampled ct data. In: IEEE Computer Society, ICPR, vol. 3, pp. 778–781. Kremer, J.R., Mastronarde, D.N., McIntosh, J.R., 1996. Computer visualization of three-dimensional image data using IMOD. J. Struct. Biol. 116, 71–76. Kudryashev, M., Lepper, S., Stanway, R., Bohn, S., Baumeister, W., et al., 2010. Positioning of large organelles by a membrane-associated cytoskeleton in Plasmodium sporozoites. Cell. Microbiol. 12, 362–371. Leis, A., Rockel, B., Andrees, L., Baumeister, W., 2009. Visualizing cells at the nanoscale. Trends Biochem. Sci. 34, 60–70. Maiorca, M., Hanssen, E., Kazmierczak, E., Maco, B., Kudryashev, M., et al., 2012. Improving the quality of electron tomography image volumes using prereconstruction filtering. J. Struct. Biol. 180, 132–142. Mastronarde, D.N., 1997. Dual-axis tomography: an approach with alignment methods that preserve resolution. J. Struct. Biol. 120, 343–352. McDonald, K.L., Auer, M., 2006. High-pressure freezing, cellular tomography, and structural cell biology. Biotechniques 41, 137, 139, 141 passim. McIntosh, R., Nicastro, D., Mastronarde, D., 2005. New views of cells in 3D: an introduction to electron tomography. Trends Cell Biol. 15, 43–51. McMillan, P.J., Millet, C., Batinovic, S., Maiorca, M., Hanssen, E., et al., 2013. Spatial and temporal mapping of the PfEMP1 export pathway in Plasmodium falciparum. Cell. Microbiol. 15, 1401–1418. Mendrik, A.M., Vonken, E.J., Rutten, A., Viergever, M.A., van Ginneken, B., 2009. Noise reduction in computed tomography scans using 3-D anisotropic hybrid diffusion with continuous switch. IEEE Trans. Med. Imaging 28, 1585–1594. Milne, J.L., Subramaniam, S., 2009. Cryo-electron tomography of bacteria: progress, challenges and future prospects. Nat. Rev. Microbiol. 7, 666–675. Moritz, M., Braunfeld, M.B., Sedat, J.W., Alberts, B., Agard, D.A., 1995. Microtubule nucleation by gamma-tubulin-containing rings in the centrosome. Nature 378, 638–640. Mumcuoglu, E.U., Hassanpour, R., Tasel, S.F., Perkins, G., Martone, M.E., et al., 2012. Computerized detection and segmentation of mitochondria on electron microscope images. J. Microsc. 246, 248–265. Nguyen, H., Ji, Q., 2008. Shape-driven three-dimensional watersnake segmentation of biological membranes in electron tomography. IEEE Trans. Med. Imaging 27, 616–628. Penczek, P., Marko, M., Buttle, K., Frank, J., 1995. Double-tilt electron tomography. Ultramicroscopy 60, 393–410. Perona, P., Malik, J., 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639. Pierson, J., Vos, M., McIntosh, J.R., Peters, P.J., 2011. Perspectives on electron cryotomography of vitreous cryo-sections. J. Electron. Microsc. (Tokyo) 60 (Suppl. 1), S93–S100. Radermacher, M., 1992. Weighted back-projection methods. In: Frank, J. (Ed.), Electron Tomography: Three-dimensional Imaging with the Transmission Microscope. Plenum Press, New York, pp. 91–115.

M. Maiorca et al. / Journal of Structural Biology 186 (2014) 28–37 Saghi, Z., Midgley, P.A., 2012. Electron tomography in the (S) TEM: from nanoscale morphological analysis to 3D atomic imaging. Annu. Rev. Mater. Res. 42, 59–79. Sorzano, C.O., Marabini, R., Velazquez-Muriel, J., Bilbao-Castro, J.R., Scheres, S.H., et al., 2004. XMIPP: a new generation of an open-source image processing package for electron microscopy. J. Struct. Biol. 148, 194–204. Spycher, C., Rug, M., Pachlatko, E., Hanssen, E., Ferguson, D., et al., 2008. The Maurer’s cleft protein MAHRP1 is essential for trafficking of PfEMP1 to the surface of Plasmodium falciparum-infected erythrocytes. Mol. Microbiol. 68, 1300–1314. Tchelidze, P., Sauvage, C., Bonnet, N., Kilian, L., Beorchia, A., et al., 2006. Electron tomography of amplified nanogold immunolabelling: improvement of quality based on alignment of projections with sinograms and use of postreconstruction deconvolution. J. Struct. Biol. 156, 421–431.

37

van der Heide, P., Xu, X.-P., Marsh, B.J., Hanein, D., Volkmann, N., 2007. Efficient automatic noise reduction of electron tomographic reconstructions based on iterative median filtering. J. Struct. Biol. 158, 196–204. Vanhecke, D., Asano, S., Kochovski, Z., Fernandez-Busnadiego, R., Schrod, N., et al., 2011. Cryo-electron tomography: methodology, developments and biological applications. J. Microsc. 242, 221–227. Weickert, J., 1996a. Theoretical foundations of anisotropic diffusion in image processing. In: Kropatsch, W. et al. (Eds.), Theoretical Foundations of Computer Vision, vol. 11. Springer Vienna, pp. 221–236. Weickert, J., 1996b. Anisotropic diffusion in image processing. University of Kaiserslautern.