Three-dimensional reconstruction of neuronal forests

Three-dimensional reconstruction of neuronal forests

Neurocomputing 38}40 (2001) 1643}1650 Three-dimensional reconstruction of neuronal forests Brent P. Burton , Bruce H. McCormick *, Reidun Torp, Jam...

259KB Sizes 1 Downloads 17 Views

Neurocomputing 38}40 (2001) 1643}1650

Three-dimensional reconstruction of neuronal forests Brent P. Burton , Bruce H. McCormick *, Reidun Torp, James H. Fallon NVIDIA, Austin, TX 78727, USA Department of Computer Science, Scientixc Visualization Laboratory, Texas A&M University, College Station, TX 778-3112, USA University of Oslo, Norway University of California, Irvine, CA, USA

Abstract Three-dimensional reconstruction of neuronal forests from a coherent volumetric data set representing Golgi-stained cortical material is demonstrated. The system automates in parallel neuron feature extraction and reconstruction, replacing largely manual techniques for tracing individual neurons. Serial sections were digitized at 1024;1024 pixels at a linear resolution of 0.37 m, and then digitally aligned into a coherent volume data set. Our reconstruction program, Recon, determined regions of interest (ROIs) in each image and culled data aggressively, reducing the original volumetric data 70-fold into an ROI-based aligned image stack. The neuronal forest of Fig. 1 below was treaded from this reduced data set.  2001 Published by Elsevier Science B.V. Keywords: Brain microstructure; 3D reconstruction of neurons; Physical sectioning; Brain tissue scanning

1. Introduction Virtually, our entire knowledge of the geometry and statistics of brain microstructure has been drawn from two-dimensional (planar) measurements [1]. From two dimensions, the three-dimensional environment of neurons (their somata, and dendritic and axonal arbors) and their interconnections can be only partially inferred.

* Corresponding author. Tel.:#1-979-845-8870; fax:#1-979-847-8578. E-mail address: [email protected] (B.H. McCormick). 0925-2312/01/$ - see front matter  2001 Published by Elsevier Science B.V. PII: S 0 9 2 5 - 2 3 1 2 ( 0 1 ) 0 0 5 2 3 - 9

1644

B.P. Burton et al. / Neurocomputing 38}40 (2001) 1643}1650

Fig. 1. Reconstruction viewed from top of slab. Neurons are represented by their central trajectories. Vertical axis has been stretched 3.5;.

For example, the columnar organization of the neocortex at a neuronal level of detail is now virtually indecipherable [5]. The central contribution of the research described below is to reconstruct dendritic and axonal arbors in their three-dimensional con"guration, as a step towards building a coherent geometric model of the large-scale network of their neuronal connections.

2. Three-dimensional reconstruction experiments In early parallel reconstruction experiments described here, Golgi-stained cortical material was sectioned at 0.5 m increments using an ultramicrotome equipped with a diamond knife designed for histological sectioning. Forty-two consecutive sections were digitized at 1024;1024 pixels at a linear resolution of 370 nm. Successive images of the image stack were then digitally aligned into a coherent volume data set. Our reconstruction program, Recon [2,3], determined regions of interest (ROIs) separately in each image and culled data aggressively, reducing the original volumetric data 70-fold into an ROI-based aligned image stack. The neuronal forest in Fig. 1 was threaded from this reduced data set.

3. Three-dimensional reconstruction process Image modulation is used to form new images from consecutive pairs of images in the aligned image stack: corresponding pixels from the two images are multiplied (Fig. 2). When neuronal features from both images overlap, the two areas are

B.P. Burton et al. / Neurocomputing 38}40 (2001) 1643}1650

1645

Fig. 2. Image modulation of successive images in image stack. (a) and (b) Slices 5 and 6, respectively. (c) Modulated image from the previous pair.

near-black, and the result is also black. When a feature resides in one section (black) but not in the other section (gray or white), the result will be either dark gray or lighter. Thresholding is required to de"ne feature contours. However, the overall darkening means a lower threshold (about 15% gray) that can be used to de"ne the border of features (ROIs). The reconstruction process comprises three phases (using the modulated image stack). First, each image is independently segmented into ROIs (oriented bounding boxes). Second, ROIs from successive images are threaded together into cell trajectories (indicated by thin black lines in Fig. 1). Pathway branching and merging and bends are allowed. Third, identi"ed cell segments (the generalized cylinders centered about the trajectories) are edited into viable dendritic or axonal arbors, or somata. Selective visualization of the reconstructed cells is provided (Figs. 3 and 4). The Visualization Toolkit (http://www.kitware.com) was used to perform isosurface reconstruction. Section thickness was distorted (set to 3.5 m) to give the slab more depth. The trajectories are rendered as black lines inside the cell `skinsa in the slab. The cell isosurfaces have a particularly smooth, globular shape because they were generated from half-resolution data (512;512) and decimated, while the trajectories were created from 1k;1k data. If a path is outside a `skina, it may be due to either noise (and wrong) or the lack of an isosurface in that region. Also the Golgi staining is incomplete in this data set. The data set of Fig. 1 used 14 sections, each image cropped to 1k;1k resolution. Here, 647 ROIs (contours) were detected in the data set and 414 segments were matched. Unthreaded ROIs are from noise and other artifacts. The reconstruction program read the 14MB of input data and performed the reconstruction in less than 4 s. The reconstruction program, using lower resolution (1 m) imaging, is trivially extensible to cell counting and in determining the spatial distribution and morphology of cell bodies, as in Nissl stained tissue. Lower resolution scanning (1 voxel"(1 m)) is used. Somata, in common with reconstruction methods for Golgi-stained material, are treated as segments at level zero (root dendritic segments are at level 1, etc.).

1646

B.P. Burton et al. / Neurocomputing 38}40 (2001) 1643}1650

Fig. 3. Isosurface visualization. Overhead view showing both the reconstructed trajectories (black lines) and isosurfaces (gray) generated using the Marching Cubes algorithm.

Fig. 4. Isosurface visualization. Left side of slab.

B.P. Burton et al. / Neurocomputing 38}40 (2001) 1643}1650

1647

4. Data compression Computationally, the tri-partite partitioning of the reconstruction process is important: the image-based segmentation can be run in real-time, and reduces 70-fold the requirements for secondary storage. This image compression is critical in a production environment potentially generating one teravoxel per day (see below). The maximum data compression factor one can expect is 150 : 1 (as compared to the attainable 70 : 1 factor) and is approximately independent of pixel or voxel size. The volume of stained material in the tissue block can be integrated from the stained neuronal cross sections seen on successive images of the image stack. On average, then, the fraction of pixels in an image that exhibit Golgi stain equals the volume fraction (approximately 2%) of Golgi-stained neurons in the brain tissue. Neurons constitute one-third of the volume of cortical tissue, and stained neurons occupy  ths by volume. Oriented bounding boxes and the data structures used provide  additional overhead.

5. Circumventing image alignment error The main problem in performing the reconstruction is image misalignment: observe the jitter in the cell trajectories arising from manual alignment in Fig. 1. For this reason, the reconstruction of neuronal forests from an image stack using physical sectioning has hitherto not been considered a viable procedure. Unlike these manual procedures, the design of the instrument described below is predicated to circumventing image alignment error. The instrument, the brain tissue scanner, scans repeatedly the surface of the work piece*that is, the block of stained brain tissue embedded in plastic. As consecutive thin sections are micro-machined o!, using a diamond knife or sectioning mill, concurrent digital scanning generates a volume data set in the form of an aligned image stack. All sectioned material is destroyed by the physical sectioning process.

6. Brain tissue scanner The physical sectioning methods of the brain tissue scanner preserve image registration. The brain tissue scanner allows two very di!erent modes of section scanning: (1) knife-edge scanning and (2) surface scanning. Though both scanning modes use a line-scan camera, they di!er signi"cantly in their scanning characteristics. Knifeedge scanning (Fig. 5) preserves vertical registration. Knife-edge scanning also gives complete isolation of the thin tissue being imaged from tissue at greater depths in the tissue block. In particular, knife-edge scanning prevents back-scattering of light from

 The Knife-edge scanner has been granted a Provisional Patent Application (McCormick/Texas Engineering Experiment Station, College Station, TX, USA) by the United States PTO.

1648

B.P. Burton et al. / Neurocomputing 38}40 (2001) 1643}1650

Fig. 5. Cross section of specimen being sectioned and knife-edge scanned. Note deep voxels do not participate in light back-scattering, nor are they bleached in #uorescence microscopy. (Thickness of section exaggerated.)

lower tissue layers and allows the imaging of #uorescent-stained material without bleaching the tissue to be sectioned subsequently. Image data acquisition rates are bounded by the maximum permissible tissue cutting velocity of the diamond knife. Knife-edge scanning supports all known forms of microscopy: absorption imaging using transmitted light; bright-"eld, dark-"eld, and DIC imaging using re#ected light; and #uorescence imaging using re#ected light. In surface scanning, successive thin sections are removed from the top of the specimen block by knife-based or mill-based microtomy [4]. Unlike knife-edge scanning, surface scanning separates in time; (1) removal of tissue from the face of the specimen block and (2) scanning the newly-exposed surface. Physical sectioning allows intense epi-illumination of sections and e$cient collection of the scattered/#uorescent photons. Sampling at 200 MHz, surface scanning generates a volume database of brain tissue at a signi"cant speed advantage ('100 : 1) over optical sectioning in confocal microscopy. Only re#ected light microscopy (bright-"eld, dark-"eld, or DIC) or #uorescence imaging can be used. The speed of our methods will permit imaging at a neuronal level of detail the entire mouse brain (&1 cm) or surveys of comparable size of the developing human cerebral cortex in 27 days. We have set our speci"cations to scan 1 teravoxel (10 volume elements) per 8 h working day, a mean sampling rate of 35 MHz.

Acknowledgements Exploring the Brain Forest was supported by Texas Advanced Technology Program Grant 999903-124 (McCormick) from the Texas Higher Education Coordinating Board. Development of Brain Tissue Scanner is supported by a Major Research Instrumentation (MRI) Grant 0079874 (McCormick) from NSF initiated September 1, 2000.

B.P. Burton et al. / Neurocomputing 38}40 (2001) 1643}1650

1649

References [1] V. Braitenberg, A. Schuz, Cortex: Statistics and Geometry of Neuronal Connectivity, 2nd Edition, Springer, Berlin, 1998. [2] B.P. Burton, B.H. McCormick, Virtual microscopy of brain tissue, Neurocomputing 26}27 (1999) 981}987. [3] B.P. Burton, Automated 3D reconstruction of neuronal structures from serial sections, MS Thesis, Department of Computer Science, Texas A&M University, 1998. [4] B.H. McCormick, Design of a brain tissue scanner, Neurocomputing 26-27 (1999) 1025}1032. [5] V.B. Mountcastle, Perceptual Neuroscience: the Cerebral Cortex, Harvard University Press, Cambridge, MA, 1998.

Bruce H. McCormick is Professor of Computer Science and the Director of the Scienti"c Visualization Laboratory at Texas A&M University. His research interests include scienti"c visualization, brain mapping, computational neuroscience, and neural networks. He received his B.S. and Ph.D. degrees in Physics from MIT and Harvard University, respectively. He was Professor of Computer Science and Physics at the University of Illinois at Urbana-Champaign. At the University of Illinois at Chicago he served as Head of the Department of Information Engineering, and at Texas A&M University, as the "rst head of the Department of Computer Science.

Brent P. Burton is a senior software engineer at NVIDIA in Austin, Texas, performing optimization of 3D graphics drivers for hardware accelerators. He has a B.S. and MS. from Texas A&M University in Computer Science. His research interests are scienti"c visualization and reconstruction techniques as applied to medical image data. Other interests include 3D rendering techniques and programming languages.

Reidun Torp is a Visiting Assistant Researcher, Institute of Brain Aging and Dementia, University of California, Irvine and a Senior Scientist, Anatomical Institute, University of Oslo. She received her B.S. in Cell Biology and her M.S. and Ph.D. degrees in Neurobiology from the University of Oslo. Her recent studies have centered on Alzheimer's disease, glutamate transporters in rat cerebral cortex and thalamus, and the ultrastructural study of pathology and amyloid deposition in aged canine models of human aging and dementia. She is now collaborating with James Fallon and Bruce McCormick on the threedimensional reconstruction of brain microstructure in the transgenic mouse and the postnatal human cortex.

1650

B.P. Burton et al. / Neurocomputing 38}40 (2001) 1643}1650 James H. Fallon is Professor of Anatomy and Neurobiology in the College of Medicine at the University of California, Irvine. His areas of interest are the function of neurotropic factors in neurodegenerative disorders, aging and development, and their use in neural stem cell therapies. He was the "rst to localize a characterized growth factor in the brain, and the "rst to localize EGF, TGF, and FGF in the brain. He has a long-standing interest in neurochemical anatomy of monoamine, opioid, and other neurotransmitter systems in the mamallian brain. He is collaborating with Junko Hara and Rod Shankle on analyses of developing human cortex, and with Bruce McCormick on geometric modeling of the transgenic mouse brain and human cerebral cortex.