COMPUTERS
AND
BIOMEDICAL
RESEARCH
15,
598-604 (1982)
Three-Dimensional Reconstruction from Serial Sections. I. A Portable Microcomputer-Based Software Package in Fortran JEFFREY Department of Biological
PROTHEROANDJOHN
PROTHERO
Structure, University of Washington School of Medicine. Seattle, Washington 98195 Received May 12, 1982
A software package is described that is designed to simplify the collection, storage, and processing of data to be used for three-dimensional reconstruction from serial sections. The package, called MORPHO, consists of four independent programs, all written in FORTRAN, running under CP/M, and called ISPY, DMPSPY, EDSPY, and STATSPY. The function of ISPY is to guide the user in the documentation, digitization, and storage of data derived from serial section images. The serial sections may be at any level of resolution: ultrastructural, microscopic, or macroscopic. DMPSPY allows the user to display and to make hard copies of the digitized contours. EDSPY allows any and all errors in data entry to be rectified simply and directly. STATSPY carries out statistical calculations, such as crosssectional areas and volumes. A useful feature of STATSPY is the provision for easy insertion of additional subroutines to compute special statistics not currently provided.
1. INTRODUCTION
Many problems arising in fundamental studies, in such fields as biomechanits, morphogenesis, pattern formation, scaling and transport fluxes, as well as in applied biomedical studies, require quantitative structural data in three dimensions for their solution. The need is a generic one, cutting across many fields of biology and medicine. Until recently, the complexity of biological shapes has made quantitation in three dimensions a prohibitively tedious and expensive undertaking. The advent of computers has substantially reduced the time and effort needed to collect three-dimensional data; furthermore, computer graphics have greatly enhanced the value of the resultant data. Neurobiologists, in particular, have been quick to take advantage of minicomputers for three-dimensional reconstruction (1, 2). Our first work in the development of this methodology made use of mainframe (3) (see also (4)). We have now developed a microcomputer-based morphometric facility for quantitative three-dimensional reconstruction. The system is organized around a digitizer table (O.l-mm resolution), a 64K microcomputer with three floppydisk drives (170K each), a graphics display terminal (512 x 256), a printer, and a digital XY plotter. 598 0010-4809/82/060598-07$02.00/O Copyright0 1982 by Academic Press, Inc. All rights of reproduction
in any form reserved.
3D RECONSTRUCTION
FROM
SERIAL
SECTIONS
599
A brief description of the design of the facility may be helpful as background for the subsequent discussion. In essence, it is assumed that the user’s objective is to produce an anatomical atlas, each page of which represents a serial section. Input data can be at the ultrastructural level of resolution, or more typically at the microscopic or macroscopic level. The serial sections to be measured are assumed to be in their correct sequential order, but need not be contiguous or even equally spaced. Images of each serial section to be digitized must be superimposed on the digitizer tablet. This can be accomplished, for example, with tracings that have been prepared off line. In our practice, the images of the serial sections are usually recorded on a 35-mm film strip and projected onto the digitizer tablet from above. The design of the software must take into account two special constraints that are inherent in quantitative three-dimensional reconstruction. The first constraint concerns the need to be able to put successive (but not necessarily contiguous) serial sections in their correct register. For microscopic and macroscopic work we have found it sufficient to introduce fiducials, in the form of holes, extending through the tissue block and threaded with “fixed” nerves. The second constraint arises from the need to document the magnification. We have made provision for two magnification factors: one gives the magnification from the specimen to the 35-mm film and the second gives the magnification from the 35-mm film to the digitizer tablet. We determine these magnification factors in separate operations and enter them at the beginning of each atlas (for further details on these and other nonsoftware aspects of the facility see (5)). The rest of our discussion proceeds on the assumption that one image of each serial section to be digitized is recorded, sequentially, on 35-mm film; that is, with one section per frame on the film (reconstruction is possible where each frame contains only a subfield of one serial section, but that application of the system is not discussed here). At the beginning of each atlas the user enters one-time-only data, such as atlas name, as well as a variety of bookkeeping data: date, tissue code, magnification factors, etc. (see below). The user then goes on to digitize the first frame. In digitizing a frame, the user first records the fiducials and then those contours within that frame that are of interest. As fiducials or contours are digitized, their images appear on the graphic display terminal. At the end of each record (fiducials or contour) the user elects to store or reject the digitized points. (Note that the fiducials are stored as the zeroth-order contour.) When all the contours of interest within the first frame have been digitized, the user either exits from the menu, or more usually, goes on to the next frame, repeating all the above steps, including digitization of the fiducials. The user continues in this fashion, with the option of stopping at the end of any frame, exiting, and restarting later at the next frame. A total of about 40,000 points, constituting the maximum number for one atlas, can be recorded on one diskette (i.e., allowing for some overhead, one diskette has about 160,000 bytes; 2 bytes/integer and 2 integers/point gives 40,000 points).
600
PROTHERO
AND PROTHERO
A more in-depth discussion of the hardware, and logistics of the facility is in preparation (5). Our purpose here is to describe a software package, called MORPHO, that manages data storage and guides the user in the collection, display, editing, and processing of morphometric data. MORPH0 consists of four major programs: ISPY, DMPSPY, EDSPY and STATSPY-as well as a number of subsidiary programs (e.g., one to specifically initialize diskettes for MORPHO), all of which run under CP/M and all but one of which are organized in a menu format. We consider first the general properties of MORPH0 and then go on to discuss each of the constituent programs separately. 2. MORPH0 The structure of MORPH0 and the flow of information is illustrated in Fig. 1. It should be noted that each program functions independently of the others. This independence is mandated in any case by current limits in microcomputer fast memory, but is also in conformity with good programming practice. Our discussion from here on assumes that only two disk drives are available (i.e., only two are required), but it must be admitted that the system is much easier to use with three disk drives. For simplicity, we shall assume that properly formated and initialized data disks have already been prepared. A typical sequence of data collection and data processing begins with loading the MORPH0 system disk and reading ISPY into the computer memory. The system disk is then removed and two MORPHO-initialized data disks are loaded in the A and B drives. (The use of two data disks allows digitization to be discontinued and restarted-after an automatic copy operation-and also allows either of the disks to be fully utilized with an essentially complete backup copy at all times.) The philosophy adopted in the design of MORPH0 is to keep all of the
TERMINAL DISPLAY
i TERTIARY
FIG.
3 o-----
1. MORPH0
(STANDARD ASCII) -
programs
and
c 3D COMPUTER GRAPHICS
information Row.
3D RECONSTRUCTION
FROM
SERIAL
SECTIONS
601
“raw” digitized coordinates as permanent archival files. These are referred to as primary data files (see Fig. 1) and are not subject to editing. As a rule, it is assumed that one primary file corresponds to one atlas. The data on the primary files can be inspected at any time by calling DMPSPY. This program allows the user to make video displays and plots (using either a graphics printer, a digital XY plotter, or both) on a frame-by-frame basis. As the data are inspected with DMPSPY, various errors may be discovered. All such errors are correctabie, using EDSPY (see below). The output of EDSPY is a secondary data file, stored on a separate disk. The last step in data processing is to read in the corrected data from the secondary data file (or if no errors are present, from a primary file) and to compute the desired statistics, using STATSPY. The output of STATSPY is stored in a tertiary file, which may or may not be on the same disk as the secondary file. The data in the tertiary file are in a format that is convenient from the standpoint of input into a special-purpose graphics computer capable of generating three-dimensional displays, rotating in real time. We present now a brief discussion of each of the programs. 3. ISPY All data collection takes place under the supervision of ISPY (see Fig. 1). When the user accesses ISPY, a “master” menu is displayed (see Appendix). The significance of most of the options is self evident, but a few may be obscure. Thus, the stream spacing represents the interval between adjacent digitized points. Usually this is set to the resolution of the digitizer (i.e., 0.1 mm), but this can be reduced by the user in increments of one if it is desired to reduce the number of points recorded. The Z spacing corresponds to the absolute spacing between successive digitized serial sections. Changes in Z spacing are currently taken into account by altering the frame number. When the user has entered the above bookkeeping data, a submenu (see Appendix). is accessed by typing “M.” Data entry hereafter is controlled by pressing one of four colored buttons on the digitizer stylus or mouse. This submenu, in turn, accesses other submenus in order to accomplish specific tasks. One such submenu (not shown) permits the user to select the “window” size for the graphics display (i.e., this submenu governs the size of the display on the graphics terminal, but has no effect on the value of the coordinates recorded). This option allows small contours to be displayed using the full screen size. The submenu also allows the user to control the position of the contour on the display screen. Hence, over a significant range there is no necessary relationship between the size or position of a contour on the digitizer table and its size and position on the display screen. Data entry proceeds in either of two distinct modes: point digitization or stream digitization. In point digitization a point is recorded each time a button is pressed (i.e., one by one), whereas in stream digitization, points are recorded continuously as the mouse is moved. Under ISPY the fiducials are entered in
602
PROTHERO
AND
PROTHERO
the point digitization mode and the XY coordinates of preselected contours are digitized in the stream mode. Each contour is automatically indexed (1, 2, 3, etc.), but this may be overridden manually. When a displayed contour is examined and an error is found to be present, then the contour can be rejected and redigitized before initiating disk storage. This process is repeated until all the contours of interest on the section have been digitized and stored. It may be noted that ISPY continuously records the current frame, contour number, and number of points on each contour and displays these values. When the first frame is completed the user advances to the next frame, records the fiducials, then digitizes preselected contours. When digitization is to be ended, the user exits from the submenu and closes the file from the master menu. 4. DMPSPY The primary purpose of the DMPSPY program is to allow any and all frames to be displayed and plotted, both for error checking and for the production of permanent records. For the convenience of the user who is chiefly interested in two-dimensional data, we have made provision to allow the user to print out a list of the coordinates for each contour or to compute the cross-sectional area within each contour, or both. (Note that the list of coordinates obtained in this way is designed for off-line data processing only; the list is incompatible with EDSPY and STATSPY .) The options available with DMPSPY are shown in the Appendix. 5. EDSPY This program allows the user to correct errors made in data entry under ISPY. Two classes of errors need to be considered. The first includes “bookkeeping” errors (e.g., date, scale factors, contour numbers). All errors of this type can be corrected directly by simply typing in the correct values. The second class encompasses errors in digitizing (e.g., contours omitted or digitized incorrectly). Errors of this kind are corrected by redigitizing fiducials and all of the contour data on the specific frame in which the error occurs. The menus displayed when EDSPY is called are shown in the Appendix. 6. STATSPY The purpose of STATSPY is to produce morphometric statistics, such as cross-sectional areas and volumes, in tabular form. Typically one reads in data from a secondary file and stores the tabular output on a tertiary file (see Fig. 1). A useful feature of STATSPY is the provision for easy addition of supplementary statistical routines. For this purpose, a “dummy” program has been written that reads through a tertiary file in the proper order (i.e., frame, contour, coordinates). To compute a new statistic, the user need merely write a subroutine to perform the required computation, add it on to a (renamed) copy of the “dummy” program and insert a subroutine call at the prescribed place in
3D RECONSTRUCTION
FROM
SERIAL
SECTIONS
603
the new version of the dummy program. Thus the user need not replicate the software to read the data files or even know the details of the file structure to incorporate new statistical routines. 7. SUMMARY
MORPH0 is a well-documented, microcomputer-based FORTRAN package for carrying out three-dimensional reconstruction from serial sections. A series of programs guide the user in the collection, display, editing, and processing of morphometric data. The software can be adapted readily to any CP/M operating system, given the appropriate hardware, and the statistical routines can easily be expanded to meet special needs. APPENDIX ZSPY Master
A B D F M N 0 P S W X Z
Menu
Atlas name Bitpad least count Date Film scale (subject-to-film) Measure frames Name (of user) Organ name Pad scale (film-to-bitpad) Stream spacing (in least counts) ‘ ‘Whistle” contour number exit to CP/M, saving data Z spacing
ZSPY Submenu (Mouse
Menu)
Blue then white Skip current contour Green Change frame number White Measure contour Blue then green Increment frame number Yellow Return to master menu DMPSPY
A C D F 0 P R T
Master
Menu
Areas of contours Current frame (redisplay) Dump XY coordinates File i/o submenu Output frame to printer Plot current frame Restart program Translation submenu
PROTHEROANDPROTHERO
604 W Y X
Window Yank in next frame exit
First EDSP Y Menu A B P R
Atlas name Bitpad least count spacing Proceed Reference points (fiducials)
Second EDSPY Menu C D F I K M N 0 P R S W X Y Z
Current frame (redisplay) Date Film scale Insert new frame Kill current frame reMeasure frame Name of operator Organ name film-Pad scale factor Renumber contour Substitute frame number Window exit Yank in next frame Z spacing ACKNOWLEDGMENTS
We are grateful to Alan Lindsay, who wrote the “driver” programs which MORPH0 accesses, and who also provided helpful advice at a number of points. Our thanks to Ms. Doris Ringer for preparation of the manuscript. This work was supported in part by USPHS Grants AG-01751 to Dr. G. Martin and JR-OlGM13626 to Dr. J. Gallant, from the National Institutes of Health. REFERENCES 1. MACAGNO, E. R., LEVINTHAL, C., AND SOBEL, I. Three-dimensional computer reconstruction of neurons and neuronal assemblies. Ann. Rev. Bioeng. 8, 323 (1979). 2. LINDSAY, R. D. (Ed.) “Computer Analysis of Neuronal Structure.” Plenum, New York, 1977. 3. PROTHERO, J. W.. TAMARIN, A., AND PICKERING, R. Morphometrics of living specimens. A methodology for the quantitative three-dimensional study of growing microscopic embryos. J. Microsc. 101, 31 (1974). 4. TENNY, J. R., LONG, J. W., JR., MCFARLAND, W. D., VORBECK, M. L., TOWNSEND. J. F.. AND MARTIN, A. P. Computerized 3-dimensional analysis of mitochondrial structure. Compur. Prog. Biomed. x2, 1 (1980). 5. PROTHERO, J. W., AND SUNDSTEN, J. W. Three-dimensional reconstruction from serial sections. II. A microcomputer-based morphometric facility, in preparation.