- Email: [email protected]
Display systems in nuclear medicine
Display Systems in Nuclear Medicine A B Brdl and J J Erickson Image display has t r a d i t i o n a l l y been implicated as the l i m i t i n g f a c t o r in t h e u s e f u l n e s s o f d i g i t a l d a t a processing in nuclear medicine. This w a s partly due to limited t e c h n o l o g y and partly to an a t t e m p t to use a single system for all displays. Because of the w i d e l y varying characteristics of the various nuclear medicine procedures, no single system can p e r f o r m optimally in all studies. The various uses f o r the display in data processing are: user interaction, f e a t u r e extraction, q u a n t i t a t i v e p a r a m e t e r e s t i m a t i o n , correlation w i t h previous studies and reporting. User interaction poses the least stringent r e q u i r e m e n t s on the display. T y p e w r i t e r s , line printers, and standard c o m p u t e r t e r m i n a l s are adequate for this m o d e of operation. Display of m o n i t o r images to a l l o w the o p e r a t o r to f o l l o w the processing of the d a t a requires only a m o d e r a t e quality display, and, for this, a storage scope display or gray scale w i t h a limited n u m b e r of gray scales is useful. Interactive displays, which are used to guide the processing, must have s o m e means of a l l o w i n g the user to indicate regions of interest. Joysticks, cursors, or graphic t a b l e t s can p e r f o r m this function. For display of images t o be used f o r d i a g n o s t i c i n t e r p r e t a t i o n , a h i g h - q u a l i t y
image m u s t be produced, w h i c h a l l o w s the entire dynamic r ange of data to be s h o w n t o the clinician. Printing of multiple images on an electrostatic p l o t t e r or on f i l m used in a f l yi ng- spot scanner m o d e can provide the clinician w i t h such images. Display of dynamic d a t a places a n o t h e r restriction on the system in t h a t not only must spatial d i s t r i b u t i o n be represented, but the t e m p o r a l distribution o f activity must also be shown. This is approached in one of t hr ee ways. If t h e p r o b l e m is one of changing organ size, as in a cardiac ventricular ejection f r a c t i o n measurement, t h e n the overlaying of the o r g a n edges as a function o f time, along with a curve of t h e change in the area (or counts) as a function of t i me, is adequate. If, h o w e v e r , the problem is one of measuring a single organ f u n c t i o n as a f unct i on of t i m e as in a cardiac b o l u s s t u d y w h e r e t h e p r e s e n c e o f a s h u n t is s u s p e c t e d , t h e n o n e can use t i m e - c o d e d c o l o r displays in w h i c h the activity d i s t r i b u t i o n in selected t i m e s e g m e n t s is displayed in d i f f e r e n t colors. In this case, the over l appi ng activity areas are s h o w n as the summation of the individual colors, while the n o n o v e r l a p p i n g a r e a s r e m a i n in t h e i r r e s p e c t i v e colors.
HE A P P L I C A T I O N S of digital computers to image processing in medicine have grown significantly in recent years. In many instances, such as fast dynamic flow studies in nuclear medicine, the computer has made possible the performance of otherwise impossible studies. Often, however, the data processing systems in current use are carefully designed and optimized with regard to data collection and processing, but with only cursory attention to the display portion of the system. This is unf o r t u n a t e , since the inability to p r o p e r l y represent the images or data that have been processed will very quickly reduce the utility of the system to zero. This is especially so in nuclear medicine, where the computer is used to process and modify the data before presentation to the physician for diagnosis. In the past, it was not uncommon to hear discussions of sophisticated image-processing procedures purported to significantly i n c r e a s e the d i a g n o s t i c readability of images but which, in fact, were so limited by the display system that the unmodified analogue images were immensely superior to any of the processed images. In recent years, the growth of image technology and solid state electronics have produced many systems in
which it is difficult to tell that the image was ever anything but a high quality analogue image. Since no single system is ideal for all purposes, limitations of various kinds are encountered when any single type of display is required to perform all tasks. In this review, we shall review the different types of display systems available for use in nuclear medicine and indicate the applications for which each is best suited. Even though the display of the image is an extremely important element in the processing system, there are very few articles in the literature that deal solely with this subject. In
T
Seminars/n Nuclear Medicine, Vol VIII. No. 2 (April), 1978
From the Department of Radiology and Radiological Sciences, Vanderbilt University, Nashville, Tenn. A. B. Brill, M.D., Ph.D.: Professor of Radiology and Radiological Sciences, Vanderbilt University Medical Center; J. J. Erickson, Ph.D.: Assistant Professor of Radiology, Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, Tenn. Reprint requests should be addressed to ,Ion J. Erickson, Ph. D., Department of Radiology and Radiological Sciences, Vanderbilt University Medtcal Center, Nashville, Tenn 37232. ~ 1978 by Grune & Stratton, Inc. 0001-2998/78/0802-000151 00/0 155
156
BRILL AND ERICKSON
general, references to different display techniques and the particular attributes of each are found in articles whose main objective is the discussion of processing techniques or systems used at the various institutions. The references given in this article should provide a starting point for anyone interested in display processes, and as such are not intended to be an exhaustive list of information sources. Todd-Pokropek and Pizer, ~ in their detailed review of the factors affecting the use of scintigraphic displays, have summarized the functional tasks for which display systems are used in nuclear medicine as follows: 1. To detect those features of interest using a good display of the whole image. 2. To interact with the system to improve the display of these features. 3. To obtain a quantitative estimate of some parameter related to those features. 4. To relate this estimate to previous values obtained in other cases. 5. To reach some conclusion about the clinical relevance of these features. 6. To report in such a way as to influence p a t i e n t m a n a g e m e n t (and convince o t h e r clinicians). We will focus attention on the types of displays that we believe are best suited for specific nuclear medicine tasks. Such an appraisal is based upon personal experience and hence is necessarily biased. We will not avoid naming specific products when this is essential to the discussion, but will make no pretense of presenting a market survey in which the pros and cons of all products are reviewed. COMPUTER PROGRAM DEVELOPMENT, TESTING, AND NUMERICAL DATA ANALYSIS
Machine-readable punched cards have been used on large systems for many years for data and program inputs, while line printers are used for listings of programs, verifications of data entering, and printout of results. With the development of large r e m o t e time-sharing systems, terminations coupled by communication data links (commonly the telephone) have come to be the most common means of entering programs, data, interacting with programs at multiple stages, and receiving the results. These terminals utilize typewriters, lineprinters, and
cathode ray tube displays. With the rapid advances in computing technology, and the falling prices, "intelligent terminals" are available now at reasonable prices. These systems include microprocessors, data buffers (typically floppy disk or tape casettes), and CRY displays. Such terminals can be used for data entry, program development, and interactive data processing. Due to the fact that communications errors and system m a l f u n c t i o n s do o c c u r , it is necessary to include on-site data buffering, such as on casettes or floppy disks, to protect data files from inadvertent losses. MONITOR IMAGE PROCESSING
If the display is to be used for diagnostic purposes, that is, if the clinician is going to view the display directly, rather than some pbotographic hard copy, the images must be high-quality, flicker-free, and with sufficient intensity range to reproduce the data. Figure 1 shows an image of an anterior view of a canine chest. This is a photograph of the screen of a storage scope display. In this type of display, the gray scale is achieved by increasing the dot density to simulate increased brightness. This method of display is relatively inexpensive because it requires only a very simple computer interface and no external memory other than the storage scope. It does provide a very satisfactory monitor image, but suffers from several drawbacks that limit its usefulness in the clinical situation. Primary among these are the fact that the entire image must be erased and rewritten to display alterations in the image caused by data processing, plus the fact that the dot density method of intensity modulation causes loss of resolution in low-intensity regions. Figure 2A is a photograph of the same data as in Fig. 1, as shown on a display with 16 shades of gray. This display utilizes a digital memory for image storage on a color video monitor in a 128 • 128 format. This array size removes most of the edge distortion which results from the digital array. However, the 16 gray shades available in this system are not sufficient to produce a smooth image, free from the contouring introduced by limited gray scales. Satisfactory techniques for interacting with the image require the use of light pens, joysticks, cursors, or graphic tablets, probably in that order of decreasing general acceptability.
DISPLAY SYSTEMS
1 57
Fig. 1. A storage scope display of a canine chest. Pseudo-gray-scale is achieved in this display by variations in dot density and provides an adequate monitor image, but is limited in its diagnostic value.
The ability to edit the chosen region of interest without having to reenter the entire region is a software feature that should be adaptable to any of these modes of interaction. DISPLAYS OF STATIC IMAGES FOR DIAGNOSTIC INTERPRETATION
In reviewing the computer processed data, the physician can look at hard-copy images or at on-line displays. In the former case, he is restricted to the particular display factor choices made by the operator. The use of an online display system permits him to choose factors which reveal detail in either the high- or
low-intensity regions, both of which can not be viewed simultaneously with any single set of display factors. This mode of interaction is very useful for the experienced computer user, but it often limits the performance of persons unaccustomed to the use of such systems. Hard-copy displays of static images can be p r o d u c e d by r e g u l a r line p r i n t e r s using characters to encode gray levels, or contour elevations. 8 To enhance the range of gray scales available, one may overprint the same line multiple times. Recently, several companies have offered thermal printers that produce character and continuous gray scale images of high quality? One which we have used for the
Fig. 2. Part A shows the same data as Fig. 1, but as produced by a display having 16 true shades of gray. Contours produced by the limited number of shades of gray is evident. Part B is this image photographed by a flying spot scanner technique as described in the text, and shows no contouring as in image A or loss of resolution in the low count areas as in Fig. 1.
158
BRILL AND ERICKSON
A P
A
P
A
Fig. 3. An anterior and posterior whole body scan as displayed on an electrostatic printer/plotter, The image on the left (A) was generated in the "positive" mode of the printer, while those in the right (B) were produced in the "negative" mode~
past year* produces images with 2048 resolution etements across the paper, functions well for typical nuclear medicine, CT, and ultrasound images (It takes approximately 1 min to produce a 2048 • 2048 image (11" • 11") and costs approximately 10e/image for the paper.) Figure 3 shows an anterior and posterior whole body scan as produced on this printer. The images on the left were printed in "positive" mode, while those on the right were p r i n t e d in the " n e g a t i v e " m o d e o f the printer/plotter. One of the key advantages of such displays is that they can be used for the display of images with various aspect rations. Thus, a whole body image could be displayed as 11" • 44", 3" • 12" as desired. Similarly, such displays can be printed in color, using ink jet printers as developed in Sweden. Film is the usual means for generating hardcopy images from computers. High-quality flying spot scanners produce the best images. At the other end of the quality scale is the optical camera set up to photograph a TV
*ANAC model 911-ANAC Limited, 30 Trudy Lane, Menlo Park, CA 94045.
screen display. Photographic recording from CRTs run in a raster mode are intermediate in quality. In this case, gray scale can be achieved by differential intensification of the beam at each pixel location, or the beam can be energized a number of times proportional to desired image density at that spot. Figure 2B shows an image produced in this manner. Knowledge of the film H and D curve is essential to the proper exploitation of film properties for displaying image data. ~2 CRTs should have reasonably good linearity, stable electronics, and a phosphor properly matched to the film chosen for photography. The phosphor should be reasonably uniform over the surface of the tube, and for highest quality work, this surface should be mapped densitometrically, to establish correction factors. Standard interfaces exist on all computers for coupling CRTs (through D / A converters) and line printers are supported by hardware and software systems. These systems usually are driven by the c o m p u t e r memory and only minimally degrade computing capacity. When higher quality images are needed for graphs, contour, or volumetric displays, hardcopy point plotting devices (e.g., Calcomb or Houston Plotters) may be used. These devices
DISPLAY SYSTEMS
159
can produce high-quality multicolored plots (with changes in pens during the course of display generation) which can be used for publications directly, without further artwork required. These displays take longer to generate, and in general are not used for routine studies. DISPLAY OF DYNAMIC IMAGES
Physiological processes can be displayed by curves that show how (a) a function changes with time (e.g., renogram), (b) the outline of an anatomic region changes with time (radionuclide ventriculogram), (c) material moving through different subregions of the system with time, or (d) how the same system behaves with respect to different processes (ventilation/perfusion studies). In some cases, the same systems can be used for more than one type of study, but for simplicity, we will discuss these typical studies separately.
Changes in Parameters with Time
Fig. 4. A display of the results of an automated computer analysis of a ventricular volume measurement. The inset image shows the ventricular wall position as determined by the computer for each time frame in the cardiac cycle. The large curve in the ventricular volume as a function of time, while the smaller is its derivative.
In the evaluation of patients with renal transplants, we have found it convenient to prepare a sequential listing of the parameters from the renogram each time the patient has been studied. An updated list replaces the earlier lists and is stored in the patients file, which is reviewed each time the patient is studied. In addition, the kidney and bladder curves are displayed on a line printer output where the current study is superimposed on the temporal curve obtained at the last study. This permits a direct correlation of the two studies, limited only by knowledge of the comparability of the flagged regions of interest. The comprehensive study in our laboratory involves the use of a video disk in addition to the line printer record. On the video disk, we display six successive 5min images o f the h i p p u r a n d i s t r i b u t i o n (intensity modulated color-coded, hot-body spectrum), and an image of the flagged regions utilized for the current and previous study. Below these images are shown superimposed current and previous study time curves for kidney and bladder, separately. The color display is located in the scan-reading room, and the output of the video disk is controlled by a touchtone pad at the TV display.
cardiac cycle is the gated film recorder. Commercially available ECG gating systems permit the selection of time intervals referable to the QRS pulse. Images of the different selected phases are displayed on multiformated film recorders with suitable biases added to the X and Y signals, which occur in the selected time phases. Such systems can be used to make m a n y images t h r o u g h t e m p o r a l l y varying cyclical processes. By outlining one image on a transparency and overlaying it on the other image, the border behavior can be depicted accurately and cheaply. If the data are collected in a data processing system, it is possible to use software to determine the border motion. Figure 4 shows the results of such an analysis in the determination of left ventricular ejection fraction. The small inset image shows the ventricular borders as determined in each frame of the dynamic study. The large curve is the plot of counts in the ventricle versus time in the cardiac cycle, and is used for the calculation of ejection fraction. The smaller curve is the derivative of the larger curve and shows the time rate of change of the ventricular volume.
Changing Outline of a Moving Structure
Material Flow Through the System
The simplest hard-copy system for displaying the outline of the left ventricle during the
Three approaches have been taken to this four-dimensional display problem. Functional
160
images have been used to display transients through artificial organs and through stationary (i.e., nonpulsating) systems, such as the kidney and brain. In these cases, parameters like t~,, t.... and clearance times provide useful diagnostic information that can be recorded effectively on photographic film as intensity modulated so-called functional images. The display of flow through a pulsating system, i.e., blood flow through the heart or gas flow through the lungs, poses a more difficult problem. Mosaic displays, as suggested for outlining the ventricle, have been used by many workers. The most vivid displays, however, of cardiac and pulmonary dynamics show a cinematographic sequence. The eye-brain "computer" is remarkably good at identifying the different structures and processes by sequential replay of the same scene. Software systems on many of the computer devices used in nuclear medicine provide on-line displays of black/white and color images of the beating heart, which permits optimum utilization of the available data. In fact, such displays are limited mainly by technical factors in study conduct and in physician knowledge/judgment. The major problem is encountered when you want to transport information for presentation at conferences or meetings. This requires a communication line, the production of a movie film, or video recording. In our laboratory we have used a video disk for producing these displays with and without a computer. The system with a computer involves collecting a gated series of images in memory and then writing that sequence onto the disk through a special highspeed interface. Two methods that can be used without a computer are the collection of a series of time-gated images (either on an electrical storage tube ( E S T ) - - a s with the Unirad cardioniner, or on film--as described below). With the EST, a television readout is inherent in the device, and the study results are immediately available at the end of imaging. With the film mosaic, it is necessary to translate that image into a spatially registered sequence. We have done this by writing each image onto the video disk in sequence using a TV camera viewing one image at a time and writing it onto one or more sequential tracks. The disk can be replayed at various rates enabling careful study of temporal transients in the scene.
BRILL AND ERICKSON
The use of color for time coding lends itself to selection of nonoverlapping ROIs, e.g., in cardiac studies. When one images cardiac flow in successive phases through different color filters, the chambers and vessels seen in the original unblended colors represent nonoverlapping regions.
Multiple Functions in a Single Organ The correlation between ventilation and perfusion has been analyzed using wash-in and wash-out maps of radiogas and radiolabeled microspheres. The display of data relating ventilation, perfusion, and ratio parameters as a series of numbers may be accomplished with line printers or on storage scopes and associated hard-copy printouts (commercial systems). These systems are plagued by the fact that it is difficult to get people to study displays carefully that have many numbers. Thus, to achieve user acceptance one reduces the number of values and, consequently, also the resolution of the information. One way of overcoming this difficulty involves the use of twostage strategy: (1) qualitative viewing of the changing scene to identify ROI, followed by (2) quantitative analysis of the data in the selected ROI. This is the same strategy used in the analysis of static images. In the case of lung imaging, the following strategy is used. Timegated ventilation maps are viewed, using the same systems as for temporally varying cardiac studies. This provides information on the distribution of ventilation. Ventilation parameters are computed using washout slopes or other flow parameters (height/area, centroids, etc.). The perfusion image is acquired without moving the patient, so the images are properly registered. The images are displayed in two colors, e.g., the ventilation parameters in green, the perfusion parameter in red, with areas that have V and P shown in yellow (the mixture of R + G). In this way, one identifies ROI where unbalanced function occurs, and then tests the V and P parameters therein for statistical significance. One views x-ray findings in the areas with extra care, as well as the time histories of ventilation and perfusion. WHICH DISPLAY SYSTEM TO BUY
The cost of computers is coming down at a rapid rate. In fact, the cost of the central
DISPLAY SYSTEMS
1 61
p r o c e s s o r and m e m o r y is a very s m a l l fraction o f t h e cost o f the s y s t e m . T h e choice o f d i s p l a y s y s t e m s is v e r y m u c h d e p e n d e n t on the studies t h a t will be c a r r i e d out on the system. T h e cost o f high p e r f o r m a n c e d i s p l a y s y s t e m s m a y exceed t h a t o f t h e c o m p u t e r itself, and it m a y be n e c e s s a r y to s h a r e such systems. The cost of intelligent terminals is $2,000 $10,000. T h e cost o f line p r i n t e r s varies from $4,000 to very l a r g e costs, d e p e n d i n g on speed. T h e usual small c o m p u t e r in n u c l e a r medicine should have such a device if a significant a m o u n t o f p r o g r a m d e v e l o p m e n t is anticipated. Line p r i n t e r s that can be u s e d for g r a y scale, as well as text images, should be considered when l a r g e volumes o f printing a r e not expected, S i m p l e C R T d i s p l a y s for photo g e n e r a t i o n can be a c q u i r e d for $2,000-$5,000 and should be quite s a t i s f a c t o r y for routine static displays. Int e r f a c e s to m o s a i c displays such as M i c r o d o t ( S e a r l e ) a r e a v a i l a b l e and these devices p r o d u c e high-quality i m a g e s , at higher cost.
High-quality buffered disp/ay s y s t e m s a r e now available, and s o m e o f t h e m c a r r y o u t h a r d w a r e p r o c e s s i n g options as well as i m a g e buffering. Real t i m e fast F o u r i e r t r a n s f o r m s , convolution and deconvolution, i m a g e p e r s p e c tive changes, etc., d i s p l a y e d in b l a c k and white, or color, and s u p e r i m p o s e d i m a g e s can be achieved with off-the-shelf turn k e y s y s t e m s . The costs of such systems range from $15,000-$75,000, depending on t h e options selected. M o r e e m p h a s i s and m o n e y should be invested in t h e display s y s t e m when a system is p u r c h a s e d for m e d i c a l imaging. T h e possibility o f sharing such s y s t e m s with CT, and U l t r a sound devices m a y m a k e such acquisitions m o r e feasible, and the sharing o f unusual display s y s t e m s should be explored when p u r c h a s i n g a c o m p u t e r for use in medical imaging analysis. ACKNOWLEDGMENT
The authors would like to acknowledge the assistance of Drs. James A. Patton, Jerome Jones, and Stanley Higgins. and Mr. Robert Bowen for their assistance in obtaining the illustrations.
REFERENCES
1. Todd-Pokropek AE, Pizer, SM: Displays in Scintigraphy in Medical Radionuclide Imaging, Vol t. Proceedings of a Symposium, Los Angeles, (1976). Vienna, IAEA, 1977, p 505 2. Budinger TF: Clinical and Research Quantitative Nuclear Medicine System in Medical Radioisotope Scintigraphy, Vol 1. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, IAEA, 1973, p 501 3, Adams R, Jansen C, Braun E, et al: Quantitative Display of Image Data as Colour-Coded Isocount Contours in Medical Radioisotope Scintigraphy, Vol I. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, IAEA, 1973, p 557 4. Lorenz WJ, Georgi P, Meder HG, et al: Interactive Processing and D~splayingof Digital Scintigrams in Medical Radioisotope Scintigraphy, Vol 1. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, IAEA, 1973, p613 5. Jordan K, Geisler S: Data Display in Scintigraphy by Means of a High Speed Electrostatic Digital Plotter and Special Computer Averaging Techniques in Medical Radioisotope Scintigraphy, Vol 1. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, 1AEA, 1973, p 635 6. Todd-Pokropek AE: An Investigation Using MonteCarlo Techniques and Gaming Theory into the Value of Digital Radioisotope Display Systems in Medical Radio-
isotope Scintigraphy, Vol 1. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, IAEA, 1973, p 705 7. Metz CE, Starr S J, Lusted LB: Quantitatwe Evaluation of Medical Imaging in Medical Radionuclide Imaging, Vol I. Proceedings of a Symposium, Los Angeles, 1976. IAEA, Vienna, 1977, p 491 8. Tauxe WN, Chaapel DW, Sprau AC: Use of HighSpeed Digital Computers in Processing Radioisotope Scintiscan Matrices, Stacy RW and Waxman BD (eds): Computers in Biomedical Research, Vol III. New York. Academic Press, 1966, p 145 9. Wemer SM, Borkat FR, Floyd RM: Functional imaging: A method of analysis and display of regional rate constants, J Nucl Med 15:65-68, 1974 10. Toyama H, Lil M, Iisaka J, et al: Color functional images of the cerebral blood flow, J Nucl Med 17:953, 1976 11. Todd-Pokropek AE: Display Systems in Clinical Practice, in Kenny PJ, Smith EM (eds): Quantitative Organ Visualization in Nuclear Medicine. University of Miami Press, Coral Gables, FI, 1971, p 717 12. Carlson JC: Greyscale Distortion of Computer Displays in Brain Imaging in Proceedings of the Third Symposium on Sharing of Computer Programs and Technology in Nuclear Medicine, Miami, Florida, U. S. Technical Information Center, AEC CONF-730627, p 233
image m u s t be produced, w h i c h a l l o w s the entire dynamic r ange of data to be s h o w n t o the clinician. Printing of multiple images on an electrostatic p l o t t e r or on f i l m used in a f l yi ng- spot scanner m o d e can provide the clinician w i t h such images. Display of dynamic d a t a places a n o t h e r restriction on the system in t h a t not only must spatial d i s t r i b u t i o n be represented, but the t e m p o r a l distribution o f activity must also be shown. This is approached in one of t hr ee ways. If t h e p r o b l e m is one of changing organ size, as in a cardiac ventricular ejection f r a c t i o n measurement, t h e n the overlaying of the o r g a n edges as a function o f time, along with a curve of t h e change in the area (or counts) as a function of t i me, is adequate. If, h o w e v e r , the problem is one of measuring a single organ f u n c t i o n as a f unct i on of t i m e as in a cardiac b o l u s s t u d y w h e r e t h e p r e s e n c e o f a s h u n t is s u s p e c t e d , t h e n o n e can use t i m e - c o d e d c o l o r displays in w h i c h the activity d i s t r i b u t i o n in selected t i m e s e g m e n t s is displayed in d i f f e r e n t colors. In this case, the over l appi ng activity areas are s h o w n as the summation of the individual colors, while the n o n o v e r l a p p i n g a r e a s r e m a i n in t h e i r r e s p e c t i v e colors.
HE A P P L I C A T I O N S of digital computers to image processing in medicine have grown significantly in recent years. In many instances, such as fast dynamic flow studies in nuclear medicine, the computer has made possible the performance of otherwise impossible studies. Often, however, the data processing systems in current use are carefully designed and optimized with regard to data collection and processing, but with only cursory attention to the display portion of the system. This is unf o r t u n a t e , since the inability to p r o p e r l y represent the images or data that have been processed will very quickly reduce the utility of the system to zero. This is especially so in nuclear medicine, where the computer is used to process and modify the data before presentation to the physician for diagnosis. In the past, it was not uncommon to hear discussions of sophisticated image-processing procedures purported to significantly i n c r e a s e the d i a g n o s t i c readability of images but which, in fact, were so limited by the display system that the unmodified analogue images were immensely superior to any of the processed images. In recent years, the growth of image technology and solid state electronics have produced many systems in
which it is difficult to tell that the image was ever anything but a high quality analogue image. Since no single system is ideal for all purposes, limitations of various kinds are encountered when any single type of display is required to perform all tasks. In this review, we shall review the different types of display systems available for use in nuclear medicine and indicate the applications for which each is best suited. Even though the display of the image is an extremely important element in the processing system, there are very few articles in the literature that deal solely with this subject. In
T
Seminars/n Nuclear Medicine, Vol VIII. No. 2 (April), 1978
From the Department of Radiology and Radiological Sciences, Vanderbilt University, Nashville, Tenn. A. B. Brill, M.D., Ph.D.: Professor of Radiology and Radiological Sciences, Vanderbilt University Medical Center; J. J. Erickson, Ph.D.: Assistant Professor of Radiology, Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, Tenn. Reprint requests should be addressed to ,Ion J. Erickson, Ph. D., Department of Radiology and Radiological Sciences, Vanderbilt University Medtcal Center, Nashville, Tenn 37232. ~ 1978 by Grune & Stratton, Inc. 0001-2998/78/0802-000151 00/0 155
156
BRILL AND ERICKSON
general, references to different display techniques and the particular attributes of each are found in articles whose main objective is the discussion of processing techniques or systems used at the various institutions. The references given in this article should provide a starting point for anyone interested in display processes, and as such are not intended to be an exhaustive list of information sources. Todd-Pokropek and Pizer, ~ in their detailed review of the factors affecting the use of scintigraphic displays, have summarized the functional tasks for which display systems are used in nuclear medicine as follows: 1. To detect those features of interest using a good display of the whole image. 2. To interact with the system to improve the display of these features. 3. To obtain a quantitative estimate of some parameter related to those features. 4. To relate this estimate to previous values obtained in other cases. 5. To reach some conclusion about the clinical relevance of these features. 6. To report in such a way as to influence p a t i e n t m a n a g e m e n t (and convince o t h e r clinicians). We will focus attention on the types of displays that we believe are best suited for specific nuclear medicine tasks. Such an appraisal is based upon personal experience and hence is necessarily biased. We will not avoid naming specific products when this is essential to the discussion, but will make no pretense of presenting a market survey in which the pros and cons of all products are reviewed. COMPUTER PROGRAM DEVELOPMENT, TESTING, AND NUMERICAL DATA ANALYSIS
Machine-readable punched cards have been used on large systems for many years for data and program inputs, while line printers are used for listings of programs, verifications of data entering, and printout of results. With the development of large r e m o t e time-sharing systems, terminations coupled by communication data links (commonly the telephone) have come to be the most common means of entering programs, data, interacting with programs at multiple stages, and receiving the results. These terminals utilize typewriters, lineprinters, and
cathode ray tube displays. With the rapid advances in computing technology, and the falling prices, "intelligent terminals" are available now at reasonable prices. These systems include microprocessors, data buffers (typically floppy disk or tape casettes), and CRY displays. Such terminals can be used for data entry, program development, and interactive data processing. Due to the fact that communications errors and system m a l f u n c t i o n s do o c c u r , it is necessary to include on-site data buffering, such as on casettes or floppy disks, to protect data files from inadvertent losses. MONITOR IMAGE PROCESSING
If the display is to be used for diagnostic purposes, that is, if the clinician is going to view the display directly, rather than some pbotographic hard copy, the images must be high-quality, flicker-free, and with sufficient intensity range to reproduce the data. Figure 1 shows an image of an anterior view of a canine chest. This is a photograph of the screen of a storage scope display. In this type of display, the gray scale is achieved by increasing the dot density to simulate increased brightness. This method of display is relatively inexpensive because it requires only a very simple computer interface and no external memory other than the storage scope. It does provide a very satisfactory monitor image, but suffers from several drawbacks that limit its usefulness in the clinical situation. Primary among these are the fact that the entire image must be erased and rewritten to display alterations in the image caused by data processing, plus the fact that the dot density method of intensity modulation causes loss of resolution in low-intensity regions. Figure 2A is a photograph of the same data as in Fig. 1, as shown on a display with 16 shades of gray. This display utilizes a digital memory for image storage on a color video monitor in a 128 • 128 format. This array size removes most of the edge distortion which results from the digital array. However, the 16 gray shades available in this system are not sufficient to produce a smooth image, free from the contouring introduced by limited gray scales. Satisfactory techniques for interacting with the image require the use of light pens, joysticks, cursors, or graphic tablets, probably in that order of decreasing general acceptability.
DISPLAY SYSTEMS
1 57
Fig. 1. A storage scope display of a canine chest. Pseudo-gray-scale is achieved in this display by variations in dot density and provides an adequate monitor image, but is limited in its diagnostic value.
The ability to edit the chosen region of interest without having to reenter the entire region is a software feature that should be adaptable to any of these modes of interaction. DISPLAYS OF STATIC IMAGES FOR DIAGNOSTIC INTERPRETATION
In reviewing the computer processed data, the physician can look at hard-copy images or at on-line displays. In the former case, he is restricted to the particular display factor choices made by the operator. The use of an online display system permits him to choose factors which reveal detail in either the high- or
low-intensity regions, both of which can not be viewed simultaneously with any single set of display factors. This mode of interaction is very useful for the experienced computer user, but it often limits the performance of persons unaccustomed to the use of such systems. Hard-copy displays of static images can be p r o d u c e d by r e g u l a r line p r i n t e r s using characters to encode gray levels, or contour elevations. 8 To enhance the range of gray scales available, one may overprint the same line multiple times. Recently, several companies have offered thermal printers that produce character and continuous gray scale images of high quality? One which we have used for the
Fig. 2. Part A shows the same data as Fig. 1, but as produced by a display having 16 true shades of gray. Contours produced by the limited number of shades of gray is evident. Part B is this image photographed by a flying spot scanner technique as described in the text, and shows no contouring as in image A or loss of resolution in the low count areas as in Fig. 1.
158
BRILL AND ERICKSON
A P
A
P
A
Fig. 3. An anterior and posterior whole body scan as displayed on an electrostatic printer/plotter, The image on the left (A) was generated in the "positive" mode of the printer, while those in the right (B) were produced in the "negative" mode~
past year* produces images with 2048 resolution etements across the paper, functions well for typical nuclear medicine, CT, and ultrasound images (It takes approximately 1 min to produce a 2048 • 2048 image (11" • 11") and costs approximately 10e/image for the paper.) Figure 3 shows an anterior and posterior whole body scan as produced on this printer. The images on the left were printed in "positive" mode, while those on the right were p r i n t e d in the " n e g a t i v e " m o d e o f the printer/plotter. One of the key advantages of such displays is that they can be used for the display of images with various aspect rations. Thus, a whole body image could be displayed as 11" • 44", 3" • 12" as desired. Similarly, such displays can be printed in color, using ink jet printers as developed in Sweden. Film is the usual means for generating hardcopy images from computers. High-quality flying spot scanners produce the best images. At the other end of the quality scale is the optical camera set up to photograph a TV
*ANAC model 911-ANAC Limited, 30 Trudy Lane, Menlo Park, CA 94045.
screen display. Photographic recording from CRTs run in a raster mode are intermediate in quality. In this case, gray scale can be achieved by differential intensification of the beam at each pixel location, or the beam can be energized a number of times proportional to desired image density at that spot. Figure 2B shows an image produced in this manner. Knowledge of the film H and D curve is essential to the proper exploitation of film properties for displaying image data. ~2 CRTs should have reasonably good linearity, stable electronics, and a phosphor properly matched to the film chosen for photography. The phosphor should be reasonably uniform over the surface of the tube, and for highest quality work, this surface should be mapped densitometrically, to establish correction factors. Standard interfaces exist on all computers for coupling CRTs (through D / A converters) and line printers are supported by hardware and software systems. These systems usually are driven by the c o m p u t e r memory and only minimally degrade computing capacity. When higher quality images are needed for graphs, contour, or volumetric displays, hardcopy point plotting devices (e.g., Calcomb or Houston Plotters) may be used. These devices
DISPLAY SYSTEMS
159
can produce high-quality multicolored plots (with changes in pens during the course of display generation) which can be used for publications directly, without further artwork required. These displays take longer to generate, and in general are not used for routine studies. DISPLAY OF DYNAMIC IMAGES
Physiological processes can be displayed by curves that show how (a) a function changes with time (e.g., renogram), (b) the outline of an anatomic region changes with time (radionuclide ventriculogram), (c) material moving through different subregions of the system with time, or (d) how the same system behaves with respect to different processes (ventilation/perfusion studies). In some cases, the same systems can be used for more than one type of study, but for simplicity, we will discuss these typical studies separately.
Changes in Parameters with Time
Fig. 4. A display of the results of an automated computer analysis of a ventricular volume measurement. The inset image shows the ventricular wall position as determined by the computer for each time frame in the cardiac cycle. The large curve in the ventricular volume as a function of time, while the smaller is its derivative.
In the evaluation of patients with renal transplants, we have found it convenient to prepare a sequential listing of the parameters from the renogram each time the patient has been studied. An updated list replaces the earlier lists and is stored in the patients file, which is reviewed each time the patient is studied. In addition, the kidney and bladder curves are displayed on a line printer output where the current study is superimposed on the temporal curve obtained at the last study. This permits a direct correlation of the two studies, limited only by knowledge of the comparability of the flagged regions of interest. The comprehensive study in our laboratory involves the use of a video disk in addition to the line printer record. On the video disk, we display six successive 5min images o f the h i p p u r a n d i s t r i b u t i o n (intensity modulated color-coded, hot-body spectrum), and an image of the flagged regions utilized for the current and previous study. Below these images are shown superimposed current and previous study time curves for kidney and bladder, separately. The color display is located in the scan-reading room, and the output of the video disk is controlled by a touchtone pad at the TV display.
cardiac cycle is the gated film recorder. Commercially available ECG gating systems permit the selection of time intervals referable to the QRS pulse. Images of the different selected phases are displayed on multiformated film recorders with suitable biases added to the X and Y signals, which occur in the selected time phases. Such systems can be used to make m a n y images t h r o u g h t e m p o r a l l y varying cyclical processes. By outlining one image on a transparency and overlaying it on the other image, the border behavior can be depicted accurately and cheaply. If the data are collected in a data processing system, it is possible to use software to determine the border motion. Figure 4 shows the results of such an analysis in the determination of left ventricular ejection fraction. The small inset image shows the ventricular borders as determined in each frame of the dynamic study. The large curve is the plot of counts in the ventricle versus time in the cardiac cycle, and is used for the calculation of ejection fraction. The smaller curve is the derivative of the larger curve and shows the time rate of change of the ventricular volume.
Changing Outline of a Moving Structure
Material Flow Through the System
The simplest hard-copy system for displaying the outline of the left ventricle during the
Three approaches have been taken to this four-dimensional display problem. Functional
160
images have been used to display transients through artificial organs and through stationary (i.e., nonpulsating) systems, such as the kidney and brain. In these cases, parameters like t~,, t.... and clearance times provide useful diagnostic information that can be recorded effectively on photographic film as intensity modulated so-called functional images. The display of flow through a pulsating system, i.e., blood flow through the heart or gas flow through the lungs, poses a more difficult problem. Mosaic displays, as suggested for outlining the ventricle, have been used by many workers. The most vivid displays, however, of cardiac and pulmonary dynamics show a cinematographic sequence. The eye-brain "computer" is remarkably good at identifying the different structures and processes by sequential replay of the same scene. Software systems on many of the computer devices used in nuclear medicine provide on-line displays of black/white and color images of the beating heart, which permits optimum utilization of the available data. In fact, such displays are limited mainly by technical factors in study conduct and in physician knowledge/judgment. The major problem is encountered when you want to transport information for presentation at conferences or meetings. This requires a communication line, the production of a movie film, or video recording. In our laboratory we have used a video disk for producing these displays with and without a computer. The system with a computer involves collecting a gated series of images in memory and then writing that sequence onto the disk through a special highspeed interface. Two methods that can be used without a computer are the collection of a series of time-gated images (either on an electrical storage tube ( E S T ) - - a s with the Unirad cardioniner, or on film--as described below). With the EST, a television readout is inherent in the device, and the study results are immediately available at the end of imaging. With the film mosaic, it is necessary to translate that image into a spatially registered sequence. We have done this by writing each image onto the video disk in sequence using a TV camera viewing one image at a time and writing it onto one or more sequential tracks. The disk can be replayed at various rates enabling careful study of temporal transients in the scene.
BRILL AND ERICKSON
The use of color for time coding lends itself to selection of nonoverlapping ROIs, e.g., in cardiac studies. When one images cardiac flow in successive phases through different color filters, the chambers and vessels seen in the original unblended colors represent nonoverlapping regions.
Multiple Functions in a Single Organ The correlation between ventilation and perfusion has been analyzed using wash-in and wash-out maps of radiogas and radiolabeled microspheres. The display of data relating ventilation, perfusion, and ratio parameters as a series of numbers may be accomplished with line printers or on storage scopes and associated hard-copy printouts (commercial systems). These systems are plagued by the fact that it is difficult to get people to study displays carefully that have many numbers. Thus, to achieve user acceptance one reduces the number of values and, consequently, also the resolution of the information. One way of overcoming this difficulty involves the use of twostage strategy: (1) qualitative viewing of the changing scene to identify ROI, followed by (2) quantitative analysis of the data in the selected ROI. This is the same strategy used in the analysis of static images. In the case of lung imaging, the following strategy is used. Timegated ventilation maps are viewed, using the same systems as for temporally varying cardiac studies. This provides information on the distribution of ventilation. Ventilation parameters are computed using washout slopes or other flow parameters (height/area, centroids, etc.). The perfusion image is acquired without moving the patient, so the images are properly registered. The images are displayed in two colors, e.g., the ventilation parameters in green, the perfusion parameter in red, with areas that have V and P shown in yellow (the mixture of R + G). In this way, one identifies ROI where unbalanced function occurs, and then tests the V and P parameters therein for statistical significance. One views x-ray findings in the areas with extra care, as well as the time histories of ventilation and perfusion. WHICH DISPLAY SYSTEM TO BUY
The cost of computers is coming down at a rapid rate. In fact, the cost of the central
DISPLAY SYSTEMS
1 61
p r o c e s s o r and m e m o r y is a very s m a l l fraction o f t h e cost o f the s y s t e m . T h e choice o f d i s p l a y s y s t e m s is v e r y m u c h d e p e n d e n t on the studies t h a t will be c a r r i e d out on the system. T h e cost o f high p e r f o r m a n c e d i s p l a y s y s t e m s m a y exceed t h a t o f t h e c o m p u t e r itself, and it m a y be n e c e s s a r y to s h a r e such systems. The cost of intelligent terminals is $2,000 $10,000. T h e cost o f line p r i n t e r s varies from $4,000 to very l a r g e costs, d e p e n d i n g on speed. T h e usual small c o m p u t e r in n u c l e a r medicine should have such a device if a significant a m o u n t o f p r o g r a m d e v e l o p m e n t is anticipated. Line p r i n t e r s that can be u s e d for g r a y scale, as well as text images, should be considered when l a r g e volumes o f printing a r e not expected, S i m p l e C R T d i s p l a y s for photo g e n e r a t i o n can be a c q u i r e d for $2,000-$5,000 and should be quite s a t i s f a c t o r y for routine static displays. Int e r f a c e s to m o s a i c displays such as M i c r o d o t ( S e a r l e ) a r e a v a i l a b l e and these devices p r o d u c e high-quality i m a g e s , at higher cost.
High-quality buffered disp/ay s y s t e m s a r e now available, and s o m e o f t h e m c a r r y o u t h a r d w a r e p r o c e s s i n g options as well as i m a g e buffering. Real t i m e fast F o u r i e r t r a n s f o r m s , convolution and deconvolution, i m a g e p e r s p e c tive changes, etc., d i s p l a y e d in b l a c k and white, or color, and s u p e r i m p o s e d i m a g e s can be achieved with off-the-shelf turn k e y s y s t e m s . The costs of such systems range from $15,000-$75,000, depending on t h e options selected. M o r e e m p h a s i s and m o n e y should be invested in t h e display s y s t e m when a system is p u r c h a s e d for m e d i c a l imaging. T h e possibility o f sharing such s y s t e m s with CT, and U l t r a sound devices m a y m a k e such acquisitions m o r e feasible, and the sharing o f unusual display s y s t e m s should be explored when p u r c h a s i n g a c o m p u t e r for use in medical imaging analysis. ACKNOWLEDGMENT
The authors would like to acknowledge the assistance of Drs. James A. Patton, Jerome Jones, and Stanley Higgins. and Mr. Robert Bowen for their assistance in obtaining the illustrations.
REFERENCES
1. Todd-Pokropek AE, Pizer, SM: Displays in Scintigraphy in Medical Radionuclide Imaging, Vol t. Proceedings of a Symposium, Los Angeles, (1976). Vienna, IAEA, 1977, p 505 2. Budinger TF: Clinical and Research Quantitative Nuclear Medicine System in Medical Radioisotope Scintigraphy, Vol 1. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, IAEA, 1973, p 501 3, Adams R, Jansen C, Braun E, et al: Quantitative Display of Image Data as Colour-Coded Isocount Contours in Medical Radioisotope Scintigraphy, Vol I. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, IAEA, 1973, p 557 4. Lorenz WJ, Georgi P, Meder HG, et al: Interactive Processing and D~splayingof Digital Scintigrams in Medical Radioisotope Scintigraphy, Vol 1. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, IAEA, 1973, p613 5. Jordan K, Geisler S: Data Display in Scintigraphy by Means of a High Speed Electrostatic Digital Plotter and Special Computer Averaging Techniques in Medical Radioisotope Scintigraphy, Vol 1. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, 1AEA, 1973, p 635 6. Todd-Pokropek AE: An Investigation Using MonteCarlo Techniques and Gaming Theory into the Value of Digital Radioisotope Display Systems in Medical Radio-
isotope Scintigraphy, Vol 1. Proceedings of a Symposium, Monte Carlo, 1972. Vienna, IAEA, 1973, p 705 7. Metz CE, Starr S J, Lusted LB: Quantitatwe Evaluation of Medical Imaging in Medical Radionuclide Imaging, Vol I. Proceedings of a Symposium, Los Angeles, 1976. IAEA, Vienna, 1977, p 491 8. Tauxe WN, Chaapel DW, Sprau AC: Use of HighSpeed Digital Computers in Processing Radioisotope Scintiscan Matrices, Stacy RW and Waxman BD (eds): Computers in Biomedical Research, Vol III. New York. Academic Press, 1966, p 145 9. Wemer SM, Borkat FR, Floyd RM: Functional imaging: A method of analysis and display of regional rate constants, J Nucl Med 15:65-68, 1974 10. Toyama H, Lil M, Iisaka J, et al: Color functional images of the cerebral blood flow, J Nucl Med 17:953, 1976 11. Todd-Pokropek AE: Display Systems in Clinical Practice, in Kenny PJ, Smith EM (eds): Quantitative Organ Visualization in Nuclear Medicine. University of Miami Press, Coral Gables, FI, 1971, p 717 12. Carlson JC: Greyscale Distortion of Computer Displays in Brain Imaging in Proceedings of the Third Symposium on Sharing of Computer Programs and Technology in Nuclear Medicine, Miami, Florida, U. S. Technical Information Center, AEC CONF-730627, p 233