- Email: [email protected]
Applications of three-dimensional display techniques in medical imaging
Applications of three-dimensional techniques in medical imaging A. B. Strong, S. Lobregt*
and F. W. Zonneveld*
Medical Imaging Technology, Netherlands
Aylesbury,
Bucks, UK; *Philips
display
Medical Systems,
Best, The
ABSTRACT This paper outlines some of the current uses of three-dimensional technique-s in medical imaging applications and their potentialfir thefuture. As an exam@, three-dimensional imaging is described using a CTscanner, as it applies to a case involving craniofacil surgery. K%is includes defining the relationship between the requirements on tire data acquisition system, as well as the spec$cation of the hardware andsojware for the display. A current4 used a&orithm is &scribedfor the display of surjaces as a fin&ion of localposition, orientation of the sutface and theposition of a virtual light source. lItis includes the use of transparency and cut plane greyscale techniques, in addition to the display of the surfaces. A speculation is ma& regarding the use of a three-dimensional display as the standard viewing mode in CT, with slice and multiplanar imaging as submodes. Keywords:
Three-dimensional imaging, CT displays, image processing
INTRODUCTION
Table 1
Medical three-dimensional imaging’” differs from that applied to computer-aided design (CAD) graphics in that the object points are given. The key is the appropriate selection of the points, from a volume of interest (VOI) and their subsequent display, as grey values. It should not be underestimated how well a skilled radiologist can ‘reconstruct’ in his/her mind a threedimensional image from conventional two-dimensional lane films. There are, though, occasions when th ree B imensions can be of great assistance in he1 ing to visualize a lesion. This is particularly so when tI: ere has been severe distortion, caused by injury or the presence of an alien, space-occupying structure. It is to overcome the disorientating effect of distortions that anatomy is often taught in layers, rather than absolute positions. Even in the case of a misplacement, the various structures tend to remain in the same order. In medical imaging, three dimensions can be applied to a number of imaging modalities such as CT and MRI (see Table 7). The list in Table 7 is divided into those imaging modalities which inherently produce three-dimensional images and those modalities where three-dimensional images are derived from a series of reconstructed slices. This article describes an example of the latter.
Three-dimensional imaging from slices
Correspondence and reprint requests to: Mr Antony B. Strong, Medical Imaging Technology, 2 Castle Court, Aylesbury, Bucks HP20 2RD, UK
Imaging modalities applicable to three-dimensions
Computer tomography (CT) Magnetic resonance imaging (MRI) Positron emission tomography (PET) Single photon emission computed tomography (SECT) Ultrasonics (US) Direct three-dimensional imaging (without reconstruction) Flashing tomosynthesis Compton scatter imaging
APPLICATION OF THREE-DIMENSIONAL IMAGINGINCTAND~SFU’IVREPOTENTIAL The applications of three-dimensional images to CT have increased over the last few years. Examples of clinical applications include those shown in Table 2. The use of three-dimensional images in surgical planning has shown a particularly strong growth as a tool to minimize the time for interventional procedures and to speed up time to full recovery. In this article an example of three-dimensional imaging using a CT scanner as it applies to a craniofacial surgery case is described-. It is interesting to see that the use of three-dimensional imaging has changed the Table 2
Applications of three-dimensional imaging
Surgical planning Prosthesis design Stereotaxis in the head Radiation treatment planning
0 1990 Butterworths for BES 0141.5425/90/030233-06 J. Biomed. Eng. 1990, Vol 12, May
233
Three-dimtnsionalimaging:A.B. Stronget al.
roles of those involved in the hospital. It means, for example, that the surgeon is interacting with the display system himself directly, as well as receiving advice from the radiologist. The radiologist is, on the other hand, also having his/her role changed, because the use of injected contrast media and active catheters, becomes more widespread. Prosthesis, or the making of implants for patients, has also become a key application of threedimensional techniques. Again, much time can be saved by designing bone replacements or artificial hip ‘oints to fit first time, rather than having to be mo d ified, at the time of insertion. It is, in fact, not uncommon to feed the output of the three-
dimensional reconstruction directly to a numerically controlled milling machine, to make the correct shaped part. Stereotaxis has come into its own for doing tissue biopsies or inserting radioactive beads for treating tumours. The position and best direction of insertion, can be better seen with the use of a three-dimensional presentation. Three-dimensional imaging can also play a part in ensuring maximum exposure to a tumour during radiation treatment and minimizing the dose to the surrounding tissues. This can be done by overlaying a three-dimensional image of the tumour, with a tlireedimensional reconstruction of the dose profiles. The
Figure 1 a, Surface display of a skull; b, cut plane showing brain tissue; c, skull with internal cut plane; d, partial transparency showing skull and relative position of ventricles
234
J. Biomed. Eng. 1990, Vol 12, May
Three-dimensional imaging:A.B. Strong et al.
spatial matching can be seen more clearly using this viewing technique. In all of the above applications and many others, three-dimensional imaging as it applies to the display s stem usually refers to the use of a pseudo threedy imensional image where a two-dimensional projection of a three-dimensional object is displayed. The impression of three dimensions is given by the use of shading, perspective and rotatin the pro’ected object on the screen. This type of disp Hay may $ e enhanced with effect by the use of a stereoscopic technique. For this method, the alternate half frames of a video image can be used to dis lay the two images, with Polaroid screens to m 9 e the three-dimensional image v&able to the observer. True three-dimensional images may also be displayed by using holograms or the vibrating mirrors technique, but these methods are rarely on commercial offer. In addition to the dis lay of surfaces, it may be possible to show cut p Panes, where the greyscale values in the source slices are used to form the images in the planes. It may also be useful to combine soft tissues and bone in the same image, or to show degrees of transparency, so the relative osition of structures may be appreciated more rea B ily. Examples of some of the above three-dimensional images, as obtained with CT, are shown in Figure 1. FtEQ-S PRODUCING
IN A CT SYSTEM THREE-DIMENSIONAL
FOR IMAGES
The requirements for the system architecture in a CT scanner, to be used to generate three-dimensional images, involves the whole system and is not confined to the display system alone. All of the building blocks must be matched together to ensure optimum erformance. The system must be capable of taking t! in, well collimated slices, to ensure adequate resolution in the 2 direction and to reduce dose to the patient. It is also necessary to have a fast serial scan mode, to produce the 100 or so images required, in an acceptable lapsed time. These must also be stored usin a medium from which they can be accessed quit k ly. Any delay for tube cooling, image reconstruction or storage will tend to reduce the resultant image quality because there is more time and hence more chance for patient movement. Last, but not least, the table must have a high index accuracy, to ensure consistency in the slice thickness in the images produced. The reconstruction and display/archive modules must also exhibit important features. These include the ability to reconstruct the slice images quickly. The array processor used for performing the correction and calibration steps may also be used for doing the three-dimensional reconstructions as well. The computer power necessary for producing threedimensional images is of the same order as that required for the tomogram reconstructions. It may also be useful if the same array processor, used in the basic scanner, is available in an inde endent console, or workstation for three-dimensiona P viewing. This is likely to improve corn atability in user interface, as uplication in hardware and, well as minimizing dp
particularly, software developments. New software releases on the basic scanner offering new threedimensional functionality are most likely to be available on the independent system, at the same time. The selection of the parameters required to select the best three-dimensional result must be user friendly and given in the language of the user. Thus selection must not only be clear, but should avoid the use of computer or mathematical terms.
AN ALGORITHM IMAGING USING
FOR THREE-DIMENSIONAL A Cl” SCANNER
A flow diagram describing the steps used to create a three-dimensional image is shown in Figure 2. The input is a series of slice images. Because the 2 (slice thickness) resolution is typically half that of the X, Y resolution, it is first necessary to do an interpolation to obtain cubic voxels. The interpolated images can each be derived from two adjacent images by linear interpolation. In this implementation a twodimensional filtering (3 x 3) is first used in the X and Y plane of the adjacent images, followed by a low pass function inte olation in the Z direction. The two-dimensional fi‘ptering method gives lower noise and hence improved picture quality of the final threedimensional image (Figure 3). After the interpolation step, it may be convenient to select a volume of interest. This is done if cut plane images are later required or to reduce the volume of
If
fP Y
Slices in
1
>
X
t
1,
Equal X, Y, Z volume of interest
Interpolation 1
Pre Proc
I Crey
to binary
Segmentation I
I
I
3D-2D
projection
1lisplay
(1
41mageout
I Figure
J
-I
I 2
An algorithm
for three-dimensional
imaging in CT
J. Eiomed. Eng. 1990, Vol 12, May
235
Three-dimensionalimaging:A.B. Stronget
al. 2D filter .
64 x 64 = 4096 directions Cube
of greyscale
Figure 5 3D
1D
Figure 3 voxels
One or three-dimensional
interpolation
to obtain cubic
Cube elements checked for segmentation criteria Volume of interest equal X, Y, Z If positive vertex of elements on surface
Figure 4 Segmentation or selection of which voxels are to form the three-dimensional image
data that has to be processed and thus speed up the three-dimensional reconstruction. The most critical step of segmentation is shown in Figure4. The purpose is to find the surface which is to be visualized. The process results in each greyscale voxel in the volume of interest, that is part of the final image, being converted to a binary value. Most methods utilize a thresholding to select those voxels, that would form part of the three-dimensional image. a 2~2x2 cube of voxel In this implementation, sized elements is moved through the volume of interest and at each step, the elements of the cube are checked against a selection criterion. In the current implementation, the criterion is also one of thresholding from the Hounsfield levels in the source slices. It is in this area of choice of the criterion that most research is being carried out. If the selection criterion is met, it is then known that the vertex in the centre of the cube is on the required surface. After segmentation, the step of surface formation is carried out. The grey value of each oint on the surface is dependent on the position of J: e surface, its orientation and the position of a virtual light source. Orientation of the surface at each selected point is determined and subsequently used as one of the factors selecting the grey value in the image. This type of grey value determination is usually referred to as surface orientation dependent shading as opposed to depth shading, where the distance from the observer is the key criteria in determining the grey value. To calculate the orientation, a 2 x 2 x 2 cube, the same size as that for segmentation, is used, as indicated in Figure 5 The cube elements are from the greyscale
236
J. Biomed. Eng. 1990, Vol 12, May
elements
Surface orientation calculation
levels in the original slices. The important parameter required is the direction and not the magnitude. There are a large number of possible directions, but to limit calculation time, 64 possible directions in both rotation and elevation circles are calculated. In the special case where cut planes are displayed, the grey value is also calculated from a cube of elements from the greyscale in the original slices. This is done as shown in Figure 6, by averaging the elements of the cube. Again the use of averaging improves the noise characteristics. All the calculations and manipulations up to this point are referred to as pre-processing. The resultant surface description file, containing details of the position and orientation of the surface, is stored for the following projection on the screen to form a threedimensional image. The reconstruction time for a three-dimensional image is often dominated by the pre-processing. When considering the specification of a three-dimensional imaging system, it is important to distinguish between the time for pre-processing, as opposed to that for projection. The final step of projection now follows. First, the view point direction and distance are selected as shown in Figure 7. Next, the two-dimensional projec-
Cube of greyscale elements are averaged
Figure 6
Cut plane grey value determination
Viewpoint
perspective
Figure 7
Projection view point definition
Three-dimensionalimaging: A.B. Strong et al. Object
space
Projection
V
rotated object space, onto the two-dimensional projection plane. After determining the position of the displayed ixel as indicated above, a grey value can be assigned g ased on the orientation and the position of virtual light source. In showing the three-dimensional image it is essential to remove hidden surfaces. This is done by checking which points on the surface in the volume have the greater Wvalues (see Figure 8), and hence are further from the viewing point.
plane
D
P
D”
e--_--
Figure 8
c
Two-dimensional projection plane definition
CLIMCAL RESULTS
tion plane is defined, as shown in Figure 8. It is defined as the distance between the viewing point and the two-dimensional projection plane. This distance is im ortant as it defines the perspective that will be uti Pized. The projection algorithm then transforms the object space coordinates $ K 2 of the surface vertex ositions, to the rotated space coordinates U, K W may be a simple Psee Figure 8). This projection parallel projection or, for greater effect, is scaled to show true perspective in the final displayed image. Next the vertex positions are projected, from the Craniosynostosis
Premature
fusion
Metopic lines /
Coronal
Sagittal Fontenel
@ Figure 9
Figure
10
Lambdoid
Plagiocephaly
The example used is a case of plagiocephaly or skewing of the skull, as indicated in Fijpre 9. In this case the cause is craniosynostosis or premature fusion, of the right coronal suture. The source slices for this example were obtained on a Philips T 350 scanner and subsequent reconstruction was done on a research configuration, using a VAX computer coupled to an array processor. Reconstruction time, including pre-processing and display, was approximately 15 min. Premature fusion results in distortions in the skull that will become permanent, if not corrected at an early date. The status of the patient at l-year old is shown in the three-dimensional reconstructions of Figure 70. The distortion at this a e is small, but it can be seen that the right eye ten %s to look u wards, while the left eye looks straight ahead. Inside t!l e skull the asymme can be seen clearly in a difference between the ‘r eft and right middle cranial fossa. A post-operative three-dimensional reconstruction is shown in Figure 77 where, thou h the patient still has much growing to do, the skul H has been improved internally and external1 . the The surgical proce crure followed involved
Three-dimensional image of patient at 1 year before the operation. a, External view; b, internal view
J. Biomed. Eng. 1990, Vol 12, May
237
llree-dimensional imaging: A.B. Stronget al.
Figure 11 Three-dimensionalimages of patient at 2 years 5 months after the operation. a, External view; b, internal view
removal of a section of the skull bone and its replacement, leaving the right suture open, to avoid a repetition of the problem. CONCLUSIONS In considering the future of three-dimensional imaging, a few key conclusions can be reached. First, three-dimensional imaging is a multi-modality displa tool, equally a plicable to any slice imaging tee h nique. In those B irect three-dimensional modalities, such as flashing tomosynthesis or Compton the algorithm described is not scatter imagers, appropriate but, nevertheless, a form of threedimensional display can be done. Three-dimensional imaging, as applied to CT, requires a full system approach, because the right data acquisition characteristics are essential to produce optimum image quality, as well as having a display s stem capable of showing the images. ThreeJ imensional ima ‘ng should be an interactive procedure (real time Y, both as regards parameter selection and image reconstruction. The selection of the relevant parameters must be user friendly, using the langua e of the radiologist and surgeon. Thae ey area for ongoing research is the segmentaa thresholding method at too low tion. By considerin a level, structures 8 at are not clinically relevant can be created, while at too high a level, structures will be lost. Artificial intelligence may be required to enable soft tissue and bone to be correctly visualized in the same image. It may also be effective to combine the results from more than one modality. For example,
238 J. Biomed. Eng. 1990, Vol 12, May
the soft tissue from MRI and the bone from CT. When fully developed, it may be anticipated that three-dimensional images will become the normal viewing mode, with multi-planar9 and slice imaging as subsets. REFERENCES 1. Herman GT, Liu HK. Display of three dimensional information in computed tomography. J Cornput Assist Tomogr1977; 1: 155. 2. Lobregt S, Kliene Schaars HWG. Three-dimensional imaging and manipulation of CT data. Part I: General principles Medica Mundi 1987; 32. 3. Zonneveld FW, van der Meulen JCH, van Akkerveeken PF Koomeef L, Vaandrager JM, van der Horst CM. Threedimensional imaging and manipulation of CT data. Part II: Clinical applications. Medica Mundi 1987; 32: 3. 4. Jackson IT, ed. Progress symposium. Progress in craniofacial surgery. WorldJ&g 1989; 13: 327. 5. Jones BM. Advances in the treatment of facial deformity. Br Med J (Clin Res) 1986; 29: 389. 6. Zonneveld FW, Lobegt S, van der Meulen JCH, Vaandrager JM. 3D imaging in craniofacial surgery. World J Surgery1989; 13: 328-420. 7. Vannier MW, Marsh JL. 3D imaging aids skull surgeons. Adding depth to CAT scans clarifies diagnoses. Cornput Graph World 1985; 7: 49. 8. Hemmy DC, David DJ, Herman GT. Three-dimensional reconstruction of craniofacial deformity using computed tomography. Neurosurgery1983; 13: 534. 9. Gillespie JE, Quayle AA, Barker G, Isherwood I. Threedimensional CT reformations in the assessment of congenital and traumatic crania-facial deformities. Br J Oral Maxillofac Surg 1987; 25: 17 1.
and F. W. Zonneveld*
Medical Imaging Technology, Netherlands
Aylesbury,
Bucks, UK; *Philips
display
Medical Systems,
Best, The
ABSTRACT This paper outlines some of the current uses of three-dimensional technique-s in medical imaging applications and their potentialfir thefuture. As an exam@, three-dimensional imaging is described using a CTscanner, as it applies to a case involving craniofacil surgery. K%is includes defining the relationship between the requirements on tire data acquisition system, as well as the spec$cation of the hardware andsojware for the display. A current4 used a&orithm is &scribedfor the display of surjaces as a fin&ion of localposition, orientation of the sutface and theposition of a virtual light source. lItis includes the use of transparency and cut plane greyscale techniques, in addition to the display of the surfaces. A speculation is ma& regarding the use of a three-dimensional display as the standard viewing mode in CT, with slice and multiplanar imaging as submodes. Keywords:
Three-dimensional imaging, CT displays, image processing
INTRODUCTION
Table 1
Medical three-dimensional imaging’” differs from that applied to computer-aided design (CAD) graphics in that the object points are given. The key is the appropriate selection of the points, from a volume of interest (VOI) and their subsequent display, as grey values. It should not be underestimated how well a skilled radiologist can ‘reconstruct’ in his/her mind a threedimensional image from conventional two-dimensional lane films. There are, though, occasions when th ree B imensions can be of great assistance in he1 ing to visualize a lesion. This is particularly so when tI: ere has been severe distortion, caused by injury or the presence of an alien, space-occupying structure. It is to overcome the disorientating effect of distortions that anatomy is often taught in layers, rather than absolute positions. Even in the case of a misplacement, the various structures tend to remain in the same order. In medical imaging, three dimensions can be applied to a number of imaging modalities such as CT and MRI (see Table 7). The list in Table 7 is divided into those imaging modalities which inherently produce three-dimensional images and those modalities where three-dimensional images are derived from a series of reconstructed slices. This article describes an example of the latter.
Three-dimensional imaging from slices
Correspondence and reprint requests to: Mr Antony B. Strong, Medical Imaging Technology, 2 Castle Court, Aylesbury, Bucks HP20 2RD, UK
Imaging modalities applicable to three-dimensions
Computer tomography (CT) Magnetic resonance imaging (MRI) Positron emission tomography (PET) Single photon emission computed tomography (SECT) Ultrasonics (US) Direct three-dimensional imaging (without reconstruction) Flashing tomosynthesis Compton scatter imaging
APPLICATION OF THREE-DIMENSIONAL IMAGINGINCTAND~SFU’IVREPOTENTIAL The applications of three-dimensional images to CT have increased over the last few years. Examples of clinical applications include those shown in Table 2. The use of three-dimensional images in surgical planning has shown a particularly strong growth as a tool to minimize the time for interventional procedures and to speed up time to full recovery. In this article an example of three-dimensional imaging using a CT scanner as it applies to a craniofacial surgery case is described-. It is interesting to see that the use of three-dimensional imaging has changed the Table 2
Applications of three-dimensional imaging
Surgical planning Prosthesis design Stereotaxis in the head Radiation treatment planning
0 1990 Butterworths for BES 0141.5425/90/030233-06 J. Biomed. Eng. 1990, Vol 12, May
233
Three-dimtnsionalimaging:A.B. Stronget al.
roles of those involved in the hospital. It means, for example, that the surgeon is interacting with the display system himself directly, as well as receiving advice from the radiologist. The radiologist is, on the other hand, also having his/her role changed, because the use of injected contrast media and active catheters, becomes more widespread. Prosthesis, or the making of implants for patients, has also become a key application of threedimensional techniques. Again, much time can be saved by designing bone replacements or artificial hip ‘oints to fit first time, rather than having to be mo d ified, at the time of insertion. It is, in fact, not uncommon to feed the output of the three-
dimensional reconstruction directly to a numerically controlled milling machine, to make the correct shaped part. Stereotaxis has come into its own for doing tissue biopsies or inserting radioactive beads for treating tumours. The position and best direction of insertion, can be better seen with the use of a three-dimensional presentation. Three-dimensional imaging can also play a part in ensuring maximum exposure to a tumour during radiation treatment and minimizing the dose to the surrounding tissues. This can be done by overlaying a three-dimensional image of the tumour, with a tlireedimensional reconstruction of the dose profiles. The
Figure 1 a, Surface display of a skull; b, cut plane showing brain tissue; c, skull with internal cut plane; d, partial transparency showing skull and relative position of ventricles
234
J. Biomed. Eng. 1990, Vol 12, May
Three-dimensional imaging:A.B. Strong et al.
spatial matching can be seen more clearly using this viewing technique. In all of the above applications and many others, three-dimensional imaging as it applies to the display s stem usually refers to the use of a pseudo threedy imensional image where a two-dimensional projection of a three-dimensional object is displayed. The impression of three dimensions is given by the use of shading, perspective and rotatin the pro’ected object on the screen. This type of disp Hay may $ e enhanced with effect by the use of a stereoscopic technique. For this method, the alternate half frames of a video image can be used to dis lay the two images, with Polaroid screens to m 9 e the three-dimensional image v&able to the observer. True three-dimensional images may also be displayed by using holograms or the vibrating mirrors technique, but these methods are rarely on commercial offer. In addition to the dis lay of surfaces, it may be possible to show cut p Panes, where the greyscale values in the source slices are used to form the images in the planes. It may also be useful to combine soft tissues and bone in the same image, or to show degrees of transparency, so the relative osition of structures may be appreciated more rea B ily. Examples of some of the above three-dimensional images, as obtained with CT, are shown in Figure 1. FtEQ-S PRODUCING
IN A CT SYSTEM THREE-DIMENSIONAL
FOR IMAGES
The requirements for the system architecture in a CT scanner, to be used to generate three-dimensional images, involves the whole system and is not confined to the display system alone. All of the building blocks must be matched together to ensure optimum erformance. The system must be capable of taking t! in, well collimated slices, to ensure adequate resolution in the 2 direction and to reduce dose to the patient. It is also necessary to have a fast serial scan mode, to produce the 100 or so images required, in an acceptable lapsed time. These must also be stored usin a medium from which they can be accessed quit k ly. Any delay for tube cooling, image reconstruction or storage will tend to reduce the resultant image quality because there is more time and hence more chance for patient movement. Last, but not least, the table must have a high index accuracy, to ensure consistency in the slice thickness in the images produced. The reconstruction and display/archive modules must also exhibit important features. These include the ability to reconstruct the slice images quickly. The array processor used for performing the correction and calibration steps may also be used for doing the three-dimensional reconstructions as well. The computer power necessary for producing threedimensional images is of the same order as that required for the tomogram reconstructions. It may also be useful if the same array processor, used in the basic scanner, is available in an inde endent console, or workstation for three-dimensiona P viewing. This is likely to improve corn atability in user interface, as uplication in hardware and, well as minimizing dp
particularly, software developments. New software releases on the basic scanner offering new threedimensional functionality are most likely to be available on the independent system, at the same time. The selection of the parameters required to select the best three-dimensional result must be user friendly and given in the language of the user. Thus selection must not only be clear, but should avoid the use of computer or mathematical terms.
AN ALGORITHM IMAGING USING
FOR THREE-DIMENSIONAL A Cl” SCANNER
A flow diagram describing the steps used to create a three-dimensional image is shown in Figure 2. The input is a series of slice images. Because the 2 (slice thickness) resolution is typically half that of the X, Y resolution, it is first necessary to do an interpolation to obtain cubic voxels. The interpolated images can each be derived from two adjacent images by linear interpolation. In this implementation a twodimensional filtering (3 x 3) is first used in the X and Y plane of the adjacent images, followed by a low pass function inte olation in the Z direction. The two-dimensional fi‘ptering method gives lower noise and hence improved picture quality of the final threedimensional image (Figure 3). After the interpolation step, it may be convenient to select a volume of interest. This is done if cut plane images are later required or to reduce the volume of
If
fP Y
Slices in
1
>
X
t
1,
Equal X, Y, Z volume of interest
Interpolation 1
Pre Proc
I Crey
to binary
Segmentation I
I
I
3D-2D
projection
1lisplay
(1
41mageout
I Figure
J
-I
I 2
An algorithm
for three-dimensional
imaging in CT
J. Eiomed. Eng. 1990, Vol 12, May
235
Three-dimensionalimaging:A.B. Stronget
al. 2D filter .
64 x 64 = 4096 directions Cube
of greyscale
Figure 5 3D
1D
Figure 3 voxels
One or three-dimensional
interpolation
to obtain cubic
Cube elements checked for segmentation criteria Volume of interest equal X, Y, Z If positive vertex of elements on surface
Figure 4 Segmentation or selection of which voxels are to form the three-dimensional image
data that has to be processed and thus speed up the three-dimensional reconstruction. The most critical step of segmentation is shown in Figure4. The purpose is to find the surface which is to be visualized. The process results in each greyscale voxel in the volume of interest, that is part of the final image, being converted to a binary value. Most methods utilize a thresholding to select those voxels, that would form part of the three-dimensional image. a 2~2x2 cube of voxel In this implementation, sized elements is moved through the volume of interest and at each step, the elements of the cube are checked against a selection criterion. In the current implementation, the criterion is also one of thresholding from the Hounsfield levels in the source slices. It is in this area of choice of the criterion that most research is being carried out. If the selection criterion is met, it is then known that the vertex in the centre of the cube is on the required surface. After segmentation, the step of surface formation is carried out. The grey value of each oint on the surface is dependent on the position of J: e surface, its orientation and the position of a virtual light source. Orientation of the surface at each selected point is determined and subsequently used as one of the factors selecting the grey value in the image. This type of grey value determination is usually referred to as surface orientation dependent shading as opposed to depth shading, where the distance from the observer is the key criteria in determining the grey value. To calculate the orientation, a 2 x 2 x 2 cube, the same size as that for segmentation, is used, as indicated in Figure 5 The cube elements are from the greyscale
236
J. Biomed. Eng. 1990, Vol 12, May
elements
Surface orientation calculation
levels in the original slices. The important parameter required is the direction and not the magnitude. There are a large number of possible directions, but to limit calculation time, 64 possible directions in both rotation and elevation circles are calculated. In the special case where cut planes are displayed, the grey value is also calculated from a cube of elements from the greyscale in the original slices. This is done as shown in Figure 6, by averaging the elements of the cube. Again the use of averaging improves the noise characteristics. All the calculations and manipulations up to this point are referred to as pre-processing. The resultant surface description file, containing details of the position and orientation of the surface, is stored for the following projection on the screen to form a threedimensional image. The reconstruction time for a three-dimensional image is often dominated by the pre-processing. When considering the specification of a three-dimensional imaging system, it is important to distinguish between the time for pre-processing, as opposed to that for projection. The final step of projection now follows. First, the view point direction and distance are selected as shown in Figure 7. Next, the two-dimensional projec-
Cube of greyscale elements are averaged
Figure 6
Cut plane grey value determination
Viewpoint
perspective
Figure 7
Projection view point definition
Three-dimensionalimaging: A.B. Strong et al. Object
space
Projection
V
rotated object space, onto the two-dimensional projection plane. After determining the position of the displayed ixel as indicated above, a grey value can be assigned g ased on the orientation and the position of virtual light source. In showing the three-dimensional image it is essential to remove hidden surfaces. This is done by checking which points on the surface in the volume have the greater Wvalues (see Figure 8), and hence are further from the viewing point.
plane
D
P
D”
e--_--
Figure 8
c
Two-dimensional projection plane definition
CLIMCAL RESULTS
tion plane is defined, as shown in Figure 8. It is defined as the distance between the viewing point and the two-dimensional projection plane. This distance is im ortant as it defines the perspective that will be uti Pized. The projection algorithm then transforms the object space coordinates $ K 2 of the surface vertex ositions, to the rotated space coordinates U, K W may be a simple Psee Figure 8). This projection parallel projection or, for greater effect, is scaled to show true perspective in the final displayed image. Next the vertex positions are projected, from the Craniosynostosis
Premature
fusion
Metopic lines /
Coronal
Sagittal Fontenel
@ Figure 9
Figure
10
Lambdoid
Plagiocephaly
The example used is a case of plagiocephaly or skewing of the skull, as indicated in Fijpre 9. In this case the cause is craniosynostosis or premature fusion, of the right coronal suture. The source slices for this example were obtained on a Philips T 350 scanner and subsequent reconstruction was done on a research configuration, using a VAX computer coupled to an array processor. Reconstruction time, including pre-processing and display, was approximately 15 min. Premature fusion results in distortions in the skull that will become permanent, if not corrected at an early date. The status of the patient at l-year old is shown in the three-dimensional reconstructions of Figure 70. The distortion at this a e is small, but it can be seen that the right eye ten %s to look u wards, while the left eye looks straight ahead. Inside t!l e skull the asymme can be seen clearly in a difference between the ‘r eft and right middle cranial fossa. A post-operative three-dimensional reconstruction is shown in Figure 77 where, thou h the patient still has much growing to do, the skul H has been improved internally and external1 . the The surgical proce crure followed involved
Three-dimensional image of patient at 1 year before the operation. a, External view; b, internal view
J. Biomed. Eng. 1990, Vol 12, May
237
llree-dimensional imaging: A.B. Stronget al.
Figure 11 Three-dimensionalimages of patient at 2 years 5 months after the operation. a, External view; b, internal view
removal of a section of the skull bone and its replacement, leaving the right suture open, to avoid a repetition of the problem. CONCLUSIONS In considering the future of three-dimensional imaging, a few key conclusions can be reached. First, three-dimensional imaging is a multi-modality displa tool, equally a plicable to any slice imaging tee h nique. In those B irect three-dimensional modalities, such as flashing tomosynthesis or Compton the algorithm described is not scatter imagers, appropriate but, nevertheless, a form of threedimensional display can be done. Three-dimensional imaging, as applied to CT, requires a full system approach, because the right data acquisition characteristics are essential to produce optimum image quality, as well as having a display s stem capable of showing the images. ThreeJ imensional ima ‘ng should be an interactive procedure (real time Y, both as regards parameter selection and image reconstruction. The selection of the relevant parameters must be user friendly, using the langua e of the radiologist and surgeon. Thae ey area for ongoing research is the segmentaa thresholding method at too low tion. By considerin a level, structures 8 at are not clinically relevant can be created, while at too high a level, structures will be lost. Artificial intelligence may be required to enable soft tissue and bone to be correctly visualized in the same image. It may also be effective to combine the results from more than one modality. For example,
238 J. Biomed. Eng. 1990, Vol 12, May
the soft tissue from MRI and the bone from CT. When fully developed, it may be anticipated that three-dimensional images will become the normal viewing mode, with multi-planar9 and slice imaging as subsets. REFERENCES 1. Herman GT, Liu HK. Display of three dimensional information in computed tomography. J Cornput Assist Tomogr1977; 1: 155. 2. Lobregt S, Kliene Schaars HWG. Three-dimensional imaging and manipulation of CT data. Part I: General principles Medica Mundi 1987; 32. 3. Zonneveld FW, van der Meulen JCH, van Akkerveeken PF Koomeef L, Vaandrager JM, van der Horst CM. Threedimensional imaging and manipulation of CT data. Part II: Clinical applications. Medica Mundi 1987; 32: 3. 4. Jackson IT, ed. Progress symposium. Progress in craniofacial surgery. WorldJ&g 1989; 13: 327. 5. Jones BM. Advances in the treatment of facial deformity. Br Med J (Clin Res) 1986; 29: 389. 6. Zonneveld FW, Lobegt S, van der Meulen JCH, Vaandrager JM. 3D imaging in craniofacial surgery. World J Surgery1989; 13: 328-420. 7. Vannier MW, Marsh JL. 3D imaging aids skull surgeons. Adding depth to CAT scans clarifies diagnoses. Cornput Graph World 1985; 7: 49. 8. Hemmy DC, David DJ, Herman GT. Three-dimensional reconstruction of craniofacial deformity using computed tomography. Neurosurgery1983; 13: 534. 9. Gillespie JE, Quayle AA, Barker G, Isherwood I. Threedimensional CT reformations in the assessment of congenital and traumatic crania-facial deformities. Br J Oral Maxillofac Surg 1987; 25: 17 1.