A novel 3D volumetric voxel registration technique for volume-view-guided image registration of multiple imaging modalities

Int. J. Radiation Oncology Biol. Phys., Vol. 63, No. 1, pp. 261–273, 2005 Copyright © 2005 Elsevier Inc. Printed in the USA. All rights reserved 0360-3016/05/$–see front matter

doi:10.1016/j.ijrobp.2005.05.008

PHYSICS CONTRIBUTION

A NOVEL 3D VOLUMETRIC VOXEL REGISTRATION TECHNIQUE FOR VOLUME-VIEW-GUIDED IMAGE REGISTRATION OF MULTIPLE IMAGING MODALITIES GUANG LI, PH.D., HUCHEN XIE, PH.D., HOLLY NING, PH.D., JACEK CAPALA, PH.D., BARBARA C. ARORA, M.S., C. NORMAN COLEMAN, M.D., KEVIN CAMPHAUSEN, M.D., AND ROBERT W. MILLER, PH.D. Radiation Oncology Branch, National Cancer Institute, National Institutes of Health, Bethesda, MD Purpose: To provide more clinically useful image registration with improved accuracy and reduced time, a novel technique of three-dimensional (3D) volumetric voxel registration of multimodality images is developed. Methods and Materials: This technique can register up to four concurrent images from multimodalities with volume view guidance. Various visualization effects can be applied, facilitating global and internal voxel registration. Fourteen computed tomography/magnetic resonance (CT/MR) image sets and two computed tomography/positron emission tomography (CT/PET) image sets are used. For comparison, an automatic registration technique using maximization of mutual information (MMI) and a three-orthogonal-planar (3P) registration technique are used. Results: Visually sensitive registration criteria for CT/MR and CT/PET have been established, including the homogeneity of color distribution. Based on the registration results of 14 CT/MR images, the 3D voxel technique is in excellent agreement with the automatic MMI technique and is indicatory of a global positioning error (defined as the means and standard deviations of the error distribution) using the 3P pixel technique: 1.8° ⴞ 1.2° in rotation and 2.0 ⴞ 1.3 (voxel unit) in translation. To the best of our knowledge, this is the first time that such positioning error has been addressed. Conclusion: This novel 3D voxel technique establishes volume-view-guided image registration of up to four modalities. It improves registration accuracy with reduced time, compared with the 3P pixel technique. This article suggests that any interactive and automatic registration should be safeguarded using the 3D voxel technique. © 2005 Elsevier Inc. Multimodality image registration, 3D voxel image registration, Volume-view-guided image registration, Accelerated image registration with improved accuracy, Radiation treatment planning.

Multimodality image registration combines complementary information from different imaging technologies, providing comprehensive information about a patient. It has been increasingly used in radiation diagnosis and treatment planning (1–3). By definition, image registration or fusion is to spatially superimpose three-dimensional (3D) images obtained from different imaging modalities. Co-registered images combine medical information, including the patient’s anatomy and physiologic activity, which assist the physician with diagnosis and treatment. All interactive image registration involves three steps (1–3). First, registration landmarks, including the entire voxel volume, must be identified in all relevant imaging modalities. Both anatomic and fiducial landmarks can be used, but the

former is more favorable as they are intrinsic to the images, abundant for matching, and redundant for cross-verification. Second, a strategy for registering the landmarks must be available, either with an automatic registration algorithm or with interactive controls using graphical user interface (GUI). Although the former is dominant in the literature with different algorithms, various cost functions, and elastic deformation models (4 –18), interactive registration is still predominantly used in clinical treatment planning (19). Finally, the result should be verified with visual representation as a necessary step for quality assurance (9, 10). Previously, interactive registration has been reported judgmental and time-consuming (19). The former is a characteristic of any manual method where the registration accuracy is dependent on the quality of the visual representation, whereas

Reprint requests to: Guang Li, Ph.D., Radiation Oncology Branch, National Cancer Institute, NIH, Bldg. 10, CRC, Rm. B2-3561, MSC-1682, 9000 Rockville Pike, Bethesda, MD 20892. Tel: (301) 451-8955; Fax: (301) 480-1064; E-mail: ligeorge@ mail.nih.gov

Acknowledgements—The authors thank Jan Hardenbergh and Frank Baker (Terarecon, Inc.) for their support and in-depth discussion on the volume rendering board and its supporting toolkit. Received Dec 16, 2004, and in revised form May 6, 2005. Accepted for publication May 8, 2005.

INTRODUCTION

261

262

I. J. Radiation Oncology

● Biology ● Physics

the latter is mostly caused by the cumbersome 3D operations inherent in any conventional registration program. These conventional registration techniques visualize and align the 3D volume based on three orthogonal planar (3P) images, and here they are referred to as the 3P pixel technique. Therefore, they require the user to have sophisticated spatial imagination to coordinate the overall position change based on the three orthogonal planar views. In other words, accuracy and speed depend upon the user’s ability to form a mental model of the reconstructed 3D image volume and to determine how moving the image in one planar view will affect its alignment in the other two. For the same reason, identification of registration landmarks and verification of the result are also slow. Even for experienced users, the correlation across all slices in each of the orthogonal directions must be memorized to detect any global positioning error and to identify any distortional error. For automatic registration, the method of maximization of mutual information (MMI) (7, 8) is a theoretically sound, well tested, and increasingly accepted image registration technique in clinic. However, it requires that the anatomy underlying the multimodal images be unchanged both geometrically and materially. The geometrical elasticity and flexibility of human anatomy have been studied in registration using various deformation models (3, 15–18) to meet clinical needs, while the material change, which can be arbitrary and difficult to consider mathematically, occurs clinically as well. For instance, under some protocols, the magnetic resonance (MR) images collected are mostly presurgical whereas the computed tomographic (CT) images are postsurgical, preirradiation treatment planning images. Therefore, the differing anatomy, as a result of surgery, may complicate the determination of automatic registration and produce inferior results. To overcome these shortcomings, a novel 3D volumetric voxel registration technique has been developed that provides volume-view-guided image registration with straightforward means for manipulating multiple rigid image volumes. This 3D global-view formalism significantly reduces the demand for cognitive ability and therefore can reduce the registration time. More importantly, owing to the superior visual representation of the 3D voxel volumes, small positioning errors become readily identifiable. Unlike most registration software that can only handle two image sequences from multiple modalities simultaneously (1–3), this technique can register up to four image sets. Although this feature may be ahead of the current clinical need, the trend is to incorporate an increasing number of imaging modalities to meet potential challenges in clinical diagnosis and treatment. Registration results based on statistical analysis using CT images and MR images are presented. Registration results of CT and positron emission tomography (PET) images are also demonstrated. In addition, with the 3D volumetric voxel registration, the alignment is determined based on the voxel intensity (gray level) and can be verified volumetrically. Therefore, its registration results should be comparable to the automatic MMI registration (7, 8), if the change in the underlying

Volume 63, Number 1, 2005

anatomy can be neglected. For dramatic anatomic changes, the 3D voxel registration technique can be a good alternative to the automatic registration technique.

METHODS AND MATERIALS The 3D voxel registration software is performed on a personal computer (Pentium IV-1.7 GHz XEON, 1 GB RAM) with a volume rendering board (VolumePro1000, TeraRecon, Inc., San Matteo, CA). The program is written in C⫹⫹ and Java, linked by Java Native Interface, under Windows 2000 operating system. Many advanced visual representation features are provided with the board. However, the workhorse for the 3D voxel registration is built using the volume rendering board, as well as the resources of the host computer. For each voxel volume, a look-up table (LUT) that overlays the image histogram is used to control voxel color and transparency, including three color channels: red (R), green (G), and blue (B), as well as a transparency channel: alpha (A). It provides a comprehensive way to alter window (W) range and level (L) function.

Volumetric voxel rendering and representation Ray casting is a well-established rendering algorithm and integrated in the volume rendering board. Because of the on-board hardware support, the image display and basic volume operation are done in real-time. Figure 1 shows a schematic drawing of how voxel registration operation is incorporated in the volume rendering pipeline. The ray casting algorithm manages the rendering from the user’s eye view (UEV), along a ray within a bundle of parallel rays that cover the whole image area. RGBA (R, G, B, and A) contribution from any voxel along the ray is weighted and accumulated usually in front-to-back order. In parallel, the voxel gradient estimation is done to improve image quality. The reconstructed point along the ray will also be affected by the lighting effect and color blending. The final image therefore contains both the voxel information and the environmental information. In this study, however, illumination lights are all turned off during registration, eliminating the environmental factors. Most of the key parameters in the rendering pipeline can be changed by users at any time via the GUI panel and will be reflected in the next rendering. Because of the real-time performance, such changes are done smoothly without visual disruption or interference.

Input and output of the 3D volumetric voxel registration Each volumetric voxel contains 32 bits, which can be divided into four volume fields with 8 bits each reflecting their voxel intensity (gray level). Thus, for each individual imaging modality, the original 16-bit gray level must be compressed to 8 bits based on optimal visibility. Up to four image sequences can be registered simultaneously in real-time with straightforward GUI operations (20). The multiple image sets can be processed so as to produce a uniform volume size and isotropic voxel size using linear interpolation. The registration result can be recorded via six parameters, reflecting the six degrees of freedom in rotation and translation in the 3D volume coordinate, in which the center of rotation is fixed at the center of the volume. Four sets of the six parameters are then used to communicate with a treatment planning system, as shown in Fig. 2.

Volumetric voxel registration for multiple imaging modalities

● G. LI et al.

263

for fine changes has been identified and they should be used for future image registration in clinic.

Mono-color visual representation and registration criterion To distinguish each individual volume in the registration process, mono-color visual representation is used, where each image modality is set to a single color, such as R, G, or B. The color distribution on the voxel volume or a subvolume can be used as registration criterion: the homogeneity of the color distribution would be an indicator for an optimal match, provided that the voxel content of interest exists in the multiple modalities involved. This can be illustrated with an extreme case where a volume is registered with itself and a successful registration will result in a homogeneous color distribution, causing color change effect, which will be discussed in later sections. The supportive results produced by the automatic MMI registration will also be presented in the later sections.

Pseudocolor visual representation and cross-verification

Fig. 1. Image volume rendering pipeline of ray-casting algorithm. The registration operation is done in the early stage of the rendering pipeline with manipulation of the voxel fields.

Voxel transformation operation for volumetric registration Although the volume rendering board has facilitated real-time volume rendering, it does not support rendering two or more independent voxel volumes simultaneously through the same rendering process. To realize registration operations, the host computer resources are used to buffer and manipulate the voxel volume in the early stage of the rendering pipeline, as shown in Fig. 1. A simplified 3D affine transformation is applied to an image volume at a time, and the transformed volume voxels are inversely reassigned to the original voxel coordinate with linear interpolation and rendered with others. An arbitrary voxel, Pi,j,k, belonging to a set or a volume, can be expressed as V ⫽ { Pi,j,k }, where i, j, and k are the indices in the volume array in x, y, and z directions, respectively. The coordinate transformation can be expressed using a 4 ⫻ 4 matrix T. If there is P=i,j,k ⫽ Pi,j,k ⫻ T, then,

The pseudocolor representation uses a predefined LUT overlaid on the image histogram to visualize the voxel intensity in colorcoded spectra (RGBA) for various purposes, including enhancing visual differentiation, mimicking real tissue color, or displaying interior voxel. An alignment based on an interior landmark (subvolume) can be used for registration or verification. Because an imaging modality for certain anatomic segments usually provides similar histograms, a library of LUTs can be built based on the “typical” histograms before use. Such a library can be loaded as desired, leaving the time-consuming LUT setup and adjustment as a preperformed operation. This feature facilitates the image view selectivity, and is therefore useful to speed up the image registration.

“X-ray vision” visual representation for interior voxel Taking the advantage of the ray-casting algorithm in the volume rendering process, the voxel with nontransparent content first reached by the ray from UEV can be identified, accessed, and manipulated. Based on user-defined regions of the volume on screen, the voxel of interest can be located and manipulated, such

V
⫽ 兵P
i,j,k其 ⫽ 兵Pi,j,k ⫻ T其 ⫽ V ⫻ T is true for any 1:1 transformation between the two sets of voxels, namely V and V=, according to the theory of set. Integer precision is used for minimum transformation steps in both rotation and translation in the current version of the program. This is adequate for the purpose of comparison, although the need

Fig. 2. The volume display window (left) and graphical user interface control panel (right) of the three-dimensional volumetric voxel registration software. A set of rulers and a cursor can be overlaid with the voxel volume as visual measures, and all the control functions are shown in a set of tab panels.

264

I. J. Radiation Oncology

● Biology ● Physics

Volume 63, Number 1, 2005

assessment and target allocation, whereas the CT images are for treatment planning before radiation therapy. Therefore, partial anatomic changes are present in these images. Although multiple MR scans are collected with different relaxation times and pulse sequences, the one that provides the best target view is used in the registration and in defining gross tumor volume. The two CT/PET images are collected from separate scanners using 18F-fluorodeoxyglucose (18F-FDG) as the PET imaging agent.

RESULTS Fig. 3. Demonstration of the three-dimensional voxel registration process with rotational (A) and translational (B) shifts in three imaging modalities, computed tomography (red), T2-based magnetic resonance (blue), and T1-based magnetic resonance (green), in mono-color representation.

as changing voxel transparency. Therefore, the first nontransparent voxel layer along the UEV can be “peeled off” layer by layer, so that the interior voxel can be viewed without interference from the exterior. We call this “onion-peeling” representation X-ray vision.

Conventional three orthogonal planar pixel registration The 3P pixel technique used in this work has been developed and used in clinic for many years. It provides a detailed comparison slice-by-slice in the three orthogonal directions and is useful for checking details in the pixel alignment on the edges of segmented display windows, showing the two registered images alternately. As all of the registration programs in this category, it uses three orthogonally sliced pixel views to represent the entire voxel volume. Therefore, the tissue continuity or integrity is not completely visualized and presented to the user. As a consequence, the registration that suffers from poor reproducibility and long registration time has been reported (19). In addition, only two image modalities can be registered at a time.

Automatic image registration using maximization of mutual information The automatic MMI registration program is developed based on the Insight Toolkit (ITK, version 2.0.0) (21). A mutual information metric (22) and an evolutionary optimizer (23) are used. The fundamental assumption for mutual information is that the underlying anatomy in two different imaging modalities is the same. For rigid registration, any deformational changes or material changes in patient’s anatomy in the two image sets can reduce the reliability of the registration, depending upon the severity of the changes. The key tunable parameters used in this automatic MMI registration are described as follow: 10,000 voxels are randomly sampled to compute the mutual information metric value in each iteration, and totally 10,000 iterations are used to achieve a good registration, using the evolutionary optimizer. The “growing” and “shrinking” factors, which are used to control the forward and backward step sizes in the optimizer, are set to be 1.05 and (1.05)⫺1/4, respectively (21, 23).

Multimodal images of CT/MR and CT/PET A set of 14 CT/MR brain images are used in this study. The MR images are diagnostic images used for presurgical tumor volume

3D volumetric voxel registration of up to four image sets The key to registration is the ability to shift the individual voxel volumes while they are still rendered with the other image volumes in the rendering pipeline. Therefore, the spatial relationship (correct depth information in UEV) among the volumes can be properly visualized throughout the registration process. This basic functionality is demonstrated using three voxel volumes in mono-color as shown in Fig. 3. Users are allowed to adjust one imaging modality at a time, to select a particular modality for active operation, and to turn on or off any modality as desired, as illustrated in Fig. 4. The performance of the transformation operation, which is the bottle neck in the rendering pipeline, is dependent upon the size of the voxel volume. In the specific case shown in Figs. 3 and 4, the volume contains 55 slices of 320 ⫻ 320 pixels for all modalities. A single translational operation takes about 100 –200 ms, about four times faster than a rotational operation, because the latter involves a more complex transformation. The performance also depends upon the speed and availability of host computer resources. Using the same data set, the performance is improved by ⬃2–3 folders using a more powerful computer (dual Pentium IV 2.8 GHz XEON with 2 GB RAM and 4 GB VolumePro1000 board) without any change in the source code. Therefore, the operations can be done virtually in real-time.

Fig. 4. Demonstration of combined modalities with pseudocolor representation. (A) Computed tomography bone structure. (B) Dual modality display with some magnetic resonance voxels (muscle tissues and eyeballs) turned on. Illumination light is turned on in this figure for the best 3D views.

Volumetric voxel registration for multiple imaging modalities

● G. LI et al.

265

Table 1. Comparison of registration results from the 3D voxel interactive registration technique and from the automatic registration technique using maximization of mutual information Patients (images)

␦Xr*

␦Yr*

␦Zr*

⌺|␦|/3¶

(⌺␦2)1/2¶

␦Xt†

␦Yt†

␦Zt†

⌺|␦|/3¶

(⌺␦2)1/2¶

1 (CT/MR_T1Flr)‡ 2 (CT/MR_T2) 3 (CT/MR_T1Flr)‡ 4 (CT/MR_T1Gd) 5 (CT/MR_T1Gd)§# 6 (CT/MR_T13D)储 7 (CT/MR_T1Flr)‡ 8 (CT/MR_T1Gd)§ 9 (CT/MR_T2) 10 (CT/MR_T1Flr)‡ 11 (CT/MR_T13D)储 12 (CT/MR_T1Flr)‡# 13 (CT/MR_T1Gd)§ 14 (CT/MR_T1Gd)§ Ave (⌺|␦|/N) Std Dev (␴)

0.00 1.00 0.00 2.00 off ⫺1.00 0.00 0.00 0.00 0.00 0.00 off 0.00 0.00 0.33 0.67

0.00 0.00 0.00 0.00 off 0.00 0.00 0.00 0.00 0.00 0.00 off 0.00 0.00 0.00 0.00

⫺1.00 0.00 0.00 0.00 off 0.00 0.00 0.00 0.00 0.00 0.00 off 0.00 0.00 0.08 0.29

0.33 0.33 0.00 0.67

1.00 1.00 0.00 2.00

0.00 0.33 0.33 0.33 0.00 0.00

0.00 1.00 1.00 1.00 0.00 0.00

0.00 0.00 0.1 0.3

0.00 0.00 0.4 0.7

0.00 0.00 0.00 0.00 off 0.00 0.00 0.00 0.00 0.00 0.00 off 0.00 0.00 0.00 0.00

1.00 0.00 0.00 0.00

1.00 0.00 0.00 0.00 0.00 0.00

⫺1.00 0.00 0.00 0.00 off 0.00 0.00 ⫺1.00 ⫺1.00 0.00 0.00 off 0.00 0.00 0.25 0.45

0.33 0.00 0.00 0.00

0.33 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00 off 0.00 ⫺1.00 0.00 0.00 0.00 0.00 off 0.00 0.00 0.08 0.29

0.00 0.00 0.1 0.3

0.00 0.00 0.3 0.5

The homogeneity of the color distribution on the skin/muscle voxels is used for the 3D voxel registration and for the adjustments on the automatic registration results. * Rotational adjustments/deviations in degree (°). † Translational adjustments/deviations in voxel unit (usually ⬇ or ⬍1 mm). ‡ T1Flr: T1-based FLAIR (fluid-attenuated inversion-recovery sequence) (fast spin echo). § T1Gd: T1 with gadolinium contrast. 储 T13D: 3D spoiled GRASS (gradient recalled acquisition in a steady state) sequence. ¶ Two ways of evaluating the deviations between techniques. The arithmetic mean (⌺|␦|/3) and “distance” ((⌺␦2)1/2) of both rotational and translational deviations are provided for assessment of the registration differences. The precise assessment of the overall voxel deviation is the arithmetic means of all voxel distance deviations, which are no longer uniformly distributed if a rotation is involved. # Where the automatic registrations failed, identified by visual checking of misalignment.

3D voxel registration criterion and validation using mutual information technique Another key to registration is the ability to identify a small misalignment among the voxel volumes. The highest achievable homogeneity of the color distribution on a given anatomic volume is used to provide visual criterion for alignment. To validate such an a priori assumption for registration, a comparison between the 3D voxel interactive registration and the automatic MMI registration is done over a set of 14 CT/MR images. Table 1 shows that the two registration results are in good agreement, as the overall “distance” of deviations for both rotation and translation are about less than a half of a single degree or voxel unit for rotation and translation, respectively. The automatic MMI registration technique is theoretically sound and has been widely tested and increasingly accepted in medical image registration. Therefore, the consistency between the two registration techniques supports the validity of the registration criterion, which is heavily used in the 3D voxel registration technique for most of this work. There are two cases in which the automatic registration failed and significant misalignments are identified visually. Likely, this is caused by a potential local maximum trap as these results are reproducible. Surgery performed during the interval between the CT and MR scan acquisition may play a role.

Registration landmarks, criteria, and improved accuracy The selected anatomic landmarks usually have either a clearly definable surface (1, 5) or valid voxel contents (2– 4) for all the imaging modalities studied, for both interactive and automatic registration. In this study, the skin/muscle voxels (or the entire volume) are frequently used as the

Fig. 5. Improved registration accuracy with higher homogeneity of the color distribution on the head voxel surface of computed tomography image (red) and TI-based magnetic resonance image (green). (A) The registration result obtained from 3P pixel technique: where a positioning error is identified. (B) The registration with the three-dimensional voxel technique shows higher degree of homogeneity in the color distribution on the head volume. From (A) to (B) the voxel shift is made: ␦Xt ⫽ ⫹1; ␦Yt ⫽ ⫺1; and ␦Zt ⫽ ⫹1 in voxel unit (0.94 mm).

266

I. J. Radiation Oncology

● Biology ● Physics

Fig. 6. Improved registration accuracy with correction of a positioning error. The computed tomography image is in red and the magnetic resonance image is in green. (A) The registration result from 3P pixel technique. (B) The refined registration result from the three-dimensional voxel technique with ␦Zr ⫽ ⫹3 (degree) and ␦Zt ⫽ ⫺3 voxel unit (1.09 mm). The center of rotation is fixed at the center of each individual volume.

landmark for CT/MR registration, as they are readily distinguishable from surrounding air voxels and reside as the outmost layers of the voxel volume. The homogeneity of the color distribution, when mono-color representation is applied, is used as an effective and accurate registration indicator, as shown in Table 1 and Figs. 5 and 6. With this unique visual alignment criterion, the registration becomes very sensitive to a slight positioning change of a voxel volume, especially for the global misalignment of the 3D voxel images, as shown in Figs. 5a and 6a. The registration results using the 3P pixel technique are compared with those from the 3D voxel technique, illustrated in Figs. 5 and 6. In both CT and MR modalities, the surface voxels of the patient’s head are used as registration landmark with minimal window/level adjustment. In Fig. 5, to make the color distribution more homogeneous, translational shifts are made with ⫹1, ⫺1, and ⫹1 voxel unit along the x, y, and z direction, respectively. Integer precision is used, which seems adequate for the comparison purpose. The changes are shown in the GUI panel of Fig. 2 and recorded into a common file, for communicating with a treatment planning system. Figure 6 shows the result of a 3P pixel registration containing both rotational and translational misalignment, which can be identified visually and corrected to achieve more homogeneous color distribution on the landmark. All the fine-tuned registration results using the 3D voxel technique are verified with segmented views, as well as other internal voxel checking tools for details. This is the methodology with which any interactive registration should be performed: the global view first, followed by the details. It has been found that the registration is not sensitive to the precise display of the entire landmark volume, as long as the displayed volume is within the desired anatomy, as defined by its contour. This is because the 3D voxel registration is voxel-based, which includes both surface and internal voxels. In practice, it is straightforward to display similar voxel volumes for different modalities by

Volume 63, Number 1, 2005

Fig. 7. Cross-verification of the registration result with eyeball as reference. The voxel in front of patient’s left eyeball is set to be transparent. (A) The computed tomography bone structure with some magnetic resonance soft tissues. (B) The same as (A) but showing more soft tissue voxel (orange) from the computed tomography image. From (A) to (B), the eyeball color turns from green to yellow as the orange voxel from computed tomography is shown, indicating a superimposed position on this object.

changing the window/level in the individual LUT of the corresponding image histogram. Cross-verification of 3D voxel image registration result With the superior visualization features, multiple registration landmarks can be determined. Therefore, additional landmarks can be used for cross-verification as a quality assurance tool. Registration using multiple landmarks should dramatically increase the reliability of the result. As an example, Fig. 7 shows a patient’s orbits used as additional registration landmarks, together with their surrounding anatomy. The two independent image registration results confirm each other. On the other hand, if conflicting results are obtained using different, but valid landmarks, then the registration requires additional validation. This verification can also be done using various geometrical segmentation tools embedded in the 3D voxel technique, such as multiple cut, trim and crop planes, either in the

Fig. 8. Various views of the target (large green area) in the registered image, where computed tomography (gray) skull is shown in the magnetic resonance voxel volume with pseudocolor representation. (A) A planar view with a cursor. (B) A view with the corner segment removed at the tumor location on the patient’s right side.

Volumetric voxel registration for multiple imaging modalities

● G. LI et al.

267

Table 2. Comparison of registration results from the 3D voxel interactive registration technique and from the conventional 3P pixel registration technique Patients (images)

␦Xr*

␦Yr*

␦Zr*

⌺|␦|/3¶

(⌺␦2)1/2¶

␦Xt†

␦Yt†

␦Zt†

⌺|␦|/3¶

(⌺␦2)1/2¶

1 (CT/MR_T1Flr)‡ 2 (CT/MR_T2) 3 (CT/MR_T1Flr)‡ 4 (CT/MR_T1Gd)§ 5 (CT/MR_T1Gd)§ 6 (CT/MR_T13D)储 7 (CT/MR_T1Flr)‡ 8 (CT/MR_T1Gd)§ 9 (CT/MR_T2) 10 (CT/MR_T1Flr)‡ 11 (CT/MR_T13D)储 12 (CT/MR_T1Flr)‡ 13 (CT/MR_T1Gd)§ 14 (CT/MR_T1Gd)§ Ave (⌺|␦|/N) Std Dev (␴)

0 ⫺1 0 0 0 ⫺1 0 ⫺1 2 ⫺1 0 0 ⫺2 0 0.57 0.77

0 1 0 0 2 2 0 0 ⫺1 1 1 0 0 1 0.64 0.74

0 0 ⫺3 ⫺1 0 0 ⫺2 0 0 1 ⫺2 0 4 1 1.00 1.30

0.00 0.67 1.00 0.33 0.67 1.00 0.67 0.33 1.00 1.00 1.00 0.00 2.00 0.67 0.7 0.5

0.00 1.41 3.00 1.00 2.00 2.24 2.00 1.00 2.24 1.73 2.24 0 4.47 1.41 1.8 1.2

1 1 0 0 0 2 0 1 0 1 0 0 1 1 0.57 0.65

⫺1 ⫺1 0 0 ⫺1 0 1 0 4 0 0 0 ⫺3 1 0.86 1.23

1 2 3 1 0 0 0 0 ⫺1 0 4 0 ⫺1 ⫺2 1.07 1.27

1.00 1.33 1.00 0.33 0.33 0.67 0.33 0.33 1.67 0.33 1.33 0.00 1.67 1.33 0.8 0.6

1.73 2.45 3.00 1.00 1.00 2.00 1.00 1.00 4.12 1.00 4.00 0.00 3.32 2.45 2.0 1.3

The homogeneity of the color distribution on the skin/muscle voxels is used for the 3D voxel registration and for the adjustments on the 3P pixel registration results. * Rotational adjustments/deviations in degree (°). † Translational adjustments/deviations in voxel unit (usually ⬇ or ⬍1 mm). ‡ T1Flr: T1 based FLAIR (fluid-attenuated inversion-recovery sequence) (fast spin echo). § T1Gd: T1 with gadolinium contrast. 储 T13D: 3D spoiled GRASS (gradient recalled acquisition in a steady state) sequence. ¶ Two ways of evaluating the deviations between techniques. The arithmetic mean (⌺|␦|/3) and “distance” ((⌺␦2)1/2) of both rotational and translational deviations are provided for assessment of the registration differences. The precise assessment of the overall voxel deviation is the arithmetic means of all voxel distance deviations, which are no longer uniformly distributed if a rotation is involved.

conventional manner or by a 3D voxel means, combining with 3D volumetric view. Using these tools, the clinical target in the co-registered image can also be viewed in various ways, as shown in Fig. 8. Statistical comparison with conventional interactive registration The identical set of 14 CT/MR images has been used for this comparison study. The 3P pixel registrations are performed by two medical physicists, independently, and the 3D voxel registration is performed independently by a third. Virtually all of the registration results from the 3P pixel technique show positioning errors to various degrees, as identified visually by the 3D voxel registration criterion and shown in Table 2. The refined registration results are also checked with internal views, including planar views, for verification. For the data presented in Table 2, the registration refinement was done on the 3P pixel registration results using the 3D voxel technique. The logical next step is to adjust any misalignment after it is identified. This provides a straightforward comparison. It is not necessary, however, to start with a coarse alignment for the 3D voxel registration technique; in fact, one of the advantages of this technique is to achieve a rapid coarse alignment compared with the 3P pixel technique. The MR images in Table 1 and 2 are collected under different relaxation times and pulse sequences. For all T1based MR images, the body surface is well defined, whereas

T2-based MR images have relatively vague surface. In the former group, the fused images have homogeneous color distribution of red and green, whereas in the latter, MR voxels should be either confined within the CT volume or overlapped with slightly reduced CT volume by window/ level adjustment. The positioning errors of the 14 registration results are about 2 degrees and 2 voxel units in terms of “distance” deviations and show no systematic trends. This suggests that the users are “blind” to the existence of such positioning error in the results, due to the poor visual representation provided by the 3P pixel technique. However, when using the 3D voxel technique with the proper criterion, this global positioning error is clear. Theoretically both techniques should have similar precision because they all apply 3D voxel information into the registration; but practically, only the 3D volumetric voxel technique with its superior visual representation can match the accuracy to the precision. This improved accuracy results primarily from eliminating or minimizing the global positioning errors. Reduced registration time in comparison with 3P pixel registration Comparing with the 3P pixel technique, the 3D voxel technique can reduce the registration time. On average, a two-modality image registration done by a skilled user using the 3P pixel interactive technique takes about 15 min

268

I. J. Radiation Oncology

● Biology ● Physics

Volume 63, Number 1, 2005

Fig. 9. A three-dimensional voxel registration result of head computed tomography/positron emission tomography (CT/PET) images. The computed tomography (red) bone voxel is set semitransparent, so that the inner positron emission tomography (green) voxel can be viewed at the same time. The inner skull surface in computed tomography is used to fit the brain surface in positron emission tomography. (A) Side view. (B) Back view.

Fig. 10. A three-dimensional voxel registration result of body computed tomography/positron emission tomography images with distortion (bended legs). The computed tomography (red) muscle voxel is set semitransparent and the overlapped voxel with positron emission tomography (green) becomes yellow color. The registration is focused on the body trunk, where the tumor locates, ignoring the bended legs. (A) Head-to-foot view. (B) Side view.

or more for the 14 CT/MR image registrations. In contrast, the identical registration done using the 3D voxel technique takes about 5 min, including the pre-process (bit compression) of the images. The identification of any positioning error is almost instantaneous when the resulting volumes are displayed, and correction (fine adjustment) usually takes about 2–3 min. Both initial coarse alignment and final fine adjustment are quite straightforward, guided by the 3D global views with distinguishable colors. If an additional modality is included, the 3D voxel technique will save even more time because the pre-process is shared and the three combinations of the bi-modal image registration are performed as a single process. For 4-modality image registration, six combinations of the bi-modal image registration will be condensed in one process. Therefore, this technique provides better and quicker multimodal image registration, in comparison with the 3P pixel technique.

tating the registration. This is another way to monitor the registration process from a global point of view.

Registration of anatomic and functional images—a head case A case using images from CT and PET is shown in Fig. 9. The registration landmark is identified as the inner surface of the skull from CT and the brain surface from PET, as previously reported (6). Both voxel contents are valid, lying in neighborhood anatomic structures. The PET brain voxel is relatively insensitive to window/ level change in the central range of the gray scale, because its intensity covers the higher end of the histogram. This facilitates the registration, as the nonbrain voxels can be visually eliminated, while the brain surface remains relatively unchanged. In CT, all soft tissue voxels are set transparent and the skull voxels are set semitransparent, so that both PET brain voxels and CT bone voxels can be viewed simultaneously and from different view perspectives, as illustrated in Fig. 9. The particular LUT setup can be saved in a library and loaded for use as needed, facili-

Registration of anatomic and functional images—a torso case Because of the 3D global visualization, a distortional error can be readily identified in a case involving the torso with CT and PET images as illustrated in Fig. 10. No satisfactory registration result can be achieved without either correcting for or ignoring the position of the legs. In this instance, the leg positions are ignored and the registration is done on torso, where the treatment target resides. Although this is not mathematically sound, it is clinically valid. The registration for CT/PET images of the torso is more complex than the case involving the head, because welldefined anatomic landmarks are hard to find. Here the total information of PET image is used to provide a reference for the registration without differentiating detailed anatomic structure. During the process, the window/level or the A-

Fig. 11. Critical voxel for computed tomography (red) bone structure and positron emission tomography (green) lung and other internal tissues. The most intense positron emission tomography signal seems confined inside the chest (rib cage) well. (A) Top view. (B) Back view.

Volumetric voxel registration for multiple imaging modalities

spectrum setting is changed, as well as the usual six degrees of freedom. Therefore, the voxel transparency in the histogram is treated as an additional degree of freedom. The most intense PET signal, from lung and tissue inside the chest wall, seems confined by the chest volume rather well, as shown in Fig. 11. Therefore, confining the PET voxel volume within the rib cage provides a good initial fit. Then the fine adjustment of the registration can be performed with increased PET voxel contribution and additional soft tissue voxels from the CT image, shown in Fig. 10. Here, the aim is an overall voxel fitting as a global criterion without a detailed assessment.

● G. LI et al.

269

Owing to the superior visual representation, the three steps in the interactive image registration process: (a) identification of appropriate landmark, (b) registration, and (c) verification, are integrated and iterated as the registration proceeds. Throughout the interactive registration, the user is guided by the 3D volumetric views, aiming to achieve best homogeneity of the color distribution on the landmark volume. The preprocessing of the image for stacking the volume buffer is a generic formatting process, which may vary slightly with modality but is not image-specific. The 3D voxel technique can register up to four image sequences (including multiple imaging modalities) simultaneously. Therefore, it provides multimodal image registration in a single process, providing room for additional modalities to accommodate future clinical needs. Typically, most commercial software can register only two imaging modalities at a time. Registration of three or more image sets using the two-at-a-time approach tends to forwardpropagate registration errors. The 3D voxel technique can minimize or even eliminate such potential errors.

ences in signal intensities, noise levels, and sensitivities to different tissues themselves or their physiologic activities. Therefore, we extended the criterion from the above ideal case to the practical as: the highest achievable homogeneity of color distribution represents the best alignment among voxel volumes when mono-color representation is applied. As a validation to the assumption, comparisons between the automatic registration using mutual information and the 3D volumetric voxel registration are performed, as shown in Table 1. Because of the two failures in the automatic registration, the statistics are done over the other 12 cases. The overall “distances” of the deviations are 0.4° ⫾ 0.7° in rotation and 0.3 ⫾ 0.5 (voxel unit) in translation. Part of the cause for the slight difference between the two sets of registration may be attributed to the integer precision for coordinate transformation in the current version of the 3D voxel registration program. In some cases, a change with fractions of a degree or a voxel unit should better reflect the differences between these registration results. So, a “fine transformation” step will be incorporated in future work. Another cause may result from the change in the underlying anatomy as a result of the surgical removal of the tumor and some degree of weight loss shown in some of the patients. This may result in two mismatched registrations using the MMI technique. Regardless of the causes of deviation, an excellent agreement between these two techniques can be seen, as shown in Table 1, indicating that the registration criteria are robust and valid for CT/MR registration. For CT/PET cases, there are not enough image data for such comparison. The parameters for the automatic algorithm need to be adjusted and tested over a set of cases before they can be used for such comparison, owing to the poor resolution and incomplete anatomic information of PET images. Therefore, the registration criteria are used as a reasonable assumption without such comparison.

The foundation of 3D voxel image registration In addition to the technical aspects of the 3D voxel registration technique, such as capability of the position adjustment for any of the voxel volumes in the image rendering pipeline, the registration criterion is another central foundation of this technique. This criterion is the “sensor” that allows the user to sense/see any misalignment present in a coarse registration, as well as to identify an optimum registration. In an extreme case, where an image is registered to itself, a perfect alignment between the identical two images colored with red and green will result in a yellow voxel volume. This is solely because the completely overlapped voxels with equally weighted red and green will synthesize the color of yellow on the display screen. It is worthwhile to mention that the yellow color exists for all voxels: external and internal. A slight misalignment will demonstrate an inhomogeneous color distribution and partial disappearance of the color change effect. When images from different modalities are used, the registration criterion should be modified to adapt the differ-

Global volume-view-guided image registration In the combined 3D global view of multi-image registration, the user is able to observe and subsequently change the active voxel volume position spatially to superimpose it onto the others. The tissue continuity or integrity among slices in three orthogonal directions is unveiled in the global views, permitting visualization of reconstructed individual slices in 3D voxel volume format. With the global view using 3D visual representation and the color distribution criterion, any positioning error becomes readily identifiable and can be avoided in practice. Therefore, potential errors, shown in Table 2, can be significantly reduced. For 3P pixel interactive registration, human errors are often expressed as poor reproducibility (19), as the result of poor visual representation and the variation of the ability of inherent spatial orientation among observers. In addition to the improved accuracy, the 3D voxel technique can also accelerate the registration process. The 3D voxel registration results in Table 2 are checked using multiple planar views to determine if the registration is acceptable in the conventional view, and how and why the

DISCUSSION

270

I. J. Radiation Oncology

● Biology ● Physics

3P pixel registration often produces misalignment. These results are all acceptable from the conventional views. This finding suggests that there is an ambiguity among the best fit candidates produced by the 3P pixel technique. In fact, the users tend to drop their efforts to continue as soon as an acceptable registration solution is identified using the 3P pixel technique, unaware of the existence of better alignments. This would be the manual version of a “local minimum or maximum,” equivalent to that found in automatic optimization. Compared with automatic registration of rigid images, this global view eliminates erroneous results that occur when the cost function is caught in a local minimum/ maximum. Two misaligned results are seen in Table 1, consistent with previous reports that automatic registration should be validated visually (9, 10). In case of a relatively flat or noisy surface of the cost function around the optimal fitting point, ambiguity may exist in the automated registration results, depending on which stopping criterion is used. Therefore, several registrations with similar cost function values may be randomly picked with an arbitrary stopping criterion. Moreover, it is mathematically difficult to distinguish a random error from a systematic positioning error based on the cost function alone. A check on the randomness of the fitting residue should be done either visually or in other formats. Such visual checking, together with ability of fine adjustment, is essential to ensure a correct registration. The 3D voxel technique can provide such a visual assessment on the fitting residue. The color distribution can be regarded as color-coded fits overlaid on a curved surface, namely the landmark voxel volume surface. The randomness (or homogeneity) of the color distribution suggests the randomness of the fitting residue. In other words, inhomogeneous color distribution indicates a systematic bias in the image registration. Although this may only provide a partial view of the fitting residue, no previous report of this assessment has been seen. Therefore, the 3D voxel technique provides a better method for quality assurance, eliminating potential global misalignment or systematic bias, in comparison to the 3P pixel technique. The global view of the registration images can also be helpful to identify distortional error if it is present in any of the imaging modalities, as shown in Fig. 10. As a matter of fact, the initial check or correction for possible distortion should always be done for rigid image registration, as was reported for automated software (9). More advanced software can do auto-correction using an elastic deformation algorithm if the distortion is expected (15–18). Here, as all images are treated rigid volumes, distortion, if found, can be avoided, focusing instead on treatment target, rather than the overall volume. For CT/MR brain images where patients’ anatomy has changed over the course of treatment, the 3D voxel registration is one of the best choices. Note that the CT images do not contain the target, which is surgically removed but can be identified using registered presurgical MR images for

Volume 63, Number 1, 2005

the gross tumor volume. In addition to the tumor removal, some patients have shown significant weight lost as a result of the surgery, which is reflected in the two images. In these cases, automatic MMI registration (7, 21, 22) may encounter difficulties as the underlying anatomy has changed. In fact, the two misaligned registrations in Table 1 may be directly related to this reason. However, an earlier automatic registration algorithm using chamfer matching based on anatomic surface (5) may be useful in such cases, if the bone surface is used as the primary registration landmark. This suggests that any previously developed techniques may still have merit in particular clinical cases. The 3D voxel technique does not exclude 3P pixel registration tools. In addition to the three orthogonal slice planes, planar views with arbitrary orientation, location, and various geometrical shapes, shown in Figs. 7 and 8, are provided. One of the frequently used viewing tools is to view slice in UEV while the volume is rotated either manually or through animation. Here, planar view is combined with volumetric view. There are several methods by which examinations for registration and interior voxel matching can be performed. In this sense, the 3P pixel technique is a subset of the 3D voxel technique. In terms of registration methodology, it should always be starting from a global picture and then going into details. Similar methodology has been employed in automatic optimization related programs where the coarse fitting is done differently from that of fine-tuning with different resolutions. The global views not only can serve as the big picture for initial coarse fitting but also contain details for the final fine-tuning, owing to the visually sensitive registration criteria provided by the 3D voxel technique. The 3P pixel technique, however, is incapable of providing such global views and therefore the registration is buried in details, often producing a global positioning error.

Improved registration accuracy and performance The key to the improved registration accuracy with the 3D voxel technique is the ability to identify global positioning errors. With the registration criterion of color homogeneity, these errors are readily identifiable visually, as shown in Figs. 5a and 6a, and can be corrected with minimal effort, as shown in Figs. 5b and 6b. This is one of the most significant advantages of the 3D voxel technique over the 3P pixel technique. By comparison with the automatic MMI registration, the resulting accuracy is within a single voxel unit, better than the inherent image resolution, especially along the direction normal to the slices. Additionally, the various visualization tools provided in this study can help users to identify and register the landmarks. As stated above, all three steps in the interactive image registration have become an integrated process for this technique. In the initial stage, it is easy to find a start point that is close to the final result. In the final stage of the registration, as shown in Figs. 5 and 6, this technique can also improve accuracy, as long as

Volumetric voxel registration for multiple imaging modalities

appropriate landmarks are identified and a good criterion is used. The sensitive registration criteria used in the 3D voxel technique are mostly based upon the color distribution on a given voxel volume. Such criteria can operate not only on a contour surface but also on layers beneath. The visible volume can be changed using window/level or transparency curve based on the histogram of the voxel intensity for all modalities involved. Therefore, the 3D voxel registration criteria are fundamentally based on the voxel intensity, rather than surface information; so it can be interpreted as the visual and interactive version of maximization of mutual information. It is rather stunning that 13 of the 14 registration results (93%) done with the 3P pixel interactive technique contains positioning errors, as shown in Table 2. Two typical cases are shown in Figs. 5 and 6, where the inhomogeneous color distribution indicates the error. The “distance” of the deviations is 1.8° ⫾ 1.2° in rotation and 2.0 ⫾ 1.3 (voxel unit) in translation, as shown in Table 2. It is worthwhile to note that the mean of the deviations reflects the global positioning errors in the 3P pixel technique, whereas the standard deviations are the measure of the width of the error distribution. Here the normal distribution of the errors is assumed because the nature of the errors across different patients is generally random. Although these errors may be within clinical tolerance, the magnitude and frequency of the positioning errors associated with the 3P pixel registration is rather alarming. To the best of our knowledge, this is the first time that such global positioning errors have been addressed. This suggests that one should be cautious in using the conventional 3P pixel technique in clinic, and its results should be safeguard by the 3D voxel technique. Another key to a good registration is to have clearly defined anatomic landmarks. In the CT/MR cases, the skin/muscle voxels are often used as the landmarks because they are readily distinguishable from the surrounding air and they are on the surface. For registration with rigid image volumes, the landmarks are more reliable if they are rigid themselves or they are confined within a rigid environment. In this aspect, the quality of registration for the head is generally better than the rest of the body. Therefore, the quality of the registration is strongly correlated with landmark selection and reliability. If severe distortions or anatomic changes exist in one of the modalities, rigid automatic registration will fail. However, the 3D voxel technique can provide a clinically valid result by registering the valid anatomy only around the treatment target. In addition to the improved accuracy, the reduced registration time is another direct result of the global volumeview-guided registration. The tissue continuity and integrity is visually presented to the user throughout the entire process, and the global view allows the user to have a clear assessment about how well the current operation is and what the next adjustment should be. So, the user can focus on the

● G. LI et al.

271

registration itself, rather than struggling to fill the gap from the 3P pixel views to the 3D voxel views. As a consequence, the registration can be performed in an accelerated manner. Registration with functional imaging modality It has been demonstrated that the 3D volumetric voxel technique can register PET images with CT images, as shown in Figs. 9, 10, and 11. The image quality, resolution, and voxel coverage on the anatomic structure are critical, because they affect all of the three aspects of an interactive registration. Many functional images, such as PET and MR spectroscopy, usually suffer from low resolution. Moreover, they often provide incomplete anatomic structure, because their signal is more physiologically (cancer activity) than anatomically related. Therefore, the voxel coverage is expected to be different between anatomic images and functional images (11). For all registration strategies, including the 3D voxel technique, the common anatomic structures must reside in the imaging modalities involved in order for registration to be possible. Previous reports have indicated that brain registration criterion can be extracted from PET when the tracer is distributed throughout the whole brain (6). Therefore, some anatomic information can be obtained from the PET image as well. For other organs and anatomic structures, the coverage varies dramatically, depending upon the uptake and distribution of the PET marker. In this study, it has been demonstrated that the 3D voxel technique can be applied to CT/PET image registration, as shown in Figs. 9, 10, and 11. However, it is not possible to provide registration results by automatic MMI technique in this study until adequate CT/PET data sets are available to optimize the parameter settings in the MMI algorithms. Images of the torso, in general, are more difficult to register than images of the head, because the flexibility of the torso as well as the movement from respiration and cardiac activity reduce accuracy. For more sophisticated registration, internal organ motion should be taken into account. As reported in literature, organ motion can be corrected with an elastic registration algorithm (15–18) and has been considered in radiation treatment (24). Registered, multiple modality images are expected to guide the targeting of radiation dose more precisely to the functionally active site (25, 26). Limitations and perspectives of the 3D voxel registration All techniques and technologies have limitations, and the 3D voxel registration technique is no exception. First, the registration criteria are heavily dependent upon the visual distinction in color. Therefore, any color-blind person cannot use this tool for registration. Second, it requires at least one common anatomic structure that has enough voxel intensity in the different modalities involved. Special attention is needed when internal landmarks are used for the first time for registration. Third, it is relatively time-consuming to register functional images primarily because of incomplete anatomic information, compared with anatomic image

272

I. J. Radiation Oncology

● Biology ● Physics

registration. Finally, the 3D voxel registration technique can only register rigid image volumes and does not take into account any image deformation, at least in the current version of the program. For future research, nonrigid image registration can be incorporated into the 3D voxel technique as the interface between the volume rendering pipeline and the voxel transformation has been established (Fig. 1). Any existing deformation algorithms, therefore, can be applied and the result can be visualized in 3D volumetric representation. The homogeneity of color distribution on a given volumetric landmark, as a sensitive registration criterion, can also be used to verify or guide the nonrigid registration. It has been one of our future research interests. In addition, functional image registration remains more challenging than anatomic image registration. Further explorations of this 3D voxel technique for functional image registration need to be done, and the results should be compared with those from other independent registration techniques. In summary, we have developed a technique that utilizes global, integrated views on the patient’s anatomic continuity or physiologic activity as visual guidance during the image registration process. The homogeneity of the color distribution is validated for CT/MR images to be a sensitive visual criterion for both registration and fitting residue checking on a given anatomic landmark volume. The registration results from the 3D voxel technique are in a good agreement with those from the automatic MMI technique. This technique integrates all three common steps in one and provides the user with a visual guidance throughout the registration. With the visual and operational advantages provided by the 3D voxel technique, the shortcomings of conven-

Volume 63, Number 1, 2005

tional 3P pixel technique have been largely overcome. Based on this comparison study, it has been demonstrated that the 3D voxel technique is considered superior to the conventional 3P pixel technique in many aspects, including improved registration accuracy and reduced registration time. CONCLUSION The 3D volumetric voxel technique is a novel approach for volume-view-guided image registration of up to four imaging modalities simultaneously. It facilitates interactive image registration with visualized volume views, sensitive registration criteria, and straightforward operations. The 3D voxel technique can be applied to register anatomic and functional images, such as CT, MR, and PET. The 3D voxel technique has subvoxel accuracy in CT/MR image registration, which is comparable to that of automatic registration using mutual information. The registration result is generally superior to and more reliable than those from the 3P pixel technique. A global positioning error is often found in the 3P pixel registration, and it is readily identifiable visually and can be corrected by achieving the highest homogeneity of color distribution on the volumetric landmark. With the 3D voxel technique, the registration accuracy is improved to be within a single voxel using a fraction of the time, in comparison with the 3P pixel technique. Further explorations of internal voxel registration for functional images and of nonrigid registration for distorted images are needed and will be the focus of further research using the 3D volumetric voxel registration technique.

REFERENCES 1. Maurer CR, Fitzpatrick JM. A review of medical image registration. In: Maciunas RJ, editor. Interactive image-guided neurosurgery. Parkridge, IL: American Association of Neurological Surgeons; 1993. p. 17– 44. 2. Maintz JBA, Viergever MA. A survey of medical image registration. Med Image Analysis 1998;2:1–37. 3. Pluim JPW, Maintz JBA, Viergever MA. Mutual informationbased registration of medical images: A survey. IEEE Trans Med Imag 2003;22:986 –1004. 4. Woods RP, Mazziotta JC, Cherry SR. MR-PET registration with automated algorithm. J Comput Assist Tomogr 1993;17: 536 –546. 5. van Herk M, Kooy HM. Automatic three-dimensional correlation of CT-CT, CT-MRI, and CT-SPECT using chamfer matching. Med Phys 1994;21:1163–1178. 6. Mangin J-F, Frouin V, Bloch I, et al. Fast nonsupervised 3-D registration of PET and MR images of the brain. J Cereb Blood Flow Metab 1994;14:749 –762. 7. Wells WM III, Viola P, Atsumi H, et al. Multi-modal volume registration by maximization of mutual information. Med Image Analysis 1996;1:35–51. 8. Maes F, Collignon A, Vandermeulen D, et al. Multimodality

9.

10.

11.

12.

13.

14.

15.

image registration by maximization of mutual information. IEEE Trans Med lmag 1997;16:187–198. West J, Fitzpatrick JM, Wang MY, et al. Comparison and evaluation of retrospective intermodality brain image registration techniques. J Comput Assist Tomogr 1997;21:554 –566. Fitzpatrick JM, Hill DLG, Shyr Y, et al. Visual assessment of the accuracy of retrospective registration of MR and CT images of the brain. IEEE Trans Med lmag 1998;17:571–585. de Munck JC, Verster FC, Dubois, EA, et al. Registration of MR and SPECT without using external fiducial markers. Phys Med Biol 1998;43:1255–1269. West J, Fitzpatrick JM, Wang MY, et al. Retrospective intermodality registration techniques for images of the head: Surface-based versus volume-based. IEEE Trans Med Imag 1999; 18:144 –150. Maintz JBA, van den Elsen PA, Viergever MA. 3D multimodality medical image registration using morphological tools. Image Vision Computing 2001;19:53– 62. Barra V, Boire J-Y. A general framework for the fusion of anatomical and functional medical images. NeuroImage 2001; 13:410 – 424. Rueckert D, Sonoda LI, Hayes C, et al. Non-rigid registration

Volumetric voxel registration for multiple imaging modalities

16. 17. 18.

19.

20.

using free-form deformations: Applications to breast MR images. IEEE Trans Med Imag 1999;18:712–721. Shen D, Davatzikos C. Very high-resolution morphometry using mass-preserving deformations and HAMMER elastic registration. NeuroImage 2003;18:28 – 41. Makela T, Pham QC, Clarysse P, et al. A 3-D model-based registration approach for the PET, MR and MCG cardiac data fusion. Med Image Analysis 2003;7:377–389. Slomka PJ, Dey D, Przetak C, et al. Automated 3-dimensional registration of stand-alone 18F-FDG whole-body PET with CT. J Nucl Med 2003;44:1156 –1167. Vaarkamp J. Reproducibility of interactive registration of 3D CT and MR pediatric treatment planning head images. J Appl Clin Med Phys 2001;2:131–137. Xie H, Li G, Ning H, et al. 3D voxel fusion of multi-modality medical images in a clinical treatment planning system. In: Long R, Antani S, Lee DJ, et al., editors. Computer-based medical systems. Los Alamitos, CA: IEEE Computer Society; 2004. p. 48 –53.

● G. LI et al.

273

21. Ibanez L, Scehroeder W, Ng L, et al. The ITK software guide. 2nd edition. Clifton Park, NY: Kitware Inc.; 2005. 22. Mattes D, Haynor DR, Vesselle H, et al. PET-CT image registration in the chest using free-form deformations. IEEE Trans Med Imag 2003;22:120 –128. 23. Styner M, Gerig G. Evaluation of 2D/3D bias correction with 1⫹1ES-optimization. Tech. Rept. 179. Zurich: Image Sci Lab, ETH; 1997. 24. Jiang SB, Pope C, Jarrah KMA, et al. An experimental investigation on intra-fractional organ motion effects in lung IMRT treatments. Phys Med Biol 2003;48:1773–1784. 25. Esthappan J, Mutic S, Malyapa RS, et al. Treatment planning guidelines regarding the use of CT/PET-guided IMRT for cervical carcinoma with positive paraaortic lymph nodes. Int J Radiat Oncol Biol Phys 2004;58:1289 –1297. 26. Paulino AC, Johnstone FAS. FDG-PET in radiotherapy treatment planning: Pandora’s box? Int J Radiat Oncol Biol Phys 2004;59:4 –5.