- Email: [email protected]
Computer-aided lesion detection for brain pet images
Copyright i£; IFAC Modelling and Control in Biomedical Systems, Melbourne, Australia, 2003
IFAC PUBLICATIONS www.elsevier.com/locale/ifac
COMPUTER-AIDED LESION DETECTION FOR BRAIN PET IMAGES
Zhe Chen l , David Dagan Feng l ,2, Weidong Cai l
Biomedical & Multimedia Information Technology (BMIT) Group School ofInformation technologies The University ofSydney 2 Center for Multimedia Signal Processing (CMSP) Department ofElectronic & Information Engineering. Hong Kong Polvtechnic University
J
Abstract: We proposed a new method for the automatic detection of lesions in brain PET images based on the asymmetry across segments of the segmented PET data. Per-pixel asymmetry feature detection is experimentally compared with our per-segment approach. The proposed per-segment method is shown to produce fewer false positives in the brain PET data examples presented. Also, the algorithm is shown to provide accurate demarcation of the tumour in each plane of PET data. Therefore, this method offers potential advantages in region-of-interest (ROI) extraction for kinetic modelling and quantitative analysis. Copyright © 20031FAC Keywords: Positron emission tomography, Symmetry, Image alignment, Segmentation.
1. INTRODUCTION
lesion detection and region-of-interest (ROI) extraction for kinetic modelling and quantitative analysis.
A variety of imaging modalities are used to diagnose various disease states that manifest themselves as structural or functional changes. These modalities can be divided into two main categories: structural imaging modalities such as X-ray, Computed Tomography (CT). Magnetic Resonance Imaging (MRI). Ultrasound Imaging: and functional imaging modalities such as Single Photon Emission CT (SPECT), and Positron Emission Tomography
Due to the approximate bilateral symmetry of the human body, the measurement of bilateral asymmetry is a particularly useful feature in medical imaging applications. For example, Liu (200 I) performed 3D brain CT image retrieval based on semantic features using statistical bilateral asymmetry measures for normal and pathological human brains. Batty et al (2002) presented a PET brain image retrieval method using detection and measurement of asymmetrical features. However these methods are based on a per-pixel approach, and on noisy PET images this can lead to many false positives, necessitating a large difference threshold and hence lowered sensitivity.
(PET).
Unlike the anatomical/structural imaging techniques that delineate anatomy, biomedical functional imaging techniques allow us to see dynamic processes quantitatively in the living human body (Feng et al.. 1997). The main advantage of functional imaging is that it enables the quantitative detection of functional changes in tissues that is complementary information to the structural modalities. However, PET images typically have relatively low signal to noise ratio and low resolution that limits the use of anatomical content features such as shape for PET
In this paper, we describe a novel per-segment asymmetry feature detection approach for brain PET images to automatically extract pathological lesions, which is an essential initial step in kinetic modelling and quantitative analysis. This paper is organized as
163
subtraction creates the per-pixel difference Image.
follows: section 2 gives a description of the proposed method for brain PET lesion detection based on asymmetrical feature measurement: in sections 3, the clinical case studies and related experimental results are described and discussed; and finally, conclusions are presented in section 4.
In step 3, over each segment from step 2A. we average the per-pixel difference image from step 2B, to create a per-segment asymmetry measure. In step 4, each segment having an average asymmetry value greater than some threshold is flagged as significant and displayed.
2. METHOD Our approach to asymmetry feature detection in PET brain lesions is illustrated in figure I with four major steps: image alignment, image segmentation & symmetric difference image creation, averaging asymmetry over each segment, and thresholding based on the segment asymmetry value.
2.
Step I is image alignment. To calculate asymmetry in the later steps we first need to ensure that the midsagittal line of the PET image is centred and vertical. In this paper we use a moment-based approach (Prokop 1992). We start with a binary valued thresholded version of the original image. The translation and rotation of the principal symmetry axis of the image is then calculated by a moment analysis of the binary image. If the ijth discrete central moment m" of a binary region is defined by:
m = L(X -~Hy- yY
3. Average asymmetry difference over each segment
4. Threshold based on segment asymmetry
Figure I. Four steps of Asymmetry measurement
where the sums are taken over all points (x,y)
-
I
-
(~, y) is the centroid 3. RESULTS AND DISCUSSION
I
x=-LxandY=-LY n n ,.
(2)
To evaluate the algorithm, clinical human brain FOG-PET studies of two patients were used: one is a normal data set, and the other is a pathological data set with an obvious tumour. These image data sets were acquired by a SIEMENS ECAT 951 R PET scanner at the PET and Nuclear Medicine Department. Royal Prince Alfred Hospital, Sydney. The number of cross-sectional image planes was 31. A typical sampling schedule consisting of 22 temporal frames was used to acquire the PET projection data. The PET scanning schedule was 6 x 10.0 second scans, 4 x 30.0 second scans, I x 120.0 second scans, and 1I x 300 second scans. In this paper, we only used the last frame of each 22 frame temporal sequence for each plane.
x
Then the angle of rotation of the principal axis of the image is given by B where
I -'[ B =-tao 2
2m " ] - m 02
(3)
m 20
The original image is then aligned by translating the centroid to the image centre and then rotating by
-B. Step 2 includes two parts, one is image segmentation, and the other is symmetric difference image creation. • A. The segmentation method used is a Mumford-Shah energy-minimization, region-merging algorithm. This algorithm has been demonstrated to be effective in segmenting features even in noisy, low resolution PET images. The details of the segmentation algorithm used are provided in Parker (2001 and 2002). Currently each PET data set plane is segmented using a two-dimensional segmentation. •
Subtract reflection image from the original image
(I)
ij
contained within the image and of the image:
Image segmentation
The performance of the full per-segment algorithm was compared against a simple per-pixel approach that did not apply the segmentation, on plane # 16 of the normal data set and plane #20 of the tumour data set. The full per-segment algorithm was then applied to planes 16-30 for the tumour data set, and visually evaluated. Note that all of the images presented here have been individually contrast-adjusted such that the highest value has been set to I and the lowest value set to 0, except figures 3(d) and 4(d) which have been scaled the same to be comparable.
B. We create a reflection image about the vertical mid plane of the aligned original image and then subtract it from the original image. The absolute value of this image
164
detected but with many false positives as well. Note that the tumour in figure 4-f is very well demarcated.
a
b
c
d
In figures 5 and 6 the per-segment asymmetry algorithm has correctly detected and localized the right hemisphere tumour present in planes 16 to 26, with no false positives. In addition to the tumour from planes 27 to 30 it can be seen that there i~ asymmetry between the left and right cerebellar hemispheres. In plane 30 this cerebellar asymmetry becomes large enough to be detected by our method. Also, in planes 27 and 28 the left temporal lobe is asymmetric compared with the right and so is detected. This is an example of how asymmetry features alone are not entirely sufficient to distinguish different causes of asymmetry. Other features such as spatial location or perhaps texture would be needed to distinguish different types of pathological asymmetries from other causes of asymmetry.
Figure. 2. Image alignment a. Misaligned normal image. b. Misaligned tumour image. c. Aligned normal image d. Aligned tumour image
4. CONCLUSIONS In this paper, we proposed an asymmetry detection algorithm based on the asymmetry of individual segments as opposed to a per-pixel comparison, and applied it to brain PET data sets. In comparison to a simple per-pixel asymmetry feature detection, our per-segment approach was shown to provide fewer false positives at a given threshold level in the PET data-set sample tested. Because of the noisy nature of PET imaging, the simpler per-pixel approach would require a higher threshold to avoid false positives, and this would lead to a lower sensitivity for asymmetric lesion detection compared with the persegment approach.
Figure 2 illustrates the results of image alignment. To test the performance of the alignment code, the original data was first artificially severely displaced and rotated. Figure 2-a and Figure2-b are the rotated and shifted versions of the original datasets. Figure 2-c and figure 2-d are the aligned normal image and tumour image respectively. The image asymmetry detection steps for the normal data set are shown in figure 3. Figure 3-a shows the original normal image. Figure 3-b shows the normal data set after segmentation. Figure 3-c illustrates the absolute difference image created by the absolute value of the subtraction of the reflection image and the original image. Figure 3-d is the result of averaging the absolute difference image (figure 3-c) over each segment of figure 3-b. Figure 3-e shows the final per-pixel result of thresholding the difference image of figure 3-c with a threshold value of 0.15. Figure 3-f shows the final per-segment result of thresholding the segments of figure 3-d with a threshold value of 0.15. Figures 4-a to 4-f show the corresponding results for the tumour data set.
Also, the results demonstrate very accurate demarcation and localization of the tumour in each plane of a PET data set. This is a significant advantage of our approach as having well demarcated and segmented lesion data is an important initial step for later processing, and in particular for region-of-interest (ROI) extraction for kinetic modelling and quantitative analysis.
ACKNOWLEDGEMENT This work is partially supported by the ARC and UGC grants. Thanks to Brian Parker for explaining his segmentation algorithm.
Figure 5 shows the original planes 16 to 30 of the tumour data set. Figure 6 shows the results of the full per-segment asymmetry detection algorithm applied to planes 16-30 of the tumour data set.
REFERENCES
Figure 3-f shows the per-segment algorithm has no false positives for this image. By contrast figure 3-e shows that a simple per-pixel approach produces many false positives at the same threshold level. Figure 4-f shows the asymmetric tumour correctly detected by the per-segment algorithm with no false positives. Figure 4-e shows the tumour asymmetry
Batty, S., J. Clark, T. Fryer, X.W. Gao, (2002). Extraction of Features from 3D PET Images Medical Image Understanding and Analvsis 2002. 22-23 July 2002, The universitv' of Portsmouth, Portsmouth, U.K. Feng, D., D. Ho, H. lida, and K. Chen (1997). Techniques for Functional Imaging, invited
165
chapter, in: C.T. Leondes ed., Medicallmaging Systems Techniques and Applications: General Anatomy. Amsterdam: Gordon and Breach Science Publishers, 85-145. Liu, Y., F. Dellaert, W.E. Rothfus, A. Moore, J. Schneider, T. Kanade (2001). ClassificationDriven Pathological Neuroimage Retrieval Using Statistical Asymmetry Measures', Proceedings of International Conference of Medical Image Computing and Computer Assisted Intervention (MICCAI 2001), October 14-17, 2001, Utrecht, The Netherlands. Prokop RJ. and Reeves A.P. (1992), A survey of moment-based techniques for unoccluded object representation and recognition, CVGJP:Graphical Models and Image Processing, Vol. 54, No. 5, pp.438-460, Sept. 1992. Parker, B. (2001) Three-Dimensional Medical Image Segmentation Using a Graph-Theoretic EnergyMinimisation Approach. Proceedings of PanSydney Area Workshop on Visual Information processing. Sydney. 2001. Parker, B. and D. D. FENG (2002) Variational Segmentation and PCA Applied to Dynamic PET Analysis. Proceedings of Pan-Sydney Area Workshop on Visual Information processing. Sydney, 2002.
a
b. Segmented image c. Absolute difference image between the original image (Figure 3-a) and its own vertical reflection d. Average absolute difference image (Figure 3-c) over each segment in the segmentation image (Figure 3-b). e. Thresholded image of the difference image (Figure 3-c) with Threshold value = O. I 5 f. Thresholded segments of Figure 3-d with Threshold value = 0.15.
b
c
d
e
f
a
b
c
d
e
f
Figure 4. Asymmetry feature detection for PET image with tumour a. Original image with its midsagittal line aligned and vertical. b. Segmented image c. Absolute difference image between the original image (Figure 4-a) and its own vertical reflection d. Average absolute difference image (Figure 4-c) over each segment in the segmentation image (Figure 4-b). e. Thresholded image of the difference image (Figure 4-c) with Threshold value = 0.15 f. Thresholded segments of Figure 4-d with Threshold value 0.15.
Figure 3. Asymmetry feature detection for PET image without tumour a. Original image with its midsagittal line aligned and vertical.
166
Figure 5. Original Tumour images (aligned) of plane 16 to plane 30 from left to right and top to bottom.
Figure 6. Corresponding asymmetry feature extraction images of plane 16 to plane 30.
167
IFAC PUBLICATIONS www.elsevier.com/locale/ifac
COMPUTER-AIDED LESION DETECTION FOR BRAIN PET IMAGES
Zhe Chen l , David Dagan Feng l ,2, Weidong Cai l
Biomedical & Multimedia Information Technology (BMIT) Group School ofInformation technologies The University ofSydney 2 Center for Multimedia Signal Processing (CMSP) Department ofElectronic & Information Engineering. Hong Kong Polvtechnic University
J
Abstract: We proposed a new method for the automatic detection of lesions in brain PET images based on the asymmetry across segments of the segmented PET data. Per-pixel asymmetry feature detection is experimentally compared with our per-segment approach. The proposed per-segment method is shown to produce fewer false positives in the brain PET data examples presented. Also, the algorithm is shown to provide accurate demarcation of the tumour in each plane of PET data. Therefore, this method offers potential advantages in region-of-interest (ROI) extraction for kinetic modelling and quantitative analysis. Copyright © 20031FAC Keywords: Positron emission tomography, Symmetry, Image alignment, Segmentation.
1. INTRODUCTION
lesion detection and region-of-interest (ROI) extraction for kinetic modelling and quantitative analysis.
A variety of imaging modalities are used to diagnose various disease states that manifest themselves as structural or functional changes. These modalities can be divided into two main categories: structural imaging modalities such as X-ray, Computed Tomography (CT). Magnetic Resonance Imaging (MRI). Ultrasound Imaging: and functional imaging modalities such as Single Photon Emission CT (SPECT), and Positron Emission Tomography
Due to the approximate bilateral symmetry of the human body, the measurement of bilateral asymmetry is a particularly useful feature in medical imaging applications. For example, Liu (200 I) performed 3D brain CT image retrieval based on semantic features using statistical bilateral asymmetry measures for normal and pathological human brains. Batty et al (2002) presented a PET brain image retrieval method using detection and measurement of asymmetrical features. However these methods are based on a per-pixel approach, and on noisy PET images this can lead to many false positives, necessitating a large difference threshold and hence lowered sensitivity.
(PET).
Unlike the anatomical/structural imaging techniques that delineate anatomy, biomedical functional imaging techniques allow us to see dynamic processes quantitatively in the living human body (Feng et al.. 1997). The main advantage of functional imaging is that it enables the quantitative detection of functional changes in tissues that is complementary information to the structural modalities. However, PET images typically have relatively low signal to noise ratio and low resolution that limits the use of anatomical content features such as shape for PET
In this paper, we describe a novel per-segment asymmetry feature detection approach for brain PET images to automatically extract pathological lesions, which is an essential initial step in kinetic modelling and quantitative analysis. This paper is organized as
163
subtraction creates the per-pixel difference Image.
follows: section 2 gives a description of the proposed method for brain PET lesion detection based on asymmetrical feature measurement: in sections 3, the clinical case studies and related experimental results are described and discussed; and finally, conclusions are presented in section 4.
In step 3, over each segment from step 2A. we average the per-pixel difference image from step 2B, to create a per-segment asymmetry measure. In step 4, each segment having an average asymmetry value greater than some threshold is flagged as significant and displayed.
2. METHOD Our approach to asymmetry feature detection in PET brain lesions is illustrated in figure I with four major steps: image alignment, image segmentation & symmetric difference image creation, averaging asymmetry over each segment, and thresholding based on the segment asymmetry value.
2.
Step I is image alignment. To calculate asymmetry in the later steps we first need to ensure that the midsagittal line of the PET image is centred and vertical. In this paper we use a moment-based approach (Prokop 1992). We start with a binary valued thresholded version of the original image. The translation and rotation of the principal symmetry axis of the image is then calculated by a moment analysis of the binary image. If the ijth discrete central moment m" of a binary region is defined by:
m = L(X -~Hy- yY
3. Average asymmetry difference over each segment
4. Threshold based on segment asymmetry
Figure I. Four steps of Asymmetry measurement
where the sums are taken over all points (x,y)
-
I
-
(~, y) is the centroid 3. RESULTS AND DISCUSSION
I
x=-LxandY=-LY n n ,.
(2)
To evaluate the algorithm, clinical human brain FOG-PET studies of two patients were used: one is a normal data set, and the other is a pathological data set with an obvious tumour. These image data sets were acquired by a SIEMENS ECAT 951 R PET scanner at the PET and Nuclear Medicine Department. Royal Prince Alfred Hospital, Sydney. The number of cross-sectional image planes was 31. A typical sampling schedule consisting of 22 temporal frames was used to acquire the PET projection data. The PET scanning schedule was 6 x 10.0 second scans, 4 x 30.0 second scans, I x 120.0 second scans, and 1I x 300 second scans. In this paper, we only used the last frame of each 22 frame temporal sequence for each plane.
x
Then the angle of rotation of the principal axis of the image is given by B where
I -'[ B =-tao 2
2m " ] - m 02
(3)
m 20
The original image is then aligned by translating the centroid to the image centre and then rotating by
-B. Step 2 includes two parts, one is image segmentation, and the other is symmetric difference image creation. • A. The segmentation method used is a Mumford-Shah energy-minimization, region-merging algorithm. This algorithm has been demonstrated to be effective in segmenting features even in noisy, low resolution PET images. The details of the segmentation algorithm used are provided in Parker (2001 and 2002). Currently each PET data set plane is segmented using a two-dimensional segmentation. •
Subtract reflection image from the original image
(I)
ij
contained within the image and of the image:
Image segmentation
The performance of the full per-segment algorithm was compared against a simple per-pixel approach that did not apply the segmentation, on plane # 16 of the normal data set and plane #20 of the tumour data set. The full per-segment algorithm was then applied to planes 16-30 for the tumour data set, and visually evaluated. Note that all of the images presented here have been individually contrast-adjusted such that the highest value has been set to I and the lowest value set to 0, except figures 3(d) and 4(d) which have been scaled the same to be comparable.
B. We create a reflection image about the vertical mid plane of the aligned original image and then subtract it from the original image. The absolute value of this image
164
detected but with many false positives as well. Note that the tumour in figure 4-f is very well demarcated.
a
b
c
d
In figures 5 and 6 the per-segment asymmetry algorithm has correctly detected and localized the right hemisphere tumour present in planes 16 to 26, with no false positives. In addition to the tumour from planes 27 to 30 it can be seen that there i~ asymmetry between the left and right cerebellar hemispheres. In plane 30 this cerebellar asymmetry becomes large enough to be detected by our method. Also, in planes 27 and 28 the left temporal lobe is asymmetric compared with the right and so is detected. This is an example of how asymmetry features alone are not entirely sufficient to distinguish different causes of asymmetry. Other features such as spatial location or perhaps texture would be needed to distinguish different types of pathological asymmetries from other causes of asymmetry.
Figure. 2. Image alignment a. Misaligned normal image. b. Misaligned tumour image. c. Aligned normal image d. Aligned tumour image
4. CONCLUSIONS In this paper, we proposed an asymmetry detection algorithm based on the asymmetry of individual segments as opposed to a per-pixel comparison, and applied it to brain PET data sets. In comparison to a simple per-pixel asymmetry feature detection, our per-segment approach was shown to provide fewer false positives at a given threshold level in the PET data-set sample tested. Because of the noisy nature of PET imaging, the simpler per-pixel approach would require a higher threshold to avoid false positives, and this would lead to a lower sensitivity for asymmetric lesion detection compared with the persegment approach.
Figure 2 illustrates the results of image alignment. To test the performance of the alignment code, the original data was first artificially severely displaced and rotated. Figure 2-a and Figure2-b are the rotated and shifted versions of the original datasets. Figure 2-c and figure 2-d are the aligned normal image and tumour image respectively. The image asymmetry detection steps for the normal data set are shown in figure 3. Figure 3-a shows the original normal image. Figure 3-b shows the normal data set after segmentation. Figure 3-c illustrates the absolute difference image created by the absolute value of the subtraction of the reflection image and the original image. Figure 3-d is the result of averaging the absolute difference image (figure 3-c) over each segment of figure 3-b. Figure 3-e shows the final per-pixel result of thresholding the difference image of figure 3-c with a threshold value of 0.15. Figure 3-f shows the final per-segment result of thresholding the segments of figure 3-d with a threshold value of 0.15. Figures 4-a to 4-f show the corresponding results for the tumour data set.
Also, the results demonstrate very accurate demarcation and localization of the tumour in each plane of a PET data set. This is a significant advantage of our approach as having well demarcated and segmented lesion data is an important initial step for later processing, and in particular for region-of-interest (ROI) extraction for kinetic modelling and quantitative analysis.
ACKNOWLEDGEMENT This work is partially supported by the ARC and UGC grants. Thanks to Brian Parker for explaining his segmentation algorithm.
Figure 5 shows the original planes 16 to 30 of the tumour data set. Figure 6 shows the results of the full per-segment asymmetry detection algorithm applied to planes 16-30 of the tumour data set.
REFERENCES
Figure 3-f shows the per-segment algorithm has no false positives for this image. By contrast figure 3-e shows that a simple per-pixel approach produces many false positives at the same threshold level. Figure 4-f shows the asymmetric tumour correctly detected by the per-segment algorithm with no false positives. Figure 4-e shows the tumour asymmetry
Batty, S., J. Clark, T. Fryer, X.W. Gao, (2002). Extraction of Features from 3D PET Images Medical Image Understanding and Analvsis 2002. 22-23 July 2002, The universitv' of Portsmouth, Portsmouth, U.K. Feng, D., D. Ho, H. lida, and K. Chen (1997). Techniques for Functional Imaging, invited
165
chapter, in: C.T. Leondes ed., Medicallmaging Systems Techniques and Applications: General Anatomy. Amsterdam: Gordon and Breach Science Publishers, 85-145. Liu, Y., F. Dellaert, W.E. Rothfus, A. Moore, J. Schneider, T. Kanade (2001). ClassificationDriven Pathological Neuroimage Retrieval Using Statistical Asymmetry Measures', Proceedings of International Conference of Medical Image Computing and Computer Assisted Intervention (MICCAI 2001), October 14-17, 2001, Utrecht, The Netherlands. Prokop RJ. and Reeves A.P. (1992), A survey of moment-based techniques for unoccluded object representation and recognition, CVGJP:Graphical Models and Image Processing, Vol. 54, No. 5, pp.438-460, Sept. 1992. Parker, B. (2001) Three-Dimensional Medical Image Segmentation Using a Graph-Theoretic EnergyMinimisation Approach. Proceedings of PanSydney Area Workshop on Visual Information processing. Sydney. 2001. Parker, B. and D. D. FENG (2002) Variational Segmentation and PCA Applied to Dynamic PET Analysis. Proceedings of Pan-Sydney Area Workshop on Visual Information processing. Sydney, 2002.
a
b. Segmented image c. Absolute difference image between the original image (Figure 3-a) and its own vertical reflection d. Average absolute difference image (Figure 3-c) over each segment in the segmentation image (Figure 3-b). e. Thresholded image of the difference image (Figure 3-c) with Threshold value = O. I 5 f. Thresholded segments of Figure 3-d with Threshold value = 0.15.
b
c
d
e
f
a
b
c
d
e
f
Figure 4. Asymmetry feature detection for PET image with tumour a. Original image with its midsagittal line aligned and vertical. b. Segmented image c. Absolute difference image between the original image (Figure 4-a) and its own vertical reflection d. Average absolute difference image (Figure 4-c) over each segment in the segmentation image (Figure 4-b). e. Thresholded image of the difference image (Figure 4-c) with Threshold value = 0.15 f. Thresholded segments of Figure 4-d with Threshold value 0.15.
Figure 3. Asymmetry feature detection for PET image without tumour a. Original image with its midsagittal line aligned and vertical.
166
Figure 5. Original Tumour images (aligned) of plane 16 to plane 30 from left to right and top to bottom.
Figure 6. Corresponding asymmetry feature extraction images of plane 16 to plane 30.
167