- Email: [email protected]
Improved watershed transform for tumor segmentation: Application to mammogram image compression
Expert Systems with Applications 39 (2012) 3950–3955
Contents lists available at SciVerse ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Improved watershed transform for tumor segmentation: Application to mammogram image compression Wei-Yen Hsu Graduate Institute of Biomedical Informatics, Taipei Medical University, 250 Wu-Xin Street, Taipei 110, Taiwan
a r t i c l e
i n f o
Keywords: Image segmentation Mammogram Tumor Improved watershed transform Vector quantization Competitive Hopfield neural network (CHNN)
a b s t r a c t In this study, an automatic image segmentation method is proposed for the tumor segmentation from mammogram images by means of improved watershed transform using prior information. The segmented results of individual regions are then applied to perform a loss and lossless compression for the storage efficiency according to the importance of region data. These are mainly performed in two procedures, including region segmentation and region compression. In the first procedure, the canny edge detector is used to detect the edge between the background and breast. An improved watershed transform based on intrinsic prior information is then adopted to extract tumor boundary. Finally, the mammograms are segmented into tumor, breast without tumor and background. In the second procedure, vector quantization (VQ) with competitive Hopfield neural network (CHNN) is applied on the three regions with different compression rates according to the importance of region data so as to simultaneously reserve important tumor features and reduce the size of mammograms for storage efficiency. Experimental results show that the proposed method gives promising results in the compression applications. Ó 2011 Elsevier Ltd. All rights reserved.
1. Introduction Breast cancer is the most common one for women in worldwide. The women disease incidence rate that the breast cancer leaps to first poses the quite big threat to the domestic women. The X-ray mammography is an effective method to achieve the goal of early diagnoses. Accordingly, the accurate segmentation of tumor in mammogram images in very important. Image segmentation plays an important role and is an essential process in medical images. General segmentation is the process of partitioning the image into disjointed regions so as to the characteristics of each region are homogeneous (Harlick & Shapiro, 1985). A large variety of image segmentation methods have been presented (Dokur, 2008; Dokur & Ölmez, 2008; Hsu, 2010; Hsu, Poon, & Sun, 2008; Lai & Chang, 2009; Lin, Tsai, Hung, & Shih, 2006; Pal & Pal, 1993; Sahba, Tizhoosh, & Salama, 2008). Among these methods, the watershed (Beucher & Meyer, 1993) that is a region-based approach is a traditional but popular method from mathematical morphology. The region-based approaches group similar pixels into a region based on some pixel information. The advantages of watershed approach are that it is fast, it can be parallelized (Moga & Gabbouj, 1997), and it produces a complete division of image even if its contrast is low. Many techniques related to the watersheds have been presented in recent decades
E-mail addresses: [email protected], [email protected] 0957-4174/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2011.08.148
(Chen, Lien, & Tzeng, 2010; Hsu, Lin, Ju, & Sun, 2007; Vincent & Soille, 1991). The main drawback of watershed algorithm is that it produces over-segmented results. That is, when the watershed obtains catchment basins from the gradient of image, the results of watershed contain too many small regions. Moreover, it is sensitive to noise. Local variations of the image can significantly change the results. It results from the use of high pass filters to estimate the gradient. In addition, it is poor detection in significant areas with low contrast boundaries. If the signal to noise ratio is not high enough at the contour, the watershed transform will be unable to detect it accurately. Accordingly, the improved version of watershed algorithm may be overcome the intrinsic problems. In addition, various kinds of preprocessing have been developed to solve the problems of over-segmentation, such as the median filter and anisotropic diffusion filter (Weickert, 1998). In this study, an automatic image segmentation method based on improved watershed transform using prior information is proposed for the tumor segmentation from mammogram images. Moreover, for medical applications, we should be very cautious to retain sufficient image information in supporting different diagnosis purposes during the compression. A common approach is to perform different compression rates, including loss and lossless, for different degrees of important regions. Hence, the segmented results are separated original mammogram image into the tumor, the breast without tumor, and background. Since these two regions, the breast without tumor and background, are not important in diagnosis, the vector quantization with competitive Hopfield
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955
neural network (CHNN) is applied for loss compression. The Hopfield neural network (Hopfield & Tank, 1985) is a well-known technique used for solving optimization problems based on the Lyapunov energy function. In the study, the competitive learning rule for CHNN is modified to address the codebook design. The CHNN is constructed as a two-dimensional fully interconnected array with the rows representing the training vectors and columns standing for the codevectors in the codebook. This paper is organized as follows: Section 2 presents the segment and compression methods. It describes the results and discussion in Section 3. A conclusion is given in Section 4.
3951
to calculate the watershed transform. The lower neighbors are directly derived from the lower slope. That is, for each image pixel p, lower neighbors LN(p) are denoted,
f ðpÞ f ðp0 Þ 0 LNðpÞ ¼ p 2 NðpÞ 0 dðp; p Þ
ð2Þ
2. Methods
The set of lower neighbors is the subset of neighboring pixels for which the directed gradient to the pixel p equals its lower slope. In practical applications, the watershed transform is not calculated directly on the image, but rather on the absolute value of its gradient, which has high values at the contours. Gradient estimation at the center of the pixels reduces the original resolution of images. Thus, the lower slope is redefined as
2.1. Canny edge detection
LSðpÞ ¼ max
Edge detection means to extract sharp content information from the image. It plays an important role in feature-based image analysis. The most important topic in edge detection is how to extract representative features from the image in low contrast. Furthermore, three criteria, including detection (important edges should not be missed and there should be no spurious responses), localization (the distance between the actual and located position of the edge should be minimal) and one response (minimizing multiple responses to a single edge) are used to judge if the edge detector is optimal or not. Hence, the Canny edge detector (Canny, 1986) is adopted to coincide with the above-mentioned three criteria owing to its good localization and elastic thresholding. In this study, the Canny edge detector is used to detect the edge between the background and breast. In addition, the parameter r, the Gaussian scaling, of the Canny edge detector decides the amount and the fineness of extracted edges. In other words, there are fewer but more significant edges extracted when the image is convoluted with larger scaling r; while convoluting with a smaller scaling r, there are more and detailed edge responses as well as better edge localization. We detect edge locations from the acquired image after convolution. In other words, we use the Gaussian scaling r to adjust the amount and the fineness of edges. In addition to the scaling r, the other two parameters, high and low thresholds, of the Canny edge detector are also taken into account. If an edge response is over the high threshold, it is considered as a definite edge for a particular scaling. Individual weak responses usually coincide with noise except when they are neighbor to the pixels with strong responses, and they are likely to be true edges in the acquired image. These points are regarded as true edge points as long as their edge responses are over the low threshold. Both of them are set according to the estimated signal-to-noise ratio. Summarily speaking, edges are detected by adjusting these three parameters according to the characteristics and quality of images.
where jf ðpÞ f ðqÞj is calculated for the link (p, q). We have available prior information in the absolute or relative intensities of images. To take advantage of prior information, a set of lower cost functions for each object is used
jf ðpÞ f ðqÞj q2NðpÞ[p dðp; qÞ
2.2. Improved watershed transform An improved watershed transform based on intrinsic prior information (Grau, Mewes, Alcañiz, Kikinis, & Warfield, 2004) is adopted to extract tumor boundary from the breast. We first introduce important definitions of watershed transform about the lower slope and lower neighbors. Let f be a gray image. The lower slope of f at a pixel p, LS(p) is defined,
f ðpÞ f ðqÞ q2NðpÞ[p dðp; qÞ
LSðpÞ ¼ max
ð1Þ
where N(p) is the set of neighbors of p, and d(p, q) is the Euclidean distance between pixels p and q. To define a steepest slope relation between pixels is necessary for the lower slope, which will be used
fe ðp; qÞ q2NðpÞ[p dðp; qÞ
LSe ðpÞ ¼ max
ð3Þ
ð4Þ
where fe ðp; qÞ is to quantify the probability of having an edge between the pixels p and q. To calculate the functions, it is assumed that the label of pixel p is known before labeling pixel q. This is reached if a region-growing algorithm is used for watershed calculation. The improved watershed transform produces a possibility for different applications, depending on the amount of knowledge available on the objects. A function is used to measure the difference in class probability between two neighboring pixels for generic image segmentation. Normal distributions are assumed for the objects in the image, for which mean and variance are calculated using a set of seed pixels for each class. The seeds are selected using automatic techniques. Posterior probabilities for each class i at each pixel p are calculated using Bayes’ rule as
PðIp jiÞPðiÞ PðijIp Þ ¼ P i PðI p jiÞPðiÞ
ð5Þ
where Ip is the intensity of the image at pixel p. The structures exhibit a significant spatial homogeneity, and so it is desirable to achieve this behavior in the obtained probability values. Markov random fields (Besag, 1986; Geman & Geman, 1984) provide a way to model local correlations between pixels. To estimate the model, iterative conditional modes (Besag, 1986) are used to iteratively solve. 2.3. Vector quantization with competitive Hopfield neural network The Hopfield neural network with simple architecture and parallel potential has been applied in many fields (Hsu & Sun, 2009; Wang & Zhou, 2009). It consists of a single layer of processing elements where each neuron is connected to every other neuron in the network. The Hopfield neural network is a well-known technique used for solving optimization problems based on the Lyapunov energy function. In this study, we use a discrete competitive Hopfield neural network (CHNN), where the winner-takes-all method is adopted to learn weighting factors in the energy function, for vector quantization. That is, the CHNN is used to quantize the vectors from segmented regions, and compress and reconstruct the regions. Suppose an image is divided into n blocks (vectors of pixels) and each block occupies ‘ ‘ pixels. A vector quantizer is a technique that maps the Euclidean ‘ ‘-dimensional space R‘‘ into a set
3952
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955
fxj ; j ¼ 1; 2; . . . ; cg of points in R‘‘ , called a codebook. It looks for a codebook such that each training vector is approximated as close as possible by one of the code vectors in the codebook. A codebook is optimal if the average distortion is at the minimum value. The average distortion E½dðxy ; xj Þ between an input sequence of training vectors fxy ; y ¼ 1; 2; . . . ; ng and its corresponding output sequence of code vectors fxj ; j ¼ 1; 2; . . . ; cg is defined as n 1X D ¼ E½dðxy ; xj Þ ¼ dðxy ; xj Þ n y¼1
ð6Þ
To update the training performance, the simplified object function for the competitive Hopfield neural network (CHNN) based on competitive learning can be modified as
E¼
n X n X c X c X x¼1 y¼1 i¼1
V x;i W x;i;y;j V y;j
ð7Þ
j¼1
and
Netx;i ¼
n X c X
W x;i;y;j V y;j
ð8Þ
y¼1 j¼1
Fig. 1. Procedure of proposed segmentation method. (a) A test image. (b) The edge between the background and breast detected by Canny edge detector. (c) The tumor extracted by improved watershed transform. (d) and (e) The tumor outlined by experts 1 and 2, respectively.
3953
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955
where W x;i;y;j is the weight between neurons (x, i) and (y, j), and Ix;i ¼ 0.Using Eqs. (7) and (8), the Hopfield neural network based on competitive learning can be used for vector quantization in a parallel manner are given as follows: Step 1: Input a set training vector X ¼ fx1 ; x2 ; . . . ; xn g, and the number of class c. c Step 2: Compute the weight matrix W ¼ fwx;i;y;j gn; y¼1;j¼1 with training samples and codevectors. Step 3: Set the initial number of the vectors to be n. Each class contains at least one vector. Step 4: Calculate the input to each neuron (x, i) by Eq. (8). Step 5: Apply below equation to update the neurons’ output states for each neuron in a row
V x;i ¼
1 if Net x;i ¼ maxfNet x;1 ; Net x;2 ; . . . Netx;c g 0 otherwise
Step 6: Repeat Steps 4 and 5 for all rows, and count the number of neurons for the new state. If no neuron is changed go to Step 7, otherwise go to Step 4. Step 7: Complete the codebook design.
features from images in low contrast. More specifically, the Canny edge detector can achieve the criteria of detection, localization and on response due to its good localization and elastic thresholding. In addition, to extract tumor boundary from the breast, an improved watershed transform based on intrinsic prior information is applied. The Bayes’ rule is used to calculate posterior probabilities of each pixel for each class. The local correlations between pixels are then modeled by Markov random fields. To demonstrate the results of proposed segmentation method, the public database Mammographic Image Analysis Society MiniMammographic Database (MiniMIAS) is adopted in the experiments. The size of all the images in MiniMIAS database has been reduced to 1024 1024 pixels. The procedure of proposed segmentation method is illustrated in Fig. 1. A test image is shown in Fig. 1(a). Fig. 1(b) shows the edge between the breast and background detected by Canny edge detector, whereas the tumor extracted by improved watershed transform is shown in Fig. 1(c). Fig. 1(d) and 1(e) shows the tumor outlined by experts 1 and 2, respectively. In addition, Fig. 2 illustrates the segmented results
Table 1 Average similarity measure with Jaccard and Dice indexes between the regions segmented by improved watershed transform and regions outlined by two experts.
3. Results and discussion 3.1. Region segmentation The boundary between the background and breast is detected by Canny edge detector. This is because it can extract important
Average similarity measure
Jaccard index J
Dice index D
vs. Expert 1 vs. Expert 2
0.937 0.892
0.968 0.943
Fig. 2. Segmented results. (a) The tumor. (b) The breast without the tumor. (c) The background.
3954
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955
Fig. 3. Reconstruction of mammogram image. (a) A test image. (b) The reconstruction image.
of a mammogram image shown in Fig. 1(a). The tumor, the breast without the tumor, and the background are shown in Fig. 2(a)–(c), respectively. To evaluate the performance of proposed segmentation method, the standard Jaccard similarity indexes (Shattuck, Sandor-Leahy, Schaper, Rottenberg, & Leahy, 2001) is adopted. The index is calculated by comparing the region I segmented by improved watershed transform with region E outlined by the experts. Specifically, the Jaccard index J(I, E), which measures the similarity between these two regions, is defined as
I\E JðI; EÞ ¼ I[E
ð9Þ
The value of J(I, E) lies between 0 and 1. It would be 1 if regions I and E are completely overlapping, while when these two regions are entirely different, it would be 0. In addition, the Dice index is also widely used for comparing between two regions in the literature (Dice, 1945). There is a one-to-one correspondence between the two similarity indexes. More specifically, Dice index D(I, E) is defined as
DðI; EÞ ¼
2DðI; EÞ 1 þ DðI; EÞ
ð10Þ
The Dice index is also calculated for evaluating all the mammogram images provided from the MiniMIAS database. The value of D(I, E) is also situated between 0 and 1. If regions I and E are the same, it obtains 1, whereas it gets 0 when these two regions are disjoint. The average similarity measure with Jaccard and Dice indexes between the regions segmented by improved watershed transform and regions outlined by these two experts are listed in Table 1. The results indicate that the proposed method is promising in tumor segmentation. 3.2. Region compression After mammogram images are segmented into the tumor, the breast without the tumor, and the background, the vector quantization with CHNN is applied to all the separated parts with different compression rate according to their importance by simultaneously reserving important region information and reducing the size of mammogram images for more efficient storage or transmission. Fig. 3 shows the reconstruction of a mammogram image. A test image the same as Fig. 1(a) is shown in Fig. 3(a), while Fig. 3(b) shows the reconstruction result of Fig. 1(a). Since it is very important in the diagnoses of breast diseases, the tumor region is lossless compressed or not compressed. The breast without tumor is less
important than the tumor, so it is compressed with lower compression rate (bit rate = 0.6), whereas the background is almost useless in diagnosis, therefore it is compressed with higher compression rate (bit rate = 0.08). The peak signal-to-noise ratio (PSNR) of reconstructed image in Fig. 3(b) is 42.97. From the experimental results, the reconstructed mammogram images obtained from vector quantization with CHNN can provide much low bit rates and high PSNR in the applications of mammogram compression. 4. Conclusion We have proposed a region segmentation and compression method in this study. The mammogram images are segmented into tumor, breast without tumor and background automatically by means of improved watershed transform using prior information. The individual segmented regions are then applied to loss or lossless compression for the storage efficiency according to the importance of region data. More specifically, we detect the edge between the breast and background with canny edge detector. The improved watershed transform based on intrinsic prior information is then used to segment tumor boundary. The experimental results show that it is an effective method for tumor segmentation while comparing to those outlined by an expert. After segmentation, original mammogram images are separated into tumor, breast without tumor and background for further compression. The vector quantization with CHNN is applied to all the separated parts with different compression rate according their importance to simultaneously reserve important details and reduce the size of the storage or transmission. The experiment results also show the presented method can reconstruct very well in the applications of mammogram image compression. Acknowledgement The author would like to express his sincerely appreciation for grant under shared facilities supported by the Program of Top 100 Universities Advancement, Ministry of Education, Taiwan. References Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society B, 48, 259–302. Beucher, S., & Meyer, F. (1993). The morphological approach to segmentation: The watershed transform. In E. R. Dougherty (Ed.). Mathematical morphology in image processing (Vol. 12, pp. 433–481). New York: Marcel Dekker. Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 679–698.
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955 Chen, Y. C., Lien, H. P., & Tzeng, G. H. (2010). Measures and evaluation for environment watershed plans using a novel hybrid MCDM model. Expert Systems with Applications, 37(2), 926–938. Dice, L. R. (1945). Measures of the amount of ecologic association between species. Ecology, 26(3), 297–302. Dokur, Z. (2008). A unified framework for image compression and segmentation by using an incremental neural network. Expert Systems with Applications, 34(1), 611–619. Dokur, Z., & Ölmez, T. (2008). Tissue segmentation in ultrasound images by using genetic algorithms. Expert Systems with Applications, 34(4), 2739–2746. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741. Grau, V., Mewes, A. U. J., Alcañiz, M., Kikinis, R., & Warfield, S. K. (2004). Improved watershed transform for medical image segmentation using prior information. IEEE Transactions on Medical Imaging, 23(4), 447–458. Harlick, R. M., & Shapiro, L. G. (1985). Image segmentation techniques. CVGIP, 29, 100–132. Hopfield, J. J., & Tank, D. W. (1985). Neural computation of decisions in optimization problems. Biological Cybernetics, 52, 141–152. Hsu, W. Y. (2010). EEG-based motor imagery classification using neuro-fuzzy prediction and wavelet fractal features. Journal of Neuroscience Methods, 189(2), 295–302. Hsu, W. Y., Lin, C. C., Ju, M. S., & Sun, Y. N. (2007). Wavelet-based fractal features with active segment selection: Application to single-trial EEG data. Journal of Neuroscience Methods, 163(1), 145–160. Hsu, W. Y., Poon, W. F. P., & Sun, Y. N. (2008). Automatic seamless mosaicing of microscopic images: Enhancing appearance with color degradation
3955
compensation and wavelet-based blending. Journal of Microscopy-Oxford, 231(3), 408–418. Hsu, W. Y., & Sun, Y. N. (2009). EEG-based motor imagery analysis using weighted wavelet transform features. Journal of Neuroscience Methods, 167(2), 310–318. Lai, C. C., & Chang, C. Y. (2009). A hierarchical evolutionary algorithm for automatic medical image segmentation. Expert Systems with Applications, 36(1), 248–259. Lin, Y. C., Tsai, Y. P., Hung, Y. P., & Shih, Z. C. (2006). Comparison between immersion-based and toboggan-based watershed image segmentation. IEEE Transactions on Medical Imaging, 15(3), 632–640. Moga, A. N., & Gabbouj, M. (1997). Parallel image component labeling with watershed transformation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 441–450. Pal, N. R., & Pal, S. K. (1993). A review on image segmentation techniques. Pattern Recognition, 26, 1277–1294. Sahba, F., Tizhoosh, H. R., & Salama, M. M. M. A. (2008). A reinforcement agent for object segmentation in ultrasound images. Expert Systems with Applications, 35(3), 772–780. Shattuck, D. W., Sandor-Leahy, S. R., Schaper, L. A., Rottenberg, D. A., & Leahy, R. M. (2001). Magnetic resonance image tissue classification using a partial volume model. NeuroImage, 13, 856–876. Vincent, L., & Soille, P. (1991). Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6), 583–593. Wang, J., & Zhou, Y. (2009). Stochastic optimal competitive Hopfield network for partitional clustering. Expert Systems with Applications, 36(2), 2072–2080. Weickert, J. (1998). Anisotropic diffusion in image processing. Stuttgart, Germany: Teubner-Verlag.
Contents lists available at SciVerse ScienceDirect
Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa
Improved watershed transform for tumor segmentation: Application to mammogram image compression Wei-Yen Hsu Graduate Institute of Biomedical Informatics, Taipei Medical University, 250 Wu-Xin Street, Taipei 110, Taiwan
a r t i c l e
i n f o
Keywords: Image segmentation Mammogram Tumor Improved watershed transform Vector quantization Competitive Hopfield neural network (CHNN)
a b s t r a c t In this study, an automatic image segmentation method is proposed for the tumor segmentation from mammogram images by means of improved watershed transform using prior information. The segmented results of individual regions are then applied to perform a loss and lossless compression for the storage efficiency according to the importance of region data. These are mainly performed in two procedures, including region segmentation and region compression. In the first procedure, the canny edge detector is used to detect the edge between the background and breast. An improved watershed transform based on intrinsic prior information is then adopted to extract tumor boundary. Finally, the mammograms are segmented into tumor, breast without tumor and background. In the second procedure, vector quantization (VQ) with competitive Hopfield neural network (CHNN) is applied on the three regions with different compression rates according to the importance of region data so as to simultaneously reserve important tumor features and reduce the size of mammograms for storage efficiency. Experimental results show that the proposed method gives promising results in the compression applications. Ó 2011 Elsevier Ltd. All rights reserved.
1. Introduction Breast cancer is the most common one for women in worldwide. The women disease incidence rate that the breast cancer leaps to first poses the quite big threat to the domestic women. The X-ray mammography is an effective method to achieve the goal of early diagnoses. Accordingly, the accurate segmentation of tumor in mammogram images in very important. Image segmentation plays an important role and is an essential process in medical images. General segmentation is the process of partitioning the image into disjointed regions so as to the characteristics of each region are homogeneous (Harlick & Shapiro, 1985). A large variety of image segmentation methods have been presented (Dokur, 2008; Dokur & Ölmez, 2008; Hsu, 2010; Hsu, Poon, & Sun, 2008; Lai & Chang, 2009; Lin, Tsai, Hung, & Shih, 2006; Pal & Pal, 1993; Sahba, Tizhoosh, & Salama, 2008). Among these methods, the watershed (Beucher & Meyer, 1993) that is a region-based approach is a traditional but popular method from mathematical morphology. The region-based approaches group similar pixels into a region based on some pixel information. The advantages of watershed approach are that it is fast, it can be parallelized (Moga & Gabbouj, 1997), and it produces a complete division of image even if its contrast is low. Many techniques related to the watersheds have been presented in recent decades
E-mail addresses: [email protected], [email protected] 0957-4174/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2011.08.148
(Chen, Lien, & Tzeng, 2010; Hsu, Lin, Ju, & Sun, 2007; Vincent & Soille, 1991). The main drawback of watershed algorithm is that it produces over-segmented results. That is, when the watershed obtains catchment basins from the gradient of image, the results of watershed contain too many small regions. Moreover, it is sensitive to noise. Local variations of the image can significantly change the results. It results from the use of high pass filters to estimate the gradient. In addition, it is poor detection in significant areas with low contrast boundaries. If the signal to noise ratio is not high enough at the contour, the watershed transform will be unable to detect it accurately. Accordingly, the improved version of watershed algorithm may be overcome the intrinsic problems. In addition, various kinds of preprocessing have been developed to solve the problems of over-segmentation, such as the median filter and anisotropic diffusion filter (Weickert, 1998). In this study, an automatic image segmentation method based on improved watershed transform using prior information is proposed for the tumor segmentation from mammogram images. Moreover, for medical applications, we should be very cautious to retain sufficient image information in supporting different diagnosis purposes during the compression. A common approach is to perform different compression rates, including loss and lossless, for different degrees of important regions. Hence, the segmented results are separated original mammogram image into the tumor, the breast without tumor, and background. Since these two regions, the breast without tumor and background, are not important in diagnosis, the vector quantization with competitive Hopfield
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955
neural network (CHNN) is applied for loss compression. The Hopfield neural network (Hopfield & Tank, 1985) is a well-known technique used for solving optimization problems based on the Lyapunov energy function. In the study, the competitive learning rule for CHNN is modified to address the codebook design. The CHNN is constructed as a two-dimensional fully interconnected array with the rows representing the training vectors and columns standing for the codevectors in the codebook. This paper is organized as follows: Section 2 presents the segment and compression methods. It describes the results and discussion in Section 3. A conclusion is given in Section 4.
3951
to calculate the watershed transform. The lower neighbors are directly derived from the lower slope. That is, for each image pixel p, lower neighbors LN(p) are denoted,
f ðpÞ f ðp0 Þ 0 LNðpÞ ¼ p 2 NðpÞ 0 dðp; p Þ
ð2Þ
2. Methods
The set of lower neighbors is the subset of neighboring pixels for which the directed gradient to the pixel p equals its lower slope. In practical applications, the watershed transform is not calculated directly on the image, but rather on the absolute value of its gradient, which has high values at the contours. Gradient estimation at the center of the pixels reduces the original resolution of images. Thus, the lower slope is redefined as
2.1. Canny edge detection
LSðpÞ ¼ max
Edge detection means to extract sharp content information from the image. It plays an important role in feature-based image analysis. The most important topic in edge detection is how to extract representative features from the image in low contrast. Furthermore, three criteria, including detection (important edges should not be missed and there should be no spurious responses), localization (the distance between the actual and located position of the edge should be minimal) and one response (minimizing multiple responses to a single edge) are used to judge if the edge detector is optimal or not. Hence, the Canny edge detector (Canny, 1986) is adopted to coincide with the above-mentioned three criteria owing to its good localization and elastic thresholding. In this study, the Canny edge detector is used to detect the edge between the background and breast. In addition, the parameter r, the Gaussian scaling, of the Canny edge detector decides the amount and the fineness of extracted edges. In other words, there are fewer but more significant edges extracted when the image is convoluted with larger scaling r; while convoluting with a smaller scaling r, there are more and detailed edge responses as well as better edge localization. We detect edge locations from the acquired image after convolution. In other words, we use the Gaussian scaling r to adjust the amount and the fineness of edges. In addition to the scaling r, the other two parameters, high and low thresholds, of the Canny edge detector are also taken into account. If an edge response is over the high threshold, it is considered as a definite edge for a particular scaling. Individual weak responses usually coincide with noise except when they are neighbor to the pixels with strong responses, and they are likely to be true edges in the acquired image. These points are regarded as true edge points as long as their edge responses are over the low threshold. Both of them are set according to the estimated signal-to-noise ratio. Summarily speaking, edges are detected by adjusting these three parameters according to the characteristics and quality of images.
where jf ðpÞ f ðqÞj is calculated for the link (p, q). We have available prior information in the absolute or relative intensities of images. To take advantage of prior information, a set of lower cost functions for each object is used
jf ðpÞ f ðqÞj q2NðpÞ[p dðp; qÞ
2.2. Improved watershed transform An improved watershed transform based on intrinsic prior information (Grau, Mewes, Alcañiz, Kikinis, & Warfield, 2004) is adopted to extract tumor boundary from the breast. We first introduce important definitions of watershed transform about the lower slope and lower neighbors. Let f be a gray image. The lower slope of f at a pixel p, LS(p) is defined,
f ðpÞ f ðqÞ q2NðpÞ[p dðp; qÞ
LSðpÞ ¼ max
ð1Þ
where N(p) is the set of neighbors of p, and d(p, q) is the Euclidean distance between pixels p and q. To define a steepest slope relation between pixels is necessary for the lower slope, which will be used
fe ðp; qÞ q2NðpÞ[p dðp; qÞ
LSe ðpÞ ¼ max
ð3Þ
ð4Þ
where fe ðp; qÞ is to quantify the probability of having an edge between the pixels p and q. To calculate the functions, it is assumed that the label of pixel p is known before labeling pixel q. This is reached if a region-growing algorithm is used for watershed calculation. The improved watershed transform produces a possibility for different applications, depending on the amount of knowledge available on the objects. A function is used to measure the difference in class probability between two neighboring pixels for generic image segmentation. Normal distributions are assumed for the objects in the image, for which mean and variance are calculated using a set of seed pixels for each class. The seeds are selected using automatic techniques. Posterior probabilities for each class i at each pixel p are calculated using Bayes’ rule as
PðIp jiÞPðiÞ PðijIp Þ ¼ P i PðI p jiÞPðiÞ
ð5Þ
where Ip is the intensity of the image at pixel p. The structures exhibit a significant spatial homogeneity, and so it is desirable to achieve this behavior in the obtained probability values. Markov random fields (Besag, 1986; Geman & Geman, 1984) provide a way to model local correlations between pixels. To estimate the model, iterative conditional modes (Besag, 1986) are used to iteratively solve. 2.3. Vector quantization with competitive Hopfield neural network The Hopfield neural network with simple architecture and parallel potential has been applied in many fields (Hsu & Sun, 2009; Wang & Zhou, 2009). It consists of a single layer of processing elements where each neuron is connected to every other neuron in the network. The Hopfield neural network is a well-known technique used for solving optimization problems based on the Lyapunov energy function. In this study, we use a discrete competitive Hopfield neural network (CHNN), where the winner-takes-all method is adopted to learn weighting factors in the energy function, for vector quantization. That is, the CHNN is used to quantize the vectors from segmented regions, and compress and reconstruct the regions. Suppose an image is divided into n blocks (vectors of pixels) and each block occupies ‘ ‘ pixels. A vector quantizer is a technique that maps the Euclidean ‘ ‘-dimensional space R‘‘ into a set
3952
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955
fxj ; j ¼ 1; 2; . . . ; cg of points in R‘‘ , called a codebook. It looks for a codebook such that each training vector is approximated as close as possible by one of the code vectors in the codebook. A codebook is optimal if the average distortion is at the minimum value. The average distortion E½dðxy ; xj Þ between an input sequence of training vectors fxy ; y ¼ 1; 2; . . . ; ng and its corresponding output sequence of code vectors fxj ; j ¼ 1; 2; . . . ; cg is defined as n 1X D ¼ E½dðxy ; xj Þ ¼ dðxy ; xj Þ n y¼1
ð6Þ
To update the training performance, the simplified object function for the competitive Hopfield neural network (CHNN) based on competitive learning can be modified as
E¼
n X n X c X c X x¼1 y¼1 i¼1
V x;i W x;i;y;j V y;j
ð7Þ
j¼1
and
Netx;i ¼
n X c X
W x;i;y;j V y;j
ð8Þ
y¼1 j¼1
Fig. 1. Procedure of proposed segmentation method. (a) A test image. (b) The edge between the background and breast detected by Canny edge detector. (c) The tumor extracted by improved watershed transform. (d) and (e) The tumor outlined by experts 1 and 2, respectively.
3953
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955
where W x;i;y;j is the weight between neurons (x, i) and (y, j), and Ix;i ¼ 0.Using Eqs. (7) and (8), the Hopfield neural network based on competitive learning can be used for vector quantization in a parallel manner are given as follows: Step 1: Input a set training vector X ¼ fx1 ; x2 ; . . . ; xn g, and the number of class c. c Step 2: Compute the weight matrix W ¼ fwx;i;y;j gn; y¼1;j¼1 with training samples and codevectors. Step 3: Set the initial number of the vectors to be n. Each class contains at least one vector. Step 4: Calculate the input to each neuron (x, i) by Eq. (8). Step 5: Apply below equation to update the neurons’ output states for each neuron in a row
V x;i ¼
1 if Net x;i ¼ maxfNet x;1 ; Net x;2 ; . . . Netx;c g 0 otherwise
Step 6: Repeat Steps 4 and 5 for all rows, and count the number of neurons for the new state. If no neuron is changed go to Step 7, otherwise go to Step 4. Step 7: Complete the codebook design.
features from images in low contrast. More specifically, the Canny edge detector can achieve the criteria of detection, localization and on response due to its good localization and elastic thresholding. In addition, to extract tumor boundary from the breast, an improved watershed transform based on intrinsic prior information is applied. The Bayes’ rule is used to calculate posterior probabilities of each pixel for each class. The local correlations between pixels are then modeled by Markov random fields. To demonstrate the results of proposed segmentation method, the public database Mammographic Image Analysis Society MiniMammographic Database (MiniMIAS) is adopted in the experiments. The size of all the images in MiniMIAS database has been reduced to 1024 1024 pixels. The procedure of proposed segmentation method is illustrated in Fig. 1. A test image is shown in Fig. 1(a). Fig. 1(b) shows the edge between the breast and background detected by Canny edge detector, whereas the tumor extracted by improved watershed transform is shown in Fig. 1(c). Fig. 1(d) and 1(e) shows the tumor outlined by experts 1 and 2, respectively. In addition, Fig. 2 illustrates the segmented results
Table 1 Average similarity measure with Jaccard and Dice indexes between the regions segmented by improved watershed transform and regions outlined by two experts.
3. Results and discussion 3.1. Region segmentation The boundary between the background and breast is detected by Canny edge detector. This is because it can extract important
Average similarity measure
Jaccard index J
Dice index D
vs. Expert 1 vs. Expert 2
0.937 0.892
0.968 0.943
Fig. 2. Segmented results. (a) The tumor. (b) The breast without the tumor. (c) The background.
3954
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955
Fig. 3. Reconstruction of mammogram image. (a) A test image. (b) The reconstruction image.
of a mammogram image shown in Fig. 1(a). The tumor, the breast without the tumor, and the background are shown in Fig. 2(a)–(c), respectively. To evaluate the performance of proposed segmentation method, the standard Jaccard similarity indexes (Shattuck, Sandor-Leahy, Schaper, Rottenberg, & Leahy, 2001) is adopted. The index is calculated by comparing the region I segmented by improved watershed transform with region E outlined by the experts. Specifically, the Jaccard index J(I, E), which measures the similarity between these two regions, is defined as
I\E JðI; EÞ ¼ I[E
ð9Þ
The value of J(I, E) lies between 0 and 1. It would be 1 if regions I and E are completely overlapping, while when these two regions are entirely different, it would be 0. In addition, the Dice index is also widely used for comparing between two regions in the literature (Dice, 1945). There is a one-to-one correspondence between the two similarity indexes. More specifically, Dice index D(I, E) is defined as
DðI; EÞ ¼
2DðI; EÞ 1 þ DðI; EÞ
ð10Þ
The Dice index is also calculated for evaluating all the mammogram images provided from the MiniMIAS database. The value of D(I, E) is also situated between 0 and 1. If regions I and E are the same, it obtains 1, whereas it gets 0 when these two regions are disjoint. The average similarity measure with Jaccard and Dice indexes between the regions segmented by improved watershed transform and regions outlined by these two experts are listed in Table 1. The results indicate that the proposed method is promising in tumor segmentation. 3.2. Region compression After mammogram images are segmented into the tumor, the breast without the tumor, and the background, the vector quantization with CHNN is applied to all the separated parts with different compression rate according to their importance by simultaneously reserving important region information and reducing the size of mammogram images for more efficient storage or transmission. Fig. 3 shows the reconstruction of a mammogram image. A test image the same as Fig. 1(a) is shown in Fig. 3(a), while Fig. 3(b) shows the reconstruction result of Fig. 1(a). Since it is very important in the diagnoses of breast diseases, the tumor region is lossless compressed or not compressed. The breast without tumor is less
important than the tumor, so it is compressed with lower compression rate (bit rate = 0.6), whereas the background is almost useless in diagnosis, therefore it is compressed with higher compression rate (bit rate = 0.08). The peak signal-to-noise ratio (PSNR) of reconstructed image in Fig. 3(b) is 42.97. From the experimental results, the reconstructed mammogram images obtained from vector quantization with CHNN can provide much low bit rates and high PSNR in the applications of mammogram compression. 4. Conclusion We have proposed a region segmentation and compression method in this study. The mammogram images are segmented into tumor, breast without tumor and background automatically by means of improved watershed transform using prior information. The individual segmented regions are then applied to loss or lossless compression for the storage efficiency according to the importance of region data. More specifically, we detect the edge between the breast and background with canny edge detector. The improved watershed transform based on intrinsic prior information is then used to segment tumor boundary. The experimental results show that it is an effective method for tumor segmentation while comparing to those outlined by an expert. After segmentation, original mammogram images are separated into tumor, breast without tumor and background for further compression. The vector quantization with CHNN is applied to all the separated parts with different compression rate according their importance to simultaneously reserve important details and reduce the size of the storage or transmission. The experiment results also show the presented method can reconstruct very well in the applications of mammogram image compression. Acknowledgement The author would like to express his sincerely appreciation for grant under shared facilities supported by the Program of Top 100 Universities Advancement, Ministry of Education, Taiwan. References Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society B, 48, 259–302. Beucher, S., & Meyer, F. (1993). The morphological approach to segmentation: The watershed transform. In E. R. Dougherty (Ed.). Mathematical morphology in image processing (Vol. 12, pp. 433–481). New York: Marcel Dekker. Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 679–698.
W.-Y. Hsu / Expert Systems with Applications 39 (2012) 3950–3955 Chen, Y. C., Lien, H. P., & Tzeng, G. H. (2010). Measures and evaluation for environment watershed plans using a novel hybrid MCDM model. Expert Systems with Applications, 37(2), 926–938. Dice, L. R. (1945). Measures of the amount of ecologic association between species. Ecology, 26(3), 297–302. Dokur, Z. (2008). A unified framework for image compression and segmentation by using an incremental neural network. Expert Systems with Applications, 34(1), 611–619. Dokur, Z., & Ölmez, T. (2008). Tissue segmentation in ultrasound images by using genetic algorithms. Expert Systems with Applications, 34(4), 2739–2746. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741. Grau, V., Mewes, A. U. J., Alcañiz, M., Kikinis, R., & Warfield, S. K. (2004). Improved watershed transform for medical image segmentation using prior information. IEEE Transactions on Medical Imaging, 23(4), 447–458. Harlick, R. M., & Shapiro, L. G. (1985). Image segmentation techniques. CVGIP, 29, 100–132. Hopfield, J. J., & Tank, D. W. (1985). Neural computation of decisions in optimization problems. Biological Cybernetics, 52, 141–152. Hsu, W. Y. (2010). EEG-based motor imagery classification using neuro-fuzzy prediction and wavelet fractal features. Journal of Neuroscience Methods, 189(2), 295–302. Hsu, W. Y., Lin, C. C., Ju, M. S., & Sun, Y. N. (2007). Wavelet-based fractal features with active segment selection: Application to single-trial EEG data. Journal of Neuroscience Methods, 163(1), 145–160. Hsu, W. Y., Poon, W. F. P., & Sun, Y. N. (2008). Automatic seamless mosaicing of microscopic images: Enhancing appearance with color degradation
3955
compensation and wavelet-based blending. Journal of Microscopy-Oxford, 231(3), 408–418. Hsu, W. Y., & Sun, Y. N. (2009). EEG-based motor imagery analysis using weighted wavelet transform features. Journal of Neuroscience Methods, 167(2), 310–318. Lai, C. C., & Chang, C. Y. (2009). A hierarchical evolutionary algorithm for automatic medical image segmentation. Expert Systems with Applications, 36(1), 248–259. Lin, Y. C., Tsai, Y. P., Hung, Y. P., & Shih, Z. C. (2006). Comparison between immersion-based and toboggan-based watershed image segmentation. IEEE Transactions on Medical Imaging, 15(3), 632–640. Moga, A. N., & Gabbouj, M. (1997). Parallel image component labeling with watershed transformation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 441–450. Pal, N. R., & Pal, S. K. (1993). A review on image segmentation techniques. Pattern Recognition, 26, 1277–1294. Sahba, F., Tizhoosh, H. R., & Salama, M. M. M. A. (2008). A reinforcement agent for object segmentation in ultrasound images. Expert Systems with Applications, 35(3), 772–780. Shattuck, D. W., Sandor-Leahy, S. R., Schaper, L. A., Rottenberg, D. A., & Leahy, R. M. (2001). Magnetic resonance image tissue classification using a partial volume model. NeuroImage, 13, 856–876. Vincent, L., & Soille, P. (1991). Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6), 583–593. Wang, J., & Zhou, Y. (2009). Stochastic optimal competitive Hopfield network for partitional clustering. Expert Systems with Applications, 36(2), 2072–2080. Weickert, J. (1998). Anisotropic diffusion in image processing. Stuttgart, Germany: Teubner-Verlag.