- Email: [email protected]
A hierarchical evolutionary algorithm for automatic medical image segmentation
Available online at www.sciencedirect.com
Expert Systems with Applications Expert Systems with Applications 36 (2009) 248–259 www.elsevier.com/locate/eswa
A hierarchical evolutionary algorithm for automatic medical image segmentation Chih-Chin Lai a, Chuan-Yu Chang b
b,*
a Department of Electrical Engineering, National University of Kaohsiung, Kaohsiung 81148, Taiwan Department of Computer and Communication Engineering National Yunlin University of Science & Technology, Yunlin 640, Taiwan
Abstract Image segmentation denotes a process of partitioning an image into distinct regions. A large variety of different segmentation approaches for images have been developed. Among them, the clustering methods have been extensively investigated and used. In this paper, a clustering based approach using a hierarchical evolutionary algorithm (HEA) is proposed for medical image segmentation. The HEA can be viewed as a variant of conventional genetic algorithms. By means of a hierarchical structure in the chromosome, the proposed approach can automatically classify the image into appropriate classes and avoid the difficulty of searching for the proper number of classes. The experimental results indicate that the proposed approach can produce more continuous and smoother segmentation results in comparison with four existing methods, competitive Hopfield neural networks (CHNN), dynamic thresholding, k-means, and fuzzy c-means methods. 2007 Elsevier Ltd. All rights reserved. Keywords: Image segmentation; Hierarchical evolutionary algorithm
1. Introduction Image processing covers various techniques that are applicable to a wide range of applications. Image processing can be viewed as a special form of two-dimensional signal processing used to uncover information about images. Among various image processing tasks, segmentation can be viewed as the first essential and important step of low level vision (Gonzalez & Woods, 1992). Image segmentation is a process by which an image is partitioned into non-intersecting regions. These regions have two properties: (1) homogeneity within a region, i.e., the texture or color in a region should be as similar as possible, and (2) heterogeneity between the regions, i.e., texture or color that in one region should be distinct from those in another region. A more formal definition of segmentation is given in the following way: Let I be
*
Corresponding author. Tel.: +886 5 5342601 4337. E-mail address: [email protected] (C.-Y. Chang).
0957-4174/$ - see front matter 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2007.09.003
an image and H be a homogeneity predicate defined over groups of connected pixels; then the image segmentation is a partition of I into a set of regions {R1, R2, . . . , Rn} such that n
[ Ri ¼ I
i¼1
with Ri \ Rj ¼ /; i–j;
H ðRi Þ ¼ true;
8i;
ð1Þ ð2Þ
and H ðRi [ Rj Þ ¼ false;
8Ri and Rj adjacent:
ð3Þ
A variety of approaches have been proposed for image segmentation (Chang & Chung, 2001; Chen & Zhang, 2004; Cheng, Lin, & Mao, 1996; Cheriet, Said, & Suen, 1998; Chuang, Tzeng, Chen, Wu, & Chen, 2006; Felzenszwalb & Huttenlocher, 2004; Grau, Mewes, Alcaniz, Kikinis, & Wardield, 2004; Karayiannis & Pai, 1999; Maulik & Banyopadhyay, 2000; Sammouda, Niki, & Nishitani, 1997; Sarkar, Yegnanarayana, & Khemani, 1997; Xu, Olman, & Uberbacher, 1998; Yan & Kassim, 2006; Coleman & Andrews, 1979; Singh, Patel, Khosla, & Kim,
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
1996; Pham & Prince, 1999; Ahmed, Yamany, Mohamed, Farag, & Moriarty, 2002; Shen, Sandham, Granat, Dempsey, & Patterson, 2003). Xu et al. (1998) summarized these methods into two categories: (1) boundary detection-based approaches, which try to search closed boundary contours for segmenting an image, and (2) region clustering-based approach, which group ‘‘similar’’ neighboring pixels into clusters. Cheriet et al. (1998) proposed a modified Otsu’s approach (Otsu, 1979) called recursive thresholding technique (dynamic thresholding) for image segmentation. Grau et al. (2004) proposed an improvement to the watershed transform that enables the introduction of prior information for medical image segmentation. Yan and Kassim (2006) used minimal path deformable models incorporated with statistical shape priors to extract organ contours. Karayiannis and Pai (1999) used a fuzzy algorithm for learning vector quantization in MRI segmentation. Fuzzy c-means clustering algorithms with spatial information were also proposed for medical image segmentation (Chen & Zhang (2004); Chuang et al., 2006). Recently, the Hopfield neural networks have been proposed as alternative approaches (Chang & Chung, 2001; Cheng et al., 1996; Sammouda et al., 1997). Among them, the segmentation using competitive Hopfield neural networks (CHNN) are formulated as a cost-function-minimization problem to perform gray level thresholding on the image histogram or the pixels’ gray levels arranged in a one-dimensional array (Chang & Chung, 2001; Cheng et al., 1996). Even though many segmentation methods have been presented, most of them are still limited in two respects: First, the number of classes is predetermined, which implies that users must identify the number of regions beforehand. Second, most of the proposed methods need some preprocessing to reduce or remove the noise. Recently, to rectify these limitations, Felzenszwalb and Huttenlocher (2004) proposed a graph-based image segmentation method, which obeys the properties of being neither too coarse nor too fine according to a particular region comparison function. In spirit of the segmentation property, an automatic hierarchical evolutionary based image segmentation approach is proposed in this paper. Unlike the conventional genetic algorithm (Goldberg, 1989), which uses a fixed or pre-defined chromosome and the phenotype structure, the hierarchical evolutionary algorithm (HEA) (Tang, Man, Kwong, & Liu, 1998) can relax these constraints. The intrinsic property of the HEA is its ability to code the parameters of the considered problem in a hierarchical structure. This particular property makes it a potential technology for automatic medical image segmentation. Although the concept of HEA was first proposed by Tang et al. (1998) for infinite-impulse-response (IIR) filter design, as far as we know, no one has applied the HEA method to image segmentation. The main objective of our contribution is to successfully employ HEA in medical image segmentation without considering any auxiliary or extra medical image information, such as contextual or
249
textual properties, in given medical images. On the other hand, when we apply this method, the number of clusters in the given image does not need to be known in advance. 2. The proposed approach 2.1. Image segmentation formulation Let us consider a gray level image I. In general, an image can be described by a two-dimensional function f(x, y), where (x, y) denotes the spatial coordinates and the value of f is the feature at (x, y), and f(x, y) 2 [0, L], the set of discrete levels of the feature value. An image segmentation problem can be formulated as follows. Find a partition {R1, R2, . . . , Rk}of I with each Rj being a connected region of I, such that min z ¼
k X X
2
½ RepðRi Þ f ðx; yÞ ;
ð4Þ
i¼1 ðx;yÞ2Ri
where Rep(Ri) denotes the representative gray level of some region Ri, and k is the number of regions. This formulation can capture the intuition of segmenting an image; however, it is difficult to solve due to two reasons: first, the proper number of regions is not known beforehand. Observing Eq. (4), the minimum z is obtained when k is equal to L. However, this is not a reasonable solution. According to Felzenszwalb and Huttenlocher (2004), the segmented regions are neither too fine nor too coarse. The second difficulty is the decision of the representative gray level implicitly requires considering all the possible partitions. The k-means algorithm, one of the most widely used methods, can be applied to the image segmentation problem; however, it suffers from a couple of drawbacks, the k-means algorithm is very sensitive to the presence of noise as well as the initial cluster centers, and to apply this method, the number of clusters in the given data set should be known in advance (Otsu, 1979; Sarkar et al., 1997). In order to tackle these problems, we developed a hierarchical evolutionary algorithm based approach which can be employed for automatically searching the number of regions as well as properly finding the representative gray level for each region. 2.2. Hierarchical evolutionary algorithm The hierarchical evolutionary algorithm can be viewed as a variant of conventional genetic algorithm (GA). Genetic algorithms (GAs) are randomized search and optimization techniques guided by the principles of evolution and natural genetics, and have a large amount of implicit parallelism. They provide near optimal solutions of an objective or fitness function in complex, large, and multimodal landscapes. In general, a GA contains a fixed-size population of potential solutions over the search space. These potential solutions of the search space are encoded as binary or floating-point strings, and called individuals
250
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
or chromosomes. The initial population can be created randomly or based on the problem-specific knowledge. In each iteration, called a generation, a new population is created based on a preceding one through the following three steps: (1) evaluation: each individual of the old population is evaluated using a fitness function and given a value to denote its merit, (2) selection: individuals with better fitness are selected to generate the next population, and (3) mating: genetic operators such as crossover and mutation are applied to the selected individuals to produce new individuals for the next generation. The above three steps are iterated for many generations until a satisfactory solution is found or a termination criterion is met. From the biological and medical perspectives, the genetic structure of a chromosome is formed by a number of gene variations, which are arranged in a hierarchical manner. In the light of this issue, Tang et al. (1998) proposed a hierarchical evolutionary algorithm (HEA) to simulate this phenomenon. In HEA, the chromosome consists of two types of genes – control genes and parametric genes. The purpose of control genes is to determine which parametric gene should be utilized and which one can be disabled during the evolution process of an HEA. Generally, control genes are coded as binary digits, while parametric genes can be coded as any type of data structure. If the value of a control gene is ‘‘1’’, then the associated parametric genes are activated, otherwise the associated parametric genes are disabled. Two examples of the hierarchical chromosome structure are shown in Fig. 1. Fig. 1a is a two-level gene structure that each control gene corresponds to one parametric gene. The genes 53.2, 34.7, and 68.2 are active, but 19.6 and 75.3 are disabled. Fig. 1b represents a three-level gene structure that contains two levels of control genes. The activation of the parametric genes is governed by the second-level control genes, which are governed by the first-level control genes. Thus, only gene 78.5 is enabled. Although the majority of work with GA is focused on fixed-length chromosomes, a substantial amount of research has been performed on variable-length chromosomes and other structures. To some extent, HEA can be
viewed as a concise representation of variable-length chromosome. However, by means of hierarchical structure, not only is no extra effort required for reconfiguring the usual genetic computations, but also the robustness and the complexity of parametric modeling are improved. Since the intrinsic property of the HEA is its ability to code the parameters of the problem considered in a hierarchical structure, this flexible representation has an advantage that it can simultaneously consider a few demand functions/ constraints if given in the problem. Therefore, it is more appropriate for solving the automatic clustering problem compared with conventional GA. 2.3. The proposed HEA approach Here, we propose an HEA-based approach to solve the medical image segmentation problem as shown in Fig. 2. When we apply the HEA to automatically solve an image segmentation problem, we must consider the following components: (1) a genetic representation of solutions to the problem, (2) a way to create the initial population of solutions, (3) an evaluation function that rates all candidate solutions according to their ‘‘fitness’’, (4) genetic operators that alter the genetic composition of children during reproduction. 2.3.1. Solution representation In the proposed approach, the chromosome is made up of binary digits (the total number of ‘‘1’’ implicitly represents the number of regions) as well as integer numbers (representing the representative gray levels). The number of control genes is decided by a soft estimate of the upper bound of the number of regions. An example of the hierarchical chromosome structure in our approach is illustrated as follows, where it represents a partition of an image with three regions, and the associated representative gray levels are 25, 113, and 178, respectively. 2control genes 3 parametric genes zfflffl}|fflffl{ zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{ chromosomei ¼ 4 10110 :: 25 69 113 178 2105
2.3.2. Initial population A GA requires a population of potential solutions to be initialized at the beginning of the GA process. In our approach, we randomly select a few gray levels from [0, L] as the initial parametric genes. As for the control genes, they are generated randomly from {0, 1}.
Fig. 1. Two examples of the chromosome of the HEA. (a) Two-level structure and (b) three-level structure.
2.3.3. Fitness function A fitness function is the survival arbiter for chromosomes. Since the objective of image segmentation is to make pixels in the same region similar to each other, the fitness function can be defined as Eq. (4). For each chromosome, the parametric genes are extracted, and then a
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Input an image
Initialize population
251
Hierarchical Evolutionary Algorithm
Fitness evaluation
Mutation
Crossover
Termination
No
Yes
Selection
The proper number of regions and The representative gray levels of regions
Segmented image
Fig. 2. The schematic flow chart of the proposed method.
segmented image is obtained by assigning the pixels to their corresponding regions. 2.3.4. Selection The selection operator determines which individuals are chosen for mating and how many offspring each selected chromosome produces. Here we adopt the tournament selection method (Deb, 2001) because the time complexity of it is low. The basic concept of the tournament method is as follows: Randomly select a positive number Ntour of chromosomes from the population and copy the best fitted item from them into an intermediate population. The process is repeated P times, and here P is the population size. The algorithm of tournament selection is shown below.
Algorithm: Tournament selection Input: Population P (size of P is Ppop), tournament size Ntour (a positive number) Output: Population after selection P 0 (size of P 0 is also Ppop)
2001) in the proposed approach. The uniform crossover is applied to the control genes as well as the parametric genes, simultaneously. Two chromosomes are randomly selected as parents from the current population. The crossover creates the offspring chromosome on a bitwise basis, copying each allele from each parent with a probability pi. The pi is a random real number uniformly distributed in the interval [0, 1]. Let P1 and P2 be two parents, and C1 and C2 are offspring chromosomes; the ith allele in each offspring is defined as C 1 ðiÞ ¼ P 1 ðiÞ
and
C 2 ðiÞ ¼ P 2 ðiÞ if pi P 0:5;
ð5Þ
C 1 ðiÞ ¼ P 2 ðiÞ
and
C 2 ðiÞ ¼ P 1 ðiÞ if pi < 0:5:
ð6Þ
An example of this operator is shown in Fig. 3. 2.3.6. Mutation The mutation operator is needed to explore new areas of the search space and helps the search procedure avoid sticking in local optima. Here we apply bit mutation to the control genes. This results in some bits in control genes of the children being reversed: ‘‘1’’ is changed to ‘‘0’’ and
begin for i 1 to Ppop do best fitted item among Ntour elements ranP0 domly selected from P; return P 0 end 2.3.5. Crossover The crossover operator randomly pairs chromosomes and swaps parts of their genetic information to produce new chromosomes. We use the uniform crossover (Deb,
Fig. 3. A uniform crossover used in the proposed approach.
252
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Fig. 4. Examples of the bit mutation process used in the proposed approach.
‘‘0’’ is changed to ‘‘1’’. Either of these cases will change the number of regions. In the former, the associated parametric genes are disabled, while in the latter, the associated parametric genes are activated and the gene values are modified as follows: ^ p ¼ p N ð0; rÞ p;
ð7Þ
where ^p is the mutated gene, p is the existing gene value, and N(0, r) is a normally distributed random number with mean 0 and standard deviation r. After some trial and error procedures, we find that when r is equal to 0.1, the HEA obtains the best performance; that is, it achieves the ideal result by subjective visual assessment of the final segmented image. The mutation operator works as shown in Fig. 4. The example shows the part of control genes in three chromosomes and random numbers generated for each gene. The gene changes its value when the random number test is passed. The random number that causes a gene to change is printed in bold face. 3. Experimental results To show the proposed approach has the capability of image segmentation and robustness to noise, because of space limitations, five cases of different modality medical images, including skull-based CT (Fig. 5a), abdominal MRI (Fig. 6a), brain MRI (Fig. 7a), knee MRI (Fig. 8a), and a computer generated phantom image (Fig. 9a) were tested. The proposed method was compared with dynamic thresholding (Otsu, 1979), k-means (Singh et al., 1996), fuzzy c-means (Pham & Prince, 1999), and CHNN (Cheng et al., 1996) methods. All the cases used for evaluating the proposed method were collected from National Cheng Kung University Hospital. The CT image was taken from the GE 9800 CT scanner, and the MRI image was taken from the Siemen’s Magnetom 63SPA, T2 weighted spinecho sequences. The image sizes of CT and MR images are 256 · 256 pixels, with each pixel the 256 gray level. It should be noted that though the values of parameters in GAs, such as maximum generation number and probabilities of genetic operator, always influence the performance of the algorithms, only a few tries are needed to specify the parameter values. Certainly, reasonable parameters ensure good results and give rise to quick convergence. The parameters used in the experiments are as follows. The generation number is 500, the population size is 20, the probabilities of crossover and mutation are 0.8 and 0.15, respectively, and the soft estimation of the upper bound of the number of clusters is 10.
Fig. 5a is a CT head image in which a number of anatomical structures can be observed. Fig. 5b is the segmented result by means of the dynamic thresholding method. Obviously, the multi-threshold of the dynamic thresholding method cannot effectively separate the skull from the image. Thus, the segmentation results are quite messy. Fig. 5c and d show the segmented results of k-means and fuzzy c-means, respectively. Due to the intensity of different components are separated far from each other. Thus, the segmentation results of k-means and fuzzy c-means are acceptable. However, the major concern of k-means and fuzzy c-means methods is the number of clustering is unknown in advance. Fig. 5e is the segmented result of CHNN method. From the result, we can see that, although CHNN has the ability to separate the skull, mandible, other tissues and background, the tiny features of the image result in many fragments and little holes in it. Fig. 5f shows the segmented result using our approach, and we can see that most of the fragments and little holes are removed. Thus, the proposed approach can obtain clearer and more complete segmentation images. Fig. 6a is an abdominal MR image in which many artifact-caused features can be found near the spine. The dynamic thresholding method fails to divide the liver, tissues and background from the abdominal MR image into three classes. The result is shown in Fig. 6b, from which we can see that using the dynamic thresholding method produces a number of disconnected fragments as a result of noise and other artifacts. The CHNN method is better than the dynamic thresholding method in that the former results in fewer fractions, as shown in Fig. 6e. However, a number of artifact-caused fractions still exist at the vertical orientation of spine. The result obtained from the kmeans, fuzzy c-means, and proposed HEA method are shown in Fig. 6c, d and f, in which the liver, lung, and heart are segmented, indicating that the HEA can limit the effect of artifacts on image segmentation. Figs. 7 and 8 show the segmentation results of brain and knee MR images. Obviously, the proposed method can segment medical images into regions more intuition for human percept than other four methods. It is notable, k-means and fuzzy c-means have better segmentation capabilities for CT images. Observing Figs. 7(c–d) and 8(c–d), we can find that the brain regions and the knee joint cannot be segmented completely by k-means and fuzzy c-means methods. This is because there are many artifacts and the contrast of the MR images is relatively lower than CT images. In addition, since the actual segmentation results are hard to obtain and there maybe have varied judgments by different
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
253
Fig. 5. Comparison of dynamic thresholding, CHNN and proposed segmentation methods on the CT head image. (a) The original head CT image, (b) the segmentation result of dynamic thresholding method, (c) the segmentation result of k-means, (d) the segmentation result of fuzzy c-mean, (e) the segmentation result of CHNN, and (f) the segmentation result of the proposed approach.
radiologists, it is very difficult to provide the quality assessment of medical images. Thus, all the segmentation results on the real medical images were evaluated by two experienced radiologists from Kaohsiung Veterans General Hospital and National Cheng Kung University Hospital. In order to illustrate the segmentation performance of the proposed method and provide an objective evaluation, a computer generated phantom image (Fig. 9a) was tested.
This image was made up of seven overlapping ellipses. Each ellipse represents one structural area of tissue. From the periphery to the center, they were tissue 1 (T1, gray level = 30), tissue 2 (T2, gray level = 120), tissue 3 (T3, gray level = 165), tissue 4 (T4, gray level = 75), and tissue 5 (T5, gray level = 210). The gray levels for each tissue were set to a constant value. A 5 · 5 low-pass filter was applied to the original phantom image so as to obtain more
254
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Fig. 6. Comparison of dynamic thresholding, CHNN and proposed segmentation methods applied to the abdominal MR image: (a) the original abdominal MR image, (b) the segmentation result of dynamic thresholding method, (c) the segmentation result of k-means, (d) the segmentation result of fuzzy c-mean, (e) the segmentation result of CHNN, and (f) the segmentation result of the proposed approach.
blurred boundaries. In addition, noise of uniform distribution with the gray levels ranging from N to N was then added to this simulated blurred phantom. In our experiment, the noise for different cases was set to N = 30 and N = 45, respectively. The resulting images for the simulated noisy image (N = 45) are shown in Fig. 9. The accuracy for the dynamic thresholding, CHNN, k-means, fuzzy c-means and our approach are listed in Tables 1 and 2. The misclassified rate is defined as
misclassified rate ¼
jA Dj 100%; A
ð18Þ
where A is the number of the actual pixels, and D is the number of the segmented pixel. As for the average PK error, it is obtained from the equation 1=K i¼1 ðmisclassified rateÞi , here K = 5. From Fig. 9 it can be seen that the boundaries of tissues will be misclassified seriously in dynamic
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
255
Fig. 7. Comparison of dynamic thresholding, CHNN and proposed segmentation methods applied to the brain MR image: (a) the original brain MR image, (b) the segmentation result of dynamic thresholding method, (c) the segmentation result of k-means, (d) the segmentation result of fuzzy c-mean, (e) the segmentation result of CHNN, and (f) the segmentation result of the proposed approach.
thresholding, CHNN, k-means and fuzzy c-means methods for noise level N = 45. On the other hand, the average error rates of the proposed method with noise level N = 30 and N = 45 were 0.16 and 2.65, respectively. Both the average error rates were smaller than
the other methods, indicating that the proposed approach obtained more correct segmentation results than the other methods. To evaluate the performance of the proposed method, we also adopt the criteria suggested in Penedo, Carreria,
256
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Fig. 8. Comparison of dynamic thresholding, CHNN and proposed segmentation methods applied to the knee MR image: (a) the original kneel MR image, (b) the segmentation result of dynamic thresholding method, (c) the segmentation result of k-means, (d) the segmentation result of fuzzy c-mean, (e) the segmentation result of CHNN, and (f) the segmentation result of the proposed approach.
Mosquera, and Cabello (1998) to define sensitivity and specificity for performance evaluation. Let Np be the total pixel number of positive regions of interest (ROI) and Nn denote the total pixel number of negative ROI. We also define Ntp be the pixel number of detected regions which contain ROI and are actually detected, and Nfp to be the
pixel number of regions which contain no ROI but were falsely detected. Similarity, the true negative number (Ntn) and false negative number (Nfn) can be defined by Ntn = Nn Nfp and Nfn = Np Ntp, respectively. According to Penedo et al. (1998), we can further define sensitivity and specificity by:
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
257
Fig. 9. Comparison of different segmentation methods applied to the simulated image with added noise (N = 45). (a) Original phantom image; (b) blurred phantom image; (c) blurred noisy phantom image; (d) the segmentation result of dynamic thresholding method; (e) the segmentation result of CHNN; (f) the segmentation result of the k-means method; (g) the segmentation result of the fuzzy c-means method and (h) the segmentation result of the proposed approach. Table 1 The segmentation performance for five methods using the simulated image N = 30 Actual pixels
T5 T4 T3 T2 T1
3282 8745 9040 5590 38,879
D-thresholding
CHNN
k-Means
Fuzzy c-means
Our approach
Segmented Misclassified Segmented Misclassified Segmented Misclassified Segmented Misclassified Segmented Misclassified pixels rate (%) pixels rate (%) pixels rate (%) pixels rate (%) pixels rate (%) 2106 9045 9004 7624 37,857
Average error
35.83 3.43 0.40 34.60 2.6 15.38
2566 8810 9293 6631 38,236
21.82 0.70 2.80 18.62 1.65
2594 8905 9158 6649 38,230
9.13
20.96 1.83 1.31 19.94 1.67 8.94
Sensitivity ¼
N tp Np
ð19Þ
Specificity ¼
N tn Nn
ð20Þ
The sensitivity and specificity of the simulated image (N = 45) for the five methods are shown in Table 3. Obvi-
2594 8872 9158 6732 38,180
20.96 1.45 1.31 20.43 1.8 9.19
3290 8767 9029 5597 38,853
0.24 0.25 0.12 0.13 0.06 0.16
ously, the average sensitivity and specificity of the proposed method are 0.993 and 0.970, respectively, which are higher than dynamic thresholding, CHNN, k-means and fuzzy c-means. This means that the proposed method can segment medical images into regions more accuracy than dynamic thresholding, CHNN, k-means and fuzzy c-means methods. Although the sensitivity of the T5 of
258
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Table 2 The segmentation performance for five methods using the simulated image N = 45 Actual pixels
T5 T4 T3 T2 T1
3282 8745 9040 5590 38,879
D-thresholding
CHNN
k-Means
Fuzzy c-means
Our approach
Segmented Misclassified Segmented Misclassified Segmented Misclassified Segmented Misclassified Segmented Misclassified pixels rate (%) pixels rate (%) pixels rate (%) pixels rate (%) pixels rate (%) 1917 11069 9015 7866 35,669
Average error
41.59 26.58 0.28 40.72 8.26
2846 9056 9595 6858 37,181
23.48
13.28 3.56 6.14 22.68 4.37
2742 9276 8848 6782 37,888
10.10
16.45 6.07 2.12 21.32 2.55
2742 9539 9103 6843 37,309
9.7
16.45 9.08 0.7 22.42 4.04
3373 8893 9429 5727 38,114
2.84 1.69 4.30 2.45 1.97
10.54
2.65
Table 3 The sensitivity and specificity for five methods using the simulated image N = 45 Simulated object
D-thresholding
CHNN
Fuzzy c-means
Our approach
Sensitivity
Specificity
Sensitivity
Specificity
Sensitivity
k-Means Specificity
Sensitivity
Specificity
Sensitivity
Specificity
T5 T4 T3 T2 T1
1 0.970 0.980 0.957 0.999
0.690 0.852 0.882 0.893 0.974
0.999 0.976 0.984 0.968 0.999
0.812 0.862 0.933 0.850 0.983
0.997 0.966 0.983 0.962 0.996
0.815 0.851 0.884 0.827 0.972
0.997 0.959 0.981 0.962 0.998
0.815 0.837 0.899 0.835 0.958
0.996 0.982 0.996 0.994 0.996
0.986 0.963 0.963 0.964 0.976
Average
0.981
0.858
0.985
0.888
0.981
0.870
0.979
0.869
0.993
0.970
the dynamic thresholding has the perfect value of one, the corresponding specificity value is 0.69. This means that the problem of false positives is a serious one. Despite that the proposed method is more robustness than other methods; however, compared with traditional methods, the proposed method is time-consuming. 4. Conclusion In this paper, a hierarchical evolutionary algorithm based approach was proposed to solve the automatic medical image segmentation problem. The proposed segmentation approach can automatically determine the proper number of regions. On the other hand, the representative gray levels of regions are also determined, and then a partitioning of the given image is done. The hierarchical evolutionary algorithm is a conventional genetic algorithm with hierarchical genetic structure. By means of the hierarchical genetic structure, not only the basic search capability is maintained, but also the flexibility and the complexity of parametric modeling are improved. The utility of the proposed approach is demonstrated for a skull-based CT, abdominal MRI and a phantom-based image. From the experimental results, the proposed approach can actually find the appropriate number of regions as well as proper segmenting of an image. The segmentation results are more continued and smoother than dynamic thresholding, CHNN methods, k-means and fuzzy c-means. These results are useful to doctors for recognizing organs and tissues correctly, thus enhancing their diagnostic efficiency and minimizing their workload in medical image analysis.
Acknowledgements This work is partially supported by the National Science Council, Taiwan, R.O.C. under grants NSC 95-2221-E024-015 and NSC 95-2212-E-224-059. The author would like to thank the three anonymous referees and Associate Editor for their valuable comments and suggestions on earlier versions of this paper. References Ahmed, M. N., Yamany, S. M., Mohamed, N., Farag, A. A., & Moriarty, T. (2002). A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI data. IEEE Transactions on Medical Imaging, 21, 193–199. Chang, C.-Y., & Chung, P.-C. (2001). Medical image segmentation using a contextual-constraint-based Hopfield neural cube. Image and Vision Computing, 19, 669–678. Chen, S., & Zhang, D. (2004). Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure. IEEE Transactions on Systems Man and Cybernetics B, 34, 1907–1916. Cheng, K. S., Lin, J. S., & Mao, C. W. (1996). The application of competitive Hopfield Neural Network to medical image segmentation. IEEE Transactions on Medical Imaging, 15, 560–567. Cheriet, M., Said, J. N., & Suen, C. Y. (1998). A recursive thresholding technique for image segmentation. IEEE Transactions on Image Processing, 7, 918–921. Chuang, K.-S., Tzeng, H.-L., Chen, S., Wu, J., & Chen, T.-J. (2006). Fuzzy c-means clustering with spatial information for image segmentation. Computerized Medical Imaging and Graphics, 30, 9–15. Coleman, G. B., & Andrews, H. C. (1979). Image segmentation by clustering. Proceedings oF IEEE, 67, 773–785. Deb, K. (2001). Multi-objective optimization using evolutionary algorithms. Chichester, England: John Wiley & Sons, Ltd.
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259 Felzenszwalb, P. F., & Huttenlocher, D. P. (2004). Efficient graph-based image segmentation. International Journal of Computer Vision, 59, 1–26. Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. New York: Addison-Wesley. Gonzalez, R. C., & Woods, R. E. (1992). Digital image processing. Reading, MA: Addison-Wesley. Grau, V., Mewes, A. U. J., Alcaniz, M., Kikinis, R., & Wardield, S. K. (2004). Improved watershed transform for medical image segmentation using prior information. IEEE Transactions on Medical Imaging, 23, 447–458. Karayiannis, N. B., & Pai, P.-I. (1999). Segmentation of magnetic resonance image using fuzzy algorithms for learning vector quantization. IEEE Transactions on Medical Imaging, 18, 172–180. Maulik, U., & Banyopadhyay, S. (2000). Genetic algorithm-based clustering technique. Pattern Recognition, 33, 1455–1465. Otsu, N. (1979). A threshold selection method from gray level histograms. IEEE Transactions on Systems Man and Cybernetics B, 9, 62–66. Penedo, M. G., Carreria, M. J., Mosquera, A., & Cabello, D. (1998). Computer-aided diagnosis: A neural-network-based approach to lung nodule detection. IEEE Transactions on Medical Imaging, 17, 872–880. Pham, D. L., & Prince, J. L. (1999). Adaptive fuzzy segmentation of magnetic resonance images. IEEE Transactions on Medical Imaging, 18, 737–752.
259
Sammouda, R., Niki, N., & Nishitani, H. (1997). Hopfield neural network for the multichannel segmentation of magnetic resonance cerebral images. Pattern Recognition, 30, 921–927. Sarkar, M., Yegnanarayana, B., & Khemani, D. (1997). A clustering algorithm using an evolutionary programming-based approach. Pattern Recognition Letters, 18, 975–986. Shen, S., Sandham, W. A., Granat, M. H., Dempsey, M. F., & Patterson, J. (2003). Fuzzy clustering based applications to medical image segmentation. In Proceedings of the 25th annual international conference of the IEEE engineering in medicine and biology society, Vol. 1. (pp. 747–750). Singh, M., Patel, P., Khosla, D., & Kim, T. (1996). Segmentation of functional MRI by K-means clustering. IEEE Transactions on Nuclear Science, 43, 2030–2036. Tang, K. S., Man, K., Kwong, F. S., & Liu, Z. F. (1998). Design and optimization of IIR filter structure using hierarchical genetic algorithms. IEEE Transactions on Industrical Electronics, 45, 481–487. Xu, Y., Olman, V., & Uberbacher, E. C. (1998). A segmentation algorithm for noisy images: Design and evaluation. Pattern Recognition Letters, 19, 1213–1224. Yan, P., & Kassim, A. A. (2006). Medical image segmentation using minimal path deformable models with implicit shape priors. IEEE Transactions on Information Technology B, 10, 677–684.
Expert Systems with Applications Expert Systems with Applications 36 (2009) 248–259 www.elsevier.com/locate/eswa
A hierarchical evolutionary algorithm for automatic medical image segmentation Chih-Chin Lai a, Chuan-Yu Chang b
b,*
a Department of Electrical Engineering, National University of Kaohsiung, Kaohsiung 81148, Taiwan Department of Computer and Communication Engineering National Yunlin University of Science & Technology, Yunlin 640, Taiwan
Abstract Image segmentation denotes a process of partitioning an image into distinct regions. A large variety of different segmentation approaches for images have been developed. Among them, the clustering methods have been extensively investigated and used. In this paper, a clustering based approach using a hierarchical evolutionary algorithm (HEA) is proposed for medical image segmentation. The HEA can be viewed as a variant of conventional genetic algorithms. By means of a hierarchical structure in the chromosome, the proposed approach can automatically classify the image into appropriate classes and avoid the difficulty of searching for the proper number of classes. The experimental results indicate that the proposed approach can produce more continuous and smoother segmentation results in comparison with four existing methods, competitive Hopfield neural networks (CHNN), dynamic thresholding, k-means, and fuzzy c-means methods. 2007 Elsevier Ltd. All rights reserved. Keywords: Image segmentation; Hierarchical evolutionary algorithm
1. Introduction Image processing covers various techniques that are applicable to a wide range of applications. Image processing can be viewed as a special form of two-dimensional signal processing used to uncover information about images. Among various image processing tasks, segmentation can be viewed as the first essential and important step of low level vision (Gonzalez & Woods, 1992). Image segmentation is a process by which an image is partitioned into non-intersecting regions. These regions have two properties: (1) homogeneity within a region, i.e., the texture or color in a region should be as similar as possible, and (2) heterogeneity between the regions, i.e., texture or color that in one region should be distinct from those in another region. A more formal definition of segmentation is given in the following way: Let I be
*
Corresponding author. Tel.: +886 5 5342601 4337. E-mail address: [email protected] (C.-Y. Chang).
0957-4174/$ - see front matter 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2007.09.003
an image and H be a homogeneity predicate defined over groups of connected pixels; then the image segmentation is a partition of I into a set of regions {R1, R2, . . . , Rn} such that n
[ Ri ¼ I
i¼1
with Ri \ Rj ¼ /; i–j;
H ðRi Þ ¼ true;
8i;
ð1Þ ð2Þ
and H ðRi [ Rj Þ ¼ false;
8Ri and Rj adjacent:
ð3Þ
A variety of approaches have been proposed for image segmentation (Chang & Chung, 2001; Chen & Zhang, 2004; Cheng, Lin, & Mao, 1996; Cheriet, Said, & Suen, 1998; Chuang, Tzeng, Chen, Wu, & Chen, 2006; Felzenszwalb & Huttenlocher, 2004; Grau, Mewes, Alcaniz, Kikinis, & Wardield, 2004; Karayiannis & Pai, 1999; Maulik & Banyopadhyay, 2000; Sammouda, Niki, & Nishitani, 1997; Sarkar, Yegnanarayana, & Khemani, 1997; Xu, Olman, & Uberbacher, 1998; Yan & Kassim, 2006; Coleman & Andrews, 1979; Singh, Patel, Khosla, & Kim,
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
1996; Pham & Prince, 1999; Ahmed, Yamany, Mohamed, Farag, & Moriarty, 2002; Shen, Sandham, Granat, Dempsey, & Patterson, 2003). Xu et al. (1998) summarized these methods into two categories: (1) boundary detection-based approaches, which try to search closed boundary contours for segmenting an image, and (2) region clustering-based approach, which group ‘‘similar’’ neighboring pixels into clusters. Cheriet et al. (1998) proposed a modified Otsu’s approach (Otsu, 1979) called recursive thresholding technique (dynamic thresholding) for image segmentation. Grau et al. (2004) proposed an improvement to the watershed transform that enables the introduction of prior information for medical image segmentation. Yan and Kassim (2006) used minimal path deformable models incorporated with statistical shape priors to extract organ contours. Karayiannis and Pai (1999) used a fuzzy algorithm for learning vector quantization in MRI segmentation. Fuzzy c-means clustering algorithms with spatial information were also proposed for medical image segmentation (Chen & Zhang (2004); Chuang et al., 2006). Recently, the Hopfield neural networks have been proposed as alternative approaches (Chang & Chung, 2001; Cheng et al., 1996; Sammouda et al., 1997). Among them, the segmentation using competitive Hopfield neural networks (CHNN) are formulated as a cost-function-minimization problem to perform gray level thresholding on the image histogram or the pixels’ gray levels arranged in a one-dimensional array (Chang & Chung, 2001; Cheng et al., 1996). Even though many segmentation methods have been presented, most of them are still limited in two respects: First, the number of classes is predetermined, which implies that users must identify the number of regions beforehand. Second, most of the proposed methods need some preprocessing to reduce or remove the noise. Recently, to rectify these limitations, Felzenszwalb and Huttenlocher (2004) proposed a graph-based image segmentation method, which obeys the properties of being neither too coarse nor too fine according to a particular region comparison function. In spirit of the segmentation property, an automatic hierarchical evolutionary based image segmentation approach is proposed in this paper. Unlike the conventional genetic algorithm (Goldberg, 1989), which uses a fixed or pre-defined chromosome and the phenotype structure, the hierarchical evolutionary algorithm (HEA) (Tang, Man, Kwong, & Liu, 1998) can relax these constraints. The intrinsic property of the HEA is its ability to code the parameters of the considered problem in a hierarchical structure. This particular property makes it a potential technology for automatic medical image segmentation. Although the concept of HEA was first proposed by Tang et al. (1998) for infinite-impulse-response (IIR) filter design, as far as we know, no one has applied the HEA method to image segmentation. The main objective of our contribution is to successfully employ HEA in medical image segmentation without considering any auxiliary or extra medical image information, such as contextual or
249
textual properties, in given medical images. On the other hand, when we apply this method, the number of clusters in the given image does not need to be known in advance. 2. The proposed approach 2.1. Image segmentation formulation Let us consider a gray level image I. In general, an image can be described by a two-dimensional function f(x, y), where (x, y) denotes the spatial coordinates and the value of f is the feature at (x, y), and f(x, y) 2 [0, L], the set of discrete levels of the feature value. An image segmentation problem can be formulated as follows. Find a partition {R1, R2, . . . , Rk}of I with each Rj being a connected region of I, such that min z ¼
k X X
2
½ RepðRi Þ f ðx; yÞ ;
ð4Þ
i¼1 ðx;yÞ2Ri
where Rep(Ri) denotes the representative gray level of some region Ri, and k is the number of regions. This formulation can capture the intuition of segmenting an image; however, it is difficult to solve due to two reasons: first, the proper number of regions is not known beforehand. Observing Eq. (4), the minimum z is obtained when k is equal to L. However, this is not a reasonable solution. According to Felzenszwalb and Huttenlocher (2004), the segmented regions are neither too fine nor too coarse. The second difficulty is the decision of the representative gray level implicitly requires considering all the possible partitions. The k-means algorithm, one of the most widely used methods, can be applied to the image segmentation problem; however, it suffers from a couple of drawbacks, the k-means algorithm is very sensitive to the presence of noise as well as the initial cluster centers, and to apply this method, the number of clusters in the given data set should be known in advance (Otsu, 1979; Sarkar et al., 1997). In order to tackle these problems, we developed a hierarchical evolutionary algorithm based approach which can be employed for automatically searching the number of regions as well as properly finding the representative gray level for each region. 2.2. Hierarchical evolutionary algorithm The hierarchical evolutionary algorithm can be viewed as a variant of conventional genetic algorithm (GA). Genetic algorithms (GAs) are randomized search and optimization techniques guided by the principles of evolution and natural genetics, and have a large amount of implicit parallelism. They provide near optimal solutions of an objective or fitness function in complex, large, and multimodal landscapes. In general, a GA contains a fixed-size population of potential solutions over the search space. These potential solutions of the search space are encoded as binary or floating-point strings, and called individuals
250
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
or chromosomes. The initial population can be created randomly or based on the problem-specific knowledge. In each iteration, called a generation, a new population is created based on a preceding one through the following three steps: (1) evaluation: each individual of the old population is evaluated using a fitness function and given a value to denote its merit, (2) selection: individuals with better fitness are selected to generate the next population, and (3) mating: genetic operators such as crossover and mutation are applied to the selected individuals to produce new individuals for the next generation. The above three steps are iterated for many generations until a satisfactory solution is found or a termination criterion is met. From the biological and medical perspectives, the genetic structure of a chromosome is formed by a number of gene variations, which are arranged in a hierarchical manner. In the light of this issue, Tang et al. (1998) proposed a hierarchical evolutionary algorithm (HEA) to simulate this phenomenon. In HEA, the chromosome consists of two types of genes – control genes and parametric genes. The purpose of control genes is to determine which parametric gene should be utilized and which one can be disabled during the evolution process of an HEA. Generally, control genes are coded as binary digits, while parametric genes can be coded as any type of data structure. If the value of a control gene is ‘‘1’’, then the associated parametric genes are activated, otherwise the associated parametric genes are disabled. Two examples of the hierarchical chromosome structure are shown in Fig. 1. Fig. 1a is a two-level gene structure that each control gene corresponds to one parametric gene. The genes 53.2, 34.7, and 68.2 are active, but 19.6 and 75.3 are disabled. Fig. 1b represents a three-level gene structure that contains two levels of control genes. The activation of the parametric genes is governed by the second-level control genes, which are governed by the first-level control genes. Thus, only gene 78.5 is enabled. Although the majority of work with GA is focused on fixed-length chromosomes, a substantial amount of research has been performed on variable-length chromosomes and other structures. To some extent, HEA can be
viewed as a concise representation of variable-length chromosome. However, by means of hierarchical structure, not only is no extra effort required for reconfiguring the usual genetic computations, but also the robustness and the complexity of parametric modeling are improved. Since the intrinsic property of the HEA is its ability to code the parameters of the problem considered in a hierarchical structure, this flexible representation has an advantage that it can simultaneously consider a few demand functions/ constraints if given in the problem. Therefore, it is more appropriate for solving the automatic clustering problem compared with conventional GA. 2.3. The proposed HEA approach Here, we propose an HEA-based approach to solve the medical image segmentation problem as shown in Fig. 2. When we apply the HEA to automatically solve an image segmentation problem, we must consider the following components: (1) a genetic representation of solutions to the problem, (2) a way to create the initial population of solutions, (3) an evaluation function that rates all candidate solutions according to their ‘‘fitness’’, (4) genetic operators that alter the genetic composition of children during reproduction. 2.3.1. Solution representation In the proposed approach, the chromosome is made up of binary digits (the total number of ‘‘1’’ implicitly represents the number of regions) as well as integer numbers (representing the representative gray levels). The number of control genes is decided by a soft estimate of the upper bound of the number of regions. An example of the hierarchical chromosome structure in our approach is illustrated as follows, where it represents a partition of an image with three regions, and the associated representative gray levels are 25, 113, and 178, respectively. 2control genes 3 parametric genes zfflffl}|fflffl{ zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{ chromosomei ¼ 4 10110 :: 25 69 113 178 2105
2.3.2. Initial population A GA requires a population of potential solutions to be initialized at the beginning of the GA process. In our approach, we randomly select a few gray levels from [0, L] as the initial parametric genes. As for the control genes, they are generated randomly from {0, 1}.
Fig. 1. Two examples of the chromosome of the HEA. (a) Two-level structure and (b) three-level structure.
2.3.3. Fitness function A fitness function is the survival arbiter for chromosomes. Since the objective of image segmentation is to make pixels in the same region similar to each other, the fitness function can be defined as Eq. (4). For each chromosome, the parametric genes are extracted, and then a
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Input an image
Initialize population
251
Hierarchical Evolutionary Algorithm
Fitness evaluation
Mutation
Crossover
Termination
No
Yes
Selection
The proper number of regions and The representative gray levels of regions
Segmented image
Fig. 2. The schematic flow chart of the proposed method.
segmented image is obtained by assigning the pixels to their corresponding regions. 2.3.4. Selection The selection operator determines which individuals are chosen for mating and how many offspring each selected chromosome produces. Here we adopt the tournament selection method (Deb, 2001) because the time complexity of it is low. The basic concept of the tournament method is as follows: Randomly select a positive number Ntour of chromosomes from the population and copy the best fitted item from them into an intermediate population. The process is repeated P times, and here P is the population size. The algorithm of tournament selection is shown below.
Algorithm: Tournament selection Input: Population P (size of P is Ppop), tournament size Ntour (a positive number) Output: Population after selection P 0 (size of P 0 is also Ppop)
2001) in the proposed approach. The uniform crossover is applied to the control genes as well as the parametric genes, simultaneously. Two chromosomes are randomly selected as parents from the current population. The crossover creates the offspring chromosome on a bitwise basis, copying each allele from each parent with a probability pi. The pi is a random real number uniformly distributed in the interval [0, 1]. Let P1 and P2 be two parents, and C1 and C2 are offspring chromosomes; the ith allele in each offspring is defined as C 1 ðiÞ ¼ P 1 ðiÞ
and
C 2 ðiÞ ¼ P 2 ðiÞ if pi P 0:5;
ð5Þ
C 1 ðiÞ ¼ P 2 ðiÞ
and
C 2 ðiÞ ¼ P 1 ðiÞ if pi < 0:5:
ð6Þ
An example of this operator is shown in Fig. 3. 2.3.6. Mutation The mutation operator is needed to explore new areas of the search space and helps the search procedure avoid sticking in local optima. Here we apply bit mutation to the control genes. This results in some bits in control genes of the children being reversed: ‘‘1’’ is changed to ‘‘0’’ and
begin for i 1 to Ppop do best fitted item among Ntour elements ranP0 domly selected from P; return P 0 end 2.3.5. Crossover The crossover operator randomly pairs chromosomes and swaps parts of their genetic information to produce new chromosomes. We use the uniform crossover (Deb,
Fig. 3. A uniform crossover used in the proposed approach.
252
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Fig. 4. Examples of the bit mutation process used in the proposed approach.
‘‘0’’ is changed to ‘‘1’’. Either of these cases will change the number of regions. In the former, the associated parametric genes are disabled, while in the latter, the associated parametric genes are activated and the gene values are modified as follows: ^ p ¼ p N ð0; rÞ p;
ð7Þ
where ^p is the mutated gene, p is the existing gene value, and N(0, r) is a normally distributed random number with mean 0 and standard deviation r. After some trial and error procedures, we find that when r is equal to 0.1, the HEA obtains the best performance; that is, it achieves the ideal result by subjective visual assessment of the final segmented image. The mutation operator works as shown in Fig. 4. The example shows the part of control genes in three chromosomes and random numbers generated for each gene. The gene changes its value when the random number test is passed. The random number that causes a gene to change is printed in bold face. 3. Experimental results To show the proposed approach has the capability of image segmentation and robustness to noise, because of space limitations, five cases of different modality medical images, including skull-based CT (Fig. 5a), abdominal MRI (Fig. 6a), brain MRI (Fig. 7a), knee MRI (Fig. 8a), and a computer generated phantom image (Fig. 9a) were tested. The proposed method was compared with dynamic thresholding (Otsu, 1979), k-means (Singh et al., 1996), fuzzy c-means (Pham & Prince, 1999), and CHNN (Cheng et al., 1996) methods. All the cases used for evaluating the proposed method were collected from National Cheng Kung University Hospital. The CT image was taken from the GE 9800 CT scanner, and the MRI image was taken from the Siemen’s Magnetom 63SPA, T2 weighted spinecho sequences. The image sizes of CT and MR images are 256 · 256 pixels, with each pixel the 256 gray level. It should be noted that though the values of parameters in GAs, such as maximum generation number and probabilities of genetic operator, always influence the performance of the algorithms, only a few tries are needed to specify the parameter values. Certainly, reasonable parameters ensure good results and give rise to quick convergence. The parameters used in the experiments are as follows. The generation number is 500, the population size is 20, the probabilities of crossover and mutation are 0.8 and 0.15, respectively, and the soft estimation of the upper bound of the number of clusters is 10.
Fig. 5a is a CT head image in which a number of anatomical structures can be observed. Fig. 5b is the segmented result by means of the dynamic thresholding method. Obviously, the multi-threshold of the dynamic thresholding method cannot effectively separate the skull from the image. Thus, the segmentation results are quite messy. Fig. 5c and d show the segmented results of k-means and fuzzy c-means, respectively. Due to the intensity of different components are separated far from each other. Thus, the segmentation results of k-means and fuzzy c-means are acceptable. However, the major concern of k-means and fuzzy c-means methods is the number of clustering is unknown in advance. Fig. 5e is the segmented result of CHNN method. From the result, we can see that, although CHNN has the ability to separate the skull, mandible, other tissues and background, the tiny features of the image result in many fragments and little holes in it. Fig. 5f shows the segmented result using our approach, and we can see that most of the fragments and little holes are removed. Thus, the proposed approach can obtain clearer and more complete segmentation images. Fig. 6a is an abdominal MR image in which many artifact-caused features can be found near the spine. The dynamic thresholding method fails to divide the liver, tissues and background from the abdominal MR image into three classes. The result is shown in Fig. 6b, from which we can see that using the dynamic thresholding method produces a number of disconnected fragments as a result of noise and other artifacts. The CHNN method is better than the dynamic thresholding method in that the former results in fewer fractions, as shown in Fig. 6e. However, a number of artifact-caused fractions still exist at the vertical orientation of spine. The result obtained from the kmeans, fuzzy c-means, and proposed HEA method are shown in Fig. 6c, d and f, in which the liver, lung, and heart are segmented, indicating that the HEA can limit the effect of artifacts on image segmentation. Figs. 7 and 8 show the segmentation results of brain and knee MR images. Obviously, the proposed method can segment medical images into regions more intuition for human percept than other four methods. It is notable, k-means and fuzzy c-means have better segmentation capabilities for CT images. Observing Figs. 7(c–d) and 8(c–d), we can find that the brain regions and the knee joint cannot be segmented completely by k-means and fuzzy c-means methods. This is because there are many artifacts and the contrast of the MR images is relatively lower than CT images. In addition, since the actual segmentation results are hard to obtain and there maybe have varied judgments by different
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
253
Fig. 5. Comparison of dynamic thresholding, CHNN and proposed segmentation methods on the CT head image. (a) The original head CT image, (b) the segmentation result of dynamic thresholding method, (c) the segmentation result of k-means, (d) the segmentation result of fuzzy c-mean, (e) the segmentation result of CHNN, and (f) the segmentation result of the proposed approach.
radiologists, it is very difficult to provide the quality assessment of medical images. Thus, all the segmentation results on the real medical images were evaluated by two experienced radiologists from Kaohsiung Veterans General Hospital and National Cheng Kung University Hospital. In order to illustrate the segmentation performance of the proposed method and provide an objective evaluation, a computer generated phantom image (Fig. 9a) was tested.
This image was made up of seven overlapping ellipses. Each ellipse represents one structural area of tissue. From the periphery to the center, they were tissue 1 (T1, gray level = 30), tissue 2 (T2, gray level = 120), tissue 3 (T3, gray level = 165), tissue 4 (T4, gray level = 75), and tissue 5 (T5, gray level = 210). The gray levels for each tissue were set to a constant value. A 5 · 5 low-pass filter was applied to the original phantom image so as to obtain more
254
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Fig. 6. Comparison of dynamic thresholding, CHNN and proposed segmentation methods applied to the abdominal MR image: (a) the original abdominal MR image, (b) the segmentation result of dynamic thresholding method, (c) the segmentation result of k-means, (d) the segmentation result of fuzzy c-mean, (e) the segmentation result of CHNN, and (f) the segmentation result of the proposed approach.
blurred boundaries. In addition, noise of uniform distribution with the gray levels ranging from N to N was then added to this simulated blurred phantom. In our experiment, the noise for different cases was set to N = 30 and N = 45, respectively. The resulting images for the simulated noisy image (N = 45) are shown in Fig. 9. The accuracy for the dynamic thresholding, CHNN, k-means, fuzzy c-means and our approach are listed in Tables 1 and 2. The misclassified rate is defined as
misclassified rate ¼
jA Dj 100%; A
ð18Þ
where A is the number of the actual pixels, and D is the number of the segmented pixel. As for the average PK error, it is obtained from the equation 1=K i¼1 ðmisclassified rateÞi , here K = 5. From Fig. 9 it can be seen that the boundaries of tissues will be misclassified seriously in dynamic
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
255
Fig. 7. Comparison of dynamic thresholding, CHNN and proposed segmentation methods applied to the brain MR image: (a) the original brain MR image, (b) the segmentation result of dynamic thresholding method, (c) the segmentation result of k-means, (d) the segmentation result of fuzzy c-mean, (e) the segmentation result of CHNN, and (f) the segmentation result of the proposed approach.
thresholding, CHNN, k-means and fuzzy c-means methods for noise level N = 45. On the other hand, the average error rates of the proposed method with noise level N = 30 and N = 45 were 0.16 and 2.65, respectively. Both the average error rates were smaller than
the other methods, indicating that the proposed approach obtained more correct segmentation results than the other methods. To evaluate the performance of the proposed method, we also adopt the criteria suggested in Penedo, Carreria,
256
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Fig. 8. Comparison of dynamic thresholding, CHNN and proposed segmentation methods applied to the knee MR image: (a) the original kneel MR image, (b) the segmentation result of dynamic thresholding method, (c) the segmentation result of k-means, (d) the segmentation result of fuzzy c-mean, (e) the segmentation result of CHNN, and (f) the segmentation result of the proposed approach.
Mosquera, and Cabello (1998) to define sensitivity and specificity for performance evaluation. Let Np be the total pixel number of positive regions of interest (ROI) and Nn denote the total pixel number of negative ROI. We also define Ntp be the pixel number of detected regions which contain ROI and are actually detected, and Nfp to be the
pixel number of regions which contain no ROI but were falsely detected. Similarity, the true negative number (Ntn) and false negative number (Nfn) can be defined by Ntn = Nn Nfp and Nfn = Np Ntp, respectively. According to Penedo et al. (1998), we can further define sensitivity and specificity by:
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
257
Fig. 9. Comparison of different segmentation methods applied to the simulated image with added noise (N = 45). (a) Original phantom image; (b) blurred phantom image; (c) blurred noisy phantom image; (d) the segmentation result of dynamic thresholding method; (e) the segmentation result of CHNN; (f) the segmentation result of the k-means method; (g) the segmentation result of the fuzzy c-means method and (h) the segmentation result of the proposed approach. Table 1 The segmentation performance for five methods using the simulated image N = 30 Actual pixels
T5 T4 T3 T2 T1
3282 8745 9040 5590 38,879
D-thresholding
CHNN
k-Means
Fuzzy c-means
Our approach
Segmented Misclassified Segmented Misclassified Segmented Misclassified Segmented Misclassified Segmented Misclassified pixels rate (%) pixels rate (%) pixels rate (%) pixels rate (%) pixels rate (%) 2106 9045 9004 7624 37,857
Average error
35.83 3.43 0.40 34.60 2.6 15.38
2566 8810 9293 6631 38,236
21.82 0.70 2.80 18.62 1.65
2594 8905 9158 6649 38,230
9.13
20.96 1.83 1.31 19.94 1.67 8.94
Sensitivity ¼
N tp Np
ð19Þ
Specificity ¼
N tn Nn
ð20Þ
The sensitivity and specificity of the simulated image (N = 45) for the five methods are shown in Table 3. Obvi-
2594 8872 9158 6732 38,180
20.96 1.45 1.31 20.43 1.8 9.19
3290 8767 9029 5597 38,853
0.24 0.25 0.12 0.13 0.06 0.16
ously, the average sensitivity and specificity of the proposed method are 0.993 and 0.970, respectively, which are higher than dynamic thresholding, CHNN, k-means and fuzzy c-means. This means that the proposed method can segment medical images into regions more accuracy than dynamic thresholding, CHNN, k-means and fuzzy c-means methods. Although the sensitivity of the T5 of
258
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259
Table 2 The segmentation performance for five methods using the simulated image N = 45 Actual pixels
T5 T4 T3 T2 T1
3282 8745 9040 5590 38,879
D-thresholding
CHNN
k-Means
Fuzzy c-means
Our approach
Segmented Misclassified Segmented Misclassified Segmented Misclassified Segmented Misclassified Segmented Misclassified pixels rate (%) pixels rate (%) pixels rate (%) pixels rate (%) pixels rate (%) 1917 11069 9015 7866 35,669
Average error
41.59 26.58 0.28 40.72 8.26
2846 9056 9595 6858 37,181
23.48
13.28 3.56 6.14 22.68 4.37
2742 9276 8848 6782 37,888
10.10
16.45 6.07 2.12 21.32 2.55
2742 9539 9103 6843 37,309
9.7
16.45 9.08 0.7 22.42 4.04
3373 8893 9429 5727 38,114
2.84 1.69 4.30 2.45 1.97
10.54
2.65
Table 3 The sensitivity and specificity for five methods using the simulated image N = 45 Simulated object
D-thresholding
CHNN
Fuzzy c-means
Our approach
Sensitivity
Specificity
Sensitivity
Specificity
Sensitivity
k-Means Specificity
Sensitivity
Specificity
Sensitivity
Specificity
T5 T4 T3 T2 T1
1 0.970 0.980 0.957 0.999
0.690 0.852 0.882 0.893 0.974
0.999 0.976 0.984 0.968 0.999
0.812 0.862 0.933 0.850 0.983
0.997 0.966 0.983 0.962 0.996
0.815 0.851 0.884 0.827 0.972
0.997 0.959 0.981 0.962 0.998
0.815 0.837 0.899 0.835 0.958
0.996 0.982 0.996 0.994 0.996
0.986 0.963 0.963 0.964 0.976
Average
0.981
0.858
0.985
0.888
0.981
0.870
0.979
0.869
0.993
0.970
the dynamic thresholding has the perfect value of one, the corresponding specificity value is 0.69. This means that the problem of false positives is a serious one. Despite that the proposed method is more robustness than other methods; however, compared with traditional methods, the proposed method is time-consuming. 4. Conclusion In this paper, a hierarchical evolutionary algorithm based approach was proposed to solve the automatic medical image segmentation problem. The proposed segmentation approach can automatically determine the proper number of regions. On the other hand, the representative gray levels of regions are also determined, and then a partitioning of the given image is done. The hierarchical evolutionary algorithm is a conventional genetic algorithm with hierarchical genetic structure. By means of the hierarchical genetic structure, not only the basic search capability is maintained, but also the flexibility and the complexity of parametric modeling are improved. The utility of the proposed approach is demonstrated for a skull-based CT, abdominal MRI and a phantom-based image. From the experimental results, the proposed approach can actually find the appropriate number of regions as well as proper segmenting of an image. The segmentation results are more continued and smoother than dynamic thresholding, CHNN methods, k-means and fuzzy c-means. These results are useful to doctors for recognizing organs and tissues correctly, thus enhancing their diagnostic efficiency and minimizing their workload in medical image analysis.
Acknowledgements This work is partially supported by the National Science Council, Taiwan, R.O.C. under grants NSC 95-2221-E024-015 and NSC 95-2212-E-224-059. The author would like to thank the three anonymous referees and Associate Editor for their valuable comments and suggestions on earlier versions of this paper. References Ahmed, M. N., Yamany, S. M., Mohamed, N., Farag, A. A., & Moriarty, T. (2002). A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI data. IEEE Transactions on Medical Imaging, 21, 193–199. Chang, C.-Y., & Chung, P.-C. (2001). Medical image segmentation using a contextual-constraint-based Hopfield neural cube. Image and Vision Computing, 19, 669–678. Chen, S., & Zhang, D. (2004). Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure. IEEE Transactions on Systems Man and Cybernetics B, 34, 1907–1916. Cheng, K. S., Lin, J. S., & Mao, C. W. (1996). The application of competitive Hopfield Neural Network to medical image segmentation. IEEE Transactions on Medical Imaging, 15, 560–567. Cheriet, M., Said, J. N., & Suen, C. Y. (1998). A recursive thresholding technique for image segmentation. IEEE Transactions on Image Processing, 7, 918–921. Chuang, K.-S., Tzeng, H.-L., Chen, S., Wu, J., & Chen, T.-J. (2006). Fuzzy c-means clustering with spatial information for image segmentation. Computerized Medical Imaging and Graphics, 30, 9–15. Coleman, G. B., & Andrews, H. C. (1979). Image segmentation by clustering. Proceedings oF IEEE, 67, 773–785. Deb, K. (2001). Multi-objective optimization using evolutionary algorithms. Chichester, England: John Wiley & Sons, Ltd.
C.-C. Lai, C.-Y. Chang / Expert Systems with Applications 36 (2009) 248–259 Felzenszwalb, P. F., & Huttenlocher, D. P. (2004). Efficient graph-based image segmentation. International Journal of Computer Vision, 59, 1–26. Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. New York: Addison-Wesley. Gonzalez, R. C., & Woods, R. E. (1992). Digital image processing. Reading, MA: Addison-Wesley. Grau, V., Mewes, A. U. J., Alcaniz, M., Kikinis, R., & Wardield, S. K. (2004). Improved watershed transform for medical image segmentation using prior information. IEEE Transactions on Medical Imaging, 23, 447–458. Karayiannis, N. B., & Pai, P.-I. (1999). Segmentation of magnetic resonance image using fuzzy algorithms for learning vector quantization. IEEE Transactions on Medical Imaging, 18, 172–180. Maulik, U., & Banyopadhyay, S. (2000). Genetic algorithm-based clustering technique. Pattern Recognition, 33, 1455–1465. Otsu, N. (1979). A threshold selection method from gray level histograms. IEEE Transactions on Systems Man and Cybernetics B, 9, 62–66. Penedo, M. G., Carreria, M. J., Mosquera, A., & Cabello, D. (1998). Computer-aided diagnosis: A neural-network-based approach to lung nodule detection. IEEE Transactions on Medical Imaging, 17, 872–880. Pham, D. L., & Prince, J. L. (1999). Adaptive fuzzy segmentation of magnetic resonance images. IEEE Transactions on Medical Imaging, 18, 737–752.
259
Sammouda, R., Niki, N., & Nishitani, H. (1997). Hopfield neural network for the multichannel segmentation of magnetic resonance cerebral images. Pattern Recognition, 30, 921–927. Sarkar, M., Yegnanarayana, B., & Khemani, D. (1997). A clustering algorithm using an evolutionary programming-based approach. Pattern Recognition Letters, 18, 975–986. Shen, S., Sandham, W. A., Granat, M. H., Dempsey, M. F., & Patterson, J. (2003). Fuzzy clustering based applications to medical image segmentation. In Proceedings of the 25th annual international conference of the IEEE engineering in medicine and biology society, Vol. 1. (pp. 747–750). Singh, M., Patel, P., Khosla, D., & Kim, T. (1996). Segmentation of functional MRI by K-means clustering. IEEE Transactions on Nuclear Science, 43, 2030–2036. Tang, K. S., Man, K., Kwong, F. S., & Liu, Z. F. (1998). Design and optimization of IIR filter structure using hierarchical genetic algorithms. IEEE Transactions on Industrical Electronics, 45, 481–487. Xu, Y., Olman, V., & Uberbacher, E. C. (1998). A segmentation algorithm for noisy images: Design and evaluation. Pattern Recognition Letters, 19, 1213–1224. Yan, P., & Kassim, A. A. (2006). Medical image segmentation using minimal path deformable models with implicit shape priors. IEEE Transactions on Information Technology B, 10, 677–684.