- Email: [email protected]
Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation
G Model
ARTICLE IN PRESS
ASOC 2858 1–11
Applied Soft Computing xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Applied Soft Computing journal homepage: www.elsevier.com/locate/asoc
Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation
1
2
3
Q1
Ahmad Khan a,b , Muhammad Arfan Jaffar a,c,∗ a
4
b
5
c
6
National University of Computer and Emerging Sciences, H-11/4 Islamabad, Pakistan COMSATS Institute of Information Technology, Abbottabad, Pakistan College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia
7
a r t i c l e
8 24
i n f o
a b s t r a c t
9
Article history: Received 29 June 2011 Received in revised form 16 March 2015 Accepted 16 March 2015 Available online xxx
10 11 12 13 14 15
23
Keywords: Self organizing map (SOM) Segmentation Fuzzy Genetic algorithm (GA) Color image Centroid Segment
25
1. Introduction
16 17 Q3 18 19 20 21 22
The grouping of pixels based on some similarity criteria is called image segmentation. In this paper the problem of color image segmentation is considered as a clustering problem and a fixed length genetic algorithm (GA) is used to handle it. The effectiveness of GA depends on the objective function (fitness function) and the initialization of the population. A new objective function is proposed to evaluate the quality of the segmentation and the fitness of a chromosome. In fixed length genetic algorithm the chromosomes have same length, which is normally set by the user. Here, a self organizing map (SOM) is used to determine the number of segments in order to set the length of a chromosome automatically. An opposition based strategy is adopted for the initialization of the population in order to diversify the search process. In some cases the proposed method makes the small regions of an image as separate segments, which leads to noisy segmentation. A simple ad hoc mechanism is devised to refine the noisy segmentation. The qualitative and quantitative results show that the proposed method performs better than the state-of-the-art methods. © 2015 Elsevier B.V. All rights reserved.
The fundamental purpose of color image segmentation is to divide or partition a color image into a set of disjoint non27 overlapping regions (segments) such that each segment represents 28 29Q4 a particular object. Since last decades, abundance of techniques have been proposed to address the problem of an image segmen30 tation. Although, color image segmentation is still considered as a 31 challenging problem in computer vision research community. Par32 ticularly, the segmentation of colored natural images is the most 33 difficult job because of the random inhomogeneities that exist in 34 color and texture. Most of the approaches address the segmen35 tation of a particular type of an image for particular application. 36 Moreover, these approaches are not fully automatic and need a 37 significant amount of user intervention to segment the objects of 38 interest accurately. 39 In literature there are many color image segmentation tech40 niques which can be divided into [1] image-domain based, physics 41 26
∗ Corresponding author at: College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia. Q2 Tel.: +966 553605730. E-mail addresses: [email protected] (A. Khan), [email protected] (M.A. Jaffar).
based and feature-space based approaches. Image-domain based methods use both color and spatial features to carry out the segmentation task. The efficiency of these methods is depending on the selection of seed regions, which are normally set manually by the user. On the other hand the physics based methods use the physical models of material reflection to carry out color image segmentation [2]. It needs the reflection property of the underlying object which are not known in most of the cases and hence it is an application specific. The feature-based methods use color features only which leads to inaccurate results due to the lack of spatial relationship among the pixels [3]. Contour based segmentation algorithms can be found in literature [4–7]. It starts from an initial boundary in the form of a spline curve; the curve is iteratively evolved to minimize some energy function. It suffers from the manual initialization of the curve or boundary to the object of interest. It is trapped in local optima and leads to unsatisfactory results. Markov random field based on statistical models [8] uses spatial connectivity information among the neighboring pixels to perform segmentation. Graph cut [9–12] is another segmentation approach based upon Markov random fields. These algorithms solve a two class problem, i.e., separating the background and foreground objects. Watershed segmentation [13,14] has been successfully adopted for the two class segmentation problems. It often produces bad results for image having many regions.
http://dx.doi.org/10.1016/j.asoc.2015.03.029 1568-4946/© 2015 Elsevier B.V. All rights reserved.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66
G Model
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116
ASOC 2858 1–11
ARTICLE IN PRESS
2
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Image segmentation can be treated as a clustering problem. The feature of each pixel corresponds to a pattern and the combination of the pixels (i.e. segment) corresponds to a cluster. Keeping this analogy in mind, many clustering algorithms have been used for image segmentation [15–18]. Among them K-means [15] is the basic clustering algorithm adopted for color image segmentation. Similarly, fuzzy C-means (FCM) [16] has been used for image segmentation extensively. It is a clustering process which decrease intra clusters distances and increase inter clusters distances. However, its implementation faces three problems: (1) cluster initialization, (2) determining number of the clusters, and (3) lacking of the robust cluster validity index. The performance of the algorithm depends on the initialization of clusters centroids. If the cluster centroids represent the search space uniformly then the results will be better on the other hand the biased initialization will lead to poor performance. Similarly the number of clusters have considerable impact on the quality of segmentation. The clustering validity index (CVI) performs a vital role in the process of image segmentation and data clustering [19]. It checks whether the provided clustering or grouping of the given data reflects the original structure of the data or not. The cluster validity index (CVI) adopted for the FCM takes into account both compactness and separation for reliable segmentation. Note that if the first (i.e., compactness) is adopted as a cluster validly index then the best result would contain the cluster/segments number as that of the number of points present in the underlying search space. Similarly if the second (i.e., separation) is considered as a cluster validity index then the best result would contain only one segment/cluster. The CVI that combines compactness and separation is insufficient to represent the geometrical structure of the data because it normally finds the distances between the centroids of different clusters. In this paper, a fixed length fuzzy genetic algorithm (GA) is used to perform color image segmentation. A new objective function or clustering validity index (CVI) is proposed which incorporate the fuzzy compactness of data points and fuzzy separation of the segments centroids. In the new objective function, the compactness is defined term of fuzzy deviation of data points while the separation is based on the average variance of segments centroids. A self organizing map (SOM) is used to determine the optimal number of segments in order to set the length of a chromosome. In some cases the proposed method makes some small regions of an image as separate segments which leads to noisy segmentation. A simple merging technique is proposed to address this problem and make segmentation refine. All the operations are performed in the Lab color space, which is near to the human level of perception. The block diagram of the proposed method is given in Fig. 1. The rest of the paper is organized as follows. Section 2 presents a mechanism to set the length of a chromosome automatically. In Section 3 the GA based color image segmentation is discussed. Section 4 presents the detail of the merging algorithm to handle the
over-segmentation problem. Experimental results are discussed in Section 5 followed by the conclusion. 2. Determining length of the chromosome
118
119
We have used a fixed length genetic algorithm in which the length of a chromosome depends on the number of segments. Generally in segmentation algorithms the number of segments is set by user manually using some sort of domain knowledge. In this work a self organizing map (SOM) [20] is used to determine the number of segments that are hiddenly present in the underlying distribution (input image). SOM is an unsupervised type of artificial neural network (ANN) which is used to produce the low dimensional map of complex input space. Here we use a two-dimensional (2D) SOM network that consists of nodes or cells. Each node Ni of the SOM, where i ) having three elements. The i ∈ 16, is assigned a weight vector (w training dataset (in the form of input image) is organized as a vector V j in a raster scanning manner, where j = 1, . . ., N and N represents the total number of pixels in the input image. Note that each element of the V j (training set) consists of the three channels of the input image. The SOM network is randomly initialized. Then each input vector is iteratively feeded to the SOM network. The input vector is assigned to a best matching unit (BMU), a node in the SOM network which has maximum similarity to the given input vector. The best matching unit (BMU) to a given input vector V j is determined by using the following Eq. (1).
i , j ∈ [1, N] BMU = argmin V j − w
117
(1)
120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142
143
i∈[1,16]
where 16 is the number of nodes/cells in the SOM network. Here, 16 is to be considered the maximum number of segment s present in a particular image. The weights of NBMU (a node in the network which has maximum matching to the input vector V j ) and its neighboring nodes are updated using Eq. (2) i (t + 1) w
=
144 145 146 147 148
149
i (t)], i (t) + (t)(t)[V j (t) − w w
ifNBMU − Ni ≤ v(t)
i (t), w
ifNBMU − Ni > v(t)
(2)
150
151
In Eq. (2) t, and v(t) represent the iteration step and the neighborhood radius of the network respectively. Initially the neighborhood radius v0 is kept high to cover all the nodes of the network. Then the size of neighborhood radius v(t) reduces exponentially with the increase in number of iterations t, as given in Eq. (3).
v(t) = v0 · e−t/
(3)
Fig. 1. Block diagram of the proposed method.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
152 153 154 155 156
157
G Model
ARTICLE IN PRESS
ASOC 2858 1–11
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx 158 159 160 161 162 163 164 165 166 167 168 169 170
171
172 173 174 175 176 177
178
179 180 181 182 183 184
185
186 187 188
where t is the iteration number and is a constant usually set to the ratio between the initial radius and the maximum number of iterations. The weights that lie inside the neighborhood of BMU are updated. It is clear from Eq. (2) that the weight updating process of the nodes, that lies inside in the neighborhood of BMU, are controlled by two functions (t) and (t). The function (t) represents the learning rate which shows that what percent of change should be incorporated in the weight of a particular node. In the initial stages more changes are made to the weights of SOM network in order to speed up the convergence. However to ensure maximum exploitation, the effect of learning rate is reduced with the increase in number of iteration t. The learning rate function (t) is given by Eq. (4). (t) = 0 · e
−t/
(4)
where 0 is a constant value which may be fixed according to the requirement of learning process. The neighborhood function (t) controls the updating strength of the neighbor nodes of the best matching unit (BMU). It is implemented as a Gaussian function, in order to make changes according to proximity of neighbors, as given in Eq. (5). (t) = e
−
NBMU −Ni 2 2v2 (t)
(5)
The training of the SOM network continues until the radius v(t) reach the size of NBMU and the weights of the network are not changed for a few successive iterations. Using the procedure of [20], the number of segments are determined from the trained SOM network. For each pixel a mapping is performed to the final weights of the nodes using Eq. (6). g , g=arg min V j − w i , j ∈ [1, N] V j ← w i∈[1,16]
Then a variance map is formed based on the minimum cumulative distance among the weights of the SOM and pixels of input image. i , the variance is given as: For all pixels V j labeled with w
189
(6)
i) = variance(w
j∈Dw
i
3
m : mutation rate c : crossover rate Np : population size P: is a matrix used to represent the population. Note that, there are Np rows in P and each row represents a particular chromosome. aj : the maximum value in the jth channel of an image bj : the minimum value in the jth channel of an image : represents the logical OR operator Pi,g : is a row vector represents the ith chromosome at generation (iteration) g F i,g : is a scalar value which represents the fitness of the ith chromosome at generation g 3.1. Initialization A genetic algorithm needs to be initialized before starting the iterative procedure. Usually the initialization is performed by user using some sort of domain knowledge or random procedure. The pure random initialization is not good enough because it forces the segmentation algorithm to converge to local optima [22]. The initialization process can be diversified by opposite number generation. The opposite number diversify the initialization process and tends to select the initial solutions from the productive regions of the underlying search space. Here we used the opposition based strategy [23] to generate the initial population. The best individuals from the union of purely random solutions and their opposites are selected. This strategy avoids the unproductive samples or solutions and initialize the population to the most fertile regions. In Algorithm 1 lines 3–8, demonstrate the initialization process. The aj and bj (Algorithm 1, line 6) are the minimum and maximum values of a particular dimension, which ensure to select a value in the given range.
i) no pixels labeled(w
, i ∈ [1, 16]
A genetic algorithm (GA) is a population based probabilistic search technique used to solve the optimization problems. This section presents the details of genetic algorithm and its use in the area of color image segmentation. The basic mechanism of GA is outlined in Algorithm 1. The following are the terminologies used in the given algorithm.
Here a chromosome is represented as a series of real values to encode K segments centroids. Each segment centroid consists of three colored channel of an image. So a particular chromosome can be represented as a string of K × 3 real values. In each chromosome the first three values represent the centroid of first segment, the second three values represent the centroid of second segment and
195 196 197 198 199 200 201 202 203 204
207 208 209 210 211 212
218 219 220 221 222
223 224
225
226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242
244
3.2. A chromosome encoding
194
217
(7)
3. Genetic algorithm based color image segmentation
193
216
i V j − w
206
192
215
243
205
191
214
Algorithm 1.
where Dw i is the domain set of pixels V j that has maximum match i . The smaller variance (w i ) value represents a ing to the weights w more reliable estimate of a compact segment. In the last step the inter-segment variability is calculated to find out the final number of segments. A similarity matrix is constructed in order to store the Euclidian distances between the neighboring nodes in the SOM network. An elimination process is exploited to minimize the number of nodes/segment centroids. If the distance between two neighboring nodes is less than a pre-defined threshold ˇ, inter segment distance, then a node having highest variance score is removed. The elimination process continues until the distances among all the neighboring nodes remain greater than a pre-defined threshold value of ˇ. In this work the inter-segment threshold ˇ is set to 0.4 based on the empirical study of BSD images [21]. This value proved good enough for all images analyzed in the experimentation.
190
213
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
245 246 247 248 249 250
G Model
251 252 253
ASOC 2858 1–11
ARTICLE IN PRESS
4
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
so on. Suppose we have K = 3 then a particular chromosome can be represented as: C11
C12
C13
C21
C22
C23
C31
C32
C33
the separation determines that how much the given segments centroids are different. The smaller the compactness and the greater the separation leads to better segmentation. Here we determine the compactness in the following way:
254 255 256 257 258
259
260
The above illustration demonstrates three segment centroid s (C11 , C12 , C13 ), (C21 , C22 , C23 ), and (C31 , C32 , C33 ) respectively, where Cij represents the jth value of the ith segment centroid such that 1 ≤ i ≤ K, and, j ∈ {1, 2, 3}.
In fuzzy C-mean (FCM) algorithm [24] the membership ij of a data point xj to a segment centroid vi is given as:
262
ij =
(8)
D(vk ,xj )
280
281
sij =
264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279
ik
(9)
k∈(xj ) 282 283 284 285 286 287 288 289
where (xj ) denote a square window, in the spatial domain centered around a pixel xj . The spatial function sij represents the relationship among the neighboring pixels – that how much they are similar. If the majority of pixels, in the same neighborhood, belongs to one segment then the value of spatial function will be greater and vice versa. The spatial information can be included in the membership function by the following way: p q
290
ij =
ij sij
K
k=1 291 292 293 294 295 296 297 298
p q
(10)
kj skj
where p and q are parameters used to give relative weights to each function. Here we set the values of p and q to 1 and 2 respectively. The effect of p and q on the final segmentation is discussed in qualitative evaluation Section 5.1. In this paper we use the compactness and separation measures to develop a new objective function (fitness function) in order to asses the quality of solution/chromosome. The compactness determines that how much the points in one segment are similar while
300 301 302
(11)
303
where ij D(vi , xj ) is the weighed distance of a point xj to a segment
304
i=1
i=1
ij
K
is the fuzzy cardinality of the segment cencentroid vi and troid vi . Note that the sum of the weighted distances of a all points give the fuzzy deviation of the data. For the fuzzy separation we find the distances between each pair of segments centroids. If there are total K centroids, then K(K − 1)/2 distances can be calculated. So, the fuzzy separation of the segments centroids is given as:
2
where N and K represent the number of data points and the number of segments, D(vi , xj ) represents a distance between a cluster centroid vi and a point xj , m is a fuzzifier which determines the level of cluster fuzzyness. It is important to note that, when calculating the membership value ij in Eq. (8) if D(vi , xj ) becomes zero for any value of k then ij is assigned 0, ∀i = 1, / k, while the value of ij is set to 1. It means 2, . . ., K, i = that a particular point xj has complete matching with the segment centroid vi therefore the difference or distance between them becomes 0. Then the membership of this point to the corresponding segment centroid is set to maximum,i.e., 1 and the membership to the rest of the cluster centroids is set to minimum, i.e., 0. In images the neighboring pixels have maximum similarities. In other words, they have similar features or characteristics. So there is a maximum probability that they belong to the same segment. To exploit this idea in color image segmentation a spatial function is constructed as [25]:
263
ij D(vi , xj )
K
Mediani =/ k (D(vi , vk )) − Mini =/ k (D(vi , vk )) + Maxi =/ k (D(vi , vk )) − Mediani =/ k (D(vi , vk ))
, for1 ≤ i ≤ K, 1 ≤ j ≤ N 2 K D(vi ,xj ) m−1 k=1
i=1
261
1
K
i=1 ij
3.3. The proposed objective function
=
˝=
K
299
(12)
The compactness ‘˝’ should be minimized to get compact segments (i.e. to maximize the inter-segment similarities). On the contrary the fuzzy separation should be maximized in order to decrease the intra-segments similarities. Finally the fitness function is determined by Eq. (13). F=
˝
(13)
The chromosome/solution which has higher fitness value is considered the best. It is a maximization case so the aim is to maximize the fitness value. The chromosomes having highest fitness values are selected for the reproduction. 3.4. Selection In GA, selection is one of the most important operator. The main purpose of the selection is to select the best chromosomes for reproduction. To determine the quality of a solution/chromosome the fitness function is used. The chromosome which has best fitness should have higher probability for the selection. Here we use the binary tournament selection. 3.5. Reproduction Reproduction is the process to generate new solutions (chromosomes) from the current selected solutions. We carried out a single point crossover to reproduce new child solutions. The cut point for crossover is selected randomly. Note that, the crossover probability c is initialized to 0.8, while the mutation probability m is set to 0.05. If a chromosome is selected for mutation then the allele position, which is expose to the mutation process, is randomly selected. The allele value of that particular position is replaced by a randomly generated value from the respective component of an image. The parent and new offspring populations are exposed together for the selection of new generation in order to ensure the elitism operation. 3.6. Stoping criteria It a is condition to terminate the iterative execution of the GA. / true) as a stoping criteria. Here, G is the maxiWe use (g ≤ G ı = mum number of iterations and ı is a flag which is set to true if < (Algorithm 1, lines 20–23). Note that, here we set to 0.0001. Moreover, it each iteration g the is updated as (Algorithm 1, lines 20): ← F best,g − F best,g−1 , where the Fbest,g represents the fitness of the best chromosome at the current iteration while Fbest,g−1 represents the fitness of the best chromosome at previous iteration Table 1
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
305 306 307 308 309 310 311
312
313 314 315 316 317
318
319 320 321 322
323
324 325 326 327 328 329
330
331 332 333 334 335 336 337 338 339 340 341
342
343 344 345 346 347 348 349 350
G Model ASOC 2858 1–11
ARTICLE IN PRESS A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Table 1 The proposed method parameter setting
351
Parameter
Value/type
Crossover type Crossover cut point selection Mutation type Crossover rate (c ) Mutation rate (m ) Selection type Population size
Single point Random Random 0.8 0.05 Binary tournament 20
4. Elimination of the small segments
Q5
357
. In some cases the given genetic algorithm makes the small regions as separate segments, which leads to over-segmentation. A simple ad hoc mechanism is used to eliminate the small segments by merging the to their matching neighbors. The pseudocode of the elimination process is given in Algorithm 2.
358
Algorithm 2.
352 353 354 355 356
359 360 361 362 363 364 365 366 367 368 369 370 371 372
373
374 375 376 377 378 379 380 381 382 383 384 385
If the size of a segment x is less than a threshold then its neighboring segments are find out. In line 4, y ∈ NB(Lx ) represent that segment y is the neighbor of the segment x. Then the average distance among the segment y and segment x in the corresponding color image I is calculated. Note that, the argmin(·) returns the label of the best matching neighbor (BMN) which has the smallest distance to the target segment x. After finding the best matching neighbor (BMN) the two segments are merged by giving same label to their union in the corresponding newLp∪q . Fig. 2 shows the results of the elimination process. The first column shows the original images, while the second and third columns show the segmentation results before and after the elimination process. Note that, some small dots can be seen in Fig. 2 (column 2 before elimination) which are eliminated in column 2 by the elimination process. 5. Experimental results The experimentations are carried out on images taken from the Berkeley image segmentation database (BSD) [21]. The results of the proposed method are compared with seven well known image segmentation methods: statistical region merging (SRM) [26], segmentation via lossy data compression (LDC) [27], image segmentation in the presence of shadows and highlights using a ridge based histogram analysis (RADp) [2], multiregion image segmentation by parametric kernel graph cuts (KGC) [28], a level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI (LS) [29], efficient graph-based image segmentation (EG) [30], and normalized cuts and image segmentation (NC) [31].
5
The different parameters of the algorithms (used for comparison) are adjusted as suggested by the authors. For illustration, in (LDC) [27] the window size w and the texture difference metric ˛ are initialized to 7 and 2 respectively. In the case of EG [30], the parameter is set to 0.8, the threshold value k is initialized to 300, and the minimal region= 50 pixels. The number of regions in normalized cuts (NC) [31] is set to 20, which is the average number of segments in BSD. In the case of (SRM) [26] the segmentation parameter Q is given as: Q = 2 , where = 0, 1, 2, . . ., 8. The scale parameters in (LS) algorithm [29] are set to 4. For (KGC) [28] the weight of the smoothness constraint ˛ is initialized to 1 and the number of regions k to 20. The codes of all of the 7 algorithms are downloaded from their respective web sites. Due to the overflow of memory NC and LDC are unable to process an image of size (321 × 481) or (481 × 321), so for these algorithms all of the images are reduced to size 314 × 208 (or 208 × 314), and finally the segmented results are restored to their actual sizes. 5.1. Qualitative evaluation Effect of spatial information on final segmentation: Fig. 3 demonstrates the effect of p and q (the parameters given in Eq. (10)) on final segmentation. It is clear from the figure that only the color information of the pixels lead to over-segmentation but the incorporation of spatial information (based on color) turns-out improvement in the segmentation process. Note that, in the case of building image (Fig. 3, first row), when p = 1 and q = 1, a slight variation in the color (due to light) are marked as separate segments, which are not desired. But for the same image, when p = 1 and q = 2, the small variation in colors are ignored and optimal segmentation is performed. Similarly for the bear image (Fig. 3, second row) when p = 1 and q = 2 it performs well. To qualitatively demonstrate the usefulness of the proposed method 6 images (each contains two different versions of human segmentation) are selected from the Berkeley segmentation database. These images include: an Aeroplane (481 × 321), Tree (481 × 321), Plant (481 × 321), Horse (321 × 481), Elephant (481 × 321) and Bear (481 × 321). Fig. 4 shows the segmentation results of the seven algorithms for an image in which an aeroplane is flying in a cloudy environment. It is clear from the Fig. 4c that the proposed method outperforms the rest of the other methods and perfectly separate the cloudy background and the foreground aeroplane. For the same image SRM, LDC, and RADp (Figs. 4(d)–(f)) detect the foreground aeroplane but further divide the cloudy background into two or three more segments. Note that, KGC (Fig. 4(g)), LS (Fig. 4(h)) and NC (Fig. 4(j)) perform over-segmentation and divide the foreground and background into multiple segments. The EG (Fig. 4(i)) fails to detect the actual boundaries of the foreground aeroplane. Similarly for the tree image (Fig. 5), the performance of the proposed method is perfect. The remaining methods over segment the image by dividing the sky area, grass region and the tree into undesired segments. In the case of plant image (Fig. 6(c)) the proposed method performs better to detect the boundaries of the plant and divide the background into two separate segments. Similarly, the SRM (Fig. 6(d)) performs better to detect the plant and segment the background but it further divide the plant region into small unnecessary segments. The rest of the algorithms (Figs. 6(e)–(j) make over-segmentation either divide the background, foreground or both into undesirable multiple segments. For the horse image (Fig. 7(c)) the proposed method performs better than the rest of the methods. The SRM (Fig. 7(d)) performs nearly equal to the proposed method but it makes slight oversegmentation in the foreground horses. The LDC (Fig. 7(e)) and KGC (Fig. 7(g)) over-segment the foreground horses while the RADp (Fig. 7(f)) fails to detect the horses region and perform a noisy segmentation of the background region. Similarly, the LS (Fig. 7(h))
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402
403
404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449
G Model ASOC 2858 1–11
ARTICLE IN PRESS
6
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Fig. 2. First column contains the original images while the second and third columns show the images before and after the elimination process.
Fig. 3. Effect of spatial information on final segmentation. The first column contains the original images, the second column shows the segmentation results when p = 1 and q = 1, while the third column shows the segmentation results when p = 1 and q = 2.
450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467
makes a noisy segmentation for the horse image. The EG (Fig. 7(i)) performs good to detect the boundaries of the horses but makes a noisy segmentation. Likewise the previous cases the NC (Fig. 7(j)) again make equal size non-uniform segments. Fig. 8 demonstrates that the proposed method outperforms the remaining methods by producing the result similar to that of the corresponding human subject. In bear image (Fig. 9) the proposed method achieves best results as compared to the other state-of-the-art methods. For the same bear image (Fig. 9) the remaining methods make oversegmentation while RADp is unable to perform segmentation at all. From the above illustrations the following general observations are concluded. Generally the SRM has the tendency to divide an image into regions less than the required number of segments. But for some images SRM produces over-segmented results by making the small regions as separate segments. Usually, LDC fails to get the actual edges and tends to produce over-segmented results. In most of the cases RADp is unable to produce the real boundaries of the
segments and leads to under-segmentation. Moreover, in many situations the RADp makes the uniform regions as separate segments and produce a noisy segmentation. Generally, KGC performs an over-segmentation but in some cases it perform better to get the actual segments. The performance of LS is unsatisfactory because it fails to get the real boundaries and edges of the objects. For some images EG perform good enough but in many cases it spawns oversegmented results and fails to get the boundaries of the objects. The NC tends to partition an image into equal size segments and fails to get the real boundaries. It is clear from the mentioned results that the proposed method performs better than the other state-of-theart methods in term of boundaries detection and number of actual segments generation. 5.2. Quantitative comparisons Quantitative evaluation has a significant importance to assess the effectiveness of an algorithm objectively. The variation of
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
468 469 470 471 472 473 474 475 476 477 478 479 480
481
482 483
G Model ASOC 2858 1–11
ARTICLE IN PRESS A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
7
Fig. 4. Segmentation of an aeroplane image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Fig. 5. Segmentation of the Tree Image: (a) and (b) show the two different versions of human segmented images while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
G Model ASOC 2858 1–11
ARTICLE IN PRESS
8
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Fig. 6. Segmentation of the plant image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Fig. 7. Segmentation of the sparrow image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
G Model ASOC 2858 1–11
ARTICLE IN PRESS A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
9
Fig. 8. Segmentation of the elephant image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Fig. 9. Segmentation of the bear image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
G Model ASOC 2858 1–11
ARTICLE IN PRESS
10
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Table 2 Average values of PRI and VOI for the eight algorithms on the images in the Berkeley segmentation database. Measures
Proposed method
SRM
LDC
RADp
KGC
LS
EG
NC
PRI VoI
0.8332 1.9239
0.7681 2.0551
0.7833 2.6996
0.5366 2.119
0.7343 2.8982
0.6738 4.9686
0.7857 2.7589
0.7212 2.9893
Fig. 10. Probabilistic rand index (PRI) score of the 8 algorithms on 300 images of the Berkeley segmentation database.
484 485 486 487 488 489 490 491 492 493
information (VoI) [32] and probabilistic rand index (PRI) [33] are used to objectively assess the performance of the proposed algorithm. Variation of information (VoI): variation of information (VoI) [32] is a measure of information contents which depict that how much one segmentation reflects the information of the other segmentation. Actually it is the conditional entropy among the distributions of the segments labels. Let a set of ground truth images {S1 ; S2 , . . ., SN } and the actually segmented image Ss are given. Then the variation of information can be calculated in the following way: 1 VoI(Ss , {Sk }) = [H(Ss ) + H(Sk ) − 2I(Ss , Sk )] N N
494
(14)
k=1 495 496 497
where H and I represent entropy and mutual information respectively. The entropy H is always nonnegative. It is zero when there is no uncertainty. The uncertainty depends on the relative proportion
of the segments, it is not influenced by the number of pixels in a segment. The mutual information I represents that how much one segment have information about the other segment. The values of the variation of information (VoI) lies in [0, ∞). The 0 means that the two segmentations are perfectly matching. The smaller the VoI value the better will be the results. Probabilistic rand index (PRI): suppose a set of human segmented images, labeled by K persons, {S1 , S2 , . . ., SK }; of an image containing N pixels. Let Ss be the result of a source segmentation and we want to compare it with ground truth images, {S1 , S2 , . . ., SK }. Then the PRI [33] can be calculated as: PRI(Ss , {Sk }) =
1
N
c
pq [ppq (1 − ppq )
1−cpq
]
(15)
498 499 500 501 502 503 504 505 506 507 508
509
p
2 where (p, q) is the pair of a pixels in the given image, cpq = T (lpStest = lqSs ) denotes that the pixels p and q have the same label in the source
Fig. 11. Variation of information (VoI) score of the 8 algorithms on 300 images of the Berkeley segmentation database.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
510 511
G Model
ARTICLE IN PRESS
ASOC 2858 1–11
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
K
540
segmentation Ss , and ppq = 1/K k=1 T (lpSk = lqSk ) is related to the probability that p and q has the same label. Probabilistic rand index (PRI) compares the segmentation results generated by a particular algorithm with a number of human segmented images. The comparison is performed by inducing the variability in human segmented images through variable weights of pixels pairs. The value of PRI lies in the range of [0, 1], the larger the value, the better is the result. Table 2 shows the average scores of PRI and VoI measures for the 8 algorithms on 300 images in the Berkeley segmentation database (BSD). It is clear from Table 2 that the proposed method performs better than the other state-of-the-art algorithms in terms of VoI and PRI values. The average PRI values of LDC and EG are close to the proposed algorithm but their average VoI values are much greater. Similarly the average VoI values of the SRM and RADp are nearly approaching the proposed algorithm but their average PRI scores are far away from the desired target. Moreover, KGC is not able to reach the proposed method in average PRI nor in average VoI scores. It is clear from Table 2 that LS and NC have very poor results in term of both PRI and VoI values. The performance of each of the eight algorithms, on each image in the Berkeley segmentation database (BSD), are graphically represented in Figs. 10 and 11. Fig. 10 graphically depicts the PRI values of the given eight algorithms for 300 images in the Berkeley segmentation database. Whereas the VoI scores of the given eight algorithms for 300 images in the Berkeley segmentation database are shown in Fig. 11. It is clear from the curves in Figs. 10 and 11 that the proposed algorithm performs better than the other stateof-the-art methods.
541
6. Conclusion
512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539
560
In this paper the color image segmentation is performed by a fixed length genetic algorithm (GA). A self organizing map (SOM) is used to determine the number of segments in order to set length of a chromosome. A new objective function based on compactness and separation is proposed to asses the quality of the solution. Note that, an opposition based strategy is used to initialize the GA to the productive search areas. The new objective function and opposition based initialize strategy make the GA to come out of the local optima and produce best results. In some cases the given genetic algorithm makes the small regions of an image as separate segments and leads to a nearly noisy segmentation. A simple ad hoc mechanism is adopted to refine segmentation by merging the small segments to their matching neighbors. The results of the proposed method are compared with eight state-of-the-art image segmentation algorithms. The qualitative and quantitative results demonstrate that the proposed algorithm performs better than the other eight algorithms. In the future, the texture and geometrical features of the objects will be combined with color features to perform robust segmentation.
561
References
542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559
562 563
[1] L. Lucchese, S.K. Mitra, Color image segmentation: a state of the art survey, Proc. Indian Natl. Sci. Acad. Image Process. Vis. Pattern Recogn. 67 (2) (2001) 207–221.
11
[2] E. Vazquez, J.V.D. Weijer, R. Baldrich, Image segmentation in the presence of shadows and highlights using a ridge based histogram analysis, in: Proc. ECCV08, vol. 5305, 2008, pp. 1–14. [3] P.K. Loo, C.L. Tan, Adaptive region growing color segmentation for text using irregular pyramid, in: International Workshop on Document Analysis Systems, vol. 3163, 2004, pp. 264–275. [4] M. Kass, A. Witkin, D. Terzopoulos, Snakes: active contour models, Int. J. Comput. Vis. 1 (4) (1988) 321–331. [5] D. Cohen, et al., On active contour models and balloons, CVGIP: Image UnderQ6 stand. 53 (2) (1991) 211–218. [6] Xu. Chenyang, J. Prince, Snakes, shapes, and gradient vector flow, IEEE Trans. Q7 Image Process. 7 (1998) 3. [7] E. Mortensen, W. Barrett, Interactive segmentation with intelligent scissors, Graph. Models Image Process. 60 (1998) 5. [8] J. Besag, On the statistical analysis of dirty pictures, J. Roy. Stat. Soc. B 48 (1986) 259–302. [9] Y.B. Kov, M. Jolly, Interactive graph cuts for optimal boundary region segmentation of objects in N-D images, IEEE Int. Conf. Comp. Vis. 1 (2001) 105–112. [10] V. Kolmogorov, R. Zabih, What energy functions can be minimized via graph cuts ? IEEE Trans. Pattern Anal. Mach. Intell. 26 (2) (2004) 147–159. [11] C. Rother, V. Kolmogorov, A. Blake, Grabcut interactive foreground extraction using iterated graphcuts, ACM Trans. Graph. 23 (3) (2004) 309–314. [12] Y. Li, J. Sun, C.K. Tang, H.Y. Shum, Lazy snapping, ACM Trans. Graph. 23 (3) (2004) 303–308. [13] S. Beucher, F. Meyer, The morphological approach to segmentation: the watershed transformation, Math. Morphol. Image Process. 34 (1993) 433–481. [14] L. Vincent, P. Soille, Watersheds in digital spaces: an efficient algorithm based on immersion simulations, IEEE Trans. Pattern Anal. Mach. Intell. 13 (6) (1999) 583–598. [15] R. Duda, P. Hart, D. Stork, Pattern Classification, 2nd ed., Wiley, Hoboken, NJ, 2001. [16] M.N. Ahmed, S.M. Yamany, A.A.F. Nevin Mohamed, T. Moriarty, A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI data, IEEE Trans. Med. Imaging 21 (March (3)) (2002) 193–199. [17] K.S. Chuanga, et al., Fuzzy c-means clustering with spatial information for image segmentation, Comp. Med. Imaging Graph. 30 (2006) 9–15. [18] A. Khan, M.A. Jaffar, T.S. Choi, Som and fuzzy based color image segmentation, Multimedia Tools Appl. 64 (2) (2013) 331–344. [19] L. Vendramin, R.J.G.B. Campello, E.R. Hruschka, Relative clustering validity criteria: a comparative overview, Stat. Anal. Data Min. 3 (4) (2010) 209–235. [20] D.E. Ilea, P.F. Whelan, Ctex – an adaptive unsupervised segmentation algorithm based on color-texture coherence, IEEE Trans. Image Process. 17 (10) (2008) 1926–1939. [21] D. Martin, C. Fowlkes, D. Tal, J. Malik, A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, in: IEEE International Conference on Computer Vision, vol. 2, 2001, pp. 416–423. [22] A. Khan, M.A. Jaffar, L. Shao, A modified adaptive differential evolution algorithm for color image segmentation, Knowl. Inform. Syst. (2014) 1233–1243. [23] S. Rahnamayan, H.R. Tizhoosh, M.M.A. Salama, Opposition-based differential evolution, IEEE Trans. Evol. Comput. 12 (1) (2008) 64–79. [24] J.C. Bezdek, Cluster validity with fuzzy sets, J. Cybern. 3 (3) (1973) 58–73. [25] K.S. Chuang, H.L. Tzeng, S. Chen, T.J. Chen, Fuzzy c-means clustering with spatial information for image segmentation, Comp. Med. Imaging Graph. 30 (2006) 9–15. [26] R. Nock, F. Nielsen, Statistical region merging, IEEE Trans. Pattern Anal. Mach. Intell. 26 (11) (2004) 1452–1458. [27] A.Y. Yang, J. Wright, S.S. Sastry, Y. Ma, Unsupervised segmentation of natural images via lossy data compression, Comp. Vis. Image Understand. (2008) 212–225. [28] M. Ben Salah, A. Mitiche, I.B. Ayed, Multiregion image segmentation by parametric kernel graph cuts, IEEE Trans. Image Process. 20 (2) (2011) 545–557. [29] C. Li, R. Huang, Z. Ding, J.C. Gatenby, D.N. Metaxas, J.C. Gore, A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI, IEEE Trans. Image Process. 20 (7) (2011) 2007–2016. [30] P. Felzenszwalb, D. Huttenlocher, Efficient graph-based image segmentation, Int. J. Comput. Vis. 59 (02) (2004) 167–181. [31] J. Shi, J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell. 22 (8) (2000) 888–905. [32] M. Meila, Comparing clusterings by the variation of information, Lect. Note Comp. Sci. 2777 (2003) 173–187. [33] R. Unnikrishnan, M. Hebert, Measures of similarity, in: Proceedings of the IEEE Workshop on Computer Vision Applications, vol. 1, 2005, pp. 394–401.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637
ARTICLE IN PRESS
ASOC 2858 1–11
Applied Soft Computing xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Applied Soft Computing journal homepage: www.elsevier.com/locate/asoc
Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation
1
2
3
Q1
Ahmad Khan a,b , Muhammad Arfan Jaffar a,c,∗ a
4
b
5
c
6
National University of Computer and Emerging Sciences, H-11/4 Islamabad, Pakistan COMSATS Institute of Information Technology, Abbottabad, Pakistan College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia
7
a r t i c l e
8 24
i n f o
a b s t r a c t
9
Article history: Received 29 June 2011 Received in revised form 16 March 2015 Accepted 16 March 2015 Available online xxx
10 11 12 13 14 15
23
Keywords: Self organizing map (SOM) Segmentation Fuzzy Genetic algorithm (GA) Color image Centroid Segment
25
1. Introduction
16 17 Q3 18 19 20 21 22
The grouping of pixels based on some similarity criteria is called image segmentation. In this paper the problem of color image segmentation is considered as a clustering problem and a fixed length genetic algorithm (GA) is used to handle it. The effectiveness of GA depends on the objective function (fitness function) and the initialization of the population. A new objective function is proposed to evaluate the quality of the segmentation and the fitness of a chromosome. In fixed length genetic algorithm the chromosomes have same length, which is normally set by the user. Here, a self organizing map (SOM) is used to determine the number of segments in order to set the length of a chromosome automatically. An opposition based strategy is adopted for the initialization of the population in order to diversify the search process. In some cases the proposed method makes the small regions of an image as separate segments, which leads to noisy segmentation. A simple ad hoc mechanism is devised to refine the noisy segmentation. The qualitative and quantitative results show that the proposed method performs better than the state-of-the-art methods. © 2015 Elsevier B.V. All rights reserved.
The fundamental purpose of color image segmentation is to divide or partition a color image into a set of disjoint non27 overlapping regions (segments) such that each segment represents 28 29Q4 a particular object. Since last decades, abundance of techniques have been proposed to address the problem of an image segmen30 tation. Although, color image segmentation is still considered as a 31 challenging problem in computer vision research community. Par32 ticularly, the segmentation of colored natural images is the most 33 difficult job because of the random inhomogeneities that exist in 34 color and texture. Most of the approaches address the segmen35 tation of a particular type of an image for particular application. 36 Moreover, these approaches are not fully automatic and need a 37 significant amount of user intervention to segment the objects of 38 interest accurately. 39 In literature there are many color image segmentation tech40 niques which can be divided into [1] image-domain based, physics 41 26
∗ Corresponding author at: College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia. Q2 Tel.: +966 553605730. E-mail addresses: [email protected] (A. Khan), [email protected] (M.A. Jaffar).
based and feature-space based approaches. Image-domain based methods use both color and spatial features to carry out the segmentation task. The efficiency of these methods is depending on the selection of seed regions, which are normally set manually by the user. On the other hand the physics based methods use the physical models of material reflection to carry out color image segmentation [2]. It needs the reflection property of the underlying object which are not known in most of the cases and hence it is an application specific. The feature-based methods use color features only which leads to inaccurate results due to the lack of spatial relationship among the pixels [3]. Contour based segmentation algorithms can be found in literature [4–7]. It starts from an initial boundary in the form of a spline curve; the curve is iteratively evolved to minimize some energy function. It suffers from the manual initialization of the curve or boundary to the object of interest. It is trapped in local optima and leads to unsatisfactory results. Markov random field based on statistical models [8] uses spatial connectivity information among the neighboring pixels to perform segmentation. Graph cut [9–12] is another segmentation approach based upon Markov random fields. These algorithms solve a two class problem, i.e., separating the background and foreground objects. Watershed segmentation [13,14] has been successfully adopted for the two class segmentation problems. It often produces bad results for image having many regions.
http://dx.doi.org/10.1016/j.asoc.2015.03.029 1568-4946/© 2015 Elsevier B.V. All rights reserved.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66
G Model
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116
ASOC 2858 1–11
ARTICLE IN PRESS
2
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Image segmentation can be treated as a clustering problem. The feature of each pixel corresponds to a pattern and the combination of the pixels (i.e. segment) corresponds to a cluster. Keeping this analogy in mind, many clustering algorithms have been used for image segmentation [15–18]. Among them K-means [15] is the basic clustering algorithm adopted for color image segmentation. Similarly, fuzzy C-means (FCM) [16] has been used for image segmentation extensively. It is a clustering process which decrease intra clusters distances and increase inter clusters distances. However, its implementation faces three problems: (1) cluster initialization, (2) determining number of the clusters, and (3) lacking of the robust cluster validity index. The performance of the algorithm depends on the initialization of clusters centroids. If the cluster centroids represent the search space uniformly then the results will be better on the other hand the biased initialization will lead to poor performance. Similarly the number of clusters have considerable impact on the quality of segmentation. The clustering validity index (CVI) performs a vital role in the process of image segmentation and data clustering [19]. It checks whether the provided clustering or grouping of the given data reflects the original structure of the data or not. The cluster validity index (CVI) adopted for the FCM takes into account both compactness and separation for reliable segmentation. Note that if the first (i.e., compactness) is adopted as a cluster validly index then the best result would contain the cluster/segments number as that of the number of points present in the underlying search space. Similarly if the second (i.e., separation) is considered as a cluster validity index then the best result would contain only one segment/cluster. The CVI that combines compactness and separation is insufficient to represent the geometrical structure of the data because it normally finds the distances between the centroids of different clusters. In this paper, a fixed length fuzzy genetic algorithm (GA) is used to perform color image segmentation. A new objective function or clustering validity index (CVI) is proposed which incorporate the fuzzy compactness of data points and fuzzy separation of the segments centroids. In the new objective function, the compactness is defined term of fuzzy deviation of data points while the separation is based on the average variance of segments centroids. A self organizing map (SOM) is used to determine the optimal number of segments in order to set the length of a chromosome. In some cases the proposed method makes some small regions of an image as separate segments which leads to noisy segmentation. A simple merging technique is proposed to address this problem and make segmentation refine. All the operations are performed in the Lab color space, which is near to the human level of perception. The block diagram of the proposed method is given in Fig. 1. The rest of the paper is organized as follows. Section 2 presents a mechanism to set the length of a chromosome automatically. In Section 3 the GA based color image segmentation is discussed. Section 4 presents the detail of the merging algorithm to handle the
over-segmentation problem. Experimental results are discussed in Section 5 followed by the conclusion. 2. Determining length of the chromosome
118
119
We have used a fixed length genetic algorithm in which the length of a chromosome depends on the number of segments. Generally in segmentation algorithms the number of segments is set by user manually using some sort of domain knowledge. In this work a self organizing map (SOM) [20] is used to determine the number of segments that are hiddenly present in the underlying distribution (input image). SOM is an unsupervised type of artificial neural network (ANN) which is used to produce the low dimensional map of complex input space. Here we use a two-dimensional (2D) SOM network that consists of nodes or cells. Each node Ni of the SOM, where i ) having three elements. The i ∈ 16, is assigned a weight vector (w training dataset (in the form of input image) is organized as a vector V j in a raster scanning manner, where j = 1, . . ., N and N represents the total number of pixels in the input image. Note that each element of the V j (training set) consists of the three channels of the input image. The SOM network is randomly initialized. Then each input vector is iteratively feeded to the SOM network. The input vector is assigned to a best matching unit (BMU), a node in the SOM network which has maximum similarity to the given input vector. The best matching unit (BMU) to a given input vector V j is determined by using the following Eq. (1).
i , j ∈ [1, N] BMU = argmin V j − w
117
(1)
120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142
143
i∈[1,16]
where 16 is the number of nodes/cells in the SOM network. Here, 16 is to be considered the maximum number of segment s present in a particular image. The weights of NBMU (a node in the network which has maximum matching to the input vector V j ) and its neighboring nodes are updated using Eq. (2) i (t + 1) w
=
144 145 146 147 148
149
i (t)], i (t) + (t)(t)[V j (t) − w w
ifNBMU − Ni ≤ v(t)
i (t), w
ifNBMU − Ni > v(t)
(2)
150
151
In Eq. (2) t, and v(t) represent the iteration step and the neighborhood radius of the network respectively. Initially the neighborhood radius v0 is kept high to cover all the nodes of the network. Then the size of neighborhood radius v(t) reduces exponentially with the increase in number of iterations t, as given in Eq. (3).
v(t) = v0 · e−t/
(3)
Fig. 1. Block diagram of the proposed method.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
152 153 154 155 156
157
G Model
ARTICLE IN PRESS
ASOC 2858 1–11
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx 158 159 160 161 162 163 164 165 166 167 168 169 170
171
172 173 174 175 176 177
178
179 180 181 182 183 184
185
186 187 188
where t is the iteration number and is a constant usually set to the ratio between the initial radius and the maximum number of iterations. The weights that lie inside the neighborhood of BMU are updated. It is clear from Eq. (2) that the weight updating process of the nodes, that lies inside in the neighborhood of BMU, are controlled by two functions (t) and (t). The function (t) represents the learning rate which shows that what percent of change should be incorporated in the weight of a particular node. In the initial stages more changes are made to the weights of SOM network in order to speed up the convergence. However to ensure maximum exploitation, the effect of learning rate is reduced with the increase in number of iteration t. The learning rate function (t) is given by Eq. (4). (t) = 0 · e
−t/
(4)
where 0 is a constant value which may be fixed according to the requirement of learning process. The neighborhood function (t) controls the updating strength of the neighbor nodes of the best matching unit (BMU). It is implemented as a Gaussian function, in order to make changes according to proximity of neighbors, as given in Eq. (5). (t) = e
−
NBMU −Ni 2 2v2 (t)
(5)
The training of the SOM network continues until the radius v(t) reach the size of NBMU and the weights of the network are not changed for a few successive iterations. Using the procedure of [20], the number of segments are determined from the trained SOM network. For each pixel a mapping is performed to the final weights of the nodes using Eq. (6). g , g=arg min V j − w i , j ∈ [1, N] V j ← w i∈[1,16]
Then a variance map is formed based on the minimum cumulative distance among the weights of the SOM and pixels of input image. i , the variance is given as: For all pixels V j labeled with w
189
(6)
i) = variance(w
j∈Dw
i
3
m : mutation rate c : crossover rate Np : population size P: is a matrix used to represent the population. Note that, there are Np rows in P and each row represents a particular chromosome. aj : the maximum value in the jth channel of an image bj : the minimum value in the jth channel of an image : represents the logical OR operator Pi,g : is a row vector represents the ith chromosome at generation (iteration) g F i,g : is a scalar value which represents the fitness of the ith chromosome at generation g 3.1. Initialization A genetic algorithm needs to be initialized before starting the iterative procedure. Usually the initialization is performed by user using some sort of domain knowledge or random procedure. The pure random initialization is not good enough because it forces the segmentation algorithm to converge to local optima [22]. The initialization process can be diversified by opposite number generation. The opposite number diversify the initialization process and tends to select the initial solutions from the productive regions of the underlying search space. Here we used the opposition based strategy [23] to generate the initial population. The best individuals from the union of purely random solutions and their opposites are selected. This strategy avoids the unproductive samples or solutions and initialize the population to the most fertile regions. In Algorithm 1 lines 3–8, demonstrate the initialization process. The aj and bj (Algorithm 1, line 6) are the minimum and maximum values of a particular dimension, which ensure to select a value in the given range.
i) no pixels labeled(w
, i ∈ [1, 16]
A genetic algorithm (GA) is a population based probabilistic search technique used to solve the optimization problems. This section presents the details of genetic algorithm and its use in the area of color image segmentation. The basic mechanism of GA is outlined in Algorithm 1. The following are the terminologies used in the given algorithm.
Here a chromosome is represented as a series of real values to encode K segments centroids. Each segment centroid consists of three colored channel of an image. So a particular chromosome can be represented as a string of K × 3 real values. In each chromosome the first three values represent the centroid of first segment, the second three values represent the centroid of second segment and
195 196 197 198 199 200 201 202 203 204
207 208 209 210 211 212
218 219 220 221 222
223 224
225
226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242
244
3.2. A chromosome encoding
194
217
(7)
3. Genetic algorithm based color image segmentation
193
216
i V j − w
206
192
215
243
205
191
214
Algorithm 1.
where Dw i is the domain set of pixels V j that has maximum match i . The smaller variance (w i ) value represents a ing to the weights w more reliable estimate of a compact segment. In the last step the inter-segment variability is calculated to find out the final number of segments. A similarity matrix is constructed in order to store the Euclidian distances between the neighboring nodes in the SOM network. An elimination process is exploited to minimize the number of nodes/segment centroids. If the distance between two neighboring nodes is less than a pre-defined threshold ˇ, inter segment distance, then a node having highest variance score is removed. The elimination process continues until the distances among all the neighboring nodes remain greater than a pre-defined threshold value of ˇ. In this work the inter-segment threshold ˇ is set to 0.4 based on the empirical study of BSD images [21]. This value proved good enough for all images analyzed in the experimentation.
190
213
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
245 246 247 248 249 250
G Model
251 252 253
ASOC 2858 1–11
ARTICLE IN PRESS
4
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
so on. Suppose we have K = 3 then a particular chromosome can be represented as: C11
C12
C13
C21
C22
C23
C31
C32
C33
the separation determines that how much the given segments centroids are different. The smaller the compactness and the greater the separation leads to better segmentation. Here we determine the compactness in the following way:
254 255 256 257 258
259
260
The above illustration demonstrates three segment centroid s (C11 , C12 , C13 ), (C21 , C22 , C23 ), and (C31 , C32 , C33 ) respectively, where Cij represents the jth value of the ith segment centroid such that 1 ≤ i ≤ K, and, j ∈ {1, 2, 3}.
In fuzzy C-mean (FCM) algorithm [24] the membership ij of a data point xj to a segment centroid vi is given as:
262
ij =
(8)
D(vk ,xj )
280
281
sij =
264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279
ik
(9)
k∈(xj ) 282 283 284 285 286 287 288 289
where (xj ) denote a square window, in the spatial domain centered around a pixel xj . The spatial function sij represents the relationship among the neighboring pixels – that how much they are similar. If the majority of pixels, in the same neighborhood, belongs to one segment then the value of spatial function will be greater and vice versa. The spatial information can be included in the membership function by the following way: p q
290
ij =
ij sij
K
k=1 291 292 293 294 295 296 297 298
p q
(10)
kj skj
where p and q are parameters used to give relative weights to each function. Here we set the values of p and q to 1 and 2 respectively. The effect of p and q on the final segmentation is discussed in qualitative evaluation Section 5.1. In this paper we use the compactness and separation measures to develop a new objective function (fitness function) in order to asses the quality of solution/chromosome. The compactness determines that how much the points in one segment are similar while
300 301 302
(11)
303
where ij D(vi , xj ) is the weighed distance of a point xj to a segment
304
i=1
i=1
ij
K
is the fuzzy cardinality of the segment cencentroid vi and troid vi . Note that the sum of the weighted distances of a all points give the fuzzy deviation of the data. For the fuzzy separation we find the distances between each pair of segments centroids. If there are total K centroids, then K(K − 1)/2 distances can be calculated. So, the fuzzy separation of the segments centroids is given as:
2
where N and K represent the number of data points and the number of segments, D(vi , xj ) represents a distance between a cluster centroid vi and a point xj , m is a fuzzifier which determines the level of cluster fuzzyness. It is important to note that, when calculating the membership value ij in Eq. (8) if D(vi , xj ) becomes zero for any value of k then ij is assigned 0, ∀i = 1, / k, while the value of ij is set to 1. It means 2, . . ., K, i = that a particular point xj has complete matching with the segment centroid vi therefore the difference or distance between them becomes 0. Then the membership of this point to the corresponding segment centroid is set to maximum,i.e., 1 and the membership to the rest of the cluster centroids is set to minimum, i.e., 0. In images the neighboring pixels have maximum similarities. In other words, they have similar features or characteristics. So there is a maximum probability that they belong to the same segment. To exploit this idea in color image segmentation a spatial function is constructed as [25]:
263
ij D(vi , xj )
K
Mediani =/ k (D(vi , vk )) − Mini =/ k (D(vi , vk )) + Maxi =/ k (D(vi , vk )) − Mediani =/ k (D(vi , vk ))
, for1 ≤ i ≤ K, 1 ≤ j ≤ N 2 K D(vi ,xj ) m−1 k=1
i=1
261
1
K
i=1 ij
3.3. The proposed objective function
=
˝=
K
299
(12)
The compactness ‘˝’ should be minimized to get compact segments (i.e. to maximize the inter-segment similarities). On the contrary the fuzzy separation should be maximized in order to decrease the intra-segments similarities. Finally the fitness function is determined by Eq. (13). F=
˝
(13)
The chromosome/solution which has higher fitness value is considered the best. It is a maximization case so the aim is to maximize the fitness value. The chromosomes having highest fitness values are selected for the reproduction. 3.4. Selection In GA, selection is one of the most important operator. The main purpose of the selection is to select the best chromosomes for reproduction. To determine the quality of a solution/chromosome the fitness function is used. The chromosome which has best fitness should have higher probability for the selection. Here we use the binary tournament selection. 3.5. Reproduction Reproduction is the process to generate new solutions (chromosomes) from the current selected solutions. We carried out a single point crossover to reproduce new child solutions. The cut point for crossover is selected randomly. Note that, the crossover probability c is initialized to 0.8, while the mutation probability m is set to 0.05. If a chromosome is selected for mutation then the allele position, which is expose to the mutation process, is randomly selected. The allele value of that particular position is replaced by a randomly generated value from the respective component of an image. The parent and new offspring populations are exposed together for the selection of new generation in order to ensure the elitism operation. 3.6. Stoping criteria It a is condition to terminate the iterative execution of the GA. / true) as a stoping criteria. Here, G is the maxiWe use (g ≤ G ı = mum number of iterations and ı is a flag which is set to true if < (Algorithm 1, lines 20–23). Note that, here we set to 0.0001. Moreover, it each iteration g the is updated as (Algorithm 1, lines 20): ← F best,g − F best,g−1 , where the Fbest,g represents the fitness of the best chromosome at the current iteration while Fbest,g−1 represents the fitness of the best chromosome at previous iteration Table 1
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
305 306 307 308 309 310 311
312
313 314 315 316 317
318
319 320 321 322
323
324 325 326 327 328 329
330
331 332 333 334 335 336 337 338 339 340 341
342
343 344 345 346 347 348 349 350
G Model ASOC 2858 1–11
ARTICLE IN PRESS A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Table 1 The proposed method parameter setting
351
Parameter
Value/type
Crossover type Crossover cut point selection Mutation type Crossover rate (c ) Mutation rate (m ) Selection type Population size
Single point Random Random 0.8 0.05 Binary tournament 20
4. Elimination of the small segments
Q5
357
. In some cases the given genetic algorithm makes the small regions as separate segments, which leads to over-segmentation. A simple ad hoc mechanism is used to eliminate the small segments by merging the to their matching neighbors. The pseudocode of the elimination process is given in Algorithm 2.
358
Algorithm 2.
352 353 354 355 356
359 360 361 362 363 364 365 366 367 368 369 370 371 372
373
374 375 376 377 378 379 380 381 382 383 384 385
If the size of a segment x is less than a threshold then its neighboring segments are find out. In line 4, y ∈ NB(Lx ) represent that segment y is the neighbor of the segment x. Then the average distance among the segment y and segment x in the corresponding color image I is calculated. Note that, the argmin(·) returns the label of the best matching neighbor (BMN) which has the smallest distance to the target segment x. After finding the best matching neighbor (BMN) the two segments are merged by giving same label to their union in the corresponding newLp∪q . Fig. 2 shows the results of the elimination process. The first column shows the original images, while the second and third columns show the segmentation results before and after the elimination process. Note that, some small dots can be seen in Fig. 2 (column 2 before elimination) which are eliminated in column 2 by the elimination process. 5. Experimental results The experimentations are carried out on images taken from the Berkeley image segmentation database (BSD) [21]. The results of the proposed method are compared with seven well known image segmentation methods: statistical region merging (SRM) [26], segmentation via lossy data compression (LDC) [27], image segmentation in the presence of shadows and highlights using a ridge based histogram analysis (RADp) [2], multiregion image segmentation by parametric kernel graph cuts (KGC) [28], a level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI (LS) [29], efficient graph-based image segmentation (EG) [30], and normalized cuts and image segmentation (NC) [31].
5
The different parameters of the algorithms (used for comparison) are adjusted as suggested by the authors. For illustration, in (LDC) [27] the window size w and the texture difference metric ˛ are initialized to 7 and 2 respectively. In the case of EG [30], the parameter is set to 0.8, the threshold value k is initialized to 300, and the minimal region= 50 pixels. The number of regions in normalized cuts (NC) [31] is set to 20, which is the average number of segments in BSD. In the case of (SRM) [26] the segmentation parameter Q is given as: Q = 2 , where = 0, 1, 2, . . ., 8. The scale parameters in (LS) algorithm [29] are set to 4. For (KGC) [28] the weight of the smoothness constraint ˛ is initialized to 1 and the number of regions k to 20. The codes of all of the 7 algorithms are downloaded from their respective web sites. Due to the overflow of memory NC and LDC are unable to process an image of size (321 × 481) or (481 × 321), so for these algorithms all of the images are reduced to size 314 × 208 (or 208 × 314), and finally the segmented results are restored to their actual sizes. 5.1. Qualitative evaluation Effect of spatial information on final segmentation: Fig. 3 demonstrates the effect of p and q (the parameters given in Eq. (10)) on final segmentation. It is clear from the figure that only the color information of the pixels lead to over-segmentation but the incorporation of spatial information (based on color) turns-out improvement in the segmentation process. Note that, in the case of building image (Fig. 3, first row), when p = 1 and q = 1, a slight variation in the color (due to light) are marked as separate segments, which are not desired. But for the same image, when p = 1 and q = 2, the small variation in colors are ignored and optimal segmentation is performed. Similarly for the bear image (Fig. 3, second row) when p = 1 and q = 2 it performs well. To qualitatively demonstrate the usefulness of the proposed method 6 images (each contains two different versions of human segmentation) are selected from the Berkeley segmentation database. These images include: an Aeroplane (481 × 321), Tree (481 × 321), Plant (481 × 321), Horse (321 × 481), Elephant (481 × 321) and Bear (481 × 321). Fig. 4 shows the segmentation results of the seven algorithms for an image in which an aeroplane is flying in a cloudy environment. It is clear from the Fig. 4c that the proposed method outperforms the rest of the other methods and perfectly separate the cloudy background and the foreground aeroplane. For the same image SRM, LDC, and RADp (Figs. 4(d)–(f)) detect the foreground aeroplane but further divide the cloudy background into two or three more segments. Note that, KGC (Fig. 4(g)), LS (Fig. 4(h)) and NC (Fig. 4(j)) perform over-segmentation and divide the foreground and background into multiple segments. The EG (Fig. 4(i)) fails to detect the actual boundaries of the foreground aeroplane. Similarly for the tree image (Fig. 5), the performance of the proposed method is perfect. The remaining methods over segment the image by dividing the sky area, grass region and the tree into undesired segments. In the case of plant image (Fig. 6(c)) the proposed method performs better to detect the boundaries of the plant and divide the background into two separate segments. Similarly, the SRM (Fig. 6(d)) performs better to detect the plant and segment the background but it further divide the plant region into small unnecessary segments. The rest of the algorithms (Figs. 6(e)–(j) make over-segmentation either divide the background, foreground or both into undesirable multiple segments. For the horse image (Fig. 7(c)) the proposed method performs better than the rest of the methods. The SRM (Fig. 7(d)) performs nearly equal to the proposed method but it makes slight oversegmentation in the foreground horses. The LDC (Fig. 7(e)) and KGC (Fig. 7(g)) over-segment the foreground horses while the RADp (Fig. 7(f)) fails to detect the horses region and perform a noisy segmentation of the background region. Similarly, the LS (Fig. 7(h))
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402
403
404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449
G Model ASOC 2858 1–11
ARTICLE IN PRESS
6
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Fig. 2. First column contains the original images while the second and third columns show the images before and after the elimination process.
Fig. 3. Effect of spatial information on final segmentation. The first column contains the original images, the second column shows the segmentation results when p = 1 and q = 1, while the third column shows the segmentation results when p = 1 and q = 2.
450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467
makes a noisy segmentation for the horse image. The EG (Fig. 7(i)) performs good to detect the boundaries of the horses but makes a noisy segmentation. Likewise the previous cases the NC (Fig. 7(j)) again make equal size non-uniform segments. Fig. 8 demonstrates that the proposed method outperforms the remaining methods by producing the result similar to that of the corresponding human subject. In bear image (Fig. 9) the proposed method achieves best results as compared to the other state-of-the-art methods. For the same bear image (Fig. 9) the remaining methods make oversegmentation while RADp is unable to perform segmentation at all. From the above illustrations the following general observations are concluded. Generally the SRM has the tendency to divide an image into regions less than the required number of segments. But for some images SRM produces over-segmented results by making the small regions as separate segments. Usually, LDC fails to get the actual edges and tends to produce over-segmented results. In most of the cases RADp is unable to produce the real boundaries of the
segments and leads to under-segmentation. Moreover, in many situations the RADp makes the uniform regions as separate segments and produce a noisy segmentation. Generally, KGC performs an over-segmentation but in some cases it perform better to get the actual segments. The performance of LS is unsatisfactory because it fails to get the real boundaries and edges of the objects. For some images EG perform good enough but in many cases it spawns oversegmented results and fails to get the boundaries of the objects. The NC tends to partition an image into equal size segments and fails to get the real boundaries. It is clear from the mentioned results that the proposed method performs better than the other state-of-theart methods in term of boundaries detection and number of actual segments generation. 5.2. Quantitative comparisons Quantitative evaluation has a significant importance to assess the effectiveness of an algorithm objectively. The variation of
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
468 469 470 471 472 473 474 475 476 477 478 479 480
481
482 483
G Model ASOC 2858 1–11
ARTICLE IN PRESS A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
7
Fig. 4. Segmentation of an aeroplane image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Fig. 5. Segmentation of the Tree Image: (a) and (b) show the two different versions of human segmented images while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
G Model ASOC 2858 1–11
ARTICLE IN PRESS
8
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Fig. 6. Segmentation of the plant image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Fig. 7. Segmentation of the sparrow image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
G Model ASOC 2858 1–11
ARTICLE IN PRESS A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
9
Fig. 8. Segmentation of the elephant image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Fig. 9. Segmentation of the bear image: (a) and (b) show the two different versions of human segmentation while (c)–(j) present the results of the proposed method, SRM, LDC, RADp, KGC, LS, EG and NC respectively.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
G Model ASOC 2858 1–11
ARTICLE IN PRESS
10
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
Table 2 Average values of PRI and VOI for the eight algorithms on the images in the Berkeley segmentation database. Measures
Proposed method
SRM
LDC
RADp
KGC
LS
EG
NC
PRI VoI
0.8332 1.9239
0.7681 2.0551
0.7833 2.6996
0.5366 2.119
0.7343 2.8982
0.6738 4.9686
0.7857 2.7589
0.7212 2.9893
Fig. 10. Probabilistic rand index (PRI) score of the 8 algorithms on 300 images of the Berkeley segmentation database.
484 485 486 487 488 489 490 491 492 493
information (VoI) [32] and probabilistic rand index (PRI) [33] are used to objectively assess the performance of the proposed algorithm. Variation of information (VoI): variation of information (VoI) [32] is a measure of information contents which depict that how much one segmentation reflects the information of the other segmentation. Actually it is the conditional entropy among the distributions of the segments labels. Let a set of ground truth images {S1 ; S2 , . . ., SN } and the actually segmented image Ss are given. Then the variation of information can be calculated in the following way: 1 VoI(Ss , {Sk }) = [H(Ss ) + H(Sk ) − 2I(Ss , Sk )] N N
494
(14)
k=1 495 496 497
where H and I represent entropy and mutual information respectively. The entropy H is always nonnegative. It is zero when there is no uncertainty. The uncertainty depends on the relative proportion
of the segments, it is not influenced by the number of pixels in a segment. The mutual information I represents that how much one segment have information about the other segment. The values of the variation of information (VoI) lies in [0, ∞). The 0 means that the two segmentations are perfectly matching. The smaller the VoI value the better will be the results. Probabilistic rand index (PRI): suppose a set of human segmented images, labeled by K persons, {S1 , S2 , . . ., SK }; of an image containing N pixels. Let Ss be the result of a source segmentation and we want to compare it with ground truth images, {S1 , S2 , . . ., SK }. Then the PRI [33] can be calculated as: PRI(Ss , {Sk }) =
1
N
c
pq [ppq (1 − ppq )
1−cpq
]
(15)
498 499 500 501 502 503 504 505 506 507 508
509
p
2 where (p, q) is the pair of a pixels in the given image, cpq = T (lpStest = lqSs ) denotes that the pixels p and q have the same label in the source
Fig. 11. Variation of information (VoI) score of the 8 algorithms on 300 images of the Berkeley segmentation database.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
510 511
G Model
ARTICLE IN PRESS
ASOC 2858 1–11
A. Khan, M.A. Jaffar / Applied Soft Computing xxx (2015) xxx–xxx
K
540
segmentation Ss , and ppq = 1/K k=1 T (lpSk = lqSk ) is related to the probability that p and q has the same label. Probabilistic rand index (PRI) compares the segmentation results generated by a particular algorithm with a number of human segmented images. The comparison is performed by inducing the variability in human segmented images through variable weights of pixels pairs. The value of PRI lies in the range of [0, 1], the larger the value, the better is the result. Table 2 shows the average scores of PRI and VoI measures for the 8 algorithms on 300 images in the Berkeley segmentation database (BSD). It is clear from Table 2 that the proposed method performs better than the other state-of-the-art algorithms in terms of VoI and PRI values. The average PRI values of LDC and EG are close to the proposed algorithm but their average VoI values are much greater. Similarly the average VoI values of the SRM and RADp are nearly approaching the proposed algorithm but their average PRI scores are far away from the desired target. Moreover, KGC is not able to reach the proposed method in average PRI nor in average VoI scores. It is clear from Table 2 that LS and NC have very poor results in term of both PRI and VoI values. The performance of each of the eight algorithms, on each image in the Berkeley segmentation database (BSD), are graphically represented in Figs. 10 and 11. Fig. 10 graphically depicts the PRI values of the given eight algorithms for 300 images in the Berkeley segmentation database. Whereas the VoI scores of the given eight algorithms for 300 images in the Berkeley segmentation database are shown in Fig. 11. It is clear from the curves in Figs. 10 and 11 that the proposed algorithm performs better than the other stateof-the-art methods.
541
6. Conclusion
512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539
560
In this paper the color image segmentation is performed by a fixed length genetic algorithm (GA). A self organizing map (SOM) is used to determine the number of segments in order to set length of a chromosome. A new objective function based on compactness and separation is proposed to asses the quality of the solution. Note that, an opposition based strategy is used to initialize the GA to the productive search areas. The new objective function and opposition based initialize strategy make the GA to come out of the local optima and produce best results. In some cases the given genetic algorithm makes the small regions of an image as separate segments and leads to a nearly noisy segmentation. A simple ad hoc mechanism is adopted to refine segmentation by merging the small segments to their matching neighbors. The results of the proposed method are compared with eight state-of-the-art image segmentation algorithms. The qualitative and quantitative results demonstrate that the proposed algorithm performs better than the other eight algorithms. In the future, the texture and geometrical features of the objects will be combined with color features to perform robust segmentation.
561
References
542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559
562 563
[1] L. Lucchese, S.K. Mitra, Color image segmentation: a state of the art survey, Proc. Indian Natl. Sci. Acad. Image Process. Vis. Pattern Recogn. 67 (2) (2001) 207–221.
11
[2] E. Vazquez, J.V.D. Weijer, R. Baldrich, Image segmentation in the presence of shadows and highlights using a ridge based histogram analysis, in: Proc. ECCV08, vol. 5305, 2008, pp. 1–14. [3] P.K. Loo, C.L. Tan, Adaptive region growing color segmentation for text using irregular pyramid, in: International Workshop on Document Analysis Systems, vol. 3163, 2004, pp. 264–275. [4] M. Kass, A. Witkin, D. Terzopoulos, Snakes: active contour models, Int. J. Comput. Vis. 1 (4) (1988) 321–331. [5] D. Cohen, et al., On active contour models and balloons, CVGIP: Image UnderQ6 stand. 53 (2) (1991) 211–218. [6] Xu. Chenyang, J. Prince, Snakes, shapes, and gradient vector flow, IEEE Trans. Q7 Image Process. 7 (1998) 3. [7] E. Mortensen, W. Barrett, Interactive segmentation with intelligent scissors, Graph. Models Image Process. 60 (1998) 5. [8] J. Besag, On the statistical analysis of dirty pictures, J. Roy. Stat. Soc. B 48 (1986) 259–302. [9] Y.B. Kov, M. Jolly, Interactive graph cuts for optimal boundary region segmentation of objects in N-D images, IEEE Int. Conf. Comp. Vis. 1 (2001) 105–112. [10] V. Kolmogorov, R. Zabih, What energy functions can be minimized via graph cuts ? IEEE Trans. Pattern Anal. Mach. Intell. 26 (2) (2004) 147–159. [11] C. Rother, V. Kolmogorov, A. Blake, Grabcut interactive foreground extraction using iterated graphcuts, ACM Trans. Graph. 23 (3) (2004) 309–314. [12] Y. Li, J. Sun, C.K. Tang, H.Y. Shum, Lazy snapping, ACM Trans. Graph. 23 (3) (2004) 303–308. [13] S. Beucher, F. Meyer, The morphological approach to segmentation: the watershed transformation, Math. Morphol. Image Process. 34 (1993) 433–481. [14] L. Vincent, P. Soille, Watersheds in digital spaces: an efficient algorithm based on immersion simulations, IEEE Trans. Pattern Anal. Mach. Intell. 13 (6) (1999) 583–598. [15] R. Duda, P. Hart, D. Stork, Pattern Classification, 2nd ed., Wiley, Hoboken, NJ, 2001. [16] M.N. Ahmed, S.M. Yamany, A.A.F. Nevin Mohamed, T. Moriarty, A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI data, IEEE Trans. Med. Imaging 21 (March (3)) (2002) 193–199. [17] K.S. Chuanga, et al., Fuzzy c-means clustering with spatial information for image segmentation, Comp. Med. Imaging Graph. 30 (2006) 9–15. [18] A. Khan, M.A. Jaffar, T.S. Choi, Som and fuzzy based color image segmentation, Multimedia Tools Appl. 64 (2) (2013) 331–344. [19] L. Vendramin, R.J.G.B. Campello, E.R. Hruschka, Relative clustering validity criteria: a comparative overview, Stat. Anal. Data Min. 3 (4) (2010) 209–235. [20] D.E. Ilea, P.F. Whelan, Ctex – an adaptive unsupervised segmentation algorithm based on color-texture coherence, IEEE Trans. Image Process. 17 (10) (2008) 1926–1939. [21] D. Martin, C. Fowlkes, D. Tal, J. Malik, A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, in: IEEE International Conference on Computer Vision, vol. 2, 2001, pp. 416–423. [22] A. Khan, M.A. Jaffar, L. Shao, A modified adaptive differential evolution algorithm for color image segmentation, Knowl. Inform. Syst. (2014) 1233–1243. [23] S. Rahnamayan, H.R. Tizhoosh, M.M.A. Salama, Opposition-based differential evolution, IEEE Trans. Evol. Comput. 12 (1) (2008) 64–79. [24] J.C. Bezdek, Cluster validity with fuzzy sets, J. Cybern. 3 (3) (1973) 58–73. [25] K.S. Chuang, H.L. Tzeng, S. Chen, T.J. Chen, Fuzzy c-means clustering with spatial information for image segmentation, Comp. Med. Imaging Graph. 30 (2006) 9–15. [26] R. Nock, F. Nielsen, Statistical region merging, IEEE Trans. Pattern Anal. Mach. Intell. 26 (11) (2004) 1452–1458. [27] A.Y. Yang, J. Wright, S.S. Sastry, Y. Ma, Unsupervised segmentation of natural images via lossy data compression, Comp. Vis. Image Understand. (2008) 212–225. [28] M. Ben Salah, A. Mitiche, I.B. Ayed, Multiregion image segmentation by parametric kernel graph cuts, IEEE Trans. Image Process. 20 (2) (2011) 545–557. [29] C. Li, R. Huang, Z. Ding, J.C. Gatenby, D.N. Metaxas, J.C. Gore, A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI, IEEE Trans. Image Process. 20 (7) (2011) 2007–2016. [30] P. Felzenszwalb, D. Huttenlocher, Efficient graph-based image segmentation, Int. J. Comput. Vis. 59 (02) (2004) 167–181. [31] J. Shi, J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell. 22 (8) (2000) 888–905. [32] M. Meila, Comparing clusterings by the variation of information, Lect. Note Comp. Sci. 2777 (2003) 173–187. [33] R. Unnikrishnan, M. Hebert, Measures of similarity, in: Proceedings of the IEEE Workshop on Computer Vision Applications, vol. 1, 2005, pp. 394–401.
Please cite this article in press as: A. Khan, M.A. Jaffar, Genetic algorithm and self organizing map based fuzzy hybrid intelligent method for color image segmentation, Appl. Soft Comput. J. (2015), http://dx.doi.org/10.1016/j.asoc.2015.03.029
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637