- Email: [email protected]
Rule-based automatic segmentation of color images
Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442 www.elsevier.de/aeue
Rule-based automatic segmentation of color images Recep Demirci∗ Electrical Education Department, Technical Education Faculty, Abant Izzet Baysal University, Beciyoruk Kampusu, 81620 Duzce, Turkey Received 3 April 2005
Abstract A rule-based segmentation algorithm for color images has been presented in this paper. The proposed strategy is similar to region growing algorithm where the seed points are automatically selected and grown. The similarity percents of neighboring pixels are calculated by means of fuzzy reasoning rules, and the merging of the pixels with regions is performed by comparing the similarity percent with the similarity threshold value. The algorithm does not require any prior knowledge of the number of regions existing in the image and decreases the computational load required for the fuzzy c-means (FCM). Several computer simulations have been performed and the results have been discussed. The simulation results indicate that the proposed algorithm yields segmented color image of perfect accuracy. 䉷 2005 Elsevier GmbH. All rights reserved. Keywords: Rule-based image segmentation; Color image; Fuzzy logic; Similarity percent threshold
1. Introduction The computer vision branch of the field of artificial intelligence is concerned with developing algorithms for analyzing the content of an image. In other words, it is an attempt to find out what is in the picture. This process is generally called pattern recognition. Given a digitized image containing a number of objects, the pattern recognition process consists of three major phases. The first phase is called image segmentation or object isolation, where each object is found and its image is isolated from the rest of the scene. The second phase is called feature extraction. This is where the object is measured. A measurement is the value of some quantifiable property of an object. A feature is a function of one or more measurements, computed so that it quantifies some important characteristic of the object. The feature extraction process produces a set of features that, taken together, comprises the feature vector. The third phase of
∗ Tel.: +90 380 5413344; fax :+90 380 5413184.
E-mail address: [email protected]. 1434-8411/$ - see front matter 䉷 2005 Elsevier GmbH. All rights reserved. doi:10.1016/j.aeue.2005.09.002
pattern recognition is classification. Its output is merely a decision regarding the class to which each object belongs. Each object is recognized as being of one particular type, and the recognition is implemented as a classification process [1,2]. As could be seen, image segmentation is the first step in pattern recognition and image analysis. It is a critical and essential component of image analysis system and one of the most difficult tasks in image processing, and determines the quality of the final result of analysis. Segmentation is basically clustering of the pixels in the image according to some criteria. The main aim is to recognize homogeneous regions within an image as distinct and belonging to different objects. Segmentation phase does not worry about the identity of the objects. They can be labeled later. The segmentation process can be based on finding the maximum homogeneity in gray levels within the regions identified. There are several issues related to image segmentation that require detailed review. One of the common problems encountered in image segmentation is to choose a suitable approach for isolating different objects from the background and each other. The segmentation does not perform well if
436
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
the gray levels of different objects are quite similar. Issues related to segmentation involve choosing good segmentation algorithms, measuring their performance, and understanding their impact on the scene analysis system. Image segmentation has been approached from a wide variety of perspectives: histogram thresholding, edge-based segmentation, three graph-based approaches, region growing, clustering, probabilistic or Bayesain approaches, neural networks for segmentation, and other approaches [3–5]. Among these, the fuzzy c-means (FCM), region growing, histogram thresholding techniques have received significant attentions [6]. In segmentation strategies with the histogram analysis, choosing threshold points has always been a problem. Although Lim et al. [7] proposed a method based on derivative and convolution of the histogram of image and scale-space filter in order to find the threshold point, it has still computational complexity. Region growing algorithms take one or more pixels, called seeds, and grow in the regions around them based upon a certain homogeneity criteria. If the adjoining pixels are similar to the seed, they are merged with them within a single region. The process continues until all the pixels in the image are assigned to one or more regions. The selection of seeds can be automatically or manually done. The automated selection can be based on finding pixels that are of interest. Alternatively, they could be determined from the peaks found in an image histogram. On the other hand, seeds could be selected manually for each object present in the image [8]. Image segmentation could be performed effectively by clustering image pixels. For any given finite data set X of objects, the problem of clustering in X is to assign object labels that identify natural subgroups in the set. Because the data are unlabeled, this problem is often called unsupervised learning. Here, the word learning refers to learning the correct labels for good subgroups. The objective is the partition of X into a certain number (c) of natural and homogeneous subsets, where the elements of each set are as similar as possible to each other and at the same time, as different from those of the other sets as possible. The number (c) can be fixed beforehand, or may result as a consequence of physical or mathematical constraints [9–11]. The FCM is one of the best-known clustering techniques. However, it is widely recognized that the clustering techniques including FCM, suffer from problems related to (1) adjacent clusters frequently overlapping in color space, causing incorrect pixel classification; and (2) clustering being more difficult when the number of clusters is unknown, as is typical for segmentation applications. Especially, problem (2) is critical to the success of FCM (i.e. it requires prior knowledge of the number of regions existing in the image). Otherwise, false results may be the case. Also this problem is at the heart of the cluster validity question. Moreover, the FCM has a problem of exhaustive computational burden [12–15]. In this paper, a segmentation algorithm for color image based on the threshold of similarity percent of neighbor-
ing pixels is presented. The proposed segmentation algorithm uses rule-based segmentation concept and therefore reduces the computational complexity required by the FCM and histogram-based segmentation algorithms.
2. Proposed algorithm The proposed strategy is similar to the region growing algorithm where the seed points are automatically selected and grown. The algorithm starts with choosing the pixel at the upper left corner of the image as a seed and grows. Then the next seed is selected from pixels which are not assigned to any of previous regions. It continues from left to right pixels for successive lines. The similarity percent of the pixels are calculated in terms of fuzzy rules. The merging of the pixels with regions is performed by comparing the similarity percent with threshold value. Accordingly, the final decision whether the pixel belongs to the region is made, and the pixel is combined.
2.1. Pixel membership The proposed color image segmentation method could be described as a system whose inputs are a color image and the similarity percent threshold value ST (ST,R , ST,G , ST,B ), as shown in Fig. 1. The outputs of the system are a segmented image, the region numbers and region’s pixel counts. As each pixel has three color components, the similarity percent is calculated for each component independently. Therefore, the system has three threshold inputs. In this study, the same similarity percent threshold for each color has been selected. It is well known that the outputs of a system depend on the input variables. For the time-invariant images, the input variables could be reduced into single variable: ST . Therefore, the user or operator of the automatic image segmentation system in our case must provide the necessary input variables although they could be fixed in advance. For the proposed algorithm, a user graphic interface in Windows has been developed and a scrollbar has been assigned for ST . Therefore, the sample images have been easily tested many times with different ST values. An image consists of pixels, which are neighbors of each other. Let us consider two neighbor pixels, which are in an image, as shown in Fig. 2.
Fig. 1. Automatic image segmentation system.
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
437
Fig. 2. Neighboring pixels in a color image.
Fig. 5. Membership functions for similarity percent of red component.
Table 1. Fuzzy rule table for red component
R,1 Fig. 3. Membership functions for gray levels of red component of pixel 1.
Fig. 4. Membership functions for gray levels of red component of pixel 2.
We define membership functions for the gray levels of each color component of pixels P1 and P2 as shown in Figs. 3 and 4, respectively. Subsequently, gray levels for each color components have been partitioned by five linguistic values, which are Not Red:NR, Little Red:LR, Medium Red:MR, Quite Red:QR and Pure Red:PR for red; Not Green:NG, Little Green:LG, Medium Green:MG, Quite Green:QG and Pure Green:PG for green and; Not Blue:NB, Little Blue:LB, Medium Blue:MB, Quite Blue:QB and Pure Blue:PB for blue. In the same way, the membership functions for similarity percents of each color component of P1 and P2 were also defined as shown in Fig. 5. Five linguistic values for similarity percent have been assigned, which are Not Similar:NS,
NR LR MR QR PR
R,2 NR
LR
MR
QR
PR
ES QS MS LS NS
QS ES QS MS LS
MS QS ES QS MS
LS MS QS ES QS
NS LS MS QS ES
Little Similar:LS, Medium Similar:MS, Quite Similar:QS and Exactly Similar:ES. Generally a fuzzy system is a static mapping between its inputs and outputs. For a fuzzy system, the mapping of the inputs to the outputs is characterized by a set of condition–action rules, or in modus-pones (If–Then) form; If premise then consequent. Generally, the inputs of the fuzzy system are associated with the premise, and the outputs are associated with the consequences. These if–then rules can be represented in many forms. Multi-input multi-output (MIMO) and multi-input single-output (MISO) are some of standard forms. The MISO form of linguistic rules in our case is as follows: Rule 1: If P1 is Little Red and P2 is Little Red, Then P1 and P2 are Exactly Similar in terms of red component, Rule 2: If P1 is Not Red and P2 is Little Red, Then P1 and P2 are Quite Similar in terms of red component, Rule 3: If P1 is Medium Red and P2 is Pure Red, Then P1 and P2 are Medium Similar in terms of red component, so on. As a result, rules to compare neighboring pixels in the image have been constructed according to Table 1. When the center-average defuzzification and product to represent the conjunctions in the premise are used, then the similarity percent in terms of red components of P1 and P2 could be
438
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
explicitly represented: R j j j =1 SR prem (LR ) . SR = R j j =1 prem (LR )
(1)
where R is the total number of the rules for red component, j and prem (LR ) is the certainty of the premise of the jth rule j for red component. SR is center of the similarity percent membership function for the jth rule for red component. The certainty of the premise is j
j
jprem (LR ) = R,1 (LR,1 ) · R,2 (LR,2 ).
(2)
As there are five linguistic variables, 25 rules in total were established for each color component. The similarity percentages in terms of green and blue components of P1 and P2 have been individually calculated in the same way.
2.2. Similarity percent thresholding Once similarity percent of the neighboring pixels is calculated, it is compared with the similarity percent threshold value ST to decide whether the neighboring pixel belongs to the same region as shown in Fig. 6. Since the proposed technique is non-supervised, we do not have any pre-information on image or any labeled sample of the segments, the clustering algorithm attempts to partition the image based on criteria, which is threshold value ST (ST,R , ST,G , ST,B ). Then the algorithm produces the segmented image depending on ST . The larger the threshold value ST selected, the more the homogeneous subimages obtained. On the other hand, the number of subimages is increased too. The volume of similarity cube CS gives homogeneity of regions in the image. The lesser the volume of similarity cube CS used, the more the homogeneous regions obtained. The one advantage of this approach is that the homogeneity of any color
Fig. 7. A pixel in image: (a) spatial coordinates and (b) mask used for similarity percent.
component in the region could also be separately chosen. Consequently, dominant colors in the region could be made dominant while segmenting the image. However, if the similarity percent threshold values are made the same, there would be no dominant colors while segmenting the image.
2.3. Region labeling
Fig. 6. Similarity cube.
A pixel and its neighbors in an image could be represented as shown in Fig. 7(a). The value of similarity percent is calculated for all neighbors. In the proposed algorithm, a dynamic array which has the same size of the image is used to hold the region numbers of pixels. Initially, the array is filled with −1, which means the pixel has not assigned to any region yet and the similarity percent must be calculated. If the value in the array is positive, it means the corresponding pixel has already been assigned to any of previous region and there is no need to further consider it. The direction of calculation of similarity percent is the same as clockwise direction.
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
Once similarity percent value for any of neighboring pixels is calculated, it is compared with the similarity percent threshold value ST to decide whether the neighboring pixel P2 belongs to the same region. If true, then the region number of P1 is assigned to P2 . However, if it is not
439
the case, the next pixel is checked and a new region is created, and so on. Accordingly, the mask shown in Fig. 7(b) is moved through the image from left to right and from top to bottom. The proposed algorithm could be described as follows:
Do begin define global SR , SG , SB ; define global, ST,R ST,G ST,B for thresholds define global, c as a region counter; define global array, R[M − 1][N − 1] for labels; fill R[M − 1][N − 1] with −1; set ST,R ST,G ST,B by default or user selected; set c = 0; do for x = 0 to M − 1 begin do for y = 0 to N − 1 begin c = c + 1; Call Subroutine Connect Pixel (x, y, c) end end Subroutine Connect Pixel (x, y, c) begin if R[x][y] < 0 then begin set R[x][y] = c; Call Subroutine Fuzzy, S (x, y + 1) if SR (x, y + 1) > ST,R and SG (x, y + 1) > ST,G and SB (x, y + 1) > ST,B then begin Call Subroutine Connect Pixel (x, y + 1, c) end Call Subroutine Fuzzy, S (x + 1, y + 1) if SR (x + 1, y + 1) > ST,R and SG (x + 1, y + 1) > ST,G and SB (x + 1, y + 1) > ST,B then begin Call Subroutine Connect Pixel (x + 1, y + 1, c) end Call Subroutine Fuzzy, S (x + 1, y); if SR (x + 1, y) > ST,R and SG (x + 1, y) > ST,G and SB (x + 1, y) > ST,B then begin Call Subroutine Connect Pixel (x + 1, y, c) end Call Subroutine Fuzzy, S (x + 1, y − 1); if SR (x + 1, y − 1) > ST,R and SG (x + 1, y − 1) > ST,G and SB (x + 1, y − 1) > ST,B then begin Call Subroutine Connect Pixel (x + 1, y − 1, c) end Call Subroutine Fuzzy, S (x, y − 1) if SR (x, y − 1) > ST,R and SG (x, y − 1) > ST,G and SB (x, y − 1) > ST,B then begin Call Subroutine Connect Pixel (x, y − 1, c) end
440
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
Call Subroutine Fuzzy, S (x − 1, y − 1) if SR (x − 1, y − 1) > ST,R and SG (x − 1, y − 1) > ST,G and SB (x − 1, y − 1) > ST,B then begin Call Subroutine Connect Pixel (x − 1, y − 1, c) end Call Subroutine Fuzzy, S (x − 1, y) if SR (x − 1, y) > ST,R and SG (x − 1, y) > ST,G and SB (x − 1, y) > ST,B then begin Call Subroutine Connect Pixel (x − 1, y, c) end Call Subroutine Fuzzy, S (x − 1, y + 1) if SR (x − 1, y + 1) > ST,R and SG (x − 1, y + 1) > ST,G and SB (x − 1, y + 1) > ST,B then begin Call Subroutine Connect Pixel (x − 1, y + 1, c) end end else return end
Fig. 8. Lena: (a) color image of size 256 × 256 with 255 gray levels, (b) some of regions, (c) variation of maximum region number with ST and (d) region histogram of segmented image: ST = 70.
3. Simulation results and discussion The proposed algorithm was tested with the well-known Lena image shown in Fig. 8(a), which is a color image of size 256 × 256 with 255 gray levels. As ST is the input variable of the image segmentation system, the maximum region number of the segmented image varies with ST as shown in Fig. 8(c) where the maximum region number increases with ST .
Although each point on the curve in Fig. 8(c) corresponds different segmented image and region count, the segmented image shown in Fig. 8(b), when the ST is selected to be 70, consists of 215 regions. The region histogram shown in Fig. 8(d) was also obtained while the ST was set to 70. Image of electronic components shown in Fig. 9(a) was also tested with the proposed algorithm. Fig. 9(c) shows the variation of total number of regions with the similarity threshold while the region histogram is shown in Fig. 9(d).
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
441
Fig. 9. Electronic components: (a) color image of electronic circuit components, (b) some of segmented letters, (c) variation of maximum region number with ST and (d) region histogram of segmented image: ST = 80.
The image was segmented 794 regions when ST was equal to 80. The symbol of integrated circuit, letters: ‘M’, ‘9’, ‘K’, ‘L’, ‘M’, ‘4’, ‘B’, ‘8’ and ‘2’ are segmented regions of the image as shown in Fig. 9(b). It has been noticed from applications that the most of regions have between one and ten pixels. This shows sensitivity of the algorithm which means that any object consisting of even a single pixel could be isolated from the rest of the image. It could be essential for some particular applications. On the other hand, it may not be necessary for coarse applications where the merging of tiny regions with other big regions is required. The bigger the ST chosen, the more the homogeneity within subimages attained and at the same time the more the number of the subimages obtained. Consequently, the number of regions and homogeneity are inversely related.
4. Conclusion A new method for segmentation of color images was described. The proposed approach is similar to region growing algorithm where the seed points are automatically selected and grown. The similarity percent of neighboring pixels is
calculated by means of fuzzy reasoning rules and the merging of the pixels with regions is performed by comparing the similarity percent with the similarity threshold value. The algorithm does not require any prior knowledge of number of regions existing in the image and decreases the computational load required by other segmentation algorithms since the complex computations such as statistical calculations, gradients, Laplace and Green’s functions were not used. On the other hand, a drawback of the proposed method for segmentation is that an array of the same size as the image is required and some regions with one pixel may be created. The further research to avoid formation of regions with one pixel and different applications are still in progress.
References [1] Gonzalez RC, Woods RE. Digital image processing. Cambridge, MA: Addison-Wesley; 1993. [2] Castleman KR. Digital image processing. Englewood Cliffs, NJ: Prentice-Hall; 1996. [3] Cheng HD, Jiang XH, Sun Y, Wang J. Color image segmentation: advances and prospects. Pattern Recognition 2001;34:2259–81.
442
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
[4] Pal NR, Pal SK. A review on image segmentation techniques. Pattern Recognition 1993;26:1277–94. [5] Fu KS, Mui JK. A survey on image segmentation. Pattern Recognition 1981;13:3–16. [6] Bezdek JC, Trevide MM. Low level segmentation of aerial images with fuzzy clustering. IEEE Trans Systems Man Cybern 1986;SMC-16:589–98. [7] Lim YW, Lee SU. On the color image segmentation algorithm based on the thresholding and the fuzzy c-means technique. Pattern Recognition 1990;23(9). [8] Chang YL, Li X. Adaptive image region growing. IEEE Trans Image Process 1994;3(6):868–73. [9] Zadeh LA. Fuzzy sets. Inf Control 1965;8:338–53. [10] Ross TJ. Fuzzy logic with engineering application. New York: McGraw-Hill; 1995. [11] Pal SK, King RA. Image enhancement using smoothing with fuzzy sets. IEEE Trans Systems Man Cybern 1981;SMC11(7):494–501. [12] Pal SK, Rosenfeld A. Image enhancement and threshold by optimization of fuzzy compactness. Pattern Recognition Lett 1988;7:77–86. [13] Paclik P, Duin RPW, Kempen GMP, Kohlus R. Segmentation of multi-spectral images using the combined classifier approach. Image Vision Comput 2003;21:473–82.
[14] Cheng YC, Chen SY. Image classification using color, texture and regions. Image Vision Comput 2003;21:759–76. [15] Mahmood TS, Petkovic D. On describing color and shape information in images. Signal Process Image Commun 2000;16:15–31. Recep Demirci was born in K. Maras, Turkey in 1968. He got his B.Sc. degree from the Gazi University in 1990 where he studied electrical education. He was a visiting researcher at the Purdue University, USA in 1992. He has got his Ph.D. degree from the Electrical Engineering Department, University of Wales Cardiff, UK in 1998. He completed his military service as a Lecturer at Engineering Brigade in Izmir between 1998 and 2000. He has been working as an Assistant Professor at Electrical Education Department, Technical Education Faculty, Abant Izzet Baysal University since 2000.
Rule-based automatic segmentation of color images Recep Demirci∗ Electrical Education Department, Technical Education Faculty, Abant Izzet Baysal University, Beciyoruk Kampusu, 81620 Duzce, Turkey Received 3 April 2005
Abstract A rule-based segmentation algorithm for color images has been presented in this paper. The proposed strategy is similar to region growing algorithm where the seed points are automatically selected and grown. The similarity percents of neighboring pixels are calculated by means of fuzzy reasoning rules, and the merging of the pixels with regions is performed by comparing the similarity percent with the similarity threshold value. The algorithm does not require any prior knowledge of the number of regions existing in the image and decreases the computational load required for the fuzzy c-means (FCM). Several computer simulations have been performed and the results have been discussed. The simulation results indicate that the proposed algorithm yields segmented color image of perfect accuracy. 䉷 2005 Elsevier GmbH. All rights reserved. Keywords: Rule-based image segmentation; Color image; Fuzzy logic; Similarity percent threshold
1. Introduction The computer vision branch of the field of artificial intelligence is concerned with developing algorithms for analyzing the content of an image. In other words, it is an attempt to find out what is in the picture. This process is generally called pattern recognition. Given a digitized image containing a number of objects, the pattern recognition process consists of three major phases. The first phase is called image segmentation or object isolation, where each object is found and its image is isolated from the rest of the scene. The second phase is called feature extraction. This is where the object is measured. A measurement is the value of some quantifiable property of an object. A feature is a function of one or more measurements, computed so that it quantifies some important characteristic of the object. The feature extraction process produces a set of features that, taken together, comprises the feature vector. The third phase of
∗ Tel.: +90 380 5413344; fax :+90 380 5413184.
E-mail address: [email protected]. 1434-8411/$ - see front matter 䉷 2005 Elsevier GmbH. All rights reserved. doi:10.1016/j.aeue.2005.09.002
pattern recognition is classification. Its output is merely a decision regarding the class to which each object belongs. Each object is recognized as being of one particular type, and the recognition is implemented as a classification process [1,2]. As could be seen, image segmentation is the first step in pattern recognition and image analysis. It is a critical and essential component of image analysis system and one of the most difficult tasks in image processing, and determines the quality of the final result of analysis. Segmentation is basically clustering of the pixels in the image according to some criteria. The main aim is to recognize homogeneous regions within an image as distinct and belonging to different objects. Segmentation phase does not worry about the identity of the objects. They can be labeled later. The segmentation process can be based on finding the maximum homogeneity in gray levels within the regions identified. There are several issues related to image segmentation that require detailed review. One of the common problems encountered in image segmentation is to choose a suitable approach for isolating different objects from the background and each other. The segmentation does not perform well if
436
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
the gray levels of different objects are quite similar. Issues related to segmentation involve choosing good segmentation algorithms, measuring their performance, and understanding their impact on the scene analysis system. Image segmentation has been approached from a wide variety of perspectives: histogram thresholding, edge-based segmentation, three graph-based approaches, region growing, clustering, probabilistic or Bayesain approaches, neural networks for segmentation, and other approaches [3–5]. Among these, the fuzzy c-means (FCM), region growing, histogram thresholding techniques have received significant attentions [6]. In segmentation strategies with the histogram analysis, choosing threshold points has always been a problem. Although Lim et al. [7] proposed a method based on derivative and convolution of the histogram of image and scale-space filter in order to find the threshold point, it has still computational complexity. Region growing algorithms take one or more pixels, called seeds, and grow in the regions around them based upon a certain homogeneity criteria. If the adjoining pixels are similar to the seed, they are merged with them within a single region. The process continues until all the pixels in the image are assigned to one or more regions. The selection of seeds can be automatically or manually done. The automated selection can be based on finding pixels that are of interest. Alternatively, they could be determined from the peaks found in an image histogram. On the other hand, seeds could be selected manually for each object present in the image [8]. Image segmentation could be performed effectively by clustering image pixels. For any given finite data set X of objects, the problem of clustering in X is to assign object labels that identify natural subgroups in the set. Because the data are unlabeled, this problem is often called unsupervised learning. Here, the word learning refers to learning the correct labels for good subgroups. The objective is the partition of X into a certain number (c) of natural and homogeneous subsets, where the elements of each set are as similar as possible to each other and at the same time, as different from those of the other sets as possible. The number (c) can be fixed beforehand, or may result as a consequence of physical or mathematical constraints [9–11]. The FCM is one of the best-known clustering techniques. However, it is widely recognized that the clustering techniques including FCM, suffer from problems related to (1) adjacent clusters frequently overlapping in color space, causing incorrect pixel classification; and (2) clustering being more difficult when the number of clusters is unknown, as is typical for segmentation applications. Especially, problem (2) is critical to the success of FCM (i.e. it requires prior knowledge of the number of regions existing in the image). Otherwise, false results may be the case. Also this problem is at the heart of the cluster validity question. Moreover, the FCM has a problem of exhaustive computational burden [12–15]. In this paper, a segmentation algorithm for color image based on the threshold of similarity percent of neighbor-
ing pixels is presented. The proposed segmentation algorithm uses rule-based segmentation concept and therefore reduces the computational complexity required by the FCM and histogram-based segmentation algorithms.
2. Proposed algorithm The proposed strategy is similar to the region growing algorithm where the seed points are automatically selected and grown. The algorithm starts with choosing the pixel at the upper left corner of the image as a seed and grows. Then the next seed is selected from pixels which are not assigned to any of previous regions. It continues from left to right pixels for successive lines. The similarity percent of the pixels are calculated in terms of fuzzy rules. The merging of the pixels with regions is performed by comparing the similarity percent with threshold value. Accordingly, the final decision whether the pixel belongs to the region is made, and the pixel is combined.
2.1. Pixel membership The proposed color image segmentation method could be described as a system whose inputs are a color image and the similarity percent threshold value ST (ST,R , ST,G , ST,B ), as shown in Fig. 1. The outputs of the system are a segmented image, the region numbers and region’s pixel counts. As each pixel has three color components, the similarity percent is calculated for each component independently. Therefore, the system has three threshold inputs. In this study, the same similarity percent threshold for each color has been selected. It is well known that the outputs of a system depend on the input variables. For the time-invariant images, the input variables could be reduced into single variable: ST . Therefore, the user or operator of the automatic image segmentation system in our case must provide the necessary input variables although they could be fixed in advance. For the proposed algorithm, a user graphic interface in Windows has been developed and a scrollbar has been assigned for ST . Therefore, the sample images have been easily tested many times with different ST values. An image consists of pixels, which are neighbors of each other. Let us consider two neighbor pixels, which are in an image, as shown in Fig. 2.
Fig. 1. Automatic image segmentation system.
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
437
Fig. 2. Neighboring pixels in a color image.
Fig. 5. Membership functions for similarity percent of red component.
Table 1. Fuzzy rule table for red component
R,1 Fig. 3. Membership functions for gray levels of red component of pixel 1.
Fig. 4. Membership functions for gray levels of red component of pixel 2.
We define membership functions for the gray levels of each color component of pixels P1 and P2 as shown in Figs. 3 and 4, respectively. Subsequently, gray levels for each color components have been partitioned by five linguistic values, which are Not Red:NR, Little Red:LR, Medium Red:MR, Quite Red:QR and Pure Red:PR for red; Not Green:NG, Little Green:LG, Medium Green:MG, Quite Green:QG and Pure Green:PG for green and; Not Blue:NB, Little Blue:LB, Medium Blue:MB, Quite Blue:QB and Pure Blue:PB for blue. In the same way, the membership functions for similarity percents of each color component of P1 and P2 were also defined as shown in Fig. 5. Five linguistic values for similarity percent have been assigned, which are Not Similar:NS,
NR LR MR QR PR
R,2 NR
LR
MR
QR
PR
ES QS MS LS NS
QS ES QS MS LS
MS QS ES QS MS
LS MS QS ES QS
NS LS MS QS ES
Little Similar:LS, Medium Similar:MS, Quite Similar:QS and Exactly Similar:ES. Generally a fuzzy system is a static mapping between its inputs and outputs. For a fuzzy system, the mapping of the inputs to the outputs is characterized by a set of condition–action rules, or in modus-pones (If–Then) form; If premise then consequent. Generally, the inputs of the fuzzy system are associated with the premise, and the outputs are associated with the consequences. These if–then rules can be represented in many forms. Multi-input multi-output (MIMO) and multi-input single-output (MISO) are some of standard forms. The MISO form of linguistic rules in our case is as follows: Rule 1: If P1 is Little Red and P2 is Little Red, Then P1 and P2 are Exactly Similar in terms of red component, Rule 2: If P1 is Not Red and P2 is Little Red, Then P1 and P2 are Quite Similar in terms of red component, Rule 3: If P1 is Medium Red and P2 is Pure Red, Then P1 and P2 are Medium Similar in terms of red component, so on. As a result, rules to compare neighboring pixels in the image have been constructed according to Table 1. When the center-average defuzzification and product to represent the conjunctions in the premise are used, then the similarity percent in terms of red components of P1 and P2 could be
438
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
explicitly represented: R j j j =1 SR prem (LR ) . SR = R j j =1 prem (LR )
(1)
where R is the total number of the rules for red component, j and prem (LR ) is the certainty of the premise of the jth rule j for red component. SR is center of the similarity percent membership function for the jth rule for red component. The certainty of the premise is j
j
jprem (LR ) = R,1 (LR,1 ) · R,2 (LR,2 ).
(2)
As there are five linguistic variables, 25 rules in total were established for each color component. The similarity percentages in terms of green and blue components of P1 and P2 have been individually calculated in the same way.
2.2. Similarity percent thresholding Once similarity percent of the neighboring pixels is calculated, it is compared with the similarity percent threshold value ST to decide whether the neighboring pixel belongs to the same region as shown in Fig. 6. Since the proposed technique is non-supervised, we do not have any pre-information on image or any labeled sample of the segments, the clustering algorithm attempts to partition the image based on criteria, which is threshold value ST (ST,R , ST,G , ST,B ). Then the algorithm produces the segmented image depending on ST . The larger the threshold value ST selected, the more the homogeneous subimages obtained. On the other hand, the number of subimages is increased too. The volume of similarity cube CS gives homogeneity of regions in the image. The lesser the volume of similarity cube CS used, the more the homogeneous regions obtained. The one advantage of this approach is that the homogeneity of any color
Fig. 7. A pixel in image: (a) spatial coordinates and (b) mask used for similarity percent.
component in the region could also be separately chosen. Consequently, dominant colors in the region could be made dominant while segmenting the image. However, if the similarity percent threshold values are made the same, there would be no dominant colors while segmenting the image.
2.3. Region labeling
Fig. 6. Similarity cube.
A pixel and its neighbors in an image could be represented as shown in Fig. 7(a). The value of similarity percent is calculated for all neighbors. In the proposed algorithm, a dynamic array which has the same size of the image is used to hold the region numbers of pixels. Initially, the array is filled with −1, which means the pixel has not assigned to any region yet and the similarity percent must be calculated. If the value in the array is positive, it means the corresponding pixel has already been assigned to any of previous region and there is no need to further consider it. The direction of calculation of similarity percent is the same as clockwise direction.
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
Once similarity percent value for any of neighboring pixels is calculated, it is compared with the similarity percent threshold value ST to decide whether the neighboring pixel P2 belongs to the same region. If true, then the region number of P1 is assigned to P2 . However, if it is not
439
the case, the next pixel is checked and a new region is created, and so on. Accordingly, the mask shown in Fig. 7(b) is moved through the image from left to right and from top to bottom. The proposed algorithm could be described as follows:
Do begin define global SR , SG , SB ; define global, ST,R ST,G ST,B for thresholds define global, c as a region counter; define global array, R[M − 1][N − 1] for labels; fill R[M − 1][N − 1] with −1; set ST,R ST,G ST,B by default or user selected; set c = 0; do for x = 0 to M − 1 begin do for y = 0 to N − 1 begin c = c + 1; Call Subroutine Connect Pixel (x, y, c) end end Subroutine Connect Pixel (x, y, c) begin if R[x][y] < 0 then begin set R[x][y] = c; Call Subroutine Fuzzy, S (x, y + 1) if SR (x, y + 1) > ST,R and SG (x, y + 1) > ST,G and SB (x, y + 1) > ST,B then begin Call Subroutine Connect Pixel (x, y + 1, c) end Call Subroutine Fuzzy, S (x + 1, y + 1) if SR (x + 1, y + 1) > ST,R and SG (x + 1, y + 1) > ST,G and SB (x + 1, y + 1) > ST,B then begin Call Subroutine Connect Pixel (x + 1, y + 1, c) end Call Subroutine Fuzzy, S (x + 1, y); if SR (x + 1, y) > ST,R and SG (x + 1, y) > ST,G and SB (x + 1, y) > ST,B then begin Call Subroutine Connect Pixel (x + 1, y, c) end Call Subroutine Fuzzy, S (x + 1, y − 1); if SR (x + 1, y − 1) > ST,R and SG (x + 1, y − 1) > ST,G and SB (x + 1, y − 1) > ST,B then begin Call Subroutine Connect Pixel (x + 1, y − 1, c) end Call Subroutine Fuzzy, S (x, y − 1) if SR (x, y − 1) > ST,R and SG (x, y − 1) > ST,G and SB (x, y − 1) > ST,B then begin Call Subroutine Connect Pixel (x, y − 1, c) end
440
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
Call Subroutine Fuzzy, S (x − 1, y − 1) if SR (x − 1, y − 1) > ST,R and SG (x − 1, y − 1) > ST,G and SB (x − 1, y − 1) > ST,B then begin Call Subroutine Connect Pixel (x − 1, y − 1, c) end Call Subroutine Fuzzy, S (x − 1, y) if SR (x − 1, y) > ST,R and SG (x − 1, y) > ST,G and SB (x − 1, y) > ST,B then begin Call Subroutine Connect Pixel (x − 1, y, c) end Call Subroutine Fuzzy, S (x − 1, y + 1) if SR (x − 1, y + 1) > ST,R and SG (x − 1, y + 1) > ST,G and SB (x − 1, y + 1) > ST,B then begin Call Subroutine Connect Pixel (x − 1, y + 1, c) end end else return end
Fig. 8. Lena: (a) color image of size 256 × 256 with 255 gray levels, (b) some of regions, (c) variation of maximum region number with ST and (d) region histogram of segmented image: ST = 70.
3. Simulation results and discussion The proposed algorithm was tested with the well-known Lena image shown in Fig. 8(a), which is a color image of size 256 × 256 with 255 gray levels. As ST is the input variable of the image segmentation system, the maximum region number of the segmented image varies with ST as shown in Fig. 8(c) where the maximum region number increases with ST .
Although each point on the curve in Fig. 8(c) corresponds different segmented image and region count, the segmented image shown in Fig. 8(b), when the ST is selected to be 70, consists of 215 regions. The region histogram shown in Fig. 8(d) was also obtained while the ST was set to 70. Image of electronic components shown in Fig. 9(a) was also tested with the proposed algorithm. Fig. 9(c) shows the variation of total number of regions with the similarity threshold while the region histogram is shown in Fig. 9(d).
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
441
Fig. 9. Electronic components: (a) color image of electronic circuit components, (b) some of segmented letters, (c) variation of maximum region number with ST and (d) region histogram of segmented image: ST = 80.
The image was segmented 794 regions when ST was equal to 80. The symbol of integrated circuit, letters: ‘M’, ‘9’, ‘K’, ‘L’, ‘M’, ‘4’, ‘B’, ‘8’ and ‘2’ are segmented regions of the image as shown in Fig. 9(b). It has been noticed from applications that the most of regions have between one and ten pixels. This shows sensitivity of the algorithm which means that any object consisting of even a single pixel could be isolated from the rest of the image. It could be essential for some particular applications. On the other hand, it may not be necessary for coarse applications where the merging of tiny regions with other big regions is required. The bigger the ST chosen, the more the homogeneity within subimages attained and at the same time the more the number of the subimages obtained. Consequently, the number of regions and homogeneity are inversely related.
4. Conclusion A new method for segmentation of color images was described. The proposed approach is similar to region growing algorithm where the seed points are automatically selected and grown. The similarity percent of neighboring pixels is
calculated by means of fuzzy reasoning rules and the merging of the pixels with regions is performed by comparing the similarity percent with the similarity threshold value. The algorithm does not require any prior knowledge of number of regions existing in the image and decreases the computational load required by other segmentation algorithms since the complex computations such as statistical calculations, gradients, Laplace and Green’s functions were not used. On the other hand, a drawback of the proposed method for segmentation is that an array of the same size as the image is required and some regions with one pixel may be created. The further research to avoid formation of regions with one pixel and different applications are still in progress.
References [1] Gonzalez RC, Woods RE. Digital image processing. Cambridge, MA: Addison-Wesley; 1993. [2] Castleman KR. Digital image processing. Englewood Cliffs, NJ: Prentice-Hall; 1996. [3] Cheng HD, Jiang XH, Sun Y, Wang J. Color image segmentation: advances and prospects. Pattern Recognition 2001;34:2259–81.
442
R. Demirci / Int. J. Electron. Commun. (AEÜ) 60 (2006) 435 – 442
[4] Pal NR, Pal SK. A review on image segmentation techniques. Pattern Recognition 1993;26:1277–94. [5] Fu KS, Mui JK. A survey on image segmentation. Pattern Recognition 1981;13:3–16. [6] Bezdek JC, Trevide MM. Low level segmentation of aerial images with fuzzy clustering. IEEE Trans Systems Man Cybern 1986;SMC-16:589–98. [7] Lim YW, Lee SU. On the color image segmentation algorithm based on the thresholding and the fuzzy c-means technique. Pattern Recognition 1990;23(9). [8] Chang YL, Li X. Adaptive image region growing. IEEE Trans Image Process 1994;3(6):868–73. [9] Zadeh LA. Fuzzy sets. Inf Control 1965;8:338–53. [10] Ross TJ. Fuzzy logic with engineering application. New York: McGraw-Hill; 1995. [11] Pal SK, King RA. Image enhancement using smoothing with fuzzy sets. IEEE Trans Systems Man Cybern 1981;SMC11(7):494–501. [12] Pal SK, Rosenfeld A. Image enhancement and threshold by optimization of fuzzy compactness. Pattern Recognition Lett 1988;7:77–86. [13] Paclik P, Duin RPW, Kempen GMP, Kohlus R. Segmentation of multi-spectral images using the combined classifier approach. Image Vision Comput 2003;21:473–82.
[14] Cheng YC, Chen SY. Image classification using color, texture and regions. Image Vision Comput 2003;21:759–76. [15] Mahmood TS, Petkovic D. On describing color and shape information in images. Signal Process Image Commun 2000;16:15–31. Recep Demirci was born in K. Maras, Turkey in 1968. He got his B.Sc. degree from the Gazi University in 1990 where he studied electrical education. He was a visiting researcher at the Purdue University, USA in 1992. He has got his Ph.D. degree from the Electrical Engineering Department, University of Wales Cardiff, UK in 1998. He completed his military service as a Lecturer at Engineering Brigade in Izmir between 1998 and 2000. He has been working as an Assistant Professor at Electrical Education Department, Technical Education Faculty, Abant Izzet Baysal University since 2000.