- Email: [email protected]
Self-Organizing Map based Extended Fuzzy C-Means (SEEFC) algorithm for image segmentation
Accepted Manuscript Title: Self-Organizing Map based Extended Fuzzy C-Means (SEEFC) Algorithm for Image Segmentation Author: Ebrahim Aghajari Gharpure Damayanti Chandrashekhar PII: DOI: Reference:
S1568-4946(17)30007-8 http://dx.doi.org/doi:10.1016/j.asoc.2017.01.003 ASOC 3997
To appear in:
Applied Soft Computing
Received date: Revised date: Accepted date:
17-7-2016 28-11-2016 4-1-2017
Please cite this article as: Ebrahim Aghajari, Gharpure Damayanti Chandrashekhar, Self-Organizing Map based Extended Fuzzy C-Means (SEEFC) Algorithm for Image Segmentation, Applied Soft Computing Journal http://dx.doi.org/10.1016/j.asoc.2017.01.003 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Self-Organizing Map based Extended Fuzzy C-Means (SEEFC) Algorithm for Image Segmentation Ebrahim Aghajari a, Gharpure Damayanti Chandrashekhar b a b
Department of Electrical Engineering, Ahvaz Branch ,Islamic Azad University ,Ahvaz, Iran Department of Electronic Science, University of Pune, Pune, Maharashtra, India
Graphical Abstract
Figure: SOM and EFCM Integration in a Glance View
*Highlights (for review) The motivation for this integration was the difficulties faced in using SOM for Image segmentation. The number of clusters needs to be specified for improving the performance. For different images the map sizes have to be optimized. It was also observed that SOM has limited capabilities for the representation of hierarchical relation of mapped data making interpretation difficult. In this work a novel attempt taking advantage of capabilities of self-organizing map and extended fuzzy clustering for Image segmentation is reported. Incorporating SOM we can find the essential information by mapping image data to a two dimensional space and with the help of EFCM we estimate the number of clusters. Finally the cluster centers are used for clustering purpose and image segmentation.
Original Image Features selection & Feature Extraction
Prior Segmentation Using SOM
EFCM
Labeling using EFCM Output
Final Segmentation Segmented Image
Answer to reviewer Comments: We believe that we have answered all the queries of the referees. The paper has been checked word by word and all the points has been considered and corrected as per referee's comment. We hope that the revised version of manuscript considered all the points.
Abstract: A Novel hybrid algorithm based on Self-Organizing-Map (SOM) and Extended Fuzzy CMeans (EFCM) named self-organizing-map based extended fuzzy c-means (SEEFC) has been designed and implemented for image segmentation. The proposed algorithm works in three stages. At first, the images are decomposed by discrete wavelet transform (DWT) into various frequencies and gradient, pixel value and statistical parameters are obtained to form a feature prototype. The feature prototypes are input to the Self Organizing Map (SOM). Finally, the codebook vectors of the trained SOM have been clustered automatically using Extended Fuzzy C-Means (EFCM). The clustered codebook vector centers are used for image segmentation based on minimum distance criterion. The proposed method has been tested with images from Berkeley’s database and results obtained are promising. The segmentation results are evaluated against the ground truth. Comparisons with another state of the art approaches indicate advantages of the SOM based EFCM algorithm.
Keywords: Self Organizing Map (SOM), Extended Fuzzy C-Means (EFCM), Discrete Wavelet Transform (DWT), Image Segmentation, Feature Extraction.
1. Introduction: Image segmentation is one of the most interesting and critical steps towards image understanding and analysis. It can be defined as a process which aims at partitioning an image into meaningful and separate regions. Indeed the goal of segmentation is to simplify or change the representation of an image into something that is more meaningful and easier to analyze. Extensive research has been carried out in developing different algorithms and methods for image segmentation but even today, no single standard method of image segmentation has emerged [1].Rather, there are collections of ad hoc methods that have received some degree of popularity. However all the research work performed on image segmentation can be classified into two broad categories. Traditional methods like thresholding [2], morphological methods, edge-based segmentation [3], Normalized Cut method (NC)[4], Efficient graph-based method (EG)[5] Mean shift method (MS)[6] ,Level-set method (LS)[7],Ratio-contour method (RC) [8] and many more. Soft computing techniques based on Fuzzy Theory [8][9][11][12],Artificial Neural Network (ANN)[13][14] and Genetic Algorithm (GA)[12][13][17], are predominantly used for segmentation and are preferred by researchers because of the adaptive nature and accuracy. The theory of fuzzy set has been introduced in 1965 by Zadeh [18] as a way of representing uncertainties in everyday life. Challenges in image segmentation have encouraged researchers to develop fuzzy segmentation algorithms by considering image regions as fuzzy objects, where an image pixel may be partially classified into multiple potential classes and the boundaries between intensities of different objects can be well defined. The theory of fuzzy sets is adopted to effectively model the fuzziness of image pixels which might be caused by inherent heterogeneity and imaging device artifacts (e.g., blurring, imposed noise and background variation).There are several image segmentation methods based on fuzzy concept reported in[19][20][21]among which fuzzy clustering [21] and fuzzy connectedness[22] are two well-known techniques for this purpose. FCM is used for data clustering, with the drawback that the number of clusters has to be specified. Extended fuzzy CMeans (EFCM), has the capabilities of finding an optimum number of clusters automatically. The major problem in applying EFCM to Image segmentation is that it becomes computationally expensive as the dimensions of the data increases [23]. Artificial neural networks (ANN) [24][25], are signal processing systems that try to emulate the biological nervous systems, by providing a mathematical model of the combination of numerous neurons connected in a network. In general, neural networks fall into one of the two following categories: Supervised and unsupervised neural networks[26]. Our review of literature related in this areas shows that among supervised NN’s the Hopfield neural network, multilayer perceptron and back propagation are well-known techniques for segmentation purpose. Among unsupervised methods the self-organizing map (SOM) [27] is one of the best approaches for this purpose. Although it has been proved that SOM makes itself a promising clustering tool and it is very helpful for visualization and cluster extraction but it has limitations from Image segmentation point of view. Some researchers have used SOM with other methods for better performance [28]–[33].It has been noted that the map size affects the clustering ability of SOM. Araujo and Costa have proposed SOM network with a self-adaptive topology for color image segmentation [34]. Dongxiang chi [29]combined SOM with K-means and saliency map for clustering. It is observed that SOM-K method is an unsupervised method requiring less computation and no human intervention. The saliency map was used to improve the segmentation performance. An optimizing number of clusters for k-means from 2 clusters to √𝑛 cluster (where n is the map size) was done by applying entropy index. Concalves et al [31] presented SOM-based clustering method for remotely sensed images. The hit map information is investigated and zero-hits nodes and heterogeneous nodes are filtered. Secondly, the clustering index is applied to get the best clusters. This method has been applied for natural scene image segmentation. Review of the literature indicates that most of the SOM-based clustering methods emphasize on analyzing the SOM derived prototype parameters [35]–[37]. Enlightened by [29] and [37] the prototypes are often integrated through the second stage or combined with hit numbers for automatic detection of clusters.
It may be noted that Neuro-fuzzy hybridization is a widely used tool of soft computing paradigm and has emerged as a promising field of research in recent years [38] [39]. Neuro-fuzzy systems combine the advantages of both the uncertainty handling capability of fuzzy systems and the parallel learning ability of neural networks [40].The motivation for this integration was the difficulties faced in using SOM for Image segmentation. The number of clusters needs to be specified far improving the performance. For different images, the map sizes have to be optimized. It was also observed that SOM has limited capabilities for the representation of hierarchical relation of mapped data making interpretation difficult. [27] In this work a novel attempt taking advantage of capabilities of a self-organizing map and extended fuzzy clustering for image segmentation is reported. Incorporating SOM we can find the essential information by mapping image data to a two-dimensional space and with the help of EFCM we estimate the number of clusters. Finally the cluster centers are used for clustering purpose and image segmentation [41]. The new algorithm is named as Self-Organizing Map based Extended Fuzzy C-Means (SEEFC). This paper is organized as follows. Section 2 presents the methodology and section3 deals with the experimental results and discussions. Section 4 presents the comparison of results and finally section 5 concludes the paper. 2. Methodology This study explores the use of SOM followed by EFCM to get the number of clusters and the cluster centers automatically, for image segmentation. The algorithm extracts the features initially followed by a module for features selection. The selected features are used for prior clustering using SOM and the SOM map is given to EFCM algorithm for further processing. The EFCM is used to optimize a number of clusters based on coarse clustering carried out by SOM. The cluster centers are obtained and individual pixels are labeled using minimum distance criterion for final segmentation task. The flow chart of the methodology implemented is shown in Fig.1. 2.1 Feature extraction The main purpose of feature extraction is representing the original data set by proper features to distinguish one input pattern from another. The selected features must represent the characteristics of the input image regions. The features that will be used as inputs are extracted as discrete wavelet transforms features including four features namely approximation, diagonal, horizontal and vertical in addition to pixel values, Gradient and statistical features including variance, standard deviation and energy. The following subsection will describe the features in detail. 2.1.1
Discrete wavelet transform (DWT)
The wavelet transform is important to provide a compact description of signals (or images) that are limited in time (or spatial extent) and it is very helpful in the description of edge and line that are highly localized [42]. Wavelet transform decomposes a signal into a set of basis functions. These basis functions are called wavelets. Wavelets are obtained from a single prototype wavelet called mother wavelet by dilations and shifting:
a ,b (t )
1 t b ( ) a a
a ,b (t )
(1)
Where (a) is the scaling parameter and (b) is the shifting parameter. The scaling and wavelet function (x , y ) and (x , y ) here. The scaled and translated basis are two variable functions denoted functions are defined as: ji , m , n ( x, y ) 2 j / 2 (2 j x m, 2 j y n)
(2)
i j /2 j j i {H ,V , D }. j , m , n (x , y ) 2 (2 x m , 2 y n ),
(3)
By using wavelet on an image for one level, four images are obtained which correspond to the approximation and details in the images. The approximation coefficients matrix CA and details coefficients matrices CH, CV, and CD(horizontal, vertical, and diagonal, respectively), obtained by wavelet decomposition of the input matrix X which is the same as (LL,LH, HL, HH) have been used as features for proposed algorithm.
2.1.2 Statistical Parameters 2.1.2.1 Variance The variance (σ2) is a measure of how intensity values in the image are far from the mean. We use the 3*3 window template for calculating the variance. Here is how it is defined: n
2 =
(X i 1
i
)2 (4)
N
Where the variable is the mean of intensity values and N is the number of pixels in an image. 2.1.2.2 Standard deviation The Standard Deviation (σ) is a measure of how spread out intensity values in image is. A low standard deviation indicates that the pixel values tend to be very close to the mean intensity, whereas high standard deviation indicates that the pixels values are spread out over a large range of values. 2.1.2.3 Energy There is more than one definition of "energy" in image processing but in general the energy can be defined as spectral density which describes how the energy of a signal or a time series is distributed with frequency. Here, the term energy is used in the generalized sense of signal processing to denote a variance of the signal. The 3*3 window template has been used for energy calculation.
ERG = P(i, j)2
(5)
i, j
2.1.3
Feature Selection
The optimized feature vectors for this study were based on two domains, frequency and statistics. Deployment of features of all pixels in the image in the training process can be quite onerous. So it is preferable to construct a set of feature vectors in such a way that the original data variance in the image is retained. The training data selection is carried out based on the following principals: At first the selected data should accurately represent the characteristics of input image. Secondly it has to address the significant information and component of the image. Different methods were examined for pixel selection in order to choose a subset as training data. As shown in following figure, the training candidate pixels were selected randomly from each quadrant as shown in Fig.3.5 (a) and the total features vectors were 10,000. Use of equal number of training feature vectors from each quadrant and otherwise, did not make much difference in the final result. The other method, was the random selection of data in entire image as shown in Fig.3.5 (b & c). Various experiments carried out shows that the technique of random data selection for generating a subset of feature vector is faster and easier. 2.2 Self organizing map (SOM) Self-organizing map was introduced by Kohonen in 1982 [43]. The idea was to represent, huge amount of data by finite number of samples. SOM is an unsupervised neural network and it is able to organize itself. It consists of n dimensional input space with m dimension of output space. The map is usually 2D structure with unit of map associated with a weight vector. (6) Nij Wij ;1 i L,1 j M
Where Nij is a 2D grid of map (also called neuron),Wij is the weights vector (also called as prototype vector) assigned to the (i,j)’th unit of SOM architecture and L& M are number of rows and columns. The formation of the self-organizing map consists of 3 steps.1-competition, 2- cooperation, 3-synaptic adaption. 1- Competition In this section calculation of best matching unit (BMU) is done according to the Euclidean distance between weight vector and training input patterns. The weight of the neurons is defined as Wij w1 , w2 ,..., wd where d is the dimension of weight vectors and in this case based on the number of features chosen it is nine. The features, namely discrete wavelet transform (CA, CD, CV, CH), pixel value (PV), gradient (GXY), standard deviation (STD), variance (VAR), and energy (ERG) are used to form the feature vector. In this study, training pattern vector set is defined as T {Vp ,1 p n}where the Vp is the P’th training feature vector and n is the total number of training pattern vectors which is 10’000 in this study. During training the SOM tries to find the neuron in its map whose weight is nearest to the training pattern vector (Vp) based on Euclidean distance. The Neuron in the SOM architecture whose distance from training pattern vector (Vp) is minimum, is called as best matching unit (BMU) of that training pattern vector (Vp) and denoted as Wm. The Euclidean distance and BMU are determined as: (7) dist (Vp Wm ) (8) || Vp - Wm || min ij {|| Vp - Wij ||} 2- Synaptic Adaption The BMU and neighbors around it have their weights W ij updated with step t and will be represent by (9) W (t 1) W (old ) (t ) (t )(V (t ) W (t )) ij
ij
The new weight for a node is the old weight, plus a fraction of the old weight and the input vector wherein𝜃(𝑡)is based on distance from the BMU, (t ) is the learning rate, exponential decay function that insure the SOM will converge.
t
(t ) 0 exp( ) Represent time constant and t is the time step.
(10)
3- Cooperation Ideally after finding the BMU the winner neuron weight has to be updated. But in practice the BMU and neighbors around it will be updated based on Eq. (11)
dist 2 (t ) exp( 2 ) 2 (t )
(11)
In the above expression (t ) is the topological neighborhood center on BMU where dist is the position of BMU in SOM unit. isan exponential decay function that shrinks on each iteration until eventually the neighborhood is just BMU itself. The amplitude of the topological neighbors decreases monotonically with the increasing lateral distance.
t
(t ) 0 exp( ) 2.2.1
(12)
SOM Segmentation
The interpretation of output of SOM is a tricky issue as reported in the literature. The Umatrix is one of the simplest options for that. In this paper we use the following method for this purpose[44]. Here we consider that the winner neuron hits for interpretation of results using SOM.Fig.3-a shows the flowchart of proposed algorithm. First of all the extracted features of various
pixels in the image will be generated. Considering the features of all the pixels in the image would take lots of time for SOM training. To overcome the problem, the proposed algorithm uses random selection of pixels in the entire image as a representative training set. In this algorithm number of feature vectors selected for training purpose was 10’000. In the next step the sample features are given to the created SOM architecture for training purpose. The topology lattice is hexagonal and its size is optimized to(15*15). Initially use of SOM alone for segmentation was investigated. Herein the number of hits of winner neurons and Euclidean distance between two winners has been used for segmentation. We predefine number of clusters and the Euclidean distance threshold on the hexagonal map Fig.3-b. The threshold value can vary from 0 to maximum distance between the 1st neuron position W11 and the last position W1515 i.e.is 19.79. Initially the neuron with maximum hits is considered as first cluster representative. Then the second neuron with maximum hits is obtained. If its distance from the first winner neuron is more than threshold value (13 in this case) it will be considered as representative of second cluster and else the algorithm will select the next highest one and the loop goes on. As per the predefined number of clusters (3 in this case) the process will continue till the algorithm meets conditions. Finally the weights of selected winning neurons are considered for image segmentation by minimum distance criterion, calculating Euclidean distance of prototype vectors from cluster representative vectors as seen in Fig.3-c. The problem with SOM is the number of segments has to be specified in prior and the map size has to be big enough to give acceptable cluster separation. For the said purpose different methods has been studied and extended fuzzy clustering found to be suitable because of its robustness and accuracy [23]. 2.3 Extended Fuzzy Clustering Fuzzy clustering is a process which allocates data points to clusters based on the fuzzy memberships related to Euclidean distance. Most widely used fuzzy clustering is Fuzzy C-Means (FCM) Algorithm[20] which partition a collection of n data elements into collection of ‘C’fuzzy clusters with respect to some given criterion. But FCM algorithm presents two shortcomings: one, a priori the number ‘C’ of prototypes must be defined or one calculates C as minimum or maximum of a suitable function. Secondly, the cluster centers tend to locate in areas with high concentrations of features and at times, the zones with low density data points could be relevant. Generally speaking, the distribution of the data is sensitive to the initialization phase. The EFCM algorithm was first proposed in[23] to overcome the above shortcoming of the FCM method . Let’s consider X R as a sample dataset with the following matrix:
x11 X x M1
x1N xMN
(13)
n
Where X={x11,...,xMN}⊂R , is composed of M input vectors having N features defined as xj= (x1j,x2j, x3j…,xnj)T. For implementation of the EFCM algorithm with matrix X as input one can follow the following steps. Step 1: Compute center of cluster prototype
(U ) X (U ) N
i
m
k 1 N
k 1
ik
m
k
1
(14)
ik
Where Xk is an input feature vectors,m≥1 is the fuzzifier parameter. Please note that in the work done, all tests and examples have been calculated, with a fuzziness parameter m=2.Uik is the membership
degree of Xkto the i-th cluster where set of clusters are i=1,…,C. Viis the center of I’th cluster and the set can be represent as V={v1,…,vC}⊂Rn . Step 2: Computing the radius of cluster prototype using Eq. (15)
Pi
ri
N k 1
(U ik )m (X k V i )(X k V i )T
N k 1
(U ik )
(15)
m
1/n
Pi
(16)
M
The Pi is the covariance matrix whose determinant gives the volume of the i-th cluster and β is thevolume prototype coefficient and m defines the number of clusters in each iteration. β, initially has to be predefined by the user and in this study selected as 1 and it will be improved duration of process under circumstances. Step3: Compute the distance to the volume cluster In the EFCM algorithm the objective function is the following formula M
J i 1
N
U k 1
m ik
(dik ri )
(17)
The term dikis the distance of each volume prototype x from the center of volume cluster i. Unlike FCM this distance is calculated as given below in EFCM algorithm. (18)
d ik max(0, (x k v i )(x k v i )T ri ) Where in xk and vi are k-th prototype vector and i’th cluster center respectively. Step4: Update the membership matrix For 1≤k≤N, let k {i | d ik 0}
(19) For calculation of membership function we consider the parameter k equal to the number of clusters I for which dik (distance) =0.i.e for prototype vectors inside the clusters. Thus membership function is determined using the following formula when the k .
U ik
1
M j 1
(d ik / d jk )
2/ m 1
, k 0
(20)
1≤i≤M
Otherwise 0
ifdik> 0 1≤i≤M , k
Uik=
0
1/ k ifdik = 0
Step5: Select the most similar cluster
S ij
N k 1
min(U ik ,U jk )
N
U ik
j 1
1≤i,j≤M
(21)
(i * , j * ) arg max(S i , j ) (i , j ) i j Determination of the number of clusters is achieved by computing the Sij (similarity measure)for the ith cluster and j-th cluster.
Step6: Merging similar clusters The merging between the two clusters i and j is done when at l-th iteration we have the following condition. Note that the 𝜀1 is user predefined criterion which is set to 0.01 in this paper.
If S (l )i* , j* S (l 1)i* , j* 1
(22)
Parameter 𝛼 ∈ [0,1] is the merging criteria which is determined with the following formula: Let
1 M 1
(23)
If Si* , j* So that cluster i-th and j-th will be merged together if above condition is true.
U l i *k : (U ( l )i *K U ( l ) j *K ) Remove j from U After merging the cluster M becomes:
M M 1 Else the program enlarge the radius (ri) volume prototype by a parameter𝛽factor which is defined recursively as follows
(l ) min(M (l 1) , (l 1) 1)
(24)
Wherein 𝛽is initially set to 1.𝛽(0) = 1 Step7: the process stops when the condition below is met. Note that the 𝜀2 is user predefined criterion which is set to 0.001 in this paper. Until U (l ) U (l 1) 2
(25)
As it is mentioned in paper [45],[46] the problem with this method is that, in fact it is computationally expensive and with increasing the number of input data the program would take a very long time. The idea of using EFCM for clustering and image segmentation was explored but it was observed that amount of time required with a minimal data set of 10,000, is prohibitive. Considering the size of the image, for good segmentation the data set used has to be exhaustive, ruling out the use of EFCM on the image directly. However as it is known, the beauty of self-organizing map is its ability to represent huge amount of input data in a finite 2D representation (15*15 in our case). Using the SOM output map as input representative input, one can overcome the time limitation of robust EFCM. The new algorithm utilizes the codebook i.e. 2D map of SOM as input data to EFCM algorithm to get number
of clusters in minimum time automatically. In the following subsection 2.4 the proposed method will be elaborated in detail.
2.4 Developed Neuro-Fuzzy Segmentation Method (SEEFC) This study explored the use of SOM followed by EFCM to get the number of clusters and the use of cluster centers as a reference automatically, for image segmentation. The technique is named Self Organizing Map EFCM (SEEFC). Fig.4 shows an image (321*481) pixel from Berkley Database (#3096). The optimized 9 feature vectors for the image are calculated resulting in: (321*481=154401 Feature Vectors). 10000 random samples are selected for training the SOM as described before. Fig.4 (b) shows a SOM of (15*15) size and the huge image data can be represented with 15*15 =625 weight vectors which are only 2.25% of the image data. The size of SOM map was varied from 10*10 to 25*25 for which the data reduction is over a range of 1% to 6.25%. With experimentation, the size of SOM was optimized to 15*15 for this work. The reduced data, clustered using SOM, is input to EFCM algorithm for finding optimum number of clusters. The cluster centers are used for segmentation. This method is named as SEEFC. Regardless of the size of SOM and its related topology the SEEFC technique can perform segmentation smartly as per the methodology shown in Fig.5. In the beginning, the input image is normalized. The next module extracts features using Discrete Wavelet Transform, Gradient and pixel value, as well as Statistical parameter (variance, standard deviation and energy). The proposed method SEEFC uses the random selection of pixels throughout entire image to be used by self-organizing map neural network for training purpose. Once the SOM is trained, the map weight vectors created by SOM are used for obtaining the optimum number of clusters and cluster centers using extended fuzzy clustering method (EFCM). For segmentation the Euclidean distance between the cluster centers obtained from EFCM output and feature vector of the original image is calculated. The pixel is labeled based on the minimum distance criterion.
Different images from Berkeley database have been used for testing the performance of the proposed method. The results show the ability of the SEEFC method in segmentation. A typical result for the airplane image from Berkley database (code no: #3096) in Fig.6 (a),Fig.6 (b) and Fig.6 (c) shows the segmented result by SOM and Neuro-Fuzzy segmentation techniques (SEEFC) respectively. The SEEFC algorithm found 3 clusters for the image and the process of segmentation of the image was carried out on this basis. As it is evident visually, the refinement in segmented airplane image and background is clearly visible. The airplane nose seems mixed with background using SOM technique Fig.6 (b) for segmentation but the segmentation is refined by proposed method SEEFC and good separation can be seen between object and background Fig.6 (c). Also, while observing the segmentation result from background point of view, the uniformity in background is refined compared to SOM method. 2.5 SEEFC Performance Evaluation Despite of many approaches for image segmentation validation, it is still difficult to characterize whether one particular segmentation technique, produces better segmentation than another. However various attempts have been made and studied [47][48] to define parameters to assess the performance of image segmentation. The reasons for this are manifold. One among them is the difficulties that exist in validating the segmentation algorithm due to variability of the region causes difficulty in defining ground truth. In reality as segmentation is an ill-defined problem, there is no unique ground truth segmentation of an image against which the output of an algorithm may be compared. However one of the common segmentation assessments is based on comparison of segmented image against manually segmented or pre-processed reference image. In present work two
approaches namely qualitative comparison and quantitative evaluation have been used. All the images and ground truth used for evaluation are available at Berkeley database. Qualitative comparison is the most common method for evaluation the effectiveness of a segmentation algorithm, in which a human visually compares the image segmentation result for separate segmentation algorithm [49]. Quantitative analysis enables the performance of segmentation results to be evaluated statistically against ground truth. For these reason there are three parameters which have been selected for this evaluation which are Sensitivity, Accuracy and Q-index. The first two parameters are based on four different criterion parameters such as True-Positive (TP), True-Negative (TN), False-Positive (FP) and False-Negative (FN) as shown in Fig.7 and will be defined as following equations, Eq.1 Eq.2
Sensitivity = ((TP/ (TP+FN))*100% Accuracy=((TP+TN)/TP+FN+TN+FP))*100%
TP is True-Positive which is defined as number of pixels correctly labeled as belonging to the object class, TN is True Negative and could be explained as number of pixels correctly labeled as belonging to the not object class, FP is False-Positive and defined as number of pixels incorrectly labeled as belonging to the object class and finally FN (False-Negative) defined as number of pixels which were not labeled as belonging to the object class but should have been. An ideal segmentation is the one which achieves 100% sensitivity and accuracy. Sensitivity can give the information about the object of interest and how segmentation algorithm segmented the object perfectly. In this evaluation the image is segmented into two classes, “object of interest” and everything else as “not object of interest”. The terms accuracy points to the overall ability of the segmentation algorithm. The other quantitative parameter for segmentation assessment is Q (I) index proposed by Borsotti et al [50] and defined as follows. Eq.(1.3)
Q( I )
R ei2 R( Ai ) 2 1 R * [ ( ) ] 10000( N * M ) Ai i 1 1 log Ai
Where, I is the resultant image, N*M is the size of the image (I).R is the number of regions of the segmented image. ei is the sum of the Euclidean distance between the vectors of the pixels of region i and the vector attributed to region i in the segmented image. Ai is the area (as measured by the number of pixels) of the ith region. The R (Ai) represents the number of regions having an area equal to Ai. The lower Q value indicates better segmentation. This parameter gives the overall idea of how well the algorithm was able to perform the segmentation task. In the following description some sample images is selected for quantitative evaluation of the developed algorithms in terms of sensitivity and accuracy. 2.5.1 Segmentation of Simple Images Using SEEFC To analyse the performance of the developed algorithm few simple images of size 300*300 pixels images have been chosen as shown in Fig.8 (a & b).This experiment was carried out to test the ability of the SEEFC algorithm for the known number of clusters. Fig.8 (a) is an image, with the 2 different colors (white and Black) and the expected number of clusters are two. The SEEFC algorithm detected 2 clusters and the result is shown in Fig.8 (c). In the segmented image all the pixels are assigned perfectly as per expectation automatically. The other image is a text in which different gray shades color shown in Fig.8 (b). The SOM trained network weight vectors were given to the EFCM. The output of EFCM detected 3 optimum clusters and the segmented output can be seen in Fig.8 (d). In the segmented image all the similar gray shades belongs to the same cluster. The SEEFC algorithm has been tested on Berkley image segmentation database to study and understand its performance. For this analysis the algorithm is firstly tested on some simple images for which the number of clusters is known and then it is tested on a number of images.
2.5.2 SEEFC Result on Standard Images After evaluation of the developed Neuro-Fuzzy technique for simple images, the algorithm was tested on some standard image .Results obtained for two typical images reported in papers[51] have been presented in Fig.9.The figure shows segmentation using SEEFC algorithm along with other techniques. The first image is Rice image shown in Fig.9 (a), in fact the SEEFC algorithm processed the image and find the number of cluster as 3 in the image and the result is mapped the output of segmentation as shown in Fig.9 (c). The second image is a pepper image wherein SEEFC detected 4cluster and the result illustrated in Fig.10 (d).The pepper image has been segmented using different techniques as shown in Fig.10 (a & c) shows fuzzy weight gradient techniques (FWG) segmentation result on this particular image with 3 and 9 clusters respectively[51]. Fig.10 shows the segmentation results using various algorithms. Fig.9 (b) shows the segmented output of WINN technique for the pepper image with the number of cluster selected as 7. The Fig.10 (d&e) shows the result of SOM neural network and fuzzy C-means (FCM) segmentation output for 4 clusters. Fig.10 (f) shows ability of SEEFC technique on cluster selection and segmentation. To analyses the SEEFC performance in more detail on other images, Berkley image segmentation database was selected [52]. These images are standard images which have been used for different purpose and it provides the ground truth which can be used for evaluation purpose.
2.6 Segmentation of images from Berkeley Database Using SEEFC The SEEFC algorithm has been evaluated on single object Berkeley database. Throughout the assessment stage, Segmentation results have been compared against ground truths provided by Berkeley database.Fig.11 illustrates the ground truths and segmented images. The segmentation results have been analyzed considering two criteria: Object extraction and background detection. Two segmentation methods one based on Self Organizing Map (SOM) alone and the other combination of self-organizing map and Extended of fuzzy clustering method (SEEFC), have been implemented. A number of images have been used for testing the performance of the two and the results have been evaluated. For this purpose ten out of those tested images are chosen to illustrate the performance of proposed algorithms. These images are named as Plane (#3096), Bird (#42049), Elephant (#296059), Shipman (#62096), Fox (#167062), Garden (#86016), Eagle (#135069), moon (#238011), church (#118035) and paratrooper (#60079). The results obtained using SOM and SEEFC are shown in Fig.10. The significant changes and information is highlighted with arrows in segmented images. First we consider the plane image (#3096). The tip of plane in SOM output is mixed with background information while in SEEFC output it is completely refined and segmented out properly. The same scenario is repeated for the background of the plane image. As it is highlighted by arrows, the background of SEEFC algorithm is distributed between two clusters but in SOM the combination of all clusters can be seen in the resultant image. Further, consider the segmentation of letter ‘A’ on the vertical stabilizer section of plane, letter A is detected with SEEFC algorithm much better than SOM algorithm. Let’s consider the bird image (#42094).The result clearly shows that SOM fails to detect the entire background as it is expected via ground truth while SEEFC algorithm has better capability to detect background. Considering the bird object in the segmentation result one can say that SOM had better performance in detecting the branches compared to SEEFC algorithm. The detail information of Bird body is segmented more clearly particularly the feet area and the branches in the lower right of the image are detected and segmented properly with SEEFC. In the case of the elephant image (#296059) by SEEFC, both the elephant are segmented as a similar object while SOM segments the elephant body into two clusters. The problem of both methods is the outline between two elephants is not separated properly. The ship man image (#62096) includes three segments as shown in the ground truth. Object
including the boat and man, background including river and mountain area. The SOM algorithm segmented the river and the mountain area almost similar to ground truth. The object is detected but it is combination of 2 clusters. The shipman and the sail of boat are segmented as different objects while it is one object as per ground truth. Looking at results of SEEFC algorithm, it is evident that the river and its outline are segmented properly but the mountain area and object are segmented as one cluster. Thus, the shipman and the sail are fully segmented as one object but at the same time it is mixed with mountain area which does not match with ground truth. In the Fox image, SEEFC algorithm is successful in detecting the Fox better in comparison with SOM algorithm. The SEEFC segmented the background better near ground truth as compared to SOM. For the Garden image in both the cases, the center garden is fully detected as a separate object according to ground truth. But background, is segmented better by SEEFC method. The image (#135069) is the Eagle image. Both the methods detect the object quite well but SEEFC method segments the eagle and hunted bird by eagle in a proper manner, as has been shown in ground truth. SEEFC was able to easily distinguish between the object itself in the image by showing that in another cluster. In case of the moon image (#238011) SEEFC method detected all objects and background as per ground truth. The moon in this image is a small object, but it is easily segmented by SEEFC method while SOM did not detect that properly. The trees are detected by both methods properly. Background of the image, detected by SEEFC is better compared to the other. The image (#135069) is the Eagle image. Both the methods detect the object quite well but SEEFC method segments the eagle and hunted bird by eagle in a proper manner, as has been shown in ground truth. It was able to easily distinguish between the object itself in the image by showing that in another cluster. In case of the moon image (#238011) SEEFC method detected all objects and background as per ground truth. The moon in this image is a small object, but it is easily segmented by SEEFC method while SOM did not detect that properly. Background of the image detected by SEEFC is better while SOM does not detect that properly. The trees are detected by both of methods. Now we consider the church image (#118035). SEEFC detected 4 clusters for this image. The dome area of the church image and its arch has been properly detected as a separate cluster. The significant difference is highlighted by arrows in segmented images. The church building details and crosses are detected in both the segmented images properly but the SEEFC output is comparatively refined. Let's consider the paratrooper (#60079) image. The paratrooper is the main object. The paratrooper man has been detected as one object in both of the methods. The parachute part has been detected by two different clusters in both methods. The mountain is a combination of 4 different clusters in SOM method while it is combination of 3 clusters in SEEFC method. It seems to be more near to reality compared to the SOM result. From background point of view the SEEFC segmentation is better than SOM. The white shade clusters has been expanded and occupies significant part of the background which is quite far from the ground truth. The same scenario has been repeated for SEEFC method but it is less as compared to SOM. To check the SEEFC segmentation performance quantitavely, the sensitivity and accuracy of the object for images calculated as shown in Table2. The sensitivity value shows the ability of the SEEFC algorithm for object detection. The average accuracy of SEEFC algorithm for object detection is 88.1%. The other parameter which was calculated for SEECF quantitative evaluation is Q index as discussed in Section3.4.The lower is the Q value, the better is the segmentation. In fact the range of Q index varied from “0” to “1”. Q value “0” indicates ideal segmentation. As shown in Table3 it can be seen that the SEEFC algorithm has successfully produced very small Q(I) values for almost all images. The value indicates the ability of SEEFC in correct decision making close to the reality. 2.7 Comparison of SEEFC with other segmentation algorithm These experiments were carried out to study and understand the performance of developed SEEFC algorithm compared with two popular segmentation algorithm namely , Normalize Cut (NC) and Contour Detection (CD)[53]. For evaluation of algorithms the images were selected based on Distinctiveness factor as per Eq.(4.1) . In fact the difficulty of segmentation is measured by “Distinctiveness” of the salient object versus its background.
4.1 Where x2 is difference between the brightness distribution of object in the image and its background. Pf and Pb are the figure and background distribution. So lesser distinctiveness of an image, means more difficulty in image segmentation. In fact distinguish between object and background will be more difficult for segmentation algorithm. Generally Distinctiveness falls into range of [0 to 1] and on this basis the test images are categorized into five different groups. However to assess and compare the result of image segmentation techniques proposed in this work the evaluation is carried out based on qualitative and quantitative analyses. Qualitative analysis can address visual interpretation of the result as observed via human visual perception. Though the qualitative analysis can determine the ability of segmentation algorithm to distinguish between object of interest and background quantitative analysis enables the performance of segmentation results to be evaluated statistically. 2.7.1 Qualitative Analysis This section deals with visual evaluation of the performance of segmentation of each individual algorithm based on their capability to detect the important regions in the image. Four different ranges of distinctiveness from (0-0.2),(0.2-0.4),(0.4-0.6) and (0.6-1) have been used and the segmentation results of SEEFC along with reported data are shown. Segmentation of images with distinctiveness<=0.2 Fig.12shows the two Berkley database images #21077 and #227092 which are Car and Pot with Distinctiveness factor of 0.1 and 0.2 respectively. The ground truth related to each of these images is shown. SEEFC selected 4 clusters and 3 clusters for Car and Pot image respectively. The objects in the SEEFC, CD & NC is shown with the red arrow. Looking at SEEFC result it can be observed that the car object does not have over segmentation with the background which is the problem seen in other method. In the case of Pot image the boundary of the segmented object is isolated from background. The performance of other technique can be seen in the Fig.12. Segmentation of images with 0.2
Second example shows a Surfing image with its related Ground Truth #1109.Looking at SEEFC performance it can be easily observed that the object is detected very nicely while CD and NC failed in segmentation of the object from background. Observing the SEEFC performance on surfing image shows how 3 clusters are assigned to the image. For example the gray color in the segmented image is around the object. Comparing this shade of color in original image one can judge how perfectly it is segmented. Segmentation of images with distinctiveness>0.6 The final experiments evaluated the result of developed algorithm for the group of images with distinctiveness more than 0.6.In this specific groups the clearance of objects from background make the segmentation task easier. From Berkley database the images #3096 and #42049 was selected in which their distinctiveness factors are 0.76 and 0.8 correspondingly. The Ground Truth of images #3096 or plane image shows the plane itself as an object and rest of the image as a background. SEEFC optimizes the number of clusters as 3 and segmented the image accordingly. Contour detection and normalized cut also detected the object properly. All above description could be extended to the image #42049 which is called as bird image. The bird is an object of interest. The SEEFC performs the task well by minimizing the number of clusters to 3. But visual inspection of the result shows the ability of NC method in object isolation compare to other technique is better in this case. In conclusion it can be seen that the result of developed Neuro-fuzzy technique is comparable with well-known techniques of image segmentation. Qualitative comparison of the result show SEEFC has satisfactory performance. Though qualitative analysis shows the ability of the algorithm but next section deals with quantitative analysis of the SEEFC performance. 2.7.2 Quantitative Analysis based on Distinctiveness Factor The proposed algorithm has been tested on Berkley image segmentation database as per the Distinctiveness factor. The main purpose of these experiments was to study and understand the performance of SEEFC algorithm in comparison with Normalize Cut (NC), Contour Detection (CD)[53],SOM and FCM in quantitative manner. The accuracy value of each method can be observed in Table 1: It can be observed that the accuracy of SEEFC segmentation algorithm is better than other techniques especially in case of images with lower distinctiveness factor. That indicates the ability of SEEFC in detection of salient object when there is a smooth edge between object and background. The analysis of the results also indicates the ability of SEEFC as a Neuro-Fuzzy approach to segmentation problem compared to each of FCM and SOM segmentation performance individually. The Fig.16 also represents the plot of SEEFC accuracy in comparison with other approaches. The plot shows that NC technique has the worst performance to detect the salient object. But it is observed that FCM, SOM and NC techniques have acceptable accuracy rate in salient object segmentation in middle range of Distinctiveness. 2.8 Reproducibility of SEEFC Algorithm SEEFC shows better accuracy for lower distinctiveness. This could be attributed to complimentary functioning of SOM-EFCM. However as SOM and EFCM are initialized randomly, however it causes some differences in segmented result .Hence reproducibility performance of SEEFC was studied. The reproducibility of SEEFC was tested in terms of sensitivity and accuracy and the results are shown in Table2. These variations are illustrated in Fig.17 and Fig.18.In order to find the reproducibility, the algorithm executed 10 times and the sensitivity and accuracy parameters are calculated for each run. The analysis of the reproducibility of the algorithm indicates that there is not much variation in sensitivity which addresses the salient object detection in the image except in few cases. However the variation in terms of accuracy shows that SEEFC segmentation results variations is still lesser as compare to sensitivity variations.
2.9 Effect of noise on SEEFC To assess the effect of noise on the proposed algorithm we corrupted the images by Gaussian noise as well as Salt& Pepper noise. The result shows that the algorithm performs very well up to 15 % of variance value of Gaussian noise and 5% of Salt &Pepper noise.Fig.19 illustrates the segmentation result of Bird image (#42049) in presence of noise as a sample.Table5 shows the statistics parameter of stability evaluation in the presence of noises. Increasing the Gaussian noise causes decreasing the sensitivity and accuracy to 81% and 73.5% respectively. The experiments were repeated using runs on the SEEFC algorithm 10 times and the average performance in presence of noises, are as per Table5. However Gaussian noise more than 15%, causes a decrease in accuracy and object sensitivity. The above scenario was repeated for Salt & Pepper noise. The image corruption by this type of noise can lead increases to 5% and result shows that the SEEFC algorithm has acceptable performance on segmentation of such an images. Looking at Table5, it shows that the average accuracy does not decrease below 87%.as it can be observed. In Fig.19, the 5% of Salt & Pepper noise almost change the view of the original images. The SEEFC algorithm can tolerate this situation with minimum effect of these phenomena on the segmented output. However increasing the noise more than 5% causes the algorithm to fail in segmentation process. 3. Application As it is discussed, the developed Neuro- Fuzzy algorithm (SEEFC) technique is completely unsupervised and it is able to find the number of cluster automatically. It’s performance shows good accuracy in presence of abnormal condition like noise as tested. These ability could be utilized in such conditions when there is a change in image backgrounds or unwanted phenomena to make the algorithm robust for segmentation. One of this criteria generally occurs in segmentation of hand for Indoor Robot Navigation (IRN) using gesture recognition [54]. In fact vision based indoor robot navigation imposes some requirements on the system to make it applicable in real life. Some of them are lighting condition or illumination variation, type of backgrounds (simple, complex and variant), skin colors, size of different people’s hand and complex nature of the hand. This makes segmentation and extraction of the hand a difficult task. Capability of SEEFC to adapt to these variations for hand segmentation was studied and tested on hand database images available in the literature [53] and Cambridge dataset[55].As an example The said database includes different images under different background and different illumination conditions. The SEEFC segmentation of images in this database shows comparable performance for hand under different condition. Thus the future work is going to focus on such an application.Fig.20 shows the result of SEEFC algorithm on some sample images under 3 different backgrounds under different illumination condition. Detection of 3 & 4 clusters using SEEFC algorithm result in automatic segmentation, with the hand segmented as depicted in following figure. 4. Conclusion A novel methodology based on the self-organizing map and extended fuzzy clustering has been introduced (SEEFC). SOM is used for coarse clustering of the feature vectors. The basics advantage of the method is the use of EFCM to estimate the number of clusters in the image automatically. Thus the method takes advantage of the self-organizing capability of the SOM as well as automatic estimation of clusters using EFCM. The performance of SEEFC has been evaluated with a number of images from the Berkley database. The results demonstrate the ability of SEEFC for image segmentation. The overall performance of SEEFC, to detect the objects in the image is promising. Qualitative as well as quantitative analysis based on the ground truths indicates the usefulness of SEEFC for Image segmentation. Evaluation of segmented image with standard ground truth images reveals that the SEEFC method’s sensitivity, specificity and accuracy values are better than SOM in the case of object extraction. Even in the case of
background detection, the performance of proposed method is better in terms of sensitivity and accuracy but the specificity of SOM is better when the background is in priority. Qualitative evaluation of the SEEFC algorithm shows an overall improvement of the segmented output. In the quantitative evaluation, sensitivity and accuracy of SEEFC algorithm have been calculated and it shows 84.56% average sensitivity to isolate an object from background rate and an average of 88% success in terms of accuracy. The reproducibility of the algorithm was tested by running the algorithm in a number of times and it shows the good capability of reproducing the result. Finally, the developed algorithm was tested against the different type of noises and it shows good robustness to reproduce the result and can tolerate 15% of Gaussian noise and 5% of salt and pepper noise as well. In contrast, the proposed Neuro-Fuzzy method acts very well for detecting the object from background. Also, SEEFC preserves minor details in the image as can be seen from the moon and Eagle image. Comparison with standard methods shows improved performance of SEEFC. The experiments indicate that the proposed method has the potential to improve segmentation.
References: [1] W. K. Pratt, Digital image processing: PIKS inside. Wiley, 2002. [2] X. Zhao, M. Turk, W. Li, K. Lien, and G. Wang, “A multilevel image thresholding segmentation algorithm based on two-dimensional K–L divergence and modified particle swarm optimization,” Appl. Soft Comput., vol. 48, pp. 151–159, Nov. 2016. [3] R. C. Gonzalez and R. E. Woods, Digital Image Processing, vol. 49, no. 3. Prentice Hall, 2008. [4] J. Shi and J. Malik, “Normalized cuts and image segmentation,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 22, no. 8, pp. 888–905, Aug. 2000. [5] P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient graph-based image segmentation,” Int. J. Comput. Vis., vol. 59, no. 2, pp. 167–181, 2004. [6] D. Comaniciu and P. Meer, “Mean shift: A robust approach toward feature space analysis,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 24, no. 5, pp. 603–619, 2002. [7] J. A. Sethian, “Level set methods and fast marching methods,” J. Comput. Inf. Technol., vol. 11, no. 1, pp. 1–2, 2003. [8] S. Wang, T. Kubota, J. M. Siskind, and J. Wang, “Salient closed boundary extraction with ratio contour,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 27, no. 4, pp. 546–561, 2005. [9] W. Cai, S. Chen, and D. Zhang, “Fast and robust fuzzy< i> c-means clustering algorithms incorporating local information for image segmentation,” Pattern Recognit., vol. 40, no. 3, pp. 825–838, 2007. [10] X. Y. Wang and J. Bu, “A fast and robust image segmentation using FCM with spatial information,” Digit. Signal Process., vol. 20, no. 4, pp. 1173–1182, 2010. [11] J. K. Udupa and S. Samarasekera, “Fuzzy connectedness and object definition: theory, algorithms, and applications in image segmentation,” Graph. Model. Image Process., vol. 58, no. 3, pp. 246–261, 1996. [12] J. P. Sarkar, I. Saha, and U. Maulik, “Rough Possibilistic Type-2 Fuzzy C-Means clustering for MR brain image segmentation,” Appl. Soft Comput., vol. 46, pp. 527–536, Sep. 2016. [13] M. Egmont-Petersen, D. de Ridder, and H. Handels, “Image processing with neural networks-a review,” Pattern Recognit., vol. 35, no. 10, pp. 2279–2301, 2002. [14] S. Bhattacharyya, P. Pal, and S. Bhowmick, “Binary image denoising using a quantum multilayer self organizing neural network,” Appl. Soft Comput., vol. 24, pp. 717–729, Nov. 2014. [15] W. B. Tao, J. W. Tian, and J. Liu, “Image segmentation by three-level thresholding based on maximum fuzzy entropy and genetic algorithm,” Pattern Recognit. Lett., vol. 24, no. 16, pp. 3069–3078, 2003. [16] K. Hammouche, M. Diaf, and P. Siarry, “A multilevel automatic thresholding method based on a genetic algorithm for a fast image segmentation,” Comput. Vis. Image Underst., vol. 109, no. 2, pp. 163–175, 2008. [17] B. Bhanu, S. Lee, and J. Ming, “Adaptive image segmentation using a genetic algorithm,” Syst. Man Cybern. IEEE Trans., vol. 25, no. 12, pp. 1543–1567, 1995. [18] L. A. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, no. 3, pp. 338–353, 1965. [19] G. C. Karmakar, L. Dooley, and S. M. Rahman, “Review on fuzzy image segmentation techniques,” Des. Manag. Multimed. Inf. Syst. Oppor. challenges, pp. 282–313, 2001. [20] J. C. Bezdek, R. Krisnapuram, and N. R. Pal, Fuzzy models and algorithms for pattern recognition and image processing, vol. 4. Springer-Verlag New York Inc, 2005. [21] J. V De Oliveira and W. Pedrycz, Advances in fuzzy clustering and its applications. Wiley Online Library, 2007. [22] M. Hasanzadeh and S. Kasaei, “Fuzzy image segmentation using membership connectedness,” EURASIP J. Adv. Signal Process., vol. 2008, no. 1, p. 417293, 2008. [23] U. Kaymak and M. Setnes, “Fuzzy clustering with volume prototypes and adaptive cluster merging,” Fuzzy Syst. IEEE Trans., vol. 10, no. 6, pp. 705–712, Dec. 2002. [24] M. T. Hagan, H. B. Demuth, M. H. Beale, and B. University of Colorado, Neural network design. PWS Pub, 1996. [25] R. Lippmann, “An introduction to computing with neural nets,” ASSP Mag. IEEE, vol. 4, no.
[26] [27] [28]
[29] [30]
[31]
[32] [33]
[34] [35] [36] [37] [38] [39]
[40] [41] [42]
[43] [44]
[45] [46]
[47]
[48]
2, pp. 4–22, 1987. M. Acharyya and M. K. Kundu, “Image segmentation using wavelet packet frames and neurofuzzy tools,” Int. J. Comput. Cogn., vol. 5, no. 4, pp. 27–43, 2007. T. Kohonen, “The self-organizing map,” Proc. IEEE, vol. 78, no. 9, pp. 1464–1480, Sep. 1990. V. V Srinivas, S. Tripathi, A. R. Rao, and R. S. Govindaraju, “Regional flood frequency analysis by combining self-organizing feature map and fuzzy clustering,” J. Hydrol., vol. 348, no. 1, pp. 148–166, 2008. C. Dongxiang, “Self-Organizing Map-Based Color Image Segmentation with k-Means Clustering and Saliency Map,” ISRN Signal Process., vol. 2011, 2011. M. Sezgin, O. K. Ersoy, and B. l Yazgan, “Segmentation of remote sensing images using multistage unsupervised learning,” in Optical Science and Technology, the SPIE 49th Annual Meeting, 2004, pp. 616–623. M. L. Goncalves, M. L. A. Netto, J. A. F. Costa, and J. Z. Junior, “An unsupervised method of classifying remotely sensed images using Kohonen self‐organizing maps and agglomerative hierarchical clustering methods,” Int. J. Remote Sens., vol. 29, no. 11, pp. 3171–3207, 2008. G. W. Jiji and L. Ganesan, “A new approach for unsupervised segmentation,” Appl. Soft Comput., vol. 10, no. 3, pp. 689–693, 2010. D. M. Ristic, M. Pavlovic, and I. Reljin, “Image segmentation method based on selforganizing maps and K-means algorithm,” in Neural Network Applications in Electrical Engineering, 2008. NEUREL 2008. 9th Symposium on, 2008, pp. 27–30. A. R. F. Araújo and D. C. Costa, “Local adaptive receptive field self-organizing map for image color segmentation,” Image Vis. Comput., vol. 27, no. 9, pp. 1229–1239, 2009. J. Vesanto and E. Alhoniemi, “Clustering of the self-organizing map,” Neural Networks, IEEE Trans., vol. 11, no. 3, pp. 586–600, 2000. K. Tasdemir and E. Merényi, “Exploiting data topology in visualization and clustering of selforganizing maps,” Neural Networks, IEEE Trans., vol. 20, no. 4, pp. 549–562, 2009. D. Brugger, M. Bogdan, and W. Rosenstiel, “Automatic cluster detection in Kohonen’s SOM,” Neural Networks, IEEE Trans., vol. 19, no. 3, pp. 442–459, 2008. A. K. Helmy and G. S. El-Taweel, “Image segmentation scheme based on SOM–PCNN in frequency domain,” Appl. Soft Comput., vol. 40, pp. 405–415, Mar. 2016. M. I. Chacon-Murguia and G. Ramirez-Alonso, “Fuzzy-neural self-adapting background modeling with automatic motion analysis for dynamic object detection,” Appl. Soft Comput., vol. 36, pp. 570–577, Nov. 2015. K. Deshmukh and G. N. Shinde, “An adaptive neuro-fuzzy system for color image segmentation,” J. Indian Inst. Sci, vol. 86, pp. 493–506, 2006. E. Aghajari, “Incorporating FCM and Back Propagation Neural Network for Image Segmentation,” Int. J. Comput. Commun. Technol., vol. 2, no. Viii, pp. 121–126, 2011. L. C. M. Andrade, M. Oleskovicz, and R. A. S. Fernandes, “Adaptive threshold based on wavelet transform applied to the segmentation of single and combined power quality disturbances,” Appl. Soft Comput., vol. 38, pp. 967–977, Jan. 2016. T. Kohonen, “The self-organizing map,” Proc. IEEE, vol. 78, no. 9, pp. 1464–1480, 1990. E. Aghajari, H. Lotfi, and D. Gharpure, “Towards interpretation of self organizing map for image segmentation,” in Informatics, Electronics & Vision (ICIEV), 2013 International Conference on, 2013, pp. 1–6. U. Kaymak and M. Setnes, “Fuzzy clustering with volume prototypes and adaptive cluster merging,” IEEE Trans. Fuzzy Syst., vol. 10, no. 6, pp. 705–712, Dec. 2002. F. Di Martino_affanb, V. Loia, and S. Sessa, “Extended fuzzy C-means clustering algorithm for hotspot events in spatial analysis,” Int. J. Hybrid Intell. Syst., vol. 5, no. 1, pp. 31–44, Jan. 2008. R. Unnikrishnan, C. Pantofaru, and M. Hebert, “Toward objective evaluation of image segmentation algorithms.,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 929–44, Jun. 2007. Y. J. Zhang, “A survey on evaluation methods for image segmentation,” Pattern Recognit., vol. 29, no. 8, pp. 1335–1346, 1996.
[49] [50]
[51]
[52]
[53] [54] [55]
[56]
H. Zhang, J. E. Fritts, and S. A. Goldman, “Image segmentation evaluation: A survey of unsupervised methods,” Comput. Vis. image Underst., vol. 110, no. 2, pp. 260–280, 2008. Z. T. Birch, “an efficient data clustering method for very large databases/T. Zhang, R. Ramakrishnan, M. Livny,” in Proc. of the ACM SIGMOD Conf. on Management of Data.– Montreal: ACM Press, 1996, pp. 103–114. H. H. Muhammed, “Unsupervised Image Segmentation Using New Neuro-Fuzzy Systems,” in Swedish Society for Automated Image Analysis Symposium-Proceedings of SSAB 2002, 2002, pp. 83–87. D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, 2001, vol. 2, pp. 416–423. H. Wang and J. Oliensis, “Generalizing edge detection to contour detection for image segmentation,” Comput. Vis. Image Underst., vol. 114, no. 7, pp. 731–744, 2010. S. S. Rautaray and A. Agrawal, “Vision based hand gesture recognition for human computer interaction: a survey,” Artif. Intell. Rev., vol. 43, no. 1, pp. 1–54, 2015. T.-K. Kim, K.-Y. K. Wong, and R. Cipolla, “Tensor canonical correlation analysis for action classification,” in Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, 2007, pp. 1–8. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 33, no. 5, pp. 898–916, 2011.
Original Image Features selection & Feature Extraction
Prior Segmentation Using SOM
EFCM
Labeling using EFCM Output
Final Segmentation Segmented Image
Fig.1.The proposed SEEFC Algorithm
Fig.2: Random data selection a) four quadrant division of the image for equal distribution, b) Random distribution, c) Random data map on image, (Berkley #3096) [56]
W1M
W11 W12
(BMU) V1 V2 V3 V4 V5 V6 V7 V8 V9
WL1
Hexagonal Grid
WLM
Fig.2.The training pattern vector (Vp) consist of Discrete wavelet filtered images, Pixel Values, Gradient, Standard deviation, Variance &Energy, The BMU (wm) of training pattern vector (Vp) is shown in colorful shade on self-organizing map of hexagonal grid. The first and second neighborhood unit of BMU has been shown in red and green shades, respectively. The weights of the individual units (w11, w12,w1M, wL1, wLM) are also be seen on SOM map.
2nd Max Selected Neuron
START
W11
Feature Extraction
Random Selection of pixels for Training
Creating SOM and Network Training
Hits map of Best Match Units (BMU)
3rd Max Selected Neuron
Find Max Neuron Hits & Save Its Weights
W1515 Find Next Max Neuron Hits
Yes
1st Max Selected Neuron
(b)
If Max Neuron Distance
(c)
No
No
If Selected neurons = Cluster No.
Yes
Find Euclidean Distance of Each Prototype Vectors
Prototype Vectors END to Their Assignment Corresponding Cluster
Segmented Image
END
(a) Fig. 3. a) Overview of SOM segmentation, b)Illustration of best matching units hits of training pattern vectors and selected number of neurons as a reference for each cluster in order to interpret SOM network result.(threshold value =13) ,c) segmentation Image for airplane image from Berkeley database.
Fig.4. Original Image (#3096) from Berkley Database, b) SOM Trained Map
Fig.5. SOM and EFCM Integration in a Glance View
Fig.6. Sample Image and segmented images for image #3096
Fig.7.Illustration of TP-TN-FP & FN
Fig.8.a,b) Original Image, c,d) SEEFC Segmentation Result
Fig.9.State of the Art Images a,b)Original Image, c,d)SEEFC Segmentation Result
Fig.10.Result of State of the art techniques on pepper image
Fig.11. Image samples and Ground truths and segmented images results.
Fig.12.Comparison Of SEEFC Algorithm Performance
With Contour Detection and Normalize Cut Techniques For Images With Distinctiveness<=0.2
Fig.13.Comparison Of SEEFC Algorithm Performance
With Contour Detection and Normalize Cut Techniques For Images With 0.2
Fig.14.Comparison Of SEEFC Algorithm Performance With Contour Detection and Normalize Cut Techniques For Images With 0.4
Fig.15.Comparison of SEEFC Algorithm Performance with Contour Detection and Normalize Cut Techniques for Images with Distinctiveness>0.6
100 CD SEEFC
Accuracy %
80
FCM
60
SOM
40
NC
SEEFC NC CD SOM FCM
20
0
0
0.2
0.4
0.6
0.8
Distinctiveness Factor Fig.16: Accuracy Plot of SEEFC Comparison with Other Algorithm
100
Sensitivity Variation
90 80 70 60 50 40 30 20 10 0
Elephant
Plane Bird
Eagle
Fox
Shipman
Garden
Church
Moon
Paratrooper
Images
Fig.17:The Performance Variation of SEEFC Algorithm for Object Sensitivity
100 90
Accuracy Variation
80 70 60 50 40 30 20 10 0
Plane
Elephant Bird
Shipman
Fox
Eagle
Garden
Church
Moon
Paratrooper
Images
Fig.18:The Performance Variation of SEEFC Algorithm for Object Accuracy
Fig.19: Stability of SEEFC on presence of Gaussian Noise and Salt &Pepper Noise
Fig.20:SEEFC Algorithm Result for Hand Databases available in the literature
Table 1: Accuracy of Different Algorithm Performance Based on Distinctiveness Factor
DISTICTIVNESS
IMAGES WITH DIFFERENT DISTINCTIVENESS FACTOR 0-0.2 0.2-0.4 0.4-0.6 >0.6
Car Pot Statue Boatman Man Surfing Plane Bird
SEEFC 73.52* 87.9* 97.8* 83.4 87.9* 65* 80.08 88.79*
SEGMENTATION ALGORITHM NC CD SOM FCM 67 46 73.31 68.52 61 62 45.91 51.47 45 68 71.5 92.26 20 87* 69 83 71 73 81.7 82.42 15 40 43.2 45.56 84 85* 73.2 81 86 79 88.76 87.9
Table 2: Average Sensitivity and Accuracy of SEEFC algorithm (%)
Image Plane Bird Elephant Shipman Fox Garden Eagle Moon Church Paratrooper %Sensitivity
77.77 83.04
90.17
94.87 81.12
%Accuracy 80.08 88.79
78.63
65 98.48
98.64
Average
88.31
80.67
74.43
76.59
84.56
97.97 95.13
95.7
83.3
91.76
88.1
Table 3: Q(I) value for SEEFC performance
Images
Q(I)
Plane Bird Elephant Shipman Fox Eagle Moon Garden Church Paratrooper
SEEFC 0.024±0.25% 0.0797±0.11% 0.0546±0.38% 0.1348±0.21% 0.0140±0.42% 0.0295±0.06% 0.0042±0% 0.0477±0.37% 0.0422±0.2% 0.2881±1.72%
Table 4: The Average performance of SEEFC Algorithm
Images Plane Bird Elephant Shipman Fox Eagle Moon Garden Church Paratrooper
∆% (average performance of SEEFC ) Sensitivity Accuracy Object Object 77.77 ± 9.35 90.72 ±7.14 83.04 ± 6.66 91.8 ±0.85 90.17 ± 0.02 78.63 ± 0.04 94.87 ± 3.14 65 ± 2.74 81.12 ± 7.98 98.48±0.18 98.64± 0.53 97.97 ±0.57 88.31 ± 0.045 95.13 ±0.008 80.67 ± 0.002 95.7 ±0 74.43 ± 3.69 83.3 ±0.59 76.59 ± 2.33 91.76 ±5.83
Table5: Evaluation Parameters of Stability of SEEFC on presence of Noises
Images
Evaluation Parameter
Object
Sensitivity Accuracy
5% 90.5±3.66 94±1.5
10% 89±4.1 85±2
Gaussian 15% 81±2.54 73.5±5.25
Type of Noise Salt &Pepper 2% 5% 87±4.6 95.8±2.36 88.9±0.2 91.6±3.6
S1568-4946(17)30007-8 http://dx.doi.org/doi:10.1016/j.asoc.2017.01.003 ASOC 3997
To appear in:
Applied Soft Computing
Received date: Revised date: Accepted date:
17-7-2016 28-11-2016 4-1-2017
Please cite this article as: Ebrahim Aghajari, Gharpure Damayanti Chandrashekhar, Self-Organizing Map based Extended Fuzzy C-Means (SEEFC) Algorithm for Image Segmentation, Applied Soft Computing Journal http://dx.doi.org/10.1016/j.asoc.2017.01.003 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Self-Organizing Map based Extended Fuzzy C-Means (SEEFC) Algorithm for Image Segmentation Ebrahim Aghajari a, Gharpure Damayanti Chandrashekhar b a b
Department of Electrical Engineering, Ahvaz Branch ,Islamic Azad University ,Ahvaz, Iran Department of Electronic Science, University of Pune, Pune, Maharashtra, India
Graphical Abstract
Figure: SOM and EFCM Integration in a Glance View
*Highlights (for review) The motivation for this integration was the difficulties faced in using SOM for Image segmentation. The number of clusters needs to be specified for improving the performance. For different images the map sizes have to be optimized. It was also observed that SOM has limited capabilities for the representation of hierarchical relation of mapped data making interpretation difficult. In this work a novel attempt taking advantage of capabilities of self-organizing map and extended fuzzy clustering for Image segmentation is reported. Incorporating SOM we can find the essential information by mapping image data to a two dimensional space and with the help of EFCM we estimate the number of clusters. Finally the cluster centers are used for clustering purpose and image segmentation.
Original Image Features selection & Feature Extraction
Prior Segmentation Using SOM
EFCM
Labeling using EFCM Output
Final Segmentation Segmented Image
Answer to reviewer Comments: We believe that we have answered all the queries of the referees. The paper has been checked word by word and all the points has been considered and corrected as per referee's comment. We hope that the revised version of manuscript considered all the points.
Abstract: A Novel hybrid algorithm based on Self-Organizing-Map (SOM) and Extended Fuzzy CMeans (EFCM) named self-organizing-map based extended fuzzy c-means (SEEFC) has been designed and implemented for image segmentation. The proposed algorithm works in three stages. At first, the images are decomposed by discrete wavelet transform (DWT) into various frequencies and gradient, pixel value and statistical parameters are obtained to form a feature prototype. The feature prototypes are input to the Self Organizing Map (SOM). Finally, the codebook vectors of the trained SOM have been clustered automatically using Extended Fuzzy C-Means (EFCM). The clustered codebook vector centers are used for image segmentation based on minimum distance criterion. The proposed method has been tested with images from Berkeley’s database and results obtained are promising. The segmentation results are evaluated against the ground truth. Comparisons with another state of the art approaches indicate advantages of the SOM based EFCM algorithm.
Keywords: Self Organizing Map (SOM), Extended Fuzzy C-Means (EFCM), Discrete Wavelet Transform (DWT), Image Segmentation, Feature Extraction.
1. Introduction: Image segmentation is one of the most interesting and critical steps towards image understanding and analysis. It can be defined as a process which aims at partitioning an image into meaningful and separate regions. Indeed the goal of segmentation is to simplify or change the representation of an image into something that is more meaningful and easier to analyze. Extensive research has been carried out in developing different algorithms and methods for image segmentation but even today, no single standard method of image segmentation has emerged [1].Rather, there are collections of ad hoc methods that have received some degree of popularity. However all the research work performed on image segmentation can be classified into two broad categories. Traditional methods like thresholding [2], morphological methods, edge-based segmentation [3], Normalized Cut method (NC)[4], Efficient graph-based method (EG)[5] Mean shift method (MS)[6] ,Level-set method (LS)[7],Ratio-contour method (RC) [8] and many more. Soft computing techniques based on Fuzzy Theory [8][9][11][12],Artificial Neural Network (ANN)[13][14] and Genetic Algorithm (GA)[12][13][17], are predominantly used for segmentation and are preferred by researchers because of the adaptive nature and accuracy. The theory of fuzzy set has been introduced in 1965 by Zadeh [18] as a way of representing uncertainties in everyday life. Challenges in image segmentation have encouraged researchers to develop fuzzy segmentation algorithms by considering image regions as fuzzy objects, where an image pixel may be partially classified into multiple potential classes and the boundaries between intensities of different objects can be well defined. The theory of fuzzy sets is adopted to effectively model the fuzziness of image pixels which might be caused by inherent heterogeneity and imaging device artifacts (e.g., blurring, imposed noise and background variation).There are several image segmentation methods based on fuzzy concept reported in[19][20][21]among which fuzzy clustering [21] and fuzzy connectedness[22] are two well-known techniques for this purpose. FCM is used for data clustering, with the drawback that the number of clusters has to be specified. Extended fuzzy CMeans (EFCM), has the capabilities of finding an optimum number of clusters automatically. The major problem in applying EFCM to Image segmentation is that it becomes computationally expensive as the dimensions of the data increases [23]. Artificial neural networks (ANN) [24][25], are signal processing systems that try to emulate the biological nervous systems, by providing a mathematical model of the combination of numerous neurons connected in a network. In general, neural networks fall into one of the two following categories: Supervised and unsupervised neural networks[26]. Our review of literature related in this areas shows that among supervised NN’s the Hopfield neural network, multilayer perceptron and back propagation are well-known techniques for segmentation purpose. Among unsupervised methods the self-organizing map (SOM) [27] is one of the best approaches for this purpose. Although it has been proved that SOM makes itself a promising clustering tool and it is very helpful for visualization and cluster extraction but it has limitations from Image segmentation point of view. Some researchers have used SOM with other methods for better performance [28]–[33].It has been noted that the map size affects the clustering ability of SOM. Araujo and Costa have proposed SOM network with a self-adaptive topology for color image segmentation [34]. Dongxiang chi [29]combined SOM with K-means and saliency map for clustering. It is observed that SOM-K method is an unsupervised method requiring less computation and no human intervention. The saliency map was used to improve the segmentation performance. An optimizing number of clusters for k-means from 2 clusters to √𝑛 cluster (where n is the map size) was done by applying entropy index. Concalves et al [31] presented SOM-based clustering method for remotely sensed images. The hit map information is investigated and zero-hits nodes and heterogeneous nodes are filtered. Secondly, the clustering index is applied to get the best clusters. This method has been applied for natural scene image segmentation. Review of the literature indicates that most of the SOM-based clustering methods emphasize on analyzing the SOM derived prototype parameters [35]–[37]. Enlightened by [29] and [37] the prototypes are often integrated through the second stage or combined with hit numbers for automatic detection of clusters.
It may be noted that Neuro-fuzzy hybridization is a widely used tool of soft computing paradigm and has emerged as a promising field of research in recent years [38] [39]. Neuro-fuzzy systems combine the advantages of both the uncertainty handling capability of fuzzy systems and the parallel learning ability of neural networks [40].The motivation for this integration was the difficulties faced in using SOM for Image segmentation. The number of clusters needs to be specified far improving the performance. For different images, the map sizes have to be optimized. It was also observed that SOM has limited capabilities for the representation of hierarchical relation of mapped data making interpretation difficult. [27] In this work a novel attempt taking advantage of capabilities of a self-organizing map and extended fuzzy clustering for image segmentation is reported. Incorporating SOM we can find the essential information by mapping image data to a two-dimensional space and with the help of EFCM we estimate the number of clusters. Finally the cluster centers are used for clustering purpose and image segmentation [41]. The new algorithm is named as Self-Organizing Map based Extended Fuzzy C-Means (SEEFC). This paper is organized as follows. Section 2 presents the methodology and section3 deals with the experimental results and discussions. Section 4 presents the comparison of results and finally section 5 concludes the paper. 2. Methodology This study explores the use of SOM followed by EFCM to get the number of clusters and the cluster centers automatically, for image segmentation. The algorithm extracts the features initially followed by a module for features selection. The selected features are used for prior clustering using SOM and the SOM map is given to EFCM algorithm for further processing. The EFCM is used to optimize a number of clusters based on coarse clustering carried out by SOM. The cluster centers are obtained and individual pixels are labeled using minimum distance criterion for final segmentation task. The flow chart of the methodology implemented is shown in Fig.1. 2.1 Feature extraction The main purpose of feature extraction is representing the original data set by proper features to distinguish one input pattern from another. The selected features must represent the characteristics of the input image regions. The features that will be used as inputs are extracted as discrete wavelet transforms features including four features namely approximation, diagonal, horizontal and vertical in addition to pixel values, Gradient and statistical features including variance, standard deviation and energy. The following subsection will describe the features in detail. 2.1.1
Discrete wavelet transform (DWT)
The wavelet transform is important to provide a compact description of signals (or images) that are limited in time (or spatial extent) and it is very helpful in the description of edge and line that are highly localized [42]. Wavelet transform decomposes a signal into a set of basis functions. These basis functions are called wavelets. Wavelets are obtained from a single prototype wavelet called mother wavelet by dilations and shifting:
a ,b (t )
1 t b ( ) a a
a ,b (t )
(1)
Where (a) is the scaling parameter and (b) is the shifting parameter. The scaling and wavelet function (x , y ) and (x , y ) here. The scaled and translated basis are two variable functions denoted functions are defined as: ji , m , n ( x, y ) 2 j / 2 (2 j x m, 2 j y n)
(2)
i j /2 j j i {H ,V , D }. j , m , n (x , y ) 2 (2 x m , 2 y n ),
(3)
By using wavelet on an image for one level, four images are obtained which correspond to the approximation and details in the images. The approximation coefficients matrix CA and details coefficients matrices CH, CV, and CD(horizontal, vertical, and diagonal, respectively), obtained by wavelet decomposition of the input matrix X which is the same as (LL,LH, HL, HH) have been used as features for proposed algorithm.
2.1.2 Statistical Parameters 2.1.2.1 Variance The variance (σ2) is a measure of how intensity values in the image are far from the mean. We use the 3*3 window template for calculating the variance. Here is how it is defined: n
2 =
(X i 1
i
)2 (4)
N
Where the variable is the mean of intensity values and N is the number of pixels in an image. 2.1.2.2 Standard deviation The Standard Deviation (σ) is a measure of how spread out intensity values in image is. A low standard deviation indicates that the pixel values tend to be very close to the mean intensity, whereas high standard deviation indicates that the pixels values are spread out over a large range of values. 2.1.2.3 Energy There is more than one definition of "energy" in image processing but in general the energy can be defined as spectral density which describes how the energy of a signal or a time series is distributed with frequency. Here, the term energy is used in the generalized sense of signal processing to denote a variance of the signal. The 3*3 window template has been used for energy calculation.
ERG = P(i, j)2
(5)
i, j
2.1.3
Feature Selection
The optimized feature vectors for this study were based on two domains, frequency and statistics. Deployment of features of all pixels in the image in the training process can be quite onerous. So it is preferable to construct a set of feature vectors in such a way that the original data variance in the image is retained. The training data selection is carried out based on the following principals: At first the selected data should accurately represent the characteristics of input image. Secondly it has to address the significant information and component of the image. Different methods were examined for pixel selection in order to choose a subset as training data. As shown in following figure, the training candidate pixels were selected randomly from each quadrant as shown in Fig.3.5 (a) and the total features vectors were 10,000. Use of equal number of training feature vectors from each quadrant and otherwise, did not make much difference in the final result. The other method, was the random selection of data in entire image as shown in Fig.3.5 (b & c). Various experiments carried out shows that the technique of random data selection for generating a subset of feature vector is faster and easier. 2.2 Self organizing map (SOM) Self-organizing map was introduced by Kohonen in 1982 [43]. The idea was to represent, huge amount of data by finite number of samples. SOM is an unsupervised neural network and it is able to organize itself. It consists of n dimensional input space with m dimension of output space. The map is usually 2D structure with unit of map associated with a weight vector. (6) Nij Wij ;1 i L,1 j M
Where Nij is a 2D grid of map (also called neuron),Wij is the weights vector (also called as prototype vector) assigned to the (i,j)’th unit of SOM architecture and L& M are number of rows and columns. The formation of the self-organizing map consists of 3 steps.1-competition, 2- cooperation, 3-synaptic adaption. 1- Competition In this section calculation of best matching unit (BMU) is done according to the Euclidean distance between weight vector and training input patterns. The weight of the neurons is defined as Wij w1 , w2 ,..., wd where d is the dimension of weight vectors and in this case based on the number of features chosen it is nine. The features, namely discrete wavelet transform (CA, CD, CV, CH), pixel value (PV), gradient (GXY), standard deviation (STD), variance (VAR), and energy (ERG) are used to form the feature vector. In this study, training pattern vector set is defined as T {Vp ,1 p n}where the Vp is the P’th training feature vector and n is the total number of training pattern vectors which is 10’000 in this study. During training the SOM tries to find the neuron in its map whose weight is nearest to the training pattern vector (Vp) based on Euclidean distance. The Neuron in the SOM architecture whose distance from training pattern vector (Vp) is minimum, is called as best matching unit (BMU) of that training pattern vector (Vp) and denoted as Wm. The Euclidean distance and BMU are determined as: (7) dist (Vp Wm ) (8) || Vp - Wm || min ij {|| Vp - Wij ||} 2- Synaptic Adaption The BMU and neighbors around it have their weights W ij updated with step t and will be represent by (9) W (t 1) W (old ) (t ) (t )(V (t ) W (t )) ij
ij
The new weight for a node is the old weight, plus a fraction of the old weight and the input vector wherein𝜃(𝑡)is based on distance from the BMU, (t ) is the learning rate, exponential decay function that insure the SOM will converge.
t
(t ) 0 exp( ) Represent time constant and t is the time step.
(10)
3- Cooperation Ideally after finding the BMU the winner neuron weight has to be updated. But in practice the BMU and neighbors around it will be updated based on Eq. (11)
dist 2 (t ) exp( 2 ) 2 (t )
(11)
In the above expression (t ) is the topological neighborhood center on BMU where dist is the position of BMU in SOM unit. isan exponential decay function that shrinks on each iteration until eventually the neighborhood is just BMU itself. The amplitude of the topological neighbors decreases monotonically with the increasing lateral distance.
t
(t ) 0 exp( ) 2.2.1
(12)
SOM Segmentation
The interpretation of output of SOM is a tricky issue as reported in the literature. The Umatrix is one of the simplest options for that. In this paper we use the following method for this purpose[44]. Here we consider that the winner neuron hits for interpretation of results using SOM.Fig.3-a shows the flowchart of proposed algorithm. First of all the extracted features of various
pixels in the image will be generated. Considering the features of all the pixels in the image would take lots of time for SOM training. To overcome the problem, the proposed algorithm uses random selection of pixels in the entire image as a representative training set. In this algorithm number of feature vectors selected for training purpose was 10’000. In the next step the sample features are given to the created SOM architecture for training purpose. The topology lattice is hexagonal and its size is optimized to(15*15). Initially use of SOM alone for segmentation was investigated. Herein the number of hits of winner neurons and Euclidean distance between two winners has been used for segmentation. We predefine number of clusters and the Euclidean distance threshold on the hexagonal map Fig.3-b. The threshold value can vary from 0 to maximum distance between the 1st neuron position W11 and the last position W1515 i.e.is 19.79. Initially the neuron with maximum hits is considered as first cluster representative. Then the second neuron with maximum hits is obtained. If its distance from the first winner neuron is more than threshold value (13 in this case) it will be considered as representative of second cluster and else the algorithm will select the next highest one and the loop goes on. As per the predefined number of clusters (3 in this case) the process will continue till the algorithm meets conditions. Finally the weights of selected winning neurons are considered for image segmentation by minimum distance criterion, calculating Euclidean distance of prototype vectors from cluster representative vectors as seen in Fig.3-c. The problem with SOM is the number of segments has to be specified in prior and the map size has to be big enough to give acceptable cluster separation. For the said purpose different methods has been studied and extended fuzzy clustering found to be suitable because of its robustness and accuracy [23]. 2.3 Extended Fuzzy Clustering Fuzzy clustering is a process which allocates data points to clusters based on the fuzzy memberships related to Euclidean distance. Most widely used fuzzy clustering is Fuzzy C-Means (FCM) Algorithm[20] which partition a collection of n data elements into collection of ‘C’fuzzy clusters with respect to some given criterion. But FCM algorithm presents two shortcomings: one, a priori the number ‘C’ of prototypes must be defined or one calculates C as minimum or maximum of a suitable function. Secondly, the cluster centers tend to locate in areas with high concentrations of features and at times, the zones with low density data points could be relevant. Generally speaking, the distribution of the data is sensitive to the initialization phase. The EFCM algorithm was first proposed in[23] to overcome the above shortcoming of the FCM method . Let’s consider X R as a sample dataset with the following matrix:
x11 X x M1
x1N xMN
(13)
n
Where X={x11,...,xMN}⊂R , is composed of M input vectors having N features defined as xj= (x1j,x2j, x3j…,xnj)T. For implementation of the EFCM algorithm with matrix X as input one can follow the following steps. Step 1: Compute center of cluster prototype
(U ) X (U ) N
i
m
k 1 N
k 1
ik
m
k
1
(14)
ik
Where Xk is an input feature vectors,m≥1 is the fuzzifier parameter. Please note that in the work done, all tests and examples have been calculated, with a fuzziness parameter m=2.Uik is the membership
degree of Xkto the i-th cluster where set of clusters are i=1,…,C. Viis the center of I’th cluster and the set can be represent as V={v1,…,vC}⊂Rn . Step 2: Computing the radius of cluster prototype using Eq. (15)
Pi
ri
N k 1
(U ik )m (X k V i )(X k V i )T
N k 1
(U ik )
(15)
m
1/n
Pi
(16)
M
The Pi is the covariance matrix whose determinant gives the volume of the i-th cluster and β is thevolume prototype coefficient and m defines the number of clusters in each iteration. β, initially has to be predefined by the user and in this study selected as 1 and it will be improved duration of process under circumstances. Step3: Compute the distance to the volume cluster In the EFCM algorithm the objective function is the following formula M
J i 1
N
U k 1
m ik
(dik ri )
(17)
The term dikis the distance of each volume prototype x from the center of volume cluster i. Unlike FCM this distance is calculated as given below in EFCM algorithm. (18)
d ik max(0, (x k v i )(x k v i )T ri ) Where in xk and vi are k-th prototype vector and i’th cluster center respectively. Step4: Update the membership matrix For 1≤k≤N, let k {i | d ik 0}
(19) For calculation of membership function we consider the parameter k equal to the number of clusters I for which dik (distance) =0.i.e for prototype vectors inside the clusters. Thus membership function is determined using the following formula when the k .
U ik
1
M j 1
(d ik / d jk )
2/ m 1
, k 0
(20)
1≤i≤M
Otherwise 0
ifdik> 0 1≤i≤M , k
Uik=
0
1/ k ifdik = 0
Step5: Select the most similar cluster
S ij
N k 1
min(U ik ,U jk )
N
U ik
j 1
1≤i,j≤M
(21)
(i * , j * ) arg max(S i , j ) (i , j ) i j Determination of the number of clusters is achieved by computing the Sij (similarity measure)for the ith cluster and j-th cluster.
Step6: Merging similar clusters The merging between the two clusters i and j is done when at l-th iteration we have the following condition. Note that the 𝜀1 is user predefined criterion which is set to 0.01 in this paper.
If S (l )i* , j* S (l 1)i* , j* 1
(22)
Parameter 𝛼 ∈ [0,1] is the merging criteria which is determined with the following formula: Let
1 M 1
(23)
If Si* , j* So that cluster i-th and j-th will be merged together if above condition is true.
U l i *k : (U ( l )i *K U ( l ) j *K ) Remove j from U After merging the cluster M becomes:
M M 1 Else the program enlarge the radius (ri) volume prototype by a parameter𝛽factor which is defined recursively as follows
(l ) min(M (l 1) , (l 1) 1)
(24)
Wherein 𝛽is initially set to 1.𝛽(0) = 1 Step7: the process stops when the condition below is met. Note that the 𝜀2 is user predefined criterion which is set to 0.001 in this paper. Until U (l ) U (l 1) 2
(25)
As it is mentioned in paper [45],[46] the problem with this method is that, in fact it is computationally expensive and with increasing the number of input data the program would take a very long time. The idea of using EFCM for clustering and image segmentation was explored but it was observed that amount of time required with a minimal data set of 10,000, is prohibitive. Considering the size of the image, for good segmentation the data set used has to be exhaustive, ruling out the use of EFCM on the image directly. However as it is known, the beauty of self-organizing map is its ability to represent huge amount of input data in a finite 2D representation (15*15 in our case). Using the SOM output map as input representative input, one can overcome the time limitation of robust EFCM. The new algorithm utilizes the codebook i.e. 2D map of SOM as input data to EFCM algorithm to get number
of clusters in minimum time automatically. In the following subsection 2.4 the proposed method will be elaborated in detail.
2.4 Developed Neuro-Fuzzy Segmentation Method (SEEFC) This study explored the use of SOM followed by EFCM to get the number of clusters and the use of cluster centers as a reference automatically, for image segmentation. The technique is named Self Organizing Map EFCM (SEEFC). Fig.4 shows an image (321*481) pixel from Berkley Database (#3096). The optimized 9 feature vectors for the image are calculated resulting in: (321*481=154401 Feature Vectors). 10000 random samples are selected for training the SOM as described before. Fig.4 (b) shows a SOM of (15*15) size and the huge image data can be represented with 15*15 =625 weight vectors which are only 2.25% of the image data. The size of SOM map was varied from 10*10 to 25*25 for which the data reduction is over a range of 1% to 6.25%. With experimentation, the size of SOM was optimized to 15*15 for this work. The reduced data, clustered using SOM, is input to EFCM algorithm for finding optimum number of clusters. The cluster centers are used for segmentation. This method is named as SEEFC. Regardless of the size of SOM and its related topology the SEEFC technique can perform segmentation smartly as per the methodology shown in Fig.5. In the beginning, the input image is normalized. The next module extracts features using Discrete Wavelet Transform, Gradient and pixel value, as well as Statistical parameter (variance, standard deviation and energy). The proposed method SEEFC uses the random selection of pixels throughout entire image to be used by self-organizing map neural network for training purpose. Once the SOM is trained, the map weight vectors created by SOM are used for obtaining the optimum number of clusters and cluster centers using extended fuzzy clustering method (EFCM). For segmentation the Euclidean distance between the cluster centers obtained from EFCM output and feature vector of the original image is calculated. The pixel is labeled based on the minimum distance criterion.
Different images from Berkeley database have been used for testing the performance of the proposed method. The results show the ability of the SEEFC method in segmentation. A typical result for the airplane image from Berkley database (code no: #3096) in Fig.6 (a),Fig.6 (b) and Fig.6 (c) shows the segmented result by SOM and Neuro-Fuzzy segmentation techniques (SEEFC) respectively. The SEEFC algorithm found 3 clusters for the image and the process of segmentation of the image was carried out on this basis. As it is evident visually, the refinement in segmented airplane image and background is clearly visible. The airplane nose seems mixed with background using SOM technique Fig.6 (b) for segmentation but the segmentation is refined by proposed method SEEFC and good separation can be seen between object and background Fig.6 (c). Also, while observing the segmentation result from background point of view, the uniformity in background is refined compared to SOM method. 2.5 SEEFC Performance Evaluation Despite of many approaches for image segmentation validation, it is still difficult to characterize whether one particular segmentation technique, produces better segmentation than another. However various attempts have been made and studied [47][48] to define parameters to assess the performance of image segmentation. The reasons for this are manifold. One among them is the difficulties that exist in validating the segmentation algorithm due to variability of the region causes difficulty in defining ground truth. In reality as segmentation is an ill-defined problem, there is no unique ground truth segmentation of an image against which the output of an algorithm may be compared. However one of the common segmentation assessments is based on comparison of segmented image against manually segmented or pre-processed reference image. In present work two
approaches namely qualitative comparison and quantitative evaluation have been used. All the images and ground truth used for evaluation are available at Berkeley database. Qualitative comparison is the most common method for evaluation the effectiveness of a segmentation algorithm, in which a human visually compares the image segmentation result for separate segmentation algorithm [49]. Quantitative analysis enables the performance of segmentation results to be evaluated statistically against ground truth. For these reason there are three parameters which have been selected for this evaluation which are Sensitivity, Accuracy and Q-index. The first two parameters are based on four different criterion parameters such as True-Positive (TP), True-Negative (TN), False-Positive (FP) and False-Negative (FN) as shown in Fig.7 and will be defined as following equations, Eq.1 Eq.2
Sensitivity = ((TP/ (TP+FN))*100% Accuracy=((TP+TN)/TP+FN+TN+FP))*100%
TP is True-Positive which is defined as number of pixels correctly labeled as belonging to the object class, TN is True Negative and could be explained as number of pixels correctly labeled as belonging to the not object class, FP is False-Positive and defined as number of pixels incorrectly labeled as belonging to the object class and finally FN (False-Negative) defined as number of pixels which were not labeled as belonging to the object class but should have been. An ideal segmentation is the one which achieves 100% sensitivity and accuracy. Sensitivity can give the information about the object of interest and how segmentation algorithm segmented the object perfectly. In this evaluation the image is segmented into two classes, “object of interest” and everything else as “not object of interest”. The terms accuracy points to the overall ability of the segmentation algorithm. The other quantitative parameter for segmentation assessment is Q (I) index proposed by Borsotti et al [50] and defined as follows. Eq.(1.3)
Q( I )
R ei2 R( Ai ) 2 1 R * [ ( ) ] 10000( N * M ) Ai i 1 1 log Ai
Where, I is the resultant image, N*M is the size of the image (I).R is the number of regions of the segmented image. ei is the sum of the Euclidean distance between the vectors of the pixels of region i and the vector attributed to region i in the segmented image. Ai is the area (as measured by the number of pixels) of the ith region. The R (Ai) represents the number of regions having an area equal to Ai. The lower Q value indicates better segmentation. This parameter gives the overall idea of how well the algorithm was able to perform the segmentation task. In the following description some sample images is selected for quantitative evaluation of the developed algorithms in terms of sensitivity and accuracy. 2.5.1 Segmentation of Simple Images Using SEEFC To analyse the performance of the developed algorithm few simple images of size 300*300 pixels images have been chosen as shown in Fig.8 (a & b).This experiment was carried out to test the ability of the SEEFC algorithm for the known number of clusters. Fig.8 (a) is an image, with the 2 different colors (white and Black) and the expected number of clusters are two. The SEEFC algorithm detected 2 clusters and the result is shown in Fig.8 (c). In the segmented image all the pixels are assigned perfectly as per expectation automatically. The other image is a text in which different gray shades color shown in Fig.8 (b). The SOM trained network weight vectors were given to the EFCM. The output of EFCM detected 3 optimum clusters and the segmented output can be seen in Fig.8 (d). In the segmented image all the similar gray shades belongs to the same cluster. The SEEFC algorithm has been tested on Berkley image segmentation database to study and understand its performance. For this analysis the algorithm is firstly tested on some simple images for which the number of clusters is known and then it is tested on a number of images.
2.5.2 SEEFC Result on Standard Images After evaluation of the developed Neuro-Fuzzy technique for simple images, the algorithm was tested on some standard image .Results obtained for two typical images reported in papers[51] have been presented in Fig.9.The figure shows segmentation using SEEFC algorithm along with other techniques. The first image is Rice image shown in Fig.9 (a), in fact the SEEFC algorithm processed the image and find the number of cluster as 3 in the image and the result is mapped the output of segmentation as shown in Fig.9 (c). The second image is a pepper image wherein SEEFC detected 4cluster and the result illustrated in Fig.10 (d).The pepper image has been segmented using different techniques as shown in Fig.10 (a & c) shows fuzzy weight gradient techniques (FWG) segmentation result on this particular image with 3 and 9 clusters respectively[51]. Fig.10 shows the segmentation results using various algorithms. Fig.9 (b) shows the segmented output of WINN technique for the pepper image with the number of cluster selected as 7. The Fig.10 (d&e) shows the result of SOM neural network and fuzzy C-means (FCM) segmentation output for 4 clusters. Fig.10 (f) shows ability of SEEFC technique on cluster selection and segmentation. To analyses the SEEFC performance in more detail on other images, Berkley image segmentation database was selected [52]. These images are standard images which have been used for different purpose and it provides the ground truth which can be used for evaluation purpose.
2.6 Segmentation of images from Berkeley Database Using SEEFC The SEEFC algorithm has been evaluated on single object Berkeley database. Throughout the assessment stage, Segmentation results have been compared against ground truths provided by Berkeley database.Fig.11 illustrates the ground truths and segmented images. The segmentation results have been analyzed considering two criteria: Object extraction and background detection. Two segmentation methods one based on Self Organizing Map (SOM) alone and the other combination of self-organizing map and Extended of fuzzy clustering method (SEEFC), have been implemented. A number of images have been used for testing the performance of the two and the results have been evaluated. For this purpose ten out of those tested images are chosen to illustrate the performance of proposed algorithms. These images are named as Plane (#3096), Bird (#42049), Elephant (#296059), Shipman (#62096), Fox (#167062), Garden (#86016), Eagle (#135069), moon (#238011), church (#118035) and paratrooper (#60079). The results obtained using SOM and SEEFC are shown in Fig.10. The significant changes and information is highlighted with arrows in segmented images. First we consider the plane image (#3096). The tip of plane in SOM output is mixed with background information while in SEEFC output it is completely refined and segmented out properly. The same scenario is repeated for the background of the plane image. As it is highlighted by arrows, the background of SEEFC algorithm is distributed between two clusters but in SOM the combination of all clusters can be seen in the resultant image. Further, consider the segmentation of letter ‘A’ on the vertical stabilizer section of plane, letter A is detected with SEEFC algorithm much better than SOM algorithm. Let’s consider the bird image (#42094).The result clearly shows that SOM fails to detect the entire background as it is expected via ground truth while SEEFC algorithm has better capability to detect background. Considering the bird object in the segmentation result one can say that SOM had better performance in detecting the branches compared to SEEFC algorithm. The detail information of Bird body is segmented more clearly particularly the feet area and the branches in the lower right of the image are detected and segmented properly with SEEFC. In the case of the elephant image (#296059) by SEEFC, both the elephant are segmented as a similar object while SOM segments the elephant body into two clusters. The problem of both methods is the outline between two elephants is not separated properly. The ship man image (#62096) includes three segments as shown in the ground truth. Object
including the boat and man, background including river and mountain area. The SOM algorithm segmented the river and the mountain area almost similar to ground truth. The object is detected but it is combination of 2 clusters. The shipman and the sail of boat are segmented as different objects while it is one object as per ground truth. Looking at results of SEEFC algorithm, it is evident that the river and its outline are segmented properly but the mountain area and object are segmented as one cluster. Thus, the shipman and the sail are fully segmented as one object but at the same time it is mixed with mountain area which does not match with ground truth. In the Fox image, SEEFC algorithm is successful in detecting the Fox better in comparison with SOM algorithm. The SEEFC segmented the background better near ground truth as compared to SOM. For the Garden image in both the cases, the center garden is fully detected as a separate object according to ground truth. But background, is segmented better by SEEFC method. The image (#135069) is the Eagle image. Both the methods detect the object quite well but SEEFC method segments the eagle and hunted bird by eagle in a proper manner, as has been shown in ground truth. SEEFC was able to easily distinguish between the object itself in the image by showing that in another cluster. In case of the moon image (#238011) SEEFC method detected all objects and background as per ground truth. The moon in this image is a small object, but it is easily segmented by SEEFC method while SOM did not detect that properly. The trees are detected by both methods properly. Background of the image, detected by SEEFC is better compared to the other. The image (#135069) is the Eagle image. Both the methods detect the object quite well but SEEFC method segments the eagle and hunted bird by eagle in a proper manner, as has been shown in ground truth. It was able to easily distinguish between the object itself in the image by showing that in another cluster. In case of the moon image (#238011) SEEFC method detected all objects and background as per ground truth. The moon in this image is a small object, but it is easily segmented by SEEFC method while SOM did not detect that properly. Background of the image detected by SEEFC is better while SOM does not detect that properly. The trees are detected by both of methods. Now we consider the church image (#118035). SEEFC detected 4 clusters for this image. The dome area of the church image and its arch has been properly detected as a separate cluster. The significant difference is highlighted by arrows in segmented images. The church building details and crosses are detected in both the segmented images properly but the SEEFC output is comparatively refined. Let's consider the paratrooper (#60079) image. The paratrooper is the main object. The paratrooper man has been detected as one object in both of the methods. The parachute part has been detected by two different clusters in both methods. The mountain is a combination of 4 different clusters in SOM method while it is combination of 3 clusters in SEEFC method. It seems to be more near to reality compared to the SOM result. From background point of view the SEEFC segmentation is better than SOM. The white shade clusters has been expanded and occupies significant part of the background which is quite far from the ground truth. The same scenario has been repeated for SEEFC method but it is less as compared to SOM. To check the SEEFC segmentation performance quantitavely, the sensitivity and accuracy of the object for images calculated as shown in Table2. The sensitivity value shows the ability of the SEEFC algorithm for object detection. The average accuracy of SEEFC algorithm for object detection is 88.1%. The other parameter which was calculated for SEECF quantitative evaluation is Q index as discussed in Section3.4.The lower is the Q value, the better is the segmentation. In fact the range of Q index varied from “0” to “1”. Q value “0” indicates ideal segmentation. As shown in Table3 it can be seen that the SEEFC algorithm has successfully produced very small Q(I) values for almost all images. The value indicates the ability of SEEFC in correct decision making close to the reality. 2.7 Comparison of SEEFC with other segmentation algorithm These experiments were carried out to study and understand the performance of developed SEEFC algorithm compared with two popular segmentation algorithm namely , Normalize Cut (NC) and Contour Detection (CD)[53]. For evaluation of algorithms the images were selected based on Distinctiveness factor as per Eq.(4.1) . In fact the difficulty of segmentation is measured by “Distinctiveness” of the salient object versus its background.
4.1 Where x2 is difference between the brightness distribution of object in the image and its background. Pf and Pb are the figure and background distribution. So lesser distinctiveness of an image, means more difficulty in image segmentation. In fact distinguish between object and background will be more difficult for segmentation algorithm. Generally Distinctiveness falls into range of [0 to 1] and on this basis the test images are categorized into five different groups. However to assess and compare the result of image segmentation techniques proposed in this work the evaluation is carried out based on qualitative and quantitative analyses. Qualitative analysis can address visual interpretation of the result as observed via human visual perception. Though the qualitative analysis can determine the ability of segmentation algorithm to distinguish between object of interest and background quantitative analysis enables the performance of segmentation results to be evaluated statistically. 2.7.1 Qualitative Analysis This section deals with visual evaluation of the performance of segmentation of each individual algorithm based on their capability to detect the important regions in the image. Four different ranges of distinctiveness from (0-0.2),(0.2-0.4),(0.4-0.6) and (0.6-1) have been used and the segmentation results of SEEFC along with reported data are shown. Segmentation of images with distinctiveness<=0.2 Fig.12shows the two Berkley database images #21077 and #227092 which are Car and Pot with Distinctiveness factor of 0.1 and 0.2 respectively. The ground truth related to each of these images is shown. SEEFC selected 4 clusters and 3 clusters for Car and Pot image respectively. The objects in the SEEFC, CD & NC is shown with the red arrow. Looking at SEEFC result it can be observed that the car object does not have over segmentation with the background which is the problem seen in other method. In the case of Pot image the boundary of the segmented object is isolated from background. The performance of other technique can be seen in the Fig.12. Segmentation of images with 0.2
Second example shows a Surfing image with its related Ground Truth #1109.Looking at SEEFC performance it can be easily observed that the object is detected very nicely while CD and NC failed in segmentation of the object from background. Observing the SEEFC performance on surfing image shows how 3 clusters are assigned to the image. For example the gray color in the segmented image is around the object. Comparing this shade of color in original image one can judge how perfectly it is segmented. Segmentation of images with distinctiveness>0.6 The final experiments evaluated the result of developed algorithm for the group of images with distinctiveness more than 0.6.In this specific groups the clearance of objects from background make the segmentation task easier. From Berkley database the images #3096 and #42049 was selected in which their distinctiveness factors are 0.76 and 0.8 correspondingly. The Ground Truth of images #3096 or plane image shows the plane itself as an object and rest of the image as a background. SEEFC optimizes the number of clusters as 3 and segmented the image accordingly. Contour detection and normalized cut also detected the object properly. All above description could be extended to the image #42049 which is called as bird image. The bird is an object of interest. The SEEFC performs the task well by minimizing the number of clusters to 3. But visual inspection of the result shows the ability of NC method in object isolation compare to other technique is better in this case. In conclusion it can be seen that the result of developed Neuro-fuzzy technique is comparable with well-known techniques of image segmentation. Qualitative comparison of the result show SEEFC has satisfactory performance. Though qualitative analysis shows the ability of the algorithm but next section deals with quantitative analysis of the SEEFC performance. 2.7.2 Quantitative Analysis based on Distinctiveness Factor The proposed algorithm has been tested on Berkley image segmentation database as per the Distinctiveness factor. The main purpose of these experiments was to study and understand the performance of SEEFC algorithm in comparison with Normalize Cut (NC), Contour Detection (CD)[53],SOM and FCM in quantitative manner. The accuracy value of each method can be observed in Table 1: It can be observed that the accuracy of SEEFC segmentation algorithm is better than other techniques especially in case of images with lower distinctiveness factor. That indicates the ability of SEEFC in detection of salient object when there is a smooth edge between object and background. The analysis of the results also indicates the ability of SEEFC as a Neuro-Fuzzy approach to segmentation problem compared to each of FCM and SOM segmentation performance individually. The Fig.16 also represents the plot of SEEFC accuracy in comparison with other approaches. The plot shows that NC technique has the worst performance to detect the salient object. But it is observed that FCM, SOM and NC techniques have acceptable accuracy rate in salient object segmentation in middle range of Distinctiveness. 2.8 Reproducibility of SEEFC Algorithm SEEFC shows better accuracy for lower distinctiveness. This could be attributed to complimentary functioning of SOM-EFCM. However as SOM and EFCM are initialized randomly, however it causes some differences in segmented result .Hence reproducibility performance of SEEFC was studied. The reproducibility of SEEFC was tested in terms of sensitivity and accuracy and the results are shown in Table2. These variations are illustrated in Fig.17 and Fig.18.In order to find the reproducibility, the algorithm executed 10 times and the sensitivity and accuracy parameters are calculated for each run. The analysis of the reproducibility of the algorithm indicates that there is not much variation in sensitivity which addresses the salient object detection in the image except in few cases. However the variation in terms of accuracy shows that SEEFC segmentation results variations is still lesser as compare to sensitivity variations.
2.9 Effect of noise on SEEFC To assess the effect of noise on the proposed algorithm we corrupted the images by Gaussian noise as well as Salt& Pepper noise. The result shows that the algorithm performs very well up to 15 % of variance value of Gaussian noise and 5% of Salt &Pepper noise.Fig.19 illustrates the segmentation result of Bird image (#42049) in presence of noise as a sample.Table5 shows the statistics parameter of stability evaluation in the presence of noises. Increasing the Gaussian noise causes decreasing the sensitivity and accuracy to 81% and 73.5% respectively. The experiments were repeated using runs on the SEEFC algorithm 10 times and the average performance in presence of noises, are as per Table5. However Gaussian noise more than 15%, causes a decrease in accuracy and object sensitivity. The above scenario was repeated for Salt & Pepper noise. The image corruption by this type of noise can lead increases to 5% and result shows that the SEEFC algorithm has acceptable performance on segmentation of such an images. Looking at Table5, it shows that the average accuracy does not decrease below 87%.as it can be observed. In Fig.19, the 5% of Salt & Pepper noise almost change the view of the original images. The SEEFC algorithm can tolerate this situation with minimum effect of these phenomena on the segmented output. However increasing the noise more than 5% causes the algorithm to fail in segmentation process. 3. Application As it is discussed, the developed Neuro- Fuzzy algorithm (SEEFC) technique is completely unsupervised and it is able to find the number of cluster automatically. It’s performance shows good accuracy in presence of abnormal condition like noise as tested. These ability could be utilized in such conditions when there is a change in image backgrounds or unwanted phenomena to make the algorithm robust for segmentation. One of this criteria generally occurs in segmentation of hand for Indoor Robot Navigation (IRN) using gesture recognition [54]. In fact vision based indoor robot navigation imposes some requirements on the system to make it applicable in real life. Some of them are lighting condition or illumination variation, type of backgrounds (simple, complex and variant), skin colors, size of different people’s hand and complex nature of the hand. This makes segmentation and extraction of the hand a difficult task. Capability of SEEFC to adapt to these variations for hand segmentation was studied and tested on hand database images available in the literature [53] and Cambridge dataset[55].As an example The said database includes different images under different background and different illumination conditions. The SEEFC segmentation of images in this database shows comparable performance for hand under different condition. Thus the future work is going to focus on such an application.Fig.20 shows the result of SEEFC algorithm on some sample images under 3 different backgrounds under different illumination condition. Detection of 3 & 4 clusters using SEEFC algorithm result in automatic segmentation, with the hand segmented as depicted in following figure. 4. Conclusion A novel methodology based on the self-organizing map and extended fuzzy clustering has been introduced (SEEFC). SOM is used for coarse clustering of the feature vectors. The basics advantage of the method is the use of EFCM to estimate the number of clusters in the image automatically. Thus the method takes advantage of the self-organizing capability of the SOM as well as automatic estimation of clusters using EFCM. The performance of SEEFC has been evaluated with a number of images from the Berkley database. The results demonstrate the ability of SEEFC for image segmentation. The overall performance of SEEFC, to detect the objects in the image is promising. Qualitative as well as quantitative analysis based on the ground truths indicates the usefulness of SEEFC for Image segmentation. Evaluation of segmented image with standard ground truth images reveals that the SEEFC method’s sensitivity, specificity and accuracy values are better than SOM in the case of object extraction. Even in the case of
background detection, the performance of proposed method is better in terms of sensitivity and accuracy but the specificity of SOM is better when the background is in priority. Qualitative evaluation of the SEEFC algorithm shows an overall improvement of the segmented output. In the quantitative evaluation, sensitivity and accuracy of SEEFC algorithm have been calculated and it shows 84.56% average sensitivity to isolate an object from background rate and an average of 88% success in terms of accuracy. The reproducibility of the algorithm was tested by running the algorithm in a number of times and it shows the good capability of reproducing the result. Finally, the developed algorithm was tested against the different type of noises and it shows good robustness to reproduce the result and can tolerate 15% of Gaussian noise and 5% of salt and pepper noise as well. In contrast, the proposed Neuro-Fuzzy method acts very well for detecting the object from background. Also, SEEFC preserves minor details in the image as can be seen from the moon and Eagle image. Comparison with standard methods shows improved performance of SEEFC. The experiments indicate that the proposed method has the potential to improve segmentation.
References: [1] W. K. Pratt, Digital image processing: PIKS inside. Wiley, 2002. [2] X. Zhao, M. Turk, W. Li, K. Lien, and G. Wang, “A multilevel image thresholding segmentation algorithm based on two-dimensional K–L divergence and modified particle swarm optimization,” Appl. Soft Comput., vol. 48, pp. 151–159, Nov. 2016. [3] R. C. Gonzalez and R. E. Woods, Digital Image Processing, vol. 49, no. 3. Prentice Hall, 2008. [4] J. Shi and J. Malik, “Normalized cuts and image segmentation,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 22, no. 8, pp. 888–905, Aug. 2000. [5] P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient graph-based image segmentation,” Int. J. Comput. Vis., vol. 59, no. 2, pp. 167–181, 2004. [6] D. Comaniciu and P. Meer, “Mean shift: A robust approach toward feature space analysis,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 24, no. 5, pp. 603–619, 2002. [7] J. A. Sethian, “Level set methods and fast marching methods,” J. Comput. Inf. Technol., vol. 11, no. 1, pp. 1–2, 2003. [8] S. Wang, T. Kubota, J. M. Siskind, and J. Wang, “Salient closed boundary extraction with ratio contour,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 27, no. 4, pp. 546–561, 2005. [9] W. Cai, S. Chen, and D. Zhang, “Fast and robust fuzzy< i> c-means clustering algorithms incorporating local information for image segmentation,” Pattern Recognit., vol. 40, no. 3, pp. 825–838, 2007. [10] X. Y. Wang and J. Bu, “A fast and robust image segmentation using FCM with spatial information,” Digit. Signal Process., vol. 20, no. 4, pp. 1173–1182, 2010. [11] J. K. Udupa and S. Samarasekera, “Fuzzy connectedness and object definition: theory, algorithms, and applications in image segmentation,” Graph. Model. Image Process., vol. 58, no. 3, pp. 246–261, 1996. [12] J. P. Sarkar, I. Saha, and U. Maulik, “Rough Possibilistic Type-2 Fuzzy C-Means clustering for MR brain image segmentation,” Appl. Soft Comput., vol. 46, pp. 527–536, Sep. 2016. [13] M. Egmont-Petersen, D. de Ridder, and H. Handels, “Image processing with neural networks-a review,” Pattern Recognit., vol. 35, no. 10, pp. 2279–2301, 2002. [14] S. Bhattacharyya, P. Pal, and S. Bhowmick, “Binary image denoising using a quantum multilayer self organizing neural network,” Appl. Soft Comput., vol. 24, pp. 717–729, Nov. 2014. [15] W. B. Tao, J. W. Tian, and J. Liu, “Image segmentation by three-level thresholding based on maximum fuzzy entropy and genetic algorithm,” Pattern Recognit. Lett., vol. 24, no. 16, pp. 3069–3078, 2003. [16] K. Hammouche, M. Diaf, and P. Siarry, “A multilevel automatic thresholding method based on a genetic algorithm for a fast image segmentation,” Comput. Vis. Image Underst., vol. 109, no. 2, pp. 163–175, 2008. [17] B. Bhanu, S. Lee, and J. Ming, “Adaptive image segmentation using a genetic algorithm,” Syst. Man Cybern. IEEE Trans., vol. 25, no. 12, pp. 1543–1567, 1995. [18] L. A. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, no. 3, pp. 338–353, 1965. [19] G. C. Karmakar, L. Dooley, and S. M. Rahman, “Review on fuzzy image segmentation techniques,” Des. Manag. Multimed. Inf. Syst. Oppor. challenges, pp. 282–313, 2001. [20] J. C. Bezdek, R. Krisnapuram, and N. R. Pal, Fuzzy models and algorithms for pattern recognition and image processing, vol. 4. Springer-Verlag New York Inc, 2005. [21] J. V De Oliveira and W. Pedrycz, Advances in fuzzy clustering and its applications. Wiley Online Library, 2007. [22] M. Hasanzadeh and S. Kasaei, “Fuzzy image segmentation using membership connectedness,” EURASIP J. Adv. Signal Process., vol. 2008, no. 1, p. 417293, 2008. [23] U. Kaymak and M. Setnes, “Fuzzy clustering with volume prototypes and adaptive cluster merging,” Fuzzy Syst. IEEE Trans., vol. 10, no. 6, pp. 705–712, Dec. 2002. [24] M. T. Hagan, H. B. Demuth, M. H. Beale, and B. University of Colorado, Neural network design. PWS Pub, 1996. [25] R. Lippmann, “An introduction to computing with neural nets,” ASSP Mag. IEEE, vol. 4, no.
[26] [27] [28]
[29] [30]
[31]
[32] [33]
[34] [35] [36] [37] [38] [39]
[40] [41] [42]
[43] [44]
[45] [46]
[47]
[48]
2, pp. 4–22, 1987. M. Acharyya and M. K. Kundu, “Image segmentation using wavelet packet frames and neurofuzzy tools,” Int. J. Comput. Cogn., vol. 5, no. 4, pp. 27–43, 2007. T. Kohonen, “The self-organizing map,” Proc. IEEE, vol. 78, no. 9, pp. 1464–1480, Sep. 1990. V. V Srinivas, S. Tripathi, A. R. Rao, and R. S. Govindaraju, “Regional flood frequency analysis by combining self-organizing feature map and fuzzy clustering,” J. Hydrol., vol. 348, no. 1, pp. 148–166, 2008. C. Dongxiang, “Self-Organizing Map-Based Color Image Segmentation with k-Means Clustering and Saliency Map,” ISRN Signal Process., vol. 2011, 2011. M. Sezgin, O. K. Ersoy, and B. l Yazgan, “Segmentation of remote sensing images using multistage unsupervised learning,” in Optical Science and Technology, the SPIE 49th Annual Meeting, 2004, pp. 616–623. M. L. Goncalves, M. L. A. Netto, J. A. F. Costa, and J. Z. Junior, “An unsupervised method of classifying remotely sensed images using Kohonen self‐organizing maps and agglomerative hierarchical clustering methods,” Int. J. Remote Sens., vol. 29, no. 11, pp. 3171–3207, 2008. G. W. Jiji and L. Ganesan, “A new approach for unsupervised segmentation,” Appl. Soft Comput., vol. 10, no. 3, pp. 689–693, 2010. D. M. Ristic, M. Pavlovic, and I. Reljin, “Image segmentation method based on selforganizing maps and K-means algorithm,” in Neural Network Applications in Electrical Engineering, 2008. NEUREL 2008. 9th Symposium on, 2008, pp. 27–30. A. R. F. Araújo and D. C. Costa, “Local adaptive receptive field self-organizing map for image color segmentation,” Image Vis. Comput., vol. 27, no. 9, pp. 1229–1239, 2009. J. Vesanto and E. Alhoniemi, “Clustering of the self-organizing map,” Neural Networks, IEEE Trans., vol. 11, no. 3, pp. 586–600, 2000. K. Tasdemir and E. Merényi, “Exploiting data topology in visualization and clustering of selforganizing maps,” Neural Networks, IEEE Trans., vol. 20, no. 4, pp. 549–562, 2009. D. Brugger, M. Bogdan, and W. Rosenstiel, “Automatic cluster detection in Kohonen’s SOM,” Neural Networks, IEEE Trans., vol. 19, no. 3, pp. 442–459, 2008. A. K. Helmy and G. S. El-Taweel, “Image segmentation scheme based on SOM–PCNN in frequency domain,” Appl. Soft Comput., vol. 40, pp. 405–415, Mar. 2016. M. I. Chacon-Murguia and G. Ramirez-Alonso, “Fuzzy-neural self-adapting background modeling with automatic motion analysis for dynamic object detection,” Appl. Soft Comput., vol. 36, pp. 570–577, Nov. 2015. K. Deshmukh and G. N. Shinde, “An adaptive neuro-fuzzy system for color image segmentation,” J. Indian Inst. Sci, vol. 86, pp. 493–506, 2006. E. Aghajari, “Incorporating FCM and Back Propagation Neural Network for Image Segmentation,” Int. J. Comput. Commun. Technol., vol. 2, no. Viii, pp. 121–126, 2011. L. C. M. Andrade, M. Oleskovicz, and R. A. S. Fernandes, “Adaptive threshold based on wavelet transform applied to the segmentation of single and combined power quality disturbances,” Appl. Soft Comput., vol. 38, pp. 967–977, Jan. 2016. T. Kohonen, “The self-organizing map,” Proc. IEEE, vol. 78, no. 9, pp. 1464–1480, 1990. E. Aghajari, H. Lotfi, and D. Gharpure, “Towards interpretation of self organizing map for image segmentation,” in Informatics, Electronics & Vision (ICIEV), 2013 International Conference on, 2013, pp. 1–6. U. Kaymak and M. Setnes, “Fuzzy clustering with volume prototypes and adaptive cluster merging,” IEEE Trans. Fuzzy Syst., vol. 10, no. 6, pp. 705–712, Dec. 2002. F. Di Martino_affanb, V. Loia, and S. Sessa, “Extended fuzzy C-means clustering algorithm for hotspot events in spatial analysis,” Int. J. Hybrid Intell. Syst., vol. 5, no. 1, pp. 31–44, Jan. 2008. R. Unnikrishnan, C. Pantofaru, and M. Hebert, “Toward objective evaluation of image segmentation algorithms.,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 929–44, Jun. 2007. Y. J. Zhang, “A survey on evaluation methods for image segmentation,” Pattern Recognit., vol. 29, no. 8, pp. 1335–1346, 1996.
[49] [50]
[51]
[52]
[53] [54] [55]
[56]
H. Zhang, J. E. Fritts, and S. A. Goldman, “Image segmentation evaluation: A survey of unsupervised methods,” Comput. Vis. image Underst., vol. 110, no. 2, pp. 260–280, 2008. Z. T. Birch, “an efficient data clustering method for very large databases/T. Zhang, R. Ramakrishnan, M. Livny,” in Proc. of the ACM SIGMOD Conf. on Management of Data.– Montreal: ACM Press, 1996, pp. 103–114. H. H. Muhammed, “Unsupervised Image Segmentation Using New Neuro-Fuzzy Systems,” in Swedish Society for Automated Image Analysis Symposium-Proceedings of SSAB 2002, 2002, pp. 83–87. D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” in Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, 2001, vol. 2, pp. 416–423. H. Wang and J. Oliensis, “Generalizing edge detection to contour detection for image segmentation,” Comput. Vis. Image Underst., vol. 114, no. 7, pp. 731–744, 2010. S. S. Rautaray and A. Agrawal, “Vision based hand gesture recognition for human computer interaction: a survey,” Artif. Intell. Rev., vol. 43, no. 1, pp. 1–54, 2015. T.-K. Kim, K.-Y. K. Wong, and R. Cipolla, “Tensor canonical correlation analysis for action classification,” in Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, 2007, pp. 1–8. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 33, no. 5, pp. 898–916, 2011.
Original Image Features selection & Feature Extraction
Prior Segmentation Using SOM
EFCM
Labeling using EFCM Output
Final Segmentation Segmented Image
Fig.1.The proposed SEEFC Algorithm
Fig.2: Random data selection a) four quadrant division of the image for equal distribution, b) Random distribution, c) Random data map on image, (Berkley #3096) [56]
W1M
W11 W12
(BMU) V1 V2 V3 V4 V5 V6 V7 V8 V9
WL1
Hexagonal Grid
WLM
Fig.2.The training pattern vector (Vp) consist of Discrete wavelet filtered images, Pixel Values, Gradient, Standard deviation, Variance &Energy, The BMU (wm) of training pattern vector (Vp) is shown in colorful shade on self-organizing map of hexagonal grid. The first and second neighborhood unit of BMU has been shown in red and green shades, respectively. The weights of the individual units (w11, w12,w1M, wL1, wLM) are also be seen on SOM map.
2nd Max Selected Neuron
START
W11
Feature Extraction
Random Selection of pixels for Training
Creating SOM and Network Training
Hits map of Best Match Units (BMU)
3rd Max Selected Neuron
Find Max Neuron Hits & Save Its Weights
W1515 Find Next Max Neuron Hits
Yes
1st Max Selected Neuron
(b)
If Max Neuron Distance
(c)
No
No
If Selected neurons = Cluster No.
Yes
Find Euclidean Distance of Each Prototype Vectors
Prototype Vectors END to Their Assignment Corresponding Cluster
Segmented Image
END
(a) Fig. 3. a) Overview of SOM segmentation, b)Illustration of best matching units hits of training pattern vectors and selected number of neurons as a reference for each cluster in order to interpret SOM network result.(threshold value =13) ,c) segmentation Image for airplane image from Berkeley database.
Fig.4. Original Image (#3096) from Berkley Database, b) SOM Trained Map
Fig.5. SOM and EFCM Integration in a Glance View
Fig.6. Sample Image and segmented images for image #3096
Fig.7.Illustration of TP-TN-FP & FN
Fig.8.a,b) Original Image, c,d) SEEFC Segmentation Result
Fig.9.State of the Art Images a,b)Original Image, c,d)SEEFC Segmentation Result
Fig.10.Result of State of the art techniques on pepper image
Fig.11. Image samples and Ground truths and segmented images results.
Fig.12.Comparison Of SEEFC Algorithm Performance
With Contour Detection and Normalize Cut Techniques For Images With Distinctiveness<=0.2
Fig.13.Comparison Of SEEFC Algorithm Performance
With Contour Detection and Normalize Cut Techniques For Images With 0.2
Fig.14.Comparison Of SEEFC Algorithm Performance With Contour Detection and Normalize Cut Techniques For Images With 0.4
Fig.15.Comparison of SEEFC Algorithm Performance with Contour Detection and Normalize Cut Techniques for Images with Distinctiveness>0.6
100 CD SEEFC
Accuracy %
80
FCM
60
SOM
40
NC
SEEFC NC CD SOM FCM
20
0
0
0.2
0.4
0.6
0.8
Distinctiveness Factor Fig.16: Accuracy Plot of SEEFC Comparison with Other Algorithm
100
Sensitivity Variation
90 80 70 60 50 40 30 20 10 0
Elephant
Plane Bird
Eagle
Fox
Shipman
Garden
Church
Moon
Paratrooper
Images
Fig.17:The Performance Variation of SEEFC Algorithm for Object Sensitivity
100 90
Accuracy Variation
80 70 60 50 40 30 20 10 0
Plane
Elephant Bird
Shipman
Fox
Eagle
Garden
Church
Moon
Paratrooper
Images
Fig.18:The Performance Variation of SEEFC Algorithm for Object Accuracy
Fig.19: Stability of SEEFC on presence of Gaussian Noise and Salt &Pepper Noise
Fig.20:SEEFC Algorithm Result for Hand Databases available in the literature
Table 1: Accuracy of Different Algorithm Performance Based on Distinctiveness Factor
DISTICTIVNESS
IMAGES WITH DIFFERENT DISTINCTIVENESS FACTOR 0-0.2 0.2-0.4 0.4-0.6 >0.6
Car Pot Statue Boatman Man Surfing Plane Bird
SEEFC 73.52* 87.9* 97.8* 83.4 87.9* 65* 80.08 88.79*
SEGMENTATION ALGORITHM NC CD SOM FCM 67 46 73.31 68.52 61 62 45.91 51.47 45 68 71.5 92.26 20 87* 69 83 71 73 81.7 82.42 15 40 43.2 45.56 84 85* 73.2 81 86 79 88.76 87.9
Table 2: Average Sensitivity and Accuracy of SEEFC algorithm (%)
Image Plane Bird Elephant Shipman Fox Garden Eagle Moon Church Paratrooper %Sensitivity
77.77 83.04
90.17
94.87 81.12
%Accuracy 80.08 88.79
78.63
65 98.48
98.64
Average
88.31
80.67
74.43
76.59
84.56
97.97 95.13
95.7
83.3
91.76
88.1
Table 3: Q(I) value for SEEFC performance
Images
Q(I)
Plane Bird Elephant Shipman Fox Eagle Moon Garden Church Paratrooper
SEEFC 0.024±0.25% 0.0797±0.11% 0.0546±0.38% 0.1348±0.21% 0.0140±0.42% 0.0295±0.06% 0.0042±0% 0.0477±0.37% 0.0422±0.2% 0.2881±1.72%
Table 4: The Average performance of SEEFC Algorithm
Images Plane Bird Elephant Shipman Fox Eagle Moon Garden Church Paratrooper
∆% (average performance of SEEFC ) Sensitivity Accuracy Object Object 77.77 ± 9.35 90.72 ±7.14 83.04 ± 6.66 91.8 ±0.85 90.17 ± 0.02 78.63 ± 0.04 94.87 ± 3.14 65 ± 2.74 81.12 ± 7.98 98.48±0.18 98.64± 0.53 97.97 ±0.57 88.31 ± 0.045 95.13 ±0.008 80.67 ± 0.002 95.7 ±0 74.43 ± 3.69 83.3 ±0.59 76.59 ± 2.33 91.76 ±5.83
Table5: Evaluation Parameters of Stability of SEEFC on presence of Noises
Images
Evaluation Parameter
Object
Sensitivity Accuracy
5% 90.5±3.66 94±1.5
10% 89±4.1 85±2
Gaussian 15% 81±2.54 73.5±5.25
Type of Noise Salt &Pepper 2% 5% 87±4.6 95.8±2.36 88.9±0.2 91.6±3.6