Cluster analysis using optimization algorithms with newly designed objective functions

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 Expert Systems with Applications xxx (2015) xxx–xxx 1

Contents lists available at ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa 5 6 3 4

Cluster analysis using optimization algorithms with newly designed objective functions

7

D. Binu

8

Aloy Labs, Bengaluru, India

10 9 11 1 4 3 2 14 15 16 17 18 19 20 21 22 23

a r t i c l e

i n f o

Article history: Available online xxxx Keywords: Clustering Optimization Genetic algorithm (GA) Cuckoo search (CS) Particle swarm optimization (PSO) Kernel space

a b s t r a c t Clustering finds various applications in the field of medical and telecommunication for unsupervised learning which is much required in expert system and its application. Various algorithms have been developed to clustering for the past fifty years after the introduction of k-means clustering. Recently, optimization algorithms are applied for clustering to find optimal clusters with the help of different objective functions. Accordingly, in this research, clustering is performed using three newly designed objective functions along with four existing objective functions with the help of optimization algorithms like, genetic algorithm, cuckoo search and particle swarm optimization algorithm. Here, three different objective functions are designed including the cumulative summation of fuzzy membership and distance value with normal data space, kernel space as well as multiple kernel space. In addition to the existing seven objective functions, totally, 21 different clustering algorithms are discussed and the performance is validated with 16 different datasets which are synthetic, small and large scale real data. The comparison is made with five different evaluation metrics to validate the effectiveness and efficiency. From the research outcome, the suggestion is presented to select a suitable algorithm among 21 algorithms for a particular data and results proved that the effectiveness of cluster analysis is mainly dependent on objective function and the efficiency of cluster analysis is based on search algorithm. Ó 2015 Published by Elsevier Ltd.

25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

43 44

1. Introduction

45

Expert systems are intelligent software programs designed for taking useful and intelligent managerial decisions for various domains like, agriculture, finance, education, medicine to military science, process control, space technology and engineering. The expert systems require different data mining methods to support decision making process. Among different data mining methods, classification and clustering are two important methods applied widely for expert system. Clustering which is unsupervised learning has received significant attention among the researchers due to its wide applicability for the past fifty years after the introduction of k-means clustering algorithm (McQueen, 1967), which is wellknown algorithm for clustering due to its simplicity. Due to the reception of k-mean clustering, variants of k-means clustering algorithms are introduced by different researchers by pointing out various problems like, initialization (Khan & Ahmad, 2004), k-value (Pham, Dimov, & Nguyen, 2004), and distance computation. One of the most accepted methods of clustering after the introduction of k-means clustering is fuzzy c-means clustering (FCM) (Bezdek, 1981), which is a popular algorithm after the

46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63

E-mail address: [email protected]

k-means clustering, including fuzzy concept in computing the cluster centroids. FCM- algorithm is also found at various variants among the researchers (Ji, Pang, Zhou, Han, & Wang, 2012; Ji et al., 2012; Kannana, Ramathilagam, & Chung, 2012; Linda & Manic, 2012; Maji, 2011). The important variants of FCM algorithm is kernel fuzzy clustering (Zhang & Chen, 2004) and multiple kernel-based clustering algorithm (Chen, Chen, & Lu, 2011) which are based on the concept of FCM algorithm with the inclusion of kernels, accepted widely for its capability of doing the task for non-linear data. More interestingly, these entire algorithms have found importance in image segmentation (Chen et al., 2011; Zhang & Chen, 2004) and the relevant applications related to image segmentation (Ji, Pang, et al., 2012; Ji et al., 2012; Li & Qi, 2007; Sulaiman & Isa, 2010; Szilágyi, Szilágyi, Benyób, & Benyó, 2011; Zhao, Jiao, & Liu, 2013). After the introduction of soft computing techniques, the clustering problem is transformed to optimization problem, finding the optimal clusters in the defined search space. Accordingly most of the optimization algorithms are applied to clustering problems. For example, the first and pioneer optimization algorithm called, GA (Mualik & Bandyopadhyay, 2002) is applied for clustering initially and then, PSO algorithm (Premalatha & Natarajan, 2008), Artificial Bee Colony (Zhang, Ouyang, & Ning, 2010), Bacterial

http://dx.doi.org/10.1016/j.eswa.2015.03.031 0957-4174/Ó 2015 Published by Elsevier Ltd.

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 2 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

Foraging Optimization (Wan, Li, Xiao, Wang, & Yang, 2012), Simulated Annealing (Selim & Alsultan, 1991), Differential Evolution Algorithm (Das, Abraham, & Konar, 2008), and Evolutionary algorithm (Castellanos-Garzón & Diaz, 2013) and Firefly (Senthilnath, Omkar, & Mani, 2011) are subsequently _ applied for clustering. Recently, Inkaya, Kayalıgil, and Özdemirel, 2015 utilized Ant Colony Optimization for clustering methodology using two objective functions, namely adjusted compactness and relative separation. Liyong, Witold, Wei, Xiaodong, and Li (2014) utilized genetically guided alternating optimization for fuzzy cmeans clustering. Here, interval number was introduced for attribute weighting in the weighted fuzzy c-means (WFCM) clustering to obtain appropriate weights more easily from the viewpoint of geometric probability. Hoang, Yadav, Kumar, and Panda (2014) have utilized the recent optimization algorithm called, Harmony Search Algorithm for clustering. Yuwono, Su, Moulton, and Nguyen (2014) have developed Rapid Centroid Estimation utilizing the rules of PSO algorithm to reduce the computational complexity and produced the clusters with higher purity. These recent algorithms utilized the traditional objective function for evaluating the clustering solution. After that, hybrid algorithms are in the field of doing clustering process over the datasets to utilize the advantages of both the algorithms taken for hybridization. Here, two optimization algorithms are combined to do the clustering task as like, GA with PSO (Kuo, Syu, Chen, & Tien, 2012). From this, we can say that if any new optimization algorithms are being done, researchers are waiting to utilize the updating algorithm for clustering process. Due to the successful application of hybrid algorithms in clustering process, researchers are then hybridized the traditional clustering algorithms with the optimization algorithm. For example, GA is combined with k-means clustering, called genetic-k-means (Krishna & Murty, 1999) and the similar type of work is given in Niknam and Amiri (2010). Recently, Krishnasamy, Kulkarni, and Paramesran (2014) have proposed hybrid evolutionary data clustering algorithm referred to as K-MCI, whereby, K-means with modified cohort intelligence are combined for data clustering. Wei, Yingying, Soon Cheol, and Xuezhong (2015) have developed hybrid evolutionary computation approach utilizing Quantum-behaved particle swarm optimization for data clustering. Garcia-Piquer, Fornells, Bacardit, Orriols-Puig, and Golobardes (2014) have developed Multiobjective Clustering to guide the search following a cycle based on evolutionary algorithms. Tengke, Shengrui, Qingshan, and Huang (2014) have proposed a cascade optimization framework that combines the weighted conditional probability distribution (WCPD) and WFI models for data clustering. In optimization-based clustering applications, clustering practices are operated, based on the fitness function, which validates the optimal cluster achieved. Here, the constraint is that fitness function developed should be capable of providing the good clusters’ quality. The objective function is also responsible for the validation of the clustering output and directing it through the optimal cluster centroids. However, when looking into clustering’ fitness functions, most of the optimization-based algorithm utilized the k-mean objective (minimum mean square distance) as fitness function for optimal searching of cluster task (Wan et al., 2012) because of its simplistic computation. Similarly, FCM objective is also applied as fitness function for finding the optimal cluster centroids (Ouadfel & Meshoul, 2012) due to its flexibility and its effectiveness. Also, authors utilize some cluster validity indices to apply on swarm intelligence-based optimization algorithm (Xu, Xu, & Wunsch, 2012) with the different perspective of cluster quality. In addition to, fuzzy cluster validity indices are developed with the inclusion of fuzzy theory and then, it is applied on optimization algorithm, GA (Pakhira, Bandyopadhyay, & Maulik, 2005). In a

further way, the multiple objectives are combined to do the clustering optimization as like (Bandyopadhyay, 2011). Here, cluster stability and validity are combined as fitness and then it is solved using optimization algorithm, simulated annealing (Saha & Bandyopadhyay, 2009). By means of the overall analysis, our finding is that most of the optimization algorithm utilizes k-means (KM) and FCM objective for their clustering optimization. Moreover with the best of our knowledge, MKFCM (Multiple Kernel FCM) objective is not solved previously through optimization clustering. So, with the intention of doing clustering task with optimization totally, two well known objectives (k-means and FCM), two recent objectives (KFCM and MKFCM) and three newly designed objective functions are utilized here. The reason of selecting these objectives is its (i) applicability and popularity (k-means and FCM objectives are chosen), (ii) regency and standard (KFCM and MKFCM are chosen), (iii) effectiveness and importance (three newly designed objective function). Then, we are in need of optimization algorithm for solving these objectives. Even though various optimization algorithms are presented in the literature, three optimization algorithms are chosen for our task of applying clustering process. Here, GA, PSO algorithm and CS algorithm are chosen because GA is traditional and popular one (Goldberg & David, 1989), PSO is an intelligent algorithm accepted by various researchers to its capability of changing the condition according to its most optimistic position (Kennedy & Eberhart, 1995), CS is a recent and effective algorithm proved better for various complex task of engineering problems (Yang & Deb, 2010). The basic organization of the paper is given as follows: Section 2 provides contributions of the paper and Section 3 discusses objective measures taken from the literature. Section 4 presents new objective functions designed and Section 5 provides the solution encoding procedure. Section 6 discusses optimization algorithms taken for data clustering and Section 7 discusses the experimentation with detailed results. Finally, the conclusion is summed up in Section 8.

152

2. Contributions of the paper

188

The most important contributions of the paper are discussed as follows:

189

(i) Clustering process with optimization: We have developed clustering process through optimization technique in order to accomplish the optimal cluster quality. So, two traditional objective function (KM and FCM), two recent objective functions (KFCM and MKFCM) and three newly developed objective functions are operated to do the task. Moreover, the optimization algorithms such as, GA, PSO and CS algorithm are considered. (ii) Hybridization: With the best of our knowledge, MKFCM objective is firstly solved with the optimization algorithms in this work. Hence, three optimization algorithms such as, GA, PSO and CS algorithm are combined with MKFCM objective functions to get three new hybridization algorithms, (GA-MKFCM, PSO-MKFCM, and CS-MKFCM) which are not presented in the literature previously. (iii) New objective functions: We have designed three new objective functions (FCM + CF, KFCM + KCF, MKFCM + MKCF), including the cumulative summation of fuzzy membership and distance value. Here, the same cumulative summation is also performed with kernel space as well as multiple kernel space. Again, these three new objective functions are derived with good mathematical formulation and the corresponding theorem and the proof is moreover provided.

191

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187

190

192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 3

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx Table 1 Algorithms. GA KM FCM KFCM MKFCM FCM + CF KFCM + KCF MKFCM + MKCF a b c

214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236

PSO c

GA-KM GA-FCMc GA-KFCMc GA-MKFCMb GA-FCM + CFa GA-KFCM + KCFa GAMKFCM + MKCFa

CS c

PSO-KM PSO-FCMc PSO-KFCMc PSO-MKFCMb PSO-FCM + CFa PSO-KFCM + KCFa PSOMKFCM + MKCFa

c

CS-KM CS-FCMc CS-KFCMc CS-MKFCMb CS-FCM + CFa CS-KFCM + KCFa CSMKFCM + MKCFa

2.1. Problem definition Let X be the database, consisting of n data points located in d-dimensional real space of xi 2 Rd . The definition of clustering is to divide the n data points into m clusters, which means that mj ð1 6 j 6 gÞ cluster centre should be identified from the input database.

243

245

3.1. Objective function 1: (KM)

246

The clustering problem defined above is converted into optimization problem in such a way that minimizing the summation of distance between the entire data points with its nearest cluster. Let the objective function of k-means clustering (McQueen, 1967)

248 249 250

251

OBKM ¼ 253 254

255

g n X X   xi  mj 2

ð1Þ

i¼1 j¼1

ðiÞ mi \ mj ¼ /;

261 262

ð3Þ 265

i¼1 j¼1

ðiiiÞ ðiv Þ

266

267

i; j ¼ 1; 2; . . . ; g; i – j

i ¼ 1; 2; . . . ; g

g

[ xi 2 mj ¼ X;

j¼1

g X

uij ¼ 1;

i ¼ 1; 2; . . . ; n

i ¼ 1; 2; . . . ; n

ð4Þ

j¼1

  1  xi  mj b1 ðv Þ ubij ¼ P   1  g xi  mj b1 j¼1

269

3.3. Objective function 3: (KFCM)

270

The same clustering problem can be indicated as optimization problem including, distance and fuzzy membership as variables but, the computation is done in kernel space instead of original data space. The kernel-based clustering problem is given as (Zhang & Chen, 2004),

271 272 273 274 275

276

OBKFCM

g n X X ¼ ubij ð1  kðxi ; mj ÞÞ

ð5Þ 278

i¼1 j¼1

subject to the following constraints:

ðiÞ mi \ mj ¼ /;

279

280

i; j ¼ 1; 2; . . . ; g; i – j

ðiiÞ mi – /; i ¼ 1; 2; . . . ; g g [ xi 2 mj ¼ X; i ¼ 1; 2; . . . ; n ðiiiÞ j¼1

ðiv Þ

g X uij ¼ 1;

i ¼ 1; 2; . . . ; n

j¼1

ð6Þ

1

ð1  kðxi ; mj ÞÞb1 g j¼1 ð1

1

282

As suggested in Chen et al. (2011), the value of r and fuzzification coefficient, b are fixed as 150 and 2 respectively.

283

3.4. Objective function 4: (MKFCM)

285

The purpose of grouping the data points presented in the original data base, X can be also denoted as optimization problem (Chen et al., 2011) like,

286

284

287 288

289

subject to the following constraints:

OBMKFCM

i; j ¼ 1; 2; . . . ; g; i – j

ðiiÞ mi – /; i ¼ 1; 2; . . . ; g g [ xi 2 mj ¼ X; i ¼ 1; 2; . . . ; n ðiiiÞ 257

OBFCM

 kðxi ; mj ÞÞb1   ! xi  mj 2 ðv iÞ kðxi ; mj Þ ¼ exp  r2

3. Objective measures considered from the literature

260

263 g n X X  2 ¼ ubij xi  mj 

ðv Þ ubij ¼ P

244

247

259

ðiiÞ mi – /;

(iv) Algorithms: We have presented nine algorithms (GAFCM + CF, PSO-FCM + CF and CS-FCM + CF, GA-KFCM + KCF, PSO-KFCM + KCF, CS-KFCM + KCF, GA-MKFCM + MKCF, PSOMKFCM + MKCF, and CS-MKFCM + MKCF) recently with the help of three newly designed objective functions. Three algorithms (GA-MKFCM, PSO-MKFCM, and CS-MKFCM) are presented by hybridizing in a new way as not presented in the literature. Nine obtainable algorithms are also considered subsequently. So, totally, 21 algorithms are discussed in this paper. These algorithms are the major contribution of this paper after detailed analysis with the literature. The clustering algorithms formulated based on the hybridization of optimization algorithms and its objective function is specified in Table 1. (v) Validation: To validate the 21 algorithms, five evaluation metrics and 16 datasets where, eight real datasets, two image data and six datasets generated are synthetically utilized. Then, performance of algorithms is extensively analyzed with three different perspectives like, search algorithm, objective functions and hybridization view to encounter the effect of clustering results. Most excellent algorithms for clustering are suggested at last with respect to the characteristics of input data.

240 242

The objective of clustering problem can be represented in another way utilizing the fuzzy membership function along with the distance variable. Let the objective function of FCM (Bezdek, 1981)

ðiÞ mi \ mj ¼ /;

239

241

258

subject to the following constraints:

Denotes algorithms with new objective function (noval works). Denotes algorithms with old objective function (no similar works). Denotes existing algorithms.

237 238

3.2. Objective function 2: (FCM)

g n X X ¼ ubij ð1  kcom ðxi ; mj ÞÞ

ð7Þ

i¼1 j¼1

ð2Þ

subject to the following constraints:

j¼1

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

291 292

293

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 4

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

ðiÞ mi \ mj ¼ /;

i; j ¼ 1; 2; . . . ; g; i – j

utilized to do the membership and distance computation instead of the original data space. Hence, we named it as, kernel cumulative functions (KCF). Let the new definition of objective function for clustering considering KCF

ðiiÞ mi – /; i ¼ 1; 2; . . . ; g g [ xi 2 mj ¼ X; i ¼ 1; 2; . . . ; n ðiiiÞ j¼1

ðiv Þ

g X uij ¼ 1;

OBKFCMþKCF ¼

i ¼ 1; 2; . . . ; n 1

ðv Þ ubij ¼ P

ð1  kcom ðxi ; mj ÞÞb1 m j¼1 ð1

n X

ubij ð1  kðxi ; mj ÞÞ þ

j¼1 i¼1;i2j

ð8Þ

j¼1

g X

n X

 1 b1

!

!

g X

n X

j¼1

i¼1;i2j

295

ðv iÞ kcom ðxi  mj Þ ¼ k1 ðxi ; mj Þ  k2 ðxi ; mj Þ   ! xi  mj 2 ðv iiÞ k1 ðxi ; mj Þ ¼ k2 ðxi ; mj Þ ¼ exp  r2

296

4. Devising of new objective functions

297

4.1. Objective function 5: (FCM + CF)

329 330 331

332

ubij

ð1  kðxi ; mj ÞÞ

ð11Þ 334

i¼1;i2j

 kcom ðxi ; mj ÞÞ

328

subject to the following constraints:

335

336

ðiÞ mi \ mj ¼ /; i; j ¼ 1; 2; . . . ; g; i – j ðiiÞ mi – /; i ¼ 1; 2; . . . ; g g [ ðiiiÞ xi 2 mj ¼ X; i ¼ 1; 2; . . . ; n ðiv Þ

j¼1 g X

uij ¼ 1;

i ¼ 1; 2; . . . ; n

ð12Þ

j¼1 1 b1

298 299 300 301 302 303 304 305 306 307 308 309 310 311 312

313

The new objective function is introduced with the concern of distance, fuzzy variable along with two additional variables that are not defined previously, called cumulative distance and cumulative fuzzy values. These two additional variables are added into the objective function of clustering because outlier data points or noises can contribute to objective function even more with the original data points. This contribution can be omitted by adding the cumulative distance and cumulative membership value with the old objective function. With the intention of this, two variables are additionally added with FCM objective function to make more suitable for clustering even outlier data points are presented. The following objective function is designed for clustering with the addition of FCM objective with the newly introduced term called, cumulative function (CF). Let the objective function of clustering based on new term defined,

OBFCMþCF ¼

g n X X



ubij xi

g 2 X  mj  þ

j¼1 i¼1;i2j

 315 316

317

! n  X   xi  mj 2

j¼1

n X

!

ubij i¼1;i2j

g m X X

ubij ð1  kcom ðxi ; mj ÞÞ þ

j¼1 i¼1;i2j n X 

1  kcom ðxi ; mj Þ

!

n X

j¼1

i¼1;i2j

!

subject to the following constraints:

ðiiÞ mi – /; i ¼ 1; 2; . . . ; g g [ xi 2 mj ¼ X; i ¼ 1; 2; . . . ; n ðiiiÞ

ðiÞ mi \ mj ¼ /;

ð10Þ

4.2. Objective function 6: (KFCM + KCF)

321

The new definition of cumulative function by considering the kernel space is given and it is utilized along with the objective function of KFCM to define the clustering optimization. Absolutely, it is a small variant of cumulative function which is defined above but, it affects the performance of the algorithm very well. The difference is very simple that the cumulative function is performed in the original data space but here, the kernel space is

j¼1

341 342 343 344 345 346 347 348 349

ubij ð13Þ 351

ðiÞ mi \ mj ¼ /; i; j ¼ 1; 2; . . . ; g; i – j

g X uij ¼ 1; i ¼ 1; 2; . . . ; n



g X

i¼1;i2j

320

327

340



319

326

Even though the kernel space played a significant role in distance computation of clustering algorithm, multiple kernels also played key role in differentiating the data points. Multiple kernels are combined to act as kernel at this point. It is just the variant of kernel-based objective function in addition with combined kernel. Similarly, cumulative function is also performed with multiple kernels so that the name of the function we utilized after this is multiple kernel cumulative function (MKCF). Let the addition of these two objective function taken for clustering optimization

subject to the following constraints:

  1  xi  mj b1 ðv Þ ubij ¼ P   1  g xi  mj b1

325

339

ð9Þ

j¼1

324

4.3. Objective function 7: (MKFCM + MKCF)

i¼1;i2j

ðiv Þ

323

338

OBMKFCMþMKCF ¼

j¼1

322

ð1  kðxi ; mj ÞÞ ðv Þ ubij ¼ P 1 g b1 j¼1 ð1  kðxi ; mj ÞÞ   ! xi  mj 2 ðv iÞ kðxi ; mj Þ ¼ exp  r2

352

353

i; j ¼ 1; 2; . . . ; g; i – j

ðiiÞ mi – /; i ¼ 1; 2; . . . ; g g [ xi 2 mj ¼ X; i ¼ 1; 2; . . . ; n ðiiiÞ j¼1

ðiv Þ

g X uij ¼ 1;

i ¼ 1; 2; . . . ; n ð14Þ

j¼1 1

ð1  kcom ðxi ; mj ÞÞb1

ðv Þ ubij ¼ P g

j¼1 ð1

1

 kcom ðxi ; mj ÞÞb1

ðv iÞ kcom ðxi ; mj Þ ¼ k1 ðxi ; mj Þ  k2 ðxi ; mj Þ   ! xi  mj 2 ðv iiÞ k1 ðxi ; mj Þ ¼ k2 ðxi ; mj Þ ¼ exp  r2

355

5. Solution encoding

356

The explanation encoding procedure employed in the clustering is given in Fig. 1. Every solution defined in the problem space is a

357

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

358

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

5

Fig. 1. Solution encoding. 399

367

vector which consists of d  g v alues: This signifies that the solution is centroid values for the given dataset, X. Suppose, the datasets is having 3 attributes (3 dimensions), and then the solution vector will contain six elements if the number of cluster needed is two. The first three elements in the solution vector is the first centroid and the last three elements in the solution vector the second centroid. This way of representing the solution can fulfill the optimization criterion with less computation times even though the dimension of the solution is high.

368

6. Optimization algorithms for data clustering

369

381

Solving optimization problems requires well-established heuristic procedures from the field of intelligent search towards using decision making in expert system. Metaheuristics are extensively renowned as efficient approaches for many hard optimization problems including cluster analysis. The field of metaheuristics for the application to real world optimization problems is a rapidly growing field of research. This is due to the importance of optimization problems for the scientific as well as the industrial world to do decision making much faster. As these metaheuristics technologies mature and lead to widespread deployment of expert systems, optimal finding of solution becomes more important. Here, three different heuristic search algorithms are effectively utilized for solving clustering problem.

382

6.1. Genetic algorithm

384 383

Step 1: Initial population: Initially, P solutions are given in a population and every chromosome with d  m vector. Step 2: Fitness computation: For every chromosome (centroid sets), the objective function is computed. Step 3: Selection: From the initial population, set of chromosomes are selected randomly based on selection rate. Step 4: Crossover: The crossover operator is applied on the selected two candidates, and this produces two individuals newly. Step 5: Mutation: The obtained new set of individuals is then fed to the mutation operator that also provides a new set of chromosomes. Step 6: Termination: After performing cross over and mutation operators, the algorithm go to step 2 up to the maximum number of iteration specified by the user are reached.

359 360 361 362 363 364 365 366

370 371 372 373 374 375 376 377 378 379 380

385 386 387 388 389 390 391 392 393 394 395 396 397 398

6.2. PSO algorithm

400

Step 1. Initially, P solutions (centroid sets) are given in an initial set of particles. Initial solutions are taken from the input dataset X and the velocity value of every particle is set to zero. Step 2. For every particle (centroid), the position is computed based on the objective function and the particle with minimum fitness is assigned as pbest for the current iteration. pbest is local best guided towards reaching best position based on one particle. Step 3. Determine the particle that have minimum fitness with respect to entire iterations executed and update it as g best which is global best particle found by all the particles in search space. Step 4. Once we find pbest and g best , the particles’ (centroids’) velocities are newly generated using the following equation.

402 401 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417

418

v tþ1 ¼ w  v t þ /1  rndðÞ  ðpbest  xt Þ þ /2  rndðÞ  ðg best  xt Þ

ð15Þ

420

where /1 and /2 are set as two. v t is the old velocity of the particle and rndðÞ is a random number between (0, 1). xt is the current particle taken for finding new velocity. Step 5. Then, new positions for all the particles are found out using the new velocity and previous positions. The formulae used for computing new position are given as follows:

421

xtþ1 ¼ xt þ v tþ1

ð16Þ

Once we generate new position, lower bound and upper bound conditions are checked based on lower bound and upper bound values availed in the input database. If the new position value is less than the lower bound, then, the new value is replaced with lower bound value and if it is greater, replace with upper bound value. 7. Go to step 2 until it reach maximum iteration.

422 423 424 425 426

427 429 430 431 432 433 434 435 436 437

6.3. Cuckoo search algorithm

438

Step 1: Initially, P solutions are given in an initial set of nests and every nest represents with d  m matrix.

440 439

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

441

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 6 442 443 444 445 446 447 448 449 450 451 452 453 454

455 457

Step 2: Choose a random number ðjÞ through levy flight equation in between 1 to P and the corresponding solution (centroids sets) is chosen. Step 3: Evaluate the fitness of nest in j th location based on the objective function and a random number generated in between 1 to P blindly and the solution given in i th location of initial population is chosen to find the fitness of the solution. Step 4: Replacing nest j by new solution if the fitness belongs to j is less than i : The evaluation of the fitness of both solutions taken from the previous steps is found out by comparing the fitness. The new solution xðtþ1Þ for the worst nest is performed by,

xðtþ1Þ ¼ xðtÞ þ a  Lev yðkÞ

459 460 461 462

463 465

467 468 469 470 471 472 473 474 475 476 477 478

ð17Þ

where a > 0 is the step size which should be related to the scales of the problem of interest. The product  means entry-wise multiplications. Levy flights essentially provide a random walk whereas their random steps are drawn from a Levy distribution for large steps,

458

466

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

Lev y  u ¼ t k ;

ð1 < k 6 3Þ

Once we generate new position, lower bound and upper bound conditions are checked based on lower bound and upper bound values availed in the input database. If the value obtained in new solution based on levy flight is greater than upper bound, the values are replaced with upper bound value. If the value is less than lower bound, new value is replaced with lower bound value. Step 5: Based on the probability pa given in the algorithm, the worst set of nests are identified and building a new one in the corresponding location. Step 6: The best set of nest is maintained in every iteration based on the objective function and the process is continued from step 2 to 5 until the maximum iteration is reached.

480

7. Results and discussion

481

486

This section presents experimental validation of the clustering techniques. In order to handle with, evaluation metrics and dataset taken for the validation of the clustering techniques is given with full description. Then, detailed experimental results are given with tables and the corresponding discussion is furthermore given in this section.

487

7.1. Evaluation metrics

488

The performance of the algorithm is measured through effectiveness and efficiency where, the effectiveness of the algorithms is evaluated with three different evaluation metrics, like clustering accuracy (CA) given in Yang and Chen (2011), Rand efficient (RC), Jaccard co-efficient (JC) given in Wan et al. (2012) and Adjusted rand index (ARI). The efficiency is measured with computation time. The definition of these metrics is given as follows: Clustering accuracy,

483 484 485

489 490 491 492 493 494 495 496

497

PK

i¼1 maxjf1;2;...;gg

499 500 501 502

503 505

CA ¼

 

jC \P j 2 C þi Pmj j i j mj jj

K

ð19Þ

Here, C ¼ fC 1 ; . . . ; C K g is a labeled data set that offers the ground  truth and Pm ¼ Pm1 ; . . . ; P mg is a partition produced by a clustering algorithm for the data set.

Rand co-efficient; RC ¼ ðSS þ DDÞ=ðSS þ SD þ DS þ DDÞ

ð21Þ

ð20Þ

506 508

Adjusted rand Index,

509



n ðSS þ DDÞ  ½ðSS þ SDÞðSS þ DSÞ þ ðDS þ DDÞðSD þ DDÞ 2 ARI ¼ 2 n  ½ðSS þ SDÞðSS þ DSÞ þ ðDS þ DDÞðSD þ DDÞ 2

510

ð22Þ

512

Here, SS, SD, DS, DD represent the number of possible pairs of data points where,

513

SS: both the data points belong to the same cluster and same group. SD: both the data points belong to the same cluster but different groups. DS: both the data points belong to different clusters but same group. DD: both the data points belong to different clusters and different groups.

515

ð18Þ

479

482

Jaccard co-efficient; JC ¼ ðSSÞ=ðSS þ SD þ DSÞ

514

516 517 518 519 520 521 522 523

Computation time: The efficiency of all the algorithms is evaluated with the execution time which is measured by Matlab syntax, ‘‘tic’’ and tac’’. ‘‘toc’’ reads the elapsed time from the stopwatch timer started by the ‘‘tic’’ function.

524

7.2. Datasets description

528

The experimental validation is performed with 16 different datasets which are taken under four different categories like, small scale real data, synthetic data, large scale real data and image data. Table 2 details total data objects, total attributes number of classes and dimension of solution for all the 16 datasets taken for the experimentation and Fig. 2 views the synthetic data generated and the image data. Small scale real data: For small scale data validation, eight datasets are captured from UCI learning repository (UCI, 2013). The datasets are Iris, PID, Wine, Blood transfusion, Mammogram and sonar dataset which are used to evaluate the clustering. Synthetic data: The synthetic information is generated to validate the performance of the algorithm with respect to various size, shape, overlapping and classes. This synthetic data-based experimentation provides to what extent the algorithms are better for different shapes, density of data records, data size, overlapping and classes.

529

Table 2 Description of datasets. Dataset name

Notation

Data objects

Attributes

No. of class

Dimension of solution

Iris PID Wine Sonar Blood Transfusion Mammogram Secom Madelon Synthetic Synthetic Synthetic Synthetic Synthetic Synthetic Image Image

RD1 RD2 RD3 RD4 RD5 RD6 LRD1 LRD2 SD1 SD2 SD3 SD4 SD5 SD6 Micro MRI

150 768 178 208 748 961 1567 4400 10,029 12,745 42,248 55,647 5761 22,162 65,025 65,025

4 8 13 60 5 6 591 500 3 3 3 3 3 3 6 6

3 2 3 2 2 2 2 2 2 3 2 2 2 2 2 3

9 14 36 118 8 10 1180 998 4 6 4 4 4 4 10 15

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

525 526 527

530 531 532 533 534 535 536 537 538 539 540 541 542 543 544

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 7

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

SD1

SD5

SD2

SD3

SD4

SD6

Microarray image

MRI image

Fig. 2. Visualization of synthetic and image data.

564

Accordingly, SD1 and SD5 datasets are generated with the intention of evaluating the performance in clustering of letter symbols. SD2 is generated to evaluate the performance of clustering algorithm for the widely accepted shapes. SD6 is generated to validate the performance of clustering algorithm in overlapping. Similarly, SD3 and SD4 are used for validating the performance of the algorithms in irregular shape of clusters. The synthetic dataset are generated through the synthetic code which gets an input as an image drawn by the users. Then, the image is converted to data sample by finding the location of the pixel with intensity as more than 0. Based on the pixel location, 2D data is formed for every pixel; refereeing x and y coordinate values. Large scale real data: Here, two datasets such as secom and madelon are chosen with large dimension of solution nearly 1000 from UCI learning repository (UCI, 2013). Image data: Two medical images such as MRI and microarray image are taken and then, data is formulated based on the gray level intensity. Here, the total number of data records will be equal to the total number of pixels in the image and attributes of every pixel will be its intensity value and its four neighbor pixels value.

565

7.3. Experimental set up

566

The clustering algorithms are written in MATLAB programming (Version: R2011a) and the results are taken after running with a system of having 2.13 GHz Intel (R) Pentium (R) CPU with 2 GB RAM. For genetic algorithm, we have fixed a Cross over rate and Mutation rate as 0.8 and 0.005 respectively, based on the suggestion given in Hong and Kwong (2008). For PSO algorithm, the parameter fixed are, w ¼ 0:72; /1 ¼ 1:49; /2 ¼ 1:49 based on recommendation (Merwe & Engelbrecht, 2003). Also, pa ¼ 0:25; a ¼ 1 is set for cuckoo search algorithm as per (Elkeran, 2013). Another one operator considered is cluster size which is fixed based on the ground truth classes in the dataset, means that the number of classes in the dataset is equivalent to cluster size given for clustering. Since the cluster size and attributes are fixed, the dimension of the solution is also fixed. The other two common parameters in the optimization algorithm are iteration and population. These two operators are fixed as 100

545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563

567 568 569 570 571 572 573 574 575 576 577 578 579 580 581

and 10, respectively as these two are global convergence operator which is common for all optimization algorithms.

582

7.4. Performance evaluation

584

7.4.1. Experimentation with synthetic data The experimentation of all the 21 algorithms and k-means for the synthetic data is executed in this section. The average performance of the algorithms for synthetic data is given with three tables that are plotted based on three different views like, algorithm perspective, objective perspective and hybridization perspective. After finding performance measures for all the algorithms, results are summarized by taking mean value of the six synthetic data. So, for the hybridization perspective, six samples (measures of all the six synthetic data) are collected for all the 21 algorithms and k-means to fill up Table 5. Similarly, the summary of algorithmic perspective given in Table 3 for synthetic data is filled up by taking the seven samples (out of 21 algorithms, every search algorithms are used seven times) for every data so globally, the average performance of 42 samples (7 algorithms ⁄ 6 synthetic data). It means that every tuple of Table 3 is the average performance of seven genetic-based algorithms given in Table 1 with six synthetic data. Table 4 is the performance summary of all the clustering algorithms with the perspective of objective formulation. The values are filled by taking 3 samples (out of 21 algorithms, three unique objective functions are used) for every data so globally, the average performance of 18 samples (3 same objectives algorithms⁄6 synthetic data) which are collected after performing the clustering with three same objective-based algorithms given in Table 1 with six synthetic data samples. For all the six synthetic datasets, the maximum performance (one) is achieved by any one of the 21 algorithms in terms of CA, RC and JC. In terms of ARI, CS-KM achieved maximum performance by reaching 0.9936. The minimum computation time for SD1, SD2, SD3, SD4, SD5 and SD6 data is 148.206113 s, 280.510192 s, 701.163344 s, 950.327078 s, 87.518454 s and 5415 s. The minimization will be varied based on datasets and the objective functions so, through these objectives, the performance cannot be compared. From Table 3, CS in terms of CA is obtained 0.9164

585

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

583

586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 8

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

Table 3 Summary of data experimentation in the view of search algorithm. GA

PSO

CS

K-means

GA

PSO

CS

K-means

Synthetic data

CA RC JC ARI Time

0.8921 0.8724 0.8609 0.9028 49288.0

0.8748 0.8548 0.7647 0.8885 2237.0

0.9164 0.8735 0.8745 0.9179 5665.1

0.8944 0.8463 0.7747 0.9014 5415.2

Image data

CA RC JC ARI Time

0.9229 0.8772 0.8833 0.9338 55994.0

0.911 0.8322 0.7506 0.9277 1665.0

0.9293 0.8506 0.7910 0.9287 3771.0

0.9211 0.8567 0.8083 0.9286 13490.0

Small scale real data

CA RC JC ARI Time

0.809386 0.59194 0.45826 0.8047 54.9

0.808186 0.611712 0.481398 0.7975 2.8

0.819169 0.599895 0.441381 0.8168 7.2

0.7535 0.6522 0.4768 0.7605 24.3

Large scale real data

CA RC JC ARI Time

0.7961 0.5967 0.6274 0.7513 903.6

0.8104 0.6094 0.6374 0.7584 34.9

0.8059 0.5944 0.6337 0.7562 79.7

0.7625 0.5149 0.5769 0.7346 295.3

Table 4 Summary of data experimentation in the view of objective formalization. KM

FCM

KFCM

MKFCM

FCM + CF

KFCM + KCF

MKFCM + MKCF

Synthetic data

CA RC JC ARI Time

0.9870 0.9876 0.9728 0.9772 5415.0

0.9620 0.9436 0.9146 0.9187 7341.0

0.8030 0.8226 0.8707 0.9555 8979.0

0.7418 0.6556 0.5145 0.7568 5079.0

0.9486 0.8854 0.8334 0.9464 3980.0

0.9227 0.8775 0.8140 0.8633 3015.0

0.7392 0.6513 0.4968 0.7660 6067.0

Small scale real data

CA RC JC ARI Time

0.7539 0.6523 0.4768 0.7608 24.3

0.7944 0.6215 0.4629 0.7893 24.4

0.8670 0.6194 0.4777 0.8769 21.0

0.6135 0.4806 0.3678 0.5803 20.2

0.8409 0.6431 0.4586 0.8458 23.8

0.9309 0.6744 0.5465 0.9322 18.6

0.8848 0.5167 0.4319 0.8592 19.3

Image data

CA RC JC ARI Time

0.9362 0.9068 0.8728 0.9212 13490

0.9564 0.8886 0.8287 0.9500 22132

0.9641 0.8938 0.8537 0.9780 22138

0.8115 0.7312 0.6862 0.8252 30323

0.9769 0.9141 0.8759 0.9948 10477

0.9456 0.8699 0.8181 0.9654 14392

0.8568 0.7924 0.7228 0.8759 30387

Large scale real data

CA RC JC ARI Time

0.7628 0.5149 0.5770 0.7228 295.4

0.7235 0.4738 0.5496 0.7150 306.1

0.8525 0.6872 0.6873 0.7797 359.1

0.8527 0.6874 0.6877 0.7796 364.4

0.7321 0.4636 0.5536 0.7188 306.2

0.8529 0.6872 0.6873 0.7797 434.5

0.8529 0.6872 0.6873 0.7797 376.7

Table 5 Summary of synthetic data experimentation in the view of objective formalization and search algorithm.

K-means GA-KMc PSO-KMc CS-KMc GA-FCMc PSO-FCMc CS-FCMc GA-KFCMc PSO-KFCMc CS-KFCMc GA-MKFCMb PSO-MKFCMb CS-MKFCMb GA-FCM + CFa PSO- FCM + CFa CS- FCM + CFa GA-KFCM + KCFa PSO- KFCM + KCFa CS- KFCM + KCFa GA-MKFCM + MKCFa PSO- MKFCM + MKCFa CS- MKFCM + MKCFa

619 620 621 622

CA

RC

JC

ARI

Time

0.8944 0.9792 1 1 0.9713 1 0.9844 0.9751 0.8884 0.9864 0.7259 0.7244 0.7752 0.8904 0.9742 0.9212 0.9752 0.7232 0.7168 0.7712 0.7186 0.7711

0.8463 0.9628 1 1 0.9296 0.9578 0.9615 0.9254 0.8648 0.9608 0.6422 0.6390 0.6857 0.8623 0.9140 0.8799 0.9243 0.6664 0.6148 0.6627 0.6189 0.6722

0.7747 0.9395 1 1 0.8826 0.9297 0.9315 0.8742 0.8054 0.9308 0.4996 0.5124 0.5314 0.7835 0.8627 0.8540 0.8725 0.5015 0.4518 0.5035 0.4510 0.5357

0.9014 0.9711 0.9669 0.9936 0.9740 0.9749 0.9872 0.9699 0.9151 0.9815 0.7741 0.7221 0.7742 0.9233 0.9741 0.9419 0.9524 0.6533 0.5814 0.7547 0.7776 0.7655

5415.2 14735.7 454.9 1055.1 12661.2 455.2 1174.6 15075.4 650.9 1224.7 25063.1 604.7 1611.6 13021.1 555.5 2584.4 8800.7 628.5 2616.8 14249.6 986.4 2965.1

which is higher than the genetic and PSO algorithm. For RC, CS algorithm is better and achieved 0.8735 which is higher than other two algorithms. Similarly, in terms of ARI, CS algorithm is better and achieved 0.9179 which is higher than other two algorithms

The CS algorithm outperformed in synthetic data as compared with existing algorithms. For the objective minimization, KM objective achieved 0.9870 which is higher than the existing and the proposed objective function. For the algorithmic perspective also, KM combined algorithms provided better results.

623

7.4.2. Experimentation with real data with small and medium scale dimension The summary experimental results of iris, PID, wine, sonar, transfusion and mammogram data are given in Table 3, 4 and 6. This experimentation is prepared with the intention of testing the effectiveness and efficiency of the clustering algorithm in small and medium scale dimension problems. In the datasets of PID, sonar, transfusion and mammogram, the maximum CA has been reached. The maximum accuracy reached by iris data is 0.9867. In iris data, maximum RC is 0.8923 and JC is 0.719. In terms of ARI, maximum value is 0.9480. Every clustering algorithm has taken six different real data as input and the clustering output are measured with CA, JC, RC, ARI and computation time. Then, summary of experimental results are obtained with three different perspectives and presented in Tables 3, 4 and 6. In Table 3, every tuples belonging to real data are found out after taking mean value from CA, JC and RC out of 42 samples (seven genetic-based algorithms⁄ six different data). So, Table 3 of real data has been the mean value of 42 samples. Based on this analysis, PSO algorithm provided better performance in terms of RC and JC but for cuckoo search, the summary value is 0.819 in terms of CA and 0.8168 in terms of ARI which is higher

628

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

624 625 626 627

629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 9

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx Table 6 Summary of real data experimentation in the view of objective formalization and search algorithm.

K-means GA-KMc PSO-KMc CS-KMc GA-FCMc PSO-FCMc CS-FCMc GA-KFCMc PSO-KFCMc CS-KFCMc GA-MKFCMb PSO-MKFCMb CS-MKFCMb GA-FCM + CFa PSO-FCM + CFa CS-FCM + CFa GA-KFCM + KCFa PSO- KFCM + KCFa CS-KFCM + KCFa GA-MKFCM + MKCFa PSO-MKFCM + MKCFa CS-MKFCM + MKCFa

650 651 652 653 654 655 656 657 658 659 660 661 662 663

664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688

CA

RC

JC

ARI

Time

0.7535 0.7729 0.7148 0.7741 0.7489 0.8179 0.8163 0.9144 0.8699 0.8167 0.5833 0.5927 0.6646 0.7900 0.8846 0.8480 0.9480 0.9458 0.8990 0.9079 0.8312 0.9152

0.6522 0.6550 0.6457 0.6562 0.5913 0.6376 0.6356 0.6380 0.6266 0.5935 0.4631 0.4621 0.5165 0.6228 0.6620 0.6444 0.7089 0.6404 0.6740 0.4641 0.6073 0.4787

0.4768 0.4848 0.4590 0.4867 0.4332 0.4783 0.4772 0.5049 0.4806 0.4475 0.3939 0.3807 0.3287 0.4523 0.4734 0.4502 0.5494 0.5892 0.5006 0.3890 0.5082 0.3984

0.7605 0.7771 0.7268 0.7780 0.7323 0.8276 0.8079 0.9069 0.8804 0.8441 0.5814 0.5223 0.6374 0.7912 0.8889 0.8572 0.9585 0.9464 0.8918 0.8858 0.7909 0.9011

24.3 62.5 3.3 7.2 62.6 3.2 7.3 52.8 3.1 7.1 50.2 3.2 7.1 56.3 2.9 12.2 48.7 2.1 4.9 51.2 2.0 4.6

than other two algorithms. The computation complexity is much less for PSO algorithm for these real datasets. In Table 4, summary is taken based on objective function-based performance of real data. Here, each values are obtained by finding the mean value of 18 samples (three same objective functions out of 21⁄six datasets). Here, the proposed kernel-based objective outperformed all other objectives in terms of CA, RC and JC. The proposed KFCM + KCF have achieved 0.9309 in CA and 0.9322 in ARI which is higher than existing objective functions. Likewise, the summary of hybridized algorithm for real data is given in Table 6 in which the values are taken by taking the average performance of six different real data considered. For this, GA-KFCM + KCF provided 0.94807 as CA value and 0.7089 as RC value. In terms of JC, PSO-KFCM + KCF provided better results.

7.4.3. Experimentation with image data The experimentation of all the 21 algorithms for the image data is performed in this section. To further analyze the clustering algorithm, the image data is taken and we apply all the algorithms to the image data like micro array and MRI image. The maximum CA reached by any one of the algorithm is 0.9895 and RC is 0.9792. The JC reached for Microarray data is 0.9751. Similarly, for the MRI image, the maximum value obtained by analyzing the entire algorithms is 0.9992, 0.9774, and 0.9639 in terms of CA, RC and JC respectively. The minimum time taken is 1159.035 s for microarray data and 1601.326 s for MRI image data. The summarized result of two image data is given in Tables 3, 4 and 7. In Table 3, the performance metrics are found out by taking the average performance over the same search algorithms with two different data (totally, 14 samples). Based on this Table 3, CS algorithm has given 0.9293 in terms of CA and other metrics are better outperformed by GA. The summarized results of image data with the perspective of objective functions are given in Table 4. Here, FCM + CF-based objectives (proposed) are outperformed over all the presented objectives. The proposed FCM + CF have achieved nearly a value of 0.9 for all the evaluation metrics, CA, RC, ARI and JC. The summarized result of hybridized form of algorithms is given in Table 7. Here, average value of two datasets for every algorithm is given. Based on this Table 7, except RC, the proposed CSFCM + CF have provided the better results of 0.987 through CA

Table 7 Summary of image data experimentation in the view of objective formalization and search algorithm.

K-means GA-KMc PSO-KMc CS-KMc GA-FCMc PSO-FCMc CS-FCMc GA-KFCMc PSO-KFCMc CS-KFCMc GA-MKFCMb PSO-MKFCMb CS-MKFCMb GA-FCM + CFa PSO-FCM + CFa CS-FCM + CFa GA-KFCM + KCFa PSO-KFCM + KCFa CS-KFCM + KCFa GA-MKFCM + MKCFa PSO-MKFCM + MKCFa CS-MKFCM + MKCFa

CA

RC

JC

ARI

Time

0.9211 0.9480 0.8820 0.9787 0.9722 0.9114 0.9857 0.9563 0.9706 0.9656 0.8783 0.7475 0.8088 0.9689 0.9749 0.9870 0.8970 0.9542 0.9857 0.8401 0.9363 0.7941

0.8567 0.9191 0.8382 0.9632 0.9606 0.7962 0.9090 0.9342 0.8777 0.8695 0.8212 0.6918 0.6807 0.9469 0.8857 0.9096 0.8544 0.8497 0.9055 0.7740 0.8861 0.7171

0.8083 0.9401 0.7335 0.9447 0.9383 0.6981 0.8497 0.9381 0.8170 0.8061 0.8783 0.5933 0.5870 0.9492 0.8258 0.8529 0.8194 0.7850 0.8500 0.7199 0.8016 0.6471

0.9286 0.9366 0.8512 0.9786 0.9745 0.8896 0.9855 0.9615 0.8239 0.9706 0.9106 0.7762 0.7892 0.9963 0.9822 0.8859 0.8942 0.8954 0.9928 0.8621 0.9830 0.7822

13490.6 35772.9 1683.7 3013.7 60674.2 1903.7 3818.4 61348.8 1658.9 3406.6 85831.7 1673.6 3463.7 27193.3 1380.1 2858.5 38051.3 1658.3 3468.1 83090.0 1699.5 6371.3

and 0.9492 in terms of JC. The proposed GA-FCM + CF have provided the better results of 0.9963 through ARI.

689

7.4.4. Experimentation with real data of large scale dimension For the large scale dimension analysis, two real datasets such as, secom and madelon are taken and the performance of algorithms is analyzed to find the suitability of algorithm in large scale cluster analysis. Based on the experimentation with large scale data, the maximum CA reached for secom data is 0.9994 and the maximum RC reached is 0.877. The maximum ARI reached for secom data is 0.8525. The minimum computation time required in secom data is 26.73115 s. For the madelon data, the maximum performance in terms of CA, RC and JC is 0.7065, 0.4995 and 0.5085. The computation time required for the minimum case is 34.75245 s. The summarized results are presented in Tables 3, 4 and 8. In Table 3, the average performance is computed by taking mean value of seven same search-based algorithms belonging to two different data and so it is the mean value of 14 samples. Table 3 clearly indicated that PSO algorithm outperformed in all the evaluation metrics when compared with other search algorithms. The PSO algorithm has given 0.8104 as CA whereas the second rank is for CS which reached only 0.8054. The computation time of PSO algorithm for large scale data is half of the CS algorithm. From Table 4, MKFCM objectives are better as compared with other objective functions. The proposed MKFCM + MKCF have provided 0.8529 as CA which is higher than the existing algorithms. From Table 8, PSO-MKFCM achieved better results in all the format of evaluation metrics taken, CA, RC, ARI and JC. The values for this objective function are 0.85295, 0.68825 and 0.68845.

691

7.5. Discussion

717

A thorough discussion about the experimentation of 21 different algorithms with 16 different datasets is provided in this section. The performance of clustering algorithms is analyzed and discussed through four different categories (based on datasets) with respect to effectiveness and efficiency along with three perspectives (search algorithm, objective function and hybridized form). The effectiveness has found out based on CA, JC and RC. The efficiency is found out using computation time.

718

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

690

692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716

719 720 721 722 723 724 725

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 10

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

Table 8 Summary of large data experimentation in the view of objective formalization and search algorithm.

K-means GA-KMc PSO-KMc CS-KMc GA-FCMc PSO-FCMc CS-FCMc GA-KFCMc PSO-KFCMc CS-KFCMc GA-MKFCMb PSO-MKFCMb CS-MKFCMb GA-FCM + CFa PSO-FCM + CFa CS-FCM + CFa GA-KFCM + KCFa PSO-KFCM + KCFa CS-KFCM + KCFa GA-MKFCM + MKCFa PSO-MKFCM + MKCFa CS-MKFCM + MKCFa

726 727 728 729 730

731 733 734

CA

RC

JC

ARI

Time

0.7625 0.7601 0.7664 0.7620 0.6678 0.7616 0.7412 0.8529 0.8529 0.8529 0.8526 0.8529 0.8526 0.7333 0.7333 0.7272 0.8529 0.8529 0.8529 0.8529 0.8529 0.8529

0.5149 0.5015 0.5565 0.4868 0.4609 0.4944 0.4663 0.6872 0.6872 0.6872 0.6866 0.6882 0.6875 0.4667 0.4651 0.4591 0.6872 0.6872 0.6872 0.6872 0.6872 0.6872

0.5765 0.5739 0.5800 0.5771 0.5156 0.5753 0.5581 0.6873 0.6873 0.6873 0.6868 0.6884 0.6878 0.5538 0.5561 0.5508 0.6873 0.6873 0.6873 0.6873 0.6873 0.6873

0.7346 0.7333 0.7364 0.7340 0.6871 0.7345 0.7275 0.7795 0.7795 0.7797 0.7795 0.7795 0.7795 0.7199 0.7199 0.7165 0.7795 0.7795 0.7795 0.7795 0.7795 0.7795

590.7 784.3 30.9 70.9 816.3 30.9 71.1 956.3 36.8 84.5 968.6 38.0 86.5 816.5 31.3 70.9 979.4 41.2 86.3 1004.2 38.4 87.6

7.5.1. Complexity analysis The computational complexity of objective function is computed based on big O notation. For worst case, the computational complexity (CC) of the proposed objective function is computed as follows:

CC ¼ Oðngðd þ 1Þ þ n log g þ 2ng log gÞ For best case,

735 737

CC ¼ Oðnðgðd þ 3ÞÞ þ 1Þ

738

For average case,

739 741 742 743

744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768

ð22Þ

CC ¼ Oðngðd þ 1:5 þ gÞÞ

ð23Þ

ð24Þ

where n are the total data points and d is the dimension of real data space and g is the number of clusters. 7.5.2. Findings For the first category (experimentation with synthetic data), CSbased algorithms are outperformed in three different effectiveness metrics and the efficiency is better for PSO-based algorithms in the case of search algorithm dependent perspective. In the perspective of objective functions, KM objective outperformed for all three evaluation metrics considered to prove the effectiveness. With respect to the hybridized clustering, PSO-KM and CS-KM provided the maximum accuracy and the PSO-KM provides less computation time. This performance ensures that the existing KM is better suitable if the data is integer. From the results, we can easily understand that the multiple kernel-based algorithms is struggled to find the exact number of clusters as compared with other clustering algorithms. While analyzing the performance based on optimization algorithms, there is no big performance deviation for GA, PSO and CS algorithm. Also, objective function-based performance is changed for every objective function which is the core of the analysis of effectiveness. Here, multiple kernel-based algorithms which may be proposed objective function or the existing function does not contribute much on the performance of clustering process as compared with other objective functions. For the second category (experimentation with small and medium scale real data), PSO algorithm has outperformed in two evaluation metrics and other one by cuckoo search algorithm.

The computation effort is minimized for PSO algorithm. For the objective-based analysis, the proposed KFCM + KCF proved better in all the effectiveness measure as well as efficiency as compared with other objective function. In terms of hybridized results, GAKFCM + KCF scored the highest rank in terms of CA and RC. On the other hand, PSO-KFCM + KCF are better in JC. The better efficiency is achieved by the PSO-KFCM + KCF. The existing objective functions are not much provided the effectiveness as compared with the proposed kernel-based objective function. The kernelbased proposed objective function provided better result for the real data which are from different application. The performance achieved by the proposed algorithm ensures that the application and its data range is not a problem for kernel-based objective function. For the third category (experimentation with image data), GA algorithm performed better in two evaluation metrics and other one by CS algorithm. Again, the time is minimized for PSO algorithm. In the perspective of objective formulation, the proposed FCM + CF has outperformed in both effectiveness and efficiency. In the hybridized form, GA-FCM + CF have provided better in terms of JC and CS-FCM + CF has provided better in terms of CA. The computation effort is minimized when PSO-FCM + CF is used for the image data, so this ensures that the proposed FCM + CF provided good significance if the data range is constant for all the attributes. For the fourth category (experimentation with large scale data), PSO-based algorithm outperformed in both effectiveness and efficiency. In the case of objective function-based analysis, the effectiveness-based measures are better improved by MKFCM and MKFCM + MKCF. The better efficiency is obtained by k-means for the large scale data. For the hybridized form, PSO-MKFCM is better choice to obtain more effective in clustering. The efficient result is achieved by the PSO-FCM algorithm for the large scale data. The conclusion from the experimentation with large scale data is that if the attribute size is very large along with different data range and its variance are not uniform, proposed MKFCM + MKCF provided the better results as compared with existing objective functions.

769

7.5.3. Suggestions Based on the analysis, we can easily say that the algorithmic effectiveness is decided by objective function and the algorithmic efficiency is decided by the search algorithm. For better efficiency, PSO algorithm is a right choice for all the different set of data which may be small scale or large scale. For effectiveness, the right objective function should be taken based on the characteristics of datasets such as, range of values, dimension, image and data type (integer or floating point). Based on this data characteristic, the suggestion is that KM can be chosen if (i) the data is fully integers and within constant interval, (ii) the dimension of solution is less. The KFCM + KCF can be chosen if, (i) the data may be any integer or floating point value data, (ii) the dimension may be small or medium. The FCM + CF can be chosen only if (i) the range of value is constant for all attributes, (ii) the values are medium, (iii) suitable for image analysis. Finally, the MKFCM + MKCF can be chosen only if, (i) the dimension of data is high, (iii) the range of values is not constant, (iii) the data may be integer or floating point.

806

8. Conclusion

824

We have presented 21 different techniques to find optimal clusters with the help of different objective functions and optimization algorithms for expert systems and its applications in the field of medical, telecommunication and engineering. Here, three objective functions are newly designed with four existing objective functions to incorporate with optimization algorithms. The new objective

825

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805

807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823

826 827 828 829 830

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

873

functions was introduced with the consideration of distance, fuzzy variable along with two additional variables that are not defined previously, called cumulative distance and cumulative fuzzy values. Totally, 21 different clustering algorithms are discussed with mathematical formulation after blending objective function with search algorithm like, GA, CS and PSO algorithm. The performance of algorithms is evaluated with 16 different datasets of different shape and characteristics. The effectiveness and efficiency of algorithms are compared using three different evaluation metrics with computation time. From the research outcome, we can easily conclude that the algorithmic effectiveness is better dependent on objective function and the algorithmic efficiency is decided based on search algorithm. Finally, the right choice of algorithms is suggested depending on the characteristic of input data. The clustering is a potential method of having various applications especially, in expert systems. The various experts systems needs of good unsupervised learning strategy to fulfill their requirements. The important practical application found out from the literature for the clustering is given as, speaker clustering (Tang, Palo Alto, Chu, Hasegawa-Johnson, & Huang, 2012), analysis of fMRI Data (Zhang, Xianguo, Zhen, Wei, & Huafu, 2011), wireless sensor network (Youssef, Youssef, & Younis, 2009), grouping of text documents (Cao, Zhiang, Junjie, & Hui, 2013), Auditory Scene Categorization (Cai, Lie, & Hanjalic, 2008), News Story Clustering (Xiao, Chong-Wah, & Hauptmann, 2008), Target Tracking (Liang et al., 2010), network (Huang, Heli, Qinbao, Hongbo, & Jiawei, 2013), Cancer Gene Expression Profiles (Zhiwen, Le, You, Hau-San, & Guoqiang, 2012), social networks (Caimei, Xiaohua, & Jung-ran, 2011). The major limitation of the proposed algorithm is user given cluster size which requires data knowledge for the user. The second limitation is thenumber of iteration which is a termination criteria utilized here for convergence. The critical analysis is required on defining the better termination criteria. Also, multiple parametric inputs and optimal fixing of threshold values are to overcome for better application of clustering process. The proposed clustering algorithm can be applied to clinical decision support system, disease diagnosis, agricultural research, forecasting and routing to obtain more effective results based on the findings of research. Also, clustering can be extended by modifying the optimization search algorithm towards reducing the time complexity. Again, the effectiveness can be even improved including the different constraints as per the datasets or user in the objective functions.

874

9. Uncited reference

831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872

875

Graves and Pedrycz (2010).

876

Appendix A. Supplementary data

877 879

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.eswa.2015.03. 031.

880

References

881 882 883 884 885 886 887 888 889 890 891 892 893

Bandyopadhyay, S. (2011). Multiobjective simulated annealing for fuzzy clustering with stability and validity. IEEE Transactions on Systems, Man, and Cybernetics – Part C: Applications and Reviews, 41(5), 682–691. Bezdek, J. C. (1981). Pattern recognition with fuzzy objective function algorithms. 0306406713. New York: Plenum Press. Cai, R., Lie, L., & Hanjalic, A. (2008). Co-clustering for auditory scene categorization. IEEE Transactions on Multimedia, 10(4), 596–606. Caimei, L., Xiaohua, H., & Jung-ran, P. (2011). The social tagging network for web clustering. IEEE Transactions on Systems, Man and Cybernetics. Part A: Systems and Humans, 41(5), 840–852. Cao, J., Zhiang, W., Junjie, W., & Hui, X. (2013). SAIL: Summation-based incremental learning for information-theoretic text clustering. IEEE Transactions on Cybernetics, 43(2), 570–584.

878

11

Castellanos-Garzón, J. A., & Diaz, F. (2013). An evolutionary computational model applied to cluster analysis of DNA microarray data. Expert Systems with Applications, 40(7), 2575–2591. Chen, L., Chen, C. L. P., & Lu, M. (2011). A multiple-kernel fuzzy c-means algorithm for image segmentation. IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics, 41(5). Das, S., Abraham, A., & Konar, A. (2008). Automatic clustering using an improved differential evolution algorithm. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 38(1), 218–237. Hoang, D. C., Yadav, P., Kumar, R., & Panda, S. K. (2014). Implementation of a harmony search algorithm-based clustering protocol for energy-efficient wireless sensor networks. IEEE Transactions on Industrial Informatics, 10(1), 774–783. Elkeran, A. (2013). A new approach for sheet nesting problem using guided cuckoo search and pairwise clustering. European Journal of Operational Research, 231(3), 757–769. Krishnasamy, G., Kulkarni, A. J., & Paramesran, R. (2014). Hybrid approach for data clustering based on modified cohort intelligence and K-means. Expert Systems with Applications, 41(13), 6009–6016. Garcia-Piquer, A., Fornells, A., Bacardit, J., Orriols-Puig, A., & Golobardes, E. (2014). Large-scale experimental evaluation of cluster representations for multiobjective evolutionary clustering. IEEE Transactions on Evolutionary Computation, 18(1), 36–53. Goldberg & David (1989). Genetic algorithms in search, optimization and machine learning. 978-0201157673. Reading, MA: Addison-Wesley Professional. Graves, D., & Pedrycz, W. (2010). Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study. Fuzzy Sets and Systems, 161, 522–543. Hong, Y., & Kwong, S. (2008). To combine steady-state genetic algorithm and ensemble learning for data clustering. Journal Pattern Recognition Letters, 29(9), 1416–1423. Huang, J., Heli, S., Qinbao, S., Hongbo, D., & Jiawei, H. (2013). Revealing densitybased clustering structure from the core-connected tree of a network. IEEE Transactions on Knowledge and Data Engineering, 25(8), 1876–1889. _ Inkaya, T., Kayalıgil, S., & Özdemirel, N. E. (2015). Colony optimization based clustering methodology. Applied Soft Computing, 28, 301–311. Ji, J., Pang, W., Zhou, C., Han, X., & Wang, Z. (2012). A fuzzy k-prototype clustering algorithm for mixed numeric and categorical data. Journal of Knowledge-Based Systems, 30, 129–135. Ji, Z., Xi, Y., Chen, Q., Sun, Q., Xia, D., & Feng, D. D. (2012). Fuzzy c-means clustering with weighted image patch for image segmentation. Applied Soft Computing, 12, 1659–1667. Kannana, S. R., Ramathilagam, S., & Chung, P. C. (2012). Effective fuzzy c-means clustering algorithms for data clustering problems. Expert Systems with Applications, 39(7), 6292–6300. Kennedy, J., & Eberhart, R. C. (1995). Particle swarm optimization. Proc. of the IEEE int’l conf. on neural networks (Vol. IV, pp. 1942–1948). Piscataway, NJ: IEEE service center. Khan, S. S., & Ahmad, A. (2004). Cluster center initialization algorithm for K-means clustering. Pattern Recognition Letters, 25, 1293–1302. Krishna, K., & Murty (1999). Genetic K-means algorithm. IEEE Transactions on Systems Man and Cybernetics B Cybernetics, 29, 433–439. Kuo, R. J., Syu, Y. J., Chen, Z. Y., & Tien, F. C. (2012). Integration of particle swarm optimization and genetic algorithm for dynamic clustering. Journal of Information Sciences, 19, 124–140. Liang, Z., Chaovalitwongse, W. A., Rodriguez, A. D., Jeffcoat, D. E., Grundel, D. A., & O’Neal, J. K. (2010). Optimization of spatiotemporal clustering for target tracking from multisensor data. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 40(2), 176–188. Linda, O., & Manic, M. (2012). General type-2 fuzzy c-means algorithm for uncertain fuzzy clustering. IEEE Transactions on Fuzzy Systems, 20(5), 883–897. Li & Qi (2007). Spatial kernel K-harmonic means clustering for multi-spectral image segmentation. IET Image Processing, 1(2), 156–167. Liyong, Z., Witold, P., Wei, L., Xiaodong, L., & Li, Z. (2014). An interval weighed fuzzy c-means clustering by genetically guided alternating optimization. Expert Systems with Applications, 41(13), 5960–5971. Maji, P. (2011). Fuzzy-rough supervised attribute clustering algorithm and classification of microarray data. IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics, 41(1), 222–233. McQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proc. of fifth berkeley symposium on math. Vol: Statistics and probability (pp. 281–297). Merwe, D. W. V. & Engelbrecht, A. P. (2003). Data clustering using particle swarm optimization. In The congress on evolutionary computation (Vol. 1, pp. 215–220). Mualik, U., & Bandyopadhyay, S. (2002). Genetic algorithm based clustering technique. Pattern Recognition, 33, 1455–1465. Niknam, T., & Amiri, B. (2010). An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis. Applied Soft Computing, 10, 183–197. Ouadfel, S., & Meshoul, S. (2012). Handling fuzzy image clustering with a modified ABC algorithm. International Journal of Intelligent Systems and Applications, 12, 65–74. Pakhira, M. K., Bandyopadhyay, S., & Maulik, U. (2005). A study of some fuzzy cluster validity indices, genetic clustering and application to pixel classification. Fuzzy Sets and Systems, 155(2). Pham, D. T., Dimov, S. S. & Nguyen, C. D. (2004). Selection of K in K-means clustering, In Proc. IMechE (p. 219).

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031

894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979

ESWA 9942

No. of Pages 12, Model 5G

10 April 2015 12 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010

D. Binu / Expert Systems with Applications xxx (2015) xxx–xxx

Premalatha, K., & Natarajan, A. M. (2008). A new approach for data clustering based on PSO with local search. Computer and Information Science, 1(4). Saha, S., & Bandyopadhyay, S. (2009). A new multiobjective simulated annealing based clustering technique using symmetry. Pattern Recognition Letters, 30, 1392–1403. Selim, S. Z., & Alsultan, K. (1991). A simulated annealing algorithm for the clustering problem. Pattern Recognition, 10(24), 1003–1008. Senthilnath, J., Omkar, S. N., & Mani, V. (2011). Clustering using firefly algorithm: Performance study. Swarm and Evolutionary Computation, 1, 164–171. Sulaiman, S. N., & Isa, N. A. M. (2010). Adaptive fuzzy-K-means clustering algorithm for image segmentation. IEEE Transactions on Consumer Electronics, 56(4), 2661–2668. Szilágyi, L., Szilágyi, S. M., Benyób, B., & Benyó, Z. (2011). Intensity inhomogeneity compensation and segmentation of MR brain images using hybrid c-means clustering models. Biomedical Signal Processing and Control, 6, 3–12. Tang, H., Palo Alto, C. A., Chu, S. M., Hasegawa-Johnson, M., & Huang, T. S. (2012). Partially supervised speaker clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(5), 959–971. Tengke, X., Shengrui, W., Qingshan, J., & Huang, J. Z. (2014). A novel variable-order Markov model for clustering categorical sequences. IEEE Transactions on Knowledge and Data Engineering, 26(10), 2339–2353. UCI machine learning repository from . Wan, M., Li, L., Xiao, J., Wang, C., & Yang, Y. (2012). Data clustering using bacterial foraging optimization. Journal of Intelligent Information Systems, 38(2), 321–341. Wei, S., Yingying, Q., Soon Cheol, P., & Xuezhong, Q. (2015). A hybrid evolutionary computation approach with its application for optimizing text document clustering. Expert Systems with Applications, 42(5), 2517–2524. Xiao, W., Chong-Wah, N., & Hauptmann, A. G. (2008). Multimodal news story clustering with pairwise visual near-duplicate constraint. IEEE Transactions on Multimedia, 10(2), 188–199.

Xu, R., Xu, J., & Wunsch, D. C. (2012). A comparison study of validity indices on swarm-intelligence-based clustering. IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics, 42(4), 1243–1256. Yang, Y., & Chen, K. (2011). Temporal data clustering via weighted clustering ensemble with different representations. IEEE Transactions on Knowledge and Data Engineering, 23(2). Yang, X. S., & Deb, S. (2010). Engineering optimization by cuckoo search. International Journal of Mathematical Modelling and Numerical Optimisation, 1(4), 330–343. Youssef, M., Youssef, A., & Younis, M. (2009). Overlapping multihop clustering for wireless sensor networks. IEEE Transactions on Parallel and Distributed Systems, 20(12), 1844–1856. Yuwono, M., Su, S. W., Moulton, B. D., & Nguyen, H. T. (2014). Clustering using variants of rapid centroid estimation. IEEE Transactions on Evolutionary Computation, 18(3), 366–377. Zhang, D. Q., & Chen, S. C. (2004). A novel kernelized fuzzy C-means algorithm with application in medical image segmentation. Artificial Intelligence in Medicine, 32(1), 37–50. Zhang, C., Ouyang, D., & Ning, J. (2010). An artificial bee colony approach for clustering. Expert Systems with Applications, 37, 4761–4767. Zhang, J., Xianguo, T., Zhen, Y., Wei, L., & Huafu, C. (2011). Analysis of fMRI data using an integrated principal component analysis and supervised affinity propagation clustering approach. IEEE Transactions on Biomedical Engineering, 58(11), 3184–3196. Zhao, F., Jiao, L., & Liu, H. (2013). Kernel generalized fuzzy c-means clustering with spatial information for image segmentation. Digital Signal Processing, 23, 184–199. Zhiwen, Y., Le, L., You, J., Hau-San, W., & Guoqiang, H. (2012). SC(3): Triple spectral clustering-based consensus clustering framework for class discovery from cancer gene expression profiles. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 9(6), 1751–1765.

1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042

Please cite this article in press as: Binu, D. Cluster analysis using optimization algorithms with newly designed objective functions. Expert Systems with Applications (2015), http://dx.doi.org/10.1016/j.eswa.2015.03.031