Teaching-learning-based optimization with variable-population scheme and its application for ANN and global optimization

Author’s Accepted Manuscript Teaching-learning-based optimization with variable-population scheme and its application for ANN and global optimization Debao Chen, Renquan Lu, Feng Zou, Suwen Li www.elsevier.com/locate/neucom

PII: DOI: Reference:

S0925-2312(15)01254-0 http://dx.doi.org/10.1016/j.neucom.2015.08.068 NEUCOM16014

To appear in: Neurocomputing Received date: 29 June 2015 Revised date: 15 August 2015 Accepted date: 24 August 2015 Cite this article as: Debao Chen, Renquan Lu, Feng Zou and Suwen Li, Teaching-learning-based optimization with variable-population scheme and its application for ANN and global optimization, Neurocomputing, http://dx.doi.org/10.1016/j.neucom.2015.08.068 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting galley proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Teaching-learning-based optimization with variable-population scheme and its application for ANN and global optimization Debao Chena, Renquan Lub*, Feng Zoua, Suwen Li a (aSchool of Physics and Electronic Information, HuaiBei Normal University, Huaibei, 235000, China b

Institute of Information and Automation, Hangzhou Dianzi University, Hangzhou,31000,China)

Abstract: A teaching-learning-based optimization algorithm (TLBO) which uses a variable population size in the form of a triangle form (VTTLBO) is proposed in the paper. The main goal of the proposed method is to decrease the computing cost of original TLBO and extend it for optimizing the parameters of artificial neural network (ANN). In the proposed algorithm, the evolutionary process is divided into some equal periods according to the maximal generation. The population size in each period is changed in form of a triangle. In the linear increasing phase of population’s number, some new individuals are generated with gauss distribution by using the adaptive mean and variance of the population. In the linear decreasing phase of population’s number, some highly similar individuals are deleted. To compare the performance of the proposed method with some other methods, Saw-tooth teaching-learning-based optimization algorithm is also designed with simulating the basic principle of Saw-tooth genetic algorithm (STGA), and some other EAs with the fixed population size are also simulated. A variety of benchmark problems and system modeling and prediction problems with ANN are tested in this paper, the results indicate that the computation cost of the given method is small and the convergence accuracy and speed of it are high. Keywords: Teaching-learning-based optimization (TLBO); Saw-tooth genetic algorithm (STGA); Variable-population, Triangle Teaching-learning-based optimization

*Corresponding author. E-mail address: [email protected]

1. Introduction The teaching-learning-based optimization (TLBO) algorithm [1, 2] is a population-based algorithm which simulates the influence of a teacher on the output of learners in a class. The algorithm requires only the common control parameters and does not require any algorithm-specific control parameters. As it has been empirically shown to perform well on many optimization problems, TLBO has emerged as one of the simplest and efficient techniques. The research works for TLBO can generally divided into two kinds, the one is to extend the application fields of it, and the other is to improve its global performance with different evolutionary operators. In the first class, Elitism concept in EAs is introduced to improve the convergence performance of TLBO algorithm [3] and ETLBO has been used for multi-level production planning in a petrochemical industry [4]. The optimal power flow (OPF) problem is solved by using multi-objective teaching-learning-based optimization algorithm, and the results indicate that the algorithm is able to produce true and well distributed Pareto optimal solutions [5]. TLBO is proposed to sole the clustering problem [6], and the clustering ability of it is tested on some well-known datasets. Discrete optimization of planar steel frames is optimized by TLBO [7], and the results of it are compared to those of other algorithms. In [8], TLBO is used to solve high dimensional function optimization problems. In [9], TLBO is used for clustering problems. In this method, TLBO is used to find the near-optimal cluster centers and then the cluster centers found by TLBO are evaluated using reformulated C-means objective function. More than one teacher and adaptive teacher factor methods are used to modify the TLBO, and the modified algorithm is used for multi-objective optimization of heat exchangers [10]. TLBO is modified and used for dynamic economic emission dispatch [11]. To optimize large number of input parameters of manufacturing industries, TLBO has been used for the process parameter optimization of selected modern machining processes [12]. In addition, TLBO is used in some engineering problems [13-20]. In the second class, the concept of number of teachers, adaptive teaching factor, tutorial training and self-motivated learning are introduced in TLBO algorithm to improve its performance [21]. A weighted teaching-learning-based optimization (WTLBO) [22] which uses a parameter in TLBO algorithm to increase convergence rate, the equation in teacher phase is modified. A modified teaching factor and mutation operator are introduced into TLBO to adjust the convergence speed and avoid premature convergence of the original TLBO [23]. Orthogonal design with a new selection strategy is applied to decrease the number of generations and make the algorithm converge faster [24]. Producer–scrounger model is proposed by our research group to decrease the computation cost of TLBO [25]. The idea of group discussion is introduced into TLBO to improve the performance of TLBO by the group leaders [26]. Artificial bee colony algorithm (ABC) is combined with TLBO to predict the berm geometry with a set of laboratory

tests [27]. Double differential evolution (DDE) algorithm is combined with teaching learning algorithm is used to handle the ORPD problem [28]. Neighborhood search is introduced into TLBO for applications of ANN [29]. Variable neighborhood search is introduced into TLBO to improve the global performance of it and it is used for permutation flow shop scheduling problem [30]. A scale factor is used to modify the updating equation of learners, and the improved algorithm is evaluated by testing for benchmark functions [31]. Dynamic group strategy, where each learner learns knowledge from the mean of his corresponding group, is introduced in TLBO to improve the performance of it for global optimization problems [32]. A bi-phase crossover scheme and special local search operators are incorporated into the TLBO to balance the exploration and exploitation capabilities [33]. Some experiments have proved that large number of population might be unhelpful in evolutionary algorithms [34], and the computation cost was increased, a small number of it might make local convergence of algorithm. Variable population size is a simple technology to adjust the computation cost of EAs, the method is easy to be realized. Saw-tooth GA [35] is proposed to decrease the computation cost of GA, the population in each period is linear decreased from the maximal value to the minimal value, and the population size is directly increased to maximal value at the end of every period. The computation cost is determined by the average size of population. A variable-population evolutionary game model is used to allocate the resource for cooperative cognitive relay networks [36]. An adaptive population tuning scheme (APTS) for differential evolution is proposed to dynamically adjust the population size [37]. In this method, the redundant individuals are removed from the population according to their ranking order and the new individuals are also generated from them. The dynamic population size is used in artificial bee colony algorithm to solve the combined economic and emission dispatch problem [38]. The lifetime and extinction methods are used in DE to adjust the population size of the algorithm. The population size is adjusted according to the online progress of fitness improvement [39]. The population variation (PV) scheme, where the population could be increased and/or decreased with a variable profile, where the increment or reduction of population size could take on any flexible profile such as linear, exponential, hyperbolic, sinusoidal or even random, has been introduced in [40]. To improve the performance of PV, a dynamic population variation (DPV) algorithm has been proposed in genetic programming [41]. Adaptive elitist-population search method combining with GA has been introduced to solve multimodal function optimization [42]. Dynamic population multiple-swarm (MOPSO) algorithm [43] is used to solve multi-objective problem, the computation cost is decreased. A particle swarm optimization (PSO) that uses an adaptive variable population size in the form of ladder is proposed by our group [44]. In most variable-population based method, randomly reinitialize some new individuals is often adapted in the increasing phase of population size. The method might make the average

fitness of population shake severely. To decrease the computation cost at each iteration and make the evolutionary process smooth, a variable population size in the form of a triangle form is used to improve the performance of TLBO. The population size of TLBO linearly vary from the minimal value to the maximal value and then from the maximal value to the minimal value. The new individuals are generated with gauss distribution with adaptive mean and variance in the increasing phase. Some individuals with high similarity are removed from the current population in the decreasing phase of population. The saw-tooth TLBO is also designed in the paper, the comparison study of it with the proposed algorithm is also given. Moreover, The proposed algorithm is test on some benchmark functions with compare to some other EAs. The results indicated that the given algorithm is also a challenge improved algorithm for TLBO. The rest of the paper is organized as follows. Section 2 describes the main aspects of standard TLBO. TLBO with a variable population size in the form of a triangle is described in Section 3. Functions optimization problems and some simulation experiments are shown in Section 4, and in Section 5, the parameters of ANN are optimized by VTTLBO. Some conclusions are given in Section 6.

2. The standard teaching-learning-based optimization algorithm (STLBO) The TLBO is an efficient population based evolutionary computation algorithm, it mimics the teaching-learning relation in a class. The students improve their grades through learning from the teacher and other students. The best student in the current generation is chosen as the teacher, the teacher shares his or her knowledge with the learners in the class, and the quality of teacher affects the outcome of the learners. In addition, the learners also learn knowledge from other learners. The TLBO algorithm is composed of two main phases (teacher phase and learner phase) [1]. The two main phases are described as follow.

2.1 Teacher phase In teacher phase, the learners learn knowledge from the teacher of current generation. During this phase, the teacher distributes his or her knowledge for all learners in the class and makes an effort to increase the mean grade of the class. For a n dimension optimization problem, assume that the solution of i-th learner is X i  {xi ,1 , xi ,2 , M k  {m1 , m2 ,

, xi ,2 } , the mean solution of the class is

, mn } ，the best solution in the current generation is X teacher . The goal of the

teacher is to improve the mean solution M k . The teacher often improves the mean solution of the class from M k to any other value M k' which is better than it, which depending on his or her capability. The new solution of i-th learner is shown in Eq. 1. X new,k ,i  X old ,k ,i  rand (.)( X teacher ,k  TF M k )

(1)

Where, X new,k ,i and X old ,k ,i are the new and old positions of the i-th learner in the k-th generation, X teacher ,k is the best solution of the class in the k-th generation, rand(.) is the random number in the range [0,1]. TF is the teaching factor which decides the value of the mean to be changed. The value of TF can be either 1 or 2. The TF is decided randomly with equal probability as shown in Eq. 2 [1]. TF  round[1  rand (0,1){2  1}]

(2)

2.2 Leaner phase The second part of TLBO is the leaner phase. In this phase, the learners increase their knowledge by interaction with another randomly selected learner in the class. A learner learns new knowledge if the randomly selected learner is better than him or her. The learner phase of the i-th learner is mathematically expressed as follow. For minimal optimization problem, randomly select another learner j from the class at any iteration k, where i  j . X new,i ,k  X old ,i ,k  rand (.)( X old ,i ,k  X old , j ,k )

if f ( X old ,i ,k )  f ( X old , j ,k )

(3)

X new,i ,k  X old ,i ,k  rand (.)( X old , j ,k  X old ,i ,k )

if f ( X old , j ,k )  f ( X old ,i ,k )

(4)

Accept X new,i ,k if it gives better function value. Where, X new,i ,k is the new position of the

i-th learner in the k-th generation, X old ,i ,k and X old , j ,k are the old positions of the i-th and the j-th learners. In original TLBO, duplicate elimination process is used to increase the diversity of the class. The experimental results indicate that non-removal of duplicate solutions does not always affect the final solution [10]. For the function evaluations (FEs) in duplicate elimination process is not clearly, the process in this paper is not adopted. The number of function evaluations of the algorithm will change from {(2  population size  number of generations)+(FEs needed for duplicate elimination)} (in STLBO) to {2  average population size  number of generations} (VTTLBO). The computation cost is decreased when the maximum generation is the same. The pseudo-code for the implementation of TLBO is summarized in Algorithm 1. Algorithm 1: TLBO( ) Begin Initialize the population size N and the dimension D of the give problem Initialize the population X and evaluate them Save the best learner as Teacher and calculate the mean of all learners while the ended condition is not satisfied for each learner xi of the class in the k-th generation TF = round(1+rand(0,1)) for j = 1:D

xnew,k ,i ( j )  xold ,k ,i ( j )  rand (.)( xteacher ,k ( j )  TF M k ( j )) end for Accept the new position if it is better than the old one end for for each learner xi of the class in the k-th generation Randomly select another learner Xl , such that i≠l if f(Xi ) is better than f(Xl ) for j = 1:D

X new,i ,k ( j )  X old ,i ,k ( j )  rand (.)( X old ,i ,k ( j )  X old ,l ,k ( j )) end for else for j = 1:D

X new,i ,k ( j )  X old ,i ,k ( j )  rand (.)( X old ,l ,k ( j )  X old ,i ,k ( j )) end for end if Accept the new position if it is better than the old one end for Update the Teacher and the mean solution of the class end while end

3 TLBO with a variable population size in the form of a triangle (VTTLBO) 3.1 Algorithm overview In the saw-tooth GA and the Ladder EAs (variable population size in the form of ladder), the mean of fitness of the population might severely shake for randomly increasing a lot of individuals when the population size changes from a small number to a large number. This phenomenon can be explained as follow. The sketch maps of the variable population size in saw-tooth EAs and Ladder EAs are shown in Fig. 1 and Fig. 2. Population size

Population size

N max

N max

…

N av N min

…

N av N min Generation T

2T

kT

(k+1)T

Fig.1 the sketch map of saw-tooth EAs

Generation T

2T

kT

(k+1)T

Fig.2 the sketch map of Ladder EAs

In Fig. 1 and Fig. 2, if the population size should be increased in any end of the periods, such as T, 2T, kT, (k+1)T in the Fig. 1 and T, kT in the Fig. 2, some new individuals will be randomly reinitialized in a given range, the randomly increased individuals will make the average fitness of the population severely shake. Though the crossover operator is used in population size increased phase of ladder EAs, the distribution character of the increased individual is not considered. The algorithm with the variable population size in the form of a triangle is proposed to solve these problems. The size of the population varied from the minimal value to the maximum value and then from the maximum value to the minimal value in a period, the number of increased individual is not suddenly changed compared to saw-tooth and ladder EAs, and the increased individuals are generated according to the gauss distribution with considering the mean and variance of current population. This method will be introduced in the part of individuals increased phase.

3.2 VTTLBO algorithm 3.2.1 The basic framework of the VTTLBO The basic framework of the VTTLBO is described as follow. Step 1: Randomly initialize the population. Step 2: Divide the maximum generation into some periodic. Step 3: Increase the population size from the minimal value to the maximal value and execute TLBO operators. Step 4: Decrease the population size from the maximal value to the minimal value of the population size and execute TLBO operators. Step 5: If the ended condition is satisfied, output the best solutions, else go to Step 3. 3.2.2 The main aspects of the VTTLBO A. The variable population size in the form of triangle First of all, the evolutionary process of the VTTLBO is divided into R periodic, each periodic has the same evolutionary generation T. The sketch map of variable population size in the form of triangle is shown in Fig. 3. Population size

N max N av N min

… Generation

 T

2T

kT

(k+1)T

Fig.3 the sketch map of VTTLBO algorithm

In Fig. 3, Nmax , Nmin , Nav are the maximal, the minimal and the average number of the population, respectively. The average number of population N av is determined by the parameters T and  . The relation among the maximal evolutionary generations maxgen, R and T is shown in Eq. 5. R  max gen / T

(5)

The parameter  will affect the FES of the algorithm, if the value of  is large, the time of increase process will be long, if the value of  is small, the time of decrease process will be long. If  equals to zero, the variable population size of the algorithm will in saw-tooth function. To simplify the program,  is chosen as 0.5T in this paper, the form of the variable population size is in the form of isosceles triangle. The average population size of VTTLBO is shown in Eq. 6.



N av  R[ floor (  ( gen 1

gen *( N max  N min )



 N min ) 

T



gen  1

(

(T  gen)*( N max  N min )  N min ))] / max gen T  (6)

B. Population increasing strategy Some variable population size EAs increase the population number based on the diversity of the current population, if the diversity of the current population is bad, randomly initialize a certain number of new individuals and add them in the population of next generation. The diversity of population should be calculated in every generation. The computation cost of the algorithm is increased. Moreover, accurately determine the diversity of population is also a difficult task. In addition, randomly generate new individuals for the population in the next generation is often adopted in the traditional method, the method has some blindness, the distribution of the new individuals is not considered, and the average fitness of the population might shake. In VTTLBO, the process of calculating the diversity is removed. To simplify the increasing process and control the distribution of the new individuals, the new individuals generated in VTTLBO are in gauss distribution with a designed mean and variance. The process is introduced in the D part. For a D-dimension problem, the population increasing process is described as Algorithm 2.

Algorithm 2: Increased( ) For k=1:R for gen = (k-1)* T+1:k*T w = wmin +(gen-(k-1)* T)*( wmax - wmin )/T; if (gen>=(k-1)* T+1)&&(gen<=(k-1)* T+ ) VariPopSize = N min +floor(( N max - N min )*(gen-(k-1)* T)/ ); if VariPopSize <= N max newPopSize = VariPopSize; for i = PopSize(gen-1)+1:newPopSize for j = 1:D Pop(i,j) = normrnd(w*(Teacher(1,j)+MeanPop(1,j))/2,w*abs(Teacher(1,j)-MeanPop(1,j))); end Evaluate the increased individuals; end Pop = Pop(1:newPopSize,1:opt.DimSize+1);/*form the new population*/ end PopSize(gen)=newPopSize; End End End

In Algorithm 2, VariPopSize is the population size of current generation, w is the adaptive factor for the increasing process. C. Population decreasing strategy In the conventional methods, all the individuals are sorted according to their fitness value from high to low, and some individuals at the latter of the alignment will be removed from the current generation. This might not benefit for evolution because the individuals with bad fitness might play more important role than individuals with good fitness in some generations. In VTTLBO, the individuals are sorted according to the similarity, and parts of similar individuals are deleted in every periodic. The teacher is saved in every generation to save the best solution of the algorithm. The specific decreasing process is described in Algorithm 3.

Algorithm 3: Decreased( ) For k=1:R for gen = (k-1)* T+1:k*T if (gen>(k-1)* T+ )&&(gen<=k*T) VariPopSize = N max -floor(( N max - N min )*(gen-(K-1)* T- )/ ); if VariPopSize >= minN newPopSize = VariPopSize; end dis = []; for i = 1: PopSize(gen-1) dis(i) = sqrt(sum((Pop(i,1:D) - MeanPop(1,1:D)).^2)); end [p,q] = sort(dis,'descend'); /* sorting individuals according to similarity*/ Pop = Pop(q(1:newPopSize-1),1:opt.DimSize+1); Pop = [Pop;Teacher]; PopSize(gen) = newPopSize; End End End

In Algorithm 3, dis(i) is the similarity of the i-th individual. It is calculated as follow.

dis(i) 

D

 ( Pop(i, j)  MeanPop(1, j))

2

(7)

j 1

D. Generate new individuals and adaptive parameter w In the bare bones PSO algorithm [45], the social communication is maintained but the dynamical particle is updated with sampling from a probability distribution with considering the gbest (the best solution in current generation) and pbesti (the best position which the i-th particle has been achieved so far), the method is shown in equation 8. gbest  pbesti , j (t ) xi , j (t  1)  N ( , gbest  pbesti , j (t ) ) 2

(8)

Where, xi , j (t  1) is the j-th dimension of the i-th particle in the population and N(.) represents a Gaussian distribution with a designed mean and standard deviation. Based on the above idea, the gauss distribution with a designed mean and variance is used to generate the new individuals for the population in VTTLBO. The method is described in equation 9.

X (i, j )  Normrnd (( X teacher (1, j )  X mean (1, j )) / 2, X teacher (1, j )  w * X mean (1, j ) )

(9)

In TLBO, the goal of teacher is to improve the average grade of the class. In the evolution of VTTLBO, the mean of the class will gradually approach to the teacher. The first part of Eq. 9 indicates that the mean of increased individual will gradually close to the teacher with the development of evolution. The second part of Eq. 9 is used to adjust the distribution of the increased individual. The generated new individuals with small variance will not benefit for getting away from the local optima of the algorithm. To improve the global performance of the

algorithm, a factor w is used to adjust the variance of the new individuals. The next task is to determine the parameter w. For the distance between the mean position and teacher is decreased with the increasing of generation, the mean position X mean (1, j ) might equal the position of the teacher X teacher (1, j ) especially in the latter of the evolution, if there is no factor w in the equation or w equals 1, the variance of the new individuals will become zero, the distribution of the added individuals will become bad, the convergence character of the algorithm will difficult to improve. In general, the distribution of population is good in the beginning of the evolution, and it is bad in the latter of the evolution. In VTTLBO, w is changed from one to a small value, the minimal value (wmin ) of it is chosen with trail and error method. The adaptive parameter is shown in equation 10. w = wmax -(gen-(k-1)* T)*(wmax - wmin)/T

(10)

where, the wmin and wmax is the low and up boundaries of w. In general, wmax equals to one. The wmin is chosen with trial and error method. For example, the 30 dimensional function f18 are simulated 30 independent runs with wmin changing from 0.01 to 0.2, the best average solutions are shown in Fig. 4. In Fig. 4, when wmax equals to 0.05, the best average fitness is the smallest. In the paper, wmax is selected as 0.05 in the experiments. This might not the precise solution for some other problems. 5400 5300

The best average fitness

5200 5100 5000 4900 4800 4700 4600 4500

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

wmin

Fig.4 the best average fitness for function f18 with wmin changed from 0.01 to 0.2

E. The pseudocode of the VTTLBO Synthesizing the previous analysis, the VTTLBO is described in Algorithm 4. Algorithm 4: ( ) Begin: Initialize the parameters: T, R, N max , N min , wmax , wmin , D, Xmax, Xmin. Initialize the population: Pop= Xmin+ rand* (Xmax - Xmin), PopSize =Nmin. Evaluate the population and save the best solution as Xteacher, and calculate the mean of the learners. While the ended condition is not satisfied For k=1:R for gen = (k-1)* T+1:k*T w = wmin +(gen-(k-1)* T)*( wmax - wmin )/T for i=1:popsize TF=round(1+rand(1)). Execute the teacher phase according to equation 1. Accept the good solution for the next generation. End for i=1:popsize Execute the learner phase according to equations 3 and 4. Accept the good solution for the next generation. End if (gen>=(k-1)* T+1)&&(gen<=(k-1)*T+ ) Execute the increasing strategy. Else Execute the decreasing strategy. End End End End

4 Function optimization problems To illustrate the effectiveness of the proposed method, 18 benchmark functions are used to test the efficiency of VTTLBO. To compare the performance of VTTLBO with some other methods, the other 9 algorithms are simulated and the results are compared in the paper. 4.1 Benchmark functions The details of 18 benchmark functions are shown in Table 1. Among 18 benchmark functions, f1 to f5 are unimodal functions, and f6 to f10 are multimodal functions, and the others are the rotation functions. The searching range, the theory optima and the acceptable solutions for all functions are shown in the table.

Table 1 Details of numerical benchmarks used Function

Formula

Range

f1 (x)  i 1 xi2 D

f1(Sphere)

[-100,100]

min

0

f 2 (x)  i 1 ( j 1 x j )2

[-100,100]

f3(Sum Square)

f3 (x)  i 1 ixi2

ccepta1

[-100,100]

1

i

0 e-6

D

1

0 e-6

f4 (Zakharov)

4 f 4 (x)  i 1 xi2  (i 1 0.5ixi )2  (i 1 0.5ix[-10,10] i)

0

f5(Rosenbrock)

2 D 1 2 f5 (x)  i 1 100  xi2  xi 1    xi  1   

0

f6(Ackley)

f 6 (x)  20  20exp ( 

D

D

D

1 e-6

[-2.048,2.0

0

48]

.1

1 1 1 D 2 1[-32.768,3 i 1 xi )-exp ( iD1 cos (2 xi0))  e 2.768]D e-6 5 D [-5.12,5.12

f7 (x)  iD1 (xi2  10cos (2 xi )  10)

f7(Rastrigin)

A

e-6 nce

f2(Quadric)

D

f

2 0

]

.5

f8 (x)   i 1 ( k 0  a k cos (2 b k (xi  0.5)) )  D

k max

f8(Weierstrass)

D k 0  a cos (2 b  0.5)  a  0.5 b  3 k max  20

f9 (Griewank)

f 9 ( x) 

f10(Schwefel’s)

f10 (x)  418.9829 D   xi sin

k max

k

[-0.5,0.5]

k

1 0 e-6

x 1 D 2 D xi   cos( i )  1  4000 i 1 i i 1

0 [-600,600]

0 .1

D

i 1

f11(Rotated

Sum

Square) f12(Rotated Zakharov) f13(Rotated Rosenbrock)

f14(Rotated Ackley)

5

xi

[-500,500]

00

f11 (y)  i 1 iyi2 yi  M  xi

1

D

[-100,100]

D

0 e-6

4 [-10,10] f12 (y )   i 1 yi2  ( i 1 0.5iyi ) 2  ( i 1 0.5iy i) D

0

D

1

0 e-6

yi  M  xi D 1 2 [-2.048,2.0 1 2 f13 (y )   i 1 100  yi2  yi 1    yi  1  0  48] .5 yi  M  xi 1 1 D 2 1 D f14 (y )  20  20exp (   i 1 yi )-exp ( [-32.768,3  i 1 cos (2 yi ))  e1 5 D D 0 2.768] e-6 yi  M  xi

f15(Rotated

f15 (y)  iD1 (yi2  10cos (2 yi )  10) yi  M[-5.12,5.12  xi

Rastrigin)

f16 (y )   i 1 ( k 0  a k cos (2 b k (yi  0.5)) )  D

f16(Rotated Weierstrass)

f17(Rotated Griewank)

Schwefel’s)

5

k max

[-0.5,0.5]

D k 0  a k cos (2 b k  0.5)  a  0.5 b D 3 k max  20 yi  M  xi D y 1 2 f17 ( y )  yi   cos( i )  1 yi [-600,600] M  xi  4000 i 1 i i 1 k max

D

f18(Rotated

0

]

f18 (x)  418.9829 D   xi sin i 1

xi

yi  M  xi [-500,500]

1 0 e-6

0 0 .01

2 0 40

4.2 Parameter settings All the experiments are carried out on the same machine with a Celoron 2.26 GHz CPU, 1GB memory, and windows XP operating system with Matlab 7.2. For the purpose of reducing statistical errors, each algorithm is independently simulated 30 runs. The maximal number of population size N max of VTTLBO and SawTLBO is 30, and the minimal number N min of them is 10 for 10 dimensional functions. The maximal number of population size N max of VTTLBO and SawTLBO is 40, and the minimal number of population size N min of them is 20 for 30 dimensional functions. For all other algorithms, the population size is 30 for 10 dimensional functions, and 40 for 30 dimensional functions. The other parameters for all the other algorithms are the same as them in the references. (DE [39], ABC [46],jDE [47], SaDE [48], PSOwFIPS [49] CLPSO [50], SawTLBO, ETLBO[3] (the number of elites is 2), and TLBO [1]). The maximal function evaluations (FES) 50000 are used as the stopping criterion for all algorithms. 4.3 Results and Comparisons 4.3.1 Comparisons on the Solution Accuracy The performance of different algorithms in terms of the mean and standard deviation (Std) of the solutions obtained in the 30 independent runs for 10 and 30 dimensional functions are listed in Table 2 and 3, respectively. Boldface in the tables indicates the best result among those obtained by all ten contenders. Fig. 5 graphically presents the comparison in terms of convergence characteristics for solving the 18 different functions with 30 dimensions.

Table 2 The mean solutions and standard deviation of the 30 trials obtained by various method for 10 dimensional functions F f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18

Perf. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std.

DE 7.13e-073 7.18e-073 4.18e-012 9.35e-012 2.47e-074 4.86e-074 1.08e-005 2.41e-005 6.24e+000 9.02e-001 3.48e-015 2.37e-016 1.46e+000 5.02e-001 0.00e+000 0.00e+000 3.50e-002 2.22e-002 2.18e+002 1.96e+002 .66e-070 1.10e-069 3.99e-038 6.82e-038 5.83e+000 1.38e+000 3.31e-015 4.25e-016 4.15e+000 2.69e+000 1.18e-001 2.64e-001 2.41e-002 2.08e-002 2.57e+002 1.27e+002

jDE 1.31e-076 1.58e-076 1.14e-021 1.52e-021 6.97e-078 1.39e-077 1.31e-031 1.30e-031 5.14e-007 9.47e-007 3.36e-015 4.27e-016 0.00e+000 0.00e+000 0.00e+000 0.00e+000 0.00e+000 0.00e+000 1.27e-004 0.00e+000 5.63e-058 1.26e-057 7.13e-031 1.31e-030 3.61e-002 7.94e-002 2.84e-015 1.59e-015 3.59e+000 1.84e+000 9.27e-002 2.07e-001 1.40e-002 9.09e-003 1.27e-004 0.00e+000

SaDE 1.35e-071 2.02e-071 1.89e-019 3.54e-019 1.28e-074 2.52e-074 6.65e-031 1.48e-030 2.62e+000 1.50e+000 3.28e-015 2.51e-016 0.00e+000 0.00e+000 0.00e+000 0.00e+000 1.48e-003 3.31e-003 1.27e-004 0.00e+000 3.32e-046 7.42e-046 6.80e-036 5.81e-036 2.08e+000 1.82e+000 3.21e-015 2.38e-016 5.78e+000 1.85e+000 0.00e+000 0.00e+000 6.90e-003 6.83e-003 3.59e+002 3.54e+002

PSOwFIPS 3.98e-016 6.09e-016 6.19e-006 2.15e-006 2.18e-017 1.39e-017 3.23e-009 2.23e-009 4.51e+000 7.17e-002 8.04e-009 4.33e-009 1.89e+000 1.03e+000 4.24e-004 6.60e-004 7.59e-002 5.19e-002 3.69e-002 3.41e-002 8.42e-015 8.03e-015 2.88e-009 9.64e-010 4.55e+000 5.29e-001 9.98e-009 2.82e-009 9.31e+000 1.96e+000 2.55e-002 2.95e-002 1.02e-001 3.85e-002 2.81e+002 1.91e+002

CLPSO 1.09e-018 1.50e-018 5.37e-001 1.38e-001 2.59e-020 1.76e-020 2.66e-003 2.37e-003 2.45e+000 1.00e+000 4.28e-010 2.89e-010 2.76e-009 3.94e-009 6.33e-012 6.08e-012 4.15e-003 5.68e-003 1.27e-004 1.22e-009 7.05e-011 1.55e-010 1.78e-003 1.31e-003 7.25e+000 6.39e+000 1.68e-007 2.04e-007 6.11e+000 3.00e+000 1.44e-001 1.21e-001 2.73e-002 2.35e-002 2.32e+002 1.04e+002

ABC 8.02e-017 3.22e-017 4.04e+001 2.35e+001 7.18e-017 3.94e-017 1.31e+001 8.38e+000 2.84e-001 3.23e-001 8.53e-015 3.18e-015 0.00e+000 0.00e+000 0.00e+000 0.00e+000 3.94e-003 5.67e-003 1.27e-004 4.98e-013 5.50e-010 1.23e-009 1.72e+001 6.06e+000 5.89e-001 8.58e-001 5.83e-005 9.40e-005 1.33e+001 7.00e+000 1.69e+000 6.25e-001 9.56e-003 1.12e-002 3.47e+002 2.62e+002

TLBO 3.29e-184 3.08e-185 2.56e-082 5.58e-082 9.94e-187 1.25e-187 1.51e-089 1.62e-089 4.96e-001 4.21e-001 3.43e-015 2.17e-015 3.06e+000 1.52e+000 0.00e+000 0.00e+000 6.48e-003 9.71e-003 6.68e+002 1.51e+002 1.23e-173 2.37e-174 4.18e-089 6.95e-089 2.50e+000 2.33e+000 3.52e-015 6.05e-016 4.38e+000 8.91e-001 0.00e+000 0.00e+000 2.81e-003 6.29e-003 5.58e+002 2.76e+002

ETLBO 2.84e-166 4.27e-167 3.22e-079 5.07e-079 6.50e-169 5.49e-170 2.94e-087 3.10e-087 1.46e-001 1.38e-001 3.37e-015 1.05e-015 3.02e+000 1.86e+000 0.00e+000 0.00e+000 2.42e-002 3.69e-002 7.03e+002 1.81e+002 2.21e-161 3.04e-161 2.18e-086 4.82e-086 2.63e+000 2.87e+000 3.49e-015 3.27e-016 3.00e+000 1.22e+000 0.00e+000 0.00e+000 6.77e-003 5.15e-003 5.69e+002 2.63e+002

sawTLBO 3.01e-064 4.86e-064 4.59e-050 1.12e-049 4.86e-067 1.54e-066 6.87e-053 1.61e-052 1.90e+000 5.28e-001 2.84e-015 1.50e-015 8.57e+000 3.23e+000 0.00e+000 0.00e+000 1.33e-002 2.22e-002 8.25e+002 1.22e+002 5.20e-065 1.62e-064 4.89e-061 8.08e-061 2.38e+000 2.03e+000 3.20e-015 1.12e-015 1.34e+001 4.28e+000 0.00e+000 0.00e+000 1.91e-002 2.92e-002 9.71e+002 4.48e+002

VTTLBO 3.56e-296 0.00e+000 3.50e-130 5.05e-130 4.63e-298 0.00e+000 1.02e-139 3.21e-139 1.13e+000 5.06e-001 1.78e-015 1.87e-015 1.09e+000 1.52e+000 0.00e+000 0.00e+000 3.82e-008 1.21e-007 7.92e+002 1.90e+002 3.09e-279 0.00e+000 1.50e-138 4.75e-138 5.02e+000 2.53e+000 2.13e-015 1.83e-015 2.55e+000 2.14e+000 0.00e+000 0.00e+000 5.40e-010 1.70e-009 7.77e+002 3.05e+002

Table 3 The mean solutions and standard deviation of the 30 trials obtained by various method for 30 dimensional functions F f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17

Perf. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std.

DE 4.90e-014 1.08e-013 4.14e+000 3.22e+000 7.99e-017 8.83e-017 8.79e-001 6.15e-001 2.54e+001 5.86e-001 7.49e-009 5.58e-009 7.75e+001 3.01e+001 1.20e-002 2.67e-002 3.94e-003 5.40e-003 3.33e+003 1.56e+003 4.45e-005 6.40e-005 6.62e-001 2.77e-001 5.14e+001 3.03e+001 3.31e-008 3.18e-008 1.28e+002 6.97e+001 4.17e-001 6.25e-001 1.97e-003 4.41e-003

jDE 1.95e-022 2.76e-022 2.06e+001 6.71e+000 3.92e-023 3.86e-023 1.35e+000 1.68e+000 2.18e+001 2.59e-001 2.82e-012 1.77e-012 2.24e-009 4.15e-009 3.16e-001 5.07e-001 0.00e+000 0.00e+000 2.37e+001 5.30e+001 2.57e-006 5.75e-006 6.79e-001 6.36e-001 2.42e+001 1.03e+000 7.24e-012 3.21e-012 6.18e+001 1.29e+001 2.20e+000 4.19e+000 2.46e-003 5.51e-003

SaDE 3.84e-023 2.15e-023 1.06e+001 6.53e+000 3.00e-024 2.47e-024 1.50e-001 1.31e-001 2.52e+001 1.36e+000 1.30e-012 8.26e-013 5.53e-001 7.55e-001 6.56e-011 8.87e-011 0.00e+000 0.00e+000 3.82e-004 1.99e-010 1.21e-004 2.54e-004 1.51e-001 6.29e-002 2.74e+001 2.04e+000 1.36e-012 9.25e-013 9.63e+001 1.50e+001 9.96e-003 2.18e-002 7.98e-013 1.26e-012

PSOwFIPS 1.43e+000 2.78e-001 3.82e+003 1.01e+003 2.17e-001 7.69e-002 1.11e+002 1.89e+001 2.73e+001 2.99e-001 5.21e-001 1.78e-001 1.23e+002 1.57e+001 2.46e+000 7.11e-001 8.92e-001 5.53e-002 4.99e+003 5.50e+002 7.91e-001 6.74e-001 1.13e+002 1.65e+001 2.83e+001 5.57e-001 5.96e-001 1.55e-001 1.52e+002 1.53e+001 5.03e+000 6.56e-001 9.34e-001 6.89e-002

CLPSO 1.94e-001 7.79e-002 1.16e+004 2.72e+003 2.25e-002 6.43e-003 2.30e+002 5.45e+001 6.84e+001 2.84e+001 7.13e-001 5.74e-001 2.91e+001 4.77e+000 5.18e-001 6.83e-002 2.68e-001 4.38e-002 1.72e+003 3.21e+002 1.37e+000 5.16e-001 2.27e+002 1.60e+001 5.61e+001 1.63e+001 2.68e+000 1.40e-001 1.10e+002 1.14e+001 1.38e+001 1.21e+000 3.64e-001 1.05e-001

ABC 2.45e-006 1.21e-006 1.52e+004 2.77e+003 6.88e-007 3.24e-007 5.48e+002 6.58e+001 2.16e+001 4.59e+000 9.08e-003 4.76e-003 4.22e+000 5.91e-001 3.66e-002 8.53e-003 3.32e-003 6.99e-003 5.93e+002 1.45e+002 3.65e-002 2.22e-002 5.27e+002 6.97e+001 4.92e+001 2.60e+001 2.24e+000 4.15e-001 7.67e+001 1.04e+001 1.18e+001 9.05e-001 1.62e-003 1.89e-003

TLBO 4.04e-111 3.20e-111 1.08e-022 1.43e-022 5.38e-111 3.43e-111 5.11e-011 8.59e-011 2.38e+001 7.01e-001 3.55e-015 0.00e+000 1.17e+001 3.71e+000 0.00e+000 0.00e+000 0.00e+000 0.00e+000 4.29e+003 9.40e+002 1.56e-106 3.33e-106 3.24e-012 3.24e-012 5.10e+001 2.82e+001 3.55e-015 0.00e+000 2.17e+001 1.64e+001 0.00e+000 0.00e+000 1.15e-003 2.57e-003

ETLBO 2.66e-095 1.84e-095 3.42e-022 4.72e-022 8.21e-096 1.11e-095 1.92e-011 1.93e-011 2.38e+001 8.57e-001 3.55e-015 0.00e+000 1.22e+001 9.07e+000 0.00e+000 0.00e+000 0.00e+000 0.00e+000 4.55e+003 7.81e+002 1.25e-092 1.37e-092 6.28e-011 6.35e-011 5.50e+001 2.95e+001 3.55e-015 0.00e+000 1.36e+001 7.85e+000 0.00e+000 0.00e+000 0.00e+000 0.00e+000

sawTLBO 2.61e-065 3.09e-065 1.68e-026 1.78e-026 6.13e-066 1.12e-065 1.23e-008 2.54e-008 2.36e+001 8.68e-001 3.55e-015 0.00e+000 2.36e+001 6.04e+000 0.00e+000 0.00e+000 0.00e+000 0.00e+000 5.26e+003 6.41e+002 7.57e-065 1.08e-064 2.89e-008 5.42e-008 5.52e+001 3.42e+001 3.55e-015 0.00e+000 4.09e+001 1.15e+001 0.00e+000 0.00e+000 0.00e+000 0.00e+000

VTTLBO 4.85e-158 1.06e-157 1.24e-032 1.79e-032 2.50e-158 2.48e-158 8.06e-018 1.11e-017 2.28e+001 4.23e-001 3.55e-015 0.00e+000 1.25e+001 7.27e+000 0.00e+000 0.00e+000 0.00e+000 0.00e+000 4.70e+003 4.22e+002 2.58e-153 5.31e-153 1.99e-017 4.29e-017 7.05e+001 2.56e+001 3.55e-015 0.00e+000 1.41e+001 5.55e+000 0.00e+000 0.00e+000 0.00e+000 0.00e+000

1.67e+003 8.23e+002

3.49e+003 2.98e+002

4.66e+003 6.74e+0 02

5.86e+003 4.95e+002

4.34e+003 2.33e+002

3.96e+003 7.03e+002

4.53e+003 6.01e+002

4.78e+003 1.09e+003

4.58e+003 5.53e+002

4.70e+003 1.04e+003

f18

Table 2 and table 3 display that VTTLBO outperforms some other algorithms in terms of the mean and standard deviation for some functions. Table 2 displays that VTTLBO has better performance in terms of the mean and standard deviation than all other algorithms for functions

f1 , f 2 , f 3 , f 4 , f 6 , f11 , f12 , f14 , f15 and f17 . jDE has the smallest mean and Std for functions f 5 , f 9 , f13 and f18 . For function f 7 , the means and Stds of jDE, SaDE and ABC are zero. jDE, SaDE, CLPSO and ABC have the same smallest mean for function f10 , and the Stds of jDE and SaDE are zero. The means and standard deviations of variant DEs, TLBOs and ABC are zero for function f8 . For function f16 , SaDE and variant TLBOs have the smallest means and Stds. Among variant TLBOs, the performance of ETLBO is better than other TLBOs for function f 5 . The mean

of VTTLBO for function f 7 is better than other TLBOs, and the standard deviations of TLBO and VTTLBO are the same. The performance of VTTLBO is better than the other TLBOs for function f 9 . The performance in terms of the mean and the standard deviation of original TLBO for functions f10 , f13 and f18 is the best among variant TLBOs. Table 3 indicates that the mean and standard deviation of VTTLBO is better than all other algorithms for functions f1 , f 2 , f 3 , f 4 ,

f11 and f12 , and it can converge to global optimization for functions f8 , f 9 , f16 and f17 . For function f 5 , ABC has the smallest mean, jDE has the smallest standard deviation, and the performance of VTTLBO is the best among variant TLBOs. The performance of variant TLBOs is the same and it is better than other algorithms for function f 6 . For function f 7 , SaDE has the best performance and original TLBO has the best performance among all variant TLBOs, the same results are also derived for function f10 except that the standard deviation of VTTLBO is the smallest among variant TLBOs. jDE outperforms all the other algorithms for function f13 . The performance of ETLBO is the best among all algorithms for function f15 , and DE is the best among all algorithms for function f18. As previous analysis, the VTTLBO outperforms a majority of algorithms in terms of solution accuracy. Fig. 5 also shows the convergence process of different algorithm for the eighteen benchmark functions. The figurer indicates that VTTLBO offers the best performance on most tested functions. Table 2 and table 3 also indicate that VTTLBO has the best accuracy for 12 functions with 10 dimensions and 30 dimensions. Among variant TLBOs, the VTTLBO outperforms other algorithms for 14 functions with 10 dimensions and 12 functions with 30 dimensions.

20

10

0

5

0

-20

0

-20

-120 -140 -160

0

0.5

1

1.5

-20 -25 -30

2

2.5 FEs

3

3.5

4

4.5

-35

5

log (Mean Fitness)

0

4

0.5

1

2

2.5 FEs

3

3.5

4

10

4

5

3.5

0.5

1

1.5

2

2.5 FEs

3

3.5

4

4.5

5 4

x 10

(c) f3 f6

1.35

0

1

2

3

4

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

5

FEs

4

x 10

2.5

0

-2

-4

-6 DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-8

-10

2 -12

1.5 -14

1.5

2

2.5 FEs

3

3.5

4

4.5

1 0

5

0.5

1

1.5

2

4

2.5 FEs

3

3.5

4

4.5

-16

5

0

0.5

1

1.5

2

4

(e) f 5

3

3.5

4

4.5

5 4

x 10

(f) f 6

f8

f7

2.5 FEs

x 10

x 10

f9

2

4

5

0

2

0

-2

-6

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-6 -8 -10 -12

-8

-14

-10 0

0.5

1

1.5

2

2.5 FEs

3

3.5

4

4.5

-16

5

-10

-15

-20

0

0.5

1

1.5

2

4

x 10

(g) f 7

2.5 FEs

3

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-5

10

-4

-4

10

-2

log (Mean Fitness)

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

log (Mean Fitness)

0

10

1

2

(d) f 4

log (Mean Fitness)

0.5

1.45

3

10

10

log (Mean Fitness)

log (Mean Fitness)

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO 0

0

4

x 10

1.4

-20

-160

5

f5 4.5

-15

4.5

(b) f2

f4

-10

-100

-140

1.5

x 10

15

-5

-80

-120

(a) f1

0

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-60

10

-15

-40

10

-100

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

log (Mean Fitness)

10

-80

-10

10

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-60

-5

log (Mean Fitness)

-40

log (Mean Fitness)

f3

f2

f1 20

3.5

4

4.5

5

0

0.5

1

1.5

(h) f 8

f10

0

3

-20

3

3.5

4

4.5

5 4

x 10

f12 15

20

4

2.5 FEs

(i) f 9

f11

5

2

4

x 10

10

-100 -120 -140

-3 -4

-160

0

0.5

1

1.5

2

2.5 FEs

3

3.5

4

4.5

5

log (Mean Fitness)

0

0.5

1

1.5

-15

2

4

f13

10

3 2.5

0

0.5

1

1.5

2

4

x 10

2.4

3

(m) f 13

3.5

4

4.5

5

3.5

4

4.5

5 4

x 10

2.2

-6 DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-8

-16

3

f15

-2

-14

2.5 FEs

(l) f 12

2.6

1.5

2.5 FEs

-20

5

2.8

-12

2

4.5

0

-10

1.5

4

2

2

1

3.5

-4

log (Mean Fitness)

10

log (Mean Fitness)

4 3.5

0.5

3

f14

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

4.5

0

2.5 FEs

(k) f 11

5

1

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-10

x 10

(j) f 10

0

-5

10

-80

10

-2

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

log (Mean Fitness)

-1

-60

10

0

-40

log (Mean Fitness)

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

1

10

log (Mean Fitness)

5

2

2

1.8 DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

1.6

1.4

1.2

0

0.5

1

1.5

2

2.5 FEs

3

3.5

4

4.5

5 4

x 10

1

0

0.5

1

1.5

2

2.5 FEs

4

x 10

(n) f 14

(o) f 15

3

3.5

4

4.5

5 4

x 10

f16

f18

f17

2

4

0

2

4.2 DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

4.1

4

0

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-8 -10

-14

-4

-6

-8

-12

-14

0

0.5

1

1.5

2

2.5 FEs

(p) f 16

3

3.5

4

4.5

5

0

0.5

1

1.5

2

3.8

3.7

10

DE jDE SaDE PSOwFIPS CLPSO ABC TLBO ETLBO SawTLBO VTTLBO

-10

-12

log (Mean Fitness)

-6

3.9

-2

10

-4

log (Mean Fitness)

10

log (Mean Fitness)

-2

3.6

3.5

3.4

3.3

2.5 FEs

4

3

3.5

4

4.5

5 4

x 10

3.2

0

0.5

1

1.5

2

2.5 FEs

3

3.5

4

x 10

(q) f 17

(r) f 18

Fig.5 Convergence curves of the different methods on 18 test functions with 30 dimensions.

4.5

5 4

x 10

4.3.2 Comparisons on the Convergence Speed The speed in obtaining the global optimum is also a salient yardstick for measuring the performance of an algorithm. The mean number of function evolutions (FES) is often used to measure the speed of algorithms. The mean of function evolutions (FEs) is also used in the paper to test the convergence speed of different algorithms. The mean function evolutions (mFEs) of all algorithms for eighteen functions with 10 and 30 dimensions are shown in Table 4 and 5. Boldface in the tables indicates the best result among ten algorithms. “mFEs” in the tables represents the mean of FEs when the algorithm can converge to the acceptable solutions, and “NaN” represents the algorithm is not convergent. The acceptable solutions of eighteen are listed in table 1. The tables reveal that the VTTLBO generally offers a much higher speed. Table 4 displays that the mFEs of VTTLBO almost the smallest except that for functions f 5 , f 7 , f10 , f13 ,

f15 and f18 with 10 dimensions. The mFEs of DE is the smallest for function f18 . jDE has better performance in terms of mFEs for functions f 5 , f 7 and f10 . ABC has the smallest mFEs for function f13 , and original TLBO is the best for f15 .Table 5 indicates that VTTLBO almost better than all other algorithms in terms of mFEs for 30 dimensional functions when the algorithms is convergent. 4.3.3 Comparisons on the Reliability Table 4 and table 5 also indicate that VTTLBO can reach the acceptable solutions with a high successful ratio for a majority of functions. The Tables display that the successful ratios of VTTLBO is 100% for all test functions with 10 dimensions except for f 5 , f 7 , f10 , f13 , f15 and f18 . According to the theorem of “no free lunch” [51], one algorithm cannot offer better performance than all the others on every aspect or on every kind of problem. This is also observed in our experimental results. For f10 in table 5, the successful ratio of VTTLBO is 0.0%, jDE and SaDE algorithms have the highest successful ratios. The mean of SaDE is the smallest among the results of all algorithms. 4.3.4 Comparisons Using t-Tests For a thorough comparison, the t-test [52] has also been carried out. Table 6 and 7 present the t values and the P values on every function of this two-tailed test with a significant level of 0.05 between the VTTLBO and other algorithms. Rows “B”, “S”, and “W” represent the number of functions that VTTLBO performs significantly better than, almost the same as, and significantly worse than the compared algorithm, respectively. “M” represents the difference between the number of “B” and number of “W”, which is used to give an overall comparison between the two algorithms. For example, in table 6, comparing VTTLBO and DE, the former significantly outperformed the latter on thirteen functions ( f1 , f 2 , f 3 , f 4 , f 5 , f 6 , f 9 , f11 , f12 , f14 , f15 , f16 , f17 )， does as better as the latter on three functions ( f 7 , f8 , f13 ), and does worse on two functions ( f10 ,

f18 ), yielding a “M=13-2=11”, indicates that VTTLBO generally outperforms the DE.

Table 4 The mean FEs and reliability “ratio” being the percentage of trial runs reaching acceptable solutions for 10 dimensional functions F f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18

Index mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%)

DE 6733 100 11892 100 5947 100 9372 80 NaN 0 10121 100 31720 100 15015 100 11865 100 11045 80.7 6015 100 9085 100 NaN 0 10396 100 25715 80.5 16937 80 10522 40.2 4772 40.1

jDE 6218 100 17719 100 5628 100 13083 100 30868 100 9619 100 6297 100 16636 100 5231 100 2970 100 6201 100 13662 100 26724 100 10097 100 30618 80.2 33109 80 20995 40.3 17279 100

SaDE 6395 100 19236 100 5635 100 13163 100 NaN 0 9586 100 9132 100 12775 100 8896 100 4031 100 6329 100 12509 100 18602 60.6 9963 100 37112 40.6 19909 100 23761 60.6 16445 60.2

PSOwFIPS 24291 100 NaN 0 20803 100 37939 100 NaN 0 38944 100 39383 80.3 NaN 0 32582 80.5 14641 100 24624 100 38992 100 NaN 0 39525 100 NaN 0 NaN 0 NaN 0 21843 60.4

CLPSO 27464 100 NaN 0 24573 100 NaN 0 NaN 0 37753 100 28860 100 41247 100 23562 100 16295 100 29864 100 NaN 0 32232 20.5 45103 100 44141 40.9 NaN 0 41831 40.5 34275 40.2

ABC 11300 100 NaN 0 9627 100 NaN 0 46110 60.3 21143 100 8529 100 25594 100 7392 100 4644 100 25430 100 NaN 0 12828 80.1 41642 40.7 NaN 0 NaN 0 22955 60.8 24089 40.2

TLBO 2728 100 5659 100 2400 100 5814 100 NaN 0 4126 100 27555 40.7 6093 100 5084 100 44232 20.8 2444 100 5952 100 23392 40.5 4166 100 15690 80.2 7071 100 5473 80.2 5427 20.8

ETLBO 3038 100 5968 100 2584 100 6062 100 44657 60.8 4563 100 41698 20.4 6604 100 5604 100 9000 20.6 2616 100 5955 100 26534 60.7 4509 100 19242 100 7219 100 22714 80.3 14408 20.6

sawTLBO 2381 100 4775 100 2066 100 4863 100 NaN 0 3472 100 NaN 0 4876 100 9030 100 NaN 0 2179 100 4798 100 19728 40.7 3485 100 NaN 0 5368 100 18404 60.5 36033 10.7

VTTLBO 1201 100 3343 100 975 100 3563 100 NaN 0 2089 100 15596 80.6 3817 100 1940 100 NaN 0 1028 100 3425 100 NaN 0 2146 100 17338 90.4 4482 100 3939 100 NaN 0

Table 5 The mean FEs and reliability “ratio” being the percentage of trial runs reaching acceptable solutions for 30 dimensional functions F f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18

Index mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%) mFEs ratio(%)

DE 27150 100 NaN 0 24454 100 NaN 0 NaN 0 39175 100 NaN 0 NaN 0 15701 100 NaN 0 43848 40 NaN 0 NaN 0 41747 100 NaN 0 NaN 0 19680 100 NaN 0

jDE 20132 100 NaN 0 18788 100 NaN 0 NaN 0 29075 100 29888 100 NaN 0 11780 100 18013 100 24552 80 NaN 0 NaN 0 30421 100 NaN 0 NaN 0 17233 80.4 NaN 0

SaDE 19179 100 NaN 0 17115 100 NaN 0 NaN 0 27726 100 45886 100 39975 100 11037 100 21675 100 39593 60 NaN 0 NaN 0 28286 100 NaN 0 NaN 0 14234 100 NaN 0

PSOwFIPS NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0

CLPSO NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0

ABC NaN 0 NaN 0 48083 60 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 29454 100 48980 40 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 NaN 0 40107 100 NaN 0

TLBO 4724 100 19289 100 4397 100 36703 100 NaN 0 6813 100 NaN 0 9809 100 2808 100 NaN 0 4432 100 34724 100 NaN 0 6853 100 NaN 0 10243 100 8751 100 NaN 0

ETLBO 5521 100 19131 100 5119 100 36607 100 NaN 0 8024 100 9069 20 11236 100 3192 100 NaN 0 5184 100 37788 100 NaN 0 7966 100 NaN 0 11514 100 3826 100 NaN 0

sawTLBO 4065 100 16458 100 3833 100 38919 100 NaN 0 5730 100 NaN 0 7773 100 2495 100 NaN 0 3856 100 41165 100 NaN 0 5744 100 NaN 0 8092 100 2962 100 NaN 0

VTTLBO 2896 100 13656 100 2571 100 26751 100 NaN 0 4580 100 3841 20.5 7395 100 1663 100 NaN 0 2709 100 25608 100 NaN 0 4687 100 44773 20 7674 100 1904 100 NaN 0

Table 6 Comparisons between VTTLBO and other algorithms on t-tests for 10 dimensional functions F f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18

Index t p t p t p t p t p t p t p t p t p t p t p t p t p t p t p t p t p t p B W S M

DE 5.43544 0.00000 2.44949 0.01735 2.78652 0.00719 2.44949 0.01735 27.95752 0.00000 9.79793 0.00000 0.18678 0.85248 0.00000 0.00000 8.60781 0.00000 -10.56084 0.00000 2.82975 0.00639 3.20093 0.00222 0.53590 0.59408 4.00000 0.00018 2.50931 0.01491 2.44949 0.01735 6.36069 0.00000 -9.83875 0.00000 13 2 3 11

jDE 4.54976 0.00003 4.09832 0.00013 2.75199 0.00789 5.53630 0.00000 -10.38811 0.00000 9.79792 0.00000 -3.93366 0.00023 0.00000 0.00000 -2.45392 0.01716 -18.49128 0.00000 2.45039 0.01731 2.97901 0.00422 -10.58683 0.00000 1.54919 0.12678 1.96635 0.05405 2.44949 0.01735 8.43912 0.00000 -15.68989 0.00000 9 6 3 3

SaDE 3.66971 0.00053 2.92125 0.00496 2.78247 0.00727 2.45131 0.01727 5.72160 0.00000 9.79795 0.00000 -3.93366 0.00023 0.00000 0.00000 2.44936 0.01735 -18.49128 0.00000 2.44949 0.01735 6.41658 0.00000 -5.58600 0.00000 4.00000 0.00018 6.30646 0.00000 0.00000 0.00000 5.52823 0.00000 -5.52836 0.00000 12 4 2 8

PSOwFIPS 3.58350 0.00070 15.73810 0.00000 8.58049 0.00000 7.95848 0.00000 37.96791 0.00000 10.16731 0.00000 1.24785 0.21710 3.52008 0.00085 8.00362 0.00000 -18.49045 0.00000 5.73997 0.00000 16.35435 0.00000 -1.84232 0.07054 19.40931 0.00000 12.95096 0.00000 4.72886 0.00001 14.56882 0.00000 -8.55744 0.00000 14 2 2 12

CLPSO 3.98012 0.00019 21.24876 0.00000 8.04327 0.00000 6.16582 0.00000 7.27535 0.00000 8.10079 0.00000 -3.93366 0.00023 5.69992 0.00000 4.00454 0.00018 -18.49128 0.00000 2.49999 0.01527 7.46973 0.00000 1.35688 0.18008 4.50827 0.00003 5.29565 0.00000 6.53114 0.00000 6.35406 0.00000 -10.56140 0.00000 14 3 1 11

ABC 13.64902 0.00000 9.41908 0.00000 9.99281 0.00000 8.57776 0.00000 -6.16305 0.00000 12.04990 0.00000 -3.93366 0.00023 0.00000 0.00000 3.80716 0.00034 -18.49128 0.00000 2.45178 0.01725 15.51211 0.00000 -9.11518 0.00000 3.39652 0.00124 8.05539 0.00000 14.77781 0.00000 4.68876 0.00002 -6.64853 0.00000 12 5 1 7

TLBO 36.42932 0.00000 2.50909 0.01492 64.84875 0.00000 5.10861 0.00000 -3.85903 0.00029 9.79794 0.00000 3.70712 0.00047 0.00000 0.00000 3.65635 0.00055 -2.87856 0.00559 28.42611 0.00000 3.29901 0.00166 -4.50287 0.00003 4.00000 0.00018 4.36704 0.00005 0.00000 0.00000 2.44949 0.01735 -3.59099 0.00068 12 4 2 8

ETLBO 3.39227 0.00125 3.48099 0.00096 1.72853 0.08921 5.19353 0.00000 -8.49870 0.00000 9.79792 0.00000 3.31380 0.00159 0.00000 0.00000 3.59150 0.00068 -2.09365 0.04067 4.05413 0.00015 2.47387 0.01631 -3.92593 0.00023 4.00000 0.00018 0.91675 0.36307 0.00000 0.00000 7.20373 0.00000 -3.51902 0.00085 10 4 4 6

sawTLBO 4.62509 0.00002 2.45002 0.01732 2.80395 0.00686 3.49252 0.00092 6.20717 0.00000 9.79796 0.00000 8.91036 0.00000 0.00000 0.00000 2.45027 0.01731 0.60236 0.54928 2.56071 0.01307 3.96554 0.00020 -5.76149 0.00000 1.54919 0.12678 18.42593 0.00000 0.00000 0.00000 3.20821 0.00218 0.75452 0.45359 12 1 5 11

Table 7 Comparisons between VTTLBO and other algorithms on t-tests for 30 dimensional functions F f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15 f16 f17 f18

Perf. t p t p t p t p t p t p t p t p t p t p t p t p t p t p t p t p t p t p B W S M

DE 2.49786 0.01535 7.03664 0.00000 4.95393 0.00001 7.82569 0.00000 19.96823 0.00000 7.34904 0.00000 11.49504 0.00000 2.45517 0.01710 4.00000 0.00018 -4.64776 0.00002 3.81035 0.00034 13.07114 0.00000 -2.63596 0.01075 5.70043 0.00000 8.94559 0.00000 3.65465 0.00056 2.44949 0.01735 -12.53139 0.00000 15 3 0 12

jDE 3.88020 0.00027 16.78121 0.00000 5.55632 0.00000 4.38946 0.00005 -10.63733 0.00000 8.69932 0.00000 -9.45370 0.00000 3.41449 0.00117 0.00000 0.00000 -60.24449 0.00000 2.44949 0.01735 5.84512 0.00000 -9.90243 0.00000 12.36021 0.00000 18.58793 0.00000 2.87731 0.00560 2.44949 0.01735 -6.14918 0.00000 12 5 1 7

SaDE 9.79973 0.00000 8.92754 0.00000 6.66086 0.00000 6.27699 0.00000 9.25173 0.00000 8.57171 0.00000 -8.98867 0.00000 4.04876 0.00015 0.00000 0.00000 -61.02415 0.00000 2.59909 0.01183 13.18648 0.00000 -9.18987 0.00000 8.03982 0.00000 28.11787 0.00000 2.50713 0.01499 3.48211 0.00095 -0.16659 0.86827 13 3 2 10

PSOwFIPS 28.25896 0.00000 20.74920 0.00000 15.48621 0.00000 32.05225 0.00000 42.19516 0.00000 16.02194 0.00000 38.88453 0.00000 18.95497 0.00000 88.32375 0.00000 2.47513 0.01626 6.42189 0.00000 37.52055 0.00000 -5.99601 0.00000 21.09418 0.00000 47.97259 0.00000 42.02964 0.00000 74.18339 0.00000 3.43627 0.00110 17 1 0 16

CLPSO 13.61658 0.00000 23.45663 0.00000 19.20653 0.00000 23.13658 0.00000 8.81290 0.00000 6.79885 0.00000 10.44619 0.00000 41.50697 0.00000 33.43644 0.00000 -30.76707 0.00000 14.52254 0.00000 77.87243 0.00000 -2.60189 0.01175 104.62911 0.00000 41.29552 0.00000 62.46333 0.00000 19.04223 0.00000 -1.86381 0.06741 15 2 1 13

ABC 11.03709 0.00000 30.08164 0.00000 11.64081 0.00000 45.62848 0.00000 -1.35155 0.18177 10.44280 0.00000 -6.25481 0.00000 23.49313 0.00000 2.60135 0.01176 -50.44439 0.00000 9.03218 0.00000 41.40965 0.00000 -3.19439 0.00227 29.61358 0.00000 29.09206 0.00000 71.24562 0.00000 4.69950 0.00002 -3.24173 0.00197 13 4 1 9

TLBO 6.91940 0.00000 4.11085 0.00013 8.57662 0.00000 3.25746 0.00188 6.89769 0.00000 0.00000 0.00000 -0.58148 0.56317 0.00000 0.00000 0.00000 0.00000 -2.18595 0.03287 2.56213 0.01302 5.47976 0.00000 -2.79744 0.00698 0.00000 0.00000 2.40937 0.01917 0.00000 0.00000 2.44949 0.01735 -0.77658 0.44056 9 2 7 7

ETLBO 7.91090 0.00000 3.96634 0.00020 4.05791 0.00015 5.45051 0.00000 -2.59283 0.01203 0.00000 0.00000 1.50707 0.13722 0.00000 0.00000 0.00000 0.00000 0.62962 0.53142 5.01342 0.00001 5.42230 0.00000 -2.48376 0.01591 0.00000 0.00000 0.97850 0.33189 0.00000 0.00000 0.00000 0.00000 -3.19130 0.00229 6 3 9 3

sawTLBO 4.62468 0.00002 5.16024 0.00000 2.99685 0.00401 2.64679 0.01044 4.74035 0.00001 0.00000 0.00000 6.38780 0.00000 0.00000 0.00000 0.00000 0.00000 3.97156 0.00020 3.85693 0.00029 2.91858 0.00500 -1.95803 0.05504 0.00000 0.00000 11.50420 0.00000 0.00000 0.00000 0.00000 0.00000 -0.56481 0.57438 10 3 5 7

5. Time series prediction To deeply test the performance of the proposed algorithm, complex system modeling and prediction problems with ANN are evaluated and the performances of some EAs are compared. 5.1 Definition of the problem In general, complex system modeling and prediction uses a sequence of historical values to develop a model for building the approximation model and forecasting future values. It implies that laws underlying these complex systems can be expressed as a dynamical model. In this part, Multilayer perception (MLP) is used as a prediction model of time series. For MLP training, the goal is to find a set of weights and biases with the smallest error measure. Hence, it can be considered as a minimization problem. The most commonly cost function is shown as follow. 1 Q K 1 (11) MSE  (dij  yij )2  i 1  j Q*K 2 Where, Q is the number of training data set, K is the number of output samples, dij is desired output and yij is output inferred from MLP. 5.2 Examples MISO Nonlinear System modeling and Mackey-Glass chaotic system (Mackey-Glass) prediction are simulated in this section. The maximal number of function evaluations (FEs) is used as the ended condition. The minimum and maximum values of the variables are -10 and 10. The maximal numbers of FEs are 10000 and 25000 for MISO System modeling and Mackey-Glass system prediction respectively. To compare the performance of different algorithms, GA, PSO, TLBO and VTTLBO are also used to optimize the weights of MLP over 30 random runs. The number of population is set to 50, and the initial population is selected randomly. 5.2.1 MISO Nonlinear System For this problem, 100 pairs of data are chosen from the real model as follow. y(t ) y(t  1)[ y(t )  2.5] 2 t y(t  1)   u (t ) u (t )  sin( ) 2 2 1  y (t )  y (t  1) 25

(12)

The goal is to build the approximation model of the non-linear function at the point y(t+1) from the earlier points u(t), y(t-1) and y(t) . y(t  1)  f (u(t ), y(t 1), y(t ))

(13)

For MLP with 2 input units, 5 hidden units and one output units, the inputs are connected to all the hidden units, the variables are consisted by weights and biases of MLP. Table 8 shows the mean and the standard deviation of the errors and successful rates. Fig.6 shows ideal curves, approximation curves and the mean sum of squared errors curves of the four algorithms.

Table 8 Comparisons between VTTLBO and other algorithms on MSE GA

PSO

TLBO

VTTLBO

Mean

Std

Mean

Std

Mean

Std

Mean

Std

0.0405

0.0194

0.0201

0.0136

0.0144

0.0080

0.0085

0.0064

Table 8 displays that the results of VTTLBO are better than GA, PSO and TLBO in terms of the mean MSE and the standard deviation of MSE, and the results of TLBO are better than those of GA and PSO. Fig.6 shows that the outputs of the identified systems which track the target output with different algorithms. It can be concluded that VTTLBO has the best performance compared with GA, PSO TLBO for building the approximation model. Problem: MISO Method: GA

Convergence curves using different methods

4

3 GA PSO TLBO VTTLBO

3

2

Output

1

0

1

1

log 0(Best Value)

2

-1

0

-2

-1

-3

-2 0

0.2

0.4

0.6

0.8

1 FEs

1.2

1.4

1.6

1.8

2

0

10

20

(a) Convergence curves using different algorithms Problem: MISO Method: PSO

90

100

Problem: MISO Method: TLBO

3

3

2

2

Output

4

1

0

1

0

-1

-1

Ideal Output Actual Output Output Error 0

10

20

30

40 50 60 Sample number

70

80

90

-2

100

(c) Approximation curves using PSO

0

10

20

30

Ideal Output Actual Output Output Error 40 50 60 70 Sample number

Problem: MISO Method: VTTLBO

3

2

1

0

-1

-2

0

10

20

80

90

100

(d) Approximation curves using TLBO

4

Output

Output

80

(b) Approximation curves using GA

4

-2

30

4

x 10

Ideal Output Actual Output Output Error 40 50 60 70 Sample number

30

Ideal Output Actual Output Output Error 40 50 60 Sample number

70

80

90

100

(e) Approximation curves using VTTLBO Fig.6 Comparison of the performance curves using different algorithms

5.2.2 Mackey-Glass chaotic system prediction For Mackey-Glass chaotic system, the goal is to predict the value of the time series at the point x(t+1) from the earlier points x(t 18), x(t 12), x(t  6), x(t ) u(t-4) and x(t-1) [53]. xˆ(t  1)  f ( x(t 18), x(t 12), x(t  6), x(t ))

(14)

The normalized 2000 data pairs after the initial transients are chosen from the real model. The former 1500 points are selected as the training data points and the rest 500 points are selected as the testing data points. For MLP with 4 input units, 5 hidden units and one output units, the inputs are connected to all the hidden units, the variables are consisted by weights and biases. Table 9 shows the mean and the standard deviations of the errors and successful rates of different algorithms. Fig.7 shows ideal curves, the training and prediction curves and the mean sum of squared errors curves of all algorithms. Table 9 Comparisons of differential algorithms on time series over 30 runs Type of Filter

Algorithm

Training Error

Testing Error

Successful Rate

CPU Time(s)

GA

0.0032±0.0013

0.0031±0.0013

24/24

269.64±30.28

PSO

0.0042±0.0130

0.0041±0.0129

14/16

302.04±103.92

TLBO

0.0051±0.0020

0.0049±0.0019

18/19

283.46±60.99

VTTLBO

0.0030±0.0018

0.0030±0.0017

28/27

291.80±81.18

Mackey-Glass

Table 9 displays that VTTLBO attains the best performances in both training and testing cases in terms of the mean MSE and testing error. The table also shows that the successful rate of VTTLBO is the highest, the successful times are 28 and 27, respectively, where “28” is the successful rate in training process and “27” is for testing process. PSO behaves the worst, only 14 for training and 16 for testing, respectively. The experiment results show that, although VTTLBO spends much more time than GA and PSO, it holds excellent prediction precision. The model trained by VTTLBO tracks the chaotic behavior of the Mackey-Glass chaotic time series very well as is demonstrated in Fig. 7. Convergence curves using different methods

Problem: MackeyGlass Method: GA

4

1.4

GA PSO TLBO VTTLBO

3 2

1.2

Output

0

1

log 0(Best Value)

1

1 0.8

0.6

-1 0.4

-2

-4

Ideal Output Actual Output Output Error

0.2

-3

0

0

0.5

1

1.5 FEs

2

2.5 4

0

200

400

600

800 1000 1200 Sample number

1400

1600

1800

2000

x 10

(a) Convergence curves of different algorithms

(b) The training and prediction results of GA

Problem: MackeyGlass Method: TLBO 1.4

1.2

1.2

1

1

0.8

0.8

Output

Output

Problem: MackeyGlass Method: PSO 1.4

0.6

0.4

0.4 Ideal Output Actual Output Output Error

0.2

0

0.6

0

0

200

400

600

800 1000 1200 Sample number

Ideal Output Actual Output Output Error

0.2

1400

1600

1800

2000

(c) The training and prediction results of PSO

0

200

400

600

800 1000 1200 Sample number

1400

1600

1800

2000

(d) The training and prediction results of TLBO

Problem: MackeyGlass Method: VTTLBO 1.4

1.2

Output

1

0.8

0.6

0.4 Ideal Output Actual Output Output Error

0.2

0

0

200

400

600

800 1000 1200 Sample number

1400

1600

1800

2000

(E) The training and prediction results of VTTLBO Fig.7 Comparison of the performance curves for different algorithms

6. Conclusion In this paper, a variant of TLBO which is called VTTLBO is proposed for improving the performance of TLBO, in which a variable population size in the form of a triangle function is designed. In VTTLBO algorithm, the population size linearly vary from the minimal value to the maximal value and then from the maximal value to the minimal value. The new individuals are generated with gauss distribution with adaptive mean and variance in the increasing phase and the individuals with high similarity are removed from the current population in the decreasing phase of population size. The saw-tooth TLBO is also designed in the paper, and the results of it are also tested on some benchmark functions. The results indicate that the proposed algorithm is also a challenge swarm based algorithm with comparing with some other EAs. In addition, the VTTLBO algorithm is also extended for modeling MISO system and predicting Mackey-Glass chaotic system (Mackey-Glass) with ANN, the parameters of ANN is optimized by the different algorithms, the results indicate that VTTLBO also has some good performance. In the future, it is expected that VTTLBO will be applied to constrained, dynamic and noisy single-objective, multi-objective optimization and other real-world optimization problems.

Acknowledgement: This work was supported in part by Major Project of Natural Science Research in Anhui Province under Grant KJ2015ZD36, in part by the Natural Science Foundation of Anhui Province under Grant 1308085MF82, in part by National Natural Science Foundations of China under Grants 61304082 and 41475017, in part by The National Science Fund for Distinguished Young Scholars under Grant 61425009

References [1] R. V. Rao, V. J. Savsani, D. P. Vakharia, Teaching-learning-based optimization: An optimization method for continuous non-linear large scale problems. Information Sciences, 2012, 183(1): 1-15. [2] R.V. Rao, V. J. Savsani, D. P. Vakharia, Teaching-learning-based optimization: A novel method for constrained mechanical design optimization problems, Computer-Aided Design, 2011, 43: 303-315. [3] R. V. Rao and V. Patel, An elitist teaching-learning-based optimization algorithm for solving complex constrained optimization problems, International Journal of Industrial Engineering Computations, 2012, 3: 535-560. [4] K. Rajasekhar, K. Prakash. Multi-level production planning in a petrochemical industry using elitist Teaching–Learning-Based-Optimization. Expert Systems with Applications, 2015, 42 : 628–641 [5] M. R. Nayak, C. K. Nayak and P. K. Rout, Application of Multi-Objective Teaching Learning Based Optimization Algorithm to Optimal Power Flow Problem, Procedia Technology, 2012, 6: 255-264. [6] B. Amiri, Application of Teaching-Learning-Based Optimization Algorithm on Cluster Analysis, Journal of Basic and Applied Scientific Research, 2012, 2(11): 11795-11802. [7] V. Toğan, Design of planar steel frames using Teaching–Learning Based Optimization, Engineering Structures, 2012, 34: 225-232. [8] S. C. Satapathy, A. Naik and K. Parvathi, High dimensional real parameter optimization with teaching learning based optimization, International Journal of Industrial Engineering Computations, 2012, 3(5): 807-816. [9] A. Naik, S. C. Satapathy and K. Parvathi, Improvement of initial cluster center of c-means using Teaching learning based optimization, Procedia Technology, 2012, 6: 428-435. [10] R. V. Rao, V. Patel, Multi-objective optimization of heat exchangers using a modified teaching-learning-based optimization algorithm, Applied Mathematical Modelling, 2013, 37(3): 1147-1162. [11] T. Niknam, F. Golestaneh, and M. S. Sadeghi, θ-Multiobjective Teaching-Learning-Based Optimization for Dynamic Economic Emission Dispatch, IEEE systems journal, 2012, 6(2): 341-351. [12] R. V. Rao, V. D. Kalyankar, Parameter optimization of modern machining processes using teaching– learning-based optimization algorithm, Engineering Applications of Artificial Intelligence, 2013, 26(1): 524-531. [13] F. Zou, L. Wang, X. H. Hei, et.al. Multi-objective optimization using teaching-learning-based optimization algorithm, Engineering Applications of Artificial Intelligence, 2013,26:1291–1300 [14] A. M. Miguel, A. C. C. Carlos, M. R. Juan, Reactive Power Handling by a Multi-Objective Teaching Learning Optimizer Based on Decomposition. IEEE Transactions on power systems, 2013, 28(4):3629-3637 [15] P. K. Roy, A. Sur, D. K.Pradhan. Optimal short-term hydro-thermal scheduling using quasi-oppositional teaching learning based optimization. Engineering Applications of Artificial Intelligence, 2013,26:2516–2524 [16]K. Xia, L. Gao, W. D. Li, et.al. Disassembly sequence planning using a simplified Teaching–Learning-Based Optimization algorithm. Advanced Engineering Informatics, 2014, 28: 518–527

[17] B. Mandal, P. K. Roy. Multi-objective optimal power flow using quasi-oppositional teaching learning based optimization. Applied Soft Computing, 2014, 21:590–606 [18]J. Q. Li, Q. K. Pan, K. Mao, A discrete teaching-learning-based optimization algorithm for realistic flowshop rescheduling problems. Engineering Applications of Artificial Intelligence 2015,37:279–292 [19] R. Singh, H. K. Verma. Teaching–learning-based Optimization Algorithm for Parameter Identification in the Design of IIR Filters. J. Inst. Eng. India Ser. B,2014,94(4):285–294 [20] S. Sultana, P. K. Roy. Multi-objective quasi-oppositional teaching learning based optimization for optimal location of distributed generator in radial distribution systems. Electrical Power and Energy Systems, 2014, 63: 534–545 [21] R. V. Rao, V. Patel, An improved teaching-learning-based optimization algorithm for solving unconstrained optimization problems, Scientia Iranica, 2013, 20(3): 710-720. [22] S. C. Satapathy, A. Naik and K. Parvathi, Weighted Teaching-Learning-Based Optimization for Global Function Optimization, Applied Mathematics, 2013, 4, 429-439. [23] N. Taher, A. A. Rasoul, R. N. Mohammad. A new multi-objective optimization approach based on TLBO for location of automatic voltage regulators in distribution systems. Engineering Applications of Artificial Intelligence, 2012,25:1577–1588 [24] Suresh C. S., A. Naik K. Parvathi. A teaching learning based optimization based on orthogonal design for solving global optimization problems. Springer Plus, 2013, 2:130 [25] D. B. Chen, F. Zou, J. T. Wang, et.al. A teaching–learning-based optimization algorithm with producer– scrounger model for global optimization. Soft Computing, 2015, 19:745–762 [26] C. H. Chen. Group leader dominated teaching-learning based optimization. International conference on parallel and distributed computing, applications and technologies, 2013:304-308 [27] E. Uzlu, M. I. Kömürcü, M. Kankal, et.al. Prediction of berm geometry using a set of laboratory tests combined with teaching–learning-based optimization and artificial bee colony algorithms. Applied Ocean Research, 2014, 48 :103–113 [28]M. Ghasemi, M. M. Ghanbarian, S. Ghavidel, et.al. Modified teaching learning algorithm and double differential evolution algorithm for optimal reactive power dispatch problem: A comparative study. Information Sciences, 2014, 278:231–249 [29] L. Wang, F. Zou, D. Yang, et al. An improved teaching-learning-based optimization with neighborhood search for applications of ANN. Neurocomputing, 2014, 143, 231-247.

[30] Z. P. Xie, C. Y. Zhang, X. Y. Shao, et.al. An effective hybrid teaching-learning-based optimization algorithm for permutation flow shop scheduling problem. Advances in Engineering Software, 2014, 77: 35–47 [31] C. S. Suresh, N. Anima. Modified Teaching–Learning-Based Optimization algorithm for global numerical optimization—A comparative study. Swarm and Evolutionary Computation, 2014,16:28–37 [32] F. Zou, L. Wang, X. H. Hei, et.al. Teaching–learning-based optimization with dynamic group strategy for global optimization. Information Sciences, 2014, 273: 112-131 [33] Y. Xu., L. Wang,S. Y Wang, et.al. An effective teaching–learning-based optimization algorithm for the flexible job-shop scheduling problem with fuzzy processing time. Neurocomputing, 2015,148:260–268 [34] T. Chen, K. Tang, G. Chen, X. Yao, A large population size can be unhelpful in evolutionary algorithms, Theoretical Computer Science, 2012, 436, 54-70. [35] V. K. Koumousis and C. P. Katsaras, A Saw-Tooth Genetic Algorithm Combining the Effects of Variable Population Size and Reinitialization to Enhance Performance, IEEE Transactions on Evolutionary Computation, 2006,10(1): 19-28. [36] Z. Zhang and H. Zhang, A Variable-Population Evolutionary Game Model for Resource Allocation in Cooperative Cognitive Relay Networks, IEEE communications letters, 2013, 17(2): 361-364. [37] W. Zhu, Y. Tang, J. Fang, W. Zhang, Adaptive population tuning scheme for differential evolution, Information Sciences, 2013, 223, 164-191. [38] D. Aydin, S. Özyön, C. Yaşar, et al., Artificial bee colony algorithm with dynamic population size to combined economic and emission dispatch problem, Electrical Power and Energy Systems, 2014, 54, 144-153. [39] C. Zhang, J. Chen, B. Xin, et al., Differential Evolution with Adaptive Population Size Combining Lifetime and Extinction Mechanisms, Proceedings of 2011 8th Asian Control Conference (ASCC) Kaohsiung, Taiwan, May 15-18, 2011, 1220-1226. [40] P. Kouchakpour, A. Zaknich, T. Bräunl, Population variation in genetic programming, Information Sciences, 2007, 177(17): 3438-3452. [41] P. Kouchakpour, A. Zaknich, T. Bräunl, Dynamic population variation in genetic programming, Information Sciences, 2009,179, 1078-1091. [42] Y. Liang, K. S. Leung, Genetic Algorithm with adaptive elitist-population strategies for multimodal function optimization, Applied Soft Computing, 2011, 11, 2017-2034. [43] W. F. Leong and G. G. Yen, PSO-Based Multiobjective Optimization With Dynamic Population Size and Adaptive Local Archives, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2008, 38(5):1270-1293. [44] D. Chen, C. Zhao, Particle swarm optimization with adaptive population size and its application, Applied Soft Computing, 2009, 9, 39-48.

[45] J. Kennedy. Bare bones particle swarms. in Proceedings of the IEEE Swarm Intelligence Symposium, 2003, 80-87. [46] D. Karaboga, B. Basturk. On the performance of artificial bee colony (ABC) algorithm. Applied Soft Computing , 2008, 8(1): 687-697 [47] J. Brest, S. Greiner, B. Boskovic, M. Mernik, & V. Zumer. Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems. IEEE Transactions on Evolutionary Computation, 2006, 10(6): 646-657 [48] A. K. Qin, V. L. Huang, and P. N. Suganthan. Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Transactions on Evolutionary Computation, 2009, 13(2): 398-417 [49] R. Mendes, J. Kennedy, J. Neves. The fully informed particle swarm: simpler, maybe better. IEEE Transactions on Evolutionary Computation, 2004, 8(3): 204-210 [50] J. J. Liang, A. K. Qin, P. N. Suganthan and S. Baskar. Comprehensive learning particle swarm optimizer for global optimization of multimodal functions, IEEE Transactions on Evolutionary Computation. 2006, 10 (3): 281-295. [51] D. H. Wolpert and W. G. Macready, No free lunch theorems for optimization, IEEE Transactions on evolutionary computation, 1997, 1(1): 67-82. [52] X. Yao, Y. Liu and G. M. Lin, Evolutionary programming made faster, IEEE Transactions on evolutionary computation, 1999, 3(2): 82-102. [53] M. C. Mackey, L. Glass. Oscillation and chaos in physiological control systems. Science, 1977, 197(4300): 287-289.

Debao Chen received the Ph.D. degree in the School of Computer Science from NanJing University of Science and Technology, Nanjing, China, in 2008. Currently, he is a full professor in Huaibei Normal University, Huaibei, China. His current research interests include evolutionary computation, global optimization, multiobjective optimization, neural network, etc. Renquan Lu (M’08) received the Ph.D. degree in control science and engineering from Zhejiang

University,

Hangzhou, China, in 2004. Currently, he is a hull Professor in the Institute of Information and Control, Hangzhou Dianzi University, Hangzhou. He has published more than 30 journal papers in the fields of robust control and complex systems. His current research interests include robust control, singular systems, and complex systems. Feng Zou received the Ph.D. degree in the School of Computer Science and Engineering from Xi’an University of Technology, Xi’an, China, in 2015. Currently, he is an associate professor in Huaibei Normal University, Huaibei, China. His research interests mainly include evolutionary algorithms, swarm intelligence and multi-objective optimization. Suwen Li received the Ph.D. degree in Hefei Institutes of Physical Science, Chinese Academy Of Sciences, Hefei, China, in 2008. She is currently a full professor in Huaibei Normal University, Huaibei, China. Her current research interests include algorithm optimization and optoelectronic information science and engineering, etc.