Probabilistic properties of fitness-based quasi-reflection in evolutionary algorithms

Computers & Operations Research 63 (2015) 114–124

Contents lists available at ScienceDirect

Computers & Operations Research journal homepage: www.elsevier.com/locate/caor

Probabilistic properties of fitness-based quasi-reflection in evolutionary algorithms M. Ergezer n, D. Simon Cleveland State University, Department of Electrical and Computer Engineering 2121 Euclid Avenue, FH 332 Cleveland, OH, USA

art ic l e i nf o

a b s t r a c t

Available online 30 March 2015

Evolutionary algorithms (EAs) excel in optimizing systems with a large number of variables. Previous mathematical and empirical studies have shown that opposition-based algorithms can improve EA performance. We review existing opposition-based algorithms and introduce a new one. The proposed algorithm is named fitness-based quasi-reflection and employs the relative fitness of solution candidates to generate new individuals. We provide the probabilistic analysis to prove that among all the opposition-based methods that we investigate, fitness-based quasi-reflection has the highest probability of being closer to the solution of an optimization problem. We support our theoretical findings via Monte Carlo simulations and discuss the use of different reflection weights. We also demonstrate the benefits of fitness-based quasi-reflection on three state-of-the-art EAs that have competed at IEEE CEC competitions. The experimental results illustrate that fitness-based quasi-reflection enhances EA performance, particularly on problems with more challenging solution spaces. We found that competitive DE (CDE) which was ranked tenth in CEC 2013 competition benefited the most from opposition. CDE with fitnessbased quasi-reflection improved on 21 out of the 28 problems in the CEC 2013 test suite and achieved 100% success rate on seven more problems than CDE. & 2015 Elsevier Ltd. All rights reserved.

Keywords: Evolutionary algorithms Continuous optimization Opposition Fitness-based quasi-reflection

1. Introduction Evolutionary algorithms (EAs) are computational intelligence techniques that excel in optimizing systems with a large number of variables. EAs are effective numerical methods for finding global solutions to complex problems because they do not require a differentiable or a continuous objective function and can successfully escape local minima. The cost of these benefits is the increased convergence time due to the large number of function evaluations performed by the EAs. This paper proposes a novel algorithm which generates perturbed solution candidates that have an increased probability (relative to other opposition-based methods) of being closer to the solution(s) of the optimization problem. Consequentially, EAs are expected to achieve lower costs results to optimization problems. We provide the mathematical proofs to demonstrate the benefits that can be gained by employing this algorithm as a component of other heuristic search techniques. The proposed algorithm is inspired by opposition-based learning (OBL). A shortcoming of most EAs that are based on natural

n

Corresponding author. E-mail addresses: [email protected] (M. Ergezer), [email protected] (D. Simon). http://dx.doi.org/10.1016/j.cor.2015.03.013 0305-0548/& 2015 Elsevier Ltd. All rights reserved.

processes is that they are modeled after very slow processes. On the other hand, human society progresses at a much faster rate via “social revolutions.” Hence, an EA based on such a model can accelerate the learning process. Tizhoosh maps this theory to machine learning and proposes to use opposite numbers instead of random mutations to quickly evolve the EA population [1]. OBL was first proposed in 2005 [2] and was applied to a popular reinforcement learning algorithm, Q-learning. It was concluded that opposition-based extension reduces the algorithm's convergence time [3]. Opposition can also accelerate learning in machine intelligence [1]. Preliminary results with anti-chromosomes for genetic algorithms, contrariness values for Q-learning and opposite weights for artificial neural networks illustrate the numerous possibilities for opposition in the field of artificial intelligence. The benefits of OBL in solving global optimization problems were first published in [4]. In that paper, concepts of opposition-based initialization and generation jumping were proposed to improve the solution accuracy of differential evolution (DE) and were tested on a limited test set. The research was extended to empirical analysis on an extensive collection of benchmark functions and the experimental results illustrated that opposition-based DE outperforms fuzzyadaptive DE and standard DE [5]. The application of OBL in evolutionary computation is not limited to DE. Other EAs have also benefited from the idea of opposition. OBL

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

has been paired with biogeography-based optimization (BBO) to form oppositional BBO (OBBO) [6,7]. Variations of OBBO have been introduced in the literature [8,9]. Particle swarm optimization (PSO) is another EA that benefited from OBL. OBL helps PSO to enhance swarm diversity [10]. Different opposition-based PSO algorithms have been introduced [11] with velocity clamping [12] or with Cauchy mutation [13]. More recent opposition-based EAs include appending artificial bee colony algorithm [14] with opposition to form generalized oppositionbased ABC [15]; CODEQ, a parameter-free algorithm that combines chaotic search, opposition-based learning, differential evolution and quantum mechanics for optimizing constrained problems [16]; and opposition-based gravitational search algorithm [17]. Applications of OBL in other fields of evolutionary computation are also continuing to expand. A survey on the state-of-the-art opposition research is presented in [18]. Most recent uses of OBL include a new framework for OBL and cooperative co-evolution to solve large-scale global optimization problems [19], multiobjective optimization problems based on decomposition [20] and mathematical analysis demonstrating an opposition-based EA's probability of converging to optimum for discrete-domain scheduling problems [21]. The remainder of this paper is structured as follows. Section 2 provides a historical background for opposition in different fields of science and philosophy. This section also discusses different opposition-based techniques as employed in continuous optimization and their statistical characteristics. Section 3 introduces fitness-based opposition and fitness-based quasi-reflection. We then present the probabilistic analysis of fitness-based quasireflection. Section 4 provides a discussion on various algorithm parameters. Section 5 analyzes the performance of the proposed method on CEC 2013 competition on real-parameter single objective optimization test suite and some of the competing algorithms. In Section 6, we draw conclusions and outline potential next steps for this research.

2. Background 2.1. Introduction to opposition In this section, we discuss the definitions of opposition in various fields, and explain how it can be applied to optimization problems. Opposition is encountered in different fields under different names. In Euclidean geometry it is called inverse geometry, in physics it is called the parity transformation, and in mathematics it is called reflection. All of these definitions involve isometric selfmapping of a function. Other examples include astronomy where planets that are 1801 apart are considered to be opposing each other. Opposites also have a significant meaning in semantics as generalizations of antonyms. Where antonyms are limited to gradable terms, such as thin and thick, the term “opposite” can be applied to gradable, non-gradable and pseudo-opposite terms. Akin to the use of opposition in semantics, OBL has evolved many variations in computational intelligence. These variations generate opposite samples in different intervals of the search space. In the next section, we provide an overview of the opposition-based types. 2.2. Variations of opposition-based algorithms Different opposition-based algorithms have been introduced in the literature to accelerate EA convergence. This section presents an overview of selected OBL techniques. The definitions of these algorithms are given in Table 1 and a graphical representation is given in Fig. 1.

115

Table 1 Mathematical definitions of existing opposition-based algorithms where c is the center of the search interval ½a; b and can be calculated as ða þ bÞ=2, and rand(α; β) is a random number uniformly distributed between α and β. For any x^ A ½a; b, its opposite values are defined below. Method

Definition

Opposition Quasi-opposition Quasi-reflection Center-based sampling

x^ o ¼ a þ b  x^ x^ qo ¼ randðc; x^ o Þ ^ x^ qr ¼ randðc; xÞ ^ x^o Þ x^ cb ¼ randðx;

Fig. 1. Opposite points defined in domain ½a; b. c is the center of the domain and x^ ^ and x^ qo and x^ qr are the quasiis an arbitrary EA individual. x^ o is the opposite of x, opposite and quasi-reflected points, respectively. x^ cb is the center-based point.

The original opposite point is proposed in [1] and is shown in Table 1 as x^ o . The central opposition theorem proves that the opposite point has a higher probability of being closer than a random guess to the solution [22]. However, central opposition theorem does not give the exact probabilities, but rather illustrates the probabilistic relationships between the opposite point and the solution, and a random point and the solution. An intuitive analysis can be used to show that the distance to the optimal solution is less with opposite sampling than random sampling [23]. This proof is also extended for N-dimensional search spaces. Other research performed empirical analysis of the performance of opposite points using 58 benchmark test functions [5]. The effects of population size, problem dimensionality and opposition jumping rate (Jr) on opposition-based differential evolution are studied [5]. A quasi-opposite point is randomly placed between the center of the search domain and the opposite point. The notation for a quasiopposite point is given in Table 1 as x^ qo . Empirical studies on 30 benchmark functions indicate that quasi-oppositional optimization outperforms opposition [24]. Mathematical properties of quasiopposition are given in [25]. We also computed the probability of x^ qo being closer than the opposite point to the solution [7]. A newer OBL algorithm called quasi-reflection is denoted as x^ qr in Table 1. A quasi-reflected point is placed randomly between the solution candidate and the center of the solution space. Empirical studies illustrate the performance gained by quasi-reflection [7,8]. Mathematical proofs given in [6] demonstrate that quasi-reflection has a higher probability of being closer to the solution than opposition and quasi-opposition. One of the latest OBL variations is named center-based sampling and is denoted as x^ cb in Table 1. The closeness of center-based candidates to solutions via Monte Carlo simulations for high dimensional problems is studied in [26]. Empirical studies given in [27–29] show the convergence speed gains that population-based algorithms achieve via this method. This algorithm is generalized in [30].

2.3. Probabilities of previously developed opposition-based algorithms This section reviews the probabilities for two opposition-based algorithms as presented in the literature (Table 2). The results assume that the solution is uniformly distributed in the search space. Without any prior knowledge about the problem, assuming

116

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

Table 2 Probability that an opposite point is closer than an EA individual or its opposite to the solution of an optimization problem. Row

Methods

1




 
Pr
x^ o  x
o
x^  x


2




 
Pr
x^ qo  x
o
x^  x


3




 
Pr
x^ qr  x
o
x^  x


4




 
Pr
x^ qo  x
o
x^ o  x


5




 
Pr
x^ qr  x
o
x^ o  x


Probability 1 2 9 16 11 16 11 16 9 16

“equal probability” for the location of the optimal solution [31] is the reasonable course of action [32]:


 
 Pr
x^ qo x
o
x^  x
: In Table 2 Row 2, we show how much

 



more likely it is that a quasi-opposite point is closer than an EA individual of an optimization problem.
to
the solution
 
Pr
x^ qr  x
o
x^  x
: In Table 2 Row 3, we show how much more likely it is that a quasi-reflected point is closer than an EA individual of an optimization problem.
to
the solution
 
Pr
x^ qo x
o
x^ o  x
: In Table 2 Row 4, we show how much more likely it is that a quasi-opposite point is closer than the opposite of an EA individual to the solution of an optimization problem.


 
Pr
x^ qr  x
o
x^ o  x
: In Table 2 Row 5, we show how much more likely it is that a quasi-reflected point is closer than the opposite of an EA individual to the solution of an optimization problem.

We should note that all these probabilities are in one dimensional space and we assume that the solution x of the optimization problem has a uniform distribution. However, empirical results for higher dimensions are given in [28]. These simulations illustrated that probabilities of quasi-populations being closer to the solution increase with the problem dimension. For instance, the quasireflected and quasi-opposite solution candidates were 91% and 78% more likely to be closer than an EA individual to the solution of a 20-dimensional optimization problem.

3. Fitness-based quasi-reflection algorithm Fitness-based opposition (FBO) and quasi-based reflection are introduced as a single algorithm in [7]. Empirical analysis of FBO with center-based sampling against four other opposition-based algorithms is performed in [28]. That paper compared FBO's performance on 21 well-known benchmark problems. Here, we introduce fitness-based quasi-reflection and present a probabilistic analysis for it. Note that the FBO proof provided below can be extended to other opposition-based algorithms. In this section, we define fitness-dependent quasi-reflection or ^ the solution candidate x^ Kr as a function of random variable x, generated by an EA. With this OBL method, we can control the amount of reflection based on the fitness of the individual. Thus, fit solutions can be reflected by a smaller amount than less fit solutions. x^ Kr is defined as ( ^ x^ þ ðc  xÞK if x^ r c ð1Þ x^ Kr ¼ c þ ðx^  cÞð1  KÞ if x^ 4 c where c is the center of the domain and K A ð0; 1 the reflection weight and is discussed further in Section 4.2. Pseudo-code for fitness-based quasi-reflection is provided in Algorithm 1.

We also provide an alternative definition of Eqs. (1) and (2) which employs the unit step function, U(x), to calculate the fitness-based quasi-reflected point. The unit step function is a discontinuous function and it is defined as 0 for negative values of x and 1 for all other values of x:     ^ ^ þ c þðx^  cÞð1  KÞ Uðx^  cÞ x^ Kr ¼ x^ þðc  xÞK Uðc  xÞ ð2Þ Unlike original opposition, which is individual based, x^ Kr is population oriented. This is because x^ Kr considers the ranking of an individual among its peers in a given population and determines the amount of opposition based on that. Furthermore, x^ Kr eliminates the need for the previously defined random function by considering the relative fitness of the individual. Let the center of the domain, c, be zero (without loss of generality). Then Eq. (1) can be simplified as ^ KÞ x^ Kr ¼ xð1

ð3Þ

Algorithm 1. Algorithm for creating fitness-based quasi-reflected population (FBQRP) where the reflection weight is linearly proportional to a solution candidate's relative ranking in the population. Note that rank¼1 for the best individual in the population. 1: procedure FITNESS-BASED QUASI-REFLECTION(P, K) ▹P is the current population of solution candidates and K is the individual reflection weight 2: for each solution candidate s do 3: N ¼ Population size 4: Various K is defined in Section 4.2 5: foreach dimension d do 6: M d ¼ average value for d in P 7: if P s;d o M d then 8: FBQRP s;d ¼ P s;d þK s ðM d  P s;d Þ 9: else 10: FBQRP s;d ¼ M d þ K s ðP s;d  M d Þ 11: end if 12: end for 13: end for 14: Calculate the cost of FBQRP 15: return fittest set of N individuals amongst (FBQRP; P) 16: end procedure

Based on these definitions, x^ can be positive or negative. Recall ^ that when x^ is negative, Eq. (1) defines x^ Kr ¼ x^ þ ðc  xÞK. If we let ^  KÞ, or Eq. (3). c¼0, we obtain x^ Kr ¼ xð1 Now, we present the probability of x^ Kr being closer than an EA solution candidate to the solution of an optimization problem, x, and the expected value of this probability as a function of the reflection weight, K. Theorem 3.1. Assume that the solution x of an optimization problem is uniformly distributed in a one-dimensional search space. Then the probability averaged over all x and all candidate solutions x^ that a fitness-weighted quasi-reflected point is closer to the solution than a candidate solution is ð6 KÞ=8. Proof. See Appendix A. 4. Discussion 4.1. Monte Carlo experiments Fig. 2 plots the expected probability of x^ Kr being closer to the solution than a random candidate solution x^ as a function of reflection weight. The results are derived theoretically and verified via simulation. The theoretical results are based on Theorem 3.1.

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

0.8

expected value of Eq. (5) becomes equal to Eq. (4):




  
6K 11 EK Pr
x^ Kr  x
o
x^ x
¼ EK ¼ 8 16

0.7 0.6

ð6Þ

where EK indicates the expected value with respect to the variable K. In Algorithm 1, K A ð0; 1 is the reflection weight and is described as

0.5 0.4

K¼

0.3 0.2 0.1 0

117

0

0.2

0.4

0.6

0.8

1

Fig. 2. Probability that x^ Kr is closer to the solution of an optimization problem than an EA individual as a function of the reflection weight, K.

The simulation results are obtained by dividing the solution space into equal subspaces and performing 1000 Monte Carlo simulations for each subspace. The pseudo-code for the simulation is provided in Algorithm 2. Note that there is a discontinuity in this figure when K ¼ 0, where the probability is 0. After this point, at K ¼ 0 þ , the probability jumps to about 75%. We note that as K increases, the probability of x^ Kr being closer to the solution decreases. This is why we recommend using lower reflection weights to fitter candidate solutions, as shown in Algorithm 1. Algorithm 2. Algorithm for simulating the probability of x^ Kr being closer than a solution candidate to the optimal solution. This algorithm was used to create Fig. 2.

Solution rank Population size

ð7Þ

and solution rank¼ 1 for the best individual in the population. However, K can be designed to have a non-uniform distribution so different reflection patterns can be developed to better fit a given problem. Table 3 lists four complementary functions, quadratic and sinusoidal, that could be used to create the reflection weights. Plots of these nonlinear functions are presented in Fig. 3. These functions are inspired from the BBO migration models presented in [33]. In this study, we are suggesting smaller K values for lower ranked solution candidates, such as the ones provided by f1 and f4. 4.3. Distance and reflection weight After deriving the probabilities for x^ Kr , we investigate the relative distances between x^ Kr and a randomly distributed optimization problem solution. A symmetric problem domain ½  b; b is assumed. Fig. 4 plots the simulation results for three distances as a function of the reflection weight. These results are obtained by randomly generating 10,000 values for x, x^ and x^ Kr . We plot the expected distance between x^ and the solution, the difference between x^ Kr and the solution, and the difference between these distances. From this figure, we note that as the reflection weight increases, the distance between x^ Kr and the solution decreases. Table 3 Example of quadratic and sinusoidal functions that can be used to create reflection weights where r is the rank of an individual and N is the population size.

1: procedure 1-DIMENSIONAL PROBABILITY SIMULATION 2: for each possible solution x A ½  1; 1 do 3: for each Monte Carlo simulation do 4: Generate random solution candidate, x^ 5: Initialize probability count (CK) to 0 for all K 6: for each reflection weight K A ð0; 1 do



7: if
x^ Kr;K  x
o
x^  x
then 8: Increment probability count (CK) 9: end if 10: end for 11: end for

Label

Reflection weight

f1

 r 2 N r 2 1 N rπ 1 cos þ1 2 N rπ 1  cos þ1 2 N

f2 f3 f4

count ðC K Þ 12: Probabilityx;K ¼ totalprobability number of simulations 13: end for 14: return ProbabilityK 15: end procedure

1

f1 f2 f3 f4

0.9 0.8

4.2. Discussion on the reflection weight

0.7




 11 
Pr
x^ qr  x
o
x^  x
¼ 16

ð4Þ

0.5 0.4 0.3 0.2

In Section 3, we defined x^ Kr ¼ x^  K x^ and proved that


 6  K 
Pr
x^ Kr  x
o
x^  x
¼ 8

0.6

K

Ref. [6] derives the probabilities for x^ qr , which is a random ^ In this paper, we analyze x^ Kr , which is dependent on function of x. x^ via its fitness rank. As stated in Table 2,

0.1

ð5Þ

for any user-selectable value of K A ð0; 1. If we assume that K, the reflection weight, is uniformly distributed, then E½K ¼ 12 and the

0

0

20

40

60

80

100

Percentile Rank Fig. 3. Four possible nonlinear reflection weights based on individual rankings.

118

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

Table 4 Summary of opposition probabilities. Methods


 
Pr
x^ o  x
o
x^  x



 
Pr
x^ qo  x
o
x^  x



 
Pr
x^ qr  x
o
x^  x



 
Pr
x^ qo  x
o
x^ o  x



 
Pr
x^ qr  x
o
x^ o  x



 
Pr
x^ Kr  x
o
x^  x


Probability 1 2 9 16 11 16 11 16 9 16 6K 8

individual for multidimensional search spaces. This algorithm was used to create Fig. 5.

Fig. 4. The distances between x^ Kr , x^ and the solution, x.

1: procedure MULTI-DIMENSIONAL PROBABILITY SIMULATION 2: for each dimension d do 3: for each Monte Carlo simulation do 4: x^ is randomly selected A ½a; b 5: x is randomly selected A ½a; b 6: Initialize probability count (CK) to 0 for all K in d 7: for each reflection weight, K do




d

d
8: if
x^ Kr;K  xd
o
x^  xd
then

1

0.9

9: 10: 11: 12: 13

0.8

0.7

Increment probability count (CK) in d end if end for end for end for

count ðC K Þ 14: Probabilityx;K ¼ totalprobability for each d number of simulations 15: end procedure d

0.6 100 50 8 0

0

0.2

0.4

0.6

0.8

1

Fig. 5. Probability that x^ Kr is closer to the solution of an optimization problem than an EA individual as a function of the reflection weight, K, and problem dimension.

But, based on Fig. 2, we notice that the probability of being closer to the solution and the distance to the solution are inversely related. Thus, x^ Kr is less likely to be closer to the solution when using a higher reflection weight than a lower reflection weight. However, in the event that the larger K solution is closer, then its distance to the solution is expected to be less as well.

4.4. Simulations on higher dimensions Theorem 3.1 shows the probabilities for x^ Kr and x^ for a single dimensional search space. In this section, we extend the results to higher dimensions via computer simulations. Algorithm 3 outlines the logic employed to generate multi-dimensional probabilities and Fig. 5 plots the obtained probabilities as functions of the reflection weight and problem dimension. From Fig. 5, we note that for a one-dimensional problem, we obtain the result in Theorem 3.1 and as the problem dimensionality increases, the probability of x^ Kr being closer than x^ to the solution of an optimization problem increases as well. Algorithm 3. Algorithm for determining the probabilities of x^ Kr being closer to the solution of an optimization problem than an EA

4.5. Overview of probabilities of opposition-based algorithms Table 4 lists the probabilities of being closer to the solution of an optimization problem for all of the discussed opposition points in Section 2. Rows 1–3 compare the probability of the opposition points to an EA individual, and Rows 4 and 5 compare the probability of the quasi-opposite points to the opposition of the EA individual. In the last row of Table 4, we show the probability of fitness-dependent quasi-reflection being closer to the solution than an EA individual as shown in Theorem 3.1. The fitness-dependent quasi-reflection's probability of being closer to the solution is presented in Row 6. This probability is dependent on the reflection weight K, which in turn depends on the individual's relative fitness in the population. Note that, as illustrated in Eq. (6), an average individual (K ¼ 12) will have the same probability of being closer to the solution as the quasireflected point and amongst all the other opposite points, x^ qr has the highest probability of being closer to the solution.

5. Experimental results This section analyzes the performance of the proposed algorithm via simulations on well-known and state-of-the-art EAs. We employ the benchmark suite developed for the IEEE CEC 2013 realparameter single objective optimization competition [34]. The test suite includes 28 benchmark functions and can be categorized into three groups as shown in Table 5. The final cost values of all the problems have been shifted, thus the optimal cost values for all the problems are 0 and the lower cost values indicate a better solution.

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

Table 5 CEC 2013 test suite problem categories. Function type

Function number

Unimodal Multimodal Composition

1–5 6–20 21–28

Table 6 CEC 2013 test suite configuration parameters. Variable

Setting

Dimension Simulations per problem Population size Search range

30 25 50

Max cost function evaluation Error tolerance

ð  100; 100ÞD 10,000 10  8

Table 6 defines the benchmark function and problem setup parameters [34]. 5.1. Effects of jumping rate Similar to other opposition-based algorithms, x^ Kr relies on a single parameter called the jumping rate, Jr, which determines how often the opposition function will be executed and it is similar in concept to mutation rate in other EAs. In order to present the effects of Jr, we run two EAs with three different Jr values and compare their average final cost to EA with no opposition. The first EA that we analyze is DE [35,36]. DE is selected for this study because most of the algorithms that competed in CEC are variations of DE. Here we simulate the standardized /rand/bin/1 configuration. Table 7 presents the results. The second EA of interest is BBO [37]. BBO models the mathematics of biogeography and one of the competing algorithms that we study in Section 5.2 is a modification of BBO. BBO's results are presented in Table 8. Tables 7 and 8 illustrate that for unimodal problems having no opposition or minimal opposition (Jr ¼0.05) yields the best results. For the multimodal functions, opposition aids the EA; however the optimal frequency of opposition depends on the problem and, in general, Jr of 0.05 and 0.2 are both equally successful. More challenging composition problems benefit the most from opposition and higher Jr values of 0.1 and 0.2. For instance, both for DE 0:05 and BBO, x^ kr outperforms non-opposition DE and BBO in 17 out of 28 problems. Based on our experiments, for the analysis of our competitive algorithms in the following section, we employ a lower Jr value of 0.01 across all algorithms. 5.2. Competitive algorithms In this section, we add x^ Kr to competitive EAs that have been tested on CEC test suites and observe its benefits. Most of the CEC algorithms can benefit from x^ Kr . One exception is the mean– variance mapping optimization (MVMO) since MVMO evaluates fitness per individual, rather than evaluating the whole population [38]. Due to the population-based framework of x^ Kr , one could not add opposition to MVMO. On the other hand, another CEC algorithm, adaptive differential evolution (ADE), already includes quasi-opposition logic [39]. Our simulations focus on four algorithms. The first EA is particle swarm optimization (PSO) [40]. PSO was provided by

119

CEC along with the problem set. An opposition-based PSO algorithm was previously published [41]. Besides the addition of x^ Kr , we had to modify the provided algorithm to increment the function evaluation counter at each cost function evaluation. This provided a fair comparison to the other EAs. The second algorithm that we investigate is success-history based adaptive DE (SHADE) [42]. SHADE ranked fourth out of 23 competitors in CEC 2013 and its offline analysis presented in [42] illustrates its performance against competitive algorithms of CEC 2005. Competitive DE (CDE) is another DE variation that is employed in this study [43]. CDE ranked tenth out of 23 competitors in CEC 2013 and uses adaptivity to avoid tuning the numerous DE parameters. BBO has many state-of-art variations [44–47]. In this section, we incorporate opposition to a variant called linearized BBO (LBBO) [48]. LBBO did not compete in CEC 2013; however, it is included in this study as a newer algorithm and due to its promising performance on the CEC 2005 and CEC 2011 test suites [48]. We find here that on CEC 2013 problem set, LBBO outperforms SHADE in 9 problems, CDE in 12 problems and PSO in 16 problems, out of 28 total problems. Analysis not included here shows that opposition-based LBBO is also less sensitive to jumping rate variations. The source code for all the modified EA is provided at [49]. Table 9 lists the mean of final cost for these four EA for the CEC 2013 test suite after 25 Monte Carlo simulations. The best result for each benchmark function is written in boldface. CEC 2013 competition also requires the algorithm complexity to be reported along with algorithm results as a metric of EA performance. The steps to calculate algorithm complexity require recording computation time of the algorithms for 200,000 cost function evaluations. We do not include an analysis on computational times as the implemented algorithms are written in different computational languages (C, C þ þ and MATLAB) and since addition of x^ Kr does not significantly affect the computational costs of EA. Results presented in Table 9 indicate that for unimodal problems 1 and 5, all algorithms can find the optimal solution consistently. For the multimodal functions, the best performance 0:01 is achieved by CDE/x^ kr and SHADE. For the most challenging 0:01 composition functions, CDE/x^ kr outperforms all other algorithms. 0:01 Overall, CDE/x^ kr has the best performance by reaching the best solution for 15 problems. We also note that SHADE outperformed CDE in CEC2013. However, based on the presented results, CDE/ 0:01 x^ kr would be one of the best algorithms in CEC 2013. 0:01 We also observe that PSO/x^ kr reaches a solution as good as or 0:01 0:01 better than PSO on 17 problems. SHADE/x^ kr , CDE/x^ kr and LBBO/ 0:01 x^ kr reach a solution as good as or better than their nonopposition-based counter parts on 16, 21 and 14 problems, 0:01 respectively. These findings indicate that CDE/x^ kr benefits the most from fitness-based quasi-reflection. Furthermore, for problems 16, 19, 20, 22, 24, 25 and 26, CDE/ 0:01 x^ kr achieves a 100% success rate whereas CDE has mediocre performance. The majority of x^ kr benefits are observable on more challenging multi-modal and composition problems. We also observe the benefits of opposition in function 28 with SHADE. We note that SHADE converges to 300 for every Monte Carlo run as indicated by zero standard deviation. On the other 0:01 hand, SHADE/x^ kr achieves a lower average final fitness value by avoiding this local minimum at the cost of a larger average standard variation. 5.3. Statistical analysis Statistical significance of our findings are analyzed using a non-parametric Wilcoxon test. Table 10 lists the p-values of a two-sided Wilcoxon rank sum test. The null hypothesis is that

120

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

Table 7 Mean (standard deviation) of final cost function after 25 Monte Carlo simulations with DE and its opposition-based versions. The superscript indicates the opposition jumping rate, Jr. Function

DE

DE/x^ kr

DE/x^ kr

DE/x^ kr

1 2 3 4 5

4.88(3.36) 13,876,267(4,767,202) 565,430,885(669,939,428) 6313.83(1820.87) 1.91(1.56)

4.60(5.69) 11,127,525(4,398,381) 867,735,166(1,249,978,816) 7002.93(2329.36) 1.67(1.10)

2.60(1.82) 13,543,785(6,591,510) 1,129,293,332(1,376,815,517) 7085.94(2279.92) 2.34(1.99)

2.05(1.30) 11,351,027(3,526,607) 1,119,792,107(1,164,774,513) 7574.99(2141.63) 1.87(1.03)

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

31.85(23.52) 80.69(24.02) 20.95(0.06) 26.44(2.84) 15.57(13.19) 15.57(10.25) 100.10(30.33) 178.19(32.68) 524.40(560.14) 4702.66(933.15) 2.32(0.32) 59.61(13.23) 259.65(28.85) 5.63(1.98) 12.65(0.58)

34.53(22.65) 79.26(27.93) 20.94(0.08) 25.92(3.37) 8.72(4.15) 18.10(11.48) 97.35(35.08) 179.88(40.09) 427.39(304.75) 4620.35(871.51) 2.26(0.42) 68.86(17.31) 266.67(26.22) 6.66(3.04) 13.01(0.94)

38.12(27.92) 81.38(24.73) 20.97(0.05) 26.01(3.72) 10.51(7.12) 15.27(7.05) 103.05(27.06) 199.19(38.62) 382.03(306.34) 4414.95(895.64) 2.20(0.38) 67.93(20.11) 268.55(27.10) 6.46(2.50) 12.95(1.23)

55.04(31.06) 85.31(26.96) 20.96(0.04) 26.26(4.68) 6.61(4.00) 17.65(10.43) 124.94(53.61) 208.82(27.32) 335.83(285.34) 4258.66(841.08) 2.27(0.36) 65.05(22.27) 278.69(22.72) 5.56(1.92) 13.78(1.31)

21 22 23 24 25 26 27 28

336.40(67.14) 438.66(307.80) 5121.93(1048.98) 269.73(10.25) 284.01(9.47) 234.44(68.42) 983.29(119.66) 373.63(33.81)

349.39(85.79) 425.53(297.54) 5236.28(1065.00) 265.36(9.98) 278.40(8.01) 233.12(66.15) 978.93(125.89) 361.28(28.71)

348.36(79.33) 368.81(167.51) 5247.86(1379.53) 265.60(10.82) 286.14(9.28) 225.71(58.59) 932.02(104.97) 335.79(49.86)

352.57(74.66) 415.42(351.64) 4691.24(1082.09) 263.26(13.87) 284.08(12.21) 239.89(71.73) 955.11(135.42) 471.31(475.69)

0:05

0:1

0:2

Table 8 Mean (standard deviation) of final cost function after 25 Monte Carlo simulations with BBO and its opposition-based versions. The superscript indicates the opposition jumping rate, Jr. Fn

BBO

BBO/x^ kr

BBO/x^ kr

BBO/x^ kr

1 2 3 4 5

0.08(0.03) 13,255,044(5,896,831) 979,635,377(857,927,435) 59,538.46(19,821.06) 0.17(0.04)

0.00(0.00) 11,718,904(3,279,651) 897,989,310(989,407,622) 38,940.14(7951.00) 0.02(0.01)

0.00(0.00) 11,980,358(4855300) 1,245,990,793(1097481556) 37,857.02(8027.02) 0.01(0.01)

0.00(0.00) 14,064,349(3881598) 2,111,463,052(1187093430) 42,656.45(6423.93) 56.79(283.91)

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

47.29(26.82) 85.99(16.85) 20.98(0.05) 29.33(3.06) 4.70(1.43) 0.16(0.08) 65.16(17.61) 132.57(37.03) 1.34(0.33) 4274.24(618.74) 1.11(0.30) 32.15(0.36) 148.76(22.18) 1.26(0.30) 13.16(0.85)

68.42(22.33) 85.11(15.58) 20.94(0.04) 30.77(3.25) 2.37(0.53) 0.01(0.01) 103.46(41.30) 199.98(57.94) 0.54(0.16) 3839.04(706.97) 1.30(0.38) 31.42(0.24) 156.40(40.65) 1.08(0.33) 13.72(1.07)

64.14(26.87) 80.94(15.35) 20.94(0.05) 29.84(4.43) 2.43(0.75) 0.00(0.00) 139.51(58.68) 238.51(49.06) 0.45(0.23) 3973.82(795.76) 1.21(0.40) 31.05(0.18) 150.30(39.60) 1.07(0.32) 13.82(1.12)

77.65(9.89) 80.78(12.88) 20.95(0.06) 26.69(3.35) 5.25(2.51) 0.00(0.00) 169.19(63.56) 263.22(61.58) 0.40(0.32) 3812.96(606.45) 1.31(0.46) 30.89(0.16) 208.68(43.13) 1.09(0.51) 14.10(0.92)

21 22 23 24 25 26 27 28

299.18(77.75) 110.64(19.27) 5029.33(771.98) 270.14(9.48) 290.51(8.81) 263.20(84.97) 990.37(64.63) 424.19(401.65)

323.54(53.46) 113.89(29.13) 4929.98(712.03) 252.81(9.08) 293.40(6.79) 248.34(71.68) 908.07(107.27) 1070.33(957.39)

334.67(62.44) 114.58(29.20) 4750.22(1133.79) 249.34(9.53) 294.39(7.44) 250.65(75.00) 846.21(130.34) 1407.19(948.98)

320.97(74.12) 103.42(19.44) 4964.18(1166.39) 247.22(10.37) 294.45(8.91) 259.34(73.90) 820.60(81.72) 2132.76(645.13)

0:05

mean final costs are samples from continuous distributions with equal medians. We assume that the samples are independent.

0:1

0:2

P-values of less than 0.05 indicate the rejection of the null hypothesis at the 5% significance level. We find that all results are 0:01 statistically significant relative to CDE/x^ kr , except SHADE and

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

121

Table 9 Mean (standard deviation) of final cost function after 25 Monte Carlo simulations with competitive EA and their opposition-based versions. The superscript indicates the opposition jumping rate, Jr. Fn

PSO

PSO/x^ kr

SHADE

SHADE/x^ kr

CDE

CDE/x^ kr

LBBO

LBBO/x^ kr

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

0.00(0) 13.4E6(68E5) 17.9E7(33E7) 5738.70(1477) 0.00(0) 69.29(36) 34.04(11) 20.93(0) 20.10(3) 0.23(0) 23.52(6) 82.55(34) 151.39(35) 828.04(266) 6684.06(872) 2.30(0) 57.61(17) 228.73(31) 3.57(1) 14.22(1) 300.71(85) 869.79(293) 6472.18(980) 263.90(10) 282.13(8) 274.85(79) 880.09(99) 352.31(262)

0.00(0) 11.6E6(76E5) 10.1E7(18E7) 7554.94(2290) 0.00(0) 82.87(31) 36.03(12) 20.93(0) 19.96(4) 0.13(0) 20.97(6) 75.82(21) 149.04(21) 938.92(249) 4846.90(1542) 2.03(1) 72.63(11) 128.64(34) 3.95(1) 14.01(2) 343.67(86) 892.76(230) 5561.80(1496) 262.32(11) 280.57(8) 286.13(78) 858.95(102) 398.30(340)

0.00(0) 10.9E3(86E2) 76.7E4(22E5) 0.00(0) 0.00(0) 1.06(5) 2.77(3) 20.89(0) 28.10(1) 0.09(0) 0.00(0) 21.26(4) 51.50(10) 0.02(0) 3243.15(292) 0.91(0) 30.43(0) 73.76(5) 1.36(0) 10.61(1) 299.48(53) 98.71(21) 3491.58(440) 205.95(5) 264.85(15) 200.18(1) 370.61(46) 300.00(0)

0.00(0) 11.5E3(92E2) 52.3E3(26E4) 0.00(0) 0.00(0) 0.00(0) 7.26(4) 20.96(0) 21.30(2) 0.07(0) 0.00(0) 40.00(12) 72.22(24) 0.04(0) 2945.64(279) 0.86(0) 30.43(0) 56.27(9) 1.24(0) 10.33(1) 330.45(66) 110.68(21) 3324.41(549) 213.37(9) 252.03(13) 214.21(38) 472.98(104) 284.00(54)

0.00(0) 67.1E3(46E3) 84.4E4(21E5) 0.39(1) 0.00(0) 8.96(7) 18.24(14) 20.97(0) 28.91(1) 0.06(0) 0.00(0) 61.89(15) 97.66(18) 0.10(0) 4228.74(391) 1.84(1) 30.43(0) 138.37(18) 1.33(0) 11.45(0) 290.97(82) 108.23(15) 4452.88(321) 221.96(13) 250.61(14) 213.29(46) 778.91(187) 300.00(0)

0.00(0) 63.1E3(41E3) 18.1E5(33E5) 0.56(1) 0.00(0) 8.66(6) 15.95(11) 21.00(0) 14.98(4) 0.06(0) 0.00(0) 56.94(15) 102.39(27) 0.21(0) 3367.34(613) 0.00(0) 4.37(11) 47.65(105) 0.00(0) 0.00(0) 618.45(659) 0.00(0) 3079.22(607) 0.00(0) 0.00(0) 0.00(0) 89.11(106) 478.17(788)

0.00(0) 27.1E2(20E2) 21.9E4(44E4) 1.62(1) 0.00(0) 0.00(0) 89.63(16) 21.00(0) 27.57(2) 0.00(0) 27.86(4) 196.13(31) 244.77(24) 330.71(127) 3636.24(373) 0.77(0) 71.73(6) 226.87(33) 5.41(2) 12.96(1) 140.00(50) 474.21(144) 4411.38(458) 278.46(8) 299.85(7) 200.02(0) 1003.89(81) 220.00(100)

0.00(0) 21.5E2(10E2) 24.8E4(56E4) 1.06(1) 0.00(0) 0.00(0) 82.40(14) 20.97(0) 27.46(3) 0.00(0) 28.66(3) 221.04(28) 238.52(33) 338.44(88) 3596.70(424) 0.73(0) 74.72(6) 242.33(39) 5.33(2) 12.97(1) 144.00(51) 497.29(147) 4374.52(473) 277.11(9) 301.61(6) 200.02(0) 999.06(80) 196.00(102)

Best

2

2

9

7

3

15

5

6

0:01

0:01

0:01

0:01

Table 10 P-values of a two-sided Wilcoxon test. The statistically significant results are indicated in bold font. 0:01

CDE/x^ kr

vs.

p-value

PSO

PSO/x^ kr

SHADE

SHADE/x^ kr

CDE

LBBO

LBBO/x^ kr

0.0003

0.0002

0.8824

0.9893

0.0247

0.0388

0.0461

0:01

0:01

SHADE/x^ kr . We fail to reject the null hypothesis with a high 0:01 probability while comparing the final mean values of CDE/x^ kr 0:01 and SHADE/x^ kr . This is probably due to the fact that both algorithms are adaptive DE-based and benefit similarly from x^ Kr . 0:01 We also note that results provided by CDE and CDE/x^ kr are statistically significant.

6. Conclusions This paper introduced a technique called fitness-based quasireflection, x^ Kr , to accelerate EA performance. x^ Kr is based on our earlier empirical results in [7] and is characterized by assigning a reflection weight proportional to a solution candidate's relative fitness. This allows exploitation of the search space by the weaker solution candidates and smaller perturbations for fitter solution candidates. We provide a probabilistic analysis of x^ Kr and support our theoretical findings via Monte Carlo simulations. The mathematical derivations indicate that among all opposition methods that we consider, the proposed algorithm has the highest expected probability to be closer to the solution of an optimization problem. It is worth noting that fitness-based opposition can be applied to any other OBL algorithm and the analysis here can be modified to derive probabilistic properties. We suggest this idea for future work. Numerical experiments were carried out on three state-of-theart algorithms with a strong track record at the recent IEEE CEC competitions. These algorithms were modified with the addition

0:01

0:01

of x^ Kr logic. Simulations on CEC 2013 test suite problems illustrated that the opposition-based version of the algorithms outperforms the IEEE CEC competing algorithms. The mathematical results offer a theoretical understanding for our empirical results. Future work can include further development of the included proofs. The presented results assume that the problem space is one-dimensional; however, they can be extended for higher dimensions. We assumed that the solution and the estimate have uniform distributions as in [32] and that the problem domain is symmetric such that b ¼  a to simplify the resulting mathematical expressions. As the algorithm converges, the distribution of the solution space may change. Thus, the next step should include studying the distribution of the search space and the corresponding opposition-based probabilities. As discussed in Algorithm 1, x^ Kr applies opposition to the whole population. Future work should be made analyzing the effects of opposition on selected individuals. Finally, we limited the reflection weight to K A ð0; 1 in this paper. Varying the range of K will create different opposition algorithms and can be a topic of further research.

Appendix A. Probabilistic analysis of fitness-weighted quasireflection

Theorem 3.1. Assume that the solution x of an optimization problem is uniformly distributed in a one-dimensional search space. Then the

122

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

Since by definition K 4 0 and x^ o0


^  KÞ  x
ox  x^ j xð1 ^  KÞ o x ¼ 1 Pr xð1 Fig. 6. Opposite points defined in domain ½a; b. c is the center of the domain and x^ ^ and x^ Kr is the is an EA individual, generated by an EA. x^ o is the opposite of x, fitness-based quasi-reflected point.

^ Fig. 7. Solution domain if x A ½a; x.

^ c. Fig. 8. Solution domain if x A ½x;

A.1. Case (A) ^ as shown in Fig. 7. From Fig. 7, we note For this case, x A ½a; x ^ that x^ is always closer than x^ Kr to solution, x. Hence, when x A ½a; x, the probability that the quasi-reflected point is closer than the opposite point to the solution is   ^ Pr j x^ Kr  xj o j x^  xj ¼ 0 for x A ½a; x ðA:1Þ A.2. Case (B) ^ c as seen in Fig. 8: This case investigates the probability if x A ½x;


 




Pr x^ Kr  x o x^  x


 
^  KÞ  x
o
x^  x
¼ Pr
xð1 ðA:2Þ From Fig. 8, we note that x^ o x. Then, Eq. (A.2) can be simplified as



 ^  KÞ
o x  x^ Pr
x  xð1

ðA:3Þ

We now use the Total Probability Theorem from [50, Eq. 241], and obtain four probabilities:


 
 ^  KÞ
o x  x^ ¼ Pr
xð1 ^  KÞ  x
o x  x^ j xð1 ^  KÞ o x Pr
x  xð1   ^  KÞ o x Pr xð1

 ^ KÞ  x
o x  x^ j xð1 ^ KÞ 4 x þ Pr
xð1   ^  KÞ 4 x Pr xð1 ðA:4Þ The subsequent sections analyze these four terms individually. A.2.1. Case (B1)

 ^ KÞ o x ^ KÞ  x
ox  x^ j xð1 This case looks at the term Pr
xð1 from Eq. (A.4):

 ^  KÞ x
o x  x^ j xð1 ^  KÞ ox Pr
xð1   ^  KÞ o x  x^ j xð1 ^  KÞ o x ¼ Pr x  xð1   ^ o 0j xð1 ^ KÞ o x ðA:5Þ ¼ Pr xK

ðA:6Þ

A.2.2. Case (B2)   ^  KÞ o x from Eq. (A.4) and it This case looks at the term Pr xð1 is solved in two steps. We first hold x^ fixed and find the probability over x. Then, we let x^ vary and calculate the corresponding expected probability.     ^ 0 , that is, x  U x; ^ 0 , we When x^ is fixed and x is uniform in x; obtain the conditional probability as Z 0 Z 1

  1 ^  KÞ o x

x^ ¼ f ðxÞ dx ¼  dx Pr j xð1 x^ ^  KÞ ^  KÞ xð1 xð1
0
1 ¼  x

¼ 1K ðA:7Þ x^ xð1 ^  KÞ   Now, we let x^  U b; 0 and calculate the probability: Z 0     ^  KÞ f ðxÞ ^ dx^ ^  KÞ o x ¼ Pr x 4 xð1 Pr xð1

probability averaged over all x and all candidate solutions x^ that a fitness-weighted quasi-reflected point is closer to the solution than a candidate solution is ð6  KÞ=8. Proof. Given the scenario in Fig. 6 where a and b are the end points of the solution domain and c is the center of this domain, the solution, x, will always be in one of these four segments: ^ (B) x A ½x; ^ c, (C) x A ½c; x^ o  or (D) x A ½x^ o ; b. We examine (A) x A ½a; x, each scenario separately in Cases A, B, C and D below.

^ c for x A ½x;

Z ¼

b 0




 1 1  K

0 dx^ ¼ x^
¼ 1  K ð1  KÞ b b b b

ðA:8Þ

A.2.3. Case (B3)

 ^  KÞ  x
o x  x^ j xð1 ^  KÞ 4 x This case solves the term Pr
xð1 from Eq. (A.4):

 ^  KÞ  x
ox  x^ j xð1 ^  KÞ 4 x Pr
xð1   ^  KÞ o2x; xð1 ^  KÞ 4 x Pr xð2   ¼ ^  KÞ 4 x Pr xð1

 2K ^  KÞ 4 x x^ o x; xð1 Pr 2   ¼ ^ KÞ 4 x Pr xð1

 2K ^  KÞ x^ o x o xð1 Pr 2   ðA:9Þ ¼ ^  KÞ 4 x Pr xð1 Eq. (A.9) consists of two probabilities. The second probability is solved in Section A.2.4. The first probability  can  be calculated in ^ 0 : two steps where we hold x^ fixed and x  U x;




2  K

^  KÞ


x^ x^ o x o xð1 Pr

2 Z ð1  KÞx^ Z ð1  KÞx^ 1 ¼ f ðxÞ dx ¼  dx x^ ^ ^ ðð2  KÞ=2Þx ðð2  KÞ=2Þx
ð1  KÞx^

 1
1 K x^ ¼  x^  K x^  x^ þ ¼  x

2 x^ ðð2  KÞ=2Þx^ x^ ¼

K 2

ðA:10Þ

  We let x^  U  b; 0 and calculate the probability as

 2K ^  KÞ x^ ox o xð1 Pr 2  Z 0 2K ^  KÞ f ðxÞ ^ dx^ x^ o x o xð1 ¼ Pr 2 b
Z 0 K1 K
K ¼ dx^ ¼ x^

¼ 2b 2b 2 b

 b;0

ðA:11Þ

We then combine Eqs. (A.8) and (A.11) to solve Eq. (A.9):


^  KÞ  x
ox  x^ j xð1 ^  KÞ 4 x Pr xð1

 2K ^  KÞ x^ o x o xð1 Pr 2   ¼ ^  KÞ 4 x Pr xð1

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

be simplified as   ^  KÞ o x  x^ Pr x  xð1     ^ þ 1  1Þ o xð1 1Þ ¼ Pr K x^ o 0 ¼ Pr xðK Fig. 9. Solution domain if x A ½c; x^ o .

123

ðA:18Þ

Thus,


 
Pr
x^ Kr  x
o
x^  x
¼ 1

for x A ½c; x^ o 

ðA:19Þ

A.4. Case (D) Fig. 10. Solution domain if x A ½x^ o ; b.

¼

K1 1 ¼ 2K 2

ðA:12Þ

A.2.4. Case (B4)   ^  KÞ 4 x from Eq. (A.4). This is This case solves the term Pr xð1 solved in two steps. We first hold x^ fixed and find the conditional probability over x. Then, we let x^ vary and calculate the corresponding probability.   ^ 0 , we obtain When x^ is fixed and x  U x;
  ^  KÞ oxj
x^ Pr j xð1

 ^  KÞj
x^ ¼ F x ðj xð1 ^  KÞj
x^ Þ ¼ Pr j x o xð1 Z xð1 Z ^  KÞ ^  KÞ xð1 1 ¼ f ðxÞ dx ¼  dx x^ 1 x^
xð1 ^  KÞ  1
1 ¼  x

¼  x^  K x^  x^ x^ x^ x^ ¼K ðA:13Þ   ^ Now, we let x  U  b; 0 and calculate the probability as   ^ KÞ 4 x Pr xð1 Z 0   ^  KÞ f ðxÞ ^ dx^ ¼ Pr x o xð1 Z ¼

b 0

K b

1 K dx^ ¼ b b


0
x^



b

¼K

ðA:14Þ

A.2.5. Case (B) Conclusion We can now solve Eqs. (A.2) and (A.4) using Eqs. (A.6), (A.8), (A.12) and (A.14):


 
Pr
x^ Kr  x
o
x^  x



 
^  KÞ  x
o
x^ x
¼ Pr
xð1

 ^  KÞ  x
ox  x^ j xð1 ^  KÞ o x ¼ Pr
xð1   ^  KÞ o x  Pr xð1

 ^ KÞ  x
ox  x^ j xð1 ^ KÞ 4 x þ Pr
xð1   ^  KÞ 4 x  Pr xð1 1 K ¼ 1ð1 KÞ þ ðKÞ ¼ 1  2 2

ðA:15Þ

Thus,


 
K Pr
x^ Kr  x
o
x^  x
¼ 1  2

^ c for x A ½x;

ðA:16Þ

A.3. Case (C) For this case x A ½c; x^ o  as shown in Fig. 9. When x A ½c; x^ o , the probability that the quasi-reflected point is closer than the estimated point to the solution is



Pr
x^ Kr  x
o
x^  x



 
^  KÞ  x
o
x^ x
¼ Pr
xð1 ðA:17Þ ^  KÞ o x. Then, Eq. (A.2) can From Fig. 9, we note that x^ o x and xð1

This is the case if x A ½x^ o ; b as shown in Fig. 10. This case is very similar to Case (C). From Fig. 10, we again note ^  KÞ o x: that x^ o x and xð1


 
Pr
x^ Kr  x
o
x^  x
  ðA:20Þ ¼ Pr K x^ o 0 ¼ 1 for x A ½x^ o ; b A.5. Conditional probability We can now combine all of the cases to calculate the conditional probability in the domain ½a; b:



 
Pr
x^ Kr  x
o
x^  x

x^

 K ^ þ 1ðx^ o  0Þ þ 1ðb  x^ o Þ ð0  xÞ 0ðx^ þ bÞ þ 1  2 ¼ 2b

 K þb  x^ 1  2 ðA:21Þ ¼ 2b A.6. Probability We now take the previous results to prove Theorem 3.1. The probability for uniform x^ can be calculated as Z 0





 

^ dx^ Pr
x^ Kr  x
o
x^  x
¼ Pr
x^ Kr  x
o
x^  x
f ðxÞ b

 0 1 K Z 0 þb   x^ 1  B C 1 2 B C ¼ @ A b dx^ 2b b
^ 2Þ þ4b
0 xðK x^

2 8b b 6 K ¼ 8 ¼

ðA:22Þ

This gives the result as in Theorem 3.1 References [1] Tizhoosh H. Opposition-based learning: a new scheme for machine intelligence. In: Proceedings of international conference on computational intelligence for modelling control and automation, vol. 1; 2005. p. 695–701. [2] Tizhoosh H. Reinforcement learning based on actions and opposite actions. In: International conference on artificial intelligence and machine learning; 2005. p. 94–8. [3] Tizhoosh H. Opposition-based reinforcement learning. J Adv Comput Intell Intell Informatics 2006;10(4):578–85. [4] Rahnamayan S, Tizhoosh HR, Salama MM. Opposition-based differential evolution algorithms. In: IEEE congress on evolutionary computation; 2006. p. 2010–7. [5] Rahnamayan S, Tizhoosh HR, Salama MMA. Opposition-based differential evolution. IEEE Trans Evol Comput 2008;12(1):64–79. [6] Ergezer M, Simon D. Mathematical and experimental analyses of oppositional algorithms. IEEE Trans Cybern 2014;44(11):2178–89. [7] Ergezer M, Simon D, Du D. Oppositional biogeography-based optimization. In: IEEE international conference on systems, man and cybernetics; 2009. p. 1009–14. [8] Yang X, Cao J, Li K, Li P. Improved opposition-based biogeography optimization. In: International workshop on advanced computational intelligence; 2011. p. 642–7.

124

M. Ergezer, D. Simon / Computers & Operations Research 63 (2015) 114–124

[9] Ergezer M, Simon D. Oppositional biogeography-based optimization for combinatorial problems. In: Smith AE, editor. Proceedings of the 2011 IEEE congress on evolutionary computation, New Orleans, USA; 2011. p. 1490–7. [10] Chi Y, Cai G. Particle swarm optimization with opposition-based disturbance. In: International Asia conference on informatics in control, automation and robotics, vol. 2; 2010. p. 223–6. [11] Han L, He X. A novel opposition-based particle swarm optimization for noisy problems. In: International conference on natural computation, vol. 3, Los Alamitos, CA, USA; 2007. p. 624–629. [12] Shahzad F, Baig A, Masood S, Kamran M, Naveed N. Opposition-based particle swarm optimization with velocity clamping. Adv Comput Intell 2009:339–48. [13] Wang H, Wu Z, Rahnamayan S, Liu Y, Ventresca M. Enhancing particle swarm optimization using generalized opposition-based learning. Inf Sci 2011;181(20):4699–714. [14] Karaboga D, Basturk B. A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Glob Optim 2007;39(3):459–71. [15] El-Abd M. Generalized opposition-based artificial bee colony algorithm. In: IEEE congress on evolutionary computation; 2012. p. 1–4. [16] Omran MG, Salman A. Constrained optimization using CODEQ. Chaos Solitons Fract 2009;42(2):662–8. [17] Shaw B, Mukherjee V, Ghoshal S. A novel opposition-based gravitational search algorithm for combined economic and emission dispatch problems of power systems. Int J Electr Power Energy Syst 2012;35(1):21–33. [18] Xu Q, Wang L, Wang N, Hei X, Zhao L. A review of opposition-based learning from 2005 to 2012. Eng Appl Artif Intell 2014;29:1–12. [19] Kazimipour B, Omidvar MN, Li X, Qin A. A novel hybridization of oppositionbased learning and cooperative co-evolutionary for large-scale optimization. In: IEEE congress on evolutionary computation; 2014. p. 2833–40. [20] Ma X, Liu F, Qi Y, Gong M, Yin M, Li L, et al. Moea/d with opposition-based learning for multiobjective optimization problem. Neurocomputing 2014;146:48–64. [21] Zhao F, Zhang J, Wang J, Zhang C. A shuffled complex evolution algorithm with opposition-based learning for a permutation flow shop scheduling problem, Int J Comput Integr Manuf 2014, http://dx.doi.org/10.1080/0951192X.2014. 961965, in press. [22] Rahnamayan S, Tizhoosh HR, Salama MM. Opposition versus randomness in soft computing techniques. Appl Soft Comput 2008;8(2):906–18. [23] Rahnamayan S, Wang GG, Ventresca M. An intuitive distance-based explanation of opposition-based sampling. Appl Soft Comput 2012;12(9):2828–39. [24] Rahnamayan S, Tizhoosh HR, Salama MMA. Quasi-oppositional differential evolution. In: IEEE congress on evolutionary computation; 2007. p. 2229–36. [25] Rahnamayan S. Opposition-based differential evolution [Ph.D. dissertation]. University of Waterloo; 2007. [26] Rahnamayan S, Wang G. Center-based sampling for population-based algorithms. In: IEEE congress on evolutionary computation, 2009. p. 933–8. http:// dx.doi.org/10.1109/CEC.2009.4983045. [27] Esmailzadeh A, Rahnamayan S. Enhanced differential evolution using centerbased sampling. In: IEEE congress on evolutionary computation; 2011. p. 2641–8. [28] Ergezer M, Sikder I. Survey of oppositional algorithms. In: The 14th international conference on computer and information technology; 2011. p. 623–8. http://dx.doi.org/10.1109/ICCITechn.2011.6164863. [29] Esmailzadeh A, Rahnamayan S. Center-point-based simulated annealing. In: IEEE Canadian conference on electrical & computer engineering; 2012. p. 1–4.

[30] Wang H, Rahnamayan S, Zeng S. Generalised opposition-based differential evolution: an experimental study. Int J Comput Appl Technol 2012;43(4):311–9. [31] Bernoulli J. Ars conjectandi [The art of conjecture], Basel; 1713. [32] Dembski W, Marks R. Conservation of information in search: measuring the cost of success. IEEE Trans Syst Man Cybern Part A Syst Hum 2009;39(5):1051–61. [33] Ma H. An analysis of the equilibrium of migration models for biogeographybased optimization. Inf Sci 2010;180(18):3444–64. [34] Liang JJ, Qu B-Y, Suganthan PN, Hernández-Dıaz AG. Problem definitions and evaluation criteria for the CEC 2013 special session and competition on realparameter optimization, Technical report 201212. Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou China; 2013. [35] Storn R, Price K. Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 1997;11(4):341–59. [36] Storn R, Price K. Differential evolution-a simple and efficient adaptive scheme for global optimization over continuous spaces, Technical report. International Computer Science Institute Publications; 1995. [37] Simon D. Biogeography-based optimization. IEEE Trans Evol Comput 2008;12(6):702–13. [38] Rueda J, Erlich I. Hybrid mean-variance mapping optimization for solving the IEEE-CEC 2013 competition problems. In: IEEE congress on evolutionary computation; 2013. p. 1664–71. [39] dos Santos Coelho L, Ayala H, Zanetti Freire R. Population's variance-based adaptive differential evolution for real parameter optimization. In: IEEE congress on evolutionary computation; 2013. p. 1672–7. [40] Eberhart R, Kennedy J. A new optimizer using particle swarm theory, in: Proceedings of the sixth international symposium on micro machine and human science; 2002. p. 39–43. [41] Wang H, Liu Y, Zeng S, Li H, Li C. Opposition-based particle swarm algorithm with Cauchy mutation. In: IEEE congress on evolutionary computation; 2007. p. 4750–6. [42] Tanabe R, Fukunaga A. Evaluating the performance of SHADE on CEC 2013 benchmark problems. In: IEEE congress on evolutionary computation; 2013. p. 1952–9. [43] Tvrdík J, Polakova R. Competitive differential evolution applied to CEC 2013 problems. In: IEEE congress on evolutionary computation; 2013. p. 1651–7. [44] Xiong G, Shi D, Duan X. Enhancing the performance of biogeography-based optimization using polyphyletic migration operator and orthogonal learning. Comput Oper Res 2014;41:125–39. [45] Zheng Y-J, Ling H-F, Xue J-Y. Ecogeography-based optimization: enhancing biogeography-based optimization with ecogeographic barriers and differentiations. Comput Oper Res 2014;50:115–27. [46] Feng Q, Liu S, Zhang J, Yang G, Yong L. Biogeography-based optimization with improved migration operator and self-adaptive clear duplicate operator. Appl Intell 2014;41(2):563–81. [47] Xu Q, Guo L, Wang N, Pan J, Wang L. A novel oppositional biogeography-based optimization for combinatorial problems. In: International conference on natural computation; 2014. p. 412–8. [48] Simon D, Omran MG, Clerc M. Linearized biogeography-based optimization with re-initialization and local search. Inf Sci 2014;267:140–57. [49] Ergezer M. Quasi-reflection, Online, retrieved from 〈https://sites.google.com/ site/mehmetergezer/quasi_reflection/〉; March 2015. [50] Papoulis A, Pillai S. Probability. Random variables, and stochastic processes. New York: McGraw-Hill; 1965.