Dynamics of fitness sharing evolutionary algorithms for coevolution of multiple species

Applied Soft Computing 10 (2010) 832–848

Contents lists available at ScienceDirect

Applied Soft Computing journal homepage: www.elsevier.com/locate/asoc

Dynamics of fitness sharing evolutionary algorithms for coevolution of multiple species Minqiang Li *, Dan Lin, Jisong Kou School of Management, Tianjin University, 92 Weijin Road, Nankai District, Tianjin 300072, PR China

A R T I C L E I N F O

A B S T R A C T

Article history: Received 25 April 2007 Received in revised form 16 May 2009 Accepted 3 October 2009 Available online 13 October 2009

This paper builds the normal model of fitness sharing with proportionate selection on real-valued functions, and derives the dynamic formula to describe the evolution process of the population with the fitness sharing. The normal modeling simulation is investigated on specific test functions, and experimental results illustrate that the normal model is able to describe exactly the dynamics of the fitness sharing EAs and is a good platform to study the behavior of the fitness sharing EAs with regard to niching radius. The experimental results of the normal modeling simulation and the fitness sharing EAs verify the dilemma in finding optimal niche radius to achieve both good niching convergence and niching efficiency, for which a hybrid scheme is proposed to carry out the niching task. ß 2009 Elsevier B.V. All rights reserved.

Keywords: Evolutionary algorithms Fitness sharing Niching Normal modeling Multimodal functions optimization

1. Introduction Many real-world optimization problems are multimodal, and it is usually required that multiple optima, either global or local, are identified so as to obtain more information for decision making tasks in both engineering and management. Since the standard evolutionary algorithms (EAs) or genetic algorithms (GAs) tend to evolve the population toward only one of the optima with the commonly used elitist selection strategy, where all individuals evolve to the neighborhood of an optimum after the termination of the algorithms, it is not suitable to handle the task of the multimodal optimization problems. For this purpose, many niching mechanisms in the research community of evolutionary computation (EC) have been proposed to realize the coevolution of multiple species and the formation of stable subpopulations among all or most of the global or local optima [1–3]. In order to reduce the effects of genetic drift in the standard EAs (including all kinds of the standard genetic and evolutionary algorithms), various niching approaches, usually called niching techniques, were developed to maintain the population diversity and evolve subpopulations to search in parallel multiple peaks when the EAs were used to solve multimodal function optimization problems. Among all of the niching methods (including

* Corresponding author. Tel.: +86 22 27404796; fax: +86 22 27404796. E-mail addresses: [email protected], [email protected] (M. Li). 1568-4946/$ – see front matter ß 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.asoc.2009.10.001

crowding and preselection, island model GAs and parallel GAs, immune systems, and the clustering-based niching methods) [2– 8], the fitness sharing is a well-known niching technique [1,9,10], and a variety of modified fitness sharing schemes [11–18] were presented. Recently, there were two studies on the successfully application of the fitness sharing technique. Solteiro Pires et al. [19] employed the fitness sharing technique to promote population diversity in the design of the multi-objective genetic algorithm for solving the manipulator trajectory generation problems, which produced the converged non-dominated Pareto front with a good spread and distribution. Hwang and Cho [20] used the fitness sharing method as the speciation technique to deal with the massive multimodal and deceptive landscape of the evolvable hardware problems, and the efficiency of the fitness sharing scheme in overcoming the deceptions and obtaining diverse hardware modules was validated in experiments. The fitness sharing EAs (FSEA) works by emulating the mechanism of natural coevolution of multiple species. It is able to accomplish the identification, maintenance, and coevolution of the multiple species in the exploration of all or most niches on the fitness landscape with the proper parameterization. Although it has been adopted to deal with complicated real-world problems successfully, there still remain many facets about the fitness sharing mechanism that we have not fully probed. What we attempt to do in the paper is to investigate the features and properties of the FSEA with regard to the proportionate selection by building normal models and making facet analyses so as to

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

reveal the dynamics of the FSEA with regard to the niche radius on specific problems. The conclusions to be drawn will benefit the design and application of the FSEA to complex multimodal optimization problems. This paper gives a brief introduction of the principle and main characteristics of the fitness sharing technique in Section 2. In Section 3, we review the theoretical propositions and properties of perfect sharing and overlapped sharing, and raise the issues related to the fitness sharing scheme in terms of the niching quality, niching convergence and efficiency. In Section 4, we build the normal model of the fitness sharing on the real-valued functions, and derive the dynamic formula to describe the evolution process of the population with the fitness sharing. The boundary effect of fitness sharing with regard to niche counts is pointed out and analyzed. Experiments are conducted on a set of test functions, and results are reported and compared between the normal model simulation and the FSEA in Section 5. Finally, we conclude this paper with a summary and suggest the feasible research directions in the future work. 2. Principle of fitness sharing A niche represents a local optimum and its attraction basin (or region) in the feasible solution space of a multimodal function [1]. The niching EAs tends to carry out in parallel exploitation of a number of different niches in a multimodal optimization problem, and allows only a subset of individuals (called a species) to adapt to a niche (called the local adaptation) [2,3,12,21]. Goldberg defined the niche and speciation techniques as having the following functions required (pp. 185–197 [1]): (1) stable and non-competitive subpopulations, formed around all peaks or niches, serve different domains of a function, and the uncontrolled growth of a particular species within a population is limited; (2) it is expected to allocate subpopulations to peaks in proportion to their magnitude (fitness). The principle of the fitness sharing was initially proposed by Goldberg and Richardson [9] to deal directly with the location and preservation of multiple solutions in the GAs. With the fitness sharing approach, the search landscape is modified by reducing the payoff in densely populated regions. It reduces the fitness of an individual by an amount proportional to the number of similar individuals in the population. A sharing function is defined to calculate the niche counts which are further used, immediately prior to selection operation, to derate the fitness of individuals in densely populated subspaces. The most commonly used sharing function has the following form:

shðdi; j Þ ¼

8 <

1

:

0



di; j

s share

a

if di; j < s share ;

(2.1)

otherwise

where sshare, representing the threshold of dissimilarity, is the niche radius defined for specific problems. a controls the shape of the sharing function, and the triangular sharing function is obtained with a = 1. di,j is the distance between two individuals {i,j} in the current population P, i,j = 1,2,. . .,N, where N denotes the population size. The similarity between individuals for the realcoded EAs is computed by the Euclidean distance in the real-valued space. For each individual i, there are some individuals that are measured as similar by sshare and with whom the raw fitness fi is to be shared. Its niche counts mi is calculated as: mi ¼

N X shðdi; j Þ: j¼1

(2.2)

833

Hence, the shared fitness of individual i with raw fitness fi is given by: f sh ðiÞ ¼

fi : mi

(2.3)

By fitness sharing, the replicas and offspring of an individual are produced in proportion inversely to its similar ones in the same niche, and even the elitists could not take over the population, which means that the fitness sharing scheme is able to counterbalance the genetic drift. Therefore, the fitness sharing techniques enable the exploration and exploitation of the fitness landscape by favoring the formation of stable subpopulations. With the properly parameterized fitness sharing (population size N, and niche radius sshare), the niching equilibrium is eventually reached, where all of the individuals are distributed among niches according to their fitness, and all species of the identified niches are maintained to the final population. 3. Review of related work The fitness sharing is an effective implicit niching scheme, where we do not need to identify niches by implementing heuristic procedures (such as clustering methods) that are commonly used in the explicit niching, nor do we use multiple-demes paradigm of the EAs by employing multiple segregated populations. Since the fitness sharing aims at identifying multiple niches in a single population, a large population size is usually required to accommodate the coevolution and maintenance of multiple species, and individuals in the same niche are identified to share resource. Since the introduction of the fitness sharing by Goldberg and Richardson in 1987, there have been more works about the application of fitness sharing techniques than theoretical researches. It was proved that the shared fitness of individuals in two niches h,k at the niching equilibrium is equal [1,12,21]: fh f ¼ k; mh mk

(3.1)

where fh, fk are the raw fitness of the peaks in the two niches. Deb, and Goldberg [1] investigated the parameterization of the niche radius, and compared the performance of the fitness sharing scheme and De Jong’s crowding on multimodal functions in the paradigm of the GAs. Horn [22] employed finite discrete-time Markov chain to investigate the niching pressure of the GAs on the steady-state distribution of the population with the perfect sharing and overlapped sharing. He used the one-gene binary chromosome to analyze the effects of the fitness sharing on the niching equilibrium of population when the species are converged in all niche. Horn and Goldberg [23] derived the expected proportion of individual distribution when the niching equilibrium is achieved in more general case. Suppose that there are two niches A, B with raw fitness fA, fB, and the species in each niche is converged and all individuals in a species have identical fitness. For perfect sharing (no overlap between species when ssh is set properly), the individuals in a niche only contribute to the niche counts in the same niche. Otherwise, it is called the overlapped sharing (when ssh is greater than the distance between two niches). There are following results for the niching equilibrium. (a) Perfect sharing: rf ; P A;eq ¼ rf þ1

(3.2)

where PA,eq is the proportion of individuals adapted to niche A at the niching equilibrium, rf = fA/fB. It is noted that the niching

834

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

GAs get to equilibrium in only one generation with perfect sharing, no matter what the initial individual distribution is. (b) Overlapped sharing: P A;eq ¼

ðr f  1Þs sh þ 1 ; rf þ1

(3.3)

where the distance between niches A, B are treated as 1. The niching GAs need more than one generation to get to equilibrium with overlapped sharing. The expected convergence time is related to the individual distribution in the initial population, the population size, and parameters rf and ssh [21,23]. In order to improve the sharing efficiency, two scaling techniques: the b power sharing f sh ðiÞ ¼ fi =mi (b > 1), and the root sharing f sh ðiÞ ¼ ffiffiffiffiffiffi p b f i = mi were proposed [21,24], which were able to increase the carrying capacity of the higher fitness niches at the expense of losing smaller fitness ones. Darwen and Yao [25] investigated the dilemma of the power scaling in the fitness sharing, and pointed out that a big population was needed for the fitness sharing for the power scaling to work. Recently, Cippa et al. [18], studied empirically the dynamic behavior of the fitness sharing EA with niche radius sshare and population size N, and showed empirically the evidence on the key role played by the niche radius on the performance of the fitness sharing. Only when the niche radius is equal to the actual width of the niches, there is a perfect correspondence between the niches found by the algorithm and the actual number of peaks on the fitness landscape. A procedure for finding the optimal values of niche radius sshare and population size N was proposed, and it provided the estimates of both the population size and the niche radius when there was a perfect peak discrimination on the fitness landscape. However, when the peaks are not equidistantly located, there is not a one-toone peak-niche correspondence although it could still provide some heuristic information. This procedure may become prohibitive in computation for real-world multimodal optimization problems. However, we are still very confined in understanding the dynamics and behavior of the fitness sharing when it is used to solve real-world problems. Experiments on specific functions can sometimes reveal evidences to our intuitions, but theoretical analyses are urgently needed. What we attempt to do in this paper is to make a systematical investigation on the dynamics of the FSEA with the proportionate selection based on normal models, and further to evaluate the effects of the parameterization of the niching radius on the performance of the FSEA. And more, we tend to present our discussions in respect of the niching quality, niching convergence, and niching efficiency. 4. Normal model of FSEA

subsets, and each division is represented as a point. For a fitness function, the definition domain [a,b] is divided into K subsets, Dx = (b  a)/K, and the fitness of individuals belonging to each subset is denoted as f ðxk Þ ¼ f ðððk  1Þ  Dx þ k  DxÞ=2Þ, k ¼ 1; 2; . . . ; K. We assume that the niche radius is set as sshare = l, and number of individuals belonging to kth subset is denoted as nk (k = 1,2,. . .,K). Then the niche counts of the kth subset of individuals is calculated as: mk ¼ mk;þ þ mk;0 þ mk; ;

(4.1)

where mk; ¼

  dð j; kÞ n j; 1 l j¼maxf1;klg k1 X

¼ nk ;

mk;þ ¼

minfkþl;Kg X  i¼kþ1

1

dð j; kÞ ¼ k  j;  dðk; iÞ ni ; l

mk;0

dðk; iÞ ¼ i  k:

Let us define:      k  ðmaxf1; k  lgÞ k j ;...; 1  ;...; Q k ¼ 0; . . . ; 0; 0; 1  l l      1 1 ik ;...;  1  ; 1; 1  ; . . . ; 1  l l l    minfl; i  kg ; 0; 0; . . . ; 0 ; NP ¼ ðn1 ; n2 ; . . . ; nK n ÞT ;  1 l where the numbers of consecutive entries ‘‘0’’ in the left or the right of the vector Qk are max{0,k  l  1} or max{0,K  k  l}, respectively. Then formula (4.1) can be rewritten as: mk ¼ Q k  NP :

(4.2)

Consider all subsets of individuals, we get the niches counts vector: 2 3 2 3 Q1 m1 6 m2 7 6 Q 2 7 7 6 7 (4.3) M¼6 4 . . . 5 ¼ 4 . . . 5  NP : mK n Q Kn 4.2. n-Dimensional real-valued function n Y For the n-dimensional real-valued function f ðXÞ; X 2 ½a j ; b j , the niche counts of individuals in the cell X i ¼ ðxi1 ; xi2 ; . .j¼1 . ; xin Þði ¼ n 1; 2; . . . ; K Þ is computed as: n

mi ¼

K X shðdi; j Þ  n j ;

(4.4)

j¼1

For the sake of generality, we consider the n-dimensional realvalued optimization function f(X), X ¼ ðx1 ; x2 ; . . . ; xn Þ, n Y X 2 ½a j ; b j   Rn . A dimension domain [aj,bj] (j = 1,2,. . .,n) is j¼1

divided into K subsets, the span of a subset is Dxj = (bj  aj)/K, and the real solution space is split into the Kn n-dimensional cells with equal size. A cell is denoted as a point X ¼ ðx1;k1 ; x2;k2 ; . . . ; xn;kn Þ, x j;k j ¼ ððk j  1Þ  Dx j þ k j  Dx j Þ=2, its fitness is f(X), j = 1,2,. . .,n, kj = 1,2,. . .,K. All cells are arranged into a list X ¼ fX 1 ; X 2 ; . . . ; X K n g according to a specific order, and the niches counts vector is represented as M ¼ ðm1 ; m2 ; . . . ; mK n Þ. The niche counts are computed by consider two cases. 4.1. One-dimensional real-valued function For the one-dimensional real-valued function f(x), x 2 [a,b], and the definition domain is divided evenly into multiple segments or

where di; j ¼ jjX i  X j jj2 , nj is the number of individuals in jth cell. In order to implement the operations of matrix algebra, we define the niches counts as a diagonal matrix 2 3 m1 0 . . . 0 6 0 m2 . . . 0 7 6 7 W ¼ 6 .. .. 7, and define the fitness of all subsets as .. .. 4 . . 5 . . 0 0 . . . mK n T

a vector F ¼ ð f 1 ; f 2 ; . . . ; f K n Þ . Then the shared fitness is computed as: Fsh ¼ W1  F:

(4.5)

The proportionate selection will produce a new population with survival probability of individuals in all subsets as: P0 ¼

Vsh  NP FTsh  NP

;

(4.6)

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

2

where Vsh

f sh;1 6 0 6 ¼6 . 4 .. 0

0 f sh;2 .. . 0

... ... .. . ...

0 0 .. .

3 7 7 7: 5

f sh;K n

The above formula can be changed into the representation of the proportion of individuals or the individual distribution as between tth generation and t + 1th generation: Pðt þ 1Þ ¼

Vsh ðtÞ  PðtÞ FTsh ðtÞ  PðtÞ

(4.7)

;

where PðtÞ ¼ ½P 1 ðtÞ; P 2 ðtÞ; . . . ; P K n ðtÞT . It describes the non-linear changes of the proportion of individuals in the population with regard to generation. By starting from the initial population, we get: t1 Y

PðtÞ ¼

Vsh ðt Þ  Pð0Þ

t ¼0

FTsh ðt  1Þ 

t2 Y

:

manipulate to study the dynamics and behavior of fitness sharing, the formula (4.8) serves our purpose very well. It is assumed that the initial individuals are generated uniformly in the definition space, the individuals distribution of initial population might be taken ideally as equal for all subsets or cells, n0 = N/Kn. The niche counts are calculated regarding the onedimensional and two-dimensional real-valued functions as follows. 4.3. Niche counts for real-valued functions The niche counts for the one-dimensional real-valued function are calculated by considering three types of subsets. P (a) For the leftist subset, we have: mle ftist ¼ n0 li¼0 ð1  ði=lÞÞ ¼ n0  ððl þ 1Þ=2Þ. P (b) For the rightist subset, we have: mrightist ¼ n0 lj¼0 ð1  ll jÞ ¼ n0  ððl þ 1Þ=2Þ. (c) For subsets that have neighbors on both left and right:

(4.8) 2

Vsh ðt Þ  Pð0Þ

t ¼0

3    k1 X ik k j 5 þ1þ 1 1 l l j¼maxf1;klg i¼kþ1     lr ðlr þ 1Þ l ðl þ 1Þ þ 1 þ ll  l l ¼ n0  lr  2l 2l

mk ¼ n0 4

This formula illustrates the dynamics of fitness sharing under the proportionate selection for n-dimensional real-valued functions. Remember that our goal is to build a normal model that is easy to

835

minfkþl;Kg X 

Fig. 1. Initial niche counts of fitness sharing and boundary effect.

836

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

where 1 k  K, lr = min(l, K  k), ll = min(l,k  1). When 1 < l < K, we have mk  mleftest and mk  mrightest, as shown in Fig. 1, which we call the boundary effect of niche counts in fitness sharing. The niche counts for the two-dimensional real-valued function are calculated based on formula (4.4), and are shown in Fig. 1(b). The boundary effect becomes more evident when a bigger niche radius is used, and it is also true for high-dimensional real-valued functions. The boundary area in the definition domain has smaller niche counts. Since the niche counts are used to derate the raw fitness, the share fitness of the individuals in the boundary area will has a smaller derating. This will lead to a bigger survival probability of these individuals in the boundary area, which forms the species that do not correspond to real niches. It is called that these species correspond to fake niches. The boundary effects

spread over the definition space, and there appear fake niches on the whole landscape of the shared fitness. The boundary effect of niches counts is good for maintaining niches located at boundaries of the solution space while derating more hardly the niches locate in the interior area of the definition space. Generally speaking, when the niche radius is quite small compared with the span of the definition space, it does not change significantly the dynamics of the fitness sharing. But it may lead to fake niches (niches with higher density of individuals on the shared fitness landscape, but they are not the real ones on the original landscape) which we will discuss later. The normal model constitutes a good platform to simulate and analyze the properties of the FSEA. It is a powerful tool to investigate the dynamics and behavior of the FSEA with regard to

Fig. 2. Shared fitness landscape of the NMS and FSEA.

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

837

Fig. 2. (Continued )

the niching radius sshare, and multiple facets about the fitness sharing can be observed in the simulations by setting different parameters. 5. Empirical investigation of normal model and FSEA In this section, the normal model is employed to simulate the generational FSEAs with real-coded representation. In order to get

intuitive insights about the normal model simulation (NMS), we select a set of special functions for experiments in Section 5.1. In Section 5.2, we use a big population to do simulations of the normal model so that we can observe the effects of the niche radius on the niching equilibrium of the population under fitness sharing. We also conduct experiments of the FSEA without crossover and mutation operations so as to compare the dynamics and behavior of the NMS and the FSEA.

838

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

Fig. 2. (Continued ).

5.1. Test functions A set of typical functions (the maximization is assumed) are selected to do simulations with the NMS and the FSEA so as to investigate the properties of the fitness sharing. A monotone function is included as well to investigate interesting facets of the fitness sharing.

{m1 = (0.3,0.3), m2 = (0.7,0.7)} and {r1 = 0.15  2, r2 = 0.15  2}. This function is represented as f b;1d and f b;2d for the 1dimension and 2-dimension, respectively. b regulates the height of the two peaks. The two peaks and their attraction basins are labeled as A and B niches from the origin to 1 or (1,1). (c) Multimodal sine function: f c ðXÞ ¼

(a) Monotone function: vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u n u1 X x2 ; xi 2 ½0; 1; i ¼ 1; 2; . . . ; n; f a ðXÞ ¼ t n i¼1 i

where n = 1 and n = 2 are considered in experiments. This function is represented as f a;1d , and f a;2d for the 1dimension and 2-dimension, respectively. n   jjXm1 jj2 ;b (b) Two-modal Gaussian function: f b ðXÞ ¼ max exp   2 r 1 jjXm2 jj2 g; xi 2 ½0; 1; i ¼ 1; 2; . . . ; n;where {m1, m2}, {r1, exp r2 2 r2} are the centers and widths of the two radius basis functions. For the 1-dimensional function, the center and width are set as {m1 = 0.3, m2 = 0.7} and {r1 = 0.15, r2 = 0.15}. For the 2dimensional function, the center and width are set as

n X sin6 ð5pxi Þ;

xi 2 ½0; 1; i ¼ 1; 2; . . . ; n;

i¼1

where the 1-dimensional function has five peaks that are equally spaced at {0.1, 0.3, 0.5, 0.7, 0.9} with equal peak magnitude 1, and the 2-dimensional function has 25 peaks (with equal peak magnitude 2) that are located at {0.1, 0.3, 0.5, 0.7, 0.9} for each coordinate. This function is represented as fc,1d and fc,2-d for the 1-dimension and 2-dimension, respectively. (d) Multimodal exponential sine function: n h i X 2 3=4 f d ðXÞ ¼ e2ðln 2Þðððxi 0:1Þ=0:854Þ Þ sin6 5pðxi  0:05Þ ; i¼1

xi 2 ½0; 1; i ¼ 1; 2; . . . ; n; where the 1-dimensional function has five peaks that are unequally spaced at {0.080, 0.247, 0.451, 0.681, 0.934} with

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

decreasing peak magnitude {1.000, 0.948, 0.770, 0.503, 0.250} and the 2-dimensional function has 25 peaks that are located at {0.080, 0.247, 0.451, 0.681, 0.934} for each coordinate. This function is represented as fd,1-d and fd,2-d for the 1-dimension and 2-dimension, respectively. For both fc,1-d and fd,1-d, the five peaks and their attraction basins are labeled as 1st, 2nd, 3rd, 4th, and 5th niches from the origin to 1 on the horizontal axis. 5.2. Measures for performance estimation What we care in the simulation experiments are the landscape, population diversity, niching quality, niching convergence and efficiency with regard to the niching radius sshare when a big population size is used. The measures for these above features are defined as follows. (a) Landscape: A group of intuitive graphs are used to describe the shared fitness function, which shows clearly the multimodal landscape at the niching equilibrium with different niche radius. The landscape of the final population provides information about the niching quality, niching convergence and efficiency as well. (b) Population diversity: The population diversity is defined to measure the evolution process of the NMS and the FSEA with the niche radius. We adopt the Shannon entropy as the metric for the population diversity: K X Pk logðP k Þ;

(5.1)

k¼1

where 0 < D(P)  n log(K), K > 1. A bigger value of D(P) means that the landscape at the niching equilibrium of population with fitness sharing is closer to the original landscape of the fitness function with regard to the niches maintenance or local features conservation. (c) Niching quality: Instead of using the x2-like performance measure [10,15], the standard x2 statistic test is employed to measure the difference between the ideal niching equilibrium and experimental results of the individual distribution in the final population: Q x ðPÞ ¼

c X ðNPi  neq;i Þ2 i¼1

neq;i

the peak in the ith niche, i = 1,2,. . .,c (c is the number of niches of a function). If Qx(P) is small, the actual distribution is close to the ideal one. Otherwise, the actual distribution is dissimilar to the ideal one. (d) Niching convergence and niching efficiency: When the convergence is associated to the niching EAs, its meaning is quite different to the standard EAs or GAs. The convergence of niching EAs with fitness sharing describes the deviation of individuals in species adapting to different niches. When all individuals of a species adapted to a niche are copies of the peak point, it is called that the species or the subpopulation converges to the niche, or the niching convergence is achieved. So, we define the niching convergence as: vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Ni c u u1 X 1X t CðPÞ ¼ ðx  xi Þ2 ; (5.3) c i¼1 N i j¼1 i; j where xi,j is an individual belonging to niche i, xi is the peak point in niche i, Ni is the number of individuals adapting to niche i, j = 1,2,. . .,Ni, i = 1,2,. . .,c. The niching efficiency is a metric of computation for achieving niching convergence. Just as the standard EAs or GAs aiming to obtain the global optimum in fewer generations, the efficiency of the niching EAs is measured by the minimum generations for the population to get to a certain degree of niching convergence. t  ¼ minftj½CðPt Þ  e ^ ½Lg;

n

DðPÞ ¼ 

;

(5.2)

P where Q x ðPÞ  0; neq;i ¼ f i = cj¼1 f j is the ideal number of individuals that should adapt to ith the niche, fi is the fitness of

839

(5.4)

where L denotes the parameter set of the niching EAs, including population size N, niche radius s, etc. Since we only consider the algorithm performance when all niches are maintained, a relative minimum generation in terms of the niching convergence is designed as: tr ¼

CðP; T max Þ  T max ; CðP; 0Þ

(5.5)

where Tmax is the maximum generation of evolution before which the population reaches the niching equilibrium, C(P,0) and C(P,Tmax) are the niching convergence measures of the initial population and final population separately. 5.3. Simulations and analyses When we use the proposed normal model to simulate the behavior of the FSEA, the initial population is generated in two ways: the ideal uniform distribution and random uniform

Fig. 3. Boundary effects (observed via individual distribution) on the fitness landscape.

840

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

Fig. 3. (Continued )

distribution. The former sets the number of individuals in all subsets or cells of the solution space as equal, while the latter just produce individuals randomly in the definition domain. For the 1-dimensional and 2-dimensional functions, the division number of definition domain with each coordinate is set as K = 128 and K = 64, and the population size is fixed as N = 3200 and N = 40,960. The maximum generation is taken as Tmax = 30. The experimental results are averaged over 30 independent executions. The t-test is implemented with a confidence level of 95% whenever it is employed to check the difference of the NMS and the FSEAs with regard to a performance measure.

5.3.1. Landscape and boundary effects First, let us observe the changes of shared fitness landscape with different niche radius for the NMS and the FSEA separately, see Fig. 2. The fitness landscape of the NMS is an exact simulation for the fitness sharing mechanism with the infinite population, whereas the FSEA only presents an approximation to the fitness landscape under finite population. The shared fitness of a division of the definition domain is not calculated when there is no individual in it, so the shared fitness landscape of the FSEA on test functions is rugged because of the randomness of the proportionate selection. But the envelope of the shared fitness landscape resembles that of the NMS on all test functions.

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

841

Fig. 3. (Continued )

There is a good similarity between the shared fitness landscape of the NMS and that of the FSEA on the test functions with different niching radius (figures with other radius values are not drawn here), which illustrates that the NMS is able to emulate the mechanism and performance of the FSEA. What we find with the NMS reveals exactly the characteristics of the FSEA with a big

population. Thus, the NMS constitutes a manipulable platform to study the dynamics and behavior of the FSEA. The boundary effect leads to fake niches on the fitness landscapes effect, see Fig. 3. Some species at the niching equilibrium are not located at the real niches, such as fa,1-d and fb,1-d (Fig. 3(a.1)(a.2)(b.1)(b.2)) when the smaller niche radii are

842

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

Fig. 3. (Continued ).

used. This happens when the niche radius is smaller than the ideal value, sshare < 1 for fa,1-d and sshare < 0.4 for fb,1-d. For the fc,1-d and fd,1-d, we can see another type of the boundary effect (Fig. 3(c.1)(c.2)(d.1)(d.2)). The individual distrbution among niches at the niching equilibrium do not depend on the fitness of niches on the landscape of the fc,1-d and fd,1-d when the niche radius is bigger than ideal values (sshare < 0.2). Althrough the fc,1-d has equally spaced niches with equal fitness, the number of individuals adapting to each niche is not equal. The first and last niches take a good advantage over the second and fourth ones when 0.5 > sshare < 0.2 because of the boundary effect of the fitness sharing. Meanwhile, the third niche enjoys more resource while the second and fourth ones in neighbourhood are deprived. If sshare  0.5, there are fewer and fewer individuals adapting to the

second, third, and fourth niches. The first and last niches nearly take over and share most of the population when sshare = 1. This is called the biased individual distribution at niching equilibrium. For the 2-dimensional functions, there are species that are not located in niches on the landscape of the fa,2-d or fb,2-d with the niche radius sshare = 0.5 or sshare = 0.25, see Fig. 3(e.1)(e.2)(f.1)(f.2). pffiffiffi This happenspdefinitely for the fa,2-d with s share < 2 and the fb,2-d ffiffiffi with s share < 2=2. A significant part of individuals are attracted by these species at equilibrium in the final population, and they do not contribute to the exploration or exploitation of the global or local optima. There also exist the biased individual distribution at pffiffiffi equilibrium on the fc,2-d and fd,2-d with s share > 2=5, see Fig. 3(g.1)(g.2)(h.1)(h.2). The niches located (0.1,0.1), (0.1,0.9), (0.9,0.1), (0.9,0.9) are more competitive and attract more

Fig. 4. Comparison on niching measures with the NMS and FSEA.

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

843

Fig. 4. (Continued )

pffiffiffi individuals than others for the fc,2-d with s share > 2=5. As for the fd,2-d, the species adapting to niches located (0.080,0.080) is the biggest one, and boundary niches pffiffiffialigning to each coordinate are more competitive with s share > 2=6 approximately. The fake niches or the biased individual distribution at niching equilibrium provide false information about the real fitness lanscape althrough the shared fitness is correct. The phenomena, caused by the boundary effect of the fitness sharing, should be examinated after the termination of the algorthm to make sure what the species adapt to at niching equilibrium are real niches or fake ones, or to check the bias in the individual distribution at equilibrium among niches for the sake of finding the true features of niches.

5.3.2. Population diversity and niche convergence The NMS and the FSEA share similar relationship between the niche radius and any of the three measures (population diversity, niche convergence, x2 statistic (Chi-square)), see Fig. 4. Although there are still some minor gaps between the two curves especially when the niche radius gets bigger because of the randomness of the proportionate selection in the FSEA, the correlation coefficients of a measure on all test functions reveal the statistically significant consistency of performance with regard to the NMS and the FSEA (see Table 1). As for the niching quality of the fitness niching with the x2 statistic, all niches can be maintained with a smaller niche radius when a big population size is employed, for instance the fb,1-d with

844

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

Fig. 4. (Continued ).

s share < 0:4, the fc,1-d and fd,1-d with s share < 0:2. They accommodate approximately species according to the fitness-dependent distribution among all niches. Otherwise, the individual distribution at equalibrium has an evident divergence to the ideal ones. It is

interesting to note that the fitness sharing works well when the niche radius is smaller than the ideal value in terms of the x2 statistic. In contrast to the empirical conclusion that it is critical to find the ideal niche radius for the fitness sharing to perform well in

Fig. 5. Niching quality of the NMS and FSEA with different niche radius (ideal indicates the individual distribution at equilibrium exactly proportional to the niche fitness).

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

845

Table 1 Correlation coefficients of performance estimation measures with NMS and FSEA. Measures

Population diversity Niche convergence x2-Statistic

Test functions fa,1-d

fb,1-d

fc,1-d

fd,1-d

fa,2-d

fb,2-d

fc,2-d

fd,2-d

0.9488 0.8909 –

0.9843 0.9836 0.9986

0.9753 0.9986 0.9903

0.9849 0.9953 0.9914

0.9245 0.9245 –

0.9968 0.9968 0.9996

0.9881 0.9881 0.9707

0.9754 0.9754 0.9851

convention researches [1,9,13,15,18], this empirical investigation based on the NMS and the FSEA illustrates that we do not require an exact value of the niche radius to maintain all niches according to the x2 statistic. As shown in Fig. 5, there is no significant difference by the t-test between the NMS and the FSEA on all of the test functions with regard to the individual distribution at equilibrium with different values of the niche radius. As for the 2-dimensional test functions fb,2-d, fc,2-d, and fd,2-d, similar results are obtained regarding the individual distribution at equilibrium (see Fig. 4(e)–(h)). Fig. 5 indicates that the fitness sharing is able to maitain all niches when the niche radius is smaller than a threshold (sshare < 0.5 for fb,1-d, sshare  0.1875 for fc,1-d and fd,1-d acccording to the discretized niche radius), and the individual distribution among niches at equilibrium is closely fitness-proportionate. Thus, we have quite a big feasible range for the parameterization of the niche radius. For multimodal optimization problems, a niche

radius smaller than the smallest distance between any pair of niches is able to realize the coevolution of mutliple species with the individual distribution being aproximately fitness-proportionate amomg all niches. This discovery reveals a novel property of the fitness sharing scheme. 5.3.3. Niching equilibrium with niche radius In order to compare the individual distribution at niching quilibrium of the NMS and the FSEA with the ideal cases by formula (3.2) for the perfect sharing and formula (3.3) for the overlapped sharing, we carry out experiments on the function fb,1d, and the results are shown in Fig. 6. Similar results are obtained on the fb,2-d. With b = 2 and r1/r2 = 1 for fb,1-d, the results of the NMS and the FSEA are consistent with the perfect sharing or overlapped sharing by the t-test, which indicates that formulas (3.2) and (3.3) hold for functions with niches of approximately equal attraction basin sizes

Fig. 6. Individuals distribution at equilibrium of the NMS and FSEA on fb,1-d.

846

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

Fig. 7. Niching efficiency (tr ) of the NMS and FSEA with different niche radius.

although the niching convergence are not achieved with sshare < 0.4 (Fig. 6(a)). For the NMS and the FSEA on fb,1-d with b = 2 and r1/r2 6¼ 1, it is observed that the inidividual distribution among niches is quite different by the t-test from the perfect sharing sharing. When the first niche has a bigger niche radius than the second one (r1 > r2), the species adapting to the niche A has more individuals than the perfect sharing with sshare < 0.4; and the larger of r1/r2, the bigger of the number of individuals adapting to the niche A. In contrast, for r1 < r2 with sshare < 0.4, the niche A will attract fewer individuals; and the lower of r1/r2, the smaller of the number of individuals adapting to the niche A. For all cases of fb,1-d with sshare > 0.4, the niching equilibria of both the NMS and the FSEA are very close to the overlapped sharing (formula (3.3)). If sshare > 1.0 for fb,1-d, the niche A will be taken over totally by the niche B. These results verify the correctness of the niching equilibrium formula (3.3) with converged niches.

5.3.4. Niching efficiency When the FSEA is applied to real-world multimodal problems, it is usually aimed to achieve efficiently niching convergence in the final population, where the subpopulations adapting to different niches all converge to the peak points of niches. The niching efficiency is also an important measure on the performance of the FSEA. Fig. 7 illustrates the niching efficiency (relative minimum generation tr ) of the NMS and the FSEA with different niche radius. It is only when the niche radius takes values bigger than the ideal that the NMS and the FSEA are able to obtain a good niching efficiency. However, it is very difficult to find the ideal niche radius without a priori information about a problem, and there is not an ideal niche radius on functions with non-equal spaced niches. There is a difficulty on the parameterization of the niche radius regarding niching efficiency, which has been shown in Fig. 5. When a radius is bigger than the ideal value, the competition between

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

niches may lead to genetic drift while only competitive niches are maintained. On the other hand, smaller radius usually yields fake niches, and it is hard to achieve the niching convergence for all niches. This is also proved saliently on the monotone function (fa) in terms of the niching convergence measure C(P). The NMS and the FSEA yield divergent populations with a small radius (sshare < 1 for pffiffiffi fa,1-d, s share < 2 for fa,2-d). The final population gets more converged when sshare takes bigger values, pffiffiffi and become fully converged if sshare > 1 for fa,1-d and s share > 2 for fa,2-d. The bigger niche radius compared with the ideal one is surely necessary for achieving good niching convergence and efficiency. The proper parameterization of sshare to meet the requirements of both maintaining niches and achieving niching efficiency is actually a dilemma, which becomes more intricate on the multimodal functions with non-equal spaced and non-equal height niches. This is consistent with the conclusion of conventional researches [16,18]. 5.4. Discussions In the studies of Goldberg et al. [24], and Horn [21], it was suggested to use power sharing or root sharing to improve sharing efficiency. But this strategy gave rise to the disappearance of niches with smaller fitness. Darwen and Yao [25] studied the behavior of the fitness sharing with power scaling, and presented an annealing power scaling method. To fix the issue with the niching efficiency, the mating restriction was firstly proposed [10] to achieve a good x2-like measure. The clustering and dynamic clustering methods or distance-based niche identification methods were employed to estimate the diverse radii of niches and to find the center and width of niches [4–8,11,13,14,16,17,26] by using heuristic procedures when apriori knowledge is avaliable about a problem. These approaches resort to specific procedures (clustering or distance-based niche identification) to identify niches and estimate niche radius, which usually introduces extra parameters and is also computational prohibitive when large multimodal problems with a lot of non-equal spaced niches are handled. What we discover from the experimental results of the NMS and the FSEA is very illuminative to the application of the FSEA to realworld multimodal optimization problems. Instead of trying to find optimal niche radius to achieve both good niching quality and niching convegence (and niching effciency), why do not we adopt the hybrid schemes to realize the two goals? The fitness sharing is a very effective technique to locate and maintain niches when the niche radius smaller than the smallest size of niches is employed. In other words, supposed that we take sshare,minimum as the threshold to filter niches with radii greater than it, it is theoretically sound to use sshare,minimum as the niching radius. Meanwhile, we are exempted of the task of finding different radii for niches with diverse attraction basin sizes. Then local search algorithms are then used to find the peak points in niches, which are done only on the promising points in the population. The promising points are those with both bigger niche counts and bigger fitness values in the hyperspherical neighborhood with a diameter of 2sshare,minimum. Deb and Goldberg [10] suggested that the radius was estimated by calculating the smallest n-dimensional hypersphere containing ffi pffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn u  xl Þ2 , a niche in the feasible space: s share ¼ ð1=2 n Q Þ ðx k¼1 k k where fxuk ; xlk g are the upper and lower bounds of kth dimension of the domain. Q denotes the estimated number of niches. It can only be used in cases where the niches are evenly distributed over the feasible solution space. When the population reaches niching equilibrium by the measure of population diversity or niching convergence, various

847

local search methods may be utilized to search the local fitness landscape of niches which are usually unimodal. Here, the FSEA is only responsible for explorative searching of feasible solution space and maintaining niches. In contrast, the local searching methods are assigned to do exploitative refining of the local optimal solutions which can be identified by a heuristic procedure based on niche counts of individuals. This hybrid scheme is simple to control, easy to implement, and meets the twin goals of both niching convergence and niching efficiency. Hybrid schemes of the EAs and other local search methods also proved their efficiency in the memetic algorithms (MA) for solving global optimization problems [27]. When we look back further into the research domain of global optimization, we can find that most of the stochastic searching methods [28] divided the searching process into two phases (global phase and local phase) to guarantee the global optimality and convergence. This two-phase scheme was also adopted in the GAs-based methods for global optimization [2] just as what was suggested by Holland [29]: genetic algorithms should be used as a preprocessor to perform the initial search before tuning the search with local methods. The work by Yin and Germany [11] concluded that a simple extension (to incorporate local search algorithms) such as hillclimbing (or other gradient-based search methods) is necessary, for which we provide theoretical and empirical proof in this paper. However, there has been no attempt based on this hybrid scheme in the paradigm of the single population FSEA. We have done tentatively experiments to test the real-valued multimodal functions with the two-phase hybrid scheme of the fitness sharing FSEA and the simplex method, where the simplex method is a good non-linear local search algorithm. The initial experimental results are promising, and further work is needed to be done on more multimodal problems. 6. Conclusions The main contribution of this paper is the formulation of the normal model of fitness sharing with the proportionate selection on real-valued function. The normal model constitutes a good platform to investigate the dynamics and behavior of the FSEA with regard to the niche radius sshare. Experiments on test functions prove the consistency of the NMS and the FSEA, and illustrate the efficacy of the normal model simulation for studying the characteristics of the fitness sharing technique. From the experimental results of the NMS and the FSEA, there exists the dilemma to find the optimal niche radius to achieve good niching quality, niching convergence and niching effciency. The proposal is to devise the hybrid schemes to realize the two goals. The FSEA is first used to explore and maintain niches with a niche radius smaller than the smallest size of niches, and local searching methods are then employed to find the peak points of niches. This hybrid scheme enhances the practical applicability of the FSEA to complex real-world multimodal optimization problems by achieveing good niching quality, niching convergence and niching efficiency. The incorporation of crossover and mutation operations into the normal model is to be done so as to get an integrated normal model of the FSEA. Meanwhile, the proposed hybrid scheme is further investigated empirically to verify its effectiveness and efficiency, and initial results look quite promising. Acknowledgments The work was supported by the National Science Foundation of China for Distinguished Young Scholars (Grant No. 70925005) and by the Program for New Century Excellent Talents in Universities of China (NCET-05-0253).

848

M. Li et al. / Applied Soft Computing 10 (2010) 832–848

References [1] D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley Publishing Company, New York, 1989. [2] T. Ba¨ck, D.B. Fogel, Z. Michalewicz, Evolutionary Computation Bristol and Philadelphia, Institute of Physics Publishing, 2000. [3] A.E. Eiben, J.E. Smith, Introduction to Evolutionary Computing, Springer-Verlag, Berlin, Heidelberg, 2003, p. 2003. [4] C.-Y. Lin, Y.-J. Yang, Cluster identification techniques in genetic algorithms for multimodal optimization, Computer-Aided Civil and Infrastructure Engineering 13 (January (10)) (1998) 53–62. [5] M. Jelasity, P.M. Ortigosa, I. Garcia, UEGO, an abstract clustering technique for multimodal global optimization, Journal of Heuristics 7 (3) (2001) 215–233. [6] F. Streichert, G. Stein, H. Ulmer, A. Zell, A clustering based niching EA for multimodal search spaces, in: Proceedings of the International Conference Evolution Artificielle Volume 2936 of Lecture Notes in Computer Science, Springer-Verlag, 2003, pp. 293–304. [7] J. Alami, A.E. Imrani, A. Bouroumi, A multipopulation cultural algorithm using fuzzy clustering, Applied Soft Computing 7 (March (2)) (2007) 506–519. [8] Q. Ling, G. Wu, Z. Yang, Q. Wang, Crowding clustering genetic algorithm for multimodal function optimization, Applied Soft Computing 8 (January (1)) (2008) 88–95. [9] D.E. Goldberg, J.J. Richardson, Genetic algorithms with sharing for multimodal function optimization, in: Proceedings of the Second International Conference on Genetic Algorithms (ICGA 2nd), Cambridge, MA, USA, July, (1987), pp. 41–49. [10] K. Deb, D.E. Goldberg, An investigation of niche and species formation in genetic function optimization, in: J.B. Schaffer (Ed.), Proceedings of the Third International Conference on Genetic Algorithms, San Mateo, California, June 1989, George Mason University, Morgan Kaufmann Publishers, (1989), pp. 42–50. [11] X. Yin, N. Germany, A fast algorithm with sharing scheme using cluster analysis methods in multimodal function optimization, in: R.F. Albrecht, C.R. Reeves, N.C. Steel (Eds.), Proceedings of the International Conference on Artificial Neural Nets and Genetic Algorithms, Springer, Berlin, August, 1993, pp. 450–457. [12] S.W. Mahfoud, Niching methods for genetic algorithms. Doctoral dissertation, University of Illinois at Urbana-Champaign, Urbana, IL 61801, 1995 (Also IlliGAL Report No. 95001). [13] B.L. Miller, M.J. Shaw, Genetic algorithms with dynamic niche sharing for multimodal function optimization, in: International Conference on Evolutionary Computation, IEEE Press, Piscataway, NJ, 1996, pp. 786–791 (Also IlliGAL Report No. 95010). [14] D.E. Goldberg, L. Wang, Adaptive niching via coevolutionary sharing, in: D. Quagliarella, et al. (Eds.), Genetic Algorithms and Evolution Strategy in Engineering and Computer Science, John Wiley and Sons, Ltd., Chichester, NY, 1998, pp. 21–38 (Also IlliGAL Report No. 97007).

[15] B. Sareni, L. Kra¨henbu¨hl, Fitness sharing and niching methods revisited, IEEE Transactions On Evolutionary Computation 2 (September (3)) (1998) 97–106. [16] J. Gan, K. Warwick, A variable radius niching technique for speciation in genetic algorithms, in: Proceedings of the Genetic and Evolutionary Compuational Conference (GECCO2000), Morgan-Kaufmann, Las Vegas, USA, July 8–12, (2000), pp. 96–103. [17] J.-P. Li, M.E. Balazs, G.T. Parks, P.J. Clarkson, A species conserving genetic algorithm for multimodal function optimization, Evolutionary Computation 10 (Fall (3)) (2002) 207–234. [18] A.D. Cioppa, C. Stefano, A. De Marcelli, On the role of population size and niche radius in fitness sharing, IEEE Transactions on Evolutionary Computation 8 (December (6)) (2004) 580–592. [19] E.J. Solteiro Pires, P.B. de Moura Oliveira, J.A. Tenreiro Machado, Manipulator trajectory planning using a MOEA, Applied Soft Computing 7 (June (3)) (2007) 659–667. [20] K.-S. Hwang, S.-B. Cho, Improving evolvable hardware by applying the speciation technique, Applied Soft Computing 9 (January (1)) (2009) 254–263. [21] Horn, J., The nature of niching: genetic algorithms and the evolution of optimal, cooperative populations. Doctoral dissertation, University of Illinois at UrbanaChampaign, Urbana, IL 61801, 1997 (Also IlliGAL Report No. 97008). [22] J. Horn, Finite Markov chain analysis of genetic algorithms with niching, in: S. Forrest (Ed.), Proceedings of the 5th International Conference on Genetic Algorithms, Morgan Kaufmann Publishers, San Mateo, CA, (1993), pp. 110–117. [23] J. Horn, D.E. Goldberg, A timing analysis of convergence to fitness sharing equilibrium, in: A.E. Eiben, T. Ba¨ck, M. Schoenauer, H.-P. Schwefel (Eds.), Proceedings of Parallel Problem Solving From Nature, PPSN V, Lecture Notes in Computer Science 1498, Springer, Berlin, Germany, October, (1998), pp. 23–33. [24] D.E. Goldberg, K. Deb, J. Horn, Massive multimodality, deception, and genetic algorithms, in: R. Manner, B. Manderick (Eds.), Parallel Problem Solving from Nature 2, North-Holland, 1992, pp. 37–46, September (Also IlliGAL Report No. 92007). [25] P. Darwen, X. Yao, A dilemma for fitness sharing with a scaling function, in: Proceedings of IEEE International Conference on Evolutionary Computation, vol. 1, IEEE Press, Piscataway, NJ, November 29–December 1, (1995), pp. 166–171. [26] C.-Y. Lin, W.-H. Wu, Niche identification techniques in multimodal genetic search with sharing scheme, Advances in Engineering Software 33 (November–December (11–12)) (2002) 779–791. [27] P. Moscato, Memetic algorithms: a short introduction, in: D. Corne, M. Dorigo, F. Glover (Eds.), New Ideas in Optimization, McGraw Hill, 1999, pp. 219–234. [28] A.H.G. Rinnooy-Kan, G. Timmer, A stochastic approach to global optimization, in: P. Boggs, R. Byrd, R.B. Schnabel (Eds.), Numerical Optimization, SIAM, Philadelphia, 1984, pp. 245–262. [29] J.H. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence, 2nd ed., MIT Press, Cambridge, MA, 1992.