- Email: [email protected]
An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems
Journal Pre-proof An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems Enying Li
PII: DOI: Reference:
S1568-4946(19)30533-2 https://doi.org/10.1016/j.asoc.2019.105752 ASOC 105752
To appear in:
Applied Soft Computing Journal
Received date : 7 November 2018 Revised date : 2 August 2019 Accepted date : 2 September 2019 Please cite this article as: E. Li, An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems, Applied Soft Computing Journal (2019), doi: https://doi.org/10.1016/j.asoc.2019.105752. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Β© 2019 Elsevier B.V. All rights reserved.
Journal Pre-proof
1
An adaptive surrogate assisted differential evolutionary
2
algorithm for high dimensional constrained problems Enying Liο College of Mechanical & Electrical Engineering, Central South University of Forestry and Technology, Changsha, 41004, P.R. China Nomenclature
pro of
3 4 5 6
DE
constrained optimization problems
congress on evolutionary computation
CEC
Evolutionary Algorithm
EA
Function Evaluations
FEs
Kriging
KG
Radial Basis Function
RBF
leave-one-out cross validation
LOOCV
Polynomial Regression
PR
Support Vector Regression
SVR
scaling factor
F Np
exclusive integers
the best individual in the population at the iteration t
π 1π ,
π 2π , π 3π (π‘)
target vector
π₯π,π‘
mutation vector
π₯πππ π‘ π£π,π‘
trail vector
π’π,π‘
decision variable
πππππ
inequality constraint
ππ (π₯)
design space
π
tolerance parameter feasible ratio normalized fitness value population
βπ (π₯)
number of inequality constraints
lP
quality constraint
q
constraint violation
π(π₯)
πππππ
overall constraint violation
π£(π₯)
π β²β² (π₯)
penalty value
π(π₯)
πΏ
π(πΊ)
urn a
the kth subpopulation of the Gth generation
πππΊ
present number of the successful evolutions in sth subpopulation
ππ’π
entire population size in the Gth generation
πππΊ
small positive constant
ππ’0
preset minimum population size
πππππ
initial population size
πππππ₯
population size in the generation
πππππ₯
Abstract
Jo
8 9 10 11 12 13
Cr
crossover rate
re-
population size in the tth generation
7
COPs
differential evolution
Differential evolution (DE) is a competitive algorithm for constrained optimization problems (COPs). In this study, in order to improve the efficiency and accuracy of the DE for high dimensional problems, an adaptive surrogate assisted DE algorithm, called ASA-DE is suggested. In the ASA, several kinds of surrogate modeling techniques are integrated. Furthermore, to avoid violate the constraints and obtain better solution simultaneously, adaptive strategies for population size and mutation are also suggested in this study. The
ο
Corresponding author [email protected]
1 / 31
Journal Pre-proof suggested adaptive population strategy which controls the exploring and exploiting states according to whether algorithm find enough feasible solution is similar to a state switch. The mutation strategy is used to enhance the effect of state switch based on adaptive population size. Finally, the suggested ASA-DE is evaluated on the benchmark problems from congress on evolutionary computation (CEC) 2017 constrained real parameter optimization. The experimental results show the proposed algorithm is a competitive one compared to other state-of-the-art algorithms.
8
Keywords
9
Surrogate assisted, Differential evaluation, Adaptive strategy, High dimensional problem
pro of
1 2 3 4 5 6 7
1. Introduction
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
Evolutionary Algorithm (EA) is a useful optimization method for complicated problems and has been applied to multidiscipline [1-5]. Differential Evolutions (DE) is an important algorithm of EAs and attracts lots of scholarsβ attention due to its competitive performance on constrained optimization problems over the past decades [6-8]. Additionally, most of practical optimization problems are Constrained Optimization Problems (COPs), which might lead to the difficulty for convergence. The purpose of this study is to find the global optimization solutions as accurately as possible for COPs. Since the DE was proposed [9], many efforts have been designed to enhance the performance of the DE for constrained problems. Most of them focused on the setting of control parameters such as scaling factor (F), crossover rate (Cr), selection mechanism, mutation strategies and population size (Np). However, parameter settings commonly are diverse for different cases. Furthermore, the interactions between performance and parameter are also complicated. It motivated many scholars to develop some adaptive parameter setting based DEs such as jDE [10], SaDE [11], JADE [12], SHADE [13], IDE [14] and JADE_sort [15]. The essential of these strategies is to tune control parameters in the present iteration according to feedbacks from the previous ones. These controlling strategies are commonly used to adjust Cr and F. Moreover, other kinds of parameter setting strategies were widely used such as EPSDE [16], adaptive DE [17] and CoDE [18], which select the parameters from a pool involving sets of candidate populations. The selection mechanism is another critical issue for the DE. The essential of this mechanism is based on greedy choice in the classical DE. These strategies have also improved the performance of the DE. Population size control is another important mechanism for the DE. Several adaptive or non-adaptive population size control methods have been proposed and achieved competitive performances. For instance, Zamuda et al. [19] controlled the population size based on increasing number of FEs, a set of novel mutation strategies which depending on population size were also suggested. It showed its superior performance on the CEC2011 real life problems. Tanabe et al. [20] employed the linear population size reduction strategy to improve the SHADE algorithm, the population size is reduced with increasing FEs (Function Evaluations) and called as L-SHADE. It will stop if the population size reaches the present minimum (Npmin). The L-SHADE also won the CEC2014 competition on real parameter single-objective optimization.
Jo
urn a
lP
re-
10
2 / 31
Journal Pre-proof
urn a
lP
re-
pro of
Another population size controlling-method called Linear Population Size Reduction (LPSR) requires only one parameter (the initial population size). Similar algorithm Dynamic Population Size Reduction (DPSR) [21] was suggested to reduce the population by half at preset intervals. These non-adaptive methods can increase or decrease (decrease in most cases) the population size according to preset conditions and simplify the method of controlling population size. Moreover, instead of reduction of the population size by fixed formula or predetermined path, several adaptive population size techniques were proposed recently. Yang et al. [22] proposed a novel strategy, which varies the population size in a predetermined range. It will detect the present state and decrease or increase the present population size according to the feedbacks from the previous iterations, and the population size will be expanded by introducing more solutions when the algorithm is unable to find a better solution within a preset number of consequent generations. On the contrary, if the algorithm updates the best solution successfully in successive generations, the population size will reduce in order to purge the redundant solutions. Many experiments and studies have proved the effectiveness of the population size control method. More recently, some novel strategies have been introduced to improve the performance of the DE. Du et al. introduced an Event-Triggered Impulsive (ETI) to improve the performance of DE [23]. Awad et al. presented an enhanced algorithm based on LSHADE with ensemble parameter sinusoidal adaptation called LSHADE-EpSin [24]. The LSHADE-EpSin suggested a mixture of two sinusoidal formulas and a Cauchy distribution to balance the exploration and the exploitation of existed best solutions. A restart method is used at later generations to enhance the quality of the found solutions. By introducing Impulsive Control (IPC) and ETM into DE, they hope to change the search performance of the population in a positive way after revising the positions of some individuals. Wang et al. suggested a self-adaptive mutation DE algorithm based on particle swarm optimization (DEPSO) [25]. Compared with the unconstrained problems, the COPs are more difficult to be handled because both constraint violation and objective function should be considered. The present popular constraint-handling strategies can be mainly categorized three types: penalty function methods [26-27]; methods based on preference of feasible solutions over infeasible ones [28-31]; multi-objective based methods [32-37]. Briefly, the purpose of the COPs is to find the feasible region efficiently and obtain the solution finally. For the DEs, some novel algorithms have been suggested. Wang and Cai [38] proposed (ΞΌ+Ξ»)-CDE framework. They also [39] developed CMODE algorithm by integration of multi-objective optimization with DE. Mohamed and Sabry [40] employed several strategies on the DE, involving mutation operators, F, and CR. All mentioned algorithms used direct evaluation way to obtain the response of objective function. Hernandez et al. [41] suggested a hybridization of DE and hill climbing, which uses static penalty to handle constraints. Sarker et al. [42] developed a DE with dynamic parameter selection. Zhang et al. [43] suggested a constrained artificial immune system based on immune response principle. Considering the number of evaluations, the computational cost and convergence ratio still need to be addressed, especially for the complicated problems. Therefore, Surrogate Assisted Evolutionary Algorithm (SAEA) was motived by this issue [44]. Theoretically, surrogates can be applied to most of operations of evolutionary algorithms, such as population initialization, cross-over, mutation, local search and fitness evaluation. In such algorithms, a surrogate model is used to evaluate the objective function by constructing an
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
3 / 31
Journal Pre-proof approximate model [45]. Over the last few decades, many surrogate models have been proposed, such as, Kriging(KG), Support Vector Regression(SVR), Polynomial Regression(PR), and Radial Basis Function (RBF) [46]. Similar to other algorithms, the DE has been incorporated with surrogate modeling techniques and achieved good results [47-50]. Awad et al. developed a novel DE with a novel adapted KG surrogate model called iDEaSm. During the modeling in the iDEaSm, the covariance matrix is used to adapt the theta parameter in the KG model at each generation [51]. Taran et al. suggested a two-level surrogate-assisted optimization algorithm is proposed for electric machine design [52]. Cai et al. introduced an efficient surrogate-guided DE based on JADE algorithm [53]. In this study, we hope to find an alternative way to improve the efficiency of surrogate assisted DE algorithm. Compared with other surrogate assisted algorithms, such as iDEaSm, several surrogate-modeling techniques are employed. The diversity of surrogate modeling should be enhanced and the best of them should be selected. by the given criteria. Therefore, the suggested method is called as Adaptive Surrogate Assisted DE (ASA-DE) algorithm. To evaluate performance of the ASA-DE, 28 benchmark functions from CEC2017 are tested. The results demonstrate that performance of the ASA-DE is better than, or at least competitive to the stateof-the-art constraint optimization algorithms, and show the competitiveness relative to the other excellent algorithm. The rest of the paper is organized as follows. The classical DE algorithm and several constraint handling methods for constrained optimization are briefly introduced in Section 2, Section 3 gives the detailed description of the proposed ASA-DE, and results of benchmark functions and comparison with other state-of-the-art algorithms are shown in Section 4. Finally, Section 5 summarizes the conclusions and discusses the direction of future work.
24
2. Classical DE algorithm and constraint handling
re-
lP
methods
urn a
25
pro of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
2.1 DE Algorithm
27 28 29 30 31 32 33 34 35 36 37
The process of the DE consists of three main operators: mutation, crossover and selection. Similar to other Evolutionary Algorithms (EAs) for optimization problems, the DE is based on population. A DE population consists of Np individual vectors π₯π,π‘ = (π₯1,π‘ , β¦ , π₯π,π‘ ), π = 1, β¦ , ππ, (1) where n is the number of design variables, Np is the population size in the present generation, the population individual vectors are generated randomly at the beginning of optimization. Np trail vectors are generated from the survived population individual by mutation and crossover operations in each generation t until termination criteria are satisfied. The details of DE are discussed according to Ref. [6] as follows. Mutation Operation: After the initialization, the DE adopts the mutation operation based on the difference of other individuals to generate mutant vector ππ,π‘ with respect to each target vector π₯π,π‘ , in the present generation. Mutation strategies influence the balance of exploration
38
and exploitation during the evolutionary process, some most frequently referred mutation
Jo
26
4 / 31
Journal Pre-proof strategies are listed below: "DE/rand/1": (π‘)
π£π 3
(π‘)
(π‘)
1
2
3
(π‘)
(π‘)
1
2
(2)
"DE/best/1": (π‘)
π£π 4
(π‘)
= π₯π π + πΉ (π₯π π β π₯π π )
(π‘)
(3)
= π₯πππ π‘ + πΉ (π₯π π β π₯π π )
"DE/current-to-best/1": (π‘)
π£π
(π‘)
= π₯π
(π‘)
(π‘)
(π‘)
(π‘)
1
2
pro of
1 2
+ πΉ (π₯πππ π‘ β π₯π ) + πΉ (π₯π π β π₯π π )
(4)
5 6
where the indices π 1π , π 2π , and π 3π are exclusive integers randomly chosen from the range [1, Np], (Np is population size in the tth generation). F is the scaling factor which usually lies in [0,
7
1] for scaling the difference vectors. π₯πππ π‘ is the best individual in the population at the
8 9 10 11 12 13 14 15 16 17
iteration t. Generally, mutation strategies can be classified into three categories: exploring biased, exploiting biased and balance modes. The DE/rand/1 belongs to the type of exploring biased; the DE/best/1 belongs to the type of exploiting biased. Moreover, DE/current-tobest/1 is a balance mode because the diversity is enhanced by adding a difference vector composed of random individuals and the use of best individual. The suitable balance point of exploration and exploitation is updating dynamically in the evolutionary process. Therefore, an adaptive mutation strategy based on feasible rate is suggested in this study to switch state between exploring biased and exploiting biased in order to keep the efficiency of algorithm. Crossover Operation: After the mutation, crossover operation is used to generate trail vector π’π,π‘ by replacing the components between the target vector π₯π,π‘ and the mutation vector π£π,π‘
18 19
with probability. Binomial crossover, which is the most frequently employed crossover operator in the DE used as follows: π
π£π,π‘ ππ ππππ[0,1) β€ πΆπ ππ π = π π ππππ ={ π ππ‘βπππ€ππ π π₯π,π‘
urn a
π π’π,π‘
lP
re-
(π‘)
(5)
where πππππ is a decision variable. Selection Operation: The selection operation based on the greedy algorithm chooses the survived vector from trail and target vectors, and can be represented as π’π,π‘ ππ π(π’π,π‘ ) β€ π(π₯π,π‘ ) π₯π,π‘+1 = {π₯ (6) π,π‘ ππ‘βπππ€ππ π
23 24
The selection operation in the classical DE compares each target vector π₯π,π‘ against corresponding trial vector π’π,π‘ based on the objective function, and keeping the evolution
25
continuing.
26
2.2 Constraint strategies
27 28 29
To solve COPs using EAs, the constraint-handling strategies are very important. A constrained optimization problem is usually represented as a nonlinear programming of the following form for the minimization problem [54] as
Jo
20 21 22
5 / 31
Journal Pre-proof Minimize:
π(π₯), π₯ = (π₯1 , π₯2 , β¦ , π₯π ) πππ π₯ β π ππ (π₯) β€ 0, π = 1, β¦ , π Subject to: βπ (π₯) = 0, π = π + 1, β¦ , π
(7)
where π is the design space, ππ (π₯) is the inequality constraint, and βπ (π₯) is the equality
2 3 4 5 6 7
constraint. The number of inequality constraints is q, (m-q) is the number of the equality constraints. Generally, if the global optimum satisfies the constraints ππ (π₯) = 0, the inequality constraints are called active constraints. Similarly, Active constraints contain all equality constraints. For convenience, equality constraints can be transformed into the form of inequality constraints. The inequality constraints and equality constraints can be integrated into a combined form as πΊπ (π₯) = {
pro of
1
πππ₯{ππ (π₯), 0} π = 1, β¦ , π πππ₯{|βπ (π₯)| β πΏ, 0} π = π + 1, β¦ , π
(8)
8 9
where πΏ is the tolerance parameter which is usually set as 1e-4. The overall constraint violation for a solution can be expressed as
10 11 12 13 14 15 16 17 18 19 20 21 22 23
βπ π=1 πΊπ (π₯) (9) π where π£(π₯) is zero for the feasible solution and positive when at least one constraint is violated [55], and the overall constraint violation is an important parameter in order to guide the evolutionary process towards feasible areas. In this study, three popular constraint handling strategies should be introduced as follows Superiority of Feasible Solution (SF): Two individual π₯1 and π₯2 are compared in SF. π₯1 is assumed to be superior to π₯2 if the following conditions are satisfied: 1. π₯1 is a feasible solution but π₯2 is an unfeasible solution. 2. Both π₯1 and π₯2 are feasible solutions and the objective function value of π₯1 is smaller than π₯2 . 3. Both π₯1 and π₯2 are infeasible solutions, and the overall constraint violation π£(π₯) of π₯1 is smaller than π₯2 . Ξ΅-Constraint (EC): The method of constraint violation and Ξ΅ level comparisons was proposed in [21]. The essential of Ξ΅-Constraint handling strategy is the tradeoff between the constraint violation and the objective function. The parameter Ξ΅ was employed to adjust the balance,
lP
urn a
26 27 28
and the constraint violation π(π₯) was defined as a form based on the maximum of all constraint violations. π(π₯) = πππ₯{πππ₯{0, π1 (π₯), β¦ , ππ (π₯)}, πππ₯|βπ+1 (π₯), β¦ , βπ (π₯)|} (10) In constrained optimization problems, an individual is assumed as infeasible solution and its precedence is low if the constraint violation is greater than 0, This precedence is controlled by the parameter Ξ΅ . When the target and trail vectors are compared based on Ξ΅ -Constraint
29 30
method, a set of objective functions and constraint violations (π(π₯), π(π₯)) will be evaluated according to the Ξ΅ -level precedence. Similar to previous discussion, π₯π is considered as
31
superior to π₯π when the following criteria are satisfied
Jo
24 25
re-
π£(π₯) =
(π(π₯π ), π(π₯π )) β€π (π(π₯π ), π(π₯π ))
π(π₯π ) β€ π(π₯π ), ππ π(π₯π ), π(π₯π ) β€ π {π(π₯π ) β€ π(π₯π ), ππ π(π₯π ) = π(π₯π ) π(π₯π ) < π(π₯π ), ππ‘βπππ€ππ π 32
(11)
The β€π between the individual π₯π and π₯π means π₯π is superior to π₯π according the Ξ΅6 / 31
Journal Pre-proof level precedence. When Ξ΅ is set as β, the value of objective function is the only criterion for determining superiority of two individuals. The property of the Ξ΅-Constraint method is the flexibility. The parameter Ξ΅ can be modified according to the characteristics of optimization
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
problem in order to obtain the suitable balance between lower values of objective function value and constraint violation. However, the suitable balance point is difficult to be determined for different COPs. Self-Adaptive Penalty (SP): Penalty function method keeps the efficiency even after achieving enough feasible individuals with simple structure [56]. Static penalty function method is classical in COPs due to its simplicity. Many methods based on penalty functions provide competitive results [57]. However, a major drawback of static penalty function method is difficult to determine the suitable penalty coefficients. Generally, the penalty coefficients are commonly case-dependent. Therefore, adaptive penalty functions [58] have been proposed in order to overcome this bottleneck. The penalty coefficients can be determined adaptively according to the previous iterations. In constrained optimizations, it is important to extract useful information from the infeasible individual whose worth is low but can guide the evolutionary path to the feasible region. The main difference is their precedence of distinct types of infeasible individuals [59]. An adaptive penalty function method based on distance value and two penalties was proposed [59]. The amount of added penalties are affected by the feasible ratio of population individuals. A greater penalty will be added to infeasible individuals if the feasible ratio is low. In contrast, if there are enough feasible individuals, then the infeasible individuals with high-fitness values will obtain relatively small penalties. It means that the algorithm can change the status between seeking the optimal solution and finding feasible solutions without presetting parameters. The details of this method are represented as follows πΉ(π₯) = π(π₯) + π(π₯) (12) π£(π₯), ππ πππππ = 0 π(π₯) = { β²β² 2 (13) 2 βπ (π₯) + π£(π₯) , ππ‘βπππ€ππ π
lP
re-
pro of
1 2 3
urn a
π(π₯) = (1 β πππππ )π(π₯) + πππππ π(π₯)
πππππ =
ππ’ππππ ππ ππππ ππππ πππππ£πππ’ππ ππππ’πππ‘πππ π ππ§π
(15)
0, ππ πππππ = 0 π£(π₯), ππ‘βπππ€ππ π
(16)
M(x) = {
ππ π₯ ππ π ππππ ππππ π πππ’π‘πππ 0, π(π₯) = { β²β² (π₯), π ππ π₯ ππ ππ ππππππ ππππ π πππ’π‘πππ π β²β² (π₯) =
Jo
(14)
(π(π₯) β ππππ ) (ππππ₯ β ππππ )
(17) (18)
25 26
where πππππ is the feasible ratio, π£(π₯) is the overall constraint violation, π β²β² (π₯) is the normalized fitness value οΌ ππππ₯ πππ ππππ are the maximum and minimum values of the
27
objective function in the present generation, π(π₯) is the penalty value.
28
3 Surrogate assisted adaptive DE algorithm
29
It is well known that the performance of the DE is a case-dependent with different parameter 7 / 31
Journal Pre-proof ranges when handling with different problems. Except for parameters like Cr, F, Np, mutation strategies employed in evolution are also sensitive to the performance during optimization. Actually, even for a simple problem, which has been determined its characteristics; the suitable ranges of parameters like Cr, F and Np should be updated in different stages of the DE. Therefore, the requirements for mutation strategies are also changed especially in constrained optimizations. To overcome these difficulties, several strategies have been suggested. For example, parameter adaptation method for Cr, F based on success-history has been employed in many algorithms like SaDE [11], JADE [12], SHADE [13] and JADE_sort [15]; population size controlling strategy for Np has been applied to L-SHADE [20], sTDE-dR [67]. Although these adaptive methods have successfully improved the performances, the efficiency of the DE still has room for improvement. Therefore, several adaptive strategies are suggested in this study and the framework of the suggested algorithm is presented in Fig.1. Initialize control parameters
Generate the initial four equal size subpopulations randomly and evaluate the individuals by real functions
re-
Mutation (DE/current-to-best/1) is assigned for each subpopulation
pro of
1 2 3 4 5 6 7 8 9 10 11 12 13
Initialize the subpopulations and assign the number of size of each subpopulation
Local surrogate assisted evaluation strategy
Search the neighbor populations around new one
EFs<=EFsmax
lP
Assign Cr, F to individuals in each subpopulation to create the trail vectors according subpopulation archives
NO
Terminate
Update the size of subpopulation based on pfeas and save failed individuals, update the success-memory of each subpopulation
EGO (1) KG (2) SVR (3) RBF
(1) KG-HDMR (2) RBF-HDMR (3) SVR-HDMR
Projection sampling method
Generate and update surrogate by different techniques
No
Select the best surrogate model
urn a
Parameter adaptation strategy based on success-history
Calculate size of the next generation
14 15
0.5<=R2<=1
Yes
Fig.1 Framework of the ASA-DE
3.1 Notes of Adaptive population clustering strategy
17 18 19 20 21 22
In order to control the state switch efficiently, adaptive population clustering strategy is employed for the suggested algorithm. This strategy has been widely used by several scholars. In Zhouβs work [68], the population should be classed into better and worse groups according to fitness values. In the L-SHADE44 [66], population can be divided into four subpopulations, each subpopulation employs an independent mutation strategy. πΊ In this study, the population π(πΊ) = [π₯1πΊ , π₯2πΊ , β¦ , π₯ππ ] should be clustered into k
23 24 25
subpopulations [π1πΊ , π2πΊ , β¦ , πππΊ ] , πππΊ is the kth subpopulation of the Gth generation. Each subpopulation should be assigned by DE/current-to-best/1 strategy and has independent adaptive parameter archive. The size of each subpopulation should be the same in the first
Jo
16
8 / 31
Journal Pre-proof generation and the fitness value of each population should be evaluated by the real function or simulation. Commonly, the size of each subpopulation will be updated according to the π success rate of evolution. If the trail vector generated from individual π₯π,πΊ is better than its
4 5
parents in term of the fitness, the ith individual in sth subpopulation should be considered as a successful one. The size of each subpopulation is determined as ππ’π + ππ’0 ππ π = π ππ (19) βπ=1(ππ’π + ππ’0 ) πΊ
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
where ππ’π is the present number of the successful evolutions in sth subpopulation. ππ’0 is a small positive constant in order to avoid zero denominator when the algorithm is close to convergence. πππΊ is the entire population size in the Gth generation. However, according to the test, this size control strategy still has room for modification, a more suitable strategy is suggested according to exploitation and exploration and should be discussed in Section 3.2. Moreover, it should be noted, the new population except for the first iteration should be generated and evaluated by the surrogate model according to the assigned criterion. Some of new populations cannot be directly generated by the HDMR-based surrogate techniques directly due to the characteristic of the HDMR. For the HDMR-based modeling technique, the new population should be generated on the axil of hypercube. Therefore, when a new population is generated inside of the hypercube, the project sampling strategy [65] should be implemented to generate the new population on the each axil. Thus, the number of populations should be significantly improved. However, these new populations generated by the project strategy should be evaluated by the surrogate model, the computational cost should be almost same as the others. Another important issue should be considered is the number of clusters. Theoretically, the number of clusters should be determined by the complexity of each problem. However, consider the diversity of problems, the number of clusters is difficult to be obtained. Therefore, several tests have been carried out with 4~10 clusters. In this study, the test results suggested that 4-cluster is the best choices.
26
3.2 Adaptive population size based on pfeas
27 28 29 30 31 32 33 34
As mentioned previous section, the population size is an important parameter to improve the performance of algorithm. In this study, after cluttering, the size of each subpopulation should updated according to the suggested adaptive population size strategy based on pfeas. In this strategy, the evolutionary procedure can be classified into three states, exploring biased, balance search and exploiting biased states. Exploring biased state: The optimization procedure enters the exploring state before the FEs exceeds the 0.5 πΉπΈπ πππ₯ . The control strategy of population size is linear reduction. The population size in this stage is calculated according to the formula as follows: πππππ₯ β πππππ (20) Np(G + 1) = Round [πππππ₯ β ( ) πΉπΈπ ] πΉπΈπ πππ₯
35 36 37 38
where πππππ is the preset minimum population size, πππππ₯ is the initial population size. Before the FEs exceeds the πΉπΈπ π π€ππ‘πβ1 (set as 0.5πΉπΈπ πππ₯ , the choice of values should be discussed in numerical test section), the population is large enough to keep exploration and population diversity. The minimum of the population size is 0.5πππππ₯ . Experiments have
Jo
urn a
lP
re-
pro of
1 2 3
9 / 31
Journal Pre-proof proved that it is a suitable value choice for the algorithm when it is difficult to determine whether the constraints are loose. The balance search state: The optimization procedure will enter the balance search state if no feasible solution is found when FEs reaches πΉπΈπ π π€ππ‘πβ2 . In this state, the population size will keep the same size (0.5πππππ₯ ) at the end of the previous state to ensure a high-level diversity. Meanwhile, the mutation strategies will switch to highly exploiting biased state to enhance the efficiency. Once the first feasible solution is successfully found, the population size will be adaptively reduced based on the feasible rate (πππππ ) which can be formulated as (πππππ₯ β πππππ ) NP(G + 1) = Round [πππππ₯ β (πΉπΈπ β πΉπΈπ πππ₯)πππππ ] (21) (πΉπΈπ πππ₯ β πΉπΈπ πππ₯)
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
where πππππ₯ is the population size in the generation when the algorithm enter to the balance search state, πΉπΈπ πππ₯ is the number of function evaluations when the algorithm switches to the balance search state. If pfeas is 0, the population size will be stopped to be reduced to keep the diversity of populations to keep enough diversity. When pfeas exceeds pfeasNp (set as 0.5, the choice of value should be discussed in section 4), the population size is calculated as Eq.(21). If pfeas has reached pfeasNp before the balance search state begins, the method based on the linear reduction will be employed continuously and the balance search state is not employed during this evolutionary process. Exploiting biased state: The algorithm will switch to the exploiting biased state when FEs reaches πΉπΈπ π π€ππ‘πβ3 . The population linear reduction method and an exploiting protection mechanism will be employed at this stage. The population will initialize as πππππ₯ /6 if the population size is larger than πππππ₯ /6 (pfeas remains low or no feasible solution can be found in the first two stages). The mutation strategies will switch to balance mode to enhance the effect. For clarity, the details of this strategy is presented as follows.
lP
re-
pro of
1 2 3 4 5 6 7 8
urn a
Adaptive population size based on pfeas Start 1. If πΈπΉπ < 0.5 Γ πΉπΈπ πππ₯ 2. The strategy chooses the exploring state and the population size is calculated as Eq. 20. 3. Else 4. ..If πππππ > 0.5 && πΈπΉπ < 0.9 Γ πΉπΈπ πππ₯ 5. Keep the exploring state and the population size is calculated by Eq. 20. 6. If πππππ = 0 && πΈπΉπ > 0.9 Γ πΉπΈπ πππ₯ 7. Switch to the exploiting biased state and the population size is initialized to πππππ₯ /6 and is calculated as Eq. 20.
Jo
8. Else 9. Switch to the balance search state and the population size is calculated by Eq. 21. 10. End if 11.End if
24
3.3 Local surrogate assisted evaluation strategy
25 26
The surrogate model is another important issue in this framework. In this study, surrogate models should be constructed by several kinds of modeling techniques, KG [60], SVR [61], RBF 10 / 31
Journal Pre-proof [62], high dimensional modal representation (HDMR) [63] and other surrogate assisted HDMR methods, such as KG-HDMR, RBF-HDMR and SVR-HDMR in sequential optimization as shown in Fig.1. The details of these surrogate modeling techniques can be found in [64]. In this procedure, these surrogate modeling techniques are implemented to improve the efficiency of suggested strategy instead of real function evaluations. In this procedure, R 2 should be the criterion to verify the accuracy of the constructed surrogate models based on leave-one-out cross validation (LOOCV). To save the computational cost, the samples from other subpopulation should be the test population for validation. Compared with the LOOCV based on the training samples, the result of test obviously more objective. Moreover, it also means that the additional test samples for the subpopulation can be avoided. After test, the best surrogate model should be used to generate new populations for the next generation. Compared with direct evaluation strategy, the accuracy of a new population derived from the surrogate model should be decreased. Actually, it is difficult for the popular surrogate modeling techniques to construct highly accurate model for high dimensional problems (over 50D). Therefore, in the suggested algorithm, the populations used for the following strategies should be used to generate the local surrogate model instead of global one. In the suggested
17
π·ππ πππ π ππππβ strategy, the range for each subdomain should be limited for π. For example, if
18
the dimension of assigned case is 30, the surrogate model should be constructed based on the
19
neighbor samples limited in
20
local surrogate model is commonly more accurate than the global one.
21
3.4 Parameter adaptation strategy based on success-history
22 23 24
The choice of parameter F and Cr is also important to the performance of suggested algorithm. In the ASA-DE, a parameter adaptation strategy based on success-history is also employed to control F and Cr [12-13, 74]. Each subpopulation has an independent archive ππ (π =
25 26 27 28 29
1,2, β¦ , π) to store suitable mean values of Cr and F (ππΆπ πππ ππΉ ). The size of archive ππ ππ§π is a preset constant. The values of F and Cr used by successful individuals in the current generation should be stored in temporary archives πππΆπ and πππΉ to calculate the ππΆπ πππ ππΉ . The values of ππΆπ,π and ππΉ,π (π = 1,2, β¦ , ππ ππ§π ) are set to 0.5 at the beginning. In each generation, once archive ππ is full, a random element in the archive should be replaced. The
30 31
details of mechanism are shown as follows ππΆπ ,πΉ = ππππππΏ (πππΆπ ,πΉ ) ππ πππΆπ,πΉ β β (22) where π(π₯π ) is the objective function of the individual which employs the mth F stored in πππΉ . π(π’π ) is the objective of the trail vector of the individual. When the population is completely partitioned, each individual in subpopulations selects F and Cr randomly from the corresponding archive ππ . The final values of Cr and F are calculated
36 37 38 39 40
re-
urn a
lP
(π΅ππ’πππππππ β π΅ππ’πππΏππ€ππ )β 30 . Theoretically, the accuracy of
Jo
32 33 34 35
pro of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
as follows
πΆπ = πππππ(ππΆπ , 0.1) (23) πΉ = πππππ(ππΉ , 0.1) (24) The final values of Cr and F are generated based on normal distributions. The mechanism is employed in our algorithm directly, the parameter settings can be referred to [13], [74]. 11 / 31
Journal Pre-proof 1
4 Numerical test The suggested ASA-DE is tested on 28 widely used benchmark functions from the IEEE CEC2017 benchmarks on constrained real-parameter optimization [69] on 10D, 30D, 50D and 100D. The algorithm runs 50 times for each test function with a number of function evaluations equals to 20,000π· according to the guidelines of CEC2017 [69]. Because the surrogate modeling techniques are used to evaluate the fitness in the ASA-DE, the number of populations evaluated by the surrogate should be recorded. In this test, the percentage of number of surrogate evaluated populations should also be listed. The criteria employed to evaluate the performance of the ASA-DE can be found in [69] and includes feasibility rate of all runs (FR), mean violation amounts (π£ππ Μ Μ Μ Μ ) and mean of objective function value (mean). Moreover, the priority level of the evaluation criteria can be according to following order [71], FR. Μ Μ Μ Μ π£ππ and mean. In this study, three evaluation criteria are used for performance comparison. The first criterion is referenced in the guidelines of CEC2017 [69] and employed in results of all comparison algorithms. Pairwise comparisons method the sign test [70] as the second evaluation criterion is also employed to compare the ASA-DE with other state-of-the-art algorithms including the top four algorithms for constrained optimization in CEC2017 are compared with the ASA-DE. The detailed description of the evaluation criterion is showed above. The sign test [71] is used to determine the significance of the results (CAL-SHADE, L-SHADE44, SaDE, SajDE and DEbin versus ASA-DE) at the 0.05 significance. The symbols "+", " β ", "=" are employed to indicate the ASA-DE performs significantly better (+ ), no significantly difference (= ), significantly worse (β) than the algorithms in comparison. Considering the surrogate modeling is imported for evaluating the real response of each test function, the percentage of number of samples evaluated by the surrogate model is also presented. More cheap samples are used, more efficiency ASA-DE is.
26
4.1 Parameters setting
27 28 29 30 31 32
The parameter settings are shown as follows 1. The initial population size πππππ₯ is set to 24D in 10-D problems, 19D in 30-D, 14D in 50-D and 9D in 100D problems. 2. The minimum population size πππππ is set to 6. 3. The number of subpopulation is set to 4. 4. The size of archive ππ (π = 1,2, β¦ , π) is set to 6.
Jo
urn a
lP
re-
pro of
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
33 34 35 36 37
5.
πΉπΈπ π π€ππ‘πβ1 , πΉπΈπ π π€ππ‘πβ2 and πΉπΈπ π π€ππ‘πβ3 are respectively set 0. 08πΉπΈπ πππ₯ , 0.5πΉπΈπ πππ₯ , 0.9πΉπΈπ πππ₯ .
The selection of these parameters is discussed below: The number of subpopulations, the initial population size and ππ are set with reference to the
38 39 40
L-SHADE44. πππππ is slightly lager than the number of subpopulation. Experiments show that πππππ has little effect on the performance of the algorithm and is set as 6 finally. πΉπΈπ π π€ππ‘πβ1 and πΉπΈπ π π€ππ‘πβ3 are more like a protective threshold. πΉπΈπ π π€ππ‘πβ1 should be 12 / 31
Journal Pre-proof 1 2 3 4
enough close to the beginning of the evolutionary process. In contrast, πΉπΈπ π π€ππ‘πβ3 should close to the end of the evolutionary process. Therefore, πΉπΈπ π π€ππ‘πβ1 and πΉπΈπ π π€ππ‘πβ3 are set as 0.08πΉπΈπ πππ₯ and 0.9πΉπΈπ πππ₯ , respectively. Table 1 Configuration of parameters and the initial values of the surrogate modeling Basis functions
Expression
Values
Initial correlation coefficient π
π(β1/m)
Turning parameter ππ
3
RBF
Polyharmonic function order
3
LSSVR
Turning parameter ππΏ
2
Initial correlation coefficient ππΎπ»
0.1
Lower boundary on the π
0.1
Upper boundary on the π
10
Turning parameter ππ π» for thin plate spline function
1
pro of
Kriging
Kriging-HDMR
SVR-HDMR
Others for HDMR
Tolerance to judge the convergence of ππ (π₯π )
10e-4
Tolerance to judge the convergence of πππ (π₯π , π₯π )
10e-2
Tolerance to judge the interaction of π₯π and π₯π
10e-6
Table 2 Comparison results of variants with different πΉπΈπ π π€ππ‘πβ2 with selected functions πΉπΈπ π π€ππ‘πβ2 F3
0.1 β πΉπΈπ πππ₯
0.2 β πΉπΈπ πππ₯
0.5 β πΉπΈπ πππ₯
0.7 β πΉπΈπ πππ₯
0.9 β πΉπΈπ πππ₯
376300
476300
319459
561234
656123
1
1
0.85
0.87
0
0
9.7363e-06
8.1236e-05
4681.12
4453.81
4262.0712
4132.0782
0.26
0.32
0.3023
0.2931
0.01121
0.01306
0.01278
0.01278
-0.5622
-0.5622
-0.8736
-0.8651
-0.8551
0.6210
0.6210
1
1
1
8.2621e-10
9.4621e-11
0
0
0
478.7812
401.3732
212.5621
388.2108
461.9812
1
1
1
0.6800
0.6980
0
0
0
0.2837
0.3865
1 8.7876e-06 4781.12
F6
0.26
F11
F22
urn a
0.01121
For more clarity, the suggested algorithm with 5 different values of πΉπΈπ π π€ππ‘πβ2 (0.1 β πΉπΈπ πππ₯ , 0.3 β πΉπΈπ πππ₯ , 0.5 β πΉπΈπ πππ₯ 0.7 β πΉπΈπ πππ₯ and 0.9 β πΉπΈπ πππ₯ ) on 30D problems according to the guidelines of CEC2017. The parameter πΉπΈπ π π€ππ‘πβ2 is used in adaptively population size mechanism which is designed for solving medium strict level problems. Therefore, f3, f6, f11, f22 are employed to analyze πΉπΈπ π π€ππ‘πβ2 . Three variant algorithms are as follows. ASA-DE-FEs1: πΉπΈπ π π€ππ‘πβ2 is set as 0.1 β πΉπΈπ πππ₯ ASA-DE-FEs2: πΉπΈπ π π€ππ‘πβ2 is set as 0.3 β πΉπΈπ πππ₯ ASA-DE -FEs3: πΉπΈπ π π€ππ‘πβ2 is set as 0.5 β πΉπΈπ πππ₯ ASA-DE -FEs4: πΉπΈπ π π€ππ‘πβ2 is set as 0.7 β πΉπΈπ πππ₯
Jo
7 8 9 10 11 12 13 14 15 16
2 10e-6
lP
5 6
Turning parameter ππΏπ»
Tolerance to judge the nonlinearity of ππ (π₯π )
re-
RBF-HDMR
13 / 31
Journal Pre-proof ASA-DE -FEs5: πΉπΈπ π π€ππ‘πβ2 is set as 0.9 β πΉπΈπ πππ₯ According to Table 2, it can be found that when πΉπΈπ π π€ππ‘πβ2 is set as 0.5 β πΉπΈπ πππ₯ , the performance of the algorithm is the best one. Therefore, we set πΉπΈπ π π€ππ‘πβ2 as 0.5 β πΉπΈπ πππ₯ . Moreover, because several surrogate models are integrated in the suggested method, the initial values of parameters used in these methods are listed in Table.1. It means that Kriging, RBF, LSSVR, Kriging-HDMR, RBF-HDMR and SVR-HDMR should be employed to construct the surrogate model for each subpopulation and the best one of them should be selected for the next iteration.
9
4.2 Comparisons with other popular algorithms
10 11 12 13
pro of
1 2 3 4 5 6 7 8
In order to evaluate the performance of the suggested ASA-DE algorithm, the following popular algorithms including the top 4 algorithms for the constrained optimization in CEC2017 are compared with the ASA-DE.
Name
Full Name
Year
CAL-SHADE
re-
Table 3 Other DE algorithms for comparison
Adaptive constraint handling and success history differential evolution[72]
2017
L-SHADE44+IDE
SajDE SaDE DEbin
[74].
A unified differential evolution algorithm for constrained optimization problems (UDE) [18].
An Improved Self-adaptive Differential Evolution Algorithm in Single Objective Constrained Real-Parameter Optimization [55]. Self-adaptive differential evolution algorithm for constrained realparameter optimization [75]. The standard differential evolution algorithm (DEbin) [3].
2017 2017 2017 2010 2006 1997
For the comparison result with 10D, as listed in Tables 4 and 5, the ASA-DE is better than or at least as good as the other algorithms. The ASA-DE also performs better than the ranked first algorithm of the CEC2017 constrained optimization (L-SHADE-44) and performs better than the L-SHADE-44 on 15 functions, while is worse by the L-SHADE-44 on 7 functions. The ASADE is also efficient at solving f6, while the L-SHADE-44 performs worse on this one. It should be noted that most of the mean percentages of the number of surrogate evaluated populations are closed to 35% and corresponding variances are near 5%. It suggests the ASA-DE might save nearly 30% computational cost compared with the directly evaluated algorithm. For f17, f18, f19, f26, f27, f28, it is difficult for the ASA-DE to construct high accurate surrogate modeling due to its complexity even the original function has been clustered. For these problems, the variances are also smaller. Therefore, for 10D problems, the ASA-DE generally obtains the high accurate result efficiently. Moreover, because the adaptive parameter strategy is used, Μ Μ Μ Μ π£ππ also outperforms than most of other algorithms.
Jo
14 15 16 17 18 19 20 21 22 23 24 25 26 27
L-SHADE with competing strategies applied to constrained optimization
lP
UDE
SHADE44 and IDE (L) [73].
urn a
L-SHADE44
A simple framework for constrained problems with application of L-
14 / 31
Journal Pre-proof For 30D problems, as shown in Tables 6 and 7, the ASA-DE also performs better than or at least equal to the other algorithms and keep the ability to solve f6. The ASA-DE achieves competitive results that are equal or better than the L-SHADE-44 in most benchmark functions. Except f17, f18, f19, f26, f27, f28, the ASA-DE optimizes all other functions successfully. The other algorithms are also inefficient for it. Although the UDE is also effective in optimizing f6, it fails to optimize f11. As a result, the ASA-DE has an excellent constraint control capability and can follow the suitable balance point between exploration and exploitation during evolutionary process through state switching and population size adaptive mechanisms. Compared with the test result for 10D problems, the percentages of surrogate evaluated population are decreased, the mean values are over 30%. With the increasing of the dimensionality, the performance of the ASA-DE is decreased due to the accuracy of surrogate modeling techniques. However, the computational cost for each problem can be saved over 20%-30%, it demonstrates that the ASA-DE is still suitable for 30D problems. It demonstrates that the local modeling strategy becomes more important for higher dimensional problems. For 50D and 100D problems, as presented in Tables 7-11, the ASA-DE outperforms the other algorithms in most benchmark functions. The ASA-DE obtains competitive results that are equal or better than L-SHADE-44 in 28 benchmark functions. Compared with 10D and 30D problems, although the performance of the ASA-DE still prevails, the influence from the surrogate model seems to be weak. According to Tables 8 and 9, the number of samples evaluated the surrogate models decreases. For 100D problems, as presented in Tables 10 and 11, due to the limitation of existed test data, four novel algorithms are compared with the ASADE. It can be found that the result is similar to 50D, even better. Successful optimization rates for benchmark functions are presented in Table.12, it demonstrates that the suggested algorithm is better than other algorithms in this test. Moreover, it should be noted that the number of samples evaluated by surrogates should be decreased in 50D and 100D. However, compared with 50D problems, the number of samples evaluated by surrogates in 100D is not decreased significantly. It suggests that the number of clusters for different dimensions is reasonable. In our opinion, the local modeling technique plays a more important role for highly dimensional problems.
31
Table 4. Experimental results in mean, FR and Μ Μ Μ Μ Μ ππΌπ on functions 1-28 for 10D problems
F1
F2
F3
F4
ASA-DE
CALSHADE
LSHADE44
SaDE
SajDE
DEbin
LS44+IDE
UDE
0.0000E+00
6.4300E-30
0.0000E+00
0.0000E+00
6.1800E-30
2.7200E-29
0.0000E+00
5.0300E-15
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
9.1000E-17
0.0000E+00
0.0000E+00
2.6900E-29
1.2100E-28
0.0000E+00
6.4212E-15
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1233E+03
1.1892E+05
8.2093E+04
1.0499E+05
1.2000E-29
1.0892E+05
3.2601E+05
7.7386E+01
1.0000E+00
4.8000E-01
9.2000E-01
8.0000E-02
1.0000E+00
2.8000E-01
1.0000E+00
9.6000E-01
0.0000E+00
6.2600E-05
9.2600E-06
1.9000E-04
0.0000E+00
1.2600E-04
0.0000E+00
4.3000E-06
1.3291E+01
1.3921E+01
1.3629E+01
1.3273E+01
1.5120E+01
1.4793E+01
1.4420E+01
2.5117E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
Jo
Func
urn a
lP
re-
pro of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
15 / 31
Journal Pre-proof
F5
F6
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
5.0100E-02
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.1890E-01
0.0000E+00
0.0000E+00
3.1921E-01
1.2341E+00
0.0000E+00
1.6542E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.2741E+01
8.28299E+02
7.2049E+02
7.9512E+02
0.0000E+00
4.2825E+02
8.0836E+02
8.7100E+01
1.0000E+00
0.0000E+00
4.0000E-02
0.0000E+00
1.0000E+00
1.6000E-01
0.0000E+00
4.4000E-01
0.0000E+00
1.4620E-01
3.1950E-02
1.8270E-01
0.0000E+00
1.4180E-01
3.7660E-02
2.0400E-02
1.7993E+01
-6.5994E+00
2.1918E+01
-5.2247E+02
3.0619E+01
-3.4003E+01
-6.4624E+00
1.0000E+00
6.8000E-01
7.6000E-01
0.0000E+00
0.0000E+00
8.0000E-02
8.0000E-01
7.2000E-01
0.0000E+00
1.0000E-04
3.2500E-05
1.1440E-01
1.0216E+03
1.2150E-03
3.1900E-05
7.0610E-02
-1.350E-03
-1.350E-03
-1.3500E-03
-1.350E-03
-9.0457E+01
-2.9000E-04
-1.3500E-03
-1.3400E-03
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.7650E+05
0.0000E+00
0.0000E+00
0.0000E+00
-4.980E-03
1.0032E+00
-4.9800E-03
-4.980E-03
-6.0920E-01
9.9100E-02
-4.9800E-03
-4.9800E-03
1.0000E+00
5.2000E-01
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
4.0900E-02
0.0000E+00
0.0000E+00
2.5330E+02
0.0000E+00
0.0000E+00
0.0000E+00
-5.100E-04
-5.100E-04
-5.1000E-04
-5.100E-04
-5.9754E+01
-4.5000E-04
-5.1000E-04
-5.1000E-04
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1100E+06
0.0000E+00
0.0000E+00
0.0000E+00
-1.5250E-01
-1.5950E-01
-1.6340E-01
8.5700E-02
-9.8610E+02
-6.4520E-02
-1.6880E-01
-5.9955E+00
1.0000E+00
8.8000E-01
1.0000E+00
6.8000E-01
0.0000E+00
8.8000E-01
1.0000E+00
0.0000E+00
0.0000E+00
1.8000E-04
0.0000E+00
1.8500E-07
4.2500E+19
7.7200E-15
0.0000E+00
3.0000E-03
3.9880E+00
4.0037E+00
3.9932E+00
3.9934E+00
1.9900E-01
3.9879E+00
3.9879E+00
3.9879E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.9005E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.8670E-01
1.9700E-31
0.0000E+00
4.7840E-01
2.5780E-02
0.0000E+00
1.1124E+01
F11
F12
F13
F14
F15
F16
F17
F18
re-
F10
lP
F9
urn a
F8
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.8570E+00
3.2745E+00
2.8391E+00
2.8478E+00
1.4200E-16
3.1894E+00
3.0000E+00
2.7362E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.5000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.2666E+01
1.4925E+01
1.4294E+01
1.2535E+01
2.2000E-15
1.4797E+01
1.1278E+01
6.7549E+00
1.0000E+00
0.0000E+00
2.8000E-01
9.2000E-01
0.0000E+00
1.0000E+00
1.0000E+00
9.2000E-01
0.0000E+00
4.7000E-03
1.8900E-04
4.3000E-06
5.0000E-01
0.0000E+00
0.0000E+00
3.7000E-04
Jo
F7
2.2418E+01
pro of
-
3.7448E+01
4.9009E+01
4.0904E+01
5.1271E+01
0.0000E+00
4.7752E+01
4.0401E+01
6.2830E+00
1.0000E+00
9.6000E-01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
9.6000E-01
0.0000E+00
2.0900E-06
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1100E-05
1.0521E+00
9.2130E-01
9.1933E-01
9.1520E-01
1.2800E-03
1.0273E+00
8.8240E-01
1.0460E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.6600E+00
5.5000E+00
5.2600E+00
4.5000E+00
2.2939E+01
4.9400E+00
5.2200E+00
5.4200E+00
3.8204E+02
7.9760E+02
3.0489E+03
7.7489E+02
8.0000E-02
6.2531E+02
3.1663E+03
2.3591E+03
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
16 / 31
Journal Pre-proof
F22
F23
F24
F25
F26
F28 +ο―=ο―β +ο―=ο―β
1.1437E+02
8.2300E+06
1.5100E+07
0.0000E+00
2.3900E-06
1.5800E-06
0.0000E+00
-9.2521E+00
1.1200E-06
0.0000E+00
2.7200E-03
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
6.6336E+03
6.6336E+03
6.6336E+03
6.6336E+03
6.6464E+03
6.6336E+03
6.6336E+03
6.6336E+03
4.5110E-01
2.4900E-01
1.8010E-01
4.5800E-01
6.0890E-01
4.2790E-01
4.1560E-01
1.6464E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.9879E+00
3.9901E+00
3.9884E+00
3.9880E+00
1.0777E+01
3.9879E+00
3.9879E+00
6.2436E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.1860E-01
1.1011E+02
3.1890E-01
4.7840E-01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.7089E+00
3.3359E+00
2.5320E+00
3.3042E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.3289E+01
1.0022E+01
8.1367E+00
2.0452E+01
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1196E+02
4.6244E+01
4.6307E+01
3.7008E+01
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.2066E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.9313E+01
8.3719E+00
1.5950E-01
1.2504E+01
8.4000E-01
1.0000E+00
1.0000E+00
1.0000E+00
2.3755E+02
0.0000E+00
0.0000E+00
0.0000E+00
2.0496E+01
3.0997E+00
3.0212E+00
2.7456E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
4.0874E+04
0.0000E+00
0.0000E+00
0.0000E+00
1.4100E-16
1.1781E+01
8.7650E+00
6.0006E+00
1.0000E+00
1.0000E+00
9.6000E-01
5.0000E-01
0.0000E+00
0.0000E+00
1.5200E-04
6.8361E+01
7.1500E-15
4.2327E+01
3.7699E+01
6.3459E+00
1.0000E+00
9.2000E-01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
8.1100E-06
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.0547E+00
9.5090E-01
1.0059E+00
1.1173E+00
2.0580E-02
1.0636E+00
9.3740E-01
1.0217E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.7000E+00
5.4600E+00
5.1000E+00
4.5000E+00
2.9816E+01
4.9000E+00
5.0526E+00
5.4611E+00
4.3869E+02
3.1764E+03
2.6079E+03
7.3171E+02
3.1806E+01
5.8606E+03
7.9442E+03
6.7179E+03
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.1842E+04
8.6413E+04
3.5710E+05
1.7742E+02
1.8585E+03
2.6471E+05
4.0500E+07
2.2300E+08
2.1103E+01
3.5925E+01
2.5750E+01
1.0419E+01
-8.0290E-02
2.3984E+01
1.0763E+01
9.7567E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
6.6480E+03
6.6604E+03
6.6522E+03
6.6396E+03
6.6475E+03
6.6496E+03
6.6409E+03
6.6410E+03
Criterion I Criterion II
23/2/3 18/8/2
15/6/7 7/17/4
13/8/7 11/14/3
21/0/7 19/3/6
23/1/4 15/12/1
11/9/8 NA
22/2/4 NA
Table 5. The mean percentage of number of surrogate evaluated populations for each 10D problems
Jo
1 2 3
7.4797E+00
urn a
F27
5.7405E+01
pro of
F21
1.7100E+06
re-
F20
1.1585E+02
lP
F19
1.3543E+04
Function
F1
F2
F3
F4
F5
F6
F7
Mean
38.51%
39.22%
28.21%
18.91%
35.12%
38.72%
37.32%
Variance
5.29%
4.39%
6.21%
7.53%
4.56%
5.12%
4.34%
Function
F8
F9
F10
F11
F12
F13
F14
Mean
34.51%
40.21%
32.21%
34.78%
36.82%
39.12%
33.39%
Variance
5.12%
8.39%
3.76%
4.78%
5.11%
6.18%
5.21%
Function
F15
F16
F17
F18
F19
F20
F21
Mean
33.51%
34.21%
21.2%
18.67%
15.89%
32.121%
37.21% 17 / 31
Journal Pre-proof Variance
4.62%
3.89%
2.34%
3.89%
2.76%
5.12%
6.12%
Function
F22
F23
F24
F25
F26
F27
F28
Mean
33.31%
41.22%
37.02%
37.91%
25.2%
20.12%
21.32%
Variance
6.12%
5.81%
5.78%
4.23%
1.89%
2.12%
3.23%
1 Table 6. Experimental results in mean, FR and Μ Μ Μ Μ Μ ππΌπ on functions 1-28 for 30D problems
F3
F4
F5
F6
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
9.6300E-29
7.2300E-30
1.1600E-07
4.6900E-27
1.0100E-11
0.0000E+00
7.3400E-29
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.6700E-30
8.6600E-29
4.7400E-30
6.3100E-08
1.1700E-26
7.1700E-12
0.0000E+00
7.3900E-29
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.4280E+05
3.6912E+05
3.1728E+05
9.6069E+05
7.4200E-27
1.1600E+06
6.7000E+06
7.3254E+01
1.0000E+00
1.0000E+00
1.0000E+00
4.4000E-01
1.0000E+00
4.4000E-01
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.0000E-04
0.0000E+00
1.2000E-04
0.0000E+00
0.0000E+00
1.3573E+01
1.3833E+01
1.3573E+01
1.3573E+01
3.5820E-01
1.3667E+01
1.3854E+01
8.2422E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.5020E-01
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.5950E-01
3.6500E-31
1.4700E-07
5.4837E+00
1.0157E+00
0.0000E+00
2.3200E-17
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.4538E+03
3.9102E+03
3.7330E+03
4.2628E+03
4.7760E-01
3.7296E+03
5.5264E+03
3.0362E+02
3.2000E-01
0.0000E+00
0.0000E+00
0.0000E+00
5.6000E-01
0.0000E+00
0.0000E+00
1.0000E+00
1.3060E-02
2.0800E-02
1.7800E-02
2.9250E-01
2.9140E-01
3.1080E-01
2.5670E-02
0.0000E+00
-9.3038E+01
-1.5046E+02
-1.0888E+02
-3.5931E+01
-1.5460E+03
2.5535E+01
-8.1088E+01
F7
F9
F10
F11
5.9813E+02
1.0000E+00
1.0000E+00
9.6000E-01
1.2000E-01
0.0000E+00
1.6000E-01
9.6000E-01
1.0000E+00
0.0000E+00
0.0000E+00
5.5100E-06
1.1000E-03
2.9055E+03
1.6800E-03
4.0600E-06
0.0000E+00
-2.800E-04
-2.8000E-04
-2.8000E-04
-1.8000E-04
-6.3808E+01
-2.1000E-04
-2.6000E-04
-2.8000E-04
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
7.8000E+06
0.0000E+00
0.0000E+00
0.0000E+00
-2.670E-03
-2.6700E-03
-2.6700E-03
-2.670E-03
-5.8710E-01
3.4250E-02
-2.6700E-03
-2.6700E-03
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.2749E+04
0.0000E+00
0.0000E+00
0.0000E+00
-1.000E-04
-1.0000E-04
-1.0000E-04
-9.9800E-05
-5.0439E+01
-1.000E-04
-9.8000E-05
-1.0000E-04
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.9500E+07
0.0000E+00
0.0000E+00
0.0000E+00
-8.5360E-01
-4.5390E-01
-8.6230E-01
1.9375E+00
-2.9687E+03
-2.0513E+01
-8.6510E-01
-2.8348E+01
1.0000E+00
8.8000E-01
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
1.0000E-19
0.0000E+00
1.1600E+22
3.4700E+59
1.5547E+00
0.0000E+00
2.0700E-02
3.9856E+00
2.1186E+01
3.9856E+00
3.9963E+00
9.9500E-01
3.9826E+00
6.0679E+00
1.8679E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
urn a
F8
pro of
F2
CAL-SHADE
re-
F1
ASA-DE 5.7500E-30
lP
Func
Jo
2
F12
18 / 31
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
1.6219E+00
0.0000E+00
0.0000E+00
0.0000E+00
6.9248E+00
1.5040E+01
4.8409E+00
1.3427E+01
5.9520E+00
2.3195E+01
2.6011E+01
8.1490E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.8808E+00
2.0530E+00
1.8330E+00
1.8373E+00
3.7250E-02
2.1033E+00
1.9086E+00
1.5258E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.4701E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.8693E+01
1.9447E+01
1.8818E+01
1.6305E+01
1.5321E+01
1.8567E+01
1.2912E+01
9.1420E+00
9.6000E-01
1.6000E-01
8.4000E-01
1.0000E+00
3.2300E-06
4.3000E-04
1.8600E-05
0.0000E+00
1.4652E+02
1.6072E+02
1.4853E+02
1.9434E+02
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.0019E+00
1.0141E+00
9.9840E-01
1.0221E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.5500E+01
1.5500E+01
1.5500E+01
1.5500E+01
8.2729E+02
4.8317E+03
1.5830E+03
5.8927E+02
pro of
F16
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.5366E+05
1.1100E+06
1.5794E+04
5.3500E-06
7.3300E-06
0.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
8.8000E-01
3.5785E+02
0.0000E+00
0.0000E+00
2.6200E-05
4.9700E-16
1.7851E+02
1.4301E+02
8.4193E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.5600E-03
1.0263E+00
1.0128E+00
1.0214E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
7.0390E+01
1.5500E+01
1.5500E+01
1.5500E+01
6.8000E-01
6.1701E+03
4.0096E+03
3.5682E+03
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.0528E+03
4.7288E+01
2.0400E+06
1.3300E+06
3.3600E+07
6.2900E-06
0.0000E+00
-2.7748E+01
7.8100E-06
0.0000E+00
3.6576E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.1375E+04
2.1375E+04
2.1375E+04
2.1375E+04
2.1416E+04
2.1375E+04
2.1375E+04
2.1379E+04
1.4602E+00
1.1764E+00
1.4034E+00
3.0345E+00
1.8854E+00
3.1429E+00
2.2476E+00
4.5816E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
9.6000E-01
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.3100E-03
0.0000E+00
0.0000E+00
0.0000E+00
2.2932E+01
1.9466E+01
1.8695E+01
9.5185E+00
8.5765E+01
1.8217E+01
2.7467E+01
1.1910E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
9.2000E-01
lP
F15
0.0000E+00
urn a
F14
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.0666E+01
0.0000E+00
0.0000E+00
1.4800E-12
2.1354E+02
3.3571E+04
3.0641E+03
2.6707E+04
1.2136E+02
3.6693E+03
8.4401E+02
9.4147E+01
1.0000E+00
8.8000E-01
1.0000E+00
3.6000E-01
7.2000E-01
9.2000E-01
1.0000E+00
1.0000E+00
0.0000E+00
4.0420E-01
0.0000E+00
3.2106E+00
2.9194E+02
5.1750E-02
0.0000E+00
0.0000E+00
1.7837E+00
2.0591E+00
1.6560E+00
1.9966E+00
2.1052E+01
2.0072E+00
1.8564E+00
1.4349E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.5224E+05
0.0000E+00
0.0000E+00
0.0000E+00
1.2032E+01
1.5048E+01
1.2158E+01
5.9028E+01
1.4898E+00
1.7059E+01
1.3917E+01
8.3879E+00
Jo
F13
0.0000E+00
re-
Journal Pre-proof
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
8.6524E+03
5.3150E-01
0.0000E+00
0.0000E+00
0.0000E+00
1.3936E+02
1.6644E+02
1.4143E+02
3.1894E+02
1.4772E+01
1.7712E+02
1.4062E+02
1.5959E+01
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1520E+03
6.7950E-01
0.0000E+00
0.0000E+00
0.0000E+00
1.0062E+00
1.0176E+00
1.0056E+00
1.0264E+00
8.5720E-03
1.0277E+00
1.0123E+00
1.0194E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.5500E+01
1.5500E+01
1.5500E+01
1.5500E+01
5.8347E+01
1.5500E+01
1.5500E+01
1.5500E+01
19 / 31
Journal Pre-proof
F28
+ο―=ο―β +ο―=ο―β
9.5444E+03
5.0072E+02
1.9890E+02
2.4305E+04
5.2293E+04
1.1876E+04
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.9800E+06
3.8300E+07
1.0000E+07
1.6451E+04
5.7098E+04
2.0800E+08
1.0800E+08
3.4100E+08
1.1185E+02
1.5971E+02
1.3606E+02
1.4627E+02
3.8200E+01
1.0807E+02
1.5285E+02
6.4323E+01
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.1459E+04
2.1492E+04
2.1478E+04
2.1491E+04
2.1466E+04
2.1462E+04
2.1482E+04
2.1446E+04
Criterion I Criterion II
22/3/3 16/11/1
13/5/10 4/21/3
20/1/7 20/5/3
22/0/6 20/4/4
24/1/3 21/5/2
20/2/6 NA
16/3/9 NA
Table 7. The mean percentage of number of surrogate evaluated populations for each 30D problems F1
F2
F3
F4
F5
F6
F7
Mean
32.34%
32.82%
22.31%
15.76%
32.56%
35.72%
33.87%
Variance
4.32%
3.76%
5.89%
5.23%
4.56%
4.82%
5.11%
Function
F8
F9
F10
F11
F12
F13
F14
Mean
30.12%
32.21%
30.11%
31.19%
32.82%
33.97%
29.95%
Variance
5.08%
5.91%
4.12%
4.21%
5.29%
4.18%
4.02%
Function
F15
F16
F17
F18
F19
F20
F21
Mean
31.12%
32.12%
16.23%
18.69%
15.89%
32.121%
32.21%
Variance
3.38%
3.45%
1.89%
3.05%
2.76%
5.12%
6.12%
Function
F22
F23
F24
F25
F26
F27
F28
Mean
30.12%
35.82%
35.14%
34.21%
22.1%
16.79%
18.19%
Variance
4.12%
5.12%
4.35%
4.23%
2.09%
2.91%
2.29%
4
Table 8. Experimental results in mean, FR and Μ Μ Μ Μ Μ ππΌπ on functions 1-28 for 50D problems F1
F2
F3
F4
F5
F6
F7
F8
F9
F10 F11
ASA-DE 8.57e-29 1 0 6.97e-29 1 0 885090 1 0 13.5728 1 0 8.35e-29 1 0 7941.5900 0.4 0 -145.6200 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 -1.4199 0.92
CAL-SHADE 6.38e-28 1 0 7.49-28 1 0 725750 1 0 13.7135 1 0 0.3189 1 0 7154.11 0 0.0116 -260.388 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 1.7584 0.44
L-SHADE44 7.75e-29 1 0 7.96e-29 1 0 946043 1 0 13.5728 1 0 8.92e-29 1 0 7946.24 0 0.01264 -172.159 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 -1.6628 1
urn a
Func
Jo
5
re-
Function
lP
1 2 3
1.2675E+04
pro of
F27
2.3376E+03
SaDE 0.02424 1 0 0.01425 1 0 6.49e+06 0.48 7.87e-05 40.796 1 0 10.9402 1 0 7615.43 0 0.2675 -21.6109 0.04 0.00102 0.00366 0.96 4.76e-06 -0.00203 1 0 5.07e-05 1 0 48.59 0
SajDE 5.10e-15 1 0 3.58e-15 1 0 1.96e-15 1 0 2.587 0 0.7517 21.2546 1 0 4.7360 0.2 0.9332 -2563.96 0 5011.44 -14.9387 0 1.82e+07 0.7054 0 21419.9 -16.7263 0 8.33e+07 -4946.35 0
DEbin 0.1425 1 0 0.0455 1 0 5.06e+06 0.44 6.51e-05 13.5728 1 0 16.6896 1 0 7550.13 0 0.3818 -13.0956 0 0.00155 0.04298 0 0.00298 -0.00204 1 0 -3.24e-05 1 0 65.2575 0
L-S44+IDE 1.49e-08 1 0 0 1 0 2.65e+07 1 0 13.9880 1 0 0 1 0 8600.818 0 0.01525 -153.963 1 0 0.000286 1 0 -0.00168 1 0 9.12e-05 1 0 -1.9254 1
UDE 1.16e-10 1 0 3.24e-11 1 0 90.6979 1 0 158.418 1 0 14.9547 1 0 687.2246 0.96 5.85e-06 -941.8285 0.88 1.66e-05 0.0002 1 0 -0.00204 1 0 5.81e-05 1 0 -160.1565 0
20 / 31
Journal Pre-proof
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28 +ο―=ο―β +ο―=ο―β
2.4313 4.1081 1 0 70.7797 1 0 1.6326 1 0 20.3261 1 0 302.85 1 0 1.0492 0 25.5 14321.2 0 5.46e+07 1.49e-05 0 36116.2 7.7770 1 0 57.1599 1 0 40693 0 43.6401 1.5225 1 0 17.9385 1 0 315.039 1 0 1.04887 0 25.5 48022.1 0 1.84e+09 248.88 0 36321.5 23/2/3 23/4/1
0 7.3566 1 0 73.6353 1 0 1.4611 1 0 13.6659 1 0 251.4531 1 0 1.0450 0 25.5 7340.76 0 8.96e+06 0 0 36116.2 4.5789 1 0 62.1685 1 0 7838.53 1 0 1.3742 1 0 14.9225 1 0 254.1548 1 0 1.0411 0 25.5 6.37e+04 0 2.00e+08 267.1889 0 36317.4 18/0/10 NA
0.3887 12.9012 1 0 1293.788 1 0 1.1840 1 0 11.2783 1 0 13.069 1 0 1.0480 0 25.5 20155.975 0 6.56e+08 8.8444 0 36130.22 6.8396 1 0 6.8466 1 0 3186.796 0.76 0.2074 1.1240 1 0 11.6550 1 0 27.52 1 0 1.0478 0 25.5 60198.128 0 6.52e+09 129.11 0 36258.2 16/1/11 NA
Table 9. The mean percentage of number of surrogate evaluated populations for each 50D problems Function
F1
F2
F3
F4
F5
F6
F7
Mean
30.21%
32.67%
18.32%
15.12%
28.32%
31.56%
29.99%
Variance
3.29%
3.39%
6.21%
7.53%
6.19%
4.87%
5.14%
Function
F8
F9
F10
F11
F12
F13
F14
Mean
28.32%
30.32%
28.13%
29.18%
36.82%
39.12%
33.39%
Variance
3.12%
8.39%
3.87%
4.78%
5.11%
6.18%
5.21%
Function
F15
F16
F17
F18
F19
F20
F21
Mean
29.12%
23.41%
16.12%
12.37%
10.21%
22.12%
32.18%
Variance
2.34%
5.12%
2.34%
5.89%
3.45%
5.77%
5.32%
Jo
1 2 3
2.70e+99 11.3024 0 4.5132 10.5261 0.8 1.58983 0.3148 0 2.5679 31.4785 0 2279.59 4.76e-15 1 0 0.00856 0 118.433 6.6 0 5542.67 -46.041 0 36185.7 3.2270 0.96 0.004264 235.007 0 114.908 85.3516 0.76 217.017 21.2107 0 443239 10.2124 0 277.846 51.0171 0 0.8301 0.003844 0 117.822 416.528 0 149219 91.0368 0 36294.1 24/0/4 21/2/5
pro of
F15
2.26e+50 4.0002 1 0 126.221 1 0 1.4327 1 0 20.7031 1 0 375.734 0.92 4.78e-06 1.0468 0 25.5 857.646 0 18625.7 6.52e-06 0 36116.2 6.2603 1 0 25.4192 1 0 74216.5 0 51.4983 1.5441 1 0 75.9954 0 21883.1 609.72 0 3346.36 1.0475 0 25.5 818.044 0 69432 275.074 0 36328 21/0/7 22/2/4
re-
F14
0 53.3338 1 0 26.824 1 0 1.4119 1 0 17.5615 1 0 260.124 1 0 1.0312 0 25.5 6219.43 0 4.05e+07 1.21e-05 0 36116.2 3.2117 1 0 63.3177 1 0 8051.21 0.96 0.0056 1.3257 1 0 13.9172 1 0 253.526 1 0 1.0366 0 25.5 13404.6 0 2.00e+07 269.195 0 36316.3 14/5/9 8/20/0
lP
F13
6.11e-12 27.5311 1 0 51.9073 1 0 1.5756 1 0 20.4518 0.12 0.00016 281.738 1 0 1.0416 0 25.5 9581.65 0 1.51e+07 1.51e-05 0 36116.2 2.5485 1 0 12.2582 1 0 64028.9 0.28 3.0501 1.5710 1 0 17.0588 1 0 299.896 1 0 1.0435 0 25.5 19907 0 2.33e+08 268.093 0 36324 20/3/5 18/7/3
urn a
F12
1.83e-32 51.6255 1 0 23.2901 1 0 1.4330 1 0 18.9438 0.96 2.63e-06 256.542 1 0 1.0384 0 25.5 854.438 0 134670 1.21e-05 0 36116.2 3.1344 1 0 62.1813 1 0 2997.47 1 0 1.3788 1 0 14.1685 1 0 252.584 1 0 1.0389 0 25.5 4866.84 0 1.53e+07 213.915 0 36273 Criterion I Criterion II
21 / 31
Journal Pre-proof Function
F22
F23
F24
F25
F26
F27
F28
Mean
24.89%
32.12%
29.02%
32.18%
24.21%
15.32%
15.32%
Variance
5.72%
7.81%
4.87%
6.71%
1.34%
1.87%
1.87%
1 Table 10. Experimental results in mean, FR and Μ Μ Μ Μ Μ ππΌπ on functions 1-28 for 100D problems
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17 F18
L-S44+IDE 7 1 0 3 1 0 2.25e+08 1 0 14.028 1 0 0 1 0 15533 0 1.19e-02 -335.49 1 0 1.48e-03 1 0 -1.45e-04 1 0 4.25e-04 1 0 -2.834 0.96 0 7.015 1 0 71.760 1 0 0.977 1 0 16.053 1 0 529.74 1 0 1.0980 0 50.5 10091 0
pro of
F3
L-SHADE44 1.03e-25 1 0 8.47e-26 1 0 2.73e+06 1 0 13.7132 1 0 3.28e-05 1 0 15562.2 0 0.0098 -302.689 1 0 -4.81e-05 1 0 -0.00143 1 0 -1.72e-05 1 0 3.6511 0.88 5.25e-43 32.5056 1 0 80.7003 1 0 0.9718 1 0 18.0641 1 0 534.51 1 0 1.0962 0 50.5 3436.37 0
re-
F2
CAL-SHADE 0.9777 1 0 0.3661 1 0 1.514e+07 0.16 0.024 413.582 1 0 0.8188 1 0 15222.9 1 0 -193.458 0.4 0.00436 0.04160 0 0.00124 0.5225 0.96 1.564e-05 0.00051 1 0 -13.7384 0 0.14709 23.8996 1 0 118844 0 18.7453 0.7949 1 0 30.8846 0.6 1253.21 712.859 0.12 0.0488 1.0980 0 54.2153 16640.7 0
lP
F1
ASA-DE 7.21e-26 1 0 6.12e-21 1 0 2.12e+04 1 0 13.8310 1 0 2.35e-06 1 0 1932.4500 0.4 0 -132.6200 1 0 -4.81e-05 1 0 -1.45e-04 1 0 4.25e-04 1 0 -2.2199 0.92 5.83e-21 51.6255 1 0 67.3201 1 0 0.7671 1 0 18.0641 0.96 2.12e-06 26.2421 1 0 1.0980 0 54.2153 854.438 0
urn a
Func
Jo
2
UDE 6.16e-05 1 0 5.15e-05 1 0 388.18 1 0 386.65 1 0 41.284 1 0 2185.52 1 0 -1649.1 0.8 0.07357 0.00046 1 0 -0.00068 0.52 1.35e-21 1.61e-05 1 0 -549.15 0 1.2268 6.13 1 0 16793.687 0.48 3.47 0.7927 1 0 16.3048 1 0 21.7397 1 0 1.0987 0 50.5 64364.38 0 22 / 31
Journal Pre-proof
F22
F23
F24
F25
F26
F27
F28 +ο―=ο―β +ο―=ο―β
Function
F3
F4
F5
F6
F7
31.33%
20.22%
16.74%
23.32%
30.12%
28.12%
3.12%
3.33%
5.99%
6.92%
5.65%
4.12%
5.08%
F8
F9
F10
F11
F12
F13
F14
29.76%
29.12%
27.43%
27.89%
32.12%
37.21%
31.27%
3.23%
7.21%
4.87%
5.18%
3.99%
6.18%
5.08%
F15
F16
F17
F18
F19
F20
F21
27.31%
27.12%
20.98%
17.17%
12.12%
24.22%
30.23%
5.12%
7.12%
3.09%
5.23%
3.87%
5.21%
5.12%
Function
F22
F23
F24
F25
F26
F27
F28
Mean
26.87%
34.56%
27.78%
32.22%
27.21%
18.76%
18.12%
Variance
6.12%
6.12%
5.12%
6.12%
2.11%
2.01%
2.12%
Mean Variance Function Mean Variance Function Mean
F1
F2
28.93%
Jo
Variance
4 5
3.02e+09 38.0444 0 73030.81 15.1336 1 0 10.2589 1 0 44948.02 0 368.69 0.7842 1 0 17.81 1 0 102.29 1 0 1.098 0 50.5 197018.15 0 3.19e+10 298.416 0 73303.66 23/0/5 23/1/4
Table 11. The mean percentage of number of surrogate evaluated populations for each 100D problems
urn a
1 2 3
pro of
F21
2.74e+07 0 0 7.30e+04 11.480 1 0 29.544 1 0 60692 0.04 10.594 0.958 1 0 17.938 1 0 571.016 1 0 1.097 0 50.5 140923 0 1.45e+09 610.2 0 734e+04 27/1/0 24/0/4
3.34e+06 4.68e-05 0 72969.5 9.3628 1 0 31.5798 1 0 50364.5 0.04 6.4572 0.9694 1 0 17.1845 1 0 544.187 1 0 1.09616 0 50.5 36875 0 4.78e+08 584.017 0 73404.3 25/1/2 15/11/2
re-
F20
4.05e+09 8.82e-05 0 48646.3 7.4038 1 0 14.9419 1 0 263043 0 46.1771 0.8144 0.92 . 3.68e-05 22.8996 0.08 234.413 724.212 0.24 0.02953 1.101 0 52.6093 47247.7 0 1.37e+09 415.526 0 48880.7 25/1/2 19/6/3
lP
F19
134670 1.12e-05 0 32116.2 2.2314 1 0 21.1813 1 0 3345.47 1 0 0.7981 1 0 13.1123 1 0 252.584 1 0 1.0389 0 50.5 4982.1289 0 1.24e+07 187.6720 0 38982.2 Criterion I Criterion II
Table 12 Successful optimization rate for benchmark functions Dimension
10D
Algorithm
ASA-DE
CAL-SHADE
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
SR
78.57%
71.42%
71.42%
64.29%
35.71%
67.86%
75%
75%
23 / 31
Journal Pre-proof Failure(f)
6
8
8
10
Dimension ASA-DE
CAL-SHADE
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
71.42%
75%
60.71%
32.14%
67.86%
75%
75%
Failure(f)
6
8
7
11
19
9
7
7
50D
Algorithm
ASA-DE
CAL-SHADE
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
SR
78.57%
67.86%
75%
57.14%
28.57%
60.71%
75%
75%
Failure(f)
6
9
7
12
20
11
7
7
100D ASA-DE
CAL-SHADE
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
SR
65.57%
45.86%
71.2%
N/A
N/A
N/A
70.21%
67.21%
Failure(f)
5
5
6
N/A
N/A
N/A
7
6
pro of
Algorithm
lP
re-
Moreover, the bar charts of statistical results of ASA-DE and 8 state-of-the-art other algorithms on 28 CEC2017 benchmark functions are presented when the dimensions are 50 and 100, where the blue and red bars represent the rank score of Friedman test and the number of functions with the best results, respectively. According to Fig.2, it can be found that the ASADE has the best overall performance and the advantages are more obvious in the 50D and 100D problems comparing with other DEs. It can be easily found that the ASA-DE is the best algorithm in terms of these criteria.
50D
Jo
urn a
10
13 14 15 16
7
78.57%
1
12
7
SR
Dimension
11
9
Algorithm
Dimension
2 3 4 5 6 7 8 9
18
30D
100D Fig.2 The rank score and the number of functions with the best results of ASA-DE on CEC2017 benchmark functions.
24 / 31
Journal Pre-proof
F4
F5
F3
pro of
F2
F6
F8
F9
F10
urn a
F11
F12
Jo
F7
lP
re-
F1
F13
F14
F15
25 / 31
Journal Pre-proof
F19
F20
F18
pro of
F17
F21
F22
F24
F25
F26
F27
F28
urn a
F23
Jo
lP
re-
F16
1 2 3 4
Fig.3 Convergence curves for 28 test function (100D) Furthermore, to investigate the convergence ratio, several representative function iteration curves with 100 variables are demonstrated in Fig. 3. In these figures, X and Y axial represent 26 / 31
Journal Pre-proof 1 2 3
the number of function evaluations (FES) and the logarithm of the function error value, respectively. According to these results, the suggested algorithm also are easier to converge for most of test functions.
4
4 Conclusions An adaptive surrogate assisted differential evolution with novel mutation strategy, called ASA-DE is suggested for constrained optimization problems. In the ASA-DE method, several kinds of surrogate-modeling techniques are integrated. Compared with other surrogate modeling assisted DE methods, more surrogate modeling techniques are used. Due the diversity of modeling performance, such strategy is easier to obtain high accurate model. Moreover, considering the curse of dimensionality, the surrogate model should be constructed based on the local region. The test results suggests that the surrogate modeling technique is still useful for the highly dimensional problems. Moreover, in order to follow the change of suitable balance point between exploration and exploitation, adaptive population size control based on pfeas and parameter adaptation strategy based on success-history are suggested. When these strategies are integrated, the performance of the optimization based on the ASA-DE can be guaranteed. According the results of comparison with other state-of-the-art algorithms, ASA-DE show a better or at least competitive performance compared to other algorithms. The experimental results also indicate that the state-switching mechanism can improve the performance of the algorithm. However, it is difficult for ASA-DE to solve f17, f18, f19, f26, f27, f28, but other algorithms also fail to search feasible solutions. Moreover, the number of benchmark functions that ASADE finds feasible solutions is the most compared to other algorithms.
23
Acknowledgment
24 25
This work has been supported by the Key Projects of the Research Foundation of Education Bureau of Hunan Province (17A224).
26
References
re-
lP
urn a
1.
Hu, W., Yao, L. G., & Hua, Z. Z. (2008). Optimization of sheet metal forming processes by adaptive response surface based on intelligent sampling method. Journal of materials processing technology, 197(1-3), 77-
Jo
27 28 29 30 31 32 33 34 35 36
pro of
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
88. 2.
Abedinia, O., Naderi, M. S., Jalili, A., & Mokhtarpour, A. (2011). A novel hybrid GA-PSO technique for optimal tuning of fuzzy controller to improve multi-machine power system stability. International Review of Electrical Engineering, 6(2).
3.
Abedinia, O., Amjady, N., Kiani, K., Shayanfar, H. A., & Ghasemi, A. (2012). Multiobjective environmental and economic dispatch using imperialist competitive algorithm. International Journal on Technical and Physical Problems of Engineering (IJTPE), (11), 63-70.
4.
Kar, A. K. (2016). Bio inspired computingβA review of algorithms and scope of applications. Expert 27 / 31
Journal Pre-proof Systems with Applications, 59, 20-32. 5.
Abedinia, O., Ghasemi, A., & Ojaroudi, N. (2016). Improved time varying inertia weight PSO for solved economic load dispatch with subsidies and wind power effects. Complexity, 21(4), 40-49.
6.
Brest, J., Zumer, V., & Maucec, M. S. (2006, July). Self-adaptive differential evolution algorithm in constrained real-parameter optimization. In Evolutionary Computation, 2006. CEC 2006. IEEE Congress on (pp. 215-222). IEEE.
7.
Hamza, N. M., Sarker, R. A., & Essam, D. L. (2013). Differential evolution with multi-constraint consensus methods for constrained optimization. Journal of Global Optimization, 57(2), 583-611. Das, S., Mullick, S. S., & Suganthan, P. N. (2016). Recent advances in differential evolutionβan updated
pro of
8.
survey. Swarm and Evolutionary Computation, 27, 1-30. 9.
Storn, R., & Price, K. (1997). Differential evolutionβa simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11(4), 341-359.
10.
Brest, J., Greiner, S., Boskovic, B., Mernik, M., & Zumer, V. (2006). Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE transactions on evolutionary computation, 10(6), 646-657.
11.
Qin, A. K., Huang, V. L., & Suganthan, P. N. (2009). Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE transactions on Evolutionary Computation, 13(2),
re-
398-417. 12.
Zhang, J., & Sanderson, A. C. (2009). JADE: adaptive differential evolution with optional external archive. IEEE Transactions on evolutionary computation, 13(5), 945-958.
13.
Tanabe, R., & Fukunaga, A. (2013, June). Success-history based parameter adaptation for differential evolution. In Evolutionary Computation (CEC), 2013 IEEE Congress on (pp. 71-78). IEEE. Tang, L., Dong, Y., & Liu, J. (2015). Differential evolution with an individual-dependent mechanism. IEEE
lP
14.
Transactions on Evolutionary Computation, 19(4), 560-574. 15.
Zhou, Y. Z., Yi, W. C., Gao, L., & Li, X. Y. (2017). Adaptive differential evolution with sorting crossover rate for continuous optimization problems. IEEE transactions on cybernetics, 47(9), 2742-2753.
16.
Bujok, P., Tvrdik, J., & Polakova, R. (2014, July). Differential evolution with rotation-invariant mutation
urn a
and competing-strategies adaptation. In Evolutionary Computation (CEC), 2014 IEEE Congress on (pp. 2253-2258). IEEE. 17.
Wang, Y., Cai, Z., & Zhang, Q. (2011). Differential evolution with composite trial vector generation strategies and control parameters. IEEE Transactions on Evolutionary Computation, 15(1), 55-66.
18.
Trivedi, A., Sanyal, K., Verma, P., & Srinivasan, D. (2017, June). A unified differential evolution algorithm for constrained optimization problems. In Evolutionary Computation (CEC), 2017 IEEE Congress on (pp. 1231-1238). IEEE.
19.
Zamuda, A., & Brest, J. (2012). Population reduction differential evolution with multiple mutation strategies in real world industry challenges. In Swarm and evolutionary computation (pp. 154-161).
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
Springer, Berlin, Heidelberg. 20.
Tanabe, R., & Fukunaga, A. S. (2014, July). Improving the search performance of SHADE using linear population size reduction. In Evolutionary Computation (CEC), 2014 IEEE Congress on (pp. 1658-1665). IEEE.
21.
Brest, J., & Maucec, M. S. (2011). Self-adaptive differential evolution algorithm using population size reduction and three strategies. Soft Computing, 15(11), 2157-2174.
22.
Yang, Z., Tang, K., & Yao, X. (2008, June). Self-adaptive differential evolution with neighborhood search. In Evolutionary Computation, 2008. CEC 2008. (IEEE World Congress on Computational Intelligence). 28 / 31
Journal Pre-proof IEEE Congress on (pp. 1110-1116). IEEE. 23.
Du, W., Leung, S. Y. S., Tang, Y., & Vasilakos, A. V. (2016). Differential evolution with event-triggered impulsive control. IEEE transactions on cybernetics, 47(1), 244-257.
24.
Awad, N. H., Ali, M. Z., & Suganthan, P. N. (2018). Ensemble of parameters in a sinusoidal differential evolution with niching-based population reduction. Swarm and evolutionary computation, 39, 141-156.
25.
Wang, S., Li, Y., & Yang, H. (2019). Self-adaptive mutation differential evolution algorithm based on particle swarm optimization. Applied Soft Computing, 81, 105496.
26.
Coit, D. W. , Smith, A. E. , & Tate, D. M. . (1996). Adaptive penalty methods for genetic optimization of
27.
pro of
constrained combinatorial problems. INFORMS Journal on Computing, 8(2), 173-182.
Farmani, R. , & Wright, J. A. . (2003). Self-adaptive fitness formulation for constrained optimization. Evolutionary Computation IEEE Transactions on, 7(5), 445-455.
28.
Deb, K. (2000). An efficient constraint handling method for genetic algorithms. Computer methods in applied mechanics and engineering, 186(2-4), 311-338.
29.
Yao, T. P. R. X. . (2000). Stochastic ranking for constrained evolutionary optimization. Evolutionary Computation IEEE Transactions on, 4(3), 284-294.
30.
Mezuramontes, E , and C. A. C. Coello . "A simple multimembered evolution strategy to solve constrained optimization problems." IEEE Transactions on Evolutionary Computation 9.1(2005):1-17. Takahama, T. , & Sakai, S. (2005). Constrained optimization by applying the Ξ± constrained method to the
re-
31.
nonlinear simplex method with mutations. IEEE Press. 32.
Coello, Carlos A. Coello.(2000) "Treating constraints as objectives for single-objective evolutionary optimization." Engineering Optimization, 32(3),34.
33.
Coello, C. A. C. , & Efren Mezura Montes. (2002). Constraint-handling in genetic algorithms through the
34.
lP
use of dominance-based tournament selection. Advanced Engineering Informatics, 16(3), 193-203. Ray, T. , & Liew, K. M. . (2003). Society and civilization: an optimization algorithm based on the simulation of social behavior. IEEE Transactions on Evolutionary Computation, 7(4), 386-396. 35.
Cai, Z., & Wang, Y. (2006). A multiobjective optimization-based evolutionary algorithm for constrained optimization. IEEE Transactions on evolutionary computation, 10(6), 658-675. Wang, Y., Cai, Z., Guo, G., & Zhou, Y. (2007). Multiobjective optimization and hybrid evolutionary
urn a
36.
algorithm to solve constrained optimization problems. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 37(3), 560-575. 37.
Wang, Y., Cai, Z., Zhou, Y., & Zeng, W. (2008). An adaptive tradeoff model for constrained evolutionary optimization. IEEE Transactions on Evolutionary Computation, 12(1), 80-92.
38.
Wang, Y., & Cai, Z. (2011). Constrained evolutionary optimization by means of (ΞΌ+ Ξ»)-differential evolution and improved adaptive trade-off model. Evolutionary Computation, 19(2), 249-285.
39.
Wang, Y., & Cai, Z. (2012). Combining multiobjective optimization with differential evolution to solve constrained optimization problems. IEEE Transactions on Evolutionary Computation, 16(1), 117-134.
40.
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
Mohamed, A. W., & Sabry, H. Z. (2012). Constrained optimization based on modified differential evolution algorithm. Information Sciences, 194, 171-208.
41.
Hernandez, S., Leguizamon, G., & Mezura-Montes, E. (2013). Hybridization of differential evolution using hill climbing to solve constrained optimization problems. Inteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial, 16(52).
42.
Sarker, R. A., Elsayed, S. M., & Ray, T. (2014). Differential evolution with dynamic parameters selection for optimization problems. IEEE Transactions on Evolutionary Computation, 18(5), 689-707.
43.
Zhang, W., Yen, G. G., & He, Z. (2014). Constrained optimization via artificial immune system. IEEE 29 / 31
Journal Pre-proof Transactions on Cybernetics, 44(2), 185-198. 44.
Ong, Y. S., Nair, P. B., & Keane, A. J. (2003). Evolutionary optimization of computationally expensive problems via surrogate modeling. AIAA journal, 41(4), 687-696.
45.
Jin, Y. (2011). Surrogate-assisted evolutionary computation: Recent advances and future challenges. Swarm and Evolutionary Computation, 1(2), 61-70.
46.
Jin, Y. (2005). A comprehensive survey of fitness approximation in evolutionary computation. Soft computing, 9(1), 3-12.
47.
Shi, L., & Rasheed, K. (2008, July). ASAGA: an adaptive surrogate-assisted genetic algorithm. In
pro of
Proceedings of the 10th annual conference on Genetic and evolutionary computation (pp. 1049-1056). ACM. 48.
Gao, W., Yen, G. G., & Liu, S. (2014). A cluster-based differential evolution with self-adaptive strategy for multimodal optimization. IEEE transactions on cybernetics, 44(8), 1314-1327.
49.
Miranda-Varela, M. E., & Mezura-Montes, E. (2016). Surrogate-assisted differential evolution with an adaptive evolution control based on feasibility to solve constrained optimization problems. In Proceedings of Fifth International Conference on Soft Computing for Problem Solving (pp. 809-822). Springer, Singapore.
Li, E., & Wang, H. (2016). Bi-direction multi-surrogate assisted global optimization. Engineering Computations, 33(3), 646-666.
51.
re-
50.
Awad, N. H., Ali, M. Z., Mallipeddi, R., & Suganthan, P. N. (2018). An improved differential evolution algorithm using efficient adapted surrogate model for numerical op51timization. Information Sciences, 451, 326-347.
52.
Taran, N., Ionel, D. M., & Dorrell, D. G. (2018). Two-level surrogate-assisted differential evolution multi-
53.
lP
objective optimization of electric machines using 3-D FEA. IEEE Transactions on Magnetics, 54(11), 1-5. Cai, X., Gao, L., Li, X., & Qiu, H. (2019). Surrogate-guided differential evolution algorithm for high dimensional expensive problems. Swarm and Evolutionary Computation, 48, 288-311. 54.
Elsayed, S. M., Sarker, R. A., & Essam, D. L. (2013). An improved self-adaptive differential evolution algorithm for optimization problems. IEEE Transactions on Industrial Informatics, 9(1), 89-99. Brest, J., Boskovic, B., & Zumer, V. (2010, July). An improved self-adaptive differential evolution algorithm
urn a
55.
in single objective constrained real-parameter optimization. In Evolutionary Computation (CEC), 2010 IEEE Congress on (pp. 1-8). IEEE. 56.
Mallipeddi, R., & Suganthan, P. N. (2010). Ensemble of constraint handling techniques. IEEE Transactions on Evolutionary Computation, 14(4), 561-579.
57.
Mezura-Montes, E., & Coello, C. A. C. (2011). Constraint-handling in nature-inspired numerical optimization: past, present and future. Swarm and Evolutionary Computation, 1(4), 173-194.
58.
Farmani, R., & Wright, J. A. (2003). Self-adaptive fitness formulation for constrained optimization. IEEE Transactions on Evolutionary Computation, 7(5), 445-455.
59.
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
Tessema, B., & Yen, G. G. (2006, July). A self adaptive penalty function based algorithm for constrained optimization. In Evolutionary Computation, 2006. CEC 2006. IEEE Congress on (pp. 246-253). IEEE.
60.
Kleijnen, J. P. (2009). Kriging metamodeling in simulation: A review. European journal of operational research, 192(3), 707-716.
61.
Clarke, S. M., Griebsch, J. H., & Simpson, T. W. (2005). Analysis of support vector regression for approximation of complex engineering analyses. Journal of mechanical design, 127(6), 1077-1087.
62.
Regis, R. G., & Shoemaker, C. A. (2005). Constrained global optimization of expensive black box functions using radial basis functions. Journal of Global optimization, 31(1), 153-171. 30 / 31
Journal Pre-proof 63.
Wang, H., Tang, L., & Li, G. Y. (2011). Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Systems with Applications, 38(11), 14117-14126.
64.
Chen, L., Wang, H., Ye, F., & Hu, W. (2018). Comparative study of HDMRs and other popular metamodeling techniques for high dimensional problems. Structural and Multidisciplinary Optimization, 1-22.
65.
Wang, H., Tang, L., & Li, G. Y. (2011). Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Systems with Applications, 38(11), 14117-14126.
66.
Piotrowski, A. P. (2013). Adaptive memetic differential evolution with global and local neighborhoodbased mutation operators. Information Sciences, 241, 164-194. Ali, M. Z., Awad, N. H., Suganthan, P. N., & Reynolds, R. G. (2017). An adaptive multipopulation differential
pro of
67.
evolution with dynamic population reduction. IEEE transactions on cybernetics, 47(9), 2768-2779. 68.
Zhou, Y., Li, X., & Gao, L. (2013). A differential evolution algorithm with intersect mutation operator. Applied Soft Computing, 13(1), 390-401.
69.
Wu, G., Mallipeddi, R., & Suganthan, P. N. (2017). Problem definitions and evaluation criteria for the cec 2017 competition on constrained real-parameter optimization. National University of Defense Technology, Changsha, Hunan, PR China and Kyungpook National University, Daegu, South Korea and Nanyang Technological University, Singapore, Technical Report.
70.
Yang, M., Cai, Z., Li, C., & Guan, J. (2013, July). An improved adaptive differential evolution algorithm with computation (pp. 145-152). ACM.
71.
re-
population adaptation. In Proceedings of the 15th annual conference on Genetic and evolutionary Derrac, J., GarcΓa, S., Molina, D., & Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation, 1(1), 3-18.
Zamuda, Ales. Adaptive constraint handling and success history differential evolution for CEC 2017
lP
72.
constrained real-parameter optimization. Evolutionary Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. 73.
TvrdΓk, Josef, and Radka Polakova. A simple framework for constrained problems with application of LSHADE44 and IDE. Evolutionary Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. Polakova, Radka. L-SHADE with competing strategies applied to constrained optimization. Evolutionary
urn a
74.
Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. 75.
Brest, Janez, Viljem Zumer, and Mirjam Sepesy Maucec. Self-adaptive differential evolution algorithm in constrained real-parameter optimization. Evolutionary Computation, 2006. CEC 2006. IEEE Congress on. IEEE, 2006.
76.
Suykens, J. A., Van Gestel, T., & De Brabanter, J. (2002). Least squares support vector machines. World Scientific.
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
31 / 31
Journal Pre-proof
Highlights An adaptive surrogate assisted DE algorithm is suggested;
2.
Several kinds of surrogate modeling techniques are integrated in suggested algorithm;
3.
An adaptive population size strategy based on pfeas is suggested;
4.
Parameter adaptation strategy based on success-history is suggested;
5.
The suggested algorithm is evaluated by the benchmark problems from CEC2017.
Jo
urn a
lP
re-
pro of
1.
Journal Pre-proof
Conflict of interest statement
pro of
We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled βAn adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problemsβ
Jo
urn a
lP
re-
Authors Enying Li Zheng Zhou College of Mechanical & Electrical Engineering, Central South University of Forestry and Technology, Changsha, 41004, P.R. China
PII: DOI: Reference:
S1568-4946(19)30533-2 https://doi.org/10.1016/j.asoc.2019.105752 ASOC 105752
To appear in:
Applied Soft Computing Journal
Received date : 7 November 2018 Revised date : 2 August 2019 Accepted date : 2 September 2019 Please cite this article as: E. Li, An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems, Applied Soft Computing Journal (2019), doi: https://doi.org/10.1016/j.asoc.2019.105752. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Β© 2019 Elsevier B.V. All rights reserved.
Journal Pre-proof
1
An adaptive surrogate assisted differential evolutionary
2
algorithm for high dimensional constrained problems Enying Liο College of Mechanical & Electrical Engineering, Central South University of Forestry and Technology, Changsha, 41004, P.R. China Nomenclature
pro of
3 4 5 6
DE
constrained optimization problems
congress on evolutionary computation
CEC
Evolutionary Algorithm
EA
Function Evaluations
FEs
Kriging
KG
Radial Basis Function
RBF
leave-one-out cross validation
LOOCV
Polynomial Regression
PR
Support Vector Regression
SVR
scaling factor
F Np
exclusive integers
the best individual in the population at the iteration t
π 1π ,
π 2π , π 3π (π‘)
target vector
π₯π,π‘
mutation vector
π₯πππ π‘ π£π,π‘
trail vector
π’π,π‘
decision variable
πππππ
inequality constraint
ππ (π₯)
design space
π
tolerance parameter feasible ratio normalized fitness value population
βπ (π₯)
number of inequality constraints
lP
quality constraint
q
constraint violation
π(π₯)
πππππ
overall constraint violation
π£(π₯)
π β²β² (π₯)
penalty value
π(π₯)
πΏ
π(πΊ)
urn a
the kth subpopulation of the Gth generation
πππΊ
present number of the successful evolutions in sth subpopulation
ππ’π
entire population size in the Gth generation
πππΊ
small positive constant
ππ’0
preset minimum population size
πππππ
initial population size
πππππ₯
population size in the generation
πππππ₯
Abstract
Jo
8 9 10 11 12 13
Cr
crossover rate
re-
population size in the tth generation
7
COPs
differential evolution
Differential evolution (DE) is a competitive algorithm for constrained optimization problems (COPs). In this study, in order to improve the efficiency and accuracy of the DE for high dimensional problems, an adaptive surrogate assisted DE algorithm, called ASA-DE is suggested. In the ASA, several kinds of surrogate modeling techniques are integrated. Furthermore, to avoid violate the constraints and obtain better solution simultaneously, adaptive strategies for population size and mutation are also suggested in this study. The
ο
Corresponding author [email protected]
1 / 31
Journal Pre-proof suggested adaptive population strategy which controls the exploring and exploiting states according to whether algorithm find enough feasible solution is similar to a state switch. The mutation strategy is used to enhance the effect of state switch based on adaptive population size. Finally, the suggested ASA-DE is evaluated on the benchmark problems from congress on evolutionary computation (CEC) 2017 constrained real parameter optimization. The experimental results show the proposed algorithm is a competitive one compared to other state-of-the-art algorithms.
8
Keywords
9
Surrogate assisted, Differential evaluation, Adaptive strategy, High dimensional problem
pro of
1 2 3 4 5 6 7
1. Introduction
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
Evolutionary Algorithm (EA) is a useful optimization method for complicated problems and has been applied to multidiscipline [1-5]. Differential Evolutions (DE) is an important algorithm of EAs and attracts lots of scholarsβ attention due to its competitive performance on constrained optimization problems over the past decades [6-8]. Additionally, most of practical optimization problems are Constrained Optimization Problems (COPs), which might lead to the difficulty for convergence. The purpose of this study is to find the global optimization solutions as accurately as possible for COPs. Since the DE was proposed [9], many efforts have been designed to enhance the performance of the DE for constrained problems. Most of them focused on the setting of control parameters such as scaling factor (F), crossover rate (Cr), selection mechanism, mutation strategies and population size (Np). However, parameter settings commonly are diverse for different cases. Furthermore, the interactions between performance and parameter are also complicated. It motivated many scholars to develop some adaptive parameter setting based DEs such as jDE [10], SaDE [11], JADE [12], SHADE [13], IDE [14] and JADE_sort [15]. The essential of these strategies is to tune control parameters in the present iteration according to feedbacks from the previous ones. These controlling strategies are commonly used to adjust Cr and F. Moreover, other kinds of parameter setting strategies were widely used such as EPSDE [16], adaptive DE [17] and CoDE [18], which select the parameters from a pool involving sets of candidate populations. The selection mechanism is another critical issue for the DE. The essential of this mechanism is based on greedy choice in the classical DE. These strategies have also improved the performance of the DE. Population size control is another important mechanism for the DE. Several adaptive or non-adaptive population size control methods have been proposed and achieved competitive performances. For instance, Zamuda et al. [19] controlled the population size based on increasing number of FEs, a set of novel mutation strategies which depending on population size were also suggested. It showed its superior performance on the CEC2011 real life problems. Tanabe et al. [20] employed the linear population size reduction strategy to improve the SHADE algorithm, the population size is reduced with increasing FEs (Function Evaluations) and called as L-SHADE. It will stop if the population size reaches the present minimum (Npmin). The L-SHADE also won the CEC2014 competition on real parameter single-objective optimization.
Jo
urn a
lP
re-
10
2 / 31
Journal Pre-proof
urn a
lP
re-
pro of
Another population size controlling-method called Linear Population Size Reduction (LPSR) requires only one parameter (the initial population size). Similar algorithm Dynamic Population Size Reduction (DPSR) [21] was suggested to reduce the population by half at preset intervals. These non-adaptive methods can increase or decrease (decrease in most cases) the population size according to preset conditions and simplify the method of controlling population size. Moreover, instead of reduction of the population size by fixed formula or predetermined path, several adaptive population size techniques were proposed recently. Yang et al. [22] proposed a novel strategy, which varies the population size in a predetermined range. It will detect the present state and decrease or increase the present population size according to the feedbacks from the previous iterations, and the population size will be expanded by introducing more solutions when the algorithm is unable to find a better solution within a preset number of consequent generations. On the contrary, if the algorithm updates the best solution successfully in successive generations, the population size will reduce in order to purge the redundant solutions. Many experiments and studies have proved the effectiveness of the population size control method. More recently, some novel strategies have been introduced to improve the performance of the DE. Du et al. introduced an Event-Triggered Impulsive (ETI) to improve the performance of DE [23]. Awad et al. presented an enhanced algorithm based on LSHADE with ensemble parameter sinusoidal adaptation called LSHADE-EpSin [24]. The LSHADE-EpSin suggested a mixture of two sinusoidal formulas and a Cauchy distribution to balance the exploration and the exploitation of existed best solutions. A restart method is used at later generations to enhance the quality of the found solutions. By introducing Impulsive Control (IPC) and ETM into DE, they hope to change the search performance of the population in a positive way after revising the positions of some individuals. Wang et al. suggested a self-adaptive mutation DE algorithm based on particle swarm optimization (DEPSO) [25]. Compared with the unconstrained problems, the COPs are more difficult to be handled because both constraint violation and objective function should be considered. The present popular constraint-handling strategies can be mainly categorized three types: penalty function methods [26-27]; methods based on preference of feasible solutions over infeasible ones [28-31]; multi-objective based methods [32-37]. Briefly, the purpose of the COPs is to find the feasible region efficiently and obtain the solution finally. For the DEs, some novel algorithms have been suggested. Wang and Cai [38] proposed (ΞΌ+Ξ»)-CDE framework. They also [39] developed CMODE algorithm by integration of multi-objective optimization with DE. Mohamed and Sabry [40] employed several strategies on the DE, involving mutation operators, F, and CR. All mentioned algorithms used direct evaluation way to obtain the response of objective function. Hernandez et al. [41] suggested a hybridization of DE and hill climbing, which uses static penalty to handle constraints. Sarker et al. [42] developed a DE with dynamic parameter selection. Zhang et al. [43] suggested a constrained artificial immune system based on immune response principle. Considering the number of evaluations, the computational cost and convergence ratio still need to be addressed, especially for the complicated problems. Therefore, Surrogate Assisted Evolutionary Algorithm (SAEA) was motived by this issue [44]. Theoretically, surrogates can be applied to most of operations of evolutionary algorithms, such as population initialization, cross-over, mutation, local search and fitness evaluation. In such algorithms, a surrogate model is used to evaluate the objective function by constructing an
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
3 / 31
Journal Pre-proof approximate model [45]. Over the last few decades, many surrogate models have been proposed, such as, Kriging(KG), Support Vector Regression(SVR), Polynomial Regression(PR), and Radial Basis Function (RBF) [46]. Similar to other algorithms, the DE has been incorporated with surrogate modeling techniques and achieved good results [47-50]. Awad et al. developed a novel DE with a novel adapted KG surrogate model called iDEaSm. During the modeling in the iDEaSm, the covariance matrix is used to adapt the theta parameter in the KG model at each generation [51]. Taran et al. suggested a two-level surrogate-assisted optimization algorithm is proposed for electric machine design [52]. Cai et al. introduced an efficient surrogate-guided DE based on JADE algorithm [53]. In this study, we hope to find an alternative way to improve the efficiency of surrogate assisted DE algorithm. Compared with other surrogate assisted algorithms, such as iDEaSm, several surrogate-modeling techniques are employed. The diversity of surrogate modeling should be enhanced and the best of them should be selected. by the given criteria. Therefore, the suggested method is called as Adaptive Surrogate Assisted DE (ASA-DE) algorithm. To evaluate performance of the ASA-DE, 28 benchmark functions from CEC2017 are tested. The results demonstrate that performance of the ASA-DE is better than, or at least competitive to the stateof-the-art constraint optimization algorithms, and show the competitiveness relative to the other excellent algorithm. The rest of the paper is organized as follows. The classical DE algorithm and several constraint handling methods for constrained optimization are briefly introduced in Section 2, Section 3 gives the detailed description of the proposed ASA-DE, and results of benchmark functions and comparison with other state-of-the-art algorithms are shown in Section 4. Finally, Section 5 summarizes the conclusions and discusses the direction of future work.
24
2. Classical DE algorithm and constraint handling
re-
lP
methods
urn a
25
pro of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
2.1 DE Algorithm
27 28 29 30 31 32 33 34 35 36 37
The process of the DE consists of three main operators: mutation, crossover and selection. Similar to other Evolutionary Algorithms (EAs) for optimization problems, the DE is based on population. A DE population consists of Np individual vectors π₯π,π‘ = (π₯1,π‘ , β¦ , π₯π,π‘ ), π = 1, β¦ , ππ, (1) where n is the number of design variables, Np is the population size in the present generation, the population individual vectors are generated randomly at the beginning of optimization. Np trail vectors are generated from the survived population individual by mutation and crossover operations in each generation t until termination criteria are satisfied. The details of DE are discussed according to Ref. [6] as follows. Mutation Operation: After the initialization, the DE adopts the mutation operation based on the difference of other individuals to generate mutant vector ππ,π‘ with respect to each target vector π₯π,π‘ , in the present generation. Mutation strategies influence the balance of exploration
38
and exploitation during the evolutionary process, some most frequently referred mutation
Jo
26
4 / 31
Journal Pre-proof strategies are listed below: "DE/rand/1": (π‘)
π£π 3
(π‘)
(π‘)
1
2
3
(π‘)
(π‘)
1
2
(2)
"DE/best/1": (π‘)
π£π 4
(π‘)
= π₯π π + πΉ (π₯π π β π₯π π )
(π‘)
(3)
= π₯πππ π‘ + πΉ (π₯π π β π₯π π )
"DE/current-to-best/1": (π‘)
π£π
(π‘)
= π₯π
(π‘)
(π‘)
(π‘)
(π‘)
1
2
pro of
1 2
+ πΉ (π₯πππ π‘ β π₯π ) + πΉ (π₯π π β π₯π π )
(4)
5 6
where the indices π 1π , π 2π , and π 3π are exclusive integers randomly chosen from the range [1, Np], (Np is population size in the tth generation). F is the scaling factor which usually lies in [0,
7
1] for scaling the difference vectors. π₯πππ π‘ is the best individual in the population at the
8 9 10 11 12 13 14 15 16 17
iteration t. Generally, mutation strategies can be classified into three categories: exploring biased, exploiting biased and balance modes. The DE/rand/1 belongs to the type of exploring biased; the DE/best/1 belongs to the type of exploiting biased. Moreover, DE/current-tobest/1 is a balance mode because the diversity is enhanced by adding a difference vector composed of random individuals and the use of best individual. The suitable balance point of exploration and exploitation is updating dynamically in the evolutionary process. Therefore, an adaptive mutation strategy based on feasible rate is suggested in this study to switch state between exploring biased and exploiting biased in order to keep the efficiency of algorithm. Crossover Operation: After the mutation, crossover operation is used to generate trail vector π’π,π‘ by replacing the components between the target vector π₯π,π‘ and the mutation vector π£π,π‘
18 19
with probability. Binomial crossover, which is the most frequently employed crossover operator in the DE used as follows: π
π£π,π‘ ππ ππππ[0,1) β€ πΆπ ππ π = π π ππππ ={ π ππ‘βπππ€ππ π π₯π,π‘
urn a
π π’π,π‘
lP
re-
(π‘)
(5)
where πππππ is a decision variable. Selection Operation: The selection operation based on the greedy algorithm chooses the survived vector from trail and target vectors, and can be represented as π’π,π‘ ππ π(π’π,π‘ ) β€ π(π₯π,π‘ ) π₯π,π‘+1 = {π₯ (6) π,π‘ ππ‘βπππ€ππ π
23 24
The selection operation in the classical DE compares each target vector π₯π,π‘ against corresponding trial vector π’π,π‘ based on the objective function, and keeping the evolution
25
continuing.
26
2.2 Constraint strategies
27 28 29
To solve COPs using EAs, the constraint-handling strategies are very important. A constrained optimization problem is usually represented as a nonlinear programming of the following form for the minimization problem [54] as
Jo
20 21 22
5 / 31
Journal Pre-proof Minimize:
π(π₯), π₯ = (π₯1 , π₯2 , β¦ , π₯π ) πππ π₯ β π ππ (π₯) β€ 0, π = 1, β¦ , π Subject to: βπ (π₯) = 0, π = π + 1, β¦ , π
(7)
where π is the design space, ππ (π₯) is the inequality constraint, and βπ (π₯) is the equality
2 3 4 5 6 7
constraint. The number of inequality constraints is q, (m-q) is the number of the equality constraints. Generally, if the global optimum satisfies the constraints ππ (π₯) = 0, the inequality constraints are called active constraints. Similarly, Active constraints contain all equality constraints. For convenience, equality constraints can be transformed into the form of inequality constraints. The inequality constraints and equality constraints can be integrated into a combined form as πΊπ (π₯) = {
pro of
1
πππ₯{ππ (π₯), 0} π = 1, β¦ , π πππ₯{|βπ (π₯)| β πΏ, 0} π = π + 1, β¦ , π
(8)
8 9
where πΏ is the tolerance parameter which is usually set as 1e-4. The overall constraint violation for a solution can be expressed as
10 11 12 13 14 15 16 17 18 19 20 21 22 23
βπ π=1 πΊπ (π₯) (9) π where π£(π₯) is zero for the feasible solution and positive when at least one constraint is violated [55], and the overall constraint violation is an important parameter in order to guide the evolutionary process towards feasible areas. In this study, three popular constraint handling strategies should be introduced as follows Superiority of Feasible Solution (SF): Two individual π₯1 and π₯2 are compared in SF. π₯1 is assumed to be superior to π₯2 if the following conditions are satisfied: 1. π₯1 is a feasible solution but π₯2 is an unfeasible solution. 2. Both π₯1 and π₯2 are feasible solutions and the objective function value of π₯1 is smaller than π₯2 . 3. Both π₯1 and π₯2 are infeasible solutions, and the overall constraint violation π£(π₯) of π₯1 is smaller than π₯2 . Ξ΅-Constraint (EC): The method of constraint violation and Ξ΅ level comparisons was proposed in [21]. The essential of Ξ΅-Constraint handling strategy is the tradeoff between the constraint violation and the objective function. The parameter Ξ΅ was employed to adjust the balance,
lP
urn a
26 27 28
and the constraint violation π(π₯) was defined as a form based on the maximum of all constraint violations. π(π₯) = πππ₯{πππ₯{0, π1 (π₯), β¦ , ππ (π₯)}, πππ₯|βπ+1 (π₯), β¦ , βπ (π₯)|} (10) In constrained optimization problems, an individual is assumed as infeasible solution and its precedence is low if the constraint violation is greater than 0, This precedence is controlled by the parameter Ξ΅ . When the target and trail vectors are compared based on Ξ΅ -Constraint
29 30
method, a set of objective functions and constraint violations (π(π₯), π(π₯)) will be evaluated according to the Ξ΅ -level precedence. Similar to previous discussion, π₯π is considered as
31
superior to π₯π when the following criteria are satisfied
Jo
24 25
re-
π£(π₯) =
(π(π₯π ), π(π₯π )) β€π (π(π₯π ), π(π₯π ))
π(π₯π ) β€ π(π₯π ), ππ π(π₯π ), π(π₯π ) β€ π {π(π₯π ) β€ π(π₯π ), ππ π(π₯π ) = π(π₯π ) π(π₯π ) < π(π₯π ), ππ‘βπππ€ππ π 32
(11)
The β€π between the individual π₯π and π₯π means π₯π is superior to π₯π according the Ξ΅6 / 31
Journal Pre-proof level precedence. When Ξ΅ is set as β, the value of objective function is the only criterion for determining superiority of two individuals. The property of the Ξ΅-Constraint method is the flexibility. The parameter Ξ΅ can be modified according to the characteristics of optimization
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
problem in order to obtain the suitable balance between lower values of objective function value and constraint violation. However, the suitable balance point is difficult to be determined for different COPs. Self-Adaptive Penalty (SP): Penalty function method keeps the efficiency even after achieving enough feasible individuals with simple structure [56]. Static penalty function method is classical in COPs due to its simplicity. Many methods based on penalty functions provide competitive results [57]. However, a major drawback of static penalty function method is difficult to determine the suitable penalty coefficients. Generally, the penalty coefficients are commonly case-dependent. Therefore, adaptive penalty functions [58] have been proposed in order to overcome this bottleneck. The penalty coefficients can be determined adaptively according to the previous iterations. In constrained optimizations, it is important to extract useful information from the infeasible individual whose worth is low but can guide the evolutionary path to the feasible region. The main difference is their precedence of distinct types of infeasible individuals [59]. An adaptive penalty function method based on distance value and two penalties was proposed [59]. The amount of added penalties are affected by the feasible ratio of population individuals. A greater penalty will be added to infeasible individuals if the feasible ratio is low. In contrast, if there are enough feasible individuals, then the infeasible individuals with high-fitness values will obtain relatively small penalties. It means that the algorithm can change the status between seeking the optimal solution and finding feasible solutions without presetting parameters. The details of this method are represented as follows πΉ(π₯) = π(π₯) + π(π₯) (12) π£(π₯), ππ πππππ = 0 π(π₯) = { β²β² 2 (13) 2 βπ (π₯) + π£(π₯) , ππ‘βπππ€ππ π
lP
re-
pro of
1 2 3
urn a
π(π₯) = (1 β πππππ )π(π₯) + πππππ π(π₯)
πππππ =
ππ’ππππ ππ ππππ ππππ πππππ£πππ’ππ ππππ’πππ‘πππ π ππ§π
(15)
0, ππ πππππ = 0 π£(π₯), ππ‘βπππ€ππ π
(16)
M(x) = {
ππ π₯ ππ π ππππ ππππ π πππ’π‘πππ 0, π(π₯) = { β²β² (π₯), π ππ π₯ ππ ππ ππππππ ππππ π πππ’π‘πππ π β²β² (π₯) =
Jo
(14)
(π(π₯) β ππππ ) (ππππ₯ β ππππ )
(17) (18)
25 26
where πππππ is the feasible ratio, π£(π₯) is the overall constraint violation, π β²β² (π₯) is the normalized fitness value οΌ ππππ₯ πππ ππππ are the maximum and minimum values of the
27
objective function in the present generation, π(π₯) is the penalty value.
28
3 Surrogate assisted adaptive DE algorithm
29
It is well known that the performance of the DE is a case-dependent with different parameter 7 / 31
Journal Pre-proof ranges when handling with different problems. Except for parameters like Cr, F, Np, mutation strategies employed in evolution are also sensitive to the performance during optimization. Actually, even for a simple problem, which has been determined its characteristics; the suitable ranges of parameters like Cr, F and Np should be updated in different stages of the DE. Therefore, the requirements for mutation strategies are also changed especially in constrained optimizations. To overcome these difficulties, several strategies have been suggested. For example, parameter adaptation method for Cr, F based on success-history has been employed in many algorithms like SaDE [11], JADE [12], SHADE [13] and JADE_sort [15]; population size controlling strategy for Np has been applied to L-SHADE [20], sTDE-dR [67]. Although these adaptive methods have successfully improved the performances, the efficiency of the DE still has room for improvement. Therefore, several adaptive strategies are suggested in this study and the framework of the suggested algorithm is presented in Fig.1. Initialize control parameters
Generate the initial four equal size subpopulations randomly and evaluate the individuals by real functions
re-
Mutation (DE/current-to-best/1) is assigned for each subpopulation
pro of
1 2 3 4 5 6 7 8 9 10 11 12 13
Initialize the subpopulations and assign the number of size of each subpopulation
Local surrogate assisted evaluation strategy
Search the neighbor populations around new one
EFs<=EFsmax
lP
Assign Cr, F to individuals in each subpopulation to create the trail vectors according subpopulation archives
NO
Terminate
Update the size of subpopulation based on pfeas and save failed individuals, update the success-memory of each subpopulation
EGO (1) KG (2) SVR (3) RBF
(1) KG-HDMR (2) RBF-HDMR (3) SVR-HDMR
Projection sampling method
Generate and update surrogate by different techniques
No
Select the best surrogate model
urn a
Parameter adaptation strategy based on success-history
Calculate size of the next generation
14 15
0.5<=R2<=1
Yes
Fig.1 Framework of the ASA-DE
3.1 Notes of Adaptive population clustering strategy
17 18 19 20 21 22
In order to control the state switch efficiently, adaptive population clustering strategy is employed for the suggested algorithm. This strategy has been widely used by several scholars. In Zhouβs work [68], the population should be classed into better and worse groups according to fitness values. In the L-SHADE44 [66], population can be divided into four subpopulations, each subpopulation employs an independent mutation strategy. πΊ In this study, the population π(πΊ) = [π₯1πΊ , π₯2πΊ , β¦ , π₯ππ ] should be clustered into k
23 24 25
subpopulations [π1πΊ , π2πΊ , β¦ , πππΊ ] , πππΊ is the kth subpopulation of the Gth generation. Each subpopulation should be assigned by DE/current-to-best/1 strategy and has independent adaptive parameter archive. The size of each subpopulation should be the same in the first
Jo
16
8 / 31
Journal Pre-proof generation and the fitness value of each population should be evaluated by the real function or simulation. Commonly, the size of each subpopulation will be updated according to the π success rate of evolution. If the trail vector generated from individual π₯π,πΊ is better than its
4 5
parents in term of the fitness, the ith individual in sth subpopulation should be considered as a successful one. The size of each subpopulation is determined as ππ’π + ππ’0 ππ π = π ππ (19) βπ=1(ππ’π + ππ’0 ) πΊ
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
where ππ’π is the present number of the successful evolutions in sth subpopulation. ππ’0 is a small positive constant in order to avoid zero denominator when the algorithm is close to convergence. πππΊ is the entire population size in the Gth generation. However, according to the test, this size control strategy still has room for modification, a more suitable strategy is suggested according to exploitation and exploration and should be discussed in Section 3.2. Moreover, it should be noted, the new population except for the first iteration should be generated and evaluated by the surrogate model according to the assigned criterion. Some of new populations cannot be directly generated by the HDMR-based surrogate techniques directly due to the characteristic of the HDMR. For the HDMR-based modeling technique, the new population should be generated on the axil of hypercube. Therefore, when a new population is generated inside of the hypercube, the project sampling strategy [65] should be implemented to generate the new population on the each axil. Thus, the number of populations should be significantly improved. However, these new populations generated by the project strategy should be evaluated by the surrogate model, the computational cost should be almost same as the others. Another important issue should be considered is the number of clusters. Theoretically, the number of clusters should be determined by the complexity of each problem. However, consider the diversity of problems, the number of clusters is difficult to be obtained. Therefore, several tests have been carried out with 4~10 clusters. In this study, the test results suggested that 4-cluster is the best choices.
26
3.2 Adaptive population size based on pfeas
27 28 29 30 31 32 33 34
As mentioned previous section, the population size is an important parameter to improve the performance of algorithm. In this study, after cluttering, the size of each subpopulation should updated according to the suggested adaptive population size strategy based on pfeas. In this strategy, the evolutionary procedure can be classified into three states, exploring biased, balance search and exploiting biased states. Exploring biased state: The optimization procedure enters the exploring state before the FEs exceeds the 0.5 πΉπΈπ πππ₯ . The control strategy of population size is linear reduction. The population size in this stage is calculated according to the formula as follows: πππππ₯ β πππππ (20) Np(G + 1) = Round [πππππ₯ β ( ) πΉπΈπ ] πΉπΈπ πππ₯
35 36 37 38
where πππππ is the preset minimum population size, πππππ₯ is the initial population size. Before the FEs exceeds the πΉπΈπ π π€ππ‘πβ1 (set as 0.5πΉπΈπ πππ₯ , the choice of values should be discussed in numerical test section), the population is large enough to keep exploration and population diversity. The minimum of the population size is 0.5πππππ₯ . Experiments have
Jo
urn a
lP
re-
pro of
1 2 3
9 / 31
Journal Pre-proof proved that it is a suitable value choice for the algorithm when it is difficult to determine whether the constraints are loose. The balance search state: The optimization procedure will enter the balance search state if no feasible solution is found when FEs reaches πΉπΈπ π π€ππ‘πβ2 . In this state, the population size will keep the same size (0.5πππππ₯ ) at the end of the previous state to ensure a high-level diversity. Meanwhile, the mutation strategies will switch to highly exploiting biased state to enhance the efficiency. Once the first feasible solution is successfully found, the population size will be adaptively reduced based on the feasible rate (πππππ ) which can be formulated as (πππππ₯ β πππππ ) NP(G + 1) = Round [πππππ₯ β (πΉπΈπ β πΉπΈπ πππ₯)πππππ ] (21) (πΉπΈπ πππ₯ β πΉπΈπ πππ₯)
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
where πππππ₯ is the population size in the generation when the algorithm enter to the balance search state, πΉπΈπ πππ₯ is the number of function evaluations when the algorithm switches to the balance search state. If pfeas is 0, the population size will be stopped to be reduced to keep the diversity of populations to keep enough diversity. When pfeas exceeds pfeasNp (set as 0.5, the choice of value should be discussed in section 4), the population size is calculated as Eq.(21). If pfeas has reached pfeasNp before the balance search state begins, the method based on the linear reduction will be employed continuously and the balance search state is not employed during this evolutionary process. Exploiting biased state: The algorithm will switch to the exploiting biased state when FEs reaches πΉπΈπ π π€ππ‘πβ3 . The population linear reduction method and an exploiting protection mechanism will be employed at this stage. The population will initialize as πππππ₯ /6 if the population size is larger than πππππ₯ /6 (pfeas remains low or no feasible solution can be found in the first two stages). The mutation strategies will switch to balance mode to enhance the effect. For clarity, the details of this strategy is presented as follows.
lP
re-
pro of
1 2 3 4 5 6 7 8
urn a
Adaptive population size based on pfeas Start 1. If πΈπΉπ < 0.5 Γ πΉπΈπ πππ₯ 2. The strategy chooses the exploring state and the population size is calculated as Eq. 20. 3. Else 4. ..If πππππ > 0.5 && πΈπΉπ < 0.9 Γ πΉπΈπ πππ₯ 5. Keep the exploring state and the population size is calculated by Eq. 20. 6. If πππππ = 0 && πΈπΉπ > 0.9 Γ πΉπΈπ πππ₯ 7. Switch to the exploiting biased state and the population size is initialized to πππππ₯ /6 and is calculated as Eq. 20.
Jo
8. Else 9. Switch to the balance search state and the population size is calculated by Eq. 21. 10. End if 11.End if
24
3.3 Local surrogate assisted evaluation strategy
25 26
The surrogate model is another important issue in this framework. In this study, surrogate models should be constructed by several kinds of modeling techniques, KG [60], SVR [61], RBF 10 / 31
Journal Pre-proof [62], high dimensional modal representation (HDMR) [63] and other surrogate assisted HDMR methods, such as KG-HDMR, RBF-HDMR and SVR-HDMR in sequential optimization as shown in Fig.1. The details of these surrogate modeling techniques can be found in [64]. In this procedure, these surrogate modeling techniques are implemented to improve the efficiency of suggested strategy instead of real function evaluations. In this procedure, R 2 should be the criterion to verify the accuracy of the constructed surrogate models based on leave-one-out cross validation (LOOCV). To save the computational cost, the samples from other subpopulation should be the test population for validation. Compared with the LOOCV based on the training samples, the result of test obviously more objective. Moreover, it also means that the additional test samples for the subpopulation can be avoided. After test, the best surrogate model should be used to generate new populations for the next generation. Compared with direct evaluation strategy, the accuracy of a new population derived from the surrogate model should be decreased. Actually, it is difficult for the popular surrogate modeling techniques to construct highly accurate model for high dimensional problems (over 50D). Therefore, in the suggested algorithm, the populations used for the following strategies should be used to generate the local surrogate model instead of global one. In the suggested
17
π·ππ πππ π ππππβ strategy, the range for each subdomain should be limited for π. For example, if
18
the dimension of assigned case is 30, the surrogate model should be constructed based on the
19
neighbor samples limited in
20
local surrogate model is commonly more accurate than the global one.
21
3.4 Parameter adaptation strategy based on success-history
22 23 24
The choice of parameter F and Cr is also important to the performance of suggested algorithm. In the ASA-DE, a parameter adaptation strategy based on success-history is also employed to control F and Cr [12-13, 74]. Each subpopulation has an independent archive ππ (π =
25 26 27 28 29
1,2, β¦ , π) to store suitable mean values of Cr and F (ππΆπ πππ ππΉ ). The size of archive ππ ππ§π is a preset constant. The values of F and Cr used by successful individuals in the current generation should be stored in temporary archives πππΆπ and πππΉ to calculate the ππΆπ πππ ππΉ . The values of ππΆπ,π and ππΉ,π (π = 1,2, β¦ , ππ ππ§π ) are set to 0.5 at the beginning. In each generation, once archive ππ is full, a random element in the archive should be replaced. The
30 31
details of mechanism are shown as follows ππΆπ ,πΉ = ππππππΏ (πππΆπ ,πΉ ) ππ πππΆπ,πΉ β β (22) where π(π₯π ) is the objective function of the individual which employs the mth F stored in πππΉ . π(π’π ) is the objective of the trail vector of the individual. When the population is completely partitioned, each individual in subpopulations selects F and Cr randomly from the corresponding archive ππ . The final values of Cr and F are calculated
36 37 38 39 40
re-
urn a
lP
(π΅ππ’πππππππ β π΅ππ’πππΏππ€ππ )β 30 . Theoretically, the accuracy of
Jo
32 33 34 35
pro of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
as follows
πΆπ = πππππ(ππΆπ , 0.1) (23) πΉ = πππππ(ππΉ , 0.1) (24) The final values of Cr and F are generated based on normal distributions. The mechanism is employed in our algorithm directly, the parameter settings can be referred to [13], [74]. 11 / 31
Journal Pre-proof 1
4 Numerical test The suggested ASA-DE is tested on 28 widely used benchmark functions from the IEEE CEC2017 benchmarks on constrained real-parameter optimization [69] on 10D, 30D, 50D and 100D. The algorithm runs 50 times for each test function with a number of function evaluations equals to 20,000π· according to the guidelines of CEC2017 [69]. Because the surrogate modeling techniques are used to evaluate the fitness in the ASA-DE, the number of populations evaluated by the surrogate should be recorded. In this test, the percentage of number of surrogate evaluated populations should also be listed. The criteria employed to evaluate the performance of the ASA-DE can be found in [69] and includes feasibility rate of all runs (FR), mean violation amounts (π£ππ Μ Μ Μ Μ ) and mean of objective function value (mean). Moreover, the priority level of the evaluation criteria can be according to following order [71], FR. Μ Μ Μ Μ π£ππ and mean. In this study, three evaluation criteria are used for performance comparison. The first criterion is referenced in the guidelines of CEC2017 [69] and employed in results of all comparison algorithms. Pairwise comparisons method the sign test [70] as the second evaluation criterion is also employed to compare the ASA-DE with other state-of-the-art algorithms including the top four algorithms for constrained optimization in CEC2017 are compared with the ASA-DE. The detailed description of the evaluation criterion is showed above. The sign test [71] is used to determine the significance of the results (CAL-SHADE, L-SHADE44, SaDE, SajDE and DEbin versus ASA-DE) at the 0.05 significance. The symbols "+", " β ", "=" are employed to indicate the ASA-DE performs significantly better (+ ), no significantly difference (= ), significantly worse (β) than the algorithms in comparison. Considering the surrogate modeling is imported for evaluating the real response of each test function, the percentage of number of samples evaluated by the surrogate model is also presented. More cheap samples are used, more efficiency ASA-DE is.
26
4.1 Parameters setting
27 28 29 30 31 32
The parameter settings are shown as follows 1. The initial population size πππππ₯ is set to 24D in 10-D problems, 19D in 30-D, 14D in 50-D and 9D in 100D problems. 2. The minimum population size πππππ is set to 6. 3. The number of subpopulation is set to 4. 4. The size of archive ππ (π = 1,2, β¦ , π) is set to 6.
Jo
urn a
lP
re-
pro of
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
33 34 35 36 37
5.
πΉπΈπ π π€ππ‘πβ1 , πΉπΈπ π π€ππ‘πβ2 and πΉπΈπ π π€ππ‘πβ3 are respectively set 0. 08πΉπΈπ πππ₯ , 0.5πΉπΈπ πππ₯ , 0.9πΉπΈπ πππ₯ .
The selection of these parameters is discussed below: The number of subpopulations, the initial population size and ππ are set with reference to the
38 39 40
L-SHADE44. πππππ is slightly lager than the number of subpopulation. Experiments show that πππππ has little effect on the performance of the algorithm and is set as 6 finally. πΉπΈπ π π€ππ‘πβ1 and πΉπΈπ π π€ππ‘πβ3 are more like a protective threshold. πΉπΈπ π π€ππ‘πβ1 should be 12 / 31
Journal Pre-proof 1 2 3 4
enough close to the beginning of the evolutionary process. In contrast, πΉπΈπ π π€ππ‘πβ3 should close to the end of the evolutionary process. Therefore, πΉπΈπ π π€ππ‘πβ1 and πΉπΈπ π π€ππ‘πβ3 are set as 0.08πΉπΈπ πππ₯ and 0.9πΉπΈπ πππ₯ , respectively. Table 1 Configuration of parameters and the initial values of the surrogate modeling Basis functions
Expression
Values
Initial correlation coefficient π
π(β1/m)
Turning parameter ππ
3
RBF
Polyharmonic function order
3
LSSVR
Turning parameter ππΏ
2
Initial correlation coefficient ππΎπ»
0.1
Lower boundary on the π
0.1
Upper boundary on the π
10
Turning parameter ππ π» for thin plate spline function
1
pro of
Kriging
Kriging-HDMR
SVR-HDMR
Others for HDMR
Tolerance to judge the convergence of ππ (π₯π )
10e-4
Tolerance to judge the convergence of πππ (π₯π , π₯π )
10e-2
Tolerance to judge the interaction of π₯π and π₯π
10e-6
Table 2 Comparison results of variants with different πΉπΈπ π π€ππ‘πβ2 with selected functions πΉπΈπ π π€ππ‘πβ2 F3
0.1 β πΉπΈπ πππ₯
0.2 β πΉπΈπ πππ₯
0.5 β πΉπΈπ πππ₯
0.7 β πΉπΈπ πππ₯
0.9 β πΉπΈπ πππ₯
376300
476300
319459
561234
656123
1
1
0.85
0.87
0
0
9.7363e-06
8.1236e-05
4681.12
4453.81
4262.0712
4132.0782
0.26
0.32
0.3023
0.2931
0.01121
0.01306
0.01278
0.01278
-0.5622
-0.5622
-0.8736
-0.8651
-0.8551
0.6210
0.6210
1
1
1
8.2621e-10
9.4621e-11
0
0
0
478.7812
401.3732
212.5621
388.2108
461.9812
1
1
1
0.6800
0.6980
0
0
0
0.2837
0.3865
1 8.7876e-06 4781.12
F6
0.26
F11
F22
urn a
0.01121
For more clarity, the suggested algorithm with 5 different values of πΉπΈπ π π€ππ‘πβ2 (0.1 β πΉπΈπ πππ₯ , 0.3 β πΉπΈπ πππ₯ , 0.5 β πΉπΈπ πππ₯ 0.7 β πΉπΈπ πππ₯ and 0.9 β πΉπΈπ πππ₯ ) on 30D problems according to the guidelines of CEC2017. The parameter πΉπΈπ π π€ππ‘πβ2 is used in adaptively population size mechanism which is designed for solving medium strict level problems. Therefore, f3, f6, f11, f22 are employed to analyze πΉπΈπ π π€ππ‘πβ2 . Three variant algorithms are as follows. ASA-DE-FEs1: πΉπΈπ π π€ππ‘πβ2 is set as 0.1 β πΉπΈπ πππ₯ ASA-DE-FEs2: πΉπΈπ π π€ππ‘πβ2 is set as 0.3 β πΉπΈπ πππ₯ ASA-DE -FEs3: πΉπΈπ π π€ππ‘πβ2 is set as 0.5 β πΉπΈπ πππ₯ ASA-DE -FEs4: πΉπΈπ π π€ππ‘πβ2 is set as 0.7 β πΉπΈπ πππ₯
Jo
7 8 9 10 11 12 13 14 15 16
2 10e-6
lP
5 6
Turning parameter ππΏπ»
Tolerance to judge the nonlinearity of ππ (π₯π )
re-
RBF-HDMR
13 / 31
Journal Pre-proof ASA-DE -FEs5: πΉπΈπ π π€ππ‘πβ2 is set as 0.9 β πΉπΈπ πππ₯ According to Table 2, it can be found that when πΉπΈπ π π€ππ‘πβ2 is set as 0.5 β πΉπΈπ πππ₯ , the performance of the algorithm is the best one. Therefore, we set πΉπΈπ π π€ππ‘πβ2 as 0.5 β πΉπΈπ πππ₯ . Moreover, because several surrogate models are integrated in the suggested method, the initial values of parameters used in these methods are listed in Table.1. It means that Kriging, RBF, LSSVR, Kriging-HDMR, RBF-HDMR and SVR-HDMR should be employed to construct the surrogate model for each subpopulation and the best one of them should be selected for the next iteration.
9
4.2 Comparisons with other popular algorithms
10 11 12 13
pro of
1 2 3 4 5 6 7 8
In order to evaluate the performance of the suggested ASA-DE algorithm, the following popular algorithms including the top 4 algorithms for the constrained optimization in CEC2017 are compared with the ASA-DE.
Name
Full Name
Year
CAL-SHADE
re-
Table 3 Other DE algorithms for comparison
Adaptive constraint handling and success history differential evolution[72]
2017
L-SHADE44+IDE
SajDE SaDE DEbin
[74].
A unified differential evolution algorithm for constrained optimization problems (UDE) [18].
An Improved Self-adaptive Differential Evolution Algorithm in Single Objective Constrained Real-Parameter Optimization [55]. Self-adaptive differential evolution algorithm for constrained realparameter optimization [75]. The standard differential evolution algorithm (DEbin) [3].
2017 2017 2017 2010 2006 1997
For the comparison result with 10D, as listed in Tables 4 and 5, the ASA-DE is better than or at least as good as the other algorithms. The ASA-DE also performs better than the ranked first algorithm of the CEC2017 constrained optimization (L-SHADE-44) and performs better than the L-SHADE-44 on 15 functions, while is worse by the L-SHADE-44 on 7 functions. The ASADE is also efficient at solving f6, while the L-SHADE-44 performs worse on this one. It should be noted that most of the mean percentages of the number of surrogate evaluated populations are closed to 35% and corresponding variances are near 5%. It suggests the ASA-DE might save nearly 30% computational cost compared with the directly evaluated algorithm. For f17, f18, f19, f26, f27, f28, it is difficult for the ASA-DE to construct high accurate surrogate modeling due to its complexity even the original function has been clustered. For these problems, the variances are also smaller. Therefore, for 10D problems, the ASA-DE generally obtains the high accurate result efficiently. Moreover, because the adaptive parameter strategy is used, Μ Μ Μ Μ π£ππ also outperforms than most of other algorithms.
Jo
14 15 16 17 18 19 20 21 22 23 24 25 26 27
L-SHADE with competing strategies applied to constrained optimization
lP
UDE
SHADE44 and IDE (L) [73].
urn a
L-SHADE44
A simple framework for constrained problems with application of L-
14 / 31
Journal Pre-proof For 30D problems, as shown in Tables 6 and 7, the ASA-DE also performs better than or at least equal to the other algorithms and keep the ability to solve f6. The ASA-DE achieves competitive results that are equal or better than the L-SHADE-44 in most benchmark functions. Except f17, f18, f19, f26, f27, f28, the ASA-DE optimizes all other functions successfully. The other algorithms are also inefficient for it. Although the UDE is also effective in optimizing f6, it fails to optimize f11. As a result, the ASA-DE has an excellent constraint control capability and can follow the suitable balance point between exploration and exploitation during evolutionary process through state switching and population size adaptive mechanisms. Compared with the test result for 10D problems, the percentages of surrogate evaluated population are decreased, the mean values are over 30%. With the increasing of the dimensionality, the performance of the ASA-DE is decreased due to the accuracy of surrogate modeling techniques. However, the computational cost for each problem can be saved over 20%-30%, it demonstrates that the ASA-DE is still suitable for 30D problems. It demonstrates that the local modeling strategy becomes more important for higher dimensional problems. For 50D and 100D problems, as presented in Tables 7-11, the ASA-DE outperforms the other algorithms in most benchmark functions. The ASA-DE obtains competitive results that are equal or better than L-SHADE-44 in 28 benchmark functions. Compared with 10D and 30D problems, although the performance of the ASA-DE still prevails, the influence from the surrogate model seems to be weak. According to Tables 8 and 9, the number of samples evaluated the surrogate models decreases. For 100D problems, as presented in Tables 10 and 11, due to the limitation of existed test data, four novel algorithms are compared with the ASADE. It can be found that the result is similar to 50D, even better. Successful optimization rates for benchmark functions are presented in Table.12, it demonstrates that the suggested algorithm is better than other algorithms in this test. Moreover, it should be noted that the number of samples evaluated by surrogates should be decreased in 50D and 100D. However, compared with 50D problems, the number of samples evaluated by surrogates in 100D is not decreased significantly. It suggests that the number of clusters for different dimensions is reasonable. In our opinion, the local modeling technique plays a more important role for highly dimensional problems.
31
Table 4. Experimental results in mean, FR and Μ Μ Μ Μ Μ ππΌπ on functions 1-28 for 10D problems
F1
F2
F3
F4
ASA-DE
CALSHADE
LSHADE44
SaDE
SajDE
DEbin
LS44+IDE
UDE
0.0000E+00
6.4300E-30
0.0000E+00
0.0000E+00
6.1800E-30
2.7200E-29
0.0000E+00
5.0300E-15
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
9.1000E-17
0.0000E+00
0.0000E+00
2.6900E-29
1.2100E-28
0.0000E+00
6.4212E-15
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1233E+03
1.1892E+05
8.2093E+04
1.0499E+05
1.2000E-29
1.0892E+05
3.2601E+05
7.7386E+01
1.0000E+00
4.8000E-01
9.2000E-01
8.0000E-02
1.0000E+00
2.8000E-01
1.0000E+00
9.6000E-01
0.0000E+00
6.2600E-05
9.2600E-06
1.9000E-04
0.0000E+00
1.2600E-04
0.0000E+00
4.3000E-06
1.3291E+01
1.3921E+01
1.3629E+01
1.3273E+01
1.5120E+01
1.4793E+01
1.4420E+01
2.5117E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
Jo
Func
urn a
lP
re-
pro of
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
15 / 31
Journal Pre-proof
F5
F6
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
5.0100E-02
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.1890E-01
0.0000E+00
0.0000E+00
3.1921E-01
1.2341E+00
0.0000E+00
1.6542E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.2741E+01
8.28299E+02
7.2049E+02
7.9512E+02
0.0000E+00
4.2825E+02
8.0836E+02
8.7100E+01
1.0000E+00
0.0000E+00
4.0000E-02
0.0000E+00
1.0000E+00
1.6000E-01
0.0000E+00
4.4000E-01
0.0000E+00
1.4620E-01
3.1950E-02
1.8270E-01
0.0000E+00
1.4180E-01
3.7660E-02
2.0400E-02
1.7993E+01
-6.5994E+00
2.1918E+01
-5.2247E+02
3.0619E+01
-3.4003E+01
-6.4624E+00
1.0000E+00
6.8000E-01
7.6000E-01
0.0000E+00
0.0000E+00
8.0000E-02
8.0000E-01
7.2000E-01
0.0000E+00
1.0000E-04
3.2500E-05
1.1440E-01
1.0216E+03
1.2150E-03
3.1900E-05
7.0610E-02
-1.350E-03
-1.350E-03
-1.3500E-03
-1.350E-03
-9.0457E+01
-2.9000E-04
-1.3500E-03
-1.3400E-03
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.7650E+05
0.0000E+00
0.0000E+00
0.0000E+00
-4.980E-03
1.0032E+00
-4.9800E-03
-4.980E-03
-6.0920E-01
9.9100E-02
-4.9800E-03
-4.9800E-03
1.0000E+00
5.2000E-01
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
4.0900E-02
0.0000E+00
0.0000E+00
2.5330E+02
0.0000E+00
0.0000E+00
0.0000E+00
-5.100E-04
-5.100E-04
-5.1000E-04
-5.100E-04
-5.9754E+01
-4.5000E-04
-5.1000E-04
-5.1000E-04
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1100E+06
0.0000E+00
0.0000E+00
0.0000E+00
-1.5250E-01
-1.5950E-01
-1.6340E-01
8.5700E-02
-9.8610E+02
-6.4520E-02
-1.6880E-01
-5.9955E+00
1.0000E+00
8.8000E-01
1.0000E+00
6.8000E-01
0.0000E+00
8.8000E-01
1.0000E+00
0.0000E+00
0.0000E+00
1.8000E-04
0.0000E+00
1.8500E-07
4.2500E+19
7.7200E-15
0.0000E+00
3.0000E-03
3.9880E+00
4.0037E+00
3.9932E+00
3.9934E+00
1.9900E-01
3.9879E+00
3.9879E+00
3.9879E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.9005E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.8670E-01
1.9700E-31
0.0000E+00
4.7840E-01
2.5780E-02
0.0000E+00
1.1124E+01
F11
F12
F13
F14
F15
F16
F17
F18
re-
F10
lP
F9
urn a
F8
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.8570E+00
3.2745E+00
2.8391E+00
2.8478E+00
1.4200E-16
3.1894E+00
3.0000E+00
2.7362E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.5000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.2666E+01
1.4925E+01
1.4294E+01
1.2535E+01
2.2000E-15
1.4797E+01
1.1278E+01
6.7549E+00
1.0000E+00
0.0000E+00
2.8000E-01
9.2000E-01
0.0000E+00
1.0000E+00
1.0000E+00
9.2000E-01
0.0000E+00
4.7000E-03
1.8900E-04
4.3000E-06
5.0000E-01
0.0000E+00
0.0000E+00
3.7000E-04
Jo
F7
2.2418E+01
pro of
-
3.7448E+01
4.9009E+01
4.0904E+01
5.1271E+01
0.0000E+00
4.7752E+01
4.0401E+01
6.2830E+00
1.0000E+00
9.6000E-01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
9.6000E-01
0.0000E+00
2.0900E-06
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1100E-05
1.0521E+00
9.2130E-01
9.1933E-01
9.1520E-01
1.2800E-03
1.0273E+00
8.8240E-01
1.0460E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.6600E+00
5.5000E+00
5.2600E+00
4.5000E+00
2.2939E+01
4.9400E+00
5.2200E+00
5.4200E+00
3.8204E+02
7.9760E+02
3.0489E+03
7.7489E+02
8.0000E-02
6.2531E+02
3.1663E+03
2.3591E+03
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
16 / 31
Journal Pre-proof
F22
F23
F24
F25
F26
F28 +ο―=ο―β +ο―=ο―β
1.1437E+02
8.2300E+06
1.5100E+07
0.0000E+00
2.3900E-06
1.5800E-06
0.0000E+00
-9.2521E+00
1.1200E-06
0.0000E+00
2.7200E-03
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
6.6336E+03
6.6336E+03
6.6336E+03
6.6336E+03
6.6464E+03
6.6336E+03
6.6336E+03
6.6336E+03
4.5110E-01
2.4900E-01
1.8010E-01
4.5800E-01
6.0890E-01
4.2790E-01
4.1560E-01
1.6464E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.9879E+00
3.9901E+00
3.9884E+00
3.9880E+00
1.0777E+01
3.9879E+00
3.9879E+00
6.2436E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.1860E-01
1.1011E+02
3.1890E-01
4.7840E-01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.7089E+00
3.3359E+00
2.5320E+00
3.3042E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.3289E+01
1.0022E+01
8.1367E+00
2.0452E+01
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1196E+02
4.6244E+01
4.6307E+01
3.7008E+01
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.2066E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.9313E+01
8.3719E+00
1.5950E-01
1.2504E+01
8.4000E-01
1.0000E+00
1.0000E+00
1.0000E+00
2.3755E+02
0.0000E+00
0.0000E+00
0.0000E+00
2.0496E+01
3.0997E+00
3.0212E+00
2.7456E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
4.0874E+04
0.0000E+00
0.0000E+00
0.0000E+00
1.4100E-16
1.1781E+01
8.7650E+00
6.0006E+00
1.0000E+00
1.0000E+00
9.6000E-01
5.0000E-01
0.0000E+00
0.0000E+00
1.5200E-04
6.8361E+01
7.1500E-15
4.2327E+01
3.7699E+01
6.3459E+00
1.0000E+00
9.2000E-01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
8.1100E-06
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.0547E+00
9.5090E-01
1.0059E+00
1.1173E+00
2.0580E-02
1.0636E+00
9.3740E-01
1.0217E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.7000E+00
5.4600E+00
5.1000E+00
4.5000E+00
2.9816E+01
4.9000E+00
5.0526E+00
5.4611E+00
4.3869E+02
3.1764E+03
2.6079E+03
7.3171E+02
3.1806E+01
5.8606E+03
7.9442E+03
6.7179E+03
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.1842E+04
8.6413E+04
3.5710E+05
1.7742E+02
1.8585E+03
2.6471E+05
4.0500E+07
2.2300E+08
2.1103E+01
3.5925E+01
2.5750E+01
1.0419E+01
-8.0290E-02
2.3984E+01
1.0763E+01
9.7567E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
6.6480E+03
6.6604E+03
6.6522E+03
6.6396E+03
6.6475E+03
6.6496E+03
6.6409E+03
6.6410E+03
Criterion I Criterion II
23/2/3 18/8/2
15/6/7 7/17/4
13/8/7 11/14/3
21/0/7 19/3/6
23/1/4 15/12/1
11/9/8 NA
22/2/4 NA
Table 5. The mean percentage of number of surrogate evaluated populations for each 10D problems
Jo
1 2 3
7.4797E+00
urn a
F27
5.7405E+01
pro of
F21
1.7100E+06
re-
F20
1.1585E+02
lP
F19
1.3543E+04
Function
F1
F2
F3
F4
F5
F6
F7
Mean
38.51%
39.22%
28.21%
18.91%
35.12%
38.72%
37.32%
Variance
5.29%
4.39%
6.21%
7.53%
4.56%
5.12%
4.34%
Function
F8
F9
F10
F11
F12
F13
F14
Mean
34.51%
40.21%
32.21%
34.78%
36.82%
39.12%
33.39%
Variance
5.12%
8.39%
3.76%
4.78%
5.11%
6.18%
5.21%
Function
F15
F16
F17
F18
F19
F20
F21
Mean
33.51%
34.21%
21.2%
18.67%
15.89%
32.121%
37.21% 17 / 31
Journal Pre-proof Variance
4.62%
3.89%
2.34%
3.89%
2.76%
5.12%
6.12%
Function
F22
F23
F24
F25
F26
F27
F28
Mean
33.31%
41.22%
37.02%
37.91%
25.2%
20.12%
21.32%
Variance
6.12%
5.81%
5.78%
4.23%
1.89%
2.12%
3.23%
1 Table 6. Experimental results in mean, FR and Μ Μ Μ Μ Μ ππΌπ on functions 1-28 for 30D problems
F3
F4
F5
F6
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
9.6300E-29
7.2300E-30
1.1600E-07
4.6900E-27
1.0100E-11
0.0000E+00
7.3400E-29
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.6700E-30
8.6600E-29
4.7400E-30
6.3100E-08
1.1700E-26
7.1700E-12
0.0000E+00
7.3900E-29
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.4280E+05
3.6912E+05
3.1728E+05
9.6069E+05
7.4200E-27
1.1600E+06
6.7000E+06
7.3254E+01
1.0000E+00
1.0000E+00
1.0000E+00
4.4000E-01
1.0000E+00
4.4000E-01
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.0000E-04
0.0000E+00
1.2000E-04
0.0000E+00
0.0000E+00
1.3573E+01
1.3833E+01
1.3573E+01
1.3573E+01
3.5820E-01
1.3667E+01
1.3854E+01
8.2422E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.5020E-01
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.5950E-01
3.6500E-31
1.4700E-07
5.4837E+00
1.0157E+00
0.0000E+00
2.3200E-17
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.4538E+03
3.9102E+03
3.7330E+03
4.2628E+03
4.7760E-01
3.7296E+03
5.5264E+03
3.0362E+02
3.2000E-01
0.0000E+00
0.0000E+00
0.0000E+00
5.6000E-01
0.0000E+00
0.0000E+00
1.0000E+00
1.3060E-02
2.0800E-02
1.7800E-02
2.9250E-01
2.9140E-01
3.1080E-01
2.5670E-02
0.0000E+00
-9.3038E+01
-1.5046E+02
-1.0888E+02
-3.5931E+01
-1.5460E+03
2.5535E+01
-8.1088E+01
F7
F9
F10
F11
5.9813E+02
1.0000E+00
1.0000E+00
9.6000E-01
1.2000E-01
0.0000E+00
1.6000E-01
9.6000E-01
1.0000E+00
0.0000E+00
0.0000E+00
5.5100E-06
1.1000E-03
2.9055E+03
1.6800E-03
4.0600E-06
0.0000E+00
-2.800E-04
-2.8000E-04
-2.8000E-04
-1.8000E-04
-6.3808E+01
-2.1000E-04
-2.6000E-04
-2.8000E-04
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
7.8000E+06
0.0000E+00
0.0000E+00
0.0000E+00
-2.670E-03
-2.6700E-03
-2.6700E-03
-2.670E-03
-5.8710E-01
3.4250E-02
-2.6700E-03
-2.6700E-03
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.2749E+04
0.0000E+00
0.0000E+00
0.0000E+00
-1.000E-04
-1.0000E-04
-1.0000E-04
-9.9800E-05
-5.0439E+01
-1.000E-04
-9.8000E-05
-1.0000E-04
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.9500E+07
0.0000E+00
0.0000E+00
0.0000E+00
-8.5360E-01
-4.5390E-01
-8.6230E-01
1.9375E+00
-2.9687E+03
-2.0513E+01
-8.6510E-01
-2.8348E+01
1.0000E+00
8.8000E-01
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
1.0000E-19
0.0000E+00
1.1600E+22
3.4700E+59
1.5547E+00
0.0000E+00
2.0700E-02
3.9856E+00
2.1186E+01
3.9856E+00
3.9963E+00
9.9500E-01
3.9826E+00
6.0679E+00
1.8679E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
urn a
F8
pro of
F2
CAL-SHADE
re-
F1
ASA-DE 5.7500E-30
lP
Func
Jo
2
F12
18 / 31
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
1.6219E+00
0.0000E+00
0.0000E+00
0.0000E+00
6.9248E+00
1.5040E+01
4.8409E+00
1.3427E+01
5.9520E+00
2.3195E+01
2.6011E+01
8.1490E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.8808E+00
2.0530E+00
1.8330E+00
1.8373E+00
3.7250E-02
2.1033E+00
1.9086E+00
1.5258E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.4701E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.8693E+01
1.9447E+01
1.8818E+01
1.6305E+01
1.5321E+01
1.8567E+01
1.2912E+01
9.1420E+00
9.6000E-01
1.6000E-01
8.4000E-01
1.0000E+00
3.2300E-06
4.3000E-04
1.8600E-05
0.0000E+00
1.4652E+02
1.6072E+02
1.4853E+02
1.9434E+02
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.0019E+00
1.0141E+00
9.9840E-01
1.0221E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.5500E+01
1.5500E+01
1.5500E+01
1.5500E+01
8.2729E+02
4.8317E+03
1.5830E+03
5.8927E+02
pro of
F16
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.5366E+05
1.1100E+06
1.5794E+04
5.3500E-06
7.3300E-06
0.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
8.8000E-01
3.5785E+02
0.0000E+00
0.0000E+00
2.6200E-05
4.9700E-16
1.7851E+02
1.4301E+02
8.4193E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.5600E-03
1.0263E+00
1.0128E+00
1.0214E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
7.0390E+01
1.5500E+01
1.5500E+01
1.5500E+01
6.8000E-01
6.1701E+03
4.0096E+03
3.5682E+03
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.0528E+03
4.7288E+01
2.0400E+06
1.3300E+06
3.3600E+07
6.2900E-06
0.0000E+00
-2.7748E+01
7.8100E-06
0.0000E+00
3.6576E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.1375E+04
2.1375E+04
2.1375E+04
2.1375E+04
2.1416E+04
2.1375E+04
2.1375E+04
2.1379E+04
1.4602E+00
1.1764E+00
1.4034E+00
3.0345E+00
1.8854E+00
3.1429E+00
2.2476E+00
4.5816E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
9.6000E-01
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
3.3100E-03
0.0000E+00
0.0000E+00
0.0000E+00
2.2932E+01
1.9466E+01
1.8695E+01
9.5185E+00
8.5765E+01
1.8217E+01
2.7467E+01
1.1910E+01
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
9.2000E-01
lP
F15
0.0000E+00
urn a
F14
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.0666E+01
0.0000E+00
0.0000E+00
1.4800E-12
2.1354E+02
3.3571E+04
3.0641E+03
2.6707E+04
1.2136E+02
3.6693E+03
8.4401E+02
9.4147E+01
1.0000E+00
8.8000E-01
1.0000E+00
3.6000E-01
7.2000E-01
9.2000E-01
1.0000E+00
1.0000E+00
0.0000E+00
4.0420E-01
0.0000E+00
3.2106E+00
2.9194E+02
5.1750E-02
0.0000E+00
0.0000E+00
1.7837E+00
2.0591E+00
1.6560E+00
1.9966E+00
2.1052E+01
2.0072E+00
1.8564E+00
1.4349E+00
1.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.5224E+05
0.0000E+00
0.0000E+00
0.0000E+00
1.2032E+01
1.5048E+01
1.2158E+01
5.9028E+01
1.4898E+00
1.7059E+01
1.3917E+01
8.3879E+00
Jo
F13
0.0000E+00
re-
Journal Pre-proof
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
8.6524E+03
5.3150E-01
0.0000E+00
0.0000E+00
0.0000E+00
1.3936E+02
1.6644E+02
1.4143E+02
3.1894E+02
1.4772E+01
1.7712E+02
1.4062E+02
1.5959E+01
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
1.0000E+00
1.0000E+00
1.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.1520E+03
6.7950E-01
0.0000E+00
0.0000E+00
0.0000E+00
1.0062E+00
1.0176E+00
1.0056E+00
1.0264E+00
8.5720E-03
1.0277E+00
1.0123E+00
1.0194E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
1.5500E+01
1.5500E+01
1.5500E+01
1.5500E+01
5.8347E+01
1.5500E+01
1.5500E+01
1.5500E+01
19 / 31
Journal Pre-proof
F28
+ο―=ο―β +ο―=ο―β
9.5444E+03
5.0072E+02
1.9890E+02
2.4305E+04
5.2293E+04
1.1876E+04
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
4.9800E+06
3.8300E+07
1.0000E+07
1.6451E+04
5.7098E+04
2.0800E+08
1.0800E+08
3.4100E+08
1.1185E+02
1.5971E+02
1.3606E+02
1.4627E+02
3.8200E+01
1.0807E+02
1.5285E+02
6.4323E+01
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
0.0000E+00
2.1459E+04
2.1492E+04
2.1478E+04
2.1491E+04
2.1466E+04
2.1462E+04
2.1482E+04
2.1446E+04
Criterion I Criterion II
22/3/3 16/11/1
13/5/10 4/21/3
20/1/7 20/5/3
22/0/6 20/4/4
24/1/3 21/5/2
20/2/6 NA
16/3/9 NA
Table 7. The mean percentage of number of surrogate evaluated populations for each 30D problems F1
F2
F3
F4
F5
F6
F7
Mean
32.34%
32.82%
22.31%
15.76%
32.56%
35.72%
33.87%
Variance
4.32%
3.76%
5.89%
5.23%
4.56%
4.82%
5.11%
Function
F8
F9
F10
F11
F12
F13
F14
Mean
30.12%
32.21%
30.11%
31.19%
32.82%
33.97%
29.95%
Variance
5.08%
5.91%
4.12%
4.21%
5.29%
4.18%
4.02%
Function
F15
F16
F17
F18
F19
F20
F21
Mean
31.12%
32.12%
16.23%
18.69%
15.89%
32.121%
32.21%
Variance
3.38%
3.45%
1.89%
3.05%
2.76%
5.12%
6.12%
Function
F22
F23
F24
F25
F26
F27
F28
Mean
30.12%
35.82%
35.14%
34.21%
22.1%
16.79%
18.19%
Variance
4.12%
5.12%
4.35%
4.23%
2.09%
2.91%
2.29%
4
Table 8. Experimental results in mean, FR and Μ Μ Μ Μ Μ ππΌπ on functions 1-28 for 50D problems F1
F2
F3
F4
F5
F6
F7
F8
F9
F10 F11
ASA-DE 8.57e-29 1 0 6.97e-29 1 0 885090 1 0 13.5728 1 0 8.35e-29 1 0 7941.5900 0.4 0 -145.6200 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 -1.4199 0.92
CAL-SHADE 6.38e-28 1 0 7.49-28 1 0 725750 1 0 13.7135 1 0 0.3189 1 0 7154.11 0 0.0116 -260.388 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 1.7584 0.44
L-SHADE44 7.75e-29 1 0 7.96e-29 1 0 946043 1 0 13.5728 1 0 8.92e-29 1 0 7946.24 0 0.01264 -172.159 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 -1.6628 1
urn a
Func
Jo
5
re-
Function
lP
1 2 3
1.2675E+04
pro of
F27
2.3376E+03
SaDE 0.02424 1 0 0.01425 1 0 6.49e+06 0.48 7.87e-05 40.796 1 0 10.9402 1 0 7615.43 0 0.2675 -21.6109 0.04 0.00102 0.00366 0.96 4.76e-06 -0.00203 1 0 5.07e-05 1 0 48.59 0
SajDE 5.10e-15 1 0 3.58e-15 1 0 1.96e-15 1 0 2.587 0 0.7517 21.2546 1 0 4.7360 0.2 0.9332 -2563.96 0 5011.44 -14.9387 0 1.82e+07 0.7054 0 21419.9 -16.7263 0 8.33e+07 -4946.35 0
DEbin 0.1425 1 0 0.0455 1 0 5.06e+06 0.44 6.51e-05 13.5728 1 0 16.6896 1 0 7550.13 0 0.3818 -13.0956 0 0.00155 0.04298 0 0.00298 -0.00204 1 0 -3.24e-05 1 0 65.2575 0
L-S44+IDE 1.49e-08 1 0 0 1 0 2.65e+07 1 0 13.9880 1 0 0 1 0 8600.818 0 0.01525 -153.963 1 0 0.000286 1 0 -0.00168 1 0 9.12e-05 1 0 -1.9254 1
UDE 1.16e-10 1 0 3.24e-11 1 0 90.6979 1 0 158.418 1 0 14.9547 1 0 687.2246 0.96 5.85e-06 -941.8285 0.88 1.66e-05 0.0002 1 0 -0.00204 1 0 5.81e-05 1 0 -160.1565 0
20 / 31
Journal Pre-proof
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28 +ο―=ο―β +ο―=ο―β
2.4313 4.1081 1 0 70.7797 1 0 1.6326 1 0 20.3261 1 0 302.85 1 0 1.0492 0 25.5 14321.2 0 5.46e+07 1.49e-05 0 36116.2 7.7770 1 0 57.1599 1 0 40693 0 43.6401 1.5225 1 0 17.9385 1 0 315.039 1 0 1.04887 0 25.5 48022.1 0 1.84e+09 248.88 0 36321.5 23/2/3 23/4/1
0 7.3566 1 0 73.6353 1 0 1.4611 1 0 13.6659 1 0 251.4531 1 0 1.0450 0 25.5 7340.76 0 8.96e+06 0 0 36116.2 4.5789 1 0 62.1685 1 0 7838.53 1 0 1.3742 1 0 14.9225 1 0 254.1548 1 0 1.0411 0 25.5 6.37e+04 0 2.00e+08 267.1889 0 36317.4 18/0/10 NA
0.3887 12.9012 1 0 1293.788 1 0 1.1840 1 0 11.2783 1 0 13.069 1 0 1.0480 0 25.5 20155.975 0 6.56e+08 8.8444 0 36130.22 6.8396 1 0 6.8466 1 0 3186.796 0.76 0.2074 1.1240 1 0 11.6550 1 0 27.52 1 0 1.0478 0 25.5 60198.128 0 6.52e+09 129.11 0 36258.2 16/1/11 NA
Table 9. The mean percentage of number of surrogate evaluated populations for each 50D problems Function
F1
F2
F3
F4
F5
F6
F7
Mean
30.21%
32.67%
18.32%
15.12%
28.32%
31.56%
29.99%
Variance
3.29%
3.39%
6.21%
7.53%
6.19%
4.87%
5.14%
Function
F8
F9
F10
F11
F12
F13
F14
Mean
28.32%
30.32%
28.13%
29.18%
36.82%
39.12%
33.39%
Variance
3.12%
8.39%
3.87%
4.78%
5.11%
6.18%
5.21%
Function
F15
F16
F17
F18
F19
F20
F21
Mean
29.12%
23.41%
16.12%
12.37%
10.21%
22.12%
32.18%
Variance
2.34%
5.12%
2.34%
5.89%
3.45%
5.77%
5.32%
Jo
1 2 3
2.70e+99 11.3024 0 4.5132 10.5261 0.8 1.58983 0.3148 0 2.5679 31.4785 0 2279.59 4.76e-15 1 0 0.00856 0 118.433 6.6 0 5542.67 -46.041 0 36185.7 3.2270 0.96 0.004264 235.007 0 114.908 85.3516 0.76 217.017 21.2107 0 443239 10.2124 0 277.846 51.0171 0 0.8301 0.003844 0 117.822 416.528 0 149219 91.0368 0 36294.1 24/0/4 21/2/5
pro of
F15
2.26e+50 4.0002 1 0 126.221 1 0 1.4327 1 0 20.7031 1 0 375.734 0.92 4.78e-06 1.0468 0 25.5 857.646 0 18625.7 6.52e-06 0 36116.2 6.2603 1 0 25.4192 1 0 74216.5 0 51.4983 1.5441 1 0 75.9954 0 21883.1 609.72 0 3346.36 1.0475 0 25.5 818.044 0 69432 275.074 0 36328 21/0/7 22/2/4
re-
F14
0 53.3338 1 0 26.824 1 0 1.4119 1 0 17.5615 1 0 260.124 1 0 1.0312 0 25.5 6219.43 0 4.05e+07 1.21e-05 0 36116.2 3.2117 1 0 63.3177 1 0 8051.21 0.96 0.0056 1.3257 1 0 13.9172 1 0 253.526 1 0 1.0366 0 25.5 13404.6 0 2.00e+07 269.195 0 36316.3 14/5/9 8/20/0
lP
F13
6.11e-12 27.5311 1 0 51.9073 1 0 1.5756 1 0 20.4518 0.12 0.00016 281.738 1 0 1.0416 0 25.5 9581.65 0 1.51e+07 1.51e-05 0 36116.2 2.5485 1 0 12.2582 1 0 64028.9 0.28 3.0501 1.5710 1 0 17.0588 1 0 299.896 1 0 1.0435 0 25.5 19907 0 2.33e+08 268.093 0 36324 20/3/5 18/7/3
urn a
F12
1.83e-32 51.6255 1 0 23.2901 1 0 1.4330 1 0 18.9438 0.96 2.63e-06 256.542 1 0 1.0384 0 25.5 854.438 0 134670 1.21e-05 0 36116.2 3.1344 1 0 62.1813 1 0 2997.47 1 0 1.3788 1 0 14.1685 1 0 252.584 1 0 1.0389 0 25.5 4866.84 0 1.53e+07 213.915 0 36273 Criterion I Criterion II
21 / 31
Journal Pre-proof Function
F22
F23
F24
F25
F26
F27
F28
Mean
24.89%
32.12%
29.02%
32.18%
24.21%
15.32%
15.32%
Variance
5.72%
7.81%
4.87%
6.71%
1.34%
1.87%
1.87%
1 Table 10. Experimental results in mean, FR and Μ Μ Μ Μ Μ ππΌπ on functions 1-28 for 100D problems
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17 F18
L-S44+IDE 7 1 0 3 1 0 2.25e+08 1 0 14.028 1 0 0 1 0 15533 0 1.19e-02 -335.49 1 0 1.48e-03 1 0 -1.45e-04 1 0 4.25e-04 1 0 -2.834 0.96 0 7.015 1 0 71.760 1 0 0.977 1 0 16.053 1 0 529.74 1 0 1.0980 0 50.5 10091 0
pro of
F3
L-SHADE44 1.03e-25 1 0 8.47e-26 1 0 2.73e+06 1 0 13.7132 1 0 3.28e-05 1 0 15562.2 0 0.0098 -302.689 1 0 -4.81e-05 1 0 -0.00143 1 0 -1.72e-05 1 0 3.6511 0.88 5.25e-43 32.5056 1 0 80.7003 1 0 0.9718 1 0 18.0641 1 0 534.51 1 0 1.0962 0 50.5 3436.37 0
re-
F2
CAL-SHADE 0.9777 1 0 0.3661 1 0 1.514e+07 0.16 0.024 413.582 1 0 0.8188 1 0 15222.9 1 0 -193.458 0.4 0.00436 0.04160 0 0.00124 0.5225 0.96 1.564e-05 0.00051 1 0 -13.7384 0 0.14709 23.8996 1 0 118844 0 18.7453 0.7949 1 0 30.8846 0.6 1253.21 712.859 0.12 0.0488 1.0980 0 54.2153 16640.7 0
lP
F1
ASA-DE 7.21e-26 1 0 6.12e-21 1 0 2.12e+04 1 0 13.8310 1 0 2.35e-06 1 0 1932.4500 0.4 0 -132.6200 1 0 -4.81e-05 1 0 -1.45e-04 1 0 4.25e-04 1 0 -2.2199 0.92 5.83e-21 51.6255 1 0 67.3201 1 0 0.7671 1 0 18.0641 0.96 2.12e-06 26.2421 1 0 1.0980 0 54.2153 854.438 0
urn a
Func
Jo
2
UDE 6.16e-05 1 0 5.15e-05 1 0 388.18 1 0 386.65 1 0 41.284 1 0 2185.52 1 0 -1649.1 0.8 0.07357 0.00046 1 0 -0.00068 0.52 1.35e-21 1.61e-05 1 0 -549.15 0 1.2268 6.13 1 0 16793.687 0.48 3.47 0.7927 1 0 16.3048 1 0 21.7397 1 0 1.0987 0 50.5 64364.38 0 22 / 31
Journal Pre-proof
F22
F23
F24
F25
F26
F27
F28 +ο―=ο―β +ο―=ο―β
Function
F3
F4
F5
F6
F7
31.33%
20.22%
16.74%
23.32%
30.12%
28.12%
3.12%
3.33%
5.99%
6.92%
5.65%
4.12%
5.08%
F8
F9
F10
F11
F12
F13
F14
29.76%
29.12%
27.43%
27.89%
32.12%
37.21%
31.27%
3.23%
7.21%
4.87%
5.18%
3.99%
6.18%
5.08%
F15
F16
F17
F18
F19
F20
F21
27.31%
27.12%
20.98%
17.17%
12.12%
24.22%
30.23%
5.12%
7.12%
3.09%
5.23%
3.87%
5.21%
5.12%
Function
F22
F23
F24
F25
F26
F27
F28
Mean
26.87%
34.56%
27.78%
32.22%
27.21%
18.76%
18.12%
Variance
6.12%
6.12%
5.12%
6.12%
2.11%
2.01%
2.12%
Mean Variance Function Mean Variance Function Mean
F1
F2
28.93%
Jo
Variance
4 5
3.02e+09 38.0444 0 73030.81 15.1336 1 0 10.2589 1 0 44948.02 0 368.69 0.7842 1 0 17.81 1 0 102.29 1 0 1.098 0 50.5 197018.15 0 3.19e+10 298.416 0 73303.66 23/0/5 23/1/4
Table 11. The mean percentage of number of surrogate evaluated populations for each 100D problems
urn a
1 2 3
pro of
F21
2.74e+07 0 0 7.30e+04 11.480 1 0 29.544 1 0 60692 0.04 10.594 0.958 1 0 17.938 1 0 571.016 1 0 1.097 0 50.5 140923 0 1.45e+09 610.2 0 734e+04 27/1/0 24/0/4
3.34e+06 4.68e-05 0 72969.5 9.3628 1 0 31.5798 1 0 50364.5 0.04 6.4572 0.9694 1 0 17.1845 1 0 544.187 1 0 1.09616 0 50.5 36875 0 4.78e+08 584.017 0 73404.3 25/1/2 15/11/2
re-
F20
4.05e+09 8.82e-05 0 48646.3 7.4038 1 0 14.9419 1 0 263043 0 46.1771 0.8144 0.92 . 3.68e-05 22.8996 0.08 234.413 724.212 0.24 0.02953 1.101 0 52.6093 47247.7 0 1.37e+09 415.526 0 48880.7 25/1/2 19/6/3
lP
F19
134670 1.12e-05 0 32116.2 2.2314 1 0 21.1813 1 0 3345.47 1 0 0.7981 1 0 13.1123 1 0 252.584 1 0 1.0389 0 50.5 4982.1289 0 1.24e+07 187.6720 0 38982.2 Criterion I Criterion II
Table 12 Successful optimization rate for benchmark functions Dimension
10D
Algorithm
ASA-DE
CAL-SHADE
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
SR
78.57%
71.42%
71.42%
64.29%
35.71%
67.86%
75%
75%
23 / 31
Journal Pre-proof Failure(f)
6
8
8
10
Dimension ASA-DE
CAL-SHADE
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
71.42%
75%
60.71%
32.14%
67.86%
75%
75%
Failure(f)
6
8
7
11
19
9
7
7
50D
Algorithm
ASA-DE
CAL-SHADE
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
SR
78.57%
67.86%
75%
57.14%
28.57%
60.71%
75%
75%
Failure(f)
6
9
7
12
20
11
7
7
100D ASA-DE
CAL-SHADE
L-SHADE44
SaDE
SajDE
DEbin
L-S44+IDE
UDE
SR
65.57%
45.86%
71.2%
N/A
N/A
N/A
70.21%
67.21%
Failure(f)
5
5
6
N/A
N/A
N/A
7
6
pro of
Algorithm
lP
re-
Moreover, the bar charts of statistical results of ASA-DE and 8 state-of-the-art other algorithms on 28 CEC2017 benchmark functions are presented when the dimensions are 50 and 100, where the blue and red bars represent the rank score of Friedman test and the number of functions with the best results, respectively. According to Fig.2, it can be found that the ASADE has the best overall performance and the advantages are more obvious in the 50D and 100D problems comparing with other DEs. It can be easily found that the ASA-DE is the best algorithm in terms of these criteria.
50D
Jo
urn a
10
13 14 15 16
7
78.57%
1
12
7
SR
Dimension
11
9
Algorithm
Dimension
2 3 4 5 6 7 8 9
18
30D
100D Fig.2 The rank score and the number of functions with the best results of ASA-DE on CEC2017 benchmark functions.
24 / 31
Journal Pre-proof
F4
F5
F3
pro of
F2
F6
F8
F9
F10
urn a
F11
F12
Jo
F7
lP
re-
F1
F13
F14
F15
25 / 31
Journal Pre-proof
F19
F20
F18
pro of
F17
F21
F22
F24
F25
F26
F27
F28
urn a
F23
Jo
lP
re-
F16
1 2 3 4
Fig.3 Convergence curves for 28 test function (100D) Furthermore, to investigate the convergence ratio, several representative function iteration curves with 100 variables are demonstrated in Fig. 3. In these figures, X and Y axial represent 26 / 31
Journal Pre-proof 1 2 3
the number of function evaluations (FES) and the logarithm of the function error value, respectively. According to these results, the suggested algorithm also are easier to converge for most of test functions.
4
4 Conclusions An adaptive surrogate assisted differential evolution with novel mutation strategy, called ASA-DE is suggested for constrained optimization problems. In the ASA-DE method, several kinds of surrogate-modeling techniques are integrated. Compared with other surrogate modeling assisted DE methods, more surrogate modeling techniques are used. Due the diversity of modeling performance, such strategy is easier to obtain high accurate model. Moreover, considering the curse of dimensionality, the surrogate model should be constructed based on the local region. The test results suggests that the surrogate modeling technique is still useful for the highly dimensional problems. Moreover, in order to follow the change of suitable balance point between exploration and exploitation, adaptive population size control based on pfeas and parameter adaptation strategy based on success-history are suggested. When these strategies are integrated, the performance of the optimization based on the ASA-DE can be guaranteed. According the results of comparison with other state-of-the-art algorithms, ASA-DE show a better or at least competitive performance compared to other algorithms. The experimental results also indicate that the state-switching mechanism can improve the performance of the algorithm. However, it is difficult for ASA-DE to solve f17, f18, f19, f26, f27, f28, but other algorithms also fail to search feasible solutions. Moreover, the number of benchmark functions that ASADE finds feasible solutions is the most compared to other algorithms.
23
Acknowledgment
24 25
This work has been supported by the Key Projects of the Research Foundation of Education Bureau of Hunan Province (17A224).
26
References
re-
lP
urn a
1.
Hu, W., Yao, L. G., & Hua, Z. Z. (2008). Optimization of sheet metal forming processes by adaptive response surface based on intelligent sampling method. Journal of materials processing technology, 197(1-3), 77-
Jo
27 28 29 30 31 32 33 34 35 36
pro of
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
88. 2.
Abedinia, O., Naderi, M. S., Jalili, A., & Mokhtarpour, A. (2011). A novel hybrid GA-PSO technique for optimal tuning of fuzzy controller to improve multi-machine power system stability. International Review of Electrical Engineering, 6(2).
3.
Abedinia, O., Amjady, N., Kiani, K., Shayanfar, H. A., & Ghasemi, A. (2012). Multiobjective environmental and economic dispatch using imperialist competitive algorithm. International Journal on Technical and Physical Problems of Engineering (IJTPE), (11), 63-70.
4.
Kar, A. K. (2016). Bio inspired computingβA review of algorithms and scope of applications. Expert 27 / 31
Journal Pre-proof Systems with Applications, 59, 20-32. 5.
Abedinia, O., Ghasemi, A., & Ojaroudi, N. (2016). Improved time varying inertia weight PSO for solved economic load dispatch with subsidies and wind power effects. Complexity, 21(4), 40-49.
6.
Brest, J., Zumer, V., & Maucec, M. S. (2006, July). Self-adaptive differential evolution algorithm in constrained real-parameter optimization. In Evolutionary Computation, 2006. CEC 2006. IEEE Congress on (pp. 215-222). IEEE.
7.
Hamza, N. M., Sarker, R. A., & Essam, D. L. (2013). Differential evolution with multi-constraint consensus methods for constrained optimization. Journal of Global Optimization, 57(2), 583-611. Das, S., Mullick, S. S., & Suganthan, P. N. (2016). Recent advances in differential evolutionβan updated
pro of
8.
survey. Swarm and Evolutionary Computation, 27, 1-30. 9.
Storn, R., & Price, K. (1997). Differential evolutionβa simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11(4), 341-359.
10.
Brest, J., Greiner, S., Boskovic, B., Mernik, M., & Zumer, V. (2006). Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE transactions on evolutionary computation, 10(6), 646-657.
11.
Qin, A. K., Huang, V. L., & Suganthan, P. N. (2009). Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE transactions on Evolutionary Computation, 13(2),
re-
398-417. 12.
Zhang, J., & Sanderson, A. C. (2009). JADE: adaptive differential evolution with optional external archive. IEEE Transactions on evolutionary computation, 13(5), 945-958.
13.
Tanabe, R., & Fukunaga, A. (2013, June). Success-history based parameter adaptation for differential evolution. In Evolutionary Computation (CEC), 2013 IEEE Congress on (pp. 71-78). IEEE. Tang, L., Dong, Y., & Liu, J. (2015). Differential evolution with an individual-dependent mechanism. IEEE
lP
14.
Transactions on Evolutionary Computation, 19(4), 560-574. 15.
Zhou, Y. Z., Yi, W. C., Gao, L., & Li, X. Y. (2017). Adaptive differential evolution with sorting crossover rate for continuous optimization problems. IEEE transactions on cybernetics, 47(9), 2742-2753.
16.
Bujok, P., Tvrdik, J., & Polakova, R. (2014, July). Differential evolution with rotation-invariant mutation
urn a
and competing-strategies adaptation. In Evolutionary Computation (CEC), 2014 IEEE Congress on (pp. 2253-2258). IEEE. 17.
Wang, Y., Cai, Z., & Zhang, Q. (2011). Differential evolution with composite trial vector generation strategies and control parameters. IEEE Transactions on Evolutionary Computation, 15(1), 55-66.
18.
Trivedi, A., Sanyal, K., Verma, P., & Srinivasan, D. (2017, June). A unified differential evolution algorithm for constrained optimization problems. In Evolutionary Computation (CEC), 2017 IEEE Congress on (pp. 1231-1238). IEEE.
19.
Zamuda, A., & Brest, J. (2012). Population reduction differential evolution with multiple mutation strategies in real world industry challenges. In Swarm and evolutionary computation (pp. 154-161).
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
Springer, Berlin, Heidelberg. 20.
Tanabe, R., & Fukunaga, A. S. (2014, July). Improving the search performance of SHADE using linear population size reduction. In Evolutionary Computation (CEC), 2014 IEEE Congress on (pp. 1658-1665). IEEE.
21.
Brest, J., & Maucec, M. S. (2011). Self-adaptive differential evolution algorithm using population size reduction and three strategies. Soft Computing, 15(11), 2157-2174.
22.
Yang, Z., Tang, K., & Yao, X. (2008, June). Self-adaptive differential evolution with neighborhood search. In Evolutionary Computation, 2008. CEC 2008. (IEEE World Congress on Computational Intelligence). 28 / 31
Journal Pre-proof IEEE Congress on (pp. 1110-1116). IEEE. 23.
Du, W., Leung, S. Y. S., Tang, Y., & Vasilakos, A. V. (2016). Differential evolution with event-triggered impulsive control. IEEE transactions on cybernetics, 47(1), 244-257.
24.
Awad, N. H., Ali, M. Z., & Suganthan, P. N. (2018). Ensemble of parameters in a sinusoidal differential evolution with niching-based population reduction. Swarm and evolutionary computation, 39, 141-156.
25.
Wang, S., Li, Y., & Yang, H. (2019). Self-adaptive mutation differential evolution algorithm based on particle swarm optimization. Applied Soft Computing, 81, 105496.
26.
Coit, D. W. , Smith, A. E. , & Tate, D. M. . (1996). Adaptive penalty methods for genetic optimization of
27.
pro of
constrained combinatorial problems. INFORMS Journal on Computing, 8(2), 173-182.
Farmani, R. , & Wright, J. A. . (2003). Self-adaptive fitness formulation for constrained optimization. Evolutionary Computation IEEE Transactions on, 7(5), 445-455.
28.
Deb, K. (2000). An efficient constraint handling method for genetic algorithms. Computer methods in applied mechanics and engineering, 186(2-4), 311-338.
29.
Yao, T. P. R. X. . (2000). Stochastic ranking for constrained evolutionary optimization. Evolutionary Computation IEEE Transactions on, 4(3), 284-294.
30.
Mezuramontes, E , and C. A. C. Coello . "A simple multimembered evolution strategy to solve constrained optimization problems." IEEE Transactions on Evolutionary Computation 9.1(2005):1-17. Takahama, T. , & Sakai, S. (2005). Constrained optimization by applying the Ξ± constrained method to the
re-
31.
nonlinear simplex method with mutations. IEEE Press. 32.
Coello, Carlos A. Coello.(2000) "Treating constraints as objectives for single-objective evolutionary optimization." Engineering Optimization, 32(3),34.
33.
Coello, C. A. C. , & Efren Mezura Montes. (2002). Constraint-handling in genetic algorithms through the
34.
lP
use of dominance-based tournament selection. Advanced Engineering Informatics, 16(3), 193-203. Ray, T. , & Liew, K. M. . (2003). Society and civilization: an optimization algorithm based on the simulation of social behavior. IEEE Transactions on Evolutionary Computation, 7(4), 386-396. 35.
Cai, Z., & Wang, Y. (2006). A multiobjective optimization-based evolutionary algorithm for constrained optimization. IEEE Transactions on evolutionary computation, 10(6), 658-675. Wang, Y., Cai, Z., Guo, G., & Zhou, Y. (2007). Multiobjective optimization and hybrid evolutionary
urn a
36.
algorithm to solve constrained optimization problems. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 37(3), 560-575. 37.
Wang, Y., Cai, Z., Zhou, Y., & Zeng, W. (2008). An adaptive tradeoff model for constrained evolutionary optimization. IEEE Transactions on Evolutionary Computation, 12(1), 80-92.
38.
Wang, Y., & Cai, Z. (2011). Constrained evolutionary optimization by means of (ΞΌ+ Ξ»)-differential evolution and improved adaptive trade-off model. Evolutionary Computation, 19(2), 249-285.
39.
Wang, Y., & Cai, Z. (2012). Combining multiobjective optimization with differential evolution to solve constrained optimization problems. IEEE Transactions on Evolutionary Computation, 16(1), 117-134.
40.
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
Mohamed, A. W., & Sabry, H. Z. (2012). Constrained optimization based on modified differential evolution algorithm. Information Sciences, 194, 171-208.
41.
Hernandez, S., Leguizamon, G., & Mezura-Montes, E. (2013). Hybridization of differential evolution using hill climbing to solve constrained optimization problems. Inteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial, 16(52).
42.
Sarker, R. A., Elsayed, S. M., & Ray, T. (2014). Differential evolution with dynamic parameters selection for optimization problems. IEEE Transactions on Evolutionary Computation, 18(5), 689-707.
43.
Zhang, W., Yen, G. G., & He, Z. (2014). Constrained optimization via artificial immune system. IEEE 29 / 31
Journal Pre-proof Transactions on Cybernetics, 44(2), 185-198. 44.
Ong, Y. S., Nair, P. B., & Keane, A. J. (2003). Evolutionary optimization of computationally expensive problems via surrogate modeling. AIAA journal, 41(4), 687-696.
45.
Jin, Y. (2011). Surrogate-assisted evolutionary computation: Recent advances and future challenges. Swarm and Evolutionary Computation, 1(2), 61-70.
46.
Jin, Y. (2005). A comprehensive survey of fitness approximation in evolutionary computation. Soft computing, 9(1), 3-12.
47.
Shi, L., & Rasheed, K. (2008, July). ASAGA: an adaptive surrogate-assisted genetic algorithm. In
pro of
Proceedings of the 10th annual conference on Genetic and evolutionary computation (pp. 1049-1056). ACM. 48.
Gao, W., Yen, G. G., & Liu, S. (2014). A cluster-based differential evolution with self-adaptive strategy for multimodal optimization. IEEE transactions on cybernetics, 44(8), 1314-1327.
49.
Miranda-Varela, M. E., & Mezura-Montes, E. (2016). Surrogate-assisted differential evolution with an adaptive evolution control based on feasibility to solve constrained optimization problems. In Proceedings of Fifth International Conference on Soft Computing for Problem Solving (pp. 809-822). Springer, Singapore.
Li, E., & Wang, H. (2016). Bi-direction multi-surrogate assisted global optimization. Engineering Computations, 33(3), 646-666.
51.
re-
50.
Awad, N. H., Ali, M. Z., Mallipeddi, R., & Suganthan, P. N. (2018). An improved differential evolution algorithm using efficient adapted surrogate model for numerical op51timization. Information Sciences, 451, 326-347.
52.
Taran, N., Ionel, D. M., & Dorrell, D. G. (2018). Two-level surrogate-assisted differential evolution multi-
53.
lP
objective optimization of electric machines using 3-D FEA. IEEE Transactions on Magnetics, 54(11), 1-5. Cai, X., Gao, L., Li, X., & Qiu, H. (2019). Surrogate-guided differential evolution algorithm for high dimensional expensive problems. Swarm and Evolutionary Computation, 48, 288-311. 54.
Elsayed, S. M., Sarker, R. A., & Essam, D. L. (2013). An improved self-adaptive differential evolution algorithm for optimization problems. IEEE Transactions on Industrial Informatics, 9(1), 89-99. Brest, J., Boskovic, B., & Zumer, V. (2010, July). An improved self-adaptive differential evolution algorithm
urn a
55.
in single objective constrained real-parameter optimization. In Evolutionary Computation (CEC), 2010 IEEE Congress on (pp. 1-8). IEEE. 56.
Mallipeddi, R., & Suganthan, P. N. (2010). Ensemble of constraint handling techniques. IEEE Transactions on Evolutionary Computation, 14(4), 561-579.
57.
Mezura-Montes, E., & Coello, C. A. C. (2011). Constraint-handling in nature-inspired numerical optimization: past, present and future. Swarm and Evolutionary Computation, 1(4), 173-194.
58.
Farmani, R., & Wright, J. A. (2003). Self-adaptive fitness formulation for constrained optimization. IEEE Transactions on Evolutionary Computation, 7(5), 445-455.
59.
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
Tessema, B., & Yen, G. G. (2006, July). A self adaptive penalty function based algorithm for constrained optimization. In Evolutionary Computation, 2006. CEC 2006. IEEE Congress on (pp. 246-253). IEEE.
60.
Kleijnen, J. P. (2009). Kriging metamodeling in simulation: A review. European journal of operational research, 192(3), 707-716.
61.
Clarke, S. M., Griebsch, J. H., & Simpson, T. W. (2005). Analysis of support vector regression for approximation of complex engineering analyses. Journal of mechanical design, 127(6), 1077-1087.
62.
Regis, R. G., & Shoemaker, C. A. (2005). Constrained global optimization of expensive black box functions using radial basis functions. Journal of Global optimization, 31(1), 153-171. 30 / 31
Journal Pre-proof 63.
Wang, H., Tang, L., & Li, G. Y. (2011). Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Systems with Applications, 38(11), 14117-14126.
64.
Chen, L., Wang, H., Ye, F., & Hu, W. (2018). Comparative study of HDMRs and other popular metamodeling techniques for high dimensional problems. Structural and Multidisciplinary Optimization, 1-22.
65.
Wang, H., Tang, L., & Li, G. Y. (2011). Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Systems with Applications, 38(11), 14117-14126.
66.
Piotrowski, A. P. (2013). Adaptive memetic differential evolution with global and local neighborhoodbased mutation operators. Information Sciences, 241, 164-194. Ali, M. Z., Awad, N. H., Suganthan, P. N., & Reynolds, R. G. (2017). An adaptive multipopulation differential
pro of
67.
evolution with dynamic population reduction. IEEE transactions on cybernetics, 47(9), 2768-2779. 68.
Zhou, Y., Li, X., & Gao, L. (2013). A differential evolution algorithm with intersect mutation operator. Applied Soft Computing, 13(1), 390-401.
69.
Wu, G., Mallipeddi, R., & Suganthan, P. N. (2017). Problem definitions and evaluation criteria for the cec 2017 competition on constrained real-parameter optimization. National University of Defense Technology, Changsha, Hunan, PR China and Kyungpook National University, Daegu, South Korea and Nanyang Technological University, Singapore, Technical Report.
70.
Yang, M., Cai, Z., Li, C., & Guan, J. (2013, July). An improved adaptive differential evolution algorithm with computation (pp. 145-152). ACM.
71.
re-
population adaptation. In Proceedings of the 15th annual conference on Genetic and evolutionary Derrac, J., GarcΓa, S., Molina, D., & Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation, 1(1), 3-18.
Zamuda, Ales. Adaptive constraint handling and success history differential evolution for CEC 2017
lP
72.
constrained real-parameter optimization. Evolutionary Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. 73.
TvrdΓk, Josef, and Radka Polakova. A simple framework for constrained problems with application of LSHADE44 and IDE. Evolutionary Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. Polakova, Radka. L-SHADE with competing strategies applied to constrained optimization. Evolutionary
urn a
74.
Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. 75.
Brest, Janez, Viljem Zumer, and Mirjam Sepesy Maucec. Self-adaptive differential evolution algorithm in constrained real-parameter optimization. Evolutionary Computation, 2006. CEC 2006. IEEE Congress on. IEEE, 2006.
76.
Suykens, J. A., Van Gestel, T., & De Brabanter, J. (2002). Least squares support vector machines. World Scientific.
Jo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
31 / 31
Journal Pre-proof
Highlights An adaptive surrogate assisted DE algorithm is suggested;
2.
Several kinds of surrogate modeling techniques are integrated in suggested algorithm;
3.
An adaptive population size strategy based on pfeas is suggested;
4.
Parameter adaptation strategy based on success-history is suggested;
5.
The suggested algorithm is evaluated by the benchmark problems from CEC2017.
Jo
urn a
lP
re-
pro of
1.
Journal Pre-proof
Conflict of interest statement
pro of
We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled βAn adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problemsβ
Jo
urn a
lP
re-
Authors Enying Li Zheng Zhou College of Mechanical & Electrical Engineering, Central South University of Forestry and Technology, Changsha, 41004, P.R. China