An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems

An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems

Journal Pre-proof An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems Enying Li PII: DOI: Re...

1MB Sizes 0 Downloads 35 Views

Journal Pre-proof An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems Enying Li

PII: DOI: Reference:

S1568-4946(19)30533-2 https://doi.org/10.1016/j.asoc.2019.105752 ASOC 105752

To appear in:

Applied Soft Computing Journal

Received date : 7 November 2018 Revised date : 2 August 2019 Accepted date : 2 September 2019 Please cite this article as: E. Li, An adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems, Applied Soft Computing Journal (2019), doi: https://doi.org/10.1016/j.asoc.2019.105752. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Β© 2019 Elsevier B.V. All rights reserved.

Journal Pre-proof

1

An adaptive surrogate assisted differential evolutionary

2

algorithm for high dimensional constrained problems Enying Li College of Mechanical & Electrical Engineering, Central South University of Forestry and Technology, Changsha, 41004, P.R. China Nomenclature

pro of

3 4 5 6

DE

constrained optimization problems

congress on evolutionary computation

CEC

Evolutionary Algorithm

EA

Function Evaluations

FEs

Kriging

KG

Radial Basis Function

RBF

leave-one-out cross validation

LOOCV

Polynomial Regression

PR

Support Vector Regression

SVR

scaling factor

F Np

exclusive integers

the best individual in the population at the iteration t

𝑅1𝑖 ,

𝑅2𝑖 , 𝑅3𝑖 (𝑑)

target vector

π‘₯𝑖,𝑑

mutation vector

π‘₯𝑏𝑒𝑠𝑑 𝑣𝑖,𝑑

trail vector

𝑒𝑖,𝑑

decision variable

π‘—π‘Ÿπ‘Žπ‘›π‘‘

inequality constraint

𝑔𝑖 (π‘₯)

design space

πœ“

tolerance parameter feasible ratio normalized fitness value population

β„Žπ‘— (π‘₯)

number of inequality constraints

lP

quality constraint

q

constraint violation

πœƒ(π‘₯)

π‘π‘“π‘’π‘Žπ‘ 

overall constraint violation

𝑣(π‘₯)

𝑓 β€²β€² (π‘₯)

penalty value

𝑝(π‘₯)

𝛿

𝑀(𝐺)

urn a

the kth subpopulation of the Gth generation

π‘šπ‘˜πΊ

present number of the successful evolutions in sth subpopulation

𝑆𝑒𝑠

entire population size in the Gth generation

𝑁𝑝𝐺

small positive constant

𝑆𝑒0

preset minimum population size

π‘π‘π‘šπ‘–π‘›

initial population size

π‘π‘π‘šπ‘Žπ‘₯

population size in the generation

𝑁𝑝𝑓𝑖π‘₯

Abstract

Jo

8 9 10 11 12 13

Cr

crossover rate

re-

population size in the tth generation

7

COPs

differential evolution

Differential evolution (DE) is a competitive algorithm for constrained optimization problems (COPs). In this study, in order to improve the efficiency and accuracy of the DE for high dimensional problems, an adaptive surrogate assisted DE algorithm, called ASA-DE is suggested. In the ASA, several kinds of surrogate modeling techniques are integrated. Furthermore, to avoid violate the constraints and obtain better solution simultaneously, adaptive strategies for population size and mutation are also suggested in this study. The



Corresponding author [email protected]

1 / 31

Journal Pre-proof suggested adaptive population strategy which controls the exploring and exploiting states according to whether algorithm find enough feasible solution is similar to a state switch. The mutation strategy is used to enhance the effect of state switch based on adaptive population size. Finally, the suggested ASA-DE is evaluated on the benchmark problems from congress on evolutionary computation (CEC) 2017 constrained real parameter optimization. The experimental results show the proposed algorithm is a competitive one compared to other state-of-the-art algorithms.

8

Keywords

9

Surrogate assisted, Differential evaluation, Adaptive strategy, High dimensional problem

pro of

1 2 3 4 5 6 7

1. Introduction

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

Evolutionary Algorithm (EA) is a useful optimization method for complicated problems and has been applied to multidiscipline [1-5]. Differential Evolutions (DE) is an important algorithm of EAs and attracts lots of scholars’ attention due to its competitive performance on constrained optimization problems over the past decades [6-8]. Additionally, most of practical optimization problems are Constrained Optimization Problems (COPs), which might lead to the difficulty for convergence. The purpose of this study is to find the global optimization solutions as accurately as possible for COPs. Since the DE was proposed [9], many efforts have been designed to enhance the performance of the DE for constrained problems. Most of them focused on the setting of control parameters such as scaling factor (F), crossover rate (Cr), selection mechanism, mutation strategies and population size (Np). However, parameter settings commonly are diverse for different cases. Furthermore, the interactions between performance and parameter are also complicated. It motivated many scholars to develop some adaptive parameter setting based DEs such as jDE [10], SaDE [11], JADE [12], SHADE [13], IDE [14] and JADE_sort [15]. The essential of these strategies is to tune control parameters in the present iteration according to feedbacks from the previous ones. These controlling strategies are commonly used to adjust Cr and F. Moreover, other kinds of parameter setting strategies were widely used such as EPSDE [16], adaptive DE [17] and CoDE [18], which select the parameters from a pool involving sets of candidate populations. The selection mechanism is another critical issue for the DE. The essential of this mechanism is based on greedy choice in the classical DE. These strategies have also improved the performance of the DE. Population size control is another important mechanism for the DE. Several adaptive or non-adaptive population size control methods have been proposed and achieved competitive performances. For instance, Zamuda et al. [19] controlled the population size based on increasing number of FEs, a set of novel mutation strategies which depending on population size were also suggested. It showed its superior performance on the CEC2011 real life problems. Tanabe et al. [20] employed the linear population size reduction strategy to improve the SHADE algorithm, the population size is reduced with increasing FEs (Function Evaluations) and called as L-SHADE. It will stop if the population size reaches the present minimum (Npmin). The L-SHADE also won the CEC2014 competition on real parameter single-objective optimization.

Jo

urn a

lP

re-

10

2 / 31

Journal Pre-proof

urn a

lP

re-

pro of

Another population size controlling-method called Linear Population Size Reduction (LPSR) requires only one parameter (the initial population size). Similar algorithm Dynamic Population Size Reduction (DPSR) [21] was suggested to reduce the population by half at preset intervals. These non-adaptive methods can increase or decrease (decrease in most cases) the population size according to preset conditions and simplify the method of controlling population size. Moreover, instead of reduction of the population size by fixed formula or predetermined path, several adaptive population size techniques were proposed recently. Yang et al. [22] proposed a novel strategy, which varies the population size in a predetermined range. It will detect the present state and decrease or increase the present population size according to the feedbacks from the previous iterations, and the population size will be expanded by introducing more solutions when the algorithm is unable to find a better solution within a preset number of consequent generations. On the contrary, if the algorithm updates the best solution successfully in successive generations, the population size will reduce in order to purge the redundant solutions. Many experiments and studies have proved the effectiveness of the population size control method. More recently, some novel strategies have been introduced to improve the performance of the DE. Du et al. introduced an Event-Triggered Impulsive (ETI) to improve the performance of DE [23]. Awad et al. presented an enhanced algorithm based on LSHADE with ensemble parameter sinusoidal adaptation called LSHADE-EpSin [24]. The LSHADE-EpSin suggested a mixture of two sinusoidal formulas and a Cauchy distribution to balance the exploration and the exploitation of existed best solutions. A restart method is used at later generations to enhance the quality of the found solutions. By introducing Impulsive Control (IPC) and ETM into DE, they hope to change the search performance of the population in a positive way after revising the positions of some individuals. Wang et al. suggested a self-adaptive mutation DE algorithm based on particle swarm optimization (DEPSO) [25]. Compared with the unconstrained problems, the COPs are more difficult to be handled because both constraint violation and objective function should be considered. The present popular constraint-handling strategies can be mainly categorized three types: penalty function methods [26-27]; methods based on preference of feasible solutions over infeasible ones [28-31]; multi-objective based methods [32-37]. Briefly, the purpose of the COPs is to find the feasible region efficiently and obtain the solution finally. For the DEs, some novel algorithms have been suggested. Wang and Cai [38] proposed (ΞΌ+Ξ»)-CDE framework. They also [39] developed CMODE algorithm by integration of multi-objective optimization with DE. Mohamed and Sabry [40] employed several strategies on the DE, involving mutation operators, F, and CR. All mentioned algorithms used direct evaluation way to obtain the response of objective function. Hernandez et al. [41] suggested a hybridization of DE and hill climbing, which uses static penalty to handle constraints. Sarker et al. [42] developed a DE with dynamic parameter selection. Zhang et al. [43] suggested a constrained artificial immune system based on immune response principle. Considering the number of evaluations, the computational cost and convergence ratio still need to be addressed, especially for the complicated problems. Therefore, Surrogate Assisted Evolutionary Algorithm (SAEA) was motived by this issue [44]. Theoretically, surrogates can be applied to most of operations of evolutionary algorithms, such as population initialization, cross-over, mutation, local search and fitness evaluation. In such algorithms, a surrogate model is used to evaluate the objective function by constructing an

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

3 / 31

Journal Pre-proof approximate model [45]. Over the last few decades, many surrogate models have been proposed, such as, Kriging(KG), Support Vector Regression(SVR), Polynomial Regression(PR), and Radial Basis Function (RBF) [46]. Similar to other algorithms, the DE has been incorporated with surrogate modeling techniques and achieved good results [47-50]. Awad et al. developed a novel DE with a novel adapted KG surrogate model called iDEaSm. During the modeling in the iDEaSm, the covariance matrix is used to adapt the theta parameter in the KG model at each generation [51]. Taran et al. suggested a two-level surrogate-assisted optimization algorithm is proposed for electric machine design [52]. Cai et al. introduced an efficient surrogate-guided DE based on JADE algorithm [53]. In this study, we hope to find an alternative way to improve the efficiency of surrogate assisted DE algorithm. Compared with other surrogate assisted algorithms, such as iDEaSm, several surrogate-modeling techniques are employed. The diversity of surrogate modeling should be enhanced and the best of them should be selected. by the given criteria. Therefore, the suggested method is called as Adaptive Surrogate Assisted DE (ASA-DE) algorithm. To evaluate performance of the ASA-DE, 28 benchmark functions from CEC2017 are tested. The results demonstrate that performance of the ASA-DE is better than, or at least competitive to the stateof-the-art constraint optimization algorithms, and show the competitiveness relative to the other excellent algorithm. The rest of the paper is organized as follows. The classical DE algorithm and several constraint handling methods for constrained optimization are briefly introduced in Section 2, Section 3 gives the detailed description of the proposed ASA-DE, and results of benchmark functions and comparison with other state-of-the-art algorithms are shown in Section 4. Finally, Section 5 summarizes the conclusions and discusses the direction of future work.

24

2. Classical DE algorithm and constraint handling

re-

lP

methods

urn a

25

pro of

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

2.1 DE Algorithm

27 28 29 30 31 32 33 34 35 36 37

The process of the DE consists of three main operators: mutation, crossover and selection. Similar to other Evolutionary Algorithms (EAs) for optimization problems, the DE is based on population. A DE population consists of Np individual vectors π‘₯𝑖,𝑑 = (π‘₯1,𝑑 , … , π‘₯𝑛,𝑑 ), 𝑖 = 1, … , 𝑁𝑝, (1) where n is the number of design variables, Np is the population size in the present generation, the population individual vectors are generated randomly at the beginning of optimization. Np trail vectors are generated from the survived population individual by mutation and crossover operations in each generation t until termination criteria are satisfied. The details of DE are discussed according to Ref. [6] as follows. Mutation Operation: After the initialization, the DE adopts the mutation operation based on the difference of other individuals to generate mutant vector 𝑉𝑖,𝑑 with respect to each target vector π‘₯𝑖,𝑑 , in the present generation. Mutation strategies influence the balance of exploration

38

and exploitation during the evolutionary process, some most frequently referred mutation

Jo

26

4 / 31

Journal Pre-proof strategies are listed below: "DE/rand/1": (𝑑)

𝑣𝑖 3

(𝑑)

(𝑑)

1

2

3

(𝑑)

(𝑑)

1

2

(2)

"DE/best/1": (𝑑)

𝑣𝑖 4

(𝑑)

= π‘₯𝑅𝑖 + 𝐹 (π‘₯𝑅𝑖 βˆ’ π‘₯𝑅𝑖 )

(𝑑)

(3)

= π‘₯𝑏𝑒𝑠𝑑 + 𝐹 (π‘₯𝑅𝑖 βˆ’ π‘₯𝑅𝑖 )

"DE/current-to-best/1": (𝑑)

𝑣𝑖

(𝑑)

= π‘₯𝑖

(𝑑)

(𝑑)

(𝑑)

(𝑑)

1

2

pro of

1 2

+ 𝐹 (π‘₯𝑏𝑒𝑠𝑑 βˆ’ π‘₯𝑖 ) + 𝐹 (π‘₯𝑅𝑖 βˆ’ π‘₯𝑅𝑖 )

(4)

5 6

where the indices 𝑅1𝑖 , 𝑅2𝑖 , and 𝑅3𝑖 are exclusive integers randomly chosen from the range [1, Np], (Np is population size in the tth generation). F is the scaling factor which usually lies in [0,

7

1] for scaling the difference vectors. π‘₯𝑏𝑒𝑠𝑑 is the best individual in the population at the

8 9 10 11 12 13 14 15 16 17

iteration t. Generally, mutation strategies can be classified into three categories: exploring biased, exploiting biased and balance modes. The DE/rand/1 belongs to the type of exploring biased; the DE/best/1 belongs to the type of exploiting biased. Moreover, DE/current-tobest/1 is a balance mode because the diversity is enhanced by adding a difference vector composed of random individuals and the use of best individual. The suitable balance point of exploration and exploitation is updating dynamically in the evolutionary process. Therefore, an adaptive mutation strategy based on feasible rate is suggested in this study to switch state between exploring biased and exploiting biased in order to keep the efficiency of algorithm. Crossover Operation: After the mutation, crossover operation is used to generate trail vector 𝑒𝑖,𝑑 by replacing the components between the target vector π‘₯𝑖,𝑑 and the mutation vector 𝑣𝑖,𝑑

18 19

with probability. Binomial crossover, which is the most frequently employed crossover operator in the DE used as follows: 𝑗

𝑣𝑖,𝑑 𝑖𝑓 π‘Ÿπ‘Žπ‘›π‘‘[0,1) ≀ 𝐢𝑅 π‘œπ‘Ÿ 𝑗 = 𝑗 𝑖 π‘Ÿπ‘Žπ‘›π‘‘ ={ 𝑗 π‘œπ‘‘β„Žπ‘’π‘Ÿπ‘€π‘–π‘ π‘’ π‘₯𝑖,𝑑

urn a

𝑗 𝑒𝑖,𝑑

lP

re-

(𝑑)

(5)

where π‘—π‘Ÿπ‘Žπ‘›π‘‘ is a decision variable. Selection Operation: The selection operation based on the greedy algorithm chooses the survived vector from trail and target vectors, and can be represented as 𝑒𝑖,𝑑 𝑖𝑓 𝑓(𝑒𝑖,𝑑 ) ≀ 𝑓(π‘₯𝑖,𝑑 ) π‘₯𝑖,𝑑+1 = {π‘₯ (6) 𝑖,𝑑 π‘œπ‘‘β„Žπ‘’π‘Ÿπ‘€π‘–π‘ π‘’

23 24

The selection operation in the classical DE compares each target vector π‘₯𝑖,𝑑 against corresponding trial vector 𝑒𝑖,𝑑 based on the objective function, and keeping the evolution

25

continuing.

26

2.2 Constraint strategies

27 28 29

To solve COPs using EAs, the constraint-handling strategies are very important. A constrained optimization problem is usually represented as a nonlinear programming of the following form for the minimization problem [54] as

Jo

20 21 22

5 / 31

Journal Pre-proof Minimize:

𝑓(π‘₯), π‘₯ = (π‘₯1 , π‘₯2 , … , π‘₯𝑛 ) π‘Žπ‘›π‘‘ π‘₯ ∈ πœ“ 𝑔𝑖 (π‘₯) ≀ 0, 𝑖 = 1, … , π‘ž Subject to: β„Žπ‘— (π‘₯) = 0, 𝑗 = π‘ž + 1, … , π‘š

(7)

where πœ“ is the design space, 𝑔𝑖 (π‘₯) is the inequality constraint, and β„Žπ‘— (π‘₯) is the equality

2 3 4 5 6 7

constraint. The number of inequality constraints is q, (m-q) is the number of the equality constraints. Generally, if the global optimum satisfies the constraints 𝑔𝑖 (π‘₯) = 0, the inequality constraints are called active constraints. Similarly, Active constraints contain all equality constraints. For convenience, equality constraints can be transformed into the form of inequality constraints. The inequality constraints and equality constraints can be integrated into a combined form as 𝐺𝑖 (π‘₯) = {

pro of

1

π‘šπ‘Žπ‘₯{𝑔𝑖 (π‘₯), 0} 𝑖 = 1, … , π‘ž π‘šπ‘Žπ‘₯{|β„Žπ‘– (π‘₯)| βˆ’ 𝛿, 0} 𝑖 = π‘ž + 1, … , π‘š

(8)

8 9

where 𝛿 is the tolerance parameter which is usually set as 1e-4. The overall constraint violation for a solution can be expressed as

10 11 12 13 14 15 16 17 18 19 20 21 22 23

βˆ‘π‘š 𝑖=1 𝐺𝑖 (π‘₯) (9) π‘š where 𝑣(π‘₯) is zero for the feasible solution and positive when at least one constraint is violated [55], and the overall constraint violation is an important parameter in order to guide the evolutionary process towards feasible areas. In this study, three popular constraint handling strategies should be introduced as follows Superiority of Feasible Solution (SF): Two individual π‘₯1 and π‘₯2 are compared in SF. π‘₯1 is assumed to be superior to π‘₯2 if the following conditions are satisfied: 1. π‘₯1 is a feasible solution but π‘₯2 is an unfeasible solution. 2. Both π‘₯1 and π‘₯2 are feasible solutions and the objective function value of π‘₯1 is smaller than π‘₯2 . 3. Both π‘₯1 and π‘₯2 are infeasible solutions, and the overall constraint violation 𝑣(π‘₯) of π‘₯1 is smaller than π‘₯2 . Ξ΅-Constraint (EC): The method of constraint violation and Ξ΅ level comparisons was proposed in [21]. The essential of Ξ΅-Constraint handling strategy is the tradeoff between the constraint violation and the objective function. The parameter Ξ΅ was employed to adjust the balance,

lP

urn a

26 27 28

and the constraint violation πœƒ(π‘₯) was defined as a form based on the maximum of all constraint violations. πœƒ(π‘₯) = π‘šπ‘Žπ‘₯{π‘šπ‘Žπ‘₯{0, 𝑔1 (π‘₯), … , π‘”π‘ž (π‘₯)}, π‘šπ‘Žπ‘₯|β„Žπ‘ž+1 (π‘₯), … , β„Žπ‘š (π‘₯)|} (10) In constrained optimization problems, an individual is assumed as infeasible solution and its precedence is low if the constraint violation is greater than 0, This precedence is controlled by the parameter Ξ΅ . When the target and trail vectors are compared based on Ξ΅ -Constraint

29 30

method, a set of objective functions and constraint violations (𝑓(π‘₯), πœƒ(π‘₯)) will be evaluated according to the Ξ΅ -level precedence. Similar to previous discussion, π‘₯π‘Ž is considered as

31

superior to π‘₯𝑏 when the following criteria are satisfied

Jo

24 25

re-

𝑣(π‘₯) =

(𝑓(π‘₯π‘Ž ), πœƒ(π‘₯π‘Ž )) β‰€πœ€ (𝑓(π‘₯𝑏 ), πœƒ(π‘₯𝑏 ))

𝑓(π‘₯π‘Ž ) ≀ 𝑓(π‘₯𝑏 ), 𝑖𝑓 πœƒ(π‘₯π‘Ž ), πœƒ(π‘₯𝑏 ) ≀ πœ€ {𝑓(π‘₯π‘Ž ) ≀ 𝑓(π‘₯𝑏 ), 𝑖𝑓 πœƒ(π‘₯π‘Ž ) = πœƒ(π‘₯𝑏 ) πœƒ(π‘₯π‘Ž ) < πœƒ(π‘₯𝑏 ), π‘œπ‘‘β„Žπ‘’π‘Ÿπ‘€π‘–π‘ π‘’ 32

(11)

The β‰€πœ€ between the individual π‘₯π‘Ž and π‘₯𝑏 means π‘₯π‘Ž is superior to π‘₯𝑏 according the Ξ΅6 / 31

Journal Pre-proof level precedence. When Ρ is set as ∞, the value of objective function is the only criterion for determining superiority of two individuals. The property of the Ρ-Constraint method is the flexibility. The parameter Ρ can be modified according to the characteristics of optimization

4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

problem in order to obtain the suitable balance between lower values of objective function value and constraint violation. However, the suitable balance point is difficult to be determined for different COPs. Self-Adaptive Penalty (SP): Penalty function method keeps the efficiency even after achieving enough feasible individuals with simple structure [56]. Static penalty function method is classical in COPs due to its simplicity. Many methods based on penalty functions provide competitive results [57]. However, a major drawback of static penalty function method is difficult to determine the suitable penalty coefficients. Generally, the penalty coefficients are commonly case-dependent. Therefore, adaptive penalty functions [58] have been proposed in order to overcome this bottleneck. The penalty coefficients can be determined adaptively according to the previous iterations. In constrained optimizations, it is important to extract useful information from the infeasible individual whose worth is low but can guide the evolutionary path to the feasible region. The main difference is their precedence of distinct types of infeasible individuals [59]. An adaptive penalty function method based on distance value and two penalties was proposed [59]. The amount of added penalties are affected by the feasible ratio of population individuals. A greater penalty will be added to infeasible individuals if the feasible ratio is low. In contrast, if there are enough feasible individuals, then the infeasible individuals with high-fitness values will obtain relatively small penalties. It means that the algorithm can change the status between seeking the optimal solution and finding feasible solutions without presetting parameters. The details of this method are represented as follows 𝐹(π‘₯) = 𝑑(π‘₯) + 𝑝(π‘₯) (12) 𝑣(π‘₯), 𝑖𝑓 π‘π‘“π‘’π‘Žπ‘  = 0 𝑑(π‘₯) = { β€²β€² 2 (13) 2 βˆšπ‘“ (π‘₯) + 𝑣(π‘₯) , π‘œπ‘‘β„Žπ‘’π‘Ÿπ‘€π‘–π‘ π‘’

lP

re-

pro of

1 2 3

urn a

𝑝(π‘₯) = (1 βˆ’ π‘π‘“π‘’π‘Žπ‘ )𝑀(π‘₯) + π‘π‘“π‘’π‘Žπ‘ π‘(π‘₯)

π‘π‘“π‘’π‘Žπ‘  =

π‘π‘’π‘šπ‘π‘’π‘Ÿ π‘œπ‘“ π‘“π‘’π‘Žπ‘ π‘–π‘π‘™π‘’ π‘–π‘›π‘‘π‘–π‘£π‘–π‘‘π‘’π‘Žπ‘™ π‘π‘œπ‘π‘’π‘™π‘Žπ‘‘π‘–π‘œπ‘› 𝑠𝑖𝑧𝑒

(15)

0, 𝑖𝑓 π‘π‘“π‘’π‘Žπ‘  = 0 𝑣(π‘₯), π‘œπ‘‘β„Žπ‘’π‘Ÿπ‘€π‘–π‘ π‘’

(16)

M(x) = {

𝑖𝑓 π‘₯ 𝑖𝑠 π‘Ž π‘“π‘’π‘Žπ‘ π‘–π‘π‘™π‘’ π‘ π‘œπ‘™π‘’π‘‘π‘–π‘œπ‘› 0, 𝑁(π‘₯) = { β€²β€² (π‘₯), 𝑓 𝑖𝑓 π‘₯ 𝑖𝑠 π‘Žπ‘› π‘–π‘›π‘“π‘’π‘Žπ‘ π‘–π‘π‘™π‘’ π‘ π‘œπ‘™π‘’π‘‘π‘–π‘œπ‘› 𝑓 β€²β€² (π‘₯) =

Jo

(14)

(𝑓(π‘₯) βˆ’ π‘“π‘šπ‘–π‘› ) (π‘“π‘šπ‘Žπ‘₯ βˆ’ π‘“π‘šπ‘–π‘› )

(17) (18)

25 26

where π‘π‘“π‘’π‘Žπ‘  is the feasible ratio, 𝑣(π‘₯) is the overall constraint violation, 𝑓 β€²β€² (π‘₯) is the normalized fitness value , π‘“π‘šπ‘Žπ‘₯ π‘Žπ‘›π‘‘ π‘“π‘šπ‘–π‘› are the maximum and minimum values of the

27

objective function in the present generation, 𝑝(π‘₯) is the penalty value.

28

3 Surrogate assisted adaptive DE algorithm

29

It is well known that the performance of the DE is a case-dependent with different parameter 7 / 31

Journal Pre-proof ranges when handling with different problems. Except for parameters like Cr, F, Np, mutation strategies employed in evolution are also sensitive to the performance during optimization. Actually, even for a simple problem, which has been determined its characteristics; the suitable ranges of parameters like Cr, F and Np should be updated in different stages of the DE. Therefore, the requirements for mutation strategies are also changed especially in constrained optimizations. To overcome these difficulties, several strategies have been suggested. For example, parameter adaptation method for Cr, F based on success-history has been employed in many algorithms like SaDE [11], JADE [12], SHADE [13] and JADE_sort [15]; population size controlling strategy for Np has been applied to L-SHADE [20], sTDE-dR [67]. Although these adaptive methods have successfully improved the performances, the efficiency of the DE still has room for improvement. Therefore, several adaptive strategies are suggested in this study and the framework of the suggested algorithm is presented in Fig.1. Initialize control parameters

Generate the initial four equal size subpopulations randomly and evaluate the individuals by real functions

re-

Mutation (DE/current-to-best/1) is assigned for each subpopulation

pro of

1 2 3 4 5 6 7 8 9 10 11 12 13

Initialize the subpopulations and assign the number of size of each subpopulation

Local surrogate assisted evaluation strategy

Search the neighbor populations around new one

EFs<=EFsmax

lP

Assign Cr, F to individuals in each subpopulation to create the trail vectors according subpopulation archives

NO

Terminate

Update the size of subpopulation based on pfeas and save failed individuals, update the success-memory of each subpopulation

EGO (1) KG (2) SVR (3) RBF

(1) KG-HDMR (2) RBF-HDMR (3) SVR-HDMR

Projection sampling method

Generate and update surrogate by different techniques

No

Select the best surrogate model

urn a

Parameter adaptation strategy based on success-history

Calculate size of the next generation

14 15

0.5<=R2<=1

Yes

Fig.1 Framework of the ASA-DE

3.1 Notes of Adaptive population clustering strategy

17 18 19 20 21 22

In order to control the state switch efficiently, adaptive population clustering strategy is employed for the suggested algorithm. This strategy has been widely used by several scholars. In Zhou’s work [68], the population should be classed into better and worse groups according to fitness values. In the L-SHADE44 [66], population can be divided into four subpopulations, each subpopulation employs an independent mutation strategy. 𝐺 In this study, the population 𝑀(𝐺) = [π‘₯1𝐺 , π‘₯2𝐺 , … , π‘₯𝑁𝑝 ] should be clustered into k

23 24 25

subpopulations [π‘š1𝐺 , π‘š2𝐺 , … , π‘šπ‘˜πΊ ] , π‘šπ‘˜πΊ is the kth subpopulation of the Gth generation. Each subpopulation should be assigned by DE/current-to-best/1 strategy and has independent adaptive parameter archive. The size of each subpopulation should be the same in the first

Jo

16

8 / 31

Journal Pre-proof generation and the fitness value of each population should be evaluated by the real function or simulation. Commonly, the size of each subpopulation will be updated according to the 𝑠 success rate of evolution. If the trail vector generated from individual π‘₯𝑖,𝐺 is better than its

4 5

parents in term of the fitness, the ith individual in sth subpopulation should be considered as a successful one. The size of each subpopulation is determined as 𝑆𝑒𝑠 + 𝑆𝑒0 𝑆𝑝 𝑠 = π‘˜ 𝑁𝑝 (19) βˆ‘π‘–=1(𝑆𝑒𝑖 + 𝑆𝑒0 ) 𝐺

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

where 𝑆𝑒𝑠 is the present number of the successful evolutions in sth subpopulation. 𝑆𝑒0 is a small positive constant in order to avoid zero denominator when the algorithm is close to convergence. 𝑁𝑝𝐺 is the entire population size in the Gth generation. However, according to the test, this size control strategy still has room for modification, a more suitable strategy is suggested according to exploitation and exploration and should be discussed in Section 3.2. Moreover, it should be noted, the new population except for the first iteration should be generated and evaluated by the surrogate model according to the assigned criterion. Some of new populations cannot be directly generated by the HDMR-based surrogate techniques directly due to the characteristic of the HDMR. For the HDMR-based modeling technique, the new population should be generated on the axil of hypercube. Therefore, when a new population is generated inside of the hypercube, the project sampling strategy [65] should be implemented to generate the new population on the each axil. Thus, the number of populations should be significantly improved. However, these new populations generated by the project strategy should be evaluated by the surrogate model, the computational cost should be almost same as the others. Another important issue should be considered is the number of clusters. Theoretically, the number of clusters should be determined by the complexity of each problem. However, consider the diversity of problems, the number of clusters is difficult to be obtained. Therefore, several tests have been carried out with 4~10 clusters. In this study, the test results suggested that 4-cluster is the best choices.

26

3.2 Adaptive population size based on pfeas

27 28 29 30 31 32 33 34

As mentioned previous section, the population size is an important parameter to improve the performance of algorithm. In this study, after cluttering, the size of each subpopulation should updated according to the suggested adaptive population size strategy based on pfeas. In this strategy, the evolutionary procedure can be classified into three states, exploring biased, balance search and exploiting biased states. Exploring biased state: The optimization procedure enters the exploring state before the FEs exceeds the 0.5 πΉπΈπ‘ π‘šπ‘Žπ‘₯ . The control strategy of population size is linear reduction. The population size in this stage is calculated according to the formula as follows: π‘π‘π‘šπ‘Žπ‘₯ βˆ’ π‘π‘π‘šπ‘–π‘› (20) Np(G + 1) = Round [π‘π‘π‘šπ‘Žπ‘₯ βˆ’ ( ) 𝐹𝐸𝑠] πΉπΈπ‘ π‘šπ‘Žπ‘₯

35 36 37 38

where π‘π‘π‘šπ‘–π‘› is the preset minimum population size, π‘π‘π‘šπ‘Žπ‘₯ is the initial population size. Before the FEs exceeds the πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž1 (set as 0.5πΉπΈπ‘ π‘šπ‘Žπ‘₯ , the choice of values should be discussed in numerical test section), the population is large enough to keep exploration and population diversity. The minimum of the population size is 0.5π‘π‘π‘šπ‘Žπ‘₯ . Experiments have

Jo

urn a

lP

re-

pro of

1 2 3

9 / 31

Journal Pre-proof proved that it is a suitable value choice for the algorithm when it is difficult to determine whether the constraints are loose. The balance search state: The optimization procedure will enter the balance search state if no feasible solution is found when FEs reaches πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 . In this state, the population size will keep the same size (0.5π‘π‘π‘šπ‘Žπ‘₯ ) at the end of the previous state to ensure a high-level diversity. Meanwhile, the mutation strategies will switch to highly exploiting biased state to enhance the efficiency. Once the first feasible solution is successfully found, the population size will be adaptively reduced based on the feasible rate (π‘π‘“π‘’π‘Žπ‘ ) which can be formulated as (𝑁𝑝𝑓𝑖π‘₯ βˆ’ π‘π‘π‘šπ‘–π‘› ) NP(G + 1) = Round [𝑁𝑝𝑓𝑖π‘₯ βˆ’ (𝐹𝐸𝑠 βˆ’ 𝐹𝐸𝑠𝑓𝑖π‘₯)π‘π‘“π‘’π‘Žπ‘ ] (21) (πΉπΈπ‘ π‘šπ‘Žπ‘₯ βˆ’ 𝐹𝐸𝑠𝑓𝑖π‘₯)

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

where 𝑁𝑝𝑓𝑖π‘₯ is the population size in the generation when the algorithm enter to the balance search state, 𝐹𝐸𝑠𝑓𝑖π‘₯ is the number of function evaluations when the algorithm switches to the balance search state. If pfeas is 0, the population size will be stopped to be reduced to keep the diversity of populations to keep enough diversity. When pfeas exceeds pfeasNp (set as 0.5, the choice of value should be discussed in section 4), the population size is calculated as Eq.(21). If pfeas has reached pfeasNp before the balance search state begins, the method based on the linear reduction will be employed continuously and the balance search state is not employed during this evolutionary process. Exploiting biased state: The algorithm will switch to the exploiting biased state when FEs reaches πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž3 . The population linear reduction method and an exploiting protection mechanism will be employed at this stage. The population will initialize as π‘π‘π‘šπ‘Žπ‘₯ /6 if the population size is larger than π‘π‘π‘šπ‘Žπ‘₯ /6 (pfeas remains low or no feasible solution can be found in the first two stages). The mutation strategies will switch to balance mode to enhance the effect. For clarity, the details of this strategy is presented as follows.

lP

re-

pro of

1 2 3 4 5 6 7 8

urn a

Adaptive population size based on pfeas Start 1. If 𝐸𝐹𝑠 < 0.5 Γ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ 2. The strategy chooses the exploring state and the population size is calculated as Eq. 20. 3. Else 4. ..If π‘π‘“π‘’π‘Žπ‘  > 0.5 && 𝐸𝐹𝑠 < 0.9 Γ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ 5. Keep the exploring state and the population size is calculated by Eq. 20. 6. If π‘π‘“π‘’π‘Žπ‘  = 0 && 𝐸𝐹𝑠 > 0.9 Γ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ 7. Switch to the exploiting biased state and the population size is initialized to π‘π‘π‘šπ‘Žπ‘₯ /6 and is calculated as Eq. 20.

Jo

8. Else 9. Switch to the balance search state and the population size is calculated by Eq. 21. 10. End if 11.End if

24

3.3 Local surrogate assisted evaluation strategy

25 26

The surrogate model is another important issue in this framework. In this study, surrogate models should be constructed by several kinds of modeling techniques, KG [60], SVR [61], RBF 10 / 31

Journal Pre-proof [62], high dimensional modal representation (HDMR) [63] and other surrogate assisted HDMR methods, such as KG-HDMR, RBF-HDMR and SVR-HDMR in sequential optimization as shown in Fig.1. The details of these surrogate modeling techniques can be found in [64]. In this procedure, these surrogate modeling techniques are implemented to improve the efficiency of suggested strategy instead of real function evaluations. In this procedure, R 2 should be the criterion to verify the accuracy of the constructed surrogate models based on leave-one-out cross validation (LOOCV). To save the computational cost, the samples from other subpopulation should be the test population for validation. Compared with the LOOCV based on the training samples, the result of test obviously more objective. Moreover, it also means that the additional test samples for the subpopulation can be avoided. After test, the best surrogate model should be used to generate new populations for the next generation. Compared with direct evaluation strategy, the accuracy of a new population derived from the surrogate model should be decreased. Actually, it is difficult for the popular surrogate modeling techniques to construct highly accurate model for high dimensional problems (over 50D). Therefore, in the suggested algorithm, the populations used for the following strategies should be used to generate the local surrogate model instead of global one. In the suggested

17

𝐷𝑒𝑠𝑖𝑔𝑛 π‘ π‘π‘Žπ‘π‘’β„ strategy, the range for each subdomain should be limited for 𝑛. For example, if

18

the dimension of assigned case is 30, the surrogate model should be constructed based on the

19

neighbor samples limited in

20

local surrogate model is commonly more accurate than the global one.

21

3.4 Parameter adaptation strategy based on success-history

22 23 24

The choice of parameter F and Cr is also important to the performance of suggested algorithm. In the ASA-DE, a parameter adaptation strategy based on success-history is also employed to control F and Cr [12-13, 74]. Each subpopulation has an independent archive 𝑆𝑗 (𝑗 =

25 26 27 28 29

1,2, … , π‘˜) to store suitable mean values of Cr and F (πœ‡πΆπ‘Ÿ π‘Žπ‘›π‘‘ πœ‡πΉ ). The size of archive 𝑆𝑠𝑖𝑧𝑒 is a preset constant. The values of F and Cr used by successful individuals in the current generation should be stored in temporary archives π‘‡π‘πΆπ‘Ÿ and 𝑇𝑝𝐹 to calculate the πœ‡πΆπ‘Ÿ π‘Žπ‘›π‘‘ πœ‡πΉ . The values of πœ‡πΆπ‘Ÿ,𝑙 and πœ‡πΉ,𝑙 (𝑙 = 1,2, … , 𝑆𝑠𝑖𝑧𝑒 ) are set to 0.5 at the beginning. In each generation, once archive 𝑆𝑗 is full, a random element in the archive should be replaced. The

30 31

details of mechanism are shown as follows πœ‡πΆπ‘…,𝐹 = π‘šπ‘’π‘Žπ‘›π‘ŠπΏ (𝑇𝑝𝐢𝑅,𝐹 ) 𝑖𝑓 π‘‡π‘πΆπ‘Ÿ,𝐹 β‰  βˆ… (22) where 𝑓(π‘₯π‘š ) is the objective function of the individual which employs the mth F stored in 𝑇𝑝𝐹 . 𝑓(π‘’π‘š ) is the objective of the trail vector of the individual. When the population is completely partitioned, each individual in subpopulations selects F and Cr randomly from the corresponding archive 𝑆𝑗 . The final values of Cr and F are calculated

36 37 38 39 40

re-

urn a

lP

(π΅π‘œπ‘’π‘›π‘‘π‘ˆπ‘π‘π‘’π‘Ÿ βˆ’ π΅π‘œπ‘’π‘›π‘‘πΏπ‘œπ‘€π‘’π‘Ÿ )⁄ 30 . Theoretically, the accuracy of

Jo

32 33 34 35

pro of

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

as follows

πΆπ‘Ÿ = π‘Ÿπ‘Žπ‘›π‘‘π‘›(πœ‡πΆπ‘Ÿ , 0.1) (23) 𝐹 = π‘Ÿπ‘Žπ‘›π‘‘π‘›(πœ‡πΉ , 0.1) (24) The final values of Cr and F are generated based on normal distributions. The mechanism is employed in our algorithm directly, the parameter settings can be referred to [13], [74]. 11 / 31

Journal Pre-proof 1

4 Numerical test The suggested ASA-DE is tested on 28 widely used benchmark functions from the IEEE CEC2017 benchmarks on constrained real-parameter optimization [69] on 10D, 30D, 50D and 100D. The algorithm runs 50 times for each test function with a number of function evaluations equals to 20,000𝐷 according to the guidelines of CEC2017 [69]. Because the surrogate modeling techniques are used to evaluate the fitness in the ASA-DE, the number of populations evaluated by the surrogate should be recorded. In this test, the percentage of number of surrogate evaluated populations should also be listed. The criteria employed to evaluate the performance of the ASA-DE can be found in [69] and includes feasibility rate of all runs (FR), mean violation amounts (π‘£π‘–π‘œ Μ…Μ…Μ…Μ… ) and mean of objective function value (mean). Moreover, the priority level of the evaluation criteria can be according to following order [71], FR. Μ…Μ…Μ…Μ… π‘£π‘–π‘œ and mean. In this study, three evaluation criteria are used for performance comparison. The first criterion is referenced in the guidelines of CEC2017 [69] and employed in results of all comparison algorithms. Pairwise comparisons method the sign test [70] as the second evaluation criterion is also employed to compare the ASA-DE with other state-of-the-art algorithms including the top four algorithms for constrained optimization in CEC2017 are compared with the ASA-DE. The detailed description of the evaluation criterion is showed above. The sign test [71] is used to determine the significance of the results (CAL-SHADE, L-SHADE44, SaDE, SajDE and DEbin versus ASA-DE) at the 0.05 significance. The symbols "+", " βˆ’ ", "=" are employed to indicate the ASA-DE performs significantly better (+ ), no significantly difference (= ), significantly worse (βˆ’) than the algorithms in comparison. Considering the surrogate modeling is imported for evaluating the real response of each test function, the percentage of number of samples evaluated by the surrogate model is also presented. More cheap samples are used, more efficiency ASA-DE is.

26

4.1 Parameters setting

27 28 29 30 31 32

The parameter settings are shown as follows 1. The initial population size π‘π‘π‘šπ‘Žπ‘₯ is set to 24D in 10-D problems, 19D in 30-D, 14D in 50-D and 9D in 100D problems. 2. The minimum population size π‘π‘π‘šπ‘–π‘› is set to 6. 3. The number of subpopulation is set to 4. 4. The size of archive 𝑆𝑗 (𝑗 = 1,2, … , π‘˜) is set to 6.

Jo

urn a

lP

re-

pro of

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

33 34 35 36 37

5.

πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž1 , πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 and πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž3 are respectively set 0. 08πΉπΈπ‘ π‘šπ‘Žπ‘₯ , 0.5πΉπΈπ‘ π‘šπ‘Žπ‘₯ , 0.9πΉπΈπ‘ π‘šπ‘Žπ‘₯ .

The selection of these parameters is discussed below: The number of subpopulations, the initial population size and 𝑆𝑗 are set with reference to the

38 39 40

L-SHADE44. π‘π‘π‘šπ‘–π‘› is slightly lager than the number of subpopulation. Experiments show that π‘π‘π‘šπ‘–π‘› has little effect on the performance of the algorithm and is set as 6 finally. πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž1 and πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž3 are more like a protective threshold. πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž1 should be 12 / 31

Journal Pre-proof 1 2 3 4

enough close to the beginning of the evolutionary process. In contrast, πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž3 should close to the end of the evolutionary process. Therefore, πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž1 and πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž3 are set as 0.08πΉπΈπ‘ π‘šπ‘Žπ‘₯ and 0.9πΉπΈπ‘ π‘šπ‘Žπ‘₯ , respectively. Table 1 Configuration of parameters and the initial values of the surrogate modeling Basis functions

Expression

Values

Initial correlation coefficient πœƒ

π‘š(βˆ’1/m)

Turning parameter πœŽπ‘…

3

RBF

Polyharmonic function order

3

LSSVR

Turning parameter 𝜎𝐿

2

Initial correlation coefficient πœƒπΎπ»

0.1

Lower boundary on the πœƒ

0.1

Upper boundary on the πœƒ

10

Turning parameter πœŽπ‘…π» for thin plate spline function

1

pro of

Kriging

Kriging-HDMR

SVR-HDMR

Others for HDMR

Tolerance to judge the convergence of 𝑓𝑖 (π‘₯𝑖 )

10e-4

Tolerance to judge the convergence of 𝑓𝑖𝑗 (π‘₯𝑖 , π‘₯𝑗 )

10e-2

Tolerance to judge the interaction of π‘₯𝑖 and π‘₯𝑗

10e-6

Table 2 Comparison results of variants with different πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 with selected functions πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 F3

0.1 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯

0.2 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯

0.5 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯

0.7 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯

0.9 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯

376300

476300

319459

561234

656123

1

1

0.85

0.87

0

0

9.7363e-06

8.1236e-05

4681.12

4453.81

4262.0712

4132.0782

0.26

0.32

0.3023

0.2931

0.01121

0.01306

0.01278

0.01278

-0.5622

-0.5622

-0.8736

-0.8651

-0.8551

0.6210

0.6210

1

1

1

8.2621e-10

9.4621e-11

0

0

0

478.7812

401.3732

212.5621

388.2108

461.9812

1

1

1

0.6800

0.6980

0

0

0

0.2837

0.3865

1 8.7876e-06 4781.12

F6

0.26

F11

F22

urn a

0.01121

For more clarity, the suggested algorithm with 5 different values of πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 (0.1 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ , 0.3 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ , 0.5 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ 0.7 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ and 0.9 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ ) on 30D problems according to the guidelines of CEC2017. The parameter πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 is used in adaptively population size mechanism which is designed for solving medium strict level problems. Therefore, f3, f6, f11, f22 are employed to analyze πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 . Three variant algorithms are as follows. ASA-DE-FEs1: πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 is set as 0.1 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ ASA-DE-FEs2: πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 is set as 0.3 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ ASA-DE -FEs3: πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 is set as 0.5 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ ASA-DE -FEs4: πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 is set as 0.7 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯

Jo

7 8 9 10 11 12 13 14 15 16

2 10e-6

lP

5 6

Turning parameter 𝜎𝐿𝐻

Tolerance to judge the nonlinearity of 𝑓𝑖 (π‘₯𝑖 )

re-

RBF-HDMR

13 / 31

Journal Pre-proof ASA-DE -FEs5: πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 is set as 0.9 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ According to Table 2, it can be found that when πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 is set as 0.5 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ , the performance of the algorithm is the best one. Therefore, we set πΉπΈπ‘ π‘ π‘€π‘–π‘‘π‘β„Ž2 as 0.5 βˆ— πΉπΈπ‘ π‘šπ‘Žπ‘₯ . Moreover, because several surrogate models are integrated in the suggested method, the initial values of parameters used in these methods are listed in Table.1. It means that Kriging, RBF, LSSVR, Kriging-HDMR, RBF-HDMR and SVR-HDMR should be employed to construct the surrogate model for each subpopulation and the best one of them should be selected for the next iteration.

9

4.2 Comparisons with other popular algorithms

10 11 12 13

pro of

1 2 3 4 5 6 7 8

In order to evaluate the performance of the suggested ASA-DE algorithm, the following popular algorithms including the top 4 algorithms for the constrained optimization in CEC2017 are compared with the ASA-DE.

Name

Full Name

Year

CAL-SHADE

re-

Table 3 Other DE algorithms for comparison

Adaptive constraint handling and success history differential evolution[72]

2017

L-SHADE44+IDE

SajDE SaDE DEbin

[74].

A unified differential evolution algorithm for constrained optimization problems (UDE) [18].

An Improved Self-adaptive Differential Evolution Algorithm in Single Objective Constrained Real-Parameter Optimization [55]. Self-adaptive differential evolution algorithm for constrained realparameter optimization [75]. The standard differential evolution algorithm (DEbin) [3].

2017 2017 2017 2010 2006 1997

For the comparison result with 10D, as listed in Tables 4 and 5, the ASA-DE is better than or at least as good as the other algorithms. The ASA-DE also performs better than the ranked first algorithm of the CEC2017 constrained optimization (L-SHADE-44) and performs better than the L-SHADE-44 on 15 functions, while is worse by the L-SHADE-44 on 7 functions. The ASADE is also efficient at solving f6, while the L-SHADE-44 performs worse on this one. It should be noted that most of the mean percentages of the number of surrogate evaluated populations are closed to 35% and corresponding variances are near 5%. It suggests the ASA-DE might save nearly 30% computational cost compared with the directly evaluated algorithm. For f17, f18, f19, f26, f27, f28, it is difficult for the ASA-DE to construct high accurate surrogate modeling due to its complexity even the original function has been clustered. For these problems, the variances are also smaller. Therefore, for 10D problems, the ASA-DE generally obtains the high accurate result efficiently. Moreover, because the adaptive parameter strategy is used, Μ…Μ…Μ…Μ… π‘£π‘–π‘œ also outperforms than most of other algorithms.

Jo

14 15 16 17 18 19 20 21 22 23 24 25 26 27

L-SHADE with competing strategies applied to constrained optimization

lP

UDE

SHADE44 and IDE (L) [73].

urn a

L-SHADE44

A simple framework for constrained problems with application of L-

14 / 31

Journal Pre-proof For 30D problems, as shown in Tables 6 and 7, the ASA-DE also performs better than or at least equal to the other algorithms and keep the ability to solve f6. The ASA-DE achieves competitive results that are equal or better than the L-SHADE-44 in most benchmark functions. Except f17, f18, f19, f26, f27, f28, the ASA-DE optimizes all other functions successfully. The other algorithms are also inefficient for it. Although the UDE is also effective in optimizing f6, it fails to optimize f11. As a result, the ASA-DE has an excellent constraint control capability and can follow the suitable balance point between exploration and exploitation during evolutionary process through state switching and population size adaptive mechanisms. Compared with the test result for 10D problems, the percentages of surrogate evaluated population are decreased, the mean values are over 30%. With the increasing of the dimensionality, the performance of the ASA-DE is decreased due to the accuracy of surrogate modeling techniques. However, the computational cost for each problem can be saved over 20%-30%, it demonstrates that the ASA-DE is still suitable for 30D problems. It demonstrates that the local modeling strategy becomes more important for higher dimensional problems. For 50D and 100D problems, as presented in Tables 7-11, the ASA-DE outperforms the other algorithms in most benchmark functions. The ASA-DE obtains competitive results that are equal or better than L-SHADE-44 in 28 benchmark functions. Compared with 10D and 30D problems, although the performance of the ASA-DE still prevails, the influence from the surrogate model seems to be weak. According to Tables 8 and 9, the number of samples evaluated the surrogate models decreases. For 100D problems, as presented in Tables 10 and 11, due to the limitation of existed test data, four novel algorithms are compared with the ASADE. It can be found that the result is similar to 50D, even better. Successful optimization rates for benchmark functions are presented in Table.12, it demonstrates that the suggested algorithm is better than other algorithms in this test. Moreover, it should be noted that the number of samples evaluated by surrogates should be decreased in 50D and 100D. However, compared with 50D problems, the number of samples evaluated by surrogates in 100D is not decreased significantly. It suggests that the number of clusters for different dimensions is reasonable. In our opinion, the local modeling technique plays a more important role for highly dimensional problems.

31

Table 4. Experimental results in mean, FR and Μ…Μ…Μ…Μ…Μ… 𝑉𝐼𝑂 on functions 1-28 for 10D problems

F1

F2

F3

F4

ASA-DE

CALSHADE

LSHADE44

SaDE

SajDE

DEbin

LS44+IDE

UDE

0.0000E+00

6.4300E-30

0.0000E+00

0.0000E+00

6.1800E-30

2.7200E-29

0.0000E+00

5.0300E-15

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

9.1000E-17

0.0000E+00

0.0000E+00

2.6900E-29

1.2100E-28

0.0000E+00

6.4212E-15

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.1233E+03

1.1892E+05

8.2093E+04

1.0499E+05

1.2000E-29

1.0892E+05

3.2601E+05

7.7386E+01

1.0000E+00

4.8000E-01

9.2000E-01

8.0000E-02

1.0000E+00

2.8000E-01

1.0000E+00

9.6000E-01

0.0000E+00

6.2600E-05

9.2600E-06

1.9000E-04

0.0000E+00

1.2600E-04

0.0000E+00

4.3000E-06

1.3291E+01

1.3921E+01

1.3629E+01

1.3273E+01

1.5120E+01

1.4793E+01

1.4420E+01

2.5117E+01

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

Jo

Func

urn a

lP

re-

pro of

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

15 / 31

Journal Pre-proof

F5

F6

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

5.0100E-02

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

3.1890E-01

0.0000E+00

0.0000E+00

3.1921E-01

1.2341E+00

0.0000E+00

1.6542E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.2741E+01

8.28299E+02

7.2049E+02

7.9512E+02

0.0000E+00

4.2825E+02

8.0836E+02

8.7100E+01

1.0000E+00

0.0000E+00

4.0000E-02

0.0000E+00

1.0000E+00

1.6000E-01

0.0000E+00

4.4000E-01

0.0000E+00

1.4620E-01

3.1950E-02

1.8270E-01

0.0000E+00

1.4180E-01

3.7660E-02

2.0400E-02

1.7993E+01

-6.5994E+00

2.1918E+01

-5.2247E+02

3.0619E+01

-3.4003E+01

-6.4624E+00

1.0000E+00

6.8000E-01

7.6000E-01

0.0000E+00

0.0000E+00

8.0000E-02

8.0000E-01

7.2000E-01

0.0000E+00

1.0000E-04

3.2500E-05

1.1440E-01

1.0216E+03

1.2150E-03

3.1900E-05

7.0610E-02

-1.350E-03

-1.350E-03

-1.3500E-03

-1.350E-03

-9.0457E+01

-2.9000E-04

-1.3500E-03

-1.3400E-03

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

4.7650E+05

0.0000E+00

0.0000E+00

0.0000E+00

-4.980E-03

1.0032E+00

-4.9800E-03

-4.980E-03

-6.0920E-01

9.9100E-02

-4.9800E-03

-4.9800E-03

1.0000E+00

5.2000E-01

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

4.0900E-02

0.0000E+00

0.0000E+00

2.5330E+02

0.0000E+00

0.0000E+00

0.0000E+00

-5.100E-04

-5.100E-04

-5.1000E-04

-5.100E-04

-5.9754E+01

-4.5000E-04

-5.1000E-04

-5.1000E-04

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.1100E+06

0.0000E+00

0.0000E+00

0.0000E+00

-1.5250E-01

-1.5950E-01

-1.6340E-01

8.5700E-02

-9.8610E+02

-6.4520E-02

-1.6880E-01

-5.9955E+00

1.0000E+00

8.8000E-01

1.0000E+00

6.8000E-01

0.0000E+00

8.8000E-01

1.0000E+00

0.0000E+00

0.0000E+00

1.8000E-04

0.0000E+00

1.8500E-07

4.2500E+19

7.7200E-15

0.0000E+00

3.0000E-03

3.9880E+00

4.0037E+00

3.9932E+00

3.9934E+00

1.9900E-01

3.9879E+00

3.9879E+00

3.9879E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.9005E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.8670E-01

1.9700E-31

0.0000E+00

4.7840E-01

2.5780E-02

0.0000E+00

1.1124E+01

F11

F12

F13

F14

F15

F16

F17

F18

re-

F10

lP

F9

urn a

F8

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.8570E+00

3.2745E+00

2.8391E+00

2.8478E+00

1.4200E-16

3.1894E+00

3.0000E+00

2.7362E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.5000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.2666E+01

1.4925E+01

1.4294E+01

1.2535E+01

2.2000E-15

1.4797E+01

1.1278E+01

6.7549E+00

1.0000E+00

0.0000E+00

2.8000E-01

9.2000E-01

0.0000E+00

1.0000E+00

1.0000E+00

9.2000E-01

0.0000E+00

4.7000E-03

1.8900E-04

4.3000E-06

5.0000E-01

0.0000E+00

0.0000E+00

3.7000E-04

Jo

F7

2.2418E+01

pro of

-

3.7448E+01

4.9009E+01

4.0904E+01

5.1271E+01

0.0000E+00

4.7752E+01

4.0401E+01

6.2830E+00

1.0000E+00

9.6000E-01

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

9.6000E-01

0.0000E+00

2.0900E-06

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.1100E-05

1.0521E+00

9.2130E-01

9.1933E-01

9.1520E-01

1.2800E-03

1.0273E+00

8.8240E-01

1.0460E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

4.6600E+00

5.5000E+00

5.2600E+00

4.5000E+00

2.2939E+01

4.9400E+00

5.2200E+00

5.4200E+00

3.8204E+02

7.9760E+02

3.0489E+03

7.7489E+02

8.0000E-02

6.2531E+02

3.1663E+03

2.3591E+03

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

16 / 31

Journal Pre-proof

F22

F23

F24

F25

F26

F28 +ο€―=ο€―βˆ’ +ο€―=ο€―βˆ’

1.1437E+02

8.2300E+06

1.5100E+07

0.0000E+00

2.3900E-06

1.5800E-06

0.0000E+00

-9.2521E+00

1.1200E-06

0.0000E+00

2.7200E-03

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

6.6336E+03

6.6336E+03

6.6336E+03

6.6336E+03

6.6464E+03

6.6336E+03

6.6336E+03

6.6336E+03

4.5110E-01

2.4900E-01

1.8010E-01

4.5800E-01

6.0890E-01

4.2790E-01

4.1560E-01

1.6464E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

3.9879E+00

3.9901E+00

3.9884E+00

3.9880E+00

1.0777E+01

3.9879E+00

3.9879E+00

6.2436E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

3.1860E-01

1.1011E+02

3.1890E-01

4.7840E-01

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.7089E+00

3.3359E+00

2.5320E+00

3.3042E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.3289E+01

1.0022E+01

8.1367E+00

2.0452E+01

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.1196E+02

4.6244E+01

4.6307E+01

3.7008E+01

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.2066E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.9313E+01

8.3719E+00

1.5950E-01

1.2504E+01

8.4000E-01

1.0000E+00

1.0000E+00

1.0000E+00

2.3755E+02

0.0000E+00

0.0000E+00

0.0000E+00

2.0496E+01

3.0997E+00

3.0212E+00

2.7456E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

4.0874E+04

0.0000E+00

0.0000E+00

0.0000E+00

1.4100E-16

1.1781E+01

8.7650E+00

6.0006E+00

1.0000E+00

1.0000E+00

9.6000E-01

5.0000E-01

0.0000E+00

0.0000E+00

1.5200E-04

6.8361E+01

7.1500E-15

4.2327E+01

3.7699E+01

6.3459E+00

1.0000E+00

9.2000E-01

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

8.1100E-06

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.0547E+00

9.5090E-01

1.0059E+00

1.1173E+00

2.0580E-02

1.0636E+00

9.3740E-01

1.0217E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

4.7000E+00

5.4600E+00

5.1000E+00

4.5000E+00

2.9816E+01

4.9000E+00

5.0526E+00

5.4611E+00

4.3869E+02

3.1764E+03

2.6079E+03

7.3171E+02

3.1806E+01

5.8606E+03

7.9442E+03

6.7179E+03

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.1842E+04

8.6413E+04

3.5710E+05

1.7742E+02

1.8585E+03

2.6471E+05

4.0500E+07

2.2300E+08

2.1103E+01

3.5925E+01

2.5750E+01

1.0419E+01

-8.0290E-02

2.3984E+01

1.0763E+01

9.7567E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

6.6480E+03

6.6604E+03

6.6522E+03

6.6396E+03

6.6475E+03

6.6496E+03

6.6409E+03

6.6410E+03

Criterion I Criterion II

23/2/3 18/8/2

15/6/7 7/17/4

13/8/7 11/14/3

21/0/7 19/3/6

23/1/4 15/12/1

11/9/8 NA

22/2/4 NA

Table 5. The mean percentage of number of surrogate evaluated populations for each 10D problems

Jo

1 2 3

7.4797E+00

urn a

F27

5.7405E+01

pro of

F21

1.7100E+06

re-

F20

1.1585E+02

lP

F19

1.3543E+04

Function

F1

F2

F3

F4

F5

F6

F7

Mean

38.51%

39.22%

28.21%

18.91%

35.12%

38.72%

37.32%

Variance

5.29%

4.39%

6.21%

7.53%

4.56%

5.12%

4.34%

Function

F8

F9

F10

F11

F12

F13

F14

Mean

34.51%

40.21%

32.21%

34.78%

36.82%

39.12%

33.39%

Variance

5.12%

8.39%

3.76%

4.78%

5.11%

6.18%

5.21%

Function

F15

F16

F17

F18

F19

F20

F21

Mean

33.51%

34.21%

21.2%

18.67%

15.89%

32.121%

37.21% 17 / 31

Journal Pre-proof Variance

4.62%

3.89%

2.34%

3.89%

2.76%

5.12%

6.12%

Function

F22

F23

F24

F25

F26

F27

F28

Mean

33.31%

41.22%

37.02%

37.91%

25.2%

20.12%

21.32%

Variance

6.12%

5.81%

5.78%

4.23%

1.89%

2.12%

3.23%

1 Table 6. Experimental results in mean, FR and Μ…Μ…Μ…Μ…Μ… 𝑉𝐼𝑂 on functions 1-28 for 30D problems

F3

F4

F5

F6

L-SHADE44

SaDE

SajDE

DEbin

L-S44+IDE

UDE

9.6300E-29

7.2300E-30

1.1600E-07

4.6900E-27

1.0100E-11

0.0000E+00

7.3400E-29

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.6700E-30

8.6600E-29

4.7400E-30

6.3100E-08

1.1700E-26

7.1700E-12

0.0000E+00

7.3900E-29

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

3.4280E+05

3.6912E+05

3.1728E+05

9.6069E+05

7.4200E-27

1.1600E+06

6.7000E+06

7.3254E+01

1.0000E+00

1.0000E+00

1.0000E+00

4.4000E-01

1.0000E+00

4.4000E-01

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.0000E-04

0.0000E+00

1.2000E-04

0.0000E+00

0.0000E+00

1.3573E+01

1.3833E+01

1.3573E+01

1.3573E+01

3.5820E-01

1.3667E+01

1.3854E+01

8.2422E+01

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.5020E-01

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.5950E-01

3.6500E-31

1.4700E-07

5.4837E+00

1.0157E+00

0.0000E+00

2.3200E-17

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

4.4538E+03

3.9102E+03

3.7330E+03

4.2628E+03

4.7760E-01

3.7296E+03

5.5264E+03

3.0362E+02

3.2000E-01

0.0000E+00

0.0000E+00

0.0000E+00

5.6000E-01

0.0000E+00

0.0000E+00

1.0000E+00

1.3060E-02

2.0800E-02

1.7800E-02

2.9250E-01

2.9140E-01

3.1080E-01

2.5670E-02

0.0000E+00

-9.3038E+01

-1.5046E+02

-1.0888E+02

-3.5931E+01

-1.5460E+03

2.5535E+01

-8.1088E+01

F7

F9

F10

F11

5.9813E+02

1.0000E+00

1.0000E+00

9.6000E-01

1.2000E-01

0.0000E+00

1.6000E-01

9.6000E-01

1.0000E+00

0.0000E+00

0.0000E+00

5.5100E-06

1.1000E-03

2.9055E+03

1.6800E-03

4.0600E-06

0.0000E+00

-2.800E-04

-2.8000E-04

-2.8000E-04

-1.8000E-04

-6.3808E+01

-2.1000E-04

-2.6000E-04

-2.8000E-04

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

7.8000E+06

0.0000E+00

0.0000E+00

0.0000E+00

-2.670E-03

-2.6700E-03

-2.6700E-03

-2.670E-03

-5.8710E-01

3.4250E-02

-2.6700E-03

-2.6700E-03

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.2749E+04

0.0000E+00

0.0000E+00

0.0000E+00

-1.000E-04

-1.0000E-04

-1.0000E-04

-9.9800E-05

-5.0439E+01

-1.000E-04

-9.8000E-05

-1.0000E-04

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.9500E+07

0.0000E+00

0.0000E+00

0.0000E+00

-8.5360E-01

-4.5390E-01

-8.6230E-01

1.9375E+00

-2.9687E+03

-2.0513E+01

-8.6510E-01

-2.8348E+01

1.0000E+00

8.8000E-01

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

1.0000E-19

0.0000E+00

1.1600E+22

3.4700E+59

1.5547E+00

0.0000E+00

2.0700E-02

3.9856E+00

2.1186E+01

3.9856E+00

3.9963E+00

9.9500E-01

3.9826E+00

6.0679E+00

1.8679E+01

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

urn a

F8

pro of

F2

CAL-SHADE

re-

F1

ASA-DE 5.7500E-30

lP

Func

Jo

2

F12

18 / 31

F17

F18

F19

F20

F21

F22

F23

F24

F25

F26

1.6219E+00

0.0000E+00

0.0000E+00

0.0000E+00

6.9248E+00

1.5040E+01

4.8409E+00

1.3427E+01

5.9520E+00

2.3195E+01

2.6011E+01

8.1490E+01

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.8808E+00

2.0530E+00

1.8330E+00

1.8373E+00

3.7250E-02

2.1033E+00

1.9086E+00

1.5258E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.4701E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.8693E+01

1.9447E+01

1.8818E+01

1.6305E+01

1.5321E+01

1.8567E+01

1.2912E+01

9.1420E+00

9.6000E-01

1.6000E-01

8.4000E-01

1.0000E+00

3.2300E-06

4.3000E-04

1.8600E-05

0.0000E+00

1.4652E+02

1.6072E+02

1.4853E+02

1.9434E+02

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.0019E+00

1.0141E+00

9.9840E-01

1.0221E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.5500E+01

1.5500E+01

1.5500E+01

1.5500E+01

8.2729E+02

4.8317E+03

1.5830E+03

5.8927E+02

pro of

F16

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.5366E+05

1.1100E+06

1.5794E+04

5.3500E-06

7.3300E-06

0.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

8.8000E-01

3.5785E+02

0.0000E+00

0.0000E+00

2.6200E-05

4.9700E-16

1.7851E+02

1.4301E+02

8.4193E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.5600E-03

1.0263E+00

1.0128E+00

1.0214E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

7.0390E+01

1.5500E+01

1.5500E+01

1.5500E+01

6.8000E-01

6.1701E+03

4.0096E+03

3.5682E+03

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.0528E+03

4.7288E+01

2.0400E+06

1.3300E+06

3.3600E+07

6.2900E-06

0.0000E+00

-2.7748E+01

7.8100E-06

0.0000E+00

3.6576E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.1375E+04

2.1375E+04

2.1375E+04

2.1375E+04

2.1416E+04

2.1375E+04

2.1375E+04

2.1379E+04

1.4602E+00

1.1764E+00

1.4034E+00

3.0345E+00

1.8854E+00

3.1429E+00

2.2476E+00

4.5816E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

9.6000E-01

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

3.3100E-03

0.0000E+00

0.0000E+00

0.0000E+00

2.2932E+01

1.9466E+01

1.8695E+01

9.5185E+00

8.5765E+01

1.8217E+01

2.7467E+01

1.1910E+01

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

9.2000E-01

lP

F15

0.0000E+00

urn a

F14

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

4.0666E+01

0.0000E+00

0.0000E+00

1.4800E-12

2.1354E+02

3.3571E+04

3.0641E+03

2.6707E+04

1.2136E+02

3.6693E+03

8.4401E+02

9.4147E+01

1.0000E+00

8.8000E-01

1.0000E+00

3.6000E-01

7.2000E-01

9.2000E-01

1.0000E+00

1.0000E+00

0.0000E+00

4.0420E-01

0.0000E+00

3.2106E+00

2.9194E+02

5.1750E-02

0.0000E+00

0.0000E+00

1.7837E+00

2.0591E+00

1.6560E+00

1.9966E+00

2.1052E+01

2.0072E+00

1.8564E+00

1.4349E+00

1.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.5224E+05

0.0000E+00

0.0000E+00

0.0000E+00

1.2032E+01

1.5048E+01

1.2158E+01

5.9028E+01

1.4898E+00

1.7059E+01

1.3917E+01

8.3879E+00

Jo

F13

0.0000E+00

re-

Journal Pre-proof

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

8.6524E+03

5.3150E-01

0.0000E+00

0.0000E+00

0.0000E+00

1.3936E+02

1.6644E+02

1.4143E+02

3.1894E+02

1.4772E+01

1.7712E+02

1.4062E+02

1.5959E+01

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

1.0000E+00

1.0000E+00

1.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.1520E+03

6.7950E-01

0.0000E+00

0.0000E+00

0.0000E+00

1.0062E+00

1.0176E+00

1.0056E+00

1.0264E+00

8.5720E-03

1.0277E+00

1.0123E+00

1.0194E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

1.5500E+01

1.5500E+01

1.5500E+01

1.5500E+01

5.8347E+01

1.5500E+01

1.5500E+01

1.5500E+01

19 / 31

Journal Pre-proof

F28

+ο€―=ο€―βˆ’ +ο€―=ο€―βˆ’

9.5444E+03

5.0072E+02

1.9890E+02

2.4305E+04

5.2293E+04

1.1876E+04

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

4.9800E+06

3.8300E+07

1.0000E+07

1.6451E+04

5.7098E+04

2.0800E+08

1.0800E+08

3.4100E+08

1.1185E+02

1.5971E+02

1.3606E+02

1.4627E+02

3.8200E+01

1.0807E+02

1.5285E+02

6.4323E+01

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

0.0000E+00

2.1459E+04

2.1492E+04

2.1478E+04

2.1491E+04

2.1466E+04

2.1462E+04

2.1482E+04

2.1446E+04

Criterion I Criterion II

22/3/3 16/11/1

13/5/10 4/21/3

20/1/7 20/5/3

22/0/6 20/4/4

24/1/3 21/5/2

20/2/6 NA

16/3/9 NA

Table 7. The mean percentage of number of surrogate evaluated populations for each 30D problems F1

F2

F3

F4

F5

F6

F7

Mean

32.34%

32.82%

22.31%

15.76%

32.56%

35.72%

33.87%

Variance

4.32%

3.76%

5.89%

5.23%

4.56%

4.82%

5.11%

Function

F8

F9

F10

F11

F12

F13

F14

Mean

30.12%

32.21%

30.11%

31.19%

32.82%

33.97%

29.95%

Variance

5.08%

5.91%

4.12%

4.21%

5.29%

4.18%

4.02%

Function

F15

F16

F17

F18

F19

F20

F21

Mean

31.12%

32.12%

16.23%

18.69%

15.89%

32.121%

32.21%

Variance

3.38%

3.45%

1.89%

3.05%

2.76%

5.12%

6.12%

Function

F22

F23

F24

F25

F26

F27

F28

Mean

30.12%

35.82%

35.14%

34.21%

22.1%

16.79%

18.19%

Variance

4.12%

5.12%

4.35%

4.23%

2.09%

2.91%

2.29%

4

Table 8. Experimental results in mean, FR and Μ…Μ…Μ…Μ…Μ… 𝑉𝐼𝑂 on functions 1-28 for 50D problems F1

F2

F3

F4

F5

F6

F7

F8

F9

F10 F11

ASA-DE 8.57e-29 1 0 6.97e-29 1 0 885090 1 0 13.5728 1 0 8.35e-29 1 0 7941.5900 0.4 0 -145.6200 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 -1.4199 0.92

CAL-SHADE 6.38e-28 1 0 7.49-28 1 0 725750 1 0 13.7135 1 0 0.3189 1 0 7154.11 0 0.0116 -260.388 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 1.7584 0.44

L-SHADE44 7.75e-29 1 0 7.96e-29 1 0 946043 1 0 13.5728 1 0 8.92e-29 1 0 7946.24 0 0.01264 -172.159 1 0 -0.00013 1 0 -0.00204 1 0 -4.83e-05 1 0 -1.6628 1

urn a

Func

Jo

5

re-

Function

lP

1 2 3

1.2675E+04

pro of

F27

2.3376E+03

SaDE 0.02424 1 0 0.01425 1 0 6.49e+06 0.48 7.87e-05 40.796 1 0 10.9402 1 0 7615.43 0 0.2675 -21.6109 0.04 0.00102 0.00366 0.96 4.76e-06 -0.00203 1 0 5.07e-05 1 0 48.59 0

SajDE 5.10e-15 1 0 3.58e-15 1 0 1.96e-15 1 0 2.587 0 0.7517 21.2546 1 0 4.7360 0.2 0.9332 -2563.96 0 5011.44 -14.9387 0 1.82e+07 0.7054 0 21419.9 -16.7263 0 8.33e+07 -4946.35 0

DEbin 0.1425 1 0 0.0455 1 0 5.06e+06 0.44 6.51e-05 13.5728 1 0 16.6896 1 0 7550.13 0 0.3818 -13.0956 0 0.00155 0.04298 0 0.00298 -0.00204 1 0 -3.24e-05 1 0 65.2575 0

L-S44+IDE 1.49e-08 1 0 0 1 0 2.65e+07 1 0 13.9880 1 0 0 1 0 8600.818 0 0.01525 -153.963 1 0 0.000286 1 0 -0.00168 1 0 9.12e-05 1 0 -1.9254 1

UDE 1.16e-10 1 0 3.24e-11 1 0 90.6979 1 0 158.418 1 0 14.9547 1 0 687.2246 0.96 5.85e-06 -941.8285 0.88 1.66e-05 0.0002 1 0 -0.00204 1 0 5.81e-05 1 0 -160.1565 0

20 / 31

Journal Pre-proof

F16

F17

F18

F19

F20

F21

F22

F23

F24

F25

F26

F27

F28 +ο€―=ο€―βˆ’ +ο€―=ο€―βˆ’

2.4313 4.1081 1 0 70.7797 1 0 1.6326 1 0 20.3261 1 0 302.85 1 0 1.0492 0 25.5 14321.2 0 5.46e+07 1.49e-05 0 36116.2 7.7770 1 0 57.1599 1 0 40693 0 43.6401 1.5225 1 0 17.9385 1 0 315.039 1 0 1.04887 0 25.5 48022.1 0 1.84e+09 248.88 0 36321.5 23/2/3 23/4/1

0 7.3566 1 0 73.6353 1 0 1.4611 1 0 13.6659 1 0 251.4531 1 0 1.0450 0 25.5 7340.76 0 8.96e+06 0 0 36116.2 4.5789 1 0 62.1685 1 0 7838.53 1 0 1.3742 1 0 14.9225 1 0 254.1548 1 0 1.0411 0 25.5 6.37e+04 0 2.00e+08 267.1889 0 36317.4 18/0/10 NA

0.3887 12.9012 1 0 1293.788 1 0 1.1840 1 0 11.2783 1 0 13.069 1 0 1.0480 0 25.5 20155.975 0 6.56e+08 8.8444 0 36130.22 6.8396 1 0 6.8466 1 0 3186.796 0.76 0.2074 1.1240 1 0 11.6550 1 0 27.52 1 0 1.0478 0 25.5 60198.128 0 6.52e+09 129.11 0 36258.2 16/1/11 NA

Table 9. The mean percentage of number of surrogate evaluated populations for each 50D problems Function

F1

F2

F3

F4

F5

F6

F7

Mean

30.21%

32.67%

18.32%

15.12%

28.32%

31.56%

29.99%

Variance

3.29%

3.39%

6.21%

7.53%

6.19%

4.87%

5.14%

Function

F8

F9

F10

F11

F12

F13

F14

Mean

28.32%

30.32%

28.13%

29.18%

36.82%

39.12%

33.39%

Variance

3.12%

8.39%

3.87%

4.78%

5.11%

6.18%

5.21%

Function

F15

F16

F17

F18

F19

F20

F21

Mean

29.12%

23.41%

16.12%

12.37%

10.21%

22.12%

32.18%

Variance

2.34%

5.12%

2.34%

5.89%

3.45%

5.77%

5.32%

Jo

1 2 3

2.70e+99 11.3024 0 4.5132 10.5261 0.8 1.58983 0.3148 0 2.5679 31.4785 0 2279.59 4.76e-15 1 0 0.00856 0 118.433 6.6 0 5542.67 -46.041 0 36185.7 3.2270 0.96 0.004264 235.007 0 114.908 85.3516 0.76 217.017 21.2107 0 443239 10.2124 0 277.846 51.0171 0 0.8301 0.003844 0 117.822 416.528 0 149219 91.0368 0 36294.1 24/0/4 21/2/5

pro of

F15

2.26e+50 4.0002 1 0 126.221 1 0 1.4327 1 0 20.7031 1 0 375.734 0.92 4.78e-06 1.0468 0 25.5 857.646 0 18625.7 6.52e-06 0 36116.2 6.2603 1 0 25.4192 1 0 74216.5 0 51.4983 1.5441 1 0 75.9954 0 21883.1 609.72 0 3346.36 1.0475 0 25.5 818.044 0 69432 275.074 0 36328 21/0/7 22/2/4

re-

F14

0 53.3338 1 0 26.824 1 0 1.4119 1 0 17.5615 1 0 260.124 1 0 1.0312 0 25.5 6219.43 0 4.05e+07 1.21e-05 0 36116.2 3.2117 1 0 63.3177 1 0 8051.21 0.96 0.0056 1.3257 1 0 13.9172 1 0 253.526 1 0 1.0366 0 25.5 13404.6 0 2.00e+07 269.195 0 36316.3 14/5/9 8/20/0

lP

F13

6.11e-12 27.5311 1 0 51.9073 1 0 1.5756 1 0 20.4518 0.12 0.00016 281.738 1 0 1.0416 0 25.5 9581.65 0 1.51e+07 1.51e-05 0 36116.2 2.5485 1 0 12.2582 1 0 64028.9 0.28 3.0501 1.5710 1 0 17.0588 1 0 299.896 1 0 1.0435 0 25.5 19907 0 2.33e+08 268.093 0 36324 20/3/5 18/7/3

urn a

F12

1.83e-32 51.6255 1 0 23.2901 1 0 1.4330 1 0 18.9438 0.96 2.63e-06 256.542 1 0 1.0384 0 25.5 854.438 0 134670 1.21e-05 0 36116.2 3.1344 1 0 62.1813 1 0 2997.47 1 0 1.3788 1 0 14.1685 1 0 252.584 1 0 1.0389 0 25.5 4866.84 0 1.53e+07 213.915 0 36273 Criterion I Criterion II

21 / 31

Journal Pre-proof Function

F22

F23

F24

F25

F26

F27

F28

Mean

24.89%

32.12%

29.02%

32.18%

24.21%

15.32%

15.32%

Variance

5.72%

7.81%

4.87%

6.71%

1.34%

1.87%

1.87%

1 Table 10. Experimental results in mean, FR and Μ…Μ…Μ…Μ…Μ… 𝑉𝐼𝑂 on functions 1-28 for 100D problems

F4

F5

F6

F7

F8

F9

F10

F11

F12

F13

F14

F15

F16

F17 F18

L-S44+IDE 7 1 0 3 1 0 2.25e+08 1 0 14.028 1 0 0 1 0 15533 0 1.19e-02 -335.49 1 0 1.48e-03 1 0 -1.45e-04 1 0 4.25e-04 1 0 -2.834 0.96 0 7.015 1 0 71.760 1 0 0.977 1 0 16.053 1 0 529.74 1 0 1.0980 0 50.5 10091 0

pro of

F3

L-SHADE44 1.03e-25 1 0 8.47e-26 1 0 2.73e+06 1 0 13.7132 1 0 3.28e-05 1 0 15562.2 0 0.0098 -302.689 1 0 -4.81e-05 1 0 -0.00143 1 0 -1.72e-05 1 0 3.6511 0.88 5.25e-43 32.5056 1 0 80.7003 1 0 0.9718 1 0 18.0641 1 0 534.51 1 0 1.0962 0 50.5 3436.37 0

re-

F2

CAL-SHADE 0.9777 1 0 0.3661 1 0 1.514e+07 0.16 0.024 413.582 1 0 0.8188 1 0 15222.9 1 0 -193.458 0.4 0.00436 0.04160 0 0.00124 0.5225 0.96 1.564e-05 0.00051 1 0 -13.7384 0 0.14709 23.8996 1 0 118844 0 18.7453 0.7949 1 0 30.8846 0.6 1253.21 712.859 0.12 0.0488 1.0980 0 54.2153 16640.7 0

lP

F1

ASA-DE 7.21e-26 1 0 6.12e-21 1 0 2.12e+04 1 0 13.8310 1 0 2.35e-06 1 0 1932.4500 0.4 0 -132.6200 1 0 -4.81e-05 1 0 -1.45e-04 1 0 4.25e-04 1 0 -2.2199 0.92 5.83e-21 51.6255 1 0 67.3201 1 0 0.7671 1 0 18.0641 0.96 2.12e-06 26.2421 1 0 1.0980 0 54.2153 854.438 0

urn a

Func

Jo

2

UDE 6.16e-05 1 0 5.15e-05 1 0 388.18 1 0 386.65 1 0 41.284 1 0 2185.52 1 0 -1649.1 0.8 0.07357 0.00046 1 0 -0.00068 0.52 1.35e-21 1.61e-05 1 0 -549.15 0 1.2268 6.13 1 0 16793.687 0.48 3.47 0.7927 1 0 16.3048 1 0 21.7397 1 0 1.0987 0 50.5 64364.38 0 22 / 31

Journal Pre-proof

F22

F23

F24

F25

F26

F27

F28 +ο€―=ο€―βˆ’ +ο€―=ο€―βˆ’

Function

F3

F4

F5

F6

F7

31.33%

20.22%

16.74%

23.32%

30.12%

28.12%

3.12%

3.33%

5.99%

6.92%

5.65%

4.12%

5.08%

F8

F9

F10

F11

F12

F13

F14

29.76%

29.12%

27.43%

27.89%

32.12%

37.21%

31.27%

3.23%

7.21%

4.87%

5.18%

3.99%

6.18%

5.08%

F15

F16

F17

F18

F19

F20

F21

27.31%

27.12%

20.98%

17.17%

12.12%

24.22%

30.23%

5.12%

7.12%

3.09%

5.23%

3.87%

5.21%

5.12%

Function

F22

F23

F24

F25

F26

F27

F28

Mean

26.87%

34.56%

27.78%

32.22%

27.21%

18.76%

18.12%

Variance

6.12%

6.12%

5.12%

6.12%

2.11%

2.01%

2.12%

Mean Variance Function Mean Variance Function Mean

F1

F2

28.93%

Jo

Variance

4 5

3.02e+09 38.0444 0 73030.81 15.1336 1 0 10.2589 1 0 44948.02 0 368.69 0.7842 1 0 17.81 1 0 102.29 1 0 1.098 0 50.5 197018.15 0 3.19e+10 298.416 0 73303.66 23/0/5 23/1/4

Table 11. The mean percentage of number of surrogate evaluated populations for each 100D problems

urn a

1 2 3

pro of

F21

2.74e+07 0 0 7.30e+04 11.480 1 0 29.544 1 0 60692 0.04 10.594 0.958 1 0 17.938 1 0 571.016 1 0 1.097 0 50.5 140923 0 1.45e+09 610.2 0 734e+04 27/1/0 24/0/4

3.34e+06 4.68e-05 0 72969.5 9.3628 1 0 31.5798 1 0 50364.5 0.04 6.4572 0.9694 1 0 17.1845 1 0 544.187 1 0 1.09616 0 50.5 36875 0 4.78e+08 584.017 0 73404.3 25/1/2 15/11/2

re-

F20

4.05e+09 8.82e-05 0 48646.3 7.4038 1 0 14.9419 1 0 263043 0 46.1771 0.8144 0.92 . 3.68e-05 22.8996 0.08 234.413 724.212 0.24 0.02953 1.101 0 52.6093 47247.7 0 1.37e+09 415.526 0 48880.7 25/1/2 19/6/3

lP

F19

134670 1.12e-05 0 32116.2 2.2314 1 0 21.1813 1 0 3345.47 1 0 0.7981 1 0 13.1123 1 0 252.584 1 0 1.0389 0 50.5 4982.1289 0 1.24e+07 187.6720 0 38982.2 Criterion I Criterion II

Table 12 Successful optimization rate for benchmark functions Dimension

10D

Algorithm

ASA-DE

CAL-SHADE

L-SHADE44

SaDE

SajDE

DEbin

L-S44+IDE

UDE

SR

78.57%

71.42%

71.42%

64.29%

35.71%

67.86%

75%

75%

23 / 31

Journal Pre-proof Failure(f)

6

8

8

10

Dimension ASA-DE

CAL-SHADE

L-SHADE44

SaDE

SajDE

DEbin

L-S44+IDE

UDE

71.42%

75%

60.71%

32.14%

67.86%

75%

75%

Failure(f)

6

8

7

11

19

9

7

7

50D

Algorithm

ASA-DE

CAL-SHADE

L-SHADE44

SaDE

SajDE

DEbin

L-S44+IDE

UDE

SR

78.57%

67.86%

75%

57.14%

28.57%

60.71%

75%

75%

Failure(f)

6

9

7

12

20

11

7

7

100D ASA-DE

CAL-SHADE

L-SHADE44

SaDE

SajDE

DEbin

L-S44+IDE

UDE

SR

65.57%

45.86%

71.2%

N/A

N/A

N/A

70.21%

67.21%

Failure(f)

5

5

6

N/A

N/A

N/A

7

6

pro of

Algorithm

lP

re-

Moreover, the bar charts of statistical results of ASA-DE and 8 state-of-the-art other algorithms on 28 CEC2017 benchmark functions are presented when the dimensions are 50 and 100, where the blue and red bars represent the rank score of Friedman test and the number of functions with the best results, respectively. According to Fig.2, it can be found that the ASADE has the best overall performance and the advantages are more obvious in the 50D and 100D problems comparing with other DEs. It can be easily found that the ASA-DE is the best algorithm in terms of these criteria.

50D

Jo

urn a

10

13 14 15 16

7

78.57%

1

12

7

SR

Dimension

11

9

Algorithm

Dimension

2 3 4 5 6 7 8 9

18

30D

100D Fig.2 The rank score and the number of functions with the best results of ASA-DE on CEC2017 benchmark functions.

24 / 31

Journal Pre-proof

F4

F5

F3

pro of

F2

F6

F8

F9

F10

urn a

F11

F12

Jo

F7

lP

re-

F1

F13

F14

F15

25 / 31

Journal Pre-proof

F19

F20

F18

pro of

F17

F21

F22

F24

F25

F26

F27

F28

urn a

F23

Jo

lP

re-

F16

1 2 3 4

Fig.3 Convergence curves for 28 test function (100D) Furthermore, to investigate the convergence ratio, several representative function iteration curves with 100 variables are demonstrated in Fig. 3. In these figures, X and Y axial represent 26 / 31

Journal Pre-proof 1 2 3

the number of function evaluations (FES) and the logarithm of the function error value, respectively. According to these results, the suggested algorithm also are easier to converge for most of test functions.

4

4 Conclusions An adaptive surrogate assisted differential evolution with novel mutation strategy, called ASA-DE is suggested for constrained optimization problems. In the ASA-DE method, several kinds of surrogate-modeling techniques are integrated. Compared with other surrogate modeling assisted DE methods, more surrogate modeling techniques are used. Due the diversity of modeling performance, such strategy is easier to obtain high accurate model. Moreover, considering the curse of dimensionality, the surrogate model should be constructed based on the local region. The test results suggests that the surrogate modeling technique is still useful for the highly dimensional problems. Moreover, in order to follow the change of suitable balance point between exploration and exploitation, adaptive population size control based on pfeas and parameter adaptation strategy based on success-history are suggested. When these strategies are integrated, the performance of the optimization based on the ASA-DE can be guaranteed. According the results of comparison with other state-of-the-art algorithms, ASA-DE show a better or at least competitive performance compared to other algorithms. The experimental results also indicate that the state-switching mechanism can improve the performance of the algorithm. However, it is difficult for ASA-DE to solve f17, f18, f19, f26, f27, f28, but other algorithms also fail to search feasible solutions. Moreover, the number of benchmark functions that ASADE finds feasible solutions is the most compared to other algorithms.

23

Acknowledgment

24 25

This work has been supported by the Key Projects of the Research Foundation of Education Bureau of Hunan Province (17A224).

26

References

re-

lP

urn a

1.

Hu, W., Yao, L. G., & Hua, Z. Z. (2008). Optimization of sheet metal forming processes by adaptive response surface based on intelligent sampling method. Journal of materials processing technology, 197(1-3), 77-

Jo

27 28 29 30 31 32 33 34 35 36

pro of

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

88. 2.

Abedinia, O., Naderi, M. S., Jalili, A., & Mokhtarpour, A. (2011). A novel hybrid GA-PSO technique for optimal tuning of fuzzy controller to improve multi-machine power system stability. International Review of Electrical Engineering, 6(2).

3.

Abedinia, O., Amjady, N., Kiani, K., Shayanfar, H. A., & Ghasemi, A. (2012). Multiobjective environmental and economic dispatch using imperialist competitive algorithm. International Journal on Technical and Physical Problems of Engineering (IJTPE), (11), 63-70.

4.

Kar, A. K. (2016). Bio inspired computing–A review of algorithms and scope of applications. Expert 27 / 31

Journal Pre-proof Systems with Applications, 59, 20-32. 5.

Abedinia, O., Ghasemi, A., & Ojaroudi, N. (2016). Improved time varying inertia weight PSO for solved economic load dispatch with subsidies and wind power effects. Complexity, 21(4), 40-49.

6.

Brest, J., Zumer, V., & Maucec, M. S. (2006, July). Self-adaptive differential evolution algorithm in constrained real-parameter optimization. In Evolutionary Computation, 2006. CEC 2006. IEEE Congress on (pp. 215-222). IEEE.

7.

Hamza, N. M., Sarker, R. A., & Essam, D. L. (2013). Differential evolution with multi-constraint consensus methods for constrained optimization. Journal of Global Optimization, 57(2), 583-611. Das, S., Mullick, S. S., & Suganthan, P. N. (2016). Recent advances in differential evolution–an updated

pro of

8.

survey. Swarm and Evolutionary Computation, 27, 1-30. 9.

Storn, R., & Price, K. (1997). Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11(4), 341-359.

10.

Brest, J., Greiner, S., Boskovic, B., Mernik, M., & Zumer, V. (2006). Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE transactions on evolutionary computation, 10(6), 646-657.

11.

Qin, A. K., Huang, V. L., & Suganthan, P. N. (2009). Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE transactions on Evolutionary Computation, 13(2),

re-

398-417. 12.

Zhang, J., & Sanderson, A. C. (2009). JADE: adaptive differential evolution with optional external archive. IEEE Transactions on evolutionary computation, 13(5), 945-958.

13.

Tanabe, R., & Fukunaga, A. (2013, June). Success-history based parameter adaptation for differential evolution. In Evolutionary Computation (CEC), 2013 IEEE Congress on (pp. 71-78). IEEE. Tang, L., Dong, Y., & Liu, J. (2015). Differential evolution with an individual-dependent mechanism. IEEE

lP

14.

Transactions on Evolutionary Computation, 19(4), 560-574. 15.

Zhou, Y. Z., Yi, W. C., Gao, L., & Li, X. Y. (2017). Adaptive differential evolution with sorting crossover rate for continuous optimization problems. IEEE transactions on cybernetics, 47(9), 2742-2753.

16.

Bujok, P., Tvrdik, J., & Polakova, R. (2014, July). Differential evolution with rotation-invariant mutation

urn a

and competing-strategies adaptation. In Evolutionary Computation (CEC), 2014 IEEE Congress on (pp. 2253-2258). IEEE. 17.

Wang, Y., Cai, Z., & Zhang, Q. (2011). Differential evolution with composite trial vector generation strategies and control parameters. IEEE Transactions on Evolutionary Computation, 15(1), 55-66.

18.

Trivedi, A., Sanyal, K., Verma, P., & Srinivasan, D. (2017, June). A unified differential evolution algorithm for constrained optimization problems. In Evolutionary Computation (CEC), 2017 IEEE Congress on (pp. 1231-1238). IEEE.

19.

Zamuda, A., & Brest, J. (2012). Population reduction differential evolution with multiple mutation strategies in real world industry challenges. In Swarm and evolutionary computation (pp. 154-161).

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

Springer, Berlin, Heidelberg. 20.

Tanabe, R., & Fukunaga, A. S. (2014, July). Improving the search performance of SHADE using linear population size reduction. In Evolutionary Computation (CEC), 2014 IEEE Congress on (pp. 1658-1665). IEEE.

21.

Brest, J., & Maucec, M. S. (2011). Self-adaptive differential evolution algorithm using population size reduction and three strategies. Soft Computing, 15(11), 2157-2174.

22.

Yang, Z., Tang, K., & Yao, X. (2008, June). Self-adaptive differential evolution with neighborhood search. In Evolutionary Computation, 2008. CEC 2008. (IEEE World Congress on Computational Intelligence). 28 / 31

Journal Pre-proof IEEE Congress on (pp. 1110-1116). IEEE. 23.

Du, W., Leung, S. Y. S., Tang, Y., & Vasilakos, A. V. (2016). Differential evolution with event-triggered impulsive control. IEEE transactions on cybernetics, 47(1), 244-257.

24.

Awad, N. H., Ali, M. Z., & Suganthan, P. N. (2018). Ensemble of parameters in a sinusoidal differential evolution with niching-based population reduction. Swarm and evolutionary computation, 39, 141-156.

25.

Wang, S., Li, Y., & Yang, H. (2019). Self-adaptive mutation differential evolution algorithm based on particle swarm optimization. Applied Soft Computing, 81, 105496.

26.

Coit, D. W. , Smith, A. E. , & Tate, D. M. . (1996). Adaptive penalty methods for genetic optimization of

27.

pro of

constrained combinatorial problems. INFORMS Journal on Computing, 8(2), 173-182.

Farmani, R. , & Wright, J. A. . (2003). Self-adaptive fitness formulation for constrained optimization. Evolutionary Computation IEEE Transactions on, 7(5), 445-455.

28.

Deb, K. (2000). An efficient constraint handling method for genetic algorithms. Computer methods in applied mechanics and engineering, 186(2-4), 311-338.

29.

Yao, T. P. R. X. . (2000). Stochastic ranking for constrained evolutionary optimization. Evolutionary Computation IEEE Transactions on, 4(3), 284-294.

30.

Mezuramontes, E , and C. A. C. Coello . "A simple multimembered evolution strategy to solve constrained optimization problems." IEEE Transactions on Evolutionary Computation 9.1(2005):1-17. Takahama, T. , & Sakai, S. (2005). Constrained optimization by applying the Ξ± constrained method to the

re-

31.

nonlinear simplex method with mutations. IEEE Press. 32.

Coello, Carlos A. Coello.(2000) "Treating constraints as objectives for single-objective evolutionary optimization." Engineering Optimization, 32(3),34.

33.

Coello, C. A. C. , & Efren Mezura Montes. (2002). Constraint-handling in genetic algorithms through the

34.

lP

use of dominance-based tournament selection. Advanced Engineering Informatics, 16(3), 193-203. Ray, T. , & Liew, K. M. . (2003). Society and civilization: an optimization algorithm based on the simulation of social behavior. IEEE Transactions on Evolutionary Computation, 7(4), 386-396. 35.

Cai, Z., & Wang, Y. (2006). A multiobjective optimization-based evolutionary algorithm for constrained optimization. IEEE Transactions on evolutionary computation, 10(6), 658-675. Wang, Y., Cai, Z., Guo, G., & Zhou, Y. (2007). Multiobjective optimization and hybrid evolutionary

urn a

36.

algorithm to solve constrained optimization problems. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 37(3), 560-575. 37.

Wang, Y., Cai, Z., Zhou, Y., & Zeng, W. (2008). An adaptive tradeoff model for constrained evolutionary optimization. IEEE Transactions on Evolutionary Computation, 12(1), 80-92.

38.

Wang, Y., & Cai, Z. (2011). Constrained evolutionary optimization by means of (ΞΌ+ Ξ»)-differential evolution and improved adaptive trade-off model. Evolutionary Computation, 19(2), 249-285.

39.

Wang, Y., & Cai, Z. (2012). Combining multiobjective optimization with differential evolution to solve constrained optimization problems. IEEE Transactions on Evolutionary Computation, 16(1), 117-134.

40.

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

Mohamed, A. W., & Sabry, H. Z. (2012). Constrained optimization based on modified differential evolution algorithm. Information Sciences, 194, 171-208.

41.

Hernandez, S., Leguizamon, G., & Mezura-Montes, E. (2013). Hybridization of differential evolution using hill climbing to solve constrained optimization problems. Inteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial, 16(52).

42.

Sarker, R. A., Elsayed, S. M., & Ray, T. (2014). Differential evolution with dynamic parameters selection for optimization problems. IEEE Transactions on Evolutionary Computation, 18(5), 689-707.

43.

Zhang, W., Yen, G. G., & He, Z. (2014). Constrained optimization via artificial immune system. IEEE 29 / 31

Journal Pre-proof Transactions on Cybernetics, 44(2), 185-198. 44.

Ong, Y. S., Nair, P. B., & Keane, A. J. (2003). Evolutionary optimization of computationally expensive problems via surrogate modeling. AIAA journal, 41(4), 687-696.

45.

Jin, Y. (2011). Surrogate-assisted evolutionary computation: Recent advances and future challenges. Swarm and Evolutionary Computation, 1(2), 61-70.

46.

Jin, Y. (2005). A comprehensive survey of fitness approximation in evolutionary computation. Soft computing, 9(1), 3-12.

47.

Shi, L., & Rasheed, K. (2008, July). ASAGA: an adaptive surrogate-assisted genetic algorithm. In

pro of

Proceedings of the 10th annual conference on Genetic and evolutionary computation (pp. 1049-1056). ACM. 48.

Gao, W., Yen, G. G., & Liu, S. (2014). A cluster-based differential evolution with self-adaptive strategy for multimodal optimization. IEEE transactions on cybernetics, 44(8), 1314-1327.

49.

Miranda-Varela, M. E., & Mezura-Montes, E. (2016). Surrogate-assisted differential evolution with an adaptive evolution control based on feasibility to solve constrained optimization problems. In Proceedings of Fifth International Conference on Soft Computing for Problem Solving (pp. 809-822). Springer, Singapore.

Li, E., & Wang, H. (2016). Bi-direction multi-surrogate assisted global optimization. Engineering Computations, 33(3), 646-666.

51.

re-

50.

Awad, N. H., Ali, M. Z., Mallipeddi, R., & Suganthan, P. N. (2018). An improved differential evolution algorithm using efficient adapted surrogate model for numerical op51timization. Information Sciences, 451, 326-347.

52.

Taran, N., Ionel, D. M., & Dorrell, D. G. (2018). Two-level surrogate-assisted differential evolution multi-

53.

lP

objective optimization of electric machines using 3-D FEA. IEEE Transactions on Magnetics, 54(11), 1-5. Cai, X., Gao, L., Li, X., & Qiu, H. (2019). Surrogate-guided differential evolution algorithm for high dimensional expensive problems. Swarm and Evolutionary Computation, 48, 288-311. 54.

Elsayed, S. M., Sarker, R. A., & Essam, D. L. (2013). An improved self-adaptive differential evolution algorithm for optimization problems. IEEE Transactions on Industrial Informatics, 9(1), 89-99. Brest, J., Boskovic, B., & Zumer, V. (2010, July). An improved self-adaptive differential evolution algorithm

urn a

55.

in single objective constrained real-parameter optimization. In Evolutionary Computation (CEC), 2010 IEEE Congress on (pp. 1-8). IEEE. 56.

Mallipeddi, R., & Suganthan, P. N. (2010). Ensemble of constraint handling techniques. IEEE Transactions on Evolutionary Computation, 14(4), 561-579.

57.

Mezura-Montes, E., & Coello, C. A. C. (2011). Constraint-handling in nature-inspired numerical optimization: past, present and future. Swarm and Evolutionary Computation, 1(4), 173-194.

58.

Farmani, R., & Wright, J. A. (2003). Self-adaptive fitness formulation for constrained optimization. IEEE Transactions on Evolutionary Computation, 7(5), 445-455.

59.

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

Tessema, B., & Yen, G. G. (2006, July). A self adaptive penalty function based algorithm for constrained optimization. In Evolutionary Computation, 2006. CEC 2006. IEEE Congress on (pp. 246-253). IEEE.

60.

Kleijnen, J. P. (2009). Kriging metamodeling in simulation: A review. European journal of operational research, 192(3), 707-716.

61.

Clarke, S. M., Griebsch, J. H., & Simpson, T. W. (2005). Analysis of support vector regression for approximation of complex engineering analyses. Journal of mechanical design, 127(6), 1077-1087.

62.

Regis, R. G., & Shoemaker, C. A. (2005). Constrained global optimization of expensive black box functions using radial basis functions. Journal of Global optimization, 31(1), 153-171. 30 / 31

Journal Pre-proof 63.

Wang, H., Tang, L., & Li, G. Y. (2011). Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Systems with Applications, 38(11), 14117-14126.

64.

Chen, L., Wang, H., Ye, F., & Hu, W. (2018). Comparative study of HDMRs and other popular metamodeling techniques for high dimensional problems. Structural and Multidisciplinary Optimization, 1-22.

65.

Wang, H., Tang, L., & Li, G. Y. (2011). Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Systems with Applications, 38(11), 14117-14126.

66.

Piotrowski, A. P. (2013). Adaptive memetic differential evolution with global and local neighborhoodbased mutation operators. Information Sciences, 241, 164-194. Ali, M. Z., Awad, N. H., Suganthan, P. N., & Reynolds, R. G. (2017). An adaptive multipopulation differential

pro of

67.

evolution with dynamic population reduction. IEEE transactions on cybernetics, 47(9), 2768-2779. 68.

Zhou, Y., Li, X., & Gao, L. (2013). A differential evolution algorithm with intersect mutation operator. Applied Soft Computing, 13(1), 390-401.

69.

Wu, G., Mallipeddi, R., & Suganthan, P. N. (2017). Problem definitions and evaluation criteria for the cec 2017 competition on constrained real-parameter optimization. National University of Defense Technology, Changsha, Hunan, PR China and Kyungpook National University, Daegu, South Korea and Nanyang Technological University, Singapore, Technical Report.

70.

Yang, M., Cai, Z., Li, C., & Guan, J. (2013, July). An improved adaptive differential evolution algorithm with computation (pp. 145-152). ACM.

71.

re-

population adaptation. In Proceedings of the 15th annual conference on Genetic and evolutionary Derrac, J., GarcΓ­a, S., Molina, D., & Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation, 1(1), 3-18.

Zamuda, Ales. Adaptive constraint handling and success history differential evolution for CEC 2017

lP

72.

constrained real-parameter optimization. Evolutionary Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. 73.

TvrdΓ­k, Josef, and Radka Polakova. A simple framework for constrained problems with application of LSHADE44 and IDE. Evolutionary Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. Polakova, Radka. L-SHADE with competing strategies applied to constrained optimization. Evolutionary

urn a

74.

Computation (CEC), 2017 IEEE Congress on. IEEE, 2017. 75.

Brest, Janez, Viljem Zumer, and Mirjam Sepesy Maucec. Self-adaptive differential evolution algorithm in constrained real-parameter optimization. Evolutionary Computation, 2006. CEC 2006. IEEE Congress on. IEEE, 2006.

76.

Suykens, J. A., Van Gestel, T., & De Brabanter, J. (2002). Least squares support vector machines. World Scientific.

Jo

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

31 / 31

Journal Pre-proof

Highlights An adaptive surrogate assisted DE algorithm is suggested;

2.

Several kinds of surrogate modeling techniques are integrated in suggested algorithm;

3.

An adaptive population size strategy based on pfeas is suggested;

4.

Parameter adaptation strategy based on success-history is suggested;

5.

The suggested algorithm is evaluated by the benchmark problems from CEC2017.

Jo

urn a

lP

re-

pro of

1.

Journal Pre-proof

Conflict of interest statement

pro of

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled β€œAn adaptive surrogate assisted differential evolutionary algorithm for high dimensional constrained problems”

Jo

urn a

lP

re-

Authors Enying Li Zheng Zhou College of Mechanical & Electrical Engineering, Central South University of Forestry and Technology, Changsha, 41004, P.R. China