An improved grasshopper optimization algorithm with application to financial stress prediction

Accepted Manuscript

An Improved Grasshopper Optimization Algorithm with Application to Financial Stress Prediction Jie Luo , Huiling Chen , Qian zhang , Yueting Xu , Hui Huang , Xuehua Zhao PII: DOI: Reference:

S0307-904X(18)30362-7 https://doi.org/10.1016/j.apm.2018.07.044 APM 12397

To appear in:

Applied Mathematical Modelling

Received date: Revised date: Accepted date:

12 March 2018 6 July 2018 24 July 2018

Please cite this article as: Jie Luo , Huiling Chen , Qian zhang , Yueting Xu , Hui Huang , Xuehua Zhao , An Improved Grasshopper Optimization Algorithm with Application to Financial Stress Prediction, Applied Mathematical Modelling (2018), doi: https://doi.org/10.1016/j.apm.2018.07.044

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

An Improved Grasshopper Optimization Algorithm with Application to Financial Stress Prediction Jie Luo1, Huiling Chen1*, Qian zhang1, Yueting Xu1, Hui Huang1, Xuehua Zhao2 1

(Department of Computer Science, Wenzhou University, Wenzhou, Zhejiang, 325035, China) (School of Digital Media, Shenzhen Institute of Information Technology, Shenzhen 518172, China)

*Corresponding author: [email protected]

CR IP T

2

AN US

Highlight 1. This paper proposes an improved grasshopper optimization strategy (GOA) for continuous optimization 2. We have applied the improved GOA successfully to the financial stress prediction problem 3. Levy flight, Gaussian mutation and opposition-based learning are embedded in GOA 4. The experimental results reveal the improved performance of the proposed algorithm

M

Abstract This study proposed an improved grasshopper optimization algorithm (GOA) for continuous optimization and applied it successfully to the financial stress prediction problem. GOA is a

ED

recently proposed metaheuristic algorithm inspired by the swarming behavior of grasshoppers. This algorithm is proved to be efficient in solving global unconstrained and constrained

PT

optimization problems. However, the original GOA has some drawbacks, such as easy to fall into local optimum and slow convergence speed. To overcome these shortcomings, an improved GOA which combines three strategies to achieve a more suitable balance between exploitation and

CE

exploration was established. Firstly, Gaussian mutation is employed to increase population diversity, which can make GOA has stronger local search ability. Then, Levy-flight strategy was

AC

adopted to enhance the randomness of the search agent's movement, which can make GOA have a stronger global exploration capability. Furthermore, opposition-based learning was introduced into GOA for more efficient search solution space. Based on the improved GOA, an effective kernel extreme learning machine model was developed for financial stress prediction. As the experimental results show, the three strategies can significantly boost the performance of GOA and the proposed learning scheme can guarantee a more stable kernel extreme learning machine model with higher predictive performance compared to others. Keywords: Kernel extreme learning machine; Grasshopper optimization algorithm; Parameter optimization; Opposition-based learning; Levy-flight; Gaussian mutation 1

ACCEPTED MANUSCRIPT

1 Introduction Due to the gradient-free stochastic operators, flexibility, and local optima escaping capabilities of metaheuristic algorithms, a large number of metaheuristic algorithms have been proposed by researchers in the past decade. For classic metaheuristic algorithms, such as genetic algorithm (GA)[1], ant colony optimization (ACO)[2] and particle swarm optimization (PSO)[3] are used to solve global optimal problems and achieve certain successes[4–6]. In recent years, many new

CR IP T

nature-inspired optimization algorithms have been proposed, such as flower pollination algorithm (FPA)[7] which is inspired by the pollination process of flowers, bat algorithm (BA)[8] which is inspired by the echo location behavior of bats, moth-flame optimization (MFO) [9] algorithm which is developed based on the navigation method of moths in nature. According to the behavior of grasshopper swarms in nature, Saremi et al. proposed a novel swarm intelligence algorithm called grasshopper optimization algorithm (GOA)[10]. This algorithm is proved to be efficient in

AN US

solving global unconstrained and constrained optimization problems.

GOA has been applied to various fields for its simple implementation and efficiency. In 2017, Lukasik et al.[11] has applied the GOA to the data clustering and the experimental results show that this method can obtain higher accuracy compared with the standard K-means procedure. Tharwat et al.[12] proposed a modified multi-objective GOA (MOGOA) to optimize multi-objective problems and the obtained results show that the MOGOA is able to provide

M

competitive results and outperform other counterparts. Mirjalili et al.[13] also developed the basic multi-objective GOA to optimize a set of diverse standard multi-objective test problems and the

ED

result show that the proposed algorithm is able to provide very competitive results. Barman et al.[14] also developed a hybrid approach, which is using GOA to tune SVM parameters. The experimental results show that the proposed model outperforms the other hybrid models.

PT

El-Fergany et al.[15] has applied the GOA to optimize the values of unknown parameters of proton exchange membrane fuel cells stack. The simulation results have shown that the proposed

CE

approach was viable and effective. Mafarja et al.[16] employed GOA as a search strategy to design a wrapper-based feature selection method. The experimental results demonstrate the superiority of the proposed approaches when compared to other similar methods. However, the

AC

original GOA has limitations in solving some practical problems. In order to improve the performance of the original GOA in solving specific problems, researchers have proposed some improvement strategies for GOA. In 2017, Wu et al. [17] proposed adaptive GOA (AGOA) for trajectory optimization of the solar-powered unmanned aerial vehicle cooperative target tracking in urban environment. In AGOA, some improvement measures were introduced into the GOA, such as the natural selection strategy, the democratic decision-making mechanism, and the dynamic feedback mechanism. The proposed method was proven to be very effective for tackling the specific problems. Arora et al. [18] introduced chaos theory into GOA to accelerate its global convergence speed. The results have shown that the chaotic maps were able to significantly boost 2

ACCEPTED MANUSCRIPT the performance of GOA. Like many metaheuristic algorithms, GOA is also faced with the problem of being trapped in local optima and slow convergence. These disadvantages limit the wider application of GOA. In this paper, an improved variant of GOA called IGOA is proposed, which introduced three strategies into the original GOA to improve its effectiveness. Firstly, Gaussian mutation (GM)[19] is employed to increase population diversity. Gaussian mutation is more likely to create a new offspring near the original parent because of its narrow tail. Then, Levy-flight (LF)[20] strategy is employed to improve the randomness, stochastic behavior, and exploration capability of original

CR IP T

GOA. Since Levy is a well-regarded class of stochastic non-Gaussian walks, search agents have more probability to jump out of local optima. Furthermore, opposition-based learning (OBL) [21] was introduced into GOA for more efficient search solution space. In order to validate effectiveness of the proposed method, we have evaluated the performance of IGOA on classical benchmark functions including unimodal and multimodal functions in comparison with the

AN US

state-of-the-art methods such as BA, dragonfly algorithm (DA)[22], differential evolution (DE)[23], FPA, MFO, PSO, sine cosine algorithm (SCA)[24] and salp swarm algorithm (SSA)[25]. The experimental results have shown that the three improvement strategies are able to significantly boost the performance of GOA.

Based on the IGOA, an effective hybrid kernel extreme learning machine (KELM)[26] model for financial stress prediction is established, called IGOA-KELM approach. KELM integrates the

M

kernel function into the extreme learning machine [27], which is a kind of single-hidden-layer feed-forward neural network. Its performance is strongly influenced by the two key parameters.

ED

The two key parameters in KELM model are kernel bandwidth γ and penalty parameter C. Parameter γ refers to the non-linear mapping from the input space to a high-dimensional feature space and C controls the trade-off between the model complexity and the fitting error

PT

minimization. Many works have proposed to use metaheuristic algorithms to find the best parameters of the KELM model. Liu et al.[28] adopted quantum genetic algorithm to optimize the

CE

parameters of the KELM. Lu et al.[29] used the PSO with active operators to obtain an optimal set of initial parameters for KELM. Wang et al.[30] proposed chaotic moth-flame optimization strategy to optimize the parameters of the KELM. Lv et al.[31] adopted the improved bacterial

AC

foraging optimization to optimize the two key parameters of KELM. In this study, the IGOA was used to optimize the two key parameters for KELM. In order to illustrate the effectiveness of the IGOA-KELM model, we have also compared the proposed method with seven metaheuristic algorithm-based KELM methods in terms of classification accuracy (ACC), Matthews Correlation Coefficients (MCC), sensitivity, and specificity. As the experimental results show, the proposed learning scheme can guarantee a more stable KELM model with higher predictive accuracy compared to other counterparts. The main contributions of this study can be summarized as follows. a) In order to achieve a better balance between exploration and exploitation for GOA, we 3

ACCEPTED MANUSCRIPT have studied the effects of seven different combinations of opposition-based learning, Levy-flight and Gaussian mutation on the performance of GOA. b) The proposed improved GOA strategy has successfully tackled the parameter setting problem of KELM. The resulting hybrid classification model has been rigorously evaluated on financial stress prediction. c) The proposed hybrid classification model delivers better classification performance and offers more stable and robust results when compared to other metaheuristic algorithms-based KELM models.

CR IP T

The remainder of this paper is as follows. Section 2 presents a brief description of the background of kernel extreme learning machine, grasshopper optimization algorithm, opposition-based Learning, Levy flight and Gaussian mutation. The improved GOA strategy and hybrid classification model are explained in detail in Section 3. The detailed experimental results of benchmark function test and specific applications of the proposed hybrid classification model are

AN US

presented in Section 4. Section 5 summarizes the conclusion and future work.

2 Material and methods

2.1 Kernel extreme learning machine (KELM)

M

KELM is a new multiclass classifier proposed by Huang et al. [26], which was developed based on the ELM classifier. ELM is a kind of single-hidden-layer feedforward neural network (SLFNN).

ED

However, unlike other neural network algorithms, ELM does not need to tune weights and biases through a time-consuming training process and the output weights are analytically determined by using Moore–Penrose generalized inverse. Therefore, ELM learns much faster than the traditional

PT

gradient-based learning algorithms but also reaches the smallest training error and the smallest norm of output weights. The output function of ELM for generalized SLFNN is 𝑓(𝐱) = ∑𝐿𝑖=1 𝛽𝑖 𝑕𝑖 (x) = 𝒉(𝐱)𝜷

CE

(1)

where 𝜷 = [𝛽1 , … , 𝛽𝐿 ] is the vector of the output weights between the output node and the hidden layer of L nodes, 𝒉(𝐱) = [𝑕1 (x), … , 𝑕𝐿 (x)] is the output vector of the hidden layer with input x.

AC

The function of the h(x) is to map the input data from the d-dimension to the L-dimension. The key to the fast speed of the ELM for solving classification problems lies in the way the output weight 𝜷 are calculated. The weight 𝜷 can be calculated according to the following formula: −1

I

𝜷 = 𝐇 T (𝐶 + 𝐇𝐇 T )

𝐓

(2)

where H is the hidden-layer output matrix and T is the target matrix, positive coefficient C is introduced for stability of the algorithm. Then, the output function of the regularized ELM is calculated. I

−1

𝑓(𝑥) = 𝒉(𝐱)𝜷 = 𝒉(𝐱)HT (𝐶 + HH T ) 4

T

(3)

ACCEPTED MANUSCRIPT Based on ELM, KELM introduce kernel functions for feature mapping. Kernel matrix can be defined as follows: 𝛀𝑲𝑬𝑳𝑴 = 𝐇𝐇 T ∶ Ω𝐾𝐸𝐿𝑀 = 𝑕(x𝑖 ) ∙ 𝑕(x𝑗 ) = 𝐾(x𝑖 , x𝑗 ) Then, the output function of ELM classifier can be presented as the following formula. −1

I

𝑓(𝑥) = 𝒉(𝐱)𝐇 T ( + 𝐇𝐇 T ) 𝐶

𝐾(𝑥, 𝑥1 ) T −1 I ⋮ 𝐓=[ ] (𝐶 + ΩKELM ) 𝐓 𝐾(𝑥, 𝑥N )

(4)

where the hidden layer feature map h(x) is replaced with K(u, v). The most commonly used kernel

CR IP T

function is a Gaussian function whose formula is as follows. 𝐾(𝑢, 𝑣) = exp(−𝛾‖𝑢 − 𝑣‖2 )

(5)

where parameter γ is used to control the width of the sample Gaussian distribution.

2.2 Grasshopper optimization algorithm (GOA)

AN US

GOA was first proposed by Saremi et al. [10] and it mimics the behavior of grasshopper swarms in nature for solving optimization problems. In GOA, the position of the grasshoppers in the swarm represents a candidate solution to a given optimization problem. Grasshoppers has a unique way of flying. According to the mathematical model proposed for this algorithm, the movement of grasshoppers are mainly influenced by three factors: social interaction, gravity force and wind advection. The position of the i-th grasshopper is denoted as Xi and represented as follows:

M

𝑋𝑖 = 𝑆𝑖 + 𝐺𝑖 + 𝐴𝑖

(6)

where Si is the social interaction, Gi is the gravity force and Ai is the wind advection.

ED

The most important factor is the social interaction, which can be calculated by the following equation:

CE

PT

̂ 𝑆𝑖 = ∑𝑁 𝑗=1 𝑠(𝑑𝑖𝑗 )𝑑𝑖𝑗

(7)

𝑗≠𝑖

𝑑𝑖𝑗 = |𝑥𝑗 − 𝑥𝑖 |

(8)

𝑑̂𝑖𝑗 = (𝑥𝑗 − 𝑥𝑖 )/𝑑𝑖𝑗

(9)

𝑠(𝑟) = 𝑓𝑒 −𝑟/𝑙 − 𝑒 −𝑟

(10)

where dij is the distance between the i-th and the j-th grasshopper, 𝑑̂𝑖𝑗 is a unit vector from the

AC

i-th grasshopper to the j-th grasshopper. The s function defines the social forces that can be adjusted by the parameters f and l. It should be noted that s function, which defines the social forces, is not able to apply strong forces between grasshoppers with large distances between them. To make it work, the distance between grasshoppers should been mapped in the interval of [1, 4]. The gravity force of grasshopper can be represented as the following equation: 𝐺𝑖 = −𝑔𝑒̂ 𝑔

(11)

where g is the gravitational constant and 𝑒̂ 𝑔 is a unity vector towards the center of earth. The wind advection of grasshopper can be calculated as follows: 𝐴𝑖 = 𝑢𝑒̂ 𝑤 5

(12)

ACCEPTED MANUSCRIPT where u is a constant drift and 𝑒̂ 𝑤 is a unity vector in the direction of wind. Then, Eq.(6) can be expanded as follows: 𝑋𝑖 = ∑𝑁 𝑗=1 𝑠(|𝑥𝑗 − 𝑥𝑖 |) 𝑗≠𝑖

𝑥𝑗 −𝑥𝑖 𝑑𝑖𝑗

− 𝑔𝑒̂ ̂ 𝑔 + 𝑢𝑒 𝑤

(13)

where N is the number of grasshoppers. In order to solve the optimization problem more effectively, some parameters are added to the mathematical model to adjust the exploration and exploitation capabilities. The influence of gravity force on the grasshopper is too weak to be ignored and assume that the wind direction (A

constructed as follows: 𝑋𝑖𝑑 = 𝑐 (∑𝑁 𝑗=1 𝑐

𝑢𝑏𝑑 −𝑙𝑏𝑑 2

𝑗≠𝑖

CR IP T

̂𝑑 . Finally, the mathematical model is component) is always towards the best solution 𝑇

𝑠(|𝑥𝑗𝑑 − 𝑥𝑖𝑑 |)

𝑥𝑗 −𝑥𝑖 𝑑𝑖𝑗

̂𝑑 )+𝑇

(14)

where ubd is the upper bound in the d-th dimension, lbd is the lower bound in the d-th dimension, ̂𝑑 is the value of the d-th dimension in the best solution found so far, and the parameter c is 𝑇 to the number of iteration:

AN US

updated with the following equation to reduce exploration and increase exploitation proportional

𝑐 = 𝑐𝑚𝑎𝑥 − 𝑙

𝑐𝑚𝑎𝑥−𝑐𝑚𝑖𝑛 𝐿

(15)

where cmax is the maximum value, cmin is the minimum value, l indicates the current iteration, and L is the maximum number of iterations.

M

The general framework of GOA is as follows:

AC

CE

PT

ED

Algorithm 1 Pseudo code of GOA Begin Initialize the swarm Xi(i=1,2,…,n); Initialize cmax, cmin and maximum number of iterations; Calculate the fitness of each search agent; T= the best search agent; while (l ≤ Max number of iterations) Update c using Eq.(15); for each search agent Normalize the distance between grasshoppers in [1,4]; Update the position of the current search agent by the Eq.(14); Bring the current search agent back if it goes outside the boundaries; end for Update T if there is a better solution l=l+1 end while return T; End

2.3 Opposition-based learning The opposition-based Learning (OBL) can be regarded as a well-regarded mathematical concept among the community of computational intelligence[32]. The original idea of OBL was initially proposed in 2005 [21]. The OBL can attain the opposite locations for candidate solutions for a 6

ACCEPTED MANUSCRIPT given task. The new location can provide a new chance to become aware of a neighboring point to the best position. The core conception of OBL optimization is, for disclosing an improved solution, simultaneously calculating and evaluating a candidate solution and related matching opposite solution, choosing the best solution as the next-generation individual. For a candidate solution 𝑋𝑖 , the related matching opposite solution 𝑋𝑖′ can be calculated according to the following formula: 𝑋𝑖′ = 𝑎 + 𝑏 − 𝑋𝑖 , 𝑋𝑖 ∈ [𝑎, 𝑏]

(16)

CR IP T

where a and b are the lower bound and the upper bound of the search space, respectively.

2.4 Levy flight

Levy-flight (LF) was originally introduced by the French mathematician in 1937 named Paul Levy. A diverse range of both natural and artificial phenomena are now being described in terms of Levy statistics[33]. The LF is a well-regarded class of stochastic non-Gaussian walks that its

AN US

step length values should be dispersed with regard to a Levy stable distribution. Levy distribution can be attained by:

𝐿𝑒𝑣𝑦(𝛽)~𝑢 = 𝑡 −1−𝛽 , 0 < 𝛽 ≤ 2

(17)

𝛽 shows an important Levy index to adjust the stability. The Levy random number can be calculated by the following formula:

𝜑×𝜇

(18)

M

𝐿𝑒𝑣𝑦(𝛽)~ |𝑣|1/𝛽

where 𝜇 and 𝑣 are both standard normal distributions, Γ is a standard Gamma function, β = 1.5,

ED

and φ is defined as follows:

1/𝛽 𝛤(1+𝛽)×sin(𝜋×𝛽/2)

φ=[

𝛽−1 1+𝛽 )×𝛽×2 2 ) 2

]

(19)

PT

𝛤((

To obtain a trade-off between the exploration and exploitation ability of metaheuristic algorithms, LF strategy can be used to update the position of search agent, which is formulated as

CE

follows:

𝑙𝑒𝑣𝑦

AC

where 𝑋𝑖

𝑙𝑒𝑣𝑦

𝑋𝑖

= 𝑋𝑖 + 𝑟 ⊕ 𝑙𝑒𝑣𝑦(𝛽)

(20)

is the new position of the ith search agent 𝑋𝑖 after updating and r is a random

vector in [0,1], ⊕represents the dot product (entry-wise multiplications).

2.5 Gaussian mutation The Gaussian mutation (GM) operation has been derived from the Gaussian normal distribution and its application to evolutionary search was formulated by Schwefel et al.[19]. This theory was referred to as classical evolutionary programming (CEP). Gaussian mutation is more likely to create a new offspring near the original parent because of its narrow tail. Due to this, the search equation will take smaller steps allowing for every corner of the search space to be explored in a 7

ACCEPTED MANUSCRIPT much better way. Hence it is expected to provide relatively faster convergence. The Gaussian density function is given by: 𝑓𝑔𝑎𝑢𝑠𝑠𝑖𝑎𝑛(0,𝜎2) (𝛼) =

1 √2𝜋𝜎

𝛼2

𝑒 −2𝜎2 2

(22)

where 𝜎 2 is the variance for each member of the population. This function is further reduced to generate a single n−dimensional random variable by setting the value of mean to zero and standard deviation to 1. The random variable generated is applied to the general equation of metaheuristic algorithm and is given as follows. 𝑋𝑖𝑑 = 𝑋𝑖 ⊕ 𝐺(𝛼)

CR IP T

(23)

where G(α) is a Gaussian step vector generated using Gaussian density function with α as a Gaussian random number between [0, 1].

AN US

3 Proposed method

3.1 Improved grasshopper optimization algorithm (IGOA) In this subsection, the improved GOA will be described in detail. In IGOA, to overcome the disadvantage of the basic GOA, three strategies including opposition-based learning, Levy flight and Gaussian mutation are introduced to GOA to keep a more suitable balance between the

M

exploration and exploitation. The basic concepts of the three strategies have been described in

ED

detail in the Section 2. The flowchart of IGOA is shown in Fig.1.

As we all know, the diversity of search agents is crucial for metaheuristics algorithms, because

PT

diversity gives the population a stronger search capability for global optimum. In IGOA, Gaussian mutation mechanism was employed to increase the diversity of GOA population. The modified

CE

mathematical model can be presented as the following formula: 𝑋𝑖𝑑 = 𝑐 (∑𝑁 𝑗=1 𝑐

𝑢𝑏𝑑 −𝑙𝑏𝑑 2

𝑗≠𝑖

𝑠(|𝑥𝑗𝑑 − 𝑥𝑖𝑑 |)

𝑥𝑗 −𝑥𝑖 𝑑𝑖𝑗

̂𝑑 ) ⊕ 𝐺(𝛼) + 𝑇

(24)

AC

After the position of the i-th grasshopper Xi is updated, Levy flight mechanism will be adopted to generate a new candidate solution, which is formulated as follows: 𝑙𝑒𝑣𝑦

X𝑖

= X 𝑖∗ + 𝑟𝑎𝑛𝑑(𝑑) ⊕ 𝑙𝑒𝑣𝑦(𝛽) 𝑙𝑒𝑣𝑦

X X 𝑡+1 ={ 𝑖∗ 𝑖 X𝑖

(25)

𝑙𝑒𝑣𝑦

𝑓𝑖𝑡𝑛𝑒𝑠𝑠(X 𝑖 ) > 𝑓𝑖𝑡𝑛𝑒𝑠𝑠(X ∗𝑖 ) 𝑜𝑡𝑕𝑒𝑟𝑤𝑖𝑠𝑒

(26)

where X ∗𝑖 is the new position of the i-th grasshopper after updating and rand(d) is a d-dimensional random vector in [0,1]. Because Levy flight is a random process in which the jump size follows the Levy probability distribution function, the new candidate solution generated by Levy flight mechanism has a high probability of jumping out of the local optimum and obtain a better solution. 8

ACCEPTED MANUSCRIPT In order to ensure the quality of the population, search agents with higher fitness will be retained in the population. Finally, the opposition-based learning strategy is used to generate the oppositional population corresponding to the current population after the positions of all search agents in the population have been updated. This mechanism helps search more effective space and enhances the overall exploration capability of the algorithm. Oppositional population can be generated by the following formula: 𝑜𝑝

𝑋𝑖

= 𝐿𝐵 + 𝑈𝐵 − 𝑇 + 𝑟(𝑇 − 𝑋𝑖 )

𝑜𝑝

(27)

CR IP T

where 𝑋𝑖 is the position of the i-th opposite grasshopper inside the search space, LB and UB are the lower bound and the upper bound of the search space respectively, T is the position of the best grasshopper, r is a random vector with elements inside (0,1), and 𝑋𝑖 is the position vector of the i-th grasshopper in population. Then, the best grasshopper is also updated based on the fitness of the opposite locations. Note that the individuals in the new population are selected from the

AN US

current population and the oppositional population according to its fitness. The opposition-based learning can boost the convergence speed of GOA by checking the suitability of a grasshopper to its opposite and holding the better one in the swarm for further improvements in the next steps. Hence, the opposition-based learning can force better grasshoppers to move toward the global optimum more quickly.

M

3.2 Proposed hybrid classification model

In this section, the hybrid KELM model based on IGOA is described in detail. In IGOA-KELM,

ED

IGOA is employed to optimize the parameters of KELM. The flowchart of the proposed IGOA-KELM is shown in Fig.2. The proposed methodology consists of two main parts. The first

PT

part involves optimizing inner parameters. The second part evaluates the outer classification performance. During the tuning of the inner parameters for KELM, the optimal parameters for the training set are dynamically adjusted by the IGOA strategy via the 5-fold cross validation (CV)

CE

analysis. Then, the obtained optimal parameters are fed into the KELM model to conduct the

AC

classification task in the outer loop via 10-fold CV analysis. The pseudo-code of the whole procedure is given below. Algorithm 2 Pseudo code of IGOA-KELM Begin Initialize the population Xi(i=1,2,…,n); Initialize cmax, cmin and L (maximum number of iterations); Evaluate the fitness of all agents by KELM with agent as parameters; T= the best search agent; while (l ≤ L) Update c according to Eq.(15); for i
ACCEPTED MANUSCRIPT

CR IP T

Bring the Xi back if it goes outside the boundaries; end for Evaluate the fitness of all agents by KELM with agent as parameters; Update T if there is a better solution; for i
The computational complexity of the proposed method IGOA-KELM depends on the number of samples (L), the number of generations (g), the population number (n) and the parameters dimensions (d). Therefore, the overall computational complexity is O(KELM, IGOA) =

AN US

g*(O(Updating the position of all search agents)+O(Evaluate the fitness of all agents)+ O(Calculate the oppositional position of all search agents and evaluate its fitness)+O(Sort search agents in population and oppositional population)). As we all know, KELM’s computational complexity on L samples is O(L3). The computational complexity of updating the position of all search agents is O(n*d). Evaluating the fitness of all agents is O(n*L3). Calculating the

M

oppositional position of all search agents and evaluate its fitness is O(n*L3). Sorting search agents in population and oppositional population is O(2n*log2n). Therefore, the final computational complexity of the proposed method is O(KELM, IGOA)≈O(g*(n*d+n*L3+n*L3+2n*log2n))=

ED

O(n*g*(d+2(L3+log2n)).

PT

4 Experimental studies In this section, the IGOA method is evaluated from various aspects using a series of

CE

experiments on benchmark functions and financial stress prediction problem. All experiments in this paper are implemented using MATLAB R2014b software under a Windows Server 2008 R2

AC

operating system with Intel (R) Xeon(R) CPU E5-2660 v3 (2.60 GHz) and 16GB of RAM.

4.1 Function optimization experiment To obtain fair results, all the implementations were conducted under the same conditions. Population size and maximum generation are set to 30 and 500 respectively. In order to decrease the influence of the randomness, we have run 30 times for every method on each function. The optimal solution for each test problem is bolded in each table. 4.1.1 The influence of GM, LF and OBL According to the Subsection 3.1, three search strategies (GM, LF and OBL) have been 10

ACCEPTED MANUSCRIPT combined with the basic GOA method. In order to investigate the influence of each strategy and its random combination, seven different IGOAs are generated and denoted as IGOA1-IGOA7 respectively. For IGOA1, it means that GOA is only combined with OBL. For IGOA2, it means that GOA is only combined with LF. For IGOA3, it is the combination of GOA, OBL and LF operators. For IGOA4, it means that GOA is only combined with GM. For IGOA5, it is the combination of GOA, OBL and GM operators. For IGOA6, it is the combination of GOA, GM and LF operators while not OBL operator. IGOA7 is the combination of GOA and the three search strategies. Their performance is benchmarked on four classical benchmark functions. These

CR IP T

benchmark functions are selected from [34]. The formulation of these problems and brief descriptions about their features can be obtained from Table 1. The four benchmark functions have different characteristics. For f1, it is a simple unimodal function with only one global minimum in the solution space. f2 is also an unimodal function, but it has an infinite number of identical global minimums. The two unimodal functions can reveal the global exploration capabilities of

AN US

optimization algorithm. f3 is a multimodal function that has only one global optimal solution but has many local minima at the same time. This requires the search agent to have a strong search ability and can jump out of the local optimum. f4 is a fixed-dimension multimodal function. It also has only one global optimal solution and many local minimum values. However, its solution space is very small, and it needs the adaptive adjustment step size of the search agent. Using these four benchmark functions can comprehensively evaluate the performance of the optimization algorithm

M

when solving different types of problems.

Table 2 presents the results obtained by various IGOAs on the four benchmark functions, where

ED

the first row of each function records the average value, the second row is the standard deviation, the third row is the elapsed time (in seconds) and the last row is the ranking. It can be seen from the table that IGOA7 is well capable of searching for the best function values for most cases. It

PT

indicates the combination of GOA, GM, LF and OBL operators leads to the grasshopper move toward the best solutions.

CE

For the original GOA, its performance is obviously worse than that of IGOA1. IGOA2 and IGOA4 combined with a new strategy, which suggests that any of these three strategies are helpful

AC

to the improvement of original GOA. For IGOA1, it did not achieve better results than the original GOA on some benchmark functions, but it was almost absent for these situations of IGOA2 and IGOA4. It indicates that the improvement effect of OBL strategy on GOA is not as significant as GM and LF. It can be also observed that the IGOA5 (GOA with GM and OBL strategies) has no advantage compared to IGOA2, which reveals that LF strategy is crucial to the improvement of GOA. Compared with other improved methods, the advantage of IGOA7 is obvious, which shows that by properly combining these three strategies, the performance of the original GOA algorithm can be significantly improved. Compared with the original GOA method, the introduction of the three strategies only adds a very small amount of time overhead, but this is negligible compared to the performance of the improvement. In the end, IGOA7 is used as the ultimate improved version 11

ACCEPTED MANUSCRIPT of GOA for its most significant performance improvement. For the sake of simplicity, IGOA is used to refer to IGAO7 in the subsequent section. 4.1.2 The scalability test for IGOA In order to compare the performance of IGOA algorithm and original GOA algorithm more comprehensively, scalability test is carried out in this subsection. Scalability test can help us to deeply measure the influence of dimension on the quality of solutions and the efficacy of optimizers. The effect of dimension on the quality of results is investigated to examine what

CR IP T

occurs for the performance of the IGOA and GOA techniques when the dimension of functions increases. For this purpose, 6 dimensions of f1-f3 functions are measured here: 10, 30, 100, 200, 500, and 1000. All conditions are the same and both algorithms utilize 30 search agent during 500 iterations. By increasing the dimension, the mean value and standard deviation values obtained by the optimizer in each fold are recorded and tabulated in Table 3. Note that the first row and second row in each dimension record the results obtained by IGOA and GOA, respectively.

AN US

From Table 3, we can see that IGOA has achieved obvious advantages compared with the original GOA in all dimensions. When dimension increase, the mean value of the minimum value also will increase. The reason is that the f1-f3 functions are more challenging to tackle in higher dimensions. In the higher dimensions, most of the results obtained by IGOA are significantly better than those obtained by GOA. In summary, through scalability test, it can be found that

M

IGOA has better scalability to deal with optimization problems with different complexity. 4.1.3 Comparisons of IGOA with other methods To verify the optimization performance of IGOA, its performance was compared with nine

ED

other optimization methods including BA, DA, DE, FPA, SSA, MFO, PSO, SCA and basic GOA on 4 optimization problems. For fair comparison, the number of populations and iterations of all

PT

algorithms is set to 30 and 500, respectively, and other parameters of all algorithms are set according to their original papers. The results obtained by the ten algorithm on benchmark functions are presented in Table 4.

CE

The Wilcoxon rank-sum test [35] at 5% significance is utilized as well to judge the meaningful improvements of the IGOA over the other optimizers. When p-values are less than 0.05, it can be

AC

determined that the results of IGOA are significantly superior to the other approach. If not, the obtained improvements are not statistically significant. Some obtained p-values are reported in Table 5. In the table, each p-value which is not lower than 0.05 is shown in bold face. It shows that the differences are not significant. From Table 4, we can see that the results obtained by IGOA on the unimodal functions f1 and f5 and the multimodal functions f11 are significantly better than those obtained by its competitors. It proves that IGOA has superior performance in solving unimodal and multimodal functions. The p-value in Table 5 indicates that this advantage is statistically significant. For fixed-dimension multimodal benchmark functions f19, we can see that all the results obtained by these algorithms are very close. This may be because the solution space of these fixed-dimensional multimode 12

ACCEPTED MANUSCRIPT functions is too small, making the three strategies embedded GOA cannot play its due role. However, in general, IGOA has achieved the best results on most of the functions, which shows that the three mechanisms have made a significant contribution in solving the general optimization problems. The convergence trends of benchmark functions depicted in Fig.3. It can be detected from the figure that the proposed IGOA has fast convergence compared with other algorithms. For f1 IGOA still maintains a strong search capability and obtains the best solution when other algorithms have fallen into a local optimum. For f2 case, the convergence speed of the IGOA is

CR IP T

significantly faster than other algorithms in the early stages, the solution eventually found is the best. For multimodal function f3, the convergence trends is very similar to that on f1, IGOA still has a strong search capability in the last stages when other algorithms have fallen into a local optimum. According to the convergence curves of fixed-dimensional multimodal function f4, it is seen that the most algorithms converge after few iterations because the solution space of f4 is very

AN US

small, but not all algorithms find a global optimal solution. The original GOA got the worst solution because it is trapped in the local optimum prematurely. However, the IGOA finds the global optimal solution with very few iterations. It reveals that the proposed enhanced strategies

AC

CE

PT

ED

M

can significantly improve the search ability of search agents on most cases.

13

AC

CE

PT

ED

M

AN US

CR IP T

ACCEPTED MANUSCRIPT

14

ACCEPTED MANUSCRIPT

4.2 Application to practical financial prediction problem In this section, in order to evaluate the performance of IGOA in optimizing real-world optimization problem, the proposed IGOA-KELM model is used for financial stress prediction. The technique used by Huang available at http://www3.ntu.edu.sg/home/egbhuang was used for the KELM classification. The data was scaled into a range of [-1, 1] before each classification was conducted. The search ranges of the penalty parameter (C) and kernel bandwidth (γ) in

CR IP T

K ( x, xi )  exp( x  xi 2 ) were defined as C∈{2-5, 2-3,…,215} and γ∈{2-15, 2-13,…,25}. In order to obtain fair results, all the implementations, such as GOA-KELM, GA-KELM, FA-KELM, PSO-KELM, DE-KELM, MFO-KELM and SCA-KELM, are conducted under the same conditions. Population size and maximum generation are set to 30 and 100 respectively. 4.2.1 Financial stress prediction

AN US

An important purpose of the corporate’s financial stress prediction is to determine whether the company has the risk of bankruptcy in the near future. This is an important and widely studied topic since it can have significant impact on bank lending decisions and profitability. Several recent and advanced techniques for predicting bankruptcy have been developed. In [37], Sharma et al. proposed a hybrid algorithm based on ant colony optimization and Nelder-Mead simplex for training neural networks with an application to bankruptcy prediction. In [38], Ravi et al. proposed

M

a modified bacterial foraging technique to train wavelet neural network in order to predict bankruptcy in banks. In [39], Wang et al. proposed an effective hybrid KELM model based on

ED

gray wolf optimization for bankruptcy prediction. The health of a bank or firm in a highly competitive business environment is dependent upon: (i) how financially solvent it is at the inception, (ii) its ability, relative flexibility and efficiency in

PT

creating cash from its continuous operations, (iii) its access to capital markets and (iv) its financial capacity and staying power when faced with unplanned cash short-falls. As a bank or firm

CE

becomes more and more insolvent, it gradually enters a danger zone. Then, changes to its operations and capital structure must be made in order to keep it solvent. For financial stress prediction, ten important financial ratios that can reflect the company’s financial status were

AC

selected for training predictive model according to the previous literatures[36]. The ten important financial ratios are shown in Table 6.

In this study, Japanese dataset were adopted to evaluate the effectiveness of the proposed IGOA-KELM, which is available at http://goo.gl/IFFzDp. The dataset was collected from some Japanese financial statements from 1995 to 2009, including 76 non-bankrupt observations and 76 bankrupt observations. Firstly, data was normalized by scaling them into the interval of [−1, 1]. In order to gain an unbiased estimate of the generalization accuracy, the 10-fold CV was used to evaluate the classification accuracy. The dataset was divided into 10 subsets. For each fold, one of 15

ACCEPTED MANUSCRIPT the 10 subsets is used as the test set and the other 9 subsets are put together to form a training set. After 10-fold CV, The final result is the average of 10 trials. 4.2.2 Measure for performance evaluation To evaluate the proposed method, commonly used evaluation criteria such as classification accuracy (ACC), sensitivity, specificity and Matthews Correlation Coefficients (MCC) were analyzed. They are defined as follows:

TP  TN 100% TP  FP  FN  TN TP Sensitivity  100% TP  FN TN Specificity  100% FP  TN ACC 

CR IP T

MCC 

(28)

TP * TN  FP * FN 100% (TP  FP)*(TP  FN )*(TN  FP)*(TN  FN )

(29) (30) (31)

AN US

where TP is the number of true positives, FN is the number of false negatives, TN is the number of true negatives, and FP is the number of false positives. 4.2.3 Result on Japanese dataset

Detailed results of 10-fold cross-validation of the IGOA-KELM model on the Japanese dataset are recorded in Table 7. As shown in the table, since the input data for each fold of the KELM

M

classifier is different, the optimal parameters set by IGOA for KELM are different on each fold. The average result obtained by the classification model on each fold is an important indicator to

ED

evaluate the performance of the classification model. As shown, IGOA-KELM model has achieved average results of 83.45% ACC, 85.97% sensitivity, 83.80% specificity, and 67.70% MCC. It can be also observed that the values of C and γ can be specified adaptively for each data

PT

fold. The explanation lies in the fact that the two parameters can be adaptively determined by the IGOA strategy based on specific distribution of the training data.

CE

Fig. 4 shows intuitively the comparison of the average results and the standard deviations of the IGOA-KELM with the other seven models on the four evaluation criteria. From the figure, we can

AC

see that the results obtained by different models have obvious differences. This is because there is a difference in the optimal parameters set by each optimization algorithm for KELM. This also reveals that the parameters of KELM have a significant impact on the classification performance of KELM. It can be seen from the figure that the results obtained by the IGOA-KELM model on the four evaluation indicators are all significantly better than the other models. Especially compared with the GOA-KELM model, the performance has been significantly improved. In terms of the most important evaluation indicator ACC, the IGOA-KELM model is much higher than that of other models. It suggests that the IGOA can always find more suitable parameters for KELM and makes the potential classification performance of KELM fulfilled. However, several algorithms including GOA, GA, FA seemed to fall into local optimums in the process of searching 16

ACCEPTED MANUSCRIPT the optimal parameters, which cannot get the appropriate parameters to achieve the satisfactory performance. In the other three evaluation indicators of Sensitivity, Specificity and MCC, the results obtained by the IGOA model are also much better than the other comparative counterparts. Fig.5 shows evolutionary curves of IGOA-KELM and the other seven models during the training stage. These curves are the average results of the 10 curves obtained by these models during the 10-fold cross validation. As shown, it is apparent that the IGOA-KELM model can quickly achieve a good performance during the model training process, which reveals that IGOA has a strong search capability and can find suitable parameters for KELM in an efficient manner.

CR IP T

Based on the analysis of the experimental results above, it is not difficult to find that the performance of the IGOA-KELM model is significantly better than the GOA-KELM model. It proves that the performance of the proposed IGOA algorithm has been significantly improved compared to the original GOA. The main reason for this is that the embedded three strategies has

AN US

enhanced a lot the search capabilities of GOA.

5 Conclusion

This paper presents an improved GOA called IGOA, which employed opposition-based learning, Levy-flight and Gaussian mutation to enhance the global and the local exploration capabilities of original GOA. First, four classical benchmark functions are used to verify the effectiveness of the

M

improved method. Experimental study of the effects of these three strategies on the performance of IGOA revealed that the Levy-flight strategy has a crucial influence on the improvement of the

ED

performance of IGOA and the performance of original GOA can be significantly improved by properly combining all three strategies. Second, compared with the well-known algorithms, such as BA, DA, DE, FPA, SSA, MFO, PSO and SCA, the IGOA can provide more competitive results

PT

on the four classical benchmark functions. Additionally, the resultant IGOA is taken to dynamically tune the optimal parameters of KELM. Experimental results on the real-life financial

CE

prediction problem has validated that the established IGOA-KELM methodology performed better than other seven advanced machine learning models in terms of the ACC, MCC, sensitivity, and specificity.

AC

Although the proposed IGOA has been proven to be effective in solving general optimization

problems, IGOA has some shortcomings that warrant further investigation. In IGOA, due to the introduction of three strategies, IGOA has increased little time complexity than the original GOA. Therefore, deploying the proposed approach to high-performance computing platform to reduce computational burden is a worthwhile direction. In the future research work, the method presented in this paper can also be extended to solving discrete optimization problems and multi-objective optimization problems. Furthermore, applying the proposed IGOA-KELM model to other fields such as medical diagnosis and face recognition is also an interesting future work.

17

ACCEPTED MANUSCRIPT

Acknowledgements

AC

CE

PT

ED

M

AN US

CR IP T

This research is supported by the National Natural Science Foundation of China (61571444, 61702376), the Science and Technology Plan Project of Wenzhou, China (ZG2017019), Zhejiang Provincial Natural Science Foundation of China (LY17F020012), Guangdong Natural Science Foundation (2016A030310072), MOE (Ministry of Education in China) Project of Humanities and Social Sciences (17YJCZH261), Special Innovation Project of Guangdong Education Department (2017GKTSCX063).

18

ACCEPTED MANUSCRIPT

References [1]

J.H. Holland, Genetic algorithms, Sci. Am. 267 (1992) 66–72. https://www.scopus.com/inward/record.uri?eid=2-s2.0-0026988817&partnerID=40&md5=430 7cdc9d8d7a487e89523df7ebad0b8.

[2]

M. Dorigo, M. Birattari, T. Stutzle, Ant colony optimization, Comput. Intell. Mag. IEEE. 1

[3]

CR IP T

(2004) 28–39. J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proc. 1995 IEEE Int. Conf. Neural

Networks. Part 1 (of 6), IEEE, Piscataway, NJ, United StatesPerth, Aust, 1995: pp. 1942–1948. https://www.scopus.com/inward/record.uri?eid=2-s2.0-0029535737&partnerID=40&md5=e6bf 04ae50f3268ae545d88ed91d1fc5. [4]

Y. Ge, Q.C. Meng, C.J. Yan, J. Xu, A hybrid ant colony algorithm for global optimization of

continuous multi-extreme functions, in: Int. Conf. Mach. Learn. Cybern., 2002: pp. 2427–2432

AN US

vol.4. [5]

R. Chelouah, P. Siarry, A Continuous Genetic Algorithm Designed for the Global Optimization of Multimodal Functions, J. Heuristics. 6 (2000) 191–213.

[6]

K.E. Parsopoulos, M.N. Vrahatis, Recent approaches to global optimization problems through Particle Swarm Optimization, Nat. Comput. 1 (2002) 235–306.

[7]

X.S. Yang, Flower Pollination Algorithm for Global Optimization, in: Int. Conf. Unconv.

[8]

M

Comput. Nat. Comput., 2012: pp. 240–249.

X.S. Yang, A New Metaheuristic Bat-Inspired Algorithm, Comput. Knowl. Technol. 284 (2010) 65–74.

S. Mirjalili, Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm,

[10]

S. Saremi, S. Mirjalili, A. Lewis, Grasshopper Optimisation Algorithm: Theory and application,

ED

[9]

Knowledge-Based Syst. 89 (2015) 228–249.

[11]

PT

Adv. Eng. Softw. 105 (2017) 30–47. S. Lukasik, P.A. Kowalski, M. Charytanowicz, P. Kulczycki, Data clustering with grasshopper optimization algorithm, in: Proc. 2017 Fed. Conf. Comput. Sci. Inf. Syst. FedCSIS 2017, 2017: [12]

CE

pp. 71–74. doi:10.15439/2017F340. A. Tharwat, E.H. Houssein, M.M. Ahmed, A.E. Hassanien, T. Gabel, MOGOA algorithm for constrained and unconstrained multi-objective optimization problems, Appl. Intell. (2017) 1–16.

AC

doi:10.1007/s10489-017-1074-1.

[13]

S.Z. Mirjalili, S. Mirjalili, S. Saremi, H. Faris, I. Aljarah, Grasshopper optimization algorithm for multi-objective optimization problems, Appl. Intell. (2017) 1–16.

[14]

M. Barman, N.B. Dev Choudhury, S. Sutradhar, A regional hybrid GOA-SVM model based on similar day approach for short-term load forecasting in Assam, India, Energy. 145 (2018) 710– 720. doi:10.1016/j.energy.2017.12.156.

[15]

A.A. El-Fergany, Electrical characterisation of proton exchange membrane fuel cells stack using grasshopper optimizer, IET Renew. Power Gener. 12 (2018) 9–17. doi:10.1049/iet-rpg.2017.0232.

[16]

M. Mafarja, I. Aljarah, A.A. Heidari, A.I. Hammouri, H. Faris, A. Al-Zoubi, S. Mirjalili, Evolutionary Population Dynamics and Grasshopper Optimization approaches for feature

ACCEPTED MANUSCRIPT

selection problems, Knowledge-Based Syst. (2018). doi:10.1016/j.knosys.2017.12.037. [17]

J. Wu, H. Wang, N. Li, P. Yao, Y. Huang, Z. Su, Y. Yu, Distributed trajectory optimization for multiple solar-powered UAVs target tracking in urban environment by Adaptive Grasshopper Optimization Algorithm, Aerosp. Sci. Technol. 70 (2017) 497–510. doi:10.1016/j.ast.2017.08.037.

[18]

S. Arora, P. Anand, Chaotic grasshopper optimization algorithm for global optimization, Neural Comput. Appl. (2018) 1–21. doi:10.1007/s00521-018-3343-2.

[19]

T. Bäck, H.P. Schwefel, An overview of evolutionary algorithms for parameter optimization, in: 1993: pp. 1–23. I. Pavlyukevich, Levy flights, non-local search and simulated annealing, Mathematics. 226

CR IP T

[20]

(2007) 1830–1844. [21]

H.R. Tizhoosh, Opposition-Based Learning: A New Scheme for Machine Intelligence, in: Int. Conf. Comput. Intell. Model. Control Autom. Int. Conf. Intell. Agents, Web Technol. Internet Commer. Vol, 2005: pp. 695–701.

[22]

S. Mirjalili, Dragonfly algorithm: a new meta-heuristic optimization technique for solving

AN US

single-objective, discrete, and multi-objective problems, Neural Comput. Appl. 27 (2016)

1053–1073. doi:10.1007/s00521-015-1920-110.1007/s00521-015-1874-3; Mirjalili, S., The ant lion optimizer Adv Eng Softw, 83, pp. 80-98; Mirjalili, S., Mirjalili, S.M., Hatamlou, A., Multi-Verse Optimizer: a nature-inspired algorithm for global optimization. Neural Comput Appl doi:10.1007/s00521-015-1870-7; Wolpert, D.H., Macready, W.G., No free lunch theorems for optimization Evolut Comput IEEE Trans, 1 (1), pp. 67-82; Thorp, J.H., Rogers, D.C., Thorp and Covich’s freshwater invertebrates: ecology and general. K. Price, K. Price, Differential Evolution – A Simple and Efficient Heuristic for Global

M

[23]

Optimization over Continuous Spaces, Kluwer Academic Publishers, 1997. [24]

S. Mirjalili, SCA: A Sine Cosine Algorithm for solving optimization problems,

[25]

ED

Knowledge-Based Syst. 96 (2016) 120–133. S. Mirjalili, A.H. Gandomi, S.Z. Mirjalili, S. Saremi, H. Faris, S.M. Mirjalili, Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems, (2017). G. Bin Huang, H. Zhou, X. Ding, R. Zhang, Extreme learning machine for regression and

PT

[26]

multiclass classification, IEEE Trans. Syst. Man Cybern. Part B. 42 (2012) 513–529. G. Bin Huang, Q.Y. Zhu, C.K. Siew, Extreme learning machine: Theory and applications,

CE

[27]

Neurocomputing. 70 (2006) 489–501.

[28]

B. Liu, L. Tang, J. Wang, A. Li, Y. Hao, 2-D defect profile reconstruction from ultrasonic

AC

guided wave signals based on QGA-kernelized ELM, Neurocomputing. 128 (2014) 217–223.

[29]

H. Lu, B. Du, J. Liu, H. Xia, W.K. Yeap, A kernel extreme learning machine algorithm based on improved particle swam optimization, Memetic Comput. 9 (2016) 1–8.

[30]

M. Wang, H. Chen, B. Yang, X. Zhao, L. Hu, Z.N. Cai, H. Huang, C. Tong, Towards an optimal kernel extreme learning machine using a chaotic moth-flame optimization strategy with applications in medical diagnoses, Neurocomputing. (2017).

[31]

X. Lv, H. Chen, Q. Zhang, X. Li, H. Huang, G. Wang, An Improved Bacterial-Foraging, Optimization-Based Machine Learning Framework for Predicting the Severity of Somatization Disorder, Algorithms. 11 (2018) 17.

[32]

Q. Xu, L. Wang, N. Wang, X. Hei, L. Zhao, A review of opposition-based learning from 2005 to 2012, Eng. Appl. Artif. Intell. 29 (2014) 1–12.

ACCEPTED MANUSCRIPT

[33]

A.F. Kamaruzaman, A.M. Zain, S.M. Yusuf, A. Udin, Levy Flight Algorithm for Optimization Problems - A Literature Review, Appl. Mech. Mater. 421 (2013) 496–501.

[34]

S. Mirjalili, S.M. Mirjalili, A. Lewis, Grey Wolf Optimizer, Adv. Eng. Softw. 69 (2014) 46–61. doi:10.1016/j.advengsoft.2013.12.007.

[35]

J. Derrac, S. García, D. Molina, F. Herrera, A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms, Swarm Evol. Comput. 1 (2011) 3–18. doi:https://doi.org/10.1016/j.swevo.2011.02.002. P.R. Kumar, V. Ravi, Bankruptcy prediction in banks and firms via statistical and intelligent techniques – A review, Eur. J. Oper. Res. 180 (2007) 1–28.

[37]

CR IP T

[36]

N. Sharma, N. Arun, V. Ravi, An ant colony optimisation and Nelder-Mead simplex hybrid

algorithm for training neural networks: an application to bankruptcy prediction in banks, Int. J. Inf. Decis. Sci. 5 (2013) 188–203. [38]

V. Ravi, Bacterial foraging trained wavelet neural networks: application to bankruptcy prediction in banks, Inderscience Publishers, 2011.

M. Wang, H. Chen, H. Li, Z. Cai, X. Zhao, C. Tong, J. Li, X. Xu, Grey wolf optimization

AN US

[39]

evolving kernel extreme learning machine: Application to bankruptcy prediction, Eng. Appl.

AC

CE

PT

ED

M

Artif. Intell. 63 (2017) 54–68. doi:10.1016/j.engappai.2017.05.003.

AN US

CR IP T

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

Fig. 1. The flowchart of IGOA

ACCEPTED MANUSCRIPT

ED

M

AN US

CR IP T

Fig. 2. Flowchart of the proposed IGOA-KELM

AC

CE

PT

Fig. 3. Convergence curves of the ten algorithms for four test functions.

AN US

CR IP T

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

Fig. 4. The classification performance obtained by the ten models in terms of ACC, sensitivity, specificity and MCC on the Japanese dataset

AN US

CR IP T

ACCEPTED MANUSCRIPT

AC

CE

PT

ED

M

Fig. 5. Comparison results of IGOA-KELM and other methods on the Japanese dataset.

ACCEPTED MANUSCRIPT

Table 1. Benchmark functions used in the experiments. Function

Dimension 𝑛

𝑓1 (𝑥) = ∑𝑖=1 𝑥𝑖2 𝑛−1

𝑓2 (𝑥) = ∑𝑖=1 [100(𝑥𝑖+1 − 𝑥𝑖2 )2 + (𝑥𝑖 − 1)2 ] 1 4000

𝑛

𝑥

∑𝑖=1 𝑥𝑖2 − ∏𝑛𝑖=1 cos ( 𝑖 ) + 1 √𝑖

3 𝑗=1

𝑎𝑖𝑗 (𝑥𝑗 − 𝑝𝑖𝑗 ])2 )

n

[-100,100]

0

n

[-30,30]

0

n

[-600,600]

0

3

[1,3]

-3.86

AC

CE

PT

ED

M

AN US

𝑓4 (𝑥) = − ∑4𝑖=1 𝑐𝑖 exp (− ∑

fmin

CR IP T

𝑓3 (𝑥) =

Range

ACCEPTED MANUSCRIPT

Table 2. Comparison of various IGOAs

f2

f3

IGOA2

IGOA3

IGOA4

IGOA5

IGOA6

IGOA7

28.2644

1.18E-07

2.5275

1.19E-07

0.0342

2.84E-09

0.0245

2.27E-09

16.0530

2.44E-07

0.8673

3.13E-07

0.0327

3.49E-09

0.0343

1.76E-09

106.04

110.35

106.81

111.62

109.07

115.75

111.16

106.04

8

3

7

4

6

2

5

1

4384.135

161.6362

771.915

336.4161

373.9811

346.0503

258.3298

49.3116

2451.053

124.7406

701.8484

344.4949

487.1353

936.8168

366.2698

35.05442

107.01

109.96

107.43

111.09

108.04

111.05

109.26

107.73

8

2

7

4

6

1.0449

2.07E-08

0.4530

3.98E-09

0.1684

4.71E-08

0.0730

107.63

107.50

8

5

3

1

0.2177

3.78E-09

0.2117

3.78E-09

5.33E-09

0.0726

2.23E-09

0.07031

4.39E-09

111.61

111.20

101.85

116.15

109.78

115.40

4

7

3

6

2

5

1

-3.6534

-3.7457

-3.8625

-3.8618

-3.8011

-3.7586

-3.8627

-3.8627

0.3101

0.0839

0.0005

0.0008

0.1832

0.1642

4.46E-06

0.0005

12.25

12.34

13.72

13.80

12.45

14.51

15.33

16.36

8

7

2

4

5

6

1

3

AC

CE

PT

ED

M

f4

IGOA1

CR IP T

f1

GOA

AN US

Function

ACCEPTED MANUSCRIPT

Table 3. Scalability results

10 30 100 200 500

f2

f3

mean

std.

mean

std.

mean

std.

1.94E-10

1.52E-10

8.30E+00

5.35E-01

5.71E-10

3.05E-10

6.42E-06

6.43E-06

2.39E+02

5.33E+02

2.47E-01

9.47E-02

1.55E-09

8.87E-10

4.55E+01

3.18E+01

3.09E-09

3.14E-09

4.53E+01

2.62E+01

4.42E+03

5.55E+03

1.09E+00

1.76E-01

3.12E-08

3.67E-08

3.07E+03

3.10E+03

4.43E-08

5.75E-08

1.23E+04

2.56E+03

3.75E+06

1.21E+06

1.05E+02

1.53E+01

2.76E-08

1.23E-08

4.45E+03

9.10E+03

4.37E+04

5.00E+03

1.82E+07

2.84E+06

2.39E-07

3.35E-07

6.84E+04

1.28E+05

1.33E+05

1.13E+04

8.08E+07

6.59E+06

9.47E-07

1.26E-06

2.27E+07

7.15E+07

2.84E+05

1.25E+04

1.97E+08

2.46E+07

7.58E-09

5.91E-09

3.85E+02

3.24E+01

1.63E-08

1.15E-08

1.21E+03

6.82E+01

5.43E-08

7.44E-08

2.60E+03

1.15E+02

AC

CE

PT

ED

M

AN US

1000

f1

CR IP T

Dim

ACCEPTED MANUSCRIPT

Table 4. Comparison results of the methods involved in this study

f3

DE

FPA

GOA

MFO

PSO

SCA

SSA

5.77E -08 7.69E -08 1

2.31E +02 1.48E +01 2 2.19E +05 3.56E +04 3 1.06E +00 1.12E -02 2 -3.84 E+00 1.47E -02 3

9.85E +03 5.93E +03 7 1.05E +07 9.53E +06 8 1.02E +02 7.99E +01 8

2.32E +03 2.48E +02 5 3.18E +06 6.69E +05 5 2.14E +01 2.24E +00 5

1.77E +04 3.41E +03 9 7.51E +06 2.76E +06 7 1.56E +02 2.93E +01 9

6.50E +03 1.45E +03 6 1.05E +06 4.19E +05 4 5.70E +01 1.18E +01 6

5.28E +04 1.23E +04 10 1.25E +08 7.18E +07 10 4.78E +02 1.08E +02 10

1.09E +04 8.77E +03 8 1.08E +08 6.32E +07 9 9.16E +01 7.19E +01 7

3.51E +02 1.33E +02 3 2.39E +04 1.80E +04 2 3.76E +00 9.58E -01 4

-3.86 E+00 9.75E -04 1

-3.86 E+00 2.71E -15 1

-3.86 E+00 4.16E -07 1

-3.86 E+00 2.85E -02 1

-3.86 E+00 2.71E -15 1

1.37E +03 1.35E +02 4 6.06E +06 1.15E +06 6 1.36E +00 2.66E -02 3 -3.85 E+00 1.07E -02 2

-3.86 E+00 2.88E -03 1

-3.86 E+00 3.92E -14 1

1.93E +03 3.75E +03 1 1.85E -08 2.59E -08 1 -3.86 E+00 3.73E -04 1

AC

CE

PT

ED

f4

DA

CR IP T

f2

BA

AN US

f1

IGOA

M

Func tion

ACCEPTED MANUSCRIPT

Table 5. The calculated p-values from the Wilcoxon test for the IGOA versus other optimizers Functi on

f2 f3

DE

FPA

GOA

MFO

PSO

SCA

SSA

1.83E04 1.83E04 1.83E04 1.83E04

1.83E04 1.83E04 1.83E04 3.85E01

1.83E04 1.83E04 1.83E04 6.39E05

1.83E04 1.83E04 1.83E04 9.11E03

1.83E04 1.83E04 1.83E04 3.12E02

1.83E04 1.83E04 1.83E04 6.39E05

1.83E04 1.83E04 1.83E04 1.83E04

1.83E04 1.83E04 1.83E04 2.46E04

1.83E04 1.83E04 1.83E04 1.83E04

AC

CE

PT

ED

M

AN US

f4

DA

CR IP T

f1

BA

ACCEPTED MANUSCRIPT

Table 6. Ten important financial ratios Feature

Financial ratios

Feature

Financial ratios

net profit/total assets

X6

sales/total assets

X2

current assets/current liabilities

X7

market value of equity/total debt

X3

retained earnings/total assets

X8

cash flow/total debt

X4

working capital/total assets

X9

current assets/total assets

X5

EBIT* /total assets

X10

cash/current liabilities

AC

CE

PT

ED

M

AN US

*(EBIT means earnings before interests and taxes)

CR IP T

X1

ACCEPTED MANUSCRIPT

Table 7. The results obtained by OBLWOA-KELM on the Japanese dataset γ

ACC

#1 #2 #3 #4 #5 #6 #7 #8 #9 #10 Avg. Dev.

883.0340 641.0326 699.5911 775.4226 643.4662 734.6934 567.7140 542.7934 828.5851 341.5971 665.7930 157.2564

9.5217 384.6332 12.5507 15.0195 8.6024 20.7143 10.3025 11.6004 8.1325 200.7144 68.1792 126.0638

0.8666 0.7333 0.8000 0.6666 0.8666 0.8000 1.0000 0.8125 0.8000 1.0000 0.8345 0.1050

Sensitivity

Specificity

MCC

0.8571 1.0000 0.7500 0.7500 0.8333 0.8571 1.0000 0.8000 0.7500 1.0000 0.8597 0.1051

0.8750 0.6363 0.8571 0.5714 1.0000 0.7500 1.0000 0.8333 0.8571 1.0000 0.8380 0.1491

0.7321 0.5640 0.6071 0.3273 0.7071 0.6071 1.0000 0.6180 0.6071 1.0000 0.6770 0.2015

CR IP T

C

AC

CE

PT

ED

M

AN US

Fold