An ensemble of intelligent water drop algorithms and its application to optimization problems

Information Sciences 325 (2015) 175–189

Contents lists available at ScienceDirect

Information Sciences journal homepage: www.elsevier.com/locate/ins

An ensemble of intelligent water drop algorithms and its application to optimization problems Basem O. Alijla a,b, Li-Pei Wong a, Chee Peng Lim c,∗, Ahamad Tajudin Khader a, Mohammed Azmi Al-Betar d a

School of Computer Sciences, Universiti Sains Malaysia, Malaysia Faculty of Information Technology, Islamic University of Gaza, Palestine c Centre for Intelligent Systems Research, Deakin University, Australia d Department of Information Technology, Al-Huson University College, Al-Balqa Applied University, P.O. Box 50, Al-Huson, Irbid, Jordan b

a r t i c l e

i n f o

Article history: Received 26 September 2014 Revised 13 May 2015 Accepted 4 July 2015 Available online 13 July 2015 Keywords: Intelligent water drops Optimization Swarm intelligence Exploitation Exploration

a b s t r a c t The Intelligent Water Drop (IWD) algorithm is a recent stochastic swarm-based method that is useful for solving combinatorial and function optimization problems. In this paper, we propose an IWD ensemble known as the Master-River, Multiple-Creek IWD (MRMC-IWD) model, which serves as an extension of the modified IWD algorithm. The MRMC-IWD model aims to improve the exploration capability of the modified IWD algorithm. It comprises a master river which cooperates with multiple independent creeks to undertake optimization problems based on the divide-and-conquer strategy. A technique to decompose the original problem into a number of sub-problems is first devised. Each sub-problem is then assigned to a creek, while the overall solution is handled by the master river. To empower the exploitation capability, a hybrid MRMC-IWD model is introduced. It integrates the iterative improvement local search method with the MRMC-IWD model to allow a local search to be conducted, therefore enhancing the quality of solutions provided by the master river. To evaluate the effectiveness of the proposed models, a series of experiments pertaining to two combinatorial problems, i.e., the travelling salesman problem (TSP) and rough set feature subset selection (RSFS), are conducted. The results indicate that the MRMC-IWD model can satisfactorily solve optimization problems using the divide-and-conquer strategy. By incorporating a local search method, the resulting hybrid MRMC-IWD model not only is able to balance exploration and exploitation, but also to enable convergence towards the optimal solutions, by employing a local search method. In all seven selected TSPLIB problems, the hybrid MRMC-IWD model achieves good results, with an average deviation of 0.021% from the best known optimal tour lengths. Compared with other state-of-the-art methods, the hybrid MRMC-IWD model produces the best results (i.e. the shortest and uniform reducts of 20 runs) for all13 selected RSFS problems. Crown Copyright © 2015 Published by Elsevier Inc. All rights reserved.

1. Introduction Optimization is a process that is concerned with finding the best solution of a given problem from among a range of possible solutions, within an affordable time and cost [66]. Optimization can be applied to many real-world problems, in a large variety of domains. As an example, mathematicians apply optimization methods to identify the best outcome pertaining to some ∗

Corresponding author. Tel.: +61 3 5227 3307. E-mail address: [email protected], [email protected] (C.P. Lim).

http://dx.doi.org/10.1016/j.ins.2015.07.023 0020-0255/Crown Copyright © 2015 Published by Elsevier Inc. All rights reserved.

176

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189 Table 1 Applications of the IWD algorithm. Disciplines

Applications

Ref.

Engineering

Multi-dimensional knapsack, n-queen puzzle Vehicle routing Economic load dispatch Flow-shop scheduling Reactive power dispatch Safety and in-transit inventory in manufacturing supply chains Robot path planning Printed circuit boards drill routing process Parallel machine scheduling Aerospace and defense QoS-aware routing algorithm for MANETs Routing protocol in mobile ad–hoc networks Sensor node organization in wireless sensor networks Feature selection for an irrigation system Rough set feature subset selection Clustering algorithm Gene selection and cancer classification Fault detection Optimization of neural network weight parameters Multi-objective job shop scheduling Vehicle guidance in road graph networks Mathematical optimization function

[51,53,54] [2] [41,45] [72] [32] [40] [16,48] [59] [26] [60] [49] [28] [21] [20] [3] [55] [36] [5] [17] [43] [61] [54]

Networking and routing

Machine learning

Multi-objective optimization Continuous optimization

mathematical functions within a range of variables [63]. In the presence of conflicting criteria, engineers often use optimization methods to find the best performance of a model subject to certain criteria, e.g. cost, profit, and quality [37]. Numerous methods have been developed and used to solve many NP-hard (i.e. problems that have no known solutions in polynomial time) [33] optimization problems [39,64,69]. A number of recent survey papers that provide comprehensive information on optimization methods and their associated categorizations are also available in the literature [11,30,33,35]. In this study, we focus on the Swarm Intelligence (SI) methodology for undertaking optimization problems. Among a variety of optimization methods, SI constitutes an innovative family of nature inspired models that has attracted much interest from researchers [8]. SI models stem from different natural phenomena pertaining to different swarms, e.g. ant colony optimization (ACO) is inspired by the foraging behavior of ants [13,14], while particle swarm optimization (PSO) is inspired by the social behaviors of bird flocking or fish schooling [56]. In this paper, we investigate a relatively recent swarmbased model known as the intelligent water drop (IWD) algorithm [50]. IWD is inspired by the natural phenomenon of water drops flowing with soil and velocity along a river. It imitates the natural phenomena of water drops flowing through an easier path, i.e., a path with less barriers and obstacles, from upstream to downstream. Specifically, IWD is a constructive-based, meta-heuristic algorithm, comprising a set of cooperative computational agents (water drops), that iteratively constructs the solution pertaining to a problem. The solution is formulated by water drops that traverse a path with a finite set of discrete movements. A water drop begins its journey with an initial state. It iteratively moves step-by-step passing through several intermediate states (partial solutions), until a final state (complete solution) is reached. A probabilistic method is used to control the movements of the water drops. Specifically, each water drop in the IWD algorithm has two key attributes: soil and velocity. They are used to control the probability distribution of selecting the movement of the water drop, and to find the partial solution. The soil represents an indirect communication mechanism, and enables the water drop to cooperate with other nearby water drops. The soil level indicates the cumulative proficiency of a particular movement. Contrary to the ant colony algorithm [15], in which the pheromone level is constantly updated, the soil level is dynamically updated with respect to the velocity of the water drop. In other words, the velocity influences the dynamics of updating the soil level, which is used to compute the probability of the movement of the water drop from the current state to the next. In addition, the velocity is related to heuristic information pertaining to the problem under scrutiny. This information is used to guide the water drop to move from one state to another. The IWD algorithm is useful for tackling combinatorial optimization problems [53]. IWD initially was applied for solving the travelling salesman problem (TSP) [50]. Over the past few years, it has been successfully adopted to solve different NP-hard optimization problems [57]. Table 1 summarizes a number of applications that have been successfully solved using the IWD algorithm. The success of the IWD algorithm stems from two salient properties [4,52,53]: (i) its cooperative learning mechanism allows water drops to exchange their search knowledge and (ii) the algorithm is able to memorize the search history. As can be seen in Table 1, most of the reported IWD investigations in the literature focus on solving optimization problems in different application domains. Only a small number of studies pertaining to the theoretical aspects of the IWD algorithm to improve its performance are available in the literature. As an example, an Enhanced IWD (EIWD) algorithm to solve jobshop scheduling problems was proposed by Niu et al. [42]. The following schemes have been introduced to increase diversity of

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

177

the search space and enhance the original IWD performance: (i) varying the initial soil and velocity values, (ii) employing the conditional probability in the selection probability, (iii) bounding the soil level, (iv) using the elite mechanism to update the soil and (v) combining the IWD algorithm with a local search method. In Alijla et al., [4], the modified IWD algorithm was introduced to address the limitations of fitness proportionate selection method in the original IWD algorithm. Two ranking-based methods i.e. linear and exponential were introduced. The extendibility of the IWD algorithm was investigated by Kayvanfar and Teymourian [25]. In particular, a hybrid IWD and local search algorithm was introduced. The variable neighborhood structure (VNS) algorithm was used to tackle scheduling problems of unrelated parallel machines. The aforementioned modifications focus on the soil level and the algorithm exploitation capability to enhance its performance. In this paper, two enhancements pertaining to the modified IWD algorithm introduced in [4] are proposed to achieve a balance between exploration (navigating through new regions of the search space) and exploitation (searching a specific region of the search space thoroughly). The proposed enhancements include: (i) an ensemble of the modified IWD algorithms in a Master-River, Multiple-Creek (MRMC) model, whereby a divide-and-conquer strategy is utilized to improve the search process and (ii) a hybrid MRMC-IWD model, which improves the exploitation capability of MRMC-IWD with a local improvement method to constraint the search to the local optimal solution, rather than the entire search space. To evaluate the proposed models (i.e. MRMC-IWD and hybrid MRMC-IWD) and to facilitate a performance comparison study with other state-of-theart methods, two case studies related to optimization problems that have been widely used in the literature [24,31,38,58,67], namely TSP and rough set features subset selection (RSFS), are conducted. These problems are selected because they are NPhard, and have different level of difficulties. The complexity of the problem (i.e., the number of alternatives) grows exponentially with the size of the problem [19]. Since TSPs have known bounds, they are useful to ascertain the effectiveness of the solutions produced by the proposed models. On the other hand, RSFS is crucial in pattern recognition applications. Contrary to TSP, RSFS presents strong inter-dependency among the decision variables (i.e., features). The feature sequences within the subset are not important, and the optimal solutions are normally unknown [70]. Therefore, both TSP and RSFS problems are selected as case studies to evaluate the usefulness of the proposed models and to benchmark the results against those published in the literature. As a result, the effectiveness of the proposed models for undertaking general optimization problems can be validated. The rest of this paper is organized as follows. In Section 2, the background of the IWD and modified IWD algorithms is described. In Section 3, the proposed MRMC-IWD and hybrid MRMC-IWD models are explained. In Section 4, the case studies, i.e. RSFS and TSP, are explained in details. The results are analyzed and discussed in Section 5. Conclusions and suggestions for future research are presented in Section 6.

2. Background of the IWD algorithm The IWD algorithm is a constructive-based, nature-inspired model introduced by Shah-Hosseini [50]. It is motivated by the dynamic of water flowing in a river, e.g. water follows an easier path which has fewer barriers and obstacles, water flows at a particular speed, water stream changes the environmental properties of the river, which subsequently changes the direction of water flow. The IWD algorithm computationally realizes some of these natural phenomena, and uses them as a computational mechanism to solve combinatorial optimization problems. The IWD algorithm comprises a number of computational agents (i.e. water drops). At each iteration, the water drops construct a solution, by following a finite set of discrete movements. Each water drop begins with an initial state, then iteratively moves step-by-step until a complete solution is produced. Each water drop cooperates with other water drops, and updates the environmental properties, i.e. soil and velocity. A water drop starts with an initial velocity and carries zero amount of soil. When the water drop moves from one location to another, its velocity and soil level are updated. The velocity is changed non-linearly, and is proportional to the inverse of the amount of soil between two locations. The water drop carries an amount of soil in each movement, which is non-linearly proportional to the inverse of the time needed by the water drop to move from the current location to the next. On the other hand, the time taken by a water drop to move from one location to another is proportional to its velocity and inversely proportional to the distance between two locations. The first step in solving an optimization problem using the IWD algorithm is to represent the problem by a fully connected weighted graph called the construction graph, i.e., G (V, E), where the set of vertices is denoted by V = {vi |i = 1, . . . , N}, and the set of arcs is denoted by E = {(i, j)|(i, j) ∈ V × V, i = j, i, j = 1, . . . , N}. Note that N is the number of decision variables. A solution, i.e., the feasible permutation of decision variables, is represented by a finite set of components (i.e. vertices or arcs), π = {ak | k = 1, . . . , D}, where D ≤ N is the dimension of the solution, and ak ∈ A, where A is the set of all possible components (vertices or arcs). In the following sections, a detailed description of the main phases of the IWD algorithm i.e. initialization, solution construction, reinforcement, and termination are presented.

2.1. Initialization phase In the initialization phase, two types of the IWD parameters, i.e. static and dynamic, are initialized.

178

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

2.1.1. Static parameters Static parameters are a set of IWD parameters that are initialized once at the beginning of the algorithm. Once they are initialized, their value remains unchanged throughout the entire search process. In principle, the static parameters can be divided into two categories. The first category controls the population dynamics of the IWD algorithm, as follows: • m: the population size (i.e. number of water drops). • Max_iter: the maximum number of iterations before terminating the IWD algorithm. • initSoil: the initial value of the local soil. The second category controls the update dynamics of soil and velocity during the search process, as follows: • Soil updating parameters (as , bs , cs ): control the soil update function, as defined in Eq. (4). • Velocity updating parameters (av , bv cv ): control the velocity update function, as defined in Eq. (7). 2.1.2. Dynamic parameters The dynamic parameters control the process of constructing the solutions. They are initialized at the beginning of each iteration, and are dynamically updated during the solution construction process. The dynamic parameters of the IWD algorithm include: • V kvisited a list of vertices visited by water drop k. • InitVel k : the initial velocity of water drop k • Soil k : the initial amount of soil loaded onto water drop k. 2.2. Solution construction phase In the solution construction phase, a population of m solutions based on m cooperative computational agents (i.e. water drops) is constructed. A solution, which is represented by a water drop, is formulated by as set of finite components, π = {c j | j = 1, . . . , D}. The solution construction phase starts by initializing a set of m water drops. They are then randomly spread at the vertices of the construction graph. Every water drop, k, employs a list of visited components (i.e. Vvkisited ), to maintain a

record of the components that have been added to a solution. The addition of a component to Vvkisited ) is performed such that no components are duplicated within a solution. Then, each water drop extends its partial solution, i.e. one movement of a water drop k is followed by another movement to water drop k + 1. Discrete finite steps (movements) from one location to the next are iterated by each water drop until a complete solution is constructed, i.e. the condition of completeness is met. During the movement process of the water drops, the soil and velocity attributes are constantly updated. This is important to guide other water drops in subsequent iterations to construct better solutions. The solution construction phase is composed of two steps, i.e., velocity and local soil update and selection mechanism, as described in the following sections. 2.2.1. Velocity and local soil update Whenever a water drop moves from one location to the next, it updates the soil and velocity attributes. Firstly, the velocity is updated by an amount that is non-linearly proportional to the inverse of the amount of soil between any two locations, as defined in Eq. (1), where vel k (t + 1) represents the velocity of water drop k at time t + 1, av , bv cv are the static parameters used to represent the non-linear relationship between the velocity of water drop k, i.e. vel k , and the inverse of the amount of soil in the local path, i.e. soil(i,j).

vel k (t + 1) = vel k (t ) +

av bv + cv ∗ soil 2 (i, j)

(1)

Thereafter, the soil values (i.e. the amount of soil loaded in the water drop itself and the local soil between two locations) are updated, as defined in Eqs. (2) and ( 3) respectively, where soil k is the amount of soil carried by water drop k, ρn is a small positive constant between zero and one, i.e., 0 < ρn < 1;  soil (i, j) is the amount of soil removed from the local path and carried by the water drop.

soil k = soil k +  soil (i, j) soil (i, j) = (1 − ρn )∗ soil (i, j) −

(2)

ρn ∗ soil (i, j)

(3)

This amount of soil is non-linearly proportional to the inverse of the time needed for the water drop to travel from its current location to the next, as defined in Eq. (4), where as , bs, cs are the static parameters used to represent the non-linear relationship between  soil (i, j) and the inverse of vel k . Note that time(i, j : vel k (t + 1)) is the time needed for water drop k to transit from location i to location j at time t + 1.

 soil (i, j) =

as bs + cs ∗ time2 (i, j : vel k (t + 1))

(4)

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

179

On the other hand, the time taken by the water drop to move from one location to the next is proportional to its velocity and inversely proportional to the distance between two locations, as defined in Eq. (5), where HUD(i, j) is a heuristic desirability degree between locations i and j.

time (i, j : vel k (t + 1)) =

HUD(i, j) vel k (t + 1))

(5)

2.2.2. Selection mechanism The original IWD algorithm adopts the fitness proportionate selection method, which is a parameterized probabilistic transition rule, as defined in Eq. (6).

pki ( j) = 

f (soil (i, j)) f (soil (i, l )) ∀l ∈V / k

(6)

visited

The probability of moving from one location to the next location is proportional to the inverse of the amount of soil between two locations, as defined in Eqs. (7) and ( 8), where ε is a small positive number used to prevent division by zero in function f (.), and soil (i, j) refers to the amount of soil within the local path between locations i and j.

1

f (soil (i, j)) =

g(soil (i, j)) =

(7)

ε + g(soil (i, j)) ⎧ ⎨soil (i, j)

if

(soil (i, l )) ⎩soil (i, j) − min k

Otherwise

∀l ∈V / visited

min

∀l ∈V / vkisited

(soil (i, l )) ≥ 0 (8)

In our proposed modified IWD algorithm [4], the selection method in Eq. (6) is replaced by two ranking-based selection methods, i.e. linear and exponential ranking, as in Eqs. (9) and ( 10), respectively.

P (i) =

1 × N



i−1 N−1

SP − 2(SP − 1) ×

P (i) = SP i−1 ∗



(9)

1 − SP 1 − SP N

(10)

where i ∈ {1, ..., N} represents an arc’s rank. All arcs are ranked in a descending order according to their amount of soil. The fittest arc (i.e. the arc with the lowest amount of soil) is denoted as rank 1, and the least-fit arc is denoted as rank N. P (i) refers to the probability of selecting element i. 2.3. Reinforcement phase The second step in updating the soil is known as the global soil update. In this step, the original IWD algorithm updates the amount of soil pertaining to all components included in the fittest solution, which is known as the iteration-best solution, and is denoted as T IB . It is determined using Eq. (11)

T IB = arg min / max q(x)

(11)

∀x∈T IW B

where q(.) is the fitness function used to evaluate the quality of solutions, and T IW B is the population of solutions. The global soil update is performed, as defined in Eq. (12) k soil (i, j) = (1 + ρIW D ) ∗ soil (i, j) − ρIW D ∗ soilIB ∗

1 q(T IB )

(12)

where ρIW D is a positive constant. During each iteration, the best solution (global best), i.e., T T B , is either replaced by T IB , or is maintained, as defined in Eq. (13)



T

TB

=



T IB

if q(T IB ) < q T T B

T TB

Otherwise



(13)

2.4. Termination phase The solution construction and reinforcement processes are iterated until the termination condition is met. At the beginning of each iteration, the dynamic parameters are reverted to their initial values. A number of termination conditions can be used, e.g. the maximum number of iterations (i.e. MaxIter) and this is a straightforward termination condition for the IWD algorithm.

180

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

3. The proposed models Two modifications are proposed to enhance the performance of the modified IWD algorithm. Firstly, an ensemble of the modified IWD algorithms, in a Master-River, Multiple-Creek IWD (MRMC-IWD) model is proposed. Secondly, the MRMC-IWD model is integrated with a local search method to enhance the performance by empowering the exploitation capability. These modifications are explained in detail in the following sub-sections. 3.1. The master-river, multiple-creek IWD model The MRMC-IWD model is proposed to enhance the search capability of the modified IWD algorithm, by using the divideand-conquer strategy. It is inspired by the natural phenomena pertaining to a main river with multiple independent creeks flowing down the stream. The rationale is based on dividing a complex problem into a number of sub-problems, (i.e., divide-andconquer). Fig. 1 depicts the structure of MRMC-IWD and its communication scheme. Firstly, a suitable decomposition technique (e.g. clustering algorithm) can be used to decompose the entire problem into a number of sub-problems, e.g., the k-means algorithm is used to cluster the entire TSP cities into several sub-sets of cities. Note that other clustering algorithms can be used too. The master river handles the entire problem (i.e., complete cities), while each creek handles a sub-problem (i.e., a subset of cities). In other words, the master river constructs a complete solution of the problem, while each creek contributes a partial solution. Both the master river and independent creeks maintain their parameters, (i.e. static and dynamic parameters). A bilateral cooperative scheme between the master river and multiple creeks is introduced, in order to enable exchange of partial solutions between the master river and each creek, as shown in Fig. 1. The partial solutions are known as the creek local best (CLB) water drops. The complete solution is known as the master local best (MLB) water drop. A sequential optimization process is adopted. The detailed operation is as follows. During the sequential optimization process, the first creek receives the MLB water drop from the master river. It uses the MLB water drop to update the soil and velocity levels so that the search for a (partial) solution starts with a certain level of “knowledge” (with respect to the region of optimality) provided by the master river. Note that in the first round, the MLB water drop received by the first creek is based on random initialization. However, in subsequent rounds, the MLB water drop provided to the first creek contains useful information that can help accelerate the search process in its local space. Upon completing the search process, the first creek propagates back its CLB water drop to the master river. The master river updates the relevant soil level pertaining to the first creek based on the received CLB water drop. The master river kicks off its search for a complete solution using the modified IWD algorithm, based on the updated soil level. Upon completion of the search process (usually just a few nominal iterations), the relevant MLB water drop pertaining to the next (e.g., second) creek is retrieved and sent to the target creek for it to carry out its search process. After completing the search process, the corresponding creek sends the resulting CLB water drop to the master river for it to evolve a complete solution again. This process is carried out in a sequential manner until the master river arrives at a complete solution based on the CLB water drop from the last creek. A termination criterion (e.g. a maximum round of iterations) is imposed. If the termination criterion is satisfied, the global best solution is retrieved from the master river. If not, the same sequential process re-iterates from the first creek. The aforementioned mechanism enhances the search capability of the modified IWD algorithm because, on one hand, the MLB water drop provided by the master river allows each creek to accelerate its search process around the optimal regions of the sub-problem. On the other hand, the master river adopts the CLB water drop from each creek and focuses its search for a complete solution. In other words, MRMC-IWD aims to exploit the exploration capability of the modified IWD algorithm. It decomposes the exploration process of the search space into two distinctive levels:

Fig. 1. The structure of the MRMC-IWD and its communication model.

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

181

I. sub-exploration: since each creek handles a sub-problem, its respective modified IWD algorithm performs sub-exploration of the search space to find the best solution of each sub-problem; II. global exploration: since the master river handles the complete solution of the problem, its modified IWD algorithm explores the entire search space to find the best solution for the overall problem. The pseudo-code of MRMC-IWD is presented in Algorithm 1. Algorithm 1 Pseudo-code of the MRMC-IWD model. 1. Initialize the master river and the C numbers of creeks. 2. while termination condition is not met do 3. for each creek i = { 1,... ,C} do 4. The master applies the modified IWD to construct the complete solution 5. The master river passes its MLB water drop to creek i 6. Creek i applies the modified IWD algorithm to construct its solutions 7. Creek i passes its CLB water drop back to the master river 8. End_for 9. End_while

In the following sections, a detailed description of the three main phases of MRMC-IWD, namely initialization, master river construction, and creek construction is presented. 3.1.1. Initialization In this phase, a master river and a total of C independent creeks are initialized. A problem decomposition technique is employed to decompose the original problem into a set of sub-problems. As an example, the k-means clustering algorithm [27] is used to cluster the decision variables into different sub-groups, i.e. the complete problem is decomposed into a number of sub problems. Each sub-problem is assigned to a creek, while the entire problem is handled by the master river. The decomposition strategy not only reduces the problem size but also allows the sub-problems to be solved independently. In MRMC-IWD, the modified IWD algorithm is used in the master river and all creeks to solve the optimization problem. Algorithm 2 presents a high level pseudo-code of the k-means clustering algorithm. The goal is to form a set of compact clusters pertaining to the decision variables, with each cluster having small Euclidian distances among the cluster members (decision variables). Each cluster of decision variables is handled by a creek. Algorithm 2 The k-means clustering algorithm. 1. Randomly selects C cluster centers (c1 ,… cc ) 2. Calculate the Euclidian distance of all D (decision variables) from the C centres 3. Assign every d ∈ D to the cluster ci whose center closest to it.  d 4. Set |c1 | i

d∈ci

5. Loop to step 2 until centers stabilize with a given threshold.

3.1.2. Master river construction The master river executes the modified IWD algorithm. Then, it sends the MLB water drops to a creek. When the master river receives the CLB water drop from a creek (e.g. creek k), it updates the soil level of the arcs that corresponds to the CLB water drop using Eq. (14)

Soil (i, j) =

ρ ∗Soil (i, j) −

SoilCLB Lk

(14)

where SoilCLB is the soil level of the CLB water drop, and Lk is the number of decision variables pertaining to creek k. Then, the master river executes the modified IWD algorithm to construct a set of complete solutions. It sends the MLB water drop to the next creek, i.e. creek (k + 1). Given C creeks, the master river executes the modified IWD algorithm C times in each round of its iterative process. The final solution of the original problem is obtained from the master river by iterating through the whole process until the termination condition is met. 3.1.3. Creek construction Two main steps are included in the creek construction phase, i.e., identification of the partial solution and communication between the master river and the corresponding creek. Firstly, each creek constructs a set of partial solutions using the modified IWD algorithm. The creek benefits from the information received from the master river (i.e. the MLB water drop) by updating the soil level that corresponds to the MLB water drop using Eq. (15).

Soil (i, j) =

ρ ∗Soil (i, j) −

Soil MLB N

(15)

182

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

where Soil (i, j) is the soil level of the local arc between locations i, and j; 0 < ρ < 1 is the soil update control parameter; Soil MLB : is the soil level of the MLB water drop; and N is the number of decision variables of the MLB water drop (the entire decision variables). 3.2. A hybrid MRMC-IWD model Population-based optimization methods, which include IWD, are exploration oriented, i.e., bias toward global exploration rather than local exploitation. On the other hand, local-based search methods are exploitation oriented, i.e., intensifying the search at a specific region of the search space to find the local optima [11]. While MRMC-IWD exploits the exploration capability of the modified-IWD algorithm, empowering the exploitation capability into MRMC-IWD is required. To achieve this goal, hybridizing population-based methods with local-based search methods is a common technique used in the literature [9]. Based on this rationale, we integrate MRMC-IWD with a local search to produce a hybrid MRMC-IWD model. As a result, a balance between exploration and exploitation can be achieved in the proposed hybrid MRMC-IDW model. A high level pseudo-code of the hybrid MRMC-IWD model is presented in Algorithm 3. The local search process (i.e. step 9 in Algorithm 3) improves the best local solution provided by the master river. Algorithm 3 Pseudo-code of the hybrid MRMC-IWD model. 1. Initialize the master river and the C numbers of creeks. 2. while termination condition is not met do 3. for each creek i = {1,…, C} do 4. The master applies the modified IWD to construct the complete solution 5. The master river passes its MLB water drop to creek i 6. Creek i applies the modified IWD algorithm to construct its solutions 7. Creek i passes its CLB water drop back to the master river 8. End_for 9. Local search 10. End_while

In general, any local search method can be integrated with MRMC-IWD. In this paper, the iterative improvement local search (IILS) method [10] is adopted. IILS is a trajectory-based meta-heuristic local search method [10]. It can be used for different optimization problems by adopting the appropriate problem neighborhood. It improves a given solution by applying a neighborhood structure in an iterative manner. The IILS method is used to improve the best local solution provided by the master river (i.e. step 9 in Algorithm 3). Algorithm 4 presents a high level pseudo code of the IILS method. ImportInitialSoltion() (i.e. step 1 in Algorithm 4) is used to obtain a solution from the master river and this solution serves as the initial input for the local search process. Scan(N(S)) (i.e. step 3 in Algorithm 4) navigates the solution set, seeking a better solution than the current one. N(S) is the set of solutions, which is produced by applying a specific neighborhood structure. As in steps 4 and 5 in Algorithm 4, the first improved solution is accepted for further search. Algorithm 4 The iterative improvement local search (IILS) algorithm. 1. S ← ImportInitialSoltion() 2. while local optima not reached do 3. S Scan(N(S)) 4. If (f (S)< f (S )) 5. SS 6. end if 7. end while

4. Case studies Two optimization problems, namely TSP and RSFS, were considered to validate the effectiveness of the proposed models, and to facilitate the comparison against other state-of-the-art methods in the literature. They are NP-hard combinatorial optimization problems, and have different levels of difficulty (as explained in Section 1). A detailed description of each problem is as follows. 4.1. The travelling salesman problem (TSP) The aim of TSP is to find the shortest route of a salesperson, who is required to visit a given number of cities, exactly once, and to return to the starting city. As such, TSP is the problem of choosing a candidate solution x, with a minimum tour length/cost,

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

183

from the set of feasible solutions. With V cities, there exists (V−1)! /2 feasible solutions, in which the global optimum solution is sought. TSP can be formulated as a complete weighted directed graph, G = (V, E, d), where V is a set of vertices (cities), E = {(i, j)|(i, j) ∈ V × V } is a set of arcs associated with a pair of vertices i, and j, and d: EN is a weighting function linking a positive number as the weight of cost or the distance d(i, j) between vertices i and j. The objective function corresponding to the total tour length or cost is formulated in Eq. (16)



f ( x) =

min x

n−1

d(xk , xk+1 ) + d(xn , x1 )

(16)

k=1

where d(xi , x j ) is the distance from the ith city to the jth city of tour x 4.2. The rough set feature subset selection (RSFS) problem Feature selection is a process of selecting a minimum subset of features that represents the original data set [12]. The main aim of feature selection is to reduce the number of features, thereby increasing accuracy of the resulting model and decreasing the execution time. Feature selection methods can be broadly divided into two categories: filter-based and wrapper-based [18]. Filter-based methods identify a subset of features using an independent evaluation function, rather than a learning algorithm (predicator). They are computationally less intensive, as compared with wrapper-based methods that integrate a learning algorithm as a black box to evaluate the feature subsets. However, wrapper-based methods can be more efficient, e.g. the selected feature subset provides better classification accuracy as compared with that from filter-based methods because it directly utilizes the underlying learning algorithm in feature selection [18]. In this paper, a filter-based method, i.e., RSFS, is examined. Rough Set Theory (RST) was introduced by Pawlak [44]. It is one of the important mathematical theories employed for data analysis, based on the approximation of concepts in information systems. It uses the indiscernibility relation to approximate imperfect knowledge with a pair of precise sets called the lower and upper approximations. The lower approximation is a set that includes objects definitely belonging to the subset of interest. The upper approximation is a set that includes objects possibly belonging to the subset of interest. The rough set concepts can be employed to define a heuristic function, e.g. significant features, which are the features that cause a rapid increase of the positive region (the set of objects that are definitely belonging to the subset of interest) [23,47]. These heuristics can be used to evaluate the importance of the feature subset. The main rationale of using rough set for feature subset selection is to search for the minimum number of significant features. As such, feature subset selection is the problem of choosing a solution x, such that max{f(x) ∈ X}, where f is the objective function corresponding to the significance of the feature subset proportional to its cardinality, and X is the set of all feasible feature subsets. A representation of the objective function is as follows [22].



|D| − S(x) max f (x) = γξ (x)∗ x |D|



(17)

where γξ (x) is the significance score of solution x, |D| is the dimension of the complete set of features, and S(x) is the dimension of the subset, i.e. solution x. The simplest solution of feature subset selection is to generate all possible combinations, and choose the subset with the minimum degree of cardinality [24,31]. Obviously, this is an exhaustive search, and is impractical for large data sets [70]. To manage the complexity of the search process, several stochastic optimization methods, such as hill climbing with forward selection and backward elimination [62] and meta-heuristic methods such as the genetic algorithm [62], particle swarm optimization [7,65], ant colony optimization [22,71], as well as great deluge and non-linear great deluge [1,34] can be used. In this study, we have used MRMC-IWD and hybrid MRMC-IWD to tackle this challenging problem. 5. Experiments and results A series of experiments pertaining to the TSP and RSFS optimization problems using benchmark data sets was conducted to evaluate both MRMC-IWD and hybrid MRMC-IWD models. The details are as follows. 5.1. Data sets and experimental settings For the TSP problem, a total of seven benchmark data sets from TSPLIB [46], a library of sample instances for TSP from different sources and of various types, were considered. The chosen data sets comprise varying degrees of difficulties (i.e. data set sizes), i.e., from 51 cities in eil51 to 318 cities in lin318 (as shown in Table 4). All the data sets belong to the symmetric type of TSP, i.e. given a data set with n cities, for any two cities, i and j, the distance from city i to city j is the same as that from city j to city i. For these data sets, as shown in Table 4, the number associated with the name of the data set indicates the number of cities. Each city is represented by the X and Y coordination. The travel cost between two cities is specified by the Euclidean distance rounded to the nearest whole number. For the RSFS problem, a total of 13 benchmark data sets from [22] were considered. Most of them are archived in the UCI (University of California Irvine) Machine Learning Repository [6]. Table 2 shows the main properties of the data sets. They have

184

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189 Table 2 The main properties of the data sets used for RSFS experiments. Data sets

No. of features

No. of data samples

M-of-N Exactly Exactly2 HEART VOTE CREDIT MUSHROOM LED LETTERS DERM DERM2 WQ LUNG

13 13 13 13 16 20 22 24 25 34 34 38 56

1000 1000 1000 294 300 1000 8124 2000 26 366 358 521 32

Table 3 The results of MRMC-IWD for three TSPLIB data sets with various C settings. Avg. indicates the average tour length of 10 runs. Data set

Best known tour length

C =1 Avg.

C =2 Avg.

C =3 Avg.

C =4 Avg.

C =5 Avg.

C =6 Avg.

eil51 eil76 eil101

426 538 629

440.3 551.8 671.3

428.9 547.7 641.6

430.2 544.2 630.7

427.0 540.6 632.1

427.1 539.6 638.7

429.3 541.1 640.6

different degrees of difficulties (dimensions), i.e. the numbers of features and data samples. As shown in Table 2, the number of features ranges from 13 in M-of-N to 56 in LUNG, while the number of data samples ranges from 26 in LETTER to 8124 in MUSHROOM. The static and dynamic parameters of modified IWD for both problems (i.e., TSP and RSFS) were set to those reported in [4], in order to facilitate a fair performance comparison between the proposed models and the modified IWD algorithm. 5.2. TSP results The number of creeks C of MRMC-IWD was determined based on the k-means clustering algorithm. A number of preliminary experiments were conducted to investigate the effects of C on the MRMC-IWD performance using three TSPLIB data sets (i.e., eil51, eil76, and eil10). Table 3 presents the results; whereby the average tour length (Avg.) of 10 runs with various C settings are shown. It can be observed from Table 3, that the performance of MRMC-IWD improved in line with the increase in the number of creeks until C = 4. With C = 5, an enhanced result was achieved for one out of three problems, while less accurate results were obtained for the remaining two problems, as compared with those using 4 creeks. With C = 6, the worst results were produced for all three problems as compared with those from C = 4. Therefore, the setting of C = 4 was adopted for the rest of the experiments. For hybrid MRMC-IWD, the 2-opt local search [38] was employed as the designated neighborhood of the IILS method. It is a simple neighborhood structure widely used for TSP. It incrementally improves a given tour by exchanging two arcs of the tour with the other two. More precisely, the 2-opt local search selects two arcs, {u1 , u2 } and {v1 , v2 }, from the tour such that u1 , u2 , v1 , v2 are distinctive. It then replaces these two arcs by {u1 , v1 } and {u2 , v2 } in an attempt to search for a shorter tour. A thorough evaluation using the seven selected TSPLIB data sets with C = 4 for both MRMC-IWD and hybrid MRMC-IWD was conducted. Table 4 shows the comparison results between MRMC-IWD and those from the modified IWD algorithm published in [4], and the generalized chromosome genetic algorithm (GCGA) published in [68]. It summarizes the average tour lengths of 10 runs (Avg.) and the error rates (i.e., percentages of deviation between the average tour lengths and the best known lengths). The results of modified IWD, GCGA, and MRMC-IWD were obtained using the respective algorithm without a local search. MRMCIWD was able to improve the performance of the modified IWD algorithm, while MRMC-IWD outperformed both modified IWD and GCGA for all seven TSP problems. It achieved good results with an average deviation of 0.810% to the best known results. To further compare the performance in term of the solution quality (i.e. reduction in the tour length), the t-test was carried out to indicate the reliability of the result statistically. Specifically, the paired t-test procedure, with ∝< 0.05 was conducted. The results clearly indicated that MRMC-IWD outperformed modified IWD with a p-value of 1.89982E-09. MRMC-IWD therefore enhances the performance of modified IWD by improving the search capability through the subexploration of several regions pertaining to the search space, using multiple creeks. The master river then performs global exploration, based on the results returned by the creeks, in order to achieve the optimal results.

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

185

Table 4 The results of MRMC-IWD as compared with those of the modified IWD algorithm [4] and GCGA without local search [68]. The Error (%) denotes the percentage deviation of the average tour length from the best known results. Avg. indicates the average tour length. Data set

eil51 eil76 eil101 kroA100 kroA200 lin105 lin318

Best known tour length

426 538 629 21,282 29,368 14,379 42,029

Modified IWD [4]

GCGA [68] without local search

MRMC-IWD

Avg.

Error (%)

Avg.

Error (%)

Avg.

Error (%)

440.3 551.8 671.3 22298.5 32032.5 14801.8 47481.1

3.357 2.565 6.725 4.776 9.073 2.940 12.972

437 551 661 21,768 30,707 14,621 45,535

2.582 2.416 5.087 2.284 4.559 1.683 8.342

427.0 540.6 632.1 21426.5 29726.9 14429.1 42958.1

0.235 0.483 0.493 0.679 1.222 0.348 2.211

Table 5 The results of hybrid MRMC-IWD as compared with those of MRMC-IWD, GCGA with local search [68], improved ABC with local search [29] and PSO [29] . The Error (%) denotes the percentage of deviation of the average from the best known result, Avg. indicates the average tour length, “-“ indicates that result is not available in publications. Data set

eil51 eil76 eil101 kroA100 kroA200 lin105 lin318

Best known tour length

426 538 629 21,282 29,368 14,379 42,029

MRMC-IWD

Hybrid MRMC-IWD

GCGA with local search [68]

ABC with local search [29]

PSO [29]

Avg.

Error (%)

Avg.

Error (%)

Avg.

Error (%)

Avg.

Error (%)

Avg.

Error (%)

427.0 540.6 632.1 21426.5 29726.9 14429.1 42958.1

0.235 0.483 0.493 0.679 1.222 0.348 2.211

426.2 538.0 629.2 21282.0 29371.0 14379.0 42054.2

0.047 0.000 0.032 0.000 0.010 0.000 0.060

430 551 646 21,543 29,910 14,544 44,191

0.939 2.416 2.703 1.226 1.846 1.148 5.144

432 561 657 21,688 – – –

1.408 4.275 4.452 1.908 – – –

436 555 652 22071 – – –

2.347 3.160 3.657 3.707 – – –

Table 5 shows the comparison results between MRMC-IWD, as well as hybrid MRMC-IWD and those from GCGA [68] with local search, improved artificial bee colony (ABC) with local search [29], and PSO [29]. Note that "-" in Table 5 denotes that the results are not available in the respective publications. For all seven selected TSPLIB problems, hybrid MRMC-IWD achieves good results, which deviated from the best known tour lengths by an average of 0.021%. It performed better than MRMC-IWD, GCGA, ABC, and PSO. In addition, hybrid MRMC-IWD outperformed MRMC-IWD in terms of the solution quality and convergence rate, as shown in Fig. 2. The plots in Fig. 2(a)–(g) depict that hybrid MRMC-IWD performed better than MRMC-IWD and converged faster to the global optimal solutions. The use of IILS as the local search method to intensify the search process towards the local optimal solutions proved useful. Therefore, hybrid MRMC-IWD not only was able to provide diversity in the search process, but also to strike a balance between exploration and exploitation, and to accelerate convergence towards the optimal solutions through its local search method. 5.3. RSFS results The decision variables of the RSFS problem, i.e. the number of features, were divided randomly into C. groups. An equal number of decision variables were assigned to each group. Using k-means clustering was not useful owing to the required computation complexity in determining the rough set dependency among the features. To investigate the influence of C, a series of preliminary experiments with three RSFS data sets (i.e., CREDIT, DERM2, and WQ) were conducted. For these data sets, the modified IWD algorithm was not able to produce the optimal solutions [4]. The experiments were repeated 20 runs for each data set. Table 6 presents the reduct lengths and their associated frequencies (denoted as superscripts) from 20 runs. Note that C = 3 produced the best performance of MRMC-IWD, i.e. smaller and stable (almost the same reducts length for several runs) results. A thorough evaluation with C = 3 for the RSFS problem with 13 data sets was conducted. Table 7 summarizes the overall results. The results are compared with those from modified IWD and three optimization methods reported in Jensen and Shen [22], i.e., greedy hill climbing with forward selection (RSAR), RSAR with the genetic algorithm (GenRSAR), and RSAR with ant colony (AntRSAR). Being a deterministic method, RSAR produced only one reduct from the entire 20 runs. Because of the stochastic nature of MRMC-IWD and other methods, different reduct lengths (with the possibility of the same cardinality) could be produced from different runs. The results in Table 7 show that MRMC-IWD achieved the optimal reduct cardinality for all data sets, except DERM2 and WQ, which had a high degree of complexity [6]. RSAR and GenRSAR did not manage to produce the optimal results for most of the data sets, as compared with those from AntRSAR, modified IWD, and MRMC-IWD. Modified IWD was not able to provide the optimal reduct lengths for 6 out of 13 data sets. Furthermore, MRMC-IWD performed more stably as compared with

186

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

MRMC-IWD

500

Hybrid MRMC-IWD

490 Tour Length

470 460 450

550 500

440

450

430

400 0

420 0

100 Iteraons

200

300

100

MRMC-IWD

760

Hybrid MRMC-IWD

Tour Length

700 680 660 640 620 600 100 Iteraons

200

MRMC-IWD

25500 25000 24500 24000 23500 23000 22500 22000 21500 21000

300

0

MRMC-IWD

Hybrid MRMC-IWD

100 Iteraons

(c) eil101

200

300

(d) kroA100

Hybrid MRMC-IWD

MRMC-IWD

16500

35000

Hybrid MRMC-IWD

16000

Tour Length

34000 33000 32000

31000

15500

15000 14500

30000 29000

14000 0

100

Iteraons

200

300

0

100

(e) kroA200

Iteraons

200

(f) lin105 MRMC-IWD

52000

Hybrid MRMC-IWD

50000 Tour Length

Tour Length

720

36000

300

(b) eil76

740

0

200

Iteraons

(a) eil51

Tour Length

Hybrid MRMC-IWD

600

480 Tour Length

MRMC-IWD

650

48000 46000 44000 42000 40000

0

100

Iteraons

200

300

(g) lin318 Fig. 2. The convergence trends of MRMC-IWD and hybrid MRMC-IWD for the first 300 iterations of seven selected TSPLIB data sets.

300

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

187

Table 6 The results of MRMC-IWD for three RSFS data sets with various C settings. Data set

Features

C=1

C=2

C=3

C=4

CREDIT DERM2 WQ

20 34 38

8(3) 9(15) 10(2) 9(16) 10(4) 13(4) 14(15) 15(1)

8(16) 9(4) 8(10) 9(10) 13(16) 14(4)

8(20) 8(5) 9(15) 12(13) 13(7)

8(19) 9(1)) 8(12) 9(8) 12(7) 13(11) 14(2)

Table 7 The results of MRMC-IWD as compared with those from the modified IWD algorithm [4] and other models published by [22]. Data set

Features

RSAR [22]

GenRSAR [22]

AntRSAR [22]

Modified IWD [4]

MRMC-IWD

M-of-N Exactly Exactly2 HEART VOTE CREDIT MUSHROOM LED LETTERS DERM DERM2 WQ LUNG

13 13 13 13 16 20 22 24 25 34 34 38 56

8(20) 9(20) 13(20) 7(20) 9(20) 9(20) 5(20) 12(20) 9(20) 7(20) 10(20) 14(20) 4(20)

6(6) 7(12) 6(10) 7(10) 10(9) 11(11) 6(18) 7(2) 8(2) 9(18) 10(6) 11(14) 5(1) 6(5) 7(14) 6(1) 7(3) 8(16) 8(8) 9(12) 10(6) 11(14) 10(4) 11(16) 16(20) 6(8) 7(12)

6(20) 6(20) 10(20) 6(18) 7(2) 8(20) 8(12) 9(4) 10(4) 4(20) 5(12) 6(4) 7(3) 8(20) 6(17) 7(3) 8(3) 9(17) 12(2) 13(7) 14(11) 4(20)

6(20) 6(20) 10(20) 6(20) 8(11) 9(9) 8(3) 9(15) 10(2) 4(20) 5(20) 8(13) 9(7) 6(8) 7(12) 9(16) 10(4) 13(4) 14(15) 15(1) 4(20)

6(20) 6(20) 10(20) 6(20) 8(20) 8(20) 4(20) 5(20) 8(20) 6(20) 8(5) 9(15) 12(13) 13(7) 4(20)

Table 8 Comparison between hybrid MRMC-IWD and MRMC-IWD. Note that column “Iterations” indicates the average number of iterations required to achieve the optimal solution. Data set

M-of-N Exactly Exactly2 HEART VOTE CREDIT MUSHROOM LED LETTERS DERM DERM2 WQ LUNG

Features

13 13 13 13 16 20 22 24 25 34 34 38 56

MRMC-IWD

Hybrid MRMC-IWD

Reducts-frequency

Iterations

Reducts-frequency

Iterations

6(20) 6(20) 10(20) 6(20) 8(20) 8(20) 4(20) 5(20) 8(20) 6(20) 8(5) 9(15) 12(13) 13(7) 4(20)

9.75 7.85 5.25 12 15.95 48.25 18.6 30.25 29.5 39.9 198.1 175.8 24.45

6(20) 6(20) 10(20) 6(20) 8(20) 8(20) 4(20) 5(20) 8(20) 6(20) 8(20) 12(20) 4(20)

2 2 2 2.4 2 3.1 2.2 3.3 2.4 2.6 3.4 5.4 2.4

the other methods. In general, the experimental results indicated that MRMC-IWD outperformed the other methods, including the modified IWD algorithm. For hybrid MRMC-IWD, the two-feature swap mechanism was used as the neighborhood structure of the IILS method to enhance the solutions locally. A series of experiments with the same 13 RSFS date sets was carried out. The results are summarized in Table 8. As can be observed, hybrid MRMC-IWD produced good results either in terms of the reduct size or the stability of the results for 20 runs. Compared with other state-of-the-art methods, hybrid MRMC-IWD produced the best results (i.e. the shortest and uniform reducts of 20 runs) for all RSFS problems. Again, the incorporation of a local search method into MRMC-IWD proved useful to yielding a better quality solution, with less computation time (number of iterations). 6. Conclusions In this paper, two new models, i.e., MRMC-IWD and hybrid MRMC-IWD, have been proposed as extensions of the modified IWD algorithm. MRMC-IWD comprises an ensemble of the modified IWD algorithms. It consists of a master river and multiple independent creeks, in order to exploit the exploration capability of the modified IWD algorithm. The hybrid MRMC-IWD model integrates a local search method (i.e., the IILS method) into MRMC-IWD, in order to improve the exploitation capability by focusing the search on a particular region of the search space; therefore achieving a balance between exploration and exploitation. In MRMC-IWD, a decomposition technique is first employed to decompose the entire problem into a few sub-problems. Each sub-problem is assigned to a creek, while the optimization of the entire problem is handled by the master river. A bilateral co-

188

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

operative scheme between the master river and multiple creeks is introduced, which is motivated by the divide-and-conquer strategy. On the other hand, hybrid MRMC-IWD further enhances the search capability of MRMC-IWD by using a local search. The performance of both MRMC-IWD and hybrid MRMC-IWD has been evaluated using a series of experiments pertaining to two combinatorial optimization problems, i.e. TSP and RSFS. A preliminary analysis has been conducted to select the appropriate number of creeks (C) in MRMC-IWD. In general, the results demonstrate that MRMC-IWD is able to balance between exploration and exploitation of the search space. Hybrid MRMC-IWD further improves the search process of MRMC-IWD in the local neighborhood region. From the experimental results, it is evidenced that both models are useful in producing good results in tackling optimization problems, as compared with other state-of-the-art methods in the literature. For future work, more experiments on other optimization problems can be conducted. Besides that, further investigations pertaining to the decomposition technique for constraint based optimization problems can be examined. The applicability of MRMC-IWD and hybrid MRMC-IWD to tackle multi-objective optimization problems can be investigated as well. Acknowledgment The first author acknowledges the Islamic Development Bank (IDB) for the scholarship to study the Ph.D. degree at the Universiti Sains Malaysia (USM). The authors would like to thank the anonymous referees for their careful reading of the paper and their valuable comments and suggestions, which greatly improved the paper. References [1] S. Abdullah, N.S. Jaddi, Great Deluge algorithm for rough set attribute reduction, Database Theory Appl. Bio-Sci. Bio-Technol. 118 (2010) 189–197. [2] M.R. Akbarzadeh Totonchi, Intelligent water drops a new optimization algorithm for solving the vehicle routing problem, in: Proceedings of Paper Presented at the International Conference on Electronics and Information Engineering, Turkey, ICEIE, 2010. [3] B.O. Alijla, L.C. Peng, A.T. Khader, M.A. Al-Betar, Intelligent water drops algorithm for rough set feature selection, Intelligent Information and Database Systems, Springer, 2013, pp. 356–365. [4] B.O. Alijla, L.-P. Wong, C.P. Lim, A.T. Khader, M.A. Al-Betar, A modified intelligent water drops algorithm and its application to optimization problems, Expert Syst. Appl. 41 (15) (2014) 6555–6569. [5] B.V. Babu, A. Nagar, K. Deep, M. Pant, J.C. Bansal, K. Ray, et al., Variable selection and fault detection using a hybrid intelligent water drop algorithm, in: Proceedings of the Second International Conference on Soft Computing for Problem Solving, SocProS, vol. 236, India, Springer, 2012, pp. 225–231. [6] Bache, K., & Lichman, M. (2013). UCI Machine Learning Repository. Irvine, CA: University of California, School of Information and Computer Science. See http://archive.ics.uci.edu/ml. [7] C. Bae, W.-C. Yeh, Y.Y. Chung, S.-L. Liu, Feature selection with intelligent dynamic swarm and rough set, Expert Syst. Appl. 37 (10) (2010) 7026–7032. [8] C. Blum, X. Li, Swarm Intelligence in Optimization, Springer Verlag, Berlin, 2008. [9] C. Blum, J. Puchinger, G.R. Raidl, A. Roli, Hybrid metaheuristics in combinatorial optimization: a survey, Appl. Soft Comput. 11 (6) (2011) 4135–4151. [10] C. Blum, A. Roli, Metaheuristics in combinatorial optimization: overview and conceptual comparison, ACM Comput. Surv. 35 (3) (2003) 268–308. [11] I. Boussaid, J. Lepagnot, P. Siarry, A survey on optimization metaheuristics, Inf. Sci. 237 (2013) 82–117. [12] M. Dash, H. Liu, Feature selection for classification, Intell. Data Anal. 1 (1-4) (1997) 131–156. [13] M. Dorigo, C. Blum, Ant colony optimization theory: a survey, Theor. Comput. Sci. 344 (2) (2005) 243–278. [14] M. Dorigo, G. Di Caro, Ant colony optimization: a new meta-heuristic, in: Proceedings of the Paper Presented at 1999 Congress on Evolutionary Computation, Washington, DC, 1999. [15] M. Dorigo, L.M. Gambardella, Ant colony system: a cooperative learning approach to the traveling salesman problem, IEEE Trans. Evol. Comput. 1 (1) (1997) 53–66. [16] H. Duan, S. Liu, X. Lei, Air robot path planning based on intelligent water drops optimization, in: Proceedings of the Paper Presented at the International Joint Conference on Neural Networks, IJCNN (IEEE World Congress on Computational Intelligence), 2008. [17] M.T. Hagh, H. Ebrahimian, N. Ghadimi, Hybrid intelligent water drop bundled wavelet neural network to solve the islanding detection by inverter-based DG, Front. Energy 9 (1) (2015) 78–90. [18] X. Han, X. Chang, L. Quan, X. Xiong, J. Li, Z. Zhang, et al., Feature subset selection by gravitational search algorithm optimization, Inf. Sci. 281 (0) (2014) 128–146. [19] K. Helsgaun, An effective implementation of the Lin–Kernighan traveling salesman heuristic, Eur. J. Oper. Res. 126 (1) (2000) 106–130. [20] Y. Hendrawan, H. Murase, Neural-intelligent water drops algorithm to select relevant textural features for developing precision irrigation system using machine vision, Comput. Electron. Agric. 77 (2) (2011) 214–228. [21] D.C. Hoang, R. Kumar, S.K. Panda, Optimal data aggregation tree in wireless sensor networks based on intelligent water drops algorithm, IET Wirel. Sens. Syst. 2 (3) (2012) 282–292. [22] R. Jensen, Q. Shen, Finding rough set reducts with ant colony optimization, in: Proceedings of the Paper Presented at the Workshop on Computational Intelligence, UK, 2003. [23] R. Jensen, Q. Shen, Computational Intelligence and Feature Selection: Rough and Fuzzy Approaches, vol. 8, Wiley-IEEE Press, 2008. [24] R. Jensen, A. Tuson, Q. Shen, Finding rough and fuzzy-rough set reducts with SAT, Inf. Sci. 255 (10) (2014) 100–120. [25] V. Kayvanfar, E. Teymourian, Hybrid intelligent water drops algorithm to unrelated parallel machines scheduling problem: a just-in-time approach, Int. J. Prod. Res. 52 (19) (2014) 5857–5879. [26] V. Kayvanfar, M. Zandieh, E. Teymourian, An intelligent water drop algorithm to identical parallel machine scheduling with controllable processing times: a just-in-time approach, Comput. Appl. Math. (2015), doi:10.1007/s40314-015-0218-3. [27] J. Kennedy, Stereotyping: improving particle swarm performance with cluster analysis stereotyping: improving particle swarm performance with cluster analysis, in: Proceedings of the Paper Presented at the Congress on Evolutionary Computation, 2000. [28] T.A. Khaleel, M.Y. Ahmed, Using intelligent water drops algorithm for optimisation routing protocol in mobile ad-hoc networks, Int. J. Reason.-based Intell. Syst. 4 (4) (2012) 227–234. [29] H.E. Kocer, M.R. Akca, An improved artificial bee colony algorithm with local search for traveling salesman problem, Cybern. Syst. 45 (8) (2014) 635–649. [30] J. Krause, J. Cordeiro, R.S. Parpinelli, H.S. Lopes, A survey of swarm algorithms applied to discrete optimization problems, in: X. Yang, Z. Cui, R. Xiao, A. Hossein Gandomi, M. Karamanoglu (Eds.), Swarm Intelligence and Bio-inspired Computation: Theory and Applications, Elsevier, 2013, pp. 169–191. [31] J. Lee, D.-W. Kim, Memetic feature selection algorithm for multi-label classification, Inf. Sci. 293 (1) (2015) 80–96. [32] K. Lenin, M.S. Kalavathi, An intelligent water drop algorithm for solving optimal reactive power dispatch problem, Int. J. Electr. Eng. Inf. 4 (3) (2012) 450–463. [33] M.-H. Lin, J.-F. Tsai, C.-S. Yu, A review of deterministic optimization methods in engineering and management, Math. Probl. Eng. 15 (2012). [34] M. Mafarja, S. Abdullah, Modified great deluge for attribute reduction in rough set theory, in: Proceedings of Paper Presented at the Eighth International Conference on Fuzzy Systems and Knowledge Discovery, FSKD, 2011. [35] S. Mahdavi, M.E. Shiri, S. Rahnamayan, Metaheuristics in large-scale global continues optimization: a survey, Inf. Sci 295 (20) (2015) 407–428.

B.O. Alijla et al. / Information Sciences 325 (2015) 175–189

189

[36] P. Maji, A. Ghosh, M.N. Murty, K. Ghosh, S. Pal, M. Kumar, et al., Simultaneous gene selection and cancer classification using a hybrid intelligent water drop approach, Pattern Recognition and Machine Intelligence, vol. 8251, Springer Berlin, Heidelberg, 2013, pp. 641–646. [37] R.T. Marler, J.S. Arora, Survey of multi-objective optimization methods for engineering, Struct. Multidiscip. Optim. 26 (6) (2004) 369–395. [38] M. Englert, H. Roglin, B. Vocking, Worst case and probabilistic analysis of the 2-opt algorithm, Algorithmica 68 (1) (2014) 190–264. [39] M. Mavrovouniotis, S. Yang, Ant algorithms with immigrants schemes for the dynamic vehicle routing problem, Inf. Sci. 294 (10) (2015) 456–477. [40] L. Moncayo-MartÃnez, D. Zhang, Optimisation of safety and in-transit inventory in manufacturing supply chains by intelligent water drop metaheuristic, in: Proceedings of the Paper Presented at the International Working Seminar on Production Economics, 2012. [41] P. Nagalakshmi, Y. Harish, R.K. Kumar, C. Jada, Combined economic and emission dispatch using intelligent water drops-continuous optimization algorithm, in: Proceedings of the Paper Presented at the International Conference on Recent Advancements in Electrical, Electronics and Control Engineering, ICONRAEeCE, 2011. [42] S.H. Niu, S.K. Ong, A.Y.C. Nee, An improved intelligent water drops algorithm for achieving optimal job-shop scheduling solutions, Int. J. Prod. Res. 50 (15) (2012) 4192–4205. [43] S.H. Niu, S.K. Ong, A.Y.C. Nee, An improved intelligent water drops algorithm for solving multi-objective job shop scheduling, Eng. Appl. Artif. Intell. 26 (10) (2013) 2431–2442. [44] Z. Pawlak, Rough sets, Int. J. Parallel Program. 11 (5) (1982) 341–356. [45] S.R. Rayapudi, An intelligent water drop algorithm for solving economic load dispatch problem, Int. J. Electr. Electron. Eng. 5 (2) (2011) 43–49. [46] G. Reinelt, TSPLIB: a traveling salesman problem library, ORSA J. Comput. 3 (4) (1991) 376–384. ˚ [47] L.S. Riza, A. Janusz, C. Bergmeir, C. Cornelis, F. Herrera, D. Ašle¸zak, et al., Implementing algorithms of rough set theory and fuzzy rough set theory in the R package “roughsets”, Inf. Sci. 287 (0) (2014) 68–89. [48] S. Salmanpour, H. Omranpour, H. Motameni, An intelligent water drops algorithm for solving robot path planning problem, in: Proceedings of the Paper Presented at the 14th International Symposium on Computational Intelligence and Informatics, CINTI, 2013. [49] S.C. Satapathy, S.K. Udgata, B.N. Biswal, D. Sensarma, K. Majumder, IWDRA: an intelligent water drop based QoS-aware routing algorithm for MANETs, in: Proceedings of the International Conference on Frontiers of Intelligent Computing: Theory and Applications, FICTA, Springer International Publishing, vol. 247, 2013, pp. 329–336. [50] H. Shah-Hosseini, Problem solving by intelligent water drops, in: Proceedings of the Paper presented at the IEEE Congress on Evolutionary Computation, Singapore, 2007. [51] H. Shah-Hosseini, Intelligent water drops algorithm: a new optimization method for solving the multiple knapsack problem, Int. J. Intell. Comput. Cybern. 1 (2) (2008) 193–212. [52] H. Shah-Hosseini, The intelligent water drops algorithm: a nature-inspired swarm-based optimization algorithm, Int. J. Bio-Inspired Comput. 1 (1) (2009) 71–79. [53] Shah-Hosseini, H. (2009b). Optimization with the nature-inspired intelligent water drops algorithm (pp. 297-320): InTech. [54] H. Shah-Hosseini, An approach to continuous optimization by the intelligent water drops algorithm, Procedia-Soc. Behav. Sci. 32 (2012) 224–229. [55] H. Shah-Hosseini, Improving K-means clustering algorithm with the intelligent water drops (IWD) algorithm, Int. J. Data Min. Model. Manag. 5 (4) (2013) 301–317. [56] Y. Shi, Particle swarm optimization: developments, applications and resources, in: Proceedings of the Paper Presented at the Congress on Evolutionary Computation, 2001. [57] N. Siddique, H. Adeli, Water drop algorithms, Int. J Artif. Intell Tools 23 (06) (2014) 1430002. [58] K. Smith-Miles, L. Lopes, Measuring instance difficulty for combinatorial optimization problems, Comput. Oper. Res. 39 (5) (2012) 875–889. [59] P.R. Srivastava, A cooperative approach to optimize the printed circuit boards drill routing process using intelligent water drops, Comput. Electr. Eng. 43 (2015) 270–277, doi:10.1016/j.compeleceng.2014.10.005. [60] Straub, J., Intelligent water drops for aerospace and defense applications, Paper Presented at the Machine Intelligence and Bio-inspired Computation: Theory and Applications VIII (SPIE 9119), 2014. [61] C. Sur, S. Sharma, A. Shukla, Multi-objective adaptive intelligent water drops algorithm for optimization & vehicle guidance in road graph network, in: Proceedings of the Paper Presented at the International Conference on Informatics, Electronics & Vision, ICIEV, 2013. [62] H. Vafaie, I.F. Imam, Feature selection methods: genetic algorithms vs. greedy-like search, in: Proceedings of the Paper Presented at International Conference on Fuzzy and Intelligent Control Systems, 1994. [63] J. Vesterstrom, R. Thomsen, A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems, in: Proceedings of the Paper Presented at the Congress on Evolutionary Computation, CEC, 2004. [64] C. Wang, L. Deng, G. Zhou, M. Jiang, A global optimization algorithm for target set selection problems, Inf. Sci. 267 (20) (2014) 101–118. [65] X. Wang, J. Yang, X. Teng, W. Xia, R. Jensen, Feature selection based on rough sets and particle swarm optimization, Pattern Recognit. Lett. 28 (4) (2007) 459–471. [66] T. Weise, M. Zapf, R. Chiong, A.J. Nebro, Why is optimization difficult?, Nature-Inspired Algorithms for Optimisation, Springer, 2009, pp. 1–50. [67] X.-F. Xie, J. Liu, Multiagent optimization system for solving the traveling salesman problem (TSP), IEEE Trans. Syst. Man Cybern. Part B: Cybern. 39 (2) (2009) 489–502. [68] J. Yang, C. Wu, H.P. Lee, Y. Liang, Solving traveling salesman problems using generalized chromosome genetic algorithm, Prog. Nat. Sci. 18 (7) (2008) 887–892. [69] A.R. Yildiz, Optimization of cutting parameters in multi-pass turning using artificial bee colony-based approach, Inf. Sci. 220 (20) (2013) 399–407. [70] L. Yu, H. Liu, Efficient feature selection via analysis of relevance and redundancy, J. Mach. Learn. Res. 5 (2004) 1205–1224. [71] M.F. Zaiyadi, B. Baharudin, A proposed hybrid approach for feature selection in text document categorization, World Acad. Sci. Eng. Technol. 48 (2010) 111–116. [72] J.-h. Zhou, C.-m. Ye, X.-h. Sheng, Research on permutation flow-shop scheduling problem by intelligent water drop algorithm, Comput. Sci. 9 (2013) 056.