A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm

Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

Contents lists available at ScienceDirect

Neurocomputing journal homepage: www.elsevier.com/locate/neucom

A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm Xin-Yuan Luan, Zhan-Pei Li, Ting-Zhang Liu n School of Mechatronic Engineering and Automation, Shanghai Key Laboratory of Power Station Automation Technology, Shanghai University, Shanghai 200072, China

art ic l e i nf o

a b s t r a c t

Article history: Received 7 November 2014 Received in revised form 4 June 2015 Accepted 5 June 2015 Chennai Guest Editor

Attribute Reduction (AR) is an important preprocessing step for data mining. AR based on rough set is an efficient method. Its reduction performance has been verified to be better or comparable with other methods in large amount of works, but existing reduction algorithms have some problems such as slow convergent speed and probably converging to a local optimum. A novel attribute reduction algorithm based on Artificial Fish Swarm Algorithm (AFSA) and rough set is proposed. For AFSA has a slow convergence rate in the later phase of iterations, normal distribution function, Cauchy distribution function, multi-parent crossover operator, mutation operator and modified minimal generation gap model are adopted to improve AFSA. The attribute reduction algorithm based on improved AFSA and rough set takes full advantages of the improved AFSA and rough set,which are faster, more efficient, simpler, and easier to be implemented. Datasets in the UC Irvine (UCI) Machine Learning Repository are selected to verify the aforementioned new method. The results show that above algorithm can search the attribute reduction set effectively, and it has low time complexity and the excellent global search ability. & 2015 Elsevier B.V. All rights reserved.

Keywords: Attribute reduction Rough set AFSA Cauchy distribution

1. Introduction The data mining, also known as knowledge discovery in database, includes extracting/mining knowledge from large amounts of data, discovering new patterns, and predicting the future trends. The rough set theory is an important means of data mining [1–3]. Nowadays, with the explosive growth of web information, the webpage classification faces the great challenge. Wang [4] proposed a classification approach for less popular webpages based on rough set model. Rough set can only deal with attributes of a specific type in the information system by using a specific binary relation [5]. In order to deal with the missing data and incomplete information in real decision problems, Liu [6] presents a matrix based incremental approach in dynamic incomplete information systems. It is very difficult to select significant genes closely related to classification because of the high dimension and small sample size of gene expression data. Rough set has been successfully applied to gene selection, as it selects attributes without redundancy and deals with numerical attributes directly [7]. Rough set has achieved encouraging results which extract target features well and mine data statistical correlation in the financial, industrial production, marketing information system and other fields. Hence, the rough set theory is becoming a hot research spot [8,9]. n

Corresponding author. Tel.: þ 86 021 56331563. E-mail address: [email protected] (T.-Z. Liu).

As known to all, some attributes are useful, while some attributes are redundant or meaningless, which not only occupy extensive computing resources, but also seriously impact the decision making process [10,11]. Attribute Reduction (AR) removes redundant or insignificant information and retains the classification ability of the information system same as before. It is viewed as an important preprocessing step for pattern recognition and data mining. It is also one of the important applications in the rough set theory [12,13]. Rough set method has some advantages: it has explicit stopping criterion and no parameters. And its reduction performance is comparable with other methods or even better [14,15]. Most of researches are focused on attribute reduction by using rough set [16,17]. Some methods of attribute reduction are based on discernibility matrix [1,18,19], some ones on neighborhood rough set model [20– 22] and some ones on the variable precision rough set model [23,24]. Minimal reduction problem is even NP (non-deterministic polynomial)-hard problem [18], where the number of attributes is smallest among all possible reductions [14]. Because heuristic algorithms can be used to solve many kinds of NP-hard problems, recently, heuristic attribute reduction algorithm is the main research direction in the field of attribute reduction. They are usually implemented through a certain measure to evaluate the significance of attributes and a heuristic searching strategy [10]. Heuristic attribute reduction algorithms include the algorithm based on attribute significance [25], algorithm based on dependency of attribute [26], and algorithm based

http://dx.doi.org/10.1016/j.neucom.2015.06.090 0925-2312/& 2015 Elsevier B.V. All rights reserved.

Please cite this article as: X.-Y. Luan, et al., A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.06.090i

X.-Y. Luan et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

2

on attribute frequency. Specially, the reduction algorithm based on Genetic Algorithm (GA) optimization is proposed by Wrobleswski and other researchers [27,28], and the Particle Swarm Optimization (PSO) algorithm is introduced into reduction algorithm [29,30]. However, such approaches are not difficult to fall into local optimal solution as well. Therefore, searching for a fast and efficient reduction algorithm continues to be one of the major concerns in the field of rough set theory. According to the above analysis, a novel attribute reduction algorithm based on Artificial Fish Swarm Algorithm (AFSA) and rough set is proposed in this paper. It modifies the heuristic searching strategy and can find the minimal reduction more quickly. AFSA is a new bionic optimization algorithm, which searches for an optimal solution in the target solution space by simulating fish preying, swarming, random and following behaviors [31]. AFSA has some advantages: it can use the target function as algorithm evaluation function directly, and it can get an appropriate solution quickly. Nevertheless AFSA has a slow convergence rate in the later phase of iterations, so several new approaches are adopted to improve AFSA. Normal and Cauchy distribution functions are utilized to optimize visual scope, try number and step. Based on these functions, AFSA can get better balance between convergence speed and solution accuracy. Crossover and mutation operators are introduced to enhance population diversity. Modified Minimal Generation Gap model is employed to retain the elite individual and remove the worst. Preying behavior is removed from swarming and following behaviors to cut down the amount of calculation. Numerous experiments demonstrate that improved AFSA rough set attribute reduction (IAFSA-RSAR) algorithm has higher search efficiency and lower computational complexity than others.

2. Background 2.1. Basic notions of rough set theory Redundant attributes add both complexity and overhead of calculation. So, redundant attributes are necessary to be removed, in the meantime, the ability of information classification is kept. Rough set theory is a useful tool of reduction [32]. Let U be a certain set called the universe, and X be a certain subset of U. The least composed set in A containing X will be called the best upper approximation of X in A, in symbols AprA (X); the greatest composed set in A contained in X will be called the best lower approximation of X in A, in symbols AprA (X). The universe U is divided into three subsets [12,32]

m = r (P , Q ) = |POSP (Q )| /|U | =

∪

x∈U/X

A (x)

If BND(X) set is not a null set; the X set is called rough set.

(4)

2.2. AFSA analysis Artificial Fishes (AF) are generated by random function in AFSA, and the optimal solution is found out through iterations. At every iteration process, AFs update the maximum fitness value in the bulletin board by preying, swarming and following behaviors. Preying behavior is AFSA basic behavior, described as formula (5). The Xi|next value is updated by random function sometimes, this method lets AFSA escape from the local optimal solution. xi is the AF current state; xj is a random AF in the visual scope. If Yj 4Yi, the xi is replaced by xj directly. Otherwise, the xi is replaced by another random xj. Defining the prey() function as formula (5). Swarming behavior is described as formula (6). xi is AF current state; xc is the AF in the center of current visual scope; and nf is the AF numbers in current visual scope. If Yc/nf 4 δYi, it means that there is higher food consistency in the center of current visual scope, meanwhile, it is not too crowded then AF goes toward a step to the center. Fitness value Y (xi) is calculated. Y (xi) is compared with the value in the bulletin board, it replaces the latter if it is greater than latter. Following behavior is described as formula (7) xi is AF current state; xmax is the AF with the greatest food consistence among companion AFs in the current visual scope. nf is the AF number in current visual scope. If Ymax/nf 4 δYi, this means that xmax has the higher food consistence in the center of current visual scope, meanwhile, it is not too crowded, then AF goes toward a step to xmax. The fitness value Y (xi) is calculated. Y (xi) is compared with the value in the bulletin board; it replaces latter if it is greater than latter. Swarming and following behaviors help AFSA locate the optimal solution. For the AFSA, if visual scope is greater, global search ability will be stronger and convergence speed will be faster. Oppositely, if visual scope is smaller, the local search ability will be stronger. If the updated step is greater, convergence speed will be faster. If the updated step is smaller, convergence speed will be slower, but the precision will be higher [31].

⎧ X−Xi ⎪ Xi + Random (Step ) j ‖Xj − X i‖ =⎨ ⎪ ⎩ Xi + Random (Step )

if (Yi < Y j ) else

(5)

(1)

(2)

(3) X's boundary regions

BND (X ) = AprA (X ) − AprA (X )

| P (x)| |U |

POSp(Q) is Q's P positive region in the U space. m (0 rm r1) is called dependency degrees between space Q and P, in symbols P⇒k Q where the symbol | | means element number of set. If m ¼0, the set Q is completely independent of the set P. If 0 om o 1, Q is partly independent of P. If m ¼1, Q is totally dependent on P [33].

(2) X's negative regions

NEG (X ) = POS (~X ) = AprA (X )

∑ x∈U/Q

Xi | next

(1) X's positive regions

POS (X ) = AprA (X ) =

Definition 1. Let U be a certain set called the universe, and let R be an equivalence relation on U. The pair K ¼(U, R) will be called an approximation space. Sets P, QDR, then

(3)

Xi | next

⎧ ⎛Y ⎞ X c− X i ⎪ Xi + Random (Step ) if⎜ c > δYi⎟ ⎠ =⎨ ‖X c − X i‖ ⎝ nf ⎪ else ⎩ prey()

⎧ X max−Xi ⎪ Xi + Random (Step ) Xi | next = ⎨ ‖X max −Xi ‖ ⎪ ⎩ prey()

(6)

⎛Y ⎞ if⎜ max > δYi⎟ ⎝ nf ⎠ else

(7)

Please cite this article as: X.-Y. Luan, et al., A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.06.090i

X.-Y. Luan et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

where Xi is the current fish status, Xj is the next fish status, Yc is the fitness function value at current neighborhood center, Ymax is the maximum fitness function value at current neighborhood, δ is the crowd level in the current neighborhood, and nf is the artificial fish number in the current neighborhood.

1.0

0.6

0.4

3.1. Improvement of visual scope and step When the visual scope is smaller, preying and random behaviors are more prominent. If the visual scope is greater, swarming and following behaviors are more prominent. The step is gradually increased and the number of iteration will reduce at the same time. As the step decreases gradually, the accuracy of the solution is increased to a certain degree. Therefore, the bigger visual scope is more likely to find out the global optimal solution. The smaller visual scope is more likely to find out the local optimal solution. Thus, a three-parameter Lorentzian function (formula (8)) is introduced into AFSA to improve the visual scope. The Lorentzian function is one of the Cauchy distribution forms. The Lorentzian function is selected as the operator of visual scope self-adaptive variety. Visual scope is updated by formula (9) automatically. At the beginning of iteration, greater visual scope is adopted to enhance the global search ability and the convergence speed of the algorithm, and the coarse search is implemented in a greater scope. In the later phase of AFSA, smaller visual scope is adopted to improve the local search ability of AFSA and the accuracy of optimal solution. However, if visual scope is too small, the algorithm is easy to encounter local optimization.

⎡ ⎤ γ2 ⎥ f (x; x 0 , γ , I ) = I ⎢ 2 2 ⎣ (x−x 0 ) + γ ⎦

(8)

0.2

0.0

(9)

where I is the height of the peak, gen is the iteration generation, GEN is the maximum setting iteration number, Visuali is the current visual scope value, and Visualnext is the next visual scope value. For the sake of balance between the speed of iteration and the accuracy of the solution, the variable step adaptive operator is introduced into AFSA. The operator is a normal distribution function (formula (10)) [34].

g (x) = e

−πx 2

Stepnext = Stepi × g (gen/GEN )

(10) (11)

The step is updated by formula (11) self-adaptively. At the beginning of iteration, normal distribution function operator has a slower rate of descent than the linear operator (Fig. 1), and AFSA has a faster moving speed. In the later phase of AFSA, normal distribution function operator has a smaller step, where AFSA is more likely to obtain the global optimal solution. Visual scope is not as small as possible. When visual scope is small to a certain extent, the rule of iteration number variety is irregular. Hence, the variety of visual scope needs a slower rate of descent. The probability density function curves of Cauchy distribution and normal distribution are shown in one figure (Fig. 1), it is obvious that Cauchy distribution has a much slower rate to zero. This is the reason why Cauchy distribution is selected as the variable visual scope adaptive operator.

0

1

2

x

3

4

Fig. 1. The function f (x; 0, 2, 1) and g(x).

3.2. Crossover and mutation operator The basic AFSA has some obvious weaknesses: the search speed is fast at the beginning and very slow in a large flat area. Moreover, it easily runs into the local optimal solution [35]. In this paper, multi-parent crossover operator and the new mutation operator are introduced into AFSA. The variety of the population and the ability of AFSA breaking away from the local optimal solution are effectively improved. The crossover operator comes from Guo's algorithm [36]. Numerous individuals are selected to mingle and recombine based on formula (12). Because of nonconvex combination technology of Guo's operator, the algorithm has good ergodicity of the random search. The blind angle of the search is no longer available in the subspace. m

X⁎ =

m

∑ Xi ai ∑ ai = 1 i=1

Visualnext = Visuali × f (4 × gen/GEN; 0, 2, 1)

f(x;0,2,1) g(x) y=1-x

0.8

Value

3. Improvment of AFSA

3

− 0.5 ≤ ai ≤ 1.5

i=1

(12)

where Xi is an individual in the original population, m is the number of selected individuals, X* is the new individual. The Cauchy mutation operator is used in evolutionary programming [37], and it has better performance. Here, it is introduced into AFSA to improve the population diversity of AF and convergence speed (formula (13)). Thus, the algorithm gains the better ability to avoid falling into local optimal solution.

Stepi′ = Stepi × e

2 N (0,1)

Xi′ = Xi + Stepi′ × C (0, 1)

(13)

where N (0, 1) is the random function of normal distribution, C (0, 1) is the random function of Cauchy distribution, Stepi is the current step size value, Stepi‵ is the step size value after mutation, Xi is the current position value, and Xi‵ is the new position value after mutation. Minimal Generation Gap (MGG) model is proposed by Satoh [38] firstly, which obtains good balance between exploration and exploitation ability. In the attribute reduction algorithm, the calculation of dependency degree is very time-consuming. Modified MGG model is described as follows. Algorithm 3.1. Modified MGG model algorithm. Input: The individual of minimum fitness value is named as Xmin and the individual of maximum fitness value is named as Xmax in the original population. The Xrandom is selected randomly in the original population excluding Xmax and Xmin. Y(Xmax) means the fitness value of Xmax.

Please cite this article as: X.-Y. Luan, et al., A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.06.090i

X.-Y. Luan et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

4

X1* and X2* are generated by the crossover operator (formula (12)). Based on X1* and X2*, X1‵ and X2‵ are generated by the mutation operator (formula (13)). ‵ ‵ If ( X1‵ > X2‵ ) { Xbetter = X1‵; Xworse = X2‵ ;} ‵ ‵ Else { Xbetter = X2‵ ; Xworse = X1‵;} End if ‵ If(Y({ Xbetter )4Y(Xmax})) ‵ {Xmax ¼ Xbetter } End if ‵ If(Y( Xworse )4 Y(Xmin)) ‵ ‵ {Xmin ¼ Xworse ; Xrandom ¼ Xbetter ;} ‵ Else if(Y({ Xbetter ) 4Y(Xmin})) ‵ {Xmin ¼ Xbetter ;} End if End if Output: Optimal Xmax and Xmin The superior individual is reserved and worst individual is eliminated in every generation.

3.3. Improvement of AF behaviors A new position Xj is selected randomly in current visual scope. If the fitness value of Xj is larger than the fitness value of Xi, which is the current position, the Xi is replaced by Xj directly. The convergence rate of the algorithm is improved by adding try number, but the running time of the algorithm also is increased. If try number is reduced, algorithm is more likely to jump out of local optimization. However, convergence speed of the algorithm is slow. Thus, β operator is applied to control the variety of try number.

⎧ 0.2 gen/GEN ≥ 0.8 β=⎨ ⎩ (1 − g (gen/GEN )) else try _numbernext = try _numberi *β

(14)

where gen is iteration generation, GEN is the maximum setting iteration number, try_numberi is the current try number value, and try_numbernext is the next try number value. In order to decrease the calculation amount in the process of swarming and following, new acceleration method is adopted. The swarming and following behaviors include preying behavior in basic AFSA. Preying behavior is removed from swarming and following behaviors. The approach enhances the efficiency of search and reduces the time of calculating [39].

3.4. Integer arithmetic The shift is defined as the increment of Xi. Because integer encoding method will be adopted in IAFSA-RSAR, solution space is the integer type. The original Xi increment may be decimal, and it has the obvious effect on searching efficiency. If the shift is a decimal, shift will be zero after the shift is rounded. This implies that one moving chance is wasted. So by setting minimum shift as 1, the algorithm has an actual moving. The new Xi increment method is shown in formula (15).

⎧ X + round(shift ) + 1 if (0 < shift ) i Xi | next = ⎨ ⎩ Xi −round( shift ) − 1 else ⎪

⎪

(15)

Ygenþ 2, Ygenþ 1, Ygen are the best fitness value of solutions which are searched by the algorithm at the (genþ2)th iteration, the (genþ1)th iteration and the genth iteration. If the differences of the best fitness value of solutions among these 3 iterations are less than ε, algorithm iteration stops. Inequation is shown in formula (16).

⎧ Y + 2−Y + 1 < ε ⎪ gen gen ⎨ ⎪ ⎩ Ygen + 1−Ygen < ε

(16)

4. IAFSA-RSAR algorithm 4.1. Encoding method Space U's attribute set is Q¼{q1, q2, q3…qn}. By adopting binary encoding method, attributes can be expressed as binary strings p which are combinations of 0 and 1,and p¼ {p1, p2, p3…pk}, k∈[1 n]. When pk ¼0, this means attribute qk is not selected. When pk ¼ 1, this means attribute qk is selected. In this way, the attributes can be converted into the data which AFSA can deal with. For example, attribute numbers are 4, n ¼4. p ¼{0,1,1,0}, this mean: q1 and q4 are not selected, q2 and q1 are selected. So, the binary string {0,1,1,0} is equivalent to attribute set {q2, q3}. Number of condition attribute set C is n. So, C's attribute subset numbers are 2n  1, then this information system's minimal attribute reduction set is within these 2n 1 subsets. AFSA search range is [1 2n  1], and solution is an integer value. 4.2. Fitness function Two issues are considered when attribute reduction set is searched for decision tables 1. New attribute set retains the accuracy of classification. 2. New attribute set has minimal attributes. So, the new fitness function must meet those 2 issues. That is the reason why the new fitness function in this paper includes two parts, as shown in formula (17)

Fitness = k +

|C|−|p| |C |

(17)

In the first part, symbol k means the decision attribute dependence of condition attribute, as shown in formula (4). This part is utilized to validate whether the selected attribute set is a reduction attribute set or not. When the value of k is 1, this set is a reduction attribute set at this time. In the second part, the |C| means the number of all attributes in condition attributes set C, the |p| means the number of attributes in the selected attribute subset. This part is utilized to validate whether attributes are less in the selected attribute set or not. The bigger fitness value is, the higher accuracy of the subset is, the less the numbers of redundant attributes are. Simultaneously, the fitness value is more likely to be preserved in AFSA. This trend would be doomed after several iterations, and then the AFSA finds out the minimal reduction attribute set quickly. 4.3. Algorithm description

Preparation: Binary decision table is encoded based on the above encoding method. Input: A decision table S ¼(U,Q,V,F). Set U is the universe, Q¼C∪D, set C is the condition attribute set, set D is the decision attribute set. Output: Reduction attribute set of decision table. IAFSA-RSAR algorithm process is described as follows (see Fig. 2):

Please cite this article as: X.-Y. Luan, et al., A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.06.090i

X.-Y. Luan et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

5

number of calculated values, and provides the query of the calculation values.

4.4. Algorithm analysis The reduction algorithm based on discernibility matrix is the very common reduction algorithm. Hu's [1,19] time complexity of the reduction algorithm based on discernibility matrix is O (n2  m  log m). The time complexity of reduction algorithm based on GA [27] is O (Gen  w  n  m2).The time complexity of reduction algorithm based on PSO [30] is O (Gen  w  n  m2). A conclusion comes from paper [9]: the time complexity of solving POSc(D) is O (n  m  log m) where, C is the condition attribute set, D is the decision attribute set, n is the number of condition attributes, m means all attribute numbers of the universe set. So, the time complexity of attribute dependency also is O (n  m  log m). By defining Gen as the number of iterations, w as the number of population, and l as the number of core attribute population, the time complexity of IAFSA-RSAR is O (Gen  (w-l)  n  m  log m). So, comparing with the above algorithms (Table 1), the new algorithm is manifestly faster. Typically, the calculation of equivalence classes is the most timeconsuming task, which has a great deal of repetition calculations.

5. Results and discussion In order to verify the correctness and effectiveness of IAFSARSAR algorithm, several representative samples are selected. Several kinds of reduction algorithms run in the MATLAB R2010a. Computer configurations are as follows: PC CPU is Pentium G2030@3 GHz, RAM is 2G bytes and OS is Windows XP.

Table 2 Decision table DT-1. U

a

b

c

d

e

F

U

a

b

c

d

e

f

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

1 2 3 4 5 6 7 8 9 10 11 11 13 13 15 15 17 17

1 1 3 3 5 5 7 7 9 9 11 12 13 14 15 16 17 18

1 1 3 3 5 5 7 7 9 9 11 11 13 13 15 15 17 17

1 1 3 3 5 5 7 7 9 9 11 11 13 13 15 15 17 17

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

19 20 21 22 23 23 25 25 27 27 29 29 31 31 33 33 35 35

19 19 21 21 23 24 25 26 27 28 29 30 31 32 33 33 35 35

19 20 21 22 23 24 25 26 27 27 29 29 31 31 33 34 35 36

19 19 21 21 23 23 25 25 27 27 29 29 31 31 33 33 35 35

19 19 21 21 23 23 25 25 27 27 29 29 31 31 33 33 35 36

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

6.2 6.0

Fitness value

Heuristic-GA-RSAR IAFSA-RSAR

Optimal solution

5.8 5.6 5.4 5.2 5.0 4.8 4.6 Fig. 2. The flow chart IAFSA-RSAR.

0

2

4

6

8

10

12

14

16

18

20

Generations

Therefore, a look-up table is created. Look-up table records a large

Fig. 3. The comparison of optimal solution. Table 1 Time complexity.

Table 3 Comparison with other methods for sample 1.

Algorithm

Time complexity

Hu's [19] GA-RSAR [27] PSO-RSAR [30] IAFSA-RSAR

O O O O

(n2  m  log m) (Gen  w  n  m2) (Gen  w  n  m2) (Gen  (w-l)  n  m  log m)

Algorithm

Iterations

Reduction result

GA-RSAR [27] Heuristic-GA-RSAR [28] IAFSA-RSAR

30 20 1

{b, c} {b, c} {b, c}

Please cite this article as: X.-Y. Luan, et al., A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.06.090i

X.-Y. Luan et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

6

Table 4 Comparison with other methods for sample 2–4. Name

Soybean-large Wine Zoo

Attribute

Instance

GA-RSAR [27]

Algorithm [28]

number

number

Iterations

Time/s

Iterations

Time/s

Iterations

Time/s

35 13 17

307 178 101

450 450 38

43501 41420 34.581

4 50 30 24

42234 1128.3 14.336

16 12 7

1226.6 447.42 6.142

IAFSA-RSAR AFSA-RSAR

The optimal solution of Zoo moving process

IAFSA-RSAR

The optimal solutions of soybean Gen:16 X:28399697920 Y:6.2

6.35 6.30

6.3254

6.25

7

Fitness value

6.20

6

6.15

5

6.10

4

Y

6.05

3

6.00 5.95 5.90

3

2 2

1 0

2

4

6

8

10

12

14

16

18

20

0

Generations

x 10

10

X 0

5

10

15

20

25

1

Gen

Fig. 4. The comparison of convergence speed.

Fig. 6. The changing of optimal solution by IAFSA-RSAR for the Soybean-large set. The optimal solutions of Wine

Gen:12 X:5127 Y:6.7

Table 5 Optimal results of attribute reduction.

7 6

Name

Attributes

GA-RSAR attributes

PSO-RSAR attributes

IAFSA-RSAR attributes

Soybean-large Wine Zoo

35 13 17

10 5 6

11 5 6

9 5 5

5

Y

4 3 2 1 0

5

10

15

5000 4000 3000 2000 X 1000

20

Gen Fig. 5. The changing of optimal solution by IAFSA-RSAR for the Wine set.

into the minus. IAFSA-RSAR algorithm finds out the optimal solution at the very beginning. The results also are presented in Fig. 3. After several iterations, the optimal solution is still not changed. This result is the same as paper [27,28], which is shown in Table 3. These results show that IAFSA-RSAR algorithm has faster convergence speed and retains the accuracy of classification. The optimal solution is obtained at the first iteration in IAFSA-RSAR algorithm (in Fig. 3).

5.1. Sample 1

5.2. Sample 2–4

The decision table is shown as in Table 2 [11], which has 36 instances and 6 attributes. Set {a, b, c, d, e} is the condition attribute set. Set {f} is the decision attribute set. According to the encoding method in Section 4.1, the solution space is [1,31]. For the IAFSA-RSAR algorithm, fish number N is 20, difference value ε is 0.01, moving step is 0.5, visual scope is 5, crowd level δ is 0.68, maximum trying number is 20 and maximum iteration is 5. After several runs of IAFSA-RSAR algorithm, optimal fitness value is 6.2, optimal solution is 12, binary code is {0 1 1 0 0}. Minimal attribute reduction set is {b, c}. The performances of the heuristic GA rough set attribute reduction (GA-RSAR) algorithm are presented in Fig. 3, which takes 20 iterations to find out minimal attribute reduction set. Default GA function in the GA tool box is utilized to search for the minimum of the objective function, then the objective function in this case need to be converted

For further evaluating the IAFSA-RSAR algorithm, 3 decision tables are selected in the UC Irvine (UCI) Machine Learning Repository. Instance and attribute numbers both have larger difference among these decision tables. Using the same encoding method as sample 1, these solution spaces are [1 34,359,738,367], [1 8191] and [1 131,071]. After finding out core attribute numbers l based on discernibility matrix, solution space is shrunk in size [1 2n-l]. For IAFSA-RSAR algorithm, Fish number N is [50 500], difference value ε is 0.01, moving step is [2 400], visual scope is [5 10,000], crowd level δ is 0.68, maximum try number is 20 and maximum iteration is 50. The IAFSA-RSAR algorithm has higher efficiency of attribute reduction than another two algorithms [27,28].These results are shown in Table 4. Table 4 shows that with the increase of instance numbers or attribute numbers, the traditional GA's time complexity increases

Please cite this article as: X.-Y. Luan, et al., A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.06.090i

X.-Y. Luan et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

7

Table 6 Comparison with other population-based algorithms. Name

Attributes

Instances

GA-RSAR average attributes

PSO-RSAR average attributes

AFSA-RSAR average attributes

IAFSA-RSAR average attributes

Soybean-small Cong. voting Breast cancer

35 16 32

47 435 569

2.2 11.2 13.5

2 9.2 13.2

2 9.3 12.6

2 9.1 11

sharply. Conversely, IAFSA-RSAR algorithm deceases the time complexity and reduces iterations. IAFSA-RSAR algorithm can effectively search for attribute reduction set for many kinds of decision tables,and it has low time complexity and strong global search ability. The Fig. 4 shows the movement of optimal solution for the Zoo set. IAFSA-RSAR algorithm gets maximum fitness value 6.3254, when solution is 13,576 at the 7th generation. Attributes are reduced from 17 to 5 after attribute reduction. AFSA-RSAR algorithm reaches maximum fitness value at the19th generation. The optimal solution of IAFSA-RSAR algorithm moves faster than AFSA-RSAR algorithms. This is because modified MGG model algorithm can quicken the searching process and improve the evolutionary efficiency. These comparisons demonstrate that IAFSA-RSAR algorithm has faster convergence speed. At the 12th generation, IAFSA-RSAR algorithm finds out the optimal solution for the Wine set (see Fig. 5). The best fitness value is 6.7 when solution is 5127. Attributes are reduced from 13 to 5 after attribute reduction. For the Soybean-large set,the optimal solution of IAFSA-RSAR algorithm is shown in Fig. 6. At the 16th generation, optimal solution 28,399,697,920 is obtained and the fitness value is 6.2. Attributes are reduced from 35 to 9 after attribute reduction. GA-RSAR and PSO rough set attribute reduction (PSO-RSAR) algorithms are from paper [27–30]. After several runs of these algorithms, optimal results of attribute reduction are summarized in the Table 5. IAFSA-RSAR algorithm has minimal attribute reduction set in the Table 5. Another 3 UCI datasets are selected to evaluate IAFSA-RSAR algorithm further. Table 6 shows that average attributes are average optimal results of attribute reduction with one algorithm running 10 times. The comparison results demonstrate that the attribute reduction performance of IAFSA-RSAR algorithm is better. IAFSA-RSAR algorithm has smaller attribute reduction set. The more the data in the dataset, the more the advantages for IAFSA-RSAR algorithm.

6. Conclusions To resolve the problems of low solution precision and slow resolving speed by existing reduction algorithms,an improved AFSA is introduced into rough set attribute reduction algorithm in this paper. IAFSA-RSAR algorithm takes full advantage of the improved AFSA and rough set, which is faster, more efficient, simpler, and easier to implement. As the AFSA has a slow convergence rate in the later phase of iterations, several approaches are adopted to improve AFSA. Normal and Cauchy distribution functions are employed to optimize visual scope, try number and step. Crossover operator, mutation operator and modified MGG model are adopted to enhance population diversity and remove the worst individual. These methods improve the convergence speed significantly. Numerous test results show that IAFSA-RSAR algorithm can obtain not only the fast convergence speed, but also the high accuracy of attribute reduction. Attribute reduction can effectively improve the recognition accuracy and the running speed for information systems. IAFSA-

RSAR has not only the theoretical significance but also the practical value. About the future work, because IAFSA-RSAR algorithm can only improve the heuristic searching strategy, with the view of characteristics of time-consuming calculation of individual fitness, new calculation methods are to be further developed.

Acknowledgments This work is supported by National Natural Science Foundation of China under Grant 61273190.

References [1] J. Dai, Rough set approach to incomplete numerical data, Inf. Sci. 241 (2013) 43–57. [2] A. Phophalia, A. Rajwade, S.K. Mitra, Rough set based image denoising for brain MR images, Signal Process. 103 (2014) 24–35. [3] M. Ye, X. Wu, X. Hu, D. Hu, Anonymizing classification data using rough set theory, Knowl.-Based Syst. 43 (2013) 82–94. [4] J. Wang, J. Peng, O. Liu, A classification approach for less popular webpages based on latent semantic analysis and rough set model, Expert Syst. Appl. 42 (2015) 642–648. [5] J. Zhang, T. Li, H. Chen, Composite rough sets for dynamic data mining, Inf. Sci. 257 (2014) 81–100. [6] D. Liu, T. Li, J. Zhang, A rough set-based incremental approach for learning knowledge in dynamic incomplete information systems, Int. J. Approx. Reason. 55 (2014) 1764–1786. [7] J. Meng, J. Zhang, R. Li, Y. Luan, Gene selection using rough set based on neighborhood for the analysis of plant stress response, Appl. Soft Comput. 25 (2014) 51–63. [8] D. Liang, W. Pedrycz, D. Liu, P. Hu, Three-way decisions based on decisiontheoretic rough sets under linguistic assessment with the aid of group decision making, Appl. Soft Comput. 29 (2015) 256–269. [9] S.H. Nguyen, Some efficient algorithms for rough set methods, in: International Conference of Information Processing and Management of Uncertainty in Knowledge Based Systems, Spain, Granada, 1996, pp. 1451–1456. [10] K. Zheng, J. Hu, Z. Zhan, J. Ma, J. Qi, An enhancement for heuristic attribute reduction algorithm in rough set, Expert Syst. Appl. 41 (2014) 6748–6754. [11] S.H. Liu, Q.J. Sheng, B. Wu, Z.Z. Shi, F. Hu, Research on Efficient Algorithms for Rough Set Methods, Chinese, J. Comput. 26 (2003) 524–529. [12] X.Y. Zhang, D.Q. Miao, Region-based quantitative and hierarchical attribute reduction in the two-category decision theoretic rough set model, Knowl.Based Syst. 71 (2014) 146–161. [13] X. Zhang, D. Miao, Reduction target structure-based hierarchical attribute reduction for two-category decision-theoretic rough sets, Inf. Sci. 277 (2014) 755–776. [14] B. Li, T.W.S. Chow, P. Tang, Analyzing rough set based attribute reductions by extension rule, Neurocomputing 123 (2014) 185–196. [15] R. Jensen, C. Cornelis, Fuzzy-rough nearest neighbour classification and prediction, Theor. Comput. Sci. 412 (2011) 5871–5884. [16] C. Wang, M. Shao, B. Sun, Q. Hu, An improved attribute reduction scheme with covering based rough sets, Appl. Soft Comput. 26 (2015) 235–243. [17] F. Jiang, Y. Sui, A novel approach for discretization of continuous attributes in rough set theory, Knowl.-Based Syst. 73 (2015) 324–334. [18] A. Skowron, C. Rauszer, The discernibility matrices and functions in information systems, in: S.-Y. Huang (Ed.), Handbook of Application and Advances of the Rough Sets Theory, Springer, Netherlands, London, 1992, pp. 331–362 , Intelligent Decision Support. [19] X.H. Hu, N. Cercone, Learning in relational databases: a rough set approach, Comput. Intell. 11 (1995) 323–337. [20] Z. Lu, Z. Qin, Y. Zhang, J. Fang, A fast feature selection approach based on rough set boundary regions, Pattern Recognit. Lett. 36 (2014) 81–88. [21] L. Yong, H. Wenliang, J. Yunliang, Z. Zhiyong, Quick attribute reduct algorithm for neighborhood rough set model, Inf. Sci. 271 (2014) 65–81.

Please cite this article as: X.-Y. Luan, et al., A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.06.090i

8

X.-Y. Luan et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

[22] D. Chen, W. Li, X. Zhang, S. Kwong, Evidence-theory-based numerical algorithms of attribute reduction with neighborhood-covering rough sets, Int. J. Approx. Reason. 55 (2014) 908–923. [23] Y. Yang, D. Chen, Z. Dong, Novel algorithms of attribute reduction with variable precision rough set model, Neurocomputing 139 (2014) 336–344. [24] Y. Yao, J. Mi, Z. Li, A novel variable precision -fuzzy rough set model based on fuzzy granules, Fuzzy Sets Syst. 236 (2014) 58–72. [25] X. Hu, Knowledge Discovery In Databases: An Attribute-oriented Rough Set Approach, University of Regina, Saskatchewan, Canada, 1995. [26] Y. Zhou, X.J. Yang, Y. Xu, Research on dependability algorithm of attribute reduction, Comput. Eng. Appl. 4 (2004) 78–79. [27] J. Wroblewski, Finding minimal redacts using genetic algorithms, in: Proceedings of the Second Annual Join Conference on Information Sciences, Wrightsville Beach, USA, 1995, pp. 186–189. [28] J.H. Dai, Y.X. Li, Heuristic genetic algorithm for minimal reduction decision system based on rough set theory, in: International Conference on Machine Learning and Cybernetics, Beijing, 2002, pp. 4–5. [29] F. Khoshahval, A. Zolfaghari, H. Minuchehr, A new method for multi-objective in core fuel management optimization using biogeography based algorithm, Ann. Nucl. Energy 73 (2014) 294–303. [30] J.K. Liao, D.Y. Ye, Minmial attribute reduction algorithm based on particle swarm optmization with mimunity, Comput. Appl. 27 (2007) 550–555. [31] X.Y. Luan, B. Jin, T.Z. Liu, An improved artificial fish swarm algorithm and application, Commun. Comput. Inf. Sci. 462 (2014) 99–110. [32] Y. Yao, The two sides of the theory of rough sets, Knowl.-Based Syst. 80 (2015) 67–77. [33] M.Q. Ye, X.D. Wu, X.G. Hu, D.H. Hu, Knowledge reduction for decision tables with attribute value taxonomies, Knowl.-Based Systems 56 (2014) 68–78. [34] M. Stigler, A modest proposal: a new standard for the normal,, Am. Stat. 36 (1982) 137–138. [35] N. Fang, J. Zhou, R. Zhang, Y. Liu, Y. Zhang, A hybrid of real coded genetic algorithm and artificial fish swarm algorithm for short-term optimal hydrothermal scheduling, Int. J. Electr. Power Energy Syst. 62 (2014) 617–629. [36] C. Ronco, E. Benini, A Simplex Crossover based evolutionary algorithm including the genetic diversity as objective, Appl. Soft Comput. 13 (2013) 2104–2123. [37] Y.Q. Wang, J.Z. Zhou, L. Mo, et al., A clonal real-coded quantum-inspired evolutionary algorithm with Cauchy mutation for short-term hydrothermal generation scheduling, International, J. Electr. Power Energy Syst. 43 (2012) 1228–1240. [38] H. Kita, A comparison study of self-adaptation in evolution strategies and realcoded genetic algorithms, Evol. Comput. 9 (2001) 223–241. [39] W. Zhu, J. Jiang, C. Song, L. Bao, Clustering algorithm based on fuzzy C-means and artificial fish swarm, Procedia Eng. 29 (2012) 3307–3311.

Zhanpei Li received the B.S. degree in Automation from Shanghai University, China, in 2011. He is currently working for Ph.D. degree in control science and engineering from Shanghai University. He main research interests are modeling and control of complex system.

Tingzhang Liu received the Ph.D. in machinery manufacturing from Xian Jiaotong University, Xian, China, in 1996. He is currently working as Professor of School of Mechatronic Engineering and Automation in Shanghai University. His research interests include complex systems theory, industrial energy conservation and solid state lighting.

Xinyuan Luan received the M.S. degree in integrated circuit engineering from Shanghai University, China, in 2011. He is currently working for Ph.D. degree in control science and engineering from Shanghai University. His main research interests are optimization algorithms and applications.

Please cite this article as: X.-Y. Luan, et al., A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.06.090i