Novel adaptive bacterial foraging algorithms for global optimisation with application to modelling of a TRS

Expert Systems with Applications 42 (2015) 1513–1530

Contents lists available at ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa

Novel adaptive bacterial foraging algorithms for global optimisation with application to modelling of a TRS A.N.K. Nasir a,b,⇑, M.O. Tokhi a, N.M.A. Ghani a a b

Department of Automatic Control and Systems Engineering, University of Sheffield, UK Department of Electric & Electronics, University Malaysia Pahang, 26600 Pekan Pahang, Malaysia

a r t i c l e

i n f o

Article history: Available online 16 September 2014 Keywords: Adaptive bacterial foraging Optimisation algorithm Nonparametric modelling Twin rotor system

a b s t r a c t In this paper, adaptive bacterial foraging algorithms and their application to solve real world problems is presented. The constant step size in the original bacterial foraging algorithm causes oscillation in the convergence graph where bacteria are not able to reach the optimum location with large step size, hence reducing the accuracy of the final solution. On the contrary, if a small step size is used, an optimal solution may be achieved, but at a very slow pace, thus affecting the speed of convergence. As an alternative, adaptive schemes of chemotactic step size based on individual bacterium fitness value, index of iteration and index of chemotaxis are introduced to overcome such problems. The proposed strategy enables bacteria to move with a large step size at the early stage of the search operation or during the exploration phase. At a later stage of the search operation and exploitation stage where the bacteria move towards an optimum point, the bacteria step size is kept reducing until they reach their full life cycle. The performances of the proposed algorithms are tested with various dimensions, fitness landscapes and complexities of several standard benchmark functions and they are statistically evaluated and compared with the original algorithm. Moreover, based on the statistical result, non-parametric Friedman and Wilcoxon signed rank tests and parametric t-test are performed to check the significant difference in the performance of the algorithms. The algorithms are further employed to predict a neural network dynamic model of a laboratory-scale helicopter in the hovering mode. The results show that the proposed algorithms outperform the predecessor algorithm in terms of fitness accuracy and convergence speed. Ó 2014 Elsevier Ltd. All rights reserved.

1. Introduction The twin rotor system (TRS) is a highly nonlinear dynamic system, which is often used as a platform to test controllers. It is a laboratory scale equipment that mimics the behaviour of a real helicopter. A schematic diagram of the twin rotor system is shown in Fig. 1. The main rotor moves the system up and down while the tail rotor rotates the system horizontally about the yaw-axis. The vertical and horizontal motions and multi-input multi-output nature of the system lead to complex characteristics which are difficult to model. Several conventional techniques have been used to acquire dynamic model of a twin rotor system. The techniques are based on system identification or parametric method, time-series regression, auto regressive with exogenous inputs (ARX) model, auto ⇑ Corresponding author at: Department of Electric & Electronics, University Malaysia Pahang, 26600 Pekan Pahang, Malaysia. E-mail addresses: [email protected] (A.N.K. Nasir), niha.ghani@sheffield.ac. uk (N.M.A. Ghani). http://dx.doi.org/10.1016/j.eswa.2014.09.010 0957-4174/Ó 2014 Elsevier Ltd. All rights reserved.

regressive moving average (ARMA) model and auto regressive moving average with exogenous inputs (ARMAX) model. These estimated models are mostly linear and have limited capability to capture non-linearity behaviours of the twin rotor system thus result in inaccurate model. Non-parametric approaches such as expert systems, artificial neural networks (ANNs) and fuzzy logic have been developed more recently to estimate dynamic model of various types of flexible systems with more promising and accurate results. The nonparametric approach is more robust and more accurate than the parametric approach. The prediction of a dynamic model for a flexible system using ANN is gaining attention from researchers due to its learning ability. However, the performance of ANN to predict the behaviour of a flexible system is still facing a drawback because of its complex and nonlinear nature, which is difficult to determine. Moreover, the optimum model of the ANN using conventional optimization algorithm such as gradient-based algorithm, steepest descent algorithm and least square algorithm is likely difficult to achieve since they tend to get stuck into local optima solution. Recently, the application of bio-inspired optimization algorithm like Bacteria foraging algorithm (BFA) to

1514

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

Start Initialize variables

Elimination and dispersal loop, L=1,2,3...Ned Reproduction loop, k=1,2,3...Nre Chemotaxis loop, j=1,2,3...Nc L=L+1

Compute fitness J(I,P) for I=1,2,3,…,S Tumble

Fig. 1. Schematic diagram of a twin rotor system (Reprinted from: Toha et al., 2012).

Parameter

Description

p J(I, P) S C Nc Ns sw Nre Ned i j k L

Dimension of search space Fitness cost of ith bacterium at current iteration Total number of bacteria Constant step size Total number of chemotaxis Maximum number of swim Index of swim Maximum number of reproduction Maximum number of elimination and dispersal Index of a bacterium Index of chemotaxis Index of reproduction Index of elimination and dispersal

No

Yes Swim

SW(I)=SW(I)+1

SW(I)
Yes

No Yes

I
Compute J(I,P)

j
Yes

No Reproduction k
predict nonlinear behaviour of a dynamic model is gaining more interest and has resulted in more accurate performance (Devi & Geethanjali, 2014; Ulagammai, Venkatesh, Kannan, & Padhy, 2007). Bio-inspired optimization algorithms have gained attention from researchers around the world due to their capability in dealing with numerous real world applications and the reliability of producing optimum solutions. Some of the well known bioinspired algorithms include genetic algorithm (Goldberg, 1989), particle swarm (Kennedy & Eberhart, 1995), ant colony (Dorigo & Di caro, 1999), artificial bee colony (Karaboga & Basturk, 2007) and BFA (Passino, 2002). BFA is one type of bio-inspired optimization algorithm which mimics the foraging strategy of Escherichia coli (E. coli) bacteria. The strategy is based on the bacteria behaviour to find rich nutrient or optimum food source during their full lifetime cycle which consists of chemotaxis, reproduction, elimination and dispersal phases. The chemotaxis phase through random tumble and swim mechanism is the most prominent phase of the foraging strategy where it mostly affects the success and performance of the bacteria to look for highest nutrient source. In this phase, the movement of bacteria from one location to another location is faster if a large step size is defined. However, the drawback of this choice is that the bacteria may be unable to locate the rich nutrient source if it is located at a remote area or at a minimum point of a curve. From an optimization point of view, this option can expedite the algorithm’s convergence but it produces relatively low accurate solution and leads to oscillation around the optimum point. On the contrary, the optimum nutrient source location might be easily found if the bacteria are moving with smaller step size. Nevertheless, the limitation of this option is that the bacteria require more steps and more time to reach the optimum food source. In other words, the convergence speed of the algorithm to an optimum point is slower but high accurate solution can be

j=j+1

J(I,P)
I=I+1 Table 1 Parameter of the BFA and adaptive BFA.

k=k+1

Yes

Elimination and dispersal L
Yes

End Fig. 2. Flowchart of the original and adaptive BFA.

attained. In order to overcome such problems, variation of bacteria step size in both tumble and swim actions throughout the search operation might be introduced. This can be realised through adaptation of a certain relationship such as using mathematical formulation, fuzzy logic approach, etc. The reproduction phase is an important process to preserve the fittest and healthiest bacteria in the population. After completing the chemotaxis phase, all the bacteria are classified based on their health and fitness level. The first half of the classified bacteria, which is considered as healthier than the other half of the bacteria population is reproduced. In order to maintain only the fittest bacteria in the population, the other half of the bacteria with lower health and fitness is eliminated. The reproduction process of the fittest bacteria can speed up the foraging strategy of the bacteria. To make the foraging strategy faster, a dispersal phase is introduced where all bacteria in the population are distributed randomly within the search area. With this strategy, the probability for the bacteria to be relocated at closer position to the optimum nutrient or food location is increased. This process takes place in the elimination and dispersal phase. Those three major processes of foraging strategy are performed in a sequential order and they are continuously repeated until full life cycle of bacteria is reached. There are several adaptation mechanisms previously adopted by researchers to improve BFA performance. Mishra (2005)

1515

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530 Table 2 Mathematical formulation and description of the benchmark functions. Function Rastrigin (f1) Sphere (f2) Griewank (f3)

Mathematical representation and description P f 1 ðxÞ ¼ pi¼1 ½x2i  10 cosð2pxi Þ þ 10 continuous, differentiable, separable, scalable, multi-modal P f 2 ðxÞ ¼ pi¼1 x2i continuous, differentiable, separable, scalable, uni-modal Pp 2 Qp 1 x  i¼1 cos pxiffi þ 1 continuous, differentiable, non-separable, scalable, f 3 ðxÞ ¼ 4000 i¼1 i i

Range

Theoretical global optima p

(5, 5)

f1(0) = 0

(5.12, 5.12)p

f2(0) = 0

(600, 600)p

f3(0) = 0

(15, 30)p

f4(0) = 0

(10, 10)p

 i  2 2 f 5 2 2i ¼ 0

(5, 10)p

f6(1) = 0

(2, 2)2

f7(0, 1) = 3

(5, 5)2

f8(0.0898, 0.71) = 1.0316

multi-modal

Ackley (f4)

Dixon & Price (f5) Rosenbrock (f6) Goldstein & Price (f7)

  qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  P  P 2 f 4 ðxÞ ¼ 20 exp 20 D1 D  exp D1 D i¼1 xi i¼1 cos 2pxi þ 20 þ e continuous, differentiable, non-separable, scalable, multi-modal P 2 f 5 ðxÞ ¼ ðx1  1Þ2 þ pi¼1 ið2x2i  xi1 Þ continuous, differentiable, non-separable, scalable, uni-modal P 2 ½100ðxiþ1  x2i Þ þ ðxi  1Þ2  continuous, differentiable, non-separable, scalable, f 6 ðxÞ ¼ p1 i¼1 uni-modal f 7 ðxÞ ¼ f1 þ ðx0 þ x1 þ 1Þ2 ð19  14x0 þ 3x20  14x1  6x0 x1 þ 3x21 Þgf30 þ ð2x0  3x1 Þ2 12x20

Six hump camel (f8)

27x21 Þg

ð18  32x0 þ þ 48x1  36x0 x1 þ continuous, differentiable, non-separable, non-scalable, multi-modal f 8 ðxÞ ¼ 4x21  2:1x41 þ 13 x61 þ x1 x2  4x22 þ 4x62 continuous, differentiable, non-separable, non-scalable, multi-modal

employed Takagi–Sugeno fuzzy logic scheme to establish relationship between bacteria step size and minimum value of actual cost function at every iteration. Four fuzzy rules based on triangular membership function were used in the adaptation scheme. Another adaptive scheme using fuzzy logic approach was presented by Supriyono and Tokhi (2011), where the authors used Mamdani-type fuzzy inference with Gaussian membership function to establish relationship between bacteria step size and nutrient value. This technique also was used by Venkiah and Vinod Kumar (2011) to solve congestion management problem. Another popular adaptation technique is through incorporation of mathematical formulation. Majhi, Panda, Majhi, and Sahoo (2009), Sathyaa and Kayalvizhib (2011) proposed a simple linear formulation to vary the step size within [0, 1] based on fitness value at every iteration. The analytical work on the formulation was conducted by Dasgupta, Das, Abraham, and Biswas (2009), which showed that it was capable of improving convergence speed of BFA. Supriyono and Tokhi (2011) presented a similar adaptive scheme with nonlinear equations where they offered more flexible range of bacteria step size defined within [0, Cmax], where Cmax is maximum step size. Alternatively, instead of using cost function value, the bacteria step size can be varied based on number of iterations. Goshal, Chatterjee, and Mukherjee (2009) utilised the ratio of current cycle and total number of cycles to vary bacteria step size. The bacteria step size is constantly reduced as the number of current cycle increases. The algorithm was used to solve power system stabilizer problem. Chen and Lin (2009), Farhat and ElHawary (2009), Huang and Lin (2010) and Das and Mishra (2013) utilised the total number of chemotaxis or bacteria lifetime and index of chemotaxis to vary the step size within a specified range [Cmin, Cmax]. The bacteria step size is constantly reduced as the index of chemotaxis increases. Chu, Mi, Liao, Ji, and Wu (2008) used the index of reproduction, elimination and dispersal loops to adjust the step size. In the strategy, the bacteria step size and the index have an inverse proportional relationship. Niu et al. (2010), Yan, Zhu, Chen, and Zhang (2012) and Xu, Liu, Wang, Wang, and Chen (2012) incorporated the total number of iterations and current iteration to introduce bacteria step size within user defined range [Cmin, Cmax]. Through these approaches, bacteria move with maximum step size Cmax at the early stages of search operation and they continuously move with smaller step until reaching Cmin when approaching the last iteration. In the previously mentioned improved versions of BFA, all the adaptive formulations were introduced to vary the bacteria step size. It is found that there were two major approaches to formulate

adaptive step size. The first approach was to vary the bacteria step size based on iteration index or iteration number while the second approach was to vary the bacteria step size based on individual bacterium fitness cost value. At this point, none of them have proposed an adaptive version based on the combination of iteration index and individual bacterium fitness cost value, which is proposed in this paper. Combining the two variables, bacteria step size can be more dynamically varied and hence better exploration and exploitation strategies introduced. Moreover, bacteria step size variation based on current value of accumulated iteration index, which is different from the other types of adaptive formulation proposed in the aforementioned versions is proposed. A more simple strategy based on linear equation is presented to establish relationship between a known variable namely current value of accumulated iteration index and bacteria step size. Unlike the existing adaptive formulation in the literature, the proposed adaptive formula is developed based on the combination of index of chemotaxis, reproduction, elimination and dispersal. Comparative assessment of the adaptive formulation based on iteration index and fitness cost is also presented. In this work, adaptive BFAs are employed to optimise ANN model to mimic the dynamic behaviour of a twin rotor system. Unlike the conventional BFA which has slower performance due to incorporation of constant step size, the bacteria step size is dynamically varied based on the combination of index of iteration and index of bacteria in the population and the combination of all indices and fitness value. Nonlinear adaptive formulations are established based on simple linear and nonlinear formulations, which effectively improve the BFA performance in terms of both the convergence speed and fitness accuracy. The rest of the paper is organised as follows: Section 2 provides detailed description of the proposed adaptive BFA algorithms. Validation and statistical

Table 3 Parameters of BFA, IBFA and FIBFA for benchmark functions tests.

Parameter

BFA

IBFA

FIBFA

S = 50 Ns = 4 Nc = 100 Nre = 4 Ned = 2 C = 0.1

S = 50 Ns = 4 Nc = 100 Nre = 4 Ned = 2 C IBFA ¼

S = 50 Ns = 4 Nc = 100 Nre = 4 Ned = 2 C FIBFA ¼ 

1:5 0:9ðijkLÞ0:9 þ1

2:5

0:9ðijkLÞ0:9 0:2jJði;j;k;LÞj0:2



þ1

1516

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

Table 4 Statistical results for the BFA, IBFA and FIBFA tested on the benchmark functions. No

Function

Dim

1

Rastrigin, f1

15

30

45

60

2

Sphere, f2

15

30

45

60

3

Griewank, f3

15

30

45

60

4

Ackley, f4

15

30

45

60

5

Dixon & Price, f5

15

30

45

BFA

IBFA

FIBFA

Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst

71.7664 10.0497 42.4882 87.0167 227.8996 15.2076 191.0726 255.9851 399.5583 21.3011 365.2128 442.6647 583.9721 27.2864 507.5568 640.2427

70.9444 17.3643  27.8071 95.4059 192.7103 23.9754  120.7051 234.6574 303.2600 27.4049  238.1994 347.9168 445.5985 32.6486  376.2412 506.5929

74.6593 13.7481 34.3119 99.7257 194.1270 24.0955 147.0758 245.9961 329.6411 22.8995 281.5081 384.4826 480.4727 34.8547 418.7791 549.1033

Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst

0.1453 0.0398 0.0618 0.2139 0.6566 0.1011 0.4697 0.8894 4.1923 0.9020 2.8172 6.3640 25.9662 4.4920 15.1910 33.9146

0.0033 0.0015 0.0016 0.0080 0.1947 0.1021  0.0603 0.4287 2.8328 0.6348 1.9592 4.2139 11.8903 2.5673  7.5862 17.5606

0.0040 0.0020  9.96  104 0.0113 0.1805 0.0462 0.0926 0.2620 2.5919 0.6752  0.9752 4.3739 11.7074 2.1542 8.3884 16.8666

Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst

207.2228 40.6498 116.9222 290.1331 572.9713 42.4306 475.3584 664.2639 936.8041 74.9055 741.5544 1057.4 1320.5 96.7467 1112.7 1484.7

173.7118 32.2741  99.7750 230.0381 506.4106 55.6123 406.5475 630.6696 886.4950 101.9256  540.2632 1009.8 1273.7 92.8107  1040.3 1461.7

166.4281 27.0031 108.5993 204.5474 502.9560 44.0346  367.3516 557.1681 859.4992 64.5577 730.0950 975.6863 1254.3 57.6437 1142 1383.2

Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst

17.2150 0.8505 14.2413 18.2857 18.5364 0.3884 17.4426 19.1476 18.8840 0.2426 18.3453 19.3364 19.1621 0.1771 18.7347 19.4170

14.6295 0.7917 12.8587 16.6018 15.8148 0.8189  12.5164 16.6388 16.2660 0.3659  15.4598 16.9048 16.5049 0.3195  15.9450 17.1969

16.5155 1.6137  10.5378 18.0218 18.0933 0.4351 16.7455 18.6568 18.3492 0.1971 17.9971 18.6821 18.6493 0.2275 17.9649 18.9526

Mean SD Best Worst Mean SD Best Worst Mean SD

2.4073 0.7815 1.5245 5.7020 1644.7 1163.2 174.3992 5183.1 115,390 33,979

1.1871 0.4654  0.7391 2.4906 86.7006 35.6417  33.7115 198.2101 3135.1 1430.8

2.2865 1.1009 0.8465 5.0764 88.9737 35.4453 45.8390 198.1326 2082.9 722.7053

1517

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530 Table 4 (continued) No

Function

Dim

60

6

Rosenbrock, f6

15

30

45

60

BFA

IBFA

FIBFA

Best Worst Mean SD Best Worst

36,225 180080 778,280 115,660 450,890 969,210

 695.9797 6218.9 37936 12,347 18,423 71,816

747.6920 3779.3 16,451 4962.4  6126.0 28,229

Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst Mean SD Best Worst

28.3812 3.6979 21.5729 34.3244 699.4071 314.8351 172.2762 1756.6 30,564 13,109 8229.6 61,866 199,390 62,914 60,315 321,830

17.4122 14.3084  7.1052 91.8406 292.4348 99.7466  121.9142 557.6799 2766.2 996.6184  1292.7 5178.7 18,995 8876.1 7152.4 48,770

30.5214 20.3500 14.8497 92.9491 396.2052 125.2649 147.5378 746.8726 2349.1 486.4468 1365.2 3169.7 10,559 3194.1  5997.0 21,128

7

Goldstein & Price, f7

2

Mean SD Best Worst

3.0790 0.0765 3.0043 3.2865

3.0000 1.83  105  3.0000 3.0001

3.0002 4.40  104  3.0000 3.0018

8

Six hump camel, f8

2

Mean SD Best Worst

0.0013 0.0013 1.09  106 0.0063

1.33  106 2.33 x106  4.72  108 1.06  105

9.37  107 1.80  106  4.72  108 9.32  106

Table 5 Results of the Friedman test and Wilcoxon signed rank test based on results in Table 4. Algorithm

Friedman test Mean rank

Wilcoxon signed rank test p

v2

BFA–IBFA +

BFA IBFA FIBFA

2.92 1.44 1.63

<0.05

34.04

BFA–FIBFA 

R

R

0

351

+

p

R

<0.05

23

IBFA–FIBFA R

p

R+

R

p

328

<0.05

155

170

0.840



Table 6 Results of the t-test conducted against BFA result and the best result between IBFA and FIBFA. No

Function

Dim

t-Value

95% Confidence interval

Two tailed, p

Significant improvement

1

Rastrigin, f1

15 30 45 60

Standard error 3.6630 5.1836 6.3371 7.7685

0.2244 6.7886 15.1959 17.8122

6.5102–8.1542 24.8132–45.5654 83.6133–108.9835 122.8233–153.9239

0.8232 6.63  109 6.95  1022 3.44  1025

NO YES YES YES

2

Sphere, f2

15 30 45 60

0.0073 0.0203 0.2057 0.9096

19.5185 23.4540 7.7796 15.6767

0.1275–0.1566 0.4354–0.5167 1.1886–2.0121 12.4382–16.0795

3.60  1027 2.77  1031 1.43  1010 1.61  1022

YES YES YES YES

3

Griewank, f3

15 30 45 60

8.9099 11.1645 18.0541 20.5611

4.5786 6.2712 4.2818 3.2197

22.9596–58.6297 47.6671–92.3635 41.1657–113.4442 25.0439–107.3588

2.52  105 4.84  108 7.04  105 0.0021

YES YES YES YES

4

Ackley, f4

15 30 45 60

0.2121 0.1655 0.0802 0.0667

12.1876 16.4480 32.6635 39.8402

2.1608–3.0101 2.3904–3.0528 2.4576–2.7785 2.5237–2.7907

1.23  1017 1.65  1023 4.85  1039 7.84  1044

YES YES YES YES

5

Dixon & Price, f5

15 30 45 60

0.1661 212.4711 6205.1 21,136

7.3481 7.3326 18.2599 36.0442

0.8878–1.5526 1132.7–1983.3 100,880–125,730 719,520–804,140

7.62  1010 8.09  1010 1.01  1025 2.09  1041

YES YES YES YES

6

Rosenbrock, f6

7 8

Goldstein & Price, f7 Six hump camel, f8

15 30 45 60 2 2

2.6982 60.2966 2395 11,501 0.0140 2.40  104

4.0653 6.7495 11.7805 16.4183 5.6517 5.2730

5.5680–16.3700 286.2754–527.6690 23,421–33,009 165,810–211,850 0.0510–0.1069 0.0008–0.0017

1.46  104 7.71  109 5.07  1017 1.79  1023 5.07  107 2.07  106

YES YES YES YES YES YES

1518

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

analyses of the proposed algorithms in comparison to its predecessor algorithm with different fitness landscapes of benchmark functions are presented in Section 3. Section 4 presents the application of the algorithms to optimization of neural network dynamic model for a twin rotor system. Section 5 provides conclusion of the paper. 2. Adaptive bacterial foraging algorithms The proposed adaptive bacterial foraging algorithms are described in this section. 2.1. The iteration-based adaptive bacterial foraging algorithm The mathematical equation for iteration-based bacterial foraging algorithm (IBFA) is defined as:

C IBFA ¼

C max n m  ði  j  k  LÞ þ q

adaptive

ð1Þ

where CIBFA, i, j, k and L are bacteria step size, index of total number of bacteria in the population, index of total number of chemotaxis or bacteria lifetime, index of total number of reproduction and index of total number of elimination and dispersal events respectively. The constant m is a rate of change of bacteria step size and at the same time acts as a gain, which is able to reduce or amplify any too large or too small values of the product of iteration index and bacteria index or accumulated total iteration, [i  j  k  L]. Small value of m produces wide range of step size, which increases convergence speed and good exploration while large value yields smaller step size range thus leading to slow convergence but good exploitation. n – 0 is the order of the equation, which has a very high impact on the performance. Defining n away from zero increases the rate of change and widens the range of bacteria step size, which can accelerate bacteria movement and contribute to faster convergence while at the same time produces a more accurate solution. Defining n other than n – 0 can introduce a non-linear relationship in the equation. Cmax is bacteria maximum step size and q is a constant equal to 0 or 1. In the equation, the step size is varied within [0, Cmax/2] if q, m and n are defined as 1 while if q

Fig. 3. Comparison of convergence plots for BFA, IBFA and FIBFA on the benchmark functions.

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

1519

Fig. 3 (continued)

is set as 0, m and n are set as 1, the step size is varied within [0, Cmax]. Notice that the equation is rearranged such that it is reciprocal to [i  j  k  L]. Here, the bacteria step size has an inversely proportional relationship to the current value of the accumulated total iteration. Depending on the value of n, the relationship can be linear or nonlinear. This approach enables bacteria to move with large step if [i  j  k  L] is small, making it as a good strategy for exploration thus resulting in faster convergence. On the contrary, bacteria move with smaller step size when approaching the last iteration or when [i  j  k  L] is large so that the location of optimum point can be traced accurately due to better exploitation. Through the scheme, in the same iteration, all the bacteria move with equal step size and it is very much dependent on the value of current [i  j  k  L]. Moreover, user-defined parameters have been incorporated to increase the flexibility on the determination of bacteria step size with a user-defined range. In the proposed adaptive approaches, the maximum size of bacteria step has been defined as Cmax. User defined value for Cmax is more promising since it gives more flexibility for the user to choose step size for the bacteria while in motion within various types of unknown fitness landscapes. Another advantage of this approach is that, the

variable [i  j  k  L] is a known variable and thus the dynamic variation, rate of change and range of the bacteria step size can be controlled easily. 2.2. The fitness-iteration-based adaptive bacterial foraging algorithm Fitness and iteration index based adaptive BFA (FIBFA) is a technique to vary bacteria step size through a relationship based on a combination of fitness value of bacteria in each iteration and index of total iteration of BFA or current value of the accumulated total iteration. Eq. (1) is further modified and FIBFA can be mathematically defined as:

C FIBFA ¼ 

C max

mðijkLÞn ajJði;j;k;LÞjp



þ1

ð2Þ

where the constant, p – 0 is the order of the equation, the constant a is rate of change of bacteria step size and J(i, j, k, L) is fitness value of a bacterium in each iteration while the other parameters are the same as in Eq. (1). FIBFA is designed such that CFIBFA is increased and approach Cmax when J is large and CFIBFA is reduced towards zero when J is small. In this scheme, CFIBFA is directly proportional to J

1520

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

and inversely proportional to [i  j  k  L]. In other words, CFIBFA is very much dependent on the ratio of the accumulated total iteration and fitness cost of a bacterium. The constant a can also be defined as a gain, which is able to reduce or amplify any too large or too small values of the fitness cost, J. Large value of a tends to produce bacteria step size close to Cmax, reduces the range of step size and leads to smaller rate of change of step size, which increases convergence speed and good exploration. Small value of a tends to produce bacteria step size close to zero, yields smaller step size range but faster rate of change of step size thus leading to slow convergence but good exploitation. Defining p away from zero tends to produce bacteria step size close to Cmax, reduces the rate of change and widens the range of bacteria step size, which can accelerate bacteria movement and contribute to faster convergence. Defining p toward zero tends to produce bacteria step size close to zero, reduces the range of step size and leads to higher rate of change of step size where a good exploitation can be achieved. Moreover, unlike the first scheme, with the introduction of fitness, J into the equation, different step sizes for each bacterium can be realised in every iteration and hence the exploration and exploitation strategies in one particular iteration are presented at the same time.

3.1. Benchmark functions A brief description of the mathematical formulation and features of the benchmark functions used in the tests are presented in Table 2. All the test functions are continuous and differentiable. The f1 and f2 are separable type functions while the rest f3–f8 are non-separable functions. The f2, f5 and f6 are uni-modal type functions while the f1, f3, f4, f7 and f8 are multi-modal type functions. From the optimization point of view, the non-separable and multi-modal type functions are more difficult to optimise. The f7 and f8 are non-scalable type functions, with their number of dimensions as two. The rest of the functions are scalable type functions, hence in the tests, 15, 30, 45 and 60 dimensions are defined. A function with higher dimension is more difficult to optimise since the number of variables increases with dimension. The theoretical global optima values of functions f1–f6 are zero, while for the functions f7 and f8 the theoretical global optima values are 3 and 1.03, respectively.

2.3. Adaptive bacteria foraging algorithm The parameters and flowchart of the proposed adaptive BFA are presented in Table 1 and Fig. 2 respectively. These are the same as those of the original BFA, except for incorporation of the adaptive formulation into the chemotaxis phase of the proposed adaptive BFA. The modification here is the chemotactic step size in the chemotaxis phase of BFA, which has a significant impact on the BFA performance. The introduction of the adaptive schemes into the chemotaxis phase in BFA aims to improve the convergence speed and fitness accuracy of the algorithm. This can be done if the exploration and exploitation stages throughout the search operation are kept balanced. Adopting a relationship between bacteria step size and the combination of iteration index, bacteria index and fitness value in the population to replace constant step size, is a proposed solution to the problem. This strategy does not add extra complexity to the original structure of BFA and hence it does not affect significantly the total computation time to complete the whole search operation. The tumble and swim actions in the chemotaxis phase of the original BFA move the bacteria from one location to another using the following equation:

hi ðj þ 1; k; LÞ ¼ hi ðj; k; LÞ þ CðiÞ

DðiÞ D ðiÞDðiÞ T

ð3Þ

Fig. 4. General structure of multilayer perceptron neural network.

Fixed input Synaptic weight

Inputs

wk1

x1 x2

wk 2

xm

wkm

x0 = +1

bk

Activation function output

ϕ (•)

∑

yk

Summing junction

Fig. 5. Basic structure of single neuron.

where D represents a vector of random directions in the range [1, 1], hi(j, k, L) represents the ith bacterium, at jth chemotactic, kth reproductive and Lth elimination and dispersal phase and C(i) represents a constant step size for the ith bacterium motion throughout the whole search operation. In the proposed iterationbased and fitness-iteration-based adaptive algorithms, the constant step size in Eq. (3) is replaced by adaptive formulation as presented in Eqs. (1) and (2) respectively.

u (t )

y (t )

Twin rotor system

z −1

z −1 Neural network model

z −2

z −2

3. Validation with benchmark functions This section presents validation of the proposed algorithms with eight benchmark functions with different dimensions and fitness landscapes. In order to see the effectiveness and the significance of the proposed schemes compared to the original BFA, statistical test and analysis are conducted with each test function. Both numerical and graphical results of the tests are also presented.

z − nu

z − ny y (t ) − ABFA

∑

+ y (t ) e(t )

Fig. 6. Schematic diagram for a nonparametric modelling approach of TRS optimised by ABFA.

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

3.2. Comparative performance assessment strategy The assessment focuses on three performance metrics. First, the fitness accuracy achieved at the last fitness evaluation number of each algorithm or achieved by the best bacterium. Second, the frequency of the fittest bacterium to reach the global optimum point of each function. Third, the convergence speed to reach the global optimum point based on the number of fitness evaluations. The first two performance metrics can be evaluated by taking the mean and standard deviation (SD) values of an independent and repeated number of runs while the third performance metric is assessed by plotting a convergence graph of the respective function. Moreover, the best and worst accuracies achieved by the algorithms are also

Neural Network model

y (t ) ∑

u (t )

Twin rotor system

presented since these represent the best and the worst solutions each particular algorithm can achieve. In order to provide all the statistical data, each algorithm was independently run for 30 times and 80,000 fitness evaluations (FEs) where the best fitness accuracy of each run was recorded to compute the mean and SD values. A t-test with alpha value of 5% degree-of-significance and 28 degrees-of-freedom (DOF) was conducted to compare the significant difference of the performance between the original and the best of the adaptive BFAs. Standard parameter value for BFA, IBFA and FIBFA was used in the tests for all test functions and this is presented in Table 3. The parameters for BFA were adopted from Passino (2002) and Biswas, Dasgupta, Das, and Abraham (2007) while the parameters for the IBFA and FIBFA were heuristically selected. 3.3. Empirical result

−

+

1521

e(t ) y (t )

Fig. 7. Block diagram for nonparametric modelling used to validate the estimated model of TRS in hover.

The results of the statistical tests conducted on the benchmark functions for the BFA, IBFA and FIBFA are presented in Table 4. The best mean value among the three algorithms is highlighted in bold font. Notice that for the mean result, the proposed hybrid schemes outperformed the original BFA. The IFBA dominated the mean result for functions f1, f4 and f7 while FIBFA dominated the mean result for functions f2, f3 and f8. On the other hand, for functions

Fig. 8. Input–output data pair and its corresponding PSD for TRS in vertical motion.

1522

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

f5 and f6, IBFA and FIBFA performed the best for lower dimension and high dimension respectively. The best standard deviation, which is the lowest among the three algorithms, is italicised. The table shows that the distribution of the best SD was almost even among the three. The best SD for functions f1, and f4, function f7, functions f3, f5, f6 and f8 were achieved by BFA, IBFA and FIBFA respectively. On the other hand, IBFA and FIBFA shared the SD domination for function f2. The result with a sign ‘’ indicates the best accuracy or nearest position reached within the 30 repeated runs among the three algorithms. Notice that IBFA and FIBFA dominated the performance for all functions. IBFA had better performance for functions f1, f3, f4, f5 and f6 while for the rest of the functions, IBFA and FIBFA shared the domination. The result with a sign ‘’ indicates the worst accuracy achieved among the three algorithms. Note that BFA dominated the worst accuracy for all functions except for 15 dimension functions f1 and f6, which was presented by FIBFA. The performance of the three algorithms in terms of fitness accuracy was tested through nonparametric Frieman test and Wilcoxon signed rank test and the results are presented in Table 5. Friedman test is used to rank the performance of three or more algorithms while the Wilcoxon signed rank test is used to compare the performance between two algorithms. The algorithm with lower rank in the Friedman test is considered as the algorithm with

the highest accuracy achievement. In both tests, the algorithms are considered to have an equal performance if the two-tailed, p value is equal to or more than 0.05 with 5% degree-of-significance. Notice that, based on Friedman test, IBFA achieved the lowest ranking as highlighted in bold font followed by FIBFA and BFA. In other words, IBFA was the best algorithm in terms of accuracy achievement. Based on Wilcoxon signed rank test, it showed that the p value of BFA–IBFA and BFA–FIBFA was less than 0.05. With reference to the Friedman test and p value of Wilcoxon test, it means that IBFA and FIBFA outperformed BFA with significant improvement. For the IBFA–FIBFA, Wilcoxon test shows that it had p value greater than 0.05. It means that the difference between IBFA and FIBFA in terms of accuracy achievement was not significant and thus had equal performance. The best performance between IBFA and FIBFA is selected as an input data for t-test, which is compared against original BFA. The result of the t-test conducted against BFA result and the best result between IBFA and FIBFA is shown in Table 6. For a 5% degree-ofsignificance and 28 DOF, the improvement is considered as significant if the two-tailed, p value is less than 0.05 and if the t-value is greater than approximately 2.0. Table 6 shows that all the results were significant except for 15 dimension function f1 which had the p and t values of 0.8232 and 0.2244, respectively. The results presented in Tables 4–6 clarify that the proposed adaptive schemes

Fig. 9. Results in the modelling phase. (a) Convergence plot, (b) output response of the TRS in vertical motion and (c) error.

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

outperformed the original BFA in terms of fitness accuracy achievement and FIBFA had the highest frequency to reach the solution nearest to or exactly at the global optimum point. The results also show that the adaptive formulation based on combination of both iteration index and fitness cost had more consistent performance while the adaptive formulation based on solely iteration index resulted in best accuracy achievement. The performance in terms of convergence speed is assessed based on comparison of convergence plots for each algorithm. In this paper, as a sample of graphical result, only the plot for 30 dimensions of each function is presented. Fig. 3 presents a comparison of convergence plots for BFA, IBFA and FIBFA in terms of the average of the best fitness scaled in log 10 versus 80,000 fitness evaluations. It is shown that both adaptive schemes had faster convergence speed as well as fitness accuracy compared to the original BFA hence clarifying the results presented in Tables 4–6. 4. Application to determine neural network dynamic model for twin rotor system The application of the algorithms in solving a real world engineering problem by determining neural network dynamic model for a twin rotor system (TRS) is presented in this section.

1523

Performance assessment is made in terms of fitness accuracy, convergence speed and the responses of the estimated nonlinear model in both the frequency-domain and time-domain. 4.1. Basic structure of neural network A neural network with multilayer perceptron (MLP) is used in modelling the vertical channel dynamic behaviour of the TRS. The basic structure of the MLP is shown in Fig. 4. The structure consists of three types of layers where the first layer is considered as an input layer, the last layer is considered as an output layer while a layer between the first and the last layers is named as hidden layer. Every hidden and output layer consists of a set of data processing elements with activation functions called neurons or nodes. Every node in a layer is connected to several other nodes in the adjacent layer via a connection weight. A general structure of a single neuron is shown in Fig. 5, where x1, x2, . . . , xm represent inputs and wk1,wk2, . . . , wkm represent synaptic weights of neuron. bk is denoted as a bias or a threshold, /(  ) represents an activation function of the neuron and yk represents the output of the neuron. The positive bias bk has the effect of increasing the net input of the activation function (Haykin, 1994). The output of the neuron is mathematically expressed as:

Fig. 10. Results in the validation phase. (a) Output response of TRS in vertical motion, (b) error and (c) power spectral density.

1524

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

yk ¼ / bk þ

m X wkj ij

! ð4Þ

j¼1

A hyperbolic tangent activation function is the most popularly used since it has faster learning process, higher accuracy and it is able to represent nonlinearity. It is defined as a ratio of a hyperbolic sine and cosine functions with range [1, 1]. A mathematical representation of the hyperbolic tangent activation function with a slope parameter a is expressed as:

f ðxÞ ¼

sinhðxÞ eax  eax ¼ coshðxÞ eax þ eax

ð5Þ

4.2. Model structure formulation

nonlinear auto regression moving average with exogenous inputs (NARMAX) model structure (Chen & Billings, 1989). A general mathematical formula for the NARMAX model can be represented as:

2

3 yðt  1Þ; yðt  2Þ; . . . ; yðt  ny Þ; _ 6 7 y ðtÞ ¼ f 4 uðtÞ; uðt  1Þ; . . . ; uðt  nu Þ; 5þg gðt  1Þ; gðt  2Þ; . . . ; gðt  ng Þ;

ð6Þ

_

where y ðtÞ is the estimated output, g(t) is the noise, while y(t) and u(t) are recorded output–input from experimental data. f() is nonlinear function mapping while the ny, nu and ng are the maximum lags in the actual output, input and noise respectively. Eliminating the noise term, the model can be simplified as a non-linear auto regressive with exogenous inputs (NARX) model, which in discrete-time input–output recursive form is represented as: _

An effective strategy to represent a nonlinear system when the actual input–output data pair of the system is available is through

y ðtÞ ¼ f ½yðt  1Þ; yðt  2Þ; . . . ; yðt  ny Þ; uðtÞ; uðt  1Þ; . . . ; uðt  nu Þ ð7Þ

Table 7 Performance of BFA, IBFA and FIBFA. Algorithm

Fitness accuracy

Error range in modelling phase

Error range in validation phase

BFA IBFA FIBFA

1.5076 0.5190 0.4444

[0.2343 0.3059] [0.2069 0.1407] [0.2349 0.1861]

[0.1782 0.1861] [0.0769 0.1418] [0.0892 0.1043]

Fig. 11. Auto-correlation of the error. (a) IBFA, (b) FIBFA and (c) BFA.

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

The advantage of NARX model is that it can approximate and represent a wide variety of nonlinear dynamic behaviours (Leontaritis & Billings, 1985a, 1985b). Moreover, it has been extensively used to solve many problems in various modelling and control applications (Kalkkuhl & Castro, 1996; Palumbo & Piroddi, 2000). The NARX model is linear-in-the-parameters, hence the identification process is relatively simple (Leva & Piroddi, 2002). 4.3. ABFA-based modelling In general, in the modelling process, input–output sampled data pairs are recorded from the actual system and they are divided into two parts. In practice, the first two-third of the data pair is used to estimate the actual system’s dynamic in the modelling phase while the remaining unused data pair is used to check the estimated model validity in the validation phase (Ljung, 1999). Ideally, in the modelling phase, the output response of the predicted model must display exactly the same signal as the output data taken from the first portion of the actual signal if the input–output data taken from the first portion of the actual signal is applied to the

1525

estimated model. A schematic diagram for a nonparametric modelling approach of the TRS in hovering mode (vertical channel) optimised by adaptive bacteria foraging algorithms is represented in Fig. 6. The y(t) and u(t) represent the output and input samples from the actual system respectively and they are used as the inputs to map and estimate the NN model. Then the output response of the estimated model is compared with the output signal from the actual system. The difference between the two signals is considered as the error e(t) in the modelling phase where it is used as an objective function for the proposed adaptive BFA. In order to optimally estimate the NN model, the objective function or the error function is to be reduced optimally. The mean square of the error with a weight, w is considered as the objective function in this work, and for a maximum number of sampled data, N, the error function can be represented as:

f ¼

! N _ 1X ðyðtÞ  y ðtÞÞ  w N k¼1

Fig. 12. Cross-correlation of the input and error. (a) IBFA, (b) FIBFA and (c) BFA.

ð8Þ

1526

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

4.4. ABFA-based validation In general, for a nonparametric modelling strategy, two types of tests are conducted to evaluate the adequacy of an estimated model to represent the actual system. 1. Experimental input–output data. The second portion of the unused actual input–output sampled data pair recorded during experimental work is used in the validation part. Ideally, for a given second portion of the unused input data taken from the actual system, the output response of the estimated model must portray exactly the same signal as the second portion of the unused output data taken from the actual system. The block diagram for the nonparametric modelling used to validate the estimated model of a system is depicted as in Fig. 7. The y(t), _ u(t) and y ðtÞ denote the actual output data, actual input data and output response of the estimated model respectively. The difference between the actual output data and output response of the estimated model is considered as the error, e(t) in the validation phase. 2. Correlation test. The correlation test is a well-known approach used to check the adequacy or quality of the estimated model where it can identify whether there are any un-modelled linear

or non-linear terms in the residual, which normally lead to biased estimates. A non-linear model is considered as valid model if the following conditions are satisfied:

/ee ðtÞ ¼ E½eðt  sÞeðtÞ ¼ dðsÞ /ue ðsÞ ¼ E½uðt  sÞeðtÞ ¼ 0; 2

8s

2

/u2 e ðsÞ ¼ E½ðu ðt  sÞ  u ðtÞÞeðtÞ ¼ 0;  2 ðtÞÞe2 ðtÞ ¼ 0; /u2 e2 ðsÞ ¼ E½ðu2 ðt  sÞ  u

8s

ð9Þ

8s

/eðeuÞ ðsÞ ¼ E½eðtÞeðt  1  sÞuðt  1  sÞ ¼ 0;

sP0

where, uee(s) is the auto-correlation function of the error or residual, e(t), d(s) is impulse function and uue(s) is cross-correlation function between input, u(t) and residual, e(t). If the residuals are unpredictable from all linear and non-linear combinations of past inputs–outputs, the estimated model is considered as a valid model (Toha, Julai, & Tokhi, 2012). For sampled input and output signal, the correlation function can be normalised within a range of ±1. Moreover, the correlation between the variables can never be exactly zero for most lags and therefore, in practice, 95% confidence interval is used as a basis to show that the estimated correlation is significant. An estimated model is said to be a good model if the correlation plot does not significantly exceed the 95% confidence boundary.

Fig. 13. Cross-correlation of the input square and error. (a) IBFA, (b) FIBFA and (c) BFA.

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

4.5. Nonparametric modelling strategy for the twin rotor system In the experimental work with the actual system, a total of 10,000 input–output sampled data pairs were recorded. The first 6000 data pairs were used in the modelling phase while the remaining unused 4000 data pairs were used in the validation phase. The input–output of the TRS in vertical motion were recorded from the actual system and are shown with power spectral density of the output in Fig. 8. A random input voltage within a range of [1, 1] volts was applied to the system, which produced pitch angle output response within a range of [1, 1] radian and a dominant resonance mode at 0.35 Hz. In the modelling phase, a three-layer NN consisting of one input layer, one nine-neurons hidden layer and one output layer with eight previous actual input–output samples was considered for modelling the vertical channel of the TRS. Therefore, there were 173 total parameters to be optimised as summarized below:  Weights of the input signal entering neurons in the hidden layer: 144 parameters.  Biases of the nine neurons in the hidden layer: 9 parameters.  Hyperbolic tangent sigmoid activation function of the nine neurons in the hidden layer: 9 parameters.

1527

 Weights of the hidden layer entering the output layer: 9 parameters.  Bias of the one neuron in the output layer: 1 parameter.  Linear activation function parameter of the one neuron in the output layer: 1 parameter. The parameters for the BFA and ABFAs for modelling the TRS in vertical motion were mostly the same as the parameters for BFA and ABFA used in the benchmark function tests. Considering the complexity of the problem, the only difference was the number of elimination and dispersal Ned which was defined as four. Comparison of the convergence plot, vertical channel response and modelling error for all optimization algorithms in the modelling phase are depicted in Fig. 9. On the other hand, the vertical channel response, its corresponding error, and power spectral density, in the validation phase, are shown in Fig. 10. In terms of number of fitness evaluations, the convergence plot shows that the proposed and original BFAs had about the same convergence speed. The ABFAs had faster convergence speed at the very early stage of the search operation but they slowed down near the optimum fitness value. Notice that, after 70,000 fitness evaluations, the proposed adaptive BFAs kept moving to lower fitness values which means that they continuously moved toward an optimum location.

Fig. 14. Cross-correlation of the input square and error square. (a) IBFA, (b) FIBFA and (c) BFA.

1528

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

Fig. 15. Cross-correlation of the error and product of the input and error. (a) IBFA, (b) FIBFA and (c). BFA.

On the other hand, the original BFA stopped converging to a lower fitness value and it continuously oscillated about the same fitness level. Since a constant step size was defined for the BFA, the bacteria were not able to move to an optimum location especially when it was located in a remote area. Numerical results of modelling the TRS in vertical motion are summarized in Table 7. It shows that IBFA achieved the best accuracy with fitness value of 0.4444, followed by FIBFA and BFA, which had fitness values of 0.5190 and 1.5076, respectively. Figs. 9(b) and 10(a) show that all the vertical channel responses coincided with each other denoting that the algorithms were able to predict the vertical channel behaviour successfully. The error plots in Figs. 9(c) and 10(b) and the error range in Table 7 show that the ABFAs had lower error compared to BFA both in the modelling and validation phases. The power spectral density plot in the validation phase shows that all the algorithms successfully captured the dynamic behaviour of TRS in vertical motion where it displayed a dominant frequency at 0.35 Hz for a sample time of 0.1 s. In addition to the vertical channel’s output response, five correlation tests were conducted to check the adequacy of the estimated model in representing the flexible system. The graphical results for the auto-correlation of the error, cross-correlation of the input and error, cross-correlation of the input square and error,

cross-correlation of the input square and error square, cross-correlation of the error and multiplication of the input and error are shown in Figs. 11–15 respectively. The dotted lines represent the 95% boundary while the solid lines represent the correlations. Overall, all the correlation plots satisfied the 95% boundary except the cross-correlation plot of the input square and the error produced by original BFA. It is shown in Fig. 13(c) that the correlation graph produced by BFA tended to move outside the 95% boundary. Therefore, the NN models optimised by the proposed adaptive BFAs were considered more adequate to represent the TRS in vertical motion.

5. Conclusion Novel adaptive bacteria foraging algorithms based on index of iteration, index of chemotaxis and fitness value have been proposed to improve the performance of BFA. The incorporation of the adaptive schemes as alternative approach for the constant step size balances the exploration and exploitation strategies of the BFA. In the first adaptive scheme, the bacteria move with a larger step size at the early stage and they continuously search for a better solution with a smaller step size as the total number of itera-

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

tions approaches maximum at the later stage. Through the scheme, in the same iteration, all the bacteria move with similar step size and it depends on the value of current accumulated iteration. In the second adaptive scheme, the dynamic feature of the bacteria step size has been improved by incorporating an individual bacterium’s fitness value as an additional variable to the first adaptive scheme where a bacterium moves with respect to its fitness level as well as the current accumulated iteration of the search operation. Unlike the first scheme, each bacterium in the same iteration moves with different sizes towards the next position and hence the exploration and exploitation strategies in one particular iteration are presented at the same time. Moreover, incorporation of tuneable parameters into both schemes has increased the flexibility on the determination of bacteria step size with a predefined range while in motion. The proposed algorithms have been tested with several unimodal and multi-modal standard benchmark functions with different dimensions and fitness landscapes. The performances of the algorithms have been statistically evaluated in terms of their convergence speed, fitness accuracy and frequency to achieve optimum solution. The significant difference between the original and the proposed algorithms has been statistically assessed based on nonparametric Friedman and Wilcoxon signed rank tests and parametric t-test approaches. Moreover, the algorithms have been employed to optimise a neural network model to represent the TRS behaviour in vertical channel motion. A nonparametric modelling approach with a NARX structure has been used as the candidate dynamic model for the system. The adoption of these schemes does not add extra complexity to the original structure of BFA hence maintaining the total computational time to complete the whole search operation. In terms of algorithm performance, the numerical and graphical results of the benchmark functions test and the dynamic modelling of the TRS have shown that the proposed algorithms have significantly outperformed the original algorithm in terms of fitness accuracy and convergence speed respectively. Moreover, the oscillation problem of the BFA when approaching the optimum location has also significantly been resolved due to the dynamic step size throughout the whole search operation. For the benchmark functions tests, adaptive formulation based on iteration index has resulted in the best accuracy while the inclusion of the individual bacterium fitness value in the second scheme has resulted performance with higher consistency. In terms of real application, the results of both in the modelling and validation phases of the dynamic modelling show that the estimated neural network model optimised by the proposed algorithms has better represented the actual system with significant reduction of error. Moreover, the models optimised by the proposed algorithms have shown better correlation test results in the validation phase. In this work, the proposed algorithms have been shown to satisfactorily solve a real world problem with high dimension. At this stage, the modification of the algorithm is focusing on the adaptive approach. In the future, another possibility to look at significantly improving the algorithm performance is to synergize the algorithm with other types of nature-inspired optimization algorithms where strengths of two or more algorithms can be combined together. Second, for online tuning and real time implementation, the complex structure of the algorithms can also be simplified so that the total computation time for the whole search operation can be significantly reduced while at the same time improving the performance of the original algorithms. Third, the proposed algorithms can be enhanced into multi-objective type algorithms to solve more complex and real world problems with two or more conflicting objectives where a trade-off between the objectives can be properly handled.

1529

In terms of solving real world problems, the dynamic model of the TRS can be acquired through a combination of the proposed algorithms with NN-based type, AI-based type or combination of both types and hence their performance can be comparatively assessed. Moreover, a comparative assessment through various modelling approaches such as black box and grey box identification approaches can be explored. The proposed algorithms will be employed to optimise nonlinear controllers for the TRS. In both dynamic modelling and control design cases, online tuning approach can be implemented where all the dynamic model and controller parameters can be updated and optimised while the system is in motion. Finally, with a multi-objective feature in the proposed algorithms, dynamic modelling on both vertical and horizontal channels of TRS can be simultaneously estimated and optimised where an error function at each channel of TRS can be appropriately handled. For the case of control design, the conflicting objectives between the time-domain specification of the control performance can be efficiently compromised. Acknowledgments Ahmad Nor Kasruddin Nasir is on study leave from Faculty of Electrical and Electronics Engineering (FKEE), Universiti Malaysia Pahang (UMP) and is sponsored by Ministry of Higher Education, Malaysia (MOHE). References Biswas, A., Dasgupta, S., Das, S., & Abraham, A. (2007). Synergy of PSO and bacterial foraging optimization – a comparative study on numerical benchmarks. In Innovations in hybrid intelligent systems. Advances in soft computing (Vol. 44). Springer. Chen, Y., & Lin, W. (2009). An improved bacterial foraging optimization. In Proceeding of IEEE international conference on robotics and biomimetics, ROBIO2019, Guilin, Guangxi, China, 19–23 Dec. 2009 (pp. 2057–2062). Chen, S., & Billings, S. A. (1989). Representations of non-linear systems: The NARMAX model. International Journal of Control, 49(3), 1013–1032. Chu, Y., Mi, H., Liao, H., Ji, Z., & Wu, Q. H. (2008). A fast bacterial swarming algorithm for high-dimensional function optimization. In Proceeding of 2008 IEEE congress on evolutionary computation (CEC 2008), 1–6 June 2008, Hong Kong (pp. 3135– 3140). Das, K. N., & Mishra, R. (2013). Chemo-inspired genetic algorithm for function optimization. Applied Mathematics and Computation, 220, 394–404. Dasgupta, S., Das, S., Abraham, A., & Biswas, A. (2009). Adaptive computational chemotaxis in bacterial foraging optimization: An analysis. IEEE Transactions on Evolutionary Computation, 13(4), 919–941. Devia, S., & Geethanjalib, M. (2014). Application of modified bacterial foraging optimization algorithm for optimal placement and sizing of distributed generation. Expert Systems with Applications, 41(6), 2772–2781. Dorigo, M., & Di caro, G. (1999). Ant colony optimization: A new meta-heuristic. In Proceedings of the 1999 congress on evolutionary computation, CEC 99, Washington, DC 06–09 Jul 1999 (pp. 1470–1477). Farhat, I. A., & El-Hawary, M. E. (2009). Modified bacterial foraging algorithm for optimum economic dispatch. In Proceeding of electrical power & energy IEEE conference, EPEC 2009, Montreal Quebec, Canada, 22–23 October, 2009 (pp. 1–6). Goldberg, D. E. (1989). Genetic algorithms in search, optimisation and machine learning. New York: Addison Wesley Longman, Publishing Co., Inc.. Goshal, S. P., Chatterjee, A., & Mukherjee, V. (2009). Bio-inspired fuzzy logic basedtuning of power system stabilizer. Expert Systems with Applications, 36(5), 9281–9292. Haykin, S. (1994). Neural network: A comprehensive foundation. Upper Saddle River, NJ, USA: Prentice Hall PTR. Huang, W., & Lin, W. (2010). Parameter estimation of Wiener model based on improved bacterial foraging optimization. In Proceeding of 2010 international conference on artificial intelligence and computational intelligence, Sanya, China, 23–24 October 2010 (pp. 174–178). Kalkkuhl, J., & Castro, E. L. (1996). Nonlinear control based on the NARX plant representation. In Proceeding of IEEE international symposium on industrial electronics ISIE ’96, Warsaw, 17–20 June 1996 (pp. 115–120). Karaboga, D., & Basturk, B. (2007). A powerful and efficient algorithm for numerical function optimization: Artificial bee colony (ABC) algorithm. Journal of Global Optimization, 39(3), 459–471. Kennedy, J. F., & Eberhart, R. C. (1995). Particle Swarm optimization. In Proceedings of IEEE international conference of neural network, Perth, Australia (pp. 1942– 1948).

1530

A.N.K. Nasir et al. / Expert Systems with Applications 42 (2015) 1513–1530

Leontaritis, I. J., & Billings, S. A. (1985a). Input–output parametric models for nonlinear systems. Part I: Deterministic non-linear systems. International Journal of Control, 41(2), 303–328. Leontaritis, I. J., & Billings, S. A. (1985b). Input–output parametric models for nonlinear systems. Part II: Stochastic non-linear systems. International Journal of Control, 41(2), 329–344. Leva, A., & Piroddi, L. (2002). NARX-based technique for the modelling of magnetorheological damping devices. Smart Materials and Structures, 11(1), 79–88. Ljung, L. (1999). System identification: Theory for the user (2nd ed.). Englewood Cliffs, HJ: Prentice Hall. Majhi, R., Panda, G., Majhi, B., & Sahoo, G. (2009). Efficient prediction of stock market indices using adaptive bacterial foraging optimization (ABFO) and BFO based techniques. Expert Systems with Application, 36, 10097–10104. Mishra, S. (2005). Hybrid least square-fuzzy bacterial foraging strategy for harmonic estimation. IEEE Transaction on Evolutionary Computation, 9(1), 61–73. Niu, B., Fan, Y., Zhao, P., Xue, B., Li, L., & Chai, Y. (2010). A novel bacterial foraging optimizer with linear decreasing chemotaxis step. In Proceeding of second international workshop on intelligent systems and applications (ISA), Wuhan, China, 22–23 May 2010 (pp. 1–4). Palumbo, P., & Piroddi, L. (2000). Seismic behaviour of buttress dams: Nonlinear modeling of a damaged buttress based on ARX/NARX models. Journal of Sound Vibration, 239(3), 405–422.

Passino, K. M. (2002). Biomimicry of bacterial foraging for distributed optimization and control. IEEE Control System Magazine, 52–67. Sathyaa, P. D., & Kayalvizhib, R. (2011). Optimal segmentation of brain MRI based on adaptive bacterial foraging algorithm. Neurocomputing, 74(14–15), 2299–2313. Supriyono, H., & Tokhi, M. O. (2011). Adaptive schemes of chemotactic step size of bacterial foraging algorithm for faster convergence. Journal of Artificial Intelligence, 4(4), 207–219. Toha, S. F., Julai, S., & Tokhi, M. O. (2012). Ant colony based model prediction of a twin rotor system. Procedia Engineering, 41, 1135–1144. Ulagammaia, M., Venkatesha, P., Kannana, P. S., & Padhyb, N. P. (2007). Application of bacterial foraging technique trained artificial and wavelet neural networks in load forecasting. Neurocomputing, 70(16–18), 2659–2667. Venkiah, C., & Vinod Kumar, D. M. (2011). Fuzzy adaptive bacterial foraging congestion management using sensitivity based optimal active power rescheduling of generators. Journal of Applied Soft Computing, 11(8), 4921–4930. Xu, X., Liu, Y. H., Wang, A. M., Wang, G., & Chen, H. L. (2012). A new adaptive bacterial swarm algorithm. In Proceeding of 2012 eighth international conference on natural computing (ICNC 2012), Chongqing China, 29–31 May 2012 (pp. 991– 995). Yan, X., Zhu, Y., Chen, H., & Zhang, H. (2012). Improved bacterial foraging optimization with social cooperation and adaptive step size. Berlin Heidelberg: Springer-Verlag (pp. 634–640).