Swarm intelligence-based extremum seeking control

Expert Systems with Applications 38 (2011) 14852–14860

Contents lists available at ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa

Swarm intelligence-based extremum seeking control Chen Hong a,b,⇑, Kong Li b a b

Wuhan Second Ship Design and Research Institute, Wuhan, Hubei, China Huazhong University of Science and Technology, Wuhan, Hubei, China

a r t i c l e

i n f o

Keywords: Particle swarm optimization Extremum seeking control State regulation Adaptive control Swarm intelligence-based optimization

a b s t r a c t This paper proposes an extremum seeking control (ESC) scheme based on particle swarm optimization (PSO). In the proposed scheme, the controller steers the system states to the optimal point based on the measurement, and the explicit form of the performance function is not needed. By measuring the performance function value online, a sequence, generated by PSO algorithm, guides the regulator that drives the state of system approaching to the set point that optimizes the performance. We also propose an algorithm that first reshuffles the sequence, and then inserts intermediate states into the sequence, in order to reduce the regulator gain and oscillation induced by population-based stochastic searching algorithms. The convergence of the scheme is guaranteed by the PSO algorithm and state regulation. Simulation examples demonstrate the effectiveness and robustness of the proposed scheme.  2011 Elsevier Ltd. All rights reserved.

1. Introduction Regulation and tracking of system states to optimal setpoints or trajectories are typical tasks in control engineering. However, these optimal setpoints are sometimes difficult to be chosen a priori, or vary with the environmental condition changes. Extremum seeking control (ESC) is a kind of adaptive control schemes that can search for the optimal setpoints online, based on the measurement of the performance output or its gradient. ESC can be regarded as an optimization problem, and many of the schemes used in ESC are transferred from optimization algorithms. However, some optimization algorithms cannot be incorporated into the ESC framework easily, for the reason that, practical issues, such as stability, noise, regulation time, control gain and oscillation limitation, will prevent the use of some optimization algorithms from ESC context. Thus, the study on suitable combination of ESC and optimization algorithms is of great interest both in academics and in engineering. Unlike the traditional variational calculus-involved optimal control method, the explicit form of the performance function is not needed in ESC. Therefore, ESC is useful in the applications that the performance functions are difficult to model. After Krstic and Wang’s stability studies (Krstic & Wang, 2000), research on ESC has received significant attention in recent years. The recent ESC application examples include active flow control (Beaudoin, Cadot, Aider, & Wesfreid, 2006), bioreactor or chemical process control (Bastin, Nescaroni, Tan, & Mareels, 2009; Hudon, Guay, Perrier, & ⇑ Corresponding author at: Wuhan Second Ship Design and Research Institute, Wuhan, Hubei, China. E-mail address: [email protected] (C. Hong). 0957-4174/$ - see front matter  2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2011.05.062

Dochain, 2008; Hudon, Perrier, Guay, & Dochain, 2005), cascaded Raman optical amplifiers (Dower, Farrell, & Nesic, 2008), antilock braking system design (Zhang & Ordonez, 2007), thermoacoustic cooler (Li, Rotea, Chiu, Mongeau, & Paek, 2005), and fuel cell power plant (Zhong, Huo, Zhu, Cao, & Ren, 2008). There also have been considerable theoretical studies in ESC, such as stability studies on perturbation-based ESC (Krstic, 2000; Krstic & Wang, 2000), ESC for discrete-time systems (Joon-Young, Krstic, Ariyur, & Lee, 2002), PID tuning by ESC (Killingsworth & Krstic, 2006), ESC for nonlinear dynamic systems with parametric uncertainties (Guay & Zhang, 2003), and ESC for state-constrained nonlinear systems (DeHaan & Guay, 2005). The majority of ESC literature focused on two issues, the one is the searching for the optima, and the other is the regulation of the systems. The recent studies of Zhang and Ordonez (2007, 2009) presented a numerical optimizationbased ESC (NOESC) framework that takes the advantage of numerical algorithms to find the optima online. However, these algorithms are unable to find the global optima if the assumption that the performance functions are convex and continuous does not hold. Furthermore, the NOESC is sensitive to measurement noise, due to the poor robustness of the numerical algorithms. Particle swarm optimization (PSO) algorithm is a populationbased stochastic optimization method which first devised by Kennedy and Eberhart (1995). PSO algorithm mimics the foodseeking behavior of birds or fishes. Due to its simplicity and effectiveness, PSO algorithm witnesses a considerable interest and is applied in many areas. The convergence of PSO algorithm is studied by deterministic method (Eberhart & Shi, 2001) or stochastic process theory (Jiang, Luo, & Yang, 2007). Clerc and Kennedy (2002) presented a convergence condition of PSO algorithm. Rapaic and Kanovic (2009) studied the time-varied parameter PSO algorithm

C. Hong, K. Li / Expert Systems with Applications 38 (2011) 14852–14860

and the selection of the parameters. Studies have shown that PSO algorithm is able to handle a wide range of problems, such as integer optimization (Laskari, Parsopoulos, & Vrahatis, 2002), multiobjective optimization (Dasheng, Tan, Goh, & Ho, 2007), and global optimization of multimodal functions (Liang, Qin, Suganthan, & Baskar, 2006). The recent application of PSO algorithm includes power systems (del Valle, Venayagamoorthy, Mohagheghi, Hernandez, & Harley, 2008), flights control (Duan, Ma, & Luo, 2008), and nuclear power plants (Meneses, Machado, & Schirru, 2009), to name a few. In control engineering, PSO algorithm is usually employed to identify the models (Panda, Mohanty, Majhi, & Sahoo, 2007), or to optimize the parameters of the controller offline (ElZonkoly, 2006). PSO algorithm is usually regarded as an effective global optimization method. However, it is often used in offline optimization, and depends on the explicit form and the solvability of the performance functions. However, for some complex models, e.g. active flow control problems which described by Navier– Stokes equations, it is difficult to obtain the optimal parameters of the controllers by time-consuming numerical simulations. In this paper, we extend the numerical optimization-based ESC (Zhang & Ordonez, 2007) by incorporating PSO algorithm into the extremum seeking framework. We also address the practicability issues of this scheme, and propose a reshuffle-then-insertion algorithm to reduce the control gain and oscillation. In the proposed scheme, a sequence converging to the global optima is generated by PSO with reshuffle-then-insertion algorithm. The sequence used as a guidance to regulate the state of the plant approach to the optimal set point. This paper is organized as follows. Section 2 gives a problem statement, where a PSO-based ESC (PSOESC) framework is introduced. We then review the standard PSO algorithm in Section 3. The details of the PSOESC scheme for linear time invariant (LTI) systems and feedback linearizable systems are presented in Section 4, where the reshuffle-then-insertion approach for improving the practicability of the PSOESC is also proposed. Section 5 presents the results of the numerical experiments. Finally, Section 6 concludes the paper.

2. Problem statement In control practice, the problem of seeking for an optimal set point is encountered usually. In general, this problem can be represented as model

x_ ¼ f ðx; uÞ;

ð1Þ

y ¼ JðxÞ:

ð2Þ

14853

can apply any optimization algorithm to produce a guidance sequence to determine a trajectory to the optimal set point in the state space (Zhang & Ordonez, 2007). Similar to numerical optimization-based ESC (Zhang & Ordonez, 2009), we present a PSO-based ESC block diagram as Fig. 1, where the nonlinear plant F is modeled as (1) and the performance function J is (2). Unlike Zhang & Ordonez, 2009, we apply PSO algorithms to substitute the numerical optimization algorithms to produce the extremum seeking sequence {Xk}. The sequence {Xk} is generated as the target state in every seeking iterative step, based on the online measurement of y. The regulator K regulates the state of F follows the sequence as X1, X2, . . ., Xk, toward the optimal set point. 3. Particle swarm optimization PSO is a population-based random optimization algorithm that mimics the behavior of bird or fish swarm in searching food. In the swarm, each particle has a variable speed, moving toward the positions of its own best fitness achieved so far and the best fitness achieved so far by any of its neighbors. Let S  Rn is an n-dimensional search space. The size of the particle swarm is denoted as N. The position of the ith particle is repre~ i ¼ ðxi1 ; xi2 ; . . . ; xin ÞT 2 S, and its sented as an n-dimensional vector X velocity is denoted as Vi = (vi1, vi2, . . ., vin)T 2 S, where i = 1, 2, . . ., N is the identity of each particle. The best position it has achieved so far is P pbst ¼ ðpi1 ; pi2 ; . . . ; pin ÞT 2 S. The best position achieved so i far in its neighborhood is Pgbst. On the time step k, the update equation of the basic PSO algorithm is (Kennedy & Eberhart, 1995)

  ~ i ðkÞ V i ðk þ 1Þ ¼ V i ðkÞ þ cr1 ðkÞ Ppbst ðkÞ  X i   ~ i ðkÞ ; þ cr 2 ðkÞ P gbst ðkÞ  X

ð4Þ

~ i ðk þ 1Þ ¼ X ~ i ðkÞ þ V i ðk þ 1Þ; X

ð5Þ

where, c is an acceleration coefficient, r1, r2 are two independent random numbers with uniform distribution in the range of [0, 1]. Though this version of PSO has been shown to perform well in some optimization cases, it is not suitable in ESC context for its possible explosion property will lead to an unfeasible large control gain. Clerc and Kennedy (2002) suggested a PSO algorithm which is known as the standard PSO, by introducing constriction coefficient v. The update equation in the standard PSO is

h   ~ i ðkÞ V i ðk þ 1Þ ¼ v V i ðkÞ þ u1 r 1 ðkÞ Ppbst ðkÞ  X i  i ~ i ðkÞ ; þu2 r2 ðkÞ Pgbst ðkÞ  X

ð6Þ

~ i ðk þ 1Þ ¼ X ~ i ðkÞ þ V i ðk þ 1Þ; X

ð7Þ

n

where x 2 R is the state, u 2 R is the input, y 2 R is the performance output to be optimized, f : D  R ! Rn is a sufficiently smooth function on D, and J : D ! R is an unknown function. For simplicity, we assume D ¼ Rn in this paper. Without loss of generality, we consider the minimization of the performance function (2). Unlike optimal control, extremum seeking control finds the optimal set point by online measurement of the performance function, without the knowledge of its explicit form. ESC can be considered as a class of constrained optimization problem whose constraint is the differential Eq. (1), instead of the algebraic constrains in traditional optimization problems. Then, ESC control problem can be stated as:

min x2D

s:t:

JðxÞ x_ ¼ f ðx; uÞ:

ð3Þ

When (1) is controllable, there always exists a control u such that state x transfers to any position in Rn in a finite time. We then

Fig. 1. PSO-based extremum seeking block diagram.

14854

C. Hong, K. Li / Expert Systems with Applications 38 (2011) 14852–14860

where v, u1, u2 are nonnegative constant real numbers. Obviously, by choosing appropriate parameters, (6) can transform to

invariant systems (LTI), feedback linearizable systems, and inputoutput feedback linearizable systems.

  ~ i ðkÞ V i ðk þ 1Þ ¼ xV i ðkÞ þ c1 r 1 ðkÞ Ppbst ðkÞ  X i   ~ i ðkÞ ; þ c2 r2 ðkÞ Pgbst ðkÞ  X

4.1. PSOESC for LTI systems

ð8Þ

where x, c1, c2 are nonnegative constant real numbers. The velocity update Eq. (8) has three major components. The first component is referred to as ‘‘inertia’’ for it models the tendency of the particle to continue in the same direction it has been traveling. The second component is referred to as ‘‘memory’’ for it models the moving towards the best position ever found by the given particle. The third component is referred to as ‘‘cooperation’’ for it models the moving towards the best position by particles in neighborhood (del Valle et al., 2008). From the formulation above, the standard PSO algorithm can be described as Algorithm 1 (Standard PSO algorithm). Step (1) Initialize the swarm by assigning a random position to each particle. Step (2) Evaluate the fitness function for each particle. Step (3) Identify the best position Ppbst of each particle. i Step (4) Identify the best position Pgbst. Step (5) Update the velocities and positions of all the particles using (6)or (8) and (7). Step (6) Repeat (2)–(5), until a stop criterion is met. Because the positions of particles involve randomness, the convergence of PSO-involved algorithms is defined in a possibility sense in this paper. We say Xi(k) converges in mean square to Q⁄, when limk!1 EjX i ðkÞ  Q  j2 ¼ 0, for "i 2 {1, 2, . . ., N}, where N is the population size of the swarm, and E represents the expectation operator.

Consider a single-input-single-output (SISO) linear timeinvariant system

x_ ¼ Ax þ Bu

ð9Þ

with the performance function defined as (2). The matrices A and B are given, while J is unknown. We will find the minimum of J online by the measurement of the function value y. To do this, the following assumption is needed to ensure the feasibility of ESC for the LTI system (9). Assumption 1. The LTI system (9) is controllable. From the linear system theory, we have the following lemma that states the existence of a perfect regulator. Lemma 1. For LTI system (9), when Assumption 1 holds, given system state x(tk) on time tk, regulation time dk, and Xk+1, let tk+1 = tk + dk, the control input

uðtÞ ¼ BT eA where

Gðdk Þ ¼

Z

dk

T

ðt kþ1 tÞ

G1 ðdk Þ½eAdk xðt k Þ  X kþ1 

T eAs BBT eA s ds

Proof. Refer to the Theorem 5 in Jiang et al. (2007). h Remark 1. Theorem 1 shows that the standard PSO algorithm guarantees every particle converges in mean square to the best position of the whole swarm. However, that position may not be the global optima or the local optima. There are modified PSOs that converge in mean square to local optima ( van den Bergh & Engelbrecht, 2002) or global optima (Xin & Yangmin, 2007). In order to ensure the convergence to the optimal value, modified PSO algorithms need to make tradeoffs between exploration and exploitation, and numerous papers have presented various modified PSOs for this issue (Rapaic & Kanovic, 2009; van den Bergh & Engelbrecht, 2002; Xin & Yangmin, 2007; Zihui et al., 2008). For simplicity, in the following section we will focus on standard PSO algorithm, assuming that the standard PSO algorithm can converge in mean square to the global optima in the ESC cases. For the same reason, we also drop the ‘‘mean square’’ in the convergence analysis.

ð11Þ

0

then x(tk+1) = Xk+1. Proof. From the assumption, the LTI system (9) is controllable, then the controllability Gramian G(dk) is nonsingular, thus the control input (10) is feasible. From linear system theory, the solution of (9) gives

xðt kþ1 Þ ¼ eAdk xðtk Þ þ Theorem 1. Given x, c1, c2, when 0 6 x < 1, c1 + c2 > 0, and 0 < gð1Þ < c22 ð1 þ xÞ=6, where gðkÞ ¼ k3  ðw2 þ R  xÞk2 þ 2 xðw2  R  xÞk  x3 , and w ¼ 1 þ x  c1 þc 2 , are all satisfied, then the particle systems driven by Algorithm 1 will converge in mean square to Pgbest.

ð10Þ

Z

t kþ1

eAðtk sÞ BuðsÞds

ð12Þ

tk

Then, substituting (10) into (12) will complete the proof. h Remark 2. From Lemma 1, there always exists a control u that regulates the state to any given state from any initial state, provided the LTI system is controllable. Thus, we can device a scheme that drives the state of the system to the global optima along a sequence produced by any optimization algorithms including that of swarm intelligence-based. By combining the Algorithm 1 and ESC, the PSO-based ESC scheme can be formed as follows. Algorithm 2. PSO-based ESC scheme for LTI systems

Step (1) Initialize the particle swarm, and let i = 1, k = 1. Step (2) Let particle i’s position as the ith target state, i.e. let ~ i ðkÞ in (10) to get the control u. X kþ1 ¼ X Step (3) The u regulates the plant to the target state, and the performance output y is readily to be measured by sensors. ~i ðkÞ of the particle. Step (4) Let the output y be the fitness y Step (5) Let i = i + 1, repeat 2–4, until i = N, where N is the predefined size of the particle swarm. Step (6) Update the velocity and position according to (6) and (7). Step (7) Let k = k + 1, repeat 2–5, until a stop criterion is met.

4. PSO-based extremum seeking scheme Similar to Zhang and Ordonez (2007), we discuss the PSO-based extremum seeking control scheme in the order of linear time-

We say (2)–(4) is a PSO loop, and (2)–(7) an ESC loop. The convergence result of Algorithm 2 can be shown by Lemma 1 and Theorem 1, and is presented as the following theorem.

C. Hong, K. Li / Expert Systems with Applications 38 (2011) 14852–14860

Theorem 2. Let X⁄ be the solution of optimization problem (3), i.e. X  ¼ arg min JðxÞ, if the solution of problem minx2D JðxÞ obtained by x2D

Algorithm 1 converges to X⁄, then Algorithm 2 drives system (9) to its minimum performance function value J⁄.

~ i denote the position of the ith particle on the kth ESC Proof. Let X k loop step. The PSO scheme described as Algorithm 1 produces a sequence

~ X ~ 1 X; ~ ...;X ~i ; . . . X ~ 1; X ~2; . . . ; X ~N; X ~ 1; X ~2; . . . X ~N; . . . ; X ~ 1 X; ~Ng fX 1 1 1 2 1 2 k k k k

ð13Þ

in k steps. From Theorem 1, the sequence converges to Pgbest, and from the assumption in the statement of Theorem 2 we also have Pgbest = X⁄, when k ? 1. h Let x(t0) denote the state of LTI system (9) at time t = t0. From ~ 1 . Then, at time Algorithm 2, we have the target state X 1 ¼ X 1 ~ 1 . For the same t1 = t0 + d0, from Lemma 1, we will have xðt1 Þ ¼ X 1 ~ 2 ; xðt 3 Þ ¼ X ~ 3 ; . . . ; xðt N Þ ¼ X ~N. reason, we also have xðt2 Þ ¼ X 1 1 1 ~i , By induction, at the Kth ESC loop, we suppose xðtiK Þ ¼ X K where i = 1, 2, . . ., N. Then, at the (K + 1)th ESC loop, from the Lemma 1, we will have

xðtKNþ1 Þ ¼ eAdk xðt KN Þ þ

Z

t KNþ1

t KN

~1 ; eAðtKNþ1 sÞ BuðsÞds ¼ X Kþ1

Remark 5. The Algorithm 2 requires a regulator to drive the state of the system (9) from one to another along the sequence produced by the PSO loop. However, due to the randomness feature of the update equation (6), at some stages, the regulator gain may be very large, and the state of the system would oscillate rashly. This could cause serious problems in real-world applications. This difficulty may hinder PSO or any other swarm intelligence-based algorithms from being applied in the ESC context. We will address this issue in Section 4.4, and provide an algorithm that rearranges the sequence to reduce the control gain and oscillation. Remark 6. It is not difficult to extend the Algorithm 2 to the multiinput-multi-output (MIMO) systems, like the NOESC schemes presented in Zhang and Ordonez (2007). Moreover, other design of controller (10) can be applied for a more robust regulator as suggested in Zhang and Ordonez (2007, 2009). 4.2. PSOESC for feedback linearizable systems Consider a SISO nonlinear affine system

x_ ¼ f ðxÞ þ gðxÞu

ð17Þ

ð15Þ

with the performance function defined as (2), where f and g are smooth functions on D. Assuming the system (17) is feedback linearizable on D, we have that there exists a diffeomorphism T : D ! Rn such that Dz = T(D) contains the origin, and transforms the system (13) into the form (Khalil, 2002)

ð16Þ

z_ ¼ Az þ BcðxÞ½u  aðxÞ;

ð14Þ

where T ~ 1 ; uðtÞ ¼ BT eA ðtKNþ1 tÞ G1 ðdk Þ½eAdk x xðt KN Þ  X Kþ1 Z dk T Gðdk Þ ¼ eAs BBT eA s ds:

14855

0

~ i , where For the same reason, we will have xðt KNþi Þ ¼ X Kþ1 i = 1, 2, . . ., N. Thus, the Algorithm 2 guarantees that the state of system (9) follows exactly the sequence

~ 1; X ~2; . . . ; X ~N; X ~ 1; X ~2; . . . X ~N; . . . ; X ~ 1; X ~ 1; . . . ; X ~i ; . . . X ~ N g: fX 1 1 1 2 1 2 k k k k Therefore, the performance output of system (9) converges to its minimum value J⁄. Remark 3. Since the standard PSO algorithm and its variants (del Valle et al., 2008; Xin & Yangmin, 2007) have shown their effectiveness in global optimization problems, it is safe to assume Algorithm 2 converge to the global optima in ESC. Thanks to the global searching ability of PSO algorithm, we need not apply the assumptions in Zhang and Ordonez (2007) to ensure the global convergence condition for the numerical-based optimization algorithms. The performance function J can be nonconvex and discontinuous in ESC context now.

ð18Þ

where z = T(x), and (A, B) is controllable, c(x) is nonsingular for all x 2 D. Then, we can extend the results on LTI system in 4.1 to the feedback linearizable system (17). Let Zk+1 = T(Xk+1), then we can design a regulator

uðtÞ ¼ aðxÞ þ c1 ðxÞu1 ðtÞ

ð19Þ

where T

Adk u1 ðtÞ ¼ BT eA ðtkþ1 tÞ W 1 Tðxðtk Þ  Z kþ1 ; c ðdk Þ½e Z dk T W c ðdk Þ ¼ eAs BBT eA s ds;

ð20Þ ð21Þ

0

such that the state of system (17) transfers from x(tk) to Xk+1 in a finite time dk. Then, we have the PSOESC scheme for the feedback linearizable system (17), represented as the following algorithm. Algorithm 3. PSOESC scheme for feedback linearizable systems

Remark 4. Due to the population-based feature of PSO algorithms, one may think that the use of PSO would require N sensors, or plants, able to provide N simultaneous measurements of J every time a new element in the optimization sequence is needed. However, this is not the case, since Algorithm 2 ensures that we can track each state represented by a particle in the swarm in a serial manner, thus the sensors needed in PSOESC are no more than other ESCs. It should be noted that, compared to NOESC, PSOESC have to spend more time to obtain the best state in an ESC loop. However, this cost can be regarded as the expense paid for the global optima. For example, in some active flow control cases, the plants keep working for a long period, and the drag reduction obtained by ESC would be great beneficial (Beaudoin et al., 2006), so it is reasonable to spend some time to find the global optima.

Step (1) Initialize the particle swarm, and let i = 1, k = 1. Step (2) Let particle i’s position as the target state, i.e. let ~ i ðkÞ, then let Zk+1 = T(Xk+1) to get the control u by (15). X kþ1 ¼ X Step (3) The u regulates the plant to the target state, and the performance output y is readily to be measured by sensors. ~i ðkÞ of the particle. Step (4) Let the output y be the fitness y Step (5) Let i = i+1, repeat 2–4, until i = N, where N is the predefined size of the particle swarm. Step (6) Update the velocity and position according to (6) or (8) and 7. Step (7) Let k = k + 1, repeat 2–5, until a stop criterion is met. The convergence analysis mainly follows the proof of Theorem 2. Similar remarks like 3–6 in Section 4.1 also apply here.

14856

C. Hong, K. Li / Expert Systems with Applications 38 (2011) 14852–14860

4.3. PSOESC for input–output feedback linearizable systems The results on state feedback linearizable systems in Section 4.2 can be extend to input-output feedback linearizable systems by substituting the regulator with appropriate designs, in a similar way as NOESC. The details refer to Zhang and Ordonez (2007). 4.4. Regeneration of the sequence produced by the PSO algorithm As indicated by Remark 5, tracing the sequence produced by the standard PSO algorithm would cause serious problems in practice. The sequence has to be regenerated to suit ESC context. Without loss of generality, we discuss the regeneration of the sequence produced by Algorithm 2 presented in Section 4.1. Consider system (6), and the state from time tk to tk+1. From (10), the output of the regulator k in a state transfer procedure is determined by the regulation time dk = xk+1  xk, the initial state x(tk), and the terminal state x(tk+1). Given the regulation time dk, if we want to minimize the maximum control, then we need an algorithm to reshuffle the sequence {xk} to minimize the maxð u ðtk ÞÞ. However, the optimal regeneration is difficult to k¼1;...;N

devise because it is NP-hard as Problem 1 shows. Problem 1. Let G = {V, E} be a complete weighted graph, where V = {1, 2, . . ., N} is the vertex set, indicating the states in sequence ~ k g, and E is the edge set. We also denote D as the distance matrix, fX where D = [di,j]NN, and

   Z dk 1     T A T dk  T AT s As Adk di;j ¼ B e e BB e ds ½e X i  X j    0

ð22Þ

dii ¼ þ1

ð23Þ

1

Now, the searching for the optimal sequence is equivalent to solve the programming problem represented as:

min

maxfdi;j xi;j ; 1 6 i; j 6 n; i – jg 8P n > > > xi;j ¼ 1 > > i¼1 > > > n >P < xi;j ¼ 1 s:t: j¼1 > > > > xi;j 2 f0; 1g > > >P P > > xi;j 6 jSj  1; 8S  E :

we choose the nearest state to x2 from the remainder states in {xk} as x3, and so on. Repeating the procedure will lead to a new sequence that each distance of the first k  1 states are minimal. The distance of the last two state, i.e. xk+1 and xk may be very large. Moreover, there may still be some unacceptable large distances in the reshuffled sequence {x1, x2, . . ., xk1}. However, the requests of the insertion procedure are much less than the original sequence. By combination of the reshuffle procedure and the insertion procedure above, we have the following algorithm. Algorithm 4. Reshuffle-then-Insertion algorithm for PSOESC Step (1) Obtain the distances di,j according to (22). Step (2) Let the last state of the last ESC loop be the basis point x1. Step (3) Find the state nearest to the basis point x1 in the ~ k g, and let the point be x2. sequence fX ~ k g. Step (4) Delete the state obtained in step 2) from fX Step (5) Compute the distance d1,2 between x1 and x2. Step (6) If d1,2 > R, then insert a new state into x1 and, and let the new state be x2, then, let the original x2 be x3. Step (7) Repeat step 5 and 6, until d1,2 >R, every time add 1 to the index of every point in the new sequence after x2. Step (8) Let the last point, i.e. the x2 obtained in step 3), be the ~ k g is new basis point, and repeat step 4 7, until the sequence fX empty. The result that the Algorithm 4 is applied on a random sequence in the range of [0, 10] is shown in Fig. 2. It can be seen that the algorithm smoothes the original sequence remarkably, and the control gain in each step is limited in a predefined bound. Remark 7. Algorithm 4 is also applicable to ESC based other swarm-intelligence-based algorithms, such as Artificial Bee Colony, Ant Colony Swarm and Bacteria Foraging. Then, we have a practical PSOESC scheme as follows. Algorithm algorithm

5. PSOESC

scheme

with

reshuffle-then-insertion

ð24Þ

i2S j¼S

where, |S| is the number of the vertices contained in G in S. The Problem 1 is also known as the bottleneck traveling salesman problem (BTSP), and is shown NP-hard (Kabadi, 2002), i.e. there is no polynomial algorithm for this kind of problems so far. We propose here an algorithm to regenerate the PSO-produced sequence for the practical use of PSOESC, based on the following consideration. In practice, limiting maximum control gain within a certain bound is sufficient to have a practical controller. From (10), we have that for each predefined R > 0, there always exists a regulation time dk such that ||u(t)|| < R. Therefore, the control gain can be limited in a bound by choosing a sufficiently long regulation time dk. However, too long regulation time may be unacceptable in practice. Another feasible approach is to insert new intermediate states between two states that may lead to unacceptable large control gain. These new intermediate states would smooth the original state sequence produced by the PSO algorithm. However, this procedure may prolong the regulation time as well. Therefore, we propose a reshuffle procedure below, in order to reduce the requests of the insertion procedure as less as possible. Consider the sequence {xk} produced by PSO in an ESC loop. First, we simply choose the nearest state to x1 from {xk} as x2, then

Step (1) Initialize the particle swarm, and let i = 1, k = 1. ~ k g by Algorithm 4. Step (2) regenerate the sequence fX Step (3) Let particle i’s position as the ith target state, and get the control u by (10). Step (4) The u regulates the plant to the target state, and the performance output y is readily to be measured by sensors. ~i ðkÞ of the ith particle. Step (4) Let the output y be the fitness y Step (5) Let i = i+1, repeat 3–4, until i = N0 , where N0 is the size of the regenerated sequence. Step (6) Update the velocity and position according to (6) or (8) and 7. Step (7) Let k = k + 1, repeat 2–6, until a stop criterion is met. 5. Numerical experiments The numerical experiments were carried out on a personal computer running on the Matlab/Simulink environment. The parameters of the standard PSO were chosen as w = 0.9, c1 = 0.12, and c2 = 0.012. For the purpose of simulation, performance functions were provided explicitly, while they would be unknown in realworld applications. As mentioned above, the convergence of the PSOESC scheme is described in mean square sense, and the convergence to the global optima is not guaranteed by the standard PSO. Therefore, the given illustrations are typical results obtained by numerous simulations. We define the success rate of the PSOESC

14857

C. Hong, K. Li / Expert Systems with Applications 38 (2011) 14852–14860

in [0, 1]. The two minima 0.225, 0.883 can be easily found by the optimtool toolbox in Matlab by choosing different start point, and the function value according to the two minima are 0.080 and 0.308, respectively. When setting the population size as 10 particles, we observed a 100% success rate in 120 repeated tests. We also tested a swarm with only 2 particles, and have a success rate of 82% with much less steps for convergence than that of 10 particles cases. This is shown in Fig. 4. By comparison, the performance of NOESC is dependent on the initial point. It would convergence rather fast provided a good initial point was chosen. However, if we choose h0 = 0 as the initial point, for example, the seeking procedure would converge to the local minimum 0.225.

10 9 8 7 6 5 4 3 2

5.2. Second-order LTI system

1

original regenerated

Consider a system which has been discussed in Zhang and Ordonez (2007):

0 0

5

10

15

20

25

30

35

K

x_ ¼

Fig. 2. The regeneration of the target state sequence. The original sequence is built up with 30 random numbers, indexed by k, distributed uniformly in [0, 10]. The maximal distance between two neighbors in the original sequence is 7.2183, which is reduced to 3 by the regeneration algorithm. The two inserted intermediate states in the regenerated sequence are indicated by arrows.



0

1

2 3



xþ

  0 u; 1

ð27Þ

y ¼ JðxÞ ¼ 100ðx2  x21 Þ2 þ ð1  x1 Þ2 :

ð28Þ

scheme as Rc = Nc/Nt, where Nc is the number of times PSOESC converged to the known global optima, and Nt the number of repeated test times. In practical applications, the measurement noise, unmodeled dynamics, input disturbance, and computational error are inevitable, so the convergence to the precise optima is unnecessary. We define the error limit as ±0.05J⁄, where J⁄ is the known global minimal value.

The performance function (28) is also known as Rosenbrock function whose minimum is x⁄ = [1 1]T, and J(x⁄) = 0. In this case, we use 10 particles, and the result is shown in Fig. 5, where it can be seen that the PSOESC drives the performance function converge to the minimum successfully. To test the robustness of the PSOESC, we disturbed y with a random number uniformly distributed with amplitude 0.2. As Fig. 6 shows, the PSOESC scheme drives the state of the system approaching to the optimal set point successfully, which is consistent with the stability analysis of swarms in noisy environment (Liu & Passino, 2004).

5.1. First-order single-input–single-output (SISO) LTI system

5.3. PSOESC for ABS design

We first considered a LTI system

h_ ¼ h þ u;

ð25Þ

Consider a one wheel ABS design case which is usually used in ESC literatures (Zhang & Ordonez, 2006, 2007). The problem can be modeled as (Zhang & Ordonez, 2007)

with the performance function defined as

JðhÞ ¼ 0:4 sinð9hÞ sinðhÞ;

ð26Þ 0.1

where u is the control obtained by (10). The plot of J(h) in the range of [0, 1] is illustrated in Fig. 3. Obviously, there are two local minima J(θ)

0.3

0

0.2

-0.1 -0.2 -0.3 -0.4

0.1 0

0

20

40

60

80

100

120

140

0

20

40

60

80

100

120

140

J(θ)

0.9 0.8

-0.1

0.7

θ

-0.2

0.6 -0.3 -0.4

0.5 0.4 0

0.1

0.2

0.3

0.4

0.5

θ

0.6

0.7

0.8

0.9

Fig. 3. The plot of the function J(h) = 0.4sin(9h)sin(h) in [0, 1].

1

k Fig. 4. PSOESC for the first-order LTI system, where k indicates the iteration steps.

14858

C. Hong, K. Li / Expert Systems with Applications 38 (2011) 14852–14860

J(x)

1

g ¼ eaðtkþ1 tÞ

0

50

100

150

200

250

0

50

100

150

200

250

x1

1 0.5 0

0.5

0

50

100

k

150

200

250

Fig. 5. PSOESC for the second-order LTI system. The performance function is the Rosenbrock function. The size of the particle swarm is 15.

J(x)

4 2 0 -2

e2as ds

1

ðeadk kðt k Þ  kkþ1 Þ;

ð33Þ

where kk+1 is the target state produced by the PSO algorithm applied in ESC loop. In the simulation, the parameters were chosen as (Zhang & Ordonez, 2007): M = 400 kg, B = 0.01, R = 0.2 m , k(0) = 0, a = 1, dk = 0.5. For the simulation purpose, we also postulated that the relation of l(k) and k satisfies

lðkÞ ¼ 2l

1

0

dk

0

0.5 0

x2

Z

0

20

40

60

80

100

120

140

160

k k 2

k þ k2

ð34Þ

;

where, l⁄ = 0.25, k⁄ = 0.6. Obviously, the optimal k of (34) is k⁄ = 0.6. PSOESC scheme with 10 particles worked well, and we observed a 99% convergence rate in 120 repeated tests. We also tested a swarm with only a single particle, and have a convergence rate of 61%. This is shown in Fig. 7. To test the robustness of the PSOESC, we added on l a random disturbance with uniform distribution in the range of [0.2, 0.2] to simulate the influence of noises. As Fig. 8 shows, the PSOESC scheme drives the state of the system converging to the optimal slip. By comparison, as Fig. 9 shows, the NOESC scheme, which is based on the simplex search method in our experiment, converges fast to the optimal slip when no measurement noises are added, whereas it converges to the wrong point when measurement is disturbed by noise,. 5.4. PSOESC for a system with incontinuous performance function Finally, we discuss a LTI system with a discontinuous performance function. Consider the system (20) with performance function defined as

1 0.5 0

0

20

40

60

80

100

120

140

160

x2

1.5 1 0.5 0

0

20

40

60

80

k

100

120

140

160

Fig. 6. PSOESC for the second-order LTI system with measurement noise.

M v_ ¼ NlðkÞ;

ð29Þ

Ix ¼ Bx þ NRlðkÞ  u;

ð30Þ

u¼

  Iv R x IN lðkÞg; þ ðak þ gÞ  Bx þ Iv M v 2 R

ð31Þ

such that transform the system (29) and (30) into the form

k_ ¼ ak þ g; where g is the regulator devised according to (10), and

h 2 ½0:5; 0:6Þ;

ð35Þ

h 2 ½0:6; 0:9Þ; h 2 ½0:9; þ1Þ:

0.4 0.2 0 -0.2 -0.4

0

20

40

60

0

20

40

60

80

100

120

140

80

100

120

140

1 0.5 0 -0.5 -1

ð32Þ

h 2 ð1; 0:5Þ;

The plot of this function in range [0, 1] is illustrated in Fig. 10. The PSOESC scheme drives the performance output of the system converge to the optima, as Fig. 11 shows. This test illustrates the intrinsic advantage of the PSOESC in solving discontinuous optimization problems. By comparison, the existing NOESC (Zhang & Ordonez, 2005) is unable to deal with such kind of problems.

-λ

where v denotes the linear velocity, x is the angular velocity, N = Mg is the weight of the wheel, R is the radius of the wheel, I the moment of inertia of the wheel, Bx the braking friction torque, u braking torque, l(k) the friction force coefficient, and k ¼ v vxR is the slip, 0 6 k 6 1 for xR 6 v. There exists a maximum l⁄ for the friction force coefficient l(k) at the optimal slip k⁄. The main challenge in designing ABS systems is to devise a robust control scheme that can handle the uncertainty due to environment, for the reason that k⁄ and l⁄ will change as the road condition changes. Now, the design purpose is to device a control u such that maximize the l(k). Since the v_ can be measured by an acceleration sensor, we can let the control be

8 0:5; > > > < 0:2; y ¼ JðhÞ ¼ > 0:8; > > : 2;

-μ(λ)

x1

1.5

k

Fig. 7. A single particle-based ESC for the ABS problem.

14859

C. Hong, K. Li / Expert Systems with Applications 38 (2011) 14852–14860

0.7

0.2

0.6

0

J(θ)

-μ(λ)

0.1

-0.1 -0.2 -0.3

0

20

40

60

80

100

120

0.5 0.4 0.3

140

0.2 0.5

1

-0.5

0.8

20

40

60

0

20

40

60

80

100

120

140

80

100

120

140

-1

θ

-λ

0

0

0

20

40

60

k

80

100

120

0.6

140

0.4

Fig. 8. PSOESC for the ABS with measurement noises.

0.2

k 0.4

Fig. 11. PSOESC for a system with discontinuous performance function.

-μ(λ)

0.3 0.2

6. Conclusion

0.1 0

0

5

10

15

20

-4

8

k

25

30

35

40

45

x 10

-μ(λ)

6 4 2 0 0

50

100

k

150

200

250

Fig. 9. NOESC for the ABS without measurement noises (top), and with measurement noises (bottom).

2 1.8

Acknowledgement

1.6

The authors would like to thank Dr. C. Zhang in Etch Engineering Technology for his comments and useful suggestions, especially the naming of the reshuffle-then-insertion algorithm.

1.4

J(θ)

This paper presents a PSO-based ESC scheme that is able to find the optimal set point online. The PSO algorithm produces a sequence converging to the global optima. The sequence serves as a guidance to regulate the state of the plant approaching to the optimal set point. We also propose a reshuffle-then-insertion algorithm that is able to reduce the control gain and oscillation, thus improving the practicability of PSOESC scheme. The numerical experiments show the effectiveness of the scheme. The PSOESC scheme found the global optima successfully in various cases. The simulations also demonstrate the ability of PSOESC scheme for handling measurement noise. The disadvantage of PSOESC is that it costs more time in every ESC loop. However, the robustness to measurement noise makes PSOESC preferred to NOESC in some cases. The scheme proposed in this paper will be easily extend to other swarm intelligence-based algorithms, such as Artificial Bee Colony, Ant Colony Optimization and Bacteria Foraging Algorithm.

1.2

References

1 0.8 0.6 0.4 0.2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

θ Fig. 10. The plot of function (35).

0.8

0.9

1

Bastin, G., Nescaroni, D., Tan, Y., & Mareels, I. (2009). On extremum seeking in bioprocesses with multivalued cost functions. Biotechnology Progress, 25, 683–689. Beaudoin, J., Cadot, O., Aider, J., & Wesfreid, J. E. (2006). Drag reduction of a bluff body using adaptive control methods. Physics of Fluids, 18, 085107. Clerc, M., & Kennedy, J. (2002). The particle swarm – Explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 6, 58–73. Dasheng, L., Tan, K., Goh, C., & Ho, W. (2007). A multiobjective memetic algorithm based on particle swarm optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 37, 42–50. DeHaan, D., & Guay, M. (2005). Extremum-seeking control of state-constrained nonlinear systems. Automatica, 41, 1567–1574.

14860

C. Hong, K. Li / Expert Systems with Applications 38 (2011) 14852–14860

del Valle, Y., Venayagamoorthy, G. K., Mohagheghi, S., Hernandez, J. C., & Harley, R. G. (2008). Particle swarm optimization: Basic concepts, variants and applications in power systems. IEEE Transactions on Evolutionary Computation, 12, 171–195. Dower, P. M., Farrell, P. M., & Nesic, D. (2008). Extremum seeking control of cascaded Raman optical amplifiers. IEEE Transactions on Control Systems Technology, 16, 396–407. Duan, H., Ma, G., & Luo, D. (2008). Optimal formation reconfiguration control of multiple ucavs using improved particle swarm optimization. Journal of Bionic Engineering, 5, 340–347. Eberhart, R., & Shi, Y. (2001). Tracking and optimizing dynamic systems with particle swarms. In Proceedings of the 2001 congress on evolutionary computation (pp. 94–100). El-Zonkoly, A. (2006). Optimal tuning of power systems stabilizers and AVR gains using particle swarm optimization. Expert Systems with Applications, 31, 551–557. Guay, M., & Zhang, T. (2003). Adaptive extremum seeking control of nonlinear dynamic systems with parametric uncertainties. Automatica, 39, 1283–1293. Hudon, N., Guay, M., Perrier, M., & Dochain, D. (2008). Adaptive extremum-seeking control of convection-reaction distributed reactor with limited actuation. Computers and Chemical Engineering, 32, 2994–3001. Hudon, N., Perrier, M., Guay, M., & Dochain, D. (2005). Adaptive extremum seeking control of a non-isothermal tubular reactor with unknown kinetics. Computers and Chemical Engineering, 29, 839–849. Jiang, M., Luo, Y., & Yang, S. (2007). Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm. Information Processing Letters, 102, 8–16. Joon-Young, C., Krstic, M., Ariyur, K. B., & Lee, J. S. (2002). Extremum seeking control for discrete-time systems. IEEE Transactions on Automatic Control, 47, 318–323. Kabadi, P. S. (2002). The bottleneck TSP. In A. P. G. Gutin (Ed.), The traveling salesman problem and its variation (pp. 697–735). Netherlands: Kluwer Academic Publishers. Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of IEEE international conference on neural networks (pp. 1942–1948). Khalil, H. (2002). Nonlinear systems (3rd ed.). Englewood Cliffs, New Jersey: Prentice-Hall. Killingsworth, N. J., & Krstic, M. (2006). PID tuning using extremum seeking: Online, model-free performance optimization. IEEE Control Systems Magazine, 26, 70–79. Krstic, M. (2000). Performance improvement and limitations in extremum seeking control. Systems and Control Letters, 39, 313–326. Krstic, M., & Wang, H. (2000). Stability of extremum seeking feedback for general nonlinear dynamic systems. Automatica, 36, 595–601.

Laskari, E., Parsopoulos, K., & Vrahatis, M. N. (2002). Particle swarm optimization for integer programming. In Congress on evolutionary computation (pp. 1582–1587). Li, Y., Rotea, M., Chiu, G., Mongeau, L., & Paek, I. (2005). Extremum seeking control of a tunable thermoacoustic cooler. IEEE Transactions on Control Systems Technology, 13, 527–536. Liang, J., Qin, A., Suganthan, P., & Baskar, S. (2006). Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Transactions on Evolutionary Computation, 10, 281–295. Liu, Y., & Passino, K. M. (2004). Stable social foraging swarms in a noisy environment. IEEE Transactions on Automatic Control, 49, 30–44. Meneses Machado, M., & Schirru, R. (2009). Particle swarm optimization applied to the nuclear reload problem of a pressurized water reactor. Progress in Nuclear Energy, 51, 319–326. Panda, G., Mohanty, D., Majhi, B., & Sahoo, G. (2007). Identification of nonlinear systems using particle swarm optimization technique. In IEEE congress on evolutionary computation (pp. 3253–3257). Rapaic, M. R., & Kanovic, Z. (2009). Time-varying PSO convergence analysis, convergence-related parameterization and new parameter adjustment schemes. Information Processing Letters, 109, 548–552. van den Bergh, F., & Engelbrecht, A. (2002). A new locally convergent particle swarm optimiser. In IEEE international conference on systems, man and cybernetics (pp. 6–12). Xin, C., & Yangmin, L. (2007). A modified PSO structure resulting in high exploration ability with convergence guaranteed. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 37, 1271–1289. Zhang, C., & Ordonez, R. (2005). Numerical optimization-based extremum seeking control of lti systems. In 44th IEEE conference on decision and control (pp. 4428– 4433). Zhang, C., & Ordonez, R. (2006). Non-gradient extremum seeking control of feedback linearizable systems with application to abs design. In 45th IEEE conference on decision and control (pp. 6666–6671). Zhang, C., & Ordonez, R. (2007). Numerical optimization-based extremum seeking control with application to ABS design. IEEE Transactions on Automatic Control, 52, 454–467. Zhang, C., & Ordonez, R. (2009). Robust and adaptive design of numerical optimization-based extremum seeking control. Automatica, 45, 634–646. Zhong, Z., Huo, H., Zhu, X., Cao, G., & Ren, Y. (2008). Adaptive maximum power point tracking control of fuel cell power plants. Journal of Power Sources, 176, 259–269. Zihui, R., Jian, W., & Huizhe, Z. (2008). A new particle swarm optimization algorithm and its convergence analysis. In Second international conference on genetic and evolutionary computing (pp. 319–323).