- Email: [email protected]
Optimal Control of Switched Linear Systems by Particle Swarm Optimization
Optimal Control of Switched Linear Systems by Particle Swarm Optimization SAHBI BOUBAKER1, MOHAMED DJEMAI2,3,4, NOUREDDINE MANAMANNI5 and FAOUZI M’SAHLI6 1
High Institute of Technological Studies of Nabeul, Campus Universitaire-Merazka, 8000, Nabeul,Tunisia E-mail: [email protected] 2 Univ Lille Nord de France, F-59000 Lille, France 3 UVHC, LAMIH, F-59313 Valenciennes, France 4 CNRS, UMR 8530, F-59313 Valenciennes, France E-mail: [email protected] 5 Univ Reims, CRESTIC, F-51000 Reims, France Moulin de la Housse BP 1039, 51687 REIMS cedex 2 - France E-mail: [email protected] 6 National School of Engineers of Monastir, Rue Ibn Al-Jazzar, 5019, Monastir, Tunisia E-mail: [email protected] 1,6 Unité de Recherche en Commande Numérique de Procédés Industriels (CONPRI) National School of Engineers of Gabès, Rue Omar Ibn Al-Khattab, 6029, Gabès, Tunisia
Abstract: This paper deals with Optimal Control problems of Dynamical Switched Linear Systems (OCPDSLS). This special class of hybrid systems combines the standard control, where dynamic subsystems are described by differential linear equations with a switching law which activates, alternatively, these subsystems. For a pre-specified modal sequence, we demonstrate that switching time instants and continuous input can be solved simultaneously by using an evolutionary computation technique, namely Particle Swarm Optimization (PSO). Some structural and numerical difficulties encountered in solving such problems by conventional approaches are discussed. The mapping between PSO and OCPDSLS is established via a generic procedure to synthesize optimal control laws. An illustrative numerical nonconvex example is solved showing the effectiveness of the proposed technique in overcoming difficulties of this problem known to be NP-hard, non-linear and constrained one. Keywords: Switched Linear Systems, Switching sequence, Optimal Control, Optimization, Conventional Approaches, Particle Swarm Optimization (PSO). 1. INTRODUCTION Switched Linear Systems as a particular class of hybrid systems have been under intensive investigations during the last few years. Due to their many potential applications, they have attracted researchers in the fields of modelization [Branicky et al, 98], identification [Juloski et al, 05], observation [Saadaoui et al, 06] and optimal control [Xu, 02 and 04], [Riedinger et al, 05], etc ... This special class of systems involves interaction between continuous and discrete dynamics and has applications in various fields such as mechanical systems, automotive industry, chemical engineering and switching power converters [Benmansour et al, 07]. Recently, Optimal Control Problem of Dynamical Switched Linear Systems (OCPDSLS) has become a challenging research topic because of the large classes of phenomena they describe and the integration of computers in industrial processes control. These problems do not need only the solution of optimal continuous inputs, as in a conventional optimal control problem, but also the solution of optimal modal sequence and its corresponding set of switching instants. The basic difficulty of this kind of problems is the complexity growing exponentially on the number of the handled data, when they are solved by conventional approaches [Xu, 02 and 04].
In [Xu, 02], OCPDSLS has been solved via non-linear optimization based on direct differentiation of the objective function. The main difficulty encountered by this approach is the non-availability of information about derivatives of cost function with respect to switching instants. The General Switched Linear Quadratic (GSLQ) problem discussed in [Xu, 04] is transformed into an equivalent parameter selection problem parameterized by switching instants. For the two-stage proposed algorithm, it is obvious that the optimization of the switching instants may be usually nonconvex and the Newton iteration algorithm will easily fall in local minima. In [Luus, 03], the GSLQ problem is considered via a two-level algorithm. The high level, based on a direct search optimization, is devoted to searching the optimal switching instants. The low level based on an Iterative Dynamic Programming (IDP) procedure is used to compute optimal continuous input. The main difficulty encountered by the proposed approach is the combinatorial explosion due to the time and state spaces discretization. In this paper, we aim to apply a new evolutionary computation technique, namely PSO, to solve the optimal control of linear switched systems problem. Given a prespecified active sub-systems, one needs to seek both the optimal switching locations, over a finite time horizon, and the corresponding continuous input. For a given switching law, the continuous input has been demonstrated to be a state feedback law. Through time and state spaces discretization, we convert the original problem to a parameter selection one.
The switching instants and the gains of the continuous input law are coded as positions of the particles in the PSO paradigm. These particles freely fly through a multidimensional search space when looking for the optimal solution. A suitable mapping between PSO and OCPDSLS allows handling the continuous and the discrete variables together. Without requiring any regularity of the studied problem, the PSO algorithm provides global sub-optimal solutions and allows overcoming the difficulties discussed above. The paper is organized as follows. In section 2, we propose a unified modelling framework of switched linear continuoustime systems and the general optimal control problem for this class of systems is formulated. To be well-handled by the proposed optimization technique, we propose, an equivalent discrete-time formulation via a discretization procedure of the sub-systems, the switching law and the cost function to be minimized. Section 3 is devoted to the presentation of the PSO technique. In section 4, a generic procedure to compute sub-optimal solutions for the optimal control problem is addressed. We focus on the adaptations of the PSO-based algorithm used in the aim of overcoming the difficulties discussed in section 1. Section 5 is devoted to a literature benchmark, used in optimal control illustrations, to demonstrate the effectiveness of the proposed procedure. Finally, we draw some conclusions in section 6. 2. MODELLING AND OPTIMAL CONTROL OF A SWITCHED LINEAR SYSTEM 3.1 A continuous-time formulation In this paper, we consider linear switched systems consisting of subsystems, in the state-space continuous-time formalism:
. x (t ) = Aq x (t ) + B q u (t )
q ∈ Q = {1,..., M }
Note that the system switches from subsystem q j
subsystem q ( j +1) at the switching instant t q j .During the interval t q j −1 , t q j the configuration q j is active. A collection of states, x , continuous controls, u , and a sequence σ in t 0 , t f is known as an execution of the switched system. Assumption 2: We consider only non-Zeno executions. Assumption 3: The switched system is supposed to be without jumps. Problem P1: We consider a switched system as described by (1). Given an interval of time t 0 , t f , an initial state, x0 and a sequence of active subsystems {q1 ,..., q K } , an algorithm to solve an optimal control problem of this system has to find the optimal control continuous input, u* and the optimal switching instants (t *1 ,..., t *K ) such that the following costfunction
J (σ , u ) = ψ (x (t f )) +
j =K
∑∫ j =1
tj
t j −1
Lq j (x ,u )dt
If the cost function is quadratic, using a method similar to conventional quadratic optimal control problem, the solution of the Hamilton-Jacobi-Bellmann equations [Xu, 04], provides the continuous input as: j
Aq and Bq are matrices with suitable dimensions and Q indicates that the system will be in M configurations.
x (t ) ∈ ℝ n and u (t ) ∈ ℝ m are respectively the state vector and the continuous control input. For such a system, given the modal sequence, {q j ∈ Q , j = 1,..., K } where K is finite,
(4)
is minimized. ψ represents the terminal part of this cost function.
u (x , t , q j ) = −G q (t )x (t ) + E q j (t ), j=1,...,K (1)
to
(5)
The continuous input is then obtained through its statefeedback control law (5). The problem of computing the optimal continuous input is converted to a parameter selection one involving the gains of this law. Assumption 4: we assume that the gains vectors G q (t ) and E q j (t ), j=1,...,K , are constant for each active j
sub-system q j .
we define the sequence of active subsystems in a time interval t 0 , t f by σ , which, from a control point of view,
This assumption 4 is made in order to reduce the numerical complexity of the problem.
is viewed as a discrete input :
3.1 An equivalent discrete-time formulation
σ = {(q1 , t1 ),..., (q K , t K )}
(2)
Assumption1: The global system remains in a configuration q j during, at least, a time τ min , known as the dwell-time. The switching instants are constrained by:
t 0 < t1 < ... < t K < t f = t K +1
(3)
The interval of time t 0 , t f is divided into N sampling instants such that:
N =
tf −t0 T
(6)
T is the sampling period. Assumption 5: According to Shannon sampling theorem, the sampling period T is chosen such that its value is less than
the least value among all the time constants of all the subsystems {q j ∈ Q , j = 1,..., K } .
vk +1 = wk .vk + b1 .r1 .( pbest k − p k )
The sampled times are t h = hT , h = 0,..., N . We assume that the control and the state are piecewise constant in the interval . x (h + 1) − x (h ) , the th , th +1 . Through the relation x (t ) ≃ T continuous dynamics in equation (1) are replaced by their first order discrete-time equivalents
p k +1 = p k + v k +1
x (h + 1) = (T × Aq
+ I )x
(h ) + T × B q u (h )
q ∈ Q = {1,..., M }
(7)
I is the identity matrice having the same dimensions as Aq . Assumption 6: We assume that the switching instants coincide with entire numbers of sampling period. That is to say, for each couple of two consecutive configurations, it exists an integer N j verifying: t j ≃ N j ×T . The sequence of active subsystems is then defined as
σ ≃ {(q1 , N 1 ),..., (q K , N K )}
(8)
Problem P2: In the discrete-time equivalent formulation, an optimal control algorithm has to find the optimal continuous inputs, u* (h) , h ∈ {0,1,..., N − 1} , and the optimal switching
(11)
+ b2 .r2 .( pgbest k − pk )
w − wmin wk = wmax − max kmax where, • • • • • • •
(12) (13)
.k
p k : the position of each particle at iteration k , a candidate solution. v k : the velocity of each particle at iteration k . b1 and b2 : the cognitive and the social acceleration constants r1 and r2 : random numbers with values in the range [ 0,1] .
pbest k : the best position reached by each particle in the past, known as the “ personal_best ” pgbest k : the best position between neighbors of each particle , known as the “ global_best ” kmax : a maximum iterations number pgbest k
indices, (N *1 ,..., N *K ) , such that the following cost function
J (σ , u ) ≃ ψ (x (N )) +T ×
K
h =N
j
∑∑ j =1 h = N
Lq j (x (h ), u (h ))
j −1
p k +1
(9)
pbest k
is minimized .The continuous input law (5) is equivalent to:
u (h , q j ) ≃ −G q x (h ) + E q j , j=1,...,K j
(10)
3. PARTICLE SWARM OPTIMIZATION FOR SWITCHED LINEAR SYSTEMS OPTIMAL CONTROL 3.1 PSO description The Particle Swarm Optimization (PSO) is one of the evolutionary computation techniques. Its aim is to minimize a fitness function (the objective function in our case) by undertaking a population search in a D-dimensional search space [Abraham et al, 06], [Trelea, 03]. The swarm is composed of particles. Each particle is characterized by a position (a potential solution for the optimization problem) and a velocity (a dynamic operator). Each particle updates its position taking into account its own best realization in the past and the best realization of its neighbors. Let k be an iteration index in the optimization context. The new particle velocity and position are updated by:
pk
b 2 .r2 .( pgbest k − p k )
w k .v k
b1.r1.( pbest k − p k ) vk
Fig. 1. A simple path of a particle move in the search-space We show, respectively, in Fig.1 and Fig.2 an illustration of a particle move in the search space and the general flowchart of the PSO algorithm. By analyzing the equations (11) and (12) and the Fig.1, we can see that, during its move, each particle combines three tendencies: • to follow its own way • to go towards its best previous position • to go towards the best neighbour 3.2 PSO parameter-settings and convergence The PSO algorithm, described above, has a small number of parameters that need to be fixed. The swarm size (number of particles in a swarm), is often chosen according to the problem complexity [Poli et al, 07]. Values of 20 to 50 are widely used. We have chosen this number to be 30. The PSO version, used in our work, is known as the inertia-weight version. To ensure its convergence, the inertia weight in the
equation (10), is chosen to decrease from 0.9 to 0.4, during the optimization process according to the equation (12). In a previous work by [Trelea, 03], a particle in the swarm has been viewed as a dynamic system. Cognitive and social accelerations of 0.7 have been demonstrated to make the PSO algorithm converge. Define parameter settings of the algorithm Initialize randomly the particles by positions and velocities in the search space
4. A GENERIC PROCEDURE BASED ON PSO TO SYNTHESISE OPTIMAL SWITCHED CONTROLLERS In this section, we present the mapping between PSO and OCPDSLS via a generic procedure to synthesize an optimal control law. Step 1: The decision variables, to be optimized, are coded as the position of a particle in the swarm. Taking into account the switching sequence (8) and the input continuous law (10), the position of the ith particle is:
p i = (G j 1i ,..., G jni , E Evaluate the current fitness(i) for each particle(i) and gbest_fitness = mini(fitness(i)), I is the corresponding indice and pgbest=pI
Compare fitness(i) and best_fitnss(i), if fitness(i) is better, then set best_fitness(i)=fitness(i)and pbest=p Update velocity and position for each particle i according to equations (11), (12) and (13). Test eventual constraints and confine velocities and positions if necessary
No
Stop Condition met Yes End
Fig. 2. flowchart of the PSO 3.3 Applications of PSO on hybrid optimal control The Particle Swarm Optimization technique (PSO) becomes more and more used in the field of hybrid optimal control. The attractive features of the PSO are easy to implement and require neither regularity nor convexity of the cost function to be minimized. In [Balci, 04], a PSO, combined with the Lagrangian relaxation scheme method to schedule electric power generators, is proposed. It consists in determining the schedule and the production amounts evolving under many constraints. Although it is designed to search optimal continuous solutions, a binary PSO version has been used by many researchers. To control the evolution of the reactive power, [Khalil et al, 06] uses this version of PSO in an optimal placement and sizing of capacitor banks. A solution for fed-batch processes optimal control problem is presented in [Ismael, 06] while [Mendes, 06] presents a comparison between PSO and other metaheuristics, like genetic algorithms (GA), for an optimal control of some fermentation processes. In [Boubaker, 08] we have used the PSO-based algorithm to solve optimal switching instants of autonomous switched systems.
ji
,N
ji
, j = 1,..., K )
(14)
We assume that the gains of the optimal continuous input law are constant for each mode. Both continuous and discrete variables are coded together in the same vector and they are searched simultaneously. Step 2: For each particle of the swarm, we initialize, randomly, the position in a pre-defined search-space. The limits of this search space will take into account eventual constraints on the decision variables. Step 3: PSO algorithm is based on the objective function evaluation. So, we do not need to compute the objective function gradient with respect to decision variables. For the discrete-time problem (P2), (equivalent to (P1)), the objective function (9) is evaluated, for a particle i according to the following procedure: Procedure: objective function evaluation Given: x(0) u(0)=-Gq1x(0)+Eq1 J(0)=Lq1(x(0),u(0)) For j=1 to K %the given modal sequence For h=Nj-1 to Nj %the mode qj is active U(h) =-Gqjx(h-1)+Eqj x(h)= Aqjx(h)+Bqju(h) J(h)=J(h-1)+Lqj(x(h),u(h)) End End J=J(N-1)+Ψ(x(N))
Fig. 3. Procedure of objective function evaluation Step 4: The update equations (11) and (12) generate real values. To handle the discrete variables (the switching indices), which are integer variables; we have used a technique which rounds these values to their integer parts. Step 5: PSO algorithm offers a suitable tool to handle the constraints. In fact, if a particle violates the constraint and comes out of the search space, defined by the structural constraints, it is confined by reinitialization in the searchspace.
Step 6: Stop condition may be a prefixed maximum iterations number or a predefined value of the minimum desired for the objective function.
The PSO parameters are those fixed in section 3.3. As the PSO algorithm is stochastic, we have run it 100 hundred times. The obtained results compared to those obtained by the approach in [Xu, 02] are provided in table I below.
Step 7: As the PSO-based algorithm is stochastic, the results it provides are not reproducible. We hope at least that it converges “normally”. In other words, after many trials, the obtained results are distributed according to normal distributions centred on their averages.
TABLE I RESULTS OBTAINED BY OUR APPROACH AND CONVENTIONAL APPROACHES
5. AN ILLUSTRATIVE EXAMPLE
t *1
The approach proposed in this paper is tested on a benchmark system which has been studied in [Xu, 02].
t *2
Consider the following hybrid linear switched system: .
sub-system 1: x = A1x + B1u
(15-1)
-2 0 1 A1 = ; B1 = 0 0 1 − .
sub-system 2: x = A 2 x + B 2u 0.5 A2 = -5.3
5.3 1 ; B2 = 0.5 -1
(15-2)
.
sub-system 3: x = A3 x + B 3u 1 0 0 A3 = ; B3 = 0 1.5 1
(15-3)
The system switches at t1 ∈ [ 0,3] from sub-system1 to subsystem2 and at
Our approach
[Xu, 02]
0.98
1.0002
2
2.02
2.0008
We adopt the mean values for the switching instants which are ( t *1 = 0.98 , t *2 = 2.02 ). The standard deviations (mean square errors) are respectively 0.0003 for t1 and 0.0001 for t2. We show, respectively, in Fig. 4 and Fig.5, the continuous input and the corresponding state trajectory. Note that the values performed by the approach in [Xu, 02] are respectively 1.0002 and 2.0008. The cost function is nonconvex with respect to the switching instants. This difficulty is overcome by using the inertia-weight version of PSO which provides global solution. The difficulty of computing the derivatives of the objective function with respect to the switching instants is also overcome. In fact, the PSO technique does not require information about these derivatives. A comparative study using different particle numbers (see table II) shows that if we use a high number, the algorithm takes more time to converge without improving the obtained results compared to the theoretical values.
t 2 ∈ [ 0,3] , t1 < t 2 ,from subsystem2 to
subsystem3. We want to find the switching instants
( t1* , t 2*
and the optimal continuous control input, u [0,3] such that the following objective function 3
1 1 1 2 (x 1 (3) + 4.1437) 2 + (x 2 (3) − 9.3569)2 + u (t )dt 2 2 2
∫
TABLE II RESULTS FOR DIFFERENT PARTICLES NUMBER
)
*
J (u [0,3] , t1 , t 2 ) =
theoretical values 1
t *1 t *2
20 1.02
30 0.98
40 1.02
50 0.99
2.08
2.02
2.03
2.03
(16)
0
t
is minimized with x (0) = [ 4 4] . Note here that the desired
0.08
final values, x 1 (3) = −4.1437 and x 2 (3) = 9.3569 have been chosen deliberately to obtain the theoretical optimal switching instants as mentioned in table I.
0.06
As the PSO algorithm is especially based on the objective function evaluation, we adopt the transformation of the problem described in section 2 with a sampling period T = 0.01 . The optimal continuous input is described by:
0
u (h ) ≃ −G q 1x 1 (h ) − G q 2 x 2 (h ) + E q , q=1,...,3
(17)
The decision variable is then coded as the position of a particle:
pi = (G11i ,G12i , E1i , N 1i ,G21i ,G22i , E2i , N 2i ,G31i ,G32i , E3i )
(18)
0.04
u
0.02
-0.02 -0.04 -0.06 -0.08 0
0.5
1.00
1.50 t
2.000
2.50
3.00
Fig. 4. The optimal continuous input
10 8
6
x2
4
2
0
-2
-4 -5
-4
-3
-2
-1
0
1
2
3
4
x1
Fig. 5. The state trajectory 6. CONCLUSION In this paper, we formulate the optimal control problem of switched linear systems. In order to avoid some structural and numerical difficulties, encountered by conventional approaches, we have applied a metaheuristic optimization technique. Numerical results, on a non-convex problem, have demonstrated that although the obtained solutions are suboptimal, the PSO algorithm provides a powerful alternative technique without requiring any mathematical regularity. Future works will be devoted to a more detailed study on the convergence of this technique and its application to nonlinear hybrid systems. ACKNOWLEDGEMENT A part of the present research work has been supported by International Campus on Safety and Intermodality in Transportation the Nord-Pas-de-Calais Region, the European Community, the Regional Delegation for Research and Technology, the Ministry of Higher Education and Research, and the National Center for Scientific Research. The author, M. DJEMAI gratefully acknowledges the support of these institutions. REFERENCES Abraham, A., Guo, H., and Liu, H. (2006). Swarm Intelligence: foundations, perspectives and applications. Springer-Verlag. Balci H. H. and George, F. (2004). Scheduling electric power generators using particle swarm optimization combined with the lagrangian relaxation method. Int. J. Appl. Math. Compt. Sci. Vol. 14, No. 3, pp. 411-421. Benmansour, K., Benalia, A. Djemai, M. and Leon, J. De. (2007). Hybrid control of a multicellular converter. Nonlinear Analysis: Hybrid Systems, P.P. 16-29. Branicky, M.S., Borkar, V.S. and Mitter, S.K. (1998). A Unified Framework for Hybrid Control: Model and Optimal Control Theory. IEEE. Trans. Aut. Contr, Jan. Boubaker, S. and M’sahli, F. (2008). Solutions Based on Particle Swarm Optimization for Optimal Control Problems of Hybrid Autonomous Switched systems. International Journal of Intelligent Control and Systems (IJICS), Vol. 13, NO. 2, June, 128-135. Ismael, F. V. and E. C. Ferreira. (2006). Optimal Control of fedbatch Processes with Particle Swarm Optimization. Ed. Sicilia et al. Italia.
Juloski, A. Lj., Heemels, W. P. M. H. and Ferrari-Trecate, G. (2005). Identification of hybrid systems: a tutorial. Khalil, T. M., Youssef, H. K. M. and AbdelAziz, M. M. (2006). A Binary Particle Optimization for Optimal Placement and Sizing of Capacitor Banks in Radial Distribution Feeders with Distorted Substation Voltages. in Proc. Of AIML06. Luus, R., and Chen, Y. (2003). Optimal Switching Control Via Direct Search Optimization. in Proc. Inter. Sympos. Int. Contr, Oct,2003. Mendes, R., Rocha, I., Ferreira, E.C. and Rocha, M. (2006). A comparison of algorithms for the optimization of fermentation processes. Proc. IEEE Cong. Evolutionary computation. Poli, R., Kennedy, J., and Blackwell, T. (2007). Swarm. Intell. PP. 33-57. Riedinger, P., Daafouz, J. and Iung, C. (2005). About solving hybrid optimal control problems. IMACS’2005. France Saadaoui, H., Manammani, N., Djemai, M., Barbot, J. P and Floquet, T. (2006). Exact differentiation and sliding mode observers for switched lagrangian systems. Nonlinear Analysis, Vol. 65, P.P. 1050-1069. Trelea, I. C. (2003). PSO viewed as a dynamical system: convergence and parameter-settings. Seminar on PSO. Paris, France (in French). Xu, X. and Antsaklis, P.J. (2004). Optimal Control of Switched Systems Based on Parametrization of the Switching Instants,. IEEE Trans. Automat. Contr, Vol. 49, no. 1,Jan. Xu, X. and Antsaklis, P.J. (2002). Optimal control of switched systems via non-linear optimization based on direct differentiations of values functions. INT. J. CON, VOl. 75, NOS. 16/17, P.P. 1406-1426.
High Institute of Technological Studies of Nabeul, Campus Universitaire-Merazka, 8000, Nabeul,Tunisia E-mail: [email protected] 2 Univ Lille Nord de France, F-59000 Lille, France 3 UVHC, LAMIH, F-59313 Valenciennes, France 4 CNRS, UMR 8530, F-59313 Valenciennes, France E-mail: [email protected] 5 Univ Reims, CRESTIC, F-51000 Reims, France Moulin de la Housse BP 1039, 51687 REIMS cedex 2 - France E-mail: [email protected] 6 National School of Engineers of Monastir, Rue Ibn Al-Jazzar, 5019, Monastir, Tunisia E-mail: [email protected] 1,6 Unité de Recherche en Commande Numérique de Procédés Industriels (CONPRI) National School of Engineers of Gabès, Rue Omar Ibn Al-Khattab, 6029, Gabès, Tunisia
Abstract: This paper deals with Optimal Control problems of Dynamical Switched Linear Systems (OCPDSLS). This special class of hybrid systems combines the standard control, where dynamic subsystems are described by differential linear equations with a switching law which activates, alternatively, these subsystems. For a pre-specified modal sequence, we demonstrate that switching time instants and continuous input can be solved simultaneously by using an evolutionary computation technique, namely Particle Swarm Optimization (PSO). Some structural and numerical difficulties encountered in solving such problems by conventional approaches are discussed. The mapping between PSO and OCPDSLS is established via a generic procedure to synthesize optimal control laws. An illustrative numerical nonconvex example is solved showing the effectiveness of the proposed technique in overcoming difficulties of this problem known to be NP-hard, non-linear and constrained one. Keywords: Switched Linear Systems, Switching sequence, Optimal Control, Optimization, Conventional Approaches, Particle Swarm Optimization (PSO). 1. INTRODUCTION Switched Linear Systems as a particular class of hybrid systems have been under intensive investigations during the last few years. Due to their many potential applications, they have attracted researchers in the fields of modelization [Branicky et al, 98], identification [Juloski et al, 05], observation [Saadaoui et al, 06] and optimal control [Xu, 02 and 04], [Riedinger et al, 05], etc ... This special class of systems involves interaction between continuous and discrete dynamics and has applications in various fields such as mechanical systems, automotive industry, chemical engineering and switching power converters [Benmansour et al, 07]. Recently, Optimal Control Problem of Dynamical Switched Linear Systems (OCPDSLS) has become a challenging research topic because of the large classes of phenomena they describe and the integration of computers in industrial processes control. These problems do not need only the solution of optimal continuous inputs, as in a conventional optimal control problem, but also the solution of optimal modal sequence and its corresponding set of switching instants. The basic difficulty of this kind of problems is the complexity growing exponentially on the number of the handled data, when they are solved by conventional approaches [Xu, 02 and 04].
In [Xu, 02], OCPDSLS has been solved via non-linear optimization based on direct differentiation of the objective function. The main difficulty encountered by this approach is the non-availability of information about derivatives of cost function with respect to switching instants. The General Switched Linear Quadratic (GSLQ) problem discussed in [Xu, 04] is transformed into an equivalent parameter selection problem parameterized by switching instants. For the two-stage proposed algorithm, it is obvious that the optimization of the switching instants may be usually nonconvex and the Newton iteration algorithm will easily fall in local minima. In [Luus, 03], the GSLQ problem is considered via a two-level algorithm. The high level, based on a direct search optimization, is devoted to searching the optimal switching instants. The low level based on an Iterative Dynamic Programming (IDP) procedure is used to compute optimal continuous input. The main difficulty encountered by the proposed approach is the combinatorial explosion due to the time and state spaces discretization. In this paper, we aim to apply a new evolutionary computation technique, namely PSO, to solve the optimal control of linear switched systems problem. Given a prespecified active sub-systems, one needs to seek both the optimal switching locations, over a finite time horizon, and the corresponding continuous input. For a given switching law, the continuous input has been demonstrated to be a state feedback law. Through time and state spaces discretization, we convert the original problem to a parameter selection one.
The switching instants and the gains of the continuous input law are coded as positions of the particles in the PSO paradigm. These particles freely fly through a multidimensional search space when looking for the optimal solution. A suitable mapping between PSO and OCPDSLS allows handling the continuous and the discrete variables together. Without requiring any regularity of the studied problem, the PSO algorithm provides global sub-optimal solutions and allows overcoming the difficulties discussed above. The paper is organized as follows. In section 2, we propose a unified modelling framework of switched linear continuoustime systems and the general optimal control problem for this class of systems is formulated. To be well-handled by the proposed optimization technique, we propose, an equivalent discrete-time formulation via a discretization procedure of the sub-systems, the switching law and the cost function to be minimized. Section 3 is devoted to the presentation of the PSO technique. In section 4, a generic procedure to compute sub-optimal solutions for the optimal control problem is addressed. We focus on the adaptations of the PSO-based algorithm used in the aim of overcoming the difficulties discussed in section 1. Section 5 is devoted to a literature benchmark, used in optimal control illustrations, to demonstrate the effectiveness of the proposed procedure. Finally, we draw some conclusions in section 6. 2. MODELLING AND OPTIMAL CONTROL OF A SWITCHED LINEAR SYSTEM 3.1 A continuous-time formulation In this paper, we consider linear switched systems consisting of subsystems, in the state-space continuous-time formalism:
. x (t ) = Aq x (t ) + B q u (t )
q ∈ Q = {1,..., M }
Note that the system switches from subsystem q j
subsystem q ( j +1) at the switching instant t q j .During the interval t q j −1 , t q j the configuration q j is active. A collection of states, x , continuous controls, u , and a sequence σ in t 0 , t f is known as an execution of the switched system. Assumption 2: We consider only non-Zeno executions. Assumption 3: The switched system is supposed to be without jumps. Problem P1: We consider a switched system as described by (1). Given an interval of time t 0 , t f , an initial state, x0 and a sequence of active subsystems {q1 ,..., q K } , an algorithm to solve an optimal control problem of this system has to find the optimal control continuous input, u* and the optimal switching instants (t *1 ,..., t *K ) such that the following costfunction
J (σ , u ) = ψ (x (t f )) +
j =K
∑∫ j =1
tj
t j −1
Lq j (x ,u )dt
If the cost function is quadratic, using a method similar to conventional quadratic optimal control problem, the solution of the Hamilton-Jacobi-Bellmann equations [Xu, 04], provides the continuous input as: j
Aq and Bq are matrices with suitable dimensions and Q indicates that the system will be in M configurations.
x (t ) ∈ ℝ n and u (t ) ∈ ℝ m are respectively the state vector and the continuous control input. For such a system, given the modal sequence, {q j ∈ Q , j = 1,..., K } where K is finite,
(4)
is minimized. ψ represents the terminal part of this cost function.
u (x , t , q j ) = −G q (t )x (t ) + E q j (t ), j=1,...,K (1)
to
(5)
The continuous input is then obtained through its statefeedback control law (5). The problem of computing the optimal continuous input is converted to a parameter selection one involving the gains of this law. Assumption 4: we assume that the gains vectors G q (t ) and E q j (t ), j=1,...,K , are constant for each active j
sub-system q j .
we define the sequence of active subsystems in a time interval t 0 , t f by σ , which, from a control point of view,
This assumption 4 is made in order to reduce the numerical complexity of the problem.
is viewed as a discrete input :
3.1 An equivalent discrete-time formulation
σ = {(q1 , t1 ),..., (q K , t K )}
(2)
Assumption1: The global system remains in a configuration q j during, at least, a time τ min , known as the dwell-time. The switching instants are constrained by:
t 0 < t1 < ... < t K < t f = t K +1
(3)
The interval of time t 0 , t f is divided into N sampling instants such that:
N =
tf −t0 T
(6)
T is the sampling period. Assumption 5: According to Shannon sampling theorem, the sampling period T is chosen such that its value is less than
the least value among all the time constants of all the subsystems {q j ∈ Q , j = 1,..., K } .
vk +1 = wk .vk + b1 .r1 .( pbest k − p k )
The sampled times are t h = hT , h = 0,..., N . We assume that the control and the state are piecewise constant in the interval . x (h + 1) − x (h ) , the th , th +1 . Through the relation x (t ) ≃ T continuous dynamics in equation (1) are replaced by their first order discrete-time equivalents
p k +1 = p k + v k +1
x (h + 1) = (T × Aq
+ I )x
(h ) + T × B q u (h )
q ∈ Q = {1,..., M }
(7)
I is the identity matrice having the same dimensions as Aq . Assumption 6: We assume that the switching instants coincide with entire numbers of sampling period. That is to say, for each couple of two consecutive configurations, it exists an integer N j verifying: t j ≃ N j ×T . The sequence of active subsystems is then defined as
σ ≃ {(q1 , N 1 ),..., (q K , N K )}
(8)
Problem P2: In the discrete-time equivalent formulation, an optimal control algorithm has to find the optimal continuous inputs, u* (h) , h ∈ {0,1,..., N − 1} , and the optimal switching
(11)
+ b2 .r2 .( pgbest k − pk )
w − wmin wk = wmax − max kmax where, • • • • • • •
(12) (13)
.k
p k : the position of each particle at iteration k , a candidate solution. v k : the velocity of each particle at iteration k . b1 and b2 : the cognitive and the social acceleration constants r1 and r2 : random numbers with values in the range [ 0,1] .
pbest k : the best position reached by each particle in the past, known as the “ personal_best ” pgbest k : the best position between neighbors of each particle , known as the “ global_best ” kmax : a maximum iterations number pgbest k
indices, (N *1 ,..., N *K ) , such that the following cost function
J (σ , u ) ≃ ψ (x (N )) +T ×
K
h =N
j
∑∑ j =1 h = N
Lq j (x (h ), u (h ))
j −1
p k +1
(9)
pbest k
is minimized .The continuous input law (5) is equivalent to:
u (h , q j ) ≃ −G q x (h ) + E q j , j=1,...,K j
(10)
3. PARTICLE SWARM OPTIMIZATION FOR SWITCHED LINEAR SYSTEMS OPTIMAL CONTROL 3.1 PSO description The Particle Swarm Optimization (PSO) is one of the evolutionary computation techniques. Its aim is to minimize a fitness function (the objective function in our case) by undertaking a population search in a D-dimensional search space [Abraham et al, 06], [Trelea, 03]. The swarm is composed of particles. Each particle is characterized by a position (a potential solution for the optimization problem) and a velocity (a dynamic operator). Each particle updates its position taking into account its own best realization in the past and the best realization of its neighbors. Let k be an iteration index in the optimization context. The new particle velocity and position are updated by:
pk
b 2 .r2 .( pgbest k − p k )
w k .v k
b1.r1.( pbest k − p k ) vk
Fig. 1. A simple path of a particle move in the search-space We show, respectively, in Fig.1 and Fig.2 an illustration of a particle move in the search space and the general flowchart of the PSO algorithm. By analyzing the equations (11) and (12) and the Fig.1, we can see that, during its move, each particle combines three tendencies: • to follow its own way • to go towards its best previous position • to go towards the best neighbour 3.2 PSO parameter-settings and convergence The PSO algorithm, described above, has a small number of parameters that need to be fixed. The swarm size (number of particles in a swarm), is often chosen according to the problem complexity [Poli et al, 07]. Values of 20 to 50 are widely used. We have chosen this number to be 30. The PSO version, used in our work, is known as the inertia-weight version. To ensure its convergence, the inertia weight in the
equation (10), is chosen to decrease from 0.9 to 0.4, during the optimization process according to the equation (12). In a previous work by [Trelea, 03], a particle in the swarm has been viewed as a dynamic system. Cognitive and social accelerations of 0.7 have been demonstrated to make the PSO algorithm converge. Define parameter settings of the algorithm Initialize randomly the particles by positions and velocities in the search space
4. A GENERIC PROCEDURE BASED ON PSO TO SYNTHESISE OPTIMAL SWITCHED CONTROLLERS In this section, we present the mapping between PSO and OCPDSLS via a generic procedure to synthesize an optimal control law. Step 1: The decision variables, to be optimized, are coded as the position of a particle in the swarm. Taking into account the switching sequence (8) and the input continuous law (10), the position of the ith particle is:
p i = (G j 1i ,..., G jni , E Evaluate the current fitness(i) for each particle(i) and gbest_fitness = mini(fitness(i)), I is the corresponding indice and pgbest=pI
Compare fitness(i) and best_fitnss(i), if fitness(i) is better, then set best_fitness(i)=fitness(i)and pbest=p Update velocity and position for each particle i according to equations (11), (12) and (13). Test eventual constraints and confine velocities and positions if necessary
No
Stop Condition met Yes End
Fig. 2. flowchart of the PSO 3.3 Applications of PSO on hybrid optimal control The Particle Swarm Optimization technique (PSO) becomes more and more used in the field of hybrid optimal control. The attractive features of the PSO are easy to implement and require neither regularity nor convexity of the cost function to be minimized. In [Balci, 04], a PSO, combined with the Lagrangian relaxation scheme method to schedule electric power generators, is proposed. It consists in determining the schedule and the production amounts evolving under many constraints. Although it is designed to search optimal continuous solutions, a binary PSO version has been used by many researchers. To control the evolution of the reactive power, [Khalil et al, 06] uses this version of PSO in an optimal placement and sizing of capacitor banks. A solution for fed-batch processes optimal control problem is presented in [Ismael, 06] while [Mendes, 06] presents a comparison between PSO and other metaheuristics, like genetic algorithms (GA), for an optimal control of some fermentation processes. In [Boubaker, 08] we have used the PSO-based algorithm to solve optimal switching instants of autonomous switched systems.
ji
,N
ji
, j = 1,..., K )
(14)
We assume that the gains of the optimal continuous input law are constant for each mode. Both continuous and discrete variables are coded together in the same vector and they are searched simultaneously. Step 2: For each particle of the swarm, we initialize, randomly, the position in a pre-defined search-space. The limits of this search space will take into account eventual constraints on the decision variables. Step 3: PSO algorithm is based on the objective function evaluation. So, we do not need to compute the objective function gradient with respect to decision variables. For the discrete-time problem (P2), (equivalent to (P1)), the objective function (9) is evaluated, for a particle i according to the following procedure: Procedure: objective function evaluation Given: x(0) u(0)=-Gq1x(0)+Eq1 J(0)=Lq1(x(0),u(0)) For j=1 to K %the given modal sequence For h=Nj-1 to Nj %the mode qj is active U(h) =-Gqjx(h-1)+Eqj x(h)= Aqjx(h)+Bqju(h) J(h)=J(h-1)+Lqj(x(h),u(h)) End End J=J(N-1)+Ψ(x(N))
Fig. 3. Procedure of objective function evaluation Step 4: The update equations (11) and (12) generate real values. To handle the discrete variables (the switching indices), which are integer variables; we have used a technique which rounds these values to their integer parts. Step 5: PSO algorithm offers a suitable tool to handle the constraints. In fact, if a particle violates the constraint and comes out of the search space, defined by the structural constraints, it is confined by reinitialization in the searchspace.
Step 6: Stop condition may be a prefixed maximum iterations number or a predefined value of the minimum desired for the objective function.
The PSO parameters are those fixed in section 3.3. As the PSO algorithm is stochastic, we have run it 100 hundred times. The obtained results compared to those obtained by the approach in [Xu, 02] are provided in table I below.
Step 7: As the PSO-based algorithm is stochastic, the results it provides are not reproducible. We hope at least that it converges “normally”. In other words, after many trials, the obtained results are distributed according to normal distributions centred on their averages.
TABLE I RESULTS OBTAINED BY OUR APPROACH AND CONVENTIONAL APPROACHES
5. AN ILLUSTRATIVE EXAMPLE
t *1
The approach proposed in this paper is tested on a benchmark system which has been studied in [Xu, 02].
t *2
Consider the following hybrid linear switched system: .
sub-system 1: x = A1x + B1u
(15-1)
-2 0 1 A1 = ; B1 = 0 0 1 − .
sub-system 2: x = A 2 x + B 2u 0.5 A2 = -5.3
5.3 1 ; B2 = 0.5 -1
(15-2)
.
sub-system 3: x = A3 x + B 3u 1 0 0 A3 = ; B3 = 0 1.5 1
(15-3)
The system switches at t1 ∈ [ 0,3] from sub-system1 to subsystem2 and at
Our approach
[Xu, 02]
0.98
1.0002
2
2.02
2.0008
We adopt the mean values for the switching instants which are ( t *1 = 0.98 , t *2 = 2.02 ). The standard deviations (mean square errors) are respectively 0.0003 for t1 and 0.0001 for t2. We show, respectively, in Fig. 4 and Fig.5, the continuous input and the corresponding state trajectory. Note that the values performed by the approach in [Xu, 02] are respectively 1.0002 and 2.0008. The cost function is nonconvex with respect to the switching instants. This difficulty is overcome by using the inertia-weight version of PSO which provides global solution. The difficulty of computing the derivatives of the objective function with respect to the switching instants is also overcome. In fact, the PSO technique does not require information about these derivatives. A comparative study using different particle numbers (see table II) shows that if we use a high number, the algorithm takes more time to converge without improving the obtained results compared to the theoretical values.
t 2 ∈ [ 0,3] , t1 < t 2 ,from subsystem2 to
subsystem3. We want to find the switching instants
( t1* , t 2*
and the optimal continuous control input, u [0,3] such that the following objective function 3
1 1 1 2 (x 1 (3) + 4.1437) 2 + (x 2 (3) − 9.3569)2 + u (t )dt 2 2 2
∫
TABLE II RESULTS FOR DIFFERENT PARTICLES NUMBER
)
*
J (u [0,3] , t1 , t 2 ) =
theoretical values 1
t *1 t *2
20 1.02
30 0.98
40 1.02
50 0.99
2.08
2.02
2.03
2.03
(16)
0
t
is minimized with x (0) = [ 4 4] . Note here that the desired
0.08
final values, x 1 (3) = −4.1437 and x 2 (3) = 9.3569 have been chosen deliberately to obtain the theoretical optimal switching instants as mentioned in table I.
0.06
As the PSO algorithm is especially based on the objective function evaluation, we adopt the transformation of the problem described in section 2 with a sampling period T = 0.01 . The optimal continuous input is described by:
0
u (h ) ≃ −G q 1x 1 (h ) − G q 2 x 2 (h ) + E q , q=1,...,3
(17)
The decision variable is then coded as the position of a particle:
pi = (G11i ,G12i , E1i , N 1i ,G21i ,G22i , E2i , N 2i ,G31i ,G32i , E3i )
(18)
0.04
u
0.02
-0.02 -0.04 -0.06 -0.08 0
0.5
1.00
1.50 t
2.000
2.50
3.00
Fig. 4. The optimal continuous input
10 8
6
x2
4
2
0
-2
-4 -5
-4
-3
-2
-1
0
1
2
3
4
x1
Fig. 5. The state trajectory 6. CONCLUSION In this paper, we formulate the optimal control problem of switched linear systems. In order to avoid some structural and numerical difficulties, encountered by conventional approaches, we have applied a metaheuristic optimization technique. Numerical results, on a non-convex problem, have demonstrated that although the obtained solutions are suboptimal, the PSO algorithm provides a powerful alternative technique without requiring any mathematical regularity. Future works will be devoted to a more detailed study on the convergence of this technique and its application to nonlinear hybrid systems. ACKNOWLEDGEMENT A part of the present research work has been supported by International Campus on Safety and Intermodality in Transportation the Nord-Pas-de-Calais Region, the European Community, the Regional Delegation for Research and Technology, the Ministry of Higher Education and Research, and the National Center for Scientific Research. The author, M. DJEMAI gratefully acknowledges the support of these institutions. REFERENCES Abraham, A., Guo, H., and Liu, H. (2006). Swarm Intelligence: foundations, perspectives and applications. Springer-Verlag. Balci H. H. and George, F. (2004). Scheduling electric power generators using particle swarm optimization combined with the lagrangian relaxation method. Int. J. Appl. Math. Compt. Sci. Vol. 14, No. 3, pp. 411-421. Benmansour, K., Benalia, A. Djemai, M. and Leon, J. De. (2007). Hybrid control of a multicellular converter. Nonlinear Analysis: Hybrid Systems, P.P. 16-29. Branicky, M.S., Borkar, V.S. and Mitter, S.K. (1998). A Unified Framework for Hybrid Control: Model and Optimal Control Theory. IEEE. Trans. Aut. Contr, Jan. Boubaker, S. and M’sahli, F. (2008). Solutions Based on Particle Swarm Optimization for Optimal Control Problems of Hybrid Autonomous Switched systems. International Journal of Intelligent Control and Systems (IJICS), Vol. 13, NO. 2, June, 128-135. Ismael, F. V. and E. C. Ferreira. (2006). Optimal Control of fedbatch Processes with Particle Swarm Optimization. Ed. Sicilia et al. Italia.
Juloski, A. Lj., Heemels, W. P. M. H. and Ferrari-Trecate, G. (2005). Identification of hybrid systems: a tutorial. Khalil, T. M., Youssef, H. K. M. and AbdelAziz, M. M. (2006). A Binary Particle Optimization for Optimal Placement and Sizing of Capacitor Banks in Radial Distribution Feeders with Distorted Substation Voltages. in Proc. Of AIML06. Luus, R., and Chen, Y. (2003). Optimal Switching Control Via Direct Search Optimization. in Proc. Inter. Sympos. Int. Contr, Oct,2003. Mendes, R., Rocha, I., Ferreira, E.C. and Rocha, M. (2006). A comparison of algorithms for the optimization of fermentation processes. Proc. IEEE Cong. Evolutionary computation. Poli, R., Kennedy, J., and Blackwell, T. (2007). Swarm. Intell. PP. 33-57. Riedinger, P., Daafouz, J. and Iung, C. (2005). About solving hybrid optimal control problems. IMACS’2005. France Saadaoui, H., Manammani, N., Djemai, M., Barbot, J. P and Floquet, T. (2006). Exact differentiation and sliding mode observers for switched lagrangian systems. Nonlinear Analysis, Vol. 65, P.P. 1050-1069. Trelea, I. C. (2003). PSO viewed as a dynamical system: convergence and parameter-settings. Seminar on PSO. Paris, France (in French). Xu, X. and Antsaklis, P.J. (2004). Optimal Control of Switched Systems Based on Parametrization of the Switching Instants,. IEEE Trans. Automat. Contr, Vol. 49, no. 1,Jan. Xu, X. and Antsaklis, P.J. (2002). Optimal control of switched systems via non-linear optimization based on direct differentiations of values functions. INT. J. CON, VOl. 75, NOS. 16/17, P.P. 1406-1426.