Delay Independent Evolutionary Dynamics for Resource Allocation with Asynchronous Distributed Sensors1

Proceedings of the 3rd IFAC Workshop on Distributed Estimation and Control in Networked Systems September 14-15, 2012. Santa Barbara, CA, USA

Delay Independent Evolutionary Dynamics for Resource Allocation with Asynchronous Distributed Sensors 1 Jairo Giraldo ∗ Nicanor Quijano ∗ ∗

Universidad de los Andes Colombia, (e-mail: [email protected], [email protected]). Abstract: Many control problems can be considered as resource allocation problems, where the main objective is to allocate an amount of resources (e.g., voltage, power) among some distributed elements. However, the inclusion of communication networks adds new challenges to these kind of problems and new control strategies must be developed. This work presents an evolutionary strategy that can be used in dynamical resource allocation control problems, where sensors and actuators signals are transmitted through a communication network. We show that this strategy is independent of the delays using some passivity based analysis and an application in smart grids generation dispatch is illustrated. However, the performance of the controlled system may be degraded with the inclusion of the time delays. Hence, an observer for nonlinear delayed systems is implemented in order to improve the performance of the RD technique. Keywords: Evolutionary Game Theory, Networked Control Systems, Smart Grids. 1. INTRODUCTION The studies about the inclusion of communication networks in control have been of great interest in the last few years due to the possibility of communicating distributed elements through a communication network. This kind of connectivity may enhance scalability, diminish wiring, and improve distributed control capabilities. However, different issues are induced because of the insertion of a communication network such as delays, packet dropouts, jitter effects, and the need of data synchronization just to name a few, making the analysis and design of networked control systems (NCS) complex (see for instance, the special issue on NCS in Antsaklis and Baillieul (2007)). A very important research area in NCS lies in the design of control strategies for systems where sensors or plants are spatially distributed, and control actions are executed taking into account the measurements of all or a group of sensors. In general, its importance resides in the development of industrial or military applications, such as tracking (Liu et al. (2007)), monitoring (Sinopoli et al. (2003)), or consensus (Olfati-Saber et al. (2007)). In this respect, many of the aforementioned types of control problems can be considered as resource allocation problems (Schmidt et al. (2009)), where the main objective is to allocate an amount of resources (e.g., voltage, power) among some distributed elements of a system, in order to achieve desired references. For control applications, the use of evolutionary strategies to solve resource allocation problems is appropriated because there is no need of having a model of the plant, 1

This work has been supported in part by Project CIFI 2011, Facultad de Ingenier´ıa, Universidad de los Andes.

978-3-902823-22-9 /12/$20.00 © 2012 IFAC

121

and robust control strategies can be developed (e.g.,power resource allocation for wireless networks (Han and Liu (2008)), control of UAV (Cruz Jr et al. (2004)). In particular, an evolutionary technique that is easy to implement and that has a low computational cost is the replicator dynamics (RD) approach (Taylor and Jonker (1978)). This strategy models how natural selection affects the amount of individuals in different habitats of an environment according to a fitness value. The amount of individuals in each habitat changes as a result of the interaction of the total population. At the end, the population evolves until it maximizes the social welfare. This strategy has been used in order to solve a variety of resource allocation problems where sensors and controllers are distributed, in such a way stability can be assured (Kunigami and Terano (2003); Ram´ırez-Llanos and Quijano (2010); Niyato and Hossain (2009)). Nevertheless, in most of these applications it is usually assumed that the sensors used are synchronous (i.e., they perform the same operation at the same time), take measurements at the same time, and have no communication delays between sensor platforms and the controller. These assumptions are not entirely true when data measurements from sensors are transmitted using a communication network, because transmission times are not synchronized and data might be delayed. Tembine et. al. in (Tembine et al. (2011)) introduced the RD considering time delays, and they posed some important criteria for stability depending on the delays. On the other hand, in (Alboszta and Miekisz (2004)), stability is proved for a model of the discrete RD, which is interdependent of delays when they are synchronous. These replicator dynamics approaches consider that delays are present when the fitness function is acquired (calculated) and when the average fitness is calculated. However, we propose an RD

10.3182/20120914-2-US-4030.00019

IFAC NecSys'12 September 14-15, 2012. Santa Barbara, CA, USA

o

(2)

N P xi fi (xi ) f¯(x) =

)

f¯(x)

)

x2 f2 (x2 )

1)

i=1

¯f (x )

N

122

i=1

(x fN xN

The replicator dynamics (RD) is an evolutionary strategy that models the behavior of a population of individuals that are programmed to play a game, and it can be interpreted as the interaction of a group of animals that are seeking the best habitats to reproduce or feed (Taylor and Jonker (1978)). The objective of each individual is to choose the strategy (the habitat) that increases its utility. This utility is represented by a fitness function that varies in time according to the amount of individuals playing that strategy, which describes the quality of each habitat. One of the main properties of this evolutionary game is that, when the evolutionary process is taken into account, the population will tend to an equilibrium point where all individuals achieve the same fitness. The dynamics of the amount of individuals playing a certain strategy i can be described as
x˙ i (t) = xi (t) fi (xi (t)) − f¯(x(t)) (1) for i ∈ H, where H = {1, 2, . . . , N } being the set of pure strategies, x(t) = [x1 (t), . . . , xN (t)]⊤ , and xi (t) ≥ 0 is the ith state of the replication equation for all t. The fitness function fi (xi ) represents the utility that each individual obtains in the ith habitat, and the average fitness described PN by f¯(x) = j=1 xj fj (xj ) An important restriction of the RD strategy is that

xi = 1

Interestingly, the RD can be seen as a distributed multiagent controlled system, where each agent dynamics are described by Equation (1), and the controller that receives the state information of all agents is given by the average fitness. Figure 1 describes this fact.

(x

2. REPLICATOR DYNAMICS FOR RESOURCE ALLOCATION

XN

which means that the trajectories describing the dynamics of the portion of individuals following a strategy lies in the simplex set ∆ for all t. In order to ease notation, Equation (1) can be rewritten as x˙ = x(f − ¯ f ), where x ∈ RN , f , ¯ f : RN → RN , and ¯ f is a vector with the average fitness values for each pure strategy i ∈ H. As the average fitness is the same for all the strategies set,  ⊤ then ¯ f = f¯(x), f¯(x), . . . , f¯(x) . The replicator technique ⊤ will tend to an equilibrium point x∗ = [x∗1 , x∗2 , . . . , x∗N ] , when the same fitness value is achieved. This condition is true if the initial population state x(0) ∈ int(∆), and fi (x∗i ) = f¯(x∗ ) = f ∗ , for all i = 1, 2, . . . , N . This stability property is fundamental when a resource allocation problem is described using the RD approach, due to the capability of achieving a common fitness value for all agents. Therefore, the RD can be associated to a resource allocation problem, where a total amount of resources must be allocated between some agents. An analogy that we can draw is the following: agents are similar to the patches in the environment; the amount of resources is given by the total population; and the flow of resources might be associated with the population behavior at each habitat. The usefulness of this approach in control applications lies in the fact that the fitness function contains the information of the problem or plant that is being controlled (e.g., sensor measurement, position of an UAV). As a consequence, the RD can be treated as an interconnection of two dynamical systems, where the dynamics of the portion of individuals at each habitat (x) ˙ are directly related with the measured states of the controlled plant. However, for our analysis, we consider that the fitness function implicitly describes the behavior of the system (i.e., the plant), and we only focus on the dynamics of the RD technique.

f1

This paper is organized as follows. First, we start with a description of the replicator dynamics technique for solving resource allocation problems, where distributed sensors are included. In Section 3 an analysis for time delays independence is presented, and asymptotic stability of the RD is proved for asynchronous time delays. Section 4 presents an application in smart grids generation dispatch, where distributed generators are distributed and network communication effects are taken into account. Additionally, and state observer for nonlinear delayed systems is implemented in order to improve the performance of the dynamics of the RD approach. Finally, some conclusions and future directions are drawn in Section 5.

x ∈ RN + :

f¯( x

In other words, in this work we present a networked control systems scheme based on the RD approach, where sensors are distributed and communication with the central controller is asynchronous. Additionally, we want to show that, using the proposed scheme, asymptotic stability holds independently of time delays, assumption that can be very useful for the design of resource allocation solutions for systems whose sensor measurements cannot be delayed or synchronized. Therefore, in order to illustrate the relevance of this NCS approach, an application in smart grid distributed dispatch is introduced based on the results in (Pantoja and Quijano (2011)), taking into account the delay effects induced by the smart grid communication network. Furthermore, we also show that, even if the asymptotic stability is ensured, performance may be compromised and the implementation of an observer for delayed systems based on the results in (Cacace et al. (2010)) might be useful in order to improve the system performance.

n

∆=

x1

scheme where delays are asynchronous and they are only present in the calculation of the average fitness.

x˙ 1

x˙ 2

...

x˙ N

Fig. 1. Replicator dynamics scheme. In this context, dynamic resource allocation problems are now analyzed from a different point of view, where distributed sensors or multiple-agents are taken into account (Pantoja and Quijano (2011)). However, the communication between each agent and controller has been assumed to be ideal, which is not a real approach due to the limitations that are induced when a communication network is included in a real application. Consequently, if a communication network is considered between each agent and the controller in a bidirectional way, the replicator dynamic strategy turns into a net-

IFAC NecSys'12 September 14-15, 2012. Santa Barbara, CA, USA

worked control system (NCS). One of the problems that a NCS addresses is the fact that we have to deal with delays that may affect the performance of the system. Next, we will show how the proposed control technique does not depend on delays. 3. TIME DELAYS INDEPENDENCE OF RD 3.1 Mathematical Preliminaries In order to demonstrate the asymptotical stability of the equilibrium point and the delay independence of the RD method, some passivity concepts for feedback interconnected systems can be used. Let us consider the nonlinear system of the form n H :=

x˙ = L(x, u) y = h(x, u)

(3)

where x ∈ RN , u ∈ Rm , and y ∈ Rp . It is assumed that L is locally Lipschitz, L(0, 0) = 0 and h(0, 0) = 0. The reader is referred to (Khalil (2002)) in order to get the definitions of dissipative systems and zero-state observability for nonlinear systems. When there are two subsystems forming a feedback loop interconnection similar to the one shown in Figure 2, and the connection of the subsystems is made considering delay blocks T1 and T2 , the next definition adapted from (Chopra and Spong (2007)) can be introduced. Definition 3.1. Let us assume that we have an interconnected system similar to what it can be seen in Figure 2. The origin of the delayed closed-loop system formed by H1 and H2 is asymptotically stable independently of the delays if, for the nondelayed case, there exist storage functions V1 (x1 ),V2 (x2 ) positive definite, such that H1 , H2 are dissipative with respect to the supply rate  1 2 2 2 γn kun k − kyn k , for n = 1, 2. (4) sn = 2 with H1 ,H2 zero-state detectable, and the gain γn = 1. r1

u1

+

-

e˙ x = (ex + x∗ ) (f − (u1 + f ∗ )) y1 = (ex + x∗ ) f − x∗ f ∗

H1 :=

H2 :=

(

x˙ ¯f = 0 y2 = − 1

1×N

(5)

PN

i=1

u2i

Let us define e˙ x = L1 (ex , u1 ,), y1 = h1 (ex , u1 ) for system H1 , and x˙ ¯f = L2 (x¯f, u2 ), y2 = h2 (x¯f, u2 ), where x¯f corresponds to the states of system H2 . According to the description of Equation (3), L1 (0, 0) = h1 (0, 0) = 0, L2 (0, 0) = 0, and h2 (0, 0) = 0. This can be checked using the description of the RD in Section 2 and the definition of the error variables. The error ex is zero only when all the equilibrium points have been achieved, which implies that f = f ∗ . This also implies that the zero-state observability condition for h1 (ex , 0) is satisfied. For H2 , there is no dynamics involved (i.e., x˙ ¯f = 0), so L2 is said to be zero for all t. Besides, h2 (0, 0) = 0 when the input u2 = 0 and, as the states of this system are always the same, the zero-state observability is also accomplished. Having described the replicator dynamics approach in terms of a feedback interconnection as the one in Figure 2, now we need to define the storage functions V1 (ex ) and V2 (x¯f) for each subsystem H1 and H2 . Then, we can prove that the system reaches an asymptotically stable equilibrium point using Definition 3.1. Theorem 1. Let us consider the replicator dynamics approach described by Equation (5). We assume that the subsystems H1 and H2 are dissipative with the storage functions V1 (ex ) and V2 (x¯f), respectively. If, for a feedback interconnection of two dissipative systems, the total storage function described by
(6) VT (ex , x¯f) = V1 (ex ) + V2 (x¯f) = − min exj j=1,2,...,N

is positive definite and the interconnected system is dissipative with respect to the total supply rate sT =   2 2 2 2 1 2 2 γ ku k − ky k + γ ku k − ky k , then the origin 1 1 2 2 1 2 2 of the error state ex is asymptotically stable independently of the delays.

y1

H1 T1 (t)

T2 (t)

y1

that contains the equilibrium points of the systems. The systems can be written as n

H2

u2

+ +

r2

Fig. 2. Feedback interconnection of two dissipative delayed systems.

Proof According to Definition 3.1, each subsystem has to be dissipative with  respect to the supply rate sn = 2 2 1 2 for n = 1, 2. Hence, we can use the 2 γn kun k − kyn k property of storage function for interconnected systems (Khalil (2002)), such that VT (ex , x¯f) = V1 (ex ) + V2 (x¯f). As the derivative of each storage function must satisfied the condition V˙ 1 (ex ) ≤ s1 and V˙ 2 (x¯f) ≤ s2 , then
1 2 2 2 2 2 2 V˙ T (ex , x¯f) 6

3.2 Delayed Replicator Dynamics Analysis For our case of study, the replicator dynamics with the inclusion of delays is considered as the interconnection of two systems (Figure 2), each one described by Equation (3). We assume that each fitness function fi : ∆ 7→ R is a Lipschitz continuous mapping in ∆ and it is strictly decreasing. Then, the system H1 describes the dynamics of the replicator equation, with an input u1 that corresponds to the average fitness. System H2 can be considered as the controller, which receives the information of all the states and calculates the average fitness ¯ f . Indeed, to show that the origin is stable, the system is rewritten in terms of the error coordinates ex = x − x∗ , where x∗ is the vector 123

2

γ1 ku1 k − ky1 k + γ2 ku2 k − ky2 k

(7)

The inputs of the closed-loop system (Figure 2) r1 and r2 are 0. Hence, according to Definition 3.1, the analysis is developed for the nondelayed interconnected case. Then, the outputs of each subsystem are considered as y1 = u2 and y2 = −u1 . Therefore, the inequality in Equation (7) is given by


1 2 2 2 2 V˙ T (ex , x¯f) 6

2

γ1 − 1 ku1 k − γ2 − 1 ku2 k

(8)

where asymptotic stability holds if γ1 and γ2 are 1. Hence, the supply rate is 0, and Equation (8) becomes V˙ T (ex , x¯f) 6 0. (9) In order to ease notation, and taking into account that x¯f = 0 for all t, the total storage function will be considered as VT (ex ).

IFAC NecSys'12 September 14-15, 2012. Santa Barbara, CA, USA

Now, in order to prove that the function in Equation (6) is positive definite, the restriction in Equation (2) can be PN PN PN ∗ described as i=1 xi = i=1 (exi + xi ) = i=1 exi + PN ∗ are inside the i=1 xi = 1. As the equilibrium points PN simplex, its sum is always 1, and then i=1 exi = 0. As a consequence, there will always be at least a negative error, except when the equilibrium points have been achieved and all errors are zero. The function in Equation (6) selects the most negative error, then the storage function is positive, or zero only if VT (0) = 0. As the feedback interconnected system in Equation (5) is locally Lipschitz, the total storage function in Equation (6) is a locally Lipschitz continuous function and it is differentiable almost everywhere. However, there are points where the derivatives do not exist. Then, in order to obtain V˙ T (ex ), let us consider Hf = {k : exk = VT (ex )} the set of indices of the points where VT (ex ) is not differentiable. Therefore, applying the generalized gradient for nonsmoth functions in (Clarke (1983)), V˙ T (ex) is given by  X λk ∇exj e˙ x (10) V˙ T (ex ) = − P k∈Hf for all λk > 0 such that k∈Hf λk = 1. As the gradient ∇exk is 1, and using the description of the RD in Equation (5), Equation (10)X can be described as
V˙ T (ex ) = − λk (exk + x∗k ) fk − f¯ (11) k∈Hf

In this equation, the term λk (exk + x∗k ) = λk xk ≥ 0, then fk − f¯ has to be negative or zero in order to prove the inequality in Equation (9). This can be deduced using the characteristics of the RD and the assumption that all fitness functions are decreasing and fi (0) > 0 for all i ∈ H. A decreasing fitness fi (xi ) is more profitable when the state xi (the portion of individuals playing the strategy i) is low. Thus, the fitness corresponding to the most negative error exk is related to a small xk (i.e., an state far below its equilibrium), and fk is always greater than the average fitness f¯, for all k ∈ Hf . Therefore, the condition V˙ T 6 0 is true for this storage function for all ex .  It has been shown that the replicator dynamics approach under a certain communication scheme, satisfies the conditions of Definition 3.1, and the equilibrium points are asymptotically stable for all x(0) ∈ ∆, even when asynchronous sensor measurements are considered. Next, an application that illustrates the utility of the RD approach for a smart grid power dispatch problem is introduced, taking into account some communication constraints between the sensors and the centralized controller. 4. APPLICATION TO SMART GRID GENERATION DISPATCH Smart grid generation dispatch is a resource allocation problem that has been solved using a wide variety of methods (e.g., Gaing (2003); Tippayachai et al. (2002)). In this case, a total amount of power defined by a central controller must be supplied using N distributed generators (DGs), taking into account technical and economical aspects of each DG (Pantoja and Quijano (2011)). Then, the main objective is to maximize the general utility of all generators, where the vector of dispatched powers is p = [p1 , p2 , . . . , pN ]⊤ . The optimal power vector p∗ is achieved when all marginal utilities are equal, i.e., when the partial derivative of the ith utility function with re-

124

spect to pi evaluated in pi = p∗i is the same for all i. Hence, the fitness function fi (pi ) is defined as the marginal utility of each generator, in such a way that the optimal points are reached when all fitnesses are equal, as it was described in Section 2. In this case, technical and economical conditions are considered including nominal ⊤ powers pnom = [pnom1 , pnom2 , . . . , pnomN ] , and the costs ⊤ c = [c1 , . . . , cN ] of using a specific unit. The fitness function that relates the utility of using a generator i (goodness of an habitat) is (Pantoja and Quijano (2011)),   fi (pi ) =

1 ci

1−

pi pnomi

(12)

PN where the total amount of power Pd = j=1 pj , and the average fitness function is described as f¯ = PN 1 i=1 pi fi (pi ). Pd

In order to solve the dispatch process using the RD technique described in Section 2, we assume that xi = pi Pd , which can be seen as the portion xi of the total amount of power Pd that the ith generator has to produce. For this case, in Figure 1 each generator corresponds to each agent (habitat), whose dynamics are described by Equations (1) and (12), and the information exchanged with the central controller is asynchronous through a communication network (Figure 2). 4.1 Simulation Results

The problem we want to illustrate is based on the four distributed generators problem presented in (Pantoja and Quijano (2011)). Here, we consider the same fixed nominal powers pnom = [pnom1 , . . . , pnom4 ]⊤ = [172.4, 47.2, 66.1, 106.8]⊤ kW, and the cost factor for each generator is c = [1 0.8 0.3 0.4]⊤ . In this case, we assume that the delays blocks T1 and T2 in Figure 2 are different from 0. This means that time delays are present bidirectionally between the central controller and each DG. Figure 3 illustrates the portion of the demanded power Pd = 200 for both cases, when there are no delays (top plot), and when different delay profiles are simulated taking into account that the time delay between sensors and controller is the same in both directions (i.e, T1 = T2 = τ ). Due to the fact that the fitness is decreasing, and according to the delay independence of RD proved in Section 3, the equilibrium points are achieved even when the information exchanged between agents and the central controller is asynchronous. As depicted in Figure 3, the asymptotic stability holds and the demanded power is covered (i.e., at the steady state, the sum of the portion of individuals at each generator is 1, which is equivalent to 100% of the demanded power). However, the time that it takes to achieve the equilibrium points increases as the time delays increase due to the trajectories of the states of the system for a given x(0) ∈ ∆ are not always inside the simplex ∆ PN (i.e., i=1 xi 6= 1) until the steady state is achieved. Then, an increment in the time delay makes the trajectories to be out of the simplex for a period of time proportional to the delays because the average fitness is calculated with delayed measurements. For this case, even if the asymptotic stability is assured, the fact that the produced power exceeds the demanded

Demanded power portion of each DG

IFAC NecSys'12 September 14-15, 2012. Santa Barbara, CA, USA

where the Lkϕ λ is the k-times repeated Lie derivative of the function λ(x) with respect to the vector field ϕ, for j = 1, 2, . . . , m. With this, the Jacobian of zj can be obtained, by letting Qj (x) = J(Φj (x)).

Distributed generation dispatch with no delays 0.5

DG1 DG2

0.4

DG3 DG4

0.3

0.2

The observer proposed by (Cacace et al. (2010)) only considers output measurements y¯(t) ∈ R. However, we need to estimate m outputs. Then, the observer can be described as x ˆ˙ = f (ˆ x(t)) + g(ˆ x(t))u(t) + .. .   (15) ˆ . . . + Q−1 (ˆ x(t)) K y ¯(t) − h x ˆ(t − δ(t))

0.1

0 0

5

10

15

20

25

30

35

40

45

50

Total portion of demanded power

Distributed generation dispatch for several delay profiles 2 τ = [0.5, 0.4, 0.3, 0.2] τ = [1, 1.2, 0.5, 0.8] τ = [2.5, 2, 2.3, 2.6] τ = [3, 2.2, 2, 1.5] τ = [3.5, 1.5, 3.2, 2]

1.8 1.6

for

  Q−1 (ˆ x(t)) K = Q−1 x(t)) K1 , . . . , Q−1 x(t)) Km m (ˆ 1 (ˆ where Kj ∈ RN ×1 is a suitable gain vector associated to ˆ is an the j th output estimation for j = 1, . . . , m, and δ(t) approximation of the time delays.

1.4 1.2 1 0.8 0

5

10

15

20

25

30

35

40

45

50

Time (sec)

Fig. 3. Portion of the demanded power that each generator produces for the nondelayed case (top plot), and the total amount of demanded power for five delay profiles (bottom plot). power for an interval of time until the equilibrium point is achieved provokes overproduction of energy, which can be very expensive for the power utility and for the customer. Besides, the time that it takes to achieve the equilibrium points may cause degrading effects on the performance of a dynamical distributed system. These kind of problems arise because the average fitness value f¯, calculated by the central controller, does not correspond to the optimal one when sensor measurements are delayed. Therefore, the estimation of the RD states based on the information that it is sent to the central controller (i.e, xi fi (xi ), for each i) can improve the performance of the controller, and it can diminish the steady state time. Next, the state observer introduced in (Cacace et al. (2010)) is implemented in order to improve the performance of the smart grid generation dispatch method based on the replicator dynamics approach. 4.2 State Observer for Delayed Replicator Dynamics As it was pointed before, time delays can affect the performance of a system, even if it is stable. Hence, it is necessary to use some tools in order to deal with time delays. In the RD case, delays lead the system to leave the simplex because the average fitness f¯ is calculated using delayed measurements. In order to improve the performance, we have chosen the state observer in (Cacace et al. (2010)) in such a way that a more accurate average fitness value can be calculated, and the steady state time can be reduced. This observer allows the estimation of the states of the system when multiple outputs are measured, and it only needs the instantaneous information of the time delay of each sensor even if they are time variable. The introduced observer considers systems of the type x(t) ˙ = f (x(t)) + g (x(t)) u(t) y ¯(t) = h (x(t − δ(t)))

t > −Γ t > 0, δ(t) ∈ [0, Γ]

(13)

N

N

where x(t) ∈ R is the state of the system, u(t) ∈ R is the input, y ¯(t) ∈ Rm are the measured output, and δ(t) is the vector of varying delay measurements, bounded by some Γ > 0. In order to obtain the state observation, it is necessary to define the drift-observability map for each output hj (x), i.e., i h zj = Φj (x) = hj (x) Lf hj (x) . . . LfN −1 hj (x)

⊤

(14)

125

The selection of the parameter Kj depends on the Brunowsky Canonical form matrix (Ab , Cb ) (Ciccarella ¯ j ), where et al. (1993)), and the Vandermonde matrix Vj (λ ¯ λj = {λj1 , λj2 , . . . , λjN } are eigenvalues that are assigned to the matrix Ab −Kj Cb (see Cacace et al. (2010) for more details). Hence,  N −1 −1  N  λj1 . . . λj1 1 λj1  . . .  .  . . . Kj = − .. (16) ... . .  . −1 λN . . . λjN 1 jN

λN jN

Using this approach, the central controller is able to estimate each distributed state with a priori knowledge ¯f (ˆ of the time delays, and it calculates the ˆ x), which is an estimation of the average fitness that forces the trajectories to be closer to the simplex set. The parameter Kj is a gain vector that gives weights to the difference between the delayed ¯ and the estimated  output of the real system y ˆ output h x ˆ(t − δ(t)) . The variation of this parameter is ¯ j , and an increment directly related with the eigenvalues λ of these values provokes a faster estimation. However, there is a trade off between speed of estimation and performance, because if the estimation is very fast, the state calculation will tend to oscillate, and numerical problems might arise. Figure 4 illustrates the behavior of the total amount of power (in %) that is produced for all generators for different eigenvalues. Here, the output measurement vector is y ¯(t) = xf , which is the information that is sent to the central controller by the DGs (Figure 2). In this case, we consider one of the profile delays presented in Section 4.1, where τ1 = [3.5, 1.5, 3.2, 2], and Pd = 200. In order to see the effects in the variations of the eigenvalues, ¯ j = λ = [λ1 , . . . , λ4 ] for all the Kj . λ It can be seen that the use of observers for nonlinear systems is useful in order to diminish the effects of time delays in the system. Besides, the equilibrium points are achieved faster than the no estimation case. Here, the selection of the parameter Kj has a direct influence on the speed of estimation and in the convergence of the observer method. A faster gain value might provoke a faster stabilization of the dynamics of the system, but it can also produce convergence errors due to its fast variations. So, depending on the problem that is being addressed, each Kj has to be chosen in such a way that estimation speed and observer convergence are assured. This approach can be used not only in the smart

IFAC NecSys'12 September 14-15, 2012. Santa Barbara, CA, USA

Generation dispatch with state observation for different gains K 200 λ1= −0.0043, λ2= −0.004, λ3= −0.0037, λ4= −0.0033

Total amount of power dispatched (%)

λ1= −0.0087, λ2= −0.008, λ3= −0.0073, λ4= −0.0067 180

λ1= −0.013, λ2= −0.012, λ3= −0.011, λ4= −0.01 λ1= −0.0217, λ2= −0.02, λ3= −0.0183, λ4= −0.0167 No observation

160

140

120

100

80 0

5

10

15

20

25

30

35

40

45

50

Time (sec)

Fig. 4. Total amount of power dispatched for distinct eigenvalues. grid power dispatch problem, but also in other resource allocation problems where speed of convergence plays a primary role, and long steady state times are not allowed. 5. CONCLUSIONS The inclusion of communication networks induce new challenges in the development of resource allocation solutions due to delays and asynchronous sensor measurements. In this paper, we have presented a novel distributed control scheme for resource allocation problems based on a replicator dynamics approach, where delays are taken into account. We have shown, based on some passivity analysis, that this control approach is delay independent. Also, due to the simplicity of the model, this technique can be applied in a distributed environment where centralized controllers can be used, depending on the appropriate choice of the fitness function. For instance, the application to smart grid power dispatch shows that the optimal equilibrium point is achieved independently of time delays, although a certain performance degradation is induced. Thus, a state estimation tool is added in order to improve the systems performance, persevering the stability characteristics. In future works, the delay independent analysis should be extended for the local RD approach (Pantoja et al. (2011)), which is a decentralized strategy that could be widely used in novel applications where a high amount of distributed elements are considered and centralized control strategies are not feasible. Then, the conditions to assure asymptotic stability taking into account asynchronous elements could be also determined. REFERENCES Alboszta, J. and Miekisz, J. (2004). Stability of evolutionarily stable strategies in discrete replicator dynamics with time delay. Journal of Theoretical Biology, 231(2), 175 – 179. Antsaklis, P. and Baillieul, J. (2007). Special issue on technology of networked control systems. Proceedings of the IEEE, 95(1), 5 –8. Cacace, F., Germani, A., and Manes, C. (2010). An observer for a class of nonlinear systems with time varying observation delay. Systems & Control Letters, 59(5), 305 – 312. Chopra, N. and Spong, M. (2007). Delay-independent stability for interconnected nonlinear systems with finite L2 gain. In 46th IEEE Conference on Decision and Control, 3, 3847 –3852. 126

Ciccarella, G., Dalla Mora, M., and Germani, A. (1993). A luenberger-like observer for nonlinear systems. International Journal of Control, 57(3), 537–556. Clarke, F. (1983). Optimization and Nonsmooth Analysis. Wiley New York. Cruz Jr, J., Chen, G., Li, D., and Wang, X. (2004). Particle swarm optimization for resource allocation in uav cooperative control. In AIAA Guidance, Navigation, and Control Conference and Exhibit, 1–11. Providence, USA. Gaing, Z.L. (2003). Particle swarm optimization to solving the economic dispatch considering the generator constraints. IEEE Transactions on Power Systems, 18(3), 1187 – 1195. Han, Z. and Liu, K.J.R. (2008). Resource allocation for wireless networks : basics, techniques, and applications. Cambridge University Press. Khalil, H.K. (2002). Nonlinear Systems. Prentice Hall, 3 edition. Kunigami, M. and Terano, T. (2003). Connected replicator dynamics and their control in a learning multi-agent system. In J. Liu, Y.m. Cheung, and H. Yin (eds.), Intelligent Data Engineering and Automated Learning, volume 2690 of Lecture Notes in Computer Science, 18– 26. Springer Berlin / Heidelberg. Liu, J., Chu, M., and Reich, J. (2007). Multitarget tracking in distributed sensor networks. IEEE Signal Processing Magazine, 24(3), 36 –46. Niyato, D. and Hossain, E. (2009). Dynamics of network selection in heterogeneous wireless networks: An evolutionary game approach. IEEE Transactions on Vehicular Technology, 58(4), 2008 –2017. Olfati-Saber, R., Fax, J., and Murray, R. (2007). Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE, 95(1), 215–233. Pantoja, A. and Quijano, N. (2011). A population dynamics approach for the dispatch of distributed generators. IEEE Transactions on Industrial Electronics, 58(10), 4559 – 4567. Pantoja, A., Quijano, N., and Passino, K.M. (2011). Dispatch of distributed generators using a local replicator equation. In Proceedings of the CDC-ECE, 7494–7499. Ram´ırez-Llanos, E. and Quijano, N. (2010). A population dynamics approach for the water distribution problem. International Journal of Control, 83(9), 1947–1964. Schmidt, D., Shi, C., Berry, R., Honig, M., and Utschick, W. (2009). Distributed resource allocation schemes. IEEE Signal Processing Magazine, 26(5), 53 –63. Sinopoli, B., Sharp, C., Schenato, L., Schaffert, S., and Sastry, S. (2003). Distributed control applications within sensor networks. Proceedings of the IEEE, 91(8), 1235 – 1246. Taylor, P.D. and Jonker, L.B. (1978). Evolutionary stable strategies and game dynamics. Mathematical Biosciences, 40, 145 – 156. Tembine, H., Altman, E., El-Azouzi, R., and Hayel, Y. (2011). Bio-inspired delayed evolutionary game dynamics withnetworking applications. Telecommunication Systems, 47, 137–152. Tippayachai, J., Ongsakul, W., and Ngamroo, I. (2002). Parallel micro genetic algorithm for constrained economic dispatch. IEEE Transactions on Power Systems, 17(3), 790 – 797.