On-line control of the threshold policy parameter for multiclass systems

Automatica 46 (2010) 528–536

Contents lists available at ScienceDirect

Automatica journal homepage: www.elsevier.com/locate/automatica

Brief paper

On-line control of the threshold policy parameter for multiclass systemsI Michael M. Markou, Christos G. Panayiotou ∗ KIOS Research Center for Intelligent Systems and Networks and Department of Electrical and Computer Engineering, University of Cyprus, Nicosia, Cyprus

article

info

Article history: Received 21 May 2007 Received in revised form 11 June 2009 Accepted 20 November 2009 Available online 4 January 2010 Keywords: Stochastic fluid models Infinitesimal perturbation analysis Control Optimization Quality of service Buffer management Discrete event systems Queueing systems Communication systems

abstract Motivated by the problem of Quality of Service (QoS) provisioning, this paper investigates a system with two classes of traffic and addresses the problem of dynamically controlling the buffer threshold used with the threshold policy, such that an objective function is optimized. For the analysis, a Stochastic Fluid Model (SFM) is used to derive unbiased sensitivity estimators of various metrics (workload, loss, throughput and packet expiration) with respect to the buffer threshold. These estimators are computed using information obtained from the sample path of the ‘‘real’’ discrete event system. Thus, one can use these estimators together with stochastic approximation techniques in order to maintain the system at a near-optimum point despite any changes in the incoming traffic or the transmission capacity of the network. © 2009 Elsevier Ltd. All rights reserved.

1. Introduction In this paper we consider a node that serves two classes of traffic, with different Quality of Service (QoS) requirements. An example is a network that is required to support multimedia applications with real-time data such as MPEG video streams. In the MPEG protocol, three different types of picture (video frame) are used for video coding: I frames, which are complete images, and P and B frames, which have additional information that improves the image quality. Dropping or delaying an I frame has a significantly greater impact on the quality of the received video than dropping or delaying a B or P frame. As a result, an application has an incentive to mark its I frames as high priority and its B and P frames as low priority traffic; if any node experiences congestion, it will start dropping the low priority packets (B and P frames) and thus I frames have a better chance to get through. In this paper,

I Supported in part by the Cyprus Research Promotion Foundation under contract ENIΣ X/0504/21. The material in this paper was not presented at any conference. This paper was recommended for publication in revised form by Associate Editor Giuseppe De Nicolao under the direction of Editor Ian R. Petersen. ∗ Corresponding address: University of Cyprus, KIOS Research Center for Intelligent Systems and Networks and Department of Electrical and Computer Engineering, 75 Kallipoleos St., PO Box 20537, 1678 Nicosia, Cyprus. Tel.: +357 22 892298; fax: +357 22 892260. E-mail addresses: [email protected] (M.M. Markou), [email protected] (C.G. Panayiotou).

0005-1098/$ – see front matter © 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.automatica.2009.12.002

we assume that the source will voluntarily mark its packets; however, other marking approaches are also possible (see for example Heinanen & Guérin, 1999). In this paper, we assume that the buffer of any intermediate node (router) uses the Threshold Policy (TP) to accept or reject packets; the TP is described in detail in Section 2. Since in multimedia applications it is often required that the order of the packets received is preserved, only First-In First-Out (FIFO) policies are considered. Among this family of policies we adopted the TP, since it better protects high priority packets than other FIFO policies (e.g., pushout and limited RED policies) and it is very easy to implement (Cidon, Guérin, & Khamisy, 1994). In addition, we assume that packets have a time-to-live (TTL) field and thus a node does not forward any expired packets. In our approach, we adopt a Stochastic Fluid Model (SFM) and derive Infinitesimal Perturbation Analysis (IPA) estimators Cassandras and Lafortune (1999) (gradient estimators) of various performance metrics of interest. These SFM-based estimators are evaluated based on data observed from the ‘‘real’’ Discrete Event System (DES) and are then used together with stochastic approximation techniques Kushner and Yin (1997) to perform on-line control of the threshold parameter of the TP. We emphasize that we use SFMs only for control and optimization rather than performance evaluation, and various studies have shown that such approximate models are able to effectively determine the optimal solutions; for example, see Cassandras, Wardi, Melamed, Sun, and Panayiotou (2002) and references therein. Motivated by the approaches in

M.M. Markou, C.G. Panayiotou / Automatica 46 (2010) 528–536

Panayiotou and Cassandras (2001) and Panayiotou, Cassandras, Sun, and Wardi (2004), our goal is to develop an algorithm to dynamically control the buffer of network nodes that support multiple classes of traffic with different QoS requirements. An important advantage of the proposed methodology is that the gradient estimation process does not require any knowledge of the system’s underlying stochastic processes; in other words, it is model free. The contribution of this paper is that it derives the IPA sensitivity estimates of certain performance measures of interest with respect to the threshold parameter for a network buffer that involves two priority classes of traffic, and which limits network traffic by discarding expired packets. A similar problem has been investigated in Panayiotou et al. (2004) and Cassandras, Sun, Panayiotou, and Wardi (2003), but only for nodes that forward all packets. We point out that the introduction of packet expiration introduces a significant complication when comparing the model of this paper and the models in Panayiotou et al. (2004) and Cassandras et al. (2003). Specifically, in this paper the system dynamics depend on the state of the system x(·; ·), while Panayiotou et al. (2004) and Cassandras et al. (2003) assume no such dependency. Another important distinction is that the derived sensitivities are no longer piecewise constant functions. The authors of Yu and Cassandras (2003) and Markou and Panayiotou (2005) consider a problem of deriving IPA estimates for systems where the system dynamics depend on the state of the system; however, they only consider the single class case. Finally, this paper extends the results of Markou and Panayiotou (2006) by providing all lemma and theorem proofs as well as additional simulation results. 2. System model We investigate a single node and assume that each source marks its packets as high priority (HP) or low priority (LP). HP packets are always accepted. On the other hand, LP packets are accepted only if the state of the buffer is below a threshold. All accepted packets are served using a FIFO policy, though some packets may be removed from the buffer and discarded before service if they violate the predefined delay QoS requirements (e.g., they have expired). It is worth mentioning that this system is a Discrete Event System (DES), in the sense that its operation is defined over a set of discrete events (e.g., packet arrival/departure, packet expiration). Moreover, the buffer state (the number of packets in the buffer) can take only non-negative integer values. 2.1. The stochastic fluid modeling framework Fig. 1 shows the SFM queuing equivalent of the node with two distinct traffic streams of HP and LP packets (Cassandras et al., 2003). In this context, we assume that αh (t ) and αl (t ) correspond to the time-varying inflow processes of HP and LP traffic, respectively. The process β(t ) corresponds to the maximum service rate (successful transmission rate). The buffer is assumed to have infinite capacity and admits all HP flows; thus no HP fluid will be dropped due to buffer overflow. However, HP fluid can be dropped due to expiration. On the other hand, LP flows are accepted only when the buffer content x(t ; θ ) (queue length) is below a threshold θ ; otherwise, the LP fluid is dropped. Using the SFM framework, the TP works as follows:

• If 0 ≤ x(t ; θ ) < θ , all incoming flows are accepted. • If x(t ; θ ) = θ , HP and some LP flows are accepted.1 • If x(t ; θ ) > θ , only HP flows are accepted.

1 Note that in a discrete event modeling framework if an LP packet arrives when x(t ; θ) = θ it will always be dropped.

529

Fig. 1. The SFM of a node’s buffer.

Based on the input processes αh (t ), αl (t ) and β(t ) and the threshold policy (TP), the following processes are derived: the buffer workload x(t ; θ ), the node’s outflow δ(t ; θ ), the LP fluid overflow γ (t ; θ ) and the fluid expiration σ (x; t ; θ ) due to unfulfilled QoS requirements. For the remainder of this paper, the buffer threshold θ is the control parameter of interest. Next, we make the following assumption regarding the input processes. Assumption 1. With probability 1, αh (t ) < ∞, αl (t ) < ∞ and β(t ) < ∞ are piecewise constant functions. Assumption 1 can be extended to arbitrary piecewise differentiable functions but the piecewise constant assumption is used for simplicity. Finally, all processes evolve in a given time horizon [0, T ] for some fixed 0 < T < ∞. 2.2. System dynamics To ease the notation we also define A(t ) = αh (t ) + αl (t ) − β(t ) and B(t ) = αh (t ) − β(t ). The state dynamics of x(t ; θ ) are given by dx(t ; θ ) dt +

 0,   0, 

if x(t ; θ ) = 0 and A(t ) ≤ 0 if x(t ; θ ) = θ and A(t ) − σ (x; t ; θ ) > 0 and B(t ) − σ (x; t ; θ ) < 0 =   B(t ) − σ (x; t ; θ ), if x(t ; θ ) > θ   A(t ) − σ (x; t ; θ ), otherwise.

(1)

It is worth pointing out that the system dynamics differ from those in Cassandras et al. (2003) since they also depend on σ (x; t ; θ ) (which is a function of x(t ; θ )). In the real system, the packet expiration will be determined by a time stamp on each packet and will depend on the application’s requirements, the number of hops a packet has to travel and the network conditions. For the fluid model, we assume that σ (x; t ; θ ) = c · x(t ; θ ). (Note that this is a good approximation when the packet arrivals are Poisson and the expiration deadline of each packet is exponentially distributed. In this case, since any packet can expire irrespective of its place in the queue, the expiration process is equivalent to the birth–death process of an M /M /m system and the rate which takes the system from x to x − 1 is µx, where µ is the service rate.) 2.3. Notation and preliminaries A typical sample path can be decomposed into two types of alternating intervals: empty periods (EPs), during which the system is empty (x(t ; θ ) = 0) and non-empty periods (NEPs), during which the system is non-empty (x(t ; θ ) > 0). Fig. 2 shows a typical NEP. On the system sample path we identify the events where (i) the buffer becomes or ceases to be empty, (ii) x(t ; θ ) hits the threshold θ (either from above or below) and (iii) x(t ; θ ) becomes (and stays) equal to θ or ceases being equal to θ . These events are classified as exogenous in the sense that they do not depend on the parameters of the system, or endogenous otherwise. For example, the event buffer ceases to be empty (point v0 = ξ in Fig. 2) is exogenous because it is caused by a sign change of A(t ). Similarly, the event buffer ceases to be equal to θ (points v2 , v6 , and v8 ) is also exogenous. All other events are endogenous. Endogenous and exogenous events are important when determining the event time derivatives. Since exogenous events do not depend on the parameters of the system, their event time derivative with respect dv to θ , dθi = 0. On the other hand, in general, for endogenous events dvi dθ

6= 0.

530

M.M. Markou, C.G. Panayiotou / Automatica 46 (2010) 528–536

GT (θ ) = PT (θ ) = RT (θ ) =

1

Z

T 1 T

γ (t ; θ )dt

(5)

σ (x; t ; θ )dt

(6)

δ(t ; θ )dt .

(7)

0 T

Z 0

1 T

T

T

Z 0

All above functions are defined over a time interval [0, T ]. The performance measure of interest is a combination of the expectations of the sample functions E [QT (θ )], E [GT (θ )], E [PT (θ )], E [RT (θ )]. 3. IPA derivatives with respect to the buffer threshold

Fig. 2. A typical SFM-based NEP from the system sample.

2.4. Classification of time periods An NEP is further decomposed into three types of time period (No Loss (NL), Partial Loss (PL) and Full Loss (FL)), which are maximal intervals where the buffer level x(t ; θ ) is below, equal to or above the threshold θ , respectively. These periods are defined by the points vk,i , k = 1, 2, . . . , NT , i = 0, 1, . . . , NkS , which are the event times of all endogenous and exogenous events in the kth NEP. For the sake of brevity, when we refer to a single NEP we will drop the index k. Thus, when we refer to an NEP we imply the kth one and vi denotes the ith event time during the kth NEP. We define the subperiods pi = [vi , vi+1 ), i = 0, . . . , N S − 1, so that each belongs to one of the following three sets. Also, to simplify the notation we define the open intervals poi = (vi , vi+1 ), i = 0, . . . , N S − 1. No Loss (NL) Period Set W (θ ) := {p0 } ∪ {pi : x(vi ; θ) = θ and x(t ; θ) < θ , t ∈ poi }: During such periods the buffer content is x(t ; θ) < θ (excluding the starting point vi ) and no loss occurs. Examples of such periods are p0 = [ξ , v1 ), p3 = [v3 , v4 ), p6 = [v6 , v7 ), and p8 = [v8 , v9 ), in Fig. 2. Partial Loss (PL) Period Set U (θ ) := {pi : x(t ; θ ) = θ ; t ∈ pi }: During PL periods, the buffer content is x(t ; θ ) = θ and LP traffic experiences partial loss. Moreover, the end point vi+1 of each period in U (θ ) is locally independent of θ , since as already mentioned it corresponds to an exogenous event. Furthermore, during such periods, A(t )−σ (x; t ; θ ) > 0 and B(t )−σ (x; t ; θ ) < 0. In Fig. 2, p1 , p5 , and p7 are examples of PL periods within an NEP. Full Loss (FL) Period Set V (θ ) := {pi : x(vi ; θ ) = θ and x(t ; θ ) > θ, t ∈ poi }: In an FL period the buffer content x(t ; θ ) > θ (excluding the starting point vi ) and all LP traffic is lost. Examples of FL periods are p2 and p4 in Fig. 2. Summarizing, dx(t ; θ) dt +

A(t ) − σ (x; t ; θ ), 0, B(t ) − σ (x; t ; θ ),

( =

if t ∈ pi ∈ W (θ ) if t ∈ pi ∈ U (θ ) if t ∈ pi ∈ V (θ )

(2)

and the loss of LP traffic is given by 0, γ (t ; θ) = A(t ) − σ (x; t ; θ ), al (t )

(

if t ∈ pi ∈ W (θ ) if t ∈ pi ∈ U (θ ) if t ∈ pi ∈ V (θ ).

(3)

2.5. Performance metrics We define four sample functions. The cumulative workload (or equivalently the packet delay through Little’s law) QT (θ ), the packet loss volume (due to the rejection of LP packets in admission control) GT (θ ), the packet expiration volume (due to packet expiration of both HP and LP packets) PT (θ ), and the throughput RT (θ ). QT (θ) =

1 T

T

Z

x(t ; θ )dt 0

(4)

In this section we derive the IPA derivatives with respect to θ . For notational convenience such derivatives are denoted using the dy prime notation; thus y0 = dθ . First we present some intermediate results that are needed in the derivation. From (1), we can obtain the value of x(t ; θ ) for any value of t ∈ [vi , vi+1 ) ∈ V (θ ) ∪ W (θ ). Note that dx(t ; θ ) dt +

= I (t ) − σ (x; t ; θ ) = I (t ) − c · x(t ; θ )

(8)

where I (t ) is equal to either A(t ) or B(t ) depending on the interval (i.e., whether x(t ; θ ) is above, equal to or below θ ). Eq. (8) is a firstorder differential equation with solution x(t ; θ ) = x(vi ; θ ) · e−c (t −vi ) + e−c (t −vi )

Z

t

vi

I (τ ) · ec (τ −vi ) dτ .

(9)

Next, we evaluate x0 (t ; θ ) and x0 (t ; θ ) during the three different period types pi ∈ U (θ ), pi ∈ V (θ ) and pi ∈ W (θ ).

R

Lemma 1. The derivative of the buffer content x(t ; θ ) with respect to θ at any time t ∈ pi = [vi , vi+1 ) is

x0 (t ; θ ) =

 0,   1,


e−c (t −vi ) · 1 − vi0 · Bi (θ ) ,  
 −c (t −v ) i e · 1 − vi0 · Ai (θ ) ,

if pi = p0 = [ξ , v1 ) if pi ∈ U (θ ) (10) if pi ∈ V (θ ) if pi ∈ {W (θ ) \ p0 }.

Lemma 2. During an NEP, the integral of x0 (t ; θ ) in any time period pi is

 0, if pi = p0 = [ξ , v1 )    (vi+1 − vi ), if pi ∈ U (θ )      Z vi+1  1 · 1 − v 0 · Bi (θ )
· Ci (θ ), i 0 x (t ; θ )dt = c  if p ∈ V (θ ) i vi    
1   · 1 − vi0 · Ai (θ ) · Ci (θ ),   c if pi ∈ {W (θ ) \ p0 }

(11)

where, in both lemmas, Ai (θ ) = A(vi ) − c · θ , Bi (θ ) = B(vi ) − c · θ , and Ci (θ ) = 1 − e−c (vi+1 −vi ) . All lemma proofs are included in the Appendices. Note that these estimators require some rate information at specific points in time, namely at vi which are observable events and the event time derivatives vi0 , i = 0, . . . , N S . Next we present an iterative algorithm for determining the event time derivatives vi0 , i = 0, 1, . . . , N S − 1. For each interval pi , we evaluate vi0+1 as a function of vi0 .

M.M. Markou, C.G. Panayiotou / Automatica 46 (2010) 528–536

Lemma 3. The event time derivative vi0+1 in a period pi can be calculated as

vi0+1

 0, if pi ∈ U (θ )     1   , if pi = p0   A1 (θ )     Ci (θ) (1 − Ci (θ )) · Ai (θ) 0    · vi + ,   Ai+1 (θ ) Ai+1 (θ )   = (12) if pi ∈ W (θ ) \ (p0 ∪ pN S −1 )    1 − C (θ )) · B (θ) C (θ) ( i i i   · vi0 + , if pi ∈ V (θ )    Bi+1 (θ ) Bi+1 (θ )  
  1 − CN S −1 (θ ) · AN S −1 (θ) C S (θ) − 1    · vN0 S −1 + N −1 ,   k A (v ) A(vS )  S if pi = pN S −1 .

where Qk,i (θ ) =

vk,i+1

Z

vk,i

531

x(t ; θ )dt .

Differentiating Qk,i w.r.t. θ , we get the following lemma. Lemma 5. During any period pk,i , k = 1, . . . , NT , i = 0, . . . , NkS − 1, the derivative of Qk,i w.r.t. θ is given by Qk0,i (θ ) = θ · vk0 ,1 · 1 [pk,i = pk,0 ]

  + −vk0 ,i · θ + (vk,i+1 − vk,i ) · 1k,i [U (θ )]  
Ck,i (θ ) + θ · dk,i+1 + · 1 − vk0 ,i · Bk,i (θ ) · 1k,i [V (θ )] c  
Ck,i (θ ) + θ · dk,i+1 + · 1 − vk0 ,i · Ak,i (θ ) c

· 1k,i [W (θ ) \ (p0 )] Next we evaluate the IPA sensitivities of the different sample functions G(θ ), Q (θ ), P (θ ) and R(θ ). 3.1. IPA derivative of the loss volume G(θ )

where dk,i+1 = vk0 ,i+1 − vk0 ,i . Subsequently, one can sum all Qk0,i terms to obtain S

QT (θ ) = 0

Using the sample path decomposition described earlier, the fluid loss volume (5) can be written as

1

−1  NT NX k X

c · T k=1 i=0

c · (vk,i+1 − vk,i ) · 1k,i [U (θ )]


 + Ck,i (θ ) · 1 − vk0 ,i · Bk,i (θ ) · 1k,i [V (θ )]

S

GT (θ) =

−1 NT NX k 1X

T k=1 i=0

Gk,i (θ)

(13)

Z

vk,i+1

γ (t ; θ )dt .

vk,i






· 1k,i [W (θ ) \ (pk,0 )] .

3.3. IPA derivative of the expiration volume

Differentiating Gk,i (θ ) with respect to θ , we obtain the following result.

Substituting σ (x; t ; θ ) = c · x(t ; θ ) in (6), we get PT (θ ) = c · QT0 (θ ). 0

Lemma 4. During any period pk,i , k = 1, . . . , NT , i = 0, . . . , NkS − 1, the derivative of Gk,i (θ ) with respect to θ is given by

αl (vk,i+1 ) · Ck,i (θ) G0k,i (θ ) = · 1k,i [V (θ)] Bk,i+1 (θ )   αl (vk,i+1 ) · (1 − Ck,i (θ)) · Bk,i (θ) 0 + vk,i · αl (vk,i ) Bk,i+1 (θ )   · 1k,i [V (θ )] − Ak,i (θ ) · vk0 ,i + c · (vk,i+1 − vk,i ) · 1k,i [U (θ )]

T k=1 i=0

G0k,i (θ).

3.4. IPA derivative of the throughput

R0T (θ ) =

S

−1 NT NX k 1X

(17)

Lemma 6. The derivative of the throughput RT (θ ) in the interval [0, T ] is

where 1k,i [Ψ ] = 1 [pi,k ∈ Ψ ] is the usual indicator function that takes the value 1 if the condition holds and 0 otherwise. Lemma 4 describes how the derivative of the loss volume is computed during an arbitrary period pi . We point out that even though Lemma 4 seems complicated, it can be easily evaluated using the results of Lemma 3. Thus, the derivative of the loss volume G0 (θ) during the interval [0, T ] is G0T (θ) =

(16)

We point out that all terms θ · vk0 ,i in Lemma 5 have vanished in the sum since such terms appear at the end of a period and also at the beginning of the next period with opposite sign.

where Gk,i (θ ) =

+ Ck,i (θ ) · 1 − vk,i · Ak,i (θ ) 0



(14)

T k=1

−A(vN S ) · vN0 S . k

k

at that point. Note that A(vN S ) is negative; thus the throughput k

sensitivity is a non-negative quantity. 3.5. Properties of the derived sample derivatives In this subsection we derive two additional properties of the sample derivatives G0T (θ ), QT0 (θ ), and R0T (θ ). Assuming that at time T the buffer is empty,2 using the principle of fluid conservation, we can write T

(αh (t ) + αl (t ))dt = 0

T

Z

δ(t ; θ )dt + 0

T

Z

γ (t ; θ )dt 0

T

Z Again using the sample path decomposition of the previous section, the workload (4) can be rewritten as

(18)

k

This estimator consists of the buffer becomes empty event time derivative (end of the NEP) multiplied by the inflow rate A(vN S )

Z 3.2. IPA derivative of the workload Q (θ )

NT 1X

σ (x; t ; θ )dt .

+ 0

S

QT (θ) =

−1 NT NX k 1X

T k=1 i=1

Qk,i (θ )

(15)

2 Even if x(T ; θ) > 0, this relationship holds approximately for stable systems and large enough T .

532

M.M. Markou, C.G. Panayiotou / Automatica 46 (2010) 528–536

Using σ (x; t ; θ ) = c · x(t ; θ ), dividing both sides by T and differentiating w.r.t. θ , we get R0T (θ) + G0T (θ ) + c · QT0 (θ ) = 0,

(19)

which means that if one estimates two of the three required derivatives then the third one is also easily obtained. The second property involves the limit of the sample derivatives as c goes to zero and is given in the following proposition, stated without a proof (the proof can be found in Markou (2009)). Proposition 1. As c goes to zero, the sensitivity estimators of G(θ ) and Q (θ ) derived in (14) and (16) respectively approach the ones obtained in Cassandras et al. (2003). 4. Unbiasedness of the IPA estimators In this section it is shown that the derived IPA estimators G0T (θ ), QT (θ), PT0 (θ ) and R0T (θ ) are unbiased with respect to the SFMbased objective functions. Unbiasedness is an important property that allows us to exchange the expectation and differentiation operators, i.e., 0

dE [L(θ)] dθ

 =E

dL(θ ) dθ



Note that this assumption is mild and should be satisfied by most systems, since for a given finite observation interval T the probability of the inflow process to switch rate exactly when the buffer content equals the threshold θ is zero (notice that we have excluded all rate switches that occur during a U (θ ) period). Moreover, given a finite T we can always find a ∆min that satisfies the assumption. Next, given Assumption 2, we can obtain the following bounds. Lemma 7. With probability 1, there exists constant 0 < Rmin (T , ω) < ∞ such that |Ai (θ )|, |Bi (θ )|, |A(vi )| > Rmin (T , ω) for all event times vi , and such that E [1/Rmin (T , ω)] < ∞. Theorem 1. In the SFM context, the derived IPA estimators (14), (16)–(18) are unbiased. Proof. In general, the unbiasedness of an IPA derivative L0 (θ ) is ensured by the following two conditions (see Rubinstein & Shapiro, 1993, Lemma A2, p. 70): (i) For every θ ∈ Θ , the sample derivative L0 (θ) exists with probability 1. (ii) With probability 1, the random function L(θ ) is Lipschitz continuous throughout Θ , and the (generally random) Lipschitz constant has a finite first moment. Using Assumption 1 and Lemma 7, we guarantee that the sample derivatives exist (the derivative x0 (t ; θ ) does not exist if at some ti , x(ti ; θ) = θ and Ai (θ ) = 0 but the last condition is excluded by Lemma 7). What remains to be shown is Lipschitz continuity which is done using the Mean Value Theorem (MVT). Applying the MVT to the random function L(θ ) for any θ¯ ∈ (θ , θ + 1θ )

1θ

In this section we present some heuristics that allow the SFM-based algorithms to be computed using information from the DES sample path. The first heuristic concerns the estimation of the packet expiration rate c which is required in some of the IPA estimates. Recall that for the packet expiration process σ (x; t ; θ ) ≈ c · x(t ; θ ). The total number of expired packets can then be expressed as t

Z

σ (τ ; θ )dτ ≈ 0

Assumption 2. Let qn be the time instant that the inflow process has switched rate for the nth time during any intervals in W (θ ) ∪ V (θ), then, with probability 1, there exists ∆min such that min{θ − x(qn ; θ)} = ∆min > 0 and E [1/∆min ] < ∞.

L(θ + 1θ ) − L(θ )

5. Algorithm implementation in DES

Nexp (t ) =

.

The right-hand term is what we determine through the derived IPA algorithms; however, the left-hand term is what is required for the convergence of the stochastic approximation algorithms. To prove the unbiasedness of the derived estimators we make the following technical assumption.

¯ = L0 (θ)

which means that, if L0 (θ¯ ) is always bounded, then 1L(θ ) is also bounded and hence L(θ ) is Lipschitz continuous. Moreover, due to Assumption 1 and Lemma 7, the functions |Ai (θ )| and |Bi (θ )| are always bounded within the interval [Rmin , Rmax ] and Ci (θ ) within [0, 1]. Furthermore, a finite observation interval [0, T ] is decomposed into a finite number of periods (i.e., NT < ∞ is bounded) and each has a bounded length (i.e. vi+1 − vi < T ). Using this information in (12), we find that vi0+1 is also bounded, and from (14), (16)–(18) we conclude that all sample derivatives of interest are bounded by some random variable with bounded mean value. Thus the sample functions are also Lipschitz continuous. 

⇒

1L(θ) = L(θ + 1θ ) − L(θ ) = 1θ · L0 (θ¯ )

(20)

ct =

Nexp (t ) t · Qt (θ )

t

Z

cτ · x(τ ; θ )dτ = ct · t · Qt (θ ) ⇒ 0

(21)

where ct is an estimator of c obtained over t simulation time. Note that limt →∞ ct = c. Next, to estimate the arrival and service rates required by the IPA estimators one can maintain an exponentially weighted moving average of the interarrival times between two successive packets of the same class. For example, let τ ave be the average interarrival time between HP packets; then

τ ave = (1 − y) · τ + y · τ ave

(22)

where τ is the current sample and 0 ≤ y ≤ 1. Then αh (t ) = 1/τ ave . Similar algorithms can be used for all other rates. Finally, in the context of SFM, it is straightforward to identify the NL, PL, and FL periods; however, this is not clear in the context of a DES. In Panayiotou and Markou (2008) several alternative ways of identifying certain points on the sample path are investigated. For the purposes of this paper, we use the following simple algorithm for identifying the various points: • An NL period starts when the buffer is empty and a packet arrives or when x(t ; θ ) = θ and a packet departure (or expiration) occurs. • A PL period starts when x(t ; θ ) = θ − 1 and an arrival occurs (i.e. when the queue reaches the threshold from below) or when x(t ; θ ) = θ + 1 and a packet departure or expiration occurs (i.e. when the queue reaches the threshold from above). The end of a PL occurs when x(t ; θ ) = θ and either a packet is removed from queue or a packet arrives. • If the length of the PL period is smaller than the interarrival time of the two previous arrivals of high priority packets (occurring at time vi and vi+1 , respectively), then we consider that it is not a PL but a direct transition from NL to Full Loss (FL). The start point of the FL period is considered to be the point vi . 6. Simulation and results In this section we present some experimental results that show the properties of the proposed algorithms. The simulation study concerns a single node that accepts incoming traffic from a number of sources and uses the Threshold Policy to differentiate the QoS delivered to high and low priority streams. The objective is to

M.M. Markou, C.G. Panayiotou / Automatica 46 (2010) 528–536

(a) Workload sample derivative.

533

(b) Packet rejection volume sample derivative.

(c) Packet expiration volume sample derivative.

(d) Throughput sample derivative.

Fig. 3. The sample derivatives of the performance metrics of the DES model in Scenario I.

set the threshold parameter such that the following performance measure is minimized. min E [JT (θ )] θ

  = E wQ · QT (θ ) + wG · GT (θ ) + wPh · PTh (θ ) + wPl · PTl (θ ) (23) where wQ , wG , wPh and wPl are constant weights. The overall cost E [JT (θ )] represents a tradeoff between providing low delay, low packet expiration and low packet rejection probability. Note that in general one could also include a throughput term in (23) or one can assume that it is included in wQ , wG wPh and wPl due to (19). Note that, with the SFM presented in the previous sections, it is generally difficult to keep track of the expiration cost for the low and high priority traffic (though it is possible). For the purposes of this paper, we assume that the deadline of each packet is exponentially distributed and employ the memoryless property to get the required estimates as follows. xh xl PTh0 = PT0 , PTl0 = PT0 xh + xl xh + xl where xh and xl are the (time) average number of high and low priority packets in the buffer. dG(θ ) The derived IPA sensitivity estimates (i.e., estimates of dθ , dQ (θ ) dθ

dP (θ)

and dθ ) are used with a gradient-based stochastic approximation algorithm Kushner and Yin (1997) to obtain the optimum θ ∗ that minimizes the objective function J (θ ). An advantage of the approach is that it uses transient functions in the intervals [0, T ], and therefore, if at some point the input processes αh (t ), αl (t ), or β(t ) change, then it will automatically adjust to the new optimum (or near-optimum) threshold. The stochastic approximation algorithm used is given by

θn+1 = θn − sn · Hn (θn , ωn ),

n = 0, 1, 2, . . .

(24)

Table 1 The arrival processes for Scenario I. Source

ON time (ms)

OFF time (ms)

Arrival rate (pckts/ms)

Nh1 Nl1 Nh2 Nl2 Nh3 Nl3

Exp(45) Exp(45) Exp(40) Exp(40) Exp(60) Exp(60)

Exp(15) Exp(15) Exp(20) Exp(20) Exp(25) Exp(25)

Exp(0.33) Exp(0.33) Exp(0.20) Exp(0.25) Exp(0.20) Exp(0.20) dJ (θ )

where Hn (θn , ωn ) is an estimate of Tdθ evaluated at θ = θn , based on information obtained from a sample path ωn , and sn is an appropriate step size sequence (to maintain the adjustability properties of the algorithm, a constant step size σ was used). The estimators are evaluated based on information obtained from the real DES sample path. Next we present the simulation results using the DES model. For the first scenario, we considered three source nodes, N 1 , N 2 and N 3 , where each node N i is modeled by a pair of ON/OFF sources, Nhi and Nli , which correspond to the traffic generator of HP and LP packets respectively. The arrival rates from each traffic generator are random variables defined similarly: αhi (t ) and αli (t ) are the arrival rates of HP and LP packets respectively generated from the ith source. The arrival processes used in Scenario I are summarized in Table 1. Furthermore, the transmission time of each packet is assumed to be exponentially distributed with mean 1/λβ = 1 ms. To simulate the process σ (x; t ; θ ), each packet is marked with a time stamp Time-To-Live (TTL). If the time of a packet waiting in the buffer exceeds the TTL then the packet is considered as stale and is removed from the buffer. The TTL is set to be an exponential random variable with mean 1/λe = 20 ms. Fig. 3 presents the values of the derivatives of the performance measures GT (θ ), QT (θ ), PT (θ ) and RT (θ ), when applied

534

M.M. Markou, C.G. Panayiotou / Automatica 46 (2010) 528–536 11

Costs

10

9

J( )

8

Opt. Alg. ( =16) Opt. Alg. ( =1) 7 1

2

3

4

5

6

7

8 9 10 Threshold

11

12

13

14

15

16

Fig. 4. Optimization algorithm for Scenario I.

in the DES. The derivatives were evaluated using two methods: (i) IPA, i.e., the formulas (14), (16)–(18); the corresponding curves are dG–IPA, dQ –IPA, dP–IPA and dR–IPA, respectively, and (ii) the Finite Difference (FD) method on the performance metric data extracted from the brute force simulation; the corresponding curves are respectively dG–FD, dQ –FD, dP–FD and dR–FD. The simulation interval was set to T = 100 min. As the results of Fig. 3 show, in Scenario I the IPA estimators perform good approximation of the FD method curves, though we emphasize that FD results require at least two sample paths while the IPA approach requires only one. Fig. 4 presents the steps taken by the stochastic approximation algorithm (OptAlg) when the initial θ is set to 1 and 16. It is emphasized that the observation interval is set to just 1 min (i.e., the IPA estimates are considerably more noisy than the ones presented in Fig. 3). The cost weights are wQ = 1, wG = 20 and wP = 20. The step size σn is set to 20 for all n. Scenario II considers a situation where a number of video servers distribute a certain movie encoded using MPEG protocol and all video streams pass through a relay node. This node performs admission control according to the TP and accepts incoming packets of type B or P only if the buffer level is below the threshold; otherwise they are rejected. Type I packets are considered as high priority packets and are always accepted. For the purpose of this simulation the MPEG4 video traces of the movie ‘‘Jurassic Park’’ (Video traces research group, 2008) were used from all video servers. The time between two frames is constant and equal to 40 ms (i.e. 25 frames per second). Moreover, the bandwidth of the node is B = 10 000 bytes per second and the TTL is set to be an exponential random variable with mean 1/λe = 20 ms. Initially we performed brute force simulation to obtain the performance metric J (θ ) = wD · DT (θ ) + wG · GT (θ ) + wPh · PTh (θ ) + wPl · PTl (θ ), where DT (θ ), GT (θ ), PTh (θ ) and PTl (θ ) are the average delay, average packet rejection rate and average packet expiration rate of high and low priority packets, respectively. Moreover, the cost weights were set to wD = 1, wG = 8000, wPh = 4000 and wPl = 1000 in order to protect packets from being lost due to rejection of expiration. Three different cases were studied, in which 4, 7 or 10 video servers were active; the optimal point of operation for each case was found to be θ ∗ = 11, θ ∗ = 16 and θ ∗ = 22, respectively. We then ran OptAlg in a situation where initially seven video servers were active for the period (0, 1500) seconds, then three of them became inactive for a time period of 1500 s and finally, at time t = 3000 s the number of active video servers became equal to ten. During this time, OptAlg monitors the operation of the relay node and reconfigures the value of the threshold every 60 s to adapt to the network changes and set the operation of the node to an optimal point. Fig. 5 shows clearly that OptAlg initially converges to the optimal value of the threshold within a few iterations. When the network conditions change (at times t1 = 1500 and t2 = 3000) OptAlg quickly adapts its threshold to the new optimum point.

Fig. 5. Operation of the optimization algorithm in Scenario II.

Appendix A. Proof of Lemmas 1 and 2 To prove Lemmas 1 and 2 we consider four cases. Case I: t ∈ [ξ , v1 ), where x(θ; ξ ) = 0 and I (t ) = A(t ). Substituting in (9), we get x(t ; θ ) = e−c (t −ξ ) ·

t

Z ξ

A(τ ) · ec (τ −ξ ) dτ ,

which is independent of θ . Thus, differentiating with respect to θ , we get x0 (t ; θ ) = 0. Integrating x0 (t ; θ ) in the interval (v0 , v1 ), we also get 0. Case II: t ∈ [vi , vi+1 ) ∈ U (θ ), which implies that x(t ; θ ) = θ . Hence, x0 (t ; θ ) = 1. Integrating x0 (t ; θ ) in the interval (vi , vi+1 ) gives

Z

vi+1 vi

x0 (t ; θ )dt =

vi+1

Z

vi

1dt = vi+1 − vi .

Case III: t ∈ [vi , vi+1 ) ∈ V (θ ), where x(vi ; θ ) = x(vi+1 ; θ ) = θ and I (t ) = B(t ). Substituting in (9), we get x(t ; θ ) = θ · e−c (t −vi ) + e−c (t −vi ) ·

t

Z

B(τ ) · ec (τ −vi ) dτ ;

vi

thus, differentiating with respect to θ , x0 (t ; θ ) = e−c (t −vi ) + c · θ · vi0 · e−c (t −vi )

+ (e−c (t −vi ) )0 · −c (t −vi )

+e

t

Z

vi

Z

B(τ ) · ec (τ −vi ) dτ

t

B(τ ) · e

· vi

c (τ −vi )

dτ



= e−c (t −vi ) · 1 + c · θ · vi0 + c · vi0 ·  −

B(vi ) · vi0 + c · vi0 ·

−c (t −vi )

=e

Z

t

vi

0

Z

t

vi

B(τ ) · ec (τ −vi ) dτ

B(τ ) · ec (τ −vi ) dτ



 · 1 − vi · Bi (θ ) . 0



The integral of x0 (t ; θ ) in the interval (vi , vi+1 ) is

Z

vi+1 vi

x0 (t ; θ )dt =

Z

vi+1 vi

e−c (t −vi ) · 1 − vi0 · Bi (θ ) dt





 1 − e−c (vi+1 −vi ) = 1 − vi0 · Bi (θ ) · c   1 − vi0 · Bi (θ ) · Ci (θ ) = . 

c

Case IV: t ∈ [vi , vi+1 ) ∈ W (θ ), where either

M.M. Markou, C.G. Panayiotou / Automatica 46 (2010) 528–536

• 2 ≤ i < N S − 1, x(θ; vi ) = x(θ; vi+1 ) = θ and x(t ; θ ) < θ or • i = N S − 1, x(θ; vN S −1 ) = θ , x(θ; η) = 0 and x(t ; θ ) < θ . In both cases, I (t ) = A(t ) and similarly to Case III we get
x0 (t ; θ ) = e−c (t −vi ) · 1 − vi0 · Ai (θ ) ⇒ Z vi+1
1 x0 (t ; θ )dt = · 1 − vi0 · Ai (θ ) · Ci (θ ). c

vi

Appendix B. Proof of Lemma 3 Case I: pi (θ ) ∈ U (θ ). At time vi+1 the buffer content ceases having the value x(t ; θ ) = θ and starts either a Full Loss or a No Loss period. Using Assumption 1, this is caused by a change in the values of al (t ), ah (t ) or β(t ). Since all three processes are independent of θ , then vi0+i = 0. Case II: pi (θ ) ∈ V (θ ). During a full loss period we have x(vi ) = x(vi+1 ) = θ ⇒

vi+1

Z

vi

(B(t ) − c · x(t ; θ ))dt = 0.

Appendix D. Proof of Lemma 5 To ease the notation the index k is dropped. Differentiating Qi (θ ) w.r.t. θ , we obtain Qi0 (θ ) =

dθ

Bi+1 (θ ) ·

c

where dk,i+1 = vk0 ,i+1 − vk0 ,i .

RT (θ ) =

NT 1X

T k=1




period. The approach is similar to the previous cases; see Markou (2009) for the detailed derivation. Appendix C. Proof of Lemma 4 To ease the notation the index k is dropped. Differentiating Gi (θ) w.r.t. θ and then replacing vi0+1 using Eq. (12), we get Gi (θ) =

d dθ vi+1

Z =

vi

vi+1

Z

vi

(A(t ) − c θ )dt

· 1i [U (θ )] +

Z

vi+1 vi

al (t )dt

0

+ vi

ηk

Z

ξk

δ(t ; θ )dt =

ηk

Z

ξk

β(t )dt

Rk,EP (θ ) =

Z

ξk+1 ηk

δ(t ; θ )dt =

Z

ξk+1

ηk

(αh (t ) + αl (t ))dt

is the total outflow during the kth empty period (i.e., the EP that follows the kth NEP. Given the above sample path decomposition, we derive the following lemma. The derivatives of the Rk,NEP (θ ) and Rk,EP (θ ) w.r.t. θ are, respectively,

dRk,EP dθ

= β(ηk ) · ηk − β(ξk ) · ξk + 0

0

Z

ηk ξk

β 0 (t )dt = β(ηk ) · ηk0

= (αh (ξk+1 ) + αl (ξk+1 )) · ξk0+1 − (αh (ηk ) + αl (ηk )) · ηk0 +1 Z ξk+1 + (αh (t ) + αl (t ))0 dt ηk

= −(αh (ηk ) + αl (ηk )) · ηk0 .

= (A(vi+1 ) − c · θ ) · vi0+1 − (A(vi ) − c · θ ) · vi0 vi+1

(E.1)

is the total outflow during the kth non-empty period, and

dθ

· 1i [V (θ )] "

Z

Rk,NEP (θ ) =

dRk,NEP

γ (t ; θ )dt 0

(Rk,NEP (θ ) + Rk,EP (θ ))

where

− Bi (θ ) · vi − 1 − vi · Bi (θ ) · Ci (θ ) = 0 ⇔ C (θ ) (1 − Ci (θ )) · Bi (θ ) 0 i + · vi . vi0+1 = Bi+1 (θ ) Bi+1 (θ ) Case III: pi (θ ) ∈ W (θ ). For this period, there are four subcases depending on the value of x(t ; θ ) at the beginning and end of the

0

x(t ; θ )dt = θ · v10 · 1 [pi = p0 ]

  + −vi0 · θ + (vi+1 − vi ) · 1i [U (θ )]  
Ci (θ ) 0 + θ · dk,i+1 + · 1 − vi · Bi (θ ) · 1i [V (θ )] c  
Ci (θ ) 0 + θ · dk,i+1 + · 1 − vi · Ai (θ ) · 1i [W (θ ) \ {p0 }]

vi

0

vk,i

The throughput of the system is given by

(B(vi+1 ) − c · θ ) · vi0+1 − (B(vi ) − c · θ ) · vi0 Z vi+1 (B(t ) − c · x(t ; θ ))0 dt = 0 ⇔ + vi Z vi+1 Bi+1 (θ ) · vi0+1 − Bi (θ ) · vi0 − c · x0 (t ; θ )dt = 0 ⇔ 0

vk,i+1

Z

d

Appendix E. Proof of Lemma 6

Differentiating both sides w.r.t. θ ,

vi0+1

535

Thus, the derivative of Throughput in the interval [0, T ] is

# (A(vi+1 ) − c · θ )0 dt · 1i [U (θ )]

 Z + αl (vi+1 ) · vi0+1 − αl (vi ) · vi0 +

vi+1 vi

R0T (θ ) =



a0l (t )dt · 1i [V (θ )]

 = Ai+1 (θ ) · vi0+1 − Ai (θ ) · vi0 − c · (vi+1 − vi ) · 1i [U (θ )]   + αl (vi+1 ) · vi0+1 − αl (vi ) · vi0 · 1i [V (θ )] " Ci (θ ) = αl (vi+1 ) · Bi+1 (θ )  # αl (vi+1 ) · (1 − Ci (θ )) · Bi (θ ) 0 + vi · − αl (vi ) · 1i [V (θ )] Bi+1 (θ )   − Ai (θ ) · vi0 + c · (vi+1 − vi ) · 1i [U (θ )].

=



=

NT 1X

T k =1

(R0k,NEP (θ ) + R0k,EP (θ ))

NT 1X

T k =1

(β(ηk ) − αh (ηk ) − αl (ηk )) · ηk0

NT 1X

T k =1

−A(ηk ) · ηk0 =

NT 1X

T k=1

−A(vN S ) · vN0 S . k

k

Appendix F. Proof of Lemma 7 Without loss of generality, using Assumption 1, assume that at time t0 the inflow process A(t ) switches to a certain value and remains constant for a time interval tmax , namely A(t ) = At0 , for

536

M.M. Markou, C.G. Panayiotou / Automatica 46 (2010) 528–536

all t0 ≤ t < t0 + tmax . When at time t the buffer content x(t ; θ ) approaches the boundary θ from below, Eq. (9) becomes x(t ; θ) = x(t0 ; θ ) · e−c (t −t0 ) + e−c (t −t0 ) · At0 ·

t

Z

ec (t −t0 ) dτ

t0

= x(t0 ; θ ) · e−c (t −t0 ) +

At0 c

· e−c (t −t0 ) · (ec (t −t0 ) − 1). (F.1)

Assume that, at time t = vk,i (t0 < vk,i < t0 + tmax ), the buffer threshold x(t ; θ ) reaches θ . Then, by (F.1), At0

θ · ec (t −t0 ) = x(t0 ; θ ) + ec (t −t0 ) =

= ec (vk,i −t0 ) ≥

c

· (ec (t −t0 ) − 1) ⇔

At0 − c · x(t0 ; θ ) At0 − c · θ At0 − c · x(t0 ; θ ) + c · θ − c · θ At0 − c · θ

⇔

Ai (θ ) + c · ∆min

(F.2)

Ai (θ )

since A(t ) = At0 is constant in the interval [t0 , t ] = [t0 , vi ] and Ai (θ) = A(vi ) − c · θ . From (F.2), we get

Ai (θ) ≥

c · ∆min ec (vk,i −t0 ) − 1

> 0.

(F.3)

The maximum value that tmax = t − t0 = vi − t0 can take is T (the length of the time horizon). As a result,

Ai (θ) ≥

c · ∆min ec ·T − 1

= Rmin (T , ω) > 0.

It easily follows that E [1/Rmin (T , ω)]

(F.4)

=

h E

ec ·T −1 c ·∆min

i

=

ec · T − 1

E [1/∆min ] < ∞. The proof for |Bi (θ )| < Rmin (T , ω) and |A(vi ) < |Rmin (T , ω) can be derived in a similar way. c

References Cassandras, C. G., & Lafortune, S. (1999). Introduction to discrete event systems. Boston, MA: Kluwer Academic Publishers. Cassandras, C. G., Sun, G., Panayiotou, C. G., & Wardi, Y. (2003). Perturbation analysis and control of two-class stochastic fluid models for communication networks. IEEE Transactions on Automatic Control, 48(5), 770–782. Cassandras, C. G., Wardi, Y., Melamed, B., Sun, G., & Panayiotou, C. G. (2002). Perturbation analysis for on-line control and optimization of stochastic fluid models. IEEE Transactions on Automatic Control, 47(8), 1234–1248. Cidon, Israel, Guérin, Roch, & Khamisy, Asad (1994). On protective buffer policies. IEEE/ACM Transactions on Networking, 2(3), 240–246. Heinanen, J., & Guérin, R. (1999). A two rate three color marker. September. Kushner, H. J., & Yin, G. G. (1997). Stochastic approximation algorithms and applications. New York, NY: Springer-Verlag.

Markou, Michael M. (2009). QoS Provisioning in communication networks: A perturbation analysis of stochastic fluid models approach. Ph.D. thesis. University of Cyprus. Markou, M. M., & Panayiotou, C. G. (2005). Dynamic control and optimization of buffer size in wireless networks. In Proc. of the IEEE vehicular technology conference, VTC’05. Vol. 4, May 30–June 1. Stockholm, Sweden (pp. 2162–2166). Markou, Michael M., & Panayiotou, Christos G. (2006). Dynamic control and optimization of buffer size in multiclass wireless networks. In Proc. 49th IEEE GLOBECOM conference, GLOBECOMM’06. Panayiotou, C. G., & Cassandras, C. G. (2001). On-line predictive techniques for differentiated services networks. In Proceedings IEEE conference on decision and control (pp. 4529–4534). December. Panayiotou, C. G., Cassandras, C. G., Sun, G., & Wardi, Y. (2004). Control of communication networks using infinitesimal perturbation analysis of stochastic fluid models. In S. Tarbouriech, C. T. Abdallah, & J. Chiasson (Eds.), Lecture notes in control and information sciences (LCNCIS): Advances in communication control networks, (pp. 1–26). Springer-Verlag. Panayiotou, Christos G., & Markou, Michael M. (2008). On evaluating SFM-based infinitesimal perturbation analysis estimates from discrete event system data. IEEE Transactions on Automatic Control, 53, 560–565. Rubinstein, R. Y., & Shapiro, A. (1993). Discrete event systems: Sensitivity analysis and stochastic optimization by the score function method. New York: John Wiley and Sons. Video traces research group. Mp4 video traces of the film ‘‘Jurassic Park’’, Last Accessed On May 12th 2008. Yu, Haining, & Cassandras, Christos G. (2003). Perturbation analysis of feedbackcontrolled stochastic flow systems. In Proceedings of 42nd IEEE conference on decision and control, CDC’03 (pp. 6277–6282). Michael M. Markou received his B.Sc. degree in Computer Engineering and Informatics from the University of Patras, Greece, in 2003, and his Ph.D. in Computer Engineering from the University of Cyprus, Cyprus, in 2009. During his Ph.D. studies, Michael received funding from the Cyprus Research Promotion Foundation for two research projects under the Program for the Support of Young Researchers. He has published several articles in refereed journals and conferences and served as a reviewer in conference papers. His research interests include QoS in communication networks, optimization, discrete event systems and algorithm design in communication networks. Christos Panayiotou received his B.Sc. and Ph.D. degrees in Electrical and Computer Engineering from the University of Massachusetts at Amherst, in 1994 and 1999, respectively. He also received an MBA from the Isenberg School of Management, at the aforementioned university, in 1999. From 1999 to 2002 he was a Research Associate at the Center for Information and System Engineering (CISE) and the Manufacturing Engineering Department at Boston University. Since 2002 he has been with the Electrical and Computer Engineering Department at the University of Cyprus, where currently he is an Associate Professor and serves as the Department Vice-Chair. He is a founding member and Interim Deputy Director of the KIOS Research Center for Intelligent Systems and Networks. His research interests include distributed control systems, wireless, ad hoc and sensor networks, computer communication networks, quality of service (QoS) provisioning, optimization and control of discrete-event systems, resource allocation, simulation, transportation networks and manufacturing systems. He is an Associate Editor for the Conference Editorial Board of the IEEE Control Systems Society and the General Co-Chair of the International Conference on Artificial Neural Networks (ICANN 2009). He is a senior member of the IEEE and also a reviewer for various conferences and journals, and he has served in the organizing and program committees of various international conferences.