A Stochastic Model Predictive Controller for Systems with Unreliable Communications

Preprints, 5th IFAC Conference on Nonlinear Model Predictive Preprints, 5th IFAC Conference on Nonlinear Model Predictive Control Preprints, 5th Preprints, 5th IFAC IFAC Conference Conference on on Nonlinear Nonlinear Model Model Predictive Predictive Control September 17-20, 2015. Seville, Spain Available online at www.sciencedirect.com Control Control September 17-20, 2015. Seville, Spain September September 17-20, 17-20, 2015. 2015. Seville, Seville, Spain Spain

ScienceDirect

A A A A

IFAC-PapersOnLine 48-23 (2015) 057–064

Stochastic Model Predictive Controller Stochastic Model Predictive Controller Stochastic Model Predictive Controller Stochastic Modelwith Predictive Controller for Systems Unreliable for Systems with Unreliable for Systems with Unreliable for Systems with Unreliable Communications Communications Communications Communications

∗∗∗ Quevedo ∗∗ Prabhat K. Mishra ∗∗ ∗∗ Rolf Findeisen ∗∗∗ Quevedo ∗ Prabhat K. Mishra∗∗ ∗∗ Rolf Findeisen ∗∗∗ ∗ PrabhatChatterjee ∗∗ Rolf Findeisen ∗∗∗ Quevedo K. QuevedoDebasish PrabhatChatterjee K. Mishra Mishra∗∗ Rolf Findeisen Debasish ∗∗ Debasish Debasish Chatterjee Chatterjee ∗∗ ∗ ∗ Department of Electrical Engineering (EIM-E), University of ∗ Department of Electrical Engineering (EIM-E), University of ∗ Department of Engineering (EIM-E), Paderborn, Germany. [email protected] Department of Electrical Electrical Engineering (EIM-E), University University of of Paderborn, Germany. [email protected] ∗∗ Paderborn, Germany. [email protected] Systems & Control Engineering, Indian Institute of Technology ∗∗ Paderborn, Germany. [email protected] ∗∗ Systems & Control Engineering, Indian Institute of Technology ∗∗ Systems & Control Control Engineering, Indian Indian Institute Institute of Technology Technology Bombay, India. [email protected], [email protected] Systems & Engineering, of Bombay, India. [email protected], [email protected] ∗∗∗ Bombay, India. [email protected], [email protected] Institute for Automation Engineering (IFAT), Otto-von-Guericke ∗∗∗ Bombay, India. [email protected], [email protected] ∗∗∗ Institute for Automation Engineering (IFAT), Otto-von-Guericke ∗∗∗ Institute for Automation Engineering (IFAT), University [email protected] Institute forMagdeburg, AutomationGermany. Engineering (IFAT), Otto-von-Guericke Otto-von-Guericke University Magdeburg, Germany. [email protected] University Magdeburg, Germany. [email protected] University Magdeburg, Germany. [email protected] Abstract: There is a strong trend towards feedback control over communication channels, Abstract: There is a strong trend towards feedback control over communication channels, Abstract: There is trend towards feedback over communication channels, such as wireless networks. However communication resources are limited and unreliable, Abstract: There is a a strong strong trendoften towards feedback control control overare communication channels, such as wireless networks. However often communication resources limited and unreliable, such as wireless networks. However often communication resources are limited and unreliable, leading to random transmission loss.often To deal with this issue, model predictive control seems such as wireless networks. However communication resources are limited and unreliable, leading to random transmission loss. To deal with this issue, model predictive control seems leading to random transmission deal with this issue, model predictive seems to be well suited, since it allowsloss. to To generate future input sequences and the control consideration leading to random transmission loss. To deal with this issue, model predictive control seems to be well suited, since it allows to generate future input sequences and the consideration to be well suited, since it allows to generate future input sequences and the consideration of constraints. A variety of model predictive control formulations that take into account the to be well suited, since it allows to generate future input sequences and the consideration of constraints. A variety of model predictive control formulations that take into account the of constraints. A variety of model predictive control formulations that take into account the loss of information have been proposed in the literature. Such methods typically use online of constraints. A variety of model predictive control formulations that take into account the loss of information have been proposed in the literature. Such methods typically use online loss of information have been proposed in the literature. Such methods typically use online optimization of deterministic cost functions andliterature. compensate communication effects through loss of information have been proposed in the Such methods typically use online optimization of deterministic cost functions and compensate communication effects through optimization of cost functions and compensate communication effects buffering at receiving nodes. The a model predictive control scheme, optimization of deterministic deterministic cost present functionswork and presents compensate communication effects through through buffering at receiving nodes. The present work presents aa model predictive control scheme, buffering at receiving nodes. The present work presents model predictive control scheme, which adopts a stochastic cost function that explicitly accounts for random packet dropouts. buffering at receiving nodes. The present work presents a model predictive controldropouts. scheme, which adopts a stochastic cost function that explicitly accounts for random packet which adopts adopts a stochastic stochastic cost function that explicitly explicitly accounts for random packet dropouts. dropouts. Numerical studies illustratecost potential performance benefits of the for proposed controller. which a function that accounts random packet Numerical studies illustrate potential performance benefits of the proposed controller. Numerical studies illustrate potential performance benefits of proposed controller. Numerical studies illustrate potential benefits of the the proposed controller. © 2015, IFAC (International Federation of performance Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: model predictive control; MPC; stochastic control; networked control systems; Keywords: model predictive control; MPC; stochastic control; networked control systems; Keywords: model predictive control; stochastic packet dropouts; communication channel Keywords: model fading predictive control; MPC; MPC; stochastic control; control; networked networked control control systems; systems; packet dropouts; fading communication channel packet packet dropouts; dropouts; fading fading communication communication channel channel 1. INTRODUCTION Tsumura et al. (2009); Liang et al. (2010); K¨ogel and Find1. INTRODUCTION INTRODUCTION Tsumura et al. (2009); Liang et al. (2010); K¨ ogel and Find1. Tsumura et (2009); Liang et K¨ and Findeisen (2011); al. (2012); Antunes et al. 1. INTRODUCTION Tsumura et al. al.Donkers (2009); et Liang et al. al. (2010); (2010); K¨oogel gel and(2013); Findeisen (2011); Donkers et al. (2012); Antunes et al. (2013); eisen (2011); Donkers et al. (2012); Antunes et al. (2013); Wireless sensor-actuator technology is of growing interest eisen Yu and Fu (2016). Whilst i.i.d. communication models will (2011); Donkers et al. (2012); Antunes et al. (2013); Wireless sensor-actuator technology is of growing interest Yu and Fu (2016). Whilst i.i.d. communication models will Wireless sensor-actuator technology of growing interest Yu and Fu (2016). Whilst i.i.d. communication models will for process and automation industry,is see, e.g., Chen et al. often give better insight into networked control systems Wireless sensor-actuator technology is of growing interest Yu and Fu (2016). Whilst i.i.d. communication models will for process and automation industry, see, e.g., Chen et al. often give better insight into networked control systems for process and automation industry, see, e.g., Chen et al. often give better insight into networked control systems (2011); Kim and Kumar (2012); Johansson et al. (2014); (NCSs) than deterministic ones, fading communication for process and automation industry, see, e.g., Chen et al. often give better insight into networked control systems (2011); Kim and Kumar (2012); Johansson et al. (2014); than deterministic ones, fading communication (2011); Kim Kumar al. (NCSs) than deterministic ones, Lunze (2014); You et al. (2012); (2014). Johansson A driving et force behind (NCSs) channel network congestion levelscommunication are, in general, (2011); Kim and and Kumar al. (2014); (2014); (NCSs) gains than and deterministic ones, fading fading communication Lunze (2014); (2014); You et al. al. (2012); (2014). Johansson A driving driving et force behind channel gains and network congestion levels are, in general, Lunze You et (2014). A force behind channel gains and network congestion levels are, general, using wireless technology in monitoring and control appli- channel correlated, seeand Baddour and Beaulieu (2005); Lunze (2014); You et al. (2014). A driving force behind gains network congestion levels are, in inLindbom general, using wireless technology in monitoring and control applicorrelated, see Baddour and Beaulieu (2005); Lindbom using wireless technology in monitoring and control applicorrelated, see Baddour and Beaulieu (2005); Lindbom cations is its lower deployment and reconfiguration costs. et al. (2002); Leizarowitz et al. (2006). The time variability using wireless technology in monitoring and control applicorrelated, see Baddour and Beaulieu (2005); Lindbom cations is is its its lower lower deployment deployment and and reconfiguration reconfiguration costs. costs. et al. (2002); Leizarowitz et al. (2006). The time variability cations et al. (2002); Leizarowitz (2006). time In addition, devices can placed where costs. wires of wireless may et beal. caused byThe moving machines, cations is its wireless lower deployment andbe reconfiguration et al. (2002);channel Leizarowitz et al. (2006). The time variability variability In addition, wireless devices can be placed where wires of wireless channel may be caused by moving machines, In addition, wireless devices can be placed where wires of wireless wireless channel may be caused caused by the moving machines, cannot be placed, or where power sockets are not available. vehicles, people, andmay so forth, or when receiver or the In addition, wireless devices can be placed where wires of channel be by moving machines, cannot be be placed, placed, or or where where power power sockets sockets are are not not available. available. vehicles, people, and so forth, or when the receiver or the 2 cannot vehicles, people, and so forth, or when the receiver the However, a majorordrawback of sockets wirelessare communication transmitter are mounted on a moving object. Thisor moti2 cannot be placed, where power not available. vehicles, people, and so forth, or when the receiver or the However, a major drawback of wireless communication transmitter are mounted on a moving object. This moti2 However, a major drawback of wireless communication 2 transmitter are mounted on a moving object. This motitechnology lies in that wireless channels are subject to fadvates the adoption of models where dropouts are allowed However, a major drawback of wireless communication transmitter are mounted on a moving object. This motitechnology lies lies in in that wireless wireless channels channels are are subject subject to to fadfad- vates the adoption of models where dropouts are allowed technology vates adoption of where ing and interference, which frequently lead tosubject packet errors be the correlated. example, the dropouts work Wuare andallowed Chen technology lies in that thatwhich wireless channels areto fad- to vates the adoptionFor of models models where dropouts are allowed ing and interference, interference, frequently lead packetto errors to be correlated. For example, the work Wu and 1 lead ing and which frequently to packet errors to be be correlated. correlated. For example, the work Wu Wu and Chen Chen and consequently packet-dropouts. controller (2007) models the For times betweenthe successful transmissions 1 Therefore, ing and interference, which frequently lead to packet errors to example, work and Chen controller and consequently consequently packet-dropouts. packet-dropouts.11 Therefore, models the times between successful transmissions and Therefore, controller (2007) models times between successful transmissions design for systemspacket-dropouts. using wireless technology in general (2007) as a finite, andthe thereby bounded, Markov chain Kemeny Therefore, controller and consequently (2007) models the times between successful transmissions design for systems using wireless technology in general as aa finite, and thereby bounded, Markov chain Kemeny design systems using technology in as finite, and bounded, Markov needs tofor thewireless communication resource into and In contrast, Huang andchain Dey Kemeny (2007); design forcarefully systemstake using wireless technology in general general as a Snell finite, (1960). and thereby thereby bounded, Markov chain Kemeny needs to carefully take the communication resource into and Snell (1960). In contrast, Huang and Dey (2007); needs to carefully take the communication resource into and Snell (1960). In contrast, Huang and Dey (2007); account. For that propose, communication issues need to Smith and Seiler (2003); Xie and Xie (2009); You and Xie needs to For carefully take the communication communication issues resource into and Snell (1960). In contrast, Huang and You Dey and (2007); account. that propose, need to Smith and Seiler (2003); Xie and Xie (2009); Xie account. For that propose, communication issues need to Smith and and Seilerthe (2003); Xie process and Xie Xiedirectly (2009); You You and Xie be suitably abstracted. Of particular interestissues to the need current (2011) describe dropout as a and Markov account. For that propose, communication to Smith Seiler (2003); Xie and (2009); Xie be suitably abstracted. Of particular interest to the current (2011) describe the dropout process directly as a Markov be suitably abstracted. Of particular interest current (2011) describe dropout directly as a Markov paper are stochastic communication models, to asthe commonly chain and, thus,the allow for theprocess time between transmission be suitably abstracted. Of particular interest to the current (2011) describe the dropout process directly as a Markov paper are stochastic communication models, as commonly chain and, thus, allow for the time between transmission paper are stochastic communication models, as chain and, thus, allow time transmission adopted the communications literature; e.g., Tse successes be unbounded. general semi-Markov paper arein stochastic communication models, see, as commonly commonly chain and,to thus, allow for for the theMore time between between transmission adopted in the communications literature; see, e.g., Tse successes to be unbounded. More general semi-Markov adopted in the communications literature; see, e.g., Tse successes to be unbounded. More general semi-Markov and Viswanath (2005); Paulraj et al. (2003). models have been investigated in Censi (2011); Quevedo adopted in the (2005); communications literature; see, e.g., Tse models successeshave to been be unbounded. More general semi-Markov and Viswanath Paulraj et al. (2003). investigated in Censi (2011); Quevedo and Viswanath (2005); Paulraj et al. (2003). models have been investigated (2011); Quevedo et al. (2013a) within the contextin ofCensi Kalman filtering with and Viswanath (2005); Paulraj et al. (2003). models have been investigated in Censi (2011); Quevedo A variety of models have been utilized to describe com- et al. (2013a) within the context of Kalman filtering with et al. (2013a) within the context of Kalman filtering with intermittent observations. A variety of models have been utilized to describe comet al. (2013a) within the context of Kalman filtering with A variety variety of ofissues, modelsand have been utilized to describe describe comcommunication in been particular packet-dropouts, in intermittent observations. A models have utilized to intermittent observations. munication issues, and in particular packet-dropouts, in intermittent observations. munication issues, and in particular packet-dropouts, in estimation and control The most basic approach the use of online optimizations over finite horimunication issues, andproblems. in particular packet-dropouts, in Through estimation and control problems. The most basic approach Through the use of online optimizations over finite horiestimation and control problems. The most basic approach Through the use optimizations over horidescribes network effects via independent and identically zons, Model Control (MPC) methods a estimation and control problems. The most basic approach Through the Predictive use of of online online optimizations over finite finitehave horidescribes network effects via independent and identically zons, Model Predictive Control (MPC) methods have a describes network effects via independent and identically zons, Model Predictive Control (MPC) methods have distributed (i.i.d.) random variables; see, e.g., Imer et al. remarkable capability to handle a variety of constrained describes network effects via independent and Imer identically zons, Modelcapability PredictivetoControl (MPC) methods have a a distributed (i.i.d.) random variables; see, e.g., et al. remarkable handle a variety of constrained distributed (i.i.d.)etrandom random variables; see,and e.g., Imer et al. al. 2remarkable capability to handle aa variety of constrained (2006); Schenato al. (2007); Tabbara Neˇ si´c (2008); distributed (i.i.d.) variables; see, e.g., Imer et remarkable capability to fading handlechannel variety of partially constrained (2006); Schenato et al. (2007); Tabbara and Neˇ s i´ c (2008); The time-variability of the can be com(2006); 1 (2006); Schenato Schenato et et al. al. (2007); (2007); Tabbara Tabbara and and Neˇ Neˇssi´ i´cc (2008); (2008); 22 The time-variability of the fading channel can be partially com-

Daniel Daniel Daniel Daniel

E. E. E. E.

situation is different to that of wired channels, 1 The The situation is different to that of wired channels, 1 errors delaysis mostlyto bywired congestion. The situation different that channels, 1 The and situation isare different tocaused that of of wired channels,

where where where where

packet packet packet packet

pensated for through control the power levels by the radio of fading channel can be com2 The The time-variability time-variability of the the of fading channel can used be partially partially compensated for through control of the power levels used by the radio amplifiers, e.g., using predictive control, see levels Quevedo etby al.the (2014b). pensated for through control of the power used radio pensated for through control of the power levels used by the radio amplifiers, e.g., using predictive control, see Quevedo et al. (2014b). amplifiers, amplifiers, e.g., e.g., using using predictive predictive control, control, see see Quevedo Quevedo et et al. al. (2014b). (2014b).

errors and delays are mostly caused by congestion. errors errors and and delays delays are are mostly mostly caused caused by by congestion. congestion. Copyright © 2015 IFAC 57 Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © 2015, IFAC (International Federation of Automatic Control) Copyright © 2015 IFAC 57 Copyright © 2015 IFAC 57 Peer review under responsibility of International Federation of Automatic Copyright © 2015 IFAC 57 Control. 10.1016/j.ifacol.2015.11.262

2015 IFAC NMPC 58 September 17-20, 2015. Seville, Spain Daniel E. Quevedo et al. / IFAC-PapersOnLine 48-23 (2015) 057–064

control problems, see, e.g., Mayne (2014) for a recent survey or the textbooks Gr¨ une and Pannek (2011); Rawlings and Mayne (2009). It is therefore not surprising that significant research efforts have been devoted to develop interesting MPC formulations for NCSs. In particular, to compensate for the effect of unreliable communications, socalled packetized predictive control (PPC) methods have been proposed. These exploit the capability of modern communication protocols to send, possibly large, and timestamped packets. This opens the possibility to transmit packets of data containing finite sequences, rather than individual values. Through buffering and appropriate selection logic at the receiver node, time delays and packet dropouts can, at least to some extent, be compensated for. When dealing with controller-to-actuator connections, PPC based on MPC is especially convenient, since tentative plant input sequences are directly provided by the MPC optimizer.

γk

Controller

ck

Dropout Channel

wk Smart uk Actuator

Plant

xk

Fig. 1. Control using an unreliable channel between controller and actuator with optional receipt acknowledgements. However, the analysis of stochastic PPC for systems with unreliable communications has been largely unexplored, save for the work of Fischer (2014), which is restricted to LTI plants and i.i.d. communication effects. Unfortunately, extending this approach to nonlinear plant models will in general lead to highly involved online optimizations, which are needed to derive closed-loop control policies.

Deterministic stability results of PPC algorithms when communication effects can be deterministically bounded, have been obtained in various works, including Tang and de Silva (2007); Mu˜ noz de la Pe˜ na and Christofides (2008); Chaillet and Bicchi (2008); Findeisen and Varutti (2009); Pin and Parisini (2009); Quevedo and Neˇsi´c (2011) and, more recently, in Nagahara et al. (2014) which adopts a sparsity promoting cost function. Despite the widespread use of stochastic network models in the communications community, such models have only recently been used to investigate stability and performance of PPC formulations for unreliable channels. In particular, the work presented in Quevedo et al. (2011) studies quantized control of perturbed LTI systems with i.i.d. packet dropouts. By adapting results from Markov Jump-linear systems and investigating random-time drift conditions, Quevedo et al. (2011) derived sufficient conditions for mean-square stability and presents closed form expressions for spectra of interest, see also Quevedo et al. (2013b). For nonlinear plant models and Markovian dropouts, Quevedo and Neˇsi´c (2012) established sufficient conditions for stochastic stability of nonlinear plant models and Reble et al. (2013) developed performance estimates.

Motivated by the above, the present work presents a stochastic MPC formulation, which is related to PPC and can be applied to plant models with linear dynamics and constrained inputs. By adapting ideas from Chatterjee et al. (2011), the focus is on policies that are affine in bounded functions of the past disturbances. Thereby, our current approach directly works with hard constraints on the controls, i.e., does not require artificially relaxing the hard constraints on the controls to soft probabilistic ones (to ensure large feasible sets), and still provides a globally feasible solution to the problem. The effects of the noise and channel dropouts appear in the resulting convex optimization problem as fixed cross-covariance matrices, which may be computed off-line via Monte Carlo methods and stored. The remainder of this paper is organized as follows: Section 2 presents the control architecture of interest. The deterministic PPC method of Quevedo and Neˇsi´c (2012) is briefly revised in Section 3. Section 4 presents the stochastic MPC formulation. Section 5 documents numerical studies which compare the performance of the two controllers. Section 6 draws conclusions.

Common to the works mentioned in the above paragraph is that, whilst dropout effects are compensated through the use of PPC and some of the analysis methods allow for stochastic channels, the controller itself is deterministic, i.e., the MPC cost function does not take into account the fact that the communication channel is unreliable. It is intuitively clear that such deterministic controllers will, in general, be outperformed by carefully designed stochastic PPC formulations.

Notation: We write R for the real numbers, N for {1, 2, 3, . . .}, and N0 for N ∪ {0}. The p × p identity matrix is denoted via Ip , whereas 0p  0 · Ip . We adopt the  2 definition i= ai = 0, if 1 > 2 and irrespective of ai . 1 The weighted√Euclidean norm of a vector x is denoted via xP = xT P x; if X is a set, then |X| denotes its cardinality. For scalars, | · | refers to absolute value, and · denotes the floor function. To denote the unconditional probability of an event Ω, we write Pr{Ω}. The conditional probability of Ω given an event Γ is denoted via Pr{Ω | Γ}. The expected value of a random variable ν given Γ, is denoted by E{ν | Γ}, whereas for the unconditional expectation we will write E{ν}. We use the same notation for random variables and their realizations; what is meant will depend upon the context.

Stochastic versions of MPC-related techniques initially evolved within the operations research community, with inventory and manufacturing systems as primary application areas, and have steadily filtered into the domain of control systems, with current applications in financial engineering, process control, industrial electronics, power systems, etc.; see Chatterjee et al. (2011); de la Pe˜ na et al. (2005); Kouvaritakis et al. (2010); Bertsimas and Brown (2007); Primbs and Sung (2009); Felt (2003); Goodwin et al. (2009); Bernardini and Bemporad (2009); Batina et al. (2001); Mesbah et al. (2014); Skaf and Boyd (2009); van Hessem and Bosgra (2003) and the references therein. 58

2015 IFAC NMPC September 17-20, 2015. Seville, Spain Daniel E. Quevedo et al. / IFAC-PapersOnLine 48-23 (2015) 057–064

2. SYSTEM SETUP

59

p12 p22

p11

Throughout this work, we shall focus on a single-loop control architecture with an unreliable input communication channel, as depicted in Fig. 1. The plant model is described in discrete-time via: xk+1 = f (xk , uk , wk ), k ∈ N0 , (1) where x ∈ Rn is the plant state, and u ∈ U ⊆ Rp denotes the (constrained) plant input. The plant model is perturbed by a random process {wk }k∈N0 , which in general does not have bounded support. As is common in most MPC formulations, we shall assume that the controller has direct access to the plant state. 3 We also assume that clocks are synchronized and that there are no transmission delays.

Ξk = 1 p1

Ξk = 2 p2

”good”

”bad”

p21 Fig. 2. Transmission dropout model with a binary network state {Ξk }k∈N0 : when Ξk = 1 the channel is reliable with very low dropout probabilities; Ξk = 2 refers to a situation where the channel is unreliable and transmissions are more likely to be dropped. Key to the idea of PPC is the use of a “smart actuator”, which contains a buffer and selection logic. The buffering mechanism amounts to a parallel-in serial-out shift register, which acts as a safeguard against dropouts. For that purpose, the buffer state, say bk ∈ UN , is overwritten whenever a valid (i.e., uncorrupted and undelayed) control packet arrives. Actuator values are passed on to the plant sequentially until the next valid control packet is received. If the buffer runs out of data, then the plant input is set to zero. More formally, we have: bk = (1 − γk )Sbk−1 + γk ck , (6) u k = e T bk where   0p Ip 0p . . . 0p  .. . . . . . . ..  . . . . . pN ×pN  , S 0p . . . 0p Ip 0p  ∈ R (7) 0 . . . . . . 0 I  p p p 0p . . . . . . . . . . 0p

Transmissions of controller outputs, say ck to the actuator node is through an unreliable communication channel which is affected by packet dropouts. We shall model transmission effects via a binary random process {γk }k∈N0 :  0 if packet-dropout occurs at instant k, γk  1 if packet-dropout does not occur at instant k. (2) This leads to the following temporal ordering: xk → ck → γk → uk , wk −→ xk+1 → . . . . (3) In the simplest instance, the dropouts are modeled as i.i.d., leading to a Bernoulli channel model. More general descriptions allow {γk }k∈N0 to be correlated. For instance, Fig. 2 illustrates a simplified version of the model adopted in Quevedo et al. (2013a). Here, communication reliability is modeled via an underlying network state process {Ξk }k∈N0 taking on values in a finite set B = {1, 2, . . . , |B|}. Each i ∈ B corresponds to a different environmental condition (such as network congestion or positions of mobile objects). For a given network state the dropouts are modeled as i.i.d. with probabilities pi = Pr{γk = 1 | Ξk = i}, i ∈ B. (4) Correlations can be captured in this model by allowing {Ξk }k∈N0 to be time-homogeneous Markovian with transition probabilities:    pij = Pr Ξk+1 = j  Ξk = i , ∀i, j ∈ B, k ∈ N. (5) It is easy to see that the above model encompasses Markovian dropouts and i.i.d. dropouts as special cases.

eT  [Ip 0p . . . 0p ] ∈ Rp×pN .

The control packets ck are formed by adapting the ideas underpinning MPC: At each time instant k and for a given plant state xk , the following cost function is minimized: N −1  J(u , xk )  F (xN ) + L(x , u ). (8) =0

The cost function in (8) examines predictions of the plant (1) over a finite horizon of length N , which is taken equal to the buffer size. For simplicity, the predicted state trajectories do not take into account packet-losses or disturbances and are, thus, generated by the nominal model: x+1 = f (x , u , 0), ∈ {0, 1, . . . , N − 1} (9) starting from x0 = xk and where the entries in T  u  = (u0 )T . . . (uN −1 )T ∈ UN are the associated plant inputs. Predicted plant states and inputs are penalized via the per-stage weighting function L(·, ·) and the terminal weighting F (·). These design variables allow one to trade-off control performance versus control effort. As in control loops without dropouts, see, e.g., Rawlings and Mayne (2009), the choices made for L(·, ·), F (·) and N influence closed loop (stochastic) stability and performance, see Quevedo and Neˇsi´c (2012) for details.

3. DETERMINISTIC PACKETIZED PREDICTIVE CONTROL This section briefly revises the PPC formulation adopted in Quevedo and Neˇsi´c (2012), which is similar to other methods described in the literature. As foreshadowed in the introduction, at each time instant k and for plant state xk , with PPC the control packet ck , is computed and transmitted over the dropout channel to the actuator node. To achieve good performance despite unreliable communication, ck contains tentative plant inputs for times {k, k + 1, . . . , k + N − 1}. 3

If that were not the case, then one could use a suitably designed Kalman filter; see, e.g., Schenato et al. (2007).

59

2015 IFAC NMPC 60 September 17-20, 2015. Seville, Spain Daniel E. Quevedo et al. / IFAC-PapersOnLine 48-23 (2015) 057–064

The control packet ck , see Fig. 1, is set equal to the constrained optimizer4 , ck  arg min J(u , xk ) (10)

There are two sources of randomness in this system, namely, the disturbance process {wk }k∈N0 and the dropout process {γk }k∈N0 , see (2). Both are here taken as i.i.d. processes.

and is sent through the network to the buffer. Therefore, at each instant, N tentative plant-input values are transmitted through the communication channel.

4.2 Cost function and policy class

u ∈UN

To incorporate the random phenomena in the controller, the objective function is chosen as the expectation of the predicted quadratic cost given the current plant state, cf. (8):  N −1       2  2  2 x Q + u R  xk , V (π, xk ) = E xN P +

Following the receding horizon optimization idea, at the next sampling step and given xk+1 , the horizon is shifted by one and another optimization is carried out, providing ck+1 = arg minu ∈UN J(u , xk+1 ), sequence, which is transmitted over the unreliable channel to the smart actuator. This procedure is repeated ad infinitum.

=0

Note that ck in (10) contains tentative constrained plant input values for instants {k, . . . , k+N −1}. If ck is received at time k, then these values are written into the buffer and implemented sequentially until some future (valid) control packet arrives. If there are no dropouts then the scheme reduces to regular deterministic MPC.

(13) where P and Q are positive semi-definite, and R is a positive definite matrix. As before, primed variables x and u are predicted values which respect system dynamics and constraints (12). However, in this case the system model with disturbances in (11) and not the nominal model (9), is used.

In the case of Markovian dropouts (i.e., where in (4), p1 = 1 and p0 = 0), Quevedo and Neˇsi´c (2012) showed how to design the tuning parameters N , L(·, ·) and F (·) to ensure boundedness of moments of the plant state

A perhaps subtle but important feature of this formulation is that the optimization is not over finite-length sequences, but over causal history-dependent feedback policies of finite length, i.e., π = {π0 , π1 , . . . , πN −1 }, see, e.g., Bertsekas (2005), so that u = π (xk , xk+1 , . . . , xk+ ), ∀ ∈ {0, 1, . . . , N − 1}. (14)

It is important to emphasize that in this deterministic formulation, the plant input design is done dynamically such as to optimize performance, but without using knowledge on the dropout distribution parameters pi and pij or receipt acknowledgements, see Fig. 1. The optimization (8) does not explicitly take into account the communication issues, save for what can be captured by the tuning parameters. It is intuitively clear, that a stochastic controller, which includes a model of communications will in general give better performance. In the following section, we will present such an algorithm. It uses a quadratic cost function and focuses on plant models with linear dynamics, but goes beyond the work of Fischer (2014) by also taking into account constraints on system inputs.

Minimizing (13) over all classes of causal policies commonly leads to a computationally intractable problem. Hence, we will restrict our attention to a specific class of policies which are affine in past disturbances. Following as in Chatterjee et al. (2011), the N -history-dependent policies in (14) are parameterized as per u = η +

−1  i=0

4. A STOCHASTIC MPC FORMULATION

  θ,i ϕi+1 wk+i ,

∈ {0, 1, 2 . . . , N − 1},

(15) where η ∈ Rp and θ,i ∈ Rp×n are the policy parameters to be determined using xk , statistics of disturbances and dropouts and system parameters. In particular, η can be interpreted as predicted plant input sequences, whereas θ,i gauges how future information on disturbances is used to readjust the control signal. In (15), ϕi : Rn −→ U are given bounded maps, i.e., there exists ϕmax ∈ R, such that 6 ϕi (w)∞ ≤ ϕmax , ∀(i, w) ∈ {1, 2 . . . , N } × Rn .

With the aim of developing a high-performance MPC formulation with moderate online computational complexity, in the present section we will embellish the approach of Chatterjee et al. (2011) to handle communication issues. 4.1 Setup We shall restrict our attention to perturbed LTI plant models of the form (11) xk+1 = Axk + Buk + wk , where the elements of the plant input are constrained in magnitude as per: uk ∈ U  {ν ∈ Rp : ν∞ ≤ Umax }, ∀k ∈ N0 . (12) Communication is over an i.i.d. dropout channel with reliable causal acknowledgements as in Fig. 1, i.e., the sequence {γ0 , γ1 , . . . , γk−1 } is known to the controller at time k ∈ N, see (3). 5

The terms ϕi may represent functions that are easy or inexpensive to implement, or be the only ones available for specific applications. For instance, piecewise constant policy elements with finitely many elements in their range may be viewed as a hybrid controller with a finite control alphabet, cf., Aguilera and Quevedo (2015). Remark 1. Due to acknowledgements, at any time k the controller has access to ck−1 , xk , xk−1 and γk−1 , see (3), and thereby also knows the plant input uk−1 . Thus, using

4

We assume that the optimizer is unique. If acknowledgements are unreliable, then optimisations are significantly harder, see, e.g., Nourian et al. (2014).

5

6 Boundedness is important to be able to ensure that the controls satisfy the constraint uk ∈ U.

60

2015 IFAC NMPC September 17-20, 2015. Seville, Spain Daniel E. Quevedo et al. / IFAC-PapersOnLine 48-23 (2015) 057–064

the system model (11), disturbances can be recovered by the controller after a one-step delay, by simply evaluating wk−1 = xk − Axk−1 − Buk−1 . 

ut =

Expression (15) can be rewritten in vector form, leading to        ϕ1 wk  u0   ϕ2 wk+1  u1    .  = η + Θ  := η + Θϕ(Wk ),  ..  .    . .   uN −1 ϕN −1 wk+N −2 (16) where η ∈ RpN and Θ ∈ RpN ×n(N −1) is strictly lower block-diagonal. Thus, the policy π is characterized by its parameters contained in the pair (η, Θ). The hard constraint on control (12) for the above policy is satisfied by the following element-wise inequality: |η (i) | + ϕmax Θ(i) 1 ≤ Umax

where

for all t ∈ N0 .

(17)

k ∈ K,

is used by the controller to calculate the the pair optimal control input using the expression (see (15)):

i=0

where

(18)

γk+i = 0,

i=0

In a worst case (corresponding to p = 0), with the stochastic controller, the number of values transmitted during a period of N steps is given by N (N + 1) (N − 2)(N − 1) N −1+ = N2 − . 2 2 Thus, for Nr = N > 2 or success probabilities p > 0, the proposed stochastic MPC method uses strictly less communication resources than deterministic PPC.

Recalling Remark 1, at each time

−1 

γk+i ≥ 1

with i ∈ {0, 1, . . . , N − 2}. This stands in contrast to the deterministic PPC approach described in Section 3, where at every instant a fixed number of N values is transmitted, so that Ck = N 2 for all k ∈ N0 .

where k ∈ K.

ut = η k, +

i=0

=0

Different to the method in Section 3, the minimization of (13) is not carried out at every instant k ∈ N, but only every Nr ≤ N time steps. We shall refer to Nr as the recalculation interval, not to be confused with the constraint horizon used in some MPC schemes. Using notation as presented in the preceding section, this leads to the sequence of optimizing policy parameters:

(η k , Θk )

if

 

−1 

To elucidate this situation, we shall set N = Nr and denote the transmission success probability (assumed i.i.d.) by p = E{γk }. Then, it is easy to see that  i    i  p, if j = N − 1 + (N − ) (1 − p)   =0 Pr{Ck = j} = N −1    N −1   (1 − p) , if j = N − 1 + (N − ) 

4.3 Optimization and Transmission Protocol

t ∈ {k, k + 1, . . . , k + Nr − 1},

if γt = 0 and

As described above, if the proposed stochastic MPC algorithm is used, then what is transmitted depends on the dropout realisations. The number of values transmitted over a period of length Nr corresponding to a given optimizer in (17), say Ck , becomes a finite random process.

k ∈ K  {0, Nr , 2Nr , . . . }.

η,Θ

      0

if γt = 1

4.4 Properties

∀i ∈ {1, · · · , pN },

(η k , Θk ) = arg min V (π, xk ),

  ut ,       η  , k,

61

The optimization in (17) can be carried out by taking into account system dynamics, policy class and the effect of dropouts as in (18). Interestingly, despite the apparent complexity, it can be shown that the resulting programme is convex. Its parameters depend on the problem data (including disturbance statistics) and can be computed offline. In future work, we will investigate how to tune controller parameters P , Q, R, N and Nr such that, for Lyapunov stable plant models (e.g., featuring an orthogonal system matrix A), the system state xk is mean square bounded under the hard control constraints (12). 7

  Θk,,i ϕi wk+i ,

 = t − k ∈ {0, 1, 2, . . . , Nr − 1} and where η k, ∈ Rp denotes the -th block of η k and Θk,,i ∈ Rp×n is the (, i)-th block of Θk . To improve robustness against packet dropouts, akin to what is done in PPC, buffering at the actuator node is used. To be more specific, at each time t, the controller sends the current value ut . It also transmits the remaining Nr −  blocks of the current sequence η k , with

5. NUMERICAL STUDY Consider the two dimensional system     0.6 0 0.14 xk + u + wk , |uk | ≤ 10, xk+1 = 0 0.8 0.12 k where wk is i.i.d. Gaussian of mean zero with variance 4I2 .

k = Nr t/Nr ,

provided they have not already been received by the smart actuator.

We focus on a special case of the packet dropout model in Fig. 2: When the channel is in “good” state, packets are

Successfully received values ut are directly applied at the plant input. At instances where dropouts occur, the actuator uses the corresponding block of η k if available, or otherwise sets the plant input to zero. We thus have:

7

Note that by Chatterjee et al. (2012), mean square boundedness of the plant states is impossible to attain under bounded controls if the spectral radius of A is greater than 1.

61

2015 IFAC NMPC 62 September 17-20, 2015. Seville, Spain Daniel E. Quevedo et al. / IFAC-PapersOnLine 48-23 (2015) 057–064

14

Deterministic PPC Stochastic MPC

12

average norm x

average norm x

14

10 8 0

10

20

30

40

50

60

70

80

10

0

10

20

30

40

0

10

20

30

40

50

60

70

80

90

50 60 time steps

70

80

90

4 max control

max control

8

90

6

4

3

2

2 0

10

20

30

40

50 60 time steps

70

80

90

Fig. 4. Performance comparison with similar control effort.

Fig. 3. Performance comparison, PPC vs stochastic MPC.

cost for the deterministic PPC is 70.1112 and the average of total actuator energy is 104.8296. The performance in terms of the average norm of state and the expected total cost is observed to be approximately equal for both of the controllers but the stochastic controller has the added advantage of significantly less communication and the saving of actuator energy.

always successfully delivered, whereas in the “bad” state packets are always dropped, i.e., we set p1 = 1, p2 = 0. The transition probability to switch from a bad channel state to the good state is taken as p21 = 0.9; the failure rate is taken as p12 = 0.1. The stochastic controller proposed in Section 4 is designed with state and control weights Q = I2 , P = 1.5I2 , R = 0.95, optimization horizon N = 3 and recalculation interval Nr = 2. The disturbance feedback terms in the policy (16) are chosen as a vector of element-wise scalar sigmoidal functions ϕi (ξ) = (1 − e−ξ )/(1 + e−ξ ), which are applied to each coordinate of wk . The various covariance matrices that are required to solve the (off-line) optimization problem were computed empirically via classical Monte Carlo methods, see Robert and Casella (2013) using 106 i.i.d. samples. Computations for determining our policy were carried out in the MATLAB-based software package YALMIP, see L¨ ofberg (2004), and were solved using SDPT3-4.0, see Toh et al. (2006).

6. CONCLUSIONS Control with unreliable communications resources has received significant attention. Due to their versatility, MPC methods are good candidates to tackle a variety of design problems arising in this context. However, typical MPC formulations often adopt deterministic communication models. In contrast, communications theory and practice has shown that modeling communication links via stochastic processes is in general preferable. With the above as background, the present work has presented a stochastic MPC formulation for control when access to the actuator is affected by random dropouts. Simulation results suggest that, when comparing to earlier deterministic packetized predictive control, performance gains can be obtained with the proposed control algorithm.

For the PPC approach described in Section 3, the deterministic counterpart of the quadratic cost in (13) with parameters Q = I2 , P = 1.5I2 and N = 3 was used. 8 We consider two cases for the control weighting R = 0.95 (as in the stochastic controller case above) and also R = 1.65.

The current formulation is restricted to control with direct state feedback and where dropouts in the actuator channel are i.i.d. In future work, we will focus on overcoming these limitations. More general extensions of the ideas presented here may include multi-channel systems (see, e.g., K¨ogel and Findeisen (2013); Ljeˇsnjanin et al. (2014)), allowing for self-triggered or event-triggered operation (see, e.g., Donkers and Heemels (2012); Araujo et al. (2014); Nagahara et al. (2016); Quevedo et al. (2014a)) and distributed control; see, e.g., Maestre and Negenborn (2014). Furthermore, the challenging issue of satisfaction of state constraints might be an important issue to tackle.

Fig. 3 documents the simulation results averaged over 100 T sample paths with initial state x0 = [10 10] . In addition to the averaged Euclidean norm of the system states, the figure also illustrates the empirical bound on the absolute values of the control inputs observed. The expected total cost for the deterministic controller comes out to be 68.6168 units and that for the stochastic controller is 70.3898 units, which are roughly comparable. However, the deterministic controller transmits 300 control values through the channel while the stochastic controller needs to transmit on average only 150.67 control values, a significant reduction in communication. Further, the average of total actuator energy (as measured in uk 2 ) in a sample path for the deterministic and stochastic controllers are 267.8890 units and 99.8032 units, respectively.

REFERENCES Aguilera, R.P. and Quevedo, D.E. (2015). Predictive control of power converters: Designs with guaranteed performance. IEEE Trans. Ind. Inf., 11(1), 53–63. Antunes, D., Hespanha, J., and Silvestre, C. (2013). Stability of networked control systems with asynchronous renewal links: An impulsive systems approach. Automatica, 49(2), 402–413.

Fig. 4, illustrates results when PPC is designed with R = 1.65, leading to less control effort. Here, the expected total 8

Deterministic PPC (R=1.65) Stochastic MPC (R=0.95)

12

With PPC calculations are done at every time-step, thus, Nr = 1.

62

2015 IFAC NMPC September 17-20, 2015. Seville, Spain Daniel E. Quevedo et al. / IFAC-PapersOnLine 48-23 (2015) 057–064

Araujo, J., Mazo, M., Anta, A., Tabuada, P., and Johansson, K. (2014). System architectures, protocols and algorithms for aperiodic wireless control systems. IEEE Trans. Ind. Inf., 10(1), 175–184. Baddour, K.E. and Beaulieu, N.C. (2005). Autoregressive modeling for fading channel simulation. IEEE Trans. Wireless Commun., 4(4), 1650–1662. Batina, I., Stoorvogel, A.A., and Weiland, S. (2001). Stochastic disturbance rejection in model predictive control by randomized algorithms. In Proc. Amer. Contr. Conf., volume 2, 732–737. Bernardini, D. and Bemporad, A. (2009). Scenario-based model predictive control of stochastic constrained linear systems. In Proc. IEEE Conf. Decis. Contr., 6333–6338. Shanghai, P.R. China. Bertsekas, D.P. (2005). Dynamic Programming and Optimal Control, Vol.1. Athena Scientific, Belmont, MA. Bertsimas, D. and Brown, D.B. (2007). Constrained stochastic LQC: a tractable approach. IEEE Trans. Automat. Contr., 52(10), 1826–1841. Censi, A. (2011). Kalman filtering with intermittent observations: Convergence for semi-Markov chains and an intrinsic performance measure. IEEE Trans. Automat. Contr., 56(2), 376–381. Chaillet, A. and Bicchi, A. (2008). Delay compensation in packet-switching networked controlled systems. In Proc. IEEE Conf. Decis. Contr., 3620–3625. Canc´ un, M´exico. Chatterjee, D., Hokayem, P., and Lygeros, J. (2011). Stochastic receding horizon control with bounded control inputs: A vector space approach. IEEE Trans. Automat. Contr., 56(11), 2704–2710. Chatterjee, D., Ramponi, F., Hokayem, P., and Lygeros, J. (2012). On mean square boundedness of stochastic linear systems with bounded controls. Syst. & Contr. Lett., 61(2), 375–380. Chen, J., Johansson, K.H., Olariu, S., Paschalidis, I.C., and Stokmenovic (2011). Guest editorial special issue on wireless sensor and actuator networks. IEEE Trans. Automat. Contr., 56(11), 2244–2246. de la Pe˜ na, D.M., Bemporad, A., and Alamo, T. (2005). Stochastic programming applied to model predictive control. In Proc. IEEE Conf. Decis. Contr., 1361–1366. Donkers, M.C.F. and Heemels, M. (2012). Output-based event-triggered control with guaranteed L∞ -gain and improved and decentralized event-triggering. IEEE Trans. Automat. Contr., 57(6), 1362–1376. Donkers, M.C.F., Heemels, W.P.M.H., Bernardini, D., Bemporad, A., and Shneer, V. (2012). Stability analysis of stochastic networked control systems. Automatica, 48(5), 917–925. Felt, A.J. (2003). Stochastic linear model predictive control using nested decomposition. In Proc. Amer. Contr. Conf., volume 4, 3602–3607. Findeisen, R. and Varutti, P. (2009). Stabilizing nonlinear predictive control over nondeterministic networks. In L. Magni, D.M. Raimondo, and F. Allg¨ ower (eds.), Nonlinear Model Predictive Control: Towards New Challenging Applications, volume 384 of LNCIS, 167–179. Springer-Verlag, Berlin Heidelberg. Fischer, J. (2014). Optimal Sequence-Based Control of Networked Linear Systems. Ph.D. thesis, Department of Informatics, Karlsruhe Institute of Technology.

63

Goodwin, G.C., Østergaard, J., Quevedo, D.E., and Feuer, A. (2009). A vector quantization approach to scenario generation for stochastic NMPC. In L. Magni, D.M. Raimondo, and F. Allg¨ower (eds.), Nonlinear Model Predictive Control: Towards New Challenging Applications, volume 384 of LNCIS, 235–248. Springer-Verlag, Berlin Heidelberg. Gr¨ une, L. and Pannek, J. (2011). Nonlinear Model Predictive Control: Theory and Algorithms. Springer. Huang, M. and Dey, S. (2007). Stability of Kalman filtering with Markovian packet losses. Automatica, 43(4), 598– 607. Imer, O.C., Y¨ uksel, S., and Ba¸sar, T. (2006). Optimal control of LTI systems over unreliable communication links. Automatica, 42(9), 1429–1439. Johansson, K.H., Pappas, G.J., Tabuada, P., and Tomlin, C.J. (2014). Guest editorial special issue on control of cyber-physical systems. IEEE Trans. Automat. Contr., 59(12), 3120–3121. Kemeny, J.G. and Snell, J.L. (1960). Finite Markov Chains. D. Van Nostrand Company, Inc. Kim, K.D. and Kumar, P.R. (2012). Cyber–physical systems: A perspective at the centennial. Proc. IEEE, 100, 1287–1308. K¨ogel, M. and Findeisen, R. (2011). Robust suboptimal control over lossy networks using extended input schemes. In IEEE Intern. Symp. Computer-Aided Contr. Syst. Design (CACSD), 210–215. K¨ogel, M. and Findeisen, R. (2013). Distributed control of interconnected systems with lossy communication networks. In Proc. 4th IFAC Workshop on Distributed Estimation and Control in Networked Systems, 363–368. Koblenz, Germany. Kouvaritakis, B., Cannon, M., Rakovi´c, S.V., and Cheng, Q. (2010). Explicit use of probabilistic distributions in linear predictive control. Automatica, 46(10), 1719– 1724. Leizarowitz, A., Statonjevi´c, R., and Shorten, R. (2006). Tools for the analysis and design of communication networks with Markovian dynamics. IEE Proc.–Contr. Theory Appl., 153(5), 506–519. Liang, Y., Chen, T., and Pan, Q. (2010). Optimal linear state estimator with multiple packet dropouts. IEEE Trans. Automat. Contr., 55(6), 1428–1433. Lindbom, L., Ahl´en, A., Sternad, M., and Falkenstr¨ om, M. (2002). Tracking of time-varying mobile radio channels– part II: A case study. IEEE Trans. Commun., 50(1), 156–167. Ljeˇsnjanin, M., Quevedo, D.E., and Neˇsi´c, D. (2014). Packetized MPC with dynamic scheduling constraints and bounded packet dropouts. Automatica, 50(3), 784— 797. L¨ofberg, J. (2004). YALMIP: A toolbox for modeling and optimization in MATLAB. In Intern. Symp. Computer Aided Contr. Syst. Design (CACSD), 284–289. Lunze, J. (ed.) (2014). Control Theory of Digitally Networked Dynamic Systems. Springer Verlag. Maestre, J.M. and Negenborn, R.R. (eds.) (2014). Distributed Model Predictive Control Made Easy. Springer. Mayne, D. (2014). Model predictive control: Recent developments and future promise. Automatica, 50(12), 2967–2986.

63

2015 IFAC NMPC 64 September 17-20, 2015. Seville, Spain Daniel E. Quevedo et al. / IFAC-PapersOnLine 48-23 (2015) 057–064

1869–1872. Rawlings, J.B. and Mayne, D.Q. (2009). Model Predictive Control: Theory and Design. Nob Hill Publishing, Madison, WI. Reble, M., Quevedo, D.E., and Allg¨ower, F. (2013). Control over erasure channels: stochastic stability and performance of packetized unconstrained model predictive control. Int. J. Robust Nonlin. Contr., 23(10), 1151– 1167. Robert, C. and Casella, G. (2013). Monte Carlo statistical methods. Springer Science & Business Media. Schenato, L., Sinopoli, B., Franceschetti, M., Poolla, K., and Sastry, S.S. (2007). Foundations of control and estimation over lossy networks. Proc. IEEE, 95(1), 163– 187. Skaf, J. and Boyd, S. (2009). Nonlinear Q-design for convex stochastic control. IEEE Trans. Automat. Contr., 54(10), 2426–2430. Smith, S.C. and Seiler, P. (2003). Estimation with lossy measurements: Jump estimators for jump systems. IEEE Trans. Automat. Contr., 48(12), 2163–2171. Tabbara, M. and Neˇsi´c, D. (2008). Input-output stability of networked control systems with stochastic protocols and channels. IEEE Trans. Automat. Contr., 53(5), 1160–1175. Tang, P.L. and de Silva, C.W. (2007). Stability validation of a constrained model predictive networked control system with future input buffering. Int. J. Contr., 80(12), 1954–1970. Toh, K.C., Todd, M., and Tutuncu, R. (2006). On the implementation and usage of sdpt3–a matlab software package for semidefinite-quadratic-linear programming. Technical report, version 4.0. Technical report, Department of Mathematics, National University of Singapore. Tse, D. and Viswanath, P. (2005). Fundamentals of Wireless Communication. Cambridge University Press, UK. Tsumura, K., Ishii, H., and Hoshina, H. (2009). Tradeoffs between quantization and packet loss in networked control of linear systems. Automatica, 45, 2963–2970. van Hessem, D.H. and Bosgra, O.H. (2003). A full solution to the constrained stochastic closed-loop MPC problem via state and innovations feedback and its receding horizon implementation. In Proc. IEEE Conf. Decis. Contr., volume 1, 929–934. Wu, J. and Chen, T. (2007). Design of networked control systems with packet dropouts. IEEE Trans. Automat. Contr., 52(7), 1314–1319. Xie, L. and Xie, L. (2009). Stability analysis of networked sampled-data linear systems with Markovian packet losses. IEEE Trans. Automat. Contr., 54(6), 1375–1381. You, K., Li, Z., Quevedo, D.E., and Lewis, F.L. (2014). Editorial: Recent developments in networked control and estimation. IET Control Theory Appl., 8(18), 2123– 2125. You, K. and Xie, L. (2011). Minimum data rate for mean square stabilizability of linear systems with Markovian packet losses. IEEE Trans. Automat. Contr., 56(4), 772– 785. Yu, J. and Fu, L. (2016). An optimal compensation framework for linear quadratic Gaussian control over lossy networks. IEEE Trans. Automat. Contr. To be published.

Mesbah, A., Streif, S., Findeisen, R., and Braatz, R.D. (2014). Stochastic nonlinear model predictive control with probabilistic constraints. In Proc. Amer. Contr. Conf., 2413–2419. Portland, Oregon. Mu˜ noz de la Pe˜ na, D. and Christofides, P.D. (2008). Lyapunov-based model predictive control of nonlinear systems subject to data losses. IEEE Trans. Automat. Contr., 53(9), 2076–2089. Nagahara, M., Quevedo, D.E., and Neˇsi´c, D. (2016). Maximum hands-off control: A paradigm of control effort minimization. IEEE Trans. Automat. Contr., 61(5). http://arxiv.org/pdf/1408.3025.pdf. Nagahara, M., Quevedo, D.E., and Østergaard, J. (2014). Sparse packetized predictive control for networked control over erasure channels. IEEE Trans. Automat. Contr., 59(7), 1899–1905. Nourian, M., Leong, A., Dey, S., and Quevedo, D.E. (2014). An optimal transmission strategy for Kalman filtering over packet dropping links with imperfect acknowledgements. IEEE Trans. Contr. Network Syst., 1(3), 259–271. Paulraj, A., Nabar, R., and Gore, D. (2003). Introduction to Space-Time Wireless Communications. Cambridge University Press. Pin, G. and Parisini, T. (2009). Stabilization of networked control systems by nonlinear model predictive control: A set invariance approach. In L. Magni, D.M. Raimondo, and F. Allg¨ ower (eds.), Nonlinear Model Predictive Control: Towards New Challenging Applications, volume 384 of LNCIS, 195–204. Springer-Verlag, Berlin Heidelberg. Primbs, J.A. and Sung, C.H. (2009). Stochastic receding horizon control of constrained linear systems with state and control multiplicative noise. IEEE Trans. Automat. Contr., 54(2), 221–230. Quevedo, D.E., Ahl´en, A., and Johansson, K.H. (2013a). State estimation over sensor networks with correlated wireless fading channels. IEEE Trans. Automat. Contr., 58(3), 581–593. Quevedo, D.E., Gupta, V., Ma, W., and Y¨ uksel, S. (2014a). Stochastic stability of event-triggered anytime control. IEEE Trans. Automat. Contr., 59(12), 3373– 3379. Quevedo, D.E. and Neˇsi´c, D. (2011). Input-to-state stability of packetized predictive control over unreliable networks affected by packet-dropouts. IEEE Trans. Automat. Contr., 56(2), 370–375. Quevedo, D.E. and Neˇsi´c, D. (2012). Robust stability of packetized predictive control of nonlinear systems with disturbances and Markovian packet losses. Automatica, 48(8), 1803–1811. Quevedo, D.E., Østergaard, J., and Ahl´en, A. (2014b). Power control and coding formulation for state estimation with wireless sensors. IEEE Trans. Contr. Syst. Technol., 22(2), 413–427. Quevedo, D.E., Østergaard, J., and Neˇsi´c, D. (2011). Packetized predictive control of stochastic systems over bit-rate limited channels with packet loss. IEEE Trans. Automat. Contr., 56(12), 2854–2868. Quevedo, D.E., Østergaard, J., Silva, E.I., and Neˇsi´c, D. (2013b). Correction to ”packetized predictive control of stochastic systems over bit-rate limited channels with packet loss”. IEEE Trans. Automat. Contr., 58(7), 64