Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems

Annual Reviews in Control xxx (2013) xxx–xxx

Contents lists available at SciVerse ScienceDirect

Annual Reviews in Control journal homepage: www.elsevier.com/locate/arcontrol

Full Length Article

Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems q Andrew R. Teel ⇑ Electrical and Computer Engineering Department, University of California, Santa Barbara, CA 93106-9560, United States

a r t i c l e

i n f o

Article history: Received 3 August 2012 Accepted 16 February 2013 Available online xxxx

a b s t r a c t Lyapunov-based conditions for stability and recurrence are presented for a class of stochastic hybrid systems where solutions are not necessarily unique, either due to nontrivial overlap of the flow and jump sets, a set-valued jump map, or a set-valued flow map. Randomness enters exclusively through the jump map, yet the framework covers systems with spontaneous transitions. Regularity conditions are given that guarantee the existence of random solutions and robustness of the Lyapunov conditions. Ó 2013 Elsevier Ltd. All rights reserved.

1. Introduction Hybrid dynamical systems combine continuous change and instantaneous change. Early references on non-stochastic hybrid systems include (Witsenhausen, 1966; Tavernini, 1987). Hybrid systems have been studied extensively for over twenty years and the literature contains several books on this and related topics: (Lakshmikantham et al., 1989; van der Schaft and Schumacher, 2000; Yang, 2001; Liberzon, 2003; Haddad et al., 2006; Goebel et al., 2012). In the control field, hybrid systems can be used to develop feedback algorithms for a broad class of systems including mechanical systems with impacts. Moreover, some classical feedback control problems, like global asymptotic stabilization of a point on a compact manifold, cannot be solved robustly without using hybrid compensation. Stochastic hybrid systems include the effects of random inputs. Many classes of stochastic hybrid systems have been studied in the literature, mainly differing in where randomness enters the model, the extent to which the random effects depend on the state of the system, and the types of jumps that may occur. One class of such systems is called hybrid switching diffusions, for which the literature is extensive; see (Mariton, 1990; Ghosh et al., 1993; Hanson, 2007; Yin and Zhu, 2010) for example. In most hybrid switching diffusion models, randomness enters through white noise in stochastic differential equations and in state-dependent rates that determine spontaneous transitions, although the continuous-valued state usually does not jump. An alternative to switching diffusions are the piecewise deterministic Markov processes of Davis (1984, 1993) where the differential equations are not stochastic but random jumps of the state occur either at random times or

q Research supported in part by AFOSR FA9550-12-1-0127, and NSF ECCS0925637 and ECCS-1232035. ⇑ Tel.: +1 805 893 3616; fax: +1 805 893 3262. E-mail address: [email protected]

when the state exits an open domain. Closely related classes of stochastic hybrid systems are studied in Hespanha (2005) and Jacobsen (2006). The models of Hu et al. (2000) include stochastic differential equations and random jumps determined by non-stochastic guard conditions but do not explicitly include spontaneous transitions. Some of the models mentioned above are compared in Pola et al. (2003). General stochastic hybrid systems appear in the context of discrete event systems (Cassandras and Lafortune, 2008), and in the recent monograph (Bujorianu, 2012). In most of the models mentioned here, the solutions are assumed to be unique. An exception is (Bujorianu et al., 2012), which proposes modeling tools for uncertain hybrid systems using different techniques than are offered here. We consider a class of stochastic hybrid systems building on the non-stochastic hybrid systems in Goebel and Teel (2006), Goebel et al. (2009), and Goebel et al. (2012), which specify a flow set where continuous evolution may occur, a flow map that determines how continuous evolution may occur, a jump set where instantaneous change may occur, and a jump map that determines how instantaneous change may occur. The solutions are not necessarily unique, allowing for an adversary who affects solutions. We consider a stochastic extension where the random variables appear only in the jump map, but not explicitly in the jump set, the flow map, or the flow set. Despite this restriction, we demonstrate how randomness in the flow set and jump set is handled by adding an extra state variable. This idea permits modeling systems with spontaneous transitions. Due to space constraints and the extra machinery required to rigorously address white noise in the flow map, we do not address that extension here. Instead, we focus on the intriguing mathematical issues that arise due to the possible interplay between worst-case and stochastic effects. In addition to highlighting the interesting, perhaps counterintuitive, interaction of worst-case and stochastic effects, this paper has two main contributions. The first, less glamorous, contribution is to propose a simple but quite general class of stochastic hybrid

1367-5788/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

2

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

systems and to provide mild regularity conditions on the data of these systems to guarantee existence of nontrivial solutions and robustness of stability properties. The second contribution, which is nontrivial mainly because of the allowed interaction between stochastic and worst-case effects, involves supplying Lyapunovbased sufficient conditions for uniform global asymptotic stability in probability of closed sets and uniform global recurrence of open sets. When specialized to non-stochastic systems and stability of equilibria, these Lyapunov conditions match classical conditions (see Khalil, 2002, Chapter 4 for systems with unique solutions or the natural extension to non-uniqueness, set stability, and nonstochastic hybrid systems in Goebel et al. (2012, Chapter 3) for example) or Matrosov conditions (Matrosov, 1963; Loria et al., 2005; Teel et al., 2010; Sanfelice and Teel, 2009). Moreover, in the case of non-stochastic hybrid systems (Cai et al., 2008) as well as stochastic discrete-time systems with non-unique solutions (Teel et al., submitted for publication; Subbaraman et al., submitted for publication), the Lyapunov conditions we use are necessary and sufficient for asymptotic stability (in probability) and recurrence. When applied to systems with spontaneous transitions our Lyapunov conditions constitute a departure from conditions on infinitesimal generators as considered in Davis (1993), Ghosh et al. (1993), Hespanha (2005), Jacobsen (2006), and Yin and Zhu (2010), for example. Indeed, the derivative of the expected value plays no significant role in our analysis. To make this departure compelling, we connect our Lyapunov conditions to infinitesimal generator conditions for systems like those studied in Davis (1993). There are other forms of stability, like mean-square stability, that we do not pursue here but that can be addressed in the same framework. The paper is organized as follows: Section 2 describes a class of stochastic hybrid systems, characterizes random solutions, and defines stability concepts. Section 3 contains regularity assumptions and a result on existence of solutions. The core Lyapunov results are summarized in Section 4, with additional special cases addressed in Section 5, including an extension of Matrosov functions to stochastic hybrid systems. Section 6 contains three examples: a stochastic bouncing ball, notable for its Zeno solutions which are usually precluded in other work, and two examples, including one with spontaneous transitions, that illustrate the role of causality and how it affects the validity of the proposed Lyapunov conditions. Section 7 covers a general class of systems with spontaneous transitions, including models like in Davis (1993). Longer proofs of main results are deferred to Sections 8–10. Basic notation and definitions as well as notation and concepts from set-valued analysis, which play a prominent role in our analysis, are summarized in Appendix A. 2. Systems, solutions and stability definitions This section introduces the class of stochastic hybrid systems that we study, the definition of a solution to these systems, and the stability concepts that are considered for these solutions. 2.1. Stochastic hybrid systems Consider a stochastic hybrid system written formally as

x2C

x_ 2 FðxÞ

ð1aÞ

x 2 D x 2 Gðx; v Þ þ

þ

ð1bÞ

lðÞ

ð1cÞ n

n

n

where C  R is the flow set, F : R  R is the flow map, D  Rn is the jump set, and G : Rn  Rm  Rn is the jump map. The distribution function l is derived from a probability space ðX; F ; PÞ and a sequence of independent, identically distributed (i.i.d.) input

random variables defined on ðX; F ; PÞ. Namely, with v i : X ! Rm ; i 2 ZP1 , denoting the elements of the sequence, and thus having the property that Pfx 2 X : v i ðxÞ 2 Ag is well defined and independent of i for each A in the Borel r-field over Rm , denoted BðRm Þ, the distribution function l : BðRm Þ ! ½0; 1 is defined as lðAÞ :¼ Pfx 2 X : v i ðxÞ 2 Ag; see (Fristedt and Gray, 1997, Sections 2.1 and 11.1). We use F i to denote the collection of sets fx 2 X : ðv 1 ðxÞ; . . . ; vi ðxÞÞ 2 Ag; A 2 BððRm Þi Þ, which are the sub-r-fields of F that form the minimal filtration (Fristedt and Gray, 1997, Section 11.3, Def. 4) of v :¼ fv i g1 i¼1 . We sometimes denote (1) through its data (C, F, D, G, l). In contrast to most stochastic hybrid models, uniqueness of solutions is not assumed. In fact, because the flow and jump sets may have nontrivial overlap, the flow map may be set valued, and the jump map may be set valued, uniqueness of solutions in not typical in these models. In later sections, we elaborate on the flexibility that the model (1) confers. For example, in Section 3 we discuss state-dependent distribution functions and in Section 7 we describe models with spontaneous transitions, thereby allowing some randomness in the flow set C and the jump set D.

2.2. Random solutions Random solutions to (1), which are functions of x 2 X denoted x(x), have two distinguishing features. First, for each x 2 X, the sample path x(x), appropriately coupled with the random input sequence v(x), must be a standard solution; we adopt the solution concept of Goebel and Teel (2006), Cai and Teel (2009). Secondly, x ´ x(x) must have measurability properties that are adapted to the minimal filtration of v. This measurability is used to express stability and recurrence in terms of probabilities and to enforce causal dependence on v. We adopt measurability concepts from set-valued analysis, as reviewed in Appendix A; causality is related to non-anticipative control laws in stochastic control. For example, see (Rockafellar and Wets, 1974), the references therein, and (Davis, 1984; Davis, 1993). The definition of a random solution is now made precise. We begin by defining a standard solution. A compact hybrid time domain is a subset of RP0  ZP0 of the form [Ji¼0 ð½ti ; t iþ1   figÞ for some J 2 ZP0 and numbers 0 = t0 6 t1 6 . . . 6 tJ+1. A hybrid time domain is a set E  RP0  ZP0 with the property that for each (T, J) 2 E, E \ ([0,T]  {0, . . . , J}) is a compact hybrid time domain. A hybrid arc is a mapping / : dom / ! Rn , where dom / is a hybrid time domain and t ´ /(t, j) is locally absolutely continuous for each j 2 ZP0 . The graph of a hybrid arc / is defined as graphð/Þ :¼ fðt; j; xÞ 2 Rnþ2 : ðt; jÞ 2 dom /; x ¼ /ðt; jÞg. Let x 2 C [ D. A pair of hybrid arcs (/, m) with dom / = dom m is said to be a standard solution of (1a), (1b) starting at x if /(0, 0) = x and the following conditions hold: _ jÞ 2 Fð/ðt; jÞÞ for each j 2 ZP0 and almost all 1. /(t, j) 2 C and /ðt; t 2 Ij :¼ {s:(s, j) 2 dom /}; 2. /(t, j) 2 D and /(t, j + 1) 2 G(/(t, j), m(t, j + 1)) for all (t, j) 2 dom / such that (t, j + 1) 2 dom /. The first condition says that if it is possible to move horizontally in the time domain then the solution belongs the flow set and its derivative belongs to the flow map evaluated at the current value of the solution. For j 2 ZP0 such that Ij is empty or a single point, there is nothing to check in the first condition. The second condition says that if it is possible to move vertically in the time domain then the solution belongs to D and its next value belongs to the jump map evaluated at the current value of the solution and the next value of the input. Now we define a random solution to (1). A mapping x from X to the set of hybrid arcs is a random solution of (1) starting at x, denoted x 2 S r ðxÞ, if the following two properties hold:

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

3

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

1. (Pathwise feasibility) For each x 2 X, the pair (xx, ux) with xx :¼ x(x) and ux a hybrid arc with dom ux = dom xx and ux(t, j) :¼ vj(x) for all ðt; jÞ 2 dom xðxÞ \ ðR  ZP1 Þ, is a standard solution of (1a), (1b) starting at x. 2. (Causal measurability) For each i 2 ZP0 , the mapping x # graph6i ðxðxÞÞ :¼ graphðxðxÞÞ \ ðRP0  Z6i  Rn Þ has nonempty closed values and is F i -measurable with F 0 ¼ f£; Xg, and ðF 1 ; F 2 ; . . .Þ the minimal filtration of v. The F i -measurability of graph6i(x) confers F -measurability on graph(x), due to Rockafellar and Wets (1998, Prop. 14.11(b)), as well as F i -measurability on several other mappings derived from graph6i(x) as described in the next proposition, which uses the sets CPs :¼ fðs; tÞ 2 R2 : s þ t P sg; C6s :¼ R2 n CPs , and the mappings pi : Rnþ2 ! R; i 2 f1; 2g, given by pi(t1, t2, z) :¼ ti. Proposition 2.1. Let C  Rn be closed, O  Rn be open, ðs; xÞ 2 RP0  ðC [ DÞ, and x 2 S r ðxÞ. For each i 2 ZP0 , the following mappings are F i -measurable:

x # Ti;O ðxÞ :¼ infft 2 p1 ðgraph6i ðxðxÞÞ \ ðR2  OÞÞg

ð2aÞ

x # Ji;O ðxÞ :¼ inffj 2 p2 ðgraph6i ðxðxÞÞ \ ðR2  OÞÞg

ð2bÞ

x # xx ðTi;O ðxÞ; Ji;O ðxÞÞ 8x 2 dom Ti;O ¼ dom Ji;O

ð2cÞ

of the ith or later random input. Examples in Sections 6.2 and 6.3 highlight the role that causality plays in ensuring the validity of Lyapunov conditions for stability and recurrence. 2.3. Stability properties We use stochastic extensions of classical Lyapunov stability, Lagrange stability, attractivity, and recurrence definitions. A comparison of stochastic stability definitions and their connection to nonstochastic stability is found in Kozin (1969). We study properties for sets rather than equilibria since it is common in hybrid systems for the steady-state behavior to not be constant. See (Goebel et al., 2009, p. 58). The stability concepts are expressed in terms of probabilities on solution graphs. To save on notation, we suppress the x dependence of a random solution x when working with probabilities. Moreover, by abuse of notation, we write ‘‘x(t, j) 2 S for (t, j) 2 dom x’’ in place of ‘‘xx(t, j) 2 S for (t, j) 2 dom xx’’ where xx :¼ x(x). Let a closed set A  Rn and a function - : C [ D ! RP0 be given. Frequently -ðxÞ ¼ jxjA , though in some examples we set (x) = 1 for some x 2 C [ D. The set A is uniformly Lyapunov stable in probability relative to for (1) if there exists e⁄ > 0 such that the system x 2 C \ ðA þ e BÞ; x_ 2 FðxÞ has no finite escape times and for each e > 0 and . > 0 there exists d > 0 such that

-ðnÞ 6 d; x 2 S r ðnÞ

) 2

x # graph6i ðxðxÞÞ \ ð½0; Ti;O ðxÞ  f0; . . . ; Ji;O ðxÞg  CÞ

ð2dÞ

PðgraphðxÞ  ðR  ðA þ eB ÞÞÞ P 1  . 2

x # Ti;C ðxÞ :¼ infft 2 p1 ðgraph6i ðxðxÞÞ \ ðCPs  CÞÞg

ð2eÞ

x # Ji;C ðxÞ :¼ inffj 2 p2 ðgraph6i ðxðxÞÞ \ ðCPs  CÞÞg

ð2fÞ

x # xx ðTi;C ðxÞ; Ji;C ðxÞÞ 8x 2 dom Ti;C ¼ dom Ji;C

ð2gÞ

x # graph6i ðxðxÞÞ \ ð½0; Ti;C ðxÞ  f0; . . . ; Ji;C ðxÞg  CÞ

ð2hÞ

x # Ti;s ðxÞ :¼ supft 2 p1 ðgraph6i ðxðxÞÞ \ ðC6s  Rn ÞÞg

ð2iÞ

x # Ji;s ðxÞ :¼ supfj 2 p2 ðgraph6i ðxðxÞÞ \ ðC6s  Rn ÞÞg

ð2jÞ

x # xx ðTi;s ðxÞ; Ji;s ðxÞÞ 8x 2 X

ð2kÞ

x # tiþ1;s ðxÞ :¼ supft : ðt; iÞ 2 dom xðxÞ \ C6s g

ð2lÞ

x # xx ðtiþ1;s ðxÞ; iÞ

8x 2 dom tiþ1;s

ð2mÞ

Proof. Measurability follows from results in Rockafellar and Wets (1998) as follows:(2a) and (2b): (Rockafellar and Wets, 1998, Def. 14.1, Thm. 14.13(a), Prop. 14.2, Ex. 14.30, Ex. 14.32, Thm. 14.37); (2e) and (2f): (Rockafellar and Wets, 1998, Thm. 14.13(a), Ex. 14.30, Ex. 14.32, Thm. 14.37); (2c), (2g), (2k), and (2m): note

xx ðTi ðxÞ; Ji ðxÞÞ ¼ p3 ðgraph6i ðxðxÞÞ \ ðfTi ðxÞg  fJi ðxÞg  Rn ÞÞ where p3(t1, t2, z) = z and then use (Rockafellar and Wets, 1998, Prop. 14.11(a,d), Thm. 14.13(a)); (2d) and (2h): (Rockafellar and Wets, 1998, Prop. 14.33, Prop. 14.11(d,a)); (2i), (2j), (2l): (Rockafellar and Wets, 1998, Prop. 14.11(a), Thm. 14.13(a), Ex. 14.30, Ex. 14.32, Thm. 14.37). h The F i -measurability of the mappings in Proposition 2.1 implies that they can be expressed as measurable functions of (v1(x), . . . , vi(x)); see (Fristedt and Gray, 1997, Lemma 7, p. 411). In particular, this causality prevents the timing of the ith jump or the value of the state up to the ith jump from anticipating the value

ð3Þ



The condition graphðxÞ  ðR  ðA þ eB ÞÞ is equivalent to having xðt; jÞ 2 A þ eB for all (t, j) 2 dom x. The measurability of this condition follows from Rockafellar and Wets (1998, Theorem 14.3(h)). The inequality (3) asks that this condition on x holds with probability at least 1  .. The value d > 0 is chosen sufficiently small to accommodate e > 0 and . > 0. The set A is uniformly Lagrange stable in probability relative to for (1) if there are no finite escape times for (1a) and for each d > 0 and . > 0 there exists e > 0 such that (3) holds. Here e > 0 is chosen large to accommodate d > 0 and . > 0. The set A is uniformly globally stable in probability relative to for (1) if it is both uniformly Lyapunov stable and uniformly Lagrange stable in probability relative to - for (1). The set A is uniformly globally attractive in probability relative to - for (1) if there are no finite escape times for (1a) and for each e > 0, . > 0, and R > 0 there exists s P 0 so that

-ðnÞ 6 R; x 2 S r ðnÞ ) PððgraphðxÞ \ ðCPs  Rn ÞÞ  ðR2  ðA þ eB ÞÞÞ P 1  .

ð4Þ

By convention, the empty set is a subset of any set. The condition ðgraphðxÞ \ ðCPs  Rn ÞÞ  ðR2  ðA þ eB ÞÞ asks that xðt; jÞ 2 A þ eB for all (t, j) 2 dom x satisfying t + j P s. Measurability of this condition follows from Rockafellar and Wets (1998, Prop. 14.11(a), Thm. 14.3(h)). The inequality (4) asks that this condition on x holds with probability at least 1  .. Here s is chosen large to accommodate e > 0,. > 0, and R > 0. The set A  Rn is uniformly globally asymptotically stable (UGAS) in probability relative to - for (1) if it is uniformly globally stable in probability relative to - for (1) and uniformly globally attractive in probability relative to - for (1). We typically drop the phrase ‘‘relative to -’’ in the definitions above when -ðnÞ ¼ jnjA for all n 2 C [ D. 2.4. Uniform global recurrence In addition to asymptotic stability in probability of a closed set, recurrence of an open set is a property that is often studied for stochastic systems. For example, recurrence is one of the primary

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

4

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

topics of Meyn and Tweedie (1993) and it also receives considerable attention in Yin and Zhu (2010). Recurrence of an open set corresponds to the situation where the probability of reaching that set approaches one as time grows unbounded. Since we do not necessarily assume that the open set is bounded, we ask for some uniformity in probability over solutions from certain sets of initial conditions. The open set O  Rn is uniformly globally recurrent relative to for (1) if there are no finite escape times for (1a) and for each . > 0 and R > 0 there exists s P 0 such that

-ðnÞ 6 R; x 2 S r ðnÞ ) PððgraphðxÞ  ðC
ð5Þ

X\ :¼ fx 2 X : graphðxðxÞÞ \ ðC6s  Od Þ – £g Xa :¼ fx 2 X : graphðxðxÞÞ  ðC
PðX Þ P PðX \ ðXa [ X\ ÞÞ ¼ PðXa Þ þ PðX \ X\ Þ P 1  .r  PðX\ Þ þ PðX \ X\ Þ

We establish below that PðX \ X\ Þ P ð1  .s ÞPðX\ Þ. This bound and (6), PðX\ Þ 2 ½0; 1, and .r, .s 2 (0, ./2], give

where _ is the logical ‘‘or’’, C
PðX Þ P 1  .r  PðX\ Þ þ ð1  .s ÞPðX\ Þ ¼ 1  .r  .s PðX\ Þ

ðgraphðxÞ  ðC
se Observe that X\ ¼ [di¼0 Si , where

is equivalent to the condition that either t + j < s for all (t, j) 2 dom x or else there exists (t, j) 2 dom x such that t + j 6 s and xðt; jÞ 2 O. According to Rockafellar and Wets (1998, Thm. 14.3(a),(h); Prop. 14.11(a)) the recurrence condition is measurable. The inequality (5) asks that the recurrence condition holds with probability at least 1  .. Hence, the probability of reaching O within hybrid time s should be close to one, without penalizing solutions that stop before hybrid time s, either because they cannot be continued or simply because we have not insisted that solutions are maximal. The condition (5) can be checked only for x 2 ðRn n OÞ \ ðC [ DÞ since for x 2 O \ ðC [ DÞ the recurrence condition holds for all x 2 S r ðxÞ and all x 2 X. We typically drop the phrase ‘‘relative to -’’ when -ðnÞ ¼ jnjO for all n 2 C [ D. We can also consider uniform global recurrence of sets that are open relative to C [ D. A set O is open relative to C [ D (or simply e such that relatively open) if there exists an open set O e \ ðC [ DÞ. Uniform global recurrence of the relaO \ ðC [ DÞ ¼ O e \ ðC [ DÞ. The tively open set O entails replacing O in (5) with O resulting recurrence condition is measurable according to Rockafellar and Wets (1998, Thm. 14.3(a),(h); Prop. 14.11(a)). Recurrence e \ ðC [ DÞ is equivalent to that for O. e Indeed, recurrence of of O e \ ðC [ DÞ implies recurrence of O e since O e \ ðC [ DÞ  O. e ConO versely, by the definition of solutions, for each s > 0, the existence e implies that either of (t, j) 2 dom x such that t + j 6 s and xðt; jÞ 2 O e \ ðC [ DÞ or else s + i < s + 1 for all (s, i) 2 dom x. xðt; jÞ 2 O Sometimes uniform global recurrence is of independent interest. Other times it is considered as a means of establishing uniform global asymptotic stability in probability, as in Khasminskii et al. (1980) and Yin and Zhu (2010) for example, aided by the following proposition.

Si :¼ fx 2 X : Ji;Od ðxÞ ¼ i; Ti;Od ðxÞ þ Ji;Od ðxÞ 6 sg

Proposition 2.2 (Stability plus recurrence implies UGAS). Let C [ D be closed, -1 : C [ D ! RP0 be lower semicontinuous, -2 : C [ D ! RP0 , and -3(x) :¼ max {-1(x), -2(x)} for each x 2 C [ D. If the closed set A  Rn is uniformly globally stable in probability relative to -1 for (1) and for each d > 0 there exists a relatively open subset of {n 2 C [ D:-1(n) < d}, that is uniformly globally recurrent relative to -2 for (1) then A is uniformly globally asymptotically stable in probability relative to -3 for (1). Proof. It is enough to show that A is uniformly globally attractive in probability relative to -2 for (1). Due to uniform global recur~ > 0, and rence, there are no finite escapes for (1a). Let e > 0; . R > 0 be given. Let d > 0 be such that (3) holds with . ¼ .s 2 ð0; .~ =2 and - = -1, and so that fn 2 C [ D : -1 ðnÞ < dg  A þ ~eB for some ~e 2 ð0; eÞ. Let s P 0 satisfy (5) with - ¼ -2 ; . ¼ .r 2 ð0; .~ =2 and O ¼ Od , where Od  A þ ~eB and Od \ ðC [ DÞ  fn 2 C [ D : -1 ðnÞ < dg. We claim that (4) holds ~ . Let n 2 C [ D satisfy -2(n) 6 R and let with - = -2 and . ¼ . x 2 S r ðnÞ. Define

ð6Þ

P1.

for all i 2 {0, . . . , dse} and where Ti;Od and Ji;Od were defined in (2a), (2b). Since the sets Si are disjoint, it is enough to show

PðX \ Si Þ P ð1  .s ÞPðSi Þ 8i 2 f0; . . . ; dseg

ð7Þ

Let i 2 {0, . . . , dse} and pi;Od be the mapping in (2c). Using that C [ D is closed, -1 is lower semicontinuous, which implies that its sublevel sets are closed (Rockafellar and Wets, 1998, Thm. 1.6), t ´ xx(t, j) is continuous, and the properties of ~e and Od , for all x 2 Si,

pi;Od ðxÞ 2 fn 2 C [ D : -1 ðnÞ 6 dg [ ½ðA þ eB Þ n ðC [ DÞ

ð8Þ

For each x 2 Si, let zi(x) be the hybrid arc satisfying

graphðzi ðxÞÞ ¼ graphðxðxÞÞ  ðTi;Od ðxÞ; Ji;Od ðxÞ; 0Þ

ð9Þ

The resulting set-valued mapping is measurable due to Rockafellar and Wets (1998, Prop. 14.11(a), (c), and (d)). Also, for all x 2 Si

IX ðxÞ P IðCP0 ðAþeB ÞÞ ðgraphðzi ðxÞÞÞ

ð10Þ

Using Proposition 2.1 and (Fristedt and Gray, 1997, Lemma 7, p. 411), there exist measurable functions uT, uJ, ux, such that Ti;Od ðxÞ ¼uT ðv 1 ðxÞ; . . . ; v i ðxÞÞ; Ji;Od ðxÞ¼uJ ðv 1 ðxÞ; . . . ; v i ðxÞÞ, and pi;Od ðxÞ ¼ ux ðv 1 ðxÞ; . . . ; v i ðxÞÞ for all x 2 dom Ti;Od ¼ dom Ji;Od Si , Then, with (v1(x), . . . ,vi(x)) fixed such that Ji;Od ðxÞ ¼ i and Ti;Od ðxÞ þ Ji;Od ðxÞ 6 s, the mapping zi is a random solution with inputs (vi+1, vi+2, . . .) starting at pi;Od ðxÞ. Since the sequence v is iid, the statistics of zi are unaffected by the shifted in the inputs; cf. (Fristedt and Gray, 1997, Section 22.2). Using (10) and (Fristedt and Gray, 1997, Prop. 5, Section 23.3), we conclude that, for almost all x 2 Si

PðX jF i ÞðxÞ P E½ICP0 ðAþeB Þ ðgraphðzi ÞÞjF i ðxÞ P

inf

y2S r ðux ðv 1 ðxÞ;...;v i ðxÞÞÞ

PðgraphðyÞ  ðR2  ðA þ eB ÞÞÞ

P 1  .s Therefore, according to Fristedt and Gray (1997, Section 21.1, Prop. 2), it follows that PðX \ Si Þ ¼ E½PðX jF i ÞISi  P ð1  .s ÞPðSi Þ which is (7). h The next two results are useful for establishing uniform global recurrence. The first result says that uniform global recurrence of O for (C, F, D, G, l) is equivalent to uniform global recurrence of O for the system with truncated data (C\, F, D\, G\, l) where

C \ :¼ C \ ðRn n OÞ;

D\ :¼ D \ ðRn n OÞ;

G\ ðx; v Þ :¼ Gðx; v Þ \ ðRn n OÞ

ð11Þ

which is useful when it is easier to establish uniform global recurrence for the latter data.

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

Proposition 2.3 (Recurrence from recurrence for truncated data). Let - : C [ D ! RP0 and suppose there are no finite escape times for (1a). The open set O  Rn is uniformly globally recurrent relative to - for (C, F, D, G, l) if and only if it is uniformly globally recurrent relative to - for (C\, F, D\, G\, l). Proof. ð)Þ If x is a random solution for (C\, F, D\, G\, l) then it is also a random solution for (C, F, D, G, l), since the latter imposes less stringent constraints. (Ü) Let x be a random solution for (C, F, D, G, l) starting at x 2 C [ D. Let y be such that, for each x 2 X, its graph equals

graphðxðxÞÞ \ ð½0; TO ðxÞ  f0; . . . ; JO ðxÞg  ðRn n OÞÞ where TO and JO were defined in (2a), (2b). By construction, y is a random solution to (C\, F, D\, G\, l) starting at x, graphðyÞ \ ðC6s  OÞ ¼ £ and

ðgraphðxÞ  ðC 0. This observation is especially convee D is compact for each D > 0, as in nient when the complement of O this case extra results are available for establishing recurrence; see e D ¼ ðOþ Section 5. In particular, if O is bounded then Rn n O DBÞ n O is compact for each D > 0. As alluded to above, even though the next result is stated for a general system (C, F, D, G, l), it is usually applied to the system (C\, F, D\, G\, l) since uniform Lagrange stability in probability of O often holds for (C\, F, D\, G\, l) while not holding for (C, F, D, G, l). In fact, the result is stated in slightly more general terms than described above by assuming Lagrange stability of a general closed set A that is not necessarily equal to O.

5

2  . 6 p\ðO[Oe Þ ðs; xÞ þ pAþeB ðxÞ 6 p\O ðs; xÞ þ p\Ce ðxÞ þ pAþeB ðxÞ ¼ p\O ðs; xÞ þ 1: Subtracting one from both sides establishes uniform global recurrence of O. h 3. Basic conditions and existence of solutions 3.1. Stochastic hybrid basic conditions Regularity conditions are used to establish existence of random solutions and to guarantee that integrals appearing in the study of the system (1) are well defined. The conditions are also used to establish that Lyapunov conditions for uniform global asymptotic stability in probability are robust, and that global recurrence is equivalent to uniform global recurrence from compact sets. The first set of regularity assumptions are inherited from non-stochastic hybrid systems, as in Goebel and Teel (2006). Outer semicontinuity and local boundedness are defined in Appendix A. Assumption 1 (Hybrid Basic Conditions). 1. The sets C  Rn and D  Rn are closed. 2. The set-valued mapping F : Rn  Rn is outer semicontinuous, locally bounded, and for each x 2 C the value F(x) is nonempty and convex. 3. The set-valued mapping G : Rn  Rm  Rn is locally bounded and, for each v 2 Rm ; x # Gðx; v Þ is outer semicontinuous. G(x, v) may be empty for some x 2 D. With V :¼ [x2X;i2ZP0 v iþ1 ðxÞ, solutions can take values in the set C [ D [ GðD  VÞ. This set is equal to C [ D when GðD  VÞ  C [ D. We also impose a condition on how G depends on v: Assumption 2 (Stochastic Hybrid Basic Condition). The set-valued mapping v ´ graph(G(, v)) is measurable, where graphðGð; v ÞÞ :¼ fðx; yÞ 2 Rn  Rn : y 2 Gðx; v Þg.

ð12Þ

Assumption 2 has been exploited recently for set-valued stochastic discrete-time systems in Teel et al. (submitted for publication), Subbaraman et al. (submitted for publication), Teel (submitted for publication(a)), and Teel (submitted for publication(b)). It guarantees that v ´ G(w(v), v) is a measurable set-valued mapping when w is a measurable function. See (Rockafellar and Wets, 1998, Thm. 14.13(b)). As noted in Appendix A, Assumption 2 holds if the domain of v ´ graph(G(, v)) is countable or if (x, v) ´ G(x, v) is outer semicontinuous. It also holds if G is single valued and such that x ´ G(x, v) is continuous and v ´ G(x, v) is measurable, as shown in the details of Rockafellar and Wets (1998, Ex. 14.15). Assumption 2 implies v ´ G(x, v) is measurable for each x 2 Rn ; see Appendix A. This fact is useful for establishing well-posedness of certain integrals that appear later in Lyapunov conditions (see Lemma 4.1) as well as in the proofs that these Lyapunov conditions guarantee the associated stability properties. Common operations on (C, F, D, G) preserve Assumptions 1 and 2. The next result uses the fact that the intersection of two measurable, closed-valued mappings is measurable (Rockafellar and Wets, 1998, Prop. 14.11(a)) along with the fact that the graph of the intersection of two mappings is equal to the intersection of the mappings’ graphs.

Let R > 0 arbitrarily large and . > 0 arbitrarily small be given. Using Lagrange stability in probability of A, pick e > 0 large enough so that, for all x satisfying -(x) 6 R and all x 2 S r ðxÞ, we have 1  .=2 6 pAþeB ðxÞ. Then, using uniform global recurrence of O [ Oe , pick s > 0 large enough so that, for all x satisfying -(x) 6 R and all x 2 S r ðxÞ, we have 1  .=2 6 p\ðO[Oe Þ ðs; xÞ. Then, using (12)

Fact 3.1 (Basic conditions under intersections). Suppose G1 ; G2 : Rn  Rm  Rn satisfy Assumption 2, G1 satisfies Assumption 1.3, and x ´ G2(x, v) is outer semicontinuous for each v 2 Rm . Under these conditions, the intersection mapping G : Rn  Rm  Rn defined by (x, v) ´ G1(x, v) \ G2(x, v) =: G(x, v) satisfies Assumption 2 and Assumption 1.3.

Proposition 2.4 (Recurrence from Lagrange stability and recurrence of a larger set). Let - : C [ D ! RP0 . The open set O  Rn is uniformly globally recurrent relative to - for (1) if there are no finite escape times for (1a), the closed set A  Rn is uniformly Lagrange stable in probability relative to - for (1) and, for each e > 0 (arbitrarily large), the set O [ ðRn n ðA þ eBÞÞ is uniformly globally recurrent relative to - for (1). Proof. Let p\O ðs; xÞ denote the probability in (5) and let pAþeB ðxÞ denote the probability in (3). Given a closed set C  Rn , let p\C ðxÞ :¼ PðgraphðxÞ \ ðR2  CÞ – £Þ. Define the open set Oe :¼ Rn n ðA þ eBÞ and the closed set Ce :¼ Rn n ðA þ eB Þ ¼ Oe . It follows that

p\ðO[Oe Þ ðs; xÞ 6 p\O ðs; xÞ þ p\Ce ðxÞ p\Ce ðxÞ þ pAþeB ðxÞ ¼ 1

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

6

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

There are several useful corollaries of Fact 3.1. The first corollary pertains to the data in (11). Corollary 3.1. If O  Rn is open and the data (C, F, D, G) satisfies Assumptions 1 and 2 then the data (C\, F, D\, G\) with (C\, D\, G\) defined in (11) satisfies Assumptions 1 and 2. The next corollary establishes that we can make G(x, v) empty when x R D or v 2 A where A 2 BðRm Þ satisfies l(A) = 0 without affecting Assumptions 1 or 2. Corollary 3.2. Suppose (D, G) satisfies Assumptions 1.1, 1.3 and 2, A 2 BðRm Þ satisfies l(A) = 0, and R : Rn  Rm  Rn is defined as R(x, v) :¼ £ if x R D or v 2 A, and Rðx; v Þ :¼ Rn otherwise. Under these conditions the mapping (x, v) ´ G(x, v) \ R(x, v) satisfies Assumptions 1.3 and 2.

Proof. R(x, v) = R1(x) \ R2(v) with R1(x) :¼ £ when x 2 Rn n D; R1 ðxÞ :¼ Rn for x 2 D, R2(v) :¼ £ when v 2 A and R2 ðv Þ :¼ Rn when v 2 Rm n A. Since D is closed, the graph of R1 is closed and thus R1 is outer semicontinuous (for each v 2 Rm ). Since R1 is independent of v, it satisfies Assumption 2. Since R2 is independent of x and has closed values, it is outer semicontinuous for each v 2 Rm . Also, v ´ graph(R2(v)) is measurable since, for each closed set C  Rn  Rn ; fv 2 Rm : graphðR2 ðv ÞÞ \ C – £g ¼ Rm n A, the latter set being measurable. So the corollary follows from two applications of Fact 3.1. h The next fact states that, with d : Rn ! RP0 continuous, Assumptions 1 and 2 hold for inflated data (Cd, Fd, Dd, Gd) where

C d :¼ fx 2 Rn : ðx þ dðxÞBÞ \ C – £g F d :¼ coFððx þ dðxÞBÞ \ CÞ þ dðxÞB Dd :¼ fx 2 Rn : ðx þ dðxÞBÞ \ D – £g Gd :¼ fg 2 Rn : g ¼ g~ þ dðxÞB; g~ 2 Gððx þ dðxÞBÞ \ D; v Þg

ð13Þ

which is relevant for robustness studies. That Assumption 1 holds for (Cd, Fd, Dd, Gd) is contained in Cai et al. (2007, Prop. 3.1). That Gd satisfies Assumption 2 has been established in Teel et al. (submitted for publication, Prop. 3); the proof is based on Rockafellar and Wets (1998, Prop. 14.11(a and c) and Thm. 14.13(b)) and uses Gd ðxÞ ¼ GðHðxÞ \ D; v Þ þ dðxÞB, where HðxÞ :¼ x þ dðxÞB. The outer perturbation in Gd depends on x rather than g~; the latter is needed for converse Lyapunov theorems, as in Teel et al. (submitted for publication), whereas the former is also natural and results in more straightforward robustness proofs given a Lyapunov function. (See Theorem 4.6 below and compare with Teel et al., 2009, Section V.) n

Fact 3.2. If d : R ! RP0 is continuous and Assumptions 1 and 2 hold then (Cd, Fd, Dd, Gd) in (13) satisfies Assumptions 1 and 2. Our final fact is relevant for the implementation of state-dependent distribution functions within the framework (1). b : Rn  Rp  Rn is outer semicontinuous and locally Fact 3.3. If G bounded and H : Rn  Rm  Rp satisfies Assumptions 1.3 and 2 then b Hðx; v ÞÞ satisfies Assumptions 1.3 and 2. ðx; v Þ # Gðx; v Þ :¼ Gðx;

Proof. Local boundedness and outer semicontinuity of G for each follows from Rockafellar and Wets (1998, Prop. 5.52(a b and S(x, v) = (x, H(x, v)). Assumption 2 follows, and b)) with T ¼ G like in Teel et al. (submitted for publication, Prop. 3), using the proof of Rockafellar and Wets (1998, Prop. 5.52(b)) and Rockafellar and Wets (1998, Prop. 14.11(a) & Thm. 14.13). h

v 2 Rm

Fact 3.3 relates to state-dependent distribution functions, including time-varying distribution functions if a timer or jump counter is a state component, when the distribution function can be derived from an i.i.d. distribution. For example, consider a jump b wÞ ¼ JðwÞx, where J : ZP0 ! Rnn . Setting J(w) = £ when map Gðx; b : Rn  R  Rn that is low R ZP0 results in a set-valued mapping G cally bounded and outer semicontinuous. We address the situation where w ¼ i 2 ZP0 with probability pi(x) where pi : Rn ! ½0; 1 is P continuous for each i and i2ZP0 pi ðxÞ ¼ 1 for all x 2 D. This statedependent probability distribution function can be expressed in terms of a single distribution function that is exponentially distributed with unity mean, for example, as follows. Define Pi si ðxÞ :¼ j¼0 pi ðxÞ and

Hðx; v Þ :¼ fi 2 ZP0 : v 2 ½ lnð1  si1 ðxÞÞ;  lnð1  si ðxÞÞg: It is a straightforward exercise to verify that H is locally bounded and outer semicontinuous, so that it satisfies Assumptions 1.3 and 2. Moreover, given ði; xÞ 2 ZP0  Rn , the probability that i 2 H(x, v) is equal to

Z

 lnð1si ðxÞÞ

 lnð1si1 ðxÞÞ

 lnð1s

ðxÞÞ

expðv Þdv ¼ expðv Þj lnð1si1 i ðxÞÞ

¼ 1  si1 ðxÞ  ð1  si ðxÞÞ ¼ si ðxÞ  si1 ðxÞ ¼ pi ðxÞ Thus, we can consider the jump map G(x, v) :¼ J(H(x, v))x and the exponential, unity mean distribution function for the i.i.d. process vi in place of the state-dependent distribution for w. 3.2. Consequences of stochastic hybrid basic conditions Two consequences of the stochastic hybrid basic conditions are given. The first result establishes the existence of random solutions for which the lengths of the time domains exceed a prescribed bound almost surely under mild viability conditions, one of which uses the tangent cone to C at x, denoted TC(x); see Appendix A or (Goebel and Teel, 2006, Prop. 2.4). The proof is in Section 10. Theorem 3.1 (Basic existence). Suppose there are no finite escape times for (1a), Assumptions 1 and 2 hold and for each x 2 C [ D at least one of the following viability conditions holds: (VC) x 2 C and there exists e > 0 such that TC(n) \ F(n) – £ for all n 2 C \ ðfxg þ eB Þ; (VD) x 2 D and there exists V x 2 BðRm Þ such that lðV x Þ ¼ 1 and, for each v 2 V x ; Gðx; v Þ \ ðC [ DÞ – £. For each ðs; xÞ 2 RP0  ðC [ DÞ, there exists x 2 S r ðxÞ such that

1 ¼ PððgraphðxÞ \ ðCPs  Rn Þ – £Þ ^ ððgraphðxÞ \ ðC6s  Rn ÞÞ  ðR2  ðC [ DÞÞÞÞ

ð14Þ

here ^ denotes the logical ‘‘and’’ operation. In particular, PððgraphðxÞ \ ðCPs  Rn Þ – £ÞÞ ¼ 1. The second result establishes that if - is lower semicontinuous with bounded sublevel sets then uniform global recurrence relative to - is equivalent to global recurrence relative to -; in other words, uniformity is guaranteed for free. A similar result for asymptotic stability in probability is given as a corollary. The next proposition is established in Section 10. Proposition 3.1 (Uniform recurrence from non-uniform recurrence). Suppose there are no finite escape times for (1a), Assumptions 1 and 2 hold, and - : C [ D ! RP0 is lower semicontinuous with bounded sublevel sets. The open set O  Rn is uniformly globally recurrent relative to - for (1) if and only if

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

x 2 dom

-; x 2 S r ðxÞ ) slim PððgraphðxÞ !1

 ðC
ð15Þ

The next corollary combines Propositions 3.1 and 2.2. Corollary 3.3. Suppose there are no finite escape times for (1a), Assumptions 1 and 2 hold, -1 : C [ D ! RP0 is lower semicontinuous and has bounded sublevel sets, -2 : C [ D ! RP0 , and the closed set A  Rn is uniformly globally stable in probability for (1) relative to -1. Then A is uniformly globally asymptotically stable in probability for (1) relative to -3(x) :¼ max{-1(x), -2(x)} if, for each d > 0, there exists a relatively open set Od  fx 2 C [ D : -1 ðxÞ < dg such that (15) holds with - = -2 and O ¼ Od . 4. Lyapunov-based results for stability and recurrence We use nonnegative functions and how they change along solutions to establish stability and recurrence properties. We consider several types of functions: Lagrange functions, Lyapunov functions, Foster functions, and combined Lagrange–Foster and Lyapunov– Foster functions. The functions get their names from the type of stability property they certify, whether Lagrange stability, Lyapunov stability, or recurrence. Foster’s name is often associated with recurrence because of the results in Foster (1953) on Lyapunov-like functions that can be used to prove recurrence. Each type of function must have some basic properties, making it a certification candidate. The properties that the function possesses beyond these basic properties determine what type of function it is. Table 1 contains a summary of the required properties. In light of Lyapunov conditions for asymptotic stability in non-stochastic hybrid systems, as in Goebel et al. (2009) for example, the results of this section are unsurprising, although nontrivial. A function V : dom V ! R is a certification candidate for (C, D, G, l) if (recall the definition V :¼ [x2X;i2ZP0 v iþ1 ðxÞ) C1. C [ D [ GðD  VÞ  dom V, C2. 0 6 V(x) for all x 2 C [ D [ GðD  VÞ, and R C3. the quantity Rm supg2Gðx;v Þ VðgÞlðdv Þ is well defined for each x 2 D, using the convention that supg2G(x, v)V(g) = 0 when G(x, v) = £, justified by the preceding item. Lemma 4.1. Under Assumptions 1 and 2, if V : dom V ! R is upper semicontinuous and satisfies C1–C2 then it satisfies C3. Proof. We just need to establish that, for each x 2 D, v ´ supg2G(x, v)V(g) ¼: h(v) is a measurable function, using the convention that h(v) = 0 when G(x, v) = £. It is enough to show that ~ v Þ is measurable using the convention v # inf g2Gðx;v Þ ðVðgÞÞ :¼ hð ~ v Þ ¼ 1 when G(x, v) = £ since hðv Þ ¼ maxf0; hð ~ v Þg and a that hð continuous function of a measurable function is measurable. Since V is upper semicontinuous, V is lower semicontinuous. As such, it is a ‘‘normal integrand’’ in the sense of Rockafellar and Wets (1998, Section 14.D); see (Rockafellar and Wets, 1998, Ex. 14.30). According to Assumption 2, v ´ G(x, v) is measurable and, from Assumption 1, it has closed values. It follows from Rockafellar and ^ v ; gÞ, Wets (1998, Ex. 14.32) that the extended-value function hð defined to be equal to V(g) if g 2 G(x, v) and equal to 1 otherwise, ^ v ; gÞ. ~ v Þ ¼ inf n hð is a normal integrand. Finally, note that hð g2R According to Rockafellar and Wets (1998, Thm. 14.37) this function is measurable. h Let LV(0) :¼ {x 2 dom V:V(x) = 0}. The function V is a partially Lipschitz (resp., partially C1) certification candidate for (C, D, G, l) if V is locally Lipschitz (resp., continuously differentiable) on an open set containing CnLV(0) as well as continuous on C. Henceforth, given a real-valued function V that is locally Lipschitz on an open

7

set U  Rn a point x 2 U, and a vector f 2 Rn we use V°(x; f) to denote the Clarke generalized directional derivative of V at x in the direction f. When V is C1 near x, this quantity reduces to hrV(x), fi. See (Clarke, 1990). We use S C;F ðxÞ to denote solutions of (1a) starting at x. 4.1. Lagrange functions and uniform Lagrange stability We define Lagrange functions and assert that they certify uniform Lagrange stability in probability. Let A  Rn be closed and let - : C [ D ! RP0 . A certification candidate for (C, D, G, l) is a Lagrange function for A relative to - for (1) if there exist a1 ; a2 2 G1 such that

a1 ðjxjA Þ 6 VðxÞ 8x 2 C [ D [ GðD  VÞ VðxÞ 6 a2 ð-ðxÞÞ 8x 2 C [ D Vð/ðtÞÞ 6 VðxÞ 8x 2 C; / 2 S C;F ðxÞ; t 2 dom / Z sup VðgÞlðdv Þ 6 VðxÞ 8x 2 D: Rm g2Gðx;v Þ

ð16aÞ ð16bÞ ð16cÞ ð16dÞ

A partially Lipschitz certification candidate for (C, D, G, l) is a regular Lagrange function for A relative to - for (1) if it satisfies the conditions (16a), (16b), and (16d) and

V  ðx; f Þ 6 0 8x 2 C n LV ð0Þ; f 2 FðxÞ

ð17Þ

Proposition 4.1. Let A  Rn be closed and - : C [ D ! RP0 . A regular Lagrange function for A relative to - for (1) is a Lagrange function for A relative to - for (1). Proof. Suppose there exist x 2 C; / 2 S C;F ðxÞ, and t 2 dom / such that V(/(t)) > V(x). Due to (16a), V(x) P 0. By continuity of r ´ V(/(r)) and the fact that /(0) = x, there exists s 2 dom / with s 2 [0, t) such that V(/(r)) > 0 for all r 2 [s, t] and V(/(s)) < V(/(t)). _ Due to (17), Vð/ðrÞÞ 6 0 for almost all r 2 [s, t] (see Clarke, 1990, Teel and Praly, 2000, Section 2, or Sanfelice et al., 2007, Section IV.B). Hence, V(/(t)) 6 V(/(s)), which is a contradiction. h The main result of this subsection, proved in Section 8, establishes that the existence of a Lagrange function implies uniform Lagrange stability in probability. Theorem 4.1 (Lagrange function implies Lagrange stability). Let A  Rn be closed and - : C [ D ! RP0 . Suppose Assumptions 1 and 2 hold and that either A is bounded or there are no finite escape times for (1a). Under these conditions, if there exists a Lagrange function for A relative to - for (1) then A is uniformly Lagrange stable in probability relative to - for (1).

4.2. Lyapunov functions and uniform global stability We define Lyapunov functions and assert that they certify uniform global stability in probability. A certification candidate for (C, D, G, l) is a Lyapunov function for A for (1) relative to - if there exist a1 ; a2 2 K1 such that (16) holds. A partially Lipschitz certification candidate for (C, D, G, l) is a regular Lyapunov function for A relative to - for (1) if there exist a1 ; a2 2 K1 such that (16a), (16b), (16d) and (17) hold. The proof of the next result follows that for Proposition 4.1. Proposition 4.2. Let A  Rn be closed and - : C [ D ! RP0 . A regular Lyapunov function for A relative to - for (1) is a Lyapunov function for A relative to - for (1). The main result of this subsection, proved in Section 8, establishes that existence of a Lyapunov function implies uniform global stability in probability.

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

8

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

Table 1 Summary of certification candidates. Function

Inequalities

Qualifiers

Guarantees

Lagrange Lyapunov Foster Lagrange–Foster

(16) (16) (18) (16a), (16b), (20)

a1 ; a2 2 G1 a1 ; a2 2 K1 q > 0; a2 2 G1 a1 ; a2 2 G1 ; q 2 PDðOÞ; A ¼ O; ðC \ ; D\ ; G\ Þ given

Lyapunov-Foster

(16a), (16b), (20)

a1 ; a2 2 K1 ; q 2 PDðAÞ; ðC \ ; D\ ; G\ Þ ¼ ðC; D; GÞ

Weak Foster Matrosov-Foster

(18a), (25) (28)–(29)

a2 2 G1 ; C \ n LV ð0Þ ¼ C \ ; ðC \ ; D\ ; G\ Þ given in (11)

Uniform Lagrange stability (Theorem 4.1) Uniform global stability (Theorem 4.2) Uniform global recurrence (Theorem 4.3) Uniform global recurrence (Theorem 4.4) & robustness when C0 and partially C1 (Theorem 4.6) Uniform global asymptotic stability (Theorem 4.5) and robustness when C0 and partially C1 (Theorem 4.6) Uniform global recurrence when Rn n O is compact (Theorem 5.1) Uniform global recurrence when Rn n O is compact (Theorem 5.2)

in (11)

(C\, D\, G\) given in (11), C \ n LV i ð0Þ ¼ C \ ; uc;i ; ud;i continuous and satisfy nested Matrosov conditions

Theorem 4.2 (Lyapunov function implies global stability). Let A  Rn be closed and - : C [ D ! RP0 . Suppose Assumptions 1 and 2 hold and that either A is bounded or there are no finite escape times for (1a). Under these conditions, if there exists a Lyapunov function for A relative to - for (1) then A is uniformly globally stable in probability relative to - for (1). 4.3. Foster functions and uniform global recurrence

VðxÞ 6 a2 ð-ðxÞÞ 8x 2 C \ [ D\

ð18aÞ

Vð/ðtÞÞ 6 VðxÞ  qt 8x 2 C \ ; / 2 S C \ ;F ðxÞ; t 2 dom / Z sup VðgÞlðdv Þ 6 VðxÞ  q 8x 2 D\

ð18bÞ ð18cÞ

A partially Lipschitz certification candidate for (C\, D\, G\, l) is a regular Foster function for O relative to - for (1) if it satisfies (18a) and (18c), and

V  ðx; f Þ 6 q 8x 2 C \ n LV ð0Þ; f 2 FðxÞ:

V  ðx; f Þ 6 qðxÞ 8x 2 C \ n LV ð0Þ; f 2 FðxÞ Z sup VðgÞlðdv Þ 6 VðxÞ  qðxÞ 8x 2 D\ Rm g2G\ ðx;v Þ

We define Foster functions and assert that they certify uniform global recurrence. Let O  Rn be open and - : C [ D ! RP0 . Recall the definitions of C\, D\ and G\ in (11). A certification candidate for (C\, D\, G\, l) is a Foster function for O relative to - for (1) if there exist q > 0 and a2 2 G1 such that

Rm g2G\ ðx;v Þ

candidate for (C\, D\, G\, l) is called a regular (respectively, partially C1) Lagrange–Foster function for O with respect to - for (1) if there exist a1 ; a2 2 G1 and q 2 PDðOÞ such that (16a) and (16b) hold with C ¼ C \ ; D ¼ D\ ; A :¼ O, and

ð19Þ

The next result is like Proposition 4.1, again using the fact that V(x) P 0 for all x 2 C\. Proposition 4.3. Let O  Rn be open and - : C [ D ! RP0 . A regular Foster function for O relative to - for (1) is a Foster function for O relative to - for (1). As stated next and proved in Section 9, a Foster function certifies uniform global recurrence. n

Theorem 4.3 (Foster function implies recurrence). Let O  R be open and - : C [ D ! RP0 . Suppose Assumptions 1 and 2 hold and there are no finite escape times for (1a). Under these conditions, if there exists a Foster function for O relative to - for (1) then O is uniformly globally recurrent relative to - for (1). 4.4. Joint stability and recurrence Two additional classes of functions are considered: Lagrange– Foster functions and Lyapunov–Foster functions. The former functions combine aspects of Lagrange functions and Foster functions to establish recurrence. The latter functions combine aspects of Lyapunov functions and Foster functions to establish uniform global asymptotic stability in probability. 4.4.1. Lagrange–Foster function and recurrence Let O  Rn be open and recall the definitions of C\, D\, and G\ in (11). A partially Lipschitz (respectively, partially C1) certification

ð20aÞ ð20bÞ

The existence of a Lagrange–Foster function implies uniform global recurrence. Theorem 4.4 (Lagrange–Foster function implies recurrence). Let O  Rn be open and - : C [ D ! RP0 . Suppose Assumptions 1 and 2 hold and that either O is bounded or there are no finite escape times for (1a). Under these conditions, if there exists a regular Lagrange– Foster function for O relative to - for (1) then O is uniformly globally recurrent relative to - for (1). Proof. Since q 2 PDðOÞ, a Lagrange–Foster function for O relative to - for (1) is a regular Lagrange function for O relative to - for (C\, F, D\, G\, l). Thus, according to Proposition 4.1 and Theorem 4.1, O is uniformly Lagrange stable in probability for (C\, F, D\, G\, l). According to Propositions 2.3 and 2.4, it is sufficient e D :¼ O[ to establish that for each D > 0 the set O ðRn n ðO þ DBÞÞ is uniformly globally recurrent relative to - for (1). Since q 2 PDðOÞ, it follows that, for each D > 0, a Lagrange–Foster function for O relative to - for (1) is a regular e D relative to - for (1). The result now Foster function for O follows from Proposition 4.3 and Theorem 4.3. h 4.4.2. Lyapunov–Foster functions and asymptotic stability Let A  Rn be closed. A partially Lipschitz (respectively, partially C1) certification candidate for (C, D, G, l) is called a regular (respectively, partially C1) Lyapunov–Foster function for A with respect to - for (1) if there exist a1 ; a2 2 K1 and q 2 PDðAÞ such that (16a), (16b) hold and (20) holds with C\ = C, D\ = D and G\ = G. The following result gives conditions under which the existence of a Lyapunov–Foster function implies uniform global asymptotic stability in probability. Theorem 4.5 (Lyapunov–Foster function implies UGAS). Let A  Rn be closed, - : C [ D ! RP0 be lower semicontinuous. Suppose that

8d- > 0 9dA > 0 : ððA þ dA B Þ \ ðC [ DÞÞ  fx 2 C [ D : -ðxÞ < d- g;

ð21Þ

Assumptions 1 and 2 hold, and that either A is bounded or there are no finite escape times for (1a). Under these conditions, if there exists a regular Lyapunov–Foster function for A relative to - for (1) then A is uniformly globally asymptotically stable in probability relative to - for (1).

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

9

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

Proof. Since q 2 PDðAÞ, a Lyapunov–Foster function for A relative to - for (1) is a regular Lyapunov function for A relative to - for (1). Thus, according to Proposition 4.2 and Theorem 4.2, A is uniformly globally stable in probability relative to - for (1). In particular, A is uniformly Lagrange stable in probability relative to - for (1). Then, according to Propositions 2.2, 2.3, and 2.4, it is sufficient to establish that for each 0 < d < D < 1 the set e d;D :¼ ðA þ dB Þ [ ðRn n ðA þ DBÞÞ is uniformly globally recurO rent relative to - for (1). Since q 2 PDðAÞ, it follows that a Lyapunov–Foster function for A relative to - for (1) is a regular Foster e d;D relative to - for (1) for each 0 < d < D < 1. Therefunction for O fore, the result follows from Proposition 4.3 and Theorem 4.3. h 4.5. Robustness We establish that if a Lagrange–Foster function, respectively a Lyapunov–Foster function, is continuous and partially C1 then it guarantees that uniform global recurrence, respectively uniform global asymptotic stability in probability, holds even in the presence of small perturbations, as constructed in (13). These robustness statements are obtained by combining Theorem 4.4, respectively Theorem 4.5, with the following theorem, stated for bounded sets O or A for simplicity. Theorem 4.6 (Extra regularity implies robustness). Suppose Assumptions 1 and 2 hold. Let O  Rn be open and bounded (respectively, let A  Rn be compact) and let - : Rn ! RP0 . If V is a partially C1 certification candidate for ðC ~d ; D~d ; G~d ; lÞ (see (13)) for some ~ d 2 PDðOÞ (respectively ~ d 2 PDðAÞ), a partially C1 Lagrange–Foster (respectively, Lyapunov–Foster) function for O (respectively, A) relative to - for (C, F, D, G, l), and continuous on its domain then there exist a concave function j 2 K1 that is smooth on R>0 and d 2 PDðOÞ (respectively, d 2 PDðAÞ) such that j(V) is a Lagrange– Foster (respectively, Lyapunov–Foster) function for O (respectively, A) relative to - for (Cd, Fd, Dd, Gd, l). The function j can be taken to be the identity function when l has compact support. Proof. We use (C, D, G) in place of (C\, D\, G\) throughout the proof. Using the definition of the class PDðOÞ, if q; ~ d 2 PDðOÞ then there exist a compact set A and e > 0, and functions qm ; ~ dm 2 PDðAÞ such that A þ eB  O and qm(x) 6 q(x) and ~ dm ðxÞ 6 ~ dðxÞ for all x 2 Rn . We use q and ~ d in place of qm and ~ dm in what follows. The ideas in the proof of Cai et al. (2007, Theorem 3.2 (A) implies (B)) establishes the existence of da 2 PDðAÞ with da ðxÞ 6 ~ dðxÞ for all x 2 Rn such that (20a) holds with C replaced by C da , F replaced by F da , and q replaced by 0.5q. In turn, (20a) holds with V replaced by j(V) if j 2 K1 and j0 (s) > 0 for all s > 0. Using the continuity of V and da and the local boundedness of Gda , there exists c 2 G1 such that supg2Gda ðx;v Þ VðgÞ 6 cðjxj þ jv jÞ for all x 2 Dda . As in Subbaraman et al. (submitted for publication, Section XII), there exists a concave function j 2 K1 , necessarily R satisfying j0 (s) > 0 for all s > 0, such that Rm jð2cð2jv jÞÞlðdv Þ 6 1. Thus, for all x 2 Dda ,

Z

sup

Rm g2Gda ðx;v Þ

jðVðgÞÞlðdv Þ ¼

Z Rm

!

j

sup VðgÞ g2Gda ðx;v Þ

lðdv Þ < 1

ð22Þ

Clearly we can take j(s) = s for all s 2 RP0 when l has compact support. Without loss of generality, we assume that V(x) P q(x) for all x 2 dom V, we define

q^ ðxÞ :¼ jðVðxÞÞ  jðVðxÞ  qðxÞÞ P j0 ðVðxÞÞqðxÞ 8x 2 Dda

ð23Þ

^ 2 PDðAÞ. Now consider perturbations of the form and note that q dc(x) = cda(x) with c 2 (0, 1]. We claim that for each compact set K  Rn n A there exists c 2 (0, 1] such that

Z

sup

jðVðgÞÞlðdv Þ 6 jðVðxÞÞ  0:5q^ ðxÞ 8x 2 Ddc \ K

Rm g2Gdc ðx;v Þ

ð24Þ

1 Otherwise, suppose for each i 2 ZP0 and with ci ¼ iþ1 there exists xi 2 Ddci \ K such that

Z

sup

Rm g2Gdc ðxi ;v Þ

jðVðgÞÞlðdv Þ > jðVðxi ÞÞ  0:5q^ ðxi Þ

i

Since K is compact, we can assume the sequence xi converges to x 2 D \ K. Using the outer semicontinuity of G, the continuity of ^ , Fatou’s lemma or the Dominated Convergence da, j, V and q Theorem (Fristedt and Gray, 1997, Lemma 8 or Thm. 9, Section 8.2) with (22), Jensen’s inequality (Fristedt and Gray, 1997, Prop. 12, Section 5.3) applied with the convex function j, (20), and the def^ in (23), it follows that inition of q

jðVðxÞÞ  0:5q^ ðxÞ 6 lim sup i!1

Z

sup

6

Rm g2Gðx;v Þ

6j

Z

Z

sup

jðVðgÞÞlðdv Þ

Rm g2Gdc ðxi ;v Þ i

jðVðgÞÞlðdv Þ !

sup VðgÞlðdv Þ

Rm g2Gðx;v Þ

^ ðxÞ: 6 jðVðxÞ  qðxÞÞ ¼ jðVðxÞÞ  q

^ 2 PDðAÞ, this bound is impossible. Since x 2 K  Rn n A and q For i 2 Z, define K i :¼ fx 2 Rn : jxjA 2 ½2i ; 2iþ1 g and let ci 2 (0, 1] be such that (24) holds with c = ci and K = Ki. Let g 2 PDðAÞ satisfy g(x) 6 ci 6 1 for all x 2 Ki, and take d(x) :¼ g(x)da(x). We claim that x 2 Dd \ Ki implies x 2 Ddci \ K i . Indeed, if x is such that ðx þ dðxÞBÞ \ D – £ and x 2 Ki then d(x) 6 cida(x) so that ðx þ ci da ðxÞBÞ \ D – £, i.e., x 2 Ddci . It follows R ^ ðxÞ for from (24) that Rm supg2Gd ðx;v Þ jðVðgÞÞlðdv Þ 6 jðVðxÞÞ  0:5q all x 2 Dd, which establishes the result. h 5. Recurrence of a set having a compact complement We define additional types of functions that are useful for establishing recurrence of an open set that has a compact complement. The earlier Proposition 2.4 and the discussion that precedes it show how this situation comes up as a means to establishing recurrence of an open, bounded set. We introduce weak Foster functions and Matrosov–Foster functions. 5.1. Weak Foster functions A regular certification candidate for (C\, D\, G\, l) is called a regular weak Foster function for O relative to - for (1) if (18a) holds with a2 2 G1 , we have C\nLV(0) = C\, and

V  ðx; f Þ < 0 8x 2 C \ n LV ð0Þ; f 2 FðxÞ Z sup VðgÞlðdv Þ < VðxÞ 8x 2 D\ Rm g2G\ ðx;v Þ

ð25aÞ ð25bÞ

Theorem 5.1. Let Rn n O be compact. If Assumptions 1 and 2 hold then a regular weak Foster function for O relative to - for (1) is a regular Foster function for O relative to - for (1). Proof. We show that there exists q > 0 such that (18c) and (19) hold. Otherwise, for each i 2 ZP0 there exists xi 2 Rn such that either xi 2 C\nLV(0) and there exists fi 2 F(xi) satisfying

V  ðxi ; fi Þ P Vðxi Þ 

1 iþ1

ð26Þ

or xi 2 Di and

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

10

Z

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

sup VðgÞlðdv Þ P Vðxi Þ 

Rm g2G\ ðxi ;v Þ

1 iþ1

ð27Þ

By focusing on subsequences, we can consider the two situations separately. First, using the outer semicontinuity and local boundedness of F and the compactness of Rn n O, we suppose there exists a convergent sequence graphðFÞ \ ððC \ n LV ð0ÞÞ Rn Þ 3 ðxi ; fi Þ ! ðx; f Þ 2 graphðFÞ \ ðC \  Rn Þ such that (26) holds for all i 2 ZP0 . According to Clarke (1990, Prop. 2.1.1(b)), V° is upper semicontinuous. Using the continuity of V, we get

  1 ¼ VðxÞ V  ðx; f Þ P lim supV  ðxi ; fi Þ P lim Vðxi Þ  i!1 iþ1 i!1

sup VðgÞlðdv Þ P

Rm

lim sup sup VðgÞlðdv Þ i!1

P lim sup i!1

Z

V 0 ð/ðtÞÞ 6 V 0 ðxÞ 8x 2 C \ ; / 2 S C\ ;F ðxÞ; t 2 dom /

ð29aÞ

V i ðx; f Þ 6 uc;i ðxÞ 8x 2 C \ ; f 2 FðxÞ; i 2 f1; . . . ; mg

ð29bÞ

3. (a) uc,0(x) = 0 for all x 2 C\, (b) ud,0(x) 6 0 for all x 2 D\, (c) with the functions uc;mþ1 ; ud;mþ1 : Rn ! f1g, j 2 {0, 1, . . . , m} we have i.

which contradicts (25a) since C\nLV(0) = C\. Next, suppose there exists a convergent sequence D\ 3 xi ? x 2 D\ such that (27) holds for each i 2 ZP0 . It follows from the continuity of V and the outer semicontinuity of x ´ G\(x, v) for each v (see Fact 3.1) that z # supg2G\ ðz;v Þ VðgÞ is upper semicontinuous for each v. Moreover, from the continuity of V and the definition of G\, there exists b > 0 such that supg2G\ ðz;v Þ VðgÞ 6 b for all ðz; v Þ 2 D\  Rm . It follows from Fatou’s lemma or the Dominated Convergence Theorem (Fristedt and Gray, 1997, Lemma 8 or Thm. 9, of V that Z Z Section 8.2) and the continuity Rm g2G\ ðx;v Þ

2. in addition,

g2G\ ðxi ;v Þ

sup VðgÞlðdv Þ

Rm g2G\ ðxi ;v Þ

for

each

if x 2 C\ and uc, i(x) = 0 for all i 2 {0, 1, . . . , j} then uc, j+1(x) 6 0, if x 2 D\ and ud, i(x) = 0 for all i 2 {0, 1, . . . , j} then ud, j+1(x) 6 0.

ii.

Condition 3 imposes a nested negative semi-definite condition on the functions uc, i and ud, i. The functions uc,0 and ud,0 are never positive and, in general, uc, j+1 (respectively ud, j+1) can be positive only where at least one of the functions uc, i,i 2 {0,. . . , j}, is negative. Through the definition of uc, m+1 (respectively, ud, m+1), there are no points in C\ (respectively, D\) where all of the uc, i (respectively, ud, i) are zero. The condition (28a) holds if Vi is upper semicontinuous on C\ [ D\ since this set is compact and thus Vi is uniformly bounded in this case and we can take a2(0) to exceed this bound. However, upper semicontinuity is not a requirement. Theorem 5.2. Let O  Rn be open and such that Rn n O is compact. If there exist Matrosov-Foster functions for O then there exists a Foster function for O.

  1 P lim Vðxi Þ  ¼ VðxÞ i!1 iþ1 which contradicts (25b). The contradictions give the result. h The next two results follow by combining Theorem 5.1 with Proposition 2.4. Corollary 5.1 (Recurrence from uniform Lagrange stability plus a weak Foster function). Let O  Rn be open and bounded and - : C [ D ! RP0 . Suppose Assumptions 1 and 2 hold and O is uniformly Lagrange stable in probability relative to - for (C\, F, D\, G\, l). If for each D > 0 there exists a regular weak Foster function for O [ ðRn n ðO þ DBÞÞ relative to - for (1) then O is uniformly globally recurrent relative to - for (1). Corollary 5.2 (UGAS from uniform global stability plus a weak Foster function). Let A  Rn be compact, - : C [ D ! RP0 be lower semicontinuous, and such that (21) holds. Suppose Assumptions 1 and 2 hold and that A is uniformly globally stable in probability relative to - for (1). If for each pair of positive real numbers (d, D), there exists a regular weak Foster function for ðA þ dB Þ [ ðRn n ðA þ DBÞÞ relative to - for (1) then A is uniformly globally asymptotically stable in probability relative to - for (1). 5.2. Matrosov–Foster functions Let O  Rn be open, let - : C \ [ D\ ! RP0 , and let m 2 ZP1 . Inspired by Sanfelice and Teel (2009), we say that the m + 1 functions Vi are Matrosov-Foster functions for O relative to - if there exist a2 2 G1 and continuous functions uc;i : C \ ! R and ud;i : D\ ! R; i 2 f0; . . . ; mg, such that 1. for all i 2 {0, . . . , m}, Vi is a certification candidate for (C\, D\, G\, l) with Vi being regular for i 2 f1; . . . ; mg; C \ n LV i ð0Þ ¼ C \ , and

Z

V i ðxÞ 6 a2 ð-ðxÞÞ 8x 2 C \ [ D\ sup V i ðgÞlðdv Þ 6 V i ðxÞ þ ud;i ðxÞ 8x 2 D\

Rm g2G\ ðx;v Þ

ð28aÞ ð28bÞ

Proof. Let K i 2 RP0 for each i 2 {1, . . . , m} and define Pm V s ðxÞ :¼ i¼1 K i V i ðxÞ for all x 2 \ i2{1, . . . , m}dom Vi. Then Vs is a regular certification candidate for (C\, D\, G\, l) and

8x 2 C \ ; f 2 F \ ðxÞ V s ðx; f Þ 6 8x 2 D \

m X K i uc;i ðxÞ i¼1

Z Rm

sup V s ðgÞlðdv Þ  V s ðxÞ 6

g2G\ ðx;v Þ

m X K i ud;i ðxÞ i¼1

According to Sanfelice and Teel (2009, Thm. 5.4), there exist K i 2 RP0 ; i 2 f1; . . . ; mg, and q > 0 such that

uc;0 ðxÞ þ

m X

K i uc;i ðxÞ ¼

i¼1

ud;0 ðxÞ þ

m X

m X K i uc;i ðxÞ 6 q 8x 2 C \ i¼1

K i uc;i ðxÞ 6 q 8x 2 D\ :

i¼1

It follows V(x) :¼ V0(x) + Vs(x) is a Foster function for O. h Corollary 5.3 (Recurrence from uniform Lagrange stability plus Matrosov–Foster functions). Let O  Rn be open and bounded and - : C [ D ! RP0 . Suppose Assumptions 1 and 2 hold and O is uniformly Lagrange stable in probability relative to - for (C\, F, D\, G\, l). If for each D > 0 there exist Matrosov–Foster functions for O [ ðRn n ðO þ DBÞÞ relative to - for (1) then O is uniformly globally recurrent relative to - for (1). Corollary 5.4 (UGAS from uniform global stability plus Matrosov– Foster functions). Let A  Rn be compact, - : C [ D ! RP0 be lower semicontinuous, and such that (21) holds. Suppose Assumptions 1 and 2 hold and that A is uniformly globally stable in probability relative to - for (1). If for each pair of positive real numbers (d, D), there exist Matrosov–Foster functions for ðA þ dB Þ [ ðRn n ðA þ DBÞÞ relative to - for (1) then A is uniformly globally asymptotically stable in probability relative to - for (1).

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

11

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

6. Examples

6.2. Nefarious nurse

6.1. Stochastic bouncing ball

This section presents an example showing that if the causality condition in the definition of solution is removed then the results summarized in Table 1 may not hold.

The bouncing ball is a system of the form (1) with state x 2 R2 , flow set C :¼ RP0  R, flow map F(x) :¼ (x2, c)T for x – 0, F(0) :¼ {0}  [c, 0] where c > 0, jump set D :¼ f0g  R60 and jump map Gg(x, v) :¼  r(v)x + (0, [0, g])T where the r : R ! RP0 is measurable and g 2 RP0 . Note that the interval [0, g] is added þ to r(v)x2 in the xþ 2 inclusion, while x1 ¼ 0 for x 2 D. The data (C, F, D, Gg) satisfies Assumptions 1 and 2 for each g 2 RP0 . The case g = 0 we call the bouncing ball without offset. We assume there exists r > 0 such that

Z

rr ðv Þlðdv Þ < 1:

k :¼

ð30Þ

R

For example, if r(v) = v, l({1.1}) = p and l({0.5}) = 1  p with R p 2 (0, 1) then R rr ðv Þlðdv Þ ¼ p1:1r þ ð1  pÞ0:5r . The p that makes this expression equal to 1 is p = (1  0.5r)/(1.1r  0.5r). By l’Hopital’s rule, as r ? 0 this value approaches ln(2)/ (ln(1.1) + ln(2)) 0.879, while for r = 2 it is about 0.781. Proposition 6.1 (Origin of bouncing ball without offset is UGAS; with offset, neighborhoods proportional to g are recurrent). If there exists r > 0 such that (30) holds then the origin is uniformly globally asymptotic stability in probability for the bouncing ball without offset ~ > 0 such that, for each g > 0, the open (C, F, D, G0, l) and there exists c ~gB  R2 is uniformly globally recurrent for the bouncing set O :¼ c ball (C, F, D, Gg, l).

Proof. Let g = 0 and let A denote the origin in R2 . Consider the certification candidates U and V defined by UðxÞ :¼ x22 þ 2cx1 for all x 2 R2 and V(x) :¼ U(x)r/2 for all x 2 C [ D, which satisfy (16a) and (16b) with -ðxÞ ¼ jxj ¼ jxjA for some functions a1 ; a2 2 K1 . See (Khalil, 2002, Lemma 4.3) for example. Then hrU(x), f(x)i = 0 for all x 2 C, and therefore (16c) holds for both U and V. Moreover, R using (30), R supg2G0 ðx;v Þ VðgÞlðdv Þ 6 kVðxÞ 6 VðxÞ for all x 2 D. Hence V satisfies condition (16d). Thus V is a Lyapunov function for the origin and, according to Theorem 4.2, the origin is uniformly globally stable in probability for (C, F, D, G0, l). Let K  Rn n A be compact. Let V0(x) = V(x) and V1(x) = b1 + x2 where b1 + x2 P 0 for all x 2 K. Let b2 > 0 be such that b1 + x2 6 b2 for all x 2 K. It is straightforward to verify that (V0, V1) are Foster–Matrosov functions for Rn n K by using the functions ud,0(x) = (k  1)V(x), uc,1(x) =  c, and ud,1(x) = b2. According to Corollary 5.4, the origin is uniformly globally asymptotically stable in probability for (C, F, D, G0, l). Now we consider the case where g > 0. By using the fact that, for each c1 > 1 there exists c2 > 1 such that (a + b)r 6 c1ar + c2br, it follows from (30) that there exists e > 0 satisfying

Z ke :¼

ðrðv Þ þ eÞr lðdv Þ < 1

ð31Þ

R

~ :¼ Define c have

e1 . With the definition of O in the proposition, we

x 2 D \ ðR2 n OÞ ¼: D\ ) g 6 ejx2 j

ð32Þ R

It follows from (31) and (32) that R supGg ðx;v Þ VðgÞlðdv Þ 6 ke VðxÞ 6 VðxÞ for all x 2 D\. It follows that V is a Lagrange function for O for (C, F, D\, Gg, l). The proof of recurrence of O now appeals to Corollary 5.3 using the same Matrosov–Foster functions used for uniform global asymptotic stability in probability above. The details are omitted. h

6.2.1. Qualitative description In this example, there is a (color-blind) patient to whom a nurse may administer pills at two prescribed times during the day, say at 9:00 and 21:00. At each of the prescribed times, the nurse chooses whether or not to give a pill to the patient. There are two kinds of pills that are randomly made available to the nurse, each with equal probability, a sleeping pill and an energy pill. Taking a sleeping pill in the morning or an energy pill at night makes the patient unhappier, while taking a sleeping pill at night or an energy pill during the morning makes the patient happier. Without pills, the patient would slowly drift toward happiness, which is his desired state; the nurse, being nefarious, wants to make the patient unhappy. The average effect of the pills is assumed to make the patient happier. It would seem that, regardless of the strategy of the adversarial nurse, the probability that the patient is not happy in the long run would be zero. However, we will see that the information that is available to guide the nurse’s decisions has a significant effect on the ability of the nurse to make the patient unhappy. 6.2.2. Hybrid model with recurrence and stability analysis The hybrid model has three states ðx1 ; x2 ; x3 Þ 2 R3 , where (x1, x2) flows on the unit circle S1 , where the position on the circle denotes time on a 24-hour clock, with the 0 h corresponding to (x1, x2) = (0,1). The state x3 quantifies the unhappiness of the patient, and is restricted to the set RP0 . Ultimate happiness corresponds to x3 equal to or possibly just near zero, while unhappiness corresponds to x3 being large. The flow set and flow map are given by

2

3 rx2 6 7 FðxÞ ¼ 4 rx1 5; ef ðx3 Þ

r > 0; e > 0 f : RP0 ! R is continuous

ð33aÞ

C ¼ S1  RP0 ;

f ðsÞ > 0 8s > 0

ð33bÞ

C ¼ S1  RP0 ;

f ðsÞ > 0 8s P s > 0

ð33cÞ

C ¼ ðS1 n WÞ  RP0

f ðsÞ P 0 8s P s P 0

ð33dÞ

where (33b), (33c), and (33d) constitute different models of the patient’s slow drift toward happiness in the absence of medication, W is an open sector that covers a small amount of time immediately after 9:00 and 21:00, and r scales time so that one rotation around the circle corresponds to 24 h. The jump set is

D ¼ fðc; cÞg  RP0 ;

pffiffiffi c ¼ 1= 2

ð34Þ

which corresponds to 9:00 and 21:00 on the 24-hour clock. In the cases of (33b) and (33c) the solutions are not unique because it is possible to also flow at points in D. This feature corresponds to the fact that we have given the nurse discretion about whether to administer a pill. This discretion is removed in (33d) by removing W from the flow set. The jump map instantaneously advances time by 3 h while decreasing or increasing x3 by the amount dictated by the combination of the type and timing of the pill given. In particular, the jump map rotates (x1, x2) by 45° in the clockwise direction while for x3 it has the following effect: in the morning, if a sleeping pill is given then it amplifies x3–bx3 while if an energy pill is given then it attenuates x3 to cx3; in the evening, if a sleeping pill is given then

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

12

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

it attenuates x3 to cx3 while if an energy pill is given then it amplifies x3–bx3. Mathematically, this map is expressed as

Gðx; v Þ ¼ diagðR; hðv ; x1 ÞÞx R ¼ 45 degree clockwise rotation matrix ( hðv ; x1 Þ ¼

ð35Þ

pffiffiffi

cv þ bð1  v Þ if x1 ¼ 1= 2 pffiffiffi bv þ cð1  v Þ if x1 ¼ 1= 2

where v 2 {0, 1} with equal probability where v = 0 corresponds to a sleeping pill and v = 1 corresponds to an energy pill. The values c and b are positive (the interesting case is when c < 1 < b) and on average they lead toward happiness. In other words

0 6 c;

0 6 b;

ðc þ bÞ=2 ¼: k < 1

ð36Þ

it impossible for the nurse to wait to give a pill. In the case of (33d), the solutions are unique thereby forcing causality. 6.3. Example with ‘‘spontaneous’’ jumps We consider an example with random jump times and a setvalued jump map. Two versions of the model are given. In the first version, the model enforces an extra level of causality and uniform global asymptotic stability in probability ensues. In the second version, this extra level of causality is removed and there is a random solution for which all sample paths diverge. 6.3.1. Stability for version with extra causality Consider the hybrid model with state x ¼ ðz1 ; z2 ; s; qÞ 2 R4 and

The data (C, F, D, G) in (33) and (36) satisfies Assumptions 1 and

data C :¼ R2  RP0 f1g; FðxÞ :¼ ðz2 ; z1 ; 1; 0ÞT ;D :¼ R2  f0g  T f1; 1g; GðxÞ :¼ ðz1 ; z2 ; v ; 1ÞT when q ¼ 1; GðxÞ :¼ GTa;c ðzÞ; s; 1

Proposition 6.2. Consider the model (C, F, D, G, l) defined in (33) and (36) where W is an open sector that covers a small amount of time immediately after 9:00 and 21:00, and define the compact set A :¼ S1  f0g and the open, bounded set O :¼ A þ s B .

when q =  1, where for c P 0,Ga,c is the outer semicontinuous

2.

hull ðgraphðGa;c Þ ¼ graphðg a;c ÞÞ of the mapping

( g a;c ðzÞ ¼

fg 2 R2 : g T g ¼ ð1 þ cÞzT zg if ðz1 þ z2 Þ2 6 cz21 fg 2 R2 : g T g ¼ czT zg

if ðz1 þ z2 Þ2 > cz21

1. The set A is uniformly globally asymptotically stable in probability if (33b) holds or if (33d) holds with s⁄ = 0. 2. The set O is uniformly globally recurrent if (33c) holds or if (33d) holds with s⁄ > 0.

and l() is the exponential distribution with unity mean. Due to the definitions of C and D, two consecutive jumps occur when (s, q) = (0, 1). The random input v determines the return time to s = 0. Assumptions 1 and 2 are satisfied.

Proof. For x 2 R3 define q(x) :¼ min{ef(x3), (1  k)jx3j} and V(x) :¼ x3. The condition (36) implies the condition (20b) with D\ = D and G\ = G. In turn, if (33c) holds then (20a) holds while if (33b) holds then (20a) holds with C\ = C. So the results for (33b) and (33c) follow from Theorems 4.5 and 4.4 respectively. In the case of (33d), the condition (17) holds. Since (20b) implies (16d), uniform global stability in probability for s⁄ = 0 and uniform Lagrange stability for s⁄ > 0 then follow from Theorems 4.2 and 4.1 respectively. To conclude recurrence for s⁄ > 0 or uniform global asymptotic stability for s⁄ = 0, we appeal to Corollarys 5.3 and 5.4 respectively. We define V0(x) = V1(x) = V(x) for all x. Then we let w : R ! RP1 be a smooth, periodic function with period 2p that satisfies w0 (h) =  1 for all h except those corresponding angles contained in the sector W. The derivative of w must be positive for some angles contained in the sector W to ensure that w is periodic. Then take V2(x) = w(h(x1, x2)) where h is the angle on the clock determined by (x1, x2). With uc,1(x) = 0, uc,2(x) =  r, ud,1 (x) = (k  1)jx3j and ud,2(x) = M with M > 0 sufficiently large, it is straightforward to verify that (V0, V1, V2) are Matrosov–Foster functions of the type required to apply Corollaries 5.3 and 5.4. h

Proposition 6.3. For the model defined above with c > 0 sufficiently small, the closed set A :¼ f0g  RP0  f1; 1g  R4 is uniformly global stable in probability relative to -s(z, s, q) = jzj and, for each ~ 2 PDðAa Þ with Aa :¼ f0g  R2 , is uniformly globally asymptotiq ~ ðzÞs. cally stable in probability relative to -a ðz; s; qÞ ¼ jzj þ q

6.2.3. Non-causal solutions Now consider what can happen when the nefarious nurse has ‘‘inside information’’ and either (33b) or (33c) holds. In particular, suppose the nurse knows ahead of time which random pill will be made available. In this case, the nurse can wait to administer the given pill at the time for which the pill makes the patient unhappier. If e > 0 is small compared to b  1 and f(x3) 6 x3 for all x3 then this nefarious, non-causal, strategy will prevent the patient from becoming happy, regardless of the order in which the pills arrive. Indeed, during one 24-hour cycle at least one pill is given and it always makes the patient unhappier, to an extent more than the increase in happiness that occurs by the passage of time. Thus, the Lyapunov conditions summarized in Table 1 are not sufficient for recurrence and stability if the causality constraint on solutions is relaxed. Finally, note that the behavior described above cannot occur in the case of (33d) because the removal of W from the flow set makes

Proof. The nature of the flow map rules out finite escape times. Define P :¼ diag(2,1) and W(z) :¼ zTP z. Let F a ; Q 2 R22 satisfy Fa z = (z2, z1)T, Q = QT, and zTQ z = 2z1z2. Both P  Q and P + Q are positive definite and hrW(z), Fazi = zTQ z. Hence, there exists e > 0 such that, for (z1 + z2)2 > 0

hrWðzÞ; F a zi þ sup WðgÞ  WðzÞ ¼ zT ðP  Q Þz 6 2ezT z g2Ga;0 ðzÞ

while for (z1 + z2)2 = 0 we have 2zT(I + Q)z = 0 so that

hrWðzÞ; F a zi þ sup WðgÞ  WðzÞ ¼ zT Qz þ sup g T Pg  zT Pz g2Ga;0 ðzÞ T

T

T

g2Ga;0 ðzÞ T

T

¼ z Qz þ 2z z  z Pz  2z ðI þ Q Þz ¼ z ðP þ Q Þz 6 2ezT z Then, due to the way Ga,c depends on c, it follows that for c > 0 sufficiently small

hrWðzÞ; F a zi þ sup WðgÞ  WðzÞ 6 ezT z 8z 2 R2

ð37Þ

g2Ga;c ðzÞ

Let /(t, z) denote the solution at time t of n_ ¼ F a n starting at z. Then define

( Vðz; s; qÞ :¼

/ðs; zÞT ðP  Q Þ/ðs; zÞ 8ðz; s; qÞ 2 R2  RP0  f1g zT Pz

8ðz; s; qÞ 2 R2  f0g  f1g

Since / is continuous and P  Q and P are positive definite, V is a certification candidate. Since /(s, z)T /(s, z) = zTz and @/ð@ss;zÞ ¼ @/ðs;zÞ F a z for all ðz; sÞ 2 R2  RP0 , it follows that (16a) holds with @z a1 2 K1 , (16b) holds with x(z, s, q) = jzj and a2 2 K1 , and (16c) holds. We verify (16d). Consider (z, s, q) 2 D with (s, q) = (0, 1). We have V(z,0, 1) = zT (P  Q)z and s+ = s, q+ = 1 with V(z+,0,1) = z+P z+ = W(z+) with z+ 2 Ga,c(z). It follows from (37) that

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

sup WðgÞ 6 WðzÞ  hrWðzÞ; F a ðzÞi  ezT z ¼ zT ðP  QÞz  ezT z g2Ga;c ðzÞ

¼ Vðz; 0; 1Þ  ezT z Consider (z, s, q) 2 D with (s, q) = (0, 1). Then V(z, 0, 1) = zTPz and (z+, s+, q+) = (z, v, 1), so that V(z+, s+, q+) = /T(v, z)(P  Q)/(v, z). It straightforward to verify that

Z 0

1

/T ðv ; zÞðP  Q Þ/ðv ; zÞ expðv Þdv Z 1 d ¼ ð/T ðv ; zÞP/ðv ; zÞ expðv ÞÞdv ¼ zT Pz ¼ Vðz; 0; 1Þ d v 0

It follows that (16d) holds. According to Theorem 4.2, A is uniformly globally stable in probability relative to -s. Finally, we establish that A is uniformly globally attractive in ~ ðzÞs with probability with respect to -a ðz; s; qÞ ¼ jzj þ q q~ 2 PDðAa Þ and Aa ¼ f0g 2 R2 . We take V 2 ðz; s; qÞ ¼ Vðz; s; qÞþ ~eV 1 ðs; qÞ, where V1(s, 1) = s for s 2 RP0 and V1(0, 1) = 2 and, for each d > 0, we aim to establish the conditions (18) with the relatively open set Od;D :¼ ððAa þ dB Þ [ ðRn n ðAa þ DBÞÞÞ RP0  f1; 1g. by picking ~e > 0 sufficiently small. For flows, we get that V 2 ðzðtÞ; sðtÞ; qðtÞÞ 6 V 2 ðzð0Þ; sð0Þ; qð0ÞÞ  ~et. At jumps from (s, q) = (0, 1) with jzj P d, we get

sup V 2 ðg; s; 1Þ 6 V 2 ðz; s; 1Þ  ezT z þ 2~e g2Ga;c ðzÞ

6 V 2 ðz; s; 1Þ  ed2 þ 2~e while at jumps from (s, q) = (0, 1) we get

Z

1

V 2 ðz; v ; 1Þ expðv Þdv ¼ Vðz; 0; 1Þ þ ~e

0

¼ Vðz; 0; 1Þ þ ~eV 1 ð0; 1Þ  ~e ¼ V 2 ðz; 0; 1Þ  ~e ~ ðzÞgÞ, there exists q > 0 Thus, with ~e 2 ð0; minfed2 =2; minjzj2½d;D q such that the conditions (18) hold. It follows from Theorem 4.3 that Od;D is uniformly globally recurrent relative to -a. It then follows from Proposition 2.4 that A þ dB is uniformly globally recurrent relative to -a. Finally, it follows from Proposition 2.2 and the uniform global stability of A relative to -s that A is uniformly globally asymptotically stable in probability relative to -a. h 6.4. Instability for version without extra causality In this version, we eliminate the logic variable q, combining the jumps at q =  1 and q = 1. Thus, we take C :¼ R2  RP0  R3 ; FðxÞ ¼ ðz2 ; z1 ; 1ÞT ; D :¼ R2  f0g  R3 and GðxÞ :¼ ðGTa;c ðzÞ; v ÞT . Consider the initial condition (z(0, 0), s(0, 0)) = (n, 0) with n1 =  n2 – 0. A jump must occur giving s(0, 1) = v and zð0; 1Þ 2 Ga;c ðnÞ ¼ fg 2 R2 : g T g ¼ ð1 þ cÞnT ng. Since the flow map corresponds to a linear oscillator for the z dynamics, with knowledge of v it is possible to select z(0, 1) so that /ðv ; zð0; 1ÞÞ ¼ p ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi (v, z(0, 1)) will evolve until ð1 þ cÞn. In this case, the flowpfrom ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi s = 0 at which point z will equal ð1 þ cÞn, since t ´ /(s(t, j),z(t, j)) is constant and /(0,z(t, j)) = z(t, j). This selection can be repeated indefinitely so that z(t2n+1,2n) = (1 + c)nn, regardless of the values of v. The time domains depend on x but the values of z at the end of each flow interval do not depend on x. In particular, every solution blows up. Therefore, the set A :¼ f0g  RP0 is neither stable nor attractive. 7. Modeling and analysis of spontaneous transitions As suggested by the previous example, (Davis, 1993, pp. 59, and Hespanha, 2005, pp. 1357–1358), systems with spontaneous

13

jumps can be modeled by (1). In Section 7.1, we discuss a general model that allows both spontaneous and forced transitions, which may be generated by different jump maps, and different levels of causality. In Section 7.2, we give a Lyapunov-based stability result where we relax the condition (16a). Sections 7.3 and 7.4 contain special cases, the latter relating the model to the piecewise deterministic processes of Davis (1993) and constructing a Lyapunov function from an infinitesimal generator condition. 7.1. Accounting for both spontaneous and forced transitions; different levels of causality Consider a model with state x ¼ ðzT ; s; f; qÞT 2 Rnþ3 and a parameter p 2 {1, 1}. Let C a ; Da  Rn be closed, define D1;0 ; D1;1  Rnþ1 ; D1 ; D0 ; D1  Rnþ2 and C; D  Rnþ3 by D1;0 :¼ C a f0g; D1;1 :¼ Da  RP0 ; C :¼ C a  RP0  f0; 1gf1g; D0 :¼ ðD1;0  f0gÞ [ ðD1;1  f1gÞ; D1 ¼ D1 :¼ ðD1;0 [ D1;1 Þ  f0; 1g, and D :¼ S i2{1,0,1}Di  {i} and consider the dynamics

8 z_ 2 F a ðzÞ > > > < s_ ¼ 1 ðz; s; f; qÞ 2 C _ > f¼0 > > : q_ ¼ 0 8 þ z ¼z > > > < sþ ¼ s ðz; s; f; qÞ 2 D1  f1g þ > f 2 fi 2 f0; 1g : ðz; sÞ 2 D1;i g > > : þ q ¼0 8 þ 2 Ga ðz; s; f; v 1 Þ z > > > < sþ ¼ 1 ðð1 þ pÞs þ ð1  pÞv Þ 0 ðz; s; f; qÞ 2 D0  f0g þ 2 > f ¼f > > : þ q ¼p 8 þ z ¼z > > > < sþ ¼ v 0 ðz; s; f; qÞ 2 D1  f1g þ > f ¼ f > > : þ q ¼ 1

lðÞ ¼ ðl0 ðÞ; l1 ðÞÞ l0 ðR<0 Þ ¼ 0

ð38aÞ

ð38bÞ

ð38cÞ

ð38dÞ

ð38eÞ

where l0 is the distribution for the random input v0 and l1 is the distribution for the random input v1. (The subscripts in (Ca, Fa, Da, Ga) should not be confused with the subscripts used in (13).) When q = 1, the state flows (38a) and can jump when ðz; sÞ 2 ðC a  f0gÞ [ ðDa  RP0 Þ ¼ D1;0 [ D1;1 . The condition (z, s) 2 Ca  {0} = D1,0 corresponds to a spontaneous (time-triggered) jump; the condition ðz; sÞ 2 Da  RP0 ¼ D1;1 corresponds to a ‘‘forced’’ (state-triggered) jump. These jumps (38b) change the mode to q = 0 and set f in case the jumps from Ca  {0} should be different than the jumps from Da  RP0 . Causality motivates setting f before allowing the jumps of z. From q = 0, there are two possibilities (38c) depending on the value of p 2 {1, 1}. For p = 1, the jumps of s and z are concurrent and the mode is set back to q = 1. For p = 1, the jumps of z occur before the jumps of s, inducing an extra level of causality. Indeed, for p = 1, when q = 0 the variable z is updated and the mode is changed to q = 1. When q = 1, s is reset and the mode returns to q = 1 as in (38d). When p = 1 and Ga is independent of f, the behavior of the variable z in (38) agrees with the behavior of z in the simplified system

ðz; sÞ 2 C a  RP0



z_ 2 F a ðzÞ

s_ ¼ 1

 þ z 2 Ga ðz; s; v 1 Þ ðz; sÞ 2 ðC a  f0gÞ [ ðDa  RP0 Þ þ s ¼ v0 lðÞ ¼ ðl0 ðÞ; l1 ðÞÞ l0 ðR<0 Þ ¼ 0

ð39aÞ ð39bÞ ð39cÞ

For either (38) or (39), the situation where

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

14

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

s_ ¼ kðzÞ k : C a ! RP1 continuous

ð40Þ

can be addressed through (38) or (39) by changing Fa(z) to Fa(z)/k(z) and using s_ ¼ 1. This change corresponds to a scaling of ordinary time, slowing the solutions but not changing their values. Consequently, stability properties for the time-scaled system confers those properties to the system satisfying (40). We also address the case where k : C a ! RP0 , so that the derivative of s may be zero at some points z, in the context of a class of piecewise deterministic processes in Section 7.4. 7.2. Stability analysis Let Aa  Rn be closed. For (38), we initially focus on uniform global stability in probability of

A :¼ Aa  RP0  f0; 1g  f1; 0; 1g:

ð41Þ

While the Lyapunov conditions (16) can be applied directly to (38), it is useful to relax (16a). For simplicity we assume that

Ga ðz; s; f; v 1 Þ  C a [ Da

8ðz; s; f; v 1 Þ 2 D0  V 1 :

ð42Þ

A certification candidate V for (C, D, G, l) that satisfies (16c), (16d), and for which there exists a1 ; a2 2 G1 (respectively, a1 ; a2 2 K1 ) such that

a1 ðjzjAa Þ 6 Vðz; s; f; qÞ 6 a2 ðjzjAa Þ 8ðz; s; f; qÞ 2 D

l0 ðRPr Þa1 ðjzjAa Þ 6

Z

ð43aÞ

Vðz; v 0  r; f; qÞl0 ðdv 0 Þ 8ðz; r; f; qÞ 2 C

RPr

ð43bÞ is called a relaxed Lagrange (respectively, Lyapunov) function for A for (38). When l0 ðRPr Þ ¼ expðkrÞ for all r 2 RP0 , with k > 0, so that l0(dv0) = k exp(kv0)dv0, (43b) becomes

a1 ðjzjAa Þ 6

Z

Vðz; v 0 ; f; 1Þl0 ðdv 0 Þ 8ðz; fÞ 2 C a  f0; 1g

ð44Þ

RP0

Thus, the expected value of V should satisfy an appropriate lower bound instead of requiring (16a) to hold for all s. The next result, proved in Section 8.6, establishes stability of A in (41) relative to the function

-ðz; s; f; qÞ :¼

8 < jzjA

8ðz; s; f; qÞ 2

:

otherwise

1

[

ð45Þ

Theorem 7.1 (Relaxed Lagrange, or Lyapunov, function implies Lagrange, or global, stability). Suppose (Ca, Fa, Da, Ga) satisfies Assumptions 1 and 2, there are no finite escape times for z 2 Ca, z_ 2 Fa(z), and (42) holds. Let Aa be closed and A be as in (41). If there exists a relaxed Lagrange (respectively, Lyapunov) function for A for (38) then A is uniformly Lagrange (respectively, globally) stable in probability relative to -, as defined in (45), for (38). We turn our attention to uniform global recurrence of sets

Od :¼ fðz; s; f; qÞ 2 C [ D : -ðz; s; f; qÞ < dg; d > 0:

ð46Þ

In light of Proposition 2.2, it is worth noting that 1) - is lower semicontinuous since D0 and D1 are closed and 2) Od is open relae containtive to C [ D for each d > 0 since there exists an open set O ing [i2{0,1}(Di  {i}) and disjoint from C [ (D1  {1}) so that e \ ðC [ DÞ ¼ [i2f0;1g ðDi  figÞ and thus Od ¼ ðRn n ðAa þ dBÞ O e \ ðC [ DÞ. The recurrence will be relative to a function -q~ R3 Þ \ O defined by

-q~ ðz; s; f; qÞ :¼ -ðz; s; f; qÞ þ q~ ðzÞðs þ 1Þ q~ 2 PDðAa Þ

Vðz; s; f; qÞ 6 a2 ðjzjAa Þ 8ðz; s; f; qÞ 2 [i2f0;1g ðDi  figÞ Z

max

Rmþ1 g2Gðz;s;f;q;v Þ

ð47Þ

ð48aÞ

VðgÞlðdv Þ 6 Vðz; s; f; qÞ  qðzÞ 8ðz; s; f; qÞ 2 D0  f0g

ð48bÞ

is called a relaxed Foster function for Aa for (38). Theorem 7.2 (Relaxed Foster function implies recurrence). Suppose (Ca, Fa, Da, Ga) satisfies Assumptions 1 and 2 and that (42) holds. Let Aa be closed and suppose the set A in (41)is uniformly Lagrange stable in probability relative to - in (45). If there exists a relaxed Foster ~ 2 PDðAa Þ the set Od function for Aa for (38) then for each d > 0 and q in (46) is uniformly globally recurrent relative to -q~ in (47) for (38). ~ 2 PDðAa Þ. Since Aa is uniformly Lagrange Proof. Fix d > 0 and q stable in probability relative to -, and hence relative to -q~ since -ðz; s; f; qÞ 6 -q~ ðz; s; f; qÞ, according to Proposition 2.4 it is enough to show uniform global recurrence of

e \ ðRn n ðAa þ dBÞ  R3 ÞÞ [ ððRn n ðAa þ DBÞÞ  R3 Þ Od;D :¼ ð O e was characterized below (46). According to for each D > 0, where O Proposition 2.3, it is enough to establish uniform global recurrence of Od;D for ðC; F; D \ ðRnþ3 n Od;D Þ; G; lÞ. We begin by establishing e :¼ Rn  f0g  R2 . Due to the propuniform Lagrange stability of A e erties of O, we have that, for i 2 {0, 1},

ðRnþ3 n Od;D Þ \ ðDi  figÞ ¼ ððAa þ DBÞ n ðAa þ dB Þ  R3 Þ \ ðDi  figÞ

ð49Þ

 :¼ inf jzjA 2½d;D minfqðzÞ; q ~ ðzÞg. Pick j 2 K1 to be concave, Let q a such that j is locally Lipschitz on (0, 1) and (necessarily) @ j(s) \ (1, 0] = £ for all s > 0, and so that j(s) 6 s for all s and R jðv 0 þ 1Þl0 ðdv 0 Þ 6 1. (See the proof of Theorem 4.6.) Define RP0

V 2 ðz; s; f; qÞ :¼ Vðz; s; f; qÞ þ eðjðs þ bq Þ þ cq Þ b1 ¼ 1; b0 ¼ b1 ¼ 0; c1

ðDi  figÞ

i2f0;1g

A certification candidate V for (C, D, G, l) satisfying (16c), (16d) for which there exist a2 2 G1 ; q 2 PDðAa Þ so that

  q ¼ 1; c0 ¼ 0; c1 ¼ 3; e 2 0; : 4

ð50Þ

e for We verify that this function is a Lagrange function for A e ðC; F; D \ ðRnþ3 n Od;D Þ; G; lÞ. According to (50), the definition of A,  (49), the definition of q, the fact that j(s) 6 s for all s P 0, and since -(z, s, f, q) = 1 if q =  1, with a~ 2 ðsÞ :¼ 2  maxfa2 ðsÞ; 3sg, it follows that

ejðjðz; s; f; qÞjeA Þ 6 V 2 ðz; s; f; qÞ

8ðz; s; f; qÞ 2 C [ D

~ 2 ð-q~ ðz; s; f; qÞÞ 8ðz; s; f; qÞ V 2 ðz; s; f; qÞ 6 a 2 C [ ðD \ ðRnþ3 n Od;D ÞÞ e Therefore, (16a) and (16b) hold with A replaced by A; ~ 2 . Moreover, replaced by -q~ ; a1 replaced by e j and a2 replaced by a due to the properties of j and the fact that s_ ¼ 1, it follows that (16c) also holds. Next, we establish (16d) for V2. For (z, s, f, q) = D1  {1}, using (16d) for V and (50)

V 2 ðzþ ; sþ ; fþ ; qþ Þ ¼ Vðz; s; fþ ; qþ Þ þ eðjðs þ b0 Þ þ c0 Þ 6 Vðz; s; f; qÞ þ eðjðs þ b1 Þ þ c1 Þ þ eðc0  c1 Þ ¼ V 2 ðz; s; f; qÞ þ eðc0  c1 Þ 6 V 2 ðz; s; f; qÞ  e Let ðz; s; f; qÞ 2 ðD0  f0gÞ \ ðRnþ3 n Od;D Þ. Using (48b), (49), and (50), for p = 1 we have

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

15

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

Z

sup

Rmþ1

exists a partially C1 certification candidate ðZ; Z; Ga ; l1 Þ; a1 ; a2 2 K1 , jc ; jd 2 R; e > 0 such that

 þ eðjðs þ b1 Þ þ c1 Þ V 2 ðgÞlðdv Þ 6 Vðz; s; f; qÞ  q

g2Gðz;s;f;q;v Þ

 þ eðc1  c0 Þ ¼ V 2 ðz; s; f; qÞ  q

a1 ðjzjAa Þ 6 WðzÞ 6 a2 ðjzjAa Þ 8z 2 Z hrWðzÞ; f i 6 jc WðzÞ 8z 2 Z; f 2 F a ðzÞ

6 V 2 ðz; s; f; qÞ  e

Z

while for p =  1

Z

sup WðgÞl1 ðdv 1 Þ 6 expðjd ÞWðzÞ 8z 2 Z

Rm g 1 2Ga ðz;v 1 Þ

sup

 V 2 ðgÞlðdv Þ 6 Vðz; s; f; qÞ  q

Rmþ1 g2Gðz;s;f;q;v Þ

þe

Z

el0 ðRPr Þ 6 !

Z

jðv 0 þ b1 Þl0 ðdv 0 Þ þ c1

 þ eð1 þ c1 Þ 6 V 2 ðz; s; f; qÞ  q

For (z, s, f, q) 2 D1  {1}, using (16d) for V and (50),

sup

V 2 ðgÞlðdv Þ 6 Vðz; s; f; qÞ þe

Z

expðjc ðv 0  rÞÞl0 ðdv 0 Þ

ð51aÞ ð51bÞ ð51cÞ ð51dÞ

RPr

expðjc v 0 þ jd Þl0 ðdv 0 Þ ¼: k < 1

ð51eÞ

The condition (51d) holds with e = 1 when jc P 0. For jc < 0, it holds with e = exp(jcT) when T 2 RP0 satisfies l([0, T]) = 1 and R1 with e ¼ 0 expðjc v 0 Þl0 ðdv 0 Þ when l0 ðRPr Þ ¼ expðTrÞ with T 2 R>0 . The condition (51e) recovers the conditions of Hespanha and Teel (2006, Corollary 2) (where L plays the role of jc and ‘ plays the role of exp(jd)). Indeed:

6 V 2 ðz; s; f; qÞ  e

Rmþ1 g2Gðz;s;f;q;v Þ

Z

for

RP0

RP0

Z

W

!

jðv 0 þ b1 Þl0 ðdv 0 Þ þ c1

RP0

6 V 2 ðz; s; f; qÞ þ eð1 þ c1  c1 Þ 6 V 2 ðz; s; f; qÞ  e These bounds verify (16d) for V2 and thus, according to Theoe relative rem 4.1, establish uniform Lagrange stability of the set A to -q~ for ðC; F; D \ ðRnþ3 n Od;D Þ; G; lÞ. Finally, according to Propositions 2.3 and 2.4, it is enough to establish uniform global recurrence of Od;D for ðC \ ðRn  e; D e   R2 Þ; F; D \ ðRnþ3 n Od;D Þ; G; lÞ for each D e > 0. Here we ½ D e > 0 there use the calculations above and the fact that for each D e þ 1 to exists e3 > 0 such that @ j0 (s) \ (1, e3) = £ for all s 2 ½1; D see that V2 is a Foster function for Od;D relative to -q~ for e; D e   R2 ; F; D \ ðRnþ3 n Od;D Þ; G; lÞ for each D e > 0. C \ ðRn  ½ D This observation establishes the result. The previous two results can be combined to establish a result on uniform global asymptotic stability in probability. A certification candidate V for (C, D, G, l) is called a relaxed Lyapunov–Foster function for Aa for (38) if it is both a relaxed Lyapunov function for Aa for (38) and a relaxed Foster function for Aa for (38). The next result follows by combining Theorems 7.1 and 7.2, Proposition 2.2, and the observations below (46). Corollary 7.1 (Relaxed Lyapunov–Foster function implies UGAS). Suppose (Ca, Fa, Da, Ga) satisfies Assumptions 1 and 2, there are no finite escape times for z 2 Ca, z_ 2 Fa(z), and (42) holds. Let Aa be closed and A be as in (41). If there exists a relaxed Lyapunov-Foster ~ 2 PDðAa Þ, the set A is function for Aa for (38) then, for each q uniformly globally asymptotically stable in probability relative to -q~ in (47) for (38). 7.3. Result for weakly causal, time-triggered set-valued jumps Consider the situation where Da = £ and p =  1 in (38). When Da = £, the variable f plays no role in the dynamics of (38) since if it is initialized to zero then is remains at zero for all time. Thus, as described earlier, when Da = £ and p = 1, the behavior of (38) agrees with that of the simplified system (39). We use Z :¼ Ca and Ga(z, v1) for Ga(z, s, f, v1). Suppose there

1. for an arbitrary distribution with compact support [T1, T2], the condition (51e) requires exp(jcT1 + jd) < 1 and exp(jcT2 + jd) < 1, which covers (Hespanha and Teel, 2006, Corollary 2(i)); 2. for a uniform distribution on [0, T], (51e) becomes expðjd Þ expðjc TÞ1 < 1, which is (Hespanha and Teel, 2006, Corollary jc T 2(ii)); 3. for an exponential distribution with mean T, (51e) becomes jd Þ jc T < 1; expð < 1, which is (Hespanha and Teel, 2006, Corollary 1jc T 2(iii)). Proposition 7.1. Suppose (Ca, Fa, Da, Ga) satisfies Assumptions 1 and 2, Da = £, p =  1, either Aa is compact or there are no finite escape times for z 2 Ca, z_ 2 Fa(z), and, with Z :¼ Ca, Ga(z, v1)  Z for all ðz; v 1 Þ 2 Z  V 1 . If there exists a partially C1 certification candidate W for (Z, Z, Ga, l1), a1 ; a2 2 K1 and jc ; jd 2 R such that the conditions ~ 2 PDðAa Þ, the set Aa is uniformly globally (51) hold then, for each q asymptotically stable in probability relative to -q~ in (47) for (38). Proof. There are no finite escape times either by assumption or, if Aa is compact, because of (51b) and the lower bound in (51a). Let k > 0 satisfy k P k exp(jd). We verify that V(z, s, f, q) :¼ exp(jcs)W(z) for q 2 {1, 0} and V(z, s, f, q) :¼ k W(z) for q = 1 is a relaxed Lyapunov–Foster function for Aa for (38). It is immediate that V is a certification candidate for (C, D, G, l). Since Da = £, (z, s, f, q) 2 D implies s = 0 and thus (43a) and (48a) follow from (51a). Using (51d), for all (z, r, f, q) 2 C we have

Z

Vðz; v 0  r; f; qÞl0 ðdv 0 Þ ¼

RPr

Z

expðjc ðv 0  rÞÞl0 ðdv 0 ÞWðzÞ

RPr

P l0 ðRPr Þea1 ðjzjAa Þ so that (43b) holds. Using (51b), for all (z, s, f, q) 2 C we get

hrVðz; s; f; qÞ; faT

1 0 0

T

i 6 0 8fa 2 F a ðzÞ

so that (16c) holds. There is no change in V at jumps when q =  1. For q = 1, we get

Z RP0

Vðz; v 0 ; f; qÞl0 ðdv 0 Þ ¼ WðzÞ

Z

expðjc v 0 Þlðdv 0 Þ RP0

¼ WðzÞk expðjd Þ 6 kWðzÞ the final bound being equal to V(z, s, f,1). When q = 0, we get

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

16

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

Z

in place of (52b) with k : C a ! RP0 is appropriate for (38) with fa(z) replaced by ~f a ðzÞ :¼ f a ðzÞ=ð1 þ kðzÞÞ (or with s_ changed to s_ ¼ 1 þ kðzÞ) and Ga replaced by

sup Vðg a ; v 0 ; f; 1Þlðdv Þ

Rmþ1 g a 2Ga ðz;v 1 Þ

Z

!

Z

sup Wðg a Þl1 ðdv 1 Þ expðjc v 0 Þl0 ðdv 0 Þ

6 RP0

6 WðzÞ

Rm g a 2Ga ðz;v 1 Þ

Z

e a ðz; s; f; v 1 ; uÞ :¼ G

expðjc v 0 þ jd Þl0 ðdv 0 Þ ¼ kWðzÞ

RP0

¼ Vðz; s; f; 0Þ  ð1  kÞa1 ðjzjAa Þ Thus (16d) and (48b) hold. Corollary 7.1 gives the result. h 7.4. Lyapunov functions from extended generator conditions In this section, we consider a special case of (38), addressing systems like in Davis (1993), where Lyapunov functions can be derived from infinitesimal generator conditions. The special case we consider is delineated by the following assumption: Assumption 3. The parameter p ¼ 1; Aa is compact (Ca, fa, Da, Ga) satisfies Assumptions 1 and 2, where fa : C a ! Rn is a function, the solutions of z 2 Ca, z_ = fa(z) are unique and there are no nontrivial solutions from points in Ca \ Da, there exist mappings GC a and GDa such that, 8ðz; s; f; v 1 Þ 2 D0  V 1

Ga ðz; s; f; v 1 Þ ¼ fGDa ðz; v 1 Þ þ ð1  fÞGC a ðz; v 1 Þ  C a [ Da and l0 is an exponential distribution with unity mean. We also make a Lyapunov-type assumption, which contains a condition on the infinitesimal generator in (52b). Assumption 4. There exist a C1 on a neighborhood of Ca certification candidate W for ðC a ; C a ; GC a ; l1 Þ and ðC a ; Da ; GDa ; l1 Þ; a1 ; a2 2 K1 ; j 2 K, and q 2 PDðAa Þ such that

a1 ðjzjAa Þ 6 WðzÞ 6 a1 ðjzjAa Þ 8z 2 C a [ Da hrWðzÞ; fa ðzÞi þ

Z

Z

sup

WðgÞl1 ðdv 1 Þ 6 WðzÞ  qðzÞ 8z 2 Da

Rm g2GDa ðz;v 1 Þ

hrWðzÞ; fa ðzÞi 6 0 8z 2 C a \ Da

ð52cÞ ð52dÞ ð52eÞ

\ Da, which resembles the boundary condition in Davis (1993, Thm. 26.14). When GDa GC a , the condition (52d) rules out R this boundary condition; (52d) can be weakened to Rm supg2GDa ðz;v Þ WðgÞlðdv Þ 6 WðzÞ for all z 2 Da, which includes the boundary condition, at the expense of adding an assumption on the expected value of the number of jumps in any finite amount of ordinary time, like in Davis (1993, Assumption 24.4). To make further connection to Davis (1993, Thm. 26.14), note that

max WðgÞl1 ðdv 1 Þ

Rm g2GC a ðz;v Þ

6 WðzÞ  qðzÞ 8z 2 C a

with u uniformly distributed on [0, 1]. Indeed, dividing both sides of (53) by 1 + k(z) and using k(z) W(z)/(1 + k(z)) = W(z)/(1 + k(z))  e a in place of fa and Ga and W(z) converts (53), (52b) with ~f a and G with q replaced by q(z)/(1 + k(z)). The example of Section 6.3 shows that the next result does not hold when p =  1 in (38). ~ 2 PDðAa Þ, Proposition 7.2. Under Assumptions 3 and 4, for each q the set A :¼ Aa  R3 is uniformly globally asymptotically stable in probability relative to -q~ in (47) for (38). Proof. The Conditions (52a) and (52b) rule out finite escape times since Aa is compact. Let s⁄:Ca [ Da ? [0, 1] be such that, for each z 2 Ca, s⁄(z) is the first time the solution of z 2 Ca, z_ = fa(z) reaches Da from z, with s⁄(z) = 1 if the solution from z never reaches Da, and s⁄(z) = 0 for each x 2 Dan Ca. For z 2 Ca, let /(, z) denote the unique solution of n 2 C a ; n_ ¼ fa ðnÞ starting at z. For z 2 DanCa, let /(0, z) = z. In particular, /(s, z) is well-defined for all s 2 [0, s⁄(z)] whenever s⁄(z) < 1 and if s⁄(z) = 1 then /(s, z) is well-defined for all s 2 [0, 1). Define

W> :¼ fðz; s; f; qÞ 2 C [ ðD1  f1gÞ : s > s ðzÞg W6 :¼ fðz; s; f; qÞ 2 C [ ðD1  f1gÞ : s 6 s ðzÞg and

We verify that V is a relaxed Lyapunov–Foster function for Aa for (38). That V is nonnegative valued on C [ D follows from (52b) and (52e); V is upper semicontinuous due to (52e) and thus V is a certification candidate from Lemma 4.1. The lower bound in (43a) follows from the lower bound in (52a) and (52e). The upper bounds in (43a) and (48a) follow from the compactness of Aa , the continuity of fa and rW and rW(z) = 0 for all z 2 Aa , which follows from (52a) and the fact that W is continuously differentiable. The function V remains constant during flows since t ´ /(s(t, j), z(t, j)) is constant for each j, and

j 2 K, as it is thenR implied by (52b). The condition (52e) follows from (52b) when Rm supg2GCa ðz;v Þ WðgÞlðdv Þ ¼ WðzÞ for all z 2 Ca -

Z

kðzÞ z u > 1þkðzÞ > > > : fGa ðz; s; f; v 1 Þ; zg u ¼ kðzÞ 1þkðzÞ

ð52bÞ

Conditions (52d) and (52e) disappear when Da = £, which is the case when there are no forced transitions, and condition (52c) disR appears when qðzÞ þ Rm supg2GCa ðz;v Þ WðgÞlðdv Þ P jðjzjA Þ for some

hrWðzÞ; fa ðzÞi þ kðzÞ

kðzÞ u < 1þkðzÞ

8 WðzÞ  ð1  fÞhrWðzÞ; fa ðzÞi > > > < WðzÞ Vðz; s; f; qÞ :¼ > Wð/ðs ðzÞ; zÞÞ > > : Wð/ðs; zÞÞ  hrWð/ðs; zÞÞ; fa ð/ðs; zÞÞi

sup WðgÞl1 ðdv 1 Þ

hrWðzÞ; fa ðzÞi 6 WðzÞ  jðjzjA Þ 8z 2 C a

Ga ðz; s; f; v 1 Þ

ð52aÞ

Rm g2GC a ðz;v 1 Þ

6 WðzÞ  qðzÞ 8z 2 C a

8 > > > <

ð53Þ

t#



s ðzðt; jÞÞ  sðt; jÞ /ðs ðzðt; jÞÞ; zðt; jÞÞ

ðz; s; f; qÞ 2 D0  f0g ðz; s; f; qÞ 2 D1  f1g ðz; s; f; qÞ 2 W> ðz; s; f; qÞ 2 W6 :



is constant for each j when s⁄(z(tj, j)) < 1, while if s⁄(z(tj, j)) = 1 then s⁄(z(t, j)) = 1 for all t in the solution’s domain. This fact establishes (16c). Now consider jumps. Let (z, s, f, q) 2 D1  {1}. These jumps land in D0  {0} so that

Vðzþ ; sþ ; fþ ; qþ Þ ¼ WðzÞ  ð1  fþ ÞhrWðzÞ; fa ðzÞifþ 2 fi 2 f0; 1g : ðz; sÞ 2 D1;i g

ð54Þ

Note that (z, s) 2 D1, 0 implies s = 0 so that (z, s, f, q) 2 W6, while (z, s) = 2 D1, 1nD1, 0 implies s > 0 = s⁄(z) so that (z, s, f, q) 2 W>. Thus, according to the definition of V

Vðz; s; f; qÞ ¼ WðzÞ  hrWðzÞ; fa ðzÞi 8ðz; sÞ 2 D1;0

ð55aÞ

Vðz; s; f; qÞ ¼ WðzÞ 8ðz; sÞ 2 D1;1 n D1;0

ð55bÞ

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

17

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

It follows from (54), (55) that V does not change when (z, s) 2 (D1.1nD1, 0) [ (D1,0n D1,1). For (z, s) 2 D1,0 \ D1,1, before the jump V has the value W(z)  hrW(z),fa(z)i according to (55a). After the jump, if f+ = 0 then W(z)  hrW(z), fa(z)i is also the value of V after the jump according to (54); if f+ = 1 then, using that (z, s) 2 D1,0 \ D1,1 implies z 2 Ca \ Da, (52e), and (54), we have V(z+, s+, f+, q+) = W(z) 6 W(z)  hrW(z), fa(z)i = V(z, s, f, q). Now let (z, s, f, q) 2 D1  {1}. These jumps land in W> [ W6 For s⁄(z) < 1, we get

Z

Vðz; v 0 ; f; qÞ expðv 0 Þdv 0 ¼

Z s ðzÞ 0

RP0

þ

Z

Vðz; v 0 ; f; qÞ expðv 0 Þdv 0

1

s ðzÞ

¼

Z s ðzÞ

Vðz; v 0 ; f; qÞ expðv 0 Þdv 0

ðWð/ðv 0 ; zÞÞ

0

 hrWð/ðv 0 ; zÞÞ; fa ð/ðv 0 ; zÞÞiÞ  expðv 0 Þdv 0 þ Wð/ðs ðzÞ; zÞÞ  expðs ðzÞÞ Z s ðzÞ d ðWð/ðv 0 ; zÞÞ ¼ dv 0 0  expðv 0 ÞÞdv 0 þ Wð/ðs ðzÞ; zÞÞ  expðs ðzÞÞ ¼ WðzÞ  Wð/ðs ðzÞ; zÞÞ expðs ðzÞÞ

The quantity >2,T(/) is well defined under Assumption 5. Lemma 8.1. Under Assumption 5, each convergent sequence ðxi ; /i Þ 2 Rn  X with /i 2 S 0 ðxi Þ has limit ðx; /Þ 2 Rn  X satisfying / 2 S 0 ðxÞ. Moreover, for each bounded sequence xi 2 Rn and each sequence /i 2 S 0 ðxi Þ there exists a subsequence converging to ðx; /Þ 2 Rn  X satisfying / 2 S 0 ðxÞ. Proof. Consider the first statement. Suppose x 2 Rn n C. Since C is closed by Assumption 1, for i sufficiently large we have that /i has a trivial domain (i.e., dom /i = (0, 0)) and is given by /i(0) = xi. Hence, the sequence /i converges to the function / with trivial domain given by /(0) = x and thus / 2 S 0 ðxÞ. If x 2 C then it follows from Goebel and Teel (2006, Lemma 4.3) and the fact that the function / with trivial domain given by /(0) = x is a solution of (1a) that / 2 S 0 ðxÞ. Consider the second statement. Since the sequence xi is bounded, without loss of generality we can assume that it converges to some x 2 Rn . If x 2 Rn n C then the sequence /i is convergent according to the argument used for the first statement. If x 2 C and for each i 2 ZP0 there exists i P i⁄ such that /i has trivial domain and is given by /i(0) = xi, we can select this corresponding subsequence which is again convergent. Finally, consider the situation where x 2 C and for each i 2 ZP0 there exists i P i⁄ such that xi 2 C. Then the result follows from Goebel and Teel (2006, Theorem 4.4) with the local eventual boundedness assumption of that theorem following from Assumption 5; see (Goebel and Teel, 2006, Theorem 4.6). h

þ Wð/ðs ðzÞ; zÞÞ expðs ðzÞÞ ¼ WðzÞ ¼ Vðz; s; f; qÞ This calculation also applies to the case where s⁄(z) = 1 since lims?1 W(/(s, z))exp(s) = 0 due to (52c). In addition to establishing (16d), this calculation implies (44) and thus (43b). Now let (z, s, f, q) 2 D0  {0}. These jumps land in D1  {1} where V takes the value W(g) for g 2 fGDa ðz; v Þ þ ð1  fÞGC a ðz; v Þ. Using (52b) and (52d), we have

Z

sup

Rm g2fG ðz;v 1 Þþð1fÞG ðz;v 1 Þ

bD

WðgÞl1 ðdv 1 Þ

bC

6 WðzÞ  ð1  fÞhrWðzÞ; fa ðzÞi  qðzÞ ¼ Vðz; s; f; qÞ  qðzÞ Thus (48b) holds. h

b : Rm  Rn be measurLemma 8.2. Let Assumption 5 hold and let G able with closed values. Then the set-valued mapping M : Rm  X b v ÞÞ is measurable, in the sense that defined as Mðv Þ :¼ S 0 ð Gð M1 ðCÞ 2 BðRm Þ for each closed set C  X. Proof. The proof is based on Rockafellar and Wets (1998, Thm. 5.25(b), Thm. 14.13). Let C  X be closed. We show b v Þ \ S 1 ðCÞ – M 1 ðCÞ  BðRm Þ. Note that M 1 ðCÞ ¼ fv 2 Rm : Gð 0 1 n £g, where S 0 ðCÞ :¼ fn 2 R : S 0 ðnÞ \ C – £g. It is enough to 1 n show that S 0 ðCÞ is closed. Let the sequence ðnk ; /k Þ 2 R  X satisfy /k 2 S 0 ðnk Þ \ C with nk converging to n. Due to Lemma 8.1, we can assume without loss of generality that /k converges to some / 2 S 0 ðnÞ. Since C is closed and /k 2 C for all k 2 ZP0 , it follows that / 2 C and thus n 2 S 1 0 ðCÞ. This fact establishes the result. h

8. Proofs of Theorems 4.1, 4.2 and 7.1 Lemma 8.3. If Assumption 5 holds and w : S 0  R ! RP0 is upper semicontinuous then g : Rn  R  Rm ! RP0 , given as

8.1. Preliminaries This subsection does not depend on the previous results. It uses the following assumption: Assumption 5. There are no finite escape times for (1a), Assumption 1 holds, v ´ G(x, v) is measurable for each x 2 Rn , and G(x, v) = £ for x R D. Let X :¼ fM : R  Rn jMosc; dom M – £g. According to Rockafellar and Wets (1998, Thm. 5.50), X admits the graph distance as a metric with which it is a separable, locally compact, complete metric space. Let S 0 : Rn  X be such that, for n 2 C; S 0 ðnÞ is the set of solutions to (1a) having a closed graph (thus belonging to X) and, for n 2 Rn n C; S 0 ðnÞ is the function / with domain {0} satisfying / (0) = n. Define S 0 :¼ f/ 2 X : / 2 S 0 ðxÞ for some x 2 Rn g and, for ð/; TÞ 2 S 0  R, define

>1;T ð/Þ :¼ supft 2 dom / \ ½0; maxf0; Tgg

ð56aÞ

>2;T ð/Þ :¼ /ð>1;T ð/ÞÞ

ð56bÞ

gðx; s; v Þ :¼

sup

wð/; sÞ 8ðx; s; v Þ 2 Rn  R  Rm

/2S 0 ðGðx;v ÞÞ

ð57Þ

with the convention that g(x, s, v) = 0 when G(x, v) = £, is well defined, upper semicontinuous in (x, s), measurable in v, and if G(x, v) is not empty then there exists / 2 S 0 ðGðx; v ÞÞ such that g(x, s, v) = w(/⁄, s).  > 0 such that gðx; s; v Þ 2 ½0; g   for all In addition, if there exists g R ðx; s; v Þ 2 Rn  R  Rm then ðx; sÞ # Rm gðx; s; v Þlðdv Þ is well de , and is upper semicontinuous. fined, takes values in ½0; g Proof. Let ðx; s; v Þ 2 Rn  R  Rm and let G(x, v) be nonempty. Using the local boundedness and outer semicontinuity in x for each v for G provided by Assumption 5 via Assumption 1, it follows that G(x, v) is compact. Suppose there exists a sequence (gi, /i) with gi 2 G(x, v) and /i 2 S 0 ðg i Þ such that w(/i, s) ? 1 as i ? 1. Then, from the compactness of G(x, v) and Lemma 8.1, the sequence (gi, /i) has a subsequence converging to some (g⁄, /⁄) such that g⁄ 2 G(x, v) and / 2 S 0 ðg  Þ and, by the upper semicontinuity of w, lim supi?1

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

18

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

w(/i, s) 6 w(/⁄, s) < 1. By the same argument, there exists (g⁄, /⁄) with g⁄ 2 G(x, v) and / 2 S 0 ðg  Þ such that g(x, s, v) = w(/⁄, s). To see that (x, s) ´ g(x, s, v) is upper semicontinuous, let (xi, si) ? (x, s), with 0 < gðxi ; si ; v Þ ¼ wð/i ; si Þ, where /i 2 S 0 ðGðxi ; v ÞÞ and suppose lim supi?1g(xi, si, v) > g(x, s, v). Without loss of generality, perhaps by extracting a subsequence, we assume there exists e > 0 such that, that for all i sufficiently large,  w /i ; si P gðx; s; v Þ þ e. According to Lemma 8.1 and the outer  semicontinuity of x ´ G(x, v), the sequence xi ; /i has a subsequence converging to (x, /) with / 2 S 0 ðGðx; v ÞÞ. Again without relabeling the since w is upper semicontinuous,  subsequence, lim supi!1 w /i ; si 6 wð/; sÞ 6 gðx; s; v Þ. This contradiction establishes the result. To see that v ´ g(x, s, v) is measurable, note that v # S 0 ðGðx; v ÞÞ is measurable according to Lemma 8.2. Next, since w is upper semicontinuous and S 0 is closed according to Lemma 8.1, the set Cc;s :¼ f/ 2 S 0 : wð/; sÞ P cg is closed for each c 2 RP0 and each s 2 R. Then

fv 2 Rm : gðx; s; v Þ P cg ¼ fv 2 Rm : S 0 ðGðx; v ÞÞ \ Cc;s – £g Since v # S 0 ðGðx; v ÞÞ is measurable, this set is measurable. The upper semicontinuity in the last statement of the lemma follows from the boundedness of g, the upper semicontinuity of (x, s) ´ g(x, s, v), and Fatou’s lemma or the Dominated Convergence Theorem (Fristedt and Gray, 1997, Lemma 8 or Thm. 9, Section 8.2). h Lemma 8.4. If Assumption 5 holds and w : R  Rn ! RP0 is upper semicontinuous then w2 : S 0  R ! RP0 defined by (/, s) ´ w2(/, s) :¼ w(s  >1,s(/),>2, s(/)) is upper semicontinuous.

  p\S ðs; xÞ ¼ E max I\ðC6s SÞ ðgraphi ðxÞÞ

since the quantity inside the expected value equals one if the condition inside the probability in (58) is true, and it equals zero otherwise. The quantity pS ðs; xÞ can also be bounded by expressions generated by dynamic programming. For ðs; xÞ 2 RP0  Rn , define m\S(s, x) as follows: variables ðr; n; /Þ 2 R  Rn  S 0 m"\S ð0; r; nÞ :¼ 0

ð61aÞ

n"\S ð0; r; /Þ :¼ I\ð½0;rSÞ ðgraphð/ÞÞ for k ¼ 1 : bsc Z sup n"\S ðk  1; r  1; /Þlðdv Þ m"\S ðk; r; nÞ :¼

ð61bÞ

Rm /2S 0 ðGðn;v ÞÞ

n"\S ðk; r; /Þ m\S ðs; xÞ :¼

lim sup wðsi  >1;si ð/i Þ; >2;si ð/i ÞÞ 6 wðs  >1;s ð/Þ; >2;s ð/ÞÞ i!1

This bound establishes the result. h The proof of next lemma is straightforward. The statement uses indicator functions defined in Appendix A. In addition, it uses graphs ð/Þ :¼ graphð/Þ \ ð½0; s  Rn Þ. Lemma 8.5. If S  Rn is closed then the mappings ð/; sÞ # I\ð½0;sSÞ ðgraphð/ÞÞ and ð/; sÞ # IðRSÞ ðgraphs ð/ÞÞ (see Appendix A and definition of graphs(/) above) are upper semicontinuous on S 0  R. If w1 and w2 are upper semicontinuous then (/, s) ´ max{w1(/ , s),w2(/, s)} and (/, s) ´ w1(/, s)w2(/, s) are upper semicontinuous.

:¼ max I\ð½0;rSÞ ðgraphð/ÞÞ; m"\S ðk; r max sup n"\S ðk; ; /Þ k2f0;...;bscg/2S ðnÞ 0

ð61cÞ

  >1;r ð/Þ; >2;r ð/ÞÞ ð61dÞ

s

ð61eÞ

Proposition 8.1. Let Assumption 5 hold. For each closed S  Rn and ðs; xÞ 2 RP0  ðC [ DÞ; m\S ðs; xÞ is well defined and, for each x 2 S r ðxÞ, we have p\S(s, x) 6 m\S(s, x). Moreover, (s, x) ´ m\S(s, x) is upper semicontinuous. Proof. That the function m\S is well defined and upper semicontinuous follows from the combination of Lemmas 8.3, 8.4, and 8.5. Let x 2 C [ D; s 2 Rn and x 2 S r ðxÞ be given. According to (2l) in Proposition 2.1, there exist F i1 -measurable mappings ti, i 2 {1, . . . , bsc + 1}, such that, with t0(x) = 0 for all x 2 X,

dom xðxÞ \ C6s ¼ Proof. Let the sequence ð/i ; si Þ 2 S 0  R converge to some ð/; sÞ 2 S 0  R. It is immediate from the definitions in (56) that limi!1 >1;si ð/i Þ ¼ >1;s ð/Þ and limi!1 >2;si ð/i Þ ¼ >2;s ð/Þ. Then, by the upper semicontinuity of w

ð60Þ

i2f0;...;bscg

bsc [ ð½ti ðxÞ; tiþ1 ðxÞ  figÞ 8x 2 X:

ð62Þ

i¼0

By F 0 -measurability of t1, there exists t1 2 [0, s] such that t1(x) = t1 for all x 2 X. On the other hand, ti(x) may be empty for some pairs ðx; iÞ 2 X  ZP2 . Necessarily, ti+1(x) + i 6 s for all i 2 {0, . . . ,bsc} and all x 2 dom ti+1. Also, from (2m) in Proposition 2.1, x(ti+1, i) is measurable for each i 2 {0, . . . , bsc}. For ‘ 2 {0, . . . , bsc} and h 2 {0, . . . , ‘}, define

N"\S ðx; s; h; ‘Þ :¼ max  max I\ðC6s SÞ ðgraphi ðxÞÞ; I\ðC6s Rn Þ ðgraph‘ ðxÞÞm"\S ðbsc i2fh;...;‘g

‘; s  t‘þ1  ‘; xðt‘þ1 ; ‘ÞÞg

ð63Þ

For ‘ 2 {0, . . . , bsc  1} we have

N"\S ðx;

s; ‘ þ 1; ‘ þ 1Þ

maxfI\ðC6s SÞ ðgraph‘þ1 ðxÞÞ; I\ðC6s Rn Þ ðgraph‘þ1 ðxÞÞ  m" ðbsc  ‘  1; r‘þ1  1  ~t‘þ2 ; xðt‘þ2 ; ‘ þ 1ÞÞg

ð64Þ

\S

8.2. Reachability probabilities and dynamic programming

where r‘+1 :¼ s  t‘+1  ‘and et ‘þ2 :¼ t‘þ2  t‘þ1 6 r‘þ1  1, and

This subsection uses results from the previous subsection as well as material from Sections 2.1, 2.2, and 2.3. Given a closed set S  Rn ; s P 0; x 2 C [ D, and x 2 S r ðxÞ, define

N"\S ðx; s; 0; ‘ þ 1Þ   max max I\ðC6s SÞ ðgraphi ðxÞÞ; N "\S ðx; s; ‘ þ 1; ‘ þ 1Þ

p\S ðs; xÞ :¼ PðgraphðxÞ \ ðC6s  SÞ – £Þ

ð58Þ

This quantity represents the probability that x reaches the set S within hybrid time s. It can be expressed in terms of an expected value by defining

graphi ðxÞ :¼ graphðxÞ \ ðR  fig  Rn Þ 8i 2 ZP0 and noting that

ð59Þ

i2f0;...;‘g

ð65Þ

where r‘+1 :¼ s  t‘+1  ‘and et ‘þ2 :¼ t‘þ2  t‘þ1 6 r‘þ1  1, and

N"\S ðx;

s; 0; ‘ þ 1Þ

max



max I\ðC6s SÞ ðgraphi ðxÞÞ; N "\S ðx; s; ‘ þ 1; ‘ þ 1Þ

i2f0;...;‘g

 ð65Þ

We claim that, for each ‘ 2 {0, . . . , bsc}

E½N"\S ðx; s; 0; ‘Þ 6 m\S ðs; xÞ

ð66Þ

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

19

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

For ‘ = 0, using (63) with h = ‘ = 0, (61d), (61e), and F 0 -measurability of graph0(x), we have

E½N"\S ðx; s; 0; 0Þ ¼ E½maxfI\ðC6s SÞ ðgraph0 ðxÞÞ; I\ðC6s Rn Þ ðgraph0 ðxÞÞm"\S ðbsc; s  t1 ; xðt 1 ; 0ÞÞg 6 sup n"\S ðbsc; s; xÞ 6 m\S ðs; xÞ

Corollary 8.1. Let Assumption 5 hold, let V be a certification candidate for (C, D, G, l) and suppose (16a), (16c), and (16d) are satisfied. For each D > 0 and S :¼ Rn n ðA þ DB Þ, we have m\S(s, x) 6 V(x)/ a1(D) for all ðs; xÞ 2 R  ðC [ DÞ. Proof. With c :¼ a1(D), (16a) gives (68). The result then follows from Proposition 8.2. h

/2S 0 ðxÞ

Suppose (66) holds for some ‘ 2 {0, . . . , bsc  1}. Using (61c), (61d), (64), the F ‘ -measurability of graph‘(x), and I\ðC6s Rn Þ ðgraph‘ ðxðxÞÞÞ ¼ 0 implies N "\S ðxðxÞ; s; ‘ þ 1; ‘ þ 1Þ ¼ 0, it follows that wpo

E½N"\S ðx; s; ‘ þ 1; ‘ þ 1ÞjF ‘  6 I\ðC6s Rn Þ ðgraph‘ ðxÞÞ Z  sup n"\S ðbsc  ‘  1; s  t‘þ1  ‘  1; /Þlðdv Þ Rm /2S 0 ðGðxðt‘þ1 ;‘Þ;v ÞÞ

I\ðC6s Rn Þ ðgraph‘ ðxÞÞm"\S ðbsc  ‘; s  t‘þ1  ‘; xðt‘þ1 ; ‘ÞÞ Therefore, using (65) and the fact that

8.4. Proof of Theorem 4.1 Under the theorem’s assumptions and due to Corollary 3.2, Assumption 5 holds. Since a1 ; a2 2 G1 , given d > 0 and . > 0, we can pick e > 0 sufficiently large so that c :¼ a1(e) P a2(d)/ .. Define S :¼ Rn n ðA þ eB Þ. Using Corollary 8.1, Proposition 8.1, and (16b) we get that p\S(s, x) 6 a2(-(x))/c for all x 2 C [ D and x 2 S r ðxÞ, where p\S was defined in (58). According to that definition and the definition of S

PððgraphðxÞ \ ðC6s  Rn ÞÞ  R2  ðA þ eB ÞÞ ¼ 1  p\S ðs; xÞ

max I\ðC6s SÞ ðgraphi ðxÞÞ 2 f0; 1g

P1.

i2f0;...;‘g

it follows that

Since s is arbitrary, it follows that (3) holds.



E N"\S ðx; s; 0; ‘ þ 1Þ   

¼ E max max I\ðC6s SÞ ðgraphi ðxÞÞ; E N "\S ðx; s; ‘ þ 1;‘ þ 1ÞjF ‘ i2f0;...;‘g   6 E max max I\ðC6s SÞ ðgraphi ðxÞÞ; I\ðC6s Rn Þ ðgraph‘ ðxÞÞm"\S ðbsc i2f0;...;‘g

‘; s  t‘þ1  ‘;xðt‘þ1 ; ‘ÞÞg ¼ E N "\S ðx; s; 0; ‘Þ 6 m\S ðs; xÞ

8.5. Proof of Theorem 4.2 Using that a1 ; a2 2 K1 , let e > 0 and . > 0 be given and pick d > 0 such that c :¼ a1(e) P a2(d)/.. The rest of the proof is the same as the proof of Theorem 4.1.

The claim (66) for all ‘ 2 {0, . . . , bsc} now follows by induction. Using (60) and (61a), and the definition (63) with h = 0 and ‘ = bsc, the result follows from (66) with ‘ = bsc. h

8.3. Linking reachability probabilities and Lyapunov functions

8.6. Proof of Theorem 7.1 To distinguish the variable s in m\S(s, x) from the variable s in ~ for the latter. Also, x ¼ ðzT ; s ~; f; qÞT denotes the state (38), we use s n of (38). Let e > 0 and define Sa :¼ R n ðAa þ eB Þ; c :¼ a1 ðeÞ, and S :¼ Sa  R3 . From the lower bound in (43a)

Lemma 8.6. If V is a certification candidate for (C, D, G, l) and (16c) and (16d) hold then, for each x 2 D, the following integral is well defined and satisfies

cISa ðzÞ ¼ cIS ðxÞ 6 VðxÞ 8x 2 D

Z

c sup

Rm f/2S 0 ðGðx;v ÞÞ; t2 dom /g

Vð/ðtÞÞlðdv Þ 6 VðxÞ

ð67Þ

Proof. By (16c), for each g 2 G(x, v), V(/(t)) 6 V(g) for each / 2 S 0 ðgÞ and t 2 dom /. Thus, the integrand in (67) equals supg2G(x, v)V(g). Hence, integral in (67) equals the integral in (16d) from which the result follows. h Proposition 8.2. If Assumption 5 holds, S  Rn is closed, the conditions of Lemma 8.6 hold, and there exists c > 0 such that

IS ðxÞ 6 VðxÞ=c 8x 2 C [ D [ GðD  VÞ then m\S ðs; xÞ 6

VðxÞ

c

ð68Þ

for all ðs; xÞ 2 RP0  ðC [ D [ GðD  VÞÞ.

Proof. From (16c) and the definition of S 0 , it is enough to show that for all ðr; /Þ 2 R  S 0 ðC [ D [ GðD  VÞÞ and k 2 {0, . . . , bsc}

n"\S ðk; r; /Þ 6

max

s2½0;maxf0;rg

Vð/ðsÞÞ=c:

ð69Þ

This bound holds for k = 0 due to (68) and (61b). Suppose (69) holds for some k 2 ZP0 . Then (61c) and Lemma 8.6 give

m"\S ðk þ 1; r; nÞ 6 VðnÞ=c 8n 2 C [ D [ GðD  VÞ:

ð70Þ

Now it follows from (61d) that (69) holds with k replaced by k + 1. Hence, the result follows by induction. h

ð71Þ

We claim that

m"\S ðk; r; xÞ

6 VðxÞ 8ðk; r; xÞ 2 ZP0  R  ðC [ DÞ

ð72Þ

m"\S ð0; r; xÞ

The bound holds for k = 0 since ¼ 0 for all ðr; xÞ 2 R  ðC [ DÞ. In fact, the bound holds for all ðk; r; xÞ 2 ZP0  R  ðC n DÞ since, according to Corollary 3.2, we can consider that G(x, v) = £ for x 2 CnD. Suppose the bound holds for some k 2 ZP0 and all ðr; xÞ 2 R  ðC [ DÞ. Suppose that either x 2 D0  {0} and ~þ ¼ v 0 ; s ~_ ¼ 1 p = 1 or x 2 D1  {1}. In either case, we have that s ~ ¼ 0, during subsequent flows, and flowing is not possible from s so the maximum flow time is v0. Given v1, let v 0 be the smallest value of v 0 2 RP0 such that there exist / 2 S 0 ðGðx; v ÞÞ and s 2 dom / \ [0, max{0, min{v0, r  1}}] such that /(s) 2 S. It may be that v 0 ¼ 1.   When v 0 < 1, note  that v 0 6 maxf0; r  1g and v 0 is the smallest time such that / v 0 2 S for some / 2 S 0 ðGðx; v ÞÞ. Let /⁄ denote a  solution satisfying / v 0 2 S. Also define 9 8 sup > > = < Vð/ðsÞÞ=c m ðx; v Þ :¼ / 2 S 0 ðGðx; v ÞÞ > > ; : s 2 dom / \ ½0; maxf0; minfv 0 ; r  1gg By the definition of

v 0 , (72), and the definition of n"\S

sup n"\S ðk; r  1;/Þ 6 max /2S 0 ðGðx;v ÞÞ

in (61d)

   IRP0 v 0  v 0 ; IR<0 v 0  v 0 m ðx; v Þ

Thus, from (61c), we get the upper bound ! Z Z m"\S ðk þ 1;r;xÞ 6 l0 ð½v 0 ; 1ÞÞ þ m ðx; v Þl0 ðdv 0 Þ l1 ðdv 1 Þ Rm

ð73Þ

½0;v 0 Þ

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

20

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

However, using (16c) and (16d) and the definition of m⁄(x, v) Z VðxÞ=c P m ðx; v Þlðdv Þ Rmþ1 0 1 , Z Z Z      @ P V / v0 cþ m ðx; v Þl0 ðdv 0 ÞAl1 ðdv 1 Þ Rm

½0;v 0 Þ

R Pv 

0

ð74Þ    according to (71) and the fact that If / v 0 2 D then, / v 0 2 S; V / v 0 =c P 1 so that the combination of (73) and  (74) gives m"\S ðk þ 1; r; xÞ 6 VðxÞ=c. If / v 0 R D, from the structure of the solutions and the definition of C, there exists z⁄, f⁄, q⁄   such that z ; v 0 ; f ; q 2 C and for each v 0 P v 0 ; / v 0 ¼  T   T z ; v 0  v 0 ; f ; q . In this case, using (43b), we again get that m"\S ðk þ 1; r; xÞ 6 VðxÞ=c. For x 2 [ i2{0,1}(Di  {i}), flowing is not possible and so sup/2S0 ðxÞ n"\S ðk; r; xÞ 6 VðxÞ=c for all ðk; rÞ 2 ZP0  R. From here the proof follows the proof of Theorems 4.1 and 4.2.

Proposition 9.1. Let Assumption 5 hold. For each closed S  Rn ; s 2 RP0 , and x 2 C [ D, the value mS(s, x) is well defined and, for each x 2 S r ðxÞ; pS ðs; xÞ 6 mS ðs; xÞ. Moreover (s, x) ´ mS(s, x) is upper semicontinuous.

Proof. That the functions m\S are well defined and upper semicontinuous follows from the combination of Lemmas 8.3, 8.4, and 8.5. Let x 2 C [ D; x 2 S r ðxÞ and s P 0 be given. For ‘ 2 {0, . . . , dse} and h 2 {0, . . . , ‘}, define

N"S ðx; s; h; ‘Þ ( ‘ Y IðR2 SÞ ðgraphj;s ðxÞÞ  I\ðR2 Rn Þ ðgraph‘ ðxÞÞm"S ðdse :¼ max j¼h

 ‘; s  t‘þ1  ‘; xðt‘þ1 ; ‘ÞÞ; max

i2fh;...;‘g

i Y

IðR2 SÞ ðgraphj;s ðxÞÞ

j¼h

I\ðR2 Rn Þ ðgraphi ðxÞÞm"S ð0; s  tiþ1  i; xðtiþ1 ; iÞÞ 9. Proof of Theorem 4.3

N"S ðx;

pS ðs; xÞ :

‘  1; r‘þ1  1  ~t‘þ2 ; xðt‘þ2 ; ‘ þ 1ÞÞI\ðR2 Rn Þ   graph‘þ1 ðxÞÞm"S ð0; r‘þ1  1  ~t‘þ2 ; xðt‘þ2 ; ‘ þ 1ÞÞ

¼ PððgraphðxÞ \ ðCPs  Rn Þ – £Þ ^ ððgraphðxÞ \ ðC6s  Rn ÞÞ  ðR2  SÞÞÞ

ð75Þ

where ^ denotes the logical ‘‘and’’ operation. The value pS(s, x) denotes the probability that x remains in S for time less than or equal to s and that, moreover, the solution does not stop before time s. It is convenient to use (2h) with C ¼ Rn and i = dse to truncate the domains of x to their smallest hybrid time greater than or equal to s, if such a time exists. Hence, also using (2l), without loss of generality we can assume there exist F i1 -measurable mappings ti, i 2 {1, . . . , dse + 1} (and t0(x) = 0) such that

ð½ti ðxÞ; tiþ1 ðxÞ  figÞ 8x 2 X

ð76Þ

i¼0

where necessarily, for all i 2 {0, . . . , dse} and all x 2 dom ti+1, either ti+1(x) + i 6 s or ti+1(x)  ti(x) = 0. Then

" pS ðs; xÞ ¼E

max

i Y

i2f0;...;dseg

IðR2 SÞ ðgraphj;s ðxÞÞ

j¼0

I\ðR2 Rn Þ ðgraphi ðxÞÞIR60 ðs  tiþ1  iÞ

i

ð77Þ

where in a refinement of (59)

graphj;s ðxÞ :¼ graphj ðxÞ \ ðC6s  Rn Þ

ð78Þ

The quantity pS(s, x) can be bounded by expressions generated by dynamic programming. For ðs; xÞ 2 RP0  Rn we define m\S(s, x), using graphr ð/Þ :¼ graphð/Þ \ ð½0; r  Rn Þ, as follows: variables ðr;n;/Þ 2 R  Rn  S 0 m"S ð0;r;nÞ :¼ IR60 ðrÞ

ð79aÞ

n"S ð0;r;/Þ

ð79bÞ

:¼ IðRSÞ ðgraphr ð/ÞÞIR60 ðr  >1;r ð/ÞÞ

for k ¼ 1 : dse m"S ðk;r;nÞ :¼

Z

sup

Rm /2S 0 ðGðn;v ÞÞ

n"S ðk  1;r  1;/Þlðdv Þ

n"S ðk;r; /Þ :¼ IðRSÞ ðgraphr ð/ÞÞmax m"S ði;r  >1;r ð/Þ;>2;r ð/ÞÞ i2f0;kg

mS ðs;xÞ :¼

max

sup n"S ðk; s; /Þ

k2f0;...;dseg/2S ðxÞ 0

s; ‘ þ 1; ‘ þ 1Þ

n IðR2 SÞ ðgraph‘þ1;s ðxÞÞ  max I\ðR2 Rn Þ ðgraph‘þ1 ðxÞÞm"S ðdse

For a closed S  Rn ; s P 0; x 2 C [ D, and x 2 S r ðxÞ, define

dse [

ð80Þ

For ‘ 2 {0, . . . , dse  1} we have

9.1. Viability probabilities and dynamic programming

dom xðxÞ ¼

o

ð79cÞ

ð81Þ

where r‘+1 :¼ s  t‘+1  ‘and ~t‘þ2 :¼ t‘þ2  t‘þ1 6 maxf0; r‘þ1  1g, and

N"S ðx; s; 0; ‘ þ 1Þ ( ‘ Y IðR2 SÞ ðgraphj;s ðxÞÞ  N"S ðx; s; ‘ þ 1; ‘ þ 1Þ; max j¼0 i Y IðR2 SÞ ðgraphj;s ðxÞÞ  I\ðR2 Rn Þ ðgraphi ðxÞÞm"S max

i2f0;...;‘g

j¼0

ð0; s  tiþ1  i; xðtiþ1 ; iÞÞg

ð82Þ

We claim that for each ‘ 2 {0, . . . , dse}

E½N"S ðx; s; 0; ‘Þ 6 mS ðs; xÞ

ð83Þ

For ‘ = 0, using (80) with h = ‘ = 0, (79d) and (79e), and F 0 -measurability of graph0(x), we have



E N "S ðx; s; 0; 0Þ h n ¼ E max IðR2 SÞ ðgraph0;s ðxÞÞ  I\ðR2 Rn Þ ðgraph0 ðxÞÞm"S ðdse; s  t 1 ; xðt 1 ; 0ÞÞ; oi IðR2 SÞ ðgraph0;s ðxÞÞm"S ð0; s  t 1 ; xðt1 ; 0ÞÞ 6 sup n"S ðdse; s; xÞ 6 mS ðs; xÞ /2S 0 ðxÞ

Suppose the bound (83) holds for some ‘ 2 {0, . . . , dse  1}. Using (79c), (79d), and (81), F ‘ -measurability of graph‘(x), and I\ðR2 Rn Þ ðgraph‘ ðxðxÞÞÞ ¼ 0 implies N "S ðxðxÞ; s; ‘ þ 1; ‘ þ 1Þ ¼ 0, it follows that



wpo E N "S ðx; s; ‘ þ 1; ‘ þ 1ÞjF ‘ 6 I\ðR2 Rn Þ ðgraph‘ ðxÞÞ Z sup n"S ðdse  ‘  1; s  t‘þ1  ‘  Rm /2S 0 ðGðxðt‘þ1 ;‘Þ;v ÞÞ

ð79dÞ

 1; /Þlðdv Þ

ð79eÞ

I\ðR2 Rn Þ ðgraph‘ ðxÞÞm"S ðdse  ‘; s  t‘þ1  ‘; xðt‘þ1 ; ‘ÞÞ

ð84Þ

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

21

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

Therefore, using (82) and that the second term in the maximum in (82) takes values in {0, 1} due to (79a), it follows from (82) and (84) that

E N"S ðx; s; 0; ‘ þ 1Þ " ( ‘ Y ¼ E max IðR2 SÞ ðgraphj;s ðxÞÞ  N "S ðx; s; ‘ þ 1;‘ þ 1Þ;

q maxf0; sgn"S ðk  1; s  1; /Þ 6 qðmaxf0; s  >1 ð/Þ  1g þ >1 ð/Þ þ 1Þn"S ðk  1; s  1; /Þ 6 IðRSÞ ðgraphð/ÞÞðVð>2 ð/ÞÞ þ qð>1 ð/Þ þ 1ÞÞ Then, using Lemma 9.1, we get

j¼0

max

i2f0;...;‘g

i Y

IðR2 SÞ ðgraphj;s ðxÞÞ  I\ðR2 Rn Þ ðgraphi ðxÞÞ

j¼0



m"S ð0; s  tiþ1  i;xðtiþ1 ; iÞÞ " ( ‘ Y

IðR2 SÞ ðgraphj;s ðxÞÞ  E N"S ðx; s; ‘ þ 1; ‘ þ 1ÞjF ‘ ; ¼ E max j¼0

max

i2f0;...;‘g

i Y

q maxf0; sgm"S ðk; s; xÞ ¼

Z

sup

q maxf0; sgn"S ðk  1; s  1; /Þlðdv Þ

Rm /2S 0 ðG\ ðx;v ÞÞ

Z 6

Rm

ðVð>2 ð/ÞÞ sup / 2 S 0 ðG\ ðx; v ÞÞgraphð/Þ  ðR  SÞ

þ qð>1 ð/Þ þ 1ÞÞlðdv Þ 6 VðxÞ

The result of the proposition now follows by induction. h

IðR2 SÞ ðgraphj;s ðxÞÞ  I\ðR2 Rn Þ

j¼0





graphi ðxÞÞm"S ð0; s  tiþ1  i;xðtiþ1 ; iÞÞ " ( ‘ Y 6 E max IðR2 SÞ ðgraphj;s ðxÞÞ  I\ðR2 Rn Þ ðgraph‘ ðxÞÞm"S ðdse

Corollary 9.1. If S  Rn is closed and Assumption 5 and the conditions of Lemma 9.1 hold q max{0,s} mS(s, x) 6 V(x) holds for all ðs; xÞ 2 R  ðC [ DÞ.

j¼0

‘; s  t‘þ1  ‘;xðt‘þ1 ; ‘ÞÞ; max

i2f0;...;‘g

i Y IðR2 SÞ ðgraphj;s ðxÞÞ j¼0

I\ðR2 Rn Þ ðgraphi ðxÞÞm"S ð0; s  tiþ1  i; xðtiþ1 ; iÞÞ

oi



 ¼ E N "S ðx; s; 0; ‘Þ 6 mS ðs; xÞ

The claim (83) for ‘ 2 {0, . . . , ds e} now follows by induction. With h = 0 and ‘ = ds e in (80), (83), and (77) give pS(s, x) 6 mS (s, x) since m"S ð0; s  tiþ1  i; xðtiþ1 ; iÞÞ ¼ IR60 ðs  tiþ1  iÞ. This bound establish the proposition. h 9.2. Linking viability probabilities and Foster functions Lemma 9.1. Define C\ :¼ C \ S, D\ :¼ D \ S, and G\(x, v) :¼ G(x, v) \ S. If V is a certification candidate for (C\, D\, G\, l) and the conditions (18b) and (18c) hold then, for each x 2 D, the following integral is well defined and satisfies

Z Rm

sup 8 9ðVð/ðtÞÞ þ qðt þ 1ÞÞlðdv Þ 6 VðxÞ / 2 S ðG ðx; v ÞÞ > > 0 \ < = graphð/Þ  R  S > > : ; t 2 dom /

Proof. It follows by combining (18b), (18c) that

Z

Rm

Z 6

sup 8 9ðVð/ðtÞÞ þ qðt þ 1ÞÞlðdv Þ / 2 S ðG ðx; v ÞÞ > > 0 \ < = graphð/Þ  R  S > > : ; t 2 dom / sup ðVðgÞ þ qÞlðdv Þ 6 VðxÞ 

Rm g2G\ ðx;v Þ

Proposition 9.2. If S  Rn is closed and Assumption 5 and the conditions of Lemma 9.1 hold then q maxf0; sgm"S ðk; s; xÞ 6 VðxÞ holds for all ðk; s; xÞ 2 ZP0  R  Rn . Proof. The bound holds true for k = 0 by the fact that maxf0; sgIR60 ðsÞ ¼ 0 6 VðxÞ for all ðs; xÞ 2 R  Rn . Now suppose the bound holds for some k 2 ZP0 and all ðs; xÞ 2 R  Rn . Then, using the definition of n"S in (79d), we get

9.3. Proof of Theorem 4.3 Define S :¼ Rn n O. Using Proposition 9.1, the properties of a Foster function, Corollary 9.1, and (16b), we get that pS(s, x) 6 a2(-(x))/(qs) for all x 2 C [ D and s 2 R>0 . Given R > 0 and . > 0, pick s > 0 so that a2(R)/(qs) 6 .. Thus pS(s, x) 6 . for all x 2 C [ D satisfying -(x) 6 R. It then follows from the definition of pS in (75) that

PððgraphðxÞ  ðC
ð85Þ

It follows that (5) holds. 10. Proof of Theorem 3.1 and Proposition 3.1 This section has connections to the results in Section 8.1. It relies on the following strengthening of Assumption 5: Assumption 6. There are no finite escape times for (1a), Assumptions 1 and 2 hold, and G(x, v) = £ for x R D. For each T P 0, let XT  X be such that / 2 XT implies graphð/Þ  ½0; T  Rn . The set XT is closed. Given i 2 ZP0 and ðv 1 ; v 2 ; . . .Þ 2 ðRm Þ1 , define ~ v i :¼ ðv 1 ; . . . ; v i Þ 2 ðRm Þi . 10.1. Causal, measurable selections Proposition 10.1. Suppose Assumption 6 holds and u : S 0 ! RP0 is upper semicontinuous. Let i 2 ZP0 and x : ðRm Þi  Rn be measurable and locally bounded with closed values. Then the setvalued mapping M : ðRm Þiþ1  X defined as Mð~ v iþ1 Þ :¼ argmax/2S0 ðGðxð~v i Þ;v iþ1 ÞÞ\X T uð/Þ has closed values and is measurable, i.e., M 1 ðCÞ  BððRm Þiþ1 Þ for each closed set C  X.

v iþ1 be Proof. First we establish that M has closed values. Let ~ given. If xð~ v i Þ is empty then Mð~ v iþ1 Þ is empty and thus closed. If xð~ v i Þ is nonempty then it is compact, since it is closed and the mapping x is locally bounded. In turn, Gðxð~ v i Þ; v iþ1 Þ is compact, due to the local boundedness of G and the outer semicontinuity of x ´ G(x, v) for each v 2 Rm ; see (Rockafellar and Wets, 1998, Thm. 5.25(a)). Then S 0 ðKÞ is closed for each compact K  Rn due to the properties of S 0 ; see Lemma 8.1 and compare with (Rocka-

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

22

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

fellar and Wets, 1998, Thm. 5.25(a)). Thus, S 0 ðGðxð~ v i Þ; v iþ1 ÞÞ is closed. Finally

Mð~ v iþ1 Þ ¼ Su ðgðxð~ v i Þ; v iþ1 ÞÞ \ X T where g was defined in (57) and Su : R  X is defined as Su ðcÞ :¼ f/ 2 S 0 ðGðxð~ v i Þ; v iþ1 ÞÞ : uð/Þ P cg. Observe that Su(c) is closed for each c since u is upper semicontinuous. Indeed, for /i 2 Su(c) with /i ? /, we have u(/) P lim supi?1u(/i) P c. It follows that Mð~ v iþ1 Þ is closed. Next we establish that M is measurable. Since x and G are locally bounded,

Mð~ v iþ1 Þ ¼

[

½S 0 ðGðxð~ v i Þ; v iþ1 Þ \ kBÞ \ Rk \ Su;k ðgðxð~ v i Þ; v iþ1 ÞÞ

k2ZP0

where g was defined in (57) and

Su;k ðcÞ :¼ f/ 2 S 0 ðkBÞ : uð/Þ P cg Rk :¼ f/ 2 X : graphð/Þ  ½0; T  R6T ðC \ kBÞg

Lemma 10.2. For each each closed (respectively, compact) set C1  Rnþ2 , the set C 2 :¼ f/ 2 X : graphð/Þ  C1 g  X is closed (respectively, compact).

Proposition 10.2. Let S  Rn be closed. Under Assumption 6, for each ðs; xÞ 2 RP0  ðC [ DÞ there exists x 2 S r ðxÞ such that supz2Sr ðxÞ pS ðs; zÞ ¼ pS ðs; xÞ ¼ mS ðs; xÞ. Proof. Proposition 9.1 establishes that supz2Sr ðxÞ pS ðs; zÞ 6 mS ðs; xÞ. We construct x 2 S r ðxÞ such that pS(s, x) = mS(s, x). Let k 2 {0, . . . , dse} be such that mS ðs; xÞ ¼ sup/2S0 ðxÞ n"S ðk; s; /Þ. Let / 2 S 0 ðxÞ satisfy n"S ðk; s; / Þ ¼ " ⁄ sup/2S0 ðxÞ nS ðk; s; /Þ, define t1 :¼ >1,s(/ ), and set w(t, 0) :¼ /⁄(t) for all t 2 [0, t1]. Then, for j 2 {1, . . . , k}, let /~v j 2 S 0 ðGðw~v j1 ðtj ; j  1Þ; v j ÞÞ be measurable in ~ v j and satisfy

n"S ðk  j; s  tj  j; /~v j Þ ¼

sup /2S 0 ðGðw~ v

j1

where R6T ðC \ kBÞ denotes the reachable set for x 2 C; x_ 2 FðxÞ from C \ kB in time 6T. Since x is measurable with closed values, it follows from Assumption 2 and (Rockafellar and Wets, 1998, Thm. 14.13(b)) that ~ v iþ1 # Gðxð~ v i Þ; v iþ1 Þ is measurable. Then from Rockafellar and Wets (1998, Prop. 14.11(a)) and Lemma 8.2 it follows that the mapping ~ v iþ1 # S 0 ðGðxð~ v i Þ; v iþ1 Þ \ kBÞ is measurable. It follows from Assumption 6 and (Goebel and Teel, 2006, Corollary 4.7) that Rk is compact for each k. Then, according to Himmelberg (1975, Prop. 2.1, Prop. 2.3(i) and Thm. 4.1), for the measurability of M it is enough to establish that ~ v iþ1 # Su;k ðgðxð~ v i Þ; v iþ1 ÞÞ is measurable for each k 2 ZP0 . For this purpose, it is enough to establish that S1 v iþ1 # gðxð~ v i Þ; v iþ1 Þ is u;k ðCÞ is closed for each closed C  X and ~ measurable. We first show that S1 u;k ðCÞ :¼ fc 2 R : Su;k ðcÞ \ C – £g is closed when C  X is closed. Suppose ðci ; /i Þ 2 R  X is such that /i 2 Su;k ðci Þ \ C for each i 2 ZP0 and suppose ci ? c as i ? 1. We have that u(/i) P ci and /i ð0; 0Þ 2 kB. Since the graph of /i does not escape to infinity and since C is closed, it follows from Rockafellar and Wets (1998, Thm. 5.36) that the sequence /i has a subsequence converging to some / 2 C and /ð0; 0Þ 2 kB. Since u is upper semicontinuous, we have that c = limi?1ci 6 lim supi?1u(/i) 6 u(/). Thus, / 2 Su;k ðcÞ \ C and thus c 2 S1 u;k ðCÞ, meaning that this set is closed. Finally, we show that ~ v iþ1 # gðxð~ v i Þ; v iþ1 Þ is measurable, where g was defined in (57). It is enough to show that V c :¼ f~ v iþ1 2 ðRm Þiþ1 : gðxð~ v i Þ; v iþ1 Þ P cg 2 BððRm Þiþ1 Þ for each c 2 RP0 . Observe that, by the definition of g in (57)

V c ¼ f~ v iþ1 2 ðRm Þiþ1 : S 0 ðGðxð~ v i Þ; v iþ1 ÞÞ \ Su ðcÞ – ;g: The set Su(c) is closed since u is upper semicontinuous. As established above, the mapping ~ v iþ1 # S 0 ðGðxð~ v i Þ; v iþ1 ÞÞ is measurable. Thus, by definition, V c 2 BððRm Þiþ1 Þ. h The next result is contained in Wagner (1977, Thm. 4.1), using that X is Polish, which follows from Rockafellar and Wets (1998, Thm. 5.50) and the fact that a separable, complete, locally compact metric space is Polish. Lemma 10.1. Let i 2 ZP0 and let M : ðRm Þiþ1  X be measurable and closed valued. Then M admits a measurable selection, i.e., there exists a function /: dom M ? X such that /ð~ v iþ1 Þ 2 Mð~ v iþ1 Þ for all ~ v iþ1 2 dom M.

n"S ðk  j; s  t j  j; /Þ;

ðtj ;j1Þ;v j ÞÞ

define t jþ1;~v j :¼ t j;~v j1 þ >1;stj;~v j ð/ Þ, and set w~v j ðt; jÞ ¼ /~v j j1 ðt  t j;~v j1 Þ for all t 2 ½t j;~v j1 ; t jþ1;~v j . Now define xðxÞ :¼ wðv 1 ðxÞ;...;vk ðxÞÞ . Due to the properties of w, x is a standard solution when coupled with (v1(x), . . . , vk(x)) for each x 2 X. Next we show that graph6i(x) is F i -measurable. According to Rockafellar and Wets (1998, Prop. 14.3(i)), it is enough to show that Wi :¼ fx 2 X : graph6i ðxðxÞÞ  C1 g 2 F i for each closed set C1  Rnþ2 . It is straightforward to see that Wi ¼ fx 2 X : xðxÞ 2 C2;i g, where C2;i :¼ f/ 2 X : graph6i ð/Þ  C1 g. Let wi denote the truncation of w to RP0  Z6i and note that wi is a measurable functions of (v1, . . . , vi). Then it is also clear that 1 Wi ¼ fx 2 X : ðv 1 ðxÞ; . . . ; v i ðxÞÞ 2 w1 i ðC 2;i Þg, where wi ðC2 Þ :¼ f~ v i 2 ðRm Þi : wi;~v i 2 C2;i g. Due to the measurability of wi and the fact that C2;i is closed according to Lemma 10.2, it follows that w1 ðC2;i Þ 2 BððRm Þk Þ. Hence, Wi 2 F i . Finally, for this x the inequalities in the proof of Proposition 9.1 become equalities, which establishes the result. h

10.3. Proof of Theorem 3.1 Using the assumptions of Theorem 3.1 and Corollary 3.2, we can consider that Assumption 6 holds. Due to Proposition 10.2, it is enough to show that, under the Assumptions of Theorem 3.1, m(C[D)(s, x) = 1 for each ðs; xÞ 2 RP0  ðC [ DÞ. e be the set of points x 2 D such that condition (VD) holds. Let D Then, due to (VC) or (VD) holding for every point x 2 C [ D, it follows that from each point x 2 C [ D there exists / 2 S 0 ðxÞ such that /(s) 2 C [ D for all s 2 dom / and either / is complete or there exe In turn, it follows that for every ists t 2 RP0 such that /ðtÞ 2 D. e x 2 D and every v 2 V x , there exists / 2 S 0 ðGðx; v ÞÞ such that / (s) 2 C [ D for all s 2 dom / and either / is complete or there exists e It follows by the definition of t 2 RP0 such that /ðtÞ 2 D. e and m(C[D)(1,r, x) and (VD) that m(C[D)(1,r, x) = 1 for all x 2 D r 6 1. Now suppose that m(C[D)(k  1,r  1,x) = 1 for some e Then, by the definition of k 2 ZP2 and all r 6 k and x 2 D. m(C[D)(k, r, x) and the properties given above, it follows that e Then, by induction, m(C[D)(k, r, x) = 1 for all r 6 k and x 2 D. e m(C[D)(ds e, s, x) = 1 for all x 2 D. In turn, from the properties above, sup/2S0 ðxÞ n"ðC[DÞ ðdse; s; /Þ ¼ 1 for all x 2 C [ D. The result now follows from the definition of m(C[D)(s, x).

10.2. Optimal random solutions

10.4. Proof of Proposition 3.1

The result in this section uses Proposition 10.1 and Lemma 10.1 as well as the following elementary result.

Using the assumptions of Proposition 3.1 and Corollary 3.2, we can consider that Assumption 6 holds. Due to Proposition 10.2 and

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

the equality in (85), it is enough to show that for each . > 0 and R > 0 there exists s P 0 such that

-ðxÞ 6 R ) mS ðs; xÞ 6 .

ð86Þ

Using Proposition 10.2, Proposition 9.1, and the equality in (85), it follows that ðs; xÞ # mS ðs; xÞ ¼ supx2Sr ðxÞ pS ðs; xÞ is upper semicontinuous and, due to (15)

lim mS ðs; xÞ ¼ 0 8x 2 dom

s!1

-

ð87Þ

Suppose (86) does not hold. Then, for each i 2 ZP0 there exists ni 2 C [ D with -(ni) 6 R and such that mS(i, ni) > .. Using that C [ D is closed, the sublevel sets of - are closed since - is lower semicontinuous (Rockafellar and Wets, 1998, Thm. 1.6), and the sublevel sets of - are bounded by assumption, we can assume that the sequence ni converges to some n 2 C [ D satisfying -(n) 6 R. In particular, n 2 dom -. Using (87), let i⁄ be such that mS(i⁄, n) < .. It also follows from Proposition 10.2 and the definition of pS that i P i⁄ implies mS(i, n) 6 mS(i⁄, n). Then, from the upper semicontinuity of mS we get the contradiction

. 6 lim sup mS ði; ni Þ 6 lim sup mS ði ; ni Þ 6 mS ði ; nÞ < . i!1

i!1

11. Conclusion We have presented Lyapunov-based sufficient conditions for stability and recurrence in a class of stochastic hybrid systems. These systems are a natural extension of non-stochastic hybrid systems and include the possibility of non-unique solutions. Randomness is restricted to the jump map, although it has been shown how models with spontaneous transitions can be covered. The Lyapunov conditions constitute a departure from previous results in that they are not expressed in terms of the derivative of the expected value of a function. Instead, the Lyapunov conditions are very similar to classical Lyapunov conditions. Regularity conditions on the data have been given to guarantee existence of solutions and robustness of Lyapunov conditions. Examples have been given to illustrate the theory. Acknowledgments The author gratefully acknowledges fruitful discussions with João Hespanha on stochastic systems and Rafal Goebel on measurability results from set-valued analysis. Appendix A. Notation and supplementary definitions A.1. Basic notation and definitions RP0 denotes the nonnegative real numbers, ZP0 denotes the nonnegative integers, and RP0 :¼ RP0 [ f1g. For a closed set S  Rn and x 2 Rn , jxjS :¼ infy2Sjx  yj is the Euclidean distance to S. B (resp., B ) denotes the closed (resp., open) unit ball in Rn . For a closed set S  Rn and e > 0; S þ eB (resp., S þ eB Þ denotes the set fx 2 Rn : jxjS 6 eg (resp., fx 2 Rn : jxjS < eg). A function - : Rn ! RP0 is lower semicontinuous if lim infi?1-(xi) P -(x) whenever limi?1xi = x. As noted in Rockafellar and Wets (1998, Thm. 1.6), the sublevel sets {x: -(x) 6 a} are closed for each a 2 R when - is lower semicontinuous. A function w : X ! R, where X is a metric space, is upper semicontinuous if lim supi?1w(/i) 6 w(/) whenever limi?1/i = /. For a space W and a set S  W, define the indicator function IS : W ! f0; 1g by IS ðxÞ ¼ 1 if x 2 S and IS ðxÞ ¼ 0 if x 2 WnS, which is the complement of S in W. This indicator function is upper semicontinuous when W is a metric space and S is closed. Given sets S1 ; S2  Rp , define

23

I\S1 ðS2 Þ :¼ maxx2S2 IS1 ðxÞ and IS1 ðS2 Þ :¼ 1  maxx2S2 IRn nS1 ðxÞ, with the convention that the maximums are zero when S2 = £. Given ~ v 2 ðRm Þ1 with ~ v ¼ ðv 1 ; v 2 ; . . .Þ and i 2 ZP0 , we define ~ v i :¼ ðv 1 ; . . . ; v i Þ. A function a : RP0 ! RP0 is of class K if it is continuous, strictly increasing and a(0) = 0. It is of class K1 if it is of class K and unbounded. It is of class-G1 if it is continuous, nondecreasing and unbounded. Given a closed set A  Rn ; q : Rn ! RP0 is of class PDðAÞ if it is continuous and for each d > 0 and D > 0  > 0 such that qðxÞ P q  for all x 2 ðA þ DBÞn there exists q ðA þ dB Þ. When A is compact, this condition reduces to the condition q(x) > 0 for all x 2 Rn n A. Given an open set O  Rn ; q : Rn ! RP0 is of class PDðOÞ if it is continuous and there exists a closed set A and e > 0 such that A þ eB  O and q 2 PDðAÞ. A set O  Rn is said to be open relative to C [ D (or simply relatively e  Rn such that open) if there exists an open set O e \ ðC [ DÞ. O \ ðC [ DÞ ¼ O We use S C;F ðxÞ to denote solutions of (1a). A.2. Definitions from set-valued analysis The tangent cone (Rockafellar and Wets, 1998, Def. 6.1) to C  Rn at a point x 2 Rn , denoted TC(x), is the set of all vectors w 2 Rn for which there exist xi 2 C, si > 0 with xi ? x, si ? 0 such that (xi  x)/ si ? w. A set-valued mapping M : Rp  Rn is outer semicontinuous (osc) (Rockafellar and Wets, 1998, Def. 5.4) if, for each ðxi ; yi Þ ! ðx; yÞ 2 Rp  Rn satisfying yi 2 M(xi) for all i 2 ZP0 , y 2 M(x). A set-valued mapping is outer semicontinuous if and only if its graph, which is the set fðx; yÞ 2 Rp  Rn : y 2 MðxÞg, is closed (Rockafellar and Wets, 1998, Thm. 5.7). The outer semicontinuous hull (Rockafellar and Wets, 1998, p. 154–155) of a mapping M : Rp  Rn is the mapping N : Rp  Rn with the property that graphðNÞ ¼ graphðMÞ, i.e., the closure of the graph of M. The domain of a set-valued mapping M : Rp  Rn is the set fx 2 Rp : MðxÞ – £g. A mapping M : Rp  Rn is locally bounded (Rockafellar and Wets, S 1998, Def. 5.14) if, for each bounded set K  Rp , M(K) :¼ x2KM(x) is bounded. The composition of two set-valued mappings Mi : Rni  Rniþ1 ; i 2 f1; 2g is a set-valued mapping M 3 : Rn1  Rn3 defined by Rockafellar and Wets (1998, p. 151) M 3 ðxÞ ¼ ðM2  M 1 ÞðxÞ :¼ [u2M1 ðxÞ M 2 ðuÞ. Let ðX; F Þ be a measurable space. And example is given by ðX; F Þ ¼ ðRm ; BðRm ÞÞ, where BðRm Þ denotes the Borel field, the subsets of Rm generated from open subsets of Rm through complements and finite and countable unions. A set A  X is F -measurable, or simply measurable if the r-field F is clear, if A 2 F . A mapping M : X  Rn is F -measurable, or simply measurable, (Rockafellar and Wets, 1998, Def. 14.1) if for each open set O  Rn the set M 1 ðOÞ :¼ fx 2 X : MðxÞ \ O – £g is F -measurable. When the values of M are closed, measurability is equivalent to measurability of M1(S) for each open, closed, or compact S  Rn and measurability of {x 2 X: M(x)  S} for each closed or open set S  Rn (Rockafellar and Wets, 1998, Thm. 14.3). If the domain of M is countable then M is measurable since M 1 ðOÞ is then countable (and thus measurable) for each open set O. If G : Rn  Rm  Rn is outer semicontinuous then v # graphðGð; v ÞÞ :¼ fðx; yÞ 2 Rn  Rn : y 2 Gðx; v Þg is measurable. Indeed, given a compact set K, the set fv 2 Rm : graphðGð; v ÞÞ \ K – £g is closed, and thus measurable, since if (xi, yi, vi) satisfy yi 2 G(xi, vi) and (xi, yi) 2 K and vi is convergent to some v then (xi, yi) contains a converging subsequence converging to some (x, y) 2 K such that y 2 G(x, v) which shows that graph(G(, v)) \ K – £. If the mapping G : Rn  Rm  Rn is such that v ´ graph(G(, v)) is measurable, then, for each x 2 Rm ; v # Gðx; v Þ is measurable. Indeed, letting p : Rn  Rn ! Rn be the function p(x, y) = y, we have that for each x 2 Rn

v # Gðx; v Þ ¼ pðgraphðGð; v ÞÞ \ ðfxg  Rn ÞÞ which is measurable (Rockafellar and Wets, 1998, Prop. 14.11(a), Thm. 14.13(a)). We say a logical condition involving a measurable

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001

24

A.R. Teel / Annual Reviews in Control xxx (2013) xxx–xxx

set-valued mapping M : X  Rn is measurable if the set of x such that M(x) makes the logical expression true is measurable. From the probability space ðX; F ; PÞ and the i.i.d. wpo random variables vi, we define V :¼ [x2X;i2ZP0 v iþ1 ðxÞ. We use ‘‘ 6 ’’ for being less than or equal to for almost all x 2 X, and ‘‘ ’’ for equality for all x 2 X. References Bujorianu, M. (2012). Stochastic reachability analysis of hybrid systems. London: Springer. Bujorianu, M., & Bujorianu, M. (2012). Contingent hybrid systems. In: Proceedings of the 4th IFAC conference on design and analysis of hybrid systems (pp. 442–447). Cai, C., & Teel, A. (2009). Characterizations of input-to-state stability for hybrid systems. Systems & Control Letters, 58(1), 47–53. Cai, C., Teel, A. R., & Goebel, R. (2007). Smooth Lyapunov functions for hybrid systems – Part I: Existence is equivalent to robustness. IEEE Transactions of the Automatic Control, 52(7), 1264–1277. Cai, C., Teel, A., & Goebel, R. (2008). Smooth Lyapunov functions for hybrid systems. Part II: (Pre-)asymptotically stable compact sets. IEEE Transactions of the Automatic Control, 53(3), 734–748. Cassandras, C., & Lafortune, S. (2008). Introduction to discrete event systems (2nd ed.). Springer. Clarke, F. (1990). Optimization and nonsmooth analysis. SIAM. Davis, M. H. A. (1984). Piecewise-deterministic markov processes: A general class of non-diffusion stochastic models. Journal of the Royal Statistical Society. Series B (Methodological), 46(3), 353–388. Davis, M. (1993). Markov models and optimization, monographs on statistics and applied probability. London, UK: Chapman & Hall. Foster, F. (1953). On the stochastic matrices associated with certain queuing processes. Annual Mathematical Statistics, 24(3), 355–360. Fristedt, B., & Gray, L. (1997). A modern approach to probability theory. Birkhauser. Ghosh, M., Arapostathis, A., & Marcus, S. (1993). Optimal control of switching diffusions with application to flexible manufacturing systems. SIAM Journal of Control Optimization, 31(5), 1183–1204. Goebel, R., Sanfelice, R. G., & Teel, A. R. (2009). Hybrid dynamical systems. IEEE Control Systems Magazine, 29(2), 28–93. Goebel, R., Sanfelice, R., & Teel, A. (2012). Hybrid dynamical systems. Princeton University Press. Goebel, R., & Teel, A. R. (2006). Solutions to hybrid inclusions via set and graphical convergence with stability theory applications. Automatica, 42, 573–587. Haddad, W., Chellaboina, V., & Nersesov, S. (2006). Impulsive and hybrid dynamical systems. Princeton University Press. Hanson, F. (2007). Applied stochastic processes and control for jump-diffusions: modeling, analysis, and computation. Philadelphia, PA: SIAM. Hespanha, J. (2005). A model for stochastic hybrid systems with application to communication networks. Nonlinear Analysis, Special Issue on Hybrid Systems, 62, 1353–1383. Hespanha, J. (2005). A model for stochastic hybrid systems with application to communication networks. Nonlinear Analysis: Theory, Methods & Applications, 62(8), 1353–1383. Hespanha, J., & Teel, A. (2006). Stochastic impulsive systems driven by renewal processes. In 17th International symposium on mathematical theory of networks and systems (MTNS06). Himmelberg, C. (1975). Measurable relations. Fundamental Mathematics, 87, 53–72. Hu, J., Lygeros, J., & Sastry, S. (2000). Towards a theory of stochastic hybrid systems. In N. Lynch & B. Krogh (Eds.), Hybrid Systems: Computation and Control. Lecture Notes in Computer Science (Vol. 1790, pp. 160–173). Berlin/Heidelberg: Springer. Jacobsen, M. (2006). Point process theory and applications: Marked point and piecewise deterministic processes. Boston: Birkhauser. Khalil, H. (2002). Nonlinear systems (3rd ed.). Prentice-Hall. Khasminskii, R. Z. (1980). Stochastic stability of differential equations. Netherlands: Sijthoff & Noordhoff. Kozin, F. (1969). A survey of stability of stochastic systems. Automatica, 5(1), 95–112.

Lakshmikantham, V., Bainov, D., & Simeonov, P. (1989). Theory of impulsive differential equations. World Scientific. Liberzon, D. (2003). Switching in systems and control. Birkhauser. Loria, A., Panteley, E., Popovic, D., & Teel, A. (2005). A nested Matrosov theorem and persistency of excitation for uniform convergence in stable nonautonomous systems. IEEE Transactions of the Automatic Control, 50(2), 183–198. Mariton, M. (1990). Jump linear systems in automatic control. New York, NY: Marcel Dekker. Matrosov, V. M. (1963). On the stability of motion. Journal of Applied Mathematics and Mechanics, 26, 1337–1353. Meyn, S. P., & Tweedie, R. L. (1993). Markov chains and stochastic stability. SpringerVerlag. Pola, G., Bujorianu, M., Lygeros, J., & Benedetto, M. D. (2003). Stochastic hybrid models: An overview. In Proceedings of the 2nd IFAC conference on analysis and design of hybrid systems (pp. 45–50). Rockafellar, R., & Wets, R. (1974). Continuous versus measurable recourse in n-stage stochastic programming. Journal of Mathematical Analysis and Applications, 48(3), 836–859. Rockafellar, R. T., & Wets, R. J.-B. (1998). Variational Analysis. Springer. Sanfelice, R. G., Goebel, R., & Teel, A. R. (2007). Invariance principles for hybrid systems with connections to detectability and asymptotic stability. IEEE Transactions of the Automatic Control, 52(12), 2282–2297. Sanfelice, R. G., & Teel, A. R. (2009). Asymptotic stability in hybrid systems via nested Matrosov functions. IEEE Transactions of the Automatic Control, 54(7), 1569–1574. Subbaraman, A., & Teel, A. (submitted for publication). A converse Lyapunov theorem for strong global recurrence. Tavernini, L. (1987). Differential automata and their discrete simulators. Nonlinear Analysis, 11(6), 665–683. Teel, A. R. (2009). Preliminary results on the existence of continuous Lyapunov functions for semicontinuous, stochastic discrete-time systems. In 48th IEEE conference on decision and control (pp. 4729–4734). Teel, A. R., Hespanha, J., & Subbaraman, A. (submitted for publication). A converse Lyapunov theorem for asymptotic stability in probability. Teel, A., Nesic, D., Loria, A., & Panteley, E. (2010). Summability characterizations of uniform exponential and asymptotic stability of sets for difference inclusions. Journal of Difference Equations and Applications, 16(2–3), 173–194. Teel, A., & Praly, L. (2000). On assigning the derivative of a disturbance attenuation control Lyapunov function. MCSS, 13, 95–124. Teel, A. R. (submitted for publication). Adversarial Markov decision processes: invariance and recurrence principles. Teel, A. R. (submitted for publication). A Matrosov theorem for adversarial Markov decision processes. van der Schaft, A., & Schumacher, H. (2000). An introduction to hybrid dynamical systems. Springer. Wagner, D. H. (1977). Survey of measurable selection theorems. SIAM Journal of Control Optimization, 15(5), 859–903. Witsenhausen, H. S. (1966). A class of hybrid-state continuous-time dynamic systems. IEEE Transactions on Automatic Control, 11(2), 161–167. Yang, T. (2001). Impulsive control theory. Berlin: Springer-Verlag. Yin, G. G., & Zhu, C. (2010). Hybrid switching diffusions. Springer. Andrew R. Teel received his A.B. degree in Engineering Sciences from Dartmouth College in 1987, and his M.S. and Ph.D. degrees in Electrical Engineering from the University of California, Berkeley, in 1989 and 1992, respectively. After receiving his Ph.D., he was a postdoctoral fellow at the Ecole des Mines de Paris in Fontainebleau, France and then joined the faculty of the Electrical Engineering Department at the University of Minnesota. In 1997, he joined the faculty of the Electrical and Computer Engineering Department at the University of California, Santa Barbara, where he is currently a professor. He has received NSF Research Initiation and CAREER Awards, the 1998 IEEE Leon K. Kirchmayer Prize Paper Award, the 1998 George S. Axelby Outstanding Paper Award, and was the recipient of the first SIAM Control and Systems Theory Prize in 1998. He was the recipient of the 1999 Donald P. Eckman Award and the 2001 O. Hugo Schuck Best Paper Award, both given by the American Automatic Control Council, and also received the 2010 IEE Control Systems Magazine Outstanding Paper Award. He is an area editor for Automatica, and a Fellow of the IEEE and of IFAC.

Please cite this article in press as: Teel, A. R. Lyapunov conditions certifying stability and recurrence for a class of stochastic hybrid systems. Annual Reviews in Control (2013), http://dx.doi.org/10.1016/j.arcontrol.2013.02.001