Stability analysis of switched systems using variational principles: An introduction

Automatica 42 (2006) 2059 – 2077 www.elsevier.com/locate/automatica

Stability analysis of switched systems using variational principles: An introduction夡 Michael Margaliot ∗ School of Electrical Engineering-Systems, Tel Aviv University, Israel Received 26 September 2005; received in revised form 28 May 2006; accepted 13 June 2006 Available online 1 September 2006

Abstract Many natural and artificial systems and processes encompass several modes of operation with a different dynamical behavior in each mode. Switched systems provide a suitable mathematical model for such processes, and their stability analysis is important for both theoretical and practical reasons. We review a specific approach for stability analysis based on using variational principles to characterize the “most unstable” solution of the switched system. We also discuss a link between the variational approach and the stability analysis of switched systems using Lie-algebraic considerations. Both approaches require the use of sophisticated tools from many different fields of applied mathematics. The purpose of this paper is to provide an accessible and self-contained review of these topics, emphasizing the intuitive and geometric underlying ideas. 䉷 2006 Elsevier Ltd. All rights reserved. Keywords: Global asymptotic stability; Stability under arbitrary switching; Bilinear systems; Lie bracket; Lie algebra; Nilpotent control systems; Geometric control theory; Maximum principle; Dynamic programming; Hamilton–Jacobi–Bellman equation; Bang-bang control; Reachability with nice controls; Absolute stability; Switched controllers; Hybrid systems; Differential inclusions

1. Introduction Many natural and artificial systems and processes encompass several modes of operation with a different dynamical behavior in each mode. Our car changes its dynamic behavior every time we change gears. Our heart switches between different modes of operation every time its internal valves open and close. Such systems are sometimes referred to as hybrid systems as they combine both continuous-time and discrete-time dynamics. In the last decade, hybrid systems attracted enormous interest in many fields of science, including engineering, computer science, and applied mathematics. Switched systems are a particular case of hybrid systems, arising when it is possible to describe the behavior in each mode using continuous dynamics, and the transitions as discrete-time 夡 This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Editor Manfred Morari. This research was supported in part by the ISF under Grant number 199/03. ∗ Tel.: +972 3 640 8659; fax: +972 3 640 5027. E-mail address: [email protected].

0005-1098/$ - see front matter 䉷 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.automatica.2006.06.020

events. This yields the mathematical model x˙ (t) = f
(t) (x(t)),

(1)

where fi : Rn → Rn are vector fields and
: [0, ∞) → {1, 2, . . . , m} is a piecewise constant function of time. Every mode of operation corresponds to a specific subsystem x˙ (t) = fi (x(t)), for some i ∈ {1, . . . , m}, and the switching signal
determines which subsystem is followed at each point in time. The switching signal may be time-dependent (
=
(t)) or, more specifically, state-dependent (
=
(x(t))). It may also be deterministic or probabilistic. It is usually assumed that
contains a finite number of discontinuities on each bounded time interval, ruling out the possibility of “infinitely fast” switching. An important particular case, which we will frequently use for illustration, is when all the vector fields are linear: fi (x) = Ai x, where Ai ∈ Rn×n . This yields the linear switched system x˙ (t) = A
(t) x(t).

(2)

As for all dynamical systems, the stability analysis of (1) is of considerable interest. In many cases, the precise details of the

2060

M. Margaliot / Automatica 42 (2006) 2059 – 2077

switching mechanism are unknown. Even when they are known, they may depend on unpredictable or uncontrolled conditions. This raises the following problem. Problem 1 (Stability under arbitrary switching). Find conditions guaranteeing that (1) is asymptotically stable for any switching law. Obviously, a necessary condition is that each subsystem is asymptotically stable. However, this is not a sufficient condition. Addressing this problem naturally leads to the differential inclusion (DI) (Aubin & Cellina, 1984; Smirnov, 2002) associated with (1). This is the model x˙ ∈ D(x) := {f1 (x), f2 (x), . . . , fm (x)}.

(3)

A solution of (3) is an absolutely continuous function x : R → Rn which satisfies (3) for (almost) all t. An important particular case is when the vector fields are linear, so (3) becomes the linear differential inclusion (LDI): x˙ ∈ {A1 x, A2 x, . . . , Am x}.

(4)

In the sequel, we will also consider the relaxed version of (3) which is x˙ ∈ co D(x),

(5)

where co denotes the convex hull. By definition, the set of solutions of (1) is contained in the set of solutions of (3), which is itself contained in the set of solutions of (5). Thus, conditions guaranteeing that every solution of (5) is stable automatically guarantee that (1) is stable under arbitrary switching. Stability analysis of DIs is a formidable challenge. The main difficulty is that a DI admits an infinite number of solutions for any initial condition x(0)=x0 . The variational approach is based on the following natural idea (Pyatnitskii, 1970). Characterize the “most unstable” trajectory of the DI; if this trajectory is stable, then so are all the other trajectories.1 This reduces the problem to analyzing the behavior of this single trajectory. To make this more concrete, consider for simplicity the relaxed DI (5) with m = 2, that is, x˙ ∈ co{f1 (x), f2 (x)}.

(6)

The starting point of the variational approach is to rewrite (6) as a control system x˙ = f1 (x) + (f2 (x) − f1 (x))u,

u ∈ U,

(7)

where U is the set of measurable controls u : R → [0, 1]. It is clear that every trajectory of (7) is also a trajectory of (6). The converse implication follows from Filippov’s selection lemma (see, e.g., Vinter, 2000; Young, 2000). Hence, (6) and (7) are equivalent.2 1 This is reminiscent of bounding the complexity of algorithms using the input that yields the worst running time (Cormen, Leiserson, Rivest, & Stein, 2001). 2 See Sussmann (1997) for a discussion on the relative merits of these two alternative representations.

Note that trajectories of the original switched system (1) correspond to piecewise constant controls taking values in the set {0, 1}. In particular, x˙ = f1 (x) results by setting u ≡ 0 in (7), while x˙ = f2 (x) results by setting u ≡ 1. The next step is to characterize the “most destabilizing” control u∗ as the solution of a suitable optimal control problem. If the trajectory corresponding to u∗ is stable, then so are all the other trajectories of (7). In this case, the relaxed DI is stable, which implies that the corresponding switched system is stable under arbitrary switching. The variational approach can also be combined with another important approach to stability analysis of switched systems that is based on Lie-algebraic ideas (Liberzon, 2003). Liealgebraic tools, that play a prominent role in nonlinear control theory (Isidori, 1995), are used to analyze the new behaviors that can emerge from switching between the subsystems. If the commutation relations between the vector fields fi are “simple enough”, then the stability of the subsystems implies stability of the switched system. Recently, it was shown that if certain commutation relations between the vector fields hold, then the “most unstable” control u∗ has a simple form, and this implies that the relaxed DI is stable. This creates a link between the variational and the Liealgebraic approaches. The variational approach has several important advantages. First, it allows the application of powerful tools from optimal control theory (e.g., first- and higher-order maximum principles (MPs)) in the stability analysis of switched systems. Second, the analysis of the worst-case control reveals the exact mechanisms that lead to instability via switching. This is not so in many approaches that are based on finding sufficient conditions for stability. Third, the results automatically hold for the more general system defined by the corresponding relaxed DI. It is thus not surprising that the variational approach yields the most general results available for linear switched systems (and, in particular, low-order linear switched systems). When combined with the Lie-algebraic approach it also yields the most general results available for nonlinear switched systems with a nilpotent Lie algebra. In spite of its proven success, the variational approach is still not as well-known as it should be. Probably, one reason for this is that this approach requires the combination of various tools from many fields of applied mathematics including differential geometry, variational calculus and optimal control, Lyapunov theory, convex analysis, and more. The purpose of this survey is to provide an accessible introduction to this fascinating field of research, as well as suitable references for further study. We also discuss some open problems and possible directions for further research. The interdisciplinary nature of hybrid systems suggests that surveys of different aspects of this field, stated in as simple language as possible, will be useful to a wide audience. This is the underlying philosophy of this survey. In particular, some of the results stated here hold for time-varying and nonlinear control systems evolving on smooth manifolds. Stating and proving the results in this context requires rather sophisticated notation and tools (e.g., chronological calculus, Agrachev & Sachkov,

M. Margaliot / Automatica 42 (2006) 2059 – 2077

2004, chap. 2) and the Baker–Campbell–Hausdorff (BCH) theorem for nonlinear and time-varying flows (Strichartz, 1987). Using these tools is undoubtedly the right approach in the long run. However, they are rather abstract for a reader encountering such things for the first time. In order to increase the readability of this paper, we chose to state some of the results in the simpler setting of LDIs evolving on Rn . This allows us to provide simple, self-contained and, hopefully, intuitive proofs using completely standard notation. More general statements, as well as the technical details needed to make the analysis rigorous, can be found in the cited references. Understanding the results in the special case of LDIs is a big step towards grasping the nonlinear generalizations. The analysis and design of switched systems is a vast topic. Several monographs (see, e.g., Borrelli, 2003; Liberzon, 2003; Matveev & Savkin, 2000; van der Schaft & Schumacher, 2000; Yfoulis, 2002; Zhengguo, Yengchai, & Changyun, 2005), survey papers (Liberzon & Morse, 1999; Shorten, Wirth, Mason, Wulff, & King, 2005b) and special volumes (Antsaklis, 2000; Morse, Pantelides, Sastry, & Schumacher, 1999; Nonlinear Analysis (2005a,b); Pnueli & Sifakis, 1995) discuss various aspects of this field. In this paper, we make an attempt to describe one particular line of research, namely, that which involves the study of the most unstable trajectory of the switched system. The rest of this paper is organized as follows. Section 2 recalls the definitions of Lyapunov stability for switched systems (under arbitrary switchings) and DIs. Section 3 demonstrates the importance and relevance of stability analysis of DIs using several real-world examples. It also describes the connection between stability analysis of LDIs and the famous absolute stability problem. Section 4 describes the variational approach and its application to linear switched systems. Section 5 reviews some ideas from the field of Lie algebras that are needed for the presentation of the Lie-algebraic approach in Section 6. Section 7 reviews recent results obtained by merging the variational and the Lie-algebraic approaches. Section 8 presents several open problems and suggests directions for further research. The final section concludes.

2061

for every initial condition x(0) every solution of (3) satisfies |x(t)| (|x(0)|, t)

∀ t
0

(8)

(here and below, | · | denotes the Euclidean norm on Rn ). Note that this implies, in particular, that x(t) ≡ 0 is a trajectory of the DI, and that every trajectory satisfies lim x(t) = 0,

(9)

t→∞

that is, 0 is an attractive equilibrium. If the function  in (8) takes the particular form (s, t) = ase−bt for some a, b > 0, then the DI is called globally exponentially stable (GES). The above definitions reduce to the usual notions of global asymptotic and exponential stability in the case of a single system (that is, when f1 = f2 = · · · = fm ). If the DI (3) is homogeneous, i.e., fi (cx) = cfi (x), for all 1 i m, all c > 0, and all x ∈ Rn , then the three properties: GAS, GES, and attractivity are all equivalent (Angeli, 1999). GAS of the DI (3) implies, of course, that all the trajectories of the associated switched system (1) also satisfy (8) for any switching law. In the literature on switched systems this property is referred to as global uniform asymptotic stability (GUAS), where the uniformity is with respect to arbitrary switching signals.3 Example 1. Consider

x˙ ∈ co{A the relaxed LDI  1 x, A2 x} with −5 1 −10 −1 n = 2, A1 = 0 −2 , and A2 = 0 −3 . Note that the Ai ’s are Hurwitz, so the subsystems are asymptotically stable. Defining V : R × R → R by V (a, b) := a 2 + b2 , it is easy to verify that Vx (x)Ai x  − 9x12 − 3x22 , i = 1, 2. Hence, for any c ∈ [0, 1] Vx (x)(cA1 x + (1 − c)A2 x)  − 9x12 − 3x22  − 3V (x1 (t), x2 (t)). This implies that V (x1 (t), x2 (t))  exp(−3t)V (x1 (0), x2 (0)) along every solution of the LDI. Hence (8) holds for (s, t) := s exp(−3t/2), so the LDI is GES, and the corresponding linear switched system is GUAS.

2. Asymptotic stability In this section, we formally define the notion of Lyapunov stability for DIs and switched systems. It is possible to give the definition in the familiar – style. However, we will use a different, yet equivalent (Lin, Sontag, & Wang, 1996), formulation which is better suited for our needs. Definition 1. A function  : [0, ∞) → [0, ∞) is said to be of class K if it is continuous, strictly increasing, and (0) = 0 (e.g., (s) = s). A function  : [0, ∞) × [0, ∞) → [0, ∞) is said to be of class KL if (·, t) is of class K for each fixed t
0, and (s, t) decreases to 0 as t → ∞ for each fixed s
0 (e.g., (s, t) = s exp(−t)). Definition 2. The DI (3) is called globally asymptotically stable (GAS) if there exists a class KL function  such that

It is obvious that global asymptotic stability of the individual subsystems is necessary for GUAS of a switched system. However, the next example shows that this is not a sufficient condition. Example 2. Consider the  linear switched  (2) with n=2,

system 0 1 0 1 , and A = m = 2, A1 = −2 2 −1 −10 −1 . Note that the Ai ’s are Hurwitz, so the subsystems are asymptotically stable. Fig. 1 depicts the trajectory of this system corresponding to the state-dependent switching law  1 if x1 (t)x2 (t) > 0,
(t) = 2 otherwise 3 For a discussion on some of the subtleties of this notion, see Hespanha (2004).

2062

M. Margaliot / Automatica 42 (2006) 2059 – 2077

and (10) yields

3

x(T ) = exp(A2 (T − s)) exp(A1 s)x0 , 2

where s := t1 + t3 − t2 + · · · + tj − tj −1 . Since the Ai ’s are Hurwitz, there exist Mi , mi > 0, i = 1, 2, such that | exp(Ai t)x0 | Mi e−mi t |x0 |. Hence, (11) yields

1 x2

(11)

|x(T )| M2 e−m2 (T −s) | exp(A1 s)x0 |

0

M1 M2 e−m2 (T −s) e−m1 s |x0 | -1

M1 M2 e−mT |x0 |, where m := min{m1 , m2 } > 0. Clearly, this bound actually holds for any switching law, so the switched system is GAS.

-2

-3 -1.5

-1

-0.5

0 x1

0.5

1

1.5

Fig. 1. A trajectory x(t) of the switched system with x(0) = (1, 0)T .

with x(0) = (1, 0)T . It is easy to prove that x(t) does not converge to the origin. Hence, (8) cannot hold for any class KL function , and the switched system is not GUAS. Note that we obtained instability by repeatedly switching between the subsystems. We will see later that such repeated switching is necessary to achieve instability (see Proposition 4 in Section 7). Example 2 demonstrates a deep and important property: a switched system can demonstrate behaviors that do not exist in any of its subsystems. This gestalt principle4 makes switched systems both theoretically interesting and practically promising. It should be clear that new behaviors in a switched system or a DI can only arise when the subsystems are, in some sense, different enough (to see this, it is enough to consider the DI x˙ ∈ {Ax, Ax}). The next example demonstrates another case where the switched system is stable because the subsystems are, in some sense, not different enough. Example 3. Consider the linear switched system (2) with m=2, and suppose that exp(A1 1 ) exp(A2 2 ) = exp(A2 2 ) exp(A1 1 )

The problem of identifying conditions on the subsystems— besides the obviously necessary requirement of their global asymptotic stability—which guarantee GUAS of (1) has received considerable attention in the literature (see Liberzon, 2003 and the references therein).Even for the particular case of linear switched systems, this problem is far from trivial. In Example 1 we were able to prove GUAS using a standard tool from the theory of linear differential equations (LDEs), namely, a quadratic Lyapunov function.5 In general, however, the standard methods from the theory of LDEs do not necessarily apply to linear switched systems. To understand why, consider the linear switched system (2) and suppose that we can find an invertible matrix T such that D := T A1 T −1 has a “simple” form (e.g., diagonal or triangular). In this case, we can study the LDE x˙ = A1 x by defining y := T x and analyzing the corresponding differential equation y˙ = Dy. However, applying the same trick to (2) yields y˙ (t) = T A
(t) T −1 y and, in general, there is no reason to expect that T −1 Ai T , i > 1, also has a “simple” form. (An important exception is when the Lie algebra spanned by the matrices Ai , i = 1, . . . , m, has a special structure; see Section 6.) Of course, applying a linear change of variables is only one approach for studying LDEs. This raises the following question. Question 1. What analysis tools from the theory of LDEs can be used effectively in the analysis of linear switched systems?

(10)

for all 1 , 2
0. Consider the switching signal  1, t ∈ [0, t1 ) ∪ [t2 , t3 ) ∪ · · ·
(t) = 2, t ∈ [t1 , t2 ) ∪ [t3 , t4 ) ∪ · · ·

As we will see below, the variational approach indicates that the suitable tool is the generalized first integral of the LDE. Before explaining this, we provide a wider perspective on the importance of switched systems and DIs by considering several examples.

for some 0 < t1 < t2 < · · ·. Fix an odd integer j and a time T ∈ [tj , tj +1 ]. Then

3. Examples of switched systems and DIs

x(T ) = exp(A2 (T − tj )) . . . exp(A2 (t2 − t1 )) exp(A1 t1 )x0 , 4 Webster dictionary defines gestalt as “a structure, configuration, or pattern of physical, biological, or psychological phenomena so integrated as to constitute a functional unit with properties not derivable by summation of its parts”.

The fundamental theoretical importance of the stability analysis of LDIs can be demonstrated by noting that it is 5 For more on the use of quadratic Lyapunov functions in the stability analysis of linear switched systems see Johansson (2003) and Shorten et al. (2005b).

M. Margaliot / Automatica 42 (2006) 2059 – 2077

equivalent to one of the oldest open problems in mathematical control theory, the problem of absolute stability (see, e.g., Boyd, El Ghaoui, Feron, & Balakrishnan, 1994; Vidyasagar, 1993). 3.1. Absolute stability and LDIs

2063

x1

S0 V ±

S1

L

x2 C

R

Fig. 2. The buck converter.

Consider the system x˙ (t) = Ax(t) + b(t, cT x(t)),

(12)

where A is an asymptotically stable matrix, the pair (A, b) ((A, c)) is controllable (observable), and  : R × R → R belongs to Sk , the set of scalar time-varying functions in the sector [0, k], that is, (t, 0) = 0 for all t and 0 z(t, z)kz2 for all t, z. For example, every linear function z → cz, with c ∈ [0, k], belongs to Sk . Note that we can view (12) as the feedback connection of a linear system with output y = cT x and a function  ∈ Sk . Since A is Hurwitz, it is obvious that (12) is asymptotically stable for k = 0.

Note that Problem 3 is more difficult than the problem of determining whether (13) is asymptotically stable or not. This is so because Problem 3 requires determining the exact value k ∗ where stability is lost. In particular, solving the absolute stability problem is equivalent to providing a necessary and sufficient condition for the stability of the relaxed LDI (13). (It is well-known that an LDI is GAS if and only if its relaxed version is GAS, Molchanov & Pyatnitskiy, 1989.) A more general yet closely related problem is the computation of the maximal Lyapunov exponent of an LDI (Barabanov, 2005; Colonius, Kliemann, & Grune, 2000). 3.2. DIs and real-world systems

Problem 2 (Absolute stability). Find the value: k ∗ := inf{k > 0 : there exists  ∈ Sk such that (12) is not asymptotically stable}. In other words, for any k < k ∗ , system (12) is asymptotically stable for all  ∈ Sk , and there exists a function ∗ ∈ Sk ∗ , referred to as the worst-case nonlinearity, for which (12) is not asymptotically stable. This fundamental problem, dating back to the 1940s,6 led to numerous important results in the mathematical theories of stability and control, including: Popov’s criterion, the circle criterion, the positive-real lemma (Boyd et al., 1994), and the theory of integral quadratic constraints (Megretski & Rantzer, 1997). However, all these approaches ignore the problem of characterizing ∗ and, therefore, lead to sufficient, but not necessary and sufficient stability conditions. An analysis of the computational complexity of some closely related problems can be found in Blondel and Tsitsiklis (2000). Using the technique of global linearization (Boyd et al., 1994), we can restate Problem 2 in a different form. Since  ∈ Sk , we have (t, z) = h(t, z)z with 0 h(t, z) k. Hence, denoting Bk := A + kbcT , (12) becomes x˙ (t) ∈ co{Ax(t), Bk x(t)},

(13)

where co denotes the convex hull. Problem 2 can now be restated as: Problem 3. Find the value: k ∗ := inf{k > 0 : system (13) is not GAS}. 6 See Rasvan (2004) for an interesting discussion on the history of this problem.

Numerous real-world systems can be modeled using switched systems and DIs. These include: Internet communication protocols (Hespanha, Bohacek, Obraczka, & Lee, 2001; Shorten, Leith, Foy, & Kilduff, 2005a), mechanical engines with gear transmission (Johansson, Lygeros, & Sastry, 2004); electrical circuits containing on–off switches and in particular power converters (Erickson & Maksimovic, 2001), gene regulating networks (de Jong et al., 2004); and embedded systems in which computer software interacts with a physical device (Antsaklis, 2000; van der Schaft & Schumacher, 2000). Thus, a deeper understanding of DIs and their stability can have a major impact on the analysis and synthesis of numerous real-world systems. For the sake of concreteness, we consider two specific examples. Example 4. Fig. 2 depicts an electrical circuit known as the DC–DC buck converter (Cuk, 1992). This circuit, composed of a capacitor, an inductor, and a switch, delivers power from a source V to a load with resistance R. Denoting x(t) = (x1 (t), x2 (t))T , where x1 (t) (x2 (t)) is the inductor’s current (capacitor’s voltage), yields the switched system  Mx + b when the switch is in state S0 , (14) x˙ = Mx when the switch is in state S1 
−1/L 0 T where M := 1/C −1/(RC) and b := (V /L, 0) . Example 5. Modern controller design schemes may yield two (or more) controllers for a certain plant, together with a decision maker module that switches between the two controllers according to some criteria (see e.g., Hespanha & Morse, 2002; Liberzon, 2003; McClamroch & Kolmanovsky, 2000; Morse, 1997).

2064

M. Margaliot / Automatica 42 (2006) 2059 – 2077

4. The variational approach

Controller 1

Since a DI admits an infinite number of solutions for every initial condition, a natural idea is based on characterizing the worst-case (i.e., “most unstable”) solution. If this trajectory satisfies (8), then so do all the other trajectories. We now review the pioneering work of Pyatnitskii (1970, 1971) who developed such an approach to address the absolute stability problem. The first step is to rewrite (13) as the control system

Plant Controller 2

Fig. 3. Switching between two controllers.

x˙ (t) = Ax(t) + u(t)(Bk − A)x(t), The resulting configuration, depicted in Fig. 3, can be described mathematically using a switched system, and its stability analysis is clearly of great importance. In the sequel, we will try to characterize a switching signal for a switched system, composed of two GAS subsystems, that yields an unstable trajectory (if it exists). By inverting time (i.e, defining t  := −t), we immediately see that this is equivalent to solving the following problem. Problem 4 (Stabilization using switching). Let x˙ = f (x) and x˙ = f2 (x) be two asymptotically unstable systems. Is there a switching law
: R+ → {1, 2} such that the system x˙ (t) = f
(t) (x(t)) is asymptotically stable? 1

This problem is recently attracting considerable interest (Liberzon, 2003). One reason for this is that switched controllers are in some respects fundamentally more powerful than smooth controllers (see, e.g., Artstein, 1996). 3.3. LDIs and mathematical modeling A fundamental problem in science is developing suitable mathematical models for various phenomena. A useful model must satisfy two conflicting requirements. On the one hand, it must be “simple” enough so that it is amenable to mathematical analysis. On the other hand, it must be “complex” enough so that it can be used to faithfully model complicated, non-trivial behavior. Loosely speaking, linear (nonlinear) differential equations satisfy the first (second) requirement, but not the second (first). We believe that LDIs may provide a good compromise between these two extremes. On the one hand, the fact that each subsystem is linear suggests that powerful analysis tools from the theory of LDEs may be extended to LDIs. On the other hand, even though each subsystem is linear, LDIs can still exhibit remarkably complicated behavior (Chase, Serrano, and Ramadge, 1993; Xinghuo & Chen, 2003). At the moment, the main stumbling block in the applicability of LDIs is that their analysis remains a highly non-trivial task. Consequently, the use of LDIs as modeling tools is still less common than one might expect. A breakthrough in the mathematical analysis of LDIs is likely to make them as ubiquitous as LDEs are today. This could have a significant impact on numerous scientific disciplines. We now turn to review two important approaches for tackling the stability analysis of DIs and switched systems.

u ∈ U,

(15)

where U is the set of measurable functions taking values in [0, 1]. A system in the form (15) is sometimes referred to as a bilinear control system (see Elliott, 1999, and the references therein). The next step is to characterize the worst-case trajectory (WCT) of (15). For y ∈ Rn and T > 0, let r(T ; y) := max {|x(T )| : x(0) = y}. u∈U

In other words, r is determined by solving an optimal control problem: find the control that maximizes the distance of x(T ) from the origin for the initial condition x(0) = y.7 It can be shown, using the definition of the set of admissible controls U, that an optimal control u∗ ∈ U indeed exists (Filippov, 1962). Note that if k < k ∗ , then, by definition of k ∗ , all the trajectories of (15) converge to the origin so limT →∞ r(T ; y) = 0 for all y ∈ Rn . On the other hand, if k > k ∗ , then there exists an unbounded trajectory so r(T ; y) is unbounded for some y ∈ Rn (under suitable conditions, r(T ; y) is actually unbounded for any y ∈ Rn \{0}). The interesting case is between these two extremes, namely, when k = k ∗ . Theorem 1 (Barabanov, 1989; Pyatnitskiy and Rapoport (1996); Rapoport, 1996). If k = k ∗ , then: (1) The function ϑ(y) := lim supT →∞ r(T ; y) is finite, convex, positive for y = 0, and homogeneous (i.e., ϑ(cy) := |c|ϑ(y)). (2) For every initial point y = 0, there exists a solution x∗ (t) of (15), with x∗ (0) = y, such that ϑ(x∗ (t)) = ϑ(y) for all t
0, and the directional derivative lim s↓0

ϑ(x∗ (t) + s x˙ ∗ (t)) − ϑ(x∗ (t)) s

(16)

exists and equals zero for (almost) all t
0. Note that if ϑ is smooth (which is generally not the case, Rapoport, 1993), then the last statement is simplified to 0 = ϑx (x∗ (t))˙x∗ (t) =

d ϑ(x∗ (t)), dt

where ϑx := (jϑ/jx1 , . . . , jϑ/jxn ). 7 It is interesting to compare this with the philosophy underlying the approach used in Camilli, Grune, and Wirth (2001) to characterize the domain of attraction of an equilibrium point.

M. Margaliot / Automatica 42 (2006) 2059 – 2077

As we will see below, x∗ (t) is the trajectory corresponding to the worst-case control u∗ . The theorem implies that x∗ (t) belongs to the set {x ∈ Rn : ϑ(x) = ϑ(y)}, and that this set is homeomorphic to the unit sphere {z ∈ Rn : |z| = 1}. In particular, this means that for k = k ∗ the WST is bounded, but does not converge to the origin. Pyatnitskii (1971) and also Barabanov (1988) derived an implicit characterization of the maximizing control u∗ in terms of a two-point boundary value problem. Fixing T > 0, it follows from Theorem 1 that u∗ is the solution to the problem max ϑ(x(T )), u∈U

2065

x0 x = Ax

x1 -x0

x(0) = y.

x = Bk*x

Fig. 4. Schematic interpretation of Theorem 3.

Applying the MP yields the following result. Theorem 2 (Barabanov, 1988; Rapoport, 1996). Let x∗ (t), with x∗ (0) = y, be the optimal trajectory corresponding to the control u∗ . Suppose that ϑ is differentiable at x∗ (T ), that is, ϑx (x∗ (T )) ∈ Rn exists. Define the adjoint vector p : [0, T ] → Rn by p˙ (t) = −(A + (Bk − A)u∗ )T p(t), Then ∗

u (t) =



p(T ) = ϑx (x∗ (T )).

1

if pT (t)(Bk − A)x∗ (t) > 0,

0

if pT (t)(Bk − A)x∗ (t) < 0.

(17)

x˙ = Ax

4.1. Second-order systems The fact that trajectories of second-order systems are confined to the plane makes them more amenable to analysis. For n = 2 the unit sphere becomes a circle, so x∗ must be a closed trajectory. Theorem 3 (Rapoport, 1996). Let n=2. For k=k ∗ the equation det(exp(Bk ) exp(A) + I ) = 0

(19)

admits a solution  > 0,  > 0. Such a solution does not exist for any k ∈ [0, k ∗ ). Geometrically, this implies that there exists x0 ∈ R2 such that exp(Bk ∗ ) exp(A)x0 = −x0 . Letting := we can write this as exp(B = −x0 . In other words, taking x0 as an initial point, following the trajectory of x˙ =Ax for  sec, and then the trajectory of x˙ =Bk ∗ x for  sec, we reach −x0 (see Fig. 4). By symmetry, we see that for k = k ∗ system (15) admits a closed, periodic trajectory with four (two) switching points in every period (half period). This is exactly the trajectory x∗ from Theorem 1. Note, however, that solving Eq. (19) is not trivial since it is a nonlinear equation with three unknowns: k ∗ , , and . exp(A)x0 ,

(20)

(18)

These results provide considerable geometric insight, but they do not provide an explicit characterization of the function ϑ nor the trajectory x∗ . Stronger results can be derived for the special case of low-order systems.

x1

4.1.1. Dynamic programming Margaliot and Langholz (2003) derived a dynamic programming-type solution for the optimal control problem posed by Pyatnitskiy. This also leads to an equation that, just like (19), describes the closed trajectory that appears for n=2 and k =k ∗ . However, in this equation the times  and  do not appear, so the only unknown is k ∗ . We now briefly review their approach. If the system

k∗

)x1

is Hamiltonian (Goldstein, 1980) then it admits a classical first integral, that is, a function H : Rn → R+ satisfying H (x(t)) ≡ H (x(0)) along the trajectories of (20). In this case, the study of (20) is simplified since its trajectories are nothing but the contours H (x) = const. This idea can be extended to the case where A is asymptotically stable and, therefore, (20) is not Hamiltonian. Definition 3. A function H A : Rn → R+ is a generalized first integral of (20) if H A is not constant on any open subset of Rn ; H A (x(t)) is piecewise constant along the trajectories of (20); and it attains a finite set of values on any finite time interval. For second-order linear systems it is possible to construct explicit generalized first integrals for (20) (Margaliot
& Langholz,  2003). For example, for the Hurwitz matrix A = 01 −10 −2 , the generalized first integral is H A (x) = 10(x12 − 2x1 x2 + 10x22 )    2 −3x1 × exp . arctan 3 10x2 − x1 Note that this can be interpreted as a quadratic function xT P x multiplied by a transcendental “correction term”. This term is needed because the system x˙ = Ax is not Hamiltonian, so its solutions are not closed trajectories. Fig. 5 depicts the solution of (20) for this particular A and x(0)=(3, 0.8)T . Fig. 6 displays H A (x(t)) as a function of time. It can be seen that H A (x(t)) is a piecewise constant function. Note that H A (x1 , x2 ) is not defined on the line l := {x : 10x2 − x1 = 0}. The discontinuities of H A (x(t)) are exactly the times ti such that x(ti ) ∈ l. Fig. 7 shows three contours H A (x) ≡ ck ,

2066

M. Margaliot / Automatica 42 (2006) 2059 – 2077

1

1 x(0)

0.5

0

x2

x2

0.5

x(t2)

x(t1)

0

-0.5

-0.5

-1

-1

-4

-3

-2

-1

0 x1

1

2

3

4

Fig. 5. The trajectory for x(0) = (3, 0.8)T (solid) and the line 10x2 − x1 = 0 (dashed).

-4

-3

-2

-1

0 x1

1

2

3

4

Fig. 7. Three contours of the function H A (x): H A (x) ≡ c0 (dot-dashed), H A (x) ≡ c1 (solid), and H A (x) ≡ c2 (dashed).

60

50

40

30 Fig. 8. Schematic interpretation of Theorem 3.

20

10

0

t1=0.878

t2=1.925

t3=2.973

Fig. 6. H A (x(t)) as a function of time.

k = 0, 1, 2. Note that the trajectory x(t), seen in Fig. 5, can be retrieved by following H A (x) = c0 until the first intersection point with the line l, then continuing along H A (x) = c1 until the next intersection point, and so on. It follows from Definition 3 that any trajectory of (20) can be described as a concatenation of several equal height contours of H A (x). Thus, Fig. 4 becomes Fig. 8 (where, for the sake of simplicity, we assumed that each trajectory can be described using a single equal height contour of its generalized first integral).

Considering this figure we can immediately obtain an explicit description of the WCT x∗ using the generalized first integrals of the two linear subsystems. Let denote the closed curve obtained by following the WCT for k = k ∗ , and suppose that at some point x ∈ , the WCT satisfies x˙ = Ax. If the vector Bk ∗ x points outside of , then we can easily construct an unbounded trajectory of the LDI and this contradicts the fact that k = k ∗ . Hence, the vector Bk ∗ x must point inside , and since HxA (x) is orthogonal to at x, this can be expressed as HxA (x)Bk ∗ x < 0 (see Fig. 9). This yields the following result. Theorem 4 (Margaliot and Langholz (2003)). Consider system (15) for n = 2 and k = k ∗ . The worst-case control is u∗ (t) =



0

if x(t) ∈ P ,

1

otherwise,

where P := {x ∈ R2 : HxA (x)Bk ∗ x < 0}.

(21)

M. Margaliot / Automatica 42 (2006) 2059 – 2077

2067

integral. This  was done Filippov (1980), using polar coordinates

Fig. 9. Direction of the vector Bk ∗ x.

Note that u∗ is a state-dependent bang-bang control yielding / P ). It is easy to show that x˙ = Ax (˙x = Bk ∗ x) when x ∈ P (x ∈ u∗ is also a homogeneous switching law (i.e., u∗ (cx) = u∗ (x) for all x and all c > 0). The fact that we have an explicit expression for P implies that we have an explicit expression for u∗ . Once the WCT is known, determining k ∗ amounts to finding the value k for which an equal height contour of H A can be concatenated with an equal height contour of H Bk∗ to form (half of) a closed contour. This is equivalent to an equation in the single unknown k ∗ . For example, for system (15) with n = 2,     0 1 0 1 A= and Bk = , (22) −2 −1 −(2 + k) −1 k∗

is the solution of the transcendental equation     √ √ −2 2 exp √ arctan( 7) exp √ arctan( 7 + 4k) 7 + 4k 7   2 2 exp √ =

. 2+k 7 + 4k

This equation is easily solved numerically yielding k ∗ = 6.985 (to three digit accuracy). Thus, a necessary and sufficient condition for stability is k < 6.985. Note that Example 2 describes (22) with k = 8 and the worstcase control (21). It is thus not surprising that the resulting trajectory is unstable. Furthermore, reconsidering Fig. 9, we see that applying u∗ yields a trajectory x∗ (t) that follows the closed curve (or, depending on the initial position, a scaled version of ) and any other switching law yields a trajectory that enters into the region encircled by . Define a function V : R2 → R by V (0) = 0, and V (x) := {k : k > 0, xk ∈ } for any x = 0 (Blanchini, 1999). Note that since is a closed curve encircling the origin, V (x) assigns a unique value k to every x ∈ R2 . It follows from the discussion above that we can use V as a Lyapunov function proving that x∗ (t) is indeed the WCT. Note that the generalized first integrals of the subsystems provide us with an explicit description of V , and that V is precisely the function ϑ in Theorem 1. For n = 2 it is possible to derive the WCT using geometric ideas and without appealing to the generalized first

(r, ) := ( x12 + x22 , arctan(x2 /x1 )), and by Pukhlikov (1998) who studied the optimal control of planar bilinear systems using the Poincaré return map.8 More recently, characterizations of the WCT also appeared in Dayawansa and Martin (1999), Xu and Antsaklis (2000), and Boscain (2002). The unique advantage of the generalized first integrals is that they provide the explicit dynamic programming-type solution to the optimal control problem posed by Pyatnitskiy and Rapoport (Margaliot & Gitizadeh, 2004). Indeed, reconsidering Fig. 9, we see that the function V , defined by concatenating the contours of the two generalized first integrals, is the solution to the corresponding Hamilton–Jacobi–Bellman (HJB) equation max {Vx (x)(A + (Bk ∗ − A)u)x} = 0.

u∈[0,1]

(23)

This approach also provides an affirmative answer to Question 1. By combining the generalized first integrals of each LDE we can efficiently address the stability analysis of the corresponding LDI. Note that an asymptotically stable second-order LDE has an infinite set of quadratic Lyapunov functions. However, it follows from the construction method described in Holcman and Margaliot (2003) that its generalized first integral is unique (up to adding or multiplying by a constant). 4.2. Third-order systems When n = 3, the set S := {x ∈ Rn : ϑ(x) = ϑ(y)} is homeomorphic to the unit ball. In a seminal paper, Barabanov (1993) used a Poincaré–Bendixson-type argument to prove that for k=k ∗ there exists a point x0 ∈ S such that x∗ (t), with x∗ (0)=x0 , is a closed, periodic, trajectory with four (two) switching points in every period (half period). In other words, Theorem 3 holds for the case n = 3 as well (see also Rapoport, 1996). The next result summarizes the relationship between the value k ∗ and the eigenvalues of the matrix Q(k, , ) := exp(Bk ) exp(A). Theorem 5 (Margaliot and Yfoulis (2006)). Let (k) := ∪,
0 (k, , ), where (k, , ) is the set of real eigenvalues of Q(k, , ). (1) If k < k ∗ then any ∈ (k) satisfies > − 1. (2) If k=k ∗ then any ∈ (k) satisfies
−1 and −1 ∈ (k). (3) If k > k ∗ then there exists ∈ (k) that satisfies < − 1. A complete understanding of the structure of the value function ϑ when n = 3 is still lacking. However, Theorem 5 implies that k ∗ can be calculated, to arbitrary accuracy, using a simple binary search algorithm (Margaliot & Yfoulis, 2006). The next example demonstrates this.

8 See also the work of Levin (1961) who derived a necessary and sufficient stability condition for the second-order system x(t) ¨ + p(t)x(t) ˙ + q(t)x(t) = 0.

2068

M. Margaliot / Automatica 42 (2006) 2059 – 2077

0.3

5.1. Lie brackets and commutativity

0.2

Given two smooth vector fields f1 and f2 , their Lie bracket is another vector field defined by

0.1

x3

0

[f1 , f2 ](x) :=

-0.1

The purpose of this section is to provide some geometric interpretation of the Lie bracket. Let (exp tf)(x0 ) denote the solution at time t of the differential equation x˙ = f(x), with x(0) = x0 . By definition, (exp 0f)(x0 ) = x0 and

-0.2 x0

-0.3 -0.4 0.5 x2

0 -0.5

0

-0.5

-1

-1.5

0.5

1

1.5

x1 Fig. 10. Periodic trajectory of the system in Example 6.

Example 6 (Margaliot and Yfoulis (2006)). Consider Problem 3 with n = 3, ⎛

−1.5 −3 −2

⎞

⎜ A=⎝ 1

0

⎟ 0 ⎠,

0

1

0

b = [1 0 0]T

jf2 jf1 (x)f1 (x) − (x)f2 (x). jx jx

and

c = [0 − 1 − 1] .

d (exp tf)(x0 ) = f((exp tf)(x0 )). (24) dt Note that for the particular case of a linear vector field, i.e., f(x) = Ax, (exp tf)(x0 ) = exp(tA)x0 , and (24) becomes (d/dt) exp(tA)x0 = A exp(tA)x0 . We say that two vector fields f1 , f2 : Rn → Rn commute if (exp t1 f1 )((exp t2 f2 )(x)) = (exp t2 f2 )((exp t1 f1 )(x))

(25)

for all x ∈ Rn and all t1 , t2
0 (for the sake of simplicity, we assume that the vector fields are complete so that the terms in (25) are well-defined). Proposition 1 (Jakubczyk, 2001; Jurdjevic, 1996, chap. 2). The smooth vector fields f1 and f2 commute if and only if [f1 , f2 ](x) = 0

∀x ∈ Rn .

(26)

T

The algorithm yields k ∗ ≈ 3.82695 (to five digit accuracy). The parameters of the periodic trajectory are x0 ≈ [0.9422 0.2381 − 0.2357]T ,  ≈ 0.874, and  ≈ 0.696 (see Fig. 10). For some other related numerical algorithms, see Polanski (2000), Yfoulis and Shorten (2004), and Pyatnitskii and Sokorodinskii (1982).

5. Commutation relations and Lie algebras An interesting research avenue for tackling the GAS problem is based on studying the commutation relations between the subsystems. This requires mathematical tools from the theory of Lie groups and Lie algebras. Such techniques play an everincreasing role in the analysis of differential equations (Olver, 1986), geometric control theory (see, e.g., the excellent survey, Jakubczyk, 2001), and, more recently, in the analysis of switched systems (Liberzon, 2003). Lie theory is a very broad field and we will make no attempt to provide a comprehensive description. The reader is urged to consult the references above as well as the tutorial paper (Howe, 1983), and the excellent monograph (Hall, 2003). Instead, we provide a very simplified, yet hopefully intuitive, introduction to the main ideas needed in our context.

Proof. A calculation yields d d (exp t1 f1 )((exp t2 f2 )(x))|t1 =t2 =0 dt2 dt1 =

jf1 (x)f2 (x), jx

and d d (exp t2 f2 )((exp t1 f1 )(x))|t1 =t2 =0 dt1 dt2 =

jf2 (x)f1 (x). jx

Hence, (25) implies (26). This proves the only if part. The proof of the if part is given following the statement of Theorem 7.  More generally, the Lie bracket operator can be used to “measure” how far the vector fields are from commutativity. We show this for the particular case f1 (x) = Ax and f2 (x) = Bx. In this case, [f1 , f2 ](x) = BAx − ABx = − [A, B]x, where [A, B] := AB − BA is the Lie bracket of the matrices A and B. The next result, referred to as the pull-back formula (PBF), relates linear vector fields and flows to iterated Lie brackets.

M. Margaliot / Automatica 42 (2006) 2059 – 2077

2069

Theorem 6 (PBF). exp(A)B exp(−A) = B + [A, B] + [A, [A, B]]

2 + ···. 2 (27)

Proof. Denote C(t) := exp(At)B exp(−At). Then ˙ = exp(At)(AB − BA) exp(−At), C(t)

Fig. 11. x(4) = x(0) + 2 [A, B]x(0) + o(2 ).

˙ so C(0) = [A, B]. Inductively C (2) (0) = [A, [A, B]],

C (3) (0) = [A, [A, [A, B]]], . . . .

Hence, the right-hand side of (27) is the Taylor expansion of C(t) about t = 0. 

Eqs. (29) and (27) imply a very important result: the set of Lie brackets determines the behavior of (28). This is true in general for analytic control systems (Sussmann, 1983). 5.2. Lie algebras and geometry

A similar formula holds for nonlinear vector fields; see Hermes (1991, Appendix I). If A and B commute (i.e., AB = BA), then (27) yields exp(At)B exp(−At)=B. Thus, the iterated Lie brackets can be used to “measure” how far the pair A, B is from commutativity. For our purposes, it is more important to note a different implication of the PBF. Consider the control system x˙ = Ax + uBx,

u ∈ U.

(28)

It is natural to seek an expression for the solution x(t) = x(t; u) that separates the time-dependent contributions of the control u from the invariant role of the vector field Ax (Kawski, 2004). To do so, note that for u(t) ≡ 0, the solution is of course x(t, 0) = exp(At)x0 . Hence, we seek an expression in the form x(t, u) = exp(At)y(t; u). This implies y(0)=x0 and differentiation yields the following: Theorem 7. Let y(t; u) denote the solution at time t of y˙ () = u() exp(−A)B exp(A)y(),

y(0) = x0 .

(29)

Then, x(t; u) = exp(At)y(t; u) for any t
0. In other words, the diffeomorphism v → exp(At)v maps solutions of the driftless (yet, time-varying) system (29) to solutions of (28). Theorem 7 is sometimes referred to as the variation of constants or pull-back formula. A similar result holds also for nonlinear vector fields; see e.g., Agrachev and Sachkov (2004), Hermes (1991), and Sussmann (1979). Note that if [A, B] = 0 then (29) and (27) yield y(t; u) = t exp(B 0 u() d)y(0), so  t u() d)x(0). (30) x(t; u) = exp(At) exp(B 0

On the other hand, defining z(t; u) by x(t; u)=exp(B z(t; u), and arguing similarly yields   t  x(t; u) = exp B u() d exp(At)x(0).

t 0

u() d)

0

Combining this with (30) shows that if [A, B] = 0 then the vector fields Ax and Bx commute. This proves the if part of Proposition 1 for the particular case of linear vector fields.

The Lie algebra generated by Ax and Bx, denoted by {A, B}LA (x), is the linear span of {Ax, Bx, [A, B]x, [A, [A, B]]x, [B, [A, B]]x, . . .}, where the dots indicate all the iterated Lie brackets of A and B. The Lie algebra has an important geometric interpretation. This is demonstrated in the next example. Example 7. Consider the symmetric9 LDI x˙ ∈ {Ax, −Ax, Bx, −Bx},

x(0) = x0 .

(31)

Obviously, x() = exp(A)x0 = (I + A + o())x0 is a solution of (31), so the vector Ax0 is tangent to a solution of (31) at x0 . (Here, o() denotes terms that satisfy lim→0 o()/ = 0.) Consider the problem of characterizing the set of all such tangent directions. Formally, let Q(x0 ) be the set of all directions v ∈ Rn that satisfy the following property. There exists a solution x(t) of (31) and a continuous function T : R+ → R+ , with lim→0 T () = 0, such that x(T ()) = x0 + v + o(). The characterization of Q(x0 ) is a fundamental topic in geometric control theory (Hermes, 1978; Jakubczyk, 2001). It is obvious that ±Ax0 , ±Bx0 ∈ Q(x0 ). It is not difficult to show that any convex combination of these directions (e.g., 1 3 4 Ax0 + 4 Bx0 ) is also in Q(x0 ).  (C )k Using the expansion exp(C) = ∞ k=0 k! , it is easy to verify that exp(A) exp(B) exp(−A) exp(−B)x0 = (I + [A, B]2 + o(2 ))x0 , and since the left-hand side of this equation is a solution of (31), we see that [A, B](x0 ) ∈ Q(x0 ) (see Fig. 11). Thus, using a suitable switching, we “produced” the tangent direction [A, B]x0 . We can now find a switching that “produces” the direction [A, [A, B]]x0 and inductively, any 9 We say that a DI x˙ ∈ D(x) is symmetric if z ∈ D(x) implies that −z ∈ D(x).

2070

M. Margaliot / Automatica 42 (2006) 2059 – 2077

other direction in {A, B}LA (x0 ). Thus, {A, B}LA (x0 ) ⊆ Q(x0 ). For symmetric and smooth systems, such as (31), {A, B}LA (x0 ) = Q(x0 ) (see, e.g., Jakubczyk, 2001). In other words, the Lie algebra describes the structure of the group generated by the flows near the identity. 5.3. Nilpotent Lie algebras Recall that a matrix A ∈ Rn×n is called nilpotent if there exists an integer k > 0 such that Ak = 0. This is the case, for example, when A is strictly upper triangular. Definition 4. We say that the Lie algebra {A, B}LA is nilpotent if there exists an integer k such that all iterated Lie brackets containing k + 1 terms vanish. In this case, the smallest such k is the order of nilpotency of {A, B}LA . Example 8. Consider the strictly upper triangular matrices ⎛ ⎞ ⎛ ⎞ 0 a1 a2 0 b1 b2 ⎜ ⎟ ⎜ ⎟ A = ⎝ 0 0 a 3 ⎠ , B = ⎝ 0 0 b3 ⎠ . 0

0

0

0

0

Example 9. Consider the switched system (32). For T > 0, q > 1, and an initial value x(0) = x0 , denote  R(T ; x0 , q) :=

exp(Cq tq ) . . . exp(C1 t1 )x0 : Ci ∈ {A, B},

ti
0,

0

0

a1 b3 − a3 b1

⎜ [A, B] = ⎝ 0

0

0

0

0

0

⎞ ⎟ ⎠

ti = T

,

that is, the set of states that can be reached at time T using up to q − 1 switchings. Suppose that Ax and Bx commute. Then exp(As) exp(Bt) = exp(Bt) exp(As) so R(T ; x0 , q) = {exp(C2 2 ) exp(C1 1 )x0 : Ci ∈ {A, B}, 1 , 2
0, 1 + 2 = T }.

∀T
0 ∀x0 ∈ Rn .

In other words, adding more switchings (after the first one) does not change the reachable set. In particular, it is not difficult to show that this implies that the switched system is GAS (recall that we always assume that the subsystems are asymptotically stable, that is, A and B are Hurwitz matrices).

and

[A, [A, B]] = [B, [A, B]] = 0. Hence, if a1 b3 = a3 b1 , then {A, B}LA is first-order nilpotent and if a1 b3 = a3 b1 , then {A, B}LA is second-order nilpotent. Note that kth order nilpotency implies that the expression on the right-hand side of (29) becomes a polynomial in t with degree k − 1. This is closely related to the fact that the solution of control systems with a nilpotent Lie algebra can be represented using a differential equation with polynomial vector fields (Hermes, 1982; Sussmann, 1986). This makes systems with nilpotent Lie algebras more amenable to analysis and, therefore, suitable candidates for approximating general control systems (Brockett, 1973; Hermes, 1991, 1982; Lafferriere & Sussmann, 1993). 6. The Lie-algebraic approach Another approach for tackling the GAS problem for switched systems is based on studying the commutation relations between the subsystems using Lie-algebraic techniques (Liberzon, 2003). 6.1. Linear switched systems Consider the linear switched system x ∈ Rn ,



i=1

R(T ; x0 , q) = R(T ; x0 , 2) ⎛

q 

Thus, for any q
2

0

Then,

x˙ = A
x,

where
: R+ → {1, 2}. For notational convenience, we denote A = A1 and B = A2 . The next simple example demonstrates how the commutation relations between A and B affect the trajectories of the switched system.

(32)

Using a clever manipulation of the BCH formula, Gurvits (1995) proved that second-order nilpotency implies that R(T ; x0 , q) = R(T ; x0 , 5) for any integer q > 5. In fact, Gurvits’ proof remains valid if we change the matrix exponentials into operators corresponding to the flows of nonlinear vector fields. He also posed the following conjecture.10 Conjecture 1 (Gurvits, 1995). Consider the linear switched system (32). If {A, B}LA is kth-order nilpotent, then there exists an integer s = s(k) such that R(T ; x0 , q) = R(T ; x0 , s) for all q > s, all T > 0, and all x0 ∈ Rn . Note that the bound on the number of switches s does not depend on T. This implies that if this conjecture is true then nilpotency implies GAS for switched linear systems. Liberzon, Hespanha, and Morse (1999) used a different approach, based on Lie’s third theorem, to prove that if the Lie algebra of the matrices is solvable, then the switched linear system is GAS.11 This result includes nilpotent Lie algebras of any order as a special case. Agrachev and Liberzon 10 Gurvits’ original formulation is somewhat more general. We state it here in a form which is more suitable for our purposes. 11 This is not hard to show by a direct analysis of the transition matrix, or by constructing a quadratic common Lyapunov function via the iterative procedure proposed in Narendra and Balakrishnan (1994); see also Shorten et al. (2005b).

M. Margaliot / Automatica 42 (2006) 2059 – 2077

2071

(2001) generalized this to a larger class of matrix algebras and also showed that no further generalization based on the Lie bracket relations alone is possible.

as the solution of the adjoint state equation  T  T ˙
= − jf
− jg
u∗ ,
(T ) = x∗ (T ), jx jx

6.2. Nonlinear switched systems

and let m(t) :=
T (t)g(x∗ (t)). Then,  1 if m(t) > 0, ∗ u (t) = 0 if m(t) < 0.

Nonlinear switched systems are much less thoroughly understood than linear switched systems. Since nonlinear switched systems include the class of nonlinear differential equations (NDIs) as a special case, it is clear that one cannot hope for a general stability theory. Instead, a more modest aim must be posed, namely, determining sufficient (rather than necessary and sufficient) conditions for GAS. Mancilla-Aguilar (2000) proved that if {f1 , f2 }LA is firstorder nilpotent (i.e., [f1 , f2 ](x) = 0), then the corresponding nonlinear switched system is GAS. Until very recently, all attempts to formulate GAS conditions beyond the commuting nonlinear case have been unsuccessful, as the methods used for linear switched systems do not seem to apply. These issues are explained in Liberzon (2004), where this is presented as an open problem which seems to require a different approach altogether. 7. Merging the variational and Lie-algebraic approaches It is well-known that Lie brackets play a vital role in the classical MP of optimal control (see, e.g., Agrachev & Sachkov, 2004; Sussmann, 1983). This connection between Lie brackets and the MP suggests that it is possible to merge the variational and Lie-algebraic approaches for stability analysis. We now review some recent results obtained in this direction. To apply the variational approach, rewrite (7) as x˙ = f(x) + g(x)u,

u ∈ U,

(33)

where f := f and g := f − f . We assume from hereon that the subsystems x˙ = f(x) and x˙ = f(x) + g(x) are GAS. This implies, in particular, that the solutions of (33) are well-defined for any piecewise constant control attaining values in {0, 1}. Fixing an initial value x(0) = x0 , it is easy to show that there exists a time Tm > 0 (that depends on x0 ) such that all the solutions of (33) are well-defined for all t < Tm . Fixing a final time T < Tm , consider the following optimal control problem. 1

2

1

Problem 5. Find a control u ∈ U that maximizes J (u) := |x(T )|2 . Note that this is the finite-horizon version of the problem addressed by Pyatnitskiy. Since T < Tm an optimal control indeed exists. We will show later that under our hypotheses, solutions of (33) exist globally, so we can take T to be arbitrarily large. The MP yields the following necessary condition for optimality. Theorem 8. Suppose that u∗ is an optimal control and let x∗ denote the corresponding trajectory. Define
: R+ → Rn

(34)

The connection between the MP and the Lie algebra of the control system is most evident in a proof of the MP in a geometric framework (Agrachev & Sachkov, 2004). The general proof is beyond the scope of this paper. Yet, for the particular case of Problem 5 with linear vector fields, it is easy to give a simple and self-contained proof. Since it provides considerable insight, we include it in the Appendix. Theorem 8 implies that if m has isolated zeros on [0, T ], then u∗ (t) ∈ {0, 1} for (almost) all t ∈ [0, T ]. Such a control is called bang-bang. If there exists a time interval I ⊆ [0, T ], with positive measure, such that m(t) ≡ 0 on I, then we say that the optimal control problem is singular (Bonnard & Chyba, 2003). In this case, the MP does not necessarily yield enough information to uniquely determine u∗ (t) on I. One manifestation of the connection between the MP and the Lie-algebraic properties of the system is revealed by studying the derivatives of the (absolutely continuous) function m. Indeed, using (33) and (34) yields m(t) ˙ =
(t)([f, g] + u∗ (t)[g, g])(x∗ (t)) =
(t)[f, g](x∗ (t)).

(35)

This implies that m ˙ is absolutely continuous and differentiating again yields m(t) ¨ =
(t)([f, [f, g]] + u∗ (t)[g, [f, g]])(x∗ (t)).

(36)

In general, further differentiation cannot be carried out because of the term u∗ (t). However, we can differentiate on time intervals where u∗ (t) is smooth. Proposition 2 (Sharon and Margaliot (2005)). Suppose that {f, g}LA is nilpotent of order k. Let I be a time interval such that u∗ (t) ≡ c, t ∈ I . Then the restriction of m(t) on I is a polynomial in t with degree k − 1. Proof. Using (36) yields m(t) ¨ =
(t)([f, [f, g]] + c[g, [f, g]])(x∗ (t)),

t ∈ I,

so m ¨ is absolutely continuous on I and we can continue to differentiate it. Inductively, we see that m(k) (t) contains iterated Lie brackets of f and g with k + 1 terms. Hence, m(k) (t) = 0 on I, and using the absolute continuity of m and its derivatives completes the proof.  In particular, Proposition 2 implies that in the bang-bang case m(t) is piecewise polynomial; every piece is a polynomial in t with degree k − 1. For low orders of nilpotency, this leads to strong restrictions on the optimal control.

2072

M. Margaliot / Automatica 42 (2006) 2059 – 2077

Proposition 3 (Margaliot and Liberzon (2006)). Consider Problem 5 for the NDI (33). If {f, g}LA is first (second) order nilpotent then for any T > 0 there exists an optimal control u∗ that is bang-bang with no more than one (two) switches. Proof. Consider the case of first-order nilpotency. In this case, (35) yields m(t) ˙ = 0, so m(t) = a, for some a ∈ R. If a = 0 then Theorem 8 implies that u∗ is bang-bang with zero switches for any final time T > 0. The singular case a = 0 can be addressed using the concept of strong extremality, introduced by Sussmann (1979), proving that there exists an optimal control u∗ that is bang-bang with no more than one switch for any final time T > 0. We now turn to the case of second-order nilpotency. It follows from (36) that in this case m(t) ¨ = 0, so m(t) = at + b, and a similar argument completes the proof.  The next result links these regularity properties of u∗ and the stability of the NDI. Proposition 4. Suppose that: (1) the subsystems x˙ = f(x) and x˙ =f(x)+g(x) are GAS; and (2) there exists an absolute integer d such that the following property holds: for any T > 0 there exists an optimal control u∗ that is bang-bang with no more than d switches on [0, T ]. Then the NDI (33) is GAS. Note that the bound d does not depend on T. This makes it possible to prove the proposition by building a class KL function, using the separate class KL functions i of each GAS subsystem, such that (8) is satisfied for x∗ (t) (compare with Example 3). The definition of x∗ implies that (8) is satisfied for every solution of the NDI. Combining Propositions 3 and 4 yields the following: Theorem 9. Consider the NDI (33). Suppose that the two subsystems x˙ =f(x) and x˙ =f(x)+g(x) are GAS. If {f, g}LA is either first-order or second-order nilpotent then the NDI is GAS. The third-order nilpotent case turns out to be more difficult. In this case, m(t)=
(t)([f, ¨ [f, g]]+u∗ (t)[g, [f, g]])(x∗ (t)) is not zero, and further differentiation cannot be carried out because of the term u∗ (t). It is possible to overcome this difficulty by using a powerful second-order MP, developed by Agrachev and Gamkrelidze (1990) (see also Agrachev & Sigalotti, 2003). To state the result in this case, let x(t; c, x0 ) denote the solution at time t of (33) for the constant control u(t) ≡ c and the initial condition x(0) = x0 . Theorem 10 (Sharon and Margaliot (2005)). Consider Problem 5 for the NDI (33). Suppose that {f, g}LA is third-order nilpotent. Then for any T > 0 there exists an optimal control u∗ which is piecewise constant with no more than four switches. If, furthermore, there exists a function  ∈ KL such that for any fixed c ∈ [0, 1] |x(t; c, x0 )|(|x0 |, t) ∀x0 ∈ Rn ∀t
0, then the NDI is GAS.

(37)

Theorems 9 and 10 provide a partial solution to the open problem posed in Liberzon (2004). As we will see in Section 8.3, they are also closely related to a nonlinear version of Conjecture 1. Before concluding this section, it is important to note that the conditions of nilpotency, being based on equality relations (e.g., [f, g](x) = 0) are non-generic. Thus, results based on nilpotency are fragile with respect to perturbations (unless, of course, the problem admits some special structure guaranteeing that the perturbations do not destroy the nilpotency property). Nevertheless, it should be clear from the discussion above that the analysis of switched systems using Lie-algebraic tools is of considerable importance, as the set of Lie brackets describes the fundamental intrinsic properties of the system. 8. Open problems In this section, we describe some open problems and possible directions for further research. 8.1. The absolute stability problem and generalized first integrals Theorem 1 provides important information on the behavior of LDIs on the edge of stability (that is, when k =k ∗ ). It implies that there exists a trajectory x∗ that satisfies ϑ(x∗ (t)) = const for all t
0. At points where ϑ is differentiable, this yields 0=

d ϑ(x∗ (t)) dt

= ϑx (x∗ (t))˙x∗ (t). Recalling that x∗ (t) satisfies either x˙ ∗ (t) = Ax∗ (t) or x˙ ∗ (t) = Bk ∗ x∗ (t), we see that ϑ must be composed of generalized first integrals of the linear subsystems. For the case n = 2, discussed in Section 4.1, it is possible to construct explicitly the generalized first integrals of the linear subsystems, and to combine them to provide a complete solution to the absolute stability problem. Since Theorem 1 holds for any n, it seems that a viable line of research is to study how to construct and combine the generalized first integrals of higher-order linear systems. This may provide new insights into the absolute stability problem. 8.2. Finite-horizon optimal control An interesting topic for further research is the dynamic programming solution of the finite-horizon optimal control problem for nilpotent control systems. More specifically, fixing T > 0 and x(0) ∈ Rn , consider the problem of maximizing J (u) := |x(T )|2 for the LDI x˙ (t) = Ax(t) + u(t)(B − A)x(t),

u ∈ U.

(38)

M. Margaliot / Automatica 42 (2006) 2059 – 2077

The value function for this problem is defined as V (t, y) := max |x(T ; u, t, y)|2 , u∈U

8.3. Nilpotency and reachability with nice controls (39)

where x(T ; u, t, y) is the solution at time T of (38) satisfying the initial condition x(t) = y. The value function satisfies the HJB equation 0 = max {Vt + Vx (A + u(B − A))x} u∈U

(40)

with the boundary condition V (T , y) = |y|2 .

(41)

Deriving an explicit expression for V implies the solution of the finite-time optimal control problem since u∗ is the maximizer in (39). However, such explicit solutions are very rare (Margaliot & Langholz, 2001). In the nilpotent case, we might expect u∗ to have a simple form and, therefore, so should V . We demonstrate this using the following example.
 Example 10. Consider the LDI (38) with n = 2, A = 00 01 ,
 ∗ and B = 00 −1 0 . Since [A, B] = 0, we suspect that u has no switchings, that is, either u∗ ≡ 0 or u∗ ≡ 1. Then, V (t, y) = yT exp(AT (T − t)) exp(A(T − t))y or V (t, y) = yT exp(B T (T − t)) exp(B(T − t))y, respectively. Hence, we speculate that there exists a region S ∈ R2 such that  W (t, x; A) if x ∈ S, V (t, x) = (42) W (t, x; B) otherwise, where W (t, y; C) := yT exp(C T (T − t)) exp(C(T − t))y. Note that (42) satisfies the boundary condition (41). Consider a point x ∈ S. Then (42) yields Vt + Vx Ax = 0, and Vt + Vx Bx = −4x1 x2 − 4(T − t)x22 , so (40) holds for / S, (42) yields Vt + Vx Bx = 0, and x1 x2
0. Similarly, for x ∈ Vt +Vx Ax=4x1 x2 −4(T −t)x22 , so (40) holds if x1 x2 0. This implies that (42) is indeed the solution (in the viscosity sense, Bardi & Capuzzo-Dolcetta, 1997) of the HJB equation for S := {x ∈ R2 : x1 x2
0}. Note that this also provides an explicit expression for the optimal control, namely, ∗

u (t) =



2073

Consider the nonlinear control system x˙ = f(x) + ug(x),

x(0) = x0

(43)

with u(t) ∈ [0, 1] for all t. To guarantee that solutions to various optimal control problems exist, it is necessary to define the set of admissible controls U as the set of all measurable functions u : R+ → [0, 1] (Kupka, 1990; Young, 2000). Letting x(t; u, x0 ) denote the solution at time t of (43), R(T ; U, x0 ) := {x(T ; u, x0 ) : u ∈ U} is then the reachable set at time T. There is considerable interest in identifying sets of controls W ⊂ U, that are in some sense regular, such that R(T ; U, x0 ) = R(T ; W, x0 )

∀x0 ∈ Rn .

In other words, any point that can be reached using a control u ∈ U can also be reached using a control w ∈ W. We refer to (44) as a nice reachability property. If it holds for all final time T, and W does not depend on T, then we say that the result is global in time. If, on the other hand, (44) holds only for sufficiently small T > 0, or if the complexity of W increases with T, then we refer to it as a local result. There is rich literature on conditions guaranteeing various local regularity properties (see, e.g., Schattler, 1990, 1998; Sigalotti, 2005; Sussmann, 1979 and the references therein). The most famous nice reachability result is the classic bangbang theorem stating that for linear control systems (44) holds, where W is the set of bang-bang controls. For a precise statement see, e.g., Sussmann and Nohel, 1998 (see also Artstein, 1980). This is a local result as, in general, the number of required switches increases with the final time T. Sussmann (1979) introduced a mechanism for transforming regularity properties of time-optimal controls into general reachability with nice control results. Using this, it is possible to generalize Proposition 3 and Theorem 10. To state these generalizations, we introduce the following notation. For  ∈ R+ and an integer k > 0, let BB(, k) ⊂ U [PC(, k) ⊂ U] denote the set of bang-bang (piecewise constant) controls that have no more than k switches in the interval [0, ]. Theorem 11. If {f, g}LA is first-order nilpotent, then R(T ; U, x0 ) = R(T ; BB(T , 1), x0 )

∀T > 0 ∀x0 ∈ Rn .

If {f, g}LA is second-order nilpotent, then R(T ; U, x0 ) = R(T ; BB(T , 2), x0 )

∀T > 0 ∀x0 ∈ Rn .

0

if x(t) ∈ S,

If {f, g}LA is third-order nilpotent, then

1

otherwise.

R(T ; U, x0 ) = R(T ; PC(T , 4), x0 )

This example may seem trivial because the dynamics is actually one-dimensional. Nevertheless, the important aspect here is that the matrix exponentials of each subsystem can be combined to obtain a simple and explicit closed-form expression for the value function. In general, such solutions are very rare, even for one-dimensional problems (Bellman & Bucy, 1964).

(44)

∀T > 0 ∀x0 ∈ Rn .

The proof of the second (third) part of this theorem can be found in Margaliot and Liberzon (2006) (Sharon & Margaliot, 2005). An interesting feature of this result is that it is global in time, as the bound on the number of switchings is independent of the final time T. This has an important practical implication, as

2074

M. Margaliot / Automatica 42 (2006) 2059 – 2077

any point-to-point control problem is reduced to determining a finite (and small) set of switching points and control values between every two consecutive switchings. Thus, efficient numerical algorithms can be used to solve problems such as motion planning (Lafferriere & Sussmann, 1993; Murray & Sastry, 1993) or finding time-optimal controls (Walther, Georgiou, & Tannenbaum, 2001). Interesting topics for further research include the following. First, the study of reachability with nice controls for higher orders of nilpotency. This is of course closely related to the questions posed by Gurvits (1995) and Liberzon (2004). Second, although nilpotent control systems are quite special, there are results on both: transforming non-nilpotent systems into nilpotent ones using feedback (Lafferriere & Sussmann, 1993), and approximating non-nilpotent control systems using nilpotent ones (Hermes, 1991; Lafferriere & Sussmann, 1993). Further study of the implications of the above reachability results in this context may be of interest. 9. Summary and conclusions Switched systems and DIs provide useful mathematical models for many natural and artificial systems and processes. The stability analysis of these models is not only a formidable theoretical challenge, but may also have profound implications in many fields of science. In the arbitrary switching case, the analysis of the switched system is closely related to the analysis of the corresponding DI. In particular, conditions guaranteeing that the DI is GAS automatically imply that the switched system is GUAS. The difficulty in the stability analysis of DIs is due to two factors. First, unlike (well-behaved) ordinary differential equations, a DI admits an infinite set of trajectories for any initial condition. Second, the trajectories of a DI can be much more complex than those of its separate subsystems. We reviewed an important approach for the stability analysis of DIs that is based on the natural idea of studying the “most unstable” trajectory of the DI. We also discussed how it can be merged with another approach to stability analysis that is based on analyzing the commutation relations between the subsystems composing the DI. These approaches utilize many tools from several fields of applied mathematics including variational calculus and differential geometry. We tried to provide an accessible survey of these approaches, as well as to suggest several directions for further research. Acknowledgments I thank Daniel Liberzon for enlightening discussions. I am grateful to the anonymous reviewers and the editor for many valuable comments.

Corollary 1. Suppose that u∗ is an optimal control and let x∗ denote the corresponding trajectory. Define
: R+ → Rn as the solution of the adjoint state equation
˙ = −AT
− B T
u∗ ,


(T ) = x∗ (T ),

(45)

and let m(t) :=
T (t)Bx∗ (t). Then,  1 if m(t) > 0, u∗ (t) = 0 if m(t) < 0.

(46)

The following proof of Corollary 1 highlights the connection between the MP and {A, B}LA . Proof. It is easy to verify that x∗ (t) = C ∗ (t; t0 )x(t0 ), for any t
t0
0, where C ∗ (t; t0 ) denotes the solution at time t of the matrix differential equation C˙ = (A + u∗ B)C,

C(t0 ; t0 ) = I .

(47)

It follows from this that (C ∗ (t; t0 ))−1 exists and that for t3
t2
t1 : C ∗ (t3 ; t1 ) = C ∗ (t3 ; t2 )C ∗ (t2 ; t1 ).

(48)

Note that using (45) and (47) yields (d/dt)(
(t)C ∗ (t; t0 ))=0T , so T


T (t)C ∗ (t; t0 ) =
T (t0 ).

(49)

Fix  ∈ (0, T ) and c ∈ [0, 1], and consider the control  c if t ∈ [,  + ), u(t; ˜ c, , ) = ∗ u (t) otherwise where  > 0. Note that u(t; ˜ c, , 0)=u∗ (t) so u˜ is a perturbation of the optimal control. We will consider the case  → 0 for which u˜ ∈ U, that is, u˜ is an admissible control. Let x˜ denote the corresponding trajectory. We are interested in deriving an expression for the difference x˜ (T ) − x∗ (T ). To do so, note that x˜ satisfies x˙ = (A + uB)x ˜ = (A + u∗ B)x + (u˜ − u∗ )Bx. Arguing as in the derivation of Theorem 7 yields x˜ (t) = C ∗ (t; 0)˜y(t), where y˙˜ (s) = (u(s) ˜ − u∗ (s))(C ∗ (s; 0))−1 BC ∗ (s; 0)˜y(s) with y˜ (0) = x0 . The definition of u˜ implies y˜ () = y˜ (0) = x0 , y˜ (T ) = y˜ ( + ), and y˙˜ () = (c − u∗ ())(C ∗ (; 0))−1 BC ∗ (; 0)˜y(). Hence, y˜ (T ) − y˜ ()

Appendix

= (c − u∗ ())(C ∗ (; 0))−1 BC ∗ (; 0)˜y() + o()

Consider Problem 5 for the LDI (33) with f(x) = Ax and g(x) = Bx. In this case, Theorem 8 specializes to the following result.

= (c − u∗ ())(C ∗ (; 0))−1 BC ∗ (; 0)x0 + o() = (c − u∗ ())(C ∗ (; 0))−1 Bx∗ () + o().

(50)

M. Margaliot / Automatica 42 (2006) 2059 – 2077

Thus, x˜ (T ) − x∗ (T ) = C ∗ (T ; 0)˜y(T ) − x∗ (T ) = (c − u∗ ())C ∗ (T ; 0)(C ∗ (; 0))−1 Bx∗ () + o() = (c − u∗ ())C ∗ (T ; )Bx∗ () + o(), where the last equation follows from (48). Multiplying both sides of this equation by (x∗ (T ))T , and using (45) and (49), yields (x∗ (T ))T (˜x(T ) − x∗ (T )) = (c − u∗ ())
T (T )C ∗ (T ; )Bx∗ () + o() = (c − u∗ ())
T ()Bx∗ () + o().

(51)

If (x∗ (T ))T (˜x(T ) − x∗ (T )) > 0 then |˜x(T )| > |x∗ (T )| and this contradicts the optimality of u∗ , so (c − u∗ ())
T ()Bx∗ () + o() 0, and taking a sufficiently small  yields c
T ()Bx∗ ()u∗ ()
T ()Bx∗ (). Recalling that  ∈ (0, T ) and c ∈ [0, 1] are arbitrary, this proves (46).  The connection to {A, B}LA should be evident from (50). Indeed, reasoning as in the proof of Theorem 6 yields (C ∗ (t; 0))−1 BC ∗ (t; 0) = B + (C ∗ (t; 0))−1 [B, A]C ∗ (t; 0)t +(C ∗ (t; 0))−1 ([[B, A], A]+u∗ (t)[[B, A], B])C ∗ (t; 0)

t2 2

+ ···. References Agrachev, A. A., & Gamkrelidze, R. V. (1990). Symplectic geometry for optimal control. In H. J. Sussmann (Ed.), Nonlinear controllability and optimal control (pp. 263–277). New York: Marcel Dekker. Agrachev, A. A., & Liberzon, D. (2001). Lie-algebraic stability criteria for switched systems. SIAM Journal on Control and Optimization, 40, 253–269. Agrachev, A. A., & Sachkov, Y. L. (2004). Control theory from the geometric viewpoint. Encyclopedia of mathematical sciences (Vol. 87). Berlin: Springer. Agrachev, A. A., & Sigalotti, M. (2003). On the local structure of optimal trajectories in R3 . SIAM Journal on Control and Optimization, 42, 513–531. Angeli, D. (1999). A note on stability of arbitrarily switched homogeneous systems. Preprint. Antsaklis, P. J. (Ed.), (2000). Proceedings of the IEEE, special issue on hybrid systems: theory and applications (Vol. 88(7)). Artstein, Z. (1980). Discrete and continuous bang-bang and facial spaces or: Look for the extreme points. SIAM Review, 22, 172–185.

2075

Artstein, Z. (1996). Examples of stabilization with hybrid feedback. In R. Alur, T.A. Henzinger, E.D. Sontag (Eds.), Hybrid systems III: Verification and control. Lecture notes in computer science (Vol. 1066). Berlin: Springer. pp. 173–185. Aubin, J.-P., & Cellina, A. (1984). Differential inclusions: Set-valued maps and viability theory. Berlin: Springer. Barabanov, N. E. (1988). Absolute characteristic exponent of a class of linear nonstationary systems of differential equations. Siberian Mathematical Journal, 29, 521–530. Barabanov, N. E. (1989). Method for the computation of the Lyapunov exponent of a differential inclusion. Automation and Remote Control, 50, 475–479. Barabanov, N. E. (1993). On the Aizerman problem for third-order nonstationary systems. Differential Equations, 29, 1439–1448. Barabanov, N. E. (2005). Lyapunov exponent and joint spectral radius: Some known and new results. In Proceedings of the 44th IEEE conference on decision and control (pp. 2332–2337). Bardi, M., & Capuzzo-Dolcetta, I. (1997). Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Basel: Birkhäuser. Bellman, R., & Bucy, R. (1964). Asymptotic control theory. SIAM Journal on Control and Optimization, 2(1), 11–18. Blanchini, F. (1999). Set invariance in control. Automatica, 35(11), 1747–1767. Blondel, V. D., & Tsitsiklis, J. N. (2000). A survey of computational complexity results in systems and control. Automatica, 36, 1249–1274. Bonnard, B., & Chyba, M. (2003). Singular trajectories and their role in control theory. Berlin: Springer. Borrelli, F. (2003). Constrained optimal control of linear and hybrid systems. Lecture notes in control and information sciences (Vol. 290). Berlin, Springer. Boscain, U. (2002). Stability of planar switched systems: The linear single input case. SIAM Journal on Control and Optimization, 41, 89–112. Boyd, S., El Ghaoui, L., Feron, E., & Balakrishnan, V. (1994). Linear matrix inequalities in system and control theory. SIAM studies in applied mathematics (Vol. 15). Philadelphia, PA: SIAM. Brockett, R. W. (1973). Lie algebras and Lie groups in control theory. In: D. Q. Mayne, & R. W. Brockett (Eds.), Geometric methods in system theory (pp. 43–82). Boston: D. Reidel Publishing Company. Camilli, F., Grune, L., & Wirth, F. (2001). A generalization of Zubov’s method to perturbed systems. SIAM Journal on Control and Optimization, 40, 496 –515. Chase, C., Serrano, J., & Ramadge, P. (1993). Periodicity and chaos from switched flow systems: Contrasting examples of discretely controlled continuous systems. IEEE Transactions on Automatic Control, 38, 70–83. Colonius, F., Kliemann, W., & Grune, L. (2000). The dynamics of control. Basel: Birkhäuser. Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2001). Introduction to algorithms. (2nd ed.), Cambridge, MA: MIT Press. Cuk, S. (1992). Basics of switched-mode power conversion: Topologies, magnetics and control. In B. K. Bose (Ed.), Modern power electronics: evolution, technology, and applications (pp. 265–296). New York: IEEE Press. Dayawansa, W. P., & Martin, C. F. (1999). A converse Lyapunov theorem for a class of dynamical systems which undergo switching. IEEE Transactions on Automatic Control, 44(4), 751–760. de Jong, H., Gouze, J.-L., Hernandez, C., Page, M., Sari, T., & Geiselmann, J. (2004). Qualitative simulation of genetic regulatory networks using piecewise-linear models. Bulletin of Mathematical Biology, 66, 301–340. Elliott, D. L. (1999). Bilinear systems. In J. Webster (Ed.), Encyclopedia of electrical engineering, New York: Wiley. URL: http://www.isr.umd.edu/∼delliott/Bilinear.pdf. Erickson, R. W., & Maksimovic, D. (2001). Fundamentals of power electronics. Dordrecht: Kluwer Academic Publishers. Filippov, A. F. (1962). On certain questions in the theory of optimal control. SIAM Journal on Control and Optimization, 1, 76–84. Filippov, A. F. (1980). Stability conditions in homogeneous systems with arbitrary regime switching. Automation and Remote Control, 41(8), 1078–1085.

2076

M. Margaliot / Automatica 42 (2006) 2059 – 2077

Goldstein, H. (1980). Classical mechanics. Reading, MA: Addison-Wesley. Gurvits, L. (1995). Stability of discrete linear inclusion. Linear Algebra and its Applications, 231, 47–85. Hall, B. C. (2003). Lie groups, Lie algebras, and representations: An elementary introduction. Berlin: Springer. Hermes, H. (1978). Lie algebras of vector fields and local approximation of attainable sets. SIAM Journal on Control and Optimization, 16, 715–727. Hermes, H. (1982). Control systems which generate decomposable Lie algebras. Journal of Differential Equations, 44, 166–187. Hermes, H. (1991). Nilpotent and high-order approximations of vector field systems. SIAM Review, 33, 238–264. Hespanha, J., Bohacek, S., Obraczka, K., & Lee, J. (2001). Hybrid modeling of TCP congestion control. In M.D. Di Benedetto, & A. SangiovanniVincentelli (Eds.), Hybrid systems: Computation and control. Lecture notes in computer science (Vol. 2034. pp. 291–304). Berlin: Springer. Hespanha, J. P. (2004). Uniform stability of switched linear systems: extensions of LaSalle’s invariance principle. IEEE Transactions on Automatic Control, 49, 470–482. Hespanha, J. P., & Morse, A. S. (2002). Switching between stabilizing controllers. Automatica, 38(11), 1905–1917. Holcman, D., & Margaliot, M. (2003). Stability analysis of switched homogeneous systems in the plane. SIAM Journal on Control and Optimization, 41(5), 1609–1625. Howe, R. (1983). Very basic Lie theory. American Mathematical Monthly, 90, 600–623. Isidori, A. (1995). Nonlinear control systems. (3rd ed.), Berlin: Springer. Jakubczyk, B. (2001). Introduction to geometric nonlinear control. URL: http://www.impan.gov.pl/User/jakubczy/. Johansson, K. H., Lygeros, J., & Sastry, S. (2004). Modeling of hybrid systems. In H. Unbehauen (Ed.), Encyclopedia of life support systems (EOLSS). URL: www.s3.kth.se/∼kallej/papers/eolss_6.43.28.1.pdf. Johansson, M., (2003). Piecewise linear control systems. Lecture notes in control and information sciences (Vol. 284). Berlin: Springer. Jurdjevic, V. (1996). Geometric control theory. Cambridge: Cambridge University Press. Kawski, M. (2004). Bases for Lie algebras and a continuous CBH formula. In: V. Blondel, & A. Megretski (Eds.), Unsolved problems in mathematical systems and control theory (pp. 97–102). Princeton, NJ: Princeton University Press. Kupka, I. A. K. (1990). The ubiquity of Fuller’s phenomenon. In H. J. Sussmann (Ed.), Nonlinear controllability and optimal control (pp. 313– 350). New York: Marcel Dekker. Lafferriere, G., & Sussmann, H. J. (1993). Differential geometric approach to motion planning. In: Z. Li, & J. F. Canny (Eds.), Nonholonomic motion planning (pp. 235–270). Dordrecht: Kluwer Academic Publishers. Levin, A. Y. (1961). On the stability of solutions of equations of second order. Soviet Mathematics, 2, 1642–1646. Liberzon, D. (2003). Switching in systems and control. Boston: Birkhäuser. Liberzon, D. (2004). Lie algebras and stability of switched nonlinear systems. In: V. Blondel, & A. Megretski (Eds.), Unsolved problems in mathematical systems and control theory (pp. 203–207). Princeton, NJ: Princeton University Press. Liberzon, D., Hespanha, J. P., & Morse, A. S. (1999). Stability of switched systems: A Lie-algebraic condition. Systems and Control Letters, 37, 117–122. Liberzon, D., & Morse, A. S. (1999). Basic problems in stability and design of switched systems. IEEE Control Systems Magazine, 19, 59–70. Lin, Y., Sontag, E. D., & Wang, Y. (1996). A smooth converse Lyapunov theorem for robust stability. SIAM Journal on Control and Optimization, 34, 124–160. Mancilla-Aguilar, J. L. (2000). A condition for the stability of switched nonlinear systems. IEEE Transactions on Automatic Control, 45, 2077–2079. Margaliot, M., & Gitizadeh, R. (2004). The problem of absolute stability: A dynamic programming approach. Automatica, 40(7), 1247–1252. Margaliot, M., & Langholz, G. (2001). Some nonlinear optimal control problems with closed-form solutions. International Journal of Robust and Nonlinear Control, 11, 1365–1374.

Margaliot, M., & Langholz, G. (2003). Necessary and sufficient conditions for absolute stability: The case of second-order systems. IEEE Transactions on Circuits and Systems.-I, 50, 227–234. Margaliot, M., & Liberzon, D. (2006). Lie-algebraic stability conditions for nonlinear switched systems and differential inclusions. Systems and Control Letters, 55(1), 8–16. Margaliot, M., & Yfoulis, C., (2006). Absolute stability of third-order systems: A numerical algorithm. Automatica. URL: http://www.eng.tau. ac.il/∼michaelm, to appear. Matveev, A. S., & Savkin, A. V. (2000). Qualitative theory of hybrid dynamical systems. Boston: Birkhäuser. McClamroch, N. H., & Kolmanovsky, I., (2000). Performance benefits of hybrid control design for linear and nonlinear systems. Proceedings of the IEEE 88, 1083–1096. Megretski, A., & Rantzer, A. (1997). System analysis via integral quadratic constraints. IEEE Transactions on Automatic Control, 42(6), 819–830. Molchanov, A. P., & Pyatnitskiy, Y. S. (1989). Criteria of asymptotic stability of differential and difference inclusions encountered in control theory. Systems and Control Letters, 13, 59–64. Morse, A. S. (Ed.), 1997. Control using logic-based switching. London: Springer. Morse, A. S., Pantelides, C. C., Sastry, S. S., & Schumacher, J. M. (Eds.), (1999). Automatica, Special Issue on Hybrid Systems, 35(3). Murray, R. M., & Sastry, S. S. (1993). Nonholonomic motion planning: Steering using sinusoids. IEEE Transactions on Automatic Control, 38, 700–716. Narendra, K. S., & Balakrishnan, J. (1994). A common Lyapunov function for stable LTI systems with commuting A-matrices. IEEE Transactions on Automatic Control, 39, 2469–2471. Nonlinear Analysis (2005a), Special Issue on Hybrid Systems and Applications, 62(8). Nonlinear Analysis (2005b), Special Issue on Hybrid Systems and Applications, 63(3). Olver, P. J., 1986. Applications of Lie groups to differential equations. Graduate texts in mathematics (Vol. 107). New York: Springer. Pnueli, A., & Sifakis, J. (Eds.), (1995). Theoretical Computer Science, Special Issue on Hybrid Systems, 138(1). Polanski, A. (2000). On absolute stability analysis by polyhedral Lyapunov functions. Automatica, 36, 573–578. Pukhlikov, A. V. (1998). Optimal bilinear systems on the plane. Differential Equations, 34, 1516–1521. Pyatnitskii, E. S. (1970). Absolute stability of nonstationary nonlinear systems. Automation and Remote Control, 1, 5–15. Pyatnitskii, E. S. (1971). Criterion for the absolute stability of second-order nonlinear controlled systems with one nonlinear nonstationary element. Automation and Remote Control, 1, 5–16. Pyatnitskii, E. S., & Sokorodinskii, V. I. (1982). Numerical methods of Lyapunov function construction and their application to the absolute stability problem. Systems and Control Letters, 2, 130–135. Pyatnitskiy, E. S., & Rapoport, L. B. (1996). Criteria of asymptotic stability of differential inclusions and periodic motions of time-varying nonlinear control systems. IEEE Transactions on Circuits and Systems.-I, 43, 219–229. Rapoport, L. B. (1993). Existence of nonsmooth invariant functions on the boundary of absolute stability domain of nonlinear nonstationary systems. Automation and Remote Control, 54, 448–453. Rapoport, L. B. (1996). Asymptotic stability and periodic motions of selectorlinear differential inclusions. In: F. Garofalo, L. Glielmo (Eds.), Robust control via variable structure and Lyapunov techniques. Lecture notes in control and information sciences (Vol. 217. pp. 269–285). Berlin: Springer. Rasvan, V. (2004). Delay-independent and delay-dependent Aizerman problem. In: V. Blondel, & A. Megretski (Eds.), Unsolved problems in mathematical systems and control theory (pp. 212–220). Princeton, NJ: Princeton University Press. Schattler, H. (1990). Regularity properties of optimal trajectories: Recently developed techniques. In: H. J. Sussmann (Ed.), Nonlinear controllability and optimal control (pp. 263–277). New York: Marcel Dekker.

M. Margaliot / Automatica 42 (2006) 2059 – 2077 Schattler, H. (1998). Time optimal feedback control for nonlinear systems: A geometric approach. In: B. Jakubczyk, & W. Respondek (Eds.), Geometry of feedback and optimal control (pp. 383–421). New York: Marcel Dekker. Sharon, Y., & Margaliot, M. (2005). Third-order nilpotency, finite switchings and asymptotic stability. URL: http://www.eng.tau.ac.il/∼michaelm, submitted for publication. Shorten, R., Leith, D., Foy, J., & Kilduff, R. (2005a). Analysis and design of AIMD congestion control algorithms in communication networks. Automatica, 41, 725–730. Shorten, R., Wirth, F., Mason, O., Wulff, K., & King, C. (2005b). Stability criteria for switched and hybrid systems. URL: www.hamilton.ie/ bob/SwitchedStability.pdf. Sigalotti, M. (2005). Local regularity of optimal trajectories for control problems with general boundary conditions. Journal of Dynamical and Control Systems, 11, 91–123. Smirnov, G. V. (2002). Introduction to the theory of differential inclusions. Providence, RI: American Mathematical Society. Strichartz, R. S. (1987). The Campbell–Baker–Hausdorff–Dynkin formula and solutions of differential equations. Journal of Functional Analysis, 72, 320–345. Sussmann, H. J. (1979). A bang-bang theorem with bounds on the number of switchings. SIAM Journal on Control and Optimization, 17, 629–651. Sussmann, H. J. (1983). Lie brackets, real analyticity and geometric control. In: R. W. Brockett, R. S. Millman, & H. J. Sussmann (Eds.), Differential geometric control theory (pp. 1–116). Boston: Birkhäuser. Sussmann, H. J. (1986). A product expansion for the Chen series. In: C. I. Byrnes, & A. Lindquist (Eds.), Theory and applications of nonlinear control systems (pp. 323–335). Amsterdam: Elsevier Science Publishers. Sussmann, H. J. (1997). Some recent results on the maximum principle of optimal control. In: C. I. Byrnes, B. N. Datta, D. S. Gilliam, & C. F. Martin (Eds.), Systems and control in the twenty-first century (pp. 351– 372). Basel: Birkhäuser. Sussmann, H. J., & Nohel, J. A. (1998). Commentary on Norman Levinson’s paper ‘Minimax, Liapunov and Bang-Bang’. In: J. A. Nohel, D. H. Sattinger (Eds.), Selected papers of Norman Levinson (Vol. 2, pp. 463–475). Boston: Birkhäuser. van der Schaft, A. J., & Schumacher, H. (2000). An Introduction to Hybrid Dynamical Systems. Lecture notes in control and information sciences (Vol. 251). Berlin: Springer.

2077

Vidyasagar, M. (1993). Nonlinear systems analysis. Englewood Cliffs, NJ: Prentice Hall. Vinter, R. (2000). Optimal control. Boston: Birkhäuser. Walther, U., Georgiou, T. T., & Tannenbaum, A. (2001). On the computation of switching surfaces in optimal control: A Grobner basis approach. IEEE Transactions on Automatic Control, 46, 534–540. Xinghuo, J. L., & Chen, G. (2003). Generating chaotic attractors with multiple merged basins of attraction: a switching piecewise-linear control approach. IEEE Transactions on Circuits and Systems.-I, 50, 198–207. Xu, X., & Antsaklis, P. J. (2000). Stabilization of second-order LTI switched systems. International Journal of Control, 73(14), 1261–1279. Yfoulis, C. A. (2002). Stabilisation of nonlinear systems: The piecewise linear approach. Research Studies Press. Yfoulis, C. A., & Shorten, R. (2004). A numerical technique for the stability analysis of linear switched systems. International Journal of Control, 77, 1019–1039. Young, L. C. (2000). Lectures on the calculus of variations and optimal control theory. Providence, RI: American Mathematical Society. Zhengguo, L., Yengchai, S., & Changyun, W. (2005). Switched and impulsive systems: analysis, design and applications. Lecture notes in control and information sciences (Vol. 313). Berlin: Springer.

Michael Margaliot received the B.Sc. (cum laude) and M.Sc. degrees in electrical engineering from the Technion—Israel Institute of Technology—in 1992 and 1995, respectively, and the Ph.D. degree (summa cum laude) from Tel Aviv University in 1999. He was a postdoctoral fellow in the Department of Theoretical Mathematics at the Weizmann Institute of Science. In 2000, he joined the faculty of the Department of Electrical Engineering-Systems, Tel Aviv University. Dr. Margaliot’s research interests include stability analysis of differential inclusions and switched systems, optimal control, fuzzy control, computation with words, and fuzzy modeling of biological phenomena. He is coauthor of New Approaches to Fuzzy Modeling and Control: Design and Analysis, World Scientific, 2000.