Optimal control of discrete-time bilinear systems with applications to switched linear stochastic systems

Systems & Control Letters 94 (2016) 165–171

Contents lists available at ScienceDirect

Systems & Control Letters journal homepage: www.elsevier.com/locate/sysconle

Optimal control of discrete-time bilinear systems with applications to switched linear stochastic systems✩ Ran Huang a,∗ , Jinhui Zhang b , Zhongwei Lin c a

College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, 100029, China

b

School of Electrical and Automation, Tianjin University, Tianjin, 300072, China

c

School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China

article

info

Article history: Received 7 January 2016 Received in revised form 28 April 2016 Accepted 6 June 2016 Available online 12 July 2016 Keywords: Discrete-time Stability analysis Optimal control Bilinear system Switched linear stochastic system

abstract This paper aims at characterizing the most destabilizing switching law for discrete-time switched systems governed by a set of bounded linear operators. The switched system is embedded in a special class of discrete-time bilinear control systems. This allows us to apply the variational approach to the bilinear control system associated with a Mayer-type optimal control problem, and a second-order necessary optimality condition is derived. Optimal equivalence between the bilinear system and the switched system is analyzed, which shows that any optimal control law can be equivalently expressed as a switching law. This specific switching law is most unstable for the switched system, and thus can be used to determine stability under arbitrary switching. Based on the second-order moment of the state, the proposed approach is applied to analyze uniform mean-square stability of discrete-time switched linear stochastic systems. Numerical simulations are presented to verify the usefulness of the theoretic results. © 2016 Elsevier B.V. All rights reserved.

1. Introduction The motivation for the study of switched systems has its roots in industry for two aspects in general. On one hand, many physical systems encountered in practice exhibit switching among a set of subsystems depending on various environmental factors; see e.g., [1] for more details. On the other hand, switched multicontroller systems have numerous applications in the control of mechanical systems, process control, automotive industry, and many other fields [2]. Analysis and synthesis of a switched system have attracted much attention from control community, and fruitful achievements have been developed; see [3–8], and the references therein. A common problem for a switched linear system is that of determining whether it is stable under arbitrary switching. Much effort has been made to approach this problem, resulting in various methods and tools; see the recent survey paper [9,10] and the references therein. This problem is closely related to determine

✩ This work is supported by National Natural Science Foundation of China under Grant 61403018 and 61473024. ∗ Corresponding author. E-mail addresses: [email protected] (R. Huang), [email protected] (J. Zhang), [email protected] (Z. Lin).

http://dx.doi.org/10.1016/j.sysconle.2016.06.004 0167-6911/© 2016 Elsevier B.V. All rights reserved.

the joint spectral radius (JSR) of the set of subsystems. However, as shown in [11], computing the JSR is extremely hard. The difficulty mainly stems from the fact that there exist infinite switching laws. To circumvent this obstacle, a natural idea is characterizing the most destabilizing switching laws. This enables us to compute the JSR along several specific switching laws and analyze stability under arbitrary switching. Finding the most unstable switching law is an optimal control issue, it naturally reminds us of the variational approach [12]. Recently, considerable research efforts have been directed towards the development of variational approach in the stability analysis of switched systems [13–15]. However, it is noticed the aforementioned schemes are only applicable to deterministic switched systems. Extra efforts are needed to analyze stability under arbitrary switching in a stochastic setting. In real world, not all technological processes can be adequately represented by deterministic systems such as chemical process and biology engineering [16,17]. Besides, to handle ubiquitous uncertainties in realistic system models, one can also describe uncertainties using stochastic models and design a control strategy to meet the design criteria. Researchers are directed to approach switched stochastic systems from various directions [16–21]. Many nice works in stochastic stability of jump linear systems have been reported; see [22,23], and the references therein. Based on dwell time or average dwell time constraints, mean-square stability of switched linear stochastic systems has also been

166

R. Huang et al. / Systems & Control Letters 94 (2016) 165–171

studied in [16,19,24]. In spite of progress, it should be mentioned that these results concerning stability analysis impose restrictions on the switching signal. To the best of our knowledge, there are limited results dealing with stability analysis of switched linear stochastic systems under arbitrary switching. The difficulty lies in determining the JSR of a set of bounded linear operators. This paper attempts to characterize the ‘‘most unstable’’ switching law for discrete-time switched systems (DSS) governed by a set of bounded linear operators. To apply the variational approach pioneered by E.S. Pyatnitskii, we embed the DSS in a generalized class of discrete-time bilinear systems (DBS). A second-order necessary optimality condition is derived for the DBS associated with a Mayer-type optimal problem. Optimal equivalence between the DBS and the DSS is then analyzed, which indicates that any optimal control can be equivalently expressed as a switching law. This specific switching law is most unstable for the DSS, and thus can be used to compute the JSR. Based on the second-order moment of the state, the proposed approach is applied to analyze global uniform mean-square stability (GUMS) of discrete-time switched linear stochastic systems (DSLSS). The rest of the paper is organized as follows. The problem is formulated in Section 2. Section 3 presents the main results, followed by the application to GUMS analysis of the DSLSS in Section 4. Numerical simulations are presented in Section 5 and conclusions are made in the final section. Notations: Sn is the Hilbert space composed of n × n symmetric matrices with the inner product ⟨·, ·⟩ defined by ⟨Y1 , Y2 ⟩ = tr(Y1 Y2 ), ∀Y1 , Y2 ∈ Sn . The set of natural numbers and ndimensional real vectors are respectively denoted by N and Rn . Denote by L (Sn ) the set of all bounded linear operators from Sn to Sn . For L ∈ L (Sn ), let ∥L∥ = max∥Y ∥=1,Y ∈Sn ∥L(Y )∥. Throughout the paper, denote by ∥ · ∥ the norm of a vector in Rn , a matrix in Sn or a operator in L (Sn ) induced by ⟨·, ·⟩ without ambiguity. E stands for the mathematical expectation with respect to the given probability measure P , and tr(·) denotes the trace of a square matrix. eim−1 denotes the ith column of the (m−1)×(m−1) identity matrix. The symbol vec(·) represents the linear operator stacking the entries of a matrix columnwise, and while vec−1 (·) denotes the linear inverse operator. σmax (·) denotes the maximal eigenvalue of a positive semi-definite matrix. For a vector or a matrix Y , we write Y ≺ 0(≽ 0) if all elements of Y are (no) less than 0.

1

max{∥Lik · · · Li0 ∥ k+1 : ij ∈ M }. A natural idea is to characterize γ ∗ (k), and analyze the corresponding trajectory Z∗ (k). It is clear that if ∥Z∗ (k)∥2 converges to origin along with time evolution, so does the norm of any other solution. Via this transformation, we only need to compute the operator norm along with γ ∗ (k) for determining the JSR. To apply the variational approach, we embed DSS (1) in the following DBS described by L

 Z(k + 1) =



m−1

L0 +



ui (k)Bi

(Z(k)),

u(k) ∈ U

(2)

i=1

where Bi = Li − L0 , the control set U is given by m−1



U = u(k) ∈ Rm−1 : ui (k) ≥ 0,





ui (k) ≤ 1

(3)

i=1

with u(k) = (u1 (k) . . . um−1 (k))′ ∈ U. When u(k) is bang–bang, m−1 taking values in the set {0, e1m−1 , . . . , em −1 } for any k ∈ N, DBS (2) reduces to DSS (1). Fix an arbitrary final time N ∈ N, and consider the Mayer-type optimal problem max J (N ; u(k), Z0 ) = max ∥Z(N ; u(k), Z0 )∥2 ,

u(k)∈U

(4)

u(k)∈U

where Z(N ; u(k), Z0 ) denotes the solution of (2) corresponding to u(k) at time N. We refer such a control as an optimal control, denoted by u∗ (k). If there always exists an optimal control u∗ (k) that is bang–bang, then γ ∗ (k) for DSS (1) and u∗ (k) for DBS (2) are equivalent with respect to the issue of maximizing ∥Z(N )∥2 . Definition 3. We say DBS (2) is globally asymptotically stable (GAS) if for any u(k) ∈ U, Z0 ∈ Sn , limN →∞ Z(N ; u(k), Z0 ) = 0. Definition 4. For any u(k) ∈ U, a perturbation v (k) is said to be admissible if u(k) + v (k) ∈ U. Lemma 1. Let w = [w1 . . . wm−1 ]′ ∈ Rm−1 and G = (Gij ) ≥ 0 ∈ Sm−1 be given. For any u ∈ U, define f (v ) = v ′ w + v ′ Gv, where v ∈ Rm−1 is any admissible perturbation of u. Suppose that f (v ) ≤ 0 for all admissible perturbations, then the following statements hold: (i) If w ≺ 0, then u = 0; (ii) If wi > 0 and wi > wj for any j ̸= i, then u = eim−1 ; (iii) If w = 0, then G = 0. Proof. Firstly, we prove (i) by contradiction. If u ̸= 0, it implies that there must exist an index j such that uj > 0. Choose a

2. Problem formulation

j

Let L = {Li ∈ L (Sn )}i∈M with M = {0, 1 . . . , m − 1}. Consider the following DSS described by L Z(k + 1) = Lγ (k) (Z(k)),

As shown in [27], DSS (1) is GUAS if and only if the JSR of L satisfies ρ(L) < 1, where ρ(L) = limk→∞ ρˆ k (L) with ρˆ k (L) =

Z(0) = Z0 ,

k∈N

(1)

where Z(k) ∈ Sn is state, and γ (·) : N → M is the switching law. This models a system that can switch among the m linear subsystems Z(k + 1) = Li (Z(k)), i ∈ M with the switching law determining which system is active at each time step. DSS (1) represents various discrete-time switched linear systems in both deterministic and stochastic settings, see [25,26]. Definition 1. DSS (1) is said to be globally uniformly asymptotically stable (GUAS) if for any Z0 ∈ Sn and any switching law γ (k), limk→∞ ∥Z(k)∥2 = 0. Definition 2. For an arbitrarily fixed final time N ∈ N, a switching law maximizing ∥Z(N )∥2 is referred as the ‘‘most unstable’’ switching law and denoted by γ ∗ (k). Problem 1. For an arbitrarily fixed final time N ∈ N, find the ‘‘most unstable’’ switching law γ ∗ (k).

sufficiently small positive scalar ϵ such that v = −ϵ em−1 is an admissible perturbation, we have f (v ) = −ϵwj + ϵ 2 Gjj > 0. This contradicts f (v ) ≤ 0. Thus, we have u = 0. Secondly, assume that wi > 0 and wi > wj for any j ̸= i, and u ̸= eim−1 . Since u ̸= eim−1 , there are two cases: u = δ eim−1 with δ ∈ [0, 1) and there exists an index j such that uj > 0. For the former, choosing an admissible perturbation v = ϵ eim−1 yields j

f (v ) > 0; while for the latter, letting v = ϵ(eim−1 − em−1 ) be an admissible perturbation, we obtain f (v ) = ϵ(wi − wj ) + v ′ Gv > 0. Based on the above analysis, we have u = eim−1 . Thirdly, for any u ∈ U, we can find a group of admissible perturbations {v1 , v2 , . . . , vm−1 } that is a basis of Rm−1 . Thus, the statement (iii) holds by noting f (v ) ≤ 0 for all admissible perturbations.  Definition 5. For a given L ∈ L (Sn ), L is said to be the Hilbert adjoint operator of L if it holds

⟨L(Y1 ), Y2 ⟩ = ⟨Y1 , L(Y2 )⟩,

∀Y1 , Y2 ∈ Sn .

(5)

Lemma 2. For any α ∈ R, L1 , L2 ∈ L (Sn ), we have L1 L2 =

L2 L1 , L1 = L1 , α L1 = α L1 , ∥L1 ∥ = ∥L1 ∥.

R. Huang et al. / Systems & Control Letters 94 (2016) 165–171

167

3. Main results

Thus, we have (12) for any k ≤ N. Using (5), (8), (11) and (12), it enables us to rewrite ηi (k) as

The results obtained in this section are mainly twofold. Firstly, we provide a second-order necessary optimality condition for DBS (2). Secondly, there always exists a bang–bang optimal control u∗ (k), which implies that the problem of finding γ ∗ (k) for (1) is equivalent to that of solving u∗ (k) for DBS (2).

ηi (k) = ⟨P (k + 1), Bi (Z∗ (k))⟩ = ⟨Z∗ (N ), (8∗ (N , k + 1)Bi )(Z∗ (k))⟩

3.1. Optimal control of DBS

8(k + 1, a) =

L0 +



u∗i (k)Bi



where v (s) is an admissible perturbation of the optimal control ˜ (k), we have u∗ (s). With the new control u Z˜ (N ) =

8(k, a)

i =1

Theorem 3. Assume that u (k) is an optimal control for DBS (2), and Z∗ (k) denotes the corresponding trajectory. Let P (k) ∈ Sn be the solution of the following backward matrix difference equation ∗

   m −1   P (k) = L + ∗ ui (k)Bi (P (k + 1)), 0 i =1   P (N ) = Z∗ (N ).

i ∈ {1, . . . , m − 1}.

(7)

(8)

u∗ (l) + v (l), if k = l, u∗ (k), otherwise,



(9)

is optimal and Q1 (l) = · · · = Qm−1 (l) = 0,

(10)

where Qi (l) = (8∗ (N , l + 1)Bi )(Z∗ (l)).

0 ≤ k ≤ N.

m−1



u∗i (k)Bi

(P (k + 1))

i=1

  = 8 (k + 1, k) 8∗ (N , k + 1)(p(N ))   = 8∗ (k + 1, k) 8∗ (N , k + 1) (p(N )) ∗

= 8∗ (N , k)(p(N )).

8 (N , s + 1) L0 +



m−1



u˜ i (s)Bi

(Z∗ (s))

i=1 m−1

= Z∗ (N ) +



vi (s)Qi (s).

(14)

i =1

Define f (v (s)) = ∥Z˜ (N )∥2 − ∥Z∗ (N )∥2 . With the aid of (13) and (14), we obtain f (v (s)) =



vi (s)vℓ (s)⟨Qi (s), Qℓ (s)⟩



vi (s)ηi (s) ≤ 0

(15)

for all admissible perturbations v (s) because u∗ (k) is the optimal control. Let Giℓ = ⟨Qi (s), Qℓ (s)⟩, and it can be seen the matrix G = (Giℓ ) ∈ Sm−1 is semi-positive definite. Therefore f (v (s)) satisfies the conditions in Lemma 1, and conditions (7) and (10) naturally hold. Moreover, if η1 (l) = · · · = ηm−1 (l) = 0, it follows from (10) and (15) that all admissible perturbations at time l have no effect on the value of Z∗ (N ). Hence, any u(k) defined in (9) is optimal.  Remark 1. Theorem 3 tells us u∗ (k) is a bang–bang control under suitable conditions. However, there may also exist u∗ (k) that is not bang–bang. The correspondence [28] has pointed out that for continuous-time switched linear systems, the optimal switched solution does not exist whenever there exists a singular one to the relaxed problem studied in [29]. As a result, we have to ascertain whether singularity occurs in the discrete-time case. 3.2. Optimal equivalence

(12)

This subsection analyzes optimal equivalence between DSS (1) and DBS (2). The main result shows that there always exists an optimal control u∗ (k) that is bang–bang. Namely, there always exists an optimal control u∗ (k) under which the ‘‘most unstable’’ trajectory of DBS (2) is also that of the DSS (1).

When k = N, equality (12) holds by noting 8 (N , N ) = I. Assume that (12) holds for some k + 1 < N, then it follows from (6), the definition of 8∗ (b, a) and Lemma 2 that ∗

L0 +

=

 ∗

(11)

Proof. Firstly, we prove by induction

P (k) =

8∗ (s, 0) (Z0 )

i=1

Moreover, if η1 (l) = · · · = ηm−1 (l) = 0 for some l < N, then any u(k) ∈ U defined as follows



u˜ i (s)Bi



m−1

if ηi (k) < 0 for all i, eim−1 , if ηi (k) > 0 and ηi (k) > ηj (k)  for any j ̸= i,





+2

 0,

P (k) = 8∗ (N , k)(P (N )),



i=1 ℓ=1

where ηi (k) is defined by

u(k) =

8∗ (N , s + 1) L0 +



m−1

m−1 m−1

(6)

Then for any k ∈ {0, 1, . . . , N − 1}, we have

ηi (k) = ⟨P (k + 1), Bi (Z∗ (k))⟩,



i=1

with 8(a, a) = I, where I is the identity operator of L (Sn ). 8∗ (b, a) is the transition operator from the time a to time b corresponding to the optimal control u∗ (k).

u∗ (k) =

u∗ (s) + v (s), if k = s, u∗ (k), otherwise,





m−1

(13)

˜ (k) ∈ U by Fix an arbitrary s, and define a new control u ˜ (k) = u

With an optimal control u∗ (k) ∈ U and 0 ≤ a ≤ b ≤ N, we associate an operator 8∗ (b, a) defined as the solution at time b of the operator difference equation



= ⟨Z∗ (N ), Qi (k)⟩.

Theorem 4. Consider DBS (2) with Z0 ̸= 0. There exists an optimal m−1 control u∗ (k) satisfying u∗ (k) ∈ {0, e1m−1 , . . . , em −1 } for any k ∈ {0, 1, . . . , N − 1}. Proof. Suppose that for some s ∈ {0, 1, . . . , N − 1}, u∗ (s) ̸∈ m−1 {0, e1m−1 , . . . , em −1 }, thus there exists at least one index i ∈ {1, . . . , m − 1} such that u∗i (s) ∈ (0, 1). j Case 1: there is only one such index. Suppose that u∗ (s) = δ em−1 for some j ∈ {1, 2, . . . , m − 1} and δ ∈ (0, 1). Consider the two

168

R. Huang et al. / Systems & Control Letters 94 (2016) 165–171 j

admissible perturbations v (s) = ±ϵ em−1 with ϵ > 0 sufficiently small. Let

˜ (k) = u



u∗ (s) + v (s), if k = s, u∗ (k), otherwise.

It can be shown that H is constant along an optimal solution. Indeed, if u∗ (k) is the optimal control and Z∗ (k) is the optimal trajectory, the co-states 3 must satisfy the backward matrix difference equation (6). Consequently, for any k ∈ N, we have H (Z∗ (k), 3(k + 1), u∗ (k))

Following the line in (14) yields

 =

m−1

Z˜ (N ) = Z∗ (N ) +



vi (s)Qi (s),

(17)

Since u∗ (k) is optimal, f (v (s)) ≤ 0 for any admissible ϵ . It implies that ηj (s) = 0, and (17) reduces to f (v (s)) = ϵ 2 ∥Qj (s)∥2 . As a result, we have Qj (s) = 0. Besides, it follows from (16) that Z˜ (N ) = Z∗ (N ). More precisely,

(18)

which implies u∗j (s) has no effect on both the value of Z∗ (N ) and the cost function J. Hence, we can change u∗j (s) to 0 or 1 and obtain j

a new control that is optimal and satisfies u∗ (s) ∈ {0, em−1 }. Case 2: there are two such indexes. Suppose that there exists j ̸= ℓ such that u∗j (s), u∗ℓ (s) ∈ (0, 1), j, ℓ ∈ {1, 2, . . . , m − 1}. Choose two admissible perturbations v (s) = ±ϵ(em−1 − eℓm−1 ) with ϵ > 0 sufficiently small. Substituting v (s) into (15) yields j

f (v (s)) = ±2ϵ(ηj (s) − ηℓ (s)) + ϵ 2 ∥Qj (s) − Qℓ (s)∥2 ≤ 0 for any admissible perturbation v (s). Hence, we have ηj (s) = ηℓ (s) and Qj (s) = Qℓ (s). Using (18) yields



Z∗ (N ) = u∗j (s)Qj (s) + u∗ℓ (s)Qℓ (s) + 8∗ (N , s + 1)L0 (Z∗ (s)), j em−1

Remark 2. Theorem 4 indicates that any optimal control u∗ (k) can be equivalently expressed as a switching law. Meanwhile, all switching laws belong to the control set U. Therefore, γ ∗ (k) for DSS (1) and u∗ (k) for DBS (2) are equivalent with respect to the issue of maximizing ∥Z(N )∥2 . Remark 3. According to Theorem 4, singularity commented in [28] can be avoided in the discrete-time case. It is also possible to explain the reason from the viewpoint of Hamiltonian. Define the Hamiltonian H : Sn × Sn × Rm−1 → R by





H (Z(k), 3, u(k)) = 3, L0 +



m−1

 i=1

ui (k)Bi

= 3(k), L0 +



m−1



u∗i (k − 1)Bi

 (Z∗ (k − 1))

i =1

= H (Z∗ (k − 1), 3(k), u∗ (k − 1)). m−1 Rewrite the Hamiltonian H as H = i=0 ξi (k)τi (k), where τ0 (k) = m−1 1 − i=1 ui (k), τi (k) = ui (k), ξ0 (k) = ⟨3, L0 (Z(k))⟩, and ξi (k) = ⟨3, Li (Z(k))⟩, i = 1, . . . , m − 1. Via similar discussion in [28], we

 (Z(k)) .

Theorem 5. Consider DSS (1) with Assumption 1. The following statements are equivalent: (1) DBS (2) is GAS; (2) DSS (1) is GUAS; (3) There exists Z0 ̸= 0 ∈ Sn such that lim J (N ; u∗ (N , Z0 ), Z0 ) = 0.

N →∞

Proof. Firstly, because every trajectory of DSS (1) is also a trajectory DBS (2), the implication (1) H⇒ (2) naturally holds. On the other hand, if DSS (1) is GUAS, we have limN →∞ Z(N ; u(k), Z0 ) = 0 for any bang–bang control u(k). According to Theorem 4, any optimal control u∗ (k) can be equivalently expressed as a bang–bang control. Hence, the implication (2) H⇒ (3) holds. To complete the proof, we only need to show (3) H⇒ (1). Define two sets as follows:

O = {Y ∈ Sn : any trajectory of DSS (1) with Z0 = Y converges to origin},

ℓ

which implies if we substitute u (s) with u (s) ± δ( − em−1 ), the control remains optimal since Qj (s) = Qℓ (s). By choosing δ ∈ (0, 1), we can always obtain a new admissible control satisfying either u∗j (s) = 0 or u∗ℓ (s) = 0. Consequently, the new optimal control u∗ (s) has the structure in Case 1 above. General case: there are more than two such indexes. For the general case, we can repeat the procedure as Case 2 until we obtain a vector u∗ (s) that is still part of an optimal control and has the structure in Case 1 above. Based on the above analysis, there always exists an optimal control that is bang–bang.  ∗

(3(k + 1)), Z (k)

Assumption 1. The DSS (1) only admits trivial common invariant subspaces {0} and Sn .

i=1

  = 8∗ (N , s + 1)L0 (Z∗ (s)) + u∗j (s)Qj (s)   = 8∗ (N , s + 1)L0 (Z∗ (s)),

∗

 ∗

can prove that any optimal control can be equivalently expressed as a bang–bang control, which indicates singularity can be avoided in the discrete-time case.

m −1    8∗ (N , s + 1)L0 (Z∗ (s)) + u∗i (s)Qi (s)



ui (k)B i ∗

= ⟨3(k), Z∗ (k)⟩  

where Qi (s) is given by (11). Define f (v (s)) = ∥Z˜ (N )∥2 −∥Z∗ (N )∥2 . j Thanks to (15), (16) and v (s) = ±ϵ em−1 , we obtain f (v (s)) = ±2ϵηj (s) + ϵ 2 ∥Qj (s)∥2 .

 i=1

i=1

Z∗ (N ) =

L0 +

(16)



m−1

O1 = {Y ∈ Sn : any trajectory of DBS (2) with Z0 = Y converges to origin}. It is straightforward to verify that O and O1 are respectively a common invariant subspace of DSS (1) and DBS (2). In combination with Assumption 1 and (3), we have O = Sn . Since the set of bang–bang controls is a subset of U, it holds O1 ⊆ O . In what follows, we will prove O1 = O = Sn by contradiction. Assume that there exists a Y satisfying Y ∈ O and Y ̸∈ O1 . It follows from the definition of O1 that for some u(k) ∈ U, the trajectory of DBS (2) with Z0 = Y does not converge to origin. There must exist an optimal u∗ (k) such that the optimal trajectory of DBS (2) with Z0 = Y does not converge to origin. According to Theorem 4, there exists a switching law such that the trajectory of DSS (1) with Z0 = Y does not converge to origin. This contradicts with O = Sn , and thus O1 = O = Sn . By the definition of GAS, the implication (3) H⇒ (1) holds.  4. Application to GUMS analysis of DSLSS This section primarily applies the results obtained in Section 3 to the GUMS analysis of the following DSLSS described by



x(k + 1) = Aγ (k) x(k) + Cγ (k) x(k)w(k), x(0) = x0 ,

k∈N

(19)

R. Huang et al. / Systems & Control Letters 94 (2016) 165–171

where x(k) ∈ Rn is the state, x0 is the initial condition, γ (·) : N → M is the switching law, {w(k) ∈ R, k ∈ N} is a sequence of real random variables defined on a complete probability space {Ω , F , µ, Fk } and is a wide stationary, second-order process with Ew(k) = 0 and E(w(k)w(j)) = δjk , where δjk is the Kronecker delta. Definition 6. DSLSS (19) is said to be GUMS if for any x0 and any switching law γ (k), limk→∞ E∥x(k)∥2 = 0. Let X (k) = E(x(k)x′ (k)) and γ (k) = i ∈ M . Notice that the vectors Ai x(k) and Ci x(k) are uncorrelated with respect to w(k), which are Fk−1 measurable. In view of E(w(k)) = 0, the time evolution of X (k) obeys the following equation X (k + 1) = LAi ,Ci (X (k)),

X (0) = Ex0 x′0

(20)

Remark 4. By using the techniques in the proof of Theorem 4, we can express an optimal u∗ (k) determined by system (22) as the ‘‘most unstable’’ switching law γ ∗ (k) for system (20), and this γ ∗ (k) is also most unstable for (19) in the sense of second-order moment norm. Nevertheless, it is still uncertain whether the γ ∗ (k) is the most destabilizing law for (19) in the meaning of E∥x(k)∥2 , which merits further study. Fortunately, this phenomenon will not affect GUMS analysis. For a given DSLSS (19), denote by LA,C the set of operators

{LAi ,Ci , i ∈ M}. A potential procedure is provided to determine the GUMS: 1. According to Theorem 3 and the technique in the proof of Theorem 4, determine all possible most unstable switching law γ ∗ (k); and we denote by Γ the set of all γ ∗ (k); 2. Calculate the following index

where LAi ,Ci is defined as follows [25]

ρˆ k (Γ ) =

LAi ,Ci : Y ∈ Sn → Ai YAi + Ci YCi . ′

′

(21)

X (k + 1) =

LA0 ,C0 +



ui (k)BAi ,Ci

(X (k)),

(22)

i =1

where u(k) ∈ U, and BAi ,Ci = LAi ,Ci − LA0 ,C0 . It can be seen that system (20) and system (22) are respectively a special case of DSS (1) and DBS (2). Based on the analysis in the previous section, we are in a position to give the following theorem.

1

∥LAik ,Cik · · · LAi0 ,Ci0 ∥ k+1 .

 ∥LAi0 ,Ci0 ∥2 = σmax (A′i0 ⊗ A′i0 + Ci′0 ⊗ Ci′0 )  × (Ai0 ⊗ Ai0 + Ci0 ⊗ Ci0 ) .



m−1

max

(i0 ,...,ik )∈Γ

In addition, for i0 ∈ M , we have

Thus we have LAi ,Ci : Y ∈ Sn → A′i YAi + Ci′ YCi . The corresponding discrete-time bilinear system with respect to system (20) is given by



169

(24)

Then, we have ρ(LA,C ) = limk→∞ ρˆ k (Γ ). 3. Determine GUMS for DSLSS (19) according to the value of ρ(LA,C ). 5. Simulation example Let α be an adjustable nonzero scalar. Consider DSLSS (19)√with

Theorem 6. Assume that system (20) only admits trivial common invariant subspaces {0} and Sn . The following statements are equivalent:

the parameters given by A0 = α[1 0.5; 0 0], A1 = α[ 3 1011 0;

(1) (2) (3) (4)

X (k + 1) = vec−1 (LA0 ,C0 + u1 (k)BA1 ,C1 )vec(X (k ))

DSLSS (19) is GUMS; System (20) is GUAS; System (22) is GAS; There exists X0 ̸= 0 ∈ Sn such that lim ∥X (N ; u∗ (k), X0 )∥2 = 0.

Proof. According to Theorem 5, we have that claims (2), (3) and (4) are equivalent. To complete the proof, we only need to show (1) ⇐⇒ (2). It should be noted that the following relationship holds:

√

n∥X (k)∥.

(23)

n 

2

Ex2i (k)

i =1

≤n

n  

2

Ex2i (k)

.

i=1

By the definition, we have

; 0 0], C1 = α[0.1 0; 0.1 0]. Then system (22) 

(25)

η1 (1) = ⟨P (2), BA1 ,C1 (X ∗ (1))⟩

= α 2 (2P2 (2) + P3 (2))(X1∗ (0) + X2∗ (0) + X3∗ (0)) ≥ 0,

(26)

where for any k ∈ N, P (k) and X ∗ (k) are written by

  (Ex2i (k))2 + 2 (Exi (k)xj (k))2 , i=1

3 2

where u1 (k) ∈ [0, 1], LA0 ,C0 = A0 ⊗ A0 + C0 ⊗ C0 , and BA1 ,C1 = LA1 ,C1 − LA0 ,C0 with LA1 ,C1 = A1 ⊗ A1 + C1 ⊗ C1 . It can be seen that for any u1 (k) ∈ [0, 1], we have LA0 ,C0 + u1 (k)BA1 ,C1 ≽ 0. Thus, for any X0 ≽ 0, it yields X ∗ (k) ≽ 0 for k ≥ 1. Similarly, we have P (k) ≽ 0 for any k by noting P (N ) = X ∗ (N ) in (6). Let X ∗ (0) = X0 . With respect to different initial conditions, there are two cases: η1 (0) < 0 and η1 (0) ≥ 0. Case I: η1 (0) < 0. According to Theorem 3, we have u∗1 (0) = 0. Thus, using (25) yields

n

∥X (k)∥2 =

√

= ⟨BA1 ,C1 (P (2)), LA0 ,C0 (X ∗ (0))⟩ = vec(P (2))′ BA1 ,C1 LA0 ,C0 vec(X ∗ (0))

Firstly, it follows from Cauchy inequality

(E∥x(k)∥2 )2 =

0], C0 = α[0 reduces to



N →∞

∥X (k)∥ ≤ E∥x(k)∥2 ≤

√

3 11 10

1≤i
√

n∥X (k)∥. On the other hand, which implies that E∥x(k)∥2 ≤ by using Schwarz inequality (Exi (k)xj (k))2 ≤ Ex2i (k)Ex2j (k), we

have ∥X (k)∥2 ≤ (E∥x(k)∥2 )2 . Based on the above discussion, the relationship (23) holds. Therefore, the GUMS of DSLSS (19) is equivalent to the GUAS of system (20), which implies (1) ⇐⇒ (2) holds. 

P (k) =



P1 (k) P2 (k)

P2 (k) , P3 (k)



X ∗ (k) =

X1∗ (k) X2∗ (k)



X2∗ (k) . X3∗ (k)



If η1 (1) > 0, we have u∗1 (1) = 1 by Theorem 3. If η1 (1) = 0, we know any u1 (1) ∈ [0, 1] is optimal at time 1 according to Theorem 3. Thus, it enables us to take u∗1 (1) = 1. In combination with (25) and u∗1 (1) = 1, we have by using the same procedure in (26)

η1 (2) = α 2 (2P2 (3) + P3 (3) − 2P1 (3))X1∗ (1).

(27)

170

R. Huang et al. / Systems & Control Letters 94 (2016) 165–171

We estimate η1 (2). It follows from (6) that P (3) = (LA0 ,C0 + u∗1 (3)BA1 ,C1 )(P (4)), which implies

 P1 (3) = α 2 (P1 (4) + u∗1 (3)(2P2 (4) + P3 (4))), P (3) = 0.5α 2 (P1 (4) − u∗1 (3)P1 (4)),  2 P3 (3) = α 2 (P1 (4) − u∗1 (3)P1 (4)).

(28)

In combination with (27) and (28), it enables us to write

η1 (2) = −2α 4 u∗1 (3)(P1 (4) + 2P2 (4) + P3 (4))X1∗ (1) ≤ 0. When η1 (2) < 0, we have u∗1 (2) = 0 by Theorem 3. When η1 (2) = 0, any u1 (2) ∈ [0, 1] is optimal at time 2 according to Theorem 3. Therefore, it enables us to take u∗1 (2) = 0. Repeating this procedure yields an optimal control given by

{u∗1 (0), u∗1 (1), u∗1 (2), . . .} = {0, 1, 0, . . .} =: γ1∗ (k). Case II: η1 (0) ≥ 0. In a similar way, it enables us to conclude that the following sequence

Fig. 1. Response of E∥x(k)∥2 under γ2∗ (k).

{u∗1 (0), u∗1 (1), u∗1 (2), . . .} = {1, 0, 1, . . .} =: γ2∗ (k) is optimal along with time-evolution. Denote by Γ the set of the most unstable switching laws, i.e., Γ = {γ1∗ (k), γ2∗ (k)}. Write L1 (k) = LAi ,Ci · · · LA0 ,C0 LA1 ,C1 LA0 ,C0 , k k





k+1



L2 (k) = LAi ,Ci · · · LA1 ,C1 LA0 ,C0 LA1 ,C1 . k k





k+1



Since Γ is known, we may calculate ρ(LA,C ). The structure of γ ∗ (k) implies that for any k ≥ 0, it holds

   1 ρˆ k (LA,C ) = max σmax Li′ (k)Li (k) 2(k+1) . i∈{1,2}

√

As k → +∞, it gives ρ(LA,C ) = 3α 2 . Therefore, the considered √ DSLSS is GUMS if and only if α 2 < 3/3.

Fig. 2. Response of E∥x(k)∥2 with α = 0.7548.

Remark 5. By symmetry and some calculation, this DSLSS admits only one invariant subspace A2 besides trivial common invariant subspaces {0} and S2 , where A2 is defined by A2 =: {β Y |Y = [0 1; 1 −1], β ∈ R}. Fortunately, X (k) is the second-order moment of the state, thus we have X (k) ≥ 0. This implies that under any switching law, the trajectory of X (k) will never enter the subspace A2 along with time evolution. Therefore, the above considered DSLSS satisfies Assumption 1. To verify the efficiency of our approach, we present the simulation results corresponding to different values of α . Assume the initial state is deterministic, and x0 = [1 2]′ . In this situation, we know η1 (0) ≥ 0 by noting X0 = [1 2; 2 4]. According to the analysis in Case II, γ2∗ (k) is the most unstable switching law. Letting α = 0.7598 ± 0.005, the response of E∥x(k)∥2 under γ2∗ (k) is shown in Fig. 1. It can be observed that the state x(k) is convergent for α = 0.7548, while the state is divergent for α = 0.7648. Both α being very close indicates the DSLSS is GUMS if and only √ if α 2 < 3/3. Besides, we also select several periodic switching laws to verify that γ2∗ (k) is the most unstable switching law for the specific DSLSS. Concretely, γ1 (k) = γ1∗ (k), γ3 (k) = {0, 0, 1, 1, 0, 0, . . . , }, γ4 (k) = {1, 1, 0, 0, 1, 1, . . . , }, γ5 (k) = {1, 0, 1, 1, 0, 1, . . . , }, γ6 (k) = {0, 1, 0, 0, 1, 0, . . . , }. With α given by 0.7548, 0.7648, and 0.7698, the responses of E∥x(k)∥2 are respectively shown in Figs. 2–4. It is clearly seen that at any time k, the value of E∥x(k)∥2 under γ2∗ (k) is the largest for all α . This implies γ2∗ (k) is the most destabilizing switching law for the DSLSS.

Fig. 3. Response of E∥x(k)∥2 with α = 0.7648.

6. Conclusions The problem of characterizing the most destabilizing switching law for a class of DSSs has been studied in this paper by using optimal control theory. Based on the variational approach, a second-order necessary optimality condition has been derived for a generalized class of DBSs associated with a Mayer-type

R. Huang et al. / Systems & Control Letters 94 (2016) 165–171

Fig. 4. Response of E∥x(k)∥2 with α = 0.7698.

optimal problem. We have shown that any optimal control can be equivalently expressed as a switching law. This enables us to compute the JSR of the DSS along the specific switching law. The obtained results have been applied to GUMS analysis for a class of DSLSSs, and simulation results have verified the usefulness of the propose approach. References [1] D. Liberzon, Switching in Systems and Control, Birkhauser, Boston, Massachusetts, 2003. [2] Z. Sun, Switched Linear Systems: Control and Design, Springer-Verlag, New York, 2006. [3] J. Li, J. Zhao, Passivity and feedback passification of switched discrete-time linear systems, Systems Control Lett. 62 (11) (2013) 1073–1081. [4] D. Cheng, Controllability of switched bilinear systems, IEEE Trans. Automat. Control 50 (4) (2005) 511–515. [5] Z. Ji, G. Feng, X. Guo, A constructive approach to reachability realization of discrete-time switched linear systems, Systems Control Lett. 56 (11) (2007) 669–677. [6] Z. Sun, S. Ge, T. Lee, Controllability and reachability criteria for switched linear systems, Automatica 38 (5) (2002) 775–786. [7] B. Niu, J. Zhao, Barrier lyapunov functions for the output tracking control of constrained nonlinear switched systems, Systems Control Lett. 62 (10) (2013) 963–971.

171

[8] N. Otsuka, Disturbance decoupling via measurement feedback for switched linear systems, Systems Control Lett. 82 (2015) 99–107. [9] H. Lin, P. Antsaklis, Stability and stabilizability of switched linear systems: A survey of recent results, IEEE Trans. Automat. Control 54 (2) (2009) 308–322. [10] T. Doan, A. Kalauch, M. Klose, S. Siegmund, Stability of positive linear switched systems on ordered banach spaces, Systems Control Lett. 75 (2015) 14–19. [11] R. Jungers, Lecture Notes in Control and Information Sciences: Vol. 385. The Joint Spectral Radius: Theory and Applications, Springer, 2009. [12] E.S. Pyatnitskii, Absolute stability of nonlinear pulse systems with nonstationary nonlinearity, Autom. Remote Control 8 (1970) 50–58. [13] M. Margaliot, Stability analysis of switched systems using variational principles: An introduction, Automatica 42 (12) (2006) 2059–2077. [14] T.M.M. Margaliot, A second-order maximum principle for discrete-time bilinear control systems with applications to discrete-time linear switched systems, Automatica 47 (7) (2011) 1489–1495. [15] L. Fainshil, M.M. argaliot, A maximum principle for the stability analysis of positive binlinear systems with applications to positive linear switched systems, SIAM J. Control Optim. 50 (4) (2012) 2193–2215. [16] P. Bolzern, P. Colaneri, G. Nicolao, Markovian jump linear systems with switching transition rates: mean square stability with dwell-time, Automatica 46 (6) (2010) 1081–1088. [17] W. Zhang, J. Hu, J. Lian, Quadratic optimal control of switched linear stochastic systems, Systems Control Lett. 59 (11) (2010) 73–744. [18] Z. Lin, Y. Lin, W. Zhang, A unified design for state and output feedback h∞ control of nonlinear stochastic Markovian jump systems with state and disturbance-dependent noise, Automatica 45 (12) (2009) 2955–2962. [19] R. Huang, Y. Lin, S. Ge, Z. Lin, H∞ stabilization of switched linear stochastic systems under dwell time constraints, Internat. J. Control 85 (9) (2012) 1209–1217. [20] J.W. Lee, P. Khargonekar, Optimal output regulation for discrete-time switched and Markovian jump linear systems, SIAM J. Control Optim. 47 (1) (2008) 40–72. [21] H. Zhang, Y.Q. Xia, Instability of stochastic switched systems, Systems Control Lett. 75 (2015) 101–107. [22] X. Feng, K.A. Loparo, Y. Ji, H.J. Chizeck, Stochastic stability properties of jump linear systems, IEEE Trans. Automat. Control 37 (1) (1992) 38–53. [23] D. Marcelo, M.D. Fragoso, O.L.V. Costa, A unified approach for stochastic and mean square stability of continuous-time linear systems with Markovian jumping parameters and additive disturbances, SIAM J. Control Optim. 44 (4) (2005) 1165–1191. [24] W. Feng, J.F. Zhang, Stability analysis and stabilization control of multi-variable switched stochastic systems, Automatica 42 (1) (2006) 169–176. [25] T. Damm, Lectures Notes in Control and Information Sciences: Vol. 297. Rational Matrix Equations in Stochastic Control, Springer, 2002. [26] O.L.V. Costa, D.F. Marcelo, Discrete-time lq-optimal control problems for infinite Markov jump parameter systems, IEEE Trans. Automat. Control 40 (12) (1995) 2076–2088. [27] M.A. Berger, Y. Wang, Bounded semigroups of matrices, Linear Algebra Appl. 166 (1) (1992) 21–27. [28] P. Riedinger, Comments on ‘‘optimally switched linear systems’’, Automatica 45 (6) (2009) 1588–1590. [29] T. Das, R. Mukherjee, Optimally switched linear systems, Automatica 44 (5) (2008) 1437–1441.