Optimal control for infinite dimensional stochastic differential equations with infinite Markov jumps and multiplicative noise

J. Math. Anal. Appl. 417 (2014) 694–718

Contents lists available at ScienceDirect

Journal of Mathematical Analysis and Applications www.elsevier.com/locate/jmaa

Optimal control for infinite dimensional stochastic differential equations with infinite Markov jumps and multiplicative noise Viorica Mariela Ungureanu Department of Mathematics, “Constantin Brancusi” University, Tg. Jiu, Bulevardul Republicii, nr. 1, jud. Gorj, Romania

a r t i c l e

i n f o

Article history: Received 20 August 2013 Available online 25 March 2014 Submitted by Goong Chen Keywords: Stochastic differential equations Linear quadratic control Generalized Riccati differential equations Detectability Ito’s formula

a b s t r a c t In this paper we solve an infinite-horizon linear quadratic control problem for a class of differential equations with countably infinite Markov jumps and multiplicative noise. The global solvability of the associated differential Riccati-type equations is studied under detectability hypotheses. A nonstochastic, operatorial approach is used. Some properties of the linear stochastic systems, such as stability, stabilizability and detectability, are also discussed on the basis of a new solution representation result. A generalized Ito’s formula which applies to infinite dimensional stochastic differential equations with countably infinite Markov jumps is also provided. © 2014 Elsevier Inc. All rights reserved.

1. Introduction Stochastic differential equations (SDEs) with Markovian switching can model many physical systems which may experience abrupt changes in their dynamics. Among them we mention the manufacturing systems, the power systems, the telecommunication systems etc. All these systems suffer frequent unpredictable structural changes caused by failures or repairs, connections or disconnections of the subsystems [2]. The new applications in modern queuing network theory or in the field of safety-critical and high integrity systems [3,8] led to a revival of the study of SDEs with Markovian jumps (MJs), especially in the case where the state space of the Markov process is countably infinite. For a sample of works dealing with stability, optimal control and H∞ stabilization problems for SDEs with MJs see [19,2,5,6,4,11,8,9] and the references therein. First results on optimal control problems for linear SDEs with MJs and countably infinite state space for the Markov process (SDEs with infinite MJs) were obtained recently in [8–10]. Unlike these works, we consider in this paper an optimal control problem for time-varying linear SDEs with infinite MJs and multiplicative noise (MN) in infinite dimensions. The control objective is to find, E-mail address: [email protected]. http://dx.doi.org/10.1016/j.jmaa.2014.03.052 0022-247X/© 2014 Elsevier Inc. All rights reserved.

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

695

in a class of admissible control laws, the one which minimizes an infinite-horizon linear quadratic cost criterion under stabilizability and detectability conditions. As usually, the design of the optimal control is related to the global solvability of an associated Riccati-type differential equation (RDE) defined on a certain infinite-dimensional ordered Banach space. To avoid the complicated form of the coefficients of this particular RDE, we choose to study the solution properties for a class of generalized RDEs (GRDEs) which includes as special cases most of the known Riccati equations of control. In the general case the proofs are rather the same as in the case of RDEs, the notation is more convenient and the obtained results can be applied to other control problems. Such GRDEs were studied in [26] by using linear matrix inequality (LMI) techniques (see also [5] for the finite dimensional case). Assuming stabilizability hypotheses, [26] provides necessary and sufficient conditions for the existence of certain global solutions such as maximal, minimal and stabilizing solutions. Unlike [26], in this paper we investigate the existence of bounded and stabilizing solutions for GRDEs, under detectability conditions (see Theorem 9). The proofs are nonstochastic and based on operator theory. A key role in this operatorial approach is played by the asymptotic behavior of some positive evolution operators with Lyapunov type generators, defined on ordered Banach spaces. The elements of these Banach spaces are infinite sequences of either linear and bounded operators, trace class operators or Hilbert–Schmidt operators. This situation is characteristic to the infinite-dimensional case and increases the difficulty of the proofs (see [8–10] and [5] for a comparison with the finite dimensional case). In order to apply the results on GRDEs to the stochastic optimal control problem, we need to establish the operatorial equivalent for the stochastic notions of stability, stabilizability and detectability. To this end, we give a version of Ito’s formula which applies to infinite-dimensional stochastic processes with infinite MJs and two representation formulas for the solutions of linear SDEs with infinite MJs and MN. These results extend the ones obtained in [4,6] for Markov processes with finite state space and finite dimensional SDEs. The paper is organized as it follows. Sections 2 and 3 present the notation, some preliminary results and the statement of the optimal control problem. In Section 4 we introduce a notion of detectability for pairs of operator valuated functions which extends the one in [5] and the stochastic detectability notion. In the discrete-time framework a similar notion was considered in [27]. Then we show that a classical result from the theory of Riccati equations remains true for a more general class of nonlinear differential equations (so called GRDEs) defined on ordered Banach spaces. This is the main result of this section and proves that under detectability conditions, any global, bounded and nonnegative solution of GRDEs is stabilizing. In Section 5 we give a generalized Ito’s formula and a representation result for the solutions of linear SDEs with infinite MJs and MN. On the basis of these results we then obtain deterministic characterizations of the stochastic stability, stabilizability and detectability properties that we need in the rest of the paper. In Section 6 we solve an optimal control problem for linear SDEs with infinite MJs and MN, which consists in minimizing an infinite horizon quadratic cost functional over a class of admissible controls. In this paper the optimal control is obtained with the stabilizing solution of RDEs, under stabilizability and detectability hypotheses. The results of Sections 4 and 5 as well as the ones of [26] may be applied for solving some other optimal control problems. We mention here the infinite-dimensional versions of the optimal control problems formulated in Chapter 5 of [6]. 2. Notations Through this paper H, U , V are real separable Hilbert spaces and E is a real Banach space. 2.1. Linear and bounded operators on Hilbert spaces By L(H, U ) we denote the real Banach space of linear and bounded operators from H into U . If H = U we use the short notation L(H) instead of L(H, H). A similar notation will be used if the Hilbert spaces H

696

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

and U are replaced by real Banach spaces. If we do not specify otherwise,  · , ·,· and ∗ denote the norm of elements, the inner product and the adjoint operator, respectively. An element A ∈ L(H) is nonnegative (we write A  0), if A is self adjoint and Ax, x  0 for all x ∈ H. For the Banach subspace of L(H) formed by all self-adjoint operators we use the notation S(H). As in [27], we denote by S1 (H) the space of nuclear (trace class) operators from S(H); similarly, S2 (H) is the space of all Hilbert–Schmidt operators from S(H). Let Tr denote the trace operator √ on S(H). It is known that S1 (H) is a Banach space when endowed with the nuclear norm T 1 = Tr[ T ∗ T ] and S2 (H) is a Hilbert space with the inner product   R, T 2 = Tr T ∗ R .

(1)

The norm on S2 (H) obtained from (1) and defined by   1/2 T 2 = Tr T ∗ T

(2)

is called the Hilbert–Schmidt norm. We recall that ·  ·2  ·1 and, therefore, S1 (H) ⊂ S2 (H) ⊂ L(H). For example, if x ∈ H, then the element of L(H) defined by x ⊗ x(u) = u, xx, u ∈ H has a finite nuclear norm x ⊗ x1 = Tr[x ⊗ x] = x2

(3)

and belongs to both S1 (H) and S2 (H). For further information concerning nuclear and Hilbert–Schmidt operators, the reader is referred to [18,22,27]. 2.2. Banach spaces lEZ Let Z be an interval of integers, which may be finite or infinite. We denote by lEZ the real Banach space of all sequences g = {g[i] }i∈Z ⊂ E with the property gZ = supi∈Z g[i]  < ∞ [27]. So, if E = L(U, H) Z Z Z (or lS(H) ) for the space lEZ . An element X ∈ lS(H) is said to (or E = S(H)) we use the notation lL(U,H) be nonnegative and we write X  0 iff (if and only if) X[i]  0 for all i ∈ Z. The cone of all nonnegative Z Z is denoted by KH . Obviously lS(H) is an ordered Banach space with the order  induced elements of lS(H) Z . The following by the cone KH . If IH is the identity operator on L(H), then Φ[H] := {Φ[i] = IH }i∈Z ∈ lS(H) operations are defined as in [27]: [H]

  Z Z AB = (AB)[i] = A[i] B[i] i∈Z , A ∈ lL(U,H) , B ∈ lL(H,U )   [∗] ∗ Z A = (A[i] ) i∈Z , A ∈ lL(U,H) . −1 Z Z Z [−1] Then AB ∈ lL(H) and A[∗] ∈ lL(H,U := {(A[i] )−1 }i∈Z . ) . If A ∈ lL(H) and A[i] exists for all i ∈ Z then A

Z Note that A[−1] is not necessarily from lL(H) and it is not the inverse operator of A. Also A[∗] is not the adjoint of A; it is a sequence of adjoint operators. A sequence {X(t)}t∈ ⊂ KH , where is a family of indexes, is said to be uniformly positive and we write Z Z , lS(U X(t)  0, t ∈ iff there is γ > 0 such that X(t)  γΦ[H] for all t ∈ . An operator Γ ∈ L(lS(H) ) ) is called positive if Γ (KH ) ⊂ KU . Z Z , lS(U Also, we say that Ψ ∈ L(lS(H) ) ) is m-strongly continuous if for any increasing and bounded sequence Z {D(m)}m∈N ⊂ lS(H) we have

     lim Ψ D(m) [i] x = Ψ (D) [i] x,

m→∞

where D[i] (x) = limm→∞ D(m)[i] (x), for all i ∈ Z and x ∈ H.

x ∈ H, i ∈ Z,

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

697

 Z 2 Following [27], we introduce the linear subspaces HH := {T ∈ lS(H) , |T |2 = i∈Z T[i] 2 < ∞} and Z Z , |T |1 = i∈Z T[i] 1 < ∞} of lS(H) , formed by infinite sequences of Hilbert–Schmidt NH := {T ∈ lS(H) and nuclear operators, respectively. It is known that HH is a Hilbert space with the inner product D, T Z = ∗ i∈Z Tr[T[i] D[i] ], D, T ∈ HH and that (NH , |·|1 ) is a Banach space [27]. As shown in Lemma 20 from [27], NH ⊂ HH . For any i ∈ Z, x ∈ H, we denote by i,x P the element of HH ∩ NH defined by

i,x P[j]

=0

if i = j

and

= x ⊗ x.

i,x P[i]

(4)

The linear and bounded operators Γ1 ∈ L(HH , HU ) and Γ2 ∈ L(NH , NU ) are called positive iff Z ) has HH or NH Γ1 (KH ∩ HH ) ⊂ KU ∩ HU and Γ2 (KH ∩ NH ) ⊂ KU ∩ NU , respectively. If A ∈ L(lS(H) as invariant subspaces and no confusion is possible, the notation does not distinguish between A and its Z ) is such that A|HH ∈ L(HH ), restrictions A|HH , A|NH to HH and NH , respectively. Also, if A ∈ L(lS(H) then we denote by A the adjoint operator of the restriction A|HH on the Hilbert space HH . We point out that A is not the adjoint operator of A|H on the Banach space lZ . S(H)

H

2.3. Evolution operators If T > 0, we denote by C([0, T ], E) the Banach space of all mappings G : [0, T ] → E that are continuous [20]. The subspace of C([0, T ], E) formed by all continuously differentiable mappings G on (0, T ) will be denoted by C 1 ([0, T ], E). (We recall that a mapping G is continuously differentiable iff it is differentiable on (0, T ) and G is continuous on (0, T ).) The above notation can be extended to the case where the interval [0, T ] is replaced by R+ = [0, ∞). Also Cb (R+ , E) denotes the subspace of C(R+ , E) formed by all bounded mappings. If J = [0, T ] or J = R+ , then Cb1 (J, E) is the subspace of C 1 (J, E) formed by all mappings f with the property that f (t) and dfdt(t) are bounded. Let A ∈ Cb (R+ , L(E)). It is known that the differential equation dYdt(t) = A(t)Y (t), 0  s  t  T , c (t, s) Y (s) = x ∈ E has a unique solution Y (·, s; x) ∈ Cb1 ([s, T ], E) [20]. The strong evolution operator UA on E defined by c UA (t, s)(D) := Y (t, s; D),

D ∈ E, 0  s  t  T

is called the causal evolution operator generated by A [16]. The backward differential equation dX(t) + dt 1 A(t)X(t) = 0, 0  t  T , X(T ) = D ∈ E has also a unique solution X(·, T ; D) ∈ Cb ([0, T ], E) [20]. a (s, t)(D) := Following [16], the anticausal evolution operator generated by A is the linear operator UA X(s, t; D), s  t. The following relation between the two evolution operators is known:  a ∗ c UA , ∗ (t, s) = UA (s, t)

s  t.

(5)

As in [4], we say that the mapping A generates an anticausal uniformly exponentially stable (u.e.s. for a (t, t0 )  βα(t0 −t) for all t  t0 . Also, short) evolution on E iff there are α ∈ (0, 1), β  1 such that UA c (t, t0 )  βα(t−t0 ) A generates a causal u.e.s. evolution on E iff there are α ∈ (0, 1), β  1 such that UA for all t  t0 . 2.4. Random variables Let (Ω, F, P ) be a probability space. The mean (expectation) of an integrable random variable ξ on (Ω, F, P ) will be denoted by E[ξ]. For G a σ-algebra of subsets of Ω in F, we denote by E[ξ|G ] the conditional expectation (mean) of ξ with respect to G. Also we use the notation E[ξ|ζ ] for the conditional mean of ξ with respect to Gζ , the σ-algebra generated by a random variable ζ on Ω. If A, B ∈ F with

698

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

P (B) > 0, then we write P (A|B ) for the conditional probability of A given B and E[ξ|B ] for the conditional mean of ξ on the event B. 3. Preliminaries and the statement of the problem In this paper η(t), t ∈ R+ is a right continuous, homogeneous Markov process on Ω with the state space Z. We also assume that η(t) has a standard transition probability matrix   P (t) = pij (t) i,j∈Z ,

  pij (t) = P η(t + s) = j|η(s)=i , t > 0, s  0

and the infinitesimal matrix   λij := pij (0) = lim pij (t) − pij (0) /t t0

where pij (0) = δij , i, j ∈ Z and δij is the Kronecker delta function. We also assume that 0  λij , i, j ∈ Z, j = i

λij = −λii < c

(6)

j∈Z, j =i

for all i ∈ Z and some c > 0. A matrix Λ = (λij )i,j∈Z satisfying the above conditions with c replaced by ∞ is called a stable and conservative q-matrix [17]. The following result is proved in Appendix A. Lemma 1. The convergences limt0 pij (t) = δij , limt0 (pij (t) − δij )/t − λij = 0 are uniform with respect to i and j. Since the conditional mean E[ξ|η(t)=i ] is frequently used in the paper, we assume that P {η(0) = i} > 0 for all i ∈ Z. Then, by the law of total probability, P (η(t) = i) > 0 for all t ∈ R+ , i ∈ Z and E[ξ|η(t)=i ] can be computed. Now let r ∈ N∗ = N − {0} be fixed. We consider on Ω a standard r-dimensional Wiener process w(t) = (wk (t))k=1,r , t ∈ R+ such that the σ-algebras σ(w(s), 0  s  t) and Gt = σ(η(s), 0  s  t) are independent. Following [6], we denote by Ft , t  0 the smallest σ-algebra which contains all sets M ∈ F with P (M ) = 0 and with respect to which all random vectors w(s), s  t are measurable. Then Ht := Ft ∨ Gt , t  0 is an admissible filtration for the Wiener process {w(t)}t0 , i.e. (a) the Wiener process is adapted to the filtration, and (b) for every t  0, the process (w(t + s) − w(t))s>0 is independent of the σ-algebra Ht . Obviously Ht

contains all P -negligible sets from F. Let us consider the normal filtration Ht+ = h0 Ht+h , t  0 and let χM denote the indicator function of a set M . Lemma 2. (a) The Wiener process {w(t)}t0 is Ht+ -adapted and the process (w(t+s)−w(t))s>0 is independent of Ht+ . (b) E[χη(t+h)=i |Ht+ ] = pη(t)i (h) for all h  0. For the proof see Appendix A. Lemma 2 (a) shows that {wk (t)}t0 , k = 1, . . . , r are Wiener processes with respect to the filtration {Ht+ }t0 . So, the Ito integrals we use in the sequel will be defined with respect to these Wiener processes

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

699

and the normal filtration {Ht+ }t0 . Now let us denote by L2η,w ([a, b], H) the space of all H-valued, processes X(t), t ∈ [a, b], that are nonanticipative with respect to Ht+ (i.e. they are Ht+ -adapted and measurable) b and have the property E( a X(t)2 dt) < ∞; it is known [13] that any X ∈ L2η,w ([a, b], H) is stochastically integrable on [a, b]. We also recall here that all (right) continuous stochastic processes are progressively measurable and hence nonanticipative (see [13] and the references therein). Assume the following hypothesis. Z Z Z (H1) Ak ∈ Cb (R+ , lL(H) ), Bk ∈ Cb (R+ , lL(U,H) ), k = 0, 1, . . . , r, C0 ∈ Cb (R+ , lL(H,V ) ), D0 ∈ Cb (R+ , Z lL(U,V ) ), D0 (t)D0 (t)  0, t ∈ R+ . [∗]

We consider the linear stochastic differential equation r     Ak (t)[η(t)] x(t) + Bk (t)[η(t)] u(t) dwk (t), dx(t) = A0 (t)[η(t)] x(t) + B0 (t)[η(t)] u(t) dt +

(7)

k=1

x(t0 ) = ξ,

0  t0  t,

(8)

where u = {u(t)}tt0 ∈ L2η,w ([t0 , ∞], U ) and ξ is an H-valued and Ht0 + -measurable random variable such that E(ξ2 ) < ∞. A stochastic process x ∈ L2η,w ([t0 , T ], H) is a solution of (7), (8) if it satisfies the stochastic integral equation t x(t) = ξ +

r   A0 (s)[η(s)] x(s) + B0 (s)[η(s)] u(s) ds +

t

  Ak (s)[η(s)] x(s) + Bk (s)[η(s)] u(s) dwk (s). (9)

k=1 t 0

t0

By using successive approximations, similar to those for the finite dimensional case (see Theorem 36 from [6] or Theorem 1.1 from [12, Chapter 5]), it follows that (7), (8) has a unique continuous solution x(t) = xu (t, t0 , ξ), t  t0 which belongs to L2η,w ([t0 , ∞], H) [15]. Here, by the continuity of x(t) we mean that the trajectories t → x(t, ω) of x(t) are continuous for P -almost all ω. Moreover, we have 2   sup E x(t) η(t

t0 tT

 0 )=i

    K 1 + E ξ2 η(t

0 )=i

 ,

(10)

where K is a constant depending on supt∈[t0 ,T ] Ak (t) and supt∈[t0 ,T ] Bk (t), k = 0, . . . , r. The optimal control problem O. Assume that there is c1 > 0 such that

λij  c1 ,

j ∈ Z.

(11)

i∈Z, i =j

For t0 ∈ R+ and ξ = x ∈ H fixed, we consider the stochastic differential equation with control (7), (8) and the output y(t) = C0 (t)[η(t)] x(t) + D0 (t)[η(t)] u(t).

(12)

The problem (O) consists in minimizing the performance ∞    y(t)2  J(u, t0 , x, i) = E t0

 η(t0 )=i

dt

(13)

700

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

for arbitrary i ∈ Z over the class of admissible controls U(t0 , x0 ) formed by all controls u ∈ L2η,w ([t0 , ∞], U ) with the property that J(u, t0 , x, i) < ∞, i ∈ Z. This optimal control problem is an extension of the second linear quadratic optimization problem solved in Section 5 of [6] for finite dimensional SDEs with finite MJs. Our purpose is to give a solution to problem (O) under stochastic detectability and stochastic stabilizability conditions imposed to (7), (12). The RDE associated with this problem is defined on the ordered Z Banach space lS(H) and is more complicated than the one in the case of SDEs without MJs. For comparison, the reader may consult [24] where a similar optimal control problem is considered for infinite dimensional SDEs without MJs and a different class of admissible controls. As we know from the finite dimensional case (see [6]), the RDE associated with the optimal control problem (O) should be written in the form (18) with the coefficients defined in Section 6. Therefore, in the next section we consider (18) with more general coefficients and we study the solutions’ properties under stabilizability and detectability conditions. The relationship between this equation (so called GRDE) and the problem (O) will be established in Section 6 by using deterministic equivalents for the notions of uniform exponential stability, stochastic stabilizability and stochastic detectability as well as a generalized version of the classical Ito’s formula. We prove here that the optimal cost (13) and the optimal control can be computed with the stabilizing solution of (18). 4. Global solvability of GRDEs under detectability conditions In this section we consider a class of GRDEs sufficiently large to include the RDEs associated with the optimization problem (O) and many other known Riccati equations of control (see [26]). Assuming detectability conditions we shall prove that any bounded and nonnegative solution of a GRDE is stabilizing. Through this section we assume the following hypothesis. Z Z Z Z ), B, D ∈ Cb (R+ , lL(U,H) ), M ∈ Cb (R+ , lS(H) ), R ∈ Cb (R+ , lS(U (H2) (i) A ∈ Cb (R+ , lL(H) ) ), R(t)  0, Z Z Z Z Z t ∈ R+ , Π1 ∈ Cb (R+ , L(lS(H) )), Π12 ∈ Cb (R+ , L(lS(H) , lL(U,H) )), Π2 ∈ Cb (R+ , L(lS(H) , lS(U ) )) and Z −1 [∗] [∗] there is C ∈ Cb (R+ , lL(H,V ) ) such that M (t) − D(t)R (t)D (t) = C (t)C(t) for all t ∈ R+ ; (ii) for all t ∈ R+ , Π(t) is a positive and m-strongly continuous operator defined by

 Π(t)(X) :=

Π1 (t)(X) Π12 (t)(X) [∗] Π12 (t)(X) Π2 (t)(X)

 ,

Z X ∈ lS(H) .

From (H2) (i) and the properties of Schur complements for families of operators given by Lemma 3 of [26], we have   M (t) D(t)  0, t ∈ R+ . Q(t) = (14) D(t)[∗] R(t) Z Z Z Also we note that Π ∈ Cb (R+ , L(lS(H) , lS(H×U ) )) and Q ∈ Cb (R+ , lS(H×U ) ). For convenience, in the Z Z sequel we use the notation Πa (t, X) for Πa (t)(X). Also if {W (t)}t∈R+ ⊂ lL(H,U ) and X ∈ lS(H) we set

 Φ[H] , ΠW (t, X) = Φ W (t) Π(t, X) W (t)  [H]    Φ QW (t) = Φ[H] W (t)[∗] Q(t) , t ∈ R+ , W (t)  [∗] LB,W (t)(X) = (A + BW )(t) X + X(A + BW )(t) + ΠW (t, X) 

[H]

[∗]





(15) (16) (17)

Z In view of (H2) and (14), ΠW (t) is a positive operator on lS(H) and QW (t)  0, for all t ∈ R+ . Let us consider a class of GRDEs of the form

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

       d X(t)[i] + A(t)[∗] X(t) + X(t)A(t) + Π1 t, X(t) + M (t) − X(t)B(t) + Π12 t, X(t) + D(t) dt   [−1]    [∗]  · R(t) + Π2 t, X(t) = 0, i ∈ Z, t ∈ R+ . X(t)B(t) + Π12 t, X(t) + D(t) [i]

701

(18)

The above equation can be rewritten as   d X(t)[i] + R t, X(t) [i] = 0, dt

i ∈ Z,

(19)

Z where R : Dom R → lS(H) is defined by

R(t, X) = A(t)[∗] X + XA(t) + Π1 (t, X) + M (t)   [−1]  [∗] − XB(t) + Π12 (t, X) + D(t) R(t) + Π2 (t, X) XB(t) + Π12 (t, X) + D(t) ,

(20)

Z and Dom R = {(t, X) ∈ R+ × lS(H) | R(t, i) + Π2 (t, X)(i) is invertible for all i ∈ Z}. Let (T, X0 ) ∈ Dom R. We associate with (19) the final condition

X(T ) = X0 .

(21)

Z By a solution of (19)–(21) we mean a continuously differentiable mapping X(·) = X(T, · ; X0 ) : [0, T ] → lS(H) with the property {(t, X(t)), t ∈ [0, T ]} ⊂ Dom R that verifies (19)–(21). The following result is proved in Appendix A.

Lemma 3. Let X0 ∈ KH and t ∈ R+ . Then (t, X0 ) ∈ Dom R and there is a unique solution X(T, t; X0 ) of (19)–(21) which belongs to Cb1 ([0, T ], KH ). Z ) such that {(t, X(t)), t ∈ R+ } ⊂ Dom R and (19) is fulfilled for all t ∈ R+ A mapping X ∈ C 1 (R+ , lS(H) is called a global solution of (19); if a global solution X is bounded, we say that (19) has a bounded solution. Z A direct computation [5] shows that for any {W (t)}t∈R+ ⊂ lL(H,U ) and (t, X) ∈ Dom R we have

 [∗]    R(t, X) = LB,W (t)(X) + QW (t) − W (t) − F X (t) R(t) + Π2 (t)X W (t) − F X (t) ,

(22)

 [−1]  [∗] F X (t) = − R(t) + Π2 (t, X) XB(t) + Π12 (t, X) + D(t) .

(23)

where

Let QWD (t) and LB,WD (t), t ∈ R+ be defined by (16) and (17), respectively, for WD (t) = −R−1 (t)D[∗] (t) replacing W (t). An appeal to (22), shows that (19) can be equivalently rewritten as      [−1] d D X(t)[i] + LB,WD (t)X(t) + QWD (t) − X(t)B(t) + Π12 t, X(t) R(t) + Π2D t, X(t) dt  [∗]   D t, X(t) · X(t)B(t) + Π12 = 0, [i]

(24)

where  D

Π (t)(X) :=  =

 D Π12 (t)(X) Π2D (t)(X)    −D(t)R−1 (t) Φ[H] 0 Π(t, X) , Φ[V ] −R−1 (t)D[∗] (t) Φ[V ]

Π1D (t)(X) D [Π12 (t)(X)][∗]

Φ[H] 0

QWD (t) = M (t) − D(t)R−1 (t)D[∗] (t)

(25)

702

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

Now let us consider the Lyapunov-type operator L(t)(X) = A(t)[∗] X + XA(t) + Π1 (t, X),

Z X ∈ lS(H) , t ∈ R+

(26)

Z From (H1) and (H2) it follows easily that L ∈ Cb (R+ , L(lS(H) )) and, for all t ∈ R+ , L(t) is an m-strongly continuous operator. As we have shown in [26], L generates a positive anticausal evolution Z Z . It is not difficult to see that if W ∈ Cb (R+ , lL(H,U operator ULa (t, s) in lS(H) ) ) then LB,W has the same properties as L. Z Definition 4. The triple {A, B, Π} is stabilizable if there is F ∈ Cb (R+ , lL(H,U ) ) such that LB,F generates an anticausal u.e.s. evolution; F is called a stabilizing feedback gain. Here LB,F is defined by (17) with W replaced by F . Z ) be a global solution of (19) and let F X (t) be defined by (23) with X Definition 5. Let X ∈ C 1 (R+ , lS(H) replaced by X(t). We say that X is a stabilizing solution of (19) if F X is a stabilizing feedback gain for the triple {A, B, Π}.

Theorem 6. Eq. (19) has at most one stabilizing solution X(·) in Cb1 (R+ , KH ). Proof. Assume that (19) has two stabilizing solutions X1 (·), X2 (·) ∈ Cb1 (R+ , KH ). Let Wk (t) := F Xk (t), k = 1, 2 where F Xk (t) is defined by (23) with X replaced by Xk , k = 1, 2. The boundedness of Xk (·), k = 1, 2 and (H2) (i) ensure that Wk (·), k = 1, 2 is bounded on R+ . In view of (22), we see that (19) can be equivalently rewritten as     [∗]     d X(t)[i] + LB,W1 (t) X(t) + QW1 (t) − F X (t) − W1 (t) R(t) + Π2 t, X(t) F X (t) − W1 (t) [i] = 0. dt Replacing X(t) by X1 (t) and then by X2 (t) and subtracting the obtained relations, we get     d X1 (t) − X2 (t) [i] + LB,W1 (t) X1 (t) − X2 (t) dt  [∗]     + W2 (t) − W1 (t) R(t) + Π2 t, X1 (t) W2 (t) − W1 (t) [i] = 0.

(27)

We recall that both (W2 (t) − W1 (t))[∗] [R(t) + Π2 (t, X1 (t))](W2 (t) − W1 (t)) ∈ KH and X2 (t) − X1 (t) are bounded on R+ and LB,W1 generates a positive anticausal u.e.s. evolution. Theorem 5 d) from [26] Z ensures that X1 (t) − X2 (t), t ∈ R+ is the unique solution of (27) in Cb1 (R+ , lS(H) ) and, moreover, that it is nonnegative, i.e. X1 (t) − X2 (t)  0 for all t ∈ R+ . Interchanging the role of X1 and X2 we can prove that X2 (t) − X1 (t)  0 for all t ∈ R+ . Hence X1 (t) = X2 (t), t ∈ R+ and the conclusion follows. 2 In order to define the detectability notion we introduce the hypothesis Z (H3) (i) For any W (t) ∈ lL(H,U ) and t ∈ R+ , HH is an invariant subspace of ΠW (t) and ΠW (t)|HH ∈ L(HH ). Z  W ∈ Cb (R+ , L(HH )) and Π  W ∈ Cb (R+ , L(NH )). (ii) If W ∈ Cb (R+ , lL(H,U ) ), then Π

From (H2) (ii) and (15) it follows that ΠW (t) is m-strongly continuous. Regarding the last requirement of (H3) (ii), we see that (H2) (ii), (H3) (i) and Lemma 24 ensure that  W (t) ∈ L(NH ) for t ∈ R+ . However, the | · |1 -continuity of the mapping t → Π  W (t), t ∈ R+ cannot be Π deduced from the previous assumptions and must be presumed.

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

703

 By (H3), HH is an invariant subspace of the operator L(t) defined by (26) and L(t) ∈ L(HH ). Let L(t) be the adjoint operator of the restriction of L(t) to HH . It is given by   1 (t, X), L(t)(X) = A(t)X + XA(t)[∗] + Π

X ∈ HH .

(28)

The properties of nuclear and Hilbert–Schmidt operators (see [18] or the Appendix of [27]) (H2) (i) and (H3) imply that L ∈ Cb (R+ , L(HH )) and L ∈ Cb (R+ , L(NH )). The following lemma is proved in Appendix A. Lemma 7. Assume (H3). Then L generates a positive causal evolution operator ULc(t, s) on NH and we have      ULa (s, t) Φ[H] [i] x, x = ULc(t, s)(i,x P )1 ,



(29)

for all i ∈ Z, x ∈ H. Moreover, L generates a causal u.e.s. evolution operator ULc(t, s) on NH iff L generates Z . an anticausal u.e.s. evolution operator ULa (s, t) in lS(H) Now we are ready to introduce the following. Z  C] ∈ Cb (R+ , L(NH ))×Cb (R+ , lZ Definition 8. The pair [L, L(H,V ) ) is detectable if there is F ∈ Cb (R+ , lL(H,V ) )  such that LF (t)(X) = L(t)(X) + F (t)C(t)X + XC [∗] (t)F [∗] (t), X ∈ NH , t ∈ R+ generates a causal u.e.s.

evolution in NH .  C] is detectable iff there is F ∈ Cb (R+ , lZ Lemma 7 shows that [L, L(H,V ) ) such that LF (t)(X) = L(t)(X) + XF (t)C(t) + C [∗] (t)F [∗] (t)X,

(30)

Z Z X ∈ lS(H) , t ∈ R+ generates an anticausal u.e.s. evolution in lS(H) . The following theorem is the main result of this section. It is an infinite dimensional version of Lemma 6.2 from [5].

Theorem 9. Assume that (H2), (H3) hold and let WD (t) = −R−1 (t)D[∗] (t). If [LB,WD ; C] is detectable, then any solution X ∈ Cb1 (R+ , KH ) of the Riccati differential equation (18) is stabilizing. Proof. The proof proceeds in two stages. We first prove the theorem under the assumption D(t) ≡ 0, t ∈ R+ and then we show that the general case reduces to this one.  Π D (t) ≡ Π(t) and M (t) = Case I. Assume D(t) ≡ 0, t ∈ R+ . Obviously, WD (t) = 0, LB,WD (t) ≡ L(t), [∗] C (t)C(t) for all t ∈ R+ . Let X(t) be a global, nonnegative and bounded solution of (19) and let W = {W(t)}t∈R+ , where W(t) := F X (t) and F X (t) is defined by (23). Hypothesis (H2) ensures that {W(t)}t∈R+ Z Z is bounded in lL(H,U ) and, consequently, the sequence {ΠW (t)}t∈R+ is bounded in L(lS(H) ). By Definition 5, we have to prove that {LB,W (t)}t∈R+ generates an anticausal u.e.s. evolution operator ULaB,W (t, s). In view of (22), Eq. (19) can be equivalently rewritten as     d X(t)[i] + LB,W (t) X(t) + C [∗] (t)C(t) + W(t)[∗] R(t)W(t) [i] = 0. dt

(31)

Z Z We observe that W ∈ Cb (R+ , lL(H,U ) ) and, therefore, LB,W ∈ Cb (R+ , L(lS(H) )). Also, Lemma 7 remains valid when replacing the operators L and L by LB,W and LB,W , respectively. This is because the coefficients of

LB,W satisfy the conditions imposed to the coefficients of L. Then we consider the following equation in NH

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

704

  dZ(t) = LB,W (t) Z(t) , t  s  0 dt Z(s) = i,x P , s ∈ R+ , i ∈ Z, where

is the operator defined by (4). If U c  L

i,x P

B,W

(32) (33)

(t, s) is the causal evolution operator generated by the

sequence LB,W , then Z(t, s; i,x P ) = ULc (t, s)(i,x P )  0 is the unique solution of (32), (33) in NH . We shall B,W prove that there is γ > 0 such that ∞   Z(t, s; i,x P ) dt  γx2 1

(34)

s

for all s ∈ R+ , i ∈ Z and x ∈ H. By (29), it follows that ∞ 

   ULaB,W (s, t) Φ[H] [i] x, x dt  γx2

s Z Z and Theorem 5 of [26] shows that LB,W ∈ Cb (R+ , L(lS(H) )) generates an anticausal u.e.s. evolution in lS(H) . Let us prove (34).

 C] implies that there is Step 1. First, we establish a sufficient condition for (34). The detectability of [L, Z F   a bounded sequence F ∈ Cb (R+ , lL(V,H) ) such that L (t) = L(t)(X) + F (t)C(t)X + XC [∗] (t)F [∗] (t), t ∈ R+ generates a causal u.e.s. evolution operator ULcF (t0 , t) on (NH , | · |1 ) (see Definition 8). We define  1 (t)(X). Ω F,ε (t)(X) = LF (t)(X) + 2ε2 X + ε2 Π  1 (t) is positive for all t ∈ R+ we can apply Proposition 3.3 from [4] to deduce that Ω F,ε generates Since Π a positive causal evolution on NH . Using Gronwall’s lemma it follows that there is ε0 ∈ (0, 1) such that c Ω F,ε (t), t ∈ R+ generates a causal u.e.s. evolution operator UΩ F,ε (t, s) for all 0 < ε < ε0 . For ε ∈ (0, ε0 ), dY (t) F,ε we consider the equation dt = Ω (t)(Y (t)) + Θε (t), Y (s) = i,x P , where, for all t  s  0, Θε (t) =

 [∗] 1 1 (BW)(t)Z(t) B(t)W(t) + 2 F (t)C(t)Z(t) 2 ε ε    [∗]   1  [∗] · F (t)C(t) + 1+ 2 Π 2 (t) W(t)Z(t)W (t) ε

and Z(t) is the solution of (32). An easy computation shows that   d(Y (t) − Z(t)) = Ω F,ε Y (t) − Z(t) + Ψ (t), dt Y (s) − Z(s) = 0,

t  s, t ∈ R+ ,

where   [∗]  1 1 [H] [H] Ψ (t) = εΦ − B(t)W(t) Z(t) εΦ − B(t)W(t) ε ε   [∗]    1 1 [H] [H]  ε,W t, Z(t) , + εΦ + F (t)C(t) Z(t) εΦ + F (t)C(t) +Π ε ε

(35)

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

 and Πε,W (t, X) :=

εΦ[H] − 1ε W



[∗] Π(t, X)

εΦ[H] − 1ε W



705

Z ) satisfies the . It is not difficult to see that Πε,W (t) ∈ L(lS(H)

 ε,W (t) is positive, too. Hence, Ψ (t)  0, t ∈ R+ and the positiveness hypotheses of Lemma 24 and therefore Π c of the evolution operator UΩ F,ε (t, s) implies that Y (t) − Z(t)  0. Consequently, |Y (t)|1  |Z(t)|1 for all t ∈ R+ , t  s. So, (34) holds if there is γ > 0 such that ∞   Y (t, s; i,x P ) dt  γx2 . 1

(36)

s

Step 2. Let us prove (36). We recall that Ω F,ε (t), t ∈ R+ generates a causal u.e.s. evolution operator t−k c c UΩ for all t  s. We know that F,ε (t, s), i.e. there are β1  1, α1 ∈ (0, 1) such that |UΩ F,ε (t, s)|1  β1 α1 t c c Y (t) = UΩ F,ε (t, s)(i,x P ) + s UΩ F,ε (t, r)(Θε (r)) dr, t > s. Passing to the norm | · |1 in the above formula, we get   Y (t)  β1 αt−s x2 + 1 1

t

  β1 α1t−r Θε (r)1 dr

(37)

s

Further we need an upper bound of the last integral. Note that, since F (t), B(t) and Π2 (t) are bounded, there is l > 1 such that max{F (t), B(t), Π2 (t)}  l for all t ∈ R+ . By hypothesis (H2), R(t) is uniformly positive, i.e. there is ρ > 0 such that R(t)  ρΦ[U ] for all t ∈ R+ . Then, arguing as in the proof of Theorem 18 from [27] we obtain      (F CZ)(r)(F C)(r)[∗]   l2 Tr C [∗] (r)C(r) [j] Z(r)[j] 1 j∈Z

       B(r)W(r)Z(r) B(r)W(r) [∗]   l Tr W[∗] (r)R(r)W(r) [j] Z(r)[j] 1 ρ 2

j∈Z

      [∗]  2 (t) W(t)Z(t)W[∗] (t)   l Π Tr W (r)R(r)W(r) Z(r) . [j] 1 [j] ρ j∈Z

2

2

l Letting m1 := max{ εl 2 , ρε 2 + (1 +

1 l ε2 ) ρ }

we have

        Θε (r)  m1 Tr C [∗] (r)C(r)[j] Z(r)[j] + Tr W[∗] (r)R(r)W(r) [j] Z(r)[j] . 1

(38)

j∈Z

For fixed W(t) Eq. (31) can be seen as a Lyapunov equation in X(t). Therefore, the solution X(t) of (19) also satisfies the integral equation

X(s) =

ULaB,W (s, t)

  X(t) +

t

  ULaB,W (s, r) C [∗] (r)C(r) + W[∗] (r)R(r)W(r) dr,

s  t.

(39)

s

Lemma 21 from [27] and (5) imply    f (r, j) := Tr C [∗] (r)C(r) + W[∗] (r)R(r)W(r) [j] Z(r)[j]   a    L (s, r)(i,x P ) = Tr C [∗] (r)C(r) + W[∗] (r)R(r)W(r) [j] U B,W [j]  a  [∗]    Tr ULB,W (s, r) C (r)C(r) + W[∗] (r)R(r)W(r) [j] (i,x P )[j] .

(40)

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

706

Obviously f (r, j)  0, r ∈ R+ , j ∈ Z. Using the monotone convergence theorem of Lebesgue, (40) and (39), we get t s

f (r, j) dr =

j∈Z



t

j∈Z s

=



=

 t





ULaB,W (s, r) C [∗] (r)C(r) + W[∗] (r)R(r)W(r)

Tr

j∈Z



f (r, j) dr

[j]

 dr(i,x P )[j]

s



     Tr X(s) − ULaB,W (s, t) X(t) [j] (i,x P )[j]  X(s)[i] x, x .

j∈Z

By (38) and the above inequality, we obtain t

    Θε (r) dr  m1 X(s)[i] x, x = m  1 x2 , 1

s

where m  1 = m1 sups∈R+ X(s)Z < ∞. Taking the integral in (37) from t = s to ∞, we see that there are positive constants ρ1 and ρ2 such that ∞ ∞     Y (t) dt  ρ1 x2 + ρ2 Θε (r) dr  (ρ1 + ρ2 m  1 )x2 . 1 1 s

s

So, (36) follows and the proof of Case I is complete. Case II. If D(t) = 0, t ∈ R+ we recall that (19) as (24). Obviously (H2) and (H3) are fulfilled if  can be written  QWD (t) 0 we replace Π and Q with Π D and QD (t) = , respectively. We note that QWD (t) = C [∗] (t)C(t), 0 R(t) t ∈ R+ and the role of the Lyapunov operator L from Case I is now played by LB,WD . Applying the results of Case I to Eq. (24), we deduce that any solution X ∈ Cb1 (R+ , KH ) of the Riccati differential equation (18) is stabilizing for (24). The stabilizing feedback gain is  [−1]   [∗] D X(t)B(t) + Π12 t, X(t) F X (t) = − R(t) + Π2D (t, X)  [−1]     [∗] = − R(t) + Π2 (t, X) X(t)B(t) + Π12 t, X(t) − D(t)R(t)[−1] Π2 t, X(t) . Let LB,F X be the Lyapunov operator defined by (17) with A, W and Π replaced by A − BR−1 D[∗] , F X and Π D , respectively. By Definition 5, LB,F X generates an anticausal u.e.s. evolution. A direct computation shows that  [−1]   [∗] = R(t)[−1] D(t)[∗] . (41) −D(t) − D(t)R(t)[−1] Π2 t, X(t) F X (t) − F X (t) = − R(t) + Π2 (t, X) Combining (15), (25) and (41) it follows that (Π D )F X = ΠF X and A − BR−1 D[∗] + BF X = A + BF X . Thus LB,F X = LB,F X and, consequently, LB,F X generates an anticausal u.e.s. evolution. Therefore, the solution X is stabilizing for (19) and the proof is complete. 2 Combining Theorem 9 and Theorem 6 we have the following. Corollary 10. Assume that the hypotheses of Theorem 9 hold. Then Eq. (19) has at most one solution from Cb1 (R+ , KH ) and if this solution exists, it is stabilizing.

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

707

Proof. Assuming that X1 , X2 ∈ Cb1 (R+ , KH ) are solutions of (19), it follows by Theorem 9 that they are stabilizing. From Theorem 6 we have X1 = X2 and the conclusion follows. 2 We note that Theorem 6 and Theorem 9 do not ensure the existence of a global and nonnegative solution for (19) and, consequently, of a stabilizing solution. The next theorem shows that an additional condition of stabilizability is needed to prove the existence of stabilizing solutions. Theorem 11. Assume (H2), (H3) and let WD (t) = −R−1 (t)D[∗] (t). If {A, B, Π} is stabilizable and [LB,WD , C] is detectable, then (19) has a unique solution X from Cb1 (R+ , KH ) and this solution is stabilizing. Proof. Let Γ Σ be defined by (60). Theorem 13 from [26] states that if {A, B, Π} is stabilizable and Γ Σ = ∅,

t ∈ R+ for arbitrary then Eq. (19) has a solution X ∈ Cb1 (R+ , lEZH ) with the property X(t)  X(t), Σ Σ

X ∈ Γ . As we have shown in the proof of Lemma 3, 0 ∈ Γ and, consequently, X ∈ Cb1 (R+ , KH ). Now, the conclusion follows from Corollary 10. 2 5. Uniform exponential stability, stochastic stabilizability and detectability In this section we define the notions of uniform exponential stability in conditional mean (UESCM), stochastic detectability (SD) and stochastic stabilizability for SDE (7), (8). In order to use the results of Section 4 for solving the optimal control problem (O), we need to reformulate these notions in an operatorial language. For example we shall prove that SD is equivalent with the detectability concept introduced by Definition 8. Two representation formulas for the solutions of SDEs (Theorem 14) and an infinite dimensional version of the Ito formula (Theorem 12), which apply to SDEs with infinite MJs, are the key tools for proving the main results of this section. Similar results can be found in [6] and [4], where finite dimensional SDEs with MJs and MN are studied. 5.1. A generalized Ito’s formula Assume that a, σk ∈ L2η,w ([t0 , T ], H), k = 1, . . . , r and ξ is an H-valued, Ht0 + -measurable random variable that satisfies E(ξ2 ) < ∞. Considering the stochastic process t x(t) = ξ +

a(t) dt + t0

r  k=1 t

t

σk (t) dwk (t),

(42)

0

we have the following Ito type formula. Z Theorem 12. If v(t, x, i) = K(t)[i] x, x, where x ∈ H, i ∈ Z and K ∈ Cb1 ([t0 , T ], lS(H) ), then

    E v t, x(t), η(t) − v(t0 , ξ, i) η(t  t  =E

  2 r  ∂v    1    ∂v  ∂ v ∗ s, x(s), η(s) + s, x(s), η(s) a(s) + s, x(s), η(s) σk (s) σk (s) Tr ∂s ∂x 2 ∂x2

t0

+

  v s, x(s), j λη(s),j j∈Z

 0 )=i



   ds 

k=1

 .

(43)

η(t0 )=i

The proof (see Appendix A) is inspired by the one for finite dimensional stochastic processes with finite MJs in [6]. Instead of repeating here step by step the proof from [6], we shall point out the differences which

708

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

are mainly caused by the infiniteness of the state space Z, the use of the normal filtration Ht+ and the work with infinite dimensional stochastic processes. 5.2. Representation formulas For the rest of the paper we assume that (H1) holds and we denote by (A0 , B0 , A1 , B1 , . . . , Ar , Br ; C0 , D0 ) the stochastic system (7), (12). If any of the coefficients is zero we will remove it from the notation. For example (A0 , A1 , . . . , Ar ) denotes the stochastic Eq. (7) without control (i.e. Bk = 0, k = 0, . . . , r) etc. Also, we assume that L is the linear operator defined by (26), where A(t)[i] = A0 (t)[i] + Π1 (t, X)[i] =

r

λii IH , 2

A∗k (t)[i] X[i] Ak (t)[i] +

k=1

(44)

λij X[j] ,

(45)

j∈Z, j =i

Z for all t ∈ R+ , X ∈ lS(H) and let

Π12 (t, X)[i] =

r

A∗k (t)[i] X[i] Bk (t)[i] ,

(46)

Bk∗ (t)[i] X[i] Bk (t)[i] ,

(47)

k=1

Π2 (t, X)[i] =

r k=1

B(t)[i] = B0 (t)[i] ,

Z t ∈ R+ , X ∈ lS(H)

(48)

We note that A, B and Π1 , Π12 , Π2 satisfy (H2) and therefore L generates a positive anticausal evolution Z . operator ULa (t, s) in lS(H) Some results of this section are based on those from Section 4 obtained under hypothesis (H3). Unfortunately, (H1), the properties of the transition rate matrix Λ and the particular form of the operators defined Z , above do not ensure (H3). This is because the linear operator P(X)(i) = j∈Z, j =i λij X[j] , X ∈ lS(H) i ∈ Z, appearing in (45), is not necessarily well defined and bounded on HH . Consequently, we cannot say  and Π  W (t) defined that ΠW (t) and L(t) are linear and bounded operators on HH and their adjoints L(t) in Section 4 on HH could not exist. An example illustrating this situation is the following. Example 13. Let H = Rn and assume that, for a fixed j0 ∈ Z, i∈Z |λij0 |2 = ∞. Let X j0 ∈ HRn be j0 j0 = 0 for all j ∈ Z, j = j0 . Then P(X j0 )(i) = j∈Z, j =i λij X[j] = λij0 IH defined by X[jj00 ] = IH and X[j] and |P(X j0 )|22 = i∈Z λij0 IH 22 = n i∈Z |λij0 |2 = ∞. Thus P and the associated operators Π1 (t) and ΠW (t) are not well defined on HRn and (H3) fails to hold. To ensure (H3) we assume (11). We note here that, under the hypotheses of the above example, i∈Z |λij0 | = ∞ and (11) is not satisfied. Arguing exactly as in the proofs of Theorem 1 and Lemma 1 from [7] we can prove that P ∈ L(HH ) and P ∈ L(NH ), respectively. Using the properties of Hilbert–  exists. A standard computation Schmidt operators [22,27], it follows that L(t) ∈ L(HH ) and its adjoint L(t) shows that, for all X ∈ HH ,



r     [∗] [∗]  Ak (t)XAk (t) [i] + + L(t)(X) [i] = A(t)X + XA(t) [i] k=1

j∈Z, j =i

λji X[j] .

(49)

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

709

Hypothesis (H1) and the convergence properties of the sequences of Hilbert–Schmidt and nuclear operators Z imply that L and L belong to both Cb (R+ , L(HH )) and Cb (R+ , L(NH )). Also, for all W ∈ Cb (R+ , lL(H,U ) ), ΠW defined by (15), (44)–(48) satisfies (H3). In the absence of condition (11), the operator defined by (49)  ∈ L(NH ), t ∈ R+ , L ∈ Cb (R+ , L(NH )). However, it does not necessarily remains well defined on NH and L(t) belong to L(HH ) and (H3) fails to hold.  and the At this moment we do not know what are the relationships between the operators L(t) and L(t) evolution operators they generate in the absence of condition (11) or, even more general, in the absence of hypothesis (H3). Let us recall that not all results of Section 4 are obtained with (H3). The same occurs in this section, where some results will be obtained without (11). Therefore, in the rest of the paper we do not assume (11) implicitly. We will specify each time if we need it. Following [4] and [23] we have the next representation theorem. Theorem 14. Let x(t) be the solution of the stochastic equation (A0 , A1 , . . . , Ar ) satisfying x(t0 ) = x ∈ H, t  t0  0 and let i ∈ Z. Z , we have (i) For any D ∈ lS(H)

  E D[η(t)] x(t), x(t) η(t

 0 )=i

  = ULa (t0 , t)(D)[i] x, x .

(50)

(ii) If (11) holds then     E χη(t)=j x(t) ⊗ x(t)|η(t0 )=i = ULc(t, t0 )(i,x P ) [j] ,

j ∈ Z.

(51)

Z Proof. (i) If K ∈ C 1 (R+ , L(lS(H) )), then we apply Theorem 12 for the process x(τ ) and the function v(τ, x, i) = K(τ )[i] x, x, τ  t0 and we obtain

   E K(τ )[η(τ )] x(τ ), x(τ ) η(t )=i 0  τ         d  K(r) + L(r)K(r) = K(t0 )[i] x, x + E x(r), x(r) dr , dr [η(r)] η(t0 )=i

j ∈ Z.

t0

Z Taking K(τ ) = ULa (τ, t)(D), D ∈ lS(H) , t0  τ  t, in the above formula, we get

  E ULa (τ, t)(D)[η(τ )] x(τ ), x(τ ) η(t

 0 )=i

  − ULa (t0 , t)(D)[i] x, x = 0.

For τ = t we obtain (50). (ii) Let T ∈ HH be arbitrary chosen. Applying Theorem 12 for the process x(t) and the function v(t, x, i) = T[i] x, x = T, (i,x P )Z , we have    E T, (η(t),x(t) P ) Z η(t  t =E



 0 )=i

  − T, (i,x P ) Z

L(s)(T ), (η(s),x(s) P )

 Z

 ds

 η(t0 )=i

t0

t = t0



    T, L(s) E (η(s),x(s) P )|η(t0 )=i Z ds.

 t =E t0





T, L(s)( η(s),x(s) P )

 Z

 ds

 η(t0 )=i

710

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

This shows that E[η(t),x(t) P |η(t0 )=i ] is the unique solution of the equation Z(t) = (i,x P ) + in HH . Thus E[η(t),x(t) P |η(t )=i ] = U c (t, t0 )(i,x P ). We observe that

t t0

 L(s)Z(s) ds

 L

0

      E (η(t),x(t) P )|η(t0 )=i [j] = E χη(t)=k (k,x(t) P )|η(t0 )=i [j] = E χη(t)=j x(t) ⊗ x(t)|η(t0 )=i k∈Z

and we obtain the conclusion. 2 Z Remark 15. Let W ∈ Cb (R+ , lL(H,U ) ). If we compute L with Ak replaced by Ak + Bk W , k = 0, . . . , r, we get the formula of LB,W . Therefore, the statement of the above theorem remains true if we replace Ak , k = 0, . . . , r and L by Ak + Bk W , k = 0, . . . , r and LB,W , respectively.

5.3. Some equivalence results Definition 16. Let x(t) be the unique solution of the stochastic equation (A0 , A1 , . . . , Ar ) such that x(t0 ) = x ∈ H. We say that (A0 , A1 , . . . , Ar ) is uniformly exponentially stable in conditional mean (UESCM) iff there are β  1 and α ∈ (0, 1) such that 2   E x(t) η(t

 0 )=i

 βαt−t0 x2 ,

(52)

for all t  t0  0, x ∈ H, i ∈ Z. Definition 17. The system (A0 , A1 , . . . , Ar ; C0 ) is stochastically detectable (SD) if there is F ∈ Cb (R+ , Z lL(V,H) ) such that the stochastic equation (A0 + F C0 , A1 , . . . , Ar ) is UESCM. Definition 18. The system (A0 , B0 , A1 , B1 , . . . , Ar , Br ) is stochastic stabilizable (SS) iff there is W ∈ Z Cb (R+ , lL(H,U ) ) such that (A0 + B0 W, . . . , Ar + Br W ) is UESCM. The following theorem provides necessary and sufficient conditions for the UESCM property of the equation (A0 , A1 , . . . , Ar ). Theorem 19. The following assertions are equivalent: (i) (A0 , A1 , . . . , Ar ) is UESCM; Z (ii) L generates an anticausal u.e.s. evolution operator ULa (t, t0 ) on lS(H) . If, in addition, (11) holds, then (i) and (ii) are equivalent with (iii) L generates a causal u.e.s. evolution operator ULc(t, t0 ) on NH . Proof. Taking D = Φ[H] in (50), we obtain 2   E x(t) η(t

 0 )=i

    = ULa (t0 , t) Φ[H] [i] x, x .

(53)

Lemma 6 from [25] implies that (ii) is equivalent with the existence of β  1 and α ∈ (0, 1) such that ULa (t0 , t)(Φ[H] )  βαt0 −t . Then the equivalence (i) ⇔ (ii) follows from (53) and Definition 16. Now let us assume (11). Then (H3) is satisfied and the equivalence (ii) ⇔ (iii) follows from the last statement of Lemma 7. 2

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

711

A direct consequence of the above theorem and of Remark 15 is the next result. Corollary 20. The following assertions are equivalent. (i) The system (A0 , B0 , A1 , B1 , . . . , Ar , Br ) is stochastic stabilizable (SS). Z (ii) There is W ∈ Cb (R+ , lL(H,U ) ) such that LB,W generates an anticausal u.e.s. evolution operator a Z ULB,W (t, t0 ) on lS(H) . If (11) holds, then (i) and (ii) are equivalent with Z c  (iii) There is W ∈ Cb (R+ , lL(H,U B,W (t, t0 ) ) ) such that LB,W generates a causal u.e.s. evolution operator UL on NH .

Note that, Proposition 6.6 and Lemma 4.3 from [8] and the above corollary show that the notions of stochastic stabilizability introduced in [8] and the one in this paper are equivalent. Theorem 21. The following two statements are equivalent. (i) The system (A0 , A1 , . . . , Ar ; C0 ) is stochastically detectable (SD). Z (ii) The operator LF defined by (30) generates an anticausal u.e.s. evolution operator ULaF (t, t0 ) on lS(H) . If (11) holds, then (i) and (ii) are equivalent with  C] ∈ Cb (R+ , L(NH )) × Cb (R+ , lZ (iii) The pair [L, L(U,H) ) is detectable. Proof. We note that replacing A0 with A0 +F C0 in the formula of the operator L we obtain the operator LF defined by (30), (44) and (45). From Definition 17 and Theorem 19 it follows that statement (i) is equivalent with (ii). The last statement follows from Lemma 7 and Definition 8. 2 Theorem 21 shows that the detectability notion introduced by Definition 8 generalizes the concept of stochastic detectability defined above if (11) is satisfied. 6. Optimal control In this section we shall solve the optimal control problem (O) under stochastic stabilizability and stochastic detectability conditions. As in the case of stochastic differential equations without Markov jumps [24], the design of an optimal control which minimizes (13) subject to (7)–(8) is closely related to the existence of bounded and stabilizing solutions for an associated Riccati equation of the form (19). [∗] Assuming that the hypotheses of the previous section hold, we set M (t) = C0 (t)C0 (t), R(t) = [∗] [∗] Z D0 (t)D0 (t), D(t) = C0 (t)D0 (t). We also assume that there is C ∈ Cb (R+ , lL(H,V ) ) such that M (t) − D(t)R−1 (t)D[∗] (t) = C [∗] (t)C(t) for all t ∈ R+ . Let us consider (19) in the special case where the coefficients are defined by (44)–(48) and the above relations. Lemma 22. Let xu (t) be the solution of (7)–(8) and let X0 ∈ KH . If X(·) = X(T, · ; X0 ) ∈ C 1 ([0, T ], KH ) is the unique solution of (19)–(21), and F X is defined by (23) with X replaced by X(T, · ; X0 ) then E

     (X0 )[η(T )] xu (T ), xu (T ) − X(t)[i] xu (t), xu (t) η(t)=i

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

712

 T =E t

−



      R(s) + Π2 s, X(s) 1/2 u(s) − F X xu (s) 2 [η(s)] [η(s)]

      M (s)[η(s)] xu (s), xu (s) + 2 D(s)[η(s)] u(s), xu (s) + R(s)[η(s)] u(s), u(s) dsη(t)=i . 

(54)

Proof. We apply Theorem 12 for the process x(t) and the function v(t, x, i) = X(t)[i] x, x and we get the conclusion. 2 Lemma 22 is an infinite dimensional version of Corollary 2 in [6] and may be used in the study of finite horizon linear quadratic control problems. The next theorem shows that, in conjunction with the stabilizability and detectability conditions, the above lemma gives rise to the solution of the infinite horizon control problem (O). Now let us assume (11). Then all the hypotheses of Section 4 are satisfied and we have the following. Theorem 23. Let WD (t) = −R−1 (t)D[∗] (t). Assume that (A0 , B0 , A1 , B1 , . . . , Ar , Br ) is stochastic stabilizable and (A0 + B0 WD , . . . , Ar + Br WD ; C) is SD. Then (i) (19) has a unique solution X from Cb1 (R+ , KH ) and this solution is stabilizing; (ii) the optimal control problem (O) has a solution, uopt (t) = F X (t)[η(t)] x(t)

(55)

where x(t) is the solution of the closed loop system (7), (55) with the initial condition x(t0 ) = x. The optimal cost is J(uopt , t0 , x, i) = X(t0 )[i] x, x. Proof. The stochastic stabilizability of (A0 , B0 , A1 , B1 , . . . , Ar , Br ), Corollary 20 and Definition 4 imply that {A, B, Π} is stabilizable. On the other hand, Theorem 21 and Remark 15 show that (A0 + B0 WD , . . . , Ar + Br WD ; C) is SD if and only if the pair [LB,WD ; C] is detectable. Therefore the hypotheses of Theorem 11 are fulfilled and (i) follows. Let X(t) be the unique stabilizing solution of (19) from Cb1 (R+ , KH ) and let F X be defined by (23) with X Z replaced by X(t). From Definition 5, LB,F X generates an anticausal u.e.s. evolution on lS(H) . By Theorem 19, X the unique solution x(t) of the stochastic differential equation (A0 + B0 F , A1 + B1 F X , . . . , Ar + Br F X ) satisfying x(t0 ) = x has the property that there are β  1, α ∈ (0, 1) such that 2   E x(t) η(t

 0 )=i

 βαt−t0 x2

(56)

for all t  t0  0, x ∈ H, i ∈ Z. Taking uopt (t) = F X (t)[η(t)] x(t) in (7)–(8) and applying Lemma 22, we get E



   X(T )[η(T )] xuopt (T ), xuopt (T ) − X(t0 )[i] x, x η(t  T

= −E



   M (s)[η(s)] xuopt (s), xuopt (s) + 2 D(s)[η(s)] uopt (s), xuopt (s)

t0

 + R(s)[η(s)] uopt (s), uopt (s) dsη(t0 )=i 



 T = −E t0

 0 )=i



   C0 (s)[η(s)] xu (s) + D0 (s)[η(s)] uopt (s)2 ds opt η(t

 0 )=i

.

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

713

Passing to the limit for T → ∞ in the above relation and taking into account (56) and the boundedness of X(t), we get 

 X(t0 )[i] x, x = J(uopt , t0 , x, i) < ∞.

(57)

This proves that uopt is admissible. Let u ∈ U(t0 , x0 ) and let X(·) = X(T, · ; 0) ∈ C 1 ([0, T ], KH ) be the unique solution of (19) satisfying X(T ) = 0 (see Lemma 3). Applying (54) for X(T, · ; 0), u and t = t0 we see that  T JT (u, t0 , x, i) := E

   C0 (s)[η(s)] xu (s) + D0 (s)[η(s)] u(s)2 ds

 η(t0 )=i

t0

   X(T, t0 ; 0)[i] x, x .

(58)

From the proof of Theorem 15 from [26] we know that X(T, t0 ; 0) converges strongly and componentwise ≈ to a solution X ∈ Cb1 (R+ , KH ) of (19). According to Theorem 9 this solution is stabilizing and an appeal ≈ to Theorem 6 shows that X = X. Letting T → ∞ in (58), we get ≈    J(u, t0 , x, i)  X(t0 )[i] x, x = X(t0 )[i] x, x . The conclusion follows from (57). 2 7. Conclusions This paper solves the infinite horizon LQ optimal control problem (O) for time-varying linear SDEs with infinite MJs and MN in infinite dimensions. The optimal control result may be applied in modern queuing network theory or in the field of safety-critical and high integrity systems, where the processes may experience abrupt changes in their dynamics and are modeled by SDEs with infinite MJs. In connection with this optimization problem we study the asymptotic behavior of the infinite dimensional GRDEs (18) under detectability and stabilizability conditions imposed to the coefficients. The properties of the Lyapunov type operators L(t), t ∈ R+ generating positive evolutions on Banach spaces play the key role in obtaining the main results of this paper. They are involved in the solution representation formulas (51), (50), in stability and detectability problems (see Section 5.3), as well as in the global solvability of GRDEs. As we have shown in Section 5, the study of these operators is far to be complete, especially in the case when hypothesis (H3) (or condition (11), if L(t) is associated with SDEs) is not fulfilled. In the context of a nonstochastic approach, the sufficient conditions for the global solvability of GRDE (18), provided by this paper, as well as the properties of the operators L(t), t ∈ R+ are of interest by themselves in the framework of the theory of differential equations. Appendix A For reader convenience we recall here the following result, proved in [27]. Z Lemma 24. Assume that A ∈ L(lS(H) ) is positive, m-strongly continuous operator with the properties  A(HH ) ⊂ HH and A|H ∈ L(HH ). If A is the adjoint operator of A|H in HH , then H

H

 is positive, i.e. A(H  H ∩ KH ) ⊂ HH ∩ KH ; (a) the operator A



 (b) A(i,x P ) ∈ NH and |A(i,x P )|1 = A(Φ[H] )[i] x, x, for all i ∈ Z, x ∈ H;

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

714

 H ) ⊂ NH and A  ∈ L(NH ). Moreover, A   A, where A  and A are the norms of A  and A (c) A(N Z as elements of L(NH ) and L(lS(H) ), respectively. Proof of Lemma 1. Let T (t) denote the C0 semigroup of contractions on lRZ that corresponds to the standard transition probability matrix P (t). Similarly, let Λ be the linear operator on lRZ which matrix representation is Λ. By Theorem 3.3 from [1] and Lemma 3.1 from [1] it follows that Λ is a linear and bounded operator on lRZ and the infinitesimal generator of T (t). Thus T (t) is a uniformly continuous semigroup of contractions [20] and, consequently, it is  · Z -continuous and  · Z -differentiable on lRZ . Hence, for any φj ∈ lRZ , j ∈ Z, defined by φj[k] = δkj , k ∈ Z, we have limt0 T (t)(φj ) − φj Z = 0, uniformly with respect to (for short u.w.r.t.) j. Thus limt0 supi∈Z |pij (t) − δij | = 0, u.w.r.t. j. It follows that limt0 pij (t) = δij u.w.r.t. j j i and j. For the last convergence, we see that limt0  T (t)(φt )−φ − k∈Z λik φj[k] Z = 0 u.w.r.t. j. So, limt0 supi∈Z |

pij (t)−δij t

− λij | = 0 u.w.r.t. j and the conclusion follows. 2

Proof of Lemma 2. The proof of statement (a) is standard and will be omitted. (b) The right continuity of the filtration {Ht+ }t0 and of the Markov process η(t), combined with Theorem 34 from [6] show that there is a sequence of real numbers such that qn  0 and E[χη(t+h)=i |Ht+ ] = lim E[χη(t+h)=i |Ht+qn ] qn 0

= lim E[χη(t+h)=i |η(t+qn ) ] = lim pη(t+qn )i (h) = pη(t)i (h). qn 0

The proof is complete.

qn 0

2

Proof of Lemma 3. Following [26], the GRDE (19) can be equivalently written as We associate with (19) the dissipation operator D

Σ







t, X(t) =

dΣ 1 (t, X(t)) [∗] dΣ 2 (t, X(t))

dΣ 2 (t, X(t)) R(t) + Π2 (t)(X(t))

d dt X(t) + R(t, X(t))

= 0.

 ,

(59)

  d dΣ X(t) + L(t)X(t) + M (t), 1 t, X(t) = dt     dΣ 2 t, X(t) = XB(t) + Π12 t, X(t) + D(t) and the subset Γ Σ of Cb (R+ , lEZH ) defined by         Γ Σ = X ∈ Cb1 R+ , lEZH  R(t) + Π2 t, X(t)  0, DΣ t, X(t)  0, t ∈ R+ .

(60)

Hypothesis (H2) (ii) implies that the operator Π2 (t) is positive for all t ∈ R+ , i.e. Π2 (t)(X0 )  0 for all X0 ∈ KH . Since R(t)  0, t ∈ R+ it follows that there is γ > 0 such that R(t) + Π2 (t)(X0 )  γΦ[H]

(61)

for all t ∈ R+ and X0 ∈ KH . Thus R(t, i) + Π2 (t, X0 )(i)  γI, for all t ∈ R+ , X0 ∈ KH and i ∈ Z. Consequently, (t, X0 ) ∈ Dom R for all t ∈ R+ and X0 ∈ KH . A direct computation, (H2) (i) and (61) show that 0 ∈ Γ Σ . Corollary 8 from [26] states that if 0 ∈ Γ Σ , X0  0 and t ∈ R+ then there is a unique solution X(T, t; X0 ) of (19)–(21), which belongs to Cb1 ([0, T ], KH ). 2 Proof of Lemma 7. Arguing as in [26] we can prove that the family of linear and bounded operators X → A(t)X +XA(t)[∗] , X ∈ NH , t ∈ R+ generates a positive causal evolution operator on NH . From (H2) (ii) and

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

715

(H3) (i) we know that Π1 (t) is a positive and m-strongly continuous operator with HH as an invariant sub 1 (t)(HH ∩KH ) ⊂ HH ∩KH and Π  1 (t) ∈ L(NH ). Since NH ⊂ HH , space. Applying Lemma 24 it follows that Π  1 (t)(NH ∩ KH ) ⊂ NH ∩ KH . Then the first statement of the lemma follows by Proposition we deduce that Π 3.3 (ii) from [4]. Also, (29) follows directly from Lemma 24 and (5) by observing that the anticausal evolution operator generated by L|HH on HH is exactly ULa (s, t)|HH and that ULa (s, t) is m-strongly continuous. Now let us prove the last statement of the lemma. Assuming that L generates a causal u.e.s. evolution operator ULc(t, s) on NH and using (29) it follows that there are β  1, α ∈ (0, 1) such that ULa (s, t)(Φ[H] )[i] x, x  βαt−s |i,x P |1 = βαt−s x2 for all x ∈ H and t  s. Hence, U a (s, t)(Φ[H] )  βαt−s for all t  s and L

Lemma 6 from [25] implies that L generates an anticausal u.e.s. evolution operator ULa (s, t). For the converse,  a (s, t)  U a (s, t). From (5), we get the conclusion. 2 we apply Lemma 24 and we have U L L Proof of Theorem 12. As we have mentioned before, the proof is similar to the one of Theorem 35 from [6]. It consists of three steps. At the first step it is proved (43) under the additional hypothesis (h1 ) ξ is bounded on Ω, a, σ are bounded on [t0 , T ]×Ω and a(t), σ(t) are with probability 1, right continuous functions on [t0 , T ]. For the second step, we replace (h1 ) by (h2 ) a, σ are bounded on [t0 , T ] × Ω and a(t), σ(t) are Ht+ adapted. At the third step, formula (43) is proved in the general case. Step I. Assume that ξ, a and σ satisfy the additional hypothesis (h1 ). We have     v t + h, x(t + h), η(t + h) − v t, x(t), η(t)          = χη(t+h)=j v t + h, x(t + h), j − v t, x(t), j + v t, x(t), η(t + h) − v t, x(t), η(t)

(62)

j∈Z

For each fixed j ∈ Z, we can apply the infinite dimensional Itô’s formula (Theorem 4.17 from [21]) to the function v(t, x, j) and the process x(t) defined by (42) and we obtain v(t + h, x(t + h), j) −  t+h r  t+h ∂K(s) v(t, x(t), j) = t mj (s) ds + k=1 t 2K(s)[j] x(s), σk (s) dwk (s), where mj (s) =  ∂s [j] x(s), x(s) + r 2K(s)[j] x(s), a(s)+ k=1 K(s)[j] σk (s), σk (s). Arguing as in [6] and using Lemma 2 (a) and the properties  t+h of the conditional expectation, we get E[χη(t+h)=j t 2K(s)[j] x(s), σk (s) dwk (s)|η(t0 )=i ] = 0. Furthermore, since x(t) is continuous it follows that s → mj (s) is right continuous P -a.s., uniformly with respect to j ∈ Z. Therefore 1 lim h0 h

t+h  mj (s) ds = mj (t)

(63)

t

P -a.s., uniformly with respect to j ∈ Z. On the other hand, mj (t) is Ht+ measurable and        E χη(t+h)=j mj (t)|η(t0 )=i = E E χη(t+h)=j mj (t) H η(t )=i t+ 0       = E mj (t)E [χη(t+h)=j ]|Ht+ η(t )=i = E mj (t)pη(t)j (h)|η(t0 )=i . 0

To obtain the last equality we have applied Lemma 2 (b).

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

716

Lemma 1 ensures that limh0 pη(t)j (h) = δη(t),j u.w.r.t. η(t) and j. Now, an appeal to the bounded convergence theorem of Lebesgue shows that     lim E χη(t+h)=j mj (t)|η(t0 )=i = E mj (t)δη(t),j |η(t0 )=i

(64)

h0

u.w.r.t. j. Applying Burkholder–Davis–Gundy inequality we can prove that supt∈[t0 ,T ] E[x(t)4 ] < ∞ and  t+h that there is c > 0, independent of t and h, such that E[(χη(t+h)=j h1 t mj (s) ds)2 |η(t0 )=i ] < c. From (63), (64) and Theorem 6 on page 72 in [14], it follows that 

1 lim E χη(t+h)=j h0 h

 t+h    mj (s) ds  t



  = E mj (t)δη(t),j |η(t0 )=i ,

η(t0 )=i

u.w.r.t. j. We conclude that       1 χη(t+h)=j v t + h, x(t + h), j − v t, x(t), j η(t )=i 0 h0 h j∈Z     E mj (t)δη(t),j |η(t0 )=i = E χη(t)=j mj (t)|η(t0 )=i = 

lim E

j∈Z

  = E m(t)|  η(t0 )=i ,

j∈Z

(65)

where m(t)  := mη(t) (t). Next, we evaluate      (∗) = E v t, x(t), η(t + h) − v t, x(t), η(t) η(t

 0 )=i

.

Arguing exactly as in [6], and using Lemma 2 (b) we obtain

(∗) =

      E v t, x(t), j − v t, x(t), η(t) pη(t)j (h)η(t

 0 )=i

.

j∈Z, j =η(t)

Then Theorem 6 from [14] and Lemma 1 imply     1   E v t, x(t), η(t + h) − v t, x(t), η(t) η(t )=i 0 h       E v t, x(t), j − v t, x(t), η(t) λη(t)j η(t =

lim

h0

 0 )=i

j∈Z, j =η(t)

=



    E v t, x(t), j λη(t)j η(t

 0 )=i

.

(66)

j∈Z

Combining (62), (65) and (66) we see that     1   lim E v t, x(t + h), η(t + h) − v t, x(t), η(t) η(t )=i = E 0 h0 h



    m(t)  + v t, x(t), j λη(t)j  j∈Z

 . η(t0 )=i

If we denote by hi (t) the right member of the above equality, we observe that hi (t) and the right member of (43) coincide. Now we set Gi (t) = E[v(t, x(t), η(t))|η(t0 )=i ] and, arguing as in [6], it follows that Gi (t) − t Gi (t0 ) = t0 hi (s) ds. The proof of Step I is complete.

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

717

Step II. Assume (h2 ). Following [6], the random variable ξ and the functions a, σ can be approximated by certain sequences ξn , an and σn satisfying the hypotheses of Step I and such that supt∈[t0 ,T ] Ex(t) − xn (t)2 − −−→ 0, where xn (t), n ∈ N is defined by (42) with ξ, a and σ replaced by ξn , an and σn , respectively. n→∞ Therefore, (43) holds for v(t, x, i) and xn and, passing to the limit for n → ∞ in the obtained formula, we get the conclusion. We observe that E



     v t, xn (t), j λη(t)j − v t, x(t), j λη(t)j 

j∈Z

=E







j∈Z



η(t0 )=i

   K(s, j)xn (s), xn (s) − K(s, j)x(s), x(s) λη(t)j 

j∈Z

   K(t)Z E









   λη(t)j xn (t) − x(t)xn (t) + x(t)η(t

j∈Z

      2cK(t)Z E xn (t) − x(t)xn (t) + x(t)η(t

 0 )=i



η(t0 )=i

0 )=i

− −−→ 0 n→∞

and the last convergence is uniform with respect to t ∈ [0, T ]. The proof of the last step is the same as in [6] and will be omitted. 2 References [1] J. Baczynski, M.D. Fragoso, A note on the convergence of probability functions of Markov chains, Markov Process. Related Fields 19 (1) (2013) 141–148. [2] E.K. Boukas, Control of Singular Systems with Random Abrupt Changes, Comm. Control Engrg. Ser., Springer-Verlag, Berlin, 2008. [3] H. Daduna, Queueing Networks with Discrete Time Scale, Lecture Notes in Comput. Sci., vol. 2046, Springer, 2001. [4] V. Dragan, T. Morozan, Criteria for exponential stability of linear differential equations with positive evolution on ordered Banach spaces, IMA J. Math. Control Inform. 27 (3) (2010) 267–307. [5] V. Dragan, G. Freiling, A. Hochhaus, T. Morozan, A class of nonlinear differential equations on the space of symmetric matrices, Electron. J. Differential Equations 2004 (2004), Paper No. 96, 48 pp. [6] V. Dragan, T. Morozan, A. Stoica, Mathematical Methods in Robust Control of Linear Stochastic Systems, Springer, 2006. [7] V. Dragan, T. Morozan, V. Ungureanu, Some Lyapunov type positive operators on ordered Banach spaces, Ann. Acad. Rom. Sci. Ser. Math. Appl. 5 (1–2) (2013). [8] M.D. Fragoso, J. Baczynski, Optimal control for continuous time LQ-problems with infinite Markov jump parameters, SIAM J. Control Optim. 40 (1) (2001) 270–297. [9] M.D. Fragoso, J. Baczynski, Lyapunov coupled equations for continuous time infinite Markov jump linear systems, J. Math. Anal. Appl. 274 (2002) 319–335. [10] M.D. Fragoso, J. Baczynski, Stochastic versus mean square stability in continuous-time linear infinite Markov jump parameter systems, Stoch. Anal. Appl. 20 (2002). [11] M.D. Fragoso, J. Baczynski, On an infinite dimensional perturbed Riccati differential equation arising in stochastic control, Linear Algebra Appl. 406 (2005) 165–176. [12] A. Friedman, Stochastic Differential Equations and Applications, vol. I, Academic Press, New York, 1975. [13] L. Gawarecki, V. Mandrekar, Stochastic Differential Equations in Infinite Dimensions with Applications to Stochastic Partial Differential Equations, Springer, Berlin, Heidelberg, 2011. [14] I. Ghihman, A. Skorohod, Introduction to the Theory of Random Processes, W.B. Saunders Company, 1969, Library of Congress catalog card number 68-18281, Moscow, 1965. [15] K. Ito, Foundations of Stochastic Differential Equations in Infinite Dimensional Spaces, SIAM, Philadelphia, 1984. [16] L. Ju, M. Daleckii, G. Krein, Stability of Solutions of Differential Equations in Banach Space, Transl. Math. Monogr., vol. 43, American Mathematical Society, Providence, RI, 1974. [17] F.P. Kelly, Invariant measures and the q-matrix, in: J.F.C. Kingman, G.E.H. Reuter (Eds.), Probability, Statistics and Analysis, in: London Math. Soc. Lecture Note Ser., vol. 79, Cambridge University Press, Cambridge, UK, 1983, pp. 143–160. [18] C. Kubrusly, The Elements of Operator Theory, second edition, Birkhäuser, 2011. [19] M. Mariton, Jump Linear Systems in Automatic Control, Marcel Dekker, New York, 1990. [20] A. Pazy, Semigroups of Linear Operators and Applications to Partial Differential Equations, Appl. Math. Sci., vol. 44, Springer-Verlag, Berlin, New York, 1983. [21] G. Da Prato, J. Zabczyk, Stochastic Equations in Infinite Dimensions, University Press, Cambridge, 1992. [22] H.H. Schaefer, Banach Lattices and Positive Operators, Springer-Verlag, 1974. [23] V.M. Ungureanu, Stochastic uniform observability of general linear differential equations, Dyn. Syst. 23 (3) (2008) 333–350.

718

V.M. Ungureanu / J. Math. Anal. Appl. 417 (2014) 694–718

[24] V.M. Ungureanu, Optimal control of linear stochastic evolution equations in Hilbert spaces and uniform observability, Czechoslovak Math. J. 59 (2) (2009) 317–342. [25] V.M. Ungureanu, Optimal control for linear discrete time systems with Markov perturbations in Hilbert spaces, IMA J. Math. Control Inform. 26 (1) (2009) 105–127. [26] V.M. Ungureanu, V. Dragan, Nonlinear differential equations of Riccati type on ordered Banach spaces, in: Proc. 9th Coll. QTDE, Electron. J. Qual. Theory Differ. Equ. 17 (2012) 1–22. [27] V.M. Ungureanu, V. Dragan, Stability of discrete-time positive evolution operators on ordered Banach spaces and applications, J. Difference Equ. Appl. 19 (6) (2013) 952–980.