- Email: [email protected]
Stabilization and Random Linear Regulator Problem for Random Linear Control Processes
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS ARTICLE NO.
197, 608]629 Ž1996.
0042
Stabilization and Random Linear Regulator Problem for Random Linear Control Processes Russell Johnson* Departimento di Sistemi e Informatica, Uni¨ ersita ` di Firenze, Florence 50139, Italy
and Mahesh Nerurkar † Department of Mathematics, Rutgers Uni¨ ersity, Camden, New Jersey 08102 Submitted by U. Kirchgraber Received October 19, 1994
We study in a dynamical system context the random feedback stabilization problem for linear, random control processes. We are led to study the random linear regulator problem, which we solve by considering the spectral theory of linear time-dependent Hamiltonian systems. This is done with the aid of the concepts of exponential dichotomy and rotation number. Q 1996 Academic Press, Inc.
1. INTRODUCTION The purpose of this paper is to study in a dynamical systems context the feedback stabilization problem for random linear control processes on the semi-infinite interval. By ‘‘random,’’ we mean processes x9 s a Ž t . x q b Ž t . u
Ž x g Rn, u g Rm .
Ž 1.1.
for which aŽ?. and bŽ?. may exhibit the entire range of behaviour from periodicity to Žbounded. i.i.d.’s Ži.e., independent identically distributed processes.. Though this usage is not entirely standard, it is motivated by the meaning of the word ‘‘random’’ in the theory of the random Schro¨ * The research of this author is supported by M.U.R.S.T. 40% and M.V.R.S.T. 60%. † The research of this author is supported by a Research Council Grant from Rutgers University. 608 0022-247Xr96 $18.00 Copyright Q 1996 by Academic Press, Inc. All rights of reproduction in any form reserved.
609
RANDOM FEEDBACK STABILIZATION PROBLEM
dinger operator. Stochastic methods, powerful as they are, can only be applied when the coefficients exhibit strong independence properties. Our methods apply in a vast range of circumstances when independence is not present. In a standard way, the discussion of this problem will lead us to consider the Žrandom. linear regulator problem. This problem is usually solved by constructing a solution of a Riccati equation. Our point of view is that the solution of the Riccati equation is best obtained by showing that the associated linear Hamiltonian system has an ED Žexponential dichotomy.. This implies very strong robustness properties and very good smoothness properties with respect to parameters, of the solution as well as the feedback stabilizer. We also obtain immediately the important property of ‘‘preservation of recurrence’’ of which more below. Here we note that if aŽ t . and bŽ t . exhibit chaotic time dependence, then the feedback control k Ž t . is ‘‘no more chaotic’’ than a and b w19x. We prove the existence of an exponential dichotomy by using the rotation number of the linear Hamiltonian system. In fact, we show that under a mild controllability hypothesis on Ž1.1., the rotation number of the linear Hamiltonian system is constant on an interval. This implies the existence of ED, which then by standard results gives the robustness, smoothness, and preservation of recurrence properties mentioned above. We shall also obtain a proper random analog of the pole relocation theorem. We finish the introduction by describing more precisely our formulation of the random feedback stabilization problem. Let M Ž n, m. be the set of n = m real matrices. Fix a number 1 F p F `, and let
½
C s a: R ª M Ž n, n . ¬ Sup
tq1
H tgR t
aŽ s .
p
5
ds - ` .
If p s 1, suppose in addition that lim
tq «
H ǻ0 t
a Ž s . ds s 0
uniformly in t g R. Give C the distribution topology; that is, a n ª a in C if and only if
HR a Ž t . w Ž t . dt ª HR aŽ t . w Ž t . dt n
for each smooth function w : R ª R n of compact support. Next define D s b: R ª M Ž n, m. ¬ b is uniformly bounded and uniformly continuous4 . Give D the topology of uniform convergence on compact sets. Both C and
610
JOHNSON AND NERURKAR
D support the translation flow t t ¬ t g R4 , where t t is the translation
Ž t t a. Ž s . s aŽ t q s . , Ž t t b . Ž s . s b Ž t q s .
Ž t , s g R. .
We will choose the coefficient functions aŽ t ., bŽ t . in Ž1.1. from the sets C, D, respectively. In order to use techniques of dynamical systems, however, we will replace Ž1.1. by a family of control processes having the same form. Thus let V be a compact, translation-invariant subset of C = D Žthus in particular V is metrizable.. The pair Ž V, t t ¬ t g R4. defines a flow on V in the sense that Ži. t 0 : V ª V is the identity map; Žii. t t (ts s t tqs Ž t, s g R.; Žiii. the map Ž v , t . ª t t Ž v . from V = R to V is jointly continuous. Each element of V gives rise to a control process having the form of Ž1.1.: x9 s A Ž t t Ž v . . x q B Ž t t Ž v . . u
Ž v g V.
Ž 1.1. v
We must explain the notation. If v s Ž a, b . g V, define B Ž v . s bŽ0.; then it is easily seen that B: V ª M Ž n, m. is continuous and B Žt t Ž v .. s bŽ t ., Ž t g R.. On the other hand, there may not be a continuous function A: V ª M Ž n, n. such that AŽt t Ž v .. s aŽ t . for all t g R. Therefore we regard the expression ‘‘t ª AŽt t Ž v ..’’ as a notational device for expressing the function aŽ t .. Thus t ª AŽt t Ž v .. represents the projection of v on to its first coordinate. This framework includes all time-varying systems with bounded coefficients, from periodic and almost periodic to highly stochastic systems with positive entropy, and also the intermediate case of zero entropy with or without mixing conditions. We use the word ‘‘random’’ to refer to this very wide class of coefficients. The original control process Ž1.1. generates a family Ž1.1.v if a g C and b g D. Namely, let V s clsŽt t Ž a., t t Ž b .. ¬ t g R4 : C = D. If v 0 s Ž a, b ., then Eq. Ž1.1.v 0 coincides with Ž1.1.. Our point of view is quite general in that we do not require the existence of a point v 0 g V whose orbit t t Ž v 0 . ¬ t g R4 is dense in V. The random stabilization problem is that of finding a Žat least. continuous map K:V ª M Ž m, n. such that for each v g V the origin is an asymptotically stable fixed point of Ž1.1.v with feedback rule uv Ž t . s K Ž t t Ž v . . x Ž t . . We remark that even though results of w7, 14, 20x ensure existence of feedback matrix K v Ž t . for each v , it is not clear that the family K v 4 ‘‘lifts
RANDOM FEEDBACK STABILIZATION PROBLEM
611
consistently’’ to V; i.e., it is not clear that there exists K: V ª M Ž m, n. such that K v Ž t . s K Žt t Ž v ... This problem is non-trivial in general; this is what we mean by ‘‘preservation of recurrence properties.’’ Our techniques immediately yield a consistent lifting. The strength of our methods is also illustrated when one considers robustness and smoothness of the feedback matrix K. For example if V is a manifold and A, B are C r smooth functions then K is C r as well. Robustness refers to variation of K when V itself is varied Že.g. in some function space.; results of Sacker]Sell w27x can be used to show that K is ‘‘continuous’’ under such variation in very general circumstances. Finally, we also mention that the exponential dichotomy property is very much stronger than that of positive Lyapunov exponents and in this sense our results are stronger than those of Bougerol w5, 6x when the coefficients satisfy the boundedness criterion mentioned above. The paper is organized as follows. In Section 2, we discuss quite weak conditions which imply uniform controllability. In Section 3, we briefly discuss the random linear regulator problem, introduce the concept of ED, and discuss the rotation number of linear Hamiltonian systems. In the final section, we solve the random feedback stabilization problem.
2. A CONDITION IMPLYING UNIFORM CONTROLLABILITY We introduce some notation. Consider the family of control processes x9 s A Ž t t Ž v . . x q B Ž t t Ž v . . u
Ž v g V. ,
Ž 1.1. v
where the flow Ž V, t t 4. is as described in Section 1. Let X Ž v , t . be the fundamental matrix solution of the linear system x9 s A Ž t t Ž v . . x
Ž v g V.
Ž 2.1. v
satisfying X Ž v , 0. s I}the n = n identity matrix. Then X: V = R ª GLŽ n, R. is jointly continuous and satisfies the following ‘‘cocycle identity’’: X Ž v , t q s . s X Žt t Ž v . , s . X Ž v , t .
Ž v g V , t , s g R. .
We will also consider the adjoint system x9 s yA* Ž t t Ž v . . x
Ž v g V.
Ž 2.2. v
Žhere * denotes the transpose.; the fundamental matrix solution of this system is ZŽ v , t . s X *Ž v , t .y1 .
612
JOHNSON AND NERURKAR
Let v g V and t 1 - t 2 g R. Define the following n = n matrix-valued function of v : Ww t 1 , t 2 x Ž v . s
t2
Ht
XŽ v, t.
y1
B Ž t t Ž v . . B* Ž t t Ž v . . X * Ž v , t .
y1
dt.
1
We will write WT Ž v . for Ww0, T xŽ v .. LEMMA 2.1. With the abo¨ e notation, the following identity holds for all t, T g R, and v g V: X Ž v , t . Ww t , tqT x Ž v . X * Ž v , t . s WT Ž t t Ž v . . . Proof. This is a simple calculation using the cocycle identity. DEFINITION 2.2. Fix v g V. The process Ž1.1.v is said to be controllable Žmore precisely null controllable at v . if and only if given x 0 g R n, there exists a locally integrable function uŽ t . and a number T ) 0 such that the solution x Ž t . of Ž1.1.v with u s uŽ t . and x Ž0. s x 0 satisfies x ŽT . s 0. For each v g V, it is well known that controllability of the process Ž1.1.v is equivalent to the non-singularity of WT Ž v . for some T ) 0. DEFINITION 2.3. Fix v g V. The process Ž1.1.v is said to be uniformly controllable if and only if there exist constants a ) 0, T ) 0 such that 0 - a I - WT Ž t t Ž v . . ,
; t g R.
We prove that if the flow Ž V, t t 4. is minimal in the sense that if the orbit t t Ž v . ¬ t g R4 is dense in V for all v g V, then uniform controllability on V is equivalent to simple controllability along a single orbit. This is a generalization of a theorem of Artstein w3x and is the first step towards our sufficiency condition for uniform controllability ŽProposition 2.5.. The sufficient condition does not assume minimality but rather requires only controllability along at least one orbit in each minimal subset of V. LEMMA 2.4. Suppose Ž V, t t 4. is minimal and suppose that for some v 0 g V, the process Ž1.1.v 0 is controllable. Then e¨ ery process Ž1.1.v is uniformly controllable. In fact, there exists constants a ) 0, T ) 0, which are independent of v , such that 0 - a I - WT Ž v .
Ž v g V. .
Proof. The proof is similar to that of Theorem Ž2.10. in w16x. Since the process Ž1.1.v 0 is controllable, there exists S ) 0 such that WS Ž v 0 . is
613
RANDOM FEEDBACK STABILIZATION PROBLEM
non-singular. Choose d ) 0 such that d - inf²WS Ž v 0 . x, x : ¬ 5 x 5 s 14 . Since the map Ž v , x . ª ²WS Ž v . x, x : is continuous and x ¬ 5 x 5 s 14 is compact, there exists a compact neighbourhood N of v 0 in V such that
d 2
- inf ² WS Ž v . x, x: ¬ 5 x 5 s 1, v g N 4 .
Ž 2.3.
Let us write temporarily t Ž v , t . for t t Ž v ., Ž v g V, t g R.. By minimality of V, we can find L ) 0 and a sequence Tn ª y` such that for each n G 1: Tnq 1 - Tn ;
Ž i.
Ž 2.4.
< Tnq 1 y Tn < F L;
Ž ii .
Ž 2.5.
v n s t Ž v 0 , Tn . g N.
Ž iii .
Ž 2.6.
By Ž2.3., we have for n G 1: 0-
d 2
- inf ² WT Ž t Ž v 0 , Tn . . x, x: ¬ 5 x 5 s 1 4 .
Ž 2.7.
Let t - 0. We can write t s Tn q g for some n G 1, where g g wyL, 0x. Then for any x with 5 x 5 s 1, we have
² WSq LŽ t t Ž v 0 . . x, x: s² WSqLŽ t Ž v 0 , Tn q g . . x, x: s² X Ž v n , g . Ww g , gqSqLx Ž v n . X * Ž v n , g . x, x: s² Ww g , gqSqLx Ž v n . yn , yn: , where we have written yn s X *Ž v n , g . x. Continuing, we have
² Wwg , gqSqLx Ž vn . yn , yn: G² WgqSqLŽ vn . yn , yn: , since w0, g q S q L x ; w g , g q S q L x. Thus,
² Wwg , gqSqLx Ž vn . yn , yn: G² WS Ž v n . yn , yn: G
d 2
Ž because g q L ) 0 .
5 yn 5 2 .
Letting h s min5 X *Ž v , g . x 5 2 ¬ yL F g F 0, 5 x 5 s 1, v g N 4 , we see that inf ² WSq L Ž t t Ž v 0 . . x, x: ¬ 5 x 5 s 1, t F 0 4 G
hd 2
G 0.
614
JOHNSON AND NERURKAR
Since the semi-orbit Žt t Ž v 0 . ¬ t F 04 is dense in V, we have WTqLŽ v . G hdr2 for all v g V. This shows that the statement of Lemma 2.4 holds with T s S q L. The lemma applies in particular when Ž1.1. has almost periodic coefficients aŽ t ., bŽ t .. This is due to the fact that in this case the flow Ž V, t t 4. is minimal. If Ž V, t t 4. is not minimal, then Lemma 2.4 is false, as simple examples show w3x. We now prove the main result. PROPOSITION 2.5. Suppose that, for each minimal subset M : V, there exists at least one v 0 g M such that the process Ž1.1.v 0 is controllable. Then each process Ž1.1.v is uniformly controllable and the constants T ) 0, a ) 0 can be chosen independent of v g V. In fact, WT Ž v . G a I for each v g V. Proof. Suppose for contradiction that there exist sequences v j g V, Tj ª `, a j ª 0 and x j g R n such that 5 x j 5 s 1 and 5 WT jŽ v j . x j 5 F a j . We can assume that v j ª v , x j ª x where 5 x 5 s 1. Then clearly WT Ž v . x s 0 for all T ) 0. Now however it is easily seen that, given « ) 0, there is a minimal subset M : V, a point v 0 g M and a time t ) 0 such that dŽt t Ž v ., v 0 . « . Here d is some metric on V. By continuity in v of WT Ž v . and using Lemma 2.1, we see that there exists T# ) 0 such that 5 WT#Ž v . x 5 ) 0. This is a contradiction; the proposition is proved.
3. THE RANDOM LINEAR REGULATOR PROBLEM We give a brief introduction to the random linear regulator problem and the standard approach to solving it. In doing so, we shall have occasion to recall the ‘‘Floquet Theory’’ and, in particular, the notion of rotation number for linear Hamiltonian systems. Consider once again the family of control processes x9 s A Ž t t Ž v . . x q B Ž t t Ž v . . u
Ž v g V. ,
Ž 1.1. v
where Ž V, t t 4. is a flow as discussed in Section 1. Introduce continuous matrix-valued functions Q: V ª M Ž n, n. and R: V ª M Ž m, m. with the following properties: Q* Ž v . s Q Ž v . G 0
Ž v g V.
Ž 3.1. a
R* Ž v . s R Ž v . ) 0
Ž v g V. .
Ž 3.1. b
In particular, R is assumed to be strictly positive-definite. Define the
RANDOM FEEDBACK STABILIZATION PROBLEM
615
Lagrangian
˜v Ž x, u . s² x, Q Ž v . x: q² u, R Ž v . u: , 2L and set Lv Ž x, u . s
`
H0
˜v Ž x Ž t . , u Ž t . . dt. 2L
The linear regulator problem is the following. Fix an initial vector x 0 g R n. We are to find a control function uŽ t . s uv Ž t . g L2 Žw0, `., R m . such that, if xv Ž t . is the solution of Ž1.1.v with xv Ž0. s x 0 and u s uv Ž t ., then the pair Ž xv , uv . minimizes the quadratic cost functional Lv Ž v g V .. We now sketch the standard calculus of variations approach to finding solution u of the linear regulator problem. The idea is to convert the optimization problem in to a problem in Hamiltonian dynamics by consid˜v q Ý i yi ˙x i .’’ More precisely, set ering the Hamiltonian ‘‘Hv s L
˜v Ž x, u . q² y, AŽ v . x q B Ž v . u: Hv Ž x, y, u . s L
Ž 3.2.
for x, y g R n and u g R m . The reader may worry that ‘‘ AŽ v .’’ is not strictly speaking well-defined; it will turn out that this does not matter as far as the final formulation of the problem is concerned. Note that Du Hv Ž x, y, u . s Ru q B*y Du2 Hv Ž x, y, u . s R, where Du denotes the Frechet derivative with respect to u. Since R is ´ positive definite and symmetric, it is invertible, hence our Hamiltonian is regular and also it is convex. At this point one applies the Pontryagin maximal principle w26x to conclude that a necessary condition on any control function minimizing Lv is Du Hv s 0.
Ž 3.3.
Condition Ž3.3. yields the feedback rule u s yRy1 B*y Substituting u s yRy1 B*y in the formula for Hv , one obtains Hv Ž x, y . s
² x, Q Ž v . x: q 12² y, AŽ v . x:
1 2
y 21² y, B Ž v . Ry1 Ž v . B* Ž v . y:
Ž 3.4.
616
JOHNSON AND NERURKAR
for Ž xy . g R 2 n. One can now use general arguments involving the regularity and convexity of Hv to show that the control u which optimizes Lv is given by the feedback rule Ž3.4., where y Ž t . is obtained from a certain trajectory Ž xy ŽŽ tt .. . of the Hamiltonian differential equation generated by Hv and where x Ž0. s x 0 . This completes our sketch of the calculus of variations approach to finding an optimal solution u. We consider then the Hamiltonian system corresponding to Hv defined above, x9 s
Hv y
,
yX s y
Hv x
Ž v g V. .
These equations take the form AŽt t Ž v . . x s y dt yQ Ž t t Ž v . . d
ž/
ž
yBRy1 B* Ž t t Ž v . . yA* Ž t t Ž v . .
x y
/ž /
Ž 3.5. v
for each v g V. It will sometimes be convenient to write z s Ž xy . and fŽ v. s
ž
AŽ v . yQ Ž v .
yBRy1 B* Ž v . yA* Ž v .
/
for the coefficient matrix in Ž3.5.v . Then Eq. Ž3.5.v becomes z9 s f Ž t t Ž v . . z
Ž z g R2 n , v g V . .
Ž 3.6. v
Observe that f takes values in the Lie algebra spŽ n, R. of infinitesimally symplectic 2 n = 2 n real matrices. The function uŽ t . which minimizes Lv must lie in L2 Žw0, `., R m .. The feedback rule Ž3.4. suggests that we look for solutions Ž xy ŽŽ tt .. . of Ž3.5.v for which x Ž0. s x 0 and y Ž t . g L2 Žw0, `., R n .. We will show that each Eq. Ž3.5.v admits a unique solution with these properties under mild controllability assumptions on Eqs. Ž1.1.v . In fact we will actually show much more. We will show that Eq. Ž3.6.v admits an ED. Via rule Ž3.4., we will then obtain the solution of the random linear regulator problem. The theory of ED will then give us our robustness, stability and recurrence results. Remark 3.1. Classically, one solves Ž3.5.v with x Ž0. s x 0 and y Ž t . g L2 Žw0, `., R n . by solving the Riccati equation corresponding to Ž3.5.v . We use the theory of exponential dichotomy and do not solve the Riccati equation directly.
617
RANDOM FEEDBACK STABILIZATION PROBLEM
We now review the Floquet theory for Eqs. Ž3.5.v or Ž3.6.v as given in w15, 17x. A key result in this theory is a criterion Žinvolving the rotation number. for equations Ž3.6.v to have an exponential dichotomy. Let us define this latter concept. Let F Ž v , t . be the fundamental matrix solution of Ž3.6.v satisfying F Ž v , 0. s I}the 2 n = 2 n identity matrix. Then F: V = R ª SpŽ n, R. Žthe real symplectic group. is continuous and satisfies the cocycle identity: F Ž v , t q s . s F Žt t Ž v . , s . F Ž v , t .
Ž v g V , t , s g R. .
DEFINITION 3.2. Equations Ž3.6.v are said to have an exponential dichotomy ŽED for short. o¨ er V if there exist continuous vector subbundles V s, V u of R 2 n = V with the following properties: Ža. V s [ V u s R 2 n = V ŽWhitney sum.. Žb. V s and V u are invariant under the ‘‘skew product flow’’ ˆ t on 2n R = V defined by
ˆt t Ž ¨ , v . s Ž F Ž v , t . ¨ , t t Ž v . . ,
Ž Ž ¨ , v . g R2 n = V . .
Žc. There exist constants K ) 0, a ) 0 such that if Ž ¨ , v . g V s then F Ž v , t . ¨ F Key a t 5 ¨ 5
Ž t G 0. ,
if Ž ¨ , v . g V u then F Ž v , t . ¨ F Ke a t 5 ¨ 5
Ž t G 0. .
and
One of the main steps in the development of the Floquet theory is the introduction of a parameter l Žspectral parameter. in the following way. Let g be a continuous, 2 n = 2 n matrix-valued function on V such that
g Ž v . s g *Ž v . G 0
Ž v g V. .
Consider the family of differential equations z9 s f Ž t t Ž v . . q l Jg Ž t t Ž v . . z,
Ž 3.7. v , l
. is the usual skew-symmetric matrix of dimension 2 n = 2 n. where J s Ž 0I yI 0 These equations take the standard form ŽAtkinson w4x. Jy1 z9 s Jy1 f Ž t t Ž v . . q lg Ž t t Ž v . . z upon multiplication by Jy1 ; note that Jy1 f Ž?. is symmetric because f Ž?. takes values in spŽ n, R.. When discussing quantities related to Eqs. Ž3.7.v , l , we will omit reference to the parameter l unless confusion would result.
618
JOHNSON AND NERURKAR
The Floquet theory requires the following non-degeneracy assumption on equations Ž3.6.v . It is closely related to the condition placed on Eq. Ž3.7.v , l by Atkinson in w4, Chap. 9x. HYPOTHESIS Ž3.3.. For each minimal subset M : V, there exists at least one point v 0 g M with the following property. If ¨ is a non-zero vector in R 2 n, then `
Hy`
g Žt t Ž v 0 . . F Ž v 0 , t . ¨
2
dt ) 0.
Recall that F Ž v 0 , t . is the fundamental matrix solution of Ž3.6.v 0 . Alternatively, F Ž v 0 , t . is obtained by solving Ž3.7.v , l with l s 0. It can be shown w17, Lemma Ž2.7.x that Hypothesis Ž3.3. actually implies that `
Hy`
g Ž t t Ž v . . Fl Ž v , t . ¨
2
dt ) 0
for all v g M and all l g C, whenever M is a minimal subset of V. Here FlŽ v , t . is of course, the fundamental matrix solution of Eq. Ž3.7.v , l . We now discuss the rotation number a s a Ž l. for Eqs. Ž3.7.v , l . We need to review the rudiments of the structure of the set of Lagrange planes in R 2 n. This discussion will also help to clarify the geometric meaning of the solution of the Riccati equation associated with Ž3.5.v . So, let h : R 2 n be a linear subspace of dimension n. Cal h a Lagrange plane if ² x, J y : s 0 for all x, y g h. Let L R be the compact manifold of all real Lagrange planes h : R 2 n. Consider the open subset U : L R consisting of those Lagrange planes h which admit a basis of vectors of the form
ˆe1 ˆe n , ??? . m1 mn
ž / ž /
Here ˆ e1 , ??? ˆ e n4 is the standard basis in R n Žnot R 2 n ., and m1 , ??? m n4 are n-dimensional column vectors. Each h g U can be parametrized by the n = n real matrix m s Ž m1 , ??? m n . whose columns are m1 , ??? m n . The fact that h is a Lagrange plane implies that m is symmetric: m* s m. Next let h 0 s Span e nq1 , . . . , e2 n4 be the n-plane in R 2 n spanned by the last n unit vectors. It is easily seen that h 0 g L R . Let C s h g L R ¬ dim Ž h l h 0 . G 1 4 : L R . The set C is called Maslo¨ cycle. One can show that L R y C is simply connected. The complement of C in L R is just the open set U defined
RANDOM FEEDBACK STABILIZATION PROBLEM
619
above. Also C is of codimension 1 in L R in an appropriate sense Žthough it is not a submanifold of L R , but rather a stratified submanifold.. Furthermore, C is two sided in L R in the following sense: if an oriented curve in L R passes through C , one can assign an oriented intersection number of each point of Žtransversal . intersection. This intersection number takes values between yn and n Žsee w1x.. Fix l g R and again let FlŽ v , t . be the fundamental matrix solution of Ž3.7.v , l . Since l g R, FlŽ v , t . lies in the symplectic group SpŽ n, R. for all Ž v , t .. It follows that, if h g L R , then FlŽ v , t .h g L R where FlŽ v , t .h denotes the image of h : R 2 n under the indicated linear transformation. Thus if T ) 0, the map k :t ª FlŽ v , t .h from w0, T x to L R defines a curve in L R . Let nŽT . be the number of oriented intersections this curve makes with the Maslov cycle C . We slide over certain details involved in giving a precise definition of nŽT .; these matters are discussed in w15x. consider the limit
a Ž l . s Lim
Tª`
p nŽ T . T
.
Ž 3.8.
It exists in the following sense. Let m be an ergodic measure on V Žsee the definition below.. Then there is a subset V 1 : V, whose complement has m-measure zero, such that the limit in Ž3.8. exists and is independent of both, the choice of v g V 1 and h g L R . We call this number a Ž l. s a Ž l, m . Žor rather the function l ª a Ž l.. the rotation number of Eqs. Ž3.7.v , l with respect to the ergodic measure m. We see that a Ž l. measures the average number of points of intersection of the curve k Ž t . with the Maslov cycle C as T ª y`. It is proved in w15x that l ª a Ž l. is continuous and monotone non-decreasing. We pause to recall the definition of ergodic measure Žsee w24x for a detailed discussion.. DEFINITIONS 3.4. Let m be a Radon probability measure on V. Then m is in¨ ariant if, for each Borel subset B : V, one has m Žt t Ž B .. s m Ž B . for all t g R. The measure m is ergodic if, measure of any invariant set is either 0 or 1. We also recall that the topological support of a Radon measure m on V is the complement of the largest open set V satisfying m Ž V . s 0. We now state the basic relation between the rotation number and the existence of ED for Eqs. Ž3.7.v , L . THEOREM 3.5. Suppose that m is an ergodic measure on V whose topological support is all of V. Further suppose that the Atkinson-type Hypothesis Ž3.3. is ¨ alid. Let I : R be an open inter¨ al. Then Eqs. Ž3.7.v , l
620
JOHNSON AND NERURKAR
ha¨ e an exponential dichotomy o¨ er V for all l g I if and only if the rotation number l ª a Ž l. is constant on I. The theorem is proved in w17x. Since a is continuous and monotone, the assumption of constancy of a on I is equivalent to equality of the values of a at the end points of I. We finish Section 3 by returning for a moment to the Atkinson-type condition Ž3.3.. We show that it is actually equivalent to a controllability condition. PROPOSITION 3.6.
Hypothesis Ž3.3. holds if and only if the control process z9 s yf * Ž t t Ž v . . z q g Ž t t Ž v . . u
Ž 3.9. v
is Ž controllable and hence. uniformly controllable on each minimal subset M : V. Proof. Consider first the ‘‘only if’’ implication. Write out the condition in Ž3.3. and note that F*Ž v 0 , t .y1 is the fundamental matrix solution of the adjoint system z9 s yf * Ž t t Ž v . . z. Using symmetry of g and Lemma Ž2.4., we see that Ž3.9.v is uniformly controllable on each minimal subset M : V. The ‘‘if’’ implication follows quickly from the non-singularity criterion of the controllability matrix. We see that there is an interesting interplay between the notion of uniform controllability, the Atkinson condition, and exponential dichotomy. Palmer w25x has considered the relation between controllability, bounded input-bounded output stability and the existence of exponential dichotomy.
4. SOLUTION OF THE RANDOM FEEDBACK STABILIZATION PROBLEM We begin by introducing the basic controllability assumptions which will imply that the random regulator problem and the random feedback control problem are solvable. These assumptions are natural variants of the classical conditions found in w7, 14, 20x. Define C Ž v . s QŽ v .
'
Ž v g V.
621
RANDOM FEEDBACK STABILIZATION PROBLEM
to be the unique positive semidefinite square root of the matrix function QŽ v .. HYPOTHESES 4.1. Suppose that, for each minimal subset M : V, there exists at least one v 0 g M such that: Ža. Žb.
x9 s yA*Žt t Ž v 0 .. x q C Žt t Ž v 0 .. u1 is controllable; y9 s AŽt t Ž v 0 .. y q B Žt t Ž v 0 .. u 2 is controllable.
For later convenience we write u1 , u 2 in place of u in Hypotheses 4.1. Note that Hypothesis 4.1Ža. is automatically satisfied if QŽ v . is positive definite for all v g V. We first show that Hypotheses 4.1 implies the Atkinson-Hypothesis 3.3 for appropriate g . Define
g Ž v. s
ž
QŽ v .
0 y1
BŽ v . R
0
Ž v . B* Ž v .
/
so that g is continuous, symmetric and positive semi-definite. Consider the family of control processes Ž3.9.v encountered at end of Section 3; explicitly, d
x s dt y
ž/
ž
yA* Ž t t Ž v . .
Q Žt t Ž v . .
y1
Ž BR B* . Ž t t Ž v . .
ql
ž
AŽt t Ž v . .
Q Žt t Ž v . .
x y
/ž /
0 y1
Ž BR B* . Ž t t Ž v .
0
v1 v2 .
./ž /
Ž 4.1. v
LEMMA 4.2. Hypotheses Ž4.1. implies that, if M : V is minimal, then the control systems Ž4.1.v are uniformly controllable for each v g M. COROLLARY 4.3. Eqs. Ž3.7.v , l with
If Hypotheses 4.1 holds, then Hypothesis 3.3 is ¨ alid for
fŽ v. s
ž
AŽ v . yQ Ž v .
yBRy1 B* Ž v . yA* Ž v .
/
and
g Ž v. s
ž
QŽ v . 0
0 y1
BR
B* Ž v .
/
.
Corollary 4.3 follows from Proposition 3.6 and Lemma 4.2. It will allow us to apply Theorem 3.5 to Eqs. Ž3.7.v , l .
622
JOHNSON AND NERURKAR
Proof of Lemma 4.2. Let C Ž v , t . be the fundamental matrix solution of x9 s yA*Žt t Ž v .. x, Ž v g V .. The nonsingularity Žand hence the positive definiteness . of the controllability matrix yields positive numbers T ) 0, d ) 0 such that T
H0
C Žt t Ž v 0 . . C Ž v 0 , t . ¨
2
dt G d 5 ¨ 5 2
for all ¨ g R n. Using symmetry and semi-definiteness of C together with a compactness argument, one can find d˜ ) 0 such that T
H0
Q Žt t Ž v 0 . . C Ž v 0 , t . ¨
2
dt G d˜ 5 ¨ 5 2
for all ¨ g R n. By Lemma 2.4, we see that the control systems x9 s yA* Ž t t Ž v . . x q Q Ž t t Ž v . . u1
Ž 4.2. v
are uniformly controllable for all v g M. Using strict positive definiteness of R and an argument like that just given, one shows that, if Ž4.1.Žb. holds, then the control systems y9 s A Ž t t Ž v . . y q B Ž t t Ž v . . Ry1 Ž t t Ž v . . B* Ž t t Ž v . . u 2
Ž 4.3. v
are uniformly controllable over M. Let us return again to the point v 0 g M. Let Ž xy 00 . g R 2 n. There exists a number T ) 0 and controls u1 , u 2 :w0, T x ª R m with the following properties: Ži. the solution x Ž t . of Ž4.2.v with x Ž0. s x 0 satisfies x ŽT . s 0; 0 Žii. the solution y Ž t . of Ž4.3.v with y Ž0. s y 0 satisfies y ŽT . s 0. 0 Define w 1 Ž t . s u1 Ž t . y x Ž t . w2 Ž t . s u2 Ž t . y y Ž t . for 0 F t F T. The control Ž ww12 . steers Ž xy 00 . to zero in time T for Eq. Ž4.1.v 0 . This shows that the process Ž4.1.v 0 is controllable. By Lemma 2.4, we obtain the statement of Lemma 4.2. We are now in a position to apply Theorem 3.5. Indeed, we now consider the rotation number of Eqs. Ž3.7.v , l with f, g as above. LEMMA 4.4. Assume that Hypotheses 4.1 holds. Let I s Žy1r2, 1r2.. If l g I and M : V is a minimal set, then the rotation number of Eqs. Ž3.7.v , l
623
RANDOM FEEDBACK STABILIZATION PROBLEM
equals zero for e¨ ery ergodic measure m whose topological support is contained in M. Proof. Let v g M. Consider the boundary value problem d
x s dt y
ž/ ž
yBRy1 B* yA*
A yQ
ql J
ž
Q
0
0
y1
/Ž
tt Ž v . .
Ž BR B* .
/
Žt t Ž v . .
x y
ž/
Ž 4.4. v
x Ž 0. s x Ž T . s 0 where T is some positive number. We show that this boundary value problem has only the trivial solution if T is sufficiently large. To do so, let Ž xy ŽŽ tt .. . be a solution of Ž4.4.v . Then 0 s² x Ž T . , y Ž T .: y² x Ž 0 . , y Ž 0 .: s T
s
H0
s
H0
¦
d
dt
T
;¦
xŽ t. , yŽ t. q xŽ t. ,
T
H0
d dt
d dt
² x Ž t . , y Ž t .: dt
;
yŽ t.
dt
² Ax y Ž 1 q l . BRy1 B*y, y: q² x, Ž l y 1. Qx y A*y: T
Hence, 0 s Ž l y 1.
H0
5 Cx 5 2 dt y Ž1 q l.
T
H0 5 R
T
T
Ž l y 1 . H 5 Cx 5 2 dt s Ž 1 q l . H 0
yŽ1r2.
dt.
B*y 5 2 dt, that is,
RyŽ1r2. B*y
2
dt.
0
Since l g Žy1r2, 1r2., we must have C Žt t Ž v .. x Ž t . s 0 s B*Žt t Ž v .. y Ž t . for all 0 F t F T. We conclude that yX Ž t . s yA*Žt t Ž v .. y Ž t .. Hence using uniform controllability of y9 s Ay q Bu Žsee Lemma Ž2.4. and the relation T
H0
5 B*y 5 2 dt s 0,
we see that y Ž t . s 0 on w0, T x if T is sufficiently large. Since x Ž0. s 0, we have by uniqueness that x Ž t . is identically zero on w0, T x as well. Now we can show that a Ž l, m . s 0 if l g I. Consider the Lagrange plane h 0 s Ž 0y . ¬ y g R n4 used in defining the Maslov cycle C . Note that, if dimŽ FlŽ v , T .h 0 l h 0 . G 1 for some T ) 0 and some l g I, then there
624
JOHNSON AND NERURKAR
exists 0 / y 0 g R n such that 0 0 Fl Ž v , T . y s . yŽT . 0
ž / ž
/
But then y ŽT . s 0 by the preceding paragraph, a contradiction if T is large. Hence for all l g I and all large T, the Lagrange plane FlŽ v , T .h 0 does not lie on the Maslov cycle. By Ž3.8., we must have a Ž l. s a Ž l, m . s 0 for all l g I. LEMMA 4.5. Equations Ž3.5.v admit an exponential dichotomy o¨ er V. In other words, Eqs. Ž3.7.v , l admit an exponential dichotomy o¨ er V when l s 0. Proof. If M is a minimal subset of V, then Eqs. Ž3.5.v admit an ED over M by Lemma 4.4. By Theorem Ž3.1. of w15x, the dimensions of the stable and unstable bundles V s, V u over M are each equal to n. That is, these dimensions are independent of the minimal subset M : V. Next let v g V and suppose that Ž xy ŽŽ tt .. . is a solution of Eq. Ž3.5.v which is bounded on all of R. We claim that x Ž t . and y Ž t . are identically zero. To prove this, note first that there are sequences t n ª `, sn ª y` such that Ž xy ŽŽ tt nn .. . ª Ž 00 . and Ž xy ŽŽ ssnn .. . ª Ž 00 .. For if, for example, there was no such sequence sn4 , then it is easily seen that, for each v in the a-limit set of v , the equation Ž3.5.v would admit a non-trivial solution bounded on all of R. But the a-limit set of v contains a minimal set M. Since Eqs. Ž3.5.v have ED on M, there can be no non-trivial bounded solution of Ž3.5.v if v g M. This contradiction proves that sn4 exists, and in a similar way one proves that t n4 exists. Now, d
tn
² x Ž t n . , y Ž t n .: y ² x Ž sn . , y Ž sn .: s H
sn
sy
dt tn
Hs
² x Ž t . , y Ž t .: dt 5 RyŽ1r2. B*y 5 2 q 5 Cx 5 2 dt.
n
We conclude that B*Žt t Ž v .. y Ž t . s 0 s C Žt t Ž v .. x Ž t . for all t g R. But now using the controllability condition ŽHypothesis 4.1a, b. together with Proposition 2.5 and arguing as in the proof of Lemma Ž4.4., we see that x Ž t . and y Ž t . are identically zero. We now apply a theorem of w27x, the proof of which is based on ideas of Conley w8x. This theorem states that if the dimensions of V s, V u are independent of M and if no equation Ž3.5.v admits a non-trivial bounded solution, then Eqs. Ž3.5.v admit an ED over V. This completes the proof of Lemma 4.5.
RANDOM FEEDBACK STABILIZATION PROBLEM
625
We need one more bit of information before turning to the solution of the linear regulator problem and the feedback control problem. Let V s : R 2 n = V be the stable bundle of Eqs. Ž3.5.v . LEMMA 4.6. Let v g V and let Ž xy . g V s Ž v . s ŽR 2 n = v 4. l V s. Then ² x, y : / 0 unless x s y s 0. Proof. We essentially repeat part of the proof of Lemma 4.5. Suppose that there exists Ž xy 00 . g V s Ž v . with ² x 0 , y 0 : s 0. Let Ž xy ŽŽ tt .. . be the solution of Ž3.5.v with initial value Ž xy 00 .. Then d
T
² x Ž T . , y Ž T .: s H
dt
0
sy
T
H0
² x Ž t . , y Ž t .: dt
5 RyŽ1r2. B*y 5 2 dt y
T
H0
5 Cx 5 2 dt.
Since Ž xy 00 . g V s Ž v . we have x ŽT . ª 0, y ŽT . ª 0 as T ª `. Arguing as in the proof of Lemma 4.5, we now show that x Ž t . ' 0 ' y Ž t ., thus x 0 s y 0 s 0. This completes the proof. By a result of w15x, V s Ž v . is a Lagrange plane for each v g V and Lemma 4.6 implies that, for each v g V, the projection of V s Ž v . on to h1 s Ž 0x . ¬ x g R n4 is on to. Thus V s Ž v . has a basis of the form
ž
ˆe1 ˆe n , ??? ; m1 Ž v . mn Ž v .
/ ž
/
i.e., can be parametrized by the following Žsymmetric. matrix: m Ž v . s Ž m1 Ž v . , . . . , m n Ž v . . Žsee Section 3.. We see that, if Ž xy 00 . g V s Ž v . and Ž xy ŽŽ tt .. . is the corresponding solution of Ž3.5.v , then y Ž t . s mŽt t Ž v .. x Ž t .. The continuity of m in v follows from the continuous variation of the fibers V s Ž v .. By invariance of V s under the flow ˆ t on R 2 n = V Žsee Definition 3.2., we see that the function t ª m t Ž v . satisfies the Riccati equation m9 s yA*m y mA q mBRy1 B*m y Q. We are ready to solve the linear regulator problem. Given any control uŽ t ., let x Ž t . be the corresponding solution of x9 s A Ž t t Ž v . . x q B Ž t t Ž v . . u x Ž 0. s x 0 .
626
JOHNSON AND NERURKAR
The following formula can be verified using the Riccati equation for m: d dt
² m Ž t t Ž v . . x Ž t . , x Ž t .: s R1r2 Ž t t Ž v . . u Ž t . q Ry1 B*m Ž t t Ž v . . x Ž t .
2
˜v Ž x Ž t . , u Ž t . . . y 2L
Ž 4.5.
Thus Lv is minimized by choosing u Ž t . s yRy1 Ž t t Ž v . . B* Ž t t Ž v . . m Ž t t Ž v . . x Ž t . . The corresponding x Ž t . decreases exponentially as t ª ` and hence uŽ t . is square-integrable on w0, `.. The minimum value of the functional Lv is `
H0
˜v Ž x Ž t . , u Ž t . . dt s y 2L
`
H0
d dt
² m Ž t t Ž v . . x Ž t . , x Ž t .: dt
s ² m Ž v . x 0 , x 0: . This shows that uŽ t . is the unique control solving the random regulator problem. Now we turn to the random feedback stabilization problem. Define K Ž v . s yRy1 Ž v . B* Ž v . m Ž v .
Ž v g V. .
Clearly K is continuous in v . Let Ž m Žt t ŽxvŽ t... x Ž t . . be the unique solution of Ž3.5.v lying in the stable bundle V s which satisfies x Ž0. s x 0 . We see from Ž3.5.v that x Ž t . satisfies the closed-loop system x9 s A Ž t t Ž v . . q B Ž t t Ž v . . K Ž t t Ž v . . x.
Ž 4.6.
Since x Ž t . ª 0 exponentially as t ª `, the function K does indeed stabilize Ž1.1.v for each v g V. This solves the feedback stabilization problem. At this point we note that robustness of K and smoothness of K with respect to parameters follow from the corresponding properties for the bundle V s and V u. For a very general smoothness result, see Yi w28x. As a sample of the type of robustness result that can be obtained, we explain briefly the implications of a result of Coppel w9, p. 34x. Let A 0 and B0 be constant matrices for which the controllability assumptions Ž4.1. are satisfied. For convenience we choose C s Id Žthe identity. in this discussion. Let K 0 be the corresponding feedback matrix. p Ž We view A 0 Žrespectively B0 . as an element of L loc R, M Ž n, n.. Žrespec-
627
RANDOM FEEDBACK STABILIZATION PROBLEM
p Ž tively L loc R, M Ž n, m... where p ) 1. There exists numbers d 0 ) 0, L ) 0 p Ž with the following property. Let A 1 g L loc R, M Ž n, n .., B1 g p Ž L loc R, M Ž n, m.. be functions such that
Sup
tq1
H tgR t
A1 Ž s .
p
ds F d F d 0
and
Sup
tq1
H tgR t
B1 Ž s .
p
ds F d F d 0 .
Put aŽ t . s A 0 q A1Ž t ., bŽ t . s B0 q B1Ž t .. Then there exists a feedback matrix k Ž t . such that x9 s w aŽ t . q bŽ t . k Ž t .x x is exponentially stable and such that k Ž t . y K 0 - Ld
Ž t g R. .
Note that, if QŽ v . is strictly positive definite for all v , then Hypothesis Ž4.1.Ža. is automatically satisfied. Thus the possibility of solving the feedback stabilization problem really depends only on the verification of Hypothesis Ž4.1.Žb., i.e., on controllability of Ž1.1.v . Finally, we consider the question of pole relocation. For autonomous systems x9 s ax q bu, this refers to the possibility of choosing Q, R in such a way that the resulting feedback K has the property that the eigenvalues of a q bK take on prescribed values in the left half-plane. We wish to consider the random version of pole relocation. Recall that a real number b is a Lyapuno¨ exponent of Ž4.6. if there is a non-zero vector x 0 g R n such that, if x Ž t . is the solution of Ž4.6. with x Ž0. s x 0 , then 1
b s lim
tª`
t
log x Ž t . .
We are going to show that K Ž v . can be chosen so that all Lyapunov exponents of Ž4.6. lie as far to the left of the origin as desired. To do so, choose d ) 0 and consider the random regulator problem with cost functional Lv Ž x, u . s
`
H0
Ž d 5 x 5 2 q 5 u 5 2 . dt.
Clearly Hypotheses 4.1 are valid. The m-function is positive definite because if ² mŽ v . x 0 , x 0 : s 0 for some v g V and 0 / x 0 g R n, then by Ž4.5. y² m Ž t T Ž v . . x Ž T . , x Ž T .: s
T
H0
d 5 x 5 2 q 5 B*mx 5 2 dt,
where x Ž t . is the solution of Ž4.6. with x Ž0. s x 0 . The left-hand side tends to zero as T ª `, hence x Ž t . s 0 for al t, hence x 0 s 0.
628
JOHNSON AND NERURKAR
Now let « 1 and « 2 be respectively the minimum and the maximum of the set of numbers ² mŽ v ., x, x : ¬ v g V, 5 x 5 s 14 . Then 0 - « 1 5 x 5 2 F ² m Ž v . x, x: F « 2 5 x 5 2
forall x g R n .
Thus
d
d
² m Ž t t Ž v . . x Ž t . , x Ž t .: F yd 5 x 5 2 F y ² m Ž t t Ž v . . x Ž t . , x Ž t .: . dt « 2
Therefore ² mŽt t Ž v .. x Ž t ., x Ž t .: F MeyŽ d t r « 2 . where M is some positive constant. Hence, xŽ t.
2
F
1
«1
² m Ž t t Ž v . . x Ž t . , x Ž t .: F
M
«1
eyŽ d t r « 2 . .
Thus every solution of the feedback system decays exponentially as t ª ` with rate at least Žydr2 « 2 ., Ž v g V .. Thus the Lyapunov exponents lie always to the left of Žydr2 « 2 ., uniformly in v g V. This last assertion is not always true if one has Žunbounded. stochastic coefficients. To summarize: we used the rotation number to show that Eq. Ž3.5.v has ED. This allowed us to solve the random feedback stabilization problem and obtain robustness properties, smoothness properties, preservation of recurrence of the solution, and the random pole relocation result.
REFERENCES 1. V. Arnold, On a characteristic class entering in a quantum condition, Funct. Anal. Appl. 1 Ž1969., 1]14. 2. B. Anderson and J. Moore, Detectability and stabilizability of time-varying discrete time linear systems, SIAM J. Control Optim. 19, No. 1 Ž1981., 20]32. 3. Z. Artstein, Uniform controllability via limiting systems, Appl. Math. Optim. 9 Ž1982., 111]131. 4. F. Atkinson, ‘‘Discrete and Continuous Boundary Value Problems,’’ Academic Press, New YorkrLondon, 1964. 5. P. Bougerol, Kalman filtering with random coefficients and contraction, SIAM J. Control Optim. 31, No. 4 Ž1993., 942]959. 6. P. Bougerol, Filtre de Kalman Bucy et exposants de Lyapunov, preprint Ž1993.. 7. V. Cheng, A direct way to stabilize continuous time and discrete time linear time varying systems, IEEE Trans. Automat. Control AC-24 Ž1979., 641]643. 8. C. Conley, ‘‘Isolated Invariant Sets and the Morse Index,’’ CBMS Regional Conference Series in Mathematics, Vol. 38, SIAM, Philadelphia, 1978. 9. W. Coppel, ‘‘Dichotomies in Stability Theory,’’ Lecture Notes in Mathematics, Vol. 629, Springer-Verlag, Heidelberg, 1978. 10. G. DaPrato, Synthesis of optimal control for an infinite dimensional periodic problem, SIAM J. Control Optim. 25 Ž1987., 706]714.
RANDOM FEEDBACK STABILIZATION PROBLEM
629
11. G. DaPrato and A. Ichikawa, Quadratic control for linear time varying systems, SIAM J. Control Optim. 28, No. 2 Ž1990., 359]381. 12. I. Gibson, The Riccati integral equation for optimal control problem on Hilbert spaces, SIAM J. Control Optim. 17 Ž1979., 537]565. 13. R. Hermann, Cartanian Geometry, Nonlinear Waves and Control Theory, Part A, Interdisciplinary Mathematics Series, Vol. XX, Math. Sci. Press., Brookline, MA, 1980. 14. M. Ikeda, H. Maeda, and S. Kodama, Stabilization of linear systems, SIAM J. Control 10 Ž1972., 716]729. 15. R. Johnson, m-Functions and Floquet exponents for linear differential systems, Ann. Mat. Pura Appl. 147 Ž1987., 211]248. 16. R. Johnson and M. Nerurkar, On null controllability of linear systems with recurrent coefficients and constrained controls, J. Dynamics Differential Equations 4, No. 2 Ž1992., 259]273. 17. R. Johnson and M. Nerurkar, Exponential dichotomy and rotation number for linear Hamiltonian systems, J. Differential Equations 108 Ž1994., 201]216. 18. R. Johnson and M. Nerurkar, Null controllability of linear systems with positive Lyapunov exponents, in ‘‘Differential Equations, Dynamical Systems and Control Science;’’ ŽK. Elworthy and W. Everitt, Eds.., pp. 605]621, Dekker, New York, 1994. 19. R. Johnson and M. Nerurkar, Feedback control for linear chaotic systems, in ‘‘Proceedings of the 2nd IFAC Workshop on Systems Structures and Control, Prague, Sept. 1992,’’ pp. 272]273. 20. R. Kaman, Contributions to the theory of optimal control, Bull. Soc. Math. Mex. 5 Ž1960., 102]119. 21. S. Kultyshev and E. Tonkov, Controllability of non-stationary linear systems, Differential Equations Ž Engl. transl.. 11 Ž1975., 1206]1216. 22. W. Kwon and A. Pearson, A modified quadratic cost problem and feedback stabilization of linear systems, IEEE Trans. Automat. Control AC-22 Ž1977., 838]842. 23. E. Lee and L. Markus, ‘‘Foundations of Optimal Control Theory,’’ Wiley, New York, 1967. 24. V. Nemytskii and V. Stepanov, ‘‘Qualitative Theory of Ordinary Differential Equations,’’ Princeton Univ. Press, Princeton, NJ, 1960. 25. K. Palmer, Two linear systems criteria for exponential dichotomy, Ann. Mat. Pura Appl. 124 Ž1980.. 199]216. 26. L. Pontriyagin, V. Baltyanskii, R. Gamkrelidze, and E. Mishchenko, ‘‘The Mathematical Theory of Optimal Processes,’’ Wiley]Interscience, New York, 1962. 27. R. Sacker and G. Sell, A spectral theory for linear differential systems, J. Differential Equations 27 Ž1978., 320]358. 28. Y.-F. Yi, Smoothness of integral manifolds, J. Differential Equations 102 Ž1993., 153]187.
197, 608]629 Ž1996.
0042
Stabilization and Random Linear Regulator Problem for Random Linear Control Processes Russell Johnson* Departimento di Sistemi e Informatica, Uni¨ ersita ` di Firenze, Florence 50139, Italy
and Mahesh Nerurkar † Department of Mathematics, Rutgers Uni¨ ersity, Camden, New Jersey 08102 Submitted by U. Kirchgraber Received October 19, 1994
We study in a dynamical system context the random feedback stabilization problem for linear, random control processes. We are led to study the random linear regulator problem, which we solve by considering the spectral theory of linear time-dependent Hamiltonian systems. This is done with the aid of the concepts of exponential dichotomy and rotation number. Q 1996 Academic Press, Inc.
1. INTRODUCTION The purpose of this paper is to study in a dynamical systems context the feedback stabilization problem for random linear control processes on the semi-infinite interval. By ‘‘random,’’ we mean processes x9 s a Ž t . x q b Ž t . u
Ž x g Rn, u g Rm .
Ž 1.1.
for which aŽ?. and bŽ?. may exhibit the entire range of behaviour from periodicity to Žbounded. i.i.d.’s Ži.e., independent identically distributed processes.. Though this usage is not entirely standard, it is motivated by the meaning of the word ‘‘random’’ in the theory of the random Schro¨ * The research of this author is supported by M.U.R.S.T. 40% and M.V.R.S.T. 60%. † The research of this author is supported by a Research Council Grant from Rutgers University. 608 0022-247Xr96 $18.00 Copyright Q 1996 by Academic Press, Inc. All rights of reproduction in any form reserved.
609
RANDOM FEEDBACK STABILIZATION PROBLEM
dinger operator. Stochastic methods, powerful as they are, can only be applied when the coefficients exhibit strong independence properties. Our methods apply in a vast range of circumstances when independence is not present. In a standard way, the discussion of this problem will lead us to consider the Žrandom. linear regulator problem. This problem is usually solved by constructing a solution of a Riccati equation. Our point of view is that the solution of the Riccati equation is best obtained by showing that the associated linear Hamiltonian system has an ED Žexponential dichotomy.. This implies very strong robustness properties and very good smoothness properties with respect to parameters, of the solution as well as the feedback stabilizer. We also obtain immediately the important property of ‘‘preservation of recurrence’’ of which more below. Here we note that if aŽ t . and bŽ t . exhibit chaotic time dependence, then the feedback control k Ž t . is ‘‘no more chaotic’’ than a and b w19x. We prove the existence of an exponential dichotomy by using the rotation number of the linear Hamiltonian system. In fact, we show that under a mild controllability hypothesis on Ž1.1., the rotation number of the linear Hamiltonian system is constant on an interval. This implies the existence of ED, which then by standard results gives the robustness, smoothness, and preservation of recurrence properties mentioned above. We shall also obtain a proper random analog of the pole relocation theorem. We finish the introduction by describing more precisely our formulation of the random feedback stabilization problem. Let M Ž n, m. be the set of n = m real matrices. Fix a number 1 F p F `, and let
½
C s a: R ª M Ž n, n . ¬ Sup
tq1
H tgR t
aŽ s .
p
5
ds - ` .
If p s 1, suppose in addition that lim
tq «
H ǻ0 t
a Ž s . ds s 0
uniformly in t g R. Give C the distribution topology; that is, a n ª a in C if and only if
HR a Ž t . w Ž t . dt ª HR aŽ t . w Ž t . dt n
for each smooth function w : R ª R n of compact support. Next define D s b: R ª M Ž n, m. ¬ b is uniformly bounded and uniformly continuous4 . Give D the topology of uniform convergence on compact sets. Both C and
610
JOHNSON AND NERURKAR
D support the translation flow t t ¬ t g R4 , where t t is the translation
Ž t t a. Ž s . s aŽ t q s . , Ž t t b . Ž s . s b Ž t q s .
Ž t , s g R. .
We will choose the coefficient functions aŽ t ., bŽ t . in Ž1.1. from the sets C, D, respectively. In order to use techniques of dynamical systems, however, we will replace Ž1.1. by a family of control processes having the same form. Thus let V be a compact, translation-invariant subset of C = D Žthus in particular V is metrizable.. The pair Ž V, t t ¬ t g R4. defines a flow on V in the sense that Ži. t 0 : V ª V is the identity map; Žii. t t (ts s t tqs Ž t, s g R.; Žiii. the map Ž v , t . ª t t Ž v . from V = R to V is jointly continuous. Each element of V gives rise to a control process having the form of Ž1.1.: x9 s A Ž t t Ž v . . x q B Ž t t Ž v . . u
Ž v g V.
Ž 1.1. v
We must explain the notation. If v s Ž a, b . g V, define B Ž v . s bŽ0.; then it is easily seen that B: V ª M Ž n, m. is continuous and B Žt t Ž v .. s bŽ t ., Ž t g R.. On the other hand, there may not be a continuous function A: V ª M Ž n, n. such that AŽt t Ž v .. s aŽ t . for all t g R. Therefore we regard the expression ‘‘t ª AŽt t Ž v ..’’ as a notational device for expressing the function aŽ t .. Thus t ª AŽt t Ž v .. represents the projection of v on to its first coordinate. This framework includes all time-varying systems with bounded coefficients, from periodic and almost periodic to highly stochastic systems with positive entropy, and also the intermediate case of zero entropy with or without mixing conditions. We use the word ‘‘random’’ to refer to this very wide class of coefficients. The original control process Ž1.1. generates a family Ž1.1.v if a g C and b g D. Namely, let V s clsŽt t Ž a., t t Ž b .. ¬ t g R4 : C = D. If v 0 s Ž a, b ., then Eq. Ž1.1.v 0 coincides with Ž1.1.. Our point of view is quite general in that we do not require the existence of a point v 0 g V whose orbit t t Ž v 0 . ¬ t g R4 is dense in V. The random stabilization problem is that of finding a Žat least. continuous map K:V ª M Ž m, n. such that for each v g V the origin is an asymptotically stable fixed point of Ž1.1.v with feedback rule uv Ž t . s K Ž t t Ž v . . x Ž t . . We remark that even though results of w7, 14, 20x ensure existence of feedback matrix K v Ž t . for each v , it is not clear that the family K v 4 ‘‘lifts
RANDOM FEEDBACK STABILIZATION PROBLEM
611
consistently’’ to V; i.e., it is not clear that there exists K: V ª M Ž m, n. such that K v Ž t . s K Žt t Ž v ... This problem is non-trivial in general; this is what we mean by ‘‘preservation of recurrence properties.’’ Our techniques immediately yield a consistent lifting. The strength of our methods is also illustrated when one considers robustness and smoothness of the feedback matrix K. For example if V is a manifold and A, B are C r smooth functions then K is C r as well. Robustness refers to variation of K when V itself is varied Že.g. in some function space.; results of Sacker]Sell w27x can be used to show that K is ‘‘continuous’’ under such variation in very general circumstances. Finally, we also mention that the exponential dichotomy property is very much stronger than that of positive Lyapunov exponents and in this sense our results are stronger than those of Bougerol w5, 6x when the coefficients satisfy the boundedness criterion mentioned above. The paper is organized as follows. In Section 2, we discuss quite weak conditions which imply uniform controllability. In Section 3, we briefly discuss the random linear regulator problem, introduce the concept of ED, and discuss the rotation number of linear Hamiltonian systems. In the final section, we solve the random feedback stabilization problem.
2. A CONDITION IMPLYING UNIFORM CONTROLLABILITY We introduce some notation. Consider the family of control processes x9 s A Ž t t Ž v . . x q B Ž t t Ž v . . u
Ž v g V. ,
Ž 1.1. v
where the flow Ž V, t t 4. is as described in Section 1. Let X Ž v , t . be the fundamental matrix solution of the linear system x9 s A Ž t t Ž v . . x
Ž v g V.
Ž 2.1. v
satisfying X Ž v , 0. s I}the n = n identity matrix. Then X: V = R ª GLŽ n, R. is jointly continuous and satisfies the following ‘‘cocycle identity’’: X Ž v , t q s . s X Žt t Ž v . , s . X Ž v , t .
Ž v g V , t , s g R. .
We will also consider the adjoint system x9 s yA* Ž t t Ž v . . x
Ž v g V.
Ž 2.2. v
Žhere * denotes the transpose.; the fundamental matrix solution of this system is ZŽ v , t . s X *Ž v , t .y1 .
612
JOHNSON AND NERURKAR
Let v g V and t 1 - t 2 g R. Define the following n = n matrix-valued function of v : Ww t 1 , t 2 x Ž v . s
t2
Ht
XŽ v, t.
y1
B Ž t t Ž v . . B* Ž t t Ž v . . X * Ž v , t .
y1
dt.
1
We will write WT Ž v . for Ww0, T xŽ v .. LEMMA 2.1. With the abo¨ e notation, the following identity holds for all t, T g R, and v g V: X Ž v , t . Ww t , tqT x Ž v . X * Ž v , t . s WT Ž t t Ž v . . . Proof. This is a simple calculation using the cocycle identity. DEFINITION 2.2. Fix v g V. The process Ž1.1.v is said to be controllable Žmore precisely null controllable at v . if and only if given x 0 g R n, there exists a locally integrable function uŽ t . and a number T ) 0 such that the solution x Ž t . of Ž1.1.v with u s uŽ t . and x Ž0. s x 0 satisfies x ŽT . s 0. For each v g V, it is well known that controllability of the process Ž1.1.v is equivalent to the non-singularity of WT Ž v . for some T ) 0. DEFINITION 2.3. Fix v g V. The process Ž1.1.v is said to be uniformly controllable if and only if there exist constants a ) 0, T ) 0 such that 0 - a I - WT Ž t t Ž v . . ,
; t g R.
We prove that if the flow Ž V, t t 4. is minimal in the sense that if the orbit t t Ž v . ¬ t g R4 is dense in V for all v g V, then uniform controllability on V is equivalent to simple controllability along a single orbit. This is a generalization of a theorem of Artstein w3x and is the first step towards our sufficiency condition for uniform controllability ŽProposition 2.5.. The sufficient condition does not assume minimality but rather requires only controllability along at least one orbit in each minimal subset of V. LEMMA 2.4. Suppose Ž V, t t 4. is minimal and suppose that for some v 0 g V, the process Ž1.1.v 0 is controllable. Then e¨ ery process Ž1.1.v is uniformly controllable. In fact, there exists constants a ) 0, T ) 0, which are independent of v , such that 0 - a I - WT Ž v .
Ž v g V. .
Proof. The proof is similar to that of Theorem Ž2.10. in w16x. Since the process Ž1.1.v 0 is controllable, there exists S ) 0 such that WS Ž v 0 . is
613
RANDOM FEEDBACK STABILIZATION PROBLEM
non-singular. Choose d ) 0 such that d - inf²WS Ž v 0 . x, x : ¬ 5 x 5 s 14 . Since the map Ž v , x . ª ²WS Ž v . x, x : is continuous and x ¬ 5 x 5 s 14 is compact, there exists a compact neighbourhood N of v 0 in V such that
d 2
- inf ² WS Ž v . x, x: ¬ 5 x 5 s 1, v g N 4 .
Ž 2.3.
Let us write temporarily t Ž v , t . for t t Ž v ., Ž v g V, t g R.. By minimality of V, we can find L ) 0 and a sequence Tn ª y` such that for each n G 1: Tnq 1 - Tn ;
Ž i.
Ž 2.4.
< Tnq 1 y Tn < F L;
Ž ii .
Ž 2.5.
v n s t Ž v 0 , Tn . g N.
Ž iii .
Ž 2.6.
By Ž2.3., we have for n G 1: 0-
d 2
- inf ² WT Ž t Ž v 0 , Tn . . x, x: ¬ 5 x 5 s 1 4 .
Ž 2.7.
Let t - 0. We can write t s Tn q g for some n G 1, where g g wyL, 0x. Then for any x with 5 x 5 s 1, we have
² WSq LŽ t t Ž v 0 . . x, x: s² WSqLŽ t Ž v 0 , Tn q g . . x, x: s² X Ž v n , g . Ww g , gqSqLx Ž v n . X * Ž v n , g . x, x: s² Ww g , gqSqLx Ž v n . yn , yn: , where we have written yn s X *Ž v n , g . x. Continuing, we have
² Wwg , gqSqLx Ž vn . yn , yn: G² WgqSqLŽ vn . yn , yn: , since w0, g q S q L x ; w g , g q S q L x. Thus,
² Wwg , gqSqLx Ž vn . yn , yn: G² WS Ž v n . yn , yn: G
d 2
Ž because g q L ) 0 .
5 yn 5 2 .
Letting h s min5 X *Ž v , g . x 5 2 ¬ yL F g F 0, 5 x 5 s 1, v g N 4 , we see that inf ² WSq L Ž t t Ž v 0 . . x, x: ¬ 5 x 5 s 1, t F 0 4 G
hd 2
G 0.
614
JOHNSON AND NERURKAR
Since the semi-orbit Žt t Ž v 0 . ¬ t F 04 is dense in V, we have WTqLŽ v . G hdr2 for all v g V. This shows that the statement of Lemma 2.4 holds with T s S q L. The lemma applies in particular when Ž1.1. has almost periodic coefficients aŽ t ., bŽ t .. This is due to the fact that in this case the flow Ž V, t t 4. is minimal. If Ž V, t t 4. is not minimal, then Lemma 2.4 is false, as simple examples show w3x. We now prove the main result. PROPOSITION 2.5. Suppose that, for each minimal subset M : V, there exists at least one v 0 g M such that the process Ž1.1.v 0 is controllable. Then each process Ž1.1.v is uniformly controllable and the constants T ) 0, a ) 0 can be chosen independent of v g V. In fact, WT Ž v . G a I for each v g V. Proof. Suppose for contradiction that there exist sequences v j g V, Tj ª `, a j ª 0 and x j g R n such that 5 x j 5 s 1 and 5 WT jŽ v j . x j 5 F a j . We can assume that v j ª v , x j ª x where 5 x 5 s 1. Then clearly WT Ž v . x s 0 for all T ) 0. Now however it is easily seen that, given « ) 0, there is a minimal subset M : V, a point v 0 g M and a time t ) 0 such that dŽt t Ž v ., v 0 . « . Here d is some metric on V. By continuity in v of WT Ž v . and using Lemma 2.1, we see that there exists T# ) 0 such that 5 WT#Ž v . x 5 ) 0. This is a contradiction; the proposition is proved.
3. THE RANDOM LINEAR REGULATOR PROBLEM We give a brief introduction to the random linear regulator problem and the standard approach to solving it. In doing so, we shall have occasion to recall the ‘‘Floquet Theory’’ and, in particular, the notion of rotation number for linear Hamiltonian systems. Consider once again the family of control processes x9 s A Ž t t Ž v . . x q B Ž t t Ž v . . u
Ž v g V. ,
Ž 1.1. v
where Ž V, t t 4. is a flow as discussed in Section 1. Introduce continuous matrix-valued functions Q: V ª M Ž n, n. and R: V ª M Ž m, m. with the following properties: Q* Ž v . s Q Ž v . G 0
Ž v g V.
Ž 3.1. a
R* Ž v . s R Ž v . ) 0
Ž v g V. .
Ž 3.1. b
In particular, R is assumed to be strictly positive-definite. Define the
RANDOM FEEDBACK STABILIZATION PROBLEM
615
Lagrangian
˜v Ž x, u . s² x, Q Ž v . x: q² u, R Ž v . u: , 2L and set Lv Ž x, u . s
`
H0
˜v Ž x Ž t . , u Ž t . . dt. 2L
The linear regulator problem is the following. Fix an initial vector x 0 g R n. We are to find a control function uŽ t . s uv Ž t . g L2 Žw0, `., R m . such that, if xv Ž t . is the solution of Ž1.1.v with xv Ž0. s x 0 and u s uv Ž t ., then the pair Ž xv , uv . minimizes the quadratic cost functional Lv Ž v g V .. We now sketch the standard calculus of variations approach to finding solution u of the linear regulator problem. The idea is to convert the optimization problem in to a problem in Hamiltonian dynamics by consid˜v q Ý i yi ˙x i .’’ More precisely, set ering the Hamiltonian ‘‘Hv s L
˜v Ž x, u . q² y, AŽ v . x q B Ž v . u: Hv Ž x, y, u . s L
Ž 3.2.
for x, y g R n and u g R m . The reader may worry that ‘‘ AŽ v .’’ is not strictly speaking well-defined; it will turn out that this does not matter as far as the final formulation of the problem is concerned. Note that Du Hv Ž x, y, u . s Ru q B*y Du2 Hv Ž x, y, u . s R, where Du denotes the Frechet derivative with respect to u. Since R is ´ positive definite and symmetric, it is invertible, hence our Hamiltonian is regular and also it is convex. At this point one applies the Pontryagin maximal principle w26x to conclude that a necessary condition on any control function minimizing Lv is Du Hv s 0.
Ž 3.3.
Condition Ž3.3. yields the feedback rule u s yRy1 B*y Substituting u s yRy1 B*y in the formula for Hv , one obtains Hv Ž x, y . s
² x, Q Ž v . x: q 12² y, AŽ v . x:
1 2
y 21² y, B Ž v . Ry1 Ž v . B* Ž v . y:
Ž 3.4.
616
JOHNSON AND NERURKAR
for Ž xy . g R 2 n. One can now use general arguments involving the regularity and convexity of Hv to show that the control u which optimizes Lv is given by the feedback rule Ž3.4., where y Ž t . is obtained from a certain trajectory Ž xy ŽŽ tt .. . of the Hamiltonian differential equation generated by Hv and where x Ž0. s x 0 . This completes our sketch of the calculus of variations approach to finding an optimal solution u. We consider then the Hamiltonian system corresponding to Hv defined above, x9 s
Hv y
,
yX s y
Hv x
Ž v g V. .
These equations take the form AŽt t Ž v . . x s y dt yQ Ž t t Ž v . . d
ž/
ž
yBRy1 B* Ž t t Ž v . . yA* Ž t t Ž v . .
x y
/ž /
Ž 3.5. v
for each v g V. It will sometimes be convenient to write z s Ž xy . and fŽ v. s
ž
AŽ v . yQ Ž v .
yBRy1 B* Ž v . yA* Ž v .
/
for the coefficient matrix in Ž3.5.v . Then Eq. Ž3.5.v becomes z9 s f Ž t t Ž v . . z
Ž z g R2 n , v g V . .
Ž 3.6. v
Observe that f takes values in the Lie algebra spŽ n, R. of infinitesimally symplectic 2 n = 2 n real matrices. The function uŽ t . which minimizes Lv must lie in L2 Žw0, `., R m .. The feedback rule Ž3.4. suggests that we look for solutions Ž xy ŽŽ tt .. . of Ž3.5.v for which x Ž0. s x 0 and y Ž t . g L2 Žw0, `., R n .. We will show that each Eq. Ž3.5.v admits a unique solution with these properties under mild controllability assumptions on Eqs. Ž1.1.v . In fact we will actually show much more. We will show that Eq. Ž3.6.v admits an ED. Via rule Ž3.4., we will then obtain the solution of the random linear regulator problem. The theory of ED will then give us our robustness, stability and recurrence results. Remark 3.1. Classically, one solves Ž3.5.v with x Ž0. s x 0 and y Ž t . g L2 Žw0, `., R n . by solving the Riccati equation corresponding to Ž3.5.v . We use the theory of exponential dichotomy and do not solve the Riccati equation directly.
617
RANDOM FEEDBACK STABILIZATION PROBLEM
We now review the Floquet theory for Eqs. Ž3.5.v or Ž3.6.v as given in w15, 17x. A key result in this theory is a criterion Žinvolving the rotation number. for equations Ž3.6.v to have an exponential dichotomy. Let us define this latter concept. Let F Ž v , t . be the fundamental matrix solution of Ž3.6.v satisfying F Ž v , 0. s I}the 2 n = 2 n identity matrix. Then F: V = R ª SpŽ n, R. Žthe real symplectic group. is continuous and satisfies the cocycle identity: F Ž v , t q s . s F Žt t Ž v . , s . F Ž v , t .
Ž v g V , t , s g R. .
DEFINITION 3.2. Equations Ž3.6.v are said to have an exponential dichotomy ŽED for short. o¨ er V if there exist continuous vector subbundles V s, V u of R 2 n = V with the following properties: Ža. V s [ V u s R 2 n = V ŽWhitney sum.. Žb. V s and V u are invariant under the ‘‘skew product flow’’ ˆ t on 2n R = V defined by
ˆt t Ž ¨ , v . s Ž F Ž v , t . ¨ , t t Ž v . . ,
Ž Ž ¨ , v . g R2 n = V . .
Žc. There exist constants K ) 0, a ) 0 such that if Ž ¨ , v . g V s then F Ž v , t . ¨ F Key a t 5 ¨ 5
Ž t G 0. ,
if Ž ¨ , v . g V u then F Ž v , t . ¨ F Ke a t 5 ¨ 5
Ž t G 0. .
and
One of the main steps in the development of the Floquet theory is the introduction of a parameter l Žspectral parameter. in the following way. Let g be a continuous, 2 n = 2 n matrix-valued function on V such that
g Ž v . s g *Ž v . G 0
Ž v g V. .
Consider the family of differential equations z9 s f Ž t t Ž v . . q l Jg Ž t t Ž v . . z,
Ž 3.7. v , l
. is the usual skew-symmetric matrix of dimension 2 n = 2 n. where J s Ž 0I yI 0 These equations take the standard form ŽAtkinson w4x. Jy1 z9 s Jy1 f Ž t t Ž v . . q lg Ž t t Ž v . . z upon multiplication by Jy1 ; note that Jy1 f Ž?. is symmetric because f Ž?. takes values in spŽ n, R.. When discussing quantities related to Eqs. Ž3.7.v , l , we will omit reference to the parameter l unless confusion would result.
618
JOHNSON AND NERURKAR
The Floquet theory requires the following non-degeneracy assumption on equations Ž3.6.v . It is closely related to the condition placed on Eq. Ž3.7.v , l by Atkinson in w4, Chap. 9x. HYPOTHESIS Ž3.3.. For each minimal subset M : V, there exists at least one point v 0 g M with the following property. If ¨ is a non-zero vector in R 2 n, then `
Hy`
g Žt t Ž v 0 . . F Ž v 0 , t . ¨
2
dt ) 0.
Recall that F Ž v 0 , t . is the fundamental matrix solution of Ž3.6.v 0 . Alternatively, F Ž v 0 , t . is obtained by solving Ž3.7.v , l with l s 0. It can be shown w17, Lemma Ž2.7.x that Hypothesis Ž3.3. actually implies that `
Hy`
g Ž t t Ž v . . Fl Ž v , t . ¨
2
dt ) 0
for all v g M and all l g C, whenever M is a minimal subset of V. Here FlŽ v , t . is of course, the fundamental matrix solution of Eq. Ž3.7.v , l . We now discuss the rotation number a s a Ž l. for Eqs. Ž3.7.v , l . We need to review the rudiments of the structure of the set of Lagrange planes in R 2 n. This discussion will also help to clarify the geometric meaning of the solution of the Riccati equation associated with Ž3.5.v . So, let h : R 2 n be a linear subspace of dimension n. Cal h a Lagrange plane if ² x, J y : s 0 for all x, y g h. Let L R be the compact manifold of all real Lagrange planes h : R 2 n. Consider the open subset U : L R consisting of those Lagrange planes h which admit a basis of vectors of the form
ˆe1 ˆe n , ??? . m1 mn
ž / ž /
Here ˆ e1 , ??? ˆ e n4 is the standard basis in R n Žnot R 2 n ., and m1 , ??? m n4 are n-dimensional column vectors. Each h g U can be parametrized by the n = n real matrix m s Ž m1 , ??? m n . whose columns are m1 , ??? m n . The fact that h is a Lagrange plane implies that m is symmetric: m* s m. Next let h 0 s Span e nq1 , . . . , e2 n4 be the n-plane in R 2 n spanned by the last n unit vectors. It is easily seen that h 0 g L R . Let C s h g L R ¬ dim Ž h l h 0 . G 1 4 : L R . The set C is called Maslo¨ cycle. One can show that L R y C is simply connected. The complement of C in L R is just the open set U defined
RANDOM FEEDBACK STABILIZATION PROBLEM
619
above. Also C is of codimension 1 in L R in an appropriate sense Žthough it is not a submanifold of L R , but rather a stratified submanifold.. Furthermore, C is two sided in L R in the following sense: if an oriented curve in L R passes through C , one can assign an oriented intersection number of each point of Žtransversal . intersection. This intersection number takes values between yn and n Žsee w1x.. Fix l g R and again let FlŽ v , t . be the fundamental matrix solution of Ž3.7.v , l . Since l g R, FlŽ v , t . lies in the symplectic group SpŽ n, R. for all Ž v , t .. It follows that, if h g L R , then FlŽ v , t .h g L R where FlŽ v , t .h denotes the image of h : R 2 n under the indicated linear transformation. Thus if T ) 0, the map k :t ª FlŽ v , t .h from w0, T x to L R defines a curve in L R . Let nŽT . be the number of oriented intersections this curve makes with the Maslov cycle C . We slide over certain details involved in giving a precise definition of nŽT .; these matters are discussed in w15x. consider the limit
a Ž l . s Lim
Tª`
p nŽ T . T
.
Ž 3.8.
It exists in the following sense. Let m be an ergodic measure on V Žsee the definition below.. Then there is a subset V 1 : V, whose complement has m-measure zero, such that the limit in Ž3.8. exists and is independent of both, the choice of v g V 1 and h g L R . We call this number a Ž l. s a Ž l, m . Žor rather the function l ª a Ž l.. the rotation number of Eqs. Ž3.7.v , l with respect to the ergodic measure m. We see that a Ž l. measures the average number of points of intersection of the curve k Ž t . with the Maslov cycle C as T ª y`. It is proved in w15x that l ª a Ž l. is continuous and monotone non-decreasing. We pause to recall the definition of ergodic measure Žsee w24x for a detailed discussion.. DEFINITIONS 3.4. Let m be a Radon probability measure on V. Then m is in¨ ariant if, for each Borel subset B : V, one has m Žt t Ž B .. s m Ž B . for all t g R. The measure m is ergodic if, measure of any invariant set is either 0 or 1. We also recall that the topological support of a Radon measure m on V is the complement of the largest open set V satisfying m Ž V . s 0. We now state the basic relation between the rotation number and the existence of ED for Eqs. Ž3.7.v , L . THEOREM 3.5. Suppose that m is an ergodic measure on V whose topological support is all of V. Further suppose that the Atkinson-type Hypothesis Ž3.3. is ¨ alid. Let I : R be an open inter¨ al. Then Eqs. Ž3.7.v , l
620
JOHNSON AND NERURKAR
ha¨ e an exponential dichotomy o¨ er V for all l g I if and only if the rotation number l ª a Ž l. is constant on I. The theorem is proved in w17x. Since a is continuous and monotone, the assumption of constancy of a on I is equivalent to equality of the values of a at the end points of I. We finish Section 3 by returning for a moment to the Atkinson-type condition Ž3.3.. We show that it is actually equivalent to a controllability condition. PROPOSITION 3.6.
Hypothesis Ž3.3. holds if and only if the control process z9 s yf * Ž t t Ž v . . z q g Ž t t Ž v . . u
Ž 3.9. v
is Ž controllable and hence. uniformly controllable on each minimal subset M : V. Proof. Consider first the ‘‘only if’’ implication. Write out the condition in Ž3.3. and note that F*Ž v 0 , t .y1 is the fundamental matrix solution of the adjoint system z9 s yf * Ž t t Ž v . . z. Using symmetry of g and Lemma Ž2.4., we see that Ž3.9.v is uniformly controllable on each minimal subset M : V. The ‘‘if’’ implication follows quickly from the non-singularity criterion of the controllability matrix. We see that there is an interesting interplay between the notion of uniform controllability, the Atkinson condition, and exponential dichotomy. Palmer w25x has considered the relation between controllability, bounded input-bounded output stability and the existence of exponential dichotomy.
4. SOLUTION OF THE RANDOM FEEDBACK STABILIZATION PROBLEM We begin by introducing the basic controllability assumptions which will imply that the random regulator problem and the random feedback control problem are solvable. These assumptions are natural variants of the classical conditions found in w7, 14, 20x. Define C Ž v . s QŽ v .
'
Ž v g V.
621
RANDOM FEEDBACK STABILIZATION PROBLEM
to be the unique positive semidefinite square root of the matrix function QŽ v .. HYPOTHESES 4.1. Suppose that, for each minimal subset M : V, there exists at least one v 0 g M such that: Ža. Žb.
x9 s yA*Žt t Ž v 0 .. x q C Žt t Ž v 0 .. u1 is controllable; y9 s AŽt t Ž v 0 .. y q B Žt t Ž v 0 .. u 2 is controllable.
For later convenience we write u1 , u 2 in place of u in Hypotheses 4.1. Note that Hypothesis 4.1Ža. is automatically satisfied if QŽ v . is positive definite for all v g V. We first show that Hypotheses 4.1 implies the Atkinson-Hypothesis 3.3 for appropriate g . Define
g Ž v. s
ž
QŽ v .
0 y1
BŽ v . R
0
Ž v . B* Ž v .
/
so that g is continuous, symmetric and positive semi-definite. Consider the family of control processes Ž3.9.v encountered at end of Section 3; explicitly, d
x s dt y
ž/
ž
yA* Ž t t Ž v . .
Q Žt t Ž v . .
y1
Ž BR B* . Ž t t Ž v . .
ql
ž
AŽt t Ž v . .
Q Žt t Ž v . .
x y
/ž /
0 y1
Ž BR B* . Ž t t Ž v .
0
v1 v2 .
./ž /
Ž 4.1. v
LEMMA 4.2. Hypotheses Ž4.1. implies that, if M : V is minimal, then the control systems Ž4.1.v are uniformly controllable for each v g M. COROLLARY 4.3. Eqs. Ž3.7.v , l with
If Hypotheses 4.1 holds, then Hypothesis 3.3 is ¨ alid for
fŽ v. s
ž
AŽ v . yQ Ž v .
yBRy1 B* Ž v . yA* Ž v .
/
and
g Ž v. s
ž
QŽ v . 0
0 y1
BR
B* Ž v .
/
.
Corollary 4.3 follows from Proposition 3.6 and Lemma 4.2. It will allow us to apply Theorem 3.5 to Eqs. Ž3.7.v , l .
622
JOHNSON AND NERURKAR
Proof of Lemma 4.2. Let C Ž v , t . be the fundamental matrix solution of x9 s yA*Žt t Ž v .. x, Ž v g V .. The nonsingularity Žand hence the positive definiteness . of the controllability matrix yields positive numbers T ) 0, d ) 0 such that T
H0
C Žt t Ž v 0 . . C Ž v 0 , t . ¨
2
dt G d 5 ¨ 5 2
for all ¨ g R n. Using symmetry and semi-definiteness of C together with a compactness argument, one can find d˜ ) 0 such that T
H0
Q Žt t Ž v 0 . . C Ž v 0 , t . ¨
2
dt G d˜ 5 ¨ 5 2
for all ¨ g R n. By Lemma 2.4, we see that the control systems x9 s yA* Ž t t Ž v . . x q Q Ž t t Ž v . . u1
Ž 4.2. v
are uniformly controllable for all v g M. Using strict positive definiteness of R and an argument like that just given, one shows that, if Ž4.1.Žb. holds, then the control systems y9 s A Ž t t Ž v . . y q B Ž t t Ž v . . Ry1 Ž t t Ž v . . B* Ž t t Ž v . . u 2
Ž 4.3. v
are uniformly controllable over M. Let us return again to the point v 0 g M. Let Ž xy 00 . g R 2 n. There exists a number T ) 0 and controls u1 , u 2 :w0, T x ª R m with the following properties: Ži. the solution x Ž t . of Ž4.2.v with x Ž0. s x 0 satisfies x ŽT . s 0; 0 Žii. the solution y Ž t . of Ž4.3.v with y Ž0. s y 0 satisfies y ŽT . s 0. 0 Define w 1 Ž t . s u1 Ž t . y x Ž t . w2 Ž t . s u2 Ž t . y y Ž t . for 0 F t F T. The control Ž ww12 . steers Ž xy 00 . to zero in time T for Eq. Ž4.1.v 0 . This shows that the process Ž4.1.v 0 is controllable. By Lemma 2.4, we obtain the statement of Lemma 4.2. We are now in a position to apply Theorem 3.5. Indeed, we now consider the rotation number of Eqs. Ž3.7.v , l with f, g as above. LEMMA 4.4. Assume that Hypotheses 4.1 holds. Let I s Žy1r2, 1r2.. If l g I and M : V is a minimal set, then the rotation number of Eqs. Ž3.7.v , l
623
RANDOM FEEDBACK STABILIZATION PROBLEM
equals zero for e¨ ery ergodic measure m whose topological support is contained in M. Proof. Let v g M. Consider the boundary value problem d
x s dt y
ž/ ž
yBRy1 B* yA*
A yQ
ql J
ž
Q
0
0
y1
/Ž
tt Ž v . .
Ž BR B* .
/
Žt t Ž v . .
x y
ž/
Ž 4.4. v
x Ž 0. s x Ž T . s 0 where T is some positive number. We show that this boundary value problem has only the trivial solution if T is sufficiently large. To do so, let Ž xy ŽŽ tt .. . be a solution of Ž4.4.v . Then 0 s² x Ž T . , y Ž T .: y² x Ž 0 . , y Ž 0 .: s T
s
H0
s
H0
¦
d
dt
T
;¦
xŽ t. , yŽ t. q xŽ t. ,
T
H0
d dt
d dt
² x Ž t . , y Ž t .: dt
;
yŽ t.
dt
² Ax y Ž 1 q l . BRy1 B*y, y: q² x, Ž l y 1. Qx y A*y: T
Hence, 0 s Ž l y 1.
H0
5 Cx 5 2 dt y Ž1 q l.
T
H0 5 R
T
T
Ž l y 1 . H 5 Cx 5 2 dt s Ž 1 q l . H 0
yŽ1r2.
dt.
B*y 5 2 dt, that is,
RyŽ1r2. B*y
2
dt.
0
Since l g Žy1r2, 1r2., we must have C Žt t Ž v .. x Ž t . s 0 s B*Žt t Ž v .. y Ž t . for all 0 F t F T. We conclude that yX Ž t . s yA*Žt t Ž v .. y Ž t .. Hence using uniform controllability of y9 s Ay q Bu Žsee Lemma Ž2.4. and the relation T
H0
5 B*y 5 2 dt s 0,
we see that y Ž t . s 0 on w0, T x if T is sufficiently large. Since x Ž0. s 0, we have by uniqueness that x Ž t . is identically zero on w0, T x as well. Now we can show that a Ž l, m . s 0 if l g I. Consider the Lagrange plane h 0 s Ž 0y . ¬ y g R n4 used in defining the Maslov cycle C . Note that, if dimŽ FlŽ v , T .h 0 l h 0 . G 1 for some T ) 0 and some l g I, then there
624
JOHNSON AND NERURKAR
exists 0 / y 0 g R n such that 0 0 Fl Ž v , T . y s . yŽT . 0
ž / ž
/
But then y ŽT . s 0 by the preceding paragraph, a contradiction if T is large. Hence for all l g I and all large T, the Lagrange plane FlŽ v , T .h 0 does not lie on the Maslov cycle. By Ž3.8., we must have a Ž l. s a Ž l, m . s 0 for all l g I. LEMMA 4.5. Equations Ž3.5.v admit an exponential dichotomy o¨ er V. In other words, Eqs. Ž3.7.v , l admit an exponential dichotomy o¨ er V when l s 0. Proof. If M is a minimal subset of V, then Eqs. Ž3.5.v admit an ED over M by Lemma 4.4. By Theorem Ž3.1. of w15x, the dimensions of the stable and unstable bundles V s, V u over M are each equal to n. That is, these dimensions are independent of the minimal subset M : V. Next let v g V and suppose that Ž xy ŽŽ tt .. . is a solution of Eq. Ž3.5.v which is bounded on all of R. We claim that x Ž t . and y Ž t . are identically zero. To prove this, note first that there are sequences t n ª `, sn ª y` such that Ž xy ŽŽ tt nn .. . ª Ž 00 . and Ž xy ŽŽ ssnn .. . ª Ž 00 .. For if, for example, there was no such sequence sn4 , then it is easily seen that, for each v in the a-limit set of v , the equation Ž3.5.v would admit a non-trivial solution bounded on all of R. But the a-limit set of v contains a minimal set M. Since Eqs. Ž3.5.v have ED on M, there can be no non-trivial bounded solution of Ž3.5.v if v g M. This contradiction proves that sn4 exists, and in a similar way one proves that t n4 exists. Now, d
tn
² x Ž t n . , y Ž t n .: y ² x Ž sn . , y Ž sn .: s H
sn
sy
dt tn
Hs
² x Ž t . , y Ž t .: dt 5 RyŽ1r2. B*y 5 2 q 5 Cx 5 2 dt.
n
We conclude that B*Žt t Ž v .. y Ž t . s 0 s C Žt t Ž v .. x Ž t . for all t g R. But now using the controllability condition ŽHypothesis 4.1a, b. together with Proposition 2.5 and arguing as in the proof of Lemma Ž4.4., we see that x Ž t . and y Ž t . are identically zero. We now apply a theorem of w27x, the proof of which is based on ideas of Conley w8x. This theorem states that if the dimensions of V s, V u are independent of M and if no equation Ž3.5.v admits a non-trivial bounded solution, then Eqs. Ž3.5.v admit an ED over V. This completes the proof of Lemma 4.5.
RANDOM FEEDBACK STABILIZATION PROBLEM
625
We need one more bit of information before turning to the solution of the linear regulator problem and the feedback control problem. Let V s : R 2 n = V be the stable bundle of Eqs. Ž3.5.v . LEMMA 4.6. Let v g V and let Ž xy . g V s Ž v . s ŽR 2 n = v 4. l V s. Then ² x, y : / 0 unless x s y s 0. Proof. We essentially repeat part of the proof of Lemma 4.5. Suppose that there exists Ž xy 00 . g V s Ž v . with ² x 0 , y 0 : s 0. Let Ž xy ŽŽ tt .. . be the solution of Ž3.5.v with initial value Ž xy 00 .. Then d
T
² x Ž T . , y Ž T .: s H
dt
0
sy
T
H0
² x Ž t . , y Ž t .: dt
5 RyŽ1r2. B*y 5 2 dt y
T
H0
5 Cx 5 2 dt.
Since Ž xy 00 . g V s Ž v . we have x ŽT . ª 0, y ŽT . ª 0 as T ª `. Arguing as in the proof of Lemma 4.5, we now show that x Ž t . ' 0 ' y Ž t ., thus x 0 s y 0 s 0. This completes the proof. By a result of w15x, V s Ž v . is a Lagrange plane for each v g V and Lemma 4.6 implies that, for each v g V, the projection of V s Ž v . on to h1 s Ž 0x . ¬ x g R n4 is on to. Thus V s Ž v . has a basis of the form
ž
ˆe1 ˆe n , ??? ; m1 Ž v . mn Ž v .
/ ž
/
i.e., can be parametrized by the following Žsymmetric. matrix: m Ž v . s Ž m1 Ž v . , . . . , m n Ž v . . Žsee Section 3.. We see that, if Ž xy 00 . g V s Ž v . and Ž xy ŽŽ tt .. . is the corresponding solution of Ž3.5.v , then y Ž t . s mŽt t Ž v .. x Ž t .. The continuity of m in v follows from the continuous variation of the fibers V s Ž v .. By invariance of V s under the flow ˆ t on R 2 n = V Žsee Definition 3.2., we see that the function t ª m t Ž v . satisfies the Riccati equation m9 s yA*m y mA q mBRy1 B*m y Q. We are ready to solve the linear regulator problem. Given any control uŽ t ., let x Ž t . be the corresponding solution of x9 s A Ž t t Ž v . . x q B Ž t t Ž v . . u x Ž 0. s x 0 .
626
JOHNSON AND NERURKAR
The following formula can be verified using the Riccati equation for m: d dt
² m Ž t t Ž v . . x Ž t . , x Ž t .: s R1r2 Ž t t Ž v . . u Ž t . q Ry1 B*m Ž t t Ž v . . x Ž t .
2
˜v Ž x Ž t . , u Ž t . . . y 2L
Ž 4.5.
Thus Lv is minimized by choosing u Ž t . s yRy1 Ž t t Ž v . . B* Ž t t Ž v . . m Ž t t Ž v . . x Ž t . . The corresponding x Ž t . decreases exponentially as t ª ` and hence uŽ t . is square-integrable on w0, `.. The minimum value of the functional Lv is `
H0
˜v Ž x Ž t . , u Ž t . . dt s y 2L
`
H0
d dt
² m Ž t t Ž v . . x Ž t . , x Ž t .: dt
s ² m Ž v . x 0 , x 0: . This shows that uŽ t . is the unique control solving the random regulator problem. Now we turn to the random feedback stabilization problem. Define K Ž v . s yRy1 Ž v . B* Ž v . m Ž v .
Ž v g V. .
Clearly K is continuous in v . Let Ž m Žt t ŽxvŽ t... x Ž t . . be the unique solution of Ž3.5.v lying in the stable bundle V s which satisfies x Ž0. s x 0 . We see from Ž3.5.v that x Ž t . satisfies the closed-loop system x9 s A Ž t t Ž v . . q B Ž t t Ž v . . K Ž t t Ž v . . x.
Ž 4.6.
Since x Ž t . ª 0 exponentially as t ª `, the function K does indeed stabilize Ž1.1.v for each v g V. This solves the feedback stabilization problem. At this point we note that robustness of K and smoothness of K with respect to parameters follow from the corresponding properties for the bundle V s and V u. For a very general smoothness result, see Yi w28x. As a sample of the type of robustness result that can be obtained, we explain briefly the implications of a result of Coppel w9, p. 34x. Let A 0 and B0 be constant matrices for which the controllability assumptions Ž4.1. are satisfied. For convenience we choose C s Id Žthe identity. in this discussion. Let K 0 be the corresponding feedback matrix. p Ž We view A 0 Žrespectively B0 . as an element of L loc R, M Ž n, n.. Žrespec-
627
RANDOM FEEDBACK STABILIZATION PROBLEM
p Ž tively L loc R, M Ž n, m... where p ) 1. There exists numbers d 0 ) 0, L ) 0 p Ž with the following property. Let A 1 g L loc R, M Ž n, n .., B1 g p Ž L loc R, M Ž n, m.. be functions such that
Sup
tq1
H tgR t
A1 Ž s .
p
ds F d F d 0
and
Sup
tq1
H tgR t
B1 Ž s .
p
ds F d F d 0 .
Put aŽ t . s A 0 q A1Ž t ., bŽ t . s B0 q B1Ž t .. Then there exists a feedback matrix k Ž t . such that x9 s w aŽ t . q bŽ t . k Ž t .x x is exponentially stable and such that k Ž t . y K 0 - Ld
Ž t g R. .
Note that, if QŽ v . is strictly positive definite for all v , then Hypothesis Ž4.1.Ža. is automatically satisfied. Thus the possibility of solving the feedback stabilization problem really depends only on the verification of Hypothesis Ž4.1.Žb., i.e., on controllability of Ž1.1.v . Finally, we consider the question of pole relocation. For autonomous systems x9 s ax q bu, this refers to the possibility of choosing Q, R in such a way that the resulting feedback K has the property that the eigenvalues of a q bK take on prescribed values in the left half-plane. We wish to consider the random version of pole relocation. Recall that a real number b is a Lyapuno¨ exponent of Ž4.6. if there is a non-zero vector x 0 g R n such that, if x Ž t . is the solution of Ž4.6. with x Ž0. s x 0 , then 1
b s lim
tª`
t
log x Ž t . .
We are going to show that K Ž v . can be chosen so that all Lyapunov exponents of Ž4.6. lie as far to the left of the origin as desired. To do so, choose d ) 0 and consider the random regulator problem with cost functional Lv Ž x, u . s
`
H0
Ž d 5 x 5 2 q 5 u 5 2 . dt.
Clearly Hypotheses 4.1 are valid. The m-function is positive definite because if ² mŽ v . x 0 , x 0 : s 0 for some v g V and 0 / x 0 g R n, then by Ž4.5. y² m Ž t T Ž v . . x Ž T . , x Ž T .: s
T
H0
d 5 x 5 2 q 5 B*mx 5 2 dt,
where x Ž t . is the solution of Ž4.6. with x Ž0. s x 0 . The left-hand side tends to zero as T ª `, hence x Ž t . s 0 for al t, hence x 0 s 0.
628
JOHNSON AND NERURKAR
Now let « 1 and « 2 be respectively the minimum and the maximum of the set of numbers ² mŽ v ., x, x : ¬ v g V, 5 x 5 s 14 . Then 0 - « 1 5 x 5 2 F ² m Ž v . x, x: F « 2 5 x 5 2
forall x g R n .
Thus
d
d
² m Ž t t Ž v . . x Ž t . , x Ž t .: F yd 5 x 5 2 F y ² m Ž t t Ž v . . x Ž t . , x Ž t .: . dt « 2
Therefore ² mŽt t Ž v .. x Ž t ., x Ž t .: F MeyŽ d t r « 2 . where M is some positive constant. Hence, xŽ t.
2
F
1
«1
² m Ž t t Ž v . . x Ž t . , x Ž t .: F
M
«1
eyŽ d t r « 2 . .
Thus every solution of the feedback system decays exponentially as t ª ` with rate at least Žydr2 « 2 ., Ž v g V .. Thus the Lyapunov exponents lie always to the left of Žydr2 « 2 ., uniformly in v g V. This last assertion is not always true if one has Žunbounded. stochastic coefficients. To summarize: we used the rotation number to show that Eq. Ž3.5.v has ED. This allowed us to solve the random feedback stabilization problem and obtain robustness properties, smoothness properties, preservation of recurrence of the solution, and the random pole relocation result.
REFERENCES 1. V. Arnold, On a characteristic class entering in a quantum condition, Funct. Anal. Appl. 1 Ž1969., 1]14. 2. B. Anderson and J. Moore, Detectability and stabilizability of time-varying discrete time linear systems, SIAM J. Control Optim. 19, No. 1 Ž1981., 20]32. 3. Z. Artstein, Uniform controllability via limiting systems, Appl. Math. Optim. 9 Ž1982., 111]131. 4. F. Atkinson, ‘‘Discrete and Continuous Boundary Value Problems,’’ Academic Press, New YorkrLondon, 1964. 5. P. Bougerol, Kalman filtering with random coefficients and contraction, SIAM J. Control Optim. 31, No. 4 Ž1993., 942]959. 6. P. Bougerol, Filtre de Kalman Bucy et exposants de Lyapunov, preprint Ž1993.. 7. V. Cheng, A direct way to stabilize continuous time and discrete time linear time varying systems, IEEE Trans. Automat. Control AC-24 Ž1979., 641]643. 8. C. Conley, ‘‘Isolated Invariant Sets and the Morse Index,’’ CBMS Regional Conference Series in Mathematics, Vol. 38, SIAM, Philadelphia, 1978. 9. W. Coppel, ‘‘Dichotomies in Stability Theory,’’ Lecture Notes in Mathematics, Vol. 629, Springer-Verlag, Heidelberg, 1978. 10. G. DaPrato, Synthesis of optimal control for an infinite dimensional periodic problem, SIAM J. Control Optim. 25 Ž1987., 706]714.
RANDOM FEEDBACK STABILIZATION PROBLEM
629
11. G. DaPrato and A. Ichikawa, Quadratic control for linear time varying systems, SIAM J. Control Optim. 28, No. 2 Ž1990., 359]381. 12. I. Gibson, The Riccati integral equation for optimal control problem on Hilbert spaces, SIAM J. Control Optim. 17 Ž1979., 537]565. 13. R. Hermann, Cartanian Geometry, Nonlinear Waves and Control Theory, Part A, Interdisciplinary Mathematics Series, Vol. XX, Math. Sci. Press., Brookline, MA, 1980. 14. M. Ikeda, H. Maeda, and S. Kodama, Stabilization of linear systems, SIAM J. Control 10 Ž1972., 716]729. 15. R. Johnson, m-Functions and Floquet exponents for linear differential systems, Ann. Mat. Pura Appl. 147 Ž1987., 211]248. 16. R. Johnson and M. Nerurkar, On null controllability of linear systems with recurrent coefficients and constrained controls, J. Dynamics Differential Equations 4, No. 2 Ž1992., 259]273. 17. R. Johnson and M. Nerurkar, Exponential dichotomy and rotation number for linear Hamiltonian systems, J. Differential Equations 108 Ž1994., 201]216. 18. R. Johnson and M. Nerurkar, Null controllability of linear systems with positive Lyapunov exponents, in ‘‘Differential Equations, Dynamical Systems and Control Science;’’ ŽK. Elworthy and W. Everitt, Eds.., pp. 605]621, Dekker, New York, 1994. 19. R. Johnson and M. Nerurkar, Feedback control for linear chaotic systems, in ‘‘Proceedings of the 2nd IFAC Workshop on Systems Structures and Control, Prague, Sept. 1992,’’ pp. 272]273. 20. R. Kaman, Contributions to the theory of optimal control, Bull. Soc. Math. Mex. 5 Ž1960., 102]119. 21. S. Kultyshev and E. Tonkov, Controllability of non-stationary linear systems, Differential Equations Ž Engl. transl.. 11 Ž1975., 1206]1216. 22. W. Kwon and A. Pearson, A modified quadratic cost problem and feedback stabilization of linear systems, IEEE Trans. Automat. Control AC-22 Ž1977., 838]842. 23. E. Lee and L. Markus, ‘‘Foundations of Optimal Control Theory,’’ Wiley, New York, 1967. 24. V. Nemytskii and V. Stepanov, ‘‘Qualitative Theory of Ordinary Differential Equations,’’ Princeton Univ. Press, Princeton, NJ, 1960. 25. K. Palmer, Two linear systems criteria for exponential dichotomy, Ann. Mat. Pura Appl. 124 Ž1980.. 199]216. 26. L. Pontriyagin, V. Baltyanskii, R. Gamkrelidze, and E. Mishchenko, ‘‘The Mathematical Theory of Optimal Processes,’’ Wiley]Interscience, New York, 1962. 27. R. Sacker and G. Sell, A spectral theory for linear differential systems, J. Differential Equations 27 Ž1978., 320]358. 28. Y.-F. Yi, Smoothness of integral manifolds, J. Differential Equations 102 Ž1993., 153]187.