Linear quadratic differential games with cheap control

Systems St Control North-Holland

Letters

8 (1986)

181-188

181

Linear auadratk differential with cheap control Ian R. PETERSEN Depurment

of Electricul

and

Defence

Force

Academy,

Received Revised

24 March 1986 11 July 1986

Electronic

Campbell

ACT

This paper considers a linear game in which the weighting on the allowed to approach zero. It is shown mum phase condition is satisfied then will approach zero as the weighting on approaches zero.

Abstract

Keywords:

control,

Differential Cheap control.

games,

Riccati

Engineering, 2600,

Australiun

Aus~alia

quadratic differential minimizing control is that if a certain minithe value of the game the minimizing control

games

U

trol weighting on the minimizing player approaches zero. As in the case of references [l-3], this result has an important application in the stabilization of uncertain systems using high gain observer based feedback. Indeed, in [7] the uncertain systems under consideration are required to satisfy a certain ‘matching’ condition. The results of this paper will enable that matching condition to be significantly weakened.

2. System assumptions and definitions equations,

High

gain

We consider a linear quadratic differential game in which the cost functional is

1. Introduction

In recent years, a number of papers have dealt with the linear quadratic regulator problem with cheap control; see [l-3]. In these papers, it is shown that if a certain minimum phase condition is satisfied, then the optimal cost for the regulator problem approaches zero as the control weighting approaches zero. This result is not only useful in the theory of high gain linear regulators but it also forms the basis of the ‘asymptotic loop recovery’ approach to robust controller design; e.g. see [4-71. In particular, [4] uses the results of [l] to show that the robustness properties of a linear quadratic regulator using full state feedback can be recovered using a high gain observer. Furthermore, reference, [7] uses the results of [2] to show that a high gain observer can often be used in place of state feedback when stabilizing an uncertain linear system. In this paper, we consider the use of cheap control in a linear quadratic differential game rather than in the linear quadratic regulator problem; e.g., see [8]. In fact our main result generalizes the main result of [2] by showing that if a certain minimum phase condition is satisfied then the value of the game will approach zero as the con0167-6911/86/$3.50

6 1986, Elsevier

Science

Publishers

B.V.

-u(t)‘u(t)

i

and the system under consideration by the state equations z(t) =/ix(t)

+l?u(t)

+DJ(t),

y(r) = Cx(t).

dt

(2.1)

is described

x(0) =x0, (a

In this differential game, x(t) E R” is the state, u(t) E R”’ is the minimizing control, u(t) E RJ’ is the maximizing control, y(t j E Rq is the output and q >.O is the control weighting parameter. 2.1 (see also [8]). Given any x0 E R”, the differential game defined by cost functional Jq( -) and system (2) is said to have uafue Jg* if given any E > 0, there exist control laws u&x) and u,(x) such that the following conditions hold: Definition

6) Jqbe7u)GJ~*+E for all control strategies u(e); (ii)

J,(u, Ue)2Jq*-E

for all control strategies u(a). (North-Holland)

182

I. R. Petersetr

/ Lineur

qrtadrutic

Associated with the differential game described above is the algebraic Riccati equation A’P + PA - q2PBB’P + PDD’P + C’C = 0.

(2.2)

It is shown in [8] that if this Riccati equation has a minimal positive-definite symmetric solution P then given any x0 E R”, the value of the game is given by J; =x&&.

Furthermore, if the game has a finite value for all x0 E R” then this Riccati equation will have a positive-definite symmetric solution. Throughout this paper, we will assume that there exists a positive-definite symmetric solution to Riccati equation (2.2). However, it may be of interest to utilize some results concerning the existence of solutions to this Riccati equation. Indeed, Riccati equation (2.2) has a positive definite solution P if and only if the Riccati equation AS + SA’ - q2BB’ + DD’ + SC’CS = 0

has a positive-definite solution S = P-‘. Reference [9] gives conditions under which this Riccati equation has a positive-definite symmetric solution. These results are stated in terms of the eigenvalues and generalized eigenvectors of the associated Hamiltonian matrix A MADD’-q2BB’

C’C -A’

games

with

cheup corltrol

corresponding differential

game is given by

J;;” = x;Bx, where F is the minimal positive-definite ric solution to Riccati equation (2.4).

symmet-

Assumptions. Throughout the sequel, it will be assumed that the following assumptions are satisfied: Al. [A, B] is stabilizable. A2. [C, A] is detectable. A3. B is full rank. A4. C is full rank. A5. The transfer function matrix

G(s) = C(sI-A)-lB is right invertible and strictly minimum phase, i.e.,

1

=n+q,

Re(s)>O.

Remark. It is straightforward to verify that for any x0 E R’l, the value of the differential game satisfies Jq* > 0 and furthermore, J* is monotone non-increasing as q + 00. We w& show that under the above assumptions the value of the game will have the following property:

lim J4* = 0 9-00

1

for all x0 E R”.

’

In the sequel, it will be convenient to consider an augmented cost functional rather than the cost functional J,(u, u). Thus, we define the cost functional

where Q > 0 is a given positive-definite weighting matrix. Associated with this cost functional is the Riccati equation

3. The main result

The following theorem is the main result of this paper. Theorem 3.1. Suppose that the system (X) satisfies assumptions Al-A5 and that Riccari equarion (2.4) has a posiriue-definite symmetric solution for all q sufficiently large. Then giuen any x0 E R”,

lim J;;“=O. 9-m

Before proving this theorem, we first present some additional lemmas and notation.

AtP,i- PA - q2PBB’P + PDD’P -I- C’C +Q/q2=0.

dif/erettrial

(2.4

As above, given any x0 E R” the value of the

Notation. Let 2% R” be the maximal A, B-invariant subspace contained in ker C; e.g. see [lo].

I. R. Petersen

/ Linear

qurrdrntic

Furthermore, let F’(9) denote the set of all matrices F such that 9’ is (A + BF)-invariant. Given any FE .9( 9) let A, g A + BF. It follows from Section 0.7 of [lo] that there exists a state space transformation on (2) such that we can write (3.1) Furthermore, since YC ker C, it follows that C will be of the form C = [0 C,]. Lemma 3.1. Suppose that the system (X) satisfies

assumptions Al-A5. Then there exists a matrix F, E %( 9’) with the following property: Suppose A,, = A + BF, has been transformed into the form (3.1). Then the block A, is a stability matrix, i.e., all of its eigenvalues satisfy Re( h) < 0.

diJererhd

183

games with cheap control

solution to (2.4) with q = q. Using the definition of A,, it follows that A;jj

+ I’AFO - F,B+ - PBF, - ~~~~~~~

+i;DD’~+C’C+Q/~2=0. Let x E R” be given such that B’l?x = 0, x # 0. It follows that

= -x’Qx/q2

< 0.

Hence, Finsler’s Theorem (see [12]) implies that there exists a constant u > 0 such that the matrix -Q=A$j+ij~~~-

apBB’F + PDD’F + C’C

is negative-definite. Let S = I?-’ and 0 = SOS. It follows that S satisfies the Riccati equation A,S + SA;, - aBB’ + DD’ + SC’CS + 0 = 0.

Proof. It follows

from assumption zeros of the system matrix A-SI [ c

B 0

A5 that the

(3.2) We now partition (3.1) as follows ‘:

1

satisfy Re(s) -Z 0. However, using Lemma 3 of [ll] and Sections 5.4 and 5.5 of [lo], we conclude that the eigenvalues of A, are either zeros of the system matrix or freely assignable by choice of FE .9(Y), Hence, there exists an F, Ed such that A, is a stability matrix. In the next lemma, we will assume that the matrix F, has been constructed according to Lemma 3.1 and that the system (2) has been transformed so that A, is of the form (3.1).

the matrix S to conform with

Sl s3 s= [ S’ I s,>o, 3

s,>o.

S2’

Similarly, we also partition the matrices B, C and Q as follows: B= [;:I,

D= [;;I,

6, o;8, o2 1,

&O,

C=[O

C,],

Q220.

Lemma 3.2. Suppose that Riccati equation (2.4) has a positive-definite symmetric solution for all q z q. Then there exists a matrix Fl = [0 F] (partition conforming with (3.1)) such that the following condition holds: ‘Let

Thus, the substituting into (3.2) and taking the (2,2) block, we obtain the Riccati equation

F=F,+F,

Then the Riccati equation

We now let F= - $aBiSF1, A, = A, - B,P and a3 = A, + B,l? It follows that S, satisfies the Riccati equation

A’,P + PA, + PDD’P + C’C + F’F/q2

A,S2+S2A;,+D2Di+S2C;C2S2+~,=0.

and A,=A+BF.

A,& + $A;-

oB,Bi + D2Di + S2C;C2S2

+Q2=o.

(3.3)

+Q/q2=0 has a positive-definite sufficiently large. Proof.

symmetric solution for all q

Let P be a positive-definite

symmetric

1 Throughout the sequel. the notation N > M refers to the fact that the matrix N - M is positive-definite. Similarly, the notation N > M refers to the fact that the matrix N - M is positive-semidefinite.

I, R. Petersen

184

Furthermore

/ Lineur

quadruiic

gumes

with cheup cowol

the Ricatti equation

with

F=F,+F,=F,+ we get

d(ffererltiol

[O F]

1 A,=A+B(F,+ [OF])= [“d: 1.

A;P+PA,+PDD’P+C’C+E+g=O q2 as required.

q2

0

(3.4)

2

Lemma

3.3. Suppose that the Riccati equation

We now recall from Lemma 3.1 that the matrix A, is a stability matrix. Hence, the Lyapunov equation

A+ + PA - q2PBB’P + PDD~ i- C’C i- 2 = 0 4’ (3.7)

AlS; + SlA; + DIDi + I = 0

has a minimal positive-definite symmetric solution P. Furthermore, let A, be defined as in Lemma 3.2 and suppose that the Riccati equation

(3.5)

has a positive-definite symmetric solution Sr. Let y > 0 be given and define the positive-definite matrix S, by

A;P + PA, + PDD’P + C’C + F’F/q2 +Q/q2=0

(3.8)

It follows from (3.3)-(3.5),

has a positive-definite symmetric solu?ion. Then, there exists a minimal positive-definite solution P+ to (3.8) such that

A&

P
+ S,A; + DD’ + SYC’CS,. =-

yI+(y-i)~;~, [ -S,a; - D,D;

1

-A,s,-D,D;

=A

62

-Qy-

A,+ DD’P+

.Proof. In order to establish the existence of P+, let

We now choose y sufficiently large so that the matrix Q, is positive-definite. Letting $ = Sy ‘Q,S;‘, it follows that A&+S,A;.+DD’+S,{C’C+&}S,=O. (3.6)

Since Ricatti equation (3.6) has a positive-definite symmetric solution, it now follows from Theorem 5 and Remark 15 of [13] that given any q > 4” the Riccati equation Clc+E+9

S=O q2

cj = C’C+ Q/q2 + F’F/q2 and let S = P-’ where P is the positive-definite solution to (3.8). It follows that S isa solution to the Riccati equation - A,S - SA;. - S$S - DD’ = 0.

(3.9)

Now since the pair [-A,, Q112] is controllable, it follows from Theorem 5 of [13] that (3.9) has a unique maximal solution S+ > S > 0 such that all the eigenvalues of -A> - QS’ satisfy Re(h) Q 0. Let P+ = S+-’ G P. It follows from (3.9) that

Let the constant q > 0 be chosen so that

A,S”+SA;.+DD’+S

and all eigenvalues of the matrix satisfy Re( A) Q 0.

q2

will also have a positive-definite symmetric solution S> S,.. Letting P = S-l, we can see that P will be a positive-definite symmetric solution to

S+( -A’,-

$S+) = A$+

+ DD’

and hence S+( -A’,--

$S+)S+-’

= A,+ DD’P+.

That is, the matrix A, + DD’P+ is similar to -A, - $S+. Therefore, all of the eigenvalues of A, + DD’P+ satisfy Re(h) G 0. Furthermore, since S+ is the maximal solution to (3.9), it follows that P+ = ,S+-l is the minimal positive-definite solution to (3.8).

I. R. Peterseu

/ Linear

quudrutic

In order to complete the proof, we must now show that P+ > P. Indeed, let K” = -P+ Q 0. It follows from (3.8) that K+ satisfies the equation A;K+ + K+A,-

K+DD’K+

- & = 0.

(3.10)

Now since A, is a stability matrix and all of the eigenvalues of A,- DD’K+ satisfy Re(h) G 0, it follows from Theorem 2 of [14] that K is in fact the maximal solution of (3.10). We now consider the linear quadratic differential game described in Section 2 with performance index fq(u,u) given in (2.3). It follows from [8] that given any x,, E R” the value of this game is given by x;pxc. Hence, given any E > 0, there exists a control u,( -) such that given any control strategy ii(.), J( ii, u,) > xpxo

- E.

In particular, if we let ii(x) = Fx, it follows that co x$x - u:u,} dt z x;lpxc - E (3.11) I{0 where i(t)

+ Due(t),

-x’Qx

w=jom(

x(0) =x0.

+ u’u) dt

subject to Z(t) =AFx(f)

+ Du(t),

with cheup control

q(u) >, x&K+xo. Proof of Claim. Let x0 E R” and u( -) : R + R* be given. It follows from Lemma 6 of [13] that given any T>O, -x/ox

+ u’u)dt

=xhK+xo-x(T)‘K+x(T)

This holds for all T > 0 and hence 03 --Y&x + u’u)dt > x;K+x, I(0 as required. Applying this claim to inequality conclude that co -x/ox + u&) dt -x($x0 + E > I( 0

(3.11), we

2 x;K+xo.

That is,

for all xc E R” and E > 0. Hence, P+ 2 13. 0 Observations. Using the above lemmas, we can now investigate the behaviour of P, the minimal positive-definite solution to (3.7) as q + 00. Indeed, if we let s’= P-l, it follows that ,!? is the maximal solution to the Riccati equation

+s”(C’C+

I0

TIIu-tD’K+x112

2 x;K+x,.

Q/q2)$=0.

Furthermore, it follows from the results of [14] that s’ is monotonically increasing as q --, 00. Hence P = s-i is monotonically decreasing as q --, cc. Therefore, there exists a matrix P 2 0 such that

In order to establish the main result of this paper, we must show that P = 0. Towards this end, we also investigate the behaviour of P+, the minimal positive-definite solution to (3.8) as q + cc. Indeed, assuming that (3.7) has a positive-definite solution for all q sufficiently large, it follows from Lemma 3.2 that (3.8) will also have a positive-definite solution for all q sufficiently large, Let such a sufficiently large q be given. If P+ is the corresponding minimal positive-definite solution to (3.8), let K+ = -P+. It follows that K+ < 0 will be the maximal solution to the Riccati equation A’,K+ + K+AF-

+

185

x(0) =x0.

Claim. Given any x0 E R” and any control law u( *) : R + R*, the corresponding performance index satisfies

T

games

A??+ s;Q’ - q2BB’ + DD’

=AFx(t)

We now consider an optimal control problem in which we minimize the performance index

I(0

di//ererttial

dt -E+Q=o. q2

q2

K+DD’K:

- C’C

I. R. Petersen

186

/ Linear

quadratic

Moreover, it follows from the results of [14] that K+ is monotonically increasing as we let q + co. Hence, there exists a matrix KG 0 such that lim K+=K. if we write P= -K

gcmes

then

lim P+=P>,O.

We will show that fi = 0 by first considering the matrix F. Indeed, since P+ > ? for all q sufficiently large, it follows that

q+i=Im

(3.12)

Lemma 3.4.

ker i?

9~

control

(3.15)

SPcker3.

Ya=Im

Thus, we are now motivated space of the matrix p.

cheap

Observation. Using the above lemma and equation (3.12), it now follows that

4-03

P.P20.

with

Notation (see [16]). The minimal controllable output injection subspace ..TcR” is defined as follows:

q-m

Furthermore,

differential

to look at the null

’

Y=

B,

B+A(kerCnq);

limq. i+m

Lemma 3.5.

.7c ker

3.

Proof. Let q > 0 be a sufficiently large constant so that Riccati equation (3.7) has a minimal positive-definite symmetric solution B. Furthermore, let

Proof. Using the construction

L p qjB.

given in elements [AF, C]. struction

It follows from (3.7) that

of the matrix F Lemma 3.2, it follows that all of the of 9’ are unobservable states for the pair Furthermore, it follows from the conof p that it satisfies the Riccati equation (3.13)

A’,&FA,+FDD’hC’C=O.

Hence, K= -p < 0 satisfies the Riccati equation A$+

&I,-

kDD’i+?-

C’C = 0.

(3.14)

Moreover, if P+ is defined as in Lemma 3.3 and K+ = -P+, it follows that the eigenvalues of A, - DD’K+ satisfy Re(X) < 0 (for all q sufficiently large). Hence, taking the limit as q + co, it follows that the eigenvalues of A, - DD’T also satisfy Re(h) G 0. Therefore, using Theorem 2 of [14], we conclude that K is the maximal solution to (3.14). We will now show that p is the minimal positive-semidefinite symmetric solution to (3.13). Indeed, let P > 0 be a symmetric solution to (3.13). It follows that K= -P is a solution to (3.14). Therefore p = -Kg -K = P and hence p must solution to be the minimal positive-semidefinite (3.13). In order to complete the proof, we invoke Theorem 1 of [15] (with B = 0). Indeed, using this result, together with the fact that P is the minimal positive-semidefinite solution to (3.13) and the fact that all of the elements of Y are unobservable states for [AF, C], it follows that 9% ker i?

0

L’L=A++~A+~DD++CC’C+Q/q2.

We now take the limit as q + 00. lim L’L = A+ + PA + POD+ + C’C.

q-00

It now follows that the limit as q + co of L exists and we denote it z. Thus, we can write A+ + @A + POD+

+ 02 = ET.

(3.16)

Furthermore, if we again fix q’> 0 to be a sufficiently large constant, it follows from (3.7) that PBB+ = $ { A+ + FA + PDD+

+

02

+ Q/q2 }

Taking the limit as q + cc leads to the result (3.17)

ijB=O.

To complete the proof, we will now show that each subspace q satisfies q c ker @. The proof proceeds by induction. It follows from (3.17) that To = Im B

c ker

@.

We now suppose that @q = 0. Hence jq+,=s

Im B+>A(ker =pA(ker

Cnq

Cnq

).

)

I. R. Petersen

/ Lineur

quudrutic

To show that pA(ker C n 9J = 0, let x E ker C n q given. It follows that Cx = 0 and i% = 0. Therefore, using equation (3.16), we conclude that x’pzx = 0 and hence Lx = 0. Again using (3.16), . it follows that Affix + &IX + @D&X -!- c’cX = i?zX and hence @Ax = 0. However, since x E ker C nq was arbitrary, it follows that FA(ker Cnq

) =0

and hence @q.+, = 0. Thus, we can now use induction to conclude that $q= 0 for all i and hence @Y= 0. 0 We are now in a position to prove Theorem 3.1. 3.1. Using the fact that the transfer function G(s) is right invertible together with the results of [17], it follows that 9’+ Y= R”. However, Lemmas 3.4 and 3.5 imply that 9+Fc Ker 8. Therefore ker $ = R” and hence P = 0. That is,

di//eretltial

games

with cheap

control

187

using the above theorem, it follows that lim Jg*=O

q-00

for all x,, E R”. (ii) The above theorem shows that the strict minimum phase condition, assumption A5 (together with Al-A4) forms a sufficient condition for lim Jq*-+O.

4-00

However, using the results of [2], we can also obtain a necessary condition. Indeed{ consider the optimal control problem of minimizing the performance index

Proof of Theorem

lim B=O. q-00 However, as pointed out in Section 2, given any x0 E R”, the value of the differential game under consideration is given by

q =xp-x,.

subject to k(t)

=Ax(t)

y(t)

= Cx(t).

x(O) =x0,

Given any x0 E R”, it is straightforward to verify that the value of this optimal control problem jq* satisfies jq* 6 Jq*. Furthermore, it is shown in [2] that the non-strict minimum phase condition G(s) = C(sl-

A)-‘B

right invertible

and

1

Hence lime=0 q-* for all x0 E R”.

+Bu(t),

=n+q,

Re(s)>O,

(3.18)

is a necessary condition for 0

Observations. (i) The result of the above theorem

applies to the linear quadratic differential game with augmented performance index Ji( u, u). However, it straightforward to extend this result to the differential game with original performance index J,(u,u). Indeed, it is straightforward to verify that given any x0 E R” and q > 0, the values of these two differential games are related by Jq* < JI;” .

Hence, if assumptions AI-A5 are satisfied and if Riccati equation (2.2) has a positive-definite symmetric solution for all q sufficiently large, then

lim Q=O. ‘I-* It now follows that condition necessary condition for

(3.18) is also a

lim J;=O. q-00

At this point, the reader should note that there is a gap between the necessary condition (3.18) and our sufficient condition A.5. Further research could be directed towards closing this gap. Indeed, it might be conjectured that the non-strict minimum phase condition is both necessary and sufficient for lim J;=O.

4-m

188

I. R. Petersen

/ Linear

quadratic

References PI H. Kwakemaak and R. Sivan, The maximally achievable

accuracy of linear optimal regulators and linear optimal filters,qEEE Trans. Automat. Control 17 (1972) 79-86. PI B.A. Francis, The optimal linear-quadratic time-invariant regulator with cheap control, IEEE Trans. Auromar. Control 24 (1979) 616-621. 131 T. Fuji, On the perfect regulation of optimal regulators, Systems Control Lett. 1 (1982) 356-359. [41 J.C. Doyle and G. Stein, Robustness with observers, IEEE Trans. Automat. Control 24 (1979) 607-611. [51 J.C. Doyle and G. Stein, Multivariable feedback design: Concepts for a classical/modem synthesis, IEEE Trans. Automat. Conrrol 26 (1981) 4-16. Fl C.V. Hollot and A.R. Galimidi, Stabilizing uncertain systems: Recovering full state feedback performance via an observer, Proceedings of the 1986 American Control Conference, to appear. [7] I.R. Petersen and C.V. Hollot, Using observers in the stabilization of uncertain linear systems and in disturbance rejection problems, 1986 IEEE Conference on Decision and Control, submitted. [8] E.F. Mageirou, Values and strategies for infinite time linear quadratic games, IEEE Trans. Automat. Control 21 (1976) 547-550.

di//erenrial

games

,vil cheap control

P. Lancaster and L. Rodman, Existence and uniqueness theorems for, the algebraic Riccati equation, Inrernar. J. Control 32 (1980) 285-309. Control: A GeometWI W.M. Wonham, Linear Multivariable ric Approach (Springer, New York, 1979). 1111J.P. Corfmat and AS. Morse, Control of linear systems through specified input channels, SIAM J. Control 14 (1976) 163-175. Control, PI D.H. Jacobson, Extension of Linear-Quadratic Optimization and Matrix Theory (Academic Press. London, 1977). P31 J.C. Willems, Least squares stationary optimal control and the algebraic Riccati equation, IEEE Trans. Automat. Conrrol 16 (1971) 621-634. of maximal solutions of 1141 H.K. Wimmer, Monotonicity algebraic Riccati equations, $wtems Control Left. 5 (1985) 317-319. P51 M. Pachter, Some properties of the value matrix in infinite-time linear-quadratic differential games, IEEE Trans. Automat. Control 23 (1978) 746-748. WI A.S. Morse, Structural invariants of linear multivariable systems, SJAM J. Control 11 (1973) 446-465. 1171 A.S. Morse and W.M. Wonham, Status of noninteracting control, IEEE Trans. Automar. Control 16 (1971) 568-580. 191