Open-loop Nash equilibria in the non-cooperative infinite-planning horizon LQ game

Available online at www.sciencedirect.com

Journal of the Franklin Institute 351 (2014) 2657–2674 www.elsevier.com/locate/jfranklin

Open-loop Nash equilibria in the non-cooperative infinite-planning horizon LQ game$ Jacob Engwerda Tilburg University, The Netherlands Received 15 March 2013; received in revised form 5 December 2013; accepted 26 December 2013 Available online 13 January 2014

Abstract In this paper we reconsider Nash equilibria for the affine linear quadratic differential game for an infinite planning horizon. We consider an open-loop information structure. In the standard literature this problem is solved under the assumption that every player can stabilize the system on his own. In this note we relax this assumption and provide both necessary and sufficient conditions for the existence of Nash equilibria for this game under the assumption that the system as a whole is stabilizable. Basically, it is shown that to see whether equilibria exist for all initial states one has to restrict the analysis to the controllable modes of the different players. & 2014 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.

1. Introduction In the last few decades, there is an increased interest in studying diverse problems in economics, optimal control theory and engineering using dynamic games. Particularly the framework of linear quadratic differential games is often used to analyze problems due to its analytic tractability. In environmental economics, marketing and macroeconomic policy coordination, policy coordination problems are frequently modeled as dynamic games (see, e.g., the books and references in [7,20,11,27,24,18]). In optimal control theory the derivation of robust control strategies (in particular the H 1 control problem) can be approached using the theory of (linear quadratic zero-sum) dynamic games (see the seminal work of [3]). In the area of military operations, pursuit-evasion problems and, more recently, problems of defending assets can also be approached using linear quadratic modeling techniques (see, e.g., [16,22,23]). Furthermore, this modeling paradigm has been used in the area of robot formation and communication networks (see, e.g., [15,2]). ☆ A shortened version of this paper was presented at the 15th IFAC Workshop on Control Applications of Optimization in Rimini, Italy [13]. E-mail address: [email protected]

0016-0032/$32.00 & 2014 The Franklin Institute. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jfranklin.2013.12.019

2658

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

In this note we consider the open-loop linear quadratic differential game. This open-loop Nash strategy is often used as one of the benchmarks to evaluate outcomes of the game. Another benchmark that is often used is the state feedback strategy. Recently, [5] compares both strategies to see what the loss in performance of players may be using either one of these strategies. For the scalar game (see the paper for precise details on the game) they find that if there are a large number of players involved in the game, the ratio of losses p for player under a ffiffiffi an individual pffiffiffi feedback and open-loop information structure ranges between 2=2 and 2. This indicates that the difference in performance using either one of the information structures is not dramatic. The linear quadratic differential game problem with an open-loop information structure has been considered by many authors and dates back to the seminal work of Starr and Ho in [28] (see, e.g., [25,26,8–11,17,14,1,4,21,19]). More recently, [12] studied the (regular indefinite) infinite-planning horizon case for affine systems under the assumption that every player is capable of stabilizing the system on his own. In this note we drop this assumption and will just assume that the system as a whole is stabilizable. That is, we consider the problem to find Nash equilibria of the next, so-called, affine linear quadratic differential game. This game is defined by the cost functions lim J i ðx0 ; u1 ; u2 ; TÞ

ð1Þ

T-1

where

39 xðtÞ > = 6 7 Ji ¼ ½xT ðtÞ; uT1 ðtÞ; uT2 ðtÞM i 4 u1 ðtÞ 5 dt; > 0 > : u2 ðtÞ ; 2 3 Qi V i W i 6 T 7 M i ¼ 4 V i R1i N i 5: W Ti N Ti R2i Z

T

8 > <

2

Mi is assumed to be symmetric, Rii positive definite (40), i ¼ 1; 2, and xðtÞA Rn is the solution of the affine linear differential equation x_ ðtÞ ¼ AxðtÞ þ B1 u1 ðtÞ þ B2 u2 ðtÞ þ cðtÞ; xð0Þ ¼ x0 :

ð2Þ

The variable cðÞ A L2 here is some given trajectory. The controls ui ðtÞA R considered by player i, i¼ 1, 2, are assumed to be such that uT ≔ðuT1 ðÞ; uT2 ðÞÞ belongs to the set1   U s ðx0 Þ ¼ u A L2;loc j lim J i ðx0 ; u; TÞ exists in R [ f 1; 1g; lim xðtÞ ¼ 0 ; mi

t-1

T-1

where L2;loc is the set of locally square-integrable functions, i.e.,   Z T T u ðsÞuðsÞ dso1 : L2;loc ¼ u½0; 1Þj 8 T40; 0

Notice that we make no definiteness assumptions w.r.t. matrix Qi. We assume that the players have an open-loop information structure about the game. That is, at time t ¼ t 0 they have all information about the game, determine their actions, which are then enforced as binding agreements for the whole planning horizon. Moreover, we assume that We say that limt-1 f ðtÞ ¼ 1 ð 1Þ if 8r AR; (T AR such that t ZT implies f ðtÞ Zr ð rrÞ.

1

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

2659

players will play a Nash strategy, i.e., they look for actions that have the property that a unilateral deviation from these actions makes them worse off. As already mentioned above, problem (1), (2) has been solved in [12] under the assumption that every player is capable of stabilizing the system on his own, i.e., the pairs ðA; Bi Þ; i ¼ 1; 2; are stabilizable. In this note we will relax this assumption. We will provide both necessary and sufficient conditions for the existence of Nash equilibria for this game under the assumption that the matrix pair ðA; ½B1 B2 Þ is stabilizable. This extension is important in those cases where not every player can stabilize the system on his own, but the system as a whole can be stabilized. So, in a noncooperative setting this means that though players do not cooperate, they do take into account the fact that they cannot use arbitrary actions. That is, there is a meta-objective of stabilizing the system as a whole. For didactic reasons we, first, present in Section 2 the solution together with a complete proof for the special case that in system (1) the inhomogeneous part, c(t), is lacking and the cost function is a pure quadratic form (i.e., Mi is a block diagonal matrix). Next, in Section 3, we present the solution of problem (1), (2). The proof is along the lines of the proof of [12], where the solution for this problem was provided under the assumption that the pairs ðA; Bi Þ are stabilizable. In Section 4 we illustrate the merits of the extension of the current theory to this case in a three player policy coordination problem on debt stabilization, which can be solved now using the presented theory. Finally, Section 5 contains some concluding remarks. 2. The standard linear quadratic setting In this section we consider the standard linear quadratic differential game under the assumption that the pair ðA; ½B1 B2 Þ is stabilizable. That is, the special case of Eqs. (1) and (2) where the system is homogeneous and the cost functions have no “cross terms”. Or, more specific, to find the open-loop Nash equilibria of the game where the cost functions are given by Z T fxT ðtÞQi xðtÞ þ uTi ðtÞRii ui ðtÞg dt; ð3Þ Ji ¼ 0

subject to the dynamic state equation x_ ðtÞ ¼ AxðtÞ þ B1 u1 ðtÞ þ B2 u2 ðtÞ; xð0Þ ¼ x0

where ðu1 ; u2 Þ A U s :

ð4Þ

In the Appendix we prove the next Theorem 2.1. For ease of presentation we first introduce some notation. Throughout we will use 0 to denote a zero matrix that has dimensions that follow from the context where it is used (in particular, in some cases, it may happen that in fact it does not appear (i.e., it reduces to the empty matrix)). Let Ti be nonsingular transformation matrices which transform ðA; Bi Þ into its controllable canonical form, i ¼ 1; 2 (see Lemma A.1, with ni the dimension of the controllability subspace). Let   I ni T 1 ~ i ≔½I ni 0T  1T Qi : ; Bci ≔½I ni 0T i Bi ; S~ i ≔Bi Rii 1 Bci ; Q Ai ≔½I ni 0T i AT i i 0 Theorem 2.1. Consider the linear quadratic differential game (3), (4). Assume that ðA; ½B1 B2 Þ is stabilizable. Consider matrix 2 3 A  S~ 1  S~ 2 6 ~  AT1 0 7 ð5Þ M ¼ 4 Q 5: 1 ~2 0  AT2 Q

2660

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

If the linear quadratic differential game (3), (4) has an open-loop Nash equilibrium for every initial state, then (1) M has at least n stable eigenvalues (counted with algebraic multiplicities). More in particular, there exists a p-dimensional stable M-invariant subspace S, with pZ n, such that 2 3 In 6V 7 Im4 1 5  S; V2 for some V iA Rnin . h i T T I (2) With Q11i ≔ I ni 0 T i 1 Qi T i 1 0ni and Si ≔Bci Rii 1 Bci ; the two algebraic Riccati equations, ATi K i þ K i Ai  K i Si K i þ Q11i ¼ 0;

ð6Þ

have a symmetric solution K i ðÞ such that Ai  Si K i is stable, i¼ 1,2. Conversely, if the two algebraic Riccati equations (6) have a stabilizing solution and vT ðtÞ≕½xT ðtÞ; ψ T1 ðtÞ; ψ T2 ðtÞ is an asymptotically stable solution of v_ ðtÞ ¼ MvðtÞ;

xð0Þ ¼ x0 ;

ð7Þ

T uni ≔ Rii 1 Bci ψ i ðtÞ;

i ¼ 1; 2; provides an open-loop Nash equilibrium for the linear then, quadratic differential game (3), (4). □ ~ i that appear in M are, in general, not symmetric and the Notice that the matrices S~ i and Q dimensions of matrices Ai may differ. This is a clear distinction with the standard case considered in the literature (see, e.g., [11]). Now, let " # " # ~1 A1 0 Q ~ S~ 1 S~ 2  and Q≔ ~ A~ 2 ≔ ; B≔½B1 B2 ; S≔½ ~2 : 0 A2 Q Consider the set of (coupled) algebraic Riccati equations ~ þ Q: ~ 0 ¼ A~ 2 P þ PA  PSP T

Definition 2.2. A solution PT ≕½PT1 ; PT2 , with Pi A Rni n , to Eq. (8) is called ~  C  2; (a) stabilizing, if sðA  SPÞ 3 (b) left-right stabilizing (LRS) if (i) it is a stabilizing solution, and T ~  Cþ . (ii) sð  A~ 2 þ PSÞ 0

sðHÞ denotes the spectrum of matrix H; C  ¼ fλ ACjReðλÞo0g; Cþ 0 ¼ fλACjReðλÞ Z0g: In [11] such a solution is called strongly stabilizing.

2 3

ð8Þ

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

2661

Similar to, e.g., [11, Theorem 7.7] it can be shown that the next relationship between certain invariant subspaces of matrix M and solutions of the Riccati equation (8) applies. This property can be used to calculate the (left-right) stabilizing solutions of Eq. (8). Lemma 2.3. Let V  Rnþn1 þn2 be an n-dimensional invariant subspace of M, and let X i A Rni n , i ¼ 0; 1; 2 (with n0 ¼ n) be three real matrices such that V ¼ im½X T0 ; X T1 ; X T2 T : ~ ¼ sðMjV Þ. Furthermore, If X0 is invertible, then Pi ≔X i X 0 1 ; i ¼ 1; 2; solves Eq. (8) and sðA  SPÞ ðP1 ; P2 Þ is independent of the specific choice of basis of V. □ Lemma 2.4. 1. The set of algebraic Riccati equations (8) has a LRS solution PT ¼ ½PT1 PT2  if and only if matrix M has an n-dimensional stable graph subspace and M has n1 þ n2 eigenvalues (counting algebraic multiplicities) in Cþ 0. 2. If the set of algebraic Riccati equations (8) has a LRS solution, then it is unique. Proof. h 1. Assume that Eq. (8) has a LRS solution P. Then with T≔ " # ~ A  SP  S~ 1 : TMT ¼ T 0  A~ 2 þ PS~

In 0  P I n1 þn2

i ,

Since P is a LRS solution, by Definition 2.2, matrix M has exact n stable eigenvalues and n1 þ n2 eigenvalues (counted with algebraic multiplicities) in Cþ 0 . Furthermore, obviously, the stable subspace is a graph subspace. The converse statement is obtained similarly using the result of Lemma 2.3. 2. See, e.g., Kremer [21, Section 3.2]. □

Next, assume that for every initial state there is an open-loop equilibrium action that can be implemented as a feedback solution. In particular this implies that for every standard basis vector such an equilibrium action exists. Using this property, one can construct a set of stabilizing matrices Pi satisfying the algebraic Riccati equations (8) (see for details the proof of [11, Theorem 7.12]). Furthermore, in case there is a unique open-loop equilibrium action for every state, it follows that matrix M has precisely n stable eigenvalues (see for details the proof of [11, Theorem 7.16]). Using the above-mentioned lemmas, similar to [11, Section 7.4], it can be shown then that the next two important corollaries hold. Corollary 2.5. The infinite-planning horizon two-player linear quadratic differential game (3), (4) has for every initial state an open-loop Nash set of equilibrium actions ðun1 ; un2 Þ which permit a feedback synthesis if and only if 1. there exist P1 and P2 that are solutions of the set of coupled algebraic Riccati equations (8) satisfying the additional constraint that the eigenvalues of Acl ≔A S~ 1 P1  S~ 2 P2 are all situated in the left half complex plane, and

2662

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

2. the two algebraic Riccati equations (6) have a symmetric solution K i ðÞ such that Ai  Si K i is stable, i ¼ 1,2. If ðP1 ; P2 Þ is a set of stabilizing solutions of the coupled algebraic Riccati equations (8), the actions uni ðtÞ ¼  Rii 1 Bci Pi Φðt; 0Þx0 ; T

i ¼ 1; 2;

ð9Þ

_ 0Þ ¼ Acl Φðt; 0Þ; Φð0; 0Þ ¼ I; yield an openwhere Φðt; 0Þ satisfies the transition equation Φðt; loop Nash equilibrium. The costs, by using these actions, for the players are xT0 M i x0 ;

i ¼ 1; 2;

ð10Þ

where Mi is the unique solution of the Lyapunov equation ATcl M i þ M i Acl þ Qi þ PTi Si Pi ¼ 0:

□

ð11Þ

Corollary 2.6. The linear quadratic differential game (3), (4) has a unique open-loop Nash equilibrium for every initial state if and only if 1. The set of coupled algebraic Riccati equations (8) has a LRS solution, and 2. the two algebraic Riccati equations (6) have a stabilizing solution. □

Moreover, the unique equilibrium actions are given by Eq. (9).

In case the matrices ðA; Bi Þ are stabilizable the results from [13] are obtained by replacing ~ i by Qi and S~ i by Si ≔Bi R  1 BT . Let M0 be the with this case corresponding everywhere Q ii matrix M. With T≔diagðI n ; T 1 ; T 2 Þ, a subspace V is a n-dimensional graph subspace of matrix ~ 0 ≔T  1 M 0 T: So the study M0 if and only if T  1 V is a n-dimensional graph subspace of matrix M of Nash equilibria, which are determined by the n-dimensional stable graph subspaces of M0, can ~ 0 . Without specifying in detail the be performed by studying these subspaces for matrix M matrices with a hat below note that 2 3 A  S1  S2 T ~ 0 ¼ T  16 0 7 M 4  Q1  A 5 2

A 6  Q~ 6 1 6 6  Q^ T ¼6 1 6 6  Q~ 2 4  Q^ 2

 Q2

 AT

0

3

1 c B1  B1 R11

0

1 c  B2 R22 B2

0

 AT1

0

0

0

 A^ 12

 A^ 13

0

0

0

0

 AT2

0

0

0

 A^ 22

 A^ 23

T

T

T

T

T

T

7 7 7 7 7: 7 7 5

ð12Þ

T Since ðA; Bi Þ is stabilizable it follows that all eigenvalues of the matrices  A^ i3 ; i ¼ 1; 2, are unstable (i.e., the real parts are all strictly positive). So, the corresponding eigenspaces do not play a role in determining the Nash equilibria of the game. Consequently, to determine the equilibria of the game one can restrict to the analysis of matrix M. Or, stated differently, to find

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

2663

the Nash equilibria of the game one can restrict the analysis to the subspaces associated with the controllable modes of every player. So, the above results show that also in case ðA; Bi Þ is not individually stabilizable, but the system as a whole is stabilizable, one should restrict the analysis to the subspaces associated with the controllable modes of every player to analyze the game for open-loop Nash equilibria. Example 2.7 below shows the necessity to perform this reduction. R1 Example 2.7. Consider the system x_ i ðtÞ ¼ xi ðtÞ þ ui ðtÞ; xi ð0Þ ¼ xi0 ; with the cost function J i ¼ 0 u2i ðtÞ dt; i ¼ 1; 2. Then, with A ¼ I 2;

B1 ¼

  1 ; 0

B2 ¼

  0 ; 1



T 1 ¼ I2;

T2 ¼

0 1

 1 ; 0

Qi ¼ 0;

Ri ¼ 1 and ni ¼ 1

we have Ai ¼ 1;

Bci ¼ 1;

S~ i ¼ Bi

and

~ i ¼ 0: Q

Consequently, S~ ¼ A~ 2 ¼ I and Q~ ¼ 0. This yields 2 3 1 0 1 0 60 1 0 1 7 6 7 M¼6 7: 4 0 0 1 0 5 0 0 0 1 Notice M has precisely two stable eigenvalues  1. It is straightforwardly verified that Corollary 2.6 applies with P ¼ 2I 2 and Ki ¼ 2. So this game has a unique open-loop Nash equilibrium for every initial state and the corresponding actions are uni ðtÞ ¼  2xi ðtÞ; where xi(t) solves the differential equation x_ i ðtÞ ¼  xi ðtÞ; xi ð0Þ ¼ xi0 ; i ¼ 1; 2. In case the pairs ðA; Bi Þ would have been stabilizable the game has a unique equilibrium (see, e.g., [11, Chapter 7]), with Si ≔Bi BTi , only if the spectrum of matrix 2 3 A  S1  S2 6 T 0 7 M 0 ¼ 4  Q1  A 5;  Q2 0  AT has 2 stable eigenvalues and 4 unstable eigenvalues. A simple inspection shows that this condition is not met by matrix M0. This illustrates the necessity to restrict the analysis to the controllable modes of the system for every player to see whether an open-loop Nash equilibrium exists for every initial state. □ Remark 2.8. 1. In case ðA; Bi Þ is stabilizable but not controllable, the above results might be used to find the openloop Nash equilibria more efficiently from a numerical point of view. For instance, matrix M in Eq. (5) has size ðn þ n1 þ n2 Þ  ðn þ n1 þ n2 Þ, whereas according to the current literature, see, e.g., [11, Theorem 7.11], one has to analyze a 3n  3n matrix to obtain the equilibria. 2. The elimination procedure will destroy in most cases the structure of the plant matrices that may have a physical meaning. Sometimes this may be a problem. In order to stay as close as

2664

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

possible in those cases to the physical meaning of the plant, note that there is no need to remove uncontrollable stable modes from the system. This follows by considering a basis T i ≔fbi1 ; bi2 ; bi3 g for Rn , where bi1 constitute a basis for the with (A, Bi) corresponding controllable subspace, bi2 a basis for the corresponding unstable uncontrollable subspace and bi3 a basis for the stable uncontrollable subspace. We obtain then a similar structure for matrix T M as in Eq. (12), where A^ i3 ≕diagðA^ i31 ; A^ i32 Þ is now a block diagonal matrix containing the unstable (in A^ i31 ) and stable (in A^ i32 ) modes, respectively. Since  A^ i32 contains then only unstable modes it is clear from Eq. (12) that these modes do not play a role in determining the Nash equilibria of the game. 3. It can be straightforwardly verified that ψ i ðtÞ in Theorem 2.1 coincides with the costate variable associated with the minimization of Ji (see, e.g., (21)) subject to the state equation (see, e.g., (22)). In case Eq. (8) has a set of stabilizing solutions it follows that ψ i ðtÞ ¼ Pi xðtÞ, where x(t) solves x_ ðtÞ ¼ ðA  S~ 1 P1  S~ 2 P2 ÞxðtÞ; xð0Þ ¼ x0 , solves Eq. (7). From Eq. (10) it follows that ∂J i =∂x0 ¼ 2M i x0 which usually (see Eq. (11)) differs from ψ i ð0Þ ¼ Pi x0 . This implies that the correct shadow price interpretation of x(t) is provided by 2M i xðtÞ instead of ψ i ðtÞ. See, e.g., [6] for more details on this issue. 4. The results can be straightforwardly generalized for the N-player case. The above results continue to hold by replacing matrix M with the matrix " # A  S~ ~  A~ T ; Q 2

~ ¼ ½Q ~T ⋯ Q ~ T T and A~ 2 ¼ diagðAi Þ. In Section 4 we will illustrate this where S~ ¼ ½S~ 1 ⋯ S~ N , Q 1 N in a 3-player game. □

3. The general case In this section we present the solution for problem (1), (2). For presentation purposes some additional notation is introduced at the end of the Appendix in a separate section. Theorem 3.1. Consider the linear quadratic differential game (1), (2). Assume that ðA; ½B1 B2 Þ is stabilizable. If the linear quadratic differential game (1), (2) has an open-loop Nash equilibrium for every initial state, then: 1. the same result as in Theorem 2.1 holds with matrix M replaced by M ; 2. the two algebraic Riccati equations, ATi K i þ K i Ai  ðK i Bci þ V~ i ÞRii 1 ðBci K i þ V~ i Þ þ Q11i ¼ 0 T

T

ð13Þ

have a symmetric solution K i ðÞ such that Ai  Si K i is stable, i¼ 1,2. Conversely, if the two algebraic Riccati equations (13) have a stabilizing solution and vT ðtÞ≕½xT ðtÞ; ψ T1 ðtÞ; ψ T2 ðtÞ is an asymptotically stable solution of 2 3 cðtÞ 6 7 v_ ðtÞ ¼ MvðtÞ þ 4 0 5; xð0Þ ¼ x0 ; 0

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

then, with ψ T ðtÞ≔½ψ T1 ðtÞ; ψ T2 ðtÞ, " n # h i u1 ðtÞ T ¼  G  1 B ψðtÞ þ ZxðtÞ ; n u2 ðtÞ provides an open-loop Nash equilibrium for the linear quadratic differential game (1), (2).

2665

ð14Þ □

Next consider the set of (coupled) algebraic Riccati equations: 0 ¼ A 2 P þ PA  PBG  1 B P þ Q: T

T

ð15Þ

Theorem 3.2, below, shows that in case this set of coupled algebraic Riccati equations (8) has a stabilizing solution, the game always has at least one equilibrium that allows for a feedback synthesis. In this context, the notion of (LRS) stabilizing solution should be interpreted along the lines of Definition 2.2. The proof of both Theorem 3.2 and Corollary 3.3 can be given along the lines of [12, Theorem 3.3 and Theorem 3.6, respectively]. Theorem 3.2. Assume that 1. the set of coupled algebraic Riccati equations (15) has a set of stabilizing solutions Pi ; i ¼ 1; 2; and 2. the two algebraic Riccati equations (13) have a stabilizing solution K i ðÞ, i¼ 1,2. Then the linear quadratic differential game (1), (2) has an open-loop Nash equilibrium for every initial state. Moreover, one set of equilibrium actions is given by " n # u1 ðtÞ T T ð16Þ ¼  G  1 ðZ þ B PÞΦðt; 0Þx0  G  1 B mðtÞ; un2 ðtÞ where Φðt; 0Þ is the solution of the transition equation T ~ 0Þ ¼ I Φ_ ðt; 0Þ ¼ ðA  BG  1 ðZ þ B PÞÞΦðt; 0Þ; Φð0; T R 1 ð  A þPBG  1 B and mðtÞ ¼ t e 2 Þðt  sÞPcðsÞ ds. □ An important corollary is Corollary 3.3. The linear quadratic differential game (1), (2) has a unique open-loop Nash equilibrium for every initial state if and only if 1. the set of coupled algebraic Riccati equations (15) has a LRS solution, and 2. the two algebraic Riccati equations (13) have a stabilizing solution. □ Remark 3.4. The above results generalize, again straightforwardly, for the N-player case. They continue to hold by replacing matrix M , above, with the matrix "

A

S

Q

 A2

T

#

2666

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

where S ¼ ½S 1 ⋯ S N , Q

T ¼ ½Q 1

⋯

T Q N T

and A 2 ¼ diagðAi Þ  BG

T

 h i h i T  1 I n1 T  1 I nN Z1 T 1 . 0 ⋯Z N T N 0

The definitions of the involved matrices are along the lines of the corresponding matrices in the notation section of the Appendix. □

4. Example Consider the problem to find the open-loop Nash equilibria for the next three player game. The game might be interpreted as a debt stabilization problem within a two country setting which engaged in a monetary union. Within that setting the variables xi(t) can be interpreted as the government debt, scaled to the level of national output, of country i. ui is the primary fiscal deficit, also scaled to output, whereas the monetary financing undertaken by the central bank, measured as a fraction of aggregate output, is denoted by uE. All parameters are assumed to be positive. The welfare loss-function of country i; i ¼ 1; 2; and central bank, respectively, is given by 1 J 1 ðu1 ðÞÞ ¼ 2

J 2 ðu2 ðÞÞ ¼

J E ðu3 ðÞÞ ¼

1 2 1 2

Z

1

0

Z

u21 ðt Þ þ β1 x21 ðt Þ dt

ð17Þ

u22 ðt Þ þ β2 x22 ðt Þ dt

ð18Þ

u2E ðt Þ þ βE ðωx1 ðtÞ þ ð1  ωÞx2 ðtÞÞ2 dt:

ð19Þ

1

0

Z

1 0

The evolution of debt in both countries over time is assumed to be described by the differential equations #" " # " # " #      γ1 x1 ðtÞ x_ 1 ðtÞ α1 0 1 0 ð20Þ u ðtÞ: ¼ þ u1 ðtÞ þ u2 ðtÞ þ  γ2 E x_ 2 ðtÞ x2 ðtÞ 0 α2 0 1 Let "

α1 A≔ 0  Q1 ≔

# 0 ; α2

β1

0

0

0

 ;

" #     γ1   1 0 ; B2 ≔ ; BE ≔  B1 ≔ ; B≔ B1 B2 BE ; γ2 0 1 " # " # 0 0 ω2 ωð1  ωÞ and QE ≔βE : Q2 ≔ 0 β2 ωð1 ωÞ ð1  ωÞ2

Obviously, the pairs ðA; Bi Þ; i ¼ 1; 2, are not stabilizable. Consequently we cannot directly use the standard theory on linear quadratic differential games to find the open-loop Nash equilibria   (see, e.g., [11]). However, notice that the pair (A, B) is stabilizable. With T 1 ¼ I, T 2 ¼ 01 10 ,

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

T E ¼ I, n1 ¼ n2 ¼ 1 and nE ¼ 2 in Theorem 2.1 we have that matrix 2 α1 0 1 0  γ 21 6 0 α2 0  1  γ1γ2 6 6 6  β1 0  α1 0 0 6 M¼6 0  β2 0  α2 0 6 6 6  β ω2  βE ωð1 ωÞ 0 0  α1 E 4 2  βE ωð1  ωÞ  βE ð1 ωÞ 0 0 0

2667

M equals 3  γ1 γ2 7  γ 22 7 7 0 7 7 7: 0 7 7 0 7 5  α2

By straightforward calculating one can verify that the determinant of M  λI equals det≔λ6 þ η5 λ5  η4 λ4  η3 λ3 þ ρ2 λ2 þ η1 λ þ η0

where ηi 40:

So, independent of the sign of ρ2, this sixth order polynomial has 2 sign changes (if one orders it by descending variable exponent). So according to Descartes’ rule of signs matrix M has either two or no positive real roots. Furthermore, with λ replaced by  λ the corresponding polynomial has 4 sign changes. By Descartes’ rule M therefore has either 4, 2 or no negative real roots. So, in case det has six real roots, 2 of them will be positive and 4 of them negative. So, generically (provided the two-dimensional stable subspaces are graph subspaces), in that case the game will have an infinite number of open-loop Nash equilibria and 42 equilibria for which the corresponding equilibrium strategies permit a feedback synthesis. This is confirmed in case we chose, e.g., x1 ð0Þ ¼ 0:7, x2 ð0Þ ¼ 1:5, α1 ¼ 0:03, α2 ¼ 0:08, γ 1 ¼ 1, γ 2 ¼ 0:5, β1 ¼ 0:04, β2 ¼ 0:08, βE ¼ 0:04 and ω ¼ 0:3. In that case matrix M has the eigenvalues f0:7002; 0:2426;  0:0421;  0:0594;  0:2522;  0:6992g, and it is easily verified that there are 6 different strategies that permit a feedback synthesis. By including a discount factor into the cost the number of equilibria reduces. In this example it turns out that if the discount factor is larger than 12% there will be a unique equilibrium. We calculated the resulting equilibrium strategies if a discount factor of 14% is used. Then, P1 ¼ ½0:1028  0:0951, 0:1316 0:2666 P2 ¼ ½0:0494 0:2133 and P ¼ ðtÞ ¼  P1 xðtÞ, u2 ðtÞ ¼ P2 xðtÞ and E 0:3343 0:6574 . So, u1  uE ðt Þ ¼ 1 12 PE xðt Þ ¼ ½0:2987 0:5953xðt Þ, where xðt Þ ¼ A  B1 P1  B2 P2 þ BE 1 12 PE xðt Þ; x ð 0Þ ¼ x0 .

Fig. 1. Fiscal debt stabilization using a discount factor of 14%. (a) State trajectories. (b) Control trajectories.

2668

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

In this specific case we observe that the central bank responds approximately with a three times higher monetary policy as the corresponding fiscal authorities to stabilize debt in their country. Furthermore we see that fiscal authorities negatively respond on a debt that has occurred in the other country. We plotted both the evolution of debt and control instruments in Fig. 1. From this figure we see that the initially strong monetary policy pursued by the central bank implies that the country with the initially smallest debt runs into a surplus. Furthermore, after five years the expansionary monetary financing policy of the central bank is replaced by a small contracting monetary financing. 5. Concluding remarks In this paper we reconsidered the infinite planning horizon open-loop linear quadratic differential game. We derived under less stringent conditions than in the current literature both necessary and sufficient conditions under which this game has an open-loop Nash equilibrium. Based on the controllable canonical form of the system we showed along the lines of the proof of [11,12] that one can derive Nash equilibria for stabilizable systems similar to the standard case considered in the literature. By determining for each individual player its controllable canonical form, one can derive matrix M that has a similar structure as in the standard case. Since the size of this square matrix is usually smaller than 3n it may be advantageous from a computational point of view, also in the standard case, to use this reduced form to calculate the Nash equilibria. We illustrated the theory in a small example. Acknowledgements The author likes to thank the referees for their comments on an earlier draft of this paper which helped to improve this paper. Appendix A First we recall the next well-known controllable state-space decomposition lemma (see, e.g., [29, Theorem 3.6]). Lemma A.1. Consider the linear system x_ ðtÞ ¼ AxðtÞ þ BuðtÞ, with xðtÞA Rn and uðtÞA Rm . Assume the controllability matrix C≔½BjABj⋯jAn  1 B has rank n1 (or stated differently the controllability subspace has dimension n1). Let vi ; i ¼ 1; …; n1 be n1 linearly independent columns of C and vn1 þ1 ; …; vn be such that V≔½v1 ⋯vn  is invertible. Then, with T≔V  1 , ^ BÞ ^ ^ have the next properties: x^ ðtÞ ¼ TxðtÞ satisfies x^_ ðtÞ ¼ A^ x^ ðtÞ þ BuðtÞ, and ðA; " # " # A^ 11 A^ 12 B^ 1 ; B^ ¼ TB ¼ ; A^ ¼ TAT  1 ¼ ^ 0 A 22 0 where A^ 11 A Rn1 n1 , B^ 1 A Rn1 m and ðA^ 11 ; B^ 1 Þ are controllable.

□

Proof of Theorem 2.1. “) part” Suppose that ðun1 ; un2 Þ are a Nash solution. That is, J 1 ðu1 ; un2 ÞZ J 1 ðun1 ; un2 Þ and J 2 ðun1 ; u2 ÞZ J 2 ðun1 ; un2 Þ:

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

2669

From the first inequality we see that for every x0 A Rn the (nonhomogeneous) linear quadratic control problem to minimize Z 1 fxT ðtÞQ1 xðtÞ þ uT1 ðtÞR11 u1 ðtÞg dt; J1 ¼ 0

subject to the (nonhomogeneous) state equation x_ ðtÞ ¼ AxðtÞ þ B1 u1 ðtÞ þ B2 un2 ðtÞ;

xð0Þ ¼ x0 ;

has a solution. By assumption xðtÞ-0. Since this optimization problem has a minimum and R11 40 it follows that u1 ðtÞ-0 too. By Lemma A.1 there exists a state-space transformation x~ ðtÞ ¼ T 1 xðtÞ such that the above minimization problem can be rewritten as the (nonhomogeneous) linear quadratic control problem to minimize Z 1 f~x T1 ðtÞQ111 x~ 1 ðtÞ þ 2~x T2 ðtÞQT121 x~ 1 ðtÞ þ x~ T2 ðtÞQT221 x~ 2 ðtÞ þ uT1 ðtÞR11 u1 ðtÞg dt; ð21Þ J1 ¼ 0

subject to the (nonhomogeneous) state equation " #" " # # " #   x~ 1 ðtÞ x~_ 1 ðtÞ B~ 21 n A1 A~ 12 Bc1 u1 ðtÞ þ ~ u ðtÞ; ¼ þ x~ 2 ðtÞ x~_ 2 ðtÞ B 22 2 0 A~ 22 0 where ðA1 ; Bc1 Þ is controllable and

h

Q111 QT121

Q121 Q221

x~ ð0Þ ¼ T 1 x0 ;

ð22Þ

i T ≔T 1 1 Q1 T 1 1 . Since this minimization problem

has a solution for every initial state, by [12, Theorem A.1], the algebraic Riccati equation AT1 K 1 þ K 1 A1  K 1 S1 K 1 þ Q111 ¼ 0

1 c with S1 ≔Bc1 R11 B1 ; T

has a stabilizing solution K1. Moreover its unique solution is 1 c ~ 1 ðtÞÞ: B1 ðK 1 x~ 1 ðtÞ þ m u1 ðtÞ ¼  R11 T

ð23Þ

~ 1 ðtÞ is given by Here m Z 1 T ~ 1 ðtÞ ¼ e  ðA1  S1 K 1 Þ ðt  sÞ ½K 1 ðB~ 21 un2 ðsÞ þ A~ 12 x~ 2 ðsÞÞ þ Q121 x~ 2 ðsÞ ds: m

ð24Þ

t

Notice that, since the optimal control for this problem is uniquely determined, and by definition the equilibrium control un1 solves the optimization problem, u1 coincides with un1 . By straightforward differentiation of (24) we obtain ~ 1 ðtÞ  K 1 ðB~ 21 un ðtÞ þ A~ 12 x~ 2 ðtÞÞ  Q121 x~ 2 ðtÞ: ~_ 1 ðtÞ ¼  ðA1  S1 K 1 ÞT m ð25Þ m 2

Next, introduce ψ 1 ðtÞ≔K 1 x~ 1 ðtÞ þ m~ 1 ðtÞ:

ð26Þ

1 c With this definition u1 ðtÞ ¼  R11 B1 ψ 1 ðtÞ. Moreover, using (25), we get4 T

~_ 1 ðtÞ ¼ K 1 ðA1  S1 K 1 Þ~x 1 ðtÞ K 1 S1 m~ 1 ðtÞ þ K 1 A~ 12 x~ 2 ðtÞ ψ_ 1 ðtÞ ¼ K 1 x~_ 1 ðtÞ þ m ~ 1 ðtÞ K 1 ðB~ 21 un2 ðtÞ þ A~ 12 x~ 2 ðtÞÞ  Q121 x~ 2 ðtÞ þK 1 B~ 21 un2 ðtÞ  ðA1  S1 K 1 ÞT m 4 Notice that a more direct derivation of the differential equation for (costate vector) ψ1 is obtained by using the maximum principle conditions for the minimization problem (21), (22). An advantage of the current approach is that we have an explicit expression for this costate variable.

2670

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

~ 1 ðtÞ  ðA1  S1 K 1 ÞT m ~ 1 ðtÞ Q121 x~ 2 ðtÞ ¼ ð Q111  AT1 K 1 Þ~x 1 ðtÞ K 1 S1 m T ~ 1 ðtÞÞ  Q121 x~ 2 ðtÞ ¼  ½Q111 Q121 ~x ðtÞ  AT1 ψ 1 ðtÞ ¼  Q111 x~ 1 ðtÞ A1 ðK 1 x~ 1 ðtÞ þ m ¼  ½Q111 Q121 T 1 xðtÞ AT1 ψ 1 ðtÞ ¼  ½I n1 0T 1 1 Q1 xðtÞ  AT1 ψ 1 ðtÞ: T

ð27Þ

In a similar way it follows that the minimum of J2 is obtained by 1 c B2 ψ 2 ðtÞ; u2 ðtÞ ¼  R22 T

ð28Þ

where ψ_ 2 ðtÞ ¼  ½I n2 0T 2 1 Q2 xðtÞ AT2 ψ 2 ðtÞ: T

ð29Þ

And the algebraic Riccati equation 1 c where S2 ≔Bc2 R22 B2 ; T

AT2 K 2 þ K 2 A2  K 2 S2 K 2 þ Q112 ¼ 0

has a stabilizing solution K2. Consequently, with M as in (5) and vT ðtÞ≔½xT ðtÞ; ψ T1 ðtÞ; ψ T2 ðtÞ, by (27) and (29), v(t) satisfies v_ ðtÞ ¼ MvðtÞ

with v1 ð0Þ≔½I n 0 0vð0Þ ¼ x0 :

Since by assumption, for arbitrary x0, v1 ðtÞ converges to zero it follows from, e.g., [11, Lemma 7.36] that matrix M must have at least n stable eigenvalues (counting algebraic multiplicities). Moreover, the other statement follows from the second part of this lemma which completes this part of the proof. “( part” Let un2 be as claimed in the theorem, that is 1 c B2 ψ 2 ðtÞ: un2 ðtÞ ¼  R22 T

ð30Þ

We next show that then necessarily un1 solves the optimization problem Z 1 fx T ðtÞQ1 xðtÞ þ uT1 R11 u1 ðtÞg dt; min u1

0

_ ¼ AxðtÞ þ B1 u1 ðtÞ þ B2 un ðtÞ; xð0Þ ¼ x0 . subject to xðtÞ To that end, reconsider the state transformation x ¼ T 1 x~ . With the notation of the proof of the first part of this theorem the above problem can then be rewritten as the minimization of (21) subject to the system (22). Since, by assumption, the algebraic Riccati equation (6) has a stabilizing solution, according to [11, Theorem 5.16], the above minimization problem has a solution. This solution has the same structure as the solution advertised in (23), (24). The only difference is that un2 ðtÞ satisfies in ~ 1 ðtÞ in (23) , (24). this case (30). For that reason we will use the notation m1 ðtÞ instead of m Introducing ψ~ 1 ðtÞ≔K 1 x~ ðtÞ þ m1 ðtÞ; 1 c we have that u1 ðtÞ ¼  R11 B1 ψ~ 1 . Similar to (27) we obtain ψ~_ 1 ¼  ½Q111 Q121 ~x ðtÞ  AT1 ψ~ 1 : Consequently, xd ðtÞ≔T 1 xðtÞ x~ ðtÞ and ψ d ðtÞ≔ψ 1 ðtÞ ψ~ 1 ðtÞ satisfy T

1 c 1 c B1 ψ 1 ðtÞB2 R22 B2 ψ 2 ðtÞÞ x_ d ðtÞ ¼ T 1 ðAT 1 1 T 1 xðtÞ B1 R11 T

T

1 c 1 c  ðT 1 AT 1 1 x~ ðtÞ T 1 B1 R11 B1 ψ~ 1  T 1 B2 R22 B2 ψ 2 ðtÞÞ T

T

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

2671

~ 1 T  1 T 1 xðtÞ AT ψ 1 ðtÞ  ð  ½Q111 Q121 ~x ðtÞ AT ψ~ 1 ðtÞÞ, respectively. Or, stated and ψ_ d ðtÞ ¼  Q 1 1 1 differently, 2 3 " # # " # " # A~ 12  S1 " A1 x_ d ðtÞ xd ðtÞ xd ð0Þ 0 6 0 7 A~ 22 0 5 ¼4 ; ¼ for some pA Rn1 : ψ_ d ðtÞ ψ d ðtÞ ψ d ð0Þ p T  Q111  Q121  A1 h i Notice that matrix AQ1111  AS1T is the Hamiltonian matrix associated with the algebraic Riccati 1

equation (6). Recall that the spectrum of this matrix is symmetric w.r.t. the imaginary axis. Since by assumption the Riccati equation (6) has a stabilizing solution, we know that its stable invariant subspace is given by Span½I K 1 T . Therefore, with Eu representing a basis for the unstable subspace, we can write " # " # 0 I ¼ v1 þ E u v2 ; p K1 for some vectors vi ; i ¼ 1; 2. However, it is easily verified that due to our asymptotic stability assumption both xd(t) and ψ d ðtÞ converge to zero if t-1. So, v2 must be zero. From this it follows now directly that p ¼ 0. Since the solution of the differential equation is uniquely determined, and ½xd ðtÞ ψ d ðtÞ ¼ ½0 0 solves it, we conclude that x~ ðtÞ ¼ T 1 xðtÞ and ψ~ 1 ðtÞ ¼ ψ 1 ðtÞ. Or stated differently, un1 solves the minimization problem. In a similar way it is shown that for u1 given by un1 , for player two his optimal control is given by un2 which proves the claim. □ Proof of Theorem 3.1. “) part” Let ðun1 ; un2 Þ be a Nash solution. This implies for player one, using the state-space transformation x~ ðtÞ ¼ T 1 xðtÞ again (and the corresponding notation), that the minimization of Z 1 T T ~ T1 þ x~ T2 ðtÞQT121 Þ~x 1 ðtÞ þ uT1 ðtÞR11 u1 ðtÞ J1 ¼ f~x T1 ðtÞQ111 x~ 1 ðtÞ þ 2uT1 V~ 1 x~ 1 ðtÞ þ 2ðun2 ðtÞW 0

~ 1c x~ 2 ðtÞ þ un2 R21 un2 ðtÞg dt; þ2uT1 ðN 1 un2 ðtÞ þ V~ 1c x~ 2 ðtÞÞ þ x~ T2 ðtÞQ221 x~ 2 ðtÞ þ 2un2 ðtÞW T

T

T

T

ð31Þ w.r.t. u1 ðÞ, subject to the (nonhomogeneous) state equation "

# " x~_ 1 ðtÞ A1 ¼ x~_ 2 ðtÞ 0

A~ 12 A~ 22

#"

" # #   x~ 1 ðtÞ B~ 21 n Bc1 u1 ðtÞ þ ~ u ðtÞ þ T 1 cðtÞ; x~ ð0Þ ¼ T 1 x0 ; þ x~ 2 ðtÞ B 22 2 0

ð32Þ

where ðA1 ; Bc1 Þ is controllable, has a solution. Or, equivalently, with 1 ~ v1 ðtÞ≔u1 ðtÞ þ R11 ðV 1 x~ 1 ðtÞ þ N 1 un2 ðtÞ þ V~ 1c x~ 2 ðtÞÞ T

T

ð33Þ

the minimization of Z 1 1 ~ T J1 ¼ f~x T1 ðtÞðQ111  V~ 1 R11 V 1 Þ~x 1 ðtÞ þ vT1 ðtÞR11 v1 ðtÞ þ 2pT1 ðtÞ~x 1 ðtÞ 0

T 1 T 1 ~ T ~ T1c  N T1 R11 V 1c Þ~x 2 ðtÞ þ~x T2 ðtÞðQ221  V~ 1c R11 V 1;c Þ~x 2 ðtÞ þ 2un2 ðtÞðW

1 þun2 ðR21  N T1 R11 N 1 Þun2 ðtÞg dt; T

ð34Þ

2672

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

subject to the (nonhomogeneous) state equation 1 ~ T V 1 Þ~x 1 ðtÞ þ Bc1 v1 ðtÞ þ n1 ðtÞ; x~_ 1 ðtÞ ¼ ðA1  Bc1 R11

ð35Þ

1 1 ~ T V 1c Þ~x 2 ðtÞ þ T 11 cðtÞ; x~_ 2 ðtÞ ¼ has a solution. Here n1 ðtÞ ¼ ðB~ 21  Bc1 R11 N 1 Þun2 ðtÞ þ ðA~ 12  Bc1 R11 ~A 22 x~ 2 ðtÞ þ B~ 22 un ðtÞ þ T 12 ðtÞ, T T ≕½T T T T , with T 11 A Rn1 n and T 12 A Rðn  n1 Þn and pT ðtÞ ¼ 1 2 1 11 12 T T T T 1 ~ T 1 ~ T nT ~ ~ u2 ðtÞðW 1  N 1 R11 V 1 Þ þ x~ 2 ðtÞðQ121  V 1c R11 V 1 Þ.

This implies that the algebraic Riccati equation 1 ~ T 1 ~ 1 ~ ðA1  Bc1 R11 V 1 Þ K 1 þ K 1 ðA1  Bc1 R11 V 1 Þ K 1 S1 K 1 þ Q111  V~ 1 R11 V1 ¼ 0 T

T

T

has a stabilizing solution. It is easily verified that this equation can be rewritten as (13), with i¼ 1. Similarly we get that also the second algebraic Riccati equation must have a stabilizing solution which completes the proof of point 2. To prove point 1 we note that the minimization problem (34) , (35) has a unique solution. 1 ~ T Introducing for notational convenience A^ i ≔A1  Bc1 R11 V 1  Si K i , i ¼ 1; 2; its solution is 1 c v1 ðtÞ ¼  R11 B1 ðK 1 x~ 1 ðtÞ þ m1 ðtÞÞ T

Z

1

with m1 ðtÞ ¼

ð36Þ

^T

e  A 1 ðt  sÞ ðK 1 n1 ðsÞ þ p1 ðsÞÞ ds;

t

where K1 is the stabilizing solution of the algebraic Riccati equation (13), with i¼ 1. Consequently, see (33), T 1 ~ T ðV 1 x~ 1 ðtÞ þ N 1 un2 ðtÞ þ V~ 1c x~ 2 ðtÞÞ u1 ðtÞ≔v1 ðtÞ  R11

ð37Þ

solves the original optimization problem. Next, introduce ψ 1 ðtÞ≔K 1 x~ 1 ðtÞ þ m1 ðtÞ: Then, along the lines of [12, Proof of Theorem 3.1] we get ψ_ 1 ðtÞ ¼ K 1 x~_ 1 ðtÞ þ m_ 1 ðtÞ ~ 1 un2 ðtÞ: ¼  ½Q111 Q121 ~x ðtÞ AT1 ψ 1 ðtÞ  V~ 1 un1 ðtÞ W Similarly it follows from the minimization of J2 that the corresponding costate variable ψ 2 ðtÞ ¼ K 2 x~ 2 ðtÞ þ m2 ðtÞ satisfies ~ 2 un2 ðtÞ ψ_ 2 ðtÞ ¼  ½Q112 Q122 ~x ðtÞ AT2 ψ 2 ðtÞ V~ 2 un1 ðtÞ  W

ð38Þ

 1 cT 1 ~ T ~ T2 x~ 2 ðtÞÞ: u2 ðtÞ ¼  R22 B2 ψ 2 ðtÞ R22 ðW 2c x~ 1 ðtÞ þ N T2 un1 ðtÞ þ W

ð39Þ

and

n

n

From (36), (37) and the definition of ψ1 it follows that ðu1 ; u2 Þ satisfy R11 un1 ðtÞ þ N 1 un2 ðtÞ ¼  Bc1 ψ 1 ðtÞ V T1 xðtÞ: T

T

Similarly we obtain from (38), (39) that N T2 un1 ðtÞ þ R22 un2 ðtÞ ¼  Bc2 ψ 2 ðtÞ W T2 xðtÞ: Since, by assumption, matrix G is invertible we obtain from these two equations that " n # " # T u1 ðtÞ  Bc1 ψ 1 ðtÞ V T1 xðtÞ 1 ¼G : T un2 ðtÞ  Bc2 ψ 2 ðtÞ W T2 xðtÞ

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

2673

Substitution of this into the state equation (2) and the expressions we obtained for ψ_ i ðtÞ yields then that vT ðtÞ ¼ ½vT1 ðtÞ; vT2 ðtÞ; vT3 ðtÞ≔½xT ðtÞ; ψ T1 ðtÞ; ψ T2 ðtÞ; satisfies v_ ðtÞ ¼ M vðtÞ þ ½cT ðtÞ 0 0T ; with v1 ð0Þ ¼ x0 : Since by assumption, for arbitrary x0, v1 ðtÞ converges to zero it is clear from [11, Lemma 7.36] by choosing consecutively x0 ¼ ei ; i ¼ 1; ⋯; n; that matrix M must have at least n stable eigenvalues (counting algebraic multiplicities). Moreover, the other statement follows from the second part of this lemma which completes this part of the proof. “ ( part” This part of the proof mimics the corresponding part of the proof of Theorem 2.1. The only difference is that the exogenous term entering the system in the considered optimization problems is larger. More concrete, let un2 be as defined in (14) where x(t) satisfies x_ ðtÞ ¼ ðA BG  1 ZÞxðtÞ BG  1 B~ ψðtÞ; xð0Þ ¼ x0 : T

Then, along the lines of the proof of [12, Theorem 3.1] it follows that necessarily un1 solves the minimization problem (31), (32). In a similar way it follows that if un1 is given by (14), un2 minimizes the associated cost function J2. Consequently, the pair ðun1 ; un2 Þ provides a set of openloop Nash actions. □

Notation The next shorthand notation will be used. 2 3 " # 0 0 " # R11 N 1 ½0 I 0M 1 6 7 G≔ ; 4 I 05¼ N T2 R22 ½0 0 IM 2 0 I where we assume throughout that this matrix G is invertible. V~ i ≔½I ni 0T i T V i ; V~ ic ≔½0 I n  ni T i T V i ; ~ i ≔½I ni 0T i T V i ; W ~ ic ≔½0 I n  ni T i T W i ; W T

T

T

B ≔½B 1 B 2 ≔diagf½I ni 0T i Bi g; 2 3 " # I " T# ½0 I 0M 1 6 7 V1 Z≔ ; 405¼ ½0 0 IM 2 W T2 0 " # Q1  1 ~ i  Z i G Z; Q≔ Q i ≔Q ; Q2 " I n1 T T 1 A≔A  BG Z; A 2 ≔A~ 2   I n2

2

3 0 0 6 7 Z i ≔½I 0 0M i 4 I 0 5 ¼ ½V i ; W i ; 0 I S i ≔BG  1 B i ; T

S≔½S 1 ; S 2 ;

#  0 T 1 T Z 1 T  G  1B 0 T 2 T Z 2

" and

M≔

Notice that 2

A 6 Q M ¼4 ~1 ~2 Q

0  AT1 0

0

3

2

6 0 7 5 þ 4 I n1 I n2  AT2

i ¼ 1; 2;

3 B h  T 7 0 T 1 Z 1 5G  1 Z;  0 T 2 T Z 2

T

B1 ;

i T B2 :

A

S

Q

 A2

T

# :

2674

J. Engwerda / Journal of the Franklin Institute 351 (2014) 2657–2674

References [1] H. Abou-Kandil, G. Freiling, G. Jank, Necessary and sufficient conditions for constant solutions of coupled Riccati equations in Nash games, Syst. Control Lett. 21 (1993) 295–306. [2] E. Altman, T. Başar, Multiuser rate-based flow control, IEEE Trans. Commun. 46 (1998) 940–949. [3] T. Başar, P. Bernhard, H 1 -Optimal Control and Related Minimax Design Problems, Birkhäuser, Boston, 1995. [4] T. Başar, G.J. Olsder, Dynamic Noncooperative Game Theory, SIAM, Philadelphia, 1999. [5] T. Başar, Q. Zhu, Prices of anarchy, information, and cooperation in differential games, Dyn. Games Appl. 1 (2011) 50–73. [6] M.R. Caputo, The envelope theorem for locally differentiable Nash equilibria of finite horizon differential games, Games Econ. Behav. 61 (2007) 198–224. [7] E. Dockner, S. Jørgensen, N.V. Long, G. Sorger, Differential Games in Economics and Management Science, Cambridge University Press, Cambridge, 2000. [8] T. Eisele, Nonexistence and nonuniqueness of open-loop equilibria in linear-quadratic differential games, J. Optim. Theory Appl. 37 (1982) 443–468. [9] J.C. Engwerda, On the open-loop Nash equilibrium in LQ-games, J. Econ. Dyn. Control 22 (1998) 729–762. [10] J.C. Engwerda, Computational aspects of the open-loop Nash equilibrium in LQ-games, J. Econ. Dyn. Control 22 (1998) 1487–1506. [11] J.C. Engwerda, LQ Dynamic Optimization and Differential Games, John Wiley & Sons, Chichester, 2005. [12] J.C. Engwerda, Uniqueness conditions for the affine open-loop linear quadratic differential game, Automatica 44 (2008) 504–511. [13] J.C. Engwerda, Open-Loop Nash Equilibria in the Non-cooperative Infinite-planning Horizon LQ Game, in: Proceedings of the 15th IFAC Workshop on Control Applications of Optimization, University of Bologna, Rimini, Italy, September 13–16, 2012. CD-Rom. Also available at IFAC-PapersOnline: http://www.ifac-papersonline.net/. [14] M. Feucht, Linear-quadratische Differentialspiele und gekoppelte Riccatische Matrixdifferentialgleichungen (Ph.D. thesis), Universität Ulm, Germany, 1994. [15] D.-B. Gu, A differential game approach to formation control, IEEE Trans. Control Syst. Technol. 16 (2008) 85–93. [16] O. Hájek, Pursuit Games: An Introduction to the Theory and Applications of Differential Games of Pursuit and Evasion, Dover Publications, 2008. [17] A. Haurie, G. Leitmann, On the global asymptotic stability of equilibrium solutions for open-loop differential games, Large Scale Syst. 6 (1984) 107–122. [18] A. Haurie, J.B. Krawczyk, G. Zaccour, Games and Dynamic Games, World Scientific, 2012. [19] M. Jiminez-Lizórraga, L. Fridman, Robust Nash strategies based on integral sliding mode control for a two players uncertain linear affine-quadratic game, Int. J. Innovat. Comput. Inf. Control 5 (2009) 241–251. [20] S. Jørgensen, G. Zaccour, Differential Games in Marketing, Kluwer, 2004. [21] D. Kremer, Non-symmetric Riccati theory and noncooperative games (Ph.D. thesis), RWTH-Aachen, Germany, 2002. [22] D. Li, G. Chen, E. Blasch, K. Pham, Sensor attack avoidance: linear quadratic game approach, in: 12th International Conference on Information Fusion, Seattle, WA, USA, July 6–9, 2009. [23] D. Li, J.B. Cruz, Defending an asset: a linear quadratic game approach, IEEE Trans. Aerosp. Electron. Syst. 47 (2009) 1026–1044. [24] N.V. Long, A Survey of Dynamic Games in Economics, World Scientific, 2010. [25] D.L. Lukes, D.L. Russell, Linear-quadratic games, J. Math. Anal. Appl. 33 (1971) 96–123. [26] H.-B. Meyer, The matrix equation AZ þ B ZCZ  ZD ¼ 0, SIAM J. Appl. Math. 30 (1976) 136–142. [27] J. Plasmans, J. Engwerda, B. van, Aarle, B. Di Bartolomeo, T. Michalak, Dynamic modeling of Monetary and Fiscal Cooperation Among Nations, Series: Dynamic Modeling and Econometrics in Economics and Finance, vol. 8, Springer, Berlin, 2006. [28] A.W. Starr, Y.C. Ho, Nonzero-sum differential games, J. Optim. Theory Appl. 3 (1969) 184–206. [29] K. Zhou, J.C. Doyle, K. Glover, Robust and Optimal Control, Prentice Hall, New Jersey, 1996.