- Email: [email protected]
Stability and stabilization of a class of finite evolutionary games
Available online at www.sciencedirect.com
Journal of the Franklin Institute ] (]]]]) ]]]–]]] www.elsevier.com/locate/jfranklin
Stability and stabilization of a class of finite evolutionary games Yuanhua Wanga,b,n, Daizhan Chenga a
School of Control Science and Engineering, Shandong University, Jinan 250061, PR China College of Physics and Electronic Engineering, Qilu Normal University, Jinan 250013, PR China
b
Received 25 September 2015; received in revised form 23 March 2016; accepted 6 December 2016
Abstract The stability and stabilization of a class of finite evolutionary games, called the Markov-type evolutionary games (MtEGs), are studied by using the Lyapunov-based technique. First, an easily verifiable necessary and sufficient condition is obtained for the global stability of MtEGs, and an algorithm is presented to construct a Lyapunov function via solving a set of equality-inequalities. Second, the MtEGs with timevarying payoffs are considered and a common Lyapunov function for its global stability is explored. Finally, the stabilization of controlled MtEGs is investigated, and the corresponding conditions are revealed to assure the global stabilization under fixed and time-varying payoffs respectively. & 2016 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.
1. Introduction Evolutionary games (EG) were first developed by evolutionary biologists for explaining many complex and challenging aspects of biology [1–3]. The fundamental concept of evolutionary game (EG) was introduced by Smith and Price [4]. The EG provides a proper model for population dynamics [5,6], and it has found many applications in economical systems [7], social systems [8], engineering systems [9], etc. An important problem in evolutionary game theory is the stability of strategic behavior [10]. The evolutionary stability indicates whether an evolutionary process converges to certain pren
Corresponding author at: School of Control Science and Engineering, Shandong University, Jinan 250061, PR China. E-mail addresses: [email protected] (Y. Wang), [email protected] (D. Cheng).
http://dx.doi.org/10.1016/j.jfranklin.2016.12.007 0016-0032/& 2016 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
2
assigned state [11]. The concept of evolutionarily stable strategy (ESS) has been proposed to describe the stability of EGs. An ESS is a strategy that, when adopted by all members of a population, cannot be invaded by a mutant strategy through the operation of natural selection. Much of the literature shows that the ESS provides a sufficient condition for local stability under the replicator dynamics [12,13]. As we know, the Lyapunov-based approaches provide a powerful framework for analyzing the stability and control designs of nonlinear dynamical systems [14]. Particularly, its applications to evolutionary games are also increasing. For instance, Hofbauer and Sandholm have studied a class of population games called stable game and constructed a suitable Lyapunov function to assure the global stability [15]. Sandholm has studied the local stability and proved that any regular ESS is locally asymptotically stable by modifying the Lyapunov function [16]. Araujo and Moreira have studied the local asymptotical stability of equilibriums in an evolutionary game model of the labor market by using Lyapunov method [17]. These results showed that the Lyapunov-based approaches are also powerful in investigating the stability of EGs. However, the Lyapunov functions constructed so far are problem-depending. There are short of discussions on general Lyapunov functions for k-valued logical dynamic systems. An EG with time-varying payoffs is a natural generalization of EG with fixed payoffs. For instance in a market competition, because the prices of products are time-varying, and then so are the profits (payoffs). Time-varying payoffs may happen also because of the environment change. For instance, in the problem of distributed coverage of graphs, the payoff depends on the location of mobile agent [18]. However, to the best of our knowledge, there are only very limited results on studying the stability of the EGs with time-varying payoffs due to the short of proper mathematical tools. Recently, a new mathematical tool, called the semi-tensor product of matrices, was introduced [19], which is a generalization of the conventional matrix product to two arbitrary matrices. It has been successfully applied to the analysis and control design of Boolean networks [20–23], graph theory [24], networked evolutionary games [25,26], etc. In this paper, we investigate the stability and stabilization of MtEGs by using the Lyapunovbased approach. The main contributions of this paper are as follows: (1) a necessary and sufficient condition to assure the global stability of MtEGs is proposed; (2) for a general k-valued logical dynamic system, an algorithm is obtained to produce a general k-valued pseudo-logical function as its Lyapunov function; and (3) a common Lyapunov function is presented to assure the global stability of an MtEG with time-varying payoffs. The remainder of this paper is organized as follows. Section 2 provides some necessary preliminaries. Section 3 investigates the global stability of MtEGs. Section 4 discusses the stability of MtEGs with time-varying payoffs. Section 5 considers the stabilization of controlled MtEGs. Section 6 is a brief conclusion. 2. Preliminaries For statement ease, we first give some notations:
Mmn : the set of m n real matrices. ColðMÞ: the set of columns of M; Coli ðMÞ is the i-th column of M. Dk ≔f1; 2; …; kg; k Z 2. δin : the i-th column of the identity matrix In. Δn ≔ δin ji ¼ 1; …; n . Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
3
A matrix LA Mmn is called a logical matrix if the columns of L, denoted by ColðLÞ, are of the form δkm , 1r k r m. That is, ColðLÞ Δm :
Then Lmn denotes the set of m n logical matrices. If L A Lmn , it can be expressed as L ¼ ½δim1 ; …; δimn . For the sake of compactness, it is briefly denoted as L ¼ δm ½i1 ; i2 ; …; in :
1q ≔ð1; 1; …; 1 ÞT . |fflfflfflfflffl{zfflfflfflfflffl} q
In this paper, the mathematical tool we use is the semi-tensor product of matrices. We refer to [19] for more details. Definition 2.1. Let M AMmn , N A Mpq , and t ¼ lcmfn; pg be the least common multiple of n and p. Then the semi-tensor product of M and N is defined as M⋉N≔ M I t=n N I t=p A Mmt=nqt=p ; ð1Þ where is the Kronecker product. The semi-tensor product of matrices is a generalization of conventional matrix product, and it keeps all the properties of the conventional matrix product available. Hence we can omit the symbol “⋉” mostly. The semi-tensor product of matrices has the following properties: Proposition 2.2. Let X A Rm be a column and M be any matrix. Then X⋉M ¼ ðI m M ÞX:
ð2Þ
Proposition 2.3. Let X A Δp and define a power reducing matrix ORp ≔δp2 ½1; p þ 2; 2p þ 3; …; p2 A Lp2 p . Then X 2 ¼ ORp X: The following result is fundamental for the matrix expression of the pseudo-logical (or logical) function. Theorem 2.4. Let f : Dnk -R (or f : Dnk -Dk ) be a k-valued pseudo-logical (or logical) function. Then there exists a unique matrix M f A R1kn (or M f A Lkkn ), such that in vector form we have f ðx1 ; x2 ; …; xn Þ ¼ M f ⋉ni ¼ 1 xi ; where xi A Dk , i ¼ 1; …; n; and Mf is called the structure matrix of f. Definition 2.5. Let M A Mpm , N A Mqm . Then the Khatri–Rao Product is defined as M nN ¼ ½Col1 ðMÞ⋉Col1 ðNÞ⋯Colm ðMÞ⋉Colm ðNÞ: Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
4
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
Proposition 2.6. Let u : Dnk -Dp and v : Dnk -Dq be expressed in algebraic form as u ¼ M u ⋉ni ¼ 1 xi ;
v ¼ M v ⋉ni ¼ 1 xi ;
where M u A Lpkn and M v A Lqkn . Then uv ¼ ðM u nM v Þ⋉ni ¼ 1 xi :
ð3Þ
A finite noncooperative game is defined as follows. Definition 2.7. Consider a finite noncooperative game G ¼ ðN; S; CÞ, which has three fundamental ingredients [27]: (i) N ¼ f1; 2; …; ng is the set of players. (ii) S ¼ ∏ni ¼ 1 Si is called the strategy profile, where Si ¼ f1; 2; …; kg is the set of strategies for player i, iA N. The strategies of all players but the i-th one is denoted by s i A S i ≔∏j a i Sj . (iii) C ¼ ðc1 ; …; cn Þ A Rn with ci : S-R defined as ci ≔ci ðx1 ; …; xn Þ ¼ V ci ⋉nj ¼ 1 xj ;
xj A Sj ; iA N;
ð4Þ
is the payoff function of player i. A strategy profile sn ¼ ðsn1 ; …; snn Þ is a Nash equilibrium if ci ðsni ; sn i ÞZ ci ðxi ; sn i Þ;
8 xi A Si ; iA N:
ð5Þ
Consider a finite game G ¼ ðN; S; CÞ played repetitively and denote it by G1 . Assume that its strategy profile dynamics can be described as 8 > < x1 ðt þ 1Þ ¼ f 1 ðx1 ðtÞ; …; xn ðtÞÞ ⋮ ð6Þ > : x ðt þ 1Þ ¼ f ðx ðtÞ; …; x ðtÞÞ; n n n 1 where xi ðtÞA Dk is the strategy of player i at time t, and f i : Dnk -Dk is the k-valued logical function. Then G1 is called an Markov-type evolutionary game (MtEG). 3. Stability of Markov-type evolutionary games For the dynamics (6), if we use the vector form to express logical variables, that is, Dk Δk (precisely, i δik , i ¼ 1; …; k, and hence xi ðtÞA Δk ), then we have the algebraic expression of Eq. (6) as follows: xðt þ 1Þ ¼ MxðtÞ;
ð7Þ
xðtÞ ¼ ⋉ni ¼ 1 xi ðtÞ
where and M A Lkn kn is called the transition matrix. It is easy to see that the evolutionary dynamics (7) is a standard k-valued logical dynamic system. Then we have the following result. Theorem 3.1 ([25]). For a k-valued logical dynamic system given by Eq. (7), δikn is its fixed point, if and only if the diagonal element mii of M equals 1. To deal with the stability of MtEG, the strategy updating rule (SUR) needs to be considered first. The SUR determines the strategy profile dynamic, hence influences the properties of an Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
5
MtEG. In this paper we only consider the myopic best response adjustment rule (MBRAR) [28], which is described as follows. Construct a set of optimal response strategies at time t as BRi ðtÞ ¼ argmax ci ðsi ; s i ðtÞÞ: si A Si
Then
(Case 1) If xi ðtÞ A BRi ðtÞ, then xi ðt þ 1Þ ¼ xi ðtÞ; (Case 2) If xi ðtÞ= 2 BRi ðtÞ, then choose one corresponding to a priority. For instance, xi ðt þ 1Þ ¼ minfξjξA BRi ðtÞg. As for the updating moments, three types are considered:
(Type 1) Simultaneous Update: All the players update their strategies simultaneously. (Type 2) Random Update: At each moment a player is chosen randomly to update his strategy, and all other players keep their strategies unchanged. (Type 3) Cascading Update: If ioj, then i will update its strategy before j. Hence, when player j updates his strategy he knows and can use the ioj player's updated information. That is, xi ðt þ 1Þ ¼ f i ðx1 ðt þ 1Þ; …; xi 1 ðt þ 1Þ; …; xn ðtÞÞ:
Next, we investigate the global stability of finite MtEGs. That is, under a given SUR, starting from any strategy profile the trajectory will converge to a strategy profile xn, which might be a Nash equilibrium. This paper concerns only the global stability. First, a Lyapunov function for finite MtEGs is defined as follows. Definition 3.2. A pseudo-logical function ψðxÞ : Dnk -R is called a Lyapunov function of the MtEG (6) if ψðxðt þ 1ÞÞ ψðxðtÞÞ Z 0;
t Z 0;
ð8Þ
and ψðxðt þ 1ÞÞ ¼ ψðxðtÞÞ ) xðt þ 1Þ ¼ xðtÞ ¼ xn ;
ð9Þ
n
where x is a pre-assigned point. Based on this definition, we give a necessary and sufficient condition for the global stability of finite MtEG (6). Theorem 3.3. The finite MtEG (6) is globally stable at xn, if and only if there exists a Lyapunov function ψðxÞ : Dnk -R. To prove this theorem, we give a lemma first. Lemma 3.4. Assume that ψ is a Lyapunov function with xn satisfying Eq. (9). Then ψðxn Þ ¼ maxn ψðxÞ: x A Dk
ð10Þ
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
6
Proof. Assume that Eq. (10) is not true. Since Dnk is a finite set, then there exists an x0 A Dnk and x0 a xn such that ψðx0 Þ ¼ maxx A Dnk ψðxÞ. Set xðtÞ ¼ x0 , then ψðxðt þ 1ÞÞ Z ψðxðtÞÞ. Because ψðxðtÞÞ ¼ ψðx0 Þ is the maximum, so ψðxðt þ 1ÞÞ ¼ ψðxðtÞÞ, it follows that xðt þ 1Þ ¼ xðtÞ ¼ x0 ¼ xn , which is a contradiction. □ Proof of Theorem 3.3 (Sufficiency). First, we prove that xn is a fixed point. Assume xðtÞ ¼ xn and xðt þ 1Þ ¼ ξa xn . Then, Eq. (9) yields that ψðξÞ4ψðxn Þ, which is a contradiction to Lemma 3.4. In fact, Eq. (9) also assures that xn is the unique fixed point. Now we prove that all the trajectories will converge to xn. Assume that we have a sequence of profiles in a trajectory as: xð0Þ-xð1Þ-xð2Þ-⋯-xðqÞ-⋯: If there is an xðiÞ ¼ xn , then xðrÞ ¼ xn , 8 r4i. The convergence is obvious. Otherwise, we have a strictly increasing infinite sequence as ψðxð0ÞÞoψðxð1ÞÞo⋯oψðxðqÞÞo⋯:
ð11Þ
It was shown in the proof of Lemma 3.4 that this is impossible. (Necessity). Assume that each profile xA S will converge to xn. Since N is finite, x will converge to xn in finite steps. For each x A S, if it will converge to xn in r steps, we define ψðxÞ≔ r: It is ready to verify that ψ meets the requirements of a Lyapunov function.
□
According to Theorem 2.4, it is easy to verify that in vector form we have ψðxÞ ¼ V ψ ⋉ni ¼ 1 xi ðtÞ; where V ψ A R1kn is called the structure vector of ψðxÞ. Set V ψ ¼ ½a1 ; a2 ; …; akn , and assume that the matrix M of Eq. (6) is M ¼ δkn ½i1 ; i2 ; …; ikn . Then we have Proposition 3.5. The finite MtEG (6) has a Lyapunov function if and only if there exists an integer 1r r r kn , such that the following set of equality-inequalities (12) has a solution fa1 ; a2 ; …; akn g: ( aiq 4aq ; q ¼ 1; …; kn ; qa r; ð12Þ air ¼ ar : Proof (Sufficiency). According to Eq. (12), it is easy to verify that ψðxðtÞÞ ¼ V ψ ⋉xðtÞ satisfies the conditions (8) and (9) of a Lyapunov function in Definition 3.2. (Necessity). Assume that the finite MtEG (6) has a Lyapunov function ψðxÞ satisfying the conditions (8) and (9), and its algebraic form is ψðxÞ ¼ V ψ x ¼ ½a1 ; a2 ; …; akn x. When xðtÞ ¼ xn ¼ δrkn , we have ΔψðxðtÞÞ ¼ ψðxðt þ 1ÞÞ ψðxðtÞÞ ¼ V ψ MxðtÞ V ψ xðtÞ ¼ ½a1 ; …; akn ⋉δkn ½i1 ; …; ikn xðtÞ ½a1 ; …; akn xðtÞ Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
7
¼ ½ai1 ; …; aikn δrkn ½a1 ; …; akn δrkn ¼ air ar ¼ 0: So we can obtain the equality air ¼ ar . When xðtÞa xn , ΔψðxðtÞÞ40, it is easy to verify the following inequalities: aiq 4aq ;
q ¼ 1; …; kn ; q a r:
□
Proposition 3.5 provides an algorithm to construct a Lyapunov function ψðxÞ for the MtEG (6). Algorithm 3.6. Consider the MtEG (6) and assume ψðxðtÞÞ ¼ V ψ xðtÞ ¼ ½a1 ; a2 ; …; akn xðtÞ, then we take the following steps to construct ψðxÞ: 1. Compute the transition matrix M, and find out the unique fixed point; 2. Solve the equality-inequalities (12) to find a solution fa1 ; a2 ; …; akn g; 3. Obtain the Lyapunov function ψðxðtÞÞ ¼ ½a1 ; a2 ; …; akn xðtÞ: Remark 3.7. The set of equality-inequalities (12) has infinite number of solutions, so the Lyapunov function of the MtEG (6) is not unique.
Remark 3.8. The limitation of this algorithm is the computational complexity. It is easy to see that the computational complexity depends on the computation of matrix M. when the dimensions of matrices are large, the proposed algorithm can hardly be used. Then an efficient numerical method has to be developed to deal with it. We give an example to explain how to construct a Lyapunov function of an MtEG. Example 3.9 ([29]). Consider a two-player game with payoffs given in Table 1. Each player has three strategies (1: work; 2: shirk at office; 3: shirk at home). In this game, if both players work, then a project succeeds and they each receive a payoff of 90. If only one of them works, then the project may succeed and they each receive 48, or it may fail with a payoff of 12. If none of them work and they shirk at different places, then each receives a payoff of 24. On the other hand, if both shirk at the same place, then the row player has unit payoff, whereas the other player has 1. According to the MBRAR, it is easy to figure out the best response strategy fi for each player, which is listed in Table 2. Table 1 Payoff matrix. P1 ⧹P2
1
2
3
1 2 3
ð90; 90Þ ð 12; 12Þ ð48; 48Þ
ð 12; 12Þ ð1; 1Þ ð24; 24Þ
ð48; 48Þ ð24; 24Þ ð1; 1Þ
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
8 Table 2 Best response strategy. c⧹s
11
12
13
21
22
23
31
32
33
f1 f2
1 1
3 1
1 1
1 3
3 3
1 3
1 1
3 1
1 1
Using the cascading MBRAR, we obtain that x1 ðt þ 1Þ ¼ δ3 ½1; 3; 1; 1; 3; 1; 1; 3; 1xðtÞ ¼ M 1 xðtÞ; x2 ðt þ 1Þ ¼ δ3 ½1; 1; 1; 3; 3; 3; 1; 1; 1x1 ðt þ 1Þx2 ðtÞ ¼ M 2 x1 ðt þ 1Þx2 ðtÞ ¼ M 2 M 1 x1 ðtÞx2 ðtÞx2 ðtÞ ¼ M 2 M 1 x1 ðtÞOR3 x2 ðtÞ ¼ M 2 M 1 I 3 OR3 x1 ðtÞx2 ðtÞ ~ 2 xðtÞ: ≔M Then we have the profile dynamics as ~ 2 ÞxðtÞ xðt þ 1Þ ¼ ðM 1 nM ¼ δ9 ½1; 7; 1; 1; 7; 1; 1; 7; 1xðtÞ ¼ MxðtÞ:
ð13Þ δ19 .
According to Theorem 3.1, we can see that Eq. (13) has a unique fixed point Set M ¼ δ9 ½i1 ; i2 ; …; i9 and V ψ ¼ ½a1 ; a2 ; …; a9 , then we have an equality ai1 ¼ a1 and the following set of linear inequalities: 8 a7 4a2 > > > > > a > 1 4a3 > > > > a 4a4 > > > 1 > < a7 4a5 > a1 4a6 > > > > > a1 4a7 > > > > > a7 4a8 > > > : a 4a 1
9
Obviously, it has infinite number of solutions. If we choose a solution as fa1 ; a2 ; a3 ; a4 ; a5 ; a6 ; a7 ; a8 ; a9 g ¼ f6; 3; 1; 5; 2; 3; 4; 2; 1g, then the algebraic form of this Lyapunov function is as follows: ψðxðtÞÞ ¼ V ψ xðtÞ ¼ ½6; 3; 1; 5; 2; 3; 4; 2; 1xðtÞ: It is easy to verify that the game itself is not potential, but this Lyapunov function assumes that the dynamic game converge to a pure Nash equilibrium.
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
9
Table 3 Payoff bi-matrix of Example 3.11. P1 ⧹P2
1
2
1 2
ð3; 1Þ ð2; 4Þ
ð4; 3Þ ð5; 1Þ
Remark 3.10. 1. An important issue is: what is the relationship between the limitation and the Nash equilibrium. One conclusion is: As long as the strategy profile dynamics is obtained from MBRAR, no matter what type it is, the limitation is also a Nash equilibrium. This conclusion can easily be deduced from the definitions of the related concepts directly. 2. It is easy to see that the above conclusion is also available for the case of mixed strategies, where the dynamic equation (7) has a column random matrix M. 3. Unfortunately, the technique developed in this section for constructing Lyapunov function cannot be applicable to the case of mixed strategies, because for mixed strategies the transition matrix M obtained by MBRAR is in general a state-depending matrix. Refer to the following example:
Example 3.11. Consider a game with two players, which has the payoff bi-matrix as in Table 3 Assume that the mixed strategy adopted by P1 (P2) is xðtÞ ¼ ðx1 ðtÞ; x2 ðtÞÞT (yðtÞ ¼ ðy1 ðtÞ; y2 ðtÞÞT ). Then the expected values of the payoffs are c1 ¼ 3x1 y1 þ 4x1 y2 þ 2x2 y1 þ 5x2 y2 ; c2 ¼ x1 y1 þ 3x1 y2 þ 4x2 y1 þ x2 y2 : Using MBRAR, the profile dynamics is obtained as ( 1 ( 1 δ2 ; y1 ðtÞ4y2 ðtÞ δ2 ; 2x1 ðtÞo3x2 ðtÞ xðt þ 1Þ ¼ ; yðt þ 1Þ ¼ ; δ22 ; y1 ðtÞoy2 ðtÞ δ22 ; 2x1 ðtÞ43x2 ðtÞ where M is the strategy profile-depending.
4. Stability of MtEGs with time-varying payoffs This section considers the global stability of Markov-type evolutionary games with timevarying payoff function. Assume that a finite MtEG is based on G ¼ ðN; S; CðtÞÞ, where the payoff function CðtÞ ¼ ðc1 ðtÞ; …; cn ðtÞÞ is time-varying. Then the algebraic form of ci(t) can be expressed as ci ðtÞ ¼ V ci ðtÞ⋉ni ¼ 1 xi ðtÞ:
ð14Þ
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
10
Now for each C(t), t ¼ 0; 1; 2; …, the strategy profile dynamics can be described as 8 x ðt þ 1Þ ¼ f t1 ðx1 ðtÞ; …; xn ðtÞÞ; > < 1 ⋮ > : x ðt þ 1Þ ¼ f t ðx ðtÞ; …; x ðtÞÞ: n
n
1
ð15Þ
n
where fit, t ¼ 0; 1; …, is the time-varying k-valued logical function. By using semi-tensor product, for each fit, we can get its structure matrix Mi(t). Then the algebraic form of Eq. (15) follows as xðt þ 1Þ ¼ MðtÞxðtÞ;
t ¼ 0; 1; …;
ð16Þ
where MðtÞ ¼ M 1 ðtÞn⋯nM n ðtÞ. Now since Lkn kn is a finite set, MðtÞjt ¼ 0; 1; … contains only finite distinct matrices. Denote this set as: fM 1 ; …; M r g ¼ MðtÞjt ¼ 0; 1; … : ð17Þ Then Eq. (16) can be expressed as a switched system xðt þ 1Þ ¼ M sðtÞ xðtÞ;
t ¼ 0; 1; …;
ð18Þ
where sðtÞ A f1; 2; …; rg: In general, the existence of a common quadratic Lyapunov function is sufficient but not necessary for the stability of a switched system under arbitrary switching [30]. Similarly, we have the following result. Proposition 4.1. Consider the MtEG (15) with time-varying payoff function (14). Then the MtEG is globally stable at xn if all the r models in Eq. (18) share a common Lyapunov function ψðxÞ with a common fixed point xn. Proof. No matter which model is active, the common Lyapunov function is strictly increasing until xn is reached. Now it is clear that xn ¼ argmax ψðxÞ; xAS
which is unique. Therefore, all the trajectories will converge to xn.
□
Denote the transition matrices M j ¼ δkn i1j ; i2j ; …; ikjn ; j ¼ 1; …; r: Applying Proposition 3.5 to each model, the following result is obvious. Table 4 Payoff bi-matrix of Example 4.3. P1 ⧹P2
1
2
1 2
sin ðtπ=2Þ; sin ðtπ=2Þ 1 1
1; 1 1; 1
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
11
Corollary 4.2. Consider an MtEG based on G ¼ ðN; S; CðtÞÞ. There exists a common Lyapunov function if there exists an xn ¼ δμkn such that Colμ ðM j Þ ¼ xn ;
(i)
j ¼ 1; …; r;
ð19Þ
(ii) there exist a set of parameters fa1 ; a2 ; …; akn g, such that aijq 4aqj ; q ¼ 1; …; kn ; qa μ; j ¼ 1; …; r:
ð20Þ
Corollary 4.2 provides a method to verify the existence and construction of a common Lyapunov function. We give an example to illustrate it. Example 4.3. Assume that an MtEG has the payoff bi-matrix as in Table 4. Assume that the simultaneous MBRAR is used. Then it is easy to verify that there are only r¼ 3 different models at all time moments: (i) t ¼ 4s or t ¼ 4s þ 2, s ¼ 0; 1; 2; …: M 1 ¼ δ4 ½4; 4; 4; 4; (ii) t ¼ 4s þ 1, s ¼ 0; 1; 2; …: M 2 ¼ δ4 ½2; 4; 4; 4; (iii) t ¼ 4s þ 3, s ¼ 0; 1; 2; …: M 3 ¼ δ4 ½3; 4; 4; 4: n
Setting x ¼ δ44 and taking a1 ¼ 2;
a2 ¼ 1;
a3 ¼ 1;
a4 ¼ 0;
it is ready to verify that the requirements in Corollary 4.2 are satisfied. That is, ψðxÞ ¼ ½ 2; 1; 1; 0x1 x2 is a common Lyapunov function for the MtEG with time-varying payoffs. 5. Stabilization of controlled MtEGs In this section, we study the global stabilization problem of controlled MtEGs, which are defined as follows: Definition 5.1. Consider a finite game G ¼ ðN; S; CÞ played repetitively. Assume that the set of players can be divided into two parts as N ¼ fX; Ug, where X ¼ fx1 ; …; xn g are called the state players that follow certain SUR to update their strategies; and U ¼ fu1 ; …; um g are called the control players, that can select their strategies freely. Now assume jSi j ¼ k, 8i A N, following the same procedure as used for (not controlled) MtEG in Section 3, the strategy profile dynamics can be obtained as xðt þ 1Þ ¼ MuðtÞxðtÞ; where
xðtÞ ¼ ⋉ni ¼ 1 xi ðtÞ,
uðtÞ ¼ ⋉m j ¼ 1 uj ðtÞ,
ð21Þ M A Lkn kmþn .
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
12
In fact, Eq. (21) is a standard k-valued logical dynamic system. So the following result is applicable: Theorem 5.2 ([19]). The MtEG (21) is globally stabilizable, if and only if (i) there exists a profile xn, which is reachable from anywhere; (ii) there exists a control un, such that xn is a fixed point of Mun. Definition 5.3. A pseudo-logical function ψðxÞ : Dnk -R is called a control Lyapunov function of the MtEG (21) if there exists an xn such that (i) for each xa xn there exists ux such that V ψ ðMux I kn Þx40;
ð22Þ
(ii) for xn there exists un such that V ψ ðMun I kn Þxn ¼ 0:
ð23Þ
Similar to Theorem 3.3, we can prove the following result. Theorem 5.4. The controlled MtEG is globally stabilizable, if and only if there exists a control Lyapunov function.
Proof (Sufficiency). Assume that there exists a control Lyapunov function ψðxÞ satisfying the conditions (22) and (23), now we need to prove that the controlled MtEG satisfies (i)–(ii) of Theorem 5.2. The algebraic form of ψðxÞ can be expressed as ψðxÞ ¼ V ψ x, then we have ψðxðt þ 1ÞÞ ψðxðtÞÞ ¼ V ψ xðt þ 1Þ V ψ xðtÞ ¼ V ψ MuðtÞxðtÞ V ψ xðtÞ ¼ V ψ ðMuðtÞ I kn ÞxðtÞ: From the condition (23), one sees that xn is a fixed point of Mun. And from the condition (22), it is easy to verify that all the ðuðtÞ; xðtÞÞ trajectories can reach a fixed point ðun ; xn Þ, where ψðun ; xn Þ ¼
max
n u A Dm k ;x A Dk
ψðu; xÞ:
The sufficiency is proved. (Necessity). Let q be the shortest time for x to reach xn with proper controls. Then we define ψðxÞ ¼ q. It is ready to verify that this ψ is a control Lyapunov function. □ Next, we briefly discuss how to select the free controls for the global stabilization. For the condition (i) in Theorem 5.2, there must be a q40 such that all the trajectories from any initial profile x converge to xn ¼ δμkn in q steps. According to Eq. (21), we have xn Muðq 1ÞMuðq 2Þ⋯Muð0Þx;
8x A Δkn :
This is equivalent to ColðMuðq 1ÞMuðq 2Þ⋯Muð0ÞÞ ¼ fδμkn g:
ð24Þ
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
13
Now we can calculate Muðq 1ÞMuðq 2Þ⋯Muð0Þ ¼ MðI km MÞ⋯ðI kðq 1Þm MÞ⋉0i ¼ q 1 uðiÞ ≔½M q1 ; ⋯; M qkqm ⋉0i ¼ q 1 uðiÞ: It is clear that if there is a 1 r jr kqm such that Mqj satisfies ColðM qj Þ ¼ fδμkn g, then we can choose the control ⋉0i ¼ q 1 uðiÞ ¼ δjkqm ; such that Eq. (24) holds. For the condition (ii) in Theorem 5.2, we can choose a control un A Δkm such that Mun xn ¼ xn . That is, when t Z q, the control remains to be uðtÞ ¼ un . Now we give an example. Example 5.5. Recall Example 3.9. Let u≔P1 be the control and z≔P2 be the state. Assume that the SUR is the simultaneous MBRAR. Then from Table 2 we have the strategy profile dynamics as zðt þ 1Þ ¼ δ3 ½1; 1; 1; 3; 3; 3; 1; 1; 1uðtÞzðtÞ≔MuðtÞzðtÞ: When q¼ 1, we have M ¼ ½ M 11 ; M 12 ; M 13 ¼ δ3 ½1; 1; 1; 3; 3; 3; 1; 1; 1: It follows that ColðM 11 Þ ¼ ColðM 13 Þ ¼ fδ13 g: Now it is clear that we can choose uð0Þ ¼ δ13 1 or uð0Þ ¼ δ33 3 such that the game is globally stabilized to zn ¼ δ13 . In addition, we select uðtÞ ¼ un ¼ δ13 or uðtÞ ¼ un ¼ δ33 for t Z 1, such that Mun zn ¼ zn . Hence we set ψðzÞ ¼ V ψ z ¼ ½0; 1; 1z, which satisfies the requirements of a control Lyapunov function. Finally, we consider the case when the payoffs are time-varying. Let the controlled MtEG be determined by G ¼ ððX; UÞ; S; CðtÞÞ. Similar to the uncontrolled case, corresponding to each CðtÞ; t ¼ 0; 1; 2; …, we can figure out the corresponding transition matrices MðtÞ; t ¼ 0; 1; 2; …. Express the distinct transition matrices as: MðtÞjt ¼ 0; 1; 2; ⋯ ≔fW 1 ; W 2 ; …; W s g: Then it is easy to verify the following result: Proposition 5.6. The controlled MtEG based on G ¼ ððX; UÞ; S; CðtÞÞ is globally stabilizable, if there exists a common control Lyapunov function for all the models xðt þ 1Þ ¼ W j uðtÞxðtÞ, j ¼ 1; …; s, with the same (control-depending) fixed point xn.
Example 5.7. Recall Example 4.3. Let u≔x1 be the control and z≔x2 be the state. Then we have only two different models for the controlled MtEG with time-varying payoffs, zðt þ 1Þ ¼ δ2 ½2; 2; 2; 2uðtÞzðtÞ ¼ W 1 uðtÞzðtÞ; zðt þ 1Þ ¼ δ2 ½1; 2; 2; 2uðtÞzðtÞ ¼ W 2 uðtÞzðtÞ: Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
14
When q ¼ 1, we have that W 1 ¼ ½ðW 1 Þ11 ; ðW 1 Þ12 ¼ δ2 ½2; 2; 2; 2; W 2 ¼ ½ðW 2 Þ11 ; ðW 2 Þ12 ¼ δ2 ½1; 2; 2; 2: Similar to the analysis of Example 5.5, we can choose uð0Þ ¼ δ22 such that the controlled MtEG with time-varying payoffs is globally stabilized to the common fixed point zn ¼ δ22 . On the other hand, choosing uðtÞ ¼ un ¼ δ12 or uðtÞ ¼ un ¼ δ22 for t Z 1, we have W 1 un zn ¼ zn and W 2 un zn ¼ zn . Therefore, we can choose ψðzÞ ¼ V ψ z ¼ ½ 1; 0z as the common control Lyapunov function for the two models.
6. Conclusion The stability and stabilization of a class of finite evolutionary games were investigated in this paper. First, the Lyapunov function for MtEGs was defined. It was proved that the global stability of an MtEG is equivalent to the existence of a Lyapunov function. And the construction of Lyapunov functions was also obtained. Then the stability of MtEGs with time-varying payoffs was discussed. As a sufficient condition, the common Lyapunov function was proposed. Using the stability results, the stabilization of controlled MtEGs was then discussed. It was proved that the existence of a control Lyapunov function is necessary and sufficient for the stabilization. Finally, the common control Lyapunov function was used to assure the stabilization of controlled MtEGs with time-varying payoffs. Much of the literature focuses on global stability of equilibrium that ensures the convergence to the equilibrium regardless of players’ initial behavior. Inevitably, these requirements on payoff structure are quite demanding. Therefore, one can turn to study local stability of equilibrium, seeking conditions under which an equilibrium is robust to small changes in players’ behavior. The approach proposed in this paper can be extended to local stability/stabilization of evolutionary games as long as the Lyapunov function is locally defined. Constructing a Lyapunov function to assure the convergence to a mixed Nash equilibrium remains for further investigation. Acknowledgments This work is supported partly by the National Natural Science Foundation of China under Grants 61374025 and 61273013. References [1] [2] [3] [4] [5] [6] [7] [8]
R.A. Fisher, The Genetic Theory of Natural Selection, Clarendon Press, Oxford, 1930. W.D. Hamilton, The genetical evolution of social behaviour, J. Econ. Theory 7 (1964) 1–16. R.L. Trivers, The evolution of reciprocal altruism, Q. Rev. Biol. 46 (1) (1971) 35–57. J.M. Smith, G.R. Price, The logic of animal conflict, Nature 246 (1973) 15–18. P. Taylor, L. Jonker, Evolutionary stable strategies and game dynamics, Math. Biosci. 16 (1978) 76–83. J. Hofbauer, K. Sigmund, Evolutionary Games and Population Dynamics, Cambridge University Press, Cambridge, 1998. R. Sugden, The Economics of Rights, Cooperation and Welfare, Basil Blackwell, Oxford, 1986. H. Ohtsuki, C. Hauert, E. Lieberman, M.A. Nowak, A simple rule for the evolution of cooperation on graphs and social networks, Nature 441 (2006) 502–505.
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
15
[9] E. Mojica-Nava, C.A. Macana, N. Quijano, Dynamic population games for optimal dispatch on hierarchical microgrid control, IEEE Trans. Syst. Man Cybern.: Syst. 44 (3) (2014) 306–317. [10] G. Szabo, G. Fath, Evolutionary games on graphs, Phys. Rep. 446 (4) (2007) 97–216. [11] S. Suzuki, E. Akiyama, Evolutionary stability of first-order-information indirect reciprocity in sizable groups, Theor. Popul. Biol. 73 (2008) 426–436. [12] D. Fudenberg, D.K. Levine, The Theory of Learning in Games, MIT Press, Cambridge, MA, 1998. [13] J. Hofbauer, K. Sigmund, Evolutionary game dynamics, Bull. Am. Math. Soc. 40 (4) (2003) 479–519. [14] S.G. Nersesov, M.M. Haddad, On the stability and control of nonlinear dynamical systems via vector Lyapunov functions, IEEE Trans. Autom. Control 51 (2) (2006) 203–215. [15] J. Hofbauer, W.H. Sandholm, Stable games and their dynamics, J. Econ. Theory 144 (4) (2009) 1665–1693. [16] W.H. Sandholm, Local stability under evolutionary game dynamics, Theor. Econ. 5 (1) (2010) 27–50. [17] R.A. Araujo, H.N. Moreira, Lyapunov stability in an evolutionary game theory model of the labor market, EconomiA 15 (1) (2014) 41–53. [18] A.Y. Yazicioglu, M. Egerstedt, J.S. Shamma, A game theoretic approach to distributed coverage of graphs by heterogeneous mobile agents, Estimation and Control of Networked Systems 4 (1) (2013) 309–315. [19] D. Cheng, H. Qi, Z. Li, Analysis and Control of Boolean Networks: A Semi-tensor Product Approach, Springer, London, 2011. [20] F. Li, X. Lu, Z. Yu, Optimal control algorithms for switched Boolean network, J. Frankl. Inst. 351 (6) (2014) 3490–3501. [21] D. Laschov, M. Margaliot, Controllability of Boolean control networks via Perron–Frobenius theory, Automatica 48 (6) (2012) 1218–1223. [22] H. Li, Y. Wang, Consistent stabilizability of switched Boolean networks, Neural Netw. 46 (2013) 183–189. [23] E. Fornasini, M.E. Valcher, Optimal control of Boolean control networks, IEEE Trans. Autom. Control 59 (5) (2014) 1258–1270. [24] Y. Wang, C. Zhang, Z. Liu, A matrix approach to graph maximum stable set and coloring problems with application to multi-agent systems, Automatica 48 (7) (2012) 1227–1236. [25] D. Cheng, F. He, H. Qi, T. Xu, Modeling, analysis and control of networked evolutionary games, IEEE Trans. Autom. Control 60 (9) (2015) 2402–2415. [26] P. Guo, Y. Wang, H. Li, Algebraic formulation and strategy optimization for a class of evolutionary networked games via semi-tensor method, Automatica 49 (11) (2013) 3384–3389. [27] E. Rasmusen, Games and Information: An Introduction to Game Theory, Wiley-Blackwell, Oxford, 2007. [28] H.P. Young, The evolution of conventions, Econometrica 61 (1) (1993) 57–84. [29] O. Candogan, A. Ozdaglar, P.A. Parrilo, Dynamics in near-potential games, Games Econ. Behav. 82 (2013) 66–90. [30] W.P. Dayawansa, C.F. Martin, A converse Lyapunov theorem for a class of dynamical systems which undergo switching, IEEE Trans. Autom. Control 44 (4) (1999) 751–760.
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Journal of the Franklin Institute ] (]]]]) ]]]–]]] www.elsevier.com/locate/jfranklin
Stability and stabilization of a class of finite evolutionary games Yuanhua Wanga,b,n, Daizhan Chenga a
School of Control Science and Engineering, Shandong University, Jinan 250061, PR China College of Physics and Electronic Engineering, Qilu Normal University, Jinan 250013, PR China
b
Received 25 September 2015; received in revised form 23 March 2016; accepted 6 December 2016
Abstract The stability and stabilization of a class of finite evolutionary games, called the Markov-type evolutionary games (MtEGs), are studied by using the Lyapunov-based technique. First, an easily verifiable necessary and sufficient condition is obtained for the global stability of MtEGs, and an algorithm is presented to construct a Lyapunov function via solving a set of equality-inequalities. Second, the MtEGs with timevarying payoffs are considered and a common Lyapunov function for its global stability is explored. Finally, the stabilization of controlled MtEGs is investigated, and the corresponding conditions are revealed to assure the global stabilization under fixed and time-varying payoffs respectively. & 2016 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.
1. Introduction Evolutionary games (EG) were first developed by evolutionary biologists for explaining many complex and challenging aspects of biology [1–3]. The fundamental concept of evolutionary game (EG) was introduced by Smith and Price [4]. The EG provides a proper model for population dynamics [5,6], and it has found many applications in economical systems [7], social systems [8], engineering systems [9], etc. An important problem in evolutionary game theory is the stability of strategic behavior [10]. The evolutionary stability indicates whether an evolutionary process converges to certain pren
Corresponding author at: School of Control Science and Engineering, Shandong University, Jinan 250061, PR China. E-mail addresses: [email protected] (Y. Wang), [email protected] (D. Cheng).
http://dx.doi.org/10.1016/j.jfranklin.2016.12.007 0016-0032/& 2016 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
2
assigned state [11]. The concept of evolutionarily stable strategy (ESS) has been proposed to describe the stability of EGs. An ESS is a strategy that, when adopted by all members of a population, cannot be invaded by a mutant strategy through the operation of natural selection. Much of the literature shows that the ESS provides a sufficient condition for local stability under the replicator dynamics [12,13]. As we know, the Lyapunov-based approaches provide a powerful framework for analyzing the stability and control designs of nonlinear dynamical systems [14]. Particularly, its applications to evolutionary games are also increasing. For instance, Hofbauer and Sandholm have studied a class of population games called stable game and constructed a suitable Lyapunov function to assure the global stability [15]. Sandholm has studied the local stability and proved that any regular ESS is locally asymptotically stable by modifying the Lyapunov function [16]. Araujo and Moreira have studied the local asymptotical stability of equilibriums in an evolutionary game model of the labor market by using Lyapunov method [17]. These results showed that the Lyapunov-based approaches are also powerful in investigating the stability of EGs. However, the Lyapunov functions constructed so far are problem-depending. There are short of discussions on general Lyapunov functions for k-valued logical dynamic systems. An EG with time-varying payoffs is a natural generalization of EG with fixed payoffs. For instance in a market competition, because the prices of products are time-varying, and then so are the profits (payoffs). Time-varying payoffs may happen also because of the environment change. For instance, in the problem of distributed coverage of graphs, the payoff depends on the location of mobile agent [18]. However, to the best of our knowledge, there are only very limited results on studying the stability of the EGs with time-varying payoffs due to the short of proper mathematical tools. Recently, a new mathematical tool, called the semi-tensor product of matrices, was introduced [19], which is a generalization of the conventional matrix product to two arbitrary matrices. It has been successfully applied to the analysis and control design of Boolean networks [20–23], graph theory [24], networked evolutionary games [25,26], etc. In this paper, we investigate the stability and stabilization of MtEGs by using the Lyapunovbased approach. The main contributions of this paper are as follows: (1) a necessary and sufficient condition to assure the global stability of MtEGs is proposed; (2) for a general k-valued logical dynamic system, an algorithm is obtained to produce a general k-valued pseudo-logical function as its Lyapunov function; and (3) a common Lyapunov function is presented to assure the global stability of an MtEG with time-varying payoffs. The remainder of this paper is organized as follows. Section 2 provides some necessary preliminaries. Section 3 investigates the global stability of MtEGs. Section 4 discusses the stability of MtEGs with time-varying payoffs. Section 5 considers the stabilization of controlled MtEGs. Section 6 is a brief conclusion. 2. Preliminaries For statement ease, we first give some notations:
Mmn : the set of m n real matrices. ColðMÞ: the set of columns of M; Coli ðMÞ is the i-th column of M. Dk ≔f1; 2; …; kg; k Z 2. δin : the i-th column of the identity matrix In. Δn ≔ δin ji ¼ 1; …; n . Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
3
A matrix LA Mmn is called a logical matrix if the columns of L, denoted by ColðLÞ, are of the form δkm , 1r k r m. That is, ColðLÞ Δm :
Then Lmn denotes the set of m n logical matrices. If L A Lmn , it can be expressed as L ¼ ½δim1 ; …; δimn . For the sake of compactness, it is briefly denoted as L ¼ δm ½i1 ; i2 ; …; in :
1q ≔ð1; 1; …; 1 ÞT . |fflfflfflfflffl{zfflfflfflfflffl} q
In this paper, the mathematical tool we use is the semi-tensor product of matrices. We refer to [19] for more details. Definition 2.1. Let M AMmn , N A Mpq , and t ¼ lcmfn; pg be the least common multiple of n and p. Then the semi-tensor product of M and N is defined as M⋉N≔ M I t=n N I t=p A Mmt=nqt=p ; ð1Þ where is the Kronecker product. The semi-tensor product of matrices is a generalization of conventional matrix product, and it keeps all the properties of the conventional matrix product available. Hence we can omit the symbol “⋉” mostly. The semi-tensor product of matrices has the following properties: Proposition 2.2. Let X A Rm be a column and M be any matrix. Then X⋉M ¼ ðI m M ÞX:
ð2Þ
Proposition 2.3. Let X A Δp and define a power reducing matrix ORp ≔δp2 ½1; p þ 2; 2p þ 3; …; p2 A Lp2 p . Then X 2 ¼ ORp X: The following result is fundamental for the matrix expression of the pseudo-logical (or logical) function. Theorem 2.4. Let f : Dnk -R (or f : Dnk -Dk ) be a k-valued pseudo-logical (or logical) function. Then there exists a unique matrix M f A R1kn (or M f A Lkkn ), such that in vector form we have f ðx1 ; x2 ; …; xn Þ ¼ M f ⋉ni ¼ 1 xi ; where xi A Dk , i ¼ 1; …; n; and Mf is called the structure matrix of f. Definition 2.5. Let M A Mpm , N A Mqm . Then the Khatri–Rao Product is defined as M nN ¼ ½Col1 ðMÞ⋉Col1 ðNÞ⋯Colm ðMÞ⋉Colm ðNÞ: Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
4
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
Proposition 2.6. Let u : Dnk -Dp and v : Dnk -Dq be expressed in algebraic form as u ¼ M u ⋉ni ¼ 1 xi ;
v ¼ M v ⋉ni ¼ 1 xi ;
where M u A Lpkn and M v A Lqkn . Then uv ¼ ðM u nM v Þ⋉ni ¼ 1 xi :
ð3Þ
A finite noncooperative game is defined as follows. Definition 2.7. Consider a finite noncooperative game G ¼ ðN; S; CÞ, which has three fundamental ingredients [27]: (i) N ¼ f1; 2; …; ng is the set of players. (ii) S ¼ ∏ni ¼ 1 Si is called the strategy profile, where Si ¼ f1; 2; …; kg is the set of strategies for player i, iA N. The strategies of all players but the i-th one is denoted by s i A S i ≔∏j a i Sj . (iii) C ¼ ðc1 ; …; cn Þ A Rn with ci : S-R defined as ci ≔ci ðx1 ; …; xn Þ ¼ V ci ⋉nj ¼ 1 xj ;
xj A Sj ; iA N;
ð4Þ
is the payoff function of player i. A strategy profile sn ¼ ðsn1 ; …; snn Þ is a Nash equilibrium if ci ðsni ; sn i ÞZ ci ðxi ; sn i Þ;
8 xi A Si ; iA N:
ð5Þ
Consider a finite game G ¼ ðN; S; CÞ played repetitively and denote it by G1 . Assume that its strategy profile dynamics can be described as 8 > < x1 ðt þ 1Þ ¼ f 1 ðx1 ðtÞ; …; xn ðtÞÞ ⋮ ð6Þ > : x ðt þ 1Þ ¼ f ðx ðtÞ; …; x ðtÞÞ; n n n 1 where xi ðtÞA Dk is the strategy of player i at time t, and f i : Dnk -Dk is the k-valued logical function. Then G1 is called an Markov-type evolutionary game (MtEG). 3. Stability of Markov-type evolutionary games For the dynamics (6), if we use the vector form to express logical variables, that is, Dk Δk (precisely, i δik , i ¼ 1; …; k, and hence xi ðtÞA Δk ), then we have the algebraic expression of Eq. (6) as follows: xðt þ 1Þ ¼ MxðtÞ;
ð7Þ
xðtÞ ¼ ⋉ni ¼ 1 xi ðtÞ
where and M A Lkn kn is called the transition matrix. It is easy to see that the evolutionary dynamics (7) is a standard k-valued logical dynamic system. Then we have the following result. Theorem 3.1 ([25]). For a k-valued logical dynamic system given by Eq. (7), δikn is its fixed point, if and only if the diagonal element mii of M equals 1. To deal with the stability of MtEG, the strategy updating rule (SUR) needs to be considered first. The SUR determines the strategy profile dynamic, hence influences the properties of an Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
5
MtEG. In this paper we only consider the myopic best response adjustment rule (MBRAR) [28], which is described as follows. Construct a set of optimal response strategies at time t as BRi ðtÞ ¼ argmax ci ðsi ; s i ðtÞÞ: si A Si
Then
(Case 1) If xi ðtÞ A BRi ðtÞ, then xi ðt þ 1Þ ¼ xi ðtÞ; (Case 2) If xi ðtÞ= 2 BRi ðtÞ, then choose one corresponding to a priority. For instance, xi ðt þ 1Þ ¼ minfξjξA BRi ðtÞg. As for the updating moments, three types are considered:
(Type 1) Simultaneous Update: All the players update their strategies simultaneously. (Type 2) Random Update: At each moment a player is chosen randomly to update his strategy, and all other players keep their strategies unchanged. (Type 3) Cascading Update: If ioj, then i will update its strategy before j. Hence, when player j updates his strategy he knows and can use the ioj player's updated information. That is, xi ðt þ 1Þ ¼ f i ðx1 ðt þ 1Þ; …; xi 1 ðt þ 1Þ; …; xn ðtÞÞ:
Next, we investigate the global stability of finite MtEGs. That is, under a given SUR, starting from any strategy profile the trajectory will converge to a strategy profile xn, which might be a Nash equilibrium. This paper concerns only the global stability. First, a Lyapunov function for finite MtEGs is defined as follows. Definition 3.2. A pseudo-logical function ψðxÞ : Dnk -R is called a Lyapunov function of the MtEG (6) if ψðxðt þ 1ÞÞ ψðxðtÞÞ Z 0;
t Z 0;
ð8Þ
and ψðxðt þ 1ÞÞ ¼ ψðxðtÞÞ ) xðt þ 1Þ ¼ xðtÞ ¼ xn ;
ð9Þ
n
where x is a pre-assigned point. Based on this definition, we give a necessary and sufficient condition for the global stability of finite MtEG (6). Theorem 3.3. The finite MtEG (6) is globally stable at xn, if and only if there exists a Lyapunov function ψðxÞ : Dnk -R. To prove this theorem, we give a lemma first. Lemma 3.4. Assume that ψ is a Lyapunov function with xn satisfying Eq. (9). Then ψðxn Þ ¼ maxn ψðxÞ: x A Dk
ð10Þ
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
6
Proof. Assume that Eq. (10) is not true. Since Dnk is a finite set, then there exists an x0 A Dnk and x0 a xn such that ψðx0 Þ ¼ maxx A Dnk ψðxÞ. Set xðtÞ ¼ x0 , then ψðxðt þ 1ÞÞ Z ψðxðtÞÞ. Because ψðxðtÞÞ ¼ ψðx0 Þ is the maximum, so ψðxðt þ 1ÞÞ ¼ ψðxðtÞÞ, it follows that xðt þ 1Þ ¼ xðtÞ ¼ x0 ¼ xn , which is a contradiction. □ Proof of Theorem 3.3 (Sufficiency). First, we prove that xn is a fixed point. Assume xðtÞ ¼ xn and xðt þ 1Þ ¼ ξa xn . Then, Eq. (9) yields that ψðξÞ4ψðxn Þ, which is a contradiction to Lemma 3.4. In fact, Eq. (9) also assures that xn is the unique fixed point. Now we prove that all the trajectories will converge to xn. Assume that we have a sequence of profiles in a trajectory as: xð0Þ-xð1Þ-xð2Þ-⋯-xðqÞ-⋯: If there is an xðiÞ ¼ xn , then xðrÞ ¼ xn , 8 r4i. The convergence is obvious. Otherwise, we have a strictly increasing infinite sequence as ψðxð0ÞÞoψðxð1ÞÞo⋯oψðxðqÞÞo⋯:
ð11Þ
It was shown in the proof of Lemma 3.4 that this is impossible. (Necessity). Assume that each profile xA S will converge to xn. Since N is finite, x will converge to xn in finite steps. For each x A S, if it will converge to xn in r steps, we define ψðxÞ≔ r: It is ready to verify that ψ meets the requirements of a Lyapunov function.
□
According to Theorem 2.4, it is easy to verify that in vector form we have ψðxÞ ¼ V ψ ⋉ni ¼ 1 xi ðtÞ; where V ψ A R1kn is called the structure vector of ψðxÞ. Set V ψ ¼ ½a1 ; a2 ; …; akn , and assume that the matrix M of Eq. (6) is M ¼ δkn ½i1 ; i2 ; …; ikn . Then we have Proposition 3.5. The finite MtEG (6) has a Lyapunov function if and only if there exists an integer 1r r r kn , such that the following set of equality-inequalities (12) has a solution fa1 ; a2 ; …; akn g: ( aiq 4aq ; q ¼ 1; …; kn ; qa r; ð12Þ air ¼ ar : Proof (Sufficiency). According to Eq. (12), it is easy to verify that ψðxðtÞÞ ¼ V ψ ⋉xðtÞ satisfies the conditions (8) and (9) of a Lyapunov function in Definition 3.2. (Necessity). Assume that the finite MtEG (6) has a Lyapunov function ψðxÞ satisfying the conditions (8) and (9), and its algebraic form is ψðxÞ ¼ V ψ x ¼ ½a1 ; a2 ; …; akn x. When xðtÞ ¼ xn ¼ δrkn , we have ΔψðxðtÞÞ ¼ ψðxðt þ 1ÞÞ ψðxðtÞÞ ¼ V ψ MxðtÞ V ψ xðtÞ ¼ ½a1 ; …; akn ⋉δkn ½i1 ; …; ikn xðtÞ ½a1 ; …; akn xðtÞ Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
7
¼ ½ai1 ; …; aikn δrkn ½a1 ; …; akn δrkn ¼ air ar ¼ 0: So we can obtain the equality air ¼ ar . When xðtÞa xn , ΔψðxðtÞÞ40, it is easy to verify the following inequalities: aiq 4aq ;
q ¼ 1; …; kn ; q a r:
□
Proposition 3.5 provides an algorithm to construct a Lyapunov function ψðxÞ for the MtEG (6). Algorithm 3.6. Consider the MtEG (6) and assume ψðxðtÞÞ ¼ V ψ xðtÞ ¼ ½a1 ; a2 ; …; akn xðtÞ, then we take the following steps to construct ψðxÞ: 1. Compute the transition matrix M, and find out the unique fixed point; 2. Solve the equality-inequalities (12) to find a solution fa1 ; a2 ; …; akn g; 3. Obtain the Lyapunov function ψðxðtÞÞ ¼ ½a1 ; a2 ; …; akn xðtÞ: Remark 3.7. The set of equality-inequalities (12) has infinite number of solutions, so the Lyapunov function of the MtEG (6) is not unique.
Remark 3.8. The limitation of this algorithm is the computational complexity. It is easy to see that the computational complexity depends on the computation of matrix M. when the dimensions of matrices are large, the proposed algorithm can hardly be used. Then an efficient numerical method has to be developed to deal with it. We give an example to explain how to construct a Lyapunov function of an MtEG. Example 3.9 ([29]). Consider a two-player game with payoffs given in Table 1. Each player has three strategies (1: work; 2: shirk at office; 3: shirk at home). In this game, if both players work, then a project succeeds and they each receive a payoff of 90. If only one of them works, then the project may succeed and they each receive 48, or it may fail with a payoff of 12. If none of them work and they shirk at different places, then each receives a payoff of 24. On the other hand, if both shirk at the same place, then the row player has unit payoff, whereas the other player has 1. According to the MBRAR, it is easy to figure out the best response strategy fi for each player, which is listed in Table 2. Table 1 Payoff matrix. P1 ⧹P2
1
2
3
1 2 3
ð90; 90Þ ð 12; 12Þ ð48; 48Þ
ð 12; 12Þ ð1; 1Þ ð24; 24Þ
ð48; 48Þ ð24; 24Þ ð1; 1Þ
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
8 Table 2 Best response strategy. c⧹s
11
12
13
21
22
23
31
32
33
f1 f2
1 1
3 1
1 1
1 3
3 3
1 3
1 1
3 1
1 1
Using the cascading MBRAR, we obtain that x1 ðt þ 1Þ ¼ δ3 ½1; 3; 1; 1; 3; 1; 1; 3; 1xðtÞ ¼ M 1 xðtÞ; x2 ðt þ 1Þ ¼ δ3 ½1; 1; 1; 3; 3; 3; 1; 1; 1x1 ðt þ 1Þx2 ðtÞ ¼ M 2 x1 ðt þ 1Þx2 ðtÞ ¼ M 2 M 1 x1 ðtÞx2 ðtÞx2 ðtÞ ¼ M 2 M 1 x1 ðtÞOR3 x2 ðtÞ ¼ M 2 M 1 I 3 OR3 x1 ðtÞx2 ðtÞ ~ 2 xðtÞ: ≔M Then we have the profile dynamics as ~ 2 ÞxðtÞ xðt þ 1Þ ¼ ðM 1 nM ¼ δ9 ½1; 7; 1; 1; 7; 1; 1; 7; 1xðtÞ ¼ MxðtÞ:
ð13Þ δ19 .
According to Theorem 3.1, we can see that Eq. (13) has a unique fixed point Set M ¼ δ9 ½i1 ; i2 ; …; i9 and V ψ ¼ ½a1 ; a2 ; …; a9 , then we have an equality ai1 ¼ a1 and the following set of linear inequalities: 8 a7 4a2 > > > > > a > 1 4a3 > > > > a 4a4 > > > 1 > < a7 4a5 > a1 4a6 > > > > > a1 4a7 > > > > > a7 4a8 > > > : a 4a 1
9
Obviously, it has infinite number of solutions. If we choose a solution as fa1 ; a2 ; a3 ; a4 ; a5 ; a6 ; a7 ; a8 ; a9 g ¼ f6; 3; 1; 5; 2; 3; 4; 2; 1g, then the algebraic form of this Lyapunov function is as follows: ψðxðtÞÞ ¼ V ψ xðtÞ ¼ ½6; 3; 1; 5; 2; 3; 4; 2; 1xðtÞ: It is easy to verify that the game itself is not potential, but this Lyapunov function assumes that the dynamic game converge to a pure Nash equilibrium.
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
9
Table 3 Payoff bi-matrix of Example 3.11. P1 ⧹P2
1
2
1 2
ð3; 1Þ ð2; 4Þ
ð4; 3Þ ð5; 1Þ
Remark 3.10. 1. An important issue is: what is the relationship between the limitation and the Nash equilibrium. One conclusion is: As long as the strategy profile dynamics is obtained from MBRAR, no matter what type it is, the limitation is also a Nash equilibrium. This conclusion can easily be deduced from the definitions of the related concepts directly. 2. It is easy to see that the above conclusion is also available for the case of mixed strategies, where the dynamic equation (7) has a column random matrix M. 3. Unfortunately, the technique developed in this section for constructing Lyapunov function cannot be applicable to the case of mixed strategies, because for mixed strategies the transition matrix M obtained by MBRAR is in general a state-depending matrix. Refer to the following example:
Example 3.11. Consider a game with two players, which has the payoff bi-matrix as in Table 3 Assume that the mixed strategy adopted by P1 (P2) is xðtÞ ¼ ðx1 ðtÞ; x2 ðtÞÞT (yðtÞ ¼ ðy1 ðtÞ; y2 ðtÞÞT ). Then the expected values of the payoffs are c1 ¼ 3x1 y1 þ 4x1 y2 þ 2x2 y1 þ 5x2 y2 ; c2 ¼ x1 y1 þ 3x1 y2 þ 4x2 y1 þ x2 y2 : Using MBRAR, the profile dynamics is obtained as ( 1 ( 1 δ2 ; y1 ðtÞ4y2 ðtÞ δ2 ; 2x1 ðtÞo3x2 ðtÞ xðt þ 1Þ ¼ ; yðt þ 1Þ ¼ ; δ22 ; y1 ðtÞoy2 ðtÞ δ22 ; 2x1 ðtÞ43x2 ðtÞ where M is the strategy profile-depending.
4. Stability of MtEGs with time-varying payoffs This section considers the global stability of Markov-type evolutionary games with timevarying payoff function. Assume that a finite MtEG is based on G ¼ ðN; S; CðtÞÞ, where the payoff function CðtÞ ¼ ðc1 ðtÞ; …; cn ðtÞÞ is time-varying. Then the algebraic form of ci(t) can be expressed as ci ðtÞ ¼ V ci ðtÞ⋉ni ¼ 1 xi ðtÞ:
ð14Þ
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
10
Now for each C(t), t ¼ 0; 1; 2; …, the strategy profile dynamics can be described as 8 x ðt þ 1Þ ¼ f t1 ðx1 ðtÞ; …; xn ðtÞÞ; > < 1 ⋮ > : x ðt þ 1Þ ¼ f t ðx ðtÞ; …; x ðtÞÞ: n
n
1
ð15Þ
n
where fit, t ¼ 0; 1; …, is the time-varying k-valued logical function. By using semi-tensor product, for each fit, we can get its structure matrix Mi(t). Then the algebraic form of Eq. (15) follows as xðt þ 1Þ ¼ MðtÞxðtÞ;
t ¼ 0; 1; …;
ð16Þ
where MðtÞ ¼ M 1 ðtÞn⋯nM n ðtÞ. Now since Lkn kn is a finite set, MðtÞjt ¼ 0; 1; … contains only finite distinct matrices. Denote this set as: fM 1 ; …; M r g ¼ MðtÞjt ¼ 0; 1; … : ð17Þ Then Eq. (16) can be expressed as a switched system xðt þ 1Þ ¼ M sðtÞ xðtÞ;
t ¼ 0; 1; …;
ð18Þ
where sðtÞ A f1; 2; …; rg: In general, the existence of a common quadratic Lyapunov function is sufficient but not necessary for the stability of a switched system under arbitrary switching [30]. Similarly, we have the following result. Proposition 4.1. Consider the MtEG (15) with time-varying payoff function (14). Then the MtEG is globally stable at xn if all the r models in Eq. (18) share a common Lyapunov function ψðxÞ with a common fixed point xn. Proof. No matter which model is active, the common Lyapunov function is strictly increasing until xn is reached. Now it is clear that xn ¼ argmax ψðxÞ; xAS
which is unique. Therefore, all the trajectories will converge to xn.
□
Denote the transition matrices M j ¼ δkn i1j ; i2j ; …; ikjn ; j ¼ 1; …; r: Applying Proposition 3.5 to each model, the following result is obvious. Table 4 Payoff bi-matrix of Example 4.3. P1 ⧹P2
1
2
1 2
sin ðtπ=2Þ; sin ðtπ=2Þ 1 1
1; 1 1; 1
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
11
Corollary 4.2. Consider an MtEG based on G ¼ ðN; S; CðtÞÞ. There exists a common Lyapunov function if there exists an xn ¼ δμkn such that Colμ ðM j Þ ¼ xn ;
(i)
j ¼ 1; …; r;
ð19Þ
(ii) there exist a set of parameters fa1 ; a2 ; …; akn g, such that aijq 4aqj ; q ¼ 1; …; kn ; qa μ; j ¼ 1; …; r:
ð20Þ
Corollary 4.2 provides a method to verify the existence and construction of a common Lyapunov function. We give an example to illustrate it. Example 4.3. Assume that an MtEG has the payoff bi-matrix as in Table 4. Assume that the simultaneous MBRAR is used. Then it is easy to verify that there are only r¼ 3 different models at all time moments: (i) t ¼ 4s or t ¼ 4s þ 2, s ¼ 0; 1; 2; …: M 1 ¼ δ4 ½4; 4; 4; 4; (ii) t ¼ 4s þ 1, s ¼ 0; 1; 2; …: M 2 ¼ δ4 ½2; 4; 4; 4; (iii) t ¼ 4s þ 3, s ¼ 0; 1; 2; …: M 3 ¼ δ4 ½3; 4; 4; 4: n
Setting x ¼ δ44 and taking a1 ¼ 2;
a2 ¼ 1;
a3 ¼ 1;
a4 ¼ 0;
it is ready to verify that the requirements in Corollary 4.2 are satisfied. That is, ψðxÞ ¼ ½ 2; 1; 1; 0x1 x2 is a common Lyapunov function for the MtEG with time-varying payoffs. 5. Stabilization of controlled MtEGs In this section, we study the global stabilization problem of controlled MtEGs, which are defined as follows: Definition 5.1. Consider a finite game G ¼ ðN; S; CÞ played repetitively. Assume that the set of players can be divided into two parts as N ¼ fX; Ug, where X ¼ fx1 ; …; xn g are called the state players that follow certain SUR to update their strategies; and U ¼ fu1 ; …; um g are called the control players, that can select their strategies freely. Now assume jSi j ¼ k, 8i A N, following the same procedure as used for (not controlled) MtEG in Section 3, the strategy profile dynamics can be obtained as xðt þ 1Þ ¼ MuðtÞxðtÞ; where
xðtÞ ¼ ⋉ni ¼ 1 xi ðtÞ,
uðtÞ ¼ ⋉m j ¼ 1 uj ðtÞ,
ð21Þ M A Lkn kmþn .
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
12
In fact, Eq. (21) is a standard k-valued logical dynamic system. So the following result is applicable: Theorem 5.2 ([19]). The MtEG (21) is globally stabilizable, if and only if (i) there exists a profile xn, which is reachable from anywhere; (ii) there exists a control un, such that xn is a fixed point of Mun. Definition 5.3. A pseudo-logical function ψðxÞ : Dnk -R is called a control Lyapunov function of the MtEG (21) if there exists an xn such that (i) for each xa xn there exists ux such that V ψ ðMux I kn Þx40;
ð22Þ
(ii) for xn there exists un such that V ψ ðMun I kn Þxn ¼ 0:
ð23Þ
Similar to Theorem 3.3, we can prove the following result. Theorem 5.4. The controlled MtEG is globally stabilizable, if and only if there exists a control Lyapunov function.
Proof (Sufficiency). Assume that there exists a control Lyapunov function ψðxÞ satisfying the conditions (22) and (23), now we need to prove that the controlled MtEG satisfies (i)–(ii) of Theorem 5.2. The algebraic form of ψðxÞ can be expressed as ψðxÞ ¼ V ψ x, then we have ψðxðt þ 1ÞÞ ψðxðtÞÞ ¼ V ψ xðt þ 1Þ V ψ xðtÞ ¼ V ψ MuðtÞxðtÞ V ψ xðtÞ ¼ V ψ ðMuðtÞ I kn ÞxðtÞ: From the condition (23), one sees that xn is a fixed point of Mun. And from the condition (22), it is easy to verify that all the ðuðtÞ; xðtÞÞ trajectories can reach a fixed point ðun ; xn Þ, where ψðun ; xn Þ ¼
max
n u A Dm k ;x A Dk
ψðu; xÞ:
The sufficiency is proved. (Necessity). Let q be the shortest time for x to reach xn with proper controls. Then we define ψðxÞ ¼ q. It is ready to verify that this ψ is a control Lyapunov function. □ Next, we briefly discuss how to select the free controls for the global stabilization. For the condition (i) in Theorem 5.2, there must be a q40 such that all the trajectories from any initial profile x converge to xn ¼ δμkn in q steps. According to Eq. (21), we have xn Muðq 1ÞMuðq 2Þ⋯Muð0Þx;
8x A Δkn :
This is equivalent to ColðMuðq 1ÞMuðq 2Þ⋯Muð0ÞÞ ¼ fδμkn g:
ð24Þ
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
13
Now we can calculate Muðq 1ÞMuðq 2Þ⋯Muð0Þ ¼ MðI km MÞ⋯ðI kðq 1Þm MÞ⋉0i ¼ q 1 uðiÞ ≔½M q1 ; ⋯; M qkqm ⋉0i ¼ q 1 uðiÞ: It is clear that if there is a 1 r jr kqm such that Mqj satisfies ColðM qj Þ ¼ fδμkn g, then we can choose the control ⋉0i ¼ q 1 uðiÞ ¼ δjkqm ; such that Eq. (24) holds. For the condition (ii) in Theorem 5.2, we can choose a control un A Δkm such that Mun xn ¼ xn . That is, when t Z q, the control remains to be uðtÞ ¼ un . Now we give an example. Example 5.5. Recall Example 3.9. Let u≔P1 be the control and z≔P2 be the state. Assume that the SUR is the simultaneous MBRAR. Then from Table 2 we have the strategy profile dynamics as zðt þ 1Þ ¼ δ3 ½1; 1; 1; 3; 3; 3; 1; 1; 1uðtÞzðtÞ≔MuðtÞzðtÞ: When q¼ 1, we have M ¼ ½ M 11 ; M 12 ; M 13 ¼ δ3 ½1; 1; 1; 3; 3; 3; 1; 1; 1: It follows that ColðM 11 Þ ¼ ColðM 13 Þ ¼ fδ13 g: Now it is clear that we can choose uð0Þ ¼ δ13 1 or uð0Þ ¼ δ33 3 such that the game is globally stabilized to zn ¼ δ13 . In addition, we select uðtÞ ¼ un ¼ δ13 or uðtÞ ¼ un ¼ δ33 for t Z 1, such that Mun zn ¼ zn . Hence we set ψðzÞ ¼ V ψ z ¼ ½0; 1; 1z, which satisfies the requirements of a control Lyapunov function. Finally, we consider the case when the payoffs are time-varying. Let the controlled MtEG be determined by G ¼ ððX; UÞ; S; CðtÞÞ. Similar to the uncontrolled case, corresponding to each CðtÞ; t ¼ 0; 1; 2; …, we can figure out the corresponding transition matrices MðtÞ; t ¼ 0; 1; 2; …. Express the distinct transition matrices as: MðtÞjt ¼ 0; 1; 2; ⋯ ≔fW 1 ; W 2 ; …; W s g: Then it is easy to verify the following result: Proposition 5.6. The controlled MtEG based on G ¼ ððX; UÞ; S; CðtÞÞ is globally stabilizable, if there exists a common control Lyapunov function for all the models xðt þ 1Þ ¼ W j uðtÞxðtÞ, j ¼ 1; …; s, with the same (control-depending) fixed point xn.
Example 5.7. Recall Example 4.3. Let u≔x1 be the control and z≔x2 be the state. Then we have only two different models for the controlled MtEG with time-varying payoffs, zðt þ 1Þ ¼ δ2 ½2; 2; 2; 2uðtÞzðtÞ ¼ W 1 uðtÞzðtÞ; zðt þ 1Þ ¼ δ2 ½1; 2; 2; 2uðtÞzðtÞ ¼ W 2 uðtÞzðtÞ: Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
14
When q ¼ 1, we have that W 1 ¼ ½ðW 1 Þ11 ; ðW 1 Þ12 ¼ δ2 ½2; 2; 2; 2; W 2 ¼ ½ðW 2 Þ11 ; ðW 2 Þ12 ¼ δ2 ½1; 2; 2; 2: Similar to the analysis of Example 5.5, we can choose uð0Þ ¼ δ22 such that the controlled MtEG with time-varying payoffs is globally stabilized to the common fixed point zn ¼ δ22 . On the other hand, choosing uðtÞ ¼ un ¼ δ12 or uðtÞ ¼ un ¼ δ22 for t Z 1, we have W 1 un zn ¼ zn and W 2 un zn ¼ zn . Therefore, we can choose ψðzÞ ¼ V ψ z ¼ ½ 1; 0z as the common control Lyapunov function for the two models.
6. Conclusion The stability and stabilization of a class of finite evolutionary games were investigated in this paper. First, the Lyapunov function for MtEGs was defined. It was proved that the global stability of an MtEG is equivalent to the existence of a Lyapunov function. And the construction of Lyapunov functions was also obtained. Then the stability of MtEGs with time-varying payoffs was discussed. As a sufficient condition, the common Lyapunov function was proposed. Using the stability results, the stabilization of controlled MtEGs was then discussed. It was proved that the existence of a control Lyapunov function is necessary and sufficient for the stabilization. Finally, the common control Lyapunov function was used to assure the stabilization of controlled MtEGs with time-varying payoffs. Much of the literature focuses on global stability of equilibrium that ensures the convergence to the equilibrium regardless of players’ initial behavior. Inevitably, these requirements on payoff structure are quite demanding. Therefore, one can turn to study local stability of equilibrium, seeking conditions under which an equilibrium is robust to small changes in players’ behavior. The approach proposed in this paper can be extended to local stability/stabilization of evolutionary games as long as the Lyapunov function is locally defined. Constructing a Lyapunov function to assure the convergence to a mixed Nash equilibrium remains for further investigation. Acknowledgments This work is supported partly by the National Natural Science Foundation of China under Grants 61374025 and 61273013. References [1] [2] [3] [4] [5] [6] [7] [8]
R.A. Fisher, The Genetic Theory of Natural Selection, Clarendon Press, Oxford, 1930. W.D. Hamilton, The genetical evolution of social behaviour, J. Econ. Theory 7 (1964) 1–16. R.L. Trivers, The evolution of reciprocal altruism, Q. Rev. Biol. 46 (1) (1971) 35–57. J.M. Smith, G.R. Price, The logic of animal conflict, Nature 246 (1973) 15–18. P. Taylor, L. Jonker, Evolutionary stable strategies and game dynamics, Math. Biosci. 16 (1978) 76–83. J. Hofbauer, K. Sigmund, Evolutionary Games and Population Dynamics, Cambridge University Press, Cambridge, 1998. R. Sugden, The Economics of Rights, Cooperation and Welfare, Basil Blackwell, Oxford, 1986. H. Ohtsuki, C. Hauert, E. Lieberman, M.A. Nowak, A simple rule for the evolution of cooperation on graphs and social networks, Nature 441 (2006) 502–505.
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007
Y. Wang, D. Cheng / Journal of the Franklin Institute ] (]]]]) ]]]–]]]
15
[9] E. Mojica-Nava, C.A. Macana, N. Quijano, Dynamic population games for optimal dispatch on hierarchical microgrid control, IEEE Trans. Syst. Man Cybern.: Syst. 44 (3) (2014) 306–317. [10] G. Szabo, G. Fath, Evolutionary games on graphs, Phys. Rep. 446 (4) (2007) 97–216. [11] S. Suzuki, E. Akiyama, Evolutionary stability of first-order-information indirect reciprocity in sizable groups, Theor. Popul. Biol. 73 (2008) 426–436. [12] D. Fudenberg, D.K. Levine, The Theory of Learning in Games, MIT Press, Cambridge, MA, 1998. [13] J. Hofbauer, K. Sigmund, Evolutionary game dynamics, Bull. Am. Math. Soc. 40 (4) (2003) 479–519. [14] S.G. Nersesov, M.M. Haddad, On the stability and control of nonlinear dynamical systems via vector Lyapunov functions, IEEE Trans. Autom. Control 51 (2) (2006) 203–215. [15] J. Hofbauer, W.H. Sandholm, Stable games and their dynamics, J. Econ. Theory 144 (4) (2009) 1665–1693. [16] W.H. Sandholm, Local stability under evolutionary game dynamics, Theor. Econ. 5 (1) (2010) 27–50. [17] R.A. Araujo, H.N. Moreira, Lyapunov stability in an evolutionary game theory model of the labor market, EconomiA 15 (1) (2014) 41–53. [18] A.Y. Yazicioglu, M. Egerstedt, J.S. Shamma, A game theoretic approach to distributed coverage of graphs by heterogeneous mobile agents, Estimation and Control of Networked Systems 4 (1) (2013) 309–315. [19] D. Cheng, H. Qi, Z. Li, Analysis and Control of Boolean Networks: A Semi-tensor Product Approach, Springer, London, 2011. [20] F. Li, X. Lu, Z. Yu, Optimal control algorithms for switched Boolean network, J. Frankl. Inst. 351 (6) (2014) 3490–3501. [21] D. Laschov, M. Margaliot, Controllability of Boolean control networks via Perron–Frobenius theory, Automatica 48 (6) (2012) 1218–1223. [22] H. Li, Y. Wang, Consistent stabilizability of switched Boolean networks, Neural Netw. 46 (2013) 183–189. [23] E. Fornasini, M.E. Valcher, Optimal control of Boolean control networks, IEEE Trans. Autom. Control 59 (5) (2014) 1258–1270. [24] Y. Wang, C. Zhang, Z. Liu, A matrix approach to graph maximum stable set and coloring problems with application to multi-agent systems, Automatica 48 (7) (2012) 1227–1236. [25] D. Cheng, F. He, H. Qi, T. Xu, Modeling, analysis and control of networked evolutionary games, IEEE Trans. Autom. Control 60 (9) (2015) 2402–2415. [26] P. Guo, Y. Wang, H. Li, Algebraic formulation and strategy optimization for a class of evolutionary networked games via semi-tensor method, Automatica 49 (11) (2013) 3384–3389. [27] E. Rasmusen, Games and Information: An Introduction to Game Theory, Wiley-Blackwell, Oxford, 2007. [28] H.P. Young, The evolution of conventions, Econometrica 61 (1) (1993) 57–84. [29] O. Candogan, A. Ozdaglar, P.A. Parrilo, Dynamics in near-potential games, Games Econ. Behav. 82 (2013) 66–90. [30] W.P. Dayawansa, C.F. Martin, A converse Lyapunov theorem for a class of dynamical systems which undergo switching, IEEE Trans. Autom. Control 44 (4) (1999) 751–760.
Please cite this article as: Y. Wang, D. Cheng, Stability and stabilization of a class of finite evolutionary games, Journal of the Franklin Institute. (2016), http://dx.doi.org/10.1016/j.jfranklin.2016.12.007