- Email: [email protected]
Evolution, Partnerships and Cooperation
J. theor. Biol. (1998) 195, 315–328 Article No. jt980794
Evolution, Partnerships and Cooperation B C* C W Nuffield College, Oxford OX1 1NF, UK (Received on 9 March 1998, Accepted in revised form on 21 July 1998)
In an evolutionary game theoretic setting, players are allowed to choose not only their strategies for a particular interaction, but also to some extent with whom they will play. A stability concept is presented and examined which allows agents to endogenously construct fixed partnerships. A strategy-matching type is said to be recoverable if a population in which it is used by all players is both internally and externally stable to random mutations. Such a strategy-matching type always exists and maximises social efficiency in the population. Some examples are considered in the light of this concept. The ability to form fixed partnerships enables players to coordinate on efficient outcomes. In particular, cooperation is achieved in the Prisoners’ Dilemma without recourse to repetition or rigid spatial restriction. 7 1998 Academic Press
1. Introduction Evolutionary game theory has its origins in seminal work by Maynard Smith & Price (1973) and Maynard Smith (1982). The idea that revolutionised the analysis of evolutionary systems was that of an Evolutionarily Stable Strategy (ESS). If the interaction between members of a population is modelled as a game, with the payoffs of the game influencing reproductive success, then an ESS describes the long-run equilibrium of this evolutionary process. It does so because, if the members of a population are all playing an ESS, the population is stable to small numbers of members switching to one of the other strategies in the game. Or, as we could also express it, the population is able to recover from small invasions of ‘‘mutant’’ strategies. The evolutionary process underlying the definition of an ESS is not explicitly described, but implicit in the definition is some process *Author to whom correspondence should be addressed. 0022–5193/98/023315 + 14 $30.00/0
involving repeated random matching. That is, between each round of interaction, the population is somehow ‘‘shaken up’’, so that each member has an equal chance of meeting any other member. This is obviously unrealistic. In reality, members of a population can influence with whom they interact. We might well expect this to affect the long-run outcome of the evolutionary process. In this paper we take an important first step in allowing richer matching possibilities than the conventional paradigm in evolutionary game theory. We introduce a new stability concept for finite populations playing generic two-player finite games. We call this an Evolutionarily Recoverable Strategy-Type (ERST). It differs significantly from other stability concepts, such as ESS, in that population members have the option to form fixed partnerships, where they play the same opponent thereafter, rather than being forced to match randomly every time they play. What we have in mind is the most simple form of persistent group structure. For example, a 7 1998 Academic Press
316
. .
foraging or hunting partner could be chosen randomly, with the relationship lasting just a short time. Alternatively, partners who engage in some activity together, such as foraging or hunting (where the outcome depends on their combined interaction) could establish a permanent relationship—a fixed partnership.* By considering this most simple example of a social institution, we hope to pave the way to explanations of more complex group structures. We find that allowing partnership formation has a profound impact in the long-run outcome of the evolutionary process. For example, if the underlying game is the Prisoners’ Dilemma, then we find that the unique ERST has population members cooperating in fixed pairs. Ever since Trivers (1971), biologists have recognised the significance of cooperation in the Prisoners’ Dilemma. Explanations of cooperation have either depended on repeated versions of the game,† or on a very rigid spatial structure, such
* In what follows, we shall restrict attention to interaction in symmetric (i.e. single population) games. So while the duration of mating partnerships might seem an obvious potential application, it actually lies outside the scope of the ERST concept—although the concept could be extended to accommodate such asymmetric situations quite easily. † These draw on the Folk Theorems of repeated games, see e.g. Fudenberg & Tirole (1991), tournaments like those of Axelrod (1984), or analysis and simulations of simple repeated game strategies, such as Nowak & Sigmund (1995) or Nowak et al. (1995). ‡ We use the definition of global social efficiency as found in Weibull (1995), for example. § In a symmetric game with two pure Nash equilibria, the risk-dominant equilibrium is found by calculating the best response to an opponent’s strategy which places probability half on each of the equilibrium strategies. The importance of risk-dominance in evolutionary game theory has been stressed by Kandori et al. (1993), Young (1993a) and Kandori & Rob (1995), amongst others. ¶ That we allow players to use (and observe) mixed strategies is obviously a concession to tractability rather than to realism. Using mixed strategies allows us to approximate a model with only pure strategies without having to constantly worry about integer problems that add little or nothing to the content of our analysis but complicate things enormously. In terms of internal stability, using mixed strategies approximates a situation in which updating occurs in large group sizes and players (for example) choose the pure strategy-type with the highest average payoff in the group. In terms of external stability, using mixed strategies approximates a situation where one may have large, polymorphous groups of invading pure strategy-type mutants.
as that in Nowak & May (1992). Here, however, population members play a one-shot version of the game, and the spatial structure, so far as it exists at all, is very loose. As explained in Section 3, partnerships alter the way in which mutant strategies spread through a population, and it is this fact that allows cooperation to be sustained. Partnerships also play an interesting rox le in other games. In coordination games, they facilitate the selection of the strict equilibrium that maximises social efficiency.‡ This is true even if there are alternative equilibria that are risk-dominant.§ However, in games where partnerships would lead to inefficient outcomes, they do not survive the evolutionary process. The paper proceeds as follows: Section 2 outlines notation and the reasoning behind the ERST concept, defines it formally (in 2.1 and 2.2) and proves some general propositions on existence and payoff-maximisation (in 2.3). Section 3 looks at some specific applications of the concept, including to the Prisoners’ Dilemma. Section 4 concludes. 2. Recoverability Consider a population of N players. Each player i is described by a strategy-type. That is a vector si = (xi , ti ), where xi corresponds to a mixed strategy¶ in the symmetric two player game, G, and ti $ 4 f, f , r5, where f corresponds to symmetric fixed partnerships (both partners playing the same mixed strategy), f corresponds to asymmetric fixed partnerships (with the partners playing different mixed strategies), and r corresponds to random matching. f-types and f -types simply play their partners, while r-types are matched by a uniform random matching process such that there is an equal probability of them meeting any other r-type. The generic and symmetric two player game G has finite strategy set S. Players receive payoffs according to the function p: Y × Y : R, where Y is the set of probability distributions over all the elements of S. The payoff from playing a strategy x when one’s opponent is playing y is written p(x, y). Although strategy-type choice by players is not modelled explicitly, some notion of dynamic
updating lies behind the concept defined later, as with many stability concepts. In constructing the ERST concept, we have in mind dynamic processes in which: 1. there are two types of matching: fixed partnerships and uniform random matching; 2. there are spatial constraints on ‘‘reproduction’’. The first of these characteristics is selfexplanatory. We thus refer to both a player’s strategy in the underlying game and their matching type—i.e. a player is characterised by their ‘‘strategy–type’’. The second characteristic requires some further clarification. What we have in mind are dynamical systems in which a change in strategy or matching-type by a given population member is only influenced by the member’s local environment. This characteristic can be given an interpretation according to the application in view. It could mean, for example, that the spread of a strategy-type at a given moment is determined only by its relative performance within a given geographical region. It is helpful to divide each ‘‘generation’’ of this updating process into three phases: a matching phase, a playing phase and a reproductive phase. To describe the matching phase of updating, we use the following: Definition 1. Let p(y=x,t) denote the probability of any individual i = 41, . . . , N5 meeting an individual playing y $ Y given i is playing x $ Y in a pair-type t $ 4 f, f , r5. Let: p(y =x, f ) =
p(y =x, f ) =
p(y =x, r) =
6
6
if y = x if y $ x
(1)
6
if y = x if y $ x
(2)
1 0 0 1
(Nry − 1)(Nr − 1)−1 Nry (Nr − 1)−1
if y = x if y $ x (3)
where Nr is the total number of agents in a population who randomly match. Nry is the total
317
number in this subpopulation who are playing strategy y, so that: Sy $ Y Nry = Nr . Equation (3) is the standard notion of uniform random matching. After the playing phase, in which the population members receive their payoffs from the underlying game, comes the reproductive phase. Here we assume that the population members are still in the pairs in which they played the game, and that they are somehow spatially connected to W − 1 similar pairs, where 0 Q W Q N/2. We call W the window width. We do not model the spatial connections between pairs explicitly, but allow any arrangement into groups of size W, which may overlap. As we shall see, the spread of strategy-types depends on their relative performance within these localised groupings. Essentially, a strategy-pair type is an ERST if a population in which it is used by all players is both externally and internally stable. External stability is a similar concept to conventional evolutionary stability. Imagine a partially successful mutant invasion. Every player spatially connected to the original mutant has switched to it. Is the incumbent strategy-type able to ‘‘recover’’ from such an invasion, and drive it to extinction? If so, then the population is externally stable. However, the ERST concept differs from conventional notions of stability in that we require internal stability as well as external stability. That is, in a stable population there should be no pressure for strategies or matching types to spread within any given local region. This is because, while external stability in our concept is clearly similar to ESS-type concepts, it is somewhat weaker, for reasons that should become clear. We now define internal stability for a population. 2.1.
Definition 2. A population is internally stable iff, for all groups of 1 E W Q N/2 matched pairs, the expected payoff across players in the group is constant.
. .
318
Internal stability is designed to capture the idea that any player who observed a higher payoff than their own would wish to imitate the strategy that produced it. Hence no state where different payoffs are achieved locally should be considered stable. The proof of the following is contained in Appendix A.1. Proposition 1. A population containing one or more asymmetric fixed partnerships (type f ) cannot be internally stable. Any internally stable state has everyone gaining the same payoff in a neighbourhood. There can be no asymmetric fixed pairings, as in a generic game this would imply two players earning differing amounts. Proposition 1 means that we can ignore type f strategy-types as candidates for equilibria. Notice also that monomorphous populations consisting of all players using the strategy-type (x, t), where t $ 4 f, r5, are trivially internally stable. Since we * See also Vega-Redondo (1996).
shall henceforth be restricting our attention to populations of this type, we no longer need to test for internal stability. In the finite population ESS of Shaffer (1988),* a member of the single-strategy incumbent population is replaced with a mutant, then the expected payoff to an incumbent is compared with the expected payoff to the mutant. A similar thought experiment is followed here. Take a population consisting of players all of strategy-type s = (x, t) where t $ 4 f, r5. Consider an invasion by a mutant strategy-pair type sm = (xm , tm ) $ s such that all pairs observing sm copy it, giving an invasion zone consisting of W pairs of players. (These pairs will be either all r-types or all f-types, but cannot be f -types.) Now compare the expected payoff to a player using s to the expected payoff of a player using sm . This process is illustrated for the four different matching cases in Fig. 1. These probabilities are readily calculated for the incumbent strategy-type and the mutant 2.2.
Invasion zone (a) Random matching population; random matching mutant
(b) Random matching population; fixed partnership mutant
(c) Fixed partnership population; random matching mutant
(d) Fixed partnership population; fixed partnership mutant
F. 1. Invasion zones.
strategy-type in each of the four cases in Fig. 1, and are given below:
F1 − 2W case (a) N−1 p(x =x, t) = g case (b) (c) (d) f1 F 2W case (a) p(xm =x, t) = gN − 1 case (b) (c) (d) f0 F1 − 2W − 1 case (a) N−1 p(x =xm , tm ) = g 0 case (b) (c) (d) f F2W − 1 case (a) p(xm =xm , tm ) = g N − 1 (4) case (b) (c) (d) f1 The expected payoff to a particular player of pair-type t playing x in a population using strategies from the strategy set Y is given by: s p(x, y)p(y =x, t) y$Y
It is now possible to define formally what is meant by recoverability and an ERST. Definition 3. A strategy-pair type s = (x, t) is recoverable from a mutant strategy-pair type sm = (xm , tm ) $ s, where tm $ 4 f, r5, if \W q 0 such that [W E W , the following inequality holds: s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Y
y$Y
If a strategy-type is type r or f (and hence results in internally stable populations), and is recoverable from all possible mutations, then it is an ERST. Definition 4. A strategy-pair type s = (x, t) is an evolutionarily recoverable strategy-type (ERST) if t $ 4 f, r5 and [sm = (xm , tm ) $ s, where tm $ 4 f, r5, \W q 0 such that [W E W , the following holds: s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Y
y$Y
319
The differences between this and the definition of finite population ESS are the addition of a window width condition here to reflect the spatial nature of the implicit underlying process and, more importantly, the introduction of a variable probability of meeting particular strategies, p(y =x, t) which reflects the possibility of choosing with whom to play. It is intuitively clear that if fixed partnerships were not allowed and window widths were reduced so that only one player (the mutant itself) is able to observe the mutation, this definition would collapse to finite population ESS. This is the case since only uniform random matching remains as a possible pair type, and the locality of any mutation becomes irrelevant due to the absence of observation on the part of the players. Proposition 2 proves this formally—the proof essentially follows the intuition just described and can be found in Appendix A.2. 2.3.
Proposition 2. If players only observe their own strategies and payoffs and cannot form fixed partnerships, Definition 4 reduces to standard finite population ESS. Corollary 1. Any ERST where the population is randomly matching is an ESS. Schaffer (1988) shows how the finite population ESS relates to the standard infinite population version. It is clear that under his conditions in addition to the conditions in Proposition 2, ERST will also reduce to ESS for infinite populations, found in Weibull (1995) for example. Corollary 1 is useful since it provides us with a connection between ERST and ESS. If an ERST exists with a randomly matching population then this will be an ESS of the game. The proof of this statement follows directly from Proposition 2 and can be found in Appendix A.3. However, unlike ESS, at least one ERST will always exist. Furthermore, they are simple to find and they maximise social efficiency in the population. Social efficiency is defined here as in Weibull (1995). A population state y is said to be Globally Socially Efficient if there does not exist an x such that p(x, x) q p(y, y). These
320
. .
statements are contained formally in the following proposition and proven in Appendix A.4. Proposition 3. If s = (x, t) is an ERST then x $ arg maxy $ Y p(y, y). Conversely, if x $ arg maxy $ Y p(y, y) then s = (x, f ) is an ERST. ERST maximise social efficiency. This proposition has an intuitive explanation. Consider a population of fixed partnerships. In such a population only symmetric strategies can be played in an ERST (Proposition 1). If there were a strategy that, when meeting itself, could achieve a higher payoff than the one presently being played in the population, then it could successfully spread via the introduction of a mutant playing that action in a fixed partnership. Only the strategy that maximises the payoff it receives when meeting itself is a candidate for an ERST. Such a strategy in a population of fixed partnerships can recover from any mutation and is, therefore, an ERST. The payoff a strategy receives when matched with itself forms a continuous function on a compact domain, however, and hence always has a maximum. An ERST will always exist (see Appendix A.5 for a proof): Corollary 2. For every finite game there is always an ERST of the form s = (x, t), where x $ arg maxy $ Y p(y, y) and t = f. Given these facts it is now possible to see how well ERST predicts for a reasonable dynamic. Following the propositions in this section, the method is to first find the strategy which maximises social efficiency and this will be an ERST with fixed partnerships (Proposition 3). Secondly, find whether or not this strategy corresponds to an ESS. If so, then there is an ERST at this point where all the players randomly match (Corollary 1). If not, the only ERST is the one with fixed partnerships. The next section contains some of the examples presented in Cooper & Wallace (1997) (henceforth C&W).* We find the predicted stable points (ERSTs) and compare this with the results of our simulations. * In C&W we construct a dynamic procedure to test the results illustrated here. In doing this we run simulations on various 2 strategy games, including the Prisoners’ Dilemma. As discussed in Section 3 the model here predicts the results of the simulations very closely.
3. Examples In C&W we describe an explicit dynamic process with a specific spatial structure in which players could choose whether to match randomly or whether to form fixed partnerships. The results from simulating the process were very interesting. In some games, the ability to form partnerships strongly affected the outcome. For example, we observed the coevolution of partnerships and cooperation in the Prisoners’ Dilemma. In coordination games we observed how the temporary evolution of partnerships could switch a population from the inferior strict equilibrium to the efficient equilibrium. However, in games where partnerships would have led to inefficient outcomes, they did not evolve. Just as the ESS concept is implicitly based on some dynamic process with random matching, so ERST is implicitly based on some dynamic process similar to that in C&W. Contrariwise, C&W can be seen as an attempt to test whether or not the predictions of the ERST concept make sense for a plausible dynamical process. So we are following something like the general research programme suggested in Hofbauer & Weibull (1996) and summarised in Weibull (1995), where different stability ideas are tested for robustness against a variety of dynamics. The aim of this section is to consider the implications of the ERST concept in some specific examples. However, we shall also show that ERST turns out to predict well for the sort of dynamic modelled in C&W. ’ We consider the following version: 3.1.
a
b 3
4
a 3
1
1
2
b 4
2
The simulations in C&W show the way in which fixed partnerships lead to the cooperative outcome (both players using action a). This turns out to be the unique ERST. From the earlier results it is known that any ERST maximises expected social efficiency. In this game the function p(y, y) is simply maximised when y = (1, 0). That is, all weight is put on strategy a. Thus it follows that all players playing strategy a with probability 1 in fixed partnerships is an ERST. The only other candidate is when all players are randomly matching and playing strategy a. However, it is clear that if such a population were invaded by a randomly matching mutant playing b, the payoff to the mutant strategy would be greater than that of the incumbent:*
0
3 1−
1 0 1 0
2W 2W − 1 2W + Q2 N−1 N−1 N−1
0
+4 1−
1
2W − 1 N−1
1
Note that the mirror image of this equation is also true:
0
2 1−
1 0 1 0
2W 2W − 1 2W +4 e3 N−1 N−1 N−1
0
+ 1−
1
2W − 1 N−1
1
Hence it is clear that a population playing b and randomly matching is recoverable against any randomly matching mutant.† In other words, fixed partnerships are required to invade a ‘‘deviating’’ population, and they are also required to stabilise a ‘‘cooperating’’ population. * This follows immediately from Corollary 1. If a strategy does not constitute an ESS it cannot be an ERST with random matching players. † Including mixing mutants. This should be clear from the previous footnote. Also, the analysis of the game in Section 3.4. below should clarify the role of mixed strategies. ‡ For example, suppose that two players of a game simultaneously mutate into cooperators. Under the sort of dynamics we have in mind, all population members ‘‘near to’’ these players will then switch to cooperation.
321
This is exactly as the simulations in C&W suggest, as the proportion of the population cooperating always moves with the proportion of the population in fixed partnerships. Hence the unique ERST is as claimed above. To help understand the intuition behind this result, consider the following thought experiments. (It may be helpful to refer to Fig. 1.). First, consider a population of randomly matching cooperators invaded by randomly matching defectors. The invasion spreads out into an ‘‘invasion zone’’, the size of which depends on W. We now have a situation like that in panel (a) of Fig. 1. In this matching scenario, an invading defector will most likely be matched with a cooperator. So the expected payoff to the invaders will be greater than the expected payoff to the incumbents, and the invasion is successful. Secondly, consider the resulting population of randomly matching defectors invaded by randomly matching cooperators. Again the situation is like that in panel (a) of Fig. 1. We do not model explicitly how the cooperators spread out into the invasion zone, but just suppose that they do.‡ In this matching scenario, an invading cooperator will most likely be matched with a defector. So the expected payoff to the invaders will be less than the expected payoff to the incumbents, and the invasion is unsuccessful. The population recovers from the invasion. Thirdly, consider a population of randomly matching defectors now invaded by fixed partnership cooperators. We now have a situation like that in panel (b) of Fig. 1. In this matching scenario, an invading cooperator is always matched with another cooperator; while an incumbent defector is always matched with another defector. So the expected payoff to the invaders will be greater than the expected payoff to the incumbents, and the invasion is successful. Finally, consider the resulting population of fixed partnership cooperators invaded by randomly matching defectors. We now have a situation like that in panel (c) of Fig. 1. in this matching scenario, an invading defector is always matched with another defector; while an incumbent cooperator is always matched with another cooperator. So the expected payoff to the invaders will be less than the expected payoff to the incumbents, and the invasion is
. .
322
unsuccessful. The population recovers from the invasion. -. In the simulations of C&W the socially efficient outcome turned out to be a stable point with either random matching or fixed partnerships. Fixed partnerships were required, however, to switch from the risk-dominant outcome to the efficient outcome—but once this was achieved either pair-type could survive. Consider the following game: 3.2.
a
b 3
1 0
1 0
1
2W − 1 2W 2W − 1 e3 +4 1− N−1 N−1 N−1
In fact it is enough to choose W = (N − 2)/8. Notice that mutants in fixed partnerships playing b can invade a population playing a whereas \W q 0 such that [W E W :
0
3 1−
4
4
1 0 1 0
2W 2W − 1 2W +4 e5 N−1 N−1 N−1
1
5
b 0
0
5 1−
0
a 3
an ERST. It is straightforward to show that a randomly matching population with strategy b is also an ERST, since \W q 0 such that [W E W :*
5
There should be two ERSTs, one with the whole population playing b, and randomly matching and one with the whole population playing b, and in fixed partnerships. Further, in a purely random matching world with no fixed partnerships, the state with the population playing a should be stable. In other words the strategy (a, r) should be recoverable from the strategy (b, r). Only fixed partnerships should be able to invade such a state, and hence are required to switch to the globally socially efficient outcome although once there, both pair types with strategy b constitute ERSTs. This is indeed the case. The socially efficient strategy is to play b with probability 1. It follows that fixed partnerships along with this strategy constitute
* Since this point is also an ESS the result of Corollary 1 in conjunction with Corollary 2 can again be applied. However, notice here, that there is added interest in the size of W sufficient for the point to be an ERST. † Mixed strategies are of no interest in this game. They do not affect the results at all.
In fact W = (3N + 2)/8 will do. Hence the state with random type agents playing a is recoverable to the state with random type agents playing b. Partnerships are necessary for the switch to the socially efficient outcome. This is exactly as illustrated in the simulations in C&W for this game.† The Stag-Hunt game presented here is an example of a coordination game. In ‘‘pure’’ coordination games players get a positive payoff if they play the same action but nothing otherwise. For such games, there are multiple pure strategy Nash equilibria—each of the form (y, y) where y $ Y and Y is the set of pure strategies. However, the ERSTs can only involve play of the action that induces the payoff efficient outcome, a fact which follows trivially from Proposition 3. Another, perhaps more interesting, example of a coordination game is the Nash demand game. Here, players simultaneously make a ‘‘demand’’ for an amount of some commonly known prize. If the two demands sum to less than or equal to the value of the actual prize, each player receives their demand. If not, both players receive nothing. In this game the ERSTs involve ‘‘fair’’ demands—in which both players ask for half of the total value of the prize. More details are provided in Appendix B.
‘‘’’ In this game (illustrated below) the ERSTs are easily found. 3.3.
a
b 8
7
a 8
2
2
1
b 7
1
The strategy which puts probability 1 on a maximises social efficiency. Therefore, playing a in a fixed partnership is an ERST. Putting probability 1 on a is also an ESS in finite populations for N large enough. Hence for N large enough playing a in a randomly matching pair is an ERST. However, the simulations of C&W seemed to require fixed partnerships to stabilise strategy a. Again, the differences between the assumptions about the population in the two models are responsible and can help to explain the differences in the outcomes. Suppose the population are all randomly matching and playing a, then a mutant strategy, randomly matching and playing b, can enter the population successfully if N is small. For small population sizes, then, this strategy-type is not an ERST. To see this note that the payoff of the incumbent is:
0
8 1−
323
The first is greater than or equal to the second if and only if N e 7. So for N Q 7 the strategy a in a random pair type is not an ERST (or an ESS). This is what is known as ‘‘spiteful’’ behaviour and is a well known property of finite population ESS.* In the simulations of C&W, to speed up convergence times, a local interaction structure was utilised. Players were arranged on a circle where they observe six individuals, including themselves. It is as if the population were only of size 6 (although the total population is actually much larger, players act as if there were only six due to the local interaction structure).† This is small enough to destabilise the strategy-pair type that in a larger population would be an ERST. Hence, although playing a in both randomly matched and fixed pairs are ERSTs, the simulations showed that the interaction structure used can destabilise the randomly matching one. This outcome is as the method in this paper predicts for small populations. In the simulations, this game (illustrated below) had no pure-strategy rest points. This should be reflected in this paper by an absence of ERSTs for the pure strategies of the game. 3.4.
a 2
2
1 0 1
1 0
1
b 4
1
3
3
Whereas the mutant strategy gets:
0
4
a
2W 2W +2 N−1 N−1
2W − 1 2W − 1 +7 1− N−1 N−1
b
1
Indeed, maximising social efficiency to find the candidate ERSTs gives: max p(y, y) = max 42a 2 + 7(1 − a)a y$Y
* See Schaffer (1988) for example. † If, in the simulations, players were allowed to observe a larger window width (e.g. 10 individuals) then a randomly matching population playing a would also form a stable point.
a$[0,1]
+ (1 − a)25 =
41 16
where a is the weight on strategy a. This is maximised at ax = 58 . Hence the candidate ERSTs
. .
324
are the strategy y = (58 , 38 ) with the two types of matching. With fixed partnerships this is an ERST by Proposition 3. However, with random matching it is not an ERST. To see this it is enough to show that (58 , 38 ) is not an ESS. This is indeed the case. To clarify this point it will be shown that the candidate ERST with a randomly matching population playing (58 , 38 ) can be invaded by a mutant playing (12 , 12 ), and randomly matching. The payoff to the incumbent strategy would then be:
0
1 0
41 2W 1 5 3 1− + 5 +5 16 N−1 2 8 8
=
10 1
0
2W N−1
1 0 1
41 2W 5 2W 1− + 16 N−1 2 N−1
Whereas the mutant strategy’s payoff would be:
0
1 0
10
0
1 0
10 2W − 1 1 5 3 + 6 +4 4 N−1 2 8 8
=
1−
1
2W − 1 N−1
10 2W − 1 42 2W − 1 + 1− 4 N−1 16 N−1
1
Setting W = 12 (the smallest strictly positive value it can take) this gives:
0 1 0 1
41 N − 2 1 40 42 + Q 16 N − 1 16 N − 1 16 So that there does not exist W q 0 such that [W E W the payoff to the incumbent is bigger. A randomly matching mutant playing (12 , 12 ) can invade. The only ESS of this game turns out to be
0
2N − 5 2N − 5 ,1 − 4N − 8 4N − 8
1
(recall this is a finite population scenario), which naturally tends to (12 , 12 ) as the population gets * That is, the infinite population ESS. † Instead, the entire population chose to randomly match together and the proportion playing either strategy moved erratically around a fifty–fifty split.
larger.* Hence, it can be concluded there is only one ERST of this game, where fixed partnerships play (58 , 38 ). In the simulations of C&W, this is not the observed outcome.† However, in C&W, players cannot mix strategies. Thus, the predicted ERST cannot be observed. The actual observations in the simulation do fit with the analysis here in general though. Fixed partnerships confined to playing pure strategies would clearly take the population to a state where all players were playing strategy a. There is positive probability that a mutation would arise that was randomly matching and playing in an asymmetric pair (the only pair type that allows asymmetric pairs). This would then destabilise the population. If half the randomly matching population were playing a and half were playing b then the average payoff to a player would be 104 q 2. Hence, to increase payoffs the players need to play mixed strategies. The only way to approximate such behaviour is to randomly match, as fixed partnerships require symmetric play of pure strategies. Thus the population randomly matches in an attempt to play ‘‘mixed’’ strategies. However, the population shares playing the two strategies can never settle down because there is no ERST for randomly matching populations in this game, as has been shown above. 4. Conclusion The stability concept proposed in this paper effectively captures the following ideas: that individual players update strategies within localised groups, and that players are allowed to form fixed partnerships. That is, the probability of meeting particular strategy-types is endogenised to some extent. The ERST stability concept in this paper has some affinity to conventional stability concepts in evolutionary game theory. We have shown that under certain conditions the ERST collapses to the ESS for finite (and infinite) populations. This occurs when players can only randomly match and the window width is reduced to just 1 2 so only one individual is observed. Whole games (i.e. pairs of players) are no longer observed. The assumption that players in a pair
observe the strategies and payoffs of both the players in other pairs is therefore important. This, along with the endogenous matching probability, lead to the differences between the predictions of ERST and other stability concepts, including ESS. This paper is a first attempt to model the assortative phase of interaction.* We show that if the assumption of uniform matching is relaxed very different outcomes can be expected. For example, in the Prisoners’ Dilemma cooperative behaviour spreads throughout a population. Fixed partnerships enable this. However, when random matching is of benefit, e.g. in the game of Section 3.4, to break out of an inefficient symmetric outcome it will also be utilised by the population. In general, ERSTs maximise social efficiency in a population. This is true for all finite two player games. The other example games examined in this paper illustrate the different properties of recoverability. They also enable a comparison with the partnership dynamics simulated in C&W. The similarity in the stable points of the dynamics with the recoverable strategies in the games is striking. The simulation results show that recoverability is a reasonable approximation of the story envisioned in that paper. The results lend weight to the argument that an ERST gives a good indication of how a population might behave in a situation like the one constructed here. A natural extension of the model we have studied here would allow players to ‘‘mix’’ over their pair-types. In essence this would be equivalent to letting individuals choose endogenously the probability of staying with a particular player. A generalisation of this type should not be expected to yield significantly different results,
* Dugatkin (1995) provided the initial motivation. He states, ‘‘Any potential encounter can be divided into three phases: assortative, interactive and allocative . . . . During the assortative phase, individuals decide who to interact with, during the interactive phase, how to interact and during the allocative phase, how to divide the spoils. Most game theory models assume that strategic behaviour is employed only during the interactive phase. The allocative phase is assumed to be controlled by the payoff matrix, while the assortative phase is assumed to be purely random.’’
325
however, as it would seem likely that given the choice players would prefer to choose ‘‘pure’’ strategies. After all, immediate payoffs are affected only by action and not by pair-type. But this, or other ways of extending the choices players have over the manner in which they interact, could be an interesting avenue for further research. We would like to thank Martin Nowak, Peter So rensen, Jo¨rgen Weibull, David Myatt and seminar participants at Nuffield College, Oxford for helpful advice and comments. We are also grateful for the detailed and constructive comments of an anonymous referee. Financial support from the ESRC (grants R00429424072 and R00429534346, respectively) is acknowledged.
REFERENCES A, R. (1984). The Evolution of Cooperation. New York: Basic Books. C, B. & W, C. (1997). The evolution of partnerships. Soc. Methods Res. (in press). D, L. A. (1995). Partner choice, game theory and social behaviour. J. Quant. Anthropol. 5, 3–14. F, D. & T, J. (1991). Game Theory. Cambridge, MA: MIT Press. H, J. & W, J. W. (1996). Evolutionary selection against dominated strategies. J. Econ. Theory 71, 558–575. K, M. & R, R. (1995). Evolution of equilibria in the long run: a general theory and applications. J. Econ. Theory 65, 383–414. K, M., M, G. J. & R, R. (1993). Learning, mutation and long-run equilibria in games. Econometrica 61(1), 57–84. M S, J. (1982). Evolution and the Theory of Games. Cambridge: Cambridge University Press. M S, J. & P, G. R. (1973). The logic of animal conflict. Nature 246, 15–18. N, J. F. (1953). Two-person cooperative games. Econometrica 21, 128–140. N, M. & M, R. (1992). The spatial dilemmas of evolution. Int. J. Bifurc. Chaos 3, 35–78. N, M. & S, K. (1995). Invasion dynamics of the finitely repeated Prisoners’ Dilemma. Games Econ. Behav. 11, 364–390. N, M., S, K. & E-S, E. (1995). Automata, repeated games and noise. J. Math. Biol. 33, 703–722. S, M. E. (1988). Evolutionarily stable strategies for a finite population and a variable contest size. J. theor. Biol. 132, 469–478. T, R. (1971). The evolution of reciprocal altruism. Q. Rev. Biol. 46, 35–57. V-R, F. (1996). Evolution, Games and Economic Behaviour. Oxford: Oxford University Press. W, J. (1995). Evolutionary Game Theory. Cambridge, MA: MIT Press.
. .
326
Y, H. P. (1993a). The evolution of conventions. Econometrica 61(1), 57–84. Y, H. P. (1993b). An evolutionary model of bargaining. J. Econ. Theory 59, 145–168.
APPENDIX A Mathematical Proofs This Appendix contains the formal proofs omitted from the main text. .1. 1 A population containing one or more asymmetric fixed partnerships (type f ) cannot be internally stable. Proof This is fairly straightforward. Take a group of size W containing a player using strategy-type (x, f ). By definition of type f , the player is in a pair with a strategy-type (y, f ), where y $ x. This opponent is in the same group for all W e 1, but, since G is generic, p(x, y) $ p(y, x). By Definition 2 the population is not internally stable. q .2. 2 If players only observe their own strategies and payoffs and cannot form fixed partnerships, Definition 4 reduces to standard finite population ESS. Proof If players only observe themselves then W = 12 . Also set t = r. Now Definition 4 becomes: a strategy-pair type s = (x, r) is an Evolutionarily Recoverable Strategy-Type (ERST) if [sm $ s, such that for W = 12 , the following inequality holds: s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Y
y$Y
1
0
This is precisely the definition for finite population ESS in Schaffer (1988). q .3. 1 Any ERST where the population is randomly matching is an ESS. Proof This follows immediately. If the population are in an ERST, s = (x, r)* then the following is true: [sm $ s, \W q 0 such that [W E W , the following holds: s p(x, y)p(y =x, r) e s p(xm , y)p(y =xm , tm ) y$Y
y$Y
(A.1) Hence, in particular, if this holds [W E W then it holds for W = 12 . Further, if eqn (A.1) holds [sm $ s, then it holds for tm = r. But these are the conditions for Proposition 2 to apply. Hence s is a finite population ESS. q .4. 3 If s = (x, t) is an ERST then x $ arg maxy $ Y p(y, y). Conversely, if x $ arg maxy$Y p(y, y) then s = (x, f ) is an ERST. ERST maximise social efficiency. Proof Suppose s = (x, t) is an ERST. Then [sm $ s, \W q 0 such that [W E W , the following inequality holds: s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Y
(A.2)
2W 2W 1− p(x, x) + p(x, xm ) N−1 N−1
e 1−
N−2 1 p(x, x) + p(x, xm ) e p(xm , x) N−1 N−1
y$Y
which reduces to:
0
Hence:
1
2W − 1 2W − 1 p(xm , x) + p(xm , xm ) N−1 N−1
* That is, they are randomly matching.
Suppose further that x( arg maxy$Y p(y, y). Now it is possible to choose an xm $ arg maxy$Y p(y, y) such that xm $ x. Let sm = (xm , f ). Now following the probabilities from eqn (4), eqn (A.2) becomes: p(x, x) e p(xm , xm )
327
But this is not possible since xm maximises p(y, y) and x does not. This implies that s = (x, t) is not an ERST, a contradiction. So if s = (x, t) is an ERST then x $ arg maxy$Y p(y, y). For the second part suppose that x $ arg maxy$Y p(y, y), then p(x, x) e p(xm , xm ) for xm $ x. This is obviously true for all xm . Now s = (x, f ) is an ERST if [sm $ s, \W q 0 such that [W E W , the following inequality holds:
respectively. If x + y q 1 then neither player receives anything. The pure strategy Nash equilibria of this game are any demands (x, y) such that x + y = 1.* The notion of an ERST introduced here makes a more precise prediction: a fair split (x = y = 12 ). In order to show this, consider a discrete version of the demand game, where a pure strategy for player i is a demand xi from the set
s p(x, y)p(y =x, f ) e s p(xm , y)p(y =xm , tm )
1 2 n−1 Xn = 0, , , . . . ,1 † n n n
y$Y
6
y$Y
(A.3) Using the probability values in eqn (4), eqn (A.3) is equivalent to: p(x, x) e p(xm , xm )
There are two cases to consider, when n is even (and hence 12 $ Xn ) and when n is odd. The game below is an example of the payoff matrix when n = 2:
Which is true by assumption. q .5. 2 For every finite game there is always an ERST of the form s = (x, t), where x $ arg maxy$Y p(y, y) and t = f. Proof This follows immediately, since Proposition 3 shows that s = (x, t), where x $ arg maxy$Y p(y, y) and t = f is always an ERST, and there is always a maximum to the function p(y, y) if G is finite. q
7
1 2
0
1 1 2
0
1
0 0
0
1 2
0 1 2 1 2
0
1 2
0
0
0
0
0
1
APPENDIX B
1
0
0
The Nash Demand Game This Appendix provides an analysis of the Nash demand game. In this two-player game, each agent demands a fraction of some prize, normalised to be of value 1. Suppose player 1 demands x and player 2 demands y. If x + y E 1 then the payoffs to players 1 and 2 are x and y, * This is the standard simplification of the demand game found originally in Nash (1953). Young (1993b), using the same version of the game, provides an evolutionary analysis with uniform matching. † A discrete version is used since ERSTs have so far only been introduced for discrete action space games in Definition 4. As will be shown, there is little reason to think anything would change for the continuous action space game.
Consider first when 12 $ Xn . To find the ERST with the population in fixed partnerships, we need only maximise p(x, x). In this case p(x, x) = x if x + x = 2x E 1 and p(x, x) = 0 if 2x q 1. Clearly the maximum occurs at x = 12 . Hence playing xi = 12 in a fixed partnership is an ERST. Moreover, playing the same strategy but randomly matching is also an ERST: this follows since the demands (12 , 12 ) constitute an ESS. The case when n is odd and hence 12(Xn is somewhat different. Now p(x, x) = x if 2x Q 1 and p(x, x) = 0 if 2x q 1. The maximum occurs at x = 12 − 1/2n. The ERST therefore consists of a population demanding xi = 12 − 1/2n in fixed
. .
328
partnerships. This is still a fair outcome, although there is some wasted surplus. However, now a population playing the same strategy but randomly matching is no longer consistent with an ERST. In fact, it can be successfully invaded by a mutant which randomly matches and plays 1 1 xm = + 2 2n
0
10
1−
2W − 1 N−1
1
While an incumbent member of the population continues to receive an expected payoff of 1 1 − 2 2n
0
1 1 + 2 2n
10
1−
1
2W − 1 1 1 q − N−1 2 2n
for any 1 N−1 WQ + 2 n+1 Hence \W such that [W E W :
To see this, note that such a mutant would receive an expected payoff of 1 1 + 2 2n
Now:
s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Xn
y$Xn
[sm = (xm , tm ) $
0
1
1 1 − ,r = (x, t) 2 2n
Finally, notice how as n : a (whether even or odd, when 1/2n : 0) the ERSTs consist of a population in fixed partnerships or randomly matching playing xi = 12 . Thus for the continuous strategy space game, the prediction remains the same.
Evolution, Partnerships and Cooperation B C* C W Nuffield College, Oxford OX1 1NF, UK (Received on 9 March 1998, Accepted in revised form on 21 July 1998)
In an evolutionary game theoretic setting, players are allowed to choose not only their strategies for a particular interaction, but also to some extent with whom they will play. A stability concept is presented and examined which allows agents to endogenously construct fixed partnerships. A strategy-matching type is said to be recoverable if a population in which it is used by all players is both internally and externally stable to random mutations. Such a strategy-matching type always exists and maximises social efficiency in the population. Some examples are considered in the light of this concept. The ability to form fixed partnerships enables players to coordinate on efficient outcomes. In particular, cooperation is achieved in the Prisoners’ Dilemma without recourse to repetition or rigid spatial restriction. 7 1998 Academic Press
1. Introduction Evolutionary game theory has its origins in seminal work by Maynard Smith & Price (1973) and Maynard Smith (1982). The idea that revolutionised the analysis of evolutionary systems was that of an Evolutionarily Stable Strategy (ESS). If the interaction between members of a population is modelled as a game, with the payoffs of the game influencing reproductive success, then an ESS describes the long-run equilibrium of this evolutionary process. It does so because, if the members of a population are all playing an ESS, the population is stable to small numbers of members switching to one of the other strategies in the game. Or, as we could also express it, the population is able to recover from small invasions of ‘‘mutant’’ strategies. The evolutionary process underlying the definition of an ESS is not explicitly described, but implicit in the definition is some process *Author to whom correspondence should be addressed. 0022–5193/98/023315 + 14 $30.00/0
involving repeated random matching. That is, between each round of interaction, the population is somehow ‘‘shaken up’’, so that each member has an equal chance of meeting any other member. This is obviously unrealistic. In reality, members of a population can influence with whom they interact. We might well expect this to affect the long-run outcome of the evolutionary process. In this paper we take an important first step in allowing richer matching possibilities than the conventional paradigm in evolutionary game theory. We introduce a new stability concept for finite populations playing generic two-player finite games. We call this an Evolutionarily Recoverable Strategy-Type (ERST). It differs significantly from other stability concepts, such as ESS, in that population members have the option to form fixed partnerships, where they play the same opponent thereafter, rather than being forced to match randomly every time they play. What we have in mind is the most simple form of persistent group structure. For example, a 7 1998 Academic Press
316
. .
foraging or hunting partner could be chosen randomly, with the relationship lasting just a short time. Alternatively, partners who engage in some activity together, such as foraging or hunting (where the outcome depends on their combined interaction) could establish a permanent relationship—a fixed partnership.* By considering this most simple example of a social institution, we hope to pave the way to explanations of more complex group structures. We find that allowing partnership formation has a profound impact in the long-run outcome of the evolutionary process. For example, if the underlying game is the Prisoners’ Dilemma, then we find that the unique ERST has population members cooperating in fixed pairs. Ever since Trivers (1971), biologists have recognised the significance of cooperation in the Prisoners’ Dilemma. Explanations of cooperation have either depended on repeated versions of the game,† or on a very rigid spatial structure, such
* In what follows, we shall restrict attention to interaction in symmetric (i.e. single population) games. So while the duration of mating partnerships might seem an obvious potential application, it actually lies outside the scope of the ERST concept—although the concept could be extended to accommodate such asymmetric situations quite easily. † These draw on the Folk Theorems of repeated games, see e.g. Fudenberg & Tirole (1991), tournaments like those of Axelrod (1984), or analysis and simulations of simple repeated game strategies, such as Nowak & Sigmund (1995) or Nowak et al. (1995). ‡ We use the definition of global social efficiency as found in Weibull (1995), for example. § In a symmetric game with two pure Nash equilibria, the risk-dominant equilibrium is found by calculating the best response to an opponent’s strategy which places probability half on each of the equilibrium strategies. The importance of risk-dominance in evolutionary game theory has been stressed by Kandori et al. (1993), Young (1993a) and Kandori & Rob (1995), amongst others. ¶ That we allow players to use (and observe) mixed strategies is obviously a concession to tractability rather than to realism. Using mixed strategies allows us to approximate a model with only pure strategies without having to constantly worry about integer problems that add little or nothing to the content of our analysis but complicate things enormously. In terms of internal stability, using mixed strategies approximates a situation in which updating occurs in large group sizes and players (for example) choose the pure strategy-type with the highest average payoff in the group. In terms of external stability, using mixed strategies approximates a situation where one may have large, polymorphous groups of invading pure strategy-type mutants.
as that in Nowak & May (1992). Here, however, population members play a one-shot version of the game, and the spatial structure, so far as it exists at all, is very loose. As explained in Section 3, partnerships alter the way in which mutant strategies spread through a population, and it is this fact that allows cooperation to be sustained. Partnerships also play an interesting rox le in other games. In coordination games, they facilitate the selection of the strict equilibrium that maximises social efficiency.‡ This is true even if there are alternative equilibria that are risk-dominant.§ However, in games where partnerships would lead to inefficient outcomes, they do not survive the evolutionary process. The paper proceeds as follows: Section 2 outlines notation and the reasoning behind the ERST concept, defines it formally (in 2.1 and 2.2) and proves some general propositions on existence and payoff-maximisation (in 2.3). Section 3 looks at some specific applications of the concept, including to the Prisoners’ Dilemma. Section 4 concludes. 2. Recoverability Consider a population of N players. Each player i is described by a strategy-type. That is a vector si = (xi , ti ), where xi corresponds to a mixed strategy¶ in the symmetric two player game, G, and ti $ 4 f, f , r5, where f corresponds to symmetric fixed partnerships (both partners playing the same mixed strategy), f corresponds to asymmetric fixed partnerships (with the partners playing different mixed strategies), and r corresponds to random matching. f-types and f -types simply play their partners, while r-types are matched by a uniform random matching process such that there is an equal probability of them meeting any other r-type. The generic and symmetric two player game G has finite strategy set S. Players receive payoffs according to the function p: Y × Y : R, where Y is the set of probability distributions over all the elements of S. The payoff from playing a strategy x when one’s opponent is playing y is written p(x, y). Although strategy-type choice by players is not modelled explicitly, some notion of dynamic
updating lies behind the concept defined later, as with many stability concepts. In constructing the ERST concept, we have in mind dynamic processes in which: 1. there are two types of matching: fixed partnerships and uniform random matching; 2. there are spatial constraints on ‘‘reproduction’’. The first of these characteristics is selfexplanatory. We thus refer to both a player’s strategy in the underlying game and their matching type—i.e. a player is characterised by their ‘‘strategy–type’’. The second characteristic requires some further clarification. What we have in mind are dynamical systems in which a change in strategy or matching-type by a given population member is only influenced by the member’s local environment. This characteristic can be given an interpretation according to the application in view. It could mean, for example, that the spread of a strategy-type at a given moment is determined only by its relative performance within a given geographical region. It is helpful to divide each ‘‘generation’’ of this updating process into three phases: a matching phase, a playing phase and a reproductive phase. To describe the matching phase of updating, we use the following: Definition 1. Let p(y=x,t) denote the probability of any individual i = 41, . . . , N5 meeting an individual playing y $ Y given i is playing x $ Y in a pair-type t $ 4 f, f , r5. Let: p(y =x, f ) =
p(y =x, f ) =
p(y =x, r) =
6
6
if y = x if y $ x
(1)
6
if y = x if y $ x
(2)
1 0 0 1
(Nry − 1)(Nr − 1)−1 Nry (Nr − 1)−1
if y = x if y $ x (3)
where Nr is the total number of agents in a population who randomly match. Nry is the total
317
number in this subpopulation who are playing strategy y, so that: Sy $ Y Nry = Nr . Equation (3) is the standard notion of uniform random matching. After the playing phase, in which the population members receive their payoffs from the underlying game, comes the reproductive phase. Here we assume that the population members are still in the pairs in which they played the game, and that they are somehow spatially connected to W − 1 similar pairs, where 0 Q W Q N/2. We call W the window width. We do not model the spatial connections between pairs explicitly, but allow any arrangement into groups of size W, which may overlap. As we shall see, the spread of strategy-types depends on their relative performance within these localised groupings. Essentially, a strategy-pair type is an ERST if a population in which it is used by all players is both externally and internally stable. External stability is a similar concept to conventional evolutionary stability. Imagine a partially successful mutant invasion. Every player spatially connected to the original mutant has switched to it. Is the incumbent strategy-type able to ‘‘recover’’ from such an invasion, and drive it to extinction? If so, then the population is externally stable. However, the ERST concept differs from conventional notions of stability in that we require internal stability as well as external stability. That is, in a stable population there should be no pressure for strategies or matching types to spread within any given local region. This is because, while external stability in our concept is clearly similar to ESS-type concepts, it is somewhat weaker, for reasons that should become clear. We now define internal stability for a population. 2.1.
Definition 2. A population is internally stable iff, for all groups of 1 E W Q N/2 matched pairs, the expected payoff across players in the group is constant.
. .
318
Internal stability is designed to capture the idea that any player who observed a higher payoff than their own would wish to imitate the strategy that produced it. Hence no state where different payoffs are achieved locally should be considered stable. The proof of the following is contained in Appendix A.1. Proposition 1. A population containing one or more asymmetric fixed partnerships (type f ) cannot be internally stable. Any internally stable state has everyone gaining the same payoff in a neighbourhood. There can be no asymmetric fixed pairings, as in a generic game this would imply two players earning differing amounts. Proposition 1 means that we can ignore type f strategy-types as candidates for equilibria. Notice also that monomorphous populations consisting of all players using the strategy-type (x, t), where t $ 4 f, r5, are trivially internally stable. Since we * See also Vega-Redondo (1996).
shall henceforth be restricting our attention to populations of this type, we no longer need to test for internal stability. In the finite population ESS of Shaffer (1988),* a member of the single-strategy incumbent population is replaced with a mutant, then the expected payoff to an incumbent is compared with the expected payoff to the mutant. A similar thought experiment is followed here. Take a population consisting of players all of strategy-type s = (x, t) where t $ 4 f, r5. Consider an invasion by a mutant strategy-pair type sm = (xm , tm ) $ s such that all pairs observing sm copy it, giving an invasion zone consisting of W pairs of players. (These pairs will be either all r-types or all f-types, but cannot be f -types.) Now compare the expected payoff to a player using s to the expected payoff of a player using sm . This process is illustrated for the four different matching cases in Fig. 1. These probabilities are readily calculated for the incumbent strategy-type and the mutant 2.2.
Invasion zone (a) Random matching population; random matching mutant
(b) Random matching population; fixed partnership mutant
(c) Fixed partnership population; random matching mutant
(d) Fixed partnership population; fixed partnership mutant
F. 1. Invasion zones.
strategy-type in each of the four cases in Fig. 1, and are given below:
F1 − 2W case (a) N−1 p(x =x, t) = g case (b) (c) (d) f1 F 2W case (a) p(xm =x, t) = gN − 1 case (b) (c) (d) f0 F1 − 2W − 1 case (a) N−1 p(x =xm , tm ) = g 0 case (b) (c) (d) f F2W − 1 case (a) p(xm =xm , tm ) = g N − 1 (4) case (b) (c) (d) f1 The expected payoff to a particular player of pair-type t playing x in a population using strategies from the strategy set Y is given by: s p(x, y)p(y =x, t) y$Y
It is now possible to define formally what is meant by recoverability and an ERST. Definition 3. A strategy-pair type s = (x, t) is recoverable from a mutant strategy-pair type sm = (xm , tm ) $ s, where tm $ 4 f, r5, if \W q 0 such that [W E W , the following inequality holds: s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Y
y$Y
If a strategy-type is type r or f (and hence results in internally stable populations), and is recoverable from all possible mutations, then it is an ERST. Definition 4. A strategy-pair type s = (x, t) is an evolutionarily recoverable strategy-type (ERST) if t $ 4 f, r5 and [sm = (xm , tm ) $ s, where tm $ 4 f, r5, \W q 0 such that [W E W , the following holds: s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Y
y$Y
319
The differences between this and the definition of finite population ESS are the addition of a window width condition here to reflect the spatial nature of the implicit underlying process and, more importantly, the introduction of a variable probability of meeting particular strategies, p(y =x, t) which reflects the possibility of choosing with whom to play. It is intuitively clear that if fixed partnerships were not allowed and window widths were reduced so that only one player (the mutant itself) is able to observe the mutation, this definition would collapse to finite population ESS. This is the case since only uniform random matching remains as a possible pair type, and the locality of any mutation becomes irrelevant due to the absence of observation on the part of the players. Proposition 2 proves this formally—the proof essentially follows the intuition just described and can be found in Appendix A.2. 2.3.
Proposition 2. If players only observe their own strategies and payoffs and cannot form fixed partnerships, Definition 4 reduces to standard finite population ESS. Corollary 1. Any ERST where the population is randomly matching is an ESS. Schaffer (1988) shows how the finite population ESS relates to the standard infinite population version. It is clear that under his conditions in addition to the conditions in Proposition 2, ERST will also reduce to ESS for infinite populations, found in Weibull (1995) for example. Corollary 1 is useful since it provides us with a connection between ERST and ESS. If an ERST exists with a randomly matching population then this will be an ESS of the game. The proof of this statement follows directly from Proposition 2 and can be found in Appendix A.3. However, unlike ESS, at least one ERST will always exist. Furthermore, they are simple to find and they maximise social efficiency in the population. Social efficiency is defined here as in Weibull (1995). A population state y is said to be Globally Socially Efficient if there does not exist an x such that p(x, x) q p(y, y). These
320
. .
statements are contained formally in the following proposition and proven in Appendix A.4. Proposition 3. If s = (x, t) is an ERST then x $ arg maxy $ Y p(y, y). Conversely, if x $ arg maxy $ Y p(y, y) then s = (x, f ) is an ERST. ERST maximise social efficiency. This proposition has an intuitive explanation. Consider a population of fixed partnerships. In such a population only symmetric strategies can be played in an ERST (Proposition 1). If there were a strategy that, when meeting itself, could achieve a higher payoff than the one presently being played in the population, then it could successfully spread via the introduction of a mutant playing that action in a fixed partnership. Only the strategy that maximises the payoff it receives when meeting itself is a candidate for an ERST. Such a strategy in a population of fixed partnerships can recover from any mutation and is, therefore, an ERST. The payoff a strategy receives when matched with itself forms a continuous function on a compact domain, however, and hence always has a maximum. An ERST will always exist (see Appendix A.5 for a proof): Corollary 2. For every finite game there is always an ERST of the form s = (x, t), where x $ arg maxy $ Y p(y, y) and t = f. Given these facts it is now possible to see how well ERST predicts for a reasonable dynamic. Following the propositions in this section, the method is to first find the strategy which maximises social efficiency and this will be an ERST with fixed partnerships (Proposition 3). Secondly, find whether or not this strategy corresponds to an ESS. If so, then there is an ERST at this point where all the players randomly match (Corollary 1). If not, the only ERST is the one with fixed partnerships. The next section contains some of the examples presented in Cooper & Wallace (1997) (henceforth C&W).* We find the predicted stable points (ERSTs) and compare this with the results of our simulations. * In C&W we construct a dynamic procedure to test the results illustrated here. In doing this we run simulations on various 2 strategy games, including the Prisoners’ Dilemma. As discussed in Section 3 the model here predicts the results of the simulations very closely.
3. Examples In C&W we describe an explicit dynamic process with a specific spatial structure in which players could choose whether to match randomly or whether to form fixed partnerships. The results from simulating the process were very interesting. In some games, the ability to form partnerships strongly affected the outcome. For example, we observed the coevolution of partnerships and cooperation in the Prisoners’ Dilemma. In coordination games we observed how the temporary evolution of partnerships could switch a population from the inferior strict equilibrium to the efficient equilibrium. However, in games where partnerships would have led to inefficient outcomes, they did not evolve. Just as the ESS concept is implicitly based on some dynamic process with random matching, so ERST is implicitly based on some dynamic process similar to that in C&W. Contrariwise, C&W can be seen as an attempt to test whether or not the predictions of the ERST concept make sense for a plausible dynamical process. So we are following something like the general research programme suggested in Hofbauer & Weibull (1996) and summarised in Weibull (1995), where different stability ideas are tested for robustness against a variety of dynamics. The aim of this section is to consider the implications of the ERST concept in some specific examples. However, we shall also show that ERST turns out to predict well for the sort of dynamic modelled in C&W. ’ We consider the following version: 3.1.
a
b 3
4
a 3
1
1
2
b 4
2
The simulations in C&W show the way in which fixed partnerships lead to the cooperative outcome (both players using action a). This turns out to be the unique ERST. From the earlier results it is known that any ERST maximises expected social efficiency. In this game the function p(y, y) is simply maximised when y = (1, 0). That is, all weight is put on strategy a. Thus it follows that all players playing strategy a with probability 1 in fixed partnerships is an ERST. The only other candidate is when all players are randomly matching and playing strategy a. However, it is clear that if such a population were invaded by a randomly matching mutant playing b, the payoff to the mutant strategy would be greater than that of the incumbent:*
0
3 1−
1 0 1 0
2W 2W − 1 2W + Q2 N−1 N−1 N−1
0
+4 1−
1
2W − 1 N−1
1
Note that the mirror image of this equation is also true:
0
2 1−
1 0 1 0
2W 2W − 1 2W +4 e3 N−1 N−1 N−1
0
+ 1−
1
2W − 1 N−1
1
Hence it is clear that a population playing b and randomly matching is recoverable against any randomly matching mutant.† In other words, fixed partnerships are required to invade a ‘‘deviating’’ population, and they are also required to stabilise a ‘‘cooperating’’ population. * This follows immediately from Corollary 1. If a strategy does not constitute an ESS it cannot be an ERST with random matching players. † Including mixing mutants. This should be clear from the previous footnote. Also, the analysis of the game in Section 3.4. below should clarify the role of mixed strategies. ‡ For example, suppose that two players of a game simultaneously mutate into cooperators. Under the sort of dynamics we have in mind, all population members ‘‘near to’’ these players will then switch to cooperation.
321
This is exactly as the simulations in C&W suggest, as the proportion of the population cooperating always moves with the proportion of the population in fixed partnerships. Hence the unique ERST is as claimed above. To help understand the intuition behind this result, consider the following thought experiments. (It may be helpful to refer to Fig. 1.). First, consider a population of randomly matching cooperators invaded by randomly matching defectors. The invasion spreads out into an ‘‘invasion zone’’, the size of which depends on W. We now have a situation like that in panel (a) of Fig. 1. In this matching scenario, an invading defector will most likely be matched with a cooperator. So the expected payoff to the invaders will be greater than the expected payoff to the incumbents, and the invasion is successful. Secondly, consider the resulting population of randomly matching defectors invaded by randomly matching cooperators. Again the situation is like that in panel (a) of Fig. 1. We do not model explicitly how the cooperators spread out into the invasion zone, but just suppose that they do.‡ In this matching scenario, an invading cooperator will most likely be matched with a defector. So the expected payoff to the invaders will be less than the expected payoff to the incumbents, and the invasion is unsuccessful. The population recovers from the invasion. Thirdly, consider a population of randomly matching defectors now invaded by fixed partnership cooperators. We now have a situation like that in panel (b) of Fig. 1. In this matching scenario, an invading cooperator is always matched with another cooperator; while an incumbent defector is always matched with another defector. So the expected payoff to the invaders will be greater than the expected payoff to the incumbents, and the invasion is successful. Finally, consider the resulting population of fixed partnership cooperators invaded by randomly matching defectors. We now have a situation like that in panel (c) of Fig. 1. in this matching scenario, an invading defector is always matched with another defector; while an incumbent cooperator is always matched with another cooperator. So the expected payoff to the invaders will be less than the expected payoff to the incumbents, and the invasion is
. .
322
unsuccessful. The population recovers from the invasion. -. In the simulations of C&W the socially efficient outcome turned out to be a stable point with either random matching or fixed partnerships. Fixed partnerships were required, however, to switch from the risk-dominant outcome to the efficient outcome—but once this was achieved either pair-type could survive. Consider the following game: 3.2.
a
b 3
1 0
1 0
1
2W − 1 2W 2W − 1 e3 +4 1− N−1 N−1 N−1
In fact it is enough to choose W = (N − 2)/8. Notice that mutants in fixed partnerships playing b can invade a population playing a whereas \W q 0 such that [W E W :
0
3 1−
4
4
1 0 1 0
2W 2W − 1 2W +4 e5 N−1 N−1 N−1
1
5
b 0
0
5 1−
0
a 3
an ERST. It is straightforward to show that a randomly matching population with strategy b is also an ERST, since \W q 0 such that [W E W :*
5
There should be two ERSTs, one with the whole population playing b, and randomly matching and one with the whole population playing b, and in fixed partnerships. Further, in a purely random matching world with no fixed partnerships, the state with the population playing a should be stable. In other words the strategy (a, r) should be recoverable from the strategy (b, r). Only fixed partnerships should be able to invade such a state, and hence are required to switch to the globally socially efficient outcome although once there, both pair types with strategy b constitute ERSTs. This is indeed the case. The socially efficient strategy is to play b with probability 1. It follows that fixed partnerships along with this strategy constitute
* Since this point is also an ESS the result of Corollary 1 in conjunction with Corollary 2 can again be applied. However, notice here, that there is added interest in the size of W sufficient for the point to be an ERST. † Mixed strategies are of no interest in this game. They do not affect the results at all.
In fact W = (3N + 2)/8 will do. Hence the state with random type agents playing a is recoverable to the state with random type agents playing b. Partnerships are necessary for the switch to the socially efficient outcome. This is exactly as illustrated in the simulations in C&W for this game.† The Stag-Hunt game presented here is an example of a coordination game. In ‘‘pure’’ coordination games players get a positive payoff if they play the same action but nothing otherwise. For such games, there are multiple pure strategy Nash equilibria—each of the form (y, y) where y $ Y and Y is the set of pure strategies. However, the ERSTs can only involve play of the action that induces the payoff efficient outcome, a fact which follows trivially from Proposition 3. Another, perhaps more interesting, example of a coordination game is the Nash demand game. Here, players simultaneously make a ‘‘demand’’ for an amount of some commonly known prize. If the two demands sum to less than or equal to the value of the actual prize, each player receives their demand. If not, both players receive nothing. In this game the ERSTs involve ‘‘fair’’ demands—in which both players ask for half of the total value of the prize. More details are provided in Appendix B.
‘‘’’ In this game (illustrated below) the ERSTs are easily found. 3.3.
a
b 8
7
a 8
2
2
1
b 7
1
The strategy which puts probability 1 on a maximises social efficiency. Therefore, playing a in a fixed partnership is an ERST. Putting probability 1 on a is also an ESS in finite populations for N large enough. Hence for N large enough playing a in a randomly matching pair is an ERST. However, the simulations of C&W seemed to require fixed partnerships to stabilise strategy a. Again, the differences between the assumptions about the population in the two models are responsible and can help to explain the differences in the outcomes. Suppose the population are all randomly matching and playing a, then a mutant strategy, randomly matching and playing b, can enter the population successfully if N is small. For small population sizes, then, this strategy-type is not an ERST. To see this note that the payoff of the incumbent is:
0
8 1−
323
The first is greater than or equal to the second if and only if N e 7. So for N Q 7 the strategy a in a random pair type is not an ERST (or an ESS). This is what is known as ‘‘spiteful’’ behaviour and is a well known property of finite population ESS.* In the simulations of C&W, to speed up convergence times, a local interaction structure was utilised. Players were arranged on a circle where they observe six individuals, including themselves. It is as if the population were only of size 6 (although the total population is actually much larger, players act as if there were only six due to the local interaction structure).† This is small enough to destabilise the strategy-pair type that in a larger population would be an ERST. Hence, although playing a in both randomly matched and fixed pairs are ERSTs, the simulations showed that the interaction structure used can destabilise the randomly matching one. This outcome is as the method in this paper predicts for small populations. In the simulations, this game (illustrated below) had no pure-strategy rest points. This should be reflected in this paper by an absence of ERSTs for the pure strategies of the game. 3.4.
a 2
2
1 0 1
1 0
1
b 4
1
3
3
Whereas the mutant strategy gets:
0
4
a
2W 2W +2 N−1 N−1
2W − 1 2W − 1 +7 1− N−1 N−1
b
1
Indeed, maximising social efficiency to find the candidate ERSTs gives: max p(y, y) = max 42a 2 + 7(1 − a)a y$Y
* See Schaffer (1988) for example. † If, in the simulations, players were allowed to observe a larger window width (e.g. 10 individuals) then a randomly matching population playing a would also form a stable point.
a$[0,1]
+ (1 − a)25 =
41 16
where a is the weight on strategy a. This is maximised at ax = 58 . Hence the candidate ERSTs
. .
324
are the strategy y = (58 , 38 ) with the two types of matching. With fixed partnerships this is an ERST by Proposition 3. However, with random matching it is not an ERST. To see this it is enough to show that (58 , 38 ) is not an ESS. This is indeed the case. To clarify this point it will be shown that the candidate ERST with a randomly matching population playing (58 , 38 ) can be invaded by a mutant playing (12 , 12 ), and randomly matching. The payoff to the incumbent strategy would then be:
0
1 0
41 2W 1 5 3 1− + 5 +5 16 N−1 2 8 8
=
10 1
0
2W N−1
1 0 1
41 2W 5 2W 1− + 16 N−1 2 N−1
Whereas the mutant strategy’s payoff would be:
0
1 0
10
0
1 0
10 2W − 1 1 5 3 + 6 +4 4 N−1 2 8 8
=
1−
1
2W − 1 N−1
10 2W − 1 42 2W − 1 + 1− 4 N−1 16 N−1
1
Setting W = 12 (the smallest strictly positive value it can take) this gives:
0 1 0 1
41 N − 2 1 40 42 + Q 16 N − 1 16 N − 1 16 So that there does not exist W q 0 such that [W E W the payoff to the incumbent is bigger. A randomly matching mutant playing (12 , 12 ) can invade. The only ESS of this game turns out to be
0
2N − 5 2N − 5 ,1 − 4N − 8 4N − 8
1
(recall this is a finite population scenario), which naturally tends to (12 , 12 ) as the population gets * That is, the infinite population ESS. † Instead, the entire population chose to randomly match together and the proportion playing either strategy moved erratically around a fifty–fifty split.
larger.* Hence, it can be concluded there is only one ERST of this game, where fixed partnerships play (58 , 38 ). In the simulations of C&W, this is not the observed outcome.† However, in C&W, players cannot mix strategies. Thus, the predicted ERST cannot be observed. The actual observations in the simulation do fit with the analysis here in general though. Fixed partnerships confined to playing pure strategies would clearly take the population to a state where all players were playing strategy a. There is positive probability that a mutation would arise that was randomly matching and playing in an asymmetric pair (the only pair type that allows asymmetric pairs). This would then destabilise the population. If half the randomly matching population were playing a and half were playing b then the average payoff to a player would be 104 q 2. Hence, to increase payoffs the players need to play mixed strategies. The only way to approximate such behaviour is to randomly match, as fixed partnerships require symmetric play of pure strategies. Thus the population randomly matches in an attempt to play ‘‘mixed’’ strategies. However, the population shares playing the two strategies can never settle down because there is no ERST for randomly matching populations in this game, as has been shown above. 4. Conclusion The stability concept proposed in this paper effectively captures the following ideas: that individual players update strategies within localised groups, and that players are allowed to form fixed partnerships. That is, the probability of meeting particular strategy-types is endogenised to some extent. The ERST stability concept in this paper has some affinity to conventional stability concepts in evolutionary game theory. We have shown that under certain conditions the ERST collapses to the ESS for finite (and infinite) populations. This occurs when players can only randomly match and the window width is reduced to just 1 2 so only one individual is observed. Whole games (i.e. pairs of players) are no longer observed. The assumption that players in a pair
observe the strategies and payoffs of both the players in other pairs is therefore important. This, along with the endogenous matching probability, lead to the differences between the predictions of ERST and other stability concepts, including ESS. This paper is a first attempt to model the assortative phase of interaction.* We show that if the assumption of uniform matching is relaxed very different outcomes can be expected. For example, in the Prisoners’ Dilemma cooperative behaviour spreads throughout a population. Fixed partnerships enable this. However, when random matching is of benefit, e.g. in the game of Section 3.4, to break out of an inefficient symmetric outcome it will also be utilised by the population. In general, ERSTs maximise social efficiency in a population. This is true for all finite two player games. The other example games examined in this paper illustrate the different properties of recoverability. They also enable a comparison with the partnership dynamics simulated in C&W. The similarity in the stable points of the dynamics with the recoverable strategies in the games is striking. The simulation results show that recoverability is a reasonable approximation of the story envisioned in that paper. The results lend weight to the argument that an ERST gives a good indication of how a population might behave in a situation like the one constructed here. A natural extension of the model we have studied here would allow players to ‘‘mix’’ over their pair-types. In essence this would be equivalent to letting individuals choose endogenously the probability of staying with a particular player. A generalisation of this type should not be expected to yield significantly different results,
* Dugatkin (1995) provided the initial motivation. He states, ‘‘Any potential encounter can be divided into three phases: assortative, interactive and allocative . . . . During the assortative phase, individuals decide who to interact with, during the interactive phase, how to interact and during the allocative phase, how to divide the spoils. Most game theory models assume that strategic behaviour is employed only during the interactive phase. The allocative phase is assumed to be controlled by the payoff matrix, while the assortative phase is assumed to be purely random.’’
325
however, as it would seem likely that given the choice players would prefer to choose ‘‘pure’’ strategies. After all, immediate payoffs are affected only by action and not by pair-type. But this, or other ways of extending the choices players have over the manner in which they interact, could be an interesting avenue for further research. We would like to thank Martin Nowak, Peter So rensen, Jo¨rgen Weibull, David Myatt and seminar participants at Nuffield College, Oxford for helpful advice and comments. We are also grateful for the detailed and constructive comments of an anonymous referee. Financial support from the ESRC (grants R00429424072 and R00429534346, respectively) is acknowledged.
REFERENCES A, R. (1984). The Evolution of Cooperation. New York: Basic Books. C, B. & W, C. (1997). The evolution of partnerships. Soc. Methods Res. (in press). D, L. A. (1995). Partner choice, game theory and social behaviour. J. Quant. Anthropol. 5, 3–14. F, D. & T, J. (1991). Game Theory. Cambridge, MA: MIT Press. H, J. & W, J. W. (1996). Evolutionary selection against dominated strategies. J. Econ. Theory 71, 558–575. K, M. & R, R. (1995). Evolution of equilibria in the long run: a general theory and applications. J. Econ. Theory 65, 383–414. K, M., M, G. J. & R, R. (1993). Learning, mutation and long-run equilibria in games. Econometrica 61(1), 57–84. M S, J. (1982). Evolution and the Theory of Games. Cambridge: Cambridge University Press. M S, J. & P, G. R. (1973). The logic of animal conflict. Nature 246, 15–18. N, J. F. (1953). Two-person cooperative games. Econometrica 21, 128–140. N, M. & M, R. (1992). The spatial dilemmas of evolution. Int. J. Bifurc. Chaos 3, 35–78. N, M. & S, K. (1995). Invasion dynamics of the finitely repeated Prisoners’ Dilemma. Games Econ. Behav. 11, 364–390. N, M., S, K. & E-S, E. (1995). Automata, repeated games and noise. J. Math. Biol. 33, 703–722. S, M. E. (1988). Evolutionarily stable strategies for a finite population and a variable contest size. J. theor. Biol. 132, 469–478. T, R. (1971). The evolution of reciprocal altruism. Q. Rev. Biol. 46, 35–57. V-R, F. (1996). Evolution, Games and Economic Behaviour. Oxford: Oxford University Press. W, J. (1995). Evolutionary Game Theory. Cambridge, MA: MIT Press.
. .
326
Y, H. P. (1993a). The evolution of conventions. Econometrica 61(1), 57–84. Y, H. P. (1993b). An evolutionary model of bargaining. J. Econ. Theory 59, 145–168.
APPENDIX A Mathematical Proofs This Appendix contains the formal proofs omitted from the main text. .1. 1 A population containing one or more asymmetric fixed partnerships (type f ) cannot be internally stable. Proof This is fairly straightforward. Take a group of size W containing a player using strategy-type (x, f ). By definition of type f , the player is in a pair with a strategy-type (y, f ), where y $ x. This opponent is in the same group for all W e 1, but, since G is generic, p(x, y) $ p(y, x). By Definition 2 the population is not internally stable. q .2. 2 If players only observe their own strategies and payoffs and cannot form fixed partnerships, Definition 4 reduces to standard finite population ESS. Proof If players only observe themselves then W = 12 . Also set t = r. Now Definition 4 becomes: a strategy-pair type s = (x, r) is an Evolutionarily Recoverable Strategy-Type (ERST) if [sm $ s, such that for W = 12 , the following inequality holds: s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Y
y$Y
1
0
This is precisely the definition for finite population ESS in Schaffer (1988). q .3. 1 Any ERST where the population is randomly matching is an ESS. Proof This follows immediately. If the population are in an ERST, s = (x, r)* then the following is true: [sm $ s, \W q 0 such that [W E W , the following holds: s p(x, y)p(y =x, r) e s p(xm , y)p(y =xm , tm ) y$Y
y$Y
(A.1) Hence, in particular, if this holds [W E W then it holds for W = 12 . Further, if eqn (A.1) holds [sm $ s, then it holds for tm = r. But these are the conditions for Proposition 2 to apply. Hence s is a finite population ESS. q .4. 3 If s = (x, t) is an ERST then x $ arg maxy $ Y p(y, y). Conversely, if x $ arg maxy$Y p(y, y) then s = (x, f ) is an ERST. ERST maximise social efficiency. Proof Suppose s = (x, t) is an ERST. Then [sm $ s, \W q 0 such that [W E W , the following inequality holds: s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Y
(A.2)
2W 2W 1− p(x, x) + p(x, xm ) N−1 N−1
e 1−
N−2 1 p(x, x) + p(x, xm ) e p(xm , x) N−1 N−1
y$Y
which reduces to:
0
Hence:
1
2W − 1 2W − 1 p(xm , x) + p(xm , xm ) N−1 N−1
* That is, they are randomly matching.
Suppose further that x( arg maxy$Y p(y, y). Now it is possible to choose an xm $ arg maxy$Y p(y, y) such that xm $ x. Let sm = (xm , f ). Now following the probabilities from eqn (4), eqn (A.2) becomes: p(x, x) e p(xm , xm )
327
But this is not possible since xm maximises p(y, y) and x does not. This implies that s = (x, t) is not an ERST, a contradiction. So if s = (x, t) is an ERST then x $ arg maxy$Y p(y, y). For the second part suppose that x $ arg maxy$Y p(y, y), then p(x, x) e p(xm , xm ) for xm $ x. This is obviously true for all xm . Now s = (x, f ) is an ERST if [sm $ s, \W q 0 such that [W E W , the following inequality holds:
respectively. If x + y q 1 then neither player receives anything. The pure strategy Nash equilibria of this game are any demands (x, y) such that x + y = 1.* The notion of an ERST introduced here makes a more precise prediction: a fair split (x = y = 12 ). In order to show this, consider a discrete version of the demand game, where a pure strategy for player i is a demand xi from the set
s p(x, y)p(y =x, f ) e s p(xm , y)p(y =xm , tm )
1 2 n−1 Xn = 0, , , . . . ,1 † n n n
y$Y
6
y$Y
(A.3) Using the probability values in eqn (4), eqn (A.3) is equivalent to: p(x, x) e p(xm , xm )
There are two cases to consider, when n is even (and hence 12 $ Xn ) and when n is odd. The game below is an example of the payoff matrix when n = 2:
Which is true by assumption. q .5. 2 For every finite game there is always an ERST of the form s = (x, t), where x $ arg maxy$Y p(y, y) and t = f. Proof This follows immediately, since Proposition 3 shows that s = (x, t), where x $ arg maxy$Y p(y, y) and t = f is always an ERST, and there is always a maximum to the function p(y, y) if G is finite. q
7
1 2
0
1 1 2
0
1
0 0
0
1 2
0 1 2 1 2
0
1 2
0
0
0
0
0
1
APPENDIX B
1
0
0
The Nash Demand Game This Appendix provides an analysis of the Nash demand game. In this two-player game, each agent demands a fraction of some prize, normalised to be of value 1. Suppose player 1 demands x and player 2 demands y. If x + y E 1 then the payoffs to players 1 and 2 are x and y, * This is the standard simplification of the demand game found originally in Nash (1953). Young (1993b), using the same version of the game, provides an evolutionary analysis with uniform matching. † A discrete version is used since ERSTs have so far only been introduced for discrete action space games in Definition 4. As will be shown, there is little reason to think anything would change for the continuous action space game.
Consider first when 12 $ Xn . To find the ERST with the population in fixed partnerships, we need only maximise p(x, x). In this case p(x, x) = x if x + x = 2x E 1 and p(x, x) = 0 if 2x q 1. Clearly the maximum occurs at x = 12 . Hence playing xi = 12 in a fixed partnership is an ERST. Moreover, playing the same strategy but randomly matching is also an ERST: this follows since the demands (12 , 12 ) constitute an ESS. The case when n is odd and hence 12(Xn is somewhat different. Now p(x, x) = x if 2x Q 1 and p(x, x) = 0 if 2x q 1. The maximum occurs at x = 12 − 1/2n. The ERST therefore consists of a population demanding xi = 12 − 1/2n in fixed
. .
328
partnerships. This is still a fair outcome, although there is some wasted surplus. However, now a population playing the same strategy but randomly matching is no longer consistent with an ERST. In fact, it can be successfully invaded by a mutant which randomly matches and plays 1 1 xm = + 2 2n
0
10
1−
2W − 1 N−1
1
While an incumbent member of the population continues to receive an expected payoff of 1 1 − 2 2n
0
1 1 + 2 2n
10
1−
1
2W − 1 1 1 q − N−1 2 2n
for any 1 N−1 WQ + 2 n+1 Hence \W such that [W E W :
To see this, note that such a mutant would receive an expected payoff of 1 1 + 2 2n
Now:
s p(x, y)p(y =x, t) e s p(xm , y)p(y =xm , tm ) y$Xn
y$Xn
[sm = (xm , tm ) $
0
1
1 1 − ,r = (x, t) 2 2n
Finally, notice how as n : a (whether even or odd, when 1/2n : 0) the ERSTs consist of a population in fixed partnerships or randomly matching playing xi = 12 . Thus for the continuous strategy space game, the prediction remains the same.