A numerical analysis of the evolutionary stability of learning rules

ARTICLE IN PRESS

Journal of Economic Dynamics & Control 32 (2008) 1569–1599 www.elsevier.com/locate/jedc

A numerical analysis of the evolutionary stability of learning rules Jens Josephson Department of Economics and Business, Universitat Pompeu Fabra, Ramon Trias Fargas 25-27, 08005 Barcelona, Spain Received 10 November 2005; accepted 22 June 2007 Available online 4 July 2007

Abstract In this paper, we define an evolutionary stability criterion for learning rules. Using simulations, we then apply this criterion to three types of symmetric 2  2 games for a class of learning rules that can be represented by the parametric model of Camerer and Ho [1999. Experience-weighted attraction learning in normal form games. Econometrica 67, 827–874]. This class contains stochastic versions of reinforcement and fictitious play as extreme cases. We find that only learning rules with high or intermediate levels of hypothetical reinforcement are evolutionarily stable, but that the stable parameters depend on the game. r 2007 Elsevier B.V. All rights reserved. JEL classification: C72; C73; C15 Keywords: Monte Carlo simulation; Evolutionary stability; Learning in games; Fictitious play; Reinforcement learning; EWA learning

1. Introduction The bounded rationality paradigm is based on the assumption that people learn to play games by using simple rules of adaptation, often referred to as learning rules. In almost all of the literature, it is assumed that all players of the game employ the same Tel.: +34 93 542 22 68; fax: +34 93 542 17 46.

E-mail address: [email protected] 0165-1889/$ - see front matter r 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.jedc.2007.06.008

ARTICLE IN PRESS 1570

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

type of learning rule. Moreover, the type of rule is generally motivated by its asymptotic properties in such a homogeneous setting, such as convergence to Nash equilibrium. This seems questionable from an evolutionary perspective. Evolutionary selection should favor the use of learning rules that perform well in a heterogeneous setting with mutant learning rules present in the population. Furthermore, since most individual interactions are of limited length, survival should to a great extent be determined by the payoffs to using the learning rule in the short or intermediate run in such a heterogeneous setting. In this paper, we implement these ideas by defining an evolutionary stability criterion for learning rules and applying this criterion to a set of well-known learning rules using Monte Carlo simulations. More specifically, we ask if there is a rule such that, if applied by a homogeneous population of individuals, it will survive any sufficiently small invasion of mutants using a different rule. We call such an uninvadable rule an evolutionarily stable learning rule (ESLR). This concept is an extension of Taylor and Jonker’s (1978) definition of evolutionarily stable strategies (ESSs) (a concept originally due to Maynard Smith and Price, 1973; Maynard Smith, 1974) to learning rules and behavioral strategies. The setting is a world where the members of a large population, consisting of an even number of individuals, in each of a finite number of periods are all randomly matched in pairs to play a finite two-player game. Each individual uses a learning rule, which is a function of his private history of past play, and fitness is measured in terms of expected average payoff. This framework provides a rationale for the use of learning rules and it is of particular interest since very little analysis of learning in this ‘repeated rematching’ context has previously been done (see Hopkins, 1999, for an exception). Technically, learning rules are mappings from the history of past play to the set of pure or mixed strategies. There are many models of learning and we therefore restrict the numerical analysis to a class of learning rules that can be described by the general parametric model of Camerer and Ho (1999), called experience-weighted attraction (EWA) learning. The rules in this class have experimental support and perform well in an environment where the game changes from time to time.1 Moreover, the class contains rules which differ considerably in their use of information. Two of the most well-known learning rules, reinforcement learning and fictitious play, are special cases of this model for specific parameter values. Reinforcement learning is an important model in the psychological literature on individual learning. It was introduced by Bush and Mosteller (1951) and further developed by Erev and Roth (1998), although the principle behind the model, that choices which have led to good outcomes in the past are more likely to be repeated in the future, originally is due to Thorndike (1898). Under reinforcement learning in games, players assign probability distributions to their available pure strategies. If a pure strategy is employed in a particular period, the probability of the same pure strategy being used in the subsequent period increases as a function of the realized payoff. The model has very low information and rationality requirements in the 1

See, for example, Camerer and Ho (1999) and Stahl (2003).

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1571

sense that individuals need not know the strategy realizations of the opponents or the payoffs of the game; all that is necessary is knowledge of player-specific past strategy and payoff realizations. Fictitious play (Brown, 1951) is a model where the individuals in each of the player roles of a game in every period play a pure strategy that is a best reply to the accumulated empirical distribution of their opponents’ play. This means that knowledge of the opponents’ strategy realizations and the player’s own payoff function is required. Several different models of both reinforcement and fictitious play have been developed over the years. The ones that can be represented by Camerer and Ho’s (1999) model correspond to stochastic versions with exponential probabilities.2 This means that each pure strategy in each period is assigned an attraction, which is a function of the attraction in the previous period and the payoff to the particular strategy in the current period. The attractions are then exponentially weighted in order to determine the mixed strategy to be employed in the next period. In the case of reinforcement learning, the attractions only depend on the payoff to the pure strategy actually chosen. In the case of fictitious play, the hypothetical payoffs to the pure strategies that were not chosen are of equal importance (this is sometimes referred to as hypothetical reinforcement). However, Camerer and Ho’s (1999) model also permits intermediate cases where payoffs corresponding to pure strategies that were not chosen are given a weight strictly between zero and one. The weight of such hypothetical payoffs is given by a single parameter, d. Two other important parameters of the EWA model are the discount factor, f, which depreciates past attractions, and the attraction sensitivity, l, which determines how sensitive mixed strategies are to attractions. We assume all initial attractions are zero, such that the individuals have almost no prior knowledge of the game they are drawn to play. This implies that the analysis in this paper boils down to testing if any particular parameter vector ðd; l; fÞ of the EWA model corresponds to an ESLR. We do this by simulating a large number of runs in which all members of a finite population with a large share of incumbents and a small share of mutants are randomly matched in pairs to play a two-player game in each of a finite number of periods. We then calculate the average payoff for each share of the population. We consider games from each of the three generic categories of 2  2 games: dominance solvable, coordination, and hawk–dove. We find that in all games, we cannot reject the ESLR hypothesis for a learning rule with either high or intermediate level of hypothetical reinforcement, although the stable values of the d and s parameters depend on the payoffs. We also find that in all games, the ESLR hypothesis can be rejected for learning rules with low attraction sensitivity and for reinforcement learners. These results appear consistent with the positive levels of hypothetical reinforcement and differences in parameter estimates between games found in most experiments. 2 Fudenberg and Levine (1998) show that stochastic fictitious play can be motivated by a setting where payoffs are subject to noise and the player maximizes next period’s expected payoff given an empirical distribution of the opponents’ past play.

ARTICLE IN PRESS 1572

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

The main intuition for our results is the difference in the speed of learning. The null hypothesis of an ESLR cannot be rejected for the learning rule with the fastest convergence to the equilibrium population share in a homogeneous population in any of the games. Learning rules with low attraction sensitivity converge relatively slowly to the equilibrium strategies. Moreover, learning rules with high attraction sensitivity and either low levels of hypothetical reinforcement or high level of hypothetical reinforcement and low discount factor, risk getting stuck at an initial population distribution of strategies. This paper is organized as follows. Section 2 discusses related literature. Section 3 introduces the theoretical model underlying the simulations. Section 4 presents the results of the Monte Carlo simulations. Section 5 contains a discussion of the results and Section 6 concludes. Theoretical results can be found in Appendix A, and diagrams and tables of simulation results for homogeneous populations in Appendix B. Simulation results for heterogeneous populations can be found in the supplementary archive of this journal using the link provided in Appendix C.

2. Related literature The present paper is related to the theoretical literature on learning in games, but also to experimental tests of different learning rules. An early theoretical reference, asking similar questions, is Harley (1981). He analyzes the evolution of learning rules in the context of games with a unique ESS. He assumes the existence of an ESLR and then discusses the properties of such a rule. Harley finds that, given certain assumptions, ‘‘. . .the evolutionarily stable learning rule is a rule for learning evolutionarily stable strategies.’’ He also develops an approximation to such a rule and simulates its behavior in a homogeneous population. The current paper differs from that of Harley (1981) in that it explicitly formulates an evolutionary criterion for learning rules and does not assume the existence of an ESLR. Moreover, the analysis is not limited to games with a single ESS. Anderlini and Sabourian (1995) develop a dynamic model of the evolution of algorithmic learning rules. They find that under certain conditions, the frequencies of different learning rules in the population are globally stable and that the limit points of the distribution of strategies correspond to Nash equilibria. However, they do not investigate the properties of the stable learning rules. Heller (2004) explores the evolutionary fitness of learning to best reply (at a cost) relative to playing a pure strategy in an environment where the game changes stochastically over time. She shows asymptotic dominance of learning when it is strictly better, averaging over different regimes, than each pure strategy against any possible opponent.3 There are also a number of recent theoretical results on the asymptotic behavior of reinforcement learning and fictitious play. Hopkins (2002) shows that the expected motion of both stochastic fictitious play and perturbed reinforcement learning can be 3

For more on evolution of strategic sophistication see the surveys by Hommes (2006) and Robson (2001).

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1573

written as a perturbed form of the replicator dynamics, and that, in many cases, they will therefore have the same asymptotic behavior. In particular, he claims that they have identical local stability properties at mixed equilibria. He also demonstrates that the main difference between the two learning rules is that fictitious play gives rise to faster learning. Hofbauer and Sandholm (2002) establish global convergence results for stochastic fictitious play in games with an interior ESS, zero-sum games, potential games, and supermodular games. Berger (2005) proves convergence of fictitious play to Nash equilibria in nondegenerate 2  n games. Beggs (2005) gives convergence result under Erev and Roth (1998) type of reinforcement learning for constant-sum games and iteratively dominance solvable games. Hopkins and Posch (2005) show that this type of learning gives convergence with probability zero to points that are unstable under the replicator dynamics. In the experimental literature, the objective is generally to find the learning rule which gives the best fit of experimental data. Camerer and Ho (1999) give a concise overview of the most important findings in earlier studies. They argue that the overall picture is unclear, but that comparisons appear to favor reinforcement learning in constant-sum games and fictitious play in coordination games. In their own study of asymmetric constant-sum games, median-action games, and beautycontest games, they find support for a learning rule with parameter values in between reinforcement learning and fictitious play. Ho et al. (2002), Camerer et al. (2002) and Camerer (2003) consider extensions of the EWA model and report EWA parameter estimates for a number of experiments by themselves and other authors. Stahl (2003) compares the prediction performance of seven learning models, including a restricted version of the EWA model. He pools data from a variety of symmetric two-player games and finds a logit best-reply model with inertia and adaptive expectations to perform best, closely followed by the EWA. Hanaki et al. (2005) develop a model of learning where individuals in a first long-run phase with rematching generate limit attractions for a complete set of repeated game strategies that satisfy a complexity constraint, and in a second short-run phase, without rematching, use the limit attractions as initial attractions. They find that relative to existing models of learning, their model is better at explaining the behavior of subjects in environments where fairness and reciprocity appear to play a significant role. A pair of recent papers also demonstrate the difficulties in correctly estimating the parameters of learning models. Salmon (2001) simulates experimental data for a number of games and shows that the estimations of several well-known learning models, including EWA learning, sometimes do not accurately distinguish between the data generating processes. Wilcox (2006) shows that pooled tests of learning models tend to be biased towards reinforcement learning if the heterogeneity of subjects is not taken into account.

3. The model Let G be a symmetric two-player game in normal form, where each player has a finite pure-strategy set X ¼ f1; . . . ; Jg, with the mixed-strategy extension

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1574

P DðX Þ ¼ fp 2 RJþ j Jj¼1 pj ¼ 1g. Each player’s payoff is represented by the function p : X  X ! Rþ , where pðx; yÞ denotes the payoff to playing the pure strategy x when the opponent is playing the pure strategy y. In each of T periods, all individuals of a finite population, consisting of an even number MX4 of individuals, are drawn to play this game. The mixed strategy of individual k in period t 2 f1; 2; . . . ; Tg ¼ U is denoted by pk ðtÞ. The pure-strategy realization of individual k is denoted by xk ðtÞ and that of his opponent (in this period) by yk ðtÞ. Let ðxð0Þ; yð0ÞÞ be an initial pure-strategy profile, common to all individuals. The sequence hk ðtÞ ¼ ððxð0Þ; yð0ÞÞ; ðxk ð1Þ; yk ð1ÞÞ; . . . ; ðxk ðt  1Þ; yk ðt  1ÞÞÞ

(1)

is referred to as individual k’s history in period t. Let HðtÞ be the finite set of such possible histories at time t and let H ¼ [Tt¼1 HðtÞ. We define a learning rule as a function f : H ! DðX Þ that maps histories to mixed strategies and denote the set of possible learning rules by F. The matching procedure can be described as follows. In each of T periods, all members of the population are randomly matched in pairs to play the game G against each other. This can be illustrated by an urn with M balls, from which randomly selected pairs of balls (with equal probability) are drawn successively until the urn is empty. This procedure is repeated for a total of T periods, and the draws in each period are independent of the draws in all other periods. Each individual k receives a payoff pðxk ðtÞ; yk ðtÞÞ in each period and has a private history of realized strategy profiles. Let the expected payoff for an individual employing learning rule f in a heterogeneous population of size M, where the share of individuals employing rule f is 1  e and the share of individuals employing rule g is e, be denoted by V M ðf ; g; eÞ. Let F 0 be an arbitrary subset of F containing at least two elements. We define the following evolutionary stability criterion for learning rules.4 Definition 1. A learning rule f 2 F is evolutionarily stable in the class F 0 for 0 population   size M if, for every g 2 F nf , there exists an e^ g 2 ð2=M; 1Þ such that for all e 2 0; e^ g such that eM is integer, V M ðf ; g; eÞ4V M ðg; f ; 1  eÞ.

(2)

3.1. EWA learning In this paper, we focus on a set of learning rules that can be described by Camerer and Ho’s (1999) model of EWA learning. These are learning rules such that individual k’s probability of playing strategy j in period t 2 U can be written as j

pjk ðtÞ 4

elAk ðt1Þ

¼ PJ

l¼1 e

lAlk ðt1Þ

,

(3)

In the definition, we require a fraction of at least 2=M mutants in order to allow for interaction between them.

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1575

where l is a constant and Ajk ðt  1Þ is the attraction of strategy j.5 The latter is updated according to the formula Ajk ðtÞ ¼

fNðt  1ÞAjk ðt  1Þ þ ðd þ ð1  dÞIðj; xk ðtÞÞÞpðj; yk ðtÞÞ NðtÞ

(4)

for t 2 U, and Ajk ð0Þ is a constant. NðtÞ is a function of time given by NðtÞ ¼ fð1  kÞNðt  1Þ þ 1

(5)

for t 2 U, and Nð0Þ is a positive constant. Iðj; xk ðtÞÞ is an indicator function which takes the value of one if xk ðtÞ ¼ j and zero otherwise. Finally, yk ðtÞ is the realized pure strategy of the opponent in period t, and d, f, and k are constants in the unit interval. Note that this class of learning rules includes two of the most common learning rules used in the literature. When d ¼ 0, k ¼ 0, and Nð0Þ ¼ 1=ð1  fÞ, EWA reduces P to (average) reinforcement learning.6 When d ¼ 1, k ¼ 0 and Ajk ð0Þ ¼ Jl¼1 pðj; lÞqlk , where qlk is some initial relative frequency of strategy l, EWA becomes stochastic fictitious play. Also note that the EWA probabilities (3) depend on cardinal payoffs. However, multiplying all payoffs and the initial attractions by a positive constant has the same effect as multiplying l by the same constant. In order to make the analysis more tractable, we further restrict the set of rules to EWA learning rules such that k ¼ 0, Nð0Þ ¼ 1=ð1  fÞ, and Ajk ð0Þ ¼ 0 for all k and j.7 This means that NðtÞ  N ð0Þ and that the initial attractions will not generally correspond exactly to those of stochastic fictitious play. The assumption of k ¼ 0 implies that attractions are a weighted average of lagged attractions and past payoffs and the assumption of Nð0Þ ¼ 1=ð1  fÞ that weights are fixed. The assumption of equal initial attractions is motivated by a setting where the players have very limited information about the game before the first period and where they cannot use previous experience.8 We denote the set of rules with the above parameter values by F e . Substituting in (4) gives Ajk ðtÞ ¼ fAjk ðt  1Þ þ ð1  fÞðd þ ð1  dÞIðj; xk ðtÞÞÞpðj; yk ðtÞÞ

(6)

for t 2 U, and Ajk ð0Þ ¼ 0. The formula in (3) now corresponds to stochastic fictitious play (with modified initial weights) for d ¼ 1 and to reinforcement learning for d ¼ 0. The parameter d captures the extent to which the hypothetical payoffs of the 5

Camerer and Ho (1999) note that it is also possible to model probabilities as a power function of attractors. 6 Camerer and Ho (1999) distinguish between average and cumulative reinforcement, which results if d ¼ 0, k ¼ 1, and Nð0Þ ¼ 1. The analysis in this paper is based on average reinforcement. 7 Stahl (2003) finds that a time varying NðtÞ only improves the predictive power of the model marginally and assumes NðtÞ ¼ 1 for all t. He also assumes all initial attractors are zero and uses the updating formula to determine the attractors in period one. 8 Although the game is fixed in the analysis below, a rationale for the assumption of uniform initial weight could be a setting where the game is drawn at random from some set of games before the first round of play.

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1576

pure strategies not played in a period are taken into account and f is a constant determining the relative weights of recent and historical payoffs in the updating of mixed strategies.

4. Numerical analysis The analysis is based on Monte Carlo simulations of repeated encounters between individuals using different learning rules with parameter vectors in the finite grid D ¼ fðl; f; dÞ : l 2 f1; 10g; f; d 2 f0; 0:25; 0:5; 0:75; 1gg, with representative element d.9 We focus on games from the three generic categories of 2  2 games: dominance solvable, coordination, and hawk–dove, with payoffs in the interval ½1; 5. In the simulations, each member of a population of 20 individuals, among which 2 are mutants with a different learning rule, is randomly matched with another member in each of T ¼ 10, 20, or 30 periods. The expected payoff to a learning rule is estimated by computing the mean of the average payoff for all individuals with the same learning rule in the population and by simulating 1000 such T-period population interactions. Since the mean payoff difference in each simulation is independently and identically distributed relative to the mean payoff difference in another simulation with the same population mixture, the central limit theorem applies and the mean payoff difference is approximately normally distributed. For each value of d, the null hypothesis is that the corresponding learning rule is an ESLR. This hypothesis is rejected if the mean payoff to any mutant rule is statistically significantly higher than the mean payoff to the incumbent rule in the class, in accordance with Definition 1 above. More specifically, the decision rule is as follows. The null hypothesis, Hd0 :

f d is an ESLR in the class F e ,

is rejected in favor of the alternative hypothesis, Hd1 :

f d is not an ESLR in the class F e ,

if and only if, for some d 0 2 Dnd, M M V^ ðf d ; f d 0 ; eÞ  V^ ðf d 0 ; f d ; 1  eÞ (7) o  za , SE M ðf d ; f d 0 ; eÞ M where V^ ð:; :; :Þ denotes the estimated mean of the average payoff, SE M ð:; :; :Þ denotes the sample standard error of the difference in average payoffs, computed over the 1000 simulations, and za is the critical value of the standard normal distribution at the a% level of significance.10

9

f ¼ 1 implies that each strategy is played with equal probability in every period and it is included for the sake of comparison. 10 For detailed simulation results with heterogeneous populations, see the supplementary archive of this journal.

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

2 1

1577

4 3

Fig. 1.

3 2

5 1

Fig. 2.

4.1. Dominance solvable games In dominance solvable games, a learning rule that plays the dominant strategy will fare better than one that plays the dominated strategy against any opponent. We would thus expect the ESLR to be the rule that induces play of the dominant strategy with the highest probability. The theoretical results in Appendix A help us find the parameter values of this rule. To start with, Lemma 1 says that for any individual and any history, the attraction of the dominant strategy is strictly larger than that of the dominated strategy for ^ period tX1 if d is above d^ ¼ maxfpð2; 1Þ=pð1; 1Þ; pð2; 2Þ=pð1; 2Þg, where 0odo1 (for 3 2 ^ ^ the game in Fig. 1, d ¼ 4, and for the game in Fig. 2, d ¼ 3Þ. Lemma 2 says that if the attraction of strategy one is larger than that of strategy two, then the conditional probability of playing strategy one is increasing in l and converges to one as l tends ^ the share to infinity. Proposition 1 uses these two results to prove that, if d is above d, of incumbents playing the dominant strategy in period tX2 in an infinite population tends to one as l tends to infinity. Hence, it appears, among the learning rules with high d, that we should expect the rules with higher l to do better. According to Proposition 2, the share of the population playing the dominant strategy in period two is above 0.5, increasing in d and, for sufficiently high d, decreasing in f. Simulations of a homogeneous population (see Figs. B1–B4 and Tables B1 and B2 in Appendix B.1) show that the share is increasing in d also for later periods, but for rules with small f, differences are small and the relationship not necessarily monotonic. We would thus expect learning rules with l ¼ 10, d ¼ 1, and small f to do particularly well. Conversely, we would expect learning rules with high l and d ¼ 0 to perform particularly bad in games with strictly positive payoffs, since they have a tendency to lock in on a particular strategy in a first period. The reason is that no matter what an individual plays in the first period, this strategy will be played with a probability of approximately one in the second period. If the strategy is played in period two, then it will be repeated with a probability of approximately one in the third period, etc. This feature has previously been noted by Fudenberg and Levine (1998). Recall that the null hypothesis that a learning rule with a particular d is an ESLR can be rejected if the z-statistic of the simulated difference in average payoffs is

ARTICLE IN PRESS 1578

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

smaller than the critical value of the standard normal distribution, za , for some mutant learning rule in the class, different from f d . For the game in Fig. 1 (where payoffs correspond to the row player) and T ¼ 10; 20; 30, the null of an ESLR can be rejected at the 1% level for all learning rules except the three rules with l ¼ 10, f 2 f0; 0:25; 0:5g, and d ¼ 1. For the game in Fig. 2, we obtain similar result: at the 5% level and for T ¼ 10; 20; 30, the null of an ESLR can be rejected for all learning rules except the three rules with l ¼ 10, f 2 f0; 0:25; 0:5g, and d ¼ 1. The only difference is that at the 1% level we also cannot reject the learning rules with l ¼ 10, f 2 f0; 0:25g, and d ¼ 0:75. 4.2. Coordination games In 2  2 coordination games, we conjecture that the learning rule that gives the fastest convergence to one of the pure equilibria in a homogeneous population will be an ESLR. When we simulate such a setting, we find that the parameters of the learning rule with the fastest convergence depend on the payoffs of the game. In the game in Fig. 3, the pure-strategy profile (1,1) is the unique risk and Pareto dominant equilibrium. The learning rule with l ¼ 10, d ¼ 1 and f ¼ 0:75 tends to this equilibrium and gives the fastest rate of convergence of the average strategy to an equilibrium in a homogeneous population (see Figs. B5 and B6 and Table B3 in Appendix B.3). What is the intuition behind this result? It is clear that learning rules with high l and d ¼ 0 lock in on initial strategies for the same reasons as in dominance solvable games. However, whereas the fastest convergence is achieved for d ¼ 1 and f ¼ 0 in such games, here there is a trade-off between a high d and a low f, and the optimal combination of the two seems to depend on the payoffs of the game in a non-trivial way. A high f means that a lot of weight is put on previous period’s attractions when the attractions are updated, creating inertia on an individual level. In the extreme case when f ¼ 1, there is no updating at all and both strategies are played with equal probabilities in every period. On the other hand, if d and l are high, then a low f implies that the share of the population playing a particular strategy may get stuck at the initial level (see Table 5). Consider the extreme case when all individuals use learning rules with l ¼ 10, f ¼ 0, and d ¼ 1 to play the game in Fig. 3. In this situation, only two attraction vectors are possible in each period tX1: (A1k ðtÞ,A2k ðtÞ) is either ð5; 2Þ (if the opponent plays strategy one) or ð1; 3Þ (if the opponent plays strategy two). The first vector induces play of strategy one with a probability of approximately one and the

5 2

1 3

Fig. 3.

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

4 2

1579

1 3

Fig. 4.

second vector induces play of strategy two with a probability of approximately one. This implies that if two individuals with equal attractions are matched, they will both play the same strategy in the next period with a probability of approximately one. If two individuals with different attractions are matched, they will both switch strategy with a probability of approximately one. Hence, with a high probability the initial shares of the two strategies in the population will be constant in subsequent periods. The simulations with a heterogeneous population are consistent with the results on the speed of convergence for a homogeneous population. For the game in Fig. 3 with a heterogeneous population interacting for T ¼ 20 or 30 periods, the null hypothesis of an ESLR can be rejected at the 5% level for all learning rules except for the one with parameter vector l ¼ 10, f ¼ 0:75, d ¼ 1. If T ¼ 10 or the level of significance is 1%, we also cannot reject the rule with l ¼ 10, f ¼ 0:5, d ¼ 0:75. However, these results are sensitive to changes in payoffs. If we, for instance, change the payoff of the strategy profile (1,1) to 4, such that both equilibria become risk dominant (see Fig. 4), the learning rule with the fastest convergence of the average strategy played to one of the equilibria tends to (2,2) and has parameters l ¼ 10, f ¼ 0, and d ¼ 0:5 (see Table B4 and Figs. B7 and B8 in Appendix B.4). The rule with l ¼ 10, f ¼ 0:75, and d ¼ 1 also gives fast convergence in this case, but a histogram reveals that for this rule the population average converges to each equilibria with a probability of approximately one-half. When we simulate the population interaction for this game, the null hypothesis of an ESLR can be rejected at the 5% level for all learning rules except for the ones with parameter vectors such that l ¼ 10, f 2 f0; 0:25g, d ¼ 0:5 (for all time horizons), l ¼ 10, f ¼ 0:75, d ¼ 1, and l ¼ 10, f ¼ 0:5, d ¼ 0, 75 (for T ¼ 20, 30). At the 1% level we also cannot reject the rule with l ¼ 10, f ¼ 0:75, d ¼ 0:75 for T ¼ 20, 30. 4.3. Hawk– dove games In the hawk–dove game, there is a unique and completely mixed Nash equilibrium which is also evolutionarily stable. A conjecture is that a learning rule that converges fast to a neighborhood of this equilibrium will be an ESLR. In the hawk–dove game in Fig. 5, the unique Nash equilibrium is to play strategy one with probability 0.6. When we simulate the interaction for a homogeneous population, the learning rules with l ¼ 10, f ¼ 0:75, and d 2 f0:75; 1g appear to have the fastest convergence of the population average to the equilibrium (see Figs. B9 and B10 and Table B5 in Appendix B.5). Learning rules that will do particularly bad are those that lock in on a strategy different from the equilibrium one at an early stage. This is the case for learning rules with high d and l, and low f, for similar reasons as in the coordination games and

ARTICLE IN PRESS 1580

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1 3

5 2

Fig. 5.

1 4

3 2

Fig. 6.

for learning rules with high l and d ¼ 0 for the same reason as explained above under dominance solvable games. Consistent with the findings for the homogeneous population, the ESLR hypothesis can be rejected at the 5% level for all rules except the ones with l ¼ 10, f ¼ 0:75, and d 2 f0:75; 1g when T ¼ 10; 20; 30. At the 1% level, the hypothesis also cannot be rejected for the learning rule with l ¼ 10, f ¼ 0:5, and d ¼ 0:75. However, the evolutionarily stable value of d is affected by changes in payoffs. For the game in Fig. 6, where the unique Nash equilibrium is to play strategy one with probability 0:25, the learning rule with l ¼ 10, f ¼ 0:75, and d ¼ 0:5 appears to give the fastest convergence of the average strategy to the equilibrium in a homogeneous population (see Figs. B11 and B12 and Table B6 in Appendix B.6). The simulations with a heterogeneous population confirm this finding: the ESLR hypothesis cannot be rejected for this rule for T ¼ 20; 30. 4.4. Summary of simulation results Tables 1 and 2 summarize the results of the simulations. In all games, the ESLR hypothesis can be rejected for all learning rules with l ¼ 1 and all learning rules with d ¼ 0. Learning rules with high attraction sensitivity and either low degrees of hypothetical reinforcement or high degree of hypothetical reinforcement and low discount factor risk getting stuck at an initial population distribution of strategies. Some type of stochastic fictitious play (d ¼ 1) is an ESLR in all games except one of the hawk–dove games, but there is no learning rule which is an ESLR in all settings.

5. Discussion The results in this paper indicate that an ESLR is indeed a rule for learning an ESS, as predicted by Harley (1981) in a different framework. More precisely, it appears to be the rule with the fastest convergence of the population’s strategy distribution to such a strategy.

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1581

Table 1 Parameters for which the ESLR hypothesis cannot be rejected at the 5% level Game

Dominance solvable (Fig. 1) Dominance solvable (Fig. 2) Coordination (Fig. 3) Coordination (Fig. 4)

Hawk–dove (Fig. 5) Hawk–dove (Fig. 6)

T

10; 20; 30 10; 20; 30 10; 20; 30 10 20; 30 10; 20; 30 20; 30 10; 20; 30 20; 30 10

ESLR(s) at the 5% level l

f

d

10 10 10 10 10 10 10 10 10 No ESLR

0; 0:25; 0:5 0; 0:25; 0:5 0:75 0:5 0:75 0; 0:25 0:5 0:75 0:75

1 1 1 0:75 1 0:5 0:75 0:75; 1 0:5

Table 2 Parameters for which the ESLR hypothesis cannot be rejected at the 1% level Game

Dominance solvable (Fig. 1) Dominance solvable (Fig. 2) Coordination (Fig. 3) Coordination (Fig. 4)

Hawk–dove (Fig. 5) Hawk–dove (Fig. 6)

T

10; 20; 30 10; 20; 30 10; 20; 30 10; 20; 30 10; 20; 30 10; 20; 30 10; 20; 30 10; 20; 30 20; 30 10; 20; 30 10; 20 20; 30 10

ESLR(s) at the 1% level l

f

d

10 10 10 10 10 10 10 10 10 10 10 10 No ESLR

0; 0:25; 0:5 0; 0:25; 0:5 0; 0:25 0:75 0:5 0:75 0; 0:25 0:5 0:75 0:75 0:5 0:75

1 1 0:75 1 0:75 1 0:5 0:75 0:75 0:75; 1 0:75 0:5

The ESLR hypothesis can be rejected for reinforcement learning in all of the games analyzed. Our analysis of homogeneous populations confirms Hopkins’ (2002) finding (in a different matching environment) that stochastic fictitious play gives rise to faster learning. When attraction sensitivity is high, reinforcement learners even risk getting stuck at an initial strategy, as previously noted by Fudenberg and Levine (1998). However, it is important to bear in mind that reinforcement learning has lower information and computational requirements than other EWA rules. The perceived evolutionary disadvantage of this learning rule is thus conditional on sufficiently low costs of obtaining the additional information and performing the extra calculations necessary for hypothetical reinforcement. On the other hand, the results in this paper

ARTICLE IN PRESS 1582

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

may be interpreted as an evolutionary rationale for learning rules with such low costs (see Hommes, 2006; Robson, 2001 for more on evolutionary advantages to strategic sophistication). In all of the games studied, we cannot reject the ESLR hypothesis for at least one learning rule with high or intermediate level of hypothetical reinforcement, but there are differences in the stable values of the parameters between games. Our analysis assumes the learning rules cannot condition on the type of game being played, but the differences between games provide evolutionary support for such conditioning. This seems consistent with most experimental evidence. Many papers find that EWA learning does a good job at explaining the observed behaviors in experiments, but the parameter estimates vary widely from game to game. Camerer and Ho (1999) estimate separate sets of parameters for asymmetric constant-sum games, median-action games, and beauty-contest games. Their estimates of the degree of hypothetical enforcement d are generally around 0.5, those of the discount factor f in the range of 0.8–1.0, those of the attraction sensitivity l varies from 0:2 to 18, and those of k are in the range 0–1. Stochastic reinforcement learning and fictitious play are generally rejected in favor of an intermediate model. Camerer et al. (2002) reports EWA parameter estimates of d and f by themselves and other authors from 31 data sets. The estimated d ranges from 0 in 4  4 mixedstrategy games, patent race Games with symmetric players, and functional alliance games to 0.95 in p-beauty-contest games, with an average of about 0.3 and a median of about 0.2. The estimated f ranges from 0.11 in p-beauty games to 1.04 in 4  4 mixed-strategy games with an average and median of about 0:9. Ho et al. (2002) reports EWA parameter estimates for seven games. The estimated d ranges from 0.27 in mixed-strategy games to 0.89 in median-action games, with a value of 0.76 when all games are pooled. The estimated f ranges from 0.36 in the p-beauty contest to 0.98 in mixed-strategy games, with a pooled estimate of 0.78. The estimated value of l ranges from 0.19 in pot games to 42.21 in patent race games, with a pooled estimate of 2.95. Stahl (2003) conducts experiments with finite symmetric two-player games with and without symmetric pure-strategy equilibria. He pools data from several such games and estimates a d of 0.67, a f of 0.33, and a l of 0.098 for games with payoffs between 0 and 100. He assumes that the initial attractions of the EWA model are zero and uses the updating formula to determine their values in period one. Most experimental studies thus lend support to the hypothesis that people take hypothetical payoffs into account, but the estimated d is generally smaller than one. The estimated f is generally in the upper half of the unit interval, consistent with our findings for the hawk–dove and coordination games. Finally, the estimated l varies widely, which is not surprising given its aforementioned dependence on payoffs. One should, however, be cautious in making a direct comparison between the results in this paper and the experimental findings. Apart from certain econometric difficulties in estimating learning models (see Salmon, 2001; Wilcox, 2006), the games played in most experiments differ considerably from the ones analyzed in this paper.

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1583

Moreover, most studies (with the exception of Stahl, 2003) do not restrict initial attractions to zero or assume NðtÞ is constant as we do in this paper.

6. Conclusion In this paper, we define an evolutionary stability criterion for learning rules. We then apply this criterion to a class of rules which contains stochastic versions of two of the most well-known learning rules, reinforcement learning and fictitious play, as well as intermediate rules in terms of hypothetical reinforcement. We perform Monte Carlo simulations for three types of 2  2 games of a matching scheme where all members of a large population are randomly matched in pairs in every period. We find that the learning rule with the fastest convergence to an ESS in a homogeneous population is evolutionarily stable in all games. Reinforcement learning is never evolutionarily stable, mainly because it either gets stuck at an initial mixed strategy or converges more slowly than other learning rules. Some types of stochastic fictitious play are evolutionarily stable in all but one game, but the stable value of the discount factor for this rule depends on the type of game. For two types of games we also find evidence of ESLRs with intermediate levels of hypothetical reinforcement. The results appear consistent with the positive levels of hypothetical reinforcement and differences in parameter estimates between games found in most experiments. Acknowledgments I am indebted to Jo¨rgen W. Weibull for many inspiring conversations. I am also grateful to Colin Camerer, Francisco Pen˜aranda, Ariel Rubinstein, Maria SaezMarti, the editor, and two anonymous referees, as well as seminar audiences at the Cornell University, the Society for Economic Dynamics 2001 Annual Meeting in Stockholm, and the 16th Annual Congress of the European Economic Association in Lausanne, for helpful comments. Part of this work was completed at the Department of Economics of Cornell University, which I thank for its hospitality. Financial support from the Jan Wallander and Tom Hedelius Foundation, the BBVA Foundation through the project ‘Aprender a jugar’, the Spanish Ministry of Education and Science through Grants SEJ2004-03149 and SEJ2006-09993, and Barcelona Economics (CREA) is also gratefully acknowledged. Appendix A. Theoretical results Let G be a symmetric dominance solvable 2  2 game where strategy one strictly dominates strategy two and all payoffs are positive. Further, assume that all learning rules belong to F e and that f 2 ½0; 1Þ. Lemma 1. If d4d^ ¼ maxfpð2; 1Þ=pð1; 1Þ; pð2; 2Þ=pð1; 2Þg, then A1k ðtÞ4A2k ðtÞ for all tX1.

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1584

Proof. This follows immediately from the definition of Ajk ðtÞ by observing that ^ and any history of strategy realizations, A1k ð0Þ ¼ A2k ð0Þ ¼ 0, and that for d4d, ðd þ ð1  dÞIð1; xk ðtÞÞÞpð1; yk ðtÞÞ4ðd þ ð1  dÞIð2; xk ðtÞÞÞpð2; yk ðtÞÞ.

(A.1)

Lemma 2. 1 2 qðp1k ðtÞ=p2k ðtÞÞ ¼ ðA1k ðt  1Þ  A2k ðt  1ÞÞelðAk ðt1ÞAk ðt1ÞÞ ql

and

8 > 0 if A1k ðt  1ÞoA2k ðt  1Þ; > < 1 2 1 lim p1k ðtÞ ¼ 2 if Ak ðt  1Þ ¼ Ak ðt  1Þ; > l!1 > : 1 if A1 ðt  1Þ4A2 ðt  1Þ: k k

(A.2)

(A.3)

Proof. This follows immediately from the definition of pjk ðtÞ. Now suppose that an infinite population of measure one, composed of a share ð1  eÞ40 of incumbents and a share eX0 of mutants, is repeatedly matched to play the dominance solvable game. Let the fraction of the incumbents playing strategy one in period tX1 be denoted by x1 ðtÞ. Assume that the incumbents have parameter vector ðl; f; dÞ and that the mutants have the parameter vector ðl0 ; f0 ; d0 Þ. ^ then liml!1 x1 ðtÞ ¼ 1 for tX2. Proposition 1. If d4d, Proof. Consider the set of incumbents with minimal difference A1k ðt  1Þ  A2k ðt  1Þ at time tX2. The measure of this set is a function of l (the difference is not), but it is bounded from above by 1  e and below by zero, and by Lemma 1, A1k ðt  1Þ4A2k ðt  1Þ. Hence, by Lemma 2 the fraction of incumbents playing strategy one in this set in period tX2 converges to one as l tends to infinity. It is obvious that the same must hold for the set of all other incumbents, proving the statement. Proposition 2. The following holds for x1 ð2Þ: (i) x1 ð2Þ is strictly increasing in d. (ii) x1 ð2Þ40:5. (iii) For d sufficiently close to 1, x1 ð2Þ is strictly decreasing in f. Proof. Assume without loss of generality that strategy one is dominant. To prove (i), note that 2

1

2

1

qp1k ðt þ 1Þ=qA1k ðtÞ ¼ lelðAk ðtÞAk ðtÞÞ =ð1 þ elðAk ðtÞAk ðtÞÞ Þ2 , qp1k ðt þ 1Þ=qA2k ðtÞ ¼ le

lðA2k ðtÞA1k ðtÞÞ

=ð1 þ e

lðA2k ðtÞA1k ðtÞÞ

Þ2 .

ðA:4Þ ðA:5Þ

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1585

If player k uses strategy one and his opponent strategy j ¼ 1; 2 in period t, then qA1k ðtÞ=qd ¼ 0 and qA2k ðtÞ=qd ¼ ð1  fÞpð2; jÞ. If he uses strategy two and his opponent strategy j ¼ 1; 2, then qA1k ðtÞ=qd ¼ ð1  fÞpð1; jÞ and qA2k ðtÞ=qd ¼ 0. Exactly one-fourth of the incumbents (and mutants if they have positive measure) obtain the payoff pð1; 1Þ in period one, another fourth the payoff pð1; 2Þ, a third fourth the payoff pð1; 2Þ, and the final fourth the payoff pð2; 2Þ. This means that 4qx1 ð2Þ=qd ¼  ð1  fÞpð2; 1Þlelð1fÞðdpð2;1Þpð1;1ÞÞ =ð1 þ elð1fÞðdpð2;1Þpð1;1ÞÞ Þ2  ð1  fÞpð2; 2Þlelð1fÞðdpð2;2Þpð1;2ÞÞ =ð1 þ elð1fÞðdpð2;2Þpð1;2ÞÞ Þ2 þ ð1  fÞpð1; 1Þlelð1fÞðpð2;1Þdpð1;1ÞÞ =ð1 þ elð1fÞðpð2;1Þdpð1;1ÞÞ Þ2 þ ð1  fÞpð1; 2Þlelð1fÞðpð2;2Þdpð1;2ÞÞ =ð1 þ elð1fÞðpð2;2Þdpð1;2ÞÞ Þ2 . ðA:6Þ The function ex =ð1 þ ex Þ2 ¼ 1=ðex þ 2 þ ex Þ

(A.7)

is positive, symmetric around x ¼ 0, and decreasing for x40. Moreover, since in the dominance solvable game, 0opð2; jÞopð1; jÞ, for j ¼ 1; 2, it follows that jpð2; jÞ  dpð1; jÞjpjdpð2; jÞ  pð1; jÞj

(A.8)

with equality only for d ¼ 1. These two observations together imply that qx1 ð2Þ= qd40 for all 0pdp1. To prove (ii), note that equal shares of the incumbents obtain the four possible payoffs in period one. If d ¼ 0, then p1k ð2Þ is higher among the individuals who played strategy one and were matched with an individual playing strategy j in the first period, than p2k ð2Þ among the individuals who played strategy two and were matched with an individual playing strategy j in the first period. Since this holds for any j ¼ 1; 2, it is obvious that x1 ð2Þ40:5. By (i), this result extends to any d 2 ½0; 1. (iii) If player k uses strategy one and his opponent strategy j ¼ 1; 2 in period t, then qA1k ðtÞ=qf ¼ pð1; jÞ and qA2k ðtÞ=qf ¼ dpð2; jÞ. If he uses strategy two and his opponent strategy j ¼ 1; 2, then qA1k ðtÞ=qf ¼ dpð1; jÞ and qA2k ðtÞ=qf ¼ pð2; jÞ. This gives the following partial: 4qx1 ð2Þ=qf ¼ ðdpð2; 1Þ  pð1; 1ÞÞlelð1fÞðdpð2;1Þpð1;1ÞÞ =ð1 þ elð1fÞðdpð2;1Þpð1;1ÞÞ Þ2 þ ðdpð2; 2Þ  pð1; 2ÞÞlelð1fÞðdpð2;2Þpð1;2ÞÞ =ð1 þ elð1fÞðdpð2;2Þpð1;2ÞÞ Þ2 þ ðpð2; 1Þ  dpð1; 1ÞÞlelð1fÞðpð2;1Þdpð1;1ÞÞ =ð1 þ elð1fÞðpð2;1Þdpð1;1ÞÞ Þ2 þ ðpð2; 2Þ  dpð1; 2ÞÞlelð1fÞðpð2;2Þdpð1;2ÞÞ =ð1 þ elð1fÞðpð2;2Þdpð1;2ÞÞ Þ2 .

ðA:9Þ By continuity it immediately follows that the expression is negative for d sufficiently close to 1.

ARTICLE IN PRESS 1586

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

Appendix B. Simulation results for homogeneous populations The following diagrams and tables illustrate the average share of individuals playing strategy one in each period in a homogeneous population of size 20 over 1000 simulations with l ¼ 10. In the tables, the value of d is given by the entry in the first row, the value of f by the entry in the second row, and the time period by the entry in the left-most column. The learning rule with the fastest convergence to the equilibrium strategy is indicated in bold. The average population share playing strategy one for various cases are shown in Figs. B1–B12 and Tables B1–B6. B.1. Dominance solvable game in Fig. 1 For further details, see Table B1 and Figs. B1 and B2 B.2. Dominance solvable game in Fig. 2 For further details, see Table B2 and Figs. B3 and B4 B.3. Coordination game in Fig. 3 For further details, see Table B3 and Figs. B5 and B6 B.4. Coordination game in Fig. 4 For further details, see Table B4 and Figs. B7 and B8 B.5. Hawk– dove game in Fig. 5 For further details, see Table B5 and Figs. B9 and B10 B.6. Hawk– dove game in Fig. 6 For further details, see Table B6 and Figs. B11 and B12

Appendix C. Supplementary data Supplementary data associated with this article can be found in the online version at doi: 10.1016/j.jedc.2007.06.008.

Table B1 Average population share playing strategy one in the dominance solvable game in Fig. 1 0.00 0.25

0.00 0.50

0.00 0.75

0.00 1.00

0.25 0.00

0.25 0.25

0.25 0.50

0.25 0.75

0.25 1.00

0.50 0.00

0.50 0.25

0.50 0.50

0.50 0.75

0.50 1.00

0.75 0.00

0.75 0.25

0.75 0.50

0.75 0.75

0.75 1.00

1.00 0.00

1.00 0.25

1.00 0.50

1.00 0.75

1.00 1.00

t¼1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500

0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508 0.508

0.505 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507 0.507

0.502 0.521 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523 0.523

0.497 0.503 0.502 0.502 0.496 0.498 0.500 0.497 0.499 0.491 0.495 0.505 0.498 0.506 0.497 0.494 0.492 0.497 0.499 0.500 0.500 0.502 0.497 0.502 0.501 0.496 0.506 0.502 0.500 0.500

0.502 0.504 0.506 0.507 0.509 0.511 0.512 0.514 0.516 0.517 0.520 0.521 0.523 0.524 0.526 0.528 0.529 0.531 0.532 0.533 0.535 0.537 0.538 0.540 0.542 0.544 0.545 0.546 0.548 0.549

0.500 0.505 0.506 0.507 0.508 0.509 0.509 0.510 0.510 0.511 0.512 0.512 0.513 0.514 0.514 0.515 0.515 0.516 0.517 0.517 0.518 0.518 0.519 0.519 0.520 0.520 0.521 0.522 0.523 0.523

0.504 0.523 0.526 0.527 0.528 0.528 0.528 0.529 0.529 0.530 0.530 0.530 0.531 0.531 0.531 0.532 0.532 0.532 0.532 0.533 0.533 0.533 0.534 0.534 0.535 0.535 0.535 0.535 0.536 0.536

0.502 0.555 0.568 0.572 0.574 0.574 0.575 0.575 0.575 0.576 0.576 0.576 0.576 0.576 0.577 0.577 0.577 0.577 0.577 0.577 0.577 0.577 0.577 0.578 0.578 0.578 0.578 0.578 0.578 0.578

0.500 0.499 0.499 0.506 0.503 0.504 0.493 0.500 0.502 0.505 0.501 0.503 0.506 0.499 0.500 0.501 0.503 0.498 0.501 0.501 0.498 0.502 0.501 0.499 0.496 0.504 0.508 0.497 0.498 0.499

0.501 0.624 0.736 0.830 0.896 0.938 0.966 0.981 0.989 0.994 0.997 0.998 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.503 0.628 0.687 0.742 0.795 0.841 0.881 0.914 0.938 0.958 0.971 0.981 0.988 0.992 0.994 0.996 0.997 0.998 0.999 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.496 0.625 0.676 0.716 0.750 0.782 0.814 0.844 0.869 0.892 0.913 0.929 0.942 0.953 0.961 0.968 0.974 0.979 0.983 0.986 0.989 0.991 0.992 0.994 0.995 0.997 0.997 0.997 0.998 0.998

0.499 0.638 0.705 0.746 0.775 0.799 0.821 0.841 0.858 0.874 0.888 0.901 0.913 0.924 0.933 0.941 0.948 0.955 0.960 0.965 0.970 0.973 0.977 0.980 0.982 0.984 0.986 0.988 0.989 0.990

0.502 0.495 0.495 0.496 0.497 0.502 0.496 0.500 0.504 0.501 0.497 0.501 0.497 0.508 0.498 0.499 0.498 0.501 0.499 0.503 0.498 0.501 0.504 0.494 0.493 0.502 0.500 0.499 0.499 0.496

0.499 0.877 0.991 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.501 0.870 0.988 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.499 0.850 0.979 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.501 0.806 0.949 0.990 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.500 0.503 0.505 0.506 0.506 0.502 0.499 0.501 0.504 0.500 0.498 0.497 0.502 0.499 0.504 0.499 0.495 0.501 0.507 0.501 0.495 0.501 0.502 0.498 0.498 0.499 0.501 0.494 0.497 0.494

0.503 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.504 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.499 0.993 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.501 0.921 0.988 0.997 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.498 0.498 0.500 0.503 0.502 0.502 0.497 0.505 0.500 0.503 0.502 0.501 0.495 0.497 0.498 0.501 0.500 0.501 0.496 0.501 0.496 0.500 0.498 0.501 0.505 0.504 0.501 0.501 0.496 0.497

ARTICLE IN PRESS

0.00 0.00

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

d f

1587

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1588 1.1

Av. fraction playing strategy one

1.0

0.9

0.8

=0  = 0.25  = 0.5  = 0.75  =1

0.7

0.6

0.5

0.4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B1. Average population share playing strategy one in the dominance solvable game in Fig. 1 for different values of f, holding d ¼ 1 fixed.

1.1

Av. fraction playing strategy one

1.0

0.9

0.8

0.7

=0  = 0.25  = 0.5  = 0.75 =1

0.6

0.5

0.4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B2. Average population share playing strategy one in the dominance solvable game in Fig. 1 for different values of d, holding f ¼ 0 fixed.

Table B2 Average population share playing strategy one in the dominance solvable game in Fig. 2 0.00 0.25

0.00 0.50

0.00 0.75

0.00 1.00

0.25 0.00

0.25 0.25

0.25 0.50

0.25 0.75

0.25 1.00

0.50 0.00

0.50 0.25

0.50 0.50

0.50 0.75

0.50 1.00

0.75 0.00

0.75 0.25

0.75 0.50

0.75 0.75

0.75 1.00

1.00 0.00

1.00 0.25

1.00 0.50

1.00 0.75

1.00 1.00

t¼1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498

0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500

0.499 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501

0.502 0.523 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525 0.525

0.495 0.503 0.496 0.496 0.493 0.501 0.500 0.501 0.494 0.499 0.500 0.500 0.498 0.506 0.501 0.498 0.504 0.499 0.503 0.500 0.493 0.494 0.500 0.495 0.502 0.496 0.495 0.500 0.501 0.498

0.496 0.732 0.794 0.831 0.854 0.871 0.884 0.895 0.904 0.912 0.918 0.921 0.926 0.930 0.933 0.936 0.939 0.941 0.944 0.945 0.947 0.948 0.950 0.952 0.953 0.954 0.956 0.957 0.957 0.959

0.500 0.714 0.748 0.771 0.788 0.802 0.813 0.824 0.831 0.839 0.847 0.854 0.861 0.866 0.870 0.874 0.878 0.882 0.885 0.888 0.891 0.894 0.896 0.898 0.899 0.902 0.903 0.905 0.907 0.908

0.497 0.690 0.716 0.732 0.741 0.749 0.756 0.761 0.765 0.771 0.775 0.780 0.784 0.787 0.791 0.795 0.797 0.800 0.802 0.805 0.807 0.809 0.812 0.814 0.816 0.817 0.820 0.821 0.822 0.824

0.494 0.673 0.709 0.722 0.730 0.735 0.738 0.740 0.741 0.743 0.744 0.745 0.747 0.748 0.749 0.749 0.750 0.751 0.752 0.753 0.754 0.755 0.755 0.756 0.757 0.757 0.758 0.759 0.759 0.760

0.500 0.499 0.499 0.502 0.500 0.497 0.501 0.501 0.499 0.492 0.503 0.497 0.500 0.503 0.504 0.495 0.496 0.506 0.493 0.502 0.503 0.494 0.500 0.503 0.501 0.509 0.497 0.500 0.495 0.507

0.494 0.753 0.810 0.845 0.870 0.885 0.898 0.908 0.914 0.921 0.928 0.932 0.936 0.940 0.943 0.946 0.947 0.949 0.952 0.954 0.956 0.957 0.958 0.959 0.960 0.961 0.962 0.963 0.964 0.965

0.495 0.753 0.816 0.851 0.870 0.884 0.898 0.906 0.914 0.922 0.929 0.933 0.937 0.939 0.941 0.945 0.948 0.951 0.952 0.955 0.956 0.959 0.959 0.961 0.962 0.963 0.964 0.965 0.966 0.966

0.504 0.770 0.825 0.855 0.875 0.891 0.901 0.909 0.917 0.923 0.929 0.934 0.938 0.941 0.943 0.945 0.947 0.950 0.952 0.954 0.956 0.958 0.959 0.960 0.961 0.962 0.963 0.964 0.965 0.965

0.500 0.793 0.853 0.879 0.895 0.906 0.914 0.919 0.923 0.928 0.931 0.934 0.937 0.939 0.941 0.943 0.945 0.947 0.948 0.950 0.951 0.952 0.954 0.955 0.956 0.957 0.958 0.959 0.959 0.960

0.500 0.500 0.499 0.496 0.496 0.502 0.502 0.499 0.494 0.498 0.494 0.505 0.500 0.496 0.497 0.497 0.491 0.497 0.506 0.498 0.502 0.498 0.504 0.501 0.497 0.501 0.500 0.497 0.501 0.489

0.499 0.980 0.998 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.502 0.965 0.997 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.497 0.939 0.993 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.501 0.908 0.979 0.996 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.495 0.504 0.501 0.497 0.507 0.498 0.499 0.501 0.501 0.501 0.501 0.505 0.500 0.500 0.505 0.495 0.501 0.498 0.501 0.494 0.508 0.504 0.498 0.499 0.497 0.503 0.497 0.496 0.505 0.494

0.495 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.506 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.504 0.996 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.498 0.963 0.992 0.999 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.501 0.495 0.497 0.498 0.499 0.498 0.498 0.505 0.498 0.507 0.502 0.505 0.499 0.504 0.498 0.504 0.505 0.505 0.500 0.500 0.499 0.510 0.498 0.501 0.494 0.502 0.501 0.503 0.501 0.501

ARTICLE IN PRESS

0.00 0.00

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

d f

1589

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1590 1.1

Av. fraction playing strategy one

1.0

0.9     

0.8

0.7

=0 = 0.25 = 0.5 = 0.75 =1

0.6

0.5

0.4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B3. Average population share playing strategy one in the dominance solvable game in Fig. 2 for different values of f, holding d ¼ 1 fixed.

1.1

Av. fraction playing strategy one

1.0

0.9

0.8

0.7

    

=0 = 0.25 = 0.5 = 0.75 =1

0.6

0.5

0.4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B4. Average population share playing strategy one in the dominance solvable game in Fig. 2 for different values of d, holding f ¼ 0 fixed.

Table B3 Average population share playing strategy one in the coordination game in Fig. 3 0.00 0.25

0.00 0.50

0.00 0.75

0.00 1.00

0.25 0.00

0.25 0.25

0.25 0.50

0.25 0.75

0.25 1.00

0.50 0.00

0.50 0.25

0.50 0.50

0.50 0.75

0.50 1.00

0.75 0.00

0.75 0.25

0.75 0.50

0.75 0.75

0.75 1.00

1.00 0.00

1.00 0.25

1.00 0.50

1.00 0.75

1.00 1.00

t¼1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.504

0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510 0.510

0.501 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499

0.503 0.486 0.485 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484 0.484

0.496 0.491 0.506 0.496 0.501 0.503 0.500 0.496 0.505 0.503 0.494 0.498 0.493 0.500 0.500 0.509 0.498 0.502 0.503 0.501 0.500 0.502 0.497 0.505 0.504 0.500 0.502 0.506 0.502 0.492

0.501 0.483 0.463 0.444 0.425 0.407 0.392 0.375 0.358 0.341 0.324 0.309 0.294 0.280 0.264 0.250 0.237 0.223 0.212 0.201 0.190 0.179 0.167 0.156 0.147 0.140 0.130 0.122 0.114 0.106

0.504 0.470 0.459 0.452 0.447 0.442 0.436 0.431 0.426 0.420 0.414 0.410 0.404 0.398 0.392 0.386 0.381 0.375 0.369 0.363 0.357 0.351 0.345 0.340 0.334 0.328 0.323 0.317 0.312 0.306

0.503 0.451 0.436 0.429 0.426 0.423 0.421 0.418 0.416 0.414 0.411 0.409 0.407 0.405 0.403 0.401 0.398 0.396 0.394 0.391 0.389 0.387 0.384 0.382 0.380 0.378 0.375 0.372 0.370 0.368

0.500 0.443 0.418 0.410 0.405 0.403 0.401 0.400 0.399 0.399 0.398 0.398 0.397 0.396 0.396 0.395 0.395 0.394 0.393 0.393 0.392 0.392 0.392 0.391 0.391 0.390 0.389 0.389 0.388 0.387

0.498 0.493 0.505 0.500 0.502 0.504 0.504 0.501 0.505 0.505 0.503 0.504 0.500 0.502 0.501 0.499 0.503 0.498 0.508 0.498 0.499 0.498 0.499 0.498 0.494 0.498 0.502 0.501 0.500 0.504

0.503 0.504 0.504 0.504 0.504 0.503 0.503 0.502 0.503 0.503 0.503 0.503 0.504 0.504 0.505 0.505 0.505 0.506 0.505 0.505 0.506 0.506 0.506 0.506 0.505 0.505 0.504 0.505 0.505 0.505

0.504 0.504 0.532 0.543 0.567 0.588 0.608 0.626 0.640 0.655 0.666 0.671 0.676 0.680 0.681 0.683 0.685 0.685 0.686 0.687 0.688 0.688 0.688 0.688 0.688 0.688 0.687 0.688 0.688 0.688

0.504 0.504 0.508 0.532 0.538 0.556 0.570 0.582 0.595 0.606 0.612 0.620 0.625 0.631 0.636 0.637 0.640 0.641 0.643 0.644 0.646 0.647 0.648 0.648 0.649 0.650 0.650 0.650 0.651 0.651

0.501 0.504 0.507 0.519 0.527 0.533 0.542 0.552 0.558 0.565 0.574 0.580 0.585 0.592 0.597 0.600 0.605 0.607 0.609 0.614 0.616 0.617 0.619 0.618 0.619 0.621 0.621 0.622 0.622 0.623

0.503 0.498 0.502 0.502 0.499 0.496 0.499 0.502 0.493 0.494 0.500 0.500 0.498 0.500 0.505 0.510 0.501 0.503 0.495 0.498 0.505 0.501 0.496 0.504 0.502 0.508 0.502 0.500 0.499 0.495

0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501 0.501

0.501 0.501 0.507 0.527 0.549 0.573 0.597 0.621 0.647 0.673 0.697 0.722 0.746 0.769 0.792 0.813 0.833 0.849 0.867 0.881 0.895 0.907 0.918 0.928 0.937 0.945 0.951 0.958 0.962 0.966

0.495 0.495 0.616 0.663 0.731 0.791 0.840 0.876 0.899 0.916 0.927 0.934 0.939 0.944 0.946 0.949 0.950 0.951 0.952 0.952 0.953 0.953 0.954 0.954 0.955 0.955 0.955 0.955 0.955 0.955

0.505 0.512 0.626 0.612 0.722 0.740 0.794 0.826 0.852 0.871 0.882 0.892 0.899 0.903 0.907 0.909 0.911 0.912 0.913 0.914 0.915 0.915 0.915 0.915 0.915 0.915 0.915 0.915 0.916 0.916

0.495 0.497 0.505 0.501 0.497 0.498 0.500 0.504 0.497 0.496 0.500 0.498 0.498 0.497 0.500 0.498 0.498 0.494 0.499 0.506 0.504 0.496 0.498 0.498 0.500 0.499 0.501 0.504 0.496 0.503

0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495

0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.501 0.501 0.501 0.501 0.501 0.501

0.497 0.497 0.515 0.603 0.673 0.741 0.807 0.862 0.908 0.939 0.960 0.973 0.981 0.987 0.990 0.992 0.994 0.995 0.996 0.997 0.997 0.998 0.998 0.998 0.998 0.998 0.999 0.999 0.999 0.999

0.498 0.502 0.656 0.703 0.781 0.849 0.902 0.937 0.958 0.972 0.980 0.984 0.987 0.989 0.990 0.991 0.991 0.991 0.991 0.991 0.991 0.991 0.992 0.992 0.992 0.992 0.992 0.992 0.992 0.992

0.500 0.498 0.504 0.500 0.500 0.504 0.495 0.503 0.500 0.502 0.502 0.500 0.504 0.498 0.504 0.503 0.500 0.503 0.497 0.501 0.499 0.493 0.498 0.501 0.496 0.495 0.501 0.501 0.497 0.500

ARTICLE IN PRESS

0.00 0.00

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

d f

1591

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1592 1.1

Av. fraction playing strategy one

1.0

0.9     

0.8

0.7

=0 = 0.25 = 0.5 = 0.75 =1

0.6

0.5

0.4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B5. Average population share playing strategy one in the coordination game in Fig. 3 for different values of f, holding d ¼ 1 fixed. 1.1

1.0

Av. fraction playing strategy one

0.9

    

=0 = 0.25 = 0.5 = 0.75 =1

0.8

0.7

0.6

0.5

0.4

0.3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B6. Average population share playing strategy one in the coordination game in Fig. 3 for different values of d, holding f ¼ 0:75 fixed.

Table B4 Average population share playing strategy one in the coordination game in Fig. 4 0.00 0.25

0.00 0.50

0.00 0.75

0.00 1.00

0.25 0.00

0.25 0.25

0.25 0.50

0.25 0.75

0.25 1.00

0.50 0.00

0.50 0.25

0.50 0.50

0.50 0.75

0.50 1.00

0.75 0.00

0.75 0.25

0.75 0.50

0.75 0.75

0.75 1.00

1.00 0.00

1.00 0.25

1.00 0.50

1.00 0.75

1.00 1.00

t¼1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497 0.497

0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500

0.499 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498 0.498

0.502 0.484 0.482 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481 0.481

0.495 0.503 0.496 0.496 0.493 0.501 0.500 0.501 0.494 0.499 0.500 0.500 0.498 0.506 0.501 0.498 0.504 0.499 0.503 0.500 0.493 0.494 0.500 0.495 0.502 0.496 0.495 0.500 0.501 0.498

0.496 0.478 0.460 0.442 0.422 0.404 0.387 0.368 0.351 0.333 0.318 0.303 0.287 0.274 0.258 0.244 0.232 0.218 0.205 0.193 0.182 0.172 0.161 0.151 0.142 0.133 0.126 0.118 0.111 0.105

0.500 0.467 0.455 0.449 0.444 0.439 0.433 0.426 0.420 0.414 0.407 0.402 0.395 0.390 0.384 0.378 0.372 0.365 0.359 0.353 0.346 0.340 0.333 0.328 0.321 0.313 0.307 0.302 0.295 0.290

0.497 0.444 0.428 0.422 0.418 0.415 0.411 0.408 0.406 0.403 0.400 0.397 0.394 0.391 0.388 0.386 0.383 0.380 0.378 0.375 0.372 0.369 0.365 0.362 0.359 0.356 0.354 0.350 0.347 0.344

0.494 0.428 0.400 0.390 0.385 0.383 0.381 0.380 0.379 0.378 0.377 0.376 0.375 0.374 0.374 0.373 0.372 0.371 0.370 0.369 0.369 0.368 0.367 0.366 0.365 0.365 0.363 0.363 0.362 0.361

0.500 0.499 0.499 0.502 0.500 0.497 0.501 0.501 0.499 0.492 0.503 0.497 0.500 0.503 0.504 0.495 0.496 0.506 0.493 0.502 0.503 0.494 0.500 0.503 0.501 0.509 0.497 0.500 0.495 0.507

0.494 0.374 0.260 0.171 0.104 0.060 0.034 0.018 0.010 0.005 0.002 0.001 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

0.495 0.373 0.319 0.235 0.170 0.119 0.079 0.058 0.041 0.031 0.024 0.019 0.017 0.015 0.014 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013

0.504 0.398 0.359 0.308 0.262 0.222 0.185 0.155 0.129 0.107 0.092 0.078 0.069 0.060 0.051 0.045 0.041 0.039 0.036 0.034 0.034 0.032 0.032 0.031 0.030 0.030 0.029 0.029 0.029 0.028

0.500 0.428 0.393 0.371 0.346 0.327 0.308 0.291 0.274 0.260 0.247 0.236 0.225 0.214 0.206 0.197 0.188 0.180 0.172 0.167 0.162 0.158 0.155 0.151 0.148 0.146 0.143 0.141 0.139 0.136

0.500 0.500 0.499 0.496 0.496 0.502 0.502 0.499 0.494 0.498 0.494 0.505 0.500 0.496 0.497 0.497 0.491 0.497 0.506 0.498 0.502 0.498 0.504 0.501 0.497 0.501 0.500 0.497 0.501 0.489

0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499

0.502 0.502 0.497 0.489 0.484 0.476 0.468 0.461 0.453 0.445 0.435 0.427 0.417 0.408 0.400 0.390 0.381 0.373 0.365 0.356 0.347 0.338 0.329 0.321 0.313 0.305 0.297 0.288 0.280 0.271

0.497 0.495 0.462 0.465 0.439 0.426 0.409 0.396 0.385 0.374 0.372 0.365 0.361 0.360 0.359 0.358 0.358 0.358 0.358 0.358 0.358 0.357 0.357 0.356 0.356 0.356 0.356 0.356 0.356 0.356

0.501 0.494 0.469 0.469 0.444 0.436 0.419 0.413 0.401 0.395 0.386 0.383 0.380 0.380 0.377 0.377 0.377 0.377 0.376 0.376 0.375 0.375 0.374 0.374 0.373 0.373 0.373 0.373 0.373 0.373

0.495 0.504 0.501 0.497 0.507 0.498 0.499 0.501 0.501 0.501 0.501 0.505 0.500 0.500 0.505 0.495 0.501 0.498 0.501 0.494 0.508 0.504 0.498 0.499 0.497 0.503 0.497 0.496 0.505 0.494

0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495 0.495

0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.505 0.505 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506 0.506

0.504 0.504 0.504 0.502 0.502 0.502 0.503 0.500 0.500 0.499 0.500 0.500 0.503 0.503 0.501 0.502 0.503 0.500 0.502 0.502 0.502 0.502 0.502 0.502 0.503 0.504 0.504 0.504 0.505 0.504

0.498 0.498 0.500 0.498 0.503 0.501 0.504 0.504 0.505 0.507 0.508 0.511 0.514 0.516 0.517 0.518 0.519 0.519 0.520 0.520 0.520 0.520 0.520 0.520 0.520 0.520 0.520 0.520 0.519 0.519

0.501 0.495 0.497 0.498 0.499 0.498 0.498 0.505 0.498 0.507 0.502 0.505 0.499 0.504 0.498 0.504 0.505 0.505 0.500 0.500 0.499 0.510 0.498 0.501 0.494 0.502 0.501 0.503 0.501 0.501

ARTICLE IN PRESS

0.00 0.00

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

d f

1593

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1594 0.6

Av. fraction playing strategy one

0.5

    

0.4

0.3

=0 = 0.25 = 0.5 = 0.75 =1

0.2

0.1

0.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B7. Average population share playing strategy one in the coordination game in Fig. 4 for different values of f, holding d ¼ 0:5 fixed. 0.6

Av. fraction playing strategy one

0.5

0.4

0.3

    

=0 = 0.25 = 0.5 = 0.75 =1

0.2

0.1

0.0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B8. Average population share playing strategy one in the coordination game in Fig. 4 for different values of d, holding f ¼ 0 fixed.

Table B5 Average population share playing strategy one in the hawk–dove game in Fig. 5 0.00 0.25

0.00 0.50

0.00 0.75

0.00 1.00

0.25 0.00

0.25 0.25

0.25 0.50

0.25 0.75

0.25 1.00

0.50 0.00

0.50 0.25

0.50 0.50

0.50 0.75

0.50 1.00

0.75 0.00

0.75 0.25

0.75 0.50

0.75 0.75

0.75 1.00

1.00 0.00

1.00 0.25

1.00 0.50

1.00 0.75

1.00 1.00

t¼1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500

0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499 0.499

0.502 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500

0.494 0.479 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477 0.477

0.496 0.490 0.501 0.498 0.502 0.496 0.501 0.499 0.493 0.505 0.506 0.502 0.502 0.499 0.502 0.504 0.501 0.497 0.506 0.501 0.499 0.502 0.490 0.499 0.500 0.497 0.499 0.504 0.500 0.498

0.500 0.482 0.466 0.450 0.436 0.421 0.408 0.396 0.384 0.372 0.363 0.354 0.345 0.338 0.329 0.321 0.313 0.306 0.299 0.293 0.287 0.282 0.276 0.271 0.266 0.262 0.258 0.254 0.250 0.247

0.496 0.466 0.456 0.452 0.449 0.445 0.442 0.440 0.437 0.434 0.432 0.429 0.427 0.425 0.423 0.421 0.420 0.418 0.416 0.413 0.411 0.410 0.408 0.407 0.405 0.403 0.402 0.400 0.398 0.397

0.495 0.446 0.435 0.433 0.431 0.430 0.429 0.429 0.428 0.428 0.427 0.427 0.426 0.426 0.425 0.425 0.425 0.424 0.424 0.424 0.423 0.423 0.423 0.422 0.422 0.422 0.421 0.421 0.421 0.420

0.500 0.451 0.437 0.432 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.430 0.429 0.429 0.429 0.429 0.429 0.429

0.494 0.504 0.502 0.509 0.497 0.500 0.499 0.497 0.510 0.496 0.492 0.500 0.496 0.506 0.502 0.503 0.500 0.499 0.492 0.497 0.497 0.497 0.505 0.501 0.501 0.502 0.509 0.497 0.500 0.500

0.495 0.504 0.496 0.504 0.496 0.503 0.497 0.503 0.497 0.503 0.497 0.502 0.498 0.502 0.499 0.501 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.499 0.501 0.499 0.501 0.499

0.504 0.496 0.534 0.503 0.526 0.507 0.517 0.513 0.513 0.515 0.516 0.513 0.514 0.516 0.515 0.516 0.513 0.515 0.517 0.513 0.517 0.514 0.519 0.514 0.514 0.518 0.510 0.518 0.515 0.516

0.500 0.500 0.502 0.525 0.518 0.523 0.523 0.524 0.522 0.521 0.522 0.521 0.525 0.525 0.525 0.522 0.521 0.522 0.522 0.526 0.523 0.525 0.524 0.520 0.520 0.521 0.522 0.523 0.521 0.524

0.500 0.498 0.504 0.511 0.516 0.516 0.522 0.521 0.521 0.523 0.523 0.526 0.525 0.527 0.527 0.527 0.528 0.528 0.529 0.528 0.528 0.528 0.532 0.529 0.530 0.530 0.531 0.530 0.531 0.531

0.499 0.499 0.498 0.502 0.499 0.499 0.501 0.498 0.500 0.498 0.499 0.499 0.501 0.501 0.499 0.500 0.502 0.503 0.497 0.500 0.498 0.498 0.501 0.498 0.500 0.490 0.496 0.504 0.496 0.506

0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499 0.501 0.499

0.497 0.503 0.505 0.516 0.510 0.515 0.511 0.515 0.509 0.515 0.509 0.517 0.509 0.516 0.509 0.516 0.509 0.517 0.510 0.515 0.510 0.517 0.509 0.513 0.512 0.514 0.511 0.514 0.512 0.514

0.501 0.499 0.624 0.506 0.559 0.536 0.551 0.545 0.548 0.542 0.546 0.546 0.542 0.550 0.545 0.546 0.545 0.546 0.544 0.547 0.547 0.548 0.546 0.543 0.547 0.544 0.548 0.546 0.545 0.548

0.495 0.512 0.609 0.493 0.596 0.533 0.564 0.558 0.558 0.565 0.557 0.564 0.562 0.560 0.562 0.559 0.563 0.558 0.562 0.557 0.561 0.562 0.564 0.558 0.558 0.563 0.557 0.564 0.561 0.560

0.495 0.496 0.496 0.506 0.502 0.496 0.500 0.496 0.498 0.491 0.501 0.502 0.498 0.495 0.511 0.494 0.496 0.500 0.501 0.500 0.491 0.494 0.500 0.502 0.497 0.497 0.502 0.501 0.502 0.499

0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495 0.505 0.495

0.504 0.496 0.504 0.496 0.504 0.496 0.504 0.497 0.504 0.497 0.504 0.497 0.504 0.497 0.503 0.497 0.503 0.497 0.503 0.497 0.503 0.497 0.503 0.497 0.503 0.497 0.503 0.497 0.503 0.497

0.498 0.502 0.519 0.577 0.528 0.556 0.532 0.553 0.533 0.549 0.538 0.548 0.537 0.549 0.537 0.549 0.538 0.545 0.541 0.542 0.547 0.538 0.544 0.543 0.543 0.541 0.544 0.540 0.549 0.535

0.501 0.502 0.650 0.499 0.568 0.561 0.562 0.560 0.563 0.563 0.559 0.561 0.562 0.560 0.559 0.562 0.567 0.559 0.558 0.565 0.557 0.567 0.557 0.561 0.564 0.561 0.558 0.564 0.560 0.560

0.505 0.501 0.502 0.501 0.503 0.499 0.494 0.504 0.497 0.491 0.498 0.497 0.499 0.496 0.490 0.496 0.504 0.501 0.498 0.499 0.499 0.499 0.502 0.497 0.501 0.503 0.503 0.505 0.501 0.503

ARTICLE IN PRESS

0.00 0.00

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

d f

1595

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1596

0.65     

Av. fraction playing strategy one

0.60

=0 = 0.25 = 0.5 = 0.75 =1

0.55

0.50

0.45

0.40 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B9. Average population share playing strategy one in the hawk–dove game in Fig. 5 for different values of f, holding d ¼ 0:75 fixed.

0.70

Av. fraction playing strategy one

0.65

    

=0 = 0.25 = 0.5 = 0.75 =1

0.60

0.55

0.50

0.45

0.40 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B10. Average population share playing strategy one in the hawk–dove game in Fig. 5 for different values of d, holding f ¼ 0:75 fixed.

Table B6 Average population share playing strategy one in the hawk–dove game in Fig. 6 0.00 0.25

0.00 0.50

0.00 0.75

0.00 1.00

0.25 0.00

0.25 0.25

0.25 0.50

0.25 0.75

0.25 1.00

0.50 0.00

0.50 0.25

0.50 0.50

0.50 0.75

0.50 1.00

0.75 0.00

0.75 0.25

0.75 0.50

0.75 0.75

0.75 1.00

1.00 0.00

1.00 0.25

1.00 0.50

1.00 0.75

1.00 1.00

t¼1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502 0.502

0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500

0.507 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505 0.505

0.496 0.480 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478 0.478

0.493 0.497 0.499 0.506 0.499 0.496 0.497 0.505 0.499 0.501 0.498 0.504 0.497 0.498 0.500 0.499 0.499 0.501 0.503 0.496 0.505 0.492 0.497 0.498 0.496 0.506 0.497 0.500 0.501 0.494

0.508 0.380 0.308 0.262 0.232 0.206 0.187 0.172 0.160 0.149 0.141 0.131 0.125 0.117 0.111 0.106 0.102 0.098 0.094 0.091 0.088 0.085 0.082 0.080 0.078 0.076 0.075 0.073 0.072 0.070

0.499 0.375 0.350 0.337 0.327 0.319 0.311 0.304 0.297 0.291 0.285 0.279 0.274 0.269 0.266 0.262 0.257 0.254 0.251 0.247 0.244 0.241 0.238 0.236 0.234 0.232 0.229 0.227 0.225 0.223

0.498 0.371 0.346 0.341 0.338 0.335 0.333 0.331 0.329 0.328 0.326 0.324 0.323 0.321 0.320 0.318 0.317 0.316 0.315 0.313 0.312 0.311 0.310 0.309 0.308 0.306 0.306 0.305 0.304 0.303

0.502 0.384 0.358 0.353 0.351 0.350 0.349 0.349 0.348 0.348 0.348 0.348 0.347 0.347 0.347 0.347 0.347 0.347 0.347 0.346 0.346 0.346 0.346 0.346 0.346 0.346 0.346 0.346 0.346 0.346

0.502 0.503 0.506 0.499 0.497 0.499 0.500 0.501 0.497 0.502 0.498 0.497 0.497 0.494 0.497 0.501 0.497 0.500 0.505 0.501 0.499 0.497 0.501 0.501 0.496 0.506 0.496 0.506 0.498 0.503

0.501 0.252 0.196 0.167 0.146 0.134 0.124 0.115 0.110 0.107 0.102 0.098 0.096 0.094 0.093 0.091 0.089 0.089 0.088 0.087 0.086 0.084 0.085 0.084 0.084 0.084 0.083 0.084 0.085 0.085

0.502 0.253 0.196 0.164 0.145 0.131 0.120 0.113 0.108 0.102 0.097 0.096 0.094 0.092 0.090 0.088 0.086 0.084 0.083 0.081 0.082 0.081 0.082 0.082 0.082 0.082 0.081 0.080 0.081 0.081

0.501 0.266 0.235 0.225 0.215 0.208 0.200 0.195 0.190 0.186 0.183 0.179 0.176 0.173 0.170 0.167 0.166 0.165 0.163 0.160 0.158 0.157 0.156 0.155 0.154 0.153 0.152 0.151 0.149 0.149

0.506 0.327 0.302 0.288 0.278 0.272 0.270 0.268 0.264 0.262 0.259 0.257 0.255 0.253 0.252 0.250 0.248 0.247 0.245 0.244 0.242 0.241 0.241 0.239 0.239 0.238 0.236 0.235 0.234 0.234

0.495 0.494 0.498 0.497 0.499 0.494 0.500 0.500 0.495 0.501 0.493 0.497 0.502 0.497 0.501 0.500 0.499 0.499 0.500 0.500 0.502 0.498 0.494 0.501 0.506 0.498 0.504 0.495 0.497 0.499

0.505 0.477 0.501 0.479 0.501 0.481 0.500 0.481 0.501 0.481 0.498 0.484 0.496 0.486 0.495 0.487 0.493 0.488 0.494 0.487 0.493 0.488 0.493 0.488 0.492 0.489 0.493 0.489 0.490 0.492

0.498 0.467 0.284 0.374 0.412 0.356 0.370 0.386 0.370 0.371 0.376 0.376 0.373 0.367 0.376 0.371 0.377 0.372 0.374 0.377 0.371 0.372 0.370 0.378 0.374 0.374 0.373 0.373 0.374 0.372

0.501 0.443 0.276 0.265 0.336 0.339 0.322 0.317 0.322 0.322 0.322 0.323 0.320 0.321 0.321 0.320 0.322 0.320 0.316 0.322 0.321 0.322 0.320 0.323 0.322 0.325 0.322 0.320 0.327 0.319

0.507 0.402 0.302 0.285 0.316 0.323 0.308 0.300 0.301 0.306 0.303 0.300 0.300 0.303 0.301 0.299 0.300 0.305 0.303 0.301 0.302 0.298 0.299 0.300 0.297 0.301 0.303 0.300 0.302 0.297

0.506 0.505 0.496 0.505 0.495 0.508 0.501 0.500 0.504 0.498 0.497 0.507 0.500 0.503 0.496 0.496 0.499 0.502 0.506 0.500 0.496 0.508 0.499 0.498 0.500 0.501 0.503 0.499 0.501 0.499

0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.494 0.506 0.495 0.506 0.495 0.505 0.495 0.506 0.494

0.501 0.499 0.468 0.471 0.467 0.469 0.473 0.465 0.471 0.467 0.471 0.469 0.466 0.473 0.467 0.472 0.467 0.473 0.466 0.473 0.468 0.471 0.466 0.472 0.469 0.472 0.465 0.475 0.466 0.473

0.501 0.496 0.266 0.411 0.433 0.359 0.393 0.405 0.381 0.390 0.397 0.388 0.394 0.389 0.390 0.392 0.391 0.389 0.394 0.392 0.385 0.392 0.392 0.389 0.395 0.385 0.396 0.389 0.386 0.393

0.505 0.458 0.277 0.336 0.366 0.341 0.337 0.345 0.348 0.342 0.338 0.342 0.348 0.340 0.347 0.341 0.341 0.343 0.343 0.345 0.345 0.343 0.338 0.348 0.338 0.343 0.343 0.348 0.346 0.342

0.503 0.502 0.502 0.502 0.503 0.502 0.500 0.502 0.502 0.497 0.497 0.502 0.500 0.504 0.499 0.498 0.502 0.499 0.500 0.502 0.502 0.497 0.501 0.496 0.501 0.496 0.502 0.500 0.503 0.499

ARTICLE IN PRESS

0.00 0.00

J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

d f

1597

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1598 0.6

Av. fraction playing strategy one

0.5

    

0.4

0.3

=0 = 0.25 = 0.5 = 0.75 =1

0.2

0.1

0.0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B11. Average population share playing strategy one in the hawk–dove game in Fig. 6 for different values of f, holding d ¼ 0:5 fixed.

0.55

Av. fraction playing strategy one

0.50

    

0.45

0.40

=0 = 0.25 = 0.5 = 0.75 =1

0.35

0.30

0.25

0.20 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Period

Fig. B12. Average population share playing strategy one in the hawk–dove game in Fig. 6 for different values of d, holding f ¼ 0:75 fixed.

ARTICLE IN PRESS J. Josephson / Journal of Economic Dynamics & Control 32 (2008) 1569–1599

1599

References Anderlini, L., Sabourian, H., 1995. The evolution of algorithmic learning rules: a global stability result. Mimeo, Cambridge University. Beggs, A.W., 2005. On the convergence of reinforcement learning. Journal of Economic Theory 122, 1– 36. Berger, U., 2005. Fictitious play in 2  n games. Journal of Economic Theory 120, 139– 154. Brown, G.W., 1951. Iterative solutions of games by fictitious play. In: Koopmans, T.C. (Ed.), Activity Analysis of Production and Allocation. Wiley, New York. Bush, R., Mosteller, F., 1951. A mathematical model for simple learning. Psychological Review 58, 313– 323. Camerer, C., 2003. Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University Press, Princeton, NJ. Camerer, C., Ho, T.-H., 1999. Experience-weighted attraction learning in normal form games. Econometrica 67, 827– 874. Camerer, C., Ho, T.-H., Chong, J.-K., 2002. Sophisticated experience-weighted attraction learning and strategic teaching in repeated games. Journal of Economic Theory 104, 137– 188. Erev, I., Roth, A., 1998. Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. American Economic Review 88, 848– 881. Fudenberg, D., Levine, D., 1998. The Theory of Learning in Games. The MIT Press, Cambridge, MA. Hanaki, N., Sethi, R., Erev, I., Peterhansl, A., 2005. Learning strategies. Journal of Economic Behavior and Organization 56, 523– 542. Harley, C.B., 1981. Learning the evolutionarily stable strategy. Journal of Theoretical Biology 89, 611– 633. Heller, D., 2004. An evolutionary approach to learning in a changing environment. Journal of Economic Theory 114, 31– 55. Ho, T.-H., Camerer, C., Chong, J.-K., 2002. Functional EWA: a one-parameter theory of learning in games. Mimeo, Caltech. Hofbauer, J., Sandholm, W.H., 2002. On the global convergence of stochastic fictitious play. Econometrica 70, 2265– 2294. Hommes, C., 2006. Heterogeneous agent models in economics and finance. In: Tesfatsion, L., Judd, K.L. (Eds.), Handbook of Computational Economics, volume 2. Agent-Based Computational Economics. Elsevier, Amsterdam, pp. 1109– 1186. Hopkins, E., 1999. Learning, matching, and aggregation. Games and Economic Behavior 26, 79– 110. Hopkins, E., 2002. Two competing models of how people learn in games. Econometrica 70, 2141– 2166. Hopkins, E., Posch, M., 2005. Attainability of boundary points under reinforcement learning. Games and Economic Behavior 53, 110– 125. Maynard Smith, J., 1974. The theory of games and the evolution of animal conflicts. Journal of Theoretical Biology 47, 209– 221. Maynard Smith, J., Price, G.R., 1973. The logic of animal conflict. Nature 246, 15– 18. Robson, A., 2001. The biological basis of economic behavior. Journal of Economic Literature 39, 11– 33. Salmon, T.C., 2001. An evaluation of econometric models of adaptive learning. Econometrica 69, 1597– 1628. Stahl, D.O., 2003. Action-reinforcement learning versus rule learning. Mimeo, University of Texas. Taylor, P.D., Jonker, L.B., 1978. Evolutionary stable strategies and game dynamics. Mathematical Biosciences 40, 145– 156. Thorndike, E.L., 1898. Animal intelligence: an experimental study of the associative process in animals. Psychological Monographs 2. Wilcox, N.T., 2006. Theories of learning in games and heterogeneity bias. Econometrica 74, 1271– 1292.