Evolutionary dynamics of fairness on graphs with migration

Journal of Theoretical Biology 380 (2015) 103–114

Contents lists available at ScienceDirect

Journal of Theoretical Biology journal homepage: www.elsevier.com/locate/yjtbi

Evolutionary dynamics of fairness on graphs with migration Xiaofeng Wang a,n, Xiaojie Chen b, Long Wang c a b c

Center for Complex Systems, Xidian University, Xi'an 710071, China School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China Center for Systems and Control, College of Engineering, Peking University, Beijing 100871, China

H I G H L I G H T S








We revisit the effect of node degree on the evolution of fairness. We also study the impact of dilution and migration in the networked ultimatum game. Players can stochastically leave current sites and move into other vacant sites. Both dilution and migration lead to the drop of offer level and the rise of acceptance level. We develop an analytical method based on evolutionary graph theory.

art ic l e i nf o

a b s t r a c t

Article history: Received 1 February 2015 Received in revised form 11 May 2015 Accepted 13 May 2015 Available online 22 May 2015

Individual migration plays a crucial role in evolutionary dynamics of population on networks. In this paper, we generalize the networked ultimatum game by diluting population structures as well as endowing individuals with migration ability, and investigate evolutionary dynamics of fairness on graphs with migration in the ultimatum game. We first revisit the impact of node degree on the evolution of fairness. Interestingly, numerical simulations reveal that there exists an optimal value of node degree resulting in the maximal offer level of populations. Then we explore the effects of dilution and migration on the evolution of fairness, and find that both the dilution of population structures and the endowment of migration ability to individuals would lead to the drop of offer level, while the rise of acceptance level of populations. Notably, natural selection even favors the evolution of self-incompatible strategies, when either vacancy rate or migration rate exceeds a critical threshold. To confirm our simulation results, we also propose an analytical method to study the evolutionary dynamics of fairness on graphs with migration. This method can be applied to explore any games governed by pairwise interactions in finite populations. & 2015 Elsevier Ltd. All rights reserved.

Keywords: Evolutionary game theory Population structure Ultimatum game

1. Introduction The evolution of fairness in populations consisting of selfish individuals is particularly puzzling for social dilemmas related to sharing natural resources or common goods, and has attracted scientists from different fields, ranging from social and biological sciences to theoretical physics (Nowak et al., 2000; Szabó and Fáth, 2007; Rand et al., 2013). Suppose that two individuals have to share a sum of money. One acting as a proposer offers a split of the money, and the other one playing as a responder can either accept it or not. If an agreement is reached, the sum is allocated according to the proposal. Otherwise, neither individual obtains anything.

n

Corresponding author. E-mail address: [email protected] (L. Wang).

http://dx.doi.org/10.1016/j.jtbi.2015.05.020 0022-5193/& 2015 Elsevier Ltd. All rights reserved.

This is the rule of the famous ultimatum game firstly proposed by Güth et al. (1982). In a one-shot anonymous ultimatum game, rational responders would accept the smallest offer, or else he will be empty handed. Self-interested proposers, therefore, should claim the largest share. The subgame perfect equilibrium prediction (Selten, 1975) resulted from above backward induction has been evaluated through many behavioral experiments using the ultimatum game. Empirical results have revealed that (i) many responders reject low offers and (ii) many proposers offer more than the minimum amount required to avoid rejection (Güth et al., 1982; Camerer, 2003). The disagreement between theoretical predictions and experimental results has motivated scientists from different disciplines to explore the possible explanations for the evolution of fairness in the ultimatum game. Some game theorists demonstrate that many people are not purely self-interested, but exhibit the so-called ‘other-regarding

104

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

preferences’ (Fehr and Schmidt, 1999; Camerer, 2003). In the framework of evolutionary game theory, theoretical studies indicate that reputation (Nowak et al., 2000), empathy (Page and Nowak, 2002; Sánchez and Cuesta, 2005), randomness (Rand et al., 2013; Wang et al., 2014), population structure (Page et al., 2000; Killingback and Studer, 2001; Sinatra et al., 2009; Li and Cao, 2009; Duan and Stanley, 2010; Gao et al., 2011; Iranzo et al., 2011; Szolnoki et al., 2012b, 2012a; Wu et al., 2013) and heterogeneity (da Silva and Kellerman, 2007) play a vital role in the evolution of fairness in the ultimatum game. In particular, Nowak et al. (2000) have shown that natural selection favors the rational solution in well-mixed populations, while much fairer evolutionary outcomes can be resulted on spatial networks (Page et al., 2000). Following the line of this pioneering work, the evolutionary ultimatum game on networks has aroused great concerns from researchers (Killingback and Studer, 2001; Sinatra et al., 2009; Li and Cao, 2009; Duan and Stanley, 2010; Gao et al., 2011; Iranzo et al., 2011; Szolnoki et al., 2012b, 2012a; Wu et al., 2013). Although the differences in their model setups, all of these studies have confirmed that network structure promotes the evolution of fairness in the ultimatum game. Iranzo et al. (2011) have revisited the spatial ultimatum game in detail, and found that the quasiempathetic individuals, whose offer levels are very close to their acceptance thresholds, can spontaneously emerge and fixate in the structured population. Gao et al. (2011) have explored how fairness evolves if individuals are able to adjust both strategies and social ties. They found that the coevolution of strategies and networks leads to the establishment of non-rational large offer and acceptance level. Besides, Sinatra et al. (2009) have studied the evolutionary ultimatum game on complex networks. They show again that fairness appears in the heterogeneous structured populations. On the other hand, recent experimental studies have shown the important relationships between human mobility and population dynamics, such as epidemic spreading (Brockmann et al., 2006) and social communication patterns (Gonzalez et al., 2008). Motivated by these facts, we extend the investigations of networked ultimatum game by diluting the population structures and endowing individuals with migration ability. Recently, a few works have considered this kind of situation. However, these studies are concentrated in the evolutionary puzzle of cooperation (Vainstein et al., 2007; Sicardi et al., 2009; Meloni et al., 2009; Helbing and Yu, 2009; Yang et al., 2010; Jiang et al., 2010; Yu, 2011; Roca and Helbing, 2011; Wu et al., 2012; Cong et al., 2012; Chen et al., 2012; Fu and Nowak, 2013). In addition, the insights from evolutionary games on graphs with migration are gained mainly through numerical simulations. In this paper, we investigate the effects of dilution and migration on the evolution of fairness in the ultimatum game by both numerical simulations and theoretical analysis. Specifically, we first revisit the role of node degree in the evolution of fairness in the ultimatum game, and find that there exists an optimal value of node degree leading to the maximal offer level of populations. Then we explore the effects of dilution and migration on the evolution of fairness in the ultimatum game. Simulation results show that both dilution and migration would elicit the decline of offer level, but the rise of acceptance level of populations. To corroborate our numerical results, we further propose an analytical method to study the evolutionary dynamics of fairness on graphs with migration in the ultimatum game. This paper is organized as follows. Section 2 describes our model in detail. In Section 3 we study evolutionary dynamics of fairness on graph with migration in the ultimatum game, including the revisit of the role of node degree and the exploration of the effects of dilution and migration on the evolution of fairness. Analytical results are also presented in this section to validate our simulation treatment. Discussion and conclusion are provided in Section 4.

2. Model In the ultimatum game, the strategy of each individual can be given by a vector ½o; aT , where the component o is the amount offered when acting as a proposer, or the ‘offer level’, and a is the minimum amount demanded when acting as a responder, or the ‘rejection threshold’. The sum to be divided in each ultimatum game is set to unity. Based on rational self-interest, the two components o and a of each individual's strategy vector are constrained within the interval ð0; 0:5Þ. In our model, the continuous strategies (including the offer level o and the acceptance threshold a of individuals) in the ultimatum game are discretized in increments of 1=½2ðK  1Þ. Therefore, the strategy set Ω is defined as





Ω ¼ ½o; aT jo; aA ξ;

  1 1 K 2 1 ; ; …; ; 0:5  ξ ; K Z 2; 0 o ξ⪡ ; 2ðK  1Þ K  1 2ðK  1Þ 2ðK  1Þ

ð1Þ where ξ denotes the minimum offer that a responder would accept. There are overall K  K kinds of strategies in the discrete strategy space. For the convenience of description, hereinafter, we call a player who uses the mth strategy ½om ; am  A Ω (the nth strategy ½on ; an  A Ω) an m-player (an n-player) (for detailed definition see Fig. 1). Individuals play the ultimatum game once in the proposer role and once in the responder role. Then the payoff matrix of this 2  2 symmetric ultimatum game between an m-player and an n-player is given by "

amn cmn

bmn dmn " ¼

#

Hðom  am Þ om Hðom an Þþ ð1  on ÞHðon am Þ

# ð1 om ÞHðom  an Þ þon Hðon am Þ ; Hðon  an Þ

ð2Þ where the Heaviside step function H ðt Þ satisfies ( 1; if t≥0 H ðt Þ ¼ : 0; if t o0

ð3Þ

The population structure is defined by a graph, where each vertex represents a site and can be either occupied by an individual or just empty, and each edge denotes a possible

Fig. 1. Schematic presentation of K  K discrete strategies ½oiK þ j ; aiK þ j  with the integer i A ½0; K  1 and j A ½1; K  spreading across the continuous strategy space. Each strategy is labelled by a sequence number i  K þ j, where i is its line number and j is its column number. The line number is counted in order from bottom to top, and the column number is counted in order from left to right.

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

pairwise relationship, and thus determines who interacts with whom. On the graph, individuals can leave current sites and move to vacant ones during the evolutionary process. Each time step, a randomly chosen individual at site x plays the game with all individuals, if any, at its neighboring sites on the graph (see Fig. 2). These interactions result in the total payoff Px of the individual at site x: Px ¼

X  y A Γ ðxÞ

 aSx Sy þbSx Sy ;

ð4Þ

105

where Γ ðxÞ represents h i the neighborhood of the individual at site x, and Sx A 1; K 2 denotes the sequence number of this individual's strategy. Then the individual at site x considers changing its state. With probability μ, the individual randomly adopts any other available strategy, and with probability 1  μ, it would detect the state of its randomly selected neighboring site. Here we simply assume that individuals can only obtain local information. If the chosen neighboring site, e.g., v, is empty, the individual at site x will move to the vacant site with probability qM; else if the selected neighboring site, e.g., y, is occupied by an individual, this individual acquires its payoff Py identically the way as the individual at site x does previously. Then the individual at site x imitates the strategy of the individual at site y with likelihood given by the Fermi function (Szabó and Fáth, 2007):   f Py  Px ¼

1    ; 1 þexp  s P y  P x

ð5Þ

where s represents the intensity of natural selection. For s-0, a coin toss decides whether the strategy of the individual at site y spreads into the individual at site x. Small values of s refer to ‘weak selection’, which means that natural selection is basically random, but more successful strategies spread more often. For s-1, i.e., ‘strong selection’, a more successful individual is always imitated, while a less successful one never.

3. Results In present work, we study the case s-0, which is the limit of ‘weak selection’ (Nowak et al., 2004; Ohtsuki et al., 2006). In this limit, we have Fig. 2. Possible graph layouts: a random regular graph when the node degree k¼4, the population size M¼6 and the number of sites N¼12. The sites that are occupied by individuals are denoted by orange solid circles, while the empty sites are denoted by violet hollow circles. In this illustration, the selected individual at site x can either imitate the strategy from other ones at sites w, y or z with the probability given by Eq. (5), or move into v with the probability qM. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

  1 ðP y  P x Þ f Py  Px  þ s: 4 2

ð6Þ

Study of this limit can be justified in two ways (i) the results derived from weak selection often remain as valid approximations for larger selection strength (Ohtsuki et al., 2006, 2007a); (ii) in the many different games we are involved, each one only makes a

Fig. 3. Stationary distributions of strategies over the ½o; a concrete strategy space. The color bar indicates the equilibrium abundance of strategies: orange means high while yellow low. Simulation results: (a) k ¼4 (o  0:278 and a  0:237), (b) k ¼8 (o  0:297 and a  0:226), (c) k ¼ 12 (o  0:305 and a  0:218), and (d) k ¼ 99 (o  0:193 and a  0:158). Theoretical results (see Appendices A–C): (e) k ¼ 4 (o  0:285 and a  0:231), (f) k ¼ 8 (o  0:307 and a  0:219), (g) k ¼ 12 (o  0:318 and a  0:215), and (h) k ¼ 99 (o  0:193 and a  0:158). The simulation results are averaged over 5 independent runs, each sampling time 1010. Other parameters: N ¼ 100, M ¼ 100, μ ¼ 10  5 , s ¼ 0.005, ξ ¼ 0:01 and K¼ 11. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

106

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

small contribution to our overall performance (Ohtsuki et al., 2007b; Fu et al., 2009). Besides, in this paper we restrict our attention to random regular graphs, wherein all of N sites have the same number k of neighboring sites, and are randomly distributed with M ( r N) individuals (see Fig. 2). This is because only on random regular graphs can we investigate the role of node degree, dilution and migration in the evolution of fairness in the ultimatum game explicitly. Furthermore, we have observed that the results for random regular graphs are qualitatively the same as those for other graphs, such as random graphs and scale-free networks. 3.1. Revisiting the impact of node degree on the evolution of fairness in the ultimatum game Preliminary studies have suggested that the enlargement of neighborhood size (i.e., node degree) leads to the reduction of both offer and acceptance level of populations (Page et al., 2000; Iranzo et al., 2011). Here we comprehensively revisit the impact of node degree on the evolution of fairness in the ultimatum game. For this purpose, we present the steady-state frequency distribution over the ½o; a discrete strategy space for four specific values of node degree k (i.e., k¼4, 8, 12 and 99) via both computer simulations and theoretical analysis in Fig. 3. We find that self-compatible strategies (i.e., strategies satisfying o Z a 4 0) are much more common than self-incompatible strategies (i.e., strategies satisfying a 4 o 4 0). This is because individuals with self-incompatible strategies obtain nothing when interacting with themselves. Consequently, they have little chance of surviving in the structured population (Wang et al., 2014). Furthermore, we notice that the most common selfcompatible strategies first move towards the right, and then the left of the concrete strategy space, as the increment of node degree k. The offer and acceptance level calculated by both computer simulations and theoretical analysis support our naked-eye observation (see Fig. 3). To manifest the impact of node degree on the evolution of fairness more clearly, we study how the offer and acceptance level o and a vary systematically with the changes in node degree k in Fig. 4. In line with the preliminary studies (Page et al., 2000; Iranzo et al., 2011), the acceptance level a is decreased with node degree k across the whole applicable range. Interestingly, we note that there exists an intermediate value of node degree k resulting in the maximal offer level. While preliminary investigations indicated that the increase of node degree k would result in the

decline of the offer level (Page et al., 2000; Iranzo et al., 2011), our simulation results further reveal that the increment of node degree k may also lead to the rise of the offer level. The predictions made by theoretical analysis confirm the existence of such an intriguing phenomenon (see Fig. 4). The interpretation of the results in Fig. 4 can be provided as follows. In a previous study (Wang et al., 2014), we have shown that the tradeoff between rejecting unfair offers (achieving by maintaining high a) and making more successful deals (achieving by decreasing a) results in the best acceptance thresholds for individuals in structured populations. When node degree k is low, the former factor (i.e., rejecting unfair offers) is much more important than the latter one (i.e., making more successful deals), which leads to the maintenance of high acceptance level a. With the increment of node degree k, the performances of individuals increasingly rely on making more successful deals. Thus we can inspect the monotonous decrease of acceptance level a with node degree k (see Fig. 4). We now turn to investigate the interesting phenomenon of the optimal node degree. Note that when k⪡N, the local frequency of individuals equilibrates much faster than global density of individuals on population structures, which naturally leads to a separation of timescales regarding local and global dynamics (Ohtsuki et al., 2006, 2007a, 2007b; Fu et al., 2009, 2010). Then in the limit of small mutation rates μ-0, an n-player on a random regular graph with node degree k obtains its normalized payoff as P n =k ¼ qnj n H ðon  an Þ þ qmj n ½om H ðom  an Þ þ ð1  on ÞH ðon  am Þ;

ð7Þ

where qnjn and qmjn ¼ 1  qnjn denotes the conditional probability that an n-player has a neighboring n-player and a neighboring mplayer, respectively. It is noteworthy that the increase of k would cause the decrease of probability that homospecific individuals (i.e., individuals adopting the same strategy) encounter, but the increase of probability that heterospecific individuals (i.e., individuals adopting different strategies) meet across the whole range of pn for evolutionary dynamics on  [i.e., qnjn ¼ pn þ1=ðk  1Þ 1 pn  graphs and qmjn ¼ 1  1=ðk  1Þ 1  pn ; see Eq. (A.25)] (Ohtsuki et al., 2006). This means that the performances of individuals increasingly depend on the interactions with other heterospecific individuals [see Eq. (7)]. This impact, together with the desire to make more successful deals, drives individuals to enhance their offer levels (see Fig. 4). As the further increase of node degree k, due to the continuous decline of the overall acceptance level, it is no necessary for individuals to maintain high offer levels. As a result, individuals would lower their offer levels (see Fig. 4). 3.2. Exploring the effects of dilution and migration on the evolution of fairness in the ultimatum game

Fig. 4. Evolution of fairness in the ultimatum game: the average offer and acceptance level o (□) and a (○) versus node degree k on random regular graphs. The simulation results are averaged over 5 independent runs, each sampling time 1010. The solid lines correspond to the theoretical results obtained from either perturbation method (k⪡N) (see Appendices A and C) or mean field method (k ¼ N  1) (see Appendices B and C). Other parameters: N ¼ 100, M ¼ 100, μ ¼ 10  5 , s ¼ 0:005, ξ ¼ 0:01 and K¼ 11.

In order to explore the effects of dilution and migration on the evolution of fairness, we show the steady-state frequency distribution over the ½o; a discrete strategy space for four representative values of vacancy rate [i.e., pφ ¼ ðN  MÞ=N ¼ 0, pφ ¼ ðN  MÞ=N ¼ 0:5, pφ ¼ ðN  MÞ=N ¼ 0:8 and pφ ¼ ðN MÞ=N ¼ 0:9 for qM ¼ 0:5 and M¼100] and migration rate [i.e., qM ¼ 0:01, qM ¼ 0:1, qM ¼ 0:2 and qM ¼ 1 for M¼100 and pφ ¼ ðN  MÞ=N ¼ 0:84] respectively, through both computer simulations and theoretical analysis in Fig. 5. We can see that, with the increase of either vacancy rate pφ or migration rate qM, the advantageous strategies move towards the upper left region of the concrete strategy space, though the most common strategies are still self-compatible and remain on the main diagonal region (see Fig. 5). Interestingly, when vacancy rate pφ or migration rate qM exceeds a critical threshold, self-incompatible strategies (i.e., strategies satisfying a 4 o 4 0) become advantageous in the population (see Fig. 5). Fig. 6 shows the influences of vacancy rate pφ and migration rate qM on the evolution of fairness across the applicable parameter

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

107

Fig. 5. Stationary distributions of strategies over the ½o; a concrete strategy space. The color bar indicates the equilibrium abundance of strategies: orange means high while yellow low. The upper two rows show the results with respect to varying vacancy rate pφ ¼ ðN  MÞ=N, whereas the bottom two rows show the results with respect to changing migration rate qM on random regular graphs. Simulation results: (a) pφ ¼ 0 (o  0:278 and a  0:237), (b) pφ ¼ 0:5 (o  0:264 and a  0:247), (c) pφ ¼ 0:8 (o  0:257 and a  0:254), and (d) pφ ¼ 0:9 (o  0:255 and a  0:257), when migration rate qM ¼ 0:5; (i) qM ¼ 0:01 (o  0:261 and a  0:250), (j) qM ¼ 0:1 (o  0:258 and a  0:253), (k) qM ¼ 0:2 (o  0:257 and a  0:254), and (l) qM ¼ 1 (o  0:255 and a  0:256), when vacancy rate pφ ¼ ðN  MÞ=N ¼ 0:84. Theoretical results (see Appendices A and C): (e) pφ ¼ 0 (o  0:285 and a  0:231), (f) pφ ¼ 0:5 (o  0:267 and a  0:247), (g) pφ ¼ 0:8 (o  0:256 and a  0:256), and (h) pφ ¼ 0:9 (o  0:252 and a  0:259), when migration rate qM ¼ 0:5; (m) qM ¼ 0:01 (o  0:258 and a  0:252), (n) qM ¼ 0:1 (o  0:256 and a  0:255), (o) qM ¼ 0:2 (o  0:255 and a  0:256), and (p) qM ¼ 1 (o  0:254 and a  0:258), when vacancy rate pφ ¼ ðN  MÞ=N ¼ 0:84. The simulation results are averaged over 5 independent runs, each sampling time 1010. Other parameters: M ¼ 100, μ ¼ 10  5 , k ¼ 4, s ¼0.005, ξ ¼ 0:01 and K ¼11. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

range. Both simulation and analytical results confirm that the offer level of populations o decreases with vacancy rate pφ and migration rate qM, while the acceptance level a increases with them. Although its good performance on predictions, our analytical method cannot obtain accurate results that are in an excellent agreement with the simulation data (see Fig. 6). The deviation between theory and computer simulations is mainly due to the finite-size effects, which, unfortunately, cannot be corrected in pair approximation. To intuitively understand the nontrivial results, let us study how the mean payoffs of individuals vary with vacancy rate pφ as well as migration rate qM. In the limit of small mutation rates μ-0, the average payoff of an n-player on a random regular graph with small node degree k can be given by P n ¼ kqnjn Hðon an Þ þ kqmjn ½om Hðom an Þ þ ð1  on ÞHðon  am Þ; where

conditional

probabilities

qnjn

and

qmjn

ð8Þ satisfy

qnjn þ qmjn þqφjn ¼ 1. On the other hand, evolutionary dynamics on graphs with migration for weak selection s-0 represents a first order correction to the dynamics under neutral evolution [see Eq. (A.23)]. In this case, local configurations [e.g., qnjn and qmjn in Eq. (8)] can be approximated by neutral evolution on graphs with migration. We thus study spatial correlations (i.e., qnjn and qmjn ) under neutral evolution on random regular graphs with migration in Fig. 7. As the increase of vacancy rate pφ , both qnjn and qmjn decrease across the whole range of pn [see Fig. 7(a)], which results in the attenuation of interaction frequency of individuals on graphs [see Eq. (8)]. Furthermore, we observe that the increment of vacancy rate pφ would elicit the decrease of the normalized conditional probability qnjn =ðqnjn þ qmjn Þ while the increase of the normalized conditional probability qmjn =ðqnjn þ qmjn Þ across the whole span of pn [see Figs. 7(b) and A.1(a, c)].This indicates that the payoffs of individuals increasingly rely on the interactions with

108

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

Fig. 6. Evolution of fairness in the ultimatum game: the average offer and acceptance level o (□) and a (○) as a function of (a) vacancy rate pφ ¼ ðN  MÞ=N when migration rate qM ¼ 0:5, and (b) migration rate qM when vacancy rate pφ ¼ ðN  MÞ=N ¼ 0:84 on random regular graphs. All data points are averaged over 5 independent runs, each sampling time 1010. The solid lines correspond to the theoretical results obtained from the perturbation method (see Appendices A and C). Other parameters: M ¼100, μ ¼ 10  5 , k ¼ 4, s ¼0.005, ξ ¼ 0:01 and K¼ 11.

other heterospecific individuals [see Eq. (8)]. Such an effect, together with the attenuation of interaction frequency between individuals, leads to the result that the fate of an individual mainly depends on defeating its very few evolutionary opponents in the neighborhood. Therefore, individuals incline to reduce their offer level o, but enhance acceptance threshold a [see Fig. 6(a)]. Regarding the effects of migration rate qM, we even find the concentration of interaction frequency between heterospecific individuals (i.e., qmjn ), besides the attenuation of interaction frequency between homospecific individuals (i.e., qnjn ) [see Figs. 7(c, d) and A.1(e, g)]. Thus we observe the decrease of offer level but the increase of acceptance level with migration rate qM [see Fig. 6(b)]. Finally, we would like to point out that the results reported above remain valid if the full ultimatum game (with its continuum of strategies) is considered. Furthermore, the results are also robust against the variation of mutation rates.

4. Discussion and conclusion In our study, we have revealed that the offer level of populations would be increased with node degree. Evolutionary graph theory has shown that the increment of node degree decreases the chance that homospecific individuals meet, but increases the opportunity that heterospecific individuals encounter (Nowak

et al., 2010). Under such a circumstance, individuals prefer to enhance their offer levels to make more successful deals with others, and thus become more generous. Paradoxically, previous works have demonstrated that the enlargement of neighborhood size disfavors the evolution of generous behavior (i.e., cooperation) in either prisoner's dilemma game (Ohtsuki et al., 2006) or public goods game (Santos et al., 2008; Perc et al., 2013). In the prisoner's dilemma game or public goods game, cooperators on graphs can protect themselves from the attack of defectors, and even prevail in populations by forming network clusters, the mechanism of which can be called ‘network reciprocity’ (Nowak, 2006). However, this evolutionary mechanism is inapplicable for the explanation of the evolution of fairness in the networked ultimatum game, as clusters of individuals with different self-compatible strategies (either fair or selfish) receive the same average payoff. On the contrary, the evolutionary fate of a self-compatible strategy in the ultimatum game lies in its performance against other ones (Wang et al., 2014). This is the reason that induces the paradox evolutionary phenomenon between the ultimatum game and the prisoner's dilemma game or the public goods game on graphs. Moreover, we have shown that the dilution of population structures and the endowment of migration ability to individuals lead to the drop of offer level, while the rise of acceptance level of populations in the networked ultimatum game. In the limit of weak selection, both simulation and analytical results show that both dilution and migration cause the attenuation of interaction frequency between individuals on graphs. In addition, the payoffs of individuals increasingly rely on the interactions with other heterospecific individuals. The combination of both impacts drives individuals to lower offer level and enhance acceptance threshold so as to beat a handful of evolutionary opponents in their neighborhoods. Interestingly, we have found that, when either vacancy rate or migration rate exceeds a critical threshold, the self-incompatible strategies can survive and even become the advantageous strategies in populations. Indeed, this kind of maladaptive behaviors is present in reality. Based on our above analysis, we predict that this behavior would have a foothold in populations with sparse structures and frequent migration activities, although it still requires experimental confirmations. On the other hand, we note that individuals with such a type of strategies have double moral standards, i.e., individuals who make offers which they themselves would refuse to accept, and thus resembles immoralists, who hypercritically punish other defectors though they defect themselves (Helbing et al., 2010), or antisocial punishers, who punish cooperators even though they contribute nothing to the common pool in the public goods game (Rand and Nowak, 2011). Intriguingly, previous studies have found that moderate dilution of graphs, which is closely related to the percolation threshold of the graphs, can lead to the optimal environment for the evolution of cooperation either in the pairwise social dilemmas (Wang et al., 2012a) or in the public goods game (Wang et al., 2012b). In future studies, it will be worth investigating why the dilution of graphs has different impacts on the evolution of fairness and on the evolution of cooperation. Throughout present work, we obtain our main findings under the assumption of weak selection. However, it was found that the results under weak selection, especially for evolutionary games with multiple strategies, cannot be simply generalized to strong selection in both unstructured and structured populations. In wellmixed populations, the abundance of strategies would qualitatively change with selection intensity (Wu et al., 2013). In structured populations, the results may be divergent for a different selection intensity even for evolutionary games with only two strategies (Pinheiro et al., 2012; Mullon and Lehmann, 2014). Therefore, it is worth studying the robustness of our findings against selection intensity in future works. In summary, we have revisited the impact of node degree on the evolution of fairness in the ultimatum game. Interestingly, by means of

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

109

Fig. 7. Neutral evolution on random regular graphs with migration. (a) The equilibrium conditional probabilities qnjn (□: pφ ¼ 0; ▵: pφ ¼ 0:9) and qmjn (○: pφ ¼ 0; ▿: pφ ¼ 0:9) as a function of the normalized fraction of n-players, pn =ð1  pφ Þ, on random regular graphs, when qM ¼ 0:5. (b) The relative difference between the normalized conditional probabilities when pφ ¼ 0:9 and those when pφ ¼ 0 [◃: qnjn =ðqnjn þ qmjn Þ; ▹: qmjn =ðqnjn þ qmjn Þ]. (c) The equilibrium conditional probabilities qnjn (□: qM ¼ 0:01; ▵: qM ¼ 1) and qmjn (○: qM ¼ 0:01; ▿: qM ¼ 1) as a function of the normalized fraction of n-players, pn =ð1 pφ Þ, on random regular graphs, when pφ ¼ 0:84. (d) The relative difference between the normalized conditional probabilities when qM ¼ 1 and those when qM ¼ 0:01 [▹: qnjn =ðqnjn þ qmjn Þ; ▹: qmjn =ðqnjn þ qmjn Þ]. All data points are averaged over 3  105 times of invasion attempts starting from a single randomly placed n-player in a homogeneous population of m-player. The solid lines correspond to pair approximation results predicted by Eq. (A.24). Other parameters: M ¼100 and k ¼ 4.

computer simulations, we find that there exists an optimal value of node degree resulting in the maximal offer level of populations, which was unrevealed in previous studies. In addition, we have explored the effects of dilution and migration on the evolution of fairness in the ultimatum game. Once population structures, denoted by graphs, are diluted, individuals not only play games with others, but also have opportunities to move into vacant sites. Our simulation results demonstrate that both the dilution of population structures and the endowment of migration ability to individuals would result in the decline of offer level, and the rise of acceptance level of populations. To validate the results of numerical simulations, we also propose an analytical method, which can be considered as an extension of evolutionary graph theory (Lieberman et al., 2005; Ohtsuki et al., 2006, 2007a; Pacheco et al., 2006; Antal et al., 2006; Nowak et al., 2010), to study evolutionary dynamics on graphs with migration. Although its application to investigate evolutionary dynamics for the special case of the ultimatum game in present work, this method can be applied to explore any games governed by pairwise interactions in finite populations.

Acknowledgment Financial support from the National Natural Science Foundation of China (NSFC) (Grant nos. 61020106005 and 10972002) is gratefully acknowledged.

Appendix A. Perturbation method for calculating fixation probabilities on regular graphs with migration In this appendix, we would like to describe a perturbation method for calculating fixation probabilities on regular graphs with migration in detail. This method is based on the combination of pair approximation (Ohtsuki et al., 2006; Szabó and Fáth, 2007; Fu et al., 2010) and discrete Markov process (Fudenberg and Imhof, 2006). Note that the method presented below is valid only in the limit of s-0 and N; M⪢k on regular graphs. A.1. Calculating local configuration densities in the equilibrium under neutral evolution by using pair approximation Let pn, pm and pφ denote the fraction of n-players, m-players and vacant sites in a population, respectively. Let pnn, pnm, pmn, pmm, pnφ , pφn , pmφ , pφm and pφφ represent the fractions of nn, nm, mn, mm, nφ; φn; mφ; φm and φφ pairs, respectively. Then qXjY ¼ pXY =pY specifies the conditional probability that the neighboring site of a site of state Y is in state X. Herein, X and Y stand for the state of a site, which is occupied by either an n-player or an mplayer, or just vacant. For a pair approximation method, only frequencies of state pairs pXY are tracked. The probabilities of larger configurations are expressed and approximated by the frequencies of pair configurations. Based on the symmetry condiP tion pXY ¼ pYX , the compatibility condition pX ¼ Y pXY , and the

110

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

closure conditions, the whole system can be described by the following four variables: pn, pnn, pmm and pφφ in pair approximation. Hence, there are four events: an m-player imitating an nplayer, an m-player migrating into a neighboring vacant site, an nplayer imitating an m-player, and an n-player migrating into a neighboring vacant site, which would affect the state of the system.

A.1.1. An m-player imitating an n-player Let us first consider the case that a randomly selected focal mplayer switches to an n-player. The m-player has km m-players, kn n-players and kφ ¼ k  km  kn vacant sites in its neighborhood on a regular graph with connectivity k. The frequency of such a configuration is !km !kn    k  kφ qmjm qnjm k kφ ! k!   qkφφjm 1 qφjm   : qmjm þqnjm km ! k  kφ  km ! qmjm þ qnjm kφ ! k  kφ !

ðA:1Þ The payoff of the focal m-player is P m ðkm ; kn Þ ¼ amn km þ bmn kn and 0 0  the payoff of a neighboring n-player is P n km ; kn ¼ 0 0 0 0 cmn ðkm þ 1Þ þ dmn kn , where km and kn are the number of m- and n-players among the k  1 remaining neighbors besides the focal 0 m-player and kφ vacant sites. The frequency of this configuration is   0  k  1  k0φ k 1  kφ ! 0 ðk  1Þ! kφ   qφjnm 1  qφjnm   0 0 0 0 0 km ! k 1  kφ  km ! kφ ! k  1  kφ ! qmjnm qmjnm þqnjnm

!k0m

qnjnm qmjnm þ qnjnm

!k0n ;

ðA:2Þ

where qXjYZ gives the conditional probability that a site next to the YZ pair is in state X. Here X, Y and Z denote the state of a site, which is occupied by either an m-player or an n-player, or just vacant. The probability that the focal m-player switches to an nplayer can be written as Wm 4n ¼ 

kn k

 k  1  kφ 0 ðk  1Þ! k   qφφjnm 1  qφjnm 0 0 0 0 0 kn þ km þ kφ ¼ k  1kφ ! k 1  kφ ! X

 0 k 1  kφ !

qmjnm   0 0 0 km ! k 1  kφ  km ! qmjnm þ qnjnm   0 0   f P n kn ; km P m ðkn ; km Þ : 0

!k0m

qnjnm qmjnm þqnjnm

!k0n

ðA:3Þ

Consequently, pn increases by 1=M, where M denotes the population size, with probability

 1 T nþ ¼ Prob Δpn ¼ M ¼

  pm k!   qkφ 1  qφj m k  kφ ∑ 1  pφ kn þ km þ kφ ¼ kkφ ! k  kφ ! φj m

!km !kn   qmj m qnj m k  kφ !   W m  4 n: qmj m þ qnj m km ! k  kφ km ! qmj m þ qnj m ðA:4Þ At the same time, the number of nn pairs increases by kn, and thus pnn increases by 2kn =ðkNÞ with probability

Prob Δpnn ¼

2kn kN

 ¼ pm

 kn  k  kn kX k! qkn 1  qnjm kn !ðk  kn Þ! njm k ¼0

qmjm ðk  kn Þ! km !ðk  kn km Þ! qmjm þ qφjm

m

!km

qφjm qmjm þ qφjm

!kφ Wm  4 n:

ðA:5Þ

The number of mm pairs decreases by km, and thus pmm decreases by 2km =ðkNÞ with probability

  km  k  km kX 2km k! Prob Δpmm ¼  ¼ pm qkm 1 qmjm km !ðk  km Þ! mjm kN kn ¼ 0 !kn !kφ qφjm qnjm ðk  km Þ! Wm  4 n: ðA:6Þ kn !ðk  km  kn Þ! qnjm þqφjm qnjm þqφjm

A.1.2. An m-player migrating into a neighboring vacant site Then let us consider the case that a randomly selected focal mplayer moves to a neighboring vacant site. The m-player has km mplayers, kn n-players and kφ ¼ k km  kn vacant sites in its neighborhood on a regular graph with connectivity k. The frequency of such a configuration is !km !kn    k  k φ qmjm qnjm k  kφ ! k!   qkφφjm 1 qφjm   : qmjm þ qnjm km ! k  kφ  km ! qmjm þ qnjm kφ ! k kφ !

ðA:7Þ The frequency of the configuration, in which a neighboring vacant 0 0 0 0 0 site has kn n-players, km m-players and kφ ¼ k  1  kn  km vacant sites, is   0  k  1  k0φ k  1  kφ ! 0 ðk  1Þ! kφ   qφjφm 1  qφjφm   0 0 0 0 0 km ! k  1  kφ km ! kφ ! k  1  kφ ! !k0m !k0n qmjφm qnjφm : ðA:8Þ qmjφm þ qnjφm qmjφm þ qnjφm The probability that the focal m-player moves to one of its neighboring vacant sites can be written as  k  1  k0φ 0 X kφ ðk  1Þ! k   qφφjφm 1  qφjφm Wm 4φ ¼ 0 0 k 0 0 0 kn þ km þ kφ ¼ k  1kφ ! k  1  kφ !  0



 0 k  1  kφ ! 0

0

km ! k  1  kφ  km



qmjφm q mj φ m þ qnjφm !

!k0m

qnjφm qmjφm þqnjφm

!k0n qM ; ðA:9Þ

where qM is the migration rate. The number of mm pairs increases 0 0 by km  km , and thus pmm increases by 2ðkm km Þ=ðkNÞ with probability  0 

 k  kφ 2 km  km k!  qkφφjm 1  qφjm ¼ pm  Prob Δpmm ¼ kN kφ ! k  kφ ! !km !kn   qmjm qnjm k  kφ !   W m  4 φ: qmjm þ qnjm km ! k  kφ  km ! qmjm þ qnjm ðA:10Þ 0

The number of φφ pairs increases by kφ  kφ 1, and thus pφφ 0 increases by 2ðkφ  kφ  1Þ=ðkNÞ with probability  1 0 0  k  k φ 2 kφ kφ  1 A ¼ pm  k!  qkφφjm 1  qφjm Prob@Δpφφ ¼ kN kφ ! k  kφ ! !km !kn   qmjm qnjm k  kφ !   W m  4 φ: qmjm þ qnjm km ! k  kφ  km ! qmjm þ qnjm ðA:11Þ A.1.3. An n-player imitating an m-player Similarly, the probability that pn decreases by 1=M is given by

 1 T n ¼ Prob Δpn ¼  M

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

111

Fig. A1. Numerical solutions of Eq. (A.24). The top (bottom) row shows the normalized conditional probability [i.e., qnjn =ðqnjn þ qmjn Þ, qnjm =ðqmjm þ qnjm Þ, qmjn =ðqnjn þ qmjn Þ, and qmjm =ðqmjm þ qnjm Þ] in the equilibrium as a function of the normalized fraction of n-players, pn =ð1 pφ Þ, and pφ when qM ¼ 0:5 (qM when pφ ¼ 0:84) for k ¼4.

¼

pn 1  pφ k

n

X þ km þ kφ

 k  kφ k!   qkφφjn 1 qφjn k ! k  kφ ! ¼k φ

!km !kn   qmjn qnjn k  kφ !   W n  4 m; qmjn þ qnjn km ! k  kφ  km ! qmjn þ qnjn

neighborhood on a regular graph with connectivity k. The frequency of such a configuration is ðA:12Þ

where W n  4 m denotes the probability that the focal n-player switches to an m-player, i.e.,  k  1  k0φ 0 X km ðk  1Þ! k   qφφjmn 1  qφjmn Wn 4m ¼ 0 0 k 0 0 0 kn þ km þ kφ ¼ k  1kφ ! k  1 kφ ! 

 0 k  1  kφ !

qmjmn   0 0 0 km ! k  1  kφ  km ! qmjmn þ qnjmn   0 0   f P m kn ; km  P n ðkn ; km Þ ;

!k0m

0 qnjmn ð Þkn qmjmn þ qnjmn

!km !kn     k  kφ qmjn qnjn k  kφ ! k!   qkφφjn 1  qφjn   : qmjn þ qnjn kφ ! k  kφ ! km ! k  kφ  km ! qmjn þ qnjn

ðA:16Þ The frequency of the configuration, in which a neighboring vacant 0 0 0 0 0 site has kn n-players, km m-players and kφ ¼ k  1  kn km vacant sites, is ðk  1Þ!

  0 0 kφ ! k  1  kφ ! ðA:13Þ

 0 0 0 0 where P m km ; kn ¼ amn km þ bmn ðkn þ 1Þ and P n ðkm ; kn Þ ¼ cmn km þ dmn kn . The number of nn pairs decreases by kn and thus pnn decreases by 2kn =ðkNÞ with probability

  kn  k  kn kX 2kn k! n ¼ pn qknjn 1  qnjn Prob Δpnn ¼  kn !ðk  kn Þ! kN km ¼ 0 !km !kφ qφjn qmjn ðk  kn Þ! W n  4 m: ðA:14Þ km !ðk  kn km Þ! qmjn þ qφjn qmjn þqφjn

The number of mm pairs increases by km and thus pmm increases by 2km =ðkNÞ with probability

  km  k  km kX 2km k! ¼ pn qkm 1  qmjn Prob Δpmm ¼ km !ðk  km Þ! mjn kN kn ¼ 0 !kn !kφ qφjn qnjn ðk  km Þ! W n  4 m: ðA:15Þ kn !ðk  km kn Þ! qnjn þ qφjn qnjn þqφjn

A.1.4. An n-player migrating into a neighboring vacant site Then let us consider the case that a randomly selected focal nplayer moves to a neighboring vacant site. The n-player has km mplayers, kn n-players and kφ ¼ k km  kn vacant sites in its

0

k qφφjφn



1 qφjφn

qmjφn qmjφn þ qnjφn

!k0m

k  1  k0φ



 0 k  1  kφ !

  0 0 0 km ! k 1  kφ  km !

qnjφn qmjφn þ qnjφn

!k0n :

ðA:17Þ

The probability that the focal n-player moves to one of its neighboring vacant sites can be written as Wn  4φ ¼

kφ k 

 k  1  k φ ðk  1Þ! k0   qφφjφn 1 qφjφn 0 0 0 0 kn þ km þ kφ ¼ k  1kφ ! k  1  kφ ! X

0

0

 0 k  1 kφ !

qmjφn   0 0 0 q km ! k  1 kφ  km ! mjφn þqnjφn

!k0m

qnjφn qmjφn þqnjφn

!k0n qM : ðA:18Þ

0

The number of nn pairs increases by kn kn and thus pnn increases 0 by 2ðkn kn Þ=ðkNÞ with probability  0 

 k  k φ 2 kn  kn k!  qkφ 1  qφjn ¼ pn  Prob Δpnn ¼ kN kφ ! k  kφ ! φjn !km !kn   qmjn qnjn k kφ !   W n  4 φ: qmjn þ qnjn km ! k kφ  km ! qmjn þ qnjn

ðA:19Þ

112

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114 0

The number of φφ pairs increases by kφ  kφ  1 and thus pφφ 0 increases by 2ðkφ  kφ  1Þ=ðkNÞ with probability  1 0  k  k φ 2 kφ  kφ  1 A ¼ pn  k!  qkφ 1  qφjn Prob@Δpφφ ¼ φ jn kN kφ ! k kφ ! 0

!km !kn   qmjn qnjn k  kφ !   W n  4 φ: qmjn þ qnjn km ! k  kφ  km ! qmjn þ qnjn

ðA:20Þ

by numerically solving Eq. (A.24). Specially, if we let pφ ¼ 0 (i.e., there is no vacancy in the whole population), the results of the equilibrium local configuration probabilities qYjX obtained in Ohtsuki et al. (2006) are recovered, namely, qYjX ¼

1 k2 p ; δX;Y þ k1 k1 Y

ðA:25Þ

where X and Y denote the state of a site, which is occupied by either an m-player or an n-player, and δX;Y is equal to 1 if X¼ Y, and 0 otherwise.

Let us now suppose that one replacement event takes place in one unit of time. Hence we have 8  1 þ >  > > > Δp n ¼ M T n  T n ; > > > 2 >

 X 

3 > k k

> X 2kn 2kn 2kn 2kn > > > 6 7 Δ p ¼  Δ p ¼  Prob þ Prob > nn nn > 6 kN kN kN kN 7 > > 7 k ¼0 kn ¼ 0 > 16 > 6 n 7 >

 Δ p ¼ > 6 7; 0 0 k kX 1 nn > X > 2ðkn  kn Þ 2ðkn  kn Þ 7 N6 > > 6 7 Δ p ¼ Prob þ > nn > 4 5 kN kN > > kn ¼ 0k0n ¼ 0 > > > > > 2 > >

 X 

3 > k k

X > 2km 2km 2km 2km > > 6 7 <  Prob Δpmm ¼ þ Prob Δpmm ¼  6 7 kN kN kN kN 7 km ¼ 0 km ¼ 0 16 6 7 > >

 7; > Δpmm ¼ 6 0 0 k kX 1 X > 2ðkm  km Þ 2ðkm  km Þ 7 N6 > > 6þ 7 > Prob Δpmm ¼ > 4 5 > kN kN > 0 > km ¼ 0km ¼ 0 > > > > > 2 3 ! > 0 > k kX  1 2ðk  k0  1Þ > X 2ðkφ  kφ  1Þ > φ φ > > 6 7 Prob Δpφφ ¼ > > 6 7 kN kN > 0 > 6 7 kφ ¼ 0kφ ¼ 0 > > 7 > Δp ¼ 1 6 ! > 6 7: > φφ 0 k kX  1 2ðk  k0  1Þ > X 6 7 N 2ðk  k  1Þ > φ φ φ φ > 6 7 > þ Prob Δpφφ ¼ > 4 5 > > kN kN 0 > k ¼ 0 ¼ 0 k > φ φ :

ðA:21Þ By setting qXjYZ  qXjY and letting s-0, the above difference equation set can be simplified as

  8 Δp ≈ 1 pnm s a11 ðk  1Þqmj m  a12 ðk  1Þqnj m þ1 þa21 ðk  1Þqmj n þ1 þ a22 ðk  1Þqnj n ; > > > n M 1pφ 2  >   > > < Δpnn ≈kN1 2 ðk  1Þqnj m pnm þpnm  ðk  1Þqnj n pnm þ2pnφ ðk  1Þ qnj φ  qnj n qM ;    > Δpmm ≈ 1 2 ðk  1Þqmj n pmn þ pmn  ðk  1Þqmj m pmn þ 2pmφ ðk  1Þ qmj φ  qmj m qM ; > kN > >      > > : Δpφφ ≈2qM2 ðk  1Þ pmφ qφj m  qφj φ þpnφ qφj n  qφj φ : kN

ðA:22Þ In the limit of large site size N-1 and population size M-1, we obtain the following differential equation set:

A.2. Modeling the evolutionary dynamics on graphs with migration as a discrete Markov process Note that evolutionary dynamics on graphs with migration under weak selection can be considered as a first order correction to the dynamics under neutral evolution [see Eq. (A.23)] (Fu et al., 2009). Then we are able to couple the game according to these equilibrium local configurations. Evolutionary dynamics on graphs with migration represents a discrete Markov process on the interval ½0; M, wherein the states 0 and M being absorbing. At each time step, the number of nplayers, i, can either increase by one, decrease by one, or stay the þ  same. Let T n;m ðiÞ and T n;m ðiÞ denote the transition probability that i increases ði-i þ 1Þ or decreases ði-i 1Þ, respectively. Thus we have  " !# cmn ðk  1Þqmj n ðiÞ þ 1 þdmn ðk 1Þqnj n ðiÞ ðiÞ 1 þ s  T n;m ðiÞ≈1pnm þ ;  pφ 2 4 amn ðk 1Þqmj m ðiÞ  bmn ðk  1Þqnj m ðiÞ þ 1  !# " amn ðk 1Þqmj m ðiÞ þ bmn ðk  1Þqnj m ðiÞ þ 1 ðiÞ 1  s  ðiÞ≈1pmn þ : T n;m  pφ 2 4  cmn ðk  1Þqmj n ðiÞ þ 1  dmn ðk  1Þqnj n ðiÞ ðA:26Þ Denote xi by the fixation probability of n-player when starting from i n-players. We have the following recursive equation:   þ þ   xi ¼ T n;m ðiÞxi þ 1 þ 1  T n;m ðiÞ  T n;m ðiÞ xi þ T n;m ðiÞxi  1 ; ðA:27Þ with boundary conditions x0 ¼ 0 and xM ¼1.

8

  Δpn pnm s > > > ṗn ¼ limM↦∞ 1=M≈1  pφ 2 a11 ðk  1Þqmj m  a12 ðk  1Þqnj m þ 1 þ a21 ðk  1Þqmj n þ 1 þ a22 ðk  1Þqnj n ; > >    > 1 > nn < ṗnn ¼ limN↦∞ Δp 1=N ≈kN ðk  1Þqnj m pnm þ pnm  ðk  1Þqnj n pnm þ 2pnφ ðk  1Þ qnj φ  qnj n qM ;    1 mm > ṗmm ¼ limN↦∞ Δp > 1=N ≈kN ðk  1Þqmj n pmn þ pmn  ðk  1Þqmj m pmn þ 2pmφ ðk  1Þ qmj φ  qmj m qM ; > > >      > Δpφφ 2qM > : ṗφφ ¼ limN↦∞ 1=N ≈ kN ðk  1Þ pmφ qφj m  qφj φ þ pnφ qφj n  qφj φ :

Under the condition of neutral evolution, we can obtain the following equation set which can be used to obtain the local configuration density qYjX in the equilibrium 8   > > > ðk  1Þqnjm pnm þ pnm  ðk 1Þqnjn pnm þ2pnφ ðk 1Þ qnjφ  qnjn qM ¼ 0; > >   < ðk  1Þqmjn pmn þ pmn  ðk 1Þqmjm pmn þ 2pmφ ðk  1Þ qmjφ  qmjm qM ¼ 0; >     > > > > : pmφ qφjm qφjφ þ pnφ qφjn  qφjφ ¼ 0:

ðA:24Þ Fig. A1 shows how the normalized conditional probability in dependence on the vacancy rate, pφ , and the migration rate, qM,

ðA:23Þ

In order to solve above equation, we rewrite Eq. (A.27) as xi þ 1  xi ¼

 T n;m ðiÞ

i T  ðjÞ ðxi  xi  1 Þ ¼ ∏ n;m ðx1  x0 Þ: þ þ T n;m ðiÞ j ¼ 1 T n;m ðjÞ

ðA:28Þ

Summing Eq. (A.28) from i¼ 1 to i ¼ M  1 we obtain xN  x1 ¼

M 1 X

i

∏

 T n;m ðjÞ

ðx1  x0 Þ: T þ ðjÞ i ¼ 1 j ¼ 1 n;m

ðA:29Þ

Let ρn;m denote the probability that a single randomly placed nplayer in an otherwise homogeneous population of m-players

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

reaches fixation in the absence of mutation. Thus we have 1

ρn;m ¼ x1 ¼ 1þ

PM  1 i¼1

∏ij ¼ 1

 T n;m ðjÞ

:

ðA:30Þ

þ T n;m ðjÞ

Appendix B. Mean field method for calculating fixation probabilities in well-mixed populations For well-mixed populations, the transition probabilities for the number of i n-players are given by þ ðiÞ ¼ MM i Mi 1 þ exp½  sð1P n ðiÞ  Pm ðiÞÞ; T n;m

ðB:1Þ

 T n;m ðiÞ ¼ Mi MM i 1 þ exp½  sð1P m ðiÞ  P n ðiÞÞ;

where P m ðiÞ ¼ amn ðM  i  1Þ þ bmn i and P n ðiÞ ¼ cmn ðM  iÞ þ dmn ði  1Þ. According to Eqs. (5) and (A.30) and after some algebra, the fixation probability ρn;m (Ewens, 2004; Karlin, 2014) can be simplified to

ρn;m ¼

1þ

PM  1 i¼1

1  : P exp  s ij ¼ 1 ½P n ðjÞ  P m ðjÞ

ðB:2Þ

Appendix C. Calculating the stationary distribution Generally, the embedded Markov chain method presented below can approximate the stochastic evolutionary process well if only mutation rate is sufficient low, i.e., μ-0 (Fudenberg and Imhof, 2006). For μ-0, the population is homogeneous most of the time, and consists always of one or two types at most. In this case, the fate of a mutant (i.e., elimination or fixation) will be settled before the next mutant appears. However, if there exist coexistence games among the pair interactions between individuals, the embedded Markov chain method cannot make a good approximation of this stochastic evolutionary process (Wu et al., 2012). This is because the conditional fixation time of a strategy in a coexistence game is extremely long, which makes it hard to ensure that the whole population stays in the homogeneous state most of the time. Nevertheless, in the limit of weak selection, the conditional fixation time for evolutionary games with different dynamical properties is almost the same as that for the neutral case. Therefore, the embedded Markov chain method can be validly applied to our model. The fixation probabilities ρn;m define the transition probabilities of a Markov process between the K2 different homogeneous states of the population. The transition matrix T is given by 2

μρ

μρ

2

2;1 K ;1 6 1  K 2  1  ⋯  K 2 1 6 6 μρ 2;1 6 6 T ¼ 6 K2  1 6 6⋮ 6 μρ 2 4 K ;1 K2  1

μρ1;2

⋯

K2  1 1 ⋮

μρ1;2

K2  1

μρK 2 ;2

K2  1

⋯ 

μρK 2 ;2 K 2 1

⋯

3

μρ1;K 2 K2  1

μρ2;K 2

⋮

K2  1 ⋮

⋯

1

μρ1;K 2

K2  1

⋯

μρK 2  1;K 2

7 7 7 7 7 7 7 7 7 5

K2  1

ðC:1Þ According to T, the normalized right eigenvector Λ ¼ ½λ1 ; λ2 ; …; λK 2 T to the largest eigenvalue (which is 1 for the matrix T) can be derived to determine the stationary distribution, where λi represents the percentage of timeh spent i by the system staying in the homogeneous state ½oi ; ai ; i A 1; K 2 . References Antal, T., Redner, S., Sood, V., 2006. Evolutionary dynamics on degreeheterogeneous graphs. Phys. Rev. Lett. 96, 188104.

113

Brockmann, D., Hufnagel, L., Geisel, T., 2006. The scaling laws of human travel. Nature 439, 462–465. Camerer, C.F., 2003. Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University Press, Princeton. Chen, X., Szolnoki, A., Perc, M., 2012. Risk-driven migration and the collective-risk social dilemma. Phys. Rev. E 86, 036101. Cong, R., Wu, B., Qiu, Y., Wang, L., 2012. Evolution of cooperation driven by reputation-based migration. PLoS ONE 7, e35776. Duan, W.Q., Stanley, H.E., 2010. Fairness emergence from zero intelligence agents. Phys. Rev. E 81, 026104. Ewens, W.J., 2004. Mathematical Population Genetics 1: I. Theoretical Introduction. Springer, New York. Fehr, E., Schmidt, K., 1999. A theory of fairness, competition, and cooperation. Q. J. Econ. 114, 817–868. Fu, F., Nowak, M.A., 2013. Global migration can lead to stronger spatial selection than local migration. J. Stat. Phys. 151, 637–653. Fu, F., Nowak, M.A., Hauert, C., 2010. Invasion and expansion of cooperators in lattice populations: prisoner's dilemma vs. snowdrift games. J. Theor. Biol. 266, 358–366. Fu, F., Wang, L., Nowak, M.A., Hauert, C., 2009. Evolutionary dynamics on graphs: efficient method for weak selection. Phys. Rev. E 79, 046707. Fudenberg, D., Imhof, L.A., 2006. Imitation processes with small mutations. J. Econ. Theor. 131, 251–262. Gao, J., Li, Z., Wu, T., Wang, L., 2011. The coevolutionary ultimatum game. EuroPhys. Lett. 93, 48003. Gonzalez, M.C., Hidalgo, C.A., Barabasi, A.L., 2008. Understanding individual human mobility patterns. Nature 453, 779–782. Güth, W., Schmittberger, R., Schwarze, B., 1982. An experimental analysis of ultimatum bargaining. J. Econ. Behav. Org. 3, 367–388. Helbing, D., Szolnoki, A., Perc, M., Szabó, G., 2010. Evolutionary establishment of moral and double moral standards through spatial interactions. PLoS Comput. Biol. 6, e1000758. Helbing, D., Yu, W., 2009. The outbreak of cooperation among success-driven individuals under noisy conditions. Proc. Natl. Acad. Sci. USA 106, 3680–3685. Iranzo, J., Román, J., Sánchez, A., 2011. The spatial ultimatum game revisited. J. Theor. Biol. 278, 1–10. Jiang, L.L., Wang, W.X., Lai, Y.C., Wang, B.H., 2010. Role of adaptive migration in promoting cooperation in spatial games. Phys. Rev. E 81, 036108. Karlin, S., 2014. A First Course in Stochastic Processes. Academic Press, London. Killingback, T., Studer, E., 2001. Spatial ultimatum games, collaborations and the evolution of fairness. Proc. R. Soc. Lond. B 268, 1797–1801. Li, X., Cao, L., 2009. Largest Laplacian eigenvalue predicts the emergence of costly punishment in the evolutionary ultimatum game on networks. Phys. Rev. E 80, 066101. Lieberman, E., Hauert, C., Nowak, M.A., 2005. Evolutionary dynamics on graphs. Nature 433, 312–316. Meloni, S., Buscarino, A., Fortuna, L., Frasca, M., Gómez-Gardeñes, J., Latora, V., Moreno, Y., 2009. Effects of mobility in a population of prisoner's dilemma players. Phys. Rev. E 79, 067101. Mullon, C., Lehmann, L., 2014. The robustness of the weak selection approximation for the evolution of altruism against strong selection. J. Evol. Biol. 27, 2272–2282. Nowak, M.A., 2006. Five rules for the evolution of cooperation. Science 314, 1560–1563. Nowak, M.A., Page, K.M., Sigmund, K., 2000. Fairness versus reason in the ultimatum game. Science 289, 1773–1775. Nowak, M.A., Sasaki, A., Taylor, C., Fudenberg, D., 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646–650. Nowak, M.A., Tarnita, C.E., Antal, T., 2010. Evolutionary dynamics in structured populations. Philos. Trans. R. Soc. B 365, 19–30. Ohtsuki, H., Hauert, C., Lieberman, E., Nowak, M.A., 2006. A simple rule for the evolution of cooperation on graphs and social networks. Nature 441, 502–505. Ohtsuki, H., Nowak, M.A., Pacheco, J.M., 2007a. Breaking the symmetry between interaction and replacement in evolutionary dynamics on graphs. Phys. Rev. Lett. 98, 108106. Ohtsuki, H., Pacheco, J.M., Nowak, M.A., 2007b. Evolutionary graph theory: breaking the symmetry between interaction and replacement. J. Theor. Biol. 246, 681–694. Pacheco, J.M., Traulsen, A., Nowak, M.A., 2006. Coevolution of strategy and structure in complex networks with dynamical linking. Phys. Rev. Lett. 97, 258103. Page, K.M., Nowak, M.A., 2002. Empathy leads to fairness. Bull. Math. Biol. 64, 1101–1116. Page, K.M., Nowak, M.A., Sigmund, K., 2000. The spatial ultimatum game. Proc. R. Soc. Lond. B 267, 2177–2182. Perc, M., Gómez-Gardeñes, J., Szolnoki, A., Florìa, L.M., Moreno, Y., 2013. Evolutionary dynamics of group interactions on structured populations: a review. J. R. Soc. Interface 10, 20120997. Pinheiro, F.L., Santos, F.C., Pacheco, J.M., 2012. How selection pressure changes the nature of social dilemmas in structured populations. New J. Phys. 14, 073035. Rand, D.G., Nowak, M.A., 2011. The evolution of antisocial punishment in optional public goods games. Nat. Commun. 2, 434. Rand, D.G., Tarnita, C.E., Ohtsuki, H., Nowak, M.A., 2013. Evolution of fairness in the one-shot anonymous ultimatum game. Proc. Natl. Acad. Sci. USA 110, 2581– 2586. Roca, C.P., Helbing, D., 2011. Emergence of social cohesion in a model society of greedy, mobile individuals. Proc. Natl. Acad. Sci. USA 108, 11370–11374.

114

X. Wang et al. / Journal of Theoretical Biology 380 (2015) 103–114

Sánchez, A., Cuesta, J.A., 2005. Altruism may arise from individual selection. J. Theor. Biol. 235, 233–240. Santos, F.C., Santos, M.D., Pacheco, J.M., 2008. Social diversity promotes the emergence of cooperation in public goods games. Nature 454, 213–216. Selten, R., 1975. Reexamination of the perfectness concept for equilibrium points in extensive games. Int. J. Game Theor. 4, 25–55. Sicardi, E.A., Fort, H., Vainstein, M.H., Arenzon, J.J., 2009. Random mobility and spatial structure often enhance cooperation. J. Theor. Biol. 256, 240–246. da Silva, R., Kellerman, G.A., 2007. Analyzing the payoff of a heterogeneous population in the ultimatum game. Braz. J. Phys. 37, 1206–1211. Sinatra, R., Iranzo, J., Gómez-Gardeñes, J., Floría, L.M., Latora, V., Moreno, Y., 2009. The ultimatum game in complex networks. J. Stat. Mech., P09012. Szabó, G., Fáth, G., 2007. Evolutionary games on graphs. Phys. Rep. 446, 97–216. Szolnoki, A., Perc, M., Szabó, G., 2012a. Accuracy in strategy imitations promotes the evolution of fairness in the spatial ultimatum game. EuroPhys. Lett. 100, 28005. Szolnoki, A., Perc, M., Szabó, G., 2012b. Defense mechanisms of empathetic players in the spatial ultimatum game. Phys. Rev. Lett. 109, 078701. Vainstein, M.H., Silva, A.T.C., Arenzon, J.J., 2007. Does mobility decrease cooperation? J. Theor. Biol. 244, 722–728.

Wang, X., Chen, X., Wang, L., 2014. Random allocation of pies promotes the evolution of fairness in the ultimatum game. Sci. Rep. 4, 4534. Wang, Z., Szolnoki, A., Perc, M., 2012a. If players are sparse social dilemmas are too: importance of percolation for evolution of cooperation. Sci. Rep. 2, 369. Wang, Z., Szolnoki, A., Perc, M., 2012b. Percolation threshold determines the optimal population density for public cooperation. Phys. Rev. E 85, 037101. Wu, B., Garcìa, J., Hauert, C., Traulsen, A., 2013. Extrapolating weak selection in evolutionary games. PLoS Comput. Biol. 9, e1003381. Wu, B., Gokhale, C.S., Wang, L., Traulsen, A., 2012. How small are small mutation rates? J. Math. Biol. 64, 803–827. Wu, T., Fu, F., Zhang, Y., Wang, L., 2012. Expectation-driven migration promotes cooperation by group interactions. Phys. Rev. E 85, 066104. Wu, T., Fu, F., Zhang, Y., Wang, L., 2013. Adaptive role switching promotes fairness in networked ultimatum game. Sci. Rep. 3, 1550. Yang, H.X., Wu, Z.X., Wang, B.H., 2010. Role of aspiration-induced migration in cooperation. Phys. Rev. E 81, 065101(R). Yu, W., 2011. Mobility enhances cooperation in the presence of decision-making mistakes on complex networks. Phys. Rev. E 83, 026105.