- Email: [email protected]
Reciprocal reward promotes the evolution of cooperation in structured populations
Chaos, Solitons and Fractals 119 (2019) 230–236
Contents lists available at ScienceDirect
Chaos, Solitons and Fractals Nonlinear Science, and Nonequilibrium and Complex Phenomena journal homepage: www.elsevier.com/locate/chaos
Reciprocal reward promotes the evolution of cooperation in structured populations Yu’e Wu, Zhipeng Zhang, Shuhua Chang∗ Coordinated Innovation Center for Computable Modeling in Management Science, Tianjin University of Finance and Economics, Tianjin 300222, China
a r t i c l e
i n f o
Article history: Received 19 June 2018 Revised 25 September 2018 Accepted 6 January 2019
Keywords: Prisoner’s dilemma Reciprocal reward Cooperation Complex network
a b s t r a c t Reciprocal reward, as a common phenomenon in nature and society, plays a pivotal role in interpreting the existence and maintenance of cooperation. In this work, we introduce a reciprocal reward mechanism in the two pairwise models including the prisoner’s dilemma and the snowdrift games, where a cooperative agent with cooperative neighbors will obtain additional bonuses. Under this mechanism, we find that the evolution of cooperation is promoted to a high level. The promoting effects for the demonstrated results are independent of the applied spatial network structures and the potential evolutionary game models, and thus showing a high degree of generality. In addition, we also probe into the impact of the initial fractions of cooperators on the evolution of cooperation under this mechanism, the results of which further confirm the robustness of this mechanism. Our research may provide valuable insights into interpreting the emergence and sustainability of cooperation and thus are beneficial for resolving social dilemmas. © 2019 Published by Elsevier Ltd.
1. Introduction Cooperative behavior exists widely in nature and human society, which is contradict with Darwin’s theory of evolution [1–3]. Therefore, how to understand and explain the emergence and sustainability of cooperation among unrelated individuals becomes one of the major challenges of evolutionary biology and of behavioral sciences, thus attracting the attentions from a myriad of fields including physics, mathematics, biology and behavioral sciences [4–8]. Evolutionary game theory provides a forceful framework to elucidate this puzzle, in which social contradictions are analogous to the competition among peers for limited resources [9–11]. In particular, the prisoner’s dilemma game (PDG) and the snowdrift game (SDG) are widely applicable games to explore the evolution of cooperation [12–14]. In the traditional evolutionary PDG and SDG, two involved players can choose either to cooperate or to defect in the processes of the game [15–17]. A player who chooses cooperative strategy will obtain R when confronting a cooperator and receive S (the sucker’s payoff) when meeting a defector. Meanwhile, a defector will acquire T (P) when the opponent’s strategy is cooperation (defection). As a standard practice, the payoffs are ordered as T > R > P ≥ S in the PDG, suggesting that the defection is the
∗
Corresponding author. E-mail address: [email protected] (S. Chang).
https://doi.org/10.1016/j.chaos.2019.01.006 0960-0779/© 2019 Published by Elsevier Ltd.
optimal strategy irrespective of the opponent’s decision. While for the SDG, the payoff ranking is T > R > S > P. The minor distinction of the payoff ranking induces a significant change in the game dynamics where the optimal strategy for the individual is the opposite of the opponent’s [18–20]. Traditionally, the evolutionary games are investigated in an infinite, well-mixed population (complete graphs), where all individuals interact equally likely with each other, but cooperation cannot emerge under replicator dynamics [21]. Actually, fully-connected graphs constitute rather unrealistic representations of real-world network of contacts, in which one expects local connections (spatial structure) to coexist with long-range connections, traits recently identified as characteristics of numerous of natural, social, and technological network of contacts. Naturally, considerable attention has been shifted into structured populations, or say, people focus on studying evolutionary games on the networks [22–30]. Evolutionary graph theory provides a natural and reasonable framework for this approach: the vertices in the network topology represent players and the edges indicate links between players in terms of game dynamical interaction. The agents located on the vertices are constrained to play with their immediate neighbors along the edges. Taking these simplifying settings into consideration, Nowak and May seminally introduced the spatial structure into the PDG, where the agents were located on the square lattices [31]. They found that cooperators can build clusters via such spatial structure to resist aggression by defectors. One decade later, laboratory experiments [29,32,33] have confirmed that
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
topological constraints indeed influence the evolution of cooperation in a sizeable way. Following the pioneering work, the evolution of cooperation has been extensively explored in a variety of topologies such as regular square lattices [34], Erdös-Rényi (ER) random graphs [35], small-world networks [36], Barabási-Albert (BA) scale-free (SF) network [37] and so on. In particular, Santos et al. indicated that scale-free networks can provide a unifying framework for the appearance of cooperation [38]. The main driving force behind the flourishing cooperative behavior in scale-free networks is attributed to the strong heterogeneity of the degree distribution [39]. Besides, Perk found that cooperative behavior on scale-free networks is extremely robust against random deletion of vertices but is obstructed greatly if vertices with the maximal degree are targeted, which further confirm the significance of the degree distribution heterogeneity for the evolution of cooperation by random and intentional removal of vertices [40]. Furthermore, in the framework of graph-structured populations, a myriad of scenarios have been proposed to offset the unfavorable evolutionary outcome in the traditional games. For instance, inhomogeneous teaching activity [41], preferential selection [42], individuals’ mobility [43], different evolutionary dynamics [44], high value of the clustering coefficient and social diversity [45,46]. These examples of alternative ways have been considered as potential promoters of cooperation with fruitful success, and can be reasonable from the perspective of real society. Nowak attributed all these to five mechanisms: kin selection, direct reciprocity, indirect reciprocity, network reciprocity, and group selection [47]. Of particular renown, the influence of reward on the evolution of cooperation has attracted great attention [48–50]. Reward is an established alternative to punishment [51], albeit investigated less frequent in the past. Compared to punishment, reward incorporates a cost to bear as well, but for another individual to experience a benefit. As a matter of fact, reward may be as effective as punishment for maintaining cooperation [52,53]. In vast majority of previous studies, most of the authors focus their attentions on the cooperators alone and overlook the synergistic effect between the cooperative agent and its neighbors in the pairwise models. In realistic scenarios, mutual cooperation that makes both the two agents obtain additional benefits is a very common phenomenon, such as mutualism in nature. In human society, this phenomenon is also widespread and environmental governance is one of the most typical examples. For example, if two countries (agents) share a river, each country will benefit from its own river environmental governance (cooperation). However, if two neighboring countries cooperate, each country gains additional benefits (for example, better living conditions). When many countries are distributing along the river, as our model will describe, the more cooperative neighbors an individual has, the more additional benefits the individual receives. Moreover, in most of these reward-related articles, cooperators receive rewards as a second-stage behavior and the game models are almost limited to public goods games [54,55]. Inspired by these findings, we here study the impact of rewards on the evolution of cooperation in the structured population by means of direct reciprocity (reciprocal reward). The reciprocal reward here means that a cooperator with a cooperative neighbor will receive an extra benefit. In this scenario, cooperative behavior is not only altruistic but it is very likely to be beneficial to oneself. It is worth mentioning that this is similar to a previous work [56], which is about the impact of self-interaction on the evolution of cooperation. The authors in the reference focus on self-interaction of the agents and the cooperators can obtain additional benefits through self-interaction. Although both are cooperators who receive additional rewards, we focus on reciprocal rewards generated by the interaction between the cooperators and their cooperative neighbors. In our model, the additional rewards that a cooperator receives are not only influenced by its own strat-
231
egy, but also by the number of cooperators in its neighborhood, which is quite different from the former. Besides, it focuses on the influence of the distribution of self-interaction strength on the evolution of cooperation in the reference, while our model emphasizes the robustness of the promoting effect of cooperation to different network structures and initial fractions of cooperators. It is known that many real networks have scale-free properties and, therefore, our work has important implications to promote the application of this mechanism in real life. The simulations are conducted in the spatial PDG on three types of network structures including regular square lattices with periodic boundary conditions, Erdös-Rényi (ER) random graphs and Barabási-Albert (BA) scale-free (SF) networks. The robustness of the simulation results is tested in the SDG on the square lattice. Furthermore, we also discuss the effect of the initial proportion of the cooperators on the evolution of cooperation. Through numerical simulations, we demonstrate that this mechanism allows cooperation to prevail even if the temptation to defect is large, which is independent of the potential interaction networks and game models. More detailed results will be presented next. 2. Method We consider an evolutionary prisoner’s dilemma game, in which the players populate on the vertices of the adoptive networks. As a standard practice, the evolutionary PDG is characterized with the temptation to defect T = b (the highest payoff obtained by a defector when playing with a cooperator), the reward for mutual cooperation R=1, the punishment for mutual defection P=0 as well as the sucker’s payoff S=0 (the lowest payoff acquired by a cooperator when playing against a defector). It is mentionable that even if we choose a weak and simple PDG (namely, P = S=0), our conclusions are robust and can be concluded in the full parameterized space. For the SDG, we can simplify the model in the following way: let T=1+r, R=1, S=1-r and P=0, where r (0 < r ≤ 1) represents the cost-to-benefit ratio of mutual cooperation. In this work, we introduce reciprocal rewards in the 2-person evolutionary games (PDG and SDG). In the models, a cooperator with a cooperative neighbor will obtain an additional benefit. The extra income for the cooperator is proportional to the number of cooperative neighbors, i.e., the more the number of the cooperator’s cooperative neighbors are, the higher the additional profits of the cooperator are. That is to say, each cooperative neighbor corresponds to an additional profit β for the focal cooperator. The value of β is changed from 0.0 to 0.5 with the step of 0.1. While if the player chooses to defect, it will obtain no extra income. When β =0, the model is reduced to the traditional spatial PDG or SDG. Throughout this work, with respect to the interaction networks, we choose three types of network topologies: the regular square lattices with periodic boundary conditions, the Erdös–Rényi (ER) random graphs and the Barabási-Albert (BA) scale-free (SF) networks. The game is iterated forward in accordance with Monte Carlo simulation procedure in the following way. As initial conditions, each agent is initially designated either as a cooperator sx = C or a defector sx = D with equal probability. Then, at each time step, each individual x in the structured population plays with all its neighbors and gets a payoff Px by adding all the obtained payoffs. The fitness Fx of player x is thus calculated in the following expression:
Fx =
Px Px + nβ
sx =D, sx =C,
(1)
where n represents the number of the cooperator’s cooperative neighbors. Next, all the players synchronously update their strategies employing the finite populations analogue of replicator dynamics (to which simulation results converge in the limit of well-mixed populations) [38]. Besides, under replicator dynamics,
232
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
individuals always imitate the agents who have higher incomes than themselves, which is also reasonable in the real world. The focal player x picks up at random one of its neighbors, say y, who also obtains its fitness Fy in the same way, and compares the respective fitness Fx and Fy . If Fx > Fy , player x will keep its strategy for the next step. On the contrary, player x will copy the neighbor’s strategy if Fx < Fy with a probability proportional to the fitness difference:
Wy→x =
Fy − Fx , max{kx , ky }D
(2)
where kx and ky represent the degrees of players x and y, respectively, and D is the maximum possible payoff difference between two players. Note that from the equation, it is possible for each player to adopt one of their neighbors’ strategies once on average during one full Monte Carlo simulation (MCS). Finally, if these steps are completed, a full Monte Carlo step will be finished. The key quantity the fraction of cooperators ρ c is acquired by averaging the last 104 full MCS over the total 6 × 104 steps. To assure that the system has reached a stationary state, we analyze the size of the fluctuations in < ρ c > . If the size is smaller than 10−2 , we suppose that the stationary state has been reached. Otherwise, we wait for another 104 time-steps and redo the check. Actually, the system reaches the stationary state in all the simulations and no extra time-steps are needed. All the results are averaged over 40 independent rounds. For all the considered networks we set the same size (N=104 nodes) and the same average degree, i.e., < k > =4. 3. Results and analysis We start by examining the influence of the reward-driven strategy update mechanism on the sustainability of cooperation. Fig. 1 shows how the fraction of cooperators (ρ c ) at the stationary state varies in dependence on the temptation to defect b and the cost-to-benefit ratio r for different values of β . Panel (a) and panel (b) represent the evolutionary PDG and SDG on a square lattice with periodic boundary conditions, respectively. Each individual plays with its four nearest neighbors (Von Neumann neighborhood). A cooperator will gain an additional benefit β if there is a cooperator in its neighbours. Therefore, if the focal cooperator has n cooperative neighbors, it will obtain extra benefits nβ . To further investigate the effect of this mechanism on the evolution of cooperation, the value of β is varied from 0.0 to 0.5. In particular, when β =0.0, the model collapses into the traditional version and no additional excitation is imposed to the altruistic players, where the cooperators die out soon with the increase of b. When the reciprocal reward is introduced, the critical temptation value b = bc , which pinpoints the extinction of cooperators, gradually shifts right with the growth of β as shown in Fig. 1. It is not difficult to understand the improvement of the evolution of cooperation. The additional awards between cooperators raise the cooperators’ payoffs and then strength the advantages of the cooperation strategy during the strategy spread. The potential superiority of income drives individuals to be apt to choose cooperation rather than defection. Catalyzed by network reciprocity, the cooperators with cooperative neighbors can resist the aggression of the defectors by forming compact cooperative clusters. However, when the temptation to defect b gradually increases, the cooperators situated on the edge of the cooperative clusters are prone to be invaded by defectors, which ultimately results in the dissolution of cooperative clusters. The simulation results of the PDG and the SDG may imply that the enhancement mechanism is generally valid in promoting the sustainability of cooperation, irrespective of the potential evolutionary games.
For exploring and discussing the robustness of the introduced mechanism, the simulations of the evolutionary PDG on the ErdösRényi (ER) random graphs and the Barabási-Albert (BA) scalefree (SF) networks are conducted. The Monte Carlo results for the concentration of cooperators (ρ c ) versus the temptation to b are demonstrated in Fig. 2 (a) and Fig. 2 (b), which represent the ER and the BA network topologies, respectively. Compared to the standard case expressed in black square dot line, the evolution of cooperation is intuitively promoted. For instance, the critical value bc almost moves to 2.0 when β =0.1, suggesting that cooperators can emerge or even prevail within a large range of b values. Besides, the universality of the results has been tested in the SDG as well, as shown in Fig. 2 (c) and Fig. 2 (d), which depict the fraction of cooperators in dependence on the cost-to-benefit ratio r for the ER graphs and the BA scale-free topologies. The change trends of the cooperator frequency ρ c for different β s in the PDG and the SDG on the two network structures are along the same lines as those on the square lattice, in which the fraction of cooperators and the intensity of reciprocal rewards are positively correlated. The results for the PDG and the SDG in homogeneous graphs and heterogeneous networks may indicate that the reciprocal reward mechanism introduced here can be beneficial for sustaining cooperative behavior independent of the underlying evolutionary games and the structures of the applied interaction networks, and thus appears to be a general feature. In order to explore the potential mechanism for the promotion of cooperation, we record in Fig. 3 the temporal traits of the fraction of cooperation ρ c in the PDG on the square lattice for different values of the selection parameter β . The temptation to defect b is set to be 1.2 and the value of β is changed from 0.0 to 0.3. In the early stages of the evolutionary processes, the performance of defectors is better than that of cooperators, irrespective of the values of β . As a matter of fact, this is in accordance with what one would expect that defection is a more advantageous choice than cooperation and thus be chosen more likely as a potential strategy donor given that in mixed population. At the outset, although the cooperators attempt to form clusters, these clusters are minute and too disperse to resist the invasion of the powerful defectors. However, after this stage, the density of cooperators evolves differently. For the traditional case (β =0), as presented by the black solid line in Fig. 3, the decimation of cooperators cannot cease and yields the exclusive dominance of defectors. When the reciprocal reward is introduced, incentives of extra reward among cooperators are more conducive to the formation of cooperative clusters. For β =0.1, it is demonstrated that the spread of cooperators is amplified and the exploitation of defectors is effectively suppressed as shown by the red solid line. Intuitively, the concentration of cooperators in the evolutionary steady state is positively correlated with the value of β . Worth highlighting is that for β =0.3 cooperators completely dominate the population ultimately which is indicated in pink solid line. Great enhancement of cooperation is that, for large reciprocal reward β , cooperative agents located at the boundaries of clusters become imperious to defector aggression and even induce strategy changes of the weakened defectors (from defectors to cooperators), which eventually produces the undisputed dominance of cooperation. Based on these observations, it is suggested that the introduced reciprocal reward mechanism can significantly favor the emergence and maintenance of cooperation. To have further information about the effect of reciprocal reward, the spatial distributions of cooperators and defectors at the evolutionary stable state with four typical values of the parameter β (0.0, 0.1, 0.2 and 0.3) are demonstrated in Fig. 4. The PDG is still utilized as the basic model for the evolutionary game in this part. The results are acquired with b=1.2 and L × L=100 × 100. When β =0.0, the cooperative agents vanish finally, which conforms to what is expected in the traditional model as shown in panel
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
233
Fig. 1. Fraction of cooperators (ρ c ) for different values of reciprocal reward intensity β . Panel (a) and panel (b) represent the PDG and the SDG, respectively. All the results are acquired in the von Neumann neighborhood on a square lattice for N=104 nodes, MCS=60,0 0 0 and k=4.
Fig. 2. Fraction of cooperators (ρ c ) in the PDG (SDG) as a function of defect b (r) for different values of the reciprocal reward intensity β . The first column (panel (a) and panel (c)) and the second column (panel (b) and panel (d)) represent the ER graphs and the BA scale-free networks, respectively. All the results have been obtained for < k > =4. Other parameters are consistent with those in Fig. 1.
234
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
Fig. 3. Time courses of the fraction of cooperators ρ c for different values of the reciprocal reward β . All the results are obtained for b=1.2, L=100, MCS=60,000 and k=4.
(a) of Fig. 4. When β =0.1, numerous cooperators survive at the stationary state (the red and the blue represent the defectors and the cooperators, respectively). When the value of β continues to increase, cooperators start to mushroom. For β =0.2, as indicated in panel (c), the fraction of cooperation reaches ρ c =0.72, where the remaining defectors survive by forming ribbon-like segments among cooperative clusters. Especially, when β is increased up to 0.3, cooperators reach complete dominance, as shown in panel (d). The characteristic snapshots of the distribution of cooperators
and defectors visually display the evolutionary dynamics of the agents. The reciprocal reward mechanism helps the cooperators to avoid the full exploitation of defectors. The larger the β value is, the larger the cooperative clusters become. However, even if the cooperators prevail over a large interval of T for β =0.3, defectors start invading the cooperative clusters until they split into smaller parts as predicted in Fig. 1 for the higher values of b. For the sake of generality, we investigate the robustness of the mechanism under different initial fractions of cooperation fc for the PDG on the square lattice. The results are presented in Fig. 5, in which the temptation to defect b=1.2 and the strength of reciprocal reward β =0.2. It is observed from the graph that when we dilute the initial proportions of cooperators, for instance, fc =0.4, 0.2 or 0.1, the fractions of cooperative agents ρ c in the final evolutionary stable state are the same as the results of the initial cooperator ratio of 0.5. It is worth noting that when we fix fc =0.05, there is a divergence in the outcome of the evolution. As shown in the pink dotted line, in some scenarios, there are evolutionary results that are consistent with other initial conditions, and in some other cases, cooperation almost goes extinct. We can understand this phenomenon as follows. When the initial proportion of cooperation falls to a certain extent, the number of cooperative clusters formed in the initial phase of evolution is rare. Moreover, under this parameter configuration, the income including the reciprocal reward among cooperative agents is not dominant compared with the temptation to defect. During the evolution, which is carried out by implementing the finite population analogue of replicator dynamics, if the incompletely rational individuals make some mistakes, the few cooperative clusters are likely to disintegrate and eventually result in the failure of cooperation evolution. Theoretically, we can compensate for this evolutionary instability
Fig. 4. Characteristic snapshots of cooperators (blue) and defectors (red) for different values of the selection parameter β after 105 MCS. From panel (a) to panel (d), the values of β are 0.0, 0.1, 0.2, 0.3, respectively. The depicted results are obtained for b=1.2, k=4 and L=100. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
235
4. Conclusion
Fig. 5. Time courses of the fraction of cooperation ρ c in the PDG for different initial proportions of cooperators fc . It is obvious that the results obtained are consistent with those in Fig. 1. All the results are acquired for b=1.2, β =0.2 and L=100.
In conclusion, we have investigated the impact of reciprocal rewards on the evolution of cooperation in the spatial PDG and the SDG. The introduction of reciprocal rewards enables the cooperator with cooperative neighbors to obtain a fixed extra benefit, which is conducive to the formation of cooperative clusters. The simulation results show that the reciprocal reward mechanism favors the emergency and sustainability of cooperation, regardless of the underlying interaction networks. The network structures applied in the simulations include the square lattice with periodic boundary conditions, the ER graphs and the BA topologies. To explore the mechanism that drives the evolution of cooperation, the time courses of the density of cooperation under different reward intensity β are demonstrated. The simulation results indicate the positive correlation between the final density of the cooperation and the reciprocal reward intensity β . The snapshots of the distribution of cooperators and defectors imply that improving the value of reward intensity can be beneficial for accelerating microscopic organization of cooperative clusters, which become impervious to defector attacks even for a large value of b. Furthermore, the promoting effect is also attested to be robust to the initial fraction of cooperators. Since reciprocal altruistic behaviors are ubiquitous, we hope that this work might provide additional insights for understanding the roots of cooperation and can inspire more explorations for resolving the social dilemmas. Acknowledgments
Fig. 6. Time course of the fraction of cooperation ρ c in the PDG for the special initial state: only two cooperators in the centre. The insets of the graph represent the snapshots of the distribution of cooperators (blue) and defectors (red) for β =0.3 at 0, 40 0, 10 0 0, 20 0 0 MCS from panel (a) to panel (d). The temptation to defect b=1.2 and other parameters are the same as those in Fig. 5. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
by increasing the reciprocal reward intensity that reinforces the evolutionary advantage among cooperators. To verify our inferences above, we conduct the simulations for the PDG on the square lattice under more severe initial conditions. Specifically, there exist only two cooperators in the initial stages. The temptation to defect b is fixed to 1.2 and the reciprocal reward intensity β is set to 0.3. Therefore, under the mechanism of reciprocal reward, the total benefits (1+0.3) between two cooperators outweigh the temptation to defect b=1.2, which motivates more individuals to select cooperation strategy. The time course of the fraction of cooperation ρ c is presented in Fig. 6. We can intuitively notice that the final frequency of cooperation is 1.0, which is consistent with the result in Fig. 1 (a). On the other hand, the process of microscopic expansion of cooperator cluster could be observed explicitly from the insets of Fig. 6. Panels (a), (b), (c) and panel (d) represent the snapshots of the distribution of cooperators (blue) and defectors (red) for β =0.3 at 0, 400, 1000, 2000 MCS, respectively. It is worth noting that we have made 40 repetitions of the simulation, and no one result is inconsistent with our conclusion, which indirectly confirms our inferences above. This research verifies the robustness of this mechanism to variations of initial fraction of cooperators.
This project was supported in part by the Major Research Plan of the National Natural Science Foundation of China (91430108), the National Basic Research Program (2012CB955804), the National Natural Science Foundation of China (11771322, 71802143, 61705013), the Humanities and Social Sciences Research of the Ministry of Education of China (18YJC630260), Tianjin Philosophy and Social Science Planning Project (TJGLQN18-005), The Tianjin Natural Science Foundation (17JCYBJC43300) and the graduate research program (2016TCB05) of Tianjin University of Finance Economics. References [1] Requejo RJ, Camacho J. Evolution of cooperation mediated by limiting resources: connecting resource based models and evolutionary game theory. J Theoret Biol 2011;272(1):35–41. [2] Apicella CL, Marlowe FW, Fowler JH, et al. Social networks and cooperation in hunter-gatherers. Nature 2012;481(7382):497. [3] Wang Z, Jusup M, Wang RW, et al. Onymity promotes cooperation in social dilemma experiments. Sci Adv 2017;3(3):E1601444. [4] Smelser NJ. Theory of collective behavior. Quid Pro-Books; 2011. [5] Axelrod R, Hamilton WD. The evolution of cooperation. Science 1981;211(4489):1390–6. [6] Pennisi E. How did cooperative behavior evolve? Science 2005;309(5731):93. [7] Chazelle B. An algorithmic approach to collective behavior. J Stat Phys 2015;158(3):514–48. [8] Wang Z, Kokubo S, Jusup M, et al. Universal scaling for the dilemma strength in evolutionary games. Phys Life Rev 2015;14:1–30. [9] Smith JM. Evolution and the theory of games. Cambridge University Press; 1982. [10] Weibull JW. Evolutionary game theory. MIT Press; 1997. [11] Nowak MA. Evolutionary dynamics: exploring the equations of life. Harvard University Press; 2006. [12] Doebeli M, Hauert C. Models of cooperation based on the Prisoner’s Dilemma and the snowdrift game. Ecol Lett 2005;8(7):748–66. [13] Zhang J, Zhang C, Chu T, et al. Cooperation in networks where the learning environment differs from the interaction environment. PLoS ONE 2014;9(3):E90288. [14] Zhang J, Zhang C, Cao M, et al. Crucial role of strategy updating for coexistence of strategies in interaction networks. Phys Rev E 2015;91(4). 042101. [15] Perc M. Coherence resonance in a spatial prisoner’s dilemma game. New J Phys 2006;8(2):22. [16] Huang K, Zheng X, Li Z, et al. Understanding cooperative behavior based on the coevolution of game strategy and link weight. Sci Rep 2015;5:14783.
236
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
[17] Perc M, Wang Z. Heterogeneous aspirations promote cooperation in the Prisoner’s Dilemma game. PLoS ONE 2010;5(12):E15117. [18] Wang WX, Ren J, Chen G, et al. Memory-based snowdrift game on networks. Phys Rev E 2006;74(5):056113. [19] Boyd R. Mistakes allow evolutionary stability in the repeated Prisoner’s Dilemma game. J Theoret Biol 1989;136(1):47–56. [20] Wang Z, Bauch CT, Bhattacharyya S, et al. Statistical physics of vaccination. Phys Rep 2016;664:1–113. [21] Hofbauer J, Sigmund K. Evolutionary games and population dynamics. Cambridge University Press; 1998. [22] Perc M, Szolnoki A. Social diversity and promotion of cooperation in the spatial Prisoner’s Dilemma game. Phys Rev E 2008;77(1):011904. [23] Javarone MA. Statistical physics of the spatial Prisoner’s Dilemma with memory-aware agents. Euro Phys J B 2016;89(2):42. [24] Xia CY, Meng XK, Wang Z. Heterogeneous coupling between interdependent lattices promotes the cooperation in the Prisoner’s Dilemma game. PLoS ONE 2015;10(6):E0129542. [25] Santos FC, Pacheco JM, Lenaerts T. Evolutionary dynamics of social dilemmas in structured heterogeneous populations. Proc Natl Acad Sci 2006;103(9):3490–4. [26] Poncela J, Gómez-Gardeñes J, Traulsen A, et al. Evolutionary game dynamics in a growing structured population. New J Phys 2009;11(8):083031. [27] Yu Y, Xiao G, Zhou J, et al. System crash as dynamics of complex networks. Proc Natl Acad Sci 2016;113(42):11726–31. [28] Boccaletti S, Bianconi G, Criado R, et al. The structure and dynamics of multilayer networks. Phys Rep 2014;544(1):1–122. [29] Li X, Jusup M, Wang Z, et al. Punishment diminishes the benefits of network reciprocity in social dilemma experiments. Proc Natl Acad Sci 2018;115(1):30–5. [30] Deng WF, Huang K, Yang CH, et al. Promote of cooperation in networked multiagent system based on fitness control. Appl Math Comput 2018;339:805–11. [31] Nowak MA, May RM. Evolutionary games and spatial chaos. Nature 1992;359(6398):826–9. [32] Kerr B, Riley MA, Feldman MW, et al. Local dispersal promotes biodiversity in a real-life game of rock-paper-scissors. Nature 2002;418(6894):171–4. [33] Wang Z, Jusup M, Shi L, et al. Exploiting a cognitive bias promotes cooperation in social dilemma experiments. Nat Commun 2018;9:2954. [34] Qin SM, Chen Y, Zhao XY, et al. Effect of memory on the Prisoner’s Dilemma game in a square lattice. Phys Rev E 2008;78(1):041129. [35] Hadjichrysanthou C, Broom M, Kiss IZ. Approximating evolutionary dynamics on networks using a neighbourhood configuration model. J Theoret Biol 2012;312:13. (312C). [36] Masuda N, Aihara K. Spatial Prisoner’s Dilemma optimally played in small– world networks. Phys Lett A 2003;313(1):55–61.
[37] Zhong X. Cooperative behavior in n-person evolutionary snowdrift game on Barabási–Albert network with link rewiring mechanism. J Inf Comput Sci 2015;12(11):4519–29. [38] Santos FC, Pacheco JM. Scale-free networks provide a unifying framework for the emergence of cooperation. Phys Rev Lett 2005;95(9):098104. [39] Gómez-Gardeñes J, Campillo M, Floría LM, et al. Dynamical organization of cooperation in complex topologies. Phys Rev Lett 2007;98(10):108103. [40] Perc M. Evolution of cooperation on scale-free networks subject to error and attack. New J Phys 2009;11(3):033027. [41] Szolnoki A, Szabo G. Cooperation enhanced by inhomogeneous activity of teaching for evolutionary Prisoner’s Dilemma games. EPL 20 06;77(3):30 0 04. [42] Wu ZX, Xu XJ, Huang ZG, et al. Evolutionary Prisoner’s Dilemma game with dynamic preferential selection. Phys Rev E 2006;74(1):021107. [43] Lin H, Yang DP, Shuai JW. Cooperation among mobile individuals with payoff expectations in the spatial Prisoner’s Dilemma game. Chaos Sol Fract 2011;44(1):153–9. [44] Emmert-Streib F. Evolutionary dynamics of the spatial Prisoner’s Dilemma with self-inhibition. Appl Math Comput 2012;218(11):6482–8. [45] Xu B, Wang J, Deng R, et al. Relational diversity promotes cooperation in Prisoner’s Dilemma games. PLoS ONE 2014;9(12):E114464. [46] Huang K, Chen X, Yu Z, et al. Heterogeneous cooperative belief for social dilemma in multi-agent system. Appl Math Comput 2018;320:572–9. [47] Nowak MA. Five rules for the evolution of cooperation. Science 2006;314(5805):1560–3. [48] Wu Y, Chang S, Zhang Z, et al. Impact of social reward on the evolution of the cooperation behavior in complex networks. Sci Rep 2017;7:41076. [49] Wang XW, Nie S, Jiang LL, et al. Role of delay-based reward in the spatial cooperation. Physica A 2017;465:153–8. [50] Wu Y, Zhang Z, Chang S. Effect of self-interaction on the evolution of cooperation in complex topologies. Physica A 2017:481. [51] Sigmund K, Hauert C, Nowak MA. Reward and punishment. Proc Natl Acad Sci 2001;98(19):10757–62. [52] Rand DG, Dreber A, Ellingsen T, et al. Positive interactions promote public cooperation. Science 2009;325(5945):1272–5. [53] Wang Z, Xia CY, Meloni S, et al. Impact of social punishment on cooperative behavior in complex networks. Sci Rep 2013;3:3055. [54] Szolnoki A, Perc M. Reward and cooperation in the spatial public goods game. EPL 2010;92(3):38003. [55] Choi JK, Ahn TK. Strategic reward and altruistic punishment support cooperation in a public goods game experiment. J Econ Psychol 2013;35:17–30. [56] Ding CX, Wang J, Zhang Y. Impact of self interaction on the evolution of cooperation in social spatial dilemmas. Physica A 2016;91:393–9.
Contents lists available at ScienceDirect
Chaos, Solitons and Fractals Nonlinear Science, and Nonequilibrium and Complex Phenomena journal homepage: www.elsevier.com/locate/chaos
Reciprocal reward promotes the evolution of cooperation in structured populations Yu’e Wu, Zhipeng Zhang, Shuhua Chang∗ Coordinated Innovation Center for Computable Modeling in Management Science, Tianjin University of Finance and Economics, Tianjin 300222, China
a r t i c l e
i n f o
Article history: Received 19 June 2018 Revised 25 September 2018 Accepted 6 January 2019
Keywords: Prisoner’s dilemma Reciprocal reward Cooperation Complex network
a b s t r a c t Reciprocal reward, as a common phenomenon in nature and society, plays a pivotal role in interpreting the existence and maintenance of cooperation. In this work, we introduce a reciprocal reward mechanism in the two pairwise models including the prisoner’s dilemma and the snowdrift games, where a cooperative agent with cooperative neighbors will obtain additional bonuses. Under this mechanism, we find that the evolution of cooperation is promoted to a high level. The promoting effects for the demonstrated results are independent of the applied spatial network structures and the potential evolutionary game models, and thus showing a high degree of generality. In addition, we also probe into the impact of the initial fractions of cooperators on the evolution of cooperation under this mechanism, the results of which further confirm the robustness of this mechanism. Our research may provide valuable insights into interpreting the emergence and sustainability of cooperation and thus are beneficial for resolving social dilemmas. © 2019 Published by Elsevier Ltd.
1. Introduction Cooperative behavior exists widely in nature and human society, which is contradict with Darwin’s theory of evolution [1–3]. Therefore, how to understand and explain the emergence and sustainability of cooperation among unrelated individuals becomes one of the major challenges of evolutionary biology and of behavioral sciences, thus attracting the attentions from a myriad of fields including physics, mathematics, biology and behavioral sciences [4–8]. Evolutionary game theory provides a forceful framework to elucidate this puzzle, in which social contradictions are analogous to the competition among peers for limited resources [9–11]. In particular, the prisoner’s dilemma game (PDG) and the snowdrift game (SDG) are widely applicable games to explore the evolution of cooperation [12–14]. In the traditional evolutionary PDG and SDG, two involved players can choose either to cooperate or to defect in the processes of the game [15–17]. A player who chooses cooperative strategy will obtain R when confronting a cooperator and receive S (the sucker’s payoff) when meeting a defector. Meanwhile, a defector will acquire T (P) when the opponent’s strategy is cooperation (defection). As a standard practice, the payoffs are ordered as T > R > P ≥ S in the PDG, suggesting that the defection is the
∗
Corresponding author. E-mail address: [email protected] (S. Chang).
https://doi.org/10.1016/j.chaos.2019.01.006 0960-0779/© 2019 Published by Elsevier Ltd.
optimal strategy irrespective of the opponent’s decision. While for the SDG, the payoff ranking is T > R > S > P. The minor distinction of the payoff ranking induces a significant change in the game dynamics where the optimal strategy for the individual is the opposite of the opponent’s [18–20]. Traditionally, the evolutionary games are investigated in an infinite, well-mixed population (complete graphs), where all individuals interact equally likely with each other, but cooperation cannot emerge under replicator dynamics [21]. Actually, fully-connected graphs constitute rather unrealistic representations of real-world network of contacts, in which one expects local connections (spatial structure) to coexist with long-range connections, traits recently identified as characteristics of numerous of natural, social, and technological network of contacts. Naturally, considerable attention has been shifted into structured populations, or say, people focus on studying evolutionary games on the networks [22–30]. Evolutionary graph theory provides a natural and reasonable framework for this approach: the vertices in the network topology represent players and the edges indicate links between players in terms of game dynamical interaction. The agents located on the vertices are constrained to play with their immediate neighbors along the edges. Taking these simplifying settings into consideration, Nowak and May seminally introduced the spatial structure into the PDG, where the agents were located on the square lattices [31]. They found that cooperators can build clusters via such spatial structure to resist aggression by defectors. One decade later, laboratory experiments [29,32,33] have confirmed that
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
topological constraints indeed influence the evolution of cooperation in a sizeable way. Following the pioneering work, the evolution of cooperation has been extensively explored in a variety of topologies such as regular square lattices [34], Erdös-Rényi (ER) random graphs [35], small-world networks [36], Barabási-Albert (BA) scale-free (SF) network [37] and so on. In particular, Santos et al. indicated that scale-free networks can provide a unifying framework for the appearance of cooperation [38]. The main driving force behind the flourishing cooperative behavior in scale-free networks is attributed to the strong heterogeneity of the degree distribution [39]. Besides, Perk found that cooperative behavior on scale-free networks is extremely robust against random deletion of vertices but is obstructed greatly if vertices with the maximal degree are targeted, which further confirm the significance of the degree distribution heterogeneity for the evolution of cooperation by random and intentional removal of vertices [40]. Furthermore, in the framework of graph-structured populations, a myriad of scenarios have been proposed to offset the unfavorable evolutionary outcome in the traditional games. For instance, inhomogeneous teaching activity [41], preferential selection [42], individuals’ mobility [43], different evolutionary dynamics [44], high value of the clustering coefficient and social diversity [45,46]. These examples of alternative ways have been considered as potential promoters of cooperation with fruitful success, and can be reasonable from the perspective of real society. Nowak attributed all these to five mechanisms: kin selection, direct reciprocity, indirect reciprocity, network reciprocity, and group selection [47]. Of particular renown, the influence of reward on the evolution of cooperation has attracted great attention [48–50]. Reward is an established alternative to punishment [51], albeit investigated less frequent in the past. Compared to punishment, reward incorporates a cost to bear as well, but for another individual to experience a benefit. As a matter of fact, reward may be as effective as punishment for maintaining cooperation [52,53]. In vast majority of previous studies, most of the authors focus their attentions on the cooperators alone and overlook the synergistic effect between the cooperative agent and its neighbors in the pairwise models. In realistic scenarios, mutual cooperation that makes both the two agents obtain additional benefits is a very common phenomenon, such as mutualism in nature. In human society, this phenomenon is also widespread and environmental governance is one of the most typical examples. For example, if two countries (agents) share a river, each country will benefit from its own river environmental governance (cooperation). However, if two neighboring countries cooperate, each country gains additional benefits (for example, better living conditions). When many countries are distributing along the river, as our model will describe, the more cooperative neighbors an individual has, the more additional benefits the individual receives. Moreover, in most of these reward-related articles, cooperators receive rewards as a second-stage behavior and the game models are almost limited to public goods games [54,55]. Inspired by these findings, we here study the impact of rewards on the evolution of cooperation in the structured population by means of direct reciprocity (reciprocal reward). The reciprocal reward here means that a cooperator with a cooperative neighbor will receive an extra benefit. In this scenario, cooperative behavior is not only altruistic but it is very likely to be beneficial to oneself. It is worth mentioning that this is similar to a previous work [56], which is about the impact of self-interaction on the evolution of cooperation. The authors in the reference focus on self-interaction of the agents and the cooperators can obtain additional benefits through self-interaction. Although both are cooperators who receive additional rewards, we focus on reciprocal rewards generated by the interaction between the cooperators and their cooperative neighbors. In our model, the additional rewards that a cooperator receives are not only influenced by its own strat-
231
egy, but also by the number of cooperators in its neighborhood, which is quite different from the former. Besides, it focuses on the influence of the distribution of self-interaction strength on the evolution of cooperation in the reference, while our model emphasizes the robustness of the promoting effect of cooperation to different network structures and initial fractions of cooperators. It is known that many real networks have scale-free properties and, therefore, our work has important implications to promote the application of this mechanism in real life. The simulations are conducted in the spatial PDG on three types of network structures including regular square lattices with periodic boundary conditions, Erdös-Rényi (ER) random graphs and Barabási-Albert (BA) scale-free (SF) networks. The robustness of the simulation results is tested in the SDG on the square lattice. Furthermore, we also discuss the effect of the initial proportion of the cooperators on the evolution of cooperation. Through numerical simulations, we demonstrate that this mechanism allows cooperation to prevail even if the temptation to defect is large, which is independent of the potential interaction networks and game models. More detailed results will be presented next. 2. Method We consider an evolutionary prisoner’s dilemma game, in which the players populate on the vertices of the adoptive networks. As a standard practice, the evolutionary PDG is characterized with the temptation to defect T = b (the highest payoff obtained by a defector when playing with a cooperator), the reward for mutual cooperation R=1, the punishment for mutual defection P=0 as well as the sucker’s payoff S=0 (the lowest payoff acquired by a cooperator when playing against a defector). It is mentionable that even if we choose a weak and simple PDG (namely, P = S=0), our conclusions are robust and can be concluded in the full parameterized space. For the SDG, we can simplify the model in the following way: let T=1+r, R=1, S=1-r and P=0, where r (0 < r ≤ 1) represents the cost-to-benefit ratio of mutual cooperation. In this work, we introduce reciprocal rewards in the 2-person evolutionary games (PDG and SDG). In the models, a cooperator with a cooperative neighbor will obtain an additional benefit. The extra income for the cooperator is proportional to the number of cooperative neighbors, i.e., the more the number of the cooperator’s cooperative neighbors are, the higher the additional profits of the cooperator are. That is to say, each cooperative neighbor corresponds to an additional profit β for the focal cooperator. The value of β is changed from 0.0 to 0.5 with the step of 0.1. While if the player chooses to defect, it will obtain no extra income. When β =0, the model is reduced to the traditional spatial PDG or SDG. Throughout this work, with respect to the interaction networks, we choose three types of network topologies: the regular square lattices with periodic boundary conditions, the Erdös–Rényi (ER) random graphs and the Barabási-Albert (BA) scale-free (SF) networks. The game is iterated forward in accordance with Monte Carlo simulation procedure in the following way. As initial conditions, each agent is initially designated either as a cooperator sx = C or a defector sx = D with equal probability. Then, at each time step, each individual x in the structured population plays with all its neighbors and gets a payoff Px by adding all the obtained payoffs. The fitness Fx of player x is thus calculated in the following expression:
Fx =
Px Px + nβ
sx =D, sx =C,
(1)
where n represents the number of the cooperator’s cooperative neighbors. Next, all the players synchronously update their strategies employing the finite populations analogue of replicator dynamics (to which simulation results converge in the limit of well-mixed populations) [38]. Besides, under replicator dynamics,
232
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
individuals always imitate the agents who have higher incomes than themselves, which is also reasonable in the real world. The focal player x picks up at random one of its neighbors, say y, who also obtains its fitness Fy in the same way, and compares the respective fitness Fx and Fy . If Fx > Fy , player x will keep its strategy for the next step. On the contrary, player x will copy the neighbor’s strategy if Fx < Fy with a probability proportional to the fitness difference:
Wy→x =
Fy − Fx , max{kx , ky }D
(2)
where kx and ky represent the degrees of players x and y, respectively, and D is the maximum possible payoff difference between two players. Note that from the equation, it is possible for each player to adopt one of their neighbors’ strategies once on average during one full Monte Carlo simulation (MCS). Finally, if these steps are completed, a full Monte Carlo step will be finished. The key quantity the fraction of cooperators ρ c is acquired by averaging the last 104 full MCS over the total 6 × 104 steps. To assure that the system has reached a stationary state, we analyze the size of the fluctuations in < ρ c > . If the size is smaller than 10−2 , we suppose that the stationary state has been reached. Otherwise, we wait for another 104 time-steps and redo the check. Actually, the system reaches the stationary state in all the simulations and no extra time-steps are needed. All the results are averaged over 40 independent rounds. For all the considered networks we set the same size (N=104 nodes) and the same average degree, i.e., < k > =4. 3. Results and analysis We start by examining the influence of the reward-driven strategy update mechanism on the sustainability of cooperation. Fig. 1 shows how the fraction of cooperators (ρ c ) at the stationary state varies in dependence on the temptation to defect b and the cost-to-benefit ratio r for different values of β . Panel (a) and panel (b) represent the evolutionary PDG and SDG on a square lattice with periodic boundary conditions, respectively. Each individual plays with its four nearest neighbors (Von Neumann neighborhood). A cooperator will gain an additional benefit β if there is a cooperator in its neighbours. Therefore, if the focal cooperator has n cooperative neighbors, it will obtain extra benefits nβ . To further investigate the effect of this mechanism on the evolution of cooperation, the value of β is varied from 0.0 to 0.5. In particular, when β =0.0, the model collapses into the traditional version and no additional excitation is imposed to the altruistic players, where the cooperators die out soon with the increase of b. When the reciprocal reward is introduced, the critical temptation value b = bc , which pinpoints the extinction of cooperators, gradually shifts right with the growth of β as shown in Fig. 1. It is not difficult to understand the improvement of the evolution of cooperation. The additional awards between cooperators raise the cooperators’ payoffs and then strength the advantages of the cooperation strategy during the strategy spread. The potential superiority of income drives individuals to be apt to choose cooperation rather than defection. Catalyzed by network reciprocity, the cooperators with cooperative neighbors can resist the aggression of the defectors by forming compact cooperative clusters. However, when the temptation to defect b gradually increases, the cooperators situated on the edge of the cooperative clusters are prone to be invaded by defectors, which ultimately results in the dissolution of cooperative clusters. The simulation results of the PDG and the SDG may imply that the enhancement mechanism is generally valid in promoting the sustainability of cooperation, irrespective of the potential evolutionary games.
For exploring and discussing the robustness of the introduced mechanism, the simulations of the evolutionary PDG on the ErdösRényi (ER) random graphs and the Barabási-Albert (BA) scalefree (SF) networks are conducted. The Monte Carlo results for the concentration of cooperators (ρ c ) versus the temptation to b are demonstrated in Fig. 2 (a) and Fig. 2 (b), which represent the ER and the BA network topologies, respectively. Compared to the standard case expressed in black square dot line, the evolution of cooperation is intuitively promoted. For instance, the critical value bc almost moves to 2.0 when β =0.1, suggesting that cooperators can emerge or even prevail within a large range of b values. Besides, the universality of the results has been tested in the SDG as well, as shown in Fig. 2 (c) and Fig. 2 (d), which depict the fraction of cooperators in dependence on the cost-to-benefit ratio r for the ER graphs and the BA scale-free topologies. The change trends of the cooperator frequency ρ c for different β s in the PDG and the SDG on the two network structures are along the same lines as those on the square lattice, in which the fraction of cooperators and the intensity of reciprocal rewards are positively correlated. The results for the PDG and the SDG in homogeneous graphs and heterogeneous networks may indicate that the reciprocal reward mechanism introduced here can be beneficial for sustaining cooperative behavior independent of the underlying evolutionary games and the structures of the applied interaction networks, and thus appears to be a general feature. In order to explore the potential mechanism for the promotion of cooperation, we record in Fig. 3 the temporal traits of the fraction of cooperation ρ c in the PDG on the square lattice for different values of the selection parameter β . The temptation to defect b is set to be 1.2 and the value of β is changed from 0.0 to 0.3. In the early stages of the evolutionary processes, the performance of defectors is better than that of cooperators, irrespective of the values of β . As a matter of fact, this is in accordance with what one would expect that defection is a more advantageous choice than cooperation and thus be chosen more likely as a potential strategy donor given that in mixed population. At the outset, although the cooperators attempt to form clusters, these clusters are minute and too disperse to resist the invasion of the powerful defectors. However, after this stage, the density of cooperators evolves differently. For the traditional case (β =0), as presented by the black solid line in Fig. 3, the decimation of cooperators cannot cease and yields the exclusive dominance of defectors. When the reciprocal reward is introduced, incentives of extra reward among cooperators are more conducive to the formation of cooperative clusters. For β =0.1, it is demonstrated that the spread of cooperators is amplified and the exploitation of defectors is effectively suppressed as shown by the red solid line. Intuitively, the concentration of cooperators in the evolutionary steady state is positively correlated with the value of β . Worth highlighting is that for β =0.3 cooperators completely dominate the population ultimately which is indicated in pink solid line. Great enhancement of cooperation is that, for large reciprocal reward β , cooperative agents located at the boundaries of clusters become imperious to defector aggression and even induce strategy changes of the weakened defectors (from defectors to cooperators), which eventually produces the undisputed dominance of cooperation. Based on these observations, it is suggested that the introduced reciprocal reward mechanism can significantly favor the emergence and maintenance of cooperation. To have further information about the effect of reciprocal reward, the spatial distributions of cooperators and defectors at the evolutionary stable state with four typical values of the parameter β (0.0, 0.1, 0.2 and 0.3) are demonstrated in Fig. 4. The PDG is still utilized as the basic model for the evolutionary game in this part. The results are acquired with b=1.2 and L × L=100 × 100. When β =0.0, the cooperative agents vanish finally, which conforms to what is expected in the traditional model as shown in panel
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
233
Fig. 1. Fraction of cooperators (ρ c ) for different values of reciprocal reward intensity β . Panel (a) and panel (b) represent the PDG and the SDG, respectively. All the results are acquired in the von Neumann neighborhood on a square lattice for N=104 nodes, MCS=60,0 0 0 and k=4.
Fig. 2. Fraction of cooperators (ρ c ) in the PDG (SDG) as a function of defect b (r) for different values of the reciprocal reward intensity β . The first column (panel (a) and panel (c)) and the second column (panel (b) and panel (d)) represent the ER graphs and the BA scale-free networks, respectively. All the results have been obtained for < k > =4. Other parameters are consistent with those in Fig. 1.
234
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
Fig. 3. Time courses of the fraction of cooperators ρ c for different values of the reciprocal reward β . All the results are obtained for b=1.2, L=100, MCS=60,000 and k=4.
(a) of Fig. 4. When β =0.1, numerous cooperators survive at the stationary state (the red and the blue represent the defectors and the cooperators, respectively). When the value of β continues to increase, cooperators start to mushroom. For β =0.2, as indicated in panel (c), the fraction of cooperation reaches ρ c =0.72, where the remaining defectors survive by forming ribbon-like segments among cooperative clusters. Especially, when β is increased up to 0.3, cooperators reach complete dominance, as shown in panel (d). The characteristic snapshots of the distribution of cooperators
and defectors visually display the evolutionary dynamics of the agents. The reciprocal reward mechanism helps the cooperators to avoid the full exploitation of defectors. The larger the β value is, the larger the cooperative clusters become. However, even if the cooperators prevail over a large interval of T for β =0.3, defectors start invading the cooperative clusters until they split into smaller parts as predicted in Fig. 1 for the higher values of b. For the sake of generality, we investigate the robustness of the mechanism under different initial fractions of cooperation fc for the PDG on the square lattice. The results are presented in Fig. 5, in which the temptation to defect b=1.2 and the strength of reciprocal reward β =0.2. It is observed from the graph that when we dilute the initial proportions of cooperators, for instance, fc =0.4, 0.2 or 0.1, the fractions of cooperative agents ρ c in the final evolutionary stable state are the same as the results of the initial cooperator ratio of 0.5. It is worth noting that when we fix fc =0.05, there is a divergence in the outcome of the evolution. As shown in the pink dotted line, in some scenarios, there are evolutionary results that are consistent with other initial conditions, and in some other cases, cooperation almost goes extinct. We can understand this phenomenon as follows. When the initial proportion of cooperation falls to a certain extent, the number of cooperative clusters formed in the initial phase of evolution is rare. Moreover, under this parameter configuration, the income including the reciprocal reward among cooperative agents is not dominant compared with the temptation to defect. During the evolution, which is carried out by implementing the finite population analogue of replicator dynamics, if the incompletely rational individuals make some mistakes, the few cooperative clusters are likely to disintegrate and eventually result in the failure of cooperation evolution. Theoretically, we can compensate for this evolutionary instability
Fig. 4. Characteristic snapshots of cooperators (blue) and defectors (red) for different values of the selection parameter β after 105 MCS. From panel (a) to panel (d), the values of β are 0.0, 0.1, 0.2, 0.3, respectively. The depicted results are obtained for b=1.2, k=4 and L=100. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
235
4. Conclusion
Fig. 5. Time courses of the fraction of cooperation ρ c in the PDG for different initial proportions of cooperators fc . It is obvious that the results obtained are consistent with those in Fig. 1. All the results are acquired for b=1.2, β =0.2 and L=100.
In conclusion, we have investigated the impact of reciprocal rewards on the evolution of cooperation in the spatial PDG and the SDG. The introduction of reciprocal rewards enables the cooperator with cooperative neighbors to obtain a fixed extra benefit, which is conducive to the formation of cooperative clusters. The simulation results show that the reciprocal reward mechanism favors the emergency and sustainability of cooperation, regardless of the underlying interaction networks. The network structures applied in the simulations include the square lattice with periodic boundary conditions, the ER graphs and the BA topologies. To explore the mechanism that drives the evolution of cooperation, the time courses of the density of cooperation under different reward intensity β are demonstrated. The simulation results indicate the positive correlation between the final density of the cooperation and the reciprocal reward intensity β . The snapshots of the distribution of cooperators and defectors imply that improving the value of reward intensity can be beneficial for accelerating microscopic organization of cooperative clusters, which become impervious to defector attacks even for a large value of b. Furthermore, the promoting effect is also attested to be robust to the initial fraction of cooperators. Since reciprocal altruistic behaviors are ubiquitous, we hope that this work might provide additional insights for understanding the roots of cooperation and can inspire more explorations for resolving the social dilemmas. Acknowledgments
Fig. 6. Time course of the fraction of cooperation ρ c in the PDG for the special initial state: only two cooperators in the centre. The insets of the graph represent the snapshots of the distribution of cooperators (blue) and defectors (red) for β =0.3 at 0, 40 0, 10 0 0, 20 0 0 MCS from panel (a) to panel (d). The temptation to defect b=1.2 and other parameters are the same as those in Fig. 5. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
by increasing the reciprocal reward intensity that reinforces the evolutionary advantage among cooperators. To verify our inferences above, we conduct the simulations for the PDG on the square lattice under more severe initial conditions. Specifically, there exist only two cooperators in the initial stages. The temptation to defect b is fixed to 1.2 and the reciprocal reward intensity β is set to 0.3. Therefore, under the mechanism of reciprocal reward, the total benefits (1+0.3) between two cooperators outweigh the temptation to defect b=1.2, which motivates more individuals to select cooperation strategy. The time course of the fraction of cooperation ρ c is presented in Fig. 6. We can intuitively notice that the final frequency of cooperation is 1.0, which is consistent with the result in Fig. 1 (a). On the other hand, the process of microscopic expansion of cooperator cluster could be observed explicitly from the insets of Fig. 6. Panels (a), (b), (c) and panel (d) represent the snapshots of the distribution of cooperators (blue) and defectors (red) for β =0.3 at 0, 400, 1000, 2000 MCS, respectively. It is worth noting that we have made 40 repetitions of the simulation, and no one result is inconsistent with our conclusion, which indirectly confirms our inferences above. This research verifies the robustness of this mechanism to variations of initial fraction of cooperators.
This project was supported in part by the Major Research Plan of the National Natural Science Foundation of China (91430108), the National Basic Research Program (2012CB955804), the National Natural Science Foundation of China (11771322, 71802143, 61705013), the Humanities and Social Sciences Research of the Ministry of Education of China (18YJC630260), Tianjin Philosophy and Social Science Planning Project (TJGLQN18-005), The Tianjin Natural Science Foundation (17JCYBJC43300) and the graduate research program (2016TCB05) of Tianjin University of Finance Economics. References [1] Requejo RJ, Camacho J. Evolution of cooperation mediated by limiting resources: connecting resource based models and evolutionary game theory. J Theoret Biol 2011;272(1):35–41. [2] Apicella CL, Marlowe FW, Fowler JH, et al. Social networks and cooperation in hunter-gatherers. Nature 2012;481(7382):497. [3] Wang Z, Jusup M, Wang RW, et al. Onymity promotes cooperation in social dilemma experiments. Sci Adv 2017;3(3):E1601444. [4] Smelser NJ. Theory of collective behavior. Quid Pro-Books; 2011. [5] Axelrod R, Hamilton WD. The evolution of cooperation. Science 1981;211(4489):1390–6. [6] Pennisi E. How did cooperative behavior evolve? Science 2005;309(5731):93. [7] Chazelle B. An algorithmic approach to collective behavior. J Stat Phys 2015;158(3):514–48. [8] Wang Z, Kokubo S, Jusup M, et al. Universal scaling for the dilemma strength in evolutionary games. Phys Life Rev 2015;14:1–30. [9] Smith JM. Evolution and the theory of games. Cambridge University Press; 1982. [10] Weibull JW. Evolutionary game theory. MIT Press; 1997. [11] Nowak MA. Evolutionary dynamics: exploring the equations of life. Harvard University Press; 2006. [12] Doebeli M, Hauert C. Models of cooperation based on the Prisoner’s Dilemma and the snowdrift game. Ecol Lett 2005;8(7):748–66. [13] Zhang J, Zhang C, Chu T, et al. Cooperation in networks where the learning environment differs from the interaction environment. PLoS ONE 2014;9(3):E90288. [14] Zhang J, Zhang C, Cao M, et al. Crucial role of strategy updating for coexistence of strategies in interaction networks. Phys Rev E 2015;91(4). 042101. [15] Perc M. Coherence resonance in a spatial prisoner’s dilemma game. New J Phys 2006;8(2):22. [16] Huang K, Zheng X, Li Z, et al. Understanding cooperative behavior based on the coevolution of game strategy and link weight. Sci Rep 2015;5:14783.
236
Y. Wu, Z. Zhang and S. Chang / Chaos, Solitons and Fractals 119 (2019) 230–236
[17] Perc M, Wang Z. Heterogeneous aspirations promote cooperation in the Prisoner’s Dilemma game. PLoS ONE 2010;5(12):E15117. [18] Wang WX, Ren J, Chen G, et al. Memory-based snowdrift game on networks. Phys Rev E 2006;74(5):056113. [19] Boyd R. Mistakes allow evolutionary stability in the repeated Prisoner’s Dilemma game. J Theoret Biol 1989;136(1):47–56. [20] Wang Z, Bauch CT, Bhattacharyya S, et al. Statistical physics of vaccination. Phys Rep 2016;664:1–113. [21] Hofbauer J, Sigmund K. Evolutionary games and population dynamics. Cambridge University Press; 1998. [22] Perc M, Szolnoki A. Social diversity and promotion of cooperation in the spatial Prisoner’s Dilemma game. Phys Rev E 2008;77(1):011904. [23] Javarone MA. Statistical physics of the spatial Prisoner’s Dilemma with memory-aware agents. Euro Phys J B 2016;89(2):42. [24] Xia CY, Meng XK, Wang Z. Heterogeneous coupling between interdependent lattices promotes the cooperation in the Prisoner’s Dilemma game. PLoS ONE 2015;10(6):E0129542. [25] Santos FC, Pacheco JM, Lenaerts T. Evolutionary dynamics of social dilemmas in structured heterogeneous populations. Proc Natl Acad Sci 2006;103(9):3490–4. [26] Poncela J, Gómez-Gardeñes J, Traulsen A, et al. Evolutionary game dynamics in a growing structured population. New J Phys 2009;11(8):083031. [27] Yu Y, Xiao G, Zhou J, et al. System crash as dynamics of complex networks. Proc Natl Acad Sci 2016;113(42):11726–31. [28] Boccaletti S, Bianconi G, Criado R, et al. The structure and dynamics of multilayer networks. Phys Rep 2014;544(1):1–122. [29] Li X, Jusup M, Wang Z, et al. Punishment diminishes the benefits of network reciprocity in social dilemma experiments. Proc Natl Acad Sci 2018;115(1):30–5. [30] Deng WF, Huang K, Yang CH, et al. Promote of cooperation in networked multiagent system based on fitness control. Appl Math Comput 2018;339:805–11. [31] Nowak MA, May RM. Evolutionary games and spatial chaos. Nature 1992;359(6398):826–9. [32] Kerr B, Riley MA, Feldman MW, et al. Local dispersal promotes biodiversity in a real-life game of rock-paper-scissors. Nature 2002;418(6894):171–4. [33] Wang Z, Jusup M, Shi L, et al. Exploiting a cognitive bias promotes cooperation in social dilemma experiments. Nat Commun 2018;9:2954. [34] Qin SM, Chen Y, Zhao XY, et al. Effect of memory on the Prisoner’s Dilemma game in a square lattice. Phys Rev E 2008;78(1):041129. [35] Hadjichrysanthou C, Broom M, Kiss IZ. Approximating evolutionary dynamics on networks using a neighbourhood configuration model. J Theoret Biol 2012;312:13. (312C). [36] Masuda N, Aihara K. Spatial Prisoner’s Dilemma optimally played in small– world networks. Phys Lett A 2003;313(1):55–61.
[37] Zhong X. Cooperative behavior in n-person evolutionary snowdrift game on Barabási–Albert network with link rewiring mechanism. J Inf Comput Sci 2015;12(11):4519–29. [38] Santos FC, Pacheco JM. Scale-free networks provide a unifying framework for the emergence of cooperation. Phys Rev Lett 2005;95(9):098104. [39] Gómez-Gardeñes J, Campillo M, Floría LM, et al. Dynamical organization of cooperation in complex topologies. Phys Rev Lett 2007;98(10):108103. [40] Perc M. Evolution of cooperation on scale-free networks subject to error and attack. New J Phys 2009;11(3):033027. [41] Szolnoki A, Szabo G. Cooperation enhanced by inhomogeneous activity of teaching for evolutionary Prisoner’s Dilemma games. EPL 20 06;77(3):30 0 04. [42] Wu ZX, Xu XJ, Huang ZG, et al. Evolutionary Prisoner’s Dilemma game with dynamic preferential selection. Phys Rev E 2006;74(1):021107. [43] Lin H, Yang DP, Shuai JW. Cooperation among mobile individuals with payoff expectations in the spatial Prisoner’s Dilemma game. Chaos Sol Fract 2011;44(1):153–9. [44] Emmert-Streib F. Evolutionary dynamics of the spatial Prisoner’s Dilemma with self-inhibition. Appl Math Comput 2012;218(11):6482–8. [45] Xu B, Wang J, Deng R, et al. Relational diversity promotes cooperation in Prisoner’s Dilemma games. PLoS ONE 2014;9(12):E114464. [46] Huang K, Chen X, Yu Z, et al. Heterogeneous cooperative belief for social dilemma in multi-agent system. Appl Math Comput 2018;320:572–9. [47] Nowak MA. Five rules for the evolution of cooperation. Science 2006;314(5805):1560–3. [48] Wu Y, Chang S, Zhang Z, et al. Impact of social reward on the evolution of the cooperation behavior in complex networks. Sci Rep 2017;7:41076. [49] Wang XW, Nie S, Jiang LL, et al. Role of delay-based reward in the spatial cooperation. Physica A 2017;465:153–8. [50] Wu Y, Zhang Z, Chang S. Effect of self-interaction on the evolution of cooperation in complex topologies. Physica A 2017:481. [51] Sigmund K, Hauert C, Nowak MA. Reward and punishment. Proc Natl Acad Sci 2001;98(19):10757–62. [52] Rand DG, Dreber A, Ellingsen T, et al. Positive interactions promote public cooperation. Science 2009;325(5945):1272–5. [53] Wang Z, Xia CY, Meloni S, et al. Impact of social punishment on cooperative behavior in complex networks. Sci Rep 2013;3:3055. [54] Szolnoki A, Perc M. Reward and cooperation in the spatial public goods game. EPL 2010;92(3):38003. [55] Choi JK, Ahn TK. Strategic reward and altruistic punishment support cooperation in a public goods game experiment. J Econ Psychol 2013;35:17–30. [56] Ding CX, Wang J, Zhang Y. Impact of self interaction on the evolution of cooperation in social spatial dilemmas. Physica A 2016;91:393–9.