- Email: [email protected]
Conditional neutral punishment promotes cooperation in the spatial prisoner's dilemma game
Applied Mathematics and Computation 368 (2020) 124798
Contents lists available at ScienceDirect
Applied Mathematics and Computation journal homepage: www.elsevier.com/locate/amc
Conditional neutral punishment promotes cooperation in the spatial prisoner’s dilemma game Qun Song a, Zhaoheng Cao b,c, Rui Tao b,c, Wei Jiang d, Chen Liu e,∗, Jinzhuo Liu b,c,∗∗ a
School of Automation, Northwestern Polytechnical University, Xi’an 710072, China Key Laboratory in Software Engineering of Yunnan Province, Kunming 650091, China c School of Software, Yunnan University, Kunming, Yunnan 650504, China d Unit 78102 of the Chinese people’s liberation army, Chengdu 610000, China e Center for Ecology and Environmental Sciences, Northwestern Polytechnical University, Xi’an 710072, China b
a r t i c l e
i n f o
Article history: Received 17 July 2019 Revised 20 September 2019 Accepted 30 September 2019
Keywords: Conditional punishment Strategy-neutral punishment Evolutionary games Social dilemmas Cooperation
a b s t r a c t Punishment plays an important role in promoting cooperation. In real society, individuals tend to punish other players based on certain conditions rather than punish them directly. Thus, we introduce a conditional neutral punishment mechanism and study how this mechanism affects the evolution of cooperation. Namely, an individual can punish his/her neighbors with the opposite strategy when his/her payoff is lower than the average payoff of his/her neighbors. The simulation results show that this mechanism promotes cooperation effectively even with antisocial punishment. By adopting such a mechanism, cooperative punishers form shields to protect cooperators inside, while defective punishers hide behind defectors without punishing anyone. © 2019 Elsevier Inc. All rights reserved.
1. Introduction Since ubiquitous cooperative behavior is the foundation of social stability and the impetus to push the society forward, many biologists and sociologists have devoted themselves to the phenomenon for a long time [1–3]. However, this phenomenon is contrary to the instinct of humans in that they tend to be selfish and obtain more resources in spite of the loss of the public resources [4,5]. Thus, it is important to uncover the internal reason why this contradiction appears [6–8]. Evolutionary game theory has become the mainstream analytical framework to study the evolution and maintenance of cooperative behavior [9–11]. In particular, the PDG (prisoner’s dilemma game) is the most classical model and is considered as a paradigm. Until now, many mechanisms have been proven to validly enhance the evolution of cooperation [12–29]. Spatial reciprocity, raised by Nowak [30], allows cooperative individuals to prevent defection by forming clusters. Therefore, the population structure can determine the direction of the evolution of cooperation [31–40]. In addition, inspired by real society, many approaches that help the cooperation dominate the system have been introduced and studied [41–46]. Moreover, punishment is a significant way to promote cooperation [47–50]. However, individuals would like to punish other players based on certain conditions rather than punish them directly. Enlightened by this, a conditional neutral punishment mechanism is introduced, in which an individual can punish his/her neighbors with the opposite strategy when ∗ ∗∗
Corresponding author at: Center for Ecology and Environmental Sciences, Northwestern Polytechnical University, Xi’an 710072, China. Corresponding author at: School of Software, Yunnan University, Kunming, Yunnan 650504, China. E-mail addresses: [email protected] (C. Liu), [email protected] (J. Liu).
https://doi.org/10.1016/j.amc.2019.124798 0 096-30 03/© 2019 Elsevier Inc. All rights reserved.
2
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
Fig. 1. Fraction of cooperation on the b− β parameter space for α = 0.01.
his/her payoff is lower than the average payoff of his/her neighbors. Thus, the punishment can also be named “zealous agents” (i.e., those who levy some penalty to successful agents) or “sanctions triggered by jealousy” [51,52]. Implementing our conditional neutral punishment rule, we find that cooperation can be maintained for a high value of temptation to defect and that antisocial punishment cannot deter the dynamic of the system. The remainder of the paper is structured as follows. First, we present the detailed model. Second, we show the simulation results and explain the potential reasons by a visualization method. Finally, we discuss the main conclusions. 2. Results To examine the influence of the conditional neutral punishment mechanism on the evolution of cooperation, we show the simulation result as a function of the social fine (β ) and the temptation (b) in Fig. 1. Some materials concerning the concept of the universal scaling of dilemma strength support the results [53–55]. We set the cost to punish as α = 0.01. Compared to the traditional case in which cooperation becomes extinct for a small b (e.g., b = 1.04), the fraction of cooperation is distinctly promoted owing to the introduction of the mechanism. In fact, a very low value of social fine β (β = 0.1) can change the dynamic. Moreover, with every additional increment of β , cooperators can survive better. Fig. 2 shows the hybrid impact of both social fine (β ) and cost to punish (α ) on the evolution of cooperation. As predicted, the incremental increase of the cost α leads to abominable conditions for the survival of cooperation such that defection dominates the system for α = 1.78 regardless of the value of β . Interestingly, it is easily observed that the slope of the curve between the cooperation phase and the defection phase is almost equal to 1. The phenomenon can be explained as follows. The key point at which defection invades cooperation is whether the payoff of the defective individual on the boundary between the cooperation and defection phase is more than the payoff of the cooperative individuals next to the defective one, as shown in Fig. 3. After a simple calculation, we obtain that the payoff of the defectors indicated by a black square is equal to 2b–2β , while the payoff of the cooperators indicated by a black square is equal to 2–2α . Thus, the critical condition is b − β = 1 − α . For α = 0.01, the critical condition changes to b − β = 0.99, which is consistent with the phenomenon shown in Fig. 1. However, for b = 1.3, the critical condition changes to β − α = 0.3, which is consistent with the phenomenon shown in Fig. 2. To examine the potential reason why conditional neutral punishment can promote cooperation, we show snapshots of the stable state for different parameters, namely, fixed b (b = 1.48), fixed cost to punish α (α = 0.01) and different social fines. Since the conditional neutral punishment is introduced into the game (Fig. 3(a)), it is observed that cooperators fail to survive, although they form clusters surrounded by a cooperative punisher, because of the low punishment. As the level of punishment increases (i.e., β = 0.48), several compact cooperative clusters are formed by inner cooperators and fringe punishers. Therefore, the spreading of defectors is prevented. When the social fine increases slightly (β = 0.51), the cooperative clusters not only maintain but also expand. For a larger β (β = 0.60), cooperators dominate the system. It seems that cooperative punishers deter the evolution of cooperation while defective punishers do not work. We can explain this outcome as
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
3
Fig. 2. Fraction of cooperation on the α − β parameter space for b = 1.3.
Fig. 3. Schematic of the leading invasion process. Cooperator (defectors) are presented in red (yellow), while cooperative punisher are presented in green (blue). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
follows. Cooperative punishers can punish defectors, as they are just on the edge between cooperative and defective agents. However, defective punishers cannot punish cooperative agents, as they have no chance to interact with cooperative agents, as shown in Fig. 3. To prove the explanation above concretely, we separate the system into four regions with cooperators and defectors, as shown in Fig. 4. As seen, cooperative punishers (yellow) form a shield over cooperators, while defective punishers (green) hide behind defectors. Fig. 5(a) shows how the number of cooperative clusters (including cooperators and cooperative punishers) and the number of pure cooperative clusters evolve respectively. Meanwhile, Fig. 5(b) presents the size of the largest cooperative cluster and the size of the largest pure cooperative clusters. When the value of b is low (b < 1.30), there only exists one cooperative cluster fulfilled with pure cooperators, whose size is equal to the size of the system. This outcome indicates that cooperation dominates the system, which is consistent with the result shown in the snapshots. When b increases (1.35 1.55, the punishment mechanism fails and cooperators soon disappear spontaneously. However, what our result implies might be limited in terms of the definition of played games (Fig. 6).
4
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
Fig. 4. Characteristic snapshots of cooperators who do not punish others (red), defectors who do not punish others (blue), cooperators who punish others (yellow) and defectors who punish others (green). The four rows of snapshots correspond to β values of 0.46, 0.47, 0.5 and 0.53, respectively. The four columns of snapshots represent MCS = 0, 1, 10, 100, and 5 × 104 , respectively. The results are obtained by setting α = 0.01 and b = 1.48. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
3. Methods All players are located at the two-dimensional square lattice of size L × L, where each player occupies a node and is connected to its four neighbors. At first, each player on the network is set to be a cooperator or a defector with a probability of 0.5. At each time step, each player plays a two-stage game with his/her neighbors. In the first stage, players play the traditional PDG. In accordance with the common practice in previous studies, the payoffs matrix of PDG is set as follows:
A=
1 b
0 . 0
(1)
A randomly chosen player x obtains payoff px by interacting with all his/her neighbors, and px can be calculated as follows:
px =
4
pxy ,
(2)
y=1
where pxy denotes the earnings that individual x can obtain through the interaction with individual y. In the second stage, players decide whether to punish neighbors with the opposite strategy. First, we can get the payoffs of each player x’s neighbors and calculate their average, defined as follows:
Ave gx =
y∈Nx
py /Nx ,
(3)
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
5
Fig. 5. Initial evolution of the prepared scenario. The four rows of snapshots correspond to β values of 0, 0.47, and 0.5, 0.53, respectively. The four columns of snapshots represent MCS = 0, 1, 10, 100, and 5 × 104 , respectively. The results are obtained by setting α = 0.01 and b = 1.48. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Fig. 6. Ncc , Nc , Scc , and Sc , as a function of b for β = 0.5 and α = 0.01. Ncc is the number of pure cooperative clusters. Nc is the number of cooperative clusters. Scc is the size of the largest pure cooperative cluster. Sc is the size of the largest cooperative cluster.
6
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
where Nx denotes the aggregate of all neighbors of individual x. If px ≥ Avegx , our model is similar to the traditional PDG model. Otherwise, if px < Avegx , the strategy-neutral punishment will be introduced. In detail, player x will pay a cost to punish α to each neighbor with the opposite strategy and make these neighbors lose a social fine β. At the same time, player x may be punished by his/her four neighbors with the opposite strategy. Hence, the updated payoff of player x upx can be obtained by the following equation:
u px = px − M × β − N × α ,
(4)
where M(0 ≤ M ≤ 4) is the number of neighbors who punish player x. In addition, N represents the number of neighbors punished by player x; regardless of whether punishment is taken, player x will update its strategy as follows. Player x picks up a random neighbor y, who gets his/her updated payoff in the same way, and imitates the strategy of player y with the probability obtained by the following Fermi function:
Px→y =
1 , 1 + exp[(u px − u py )/K ]
(5)
where K represents the intensity of selection and is set to 0.1 in this example [56–60]. The evolution process is simulated for L = 200. The relaxation time of the system is set as 5 × 104 Monte Carlo steps; then, we obtain the fraction of cooperation ρ c by calculating the average fraction of cooperation for the last 5 × 103 steps. To eliminate the random errors, the results are averaged over 10 independent realizations for each set of parameter values. 4. Conclusion Inspired by the fact that individuals would like to punish other players based on certain conditions rather than punish them directly in real society, we explored the joint impact of neutral and conditional punishment in structured populations. Specifically, an individual can punish his/her neighbors with the opposite strategy when his/her payoff is lower than the average payoff of his/hers neighbors. Through numerical simulations, we find that cooperation is greatly promoted and conditional antisocial punishment does not deter the evolution of cooperation. From a micro-perspective, we provide some evidence that our mechanism enhances the spatial reciprocity and is beneficial for forming compact cooperative clusters. In our model, cooperative punishers form shields to protect cooperators inside regardless of b, while defective punishers hide behind defectors without punishing anyone. Generally, social punishment rather than antisocial punishment shapes the direction of collective behavior when an individual punishes others under the condition that their payoff is below the average. Acknowledgment We appreciate the support from (I) National Natural Science Foundation of China (Grant no. U1803263) National Natural Science Foundation of China; (II) National Natural Science Foundation of China (Grant nos. 61866039, 61662085, 61462092); (III) Science Foundation of Yunnan Province Education Department (Grant no. 2017ZZX08). References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22]
R. Axelrod, W.D. Hamilton, The evolution of cooperation, Science 211 (1981) 1390–1396. N.S. Wingreen, S.A. Levin, Cooperation among microorganisms, PLoS Biol. 4 (9) (2006) e299. V.J. Neumann, O. Morgenstern, in: Theory of Games and Economic Behavior, 26, Princeton University Press, Princeton NJ, 2007, pp. 131–141. 2007. M.A. Nowak, A. Sasaki, C. Taylor, D. Fudenberg, Emergence of cooperation and evolutionary stability in finite populations, Nature 428 (6983) (2004) 646–650. C. Castellano, S. Fortunato, V. Loreto, Statistical physics of social dynamics, Rev. Mod. Phys. 81 (2) (2009) 591–646. J.M. Smith, Evolution and the Theory of Games, Cambridge University Press, 1982. J.W. Weibull, Evolutionary selection in normal-form games, Econom. Econom. Soc. 63 (6) (1995) 1371–1399. J. Hofbauer, K. Sigmund, Evolutionary games and population dynamics, J. Am. Stat. Assoc. 95 (450) (1998) 50–54. G. Szabo, C. Toke, Evolutionary prisoner’s dilemma game on a square lattice, Phys. Rev. E 58 (1998) 69–73. H. Komatsu, Altruistic cooperation during foraging by the Ache, and the evolved predisposition to cooperate, Hum. Nat. 13 (1) (2002) 105. R.A. Hammond, R. Axelrod, Evolution of contingent altruism when cooperation is expensive, Theor. Popul. Biol. 69 (3) (2006) 333–338. G. Hardin, The tragedy of the commons. the population problem has no technical solution; it requires a fundamental extension in morality, Science 162 (1968) 1243–1248. J. Von Neumann, O. Morgenstern, Theory of Games and Economic Behavior, commemorative edition, Princeton University Press, 2007. M.A. Nowak, K. Sigmund, Evolution of indirect reciprocity by image scoring, Nature 393 (6685) (1998) 31225. M. Perc, A. Szolnoki, G. Szabó, Social diversity and promotion of cooperation in the spatial prisoner’s dilemma game, Phys. Rev. E 78 (1) (2008) 066101. A. Szolnoki, M. Perc, G. Szabó, Emergence of multilevel selection in the prisoner’s dilemma game on coevolving random networks, Phys. Rev. E 80 (9) (2009) 056104. H. Zhang, M. Small, H. Yang, B. Wang, Adjusting learning motivation to promote cooperation, Phys. A Stat. Mech. Appl. 389 (21) (2010) 4734–4739. Z. Wang, X. Zhu, J.J. Arenzon, Cooperation and age structure in spatial games, Phys. Rev. E 85 (1) (2012) 011149. C. Shen, C. Chu, L. Shi, M. Perc, Z. Wang, Aspiration-based coevolution of link weight promotes cooperation in the spatial prisoner’s dilemma game, R. Soc. Open Sci. 5 (5) (2018) 180199. Z. Wang, A. Szolnoki, M. Perc, Evolution of public cooperation on interdependent networks: the impact of biased utility functions, Sci. Rep. 2 (2012) 369. C. Liu, J. Shi, T. Li, J. Liu, Aspiration driven coevolution resolves social dilemmas in networks, Appl. Math. Comput. 342 (2019) 247–254. C. Shen, C. Chu, H. Guo, L. Shi, J. Duan, Coevolution of vertex weights resolves social dilemma in spatial networks, Sci. Rep. 7 (2017) 15213.
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
7
[23] C. Shen, C. Chu, Y. Geng, J. Jin, F. Chen, L. Shi, Cooperation enhanced by the coevolution of teaching activity in evolutionary prisoner’s dilemma games with voluntary participation, PLoS One 13 (2018) e0193151. [24] M. Perc, Z. Wang, Heterogeneous aspirations promote cooperation in the prisoner’s dilemma game, PLoS One 5 (2010) e15117. [25] Z. Wang, M. Perc, Aspiring to the fittest and promotion of cooperation in the prisoner’s dilemma game, Phys. Rev. E 82 (2010) 021115. [26] A.L. Barabási, R. Albert, Emergence of scaling in random networks, Science 286 (1999) 509–512. [27] C. Chu, C. Mu, J. Liu, C. Liu, S. Boccaletti, L. Shi, Z. Wang, Aspiration-based coevolution of node weights promotes cooperation in the spatial prisoner’s dilemma game, New J. Phys. 21 (2019) 063024. [28] D.J. Watts, S.H. Strogatz, Collective dynamics of ‘small-world’ networks, Nature 393 (1998) 440. [29] Z. Wang, L. Wang, A. Szolnoki, M. Perc, Evolutionary games on multilayer networks: a colloquium, Eur. Phys. J. B 88 (2015) 124. [30] M.A. Nowak, Five rules for the evolution of cooperation, Science 314 (5805) (2006) 1560–1563. [31] K. Huang, X. Chen, Z. Yu, C. Yang, W. Gui, Heterogeneous cooperative belief for social dilemma in multi-agent system, Appl. Math. Comput. 320 (2018) 572–579. [32] A. Szolnoki, M. Perc, Promoting cooperation in social dilemmas via simple coevolutionary rules, New J. Phys. 10 (2008) 043036. ´ , G. Szabó, Motion of influential players can support cooperation in prisoner’s dilemma, Eur. Phys. J. B 71 (2009) 579. [33] M. Droz, J. Szwabinski [34] M. Perc, A. Szolnoki, Restricted connections among distinguished players support cooperation, Phys. Rev. E 77 (2008) 011904. [35] N.J. Raihani, R. Bshary, The evolution of punishment in n-player public goods games: a volunteer’s dilemma, Evolution 65 (10) (2011) 1558–5646. [36] C. Chu, J. Liu, C. Shen, Coevolution of game strategy and link weight promotes cooperation in structured population, Chaos Solitons Fractals 104 (2017) 28–32. [37] C. Chu, X. Hu, C. Shen, T. Li, S. Boccaletti, L. Shi, Z. Wang, Self-organized interdependence among populations promotes cooperation by means of coevolution, Chaos 29 (1) (2019) 013139. [38] X. Dai, P. Zhu, Y. Guo, Z. Wang, Coevolution of vaccination opinion and awareness for the epidemic spreading process, IEEE Access 7 (2019) 61558–61569. [39] K. Huang, Z. Wang, M. Jusup, Incorporating latent constraints to enhance inference of network structure, IEEE Trans. Netw. Sci. Eng. (2018), doi:10. 1109/TNSE.2018.2870687. [40] K. Huang, L. Yishun, Z. Yichi, Understanding cooperative behavior of agents with heterogeneous perceptions in dynamic networks, Phys. A Stat. Mech. Appl. 509 (2018) 234–240. [41] Z. Wang, A. Szolnoki, M. Perc, If players are sparse social dilemmas are too: importance of percolation for evolution of cooperation, Sci. Rep. 2 (2012) 369. [42] Z. Wang, J. Marko, S. Lei, Exploiting a cognitive bias promotes cooperation in social dilemma experiments, Nat. Commun. 9 (1) (2018) 2954. [43] Z. Wang, M. Jusup, R.W. Wang, Anonymity promotes cooperation in social dilemma experiments, Sci. Adv. 3 (3) (2017) e1601444. [44] C. Chu, J. Liu, C. Shen, Win-stay-lose-learn promotes cooperation in the prisoner’s dilemma game with voluntary participation, PLoS One 12 (2) (2017) e0171680. [45] J. Liu, H. Meng, W. Wang, Evolution of cooperation on independent networks: the influence of asymmetric information sharing updating mechanism, Appl. Math. Comput. 340 (2019) 234–241. [46] C. Shen, J. Lu, L. Shi, Does coevolution setup promote cooperation in spatial prisoner’s dilemma game? Appl. Math. Comput. 290 (2016) 201–207. [47] M. Perc, Sustainable institutionalized punishment requires elimination of second order free-riders, Sci. Rep. 2 (2012) 344. [48] D. Helbing, A. Szolnoki, M. Perc, G. Szabo, Defector-accelerated cooperativeness and punishment in public goods games with mutations, Phys. Rev. E 81 (2010) 057104. [49] B. Robert, J.R. Peter, Punishment allows the evolution of cooperation (or anything else) in sizable groups, Ethol. Sociobiol. 13 (1992) 171–195. [50] D.G. Rand, M.A. Nowak, The evolution of antisocial punishment in optional public goods games, Nat. Commun. 2 (1) (2011) 434. [51] R. Matsuzawa, J. Tanimoto, Sanctions triggered by jealousy help promote the cooperation in spatial prisoner’s dilemma games, Chaos Solitons Fractals 110 (2018) 239–243. [52] R. Matsuzawa, J. Tanimoto, F. Eriko, Spatial prisoner’s dilemma games with zealous cooperators, Phys. Rev. E 94 (2016) 022114. [53] J. Tanimoto, H. Sagara, Relationship between dilemma occurrence and the existence of a weakly dominant strategy in a two-player symmetric game, BioSystems 90 (1) (2007) 105–114. [54] Z. Wang, S. Kokubo, M. Jusup, Dilemma strength as a framework for advancing evolutionary game theory: reply to comments on "Universal scaling for the dilemma strength in evolutionary games", Phys. Life Rev. 14 (2015) 56–58. [55] H. Ito, J. Tanimoto, Scaling the phase-planes of social dilemma strengths shows game-class changes in the five rules governing the evolution of cooperation, R. Soc. Open Sci. 5 (10) (2018) 181085. [56] J. Tanimoto, Evolutionary Games with Sociophysics: Analysis of Traffic Flow and Epidemics, Springer, 2019. [57] J. Tanimoto, Fundamentals of Evolutionary Game Theory and its Applications, Springer, 2015. [58] M. Alam, K. Nagashima, J. Tanimoto, Various error settings bring different noise-driven effects on network reciprocity in spatial prisoner’s dilemma, Chaos Solitons Fractals 114 (2018) 338–346. [59] M. Alam, K. Kuga, J. Tanimoto, Three-strategy and four-strategy model of vaccination game introducing an intermediate protecting measure, Appl. Math. Comput. 346 (2019) 408–422. [60] M. Alam, M. Tanaka, J. Tanimoto, A game theoretic approach to discuss the positive secondary effect of vaccination scheme in an infinite and well-mixed population, Chaos Solitons Fractals 125 (2019) 201–213.
Contents lists available at ScienceDirect
Applied Mathematics and Computation journal homepage: www.elsevier.com/locate/amc
Conditional neutral punishment promotes cooperation in the spatial prisoner’s dilemma game Qun Song a, Zhaoheng Cao b,c, Rui Tao b,c, Wei Jiang d, Chen Liu e,∗, Jinzhuo Liu b,c,∗∗ a
School of Automation, Northwestern Polytechnical University, Xi’an 710072, China Key Laboratory in Software Engineering of Yunnan Province, Kunming 650091, China c School of Software, Yunnan University, Kunming, Yunnan 650504, China d Unit 78102 of the Chinese people’s liberation army, Chengdu 610000, China e Center for Ecology and Environmental Sciences, Northwestern Polytechnical University, Xi’an 710072, China b
a r t i c l e
i n f o
Article history: Received 17 July 2019 Revised 20 September 2019 Accepted 30 September 2019
Keywords: Conditional punishment Strategy-neutral punishment Evolutionary games Social dilemmas Cooperation
a b s t r a c t Punishment plays an important role in promoting cooperation. In real society, individuals tend to punish other players based on certain conditions rather than punish them directly. Thus, we introduce a conditional neutral punishment mechanism and study how this mechanism affects the evolution of cooperation. Namely, an individual can punish his/her neighbors with the opposite strategy when his/her payoff is lower than the average payoff of his/her neighbors. The simulation results show that this mechanism promotes cooperation effectively even with antisocial punishment. By adopting such a mechanism, cooperative punishers form shields to protect cooperators inside, while defective punishers hide behind defectors without punishing anyone. © 2019 Elsevier Inc. All rights reserved.
1. Introduction Since ubiquitous cooperative behavior is the foundation of social stability and the impetus to push the society forward, many biologists and sociologists have devoted themselves to the phenomenon for a long time [1–3]. However, this phenomenon is contrary to the instinct of humans in that they tend to be selfish and obtain more resources in spite of the loss of the public resources [4,5]. Thus, it is important to uncover the internal reason why this contradiction appears [6–8]. Evolutionary game theory has become the mainstream analytical framework to study the evolution and maintenance of cooperative behavior [9–11]. In particular, the PDG (prisoner’s dilemma game) is the most classical model and is considered as a paradigm. Until now, many mechanisms have been proven to validly enhance the evolution of cooperation [12–29]. Spatial reciprocity, raised by Nowak [30], allows cooperative individuals to prevent defection by forming clusters. Therefore, the population structure can determine the direction of the evolution of cooperation [31–40]. In addition, inspired by real society, many approaches that help the cooperation dominate the system have been introduced and studied [41–46]. Moreover, punishment is a significant way to promote cooperation [47–50]. However, individuals would like to punish other players based on certain conditions rather than punish them directly. Enlightened by this, a conditional neutral punishment mechanism is introduced, in which an individual can punish his/her neighbors with the opposite strategy when ∗ ∗∗
Corresponding author at: Center for Ecology and Environmental Sciences, Northwestern Polytechnical University, Xi’an 710072, China. Corresponding author at: School of Software, Yunnan University, Kunming, Yunnan 650504, China. E-mail addresses: [email protected] (C. Liu), [email protected] (J. Liu).
https://doi.org/10.1016/j.amc.2019.124798 0 096-30 03/© 2019 Elsevier Inc. All rights reserved.
2
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
Fig. 1. Fraction of cooperation on the b− β parameter space for α = 0.01.
his/her payoff is lower than the average payoff of his/her neighbors. Thus, the punishment can also be named “zealous agents” (i.e., those who levy some penalty to successful agents) or “sanctions triggered by jealousy” [51,52]. Implementing our conditional neutral punishment rule, we find that cooperation can be maintained for a high value of temptation to defect and that antisocial punishment cannot deter the dynamic of the system. The remainder of the paper is structured as follows. First, we present the detailed model. Second, we show the simulation results and explain the potential reasons by a visualization method. Finally, we discuss the main conclusions. 2. Results To examine the influence of the conditional neutral punishment mechanism on the evolution of cooperation, we show the simulation result as a function of the social fine (β ) and the temptation (b) in Fig. 1. Some materials concerning the concept of the universal scaling of dilemma strength support the results [53–55]. We set the cost to punish as α = 0.01. Compared to the traditional case in which cooperation becomes extinct for a small b (e.g., b = 1.04), the fraction of cooperation is distinctly promoted owing to the introduction of the mechanism. In fact, a very low value of social fine β (β = 0.1) can change the dynamic. Moreover, with every additional increment of β , cooperators can survive better. Fig. 2 shows the hybrid impact of both social fine (β ) and cost to punish (α ) on the evolution of cooperation. As predicted, the incremental increase of the cost α leads to abominable conditions for the survival of cooperation such that defection dominates the system for α = 1.78 regardless of the value of β . Interestingly, it is easily observed that the slope of the curve between the cooperation phase and the defection phase is almost equal to 1. The phenomenon can be explained as follows. The key point at which defection invades cooperation is whether the payoff of the defective individual on the boundary between the cooperation and defection phase is more than the payoff of the cooperative individuals next to the defective one, as shown in Fig. 3. After a simple calculation, we obtain that the payoff of the defectors indicated by a black square is equal to 2b–2β , while the payoff of the cooperators indicated by a black square is equal to 2–2α . Thus, the critical condition is b − β = 1 − α . For α = 0.01, the critical condition changes to b − β = 0.99, which is consistent with the phenomenon shown in Fig. 1. However, for b = 1.3, the critical condition changes to β − α = 0.3, which is consistent with the phenomenon shown in Fig. 2. To examine the potential reason why conditional neutral punishment can promote cooperation, we show snapshots of the stable state for different parameters, namely, fixed b (b = 1.48), fixed cost to punish α (α = 0.01) and different social fines. Since the conditional neutral punishment is introduced into the game (Fig. 3(a)), it is observed that cooperators fail to survive, although they form clusters surrounded by a cooperative punisher, because of the low punishment. As the level of punishment increases (i.e., β = 0.48), several compact cooperative clusters are formed by inner cooperators and fringe punishers. Therefore, the spreading of defectors is prevented. When the social fine increases slightly (β = 0.51), the cooperative clusters not only maintain but also expand. For a larger β (β = 0.60), cooperators dominate the system. It seems that cooperative punishers deter the evolution of cooperation while defective punishers do not work. We can explain this outcome as
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
3
Fig. 2. Fraction of cooperation on the α − β parameter space for b = 1.3.
Fig. 3. Schematic of the leading invasion process. Cooperator (defectors) are presented in red (yellow), while cooperative punisher are presented in green (blue). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
follows. Cooperative punishers can punish defectors, as they are just on the edge between cooperative and defective agents. However, defective punishers cannot punish cooperative agents, as they have no chance to interact with cooperative agents, as shown in Fig. 3. To prove the explanation above concretely, we separate the system into four regions with cooperators and defectors, as shown in Fig. 4. As seen, cooperative punishers (yellow) form a shield over cooperators, while defective punishers (green) hide behind defectors. Fig. 5(a) shows how the number of cooperative clusters (including cooperators and cooperative punishers) and the number of pure cooperative clusters evolve respectively. Meanwhile, Fig. 5(b) presents the size of the largest cooperative cluster and the size of the largest pure cooperative clusters. When the value of b is low (b < 1.30), there only exists one cooperative cluster fulfilled with pure cooperators, whose size is equal to the size of the system. This outcome indicates that cooperation dominates the system, which is consistent with the result shown in the snapshots. When b increases (1.35 1.55, the punishment mechanism fails and cooperators soon disappear spontaneously. However, what our result implies might be limited in terms of the definition of played games (Fig. 6).
4
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
Fig. 4. Characteristic snapshots of cooperators who do not punish others (red), defectors who do not punish others (blue), cooperators who punish others (yellow) and defectors who punish others (green). The four rows of snapshots correspond to β values of 0.46, 0.47, 0.5 and 0.53, respectively. The four columns of snapshots represent MCS = 0, 1, 10, 100, and 5 × 104 , respectively. The results are obtained by setting α = 0.01 and b = 1.48. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
3. Methods All players are located at the two-dimensional square lattice of size L × L, where each player occupies a node and is connected to its four neighbors. At first, each player on the network is set to be a cooperator or a defector with a probability of 0.5. At each time step, each player plays a two-stage game with his/her neighbors. In the first stage, players play the traditional PDG. In accordance with the common practice in previous studies, the payoffs matrix of PDG is set as follows:
A=
1 b
0 . 0
(1)
A randomly chosen player x obtains payoff px by interacting with all his/her neighbors, and px can be calculated as follows:
px =
4
pxy ,
(2)
y=1
where pxy denotes the earnings that individual x can obtain through the interaction with individual y. In the second stage, players decide whether to punish neighbors with the opposite strategy. First, we can get the payoffs of each player x’s neighbors and calculate their average, defined as follows:
Ave gx =
y∈Nx
py /Nx ,
(3)
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
5
Fig. 5. Initial evolution of the prepared scenario. The four rows of snapshots correspond to β values of 0, 0.47, and 0.5, 0.53, respectively. The four columns of snapshots represent MCS = 0, 1, 10, 100, and 5 × 104 , respectively. The results are obtained by setting α = 0.01 and b = 1.48. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Fig. 6. Ncc , Nc , Scc , and Sc , as a function of b for β = 0.5 and α = 0.01. Ncc is the number of pure cooperative clusters. Nc is the number of cooperative clusters. Scc is the size of the largest pure cooperative cluster. Sc is the size of the largest cooperative cluster.
6
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
where Nx denotes the aggregate of all neighbors of individual x. If px ≥ Avegx , our model is similar to the traditional PDG model. Otherwise, if px < Avegx , the strategy-neutral punishment will be introduced. In detail, player x will pay a cost to punish α to each neighbor with the opposite strategy and make these neighbors lose a social fine β. At the same time, player x may be punished by his/her four neighbors with the opposite strategy. Hence, the updated payoff of player x upx can be obtained by the following equation:
u px = px − M × β − N × α ,
(4)
where M(0 ≤ M ≤ 4) is the number of neighbors who punish player x. In addition, N represents the number of neighbors punished by player x; regardless of whether punishment is taken, player x will update its strategy as follows. Player x picks up a random neighbor y, who gets his/her updated payoff in the same way, and imitates the strategy of player y with the probability obtained by the following Fermi function:
Px→y =
1 , 1 + exp[(u px − u py )/K ]
(5)
where K represents the intensity of selection and is set to 0.1 in this example [56–60]. The evolution process is simulated for L = 200. The relaxation time of the system is set as 5 × 104 Monte Carlo steps; then, we obtain the fraction of cooperation ρ c by calculating the average fraction of cooperation for the last 5 × 103 steps. To eliminate the random errors, the results are averaged over 10 independent realizations for each set of parameter values. 4. Conclusion Inspired by the fact that individuals would like to punish other players based on certain conditions rather than punish them directly in real society, we explored the joint impact of neutral and conditional punishment in structured populations. Specifically, an individual can punish his/her neighbors with the opposite strategy when his/her payoff is lower than the average payoff of his/hers neighbors. Through numerical simulations, we find that cooperation is greatly promoted and conditional antisocial punishment does not deter the evolution of cooperation. From a micro-perspective, we provide some evidence that our mechanism enhances the spatial reciprocity and is beneficial for forming compact cooperative clusters. In our model, cooperative punishers form shields to protect cooperators inside regardless of b, while defective punishers hide behind defectors without punishing anyone. Generally, social punishment rather than antisocial punishment shapes the direction of collective behavior when an individual punishes others under the condition that their payoff is below the average. Acknowledgment We appreciate the support from (I) National Natural Science Foundation of China (Grant no. U1803263) National Natural Science Foundation of China; (II) National Natural Science Foundation of China (Grant nos. 61866039, 61662085, 61462092); (III) Science Foundation of Yunnan Province Education Department (Grant no. 2017ZZX08). References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22]
R. Axelrod, W.D. Hamilton, The evolution of cooperation, Science 211 (1981) 1390–1396. N.S. Wingreen, S.A. Levin, Cooperation among microorganisms, PLoS Biol. 4 (9) (2006) e299. V.J. Neumann, O. Morgenstern, in: Theory of Games and Economic Behavior, 26, Princeton University Press, Princeton NJ, 2007, pp. 131–141. 2007. M.A. Nowak, A. Sasaki, C. Taylor, D. Fudenberg, Emergence of cooperation and evolutionary stability in finite populations, Nature 428 (6983) (2004) 646–650. C. Castellano, S. Fortunato, V. Loreto, Statistical physics of social dynamics, Rev. Mod. Phys. 81 (2) (2009) 591–646. J.M. Smith, Evolution and the Theory of Games, Cambridge University Press, 1982. J.W. Weibull, Evolutionary selection in normal-form games, Econom. Econom. Soc. 63 (6) (1995) 1371–1399. J. Hofbauer, K. Sigmund, Evolutionary games and population dynamics, J. Am. Stat. Assoc. 95 (450) (1998) 50–54. G. Szabo, C. Toke, Evolutionary prisoner’s dilemma game on a square lattice, Phys. Rev. E 58 (1998) 69–73. H. Komatsu, Altruistic cooperation during foraging by the Ache, and the evolved predisposition to cooperate, Hum. Nat. 13 (1) (2002) 105. R.A. Hammond, R. Axelrod, Evolution of contingent altruism when cooperation is expensive, Theor. Popul. Biol. 69 (3) (2006) 333–338. G. Hardin, The tragedy of the commons. the population problem has no technical solution; it requires a fundamental extension in morality, Science 162 (1968) 1243–1248. J. Von Neumann, O. Morgenstern, Theory of Games and Economic Behavior, commemorative edition, Princeton University Press, 2007. M.A. Nowak, K. Sigmund, Evolution of indirect reciprocity by image scoring, Nature 393 (6685) (1998) 31225. M. Perc, A. Szolnoki, G. Szabó, Social diversity and promotion of cooperation in the spatial prisoner’s dilemma game, Phys. Rev. E 78 (1) (2008) 066101. A. Szolnoki, M. Perc, G. Szabó, Emergence of multilevel selection in the prisoner’s dilemma game on coevolving random networks, Phys. Rev. E 80 (9) (2009) 056104. H. Zhang, M. Small, H. Yang, B. Wang, Adjusting learning motivation to promote cooperation, Phys. A Stat. Mech. Appl. 389 (21) (2010) 4734–4739. Z. Wang, X. Zhu, J.J. Arenzon, Cooperation and age structure in spatial games, Phys. Rev. E 85 (1) (2012) 011149. C. Shen, C. Chu, L. Shi, M. Perc, Z. Wang, Aspiration-based coevolution of link weight promotes cooperation in the spatial prisoner’s dilemma game, R. Soc. Open Sci. 5 (5) (2018) 180199. Z. Wang, A. Szolnoki, M. Perc, Evolution of public cooperation on interdependent networks: the impact of biased utility functions, Sci. Rep. 2 (2012) 369. C. Liu, J. Shi, T. Li, J. Liu, Aspiration driven coevolution resolves social dilemmas in networks, Appl. Math. Comput. 342 (2019) 247–254. C. Shen, C. Chu, H. Guo, L. Shi, J. Duan, Coevolution of vertex weights resolves social dilemma in spatial networks, Sci. Rep. 7 (2017) 15213.
Q. Song, Z. Cao and R. Tao et al. / Applied Mathematics and Computation 368 (2020) 124798
7
[23] C. Shen, C. Chu, Y. Geng, J. Jin, F. Chen, L. Shi, Cooperation enhanced by the coevolution of teaching activity in evolutionary prisoner’s dilemma games with voluntary participation, PLoS One 13 (2018) e0193151. [24] M. Perc, Z. Wang, Heterogeneous aspirations promote cooperation in the prisoner’s dilemma game, PLoS One 5 (2010) e15117. [25] Z. Wang, M. Perc, Aspiring to the fittest and promotion of cooperation in the prisoner’s dilemma game, Phys. Rev. E 82 (2010) 021115. [26] A.L. Barabási, R. Albert, Emergence of scaling in random networks, Science 286 (1999) 509–512. [27] C. Chu, C. Mu, J. Liu, C. Liu, S. Boccaletti, L. Shi, Z. Wang, Aspiration-based coevolution of node weights promotes cooperation in the spatial prisoner’s dilemma game, New J. Phys. 21 (2019) 063024. [28] D.J. Watts, S.H. Strogatz, Collective dynamics of ‘small-world’ networks, Nature 393 (1998) 440. [29] Z. Wang, L. Wang, A. Szolnoki, M. Perc, Evolutionary games on multilayer networks: a colloquium, Eur. Phys. J. B 88 (2015) 124. [30] M.A. Nowak, Five rules for the evolution of cooperation, Science 314 (5805) (2006) 1560–1563. [31] K. Huang, X. Chen, Z. Yu, C. Yang, W. Gui, Heterogeneous cooperative belief for social dilemma in multi-agent system, Appl. Math. Comput. 320 (2018) 572–579. [32] A. Szolnoki, M. Perc, Promoting cooperation in social dilemmas via simple coevolutionary rules, New J. Phys. 10 (2008) 043036. ´ , G. Szabó, Motion of influential players can support cooperation in prisoner’s dilemma, Eur. Phys. J. B 71 (2009) 579. [33] M. Droz, J. Szwabinski [34] M. Perc, A. Szolnoki, Restricted connections among distinguished players support cooperation, Phys. Rev. E 77 (2008) 011904. [35] N.J. Raihani, R. Bshary, The evolution of punishment in n-player public goods games: a volunteer’s dilemma, Evolution 65 (10) (2011) 1558–5646. [36] C. Chu, J. Liu, C. Shen, Coevolution of game strategy and link weight promotes cooperation in structured population, Chaos Solitons Fractals 104 (2017) 28–32. [37] C. Chu, X. Hu, C. Shen, T. Li, S. Boccaletti, L. Shi, Z. Wang, Self-organized interdependence among populations promotes cooperation by means of coevolution, Chaos 29 (1) (2019) 013139. [38] X. Dai, P. Zhu, Y. Guo, Z. Wang, Coevolution of vaccination opinion and awareness for the epidemic spreading process, IEEE Access 7 (2019) 61558–61569. [39] K. Huang, Z. Wang, M. Jusup, Incorporating latent constraints to enhance inference of network structure, IEEE Trans. Netw. Sci. Eng. (2018), doi:10. 1109/TNSE.2018.2870687. [40] K. Huang, L. Yishun, Z. Yichi, Understanding cooperative behavior of agents with heterogeneous perceptions in dynamic networks, Phys. A Stat. Mech. Appl. 509 (2018) 234–240. [41] Z. Wang, A. Szolnoki, M. Perc, If players are sparse social dilemmas are too: importance of percolation for evolution of cooperation, Sci. Rep. 2 (2012) 369. [42] Z. Wang, J. Marko, S. Lei, Exploiting a cognitive bias promotes cooperation in social dilemma experiments, Nat. Commun. 9 (1) (2018) 2954. [43] Z. Wang, M. Jusup, R.W. Wang, Anonymity promotes cooperation in social dilemma experiments, Sci. Adv. 3 (3) (2017) e1601444. [44] C. Chu, J. Liu, C. Shen, Win-stay-lose-learn promotes cooperation in the prisoner’s dilemma game with voluntary participation, PLoS One 12 (2) (2017) e0171680. [45] J. Liu, H. Meng, W. Wang, Evolution of cooperation on independent networks: the influence of asymmetric information sharing updating mechanism, Appl. Math. Comput. 340 (2019) 234–241. [46] C. Shen, J. Lu, L. Shi, Does coevolution setup promote cooperation in spatial prisoner’s dilemma game? Appl. Math. Comput. 290 (2016) 201–207. [47] M. Perc, Sustainable institutionalized punishment requires elimination of second order free-riders, Sci. Rep. 2 (2012) 344. [48] D. Helbing, A. Szolnoki, M. Perc, G. Szabo, Defector-accelerated cooperativeness and punishment in public goods games with mutations, Phys. Rev. E 81 (2010) 057104. [49] B. Robert, J.R. Peter, Punishment allows the evolution of cooperation (or anything else) in sizable groups, Ethol. Sociobiol. 13 (1992) 171–195. [50] D.G. Rand, M.A. Nowak, The evolution of antisocial punishment in optional public goods games, Nat. Commun. 2 (1) (2011) 434. [51] R. Matsuzawa, J. Tanimoto, Sanctions triggered by jealousy help promote the cooperation in spatial prisoner’s dilemma games, Chaos Solitons Fractals 110 (2018) 239–243. [52] R. Matsuzawa, J. Tanimoto, F. Eriko, Spatial prisoner’s dilemma games with zealous cooperators, Phys. Rev. E 94 (2016) 022114. [53] J. Tanimoto, H. Sagara, Relationship between dilemma occurrence and the existence of a weakly dominant strategy in a two-player symmetric game, BioSystems 90 (1) (2007) 105–114. [54] Z. Wang, S. Kokubo, M. Jusup, Dilemma strength as a framework for advancing evolutionary game theory: reply to comments on "Universal scaling for the dilemma strength in evolutionary games", Phys. Life Rev. 14 (2015) 56–58. [55] H. Ito, J. Tanimoto, Scaling the phase-planes of social dilemma strengths shows game-class changes in the five rules governing the evolution of cooperation, R. Soc. Open Sci. 5 (10) (2018) 181085. [56] J. Tanimoto, Evolutionary Games with Sociophysics: Analysis of Traffic Flow and Epidemics, Springer, 2019. [57] J. Tanimoto, Fundamentals of Evolutionary Game Theory and its Applications, Springer, 2015. [58] M. Alam, K. Nagashima, J. Tanimoto, Various error settings bring different noise-driven effects on network reciprocity in spatial prisoner’s dilemma, Chaos Solitons Fractals 114 (2018) 338–346. [59] M. Alam, K. Kuga, J. Tanimoto, Three-strategy and four-strategy model of vaccination game introducing an intermediate protecting measure, Appl. Math. Comput. 346 (2019) 408–422. [60] M. Alam, M. Tanaka, J. Tanimoto, A game theoretic approach to discuss the positive secondary effect of vaccination scheme in an infinite and well-mixed population, Chaos Solitons Fractals 125 (2019) 201–213.