The evolution of cooperation in spatial prisoner’s dilemma game with dynamic relationship-based preferential learning

Physica A 512 (2018) 598–611

Contents lists available at ScienceDirect

Physica A journal homepage: www.elsevier.com/locate/physa

The evolution of cooperation in spatial prisoner’s dilemma game with dynamic relationship-based preferential learning ∗

Jiaqin Sun a , Ruguo Fan a , , Ming Luo b , Yingqing Zhang c , Lili Dong a a

Economics and Management School, Wuhan University, Wuhan 430072, PR China School of Economics and Management, Guangxi Normal University, Guilin 541004, PR China c School of Management Science, Guizhou University of Finance and Economics, Guiyang 550025, PR China b

highlights • • • •

Introduce dynamic relationship-based preferential learning mechanism into evolutionary prisoner’s dilemma game model. Construct a method to update the strength of relationship according to mutual strategies. Both the sensitivity factor and the preference intensity has multiple effects on promotion of cooperation. There exists a trade-off between the influences of the sensitivity factor and the preference intensity.

article

info

Article history: Received 11 January 2018 Received in revised form 8 April 2018 Available online xxxx Keywords: Prisoner’s dilemma game Evolution of cooperation Learning preference Dynamic relationship

a b s t r a c t The relationships in human society are heterogeneous and dynamically change with interactions, which have a strong effect on individual’s learning behaviors. In this light we present a new mechanism of preferential learning based on dynamic relationship into evolutionary spatial prisoner’s dilemma game to further investigate the incentive mechanisms of cooperative behaviors. In detail, we consider that the strength of relationship between pairwise individuals adaptively changes according to their mutual strategies and the adjusting rate is related to individuals’ sensitivity to interactions. Based on the heterogeneous and dynamic relationship, individuals prefer neighbors with stronger relationship to learn from instead of learning randomly The learning preference is measured by the preference intensity. By means of Monte Carlo simulations, we find that both the sensitivity factor and the preference intensity have multiple effects on the evolution of cooperation. Furthermore, to validate the multiple effects in a microcosmic view, strategy transitions during the evolution are also discussed. Interestingly, we find that there exists a trade-off between the influence of the sensitivity factor and the preference intensity on the evolution of cooperation. Presented results are robust to variations of the network structures and may provide a new understanding to the emergence of cooperative behaviors. © 2018 Published by Elsevier B.V.

1. Introduction Evolutionary game theory has played an important role in studying cooperative behavior in the fields of biology, sociology, and economics [1,2]. Evolutionary games, including the prisoner’s dilemma game (PDG) [3–11], the snowdrift game (SG) [12,13], and the public goods game (PGG) [14–18] have been introduced to characterize the evolution of cooperation. Especially, the prisoner’s dilemma game and its extensions have been frequently used by researchers to describe the origin of social dilemma and explain the emergence of cooperation. For conventional PDG model, each player has two feasible strategies: cooperation (C) and defection (D). Mutual cooperation brings a payoff R (reward) for each player, while mutual ∗ Correspondence to: Economic and Management School, Wuhan University, Bayi Road 299#, Wuchang District, Wuhan, Hubei Province, PR China. E-mail address: [email protected] (R. Fan). https://doi.org/10.1016/j.physa.2018.08.105 0378-4371/© 2018 Published by Elsevier B.V.

J. Sun et al. / Physica A 512 (2018) 598–611

599

defection results in a payoff P (punishment). If players take different actions, the defector gets T (the temptation to defect) by exploiting the cooperator who only gets S (the sucker’s payoff). Accordingly, a PD game exists only when two conditions are satisfied: T > R > P > S and 2R > T + S. Thus mutual defection is the only equilibrium in a single encounter although mutual cooperation can bring larger payoffs for both players. However, the non-cooperative conclusion obviously contradicts the reality that cooperative behaviors are ubiquitous in the world. This contradiction has aroused great interest in the study of possible conditions for the emergence and promotion of cooperation. Motivated by the fact that individuals are embedded in a social network, Nowak and May originally integrated spatial structure with the PDG, which provides an escape for cooperators from exploitation [19]. In the same vein, small-world networks [20], scale-free networks [21], networks with communities [22] and a number of other topologies [23,24] were also employed in modeling social dilemma. In the study of spatial evolutionary games, it is found that heterogeneity of spatial-structured population is an effective mechanism for cooperation significantly conducive to the emergence of cooperation [25]. Up until now, in addition to the population structure, a series of mechanisms, which are capable of explaining the survivability and sustainability of cooperation, have been proposed in theoretical and experimental studies. In particular, Nowak discussed five rules [26] (kin selection, direct reciprocity, indirect reciprocity, network reciprocity, and group selection) for the evolution of cooperation. Moreover, many other available mechanisms were also thoroughly investigated, including image scoring [27], teaching activity [28], reward and punishment [29,30], memory and conformity [31], preferential learning [32], to name but a few. Although much progress has been made in resolving the conundrum of cooperation, there is still a key defect in most of the prior works. That is, in the real world, social systems are mostly associated with weighted networks and the link weight between pairwise individuals in social networks is often heterogeneous [33–35]. Therefore, it is often improper to place social dilemma models into un-weighted networks. Introducing weighted networks into the evolution of cooperation opened a new window for the subsequent researches. In contrast to the un-weighted networks, the weighted networks can reflect not only the presence or absence of a relation, but also the subjective closeness and duration of relationship [36,37] or frequency of contact [38–40] between two players. In the last few years, a number of evolutionary PD games built on weighted networks have been proposed [41–45]. For example, in order to know whether the heterogeneity of link weight can facilitate cooperation, researchers constructed simulation models on weighted networks where the link weights followed different pre-specified distributions [46–48]. The results showed that the network with heterogeneous link weights outperformed the homogeneous link weights in the facilitation of cooperation and a proper level of heterogeneity can lead to the highest cooperation level under a particular temptation b. In most of the works with weighted networks, link weights were set to directly influence the individuals’ payoffs or utilities (the individuals’ payoffs are weighted by the value of link weights), thus the evolutionary results were affected under this setting. However, the nature of social relation between a pair of individuals in social networks is likely to affect the behavioral preference even more directly. An empirical study conducted by Harrison et al. suggest that individuals are generally willing to suffer greater cost for close friends [49], in other words, the relationship has an inevitable impact on individuals’ reciprocal behaviors. Inspired by this, Xu et al. investigated the effect of relationship-based investment preference on the evolution of cooperation by modeling evolutionary PD game on weighted network where link weights represent the closeness of relationships [50]. And it is found that a moderate preference can greatly promote the diffusion of cooperation, while an extremely strong preference can hinder the cooperative behaviors conversely. In addition to the reciprocal preference, many other preferences are evidently influenced by relationships as well, such as altruistic preference, learning preference and so on. In particular, the learning preference is closely associated with relationships, a fact which is easily ignored in investigations but prevalent in the society. As an old saying goes, ‘‘He who stays near ink gets stained black’’. And that indeed is true in society. Because of intimacy and trust, we usually learn from our close friends thus our behaviors are assimilated by them. Given the above empirical observations, we introduce learning preference into the evolutionary PDG model to investigate how such relationship-based preferential learning mechanism affects the evolution of cooperation. As is the general case in studies of cooperation in networks, all neighbors share the same possibility to be chosen for players to learn from according to deterministic rules [19,51]. In contrast, under preferential learning mechanism, neighbors with stronger connections are more likely to be chosen. Importantly, relationships (link weights) in social networks are typically dynamic [52], changing in response to the strategies of individuals [53]. In return, the relationships affect the strategies of players. Therefore, for better mimicking the reality, we assume that link weights dynamically change with interactions. To address the preferential learning mechanism and avoid the influence of heterogeneity of network structure, we consider primarily the prisoner’s dilemma game on square lattice. Furthermore, for the sake of rigor, simulations are also conducted on heterogeneous networks. The results show that the introduction of preferential learning mechanism has multiple impacts on the promotion of cooperation, which are universal to both homogeneous networks and heterogeneous. As relationship-based learning preference is ubiquitous in human society, our study may shed some new light on resolving the social dilemmas in the real world. This paper is organized as follows. The model with preferential learning mechanism is constructed in Section 2. The main results and discussions are given in Section 3 and Section 4. Conclusions are given in Section 5. 2. Model In this study, we take into account the classical two-strategy prisoner’s dilemma game, in which the elements of payoff matrix follow the previous studies:R = 1, P = S = 0, and T = b (1 < b < 2), thus the game is controlled by a single

600

J. Sun et al. / Physica A 512 (2018) 598–611

parameter b that represents the temptation to defect. In order to avoid the influence of heterogeneity of degree, we use the regular L × L square lattice with periodic boundary conditions, where each node equally has four fixed neighbors. Each node on the lattice represents an individual, with links denoting the pairwise relationships between individuals. ( )The weights of links are introduced to measure the extent of closeness via the adjacency matrix of network W = wxy N ×N , where wxy ∈ [0, 1] and N denotes the size of lattice which is equal toL2 . A greater link weight in general indicates a stronger relationship between the two individuals, while the opposite case means a weaker relationship. Noteworthy, the links are undirected, namely, there is no difference between wxy and wyx . We emphasize that both self-connections and duplicate links are avoided. Before the game starts, without losing generality, the equal weight wxy = 0.5 is assigned to each link, which however will adaptively change in accordance with the interaction during the evolutionary process. Starting from an initial state that every individual x is designated either as a cooperator (sx = C ) or a defector (sx = D) with an equal probability, he is only allowed to play PDG with his immediate neighbors and can get a payoff πxy by interplaying with neighbory. In every round, by interacting with all his neighbors, every individual x is given the accumulated payoff πx according to

πx =

∑

πxy

(1)

y∈Vx

where Vx denotes the set of neighbors of every individual x. As a result of the interactions, individuals will adjust their relationships with their neighbors. In correspondence to three possible interactions: cooperation–cooperation, cooperation-defection and defection– defection, there are three link types between pairwise individuals marked as C–C, C–D and D–D respectively. Note that mutual cooperation is a win–win strategy from which both players can get a reward R, thus the relationship can be enhanced under cooperation–cooperation interaction. However, when two players adopt different strategies (cooperation–defection interaction), the link weight will be significantly decreased because the betrayal causes a heavy loss for cooperator though it brings a great payoff for defector. The link weights are updated accordingly in the following way: t +1 wxy

( ) ⎧ t t ⎨wxy + α 1 − wxy if sx = sy = C t t = wxy − αwxy if sx = C , sy = D ⎩ t wxy other w ise,

(2)

where α (0 ≤ α ≤ 1) is used to reflect the individual’s sensitivity to the interactions. A large value of α indicates that individuals are extremely sensitive to the interactions, thus the relationships are adjusted at a high rate. In the case of α = 1, mutual cooperation can promote closeness to such a maximal level that the link weight is equal to 1. Contrastingly, if the cooperator is exploited by the defector, the strength of relationship is weakened to a minimal value of 0. On the other hand, the case α = 0 implies that the strength of relationships keeps constant during the evolution regardless of the interplaying results. Note that, different from that in proceeding literature [54], the link weight increases (or decreases) slowly when wxy is large (or small), which is more consistent with the reality. By considering the adjusting rate based on sensitivity and current closeness, our link weight update rule is more appropriate to describe the feature of the evolving relationship. The strategies adopted by individuals influence the relationships between them, and in return, the alteration of the relationships eventually affects their behavioral traits with the learning preference based on relationships. In the real world, different friends have different impacts on one’s behaviors. Close friends may influence him more strongly than ordinary ones. In other words, the influences from neighbors in networks are asymmetric, which depend on the strength of relationship and will evolve with time. Rather than randomly selecting a neighbor and adopting his strategy with a probability related to payoff difference in previous studies [55], individuals make selection decisions according to a preferential rule: the neighbor with stronger relationship has a greater probability to be selected. For every neighbor y, his probability ϕxy to be selected by player x is given by: β

ϕxy = ∑

wxy

(3)

β

y∈Vx

wxy

where β (β ≥ 0) is a tuneable parameter controlling the intensity of preference. When β = 0, the neighbor is selected with an equal probability regardless of link weights, which result in the traditional scenario. When β = 1, the player will preferentially select the neighbor with a probability proportional to his link weight, that is, the neighbor with larger link weight is more likely to be selected. When β → +∞, the player will select the most intimate one for strategy learning. After the neighbor y is selected, player x will adopt y’s strategy with a probability given by the Fermi rule: P(sx ← sy ) =

1 1 + exp[(πx − πy )/K ]

(4)

where K (K ≥ 0) characterizes the noise or bounded rationality in the process of strategy adoption. K = 0 indicates the perfect rationality, while K → +∞ implies complete randomness. In line with previous works, we set K = 0.1 in our study. Simulations of the model are conducted by means of Monte Carlo (MC) simulation, where each individual has a chance on average to update his strategy. The Monte Carlo simulation procedure consists of the following three elementary steps.

J. Sun et al. / Physica A 512 (2018) 598–611

601

Fig. 1. Fraction of cooperators fc as a function of the sensitivity factor α for different temptation to defect b under the preference intensity β = 0.2(a) and β = 1(b).

Firstly, a playerx is randomly selected and acquires an accumulated payoff by interacting with all his neighbors. Subsequently, through the interactions, the weights of links combining player x with his neighbors are updated according to Eq. (2). Finally, following the rule Eq. (3), player x preferentially selects one of his neighbors to learn from and updates his strategy with a probability given by Eq. (4). Notably, the model is simulated in a synchronous manner. 3. Simulation results Initially, players are randomly distributed on the vertices of a 50 × 50 weighted square lattice with an equal probability to cooperate or defect. Meanwhile, the equal weight of 0.5 is assigned to each link at the beginning, representing a medium level of intimacy in the relationship. The equilibrium fraction of cooperators is obtained by averaging over the last 103 full MC steps of the total 104 steps after transients long enough were discarded, and the simulation results are averaged by over 10 independent runs. 3.1. The effects of the sensitivity factor on evolution of cooperation The sensitivity factor α is the key parameter for controlling the adjusting rate of the strength of relationship. Fig. 1 features the stationary fraction of cooperators fc dependent on α with different temptation to defect b. Intuitively, we can observe that, under the preferential learning mechanism, the sensitivity factor α has multiple effects on the promotion of cooperation in evolutionary PDG which are concerned with temptation b. In Fig. 1(a), the value of the preference intensity β is fixed at 0.2. It can be found that when b = 1, the fraction of cooperation fc significantly decreases with the increase of α , which indicates that adaptive relationship (α ̸= 0) will hinder the spread of cooperative behaviors. In contrast, when b (i.e. b = 2) is sufficiently large, we find that the larger value of α corresponds to the higher fc , which implies that cooperation is considerably enhanced by the rapidly adjusting relationship when selfish behaviors are encouraged (large b). However, in a wide range of b, after a dramatical increase with the increasing α , the cooperation level begins to decrease gradually since α ≈ 0.4 (Fig. 1(a)), suggesting that adjusting strength of relationship at a proper rate can promote cooperation most. Interestingly, it can be observed that regardless of the temptation b, the stationary fraction of cooperation fc almost approaches the initial cooperation level at α = 1. As individuals are extremely sensitive to interactions (α = 1), cooperators will establish very strong relationships with cooperative neighbors but extremely weak relationships with defective ones. Under the preferential learning mechanism, cooperators will certainly not learn from a defector with whom they formed extremely weak relationships and vice versa. Consequently, the cooperation level is almost constrained to the initial freezing state. For comparison, simulations were repeated under the preference intensity β = 1 and similar results can be obtained, as shown in Fig. 1(b). In addition, it is worth noting that, compared with Fig. 1(a), the turning point in Fig. 1(b) has moved left, which suggests that the optimal value of α is affected by the value of β . To get a clearer image about the impact of α , the contour plots in the b − α plane for the fraction of cooperators fc are shown in Fig. 2. For comparison, the traditional case (where individuals have no learning preference) is also shown in Panel(a). Obviously, the results shown in Panel(b) and Panel(c) also demonstrate the multiple effects of the sensitivity factor α on the facilitation of cooperation in the evolutionary PDG, which are consistent with the results shown in Fig. 1(a) and (b). An explanation for the multiple effects of the sensitivity factor α can be presented from a microcosmic view. Generally, as the players are sensitive to the interactions of neighbors, the link weights (strength of relationships) adaptively change

602

J. Sun et al. / Physica A 512 (2018) 598–611

Fig. 2. Contour plots for the fraction of cooperators fc in the parameter plane b − α under the preference intensity β = 0(a), β = 0.2(b) and β = 1(c).

Fig. 3. Transitions between different strategy pairs in the evolution of PD game for different values of the sensitivity factor α under temptation to defect b = 1(a), b = 1.1(b) and b = 2(c). Blue represents defection(D) turning into cooperation(C) and yellow represents the opposite. In all the three cases, β = 0.2. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 4. Difference between D→C and C→D in the evolution of PD game for different values of the sensitivity factor α under temptation to defect b = 1(a), b = 1.1(b) and b = 2(c). In all the three cases, β = 0.2.

with interactions. The link weight between cooperators (C–C link) will be enhanced; the strength of relationship between cooperator and defector (C–D link) will be weakened oppositely. Therefore, compared with the traditional case (α = 0), cooperators are easier to resist the invasion of defectors by forming compact clusters or pairs, which is the positive effect. In contrast, the negative effect is that, due to the weak relationship between cooperator and defector (C–D link), the diffusion of cooperation seems all the more difficult. Thus, the sensitivity factor α is a double-edged sword in facilitating the cooperation. To get further explanation about the multiple effects of the sensitivity factor α concerned with temptation b, we monitor the transitions between different strategies in the evolution for different values of the sensitivity factor α under different temptation b, as presented in Fig. 3, and the corresponding difference between D→C and C→D in all the three cases are counted in Fig. 4. Intuitively, irrespective of temptation b, the total number of transitions decrease with the increase of the

J. Sun et al. / Physica A 512 (2018) 598–611

603

Fig. 5. Fraction of cooperators fc as a function of temptation to defect b for different values of the preference intensity β under the sensitivity factor

α = 0.1(a) and α = 0.5(b).

sensitivity factor α , as shown in Fig. 3. Note that individuals adjusting their relationships with nearest neighbors at the rate α can result in difference of weight among C–C link, C–D link and D–D link. Under the preferential learning mechanism, cooperators are more likely to choose cooperative neighbors to learn from and defectors also prefer to learn from defective neighbors, so the strategy transitions decrease with the increasing α . Although the total number of transitions has the same trends with the increase of α and regardless of b, the trends of difference between D→C transitions and C→D transitions are different with different values of temptation b, as shown in Fig. 4. For temptation b = 1 (Fig. 4(a)), the defective strategy has no advantages in payoffs compared with mutual cooperation. For the traditional case that α = 0, cooperators can easily spread in the whole network. However, with the introduction of the sensitivity factor (α ̸ = 0), individuals will gradually enhance their relationships with homogeneous neighbors (C–C link and D–D link) and weaken their relationships with heterogeneous neighbors (C–D link). The gap between the strength of C–D link and D–D link hinders defectors from learning from cooperators, hence the difference between D→C and C→D decreases as α increases. For a moderate value of b, adjusting relationship at a moderate rate can yield an optimal outcome, as a too small α cannot prevent cooperators from learning from defectors (positive effect is too weak) while a too large α will hinder defectors from learning from cooperators (negative effect is too strong). Only a moderate value of α can balance the positive effect and the negative effect brought by preferential learning. Thus in the case of b = 1.1 (Fig. 4(b)), a moderate value of α ≈ 0.4 can optimally promote the cooperation. For a large value of b (i.e. b = 2), cooperators are invaded by defectors under the great temptation. Only a sufficiently large value of α makes it possible to avoid cooperators learning from defectors and to form compact cooperation clusters or pairs quickly, as presented in Fig. 4(c). 3.2. The effects of the preference intensity on evolution of cooperation In parallel, we investigate the impact of the preference intensity β on the stationary fraction of cooperators fc as a function of temptation to defect b in the PDG, as shown in Fig. 5. For comparison, simulations are repeated under α = 0.1 and α = 0.5. As expected, in both cases of α = 0.1 and α = 0.5, the equilibrium proportion of cooperators decreases with the increase of temptation to defect b, regardless of β . In particular, when β = 0, the traditional case happens, where the cooperation level is high at the beginning but then decreases quickly as b increases. For b ≈ 1.2, cooperation almost vanishes. Moreover, with a larger value of β , the cooperation level gets lower initially, which then drops more slowly. Particularly, for an extremely large β (i.e. β = 100), there is little decrease in the cooperation level as temptation b increases, which can be clearly observed from the figure that the red line is almost horizontal. As the individuals strongly prefer to learn from close friends, they will certainly learn from the closest neighbors, thus becoming immune to invasion of others. Therefore, the cooperation level is almost constrained to the initial freezing state even if the temptation to defect is large. From this perspective, our conclusion suggests that if individuals only learn from their best friend, cooperation can be maintained but not diffused. To give a clear understanding of the influence of the preference intensity β , we present in Fig. 6 the fraction of cooperation fc as a function of the preference intensity β under different values of temptation to defect b. In order to observe the influence of the preference intensity β on the cooperation fraction in detail, we limit β to a range of 0 to 2. We can see that, the preference intensity β also has multiple effects on the evolution of cooperation, which is similar to the sensitivity factor α . For temptation b = 1, the increasing β continuously lowers the density of cooperators, which indicates that a strong preference hinders the spread of cooperative behaviors in this case. In contrast, for a very large b (i.e. b = 2), the fraction of cooperators monotonously increases with the increase of β , suggesting that an enhancement of the learning preference

604

J. Sun et al. / Physica A 512 (2018) 598–611

Fig. 6. Fraction of cooperators fc as a function of the preference intensity β for different temptation to defect b under the sensitivity factor α = 0.1(a) and

α = 0.5(b).

Fig. 7. Contour plots for the fraction of cooperators fc in the parameter plane b − β under the sensitivity factor α = 0(a), α = 0.1(b) and α = 0.5(c).

promotes the survival of cooperators when defective behaviors are encouraged (large b). However, in a wide range of b, values around β ≈ 0.8 (Fig. 6(a)) and β ≈ 0.2 (Fig. 6(b)) yield an optimal cooperation level. It is clearly shown that for moderate temptation b, a too weak intensity cannot protect cooperators from the invasion of defectors and a too strong preference intensity hinders the spread of cooperation. In addition to the multiple effects, by comparing the case of α = 0.1 and α = 0.5, we can also find that the optimal value of the preference intensity β is related to the value of α . To show the impact of β more clearly, the contour plots in the b − β plane for the fraction of cooperators fc are shown in Fig. 7. For comparison, the traditional case is also shown in Panel(a). Note that the impacts of the preference intensity β on the evolution of cooperation are similar to the sensitivity factor α . From the obtained results, we can conclude that the preference intensity β also has multiple effects on the promotion of cooperation in evolutionary PDG which are concerned with temptation b. As the link weights adaptively change with interactions at rate α , the relationships between homogeneous individuals (C–C link and D–D link) are generally stronger than those between heterogeneous individuals (C–D link). With the introduction of preferential learning mechanism, a stronger learning preference β can make cooperators learn from homogeneous neighbors with greater probability and relationship is thus further strengthened due to mutual cooperation. In consequence, compact cooperation pairs or clusters are formed to resist the diffusion of defection. However, the same is true for defectors as well. As a result, the preference intensity β is also a double-edged sword in promoting the cooperation level. Given the above results which reveal that the preference intensity β has multiple effects on the evolution of cooperation in PD game, we take it necessary to investigate the phenomenon in more detail. In the same way with the case of α , we monitor the transitions between different strategy pairs in the evolution for different values of the preference intensity β under different temptation b and the corresponding difference between D→C and C→D is also counted. Results are presented in Figs. 8 and 9. In the case of b = 1, the difference between D→C and C→D decreases with an increasing β , indicating that strong learning preference will negatively influence the evolution of cooperation. For the moderate b = 1.1, the difference between D→C and C→D maximizes at β = 0.8, suggesting that an appropriate learning preference intensity can effectively

J. Sun et al. / Physica A 512 (2018) 598–611

605

Fig. 8. Transitions between different strategy pairs in the evolution of PD game for different values of the preference intensity β under temptation to defect b = 1(a), b = 1.1(b) and b = 2(c). Blue represents defection(D) turning into cooperation(C) and yellow represents the opposite. In all the three cases, α = 0.1.

Fig. 9. Difference between D→C and C→D in the evolution of PD game for different values of preference intensity β under temptation to defect b = 1(a), b = 1.1(b) and b = 2(c). In all the three cases, α = 0.1.

support the spread of cooperation. However, for the large temptation b = 2, only sufficiently strong preference can make cooperators immune to the large temptation to defect so that the large value of β yields an optimal result. The quantitative analysis has further validated the previous results. 3.3. The joint effects of the sensitivity factor and the preference intensity on evolution of cooperation Based on the above results, it is certainly attractive to investigate the impacts of the sensitivity factor α and the preference intensity β jointly on the promotion of cooperation under different temptation b, as illustrated in Fig. 10. It can be observed that different results are presented for different temptation b. As expected, for b = 1, the cooperation level fc decreases with the increase of α and β , as shown in Fig. 10(a). For b = 1.03 (Fig. 10(b)), there exists an obvious crescent-shaped district in which a higher cooperation level can be realized (Fig. 10(b)). In this crescent-shaped district, a higher cooperation level is reached in the case of a relatively larger sensitivity factor α and a smaller preference intensity β or in the case of a relatively smaller sensitivity factor α and a larger preference intensity β , which indicates that there exists a trade-off between the influences of the sensitivity factor α and the preference intensity β on the evolution of cooperation. Under the preferential learning mechanism, both compact cooperation clusters and defection clusters are more easily formed than in the traditional situation, with the compactness of clusters positively correlated with both α and β . When cooperation clusters are appropriately compact and temptation b is moderate, cooperation can be maintained and diffused. Therefore, there exists a crescent-shaped district where a higher cooperation level is reached. Naturally, as the temptation b gets larger, the crescent-shaped district gets smaller and the level of cooperation in this district gets lower (Fig. 10(c)). Moreover, with the further increase of b (Fig. 10(d)), the crescent-shaped district almost disappears. When b is sufficiently large (Fig. 10(e) and (f)), the crescent-shaped district completely disappears and the fraction of cooperation increases with the increase of α and β , which is in sharp contrast to the case of b = 1 (Fig. 10(a)). Thus, we can conclude that, when b is small, the increasing α and β will hinder the diffusion of cooperation; when b is getting larger (b > 1), both α and β taking the moderate values

606

J. Sun et al. / Physica A 512 (2018) 598–611

Fig. 10. Contour plots for the fraction of cooperators fc in the parameter plane β − α under temptation b = 1(a), b = 1.03(b), b = 1.1(c), b = 1.3(d), b = 1.9(e) and b = 2(f).

(in the crescent-shaped district) can greatly promote the cooperation level; and when b is large enough, both sufficiently large α and β are needed for more cooperators to survive. To give an insight of the trade-off between the influences of the sensitivity factor α and the preference intensity β on the evolution of cooperation, we plot snapshots under temptation b = 1.03 with different α and β . For clarity, an initial preparatory state has been applied, as shown in Fig. 11. Panels (a) to (e) depict the evolutionary outcome of α = 0.1, β = 0.2. It can be observed that the original square C-cluster is able to launch short-lived invasion into defectors (Panel(d)), but finally be eroded by defectors into many small C-clusters (Panel(e)) as the C-clusters are not compact enough to defend defectors when both α and β taking the small values. However, when both α and β taking the relatively large values, both the C-clusters and the D-clusters are too compact to be invaded, thus the evolution is almost constrained to the initial state, as demonstrated in Panel(p) to Panel(t). The high level of cooperation can only be realized in the middle two rows of panels, where α = 0.1, β = 1 (Panel(f) to Panel(j)) and α = 0.5, β = 0.2 (Panel(k) to Panel(o)). In both cases, we can see that cooperators can spread quickly in the network as the C-clusters are compact enough to resist the temptation, and at the steady state, only few defectors can survive. By comparing the four cases, it is easy to see that, in order to get a higher cooperation level, when α takes a small value (α = 0.1), the value of β needs to be relatively large (β = 1); when α takes a large value (α = 0.5), the value of β needs to be relatively small (β = 0.2); and the same is true for β . Thus, the snapshots clearly demonstrate that there is trade-off between the influences of the sensitivity factor α and the preference intensity β in the facilitation of cooperation under moderate b, which is in agreement with the result of Fig. 10(b). 4. Discussions The above results are obtained on square lattice network with the total population N = 2500 and average degree ⟨k⟩ = 4. To verify that our results are universal, the proposed model is also performed in two representative networks: ER random network and WS small world network. To make the results on different networks comparable, the population size is set the same with square lattice network and the average degree ⟨k⟩ is set to at 4, respectively. We separately verify the multiple effects of the sensitivity factor α and the preference intensity β in the facilitation of cooperation on ER network and WS network, and the results shown in Fig. 12(a),(c) and (b),(d) are quite similar to those obtained on square lattice (Fig. 1(b) and Fig. 6(b)). To show the results on different network structures in detail, we make a comparison among square lattice network, ER network and WS network. From Fig. 13, we can see that with the increase of α , the cooperation levels follow the same trend for a certain b on these three networks. And for the preference intensity β , similar results can be observed in Fig. 14. Meanwhile, due to the more effective network reciprocity brought by degree heterogeneity, there exist differences in cooperation level. As shown in Figs. 13 and 14, the cooperation level performs the best on ER network where degree is heterogeneous and the worst on the square lattice network where degree is homogeneous. Furthermore, we also extend our exploration to BA scale-free network, where degree heterogeneity is much stronger. We separately verify the effects of the sensitivity factor α and the preference intensity β in the facilitation of cooperation on BA network, with the population size N = 2500 and average degree ⟨k⟩ = 4. Results presented in Fig. 15(a) (Fig. 15(b)) clearly show the multiple effects of α (β ). We can observe that for small temptation b, the cooperation level fc decreases

J. Sun et al. / Physica A 512 (2018) 598–611

607

Fig. 11. Snapshots for strategy evolution on square lattices at different α and β at t = 0, 20, 50, 100 and 10 000 iterations. Cooperators are colored in blue and defectors are red. The panels in the upper row are for α = 0.1, β = 0.2; in the second row for α = 0.1, β = 1; in the third row for α = 0.5, β = 0.2; and in the bottom for α = 0.5, β = 1. The temptation to defect b is fixed at 1.03 . (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 12. Fraction of cooperators fc as a function of the sensitivity factor α under the preference intensity β = 1 and the preference intensity β under the sensitivity factor α = 0.5 for different temptation to defect b. The panels in the upper row are for ER network and in the bottom row are for WS network.

with the increase of α (β ); while for large temptation b, the cooperation level fc increases with the increasing α (β ) in the beginning, and then gradually decreases. Noteworthy, results on BA scale-free network are slightly different from results on the other three networks, especially when temptation b = 2. Due to the strong network reciprocity brought by degree heterogeneity, the cooperation can still survive and spread on BA scale-free network when temptation to defect is extremely large (b = 2). Therefore, moderate sensitivity factor α and preference intensity β can greatly enhance the positive effect and resist the negative effect brought by the preferential learning mechanism. Contrastingly, in the traditional situation, cooperators cannot survive on the other three networks when temptation to defect b is large enough. In order to make

608

J. Sun et al. / Physica A 512 (2018) 598–611

Fig. 13. Fraction of cooperators fc as a function of the sensitivity factor α under the preference intensity β = 1 for different temptation to defect b. Investigation of the effect of the sensitivity factor α , making a comparison among square lattice network, ER network and WS network.

Fig. 14. Fraction of cooperators fc as a function of the preference intensity β under the sensitivity factor α = 0.5 for different temptation to defect b. Investigation of the effect of the preference intensity β , making a comparison among square lattice network, ER network and WS network.

Fig. 15. Fraction of cooperators fc as a function of the sensitivity factor α under the preference intensity β = 0.5(a) and the preference intensity β under the sensitivity factor α = 0.2(b) for different temptation to defect b. Results are obtained on BA scale-free network with the average degree ⟨k⟩ = 4 and the popular size N = 2500.

cooperators survive from the invasion of defectors, large values of the sensitivity factor α and the preference intensity β are required to form compact cooperation clusters. Although results on BA network are slightly different intuitively, the multiple effects of α and β can also be observed in Fig. 15. In essence, results on ER network, WS network and BA network are qualitatively identical. Therefore, we can conclude that both the sensitivity factor α and the preference intensity β have multiple effects on the evolution of cooperation which are universal. We also verify the robustness of the joint effects of the sensitivity factor α and the preference intensity β on the evolution of cooperation on these three networks, and results are shown in Fig. 16. An obvious crescent-shaped district can be clearly

J. Sun et al. / Physica A 512 (2018) 598–611

609

Fig. 16. Contour plots for the fraction of cooperators fc in the parameter plane β -α under temptation b = 1, b = 1. 3, b = 1.3, b = 2. The panels in the upper row are for square lattice network; in the second row for ER network; in the third row for WS network; and in the bottom for BA network.

observed in (b), (e), (h) and (l), indicating that there exists a trade-off between the influences of the sensitivity factor α and the preference intensity β on the evolution of cooperation regardless of topology. Meanwhile, due to the difference in degree heterogeneity, the results on square lattice network, ER network, WS network and BA network are seems quantitatively different. The crescent-shaped district in ER network is larger than in square lattice network and WS network, which implies a higher cooperation level. Noteworthy, in the case of BA scale-free network, a crescent-shaped district exists when b = 2 and is not so obvious compared with other three cases, which are attributed to the strong degree heterogeneity. With the effective network reciprocity brought by strong degree heterogeneity, cooperation density keeps at a high level even when temptation to defect b = 2. And the influence of network reciprocity is stronger than that of preferential learning, so the crescent-shaped district not so obvious. Based on the above analysis, we can conclude that there exists a trade-off between the influences of the sensitivity factor α and the preference intensity β on the evolution of cooperation in both homogeneous network and heterogeneous network. 5. Conclusions In this paper, we have explored the evolution of cooperation in PD game on weighted networks under dynamic relationship-based preferential learning mechanism. Instead of randomly selecting the nearest neighbor to compare payoffs when updating strategy, individuals prefer to select a neighbor with stronger relationship based on the preference intensity β . Simultaneously, to consider the mechanism in a more realistic way, strength of relationship between pairwise individuals is dynamic and adaptively changes with their mutual strategies, the adjusting rate being controlled by the sensitivity factor α . By conducting Monte Carlo simulations, we have investigated the impact of the learning preference intensity β and the sensitivity factor α respectively on the promotion of cooperation. The results of simulation have shown that the introduction of the learning preference has multiple effects on the evolution of cooperation which are concerned with temptation to defect b. And similar results can also be obtained for the sensitivity factor α , as a too small α cannot prevent cooperators learning

610

J. Sun et al. / Physica A 512 (2018) 598–611

from defectors while a too large α hinders defectors from learning from cooperators. In addition, we have found that in a wide range of b, a district with a higher cooperation level in the parameter plane β − α can be observed, suggesting that there exists a trade-off between the influences of the sensitivity factor α and the preference intensity β on the evolution of cooperation. Moreover, in order to validate multiple effects, we have further analyzed the strategy transitions during the evolution in a microcosmic view. In addition, our results, not restricted solely to the square lattice network, remain valid also on heterogeneous networks, such as, ER random network, WS small world network and BA scale-free network. Our study offers a new perspective on the Prisoner’s dilemma game: how the learning preference and dynamic relationship affect the individuals’ strategies. It is human nature that people have a born tendency to learn from those who are close to them and the closeness between them dynamically changes with interactions. Our analysis suggests that this human nature has multiple effects in the facilitation of cooperation in human society. In the real world, some restrictive assumptions may not be germane. However, it is worth emphasizing that our work may provide a new explanation for the maintenance of cooperation in the real world. Acknowledgments This paper is supported by National Social Science Foundation of China (Grant No. 14ZDA062) and Humanities and National Natural Science Foundation of China (Grant No. 71601148). References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28]

J. Hofbauer, K. Sigmund, Evolutionary Games and Population Dynamics, Cambridge University Press, Cambridge, 1998. A.M. Colman, Game Theory and Its Applications: In the Social and Biological Sciences, Psychology Press, Oxford, 1995. R. Axelrod, W.D. Hamilton, The evolution of cooperation, Science 211 (1981) 1390–1396, http://dx.doi.org/10.1126/science.7466396. Z. Wang, M. Perc, Aspiring to the fittest and promotion of cooperation in the prisoner’s dilemma game, Phys. Rev. E 82 (2010) 021115, http: //dx.doi.org/10.1103/PhysRevE.82.021115. S.S. Van, F.C. Santos, T. Lenaerts, J.M. Pacheco, Reacting differently to adverse ties promotes cooperation in social networks, Phys. Rev. Lett. 102 (2009) 058105, http://dx.doi.org/10.1103/PhysRevLett.102.058105. Q. Wang, G. Chen, M. Perc, Synchronous bursts on scale-free neuronal networks with attractive and repulsive coupling, PLoS One 6 (2011) e15851, http://dx.doi.org/10.1371/journal.pone.0015851. H. Zhang, M. Small, H. Yang, B. Wang, Adjusting learning motivation to promote cooperation, Physica A 389 (2010) 4734–4739, http://dx.doi.org/10. 1016/j.physa.2010.06.023. M. Perc, Chaos promotes cooperation in the spatial prisoner’s dilemma game, Europhys. Lett. 75 (2006) 841–846, http://dx.doi.org/10.1209/epl/i200610217-3. J. Gómez-Gardeñes, M. Campillo, L.M. Floría, Y. Moreno, Dynamical organization of cooperation in complex topologies, Phys. Rev. Lett. 98 (2007) 108103, http://dx.doi.org/10.1103/PhysRevLett.98.108103. M.H. Vainstein, A.T.C. Silva, J.J. Arenzon, Does mobility decrease cooperation? J. Theoret. Biol. 244 (2007) 722–728, http://dx.doi.org/10.1016/j.jtbi. 2006.09.012. D. Han, D. Li, M. Sun, How the initial level of visibility and limited resource affect the evolution of cooperation, Sci. Rep. 6 (2016) 27191, http: //dx.doi.org/10.1038/srep27191. C. Hauert, M. Doebeli, Spatial structure often inhibits the evolution of cooperation in the snowdrift game, Nature 428 (2004) 643–646, http: //dx.doi.org/10.1038/nature02360. W.B. Du, X.B. Cao, M.B. Hu, W.X. Wang, Asymmetric cost in snowdrift game on scale-free networks, Europhys. Lett. 87 (2009) 60004, http://dx.doi. org/10.1209/0295-5075/87/60004. R. Fan, Y. Zhang, M. Luo, H. Zhang, Promotion of cooperation induced by heterogeneity of both investment and payoff allocation in spatial public goods game, Physica A 465 (2017) 454–463, http://dx.doi.org/10.1016/j.physa.2016.08.039. A. Szolnoki, M. Perc, G. Szabó, Topology-independent impact of noise on cooperation in spatial public goods games, Phys. Rev. E 80 (2009) 056109, http://dx.doi.org/10.1103/PhysRevE.80.056109. Z. Xu, Z. Wang, L. Zhang, Bounded rationality leads to equilibrium of public goods games, Phys. Rev. E 80 (2009) 061104, http://dx.doi.org/10.1103/ PhysRevE.80.061104. X.B. Cao, W.B. Du, Z.H. Rong, The evolutionary public goods game on scale-free networks with heterogeneous investment, Physica A 389 (2010) 1273–1280, http://dx.doi.org/10.1016/j.physa.2009.11.044. Z. Wang, T. Chen, Y. Wang, Leadership by example promotes the emergence of cooperation in public goods game, Chaos Solitons Fractals 101 (2017) 100–105, http://dx.doi.org/10.1016/j.chaos.2017.05.027. M.A. Nowak, R.M. May, Evolutionary games and spatial chaos, Nature 359 (1992) 826–829, http://dx.doi.org/10.1038/359826a0. X. Chen, L. Wang, Promotion of cooperation induced by appropriate payoff aspirations in a small-world networked game, Phys. Rev. E 77 (2008) 017103, http://dx.doi.org/10.1103/PhysRevE.77.017103. A. Szolnoki, M. Perc, Leaders should not be conformists in evolutionary social dilemmas, Sci. Rep. 6 (2016) 23633, http://dx.doi.org/10.1038/srep23633. P. Liu, J. Liu, Cooperation in the prisoner’s dilemma game on tunable community networks, Physica A 472 (2017) 156–163, http://dx.doi.org/10.1016/ j.physa.2016.12.059. Z. Wang, A. Szolnoki, M. Perc, Interdependent network reciprocity in evolutionary games, Sci. Rep. 3 (2013) 1183, http://dx.doi.org/10.1038/ srep01183. Z. Wang, A. Szolnoki, M. Perc, Optimal interdependence between networks for the evolution of cooperation, Sci. Rep. 3 (2013) 2470, http://dx.doi. org/10.1038/srep02470. F.C. Santos, J.M. Pacheco, T. Lenaerts, Evolutionary dynamics of social dilemmas in structured heterogeneous populations, Proc. Natl. Acad. Sci. USA 103 (2006) 3490–3494, http://dx.doi.org/10.1073/pnas.0508201103. M.A. Nowak, Five rules for the evolution of cooperation, Science 314 (2006) 1560–1563, http://dx.doi.org/10.1126/science.1133755. M.A. Nowak, R.M. May, Evolution of indirect reciprocity by image scoring, Nature 393 (1998) 573–577, http://dx.doi.org/10.1038/31225. A. Szolnoki, M. Perc, Coevolution of teaching activity promotes cooperation, New J. Phys. 10 (2008) 043036, http://dx.doi.org/10.1088/1367-2630/ 10/4/043036.

J. Sun et al. / Physica A 512 (2018) 598–611

611

[29] H.X. Yang, Z. Rong, Mutual punishment promotes cooperation in the spatial public goods game, Chaos Solitons Fractals 77 (2015) 230–234, http: //dx.doi.org/10.1016/j.chaos.2015.06.002. [30] A. Hintze, C. Adami, Punishment in public goods games leads to meta-stable phase transitions and hysteresis, Phys. Biol. 12 (2015) 46005, http: //dx.doi.org/10.1088/1478-3975/12/4/046005. [31] D. Han, M. Sun, Can memory and conformism resolve the vaccination dilemma? Physica A 415 (2014) 95–104, http://dx.doi.org/10.1016/j.physa.2014. 07.073. [32] Z.X. Wu, X.J. Xu, Z.G. Huang, S.J. Wang, Y.H. Wang, Evolutionary prisoner’s dilemma game with dynamic preferential selection, Phys. Rev. E 74 (2006) 021107, http://dx.doi.org/10.1103/PhysRevE.74.021107. [33] M.E.J. Newman, Clustering and preferential attachment in growing networks, Phys. Rev. E 64 (2001) 025102, http://dx.doi.org/10.1103/PhysRevE.64. 025102. [34] A. Barrat, M. Barthélemy, R. Pastor-Satorras, A. Vespignani, The architecture of complex weighted networks, Proc. Natl. Acad. Sci. USA 101 (2004) 3747–3752, http://dx.doi.org/10.1073/pnas.0400087101. [35] P.J. Macdonald, E. Almaas, A.L. Barabási, Minimum spanning trees of weighted scale-free networks, Europhys. Lett. 72 (2005) 308, http://dx.doi.org/ 10.1209/epl/i2005-10232-x. [36] M. Baer, The strength-of-weak-ties perspective on creativity: a comprehensive examination and extension, J. Appl. Psychol. 95 (2010) 592–601, http://dx.doi.org/10.1037/a0018761. [37] D. Han, M. Sun, An evolutionary vaccination game in the modified activity driven network by considering the closeness, Physica A 443 (2016) 49–57, http://dx.doi.org/10.1016/j.physa.2015.09.073. [38] R.L. Holbrook, C.T. Kulik, Customer perceptions of justice in service transactions: the effects of strong and weak ties, J. Organiz. Behav. 22 (2001) 743–757, http://dx.doi.org/10.1002/job.108. [39] J.J. Jones, J.E. Settle, R.M. Bond, C.J. Fariss, C. Marlow, J.H. Fowler, Inferring tie strength from online directed behavior, PLoS One 8 (2013) e52168, http://dx.doi.org/10.1371/journal.pone.0052168. [40] H. Louch, Personal network integration: transitivity and homophily in strong-tie relations, Social Networks 22 (2000) 45–64, http://dx.doi.org/10. 1016/S0378-8733(00)00015-0. [41] L. Cao, H. Ohtsuki, B. Wang, K. Aihara, Evolution of cooperation on adaptively weighted networks, J. Theoret. Biol. 272 (2011) 8–15, http://dx.doi.org/ 10.1016/j.jtbi.2010.12.008. [42] M. Iwata, E. Akiyama, Heterogeneity of link weight and the evolution of cooperation, Physica A 448 (2016) 224–234, http://dx.doi.org/10.1016/j. physa.2015.12.047. [43] M. Li, R. Deng, The evolution of cooperation in spatial prisoner’s dilemma games with heterogeneous relationships, Physica A 424 (2015) 168–175, http://dx.doi.org/10.1016/j.physa.2015.01.028. [44] B. Xu, J. Wang, The emergence of relationship-based cooperation, Sci. Rep. 5 (2015) 16447, http://dx.doi.org/10.1038/srep16447. [45] D. Li, J. Ma, D. Han, M. Sun, et al., The co-evolution of networks and prisoner’s dilemma game by considering sensitivity and visibility, Sci. Rep. 7 (2017) 45237, http://dx.doi.org/10.1038/srep45237. [46] W.B. Du, H.R. Zheng, M.B. Hu, Evolutionary prisoner’s dilemma game on weighted scale-free networks, Physica A 387 (2008) 3796–3800, http: //dx.doi.org/10.1016/j.physa.2008.02.036. [47] Z.Q. Ma, C.Y. Xia, S.W. Sun, L. Wang, H.B. Wang, J. Wang, Heterogeneous link weight promotes the cooperation in spatial prisoner’s dilemma, Internat. J. Modern Phys. C 22 (2011) 1257–1268, http://dx.doi.org/10.1142/S0129183111016877. [48] C. Xia, Z. Ma, Y. Wang, J.Q. Wang, Z.Q. Chen, Enhancement of cooperation in prisoner’s dilemma game on weighted lattices, Physica A 390 (2011) 4602–4609, http://dx.doi.org/10.1016/j.physa.2011.06.064. [49] F. Harrison, J. Sciberras, R. James, Strength of social tie predicts cooperative investment in a human social network, PLoS One 6 (2011) e18338, http://dx.doi.org/10.1371/journal.pone.0018338. [50] B. Xu, J. Wang, R. Deng, M. Li, Relational diversity promotes cooperation in prisoner’s dilemma games, PLoS One 9 (2014) e114464, http://dx.doi.org/ 10.1371/journal.pone.0114464. [51] G. Szabó, C. Tőke, Evolutionary prisoner’s dilemma game on a square lattice, Phys. Rev. E 58 (1998) 69–73, http://dx.doi.org/10.1103/PhysRevE.58.69. [52] G. Palla, A. Barabási, T. Vicsek, Quantifying social group evolution, Nature 446 (2007) 664–667, http://dx.doi.org/10.1038/nature05670. [53] Q. Pan, X. Liu, H. Bao, et al., Evolution of cooperation through adaptive interaction in a spatial prisoner’s dilemma game, Physica A 492 (2018) 571–581, http://dx.doi.org/10.1038/j.physa.2017.09.046. [54] K. Huang, X. Zheng, Z. Li, Y. Yang, Understanding cooperative behavior based on the coevolution of game strategy and link weight, Sci. Rep. 5 (2015) 14783, http://dx.doi.org/10.1038/srep14783. [55] C. Luo, Z. Jiang, Coevolving allocation of resources and cooperation in spatial evolutionary games, Appl. Math. Comput. 311 (2017) 47–57, http: //dx.doi.org/10.1016/j.amc.2017.05.009.