Evolution of indirect reciprocity in groups of various sizes and comparison with direct reciprocity

ARTICLE IN PRESS

Journal of Theoretical Biology 245 (2007) 539–552 www.elsevier.com/locate/yjtbi

Evolution of indirect reciprocity in groups of various sizes and comparison with direct reciprocity Shinsuke Suzuki, Eizo Akiyama a

Graduate School of Systems and Information Engineering, University of Tsukuba, 1-1-1 Tennoudai, Tsukuba, Ibaraki 305-0006, Japan Received 8 July 2006; received in revised form 2 November 2006; accepted 2 November 2006 Available online 9 November 2006

Abstract Recently many studies have investigated the evolution of indirect reciprocity through which cooperative action is returned by a third individual, e.g. individual A helped B and then receives help from C. Most studies on indirect reciprocity have presumed that only two individuals take part in a single interaction (group), e.g. A helps B and C helps A. In this paper, we investigate the evolution of indirect reciprocity when more than two individuals take part in a single group, and compare the result with direct reciprocity through which cooperative action is directly returned by the recipient. Our analyses show the following. In the population with discriminating cooperators and unconditional defectors, whether implementation error is included or not, (i) both strategies are evolutionarily stable and the evolution of indirect reciprocity becomes more difficult as group size increases, and (ii) the condition for the evolution of indirect reciprocity under standing reputation criterion where the third individuals distinguish between justified and unjustified defections is more relaxed than that under image scoring reputation criterion in which the third individuals do not distinguish with. Furthermore, in the population that also includes unconditional cooperators, (iii) in the presence of errors in implementation, the discriminating strategy is evolutionarily stable not only under standing but also under image scoring if group size is larger than two. Finally, (iv) in the absence of errors in implementation, the condition for the evolution of direct reciprocity is equivalent to that for the evolution of indirect reciprocity under standing, and, in the presence of errors, the condition for the evolution of direct reciprocity is very close to that for the evolution of indirect reciprocity under image scoring. r 2006 Elsevier Ltd. All rights reserved. Keywords: Evolution of cooperation; Reciprocity; Indirect reciprocity; Reputation; Prisoner’s dilemma

1. Introduction Reciprocal altruism has been considered an effective mechanism for the evolution of cooperation among genetically unrelated individuals. Two kinds of reciprocity exist: direct reciprocity and indirect reciprocity. In direct reciprocity (Trivers, 1971), an individual who helps others receives some return from the recipient directly. The two-person iterated prisoner’s dilemma game (2-IPD) has often been used in many studies (Axelrod and Hamilton, 1981; Axelrod, 1984; May, 1987; Nowak and Sigmund, 1992) as a mathematical model of this reciprocity. In particular, Axelrod showed that, in 2-IPD, a Corresponding author. Tel.: +81 298 53 5571.

E-mail address: [email protected] (S. Suzuki). 0022-5193/$ - see front matter r 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.jtbi.2006.11.002

directly reciprocal society can be realized by a strategy, titfor-tat (TFT), which cooperates on the first move and thereafter copies the opponent’s previous move. On the other hand, in the real world, an individual who helps others might not necessarily receive a return from the recipient directly, but get a return from someone else in the community indirectly. This case is called indirect reciprocity (Alexander, 1987; Nowak and Sigmund, 2005). Nowak and Sigmund (1998a,b) formulated a mathematical model of this reciprocity by introducing the ‘‘image scoring reputation criterion’’ (evaluating the reputation of individuals in a community), under which an individual’s cooperations (defections) gave her a good (bad) reputation. They showed that, in two-person giving games, a discriminating strategy, which cooperates only with an individual who has a good reputation, can prevail in the

ARTICLE IN PRESS 540

S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

population and result in stably reciprocal societies. Subsequent to these works, various studies on indirect reciprocity have been conducted (Fishman, 2003; Lotem et al., 1999; Mohtashemi and Mui, 2003; Ohtsuki, 2004; Panchanathan and Boyd, 2004; Suzuki and Toquenaga, 2005; Brandt and Sigmund, 2005, 2004; Leimar and Hammerstein, 2001; Ohtsuki and Iwasa, 2004, 2005; Panchanathan and Boyd, 2003 Milinski et al., 2001, 2002a,b; Wedekind and Milinski, 2000). In particular, some studies (Nowak and Sigmund, 1998a; Leimar and Hammerstein, 2001; Ohtsuki and Iwasa, 2004, 2005; Panchanathan and Boyd, 2003) have pointed out the evolutionary instability of indirect reciprocity under image scoring, and have shown that, in the presence of error, indirect reciprocity becomes evolutionarily stable under the ‘‘standing reputation criterion’’ (a way to evaluate individuals’ reputations aside from image scoring), under which an individual’s defections to former defectors can be justified and do not give her a bad reputation. Note that most direct and indirect reciprocity models have assumed dyadic interactions. However, interactions in human societies or ecosystems usually involve three or more individuals. Joshi (1987) and Boyd and Richerson (1988) analysed the n-person iterated prisoner’s dilemma game (n-IPD) and investigated the evolution of direct reciprocity in large groups.1 They showed that the evolution of direct reciprocity becomes difficult as group size increases. Regarding indirect reciprocity under interactions involving more than two individuals, our previous work (Suzuki and Akiyama, 2005) analysed an n-person prisoner’s dilemma game that included the effect of reputation and investigated the evolution of indirect reciprocity in large groups.2 The result showed that the evolution of indirect reciprocity becomes difficult with the increase in group size as well as direct reciprocity does. However, the study on indirect reciprocity under nperson interaction (Suzuki and Akiyama, 2005) investigated, using agent-based simulation, the evolution of indirect reciprocity only under image scoring where implementation error is not included. So in this paper, we investigate analytically the evolution of indirect reciprocity under n-person interactions, where implementation error is included, and analysed not only the case under 1 A reciprocal player is likely to cooperate with a former cooperator. In Joshi (1987), Boyd and Richerson (1988), and this study, reciprocity in a group of more than two individuals is defined as a situation in which a former cooperator is likely to be cooperated with. In a group of more than two, it could be that a former cooperator is not cooperated with if most of the other members in the group are former defectors. 2 Milinski et al. (2002a) and Panchanathan and Boyd (2004) dealt with a model in which players play alternately the n-person prisoner’s dilemma game and the two-person giving game where a discriminator can punish only a certain defector. They showed that a reputation facilitates the evolution of cooperation in large groups. On the other hand, in our model, an individual plays only the n-person game. Compared with the above studies (Milinski et al., 2002a; Panchanathan and Boyd, 2004), our approach allows us to investigate the evolution of indirect reciprocity in large groups where a discriminator cannot punish only a certain defector.

image scoring but also the case under standing. Furthermore, we compare the evolutionary phenomena of indirect reciprocity with that of direct reciprocity. Many studies of direct and indirect reciprocity have been conducted; however, to our knowledge, the relation between the two kinds of reciprocity has not been revealed. For two-person case, Nowak and Sigmund (1998b) and Brandt and Sigmund (2006) briefly mentioned an analogy between the evolutionary phenomena of the two kinds of reciprocity. In this paper, we investigate the n-person case. The rest of the paper is organized as follows. In the second section, we present analyses of the evolutionary phenomena of indirect reciprocity in groups of various sizes and comparisons with the evolutionary phenomena of direct reciprocity in the population consisting of the strictest discriminating strategy and the unconditionally defective strategy. Next, we check the evolutionary stability of indirect reciprocity for various group size in the population including the unconditionally cooperative strategy in addition to the above two strategies. Finally, we summarize these results. 2. Evolutionary phenomena of indirect reciprocity in the population with strictest discriminators and unconditional defectors In this section, we aim to investigate analytically the evolutionary phenomena of indirect reciprocity in groups of various sizes and compare the phenomena with that of direct reciprocity in the population consisting of strictest discriminators and unconditional defectors. For this purpose, we analyse an n-person reputation prisoner’s dilemma game (n-RPD) (Suzuki and Akiyama, 2005)—an n-person prisoner’s dilemma game in which individuals do not interact repeatedly and the respective reputations of individuals affect the decision-making process, and an nperson iterated prisoners dilemma game (n-IPD) (Joshi, 1987; Boyd and Richerson, 1988). In both n-RPD and nIPD, several rounds of the n-person prisoner’s dilemma game are played. In n-RPD, the several rounds of the game are played with n-individual groups which are randomly formed in every round, and, in n-IPD, the several rounds of the game are played in the same group of n-individuals (e.g. Fehr and Gachter, 2000). 2.1. Basic model Consider a population composed of infinite number of individuals. Each individual in the population has her own reputation either G (good) or B (bad). We call an individual whose reputation is G as a ‘‘G-individual’’ and whose reputation is B as a ‘‘B-individual’’. Furthermore, the reputation of each individual is known to all individuals in the population. Each generation comprises some rounds. After the first round, subsequent rounds occur with probability w (05wo1), i.e. the expected value of the number of rounds

ARTICLE IN PRESS S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

in a generation is 1=ð1  wÞ. At the beginning of each generation, the reputation of each individual is G. Moreover, with gi we denote the frequency of G-individuals at round i. In each round, all individuals are divided randomly into groups, each of which comprises n ðX2Þ individuals and play an n-person prisoner’s dilemma game in each group.3 In this game, each of the individuals chooses either to ‘‘cooperate (C)’’ or ‘‘defect (D)’’. In this paper, we assume that the payoffs for a cooperator, V ðCjkÞ, and that for a defector, V ðDjkÞ, when there are k cooperators in the group, are, as in Boyd and Richerson (1988, 1992), Fehr and Schmidt (1999), Hauert et al. (2002a,b), Semman et al. (2003), etc., linear functions k V ðCjkÞ ¼ b  c, n V ðDjkÞ ¼

k b, n

(1)

(2)

where b4c40 and c4b=n. This type of game is called public goods game (e.g. Hauert et al., 2002a,b). Suppose that the population consists of a discriminating strategy, DIS a (a ¼ 1; . . . ; n  1), and the unconditionally defective strategy, ALLD, in fractions p and 1  p, respectively. An individual who uses DIS a , called a DIS a individual, cooperates if a or more opponents in the group are G-individuals. On the other hand, an individual who uses ALLD, called an ALLD-individual, always defects. In this game, ALLD is evolutionarily stable strategy (as is shown later). On the other hand, among the discriminating strategies, only DIS n1 is a candidate for evolutionarily stable strategy, because other DIS a -individuals (aon  1) cooperate even if one B-individual enters the group and so they cannot resist invasion by ALLD-individuals. An individual who adopts the DIS n1 strategy cooperates if and only if all opponents in the group are G-individuals. Therefore, for this study, we adopt the strictest strategy, DIS n1 , as a discriminating strategy.4 In Sections 2.2 and 2.3, we investigate the case with implementation error. Implementation error is introduced with the parameter  (0o51). With the probability , an individual who intends to cooperate fails to cooperate due to a lack of resources or a mistake etc. We, as in Panchanathan and Boyd (2003), Fishman (2003), Brandt 3 In the case of indirect reciprocity, the setting that the n-person prisoner’s dilemma game is played in all groups simultaneously in each round might not be very realistic. It might be more realistic that, in each round, some n individuals are picked up randomly and play the game (e.g. Nowak and Sigmund, 1998a; Suzuki and Akiyama, 2005). However, Nowak and Sigmund (1998b) have claimed that, in two-person case, the difference between the two case does not change the basic result. Furthermore, in n-person case, the result of this study shown in the later differs little from that in Suzuki and Akiyama (2005). 4 Of course, it may be possible for the mixed strategy of ALLD and DIS a (aon  1) to be evolutionarily stable as Boyd and Richerson (1988), Joshi (1987), Molander (1992) showed. In our future work, we will investigate the case in which there exists a DIS a strategy ðaon  1Þ.

541

and Sigmund (2004), do not consider errors which cause unintentional cooperation, i.e. an individual who intends to defect never fails to defect. Furthermore, the perception error (e.g. Takahashi and Mashima, 2006) is also not included. In other words, an individual who intends to cooperate succeed in cooperation with probability ^ ¼ 1   (05^o1). In this study, we mainly use the probability of the success, ^. In Appendix C, we mention the case without error, i.e. ^ ¼ 1. 2.2. Under the image scoring reputation criterion In this game, the reputation of opponents affects the decision-making process. How is the reputation assigned to each individual? Here, we adopt ‘‘image scoring’’ (IMAGE) as a reputation criterion, which prescribes how to judge the reputation of others on the basis of the others’ past action. Under the criterion, IMAGE, which was used in Nowak and Sigmund (1998a,b), the reputation of an individual who has defected becomes B and that of an individual who has cooperated becomes G. Under IMAGE, a DIS n1 -individual cooperates and becomes a G-individual at round i if and only if she belonged to a group in which all the n  1 opponents have reputation G and she succeeded in cooperation at round i  1, for which the probability is ^ðgi1 Þn1 . Recall that gi1 represents the frequency of G-individuals at round i  1 and p indicates the frequency of DIS n1 -individuals in the population. Therefore, the frequency of G-individuals at round i is ( 1 if i ¼ 1; gi ¼ (3) n1 if iX2: p  ^ ðgi1 Þ for i ¼ 1; 2; . . . . When n is large, gi diminishes rapidly with time. Since we consider the case with implementation error, i.e. 05^o1, limi!1 gi ¼ g1 ¼ 0. In this case, the expected payoff for an ALLD- and a DIS n1 -individual at round 1 are b ^ ðn  1Þp, f ð1Þ ALLD ¼  n

(4)

b ^ ½1 þ ðn  1Þp  ^ c, f ð1Þ (5) DIS n1 ¼  n (see Appendix A.1). Furthermore, the expected payoffs for an ALLD- and a DIS n1 -individual at round i (iX2) are f ðiÞ ALLD ¼ 0,

(6)

^ðgi Þn1 ðb  cÞ, f ðiÞ DIS n1 ¼ 

(7)

(see Appendix A.1). We define, through this section, the fitness for ALLDindividuals, f ALLD , and that for DIS n1 -individuals, f DISn1 , as the expected value of the total payoff received by ALLDand DIS n1 -individuals, respectively, during a generation. The fitness for ALLD-individuals and for DIS n1 -individuals, expected value of the total payoff during a

ARTICLE IN PRESS S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

542

generation, are calculated as follows: ð2Þ ð3Þ 2 f ALLD ¼ f ð1Þ ALLD þ w  f ALLD þ w  f ALLD

þ w3  f ð4Þ ALLD þ    ,

ð8Þ

ð2Þ ð3Þ 2 f DISn1 ¼ f ð1Þ DIS n1 þ w  f DISn1 þ w  f DISn1 3

þw 

f ð4Þ DIS n1 f ðiÞ ALLD

þ .

ð9Þ

f ðiÞ DIS n1

Assuming that and (iX3) approximate to ð1Þ f ð1Þ DISn1 and f ALLD , respectively, f ALLD and f DISn1 are ð2Þ f ALLD ¼ f ð1Þ ALLD þ w  f ALLD þ

w2 ð1Þ f , 1  w ALLD

(10)

w2 ð1Þ f . (11) 1  w DISn1 We confirmed, by numerical calculation, that this approximation does not essentially change the stability condition of DIS n1 and the basin of the DIS n1 attractor especially for large n (see Fig. 2). Considering g1 ¼ 0 and Eqs. (4)–(7), the fitness values are written as

ð2Þ f DISn1 ¼ f ð1Þ DIS n1 þ w  f DISn1 þ

b f ALLD ¼ ^ ðn  1Þp, n

(12)

b f DISn1 ¼ ^ ½1 þ ðn  1Þp  ^c þ w^ð^pÞn1 ðb  cÞ. (13) n Note that g2 ¼ p  ^ gn1 ¼ ^p. 1 At the end of each generation, individuals leave their offspring depending on their own fitness. An individual who achieves a higher fitness leaves more offspring according to natural selection. To investigate the dynamics of the frequency of the two types of individuals, as in Nowak and Sigmund (1998b), Panchanathan and Boyd (2003), we use replicator dynamics (Taylor and Jonker, 1978), p_ ¼ ðf DISn1  f^Þp,

(14)

where f^ ¼ p  f DISn1 þ ð1  pÞ  f ALLD is the average fitness of all individuals in the population. Furthermore, we represent the value of p in the initial generation as pð0Þ. By substituting the fitness values, Eqs. (12)–(13), into Eq. (14), the replicator equation becomes p_ ¼ pð1  pÞhðpÞ,

(15)

where hðpÞ ¼ f DISn1  f ALLD ¼ ^½b=n  c þ ^n1 pn1 wðb  cÞ.

ð16Þ

Given the above equation, p ¼ 0 and 1 are clearly equilibria (fixed points). Moreover, hð0Þo0, and dhðpÞ=dp40 if 0opo1. Consequently, if hð1Þ40, i.e. c 1 þ ^n1 nw o , b nð1 þ ^n1 wÞ

(17)

which indicates that the cost to benefit ratio of cooperation is sufficiently small (see Fig. 2(a)), the equilibria at p ¼ 0; 1,

are stable and there exist one unstable internal equilibrium at  1=ðn1Þ c  b=n , (18) pu ¼ n1 ^ ðb  cÞw which is a solution of the equation, hðpÞ ¼ 0 (see Fig. 1). However, for n ¼ 2, we need not use the approximation given in Eqs. (10)–(11). Panchanathan and Boyd (2003) have shown that, for n ¼ 2, the stability condition of DIS given in Eq. (17) is c 1 þ ^w o , b 2 and the value of pu given in Eq. (18) is

(19)

2c  b . (20) ^bw Note that b and c in Panchanathan and Boyd (2003) correspond to b=2 and c  b=2 in this paper. In summary, there exist two stable equilibria at p ¼ 0; 1 and one unstable equilibrium at p ¼ pu 2 ð0; 1Þ. The frequency of DIS n1 individuals, p, converges to 1 if pð0Þ4pu and to 0 if pð0Þopu . Consequently, pu is the threshold frequency of DIS n1 -individuals to realize reciprocal societies. Intuitively speaking, if sufficiently many DIS n1 -individuals exist in the population, a reciprocal society in which all individuals adopt DIS n1 strategy will eventually be realized; otherwise, a defective society in which all individuals adopt ALLD strategy will eventually appear. On the other hand if hð1Þp0 i.e. the cost to benefit ratio of cooperation is not sufficiently small (see Fig. 2 (a)), there exists one stable equilibrium at p ¼ 0 and one unstable equilibrium at p ¼ 1. In this case, the defective society eventually will appear even if there are initially only few ALLD-individuals. Up to now, we have focused on the existence of equilibria and their stability. What if group size, n, changed? As is seen in Fig. 2(a), the larger the group size becomes, the more restrictive the condition becomes for the existence of the unstable equilibrium at pu 2 ð0; 1Þ, i.e. the stability condition of DIS n1 becomes more restrictive. Moreover, when the unstable equilibrium exists, we can illustrate the relation between the value of the unstable equilibrium at pu and group size using Eq. (18). Fig. 2(b) shows the value of the unstable equilibrium at pu as a function of group size, n. As is seen in the figure, the value of pu increases drastically with the increase in group size. The increase in the value of pu means the threshold frequency of DIS n1 individuals necessary for reciprocal society increases. In this sense, the condition that allows the evolution of indirect reciprocity becomes more restrictive as group size increases. pu ¼

2.3. Under the standing reputation criterion So far, IMAGE has been served as the reputation criterion. To see the effect of reputation criterion, here, we

ARTICLE IN PRESS S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

543

0.1

. p

0.05

0

-0.05

-0.1 0

0.2

pu

0.4

0.8

1

p Fig. 1. p_ is plotted as a function of p under IMAGE (: stable equilibrium; : unstable equilibrium. group size, n ¼ 4; the benefit produced by cooperation, b ¼ 3; the cost of cooperation, c ¼ 1; the probability that subsequent rounds occur, w ¼ 0:99; and the probability that an individual who intends to cooperate succeeds in cooperation, ^ ¼ 0:99).

adopt ‘‘standing’’ (STAND) as the reputation criterion. Under the criterion, STAND, which was presented by Sugden (1986), defections to former defectors are not judged to be B. Because the original definition of STAND cannot be used without modification in an n-person game, we extend its definition as follows: (1) Reputation of an individual who has cooperated becomes G. (2) Reputation of an individual who has defected in the group with no B-individual becomes B. (3) Reputation of an individual who has defected in the group with some B-individuals does not change. Given the above definition of STAND, the frequency of Gindividuals at round i is (see Appendix A.2) ( 1 if i ¼ 1; gi ¼ (21) ^p if iX2 for i ¼ 1; 2; . . . ; where p is the frequency of DIS n1 individuals in the population. The frequency, gi , is the same as that under IMAGE for ip2 (see Eq. (3)). By substituting the values (21) into Eqs. (4)–(11), we get the fitness for ALLD-individuals, f ALLD , and that for DIS n1 -individuals, f DISn1 , b f ALLD ¼ ^ ðn  1Þp, n

(22)

b w ð^pÞn1 ^ ðb  cÞ. f DISn1 ¼ ^ ½1 þ ðn  1Þp  ^c þ n 1w (23) Note that the fitness for ALLD-individuals and for DIS n1 individuals are defined as the expected value of the total payoff during a generation given in Eqs. (8)–(9). Moreover, in this case, we need not use the approximation given in

pÞn1 ^ðb  cÞ for iX2 Eqs. (10)–(11) because f ðiÞ DIS n1 ¼ ð^ under STAND. Following the same procedure given in the case under IMAGE, we can confirm that, if the cost to benefit ratio of cooperation is sufficiently small, i.e. c 1 þ ^n1 nw0 o , b nð1 þ ^n1 w0 Þ

(24)

where w0 ¼ w=ð1  wÞ (see Fig. 2(a)), there exist two stable equilibria, p ¼ 0; 1, and the one unstable internal equilibrium, p ¼ pu :  1=ðn1Þ c  b=n pu ¼ n1 , (25) ^ ðb  cÞw0 where w0 ¼ w=ð1  wÞ, for the replicator dynamics. If pð0Þ4pu , a reciprocal society will eventually be realized; otherwise, defection will prevail in the population. On the other hand if hð1Þp0 i.e. the cost to benefit ratio of cooperation is not sufficiently small (see Fig. 2(a)), there exists one stable equilibrium at p ¼ 0 and one unstable equilibrium at p ¼ 1. In this case, defection will dominate the society even if there are initially only few ALLDindividuals. Comparing the value of pu under IMAGE (see Eq. (18)) and that under STAND (see Eq. (25)), we can find, for any group size, that the value of the unstable internal equilibrium under STAND is smaller than that under IMAGE (see Fig. 2(b)). Because an indirectly reciprocal society will be realized if pð0Þ4pu , a decrease in the value of pu means that the threshold frequency of DIS n1 -individuals that is necessary for reciprocal society decreases. Moreover, the condition for DIS n1 to be stable under IMAGE is more restrictive than that under STAND (see Fig. 2(a)). Consequently, the condition that allows the evolution of indirect reciprocity under IMAGE is more restrictive than that under STAND.

ARTICLE IN PRESS S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

544

strategy, ALLD, and the strictest discriminating strategy, DIS n1 , in fractions p and 1  p, respectively. An individual who uses DIS n1 strategy, in n-IPD, cooperates on the first move and then cooperates on each subsequent move if the n  1 opponents in the group chose cooperation during the previous round. This strategy, in n-IPD, is usually called TFT or TFT-like strategy.

1 0.8 STAND

0.6 c/b

IMAGE∗

0.4 0.2 IMAGE

0 2

5

10

15

20 25 30 Group Size, n

35

40

45

50

1 0.8 IMAGE∗

pu

0.6

STAND

IMAGE

2.4.1. Without implementation errors First, we investigate the case without implementation errors, i.e. DIS n1 -individuals never fail to cooperate. At the first round, as in the n-RPD, all DIS n1 individuals and no ALLD-individual cooperate. Thus, the expected payoff for an ALLD-individual and a DIS n1 individual at round 1 are the same as in n-RPD (see Eqs. (4)–(5) with ^ ¼ 1). At later rounds, f ðiÞ ALLD ¼ V ðDj0Þ (iX2) because DIS n1 individuals in the group with some ALLD-individuals never cooperate. Moreover, only DIS n1 -individuals belonging to a group with no ALLD-individual, for which the probability is pn1 , carry on the cooperation. Therefore, the expected payoff for an ALLD and a DIS n1 -individual at round iX2 are

0.4

f ðiÞ ALLD ¼ V ðDj0Þ ¼ 0,

0.2

n1 V ðcjnÞ þ ð1  pn1 ÞV ðDj0Þ ¼ pn1 ðb  cÞ. f ðiÞ DIS n1 ¼ p

(26)

(27)

0 2

5

10

15

20 25 30 Group Size, n

35

40

45

50

Fig. 2. (a) Stability condition of DISn1 given in Eqs. (17), (19) and (24) (w ¼ 0:90 and ^ ¼ 0:90): solid line indicates the condition under IMAGE derived analytically with the approximation, dashed line represents that under IMAGE given by numerical calculation without the approximation, and dash-dotted line indicates that under STAND. Below the line, DISn1 is stable and the unstable internal equilibrium at pu exits. (b) Relation between group size, n, and the value of the unstable internal equilibrium at pu (b ¼ 0:85  n, c ¼ 1, w ¼ 0:90, and ^ ¼ 0:90): solid line indicates the value of pu under IMAGE derived analytically with the approximation, dashed line represents that under IMAGE given by numerical calculation without the approximation, and dash-dotted line indicates that under STAND. Note that the solid line and the dashed line almost overlap.

2.4. Evolutionary phenomena of direct reciprocity To investigate differences between indirect and direct reciprocity, we introduce the evolutionary phenomena of nIPD investigated in Joshi (1987), Boyd and Richerson (1988), which is related to direct reciprocity. In n-IPD, each individual plays the game in the same group throughout a generation. Other settings, payoff function and the probability that subsequent rounds occur etc., are the same as that in n-RPD. In order to compare with n-RPD, we assume that there exist, in the population, the unconditionally defective

Hence, the fitness for an ALLD- and a DIS n1 -individual are given as b f ALLD ¼ ðn  1Þp, n

(28)

b w pn1 ðb  cÞ. f DISn1 ¼ ½1 þ ðn  1Þp  c þ n 1w

(29)

Note that the fitness for ALLD-individuals and for DIS n1 individuals are defined as the expected value of the total payoff during a generation given in Eqs. (8)–(9). Eqs. (28) and (29) show that, when the implementation error is not included (^ ¼ 1), the fitness for ALLDindividuals in n-IPD is identical to that in n-RPD under STAND given in Eq. (22) and that the fitness for DIS n1 individuals in n-IPD is identical to that in n-RPD under STAND given in Eq. (23). Therefore, when the population consists of DIS n1 - and ALLD-individuals and implementation error is not included, the condition for the evolution of direct reciprocity is equivalent to that for the evolution of indirect reciprocity under STAND for any group size. 2.4.2. With implementation errors Next, we analyse the case including implementation errors, i.e. DIS n1 -individuals sometimes fail to cooperate. In order to compare with n-RPD, we consider only one type of error which causes failures of a cooperation and which does not cause failures of a defection.

ARTICLE IN PRESS S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

By numerical calculation, we get the stability condition of DIS n1 and the value of the unstable internal equilibrium at pu . Fig. 3 shows the stability condition and the value of pu , and those in n-RPD under IMAGE. As shown in the figure, when implementation error is included, the stability condition of DIS n1 and the value of pu in nIPD are very close to those in n-RPD under IMAGE. Namely, when the population consists of DIS n1 - and ALLD-individuals and implementation error is included, the condition for the evolution of direct reciprocity is very close to that for the evolution of indirect reciprocity under IMAGE for any group size. 2.5. Summary of results We have analysed the evolution of indirect reciprocity using the n-person reputation prisoner’s dilemma game

a

1 0.8 n-IPD with error

c/b

0.6 0.4 0.2

IMAGE

0 2

b

5

10

15

20 25 30 Group Size, n

35

20 25 30 Group Size, n

35

40

45

50

0.8

pu

0.6

IMAGE

0.2

n-IPD with error

(n-RPD), and have compared the results of the analyses with those of direct reciprocity using the n-person iterated prisoner’s dilemma game (n-IPD). In this section, we have focused on the case where the population consists of only the unconditionally defective strategy, ALLD, and the strictest discriminating strategy, DIS n1 . 2.5.1. Evolutionary phenomena of indirect reciprocity We have revealed that the following three features exist in the evolution of indirect reciprocity whether implementation error is included or not. First, if cost to benefit ratio of cooperation is sufficiently small, the bistable community is formed; that is, both the defective strategy and the discriminating strategy are evolutionarily stable. In this case, if there initially exists sufficient many discriminators, the indirectly reciprocal society will eventually be realized; otherwise, the defective society will appear. Second, the evolution of indirect reciprocity becomes more difficult as group size increases. With the increase in group size, the condition for indirect reciprocity to be stable becomes more restrictive and the initial frequency of discriminators required by the formation of the indirectly reciprocal society becomes larger to the inverse of group size minus one, ðn  1Þ1 , power just like Boyd and Richerson (1988). Finally, comparing the effect of reputation criteria on the evolution of indirect reciprocity, we have found that standing, which judges defections to former defectors not to be unjustified, has an advantage to the evolution of indirect reciprocity in comparison with image scoring which judges the defections to be unjustified unconditionally. 2.5.2. Comparison with direct reciprocity Next, comparing indirect reciprocity with direct reciprocity, we have shown the following. First, in the case without implementation error (i.e. individuals who intend to cooperate always succeed in cooperation), for any group size, the condition for the evolution of direct reciprocity is equivalent to that for the evolution of indirect reciprocity under standing. Second, in the case with implementation error (i.e. individuals who intend to cooperate sometimes fail to cooperate), for any group size, the condition for the evolution of direct reciprocity is very close to that for the evolution of indirect reciprocity under image scoring.

1

0.4

545

0 2

5

10

15

40

45

50

Fig. 3. (a) Stability condition of DISn1 (w ¼ 0:90 and ^ ¼ 0:90): solid line indicates the condition in n-RPD under IMAGE, and dashed line represents that in n-IPD with error. Below the line, DISn1 is stable and the unstable internal equilibrium at pu exits. (b) Relation between group size, n, and the value of the unstable internal equilibrium at pu (b ¼ 0:85  n, c ¼ 1, w ¼ 0:90, and ^ ¼ 0:90): solid line indicates the value of pu in n-RPD under IMAGE and dashed line represents that in n-IPD with error.

3. Evolutionary stability of indirect reciprocity in the population including unconditional cooperators In the previous section, we have investigated the evolutionary phenomena of n-RPD and n-IPD in the population consisting only of unconditional defectors and of strictest discriminators. However, recent studies (e.g. Panchanathan and Boyd, 2003; Ohtsuki and Iwasa, 2004; Brandt and Sigmund,

ARTICLE IN PRESS 546

S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

2006) on indirect reciprocity, which analysed two-person case, have pointed out that the presence of unconditional cooperators plays a crucial role on the evolutionary phenomena, especially on the stability of indirect reciprocity. That is, if an indirectly reciprocal society is once established, unconditional cooperators can spread over the society due to random drift because there is no difference between the fitness for unconditional cooperators and the fitness for discriminators. In this case, the society in which unconditional cooperators have spread is eventually taken over by unconditional defectors. Furthermore, in the population including unconditional cooperators, the existence of implementation error has a serious influence on the evolutionary phenomena. For example, Panchanathan and Boyd (2003) showed that, if implementation error is included, the indirect reciprocity based on image scoring is no longer persistent whereas that based on standing is stable. Here, in this section, we verify the evolutionary stability of indirect reciprocity, i.e. the discriminating strategy DIS n1 , in the population also including unconditional cooperators called ALLC. Since indirect reciprocity is clearly not evolutionarily stable in the absence of implementation error, which is less plausible situation, we focus on the case in the presence of error. As the stability of DIS n1 against ALLD has already been analysed in the previous section, in this section, we first investigate the stability of DIS n1 against ALLC and finally show the condition for DIS n1 to be evolutionarily stable. 3.1. Evolutionary stability of DIS n1 against ALLC in n-RPD under IMAGE First, we analyse the evolutionary stability of DIS n1 against invasion by ALLC in n-RPD under IMAGE. For this purpose, we consider the situation where a few ALLCindividual enter the population dominated by DIS n1 individuals and where the frequency of the invader is sufficiently small for us to presume that each individual always interacts only with incumbent DIS n1 -individuals. In the reminder of this section we always consider this situation. Let f ðDSjDSÞ and f ðACjDSÞ be the fitness for DIS n1 - and ALLC-individuals in this situation, respectively. Then the condition for DIS n1 to be stable against invasion by ALLC is f ðDSjDSÞ 4f ðACjDSÞ .

f ðDSjDSÞ ¼ f ð1Þ ðDSjDSÞ þ

w f ð1Þ , 1  w ðDSjDSÞ

(32)

where w represents the probability that subsequent rounds occur. We confirmed that, by numerical calculation, this approximation does not change essentially the stability condition of DIS n1 against ALLC (see Appendix D). At the first round, both DIS n1 - and ALLC-individuals ð1Þ intend to cooperate and so f ð1Þ ðDSjDSÞ ¼ f ðACjDSÞ . Since we consider the case where we can assume that each individual interacts only with DIS n1 -individuals, the frequency of G-individuals among DIS n1 -individuals can be regarded as the same as that in the population with only DIS n1 -individuals given in Eq. (3) with p ¼ 1. Namely the frequency of G-individuals among DIS n1 -individuals, g1 ¼ 0. In this case, considering Eq. (7), f ð1Þ ðDSjDSÞ is 0. On the other hand, f ð1Þ is ðACjDSÞ b n2 0 ^ ½1 þ g1 f ð1Þ g1 ðn  1Þ  ^ c, ðACjDSÞ ¼  n

(33)

(see Appendix B.1) where g01 is the frequency of Gindividuals among ALLC-individuals. Under IMAGE, g01 ¼ ^ because ALLC-individuals always intend to cooperate. Considering g1 ¼ 0 and g01 ¼ ^ , f ð1Þ ðACjDSÞ becomes 8   b > > > if n42; < ^ n  c (34) ¼ f ð1Þ ðACjDSÞ > ^ b > > : ð1 þ ^Þ  ^c if n ¼ 2: 2 n2 (Note that g1 ¼ 1 for n ¼ 2.) ð1Þ ð1Þ Since f ðDSjDSÞ ¼ f ð1Þ ðACjDSÞ and f ðDSjDSÞ ¼ 0, DIS n1 is evolutionarily stable against invasion by ALLC if f ð1Þ ðACjDSÞ o0. If group size is greater than two (n42), the

inequality f ð1Þ ðACjDSÞ o0 always holds. On the other hand, for n ¼ 2, it has been shown, without the approximation given in Eqs. (31)–(32), that DIS n1 is evolutionarily stable against invasion by ALLC if c=b4ð1 þ ^wÞ=2 which is a very strict condition (Panchanathan and Boyd, 2003). Note that b and c in Panchanathan and Boyd (2003) correspond to b=2 and c  b=2, respectively, in this paper. Consequently, DIS n1 is stable against ALLC if group size is greater than two, or if group size is two and if the cost to benefit ratio of cooperation is extremely large (see Fig. 4).

(30)

Hereafter, we represent the expected payoff for DIS n1 individuals in the population at round i as f ðiÞ ðDSjDSÞ and that for ALLC-individuals as f ðiÞ . Through this section, in ðACjDSÞ order to analyse the stability of DIS n1 analytically, we assume that the fitness, which is defined as the expected total payoff, for two types of individuals in the population approximate to w f ð1Þ f ðACjDSÞ ¼ f ð1Þ , (31) ðACjDSÞ þ 1  w ðACjDSÞ

3.2. Evolutionary stability of DIS n1 against ALLC in nRPD under STAND Here we check the stability of DIS n1 against ALLC in n-RPD under STAND. As in the case under IMAGE, we consider the situation where a few ALLC-individual enter the population dominated by DIS n1 -individuals and the frequency of the invader is sufficiently small for us to presume that each individual always interacts with only incumbent DIS n1 -individuals.

ARTICLE IN PRESS S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

547

þ^dÞ=f2ð1  ^Þg where d ¼ f1  ð1 ^Þ2 wg1 . On the other hand, the stability condition for more than two-person cases is illustrated in Fig. 4 by numerical calculation. As is shown in the figure, in n-RPD under STAND, DIS n1 is almost always stable against invasion by ALLC regardless of group size. Only when the cost to benefit ratio of cooperation is extremely small, DIS n1 is unstable against ALLC.

1

0.8

c/b

0.6

0.4

STAND IMAGE and n-IPD

3.3. Evolutionary stability of DIS n1 against ALLC in n-IPD

0.2

0 2

5

10

15

20 25 30 Group Size, n

35

40

45

50

Fig. 4. Stability condition of DISn1 against ALLC (w ¼ 0:90 and ^ ¼ 0:90): the solid line indicates the condition in n-RPD under IMAGE and that in n-IPD, and the dash-dotted line represents that in n-RPD under STAND. Above the line, DISn1 -individuals can resist invasion by a few ALLC-individuals. Below the line, DISn1 -individuals cannot resist invasion by a few ALLC-individuals or the condition for the n-person prisoner’s game, c4b=n, does not hold.

As in the case under IMAGE, both ALLC- and DIS n1 individuals intend to cooperate at the first round and so ð1Þ f ð1Þ ðACjDSÞ ¼ f ðDSjDSÞ . Since we consider the case where we can presume that each individuals interacts only with incumbent DIS n1 individuals, in n-RPD under STAND, the frequency of Gindividuals among DIS n1 -individuals can be regarded as the same as that in the population with only DIS n1 individuals given in Eq. (21) with p ¼ 1. Namely, the frequency of G-individuals among DIS n1 -individuals, gi ¼ ^ (iX2). On the other hand, among ALLC-individuals, Gindividuals lose their good reputation only when they fail to cooperate in a group with no B-individuals, the ð1  ^Þ. Furthermore, ALLCprobability of which is gn1 i individuals who have reputation B recover their G reputation when they succeed in cooperation, for which the probability is ^. Consequently, the frequency of Gindividuals among ALLC-individuals, g0i , is ( 1 if i ¼ 1; 0 gi ¼ (35) 0 n1 0 gi1 ½1  gi1 ð1  ^Þ þ ð1  gi1 Þ^ if iX2; where gi indicates the frequency of G-individuals in the population, i.e. among incumbent DIS n1 -individuals. Since gi ¼ g1 ¼ ^ (iX2), limi!1 g0i ¼ g01 ¼ 1=ð1þ ^ n2  ^n1 Þ. Substituting the values g1 and g01 into Eqs. (7) and (33), ð1Þ we get the value of f ð1Þ ðDSjDSÞ and f ðACjDSÞ . As a result, by Eqs. (31) and (32), we can show the condition for DIS n1 to be stable against ALLC as follows: only in two person case (n ¼ 2), we can derive the condition without the approximation given in Eqs. (31)–(32). In this case, DISn1 is stable against ALLC if c=b4ð1  2^

Now, we investigate the evolutionary stability of DIS n1 against ALLC in n-IPD. As in n-RPD, at the first round, both DIS n1 - and ALLC-individuals intend to cooperate and so the expected payoffs for both types of individuals are the same. Therefore, by Eqs. (31) and (32), DIS n1 is stable against ð1Þ ALLC if f ð1Þ ðDSjDSÞ 4f ðACjDSÞ . Note that we consider the situation where we can assume that all the opponents of each individuals is always incumbent DIS n1 -individuals. In this case, an incumbent DIS n1 -individual always belongs to a group consisting of only DIS n1 -individuals, where someone fails to cooperate at some time due to error which causes everyone in the group to defect. Therefore, f ð1Þ ðDSjDSÞ ¼ 0. On the other hand, an ALLC-individual also belongs to a group where all the opponents have the DIS n1 strategy. If group size is greater than two (n42), in later rounds, only the focal ALLC-individual intends to cooperate, since the other DIS n1 -individuals are going to defect in response to error defections. If group size is two (n ¼ 2), the DIS n1 -opponent in the group intends to cooperate as long as the focal ALLC-individual succeeded in cooperation, the probability of which is ^, at the previous round. Consequently, f ð1Þ ðACjDSÞ is

f ð1Þ ðACjDSÞ ¼

 8  b > > ^ >  c < n

  b > 2 > ^ ^ >  c  ðb  cÞ þ ð1   Þ^  : 2

if n42 (36) if n ¼ 2:

Recall that DIS n1 is stable against invasion by ALLC if ð1Þ f ð1Þ ALLC of DIS n1 ¼ 0. If n42, DIS n1 is always stable. On the other hand, for n ¼ 2, as well as in n-RPD under IMAGE, we can get the condition for DIS n1 to be stable against ALLC without the approximation given in Eqs. (31)–(32). The condition is c=b4ð1 þ ^wÞ=2 which is the same as that in n-RPD under IMAGE. Therefore, the condition for DIS n1 to be evolutionarily stable against invasion by ALLC in n-IPD is the same as that in n-RPD under IMAGE (see Fig. 4). Note that, if n42, the two conditions are not exactly the same but very close when we do not use the approximation given in Eqs. (31)–(32). Only in the case of n ¼ 2, the two conditions are exactly the same even without the

ARTICLE IN PRESS 548

S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

approximation. Namely, in n-IPD, DIS n1 is stable against invasion by ALLC if group size is greater than two or if group size is two and if the cost to benefit ratio of cooperation is extremely large.

0.8

3.4. Condition for DIS n1 to be evolutionarily stable

0.6 c/b

III 0.4 0.2

II I

0 2

5

10

15

20 25 30 Group Size, n

35

40

45

50

35

40

45

50

35

40

45

50

1

0.8 III

0.6 c/b

So far in this section, we have analysed the condition for DIS n1 to be stable against invasion by ALLC. In the population where there exist DIS n1 , ALLD and ALLC, DIS n1 is evolutionarily stable if DIS n1 -individuals can resist invasion by a few ALLD-individual and that by a few ALLC-individual. The conditions for the former and for the latter are revealed in the previous and this section, respectively. In Fig. 5, we illustrate the condition for DIS n1 to be evolutionarily stable. As is seen in the figure, in n-RPD under IMAGE and n-IPD, DIS n1 is never evolutionarily stable when group size is two. However, when group size is greater than two, DIS n1 is evolutionarily stable in n-RPD under IMAGE and in n-IPD if the cost to benefit ratio of cooperation is sufficiently small. On the other hand, in n-RPD under STAND, DIS n1 is evolutionarily stable in wide range of the cost to benefit ratio of cooperation.

1

0.4

II

0.2

3.5. Summary of results

I

4. Discussion In this paper, we have investigated the evolutionary phenomena of indirect reciprocity when more than two person take part in a single interaction (group) and compared the phenomena with those of direct reciprocity. Especially, we have focused on the reciprocity formed by the strictest discriminators who cooperate if and only if all the opponents have good reputation. Our analyses have shown the following.

0 2

5

10

15

20 25 30 Group Size, n

1

0.8 0.6 III

c/b

In this section, we have checked the evolutionary stability of indirect reciprocity and compared with that of direct reciprocity in the population also including unconditional cooperators. The analyses have revealed the following points. First, in two-person case of n-RPD under image scoring, indirect reciprocity is never evolutionarily stable. However, when group size is greater than two, even under image scoring, indirect reciprocity is evolutionarily stable if the cost to benefit ratio of cooperation is sufficiently small. Second, in n-RPD under standing, indirect reciprocity is evolutionarily stable within the large range of the cost to benefit ratio of cooperation. Finally, for any size of groups, the stability of direct reciprocity is very close to that of indirect reciprocity under image scoring and is different from that of indirect reciprocity under standing.

0.4 0.2

II I

0 2

5

10

15

20 25 30 Group Size, n

Fig. 5. Condition for DISn1 to be an ESS (w ¼ 0:90 and ^ ¼ 0:90): Region I: DISn1 -individuals cannot resist invasion by a few ALLCindividuals or the condition for the n-person prisoner’s game, c4b=n, does not hold. Region II: DIS n1 is evolutionarily stable. Region III: DISn1 cannot resist invasion by ALLD. (a) The condition in n-RPD under IMAGE; (b) that in n-RPD under STAND; and (c) that in n-IPD.

First, in the population consisting of discriminators and unconditional defectors, both strategies are evolutionarily stable and the evolution of indirect reciprocity becomes

ARTICLE IN PRESS S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

more difficult as group size increases. With the increase in group size, the condition for indirect reciprocity to be stable becomes more restrictive with regard to the parameter space of payoff function and the initial frequency of discriminators required by the formation of the indirectly reciprocal society. These results are consistent with the precedence study using computer simulation (Suzuki and Akiyama, 2005). Furthermore, the condition for the evolution of indirect reciprocity under standing is more relaxed than that under image scoring for any size of groups. For the evolution of indirect reciprocity, punishment (defection) on defectors and cooperation among discriminators are essential. However, in large groups, if discriminators try to defect on defectors, they punish not only the defectors but also the other discriminators. Moreover, the number of the embroiled discriminators increases as group size increases. That is, in large groups, the punishment on defectors and the cooperation among discriminators do not go together. Hence, the evolution of indirect reciprocity becomes difficult with the increase in group size. It is important to note that, in the presence of errors in implementation, the increase of indirectly reciprocal strategies (discriminators) under image scoring does not imply the formation of a cooperative society, because discriminators eventually come to defect as round passes in response to error defections. Next, in the population that also includes unconditional cooperators and in the presence of errors in implementation, indirect reciprocity under image scoring is evolutionarily stable in more than two-person case for some range of parameter of payoff function, whereas that is never in two-person case. In two-person case under image scoring, as Panchanathan and Boyd (2003) has pointed out, a few unconditional cooperators can invade the population of discriminators, because discriminators defect on each other in response to error defection whereas invading unconditional cooperators are not drawn into the chain of retaliative defections. However, in more than two-person case, a few unconditional cooperators cannot invade the population of discriminators, because in this case invading unconditional cooperators usually belong to a group with two or more discriminators and so they cannot avoid being drawn into the chain of defection. Therefore, in more than two-person case, indirect reciprocity can be evolutionarily stable under image scoring. On the other hand, indirect reciprocity under standing is evolutionarily stable for wider range of parameter settings, which agree with the twoperson case given in Panchanathan and Boyd (2003), Ohtsuki and Iwasa (2004). Note that, in the absence of errors in implementation, clearly, indirectly reciprocal strategies (discriminators) cannot resist invasion by unconditional cooperators for any group size. Discriminators decide their own actions based on the opponents’ past actions (first-order information) under image scoring. On the other hand, under standing, they decide not only on the first-order information but also on

549

the former reputation of the opponents of the opponents (second-order information). Whether people actually use the second-order information or do not is controversial issue in indirect reciprocity. In two-person case, several theoretical studies (e.g. Panchanathan and Boyd, 2003; Ohtsuki and Iwasa, 2004; Brandt and Sigmund, 2004) have pointed out that second-order information is must for the evolutionary stability of indirect reciprocity. However, experimental studies have not been able to reach the conclusion yet. For instance, Milinski et al. (2001) found that people do not actually use the second-order information, while Bolton et al. (2005) found that they do. In more than two-person case, we have shown theoretically in this paper that indirect reciprocity can be evolutionarily stable even if discriminators use only the first-order information, though experimental evidence does not exist. Finally, for any size of groups, the condition for the evolution of direct reciprocity is equivalent to that for the evolution of indirect reciprocity under standing in the absence of errors in implementation, while this is very close to the condition for the evolution of indirect reciprocity under image scoring in the presence of error.5 In n-RPD which is related to indirect reciprocity, good reputation of discriminators is maintained and so the discriminators cooperate with each other under standing. However, under image scoring, the reputation of all individuals becomes bad with time in response to error defections and so no discriminator cooperates with each other. On the other hand, in n-IPD which is related to direct reciprocity, in the absence of error, discriminators continue to cooperate with each other in a group with no unconditional defector. However, in the presence of error, discriminators eventually come to defect with time in response to error even in a group with no unconditional defector. Therefore, in the absence of error, the condition for the evolution of direct reciprocity is equivalent to that for the evolution of indirect reciprocity under standing, while, in the presence of error, it is very close to the condition for the evolution of indirect reciprocity under image scoring. Indirect reciprocity involving social reputation has been considered as important in the biological basis of human morality (Alexander, 1987). Moreover, we believe one of the features of human societies is that interactions often involve three or more individuals (e.g. human relations in your company, school and other communities). Therefore, our analyses of the evolution of indirect reciprocity in groups of various sizes can help understanding the evolution of morality in human societies.

5 We conjecture that, as Brandt and Sigmund (2006) has mentioned (for two-person case), in the presence of error in implementation, the condition for the evolution of indirect reciprocity under standing might be similar to that for the evolution of direct reciprocity formed by contrite TFT (Boerlijst et al., 1997).

ARTICLE IN PRESS 550

S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

Acknowledgments

A.2. Frequency of G-individuals under STAND (n-RPD)

We thank Nobuyuki Hanaki and three anonymous reviewers for their useful comments. This research was partially supported by the Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Scientific Research (S), 2005–2009, 17103002, and for JSPS Fellows, 2006–2007, 183845.

Here we show the derivation of the frequency of Gindividuals at round i under STAND given in Eq. (21). Let us denote the frequencies of G-individuals among ALLD- and DIS n1 -individuals at round i as gi ðADÞ and gi ðDISÞ, respectively. Then, the frequency of G-individuals among the whole population, gi , is given by ð1  pÞgi ðADÞ þpgi ðDISÞ where p represents the frequency of DIS n1 individuals in the population. At the first round, all individuals are supposed to have G reputation, i.e. g1 ¼ 1. After the first round, all ALLDindividuals have B reputation. Therefore, gi ðADÞ ¼ 0 (iX2). On the other hand, DIS n1 -individuals who have reputation G lose their G reputation at round i only when they failed to cooperate in the group with no B-individual at round i  1, the probability of which is ð1  ^Þgn1 i1 . On the other hand, DIS n1 -individuals who have reputation B get reputation G at round i only when they succeeded in cooperation at round i  1, the probability of which is ^gn1 . Therefore,

Appendix A A.1. Expected payoff for an ALLD- and a DIS n1 individual (n-RPD) We show the derivation of the expected payoff for an ALLD- and a DIS n1 -individual given in Eqs. (4)–(7). In preparation for this purpose, we first show the expected payoff for an individual when she cooperates with probability x and her opponent cooperates with probability y. In this case, the expected payoff for her is f ¼ xV ðCj1 þ ðn  1ÞyÞ þ ð1  xÞV ðDjðn  1ÞyÞ b ðA:1Þ ¼ ½x þ ðn  1Þy  cx. n Getting back to the expected payoff for the two types of individuals, at the first round, since all individuals have reputation G, all DIS n1 -individuals and no ALLDindividual intend to cooperate. In this case, the probabilities that ALLD- and DIS n1 -individuals cooperate are 0 and ^, respectively. On the other hand, an opponent in the group cooperates when the opponent has DIS n1 and succeeds in cooperation, the probability of which is ^ p. Therefore, by Eq. (A.1), the expected payoffs for ALLDand DIS n1 -individuals at the first round are

gi ðDISÞ ¼ gi1 ðDISÞ½1  ð1  ^Þgn1 i1  þ ð1  gi1 ðDISÞÞ^gn1 i1 ,

ðA:5Þ

where iX2. Since gi ðADÞ ¼ 0 (iX2), gi ¼ pgi ðDISÞ (iX2). In this case, by induction, we can confirm that gi ðDISÞ ¼ ^ (iX2). Hence the frequency of G-individuals among the whole population, gi , is ( 1 if i ¼ 1; gi ¼ (A.6) ^p if iX2: Appendix B

b ^ ðn  1Þp, f ð1Þ ALLD ¼  n

(A.2)

B.1. Expected payoff for an ALLC-individual in the population dominated by DIS n1 -individuals (n-RPD)

b ^ ½1 þ ðn  1Þp  ^c. f ð1Þ DISn1 ¼  n

(A.3)

In this section, we show the derivation of the expected payoff for an ALLC-individual in the population dominated by DIS n1 -individual, f ðiÞ ðACjDSÞ , at round i (iX2) given by Eq. (33). Clearly the probability of an ALLC-individual cooperating is ^. On the other hand, an opponent of the focal ALLC-individual intends to cooperate when the focal ALLC-individuals and the other opponents have reputation G, the probability of which is gn2 g0i where gi indicates i the frequency of G-individuals among incumbent DIS n1 individuals and g0i indicates that among a few invading ALLC-individual. Therefore, the probability that an opponent of an ALLC-individual cooperates is ^gin2 g0i . Hence, by Eq. (A.1), the expected payoff for an ALLCindividual at round i in the population dominated by DIS n1 is

At round iX2, the expected payoff for an ALLDindividual is V ðDj0Þ ¼ 0 because no DIS n1 -individual cooperates in the group with some B-individuals. Note that, at round iX2, all ALLD-individuals have reputation B because of their defection. On the other hand, at round iX2, DIS n1 -individuals intend to cooperate if and only if all the opponents have reputation G, the probability of which is gn1 . A opponent i of the focal DIS n1 -individual intends to cooperate if she has DIS n1 and her all opponents including the focal DIS n1 -individual have G reputation, the probability of which is pgn2 gi =p. Note that gi =p indicates the probability i of the focal DIS n1 -individual having reputation G. Consequently, by Eq. (A.1), the expected payoffs for DIS n1 -individuals at round i is ^ ðgi Þn1 ðb  cÞ. f ðiÞ DISn1 ¼ 

(A.4)

b ^ ½1 þ gin2 g0i ðn  1Þ  ^ c. f ðiÞ ðACjDSÞ ¼  n

(B.1)

ARTICLE IN PRESS S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

551

Appendix C 1

Here we mention the evolution of indirect reciprocity in the case without implementation error, i.e. ^ ¼ 1. Under IMAGE, considering Eq. (3), the frequency of G-individuals in the limit of round i ! 1 for ^ ¼ 1 is ( 0 if 0ppo1; g1 ¼ (C.1) 1 if p ¼ 1;

0.8

c/b

0.6 IMAGE∗

0.4

IMAGE

0.2

0 2

5

10

15

20 25 30 Group Size, n

35

40

45

50

1

0.8

c/b

0.6 STAND∗

0.4

STAND

Appendix D

0.2

0 2

5

10

15

which is different from that in the case with error. However, in order to investigate the evolutionary stability of DIS n1 and the features of the internal equilibrium at pu , it is only necessary to consider the case for 0opo1, and g1 ¼ 0 for 0opo1, which is the same as that in the case with error. Consequently, the fitness for DIS n1 - and ALLD-individuals are the same as those in the case with error given in Eqs. (12) and (13), and the results in the case without error are the same as those in the case with error given in Section 2.2. Under STAND, g1 is ^p for any ^ and p. (See Eq. (21).) Therefore, the results in the case without error are the same as those in the case with error given in Section 2.3 as well as under IMAGE.

20 25 30 Group Size, n

35

40

45

50

1

We show that, by numerical calculation, the approximation given in Eqs. (31)–(32) does not change essentially the stability condition of DIS n1 against ALLC. In Fig. 6, we compare the condition derived analytically with the approximation and that given numerically without the approximation. As shown in the figure, the approximation hardly affect the condition. References

0.6

Alexander, R.D., 1987. The Biology of Moral Systems. Aldine de Gruyter, New York. Axelrod, R., 1984. The Evolution of Cooperation. Basic Books, New York. Axelrod, R., Hamilton, W.D., 1981. The evolution of cooperation. Science 211, 1390–1396. Boerlijst, M.C., Nowak, M.A., Sigmund, K., 1997. The logic of contrition. J. Theor. Biol. 185, 281–293. Bolton, G.E., Katok, E., Ockenfels, A., 2005. Cooperation among strangers with limited information about reputation. J. Public. Econ. 89, 1457–1468. Boyd, R., Richerson, P.J., 1988. The evolution of reciprocity in sizable groups. J. Theor. Biol. 132, 337–356. Boyd, R., Richerson, P.J., 1992. Punishment allows the evolution of cooperation (or anything else) in sizable groups. Ethology and Sociobiology 13, 171–195. Brandt, H., Sigmund, K., 2004. The logic of reprobation: assessment and action rules for indirect reciprocation. J. Theor. Biol. 231, 475–486. Brandt, H., Sigmund, K., 2005. Indirect reciprocity, image scoring, and moral hazard. Proc. Natl Acad. Sci. USA 102, 2666–2670. Brandt, H., Sigmund, K., 2006. The good, the bad and the discriminator— errors in direct and indirect reciprocity. J. Theor. Biol. 239, 183–194. Fehr, E., Gachter, S., 2000. Cooperation and punishment in public goods experiments. Am. Econ. Rev. 90, 980–994.

c/b

0.8

n-IPD∗

0.4

n-IPD

0.2

0 2

5

10

15

20 25 30 Group Size, n

35

40

45

50

Fig. 6. Stability condition of DISn1 against ALLC (w ¼ 0:90 and ^ ¼ 0:90): solid line indicates the condition derived analytically with the approximation, dashed line represents the condition given by numerical calculation without the approximation. Above the line, DIS n1 -individuals can resist invasion by a few ALLC-individuals. Below the line, DISn1 -individuals cannot resist invasion by a few ALLC-individuals or the condition for the n-person prisoner’s game, c4b=n, does not holds. (a) The stability condition in n-RPD under IMAGE; (b) that in n-RPD under STAND; and (c) that in n-IPD.

ARTICLE IN PRESS 552

S. Suzuki, E. Akiyama / Journal of Theoretical Biology 245 (2007) 539–552

Fehr, E., Schmidt, K.M., 1999. A theory of fairness competition and cooperation. Q. J. Econ. 114, 817–868. Fishman, M.A., 2003. Indirect reciprocity among imperfect individuals. J. Theor. Biol. 225, 285–292. Hauert, C., Monte, S.D., Hofbauer, J., Sigmund, K., 2002a. Volunteering as red queen mechanism for cooperation in public goods games. Science 296, 1129–1132. Hauert, C., Monte, S.D., Hofbauer, J., Sigmund, K., 2002b. Replicator dynamics for optional public good games. J. Theor. Biol. 218, 187–194. Joshi, N.V., 1987. Evolution of cooperation by reciprocation within structured demes. J. Genet. 66, 69–84. Leimar, O., Hammerstein, P., 2001. Evolution of cooperation through indirect reciprocity. Proc. R. Soc. London B 268, 745–753. Lotem, A., Fishman, M.A., Stone, L., 1999. Evolution of cooperation between individuals. Nature 400, 226–227. May, R.M., 1987. More evolution of cooperation. Nature 327, 15–17. Milinski, M., Semmann, D., Bakker, T.C.M., Krambeck, H.J., 2001. Cooperation through indirect reciprocity: image scoring or standing strategy? Proc. R. Soc. London B 268, 2495–2501. Milinski, M., Semmann, D., Krambeck, H.J., 2002a. Donors to charity gain in both indirect reciprocity and political reputation. Proc. R. Soc. London B 269, 881–883. Milinski, M., Semmann, D., Krambeck, H.J., 2002b. Reputation helps solve the Tragedy of the commons. Nature 415, 424–426. Mohtashemi, M., Mui, L., 2003. Evolution of indirect reciprocity by social information: the role of trust and reputation in evolution of altruism. J. Theor. Biol. 223, 523–531. Molander, P., 1992. The prevalence of free riding. J. Conflict. Resolt. 36, 756–771. Nowak, M.A., Sigmund, K., 1992. Tit for tat in heterogeneous populations. Nature 355, 250–252. Nowak, M.A., Sigmund, K., 1998a. Evolution of indirect reciprocity by image scoring. Nature 393, 573–577. Nowak, M.A., Sigmund, K., 1998b. The dynamics of indirect reciprocity. J. Theor. Biol. 194, 561–574.

Nowak, M.A., Sigmund, K., 2005. Evolution of indirect reciprocity. Nature 437, 1291–1298. Ohtsuki, H., 2004. Reactive strategy in indirect reciprocity. J. Theor. Biol. 227, 299–314. Ohtsuki, H., Iwasa, Y., 2004. How should we define goodness?— reputation dynamics in indirect reciprocity. J. Theor. Biol. 231, 107–120. Ohtsuki, H., Iwasa, Y., 2005. The leading eight: social norms that can maintain cooperation by indirect reciprocity. J. Theor. Biol. 239, 435–444. Panchanathan, K., Boyd, R., 2003. A tale of two defectors: the importance of standing for evolution of indirect reciprocity. J. Theor. Biol. 224, 115–126. Panchanathan, K., Boyd, R., 2004. Indirect reciprocity can stabilize cooperation without the second-order free rider problem. Nature 432, 499–502. Semman, D., Krambeck, H., Milinski, M., 2003. Volunteering leads to rock-paper-scissors dynamics in a public goods game. Nature 425, 390–393. Sugden, R., 1986. The Evolution of Rights, Co-operation and Welfare. Blackwell, Oxford. Suzuki, S., Akiyama, E., 2005. Reputation and the evolution of cooperation in sizable groups. Proc. R. Soc. London B 272, 1373–1377. Suzuki, Y., Toquenaga, Y., 2005. Effects of information and group structure on evolution of altruism: analysis of two-score model by covariance and contextual analyses. J. Theor. Biol. 232, 191–201. Takahashi, N., Mashima, R., 2006. The importance of subjectivity in perceptual errors on the emergence of indirect reciprocity. J. Theor. Biol. 243, 418–436. Taylor, P., Jonker, L., 1978. Evolutionary stable strategies and game dynamics. Math. Biosci. 40, 145–156. Trivers, R., 1971. The evolution of reciprocal altruism. Q. Rev. Biol. 46, 35–57. Wedekind, C., Milinski, M., 2000. Cooperation through image scoring in humans. Science 288, 850–852.