Heteroclinic Cycles and Segregation Distortion

J. theor. Biol. (1996) 183, 363–379

Heteroclinic Cycles and Segregation Distortion B¨ M. R. S† Institut fu¨r Mathematik, Universita¨t Wien, Austria (Received on 1 December 1995, Accepted in revised form on 29 May 1996)

Segregation Distorters are genetic elements that disturb the meiotic segregation of heterozygous genotypes. The corresponding genes are ‘‘ultra-selfish’’ in that they force their own spreading in the population without contributing positively to the fitness of the organisms carrying them. We consider here autosomal two-locus drive systems consisting of a ‘‘killer locus’’ and a ‘‘target locus’’. The diploid population dynamics is approximated by a game dynamical model leading to replicator equations. We investigate in detail the dynamics of the SD-system of Drosophila melanogaster and the competition of two killer alleles at the same gene locus. We show that heteroclinic cycles are a common type of attractor in models of segregation distortion systems, while they rarely occur in models with Mendelian segregation. 7 1996 Academic Press Limited

progeny (Hiraizumi & Crow, 1960). Thus, j = 1/2 corresponds to Mendelian segregation, while j = 1 indicates complete distortion, i.e., all zygotes containing the ‘‘killer’’ produce only gametes that also carry the ‘‘killer’’ allele. The examples listed in Table 1 show that meiotic drive systems in nature can act at all different levels of the distortion strength, from a slight bias to complete killing. There are two classes of meiotic drive systems: autosomal and sex-linked segregation distorters. The latter are responsible for deviations of the sex ratio from 1/2. Both genetic systems occur in a wide range of organisms; Table 1 lists a few examples. A comparison of a number of drive systems can be found in a review by Lyttle (1991). In this contribution we will restrict our attention to autosomal drive. An early example of a segregation distorter system was found in several species of commensal mice, in particular in mus domesticus and mus musculus (Dunn & Suckling, 1956). In these species, chromosome 17 carries an array of inversions called transmission ratio distortion (TRD) which comprises roughly 1.2% of the total mouse genome. Chromosomes carrying TRD have been termed t-haplotypes. Heterozygous t/+ males transmit predominately

1. Introduction Meiotic drive is the violation of the Mendelian rules of meiotic division. The mechanisms responsible for this phenomenon cause one member of a pair of heterozygous alleles or heteromorphic chromosomes to be transmitted to progeny in excess of the Mendelian proportion of 50%. The term ‘‘meiotic drive’’ for this phenomenon was coined by Sandler & Novitski (1957). Genetic elements that exhibit the phenomenon of meiotic drive are also known as Segregation Distorters. They have been investigated since the 1920s; for a review see e.g. Lyttle (1991). They can be viewed as ‘‘ultra-selfish genes’’ since they not only show a tendency to increase in frequency without increasing the fitness of the organism harboring them, but the active element (‘‘killer’’) achieves this increase by actively promoting the destruction of its allelic alternative(s) (Crow, 1988). The effectivity of the drive mechanism is measured by the distortion strength j which is defined as the proportion of distorter alleles (chromosomes) among all viable alleles (chromosomes) transmitted into † Present address: Institut fu¨r Statistik, Operations Research und Computerverfahren, Universita¨t Wien, Universita¨tsstr. 5/9, A-1010 Wien, Austria. E-mail: baer.tbi.univie.ac.at 0022–5193/96/240363 + 17 $25.00/0

363

7 1996 Academic Press Limited

364

. . . 

t-bearing sperm (Silver, 1985). Segregation distorter chromosomes (SD) were discovered in 1956 in a natural population of Drosophila melanogaster by Y. Hiraizumi (Hiraizumi & Crow, 1960). In males, they violate the Mendelian rule of segregation. Males heterozygous for SD produce progeny among which at least j = 95% carry SD. The mechanism of distortion causes a large fraction of the sperms of the genetically normal homologue to be dysfunctional. The population dynamics of species with meiotic drive systems have received considerable interest (Lewontin, 1968; Karlin, 1972; Prout et al., 1973; Thomson & Feldman, 1974, 1976; Liberman, 1976; Charlesworth & Hartl, 1978; Lessard, 1987; Crow, 1988; Feldman & Otto, 1991; Haig & Grafen, 1991). The influence of recombination is a central theme in these references. Almost all of this analysis has been concerned with the stability of polymorphisms and questions of invadability of equilibria. In this contribution we will focus on the existence and stability of heteroclinic cycles in these models in the absence of recombination. A heteroclinic cycle is a cyclic arrangement of saddle points together with connecting orbits such that each orbit has one saddle point as its a-limit and the next one as its v-limit. In contrast to limit cycles a heteroclinic orbit is not periodic. An orbit approaching a stable heteroclinic cycle will spend increasing amounts of time close to the corners of the cycle which are formed by the saddle points. The time required to return to a particular corner increases exponentially with the number of revolutions and the orbit comes closer and closer to the corners of the heteroclinic cycle. This type of v-limit is not very common in biological models. Nevertheless they have been found in a number of simple models, in

particular in ecology. We mention a few examples here. A simple though rather artificial class of threespecies models with circulant interaction matrices allows for heteroclinic cycles. The best-known example is the May–Leonard model for cyclic competition (May & Leonard, 1975), see also (Schuster et al., 1979; Phillipson et al., 1985). The assumptions behind this special Lotka–Volterra system are so artificial that it is unlikely that they could ever be found ‘‘in the field’’. The Battle of the Sexes is an example of an asymmetric game due to Dawkins (Dawkins, 1976) modeling the conflict between males and females concerning their respective shares in parental investment (Hofbauer & Sigmund, 1988, p. 139f). In Dawkin’s original model the phase portrait consists of a continuum of periodic orbits surrounding a unique equilibrium in the interior of the square. Modifications of this model can lead to both an asymptotically stable and a repelling heteroclinic cycle (Schuster et al., 1981; Gaunersdorfer et al., 1991). Heteroclinic cycles are abundant in hypercycle-like cooperation models (Eigen & Schuster, 1979; Aulbach & Flockerzi, 1989). However, they are repelling in all these systems. Gaunersdorfer has investigated heteroclinic orbits in replicator equations, focusing on the time-averages of the orbits (Gaunersdorfer, 1991, 1992). Heteroclinic cycles in the iterated prisoners’ dilemma are discussed in Nowak & Sigmund (1989). In Section 2 we shall discuss the SD system in Drosophila melanogaster. In Section 3 we shall investigate the competition of two different killer alleles at the same locus. We shall see that in all these

T 1 Some examples of segregation distorters in nature Organism Lemming. Myopus schisticolor African Butterflies, Genera Danaus, Acrea Mosquito: Aedes aegyptii Drosophila: subgroups affinis and obscura Nasonia Fungi, e.g. Neurospora Mouse Drosophila melanogaster Siberian Mouse Maize

Genetic Element

j

Ref.

Sex Ration Distortion X-Chromosome 0.8 W-Chromosome 1.0

(Fredga et al., 1977) (Chanter & Owen, 1972)

X-Chromosome

(Wood & Newton, 1991)

X-Chromosome (SR) PSR

0.6* † ?

Autosomal Segregation Distortion Spore Killer (Sk) 1.0 t-haplotype (Chr. 17) 0.9–1.0 SD-locus 0.95 Chromosome 1 0.85 B Chromosome

* up to 1.0 in laboratory populations. † many different subgenera with different distortion ratios.

(Beckenbach, 1991) (Werren, 1991) (Turner & Perkins, 1991) (Silver, 1985) (Greenberg-Temin et al., 1991) (Agulnik et al., 1993a) (Jones, 1991)

     models heteroclinic cycles play a predominant role. Methods for the stability analysis of heteroclinic cycles are reviewed in Appendix A. Proofs of the theorems are given in Appendix B. Throughout this contribution we will always assume overlapping generations, that is, we will consider differential equations rather than (discrete time) difference equations, although the latter are more common in population genetics literature. 2. Replicator Equations for Meiotic Drive Systems In order to construct a model of the population dynamics of a meiotic drive system one needs to take into account that the species have in general two sexes. Thus, the Two Sex Equation (Hofbauer & Sigmund, 1988, p. 266) x˙k = xk (My)k + (xM)k yk − 2xk m¯ y˙k = xk (Fy)k + (xF)k yk − 2yk f

(1)

is the appropriate dynamical system. The discrete version of this equation was introduced by (Bodmer, 1965). Here xk and yk denote the frequencies of the male and female gametes, respectively. The entry mij of the matrix M is composed of the fertility m˜ij of a male with genotype i × j and the proportion jij of functional gametes carrying allele i among all the sperm produced by a i × j male. Correspondingly, fij is composed of the fertility of a i × j female and the proportion hij of gametes carrying allele i among all eggs produced by the i × j female. For convenience one sets mij = 2jij m˜ij

and

fij = 2hi j f ij .

Not much is known in general about this system for more than n = 2 alleles. For special cases see (Karlin, 1984; Karlin & Lessard, 1986). 2.1.      The ‘‘Segregation Distorter’’ system in Drosophila can be modeled as a two-locus system of meiotic drive (Prout et al., 1973). On the killer locus alleles S (Killer) and + (Non-Killer) are possible, while on the target locus, alleles R (Resistant) and + (NonResistant) can be found. The resulting four gametic types are ++ , +R, S+ , SR. With this notation, SR is the genotype of the segregation distorter. Naturally occurring SR chromosomes usually carry a strong enhancer of distortion near the tip of the right arm of the chromosome and polygenic enhancers distributed along the whole right arm. In Drosophila melanogaster segregation distortion occurs

365

only in males, while recombination occurs only in females. Since the distorting effect occurs only in males, the system reduces to a special case of the Two Sex Equation (1). It can then be shown (Stadler, 1995) that the dynamics is closely related to a second order replicator equation, at least as far as the stability of the fixed point is concerned. It is justified, therefore, to approximate the population dynamics by the following ansatz for the frequencies xk of the individual chromosome types: x˙k = xk [(Wx )k − (x, Wx)]

(2)

where the entries of the matrix W are effective fitnesses in male flies. We shall now discuss the entries wij , i = 1, . . . , 4 in detail: The effective fitness wij z 2jij uij of i in a pairing with j is composed of the relative fertility uij of i × j and the segregation ratio jij . Clearly we have uij = uji and jij + jji = 1. Consequently, fair segregation, i.e., jij = jji = 1/2 implies wij = wji . The discussion below follows Charlesworth & Hartl (1978) who based their analysis on Thomson & Feldman (1976). In Drosophila meiosis is fair in ++ /++ individuals, and their relative fertility is set to one by definition. Meiosis is also fair in ++ /+R individuals but there is a viability cost for the resistance-allele R which is parmeterized by s1 q 0. In +R+R individuals the viability cost is s3 q s1 . In a simplified model one may as well assume that the viability cost is additive, i.e., s3 = 2s1 , as in the model of Haig & Grafen (1991) discussed later in this section. S+ and ++ seem to behave equivalently in combination with SR, thus we set wSR/S+ = wSR/++ wS+/SR = w++/SR . Unfair segregation occurs in SR/++ , SR/S+ , SR/++ , +R/S+ , and +R/SR genotypes. In SR/++ individuals the gametes SR are produced in more than jSR,++ = 95% of all cases, the corresponding segregation ratio is denoted by k1 . In addition, we assume a fertility loss s2 for SR/++ males. Thus, we have wSR/++ = 2k1 (1 − s2 ) z 1 + K2 w++/SR = 2(1 − k1 )(1 − s2 ) z 1 − K1 and therefore s2 = (K1 − K2 )/2 and consequently K1 q K2 whenever s2 q 0. Naturally occurring +R chromosomes are not completely insensitive to the strongly distorting SR chromosomes. In fact, SR/+R males produce about 80% SR-bearing offspring. This segregation ratio is denoted by k3 . Assuming a viability cost s4 q 0

. . . 

366 we have

Let us denote the transversal eigenvectors (Hofbauer, 1986) at X and Y with eigenvectors within the edge [XY] by lY (X) and lX (Y). Then [X:Y] if and only if lY (X) e 0 and lX (Y) E 0 and at least one of them is non-zero. As an immediate consequence we note

w+R/SR = 2(1 − k3 )(1 − s4 ) z 1 − K5 wSR/+R = 2k3 (1 − s4 ) z 1 + K6 . As above we conclude that K5 q K6. We assume that 1 − s5 Q 1 − K5 Q 1 + K6 , that is, the product of a SR × SR mating is less viable than the offsprings of a SR × +R mating. The segregation ratio of +R/S+ males is not known exactly, but it seems to be around 60% in favor of + R. This segregation ratio is denoted k2 , the corresponding viability cost is s2 . The parameters si q 0 denote viability costs, therefore we will assume si q 0 for all i. It follows then from the above assumptions that K1 q K2 , K5 q K6 , and K4 q K3 . The above assumptions imply the following matrix for male viabilities in their discrete-time model for the segregation distorter (SD) locus of Drosophila melanogaster (Charlesworth & Hartl, 1978):

Lemma 1 . The n corner equilibria P1 , P2 , . . . , Pn of Sn form a heteroclinic cycle[P1 P2 · · · Pn ] if and only if [P1:P2 ], [P2:P3 ], . . . , [Pn − 1:Pn ], and [Pn :P1 ]. In order to avoid degenerate cases we will henceforth assume that all eigenvalues lP (Q) are non-zero in a heteroclinic cycle. In the Charlesworth & Hartl (1978) model there are at most two heteroclinic 3-cycles. These lie either in the face F0 = [++ /SR/+R] or in the face F1 = [+R/ S+ /SR]. These heteroclinic cycles exist for the

++

+R

S+

SR

1

1 − s1

1

1 − s1

1 − s3

1

2(1 − k2 )(1 − s6 ) =1 − K4 2k2 (1 − s4 ) =1 + K2

2k2 (1 − s2 ) =1 + K3 1

2(1 − k1 )(1 − s2 ) = 1 − K1 2(1 − k3 )(1 − s4 ) =1 − K5 2(1 − k1 )(1 − s2 ) =1 − K1 1 − s5 =1 + K2

2k1 (1 − s2 )

2k1 (1 − s2 ) =1 + K6

2.2.       A heteroclinic cycle is not a single orbit but it consists of two or more saddle points that are joined by connecting orbits in a cyclic arrangement, see Fig. 2. A trajectory that approaches a heteroclinic cycle does not become periodic. It spends an increasing amount of time near a saddle point before it moves to the next one, see (Hofbauer & Sigmund, 1988, Section 29.2). In most of the cases that are discussed in the following the heteroclinic cycles consist of the corners of the simplex Sn and their connecting edges (2-species sub-simplices). Definition. An edge [XY] is directed from X to Y, in symbols [X:Y], if the v-limit of every point except for X in [XY] is Y.

++ +R S+ SR

following choices of parameters: F0 :

K3 Q 0 Q s1 Q s3 Q −K6 Q s5 Q K1 , K5 Q s5 Q K1 ,

F1 :

0 Q K2 Q K1 ,

K 4 Q s3 .

0 Q s1 Q s3 Q −K6 Q s5 Q K1 , K5 Q s5 Q K1 ,

0 Q K2 Q K1

Hence both cycles exist whenever the first system of inequalities is fulfilled. The edge [++ /S+ ] is a line of fixed points, thus there are no heteroclinic cycles in the faces [++ /+R/S+ ] and [++ /SR/S+ ]. This follows immediately from the classification of all possible phase portraits of the replicator equation (Bomze, 1983, 1995).

    

367

For the same reason there are only two possible 4-cycles left, namely:

++ :SR:S+ :+R and

++ :+R:S+ :SR. In the first case we require s5 q K1 for the expanding eigenvalue SR:S+ . However, the contracting eigenvalue SR:++ has to fulfill s5 Q K1 in order to allow for the existence of a heteroclinic 4- cycle, thus this cycle never exists. The second case is the same cycle with the reverse direction; it does not exist for the very same reason. Consequently, there is no heteroclinic cycle involving all four pure genotypes in the Charlesworth–Hartl model (see Fig. 1). Stable limit cycles have been found repeatedly in models of segregation distortion with non-zero recombination rates [see for instance Maffi & Jayakar (1981), Eshel (1985), Lessard (1987), Feldman & Otto (1989), Haig & Grafen (1991)]. The heteroclinic cycles in the limit of vanishing recombination have not been dealt with so far. Heteroclinic orbits are a non-local phenomena. Nevertheless, their stability can in certain cases be determined from the linearized flow at its corners, see Appendix A for a brief overview based on Brannath (1994). Using these methods we show in Appendix B: Theorem 1 . Both heteroclinic cycles [++ /SR/+R] and [+R/S+ /SR] can be stable within their planes, but at most one of them can be relatively asymptotically stable. The explicit stability conditions can be found in Appendix B. Example. The heteroclinic cycle [++ /SR/+R] is relatively asymptotically stable for the following choice of parameters K1 = 3/5, K2 = 1/2,

K3 = −33/100,

K4 = 34/100, K5 = 4/10, K6 = −4/10, s1 = 3/10,

s3 = 7/20,

s5 = 1/2.

In this case [+R/S+ /SR] is stable in its plane but not relatively asymptotically stable. The converse holds for instance for the following choice of parameters K1 = 3/5, K2 = 1/10,

K3 = −2/10,

K4 = 1/20, K5 = 4/10, K6 = −4/10, s1 = 3/10,

s3 = 7/20,

s5 = 1/2.

F. 1. Heteroclinic cycles in the Charlesworth–Hartl model. The arrows indicate two eigenvalues at [SR] which are identical and hence prohibit the existence of a heteroclinic 4-cycle. Two heteroclinic 3-cycles are possible, in the faces F0 (which corresponds to the model of Haig & Grafen discussed) and F1 , see Section 2.2.

2.3.     Charlesworth & Hartl argue that, for biologically reasonable choice of the parameters, all equilibria containing S+ either cannot exist or, if they exist, are unstable to the introduction of new gametic types. This conclusion is based on the following analytical results for the discrete-time replicator equation with the above interaction matrix (Prout et al., 1973; Thomson & Feldman, 1976; Charlesworth & Hartl, 1978). It is not hard to check (Stadler, 1995) that analogous results hold for the continuous time model investigated here: (1) The S+ corner is unstable to SR if K2 q 0 and unstable to +R if K3 q 0. (2) A S+ /SR marginal equilibrium is unstable to the introduction of +R if K2 (K1 − K5 ) + K3 (s5 − K1 ) q 0. Since K5 Q s5 Q K1 are realistic parameter values and K2 can be fairly close to K3 , the equilibrium will be unstable in most cases. (3) An S+ /+R equilibrium is unstable to the introduction of ++ if K4 q s1 . (4) An S+ /++ equilibrium is unstable to SR if K2 q 0. (5) An equilibrium on the S+ /SR/+R face is unstable to the introduction of ++ if K4 q s1 . (6) The existence of an equilibrium on the S+ /+R/ ++ face would require K4 = s1 . (7) An equilibrium on the S+ /SR/++ face exists if and only if s5 q K1 and K2 q 0. It is unstable to +R if given that (a)K2 (K1 − K5 ) + s1 (K1 − s5 ) q 0 K3 q −s1 , and (b) K2 (K1 − K5 ) − K3 (K1 − s5 ) q 0 given that K3 Q −s1 ,

. . . 

368

(8) No equilibrium can exist within the ++ /+R/ S+ /SR simplex, except for in the degenerate case K4 = s1 , which is considered impossible. (9) A stable ++ /+R/SR equilibrium is stable to the introduction of S+ if K4 q s1 . Result (8) implies that all trajectories of the continuous-time version of the Charlesworth–Hartl model converge to the boundary of the simplex. This is an immediate consequence of the exclusion principle, see, e.g., (Hofbauer & Sigmund, 1988, p. 64). Assuming that S+ is non-viable and can be neglected therefore, we obtain the following three species model:

F 1 1 − s1 1 − K1 ++ J G 1 − s 1 − s 1 − K +R G 1 3 5 G. A= G G1 + K2 1 + K6 1 − s5 SR G f ++ +R SR j

(3)

Its interaction matrix reads:

2

0 A = −s1 K2

3

s3 − s1 s5 − K1 0 s5 − K5 s3 + K6 0

As a consequence of the discussion in Section 2.2 there is a heteroclinic cycle if and only if 0 Q s1 Q s3 Q −K6 , K5 Q s5 Q K1 , and K2 q 0. It is stable provided K2 (s5 − K5 )(s3 − s1 ) Q s1 (−s3 − K6 ) (K1 − s5 ). 2.4.       Haig & Grafen (1991) proposed a simple model with heritable targets assuming two loci, which they called Killer locus and Target locus, whose wild-type

alleles are Non-Killer and Non-Resistant, respectively. The idea is that an allele Killer can arise at the Killer locus that causes haploid products to die at the end of meiosis unless they carry an allele Resistant at the Target locus. The death of some haploid products is assumed to benefit the survivors in some way: either by the acquisition of resources from the dead haploids, or by reduction of competition for fertilizations. The possession of Resistant is assumed to impose a viability cost, as compared with its allele Non-Resistant. The model is based on an imaginary protist life cycle, in which haploid organisms occasionally meet and fuse. There are three chromosome types in the populations: Killer-Resistant (K), Non-Killer-Resistant (R), and Non-Killer-Non-Resistant (N). The fourth type Killer-Non-Resistant kills itself during the meiosis that might have created it. A variation of the model relaxing the last assumption is discussed in Stadler (1995). The detailed assumptions of Haig & Grafen’s model are the following: (1) There is a viability cost w for each Resistant allele involved in the meiosis. (2) There is a cost v for each unsuccessful attempt to kill. (3) The gain in viability obtained by the driving allele Killer is j compared to a N × N mating. (4) When two K mate, their relative viability is k. (5) In K × N matings the drive is 100% successful, i.e., no Non-Resistant gametes survive. (6) Drive does not occur in matings involving R. Standardizing the fitness of N × N as one and bringing the interaction matrix to normal form one obtains the following interaction matrix:

2

0 A = −w j

w 0 −v

3

−k 1 − k − v − 2w 0

(4)

where j, k, v, w q 0. The heteroclinic cycle is stable if the interior equilibrium is unstable and vice versa (see, e.g. Bomze, 1983, 1995; Stadler & Schuster, 1990). The discussion in Appendix A of Haig & Grafen (1991) implies: Lemma 2 . There is a heteroclinic cycle k + v + 2w Q 1 which is stable if and only if 1 − v − 2w q k q F. 2. The heteroclinic cycle corresponding to the model of Haig & Grafen.

if

j (1 − v − 2w). v+j

The model by Haig & Grafen is closely related to the earlier, and much more complicated model by

     Charlesworth & Hartl discussed in the previous sections, see Table 2. Heteroclinic cycles arise in both models for realistic parameter values (see Fig. 2). 3. Competition of Killers with Complete Distortion

T 2 Correspondence of the Haig–Grafen and the Charlesworth–Hartl models for the SD system in Drosophila melanogaster Charlesworth–Hartl

Haig–Grafen Genotypes

3.1.   Let us consider a model with two different Killer alleles K1 and K2 . Then we have two alleles (R and + ) at the target locus and three alleles (+ , K1 , and K2 ) at the killer locus. Thus there are six possible diploid genotypes. In order to reduce the model we make the following assumptions: (i) +K1 and +K2 are both lethal, and (ii) the two resistant genotypes RK1 and RK2 are (kinetically) indistinguishable and can hence be considered as single resistant genotype R. As in the model of Haig & Grafen ++ is the sensitive genotype. The remaining two genotypes RK1 and RK2 are the two different killer strains K1 and K2 , respectively. In order to reduce the number of free parameters we will assume additionally that the interaction with the resistant strain is the same for both killers. Thus we obtain the following interaction matrix:

R N K1 K2

369

+ /+ + /R S/R S/+

[NK/NR] = N [NK/R] = R [K/R] = K [K/NR] lethal Viabilities

w++/++ = 1 w++/+R = 1 − s1 w+R/+R = 1 − s3 Q 1 − s1 wSR/++ = 1 + K2 w++/SR = 1 − K1 w+R/SR = 1 − K5 wSR/+R = 1 + K6 wSR/SR = 1 − s5

wNN = 1 wNR = 1 − w wRR = 1 − 2w wKN = 1 + j wNK = 0 wRK = 1 − v − 2w wKR = wRK wKK = k Realistic values

w++/SR Q wSR/SR Q w+R/SR

wNK Q wKK Q wRK

R

N

K1

K2

1 − 2w 1−w 1 − v − 2w 1 − v − 2w

1−w 1 1 + j1 1 + j2

1 − v − 2w 0 k1 k

1 − v − 2w 0 k k2

In normal form this becomes

F 0 −w 1 − v − 2w − k1 1 − v − 2w − k2 G w 0 −k1 −k2 A= G −v j 0 k − k2 1 G f −v j2 k − k1 0

Two locus models containing two different killer alleles were introduced in Prout et al. (1973). This stability of polymorphisms in such models was also considered in Thomson & Feldman (1976) and Liberman (1976). The cycles in these models have not been analysed in detail so far. General assumptions ii(i) All the parameters in the above interaction matrix are positive: j1 , j2 , k, k1 , k2 , v, w q 0.

J G G. G j

(5)

i(ii) Without loosing generality we assume k2 E k1 ; that is, the viability of a K2 /K2 genotype is less or equal to the viability of a K1 /K1 genotype. (iii) Additionally we require that k1 Q 1 − v − 2w if k Q k1 and (k 2 − k1 k2 )/(2k − k1 − k2 ) Q (1 − v − 2w) for k q k1 , k2 . This condition guarantees that there is always a heteroclinic cycle in both the face [RNK1 ] and the face [RNK2 ].

. . . 

370

We distinguish two cases: I(I) k2 Q k Q k1 . Then there is no equilibrium on the [K1 , K2 ] edge. There are two heteroclinic cycles. (II) k2 Q k1 Q k or k Q k2 Q k1 . Then there is an equilibrium F on the [K1 , K2 ] edge. In this case there are at least three heteroclinic cycles. We will prove this statement explicitly in the next section for a special case. In general it follows from Bomze (1983) and Stadler & Schuster (1990) that there are trajectories connecting N:F and F:R, and hence we have the three heteroclinic orbits [RNK1 ], [RNK2 ], and [RNF]. 3.2.   : j1 = j2 In this section we will restrict ourselves to j1 = j2 . Biologically, this means that the killer alleles K1 and K2 differ only in their interactions with themselves and with one another, but not with the sensitive and resistant strains S and R, respectively. In the following subsection we will relax this condition and consider the general case. 3.2.1. Case I: two heteroclinic 3 -cycles Let us assume that without loosing generality k2 E k E k1 . Then there are two heteroclinic orbits, [RNK1 ] and [RNK2 ].

we cannot use Brannath’s results for determining whether [1243] can ever be relatively asymptotically stable, see Appendix A. Numerical data suggest that it is never essentially asymptotically stable. 3.2.2. Case II: three heteroclinic 3 -cycles Three heteroclinic cycles can occur if either k1 , k2 Q k or k1 , k2 q k. It is easy to check that x˙3 (k − k1 ) − x˙4 (k − k2 ) = [x3 (k − k1 ) − x4 (k − k2 )] · [jx1 − vx2 − (xWx)], and thus, the plane Z = {x $S4 =x4 (k − k2 ) = x3 (k − k1 )} is invariant. The coordinates of the fixed point F on the [K1 K2 ] edge are xˆ1 = 0, xˆ4 =

xˆ2 = 0,

xˆ3 =

k − k2 , 2k − k1 − k2

k − k1 . 2k − k1 − k2

(6)

The fixed point F lies in the invariant plane Z defined above (see Fig. 3). It will be convenient to introduce the new coordinates 1 z = [(k − k1 )x3 + (k − k2 )x4 ] N and

Theorem 2 . (i) The heteroclinic cycle [RNK1 ] is relatively asymptotically stable under the general assumptions on the parameters if and only if j(1 − v − 2w) Q k1 . (ii) The heteroclinic cycle [RNK2 ] is never relatively asymptotically stable under the general assumptions on the parameters. In addition to the two 3-cycles there is also the 4-corner cycle [1243]. Since D3 (G1 ) = (k − k1 )wj Q 0

$

x˙1 = x1 −wx2 +

+

d=

1 [(k − k1 )x3 − (k − k2 )x4 ], N

where N is a suitable normalization such that z = x3 + x4 holds on Z, i.e., [(k − k1 )x3 + (k − k2 )x4 ] = N(x3 + x4 ). One finds explicitly N = 2(k − k1 )(k − k2 )/(2k − k1 − k2 ). We obtain the following system of differential equations

(1 − v − 2w − k1 )(k − k2 ) + (1 − v − 2w − k2 )(k − k1 ) z 2k − k1 − k2

(1 − v − 2w − k1 )(k − k2 ) − (1 − v − 2w − k2 )(k − k1 ) d − (xWx) 2k − k1 − k2

$

%

x˙2 = x2 wx1 −

k1 (k − k2 ) + k2 (k − k1 ) k (k − k2 ) − k2 (k − k1 ) z− 1 d − (xWx) 2k − k1 − k2 2k − k1 − k2

$

(k − k1 )(k − k2 ) (k − k1 )(k − k2 ) 2 z − (xWx) − d 2k − k1 − k2 2k − k1 − k2

z˙ = z −vx1 + jx2 +

d = d[−vx1 + jx2 − (xWx)]

%

%

(7)

     It is easy to see that this reduces to a 3-species replicator equation for d = 0, i.e., on the invariant plane Z. The interaction matrix for the system [RNF] reads:

2

0 Q= w −v

−w 0 j

371

because k2 = min{k1 , k2 , (k 2 − k1 k2 )/(2k − k1 − k2 )}. Brannath’s results imply immediately that neither G1 nor G2 can be asymptotically stable in this case.

3

(1 − v − 2w) + (k1 k2 − k 2)/(2k − k1 − k2 ) . (k1 k2 − k 2)/(2k − k1 − k2 ) 0

(8)

Let us denote the corresponding heteroclinic cycle [RNF] by GF . It is stable in its plane iff PGF Q 0, see eqn (A.1) in Appendix A. There are two cases to distinguish: i(i) k2 Q k1 Q k, therefore 2k − k1 − k2 q 0, and (ii) k1 q k2 q k, therefore 2k − k1 − k2 Q 0.

It follows immediately from the proof of the above theorem that GF cannot be relatively asymptotically stable in case (ii). In fact, the following result follows immediately from Brannath’s conditions:

Theorem 3 . The heteroclinic cycle [RNF] is stable in its plane iff k2 Q k1 Q k and

Lemma 3 . Suppose k1 q k2 q k. Then the heteroclinic cycles G1 and G2 are relatively asymptotically stable whenever they are stable in their respective planes, i.e., if

j k2 − k1 k2 (1 − v − 2w) Q . j+v 2k − k1 − k2

(9)

Then it is also relatively asymptotically stable in S4 .

and

j(1 − v − 2w) Q k2 ( j + v),

for G1

j(1 − v − 2w) Q k2 ( j + v)

for G2 .

and

Remark. All three cycles G1 , G2 , GF are stable within their respective planes if k2 Q k1 Q k

j(1 − v − 2w) Q k1 ( j + v)

(10) Remark. Neither G1 nor G2 is essentially asymptotically stable w.r.t. S4 since the invariant plane Z cuts the respective basins of attraction into two pieces, both with non-zero measure. In particular, the two cycles both attract a part of the neighborhood of the RN-edge with non-zero measure (see Fig. 4).

In case that both G1 and G2 are relatively asymptotically stable we find that v{x $intSn = d q 0} = G1 and F. 3. The invariant plane containing the equilibrium F also contains a heteroclinic 3-cycle.

v{x $ int Sn = d Q 0} = G2 .

. . . 

372

F. 4. (a) Both G1 and G2 is stable. Parameters: v = w = 1/10, k1 = 7/20, k2 = 6/20, k = 1/5, j = 1/20. (b) In this case GF is stable. Three trajectories, one in the invariant plane Z, and one starting below and above this plane, respectively, are shown. Parameters: v = w = 1/10, k1 = 7/20, k2 = 6/20, k = 8/20, j = 1/20.

3.3.   : j1 $ j2 We have the following interaction matrix:

is relatively asymptotically stable, i.e., if j (1 − v − 2w) Q k1 j+v

F0 −w 1 − v − 2w − k1 1 − v − 2w − k2 G w 0 −k1 −k2 A =G 0 k − k2 G −v j1 f −v j2 k − k1 0

We assume w, v, k, k1 , k2 , j1 , j2 , 1 − v − 2w − k1 , 1 − v − 2w − k2 q 0. In this more general case with j1 $ j2 we cannot derive an equation for an invariant manifold corresponding to Z above. 3.3.1. Case 1: system without equilibrium on the K1 /K2 -edge In this case we have k1 q k q k2 . Again we assume without loosing generality that k1 q k2 . In this case we have two heteroclinic cycles G1 and G2 . One easily checks that the following propositions follow immediately from Brannath’s formulae: If (i) j1 q j2 then G2 is never relatively asymptotically stable, and G1 is essentially asymptotically stable if it

J G G. G j

(11)

and kwj1 Q k1 . wj1 + v( j1 − j2 ) If (ii) j1 Q j2 stability of G1 in its plane follows from stability of G2 in its plane. If G2 is relatively asymptotically stable, then it is already essentially asymptotically stable (see Fig. 5). 3.3.2. Case 2: system with equilibrium on K1 /K2 -edge Let us first consider the dynamics on the face [RK1 K2 ]; we have explicitly x˙1 = x1 [(1 − v − 2w − k1 )x3 + (1 − v − 2w − k2 )x4 − (xWx)]

     x˙3 = x3 [−vx1 + (k − k2 )x4 − (xWx)] x˙4 = x4 [−vx1 + (k − k1 )x3 − (xWx)].

A necessary and sufficient condition for the existence of an equilibrium on the line L = [RF] is that A* 12 and A* 21 have the same sign. A* 21 Q 0 then implies instability. Thus, there is an equilibrium on the line L = [RF]iff(1 − v − 2w) Q (k 2 − k1 k2 )/ (2k − k1 − k2 ). It is always unstable. Furthermore, one easily checks that (1x˙2 /1x2 )=x 1 = 1 = w q 0, i.e., R is unstable with respect to the introduction of N.

(12)

These equations are the same as in the symmetric case j1 = j2 , since neither j1 nor j2 appears in them. We can thus rewrite them in terms of z and d, knowing that L = {d = 0} + [RK1 K2 ], the straight line connecting the fixed points R and F is invariant. On L the dynamics reduces to a replicator equation with game matrix A* =

0

0 −v

373

1

1 − v − 2w + (k1 k2 − k 2)/(2k − k1 − k2 ) . 0 The coordinates of the equilibrium F on the [K1 K2 ]-edge are given by xˆ1 = 0,

xˆ2 = 0, xˆ4 =

xˆ3 =

k − k2 , 2k − k1 − k2

k − k1 . 2k − k1 − k2

(13)

There is no invariant plane for general x2 $ 0; As in the symmetric case, however, we can have three heteroclinic orbits. Theorem 4 . There is a heteroclinic cycle GF = [NRF] if G1 , G2 and F exist. Recall that k1 q k2 . We may distinguish four different sub-cases depending on the relative magnitudes of k, k1 , and k2 . ii(i) k1 q k2 q k, j1 q j2 i(ii) k1 q k2 q k, j1 Q j2 (iii) k q k1 q k2 , j1 q j2 (iv) k q k1 q k2 , j1 Q j2

F. 5. Example for (ii). Both G1 and G2 are relatively asymptotically stable. Only G2 , the cycle in the front face, is essentially asymptotically stable. The parameter values are v = w = 1/10,

k1 = 3/5,

k = 8/15, k2 = 1/2, = 1/5.

j1 = 1/20,

j2

T 3 Stability of heteroclinic cycles in the asymmetric case Case

G1

G2

k1 q k2 q k j1 q j2

s.i.p. c e.a.s.

(r.a.s.)

k1 q k2 q k j1 Q j2

(r.a.s.)

s.i.p. c e.a.s. (r.a.s.)

k q k 1 q k2 j1 q j2

r.a.s. c e.a.s.

(s.i.p.) never r.a.s.

k q k 1 q k2 j1 Q j2

(s.i.p.) never r.a.s.

(r.a.s.)

G2r.a.s. c G1 c s.i.p.

k1 q k q k2 j1 q j2 k1 q k q k2 j1 Q j2

r.a.s. c e.a.s.

(s.i.p.) never r.a.s. s.i.p. c e.a.s.

G2 s.i.p. c G1 s.i.p.

(r.a.s.)

G1gG2

G2 s.i.p. c G1 s.i.p. (both r.a.s.)

s.i.p., r.a.s., and e.a.s. means ‘‘stable within its plane’’, ‘‘relatively asymptotically stable’’, and ‘‘essentially asymptotically stable’’, respectively. Entries in parentheses indicate the existence of parameters such that the heteroclinic cycle has a certain stability property.

374

. . . 

F. 6. Examples to Section 3.3.2. Parameter values are given in the text. Equilibrium points are indicated by circles. The interior equilibrium is omitted for clarity. Case (i) Both cycles are relatively asymptotically stable. Only G1 (bottom face) is essentially asymptotically stable. v = w = 1/10, k1 = 6/10, k2 = 7/12, k = 1/12, j1 = 1/2, j2 = 1/3. Case (ii) Both cycles are relatively asymptotically stable. Only G2 (front face) is essentially asymptotically stable. v = w = 1/10, k1 = 6/10, k2 = 4/10, k = 2/20, j1 = 1/15, j2 = 1/10. Case (iii) G2 is unstable in its plane (front plane) while G1 (bottom plane) is relatively asymptotically stable. v = w = 1/10, k1 = 3/5, k = 5/8, k2 = 1/2, j1 = 55/100, j2 = 1/2. Case (iv) Both G1 and G2 are stable within their planes, but unstable against the forth direction. Numerically we find that the heteroclinic 4-cycle [RNK2 F] is stable. v = w = 1/10, k1 = 7/20, k = 9/20, k2 = 6/20, j1 = 1/25, j2 = 1/20.

In cases (iii) and (iv) we have also two 4-cycles involving F apart from the three 3-cycles, namely [RNK1 F], [RNK2 F]. A numerical example in which one of them is stable is given in Fig. 6, case iv. Analytical criteria analogous to Brannath’s results

cannot be found because the parameters for the expanding direction at R cannot be determined analytically. Partial results on the stability properties of the three-cycles are listed below. An overview is compiled in Table 3. They are all obtained by

     applying Brannath’s formalism to the interaction matrix in eqn (11). The details can be found in Stadler (1995). (i) Both cycles can be relatively asymptotically stable. If G1 is stable within its plane then it is essentially asymptotically stable. (ii) if G2 is stable within its plane, then G1 is also stable within its plane. G2 is essentially asymptotically stable iff it is stable within its plane. Both heteroclinic cycles can be relatively asymptotically stable. (iii) G2 is never relatively asymptotically stable. G1 can be relatively asymptotically stable. (iv) if G2 is stable within its plane, then G1 is also stable within its plane. G1 is never relatively asymptotically stable. Examples are shown in Fig. 6.

4. Conclusions In this work we have studied heteroclinic cycles in two-locus models of segregation distortion in the absence of recombination. While all organisms exhibiting meiotic drive are of course diploid, and therefore should be described by the ‘‘two-sex’’ equation, it can be argued that second order replicator equations are a sufficiently accurate model, as long as one is interested in stable fixed points and heteroclinic cycles on the boundary of the state space. Complicated networks of heteroclinic cycles are a typical feature of this class of population genetic models. The two locus model for the SD complex in Drosophila melanogaster (Prout et al., 1973; Thomson & Feldman, 1976; Charlesworth & Hartl, 1978) shows two competing heteroclinic 3-cycles, one of which corresponds to the attracting heteroclinic cycle in the reduced model discussed in Haig & Grafen (1991). No 4-cycles can occur in these models. The competition of two different killer alleles at the same gene locus leads to a wealth of different qualitative dynamics. The model discussed here assumes that both killers behave equally against the non-killers and exhibit complete distortion. Brannath’s formalism (Brannath, 1994) has been used to study their stability properties. We found stable heteroclinic 3-cycles and systems containing co-existing attractive heteroclinic cycles for reasonable values of the parameters. In addition, a stable heteroclinic 4-cycle was found in numerical simulations. This attractor contains three corner equilibria and a fixed point in the interior of an edge. An analytical treatment of the stability properties of the

375

latter type of heteroclinic cycles is not possible at present. Since we consider two-locus models, recombination will be an important effect in nature. Small amounts of recombination can be treated as a perturbation of the replicator equation by a vector field pointing inward on the boundary of the state space. This is similar to the effect of small mutation fields. Such perturbations will break the saddle connections along the edges of the simplex. For the case of mutation it is known that a stable heteroclinic cycle can give rise to a stable limit cycles (periodic attractor) close to the boundary of the simplex via a so-called saddle-loop bifurcation (Stadler & Schuster, 1992; Stadler & Nun˜o, 1994). Analogous results are to be expected for recombination (Stadler, 1990), see also (Haig & Grafen, 1991). The situation is more complicated, however, because recombination leaves some subspaces of the boundary invariant. Nothing can be said at the moment about the bifurcations in the case of networks of heteroclinic cycles which we have encountered repeatedly in this contribution. Extensive discussions with Karl Sigmund and Peter F. Stadler made this work possible. Special thanks to Christian Forst for his help with the figures. Valuable suggestions by an anonymous referee are gratefully acknowledged. Part of this work was performed during visits to the Santa Fe Institute in 1993 and 1994.

REFERENCES A, S. I., S, I. D. & R, A. O. (1993a). Effect of sperm genotype on chromatid segregation in female mice heterozygous for aberrant chromosome 1. Genet. Res. 61, 97–100. A, B. & F, D. (1989). The past in short hypercycles. J. Math. Biol. 27, 223–231. B, A. T. (1991). Sex ratio polymorphism in Drosophila Pseudoobscura. American Naturalist 137, 340–343. B, W. (1965). Differential fertility in population genetic models. Genetics 51, 411–424. B, I. M. (1983). Lotka–Volterra equations and replicator dynamics: a two-dimensional classification. Biol. Cybern. 48, 201–211. B, I. M. (1995). Lotka–Volterra equations and replicator dynamics: new issues in classification. Biol. Cybern. 72, 447–454. B, W. (1994). Heteroclinic networks on the tetrahedron. Nonlinearity 7, 1367–1384. C, A. O. & O, D. F. (1972). The inheritance and population genetics of sex ratio in the butterfly Acraea Encedon. J. Zool. 166, 363–383. C, B. & H, D. L. (1978). Population dynamics of the segregation distorter polymorphism of drosophila melanogaster. Genetics, 89, 171–192. C, J. F. (1988). The ultraselfish gene. Genetics 118, 389–391. D, R. (1976). The Selfish Gene. Oxford: Oxford University Press. D, L. & S, J. (1956). Studies of the genetic variability in wild populations of house mice. I. Analysis of seven alleles at locus T. Genetics 41, 344–352.

376

. . . 

E, M. & S, P. (1979). The Hypercycle. Berlin: Springer-Verlag. E, I. (1985). Evolutionary genetic stability of mendelian segregation and the role of free recombination in the chromosomal system. Amer. Nat. 125, 412–420. F, M. W. & O, S. P. (1989). More on recombination and selection in the modifier theory of sex-ratio distortion. Theor. Pop. Biol. 35, 207–225. F, M. W. & O, S. P. (1991). A comparative approach to the population-genetics theory of segregation distortion. American Naturalist 138, 443–456. F, K., G. A., W, H. & F, F. (1977). A hypothesis explaining the exceptional sex ratio in wood lemming (Myopus Schisticolor). Hereditas 85, 101–104. G, A. (1991). Dynamics of Asymmetric Games and Heteroclinic Cycles. PhD dissertation Universita¨t Wien Inst. f. Mathematik. G, A. (1992). Time averages for heteroclinic attractors. SIAM J. Appl. Math. 52, 1476–1489. G, A., H, J. & S, K. (1991). On the dynamics of asymmetric games. Theor. Pop. Biol. 39, 345–357. G-T, R., G, B., P, P. A., L, T. W., P, G., D, P., W, C.-I. & H, Y. (1991). Segregation distortion in Drosophila Melanogaster: Genetic and molecular analyses. American Naturalist 137, 287–331. H, D. & G, A. (1991). Genetic scrambling as a defence against meiotic drive. J. theor. Biol. 153, 531–558. H, Y. & C, J. F. (1960). Heterozygous effects on viability, fertility, rate of development, and longevity of Drosophila chromosomes that are lethal when homozygous. Genetics 45, 1071–1084. H, J. (1986). Saturated equilibria, permanence, and stability for ecological systems. In: Proceedings on the Second Autumn Course on Mathematical Ecology, Trieste, Italy, (Hallham, T. G., Gross, L. J. & L, S. A., eds), pp. 625–642. Singapore: World Scientific. H, J. (1987). Heteroclinic cycles on the simplex. In: Proc. Int. Conf. Nonlinear Oscillations, Budapest: Janos Bolyai Math. Soc. H, J. & S, K. (1988). The Theory of Evolution and Dynamical Systems. Cambridge: Cambridge University Press. J, R. N. (1991). B-chromosome drive. American Naturalist 137, 430–442. K, S. (1972). Some mathematical models of population genetics. Amer. Math. Monthly 79, 699–739. K, S. (1984). Mathematical models, problems and controversies of evolutionary theory. Bull. Amer. Math. Soc. 10, 221–273. K, S. & L, S. (1986). Sex Ratio Evolution, volume 22 of Monographs in Population Biology. Princeton, NJ: Princeton University Press. K, M. & M, I. (1995). Asymptotic stability of cycles in systems with symmetry. Erg. Th. Dyn. Sys. 15, 121–148. L, S. (1987). The role of recombination and selection in the modifier theory of sex ratio distortion. Theor. Pop. Biol. 31, 339–358. L, R. C. (1968). The effect of differential viability on the population dynamics of t alleles in the house mouse. Evolution 22, 262–273. L, U. (1976). Modifier theory of meiotic drive: Is mendelian segregation stable? Theor. Pop. Biol. 10, 127–132. L, T. W. (1991). Segregation distorters. Annu. Rev. Genet. 25, 511–557. M, G. & J, S. D. (1981). A two-locus model for polymorphism for sex-linked meiotic drive modifiers with possible applications to aedes aegypti. Theor. Pop. Biol. 19, 19–36. M, R. M. & L, W. J. (1975). Nonlinear aspects of competition between three species. SIAM J. Appl. Math. 29, 243–253. M, I. (1989). An example of a non-asymptotically stable attractor. Nonlinearity 4, 835–844.

N, M. A. & S, K. (1989). Oscillations in the evolution of reciprocity. J. theor. Biol. 137, 21–26. P, P. E., S, P. & J, R. G. (1985). An analytic study of the May-Leonard equations. SIAM J. Appl. Math. 45, 541–554. P, T., B, J. & B, S. (1973). Population genetics of modifiers of meiotic drive I. the solution of a special case and some general implications. Theor. Pop. Biol. 4, 446–465. S, L. & N, E. (1957). Meiotic drive as an evolutionary force. Am. Nat. 41, 105–110. S, P., S, K., H, J., G, R. & M, P. (1981). Selfregulation of behaviour in animal societies III: Games between two populations with self-interaction. Biol. Cybern. 40, 17–25. S, P., S, K. & Wff, R. (1979). On v-limits for competition between three species. SIAM J. Appl. Math. 37, 49–54. S, L. M. (1985). Mouse-t-haplotypes. Annu. Rev. Genet. 19, 179–208. S, B. M. R. (1995). Segregation distortion and heteroclinic cycles. Master’s thesis University of Vienna. S, P. F. (1990). Selection, Mutation and Catalysis. PhD thesis University of Vienna, Austria. S, P. F. & N˜, J. C. (1994). The influence of mutation on autocatalytic reaction networks. Math. Biosci. 122, 127–160. S, P. F. & S, P. (1990). Dynamics of autocatalytic reaction networks I: Bifurcations, permanence and exclusion. Bull. Math. Biol. 52, 485–508. S, P. F. & S, P. (1992). Mutation in autocatalytic networks—an analysis based on perturbation theory. J. Math. Biol. 30, 597–631. T, G. J. & F, M. W. (1974). Population genetics of modifiers of meiotic drive II. linkage modification in the segregation distortion system. Theor. Pop. Biol. 5, 155–162. T, G. J. & F, M. W. (1976). Population genetics of modifiers of meiotic drive III. equilibrium analysis of a general model for the genetic control of segregation distortion. Theor. Pop. Biol. 5, 155–162. T, B. C. & P, D. D. (1991). Meiotic drive in Neurospora and other fungi. American Naturalist 137, 416–429. U, T. (1964). On the flow outside a closed invariant set; stability, relative stability, and saddle sets. Contributions to Differential Equations 3(3), 249–294. W, J. H. (1991). The paternal-sex-ratio chromosome of Nasonia. American Naturalist 137, 392–402. W, R. J. & N, M. E. (1991). Sex ratio distortion caused by meiotic drive in mosquitos. American Naturalist 137, 379–391.

APPENDIX Stability Analysis of Heteroclinic Cycles The definitions given below are taken from (Brannath, 1994). They are based on the work of Ura (1964) and Melbourne (Melbourne, 1989; Krupa & Melbourne, 1995). Our discussion follows (Brannath, 1994). Consider a flow on a subset XURn. Definition. Given any subset NUX, a closed invariant subset A of X is said to be stable relative to the set N or stable in N, if for every neighborhood U of A there is a neighborhood V of A, such that [x $V + N: x(t)$U[t e 0.

     A is said to be asymptotically stable, relatively to N, or asymptotically stable in N, if it is stable in N and there is a neighborhood V of A such that v(x)WA[x $V + N, where v(x) denotes as usual the v-limit of x. In the context of this work we will say that a heteroclinic cycle A is relatively (asymptotically) stable if there exists a set N such that A is (asymptotically) stable in N and N + int Sn $ 9. Definition. A closed invariant subset A of X, is essentially asymptotically stable in N if it is relatively asymptotically stable to a set N which satisfies lim e:0

On S4 , networks of more than one heteroclinic cycle are possible. The most important case for the work presented here consists of two heteroclinic 3-cycles G1 and G2 , which necessarily have one edge of the simplex S4 in common. Let G1M[F1:F2:F3 ] and G2M[F1:F2:F4 ], i.e., the heteroclinic cycles share the edge [F1:F2 ] and branch at the corner F2 . Generalizing the notation introduced above we label the eigenvalues at corner Fk of the cycle Gl by ekl , ckl , and tkl , for the unstable, stable and transversal directions, respectively. See Fig. A1. The above theorem says that Gl is relatively asymptotically stable if PGl Q 0 and each Dkl Mtk,l e(k + 1),l e(k + 2),l + ck,l t(k + 1),l e(k + 2),l

m(Be (A) + N) = 1, m(Be (A)

+ ck,l c(k + 1),l t(k + 2),l Q 0,

where Be (A) = .{x $X = dist(x, A) Q e} and m is the Lebesgue measure. In this work we will say that a heteroclinic cycle is essentially asymptotically stable if it is essentially asymptotically stable in Sn . We will be concerned here only with heteroclinic cycles in S3 and S4 . The following notation is convenient in this case: Let G be a heteroclinic 3-cycle consisting of the saddle points Pk , k = 1, 2, 3 and their connecting orbits. Then we use the following notation: ek expanding (outgoing) eigenvalue at corner Pk ; −ck contracting (incoming) eigenvalue at corner Pk ; tk transversal eigenvalue at corner Pk . The stability of the heteroclinic cycle depends on the following four characteristic parameters: 3

3

k=1

k=1

PG = t ek − t ck ,

377

(A.1)

where k = 1, 2, 3 is counted cyclically. It is easy to check that t1,l , t3,l Q 0 and PGl Q 0 implies c3,l D1,l Q e3,l D3,l

and e2,l D2,l Q c2,l D3,l .

Theorem A.2 . (Brannath, 1994) (i) Gl is relatively asymptotically stable if and only if Dk,l Q 0 for all k. (ii) If both G1 and G2 are relatively asymptotically stable then G1 is essentially asymptotically stable if and only if e21 q e22 . Result (ii) has a simple interpretation: If two heteroclinic cycles ‘‘compete’’ for stability the cycle with the larger expanding eigenvalue at the branching points ‘‘wins’’. It would be interesting to see if an analogous rule holds also in higher dimensions. As for the 3-cycles discussed in the previous section it will be convenient to use ek , −ck , and tk for the expanding, contracting and transversal eigenvalue at the corner k, respectively. If there is a heteroclinic

and Dk = tk ek + 1 ek + 2 + ck tk + 1 ek + 2 + ck ck + 1 tk + 2 , where indices are taken modulo 3. The following result is one of the main tools for the analysis of heteroclinic cycles in this work: Theorem A.1 . (Brannath, 1994, proposition 3.1). Let G be a heteroclinic cycle connecting three corners in S4 . Suppose P Q 0 and Dk Q 0 for k = 1, 2, 3, then G is relatively asymptotically stable. Conversely, if P q 0 or Dk q 0 for one k ${1, 2, 3} then for each neighborhood U of G there exists a trajectory which eventually leaves U, i.e., G is unstable.

F. A1. Notation for heteroclinic cycle on S4 .

. . . 

378

4-cycle [1234] involving the four corners of the simplex S4 then the interaction matrix is of the form

F 0 −c2 t3 e4 J G e1 0 −c3 t4 G A= G G. e2 0 −c4 G t1 G e3 0 j f−c1 t2

Note that with the notation of Fig. A.1 the 4-cycle is [1243] while the two 3-cycles are [123] and [124]. APPENDIX B

(A.2)

If all transversal eigenvalues are negative, ti Q 0, i = 1, 2, 3, 4, then the replicator network is termed essentially hypercyclic (Hofbauer & Sigmund, 1988, p. 187). Theorem A.3 . (Hofbauer, 1987) Suppose a replicator network on S4 is essentially hypercyclic. Then the heteroclinic 4-cycle [1234] is asymptotically stable if P = e1 e2 e3 e4 − c1 c2 c3 c4 Q 0 and unstable if P q 0.

Proof of the Theorems Proof of Theorem 1 . A short calculation shows that the cycles are stable within their planes provided K2 (s5 − K5 )(s3 − K4 ) Q K3 (s3 + K6 )(K1 − s5 ) for [+R/S+ /SR], and K2 (s5 − K5 )(s3 − s1 ) Q s1 (−s3 − K6 )(K1 − s5 ) for [++ /SR/+R]. Assume for instance K4 q −K3 q s1 . Then the above inequalities can be rewritten as K2 (s5 − K5 )(s3 − K4 ) Q K2 (s5 − K5 )(s3 − s1 ),

The situation becomes more involved when not all transversal eigenvalues are negative. If exactly one of the transversal directions is positive we obtain a network consisting of a 4-cycle and a 3-cycle forming a ‘‘short cut’’. The stability of the 4-cycle is determined by an analogue of theorem A.4 below (Brannath, 1994). One may simply omit the condition of Di for the missing 3-cycle. This type of heteroclinic network does not occur in this paper. If two of the transversal eigenvalues are positive and the other two are negative we have two possibilities: either there is only the 4-cycle and both remaining edges contain a rest point (this case will not concern us here), or we have to deal with a network consisting of the 4-cycle and two 3-cycles.

s1 (s3 + K6 )(K1 − s5 ) Q K3 (−s3 − K6 )(K1 − s5 ), and we conclude immediately that stability of the heteroclinic cycle within [++ /SR/+R] implies stability of the cycles in [+R/S+ /SR]. Let us now turn to relative asymptotic stability. We have the following transversal eigenvectors: t1 = 0

t2 = s5 − K1 Q 0

t3 = s3 − K4 q 0 for [++ /SR/+R]

t1 = s3 − s1 q 0

t2 = 0

t3 = s5 − K1 Q 0 for [+R/S+ /SR]

The conditions for relative asymptotical stability are therefore

Theorem A.4 . (Brannath, 1994) Let A be of the form given above and suppose t1 , t2 Q 0 and t3 , t4 q 0. Assume, furthermore, that

D2 ([++ /SR/+R]) = K2 (s3 − K4 )(s5 − K1 )

D1 (G1 ) = −(c1 e2 t2 + t1 t2 t3 + t1 c2 e3 ) q 0

D3 ([+R/S+ /SR]) = K2 (s5 − K1 )(s3 − s1 )

− (s5 − K1 )(s3 − s1 )K2 Q 0

− K2 (s5 − K1 )(s3 − K4 ) Q 0.

and D2 (G2 ) = −(c2 e3 t4 + t2 t3 t4 + t2 c2 e4 ) q 0 i.e., neither one of the two 3-cycles [123] and [234] is relatively asymptotically stable. Then the 4-cycle is relatively asymptotically stable if det A q 0 or if P Q 0. If both det A Q 0 and P q 0 then there is a neighborhood U of the 4-cycle such that all trajectories beginning in U + int S4 will eventually leave U.

Hence the heteroclinic cycle in [++ /SR/+R] is relatively asymptotically stable for s1 Q K4 , while the heteroclinic cycle in [+R/S+ /SR] is relatively asymptotically stable for s1 q K4 , i.e., only one of them can be relatively asymptotically stable. Proof of Theorem 2 . (i) The heteroclinic cycle [RNK1 ] is stable in its plane provided j(1 − v − 2w)/ (j + v) Q k1 . It is relatively asymptotically stable if in

     addition to the inequality above we have Dl (G1 ) Q 0 for l = 1, 2, 3, i.e., if (k − k1 )vw Q 0, j(1 − v − 2w) Q k1 ( j + v) + w(k1 − k), (k − k1 )jw Q 0. The first and the third inequality follow immediately from k1 q k, and the second one is weaker than the condition for stability in the plane. The [RNK2 ] cycle is stable in its plane if j(1 − v − 2w) Q k2 ( j + v). For relative asymptotic stability we need in addition jw(k − k2 ) Q 0,

vw(k − k2 ) Q 0,

j(1 − v − 2w) Q k2 ( j + v) + w(k2 − k). The first two of these inequalities are never fulfilled since k2 Q k, thus [RNK2 ] is not relatively asymptotically stable in case I. Proof of Theorem 3 . The condition for stability in the plane is simply PGF Q 0. In order to determine the stability within S4 we need to consider the Jacobian of the differential equation (2) for d = 0. All entries in the last row are 0, except the entry for 1d /1d which is the transversal eigenvalue for a point in the face [RNF]. We have explicitly 1d 1d

b

1d 1d

= −v,

x1 = 1

b

z=1

1d 1d

b

= j,

x2 = 1

(k − k1 )(k − k2 ) . 2k − k1 − k2

=−

379

Therefore, the relevant transversal eigenvalues are t1 = −v,

t2 = j,

t3 = −(k − k2 )(k − k1 )/(2k − k1 − k2 ). Since t1 Q 0 and t3 Q 0 we have e3 D3 q c3 D1 and c2 D3 q e2 D2. Thus D3 Q 0 implies already D1 , D2 Q 0, i.e., we have relative asymptotic stability if (k − k1 )(k − k2 ) Q 0. 2k − k1 − k2

D3 = −

This inequality holds whenever k2 Q k1 Q k, i.e., in case (i). Proof of Theorem 4 . The transversal eigenvalues of F are 1x˙ (1 − v − 2w − k1 )(k − k2 ) l1(F) = 1 = 1x1 F 2k − k1 − k2

b

+ l2(F) =

(1 − v − 2w − k2 )(k − k1 ) 2k − k1 − k2

b

1x˙2 k (k − k2 ) k (k − k1 ) =− 1 − 2 . 1x2 F 2k − k1 − k2 2k − k1 − k2

and G2 implies The existence of G1 (1 − v − 2w − k1 ) q 0 and (1 − v − 2w − k2 ) q 0. The existence of F implies that k, k1 , k2 fulfill one of the relations (i) through (iv), and hence the fractions (k − kj )/(2k − k1 − k2 ), j = 1, 2 are positive. The classification of all possible phase-portraits of the 3-species replicator equation (Bomze, 1983; Stadler & Schuster, 1990) implies now that there are orbits connecting N:F and F:R, respectively. The remaining connection R:N is shared with both G1 and G2 .