Non-linear ESS-models and frequency dependent selection

BioSystems, 16 (1983) 87--100

87

Elsevier Scientific Publishers Ireland Ltd.

NON-LINEAR ESS-MODELS AND FREQUENCY DEPENDENT SELECTION

HEINZ-JOACHIM POHLEY and BERNHARD THOMAS Institut fiir Entwicklungsphysiologie, Arbeitsgruppe Kybernetik, der Universiti#t zu K6ln, D-5000 K6ln, F.R.G.

(Received July 15th, 1982) (Revision received January 14th, 1983) The concept of an evolutionarily stable strategy (ESS) in its original definition is inappropriate for application to models where the pay-offs to the strategies depend on the distribution of their frequencies in a non-linear manner. Therefore, an extention of the original "global" concept is given by the definition of a "local ESS", which brings into fucus the local bearing of evolutionary stability. As a consequence, we find this static concept in better agreement with the dynamical aspects of the process of natural selection. With particular regard to nonlinear and frequency dependent ESS-models, we discuss in detail the "ESS-adequacy" of a standard dynamical description of the selection process, both in continuous and discrete form, i.e. whether ESSs indeed result from the dynamics of selection. Special results about ESS-models that have an intrinsic symmetry lead to general investigations on the theoretical properties of ESS-models. It is found that equivalent models may arise from very different interpretations and that models of a wide class are essentially symmetric ESS-models. Symmetric ESS-models can be interpreted as fundamental models in population genetics. Key words: Behavioural strategies; ESS-theory; Evolution; Population biology; Selection dynamics

1. Introduction T h e c o n c e p t o f an e v o l u t i o n a r i l y stable s t r a t e g y (ESS) has b e e n i n t r o d u c e d ( M a y n a r d S m i t h a n d Price, 1 9 7 3 ; M a y n a r d S m i t h , 1 9 7 4 ) as an a t t e m p t to explain t h e e v o l u t i o n o f a n i m a l behavior. T h e essential p o i n t t h a t is t o be e x p l a i n e d is t h e c o n s t a n c y o f c e r t a i n f o r m s o f b e h a v i o r c h a r a c t e r i s t i c t o species. T h e m e t h o d is s i m p l y t h a t a v a r i e t y o f f o r m a l strategies is specified, o n e o f which m o d e l s t h e p a r t i c u l a r b e h a v i o r u n d e r consideration. T h e n t h e q u e s t i o n is raised, w h a t is special a b o u t this p a r t i c u l a r strategy. T h e a n s w e r t h a t t h e E S S - t h e o r y m a y give is t h a t it is " e v o l u t i o n a r i l y s t a b l e " in a p r e c i s e l y d e f i n e d sense. T h e p o i n t o f i n t e r e s t in E S S - t h e o r y is n o t w h a t a s t r a t e g y /s (it m e r e l y serves as a t h e o r e t i c a l t e r m ) . I m p o r t a n t is r a t h e r t h e

v a l u e t h a t is assigned to a s t r a t e g y in c o m p a r i s o n t o t h e alternatives. This value is derived f r o m t h e o r e t i c a l analysis w h i c h is based on o u r ( t h e biologist's) ideas a b o u t costs and b e n e f i t s o f a strategy. T h e t h e o r y o f ESSs r e s p o n d s b y selecting t h o s e strategies in t h e given m o d e l t h a t satisfy t h e c o n d i t i o n s o f e v o l u t i o n a r y stability. T h e original c o n c e p t s a n d m e t h o d s o f ESSt h e o r y are well a d a p t e d t o linear m o d e l s . H o w e v e r , non-linear s y s t e m s are m o s t likely to p r o v i d e an a d e q u a t e d e s c r i p t i o n o f r e l e v a n t biological p r o b l e m s (see, f o r e x a m p l e , t h e m o d e l s on t h e nesting b e h a v i o r o f a digger wasp b y B r o c k m a n n et al., 1979). We, therefore, seek to derive an e x t e n t i o n o f these c o n c e p t s f o r n o n - l i n e a r m o d e l s such t h a t t h e original idea o f an ESS is retained. T h e k e y lies in t h e global c h a r a c t e r o f the original c o n d i t i o r ~ t h a t d e f i n e an ESS. T h e s e con-

0303-2647/83/$03.00 © Elsevier Scientific Publishers Ireland Ltd. Printed and Published in Ireland

88 ditions always cover complete spaces of strategies and, for example, do not allow more than one ESS in the interior of the same space. To overcome such difficulties, we introduce the concept of a "local ESS", which states that the conditions must hold only locally, i.e. within a neighborhood of an ESS. For linear models, the original global ESS and the local ESS turn out to be equivalent. Each ESS-model can be understood on the basis of a population of individuals that are the vehicles of fixed strategies. The state of the population is described by a frequency distribution on the set of strategies. In ESStheory, it is an implicit, fundamental postulate that ESSs can be considered population states which result from natural selection, i.e. that ESSs are stable states of an appropriate dynamical system. This, however, is not selfevident, since there is nothing explicitly dynamical in ESS-theory. Hence, we are faced with the problem of ESS-adequate dynamics (Thomas and Pohley, 1982), which has now to be considered with particular regard to non-linear models and the extended ESSconcept. Further ESS-theoretical considerations show that models which are described by a " s y m m e t r i c " pay-off function are of special interest. For example, the concept of an ESS in this case turns out to be equivalent to the principle of maximum population fitness. We derive a method of analysing and classifying ESS-models, which allows us to discuss them with respect to whether they are merely different interpretations of the same theoretical structure or essentially distinct. As it turns out, there are wide classes of models that are essentially symmetric in the above sense. We finally show in detail that basic models of selection on the gene pool level are equivalent to linear or frequency dependent ESS-models.

2. Linear and non-linear ESS-models In an ESS-theoretical model, the observed

form of behavior under consideration is represented either as a p u r e or as a m i x e d strategy, i.e. by a mixture of the e l e m e n t a r y strategies that characterise the model. The success of a strategy will, in general, depend on a distribution of frequencies of the strategies involved. If a population of individuals that is endowed with an individually fixed strategy, pure or mixed, is considered, such a distribution denotes the state of the population or the p o p u l a t i o n strategy. Thus, ESS-theory is ultimately a theory of population states and not expected to provide biological significance of a "player", as might be conjectured from game-theoretical interpretations. In detail, consider a model involving a finite set of n + 1 elementary strategies. Let S n denote the space of probability vectors x = (x~ . . . . , xn+l) T , where xi > 0 is assigned to strategy i, and Z x i = 1. Then population states are described by x ~ S n , and we consider the success F i ( x ) of the i-th pure strategy to depend on x. Accordingly, the success of a mixed strategy, which may also be represented by a probability distribution y ~ S n , is given by the mathematical expectation E ( y , x ) = ~ y i F i ( x ) . Similarly, we may write F i ( x ) = E ( u i , x ) , using unit vectors ui, where the i-th component equals 1, to denote pure strategies. Models of animal conflict often yield payoff values aij, which encorporate costs and benefits to strategy i in an encounter with strategy j. Then the expected success of strategy i with respect to a population with state x, is F i ( x ) = :~aijxj, random "choice of opponents" assumed. Consequently, E ( y , x ) = Y~aijYiXj = y T A x , where A is the square array of pay-offs or the p a y - o f f m a t r i x . Hence, E is called the p a y - o f f f u n c t i o n , which completely determines the ESS-model. If E is defined by a constant pay-off matrix, the model is called linear, since the Fi are then linear functions of x. Otherwise the model is called non-linear. An interesting class of simple non-linear ESS-models has pay-off values a i j ( x ) themselves depending on x, i.e. the pay-off matrix

89 n:l,

x:(xl

x2), x2 - - 1 - x 7

[ o,,r. %c.] 02,(x)

a22(x )j

Fy (x)

: al/x)x I + % ( x ) x 2

F2 (x)

:a27(x),~ + o22(x) x2

AF(x) -- FI (x) - ~ (x)

ECy,x) : Y7 C c~) + ~ F2r~) a¢!(x)-- l-O.6x I

+

0.3X12

Pl

R(p)

"neat-

p

"

Type

% ( x ) : 0.8 - O.4t)x7 + 0.24~2

o.72 [o, 7] "> O
a2t(x)zO.~ - 0.4~X! + 0.24X 2

o7,

a22(x) : O.7 + 0.S X1 - 0.3x12 1

[o,,] empty

empty

/nstable EQ S

0,74
global ESS

Fig. 1. Frequency dependent 2-strategy-model (cf. Fig. 2b) as an illustrative example of the notions given in connection with the ESS-concept. The table lists the EQSs p by their first component Pl, the corresponding sets of alternative best replies, the maximum extent of "near p", and the character o f p (*except 0.12, J'except 0.74).

is A (x) (see Fig. 1 f o r a detailed 2-strategy example). R e f e r r i n g to p o p u l a t i o n genetics, we talk o f frequency dependent ESS-models in this case.

3. Global vs. local ESS

3.1. The original arefinition T o f o r m u l a t e the c o n d i t i o n s o f an ESS, we a d o p t t e r m s similar t o t h o s e used b y Selten (1980). L e t E d e n o t e the p a y - o f f f u n c t i o n o f an E S S - m o d e l and x,y,p E S n p o p u l a t i o n states or m i x e d strategies, d e p e n d i n g u p o n the actual i n t e r p r e t a t i o n . In an expression like E(x,p) we call x a reply t o p, and y a better reply to p t h a n x is, wheneverE(y,p) > E(x,p). I f E(x,p) > E ( p , p ) t , t h e n x is a b e t t e r r e p l y t o p t h a n p itself is, and we thus call x an im-

provement on p. T h e f o r m a l d e f i n i t i o n o f an ESS as originally

l'Comparisons involving a strategy as a reply to itself are typical in ESS-theoretical arguments.

given by M a y n a r d S m i t h and Price ( 1 9 7 3 ) is as follows: p is an e v o l u t i o n a r i l y stable strategy, if

(i) E(p,p) >_ E(x,p) (ii) if E(p,p) = E(x,p) E(p,x) > E(x,x).

for all x and for any x ¢ p, t h e n

H e n c e , p is an ESS, if it is a best r e p l y t o itself and an i m p r o v e m e n t on all its alternative best replies x. In game t h e o r y , a strategy satisfying c o n d i t i o n (i) is called an equilibrium strategy (EQS).

3.2. ESS and non-linear models It is an implication o f the original ESSd e f i n i t i o n t h a t in the interior o f the same strategy space t h e r e can be at m o s t o n e ESS. T o see this, let p be an EQS and let R ( p ) d e n o t e the set o f all alternative best replies x t o p , i.e. where E(x,p) = E(p,p). If s u p p ( x ) , the s u p p o r t o f x, d e n o t e s the set o f all elem e n t a r y strategies i t h a t a p p e a r in x with p r o b a b i l i t y xi > 0, t h e n R (p) includes at least all x ¢ p with s u p p ( x ) c supp(p), i.e. the total subspace o f p,p e x c e p t e d (cf. Fig. 1). R ( p )

90 may be empty, i f p is a pure strategy, and then condition (i) suffices f o r p to be an ESS. If R(p) is not empty, then the additional condition (ii) implies that there can be no other EQS in R (p). For, if q were an EQS and in R(p), then E(q,q) >_ E(p,q), which contradicts condition (ii). Hence, if there is another EQS in R (p), then p cannot be evolutionarily stable. Obviously, it is the "globality" of the requirement in (ii) that rules out alternative ESSs that share elementary strategies. In particular, if p is a completely mixed EQS, i.e. lies in the interior of S n (full support), it is either unique or not an ESS (see Fig. 2c). The reason we stress this point is that in biological applications complex models with non-linear Fi(x ) are likely to arise, which may determine more than one EQS within the same range of strategies (see § 4). But then none of these can be evolutionarily stable by the above definition. AF a

.~L

\.

-,...

~F "'

~

.

.

.

.

.

c

!

• U!

"-

U(qJ~'~_ X1

Fig. 2. Graphical representation of the ESS-condition (1) in § 4 for 3 two-strategy-models derived from examples of frequency dependent genotype selection appearing in literature. Global ESSs are indicated by ~, local ESSs by u, unstable EQSs by o, U's indicate maximum neighborhoods attached to local ESSs. (a) (Lewontin, 1958, example (a)) Unique ESS. In spite of the non-linearity, the A F-plot appears as typically with linear models (except for undefined values in x 1 =0,1). (b) (Clarke and O'Donald, 1964, example (iv)) Mixed local ESS with U(p) reaching to the unstable EQS. In addition to the original results, there is a pure ESS (global) u 1. A similar situation arises with the digger-wasp-models by Brockmann et al. (1979). (c) (Clarke and O'Donald, 1964, example (iii)) Two mixed local ESSs separated by an instable EQS. No global ESS!

J

L

'"

3.3. Local ESS i:

.. \ ...

[

\.

~

\..\

j

U(p) - - - -

~ Xl

AF .14F l

•0?

b ,;I //" //

\\, I

/

t

/ -

,/

. . . . . . . . . . . . .S

for all x t , and for any x ¢ p near

E(p,x) > E(x,x).

-

! \ \

Definition 1: p is a local ESS, if (i) E(p,p) >_ E(x,p) (ii) if E(x,p) = E(p,p) p, then

/

',

0

~_

/

The way to overcome this problem is to give condition (ii) a local meaning. By "local" we mean that (ii) is required only nearp rather than for all alternative best replies to p.

A precise formulation of " n e a r p " would make use of the mathematical notion of a neighborA-X 1 1

t N o t e that "near p " added in (i) as well would have no bearing at all, because E is linear in its first variable.

91 h o o d U(p) of p. Figures I and 2 give an illustration of the m a x i m u m e x t e n t of such neighborhoods in 2-strategy-models. In general, U(p) means a subset of R(p) with p located in the interior. It is evident from definition 1 that alternative EQSs and ESSs may occur in R(p) while p is evolutionarily stable in the local sense. Moreover, we emphasize that the concept of a local ESS is indeed an extention of the original or global definition, as can be seen from the following results.

Corollary 1: A global ESS is also a local ESS.

Definition 2: p is evolutionarily stable, if E(p,x) > E (x,x) for all x ¢ p near p. Note t h a t the EQS-condition (i) is now implied but no longer appears in its own right. This definition, in terms of ESS-theory, expresses an evolutionary principle by specifying the condition for a strategy to be persistent. Bearing in mind the meaning of an " i m p r o v e m e n t " , we may formulate it as thee

ESS-Principle: Evolution supports strategies that are improvements on all neighboring strategies.

The converse is not true, but we can show

Lemma 1: Global and local ESSs coincide for linear models. 3.4 The ESS-principle The conditions of a local ESS have been given in a form that stresses the differences and similarities to the original definition. There is, however, an alternative and much simpler condition to characterise local ESSs. We can show

Theorem 1 : p is a local ESS if and only if E(p,x) > E(x,x) for a l l x C p nearp. This is essentially the condition (ii) of definition 1, except that "near p " is now extended to the full space S n of strategies. The points raised in § 3.2 clarified that the global concept of an ESS is not appropriate for use with non-linear models. On the other hand, local ESSs are global ESSs in linear models, by lemma 1. Hence, the condition of theorem 1 can be used for a general formulation of the concept of evolutionary stability, regardless of "local" or "global".

$ We shall omit proofs where they would only provide technical material.

4. Calculation o f ESS in non-linear models Definition 2 establishes the principle of ESS. However, it is not an adequate tool to be applied in determining ESSs in a given model. For linear models, which are represented by a constant pay-off matrix, methods to find ESSs were first presented by Haigh (1975). These methods are implied in the following results for non-linear ESS-models: Two-strategy-models (i.e. n = 1) allow for a simple graphical discussion of the EQSs and ESSs in a model. This is because we can plot the difference between the values of the two pure strategies u, and u2,

AF(x) = F, (x) -- F2 (x) over the unit interval [0,1]. If A F (p) = 0, which is a (non-linear) equation to be solved for p, then p is an EQS with all other x being alternative best replies. To be an ESS, p must improve on all x near p, which is equivalent to

(xl--p,)"

A F ( x ) < 0,

nearp

(1)

Here, x,, pl denote the first components of x and p, respectively. Condition (1) means that, near p, AF(x) has the opposite sign of

92 x~ - - p l ,i.e. the graph of AF passes zero in p, decreasingly (see Fig. 2). By lemma 3, below, (1) can be replaced by a differential condition: d

dxl

AF(p) < 0

(2)

( x - p ) T F'(p) (x-p) < 0

for x ~ R (p).

That is, the matrix of derivatives, F'(p), must be negative definite on a subspace corresponding to R (/9). Note that in the linear case F(x) = A x and thus F ' (t9) = A. This leads to the conditions of Haigh (1975).

In addition, a pure strategy ui can be an ESS, if it is an EQS with empty R (ui), i.e. if

5. ESS-adequate dynamics

AF(u~)> 0

forul

5.1. The principle of ESS-dynamics

/"F(u2)< 0

foru2.

or

Figure 2 gives graphical representations for 3 non-linear models and indicates EQSs and ESSs. Note that in Fig. 2 (c) there are 3 EQSs none of which is an ESS by the original definition. In general (i.e. n > 1), EQSs first have to be obtained by solving simultaneous nonlinear equations:

Fi~ (p) . . . . .

Fir(P)

in S n and by making sure that this is the maximum value among all the Fi(p). Let /max(P) denote the set of all elementary strategies i such that Fi(p) = max., then p is an EQS if and only if supp(p) c /max(P), and R(p) contains all x ¢ p with supp(x) c /max (P)Once p and R(p) are determined, an additional condition must be satisfied only in R (p) for p to be an ESS. Let F(x) denote the vector of the Fi(x), then we have

Lemma 2: An EQSp is an ESS, if and only if ( x - p ) T (F(x)-F(p)) < 0

f o r x inR(p) nearp.

If F(x) has continuous first derivatives in p, lemma 2 can be replaced by a differential condition.

Lemma 3: An EQS p is an ESS, if

So far, we have dealt with the theory of evolutionarily stable strategies as a static theory to explain the constancy of certain forms of behavior in species. There is, however, the dynamical aspect of evolution that may be taken into account, which considers this constancy a result of natural selection. Hence, the question arises, how this aspect is incorporated in the ESS-theory, i.e. whether dynamical systems that describe the selection process, including the "genetics" of strategies, are ESS-adequate. We call a dynamical system ESS-adequate, if ESSs are stable attracting states (attractors) of the dynamic, and strictly ESS-adequate, if in addition, ESSs are the only attractors. The ESS-principle, formally expressed by definition 2, leads to a necessary condition on dynamics t h ~ are to be ESS-adequate. Let x denote a population state and x' the subsequent state, either following from a discrete dynamic or, appropriately defined, from a continuous system. Then the ESS-condition necessitates that the dynamical system produces an improvement x' on x, i.e. that E(x',x) > E(x,x). If there is no possible improvement on a population state, then it must be left unchanged. Note that by definition 2, an ESS must be an improvement on all neighboring strategies, whereas the requirement for a dynamical system is only that it improves on the current x. To emphasize the importance of this requirement, we formulate the

93 D y n a m i c a l ESS-principle : Natural selection produces improvements on current population states. States that allow no improvement are left unchanged.

This principle very much resembles a fundamental theorem in population genetics (Ewens, 1979), which states that, under certain circumstances, natural selection tends to improve the mean fitness in a gene pool. Here, however, " i m p r o v e m e n t " is meant in terms of ESS-theory as given in §3.1 (but cf. §6).

is not very important, since a pay-off matrix can be replaced by a matrix which has only positive values (cf. § 7). Doing so in the case of 2-strategy-models, overshooting is automatically avoided and (DSD) is ESS-adequate (even strictly). For n > 1, overshooting remains a problem (Taylor and Jonker, 1978), but it can be shown that a suitable matrix-shift, which increases E ( x , x ) totally but is irrelevant to the ESS-model, can make (DSD) a better approximation for (CSD) and thus also ESSadequate. 5.3. E S S - a d e q u a c y in non-linear E S S - m o d e l s

5.2. S t a n d a r d d y n a m i c s

Unfortunately, the dynamical ESS-principle describes only a general necessary condition for ESS-adequacy. To see if ESSs are indeed dynamical attractors, one has to consider special dynamical systems in more detail. We do this here for both the continuous (CSD) and the discrete (DSD) standard dynamic, which apply when true breeding of strategies is assumed. (CSD)

xi = x i ( F i ( x ) - / ~ ( x , x ) )

(DSD)

x i ' = xi

Fi(x)

E(x,x)

, i= 1,. . . ,n + 1

They both follow the rule: "Increase, preserve, or decrease the portion xi of the i-th strategy in a given state x, depending on whether it pays off better, equally well or poorer than average ( F i ( x ) > E ( x , x ) ) " . As a first point we can prove L e m m a 4: Both (CSD) and (DSD) agree with the dynamical ESS-principle.

Thus, it makes sense to investigate further the standard dynamics for ESS-adequacy. For linear ESS-models the following has been discussed earlier (Thomas and Pohley, 1982): ESS-adequacy holds for (CSD) in general (even strictly, if n = 1). But for (DSD) there may be non-attracting ESSs due to non-positive values in the pay-off matrix or to divergent overshooting effects. The first reason

Since our concept of a (local) ESS extends the original (global) concept to non-linear models, ESS-adequacy has to be reconsidered for both the continuous and the discrete standard dynamic. With respect to (CSD), we find: T h e o r e m 2: ESSs are attractors of (CSD).

Hence, (CSD) is ESS-adequate even in the non-linear case. For the 2-strategy-models strict adequacy is again implied. With the discrete dynamic, ESS-adequacy is violated for the same reasons as in the linear case. But in contrast to linear models, this is the case even for 2-strategy-models (see Fig. 3)! Indeed, it is interesting to compare the condition for dynamical stability to the ESS-condition (1) in § 4. We can prove T h e o r e m 3: A stationary state p E S ~ is an attractor of (DSD), if and only if, near p,

-2<

x1(1 - - x l )

AF(x)

E(x,x)

X 1 -- p ,

-<0

There is a corresponding differential condition, which should be compared to (2) in § 4: Corollary 2: A stationary s t a t e p E S 1 is an attractor of (DSD), if

--2 <

pl(1 -Pl)

d

E(p,p)

dXl

AF(p) < 0

94 T h e E S S - c o n d i t i o n s are thus o n l y necessary conditions for a dynamical attractor, but not s u f f i c i e n t ones. We have f o r n = 1:

q u e n c y d e p e n d e n t E S S - m o d e l s , since, similar t o t h e linear case, E ( x , x ) can d e l i b e r a t e l y be shifted t o w a r d larger values b y c h o o s i n g an a p p r o p r i a t e " e q u i v a l e n t " m a t r i x (cf. Fig. 3).

Corollary 3: An a t t r a c t o r o f ( D S D ) is an ESS of the c o r r e s p o n d i n g E S S - m o d e l . 6. S y m m e t r i c E S S - m o d e l s This c o r o l l a r y g u a r a n t e e s strictness, whereever (DSD) is E S S - a d e q u a t e f o r 2-strategym o d e l s , a n d d o e s n o t h o l d with t h e global ESS-concept. Figure 3 gives an e x a m p l e of a non-linear 2 - s t r a t e g y - m o d e l which has a m i x e d ESS p = {1/2, 1/2). H o w e v e r , t h e c o n d i t i o n o f c o r o l l a r y 2 is n o t m e t and t h e discrete d y n a m i c reveals a periodic a t t r a c t o r o f p e r i o d 2, r a t h e r t h a n a p o i n t a t t r a c t o r at p. ( E f f e c t s like this are r e c e n t l y discussed in p o p u l a t i o n biology, e.g. May, 1974, 1976). T h e E S S - c o n d i t i o n w o u l d suffice f o r p to be an a t t r a c t o r , if E ( x , x ) c o u l d be altered w i t h o u t a f f e c t i n g A F ( x ) , w h i c h w o u l d preserve EQSs a n d ESSs. R e f e r r i n g t o t h e results in § 7, we state t h a t this is possible for fre-

5F

ei ~

........

4i !

'\\ \

4~

.

\,

.

.

.

t

.

.

.

.

. .~

.

t

"~

1

X1

Pl 4' Fig. 3. ESS-with frequency dependent matrix ( a l l = a,I =s, a,:(x) = h(x) + s, a~l (x) = 8 -- h(x) + whereh(x) = --16x~ + 12, if 0.25 < x~ < 0.75, and a constant otherwise) and ESS p = (0.5, 0.5). Depending on the shift s, p is an attractor (e.g. s = 20, small circles above converging toward p~ = 0.5) or not (e.g. s = 0.5, small circles below diverging from x~ = 0.45 toward the stable cycle p't = 0.213, p'~ = 0.787). Similar results hold for the equivalent symmetric matrix (cf. § 7).

E v o l u t i o n a r y s t a b i l i t y has t o guished f r o m a n o t h e r o p t i m a l i t y which leads to the d e f i n i t i o n o f a strategy (MS), v e r y similar t o t h a t

be distinprinciple,

maximum o f an ESS.

Definition 3: p is a m a x i m u m strategy, if E (p,p) > E ( x , x )

f o r all x ¢ p n e a r p.

In general, ESSs are n o t m a x i m u m strategies a n d vice versa. In a p o p u l a t i o n o f s t a t e x, E (x,x) can be u n d e r s t o o d as t h e m e a n fitness. H e n c e , a MS p d e n o t e s a state o f m a x i m u m p o p u l a t i o n fitness E(p,p), a n d w o u l d be an a p p r o p r i a t e n o t i o n f o r " t h e b e s t o f the s p e c i e s " i n t e r p r e t a t i o n o f n a t u r a l selection, r a t h e r t h a n f o r individual or gene selection ( D a w k i n s , 1982), which is t h e idea b e h i n d an ESS. H o w e v e r , b o t h c o n c e p t s coincide f o r t h e special class o f s y m m e t r i c E S S - m o d e l s characterised b y p a y - o f f f u n c t i o n s w h e r e E ( x , y ) = E ( y , x ) f o r all x,y E sn. t Models o f this k i n d are necessarily linear a n d t h u s d e t e r m i n e d b y a s y m m e t r i c p a y - o f f m a t r i x A = A T . We can show

Lemma 5: F o r s y m m e t r i c E S S - m o d e l s , m a x i m u m strategies are e v o l u t i o n a r i l y stable a n d vice versa. We f u r t h e r o b t a i n an interesting result on the standard dynamics:

Theorem 4: F o r s y m m e t r i c E S S - m o d e l s , a t t r a c t o r s o f t h e s t a n d a r d d y n a m i c s are ESSs.

tThis theoretical property of ESS-models is not related to the symmetries or asymmetries in animal conflicts.

95 Consequently, the standard dynamics are strictly ESS-adequate, when restricted to symmetric ESS-models. 7. On the intrinsic structure o f ESS-models

tinguish between the two models. What matters in the ESS-concept is the relation among strategies as a reply to the same x, which, with respect to the biological interpretation of pay-off values, is only appropriate. We can make this more explicit for ESSmatrices.? We call matrices A and A * equivalent, if A* can be expressed a s A * = d A + C, where d > 0 and C is a matrix with equal rows. Then we have

The objects of ESS-theory are ESS-models. In biological applications different linear and frequency dependent ESS-models are usually designed by means of a cost-benefit analysis of the elementary strategies involved, which yields the pay-off rules that establish a particular ESS-matrix. Theoretically, it is interesting to compare ESS-models that arise in different contexts, and to determine if different models incorporate the same fundamental concepts, albeit with different interpretations. In this section, we investigate significant intrinsic properties of ESS-models.

Hence, a scalar factor or a c o m p o n e n t matrix with equal rows are not "perceived" in the static ESS-theory. They become important, however, if a dynamical description of the selection process is included {see below).

7.1 Eq uivalen t ESS-models

7.2 Canonical components

First, we would want to decide whether two ESS-models are essentially different or not, i.e. provide for identical results concerning EQSs and ESSs. As a simple example, are the models described by

In trying to describe the meaning of a matrix C as above, we were led to derive a list of canonical components of ESS-matrices. Consider pairs of pure strategies. Then -- besides the population state -- two basic facts affect the pay-off: the special combination o f strategies and their position, i.e. what is a reply to what. This suggests considering types of pay-off matrices that deal with only one of these aspects at a time. We obtain the following four unique types:

I:

V -- D V

and (the symmetric)

V

ov 2

_V 2

2

,

0

respectively, different in concept and results? Obviously, if t w o models are defined by the pay-off function E and E*, respectively, they are essentially the same, if

E* (p,x) > E* (q,x)

wheneverE (p,x) > E (q,x)

and vice versa for all p,q,x ~ S n. This is because there is nothing in the definition or theorems a b o u t EQS and ESS that can dis1"For convenience we omit the variable x for frequency dependent models in the following. Results hold for this case, too, unless stated otherwise.

Lemma 6: Equivalent ESS-matrices determine identical sets of EQSs and ESSs.

(a) Symmetric matrices Ma with equal diagonal entries differentiate the types of combinations. (/~) Matrices M~ with equal columns differentiate strategies as replies. (~/) Matrices M 7 with equal rows differentiate strategies which are replied to. For completeness we add the fourth type (5) Matrices M6 with all entries equal are indifferent with respect to the above aspects.

96 I t is n o t e d t h a t m a t r i c e s of t y p e T (including t y p e 5) are e x a c t l y t h o s e m e n t i o n e d a b o v e h a v i n g n o e f f e c t on EQSs a n d ESSs in a m o d e l . B u t t h e y m a y increase t h e values o f E a n d t h u s d e t e r m i n e w h e t h e r or n o t the discrete s t a n d a r d d y n a m i c will c o n v e r g e t o w a r d an ESS (cf. § 5.3). In synthesising E S S - m o d e l s f r o m these f o u r aspects, t h e a- a n d ~ - c o m p o n e n t s already det e r m i n e EQSs a n d ESSs. T h e T - c o m p o n e n t m a y be chosen to realise s y m m e t r i e s , e.g. t a k i n g M 7 = M~ yields a s y m m e t r i c m a t r i x , w i t h t h e c o n s e q u e n c e s as d e s c r i b e d in § 6 f o r linear m o d e l s . In a d d i t i o n , M5 m a y be introd u c e d to let E have values in a desired range. On the o t h e r h a n d , E S S - m a t r i c e s m a y be analysed for these f o u r s t r u c t u r a l c o m p o n e n t s , i.e. d e c o m p o s e d as A = M s + M ~ + M 7+M6. T h e n M s and M~ are the r e l e v a n t parts, a n d the role o f certain m o d e l p a r a m e t e r s introd u c e d with t h e biological i n t e r p r e t a t i o n can be revealed (see Fig. 4). O m i t t i n g t h e technical details o f such a p r o c e d u r e , we o n l y state here t h a t m a t r i c e s f r o m certain s u b s p a c e s m a y be d e c o m p o s e d this w a y . We have

e q u i v a l e n t to m o d e l s with a s y m m e t r i c p a y - o f f f u n c t i o n , a n d t h e i r ESSs are just the m a x i m u m strategies o f t h e e q u i v a l e n t s y m m e t r i c m o d e l

(cf. § 6). 7.3. Canonical decomposition for n = 1. Lemma 7: T w o - s t r a t e g y - m o d e l s unique decomposition.

=

F 0

s/2"

Ls12

0

a

0]

This is given b y

Ms

have

~/2

~/2

$.-

7/2M7 =

M6 =

I °

L0

7/2

6

w h e r e s = (a12 + a=l) - (all + a22), t h e diff e r e n c e b e t w e e n diagonal sums, ~ = (a2~ + a22) - {a~ + a12 ), t h e r o w s u m - d i f f e r e n c e , T = (a12 + a=2) - ( a ~ + a2~ ), the c o l u m n s u m - d i f f e r e n c e , a n d 5 = a ~ . C o n s e q u e n t l y , we have

Corollary 4: T w o - s t r a t e g y - m o d e l s have s y m Theorem 5: (i) I f A is s y m m e t r i c , t h e n A is d e c o m p o s a b l e . (ii) A is d e c o m p o s a b l e if a n d o n l y if A has a s y m m e t r i c equivalent A * This s h o w s t h a t a v a r i e t y o f E S S - m o d e l s are

I ~/2 ÷ :

2 D,~

7-V

~,'2 T

~'+T o

D/2 * T o

T v'- T

-

Fig. 4. The classical strategies Hawk, Dove, Bully, Retaliator analysed for components a and ~ in twostrategy-models. (Value to win denoted by V, loss due to injury by --D and due to wasted time by --T). Note the V does not appear in a, hence, it is only relevant as an a priori value of the strategies, irrespective of their frequencies.

metric

equivalents.

L e t t i n g 7 = /3 yields t h e s y m m e t r i c m o d e l . Accordingly, the two matrices of the above e x a m p l e are equivalent.$ Another example of model equivalency is worth mentioning. Grafen (1979) discussed ESS-models covering "games between relative" with constant pay-offs and an average relatedness r. Lewontin (1958) presented a model of selection in a plant population, which is frequency dependent due to inbreeding with the inbreeding coeffident F. If the latter model is considered a frequency dependent ESS-model (as in § 8), we can show that it is equivalent to a model with a constant pay-off matrix. Suggestively enough, it turns out that this is just a special case of a games-between-relatives matrix with F and r playing the same role. As a n o t h e r c o n s e q u e n c e , t w o strategym o d e l s are c o m p l e t e l y c h a r a c t e r i s e d b y o n l y tThe first matrix has in fact been adopted from the simple Hawk-Dove model.

97 two canonical parameters, a and /3. Hence, AF(x) (cf. § 4) reduces to its corresponding a-, a-parts, i.e. AF~.(x) =/xF6 (x) = 0. It is thus possible to give E a total shift by changing 7 or 6, without affecting AF(x). In the example in Fig. 3, increasing 6 by 1 or more made the ESS become an attractor. From the results of § 4 we find

A(x) to a path in the (a , /3)-plane. Then this path will be transformed into another equivalent path given by a*(x) = d(x) ~ (x),/3*(x) = d(x)/3 (x).

If the original path does not pass the origin (0,0) of the plane, then a standard transformation may be applied:

d ( x ) = m a x ( [ a ( x ) [ , [ / 3 ( x ) [ } -1. Lemma 8: (i) u~ is an EQS, if a + / 3 < 0 , and u2 is a n E Q S , i f a - / 3 < 0. (ii) p E S ~ is an EQS, if it satisfies a ( 1 - 2pl)=/3. For linear models, this result leads to an integrated representation of all ESS-models by points (~,~) of the R2-plane, and of the corresponding EQSs by a multi-layer surface, called the EQS-surface in (Thomas and Pohley, 1982). This allows topological considerations to be made about ESS-models. Mixed EQSs, given by pl = -

1 2

/3 1-- --

,

are ESSs, wherever a > 0. With frequer, cy dependent ESS-models, the above results have to be read accordingly, since we have a and/3 depend on x or, equivalently, on its first c o m p o n e n t x~. These models can also be discussed by means of an integrated representation on the EQSsurface.

7.4. The standard graph So far no regard has been given to the scalar factor appearing in the definition of equivalent matrices. It becomes important, however, with frequency dependent ESSmodels. Since such a model can be described by a family of matrices parameterised by x, we obtain an equivalent model, if, for each x, A (x) is modified by a scalar factor d (x) > 0, i.e. if the family of matrices is multiplied by a positive function of x. Considering 2-strategy-models, we can map

This maps each path to the boundary of the unit square in the (a,/3)-plane. We thus have

Lemma 9: Frequency dependent 2-strategymodels can uniquely be represented along the unit square, if ~ and /3 do not vanish simultaneously. This allows for an easy way to understand the complete model by graphical means, since the cut profile of the EQS-surface along the unit square yields a very simple standard graph (Fig. 5). The standard graph can be supplied with the graph of the given model (or class of models): values of xl are plotted corresponding to the pair of values (~*,/3") determined by x = ( x l , 1 - xl) (cf. Fig. 5). Then the intersection points of this graph and the standard profile mark EQSs. Among these, ESSs are located either, where the graph of the model meets the standard profile at 0 or 1, or by the following graphical condition: Let p be an EQS. Then consider the direction of the transition of the x~-graph relative to the standard profile along increasing x~. Whenever ~*(p) = 1, p is an ESS, if the direction is right-to-left in p 1. Whenever ~* (p) = - 1, it must be a left-to-right transition (as in Fig. 5).

8. The one-locus gene pool model In population genetics, selection dynamical models are used to explain the evolution of stable gene frequencies in a population. As we

98 :i W

3t:

NE

:

t

:

i

i

• 4

i

i

]

]

'1

•

.

Fig. 5. C o m p l e t e standard graph (solid), Iltarting at a* = 1, ~* = --1. The example of Fig. 2b is represented here by its characteristic x I -graph (fine), and ESSs (u) and instable EQSs (o) are indicated. (Letters above indicate correspondence to Fig. 6).

will show, this is just a special interpretation of the class of symmetric ESS-models. Thus, a noted biological theory is recognised as a nucleus of the ESS-theory. 7.1 C o n s t a n t g e n o t y p e fitness

Consider a model gene pool with respect to a locus s and alleles si, i = 1 . . . . , n + 1. Let x i denote the frequency of si in the pool and x = ( x ~ , . . . , xn÷~)r the state of the pool. Assuming random combination, we have genotypes sisj (i < j) formed with probability x~, if i = j, and 2 x i x j , if i < j. Except for the names, the gene pool is thus a valid interpretation of a population of interacting individuals usually considered in ESS-models. Let vii denote the constant selective value of genotype sisj, from which the selective value of allele si and the mean selective value in th~ pool with respect to locus s are obtained as

Vi =

~

i~j

vijx j +

~

i>j

vjix I and

Y=~_,xiVi,

respectively. These terms are used to formulate the dynamics of the change in gene frequencies (Wright,. 1969).

If we attempt to describe the gene pool model in terms of ESS-theory we have to derive a pay-off matrix A from the selective values vii such that the " p a y - o f f " to si, F i ( x ) = Y, aijxj, is given by its selective value Vi. Consequently, A must be the s y m m e t r i c matrix that consists of aij = vii,

ifi<_j,

and

aij = vji, if i > j.

Then E ( x , x ) -- x T A x equals the pool fitness V and the dynamics take the standard form described in § 5. This suggests considering selection models as a special interpretation of symmetric ESSmodels. However, the gene pool model of selection is primarily based on a dynamical formulation of stable states. Therefore the question has to be considered, whether this is completely covered by the ESS-principle. Since A is a constant symmetric matrix, the pay-off function E is symmetric. By the results of §6, ESSs and states of maximum pool fitness agree in this case, which are indeed the stable states of the gene pool. The significance of this result is twofold. First, a special class of ESS-models has an immediate biological interpretation: ESSmodels with symmetric matrix may be understood on the gene pool level. By theorem 5,

99 this extends to an even larger class of models with decomposable matrix, all two-strategymodels included. It is the fact that asymmetric pay-off matrices may be employed in ESSmodels, which makes the ESS-concept applicable to a much wider range of biological problems. Second, methods and results of the ESStheory may be applied to selection dynamical models. As an example, we apply a, ~-analysis and standard graph methods to two alleles.

7.2. The a-6-mandala The " p o s i t i o n " of an allele in the genotype is assumed to have no effect on its selective value. Consequently, we do not have to regard ~,, since this must equal 13. Whenever a > 0, then there is heterocygote advantage, provided g does n o t outweigh a. For a < 0, there is definitely a h o m o c y g o t e advantage. We consider the relations among the fitness values of the three genotypes to describe the selection structure of the pool. This is then completely determined by the values of a and/3. Using the m e t h o d of the standard graph, we may identify and arrange all essential selection structures in a global representation, which we refer to as the a-5mandala (Fig. 6). Along the square of the mandala, i.e. the unit square of the standard graph representation, the eight selection structures originating NW

\

/

+I-

^

s

I~ - o - f ~

')

N

¥

0

;3

-1

1"0

E

", / hiE

-7 I

/3" I

Fig. 6. The ~-~-mandala.

+1 I

-I-

from the neutral case (a = ~= 0) in the center are indicated as the edges and corners. In population genetics some of these are known, e.g, as complete and incomplete dominance and overdominance (Sperlich, 1973). ESS-theoretical results determined by a selection structure are immediately obtained from the standard graph. To the west of the mandala (a* = 1, i.e. a > 0, 13 < a) a mixed gene pool is evolutionarily stable and determines the mixture. To the north (/3* -- 1, i.e. > 0, a < ~) the evolutionarily stable pool is pure s2, to the south (t3* = - 1 , i.e. 13 < 0, > 13) pure sl. To the east (a* = - 1 ) , i.e. < 0, t3 > a ) either one of the pure pool states is evolutionarily stable with the EQS-line separating their regions of attraction. Northeast and southeast indicate "catastrophic" selection structures in the sense described by Thomas and Pohley (1982).

7.3. Frequency dependent genotype selection In population genetics, models of frequency dependent selection are designed by means of genotype fitness values vii(x) which depend on the frequencies of genotypes and thus on the state x of the gene pool (Lewontin, 1958). It is known that frequency dependent selection does not necessarily tend to maximise the pool fitness. Accordingly, the corresponding ESS-models are frequency dependent and make use of a family A (x) of matrices parameterised by x. It is noted that, although A (x) is symmetric, the pay-off function E is not, so that the results of § 6 do not apply. That is, the ESSprinciple does not generally cover the standard dynamical descriptions of the selection process: theoretically, there may be more stable states than ESSs in special cases. Strict adequacy holds for two alleles, although maybe only after an appropriate shift of the selective values assigned to genotypes. Examples of biological significance are found e.g. in (Lewontin, 1958; Clarke and O'Donald, 1964). We adopted some of these as ESSmodels and gave a graphical analysis in Fig. 2.

100 N o t e t h a t in the examples o f Clarke and O ' D o n a l d there is m o r e t h a n one inner stable state (Fig. 2b,c). N o n e o f these w o u l d have been evolutionarily stable in the global sense o f the original ESS-definition. Unlike linear models, f r e q u e n c y d e p e n d e n t selection m o d e l s typically d e t e r m i n e variable selection structures. D e p e n d i n g on the freq u e n c y o f alleles or, impli~itely, of genotypes, different parts o f the a-~-mandala m a y b e c o m e i m p o r t a n t . In particular, the same stable pool state m a y o c c u r u n d e r different t y p e s of selection s t r u c t u r e ruling.

References Brockmann, H.J. and Dawkins, R., 1979, Joint nesting in a digger wasp as an evolutionarily stable preadaptation to sodal life. Behaviour 71, 203-245. Brockmann, H.J., Grafen, A. and Dawkins, R., 1979, Evolutionarily stable nesting strategy in a digger wasp. J. Theor. Biol. 77,473--496. Clarke, B. and O'Donald, P., 1964, Frequency dependent selection. Heredity 19, 201--206. Dawkins, R., 1982, The extended phenotype {Freeman, Oxford).

Ewens, W.J., 1979, Mathematical population genetics (Springer, New York). Grafen, A., 1979, The Hawk-Dove game played between relatives. Anim. Behav. 27,905--907. Haigh, J., 1975, Game theory and evolution. Adv. Appl. Prob. 7, 8--11. Lewontin, R.C., 1958, A general method for investigating the equilibrium of geae frequency in a population. Genetics 43,419--434. May, R.M., 1974, Biological populations with nonoverlapping generations: stable points, stable cycles and chaos, Science 186, 645--647. May, R.M., 1976, Simple mathematical models with very complicated dynamics. Nature 216, 459-467. Maynard Smith, J., 1974, The theory of games and the evoluton of animal conflicts. J. 2~neor. Biol. 47,209--221. Maynard Smith, J. and Price, G.R., 1973, The logic of animal conflicts. Nature 246, 15--18. Selten, R., 1980, A note on evolutionarily stable strategies in asymmetric animal conflicts. J. Theor. Biol. 84, 93--102. Sperlich, D., 1973, Populationsgenetik (Fischer, Stuttgart). Taylor, PD. and Jonker, LB., 1978, Evolutionarily stable strategies and game dynamics. Math. Biosci. 40, 145--156. Thomas, B. and Pohley, H.-J., 1982. On a global representation of the dynamical characteristics in ESS-models. BioSystems 15, 141--153. Wright, S., 1969, Evolution and the genetics of populations, Vol. 2. (University Press, Chicago).