- Email: [email protected]
Best replies and adaptive learning
mathematical social sciences ELSEVIER
Mathematical Social Sciences 30 (1995) 221-234
Best replies and adaptive learning J o s h u a S. Gans* Department of Economics, Stanford University, Stanford, CA 94305-6072, USA Received July 1994: revised February 1995
Abstract
This paper discusses the relationships between learning processes based on the iterated elimination of strictly dominated strategies and their myopic and more naive counterparts. The concept of a monotone game, of which games with strategic complementarities are a subclass, is introduced. Then it is shown that convergence under best reply dynamics and dominance solvability are equivalent for all two-player (and some many-player) games in this class. Keywords: Monotone games; Strategic complementarities; Best reply dynamics; Adaptive
learning; Dominance solvability; Random matching
1. Introduction
Perhaps the simplest and most naive dynamic adjustment process is the best reply dynamic. 1 Under this dynamic, in disequilibrium, agents adjust their strategies by observing the immediate past state of play and choosing strategies as if that state will persist into the future. No account is taken of the history of strategies, their direction of movement, their signalling of another player's type, or the possibility that better strategies off the best reply path are never reached. All of these considerations might provide a better signal of the future choices of other players. None the less, despite its implied myopia, the simplicity of best reply dynamics is a virtue for the applied economist, in that it is quite easy to * Corresponding address: School of Economics, The University of New South Wales, Sydney, New South Wales, 2052, Australia ~These are sometimes referred to as best response dynamics, Marshallian dynamics, Cournot dynamics, or Cournot tatonnement. 0165-4896/95/$09.50
© 1995 - Elsevier Science B.V. All rights reserved
SSDI 0165-4896(95)00792-X
222
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
determine, in many games, the outcomes that result from such adjustment behaviour. Best reply dynamics have another quality. The paths generated under best reply dynamics are a subset of what has been termed 'adaptive learning'. As Milgrom and Roberts (1991) have shown, many learning and dynamic process- naive or sophisticated-have the quality that play eventually lies within the serially (strictly) undominated set of strategies. Such dynamic processes are defined to be consistent with adaptive learning. 2 The definition of adaptive learning is associated with strict dominance. Therefore, in what follows, when I refer to dominant or undominated strategies, or dominance solvability, it will be to the strict version of these concepts) Milgrom and Roberts (1991) show that the significant feature of dynamic processes consistent with adaptive learning is that if such processes converge, they will converge to a Nash equilibrium. As a corollary, if a game is dominance solvable, then all adaptive learning processes converge to the unique serially undominated profile. Therefore, if a n y normal form game is found to be dominance solvable, then the Milgrom-Roberts theorem implies that best reply dynamics will also converge to their unique equilibrium point. Determining whether behaviour under best reply dynamics converges is easier than showing that a given game is dominance solvable. Therefore, it is the task of this paper to search for classes of normal form games with the converse implication: that if best reply dynamics converge, then so will all other processes consistent with adaptive learning. This would establish an equivalence between convergence under best reply dynamics and dominance solvability.4 In these games then, one need only check that play is convergent under best reply dynamics to conclude that even much more sophisticated dynamic processes will converge to that same outcome.
2. Notation and formulation
There are IN[ players from a set, N, where each player is indexed by n. A pure strategy, x n =- ( x l n , . . . , x i , , ) , for an individual player comes from the set An, with
2 See also the related results of Gul (1991), which look at rationalizable as opposed to undominated strategy sets. 3 Marx (1994) has derived results based on learning concepts related to weak dominance solvability. Not surprisingly, her results do not offer the strong theorems that arise from adaptive learning as considered by Milgrom and Roberts (1991). 4 Note, however, that convergence under best reply dynamics is not guaranteed in the games to follow. It is well known that best reply dynamics do not always converge- see Milgrom and Roberts (1991) for a review of these results. Here it will be shown that when best reply dynamics converge then these games must be dominance solvable.
J.S. Gans / Mathematical Social Sciences 30 (199.5) 221-234
223
dimension I n. Let each X, be a compact lattice 5 and define X =-- Xn@NX n. The payoffs of players are given by the function, %(xn, x_n), where x , denotes the strategies of all other players. It is assumed that 7r, is upper semi-continuous in % for x ,, fixed, and continuous in x , , for fixed x n.6 A pure strategy, x, EX~, for player n is said to be strictly dominated by another pure strategy, 2,, if ( V x _ , ) % ( x , , x , , ) < r r , ( i , , x ~). Given a product set, A', of strategy profiles, the set of n's undominated responses to ~" is defined by
u n ( 2 ) = {xn
x,, I (Vx'
xo)(3
I>
•
Let U ( X ) = ( U , ( X ) ; n E N) be the set of undominated responses for each player. Define X ° = X, the full set of strategy profiles. For ~-i> 1, define X" = U ( X ' - ~). A strategy, x,,, is serially undominated if x , , G U , ( X ' ) , W-; these are just the strategies that survive the iterative process of removing strategies that are strictly dominated by other pure strategies. Observe that U is a monotone operator. That is. if )( C_X', then U ( X ) C_U(X'). The dominance solution of the game is the set of strategies that remain after iterative application of this process and the game is pure dominance solvable if the dominance solution set for each player n, U~(X*), consists of a single point.
2.1. Adaptive learning Milgrom and Roberts (1991) show how the operator, U, can be used to define what is meant by adaptive learning. Definition 1. A sequence of strategies {xn(t)} is consistent with adaptive learning by player n if ( V f ) ( 3 T ) ( V t >I T)xn(t ) E Un({x(s ) ] i ~
224
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
dominance. Adjustment processes, ranging from simple processes like fictitious play to more sophisticated ones that have players using full information from past play and even experimentation, are all consistent with adaptive learning. Milgrom and Roberts (1991, theorem 3(b)) show that processes that converge to Nash equilibria are consistent with adaptive learning. Moreover, they prove that, if a game is dominance solvable, then every process consistent with adaptive learning will converge to that profile - the unique Nash equilibrium of the game.
3. Best reply dynamics and adaptive learning Having defined the concept of adaptive learning and noted some of its implications, I will now turn to consider in more detail its relationship with best reply dynamics. First, I will state a precise definition of the best reply dynamic. Then I will show how convergence under best reply dynamics and dominance solvability are equivalent in a class of games that I term 'monotone' games. In so doing, I will review how this class of games extends previous similar results by Moulin (1984), Milgrom and Roberts (1990, 1991) and Milgrom and Shannon (1994). The set of best responses for player n to some x_n is B . ( x n) = {x. E X n
xn) o(x., x_n) >t
X-n)}.
Suppose the game is repeated over a number of periods indexed by t. Under best reply dynamics, in each period, t, each player chooses a strategy that is a best reply to the strategies played by the other players in the previous period, t - 1, i.e. xn(t ) E B n ( X _ , ( t - 1 ) ) . Here, best reply dynamics are taken to be of the simultaneous kind where, in each period, players adjust their strategies at the same time] Thus, under these dynamics, players never use strategies that are strictly dominated by another strategy. A sequence of strategies is consistent with best reply dynamics if each strategy played is a best response to the immediate past strategy of the other players. Formally:
Definition 2. A sequence of strategies {x,(t)} is consistent with best reply dynamics by player n if (Vt)xn(t) E B , ( x _ , ( t - 1)). A sequence of strategy profiles {x(t)} is consistent with best reply dynamics if each {xn(t)} has the property. Convergence under best reply dynamics is taken here to mean that every sequence of strategies consistent with best reply dynamics convergences globally. 7 Moulin (1986) makes a distinction between sequential and simultaneous best reply dynamics. In the sequential version, adjustment takes place one player at a time. Theorem 1, below, will be shown to apply to both simultaneous and sequential best reply dynamics.
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
225
Moulin (1984) was the first to identify a class of games under which convergence of best reply dynamics and dominance solvability are equivalent. In 'nice' games, players are assumed to choose their strategies from a one-dimensional compact interval, X~, and receive a payoff determined by the continuous function, ~r~, over X that is strictly quasi-concave with respect to xn. These assumptions imply that the best response correspondence, Bn, is a single-valued function. Moulin then shows that for two-player nice games, global convergence under best reply dynamics implies that the game is dominance solvable. He also provides an example indicating that this result does not extend to the three-player case. As Moulin notes, this class of nice games (even without any additional condition) is very restrictive. Indeed, it precludes the discontinuities of the best response correspondence that accompany many games of interest to economists today, in particular those games with non-concave payoffs.
3.1. Monotone games Here, I introduce a class of games that is not, in general, nice but generalizes some of the properties of nice games. In particular, this class of games allows a multidimensional strategy space and non-concave payoffs. I term these games monotone games because each player has a monotone best response correspondence .8 Before defining this class of games, it is worth re-considering the properties of a subclass of such games, i.e. games with strategic complementarities. Formally, strategic complementarities arise when payoffs, 7rn(x,,, x , , ) , on some ordering of strategy spaces, satisfy certain single-crossing and quasi-supermodularity conditions. That is, given x,, ~>x'n, a payoff satisfies the single-crossing property in (x,,;x ~,,) if
~,,(x.,x'.)>~(>)~.(x'.,x'o)~r~(x,,,x
,,)>~( > ) ~ ( x ' . . x
~), Vx ~ >~x'n ,
and is quasi-supermodular in x n if, given x , , ,
7r,,(Xn,X_D >~(> )~rn(X. ^ ~.,X °) ~ Tr.(X~ v~,X .) ~>(>)Tr.(/., x _ . ) ,
Vx.~i..
Definition 3. A game, [" = (N, {X., 7r.}.~U), is a game with strategic complementarities if, for all n, ~'.(xn,x ,,) is quasi-supermodular in x. and satisfies the single-crossing property in (x. ; x_.). Marx (1993) provides a related but alternative definition of a monotone game based on concepts from lattice theory, convex analysis and non-smooth analysis. Her focus of analysis is quite different from that here, however.
226
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
These games have received wide attention of late through the work of Topkis (1979), Vires (1990), Milgrom and Roberts (1990) and Shannon (1992). For the purposes here, the significance of these games lies in their implications for the monotonicity of best response correspondences. Milgrom and Shannon (1994, theorem 4) have shown that best response correspondences in games with strategic complementarities are monotone non-decreasing in x ,. However, their conception of monotonicity is based in Veinott's strong set order to allow for best response sets that are not singletons? To see this order, let x,, v x~, denote the component-wise maximum of x n and x ' , and x,,/x x', the component-wise minimum, and let B,, =- B.,(x .; S,,) and B~, --- B.(x' .; S,,). Then Veinott's strong set order that says that B,, ~ B . if, for every x. E B,, and x. E B,,, x,, v x. E B'. and x,, ^ x'. ¢ B~,. A best response correspondence is monotone non-decreasing if B,, >/B ~, for all x,,/> x'. and all S. C_X.. Milgrom and Shannon (1994. theorem 4) show that, with fixed strategy sets, the quasi-supermodularity and single-crossing conditions in Definition 3 are both necessary and sufficient for best response correspondences to be monotone non-decreasing in x , , ) ° Hence. one can write an alternative definition of games with strategic complementarities. p
t
t
P
Definition 3'. A game, F = (N, {X., 7rn}.~N), is a game with strategic complementarities if, for all n. B . ( x .; X,,) is monotone non-decreasing in x .,. The emphasis here on the monotonicity of best response correspondences leads, naturally, to a conception of a wider class of games. Definition 4. A game, F = (n, {X n, 7r.}ncN), is a monotone game if, for all n, B, (x_,, ; X , ) is monotone in x ,,. This class of games is broader than the class of games with strategic complementarities because it allows for the case where B , ( x , ; X , ) is monotone nondecreasing in x , , for some subset of players, while being monotone nonincreasing for the rest. To emphasize, games with strategic complementarities allow B , ( x _ , ; X n ) to be monotone non-decreasing in x_, for all players or m o n o t o n e non-increasing in x ~ for all players. On the other hand, one can imagine games where the best response correspondences are non-decreasing for some players and non-increasing for all other players - a mixed monotone game.
"This is in contrast to Moulin's (1984) nice games that are formulated to ensure that best response sets are singletons. ~°Athey (1995) extends the Milgrom and Shannon (1994) monotonicity characterizations to conditions of probability density functions that are part of expected payoff functions.
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
227
A game is mixed monotone if it is monotone and not a game with strategic complementarities. 11 None the less, by the necessity part of theorem 4 of Milgrom and Shannon (1994), all monotone games require that payoff functions are quasi-supermodular in x,, and satisfy the single-crossing property in (x,,, x ,,) or in (x,,. -x ,,). 3.2. E x a m p l e : Innovation and imitation 1~_
While examples of games with strategic complementarities abound in the literature, mixed monotone games are themselves potentially important. Consider a situation in which an established innovator in an industry decides the level at which to sink costs into developing a new product. A high level of product development has the advantage of raising revenue for the firm but, of course, at greater cost. The product development strategy of the innovator, L, is denoted by a real number x L ~ [XL,)~L]. Thus, the payoff to the innovator is, 7rL(X ) = R L ( X L , X v ) - - C L ( X L ) . Here XF~[XF,)~F] denotes the product adopted by an imitator, F. The greater this is, the less is the marginal revenue to the innovator from raising product quality, i.e. 02RL
--<~0.
Ox L Ox v
This assumption is essentially a smooth analogue of the single-crossing property (Topkis, 1979). Hence, by theorem 4 of Milgrom and Shannon (1994), this is a sufficient condition for the best response correspondence of the innovator to be monotone non-increasing in the strategy choice of imitator. The imitator's payoff is ~'F(X)= R F ( X F , X L ) - CF(XF,XL)with O2RF
--~>0
OXL OXv
O2CF
and
--~<0.
OXL OXF
That is, the higher the quality of produce the innovator develops, the higher the marginal net benefit to the imitator of increasing their product quality. This payoff function implies that the optimal strategy of the imitator is increasing in that of the innovator. Hence, the game is mixed monotone - no reordering of the strategy spaces will make it a game with strategic complementarities. Note that, unlike Moulin (1984), no assumptions are made on the concavity or otherwise of the respective payoff functions - indeed, it is unreasonable to assume
" Therefore, a mixed m o n o t o n e game cannot become a game with strategic complementarities by simply re-ordering the strategy space of players. ~2This example is of a similar flavour to that of Grossman and H e l p m a n (1991, ch. 12).
228
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
in the presence of sunk costs that payoff functions are concave. ~3 None the less, as will be shown by Theorem 1 below, convergence under best reply dynamics, and convergence under adaptive learning, are equivalent in such situationsJ 4
3.3. Equivalence result: Two-player games Milgrom and Roberts (1990, 1991) and Milgrom and Shannon (1994) show that, for games with strategic complementarities and any number of players, the coincidence of convergence under best reply dynamics and dominance solvability holds. ~5 They require neither the condition of quasi-concavity nor that the strategy space is of one dimension, as assumed by Moulin. The following theorem extends the results of Milgrom and Roberts (1990, 1991) and Milgrom and Shannon (1994) to all two-player monotone games. ~6 Recall, though, the fact that dominance solvability implies convergence under best reply dynamics was shown to hold for all normal form games (Milgrom and Roberts, 1991). Theorem 1. In two-player monotone games, if play converges globally under (simultaneous) best reply dynamics, then the game is dominance solvable. Proof. Note that Milgrom and Roberts (1991) and Milgrom and Shannon (1994) state the result for all games with strategic complementarities. Thus, all that remains is to prove the theorem for mixed monotone games. This is done in the following lemma.
13Note that one could also make the strategies multidimensional, with a few extra conditions to preserve monotonicity, again going beyond Moulin (1984). 14This would also be the case with many followers who are unaffected by each other's strategy choices, but where the innovator is affected by all their strategies in a monotone non-decreasing way. This extension is achieved using Corollary 1, below. ~5More to the point, they show that uniqueness of equilibrium implies dominance solvability for games with strategic complementarities. Then the coincidence follows since global convergence under best reply dynamics implies that the equilibrium is unique. ~6In the proof of his theorem Moulin (1984) implicitly assumes an extra condition. Consider any two ordered single-dimensional strategies x I > x', for player 1 in a two-player game. Moulin argues (p. 90) that, if player 2 uses a strategy x2 ~>x~, then x~ is dominated by x, for player 1. This is true if zq(xl,x2) crosses 7h(x~,x2) once from below over the X2 domain. In this case, ~'~(xl,x~)~> (>)Trl(x~, x ' ) implies that ,q(xl, x2)>~ (>)Trl(x~, x2). The conditions for a nice game, however, do not rule out more than one crossing. For instance, consider two non-connected intervals, [,~z,£2] and [~2, x2], subsets of X 2 with £2 < ]~. It is possible that, given x~ > x ~, Ir~(x, x2) ~>( >)~rl (x ;, x2) for all x2 E [~2, i2] LI [~2, ]2] and *r~(xl, x2) < ~r~(x~,x2) , otherwise. In this case, use of a greater strategy for player 2 does not imply that the higher strategy of player 1 is dominated. Observe, however, that the single-crossing property, which is an integral part of monotone games, rules out such difficulties.
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
229
Lemma 1. In two-player mixed monotone games, if play converges globally under (simultaneous) best reply dynamics, then the game is dominance solvable. Proof. Milgrom and Shannon (1994, p. 176) show that U~(X ~) contains smallest and largest elements, _xn and £n, for each player n, by the compactness of the strategy space and the continuity assumptions on payoffs. Without loss of generality, suppose that B~ is monotone non-decreasing and B 2 is monotone non-increasing. Now it can be shown that (i) _x~ E B~(_x2); (ii) £1 C B~(£2); (iii) 2~ E B2(x l ); and (iv) x 2 E B2(21). Suppose (i) does not hold, then there exists an x~ > x 2 such that _x1 ~ B~(x~) and there exists an x I > x I such that x I E Bl(X2), by the definition of U~(X ~) and U2(X ~) and the lemma (p. 176) of Milgrom and Shannon (1994). However, this implies that B~ is not monotone non-decreasinga contradiction. A similar argument shows (ii). Suppose (iii) does not hold, then there exists an x 1 >x~ such that £z C B2(x ~) and there exists an x, < x 2 such that x'~ ~ B2(xl). Again, we have a contradiction, since this implies that B~ is not monotone non-increasing. A similar argument shows (iv). Now observe that, using facts (i)-(iv), one can construct an infinite sequence composed of the following subsequence {(£1, £2), (£1,-x2), (_x~,_x2), (_x~,.~2)}, which is consistent with simultaneous best reply dynamics. However, for this to be convergent, x_, = £~, Vn, which in turn implies that U , ( X ~) must be a singleton for each n. Hence, the game is dominance solvable. Q.E.D. Remark 1. Note that this theorem applies even if best reply dynamics are of the sequential kind. Facts (i)-(iv) imply that an infinite sequence composed of the following subsequence {£2,£1,x2,x_~} is consistent with sequential best reply dynamics, iv Remark 2. We know, from Milgrom and Roberts (1991), that the above theorem can be proved for all monotone games that are games with strategic complementarities. Note then that, since all 2 x 2 games are monotone games, global convergence under best reply dynamics is a necessary and sufficient condition for a 2 x 2 game to be dominance solvable. Remark 3. Neither monotonicity nor the conclusions of Theorem 1 holds for general two-player games, however. Consider the following 2 × 3 non-monotone game in Fig. 1, under which best reply dynamics converge but the game is not dominance solvable. j7 Note that if players' payoffsare strictlyquasi-concavein their own strategies, then this is sufficient to ensure that the equilibrium of two-playermixed monotone games is unique and that if a best reply dynamic converges it will converge to it.
230
J.S. Gans /
MathematicalSocial Sciences 30 (1995) 221-234
II 1
2
4, 10
O, 0
3,0
3,1
O, 10
4,0
Fig. l
Remark 4. The theorem does not extend, in general, to games with more than two players. Suppose the population can be divided into two types of players, m E M C N and r n ' ~ MW. Suppose that for players with index m ' , Bm,(x_m,) is non-decreasing in x_m,, and that for players with index m, Bm(X_m) is nonincreasing in x _ m. It is then possible to show that (i) X m, EBm,(X__m,); (ii) £m' ~ Bm'(X m'); (iii) 2,,, E Bm(X_m); and (iv) X m ~ Bm(.~_m). However, in the many-player case, it is not possible to use the two-player argument to construct a non-convergent best reply sequence from these facts. I will consider, in Section 4, a particular many-player context that does allow an equivalence between dominance solvability and convergence under best reply dynamics.
4. Many-player monotone games Does the equivalence of best reply convergence and dominance solvability extend to any many-player context? It was noted earlier that this equivalence does not hold for general many-player mixed monotone games. Consider, however, a random matching game. That is, suppose that players are randomly matched to play a two-player mixed monotone game from two separate populations of equal size, m ~ M C N and m ' E M ~ N = = - M ' . In this section, assume that players' strategy spaces are uni-dimensional and are represented by intervals of the real line, X m and Am,, respectively. Under best reply dynamics, each player uses a pure strategy that is a best response to the frequency distribution of pure strategies played by the population in the immediate past periods. Let fM(Xm;~p): X M X ~ - ~ [ 0 , 1] and fM,(Xm;O): X M, x O----~ [0, 1] be parameterized families of probability density function that give the proportion of members of populations M and M' playing strategies x m and x,,., respectively, in the immediate past period. ( ~ and O are totally ordered sets). The parameters q5 and 0 order the resulting frequency distributions from these functions. Imposing some ordering on the frequency distributions is necessary to define monotonicity of best response correspondences in this context. Krishna (1992)
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
231
and Kandori and Rob (1992) assume that these parameters order these distributions (partially) according to first-order stochastic dominance (FOSD). Since such orderings are arbitrary, I assume instead that frequency distributions of the state of play are ordered according to a generalized monotone likelihood ratio property (MLRP).
Definition 5. Let S(~b) = {xm E X,, [ fM(Xm'~~) > 0}. Then fM(Xm; ~) satisfies the generalized monotone likelihood ratio property (MLRP) if given any xm E S(4~) and x'm E S(4~'), (i) min[xm, Xm, ] E S(4~) and max[x,,, x,,,] E S(~b'); and (ii) for all ~)<~)', fM(Xm;t~)/fM(Xm;(~') is monotone non-increasing in x m for all x , , ~
s(4,)
n
s(4,').
An analogous definition can be written for fM'(')" This parameterization is a generalization of the more familiar monotone likelihood ratio property that excludes condition (i) but does not allow for densities that are zero at p o i n t s - a feature that is essential for game theoretic applications. However, while it is a less complete ordering than FOSD, it allows a consideration of monotonicity in a wider class of games. That is, while a cardinal condition, such as increasing first differences, is required on payoffs when the order on frequency distributions is FOSD, the generalized M L R P derives a monotone best response correspondence for original structures on payoffs, such as the single-crossing property) 8 Ordering frequency distributions according to the generalized MLRP is similar, in two-player games, to ordering mixed strategies or players' beliefs in the same manner. Therefore, it becomes possible to prove the following theorem. Theorem 2. Take any player n in a two-player game (including random matching games). Let f(x_,; O): X , x 6)--*[0, 1] be the density function representing n's beliefs regarding the strategy choices of the players in the other population, where 0 parameterizes f according to the generalized MLRP. If 7rn(xn, x ~) has the singlecrossing property in (xn, x_,) ((x~, -x_,)), then B,(O ), is monotone non-decreasing (non-increasing) in O.
Proof. The proof relies on the following lemma, which is an extension of theorem 11.1 of Karlin (1968), to allow for functions that can take on zero values.
18 Athey (1995) addresses this point in considerably more detail by identifying the close relationship between stochastic dominance and supermodularity in stochastic contexts. She also shows that the generalized MLRP is equivalent to Ormiston and Schlee's (1993) definition of MLR dominance. Finally, Athey shows that if one imposed a cardinal complementarity condition (e.g. supermodularity) on payoff functions, then one could use FOSD to order the frequency distributions and generate monotone best response correspondences, as in Theorem 2 below.
232
J,S. Gans / Mathematical Social Sciences 30 (1995) 221-234
Lemma 2. Consider two functions g : Z x Y---~ ~ and h : Z x Y---~ ~ where Z, Y C_!~. Assume that h(z, y) >i 0 satisfies the generalized MLRP. Assume further than g satisfies the following: (a) For each z, g(z, y) changes sign at most once, and from negative to positive values, as y traverses Y; (b) For each y, g(z, y) is non-decreasing in z; and (c) k(z) = S g(z, y)h(z, y) dy exists and defines a continuous function of z. Then k(z) changes sign at most once, and from negative to positive values. Proof. Without loss of generality, assume that Z is an interval. It needs to be shown that k(zo)= 0 implies k(z)/>0, Vz > z o. Suppose, then, that k(zo)= 0 Further, there exists Y0 such that g(Zo, y) ~>O, Vy > Yo and g(z o, y) <~O, Vy < Yo. (The assumption that k(zo) = 0 and (a) ensure that Yo exists). Now, observe that
h(z o, yo)k(z) = h(z o, yo)k(z) - h(z, yo)k(zo) = f h(zo, yo)h(z, y)(g(z, y) - g(Zo, y)) dy + f (h(z0, yo)h(z, y) - h(zo, y)h(z, yo))g(Zo, y) dy. If z > Zo, then the second integral is non-negative by the choice of Yo and the generalized M L R P assumption on h(z, y). The first integral is non-negative because of assumption (b). Q.E.D. Now, it can be seen that Ex °[Tr(x,,x_n)[0] = .~x , ~r(Xn,X-n)f(x-n; O)dx_ n has the single-crossing property in (x n; 0). This is done by applying Lemma 2 with g(x, y) = 7rn(x'n, x_n) - 7r,(xn, x_n) and h = f , with y = x_ n and z = 0. In this case, k(O) = Ex ,[Tr(x'n, x_n) ]0 ] - Ex ,[Tr(Xn, x , ) ] 0 ] changes sign once from negative to positive, which is the single-crossing property. This satisfies the conditions of theorem 3 of Milgrom and Shannon (1994), completing the proof. Q.E.D. Theorem 2 extends the previous results of Krishna (1992) and Kandori and Rob (1992) to games with ordinal payoffs. They studied games where strategic complementarities arose as a result of increasing first differences, rather than the weaker condition of the single-crossing property. The above lemma extends their results to an even wider class of games, since it only requires the single-crossing property. ]9 Therefore, the above theorem shows that, if players in population M have payoffs with the single-crossing property in (xm;xm,) o r (x m ; - x m,) and players in 19Theorem 2 is related to results in Athey et al. (1994) and Athey (1995). The former allows 7r to have the weak single-crossing property, but uses the strict monotone likelihood ratio for density functions that are strictly positive everywhere. Athey (1995) allows for the possibilitythat 7r itself might depend on 0.
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
233
p o p u l a t i o n M ' have payoffs with the single-crossing p r o p e r t y in (Xm,; Xm) or (Xm,;--Xm), then the resulting game is m o n o t o n e . The following can then be p r o v e d easily, using T h e o r e m 1. C o r o l l a r y 1. In many-player random matching, monotone games, if play con-
verges globally under (simultaneous) best reply' dynamics, then the game is dominance solvable. Proof. This comes f r o m using the same m e t h o d of p r o o f as T h e o r e m 1 and noting its effective equivalence to two-player contexts. That is, the p a r a m e t e r s 0 and ~b act in a similar m a n n e r to the pure strategy choices of the o t h e r player in t w o - p l a y e r games. H o w e v e r , it should be n o t e d that, unlike T h e o r e m 1, it does not extend to the case of sequential best reply dynamics.
Acknowledgements I wish to t h a n k Paul Milgrom for suggesting this topic to me. Also, thanks go to C a t h e r i n e de F o n t e n a y , Scott Stern, seminar participants at Stanford University, and, especially, Susan A t h e y for helpful discussions. All errors remain by own.
References S. Athey, Monotone comparative statics in stochastic optimization problems, mimeo, Stanford University (1995). S. Athey, J.S. Gans, S. Schaefer and S. Stern, The allocation of decisions in organizations, Discussion Paper, No. 94/29, School of Economics, University of New South Wales (1994). G.M. Grossman and E. Helpman, Innovation and Growth in the Global Economy (MIT Press, Cambridge, MA, 1991). F. Gul, Rationality and coherent theories of strategic behavior, mimeo, Stanford University (1991). M. Kandori and R. Rob, Evolution of equilibria in the long run: A general theory and applications, mimeo, Princeton University (1992). S. Karlin, Total Positivity, Vol. I (Stanford University Press, Stanford, 1968). V. Krishna, Learning in games with strategic complementarities, Working Paper, no. 92-073, Harvard Business School (1992). L.M. Marx, Monotonicity of solution sets for parameterized optimization problems, Discussion Paper, no. 1067, Northwestern University (1993). L.M. Marx, Learning to play weakly undominated strategies, mimeo, Northwestern University (1994). P. Milgrom and J. Roberts, Rationalizability, learning, and equilibrium in games with strategic complementarities, Econometrica 58, no. 6 (1990) 1255-1277. P. Milgrom and J. Roberts, Adaptive and sophisticated learning in normal form games, Games Econ. Behav. 3 (1991) 82-100. P. Milgrom and C. Shannon, Monotone comparative statics, Econometrica 62, 1 (1994) 157-180. H. Moulin, Dominance solvability and Cournot stability, Math. Soc. Sci. 7 (1984) 83-102.
234
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
H. Moulin, Game Theory for the Social Sciences, 2nd ed., (New York University Press, New York, 1986). M.E. Ormiston and E.E. Schlee, Comparative statics under uncertainty for a class of economic agents, J. Econ. Theory 61 (1993) 412-422. C. Shannon, Complementarities, comparative statics and nonconvexities in market economies, unpublished Ph.D. Thesis, Stanford University (1992). D. Topkis, Equilibrium points in non-zero sum n-person submodular games, SlAM J. Control Optimiz. 17 (1979) 773-787. X. Vives, Nash equilibrium with strategic complementarities, J. Math. Econ. 19 (1990) 305-321.
Mathematical Social Sciences 30 (1995) 221-234
Best replies and adaptive learning J o s h u a S. Gans* Department of Economics, Stanford University, Stanford, CA 94305-6072, USA Received July 1994: revised February 1995
Abstract
This paper discusses the relationships between learning processes based on the iterated elimination of strictly dominated strategies and their myopic and more naive counterparts. The concept of a monotone game, of which games with strategic complementarities are a subclass, is introduced. Then it is shown that convergence under best reply dynamics and dominance solvability are equivalent for all two-player (and some many-player) games in this class. Keywords: Monotone games; Strategic complementarities; Best reply dynamics; Adaptive
learning; Dominance solvability; Random matching
1. Introduction
Perhaps the simplest and most naive dynamic adjustment process is the best reply dynamic. 1 Under this dynamic, in disequilibrium, agents adjust their strategies by observing the immediate past state of play and choosing strategies as if that state will persist into the future. No account is taken of the history of strategies, their direction of movement, their signalling of another player's type, or the possibility that better strategies off the best reply path are never reached. All of these considerations might provide a better signal of the future choices of other players. None the less, despite its implied myopia, the simplicity of best reply dynamics is a virtue for the applied economist, in that it is quite easy to * Corresponding address: School of Economics, The University of New South Wales, Sydney, New South Wales, 2052, Australia ~These are sometimes referred to as best response dynamics, Marshallian dynamics, Cournot dynamics, or Cournot tatonnement. 0165-4896/95/$09.50
© 1995 - Elsevier Science B.V. All rights reserved
SSDI 0165-4896(95)00792-X
222
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
determine, in many games, the outcomes that result from such adjustment behaviour. Best reply dynamics have another quality. The paths generated under best reply dynamics are a subset of what has been termed 'adaptive learning'. As Milgrom and Roberts (1991) have shown, many learning and dynamic process- naive or sophisticated-have the quality that play eventually lies within the serially (strictly) undominated set of strategies. Such dynamic processes are defined to be consistent with adaptive learning. 2 The definition of adaptive learning is associated with strict dominance. Therefore, in what follows, when I refer to dominant or undominated strategies, or dominance solvability, it will be to the strict version of these concepts) Milgrom and Roberts (1991) show that the significant feature of dynamic processes consistent with adaptive learning is that if such processes converge, they will converge to a Nash equilibrium. As a corollary, if a game is dominance solvable, then all adaptive learning processes converge to the unique serially undominated profile. Therefore, if a n y normal form game is found to be dominance solvable, then the Milgrom-Roberts theorem implies that best reply dynamics will also converge to their unique equilibrium point. Determining whether behaviour under best reply dynamics converges is easier than showing that a given game is dominance solvable. Therefore, it is the task of this paper to search for classes of normal form games with the converse implication: that if best reply dynamics converge, then so will all other processes consistent with adaptive learning. This would establish an equivalence between convergence under best reply dynamics and dominance solvability.4 In these games then, one need only check that play is convergent under best reply dynamics to conclude that even much more sophisticated dynamic processes will converge to that same outcome.
2. Notation and formulation
There are IN[ players from a set, N, where each player is indexed by n. A pure strategy, x n =- ( x l n , . . . , x i , , ) , for an individual player comes from the set An, with
2 See also the related results of Gul (1991), which look at rationalizable as opposed to undominated strategy sets. 3 Marx (1994) has derived results based on learning concepts related to weak dominance solvability. Not surprisingly, her results do not offer the strong theorems that arise from adaptive learning as considered by Milgrom and Roberts (1991). 4 Note, however, that convergence under best reply dynamics is not guaranteed in the games to follow. It is well known that best reply dynamics do not always converge- see Milgrom and Roberts (1991) for a review of these results. Here it will be shown that when best reply dynamics converge then these games must be dominance solvable.
J.S. Gans / Mathematical Social Sciences 30 (199.5) 221-234
223
dimension I n. Let each X, be a compact lattice 5 and define X =-- Xn@NX n. The payoffs of players are given by the function, %(xn, x_n), where x , denotes the strategies of all other players. It is assumed that 7r, is upper semi-continuous in % for x ,, fixed, and continuous in x , , for fixed x n.6 A pure strategy, x, EX~, for player n is said to be strictly dominated by another pure strategy, 2,, if ( V x _ , ) % ( x , , x , , ) < r r , ( i , , x ~). Given a product set, A', of strategy profiles, the set of n's undominated responses to ~" is defined by
u n ( 2 ) = {xn
x,, I (Vx'
xo)(3
I>
•
Let U ( X ) = ( U , ( X ) ; n E N) be the set of undominated responses for each player. Define X ° = X, the full set of strategy profiles. For ~-i> 1, define X" = U ( X ' - ~). A strategy, x,,, is serially undominated if x , , G U , ( X ' ) , W-; these are just the strategies that survive the iterative process of removing strategies that are strictly dominated by other pure strategies. Observe that U is a monotone operator. That is. if )( C_X', then U ( X ) C_U(X'). The dominance solution of the game is the set of strategies that remain after iterative application of this process and the game is pure dominance solvable if the dominance solution set for each player n, U~(X*), consists of a single point.
2.1. Adaptive learning Milgrom and Roberts (1991) show how the operator, U, can be used to define what is meant by adaptive learning. Definition 1. A sequence of strategies {xn(t)} is consistent with adaptive learning by player n if ( V f ) ( 3 T ) ( V t >I T)xn(t ) E Un({x(s ) ] i ~
224
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
dominance. Adjustment processes, ranging from simple processes like fictitious play to more sophisticated ones that have players using full information from past play and even experimentation, are all consistent with adaptive learning. Milgrom and Roberts (1991, theorem 3(b)) show that processes that converge to Nash equilibria are consistent with adaptive learning. Moreover, they prove that, if a game is dominance solvable, then every process consistent with adaptive learning will converge to that profile - the unique Nash equilibrium of the game.
3. Best reply dynamics and adaptive learning Having defined the concept of adaptive learning and noted some of its implications, I will now turn to consider in more detail its relationship with best reply dynamics. First, I will state a precise definition of the best reply dynamic. Then I will show how convergence under best reply dynamics and dominance solvability are equivalent in a class of games that I term 'monotone' games. In so doing, I will review how this class of games extends previous similar results by Moulin (1984), Milgrom and Roberts (1990, 1991) and Milgrom and Shannon (1994). The set of best responses for player n to some x_n is B . ( x n) = {x. E X n
xn) o(x., x_n) >t
X-n)}.
Suppose the game is repeated over a number of periods indexed by t. Under best reply dynamics, in each period, t, each player chooses a strategy that is a best reply to the strategies played by the other players in the previous period, t - 1, i.e. xn(t ) E B n ( X _ , ( t - 1 ) ) . Here, best reply dynamics are taken to be of the simultaneous kind where, in each period, players adjust their strategies at the same time] Thus, under these dynamics, players never use strategies that are strictly dominated by another strategy. A sequence of strategies is consistent with best reply dynamics if each strategy played is a best response to the immediate past strategy of the other players. Formally:
Definition 2. A sequence of strategies {x,(t)} is consistent with best reply dynamics by player n if (Vt)xn(t) E B , ( x _ , ( t - 1)). A sequence of strategy profiles {x(t)} is consistent with best reply dynamics if each {xn(t)} has the property. Convergence under best reply dynamics is taken here to mean that every sequence of strategies consistent with best reply dynamics convergences globally. 7 Moulin (1986) makes a distinction between sequential and simultaneous best reply dynamics. In the sequential version, adjustment takes place one player at a time. Theorem 1, below, will be shown to apply to both simultaneous and sequential best reply dynamics.
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
225
Moulin (1984) was the first to identify a class of games under which convergence of best reply dynamics and dominance solvability are equivalent. In 'nice' games, players are assumed to choose their strategies from a one-dimensional compact interval, X~, and receive a payoff determined by the continuous function, ~r~, over X that is strictly quasi-concave with respect to xn. These assumptions imply that the best response correspondence, Bn, is a single-valued function. Moulin then shows that for two-player nice games, global convergence under best reply dynamics implies that the game is dominance solvable. He also provides an example indicating that this result does not extend to the three-player case. As Moulin notes, this class of nice games (even without any additional condition) is very restrictive. Indeed, it precludes the discontinuities of the best response correspondence that accompany many games of interest to economists today, in particular those games with non-concave payoffs.
3.1. Monotone games Here, I introduce a class of games that is not, in general, nice but generalizes some of the properties of nice games. In particular, this class of games allows a multidimensional strategy space and non-concave payoffs. I term these games monotone games because each player has a monotone best response correspondence .8 Before defining this class of games, it is worth re-considering the properties of a subclass of such games, i.e. games with strategic complementarities. Formally, strategic complementarities arise when payoffs, 7rn(x,,, x , , ) , on some ordering of strategy spaces, satisfy certain single-crossing and quasi-supermodularity conditions. That is, given x,, ~>x'n, a payoff satisfies the single-crossing property in (x,,;x ~,,) if
~,,(x.,x'.)>~(>)~.(x'.,x'o)~r~(x,,,x
,,)>~( > ) ~ ( x ' . . x
~), Vx ~ >~x'n ,
and is quasi-supermodular in x n if, given x , , ,
7r,,(Xn,X_D >~(> )~rn(X. ^ ~.,X °) ~ Tr.(X~ v~,X .) ~>(>)Tr.(/., x _ . ) ,
Vx.~i..
Definition 3. A game, [" = (N, {X., 7r.}.~U), is a game with strategic complementarities if, for all n, ~'.(xn,x ,,) is quasi-supermodular in x. and satisfies the single-crossing property in (x. ; x_.). Marx (1993) provides a related but alternative definition of a monotone game based on concepts from lattice theory, convex analysis and non-smooth analysis. Her focus of analysis is quite different from that here, however.
226
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
These games have received wide attention of late through the work of Topkis (1979), Vires (1990), Milgrom and Roberts (1990) and Shannon (1992). For the purposes here, the significance of these games lies in their implications for the monotonicity of best response correspondences. Milgrom and Shannon (1994, theorem 4) have shown that best response correspondences in games with strategic complementarities are monotone non-decreasing in x ,. However, their conception of monotonicity is based in Veinott's strong set order to allow for best response sets that are not singletons? To see this order, let x,, v x~, denote the component-wise maximum of x n and x ' , and x,,/x x', the component-wise minimum, and let B,, =- B.,(x .; S,,) and B~, --- B.(x' .; S,,). Then Veinott's strong set order that says that B,, ~ B . if, for every x. E B,, and x. E B,,, x,, v x. E B'. and x,, ^ x'. ¢ B~,. A best response correspondence is monotone non-decreasing if B,, >/B ~, for all x,,/> x'. and all S. C_X.. Milgrom and Shannon (1994. theorem 4) show that, with fixed strategy sets, the quasi-supermodularity and single-crossing conditions in Definition 3 are both necessary and sufficient for best response correspondences to be monotone non-decreasing in x , , ) ° Hence. one can write an alternative definition of games with strategic complementarities. p
t
t
P
Definition 3'. A game, F = (N, {X., 7rn}.~N), is a game with strategic complementarities if, for all n. B . ( x .; X,,) is monotone non-decreasing in x .,. The emphasis here on the monotonicity of best response correspondences leads, naturally, to a conception of a wider class of games. Definition 4. A game, F = (n, {X n, 7r.}ncN), is a monotone game if, for all n, B, (x_,, ; X , ) is monotone in x ,,. This class of games is broader than the class of games with strategic complementarities because it allows for the case where B , ( x , ; X , ) is monotone nondecreasing in x , , for some subset of players, while being monotone nonincreasing for the rest. To emphasize, games with strategic complementarities allow B , ( x _ , ; X n ) to be monotone non-decreasing in x_, for all players or m o n o t o n e non-increasing in x ~ for all players. On the other hand, one can imagine games where the best response correspondences are non-decreasing for some players and non-increasing for all other players - a mixed monotone game.
"This is in contrast to Moulin's (1984) nice games that are formulated to ensure that best response sets are singletons. ~°Athey (1995) extends the Milgrom and Shannon (1994) monotonicity characterizations to conditions of probability density functions that are part of expected payoff functions.
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
227
A game is mixed monotone if it is monotone and not a game with strategic complementarities. 11 None the less, by the necessity part of theorem 4 of Milgrom and Shannon (1994), all monotone games require that payoff functions are quasi-supermodular in x,, and satisfy the single-crossing property in (x,,, x ,,) or in (x,,. -x ,,). 3.2. E x a m p l e : Innovation and imitation 1~_
While examples of games with strategic complementarities abound in the literature, mixed monotone games are themselves potentially important. Consider a situation in which an established innovator in an industry decides the level at which to sink costs into developing a new product. A high level of product development has the advantage of raising revenue for the firm but, of course, at greater cost. The product development strategy of the innovator, L, is denoted by a real number x L ~ [XL,)~L]. Thus, the payoff to the innovator is, 7rL(X ) = R L ( X L , X v ) - - C L ( X L ) . Here XF~[XF,)~F] denotes the product adopted by an imitator, F. The greater this is, the less is the marginal revenue to the innovator from raising product quality, i.e. 02RL
--<~0.
Ox L Ox v
This assumption is essentially a smooth analogue of the single-crossing property (Topkis, 1979). Hence, by theorem 4 of Milgrom and Shannon (1994), this is a sufficient condition for the best response correspondence of the innovator to be monotone non-increasing in the strategy choice of imitator. The imitator's payoff is ~'F(X)= R F ( X F , X L ) - CF(XF,XL)with O2RF
--~>0
OXL OXv
O2CF
and
--~<0.
OXL OXF
That is, the higher the quality of produce the innovator develops, the higher the marginal net benefit to the imitator of increasing their product quality. This payoff function implies that the optimal strategy of the imitator is increasing in that of the innovator. Hence, the game is mixed monotone - no reordering of the strategy spaces will make it a game with strategic complementarities. Note that, unlike Moulin (1984), no assumptions are made on the concavity or otherwise of the respective payoff functions - indeed, it is unreasonable to assume
" Therefore, a mixed m o n o t o n e game cannot become a game with strategic complementarities by simply re-ordering the strategy space of players. ~2This example is of a similar flavour to that of Grossman and H e l p m a n (1991, ch. 12).
228
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
in the presence of sunk costs that payoff functions are concave. ~3 None the less, as will be shown by Theorem 1 below, convergence under best reply dynamics, and convergence under adaptive learning, are equivalent in such situationsJ 4
3.3. Equivalence result: Two-player games Milgrom and Roberts (1990, 1991) and Milgrom and Shannon (1994) show that, for games with strategic complementarities and any number of players, the coincidence of convergence under best reply dynamics and dominance solvability holds. ~5 They require neither the condition of quasi-concavity nor that the strategy space is of one dimension, as assumed by Moulin. The following theorem extends the results of Milgrom and Roberts (1990, 1991) and Milgrom and Shannon (1994) to all two-player monotone games. ~6 Recall, though, the fact that dominance solvability implies convergence under best reply dynamics was shown to hold for all normal form games (Milgrom and Roberts, 1991). Theorem 1. In two-player monotone games, if play converges globally under (simultaneous) best reply dynamics, then the game is dominance solvable. Proof. Note that Milgrom and Roberts (1991) and Milgrom and Shannon (1994) state the result for all games with strategic complementarities. Thus, all that remains is to prove the theorem for mixed monotone games. This is done in the following lemma.
13Note that one could also make the strategies multidimensional, with a few extra conditions to preserve monotonicity, again going beyond Moulin (1984). 14This would also be the case with many followers who are unaffected by each other's strategy choices, but where the innovator is affected by all their strategies in a monotone non-decreasing way. This extension is achieved using Corollary 1, below. ~5More to the point, they show that uniqueness of equilibrium implies dominance solvability for games with strategic complementarities. Then the coincidence follows since global convergence under best reply dynamics implies that the equilibrium is unique. ~6In the proof of his theorem Moulin (1984) implicitly assumes an extra condition. Consider any two ordered single-dimensional strategies x I > x', for player 1 in a two-player game. Moulin argues (p. 90) that, if player 2 uses a strategy x2 ~>x~, then x~ is dominated by x, for player 1. This is true if zq(xl,x2) crosses 7h(x~,x2) once from below over the X2 domain. In this case, ~'~(xl,x~)~> (>)Trl(x~, x ' ) implies that ,q(xl, x2)>~ (>)Trl(x~, x2). The conditions for a nice game, however, do not rule out more than one crossing. For instance, consider two non-connected intervals, [,~z,£2] and [~2, x2], subsets of X 2 with £2 < ]~. It is possible that, given x~ > x ~, Ir~(x, x2) ~>( >)~rl (x ;, x2) for all x2 E [~2, i2] LI [~2, ]2] and *r~(xl, x2) < ~r~(x~,x2) , otherwise. In this case, use of a greater strategy for player 2 does not imply that the higher strategy of player 1 is dominated. Observe, however, that the single-crossing property, which is an integral part of monotone games, rules out such difficulties.
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
229
Lemma 1. In two-player mixed monotone games, if play converges globally under (simultaneous) best reply dynamics, then the game is dominance solvable. Proof. Milgrom and Shannon (1994, p. 176) show that U~(X ~) contains smallest and largest elements, _xn and £n, for each player n, by the compactness of the strategy space and the continuity assumptions on payoffs. Without loss of generality, suppose that B~ is monotone non-decreasing and B 2 is monotone non-increasing. Now it can be shown that (i) _x~ E B~(_x2); (ii) £1 C B~(£2); (iii) 2~ E B2(x l ); and (iv) x 2 E B2(21). Suppose (i) does not hold, then there exists an x~ > x 2 such that _x1 ~ B~(x~) and there exists an x I > x I such that x I E Bl(X2), by the definition of U~(X ~) and U2(X ~) and the lemma (p. 176) of Milgrom and Shannon (1994). However, this implies that B~ is not monotone non-decreasinga contradiction. A similar argument shows (ii). Suppose (iii) does not hold, then there exists an x 1 >x~ such that £z C B2(x ~) and there exists an x, < x 2 such that x'~ ~ B2(xl). Again, we have a contradiction, since this implies that B~ is not monotone non-increasing. A similar argument shows (iv). Now observe that, using facts (i)-(iv), one can construct an infinite sequence composed of the following subsequence {(£1, £2), (£1,-x2), (_x~,_x2), (_x~,.~2)}, which is consistent with simultaneous best reply dynamics. However, for this to be convergent, x_, = £~, Vn, which in turn implies that U , ( X ~) must be a singleton for each n. Hence, the game is dominance solvable. Q.E.D. Remark 1. Note that this theorem applies even if best reply dynamics are of the sequential kind. Facts (i)-(iv) imply that an infinite sequence composed of the following subsequence {£2,£1,x2,x_~} is consistent with sequential best reply dynamics, iv Remark 2. We know, from Milgrom and Roberts (1991), that the above theorem can be proved for all monotone games that are games with strategic complementarities. Note then that, since all 2 x 2 games are monotone games, global convergence under best reply dynamics is a necessary and sufficient condition for a 2 x 2 game to be dominance solvable. Remark 3. Neither monotonicity nor the conclusions of Theorem 1 holds for general two-player games, however. Consider the following 2 × 3 non-monotone game in Fig. 1, under which best reply dynamics converge but the game is not dominance solvable. j7 Note that if players' payoffsare strictlyquasi-concavein their own strategies, then this is sufficient to ensure that the equilibrium of two-playermixed monotone games is unique and that if a best reply dynamic converges it will converge to it.
230
J.S. Gans /
MathematicalSocial Sciences 30 (1995) 221-234
II 1
2
4, 10
O, 0
3,0
3,1
O, 10
4,0
Fig. l
Remark 4. The theorem does not extend, in general, to games with more than two players. Suppose the population can be divided into two types of players, m E M C N and r n ' ~ MW. Suppose that for players with index m ' , Bm,(x_m,) is non-decreasing in x_m,, and that for players with index m, Bm(X_m) is nonincreasing in x _ m. It is then possible to show that (i) X m, EBm,(X__m,); (ii) £m' ~ Bm'(X m'); (iii) 2,,, E Bm(X_m); and (iv) X m ~ Bm(.~_m). However, in the many-player case, it is not possible to use the two-player argument to construct a non-convergent best reply sequence from these facts. I will consider, in Section 4, a particular many-player context that does allow an equivalence between dominance solvability and convergence under best reply dynamics.
4. Many-player monotone games Does the equivalence of best reply convergence and dominance solvability extend to any many-player context? It was noted earlier that this equivalence does not hold for general many-player mixed monotone games. Consider, however, a random matching game. That is, suppose that players are randomly matched to play a two-player mixed monotone game from two separate populations of equal size, m ~ M C N and m ' E M ~ N = = - M ' . In this section, assume that players' strategy spaces are uni-dimensional and are represented by intervals of the real line, X m and Am,, respectively. Under best reply dynamics, each player uses a pure strategy that is a best response to the frequency distribution of pure strategies played by the population in the immediate past periods. Let fM(Xm;~p): X M X ~ - ~ [ 0 , 1] and fM,(Xm;O): X M, x O----~ [0, 1] be parameterized families of probability density function that give the proportion of members of populations M and M' playing strategies x m and x,,., respectively, in the immediate past period. ( ~ and O are totally ordered sets). The parameters q5 and 0 order the resulting frequency distributions from these functions. Imposing some ordering on the frequency distributions is necessary to define monotonicity of best response correspondences in this context. Krishna (1992)
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
231
and Kandori and Rob (1992) assume that these parameters order these distributions (partially) according to first-order stochastic dominance (FOSD). Since such orderings are arbitrary, I assume instead that frequency distributions of the state of play are ordered according to a generalized monotone likelihood ratio property (MLRP).
Definition 5. Let S(~b) = {xm E X,, [ fM(Xm'~~) > 0}. Then fM(Xm; ~) satisfies the generalized monotone likelihood ratio property (MLRP) if given any xm E S(4~) and x'm E S(4~'), (i) min[xm, Xm, ] E S(4~) and max[x,,, x,,,] E S(~b'); and (ii) for all ~)<~)', fM(Xm;t~)/fM(Xm;(~') is monotone non-increasing in x m for all x , , ~
s(4,)
n
s(4,').
An analogous definition can be written for fM'(')" This parameterization is a generalization of the more familiar monotone likelihood ratio property that excludes condition (i) but does not allow for densities that are zero at p o i n t s - a feature that is essential for game theoretic applications. However, while it is a less complete ordering than FOSD, it allows a consideration of monotonicity in a wider class of games. That is, while a cardinal condition, such as increasing first differences, is required on payoffs when the order on frequency distributions is FOSD, the generalized M L R P derives a monotone best response correspondence for original structures on payoffs, such as the single-crossing property) 8 Ordering frequency distributions according to the generalized MLRP is similar, in two-player games, to ordering mixed strategies or players' beliefs in the same manner. Therefore, it becomes possible to prove the following theorem. Theorem 2. Take any player n in a two-player game (including random matching games). Let f(x_,; O): X , x 6)--*[0, 1] be the density function representing n's beliefs regarding the strategy choices of the players in the other population, where 0 parameterizes f according to the generalized MLRP. If 7rn(xn, x ~) has the singlecrossing property in (xn, x_,) ((x~, -x_,)), then B,(O ), is monotone non-decreasing (non-increasing) in O.
Proof. The proof relies on the following lemma, which is an extension of theorem 11.1 of Karlin (1968), to allow for functions that can take on zero values.
18 Athey (1995) addresses this point in considerably more detail by identifying the close relationship between stochastic dominance and supermodularity in stochastic contexts. She also shows that the generalized MLRP is equivalent to Ormiston and Schlee's (1993) definition of MLR dominance. Finally, Athey shows that if one imposed a cardinal complementarity condition (e.g. supermodularity) on payoff functions, then one could use FOSD to order the frequency distributions and generate monotone best response correspondences, as in Theorem 2 below.
232
J,S. Gans / Mathematical Social Sciences 30 (1995) 221-234
Lemma 2. Consider two functions g : Z x Y---~ ~ and h : Z x Y---~ ~ where Z, Y C_!~. Assume that h(z, y) >i 0 satisfies the generalized MLRP. Assume further than g satisfies the following: (a) For each z, g(z, y) changes sign at most once, and from negative to positive values, as y traverses Y; (b) For each y, g(z, y) is non-decreasing in z; and (c) k(z) = S g(z, y)h(z, y) dy exists and defines a continuous function of z. Then k(z) changes sign at most once, and from negative to positive values. Proof. Without loss of generality, assume that Z is an interval. It needs to be shown that k(zo)= 0 implies k(z)/>0, Vz > z o. Suppose, then, that k(zo)= 0 Further, there exists Y0 such that g(Zo, y) ~>O, Vy > Yo and g(z o, y) <~O, Vy < Yo. (The assumption that k(zo) = 0 and (a) ensure that Yo exists). Now, observe that
h(z o, yo)k(z) = h(z o, yo)k(z) - h(z, yo)k(zo) = f h(zo, yo)h(z, y)(g(z, y) - g(Zo, y)) dy + f (h(z0, yo)h(z, y) - h(zo, y)h(z, yo))g(Zo, y) dy. If z > Zo, then the second integral is non-negative by the choice of Yo and the generalized M L R P assumption on h(z, y). The first integral is non-negative because of assumption (b). Q.E.D. Now, it can be seen that Ex °[Tr(x,,x_n)[0] = .~x , ~r(Xn,X-n)f(x-n; O)dx_ n has the single-crossing property in (x n; 0). This is done by applying Lemma 2 with g(x, y) = 7rn(x'n, x_n) - 7r,(xn, x_n) and h = f , with y = x_ n and z = 0. In this case, k(O) = Ex ,[Tr(x'n, x_n) ]0 ] - Ex ,[Tr(Xn, x , ) ] 0 ] changes sign once from negative to positive, which is the single-crossing property. This satisfies the conditions of theorem 3 of Milgrom and Shannon (1994), completing the proof. Q.E.D. Theorem 2 extends the previous results of Krishna (1992) and Kandori and Rob (1992) to games with ordinal payoffs. They studied games where strategic complementarities arose as a result of increasing first differences, rather than the weaker condition of the single-crossing property. The above lemma extends their results to an even wider class of games, since it only requires the single-crossing property. ]9 Therefore, the above theorem shows that, if players in population M have payoffs with the single-crossing property in (xm;xm,) o r (x m ; - x m,) and players in 19Theorem 2 is related to results in Athey et al. (1994) and Athey (1995). The former allows 7r to have the weak single-crossing property, but uses the strict monotone likelihood ratio for density functions that are strictly positive everywhere. Athey (1995) allows for the possibilitythat 7r itself might depend on 0.
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
233
p o p u l a t i o n M ' have payoffs with the single-crossing p r o p e r t y in (Xm,; Xm) or (Xm,;--Xm), then the resulting game is m o n o t o n e . The following can then be p r o v e d easily, using T h e o r e m 1. C o r o l l a r y 1. In many-player random matching, monotone games, if play con-
verges globally under (simultaneous) best reply' dynamics, then the game is dominance solvable. Proof. This comes f r o m using the same m e t h o d of p r o o f as T h e o r e m 1 and noting its effective equivalence to two-player contexts. That is, the p a r a m e t e r s 0 and ~b act in a similar m a n n e r to the pure strategy choices of the o t h e r player in t w o - p l a y e r games. H o w e v e r , it should be n o t e d that, unlike T h e o r e m 1, it does not extend to the case of sequential best reply dynamics.
Acknowledgements I wish to t h a n k Paul Milgrom for suggesting this topic to me. Also, thanks go to C a t h e r i n e de F o n t e n a y , Scott Stern, seminar participants at Stanford University, and, especially, Susan A t h e y for helpful discussions. All errors remain by own.
References S. Athey, Monotone comparative statics in stochastic optimization problems, mimeo, Stanford University (1995). S. Athey, J.S. Gans, S. Schaefer and S. Stern, The allocation of decisions in organizations, Discussion Paper, No. 94/29, School of Economics, University of New South Wales (1994). G.M. Grossman and E. Helpman, Innovation and Growth in the Global Economy (MIT Press, Cambridge, MA, 1991). F. Gul, Rationality and coherent theories of strategic behavior, mimeo, Stanford University (1991). M. Kandori and R. Rob, Evolution of equilibria in the long run: A general theory and applications, mimeo, Princeton University (1992). S. Karlin, Total Positivity, Vol. I (Stanford University Press, Stanford, 1968). V. Krishna, Learning in games with strategic complementarities, Working Paper, no. 92-073, Harvard Business School (1992). L.M. Marx, Monotonicity of solution sets for parameterized optimization problems, Discussion Paper, no. 1067, Northwestern University (1993). L.M. Marx, Learning to play weakly undominated strategies, mimeo, Northwestern University (1994). P. Milgrom and J. Roberts, Rationalizability, learning, and equilibrium in games with strategic complementarities, Econometrica 58, no. 6 (1990) 1255-1277. P. Milgrom and J. Roberts, Adaptive and sophisticated learning in normal form games, Games Econ. Behav. 3 (1991) 82-100. P. Milgrom and C. Shannon, Monotone comparative statics, Econometrica 62, 1 (1994) 157-180. H. Moulin, Dominance solvability and Cournot stability, Math. Soc. Sci. 7 (1984) 83-102.
234
J.S. Gans / Mathematical Social Sciences 30 (1995) 221-234
H. Moulin, Game Theory for the Social Sciences, 2nd ed., (New York University Press, New York, 1986). M.E. Ormiston and E.E. Schlee, Comparative statics under uncertainty for a class of economic agents, J. Econ. Theory 61 (1993) 412-422. C. Shannon, Complementarities, comparative statics and nonconvexities in market economies, unpublished Ph.D. Thesis, Stanford University (1992). D. Topkis, Equilibrium points in non-zero sum n-person submodular games, SlAM J. Control Optimiz. 17 (1979) 773-787. X. Vives, Nash equilibrium with strategic complementarities, J. Math. Econ. 19 (1990) 305-321.