- Email: [email protected]
The multi-player version of minimax displays game-tree pathology
Artificial Intelligence 64 (1993) 323-336 Elsevier
323
ARTINT 1107
Research Note
The multi-player version of minimax displays game-tree pathology * David Mutchler Department of Computer Science, University of Tennessee, Knoxville, TN 37996-1301, USA Received November 1989 Revised July 1993
Abstract Mutehler, D., The multi-player version of minimax displays game-tree pathology (Research Note), Artificial Intelligence 64 (1993) 323-336. It is widely believed that by searching deeper in the game tree, the decision-maker is more likely to make a better decision. Dana Nau and others have discovered pathology theorems that show the opposite: searching deeper in the game tree causes the quality of the ultimate decision to become worse, not better. The models for these theorems assume that the search procedure is minimax and the games are two-player zero-sum. This report extends Nau's pathology theorem to multi-player game trees searched with maxn, the multi-player version of minimax. Thus two-player zero-sum game trees and multi-player game trees are shown to have an important feature in common.
1. Introduction Games played by more than two players have important characteristics that mark them as altogether different from two-player zero-sum games. Correspondence to: D. Mutchler, Department of Computer Science, University of Tennessee, Knoxville, TN 37996-1301, USA. Telephone: (615) 974-5067. E-mail: [email protected]. *This research was supported by the NSF under grant IRI g9-1072g, and by AFOSR under grant 90-0135. An abridged version of this paper appeared in: Proceedings Sixth International Symposium on Methodologies for Intelligent Systems, Charlotte, NC (1991).
0004-3702/93/$ 06.00 t~ 1993 - - Elsevier Science Publishers B.V. All rights reserved
324
D. Mutchler
Consider, for instance, the concept of an equilibrium point. Such a point is a collection of strategies, one per player, such that no player can improve her score by a unilateral change in her own strategy. For two-player zero-sum games, all equilibrium points yield the same outcome for the game. For multi-player games, different equilibrium points may yield different payoffs to the players. As such, the evident concept of a "solution" for a two-player zero-sum game does not extend as clearly to multi-player games. This report shows that despite their differences, two-player games and multi-player games do share one interesting property, namely, the existence of pathological game trees for which deeper search degrades the quality of the decision made. The standard search procedure for twoplayer zero-sum games is minimax. Its pathology has been demonstrated by several researchers [7,18,20,22-24]; see [2] for an excellent summary. The contribution of this report is to extend that pathology to multi-player games searched with m a x n [ 12,13,16 ], the multi-player version of minimax. 1 Section 2 gives a formal statement of minimax and m a x n . Section 3 reviews the pathology theorems that have appeared for two-player zerosum games. Section 4 contains a new pathology theorem that applies to multi-player games.
2. Minimax and
maxn
This section reviews the game theory used in this report. The books by Shubik [27] and by Luce and Raiffa [15] are good sources for a more complete account. G a m e theory treats many kinds of games. This report restrictsits attention to perfect information games, thus prohibiting, for example, simultaneous moves. The players are assumed to alternate turns in some fixed order. The computer science literaturehas been concerned largely with two-player zero-sum games; here that restriction is removed. The treatment of multiplayer games in this report assumes a noncooperative solution theory, thus prohibiting side payments, binding agreements, and threats, for example. Although a version of minimax has been developed for games with chance nodes [6,29], such nodes are not considered in this report. The trees considered are assumed to be finite.For a moment, let us look only at two-player zero-sum games. In that case, each leaf of the game tree has an associated payoff to the player who moves to that leaf. The payoff to the other player is the negative of the payoff at the leaf. Minimax, expressed l T h r o u g h o u t we use the typographically simpler n o t a t i o n m a x n used in [ 12 ] instead of the earlier n o t a t i o n m a x n used in [ 16].
Multi-player game-tree pathology
325
here by its negamax formula [11], provides one strategy for playing such a game.
mm (g) =
the payoff at leaf g, if node g is a leaf, - max{ram (g') I g' is a child of g}, otherwise.
A player follows the minimax strategy by selecting, each time the player is to move, the node g whose minimax value (from the above formula) is largest. The minimax strategy has two virtues [27, Section 8.2.1 ]. First, it provides the optimum level of s a f e t y . The ultimate payoff to the player who is using the minimax strategy is guaranteed to be no worse than the minimax value of the node to which that player is moving. This guarantee holds no matter what strategy the opponent is using. The second virtue of the minimax strategy is that it is in e q u i l i b r i u m : even if you somehow know that your opponent is using minimax, you cannot do any better than by using minimax yourself. It is the opposite natures of these virtues---the first guaranteeing a minimal value against an opponent about whom you know nothing, the second limiting you to a maximal value against an opponent about whom you know everything--that provide the universal acceptance of the "rationality" of the minimax strategy. Turn now to multi-player games. (All that is said in this report about multi-player games applies equally well to two-player games that are not zero-sum. Hereafter, include such games within the class of "multi-player games". ) Each leaf now has an associated v e c t o r of payoffs, with the j t h component of the vector being the payoff to the j t h player. Kuhn has shown that perfect information multi-player games, like two-player zero-sum games, possess solution strategies that are in equilibrium [ 13 ]. That is, there exists a vector S of strategies, one strategy per player, such that the following can be said of each player: even if the distinguished player somehow knew that the other players would abide by their respective strategies announced in the vector S, the distinguished player could not achieve a higher payoff by using a strategy other than the strategy in S associated with the distinguished player. Here is another way to state this same property: no player can improve his or her payoff by a unilateral change in strategy. How does one compute an equilibrium point, that is, how does one find a set of strategies in equilibrium? The proof that Kuhn gave in 1953 to show the existence of an equilibrium point for perfect information games is constructive; the induction he used yields an algorithm. (See [ 13, Corollary 1, p. 209 ].) However, the algorithm is only a by-product of the proof and is not stated explicitly. The same algorithm was independently introduced to the AI community and termed m a x n by Carol Luckhardt and Keki Irani
326
D. Mutchler
Who moves:
(9,6,5)
A
(3,7,4)
(9,6,5)
B
(7,3,4)
(3,7,4)
(9,6,5)
(5,3,2)
C
(9,6,1) A
>
(7,3,4) (3,7,4) (8,2,3) (9,6,5)
A A A A A
9 6 1
4 7 2
2 8 0
7 3 4
3 7 4
2 8 9
5 0 6
8 2 3
7 5 8
9 6 5
(6,3,2)
(6,7,0) (5,3,2)
AAA
0 8 1
6 3 2
2 3 1
6 7 0
5 3 2
3 2 1
• Fig. 1. The maxn backup rule. [16]; they also discussed pruning algorithms for m a x n . (See also [12].) A player follows the m a x n strategy by moving to the node whose m a x n vector is recursively selected as follows. • If node g is a leaf, m a x n (g) = the payoff vector at leaf g. • If node g is not a leaf, let V denote the m a x n vectors of g's children, i.e., V = { m a x n ( g ' ) [ g ' is a child of g}. Let i denote the player whose turn it is to move from node g. Then m a x n ( g ) = the vector in V whose ith component is maximal. Figure 1 shows the m a x n vectors for the nodes in a sample tree. The elements of the vectors refer to players A, B, and C, respectively. (Leaf values are displayed as column vectors.) Player A moves first, then B, then C, then A again. Unfortunately, an equilibrium point in a multi-player game lacks the guarantee of safety it provides in a two-player zero-sum game. In a multi-player game, different equilibrium points may provide different payoff vectors. Nonetheless, m a x n is a reasonable strategy for multi-player games. For perfeet information games, it is perhaps the strategy that is most "rational". The fact that m a x n is analogous to minimax enhances its appeal. The above description of the m a x n vector is ambiguous: what vector is selected if two or more vectors share the same maximal value at their /th component? Luckhardt and Irani showed that m a x n with any tie-breaking
Multi-player game-treepathology
327
rule yields an equilibrium point [ 16 ]. The results in this report are oblivious to the tie-breaking rule used, so the ambiguity will be left as it is. In practice, it is computationally infeasible to compute the minimax value or maxn vector for game trees of even modest depth, even with the help of pruning algorithms like alpha-beta [ 11,28] that compute the same value as minimax, but faster. Instead, one uses a heuristic evaluator e (g) to estimate the minimax value of node g in a two-player zero-sum game, or a heuristic evaluator e(g) to estimate the maxn vector at node g in a multi-player game. A typical heuristic evaluator for chess, for instance, assigns 8 points to queens, 5 points to rooks, 3 points to bishops, etc; the evaluation of a position is the total point-value of one player's pieces subtracted from the total point-value of the other player's pieces. To select a node to which to move from a set G of possible choices, a player could simply apply the heuristic evaluator to each of the nodes in G; the player then moves to whichever node g has the highest value e(g) (for two-player zero-sum games) or whose vector e (g) (for multi-player games) has the maximal value at that player's component. Another (more common) approach is to apply the heuristic evaluator to the leaves of depth d subtrees below the nodes in G. The player then computes the minimax value or maxn vector of the nodes in G by applying the minimax or maxn backup rule as if the nodes evaluated heuristically were actual payoffs. The common belief is that this second approach--use the backed-up values/vectors instead of the direct heuristic evaluations of the nodes in G--is superior. The pathology theorems show that it need not be.
3. Pathology theorems for two-player zero-sum games In two-player zero-sum games, if the heuristic evaluator is completely accurate, then minimax using the heuristic evaluator returns the same number as minimax using the actual payoffs. In practice, the heuristic evaluator will sometimes err, thereby introducing errors into the backed-up minimax value as well. The question is now: is the minimax value computed from heuristic evaluations a good estimate of the actual minimax value? Judea Pearl comments [25, p. 332]: In fact it is surprising that the minimax method works at all . . . . Minimaxing these estimates as if they were true payoffs amounts to committing one of the deadly sins of statistics: computing a function of the estimates instead of an estimate of the function. In the same way that the average of a product generally is not equal to the product of the averages, so also the estimate of the minimax differs from the minimax of the estimates.
328
D. Mutchler
Pearl continues by explaining that one argument given to justify minimaxing the heuristic evaluations is a filtering argument: the backed-up value reflects the information contained in the entire frontier of the search. The pathology theorems show that this filtering argument is fallacious. The literature contains three classes of pathology theorems for two-player zero-sum games. All the theorems model the heuristic evaluator applied to a node as a random variable, capable of error. The conclusion of each theorem has the same consequence: it is increasingly likely as the depth of the search increases that the children of the root node will all have the same minimax value. Hence the ability of minimax to distinguish good moves from bad is lost; minimax under deep search is reduced to a decision-maker that selects moves at random. • Class 1. This class contains the theorem from Dana Nau's thesis [20] and its extension in [22]. Beal independently developed a somewhat weaker theorem [7] that falls in this class. The three important assumptions of these theorems are: (a) the probability of the heuristic evaluator being totally wrong is bounded away from zero; (b) the heuristic evaluations on the different nodes possess some degree of independence; and (c) the branching factor is sufficiently large. (The more accurate the evaluator, the larger the branching factor required.) The conclusion is that the minimax value under deep search is almost certain to return the highest possible value for the player who moved last in the search. • Class 2. Pearl has a different pathology theorem [24]. By assuming that the actual outcomes of the game are either a WIN (with probability p) or a LOSS (with probability 1 - p ) , Pearl does not need to assume a sufficiently large branching factor. In fact, Pearl's theorem assumes a binary tree and a two-valued heuristic evaluator, but he indicates how these assumptions could be lifted. • Class 3. A third pathology theorem was discovered independently by Nau and Pearl [18,23]. Here the heuristic evaluators are assumed to be identically distributed as well as independent. The conclusion is that minimax under deep search is almost certain to return a fixed value determined by the distribution of the evaluators. The assumption of identical distributions is clearly not reasonable; as such, this theorem is more a statement about game trees with random outcomes than a pathology theorem.
As a concrete example, here is a simplified version of Nau's pathology theorem in [22]. Let G be a complete, infinitely deep, b-ary game tree. For each node g of G, model the heuristic evaluation e ( g ) as a discrete random variable with range - h to h. Assume these random variables are all independent (although not necessarily identically distributed). Let tie(d, g )
Multi-playergame-treepathology
329
denote the event that all children of g receive the same m i n i m a x value when a depth d search is done. The pathology theorem is: if b is sufficiently large, 2 then the probability of tie(d, g) converges to 1 as the depth d gets large. Thus, m i n i m a x under deep search reduces to a decision-maker that selects moves at random.
4. A p a t h o l o g y t h e o r e m for multi-player games
This section presents a pathology theorem for multi-player games. The theorem fallsinto the firstof the three classes of pathology theorems listed in the previous section. Its assumptions arc very much like those of Nau's two-player pathology theorem in his thesis [20] and its extension [22]. The proof strategy has two steps. First, I will extend Nau's theorem to game trees in which the branching factor for one player need not bc the same as the branching factor for the other player. The technique for doing so will follow Nau's approach quite closely. The result will b c a lemma stating that, for cvcn search depths, the minimax value is almost certain to bc the highest value possible,under appropriate assumptions. Second, I will reduce a multi-playergame to a two-player game in which all the players but one form a coalition against the remaining player. From the lemma in the firststep, the minimax value of this two-player game is almost certain to bc the highest value possible, for cvcn search depths. The multi-player result will then follow by using the fact that the m a x n value for the distinguished player is at leastas large as the value returned by minimax in the two-player game. Let b and B bc real numbers with b > I and B > I. Let x bc any real number between 0 and I. Define
si(b,B,x) =
i'
i-1 ( B , b , x ) b, -- [ 1 - S i _ l ( B , b , x ) ]
i f / = O, if i > 0 is even,
b, i f i > O i s o d d .
Note the reversal of the b and B arguments in the recursion. W h e n b = B, this function is the same as the function si (b, x ) that N a u used. He displayed a certain function w of b such that si(b,x) converges to either O, w (b), or 1 as i grows large, depending on whether x is smaller, equal to, or greater 2Let d denote the greatest lower bound over nodes g in G, of Pr [e (g) = h ]. Let w (b) denote the unique solution in the open interval (0, 1) to the equation x = (1 - (1 -x)b) b. (Note that w(b) is between 0 and I and is a decreasing function ofb.) Then "b is sufficiently large" means that d > w (b). Thus, as long as all evaluations e (g) have a bounded-away-from-zero probability of returning their highest possible value h, then b can be made sufficiently large.
330
D. Mutchler
than w (b) [18]. The following lemma shows that the behavior in the limit o f s i ( b , B , x ) is similar. Lemma 4.1. There exists a function w (b, B) o f b and B such that the limit o f the sequence (s:i} (i.e., {si } on its even-numbered terms) satisfies
lira s 2 i ( b , B , x ) =
1, w(b,B),
if w ( b , B ) < x <~ 1,
O,
ifO <~ x < w ( b , B ) .
if x = w ( b , B ) ,
Proof. See [ 17 ] for a proof, which involves close examination of the first and second derivatives of a certain function of x that corresponds to {s2i}. [] The lemma that relates function s to minimax search is the following version of [22, L e m m a A.1 ]. Lemma 4.2. Consider a game tree T in which two players, each with perfect information, alternate moves. Suppose that one player has b > 1 choices at each turn and the other has B > 1 choices. For each node x in T, model the heuristic evaluation e (x) as a discrete random variable with range - h to h. Assume these random variables are all independent (although not necessarily identically distributed). Let c denote the greatest lower bound over nodes x in T, o f P r [ e ( x ) = h]. Then for every node g at which there are b choices and for every node G at which there are B choices, if the subtrees rooted at g and G are complete to at least depth d, we have the following bounds on the probability that the depth d minimax search returns an extreme value: P r [ m m ( d , g ) = ( - 1 ) d h ] >i sd(b,B,c), P r [ m m ( d , G ) = ( - 1 ) d h ] >f Sd(B,b,c).
Proof. The proof is an easy modification of the proof of the original lemma in [22], which is itself an easy induction on d. Two modifications are required. First, the induction hypothesis here is a pair of statements, one for nodes with branching factor b and the other for nodes with branching factor B. Second, the induction step proceeds from a g node (branching factor b) to a G node (branching factor B), or vice versa. This accounts for the reversal of the b and B arguments in the recursive definition of si (b, B, x). In short, the definition of si (b, B,x) is rigged precisely to permit the truth of this lemma. []
Multi-player game-treepathology
331
Lemmas 4.1 and 4.2 permit a two-player pathology theorem for games with a dual branching factor; see [ 17 ]. Our interest, however, is in multi-player games and m a x n . Here is the pathology theorem for that case.
Theorem 4.3 (The multi-playerversion of minimax displayspathology). Let player A denote the player to move from the root o f a game tree. Let e (g) be a vector-valued heuristic evaluator. Let e~ (g) denote the component o f e (g ) that pertains to player A. Suppose • Shape of the game tree: - Several players in a multi-player, perfect information game alternate turns. - The branching factor for player A is b > 1. - The combined branching factor for the other players is B > I. - The tree is complete and infinitely deep. • Player A's heuristic evaluator: - For each node g to which player A might move, the heuristic evaluation eA(g) is modeled as a discrete random variable with range - h to h. - These random variables are all independent (although not necessarily identically distributed). - c > w (B, b) where c denotes the greatest lower bound over nodes g to which player A might move, ofPr[eA (g) = h]. • Other players' heuristic evaluator: - The other player's components o f e ( g ) can be any functions, either random (with any distribution) or deterministic. • Search depth: - Player A computes the maxn vectors o f the children o f the root and selects whichever o f these vectors maximizes ,4 'S component. - The terminal nodes o f these maxn searches (i.e., the nodes to which A applies e ( g ) ) are nodes to which A might move. Then it is almost certain that for deep searches, player A will reduce to a decision-maker who selects moves at random. That is, if V denotes { m a x n ( g ) } such that g is a child o f the root and the maxn search is to depth d, then the probability converges to 1 that, as the depth d grows large, all o f the vectors in V will have h as their value for player .4"s component. Proof. Convert the given multi-player game into a two-player zero-sum game as follows. The structure of the game tree remains the same. The protagonists in the two-player game are, first, player A, and second, a coalition consisting of all the other players from the multi-player game. On nodes g to which player A might move, the heuristic evaluator is the function e.4 taken from the multi-player game. On other nodes in the two-player game, the heuristic
D. Mutchler
332
evaluation is identically h. Player A applies minimax with an even-depth search to the children of the root of the two-player game tree. Figure 2 shows a sample multi-player tree searched with maxn and the corresponding twoplayer tree searched with minimax. Let GI through Gb denote the b children of the root in the constructed two-player game. The probability that all b of these nodes have depth d minimax value h is, using the fact that the eA'S are independent, b
I-[ Pr[mm(d, Gi) = h]. i=1
Since each child G~ of the root of the constructed two-player game has branching factor B, and since the depth d of each minimax search is even, each of the terms in the above product is at least Sd(B,b,c), from I-emma 4.2. Conclude that the probability that all b of the minimax searches return h is at least Sd (B, b, c) b. Now apply Lemma 4.1, using the assumption that c > w (B, b) and the assumption that the search is to an even depth in the two-player game. Since SEd(B, b, c) b converges to 1 as d grows large, so does the probability that all b of the minimax searches return h. The proof is concluded by comparing A's component of the maxn(g) vector in the multi-player game to ram(g) in the constructed two-player game. When it is player A's turn, maxn and minimax proceed analogously. Consider next the behavior when it is not player A's turn. Under maxn, the other players select a vector that is certainly no worse for player A than the vector selected by conspiring against player A, as they do in the constructed two-player game. Thus A's component of the maxn(g) vector in the multi-player game is at least as large as ram(g) in the constructed two-player game. As argued, it is almost certain that all b of the minimax searches return h, the largest value possible; hence it is almost certain that all b of the maxn searches also return h in player A's component. []
5. Discussion
This paper has presented a pathology theorem for multi-player games. Its assumptions are hardly more than that which is needed for two-player games. No assumption whatever is required of the backup functions for the players other than the one player for whom pathology is proved. Thus the theorem applies not only to maxn, but also to any other algorithm that agrees with maxn whenever it is player A's turn. The assumptions of the pathology theorem for multi-player games are essentially the same assumptions as Nau's pathology theorem for two-player
333
Multi-player game-tree pathology
Who moves: A
(3,7,4)
(9,6,5)
B
(7,3,4)
(3,7,4)
(9,6,5)
(5,3,2)
C
(9,6,1)
(7,3,4)
(3,7,4)
(8,2,3)
9 6
4 7
2 8
7 3
3 7
2 8
5 0
1
2
0
4
4
9
6
(9,6,5)
(6,3,2)
(6,7,0)
(5,3,2)
AAAAAAAA
A
8 2
7 5
9 6
0 8
6 3
2 3
6 7
5 3
3 2
3
8
5
1
2
1
0
2
1
m a x n using evaluator g
Who moves: A
//
Coafiffon
-9
5
3
/
-7
-3
-8
A
9
4
2
7
3
2
5
-9
-6
-6
-5
AAAAAAAA
8
7
9
0
6
2
6
m i n i m a x using evaluator eA
Fig. 2. A sample multi-player game and the constructed two-player game.
5
3
334
D. Mutchler
zero-sum games, with one addition: the search must end at nodes to which player A might move. That is, the heuristic evaluation function is always applied to nodes that are children of player A's decision points. This assumption is not unreasonable, since game-playing programs often stop at full-ply increments to sidestep any bias in the heuristic evaluator. The assumption is also a necessary one, in some sense. Without it, player A's component of the m a x n evaluations can be forced to converge to either extreme ( + h or - h ), depending on the details of the heuristic evaluator. For example, if the other players' components of the heuristic evaluator always yield the negative of player A's component, the multi-player game reduces to a two-player zero-sum game. Then player A's component of the m a x n evaluations at odd depths converges to - h , under appropriate assumptions. On the other hand, if {eA(g)} are independent of the other components of the heuristic evaluator, then player A's component of the r n a x n evaluations converges to + h for all search depths, odd or even. See [ 17] for details. The theorem applies when the heuristic evaluator has error probability larger than w (B, b). This function can be made arbitrarily small by appropriate choice of B and b. For instance, suppose there is a constant branching factor per player, so that B = b k for some fixed positive integer k. Since w ( b k , b ) ~ 0 as b ---, c~ (see [17] for a proof), the pathology theorem applies as long as the error in player A's component of the heuristic evaluator is bounded away from zero. However, the convergence of w ( B , b ) is somewhat slow, so very large branching factors may be required for the theorem to apply to accurate heuristic evaluators. For simplicity of presentation, we have assumed that the heuristic evaluations eA (g) are independent, although a weaker condition (p-dependence bounds, as given by Nau in [22] ) would suffice. See [17] for details. Despite the pathology theorems, minimax pathology has not been observed in games like chess. Several explanations for the lack of pathology have been proposed, supported by empirical and theoretical evidence [1,2,810,14,19,21,22,24,26]. It would be interesting to see how these explanations affect m a x n and to see whether m a x n does, in practice, exhibit pathology. Experience with m a x n using both hand-crafted and "generic" heuristics (via Abramson's expected-outcome model [3-5], for example) would be welcome. There are two lessons to be learned from the pathology theorem in this report. First, it provides theoretical evidence that searching deeper in the game tree when using m a x n may not be a good idea. Second, it shows that multi-player games and two-player zero-sum games share an important property, namely, game-tree pathology. This is surprising, since traditional game theory gives the two types of games two quite different treatments.
Multi-player game-tree pathology
335
Acknowledgement M u c h o f this w o r k was d o n e while the a u t h o r was in the C o m p u t e r Science a n d Systems Branch o f the N a v a l R e s e a r c h L a b o r a t o r y , Washington, DC. T h e a u t h o r is grateful to M a r y E b e d e i n a n d Bruce A b r a m s o n for their helpful c o m m e n t s .
References [I ] B. Abramson, An explanation of and cure for minimax pathology, in: L.N. Kanal and J.F. Lemmcr, eds., Uncertainty in ArtificialIntelligence,Machine Intelligenceand Pattern Recognition 4 (North-Holland, Amsterdam, 1986) 495-504. [2] B. Abramson, Control strategies for two-player games, ACM Comput. Sure. 21 (2) (1989) 137-161. [3] B. Abramson, An analysis of expected-outcome, J. Exper. Theor. Artif. Intell. 2 (I) (1990) 55-73. [4] B. Abramson, Expected-outcome: a general model of static evaluation, IEEE Trans. Pattern Anal. Mach. Intell. 12 (2) (1990) 182-193. [5] B. Abramson, On learning and testing evaluation functions, J. Exper. Theor. Artif. Intell. 2 (3) (1990) 241-251. [6] B.W. Ballard, The .-minimax search procedure for trees containing chance nodes, Artif. Intell. 21 (1-2) (1983) 327-350. [7] D.F. Beal, An analysis of minimax, in: M.R.B. Clarke, ed., Advances in Computer Chess 2 (Edinburgh University Press, Edinburgh, Scotland, 1980) 103-109. [8] D.F. Beal, Benefits of minimax search, in: M.R.B. Clarke, ed., Advances in Computer Chess 3 (Pergamon, Elmsford, NY, 1982) 17-24. [9] I. Bratko and M. Gains, Error analysis of the minimax principle, in: M.R.B. Clarke, ed., Advances in Computer Chess 3 (Pergamon, Elmsford, NY, 1982) 1-15. [10] H. Kaindi, Minimaxing--theory and practice, AI Mag. 9 (3) (1988) 69-76. [11] D.E. Knuth and R.W. Moore, An analysis of alpha-beta pruning, Artif. Intell. 6 (4) (1975) 293-326. [12] R.E. Koff, Multi-player alpha-beta pruning, Artif. Intell. 48 (1) (1991) 99-111. [ 13 ] H.W. Kuhn, Extensive games and the problem of information, in: H.W. Kuhn and A.W. Tucker, eds., Contributions to the Theory of Games, Vol. H (Princeton University Press, Princeton, NJ, 1953) 193-216. [14] L. Li and T.A. Marsland, On minimax game tree search pathology and node-value dependence, Tech. Report 90-24, Department of Computing Science, University of Alberta, Alta. (1990). [15] R.D. Luce and H. Raiffa, Games and Decisions (Wiley, New York, 1957). [16] C.A. Luckhardt and K.B. Irani, An algorithmic solution of n-person games, in: Proceedings AAAI-86, Philadelphia, PA ( 1986) 158-162. [17] D. Mutchler, The multi-player version of minimax displays game-tree pathology, Tech. Report CS-93-198, Department of Computer Science, University of Tennessee, Knoxville, TN (1993). [18] D.S. Nau, The Last Player Theorem, Artif. lntell. 18 (1) (1982) 53-65. [19] D.S. Nau, An investigation of the causes of pathology in games, Artif. Intell. 19 (3) (1982) 257-278. [20] D.S. Nau, Decision quality as a function of search depth on game trees, J. ACM 30 (4) (1983) 687-708. [21 ] D.S. Nau, On game graph structure and its influence on pathology, Int. J. Comput. Inf. ScL 12 (6) (1983) 367-383.
336
D, Mutchler
[22] D.S. Nau, Pathology on game trees revisited, and an alternative to minimaxing, Artif. Intell. 21 (1-2) (1983) 221-244. [23] J. Pearl, Asymptotic properties of minimax trees and game-searching procedures, Artif Intell. 14 (2) (1980) 113-138. [24] J. Pearl, On the nature of pathology in game searching, Artif. Intell. 20 (4) (1983) 427-453. [25] J. Pearl, Heuristics--Intelligent Search Strategies for Computer Problem Solving (Addison-Wesley, Reading, MA, 1984). [26] A. Scheucher and H. Kaindl, The reason for the benefits of minimax search, in: Proceedings IJCAI-89, Detroit, MI (1989) 322-327. [27] M. Shubik, Game Theory in the Social Sciences (MIT Press, Cambridge, MA, 1982). [28] J.R. Slagle and J.K. Dixon, Experiments with some programs that search game trees, J. ACM 16 (2) (1969) 189-207. [29] M. van Lent, A pruning algorithm for one player games with hidden information, M.S. Thesis, University of Tennessee, Knoxville, TN (1993).
323
ARTINT 1107
Research Note
The multi-player version of minimax displays game-tree pathology * David Mutchler Department of Computer Science, University of Tennessee, Knoxville, TN 37996-1301, USA Received November 1989 Revised July 1993
Abstract Mutehler, D., The multi-player version of minimax displays game-tree pathology (Research Note), Artificial Intelligence 64 (1993) 323-336. It is widely believed that by searching deeper in the game tree, the decision-maker is more likely to make a better decision. Dana Nau and others have discovered pathology theorems that show the opposite: searching deeper in the game tree causes the quality of the ultimate decision to become worse, not better. The models for these theorems assume that the search procedure is minimax and the games are two-player zero-sum. This report extends Nau's pathology theorem to multi-player game trees searched with maxn, the multi-player version of minimax. Thus two-player zero-sum game trees and multi-player game trees are shown to have an important feature in common.
1. Introduction Games played by more than two players have important characteristics that mark them as altogether different from two-player zero-sum games. Correspondence to: D. Mutchler, Department of Computer Science, University of Tennessee, Knoxville, TN 37996-1301, USA. Telephone: (615) 974-5067. E-mail: [email protected]. *This research was supported by the NSF under grant IRI g9-1072g, and by AFOSR under grant 90-0135. An abridged version of this paper appeared in: Proceedings Sixth International Symposium on Methodologies for Intelligent Systems, Charlotte, NC (1991).
0004-3702/93/$ 06.00 t~ 1993 - - Elsevier Science Publishers B.V. All rights reserved
324
D. Mutchler
Consider, for instance, the concept of an equilibrium point. Such a point is a collection of strategies, one per player, such that no player can improve her score by a unilateral change in her own strategy. For two-player zero-sum games, all equilibrium points yield the same outcome for the game. For multi-player games, different equilibrium points may yield different payoffs to the players. As such, the evident concept of a "solution" for a two-player zero-sum game does not extend as clearly to multi-player games. This report shows that despite their differences, two-player games and multi-player games do share one interesting property, namely, the existence of pathological game trees for which deeper search degrades the quality of the decision made. The standard search procedure for twoplayer zero-sum games is minimax. Its pathology has been demonstrated by several researchers [7,18,20,22-24]; see [2] for an excellent summary. The contribution of this report is to extend that pathology to multi-player games searched with m a x n [ 12,13,16 ], the multi-player version of minimax. 1 Section 2 gives a formal statement of minimax and m a x n . Section 3 reviews the pathology theorems that have appeared for two-player zerosum games. Section 4 contains a new pathology theorem that applies to multi-player games.
2. Minimax and
maxn
This section reviews the game theory used in this report. The books by Shubik [27] and by Luce and Raiffa [15] are good sources for a more complete account. G a m e theory treats many kinds of games. This report restrictsits attention to perfect information games, thus prohibiting, for example, simultaneous moves. The players are assumed to alternate turns in some fixed order. The computer science literaturehas been concerned largely with two-player zero-sum games; here that restriction is removed. The treatment of multiplayer games in this report assumes a noncooperative solution theory, thus prohibiting side payments, binding agreements, and threats, for example. Although a version of minimax has been developed for games with chance nodes [6,29], such nodes are not considered in this report. The trees considered are assumed to be finite.For a moment, let us look only at two-player zero-sum games. In that case, each leaf of the game tree has an associated payoff to the player who moves to that leaf. The payoff to the other player is the negative of the payoff at the leaf. Minimax, expressed l T h r o u g h o u t we use the typographically simpler n o t a t i o n m a x n used in [ 12 ] instead of the earlier n o t a t i o n m a x n used in [ 16].
Multi-player game-tree pathology
325
here by its negamax formula [11], provides one strategy for playing such a game.
mm (g) =
the payoff at leaf g, if node g is a leaf, - max{ram (g') I g' is a child of g}, otherwise.
A player follows the minimax strategy by selecting, each time the player is to move, the node g whose minimax value (from the above formula) is largest. The minimax strategy has two virtues [27, Section 8.2.1 ]. First, it provides the optimum level of s a f e t y . The ultimate payoff to the player who is using the minimax strategy is guaranteed to be no worse than the minimax value of the node to which that player is moving. This guarantee holds no matter what strategy the opponent is using. The second virtue of the minimax strategy is that it is in e q u i l i b r i u m : even if you somehow know that your opponent is using minimax, you cannot do any better than by using minimax yourself. It is the opposite natures of these virtues---the first guaranteeing a minimal value against an opponent about whom you know nothing, the second limiting you to a maximal value against an opponent about whom you know everything--that provide the universal acceptance of the "rationality" of the minimax strategy. Turn now to multi-player games. (All that is said in this report about multi-player games applies equally well to two-player games that are not zero-sum. Hereafter, include such games within the class of "multi-player games". ) Each leaf now has an associated v e c t o r of payoffs, with the j t h component of the vector being the payoff to the j t h player. Kuhn has shown that perfect information multi-player games, like two-player zero-sum games, possess solution strategies that are in equilibrium [ 13 ]. That is, there exists a vector S of strategies, one strategy per player, such that the following can be said of each player: even if the distinguished player somehow knew that the other players would abide by their respective strategies announced in the vector S, the distinguished player could not achieve a higher payoff by using a strategy other than the strategy in S associated with the distinguished player. Here is another way to state this same property: no player can improve his or her payoff by a unilateral change in strategy. How does one compute an equilibrium point, that is, how does one find a set of strategies in equilibrium? The proof that Kuhn gave in 1953 to show the existence of an equilibrium point for perfect information games is constructive; the induction he used yields an algorithm. (See [ 13, Corollary 1, p. 209 ].) However, the algorithm is only a by-product of the proof and is not stated explicitly. The same algorithm was independently introduced to the AI community and termed m a x n by Carol Luckhardt and Keki Irani
326
D. Mutchler
Who moves:
(9,6,5)
A
(3,7,4)
(9,6,5)
B
(7,3,4)
(3,7,4)
(9,6,5)
(5,3,2)
C
(9,6,1) A
>
(7,3,4) (3,7,4) (8,2,3) (9,6,5)
A A A A A
9 6 1
4 7 2
2 8 0
7 3 4
3 7 4
2 8 9
5 0 6
8 2 3
7 5 8
9 6 5
(6,3,2)
(6,7,0) (5,3,2)
AAA
0 8 1
6 3 2
2 3 1
6 7 0
5 3 2
3 2 1
• Fig. 1. The maxn backup rule. [16]; they also discussed pruning algorithms for m a x n . (See also [12].) A player follows the m a x n strategy by moving to the node whose m a x n vector is recursively selected as follows. • If node g is a leaf, m a x n (g) = the payoff vector at leaf g. • If node g is not a leaf, let V denote the m a x n vectors of g's children, i.e., V = { m a x n ( g ' ) [ g ' is a child of g}. Let i denote the player whose turn it is to move from node g. Then m a x n ( g ) = the vector in V whose ith component is maximal. Figure 1 shows the m a x n vectors for the nodes in a sample tree. The elements of the vectors refer to players A, B, and C, respectively. (Leaf values are displayed as column vectors.) Player A moves first, then B, then C, then A again. Unfortunately, an equilibrium point in a multi-player game lacks the guarantee of safety it provides in a two-player zero-sum game. In a multi-player game, different equilibrium points may provide different payoff vectors. Nonetheless, m a x n is a reasonable strategy for multi-player games. For perfeet information games, it is perhaps the strategy that is most "rational". The fact that m a x n is analogous to minimax enhances its appeal. The above description of the m a x n vector is ambiguous: what vector is selected if two or more vectors share the same maximal value at their /th component? Luckhardt and Irani showed that m a x n with any tie-breaking
Multi-player game-treepathology
327
rule yields an equilibrium point [ 16 ]. The results in this report are oblivious to the tie-breaking rule used, so the ambiguity will be left as it is. In practice, it is computationally infeasible to compute the minimax value or maxn vector for game trees of even modest depth, even with the help of pruning algorithms like alpha-beta [ 11,28] that compute the same value as minimax, but faster. Instead, one uses a heuristic evaluator e (g) to estimate the minimax value of node g in a two-player zero-sum game, or a heuristic evaluator e(g) to estimate the maxn vector at node g in a multi-player game. A typical heuristic evaluator for chess, for instance, assigns 8 points to queens, 5 points to rooks, 3 points to bishops, etc; the evaluation of a position is the total point-value of one player's pieces subtracted from the total point-value of the other player's pieces. To select a node to which to move from a set G of possible choices, a player could simply apply the heuristic evaluator to each of the nodes in G; the player then moves to whichever node g has the highest value e(g) (for two-player zero-sum games) or whose vector e (g) (for multi-player games) has the maximal value at that player's component. Another (more common) approach is to apply the heuristic evaluator to the leaves of depth d subtrees below the nodes in G. The player then computes the minimax value or maxn vector of the nodes in G by applying the minimax or maxn backup rule as if the nodes evaluated heuristically were actual payoffs. The common belief is that this second approach--use the backed-up values/vectors instead of the direct heuristic evaluations of the nodes in G--is superior. The pathology theorems show that it need not be.
3. Pathology theorems for two-player zero-sum games In two-player zero-sum games, if the heuristic evaluator is completely accurate, then minimax using the heuristic evaluator returns the same number as minimax using the actual payoffs. In practice, the heuristic evaluator will sometimes err, thereby introducing errors into the backed-up minimax value as well. The question is now: is the minimax value computed from heuristic evaluations a good estimate of the actual minimax value? Judea Pearl comments [25, p. 332]: In fact it is surprising that the minimax method works at all . . . . Minimaxing these estimates as if they were true payoffs amounts to committing one of the deadly sins of statistics: computing a function of the estimates instead of an estimate of the function. In the same way that the average of a product generally is not equal to the product of the averages, so also the estimate of the minimax differs from the minimax of the estimates.
328
D. Mutchler
Pearl continues by explaining that one argument given to justify minimaxing the heuristic evaluations is a filtering argument: the backed-up value reflects the information contained in the entire frontier of the search. The pathology theorems show that this filtering argument is fallacious. The literature contains three classes of pathology theorems for two-player zero-sum games. All the theorems model the heuristic evaluator applied to a node as a random variable, capable of error. The conclusion of each theorem has the same consequence: it is increasingly likely as the depth of the search increases that the children of the root node will all have the same minimax value. Hence the ability of minimax to distinguish good moves from bad is lost; minimax under deep search is reduced to a decision-maker that selects moves at random. • Class 1. This class contains the theorem from Dana Nau's thesis [20] and its extension in [22]. Beal independently developed a somewhat weaker theorem [7] that falls in this class. The three important assumptions of these theorems are: (a) the probability of the heuristic evaluator being totally wrong is bounded away from zero; (b) the heuristic evaluations on the different nodes possess some degree of independence; and (c) the branching factor is sufficiently large. (The more accurate the evaluator, the larger the branching factor required.) The conclusion is that the minimax value under deep search is almost certain to return the highest possible value for the player who moved last in the search. • Class 2. Pearl has a different pathology theorem [24]. By assuming that the actual outcomes of the game are either a WIN (with probability p) or a LOSS (with probability 1 - p ) , Pearl does not need to assume a sufficiently large branching factor. In fact, Pearl's theorem assumes a binary tree and a two-valued heuristic evaluator, but he indicates how these assumptions could be lifted. • Class 3. A third pathology theorem was discovered independently by Nau and Pearl [18,23]. Here the heuristic evaluators are assumed to be identically distributed as well as independent. The conclusion is that minimax under deep search is almost certain to return a fixed value determined by the distribution of the evaluators. The assumption of identical distributions is clearly not reasonable; as such, this theorem is more a statement about game trees with random outcomes than a pathology theorem.
As a concrete example, here is a simplified version of Nau's pathology theorem in [22]. Let G be a complete, infinitely deep, b-ary game tree. For each node g of G, model the heuristic evaluation e ( g ) as a discrete random variable with range - h to h. Assume these random variables are all independent (although not necessarily identically distributed). Let tie(d, g )
Multi-playergame-treepathology
329
denote the event that all children of g receive the same m i n i m a x value when a depth d search is done. The pathology theorem is: if b is sufficiently large, 2 then the probability of tie(d, g) converges to 1 as the depth d gets large. Thus, m i n i m a x under deep search reduces to a decision-maker that selects moves at random.
4. A p a t h o l o g y t h e o r e m for multi-player games
This section presents a pathology theorem for multi-player games. The theorem fallsinto the firstof the three classes of pathology theorems listed in the previous section. Its assumptions arc very much like those of Nau's two-player pathology theorem in his thesis [20] and its extension [22]. The proof strategy has two steps. First, I will extend Nau's theorem to game trees in which the branching factor for one player need not bc the same as the branching factor for the other player. The technique for doing so will follow Nau's approach quite closely. The result will b c a lemma stating that, for cvcn search depths, the minimax value is almost certain to bc the highest value possible,under appropriate assumptions. Second, I will reduce a multi-playergame to a two-player game in which all the players but one form a coalition against the remaining player. From the lemma in the firststep, the minimax value of this two-player game is almost certain to bc the highest value possible, for cvcn search depths. The multi-player result will then follow by using the fact that the m a x n value for the distinguished player is at leastas large as the value returned by minimax in the two-player game. Let b and B bc real numbers with b > I and B > I. Let x bc any real number between 0 and I. Define
si(b,B,x) =
i'
i-1 ( B , b , x ) b, -- [ 1 - S i _ l ( B , b , x ) ]
i f / = O, if i > 0 is even,
b, i f i > O i s o d d .
Note the reversal of the b and B arguments in the recursion. W h e n b = B, this function is the same as the function si (b, x ) that N a u used. He displayed a certain function w of b such that si(b,x) converges to either O, w (b), or 1 as i grows large, depending on whether x is smaller, equal to, or greater 2Let d denote the greatest lower bound over nodes g in G, of Pr [e (g) = h ]. Let w (b) denote the unique solution in the open interval (0, 1) to the equation x = (1 - (1 -x)b) b. (Note that w(b) is between 0 and I and is a decreasing function ofb.) Then "b is sufficiently large" means that d > w (b). Thus, as long as all evaluations e (g) have a bounded-away-from-zero probability of returning their highest possible value h, then b can be made sufficiently large.
330
D. Mutchler
than w (b) [18]. The following lemma shows that the behavior in the limit o f s i ( b , B , x ) is similar. Lemma 4.1. There exists a function w (b, B) o f b and B such that the limit o f the sequence (s:i} (i.e., {si } on its even-numbered terms) satisfies
lira s 2 i ( b , B , x ) =
1, w(b,B),
if w ( b , B ) < x <~ 1,
O,
ifO <~ x < w ( b , B ) .
if x = w ( b , B ) ,
Proof. See [ 17 ] for a proof, which involves close examination of the first and second derivatives of a certain function of x that corresponds to {s2i}. [] The lemma that relates function s to minimax search is the following version of [22, L e m m a A.1 ]. Lemma 4.2. Consider a game tree T in which two players, each with perfect information, alternate moves. Suppose that one player has b > 1 choices at each turn and the other has B > 1 choices. For each node x in T, model the heuristic evaluation e (x) as a discrete random variable with range - h to h. Assume these random variables are all independent (although not necessarily identically distributed). Let c denote the greatest lower bound over nodes x in T, o f P r [ e ( x ) = h]. Then for every node g at which there are b choices and for every node G at which there are B choices, if the subtrees rooted at g and G are complete to at least depth d, we have the following bounds on the probability that the depth d minimax search returns an extreme value: P r [ m m ( d , g ) = ( - 1 ) d h ] >i sd(b,B,c), P r [ m m ( d , G ) = ( - 1 ) d h ] >f Sd(B,b,c).
Proof. The proof is an easy modification of the proof of the original lemma in [22], which is itself an easy induction on d. Two modifications are required. First, the induction hypothesis here is a pair of statements, one for nodes with branching factor b and the other for nodes with branching factor B. Second, the induction step proceeds from a g node (branching factor b) to a G node (branching factor B), or vice versa. This accounts for the reversal of the b and B arguments in the recursive definition of si (b, B, x). In short, the definition of si (b, B,x) is rigged precisely to permit the truth of this lemma. []
Multi-player game-treepathology
331
Lemmas 4.1 and 4.2 permit a two-player pathology theorem for games with a dual branching factor; see [ 17 ]. Our interest, however, is in multi-player games and m a x n . Here is the pathology theorem for that case.
Theorem 4.3 (The multi-playerversion of minimax displayspathology). Let player A denote the player to move from the root o f a game tree. Let e (g) be a vector-valued heuristic evaluator. Let e~ (g) denote the component o f e (g ) that pertains to player A. Suppose • Shape of the game tree: - Several players in a multi-player, perfect information game alternate turns. - The branching factor for player A is b > 1. - The combined branching factor for the other players is B > I. - The tree is complete and infinitely deep. • Player A's heuristic evaluator: - For each node g to which player A might move, the heuristic evaluation eA(g) is modeled as a discrete random variable with range - h to h. - These random variables are all independent (although not necessarily identically distributed). - c > w (B, b) where c denotes the greatest lower bound over nodes g to which player A might move, ofPr[eA (g) = h]. • Other players' heuristic evaluator: - The other player's components o f e ( g ) can be any functions, either random (with any distribution) or deterministic. • Search depth: - Player A computes the maxn vectors o f the children o f the root and selects whichever o f these vectors maximizes ,4 'S component. - The terminal nodes o f these maxn searches (i.e., the nodes to which A applies e ( g ) ) are nodes to which A might move. Then it is almost certain that for deep searches, player A will reduce to a decision-maker who selects moves at random. That is, if V denotes { m a x n ( g ) } such that g is a child o f the root and the maxn search is to depth d, then the probability converges to 1 that, as the depth d grows large, all o f the vectors in V will have h as their value for player .4"s component. Proof. Convert the given multi-player game into a two-player zero-sum game as follows. The structure of the game tree remains the same. The protagonists in the two-player game are, first, player A, and second, a coalition consisting of all the other players from the multi-player game. On nodes g to which player A might move, the heuristic evaluator is the function e.4 taken from the multi-player game. On other nodes in the two-player game, the heuristic
D. Mutchler
332
evaluation is identically h. Player A applies minimax with an even-depth search to the children of the root of the two-player game tree. Figure 2 shows a sample multi-player tree searched with maxn and the corresponding twoplayer tree searched with minimax. Let GI through Gb denote the b children of the root in the constructed two-player game. The probability that all b of these nodes have depth d minimax value h is, using the fact that the eA'S are independent, b
I-[ Pr[mm(d, Gi) = h]. i=1
Since each child G~ of the root of the constructed two-player game has branching factor B, and since the depth d of each minimax search is even, each of the terms in the above product is at least Sd(B,b,c), from I-emma 4.2. Conclude that the probability that all b of the minimax searches return h is at least Sd (B, b, c) b. Now apply Lemma 4.1, using the assumption that c > w (B, b) and the assumption that the search is to an even depth in the two-player game. Since SEd(B, b, c) b converges to 1 as d grows large, so does the probability that all b of the minimax searches return h. The proof is concluded by comparing A's component of the maxn(g) vector in the multi-player game to ram(g) in the constructed two-player game. When it is player A's turn, maxn and minimax proceed analogously. Consider next the behavior when it is not player A's turn. Under maxn, the other players select a vector that is certainly no worse for player A than the vector selected by conspiring against player A, as they do in the constructed two-player game. Thus A's component of the maxn(g) vector in the multi-player game is at least as large as ram(g) in the constructed two-player game. As argued, it is almost certain that all b of the minimax searches return h, the largest value possible; hence it is almost certain that all b of the maxn searches also return h in player A's component. []
5. Discussion
This paper has presented a pathology theorem for multi-player games. Its assumptions are hardly more than that which is needed for two-player games. No assumption whatever is required of the backup functions for the players other than the one player for whom pathology is proved. Thus the theorem applies not only to maxn, but also to any other algorithm that agrees with maxn whenever it is player A's turn. The assumptions of the pathology theorem for multi-player games are essentially the same assumptions as Nau's pathology theorem for two-player
333
Multi-player game-tree pathology
Who moves: A
(3,7,4)
(9,6,5)
B
(7,3,4)
(3,7,4)
(9,6,5)
(5,3,2)
C
(9,6,1)
(7,3,4)
(3,7,4)
(8,2,3)
9 6
4 7
2 8
7 3
3 7
2 8
5 0
1
2
0
4
4
9
6
(9,6,5)
(6,3,2)
(6,7,0)
(5,3,2)
AAAAAAAA
A
8 2
7 5
9 6
0 8
6 3
2 3
6 7
5 3
3 2
3
8
5
1
2
1
0
2
1
m a x n using evaluator g
Who moves: A
//
Coafiffon
-9
5
3
/
-7
-3
-8
A
9
4
2
7
3
2
5
-9
-6
-6
-5
AAAAAAAA
8
7
9
0
6
2
6
m i n i m a x using evaluator eA
Fig. 2. A sample multi-player game and the constructed two-player game.
5
3
334
D. Mutchler
zero-sum games, with one addition: the search must end at nodes to which player A might move. That is, the heuristic evaluation function is always applied to nodes that are children of player A's decision points. This assumption is not unreasonable, since game-playing programs often stop at full-ply increments to sidestep any bias in the heuristic evaluator. The assumption is also a necessary one, in some sense. Without it, player A's component of the m a x n evaluations can be forced to converge to either extreme ( + h or - h ), depending on the details of the heuristic evaluator. For example, if the other players' components of the heuristic evaluator always yield the negative of player A's component, the multi-player game reduces to a two-player zero-sum game. Then player A's component of the m a x n evaluations at odd depths converges to - h , under appropriate assumptions. On the other hand, if {eA(g)} are independent of the other components of the heuristic evaluator, then player A's component of the r n a x n evaluations converges to + h for all search depths, odd or even. See [ 17] for details. The theorem applies when the heuristic evaluator has error probability larger than w (B, b). This function can be made arbitrarily small by appropriate choice of B and b. For instance, suppose there is a constant branching factor per player, so that B = b k for some fixed positive integer k. Since w ( b k , b ) ~ 0 as b ---, c~ (see [17] for a proof), the pathology theorem applies as long as the error in player A's component of the heuristic evaluator is bounded away from zero. However, the convergence of w ( B , b ) is somewhat slow, so very large branching factors may be required for the theorem to apply to accurate heuristic evaluators. For simplicity of presentation, we have assumed that the heuristic evaluations eA (g) are independent, although a weaker condition (p-dependence bounds, as given by Nau in [22] ) would suffice. See [17] for details. Despite the pathology theorems, minimax pathology has not been observed in games like chess. Several explanations for the lack of pathology have been proposed, supported by empirical and theoretical evidence [1,2,810,14,19,21,22,24,26]. It would be interesting to see how these explanations affect m a x n and to see whether m a x n does, in practice, exhibit pathology. Experience with m a x n using both hand-crafted and "generic" heuristics (via Abramson's expected-outcome model [3-5], for example) would be welcome. There are two lessons to be learned from the pathology theorem in this report. First, it provides theoretical evidence that searching deeper in the game tree when using m a x n may not be a good idea. Second, it shows that multi-player games and two-player zero-sum games share an important property, namely, game-tree pathology. This is surprising, since traditional game theory gives the two types of games two quite different treatments.
Multi-player game-tree pathology
335
Acknowledgement M u c h o f this w o r k was d o n e while the a u t h o r was in the C o m p u t e r Science a n d Systems Branch o f the N a v a l R e s e a r c h L a b o r a t o r y , Washington, DC. T h e a u t h o r is grateful to M a r y E b e d e i n a n d Bruce A b r a m s o n for their helpful c o m m e n t s .
References [I ] B. Abramson, An explanation of and cure for minimax pathology, in: L.N. Kanal and J.F. Lemmcr, eds., Uncertainty in ArtificialIntelligence,Machine Intelligenceand Pattern Recognition 4 (North-Holland, Amsterdam, 1986) 495-504. [2] B. Abramson, Control strategies for two-player games, ACM Comput. Sure. 21 (2) (1989) 137-161. [3] B. Abramson, An analysis of expected-outcome, J. Exper. Theor. Artif. Intell. 2 (I) (1990) 55-73. [4] B. Abramson, Expected-outcome: a general model of static evaluation, IEEE Trans. Pattern Anal. Mach. Intell. 12 (2) (1990) 182-193. [5] B. Abramson, On learning and testing evaluation functions, J. Exper. Theor. Artif. Intell. 2 (3) (1990) 241-251. [6] B.W. Ballard, The .-minimax search procedure for trees containing chance nodes, Artif. Intell. 21 (1-2) (1983) 327-350. [7] D.F. Beal, An analysis of minimax, in: M.R.B. Clarke, ed., Advances in Computer Chess 2 (Edinburgh University Press, Edinburgh, Scotland, 1980) 103-109. [8] D.F. Beal, Benefits of minimax search, in: M.R.B. Clarke, ed., Advances in Computer Chess 3 (Pergamon, Elmsford, NY, 1982) 17-24. [9] I. Bratko and M. Gains, Error analysis of the minimax principle, in: M.R.B. Clarke, ed., Advances in Computer Chess 3 (Pergamon, Elmsford, NY, 1982) 1-15. [10] H. Kaindi, Minimaxing--theory and practice, AI Mag. 9 (3) (1988) 69-76. [11] D.E. Knuth and R.W. Moore, An analysis of alpha-beta pruning, Artif. Intell. 6 (4) (1975) 293-326. [12] R.E. Koff, Multi-player alpha-beta pruning, Artif. Intell. 48 (1) (1991) 99-111. [ 13 ] H.W. Kuhn, Extensive games and the problem of information, in: H.W. Kuhn and A.W. Tucker, eds., Contributions to the Theory of Games, Vol. H (Princeton University Press, Princeton, NJ, 1953) 193-216. [14] L. Li and T.A. Marsland, On minimax game tree search pathology and node-value dependence, Tech. Report 90-24, Department of Computing Science, University of Alberta, Alta. (1990). [15] R.D. Luce and H. Raiffa, Games and Decisions (Wiley, New York, 1957). [16] C.A. Luckhardt and K.B. Irani, An algorithmic solution of n-person games, in: Proceedings AAAI-86, Philadelphia, PA ( 1986) 158-162. [17] D. Mutchler, The multi-player version of minimax displays game-tree pathology, Tech. Report CS-93-198, Department of Computer Science, University of Tennessee, Knoxville, TN (1993). [18] D.S. Nau, The Last Player Theorem, Artif. lntell. 18 (1) (1982) 53-65. [19] D.S. Nau, An investigation of the causes of pathology in games, Artif. Intell. 19 (3) (1982) 257-278. [20] D.S. Nau, Decision quality as a function of search depth on game trees, J. ACM 30 (4) (1983) 687-708. [21 ] D.S. Nau, On game graph structure and its influence on pathology, Int. J. Comput. Inf. ScL 12 (6) (1983) 367-383.
336
D, Mutchler
[22] D.S. Nau, Pathology on game trees revisited, and an alternative to minimaxing, Artif. Intell. 21 (1-2) (1983) 221-244. [23] J. Pearl, Asymptotic properties of minimax trees and game-searching procedures, Artif Intell. 14 (2) (1980) 113-138. [24] J. Pearl, On the nature of pathology in game searching, Artif. Intell. 20 (4) (1983) 427-453. [25] J. Pearl, Heuristics--Intelligent Search Strategies for Computer Problem Solving (Addison-Wesley, Reading, MA, 1984). [26] A. Scheucher and H. Kaindl, The reason for the benefits of minimax search, in: Proceedings IJCAI-89, Detroit, MI (1989) 322-327. [27] M. Shubik, Game Theory in the Social Sciences (MIT Press, Cambridge, MA, 1982). [28] J.R. Slagle and J.K. Dixon, Experiments with some programs that search game trees, J. ACM 16 (2) (1969) 189-207. [29] M. van Lent, A pruning algorithm for one player games with hidden information, M.S. Thesis, University of Tennessee, Knoxville, TN (1993).