- Email: [email protected]
A note on Arbuckle and Larimer, “the number of two-way tables satisfying certain additivity axioms”
JOURNAL
OF MATHEMATICAL
A Note Two-Way
PSYCHOLOGY
15, 292-295
on Arbuckle and Larimer, “The Number of Tables Satisfying Certain Additivity Axioms”* GARY
Department
(1977)
of Psychology,
MCCLELLAND
University
of Colorado,
Boulder,
Colorado
80309
Arbuckle and Larimer (1976) present an ingenious Monte Carlo method for estimating the number of two-way tables that satisfy the axioms of additive conjoint measurement (cf. Krantz, Lute, Suppes, & Tversky, 1971). Arbuckle and Larimer selected the Monte Carlo approach over a more direct enumeration because for tables of size 4 x 5 and larger, the number of tables satisfying independence (for which they give an exact expression) is prohibitively large for checking the higher-order cancellation conditions. For example, they show that for the 4 x 5 case, one would need to search 1.7 million tables. This note presents the exact values for those tables for which enumeration is possible. While these exact values have the small benefit of confirming the Arbuckle and Larimer Monte Carlo results, the main advantage is that the list of tables constructed in the direct enumeration approach makes it possible to answer an important related question: What is the statistical distribution of the minimum number of pair reversals necessary to convert a random table into an additive table satisfying the conjoint measurement axioms ? Goode (1964) and Tversky and Zivian (1966) h ave proposed algorithms for finding that minimum number of pair reversals for given data tables. It is therefore important to evaluate the probability of obtaining various minimum numbers by chance. In this note are presented (1) an efficient search algorithm for enumerating the tables satisfying the additivity axioms, (2) the values obtained by that algorithm for 3 x 3, 3 x 4, 3 x 5, and 4 x 4 tables, and (3) the statistical distribution of the minimum number of pair reversals necessary to achieve additivity for a random table. The basic method is to construct all r x c tables that satisfy independence and then check those tables for violations of double- and higher-order cancellation. The construction of those tables is facilitated by numbering each cell of the table and then using a standard permutation generator to produce all orderings of the rc integers with the stipulation that the first cell in the ordering receives rank 1, the second cell rank 2, etc. Since even for the 4 x 4 tables there are more than 2 x 101a possible tables, some computation reduction techniques are required. The first computation reduction is achieved by establishing a set of equivalence classes on the set of all possible rank tables. Using the notation of Arbuckle and Larimer, a table is called a normal table if it satisfies the following definition. * Helpful suggestions on this note were made by Bruce Bloxom and Clyde putations were conducted using the facilities of the Computer Laboratory Psychology of the Department of Psychology of the University of Colorado. Copyright 0 1977 by Academic Press, Inc. All rights of reproduction in any form reserved.
Coombs. The for Instruction
comin
292 ISSN
0022-2496
TWO-WAY DEFINITION
(4
1. An r
x
TABLES
293
c table, W, containing the integers 1,2,.., YCis a normal tabZeif
run < wi5 for all i,j
(b) ruil < wkl whenever i < K,
and
(c) rulj < wllc wheneverj < k. Thus, any Y x c table of ranks can be converted to normal form by permuting the rows and columnsso that the lowest rank is in the first row and column and both the first row and column are strictly increasing. Using an argument similar to Arbuckle and Larimer (1976, p. 93) for counting their regular tables, it can be shown that the number of normal tables of size Y x c is (rc)!/r!c!, and therefore each equivalence classcontains r!c! tables. The secondcomputation reduction is achievedby the useof backtrack or tree programming (cf. Golomb & Baumert, 1965, or Wells, 1971, pp. 93-126). Tree programming is an efficient method for enumerating all integer orderings consistent with a given partial ordering (poset). In the present application, the independence axioms and the definition of a normal table generatea natural partial order. The computation reduction is achieved by searchingnode by node a lexicographic tree of all possibleorderings and at each node checking for an impasse(i.e., an inconsistency with the poset required by independence).The tree need not be searchedbeyond any node at which an impasse is detected (i.e., the tree is “pruned” of all branches beyond that node). With a highly constraining poset (such as generated by independence),the small computattional cost of impassechecking at each node results in a dramatic reduction in the total search computations. * It is then easyto check all the independentnormal tables (all of Arbuckle and Larimer’s regular tables) generatedby the above algorithm for violations of the double cancellation axioms. Those tables surviving the double cancellation test could then be explicitly checked for violations of triple- and higher-order cancellation. However, it is sufficient to check these higher-order cancellation conditions implicitly by testing the surviving tables for additivity. The ORDMET algorithm (McClelland & Coombs, 1975)was used for the additivity check; any linear programming algorithm could be used. Table 1 presentsthe results of the above enumeration strategy for four table sizes. The corresponding probabilities that a random matrix will satisfy the various additivity axioms are also presented in Table 1. Not surprisingly, all probabilities fall within the 9594 confidence intervals presentedby Arbuckle and Larimer. Becausethe number of additive tables for these table sizes is reasonably small, it is possibleto determine the distribution of the minimum number of pair reversalsnecessary to make a randomly selected table additive. The basic Monte Carlo procedure is to generate a number of random tables, permute rows and columns to obtain normal form, compare them to each additive table, and calculate the minimum number of rank pair reversals necessaryto make a given random table identical to an additive ordering. For the 3 x 3 case,it is computationally possibleto compare all possiblenormal tables to the 36 additive matrices so the results for that caseare exact. The distributions for three table sizesare presentedin Fig.1. 4w5/3-6
294
GARY
MCCLELLAND
TABLE Number Satisfying
1
of Matrices and Their Probabilities of Additive Conjoint Measurement Axioms Matrix
Measurement
condition
3x3
3X4
Number All possible matrices All normal matrices Independent normal matrices Independence and double cancellation Additive normal matrices
3.63 1.01
D All
entries
are exact
to three
4.79 3.33
36
Independence Double cancellation given independence Additivity Additivity given independence Additivity given independence and double cancellation
4X4
3x5
of matrices
x loj x 104 42 36
Probability
size
x 108 x 106 462 303
1.31 s 1012 1.82 x 10B 6006 2773 2583
295 of satisfying
2.09 x 10’3 3.63 x 10”’ 24024 7840 6660
condition”
x 10-a .857
1.39
x lo-” .656
3.31
x 10-e 462
6.61
x 10-a .857 1.ooo
8.87
x 1O-5 .639 .974
1.42
x 1O-6 .430 .931
1.83 x lo-’ .277 .849
4.17
3.57
significant
x 10-7 .326
digits.
l.OO-
10,000
roildam
PIN12b.034 PIN121=.034
0
IO
20 MINIMUM
FIG.
additivity
1.
Probability distribution for three matrix sizes.
functions
NUMBER TO ACHIEVE
30 40 OF PAIR REVERSALS ADDITIVITY
for a minimum
number
of pair
50
reversals
to
achieve
TWO-WAY
TABLES
295
Let R, be the number of pair reversals sufficient to convert at most proportion (Y of random tables to additivity. Then R,,,, eq uals 2 for 3 x 3 tables, 15 for 3 x 4 tables, and 26 for 4 x 4 tables. The corresponding values of Kendall’s T (comparing the original ordering to the best-fit additive ordering) are 0.89, 0.55, and 0.57; it would be a rare event to obtain a table by chance which had a higher r coefficient with an additive table than these values. The approximate expected numbers of pair reversals for these tables (with the corresponding 7 in parentheses) are respectively 7.53 (0.58), 22.15 (0.33), and 35.09 (0.42). It is important to note that while the probability of obtaining by chance an additive matrix decreases and R, and the expected number of pair reversals increases as table size increases, the corresponding 7 values do not necessarily decrease with table size. That is, the increase in the number of necessary pair reversals is not as fast as the increase in the number of potential pair reversals with larger table sizes. The Arbuckle and Larimer results and the results in Table 1 clearly indicate that it is highly unlikely that a table satisfying the additive conjoint measurement axioms could be obtained by chance. Further, the statistical distributions in Fig. 1 indicate that for tables larger than 3 x 3, it is highly unlikely that one would obtain by chance a matrix that would even be close to an additive table (where close is defined in terms of number of pair reversals to achieve additivity). However, it must be emphasized that the results of Fig. 1 should not be used as a classical statistical test since the null hypothesis of random ordering on which these results are based would seldom be appropriate for a real experiment (see Arbuckle & Larimer, 1976, p. 90 for a more complete discussion of this issue).
REFERENCES ARFIUCKLE, J., & LARIMER, J. The number of two-way tables satisfying certain additivity axioms. Journal of Mathematical Psychology, 1976, 13, 89-100. GOLOMB, S. W., & BAUMERT, L. D. Backtrack programming. Journal of The Association for Computing Machinery, 1965, 12, 5 16-524. GOODE, F. M. An algorithm for the additive conjoint measurment of finite data matrices. American Psychologist, 1964, 19, 579. KRANTZ, D. H., LUCE, R. D., SUPPES, P., & TVERSKY, A. Foundations of measurement, New York: Academic Press, 1971. MCCLELLAND, G. H., & COOMBS, C. H. ORDMET: A general algorithm for constructing all numerical solutions to ordered metric structures. Psychometrika, 1975, 40, 269-290. TVERSKY, A., 8: ZIVIAN, A. An IBM 7090 program for additivity analysis. Behavioral Science, 1966, 11, 78-79. WELLS, M. B. Elements of combinatorial computing. New York: Pergamon Press, 1971. RECEIVED:
September 29, 1976
OF MATHEMATICAL
A Note Two-Way
PSYCHOLOGY
15, 292-295
on Arbuckle and Larimer, “The Number of Tables Satisfying Certain Additivity Axioms”* GARY
Department
(1977)
of Psychology,
MCCLELLAND
University
of Colorado,
Boulder,
Colorado
80309
Arbuckle and Larimer (1976) present an ingenious Monte Carlo method for estimating the number of two-way tables that satisfy the axioms of additive conjoint measurement (cf. Krantz, Lute, Suppes, & Tversky, 1971). Arbuckle and Larimer selected the Monte Carlo approach over a more direct enumeration because for tables of size 4 x 5 and larger, the number of tables satisfying independence (for which they give an exact expression) is prohibitively large for checking the higher-order cancellation conditions. For example, they show that for the 4 x 5 case, one would need to search 1.7 million tables. This note presents the exact values for those tables for which enumeration is possible. While these exact values have the small benefit of confirming the Arbuckle and Larimer Monte Carlo results, the main advantage is that the list of tables constructed in the direct enumeration approach makes it possible to answer an important related question: What is the statistical distribution of the minimum number of pair reversals necessary to convert a random table into an additive table satisfying the conjoint measurement axioms ? Goode (1964) and Tversky and Zivian (1966) h ave proposed algorithms for finding that minimum number of pair reversals for given data tables. It is therefore important to evaluate the probability of obtaining various minimum numbers by chance. In this note are presented (1) an efficient search algorithm for enumerating the tables satisfying the additivity axioms, (2) the values obtained by that algorithm for 3 x 3, 3 x 4, 3 x 5, and 4 x 4 tables, and (3) the statistical distribution of the minimum number of pair reversals necessary to achieve additivity for a random table. The basic method is to construct all r x c tables that satisfy independence and then check those tables for violations of double- and higher-order cancellation. The construction of those tables is facilitated by numbering each cell of the table and then using a standard permutation generator to produce all orderings of the rc integers with the stipulation that the first cell in the ordering receives rank 1, the second cell rank 2, etc. Since even for the 4 x 4 tables there are more than 2 x 101a possible tables, some computation reduction techniques are required. The first computation reduction is achieved by establishing a set of equivalence classes on the set of all possible rank tables. Using the notation of Arbuckle and Larimer, a table is called a normal table if it satisfies the following definition. * Helpful suggestions on this note were made by Bruce Bloxom and Clyde putations were conducted using the facilities of the Computer Laboratory Psychology of the Department of Psychology of the University of Colorado. Copyright 0 1977 by Academic Press, Inc. All rights of reproduction in any form reserved.
Coombs. The for Instruction
comin
292 ISSN
0022-2496
TWO-WAY DEFINITION
(4
1. An r
x
TABLES
293
c table, W, containing the integers 1,2,.., YCis a normal tabZeif
run < wi5 for all i,j
(b) ruil < wkl whenever i < K,
and
(c) rulj < wllc wheneverj < k. Thus, any Y x c table of ranks can be converted to normal form by permuting the rows and columnsso that the lowest rank is in the first row and column and both the first row and column are strictly increasing. Using an argument similar to Arbuckle and Larimer (1976, p. 93) for counting their regular tables, it can be shown that the number of normal tables of size Y x c is (rc)!/r!c!, and therefore each equivalence classcontains r!c! tables. The secondcomputation reduction is achievedby the useof backtrack or tree programming (cf. Golomb & Baumert, 1965, or Wells, 1971, pp. 93-126). Tree programming is an efficient method for enumerating all integer orderings consistent with a given partial ordering (poset). In the present application, the independence axioms and the definition of a normal table generatea natural partial order. The computation reduction is achieved by searchingnode by node a lexicographic tree of all possibleorderings and at each node checking for an impasse(i.e., an inconsistency with the poset required by independence).The tree need not be searchedbeyond any node at which an impasse is detected (i.e., the tree is “pruned” of all branches beyond that node). With a highly constraining poset (such as generated by independence),the small computattional cost of impassechecking at each node results in a dramatic reduction in the total search computations. * It is then easyto check all the independentnormal tables (all of Arbuckle and Larimer’s regular tables) generatedby the above algorithm for violations of the double cancellation axioms. Those tables surviving the double cancellation test could then be explicitly checked for violations of triple- and higher-order cancellation. However, it is sufficient to check these higher-order cancellation conditions implicitly by testing the surviving tables for additivity. The ORDMET algorithm (McClelland & Coombs, 1975)was used for the additivity check; any linear programming algorithm could be used. Table 1 presentsthe results of the above enumeration strategy for four table sizes. The corresponding probabilities that a random matrix will satisfy the various additivity axioms are also presented in Table 1. Not surprisingly, all probabilities fall within the 9594 confidence intervals presentedby Arbuckle and Larimer. Becausethe number of additive tables for these table sizes is reasonably small, it is possibleto determine the distribution of the minimum number of pair reversalsnecessary to make a randomly selected table additive. The basic Monte Carlo procedure is to generate a number of random tables, permute rows and columns to obtain normal form, compare them to each additive table, and calculate the minimum number of rank pair reversals necessaryto make a given random table identical to an additive ordering. For the 3 x 3 case,it is computationally possibleto compare all possiblenormal tables to the 36 additive matrices so the results for that caseare exact. The distributions for three table sizesare presentedin Fig.1. 4w5/3-6
294
GARY
MCCLELLAND
TABLE Number Satisfying
1
of Matrices and Their Probabilities of Additive Conjoint Measurement Axioms Matrix
Measurement
condition
3x3
3X4
Number All possible matrices All normal matrices Independent normal matrices Independence and double cancellation Additive normal matrices
3.63 1.01
D All
entries
are exact
to three
4.79 3.33
36
Independence Double cancellation given independence Additivity Additivity given independence Additivity given independence and double cancellation
4X4
3x5
of matrices
x loj x 104 42 36
Probability
size
x 108 x 106 462 303
1.31 s 1012 1.82 x 10B 6006 2773 2583
295 of satisfying
2.09 x 10’3 3.63 x 10”’ 24024 7840 6660
condition”
x 10-a .857
1.39
x lo-” .656
3.31
x 10-e 462
6.61
x 10-a .857 1.ooo
8.87
x 1O-5 .639 .974
1.42
x 1O-6 .430 .931
1.83 x lo-’ .277 .849
4.17
3.57
significant
x 10-7 .326
digits.
l.OO-
10,000
roildam
PIN12b.034 PIN121=.034
0
IO
20 MINIMUM
FIG.
additivity
1.
Probability distribution for three matrix sizes.
functions
NUMBER TO ACHIEVE
30 40 OF PAIR REVERSALS ADDITIVITY
for a minimum
number
of pair
50
reversals
to
achieve
TWO-WAY
TABLES
295
Let R, be the number of pair reversals sufficient to convert at most proportion (Y of random tables to additivity. Then R,,,, eq uals 2 for 3 x 3 tables, 15 for 3 x 4 tables, and 26 for 4 x 4 tables. The corresponding values of Kendall’s T (comparing the original ordering to the best-fit additive ordering) are 0.89, 0.55, and 0.57; it would be a rare event to obtain a table by chance which had a higher r coefficient with an additive table than these values. The approximate expected numbers of pair reversals for these tables (with the corresponding 7 in parentheses) are respectively 7.53 (0.58), 22.15 (0.33), and 35.09 (0.42). It is important to note that while the probability of obtaining by chance an additive matrix decreases and R, and the expected number of pair reversals increases as table size increases, the corresponding 7 values do not necessarily decrease with table size. That is, the increase in the number of necessary pair reversals is not as fast as the increase in the number of potential pair reversals with larger table sizes. The Arbuckle and Larimer results and the results in Table 1 clearly indicate that it is highly unlikely that a table satisfying the additive conjoint measurement axioms could be obtained by chance. Further, the statistical distributions in Fig. 1 indicate that for tables larger than 3 x 3, it is highly unlikely that one would obtain by chance a matrix that would even be close to an additive table (where close is defined in terms of number of pair reversals to achieve additivity). However, it must be emphasized that the results of Fig. 1 should not be used as a classical statistical test since the null hypothesis of random ordering on which these results are based would seldom be appropriate for a real experiment (see Arbuckle & Larimer, 1976, p. 90 for a more complete discussion of this issue).
REFERENCES ARFIUCKLE, J., & LARIMER, J. The number of two-way tables satisfying certain additivity axioms. Journal of Mathematical Psychology, 1976, 13, 89-100. GOLOMB, S. W., & BAUMERT, L. D. Backtrack programming. Journal of The Association for Computing Machinery, 1965, 12, 5 16-524. GOODE, F. M. An algorithm for the additive conjoint measurment of finite data matrices. American Psychologist, 1964, 19, 579. KRANTZ, D. H., LUCE, R. D., SUPPES, P., & TVERSKY, A. Foundations of measurement, New York: Academic Press, 1971. MCCLELLAND, G. H., & COOMBS, C. H. ORDMET: A general algorithm for constructing all numerical solutions to ordered metric structures. Psychometrika, 1975, 40, 269-290. TVERSKY, A., 8: ZIVIAN, A. An IBM 7090 program for additivity analysis. Behavioral Science, 1966, 11, 78-79. WELLS, M. B. Elements of combinatorial computing. New York: Pergamon Press, 1971. RECEIVED:
September 29, 1976