- Email: [email protected]
Frames of Reference in Quantity Estimations by Groups and Individuals
Organizational Behavior and Human Decision Processes Vol. 80, No. 2, November, pp. 103–117, 1999 Article ID obhd.1999.2848, available online at http://www.idealibrary.com on
Frames of Reference in Quantity Estimations by Groups and Individuals Patrick R. Laughlin, Bryan L. Bonner, Andrew G. Miner, and Peter J. Carnevale University of Illinois at Urbana–Champaign
The superiority of group performance over performance of the average individual is relatively greater on world knowledge tasks than on quantity estimation tasks. Previous research on quantity estimations has involved judgments without an explicit frame of reference. We propose that a frame of reference converts a quantity estimation into a world knowledge inference by embedding the estimation in a larger cognitive structure. Individuals first estimated 30 pairs of quantities, such as the length of the Ohio River and the length of the Arkansas River, given either 2 statements as a frame of reference (the Mississippi River is 2340 miles long; the Colorado River is 1450 miles long), 1 of these statements as a frame of reference, or no frame of reference. Then they made the same 30 pairs of estimations again as 3-person groups or as individuals under the same frame-of-reference conditions. As predicted, group estimations were more accurate than individual estimations, both group and individual estimations were more accurate with either a 2-statement or a 1-statement frame of reference than without a frame of reference, and the frame of reference improved group estimations relatively more than individual estimations. q 1999 Academic Press
In both their daily lives and their occupational activities people frequently make individual and group estimations of quantities, such as the length of the Ohio River in miles, the revenues of Merck in dollars, the proven oil reserves of Iraq in barrels, the number of native speakers of Spanish in the world, or the number of Rottweilers registered with the American Kennel Club. The relative accuracy of such estimations by groups and by individuals and possible ways to improve the accuracy of group and individual estimations are two important issues for both theory and application. Address correspondence and reprint requests to Patrick R. Laughlin, Department of Psychology, University of Illinois, 603 E. Daniel Street, Champaign, IL 61820. E-mail: [email protected]. uiuc.edu. 103 0749-5978/99 $30.00 Copyright q 1999 by Academic Press All rights of reproduction in any form reserved.
104
LAUGHLIN ET AL.
Laughlin (1980) proposed a continuum of group tasks anchored at one end by intellective tasks and anchored at the other end by judgmental tasks. Intellective tasks are problems or decisions for which there exists a demonstrably correct answer within a verbal or mathematical conceptual system, such as algebra problems, logic problems, and object transfer problems (e.g., Cannibals and Missionaries). Judgmental tasks are evaluative, behavioral, or aesthetic judgments for which there does not exist a demonstrably correct answer, such as aesthetic preferences, attitudinal judgments, and most jury decisions. Laughlin and Ellis (1986) proposed that demonstrability requires four conditions: (1) agreement on a verbal or mathematical conceptual system, (2) sufficient information for task solution (a system of two equations in two unknowns has a unique solution, but one equation in two unknowns does not), (3) the ability of the group member(s) who do not know the correct answer to recognize it when proposed, (4) the ability, motivation, and time of the group member(s) who know the correct answer to demonstrate it to the member(s) who do not know the correct answer. In a comprehensive review of group versus individual accuracy on judgments for which an objective criterion of accuracy exists (intellective tasks in the terminology of Laughlin, 1980), Hastie (1986) distinguished quantity estimation, problem-solving, and world knowledge tasks. Quantity estimation tasks require specification of a known or future value on some dimension, such as the present temperature of a room or the amount of precipitation next month. Problem-solving tasks require identification of the correct solution within some system of logic, science, or mathematics. World knowledge tasks include vocabulary, analogies, or facts of geography and history. Groups perform better than the average individual on each of these three types of tasks, and the superiority of groups over individuals is relatively greater on problem-solving and world knowledge tasks than on estimation tasks. Hastie (1986) proposed that the essential underlying basis of this greater superiority on problemsolving and world knowledge tasks than on estimation tasks is solution demonstrability. Hence both Laughlin and Ellis (1986) and Hastie (1986) propose that the underlying basis of the degree of superiority of groups over the average individual is the demonstrability of the proposed solution, either within a verbal, mathematical, or logical conceptual system or within a system of world knowledge. Estimation tasks typically involve simple judgments without an explicitly given frame of reference, such as “What is the length of the Ohio River in miles?” World knowledge tasks are necessarily embedded in larger conceptual systems. For example, vocabulary questions are embedded in the general structure of the language, analogy questions are embedded in relationships of the two corresponding domains, and the facts of geography are embedded with other geographical facts and relationships. Psychological cognitive systems correspond to these conceptual systems, so that demonstrability results from the correspondence between the conceptual system of a domain of knowledge and the cognitive systems of people. We propose that both group and individual accuracy on quantity estimations
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
105
may be improved by providing a frame of reference that embeds the estimation in a larger conceptual system. To illustrate, suppose a person is told that the Mississippi River is 2340 miles long and is asked to estimate the length of the Ohio River. This embeds the estimation of the length of the Ohio River in a larger cognitive system, structure, or schema of relevant world knowledge. This cognitive or knowledge system includes the geographical facts that the Mississippi runs from northern Minnesota to the Gulf of Mexico, the Ohio runs into the Mississippi near the bottom of Illinois, the Ohio is in the northeastern quarter of the United States and runs southwest, and hence the Ohio necessarily originates west of the Appalachian Mountains. Thus, the Ohio must be less than half of the length of the Mississippi. Or, consider estimation of the total revenues of EXXON in 1994, given the frame of reference that the total revenues of AMOCO in 1994 were 27 billion dollars. A person may know that Standard of New Jersey was much larger than Standard of Indiana after the Supreme Court mandated breakup of the Standard Oil Trust into multiple units, that Standard of New Jersey was subsequently renamed EXXON, and that Standard of Indiana was subsequently renamed AMOCO. Thus, the current revenues of EXXON should be considerably larger than the given revenues of AMOCO. The frame of reference converts an estimation into an inference within the relationships of a larger cognitive system and therefore should result in a more accurate estimation. Thus, a frame of reference should improve the estimations of both groups and individuals, and groups should exhibit their customary superiority over the average individual on world knowledge tasks. An important further question is whether individual or group accuracy will be increased relatively more by the explicit frame of reference. Recall that Hastie’s (1986) review indicates that group superiority over the average individual is relatively greater on world knowledge than on estimation tasks. The frame of reference should result in greater fulfillment of the four conditions of demonstrability of Laughlin and Ellis (1986), increasing the agreement on a conceptual system, increasing the information, enabling members who do not know a plausible estimation to recognize one, and enabling the group members who can make a plausible estimation to demonstrate the plausibility to the other members. Thus, the relative superiority of group over individual estimation should be greater with a frame of reference than without a frame of reference. If a single statement provides a frame of reference that converts an estimation into an inference within a system of world knowledge and hence improves the accuracy of estimation, will a second related statement further improve accuracy? For example, given the known length of the Mississippi River, will the additional knowledge of the length of the Colorado River further improve an estimation of the length of the Ohio River? We suspect that the additional statement will not appreciably improve performance over the frame of reference provided by the first statement. Although the second statement may provide further factual information within the world knowledge system, it would not seem to appreciably expand the world knowledge system and consequently improve the estimation.
106
LAUGHLIN ET AL.
At this point it is important to distinguish our meaning of a frame of reference from the large amount of research on framing effects in judgment. In a comprehensive recent review Levin, Schneider, and Gaeth (1998) distinguish three different types of valence framing effects: (1) the standard risky choice framing effects introduced by Tversky and Kahneman (1981); (2) attributes framing, which affects the evaluation of the characteristics of objects or events; and (3) goal framing, which affects the persuasiveness of a communication. We propose that, in contrast to these three types of valence framing effects, a frame of reference may convert the estimation of quantities to an inference in the larger and richer conceptual systems of world knowledge and their corresponding cognitive systems or representations in people. It is also important to realize that we are not proposing that frames of reference are normative standards of rational decision making and judgment. In introducing his review of behavioral decision making and judgment Dawes (1998) describes a basic characteristic of research in the field: What has characterized the field both historically and theoretically is the comparison of actual decision making with certain principles of rationality in decision making—for example, that increasing the number of options available to a decision maker should not increase the probability that a particular option from the more restricted set is chosen, or that the way in which identical choices are described (“framed”) should not affect choice. When actual decisions violate such principles systematically (not just as a result of unreliability or “error”), this deviation is termed an anomaly—if the people who violate these principles simultaneously accept them as one that they believe should govern their decision making. (p. 497)
Dawes’s review summarizes the important principles that have emerged from this research on judgment and decision making, such as understanding the anomaly of sunk costs or violation of the axiom of the independence of irrelevant alternatives. The current experiment is not set within this tradition of understanding the anomalies resulting from a comparison of descriptive behavior with normative principles. Rather, we propose that estimation of quantities may be improved by a frame of reference that converts the estimation to an inference within a larger and richer cognitive system of world knowledge. Accordingly, the following experiment compared groups and individuals on 30 pairs of related quantity estimations, such as the length of the Ohio River and the length of the Arkansas River. We used pairs of related quantity estimations rather than single estimations in order to increase the reliability of estimation within each of the 30 domains. Some groups and individuals were given two statements as a frame of reference, such as the length of the Mississippi River and the length of the Colorado River. Other groups and individuals were given one statement as a frame of reference, such as the length of the Mississippi River. Other groups or individuals were not given a frame of reference. All participants first made the 30 estimations as individuals, and then some made the same estimations a second time as a three-person cooperative group and others as individuals. This design allowed both a comparison of group versus individual estimations on the second administration and a comparison of individual group member first estimations and group second estimations. In summary, we proposed and tested the following three hypotheses:
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
107
HYPOTHESIS 1. Groups will make more accurate estimations than will individuals. HYPOTHESIS 2. Both groups and individuals will make more accurate estimations with a frame of reference than without a frame of reference. HYPOTHESIS 3. A frame of reference will improve group estimations relatively more than individual estimations.
METHOD
The participants were 343 students in 10 sections of an introductory social psychology course at the University of Illinois at Urbana–Champaign. They received course credit for participation. The experimental design was a 3 (frame of reference: two statements, one statement, no statements) 3 2 (administration: individual on first and individual on second, individual on first and group on second) factorial with repeated measures on the second factor. The experiment was run in a computer laboratory with 15 terminals. Each of the six experimental conditions was randomly assigned to five different 2h sessions. One or two students in each of the 10 introductory social psychology sections were randomly assigned to each of the 30 sessions. All participants within a given session made the estimations in the same one of the six experimental conditions (e.g., individual with two-statement frame of reference on both first and second administration of the estimations). This procedure was designed to give equal numbers of replications within each of the six experimental conditions. However, because of differential show-up rates and a mistake in running one condition for six sessions and one condition for four sessions, the resulting number of replications differed across the six experimental conditions. The Appendix gives the frames of reference and the questions. Conditions with two statements for a frame of reference were given both statements for each item. Conditions with one statement for a frame of reference were given the first statement for each item. Conditions with no frame of reference were not given either statement. Correct answers for each question are in parentheses. The source of the items is Famighetti (1996). The items were presented on an interactive computer system. The participants entered their answers in boxes, with the appropriate dimension such as millions of dollars given for each item. All participants first responded to the 30 pairs of items as individuals, with no time limit. They then responded a second time to the same pairs of items as individuals (181 persons) or threeperson cooperative groups (162 persons). Seventy-one persons responded as individuals on both administrations in the two-statement frame-of-reference condition, 56 in the one-statement frame-of-reference condition, and 54 in the no-frame-of-reference condition. Sixty-six persons responded first as individuals and then as a three-person cooperative group in the two-statement frameof-reference condition, 39 in the one-statement frame-of-reference condition, and 57 in the no-frame-of-reference condition. In the group conditions the three persons were instructed to discuss the items and come to a single group estimation. A randomly selected member was
108
LAUGHLIN ET AL.
designated to enter the group estimations. Discussion was completely free within the group, and no decision rule (e.g., unanimity, majority, averaging) was imposed or implied by the instructions. As with the previous individual administration, there were no time limits. Since the sessions were scheduled for 2 h, all individuals and groups had sufficient time to complete the second set of 60 estimations. Most of them required less than 1 h. RESULTS
Group versus Individual Estimations We first determined the absolute deviations of each response from the correct response on both the first (individual) administration and the second (individual or group) administration of the 60 estimates. We then computed the standard scores for each item on each administration. Hence the best performance on each item was indicated by the largest negative standard score and the poorest performance was indicated by the largest positive standard score. We then computed the mean of the three standard scores on each item for the three first-administration responses of the three people who would respond as a cooperative group on the second administration. We then summed over the 60 standard scores for the first-administration individuals or means of sets of three people. Similarly, we summed over the 60 standard scores for the secondadministration individuals or three-person groups. Hence the best performance is indicated by the largest negative sum of standard scores, and the poorest performance is indicated by the largest positive sum of standard scores. Although we conducted the analyses on this variable, the graphic representation is counterintuitive. Accordingly, we then constructed a more intuitive index of performance by subtracting the sum of scores from 20, so that a higher number indicates better estimation. Figure 1 gives the resulting means, with a higher mean indicating better performance. We conducted a 2 (administration: individual on Time 1 and individual on Time 2 versus individual on Time 1 and group on Time 2) 3 3 (frame of reference: two statements, one statement, no statements) MANOVA for the Time 1 and Time 2 scores. The main effect of administration was significant, F(2, 228) 5 7.72, Wilks’s L 5 .937, p , .001. The main effect of frame of reference was also significant, F(4, 456) 5 16.90, Wilks’s L 5 .759, p , .001. The interaction was also significant, F(4, 456) 5 2.57, Wilks’s L 5 .956, p , .05. (The degrees of freedom for Time 1 are based on the 181 individuals who subsequently responded as individuals on the second administration and the mean of the three individuals who subsequently responded as one of the 54 groups in the group condition. The degrees of freedom for Time 2 are based on the 181 persons who responded as individuals on the second administration and the 54 groups who responded as groups on the second administration.) Univariate analyses on the first individual administration (Time 1) indicated a nonsignificant difference between those who subsequently made the estimates as individuals or as three-person groups, F(1, 229) 5 1.19, indicating
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
109
FIG. 1. Index of performance. The index is 20 minus the sum of standard scores over 60 items, so that a higher score indicates better performance. II2 5 individual time 1 estimation and individual time 2 estimation with two-statement frame of reference, II1 5 individual time 1 estimation and individual time 2 estimation with one-statement frame of reference, II0 5 individual time 1 estimation and individual time 2 estimation with no frame of reference, IG2 5 individual time 1 estimation and group time 2 estimation with two-statement frame of reference, IG1 5 individual time 1 estimation and group time 2 estimation with one-statement frame of reference, IG0 5 individual time 1 estimation and group time 2 estimation with no frame of reference. Time 1 estimation is on left and time 2 on right of each pair of estimations.
the equivalence of the initial ability of the individuals in the two conditions. (The error term is for first administration only.) Univariate analyses on the second administration (Time 2) indicated significantly better performance for groups than for individuals, F(1, 229) 5 7.15, p , .01, standardized effect size (Abelson, 1995, p. 46; subsequently abbreviated SES) 5 .36. (The error term is for the second administration only.) These results support Hypothesis 1. Univariate analyses on both the first and the second administration indicated significant effects for frame of reference, F(2, 229) 5 32.56, p , .001, and F(2, 229) 5 21.52, p , .001, respectively. Tukey comparisons (at p , .05) indicated significantly better performance for both a frame of reference of two statements and a frame of reference of one statement than for no frame of reference for the first administration, SES 5 1.32, 1.28, respectively. Similarly, there was significantly better performance for both a frame of reference of two statements and a frame of reference of one statement than for no frame of reference for the second administration, SES 5 1.26, 1.28, respectively. The difference between frames of reference of two statements and one statement was not significant on either administration. These results support Hypothesis 2. Separate repeated measures analyses of variance for the three frame-ofreference conditions for the IG2, IG1, and IG0 (individual first, group second) conditions indicated improvement from the mean group member first individual estimations to the second group estimations for each of the two-statement frame-of-reference, one-statement frame-of-reference, and no-frame-of-reference conditions, F(1, 21) 5 7.40, p , .02, SES 5 .82; F(1, 12) 5 13.28, p , .02, SES 5 1.43; and F(1, 18) 5 7.10, p , .02, SES 5 .78, respectively. In contrast,
110
LAUGHLIN ET AL.
repeated measures analyses of variance for the three frame-of-reference conditions for the II2, II1, and II0 (individual first, individual second) conditions indicated nonsignificant differences for each of the two-statement frame-ofreference, one-statement frame-of-reference, and no-frame-of-reference conditions, F(1, 70) 5 2.83, F(1, 55) , 1.00, and F(1, 53) 5 2.66, respectively. Thus, there was improvement for a given frame-of-reference condition (two statements, one statement, no statement) from the mean of the first administration members of the group to the second group administration, but there was not significant improvement for a given frame-of-reference condition from individual estimations on the first administration to individual estimations on the second administration. These results support Hypothesis 3.
Group versus Group Member Estimations We next considered the relationship between individual ability on the first estimations and group performance on the second estimations. We ranked the three group members as best, second best, and third best on the first administration. For convenience of exposition we refer to the three members as first, second, and third, and abbreviate them 1, 2, 3, respectively. We then conducted a 3 (frame of reference) 3 3 (ability) ANOVA on each of the 60 estimations. This resulted in four patterns of ability for the three group members: (1) the three members did not differ significantly from each other (1 5 2 5 3) on 28 items; (2) the first and third members differed significantly but the first and second, and second and third, did not (1 5 2, 1 . 3, 2 5 3) on 3 items; (3) the first and second members did not differ significantly and both differed significantly from the third member ((1 5 2) . 3) on 25 items; (4) the first member differed significantly from the second and the second from the third (1 . 2 . 3) on 4 items. Aggregated over the 60 items the first and second members differed significantly on only 4 items, the first and third members differed significantly on 32 items, and the second and third members differed significantly on 29 items. We then conducted a repeated measures analysis for each of the first members versus the group, the second members versus the group, and the third members versus the group, on each of the 60 items. Table 1 gives the resulting comparisons of the first members versus the group, the second members versus the group, and the third members versus the group for the four patterns of first, second, and third members. Summed over the four patterns (1 5 2 5 3, etc.), the groups performed below the level of the first member on 45 of the 60 items, at the level of the second member on 51 items, and above the level of the third member on 22 items. The group performed at the level of the first member on 13 items, but the first member did not differ significantly from either the second or the third member on 6 of these items, and did not differ significantly from the second member on the other 7. Conversely, the group performed at the level of the third member on 38 items, but the first and second members did not differ from the third in 28 of these cases.
111
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
TABLE 1 Patterns of Group versus Member Performance for Patterns of Member Ability Ranks Pattern of Group Performance Pattern of member ranks 1 5 2 5 3 (28 items)
1 5 2, 1 . 3, 2 5 3 (3 items) (1 5 2) . 3 (25 items)
1 . 2 . 3 (4 items)
G , Mi G 5 Mi G . Mi
G vs 1 G G G G G G G G G G G G G G G G
5 , , . , , , , 5 5 , , , , , , 45 13 2
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
G vs 2 G G G G G G G G G G G G G G G G
5 5 . . , 5 5 5 5 5 5 . , . 5 5 3 51 6
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
G vs 3
n
5 5 5 5 5 . 5 . . 5 5 . 5 . . 5 0 38 22
6 16 2 2 2 2 1 14 3 4 3 1 1 1 1 1
G G G G G G G G G G G G G G G G
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
Note. 1 5 best member, 2 5 second-best member, 3 5 third-best member, G 5 group.
DISCUSSION
Frames of Reference in Quantity Estimations by Groups and Individuals Supporting our three hypotheses, groups made more accurate estimations than did individuals, the frame of reference improved both group and individual estimations, and the frame of reference improved group estimations relatively more than individual estimations. Although there has been a large amount of research on group versus individual estimations, to our knowledge none of these estimations have been set within the context of an explicitly given frame of reference. This explicit frame of reference embeds the estimation within a cognitive structure and converts the estimation into an inference in a richer world knowledge task. This increases the demonstrability of the proposed solution (Hastie, 1986; Laughlin & Ellis, 1986) and hence improves performance for both groups and individuals. Consistent with the research summarized by Hastie (1986) indicating that the relative superiority of groups over the average individual is greater on world knowledge tasks than on estimation tasks, the frame of reference improved group performance relatively more than individual performance. The patterns of differences between the three group members on the first estimations and the comparisons of group estimations with the previous estimations of the best, second-best, and third-best group members (Table 1) provide
112
LAUGHLIN ET AL.
some understanding of why group estimations were more accurate than individual estimations. The best member differed significantly from the second-best member on only 4 of the 60 estimations, and from the third member on 32 of the 60 estimations. The second-best member differed significantly from the third-best member on 29 of the 60 estimations. The groups were most likely to perform below the level of their best member, at the level of their secondbest member, and better than their third-best member. Rather than performing at the level of their best member, the groups performed better than their worst member. Hence the group estimations were more accurate than the mean estimations of the group members. In contrast to these relatively small differences between the best, secondbest, and third-best group members on estimation tasks, group members differ strongly from each other on world knowledge tasks (see Hastie, 1986, Table 3, for a review). Thus, a further possible explanation of why group superiority over the average individual was greater on world knowledge tasks than on estimation tasks in the previous literature is that group members differ more from each other on world knowledge than on estimation tasks. People should differ more from each other on knowledge of larger conceptual systems than on parts of the system. All of these estimations involved matters of known fact that are available in reference books, atlases, census data, archives, production records, annual reports, and, increasingly, interconnected links on the Internet. Why provide frames of reference to estimate matters of obtainable fact? First, action is sometimes necessary on the basis of estimation when the appropriate data sources are not conveniently available but a frame of reference may be. Second, time is frequently crucial, as for traders in international capital, currency, and commodities markets. Third, a frame of reference may improve estimations of the success and necessary resources of proposed activities, such as knowing the time to conduct an audit in a comparable firm in the industry. Fourth, estimation as an inference from a frame of reference within a cognitive system of world knowledge may develop estimation skills and a desire to further investigate and understand the system of world knowledge. Future Research Both one-statement and two-statement frames of reference improved both group and individual quantity estimations, but there was no significant difference between one statement and two statements. Within the limits of estimation domain and participants of this experiment, a single statement was sufficient to embed the estimation in a larger system of world knowledge and the second statement did not appreciably improve estimation within an expanded system of world knowledge. This suggests at least two important issues for future research on frames of reference. First, will frames of reference have different effects for experts and novices in a domain, in contrast to the current college students, who would generally be somewhere between complete novices and accomplished experts in their world knowledge in the various domains of
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
113
estimation? Second, will frames of reference progressively increase the difference between group and individual estimations with increasing group size beyond the three-person groups of the current experiment? Frames of Reference in Organizations The results suggest that frames of reference may improve both group and individual estimations in widely varying organizations. Real estate appraisers should make better estimations of selling prices if they know the recent sales price of a comparable house in the neighborhood. Machinists should make better estimations of the time to make a part if they know the time for similar parts. Art dealers should make better judgments of the value of a Monet if they know the value of a Renoir. Analysts should make better estimations of the earnings of a firm if they know the earnings of another firm in the industry. Hiring committees should make better estimations of the potential productivity of a prospective assistant professor if they know the productivity of the person as a graduate student. Congress should make a better estimate of future Medicare costs if they know current costs and the size of population cohorts. In all of these and many other situations the frame of reference converts the estimation to an inference within a cognitive system of world knowledge, and may consequently improve the estimation. APPENDIX: FRAMES OF REFERENCE AND QUESTIONS
1. 13,200 people died in falling accidents in the US in 1994. 4,000 people died in drowning accidents in the US in 1994. A. How many people died in motor vehicle accidents in the US in 1994? (43,000) B. How many people died in firearms accidents in the US in 1994? (1,500) 2. The total revenues in 1994 of Chrysler were 52 billion dollars. The total revenues in 1994 of General Motors were 155 billion dollars. A. What were the total revenues of the Ford Motor Company in billions of dollars in 1994? (128) B. What were the total revenues of PACCAR in billions of dollars in 1994? (5) 3. The total revenues in 1994 of Xerox were 18 billion dollars. The total revenues in 1994 of MMM were 15 billion dollars. A. What were the total revenues of Compaq Computer in billions of dollars in 1994? (11) B. What were the total revenues of Hewlett–Packard in billions of dollars in 1994? (25) 4. The total revenues in 1994 of MOBIL were 60 billion dollars. The total revenues in 1994 of AMOCO were 27 billion dollars. A. What were the total revenues of EXXON in billions of dollars in 1994? (101)
114
5.
6.
7.
8.
9.
10.
11.
12.
LAUGHLIN ET AL.
B. What were the total revenues of CHEVRON in billions of dollars in 1994? (31) The total revenues in 1994 of Gillette were 6 billion dollars. The total revenues in 1994 of Procter & Gamble were 30 billion dollars. A. What were the total revenues of Johnson & Johnson in billions of dollars in 1994? (15) B. What were the total revenues of Merck in billions of dollars in 1994? (15) Tourists from Japan spent 17 billion dollars in the US in 1994. Tourists from the United Kingdom spent 8 billion dollars in the US in 1994. A. How many billion dollars did tourists from France spend in the US in 1994? (3) B. How many billion dollars did tourists from Canada spend in the US in 1994? (7) The total value of farm marketings in Texas in 1994 was 15 billion dollars. The total value of farm marketings in Illinois in 1994 was 8 billion dollars. A. What was the total value of farm marketings in billions of dollars in California in 1994? (20) B. What was the total value of farm marketings in billions of dollars in Iowa in 1994? (10) Indiana produced 858,240,000 bushels of corn in 1994. Illinois produced 1,786,200,000 bushels of corn in 1994. A. How many bushels of corn did Nebraska produce in 1994? (1,153,700,100) B. How many bushels of corn did Iowa produce in 1994? (1,930,400,000) Kansas produced 433,200,000 bushels of wheat in 1994. North Dakota produced 356,404,000 bushels of wheat in 1994. A. How many bushels of wheat did Montana produce in 1994? (170,590,000) B. How many bushels of wheat did Washington State produce in 1994? (134,000,000) 861,000 people were employed as lawyers and judges in the US in 1994. 838,000 people were employed as college and university teachers in the US in 1994. A. How many engineers were employed in the US in 1994? (1,866,000) B. How many elementary and high school teachers were employed in the US in 1994? (4,330,000) The population of Connecticut in 1780 was 206,000. The population of Virginia in 1780 was 538,000. A. What was the population of Pennsylvania in 1780? (327,000) B. What was the population of Delaware in 1780? (45,000) The population of California in 1994 was 31 million. The population of Texas in 1994 was 18 million. A. What was the population of Florida in millions in 1994? (14)
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
13.
14.
15.
16.
17.
18.
19.
20.
21.
115
B. What was the population of Ohio in millions in 1994? (11) In 1993 there were 15,229 miles of electrical railroad in the Far East. In 1993 there were 6,866 miles of electrical railroad in Africa. A. How many miles of electrical railroad were there in Europe in 1993? (91,814) B. How miles of electrical railroad were there in South America in 1993? (22,624) The population density of Missouri in 1994 was 74 people per square mile. The population density of Massachusetts in 1994 was 767 people per square mile. A. What was the population density in people per square mile of Illinois in 1994? (206) B. What was the population density in people per square mile of New Jersey in 1994? (1,042) The land area of California is 156,000 square miles. The land area of Kansas is 82,000 square miles. A. What is the land area of Illinois in square miles? (56,000) B. What is the land area of Nebraska in square miles? (77,000) There are 11 Indian reservations in Wisconsin. There are 3 Indian reservations in Nebraska. A. How many Indian reservations are there in Arizona? (23) B. How many Indian reservations are there in South Dakota? (9) The area of the Arctic Ocean is 5 million square miles. The area of the Pacific Ocean is 64 million square miles. A. What is the area of the Atlantic Ocean in millions of square miles? (33) B. What is the area of the Indian Ocean in millions of square miles? (28) The Mississippi River is 2,340 miles long. The Colorado River is 1,450 miles long. A. What is the length of the Ohio River in miles? (981) B. What is the length of the Arkansas River in miles? (1,149) Iraq has 100 billion barrels of proven oil reserves. The US has 23 billion barrels of proven oil reserves. A. How many billion barrels of proven oil reserves does Saudi Arabia have? (261) B. How many billion barrels of proven oil reserves does Mexico have? (51) The value of airplane exports from the US in 1994 was 19 billion dollars. The value of electrical machinery exports from the US in 1994 was 44 billion dollars. A. What was the value of office machine exports from the US in billions of dollars in 1994? (31) B. What was the value of power generator exports from the US in billions of dollars in 1994? (20) The value of petroleum imports to the US in 1994 was 39 billion dollars.
116
22.
23.
24.
25.
26.
27.
LAUGHLIN ET AL.
The value of toys, games, and sporting good imports to the US in 1994 was 11 billion dollars. A. What was the value of coffee imports to the US in billions of dollars in 1994? (2) B. What was the value of footwear imports to the US in billions of dollars in 1994? (12) There were 1,893,078 elementary and high school students in Illinois in 1994. There were 875,639 elementary and high school students in Missouri in 1994. A. How many elementary and high school students were there in California in 1994? (5,328,555) B. How many elementary and high school students were there in Florida in 1994? (2,040,763) Consumers spent 50 billion dollars on tobacco products in the US in 1994. Consumers spent 430 billion dollars on food for home consumption in the US in 1994. A. How many billion dollars did US residents spend dining out in 1994? (220) B. How many billion dollars did US residents spend on health products in 1994? (69) In 1992 the per capita income of Connecticut was 26,979 dollars. In 1992 the per capita income of Illinois was 21,608 dollars. A. What was the per capita income of Oregon in dollars in 1992? (18,202) B. What was the per capita income of Mississippi in dollars in 1992? (14,088) Consumers spent 45 billion dollars on used automobiles and trucks in the US in 1994. Consumers spent 21 billion dollars on auto and truck insurance in the US in 1994. A. How many billion dollars did consumers in the US spend on new cars and trucks in the US in 1994? (93) B. How many billion dollars did consumers in the US spend on gasoline and oil in 1994? (106) McDonalds spent 425 million dollars on advertising in the US in 1994. Ford Motor Company spent 920 million dollars on advertising in the US in 1994. A. How many million dollars did AT&T spend on advertising in the US in 1994? (700) B. How many million dollars did Anheuser–Busch spend on advertising in the US in 1994? (306) 9 billion dollars was spent on advertising automobiles in the US in 1994. 4 billion dollars was spent on advertising food in the US in 1994. A. How much in billions was spent on advertising apparel and shoes in the US in 1994? (1)
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
117
B. How much in billions was spent on advertising tobacco and makeup in the US in 1994? (3) 28. 74 million people flew on United Airlines in 1994. 46 million people flew on Northwest Airlines in 1994. A. How many million people flew on Delta Airlines in 1994? (89) B. How many million people flew on American Airlines in 1994? (81) 29. There were 78,999 German Shepherds registered with the American Kennel Club in 1994. There were 126,393 Labrador Retrievers registered with the American Kennel Club in 1994. A. How many Rottweilers were registered with the American Kennel Club in 1994? (102,596) B. How many Golden Retrievers were registered with the American Kennel Club in 1994? (64,232) 30. There were 326 million native speakers of English in 1994. There were 169 million native speakers of Russian in 1994. A. How many million native speakers of German were there in 1994? (98) B. How many million native speakers of Spanish were there in 1994? (339) Note. Participants in conditions with two statements for a frame of reference were given both statements on each item. Participants in conditions with one statement for a frame of reference were given the first statement for each item. Participants in conditions with no frame of reference were not given either statement. Correct answers are in parentheses for each question. REFERENCES Abelson, R. P. (1995). Statistics as principled argument. Hillsboro, NJ: Erlbaum. Dawes, R. M. (1998). Behavioral decision making and judgment. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The handbook of social psychology (Vol. 1, pp. 497–548). New York: McGraw–Hill. Famighetti, R. (Ed.). (1996). World almanac and book of facts, 1996. Mahwah, NJ: World Almanac Books. Hastie, R. (1986). Review essay: Experimental evidence on group accuracy. In G. Owen & B. Grofman (Eds.), Information pooling and group accuracy (pp. 129–157). Westport, CT: JAI Press. Laughlin, P. R. (1980). Social combination processes of cooperative problem-solving groups on verbal intellective tasks. In M. Fishbein (Ed.), Progress in social psychology (pp. 127–155). Hillsdale, NJ: Erlbaum. Laughlin, P. R., & Ellis, A. L. (1986). Demonstrability and social combination processes on mathematical intellective tasks. Journal of Experimental Social Psychology, 22, 177–189. Levin, I. P., Schneider, S. I., & Gaeth, G. J. (1998). All frames are not created equal: A typology and critical analysis of framing effects. Organizational Behavior and Human Decision Processes, 76, 149–188. Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211, 453–458. Received February 18, 1999
Frames of Reference in Quantity Estimations by Groups and Individuals Patrick R. Laughlin, Bryan L. Bonner, Andrew G. Miner, and Peter J. Carnevale University of Illinois at Urbana–Champaign
The superiority of group performance over performance of the average individual is relatively greater on world knowledge tasks than on quantity estimation tasks. Previous research on quantity estimations has involved judgments without an explicit frame of reference. We propose that a frame of reference converts a quantity estimation into a world knowledge inference by embedding the estimation in a larger cognitive structure. Individuals first estimated 30 pairs of quantities, such as the length of the Ohio River and the length of the Arkansas River, given either 2 statements as a frame of reference (the Mississippi River is 2340 miles long; the Colorado River is 1450 miles long), 1 of these statements as a frame of reference, or no frame of reference. Then they made the same 30 pairs of estimations again as 3-person groups or as individuals under the same frame-of-reference conditions. As predicted, group estimations were more accurate than individual estimations, both group and individual estimations were more accurate with either a 2-statement or a 1-statement frame of reference than without a frame of reference, and the frame of reference improved group estimations relatively more than individual estimations. q 1999 Academic Press
In both their daily lives and their occupational activities people frequently make individual and group estimations of quantities, such as the length of the Ohio River in miles, the revenues of Merck in dollars, the proven oil reserves of Iraq in barrels, the number of native speakers of Spanish in the world, or the number of Rottweilers registered with the American Kennel Club. The relative accuracy of such estimations by groups and by individuals and possible ways to improve the accuracy of group and individual estimations are two important issues for both theory and application. Address correspondence and reprint requests to Patrick R. Laughlin, Department of Psychology, University of Illinois, 603 E. Daniel Street, Champaign, IL 61820. E-mail: [email protected]. uiuc.edu. 103 0749-5978/99 $30.00 Copyright q 1999 by Academic Press All rights of reproduction in any form reserved.
104
LAUGHLIN ET AL.
Laughlin (1980) proposed a continuum of group tasks anchored at one end by intellective tasks and anchored at the other end by judgmental tasks. Intellective tasks are problems or decisions for which there exists a demonstrably correct answer within a verbal or mathematical conceptual system, such as algebra problems, logic problems, and object transfer problems (e.g., Cannibals and Missionaries). Judgmental tasks are evaluative, behavioral, or aesthetic judgments for which there does not exist a demonstrably correct answer, such as aesthetic preferences, attitudinal judgments, and most jury decisions. Laughlin and Ellis (1986) proposed that demonstrability requires four conditions: (1) agreement on a verbal or mathematical conceptual system, (2) sufficient information for task solution (a system of two equations in two unknowns has a unique solution, but one equation in two unknowns does not), (3) the ability of the group member(s) who do not know the correct answer to recognize it when proposed, (4) the ability, motivation, and time of the group member(s) who know the correct answer to demonstrate it to the member(s) who do not know the correct answer. In a comprehensive review of group versus individual accuracy on judgments for which an objective criterion of accuracy exists (intellective tasks in the terminology of Laughlin, 1980), Hastie (1986) distinguished quantity estimation, problem-solving, and world knowledge tasks. Quantity estimation tasks require specification of a known or future value on some dimension, such as the present temperature of a room or the amount of precipitation next month. Problem-solving tasks require identification of the correct solution within some system of logic, science, or mathematics. World knowledge tasks include vocabulary, analogies, or facts of geography and history. Groups perform better than the average individual on each of these three types of tasks, and the superiority of groups over individuals is relatively greater on problem-solving and world knowledge tasks than on estimation tasks. Hastie (1986) proposed that the essential underlying basis of this greater superiority on problemsolving and world knowledge tasks than on estimation tasks is solution demonstrability. Hence both Laughlin and Ellis (1986) and Hastie (1986) propose that the underlying basis of the degree of superiority of groups over the average individual is the demonstrability of the proposed solution, either within a verbal, mathematical, or logical conceptual system or within a system of world knowledge. Estimation tasks typically involve simple judgments without an explicitly given frame of reference, such as “What is the length of the Ohio River in miles?” World knowledge tasks are necessarily embedded in larger conceptual systems. For example, vocabulary questions are embedded in the general structure of the language, analogy questions are embedded in relationships of the two corresponding domains, and the facts of geography are embedded with other geographical facts and relationships. Psychological cognitive systems correspond to these conceptual systems, so that demonstrability results from the correspondence between the conceptual system of a domain of knowledge and the cognitive systems of people. We propose that both group and individual accuracy on quantity estimations
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
105
may be improved by providing a frame of reference that embeds the estimation in a larger conceptual system. To illustrate, suppose a person is told that the Mississippi River is 2340 miles long and is asked to estimate the length of the Ohio River. This embeds the estimation of the length of the Ohio River in a larger cognitive system, structure, or schema of relevant world knowledge. This cognitive or knowledge system includes the geographical facts that the Mississippi runs from northern Minnesota to the Gulf of Mexico, the Ohio runs into the Mississippi near the bottom of Illinois, the Ohio is in the northeastern quarter of the United States and runs southwest, and hence the Ohio necessarily originates west of the Appalachian Mountains. Thus, the Ohio must be less than half of the length of the Mississippi. Or, consider estimation of the total revenues of EXXON in 1994, given the frame of reference that the total revenues of AMOCO in 1994 were 27 billion dollars. A person may know that Standard of New Jersey was much larger than Standard of Indiana after the Supreme Court mandated breakup of the Standard Oil Trust into multiple units, that Standard of New Jersey was subsequently renamed EXXON, and that Standard of Indiana was subsequently renamed AMOCO. Thus, the current revenues of EXXON should be considerably larger than the given revenues of AMOCO. The frame of reference converts an estimation into an inference within the relationships of a larger cognitive system and therefore should result in a more accurate estimation. Thus, a frame of reference should improve the estimations of both groups and individuals, and groups should exhibit their customary superiority over the average individual on world knowledge tasks. An important further question is whether individual or group accuracy will be increased relatively more by the explicit frame of reference. Recall that Hastie’s (1986) review indicates that group superiority over the average individual is relatively greater on world knowledge than on estimation tasks. The frame of reference should result in greater fulfillment of the four conditions of demonstrability of Laughlin and Ellis (1986), increasing the agreement on a conceptual system, increasing the information, enabling members who do not know a plausible estimation to recognize one, and enabling the group members who can make a plausible estimation to demonstrate the plausibility to the other members. Thus, the relative superiority of group over individual estimation should be greater with a frame of reference than without a frame of reference. If a single statement provides a frame of reference that converts an estimation into an inference within a system of world knowledge and hence improves the accuracy of estimation, will a second related statement further improve accuracy? For example, given the known length of the Mississippi River, will the additional knowledge of the length of the Colorado River further improve an estimation of the length of the Ohio River? We suspect that the additional statement will not appreciably improve performance over the frame of reference provided by the first statement. Although the second statement may provide further factual information within the world knowledge system, it would not seem to appreciably expand the world knowledge system and consequently improve the estimation.
106
LAUGHLIN ET AL.
At this point it is important to distinguish our meaning of a frame of reference from the large amount of research on framing effects in judgment. In a comprehensive recent review Levin, Schneider, and Gaeth (1998) distinguish three different types of valence framing effects: (1) the standard risky choice framing effects introduced by Tversky and Kahneman (1981); (2) attributes framing, which affects the evaluation of the characteristics of objects or events; and (3) goal framing, which affects the persuasiveness of a communication. We propose that, in contrast to these three types of valence framing effects, a frame of reference may convert the estimation of quantities to an inference in the larger and richer conceptual systems of world knowledge and their corresponding cognitive systems or representations in people. It is also important to realize that we are not proposing that frames of reference are normative standards of rational decision making and judgment. In introducing his review of behavioral decision making and judgment Dawes (1998) describes a basic characteristic of research in the field: What has characterized the field both historically and theoretically is the comparison of actual decision making with certain principles of rationality in decision making—for example, that increasing the number of options available to a decision maker should not increase the probability that a particular option from the more restricted set is chosen, or that the way in which identical choices are described (“framed”) should not affect choice. When actual decisions violate such principles systematically (not just as a result of unreliability or “error”), this deviation is termed an anomaly—if the people who violate these principles simultaneously accept them as one that they believe should govern their decision making. (p. 497)
Dawes’s review summarizes the important principles that have emerged from this research on judgment and decision making, such as understanding the anomaly of sunk costs or violation of the axiom of the independence of irrelevant alternatives. The current experiment is not set within this tradition of understanding the anomalies resulting from a comparison of descriptive behavior with normative principles. Rather, we propose that estimation of quantities may be improved by a frame of reference that converts the estimation to an inference within a larger and richer cognitive system of world knowledge. Accordingly, the following experiment compared groups and individuals on 30 pairs of related quantity estimations, such as the length of the Ohio River and the length of the Arkansas River. We used pairs of related quantity estimations rather than single estimations in order to increase the reliability of estimation within each of the 30 domains. Some groups and individuals were given two statements as a frame of reference, such as the length of the Mississippi River and the length of the Colorado River. Other groups and individuals were given one statement as a frame of reference, such as the length of the Mississippi River. Other groups or individuals were not given a frame of reference. All participants first made the 30 estimations as individuals, and then some made the same estimations a second time as a three-person cooperative group and others as individuals. This design allowed both a comparison of group versus individual estimations on the second administration and a comparison of individual group member first estimations and group second estimations. In summary, we proposed and tested the following three hypotheses:
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
107
HYPOTHESIS 1. Groups will make more accurate estimations than will individuals. HYPOTHESIS 2. Both groups and individuals will make more accurate estimations with a frame of reference than without a frame of reference. HYPOTHESIS 3. A frame of reference will improve group estimations relatively more than individual estimations.
METHOD
The participants were 343 students in 10 sections of an introductory social psychology course at the University of Illinois at Urbana–Champaign. They received course credit for participation. The experimental design was a 3 (frame of reference: two statements, one statement, no statements) 3 2 (administration: individual on first and individual on second, individual on first and group on second) factorial with repeated measures on the second factor. The experiment was run in a computer laboratory with 15 terminals. Each of the six experimental conditions was randomly assigned to five different 2h sessions. One or two students in each of the 10 introductory social psychology sections were randomly assigned to each of the 30 sessions. All participants within a given session made the estimations in the same one of the six experimental conditions (e.g., individual with two-statement frame of reference on both first and second administration of the estimations). This procedure was designed to give equal numbers of replications within each of the six experimental conditions. However, because of differential show-up rates and a mistake in running one condition for six sessions and one condition for four sessions, the resulting number of replications differed across the six experimental conditions. The Appendix gives the frames of reference and the questions. Conditions with two statements for a frame of reference were given both statements for each item. Conditions with one statement for a frame of reference were given the first statement for each item. Conditions with no frame of reference were not given either statement. Correct answers for each question are in parentheses. The source of the items is Famighetti (1996). The items were presented on an interactive computer system. The participants entered their answers in boxes, with the appropriate dimension such as millions of dollars given for each item. All participants first responded to the 30 pairs of items as individuals, with no time limit. They then responded a second time to the same pairs of items as individuals (181 persons) or threeperson cooperative groups (162 persons). Seventy-one persons responded as individuals on both administrations in the two-statement frame-of-reference condition, 56 in the one-statement frame-of-reference condition, and 54 in the no-frame-of-reference condition. Sixty-six persons responded first as individuals and then as a three-person cooperative group in the two-statement frameof-reference condition, 39 in the one-statement frame-of-reference condition, and 57 in the no-frame-of-reference condition. In the group conditions the three persons were instructed to discuss the items and come to a single group estimation. A randomly selected member was
108
LAUGHLIN ET AL.
designated to enter the group estimations. Discussion was completely free within the group, and no decision rule (e.g., unanimity, majority, averaging) was imposed or implied by the instructions. As with the previous individual administration, there were no time limits. Since the sessions were scheduled for 2 h, all individuals and groups had sufficient time to complete the second set of 60 estimations. Most of them required less than 1 h. RESULTS
Group versus Individual Estimations We first determined the absolute deviations of each response from the correct response on both the first (individual) administration and the second (individual or group) administration of the 60 estimates. We then computed the standard scores for each item on each administration. Hence the best performance on each item was indicated by the largest negative standard score and the poorest performance was indicated by the largest positive standard score. We then computed the mean of the three standard scores on each item for the three first-administration responses of the three people who would respond as a cooperative group on the second administration. We then summed over the 60 standard scores for the first-administration individuals or means of sets of three people. Similarly, we summed over the 60 standard scores for the secondadministration individuals or three-person groups. Hence the best performance is indicated by the largest negative sum of standard scores, and the poorest performance is indicated by the largest positive sum of standard scores. Although we conducted the analyses on this variable, the graphic representation is counterintuitive. Accordingly, we then constructed a more intuitive index of performance by subtracting the sum of scores from 20, so that a higher number indicates better estimation. Figure 1 gives the resulting means, with a higher mean indicating better performance. We conducted a 2 (administration: individual on Time 1 and individual on Time 2 versus individual on Time 1 and group on Time 2) 3 3 (frame of reference: two statements, one statement, no statements) MANOVA for the Time 1 and Time 2 scores. The main effect of administration was significant, F(2, 228) 5 7.72, Wilks’s L 5 .937, p , .001. The main effect of frame of reference was also significant, F(4, 456) 5 16.90, Wilks’s L 5 .759, p , .001. The interaction was also significant, F(4, 456) 5 2.57, Wilks’s L 5 .956, p , .05. (The degrees of freedom for Time 1 are based on the 181 individuals who subsequently responded as individuals on the second administration and the mean of the three individuals who subsequently responded as one of the 54 groups in the group condition. The degrees of freedom for Time 2 are based on the 181 persons who responded as individuals on the second administration and the 54 groups who responded as groups on the second administration.) Univariate analyses on the first individual administration (Time 1) indicated a nonsignificant difference between those who subsequently made the estimates as individuals or as three-person groups, F(1, 229) 5 1.19, indicating
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
109
FIG. 1. Index of performance. The index is 20 minus the sum of standard scores over 60 items, so that a higher score indicates better performance. II2 5 individual time 1 estimation and individual time 2 estimation with two-statement frame of reference, II1 5 individual time 1 estimation and individual time 2 estimation with one-statement frame of reference, II0 5 individual time 1 estimation and individual time 2 estimation with no frame of reference, IG2 5 individual time 1 estimation and group time 2 estimation with two-statement frame of reference, IG1 5 individual time 1 estimation and group time 2 estimation with one-statement frame of reference, IG0 5 individual time 1 estimation and group time 2 estimation with no frame of reference. Time 1 estimation is on left and time 2 on right of each pair of estimations.
the equivalence of the initial ability of the individuals in the two conditions. (The error term is for first administration only.) Univariate analyses on the second administration (Time 2) indicated significantly better performance for groups than for individuals, F(1, 229) 5 7.15, p , .01, standardized effect size (Abelson, 1995, p. 46; subsequently abbreviated SES) 5 .36. (The error term is for the second administration only.) These results support Hypothesis 1. Univariate analyses on both the first and the second administration indicated significant effects for frame of reference, F(2, 229) 5 32.56, p , .001, and F(2, 229) 5 21.52, p , .001, respectively. Tukey comparisons (at p , .05) indicated significantly better performance for both a frame of reference of two statements and a frame of reference of one statement than for no frame of reference for the first administration, SES 5 1.32, 1.28, respectively. Similarly, there was significantly better performance for both a frame of reference of two statements and a frame of reference of one statement than for no frame of reference for the second administration, SES 5 1.26, 1.28, respectively. The difference between frames of reference of two statements and one statement was not significant on either administration. These results support Hypothesis 2. Separate repeated measures analyses of variance for the three frame-ofreference conditions for the IG2, IG1, and IG0 (individual first, group second) conditions indicated improvement from the mean group member first individual estimations to the second group estimations for each of the two-statement frame-of-reference, one-statement frame-of-reference, and no-frame-of-reference conditions, F(1, 21) 5 7.40, p , .02, SES 5 .82; F(1, 12) 5 13.28, p , .02, SES 5 1.43; and F(1, 18) 5 7.10, p , .02, SES 5 .78, respectively. In contrast,
110
LAUGHLIN ET AL.
repeated measures analyses of variance for the three frame-of-reference conditions for the II2, II1, and II0 (individual first, individual second) conditions indicated nonsignificant differences for each of the two-statement frame-ofreference, one-statement frame-of-reference, and no-frame-of-reference conditions, F(1, 70) 5 2.83, F(1, 55) , 1.00, and F(1, 53) 5 2.66, respectively. Thus, there was improvement for a given frame-of-reference condition (two statements, one statement, no statement) from the mean of the first administration members of the group to the second group administration, but there was not significant improvement for a given frame-of-reference condition from individual estimations on the first administration to individual estimations on the second administration. These results support Hypothesis 3.
Group versus Group Member Estimations We next considered the relationship between individual ability on the first estimations and group performance on the second estimations. We ranked the three group members as best, second best, and third best on the first administration. For convenience of exposition we refer to the three members as first, second, and third, and abbreviate them 1, 2, 3, respectively. We then conducted a 3 (frame of reference) 3 3 (ability) ANOVA on each of the 60 estimations. This resulted in four patterns of ability for the three group members: (1) the three members did not differ significantly from each other (1 5 2 5 3) on 28 items; (2) the first and third members differed significantly but the first and second, and second and third, did not (1 5 2, 1 . 3, 2 5 3) on 3 items; (3) the first and second members did not differ significantly and both differed significantly from the third member ((1 5 2) . 3) on 25 items; (4) the first member differed significantly from the second and the second from the third (1 . 2 . 3) on 4 items. Aggregated over the 60 items the first and second members differed significantly on only 4 items, the first and third members differed significantly on 32 items, and the second and third members differed significantly on 29 items. We then conducted a repeated measures analysis for each of the first members versus the group, the second members versus the group, and the third members versus the group, on each of the 60 items. Table 1 gives the resulting comparisons of the first members versus the group, the second members versus the group, and the third members versus the group for the four patterns of first, second, and third members. Summed over the four patterns (1 5 2 5 3, etc.), the groups performed below the level of the first member on 45 of the 60 items, at the level of the second member on 51 items, and above the level of the third member on 22 items. The group performed at the level of the first member on 13 items, but the first member did not differ significantly from either the second or the third member on 6 of these items, and did not differ significantly from the second member on the other 7. Conversely, the group performed at the level of the third member on 38 items, but the first and second members did not differ from the third in 28 of these cases.
111
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
TABLE 1 Patterns of Group versus Member Performance for Patterns of Member Ability Ranks Pattern of Group Performance Pattern of member ranks 1 5 2 5 3 (28 items)
1 5 2, 1 . 3, 2 5 3 (3 items) (1 5 2) . 3 (25 items)
1 . 2 . 3 (4 items)
G , Mi G 5 Mi G . Mi
G vs 1 G G G G G G G G G G G G G G G G
5 , , . , , , , 5 5 , , , , , , 45 13 2
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
G vs 2 G G G G G G G G G G G G G G G G
5 5 . . , 5 5 5 5 5 5 . , . 5 5 3 51 6
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
G vs 3
n
5 5 5 5 5 . 5 . . 5 5 . 5 . . 5 0 38 22
6 16 2 2 2 2 1 14 3 4 3 1 1 1 1 1
G G G G G G G G G G G G G G G G
3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
Note. 1 5 best member, 2 5 second-best member, 3 5 third-best member, G 5 group.
DISCUSSION
Frames of Reference in Quantity Estimations by Groups and Individuals Supporting our three hypotheses, groups made more accurate estimations than did individuals, the frame of reference improved both group and individual estimations, and the frame of reference improved group estimations relatively more than individual estimations. Although there has been a large amount of research on group versus individual estimations, to our knowledge none of these estimations have been set within the context of an explicitly given frame of reference. This explicit frame of reference embeds the estimation within a cognitive structure and converts the estimation into an inference in a richer world knowledge task. This increases the demonstrability of the proposed solution (Hastie, 1986; Laughlin & Ellis, 1986) and hence improves performance for both groups and individuals. Consistent with the research summarized by Hastie (1986) indicating that the relative superiority of groups over the average individual is greater on world knowledge tasks than on estimation tasks, the frame of reference improved group performance relatively more than individual performance. The patterns of differences between the three group members on the first estimations and the comparisons of group estimations with the previous estimations of the best, second-best, and third-best group members (Table 1) provide
112
LAUGHLIN ET AL.
some understanding of why group estimations were more accurate than individual estimations. The best member differed significantly from the second-best member on only 4 of the 60 estimations, and from the third member on 32 of the 60 estimations. The second-best member differed significantly from the third-best member on 29 of the 60 estimations. The groups were most likely to perform below the level of their best member, at the level of their secondbest member, and better than their third-best member. Rather than performing at the level of their best member, the groups performed better than their worst member. Hence the group estimations were more accurate than the mean estimations of the group members. In contrast to these relatively small differences between the best, secondbest, and third-best group members on estimation tasks, group members differ strongly from each other on world knowledge tasks (see Hastie, 1986, Table 3, for a review). Thus, a further possible explanation of why group superiority over the average individual was greater on world knowledge tasks than on estimation tasks in the previous literature is that group members differ more from each other on world knowledge than on estimation tasks. People should differ more from each other on knowledge of larger conceptual systems than on parts of the system. All of these estimations involved matters of known fact that are available in reference books, atlases, census data, archives, production records, annual reports, and, increasingly, interconnected links on the Internet. Why provide frames of reference to estimate matters of obtainable fact? First, action is sometimes necessary on the basis of estimation when the appropriate data sources are not conveniently available but a frame of reference may be. Second, time is frequently crucial, as for traders in international capital, currency, and commodities markets. Third, a frame of reference may improve estimations of the success and necessary resources of proposed activities, such as knowing the time to conduct an audit in a comparable firm in the industry. Fourth, estimation as an inference from a frame of reference within a cognitive system of world knowledge may develop estimation skills and a desire to further investigate and understand the system of world knowledge. Future Research Both one-statement and two-statement frames of reference improved both group and individual quantity estimations, but there was no significant difference between one statement and two statements. Within the limits of estimation domain and participants of this experiment, a single statement was sufficient to embed the estimation in a larger system of world knowledge and the second statement did not appreciably improve estimation within an expanded system of world knowledge. This suggests at least two important issues for future research on frames of reference. First, will frames of reference have different effects for experts and novices in a domain, in contrast to the current college students, who would generally be somewhere between complete novices and accomplished experts in their world knowledge in the various domains of
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
113
estimation? Second, will frames of reference progressively increase the difference between group and individual estimations with increasing group size beyond the three-person groups of the current experiment? Frames of Reference in Organizations The results suggest that frames of reference may improve both group and individual estimations in widely varying organizations. Real estate appraisers should make better estimations of selling prices if they know the recent sales price of a comparable house in the neighborhood. Machinists should make better estimations of the time to make a part if they know the time for similar parts. Art dealers should make better judgments of the value of a Monet if they know the value of a Renoir. Analysts should make better estimations of the earnings of a firm if they know the earnings of another firm in the industry. Hiring committees should make better estimations of the potential productivity of a prospective assistant professor if they know the productivity of the person as a graduate student. Congress should make a better estimate of future Medicare costs if they know current costs and the size of population cohorts. In all of these and many other situations the frame of reference converts the estimation to an inference within a cognitive system of world knowledge, and may consequently improve the estimation. APPENDIX: FRAMES OF REFERENCE AND QUESTIONS
1. 13,200 people died in falling accidents in the US in 1994. 4,000 people died in drowning accidents in the US in 1994. A. How many people died in motor vehicle accidents in the US in 1994? (43,000) B. How many people died in firearms accidents in the US in 1994? (1,500) 2. The total revenues in 1994 of Chrysler were 52 billion dollars. The total revenues in 1994 of General Motors were 155 billion dollars. A. What were the total revenues of the Ford Motor Company in billions of dollars in 1994? (128) B. What were the total revenues of PACCAR in billions of dollars in 1994? (5) 3. The total revenues in 1994 of Xerox were 18 billion dollars. The total revenues in 1994 of MMM were 15 billion dollars. A. What were the total revenues of Compaq Computer in billions of dollars in 1994? (11) B. What were the total revenues of Hewlett–Packard in billions of dollars in 1994? (25) 4. The total revenues in 1994 of MOBIL were 60 billion dollars. The total revenues in 1994 of AMOCO were 27 billion dollars. A. What were the total revenues of EXXON in billions of dollars in 1994? (101)
114
5.
6.
7.
8.
9.
10.
11.
12.
LAUGHLIN ET AL.
B. What were the total revenues of CHEVRON in billions of dollars in 1994? (31) The total revenues in 1994 of Gillette were 6 billion dollars. The total revenues in 1994 of Procter & Gamble were 30 billion dollars. A. What were the total revenues of Johnson & Johnson in billions of dollars in 1994? (15) B. What were the total revenues of Merck in billions of dollars in 1994? (15) Tourists from Japan spent 17 billion dollars in the US in 1994. Tourists from the United Kingdom spent 8 billion dollars in the US in 1994. A. How many billion dollars did tourists from France spend in the US in 1994? (3) B. How many billion dollars did tourists from Canada spend in the US in 1994? (7) The total value of farm marketings in Texas in 1994 was 15 billion dollars. The total value of farm marketings in Illinois in 1994 was 8 billion dollars. A. What was the total value of farm marketings in billions of dollars in California in 1994? (20) B. What was the total value of farm marketings in billions of dollars in Iowa in 1994? (10) Indiana produced 858,240,000 bushels of corn in 1994. Illinois produced 1,786,200,000 bushels of corn in 1994. A. How many bushels of corn did Nebraska produce in 1994? (1,153,700,100) B. How many bushels of corn did Iowa produce in 1994? (1,930,400,000) Kansas produced 433,200,000 bushels of wheat in 1994. North Dakota produced 356,404,000 bushels of wheat in 1994. A. How many bushels of wheat did Montana produce in 1994? (170,590,000) B. How many bushels of wheat did Washington State produce in 1994? (134,000,000) 861,000 people were employed as lawyers and judges in the US in 1994. 838,000 people were employed as college and university teachers in the US in 1994. A. How many engineers were employed in the US in 1994? (1,866,000) B. How many elementary and high school teachers were employed in the US in 1994? (4,330,000) The population of Connecticut in 1780 was 206,000. The population of Virginia in 1780 was 538,000. A. What was the population of Pennsylvania in 1780? (327,000) B. What was the population of Delaware in 1780? (45,000) The population of California in 1994 was 31 million. The population of Texas in 1994 was 18 million. A. What was the population of Florida in millions in 1994? (14)
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
13.
14.
15.
16.
17.
18.
19.
20.
21.
115
B. What was the population of Ohio in millions in 1994? (11) In 1993 there were 15,229 miles of electrical railroad in the Far East. In 1993 there were 6,866 miles of electrical railroad in Africa. A. How many miles of electrical railroad were there in Europe in 1993? (91,814) B. How miles of electrical railroad were there in South America in 1993? (22,624) The population density of Missouri in 1994 was 74 people per square mile. The population density of Massachusetts in 1994 was 767 people per square mile. A. What was the population density in people per square mile of Illinois in 1994? (206) B. What was the population density in people per square mile of New Jersey in 1994? (1,042) The land area of California is 156,000 square miles. The land area of Kansas is 82,000 square miles. A. What is the land area of Illinois in square miles? (56,000) B. What is the land area of Nebraska in square miles? (77,000) There are 11 Indian reservations in Wisconsin. There are 3 Indian reservations in Nebraska. A. How many Indian reservations are there in Arizona? (23) B. How many Indian reservations are there in South Dakota? (9) The area of the Arctic Ocean is 5 million square miles. The area of the Pacific Ocean is 64 million square miles. A. What is the area of the Atlantic Ocean in millions of square miles? (33) B. What is the area of the Indian Ocean in millions of square miles? (28) The Mississippi River is 2,340 miles long. The Colorado River is 1,450 miles long. A. What is the length of the Ohio River in miles? (981) B. What is the length of the Arkansas River in miles? (1,149) Iraq has 100 billion barrels of proven oil reserves. The US has 23 billion barrels of proven oil reserves. A. How many billion barrels of proven oil reserves does Saudi Arabia have? (261) B. How many billion barrels of proven oil reserves does Mexico have? (51) The value of airplane exports from the US in 1994 was 19 billion dollars. The value of electrical machinery exports from the US in 1994 was 44 billion dollars. A. What was the value of office machine exports from the US in billions of dollars in 1994? (31) B. What was the value of power generator exports from the US in billions of dollars in 1994? (20) The value of petroleum imports to the US in 1994 was 39 billion dollars.
116
22.
23.
24.
25.
26.
27.
LAUGHLIN ET AL.
The value of toys, games, and sporting good imports to the US in 1994 was 11 billion dollars. A. What was the value of coffee imports to the US in billions of dollars in 1994? (2) B. What was the value of footwear imports to the US in billions of dollars in 1994? (12) There were 1,893,078 elementary and high school students in Illinois in 1994. There were 875,639 elementary and high school students in Missouri in 1994. A. How many elementary and high school students were there in California in 1994? (5,328,555) B. How many elementary and high school students were there in Florida in 1994? (2,040,763) Consumers spent 50 billion dollars on tobacco products in the US in 1994. Consumers spent 430 billion dollars on food for home consumption in the US in 1994. A. How many billion dollars did US residents spend dining out in 1994? (220) B. How many billion dollars did US residents spend on health products in 1994? (69) In 1992 the per capita income of Connecticut was 26,979 dollars. In 1992 the per capita income of Illinois was 21,608 dollars. A. What was the per capita income of Oregon in dollars in 1992? (18,202) B. What was the per capita income of Mississippi in dollars in 1992? (14,088) Consumers spent 45 billion dollars on used automobiles and trucks in the US in 1994. Consumers spent 21 billion dollars on auto and truck insurance in the US in 1994. A. How many billion dollars did consumers in the US spend on new cars and trucks in the US in 1994? (93) B. How many billion dollars did consumers in the US spend on gasoline and oil in 1994? (106) McDonalds spent 425 million dollars on advertising in the US in 1994. Ford Motor Company spent 920 million dollars on advertising in the US in 1994. A. How many million dollars did AT&T spend on advertising in the US in 1994? (700) B. How many million dollars did Anheuser–Busch spend on advertising in the US in 1994? (306) 9 billion dollars was spent on advertising automobiles in the US in 1994. 4 billion dollars was spent on advertising food in the US in 1994. A. How much in billions was spent on advertising apparel and shoes in the US in 1994? (1)
GROUP AND INDIVIDUAL QUANTITY ESTIMATIONS
117
B. How much in billions was spent on advertising tobacco and makeup in the US in 1994? (3) 28. 74 million people flew on United Airlines in 1994. 46 million people flew on Northwest Airlines in 1994. A. How many million people flew on Delta Airlines in 1994? (89) B. How many million people flew on American Airlines in 1994? (81) 29. There were 78,999 German Shepherds registered with the American Kennel Club in 1994. There were 126,393 Labrador Retrievers registered with the American Kennel Club in 1994. A. How many Rottweilers were registered with the American Kennel Club in 1994? (102,596) B. How many Golden Retrievers were registered with the American Kennel Club in 1994? (64,232) 30. There were 326 million native speakers of English in 1994. There were 169 million native speakers of Russian in 1994. A. How many million native speakers of German were there in 1994? (98) B. How many million native speakers of Spanish were there in 1994? (339) Note. Participants in conditions with two statements for a frame of reference were given both statements on each item. Participants in conditions with one statement for a frame of reference were given the first statement for each item. Participants in conditions with no frame of reference were not given either statement. Correct answers are in parentheses for each question. REFERENCES Abelson, R. P. (1995). Statistics as principled argument. Hillsboro, NJ: Erlbaum. Dawes, R. M. (1998). Behavioral decision making and judgment. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The handbook of social psychology (Vol. 1, pp. 497–548). New York: McGraw–Hill. Famighetti, R. (Ed.). (1996). World almanac and book of facts, 1996. Mahwah, NJ: World Almanac Books. Hastie, R. (1986). Review essay: Experimental evidence on group accuracy. In G. Owen & B. Grofman (Eds.), Information pooling and group accuracy (pp. 129–157). Westport, CT: JAI Press. Laughlin, P. R. (1980). Social combination processes of cooperative problem-solving groups on verbal intellective tasks. In M. Fishbein (Ed.), Progress in social psychology (pp. 127–155). Hillsdale, NJ: Erlbaum. Laughlin, P. R., & Ellis, A. L. (1986). Demonstrability and social combination processes on mathematical intellective tasks. Journal of Experimental Social Psychology, 22, 177–189. Levin, I. P., Schneider, S. I., & Gaeth, G. J. (1998). All frames are not created equal: A typology and critical analysis of framing effects. Organizational Behavior and Human Decision Processes, 76, 149–188. Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211, 453–458. Received February 18, 1999