- Email: [email protected]
A note on the intrasubject similarity of subjective probabilities obtained by estimates and by bets
0RGANIZATIOI~AL BEHAVIOR AND HU~cIAN PERFORMANCE
11, 250-252 (1974)
A Note on the Intrasubject Similarity of Subjective Probabilities Obtained by Estimates and by Bets1 LEE ROy BEACH2 University o] Washington In a recent study, DuCharme and Donnell (1973) found respectably high agreement among four methods of measuring subjective probabilities. Their study differed from previous work in that they focused upon individual subjects rather than upon group data. In discussing previous research, they cited Beach and Wise (1969) and correctly remarked that the use of grouped data in the latter paper precluded any conclusions about how well response modes agree for individuals. The purpose of this note is to present an individual subject reanalysis of part of the Beach and Wise data to add support to the DuCharme and Donnell conclusion that there is a fairly high degree of accord between subjective probabilities obtained from verbal estimates and subjective probabilities inferred from bets. METHOD
Procedure. A more elaborate procedural description is presented in Beach and Wise (1969) so only the essentials need be presented here. The subjects were given experience with two multinomial distributions that consisted of two decks of cards in each of which the letters A through F appeared with different frequencies; the frequencies of each letter were different for the two decks. In one deck the letters were written in red ink (the Red Deck) and in the other they were written in green ink (the Green Deck). After having become familiar with the contents of each deck, the subjects took part in what was essentially a "bookbag-and-pokerchip" task (Phillips & Edwards, 1966); out of the subjects' sight the experimenter supposedly flipped a coin to select one of the two decks and then he drew three successive cards from it. The subjects were told the letters on the cards but not the color in which they were written, i.e., not the deck from which they were drawn. The subjects' task was to verbally estimate the 1The author thanks William B. Carter for his help with the data analysis. Reprints may be obtained from Prof. L. R. Beach, Department of Psychology, Guthrie Hall, University of Washington, Seattle, Washington 98195. 250 Copyright © 1974 by Academic Press, Inc. All rights of reproduction in any form reserved.
INTRASUBJECT SIMILARITY
251
probability that the Red Deck or the Green Deck was the one being sampled. (Later, all of these probabilities were converted to the probability of the Red Deck for simplicity in data analysis.) For present purposes only two of the original three subject conditions are of interest. One of these conditions required the subjects to revise their subjective probability estimates after each of the three successive draws from the selected deck and then to bid for an experimenter-offered bet about which of the two decks actually was being sampled. The second condition required the subjects to wait until they had been told the letters on all three cards before they made their subjective probability estimates; then they bid for the bet. The bets were constructed so that subjective probabilities could be inferred from them (Beach & Wise, l°69; Preston & Baratta, 1948) and the present analysis compares, for each subject, these inferred subjective probabilities (ISP) and the final verbally estimated subjective probabilities (ESP) that were made after the subjects had been informed about the letters on all three cards. Subjects. Originally there were 30 subjects, 15 in each condition, but one was dropped because he always bid a constant amount for the bets. This left 29 subjects, all males, for each of whom there were 18 estimates and 18 bets. RESULTS
To make the results comparable to those presented by DuCharme and Donnell (1973), all ESP and ISP were converted to log likelihood ratios (see DuCharme & Donnell's paper for the method). Then the difference D, was calculated by subtracting B, the veridieal Bayesian log likelihood ratio (computed from the letters' frequencies in the decks) from the log likelihood ratio based on either the corresponding ESP or ISP. The mean D's, across subjects, and the associated standard deviations (s) are: DEsp-B = --.01 with s = .06, D~se-~ = -.01 with s = .09, and D~sp-~sP = .00 with s = .10. An additional analysis consisted of computing correlations (and slopes) between the log likelihood ratios for the ESP and Bayes, for the ISP and Bayes, and for the ESP and ISP for each of the 29 subjects. The mean correlations and slopes were: r~sp-B = .79 with a slope of .62, r~sp-B = .59 with a slope of .57, and rEsP-~sr = .49 with a slope of .40. Twenty five of the 29 rEse-s correlations were significant at or beyond the .05 level, as were 21 of the r~sp_B correlations, and 18 of the rEs~--~sp correlations. Moreover, for 22 of the 29 subjects the r~sp-~ correlation was higher than the r~sp-B correlation, indicating that the estimates tended to be more veridical than the subjective probabilities inferred from bets. Of course, it is no
252
LEE ROY BEACH
surprise that the DEsP-ise variance should be larger and the rEs~_isF correlations should be lower than comparable calculations using the Bayes log likelihood ratios; there is no error variance in Bayes but there is in the E S P and ISP and comparisons involving both of the latter reflect, the contributions of both of these error variances. DISCUSSION Although the observed error, in terms of D, in the present results is less than that reported by DuCharme and Donnell (1973) and the correlations are somewhat lower, the results support a conclusion similar to theirs: The estimate method and the betting method yield fairly similar subjective probabilities and both sets of probabilities are conservative relative to the veridical Bayesian probabilities. REFERENCES BEACH,L. R., & WISE, J. A. Subjective probability revision and subsequent decisions. Journal of experimental psychology, 1969,81, 561-565. DuCHARME, W. M., & DONNELL, M. L. Intrasubject comparison of four response modes for "subjective probability" assessment. Organizational behavior and human per]ormance, 1973, 3, 108-117. PHILLIPS, L., & EDWARDS,W. Conservatism in a simple probability inference task. Journal o] experimental psychology, 1966, 72, 346-354. PaESTON, M. G., & BARAT~A,P. An experimental study of the auction value of an uncertain outcome. American journal o] psychology, 1948, 61, 183-193. RECEIVED: M a y 14, 1973
11, 250-252 (1974)
A Note on the Intrasubject Similarity of Subjective Probabilities Obtained by Estimates and by Bets1 LEE ROy BEACH2 University o] Washington In a recent study, DuCharme and Donnell (1973) found respectably high agreement among four methods of measuring subjective probabilities. Their study differed from previous work in that they focused upon individual subjects rather than upon group data. In discussing previous research, they cited Beach and Wise (1969) and correctly remarked that the use of grouped data in the latter paper precluded any conclusions about how well response modes agree for individuals. The purpose of this note is to present an individual subject reanalysis of part of the Beach and Wise data to add support to the DuCharme and Donnell conclusion that there is a fairly high degree of accord between subjective probabilities obtained from verbal estimates and subjective probabilities inferred from bets. METHOD
Procedure. A more elaborate procedural description is presented in Beach and Wise (1969) so only the essentials need be presented here. The subjects were given experience with two multinomial distributions that consisted of two decks of cards in each of which the letters A through F appeared with different frequencies; the frequencies of each letter were different for the two decks. In one deck the letters were written in red ink (the Red Deck) and in the other they were written in green ink (the Green Deck). After having become familiar with the contents of each deck, the subjects took part in what was essentially a "bookbag-and-pokerchip" task (Phillips & Edwards, 1966); out of the subjects' sight the experimenter supposedly flipped a coin to select one of the two decks and then he drew three successive cards from it. The subjects were told the letters on the cards but not the color in which they were written, i.e., not the deck from which they were drawn. The subjects' task was to verbally estimate the 1The author thanks William B. Carter for his help with the data analysis. Reprints may be obtained from Prof. L. R. Beach, Department of Psychology, Guthrie Hall, University of Washington, Seattle, Washington 98195. 250 Copyright © 1974 by Academic Press, Inc. All rights of reproduction in any form reserved.
INTRASUBJECT SIMILARITY
251
probability that the Red Deck or the Green Deck was the one being sampled. (Later, all of these probabilities were converted to the probability of the Red Deck for simplicity in data analysis.) For present purposes only two of the original three subject conditions are of interest. One of these conditions required the subjects to revise their subjective probability estimates after each of the three successive draws from the selected deck and then to bid for an experimenter-offered bet about which of the two decks actually was being sampled. The second condition required the subjects to wait until they had been told the letters on all three cards before they made their subjective probability estimates; then they bid for the bet. The bets were constructed so that subjective probabilities could be inferred from them (Beach & Wise, l°69; Preston & Baratta, 1948) and the present analysis compares, for each subject, these inferred subjective probabilities (ISP) and the final verbally estimated subjective probabilities (ESP) that were made after the subjects had been informed about the letters on all three cards. Subjects. Originally there were 30 subjects, 15 in each condition, but one was dropped because he always bid a constant amount for the bets. This left 29 subjects, all males, for each of whom there were 18 estimates and 18 bets. RESULTS
To make the results comparable to those presented by DuCharme and Donnell (1973), all ESP and ISP were converted to log likelihood ratios (see DuCharme & Donnell's paper for the method). Then the difference D, was calculated by subtracting B, the veridieal Bayesian log likelihood ratio (computed from the letters' frequencies in the decks) from the log likelihood ratio based on either the corresponding ESP or ISP. The mean D's, across subjects, and the associated standard deviations (s) are: DEsp-B = --.01 with s = .06, D~se-~ = -.01 with s = .09, and D~sp-~sP = .00 with s = .10. An additional analysis consisted of computing correlations (and slopes) between the log likelihood ratios for the ESP and Bayes, for the ISP and Bayes, and for the ESP and ISP for each of the 29 subjects. The mean correlations and slopes were: r~sp-B = .79 with a slope of .62, r~sp-B = .59 with a slope of .57, and rEsP-~sr = .49 with a slope of .40. Twenty five of the 29 rEse-s correlations were significant at or beyond the .05 level, as were 21 of the r~sp_B correlations, and 18 of the rEs~--~sp correlations. Moreover, for 22 of the 29 subjects the r~sp-~ correlation was higher than the r~sp-B correlation, indicating that the estimates tended to be more veridical than the subjective probabilities inferred from bets. Of course, it is no
252
LEE ROY BEACH
surprise that the DEsP-ise variance should be larger and the rEs~_isF correlations should be lower than comparable calculations using the Bayes log likelihood ratios; there is no error variance in Bayes but there is in the E S P and ISP and comparisons involving both of the latter reflect, the contributions of both of these error variances. DISCUSSION Although the observed error, in terms of D, in the present results is less than that reported by DuCharme and Donnell (1973) and the correlations are somewhat lower, the results support a conclusion similar to theirs: The estimate method and the betting method yield fairly similar subjective probabilities and both sets of probabilities are conservative relative to the veridical Bayesian probabilities. REFERENCES BEACH,L. R., & WISE, J. A. Subjective probability revision and subsequent decisions. Journal of experimental psychology, 1969,81, 561-565. DuCHARME, W. M., & DONNELL, M. L. Intrasubject comparison of four response modes for "subjective probability" assessment. Organizational behavior and human per]ormance, 1973, 3, 108-117. PHILLIPS, L., & EDWARDS,W. Conservatism in a simple probability inference task. Journal o] experimental psychology, 1966, 72, 346-354. PaESTON, M. G., & BARAT~A,P. An experimental study of the auction value of an uncertain outcome. American journal o] psychology, 1948, 61, 183-193. RECEIVED: M a y 14, 1973