- Email: [email protected]
The value of information in a sealed bidding experiment
O R G A N I Z A T I O N A L BEHAVIOR AND H U M A N PERFORMANCE
24, 269-286 (1979)
The Value of Information in a Sealed Bidding Experiment STEVEN M. M E D L I N U.S. Army Research Institute fi~r the Behavioral and Social Sciences The present study assesses directly the value of perfect information from two different data sources to a decision maker confronted with the sealed bidding task. Predictions about the value of information that eliminates uncertainty and the subsequent bid are derived following Howard's (1EEE Transactions o f Systems Science and Cybernetics, 1966, SSC-2, 22-26) analysis of the value of information. A 2 × 4 × 2 factorial design varied (i) the distributions of the data sources, (ii) the kind of information purchasable, and (iii) the order of presentation for blocks of trials where information could or could not be purchased. Each of 80 subjects performed the sealed bidding task for a series of 72 trials. The results show an almost linear relationship between the observed maximum price offered for information and the prescribed value of the information. Furthermore, subjects consistently underbid in all information conditions. Deviations from optimality are interpreted in terms of the subjects' limited information processing capacities and the nature of the decision task.
When the outcome of a decision depends upon the state of nature that obtains, it is desirable and often possible to reduce uncertainty by acquiring information about the relevant states prior to making the decision. The information is almost always costly in terms of efforts for obtaining it, time for processing it, or resources for purchasing it. There is certainly no advantage in acquiring information when the cost of additional information is greater than the consequent increase in expected gain (Fried & Peterson, 1969). Two general classes of models have dealt with the value of information in uncertain decision situations. The first class of models specifies criterion functions for a variety of decision making situations and then derives models for information purchasing that optimize these functions (Howard, 1966, 1967; Marschack, 1959; Raiffa & Schlaiffer, 1961; Wald, 1947). In these information purchasing tasks, the cost per unit is prespecified and the question under investigation is how many informational units are purchased before a terminal decision is made (Becker & McClintock, 1967; Fried & Peterson, 1969; Larsson, 1968; O'Connor, Peterson, & Palmer, 1972; Pitz, 1968; Tversky & Edwards, 1966). The second class of models assumes a fixed and known amount of information to be purchased and specifies the demand price or value of information that optimizes the prespecified critePlease send request for reprints to Steven M. Medlin, U.S. Army Research Institute, 5001 Eisenhower Ave., Alexandria, VA 22333. 269 0030-5073/79/060269-18502.00/0 Copyright © 1979 by Academic Press, Inc. All rights of reproduction in any form reserved.
270
STEVEN
M. MEDLIN
rion function (Marschak, 1959). Wendt (1969) and Shanteau and Anderson (1972) fixed the amount of information to be purchased and assessed the value of a single datum of information to the subject in a simple information purchasing task (Becker, DeGroot, & Marschak, 1964). The results of both studies indicate that subjects vary their estimates in the predicted directions as a function of prior odds, payoffs, and data source diagnosticity; furthermore, the subjects tend to overestimate the prescribed value of the information when it is relatively low or near zero. The present study is also concerned with the assessment of the value of information, but differs from the work of Wendt (1969) and Shanteau and Anderson (1972) in two significant ways. First, the present study provides two sources of uncertainty (and thus two sources of information), while in previous experiments only one source of uncertainty was available. Several interesting research questions are raised by the availability of two sources of information. What is the value of information that reduces uncertainty about qualitatively different aspects of a decision making task? What is the relationship between the sum of individual values of information, each of which is associated with a different source of uncertainty, and the value of information that eliminates the uncertainty from both these sources jointly? The second difference between this study and earlier information purchasing studies concerns the diagnosticity of the information. The present study provides perfect information, eliminating all uncertainty; in previous studies the information could reduce uncertainty, but not eliminate it. The concept of perfect information is useful in analyzing the information phase of the decision task; the value of perfect information on a task variable represents the maximum amount that a decision maker should pay to eliminate uncertainty regarding the variable (Howard, 1966, 1968). The purpose of the present research is to study information purchasing behavior in a decision task that requires the subject to assess directly the value of information that reduces uncertainty about one or more task variables. To provide an information purchasing task in which subjects directly assess the value of one or more items of information, the sealed bidding task was used. A brief, general example will serve to illustrate the task. Consider a construction firm which is bidding on a contract to build a bridge. Company A bids on the contract against several competitors. The company's actual cost of performing the contract (i.e., building the bridge) is denoted by p; as is typically the case, the company is uncertain of this exact cost. The lowest bid of the competitors is denoted by t; this exact value too is uncertain. Given these uncertainties, assumed to be characterized by two independent probability distributions, the problem confronting company A is to determine its bid, b, so as to maximize the expected value, E(v), of the contract.
SEALED BIDDING EXPERIMENT
271
By investing certain resources in research and development it is assumed that the company can completely reduce the uncertainty regarding p. The company must decide, therefore, how much to invest for gathering information that reduces the uncertainty regarding p and, if it obtains this information, how much to bid for the contract. Similarly, and independent of its first problem, the company may invest certain resources in assessing the financial situation of its competitors, their past bidding records, or may even become involved in industrial spying to learn the competitors' bids, and consequently, the exact value of the lowest bid, t. Company A must decide, therefore, how much money, if any, to invest in order to secure this information, and given t, how much to bid for the contract. Finally, if company A decides to invest its resources in reducing uncertainty about both p and t simultaneously, it must decide how much to invest to obtain this information, and having determined the exact values of p and t, how much to bid for the contract. Howard (1966) has employed this same bidding problem to illustrate his approach to the theory of the value of information. He demonstrated that numerical values could be assigned to the reduction of uncertainty, and showed how to calculate the optimal values of various sources of information in the bidding problem. He also determined the optimal bid, the one which maximizes the expected utility of the contract, when various items of information are known. (Table 1 presents the optimal values for the specific task under the present study.)
METHOD
Subjects Eighty undergraduate students at the University of North Carolina at Chapel Hill served as subjects. They were recruited by posting advertisements offering monetary rewards for participation in a decisionmaking experiment.
The Sealed Bidding Task The subjects participated individually in a computerized version of the sealed bidding task. After reading the instructions, and receiving a short verbal explanation (Medlin, 1976), which provided a cover story for the task, explained the characteristics of the cost (p) and lowest competitive bid (t) distributions, and led the subject through several examples, each subject was presented with his own cost functionf(p), lowest competitive bid (LCB) functionf(t), and the (fictitious) number of his opponents. The subject was also told what kind of perfect information, if any, could be purchased, either no information, information about cost, information about LCB, or information about both cost and LCB jointly. A single trial in the computerized sealed bidding task proceeds in the following manner. If a subject was not allowed to purchase information,
272
S T E V E N M. M E D L I N
he started the trial by placing a bid for the contract. When information could be purchased, a computerized version of the Marschak bidding procedure (Becker, DeGroot, & Marschak, 1964) was used to elicit the value of the information to the subject. Depending upon the information condition, to be described below, the information purchased by the subject was either a p value randomly selected from the cost distribution, a t value randomly selected from the LCB distribution, or both. The Marschak bidding p r o c e d u r e required the subject to state his maximum buying price (MBP) for the information. The MBP was then compared to a selling price (SP), an integer randomly selected from a uniform density function (0,50). If MBP < SL the subject did not purchase the information and placed his bid for the contract without it. If MBP >I SL, the subject purchased the information and proceeded to place his bid. As shown by Becker et al. (1964), this procedure possesses the property that the subject maximizes his subjectively expected value of the transaction by stating a MBP equal to his cash-equivalent of the information. After typing his bid, b, for the contract, the computer compared b to t. If b /> t, the subject did not win the contract; his p a y o f f in this case was zero minus the price paid for the information, if it was purchased. If b < t, the subject won the contract; his p a y o f f in this case consisted of b - p minus the price paid for the information, if it was purchased. At the end of each trial the subject was informed of the p value for that trial, the bids of his opponents, and his gain (positive or negative) for that trial. One of the competitor's bids was the predetermined t value randomly selected fromf(t); the other bids were integers randomly selected from a uniform density function over the interval (t, 350).
Experimental Design The experiment employed a 2 x 4 x 2 distribution by information by order factorial design. There were two distribution conditions, DC1 and DC2, each defined by a pair of cost (p) and LCB (t) uniform density functions. For both distribution conditions/91 = t i = 100. F o r DC1 192 = 200 and tz = 300, whereas for DC2/92 = 300 and t2 = 200. Four information conditions, IC1, IC2, IC3, and IC4, defined the second factor of the design: IC1 IC2 IC3 IC4
Information Information Information Information
about/9 about t about/9 about/9
may be purchased, m = 1, may be purchased, m = 1, and t may be purchased, m = 1, and t may be purchased, m = 5,
where m denotes the number of competitors. Subjects in IC4 were treated exactly like subjects in IC3 except that in the former condition they were informed that they were bidding against five competitors. Since the computer program for the SBT allows for the simultaneous running of up to
273
SEALED BIDDING EXPERIMENT
six subjects, each with his own teletypewriter in a time-sharing mode (Medlin, 1976), the deception in IC4 could not be detected. The third factor varied the order in which a block of 24 trials on which information c o u l d not be purchased (type A) and a block of 48 trials on which information c o u l d be purchased (type B) were presented. In order condition 1 (OC1) the subjects were first presented with 24 type A trials followed by 48 type B trials. In order condition 2 (OC2) these two blocks were presented in reverse order. Five subjects were run simultaneously in a given session with each group of subjects randomly assigned to 1 of the 16 cells of the design. A session lasted approximately 2 hr. Upon completing the 72 trials, the total number of points earned by the subject was exchanged for money at the rate of 0.4 cents per point. The mean gain per subject was $6.79. Each subject completed a postexperimental questionnaire (see Medlin, 1976), which will not be analyzed here, and was dismissed. RESULTS The data analysis is divided into two major sections. The first will consider the value of the information to the subject. The second section will be concerned with three classes of bids: bids made when information could not be purchased, bids made when information could be but was not purchased, and bids made after purchasing information. In the sealed bidding task, the value of the information to the decision maker is defined as the maximum amount a subject is willing to pay to purchase the item of information (MBP). Following Howard's (1966) approach to the value of information, the prescribed values of information and bids for the present experimental conditions were obtained (Table 1). Several predictions are implied by the model about the effects of the TABLE 1 VALUE OF INFORMATION AND OPTIMAL BID BY INFORMATION AND DISTRIBUTION CONDITIONS Information condition
Prescribed values
Distribution Condition DC 1"
DC2
1.04 ((Cost + 300)/2)
8.33 ((Cost + 300)/2)
IC 1 (cost)
Information Bid
IC2 (LCB)
Information Bid
28.12 If* t > p, bid t - 1 If* t ~< p, bid above t
IC3 and IC4 (both)
Information Bid
30.21 16.67 If t > p, bid t - 1 I f t <~ p, bid above t
*~ equals 150 for DC1 and 200 for DC2.
0
274
S T E V E N M. M E D L I N
experimental variables on the MBPs. As Table 1 shows, the model predicts two significant main effects, one due to the distribution factor and one due to the information factor, and a significant distribution by information interaction. Since the LCB is not affected by the n u m b e r of competitors, the MBPs for groups IC3 and IC4 are predicted to be the same for both distribution conditions. Since between trial learning is expected in multiple trial decisionmaking experiments, the 48 type B trials were divided into two equal blocks. The main MBP was computed separately for each block of 24 type B trials and each subject. An A N O V A was conducted on these means, treating blocks as a repeated factor with two levels. Although there was no main effect due to learning (F(1,64) = 1.26, p < .265), the analysis revealed a significant distribution by block interaction (F(1,64) = 8.67, p < .005). The mean MBP in DC1 decreased from 20.20 in the first block of 24 trials to 18.62 in the second block, whereas the mean MBP in DC2 increased from 11.75 to 12.46. Since the distribution by block interaction was significant, two separate 2 x 4 x 2 distribution by information by order A N O V A s were conducted on the individual mean MBPs, one for each block of 24 type B trials. As predicted, only the main effects due to distribution (first block, F(1,64) = 25.20, < p .001; second block, F(1,64) = 14.72, p < .001) and information (first block, F(3,64) = 2.85, p < .039; second block, F(3,64) = 4.59, p < .006) and the distribution by information interaction (first block, F(3,64) = 4.83, p < .004; second block, F(3,64) = 8.06, p < .001) were significant. The means and standard deviations of the MBPs are presented in Table 2. The results are presented separately for each block of 24 type B trials, distribution condition, and information condition. A comparison of Tables 1 and 2 shows that the M B P s - - t h e subjects' estimates of the values of information--varied consistently in the same direction as did the prescribed values. The subjects offered a higher MBP for information in DC1 than in DC2, as would be expected from Table 1. Similarly, for each distribution condition the means of the MBPs for the four information conditions were ordered as predicted (with a single exception for groups IC1 and IC3 in distribution c o n d i t i o n 2). T h e p r e d i c t e d i n t e r a c t i o n effect between information and distribution was also realized. Thus, for example, subjects in DC1 were willing to pay a considerably higher price for information about the LCB (20.88 and 21.02 for blocks 1 and 2, respectively), whose prescribed value was 28.12, than subjects in DC2 (6.45 and 6.43 for blocks 1 and 2, respectively) for whom the prescribed value of the same information was zero. A closer inspection of Tables 1 and 2 and analyses of signed deviations between prescribed and observed values of information (see Medlin, 1976) revealed another finding similar to one reported by Wendt (1969). In
275
SEALED BIDDING EXPERIMENT TABLE MEANS
AND STANDARD
DEVIATIONS
2
OF MAXIMUM
AND DISTRIBUTION
BUYING
P R I C E S BY I N F O R M A T I O N
CONDITIONS
Distribution condition DC 1 Information condition
DC2
Block 1
Block 2
Total
Block 1
Block 2
Total
IC1
Mean SD
12.51 11.05
8.14 8.03
10.32 9.77
14.12 5.34
15.26 5.40
14.69 5.37
IC2
Mean SD
20.88 5.94
21.02 7.15
20.95 6.57
6.40 5.71
6.45 5.74
6.43 5.73
IC3
Mean SD
25.18 8.18
23.42 8.65
24.30 8.42
11.65 7.62
11.63 8.64
11.64 8.16
IC4
Mean SD
22.25 6.92
21.88 6.89
22.06 6.91
14.84 4.49
16.52 3.73
15.68 4.13
20.20
18.62
19.41
11.75
12.46
12.11
Overall m e a n
general, the MBP of perfect information exceeded its prescribed value when the latter was relatively low, whereas the reverse relationship obtained when the prescribed value was relatively high. The mean MBPs of information for each distribution and information group, computed over the two blocks of 24 trials each and the two order conditions, are plotted against the prescribed values in Fig. 1. There is a monotonically increas-
i
I
i
t
t
i
210
215
ADISTRIBUTION CONDITION 1 • DISTRIBUTION CONDITION 2
25
2O ~a a.
m >-
_= IE
~E
0
i
0
i
5
i
10
lIS
310
OPTIMAL VALUE OF INFORMATION FIG. 1. M a x i m u m buying price as a function of prescribed information value.
276
S T E V E N M. M E D L I N
ing, almost linear relationship between the two values. The correlation between the MBPs and predicted values is .933 (p < .01) with a slope and intercept of .467 and 8.078, respectively. Similar results were obtained when the correlations between the MBPs and the prescribed values were computed for each block of 24 trials separately (the correlation, slope, and intercept were .904, .455, and 8.521, respectively, for block 1, and .940, .482, and 7.633 for block 2). When the same decision task is repeated for many trials and complete feedback is provided at the end of each trial, sequential effects are anticipated. In the present study, success or failure in winning the contract or in purchasing the information on a given trial might have affected the MBP on subsequent trials. To study the effects of winning or losing the contract on the MBPs, difference scores were computed for each subject between the mean MBP on trials that immediately followed a successful bid and the mean MBP on trials immediately following an unsuccessful bid. An A N O V A conducted on these difference scores failed to yield any significant effect. In particular, the null hypothesis of zero mean difference score was not rejected. The effects of information purchasing on the subsequent MBPs were then examined in a similar manner. The difference between the mean MBP for information following trials on which information was purchased and the mean MBP following trials on which the attempt to purchase information was unsuccessful was computed separately for each subject. An A N O V A conducted on these difference scores revealed that subjects paid more for information after successfully purchasing it on the immediately preceding trial than after an unsuccessful attempt (F(1,64) = 5.97, p < .017). Bids No information could be purchased on 24 of the 72 trials. To test for learning effects in bids, these 24 type A trials were divided into two equal blocks. Mean bids were computed for each block of 12 trials and each subject; and an A N O V A was conducted on these means, treating blocks as a repeated factor with two levels. Having established no significant learning effects (Medlin, 1976, Table 8), the mean bids were then computed for each subject over the 24 type A trials, and an A N O V A was conducted on these overall means to test for the main effects of the three experimental factors and their interactions. The distribution main effect (F(1,64 = 39.029, p < .001), the order main effect (F(1,64) = 6.130, p < •016), and the information by order interaction effect (F(3,64) = 2.898, p < .042) were all significant at the .05 level. Table 3 presents the means and standard deviations of the bids for the 24 type A trials. For these bids the model predicts a significant effect due to the distribution condition; the optimal bids for DC1 and DC2 are 225
277
SEALED BIDDING EXPERIMENT TABLE 3 MEANS AND STANDARD DEVIATIONS OF BIDS WHEN INFORMATION COULD NOT BE PURCHASED Distribution condition DC 1 Information condition
DC2
OC 1
OC2
OC 1
OC2
IC 1
Mean SD
204.6 19.4
218.6 8.0
172.7 7.5
171.7 13.1
IC2
Mean SD
202.2 14.9
216.9 21.9
179.2 6.1
189.6 18.9
IC3
Mean SD
188.7 12.8
218.8 24.7
175.8 10.0
196.4 22.1
IC4
Mean SD
201.9 14.4
202.8 22.0
194.8 26.9
181.5 14.6
199.4
214.3
180.6
184.8
Overall mean
and 200, respectively. Inspection of Table 3 reveals that the direction of the significant distribution effect is as predicted, with higher mean bids for DC1 than DC2. The mean bids, however, are smaller than the optimal bids for all the 16 experimental groups. The significant order effect, not accounted for by the model, is shown clearly in Table 3. The mean bids for information conditions 1, 2, and 3 are higher in OC2 than OC1. This relationship is reversed for information condition 4 in distribution condition 2; hence the marginally significant interaction effect. The significant order effect is of interest in showing that subjects with information purchasing practice bid higher than subjects who first bid on the no information trials. As indicated above, the subjects underbid for the contract in all 16 experimental groups. To test for this difference between observed mean bids and optimal bids, signed deviation scores were computed for each subject and then subjected to a 2 x 4 x 2 distribution by information by order ANOVA. There was a significant order main effect (F(1,64) = 6.763, p < .012), but no distribution or distribution by order effects as in the preceding analysis. The overall mean of the signed deviation scores was significantly different from zero (F(1,64) = 82.550, p < .001), supporting the previous observation of persistent underbidding. Information was purchasable on 48 of the 72 trials. Due to the nature of the Marschak bidding procedure, however, information was not purchased on some of these trials. Table 4 shows the means and standard deviations of the number of trials (out of 48) on which information was purchased. Since the uniform function from which the SPs were randomly
278
STEVEN M. M E D L I N TABLE 4 MEANS AND STANDARD DEVIATIONS OF NUMBER OF TRIALS ON WHICH INFORMATION WAS PURCHASED Distribution condition DC 1
Information condition
DC2
OC 1
0C2
OC 1
0C2
IC1
Mean SD
10.40 8.53
10.40 10.29
15.20 4.97
14.40 5.86
IC2
Me an SD
20.20 8.41
19.00 9.03
4.60 3.36
6.20 6.14
IC3
Mean SD
25.60 8.08
22.80 10.50
13.40 8.05
9.40 8.50
IC4
Mean SD
16.60 6.66
26.20 9.42
14.40 4.16
15.80 2.68
chosen was the same for all 16 groups, the means in Table 4 are e x p e c t e d to v a r y considerably f r o m one group of subjects to another, as they actually do. Subtracting the means presented in Table 4 f r o m 48 yields the mean n u m b e r of trials on which information could be but was not purchased. The analyses conducted on the bids made on these trials were analogous to those conducted immediately a b o v e on the bids w h e n no information could be purchased. Thus, an A N O V A was conducted on the m e a n bids taken o v e r all trials on which information was available but not purchased. The A N O V A yielded a significant distribution effect (F(1,64) = 59.94, p < .001); no other main or interaction effect, including the order main effect and the order by information interaction effect o b s e r v e d above, were significant at the .05 level. The difference b e t w e e n the two A N O V A results with respect to the order and order b y information effects is interpretable. N o t e that the 24 type A trials a p p e a r e d as a single block either at the beginning or at the end of the task. The trials on which information could be but was not p u r c h a s e d were spread out a m o n g either the first or the last 48 type B trials, with the 24 middle trials overlapping. Consequently, to the extent that order has an effect it would be e x p e c t e d to be stronger in the f o r m e r than in the latter case. Table 5 presents the means and standard deviations of bids on trials on which information could be but was not purchased. The significant distribution main effect, with higher means for DC1 than DC2, and the lack of information and order main effects are clearly shown in the table. The table also shows that, with one exception, the o b s e r v e d m e a n bids are smaller than the prescribed bids (225 and 200 for DC1 and DC2, respec-
279
SEALED BIDDING EXPERIMENT TABLE 5 MEANS AND STANDARD DEVIATIONS OF BIDS WHEN INFORMATION COULD BE BUT WAS NOT PURCHASED Distribution condition DC 1 Information condition
DC2
OC 1
OC2
OC 1
OC2
IC1
Mean SD
214.0 22.4
228.6 17.6
172.7 7.3
167.5 21.2
IC2
Mean SD
216.1 10.8
214.1 22.2
181.6 6.7
188.8 19.0
1C3
Mean SD
204.8 18.9
2!2.8 27.1
180.7 12.1
186.3 15.0
IC4
Mean SD
213.6 14.4
200.8 21.9
194.3 25.3
180,6 15.4
212.1
214.1
182.3
180.8
Overall mean
tively). An A N O V A conducted on the signed deviation scores, c o m p u t e d separately for each subject, confirmed this o b s e r v a t i o n by showing that the overall m e a n was significantly different from zero (F(1,64) = 54.854, p < .001). The A N O V A conducted on the signed deviation scores also yielded a marginally significantly distribution b y information interaction effect, which m a y be attributed to chance. T w o sets of bids have b e e n analyzed, bids made on type A trials and bids made by the same subjects on trials on which information could be but was not purchased. In addition to the availability of purchasable information, the two sets differ f r o m each other, as we h a v e already noted when discussing the presence or absence of order effects, in terms of the order of a p p e a r a n c e of the trials in the sequence. To test for differences b e t w e e n these two sets of bids, the m e a n bid of each set was c o m p u t e d separately for each subject. An A N O V A was conducted on these means, treating each set of bids as a repeated factor with two levels. The overall means did not differ f r o m one set to another but there were distribution by set (F(1,64) = 7.578, p < .008) and order by set (F(1,64) = 10.196, p < .002) significant interaction effects. The latter interaction effect is due to a significant order effect that has already been reported for the first but not the second set of bids. As a c o m p a r i s o n of the overall m e a n s in Tables 3 and 5 shows, the significant distribution b y set interaction effect is due to a larger difference in the means of DC1 and DC2 in the first than the second set of bids. The analyses of bids w h e n information was p u r c h a s e d are divided into three parts, depending on whether the value o f p (group IC 1), the value of
280
S T E V E N M. M E D L I N
t (group IC2), or the values of both p and t (groups IC3 and IC4) were known exactly. W h e n p is known exactly, an analysis of the observed bids is uninformative; it is the discrepancy between observed and predicted bids that is of interest. Following H o w a r d ' s (1966) analysis, the optimal bid was computed for each trial on which p was known and the observed bid was subtracted. The resulting signed deviation scores, computed separately for each of the 20 IC1 subjects, were averaged, and the resulting means were subjected to a 2 × 2 distribution by order A N O V A . The overall mean of the signed deviation scores, -14.58, was significantly different from zero (F(1,16) = 10.828, p < .001), but no other effects were significant. The results thus show that when cost of performing the contract was known subjects still bid significantly below the optimal level. Although the subjects in group IC1 underbid both when information about p was purchased and when it was not, the analysis just conducted does not address the question of whether information about cost affected their bids. To answer this question, correlations between cost and bid were computed separately for each of the 20 subjects in group IC 1 for trials on which information was purchased. Four subjects did not purchase sufficient information for a correlation coefficient to be computed. Fourteen of the remaining sixteen correlations were significant at the .05 level. Thus subjects employed the information about cost to determine their bids; they just did not use it optimally, showing the same tendency to underbid. I f t is known exactly, the optimal policy is to bid just below t ift >/3 and above t if t ~< ~. There were 20 IC2 subjects altogether who could and did purchase information about t, 10 in DC1, and 10 in DC2. The total number of trials on which information about t was purchased, computable from the means presented in Table 5, was 196 for the 10 subjects in DC1 and 54 for the 10 in DC2. On 185 of the 196 trials in DC1 the optimal bidding policy was followed. The results for DC2 (where information about t is worthless) were different; the optimal bidding policy was followed on only 29 of the 54 trials. The difference between these two groups of 10 subjects is attributed to the difference in prescriptions of the bidding policy. The subjects who purchased information about t under DC 1 should or should not bid depending on whether t > ,~ or t <~p (13 = 150 and 100 ~< t ~< 300 for this condition). H o w e v e r , the subjects who purchased information about t under DC2 should never bid for the contract since it was always the case that t ~
SEALED
BIDDING
EXPERIMENT
281
puted. Of the remaining 16 correlations, 14 were significant at the .05 level. If both cost and LCB are known, the optimal bidding policy is trivial (see Table 1), namely, to bid just below t if t > p, and above t if t ~< p. Forty subjects, 20 in IC3 and 20 in IC4, had opportunity to purchase information about both p and t. The subjects of IC3 used this opportunity on 356 trials (see the means on Table 4 from which these frequencies can be computed) and the subjects of IC4 on 362 trials. With very few exceptions (4 of 356 in IC3 and 6 of 362 in IC4), the bids were optimal.
DISCUSSION The present approach to model testing and building follows the research methodology outlined by Rapoport (1975). Using the maximization of expected utility model for the scaled bidding task as an " i d e a l " or baseline, the discrepancies between prescribed and observed bids and values of information (MBP) are explored. It is assumed that the dis, crepancies will be neither so large nor so unsystematic as to make the comparison between prescribed and observed behavior meaningless or useless. If systematic discrepancies are found, they may be interpretable in terms of cognitive or environmental constraints imposed on the subject (Shuford, 1964), or heuristics that the subject employs to reduce the cognitive complexity of the task (Tversky & Kahneman, 1975).
Value of Information With the exception of learning effects, the MBPs support the predictions of the model in their direction but not in their values. The model predicts significant distribution, information, and distribution by information effects in the values of information, and no significant main or interaction effects due to order of presentation or to the (fictitious) number of opponents. The ANOVA results support all of these predictions. There were significant learning and sequential effects, which are not accounted for by the single-stage model, but are not unexpected in view of the relatively small number of trials, which did not allow the subjects to reach an asymptote. The discrepancy between observed and prescribed values of information is displayed in Fig. 1. Consistent with the results reported by Wendt (1969), Fig. 1 exhibits overestimation of relatively low values and underestimation of relatively high values of information. The function relating the MBPs to the prescribed values is approximately linear in form; the linear correlation accounts for 87% of the variance. Inspection of the results reported by Wendt (1969, Figs. 1 and 2), who employed a different procedure in his experiments, reveals a similar linear relationship. At least two major explanations may be proposed for the finding displayed in Fig. 1. The first attributes this discrepancy to criteria adopted by
282
S T E V E N M. M E D L I N
some or all of the subjects that differ substantially from the one assumed by the model. Thus, the subjects might have possessed nonlinear utility functions and attempted to maximize expected utility rather than expected value. Alternatively, some or all of the subjects might have adopted more complex criteria such as maximization of probability of winning the contract subject to attaining a fixed expected value (or utility). It has frequently been noted that "discriminability among various objective criteria that subjects may adopt is difficult to achieve, since the maxima derived from optimization models are often very flat, and fairly substantial deviations from optimal decision behavior typically produce only extremely small changes in DM's (decision maker's) economic prospects" (Rapoport, 1975, p.365). Flat maxima were also demonstrated by Wendt in his experiments and proposed by him as an explanation for the failure of his subjects to match the true value of information in their decisions. Flat maxima may account for erratic decision behavior but not for systematic biases. The present study goes beyond Wendt's in showing that when the subjects deviated from optimality it was in the direction providing for smaller losses. Results reported elsewhere (Medlin, 1976) show that in those cases where the subjects offered too much for information, when the prescribed values were 0 or 1.04 (see Table 1), the expected value functions dropped more slowly for numbers exceeding the prescribed values than for numbers smaller than the prescribed values. Conversely, when the prescribed values were relatively high (information conditions 2, 3, and 4 in DC1) the expected value functions dropped slowly for numbers less than the prescribed values but dropped considerably more steeply for numbers exceeding the prescribed values when the observed MBPs were too low. Finally, when the expected value functions tapered off rather sharply on both sides of the maximum, the MBPs were close to the prescribed values, as in information conditions 1, 3, and 4 in DC2. A second explanation attributes the discrepancy between observed and prescribed values to the way the Marschak bidding procedure was employed. This procedure possesses the theoretical property that it is always optimal for a subject to state his fair price of the information as his MBP. We suspect, however, that by using a uniform distribution of integral SPs with the same range for all 16 experimental groups we might have biased the results by causing a regression toward the mean of this distribution. The argument follows by noting that when the prescribed value of information is below the mean of the uniform distribution of SPs, SPs that exceed the MBPs will be observed on the majority of the trials. Information will be purchased on a fraction of the trials in this case, the larger the difference between the mean SP and the prescribed value the
SEALED BIDDING EXPERIMENT
283
smaller the fraction. Conversely, when the mean of the SP distribution is below the prescribed value, MBPs that exceed the SPs will be observed on most trials and information will be purchased. Believing that mean SPs and mean MBPs are not far apart, the subjects might have adjusted their MBPs in the direction shown in Fig. 1. Both of these explanations are testable in principle. Flat maxima may be overcome in principle by increasing the effectiveness of the reward structure, though this may prove to be a tedious or very expensive task. Similarly, MBPs may be assessed directly or by the Marschak bidding procedure with distributions of SPs, uniform or not, with various ranges. For example, a distribution of SPs with a mean that equals or slightly exceeds the prescribed value of information for a given distribution by information condition may eliminate the possible artifact suggested above.
Bids When the subjects bid without knowledge of cost of LCB, their bids were significantly below the prescribed bids. Otherwise, with the exception of order effects found in one of the analyses, which mostly reflect within task learning, the bids were affected by the experimental factors in the direction prescribed by the model. Thus, the bids for DC1 exceeded those for DC2 as predicted, and the differences between the bids of IC3 and IC4 were not significant for each of the two distribution conditions. These results were obtained when the subject bid without information about cost or LCB, whether or not information could be purchased. When the subjects bid following purchasing of information about p (group IC1) and the differences between prescribed and observed bids were analyzed, no effects due to distribution, information, or order were predicted. The ANOVA substantiated these predictions but again revealed persistent underbidding. When the subjects bid after purchasing information about t (group IC2), they followed the optimal policy of bidding just below t if t > /5 and not bidding ift ~
284
S T E V E N M. M E D L I N
and consequently attain a positive probability of winning the contract. It is suggested that the demand characteristics of the task might have influenced the subjects in DC2 to bid on the trials where no information was available or purchased. Under such circumstances they could only underbid. In DC1, on the other hand, a subject could either overbid or underbid. The underbidding observed in this distribution condition can be shown to be consistent with maximization of expected utility with concave utility functions. Thus, for example, if a logarithmic rather than linear utility function is assumed for the case of no information, the optimal bid is about 201 rather than 225. Similarly, a concave power utility function yields a range of prescribed bids between 200 and 225, depending on the values assumed by the power parameter of the function, and can, consequently, account for 14 of the 16 mean bids presented in Tables 3 and 5. Clearly, the explanation in terms of maximization of expected utility is ad hoc without independent assessments of the subjects' utility functions. A more plausible explanation of the pervasive underbidding, that applies to both distribution conditions, attributes it to an objective criterion that considers jointly both the expected value (or utility) of the task and probability of winning the contract. Since underbidding increases the probability of winning the contract, the subjects might be willing to sac,
i
llO
120
,
,
,
,
,
,
i
'
/
0 -20 -40
-60 i
-BO -I00 3O 20 10 0 -10 .20 -30 -40 -50 100
130
140
150
160
I
170
I
1BO
I
190
200|
I
I
I
I
I
I
I
I
I
I
120
140
160
180
200
220
240
260
280
300
/t
BIDS WITH NO INFORMATION
FIG. 2. E x p e c t e d value as a function of bid for the case of no information.
SEALED BIDDING EXPERIMENT
285
rifice a portion of their expected profit to increase their probability of winning the contract. The flatness of the expected value functions for the case of no information guarantees that the magnitude of sacrifice would be small. Figure 2 displays two expected value functions for the case of no information, one for each distribution condition; their maxima are indicated in the figure by small arrows. Figure 2 shows that decreasing the bid from 225 to 200 in DC1 would cause a relative expected loss in profit of only 4% but increase the probability of winning the contract by 12.5% from 3/8 to 4/8. In DC2, decreasing the bid from 200 to 175, for example, would cause a relative expected loss of 6.25% in profit; the probability of winning the contract in this case would increase from 0 to .25. Two extensions of the SBT paradigm may enhance its appeal and applicability. The first relaxes the requirement of perfect information with regard to cost or LCB and allows for purchasing information that only reduces by a specified amount, but does not eliminate completely, the uncertainty about one or both of the task variables. The second extension permits the subject to bid against several nonfictitious opponents and provides him with the option of purchasing information that either reduces or eliminates uncertainty about the bids of a subset of his opponents rather than about the lowest competitive bid. REFERENCES Becker, G. M., DeGroot, M. H., & Marschak, J. Measuring utility by a single-response sequential method. Behavioral Science. 1964, 9, 226-232. Becker, G. M., & McClintock, C. G. Value: Behavioral decision theory. Annual Review o f Psychology, 1967, 18, 239-286. Fried, L. S., & Peterson, C. R. Information seeking: Optional versus fixed stopping. Journal o f Experimental Psychology, 1969, 80, 525-529. Howard, R . A . Information value theory. 1EEE Transactions on Systems Science and Cybernetics, 1966, SSC-2, 22-26. Howard, R. A. Value of information lotteries. IEEE Transactions on Systems Science and Cybernetics, 1967, SSC-3, 54-60. Howard, R. A. The foundation of decision analysis. 1EEE Transactions on Systems Science and Cybernetics, 1968, SSC-4, 1-9. Larsson, B. Bayes strategies and human information seeking. Lurid CWK Gleerup: Univ. of Lurid Press, 1968. Marschak, J. Remarks on the economics of information. In Contribution to scientific research in management. Los Angeles: Western Data Processing Center, University of California, 1959. Medlin, S. M. Information value in a sealed bidding experiment. Unpublished doctoral dissertation, University of North Carolina at Chapel Hill, 1976. O'Connor, M. F., Peterson, C. R., & Palmer, T. J. Stakes and probabilities in information purchase. Organization Behavior and Human Performance, 1972, 7, 43-52. Pitz, G. E. Information seeking when available information is limited. Journal o f Experimental Psychology, 1968, 76, 25-34. Raiffa, H., & Schlaifer, R. Applied statistical decision theory. Boston: Harvard Business School, Division of Research, 1961.
286
STEVEN M. MEDLIN
Rapoport, A. Research paradigms for studying dynamic decision behavior. In D. Wendt & C. Vlek (Eds.), Utility, probability, and human decision making. Dordrecht: Reidel, 1975. Shanteau, J., & Anderson, N. H. Integration theory applied to judgements of the value of information. Journal of Experimental Psychology, 1972, 92, 266-275. Shuford, E. H. Some Bayesian learning processes. In M. W. Shelly II & G. L. Bryan (Eds.), Human judgements and optimality. New York: Wiley, 1964. Tversky, A., & Edwards, W. Information versus reward in binary choices. Journal of Experimental Psychology, 1966, 71,680-683. Tversky, A., & Kahneman, D. Judgement under uncertainty: Heuristics and biases. In D. Wendt & C. Vlek (Eds.), Utility, probability, and human decision making. Dordrecht: Reidel, 1975. Wald, A. Sequential analysis. New York: Wiley, 1947. Wendt, D. Value of information for decisions. Journal of Mathematical Psychology, 1969, 6, 430-443. RECEIVED: October 19, 1976
24, 269-286 (1979)
The Value of Information in a Sealed Bidding Experiment STEVEN M. M E D L I N U.S. Army Research Institute fi~r the Behavioral and Social Sciences The present study assesses directly the value of perfect information from two different data sources to a decision maker confronted with the sealed bidding task. Predictions about the value of information that eliminates uncertainty and the subsequent bid are derived following Howard's (1EEE Transactions o f Systems Science and Cybernetics, 1966, SSC-2, 22-26) analysis of the value of information. A 2 × 4 × 2 factorial design varied (i) the distributions of the data sources, (ii) the kind of information purchasable, and (iii) the order of presentation for blocks of trials where information could or could not be purchased. Each of 80 subjects performed the sealed bidding task for a series of 72 trials. The results show an almost linear relationship between the observed maximum price offered for information and the prescribed value of the information. Furthermore, subjects consistently underbid in all information conditions. Deviations from optimality are interpreted in terms of the subjects' limited information processing capacities and the nature of the decision task.
When the outcome of a decision depends upon the state of nature that obtains, it is desirable and often possible to reduce uncertainty by acquiring information about the relevant states prior to making the decision. The information is almost always costly in terms of efforts for obtaining it, time for processing it, or resources for purchasing it. There is certainly no advantage in acquiring information when the cost of additional information is greater than the consequent increase in expected gain (Fried & Peterson, 1969). Two general classes of models have dealt with the value of information in uncertain decision situations. The first class of models specifies criterion functions for a variety of decision making situations and then derives models for information purchasing that optimize these functions (Howard, 1966, 1967; Marschack, 1959; Raiffa & Schlaiffer, 1961; Wald, 1947). In these information purchasing tasks, the cost per unit is prespecified and the question under investigation is how many informational units are purchased before a terminal decision is made (Becker & McClintock, 1967; Fried & Peterson, 1969; Larsson, 1968; O'Connor, Peterson, & Palmer, 1972; Pitz, 1968; Tversky & Edwards, 1966). The second class of models assumes a fixed and known amount of information to be purchased and specifies the demand price or value of information that optimizes the prespecified critePlease send request for reprints to Steven M. Medlin, U.S. Army Research Institute, 5001 Eisenhower Ave., Alexandria, VA 22333. 269 0030-5073/79/060269-18502.00/0 Copyright © 1979 by Academic Press, Inc. All rights of reproduction in any form reserved.
270
STEVEN
M. MEDLIN
rion function (Marschak, 1959). Wendt (1969) and Shanteau and Anderson (1972) fixed the amount of information to be purchased and assessed the value of a single datum of information to the subject in a simple information purchasing task (Becker, DeGroot, & Marschak, 1964). The results of both studies indicate that subjects vary their estimates in the predicted directions as a function of prior odds, payoffs, and data source diagnosticity; furthermore, the subjects tend to overestimate the prescribed value of the information when it is relatively low or near zero. The present study is also concerned with the assessment of the value of information, but differs from the work of Wendt (1969) and Shanteau and Anderson (1972) in two significant ways. First, the present study provides two sources of uncertainty (and thus two sources of information), while in previous experiments only one source of uncertainty was available. Several interesting research questions are raised by the availability of two sources of information. What is the value of information that reduces uncertainty about qualitatively different aspects of a decision making task? What is the relationship between the sum of individual values of information, each of which is associated with a different source of uncertainty, and the value of information that eliminates the uncertainty from both these sources jointly? The second difference between this study and earlier information purchasing studies concerns the diagnosticity of the information. The present study provides perfect information, eliminating all uncertainty; in previous studies the information could reduce uncertainty, but not eliminate it. The concept of perfect information is useful in analyzing the information phase of the decision task; the value of perfect information on a task variable represents the maximum amount that a decision maker should pay to eliminate uncertainty regarding the variable (Howard, 1966, 1968). The purpose of the present research is to study information purchasing behavior in a decision task that requires the subject to assess directly the value of information that reduces uncertainty about one or more task variables. To provide an information purchasing task in which subjects directly assess the value of one or more items of information, the sealed bidding task was used. A brief, general example will serve to illustrate the task. Consider a construction firm which is bidding on a contract to build a bridge. Company A bids on the contract against several competitors. The company's actual cost of performing the contract (i.e., building the bridge) is denoted by p; as is typically the case, the company is uncertain of this exact cost. The lowest bid of the competitors is denoted by t; this exact value too is uncertain. Given these uncertainties, assumed to be characterized by two independent probability distributions, the problem confronting company A is to determine its bid, b, so as to maximize the expected value, E(v), of the contract.
SEALED BIDDING EXPERIMENT
271
By investing certain resources in research and development it is assumed that the company can completely reduce the uncertainty regarding p. The company must decide, therefore, how much to invest for gathering information that reduces the uncertainty regarding p and, if it obtains this information, how much to bid for the contract. Similarly, and independent of its first problem, the company may invest certain resources in assessing the financial situation of its competitors, their past bidding records, or may even become involved in industrial spying to learn the competitors' bids, and consequently, the exact value of the lowest bid, t. Company A must decide, therefore, how much money, if any, to invest in order to secure this information, and given t, how much to bid for the contract. Finally, if company A decides to invest its resources in reducing uncertainty about both p and t simultaneously, it must decide how much to invest to obtain this information, and having determined the exact values of p and t, how much to bid for the contract. Howard (1966) has employed this same bidding problem to illustrate his approach to the theory of the value of information. He demonstrated that numerical values could be assigned to the reduction of uncertainty, and showed how to calculate the optimal values of various sources of information in the bidding problem. He also determined the optimal bid, the one which maximizes the expected utility of the contract, when various items of information are known. (Table 1 presents the optimal values for the specific task under the present study.)
METHOD
Subjects Eighty undergraduate students at the University of North Carolina at Chapel Hill served as subjects. They were recruited by posting advertisements offering monetary rewards for participation in a decisionmaking experiment.
The Sealed Bidding Task The subjects participated individually in a computerized version of the sealed bidding task. After reading the instructions, and receiving a short verbal explanation (Medlin, 1976), which provided a cover story for the task, explained the characteristics of the cost (p) and lowest competitive bid (t) distributions, and led the subject through several examples, each subject was presented with his own cost functionf(p), lowest competitive bid (LCB) functionf(t), and the (fictitious) number of his opponents. The subject was also told what kind of perfect information, if any, could be purchased, either no information, information about cost, information about LCB, or information about both cost and LCB jointly. A single trial in the computerized sealed bidding task proceeds in the following manner. If a subject was not allowed to purchase information,
272
S T E V E N M. M E D L I N
he started the trial by placing a bid for the contract. When information could be purchased, a computerized version of the Marschak bidding procedure (Becker, DeGroot, & Marschak, 1964) was used to elicit the value of the information to the subject. Depending upon the information condition, to be described below, the information purchased by the subject was either a p value randomly selected from the cost distribution, a t value randomly selected from the LCB distribution, or both. The Marschak bidding p r o c e d u r e required the subject to state his maximum buying price (MBP) for the information. The MBP was then compared to a selling price (SP), an integer randomly selected from a uniform density function (0,50). If MBP < SL the subject did not purchase the information and placed his bid for the contract without it. If MBP >I SL, the subject purchased the information and proceeded to place his bid. As shown by Becker et al. (1964), this procedure possesses the property that the subject maximizes his subjectively expected value of the transaction by stating a MBP equal to his cash-equivalent of the information. After typing his bid, b, for the contract, the computer compared b to t. If b /> t, the subject did not win the contract; his p a y o f f in this case was zero minus the price paid for the information, if it was purchased. If b < t, the subject won the contract; his p a y o f f in this case consisted of b - p minus the price paid for the information, if it was purchased. At the end of each trial the subject was informed of the p value for that trial, the bids of his opponents, and his gain (positive or negative) for that trial. One of the competitor's bids was the predetermined t value randomly selected fromf(t); the other bids were integers randomly selected from a uniform density function over the interval (t, 350).
Experimental Design The experiment employed a 2 x 4 x 2 distribution by information by order factorial design. There were two distribution conditions, DC1 and DC2, each defined by a pair of cost (p) and LCB (t) uniform density functions. For both distribution conditions/91 = t i = 100. F o r DC1 192 = 200 and tz = 300, whereas for DC2/92 = 300 and t2 = 200. Four information conditions, IC1, IC2, IC3, and IC4, defined the second factor of the design: IC1 IC2 IC3 IC4
Information Information Information Information
about/9 about t about/9 about/9
may be purchased, m = 1, may be purchased, m = 1, and t may be purchased, m = 1, and t may be purchased, m = 5,
where m denotes the number of competitors. Subjects in IC4 were treated exactly like subjects in IC3 except that in the former condition they were informed that they were bidding against five competitors. Since the computer program for the SBT allows for the simultaneous running of up to
273
SEALED BIDDING EXPERIMENT
six subjects, each with his own teletypewriter in a time-sharing mode (Medlin, 1976), the deception in IC4 could not be detected. The third factor varied the order in which a block of 24 trials on which information c o u l d not be purchased (type A) and a block of 48 trials on which information c o u l d be purchased (type B) were presented. In order condition 1 (OC1) the subjects were first presented with 24 type A trials followed by 48 type B trials. In order condition 2 (OC2) these two blocks were presented in reverse order. Five subjects were run simultaneously in a given session with each group of subjects randomly assigned to 1 of the 16 cells of the design. A session lasted approximately 2 hr. Upon completing the 72 trials, the total number of points earned by the subject was exchanged for money at the rate of 0.4 cents per point. The mean gain per subject was $6.79. Each subject completed a postexperimental questionnaire (see Medlin, 1976), which will not be analyzed here, and was dismissed. RESULTS The data analysis is divided into two major sections. The first will consider the value of the information to the subject. The second section will be concerned with three classes of bids: bids made when information could not be purchased, bids made when information could be but was not purchased, and bids made after purchasing information. In the sealed bidding task, the value of the information to the decision maker is defined as the maximum amount a subject is willing to pay to purchase the item of information (MBP). Following Howard's (1966) approach to the value of information, the prescribed values of information and bids for the present experimental conditions were obtained (Table 1). Several predictions are implied by the model about the effects of the TABLE 1 VALUE OF INFORMATION AND OPTIMAL BID BY INFORMATION AND DISTRIBUTION CONDITIONS Information condition
Prescribed values
Distribution Condition DC 1"
DC2
1.04 ((Cost + 300)/2)
8.33 ((Cost + 300)/2)
IC 1 (cost)
Information Bid
IC2 (LCB)
Information Bid
28.12 If* t > p, bid t - 1 If* t ~< p, bid above t
IC3 and IC4 (both)
Information Bid
30.21 16.67 If t > p, bid t - 1 I f t <~ p, bid above t
*~ equals 150 for DC1 and 200 for DC2.
0
274
S T E V E N M. M E D L I N
experimental variables on the MBPs. As Table 1 shows, the model predicts two significant main effects, one due to the distribution factor and one due to the information factor, and a significant distribution by information interaction. Since the LCB is not affected by the n u m b e r of competitors, the MBPs for groups IC3 and IC4 are predicted to be the same for both distribution conditions. Since between trial learning is expected in multiple trial decisionmaking experiments, the 48 type B trials were divided into two equal blocks. The main MBP was computed separately for each block of 24 type B trials and each subject. An A N O V A was conducted on these means, treating blocks as a repeated factor with two levels. Although there was no main effect due to learning (F(1,64) = 1.26, p < .265), the analysis revealed a significant distribution by block interaction (F(1,64) = 8.67, p < .005). The mean MBP in DC1 decreased from 20.20 in the first block of 24 trials to 18.62 in the second block, whereas the mean MBP in DC2 increased from 11.75 to 12.46. Since the distribution by block interaction was significant, two separate 2 x 4 x 2 distribution by information by order A N O V A s were conducted on the individual mean MBPs, one for each block of 24 type B trials. As predicted, only the main effects due to distribution (first block, F(1,64) = 25.20, < p .001; second block, F(1,64) = 14.72, p < .001) and information (first block, F(3,64) = 2.85, p < .039; second block, F(3,64) = 4.59, p < .006) and the distribution by information interaction (first block, F(3,64) = 4.83, p < .004; second block, F(3,64) = 8.06, p < .001) were significant. The means and standard deviations of the MBPs are presented in Table 2. The results are presented separately for each block of 24 type B trials, distribution condition, and information condition. A comparison of Tables 1 and 2 shows that the M B P s - - t h e subjects' estimates of the values of information--varied consistently in the same direction as did the prescribed values. The subjects offered a higher MBP for information in DC1 than in DC2, as would be expected from Table 1. Similarly, for each distribution condition the means of the MBPs for the four information conditions were ordered as predicted (with a single exception for groups IC1 and IC3 in distribution c o n d i t i o n 2). T h e p r e d i c t e d i n t e r a c t i o n effect between information and distribution was also realized. Thus, for example, subjects in DC1 were willing to pay a considerably higher price for information about the LCB (20.88 and 21.02 for blocks 1 and 2, respectively), whose prescribed value was 28.12, than subjects in DC2 (6.45 and 6.43 for blocks 1 and 2, respectively) for whom the prescribed value of the same information was zero. A closer inspection of Tables 1 and 2 and analyses of signed deviations between prescribed and observed values of information (see Medlin, 1976) revealed another finding similar to one reported by Wendt (1969). In
275
SEALED BIDDING EXPERIMENT TABLE MEANS
AND STANDARD
DEVIATIONS
2
OF MAXIMUM
AND DISTRIBUTION
BUYING
P R I C E S BY I N F O R M A T I O N
CONDITIONS
Distribution condition DC 1 Information condition
DC2
Block 1
Block 2
Total
Block 1
Block 2
Total
IC1
Mean SD
12.51 11.05
8.14 8.03
10.32 9.77
14.12 5.34
15.26 5.40
14.69 5.37
IC2
Mean SD
20.88 5.94
21.02 7.15
20.95 6.57
6.40 5.71
6.45 5.74
6.43 5.73
IC3
Mean SD
25.18 8.18
23.42 8.65
24.30 8.42
11.65 7.62
11.63 8.64
11.64 8.16
IC4
Mean SD
22.25 6.92
21.88 6.89
22.06 6.91
14.84 4.49
16.52 3.73
15.68 4.13
20.20
18.62
19.41
11.75
12.46
12.11
Overall m e a n
general, the MBP of perfect information exceeded its prescribed value when the latter was relatively low, whereas the reverse relationship obtained when the prescribed value was relatively high. The mean MBPs of information for each distribution and information group, computed over the two blocks of 24 trials each and the two order conditions, are plotted against the prescribed values in Fig. 1. There is a monotonically increas-
i
I
i
t
t
i
210
215
ADISTRIBUTION CONDITION 1 • DISTRIBUTION CONDITION 2
25
2O ~a a.
m >-
_= IE
~E
0
i
0
i
5
i
10
lIS
310
OPTIMAL VALUE OF INFORMATION FIG. 1. M a x i m u m buying price as a function of prescribed information value.
276
S T E V E N M. M E D L I N
ing, almost linear relationship between the two values. The correlation between the MBPs and predicted values is .933 (p < .01) with a slope and intercept of .467 and 8.078, respectively. Similar results were obtained when the correlations between the MBPs and the prescribed values were computed for each block of 24 trials separately (the correlation, slope, and intercept were .904, .455, and 8.521, respectively, for block 1, and .940, .482, and 7.633 for block 2). When the same decision task is repeated for many trials and complete feedback is provided at the end of each trial, sequential effects are anticipated. In the present study, success or failure in winning the contract or in purchasing the information on a given trial might have affected the MBP on subsequent trials. To study the effects of winning or losing the contract on the MBPs, difference scores were computed for each subject between the mean MBP on trials that immediately followed a successful bid and the mean MBP on trials immediately following an unsuccessful bid. An A N O V A conducted on these difference scores failed to yield any significant effect. In particular, the null hypothesis of zero mean difference score was not rejected. The effects of information purchasing on the subsequent MBPs were then examined in a similar manner. The difference between the mean MBP for information following trials on which information was purchased and the mean MBP following trials on which the attempt to purchase information was unsuccessful was computed separately for each subject. An A N O V A conducted on these difference scores revealed that subjects paid more for information after successfully purchasing it on the immediately preceding trial than after an unsuccessful attempt (F(1,64) = 5.97, p < .017). Bids No information could be purchased on 24 of the 72 trials. To test for learning effects in bids, these 24 type A trials were divided into two equal blocks. Mean bids were computed for each block of 12 trials and each subject; and an A N O V A was conducted on these means, treating blocks as a repeated factor with two levels. Having established no significant learning effects (Medlin, 1976, Table 8), the mean bids were then computed for each subject over the 24 type A trials, and an A N O V A was conducted on these overall means to test for the main effects of the three experimental factors and their interactions. The distribution main effect (F(1,64 = 39.029, p < .001), the order main effect (F(1,64) = 6.130, p < •016), and the information by order interaction effect (F(3,64) = 2.898, p < .042) were all significant at the .05 level. Table 3 presents the means and standard deviations of the bids for the 24 type A trials. For these bids the model predicts a significant effect due to the distribution condition; the optimal bids for DC1 and DC2 are 225
277
SEALED BIDDING EXPERIMENT TABLE 3 MEANS AND STANDARD DEVIATIONS OF BIDS WHEN INFORMATION COULD NOT BE PURCHASED Distribution condition DC 1 Information condition
DC2
OC 1
OC2
OC 1
OC2
IC 1
Mean SD
204.6 19.4
218.6 8.0
172.7 7.5
171.7 13.1
IC2
Mean SD
202.2 14.9
216.9 21.9
179.2 6.1
189.6 18.9
IC3
Mean SD
188.7 12.8
218.8 24.7
175.8 10.0
196.4 22.1
IC4
Mean SD
201.9 14.4
202.8 22.0
194.8 26.9
181.5 14.6
199.4
214.3
180.6
184.8
Overall mean
and 200, respectively. Inspection of Table 3 reveals that the direction of the significant distribution effect is as predicted, with higher mean bids for DC1 than DC2. The mean bids, however, are smaller than the optimal bids for all the 16 experimental groups. The significant order effect, not accounted for by the model, is shown clearly in Table 3. The mean bids for information conditions 1, 2, and 3 are higher in OC2 than OC1. This relationship is reversed for information condition 4 in distribution condition 2; hence the marginally significant interaction effect. The significant order effect is of interest in showing that subjects with information purchasing practice bid higher than subjects who first bid on the no information trials. As indicated above, the subjects underbid for the contract in all 16 experimental groups. To test for this difference between observed mean bids and optimal bids, signed deviation scores were computed for each subject and then subjected to a 2 x 4 x 2 distribution by information by order ANOVA. There was a significant order main effect (F(1,64) = 6.763, p < .012), but no distribution or distribution by order effects as in the preceding analysis. The overall mean of the signed deviation scores was significantly different from zero (F(1,64) = 82.550, p < .001), supporting the previous observation of persistent underbidding. Information was purchasable on 48 of the 72 trials. Due to the nature of the Marschak bidding procedure, however, information was not purchased on some of these trials. Table 4 shows the means and standard deviations of the number of trials (out of 48) on which information was purchased. Since the uniform function from which the SPs were randomly
278
STEVEN M. M E D L I N TABLE 4 MEANS AND STANDARD DEVIATIONS OF NUMBER OF TRIALS ON WHICH INFORMATION WAS PURCHASED Distribution condition DC 1
Information condition
DC2
OC 1
0C2
OC 1
0C2
IC1
Mean SD
10.40 8.53
10.40 10.29
15.20 4.97
14.40 5.86
IC2
Me an SD
20.20 8.41
19.00 9.03
4.60 3.36
6.20 6.14
IC3
Mean SD
25.60 8.08
22.80 10.50
13.40 8.05
9.40 8.50
IC4
Mean SD
16.60 6.66
26.20 9.42
14.40 4.16
15.80 2.68
chosen was the same for all 16 groups, the means in Table 4 are e x p e c t e d to v a r y considerably f r o m one group of subjects to another, as they actually do. Subtracting the means presented in Table 4 f r o m 48 yields the mean n u m b e r of trials on which information could be but was not purchased. The analyses conducted on the bids made on these trials were analogous to those conducted immediately a b o v e on the bids w h e n no information could be purchased. Thus, an A N O V A was conducted on the m e a n bids taken o v e r all trials on which information was available but not purchased. The A N O V A yielded a significant distribution effect (F(1,64) = 59.94, p < .001); no other main or interaction effect, including the order main effect and the order by information interaction effect o b s e r v e d above, were significant at the .05 level. The difference b e t w e e n the two A N O V A results with respect to the order and order b y information effects is interpretable. N o t e that the 24 type A trials a p p e a r e d as a single block either at the beginning or at the end of the task. The trials on which information could be but was not p u r c h a s e d were spread out a m o n g either the first or the last 48 type B trials, with the 24 middle trials overlapping. Consequently, to the extent that order has an effect it would be e x p e c t e d to be stronger in the f o r m e r than in the latter case. Table 5 presents the means and standard deviations of bids on trials on which information could be but was not purchased. The significant distribution main effect, with higher means for DC1 than DC2, and the lack of information and order main effects are clearly shown in the table. The table also shows that, with one exception, the o b s e r v e d m e a n bids are smaller than the prescribed bids (225 and 200 for DC1 and DC2, respec-
279
SEALED BIDDING EXPERIMENT TABLE 5 MEANS AND STANDARD DEVIATIONS OF BIDS WHEN INFORMATION COULD BE BUT WAS NOT PURCHASED Distribution condition DC 1 Information condition
DC2
OC 1
OC2
OC 1
OC2
IC1
Mean SD
214.0 22.4
228.6 17.6
172.7 7.3
167.5 21.2
IC2
Mean SD
216.1 10.8
214.1 22.2
181.6 6.7
188.8 19.0
1C3
Mean SD
204.8 18.9
2!2.8 27.1
180.7 12.1
186.3 15.0
IC4
Mean SD
213.6 14.4
200.8 21.9
194.3 25.3
180,6 15.4
212.1
214.1
182.3
180.8
Overall mean
tively). An A N O V A conducted on the signed deviation scores, c o m p u t e d separately for each subject, confirmed this o b s e r v a t i o n by showing that the overall m e a n was significantly different from zero (F(1,64) = 54.854, p < .001). The A N O V A conducted on the signed deviation scores also yielded a marginally significantly distribution b y information interaction effect, which m a y be attributed to chance. T w o sets of bids have b e e n analyzed, bids made on type A trials and bids made by the same subjects on trials on which information could be but was not purchased. In addition to the availability of purchasable information, the two sets differ f r o m each other, as we h a v e already noted when discussing the presence or absence of order effects, in terms of the order of a p p e a r a n c e of the trials in the sequence. To test for differences b e t w e e n these two sets of bids, the m e a n bid of each set was c o m p u t e d separately for each subject. An A N O V A was conducted on these means, treating each set of bids as a repeated factor with two levels. The overall means did not differ f r o m one set to another but there were distribution by set (F(1,64) = 7.578, p < .008) and order by set (F(1,64) = 10.196, p < .002) significant interaction effects. The latter interaction effect is due to a significant order effect that has already been reported for the first but not the second set of bids. As a c o m p a r i s o n of the overall m e a n s in Tables 3 and 5 shows, the significant distribution b y set interaction effect is due to a larger difference in the means of DC1 and DC2 in the first than the second set of bids. The analyses of bids w h e n information was p u r c h a s e d are divided into three parts, depending on whether the value o f p (group IC 1), the value of
280
S T E V E N M. M E D L I N
t (group IC2), or the values of both p and t (groups IC3 and IC4) were known exactly. W h e n p is known exactly, an analysis of the observed bids is uninformative; it is the discrepancy between observed and predicted bids that is of interest. Following H o w a r d ' s (1966) analysis, the optimal bid was computed for each trial on which p was known and the observed bid was subtracted. The resulting signed deviation scores, computed separately for each of the 20 IC1 subjects, were averaged, and the resulting means were subjected to a 2 × 2 distribution by order A N O V A . The overall mean of the signed deviation scores, -14.58, was significantly different from zero (F(1,16) = 10.828, p < .001), but no other effects were significant. The results thus show that when cost of performing the contract was known subjects still bid significantly below the optimal level. Although the subjects in group IC1 underbid both when information about p was purchased and when it was not, the analysis just conducted does not address the question of whether information about cost affected their bids. To answer this question, correlations between cost and bid were computed separately for each of the 20 subjects in group IC 1 for trials on which information was purchased. Four subjects did not purchase sufficient information for a correlation coefficient to be computed. Fourteen of the remaining sixteen correlations were significant at the .05 level. Thus subjects employed the information about cost to determine their bids; they just did not use it optimally, showing the same tendency to underbid. I f t is known exactly, the optimal policy is to bid just below t ift >/3 and above t if t ~< ~. There were 20 IC2 subjects altogether who could and did purchase information about t, 10 in DC1, and 10 in DC2. The total number of trials on which information about t was purchased, computable from the means presented in Table 5, was 196 for the 10 subjects in DC1 and 54 for the 10 in DC2. On 185 of the 196 trials in DC1 the optimal bidding policy was followed. The results for DC2 (where information about t is worthless) were different; the optimal bidding policy was followed on only 29 of the 54 trials. The difference between these two groups of 10 subjects is attributed to the difference in prescriptions of the bidding policy. The subjects who purchased information about t under DC 1 should or should not bid depending on whether t > ,~ or t <~p (13 = 150 and 100 ~< t ~< 300 for this condition). H o w e v e r , the subjects who purchased information about t under DC2 should never bid for the contract since it was always the case that t ~
SEALED
BIDDING
EXPERIMENT
281
puted. Of the remaining 16 correlations, 14 were significant at the .05 level. If both cost and LCB are known, the optimal bidding policy is trivial (see Table 1), namely, to bid just below t if t > p, and above t if t ~< p. Forty subjects, 20 in IC3 and 20 in IC4, had opportunity to purchase information about both p and t. The subjects of IC3 used this opportunity on 356 trials (see the means on Table 4 from which these frequencies can be computed) and the subjects of IC4 on 362 trials. With very few exceptions (4 of 356 in IC3 and 6 of 362 in IC4), the bids were optimal.
DISCUSSION The present approach to model testing and building follows the research methodology outlined by Rapoport (1975). Using the maximization of expected utility model for the scaled bidding task as an " i d e a l " or baseline, the discrepancies between prescribed and observed bids and values of information (MBP) are explored. It is assumed that the dis, crepancies will be neither so large nor so unsystematic as to make the comparison between prescribed and observed behavior meaningless or useless. If systematic discrepancies are found, they may be interpretable in terms of cognitive or environmental constraints imposed on the subject (Shuford, 1964), or heuristics that the subject employs to reduce the cognitive complexity of the task (Tversky & Kahneman, 1975).
Value of Information With the exception of learning effects, the MBPs support the predictions of the model in their direction but not in their values. The model predicts significant distribution, information, and distribution by information effects in the values of information, and no significant main or interaction effects due to order of presentation or to the (fictitious) number of opponents. The ANOVA results support all of these predictions. There were significant learning and sequential effects, which are not accounted for by the single-stage model, but are not unexpected in view of the relatively small number of trials, which did not allow the subjects to reach an asymptote. The discrepancy between observed and prescribed values of information is displayed in Fig. 1. Consistent with the results reported by Wendt (1969), Fig. 1 exhibits overestimation of relatively low values and underestimation of relatively high values of information. The function relating the MBPs to the prescribed values is approximately linear in form; the linear correlation accounts for 87% of the variance. Inspection of the results reported by Wendt (1969, Figs. 1 and 2), who employed a different procedure in his experiments, reveals a similar linear relationship. At least two major explanations may be proposed for the finding displayed in Fig. 1. The first attributes this discrepancy to criteria adopted by
282
S T E V E N M. M E D L I N
some or all of the subjects that differ substantially from the one assumed by the model. Thus, the subjects might have possessed nonlinear utility functions and attempted to maximize expected utility rather than expected value. Alternatively, some or all of the subjects might have adopted more complex criteria such as maximization of probability of winning the contract subject to attaining a fixed expected value (or utility). It has frequently been noted that "discriminability among various objective criteria that subjects may adopt is difficult to achieve, since the maxima derived from optimization models are often very flat, and fairly substantial deviations from optimal decision behavior typically produce only extremely small changes in DM's (decision maker's) economic prospects" (Rapoport, 1975, p.365). Flat maxima were also demonstrated by Wendt in his experiments and proposed by him as an explanation for the failure of his subjects to match the true value of information in their decisions. Flat maxima may account for erratic decision behavior but not for systematic biases. The present study goes beyond Wendt's in showing that when the subjects deviated from optimality it was in the direction providing for smaller losses. Results reported elsewhere (Medlin, 1976) show that in those cases where the subjects offered too much for information, when the prescribed values were 0 or 1.04 (see Table 1), the expected value functions dropped more slowly for numbers exceeding the prescribed values than for numbers smaller than the prescribed values. Conversely, when the prescribed values were relatively high (information conditions 2, 3, and 4 in DC1) the expected value functions dropped slowly for numbers less than the prescribed values but dropped considerably more steeply for numbers exceeding the prescribed values when the observed MBPs were too low. Finally, when the expected value functions tapered off rather sharply on both sides of the maximum, the MBPs were close to the prescribed values, as in information conditions 1, 3, and 4 in DC2. A second explanation attributes the discrepancy between observed and prescribed values to the way the Marschak bidding procedure was employed. This procedure possesses the theoretical property that it is always optimal for a subject to state his fair price of the information as his MBP. We suspect, however, that by using a uniform distribution of integral SPs with the same range for all 16 experimental groups we might have biased the results by causing a regression toward the mean of this distribution. The argument follows by noting that when the prescribed value of information is below the mean of the uniform distribution of SPs, SPs that exceed the MBPs will be observed on the majority of the trials. Information will be purchased on a fraction of the trials in this case, the larger the difference between the mean SP and the prescribed value the
SEALED BIDDING EXPERIMENT
283
smaller the fraction. Conversely, when the mean of the SP distribution is below the prescribed value, MBPs that exceed the SPs will be observed on most trials and information will be purchased. Believing that mean SPs and mean MBPs are not far apart, the subjects might have adjusted their MBPs in the direction shown in Fig. 1. Both of these explanations are testable in principle. Flat maxima may be overcome in principle by increasing the effectiveness of the reward structure, though this may prove to be a tedious or very expensive task. Similarly, MBPs may be assessed directly or by the Marschak bidding procedure with distributions of SPs, uniform or not, with various ranges. For example, a distribution of SPs with a mean that equals or slightly exceeds the prescribed value of information for a given distribution by information condition may eliminate the possible artifact suggested above.
Bids When the subjects bid without knowledge of cost of LCB, their bids were significantly below the prescribed bids. Otherwise, with the exception of order effects found in one of the analyses, which mostly reflect within task learning, the bids were affected by the experimental factors in the direction prescribed by the model. Thus, the bids for DC1 exceeded those for DC2 as predicted, and the differences between the bids of IC3 and IC4 were not significant for each of the two distribution conditions. These results were obtained when the subject bid without information about cost or LCB, whether or not information could be purchased. When the subjects bid following purchasing of information about p (group IC1) and the differences between prescribed and observed bids were analyzed, no effects due to distribution, information, or order were predicted. The ANOVA substantiated these predictions but again revealed persistent underbidding. When the subjects bid after purchasing information about t (group IC2), they followed the optimal policy of bidding just below t if t > /5 and not bidding ift ~
284
S T E V E N M. M E D L I N
and consequently attain a positive probability of winning the contract. It is suggested that the demand characteristics of the task might have influenced the subjects in DC2 to bid on the trials where no information was available or purchased. Under such circumstances they could only underbid. In DC1, on the other hand, a subject could either overbid or underbid. The underbidding observed in this distribution condition can be shown to be consistent with maximization of expected utility with concave utility functions. Thus, for example, if a logarithmic rather than linear utility function is assumed for the case of no information, the optimal bid is about 201 rather than 225. Similarly, a concave power utility function yields a range of prescribed bids between 200 and 225, depending on the values assumed by the power parameter of the function, and can, consequently, account for 14 of the 16 mean bids presented in Tables 3 and 5. Clearly, the explanation in terms of maximization of expected utility is ad hoc without independent assessments of the subjects' utility functions. A more plausible explanation of the pervasive underbidding, that applies to both distribution conditions, attributes it to an objective criterion that considers jointly both the expected value (or utility) of the task and probability of winning the contract. Since underbidding increases the probability of winning the contract, the subjects might be willing to sac,
i
llO
120
,
,
,
,
,
,
i
'
/
0 -20 -40
-60 i
-BO -I00 3O 20 10 0 -10 .20 -30 -40 -50 100
130
140
150
160
I
170
I
1BO
I
190
200|
I
I
I
I
I
I
I
I
I
I
120
140
160
180
200
220
240
260
280
300
/t
BIDS WITH NO INFORMATION
FIG. 2. E x p e c t e d value as a function of bid for the case of no information.
SEALED BIDDING EXPERIMENT
285
rifice a portion of their expected profit to increase their probability of winning the contract. The flatness of the expected value functions for the case of no information guarantees that the magnitude of sacrifice would be small. Figure 2 displays two expected value functions for the case of no information, one for each distribution condition; their maxima are indicated in the figure by small arrows. Figure 2 shows that decreasing the bid from 225 to 200 in DC1 would cause a relative expected loss in profit of only 4% but increase the probability of winning the contract by 12.5% from 3/8 to 4/8. In DC2, decreasing the bid from 200 to 175, for example, would cause a relative expected loss of 6.25% in profit; the probability of winning the contract in this case would increase from 0 to .25. Two extensions of the SBT paradigm may enhance its appeal and applicability. The first relaxes the requirement of perfect information with regard to cost or LCB and allows for purchasing information that only reduces by a specified amount, but does not eliminate completely, the uncertainty about one or both of the task variables. The second extension permits the subject to bid against several nonfictitious opponents and provides him with the option of purchasing information that either reduces or eliminates uncertainty about the bids of a subset of his opponents rather than about the lowest competitive bid. REFERENCES Becker, G. M., DeGroot, M. H., & Marschak, J. Measuring utility by a single-response sequential method. Behavioral Science. 1964, 9, 226-232. Becker, G. M., & McClintock, C. G. Value: Behavioral decision theory. Annual Review o f Psychology, 1967, 18, 239-286. Fried, L. S., & Peterson, C. R. Information seeking: Optional versus fixed stopping. Journal o f Experimental Psychology, 1969, 80, 525-529. Howard, R . A . Information value theory. 1EEE Transactions on Systems Science and Cybernetics, 1966, SSC-2, 22-26. Howard, R. A. Value of information lotteries. IEEE Transactions on Systems Science and Cybernetics, 1967, SSC-3, 54-60. Howard, R. A. The foundation of decision analysis. 1EEE Transactions on Systems Science and Cybernetics, 1968, SSC-4, 1-9. Larsson, B. Bayes strategies and human information seeking. Lurid CWK Gleerup: Univ. of Lurid Press, 1968. Marschak, J. Remarks on the economics of information. In Contribution to scientific research in management. Los Angeles: Western Data Processing Center, University of California, 1959. Medlin, S. M. Information value in a sealed bidding experiment. Unpublished doctoral dissertation, University of North Carolina at Chapel Hill, 1976. O'Connor, M. F., Peterson, C. R., & Palmer, T. J. Stakes and probabilities in information purchase. Organization Behavior and Human Performance, 1972, 7, 43-52. Pitz, G. E. Information seeking when available information is limited. Journal o f Experimental Psychology, 1968, 76, 25-34. Raiffa, H., & Schlaifer, R. Applied statistical decision theory. Boston: Harvard Business School, Division of Research, 1961.
286
STEVEN M. MEDLIN
Rapoport, A. Research paradigms for studying dynamic decision behavior. In D. Wendt & C. Vlek (Eds.), Utility, probability, and human decision making. Dordrecht: Reidel, 1975. Shanteau, J., & Anderson, N. H. Integration theory applied to judgements of the value of information. Journal of Experimental Psychology, 1972, 92, 266-275. Shuford, E. H. Some Bayesian learning processes. In M. W. Shelly II & G. L. Bryan (Eds.), Human judgements and optimality. New York: Wiley, 1964. Tversky, A., & Edwards, W. Information versus reward in binary choices. Journal of Experimental Psychology, 1966, 71,680-683. Tversky, A., & Kahneman, D. Judgement under uncertainty: Heuristics and biases. In D. Wendt & C. Vlek (Eds.), Utility, probability, and human decision making. Dordrecht: Reidel, 1975. Wald, A. Sequential analysis. New York: Wiley, 1947. Wendt, D. Value of information for decisions. Journal of Mathematical Psychology, 1969, 6, 430-443. RECEIVED: October 19, 1976