- Email: [email protected]
Information search and choice accuracy as a function of task complexity and task structure
Acta Psychologica North-Holland
1x5
80 (1992) 185-197
Information search and choice accuracy as a function of task complexity and task structure Jo& H. Kerstholt TN0 Instltuie
*
for Perception,
Soesterherg, The Netherlunds
Previous research has consistently shown that subjects switch to noncompensatory information search behaviour when task complexity increases. However. a rather specific class of tasks was used in these studies for which complete information search is not necessary to attain accurate task performance. In the present study information search behaviour, choice accuracy, subjective difficulty and confidence were registered under various task complexity conditions in two different task environments. In the first task, subjects had to make a choice between apartments: in the second task, a choice was required between baskets containing groceries of different prices. Forty subjects made choices by means of computerized information boards. The results showed effects of both complexity and type of task on information search behaviour. The lowest proportion of requested information and the most variable search pattern was observed for the apartments task in the complex task condition. However. accuracy remained constant over all complexity conditions. Together. the results suggest that under high task complexity levels subjects try to work smarter rather than harder. In the context of a choice requirement the specific task structure may allow them to switch to noncompensatory strategies while maintaining accuracy and avoiding the integration of large amounts of information.
Process tracing research aims to identify the cognitive processes that underlie decision-making behaviour and to specify how various task and context variables affect the way people deal with decision problems (Einhorn et al. 1979; Payne 1982). A recent review by Ford et al. (1989) revealed that the major part of this research used information boards as a technique to deduce the decision strategy that subjects employ to arrive at a decision. An information board depicts * I thank Henk van Doorne and Joep Mathijssen for technical assistance. Jos van Berkum. Jeroen Raaijmakers and an anonymous reviewer made helpful comments on a draft of this article. Correspondence to: J.H. Kerstholt, TN0 Institute for Perception, P.O. Box 23, 3769 ZG Soesterberg, The Netherlands.
OOOl-6918/92/$05.00
0 1992 - Elsevier
Science
Publishers
B.V. All rights
reserved
a matrix that specifics the potential alternatives (e.g. houses or cars) and the attributes, which are the characteristic aspects of the alternatives. Subjects arc required to successively indicate combinations of one attribute and one alternative, which results in the associated values appearing on a card or on a screen. This process continues until they feel that enough information has been obtained to make a choice. Subjects can rcqucst the available information in various ways. They can, for example, request all information or just a part of it, they can request the same or various amounts of information across alternatives and they can search for information alternative-wise or attribute-wise. A major assumption underlying the information board paradigm is that the specific pattern of information search bchaviour reflects the decision strategy that a subject has employed to solve a decision problem (Billings and Marcus 1983; Svenson lY79; Payne 1976, 1982). One task variable consistently affecting information starch is complexity, usually defined by the number of alternatives and/or attributes (Billings and Marcus 19X3; Olshavsky 1979; Payne 1976, 1982; Sundstriim 1987; Westenberg and Koele 1990). Generally, under high complexity levels depth of search decreases, i.e. subjects request less information, and search variability increases. Within the information board paradigm this finding is an indication for the employment of noncompensatory strategies. However, in spite of their incomplete information search noncompcnsatory rules may do surprisingly well in terms of the decision outcome itself (Paquette and Kida 1988; Payne ct al. 1988). More specifically, the necessity to request all information may be qualified by factors such as the goal that is set to the subject or the specific task structure. The effect of task structure on the efficiency of information search behaviour has recently been demonstrated by Payne et al. (1988). By means of simulations they determined both the accuracy and the prerequisite effort of various decision rules while varying context and task variables such as time pressure and the dispersion of probabihties. In a gamble with a low degree of dispersion each outcome has about the same probability to occur (e.g. 0.30, 0.20, 0.22, and 0.28). In gambles with a high degree of dispersion, on the other hand, some outcomes are far more likely to occur than others (e.g. 0.68, 0.12, 0.05
J.H. Kerstholt / Informution search and choice accurucy
187
and 0.15). The simulations showed that in some choice environment noncompensatory heuristic rules performed rather well. In the empirical part of their study subjects were required to make choices in various dispersion conditions and their performance was compared to the simulation outcomes. The results showed that to a large extent the subjects selected the rules that were indicated by the simulations as most efficient. Therefore, subjects took advantage of the characteristics of the choice environment in their decision-making process such that accuracy was maintained at the lowest level of invested effort. In studies investigating the effects of task complexity a rather specific class of tasks is used, for example, tasks requiring choices between apartments or consumer goods (Ford et al. 1989). However, these choice tasks may typically have a great variation in attribute weights or correlated attribute values, making complete information search no longer a prerequisite for accurate task performance. Thus, when task complexity increases subjects may well seek to employ strategies that take advantage of the task structure (Bettman and Park 1980; Huber 1983; Johnson 1987; Klein 19831, rather than try to integrate large amounts of information in a strict bottom-up way. Such a strategy can be considered as an adaptive mechanism to complexity in reducing the strain on working memory capacity while maintaining accurate task performance. In order to investigate the effects of task structure we used in addition to such a ‘conventional’ task (requiring choices between apartments), a unidimensional task (all values could be expressed in the same unit, i.e. money). This task required a choice between shopping baskets containing groceries of various prices. Since the attributes in the baskets task were equally weighted and uncorrelated this task requires the employment of a compensatory rule for accurate task performance. Therefore, from a normative point of view the tasks seem to be quite comparable, in the sense that they both require the calculation of the overall values of each alternative. The major difference results from the assumption that incomplete information search does not preclude accurate task performance in the apartments task. To summarize, in the present experiment two factors were manipulated, task complexity and type of task and both factors are predicted to have an effect on information search behaviour. For the apartments task we predicted, in line with previous research, more noncompensatory information search behaviour under complex task conditions.
IX8
J. H. Kwsfholt
/ Informatiorz search and choice accuruq
For the baskets task, on the other hand, we predicted no effect of complexity on search behaviour since in this task the use of a noncompensatory strategy would lower choice accuracy. Our assumption is therefore that subjects try to attain the goal set to them resulting in constant accuracy levels over complexity conditions in both tasks. We also registered how difficult the subjects found the choice problems and how confident they were about the actual choice. Confidence is often related to the amount and strength of information supporting the decision (Koriat et al. 1980). On the assumption that subjects try to attain accurate task performance we predicted that their confidence in the decision would remain constant over complexity conditions despite more noncompensatory search behaviour (Peterson and Pitz 1988).
Method Subjects
Forty students of the University of Utrccht participated in the experiment. Their mean age was 21.7 years (s = 1.5 years) and they were paid Dfl. 20 for participation. Material
Two different tasks were used which were both presented on a computerized information board. The first task required a choice between apartments. The apartments could be specified on maximally 8 attributes: rent (300, 600 or 900 Dutch guildcrs). size (2, 3 or 4 rooms), location (bad, moderate or good); distance to work (long, moderate, or short); distance to shops (long, moderate or short); noise level (high, moderate or low); garden or balcony (big, small or absent) and state of repair (good, reasonable or bad). The stimuli sets were constructed by means of a balanced design, implying that all value levels of each attribute appeared approximately equally often in all complexity conditions. Care was taken to avoid dominant alternatives, that is, alternatives that clearly stand out by having desirable values on all their attributes. The second task required a choice between shopping baskets, each of which contained up to 8 groceries. The groceries were indicated by single letters in order to eliminate possible cffccts of prior knowledge on product costs. The subjects could rcqucst the prices of the groceries, which all varied between 0 and 10. The prices were randomly assigned to the groceries. The alternatives (apartments or baskets) were specified in the columns and the attributes (apartment characteristics or groceries) in the rows of the information board matrix. Both the subjective difficulty scores and the certainty scores were expressed on 9-point scales.
J.H. Kerstholt / Information search and choice uccuracy
1x9
Procedure
For both choice tasks subjects sat in front of a computerized information board. They could successively indicate a combination of an alternative and an attribute after which the corresponding value appeared in the appropriate matrix cell. All requested information remained visible until a choice was made. Subjects were instructed to request as much information as they needed to make an accurate choice. Subjects in the apartments-task condition were instructed to select the best apartment and subjects in the baskets task were required to select the basket lowest in price. After each trial they indicated how confident they felt that a correct choice was made and how difficult they found the choice problem on g-point scales. Before they started with the choice task, subjects in the apartments-condition carried out two additional tasks, which were needed for the calculation of the multi-attribute utility of each apartment in the choice sets. First, they indicated the attractiveness of each attribute level on a 7-point scale. Second, in order to get the weight of each attribute, subjects were asked to judge 81 apartments on attractiveness by giving them a score between 0 and 100. Each apartment was described on 8 attributes. Presenting all possible combinations of attribute levels would result in 3’ judgements. In order to reduce this number, a subset of these possibilities was selected by means of a 34 fractional factorial design (Kirk 1968). Before the judgement scores were given subjects were first asked to describe an apartment with the worst value levels (score 0) and an apartment with the best value levels (score 100). These attribute values were printed on cards and given to the subject in order to provide subjects with anchors with which the actual descriptions of the apartments could be compared throughout the judgement task. Design
In order to manipulate the complexity of the choice task the number of attributes (4 or 8) and the number of alternatives (3 or 6) were varied. They were both manipulated as a within-subjects factor. Each subject made one choice in each complexity condition. To avoid sequence effects, the order of task complexity conditions was balanced across subjects. Type of task was a between-subjects factor. Subjects were randomly assigned to a task condition.
Results As mentioned in the introduction, we predicted that information search behaviour would be affected by task complexity in the apartments task but not in the baskets task. Since we were interested in the extent to which compensatory strategies were used we operationalised information search behaviour by depth of search (proportion requested information) and variability of search (standard deviation of the proportions of requested information units across alternatives). Unless indicated otherwise we used for each dependent variable a three-way analysis of variance with type of task
I 90
J.H. Kerstholt
/ Information
I
1.0
I
“apartments’‘-task
search and choice accuracy
I
I
“baskets’‘-task
0
\ .
\
0
.
0 4 attributes 8 attributes l
0.2
I
I
I
I
3
6
3
6
number Fig. 1. Proportion
of requested
information
of alternatives for each complexity
and task condition.
as a between-subjects variable and number of alternatives and number of attributes as within-subjects variables. The proportion of requested information significantly differed between task conditions (see fig. l), such that more information was requested in the baskets task (F(1,38) = 73.42; p < 0.0001). As predicted, the proportion of requested information was differentially affcctcd by task complexity in both task conditions (number of alternatives by type of task: (F(1,38) = 7.99; p < 0.01, number of attributes by type of task: F(1,38) = 19.00; p < 0.0001). The three-way interaction was not significant (F(1,38) < 1). The interactions between type of task and complexity conditions were further analysed by two separate analyses of variance for each task condition. In the apartments task complexity significantly affected the proportion of requested information (number of alternatives: F(1,19) = 25.27; p < 0.0001, number of attributes: F(1,19) = 38.38; p < 0.0001). In the baskets task on the other hand, neither number of alternatives nor number of attributes affected the proportion requested information (number of altcrnativcs: F(1,19) = 2.42: p > 0.1, number of attributes F(1,19) < 1). Therefore, task complexity only reduces depth of search in the apartments task and not in the baskets task. In addition to the request for more information, subjects also showed a less variable search pattern in the baskets task, indexed by the mean standard deviation of the proportions of requested information units across alternatives (F(1,38) = 24.14; p < 0.0001). The mean scores for each complexity and task condition arc shown in fig. 2. As the number of alternatives increased the search pattern became more variable (F(1,38) = 11.19; p < 0.005). Note that within the information board paradigm variability of search is directly translated to the kind of strategy that is used, i.e. increases
J.H. Kerstholt / Information search and choice accuracy
I
I
191
1
“baskets’‘-task
“apartments”-task 0.4
z .2 0.3 2 G r m g
0.2
; m n cn
. /:YO 0 4 attributes 8 attributes
0.1
l
0.0
I
I
I
I
3
6
3
6
number
Fig. 2. Mean
standard
deviation
of alternatives
of requested information units complexity and task condition.
across
alternatives
for each
in the standard deviation across alternatives indicate the employment of noncompensatory strategies. On the other hand, as the number of attributes increased search variability remained constant (F(1,38) = 1.85; p > 0.1). Furthermore, complexity did not differentially affect search variability in both task conditions (number of alternatives by type of task: F(1,38) = 1.14; p > 0.2, number of attributes by type of task: F(1,38) < 1). As a criterion for the accuracy of a choice recent research has primarily used the Compensatory weighted additive rule (Creyer et al. 1990; Johnson and Payne 1985; Payne et al. 1988; Zakay and Wooler 1984). This rule implies that all information is processed and that trade-offs between different value levels are taken into account. For the baskets task the overall value of each alternative was calculated by adding the prices of all groceries and the least expensive basket was defined as the best alternative. To get the input for the application of a weighted additive rule in the apartments task subjects had provided attractiveness scores, utilities, to each attribute level and judgment scores to 81 apartments. The weights of the attributes were deduced from linear regression analyses with the judgement scores as the dependent variable and the utilities of the attribute levels as the predictors. The mean adjusted squared multiple r over subjects was 0.64 (S = 0.12). The multi-attribute utilities for each apartment and for each subject were calculated by adding the weighted utility scores of each attribute. Following Creyer et al. (1990) we expressed the accuracy scores by relating the overall utility (or price in the baskets task) of the chosen alternative to the overall utilities of the best and the worst option: Relative
accuracy =
U(choice)
- U(min)]/[U(max)
- U(min)].
1
1.0
I
“baskets’‘-task
0 . .
0
I
I
“apartments’‘-task
+
0 0
0
.
.
-.0
0.4
.
0
0 4 attrlbutes 8 attributes actual random l
-
0.2
I
I
3
6 number
Fig. 3. Predicted accuracy under random
choice
I
I
3
6
of alternatives and actual
accuracy
scores for each complexity
and task condition.
A score of 1 implies that the alternative with the highest overall utility (price) was chosen and a score of 0 that the alternative with the lowest multi-attribute utility (price) was selected. However, before presenting the results we will first indicate the prcdictcd accuracy scores under random performance, providing a base-line for the interpretation of the results. For this purpose, one alternative was randomly selected from each choice set that was presented to the subjects in the actual experiment. The overall utilities of these alternatives were entered into the formula stated above, and the relative agreement scores were calculated. Under random performance the relative performance scores fluctuate around 0.50 (see fig. 3) and performance is neither affected by type of task nor task complexity (type of task, number of attributes, number of alternatives, attributes by type of task and attributes by alternatives: F(1,38) < 1; alternatives by type of task: F(1,38) = 1.41; p > 0.2: attributes by alternatives by type of task: F(1.38) = 1.58; p > 0.2). When the actual performance scores are considered (fig. 3) it shows that subjects in the baskets task were more accurate than subjects in the apartments task (F(1,38) = 5.92; p < 0.05). Furthermore, accuracy increased when the number of alternatives increased (F(1,38) = 9.70; p < 0.01). There were no significant interactions. We also computed the relative accuracy scores in terms of the improvement of the chosen alternative over the expected value of a random choice, a measurement that was used in the studies of Johnson and Payne (1985) and Payne ct al. (1988). A comparison of the results showed that the relative improvement of the choice over random performance closely matched the improvement of the choice over the worst possible choice.
J.H. Krrstholt / Information search and choice accuracy
I
8 -
I
“apartments’‘-task
I
--
193
1
“baskets’‘-task
7!? o 6S > p % Y- 5L
:=c:
--
.,.
O-.-----O 3-
0 4 attributes 8 attributes l
I
I
I
I
3
6
3
6
2
number
Fig. 4. Mean difficulty
of alternatives
score for each complexity and task condition very easy to very difficult).
I
I
“apartments’‘-task
1
(9.point
scale varying
from
scale varying
from
I
“baskets’‘-task
0 .
0 .
/ /
0 4 attributes 8 attributes l
I
I
3
6 number
Fig. 5. Mean confidence
3
6
of alternatives
score for each task and complexity condition very unconfident to very confident.
(9-point
WC can conclude, thcreforc. that choice accuracy did not decrease over complexity conditions. The baskets task was found easier than the apartments task (F(1,38) = 4.47: p < 0.05. set fig. 4). Furthcrmorc, the choice problem was found more difficult when the number of attributes increased (F( 1.38) = 3.65: p = 0.06). Number of alternatives had no effect on the difficulty ratings (F(1,38) < I). Subjects were more confident on their choice in the baskets task than in the apartments task (F(1.38) = 7.10; p < 0.01). As the number of alternatives increased subjects were more confident (F(1,38) = 4.59: p < 0.05). As a more direct test of the relation bctwecn depth of search, variability of starch, accuracy, difficulty and confidence we also computed the correlations between these variables over all complexity conditions for each type of task. Depth of starch was significantly correlated with the variability of search in the apartments task (r = -0.58; Bonferroni adjusted probability < 0.01101): as more information was requested the variability of search decreased. or, in other words, the search pattern became more compensatory. Furthermore, the certainty scores were significantly correlated with the subjcctivc difficulty scores in both tasks (apartments task: r = -0.43: p < 0.005. baskets task: r = -0.37; p < 0.01). Thus, choices that were found less difficult also got higher certainty ratings. Accuracy scores did not correlate with any of the other dependent variables.
Discussion
Of main interest to the present experiment was the effect of task complexity and type of task on (non)compensatory search behaviour and accuracy. The results clearly showed that accuracy remained constant even though search behaviour was differentially affected by task complexity in both task conditions. Task complexity mainly affected information search behaviour in the apartments task; depth of search decreased and the variability of search increased over task complexity conditions, indicating a switch to more noncompensatory search behaviour. Since accuracy remained constant over complexity conditions this task obviously allowed for incomplete information search. In the apartments task the attributes were differentially weighted which could be used by the subjects to eliminate alternatives early in the decision process. A switch to such a strategy under high information load seems rather adaptive since the employment of a noncompensatory strategy in combination with the specific task structure reduces the integration of large amounts of information but maintains accurate task performance. The use of a noncompensatory strategy also did not affect the subjects’ confidence
J.H. Kerstholt / Information search and choice accuracy
195
ratings. Thus, in addition to the amount of information (Peterson and Pitz 1988) and the possibility to make inferences (Levin et al. 1988>, confidence seems to be related to strategic knowledge, i.e. knowledge on the accuracy of a search strategy in a specific task environment. For the baskets task we predicted no effect of task complexity on information search behaviour. However, even though depth of search remained constant over complexity conditions variability of search increased when more alternatives had to be considered. Of major importance in this respect is whether an increased number of alternatives would induce higher search variability by chance alone, i.e. without a change in search strategy. The task required the subjects to select one best option. The most reasonable explanation for search variability in the baskets task is that an alternative was dropped from the choice set whenever the added costs of an incompletely described alternative exceeded the overall value of a completely described alternative. In order to indicate a chance effect, we simulated the employment of such a strategy in various complexity conditions. A computer program read the attribute values of the successive alternatives, calculated the value of the alternative after each attribute value read, and dropped the alternative whenever this value equalled or exceeded the lowest (overall) value of preceding alternatives in the choice set. The results from these simulations (N = 500) indeed showed that the variability of search increased when the number of alternatives increased (3 alternatives: s = 0.12; 6 alternatives: s = 0.18). In agreement with our findings, the standard deviation across alternatives marginally decreased with an increased number of attributes (4 attributes: s = 0.16; 8 attributes: s = 0.131. Thus, the observed increased search variability in the baskets task might well be explained by chance alone. The results showed that even though more information is requested in the baskets task subjects rated the decision problems as less difficult than in the apartments task. A major difference between the tasks is the dimensionality of the decision problem. In order to determine the overall values, subjects could simply add all prices in the baskets task whereas they had to integrate different units in the apartments task. Even though the subjects accurately dealt with this problem, they may for this reason have found the apartments task more difficult. Confidence levels were also higher in the baskets task than in the apartments task, which may be explained by the familiarity
of the task (Huber 1989). In the baskets task the decision rule is clearly linked to the outcome of the decision. Prices have to be added and the lowest overall price is unquestionably the best alternative. Thus, this result also suggests that knowledge on the accuracy of a specific strategy in some choice environment affects one’s confidence in the actually chosen option. To conclude, in line with the predictions subjects only switched to more noncompensatory search strategies in the apartments task, as indicated by depth of search and search variability. However, in spite of incomplete information search, accuracy remained constant over all complexity conditions. In process-tracing studies a rather specific class of tasks is used, such as choices between apartments or consumer products. By its structure, for example high dispersion of attribute weights, these tasks may have allowed for the application of noncompensatory strategies without reducing the accuracy scores. The present results suggest that more research is needed with different types of task, in order to differentiate between general task complexity effects and its interaction with specific task environments.
References Bettman.
J.R. and C.W.
Park. 1Y80. Effects of prior knowledge
choice process on consumer Research
of decision Creyer,
hchavior:
3 I.
Performance
E.H..
J.R. Bettman
H.J., D.N.
and J.W. Payne.
decision hehaviour.
problems
Decision
0.
Amsterdam:
North-Holland.
Svenson
j-21. Johnson.
models
Behavior
IYYO. The impact of accuracy and effort Journal
B.M.
of Behavioral
and
Decision
feedhnck
Making
3. I-16.
lY7Y. Lineal- regression and process-tracing
Hults
and neglected
V6ri
and M.L. research
Doherty,
questions.
lYX9. Process tracing Organizational
Behav-
Payne.
and actually processed in a decision task’. In: P.C. (eds.),
Analyzing
and
aiding
decision
pr-occsaes.
pp. 441-4.54. operators
Process and structure
E.J. and J.W.
.iYS~‘IlJ.
Organizational
Review 86. 465-385.
presented
an d A.
O., 1989. ‘Information-processing Svenson (ed\.).
and noncompensatory
Processes 43, 7.5-l 17.
O., 1983. ‘The information
Humphreys.
0.
of compensatory
and B. Kleinmuntr.
S.L. Schechtman,
methods: Contributions.
Huber,
of Consumer
33l-352.
Psychological
Ford, J.K.. N. Schmitt. ior and Human
lYN3. Measures
Kleinmuntz
models of judgment.
Huber,
and phase of the
analysis. Journal
Process tracing versus policy capturing.
and goals on adaptive Einhorn.
and experience
processes: A protocol
7. 234-2-1X.
Billings. R.S. and S.A. Marcus, Human
decision
10x5. Effort
in decision making’.
in human
In: H. Montgomery
decision making.
and accuracy
in choice.
Chichexter:
Management
Wiley.
and pp.
Science 31.
J.H. Kerstholt
/ Information
seurch and ckmce uccurucy
197
Johnson, R.D., 1987. Making judgments when information is missing: Inferences, biases, and framing effects. Acta Psychologica 66, 69-82. Kirk. R.E.. 1968. Experimental design: Procedures for the behavioral sciences. Belmont: Brooks/Cole. Klein, N.M.. 1983. Utility and decision strategies: A second look at the rational decision maker. Organizational Behavior and Human Performance 31. l-25. Koriat, A., S. Lichtenstein and B. Fischhoff, 1980. Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory 6, 1077118. Levin, I.P.. D.P. Chapman and R.D. Johnson, 1988. Confidence in judgments based on incomplete information. Journal of Behavioral Decision Making 1, 29-41. Olshavsky, R.W., 1979. Task complexity and contingent processing in decision making: A replication and extension. Organizational Behavior and Human Performance 24, 300&316. Paquette, L. and T. Kida, 1988. The effects of decision strategy and task complexity on decision performance. Organizational Behavior and Human Decision Processes 41, 128-142. Payne, J.W.. 1976. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance 16, 366-387. Payne, J.W.. 1982. Contingent decision behavior. Psychological Bulletin 92, 3X2-402. Payne. J.W., J.R. Bettman and E.J. Johnson, 19x8. Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning. Memory and Cognition 14, 5344552. Peterson, D.K. and G.F. Pitz, 1988. Confidence, uncertainty and the use of information. Journal of Experimental Psychology: Learning. Memory and Cognition 14, 85-92. Sundstrom. G.A.. 1987. Information search and decision making: The effects of information displays. Acta Psychologica 65, 1655179. Swenson. O., 1970. Process descriptions of decision making. Organizational Behavior and Human Performance 23, X6- 112. Westenherg, M.R.M. and P. Koele, 1990. ‘Response modes and decision strategies’. In: K. Borcherding. 0.1. Larichev and D.M. Messick teds.), Contemporary issues in decision making. Amsterdam: North-Holland. pp. 159%170. Zakay. B.D. and S. Wooler, 1984. Time pressure, training and decision effectiveness. Ergonomics 27, 273-284.
1x5
80 (1992) 185-197
Information search and choice accuracy as a function of task complexity and task structure Jo& H. Kerstholt TN0 Instltuie
*
for Perception,
Soesterherg, The Netherlunds
Previous research has consistently shown that subjects switch to noncompensatory information search behaviour when task complexity increases. However. a rather specific class of tasks was used in these studies for which complete information search is not necessary to attain accurate task performance. In the present study information search behaviour, choice accuracy, subjective difficulty and confidence were registered under various task complexity conditions in two different task environments. In the first task, subjects had to make a choice between apartments: in the second task, a choice was required between baskets containing groceries of different prices. Forty subjects made choices by means of computerized information boards. The results showed effects of both complexity and type of task on information search behaviour. The lowest proportion of requested information and the most variable search pattern was observed for the apartments task in the complex task condition. However. accuracy remained constant over all complexity conditions. Together. the results suggest that under high task complexity levels subjects try to work smarter rather than harder. In the context of a choice requirement the specific task structure may allow them to switch to noncompensatory strategies while maintaining accuracy and avoiding the integration of large amounts of information.
Process tracing research aims to identify the cognitive processes that underlie decision-making behaviour and to specify how various task and context variables affect the way people deal with decision problems (Einhorn et al. 1979; Payne 1982). A recent review by Ford et al. (1989) revealed that the major part of this research used information boards as a technique to deduce the decision strategy that subjects employ to arrive at a decision. An information board depicts * I thank Henk van Doorne and Joep Mathijssen for technical assistance. Jos van Berkum. Jeroen Raaijmakers and an anonymous reviewer made helpful comments on a draft of this article. Correspondence to: J.H. Kerstholt, TN0 Institute for Perception, P.O. Box 23, 3769 ZG Soesterberg, The Netherlands.
OOOl-6918/92/$05.00
0 1992 - Elsevier
Science
Publishers
B.V. All rights
reserved
a matrix that specifics the potential alternatives (e.g. houses or cars) and the attributes, which are the characteristic aspects of the alternatives. Subjects arc required to successively indicate combinations of one attribute and one alternative, which results in the associated values appearing on a card or on a screen. This process continues until they feel that enough information has been obtained to make a choice. Subjects can rcqucst the available information in various ways. They can, for example, request all information or just a part of it, they can request the same or various amounts of information across alternatives and they can search for information alternative-wise or attribute-wise. A major assumption underlying the information board paradigm is that the specific pattern of information search bchaviour reflects the decision strategy that a subject has employed to solve a decision problem (Billings and Marcus 1983; Svenson lY79; Payne 1976, 1982). One task variable consistently affecting information starch is complexity, usually defined by the number of alternatives and/or attributes (Billings and Marcus 19X3; Olshavsky 1979; Payne 1976, 1982; Sundstriim 1987; Westenberg and Koele 1990). Generally, under high complexity levels depth of search decreases, i.e. subjects request less information, and search variability increases. Within the information board paradigm this finding is an indication for the employment of noncompensatory strategies. However, in spite of their incomplete information search noncompcnsatory rules may do surprisingly well in terms of the decision outcome itself (Paquette and Kida 1988; Payne ct al. 1988). More specifically, the necessity to request all information may be qualified by factors such as the goal that is set to the subject or the specific task structure. The effect of task structure on the efficiency of information search behaviour has recently been demonstrated by Payne et al. (1988). By means of simulations they determined both the accuracy and the prerequisite effort of various decision rules while varying context and task variables such as time pressure and the dispersion of probabihties. In a gamble with a low degree of dispersion each outcome has about the same probability to occur (e.g. 0.30, 0.20, 0.22, and 0.28). In gambles with a high degree of dispersion, on the other hand, some outcomes are far more likely to occur than others (e.g. 0.68, 0.12, 0.05
J.H. Kerstholt / Informution search and choice accurucy
187
and 0.15). The simulations showed that in some choice environment noncompensatory heuristic rules performed rather well. In the empirical part of their study subjects were required to make choices in various dispersion conditions and their performance was compared to the simulation outcomes. The results showed that to a large extent the subjects selected the rules that were indicated by the simulations as most efficient. Therefore, subjects took advantage of the characteristics of the choice environment in their decision-making process such that accuracy was maintained at the lowest level of invested effort. In studies investigating the effects of task complexity a rather specific class of tasks is used, for example, tasks requiring choices between apartments or consumer goods (Ford et al. 1989). However, these choice tasks may typically have a great variation in attribute weights or correlated attribute values, making complete information search no longer a prerequisite for accurate task performance. Thus, when task complexity increases subjects may well seek to employ strategies that take advantage of the task structure (Bettman and Park 1980; Huber 1983; Johnson 1987; Klein 19831, rather than try to integrate large amounts of information in a strict bottom-up way. Such a strategy can be considered as an adaptive mechanism to complexity in reducing the strain on working memory capacity while maintaining accurate task performance. In order to investigate the effects of task structure we used in addition to such a ‘conventional’ task (requiring choices between apartments), a unidimensional task (all values could be expressed in the same unit, i.e. money). This task required a choice between shopping baskets containing groceries of various prices. Since the attributes in the baskets task were equally weighted and uncorrelated this task requires the employment of a compensatory rule for accurate task performance. Therefore, from a normative point of view the tasks seem to be quite comparable, in the sense that they both require the calculation of the overall values of each alternative. The major difference results from the assumption that incomplete information search does not preclude accurate task performance in the apartments task. To summarize, in the present experiment two factors were manipulated, task complexity and type of task and both factors are predicted to have an effect on information search behaviour. For the apartments task we predicted, in line with previous research, more noncompensatory information search behaviour under complex task conditions.
IX8
J. H. Kwsfholt
/ Informatiorz search and choice accuruq
For the baskets task, on the other hand, we predicted no effect of complexity on search behaviour since in this task the use of a noncompensatory strategy would lower choice accuracy. Our assumption is therefore that subjects try to attain the goal set to them resulting in constant accuracy levels over complexity conditions in both tasks. We also registered how difficult the subjects found the choice problems and how confident they were about the actual choice. Confidence is often related to the amount and strength of information supporting the decision (Koriat et al. 1980). On the assumption that subjects try to attain accurate task performance we predicted that their confidence in the decision would remain constant over complexity conditions despite more noncompensatory search behaviour (Peterson and Pitz 1988).
Method Subjects
Forty students of the University of Utrccht participated in the experiment. Their mean age was 21.7 years (s = 1.5 years) and they were paid Dfl. 20 for participation. Material
Two different tasks were used which were both presented on a computerized information board. The first task required a choice between apartments. The apartments could be specified on maximally 8 attributes: rent (300, 600 or 900 Dutch guildcrs). size (2, 3 or 4 rooms), location (bad, moderate or good); distance to work (long, moderate, or short); distance to shops (long, moderate or short); noise level (high, moderate or low); garden or balcony (big, small or absent) and state of repair (good, reasonable or bad). The stimuli sets were constructed by means of a balanced design, implying that all value levels of each attribute appeared approximately equally often in all complexity conditions. Care was taken to avoid dominant alternatives, that is, alternatives that clearly stand out by having desirable values on all their attributes. The second task required a choice between shopping baskets, each of which contained up to 8 groceries. The groceries were indicated by single letters in order to eliminate possible cffccts of prior knowledge on product costs. The subjects could rcqucst the prices of the groceries, which all varied between 0 and 10. The prices were randomly assigned to the groceries. The alternatives (apartments or baskets) were specified in the columns and the attributes (apartment characteristics or groceries) in the rows of the information board matrix. Both the subjective difficulty scores and the certainty scores were expressed on 9-point scales.
J.H. Kerstholt / Information search and choice uccuracy
1x9
Procedure
For both choice tasks subjects sat in front of a computerized information board. They could successively indicate a combination of an alternative and an attribute after which the corresponding value appeared in the appropriate matrix cell. All requested information remained visible until a choice was made. Subjects were instructed to request as much information as they needed to make an accurate choice. Subjects in the apartments-task condition were instructed to select the best apartment and subjects in the baskets task were required to select the basket lowest in price. After each trial they indicated how confident they felt that a correct choice was made and how difficult they found the choice problem on g-point scales. Before they started with the choice task, subjects in the apartments-condition carried out two additional tasks, which were needed for the calculation of the multi-attribute utility of each apartment in the choice sets. First, they indicated the attractiveness of each attribute level on a 7-point scale. Second, in order to get the weight of each attribute, subjects were asked to judge 81 apartments on attractiveness by giving them a score between 0 and 100. Each apartment was described on 8 attributes. Presenting all possible combinations of attribute levels would result in 3’ judgements. In order to reduce this number, a subset of these possibilities was selected by means of a 34 fractional factorial design (Kirk 1968). Before the judgement scores were given subjects were first asked to describe an apartment with the worst value levels (score 0) and an apartment with the best value levels (score 100). These attribute values were printed on cards and given to the subject in order to provide subjects with anchors with which the actual descriptions of the apartments could be compared throughout the judgement task. Design
In order to manipulate the complexity of the choice task the number of attributes (4 or 8) and the number of alternatives (3 or 6) were varied. They were both manipulated as a within-subjects factor. Each subject made one choice in each complexity condition. To avoid sequence effects, the order of task complexity conditions was balanced across subjects. Type of task was a between-subjects factor. Subjects were randomly assigned to a task condition.
Results As mentioned in the introduction, we predicted that information search behaviour would be affected by task complexity in the apartments task but not in the baskets task. Since we were interested in the extent to which compensatory strategies were used we operationalised information search behaviour by depth of search (proportion requested information) and variability of search (standard deviation of the proportions of requested information units across alternatives). Unless indicated otherwise we used for each dependent variable a three-way analysis of variance with type of task
I 90
J.H. Kerstholt
/ Information
I
1.0
I
“apartments’‘-task
search and choice accuracy
I
I
“baskets’‘-task
0
\ .
\
0
.
0 4 attributes 8 attributes l
0.2
I
I
I
I
3
6
3
6
number Fig. 1. Proportion
of requested
information
of alternatives for each complexity
and task condition.
as a between-subjects variable and number of alternatives and number of attributes as within-subjects variables. The proportion of requested information significantly differed between task conditions (see fig. l), such that more information was requested in the baskets task (F(1,38) = 73.42; p < 0.0001). As predicted, the proportion of requested information was differentially affcctcd by task complexity in both task conditions (number of alternatives by type of task: (F(1,38) = 7.99; p < 0.01, number of attributes by type of task: F(1,38) = 19.00; p < 0.0001). The three-way interaction was not significant (F(1,38) < 1). The interactions between type of task and complexity conditions were further analysed by two separate analyses of variance for each task condition. In the apartments task complexity significantly affected the proportion of requested information (number of alternatives: F(1,19) = 25.27; p < 0.0001, number of attributes: F(1,19) = 38.38; p < 0.0001). In the baskets task on the other hand, neither number of alternatives nor number of attributes affected the proportion requested information (number of altcrnativcs: F(1,19) = 2.42: p > 0.1, number of attributes F(1,19) < 1). Therefore, task complexity only reduces depth of search in the apartments task and not in the baskets task. In addition to the request for more information, subjects also showed a less variable search pattern in the baskets task, indexed by the mean standard deviation of the proportions of requested information units across alternatives (F(1,38) = 24.14; p < 0.0001). The mean scores for each complexity and task condition arc shown in fig. 2. As the number of alternatives increased the search pattern became more variable (F(1,38) = 11.19; p < 0.005). Note that within the information board paradigm variability of search is directly translated to the kind of strategy that is used, i.e. increases
J.H. Kerstholt / Information search and choice accuracy
I
I
191
1
“baskets’‘-task
“apartments”-task 0.4
z .2 0.3 2 G r m g
0.2
; m n cn
. /:YO 0 4 attributes 8 attributes
0.1
l
0.0
I
I
I
I
3
6
3
6
number
Fig. 2. Mean
standard
deviation
of alternatives
of requested information units complexity and task condition.
across
alternatives
for each
in the standard deviation across alternatives indicate the employment of noncompensatory strategies. On the other hand, as the number of attributes increased search variability remained constant (F(1,38) = 1.85; p > 0.1). Furthermore, complexity did not differentially affect search variability in both task conditions (number of alternatives by type of task: F(1,38) = 1.14; p > 0.2, number of attributes by type of task: F(1,38) < 1). As a criterion for the accuracy of a choice recent research has primarily used the Compensatory weighted additive rule (Creyer et al. 1990; Johnson and Payne 1985; Payne et al. 1988; Zakay and Wooler 1984). This rule implies that all information is processed and that trade-offs between different value levels are taken into account. For the baskets task the overall value of each alternative was calculated by adding the prices of all groceries and the least expensive basket was defined as the best alternative. To get the input for the application of a weighted additive rule in the apartments task subjects had provided attractiveness scores, utilities, to each attribute level and judgment scores to 81 apartments. The weights of the attributes were deduced from linear regression analyses with the judgement scores as the dependent variable and the utilities of the attribute levels as the predictors. The mean adjusted squared multiple r over subjects was 0.64 (S = 0.12). The multi-attribute utilities for each apartment and for each subject were calculated by adding the weighted utility scores of each attribute. Following Creyer et al. (1990) we expressed the accuracy scores by relating the overall utility (or price in the baskets task) of the chosen alternative to the overall utilities of the best and the worst option: Relative
accuracy =
U(choice)
- U(min)]/[U(max)
- U(min)].
1
1.0
I
“baskets’‘-task
0 . .
0
I
I
“apartments’‘-task
+
0 0
0
.
.
-.0
0.4
.
0
0 4 attrlbutes 8 attributes actual random l
-
0.2
I
I
3
6 number
Fig. 3. Predicted accuracy under random
choice
I
I
3
6
of alternatives and actual
accuracy
scores for each complexity
and task condition.
A score of 1 implies that the alternative with the highest overall utility (price) was chosen and a score of 0 that the alternative with the lowest multi-attribute utility (price) was selected. However, before presenting the results we will first indicate the prcdictcd accuracy scores under random performance, providing a base-line for the interpretation of the results. For this purpose, one alternative was randomly selected from each choice set that was presented to the subjects in the actual experiment. The overall utilities of these alternatives were entered into the formula stated above, and the relative agreement scores were calculated. Under random performance the relative performance scores fluctuate around 0.50 (see fig. 3) and performance is neither affected by type of task nor task complexity (type of task, number of attributes, number of alternatives, attributes by type of task and attributes by alternatives: F(1,38) < 1; alternatives by type of task: F(1,38) = 1.41; p > 0.2: attributes by alternatives by type of task: F(1.38) = 1.58; p > 0.2). When the actual performance scores are considered (fig. 3) it shows that subjects in the baskets task were more accurate than subjects in the apartments task (F(1,38) = 5.92; p < 0.05). Furthermore, accuracy increased when the number of alternatives increased (F(1,38) = 9.70; p < 0.01). There were no significant interactions. We also computed the relative accuracy scores in terms of the improvement of the chosen alternative over the expected value of a random choice, a measurement that was used in the studies of Johnson and Payne (1985) and Payne ct al. (1988). A comparison of the results showed that the relative improvement of the choice over random performance closely matched the improvement of the choice over the worst possible choice.
J.H. Krrstholt / Information search and choice accuracy
I
8 -
I
“apartments’‘-task
I
--
193
1
“baskets’‘-task
7!? o 6S > p % Y- 5L
:=c:
--
.,.
O-.-----O 3-
0 4 attributes 8 attributes l
I
I
I
I
3
6
3
6
2
number
Fig. 4. Mean difficulty
of alternatives
score for each complexity and task condition very easy to very difficult).
I
I
“apartments’‘-task
1
(9.point
scale varying
from
scale varying
from
I
“baskets’‘-task
0 .
0 .
/ /
0 4 attributes 8 attributes l
I
I
3
6 number
Fig. 5. Mean confidence
3
6
of alternatives
score for each task and complexity condition very unconfident to very confident.
(9-point
WC can conclude, thcreforc. that choice accuracy did not decrease over complexity conditions. The baskets task was found easier than the apartments task (F(1,38) = 4.47: p < 0.05. set fig. 4). Furthcrmorc, the choice problem was found more difficult when the number of attributes increased (F( 1.38) = 3.65: p = 0.06). Number of alternatives had no effect on the difficulty ratings (F(1,38) < I). Subjects were more confident on their choice in the baskets task than in the apartments task (F(1.38) = 7.10; p < 0.01). As the number of alternatives increased subjects were more confident (F(1,38) = 4.59: p < 0.05). As a more direct test of the relation bctwecn depth of search, variability of starch, accuracy, difficulty and confidence we also computed the correlations between these variables over all complexity conditions for each type of task. Depth of starch was significantly correlated with the variability of search in the apartments task (r = -0.58; Bonferroni adjusted probability < 0.01101): as more information was requested the variability of search decreased. or, in other words, the search pattern became more compensatory. Furthermore, the certainty scores were significantly correlated with the subjcctivc difficulty scores in both tasks (apartments task: r = -0.43: p < 0.005. baskets task: r = -0.37; p < 0.01). Thus, choices that were found less difficult also got higher certainty ratings. Accuracy scores did not correlate with any of the other dependent variables.
Discussion
Of main interest to the present experiment was the effect of task complexity and type of task on (non)compensatory search behaviour and accuracy. The results clearly showed that accuracy remained constant even though search behaviour was differentially affected by task complexity in both task conditions. Task complexity mainly affected information search behaviour in the apartments task; depth of search decreased and the variability of search increased over task complexity conditions, indicating a switch to more noncompensatory search behaviour. Since accuracy remained constant over complexity conditions this task obviously allowed for incomplete information search. In the apartments task the attributes were differentially weighted which could be used by the subjects to eliminate alternatives early in the decision process. A switch to such a strategy under high information load seems rather adaptive since the employment of a noncompensatory strategy in combination with the specific task structure reduces the integration of large amounts of information but maintains accurate task performance. The use of a noncompensatory strategy also did not affect the subjects’ confidence
J.H. Kerstholt / Information search and choice accuracy
195
ratings. Thus, in addition to the amount of information (Peterson and Pitz 1988) and the possibility to make inferences (Levin et al. 1988>, confidence seems to be related to strategic knowledge, i.e. knowledge on the accuracy of a search strategy in a specific task environment. For the baskets task we predicted no effect of task complexity on information search behaviour. However, even though depth of search remained constant over complexity conditions variability of search increased when more alternatives had to be considered. Of major importance in this respect is whether an increased number of alternatives would induce higher search variability by chance alone, i.e. without a change in search strategy. The task required the subjects to select one best option. The most reasonable explanation for search variability in the baskets task is that an alternative was dropped from the choice set whenever the added costs of an incompletely described alternative exceeded the overall value of a completely described alternative. In order to indicate a chance effect, we simulated the employment of such a strategy in various complexity conditions. A computer program read the attribute values of the successive alternatives, calculated the value of the alternative after each attribute value read, and dropped the alternative whenever this value equalled or exceeded the lowest (overall) value of preceding alternatives in the choice set. The results from these simulations (N = 500) indeed showed that the variability of search increased when the number of alternatives increased (3 alternatives: s = 0.12; 6 alternatives: s = 0.18). In agreement with our findings, the standard deviation across alternatives marginally decreased with an increased number of attributes (4 attributes: s = 0.16; 8 attributes: s = 0.131. Thus, the observed increased search variability in the baskets task might well be explained by chance alone. The results showed that even though more information is requested in the baskets task subjects rated the decision problems as less difficult than in the apartments task. A major difference between the tasks is the dimensionality of the decision problem. In order to determine the overall values, subjects could simply add all prices in the baskets task whereas they had to integrate different units in the apartments task. Even though the subjects accurately dealt with this problem, they may for this reason have found the apartments task more difficult. Confidence levels were also higher in the baskets task than in the apartments task, which may be explained by the familiarity
of the task (Huber 1989). In the baskets task the decision rule is clearly linked to the outcome of the decision. Prices have to be added and the lowest overall price is unquestionably the best alternative. Thus, this result also suggests that knowledge on the accuracy of a specific strategy in some choice environment affects one’s confidence in the actually chosen option. To conclude, in line with the predictions subjects only switched to more noncompensatory search strategies in the apartments task, as indicated by depth of search and search variability. However, in spite of incomplete information search, accuracy remained constant over all complexity conditions. In process-tracing studies a rather specific class of tasks is used, such as choices between apartments or consumer products. By its structure, for example high dispersion of attribute weights, these tasks may have allowed for the application of noncompensatory strategies without reducing the accuracy scores. The present results suggest that more research is needed with different types of task, in order to differentiate between general task complexity effects and its interaction with specific task environments.
References Bettman.
J.R. and C.W.
Park. 1Y80. Effects of prior knowledge
choice process on consumer Research
of decision Creyer,
hchavior:
3 I.
Performance
E.H..
J.R. Bettman
H.J., D.N.
and J.W. Payne.
decision hehaviour.
problems
Decision
0.
Amsterdam:
North-Holland.
Svenson
j-21. Johnson.
models
Behavior
IYYO. The impact of accuracy and effort Journal
B.M.
of Behavioral
and
Decision
feedhnck
Making
3. I-16.
lY7Y. Lineal- regression and process-tracing
Hults
and neglected
V6ri
and M.L. research
Doherty,
questions.
lYX9. Process tracing Organizational
Behav-
Payne.
and actually processed in a decision task’. In: P.C. (eds.),
Analyzing
and
aiding
decision
pr-occsaes.
pp. 441-4.54. operators
Process and structure
E.J. and J.W.
.iYS~‘IlJ.
Organizational
Review 86. 465-385.
presented
an d A.
O., 1989. ‘Information-processing Svenson (ed\.).
and noncompensatory
Processes 43, 7.5-l 17.
O., 1983. ‘The information
Humphreys.
0.
of compensatory
and B. Kleinmuntr.
S.L. Schechtman,
methods: Contributions.
Huber,
of Consumer
33l-352.
Psychological
Ford, J.K.. N. Schmitt. ior and Human
lYN3. Measures
Kleinmuntz
models of judgment.
Huber,
and phase of the
analysis. Journal
Process tracing versus policy capturing.
and goals on adaptive Einhorn.
and experience
processes: A protocol
7. 234-2-1X.
Billings. R.S. and S.A. Marcus, Human
decision
10x5. Effort
in decision making’.
in human
In: H. Montgomery
decision making.
and accuracy
in choice.
Chichexter:
Management
Wiley.
and pp.
Science 31.
J.H. Kerstholt
/ Information
seurch and ckmce uccurucy
197
Johnson, R.D., 1987. Making judgments when information is missing: Inferences, biases, and framing effects. Acta Psychologica 66, 69-82. Kirk. R.E.. 1968. Experimental design: Procedures for the behavioral sciences. Belmont: Brooks/Cole. Klein, N.M.. 1983. Utility and decision strategies: A second look at the rational decision maker. Organizational Behavior and Human Performance 31. l-25. Koriat, A., S. Lichtenstein and B. Fischhoff, 1980. Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory 6, 1077118. Levin, I.P.. D.P. Chapman and R.D. Johnson, 1988. Confidence in judgments based on incomplete information. Journal of Behavioral Decision Making 1, 29-41. Olshavsky, R.W., 1979. Task complexity and contingent processing in decision making: A replication and extension. Organizational Behavior and Human Performance 24, 300&316. Paquette, L. and T. Kida, 1988. The effects of decision strategy and task complexity on decision performance. Organizational Behavior and Human Decision Processes 41, 128-142. Payne, J.W.. 1976. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance 16, 366-387. Payne, J.W.. 1982. Contingent decision behavior. Psychological Bulletin 92, 3X2-402. Payne. J.W., J.R. Bettman and E.J. Johnson, 19x8. Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning. Memory and Cognition 14, 5344552. Peterson, D.K. and G.F. Pitz, 1988. Confidence, uncertainty and the use of information. Journal of Experimental Psychology: Learning. Memory and Cognition 14, 85-92. Sundstrom. G.A.. 1987. Information search and decision making: The effects of information displays. Acta Psychologica 65, 1655179. Swenson. O., 1970. Process descriptions of decision making. Organizational Behavior and Human Performance 23, X6- 112. Westenherg, M.R.M. and P. Koele, 1990. ‘Response modes and decision strategies’. In: K. Borcherding. 0.1. Larichev and D.M. Messick teds.), Contemporary issues in decision making. Amsterdam: North-Holland. pp. 159%170. Zakay. B.D. and S. Wooler, 1984. Time pressure, training and decision effectiveness. Ergonomics 27, 273-284.