- Email: [email protected]
Analysis of Predecisional Information Search Patterns1
ANALYSIS OF PREDECISIONAL INFORMATION SEARCH PATTERNS' Joshua KLAYMAN Center for Decision Research, Graduate School of Business University of Chicago, U S A .
Abstract Recent interest in complex and varied decision strategies has highlighted the need for more sophisticated process tracing analyses. e.g., in analyzing information gathering patterns. Earlier studies have classified strategies as highllow proportion of available information used, constantlvariable amount of search across alternatives, and intra-linterdimensional direction of search. However, more powerful analyses are needed, since the search characteristics of a given strategy may be variable and highly taskdependent. Two major means of improving search analysis are discussed: (a) the use of task-specific simulations to establish the search characteristics expected from different strategies; and (b) the analysis of additional search characteristics, such as the extent to which future information search is controlled by prior information (contingency), and different types of search variability. An experimental example of the use of these techniques is presented. Applications are proposed in three areas: (a) the study of sequential combinations of decision rules, and multi-phase decision making; (b) exploration of the possibility that there exists continuous variation among strategies along various parameters, rather than a set of discrete rules; and (c) the investigation of how decision makers adapt strategy t o task.
'
This paper is based in part on work supported by a Graduate Dissertation Fellowship and Grant from the University of Minnesota Additional support for computer analyses was provided by Hampshire College (Amherst, Massachusetts), and by the University of Chicago, Graduate School of Business. Simulatlon programs were developed with the collaboration of Eric Smlth at Hampshire College. Thanks to Hillel Emhorn, J.E. Russo, Paul Schoemaker, and especially Robin Hogarth for comments on an earlier draft, and to Ola Svenson and two anonymous reviewers for helpful editorial suggestions.
26
402
J. Klayman
Introduction Human decisions have, in recent years, been seen as based on complex predecision processes. The decision maker is seen as having a broad repertoire of available strategies, the choice and application of which are dependent on characteristics of the task (cf. Montgomery, this volume; Svenson, 1979). Thus, there is a need for process tracing analyses which are sensitive enough to identify complex decision rules, and to represent the effects of task characteristics on decision strategies. The present paper discusses the analysis of iriforniation gathering patterns as a tool for process tracing. Two approaches for expanding the analysis of search patterns are proposed: (a) computer simulation of decision strategies; and (b) measurement of a greater variety of search characteristics. Probably the best example of the use of information gathering data is provided by Payne (1976). In this study, people chose apartments on the basis of matrices of information provided on "information boards". Three search characteristics were used in an essentially dichotomous way to classify strategies, as follows (abbreviations mine): Proportion of information searched (prop search): Searching a large proportion of the available information (high prop search) is indicative of compensatory strategies (e.g., additive, additive-difference); low prop search, o n the other hand, indicates noncompensatory strategies (e.g., conjunctive, elimination by aspects). Variability in proportion searched across alternatives (variab by a h ) : Search of the same proportion of information on each alternative (absence of wriab by a h ) indicates compensatory strategies; presence of variab by afts indicates noncompensatory strategies. (3) Direction of search (direction): An interdimensional direction indicates alternative-wise strategies (e.g., additive, conjunctive); an intradimensional direction indicates dimension-wise search (e.g., additive difference, elimination by aspects). Such a categorical treatment of search characteristics makes only linuted use of a potentially rich source of information. The basic problem is that decision strategies -provide more detailed specifications of search patterns w h c h are task-specific and cannot be adequately captured by the broad classifications mentioned above. In particular, the nature of information search may be affected by factors such as the stringency of pass/fail criteria, the distribution of values on different dimensions across alternatives, and the number of dimensions and alternatives.
ANALYSIS OF SEARCH PATTERNS
40 3
Figures 1-4 illustrate the effect of task characteristics on the measures used by Payne (1976) for various strategies.* Each figure displays a matrix of information, with alternatives as rows (2 through S ) , and dimensions as colums (A through F). The cells are numbered to indicate the order of search which would occur with the given strategy. Information is represented by a "+" or "-", indicating a value above the subject's criterion of acceptability ("pass") or below ("fail"), respectively. Shaded cells represent information w h c h is not examined by the decision maker. Vuriab by a h is measured by the standard deviation of the proportion of information searched per alternative across the set of alternatives. For direction, a score of -1 .O represents strictly dimension-wise (intradimensional) search and +1.0 strictly alternative-wise (interdimensional) search. Inter - Intra where Inrra is the number of This measure is computed asInter + Intra instances in which the nth 1 item searched is of the same dimension as the nth, and Infer is the number in which the n t h + 1 is of the same alternative as the nth. Figure 1 shows the effects of a change in pass/fail criteriaon search, using an elimination by aspects (EBA) rule. The criteria in Example 1 b are set lower than in la, so that a few "fail" values are now considered "passing". With the relaxed criteria in Example 1 b, prop info and variab by alts are both much greater, and the direction of search is more mixed. Note that these differences result only from a change in evaluation criteria, with no change of search strategy. Figure 2 illustrates the effects of differences in the distribution of values in the matrix, using alternative focused conjunctive search. Note that Examples 2a and 2b differ only with respect t o the distribution of pass and fail values; the overall proportions of pass t o fail values are identical. The seemingly modest difference between these two examples produces important differences in search characteristics. In particular, the direction of search shifts from a lareely alternative-wise +0.71 to a randomlooking 0.00, and the proportion of search decreases considerably. In Figure 3 the two matrices are identical, except that in 3b one more of the available attributes is considered essential. As a result, 3b requires twice the amount of information search to find a satisfactory alternative, and variability is also greater.
+
* The rules modeled here are specified in the appendix. They represent the author's interpretation of the rules considered by Payne (1976) and earller by Tversky (1969, 1972).
26*
404
J. Klayrnan
A
B
C
,~
DIMENSIONS
la
D
A
E
E
C
D
E
Z Y
X W
V
U T S .50
Prop info: .35 Variob by olts: ,177 Direction; -.83
,328
- 64
Fijpre 1. Effects of Change in Stringency of PasslFail Criteria, with an EBA Search Strategy
A
2a
B
DIMENSIONS A
C
2b
B
C
U Prop info: .61 Variab by altv ,390 Direction: +.71
.44 ,344 0
Figure 2. Effects of Change in Distributionof Pass and Fail Values, with a Conjunctive Search Strategy.
ANALYSIS OF SEARCH PATTERNS
405
Figure 3. Effects of Change m Number of Critical Dimensions, with a Conjunctive Search Strategy, (Dimension D is considered critical in Example 3a, but not in 3b.)
DIMENSIONS
Figure 4. Effects of Change in Number of Available Alternatives, with a Lexicographic Search Strategy
406
J . Klayman
Finally, Figure 4 illustrates the effects of a number of available alternatives with a minimum-difference lexicographic (lexico) strategy. The most striking difference here is in the direction of search, being mostly intradimensional in 4a, but mostly interdimensional in 4b. After the first pair of alternatives, search of new information is largely alternative-wise, although comparison is being made to a previously searched alternative. The same effect is obtained with the additive-difference strategy. The above examples illustrate only a few of the ways in which search characteristics are task-dependent. The upshot is that the question of what t o expect from different decision strategies is complicated, and cannot be answered by broad generalizations. Rather, it is necessary to consider the search patterns implied by different strategies in a particular decision situation. Here, task-specific simulations of search strategies can be useful. The tasks and task-manipulations of interest can be analyzed, then, in terms of how different simulated strategies function on the tasks in question. By so doing, one can establish theoretical points of comparison for the interpretation of human search data. The first requirement for this process is the specification of the step-by-step search implications of the various decision strategies. These specifications, and simulations based o n them, also allow the measurement and analysis of a number of search characteristics beyond the three discussed above. One class of such measures are referred to here as “contingency” measures. These indicate the extent t o which future search is guided by prior information. One can measure different types of contingency as the proportion of search moves which conform to a given contingency rule, out of all instances where the rule applies. Simulations can then provide theoretical comparison values for specific decision tasks. For each noncompensatory strategy modeled here there is a different process by which ”what to search next” depends o n “what has been discovered so far” (see appendix). For example, in the conjunctive strategy, if a fail value is revealed, you change alternatives, but if a pass value is revealed, you may change dimensions, staying with the same alternative. Thus, the type of search move is contingent on the value of the immediate prior information. The lexico strategy has a similar contingency rule, but based on differences between pairs of values. The rule for the EBA strategy is also like that for the conjunctive, except that the information is stored so that one knows which alternative to consider on the next dimension (i.e., only those which passed on the previous dimension). In contrast t o these rules, strictly compensatory strategies do not operate contingently, but instead rely on exhaustive search of those dimensions deemed relevant to the formation of an overall evaluation of an alternative. Thus, contingency
ANALYSIS OF SEARCH PATTERNS
407
measures are useful because they distinguish between compensatory and noncompensatory strategies more directly than other measures do, and they can also help distinguish among different noncompensatory strategies, A second class of search measures supplement Payne’s (1976) measure of variability of search. Recall that Payne measured variability in information searched per alternative, across alternatives in a given decision ( w i o b by alts). However, one can also consider the extent to which the different dimensions receive different amounts of search. Here, the measure (variab by dims) is the standard deviation of the proportion of information searched per dimension, across the set of available dimensions. Figure 5 illustrates that the two types of variability are not redundant. In fact, the distinction between them can differentiate among sources of variability (e.g., not looking at some alternatives or some dimensions at all).
DIMENSIONS
50.
A
B
C
Variab b y alis: 204 Variab by dims:.456
D
A
5b,
B
C
D
,456 ,204
Figure 5. Illustration of the Distinction Between variab by alts and varmb by dims
In measuring variability, it is also important to take into account that the size of the information matrix and the proportion of information searched both have an important impact o n how variable the decision maker can be, and how variable one would be by chance. Thus, standardized scores should be used, based on the distributions of all possible patterns of Variability which would be generated by random search with a given size of matrix, and a given proportion of search. These standardized scores (referred t o here as *vat?&) allow search patterns to be judged against a task-specific base rate for variability.
408
J. Klayman
Empirical Illustration A recent study of decision strategies in 12-year-olds (Klayman, 1981) illustrates the use of computer simulations and measures of contingency and variability, as described above. As in Payne's (1976) study, subjects were presented with "information boards" consisting of matrices of cards, in the manner of the illustrations in Figures 1-4. Information was initially hidden from view, and subjects obtained information from the matrix one item at a time, with the understanding that they would continue until they had sufficient information to choose one of the available alternatives. The criteria for sufficient information was subjectively determined by each subject, and the costs associated with information search were simply the time and effort required in turning over the cards to reveal information. The number of alternatives and number of dimensions were varied as measures of the effects of task complexity. Patterns of information search were the principal data. In this study, 48 children made four decisions, each from a different size board (three or six alternatives, three or six dimensions). Decisions concerned one of three age-appropriate topics (e.g., buying a used bicycle). Information generally consisted of high and low value terms (e.g., "new tires" or "worn-out tires"). It was assumed that, for any item subjected t o pass/fail analysis, ths high value would be above criterion, and the low value below. The values were distributed such that neither dominance nor equivalence of alternatives was likely. The orientation of the boards (alternative as rows or as columns) was counterbalanced. The same matrices presented to subjects were used in the simulations. hgh and low values were encoded as (0, l ) , with pass/fail criteria intermediate, when used. Two compensatory strategies (additive and additive-difference) and three noncompensatory strategies (conjunctive, lexico, and EBA) were simulated, based on conceptualizations of these strategies used in previous studies (see appendix). The principal focus in this study was o n the complexity variables, i.e., number of alternatives and number of dimensions. The dependent measures were those described earlier (prop search, vuriob and *wriab measures), and two measures of contingency. One contingency measure (contificonj) was based on a conjunctive rule, the other (contin-EBA)on an EBA rule. Specifically, contin-conj was calculated as M,/M, where M is the total number of moves from the nth item searched t o the nth + l/(n = 1, ...., N; where N is total number of items), and M, is the number of such moves consistent with the following rule: If item n was a "pass", item n + 1 should be on the same alternatives as n;if n was a fail, n 1 should be on a different alternative. Contin-EBA was RE/R, where R
ANALYSIS OF SEARCH PATTERNS
409
is the total number of times that search returned to an alternative after other alternatives were search, and RE is the number of such returns consistent with this rule: Return to an alternative only if the last item searched on that alternative was a "pass". For both contingency measures, then, chance value was 0.50, and perfect adherence t o the rule scored 1.oo. The major hypothesis, as in Payne's (1976) study, was that there would be a shift from compensatory to noncompensatory strategies as complexity increased. The results with the "traditional" measures supported Payne's finding of such a shift. Prop search decreased as complexity increased, from 0.79 on the 9-item boards, to 0.54 on 36-item boards. At the same time, vuriub by alts increased from 0.219 to 0.294. However, a different picture emerged with the new search measures, described above, and with comparisons to simulation data: (1) Vurkb by alts was higher overall than the simulation values for any of the five strategies tested (a mean of 0.264, versus 0.00 for compensatory strategies, and means of 0.1 16, 0.137, and 0.234 for EBA, lexico, and conjunctive, respectively). The difference was even more pronounced with the standardized values, *vuriub by ults. (2) Variab by dims, on the other hand, was well above the value for compensatory strategies (0.199 versus O.OO), but was much lower than in the simulations of noncompensatory strategies (0.469, 0.456, and 0.369). Again, the standardized measures confirmed this relationship. (3) Continconj was moderately high (0.72), but did not vary significantly with complexity, contrary to the expectation of greater use of conjunctive strategies. (4) The EBA type of contingency was well below chance (0.37), indicating that subjects were more likely to return to an alternative which fmled on last examination, than to one which passed. This tendency increased with more dimensions. These seemingly anomalous results actually revealed an interesting pattern. First, high vuriab by ults with low vuriub by dims is characteristic of what occurs with satisficing, i.e., when some alternatives are left completely unexamined. This is illustrated in Figure 5 above. Subjects in the present study did this more often as the number of alternatives increased (on 7% of the 3-alternative boards, and 30% of the &alternative boards). Second, an EBA contingency below chance is exactly what occurs when a noncompensatory rule (e.g., conjunctive) is used for a first pass, and then more information is obtained on a second pass. In this study, multiple-pass search was operationally defined as search in which information is gathered on some alternatives after all alternatives have been searched at least once. Subjects did this more often as the number of dimensions increased (on 50% of the 3-dimension boards, and 73% of the &dimension boards).
4 10
J . Klayman
Tlurd, the moderately high value obtained on contin-conj indicates that the rules used for satisficing, or for selection for second-pass search, were partially compensatory. The data did not indicate a mix of some individual compensatory patterns and some noncompensatory. Rather, the modal individual pattern was itself mixed. In sum, the search analyses used here highlighted a pattern of both sequential and simultaneous mixing of decision rules. Changes with complexity were manifested in increased use of multiple search passes, and in increased use of satisficing. However, further investigation will be necessary to validate this interpretation, and t o test its generality, since characteristics of materials and procedures can be assumed t o affect the behavior of both human and simulated decision makers. In the study above, for example, the number of available dimensions and alternatives was finite and known, there was no missing information, and only acquisition of new information was traced; there was no information about recheclung of previously searched data. The simulations, too, involved a number of procedural assumptions (see Appendix), and the five search rules simulated by no means exhaust the set of possible strategies (cf. Svenson, 1979). Further investigation is required to understand the effects on decision strategies of variations in materials, procedures, and assump tions.
Concluding Discussion In conclusion, there are several important areas of study in which more detailed analysis of search patterns could provide important process information. One such area is the use of sequential combinations of decision rules, or multi-stage processing. The empirical investigation described above illustrates how multi-stage patterns may be identified. The investigation also suggests the intriguing possibility that decision strategies d o not exist as discrete members of a set, but instead represent points in a multidimensional space of decision strategies. Strategies would then differ along several continuous parameters. Figure 6 illustrates several possible decision rules which fall between some of the usual distinctions made among strategies. The "two-alternative EBA" (A) is a blend of the conjuno tive and EBA in its elimination rule. The "double whammy" and the "additive with truncationff rules (B) represent two different points between the purely compensatory and purely noncompensatory rules. The "three-level lexico" rule is one step along the line from dichotomous evaluation to interval scaling. Other possibilities for continuous variation among strategies are suggested by Einhorn (1970) and by Montgomery and
ANALYSIS OF SEARCH PATTERNS
41 1
A Mixing contingency rules
2-011 EBB "Consider further if pass I + ) on A and 8"
A
DIMENSIONS B C D
Note Conlunctivs contingency EBA contingency =I00
B
Parlial compensation
Double whomrny (sliqhtly compensatory) "Eliminate if two toilures"
DIMENSIONS A B C D
Additive with truncation (most compensatory) "If < 20 on f i r s t three dimensios. quit"
21' 6 1 ' 6
cn W
Z Y
t z x
ti
Note Conjunctive con tingency = 0 67
3 9 ~ 4 81
E
5
I
V
I
C In-between scalin9
3-1eve1iexico~"Ehninate if large difference. or net Iwo small"
DIMENSIONS C D E
F
W' over X
-u Figure 6. Examples of "Compromise"Strategies
~
Variability low but not 0
5a W
B
:
*,
DIMENSIONS A B C D
412
J. Klayman
Svenson (1976). The question, then, becomes not whether a strategy is compeilsatory, but rather, how compensatory it is, how variable, how conjunctive, and so on. Finally, the most important use of task-specific simulations and expanded search measures is to study how strategies are adapted to tasks. Although the question of task complexity has received some investigation, as in Payne's 1976 study, there are other important task factors which have received less attention; e.g., the demand for accuracy of judgment, costs of information search, and effort-related factors such as time restraints, competing tasks, and the availability of aids to memory, computation, e t c (cf. Hogarth, 1975). The use of more detailed analyses can make information-gathering measures more powerful as process tracing tools. By combining these measures with other process tracing methods, and by establishing taskspecific points of comparison for different strategies, one can formulate testable predictions concerning the effects of many important types of task characteristics.
Appendix The following are summaries of the search rules used in Figures 1-4, and in the simulations described (Klayman, 1981). It must be noted that these are not the only possible representations of the five named strategies (cf. Svenson, 1979) since assumptions must be made concerning the translation of algebraic models or axiomatic systems into behavioral terms. These models represent cases in which a unique choice can be obtained through application of a single "pure" strategy. A ddirive. "Search one alternative, across dimensions, until all dimensions are examined which might contribute significantly to the overall value of an alternative. When this is done, begin .examination of a new alternative. Continue until all alternatives are examined." The choice is the alternative with the b h e s t overall evaluation, based on a weighted sum of values for that alternative, across dimensions. It is assumed here that (a) the dimensions need not be searched in any particular order; (b) exact ties among alternatives do not occur; and (c) there is no "truncation", in which further search of an alternative is halted because absolute maximum values on further dimensions could not compensate sufficiently for the low values thus far obtained. Additive difference. "Examine and compare huo alternatives on one dimension. Then consider the same pair on another dimension. Continue until all dimensions have been examined which might contribute signifi-
ANALYSIS OF SEARCH PAll'ERNS
413
cantly to the overall difference in value between a pair of alternatives. When this is done, use the same procedure to compare the better of the pair to a new, third alternative. Continue until all alternatives are examined." The "better" alternative is the one favored overall in a weighted sum of value differences, across dimensions. The final choice is the better of the final pair of alternatives. Assumptions here correspond to those for the additive rule. Conjunctive (by alternatives). "Search one alternative, across dimensions, as long as values are above the pass/fail criterion for each dimension. Stop at the first observation of a 'fail' value, and begin search on a new alternative. If an alternative has been searched on all dimensions for which a criterion of minimum acceptability exists, and no 'fail' values are obtained, choose that alternative, and do not search others." Some assumptions here are: (a) search is oriented toward satisficing, in that alternatives are evaluated one at a time, and search ceases with the first completely acceptable alternative; and (b) dimensions are searched in a fixed order across alternatives. In addition, there must be some process for handling "default" (cases in which all alternatives fail). In the simulations, the choice in case of default was that alternative which was most searched before failing. If a unique choice did not result, the case was discarded. A fixed order of dimension search is modeled because searching dimensions in order of importance, or in order of likelihood of failure, enhances optimality or efficiency, respectively. Lexicographic (minimum-difference). Search is pairwise, as with additive difference, except that differences are classified as either significant or insignificant: "If a difference is significant, stop search on this pair. Next, use the same procedure to compare the favored alternative of the pair to a new, third alternative. Continue until all alternatives have been considered." The chosen alternative is the favored one of the last pair. It is assumed that (a) for each dimension considered, there exists a minimum difference, 4 such that differences less than 4 are ignored, and differences greater than Ad are sufficient to eliminate the disfavored alternative; and (b) dimensions are searched in a fixed order. Default is possible in that exact ties can occur (i.e., all differences between two alternatives may be below 4).In simulations, this was handled by allowinga temporary 3-way comparison process. If ties persisted, the case was discarded. Elimination by aspects. "Look at all alternatives on one dimension. Then, go to the next dimension and examine those alternatives which passed the acceptability criterion of the previous dimension. Continue until only one alternative passes on the dimension considered." That last alternative is chosen. Default is possible in that the final dimension may eliminate all remaining alternatives. In the simulations, those cases were excluded.
414
J. Klayman
References Einhorn, H.J., 1970. The use of nonlinear, noncompensatory models in decision making Psychological Bulletin, 73, 221 -230. Hogarth, R.M., 1975. Decision time as a function of task complexity. 1n:D. Wendt and C. Vlek (eds.), Utility, Probability, and Human Decision Making Dordrecht, Holland: Reidel. Klayman, J., 1981. The analysis of decision making strategies in children. Working paper. Center for Decision Research, University of Chicago, Graduate School of Business. Montgomery, H., 1983. Decision rules and the search for a dominance structure: Toward a process model of decision making. In this volume, 343-369. Montgomery, H. and 0. Svenson, 1976. On decision rules and information processing strategies for choice among multiattribute alternatives. Scandinavian Journal o f P s y c h o l o ~ 17, , 283-291. Payne, J.W., 1976. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance, 16, 366- 387. Svenson, O., 1979. Process descriptions of decision making, Organizational Behavior and Human Performance, 23, 86- 1 12. Tversky, A., 1969. Intransitivity of preferences. Psychological Review, 76, 3 1-48. Tversky, A., 1972. Elimination by aspects: A theory of choice. Psychological Review. 79, 28 1- 299.
Abstract Recent interest in complex and varied decision strategies has highlighted the need for more sophisticated process tracing analyses. e.g., in analyzing information gathering patterns. Earlier studies have classified strategies as highllow proportion of available information used, constantlvariable amount of search across alternatives, and intra-linterdimensional direction of search. However, more powerful analyses are needed, since the search characteristics of a given strategy may be variable and highly taskdependent. Two major means of improving search analysis are discussed: (a) the use of task-specific simulations to establish the search characteristics expected from different strategies; and (b) the analysis of additional search characteristics, such as the extent to which future information search is controlled by prior information (contingency), and different types of search variability. An experimental example of the use of these techniques is presented. Applications are proposed in three areas: (a) the study of sequential combinations of decision rules, and multi-phase decision making; (b) exploration of the possibility that there exists continuous variation among strategies along various parameters, rather than a set of discrete rules; and (c) the investigation of how decision makers adapt strategy t o task.
'
This paper is based in part on work supported by a Graduate Dissertation Fellowship and Grant from the University of Minnesota Additional support for computer analyses was provided by Hampshire College (Amherst, Massachusetts), and by the University of Chicago, Graduate School of Business. Simulatlon programs were developed with the collaboration of Eric Smlth at Hampshire College. Thanks to Hillel Emhorn, J.E. Russo, Paul Schoemaker, and especially Robin Hogarth for comments on an earlier draft, and to Ola Svenson and two anonymous reviewers for helpful editorial suggestions.
26
402
J. Klayman
Introduction Human decisions have, in recent years, been seen as based on complex predecision processes. The decision maker is seen as having a broad repertoire of available strategies, the choice and application of which are dependent on characteristics of the task (cf. Montgomery, this volume; Svenson, 1979). Thus, there is a need for process tracing analyses which are sensitive enough to identify complex decision rules, and to represent the effects of task characteristics on decision strategies. The present paper discusses the analysis of iriforniation gathering patterns as a tool for process tracing. Two approaches for expanding the analysis of search patterns are proposed: (a) computer simulation of decision strategies; and (b) measurement of a greater variety of search characteristics. Probably the best example of the use of information gathering data is provided by Payne (1976). In this study, people chose apartments on the basis of matrices of information provided on "information boards". Three search characteristics were used in an essentially dichotomous way to classify strategies, as follows (abbreviations mine): Proportion of information searched (prop search): Searching a large proportion of the available information (high prop search) is indicative of compensatory strategies (e.g., additive, additive-difference); low prop search, o n the other hand, indicates noncompensatory strategies (e.g., conjunctive, elimination by aspects). Variability in proportion searched across alternatives (variab by a h ) : Search of the same proportion of information on each alternative (absence of wriab by a h ) indicates compensatory strategies; presence of variab by afts indicates noncompensatory strategies. (3) Direction of search (direction): An interdimensional direction indicates alternative-wise strategies (e.g., additive, conjunctive); an intradimensional direction indicates dimension-wise search (e.g., additive difference, elimination by aspects). Such a categorical treatment of search characteristics makes only linuted use of a potentially rich source of information. The basic problem is that decision strategies -provide more detailed specifications of search patterns w h c h are task-specific and cannot be adequately captured by the broad classifications mentioned above. In particular, the nature of information search may be affected by factors such as the stringency of pass/fail criteria, the distribution of values on different dimensions across alternatives, and the number of dimensions and alternatives.
ANALYSIS OF SEARCH PATTERNS
40 3
Figures 1-4 illustrate the effect of task characteristics on the measures used by Payne (1976) for various strategies.* Each figure displays a matrix of information, with alternatives as rows (2 through S ) , and dimensions as colums (A through F). The cells are numbered to indicate the order of search which would occur with the given strategy. Information is represented by a "+" or "-", indicating a value above the subject's criterion of acceptability ("pass") or below ("fail"), respectively. Shaded cells represent information w h c h is not examined by the decision maker. Vuriab by a h is measured by the standard deviation of the proportion of information searched per alternative across the set of alternatives. For direction, a score of -1 .O represents strictly dimension-wise (intradimensional) search and +1.0 strictly alternative-wise (interdimensional) search. Inter - Intra where Inrra is the number of This measure is computed asInter + Intra instances in which the nth 1 item searched is of the same dimension as the nth, and Infer is the number in which the n t h + 1 is of the same alternative as the nth. Figure 1 shows the effects of a change in pass/fail criteriaon search, using an elimination by aspects (EBA) rule. The criteria in Example 1 b are set lower than in la, so that a few "fail" values are now considered "passing". With the relaxed criteria in Example 1 b, prop info and variab by alts are both much greater, and the direction of search is more mixed. Note that these differences result only from a change in evaluation criteria, with no change of search strategy. Figure 2 illustrates the effects of differences in the distribution of values in the matrix, using alternative focused conjunctive search. Note that Examples 2a and 2b differ only with respect t o the distribution of pass and fail values; the overall proportions of pass t o fail values are identical. The seemingly modest difference between these two examples produces important differences in search characteristics. In particular, the direction of search shifts from a lareely alternative-wise +0.71 to a randomlooking 0.00, and the proportion of search decreases considerably. In Figure 3 the two matrices are identical, except that in 3b one more of the available attributes is considered essential. As a result, 3b requires twice the amount of information search to find a satisfactory alternative, and variability is also greater.
+
* The rules modeled here are specified in the appendix. They represent the author's interpretation of the rules considered by Payne (1976) and earller by Tversky (1969, 1972).
26*
404
J. Klayrnan
A
B
C
,~
DIMENSIONS
la
D
A
E
E
C
D
E
Z Y
X W
V
U T S .50
Prop info: .35 Variob by olts: ,177 Direction; -.83
,328
- 64
Fijpre 1. Effects of Change in Stringency of PasslFail Criteria, with an EBA Search Strategy
A
2a
B
DIMENSIONS A
C
2b
B
C
U Prop info: .61 Variab by altv ,390 Direction: +.71
.44 ,344 0
Figure 2. Effects of Change in Distributionof Pass and Fail Values, with a Conjunctive Search Strategy.
ANALYSIS OF SEARCH PATTERNS
405
Figure 3. Effects of Change m Number of Critical Dimensions, with a Conjunctive Search Strategy, (Dimension D is considered critical in Example 3a, but not in 3b.)
DIMENSIONS
Figure 4. Effects of Change in Number of Available Alternatives, with a Lexicographic Search Strategy
406
J . Klayman
Finally, Figure 4 illustrates the effects of a number of available alternatives with a minimum-difference lexicographic (lexico) strategy. The most striking difference here is in the direction of search, being mostly intradimensional in 4a, but mostly interdimensional in 4b. After the first pair of alternatives, search of new information is largely alternative-wise, although comparison is being made to a previously searched alternative. The same effect is obtained with the additive-difference strategy. The above examples illustrate only a few of the ways in which search characteristics are task-dependent. The upshot is that the question of what t o expect from different decision strategies is complicated, and cannot be answered by broad generalizations. Rather, it is necessary to consider the search patterns implied by different strategies in a particular decision situation. Here, task-specific simulations of search strategies can be useful. The tasks and task-manipulations of interest can be analyzed, then, in terms of how different simulated strategies function on the tasks in question. By so doing, one can establish theoretical points of comparison for the interpretation of human search data. The first requirement for this process is the specification of the step-by-step search implications of the various decision strategies. These specifications, and simulations based o n them, also allow the measurement and analysis of a number of search characteristics beyond the three discussed above. One class of such measures are referred to here as “contingency” measures. These indicate the extent t o which future search is guided by prior information. One can measure different types of contingency as the proportion of search moves which conform to a given contingency rule, out of all instances where the rule applies. Simulations can then provide theoretical comparison values for specific decision tasks. For each noncompensatory strategy modeled here there is a different process by which ”what to search next” depends o n “what has been discovered so far” (see appendix). For example, in the conjunctive strategy, if a fail value is revealed, you change alternatives, but if a pass value is revealed, you may change dimensions, staying with the same alternative. Thus, the type of search move is contingent on the value of the immediate prior information. The lexico strategy has a similar contingency rule, but based on differences between pairs of values. The rule for the EBA strategy is also like that for the conjunctive, except that the information is stored so that one knows which alternative to consider on the next dimension (i.e., only those which passed on the previous dimension). In contrast t o these rules, strictly compensatory strategies do not operate contingently, but instead rely on exhaustive search of those dimensions deemed relevant to the formation of an overall evaluation of an alternative. Thus, contingency
ANALYSIS OF SEARCH PATTERNS
407
measures are useful because they distinguish between compensatory and noncompensatory strategies more directly than other measures do, and they can also help distinguish among different noncompensatory strategies, A second class of search measures supplement Payne’s (1976) measure of variability of search. Recall that Payne measured variability in information searched per alternative, across alternatives in a given decision ( w i o b by alts). However, one can also consider the extent to which the different dimensions receive different amounts of search. Here, the measure (variab by dims) is the standard deviation of the proportion of information searched per dimension, across the set of available dimensions. Figure 5 illustrates that the two types of variability are not redundant. In fact, the distinction between them can differentiate among sources of variability (e.g., not looking at some alternatives or some dimensions at all).
DIMENSIONS
50.
A
B
C
Variab b y alis: 204 Variab by dims:.456
D
A
5b,
B
C
D
,456 ,204
Figure 5. Illustration of the Distinction Between variab by alts and varmb by dims
In measuring variability, it is also important to take into account that the size of the information matrix and the proportion of information searched both have an important impact o n how variable the decision maker can be, and how variable one would be by chance. Thus, standardized scores should be used, based on the distributions of all possible patterns of Variability which would be generated by random search with a given size of matrix, and a given proportion of search. These standardized scores (referred t o here as *vat?&) allow search patterns to be judged against a task-specific base rate for variability.
408
J. Klayman
Empirical Illustration A recent study of decision strategies in 12-year-olds (Klayman, 1981) illustrates the use of computer simulations and measures of contingency and variability, as described above. As in Payne's (1976) study, subjects were presented with "information boards" consisting of matrices of cards, in the manner of the illustrations in Figures 1-4. Information was initially hidden from view, and subjects obtained information from the matrix one item at a time, with the understanding that they would continue until they had sufficient information to choose one of the available alternatives. The criteria for sufficient information was subjectively determined by each subject, and the costs associated with information search were simply the time and effort required in turning over the cards to reveal information. The number of alternatives and number of dimensions were varied as measures of the effects of task complexity. Patterns of information search were the principal data. In this study, 48 children made four decisions, each from a different size board (three or six alternatives, three or six dimensions). Decisions concerned one of three age-appropriate topics (e.g., buying a used bicycle). Information generally consisted of high and low value terms (e.g., "new tires" or "worn-out tires"). It was assumed that, for any item subjected t o pass/fail analysis, ths high value would be above criterion, and the low value below. The values were distributed such that neither dominance nor equivalence of alternatives was likely. The orientation of the boards (alternative as rows or as columns) was counterbalanced. The same matrices presented to subjects were used in the simulations. hgh and low values were encoded as (0, l ) , with pass/fail criteria intermediate, when used. Two compensatory strategies (additive and additive-difference) and three noncompensatory strategies (conjunctive, lexico, and EBA) were simulated, based on conceptualizations of these strategies used in previous studies (see appendix). The principal focus in this study was o n the complexity variables, i.e., number of alternatives and number of dimensions. The dependent measures were those described earlier (prop search, vuriob and *wriab measures), and two measures of contingency. One contingency measure (contificonj) was based on a conjunctive rule, the other (contin-EBA)on an EBA rule. Specifically, contin-conj was calculated as M,/M, where M is the total number of moves from the nth item searched t o the nth + l/(n = 1, ...., N; where N is total number of items), and M, is the number of such moves consistent with the following rule: If item n was a "pass", item n + 1 should be on the same alternatives as n;if n was a fail, n 1 should be on a different alternative. Contin-EBA was RE/R, where R
ANALYSIS OF SEARCH PATTERNS
409
is the total number of times that search returned to an alternative after other alternatives were search, and RE is the number of such returns consistent with this rule: Return to an alternative only if the last item searched on that alternative was a "pass". For both contingency measures, then, chance value was 0.50, and perfect adherence t o the rule scored 1.oo. The major hypothesis, as in Payne's (1976) study, was that there would be a shift from compensatory to noncompensatory strategies as complexity increased. The results with the "traditional" measures supported Payne's finding of such a shift. Prop search decreased as complexity increased, from 0.79 on the 9-item boards, to 0.54 on 36-item boards. At the same time, vuriub by alts increased from 0.219 to 0.294. However, a different picture emerged with the new search measures, described above, and with comparisons to simulation data: (1) Vurkb by alts was higher overall than the simulation values for any of the five strategies tested (a mean of 0.264, versus 0.00 for compensatory strategies, and means of 0.1 16, 0.137, and 0.234 for EBA, lexico, and conjunctive, respectively). The difference was even more pronounced with the standardized values, *vuriub by ults. (2) Variab by dims, on the other hand, was well above the value for compensatory strategies (0.199 versus O.OO), but was much lower than in the simulations of noncompensatory strategies (0.469, 0.456, and 0.369). Again, the standardized measures confirmed this relationship. (3) Continconj was moderately high (0.72), but did not vary significantly with complexity, contrary to the expectation of greater use of conjunctive strategies. (4) The EBA type of contingency was well below chance (0.37), indicating that subjects were more likely to return to an alternative which fmled on last examination, than to one which passed. This tendency increased with more dimensions. These seemingly anomalous results actually revealed an interesting pattern. First, high vuriab by ults with low vuriub by dims is characteristic of what occurs with satisficing, i.e., when some alternatives are left completely unexamined. This is illustrated in Figure 5 above. Subjects in the present study did this more often as the number of alternatives increased (on 7% of the 3-alternative boards, and 30% of the &alternative boards). Second, an EBA contingency below chance is exactly what occurs when a noncompensatory rule (e.g., conjunctive) is used for a first pass, and then more information is obtained on a second pass. In this study, multiple-pass search was operationally defined as search in which information is gathered on some alternatives after all alternatives have been searched at least once. Subjects did this more often as the number of dimensions increased (on 50% of the 3-dimension boards, and 73% of the &dimension boards).
4 10
J . Klayman
Tlurd, the moderately high value obtained on contin-conj indicates that the rules used for satisficing, or for selection for second-pass search, were partially compensatory. The data did not indicate a mix of some individual compensatory patterns and some noncompensatory. Rather, the modal individual pattern was itself mixed. In sum, the search analyses used here highlighted a pattern of both sequential and simultaneous mixing of decision rules. Changes with complexity were manifested in increased use of multiple search passes, and in increased use of satisficing. However, further investigation will be necessary to validate this interpretation, and t o test its generality, since characteristics of materials and procedures can be assumed t o affect the behavior of both human and simulated decision makers. In the study above, for example, the number of available dimensions and alternatives was finite and known, there was no missing information, and only acquisition of new information was traced; there was no information about recheclung of previously searched data. The simulations, too, involved a number of procedural assumptions (see Appendix), and the five search rules simulated by no means exhaust the set of possible strategies (cf. Svenson, 1979). Further investigation is required to understand the effects on decision strategies of variations in materials, procedures, and assump tions.
Concluding Discussion In conclusion, there are several important areas of study in which more detailed analysis of search patterns could provide important process information. One such area is the use of sequential combinations of decision rules, or multi-stage processing. The empirical investigation described above illustrates how multi-stage patterns may be identified. The investigation also suggests the intriguing possibility that decision strategies d o not exist as discrete members of a set, but instead represent points in a multidimensional space of decision strategies. Strategies would then differ along several continuous parameters. Figure 6 illustrates several possible decision rules which fall between some of the usual distinctions made among strategies. The "two-alternative EBA" (A) is a blend of the conjuno tive and EBA in its elimination rule. The "double whammy" and the "additive with truncationff rules (B) represent two different points between the purely compensatory and purely noncompensatory rules. The "three-level lexico" rule is one step along the line from dichotomous evaluation to interval scaling. Other possibilities for continuous variation among strategies are suggested by Einhorn (1970) and by Montgomery and
ANALYSIS OF SEARCH PATTERNS
41 1
A Mixing contingency rules
2-011 EBB "Consider further if pass I + ) on A and 8"
A
DIMENSIONS B C D
Note Conlunctivs contingency EBA contingency =I00
B
Parlial compensation
Double whomrny (sliqhtly compensatory) "Eliminate if two toilures"
DIMENSIONS A B C D
Additive with truncation (most compensatory) "If < 20 on f i r s t three dimensios. quit"
21' 6 1 ' 6
cn W
Z Y
t z x
ti
Note Conjunctive con tingency = 0 67
3 9 ~ 4 81
E
5
I
V
I
C In-between scalin9
3-1eve1iexico~"Ehninate if large difference. or net Iwo small"
DIMENSIONS C D E
F
W' over X
-u Figure 6. Examples of "Compromise"Strategies
~
Variability low but not 0
5a W
B
:
*,
DIMENSIONS A B C D
412
J. Klayman
Svenson (1976). The question, then, becomes not whether a strategy is compeilsatory, but rather, how compensatory it is, how variable, how conjunctive, and so on. Finally, the most important use of task-specific simulations and expanded search measures is to study how strategies are adapted to tasks. Although the question of task complexity has received some investigation, as in Payne's 1976 study, there are other important task factors which have received less attention; e.g., the demand for accuracy of judgment, costs of information search, and effort-related factors such as time restraints, competing tasks, and the availability of aids to memory, computation, e t c (cf. Hogarth, 1975). The use of more detailed analyses can make information-gathering measures more powerful as process tracing tools. By combining these measures with other process tracing methods, and by establishing taskspecific points of comparison for different strategies, one can formulate testable predictions concerning the effects of many important types of task characteristics.
Appendix The following are summaries of the search rules used in Figures 1-4, and in the simulations described (Klayman, 1981). It must be noted that these are not the only possible representations of the five named strategies (cf. Svenson, 1979) since assumptions must be made concerning the translation of algebraic models or axiomatic systems into behavioral terms. These models represent cases in which a unique choice can be obtained through application of a single "pure" strategy. A ddirive. "Search one alternative, across dimensions, until all dimensions are examined which might contribute significantly to the overall value of an alternative. When this is done, begin .examination of a new alternative. Continue until all alternatives are examined." The choice is the alternative with the b h e s t overall evaluation, based on a weighted sum of values for that alternative, across dimensions. It is assumed here that (a) the dimensions need not be searched in any particular order; (b) exact ties among alternatives do not occur; and (c) there is no "truncation", in which further search of an alternative is halted because absolute maximum values on further dimensions could not compensate sufficiently for the low values thus far obtained. Additive difference. "Examine and compare huo alternatives on one dimension. Then consider the same pair on another dimension. Continue until all dimensions have been examined which might contribute signifi-
ANALYSIS OF SEARCH PAll'ERNS
413
cantly to the overall difference in value between a pair of alternatives. When this is done, use the same procedure to compare the better of the pair to a new, third alternative. Continue until all alternatives are examined." The "better" alternative is the one favored overall in a weighted sum of value differences, across dimensions. The final choice is the better of the final pair of alternatives. Assumptions here correspond to those for the additive rule. Conjunctive (by alternatives). "Search one alternative, across dimensions, as long as values are above the pass/fail criterion for each dimension. Stop at the first observation of a 'fail' value, and begin search on a new alternative. If an alternative has been searched on all dimensions for which a criterion of minimum acceptability exists, and no 'fail' values are obtained, choose that alternative, and do not search others." Some assumptions here are: (a) search is oriented toward satisficing, in that alternatives are evaluated one at a time, and search ceases with the first completely acceptable alternative; and (b) dimensions are searched in a fixed order across alternatives. In addition, there must be some process for handling "default" (cases in which all alternatives fail). In the simulations, the choice in case of default was that alternative which was most searched before failing. If a unique choice did not result, the case was discarded. A fixed order of dimension search is modeled because searching dimensions in order of importance, or in order of likelihood of failure, enhances optimality or efficiency, respectively. Lexicographic (minimum-difference). Search is pairwise, as with additive difference, except that differences are classified as either significant or insignificant: "If a difference is significant, stop search on this pair. Next, use the same procedure to compare the favored alternative of the pair to a new, third alternative. Continue until all alternatives have been considered." The chosen alternative is the favored one of the last pair. It is assumed that (a) for each dimension considered, there exists a minimum difference, 4 such that differences less than 4 are ignored, and differences greater than Ad are sufficient to eliminate the disfavored alternative; and (b) dimensions are searched in a fixed order. Default is possible in that exact ties can occur (i.e., all differences between two alternatives may be below 4).In simulations, this was handled by allowinga temporary 3-way comparison process. If ties persisted, the case was discarded. Elimination by aspects. "Look at all alternatives on one dimension. Then, go to the next dimension and examine those alternatives which passed the acceptability criterion of the previous dimension. Continue until only one alternative passes on the dimension considered." That last alternative is chosen. Default is possible in that the final dimension may eliminate all remaining alternatives. In the simulations, those cases were excluded.
414
J. Klayman
References Einhorn, H.J., 1970. The use of nonlinear, noncompensatory models in decision making Psychological Bulletin, 73, 221 -230. Hogarth, R.M., 1975. Decision time as a function of task complexity. 1n:D. Wendt and C. Vlek (eds.), Utility, Probability, and Human Decision Making Dordrecht, Holland: Reidel. Klayman, J., 1981. The analysis of decision making strategies in children. Working paper. Center for Decision Research, University of Chicago, Graduate School of Business. Montgomery, H., 1983. Decision rules and the search for a dominance structure: Toward a process model of decision making. In this volume, 343-369. Montgomery, H. and 0. Svenson, 1976. On decision rules and information processing strategies for choice among multiattribute alternatives. Scandinavian Journal o f P s y c h o l o ~ 17, , 283-291. Payne, J.W., 1976. Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance, 16, 366- 387. Svenson, O., 1979. Process descriptions of decision making, Organizational Behavior and Human Performance, 23, 86- 1 12. Tversky, A., 1969. Intransitivity of preferences. Psychological Review, 76, 3 1-48. Tversky, A., 1972. Elimination by aspects: A theory of choice. Psychological Review. 79, 28 1- 299.