A Comparison of Single-Step and Multiple-Step Transition Analyses of Multiattribute Decision Strategies

ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES

Vol. 69, No. 3, March, pp. 195–204, 1997 ARTICLE NO. OB972681

A Comparison of Single-Step and Multiple-Step Transition Analyses of Multiattribute Decision Strategies CHRISTOPHER BALL University of Melbourne, Melbourne, Australia

Evaluating information is a fundamental component of multiattribute decision making that can be guided by one of many cognitive strategies. Considerable research has examined the factors that influence strategy selection; however, the identification of strategies remains problematic. The search sequence or transitions that a decision maker uses when searching a matrix of decision information can provide important clues to the strategy guiding the processing of decision information. The most common form of strategy analysis is to examine each transition from one piece of information to the next to establish whether these transitions are primarily alternative or attribute based. However, the resulting single-step transition indices often restrict strategy identification to a quantitative measure of compensatoriness and were found to provide conflicting results for the same search data. The current paper proposes a multiple-step transition analysis that records more complex, longer transitions to provide a multivariate profile of the strategy. Empirical support for the advantages of a multiple-step transition analysis over single-step transition indices is also provided. q 1997 Academic Press

Multiattribute decisions involve a choice between two or more alternatives with each alternative described by a number of attributes. The resulting set of information can vary in quantity and quality, and needs to be processed by the decision maker in a meaningful way. The adaptive model of decision making, proposed by Payne, Bettman, and Johnson (1993), examines the strategies used by a decision maker to guide this information processing. The model proposes that a decision strategy is chosen by the decision maker from a repertoire of This research was funded in part from grants provided by the Australian Research Council and the Australian Defense Department. The author is grateful to Margaret Foddy and Fiona Holmes for providing their multiattribute binary decision data for analysis and to Leon Mann for helpful discussions on this topic. Address correspondence and reprint requests to Christopher Ball, Department of Psychology, University of Melbourne, Parkville 3052, Australia.

strategies, or alternatively, constructed on the spot to deal with the problem. The strategy used by the decision maker is dependent upon characteristics of the decision maker, the decision task, and the decision environment. The decision maker adapts to changes in these variables by choosing a different strategy for processing the decision information. Payne et al. (1993) suggest that strategy selection ultimately depends on the accuracy requirements of the decision outcome and the cognitive effort involved in executing the strategy. A decision that requires shifting through a complex amount of information (e.g., a large number of alternatives or attributes) requires a search strategy that reduces the information load (i.e., cognitive effort) on the decision maker. For example, to substantially reduce the number of applicants who need to be interviewed for a job, simple criteria can be set for a small number of important attributes (e.g., minimal education requirement and years of previous experience). Those applicants who do not meet these criteria are rejected and a short list of candidates is established. Examining how decision makers adapt to changes in the complexity of decision information is one of the most productive areas of research on strategy behavior. However, the accurate and reliable measurement of strategies can be difficult. The current manuscript introduces a new multivariate method of analysis and compares this approach with current univariate measures of strategy behavior. TYPES OF STRATEGIES

The following discussion of strategy measures focuses on data collected using the behavior process procedure. This procedure provides a sequential record of each piece of information accessed by an individual in making their decision. Typically, a computer screen is used to display the decision information in the form of a matrix defined by the number of decision alternatives and the number of attributes that describe each alternative. Each cell of the matrix provides a single piece of attribute information for one alternative. The subject

195

0749-5978/97 $25.00 Copyright q 1997 by Academic Press All rights of reproduction in any form reserved.

196

CHRISTOPHER BALL

searches through the information matrix by displaying one cell at a time, and the computer records the sequence of cells accessed by the subject. Software packages exist for displaying the decision information and recording the search record (MOUSELAB—Johnson, Payne, Schkade, & Bettman, 1991; DEAN—Ball & Mann, 1992). The sequence of cell accesses provides valuable clues to the decision strategy that guided the search. However, the identification of strategies from long, messy search sequences which only provide inferences from the sequential order, has been a major problem for researchers. Figure 1 illustrates seven hypothetical searches of a 3 3 3 decision matrix to illustrate some common strategies described in the literature. Each cell access is marked by an arrow connecting it to the next cell accessed in the search sequence, and this pair of cell accesses represents a single-step transition. Compensatory strategies are strategies that characteristically require searching through attribute information slowly and deliberately, comparing combinations of attribute values, and often, weighting attributes on their relative importance to the final decision outcome. The weightedadditive (Fig. 1a) and equal-weight (Fig. 1b) strategies are both examples of compensatory strategies. The majority of single-step transitions produced by compensatory strategies involve sequences that travel across the attributes for each alternative. One would expect compensatory strategies to be preferred by decision makers because information is evaluated in a thorough and complete manner, but the heavy information processing demands associated with these strategies often limit their popularity. In addition, research reveals that non-compensatory strategies that examine considerably less of the decision information, can in many situations, provide choices of equivalent accuracy (Payne et al., 1993). The lexicographic (Fig. 1d) and elimination-by-aspects (Fig. 1e) strategies are examples of non-compensatory strategies. Decision makers who use these strategies will compare alternatives one attribute at a time; rejecting or accepting an alternative on the basis of its value on this attribute. The majority of single-step transitions produced by noncompensatory searches of the information matrix travel along an attribute from one alternative to the next. Most researchers view compensatoriness as a bipolar dimension. Strict non-compensatory strategies (i.e., those which make no comparisons of combinations of attributes) fall at one extreme of the dimension (e.g., lexicographic and elimination-by-aspects strategies) and strict compensatory strategies (i.e., those which only make comparisons of the combined influences of attributes) fall at the opposite extreme (e.g., equal

weight and weighted additive strategies). A strategy which involves both type of comparisons would fall somewhere between these two extremes; however, it is unclear where precisely such a strategy is located on this dimension. For example, the satisficing strategy (Fig. 1c) is generally regarded as a non-compensatory strategy even though equal amounts of both transition types were utilised in this example. The majority of confirming dimensions strategy (Fig. 1f) can also be difficult to classify as the decision maker may combine differences in the values of attributes or binary values that highlight one alternative’s advantage over the other (Bockenholt, Albert, Aschenbrenner & Schmalhofer, 1991). The former method resembling a compensatory strategy while the latter method resembling a non-compensatory strategy. Problems can also arise when the decision maker combines components of different strategies to process decision information. The final example in Fig. 1 highlights such a case, when a lexicographic comparison was followed by an equal weight component. The resulting search contains a mixture of single-step transitions that are alternativebased and attribute-based. These strategies will congregate near the mid-point of the compensatory dimension. Regardless of these difficulties, the compensatory dimension has provided a useful means for helping researchers examine adaptive strategy behaviour (See reviews—Ford, Schmitt, Schechtman, Hults, & Doherty, 1989; Payne et al., 1993). SINGLE-STEP TRANSITION MEASURES

Several measures have been developed to measure strategy behavior, and the most popular of these involve the analysis of single-step transitions. Figure 2a illustrates the four single-step transitions possible when an individual searches a two dimensional array of decision information. The Type I transition results when an individual immediately reaccesses the same piece of information, and therefore, this transition does not reflect an important component of a search strategy. Type II transitions result when an individual accesses one attribute value and then another attribute value for the same alternative. These attribute values are often combined to form an overall attractiveness score for each alternative when part of a compensatory strategy. Type III transitions result when an individual accesses the same attribute information from two different alternatives. These transitions allow alternatives to be rejected or accepted for further evaluation when part of a noncompensatory strategy. The Type IV transition results when an individual accesses a cell that has nothing in common with the preceding cell access. The Type IV

197

ANALYZING MULTIATTRIBUTE DECISION STRATEGIES (a) Weighted additive (WADD)

(d) Lexicographic (LEX)

(b) Equal weight (EQW)

(c) Satisficing (SAT)

(e) Elimination by aspects (EBA)

(f) Majority of confirming dimensions (MCD)

es

(g) Combined example (LEX:EQU)

FIG. 1. Seven examples of decision strategies for a hypothetical set of decision information. The alternative selected in each case is highlighted in bold. Single-step transition indexes and multiple-step transition profiles are provided for each example.

transition can reflect a random access of the information matrix or the shift to a new sequence of Type II or Type III transitions. For example, the WADD example provided in Fig. 1a requires three Type II transitions to compare all attributes for a single alternative, followed by a Type IV transition to shift to the next alternative for comparison. Figure 1 presents the frequencies of single-step transitions for the seven examples

of strategies provided. The examples of compensatory strategies exhibit a preponderance of Type II transitions while the examples of non-compensatory strategies display a majority of Type III transitions. Payne (1976) first introduced an index of strategy behavior that measured the relative frequency of these two single-step transitions in the search record. This index calculation is described in Eq. (1).

198

CHRISTOPHER BALL (a) Single-step transitions (Types I-IV)

(b) Pairwise comparison (Type V)

(c) 2 attribute comparison (Type VI)

(d) 3 attribute comparison (Type VII)

FIG. 2. Examples of single-step transitions (Type I–Type IV) and multiple-step transitions (Type V–VII).

Strategy Index 5

NII 2 NIII NII 1 NIII

(1)

NII refers to the number of Type II transitions recorded in the decision maker’s search sequence. NIII refers to the number of Type III transitions. The Strategy Index (SI) will vary from 21 to 11, depending on the ratio of Type II transitions to Type III transitions. Numerous studies have reported significant changes in the value of SI after experimental manipulations of the decision task. These findings have been interpreted to reflect changes in the strategy preferred by the decision makers. One of the most consistent findings reported by researchers is a shift to simpler, non-compensatory strategies (i.e., negative SI values) as matrix complexity increases with an increase in the number of decision alternatives or the number of attributes describing each alternative (Ford et al., 1989; Payne et al., 1993). Although, significant differences in the SI values result, the values obtained do not describe the specific strategies preferred by decision makers. Researchers will often refer to specific strategies in their conclusions pertaining to such findings, but such conclusions are extremely speculative. Recently, Bockenholt and Hynan (1994) described other problems associated with the interpretation of SI values when the numbers of alternatives and attributes are varied. They reported that when the number of attributes is greater than the number of alternatives a positive SI value is more likely to result, and when the number of attributes is less than the number of alternatives a negative SI value is more likely to be obtained. Not only did Bockenholt and Hynan find that the distribution of SI values can be skewed as a result of changes to the dimensions of the decision matrix,

but in general, extreme values of SI occur with a higher probability than do intermediate values of SI. Bockenholt and Hynan (1994) have proposed an alternative index with better statistical properties and less sensitivity to changes in the dimensions of the information matrix. Their index incorporates a chi-square calculation to compare the observed frequencies of the different single-step transitions against those expected by chance. The resulting formula is presented in Eq. (2). Strategy measure 5 !N((NaltNatt/N)(NII 2 NIII) 2 (Natt 2 Nalt)) !Nalt2 (Natt 2 1) 1 Natt2(Nalt 2 1)

(2)

NII and NIII again refer to the number of Type II and Type III transitions. Nalt is the number of decision alternatives and Natt is the number of attributes that describe each alternative. N is the total number of accesses of the information matrix undertaken by the subject. The distribution of Strategy Measure (SM) values will be normal. Unfortunately, the distribution of SM values also appears sensitive to changes in the dimensions of the information matrix, as well as, to the total number of transitions performed by the decision maker. The values obtained for SI are constrained within a range from 21 to 11, whereas values of SM are not so constrained. In fact, the range of values that can be obtained using the SM formula can differ dramatically as a function of the dimensions of the information matrix. For example, a full search of a three alternative by three attribute matrix using an additive strategy (refer to Fig. 1a or Fig. 1b) results in six Type II transitions, zero Type III transitions, and two Type IV transitions. This combination of transitions produces a value of 3.18 for SM.

ANALYZING MULTIATTRIBUTE DECISION STRATEGIES

However, the same strategy for a six by six matrix (30 Type II transitions, zero Type III transitions, and five Type IV transitions) results in a much larger value of 9.62 for SM. The SI value remains 1.0 for both searches. Clearly, the variability in SM values when comparing searches from different sized matrices can be problematic when mean values are compared, as the calculation of the mean is very sensitive to extreme values. The Monte Carlo analysis performed by Bockenholt and Hynan (1994) using simulated data, reports superior performance of the SM index over the SI index, however no empirical research has yet to be presented to confirm this advantage. One aim of the current paper is to compare the results from these two strategy indexes using search data obtained from an experimental setting. MULTIPLE-STEP TRANSITIONS

There would appear three limitations with singlestep transition measures of decision strategies. First, they restrict the analysis to single steps in the search sequence, and therefore, do not use all the available information provided by the search record. Second, the measures often restrict comparisons of strategies to a single quantitative dimension of compensatoriness, and consequently, provide little knowledge regarding the actual strategies used by the decision maker. Third, the distributions of such measures appear dependent on the dimensions of the decision matrix. The introduction of multiple-step transitions can overcome these limitations. The examples provided in Fig. 1 highlight how most strategies reveal patterns of linked single-step transitions. For example, additive strategies are comprised of Type II transitions for a specified sequence of attributes followed by a single Type IV transition. This sequence of single-step transitions can be combined to represent a multiple-step transition indicative of these strategies. The introduction of multiple-step transitions provides specific strategy classification by incorporating more of the search information and examining the relationships between patterns of single-step transitions. Keeping the resulting set of transitions as multivariate measures, rather than converting them to a univariate measure, may also help to avoid the problems associated with the statistical distribution of single-step transition indices. A thorough review of the decision strategy literature indicated the following multiple-step transitions should be recorded (in addition to the four previously identified single-step transitions) for maximum discrimination of decision strategies. Pairwise comparison (type V). A comparison of the same two alternatives on more than one attribute in

199

succession (refer to Fig. 2b). This comparison is fundamental to the majority of confirming dimensions strategy (MCD) described in Fig. 1f and has been reported previously by Russo and Rosen (1975). Note that this comparison could also be achieved by reversing the order of alternatives compared for each attribute (Russo & Dosher, 1983). This distinction will be discussed in more detail when analysing strategies for making multiattribute binary choices. Multiattribute comparisons (type VI 2 type k, where k 5 number of attributes 1 4). A comparison of two or more alternatives on two or more attributes that are examined in the same order for each alternative (refer to Fig. 2c and Fig. 2d). The maximum number of these types of transitions will be a function of the number of attributes describing the various alternatives. For example, when alternatives are described by three attributes, a subject may compare alternatives on the combined influence of two attributes (Type VI) or three attributes (Type VII). These transitions are featured in additive difference strategies. The identification of a strategy from a search sequence now requires the recording of both multiplestep and single-step transitions, and this introduces some significant changes from simply recording singlestep transitions. The number of cell accesses comprising a transition will now differ, as single-step transitions only involve two cell accesses, whereas multiattribute comparison transitions can involve much larger numbers of cell accesses. For this reason, the frequencies of cell accesses for each transition were recorded as a percentage of the total number of information items accessed in the search sequence. Percentages were chosen to provide a profile of the search strategy, rather than the frequency of different transitions, as they allow transitions involving a longer sequence of information accesses (e.g., multiattribute comparisons) to receive the weight they deserve in terms of their contribution to the total search of decision information. This weighting procedure also reduces the confounding effect of isolated single-step transitions that may result without being an important component of the strategy. This confound has been noted by proponents of singlestep transition measures for short search sequences (Bockenholt & Hynan, 1994). As multiple-step transitions are made up of single-step transitions, the more complex multiple-step transitions were identified last to override previously identified single-step transitions. It is also possible for a single cell access to belong to two different transitions, and in this case, the cell access was coded as a member of the more complex multiplestep transition, or the starting member of the second transition.

200

CHRISTOPHER BALL

Figure 1 presents multiple-step transition profiles for the seven strategy examples. The profiles clearly distinguish the different strategies, although they cannot separate strategies that differ primarily on how the information is combined (e.g., WAD vs EQU) or selected (e.g., EBA vs LEX). A closer examination of the attribute values where transitions occur can often enable the researcher to make a finer discrimination of these strategies. The multiple-step transition analysis attempts to overcome the limitations of single-step transition measures, but may not be completely resistant to incorrect measurement. For example, the SAT strategy can produce a multiple-step transition profile indicative of an additive strategy if the decision maker searches the same attributes in the same sequence for repeated alternatives. Rather than recording this search as a sequence of Type II transitions, the search would be recorded as a more complex multiattribute comparison. However, users of the SAT strategy are more likely to produce incomplete searches of options and hence will not always provide the same profile as results for additive strategies. Presenting the subject with a series of decision problems involving matrices of the same dimensions and calculating the average percentages of each transition, enables a more reliable strategy profile to be established. The next aim of the present paper is to compare the findings from a multiple-step transition analysis with those obtained for the two single-step transition indices. COMPARING SINGLE-STEP AND MULTIPLE-STEP TRANSITION MEASURES

One of the most frequently reported experimental manipulation of strategy behavior involves presenting the decision maker with information matrices of varying complexity or size (Ford et al., 1989). Findings are consistently reported that highlight the preference of decision makers for non-compensatory strategies when decision complexity increases (Ford et al., 1989; Payne et al., 1993). Therefore, data from an experiment that varied task complexity should provide a good comparison of the different approaches to strategy measurement presented in this paper. Ball, Mann, and Stamm (1994) examined the adaptive strategy behaviour of adolescent decision makers as the dimensions of the decision matrix were varied in succession from three alternatives with three attributes (3 3 3), to six alternatives with three attributes (6 3 3), to three alternatives with six attributes (3 3 6), and finally to six alternatives with six attributes (6 3 6). Three decision problems were provided to the decision makers for each level of the 2 (alternative size) 3 2 (attribute size) experimental

design. The problems were adapted from a study by Klaymann (1987) and involved selecting a bike, a lunch meal, and a holiday camp. Behavior process records were collected for the 12 decision problems from 194 adolescents aged between 12 and 16 years, making it one of the largest data bases of behaviour process data reported in the literature. The mean SI and SM values were calculated for each level of matrix complexity. The correlation between the SI and SM indexes was over 0.9 for each level of matrix complexity. However, different results were obtained for each measure when comparing the effects of the different matrix sizes with a two-way repeated-measures ANOVA. A significant main effect was obtained for the SM measure when varying the number of alternatives, F(1, 188) 5 71.97, p , .001, but not for the SI measure, F(1, 188) 5 1.13, p . .05. Conversely, a significant main effect was not found for the SM measure when varying the number of attributes, F(1, 188) 5 0.01, p . .05, but was found for the SI measure, F(1, 188) 5 23.65, p , .001. While both measures reported significant interactions, F(1, 188) 5 15.16, p , .001 for the SM measure, and F(1, 188) 5 4.01, p , .05 for the SI measure, the resulting interactions provided very different interpretations. The results of the SI interaction suggest non-compensatory strategies (i.e., negative SI) are preferred for decisions involving the least number of attributes (M333 5 2.032; M633 5 2.081), and compensatory strategies (i.e. positive SI) for decisions involving the most number of attributes (M336 5 0.061; M636 5 0.066). Conversely, the results of the SM interaction suggest non-compensatory strategies (i.e., negative SM) are preferred for the problems involving the least number of alternatives (M333 5 2.055; M336 5 2.459), and compensatory strategies (i.e., positive SM) preferred for decision problems involving the most number of alternatives (M633 5 0.757; M636 5 1.151), with this difference in preference being largest when more attributes are involved. The differences in findings for these two single-step transition measures are worrying and surprising (given the high correlation between the measures at each level of decision complexity). A multiplestep transition analysis of the same data set may help explain why such different ANOVA results have been found for the two single-step transition indices. The goal of the multiple-step transition analysis was to identify qualitative differences in strategy preferences rather than differences on a quantitative measure of compensatoriness. Three types of transitions were found to be consistent across problems for each level of decision complexity (i.e., high correlations between each value for the three decision problems) and to be reported by at least 20% of the subjects. These were the Type II transition, the Type III transition, and the

201

ANALYZING MULTIATTRIBUTE DECISION STRATEGIES

most complex multiattribute comparison (i.e., Type VII for three attribute matrices and Type X for six attribute matrices). A two-stage cluster analysis of the multiple-step transition profiles provided by these three transition types was used to identify strategy preferences. The first stage involved a hierarchical cluster analysis to identify clusters of subjects who used similar searches. A similar approach was attempted unsuccessfully by Onken, Hastie, and Revelle (1985) to cluster subjects on the basis of their decision times to various decision matrix sizes. However, the multiple-step transition profiles (averaged over the three decision problems) that result for each subject should provide a more accurate measure of strategy use. There are a number of procedures for determining the similarity of profiles, and a number of agglomeration methods for aggregating subjects with similar profiles. A well-proven approach is to cluster subjects separated by the shortest squared Euclidean distance, and then use the average profile of these subjects for future iterations of the cluster analysis (Hair, Anderson, Tatham, & Black, 1995). The hierarchical cluster analysis continues until all subjects are grouped into one cluster. A plot or dendogram of the agglomeration coefficient (smallest proximity measure at each iteration) for all the iterations of the cluster analysis should indicate the optimal number of clusters that explain the major changes in the magnitude of the agglomeration coefficient. This solution should then be examined for the “meaningfulness” of the clusters. In other words, the clusters should provide important distinctions between qualitatively different strategies, and not simply, quantitative fluctuations in profiles defined by previous cluster iterations. Various proximity measures and agglomeration techniques were tested by the author, and although no change resulted to the clusters identified, some differences in cluster membership did result. For this reason, a non-hierarchical cluster analysis is recommended to check the results of the hierarchical cluster analysis. Non-hierarchical clustering procedures require the initial entry of cluster centroids, that are subsequently used to cluster subjects who fall within a threshold distance to these centroids. Entry of the centroids obtained from the hierarchical cluster analysis provides a check of the membership of individuals to each cluster. The non-hierarchical cluster analysis found the same cluster membership and cluster centroids, regardless of the options chosen for the initial hierarchical cluster analysis. However, some changes to the final cluster solution provided by the original hierarchical cluster analysis do result from the non-hierarchical cluster analysis. This is viewed as further validation of the clustering process.

For the aims of this paper, the four-cluster solutions for each condition of decision complexity will be used to illustrate changes in strategy preferences and to highlight the relationship of multiple-step transition profiles to single-step transition indices. Table 1 provides the centroids or multiple-step transition profiles for the four-cluster solutions, along with the number of subjects who belong to each cluster. The mean SI and SM values are also provided for each group of subjects. The most popular strategy for all complexity conditions, except the most complex condition (6 3 6), was a noncompensatory strategy (A) that relied heavily on Type III transitions. Another strategy description that was obtained for all conditions of decision complexity was one that relied on a combination of Type II and Type III transitions (B). For three of the four decision complexity conditions (excluding the 6 3 6 condition), two strategies were obtained that relied on multiattribute comparisons (C and D). These two strategies were replaced in the most complex matrix condition (6 3 6) by one strategy that relied primarily on Type II transitions (F), and one that combined Type II transitions with multiattribute comparisons (E). TABLE 1 Centroids for the Four-Cluster Solutions at Each Level of Decision Complexity (Number of Alternatives 3 Number of Attributes) Clusters 333 A B C D 633 A B C D Clusters 336 A B C D 636 A B E F

% Type II

% Type III

% Type VII

125 43 19 7

16.61 41.47 21.97 9.76

75.56 48.23 33.61 12.35

151 25 8 10

18.95 44.49 19.54 8.19

n

SI

SM

0.63 5.98 40.67 76.76

2.26 0.33 0.44 0.79

21.07 1.57 1.88 2.96

69.70 45.10 23.38 4.17

5.21 6.72 53.26 85.42

2.26 0.42 0.62 0.92

2.32 3.89 4.98 6.17

% Type II

% Type III

% Type X

105 67 8 12

17.56 55.09 24.31 27.5

73.57 33.94 41.75 5.4

48 124 9 8

11.02 43.01 27.78 85.53

79.86 45.68 6.94 6.31

n

SI

SM

0.22 4.62 26.08 70.5

2.30 0.47 0.23 0.91

22.79 2.22 0.29 4.94

0.19 4.28 58.37 1.61

2.53 0.20 0.87 0.93

24.01 2.36 8.90 4.72

Note. The centroids consist of the average percentages of three transitions for each group of subjects who were selected to belong to that cluster. The means SI and SM values for each cluster of subjects are also provided.

202

CHRISTOPHER BALL

The multiple-step transition analysis highlights the adolescent decision makers’ preference for non-compensatory strategies (A and B), and that as the decision task changed, so too did the preferences for these strategies. This is in contrast to the opposite change in preference found when the number of attributes was kept small but the number of alternatives was increased (6 3 3). Compensatory strategies were only used by a minority of participants, and especially so for the decision task at its most complex (6 3 6). These results are consistent with previous research that has reported a preference for non-compensatory strategies across a wide range of decision problems (Ford et al., 1989). As can be seen in Table 1, the SI and SM measures are consistent for the most part in distinguishing noncompensatory from compensatory strategies. However, they were not able to distinguish compensatory sequences from sequences resulting from simply searching an alternative independently from others (i.e., Type VII or Type X transitions vs Type II transitions). In addition, while their exists a clear relationship between the two measures, they differ dramatically in the magnitude of the mean values obtained for some clusters of individuals. This discrepancy becomes critical in analysis-of-variance calculations and may explain the differences in findings obtained with these measures. The next section will provide further examples of multiple-step transition analyses for different types of multiattribute decisions. The first example will highlight the importance of distinguishing various multiattribute comparisons (e.g., Type VI and above) and the second example will illustrate the importance of measuring Type V transitions for multiattribute binary choices.

MULTIPLE-STEP TRANSITION ANALYSIS OF BINARY DECISIONS

MULTIPLE-STEP TRANSITION ANALYSIS OF GAMBLES

The presentation of gambles to participants in experiments allows researchers to compare the performance of various strategies with solutions calculated from normative equations (Payne & Braunstein, 1978). For example, Mann and Ball (1994) presented participants with gambles that had three outcomes: they could lose money, win money, or remain on the same amount. The participants had to choose from four gambles that differed in the probabilities of winning, losing or remaining on the same amount. The four gambles also varied in the amount that could be won or lost. Expected value theory provides an optimal choice for maximizing your gains when this task is performed a number of times, and this equation is provided in Eq. (3). Expected value 5 P*W$win 2 P*L $loss

PW and PL refer to the probabilities of winning and losing. $win and $loss refer to the amounts that could be won or lost. Each participant was presented with a sequence of 25 decision problems, and the resulting search patterns were averaged across the 25 trials. The data from 88 participants who have performed this experiment will now be reported. The multiplestep transition analysis of these data resulted in the identification of a number of strategies. The most popular strategy used by participants (n 5 48) involved a combination of Type II transitions (M 5 34.7%) and Type III transitions (M 5 47%). However, two other strategies involved more complex multiattribute comparisons. Thirteen participants relied primarily on Type VIII transitions (i.e., comparing alternatives on the same four attributes in sequence), although only one of these participants actually examined attributes in a sequence that would be best suited for performing expected value calculations (i.e., PW → $win → PL → $loss). The other twelve participants examined the same attributes but in a sequence that optimized the spatial display of the attributes on the computer screen (i.e., PW → PL → $win → $loss). A second group of 18 participants were identified as relying primarily on Type VI comparisons (i.e., comparing alternatives on the same two attributes in sequence). These participants compared alternatives on various ratios, such as, the probability of winning compared to the probability of losing, or the amount that could be won compared to the amount that could be lost. The identification of such strategies highlights the utility of the multiplestep transition method in distinguishing various multiattribute comparisons.

(3)

A number of researchers have examined the strategies used by decision makers when only choosing between two alternatives (Bockenholt et al., 1991; Russo & Dosher, 1983; Schmalhofer, Albert, Aschenbrenner, & Getzen, 1986). Schmalhofer et al. (1986) explained the strategy behavior for such decisions in terms of a criterion-dependent choice (CDC) model. This model is depicted in Eq. 4. |((v(xi) 2 v(yi))| $ k

(4)

v(xi) and v(yi) depict the subjective evaluations of alternatives x and y on attribute i. The differences between the subjective evaluations are summed for selected attributes until the absolute sum is equal to or exceeds a critical value k that is determined by the decision

ANALYZING MULTIATTRIBUTE DECISION STRATEGIES

maker. This model would assume a search pattern consisting primarily of Type V transitions (refer to Fig. 2b). However, Bockenholt et al. (1991) suggest that the subjective value differences could be processed as binary outcomes (i.e., v(xi) . v(yi) or v(xi) , v(yi)), and consequently, the search pattern could resemble a a Type V transition except the order of alternatives searched is reversed for successive attributes (see also Russo & Dosher, 1983). This multiple-step transition will be referred to as a Type Vb transition to distinguish it from a Type V transition. The CDC model does not support the use of strategies that involve multiattribute comparisons. The decision strategies used by decision makers when making multiattribute binary choices were analyzed for an experiment where participants were required to choose between two job candidates (Foddy & Holmes, 1994). The job candidates could be compared on nine attributes (e.g., age, gender, experience). The multiplestep transition analysis of data collected from 33 participants revealed six different strategies. The most popular strategy (n 5 15) described a search pattern consistent with the CDC model and involved a majority of binary decisions for repeated attribute comparisons (i.e., Type Vb transitions M 5 75%). The second most popular strategy (n 5 9) involved a combination of Type V (M 5 51%) and Type Vb (M 5 41%) transitions. Another group of participants (n 5 4) used a strategy consisting primarily of Type V transitions (M 5 78%). In all, 28 of the 33 participants used strategies consistent with the expectations of the CDC model. Only one participant used a strategy involving multiattribute comparisons, and hence the data from this experiment supports the predictions of the CDC model. The multiple-step transition analysis appears to also have valuable utility for the special case of multiattribute binary decisions. CONCLUSION

In summary, the findings of this paper raise doubts regarding the adequacy of single-step transition measures currently being used by researchers to examine adaptive strategy behavior. The two single-step transition indices were found to be highly correlated, but to provide very different results with factorial analysis-ofvariance when examining the effects of decision matrix complexity on strategy preference. These differences may reflect changes in the statistical distributions of the indexes with corresponding changes to the dimensions of the information matrix. The introduction of a multiple-step transition analysis overcomes the limitations of single-step indexes by introducing a more complex and complete range of transitions. This allowed

203

more specific, qualitative strategy preferences to be identified, and consequently, highlighted the problems with single-step transition indices when measuring specific strategies (e.g., satisficing strategy). This paper provides a set of multiple-step transition types that were identified after reviewing the decision strategy literature and isolating the transitions that distinguished the most commonly reported strategies. An alternative approach is to examine all possible transitions of varying sizes and identify the transitions that appear most frequently for a large range of decision problems. The transitions identified by this approach should provide converging evidence for the multiplestep transitions described in the current paper. Further testing of multiple-step transition analyses should incorporate experimental conditions that encourage an even greater range of strategies to be used by decision makers and hence establish whether other strategies can be classified by cluster analysis of the resulting multiple-step transition profiles. Some degree of caution should be expressed when interpreting the results of such cluster analyses as the final selection of clusters can not be supported statistically (Hair et al., 1995). Future research which includes decision measures that are deemed to vary with strategy use could help overcome this limitation by providing objective validation of final cluster selections. For example, decision outcomes are predicted to vary with strategy use under various experimental conditions (Payne et al., 1993), and therefore these measures could be examined for a range of cluster solutions. Ongoing statistical developments in multiway frequency analysis (e.g. analysis of repeated measures designs) and cluster analysis (e.g., statistical measures for determining the final number of clusters) will increase the power of the multiple-step transition approach to test for qualitative changes in strategy preference. Bockenholt and Hynan (1994) have tested an alternative multivariate approach that makes use of latent-class models. Further research could compare the results of both methods of strategy analysis for the same set of data to determine the validity of multivariate approaches for strategy measurement. REFERENCES Ball, C., & Mann, L. (1992). DEAN: Decision analyser. Unpublished manuscript, Psychology Department, University of Melbourne, Australia. Ball, C., Mann, L., & Stamm, C. (1994). Decision-making abilities of intellectually gifted and non-gifted children. Australian Journal of Psychology, 46(1), 13–20. Bockenholt, U., Albert, D., Aschenbrenner, M., & Schmalhofer, F. (1991). The effects of attractiveness, dominance, and attribute differences on information acquisition in multiattribute binary choice.

204

CHRISTOPHER BALL

Organizational Behavior and Human Decision Processes, 49, 258– 281. Bockenholt, U., & Hynan, L. S. (1994). Caveats on a process-tracing measure and a remedy. Journal of Behavioral Decision Making, 7(7), 103–117. Foddy, M. & Holmes, F. (1994). Multiple methods for studying multiple standards. Paper presented at the meeting of the International Society for Political Psychology, Santiago de Compostela, Spain. Ford, J. K., Schmitt, N., Schechtman, S. L., Hults, B. M., & Doherty, M. (1989). Process tracing methods: Contributions, problems, and neglected research questions. Organizational Behavior and Human Decision Processes, 43, 75–117. Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis with readings. New York: Prentice-Hall. Johnson, E. J., Payne, J. W., Schkade, D. A., & Bettman, J. R. (1991). Monitoring information processing and decisions: The mouselab system. Unpublished manuscript, Center for Decision Studies, Fuqua School of Business, Duke University. Klaymann, J. (1985). Children’s decision strategies and their adaptation to task characteristics. Organizational Behavior and Human Decision Processes, 35, 179–201. Received: May 20, 1996

Mann, L., & Ball, C. (1994). The relationship between search strategy and risky choice. Australian Journal of Psychology, 46(3), 131–136. Onken, J., Hastie, R., & Revelle, W. (1985). Individual differences in the use of simplification strategies in a complex decision-making task. Journal of Experimental Psychology: Human Perception and Performance, 11(1), 14–27. Payne, J. W. (1976). Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance, 16, 366–387. Payne, J. W., Bettman, J. R., & Johnson, E. L. (1993). The adaptive decision maker. Cambridge: Cambridge Univ. Press. Payne, J. W., & Braunstein, M. L. (1978). Risky choice: An examination of information acquisition behavior. Memory & Cognition, 6, 554–561. Russo, J. E., & Dosher, B. A. (1983). Strategies for multiattribute binary choice. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9(4), 676–696. Russo, J. E., & Rosen, L. D. (1975). An eye fixation analysis of multialternative choice. Memory & Cognition, 3(3), 267–276. Schmalhofer, F., Albert, D., Aschenbrenner, K. M., & Gertzen, H. (1986). Process trace of binary choices: Evidence for selective and adaptive decision heuristics. Quarterly Journal of Experimental Psychology, 38A, 59–76.