Contextual Influences on Omission Neglect in the Fault Tree Paradigm

JOURNAL OF CONSUMER PSYCHOLOGY, 15(2), 117-126 Copyright O 2005, Lawrence Erlbaum Associates, Inc.

Contextual Influences on Omission Neglect in the Fault Tree Paradigm David H. Silvera The University of Tromsl, Norway

Frank R. Kardes University of Cincinnati

Nigel Harvey University College London

Maria L. Cronley Miami University

David C. Houghton Northwest Nazarene College

Fault trees are used to organize potential causes of a problem to facilitate better judgments about potential problem solutions. However, fault trees can lead to biased judgments because decision makers tend to overestimate the likelihood of problem causes that are explicitly mentioned in the fault tree and underestimate the likelihood of problem causes that are not. In this research, we examined the impact of context information and need for cognitive closure on these estimates. In 2 experiments, participants with a low need for cognitive closure used the informational content of experimenter provided and self-generated context information as a basis for making likelihood estimates. In contrast, participants with a high need for closure did not use experimenter provided context information at all but used the ease of producing self-generated context information (rather than informational content) as a basis for their likelihood estimates. Fault trees are decision aids consisting of schematic, hierarchical representations of possible determinants of an undesired event or outcome (Fischhoff, Slovic, & Lichtenstein, 1978). They systematically organize the potential causes of a problem into a branching structure to facilitate better and more reliable judgments about potential problem solutions. Figure 1 (adapted from Fischhoff et al., 1978) shows an example of a fault tree for the problem "a car fails to start" including three levels of analysis: the problem itself, categories of potential problem causes, and specific failures belonging to each category. Fault trees were developed in the aerospace industry in the 1960s (Barlow & Lambert, 1975; Vesely, Requests for reprints should be sent to David H. Silvera, Institute of Psychology, The University of Troms0, 9037 Tromso, Norway. E-mail: [email protected]

Goldberg, Roberts, & Haash, 1981) and have been used to analyze a wide variety of problems including risk assessment in chemical process industries (Khan & Abbasi, 2000), control of manufacturing systems (Hu, Starr, & Leung, 2003), electric substation performance during earthquakes (Hwang & Chou, 1998), robotic design for hazardous waste retrieval (Walker & Cavallaro, 1996), and computer security systems (Brooke & Paige, 2003). Although fault trees are designed to facilitate systematic and accurate problem analysis, decision makers frequently overestimate the likelihood of explicitly mentioned problems (i.e., labeled problem categories in the fault tree) and underestimate the likelihood of unmentioned problems (i.e., problems included implicitly in the all other problems branch of the fault tree; Dube-Rioux & Russo, 1988; Fischhoff et al., 1978; Hirt & Castellan, 1988; Johnson, Rennie, & Wells,

118

SILVERA, KARDES, HARVEY, CRONLEY, HOUGHTON

I

insufficient

-

-

I. Faulty ground connection 2. Terminals loose or corroded 3. Battery weak

defective

1. Switches defective 2. Transmission not in park or neutral 3. Faulty starter motor 4. Starter drive defective

defective

I . Insufficient (flooding) 3. Defective choke 4. Defective air filter

Car won't start

I

defective

vandalism

1. Coil faulty

1. Oil too thick

2. Dishibutor

2. Pistons frozen

3. Spark plugs defective 4. Defective wiring between components

compression

---

1 . Paint

--

-

2. Corrosion 3. Ditt 4. Loose connections

I . Lights left on, motor off 2. Age 3. Defective generator

I . Wrong type 2. Weather too cold

breakage of vital part (ex., battery) 2. Siphoning of

3. Dismption of wiring

1 . Car out of

4 1. Ignition switch 2. Starter relay 3. Neutral stan switch 4. Solenoid

2. Clogged fuel line 3. Leaks in fuel line 4. Din in fuel

5. Fuel line frozen 6. Improperly sealed valves

--

1 . Broken ring 2. Excess heat 3. Ring g m v e damaged. fouled, or loose

I . Leaking head

- gasket 2. Cracked cylinder head

FIGURE 1 Fault tree for determining why a car fails to start. From "Fault Trees: Sensitivity of Estimated Failure Probabilities to Problem Representation," by B. Fischhoff, P. Slovic, and S. Lichtenstein, 1978, Jourrzal ofEnperimental Psychology: Human Perception and Peiformance, 3, p. 331. Copyright 1978 by the American Psychological Association. Adapted with permission.

1991; Russo & Kolzow, 1994). In a study by Fischhoff et al. (1978), for example, some participants estimated the probabilities associated with each branch of either a full fault tree consisting of six labeled branches and a seventh catchall branch (all other problems). Others made estimates pertaining to a "pruned" fault tree consisting of only three of the labeled branches from the full tree along with the all other problems branch. Normatively, the probabilities associated with the three labeled branches that were common to both fault trees should be the same, and the probabilities associated with the three additional labeled branches in the full tree should be transferred to the all other problems branch of the pruned tree. In fact, however, participants transferred less than 25% of the probability associated with the eliminated branches to the all other problems branch of the pruned fault tree, thus underestimating the probability of the all other problems branch and overestimating the probability for the labeled branches. Although previous research has attempted to explain this phenomenon, referred to as pruning bias (Russo & Kolzow, 1994), none of these explanations are fully consistent with observed data. In this article, we first review existing explanations of pruning bias and describe how each of these explanations is contradicted by research findings related to fault tree performance. Next, we present a new explanation based

on insensitivity to omitted information (omission neglect; e.g., Kardes & Sanbonmatsu, 1993) along with a rationale for how omission neglect could lead to pruning bias. Next, we review existing research related to omission neglect to generate hypotheses concerning contextual and individual difference factors that potentially influence pruning bias. Finally, we report the testing of these hypotheses in two studies.

PREVIOUS EXPLANATIONS OF PRUNING BIAS Earlier researchers have focused on three explanations of pruning bias. First, people might assume that the labeled branches of the fault tree are comprehensive and thus that the all other problems category only includes a small number of relatively unlikely potential problem causes. If this is so, however, the likelihood of all other problems should be low regardless of the number of labeled branches. Furthermore, individuals should be particularly likely to underestimate the likelihood of all other problems in pruned fault trees in which important problem categories are included among these problems (Dube-Rioux & Russo, 1988). To the extent that this explanation is correct, pruning bias should be reduced or eliminated when people doubt the

OMISSION NEGLECT IN THE FAULT TREES

completeness of the fault tree. However, this appears not to be the case, as Russo and Kolzow (1994) found no relation between perceived fault tree completeness and likelihood estimates of all other problems. Moreover, Fischhoff et al. (1978) found pruning bias even when participants were told that the labeled branches of the fault tree represented an incomplete set of problem causes. A second explanation of pruning bias is based on the premise that fault tree categories are ambiguous (Hirt & Castellan, 1988). This explanation suggests that people define the content of labeled categories differently in the pruned tree than in the full tree, with the result that examples associated with pruned categories are sometimes reassigned to another labeled category of the pruned fault tree instead of the all other problems category. To the extent that increased accessibility of examples results in increased category likelihood estimates, category ambiguity could thus result in underestimates of the likelihood of problems associated with the all other problems category. In fact, however, ambiguity only impacts likelihood estimates when the fault tree is poorly designed (i.e., problem categories are inherently ambiguous or overlapping); see Russo and Kolzow (1994). Thus, category ambiguity should not be regarded as a general explanation of pruning bias. The third explanation of pruning bias proposes that the names of labeled branches act as retrieval cues for specific failures that are associated with these branches but that the all other problems label is too general to serve in this capacity (Fischhoff et al., 1978). Consequently, the increased accessibility of specific failures associated with labeled branches results in relatively higher probability estimates for the labeled branches than for the all other problems branch. This explanation has two implications. First, fault tree branches with more detailed descriptions should facilitate retrieval of specific failures and should thus be viewed as more likely than fault tree branches that are described in less detail. Second, detailed (complete) fault trees should increase accessibility of specific failures and therefore increase estimates of the overall likelihood of a failure relative to less detailed (pruned) trees (Fischhoff et al., 1978). However, both of these supplemental predictions were tested by Fischhoff et al., and neither was supported.

OMISSION NEGLECT We propose a fourth explanation in this article. This explanation is based on the possibility that pruning bias, or insensitivity to unmentioned possibilities implied by the catchall category of a fault tree, can be interpreted in terms of omission neglect or general insensitivity to missing information (e.g., missing alternatives, attributes, features, benefits, costs, issues, and possibilities). In multiattribute evaluation, extreme judgments about an object are only justified when information about the object's important attributes is avail-

119

able (Anderson, 1982; Kaplan, 1981; Yamagishi & Hill, 1983). Nevertheless, omission neglect often leads to extreme evaluations of objects even when very little information is presented (Kardes & Gurumurthy, 1992; Kardes & Sanbonmatsu, 1993; Sanbonmatsu, Kardes, & Hen; 1992; Sanbonmatsu, Kardes, Ho, Houghton, & Posavac, 2003; Sanbonmatsu, Kardes, Posavac, & Houghton, 1997; Sanbonmatsu, Kardes, & Sansone, 1991). In the study by Sanbonmatsu et al. (1992), for example, participants evaluated cameras described by a limited amount of positive evidence just as extremely and confidently as they evaluated cameras described by a considerably greater amount of positive evidence, even though the additional information was important (Sanbonmatsu et al., 1992). Similarly, omission neglect could result in extreme likelihood judgments for the explicitly presented categories in a fault tree regardless of how many or how few categories are presented. Note that an explanation based on omission neglect is more consistent with existing data than the other explanations of pruning bias we have reviewed. Specifically, omission neglect does not predict that either perceived completeness or level of detail moderate judgments or evaluations. Thus, neither the lack of correspondence between perceived completeness and likelihood estimates nor the lack of correspondence between level of detail and likelihood estimates creates problems for this conceptualization. The omission neglect hypothesis also implies that insensitivity to unmentioned categories and pruning bias should occur even when a fault tree contains remarkably few explicit categories and examples. In prior fault-tree research, unpruned trees contained a fairly exhaustive set of lower level problem descriptions relevant to each problem category (see Figure I), and even pruned trees were relatively sophisticated and potentially useful. In this research, the target fault trees were highly impoverished to determine whether pruning bias occurs with very incomplete and therefore ineffective fault trees. Examining omission neglect in the context of pruning bias also represents a novel application of omission neglect because behavioral decision research has suggested that likelihood judgments and evaluative judgments are influenced by different sets of variables and processes. For example, likelihood judgments differ as a function of whether information is presented in terms of frequencies or probabilities (Gigerenzer, Todd, & ABC Research Group, 1999). However, it seems unlikely that this variable would have a profound influence on multiattribute evaluations. Finally, omission neglect research has examined several moderator variables that are potentially relevant to the fault tree paradigm. Previous fault tree research has focused almost exclusively on expertise as a moderator variable, with some studies showing no effect of expertise on likelihood judgments (Fischhoff et al., 1978) and others showing significant effects of expertise (Johnson et al., 1991; Ofir, 2000). Research on omission neglect has shown that sensi-

120

SILVERA, KARDES, HARVEY, CRONLEY, HOUGHTON

tivity to missing information is moderated by not only expertise (Sanbonmatsu et al., 1992,2003, 1991) but also several other variables including information processing goals (Sanbonmatsu et al., 1991), the timing of judgment (Sanbonmatsu et al., 1991), specific identification of omitted attributes (Sanbonmatsu et al., 1992), and comparative (vs. singular) judgment contexts (Sanbonmatsu, Kardes, et al., 1997; Sanbonmatsu et al., 2003).

able to participants and context information that is generated by the participants themselves. Sanbonmatsu, Kardes, et al. (1997) demonstrated that the content of accessible information from either a related or an unrelated context object could call attention to the small amount of information about a target object, resulting in judgmental moderation. In this research, we replicated the positive impact of accessible content on judgment quality and also demonstrated the negative impact of subjective ease of recall on judgment quality.

CONTEXT AND THE ROLE OF ACCESSIBLE CONTENT VERSUS ACCESSIBILITY EXPERIENCES

NEED FOR COGNITIVE CLOSURE AND INTERRUPTION

The impact of context information on judgments has been a particularly important theme in recent research. Schwarz's (1998; see also Wyer, 2004) analysis suggests that there are two mechanisms by which context influences judgments and that these two mechanisms work in opposite directions. The first of these mechanisms is based on the content of accessible information (e.g., Wyer & Srull, 1989)-as the number of accessible examples of a type of event increases, the more likely that event is perceived to be. For example, frequent (vs. infrequent) television viewers tend to rate events that are frequently depicted on television (e.g., violent crimes, affluent lifestyles) as more likely (O'Guinn & Shrum, 1997). The second mechanism is based on the ease or difficulty with which information is retrieved (e.g., Koriat, 1993)-the more difficult it is to generate examples of an occurrence, the less likely that occurrence is perceived to be. Thus, for example, people inferred that their memory of their childhood was worse after recalling 12 events from their childhood than after recalling only 4 (Winkielman, Schwarz, & Belli, 1998). Twelve events were apparently more difficult to recall than 4 events, thus producing a subjective experience of difficulty of recall that led these participants to infer that their memory was poor. Although several moderator variables are potentially relevant to the fault tree paradigm, in this research we focused on the extent to which comparativejudgment contexts moderate fault tree likelihood estimates to extend existing knowledge of both pruning bias and omission neglect. With regard to pruning bias, Ofir (2000) recently examined the effects of context on fault tree judgments by asking participants to generate few or many examples from the all other problems branch of the tree. Ofir found that nonexperts' likelihood judgments for the all other problems branch were related to the subjective ease of recalling examples. That is, participants rated the all other problems branch as more likely when the example generation task was perceived as easy (few examples were generated) than when it was difficult (many examples were generated). However, the effect of a difficult generation task may occur only when people are instructed to perform this task. In this research, we examined the differential effects of context information that is simply made avail-

The final goal of this research was to examine needfor cognitive closure as an individual difference variable that potentially influences likelihood estimates in the fault tree paradigm. The need for cognitive closure refers to a preference for any definite opinion over confusion or ambiguity (Kruglanski & Webster, 1996). As the need for cognitive closure increases, people consider fewer alternatives; consider smaller amounts of information for each alternative; draw snap conclusions that have obvious and immediate implications for action; and neglect complex, inconsistent, or otherwise difficult-to-use information. The need for cognitive closure can be either measured as an individual difference variable (Webster & Kruglanski, 1994) or manipulated as a situational variable. Any variable that makes information processing difficult or unpleasant-such as time pressure, ambient noise, fatigue, or tedium (Kruglanski & Webster, 1996)-heightens the need for cognitive closure. Need for cognitive closure was selected as a potential moderating variable for several reasons. First, need for cognitive closure was shown to increase preferences for easily used information and to decrease the tendency to generate additional information associated with a judgment task (Hirt, Kardes, & Markman, 2004). This suggests that need for cognitive closure might also be related to preference for presented versus missing branches of a fault tree. Second, Ofir (2000) recently found that pruning bias is significantly reduced in individuals with expertise in the content domain of a fault tree. Expertise is generally associated with ability or knowledge. However, because people are typically more interested in information related to their areas of expertise, expertise can also be used as a motivational variable (see Kunda, 1990, and Showers & Cantor, 1985, for discussions of this possibility). As such, it is impossible to determine whether the influence of expertise on fault tree performance in Ofir's research is a result of differences in ability, motivation, or both. By using need for cognitive closure as a moderator variable, we could isolate the contribution of motivational factors to fault tree performance. Third, omission neglect research using the multiattribute paradigm showed that individuals are more likely to make extreme and confident judgments based on insufficient infor-

OMISSION NEGLECT IN THE FAULT TREES

rnation when they are rushed rather than when they are given ample time to make these judgments (Sanbonmatsu et al., 1991). If the need for cognitive closure moderates omission as this research suggests, it might also moderate fault tree performance. Although in the second experiment we report, we used a well-established individual difference measure of need for cognitive closure, in the first experiment, we used task interruption as a novel manipulation of need for cognitive closure. Task interruption increases the accessibility of an information processing goal (Martin, 1986; Zeigarnik, 1938), and this should increase the desire to attain closure by completing the interrupted task. Furthermore, Martin (1986) found that interrupting individuals during a priming task increased their tendency to assimilate their judgments of a target person to the primed concepts. Task interruption might also encourage the use of quick and easy information-processing strategies in other judgment contexts as well. In other words, it should function like a high need for cognitive closure by encouraging the use of simplifying heuristics to increase the speed with which closure can be attained.

EXPERIMENT 1 Participants were asked to make likelihood estimates for each branch of a target fault tree after being exposed either to a large-context fault tree with many categories of possible reasons for a failure or to a small-context fault tree with the same number of categories as the target fault tree. In addition, need for cognitive closure was manipulated by interrupting half of the participants while they were making likelihood judgments for the branches of the target fault tree. Previous research suggested that merely exposing individuals to relevant alternatives is sufficient to debias likelihood judgments for low need for cognitive closure individuals but not for high need for cognitive closure individuals (Hirt et al., 2004). Consequently, it was predicted that high need for closure individuals would not use information associated with the context fault tree when making likelihood estimates for the target fault tree and that the size of the context tree would have no influence on their estimates. In contrast, participants with a low need for closure were expected to use the size of the context fault tree as a standard against which the target fault tree was evaluated. Because the target fault tree was the same size as the small-context fault tree, it should be viewed as conforming to the standard provided by the context. Therefore, participants with a low need for closure should not be motivated to engage in additional mental processing for the target fault tree and should make the same likelihood estimates as participants with a high need for closure. In contrast, participants with a low need for closure who were exposed to large-context fault trees should see the target fault tree as inadequate in comparison to the

121

standard established by the context tree. Therefore, these participants, relative to those with a high need for closure or participants who are exposed to small-context trees (regardless of their need for closure), should (a) generate more examples associated with the all other problems branch of the target fault tree, (b) make higher subjective likelihood estimates for the all other problems branch of the target fault tree, and (c) be willing to spend more money on market research related to the all other problems category.

Method

Participants. Participants were 90 psychology students at Northern Kentucky University who participated for course credit. Materials. After receiving a brief explanation of fault trees and the probabilities associated with fault tree branches, we presented participants with a context fault tree that listed either 11 (large-context tree) or 5 (small-context tree) explanation categories for why an automobile might fail to start (from Fischhoff et al., 1978) and we asked them to estimate the frequency of problems associated with each branch of the tree out of 100 failures. Participants were then presented with a target fault tree containing reasons why a new product might sell poorly. This target fault tree contained five branches: "the product was designed poorly," "poor advertising," "the price was too high," "the target market was too small," and all other problems. For each branch of the target fault tree, participants were asked to estimate the number of times out of 100 that a new product selling poorly would be due to the cause associated with that branch. After making these frequency estimates, participants completed an open-ended measure in which they listed specific, lower level explanations they had considered when making likelihood estimates for the all other problems category of the fault tree. Finally, participants were asked to imagine that they were consultants and to estimate how much they would advise the firm to spend on market research for the all other problems category on a scale ranging from 0 ($0) to 10 ($100,000). Procedure. Crossed with the manipulation of the size of the context tree, need for cognitive closure was manipulated by either interrupting or not interrupting participants while they made their likelihood judgments for the target fault tree. Specifically, in the high need for closure condition, participants were stopped just before completing their likelihood estimates for the third branch of the target fault tree and asked to complete a one-page questionnaire before continuing. In the low need for closure condition, this interruption did not occur. Participants were run individually to permit careful timing of the interruption manipulation. The

122

SILVERA, KARDES, HARVEY, CRONLEY, HOUCHTON

experiment thus involved a 2 (size of context fault tree: large or small) x 2 (need for cognitive closure: high or low) design.

Results It was hypothesized that low need for closure and largecontext tree participants, compared to participants in all other conditions, would make higher subjective likelihood estimates for the all other problems branch of the target fault tree, generate more examples of all other problems, and be willing to spend more money on marketresearch related to the all other problems category. Means for each of the dependent measures are shown for each experimental condition in Table 1. Each of the dependent measures was first submitted to a 2 (need for closure: high or low) x 2 (size of context fault tree: large or small) analysis of variance. In each of these analyses, the expected interaction between need for closure and size of the context fault tree was at least marginally significant: likelihood estimates, F(1, 86) = 2.81, p < .lo; number of generated examples, F(1, 86) = 3.47, p < .07; and willingness to spend on market research, F( l,86) = 5.12, p < .03). The only significant main effect from these analyses indicated that participants with large-context fault trees generated more examples than participants with small-context fault trees, F(1, 86) = 6.13, p < .02. To test the experimental hypotheses, participants with a low need for closure and a large-context fault tree were compared to participants in the other three conditions combined for each of the dependent measures. As expected, low need for closure and large-context tree participants rated the all other problems category as more likely (M = 14.21) than other participants did (M = 11.30), t(88) = 1.68, p < .lo. Moreover, low need for closure and large-context tree participants generated significantly more examples for the all other problems category (M = 2.38) than other participants (M = TABLE 1 Likelihood Judgments, Number of Examples Generated, and Market Research Spending Decisions as a Function of Provided Context Fault Tree Set Size and Need for Closure (Experiment 1) Variable Likelihood of all other problems High need for closure Low need for closure Number of examples generated High need for closure Low need for closure Market research spending High need for closure Low need for closure

Small-Context Fault Tree

Large-Context Faalt Tree

12.39a,b 1I . 14a,b 1.43, 1.41, 4.13a.b 3.45,

Note. For each dependent variable, means with different subscripts differ at p < .05 (one-tailed).

1.47), t(88) = 3.64, p < ,001 and were willing to spend significantly more on market research related to the all other problems category (M = 4.96) than other participants (M = 3.82), t(88) = 2.58, p < .02.

Discussion In Experiment 1, we demonstrated that a context fault tree can provide a comparison standard that is used to evaluate a target fault tree and that the application of this comparison standard is moderated by the need for cognitive closure. Specifically, participants only applied the experimenter provided context information to their judgments about the target fault tree if they were low in need for cognitive closure, and the standard provided by the context information differed from the content of the target fault tree. Thus, compared to all other participants, low need for closure participants with large-context fault trees generated more examples associated with the all other problems branch of the target fault tree, made higher subjective likelihood estimates for the all other problems branch of the target fault tree, and were willing to spend more money on market research related to the all other problems category. However, what happens when participants generate the context fault tree themselves? In this case, fault tree evaluation should differ from evaluation of experimenter provided fault trees in two ways. First, the act of generating the context fault tree should make context information readily available to all participants regardless of need for cognitive closure. Second, this readily available context information should influence fault tree evaluations through different mechanisms for low versus high need for cognitive closure individuals. Individuals with a low need for closure should focus on the content of the generation task and thus produce more examples associated with the all other problems branch of a target fault tree when their generation task requires them to produce a large (rather than small) context fault tree, and this should lead them to make higher estimates of the likelihood of the all other problems branch when their generation task is large. In contrast, individuals with a high need for closure should engage in a more "mindless," content-free evaluation of the fault tree based on their phenomenological experience of the ease or difficulty of the generation task. This should cause them to make higher frequency estimates for the all other problems branch of the fault tree when the generation task is easy (i.e., when they generated a small rather than a large-context fault tree) than when it is hard. Furthermore, because the influence of context on high need for closure participants is expected to result in superficial, content-free processing of the fault tree, the size of the fault tree should not influence their content-based cognitive elaborations as reflected in the number of examples of all other problems they generate for the target fault tree.

OMISSION NEGLECT IN THE FAULT TREES

EXPERIMENT 2

123

was used on this scale to classify participants as either high or low in need for cognitive closure.

Method

Participants. Participants were 201 marketing students at The University of Cincinnati who participated for course credit.

Materials. After receiving a brief explanation of fault trees and the meanings of probabilities associated with fault tree branches, participants were asked to create their own fault trees. Participants were asked to generate context fault trees based on one of two types of failure: In the new product .condition, they gave reasons why a new product might sell poorly; in the car condition, they generated reasons why a car might fail to start. In each case, the size of the participant-generated fault tree was also varied. In the small-context tree condition, participants were asked to provide two categories of potential causes for the failure; in the large-context tree condition, participants were asked to provide eight categories of potential causes. In control conditions, participants did not generate a fault tree. The manipulated variables thus provided a 2 (generation category: new product or car) x 3 (size of generated context fault tree: small, large, or control group) design. After the context fault tree generation phase of the experiment was complete, participants were presented with a target fault tree containing reasons why a new product might sell poorly. Thus, in the new product condition, the domains of the context and target fault trees were the same, whereas in the car condition, the domains were different. The target fault tree contained the same five branches that were used in Experiment 1. For each branch of the target fault tree, participants were asked to estimate the number of times out of 100 that a new product selling poorly would be due to a cause associated with that branch. Participants were reminded that their answers should add up to 100. After finishing the fault tree, participants completed an open-ended measure in which they listed specific, lower level explanations they had generated in relation to the all other problems category of the fault tree. Finally, participants were given the Need for Cognitive Closure Scale (Webster & Kruglanski, 1994); a median split

Results and Discussion

Preliminary analyses. The main analyses were first conducted with generation category as a predictor variable in addition to number of items generated and need for cognitive closure. Generation category had no significant main effects or interactions with the other independent variables, indicating that context effects were equally strong whether the context tree was in the same or a different domain. The remaining analyses were collapsed across generation category. Primary analyses. We hypothesized that participants with high need for closure would view the all other problems branch of the fault tree as more likely when the context tree was small than when it was large but that the number of generated examples would not depend on the size of the context tree. However, participants with low need for closure were expected to view the all other problems branch of the fault tree as more likely when the context tree was large than when it was small, and their likelihood estimates were expected to be mediated by the number of examples reported during the generation task. Data relevant to these hypotheses are summarized in Table 2. Estimates of the frequency of problems in the all other problems category were first analyzed as a function of need for closure and the size of the context fault tree. Although neither main effect was significant (Fs < 1), the interaction of these variables was quite reliable, F(2, 180) = 4.79, p < .01. As expected, participants with a high need for closure rated the all other problems category as significantly more likely when they generated a small rather than a large-context tree, t(74) = 2 . 5 2 , ~< .02, whereas participants with a low need for closure rated the all other problems category as more likely when they generated a large rather than a small-context tree, t(84) = 1.76, p < .09. Similar analyses were conducted for the number of generated examples from the all other problems category. Although the interaction of need for closure and context tree size was not significant, F(2, 180) = 2.15, p > . l o , planned

TABLE 2 Likelihood Judgments and Number of Examples Generated as a Function of Participant-generated Context Fault Tree Set Size and Need for Closure (Experiment 2) Variable

Small-Context Fault Tree

Large-Context Fault Tree

Likelihood of all other problems High need for closure Low need for closure Number of examples generated High need for closure Low need for closure Note.

For each dependent variable, means with different subscripts differ at p < .05 (one-tailed).

No-Context Fault Tree

comparisons performed separately for high and low need for cognitive closure participants supported our hypotheses. Specifically, the size of the context fault tree had no influence on the number of examples generated by participants with a high need for closure ( t < I), whereas participants with a low need for closure generated significantly more examples of all other problems when the context fault tree was large than when it was small, t(85) = 2.10, p < .04. Finally, analyses were conducted to evaluate the hypothesis that the number of examples generated would mediate the relation between the size of the generation task and likelihood estimates for all other problems for participants with a low need for closure. The number of reasons generated by these participants was significantly correlated with their perception of the likelihood of all other problems, r(99) = .27, p < .01. This was not the case, however, for participants with a high need for closure, r(100) = .09, p > .30. To determine whether the former participants' likelihood estimates were mediated by the number of reasons they generated, an analysis of covariance was performed on these estimates as a function of context tree size using the number of examples generated as a covariate. This analysis showed no effect for size of the context tree (F < 1) but a significant effect for the covariate (number of examples generated), F(1, 91) = 4.91, p < .03. Although the original effect of the size of the generation task on likelihood estimates was only marginally significant, these results suggest at least partial mediation by the number of examples generated, consistent with our hypotheses.

GENERAL DISCUSSION Experiment 1 demonstrated that an experimenter-provided context fault tree can provide a comparison standard for a target fault tree but only for individuals with a low need for cognitive closure. When these participants were presented with a large-context fault tree, they produced more examples associated with the all other problems branch of the target fault tree, made higher subjective likelihood estimates for the all other problems branch of the target fault tree, and indicated that they would spend more money on market research related to the all other problems category. The results of Experiment 2 suggest that self-generated context fault trees influenced judgments related to target fault trees for all individuals, but that the processes that mediated this influence depended on need for closure. Participants with a low need for closure appeared to focus on the content of the generation task, produced more examples associated with the all other problems branch, and made higher likelihood estimates for all other problems when they generated a large rather than a small-context fault tree. In contrast, participants with a high need for closure appeared to focus on their phenomenological experience of the ease or difficulty of the generation task and made higher frequency estimates for the

all other problems branch of the target fault tree when the generation task was easy (i.e., when they generated a small-context tree) rather than hard. Hence, the results of this study suggest that the need for cognitive closure is an important new moderator of the judgmental influence of accessible content (i.e., number of examples retrieved or generated) versus accessibility experiences (i.e, ease of retrieval or generation; Schwarz, 1998). The results of these experiments suggest that omission neglect is a useful theoretical framework for interpreting fault tree performance regardless of whether the context fault tree was provided by the experimenter (Experiment 1) or generated by the participant (Experiment 2) and regardless of whether the need for closure was manipulated via interruption (Experiment 1) or measured as an individual difference variable (Experiment 2). In both experiments, fault trees were found to be misleading when participants were insensitive to omissions. Omission neglect encourages decision makers to underestimate the likelihood of the all other problems catch-all category. Sensitivity to omissions can be heightened by encouraging decision makers to compare a target fault tree to a relatively large-context fault tree that highlights the limitations of the target. However, even when decision makers are sensitive to omissions, fault trees can backfire as decision aids when decision makers are situationally or dispositionally high in the need for closure. This is because such decision makers rely on the subjective ease with which specific instances of unmentioned problems are considered rather than on the judgmental implications of these instances per se. Although prior research on the use of fault trees has suggested that omission neglect might contribute to the pruning bias, this research is open to an alternative interpretation. Grice's (1975) cooperation principle of interpersonal communication suggests that recipients assume that communicators provide information that is relevant, truthful, informative, and clear unless there is some reason to distrust or doubt the communicator. If participants assume that experimenters are cooperative communicators, they should infer that the presented branches of a fault tree are particularly important causes of a target problem and that the unmentioned causes included in the all other problems branch are much less important. However, although this alternative explanation can account for any main effect of the size of context fault trees on likelihood estimates for all other problems, it has more difficulty accounting for interactions between context fault tree size and need for cognitive closure in predicting likelihood estimates. Thus, this explanation does not provide a viable account of the results of this research. Insensitivity to missing information has also been observed in other types of judgment tasks that are unlikely to involve conversational inference. Research on the feature-positive effect has shown that the relation between a predictive cue and a reward is learned more

OMISSION NEGLECT IN THE FAULT TREES

quickly when the cue involves the presence of a stimulus (e.g., a letter, number, or symbol) rather than its absence (Newman, Wolff, & Hearst, 1980). Research on the Ellsberg paradox e.g., Birnbaum, 1992) showed that people prefer gambles involving known probabilities (e.g., a 50% chance of drawing red or a 50% chance of drawing black) to gambles involving unknown probabilities (e.g., an unknown chance of drawing red or an unknown chance of drawing black) even when people are indifferent between red and black for both types of gambles (Fox & Weber, 2002). Research on selective hypothesis testing showed that when assessing the relation between two variables, people focus more heavily on cases involving the presence of both variables than on cases involving the absence of one variable and focus least heavily on cases involving the absence of both variables (Sanbonmatsu, Posavac, Kardes, & Mantel, 1998).

Support Theory Another alternative explanation for the results of this research could be based on support theory, which suggests that likelihood judgments are based on the strength of the evidence favoring a focal hypothesis or a description of an event one is trying to predict (Tversky & Koehler, 1994). According to support theory, each hypothesis A has a support value, s(A) for the strength of the evidence favoring A. The subjective probability that focal hypothesis A rather than alternative hypothesis B is true (assuming that only one is true) is provided by

This equation suggests that subjective probability depends on the perceived support for A relative to B and that explicating or unpacking A increases its salience and its perceived support. Consequently, the perceived likelihood of a plane crash is greater when this event is unpacked into several subcategories (e.g., a plane crash due to human error, a plane crash due to mechanical failure, a plane crash due to weather conditions, a plane crash due to terrorism) rather than described more generally and inclusively (e.g., a plane crash). The perceived likelihood of an event increases by unpacking a focal hypothesis and decreases by unpacking the alternative hypothesis. Hence, in the fault-tree paradigm, presented categories are overestimated and unmentioned categories are underestimated. Although the results of this research were consistent with the implications of support theory, this theory assumes that likelihood judgments are always relative and subadditive (i.e., the perceived likelihood of a general category is less than the sum of its subcategories) except when the focal hypothesis and the alternative hypothesis are complementary. Contrary to these assumptions, likelihood judgments are not always rela-

125

tive (Gibson, Sanbonmatsu, & Posavac, 1997). Moreover, these judgments are superadditive rather than subadditive when the supporting evidence is weak or the confirmation criteria are high (Sanbonmatsu, Posavac, & Stasny, 1997) and can be subadditive even when the focal hypothesis and the alternative hypothesis are complementary (McKenzie, 1999). Thus, support theory does not provide an adequate account of estimates of the sort investigated in this article (for an explanation of the aforementioned pattern of findings in terms of selective hypothesis testing, see Sanbonmatsu et al., 1998).

CONCLUSION This research adds to a growing body of research that has shown that omission neglect plays an important role for many different types of judgments across many different paradigms. In addition to influencing likelihood judgments, omission neglect influences evaluations (Sanbonmatsu et al., 1992; Sanbonmatsu, Kartes, et al., 1997), preferences (Kardes & Sanbonmatsu, 1993), confidence ratings (Sanbonmatsu et al., 1992; Sanbonmatsu, Posavac, et al., 1997), attribute importance ratings (Sanbonmatsu et al., 2003), and inferences about missing attributes (Sanbonmatsu et al., 1991). Omission neglect plays an important role in the fault tree paradigm, multiattribute evaluation (Sanbonmatsu et al., 1992; Sanbonmatsu, Kardes, et al., 1997), the pioneering brand advantage (Kardes & Gurumurthy, 1992), the feature-positive effect (Newman et al., 1980), the Ellsberg paradox (Fox & Weber, 2002), and selective hypothesis testing (Sanbonmatsu et al., 1998). In each of these cases, people focus too readily on whatever information they happen to encounter and fail to adjust their judgments sufficiently in light of the limitations of the available evidence.

REFERENCES Anderson, N. H. (1982). Method.? of information integration theory. San Diego, CA: Academic. Barlow, R. E., & Lambert, A. E. (1975). Introduction to fault tree analysis. In R. E. Barlow, J. B. Fussell, & N. D. Singpurwalla (Eds.), Reliability and fault tree analysis: Theoretical and applied aspects of system reliability and safety assessment (pp. 7-36). Philadelphia: Society for Industrial and Applied Mathematics. Birnbaum, M. H. (1992). Issues in utility measurement. Organizational Behavior and Human Decision Processes, 52, 3 19-330. Brooke, P. J., & Paige, R. F. (2003). Fault trees for security design and analysis. Computers and Security, 22, 256-264. Dube-Rioux, L., & Russo, J. E. (1988). An availability bias in professional judgment. Journal of Behavioral Decision Making, 1, 223-237. Fischhoff, B., Slovic, P., & Lichtenstein, S. (1 978). Fault trees: Sensitivity of estimated failure probabilities to problem representations. Journal of Experimental Psychology: Human Perception and Peformaace, 3, 330-344. Fox, C. R., & Weber, M. (2002). Ambiguity aversion, comparative ignorance, and decision context. Organizational Behavior and Human Decision Processes, 88, 476498.

Gibson, B., Sanbonmatsu, D. M., & Posavac, S. S. (1997). Focal bias, probability overestimation, and gambling choice: The influence of biased hypothesis testing on subsequent judgment. Journal of Experimental Psychology: Applied, 3, 1 2 6 142. Gigerenzer, G., Todd, P. M., & the ABC Research Group. (1999). Simple heuristics that make us smart. New York: Oxford University Press. Grice, H. P. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics: Speech acts (Vol. 3, pp. 95-1 13). New York: Academic. Hirt, E. R., & Castellan, N. J. (1988). Probability and category identification in the fault tree paradigm. Journal of Experimental Psychology: Human Perception and Performance, 14, 112-1 31. Hirt, E. R., Kardes, F. R., & Markman, K. D. (2004). Activating a mental simulation mind-set through generation of alternatives: Implications for debiasing in related and unrelated domains. Journal of Experimental Social Psychology, 40, 374-383. Hu, W., Stan, A. G., & Leung, A. Y. T. (2003). Operational fault diagnosis of manufacturing systems. Journal of Materials Processing Technology, 133, 108-1 17. Hwang, H. H. M., & Chou, T. (1998). Evaluation of seismic performance of an electric substation using event treelfault tree technique. Probabilistic Engineering Mechanics, 13, 11 7-124. Johnson, R. D., Rennie, R. D., & Wells, G. L. (1991). Outcome trees and baseball: A study of expertise and list-length effects. Organizational Behavior and Human Decision Processes, 50, 324-340. Kaplan, M. F. (1981). Amount of information and polarity of attraction. Bulletin of the Psychonomic Society, 18, 23-26. Kardes, F. R., & Gurumurthy, K. (1992). Order-of-entry effects on consumer memory and judgment: An information integration perspective. Journal of Marketing Research, 29, 343-357. Kardes, F. R., & Sanbonmatsu, D. M. (1993). Direction of comparison, expected feature correlation, and the set-size effect in preference judgment. Journal of Consumer Psychology, 2, 39-54. Khan, F. I., & Abbasi, S. A. (2000). Analytical simulation and PROFAT 11: A new methodology and a computer automated tool for fault tree analysis in chemical process industries. Journal of Hazardous Materials, 75, 1-27. Koriat, A. (1993). How do we know that we know? The accessibility model of the feeling of knowing. Psychological Review, 100, 609-639. Kruglanski, A. W., & Webster, D. M. (1996). Motivated closing of the mind: "Seizing" and "freezing." Psychological Review, 103, 263-283. Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108,480498, Martin, L. L. (1 986). Setlreset: Use and disuse of concepts in impression formation. Journal of Personality and Social Psychology, 51, 493-504. McKenzie, C. R. M. (1 999). (Non)complementary updating of belief in two hypotheses. Memory & Cognition, 27, 152-165. Newman, J., Wolff, J. T., & Hearst, E. (1 980). The feature-positive effect in adult human subjects. Journal of Experimental Psychology: Human Learning and Memory, 6, 630-650. Ofir, C. (2000). Ease of recall vs recalled evidence in judgment: Experts vs laymen. Organizational Behavior and Human Decision Processes, 81, 2842. O'Guinn, T. C., & Shrum, I. J. (1997). The role of television in the construction of social reality. Journal of Consumer Research, 23, 278-294. Russo, J. E., & Kolzow, K. J. (1994). Where is the fault in fault trees? Journal of Experimental Psychology: Human Perception and Perjormance, 20,263-283.

Sanbonmatsu, D. M., Kardes, F. R., & Herr, P. M. (1992). The role of prior knowledge and missing information in multiattribute evaluation. Organizational Behavior and Human Decision Processes, 51, 7 6 9 1 . Sanbonmatsu, D. M., Kardes, F. R., Ho, E. A., Houghton, D. C., & Posavac, S. S. (2003). Overestimating the importance of the given information in multiattribute consumer judgment. Journal of Consumer Psychology, 13, 289-300. Sanbonmatsu, D. M., Kardes, F. R., Posavac, S. S., & Houghton, D. C. (1997). Contextual influences on judgment based on limited information. Organizational Behavior and Human Decision Processes, 69, 251-264. Sanbonmatsu, D. M., Kardes, F. R., & Sansone, C. (1991). Remembering less and inferring more: The effects of timing of judgment on inferences about unknown attributes. Journal of Personality and Social Psychology, 61, 546554. Sanbonmatsu, D. M., Posavac, S. S., Kardes, F. R., &Mantel, S. P. (1998). Selective hypothesis testing. Psychonomic Bulletin & Review, 5, 197-220. Sanbonmatsu, D. M., Posavac, S. S., & Stasny, R. (1997). The subjective beliefs underlying probability overestimation. Journal of Experimental Social Psychology, 33, 276295. Schwarz, N. (1998). Accessible content and accessibility experiences: The interplay of declarative and experiential information in judgment. Personaliij and Social Psychology Review, 2, 87-99. Showers, C., & Cantor, N. (1985). Social cognition: A look at motivated strategies. Annual Review of Psychology, 36, 375-305. Tversky, A,, & Koehler, D. J. (1994). Support theory: A nonextensional representation of subjective probability. Psychological Review, IOI, 547-567. Vesely, W. E., Goldberg, F. F., Roberts, N. H., & Haash, D. F. (1981). Fault tree handbook. Washington, DC: U.S. Nuclear Regulatory Commission. Walker, I. D., & Cavallaro, J. R. (1996). Failure mode analysis for a hazardous waste clean-up manipulator. Reliability Engineering and System Safety, 53, 277-290. Webster, D. M., & Kruglanski, A. W. (1994). Individual differences in need for cognitive closure. Journal of Personality and Social Psychology, 67, 1049-1062. Winkielman, P., Schwarz, N., & Belli, R. F. (1998). The role of ease of retrieval and attribution in memory judgments. Judging your memory as worse despite recalling more events. Psychological Science, 9, 4 6 4 8 . Wyer, R. S. (2004). Social comprehension and judgment. The role of situation models, narratives, and implicit theories. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Wyer, R. S., & Srull, T. K. (1989). Memory and cognition in its social context. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Yamagishi, T., & Hill, T. C. (1983). Initial impression versus missing information as explanations of the set-size effect. Journal of Personality and Social Psychology, 44, 942-95 1. Zeigarnik, B. (1938). On finished and unfinished tasks. In W. D. Ellis (Ed.), A sourcebook of gestalt psychology (pp. 300-314). New York: Harcourt, Brace, & World.

Received: December 1,2003 Accepted: March 19,2004