- Email: [email protected]
Self-report measures of patient utility
Journal of Clinical Epidemiology 53 (2000) 469–476
Self-report measures of patient utility: should we trust them? Makoto Hanita* Houston Center for Quality of Care and Utilization Studies, Houston Veterans Affairs Medical Center, 2002 Holcombe Boulevard, Houston, TX 77030, USA; Baylor College of Medicine, One Baylor Plaza, Houston, TX 77030 Received 20 May 1998; received in revised form 22 September 1999; accepted 6 October 1999
Abstract As self-reports, measures of patient utility are susceptible to the effects of cognitive biases in patients. This article presents often overlooked problems in these measures by outlining cognitive processes involved in patient self-report. It is argued that these measures: 1) require overly complex mental operations; 2) fail to elicit thoughtful response by default; 3) may be biased by patients’ mood; 4) are affected by both researchers’ choice of measurement instruments and patients’ choice of judgment strategies; 5) tend to reflect the disproportionate influence of patients’ values that happen to be recallable at the time of measurement; and 6) are affected by patients’ fear of regret. It is suggested that solutions for these problems should involve: a) improving the methods of administration; b) developing measures that are less taxing to patients; and c) redefining the concept of patient utility as judged, as opposed to retrieved, evaluation. Published by 2000 Elsevier Science Inc. All rights reserved. Keywords: Patient preference; Expected utility; Measurement; Patient self-report; Medical decision making; Cognitive psychology
1. Introduction To provide care that will satisfy patients, care providers must know preferences of patients [1,2]. How much discomfort from side effects are patients willing to tolerate to stay symptom free? How many hospital visits are patients willing to make for painful procedures that may or may not work? Care providers must have at least tentative answers to these questions before treating patients. In this regard, research on patient preference is a cornerstone of patient-centered medicine. In research on patient preference, a set of possible health outcomes is defined for patients with a given medical condition. These health outcomes are then compared against one another, in terms of their potential for eliciting satisfaction in patients [3,4]. Various measures of expected patient satisfaction—or patient utility—are used for this comparison, to quantify the level of satisfaction that patients associate with each of the possible health outcomes. Although different measures of patient utility are based on different methods of quantifying the level of patient satisfaction, there is one property that is common to all of them: all measures of patient utility are based on selfreports. This is an important property that distinguished these measures from physiological or behavioral measures.
In self-reports, patients must take active roles: they must comprehend the questions and come up with the answers. Consequently, measures that are based on self-reports are susceptible to the effects of cognitive biases in patients. Often, these cognitive biases make a straightforward interpretation of the measures of patient utility problematic; however, this is one issue that researchers tend to overlook. The focus of this article is to present a review of psychological research on human cognition surrounding the issue of utility and preference. The purpose of this review is to outline various mechanisms through which cognitive processes of patients may bias measures of patient utility. Understanding such mechanisms is essential for improving available measures as well as developing better ones. Before turning to the discussion of cognitive processes, however, a brief description of measures of patient utility is provided for readers who are not familiar with the issue. Because some readers may want to skip this part, the description is intentionally kept at a cursory level. Readers who wish to study the measurement of patient utility in detail are thus encouraged to refer to the excellent reviews that are available (e.g., Froberg and Kane [5]). 2. Measures of patient utility
* Corresponding author. Tel.: (713) 794-7615 / 7909; Fax: (713) 7947103. E-mail address: [email protected] (M. Hanita)
Several options are available for researchers who are interested in measuring patient utility. Among those, the three
0895-4356/00/$ – see front matter. Published by 2000 Elsevier Science Inc. All rights reserved. PII: S0895-4356(99)00 2 0 5 - X
470
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
most commonly chosen are the standard gamble, the time trade-off, and the rating scale. The standard gamble method was originally developed by von Neumann and Morgenstern [6]. This method is often considered the only one that yields “true” values of utility, because of its strict adherence to the axioms of modern utility theory. In the standard gamble method, patient utility for a given health outcome is measured by asking a patient to consider a choice between a certainty and a gamble—a choice between the target health outcome, which will last for the rest of his or her life, versus a treatment with a known probability (p) of success. If successful (p), the treatment would bring the patient perfect health that will last for the rest of his or her life. If not successful (1 ⫺ p), it would bring the patient immediate death. For example, a patient may be asked to choose between staying on hospital dialysis of the rest of his or her life (e.g., 10 years) versus undergoing a risky treatment, which has a 30% probability of complete recovery lasting for life and a 70% probability of immediate death. The probability of recovery versus death in the gamble is then varied until the point (i.e., “point of indifference”) is reached where the patient finds it impossible to choose between the two treatments. In the standard gamble method, the value of patient utility for the target health state is the value of probability for the point of indifference. In the above example, if the patient reaches the point of indifference at a 25% probability of complete recovery (and a 75% probability of immediate death), the utility of staying on hospital dialysis for this patient is 0.25. Despite its strong theoretical grounding, the standard gamble is not necessarily a practical method for most patients. Success of this method relies heavily on the ability of patients to understand probabilistic information; however, psychological research has found that people are generally bad at this [7,8]. Not surprisingly, health services research has also documented the difficulties that patients experience with this method [9]. The time trade-off method [10] is a practical alternative to the standard gamble method because it does not require patients to understand probabilistic information. In the time trade-off method, patient utility for a given health outcome is measured by asking a patient to consider a choice between two certainty alternatives—a choice between the target health outcome, which will last for the rest of his or her life (t) versus the perfect health state that lasts for a shorter time (x) followed by death. As in the standard gamble method, the time x is varied until the point of indifference is reached. In the time trade-off method, the value of patient utility for the target health state is the value of x/t for the point of indifference. In the previous example, if the patient reaches the point of indifference at 2.5 years of complete health versus 10 years of dialysis, the utility of staying on hospital dialysis for this patient is 2.5/10 ⫽ 0.25. Whereas the standard gamble and time trade-off methods were adopted from decision research, the rating scale method was adopted from psychological research. In the rating scale
method, patient utility for a given health outcome is measured by asking a patient to locate the point representing the desirability of the outcome on a line with clear end points—typically, 0 for “death” and 100 for “perfect health.” Because it is customarily assumed that rating scales have the property of intervalness, the value of patient utility for the target health state is the reading of its value on the scale divided by 100. In the previous example, if the patient locates the point representing the desirability of hospital dialysis at 25 on the scale of 0 to 100, the utility of staying on hospital dialysis for this patient is 0.25. Though common, use of the rating scale method to measure patient utility without data transformation is technically incorrect. The rating scale method was designed to measure value functions but not utility functions. Therefore, data transformation such as a power curve correction is necessary to obtain accurate patient utility [11]. When the rating scale method is used without transformation, this use should be considered a special and perhaps unjustified case where researchers assume that the relationship between the value function and the underlying utility function is 1:1.
3. Cognitive processes of patients As self-reports, measures of patient utility are directly influenced by the way patients process information to provide their answers. The following is a review of representative psychological research, which sets forth possible mechanisms through which cognitive processes of patients may bias measures of patient utility. 3.1. Measures of patient utility are cognitively demanding Although measures of patient utility may appear to be simple and straightforward, they require patients to perform highly complex mental operations. For example, the standard gamble method asks patients to simulate hypothetical scenarios about their future, judge the desirability of these scenarios, and compare them against the desirability of taking a risk, which is expressed in probability. As noted above, for this method to be successful, patients need to understand probabilistic information well. However, research suggests that most patients do not [7,8]. The time trade-off and rating scale methods do not have this requirement for their success; however, they still require mental simulations of hypothetical scenarios. In general, psychological research has found that people have great difficulty in accurately simulating hypothetical scenarios in their mind [12], even when the scenarios concern predictions about themselves [13]. Meanwhile, in research on patient preference, it is common for patients to be asked to simulate scenarios about their possible future from a short paragraph describing health outcomes. Obviously, the procedure has much to be improved, given that mental simulation is cognitively taxing and prone to systematic errors. The importance of minimizing the cognitive burden on patients during measurement cannot be stressed too much,
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
given that many patients are in a state of compromised cognitive functioning. Patients are under physical, emotional, and cognitive stresses from their illness. Psychological research investigating the effect of stress on judgment has found that strong stress compromises individuals’ ability to process information. Consequently, in stressful situations, individuals have difficulty making rational decisions [14, 15]. This finding has been verified in decision behaviors of patients with life-threatening diseases: they tend to engage in nonoptimal strategies in their decision making [16]. Therefore, it is advisable to administer measures of patient preference in settings that are least stressful to the patients. In certain cases, psychological counseling may be necessary, before patients are ready to be given measures of utility. Such counseling, however, needs to be administered with caution. In particular, in answering questions that measure utility, patients may pay too much attention to the aspect of the health states that is related to the issue discussed in the preceding counseling session. As such, focusing on a particular issue in counseling may actually introduce a bias. The topic related to this problem will be discussed in detail in a later section. Given that measures of patient utility require respondents to perform complex mental operations, one might argue that researchers should not recruit actual patients who are under an enormous stress from their illness. Why not recruit unaffected and hence objective individuals instead, and treat their aggregated preference as if it were the preference of patients who are not in a state of compromised cognitive functioning? Such an argument may hold when measures of patient utility are used for policy decision making, because policy decision making should be based on social preference [11]. However, the use of social preference as a substitute for individual preference is inappropriate when the measures are used for individual clinical decision making, because each individual is different in his or her preference for health states. 3.2. Measures of patient utility may not elicit thoughtful responses by default Aware or not, individuals are constantly surrounded by a multitude of stimuli and information that needs to be processed. As there is always more information than individuals can possibly process, human cognition is designed to save processing resources (e.g., time and effort) whenever possible as a mean of coping. As a result, there is a default tendency for individuals to rely upon resource-saving instead of resource-intense mental operations—such as the conscious and deliberate mental operations prescribed by rational decision models [17]. In addition, there is always a cost involved for being rational and deliberate in decision making, and therefore thinking rationally on every occasion is actually irrational from a broader perspective [18]. In fact, psychological research has accumulated evidence that people are not by default rational thinkers who process
471
information in a conscious and deliberate manner. Rather, people routinely process information and produce behavior in a mindless manner unless the cost of being mindless is considerable [19,20]. Likewise, people often act upon their automatic affective reactions regarding an issue, not upon their careful considerations of the issue [21,22]. In addition, to make complex judgments cognitively manageable, people tend to rely on judgmental heuristics (i.e., mental short cuts) [23–25]. Psychological research has also found that individuals’ reliance on resource-saving operations is more pronounced when they are strained in cognitive resources. For example, people rely more on resource-saving mental operations when they are performing multiple tasks [26]; when they are cognitively busy attending a multitude of information [27]; when they are under time pressure [28]; or even when they are making decisions during the part of the day that is not optimal with regard to their circadian rhythms [29]. In summary, psychological research raises concerns about the way measures of patient utility are commonly administered. Specifically, research suggests that measures of patient utility may fail to elicit valid self-reports if they are administered in a way that fails to encourage thoughtful responses. For example, methods such as the standard gamble, time trade-off, and rating scale may appear to patients as artificial and trivial exercises that have little to do with the real and serious concerns that they have. Therefore, to make sure that the appearances of measurement instruments will not discourage patients from responding thoughtfully, researchers should stress the direct relevance of these measures to the welfare of patients. Also, these methods may appear too complex to handle, which should encourage patients to rely on heuristics. As judgments based on heuristics are susceptible to systematic biases [7], everything being equal, researchers should choose the measurement instrument that has the simplest appearance. Although there is no guarantee that such an instrument will encourage patients to give more thoughtful responses, it will nevertheless reduce their need to use heuristics as a means of coping. Finally, researchers should provide patients an environment where they can devote their entire attention and take as much time as they need to complete the task. This is something that researchers often overlook in practice. 3.3. Measures of patient utility may be biased by the mood of patients For measures of patient utility to be reliable, transient factors such as patients’ day-to-day fluctuation in mood states should have little influence on their evaluations of health states. However, psychological research has found that individuals’ evaluations of various aspects of their lives are significantly influenced by their transient mood states [30]. The effect of transient mood states will contribute to low test–retest reliability in measures of patient utility. The effect of a positive mood on an evaluation is quite straightforward. When in a positive mood, individuals eval-
472
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
uate nearly everything about their lives—themselves, people around them, their possessions, and their future—in a more positive light than when they are in a neutral mood [31]. In contrast, the effect of a negative mood on an evaluation is hard to predict [30]. Whereas positive moods are mostly identical to one another (happy), negative moods are more differentiated (e.g., sad, angry, fearful). This complexity of negative moods may be partly responsible for the unpredictability of their effects [17]. In short, research suggests that positive mood may bias measures of patient utility positively, and negative mood may bias these measures in unpredictable ways. Individuals’ mood states affect not only their outcome evaluations but also their risk preferences [32]. Further, as individuals’ mood states change, the relative contributions of outcome utility (i.e., how desirable is the outcome?) versus outcome probability (i.e., how likely is the outcome?) on their choice behaviors change [33]. This is particularly problematic for the standard gamble method. Because this method asks patients to make choices involving risks, it may be doubly susceptible to the biasing effect of transient mood. As evidenced by the fact that the effect of mood states biases the standard gamble method, von NeumannMorgenstern utility theory has difficulty accounting for the effect of mood states on utility. In summary, research suggests that measures of patient utility are susceptible to biases introduced by patients’ transient mood states. For this reason, it is recommended that researchers measure the mood states of patients whenever they administer measures of patient utility. The data from the former may then be used to assess the validity of the obtained self-reports of patient utility. They also may be used to investigate the source of low test–retest reliability in the measure. 3.4. Patient utility is judged while it is simultaneously measured Different measures of patient utility tend to yield different estimates for this variable [34–37], which can cause serious problems for research on patient preference. When different measures suggest different patterns of preference, which measures should be trusted? Although serious, this problem was by no means unexpected. Measures of patient utility are self-reports. As such, they are influenced by the way patients process information in providing their responses. Different measures of patient utility rely on different methods in quantifying this variable. As such, each measure encourages patients to process information differently. Obviously, most patients do not come to the clinic with already formed evaluations for the hypothetical health outcomes that will be presented to them in a measurement session. Rather, most patients evaluate the desirability of these health outcomes in the course of measurement. In other words, preferences of patients are constructed while they are simultaneously measured [38]. By influencing the way patients process information to evaluate the desirability of
health outcomes, measurement instruments can affect not only the reported utility—which may or may not truthfully reflect the underlying “true” utility—but also the underlying utility itself. What other factors may affect information processing of patients during measurement of utility, and therefore may affect patient utility? Perhaps the most well known is the effect of framing, which refers to the fact that the way outcomes are framed (phrased) affects patient utility [39]. For instance, the way outcomes are phrased for the treatments for lung cancer (surgery versus radiation) has been found to affect patient utility for treatment outcome, which can lead to a reversal in patient preference [40]. Importantly, framing can also affect preferences of physicians regarding how to treat patients [40]. Both patients and physicians show less preference for surgery when its outcome is presented in terms of mortality (e.g., “Of 100 people having surgery, 10 will die during the procedure . . .”) compared with when its outcome is presented in terms of survival (e.g., “Of 100 people having surgery, 90 will be alive immediately following the procedure . . .”). Patient preference can be assessed either by methods of matching or by methods of choice. In methods of matching, patients assign a numerical value to each health outcome reflecting its desirability. In methods of choice, patients simply rank health outcomes in the order of their desirability. Despite their apparent interchangeability, psychological research has found that these two methods often lead to assessments of preference that conflict with each other [41]. Measures of patient utility—the standard gamble, the time trade-off, and the rating scale—are based on methods of matching. As such, the patient preference that is assessed by a measure of patient utility may conflict with the patient preference that is assessed by simple rank ordering of health outcomes. The results of a recent study by Giesler et al. [34] corroborate this prediction. This is another example where methods of preference elicitation affect preferences of patients. Patients may actively seek and adopt certain cognitive strategies for judging utilities. Such strategies undoubtedly affect information processing of patients during measurement of utility. Seeking and adopting of cognitive strategies generally reflect patients’ motivation to judge utilities with less cognitive cost but with an acceptable level of accuracy [42–45]. Sometimes that may lead patients to restructure or “edit” questions posed by the researcher [46]. For example, to reduce the complexity of the standard gamble, patients may round off the probability values associated with recovery versus death (e.g., change “65%” to “70%”). Such restructuring of task is especially common when outcomes are diverse in kind [47]. In many instances, health outcomes that are considered in a study of patient preference are qualitatively diverse, and therefore restructuring of task is expected to be common for measures of patient utility. The motivation to judge utilities with less cognitive cost but with an acceptable level of accuracy may lead some pa-
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
tients to rely on intuitions, such as the use of mental images [48]. Research has found that individuals often assess preferences by the degree to which their self-images fit with each of the choice alternatives [49]. For example, a patient with an end-stage prostate cancer may intuitively judge that he prefers risking faster progression of cancer to living with side effects from the treatments, because his masculine selfimage cannot fit with either chemical or surgical means of castration. Contrary to general expectation, research suggests that intuitive judgments are not necessarily less accurate than analytical judgments [50]. In general, informal strategies that require less cognitive effort do not necessarily lead to less accurate judgments [44], although they may lead to less confidence in judgments [51]. In summary, several lines of research suggest that patient utility for health outcomes is judged in the course of measurement. Unlike evaluations that are simply retrieved from memory and reported, evaluations that are judged are products of information processing that takes place during measurement. As such, patient utility is affected by both researchers’ choice of measurement instruments as well as patients’ choice of judgment strategies. Conceptualizing patient utility as judged evaluation rather than retrieved evaluation has an important implication for measures of patient utility. Specifically, it suggests that these “measures” are not passively measuring, but actively influencing, patient utility. Because the construct of patient utility cannot be divorced from the method of measurement, comparisons of measures in terms of their ability to yield the “truest” estimate of patient utility are not meaningful. 3.5. Some values may not be accessible from memory when patient utility is judged Judging utilities of health outcomes involves evaluating health outcomes in light of one’s personal values. Therefore, to judge utilities, patients must first access relevant personal values from memory. Patient utility is a product of what cognitive psychologists call memory-based judgment [52]. As a product of memory-based judgment, patient utility reflects which personal values are accessed from memory and incorporated into a judged utility. Potentially diverse personal values are relevant for any given medical situation—such as values related to medical factors (e.g., life expectancy, morbidity), values related to financial factors (e.g., medical expense, insurance coverage), values related to self-image (e.g., physical attractiveness, sexual functioning) and values related to one’s social role and functioning (e.g., career, family). Ideally, patients can access all the relevant personal values in memory when their judgments of health outcome utilities are called for. However, given the diversity of personal values that need to be considered for any given judgment of health outcome utility, accessing all of them may not be possible for many patients. In fact, research suggests that patients may not retrieve all the relevant personal values, and therefore they may
473
judge health outcome utilities from a subset of their personal values that happen to be accessible at the time of measurement [53,54]. Consequently, it is possible for the same patient to prefer different health outcomes depending on which personal values are accessible at the time of measurement. Although this may be considered a problem of test– retest reliability, it may reflect a more fundamental problem. Test–retest reliability is at issue when the observed instability is mainly attributable to the measurement instrument. However, in the case of patient utility, the instability appears to reside within the construct itself [55]. Suppose that a measure of patient utility is administered soon after a patient is visited by her colleagues from work. In this scenario, it is not hard to imagine that her personal values concerning her career may become momentarily quite accessible to her consciousness, and therefore may exert disproportionate influence on her judgment of health outcome utility. In other words, her concerns regarding the impact of each health outcome upon her professional life may excessively influence her judgment of its utility. In contrast, if a measure of patient utility is administered soon after the same patient is visited by her family, her personal values concerning family may exert disproportionate influence. Accessibility of personal values from memory can also be affected by subtler factors. In general, accessibility of information in memory can be heightened momentarily by incidental exposure to the words or pictures that are associatively related to that information [56,57]. The effect of such incidental exposure can even be produced below an individual’s conscious awareness [58,59]. For example, an individual may be thinking of a cigarette in the museum without knowing why, when in fact this effect was produced by exposure to the picture of an Egyptian on a camel. This effect, called “priming,” constitutes a significant source of concern for researchers. It is possible that seemingly innocuous stimuli in the hospital environment that patients do not even notice consciously—such as pictures in magazines, headlines in the newspaper, or other patients in the hall, etc.— can “prime” certain personal values in their memory, and therefore affect their judgments of health outcome utility. Suppose that a measure of patient utility is administered in a waiting room where TV is available. In the first scenario, a drama on TV is showing a dinner scene of a couple. Although a patient may not be paying any conscious attention to the TV, it is quite possible that his personal values concerning physical attractiveness and sexual functioning may become momentarily more accessible than usual, and therefore may exert disproportionate influence on his judgment of health outcome utility. In contrast, if a measure of patient utility is administered to the same patient while a news broadcast is reporting a stock market crash, his personal values concerning finance and cost of medical care may exert disproportionate influence. In summary, research indicates that patient utility tends to reflect the disproportionate influence of personal values
474
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
that happen to be accessible from memory at the time of measurement. This tendency in patient utility contributes to the instability in this construct. Accessibility of personal values may further be influenced by seemingly innocuous stimuli in the environment, which may augment the instability in patient utility. However, there are steps that researchers can take to prevent this instability and bias from happening. First, researchers can encourage patients to think about as diverse personal values as possible that could be relevant to health outcomes, then administer a measure of patient utility. Discussing a wide array of effects that a health outcome may have on various domains of life could be effective in ensuring that no single personal value may exert undue influence on the judgment of health outcome utility. Such a premeasure discussion, however, needs to be carried out with caution. A careless researcher may introduce a bias by prescribing a set of personal values that he or she deems appropriate, while ignoring the possibility that they may not fit the personal or cultural background of the patient. Second, researchers can administer measures of patient utility in a room in which potentially biasing stimuli—the stimuli that can prime certain personal values—are absent. 3.6. Fear of regret affects patient utility People experience regret when they realize that they have made a bad decision. On the other hand, when people realize that they have made a good decision, they experience elation. These emotions regarding the quality of decisions enhance the disutility of bad outcomes or the utility of good outcomes. For instance, consider the case of a victim of breast-implant complications. She must suffer not only from the disutility of complications but also from the regret over her decision to have this procedure done. Feelings of regret or elation can be powerful. However, these emotions are traditionally excluded from the construct of utility, because these emotions concern the quality of decisions rather than the quality of outcomes. As such, fear of regret and anticipation of elation have been traditionally considered as barriers to rational decision making. It is also possible, however, to argue that fear of regret and anticipation of elation should be incorporated into the construct of utility. Whether such feelings themselves are rational or irrational, people do feel regret or elation. Given that, is it not more rational for people to expect these emotional consequences beforehand and take these expectations into account when they make decisions? Indeed, proponents of socalled regret theories [60,61] argue that human rationality should and does incorporate expectation of regret and elation. In practice, however, it may not be advisable to let all patients take into consideration their fear of regret or anticipation of elation. For instance, some patients may have excessive fear of regret, and letting them take this fear into consideration may lead to the unnecessary avoidance of reasonable risks. Research has found that people with low selfesteem are particularly susceptible to the fear of regret, and
consequently they tend to be overly risk averse in decision making [62,63]. Therefore, evaluations of health outcomes that are obtained from patients with low self-esteem may require some calibrations before they can be qualified as valid measurements of patient utility. Such calibrations would likely result in the upward adjustment of valuations given to health outcomes with reasonable risks. 4. Conclusion As self-reports, measures of patient utility are susceptible to the effects of cognitive biases. This article explored mechanisms through which cognitive processes of patients may bias these measures. Understanding these mechanisms is essential for improving the measures. This article identified problems in current measures of patient utility that have been mostly overlooked in the past. First, measures of patient utility demand that patients, who are under an enormous stress from their illness, perform complex mental operations, such as simulating hypothetical scenarios and understanding probabilistic information. Second, measures of patient utility do not elicit thoughtful response from patients by default. Third, measures of patient utility may be biased by the mood of patients. Fourth, patient utility is affected by both researchers’ choice of measurement instruments and patients’ choice of judgment strategies. Fifth, measures of patient utility tend to reflect the disproportionate influence of personal values that happen to be accessible from memory at the time of measurement. Sixth, patient utility is affected by patients’ fear of regret. Fortunately, most of these problems can be dealt with by improving the methods of administering currently available measures. Such improvements can be made through three means: improving the setting of administration, offering assistance to patients, and taking additional measures for calibration. First, researchers should be more careful about their choice of setting in which they administer measures of patient utility. It was suggested that researchers choose the time and setting that is: 1) least stressful to patients; 2) least distracting to patients, so that patients can devote their full attention to the task without time pressure; and 3) devoid, if possible, of stimuli that can prime certain personal values. Second, researchers should offer assistance to patients both before and during the measurement. It was suggested that researchers: 1) offer counseling to patients who are significantly stressed, before the measure is given; 2) encourage patients to consider as diverse personal values as possible before the measure; 3) let patients consider the possibility of their future regret; 4) encourage patients to be thoughtful in responding to the measure, by stressing the relevance of the measure to their welfare; and 5) offer assistance in simulating future scenarios. Third, under certain circumstances, researchers should take additional measures to calibrate the measures of patient
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
utility. When researchers suspect that patients’ mood or self-esteem is biasing measures of patient utility (e.g., a patient with low self-esteem who is in a bad mood), they should measure the respective variable and adjust the data on patient utility accordingly. More research is needed to develop a method for such calibrations. While it is true that researchers can deal with many problems associated with current measures of patient utility by improving the administration of available measures, they are also encouraged to develop better measures that are less taxing to patients. In particular, developing measures that appear less intimidating and in fact easier to process than currently available measures is needed. In developing alternative measures, however, it is important to recognize the fact that current measures of patient utility were not developed “bottom-up” from the field of clinical medicine. Instead, they were imported from other academic disciplines such as economics, decision science, and psychology, then applied “top-down” to the field of clinical medicine. In this sense, it is no surprise that these measures often fail to capture the actual preferences of patients in real clinical settings. Consequently, the development of alternative measures would benefit tremendously from the simultaneous development of clinimetrics, which focuses on the accurate and scientific description of uniquely clinical phenomena [64,65]. One problem, however, demands conceptual work. Patient utility is affected by both researchers’ choice of instruments and patients’ choice of strategies. That is because patient utility is judged by patients while it is simultaneously measured by researchers. Therefore, this problem cannot be addressed adequately unless researchers redefine patient utility as judged evaluation rather than retrieved evaluation. Such conceptual refinement will have profound implications for research on patient utility. One implication that was discussed in this article was that these “measures” are not passively measuring, but actively shaping, patient utility. Because the construct of patient utility cannot be divorced from the method of its measurement, in principle researchers cannot claim that they are comparing different measures for fidelity—an ability to yield the “best” estimate of “underlying” patient utility. In conclusion, though many problems surround current methods of measuring patient utility, equally many solutions exist that we as researchers can implement to improve the current situation. Measurement of patient utility is a basis for research on patient preference, which in turn is a basis for delivery of patient-centered medicine. It makes good sense that, to provide patient-centered care, we must also start providing measures that cater to patients’ cognitive needs.
Acknowledgments Supported by the Department of Veterans Affairs, Office of Academic Affiliations, in collaboration with the Health
475
Services Research and Development (HSR&D) Service, Associated Health Professions Post-Doctoral Fellowship Program in Health Services Research (Hanita).
References [1] Nease RF Jr., Kneeland T, O’Connor GT, Sumner W, Lumpkins C, Shaw L, Pryor D, Sox HC. Variation in patient utilities for outcomes of the management of chronic stable angina: Implications for clinical practice guidelines. JAMA 1995;273:1185–90. [2] Flood AB, Wennberg JE, Nease RF, Fowler FJ Jr., Ding J, Hynes LM. The importance of patient preference in the decision to screen for prostate cancer. J Gen Intern Med 1996;11:342–49. [3] Dunn VH, Pauker SG. An ambiguous renal cyst in a patient with a prior nephrectomy: what to do when there is only one more to lose. Med Decis Making 1986;6:49–62. [4] Zalkind DL, Shachtman RH. A decision analysis approach to the swine influenza vaccination decision for an individual. Med Care 1980;18:59–72. [5] Froberg DG, Kane RL. Methodologies for measuring health-state preferences—II: scaling methods. J Clin Epidemiol 1989;42:459–71. [6] von Neumann J, Morgenstern O. Theory of Games and Economic Behavior. New York: Wiley, 1953. [7] Kahneman D, Slovic P, Tversky A, editors. Judgment Under Uncertainty: Heuristics and Biases. New York: Cambridge University Press, 1982. [8] Arkes HR, Hammond KR, editors. Judgment and Decision Making: An Interdisciplinary Reader. Cambridge: Cambridge University Press, 1986. [9] Llewellyn-Thomas H, Sutherland HJ, Tibshirani R, Ciampi A, Till JE, Boyd NF. The measurement of patients’ values in medicine. Med Decis Making 1982;2:449–62. [10] Torrance GW, Thomas WH, Sackett DL. A utility maximization model for evaluation of health care programs. Health Serv Res 1972; 7:118–33. [11] Torrance GW, Boyle MH, Horwood SP. Application of multi-attribute utility theory to measure social preferences for health states. Operations Res 1982;30:1043–69. [12] Kahneman D, Tversky A. The simulation heuristic. In: Kahneman D, Slovic P, Tversky A, editors. Judgment Under Uncertainty: Heuristics and Biases. New York: Cambridge University Press, 1982. pp. 201–8. [13] Griffin DW, Dunning D, Ross L. The role of construal processes in overconfident predictions about the self and others. J Pers Soc Psychol 1990;59:1128–39. [14] Janis IL. Decisionmaking under stress. In: Goldberger L, Breznitz S, editors. Handbook of Stress: Theoretical and Clinical Aspects, 2nd ed. New York: Free Press; 1993. pp. 56–74. [15] Mann L. Stress, affect, and risk taking. In: Yates JF, editor. Risk-Taking Behavior. New York: Wiley; 1992. pp. 201–30. [16] Janis IL. Coping patterns among patients with life-threatening diseases. Issues Ment Health Nurs 1985;7:461–76. [17] Fiske ST, Taylor SE. Social Cognition: Second edition. New York: McGraw-Hill, 1991. [18] Jungermann H. The two camps on rationality. In: Scholz RW, editor. Decision Making Under Uncertainty. Amsterdam: Elsevier, 1983. pp. 63–86. [19] Langer EJ, Blank A, Chanowitz B. The mindlessness of ostensibly thoughtful action: the role of “placebic” information in interpersonal interaction. J Pers Soc Psychol 1978;36:635–42. [20] Langer EJ. Minding matters. In: Berkowitz L, Ed. Advances in Experimental Social Psychology, vol. 22. New York: Academic Press; 1989. pp. 137–73. [21] Fazio RH, Sanbonmatsu DM, Powell MC, Kardes FR. On the automatic activation of attitudes. J Pers Soc Psychol 1986;50:229–38.
476
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
[22] Fazio RH. On the power and functionality of attitudes: The role of attitude accessibility. In: Pratkanis AR, Breckler SJ, Greenwald AG, editors. Attitude Structure and Function. Hillsdale, NJ: Erlbaum; 1989. pp. 153–79. [23] Kahneman D, Tversky A. Subjective probability: a judgment of representativeness. Cognit Psychol 1972;3:430–54. [24] Tversky A, Kahneman D. availability: A heuristic for judging frequency and probability. Cognit Psychol 1972;4:207–32. [25] Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science 1974;185:1124–31. [26] Stangor C, Duan C. Effects of multiple task demands upon memory for information about social groups. J Exp Soc Psychol 1991;27:357– 78. [27] Gilbert DT, Hixon JG. The trouble of thinking: activation and application of stereotypic beliefs. J Pers Soc Psychol 1991;60:509–17. [28] Kruglanski AW, Freund T. The freezing and unfreezing of lay-inferences: effects of impressional primacy, ethnic stereotyping, and numerical anchoring. J Exp Soc Psychol 1983;19:448–68. [29] Bodenhausen GV. Stereotypes as judgmental heuristics: evidence of circadian variations in discrimination. Psychol Sci 1990;1:319–22. [30] Isen AM. Toward understanding the role of affect in cognition. In: Wyer RS Jr., Srull TK, editors. Handbook of Social Cognition, vol. 3. Hillsdale, NJ: Erlbaum; 1984. pp. 179–236. [31] Isen AM, Shalker TE, Clark M, Karp L. Affect, accessibility of material in memory, and behavior: a cognitive loop? J Pers Soc Psychol 1978;36:1–12. [32] Isen AM, Geva N. The influence of positive affect on acceptable level of risk: the person with a large canoe had a large worry. Organ Behav Hum Decis Processes 1987;39:145–54. [33] Nygren TE, Isen AM, Taylor PJ, Dulin J. The influence of positive affect on the decision rule in risk situations: focus on outcome (and especially avoidance of loss) rather than probability. Organ Behav Hum Decis Processes 1996;66:59–72. [34] Giesler RB, Ashton CM, Brody B, Byrne MM, Cook K, Geraci JM, Hanita M, Souchek J, Wray NP. Assessing the performance of utility techniques in the absence of a gold standard. Med Care (in press). [35] Hornberger JC, Redelmeier DA, Petersen J. Variability among methods to assess patients’ well-being and consequent effect on a costeffectiveness analysis. J Clin Epidemiol 1992;45:505–12. [36] Read JL, Quinn RJ, Berwick DM, Fineberg HV, Weinstein MC. Preferences for health outcomes: comparison of assessment methods. Med Decis Making 1984;4:315–29. [37] Stiggelbout AM, Eijkemans MJC, Kiebert GM, Kievit J, Leer JWH, de Haes JCJM. The utility of the visual analog scale in medical decision making and technology assessment. Int J Technol Assess Health Care 1996;12:291–98. [38] Slovic P. The construction of preference. Am Psychol 1995;50:364– 71. [39] Tverksy A, Kahneman D. Rational choice and the framing of decisions. J Business 1986;59S:251–78. [40] McNeil BJ, Pauker SG, Sox HC, Tversky A. On the elicitation of preferences for alternative therapies. N Engl J Med 1982;306:1259– 62. [41] Tversky A, Sattath S, Slovic P. Contingent weighting in judgment and choice. Psychol Rev 1988;95:371–84. [42] Beach LR, Mitchell TR. A contingency model for the selection of decision strategies. Acad Manage Rev 1978;3:439–49.
[43] Payne JW, Bettman JR, Johnson EJ. Adaptive strategy selection in decision making. J Exp Psychol Learn Mem Cognit 1988;14:534–52. [44] Russo JE, Dosher BA. Strategies for multiattribute binary choice. J Exp Psychol Learn Mem Cognit 1983;9:676–96. [45] Simon HA. Rational choice and the structure of the environment. Psychol Rev 1956;63:129–38. [46] Payne JW, Bettman JR, Johnson EJ. Behavioral decision research: A constructive processing perspective. Annu Rev Psychol 1992;43:87– 131. [47] Johnson MD. Consumer choice strategies for comparing noncomparable alternatives. J Consumer Res 1984;11:741–53. [48] Mitchell TR, Beach LR. “. . . Do I love thee? Let me count . . .” Toward an understanding of intuitive and automatic decision making. Organ Behav Hum Decis Processes 1990;47:1–20. [49] Niedenthal PM, Cantor N, Kihlstrom JF. Prototype matching: a strategy for social decision making. J Pers Soc Psychol 1985;48:575–84. [50] Hammond KR, Hamm RM, Grassia J, Pearson T. Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment. System Man Cybernet IEEE Trans 1987;17:753–70. [51] Paese PW, Sniezek JA. Influences on the appropriateness of confidence in judgment: practice, effort, information, and decision-making. Organ Behav Hum Decis Processes 1991;48:100–30. [52] Hastie R, Park B. The relationship between memory and judgment depends on whether the judgment task is memory-based or on-line. Psychol Rev 1986;93:258–68. [53] Salancik G, Conway M. Attitude inferences from salient and relevant cognitive content about behavior. J Pers Soc Psychol 1975;32:829–40. [54] Tourangeau R, Rasinski KA, Bradburn N, D’Andrade R. Belief accessibility and context effects in attitude measurement. J Exp Soc Psychol 1989;25:401–21. [55] Christensen-Szalanski JJJ. Discount functions and the measurement of patients’ values. Med Decis Making 1984;4:47–58. [56] Higgins ET, Rholes WS, Jones CR. Category accessibility and impression formation. J Exp Soc Psychol 1977;13:141–54. [57] Srull TK, Wyer RS. Category accessibility and social perception: some implications for the study of person memory and interpersonal judgments. J Pers Soc Psychol 1980;38:841–56. [58] Bargh JA, Pietromonaco P. Automatic information processing and social perception: the influence of trait information presented outside of conscious awareness on impression formation. J Pers Soc Psychol 1982;43:437–49. [59] Bargh JA, Bond RN, Lombardi WL, Tota ME. The additive nature of chronic and temporary sources of construct accessibility. J Pers Soc Psychol 1986;50:869–79. [60] Bell DE. Regret in decision making under uncertainty. Operations Res 1982;30:961–81. [61] Loomes G, Sugden R. Regret theory: an alternative theory of rational choice under uncertainty. Econ J 1982;92:805–24. [62] Josephs RA, Larrick RP, Steele CM, Nisbett RE. Protecting the self from the negative consequences of risky decisions. J Pers Soc Psychol 1992;62:26–37. [63] Larrick RP. Motivational factors in decision theories: the role of selfprotection. Psychol Bull 1993;113:440–50. [64] Feinstein AR. An additional basic science for clinical medicine: IV. The development of clinimetrics. Ann Intern Med 1983;99:843–48. [65] Feinstein AR. Clinical judgment revisited: the distraction of quantitative models. Ann Intern Med 1994;120:799–805.
Self-report measures of patient utility: should we trust them? Makoto Hanita* Houston Center for Quality of Care and Utilization Studies, Houston Veterans Affairs Medical Center, 2002 Holcombe Boulevard, Houston, TX 77030, USA; Baylor College of Medicine, One Baylor Plaza, Houston, TX 77030 Received 20 May 1998; received in revised form 22 September 1999; accepted 6 October 1999
Abstract As self-reports, measures of patient utility are susceptible to the effects of cognitive biases in patients. This article presents often overlooked problems in these measures by outlining cognitive processes involved in patient self-report. It is argued that these measures: 1) require overly complex mental operations; 2) fail to elicit thoughtful response by default; 3) may be biased by patients’ mood; 4) are affected by both researchers’ choice of measurement instruments and patients’ choice of judgment strategies; 5) tend to reflect the disproportionate influence of patients’ values that happen to be recallable at the time of measurement; and 6) are affected by patients’ fear of regret. It is suggested that solutions for these problems should involve: a) improving the methods of administration; b) developing measures that are less taxing to patients; and c) redefining the concept of patient utility as judged, as opposed to retrieved, evaluation. Published by 2000 Elsevier Science Inc. All rights reserved. Keywords: Patient preference; Expected utility; Measurement; Patient self-report; Medical decision making; Cognitive psychology
1. Introduction To provide care that will satisfy patients, care providers must know preferences of patients [1,2]. How much discomfort from side effects are patients willing to tolerate to stay symptom free? How many hospital visits are patients willing to make for painful procedures that may or may not work? Care providers must have at least tentative answers to these questions before treating patients. In this regard, research on patient preference is a cornerstone of patient-centered medicine. In research on patient preference, a set of possible health outcomes is defined for patients with a given medical condition. These health outcomes are then compared against one another, in terms of their potential for eliciting satisfaction in patients [3,4]. Various measures of expected patient satisfaction—or patient utility—are used for this comparison, to quantify the level of satisfaction that patients associate with each of the possible health outcomes. Although different measures of patient utility are based on different methods of quantifying the level of patient satisfaction, there is one property that is common to all of them: all measures of patient utility are based on selfreports. This is an important property that distinguished these measures from physiological or behavioral measures.
In self-reports, patients must take active roles: they must comprehend the questions and come up with the answers. Consequently, measures that are based on self-reports are susceptible to the effects of cognitive biases in patients. Often, these cognitive biases make a straightforward interpretation of the measures of patient utility problematic; however, this is one issue that researchers tend to overlook. The focus of this article is to present a review of psychological research on human cognition surrounding the issue of utility and preference. The purpose of this review is to outline various mechanisms through which cognitive processes of patients may bias measures of patient utility. Understanding such mechanisms is essential for improving available measures as well as developing better ones. Before turning to the discussion of cognitive processes, however, a brief description of measures of patient utility is provided for readers who are not familiar with the issue. Because some readers may want to skip this part, the description is intentionally kept at a cursory level. Readers who wish to study the measurement of patient utility in detail are thus encouraged to refer to the excellent reviews that are available (e.g., Froberg and Kane [5]). 2. Measures of patient utility
* Corresponding author. Tel.: (713) 794-7615 / 7909; Fax: (713) 7947103. E-mail address: [email protected] (M. Hanita)
Several options are available for researchers who are interested in measuring patient utility. Among those, the three
0895-4356/00/$ – see front matter. Published by 2000 Elsevier Science Inc. All rights reserved. PII: S0895-4356(99)00 2 0 5 - X
470
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
most commonly chosen are the standard gamble, the time trade-off, and the rating scale. The standard gamble method was originally developed by von Neumann and Morgenstern [6]. This method is often considered the only one that yields “true” values of utility, because of its strict adherence to the axioms of modern utility theory. In the standard gamble method, patient utility for a given health outcome is measured by asking a patient to consider a choice between a certainty and a gamble—a choice between the target health outcome, which will last for the rest of his or her life, versus a treatment with a known probability (p) of success. If successful (p), the treatment would bring the patient perfect health that will last for the rest of his or her life. If not successful (1 ⫺ p), it would bring the patient immediate death. For example, a patient may be asked to choose between staying on hospital dialysis of the rest of his or her life (e.g., 10 years) versus undergoing a risky treatment, which has a 30% probability of complete recovery lasting for life and a 70% probability of immediate death. The probability of recovery versus death in the gamble is then varied until the point (i.e., “point of indifference”) is reached where the patient finds it impossible to choose between the two treatments. In the standard gamble method, the value of patient utility for the target health state is the value of probability for the point of indifference. In the above example, if the patient reaches the point of indifference at a 25% probability of complete recovery (and a 75% probability of immediate death), the utility of staying on hospital dialysis for this patient is 0.25. Despite its strong theoretical grounding, the standard gamble is not necessarily a practical method for most patients. Success of this method relies heavily on the ability of patients to understand probabilistic information; however, psychological research has found that people are generally bad at this [7,8]. Not surprisingly, health services research has also documented the difficulties that patients experience with this method [9]. The time trade-off method [10] is a practical alternative to the standard gamble method because it does not require patients to understand probabilistic information. In the time trade-off method, patient utility for a given health outcome is measured by asking a patient to consider a choice between two certainty alternatives—a choice between the target health outcome, which will last for the rest of his or her life (t) versus the perfect health state that lasts for a shorter time (x) followed by death. As in the standard gamble method, the time x is varied until the point of indifference is reached. In the time trade-off method, the value of patient utility for the target health state is the value of x/t for the point of indifference. In the previous example, if the patient reaches the point of indifference at 2.5 years of complete health versus 10 years of dialysis, the utility of staying on hospital dialysis for this patient is 2.5/10 ⫽ 0.25. Whereas the standard gamble and time trade-off methods were adopted from decision research, the rating scale method was adopted from psychological research. In the rating scale
method, patient utility for a given health outcome is measured by asking a patient to locate the point representing the desirability of the outcome on a line with clear end points—typically, 0 for “death” and 100 for “perfect health.” Because it is customarily assumed that rating scales have the property of intervalness, the value of patient utility for the target health state is the reading of its value on the scale divided by 100. In the previous example, if the patient locates the point representing the desirability of hospital dialysis at 25 on the scale of 0 to 100, the utility of staying on hospital dialysis for this patient is 0.25. Though common, use of the rating scale method to measure patient utility without data transformation is technically incorrect. The rating scale method was designed to measure value functions but not utility functions. Therefore, data transformation such as a power curve correction is necessary to obtain accurate patient utility [11]. When the rating scale method is used without transformation, this use should be considered a special and perhaps unjustified case where researchers assume that the relationship between the value function and the underlying utility function is 1:1.
3. Cognitive processes of patients As self-reports, measures of patient utility are directly influenced by the way patients process information to provide their answers. The following is a review of representative psychological research, which sets forth possible mechanisms through which cognitive processes of patients may bias measures of patient utility. 3.1. Measures of patient utility are cognitively demanding Although measures of patient utility may appear to be simple and straightforward, they require patients to perform highly complex mental operations. For example, the standard gamble method asks patients to simulate hypothetical scenarios about their future, judge the desirability of these scenarios, and compare them against the desirability of taking a risk, which is expressed in probability. As noted above, for this method to be successful, patients need to understand probabilistic information well. However, research suggests that most patients do not [7,8]. The time trade-off and rating scale methods do not have this requirement for their success; however, they still require mental simulations of hypothetical scenarios. In general, psychological research has found that people have great difficulty in accurately simulating hypothetical scenarios in their mind [12], even when the scenarios concern predictions about themselves [13]. Meanwhile, in research on patient preference, it is common for patients to be asked to simulate scenarios about their possible future from a short paragraph describing health outcomes. Obviously, the procedure has much to be improved, given that mental simulation is cognitively taxing and prone to systematic errors. The importance of minimizing the cognitive burden on patients during measurement cannot be stressed too much,
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
given that many patients are in a state of compromised cognitive functioning. Patients are under physical, emotional, and cognitive stresses from their illness. Psychological research investigating the effect of stress on judgment has found that strong stress compromises individuals’ ability to process information. Consequently, in stressful situations, individuals have difficulty making rational decisions [14, 15]. This finding has been verified in decision behaviors of patients with life-threatening diseases: they tend to engage in nonoptimal strategies in their decision making [16]. Therefore, it is advisable to administer measures of patient preference in settings that are least stressful to the patients. In certain cases, psychological counseling may be necessary, before patients are ready to be given measures of utility. Such counseling, however, needs to be administered with caution. In particular, in answering questions that measure utility, patients may pay too much attention to the aspect of the health states that is related to the issue discussed in the preceding counseling session. As such, focusing on a particular issue in counseling may actually introduce a bias. The topic related to this problem will be discussed in detail in a later section. Given that measures of patient utility require respondents to perform complex mental operations, one might argue that researchers should not recruit actual patients who are under an enormous stress from their illness. Why not recruit unaffected and hence objective individuals instead, and treat their aggregated preference as if it were the preference of patients who are not in a state of compromised cognitive functioning? Such an argument may hold when measures of patient utility are used for policy decision making, because policy decision making should be based on social preference [11]. However, the use of social preference as a substitute for individual preference is inappropriate when the measures are used for individual clinical decision making, because each individual is different in his or her preference for health states. 3.2. Measures of patient utility may not elicit thoughtful responses by default Aware or not, individuals are constantly surrounded by a multitude of stimuli and information that needs to be processed. As there is always more information than individuals can possibly process, human cognition is designed to save processing resources (e.g., time and effort) whenever possible as a mean of coping. As a result, there is a default tendency for individuals to rely upon resource-saving instead of resource-intense mental operations—such as the conscious and deliberate mental operations prescribed by rational decision models [17]. In addition, there is always a cost involved for being rational and deliberate in decision making, and therefore thinking rationally on every occasion is actually irrational from a broader perspective [18]. In fact, psychological research has accumulated evidence that people are not by default rational thinkers who process
471
information in a conscious and deliberate manner. Rather, people routinely process information and produce behavior in a mindless manner unless the cost of being mindless is considerable [19,20]. Likewise, people often act upon their automatic affective reactions regarding an issue, not upon their careful considerations of the issue [21,22]. In addition, to make complex judgments cognitively manageable, people tend to rely on judgmental heuristics (i.e., mental short cuts) [23–25]. Psychological research has also found that individuals’ reliance on resource-saving operations is more pronounced when they are strained in cognitive resources. For example, people rely more on resource-saving mental operations when they are performing multiple tasks [26]; when they are cognitively busy attending a multitude of information [27]; when they are under time pressure [28]; or even when they are making decisions during the part of the day that is not optimal with regard to their circadian rhythms [29]. In summary, psychological research raises concerns about the way measures of patient utility are commonly administered. Specifically, research suggests that measures of patient utility may fail to elicit valid self-reports if they are administered in a way that fails to encourage thoughtful responses. For example, methods such as the standard gamble, time trade-off, and rating scale may appear to patients as artificial and trivial exercises that have little to do with the real and serious concerns that they have. Therefore, to make sure that the appearances of measurement instruments will not discourage patients from responding thoughtfully, researchers should stress the direct relevance of these measures to the welfare of patients. Also, these methods may appear too complex to handle, which should encourage patients to rely on heuristics. As judgments based on heuristics are susceptible to systematic biases [7], everything being equal, researchers should choose the measurement instrument that has the simplest appearance. Although there is no guarantee that such an instrument will encourage patients to give more thoughtful responses, it will nevertheless reduce their need to use heuristics as a means of coping. Finally, researchers should provide patients an environment where they can devote their entire attention and take as much time as they need to complete the task. This is something that researchers often overlook in practice. 3.3. Measures of patient utility may be biased by the mood of patients For measures of patient utility to be reliable, transient factors such as patients’ day-to-day fluctuation in mood states should have little influence on their evaluations of health states. However, psychological research has found that individuals’ evaluations of various aspects of their lives are significantly influenced by their transient mood states [30]. The effect of transient mood states will contribute to low test–retest reliability in measures of patient utility. The effect of a positive mood on an evaluation is quite straightforward. When in a positive mood, individuals eval-
472
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
uate nearly everything about their lives—themselves, people around them, their possessions, and their future—in a more positive light than when they are in a neutral mood [31]. In contrast, the effect of a negative mood on an evaluation is hard to predict [30]. Whereas positive moods are mostly identical to one another (happy), negative moods are more differentiated (e.g., sad, angry, fearful). This complexity of negative moods may be partly responsible for the unpredictability of their effects [17]. In short, research suggests that positive mood may bias measures of patient utility positively, and negative mood may bias these measures in unpredictable ways. Individuals’ mood states affect not only their outcome evaluations but also their risk preferences [32]. Further, as individuals’ mood states change, the relative contributions of outcome utility (i.e., how desirable is the outcome?) versus outcome probability (i.e., how likely is the outcome?) on their choice behaviors change [33]. This is particularly problematic for the standard gamble method. Because this method asks patients to make choices involving risks, it may be doubly susceptible to the biasing effect of transient mood. As evidenced by the fact that the effect of mood states biases the standard gamble method, von NeumannMorgenstern utility theory has difficulty accounting for the effect of mood states on utility. In summary, research suggests that measures of patient utility are susceptible to biases introduced by patients’ transient mood states. For this reason, it is recommended that researchers measure the mood states of patients whenever they administer measures of patient utility. The data from the former may then be used to assess the validity of the obtained self-reports of patient utility. They also may be used to investigate the source of low test–retest reliability in the measure. 3.4. Patient utility is judged while it is simultaneously measured Different measures of patient utility tend to yield different estimates for this variable [34–37], which can cause serious problems for research on patient preference. When different measures suggest different patterns of preference, which measures should be trusted? Although serious, this problem was by no means unexpected. Measures of patient utility are self-reports. As such, they are influenced by the way patients process information in providing their responses. Different measures of patient utility rely on different methods in quantifying this variable. As such, each measure encourages patients to process information differently. Obviously, most patients do not come to the clinic with already formed evaluations for the hypothetical health outcomes that will be presented to them in a measurement session. Rather, most patients evaluate the desirability of these health outcomes in the course of measurement. In other words, preferences of patients are constructed while they are simultaneously measured [38]. By influencing the way patients process information to evaluate the desirability of
health outcomes, measurement instruments can affect not only the reported utility—which may or may not truthfully reflect the underlying “true” utility—but also the underlying utility itself. What other factors may affect information processing of patients during measurement of utility, and therefore may affect patient utility? Perhaps the most well known is the effect of framing, which refers to the fact that the way outcomes are framed (phrased) affects patient utility [39]. For instance, the way outcomes are phrased for the treatments for lung cancer (surgery versus radiation) has been found to affect patient utility for treatment outcome, which can lead to a reversal in patient preference [40]. Importantly, framing can also affect preferences of physicians regarding how to treat patients [40]. Both patients and physicians show less preference for surgery when its outcome is presented in terms of mortality (e.g., “Of 100 people having surgery, 10 will die during the procedure . . .”) compared with when its outcome is presented in terms of survival (e.g., “Of 100 people having surgery, 90 will be alive immediately following the procedure . . .”). Patient preference can be assessed either by methods of matching or by methods of choice. In methods of matching, patients assign a numerical value to each health outcome reflecting its desirability. In methods of choice, patients simply rank health outcomes in the order of their desirability. Despite their apparent interchangeability, psychological research has found that these two methods often lead to assessments of preference that conflict with each other [41]. Measures of patient utility—the standard gamble, the time trade-off, and the rating scale—are based on methods of matching. As such, the patient preference that is assessed by a measure of patient utility may conflict with the patient preference that is assessed by simple rank ordering of health outcomes. The results of a recent study by Giesler et al. [34] corroborate this prediction. This is another example where methods of preference elicitation affect preferences of patients. Patients may actively seek and adopt certain cognitive strategies for judging utilities. Such strategies undoubtedly affect information processing of patients during measurement of utility. Seeking and adopting of cognitive strategies generally reflect patients’ motivation to judge utilities with less cognitive cost but with an acceptable level of accuracy [42–45]. Sometimes that may lead patients to restructure or “edit” questions posed by the researcher [46]. For example, to reduce the complexity of the standard gamble, patients may round off the probability values associated with recovery versus death (e.g., change “65%” to “70%”). Such restructuring of task is especially common when outcomes are diverse in kind [47]. In many instances, health outcomes that are considered in a study of patient preference are qualitatively diverse, and therefore restructuring of task is expected to be common for measures of patient utility. The motivation to judge utilities with less cognitive cost but with an acceptable level of accuracy may lead some pa-
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
tients to rely on intuitions, such as the use of mental images [48]. Research has found that individuals often assess preferences by the degree to which their self-images fit with each of the choice alternatives [49]. For example, a patient with an end-stage prostate cancer may intuitively judge that he prefers risking faster progression of cancer to living with side effects from the treatments, because his masculine selfimage cannot fit with either chemical or surgical means of castration. Contrary to general expectation, research suggests that intuitive judgments are not necessarily less accurate than analytical judgments [50]. In general, informal strategies that require less cognitive effort do not necessarily lead to less accurate judgments [44], although they may lead to less confidence in judgments [51]. In summary, several lines of research suggest that patient utility for health outcomes is judged in the course of measurement. Unlike evaluations that are simply retrieved from memory and reported, evaluations that are judged are products of information processing that takes place during measurement. As such, patient utility is affected by both researchers’ choice of measurement instruments as well as patients’ choice of judgment strategies. Conceptualizing patient utility as judged evaluation rather than retrieved evaluation has an important implication for measures of patient utility. Specifically, it suggests that these “measures” are not passively measuring, but actively influencing, patient utility. Because the construct of patient utility cannot be divorced from the method of measurement, comparisons of measures in terms of their ability to yield the “truest” estimate of patient utility are not meaningful. 3.5. Some values may not be accessible from memory when patient utility is judged Judging utilities of health outcomes involves evaluating health outcomes in light of one’s personal values. Therefore, to judge utilities, patients must first access relevant personal values from memory. Patient utility is a product of what cognitive psychologists call memory-based judgment [52]. As a product of memory-based judgment, patient utility reflects which personal values are accessed from memory and incorporated into a judged utility. Potentially diverse personal values are relevant for any given medical situation—such as values related to medical factors (e.g., life expectancy, morbidity), values related to financial factors (e.g., medical expense, insurance coverage), values related to self-image (e.g., physical attractiveness, sexual functioning) and values related to one’s social role and functioning (e.g., career, family). Ideally, patients can access all the relevant personal values in memory when their judgments of health outcome utilities are called for. However, given the diversity of personal values that need to be considered for any given judgment of health outcome utility, accessing all of them may not be possible for many patients. In fact, research suggests that patients may not retrieve all the relevant personal values, and therefore they may
473
judge health outcome utilities from a subset of their personal values that happen to be accessible at the time of measurement [53,54]. Consequently, it is possible for the same patient to prefer different health outcomes depending on which personal values are accessible at the time of measurement. Although this may be considered a problem of test– retest reliability, it may reflect a more fundamental problem. Test–retest reliability is at issue when the observed instability is mainly attributable to the measurement instrument. However, in the case of patient utility, the instability appears to reside within the construct itself [55]. Suppose that a measure of patient utility is administered soon after a patient is visited by her colleagues from work. In this scenario, it is not hard to imagine that her personal values concerning her career may become momentarily quite accessible to her consciousness, and therefore may exert disproportionate influence on her judgment of health outcome utility. In other words, her concerns regarding the impact of each health outcome upon her professional life may excessively influence her judgment of its utility. In contrast, if a measure of patient utility is administered soon after the same patient is visited by her family, her personal values concerning family may exert disproportionate influence. Accessibility of personal values from memory can also be affected by subtler factors. In general, accessibility of information in memory can be heightened momentarily by incidental exposure to the words or pictures that are associatively related to that information [56,57]. The effect of such incidental exposure can even be produced below an individual’s conscious awareness [58,59]. For example, an individual may be thinking of a cigarette in the museum without knowing why, when in fact this effect was produced by exposure to the picture of an Egyptian on a camel. This effect, called “priming,” constitutes a significant source of concern for researchers. It is possible that seemingly innocuous stimuli in the hospital environment that patients do not even notice consciously—such as pictures in magazines, headlines in the newspaper, or other patients in the hall, etc.— can “prime” certain personal values in their memory, and therefore affect their judgments of health outcome utility. Suppose that a measure of patient utility is administered in a waiting room where TV is available. In the first scenario, a drama on TV is showing a dinner scene of a couple. Although a patient may not be paying any conscious attention to the TV, it is quite possible that his personal values concerning physical attractiveness and sexual functioning may become momentarily more accessible than usual, and therefore may exert disproportionate influence on his judgment of health outcome utility. In contrast, if a measure of patient utility is administered to the same patient while a news broadcast is reporting a stock market crash, his personal values concerning finance and cost of medical care may exert disproportionate influence. In summary, research indicates that patient utility tends to reflect the disproportionate influence of personal values
474
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
that happen to be accessible from memory at the time of measurement. This tendency in patient utility contributes to the instability in this construct. Accessibility of personal values may further be influenced by seemingly innocuous stimuli in the environment, which may augment the instability in patient utility. However, there are steps that researchers can take to prevent this instability and bias from happening. First, researchers can encourage patients to think about as diverse personal values as possible that could be relevant to health outcomes, then administer a measure of patient utility. Discussing a wide array of effects that a health outcome may have on various domains of life could be effective in ensuring that no single personal value may exert undue influence on the judgment of health outcome utility. Such a premeasure discussion, however, needs to be carried out with caution. A careless researcher may introduce a bias by prescribing a set of personal values that he or she deems appropriate, while ignoring the possibility that they may not fit the personal or cultural background of the patient. Second, researchers can administer measures of patient utility in a room in which potentially biasing stimuli—the stimuli that can prime certain personal values—are absent. 3.6. Fear of regret affects patient utility People experience regret when they realize that they have made a bad decision. On the other hand, when people realize that they have made a good decision, they experience elation. These emotions regarding the quality of decisions enhance the disutility of bad outcomes or the utility of good outcomes. For instance, consider the case of a victim of breast-implant complications. She must suffer not only from the disutility of complications but also from the regret over her decision to have this procedure done. Feelings of regret or elation can be powerful. However, these emotions are traditionally excluded from the construct of utility, because these emotions concern the quality of decisions rather than the quality of outcomes. As such, fear of regret and anticipation of elation have been traditionally considered as barriers to rational decision making. It is also possible, however, to argue that fear of regret and anticipation of elation should be incorporated into the construct of utility. Whether such feelings themselves are rational or irrational, people do feel regret or elation. Given that, is it not more rational for people to expect these emotional consequences beforehand and take these expectations into account when they make decisions? Indeed, proponents of socalled regret theories [60,61] argue that human rationality should and does incorporate expectation of regret and elation. In practice, however, it may not be advisable to let all patients take into consideration their fear of regret or anticipation of elation. For instance, some patients may have excessive fear of regret, and letting them take this fear into consideration may lead to the unnecessary avoidance of reasonable risks. Research has found that people with low selfesteem are particularly susceptible to the fear of regret, and
consequently they tend to be overly risk averse in decision making [62,63]. Therefore, evaluations of health outcomes that are obtained from patients with low self-esteem may require some calibrations before they can be qualified as valid measurements of patient utility. Such calibrations would likely result in the upward adjustment of valuations given to health outcomes with reasonable risks. 4. Conclusion As self-reports, measures of patient utility are susceptible to the effects of cognitive biases. This article explored mechanisms through which cognitive processes of patients may bias these measures. Understanding these mechanisms is essential for improving the measures. This article identified problems in current measures of patient utility that have been mostly overlooked in the past. First, measures of patient utility demand that patients, who are under an enormous stress from their illness, perform complex mental operations, such as simulating hypothetical scenarios and understanding probabilistic information. Second, measures of patient utility do not elicit thoughtful response from patients by default. Third, measures of patient utility may be biased by the mood of patients. Fourth, patient utility is affected by both researchers’ choice of measurement instruments and patients’ choice of judgment strategies. Fifth, measures of patient utility tend to reflect the disproportionate influence of personal values that happen to be accessible from memory at the time of measurement. Sixth, patient utility is affected by patients’ fear of regret. Fortunately, most of these problems can be dealt with by improving the methods of administering currently available measures. Such improvements can be made through three means: improving the setting of administration, offering assistance to patients, and taking additional measures for calibration. First, researchers should be more careful about their choice of setting in which they administer measures of patient utility. It was suggested that researchers choose the time and setting that is: 1) least stressful to patients; 2) least distracting to patients, so that patients can devote their full attention to the task without time pressure; and 3) devoid, if possible, of stimuli that can prime certain personal values. Second, researchers should offer assistance to patients both before and during the measurement. It was suggested that researchers: 1) offer counseling to patients who are significantly stressed, before the measure is given; 2) encourage patients to consider as diverse personal values as possible before the measure; 3) let patients consider the possibility of their future regret; 4) encourage patients to be thoughtful in responding to the measure, by stressing the relevance of the measure to their welfare; and 5) offer assistance in simulating future scenarios. Third, under certain circumstances, researchers should take additional measures to calibrate the measures of patient
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
utility. When researchers suspect that patients’ mood or self-esteem is biasing measures of patient utility (e.g., a patient with low self-esteem who is in a bad mood), they should measure the respective variable and adjust the data on patient utility accordingly. More research is needed to develop a method for such calibrations. While it is true that researchers can deal with many problems associated with current measures of patient utility by improving the administration of available measures, they are also encouraged to develop better measures that are less taxing to patients. In particular, developing measures that appear less intimidating and in fact easier to process than currently available measures is needed. In developing alternative measures, however, it is important to recognize the fact that current measures of patient utility were not developed “bottom-up” from the field of clinical medicine. Instead, they were imported from other academic disciplines such as economics, decision science, and psychology, then applied “top-down” to the field of clinical medicine. In this sense, it is no surprise that these measures often fail to capture the actual preferences of patients in real clinical settings. Consequently, the development of alternative measures would benefit tremendously from the simultaneous development of clinimetrics, which focuses on the accurate and scientific description of uniquely clinical phenomena [64,65]. One problem, however, demands conceptual work. Patient utility is affected by both researchers’ choice of instruments and patients’ choice of strategies. That is because patient utility is judged by patients while it is simultaneously measured by researchers. Therefore, this problem cannot be addressed adequately unless researchers redefine patient utility as judged evaluation rather than retrieved evaluation. Such conceptual refinement will have profound implications for research on patient utility. One implication that was discussed in this article was that these “measures” are not passively measuring, but actively shaping, patient utility. Because the construct of patient utility cannot be divorced from the method of its measurement, in principle researchers cannot claim that they are comparing different measures for fidelity—an ability to yield the “best” estimate of “underlying” patient utility. In conclusion, though many problems surround current methods of measuring patient utility, equally many solutions exist that we as researchers can implement to improve the current situation. Measurement of patient utility is a basis for research on patient preference, which in turn is a basis for delivery of patient-centered medicine. It makes good sense that, to provide patient-centered care, we must also start providing measures that cater to patients’ cognitive needs.
Acknowledgments Supported by the Department of Veterans Affairs, Office of Academic Affiliations, in collaboration with the Health
475
Services Research and Development (HSR&D) Service, Associated Health Professions Post-Doctoral Fellowship Program in Health Services Research (Hanita).
References [1] Nease RF Jr., Kneeland T, O’Connor GT, Sumner W, Lumpkins C, Shaw L, Pryor D, Sox HC. Variation in patient utilities for outcomes of the management of chronic stable angina: Implications for clinical practice guidelines. JAMA 1995;273:1185–90. [2] Flood AB, Wennberg JE, Nease RF, Fowler FJ Jr., Ding J, Hynes LM. The importance of patient preference in the decision to screen for prostate cancer. J Gen Intern Med 1996;11:342–49. [3] Dunn VH, Pauker SG. An ambiguous renal cyst in a patient with a prior nephrectomy: what to do when there is only one more to lose. Med Decis Making 1986;6:49–62. [4] Zalkind DL, Shachtman RH. A decision analysis approach to the swine influenza vaccination decision for an individual. Med Care 1980;18:59–72. [5] Froberg DG, Kane RL. Methodologies for measuring health-state preferences—II: scaling methods. J Clin Epidemiol 1989;42:459–71. [6] von Neumann J, Morgenstern O. Theory of Games and Economic Behavior. New York: Wiley, 1953. [7] Kahneman D, Slovic P, Tversky A, editors. Judgment Under Uncertainty: Heuristics and Biases. New York: Cambridge University Press, 1982. [8] Arkes HR, Hammond KR, editors. Judgment and Decision Making: An Interdisciplinary Reader. Cambridge: Cambridge University Press, 1986. [9] Llewellyn-Thomas H, Sutherland HJ, Tibshirani R, Ciampi A, Till JE, Boyd NF. The measurement of patients’ values in medicine. Med Decis Making 1982;2:449–62. [10] Torrance GW, Thomas WH, Sackett DL. A utility maximization model for evaluation of health care programs. Health Serv Res 1972; 7:118–33. [11] Torrance GW, Boyle MH, Horwood SP. Application of multi-attribute utility theory to measure social preferences for health states. Operations Res 1982;30:1043–69. [12] Kahneman D, Tversky A. The simulation heuristic. In: Kahneman D, Slovic P, Tversky A, editors. Judgment Under Uncertainty: Heuristics and Biases. New York: Cambridge University Press, 1982. pp. 201–8. [13] Griffin DW, Dunning D, Ross L. The role of construal processes in overconfident predictions about the self and others. J Pers Soc Psychol 1990;59:1128–39. [14] Janis IL. Decisionmaking under stress. In: Goldberger L, Breznitz S, editors. Handbook of Stress: Theoretical and Clinical Aspects, 2nd ed. New York: Free Press; 1993. pp. 56–74. [15] Mann L. Stress, affect, and risk taking. In: Yates JF, editor. Risk-Taking Behavior. New York: Wiley; 1992. pp. 201–30. [16] Janis IL. Coping patterns among patients with life-threatening diseases. Issues Ment Health Nurs 1985;7:461–76. [17] Fiske ST, Taylor SE. Social Cognition: Second edition. New York: McGraw-Hill, 1991. [18] Jungermann H. The two camps on rationality. In: Scholz RW, editor. Decision Making Under Uncertainty. Amsterdam: Elsevier, 1983. pp. 63–86. [19] Langer EJ, Blank A, Chanowitz B. The mindlessness of ostensibly thoughtful action: the role of “placebic” information in interpersonal interaction. J Pers Soc Psychol 1978;36:635–42. [20] Langer EJ. Minding matters. In: Berkowitz L, Ed. Advances in Experimental Social Psychology, vol. 22. New York: Academic Press; 1989. pp. 137–73. [21] Fazio RH, Sanbonmatsu DM, Powell MC, Kardes FR. On the automatic activation of attitudes. J Pers Soc Psychol 1986;50:229–38.
476
M. Hanita / Journal of Clinical Epidemiology 53 (2000) 469–476
[22] Fazio RH. On the power and functionality of attitudes: The role of attitude accessibility. In: Pratkanis AR, Breckler SJ, Greenwald AG, editors. Attitude Structure and Function. Hillsdale, NJ: Erlbaum; 1989. pp. 153–79. [23] Kahneman D, Tversky A. Subjective probability: a judgment of representativeness. Cognit Psychol 1972;3:430–54. [24] Tversky A, Kahneman D. availability: A heuristic for judging frequency and probability. Cognit Psychol 1972;4:207–32. [25] Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science 1974;185:1124–31. [26] Stangor C, Duan C. Effects of multiple task demands upon memory for information about social groups. J Exp Soc Psychol 1991;27:357– 78. [27] Gilbert DT, Hixon JG. The trouble of thinking: activation and application of stereotypic beliefs. J Pers Soc Psychol 1991;60:509–17. [28] Kruglanski AW, Freund T. The freezing and unfreezing of lay-inferences: effects of impressional primacy, ethnic stereotyping, and numerical anchoring. J Exp Soc Psychol 1983;19:448–68. [29] Bodenhausen GV. Stereotypes as judgmental heuristics: evidence of circadian variations in discrimination. Psychol Sci 1990;1:319–22. [30] Isen AM. Toward understanding the role of affect in cognition. In: Wyer RS Jr., Srull TK, editors. Handbook of Social Cognition, vol. 3. Hillsdale, NJ: Erlbaum; 1984. pp. 179–236. [31] Isen AM, Shalker TE, Clark M, Karp L. Affect, accessibility of material in memory, and behavior: a cognitive loop? J Pers Soc Psychol 1978;36:1–12. [32] Isen AM, Geva N. The influence of positive affect on acceptable level of risk: the person with a large canoe had a large worry. Organ Behav Hum Decis Processes 1987;39:145–54. [33] Nygren TE, Isen AM, Taylor PJ, Dulin J. The influence of positive affect on the decision rule in risk situations: focus on outcome (and especially avoidance of loss) rather than probability. Organ Behav Hum Decis Processes 1996;66:59–72. [34] Giesler RB, Ashton CM, Brody B, Byrne MM, Cook K, Geraci JM, Hanita M, Souchek J, Wray NP. Assessing the performance of utility techniques in the absence of a gold standard. Med Care (in press). [35] Hornberger JC, Redelmeier DA, Petersen J. Variability among methods to assess patients’ well-being and consequent effect on a costeffectiveness analysis. J Clin Epidemiol 1992;45:505–12. [36] Read JL, Quinn RJ, Berwick DM, Fineberg HV, Weinstein MC. Preferences for health outcomes: comparison of assessment methods. Med Decis Making 1984;4:315–29. [37] Stiggelbout AM, Eijkemans MJC, Kiebert GM, Kievit J, Leer JWH, de Haes JCJM. The utility of the visual analog scale in medical decision making and technology assessment. Int J Technol Assess Health Care 1996;12:291–98. [38] Slovic P. The construction of preference. Am Psychol 1995;50:364– 71. [39] Tverksy A, Kahneman D. Rational choice and the framing of decisions. J Business 1986;59S:251–78. [40] McNeil BJ, Pauker SG, Sox HC, Tversky A. On the elicitation of preferences for alternative therapies. N Engl J Med 1982;306:1259– 62. [41] Tversky A, Sattath S, Slovic P. Contingent weighting in judgment and choice. Psychol Rev 1988;95:371–84. [42] Beach LR, Mitchell TR. A contingency model for the selection of decision strategies. Acad Manage Rev 1978;3:439–49.
[43] Payne JW, Bettman JR, Johnson EJ. Adaptive strategy selection in decision making. J Exp Psychol Learn Mem Cognit 1988;14:534–52. [44] Russo JE, Dosher BA. Strategies for multiattribute binary choice. J Exp Psychol Learn Mem Cognit 1983;9:676–96. [45] Simon HA. Rational choice and the structure of the environment. Psychol Rev 1956;63:129–38. [46] Payne JW, Bettman JR, Johnson EJ. Behavioral decision research: A constructive processing perspective. Annu Rev Psychol 1992;43:87– 131. [47] Johnson MD. Consumer choice strategies for comparing noncomparable alternatives. J Consumer Res 1984;11:741–53. [48] Mitchell TR, Beach LR. “. . . Do I love thee? Let me count . . .” Toward an understanding of intuitive and automatic decision making. Organ Behav Hum Decis Processes 1990;47:1–20. [49] Niedenthal PM, Cantor N, Kihlstrom JF. Prototype matching: a strategy for social decision making. J Pers Soc Psychol 1985;48:575–84. [50] Hammond KR, Hamm RM, Grassia J, Pearson T. Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment. System Man Cybernet IEEE Trans 1987;17:753–70. [51] Paese PW, Sniezek JA. Influences on the appropriateness of confidence in judgment: practice, effort, information, and decision-making. Organ Behav Hum Decis Processes 1991;48:100–30. [52] Hastie R, Park B. The relationship between memory and judgment depends on whether the judgment task is memory-based or on-line. Psychol Rev 1986;93:258–68. [53] Salancik G, Conway M. Attitude inferences from salient and relevant cognitive content about behavior. J Pers Soc Psychol 1975;32:829–40. [54] Tourangeau R, Rasinski KA, Bradburn N, D’Andrade R. Belief accessibility and context effects in attitude measurement. J Exp Soc Psychol 1989;25:401–21. [55] Christensen-Szalanski JJJ. Discount functions and the measurement of patients’ values. Med Decis Making 1984;4:47–58. [56] Higgins ET, Rholes WS, Jones CR. Category accessibility and impression formation. J Exp Soc Psychol 1977;13:141–54. [57] Srull TK, Wyer RS. Category accessibility and social perception: some implications for the study of person memory and interpersonal judgments. J Pers Soc Psychol 1980;38:841–56. [58] Bargh JA, Pietromonaco P. Automatic information processing and social perception: the influence of trait information presented outside of conscious awareness on impression formation. J Pers Soc Psychol 1982;43:437–49. [59] Bargh JA, Bond RN, Lombardi WL, Tota ME. The additive nature of chronic and temporary sources of construct accessibility. J Pers Soc Psychol 1986;50:869–79. [60] Bell DE. Regret in decision making under uncertainty. Operations Res 1982;30:961–81. [61] Loomes G, Sugden R. Regret theory: an alternative theory of rational choice under uncertainty. Econ J 1982;92:805–24. [62] Josephs RA, Larrick RP, Steele CM, Nisbett RE. Protecting the self from the negative consequences of risky decisions. J Pers Soc Psychol 1992;62:26–37. [63] Larrick RP. Motivational factors in decision theories: the role of selfprotection. Psychol Bull 1993;113:440–50. [64] Feinstein AR. An additional basic science for clinical medicine: IV. The development of clinimetrics. Ann Intern Med 1983;99:843–48. [65] Feinstein AR. Clinical judgment revisited: the distraction of quantitative models. Ann Intern Med 1994;120:799–805.