- Email: [email protected]
Decision Making at the Bedside: Lessons for Clinical Policy Development
Decision Making at the Bedside: Lessons for Clinical Policy Development WALLY R. SMITH, MD
H
erein I would like to review five common-sense lessons clinicians have learned from the bedside-lessons that should guide the development of clinical policy. Clinical policy and practice guidelines are being developed and enforced at the local, state, and national levels, often by nonphysicians. For example, payment is contingent on implicit guidelines in the prospective Medicare reimbursement system of Diagnostic Related Groups (DRGs). This enforcement focuses on the gross similarities of the care patients receive, such as length of hospital stay, rather than on individual differences such as severity of illness. The development of clinical guidelines by nonphysicians may reflect a general change in society's perception of the physician and of the health care system. Patients no longer see the physician as God-like, are more litigious, and are less willing to pay the price of today's health care. They are disappointed with the recent wave of technology that purports to remove the uncertainty associated with the practice of medicine. However, underneath such disappointment is a legitimate concern about medical care. Society wants to know how "good" a diagnostic or therapeutic technology is before it is broadly applied in medical practice, and how "good" a caregiver is by comparing his diagnostic and therapeutic behavior to a derived optimal strategy. These evaluations are sought by patients, hospitals, third party payers, and governments. The question they typically ask is, "What is the safest, most effective, cheapest, and most thorough care we can get for our money?" Unfortunately, physician resistance will not halt the public's invasion of the doctor-patient relationship and clinical policy making. Therefore, it is important that physicians participate in the process of the development of clinical policy-if we don't do it, others will. And others may fail to see the conflicting goals inherent in their quest for ideal care, and their From the Assistant Professor of Medicine and Epidemiology, Universityof Tennessee, Memphis, College of Medicine Correspondence: Wally R. Smith, MD, University of Tennessee College of Medicine, Memphis, TN. THE AMERICAN JOURNAL OF THE MEDICAL SCIENCES
need to decide which goals are preferable. Also, without physician participation, the resulting guidelines may be impossible to implement. Thus, five common-sense clinical lessons (Table 1) become important for the process of clinical policy development. The first of these is to consider all the options. Consider the plight of a local policy maker who must prepare guidelines for the management of ambulatory sickle cell anemia patients with low grade fever and chest complaints. Nearly all these patients have normal chest examinations. The local utilization review board has complained that several physicians empirically admit these patients unnecessarily. When questioned, these physicians justify empiric admission by noting that these patients are prone to infection and may not fare well if they are infected. The physicians worry that they must see a chest radiograph infiltrate to confirm the diagnosis of acute chest syndrome in these patients, for which the differential diagnosis (often difficult) is pneumonia or pulmonary embolus. They are unwilling to miss disease by relying on the chest film. Other physicians only admit patients if an infiltrate is found on the outpatient chest radiograph. If the radiograph is negative, they assume no chest disease and send the patients home. Finally, still other physicians empirically send these patients home with oral antibiotics. What should the policy be? As you consider your advice to this policy maker, consider also that Miller and others 1.2 have noted that the human brain can simultaneously entertain only 7 +/ - 2 alternate hypotheses or concepts at a time. However, some clinical scenarios might generate up to 100 conceivable differential diagnoses, although not all equally probable, and each diagnosis might have a number of testing strategies and/or therapeutic options associated with it. How then does the policy maker properly consider all the hypotheses and strategies associated with handling these patients? For this situation, a tool that illustrates the application of the five common sense lessons to this policy dilemma is decision analysis, a systematic approach to decision making under conditions of uncertainty.3 181
Decision Making at the Bedside
Table 1. Lessons for Clinical Policy Development I. II. III. IV. V.
result (for example, there may be invisible microvascular pulmonary embolism). Finally, if he elects the empiric admission policy, the admissions may not be justified by later testing or by the patients' clinical course. The notation used to display these chance events is a chance node, which denotes a point in time at which one of several possible events beyond the control of the decision maker may take place. 5 Another way to look at this policy decision is the threshold model (Figure 2), which graphs the prevalence of the disorder (or any variable in the decision) through a range of 0-100%.6 When the policy maker applies his three policies to this model, he sees that at low prevalences he would favor the policy of doing nothing, or in this case sending patients home with oral antibiotics. At medium prevalences, he would favor testing to see if there is disease or, in this case, performing a chest radiograph and basing his actions on the results. Finally, at high prevalences, he would favor treating empirically or, in this case, admitting patients for empiric therapy. An important practical lesson he learns here is that if patients may have a life-threatening disease, there is a lower threshold at which he should favor testing or treating than if patients may have an insignificant disease. Also, he learns that he should not favor performing a test for diagnostic purposes if physicians plan either empiric treatment or no treatment of the target disease. AI-
Consider all the options. Think before you act. Determine which outcomes matter. Use accurate data. Involve patients in policies.
A decision tree (Fig. 1) is a way of displaying the temporal and logical sequence of the clinical decision problem. 4 •5 A decision node denotes a point in time at which the decision maker can elect one of several alternative courses of action. For these patients, the policy maker can simplify the problem to a choice of three alternative policies: home (send patients home with or without oral antibiotics), radiograph (perform a chest radiograph and only if an infiltrate is found admit patients), and admit (worry about radiographs later, but begin inpatient treatment with intravenous fluids and antibiotics). The second common -sense lesson is to think before you act. Each of the three policies has associated with it attendant consequences. If the policy maker elects the home policy, physicians may send some patients with acute chest syndrome home. If he elects the radiograph policy, the films mayor may not show an infiltrate. The finding of an infiltrate may be a false positive or true positive result. Likewise, the absence of an infiltrate may be a false negative or true negative
r--"Or;.;;o"""b,;... . ....:;0..;.;.2;...4'--_-f Home, Acute Chest utility .30
Home
' -_ _ _ _ _ _--1 Home, No Acute
0.83
prob. 0.76
Chest utility
.---_~_I
prob. 0.27
P
.. oSltlve I -_ _ _
SS Chest c/o Xray (0.89)
X-ray 0.89
1.0
X-ray Positive, Acute Chest utility .60
IX-ray Positive, No Acute Chest utility .90
r=~~=-!X-ray
b pro.
o. 73 Negative
Negative, Acute Chest utility .30
L....:-----:~_f X-ray Negative,
No Acute Chest utility 1.0
o
Chance node
o
Decision node
~
I -_ _ _ _ _ _ _~
._-..:....p_ro_b_._0_.2_4_ _41Admit, Acute ~.C~h~e~s~t_ _ _~ utility .60
_ _ _ _ _~I
Admit 0.83
' - - - - - - - - 1 Admit, No Acute
prob. 0.76
Chest utility
.90
Figure 1. Decision tree for sickle cell patients with low-grade fever and chest complaints.
182
September 1990 Volume 300 Number 3
Smith
No Rx
III ~f~~ IIII
o (0%) Probability
r
Tt
No
Rx
Acute Chest
of Disease (Prevalence) 1.0 (100%)
XRay+~a+b
i
XRay,~c+d
Ttrx
Total
Figure 2. Threshold model of clinical decision making (Tt = test threshold; Ttrx = test/treatment threshold).
• pAcCh Plug in prob.'s
though testing might be indicated for prognostic or other reasons, in either case physicians would not change their management based on the test result. We will return to this model as it relates to the example later. The third common-sense lesson is to determine which outcomes matter. To quote August Bier (18611949): "It is better to cure people than to make diagnoses.,,7 For this example, the outcomes that matter are whether or not the patient has acute chest syndrome, whether or not chest radiographs are positive, and whether or not the physician admits the patient. The combinations of those circumstances, as diagrammed in the decision tree, lead to eight different valued outcomes. Reasonable outcome states include mortality, morbidity, survival time, quality oflife, and cost. The fourth common-sense lesson is to use accurate data in making policy. In decision analysis jargon, that means the policy maker must specify the probabilities of chance events, such as test results, surgical mortalities, and prevalence of disease. In the absence of data from clinical trials, the subjective judgments of experts can be elicited. Clinical policy makers may begin to rely more on electronic retrieval of such information as it is demanded by patients and their surrogates. One way to represent information contained in diagnostic tests is to use 2 by 2 tables (Figure 3). For the radiograph/acute chest syndrome problem, the policy maker needs to know four probabilities the table can provide, once he has filled it with pertinent information. First, he sets up the table showing four possibilities: the radiograph may be positive or negative, and in either case, there mayor may not be acute chest Table 2. Patient's Utilities for Various Outcomes in the Decision Tree (Highest Number = Best). X·ray negative, acute chest (falsely reassured) Home, acute chest (home with disease) X·ray positive, acute chest (diagnosed then treated) Admit, acute chest (treated disease) X-ray positive, no acute chest (falsely labeled) Admit, no acute chest (admitted unnecessarily) X-ray negative, no acute chest (correctly reassured) Home, no acute chest (home with no disease)
THE AMERICAN JOURNAL OF THE MEDICAL SCIENCES
Acute Chest Total
0.3 0.3 0.6 0.6 0.9 0.9 1 1
a+c
b+d
.24 (Prevalence, a+c/a+b+c+d)
Ne AcCh.02 (c/c+d) p g [ • pPosAcCh .86 (Positive Pred. value, a/a+b) • pXRPos .27 (a+b/a+b+c+d) •
Figure 3. A 2 X 2 table and resultant probabilities for the radio· graph in acute chest syndrome. p = probability; AcCh = acute chest syndrome; Neg = negative chest radiograph; Pos = positive chest radiograph.
syndrome. Then, of the total of observed patients, he notes how many fall into each possibility. He may then calculate to find the total number of positive radiographs, the total cases of acute chest syndrome, and describe the radiograph's performance in these patients. The first probability needed for the decision tree is the probability or prevalence of acute chest syndrome. From the literature, the policy maker uses 24% or 0.24.8 Next, he needs the probability that if the chest radiograph is positive, the patient will have acute chest syndrome, also called the positive predictive value. He uses 86% or 0.86, based on his 2 by 2 table and a physician-estimated radiograph sensitivity and specificity of95%. Next, he needs the probability that if the chest radiograph is negative, the patient will have the acute chest syndrome (0.02), and the probability that a radiograph will be positive (0.27). The last common -sense lesson is to involve patients in policy about their care. By asking for utilities, which are values placed on various outcome states, depNegAcCh 0.81 0.61 0.41 0.21 0.01 L.::::::::::......tt4!.~~~~::.:::::.:::;~~~;:;.;::.;.:;;;~~........~;;,;;; 0.00 0.80 1.00 0.40 0.60 0.20 pAcCh Figure 4. Graph of expected utility thresholds for policies Home, X-ray, and Admit. pNegAcCh = probability of acute chest syn· drome, given a negative chest radiograph. pAcCh = probability (prevalence) of acute chest syndrome.
183
Decision Making at the Bedside
cision analysis forces policy makers to decide whose values to incorporate into clinical policy. Utilities may be costs, mortality, quality-adjusted life-years, or any outcome state with variable value. Values often must be assigned to outcome states such as being crippled, blind, etc. The policy maker in this case elects to use the patie"nts' values and assigns values to each outcome based on patient interviews using a maximum utility of 1 and a minimum of 0 (Table 2). The higher the utility, the more preferable the outcome. The least preferable outcome (utility = 0.3), then, is to be wrongly sent home with untreated disease because of a false-negative chest radiograph. The most preferable outcome (utility = 1) is to be sent home with no testing, no admission, and no disease. The policy maker can now solve the decision tree to determine which policy (home, radiograph, or admit) is associated with the highest expected utility, on average. Although this leaves uncertainty for the outcome of any specific patient, it maximally reduces that uncertainty. On average, if he sets this policy, patients will get the "best" care (in this case, the care they most want). Solving the tree is done by a process of addition and multiplication called averaging out and folding back. 5 As constructed, the tree suggests the highest expected utility comes from the radiograph policy. The policy maker is relieved that many of the physicians are comfortable with that policy. Intuitively, they guessed that getting a chest radiograph was the best policy, since it is a relatively cheap, harmless, and very good test. But the empiric admittance proponents argue that they are more than 50% sure, not 24 % sure, of the presence of acute chest syndrome in the patients they admit empirically. When they are less sure, they don't admit anyway. They think 95% is an overestimate of the sensitivity of the chest radiograph. Does the decision tree differ with their intuition? Would substituting their estimates for those in the decision tree change the recommended policy? First, the threshold model of this policy shows the policy maker the point at which he would change policies, the no-test/test threshold, is at approximately 30% prevalence. There is no test/treat threshold. Above 30% prevalence, all other factors being equal, he would always favor obtaining a chest radiograph before deciding to admit. Next, a sensitivity analysis shows the policy maker how the highest expected utilities associated with each of the three policies are sensitive to prevalence (actual or estimated) and to the performance of the chest radiograph. Figure 4 is a graph of prevalence or the subjective probability of the acute chest syndrome (pAcCh) vs. the probability of the acute chest syndrome given a negative chest radiograph (pnegAcCh). Each shaded area suggests pursuing one of the three
184
policies, given the associated pAcCh and pnegAcCh. At a pAcCh of 50%, the radiograph policy predominates when pnegAcCh is lower than about 30%. But above the pnegAcCh of 30% (reflecting the poorer radiograph performance suggested by the empiric admittance proponents), empiric admission becomes the predominate policy. With higher pAcChs, one may have poorer radiograph performance (higher pN egAcChs) and still be justified in ordering the film. At the baseline pAcCh of 24 % and pnegAcCh of 0.02, the radiograph policy predominates. Significantly, regardless of the pNegAcCh, the home policy predominates if the pAcCh is lower than about 20%. Thus, under this policy, neither the empiric admittance proponents nor the physicians who radiograph or send patients home may be wrong, as long as each of their subjective probabilities falls within the correct boundaries. What one need learn from this discussion is not how to do decision analysis. Construction and analysis of decision trees may require several man-hours of effort and is not practical for a busy clinician. Rather, one should learn some important lessons for making good clinical policy. Policy makers must consider all the options, think before acting, use accurate data, determine which outcomes matter, and involve patients in the policy. Clinical policy must have broad applicability but be flexible enough to meet the unique needs of every patient. Clinical policy must also allow for uncertainty ("To be uncertain is to be uncomfortable, but to be certain is to be ridiculous"-Ancient Chinese Proverb).7 Of course, clinical policy must reflect an attempt to reduce uncertainty about the meaning of test results or the success of therapies. Statements of policy should imply evaluation of a test's or therapy's performance before its wide application to clinical practice. Such evaluations are being sought by a society that may be tiring of new technology that fails to yield better outcomes than the old. References 1. Miller GA: The Magical Number Seven, Plus or Minus Two:
2. 3. 4. 5. 6. 7. 8.
Some Limits in Our Capacity for Processing Information. Psych Rev. 1979; 63:81-97. Elstein AS, Shulman LS, Sprafka SA: Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge, Mass., Harvard University Press, 1978. Raiffa, H: The Foundations of Decision Analysis. IEEE Trans SSC 1968; 4:211-219. Raiffa, H: Decision Analysis: Introductory Lectures on Choices Under Uncertainty. Reading, Mass., Addison-Wesley Publishing Co., Inc., 1968. Weinstein MC, Fineberg HV: Clinical Decision Analysis. Philadelphia, Pa., W.B. Saunders Co., 1980. Pauker SG, Kassirer JP: The Threshold Approach to Clinical Decision Making. N Engl J Med 302:1980,1109-1117. Strauss MB: Familiar Medical Quotations. Boston, Little, Brown, and Co., 1968. Serjeant GR: The Clinical Features of Sickle Cell Disease. Amsterdam, North Holland Publishing Co., 1974.
September 1990 Volume 300 Number 3
H
erein I would like to review five common-sense lessons clinicians have learned from the bedside-lessons that should guide the development of clinical policy. Clinical policy and practice guidelines are being developed and enforced at the local, state, and national levels, often by nonphysicians. For example, payment is contingent on implicit guidelines in the prospective Medicare reimbursement system of Diagnostic Related Groups (DRGs). This enforcement focuses on the gross similarities of the care patients receive, such as length of hospital stay, rather than on individual differences such as severity of illness. The development of clinical guidelines by nonphysicians may reflect a general change in society's perception of the physician and of the health care system. Patients no longer see the physician as God-like, are more litigious, and are less willing to pay the price of today's health care. They are disappointed with the recent wave of technology that purports to remove the uncertainty associated with the practice of medicine. However, underneath such disappointment is a legitimate concern about medical care. Society wants to know how "good" a diagnostic or therapeutic technology is before it is broadly applied in medical practice, and how "good" a caregiver is by comparing his diagnostic and therapeutic behavior to a derived optimal strategy. These evaluations are sought by patients, hospitals, third party payers, and governments. The question they typically ask is, "What is the safest, most effective, cheapest, and most thorough care we can get for our money?" Unfortunately, physician resistance will not halt the public's invasion of the doctor-patient relationship and clinical policy making. Therefore, it is important that physicians participate in the process of the development of clinical policy-if we don't do it, others will. And others may fail to see the conflicting goals inherent in their quest for ideal care, and their From the Assistant Professor of Medicine and Epidemiology, Universityof Tennessee, Memphis, College of Medicine Correspondence: Wally R. Smith, MD, University of Tennessee College of Medicine, Memphis, TN. THE AMERICAN JOURNAL OF THE MEDICAL SCIENCES
need to decide which goals are preferable. Also, without physician participation, the resulting guidelines may be impossible to implement. Thus, five common-sense clinical lessons (Table 1) become important for the process of clinical policy development. The first of these is to consider all the options. Consider the plight of a local policy maker who must prepare guidelines for the management of ambulatory sickle cell anemia patients with low grade fever and chest complaints. Nearly all these patients have normal chest examinations. The local utilization review board has complained that several physicians empirically admit these patients unnecessarily. When questioned, these physicians justify empiric admission by noting that these patients are prone to infection and may not fare well if they are infected. The physicians worry that they must see a chest radiograph infiltrate to confirm the diagnosis of acute chest syndrome in these patients, for which the differential diagnosis (often difficult) is pneumonia or pulmonary embolus. They are unwilling to miss disease by relying on the chest film. Other physicians only admit patients if an infiltrate is found on the outpatient chest radiograph. If the radiograph is negative, they assume no chest disease and send the patients home. Finally, still other physicians empirically send these patients home with oral antibiotics. What should the policy be? As you consider your advice to this policy maker, consider also that Miller and others 1.2 have noted that the human brain can simultaneously entertain only 7 +/ - 2 alternate hypotheses or concepts at a time. However, some clinical scenarios might generate up to 100 conceivable differential diagnoses, although not all equally probable, and each diagnosis might have a number of testing strategies and/or therapeutic options associated with it. How then does the policy maker properly consider all the hypotheses and strategies associated with handling these patients? For this situation, a tool that illustrates the application of the five common sense lessons to this policy dilemma is decision analysis, a systematic approach to decision making under conditions of uncertainty.3 181
Decision Making at the Bedside
Table 1. Lessons for Clinical Policy Development I. II. III. IV. V.
result (for example, there may be invisible microvascular pulmonary embolism). Finally, if he elects the empiric admission policy, the admissions may not be justified by later testing or by the patients' clinical course. The notation used to display these chance events is a chance node, which denotes a point in time at which one of several possible events beyond the control of the decision maker may take place. 5 Another way to look at this policy decision is the threshold model (Figure 2), which graphs the prevalence of the disorder (or any variable in the decision) through a range of 0-100%.6 When the policy maker applies his three policies to this model, he sees that at low prevalences he would favor the policy of doing nothing, or in this case sending patients home with oral antibiotics. At medium prevalences, he would favor testing to see if there is disease or, in this case, performing a chest radiograph and basing his actions on the results. Finally, at high prevalences, he would favor treating empirically or, in this case, admitting patients for empiric therapy. An important practical lesson he learns here is that if patients may have a life-threatening disease, there is a lower threshold at which he should favor testing or treating than if patients may have an insignificant disease. Also, he learns that he should not favor performing a test for diagnostic purposes if physicians plan either empiric treatment or no treatment of the target disease. AI-
Consider all the options. Think before you act. Determine which outcomes matter. Use accurate data. Involve patients in policies.
A decision tree (Fig. 1) is a way of displaying the temporal and logical sequence of the clinical decision problem. 4 •5 A decision node denotes a point in time at which the decision maker can elect one of several alternative courses of action. For these patients, the policy maker can simplify the problem to a choice of three alternative policies: home (send patients home with or without oral antibiotics), radiograph (perform a chest radiograph and only if an infiltrate is found admit patients), and admit (worry about radiographs later, but begin inpatient treatment with intravenous fluids and antibiotics). The second common -sense lesson is to think before you act. Each of the three policies has associated with it attendant consequences. If the policy maker elects the home policy, physicians may send some patients with acute chest syndrome home. If he elects the radiograph policy, the films mayor may not show an infiltrate. The finding of an infiltrate may be a false positive or true positive result. Likewise, the absence of an infiltrate may be a false negative or true negative
r--"Or;.;;o"""b,;... . ....:;0..;.;.2;...4'--_-f Home, Acute Chest utility .30
Home
' -_ _ _ _ _ _--1 Home, No Acute
0.83
prob. 0.76
Chest utility
.---_~_I
prob. 0.27
P
.. oSltlve I -_ _ _
SS Chest c/o Xray (0.89)
X-ray 0.89
1.0
X-ray Positive, Acute Chest utility .60
IX-ray Positive, No Acute Chest utility .90
r=~~=-!X-ray
b pro.
o. 73 Negative
Negative, Acute Chest utility .30
L....:-----:~_f X-ray Negative,
No Acute Chest utility 1.0
o
Chance node
o
Decision node
~
I -_ _ _ _ _ _ _~
._-..:....p_ro_b_._0_.2_4_ _41Admit, Acute ~.C~h~e~s~t_ _ _~ utility .60
_ _ _ _ _~I
Admit 0.83
' - - - - - - - - 1 Admit, No Acute
prob. 0.76
Chest utility
.90
Figure 1. Decision tree for sickle cell patients with low-grade fever and chest complaints.
182
September 1990 Volume 300 Number 3
Smith
No Rx
III ~f~~ IIII
o (0%) Probability
r
Tt
No
Rx
Acute Chest
of Disease (Prevalence) 1.0 (100%)
XRay+~a+b
i
XRay,~c+d
Ttrx
Total
Figure 2. Threshold model of clinical decision making (Tt = test threshold; Ttrx = test/treatment threshold).
• pAcCh Plug in prob.'s
though testing might be indicated for prognostic or other reasons, in either case physicians would not change their management based on the test result. We will return to this model as it relates to the example later. The third common-sense lesson is to determine which outcomes matter. To quote August Bier (18611949): "It is better to cure people than to make diagnoses.,,7 For this example, the outcomes that matter are whether or not the patient has acute chest syndrome, whether or not chest radiographs are positive, and whether or not the physician admits the patient. The combinations of those circumstances, as diagrammed in the decision tree, lead to eight different valued outcomes. Reasonable outcome states include mortality, morbidity, survival time, quality oflife, and cost. The fourth common-sense lesson is to use accurate data in making policy. In decision analysis jargon, that means the policy maker must specify the probabilities of chance events, such as test results, surgical mortalities, and prevalence of disease. In the absence of data from clinical trials, the subjective judgments of experts can be elicited. Clinical policy makers may begin to rely more on electronic retrieval of such information as it is demanded by patients and their surrogates. One way to represent information contained in diagnostic tests is to use 2 by 2 tables (Figure 3). For the radiograph/acute chest syndrome problem, the policy maker needs to know four probabilities the table can provide, once he has filled it with pertinent information. First, he sets up the table showing four possibilities: the radiograph may be positive or negative, and in either case, there mayor may not be acute chest Table 2. Patient's Utilities for Various Outcomes in the Decision Tree (Highest Number = Best). X·ray negative, acute chest (falsely reassured) Home, acute chest (home with disease) X·ray positive, acute chest (diagnosed then treated) Admit, acute chest (treated disease) X-ray positive, no acute chest (falsely labeled) Admit, no acute chest (admitted unnecessarily) X-ray negative, no acute chest (correctly reassured) Home, no acute chest (home with no disease)
THE AMERICAN JOURNAL OF THE MEDICAL SCIENCES
Acute Chest Total
0.3 0.3 0.6 0.6 0.9 0.9 1 1
a+c
b+d
.24 (Prevalence, a+c/a+b+c+d)
Ne AcCh.02 (c/c+d) p g [ • pPosAcCh .86 (Positive Pred. value, a/a+b) • pXRPos .27 (a+b/a+b+c+d) •
Figure 3. A 2 X 2 table and resultant probabilities for the radio· graph in acute chest syndrome. p = probability; AcCh = acute chest syndrome; Neg = negative chest radiograph; Pos = positive chest radiograph.
syndrome. Then, of the total of observed patients, he notes how many fall into each possibility. He may then calculate to find the total number of positive radiographs, the total cases of acute chest syndrome, and describe the radiograph's performance in these patients. The first probability needed for the decision tree is the probability or prevalence of acute chest syndrome. From the literature, the policy maker uses 24% or 0.24.8 Next, he needs the probability that if the chest radiograph is positive, the patient will have acute chest syndrome, also called the positive predictive value. He uses 86% or 0.86, based on his 2 by 2 table and a physician-estimated radiograph sensitivity and specificity of95%. Next, he needs the probability that if the chest radiograph is negative, the patient will have the acute chest syndrome (0.02), and the probability that a radiograph will be positive (0.27). The last common -sense lesson is to involve patients in policy about their care. By asking for utilities, which are values placed on various outcome states, depNegAcCh 0.81 0.61 0.41 0.21 0.01 L.::::::::::......tt4!.~~~~::.:::::.:::;~~~;:;.;::.;.:;;;~~........~;;,;;; 0.00 0.80 1.00 0.40 0.60 0.20 pAcCh Figure 4. Graph of expected utility thresholds for policies Home, X-ray, and Admit. pNegAcCh = probability of acute chest syn· drome, given a negative chest radiograph. pAcCh = probability (prevalence) of acute chest syndrome.
183
Decision Making at the Bedside
cision analysis forces policy makers to decide whose values to incorporate into clinical policy. Utilities may be costs, mortality, quality-adjusted life-years, or any outcome state with variable value. Values often must be assigned to outcome states such as being crippled, blind, etc. The policy maker in this case elects to use the patie"nts' values and assigns values to each outcome based on patient interviews using a maximum utility of 1 and a minimum of 0 (Table 2). The higher the utility, the more preferable the outcome. The least preferable outcome (utility = 0.3), then, is to be wrongly sent home with untreated disease because of a false-negative chest radiograph. The most preferable outcome (utility = 1) is to be sent home with no testing, no admission, and no disease. The policy maker can now solve the decision tree to determine which policy (home, radiograph, or admit) is associated with the highest expected utility, on average. Although this leaves uncertainty for the outcome of any specific patient, it maximally reduces that uncertainty. On average, if he sets this policy, patients will get the "best" care (in this case, the care they most want). Solving the tree is done by a process of addition and multiplication called averaging out and folding back. 5 As constructed, the tree suggests the highest expected utility comes from the radiograph policy. The policy maker is relieved that many of the physicians are comfortable with that policy. Intuitively, they guessed that getting a chest radiograph was the best policy, since it is a relatively cheap, harmless, and very good test. But the empiric admittance proponents argue that they are more than 50% sure, not 24 % sure, of the presence of acute chest syndrome in the patients they admit empirically. When they are less sure, they don't admit anyway. They think 95% is an overestimate of the sensitivity of the chest radiograph. Does the decision tree differ with their intuition? Would substituting their estimates for those in the decision tree change the recommended policy? First, the threshold model of this policy shows the policy maker the point at which he would change policies, the no-test/test threshold, is at approximately 30% prevalence. There is no test/treat threshold. Above 30% prevalence, all other factors being equal, he would always favor obtaining a chest radiograph before deciding to admit. Next, a sensitivity analysis shows the policy maker how the highest expected utilities associated with each of the three policies are sensitive to prevalence (actual or estimated) and to the performance of the chest radiograph. Figure 4 is a graph of prevalence or the subjective probability of the acute chest syndrome (pAcCh) vs. the probability of the acute chest syndrome given a negative chest radiograph (pnegAcCh). Each shaded area suggests pursuing one of the three
184
policies, given the associated pAcCh and pnegAcCh. At a pAcCh of 50%, the radiograph policy predominates when pnegAcCh is lower than about 30%. But above the pnegAcCh of 30% (reflecting the poorer radiograph performance suggested by the empiric admittance proponents), empiric admission becomes the predominate policy. With higher pAcChs, one may have poorer radiograph performance (higher pN egAcChs) and still be justified in ordering the film. At the baseline pAcCh of 24 % and pnegAcCh of 0.02, the radiograph policy predominates. Significantly, regardless of the pNegAcCh, the home policy predominates if the pAcCh is lower than about 20%. Thus, under this policy, neither the empiric admittance proponents nor the physicians who radiograph or send patients home may be wrong, as long as each of their subjective probabilities falls within the correct boundaries. What one need learn from this discussion is not how to do decision analysis. Construction and analysis of decision trees may require several man-hours of effort and is not practical for a busy clinician. Rather, one should learn some important lessons for making good clinical policy. Policy makers must consider all the options, think before acting, use accurate data, determine which outcomes matter, and involve patients in the policy. Clinical policy must have broad applicability but be flexible enough to meet the unique needs of every patient. Clinical policy must also allow for uncertainty ("To be uncertain is to be uncomfortable, but to be certain is to be ridiculous"-Ancient Chinese Proverb).7 Of course, clinical policy must reflect an attempt to reduce uncertainty about the meaning of test results or the success of therapies. Statements of policy should imply evaluation of a test's or therapy's performance before its wide application to clinical practice. Such evaluations are being sought by a society that may be tiring of new technology that fails to yield better outcomes than the old. References 1. Miller GA: The Magical Number Seven, Plus or Minus Two:
2. 3. 4. 5. 6. 7. 8.
Some Limits in Our Capacity for Processing Information. Psych Rev. 1979; 63:81-97. Elstein AS, Shulman LS, Sprafka SA: Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge, Mass., Harvard University Press, 1978. Raiffa, H: The Foundations of Decision Analysis. IEEE Trans SSC 1968; 4:211-219. Raiffa, H: Decision Analysis: Introductory Lectures on Choices Under Uncertainty. Reading, Mass., Addison-Wesley Publishing Co., Inc., 1968. Weinstein MC, Fineberg HV: Clinical Decision Analysis. Philadelphia, Pa., W.B. Saunders Co., 1980. Pauker SG, Kassirer JP: The Threshold Approach to Clinical Decision Making. N Engl J Med 302:1980,1109-1117. Strauss MB: Familiar Medical Quotations. Boston, Little, Brown, and Co., 1968. Serjeant GR: The Clinical Features of Sickle Cell Disease. Amsterdam, North Holland Publishing Co., 1974.
September 1990 Volume 300 Number 3