- Email: [email protected]
Clinical prediction rules for the diagnosis of pulmonary embolism
EDITORIAL
Clinical Prediction Rules for the Diagnosis of Pulmonary Embolism Gabriela Ferreira, MD, Jeffrey L. Carson, MD
D
iagnosing pulmonary embolism is as challenging and controversial today as it was 25 years ago when lung scanning became widely implemented. In 1976, McNeil suggested that ventilation-perfusion lung scans in patients with suspected pulmonary embolism should be classified as normal, or low, high, or “indeterminate” or intermediate probability (1). Diagnosis is confirmed by a high-probability scan and is unlikely in patients with low-probability or normal scans. Patients with indeterminate scans need a pulmonary angiogram to establish disease. Although some authors argued that there were inadequate data to justify the widespread use of lung scans (2), this diagnostic algorithm was widely accepted and is at times used erroneously today. Two prospective series have defined the test characteristics of lung scans (3,4), showing that patients with high-probability scans had about an 87% probability of pulmonary embolism, compared with one third of patients with intermediate-probability scans, and 14% to 31% of those with low-probability scans. Missing 14% to 30% of patients with this potentially lethal disease is unacceptable even if we accept the most favorable result. For this reason, low- and intermediate-probability scans are now designated as nondiagnostic. Because pulmonary embolism is likely to originate in the deep veins of the lower extremities, subsequent albeit imperfect diagnostic strategies included testing for lower extremity venous thrombosis (5). There was much hope that computed tomography would provide more reliable information. Studies show that it is useful in diagnosing other conditions that may be confused with pulmonary embolism, and that it performs adequately to exclude or confirm pulmonary embolism in patients with large central emboli but misses pulmonary embolism in smaller pulmonary arteries (6). Magnetic resonance angiography has also been found to miss pulmonary embolism in smaller pulmonary arteries, but it can be used in patients with contraindications to
Am J Med. 2002;113:337–338. From the Division of General Internal Medicine, Department of Medicine, University of Medicine and Dentistry of New Jersey, Robert Wood Johnson Medical School, New Brunswick, New Jersey. Requests for reprints should be addressed to Jeffrey L. Carson, MD, University of Medicine and Dentistry of New Jersey, Robert Wood Johnson Medical School, 125 Paterson Street, New Brunswick, New Jersey 08903, or [email protected]. ©2002 by Excerpta Medica, Inc. All rights reserved.
contrast material (7). Echocardiography has been suggested as a diagnostic tool but awaits further study (8). The most promising new test is the D-dimer test, which has excellent sensitivity, especially if using enzyme-linked immunoassays. Most D-dimer assays are effective in excluding thromboembolism in outpatients, particularly when combined with a clinical picture consistent with low probability of disease (9 –11). The test should not be used in hospitalized patients, because the prevalence of pulmonary embolism is higher in this group, and in other conditions where test results are likely to be abnormal, because of poor specificity. A fundamental principle of diagnostic testing is to consider the likelihood of disease before testing (pretest probability) and to combine it with test results to obtain a post-test probability of disease. Investigators in the Prospective Investigation Of Pulmonary Embolism Diagnosis study demonstrated that experienced clinicians were able to classify patients with suspected pulmonary embolism with some precision based on clinical assessment (3): 9% in the low clinical pretest probability group, 30% in the intermediate group, and 68% in the high-probability group. When the clinical assessment was combined with ventilation-perfusion lung scanning, the probability of pulmonary embolism was refined further. If the clinical assessment and lung scan were concordant, pulmonary embolism was either highly likely (96%) or very unlikely (2%). However, when there was discordance between clinical assessment and lung scan results, the probability was 45% to 56%, and further testing was required. It is unclear if less experienced or nonspecialist physicians would be as successful in predicting pulmonary embolism because the probability estimate is based on clinical impression rather than specific criteria. Hence, explicit prediction rules were developed. In this issue of the Journal, Chagnon et al. (12) compare two such rules—the Wells’ rule and the Geneva rule (13,14). The Wells’ prediction rule was derived from a cohort of consecutive outpatients and inpatients with suspected pulmonary embolism (13). Seven variables associated with pulmonary embolism from a logistic regression model were assigned point values. Patients were classified as having low, intermediate, or high clinical probability (pretest probability) of pulmonary embolism based on point totals. These assessments were predictive of pulmonary embolism in their algorithm; however, their diag0002-9343/02/$–see front matter 337 PII S0002-9343(02)00000-0
Clinical Prediction Rules for the Diagnosis of Pulmonary Embolism/Ferreira and Carson
nostic algorithm was criticized for being too complicated and the derived clinical score for being too subjective for requiring clinicians to decide if an alternate diagnosis was less likely than pulmonary embolism. The Geneva rule was developed in consecutive outpatients who presented to an emergency department with suspected pulmonary embolism (14). It uses objective data only with clinical assessment override if there is disagreement. Eight predictors were found on multivariate analysis to be significantly associated with pulmonary embolism; none were subjective. Ten percent of patients in the low-probability, 38% of those in the intermediateprobability, and 81% of those in the high-probability group had pulmonary embolism (14). Chagnon et al. found that similar proportions of patients with low, intermediate, and high pretest probability had pulmonary embolism (12). Receiver operating characteristic curve analysis (15,16) showed moderate diagnostic performance and no significant difference between the two prediction rules: 0.74 for the Geneva score and 0.78 for the Wells’ score, and concordance was fair between the two rules ( coefficient ⫽ 0.43). However, when implicit evaluation was added to the Geneva score, the area under the curve was significantly better (0.82) than either score alone, suggesting that clinical judgment refines the probability beyond what is learned from the score alone. Interestingly, the criteria reported by clinicians in the Geneva study that informed their implicit assessment and led them to override the rule were very similar to the Wells’ criteria. Prediction rules should help clinicians to triage patients into clinically useful groups to avoid unnecessary testing, while minimizing risk. When combined with a negative D-dimer test, these rules exclude pulmonary embolism in outpatients with low pretest probability. In other patients, such as those with intermediate or high clinical probability, or inpatients, further testing is required. However, it is not clear that the explicit prediction rules offer enough of an advantage over empiric assessment that they are worth committing to memory or recording in a personal digital assistant (PDA). The authors suggest, and we agree, that the value of prediction rules may include training of junior physicians and resolution of disagreements among more experienced physicians using implicit evaluation. Clinicians should recognize that these rules only work in settings with similar disease prevalence, and that they are only useful if combined with the results of noninvasive tests that further refine the probability of disease because they do not adequately define the presence or absence of disease with enough precision to be used on their own. Many prediction rules have been developed and published, but few are used.
338
September 2002
THE AMERICAN JOURNAL OF MEDICINE威
Perhaps this will change with the ability to store complicated rules in the PDAs that are carried by many physicians. However, we are skeptical that these rules will be widely adopted.
REFERENCES 1. McNeil BJ. A diagnostic strategy using ventilation-perfusion studies in patients suspect for pulmonary embolism. J Nucl Med. 1976; 17:613–616. 2. Robin ED. Overdiagnosis and overtreatment of pulmonary embolism: the emperor may have no clothes. Ann Intern Med. 1977; 87:775–781. 3. The PIOPED Investigators. Value of the ventilation/perfusion scan in acute pulmonary embolism. Results of the Prospective Investigation Of Pulmonary Embolism Diagnosis (PIOPED. JAMA. 1990; 263:2753–2759. 4. Hull RD, Hirsh J, Carter CJ, et al. Diagnostic value of ventilationperfusion lung scanning in patients with suspected pulmonary embolism. Chest. 1985;88:819 –828. 5. Hull RD, Raskob GE, Coates G, et al. A new noninvasive management strategy for patients with suspected pulmonary embolism. Arch Intern Med. 1989;149:2549 –2555. 6. Perrier A, Howarth N, Didier D, et al. Performance of helical computed tomography in unselected outpatients with suspected pulmonary embolism. Ann Intern Med. 2001;135:88 –97. 7. Oudkerk M, van Beek EJ, Wielopolski P, et al. Comparison of contrast-enhanced magnetic resonance angiography and conventional pulmonary angiography for the diagnosis of pulmonary embolism: a prospective study. Lancet. 2002;359:1643–1647. 8. Goldhaber SZ. Echocardiography in the management of pulmonary embolism. Ann Intern Med. 2002;136:691–700. 9. Perrier A, Desmarais S, Miron MJ, et al. Non-invasive diagnosis of venous thromboembolism in outpatients. Lancet. 1999;353:190 – 195. 10. Wells PS, Anderson DR, Rodger M, et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and Ddimer. Ann Intern Med. 2001;135:98 –107. 11. Ginsberg JS, Wells PS, Kearon C, et al. Sensitivity and specificity of a rapid whole-blood assay for D-dimer in the diagnosis of pulmonary embolism. Ann Intern Med. 1998;129:1006 –1011. 12. Chagnon I, Bounameaux H, Aujesky D, Roy PM, et al. Comparison of two clinical prediction rules and implicit assessment among patients with suspected pulmonary embolism. Am J Med. 2002;113: 000 –000. 13. Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients’ probability of pulmonary embolism: increasing the models’ utility with the SimpliRED Ddimer. Thromb Haemost. 2000;83:416 –420. 14. Wicki J, Perneger TV, Junod AF, et al. Assessing clinical probability of pulmonary embolism in the emergency ward: a simple score. Arch Intern Med. 2001;161:92–97. 15. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143: 29 –36. 16. McNeil BJ, Hanley JA. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Med Decis Making. 1984;4:137–150.
Volume 113
Clinical Prediction Rules for the Diagnosis of Pulmonary Embolism Gabriela Ferreira, MD, Jeffrey L. Carson, MD
D
iagnosing pulmonary embolism is as challenging and controversial today as it was 25 years ago when lung scanning became widely implemented. In 1976, McNeil suggested that ventilation-perfusion lung scans in patients with suspected pulmonary embolism should be classified as normal, or low, high, or “indeterminate” or intermediate probability (1). Diagnosis is confirmed by a high-probability scan and is unlikely in patients with low-probability or normal scans. Patients with indeterminate scans need a pulmonary angiogram to establish disease. Although some authors argued that there were inadequate data to justify the widespread use of lung scans (2), this diagnostic algorithm was widely accepted and is at times used erroneously today. Two prospective series have defined the test characteristics of lung scans (3,4), showing that patients with high-probability scans had about an 87% probability of pulmonary embolism, compared with one third of patients with intermediate-probability scans, and 14% to 31% of those with low-probability scans. Missing 14% to 30% of patients with this potentially lethal disease is unacceptable even if we accept the most favorable result. For this reason, low- and intermediate-probability scans are now designated as nondiagnostic. Because pulmonary embolism is likely to originate in the deep veins of the lower extremities, subsequent albeit imperfect diagnostic strategies included testing for lower extremity venous thrombosis (5). There was much hope that computed tomography would provide more reliable information. Studies show that it is useful in diagnosing other conditions that may be confused with pulmonary embolism, and that it performs adequately to exclude or confirm pulmonary embolism in patients with large central emboli but misses pulmonary embolism in smaller pulmonary arteries (6). Magnetic resonance angiography has also been found to miss pulmonary embolism in smaller pulmonary arteries, but it can be used in patients with contraindications to
Am J Med. 2002;113:337–338. From the Division of General Internal Medicine, Department of Medicine, University of Medicine and Dentistry of New Jersey, Robert Wood Johnson Medical School, New Brunswick, New Jersey. Requests for reprints should be addressed to Jeffrey L. Carson, MD, University of Medicine and Dentistry of New Jersey, Robert Wood Johnson Medical School, 125 Paterson Street, New Brunswick, New Jersey 08903, or [email protected]. ©2002 by Excerpta Medica, Inc. All rights reserved.
contrast material (7). Echocardiography has been suggested as a diagnostic tool but awaits further study (8). The most promising new test is the D-dimer test, which has excellent sensitivity, especially if using enzyme-linked immunoassays. Most D-dimer assays are effective in excluding thromboembolism in outpatients, particularly when combined with a clinical picture consistent with low probability of disease (9 –11). The test should not be used in hospitalized patients, because the prevalence of pulmonary embolism is higher in this group, and in other conditions where test results are likely to be abnormal, because of poor specificity. A fundamental principle of diagnostic testing is to consider the likelihood of disease before testing (pretest probability) and to combine it with test results to obtain a post-test probability of disease. Investigators in the Prospective Investigation Of Pulmonary Embolism Diagnosis study demonstrated that experienced clinicians were able to classify patients with suspected pulmonary embolism with some precision based on clinical assessment (3): 9% in the low clinical pretest probability group, 30% in the intermediate group, and 68% in the high-probability group. When the clinical assessment was combined with ventilation-perfusion lung scanning, the probability of pulmonary embolism was refined further. If the clinical assessment and lung scan were concordant, pulmonary embolism was either highly likely (96%) or very unlikely (2%). However, when there was discordance between clinical assessment and lung scan results, the probability was 45% to 56%, and further testing was required. It is unclear if less experienced or nonspecialist physicians would be as successful in predicting pulmonary embolism because the probability estimate is based on clinical impression rather than specific criteria. Hence, explicit prediction rules were developed. In this issue of the Journal, Chagnon et al. (12) compare two such rules—the Wells’ rule and the Geneva rule (13,14). The Wells’ prediction rule was derived from a cohort of consecutive outpatients and inpatients with suspected pulmonary embolism (13). Seven variables associated with pulmonary embolism from a logistic regression model were assigned point values. Patients were classified as having low, intermediate, or high clinical probability (pretest probability) of pulmonary embolism based on point totals. These assessments were predictive of pulmonary embolism in their algorithm; however, their diag0002-9343/02/$–see front matter 337 PII S0002-9343(02)00000-0
Clinical Prediction Rules for the Diagnosis of Pulmonary Embolism/Ferreira and Carson
nostic algorithm was criticized for being too complicated and the derived clinical score for being too subjective for requiring clinicians to decide if an alternate diagnosis was less likely than pulmonary embolism. The Geneva rule was developed in consecutive outpatients who presented to an emergency department with suspected pulmonary embolism (14). It uses objective data only with clinical assessment override if there is disagreement. Eight predictors were found on multivariate analysis to be significantly associated with pulmonary embolism; none were subjective. Ten percent of patients in the low-probability, 38% of those in the intermediateprobability, and 81% of those in the high-probability group had pulmonary embolism (14). Chagnon et al. found that similar proportions of patients with low, intermediate, and high pretest probability had pulmonary embolism (12). Receiver operating characteristic curve analysis (15,16) showed moderate diagnostic performance and no significant difference between the two prediction rules: 0.74 for the Geneva score and 0.78 for the Wells’ score, and concordance was fair between the two rules ( coefficient ⫽ 0.43). However, when implicit evaluation was added to the Geneva score, the area under the curve was significantly better (0.82) than either score alone, suggesting that clinical judgment refines the probability beyond what is learned from the score alone. Interestingly, the criteria reported by clinicians in the Geneva study that informed their implicit assessment and led them to override the rule were very similar to the Wells’ criteria. Prediction rules should help clinicians to triage patients into clinically useful groups to avoid unnecessary testing, while minimizing risk. When combined with a negative D-dimer test, these rules exclude pulmonary embolism in outpatients with low pretest probability. In other patients, such as those with intermediate or high clinical probability, or inpatients, further testing is required. However, it is not clear that the explicit prediction rules offer enough of an advantage over empiric assessment that they are worth committing to memory or recording in a personal digital assistant (PDA). The authors suggest, and we agree, that the value of prediction rules may include training of junior physicians and resolution of disagreements among more experienced physicians using implicit evaluation. Clinicians should recognize that these rules only work in settings with similar disease prevalence, and that they are only useful if combined with the results of noninvasive tests that further refine the probability of disease because they do not adequately define the presence or absence of disease with enough precision to be used on their own. Many prediction rules have been developed and published, but few are used.
338
September 2002
THE AMERICAN JOURNAL OF MEDICINE威
Perhaps this will change with the ability to store complicated rules in the PDAs that are carried by many physicians. However, we are skeptical that these rules will be widely adopted.
REFERENCES 1. McNeil BJ. A diagnostic strategy using ventilation-perfusion studies in patients suspect for pulmonary embolism. J Nucl Med. 1976; 17:613–616. 2. Robin ED. Overdiagnosis and overtreatment of pulmonary embolism: the emperor may have no clothes. Ann Intern Med. 1977; 87:775–781. 3. The PIOPED Investigators. Value of the ventilation/perfusion scan in acute pulmonary embolism. Results of the Prospective Investigation Of Pulmonary Embolism Diagnosis (PIOPED. JAMA. 1990; 263:2753–2759. 4. Hull RD, Hirsh J, Carter CJ, et al. Diagnostic value of ventilationperfusion lung scanning in patients with suspected pulmonary embolism. Chest. 1985;88:819 –828. 5. Hull RD, Raskob GE, Coates G, et al. A new noninvasive management strategy for patients with suspected pulmonary embolism. Arch Intern Med. 1989;149:2549 –2555. 6. Perrier A, Howarth N, Didier D, et al. Performance of helical computed tomography in unselected outpatients with suspected pulmonary embolism. Ann Intern Med. 2001;135:88 –97. 7. Oudkerk M, van Beek EJ, Wielopolski P, et al. Comparison of contrast-enhanced magnetic resonance angiography and conventional pulmonary angiography for the diagnosis of pulmonary embolism: a prospective study. Lancet. 2002;359:1643–1647. 8. Goldhaber SZ. Echocardiography in the management of pulmonary embolism. Ann Intern Med. 2002;136:691–700. 9. Perrier A, Desmarais S, Miron MJ, et al. Non-invasive diagnosis of venous thromboembolism in outpatients. Lancet. 1999;353:190 – 195. 10. Wells PS, Anderson DR, Rodger M, et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and Ddimer. Ann Intern Med. 2001;135:98 –107. 11. Ginsberg JS, Wells PS, Kearon C, et al. Sensitivity and specificity of a rapid whole-blood assay for D-dimer in the diagnosis of pulmonary embolism. Ann Intern Med. 1998;129:1006 –1011. 12. Chagnon I, Bounameaux H, Aujesky D, Roy PM, et al. Comparison of two clinical prediction rules and implicit assessment among patients with suspected pulmonary embolism. Am J Med. 2002;113: 000 –000. 13. Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients’ probability of pulmonary embolism: increasing the models’ utility with the SimpliRED Ddimer. Thromb Haemost. 2000;83:416 –420. 14. Wicki J, Perneger TV, Junod AF, et al. Assessing clinical probability of pulmonary embolism in the emergency ward: a simple score. Arch Intern Med. 2001;161:92–97. 15. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143: 29 –36. 16. McNeil BJ, Hanley JA. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Med Decis Making. 1984;4:137–150.
Volume 113