- Email: [email protected]
Decision Support for Radiologist Report Recommendations
OPINION
Decision Support for Radiologist Report Recommendations Giles W. L. Boland, MD, James H. Thrall, MD, G. Scott Gazelle, MD, MPH, PhD, Anthony Samir, MD, Daniel I. Rosenthal, MD, Keith J. Dreyer, DO, PhD, Tarik K. Alkasab, MD, PhD THE CLINICAL PROBLEM
The past 2 decades have seen a remarkable increase in the capabilities, utilization, and cost burden of medical imaging [1-6]. This increase in demand for diagnostic imaging has put imaging in the spotlight as a target for cuts in reimbursement [1-7]. Much of the growth in imaging utilization is clearly beneficial to medical practice and increased quality of care [8]. However, we are also facing a challenge from some referring physicians who regard recommendations for additional imaging made by radiologists in their official reports as a form of “self-referral” [9]. Also, some physicians challenge radiologist recommendations on the grounds that they feel that such recommendations increase their risk for medical liability if they go unheeded [10]. Specialists often feel they know better than radiologists about what to do next and resent being pressured to act through the recommendation process. To promote the optimal use of imaging, the ACR established a program in 1992 to develop the ACR Appropriateness Criteria® [11]. These criteria are designed to help referring physicians select the right imaging examinations for their patients: the proverbial goal of right patient, right test, right reason, and right protocol. We now argue here that a similar set of appropriateness criteria should be developed to guide radiologists as they make recommendations for additional imaging. They should be objective, evidence based, and consensus driven—including input from nonradiologists—just as the original ACR Appropriateness Criteria are. They need to be presented to radiol-
ogists in the natural course of their work. In light of the foregoing, we have studied our own experience with recommendations for further imaging in a number of ways to determine the prevalence of recommendations and to further define the circumstances under which they are being made and by whom. Using a natural language processing system in one of our studies, we analyzed ⬎5.9 million radiology reports [12]. Of note, we demonstrated that radiologist recommendations have been steadily rising in our department for more than a decade, doubling from 6% to 12% of all imaging examinations [12]. The period of most rapid increase coincided with the introduction of PACS, and we hypothesize that fewer face-to-face consultations led to more written recommendations. Contrary to what might be expected, higher cost examinations more often result in recommendations for further studies, and the recommendations are typically for examinations of equivalent or greater cost than the original studies. Currently, up to 21% of all CT reports at Massachusetts General Hospital recommend additional imaging [12]. Recommendation rates are also unevenly distributed across anatomic sites. For example, additional imaging is recommended on up to 34% of chest examination reports [12]. This latter observation is obviously heavily influenced by recommendations that are made for the follow up of pulmonary nodules. Naturally, stakeholders are entitled to ask, Do radiologist recom-
© 2011 American College of Radiology 0091-2182/11/$36.00 ● DOI 10.1016/j.jacr.2011.08.003
mendations consistently add clinical value [10]? If this were the case, it might be expected that these recommendations would be consistent and reproducible among radiologists. However, our research has demonstrated that this is not the case. For instance, a study of interobserver variability in our department demonstrated major differences in the findings and recommendations in up to 32% of abdominal and pelvic CT imaging reports, particularly for patients with cancer [13]. Furthermore, the recommendation rates for additional imaging among radiologists within subspecialty groups varied widely: the recommendation rate for further imaging varied from 7% to 33% in reports of spinal MRI among neuroradiologists [12,14]. It has also been demonstrated in our department that the more experience radiologists have, the less likely they are to recommend further imaging studies [12]. There was a 100% difference in recommendation rates between radiologists with extensive experience and those recently graduated from residency and fellowship programs [12]. Specific factors that may help explain the substantial variation in recommendation rates include (1) radiologists’ diagnostic confidence (whether due to experience or to personality), (2) the fear of litigation, (3) the level of subspecialty expertise, (4) the need to adhere to protocols or local practices that apply unequally between specialty areas (eg, recommendations for the follow up of pulmonary nodules), (5) differing perceptions about referring physicians’ preferences con819
820 Opinion
cerning recommendations for further imaging, (6) the lack of ready access to standards and guidelines for making recommendations at the time of dictation, and (7) innate ability, and potentially others. We suspect that variations in recommendation rates similar to what we have encountered in our department also occur in other departments in the United States and abroad. Apart from the ACR’s Breast Imaging Reporting and Data System® (BI-RADS®) for breast imaging [15] and the Fleischner Society criteria [16] for pulmonary nodules, there are no widely agreed upon or widely promulgated criteria for when and under what circumstances recommendations should be made by radiologists. The ACR communications practice guideline makes reference to recommendations but does not serve as a case-level guide. Thus, no comprehensive set of criteria exist at the national level, nor are we aware of any individual departments with comprehensive data and consensusdriven guidelines for making recommendations across the practice of radiology. Variability in radiologist recommendations has negative effects on stakeholders. For referring physicians, it creates confusion and undermines confidence and trust. For instance, in a patient who presents with an uncharacterized liver lesion detected by ultrasound, one radiologist may recommend an immediate follow-up examination with MRI, another CT in 6 months, yet perhaps another follow-up ultrasound in 1 year, and perhaps another radiologist will recommend no follow up at all. Variation also likely increases systemic costs (financial, radiation dose, anxiety, and increased morbidity due to delay in diagnosis and treatment). Because a comprehensive and uniform set of consensus guidelines does not exist to guide practice, it is
not currently possible to manage recommendations in a comprehensive manner that maximizes quality of care and cost efficiency. Moreover, with the notable exception of BI-RADS, there are no imperatives for radiologists to follow guidelines should they be developed and promulgated, nor are there existing performance standards to judge appropriateness of use. Some may be skeptical that evidence-based guidelines will inform practice sufficiently to address individual patient needs. Yet many additional imaging recommendations made by radiologists result from common, well-recognized incidental findings, including small pulmonary nodules and breast, liver, pancreatic, renal, adnexal, adrenal, and thyroid lesions. Although it is not known what percentage of recommendations result from detecting incidental findings from these organs, it is likely, given their frequency in the general population, that most radiologists recommend additional follow-up examinations several times on a daily basis. Although considerable variation in the imaging recommendation rates exists for these incidental findings, some professional and subspecialty organizations have begun to publish guidelines and imaging algorithms for what to do when these kinds of incidental findings are detected, given certain extant clinical circumstances [17]. These recommendations are based on current evidence and consensus regarding best practices and are updated as new data become available. They constitute a substantial body of knowledge and a good starting point for a more comprehensive approach. These guidelines have been carefully crafted by experts using the best scientific evidence available and, although not mandatory, serve to direct interpreting radiologists toward best practices. However, although guidelines for incidental find-
ings and pulmonary nodules are available, we suspect that they are frequently not adhered to, nor is there an audit process at present to measure performance. There are a variety of reasons for this, including difficulty remembering and remaining current with all of the subspecialty guidelines, or because it is too time consuming to look up the recommendations during study reporting, or perhaps radiologists may be unaware of the guidelines in the first place, and maybe because some reporting radiologists simply ignore them. The development of more comprehensive guidelines for making recommendations for further imaging and promoting adherence to them could have several potential benefits: (1) patient care is likely to be better, (2) referring physicians will likely gain more confidence in radiology groups whose recommendations are more consistent, and (3) practice standardization will permit the creation of metrics that will inform the scientific evaluation of radiologist reporting. Moreover, the specter of other physicians’ raising questions of self-referral and concerns about their risks for not following recommendations should be mitigated. Finally, it could also serve to inform radiology benefit managers, who purport to reduce inappropriate imaging costs, particularly for cross-sectional imaging, although there are no independent data on their success to date [18]. Most radiologists would probably accept that nationally recognized guidelines should be further developed and adhered to. The ACR is the logical convening organization to accomplish the first task. The question then becomes, How does one encourage compliance? This is not as easy as it sounds: although some guidelines, such as the Fleischner Society criteria for pulmonary nodules, have existed for years, deviation from them and considerable variation among
Opinion 821
radiologists continues unabated. A number of strategies have been tried to mitigate this problem. For example, some departments post guidelines above workstations for easy access or have these guidelines electronically available. Other departments copy the Fleischner Society guidelines into their reports for consideration by referring physicians. A recent report has highlighted PACS integration with a for-profit radiologic educational Web site for real-time aid in radiologic interpretation [19]. These tactics, although useful, still have not achieved the radiologists’ goal of adhering to these guidelines, as evidenced by the research findings. THE SOLUTION
Electronic decision support (DS) is particularly promising as a method for delivering guidelines to radiologists for making recommendations for further imaging. The utilization of DS tools has been growing rapidly in clinical medicine [20-24]. The premise is to efficiently direct caregivers toward standardized evidence-based clinical decisions, with the goal of improving patient outcomes, often at lower cost [23,24]. On the basis of previous experience with computerized physician order entry (CPOE) systems combined with DS for ordering imaging procedures, it seems feasible to deliver guidelines to radiologists at the point of care in the context of their normal work processes, thereby achieving the desired goal of facilitating guideline compliance without an excessive burden of time on task [23]. The use of electronic DS tools in conjunction with CPOE has been demonstrated to help reduce unnecessary imaging and improve adherence to nationally recognized appropriateness criteria [23-25]. A report from our institution using a DS system based on modified ACR Appropriateness Criteria demon-
strated a 19% reduction in the use of CT, largely because inappropriate examination requests were diverted to other investigations or to no investigation at all [24]. It should be noted that the key to DS success was that the guidelines adhered to recognized national and institutional best practices and because DS was seamlessly integrated into the CPOE tool. Referring physicians are required to input clinical data into the DS system to proceed with their examination requests [23,24]. Approval of an examination request depends on achieving an acceptable appropriateness score, which is based on the combination of a requested examination and the reason for it. For example, if a referring physician requests MRI for a patient with back pain of ⬍1 month in duration and no other clinical symptoms or signs, it would not be approved because this test would not be considered appropriate for the reason given [23]. Had the patient demonstrated long tract signs (ie, reflex changes), the DS tool would have returned a higher appropriateness score. Because the system is electronic, physician compliance to the system is readily monitored. There are too many choices and algorithms for every physician to remember every best practice; hence DS provides real-time, context-specific assistance to referring physicians to adhere to the latest best practices: “just-in-time” knowledge delivery at the point of care. It is likely that only through such systems can a disparate number of referring physicians be directed to adhere to accepted national and institutional appropriateness criteria. The underlying dynamics that drive the inconsistency for further imaging recommendations by radiologists are similar to those that drive the challenge to referring physicians to order the correct or best test in general: there are simply too many choices and algorithms for every phy-
sician to remember and, just as important, follow [26]. A real-time reporting DS tool for radiology should serve to steer radiologists toward making recommendations that conform to institutional and national guidelines. Rather than radiologists issuing further imaging recommendations on the basis of personal preference, knowledge, or experience, the DS tool can, and should, direct them to conform to recognized best practices. For follow-up recommendations, it should also help less experienced radiologists move toward the recommendation pattern of more experienced radiologists and in so doing substantially reduce variation among radiologists. Ideally, this tool would be seamlessly integrated into existing voice recognition software dictation systems but could also conceivably be implemented as an independent, stand-alone, serverbased product (Table 1). There is already a precedent, as there are standalone products for BI-RADS to standardize the report recommendations, although the BI-RADS category is still chosen by the individual radiologist from personal experience rather than through an interactive electronic DS algorithm. The advantage of embedding DS within the normal existing reporting workflow is that a radiologist would be required to navigate through a DS tree whenever a recommendation was contemplated before report completion. This would also permit recommendations for further imaging to be automatically inserted into the report using standardized language. For example, a radiologist confronted by an incidental adrenal nodule on CT will be asked specific contextual questions by the DS system (eg, Does the patient have cancer? Is the lesion ⬍ 10 Hounsfield units on noncontrast CT?) [17]. The DS algorithm will then direct the radiologist to the appropriate recommendation for the given set of circumstances for that patient and will then place the
822 Opinion
Table 1. Necessary criteria for an effective electronic decision support recommendation tool Evidence Standards Collated from National Societies Meeting Based Appropriateness Criteria Consensus driven Effective Transparent Adaptable Iterative Reproducible Standardized Easy to use
Data mining Educational
Radiologist, institutional, and nonradiologist input Improves clinical and financial outcomes Standard recommendation methodology open to stakeholders Meets specific needs of organizations and clinical practice Can be updated as new best practices become available Can be implemented across diverse organizations Multiorganizational conformity to accepted best practices; recommendation language standardized with macros Seamless integration into existing information system platforms; fast and data input minimized to critical information Analysis of volume, cost, benchmarking to institutional and national data Opportunity to educate stakeholders to current and future best practices
appropriate standardized recommendation into the report. The key, similar to a CPOE DS system, is that radiologists are required to navigate through the DS system such that they conform to the guidelines rather than ignore them. Such systems should not be entirely prescriptive, and radiologists should be able to deviate from standardized recommendations if warranted by the clinical circumstances. For performance monitoring purposes, radiologists might be required to insert an explanatory note to state why they deviated from the guidelines. Although DS can be implemented for any incidental imaging finding or other setting triggering recommendations for further imaging, it is most likely to be maximally useful, at least initially, if targeted to the most common incidentally detected abnormalities as described above. Conforming to these relatively few DS algorithms should encourage radiologists to use them effectively; otherwise, the time to generate a report may be excessively long. Another collateral benefit is that a recommendation system based on standard guidelines would allow nonradiologists to participate in the development process, which has been a key part of the development
of the current ACR Appropriateness Criteria. Referring physicians would have access to the criteria and understand that they are evidence based and determined by consensus. This is crucial to having the criteria accepted outside radiology. The criteria would be transparent, and referring physicians could also use them to help explain to patients why a particular course of care is being recommended. CONCLUSIONS
Some best practices for further imaging recommendations for incidental findings have already been established by national professional organizations, but adherence to them is inconsistent, reducing referring physicians’ confidence and potentially increasing health care costs. These guidelines must be further developed and organized in a way that can be embedded in the work process of radiologists. Electronic DS tools are likely to be the only effective method to achieve this goal and to increase adherence to standards. Further investigation of the impact of integrated DS on health care costs is warranted, partly because it is as yet uncertain whether better adherence to best practices will increase, rather
than decrease, the number of imaging recommendations. Nonetheless, conforming to best practice standards should result in better patient care and greater trust and collaboration with referring physicians. REFERENCES 1. Bodenheimer T. High and rising health care costs. Seeking an explanation. Ann Intern Med 2005;142:847-54. 2. Iglehart JK. Health insurers and medicalimaging policy—a work in progress. N Engl J Med 2009;360:1030-7. 3. Congressional Budget Office. Budget options, volume I: health care. Available at: http:// www.cbo.gov/ftpdocs/99xx/doc9925/12-18healthoptions.pdf. Accessed June 20, 2011. 4. US Government Accountability Office. Medicare Part B imaging services: rapid spending growth and shift to physician offices indicate need for CMS to consider additional management practices (GAO 08452). Available at: http://www.gao.gov/ highlights/d08452high.pdf. Accessed June 20, 2011. 5. America’s Health Insurance Plans. Ensuring quality through appropriate use of diagnostic imaging. Available at: http://www.ahip.org/ content/default.aspx?docid⫽24057. Accessed June 20, 2011. 6. Medicare Payment Advisory Commission. Report to the Congress: Medicare payment policy, March 2011. Available at: http://www.medpac.gov/documents/ mar11_entirereport.pdf. Accessed July 6, 2011. 7. Moser JW. The Deficit Reduction Act of 2005: policy, politics, and impact on radiologists. J Am Coll Radiol 2006;10:744-50. 8. Fuchs VR, Sox HC Jr. Physicians’ views of the relative importance of thirty medical innovations. Health Aff (Millwood) 2001;20: 30-42. 9. Kilani RK, Paxton BE, Stinnett SS, Barnhart HX, et al. Self-referral in medical imaging: a meta-analysis of the literature. J Am Coll Radiol 2011;8:469-76. 10. Arenson RL. Recommendations for additional imaging in radiology reports: radiologists’ self-referral or good clinical practice? Radiology 2009;253:291-2. 11. American College of Radiology. ACR Appropriateness Criteria. Available at: http:// www.acr.org/ac. Accessed July 6, 2011. 12. Sistrom CL, Dreyer KJ, Dang PA, Weilburg JB, Boland GW, Rosenthal DI, Thrall JH. Recommendations for additional imaging in radiology reports: multifactorial analysis of 5.9 million examinations. Radiology 2009;253:453-61. 13. Abujudeh H, Boland GWL, Kaewlai R, et al. Abdominal and pelvic computed tomography (CT) interpretation: discrepancy rates
Opinion 823
among experienced radiologists. Eur Radiol 2010;20:1952-7.
Findings Committee. J Am Coll Radiol 2010;7:754-73.
port in controlling inappropriate imaging. J Am Coll Radiol 2011;8:19-25.
14. Dang PA, Boland GW, Kalra MK, Schultz TJ, Stout M, Dreyer KJ. Recommendation rates in radiology: does radiologist experience and number of studies interpreted have an effect? Presented at: 94th Annual Meeting of the Radiological Society of North America; November 2008.
18. Lee DW, Rawson JV, Wade SW. Radiology benefit managers: cost saving or cost shifting? J Am Coll Radiol 2011;8:393-401.
23. Rosenthal DI, Weilburg JB, Schultz T, et al. Radiology order entry with decision support: initial clinical experience. J Am Coll Radiol 2006;3:799-806.
15. American College of Radiology. BI-RADS® atlas. Available at: http://www.acr.org/second arymainmenucategories/quality_safety/birad satlas.aspx. Accessed July 7, 2011.
20. Sim I, Gorman P, Greenes RA, et al. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc 2001;8:527-34.
16. Heber MacMahon H, Austin JHM, Gamsu G, et al. Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology 2005;237:395-400.
21. Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011;18:327-34.
17. Berland LL, Silverman SG, Gore RM, et al. Managing incidental findings on abdominal CT: white paper of the ACR Incidental
19. Morgan MB, Branstetter BF, Clark C, et al. Just-in-time radiologist decision support: the importance of PACS-integrated workflow. J Am Coll Radiol 2011;8:497-500.
22. Blackmore CG, Mecklenburg RS, Kaplan GS. Effectiveness of clinical decision sup-
24. Sistrom CL, Dang PA, Weilburg JB, Dreyer KJ, Rosenthal DI, Thrall JH. Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: sevenyear time series analysis. Radiology 2009; 251:147-55. 25. Sistrom CL. In support of the ACR Appropriateness Criteria®. J Am Coll Radiol 2008;5:630-5. 26. Boland GWL. From herding cats toward best practices: standardizing the radiologic work process. AJR Am J Roentgenol 2009; 193:1593-5.
James H. Thrall, MD, G. Scott Gazelle, MD, MPH, PhD, Anthony Samir, MD, Daniel I. Rosenthal, MD, Keith J. Dreyer, DO, PhD, and Tarik K. Alkasab, MD, PhD, are from Massachusetts General Hospital, Department of Radiology, Boston, Massachusetts. Giles W. L. Boland, MD, Massachusetts General Hospital, Department of Radiology, White Building 270C, 55 Fruit Street, Boston, MA 02114; e-mail: [email protected].
Decision Support for Radiologist Report Recommendations Giles W. L. Boland, MD, James H. Thrall, MD, G. Scott Gazelle, MD, MPH, PhD, Anthony Samir, MD, Daniel I. Rosenthal, MD, Keith J. Dreyer, DO, PhD, Tarik K. Alkasab, MD, PhD THE CLINICAL PROBLEM
The past 2 decades have seen a remarkable increase in the capabilities, utilization, and cost burden of medical imaging [1-6]. This increase in demand for diagnostic imaging has put imaging in the spotlight as a target for cuts in reimbursement [1-7]. Much of the growth in imaging utilization is clearly beneficial to medical practice and increased quality of care [8]. However, we are also facing a challenge from some referring physicians who regard recommendations for additional imaging made by radiologists in their official reports as a form of “self-referral” [9]. Also, some physicians challenge radiologist recommendations on the grounds that they feel that such recommendations increase their risk for medical liability if they go unheeded [10]. Specialists often feel they know better than radiologists about what to do next and resent being pressured to act through the recommendation process. To promote the optimal use of imaging, the ACR established a program in 1992 to develop the ACR Appropriateness Criteria® [11]. These criteria are designed to help referring physicians select the right imaging examinations for their patients: the proverbial goal of right patient, right test, right reason, and right protocol. We now argue here that a similar set of appropriateness criteria should be developed to guide radiologists as they make recommendations for additional imaging. They should be objective, evidence based, and consensus driven—including input from nonradiologists—just as the original ACR Appropriateness Criteria are. They need to be presented to radiol-
ogists in the natural course of their work. In light of the foregoing, we have studied our own experience with recommendations for further imaging in a number of ways to determine the prevalence of recommendations and to further define the circumstances under which they are being made and by whom. Using a natural language processing system in one of our studies, we analyzed ⬎5.9 million radiology reports [12]. Of note, we demonstrated that radiologist recommendations have been steadily rising in our department for more than a decade, doubling from 6% to 12% of all imaging examinations [12]. The period of most rapid increase coincided with the introduction of PACS, and we hypothesize that fewer face-to-face consultations led to more written recommendations. Contrary to what might be expected, higher cost examinations more often result in recommendations for further studies, and the recommendations are typically for examinations of equivalent or greater cost than the original studies. Currently, up to 21% of all CT reports at Massachusetts General Hospital recommend additional imaging [12]. Recommendation rates are also unevenly distributed across anatomic sites. For example, additional imaging is recommended on up to 34% of chest examination reports [12]. This latter observation is obviously heavily influenced by recommendations that are made for the follow up of pulmonary nodules. Naturally, stakeholders are entitled to ask, Do radiologist recom-
© 2011 American College of Radiology 0091-2182/11/$36.00 ● DOI 10.1016/j.jacr.2011.08.003
mendations consistently add clinical value [10]? If this were the case, it might be expected that these recommendations would be consistent and reproducible among radiologists. However, our research has demonstrated that this is not the case. For instance, a study of interobserver variability in our department demonstrated major differences in the findings and recommendations in up to 32% of abdominal and pelvic CT imaging reports, particularly for patients with cancer [13]. Furthermore, the recommendation rates for additional imaging among radiologists within subspecialty groups varied widely: the recommendation rate for further imaging varied from 7% to 33% in reports of spinal MRI among neuroradiologists [12,14]. It has also been demonstrated in our department that the more experience radiologists have, the less likely they are to recommend further imaging studies [12]. There was a 100% difference in recommendation rates between radiologists with extensive experience and those recently graduated from residency and fellowship programs [12]. Specific factors that may help explain the substantial variation in recommendation rates include (1) radiologists’ diagnostic confidence (whether due to experience or to personality), (2) the fear of litigation, (3) the level of subspecialty expertise, (4) the need to adhere to protocols or local practices that apply unequally between specialty areas (eg, recommendations for the follow up of pulmonary nodules), (5) differing perceptions about referring physicians’ preferences con819
820 Opinion
cerning recommendations for further imaging, (6) the lack of ready access to standards and guidelines for making recommendations at the time of dictation, and (7) innate ability, and potentially others. We suspect that variations in recommendation rates similar to what we have encountered in our department also occur in other departments in the United States and abroad. Apart from the ACR’s Breast Imaging Reporting and Data System® (BI-RADS®) for breast imaging [15] and the Fleischner Society criteria [16] for pulmonary nodules, there are no widely agreed upon or widely promulgated criteria for when and under what circumstances recommendations should be made by radiologists. The ACR communications practice guideline makes reference to recommendations but does not serve as a case-level guide. Thus, no comprehensive set of criteria exist at the national level, nor are we aware of any individual departments with comprehensive data and consensusdriven guidelines for making recommendations across the practice of radiology. Variability in radiologist recommendations has negative effects on stakeholders. For referring physicians, it creates confusion and undermines confidence and trust. For instance, in a patient who presents with an uncharacterized liver lesion detected by ultrasound, one radiologist may recommend an immediate follow-up examination with MRI, another CT in 6 months, yet perhaps another follow-up ultrasound in 1 year, and perhaps another radiologist will recommend no follow up at all. Variation also likely increases systemic costs (financial, radiation dose, anxiety, and increased morbidity due to delay in diagnosis and treatment). Because a comprehensive and uniform set of consensus guidelines does not exist to guide practice, it is
not currently possible to manage recommendations in a comprehensive manner that maximizes quality of care and cost efficiency. Moreover, with the notable exception of BI-RADS, there are no imperatives for radiologists to follow guidelines should they be developed and promulgated, nor are there existing performance standards to judge appropriateness of use. Some may be skeptical that evidence-based guidelines will inform practice sufficiently to address individual patient needs. Yet many additional imaging recommendations made by radiologists result from common, well-recognized incidental findings, including small pulmonary nodules and breast, liver, pancreatic, renal, adnexal, adrenal, and thyroid lesions. Although it is not known what percentage of recommendations result from detecting incidental findings from these organs, it is likely, given their frequency in the general population, that most radiologists recommend additional follow-up examinations several times on a daily basis. Although considerable variation in the imaging recommendation rates exists for these incidental findings, some professional and subspecialty organizations have begun to publish guidelines and imaging algorithms for what to do when these kinds of incidental findings are detected, given certain extant clinical circumstances [17]. These recommendations are based on current evidence and consensus regarding best practices and are updated as new data become available. They constitute a substantial body of knowledge and a good starting point for a more comprehensive approach. These guidelines have been carefully crafted by experts using the best scientific evidence available and, although not mandatory, serve to direct interpreting radiologists toward best practices. However, although guidelines for incidental find-
ings and pulmonary nodules are available, we suspect that they are frequently not adhered to, nor is there an audit process at present to measure performance. There are a variety of reasons for this, including difficulty remembering and remaining current with all of the subspecialty guidelines, or because it is too time consuming to look up the recommendations during study reporting, or perhaps radiologists may be unaware of the guidelines in the first place, and maybe because some reporting radiologists simply ignore them. The development of more comprehensive guidelines for making recommendations for further imaging and promoting adherence to them could have several potential benefits: (1) patient care is likely to be better, (2) referring physicians will likely gain more confidence in radiology groups whose recommendations are more consistent, and (3) practice standardization will permit the creation of metrics that will inform the scientific evaluation of radiologist reporting. Moreover, the specter of other physicians’ raising questions of self-referral and concerns about their risks for not following recommendations should be mitigated. Finally, it could also serve to inform radiology benefit managers, who purport to reduce inappropriate imaging costs, particularly for cross-sectional imaging, although there are no independent data on their success to date [18]. Most radiologists would probably accept that nationally recognized guidelines should be further developed and adhered to. The ACR is the logical convening organization to accomplish the first task. The question then becomes, How does one encourage compliance? This is not as easy as it sounds: although some guidelines, such as the Fleischner Society criteria for pulmonary nodules, have existed for years, deviation from them and considerable variation among
Opinion 821
radiologists continues unabated. A number of strategies have been tried to mitigate this problem. For example, some departments post guidelines above workstations for easy access or have these guidelines electronically available. Other departments copy the Fleischner Society guidelines into their reports for consideration by referring physicians. A recent report has highlighted PACS integration with a for-profit radiologic educational Web site for real-time aid in radiologic interpretation [19]. These tactics, although useful, still have not achieved the radiologists’ goal of adhering to these guidelines, as evidenced by the research findings. THE SOLUTION
Electronic decision support (DS) is particularly promising as a method for delivering guidelines to radiologists for making recommendations for further imaging. The utilization of DS tools has been growing rapidly in clinical medicine [20-24]. The premise is to efficiently direct caregivers toward standardized evidence-based clinical decisions, with the goal of improving patient outcomes, often at lower cost [23,24]. On the basis of previous experience with computerized physician order entry (CPOE) systems combined with DS for ordering imaging procedures, it seems feasible to deliver guidelines to radiologists at the point of care in the context of their normal work processes, thereby achieving the desired goal of facilitating guideline compliance without an excessive burden of time on task [23]. The use of electronic DS tools in conjunction with CPOE has been demonstrated to help reduce unnecessary imaging and improve adherence to nationally recognized appropriateness criteria [23-25]. A report from our institution using a DS system based on modified ACR Appropriateness Criteria demon-
strated a 19% reduction in the use of CT, largely because inappropriate examination requests were diverted to other investigations or to no investigation at all [24]. It should be noted that the key to DS success was that the guidelines adhered to recognized national and institutional best practices and because DS was seamlessly integrated into the CPOE tool. Referring physicians are required to input clinical data into the DS system to proceed with their examination requests [23,24]. Approval of an examination request depends on achieving an acceptable appropriateness score, which is based on the combination of a requested examination and the reason for it. For example, if a referring physician requests MRI for a patient with back pain of ⬍1 month in duration and no other clinical symptoms or signs, it would not be approved because this test would not be considered appropriate for the reason given [23]. Had the patient demonstrated long tract signs (ie, reflex changes), the DS tool would have returned a higher appropriateness score. Because the system is electronic, physician compliance to the system is readily monitored. There are too many choices and algorithms for every physician to remember every best practice; hence DS provides real-time, context-specific assistance to referring physicians to adhere to the latest best practices: “just-in-time” knowledge delivery at the point of care. It is likely that only through such systems can a disparate number of referring physicians be directed to adhere to accepted national and institutional appropriateness criteria. The underlying dynamics that drive the inconsistency for further imaging recommendations by radiologists are similar to those that drive the challenge to referring physicians to order the correct or best test in general: there are simply too many choices and algorithms for every phy-
sician to remember and, just as important, follow [26]. A real-time reporting DS tool for radiology should serve to steer radiologists toward making recommendations that conform to institutional and national guidelines. Rather than radiologists issuing further imaging recommendations on the basis of personal preference, knowledge, or experience, the DS tool can, and should, direct them to conform to recognized best practices. For follow-up recommendations, it should also help less experienced radiologists move toward the recommendation pattern of more experienced radiologists and in so doing substantially reduce variation among radiologists. Ideally, this tool would be seamlessly integrated into existing voice recognition software dictation systems but could also conceivably be implemented as an independent, stand-alone, serverbased product (Table 1). There is already a precedent, as there are standalone products for BI-RADS to standardize the report recommendations, although the BI-RADS category is still chosen by the individual radiologist from personal experience rather than through an interactive electronic DS algorithm. The advantage of embedding DS within the normal existing reporting workflow is that a radiologist would be required to navigate through a DS tree whenever a recommendation was contemplated before report completion. This would also permit recommendations for further imaging to be automatically inserted into the report using standardized language. For example, a radiologist confronted by an incidental adrenal nodule on CT will be asked specific contextual questions by the DS system (eg, Does the patient have cancer? Is the lesion ⬍ 10 Hounsfield units on noncontrast CT?) [17]. The DS algorithm will then direct the radiologist to the appropriate recommendation for the given set of circumstances for that patient and will then place the
822 Opinion
Table 1. Necessary criteria for an effective electronic decision support recommendation tool Evidence Standards Collated from National Societies Meeting Based Appropriateness Criteria Consensus driven Effective Transparent Adaptable Iterative Reproducible Standardized Easy to use
Data mining Educational
Radiologist, institutional, and nonradiologist input Improves clinical and financial outcomes Standard recommendation methodology open to stakeholders Meets specific needs of organizations and clinical practice Can be updated as new best practices become available Can be implemented across diverse organizations Multiorganizational conformity to accepted best practices; recommendation language standardized with macros Seamless integration into existing information system platforms; fast and data input minimized to critical information Analysis of volume, cost, benchmarking to institutional and national data Opportunity to educate stakeholders to current and future best practices
appropriate standardized recommendation into the report. The key, similar to a CPOE DS system, is that radiologists are required to navigate through the DS system such that they conform to the guidelines rather than ignore them. Such systems should not be entirely prescriptive, and radiologists should be able to deviate from standardized recommendations if warranted by the clinical circumstances. For performance monitoring purposes, radiologists might be required to insert an explanatory note to state why they deviated from the guidelines. Although DS can be implemented for any incidental imaging finding or other setting triggering recommendations for further imaging, it is most likely to be maximally useful, at least initially, if targeted to the most common incidentally detected abnormalities as described above. Conforming to these relatively few DS algorithms should encourage radiologists to use them effectively; otherwise, the time to generate a report may be excessively long. Another collateral benefit is that a recommendation system based on standard guidelines would allow nonradiologists to participate in the development process, which has been a key part of the development
of the current ACR Appropriateness Criteria. Referring physicians would have access to the criteria and understand that they are evidence based and determined by consensus. This is crucial to having the criteria accepted outside radiology. The criteria would be transparent, and referring physicians could also use them to help explain to patients why a particular course of care is being recommended. CONCLUSIONS
Some best practices for further imaging recommendations for incidental findings have already been established by national professional organizations, but adherence to them is inconsistent, reducing referring physicians’ confidence and potentially increasing health care costs. These guidelines must be further developed and organized in a way that can be embedded in the work process of radiologists. Electronic DS tools are likely to be the only effective method to achieve this goal and to increase adherence to standards. Further investigation of the impact of integrated DS on health care costs is warranted, partly because it is as yet uncertain whether better adherence to best practices will increase, rather
than decrease, the number of imaging recommendations. Nonetheless, conforming to best practice standards should result in better patient care and greater trust and collaboration with referring physicians. REFERENCES 1. Bodenheimer T. High and rising health care costs. Seeking an explanation. Ann Intern Med 2005;142:847-54. 2. Iglehart JK. Health insurers and medicalimaging policy—a work in progress. N Engl J Med 2009;360:1030-7. 3. Congressional Budget Office. Budget options, volume I: health care. Available at: http:// www.cbo.gov/ftpdocs/99xx/doc9925/12-18healthoptions.pdf. Accessed June 20, 2011. 4. US Government Accountability Office. Medicare Part B imaging services: rapid spending growth and shift to physician offices indicate need for CMS to consider additional management practices (GAO 08452). Available at: http://www.gao.gov/ highlights/d08452high.pdf. Accessed June 20, 2011. 5. America’s Health Insurance Plans. Ensuring quality through appropriate use of diagnostic imaging. Available at: http://www.ahip.org/ content/default.aspx?docid⫽24057. Accessed June 20, 2011. 6. Medicare Payment Advisory Commission. Report to the Congress: Medicare payment policy, March 2011. Available at: http://www.medpac.gov/documents/ mar11_entirereport.pdf. Accessed July 6, 2011. 7. Moser JW. The Deficit Reduction Act of 2005: policy, politics, and impact on radiologists. J Am Coll Radiol 2006;10:744-50. 8. Fuchs VR, Sox HC Jr. Physicians’ views of the relative importance of thirty medical innovations. Health Aff (Millwood) 2001;20: 30-42. 9. Kilani RK, Paxton BE, Stinnett SS, Barnhart HX, et al. Self-referral in medical imaging: a meta-analysis of the literature. J Am Coll Radiol 2011;8:469-76. 10. Arenson RL. Recommendations for additional imaging in radiology reports: radiologists’ self-referral or good clinical practice? Radiology 2009;253:291-2. 11. American College of Radiology. ACR Appropriateness Criteria. Available at: http:// www.acr.org/ac. Accessed July 6, 2011. 12. Sistrom CL, Dreyer KJ, Dang PA, Weilburg JB, Boland GW, Rosenthal DI, Thrall JH. Recommendations for additional imaging in radiology reports: multifactorial analysis of 5.9 million examinations. Radiology 2009;253:453-61. 13. Abujudeh H, Boland GWL, Kaewlai R, et al. Abdominal and pelvic computed tomography (CT) interpretation: discrepancy rates
Opinion 823
among experienced radiologists. Eur Radiol 2010;20:1952-7.
Findings Committee. J Am Coll Radiol 2010;7:754-73.
port in controlling inappropriate imaging. J Am Coll Radiol 2011;8:19-25.
14. Dang PA, Boland GW, Kalra MK, Schultz TJ, Stout M, Dreyer KJ. Recommendation rates in radiology: does radiologist experience and number of studies interpreted have an effect? Presented at: 94th Annual Meeting of the Radiological Society of North America; November 2008.
18. Lee DW, Rawson JV, Wade SW. Radiology benefit managers: cost saving or cost shifting? J Am Coll Radiol 2011;8:393-401.
23. Rosenthal DI, Weilburg JB, Schultz T, et al. Radiology order entry with decision support: initial clinical experience. J Am Coll Radiol 2006;3:799-806.
15. American College of Radiology. BI-RADS® atlas. Available at: http://www.acr.org/second arymainmenucategories/quality_safety/birad satlas.aspx. Accessed July 7, 2011.
20. Sim I, Gorman P, Greenes RA, et al. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc 2001;8:527-34.
16. Heber MacMahon H, Austin JHM, Gamsu G, et al. Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology 2005;237:395-400.
21. Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011;18:327-34.
17. Berland LL, Silverman SG, Gore RM, et al. Managing incidental findings on abdominal CT: white paper of the ACR Incidental
19. Morgan MB, Branstetter BF, Clark C, et al. Just-in-time radiologist decision support: the importance of PACS-integrated workflow. J Am Coll Radiol 2011;8:497-500.
22. Blackmore CG, Mecklenburg RS, Kaplan GS. Effectiveness of clinical decision sup-
24. Sistrom CL, Dang PA, Weilburg JB, Dreyer KJ, Rosenthal DI, Thrall JH. Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: sevenyear time series analysis. Radiology 2009; 251:147-55. 25. Sistrom CL. In support of the ACR Appropriateness Criteria®. J Am Coll Radiol 2008;5:630-5. 26. Boland GWL. From herding cats toward best practices: standardizing the radiologic work process. AJR Am J Roentgenol 2009; 193:1593-5.
James H. Thrall, MD, G. Scott Gazelle, MD, MPH, PhD, Anthony Samir, MD, Daniel I. Rosenthal, MD, Keith J. Dreyer, DO, PhD, and Tarik K. Alkasab, MD, PhD, are from Massachusetts General Hospital, Department of Radiology, Boston, Massachusetts. Giles W. L. Boland, MD, Massachusetts General Hospital, Department of Radiology, White Building 270C, 55 Fruit Street, Boston, MA 02114; e-mail: [email protected].