- Email: [email protected]
Teaching Evidence-Based Imaging in the Radiology Clerkship Using the ACR Appropriateness Criteria
Radiologic Education
Teaching Evidence-Based Imaging in the Radiology Clerkship Using the ACR Appropriateness Criteria Johanne E. Dillon, MD, Priscilla J. Slanetz, MD, MPH, FACR Rationale and Objectives: Given escalating health care costs and limited resources, the effective utilization of imaging is essential to permit optimal patient care. Because only 21.4% of medical schools require radiology clerkships, most students do not understand the indications for and clinical effectiveness of most imaging tests. Therefore, the authors introduced two focused sessions on evidencebased imaging during the required radiology core clerkship at their institution. Methods: From June 2008 to September 2009, 259 students in the radiology core clerkship participated in a didactic session on principles of evidence-based imaging and/or a small-group session and self-directed learning exercise using the American College of Radiology (ACR) Appropriateness Criteria. Students evaluated the exercise on a five-point scale for its effectiveness in teaching imaging strategies and its impact on their future careers. Results: Two hundred twenty-eight of 259 students (88%; 93 third year, 135 fourth year) participated in the two sessions. Two hundred thirteen of 223 (96%) were not aware of the ACR’s Web site as a resource prior to this exercise. Two hundred fourteen of 228 (94%) found the ACR Appropriateness Criteria a useful resource. Two hundred two of 228 (89%) stated that they would likely use this resource in other clinical rotations and future practice. Two hundred three of 228 (89%) reported having a solid understanding of the indications for imaging tests following the sessions. Students praised the ACR’s Web site for its comprehensive coverage of different modalities and their relative radiation risks. They valued working collaboratively on common clinical scenarios. Conclusion: The ACR Appropriateness Criteria are a valuable resource for teaching evidence-based imaging to medical students. A majority of students indicated that they plan to use this resource in the future. Key Words: Evidence-based imaging; evidence-based medicine; cost-effective care; appropriateness criteria; radiology; medical education. ªAUR, 2010
T
he increased utilization of imaging due to the introduction of new technology over the past few decades has been linked to escalating health care costs (1,2). Often, new imaging techniques are widely adopted prior to establishing their cost-effectiveness. This widespread incorporation of technology into patient care is multifactorial and includes rising patient expectations for earlier diagnosis, improved access to imaging, added diagnostic certainty for providers, increased reliance on technology for answers, and concern over potential liability (3). Given limited resources, there is a growing need to educate physicians about the strengths and limitations of different imaging modalities for specific clinical presentations. Although imaging tests can Acad Radiol 2010; 17:912–916 From the Department of Radiology, Boston Medical Center and Boston University School of Medicine, Boston, MA (J.E.D.); and the Department of Radiology, Division of Breast Imaging, Shapiro Clinical Center, 4th Floor, Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Avenue, Boston, MA 02215 (P.J.S.). Received January 28, 2010; accepted March 3, 2010. Address correspondence to: P.J.S. e-mail: [email protected] ªAUR, 2010 doi:10.1016/j.acra.2010.03.001
912
provide useful information that may directly affect patient care, physicians must use clinical histories and findings on physical examinations to guide them to the appropriate differential diagnosis and subsequent selection of laboratory and imaging tests. Without the benefit of an adequate history and physical examination, providers are apt to rely on a battery of tests, leading to untoward risk to patients and unnecessary costs to the health care system. Knowledge regarding the cost-effectiveness of different imaging modalities is essential to optimize patient care. Over the past decade, there has been increasing interest in evidence-based medicine and, more recently, evidence-based imaging. Defined as combining the best scientific evidence with physician expertise and patient expectations (4), evidence-based imaging can be expected to play a pivotal role to help physicians choose the ‘‘best’’ or most cost effective imaging tests for their patients. This approach will likely translate into stabilization of health care costs related to imaging and more appropriate patient care, as no imaging test is without risk. The American College of Radiology (ACR) has been interested in evidence-based imaging for over a decade. The
Academic Radiology, Vol 17, No 7, July 2010
national efforts of the ACR have led to the creation of a Webbased resource, the ACR Appropriateness Criteria (5). As a peer-reviewed site, this Web site has become exponentially more robust over the past few years and now presents imaging recommendations for more than 200 clinical scenarios. The Web site summarizes current scientific data for the different imaging modalities for specific clinical situations, providing a rating of each test with its pros and cons, along with its relative radiation risk. Despite the creation of this Web site, few physicians outside of radiology are aware of this resource. With only 27 of 126 medical schools (21.4%) requiring radiology clerkships prior to graduation (6), a minority of medical students are ever exposed to imaging in a formal manner. Some radiology clerkships have incorporated imaging algorithms into their curricula. Scheiner and Novelline (7) reported that formal didactic exposure during a dedicated radiology clerkship can significantly improve medical students’ ability to choose appropriate imaging tests and showed that this exposure translates into lower costs and improved patient care. Although evidence-based medicine is covered in other clinical clerkships, evidencebased imaging is a topic that is not covered in any depth in other clinical clerkships, and there are well-documented gaps in student knowledge specific to choosing the appropriate imaging modality for common clinical problems. Given the extent to which imaging influences patient care, it is critical to educate future physicians regarding the evidence behind the imaging algorithms that they observe in practice. Therefore, at our institution, we introduced small-group sessions to teach the principles of evidence-based imaging using the ACR Appropriateness Criteria as part of a required core clerkship in radiology. We undertook a formal evaluation of the student experience to determine whether this structured activity translates into improved understanding of the indications for imaging and whether the Web-based resource may serve as a useful tool for students in their future careers. METHODS As part of the required clerkship in radiology for third-year and fourth-year students at Boston University School of Medicine, a 55-minute didactic section on the principles of evidence-based imaging and a 1.5-hour to 2-hour interactive, clinically focused, small-group session were introduced into the curriculum beginning in July 2008. The first session was given by an attending radiologist, who reviewed the definition of evidence-based imaging and the process of the evidence-based approach. The session culminated with a clinical case of a young woman with a palpable axillary mass to demonstrate how an evidence-based approach leads to using appropriate imaging to guide patient care. Using the ACR Appropriateness Criteria, students were introduced to the algorithms for both patients aged #30 years and those aged >30 years who present with a palpable breast mass. At the end of the session, students understood the evidence behind ordering diagnostic mammography and breast ultrasound for
TEACHING EVIDENCE-BASED IMAGING
this patient. For the second session, students selected one of 25 cases from a variety of topics (Table 1). Working with one or two classmates, the student group created a one-page handout summarizing the data and appropriate imaging algorithm as a way to facilitate an interactive discussion at the second session, either as an informal discussion or by presenting a PowerPoint (Microsoft Corporation, Redmond, WA) presentation. Students were given a week to prepare the case, collaborate with their colleagues, and create their presentations. They were also instructed to review the other 24 cases, on their own or in small groups, to provide an additional opportunity for self-directed learning. Between July 2008 and September 2009, students evaluated the evidence-based imaging sessions using a postclerkship questionnaire given by the clerkship director (Appendix). The questions related to educational content and value and were answered on a five-point scale. The questionnaire also included open-ended questions on the strengths and weaknesses of the evidence-based imaging exercise. Given that the data were collected anonymously and as part of a formal feedback process of the radiology clerkship, this evaluation was exempt from institutional review board approval. RESULTS From July 2008 to September 2009, 259 students completed the required radiology clerkship, with 228 respondents (88%), of whom there were 93 third-year and 135 fourth-year students. Thirty-one students (12%) did not participate in the first didactic session and/or the second small-group session, because of scheduling conflicts. Of the 228 students, 213 of 223 (96%) were not aware of the ACR Appropriateness Criteria as a resource prior to this exercise. Two hundred fourteen of 228 students (94%) found the ACR Appropriateness Criteria a useful resource (Fig 1), and 202 of 228 students (89%) stated that they would likely use this resource in other clinical rotations and in future practice (Fig 2). One hundred eighty-eight of 228 students (82%) felt challenged to think in a different way about radiology as a result of these sessions (Fig 3), while 203 of 227 students (89%) felt they had a solid understanding of the indications for certain imaging tests following the sessions (Fig 4). Only 46 of 228 students (20%) would have preferred a didactic session rather than interactive small-group and self-directed sessions (Fig 5). One hundred fifty-nine of 224 (71%) thought that navigating the ACR’s Web site on their own time to investigate the appropriate imaging tests for the 25 selected diagnoses was a useful self-directed learning exercise (Fig 6). With regard to the open-ended questions, students uniformly praised the opportunity to work in teams on clinically relevant topics. Several students commented on how this exercise was ‘‘applicable to other fields of medicine’’ and ‘‘useful for future career regardless of eventual profession’’ and that ‘‘ACR information [is] useful for upcoming internship.’’ Many students commented this activity was ‘‘a good way to get students involved with each other, a good way to learn algorithms and go through the ACR website and its 913
DILLON AND SLANETZ
Academic Radiology, Vol 17, No 7, July 2010
TABLE 1. Selected Topics for Evidence-based Imaging Session Pneumonia Pulmonary embolism Solitary pulmonary nodule on preoperative chest film Blunt chest trauma Back pain Shoulder dislocation Hematuria Renal colic Acute scrotal pain
Aortic dissection Colorectal cancer screening Suspected MI/acute chest pain New-onset seizures Acute stroke Headache in HIV-positive adult Acute head trauma in child UTI in a child
Acute appendicitis Small-bowel obstruction Abnormal screening mammographic results Jaundice with RUQ pain Acute GI bleeding Vaginal spotting in postmenopausal woman Blunt abdominal trauma Acute pancreatitis
GI, gastrointestinal; HIV, human immunodeficiency virus; MI, myocardial infarction; RUQ, right upper quadrant; UTI, urinary tract infection.
68%
160 140 120 # of 100 respondents 80
25%
60 40 20
0
5% 0%
1%
1
2
43%
100 90 80 70 60 # of respondents 50 40 30 20 10 0
15%
2% 0%
1
3
4
5
2
3
Figure 1. Responses to question 2: ‘‘I found the ACR Appropriateness Criteria a useful resource.’’ ACR, American College of Radiology. 57%
44%
80
# of respondents
100 32%
80
60 40
60
10%
20
40
0%
1%
1
2
0
<1%
5%
6%
1
2
3
3
4
5
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
0 4
5
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
Figure 2. Responses to question 3: ‘‘I will likely use the ACR Appropriateness Criteria as a resource in future clinical rotations and/or when I am eventually in clinical practice.’’ ACR, American College of Radiology.
usefulness.’’ A majority of students stated that the sessions helped students ‘‘understand what studies to order for specific diagnoses in terms of risks and benefits.’’ Overall, students found that this exercise was an easy method by which ‘‘selfdirected learning in imaging algorithms consolidates [their] book knowledge.’’ With regard to weaknesses, some students found it ‘‘difficult to determine how cost fits into decision’’ because the ACR’s Web site and few online resources provide information regarding the relative costs of different imaging tests. Some 914
46%
100
120
20
5
Figure 3. Responses to question 4: ‘‘The imaging algorithm sessions challenged me to think in a different way about radiology.’’
120
# of respondents
4
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
140
39%
Figure 4. Responses to question 5: ‘‘I left the imaging algorithm sessions with a good sense of the indications for certain imaging studies.’’
students found that the activity was ‘‘time-consuming’’ or that ‘‘certain topics are time intensive.’’ A minority commented that as the sessions necessitated small-group work, the overall benefit from the exercise was ‘‘dependent on effort of students’’ and hence did depend on which students were members of each small group. DISCUSSION In the United States, the type and frequency of diagnostic radiologic and nuclear medicine studies increased approximately 10-fold and 2.5-fold, respectively, between 1980 and 2006, while the annual per capita effective dose from radiologic and nuclear medicine studies increased by approximately
Academic Radiology, Vol 17, No 7, July 2010
80
TEACHING EVIDENCE-BASED IMAGING
32%
70 60
23%
50
# of respondents 40
25%
12%
30
8%
20 10 0 1
2
3
4
5
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
Figure 5. Responses to question 6: ‘‘I would much rather have learned this information in a didactic format given by an attending radiologist.’’
80
36%
36%
4
5
70 60
24%
50
# of respondents 40 30 20 10
4% 1%
0 1
2
3
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
Figure 6. Responses to question 7: ‘‘Navigating the ACR website on my own time to investigate the appropriate imaging test for the selected 25 diagnoses was a useful self-directed learning exercise.’’ ACR, American College of Radiology.
600% during that same period (8). In this setting, coupled with ubiquitous concerns about the cost of health care, there has perhaps never been a more important time to teach medical students the principles of evidence-based imaging. In fact, a recent proposal to the health care reform bill being considered by the Senate Finance Committee provides for higher payments to physicians who use appropriateness criteria in guiding clinical decisions (9). In this study, we assessed the educational content and value of a small-group exercise teaching the principles of evidencebased imaging using the ACR Appropriateness Criteria. We found that although students had a good understanding of the principles of evidence-based medicine, the majority were not familiar with the concepts of evidence-based imaging, and very few were aware of the ACR Appropriateness Criteria as a resource. Once introduced to this resource, however, the great majority of students found the ACR Appropriateness Criteria not only useful but helpful to the extent that they plan to use them in other clerkships and in their future practice. With only 21% of American medical schools requiring clerkships in radiology, medical student exposure to radiology
is sharply limited, including not only instruction in the interpretation of imaging studies but also teaching the evidence supporting the use of specific imaging tests for particular clinical presentations. It has been well documented that prior to formal exposure to radiology, students have substantial gaps in knowledge regarding the choice of appropriate imaging tests for their patients. In this study, following the two evidencebased imaging sessions, >80% of the students thought differently about radiology and nearly 90% felt that they had gained a solid understanding of the indications for the use of various imaging tests for specific common clinical scenarios. Although most students found the exercise useful for learning about the rationale behind imaging for various clinical scenarios, some students would have preferred to cover more scenarios per session. In addition, with only eight to 10 of the 25 possible cases covered per session, some students would have preferred to have a second small-group session to cover all of the 25 scenarios rather than being expected to review the remaining cases on their own time. A few students noted that navigating the ACR’s Web site was timeconsuming, and several students suggested that the ACR Appropriateness Criteria Web site would be enhanced if relative cost data were included similar to those provided for relative radiation exposure. One fifth of the students preferred a didactic session instead of the interactive small-group and self-directed sessions. Overall, most students (71%) found that using the ACR’s Web site on their own time was useful. They felt that they gained greater understanding of which tests to order for common clinical scenarios and increased awareness of the risks of imaging tests, including exposure to ionizing radiation and intravenous contrast media. Multiple students expressed that they enjoyed the team approach and the opportunity to explore common clinical scenarios that they would likely encounter during their careers. In conclusion, the ACR Appropriateness Criteria are a valuable and easy accessible resource for teaching evidence-based imaging to medical students. When incorporated into a structured radiology clerkship, a majority of students intend to use this resource in future clinical rotations and their career. With an improved understanding of the science behind imaging, students and future physicians will likely provide more cost-effective care to their patients. REFERENCES 1. Smith-Bindman R, Miglioretti DL, Larson EB. Rising use of diagnostic medical imaging in a large integrated health system. Health Aff (Millwood) 2008; 27:1491–1502. 2. Rowan K. Rising costs of medical imaging spur debate. J Natl Cancer Inst 2008; 100:1665–1667. 3. Studdert DM, Mello MM, Sage WM, et al. Defensive medicine among highrisk specialist physicians in a volatile malpractice environment. JAMA 2005; 293:2609–2617. 4. Ertuk SM, Ondategui-Parra S, Otero H, et al. Evidence-based radiology. J Am Coll Radiol 2006; 3:513–519. 5. American College of Radiology. ACR Appropriateness Criteria. Available at: http://www.acr.org/secondarymainmenucategories/quality_safety/app_ criteria.aspx. Accessed March 17, 2010.
915
DILLON AND SLANETZ
6. Association of American Medical Colleges. Curriculum directory. Available at: http://services.aamc.org/currdir/index.cfm. Accessed November 18, 2009. 7. Scheiner JD, Novelline RA. Radiology clerkships are necessary for teaching medical students appropriate imaging work-ups. Acad Radiol 2000; 7:40–45. 8. Mettler FA Jr, Bhargavan M, Faulkner K, et al. Radiologic and nuclear medicine studies in the United States and worldwide: frequency, radiation dose, and comparison with other radiation sources—1950-2007. Radiology 2009; 253:520–531. 9. Abella HA. Senate bill ties physicians’ pay to appropriateness criteria. Available at: http://www.diagnosticimaging.com/news/display/article/113619/ 1470726?verify=0. Accessed March 17, 2010.
APPENDIX Evidence-based Imaging Session Questionnaire
Thank you in advance for your thoughtful answers to the following questions. Your feedback regarding the small group sessions in this clerkship is greatly appreciated as we use it to continue to improve the clerkship every year. Are you a third or fourth year student (please circle): 3rd 4th If you are a fourth year student, what field will you be pursuing? Using a 5-point scale (5 being strongly agree and 1 being strongly disagree), please answer the following questions: 1. Prior to this exercise, I was not aware of the ACR Appropriateness Criteria. [ ] Yes [ ] No 2. I found the ACR Appropriateness Criteria a useful resource. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree)
916
Academic Radiology, Vol 17, No 7, July 2010
3. I will likely use the ACR Appropriateness Criteria as a resource in future clinical rotations and/or when I am eventually in clinical practice. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree) 4. The imaging algorithm sessions challenged me to think in a different way about radiology. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree) 5. I left the imaging algorithm sessions with a good sense of the indications for certain imaging studies. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree) 6. I would much rather have learned this information in a didactic format given by an attending radiologist. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree) 7. Navigating the ACR website on my own time to investigate the appropriate imaging test for the selected 25 diagnoses was a useful self-directed learning exercise. [ ] [ ] [ ] [ ] [ ] 5 4 3 2 1 (strongly agree) (strongly disagree) Please list three strengths/weaknesses of this activity:
Teaching Evidence-Based Imaging in the Radiology Clerkship Using the ACR Appropriateness Criteria Johanne E. Dillon, MD, Priscilla J. Slanetz, MD, MPH, FACR Rationale and Objectives: Given escalating health care costs and limited resources, the effective utilization of imaging is essential to permit optimal patient care. Because only 21.4% of medical schools require radiology clerkships, most students do not understand the indications for and clinical effectiveness of most imaging tests. Therefore, the authors introduced two focused sessions on evidencebased imaging during the required radiology core clerkship at their institution. Methods: From June 2008 to September 2009, 259 students in the radiology core clerkship participated in a didactic session on principles of evidence-based imaging and/or a small-group session and self-directed learning exercise using the American College of Radiology (ACR) Appropriateness Criteria. Students evaluated the exercise on a five-point scale for its effectiveness in teaching imaging strategies and its impact on their future careers. Results: Two hundred twenty-eight of 259 students (88%; 93 third year, 135 fourth year) participated in the two sessions. Two hundred thirteen of 223 (96%) were not aware of the ACR’s Web site as a resource prior to this exercise. Two hundred fourteen of 228 (94%) found the ACR Appropriateness Criteria a useful resource. Two hundred two of 228 (89%) stated that they would likely use this resource in other clinical rotations and future practice. Two hundred three of 228 (89%) reported having a solid understanding of the indications for imaging tests following the sessions. Students praised the ACR’s Web site for its comprehensive coverage of different modalities and their relative radiation risks. They valued working collaboratively on common clinical scenarios. Conclusion: The ACR Appropriateness Criteria are a valuable resource for teaching evidence-based imaging to medical students. A majority of students indicated that they plan to use this resource in the future. Key Words: Evidence-based imaging; evidence-based medicine; cost-effective care; appropriateness criteria; radiology; medical education. ªAUR, 2010
T
he increased utilization of imaging due to the introduction of new technology over the past few decades has been linked to escalating health care costs (1,2). Often, new imaging techniques are widely adopted prior to establishing their cost-effectiveness. This widespread incorporation of technology into patient care is multifactorial and includes rising patient expectations for earlier diagnosis, improved access to imaging, added diagnostic certainty for providers, increased reliance on technology for answers, and concern over potential liability (3). Given limited resources, there is a growing need to educate physicians about the strengths and limitations of different imaging modalities for specific clinical presentations. Although imaging tests can Acad Radiol 2010; 17:912–916 From the Department of Radiology, Boston Medical Center and Boston University School of Medicine, Boston, MA (J.E.D.); and the Department of Radiology, Division of Breast Imaging, Shapiro Clinical Center, 4th Floor, Beth Israel Deaconess Medical Center and Harvard Medical School, 330 Brookline Avenue, Boston, MA 02215 (P.J.S.). Received January 28, 2010; accepted March 3, 2010. Address correspondence to: P.J.S. e-mail: [email protected] ªAUR, 2010 doi:10.1016/j.acra.2010.03.001
912
provide useful information that may directly affect patient care, physicians must use clinical histories and findings on physical examinations to guide them to the appropriate differential diagnosis and subsequent selection of laboratory and imaging tests. Without the benefit of an adequate history and physical examination, providers are apt to rely on a battery of tests, leading to untoward risk to patients and unnecessary costs to the health care system. Knowledge regarding the cost-effectiveness of different imaging modalities is essential to optimize patient care. Over the past decade, there has been increasing interest in evidence-based medicine and, more recently, evidence-based imaging. Defined as combining the best scientific evidence with physician expertise and patient expectations (4), evidence-based imaging can be expected to play a pivotal role to help physicians choose the ‘‘best’’ or most cost effective imaging tests for their patients. This approach will likely translate into stabilization of health care costs related to imaging and more appropriate patient care, as no imaging test is without risk. The American College of Radiology (ACR) has been interested in evidence-based imaging for over a decade. The
Academic Radiology, Vol 17, No 7, July 2010
national efforts of the ACR have led to the creation of a Webbased resource, the ACR Appropriateness Criteria (5). As a peer-reviewed site, this Web site has become exponentially more robust over the past few years and now presents imaging recommendations for more than 200 clinical scenarios. The Web site summarizes current scientific data for the different imaging modalities for specific clinical situations, providing a rating of each test with its pros and cons, along with its relative radiation risk. Despite the creation of this Web site, few physicians outside of radiology are aware of this resource. With only 27 of 126 medical schools (21.4%) requiring radiology clerkships prior to graduation (6), a minority of medical students are ever exposed to imaging in a formal manner. Some radiology clerkships have incorporated imaging algorithms into their curricula. Scheiner and Novelline (7) reported that formal didactic exposure during a dedicated radiology clerkship can significantly improve medical students’ ability to choose appropriate imaging tests and showed that this exposure translates into lower costs and improved patient care. Although evidence-based medicine is covered in other clinical clerkships, evidencebased imaging is a topic that is not covered in any depth in other clinical clerkships, and there are well-documented gaps in student knowledge specific to choosing the appropriate imaging modality for common clinical problems. Given the extent to which imaging influences patient care, it is critical to educate future physicians regarding the evidence behind the imaging algorithms that they observe in practice. Therefore, at our institution, we introduced small-group sessions to teach the principles of evidence-based imaging using the ACR Appropriateness Criteria as part of a required core clerkship in radiology. We undertook a formal evaluation of the student experience to determine whether this structured activity translates into improved understanding of the indications for imaging and whether the Web-based resource may serve as a useful tool for students in their future careers. METHODS As part of the required clerkship in radiology for third-year and fourth-year students at Boston University School of Medicine, a 55-minute didactic section on the principles of evidence-based imaging and a 1.5-hour to 2-hour interactive, clinically focused, small-group session were introduced into the curriculum beginning in July 2008. The first session was given by an attending radiologist, who reviewed the definition of evidence-based imaging and the process of the evidence-based approach. The session culminated with a clinical case of a young woman with a palpable axillary mass to demonstrate how an evidence-based approach leads to using appropriate imaging to guide patient care. Using the ACR Appropriateness Criteria, students were introduced to the algorithms for both patients aged #30 years and those aged >30 years who present with a palpable breast mass. At the end of the session, students understood the evidence behind ordering diagnostic mammography and breast ultrasound for
TEACHING EVIDENCE-BASED IMAGING
this patient. For the second session, students selected one of 25 cases from a variety of topics (Table 1). Working with one or two classmates, the student group created a one-page handout summarizing the data and appropriate imaging algorithm as a way to facilitate an interactive discussion at the second session, either as an informal discussion or by presenting a PowerPoint (Microsoft Corporation, Redmond, WA) presentation. Students were given a week to prepare the case, collaborate with their colleagues, and create their presentations. They were also instructed to review the other 24 cases, on their own or in small groups, to provide an additional opportunity for self-directed learning. Between July 2008 and September 2009, students evaluated the evidence-based imaging sessions using a postclerkship questionnaire given by the clerkship director (Appendix). The questions related to educational content and value and were answered on a five-point scale. The questionnaire also included open-ended questions on the strengths and weaknesses of the evidence-based imaging exercise. Given that the data were collected anonymously and as part of a formal feedback process of the radiology clerkship, this evaluation was exempt from institutional review board approval. RESULTS From July 2008 to September 2009, 259 students completed the required radiology clerkship, with 228 respondents (88%), of whom there were 93 third-year and 135 fourth-year students. Thirty-one students (12%) did not participate in the first didactic session and/or the second small-group session, because of scheduling conflicts. Of the 228 students, 213 of 223 (96%) were not aware of the ACR Appropriateness Criteria as a resource prior to this exercise. Two hundred fourteen of 228 students (94%) found the ACR Appropriateness Criteria a useful resource (Fig 1), and 202 of 228 students (89%) stated that they would likely use this resource in other clinical rotations and in future practice (Fig 2). One hundred eighty-eight of 228 students (82%) felt challenged to think in a different way about radiology as a result of these sessions (Fig 3), while 203 of 227 students (89%) felt they had a solid understanding of the indications for certain imaging tests following the sessions (Fig 4). Only 46 of 228 students (20%) would have preferred a didactic session rather than interactive small-group and self-directed sessions (Fig 5). One hundred fifty-nine of 224 (71%) thought that navigating the ACR’s Web site on their own time to investigate the appropriate imaging tests for the 25 selected diagnoses was a useful self-directed learning exercise (Fig 6). With regard to the open-ended questions, students uniformly praised the opportunity to work in teams on clinically relevant topics. Several students commented on how this exercise was ‘‘applicable to other fields of medicine’’ and ‘‘useful for future career regardless of eventual profession’’ and that ‘‘ACR information [is] useful for upcoming internship.’’ Many students commented this activity was ‘‘a good way to get students involved with each other, a good way to learn algorithms and go through the ACR website and its 913
DILLON AND SLANETZ
Academic Radiology, Vol 17, No 7, July 2010
TABLE 1. Selected Topics for Evidence-based Imaging Session Pneumonia Pulmonary embolism Solitary pulmonary nodule on preoperative chest film Blunt chest trauma Back pain Shoulder dislocation Hematuria Renal colic Acute scrotal pain
Aortic dissection Colorectal cancer screening Suspected MI/acute chest pain New-onset seizures Acute stroke Headache in HIV-positive adult Acute head trauma in child UTI in a child
Acute appendicitis Small-bowel obstruction Abnormal screening mammographic results Jaundice with RUQ pain Acute GI bleeding Vaginal spotting in postmenopausal woman Blunt abdominal trauma Acute pancreatitis
GI, gastrointestinal; HIV, human immunodeficiency virus; MI, myocardial infarction; RUQ, right upper quadrant; UTI, urinary tract infection.
68%
160 140 120 # of 100 respondents 80
25%
60 40 20
0
5% 0%
1%
1
2
43%
100 90 80 70 60 # of respondents 50 40 30 20 10 0
15%
2% 0%
1
3
4
5
2
3
Figure 1. Responses to question 2: ‘‘I found the ACR Appropriateness Criteria a useful resource.’’ ACR, American College of Radiology. 57%
44%
80
# of respondents
100 32%
80
60 40
60
10%
20
40
0%
1%
1
2
0
<1%
5%
6%
1
2
3
3
4
5
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
0 4
5
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
Figure 2. Responses to question 3: ‘‘I will likely use the ACR Appropriateness Criteria as a resource in future clinical rotations and/or when I am eventually in clinical practice.’’ ACR, American College of Radiology.
usefulness.’’ A majority of students stated that the sessions helped students ‘‘understand what studies to order for specific diagnoses in terms of risks and benefits.’’ Overall, students found that this exercise was an easy method by which ‘‘selfdirected learning in imaging algorithms consolidates [their] book knowledge.’’ With regard to weaknesses, some students found it ‘‘difficult to determine how cost fits into decision’’ because the ACR’s Web site and few online resources provide information regarding the relative costs of different imaging tests. Some 914
46%
100
120
20
5
Figure 3. Responses to question 4: ‘‘The imaging algorithm sessions challenged me to think in a different way about radiology.’’
120
# of respondents
4
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
140
39%
Figure 4. Responses to question 5: ‘‘I left the imaging algorithm sessions with a good sense of the indications for certain imaging studies.’’
students found that the activity was ‘‘time-consuming’’ or that ‘‘certain topics are time intensive.’’ A minority commented that as the sessions necessitated small-group work, the overall benefit from the exercise was ‘‘dependent on effort of students’’ and hence did depend on which students were members of each small group. DISCUSSION In the United States, the type and frequency of diagnostic radiologic and nuclear medicine studies increased approximately 10-fold and 2.5-fold, respectively, between 1980 and 2006, while the annual per capita effective dose from radiologic and nuclear medicine studies increased by approximately
Academic Radiology, Vol 17, No 7, July 2010
80
TEACHING EVIDENCE-BASED IMAGING
32%
70 60
23%
50
# of respondents 40
25%
12%
30
8%
20 10 0 1
2
3
4
5
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
Figure 5. Responses to question 6: ‘‘I would much rather have learned this information in a didactic format given by an attending radiologist.’’
80
36%
36%
4
5
70 60
24%
50
# of respondents 40 30 20 10
4% 1%
0 1
2
3
5-point scale: 1 = Strongly disagree to 5 = Strongly agree
Figure 6. Responses to question 7: ‘‘Navigating the ACR website on my own time to investigate the appropriate imaging test for the selected 25 diagnoses was a useful self-directed learning exercise.’’ ACR, American College of Radiology.
600% during that same period (8). In this setting, coupled with ubiquitous concerns about the cost of health care, there has perhaps never been a more important time to teach medical students the principles of evidence-based imaging. In fact, a recent proposal to the health care reform bill being considered by the Senate Finance Committee provides for higher payments to physicians who use appropriateness criteria in guiding clinical decisions (9). In this study, we assessed the educational content and value of a small-group exercise teaching the principles of evidencebased imaging using the ACR Appropriateness Criteria. We found that although students had a good understanding of the principles of evidence-based medicine, the majority were not familiar with the concepts of evidence-based imaging, and very few were aware of the ACR Appropriateness Criteria as a resource. Once introduced to this resource, however, the great majority of students found the ACR Appropriateness Criteria not only useful but helpful to the extent that they plan to use them in other clerkships and in their future practice. With only 21% of American medical schools requiring clerkships in radiology, medical student exposure to radiology
is sharply limited, including not only instruction in the interpretation of imaging studies but also teaching the evidence supporting the use of specific imaging tests for particular clinical presentations. It has been well documented that prior to formal exposure to radiology, students have substantial gaps in knowledge regarding the choice of appropriate imaging tests for their patients. In this study, following the two evidencebased imaging sessions, >80% of the students thought differently about radiology and nearly 90% felt that they had gained a solid understanding of the indications for the use of various imaging tests for specific common clinical scenarios. Although most students found the exercise useful for learning about the rationale behind imaging for various clinical scenarios, some students would have preferred to cover more scenarios per session. In addition, with only eight to 10 of the 25 possible cases covered per session, some students would have preferred to have a second small-group session to cover all of the 25 scenarios rather than being expected to review the remaining cases on their own time. A few students noted that navigating the ACR’s Web site was timeconsuming, and several students suggested that the ACR Appropriateness Criteria Web site would be enhanced if relative cost data were included similar to those provided for relative radiation exposure. One fifth of the students preferred a didactic session instead of the interactive small-group and self-directed sessions. Overall, most students (71%) found that using the ACR’s Web site on their own time was useful. They felt that they gained greater understanding of which tests to order for common clinical scenarios and increased awareness of the risks of imaging tests, including exposure to ionizing radiation and intravenous contrast media. Multiple students expressed that they enjoyed the team approach and the opportunity to explore common clinical scenarios that they would likely encounter during their careers. In conclusion, the ACR Appropriateness Criteria are a valuable and easy accessible resource for teaching evidence-based imaging to medical students. When incorporated into a structured radiology clerkship, a majority of students intend to use this resource in future clinical rotations and their career. With an improved understanding of the science behind imaging, students and future physicians will likely provide more cost-effective care to their patients. REFERENCES 1. Smith-Bindman R, Miglioretti DL, Larson EB. Rising use of diagnostic medical imaging in a large integrated health system. Health Aff (Millwood) 2008; 27:1491–1502. 2. Rowan K. Rising costs of medical imaging spur debate. J Natl Cancer Inst 2008; 100:1665–1667. 3. Studdert DM, Mello MM, Sage WM, et al. Defensive medicine among highrisk specialist physicians in a volatile malpractice environment. JAMA 2005; 293:2609–2617. 4. Ertuk SM, Ondategui-Parra S, Otero H, et al. Evidence-based radiology. J Am Coll Radiol 2006; 3:513–519. 5. American College of Radiology. ACR Appropriateness Criteria. Available at: http://www.acr.org/secondarymainmenucategories/quality_safety/app_ criteria.aspx. Accessed March 17, 2010.
915
DILLON AND SLANETZ
6. Association of American Medical Colleges. Curriculum directory. Available at: http://services.aamc.org/currdir/index.cfm. Accessed November 18, 2009. 7. Scheiner JD, Novelline RA. Radiology clerkships are necessary for teaching medical students appropriate imaging work-ups. Acad Radiol 2000; 7:40–45. 8. Mettler FA Jr, Bhargavan M, Faulkner K, et al. Radiologic and nuclear medicine studies in the United States and worldwide: frequency, radiation dose, and comparison with other radiation sources—1950-2007. Radiology 2009; 253:520–531. 9. Abella HA. Senate bill ties physicians’ pay to appropriateness criteria. Available at: http://www.diagnosticimaging.com/news/display/article/113619/ 1470726?verify=0. Accessed March 17, 2010.
APPENDIX Evidence-based Imaging Session Questionnaire
Thank you in advance for your thoughtful answers to the following questions. Your feedback regarding the small group sessions in this clerkship is greatly appreciated as we use it to continue to improve the clerkship every year. Are you a third or fourth year student (please circle): 3rd 4th If you are a fourth year student, what field will you be pursuing? Using a 5-point scale (5 being strongly agree and 1 being strongly disagree), please answer the following questions: 1. Prior to this exercise, I was not aware of the ACR Appropriateness Criteria. [ ] Yes [ ] No 2. I found the ACR Appropriateness Criteria a useful resource. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree)
916
Academic Radiology, Vol 17, No 7, July 2010
3. I will likely use the ACR Appropriateness Criteria as a resource in future clinical rotations and/or when I am eventually in clinical practice. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree) 4. The imaging algorithm sessions challenged me to think in a different way about radiology. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree) 5. I left the imaging algorithm sessions with a good sense of the indications for certain imaging studies. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree) 6. I would much rather have learned this information in a didactic format given by an attending radiologist. [] [] [] [] [] 5 4 3 2 1 (strongly agree) (strongly disagree) 7. Navigating the ACR website on my own time to investigate the appropriate imaging test for the selected 25 diagnoses was a useful self-directed learning exercise. [ ] [ ] [ ] [ ] [ ] 5 4 3 2 1 (strongly agree) (strongly disagree) Please list three strengths/weaknesses of this activity: