- Email: [email protected]
Radiology Report Turnaround Time
Radiology Resident Education Joseph E. and Nancy O. Whitley Award Winning Paper
Radiology Report Turnaround Time: Effect on Resident Education Eric England, MD, Jannette Collins, MD, MEd, FCCP, FACR, Richard D. White, MD, FACR, FACC, FAHA, FSCCT, F. Jacob Seagull, PhD, John Deledda, MD Rationale and Objectives: To compare resident workload from Emergency Department (ED) studies before and after the implementation of a required 1-hour report turnaround time (TAT) and to assess resident and faculty perception of TAT on resident education. Materials and Methods: Resident study volume will be compared for 3 years before and 1 year after the implementation of a required 1hour TAT. Changes to resident workload will be compared among the different radiology divisions (body, muscuolskeletal (MSK), chest, and neuro), as well as during different shifts (daytime and overnight). Residents and faculty at two Midwest institutions, both of which have a required report TAT, will be invited to participate in an online survey to query the perceived effect on resident education by implementation of this requirement. A P < .05 was considered statistically significant. Results: A significant decrease in resident involvement in ED studies was noted in the MSK, chest, and neuro sections with average involvement of the 3 years before the 1-hour TAT of 89%, 88%, and 82%, respectively, which decreased to 66%, 68%, and 51% after the 1-hour TAT requirement (P < .05). The resident involvement in ED studies only mildly decreased in the body section from an average before the 1-hour TAT of 87% to 80% after the 1-hour TAT requirement (P < .1). There was an overall significant decrease in resident ED study involvement during the daytime (P = .01) but not after hours during resident call (P = .1). Seventy percent of residents (43 of 61) and 55% of faculty (63 of 114) responded to our surveys. Overall, residents felt their education from ED studies during the daytime and overnight were good. However, residents who were present both before and after the implementation of a required TAT felt their education had been significantly negatively affected. Faculty surveyed thought that the required TAT negatively affected their ability to teach and decreased the quality of resident education. Conclusions: Residents are exposed to fewer ED studies after the implementation of a required 1-hour TAT. Overall, the current residents do not feel this decreased exposure to Emergency room studies affects their education. However, residents in training before and after this requirement feel their education has been significantly affected. Faculty perceives that the required TAT negatively affects their ability to teach, as well as the quality of resident education. Key Words: Resident education; turnaround time; Emergency Department. ªAUR, 2015
O
ver the past several years, radiology residents have experienced major changes to their training and certification. From the growing emphasis on diagnostic milestones for resident training to the transition from an oral board examination to a more image-rich computer-based examination, radiology resident training is in a state of rapid evolution (1,2). In addition, the transition from film with tape/digitized recordings to a picture archiving and Acad Radiol 2015; 22:662–667 From the Department of Radiology, University of Cincinnati Medical Center, 234 Goodman Street, ML 0761, Cincinnati, OH 45267-0761 (E.E.); University of Cincinnati College of Medicine, Cincinnati, Ohio (J.C.); Department of Radiology, The Ohio State University Wexner Medical Center, Colombus, Ohio (R.D.W.); Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan (F.J.S.); and Department of Emergency Medicine, University of Cincinnati Medical Center, Cincinnati, Ohio (J.D.). Received October 3, 2014; accepted December 10, 2014. Address correspondence to: E.E. e-mail: [email protected] ªAUR, 2015 http://dx.doi.org/10.1016/j.acra.2014.12.023
662
communication system (PACS) with speech recognition has dramatically changed the way academic radiologists process and review cases with residents (3). More recently, the effect on radiology resident education related to 24/7/365 in-house attending coverage was investigated (4). Another pressure now challenges academic radiologists further with a new variable to which they must adapt: the implementation of required reduced radiology report turnaround times (RTATs), potentially taking away from time spent in instruction of residents. RTAT has increasingly become an important metric for measuring the quality of diagnostic radiology services. With the institution of PACS and speech recognition systems, RTAT has decreased over the last decade from days to nearreal time (5–7). Increasingly, teleradiology companies have emphasized decreased RTAT to hospitals as a metric for improved services to compete with local radiology groups, or even other teleradiology companies (8). Some academic teaching hospitals have experimented with pay-for-performance programs to decrease their RTAT (9). This simple metric can
Academic Radiology, Vol 22, No 5, May 2015
easily be assigned a value by hospital administration and can be used to evaluate current radiology services and compare them to competing radiology services. The increased interest in this metric by radiology departments and hospital administrators over the last several years has led to hospital radiology RTATrequirements, particularly in the cases of Emergency Department (ED) studies and ‘‘stat’’ interpretations. A survey of all Emergency Medicine chairs in 2002 showed that 49% of respondents were dissatisfied with radiology RTAT (10); it also showed that 39% of sites reported RTAT <4 hours and that only 2% of daytime radiology reports were completed in <1 hour. Although continued adoption of PACS and speech recognition systems has likely improved radiologists’ RTAT since this survey was conducted, the importance that Emergency Medicine physicians place on prompt accurate final interpretations cannot be overstated. To address this concern from Emergency Medicine physicians, the American College of Radiology issued a resolution in 2001 that ‘‘all radiologic studies performed on Emergency Department patients should be promptly interpreted by radiologists’’ (11). A coordinated effort on the part of our Radiology and Emergency Medicine departments to improve radiologists’ RTAT led to the adoption of a 24/7 1-hour turnaround time (TAT) requirement for all studies performed in the ED. The goals of this requirement were to decrease patient wait time in the ED by increasing patient throughput and to improve patient care by eliminating ‘‘resident discrepancies’’ from preliminary reports, although the latter rationale remains controversial (12–15). The requirement led to the hiring of on-site dedicated nighttime and evening radiologists in 2012 to ensure that the RTAT metric was met throughout the night. The daytime ED studies are typically interpreted by a subspecialty division but with the continued requirement for a 1-hour RTAT. We hypothesized the following: 1) 1 year after the implementation of the RTAT requirement, residents would become less involved in ED studies to comply with the metric, 2) faculty would be incentivized to forego resident involvement and dictate ED studies by themselves to decrease their RTAT, and 3) faculty and resident satisfaction with resident education would decrease as more emphasis was placed on report throughput and less emphasis was placed on resident education. MATERIALS AND METHODS Emergency Department Study Volume
A retrospective review of the medical records was performed to identify patients who had the following ED studies: radiographs of the chest, abdomen, knee, and pelvis; computed tomography (CT) examinations of the chest, abdomen, pelvis, brain, cervical spine, thoracic spine, and lumbar spine; and magnetic resonance of the lumbar spine and brain. The total number of studies performed from the ED for each of these modalities and the percent of these studies that were dictated by a resident were obtained for 1 year after the implementation of the 1-hour RTAT (July 1, 2012 to June 30, 2013) and for 3 years before the 1-hour
RTAT: EFFECT ON RESIDENT EDUCATION
RTAT (July 1, 2009 to June 30, 2010; July 1, 2010 to June 30, 2011; and July 1, 2011 to June 30, 2012). The imaging studies were divided according to subspecialty section (spine studies were interpreted by the neuroradiology division.) The studies were further divided according to when they were performed, with ‘‘daytime’’ studies performed between 8 AM–5 PM and ‘‘call’’ studies performed between 5 PM–8 AM. The percent of resident-dictated cases were compared before and after the implementation of the RTAT requirement and during the different daytime/call periods using a paired t test. Changes in resident ED study volume were also assessed by subspecialty section. A P value of #.05 was considered significant. Surveys
An eight-question radiology resident survey and a ninequestion radiology faculty survey were sent to two academic radiology programs that had adopted a 1-hour RTAT requirement (Appendix, resident survey, faculty survey). These surveys addressed resident/faculty demographics, perceived effects of resident case exposure, and effects on resident education/teaching. These surveys were distributed online (via SurveyMonkey). Each resident and faculty member was sent an email requesting voluntary participation in this anonymous online survey. Consent was given online before the survey was completed. Residents and faculty had 2 weeks to complete the survey. An email reminder was sent after 1 week. Responses are presented as frequency distributions and were assessed by calculating the 95% confidence interval (CI) for the mean (P < .05). Intergroup comparisons were assessed using t tests. The study was approved by the institutional review board at both institutions. RESULTS Emergency Department Study Volume
The average percent of ED studies dictated by residents during daytime hours (8 AM–5 PM) decreased significantly after the implementation of a required 1-hour RTAT (P = .01; Table 1). On the other hand, there was not a statistically significant decrease in the mean percent of resident ED study involvement before and after the required 1-hour RTAT during call (5–8 PM; P = .10; Table 2). However, when taking both periods into account, there was overall a significant decrease in resident ED study involvement after implementation of the 1-hour RTAT (P = .027). Among the four divisions, musculoskeletal (MSK), chest, and neuroradiology had a significant drop in resident ED study involvement after the implementation of the 1-hour RTAT. The MSK division saw a decrease from an average of 88.68% (CI = 81–96; P = .006) to 65.74% (Fig 1). The chest division saw a decrease from an average of 88.36% (CI = 78– 99; P = .015) to 68.12% (Fig 2). The neuroradiology division saw the most substantial decrease from an average of 82.89% 663
ENGLAND ET AL
Academic Radiology, Vol 22, No 5, May 2015
TABLE 1. Percent of ED Studies Dictated by Residents During Daytime (8 AM–5 PM) Hours After Average of 3 years P Before Required Implementation of Required RTAT (%) Value RTAT (%) Body Musculoskeletal Chest Neuro All sections
86.71 86.79 85.16 65.55 81.05
71.87 55.98 45.35 28.10 50.33
.05 .009 .008 .04 .01
ED, Emergency Department; RTAT, radiology report turnaround time. A significant decrease of resident ED study dictations during daytime hours before and after implementation of a required 1-hour RTAT was observed.
Seventy percent (43 of 61) of residents participated in the online survey. In general, the residents thought that there were adequate opportunities to interpret studies (range, 1–2; 1 = adequate and 2 = inadequate; mean, 1.84; CI, 1.73– 1.95) with the required 1-hour RTAT metric in place. Residents also responded that learning opportunities from ED studies both during the daytime shift (range, 1–5; 1 = strongly disagree and 5 = strongly agree; mean, 3.42; CI, 3.14–3.70) and while on-call (range, 1–5; 1 = strongly disagree and 5 = strongly agree; mean, 4.02; CI, 3.76–4.28) were good. However, those residents in training both before and after the implementation of a required RTAT thought that the required RTAT significantly decreased their education (range, 1–3; 1 = worsened and 3 = improved; mean, 1.21; CI, 1.01–1.41). Comments
TABLE 2. Percent of ED Studies Dictated by Residents While on Call (5 PM–8 AM)
Body Musculoskeletal Chest Neuro All sections
Average of 3 years Before Required RTAT (%)
After Implementation of Required RTAT (%)
P Value
87.73 89.92 90.27 90.72 89.66
84.59 73.09 85.03 64.59 76.83
.35 .025 .08 .015 .1
ED, Emergency Department; RTAT, radiology report turnaround time. No statistically significant change (P = .1) of resident ED study involvement during call hours before and after implementation of a required 1-hour RTAT was observed.
(CI = 73–93; P = .005) to 51.25% (Fig 3). The only division to not see a statistically significant decrease in resident ED study involvement was the body division which only saw a slight decrease from an average of 87.37% (CI = 77–98; P = .1) to 80.24% (Fig 4). Surveys
Fifty-five percent (63 of 114) of faculty completed the online survey. The faculty responded that the 1-hour RTAT negatively influenced the total number of studies that they completed with residents (range, 1–5; 1 = greatly decreased and 5 = greatly increased; mean, 4.08; CI, 3.84–4.33). They also noted that the 1-hour RTAT decreased their ability to teach residents (range, 1–5; 1 = greatly decreased and 5 = greatly increased; mean, 2.34; CI, 2.10–2.57). Those faculty that were on staff both before and after the implementation of the 1-hour RTAT requirement noted that the new requirement significantly decreased time spent teaching residents (range, 1–3; 1 = decreased teaching and 3 = increased teaching; mean, 1.26; CI, 1.12–1.4) and significantly impaired the quality of resident education (range, 1–3; 1 = worsened and 3 = improved; mean, 1.42; CI, 1.26–1.58). 664
One common theme among the Resident Comments section of the survey was how the required RTAT disproportionately affected ‘‘lower level’’ residents and divisions with high radiograph volume. A few select comments included the following: 1) ‘‘While the turnaround time policy is advantageous to upper level residents in increasing reading speed and efficiency, for junior level residents, there is often a focus to get the study read as quickly as possible with education as a secondary objective .’’ 2) ‘‘This affects the younger residents more than the advanced residents due to the perceived slowness by attendings. This has drastically reduced the number of acute/Emergency Room (ER) plain films read by them.’’ 3) ‘‘TAT does not seem to affect upper level residents, who are more efficient, as much as first-year residents.’’ 4) ‘‘The effect may be greater in the earlier training years when the learning curve is much steeper.’’ Neuroradiology studies demonstrated the largest decrease in percent of resident involvement after the required RTAT despite being solely cross-sectional imaging. This may be partially explained by the following resident comments: 1) ‘‘At time, lower level residents are discouraged from tackling more complex cross-sectional imaging that have a required TAT. Sometimes this is self-imposed and other times faculty actively discourage residents from reading these studies.’’ 2) ‘‘The impact on education is attending and division specific. The biggest impacts are rushing through read out and attendings stealing cases to pad their TAT stats.’’ Faculty comments echoed some of the same sentiments and frustrations communicated by residents. Some of the faculty comments included the following: 1) ‘‘There is a difference between modalities—I do not think the residents are significantly excluded from ER chest CT exams, but they certainly are excluded from ER chest Xrays .’’
Academic Radiology, Vol 22, No 5, May 2015
RTAT: EFFECT ON RESIDENT EDUCATION
Figure 1. Line graph demonstrates a significant decrease in resident Musculoskeletal Emergency Department study involvement after the 1-hour required radiology report turnaround time (2012–2013), P < .05.
Figure 2. Line graph demonstrates a significant decrease in resident chest Emergency Department study involvement after the 1-hour required radiology report turnaround time (2012–2013), P < .05.
2) ‘‘I am far less likely to engage residents in discussion of complex cases due to turnaround time .’’ 3) ‘‘Pressure to exclude residents is greatest with complicated exams. These are the cases with greatest potential to learn, but are by nature slow to read. Price has become too high.’’ 4) ‘‘Worst decision in medical education in my lifetime.’’ DISCUSSION Our retrospective review of resident exposure to ED studies demonstrated a significant decrease in resident dictations of MSK, chest, and neuroradiology studies after the implementation of a required 1-hour RTAT; a slight decline in the number of resident dictated body studies was not statistically significant. During the different periods, daytime (8 AM–5 PM) and call (5 PM–8 AM), there were also varying degrees of decreased resident involvement in ED studies, P = .01 and P = .10, respectively. Academic radiologists, unlike their private practice counterparts, must divide their time between education, research,
and clinical productivity. The strain of increasing clinical productivity is a challenge that has already been suggested to come at the expense of resident education (16,17). A required RTAT places another strain on the clinical productivity of academic radiologists that is resulting in decreased resident ED study involvement. This raises the questions of whether decreased resident ED study involvement is negatively affecting resident education or just reducing resident support of activities, especially daytime, with more of the burden falling on the radiology attendings. In 2006, Collins described one of the tenets of learning theory that ‘‘education does not take place in a vacuum and is heavily influenced by student interaction with faculty’’ (18). Through implementation of a required RTAT, the faculty must now weigh an additional factor into how much time they can take to teach. The temptation of signing off a study without involving a resident to abide by this time metric is great. However, doing so decreases faculty–resident interaction and may negatively impact the resident educational experience. Academic radiologists on busy services often dictate studies without resident involvement. It has been suggested that despite 665
ENGLAND ET AL
Academic Radiology, Vol 22, No 5, May 2015
Figure 3. Line graph demonstrates a significant decrease in resident neuro Emergency Department study involvement after the 1-hour required radiology report turnaround time (2012–2013), P < .05.
Figure 4. Line graph demonstrates a downward trend although not statistically significant in resident body Emergency Department study involvement after the 1-hour required radiology report turnaround time (2012–2013), P = .1.
increasing clinical obligations, dedicated teaching should always be set aside in the form of case review or short didactics to ensure residents on the rotation are receiving the necessary training (19). Unfortunately, our study showed that faculty who perceived the implementation of an RTAT requirement had a negative effect on the faculty’s ability to teach residents. Despite the overall concerns that faculty had toward the required RTAT and the negative resident comments, residents felt overall that their learning opportunities from ED studies were good and their opportunity to read studies was adequate. However, a significant number of residents who were in training before the implementation of a required RTAT felt that their education had worsened since the implementation of a required RTAT. This ‘‘Ignorance is Bliss’’ phenomenon may apply to current radiology residents at programs with a required RTAT in that they have not experienced training in which this system was not in place and view the current system as adequate. In 2008, DeFlorio et al. published one of the only other studies that evaluated the effect of a required 60-minute ED 666
RTAT on resident education. They found that ‘‘the percentage of resident dictated cases available in #60 minutes was similar to or greater than the percentage when the resident and staff dictated cases were pooled together’’ (20). Our results differed from these data in that the daytime ED studies performed at our institutions were divided into the different subspecialty divisions as opposed to DeFlorio’s institution in which all daytime ED studies were interpreted by a single daytime ED division. Overnight, when the studies at our institutions were reviewed by a single ED radiologist, there was overall no statistically significant difference between the percent of resident-involved ED cases before and after the implementation of a required 1-hour RTAT, findings similar to DeFlorio’s results. Forming a daytime ED radiology division could be a possible solution to the decreased resident involvement in ED studies due to the increased utilization of a required RTAT. For those academic programs that wanted to continue reading ED studies within their subspecialty divisions,
Academic Radiology, Vol 22, No 5, May 2015
assigning a resident and attending to just monitor and dictate ED studies may ease the burden of RTAT. Continued investment in technology may also improve RTAT by having attendings’ finalized reports through an iPad or tablet device. This would decrease the time spent moving from a resident workstation to a separate workstation to sign off the study. The limitations of the study included only 1 year of resident ED study volume data after the implementation of the required 1-hour RTAT compared to 3 years of data before the implementation. Also, only the most commonly performed and not the complete spectrum of ED studies in each division were counted. This could have introduced selection bias. The surveys were conducted at only two Midwest institutions with a required RTAT, and a larger survey including different regions of the country may have produced different results. In conclusion, implementation of a required 1-hour RTAT significantly affected resident ED study involvement. The general perception among faculty was that this decreased the quality of resident education. In general, residents felt that their education was still adequate, with the exception of those residents who trained under both systems.
SUPPLEMENTARY DATA Supplementary data related to this article can be found, in the online version, at http://dx.doi.org/10.1016/j.acra. 2014.12.023. REFERENCES 1. http://acgme.org/acgmeweb/Portals/0/PDFs/Milestones/DiagnosticRadi ologyMilestones.pdf accessed on 3/9/2015. 2. http://www.theabr.org/ic-dr-certifying-exam accessed on 3/9/2015.
RTAT: EFFECT ON RESIDENT EDUCATION
3. Mullins M, Mehta A, Patel H, et al. Impact of PACS on the education of radiology residents: the residents’ perspective. Acad Radiol 2001; 8:67–73. 4. Collins J, Gruppen L, Bailey J, et al. 24/7/365 in-house radiologist coverage: effect on resident education. Acad Radiol 2014; 21:842–850. 5. Mattern C, King B, Hangiandreou, et al. Electronic imaging impact on image and report turnaround times. Journal of Digital Imaging 1999; 12: 155–159. 6. White K. Speech recognition implementation in radiology. Pediatr Radiol 2005; 35:841–846. 7. Bhan, Coblentz C, Norman G, et al. Effect of voice recognition on radiologist reporting time. CARJ 2008; 59:203–209. 8. Hawk P. Teleradiology: friend or foe? What imaging’s now indispensable partner means for radiology’s future and for the quality of care. J Health Care Finance 2011; 37:71–92. 9. Boland G, Halpern E, Gazelle GS. Radiologist report turnaround time: impact of pay-for-performance measures. AJR 2010; 195:707–711. 10. Lowe R, Abbuhl S, Baumritter A, et al. Radiology services in emergency medicine residency programs: a national survey. Acad Emerg Med 2002; 9:587–594. 11. Joffe S, Burak J, Rackson M, et al. The effect of international teleradiology attending radiologist coverage on radiology residents’ perception of night call. J Am Coll 2006; 3:872–878. 12. Branstetter B, Morgan M, Nesbit C, et al. Preliminary reports in the emergency department: is a subspecialist radiologist more accurate than a radiology resident? Acad Radiol 2007; 14:201–206. 13. Dell C, Deloney L, Hicks B, Jambhekar K. Preserving the educational value of call in a diagnostic radiology residency program. January 2014. J Am Coll Radiol; 11:68–73. 14. Ruchman RB, Jaeger J, Wiggins EF, et al. Preliminary radiology resident interpretations versus final attending radiologist interpretations and the impact on patient care in a community hospital. AJR Am J Roentgenol 2007; 180:523–526. 15. Velmahos GC, Fili C, Vassiliu P, et al. Around-the-clock attending radiology coverage is essential to avoid mistakes in the care of trauma patients. Am Surg 2001; 67(12):1175–1177. 16. Cohen MD, Gunderman RB, Frank MS, et al. Challenges facing radiology educators. J Am Coll Radiol 2005; 2:681–687. 17. Lundmerer KM. Instilling professionalism in medical education. JAMA 1999; 282:881–882. 18. Collins J. Keys to educator effectiveness in radiology. Acad Radiol 2006; 13:641–643. 19. Huang BK, Lubner M, Resnik C. Balancing clinical service and education in the radiology residency. Acad Radiol 2009; 16:1161–1165. 20. DeFlorio R, Coughlin B, Coughlin R, et al. Process modification and emergency department radiology service. Emerg Radiol 2008; 15:408–412.
667
Radiology Report Turnaround Time: Effect on Resident Education Eric England, MD, Jannette Collins, MD, MEd, FCCP, FACR, Richard D. White, MD, FACR, FACC, FAHA, FSCCT, F. Jacob Seagull, PhD, John Deledda, MD Rationale and Objectives: To compare resident workload from Emergency Department (ED) studies before and after the implementation of a required 1-hour report turnaround time (TAT) and to assess resident and faculty perception of TAT on resident education. Materials and Methods: Resident study volume will be compared for 3 years before and 1 year after the implementation of a required 1hour TAT. Changes to resident workload will be compared among the different radiology divisions (body, muscuolskeletal (MSK), chest, and neuro), as well as during different shifts (daytime and overnight). Residents and faculty at two Midwest institutions, both of which have a required report TAT, will be invited to participate in an online survey to query the perceived effect on resident education by implementation of this requirement. A P < .05 was considered statistically significant. Results: A significant decrease in resident involvement in ED studies was noted in the MSK, chest, and neuro sections with average involvement of the 3 years before the 1-hour TAT of 89%, 88%, and 82%, respectively, which decreased to 66%, 68%, and 51% after the 1-hour TAT requirement (P < .05). The resident involvement in ED studies only mildly decreased in the body section from an average before the 1-hour TAT of 87% to 80% after the 1-hour TAT requirement (P < .1). There was an overall significant decrease in resident ED study involvement during the daytime (P = .01) but not after hours during resident call (P = .1). Seventy percent of residents (43 of 61) and 55% of faculty (63 of 114) responded to our surveys. Overall, residents felt their education from ED studies during the daytime and overnight were good. However, residents who were present both before and after the implementation of a required TAT felt their education had been significantly negatively affected. Faculty surveyed thought that the required TAT negatively affected their ability to teach and decreased the quality of resident education. Conclusions: Residents are exposed to fewer ED studies after the implementation of a required 1-hour TAT. Overall, the current residents do not feel this decreased exposure to Emergency room studies affects their education. However, residents in training before and after this requirement feel their education has been significantly affected. Faculty perceives that the required TAT negatively affects their ability to teach, as well as the quality of resident education. Key Words: Resident education; turnaround time; Emergency Department. ªAUR, 2015
O
ver the past several years, radiology residents have experienced major changes to their training and certification. From the growing emphasis on diagnostic milestones for resident training to the transition from an oral board examination to a more image-rich computer-based examination, radiology resident training is in a state of rapid evolution (1,2). In addition, the transition from film with tape/digitized recordings to a picture archiving and Acad Radiol 2015; 22:662–667 From the Department of Radiology, University of Cincinnati Medical Center, 234 Goodman Street, ML 0761, Cincinnati, OH 45267-0761 (E.E.); University of Cincinnati College of Medicine, Cincinnati, Ohio (J.C.); Department of Radiology, The Ohio State University Wexner Medical Center, Colombus, Ohio (R.D.W.); Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan (F.J.S.); and Department of Emergency Medicine, University of Cincinnati Medical Center, Cincinnati, Ohio (J.D.). Received October 3, 2014; accepted December 10, 2014. Address correspondence to: E.E. e-mail: [email protected] ªAUR, 2015 http://dx.doi.org/10.1016/j.acra.2014.12.023
662
communication system (PACS) with speech recognition has dramatically changed the way academic radiologists process and review cases with residents (3). More recently, the effect on radiology resident education related to 24/7/365 in-house attending coverage was investigated (4). Another pressure now challenges academic radiologists further with a new variable to which they must adapt: the implementation of required reduced radiology report turnaround times (RTATs), potentially taking away from time spent in instruction of residents. RTAT has increasingly become an important metric for measuring the quality of diagnostic radiology services. With the institution of PACS and speech recognition systems, RTAT has decreased over the last decade from days to nearreal time (5–7). Increasingly, teleradiology companies have emphasized decreased RTAT to hospitals as a metric for improved services to compete with local radiology groups, or even other teleradiology companies (8). Some academic teaching hospitals have experimented with pay-for-performance programs to decrease their RTAT (9). This simple metric can
Academic Radiology, Vol 22, No 5, May 2015
easily be assigned a value by hospital administration and can be used to evaluate current radiology services and compare them to competing radiology services. The increased interest in this metric by radiology departments and hospital administrators over the last several years has led to hospital radiology RTATrequirements, particularly in the cases of Emergency Department (ED) studies and ‘‘stat’’ interpretations. A survey of all Emergency Medicine chairs in 2002 showed that 49% of respondents were dissatisfied with radiology RTAT (10); it also showed that 39% of sites reported RTAT <4 hours and that only 2% of daytime radiology reports were completed in <1 hour. Although continued adoption of PACS and speech recognition systems has likely improved radiologists’ RTAT since this survey was conducted, the importance that Emergency Medicine physicians place on prompt accurate final interpretations cannot be overstated. To address this concern from Emergency Medicine physicians, the American College of Radiology issued a resolution in 2001 that ‘‘all radiologic studies performed on Emergency Department patients should be promptly interpreted by radiologists’’ (11). A coordinated effort on the part of our Radiology and Emergency Medicine departments to improve radiologists’ RTAT led to the adoption of a 24/7 1-hour turnaround time (TAT) requirement for all studies performed in the ED. The goals of this requirement were to decrease patient wait time in the ED by increasing patient throughput and to improve patient care by eliminating ‘‘resident discrepancies’’ from preliminary reports, although the latter rationale remains controversial (12–15). The requirement led to the hiring of on-site dedicated nighttime and evening radiologists in 2012 to ensure that the RTAT metric was met throughout the night. The daytime ED studies are typically interpreted by a subspecialty division but with the continued requirement for a 1-hour RTAT. We hypothesized the following: 1) 1 year after the implementation of the RTAT requirement, residents would become less involved in ED studies to comply with the metric, 2) faculty would be incentivized to forego resident involvement and dictate ED studies by themselves to decrease their RTAT, and 3) faculty and resident satisfaction with resident education would decrease as more emphasis was placed on report throughput and less emphasis was placed on resident education. MATERIALS AND METHODS Emergency Department Study Volume
A retrospective review of the medical records was performed to identify patients who had the following ED studies: radiographs of the chest, abdomen, knee, and pelvis; computed tomography (CT) examinations of the chest, abdomen, pelvis, brain, cervical spine, thoracic spine, and lumbar spine; and magnetic resonance of the lumbar spine and brain. The total number of studies performed from the ED for each of these modalities and the percent of these studies that were dictated by a resident were obtained for 1 year after the implementation of the 1-hour RTAT (July 1, 2012 to June 30, 2013) and for 3 years before the 1-hour
RTAT: EFFECT ON RESIDENT EDUCATION
RTAT (July 1, 2009 to June 30, 2010; July 1, 2010 to June 30, 2011; and July 1, 2011 to June 30, 2012). The imaging studies were divided according to subspecialty section (spine studies were interpreted by the neuroradiology division.) The studies were further divided according to when they were performed, with ‘‘daytime’’ studies performed between 8 AM–5 PM and ‘‘call’’ studies performed between 5 PM–8 AM. The percent of resident-dictated cases were compared before and after the implementation of the RTAT requirement and during the different daytime/call periods using a paired t test. Changes in resident ED study volume were also assessed by subspecialty section. A P value of #.05 was considered significant. Surveys
An eight-question radiology resident survey and a ninequestion radiology faculty survey were sent to two academic radiology programs that had adopted a 1-hour RTAT requirement (Appendix, resident survey, faculty survey). These surveys addressed resident/faculty demographics, perceived effects of resident case exposure, and effects on resident education/teaching. These surveys were distributed online (via SurveyMonkey). Each resident and faculty member was sent an email requesting voluntary participation in this anonymous online survey. Consent was given online before the survey was completed. Residents and faculty had 2 weeks to complete the survey. An email reminder was sent after 1 week. Responses are presented as frequency distributions and were assessed by calculating the 95% confidence interval (CI) for the mean (P < .05). Intergroup comparisons were assessed using t tests. The study was approved by the institutional review board at both institutions. RESULTS Emergency Department Study Volume
The average percent of ED studies dictated by residents during daytime hours (8 AM–5 PM) decreased significantly after the implementation of a required 1-hour RTAT (P = .01; Table 1). On the other hand, there was not a statistically significant decrease in the mean percent of resident ED study involvement before and after the required 1-hour RTAT during call (5–8 PM; P = .10; Table 2). However, when taking both periods into account, there was overall a significant decrease in resident ED study involvement after implementation of the 1-hour RTAT (P = .027). Among the four divisions, musculoskeletal (MSK), chest, and neuroradiology had a significant drop in resident ED study involvement after the implementation of the 1-hour RTAT. The MSK division saw a decrease from an average of 88.68% (CI = 81–96; P = .006) to 65.74% (Fig 1). The chest division saw a decrease from an average of 88.36% (CI = 78– 99; P = .015) to 68.12% (Fig 2). The neuroradiology division saw the most substantial decrease from an average of 82.89% 663
ENGLAND ET AL
Academic Radiology, Vol 22, No 5, May 2015
TABLE 1. Percent of ED Studies Dictated by Residents During Daytime (8 AM–5 PM) Hours After Average of 3 years P Before Required Implementation of Required RTAT (%) Value RTAT (%) Body Musculoskeletal Chest Neuro All sections
86.71 86.79 85.16 65.55 81.05
71.87 55.98 45.35 28.10 50.33
.05 .009 .008 .04 .01
ED, Emergency Department; RTAT, radiology report turnaround time. A significant decrease of resident ED study dictations during daytime hours before and after implementation of a required 1-hour RTAT was observed.
Seventy percent (43 of 61) of residents participated in the online survey. In general, the residents thought that there were adequate opportunities to interpret studies (range, 1–2; 1 = adequate and 2 = inadequate; mean, 1.84; CI, 1.73– 1.95) with the required 1-hour RTAT metric in place. Residents also responded that learning opportunities from ED studies both during the daytime shift (range, 1–5; 1 = strongly disagree and 5 = strongly agree; mean, 3.42; CI, 3.14–3.70) and while on-call (range, 1–5; 1 = strongly disagree and 5 = strongly agree; mean, 4.02; CI, 3.76–4.28) were good. However, those residents in training both before and after the implementation of a required RTAT thought that the required RTAT significantly decreased their education (range, 1–3; 1 = worsened and 3 = improved; mean, 1.21; CI, 1.01–1.41). Comments
TABLE 2. Percent of ED Studies Dictated by Residents While on Call (5 PM–8 AM)
Body Musculoskeletal Chest Neuro All sections
Average of 3 years Before Required RTAT (%)
After Implementation of Required RTAT (%)
P Value
87.73 89.92 90.27 90.72 89.66
84.59 73.09 85.03 64.59 76.83
.35 .025 .08 .015 .1
ED, Emergency Department; RTAT, radiology report turnaround time. No statistically significant change (P = .1) of resident ED study involvement during call hours before and after implementation of a required 1-hour RTAT was observed.
(CI = 73–93; P = .005) to 51.25% (Fig 3). The only division to not see a statistically significant decrease in resident ED study involvement was the body division which only saw a slight decrease from an average of 87.37% (CI = 77–98; P = .1) to 80.24% (Fig 4). Surveys
Fifty-five percent (63 of 114) of faculty completed the online survey. The faculty responded that the 1-hour RTAT negatively influenced the total number of studies that they completed with residents (range, 1–5; 1 = greatly decreased and 5 = greatly increased; mean, 4.08; CI, 3.84–4.33). They also noted that the 1-hour RTAT decreased their ability to teach residents (range, 1–5; 1 = greatly decreased and 5 = greatly increased; mean, 2.34; CI, 2.10–2.57). Those faculty that were on staff both before and after the implementation of the 1-hour RTAT requirement noted that the new requirement significantly decreased time spent teaching residents (range, 1–3; 1 = decreased teaching and 3 = increased teaching; mean, 1.26; CI, 1.12–1.4) and significantly impaired the quality of resident education (range, 1–3; 1 = worsened and 3 = improved; mean, 1.42; CI, 1.26–1.58). 664
One common theme among the Resident Comments section of the survey was how the required RTAT disproportionately affected ‘‘lower level’’ residents and divisions with high radiograph volume. A few select comments included the following: 1) ‘‘While the turnaround time policy is advantageous to upper level residents in increasing reading speed and efficiency, for junior level residents, there is often a focus to get the study read as quickly as possible with education as a secondary objective .’’ 2) ‘‘This affects the younger residents more than the advanced residents due to the perceived slowness by attendings. This has drastically reduced the number of acute/Emergency Room (ER) plain films read by them.’’ 3) ‘‘TAT does not seem to affect upper level residents, who are more efficient, as much as first-year residents.’’ 4) ‘‘The effect may be greater in the earlier training years when the learning curve is much steeper.’’ Neuroradiology studies demonstrated the largest decrease in percent of resident involvement after the required RTAT despite being solely cross-sectional imaging. This may be partially explained by the following resident comments: 1) ‘‘At time, lower level residents are discouraged from tackling more complex cross-sectional imaging that have a required TAT. Sometimes this is self-imposed and other times faculty actively discourage residents from reading these studies.’’ 2) ‘‘The impact on education is attending and division specific. The biggest impacts are rushing through read out and attendings stealing cases to pad their TAT stats.’’ Faculty comments echoed some of the same sentiments and frustrations communicated by residents. Some of the faculty comments included the following: 1) ‘‘There is a difference between modalities—I do not think the residents are significantly excluded from ER chest CT exams, but they certainly are excluded from ER chest Xrays .’’
Academic Radiology, Vol 22, No 5, May 2015
RTAT: EFFECT ON RESIDENT EDUCATION
Figure 1. Line graph demonstrates a significant decrease in resident Musculoskeletal Emergency Department study involvement after the 1-hour required radiology report turnaround time (2012–2013), P < .05.
Figure 2. Line graph demonstrates a significant decrease in resident chest Emergency Department study involvement after the 1-hour required radiology report turnaround time (2012–2013), P < .05.
2) ‘‘I am far less likely to engage residents in discussion of complex cases due to turnaround time .’’ 3) ‘‘Pressure to exclude residents is greatest with complicated exams. These are the cases with greatest potential to learn, but are by nature slow to read. Price has become too high.’’ 4) ‘‘Worst decision in medical education in my lifetime.’’ DISCUSSION Our retrospective review of resident exposure to ED studies demonstrated a significant decrease in resident dictations of MSK, chest, and neuroradiology studies after the implementation of a required 1-hour RTAT; a slight decline in the number of resident dictated body studies was not statistically significant. During the different periods, daytime (8 AM–5 PM) and call (5 PM–8 AM), there were also varying degrees of decreased resident involvement in ED studies, P = .01 and P = .10, respectively. Academic radiologists, unlike their private practice counterparts, must divide their time between education, research,
and clinical productivity. The strain of increasing clinical productivity is a challenge that has already been suggested to come at the expense of resident education (16,17). A required RTAT places another strain on the clinical productivity of academic radiologists that is resulting in decreased resident ED study involvement. This raises the questions of whether decreased resident ED study involvement is negatively affecting resident education or just reducing resident support of activities, especially daytime, with more of the burden falling on the radiology attendings. In 2006, Collins described one of the tenets of learning theory that ‘‘education does not take place in a vacuum and is heavily influenced by student interaction with faculty’’ (18). Through implementation of a required RTAT, the faculty must now weigh an additional factor into how much time they can take to teach. The temptation of signing off a study without involving a resident to abide by this time metric is great. However, doing so decreases faculty–resident interaction and may negatively impact the resident educational experience. Academic radiologists on busy services often dictate studies without resident involvement. It has been suggested that despite 665
ENGLAND ET AL
Academic Radiology, Vol 22, No 5, May 2015
Figure 3. Line graph demonstrates a significant decrease in resident neuro Emergency Department study involvement after the 1-hour required radiology report turnaround time (2012–2013), P < .05.
Figure 4. Line graph demonstrates a downward trend although not statistically significant in resident body Emergency Department study involvement after the 1-hour required radiology report turnaround time (2012–2013), P = .1.
increasing clinical obligations, dedicated teaching should always be set aside in the form of case review or short didactics to ensure residents on the rotation are receiving the necessary training (19). Unfortunately, our study showed that faculty who perceived the implementation of an RTAT requirement had a negative effect on the faculty’s ability to teach residents. Despite the overall concerns that faculty had toward the required RTAT and the negative resident comments, residents felt overall that their learning opportunities from ED studies were good and their opportunity to read studies was adequate. However, a significant number of residents who were in training before the implementation of a required RTAT felt that their education had worsened since the implementation of a required RTAT. This ‘‘Ignorance is Bliss’’ phenomenon may apply to current radiology residents at programs with a required RTAT in that they have not experienced training in which this system was not in place and view the current system as adequate. In 2008, DeFlorio et al. published one of the only other studies that evaluated the effect of a required 60-minute ED 666
RTAT on resident education. They found that ‘‘the percentage of resident dictated cases available in #60 minutes was similar to or greater than the percentage when the resident and staff dictated cases were pooled together’’ (20). Our results differed from these data in that the daytime ED studies performed at our institutions were divided into the different subspecialty divisions as opposed to DeFlorio’s institution in which all daytime ED studies were interpreted by a single daytime ED division. Overnight, when the studies at our institutions were reviewed by a single ED radiologist, there was overall no statistically significant difference between the percent of resident-involved ED cases before and after the implementation of a required 1-hour RTAT, findings similar to DeFlorio’s results. Forming a daytime ED radiology division could be a possible solution to the decreased resident involvement in ED studies due to the increased utilization of a required RTAT. For those academic programs that wanted to continue reading ED studies within their subspecialty divisions,
Academic Radiology, Vol 22, No 5, May 2015
assigning a resident and attending to just monitor and dictate ED studies may ease the burden of RTAT. Continued investment in technology may also improve RTAT by having attendings’ finalized reports through an iPad or tablet device. This would decrease the time spent moving from a resident workstation to a separate workstation to sign off the study. The limitations of the study included only 1 year of resident ED study volume data after the implementation of the required 1-hour RTAT compared to 3 years of data before the implementation. Also, only the most commonly performed and not the complete spectrum of ED studies in each division were counted. This could have introduced selection bias. The surveys were conducted at only two Midwest institutions with a required RTAT, and a larger survey including different regions of the country may have produced different results. In conclusion, implementation of a required 1-hour RTAT significantly affected resident ED study involvement. The general perception among faculty was that this decreased the quality of resident education. In general, residents felt that their education was still adequate, with the exception of those residents who trained under both systems.
SUPPLEMENTARY DATA Supplementary data related to this article can be found, in the online version, at http://dx.doi.org/10.1016/j.acra. 2014.12.023. REFERENCES 1. http://acgme.org/acgmeweb/Portals/0/PDFs/Milestones/DiagnosticRadi ologyMilestones.pdf accessed on 3/9/2015. 2. http://www.theabr.org/ic-dr-certifying-exam accessed on 3/9/2015.
RTAT: EFFECT ON RESIDENT EDUCATION
3. Mullins M, Mehta A, Patel H, et al. Impact of PACS on the education of radiology residents: the residents’ perspective. Acad Radiol 2001; 8:67–73. 4. Collins J, Gruppen L, Bailey J, et al. 24/7/365 in-house radiologist coverage: effect on resident education. Acad Radiol 2014; 21:842–850. 5. Mattern C, King B, Hangiandreou, et al. Electronic imaging impact on image and report turnaround times. Journal of Digital Imaging 1999; 12: 155–159. 6. White K. Speech recognition implementation in radiology. Pediatr Radiol 2005; 35:841–846. 7. Bhan, Coblentz C, Norman G, et al. Effect of voice recognition on radiologist reporting time. CARJ 2008; 59:203–209. 8. Hawk P. Teleradiology: friend or foe? What imaging’s now indispensable partner means for radiology’s future and for the quality of care. J Health Care Finance 2011; 37:71–92. 9. Boland G, Halpern E, Gazelle GS. Radiologist report turnaround time: impact of pay-for-performance measures. AJR 2010; 195:707–711. 10. Lowe R, Abbuhl S, Baumritter A, et al. Radiology services in emergency medicine residency programs: a national survey. Acad Emerg Med 2002; 9:587–594. 11. Joffe S, Burak J, Rackson M, et al. The effect of international teleradiology attending radiologist coverage on radiology residents’ perception of night call. J Am Coll 2006; 3:872–878. 12. Branstetter B, Morgan M, Nesbit C, et al. Preliminary reports in the emergency department: is a subspecialist radiologist more accurate than a radiology resident? Acad Radiol 2007; 14:201–206. 13. Dell C, Deloney L, Hicks B, Jambhekar K. Preserving the educational value of call in a diagnostic radiology residency program. January 2014. J Am Coll Radiol; 11:68–73. 14. Ruchman RB, Jaeger J, Wiggins EF, et al. Preliminary radiology resident interpretations versus final attending radiologist interpretations and the impact on patient care in a community hospital. AJR Am J Roentgenol 2007; 180:523–526. 15. Velmahos GC, Fili C, Vassiliu P, et al. Around-the-clock attending radiology coverage is essential to avoid mistakes in the care of trauma patients. Am Surg 2001; 67(12):1175–1177. 16. Cohen MD, Gunderman RB, Frank MS, et al. Challenges facing radiology educators. J Am Coll Radiol 2005; 2:681–687. 17. Lundmerer KM. Instilling professionalism in medical education. JAMA 1999; 282:881–882. 18. Collins J. Keys to educator effectiveness in radiology. Acad Radiol 2006; 13:641–643. 19. Huang BK, Lubner M, Resnik C. Balancing clinical service and education in the radiology residency. Acad Radiol 2009; 16:1161–1165. 20. DeFlorio R, Coughlin B, Coughlin R, et al. Process modification and emergency department radiology service. Emerg Radiol 2008; 15:408–412.
667