- Email: [email protected]
Assessing surgical residents' imaging interpretation skills
The American Journal of Surgery xxx (2016) 1e4
Contents lists available at ScienceDirect
The American Journal of Surgery journal homepage: www.americanjournalofsurgery.com
Assessing surgical residents' imaging interpretation skills Joseph J. Eid a, *, Francisco Igor B. Macedo a, Edsa Negussie b, Vijay K. Mittal a a b
Department of Surgery, Providence Hospital and Medical Centers, Michigan State University College of Human Medicine, Southfield, MI, USA Department of Radiology, Providence Hospital and Medical Centers, Michigan State University College of Human Medicine, Southfield, MI, USA
a r t i c l e i n f o
a b s t r a c t
Article history: Received 26 July 2016 Received in revised form 10 November 2016 Accepted 18 November 2016 Podium Presentation at the Midwest Surgical Association 2016 Annual Meeting, Mackinanc Island, MI, USA.
Purpose: During general surgery (GS) training, residents are expected to accurately interpret radiologic images. Objective evidence evaluating residents' ability to provide accurate interpretation of imaging studies is currently lacking. Methods: A 30-item web-based test was developed using images from different radiologic modalities. Residents from 6 ACGME accredited GS programs participated. Residents from 1 radiology program served as a control group. Results: 74 GS residents (GSR) enrolled in the online test. The mean score for GSR was 75% (±9) and 83% (±6) for RR (p ¼ 0.006). Residents correctly answered 63% x-rays, 74%, CT(head), 84% CT(body), 69% ultrasound, and 88% tube/line localization questions. Senior residents were more proficient than junior residents at interpreting CT (body) and ultrasound images. Conclusion: GS residents were able to accurately interpret 75% of basic radiology images. In an effort to improve patient care, programs should consider integrating radiological education during surgical training. © 2016 Elsevier Inc. All rights reserved.
Keywords: Radiology interpretation General surgery residency Imaging interpretation Surgical education
1. Introduction As surgeons-in-training, residents are expected to use multiple modalities to accurately diagnose surgical diseases. With technological advancement and the integration of digitally enhanced and reconstructed images, radiology is now an essential tool used to manage various pathologies. In general surgery (GS) specifically, resident and attending physicians have incorporated radiographic analysis as part of patients' perioperative care. Such skills have also become an integral part of critical and trauma care where surgeons need to provide rapid interventions. Concurrently, the American Board of Surgery has emphasized the importance of training surgeons to critically assess, interpret, and integrate clinical imaging in their patient care.1 Despite the lack of formalized curricula during training, GS programs have met the requirements for board certification by providing residents radiology education via conferences, online modules, review of patient encounters under the guidance and supervision of attending physicians. Although there is an ongoing effort by academic programs to
* Corresponding author. Department of Surgery, Providence Hospital and Medical Centers, Michigan State University College of Human Medicine, 16001 W 9 Mile Rd, Southfield, MI, 48075, USA. E-mail address: [email protected] (J.J. Eid).
provide radiology education to GS, and other specialty residents, variable results have been published when investigators assessed trainees' proficiency in diagnostic imaging evaluation. These investigational studies have limited resident assessment to one radiologic modality, focused clinical scenarios such as trauma radiographs, or to a single postgraduate year level.2e4 Similar variability in trainees' imaging interpretation accuracy is seen among different specialties including radiology residents.5e8 Given the discordance in the reported accuracy rates and the ongoing expectation of GSR to provide preliminary interpretation of clinical radiographs, we sought to assess their ability to correctly interpret basic radiology images. We also sought to identify areas of deficiency in surgical education that may ultimately improve patient care. 2. Methods A 30-item web-based test was created on Proprofs.com® that included five different categories of imaging modalities: (1) x-rays (chest/abdomen), (2) computer tomography of the Head (CT head), (3) CT (body), (4) ultrasound (solid organs/vascular), and (5) invasive lines/tube localizing images. After reviewing a previously established radiology education database, de-identified images that were considered common conditions encountered by GSR's
http://dx.doi.org/10.1016/j.amjsurg.2016.11.029 0002-9610/© 2016 Elsevier Inc. All rights reserved.
Please cite this article in press as: Eid JJ, et al., Assessing surgical residents' imaging interpretation skills, The American Journal of Surgery (2016), http://dx.doi.org/10.1016/j.amjsurg.2016.11.029
2
J.J. Eid et al. / The American Journal of Surgery xxx (2016) 1e4
were selected by an expert panel. The panel included clinical radiologists, general surgeons and both GS and radiology program directors. The questions were formatted to include a brief clinical vignette and a corresponding static radiographic image. True/False and multiple choice questions which allowed for single or multiple answers were utilized. Seven allopathic general surgery programs from the Southeast Michigan Center of Medical Education consortium were invited to participate. Radiology residents (RR) from our institution served as the control group. Resident participation was on voluntary basis. An initial invitation letter and 4 additional reminders were sent to improve participation rates. Institutional Review Board approval was obtained prior to the initiation of the study. Results were collected between September and November 2015. GS postgraduate year (PGY)1 and PGY2 trainees were considered junior residents (GS-JR) and PGY3, PGY4, and PGY5 trainees were considered senior residents (GS-SR). PGY2/R1 and PGY3/R2 RR were considered RR-JR), and PGY4/R3 and PGY5/R4 were considered senior residents (RR-SR). GSR's competency in interpreting images was established by the percentage of correct answers. Performance among GSR was compared at different levels of training, and overall performance was compared to RR. 2.1. Statistical analysis Student's t-test and analysis of variance (ANOVA) F-test followed by post-hoc Tukey-Kramer test were used to analyze mean differences in resident accuracy rates between programs, different levels of training, and specialties. Categorical proportions were analyzed using Chi-square Test (c2) or Fisher's exact test as appropriate. Data is presented as mean (±standard deviation) and counts (percent, %). P < 0.05 was considered statistically significant. Statistical analysis was performed using Prism© 7.0 software (GraphPad Inc., CA, USA). 3. Results Of the 173 eligible subjects, 74 (43%) GS residents enrolled from 6 programs of which 3 were community-based with university affiliations, 2 community-based without university affiliations, and 1 university-based. One program was excluded from the analysis due to lack of participation. Examinees were predominantly PGY1 residents (n ¼ 27) compared to PGY2 (n ¼ 14), PGY3 (n ¼ 10), PGY4 (n ¼ 12), and PGY5 (n ¼ 11) residents. Nine (75%) of 12 eligible RR participated. GSR accurately interpreted 75% (±9) of the queried images compared to 83% (±6) for RR (p ¼ 0.006). GS residents from different programs performed similarly (p ¼ 0.9) (Table 1). Table 1 highlights difference in performance among GSR across various PGY levels (p < 0.0001). This discrepancy in scores is seen on post-hoc analysis when comparing GS PGY1 residents to PGY3, 4, and 5 residents (Fig. 1). GS-SR scored higher than their junior colleagues (p ¼ 0.0007). RR-JR (n ¼ 5) scored 81% (±6) compared to their GS-JR counterparts [72% (±9), p ¼ 0.037]. Similar results were seen for SR-RR (n ¼ 4) who interpreted more radiology images correctly than GS-SR [78% (±6) vs. 86% (±4), p ¼ 0.026]. No difference in mean scores was seen between male [74% (±9)] and female residents [74% (±8), p ¼ 0.88]. GSR required 13 min less to complete the test compared to RR (p ¼ 0.019), and no difference in duration was seen between GS-JR and GS-SR (p ¼ 0.12) (Table 1). A correlation between performance and duration of the quiz was not seen (r ¼ 0.11, p ¼ 0.34). Residents who completed the test during working hours answered 76% (±8) of questions correctly, compared to those that completed the quiz after working hours [71% (±9), p ¼ 0.04]. RR were more competent the interpretation of images on CT
(head) and x-rays (chest/abdomen) compared to GSR (Table 1). No difference in performance was seen between GSR and RR in the assessment of CT (body), ultrasound, and invasive line/tube localizing images. Subgroup analysis revealed no difference in accuracy rate between GS-SR and RR in the evaluation of CT (body) [90% (±13) vs. 92% (±10), p ¼ 0.81], however RR were more proficient than GS-JR [79% (±16) vs. 92% (±10), p ¼ 0.019]. The level of training appeared to influence the interpretation of several diagnostic imaging modalities. GS-SR answered 10% more questions correctly involving images on CT (body) compared to their JR colleagues (p ¼ 0.001). Similar results were observed for the ultrasound modality, where GS-SR were more precise than their JR counterparts [65% (±17) vs. 74% (±15), p ¼ 0.03]. Seniority in GS residency did not appear to impact interpretation scores for x-rays, invasive tube/lines localizing, CT (head) images. Level of training in RR did not impact performance on any of the queried modalities. 4. Discussion GSR as well other physicians continue to find themselves providing preliminary reads of clinical images. Such instances occur Table 1 Participant demographics and performance. RR Participants N (rate %) Gender Male Female Duration of exam, mean minutes (±SD) Score, mean %(±SD) GS programs Program A Program B Program C Program D Program E Program F Modalities X-rays (Chest/Abdomen) CT (Head) CT (Body) Ultrasound Invasive Lines/Tubes GS: PGY-Level PGY1 PGY2 PGY3 PGY4 PGY5
Participants, N (%) Duration of exam, mean minutes (±SD) Score, mean %(±SD) X-rays (Chest/Abdomen) CT (Head) CT (Body) Ultrasound Invasive Lines/Tubes
p-value
GSR
p-value
9 (75%)
74 (43%)
3 (6%) 6 (17%) 32 (±31)
44 (94%) 30 (83%) 19 (±13)
83 (±6)
75 (±9)
ns
77 (±12) 100 (±0) 92 (±10) 69 (±16) 90 (±15)
74 75 73 76 70 76
(±11) (±9) (±6) (±5) (±3) (±7)
63 74 84 69 88
(±14) (±22) (±15) (±15) (±13)
70 75 80 77 79
(±10) (±6) (±7) (±3) (±7)
0.019* 0.006* ns
0.019* 0.001* ns ns ns <0.0001
RR-JR
RR-SR pGS-JR value
GS-SR p-value
5 (55%) 37 (±35) 79 (±6) 73 (±15) 100 (±0) 89 (±12) 70 (±7) 84 (±17)
4 (45%) 26 (±27) 86 (±4) 81 (±6) 100 (±0) 96 (±7) 67 (±24) 95 (±10)
33 (45%) 17 (±11) 78 (±6) 67 (±12) 76 (±23) 90 (±13) 74 (±15) 90 (±12)
ns ns ns ns ns ns ns ns
41 (55%) 21 (±14) 72 (±9) 60 (±15) 72 (±14) 79 (±16) 65 (±19) 89 (±14)
ns ns 0.0007* ns ns 0.001* 0.03* ns
*p < 0.05: statistically significant; ns: not statistically significant.
Please cite this article in press as: Eid JJ, et al., Assessing surgical residents' imaging interpretation skills, The American Journal of Surgery (2016), http://dx.doi.org/10.1016/j.amjsurg.2016.11.029
J.J. Eid et al. / The American Journal of Surgery xxx (2016) 1e4
3
Fig. 1. Tukey-Kramer test analyzing difference between group means.
when time is limited and an immediate intervention is required regardless of 24-h availability of in-house radiologists. This skill is generally used by all subspecialty physicians to a variable extent. However, a recent multi-institutional study showed that general surgeons rely greatly upon individual interpretation of clinical images in the operative and trauma setting.9 Inconsistent accuracy rates have been reported when evaluating GSR's ability to independently interpret images,2e4,8 hence, an objective measure to assess resident competency is needed. Anesthesiology, emergency and internal medicine literature have demonstrated a deficiency and lack of certainty in the interpretation of chest radiographs.6e8 In one emergency department with attending physicians, medical and surgical residents, the sensitivity in detecting pathologies on chest radiographs ranged between 20% and 65%.8 In the trauma setting, PGY four and five GSR seem to have similar accuracy rates to radiologists in detecting injuries on both chest/pelvis radiographs; and the majority of missed injuries were clinically insignificant.3 In this study GSR misdiagnosed 37% of chest/abdomen radiographs and were inferior in performance compared to RR. Given the basic nature of the queried clinical images, the rate of misdiagnoses would be expected to be lower especially for senior residents. In centers with resident intensive care unit coverage, GS trainees are expected to immediately assess postplacement radiographs to assure the proper positioning of a central venous catheters, thoracostomy tubes, and nasogastric tubes. In such instances, awaiting a formal radiologist read may not be feasible due to patients' critical condition. Despite the lower scores in the assessment of clinical x-rays (chest/abdomen), both Jr. and Sr surgical trainees were able to localize placed catheters and tubes on plain radiographs. To our knowledge, this objective assessment was the first to display GSR's safety and reliability in providing correct preliminary interpretations of postplacement radiographs. Since the first installed CT scanner in 1971, surgeons have integrated this radiographic modality into their practice. Currently, CT scans have become indispensable to general surgery patient care. Its use in the perioperative period allows surgeons to meticulously plan procedures ahead of time such as in surgical oncology, vascular surgery, and thoracic surgery cases. Due to the short time duration required to scan patients and immediate availability of images, its use has expanded to the acute and trauma surgery setting. Hence, it is of utmost importance to assess GSR's ability to precisely interpret CT images. In a level one trauma center, one study reported that GSR correctly diagnosed 96% of head injuries and 94% of abdominal injuries, but 33% of thoracic injuries were
missed.2 Vorhies et al. concluded that senior GSR were equally proficient as radiologists at interpreting CT Head/Brain.3 This examination presented traumatic and non-traumatic causes of head pathologies that are typically encountered by GSR. Although the training level did not impact GSR performance, there were more interpretation errors than their radiology colleagues. The discrepancy between our results and previously published data may be due to the non-traumatic pathologies on head CT images being included. In the non-trauma setting, second year surgical residents had lower accuracy rates in detecting acute appendicitis on CT compared to radiologists.4 The participating GSR were equally competent to RR in interpreting CT of acute care service pathologies such as small bowel obstruction, pancreatitis, and pneumatosis intestinalis; however, this similarity in proficiency was due to GSSR superior performance on this modality. As expected, training and seniority improved acquisition of this skill. In this era of trauma care, surgeons perform and interpret focused ultrasonography in the emergency department, which has been shown to reliably identify hemoperitoneum and pericardial effusions. Senior surgical residents have been found to be proficient in evaluating traumatic abdominal injuries using focused ultrasonography.10,11 Our examination included both traumatic and nontraumatic pathologies seen on ultrasound. The lack of discrepancy between GSR and RR in correctly diagnosing pathologies on ultrasound may be due to surgical training's integration of dedicated education.11,12 Senior residents may have participated in more ultrasound courses than junior residents, hence demonstrating superiority in ultrasound interpretation. To our knowledge, this is the first assessment of GSR's interpretation skills of multiple modalities with images extrapolated from both trauma and non-trauma settings. Second, the resident enrollment from various types of GS programs and training levels ensured heterogeneity in the sampled population. Third, we have demonstrated that GSR were less competent than RR in the evaluation of basic radiology images that are routinely expected to be read independently. There are several limitations to our study. Despite multiple reminders, the participation rate was 43%. A greater participation rate may have improved the validity of the results. Second, an arbitrary limited sample of images were selected. Although they may represent common conditions encountered by GSR, it does not represent all possible pathologies. The addition of more images to the test may have altered our results, but it also may have induced decreased participation rates. Third, residents from the same program who enrolled at the same time may have shared and
Please cite this article in press as: Eid JJ, et al., Assessing surgical residents' imaging interpretation skills, The American Journal of Surgery (2016), http://dx.doi.org/10.1016/j.amjsurg.2016.11.029
4
J.J. Eid et al. / The American Journal of Surgery xxx (2016) 1e4
discussed answers which may have skewed results. Finally, the images incorporated into the exam were static, making the interpretation of dynamic modalities like CT and ultrasound more difficult. In summary, GSR were able to interpret 75% of basic radiology images, however major deficiencies were identified that may have a negative impact on patient care. In an attempt to provide high quality care, GS program directors should consider identifying gaps in medical education and tailor radiology curricula to both incoming and current general surgery residents. References 1. ABS Booklet of Information e Residency Experience for Initial Certification in General Surgery. 2. Arentz C, Griswold JA, Halldorsson A, et al. Accuracy of Surgery residents' interpretation of computed tomography scan in trauma. Am J Surg. 2008;196: 809e812. 3. Vorhies RW, Harrison PB, Smith RS, Helmer SD. Senior surgical residents can accurately interpret trauma radiographs. Am Surg. 2002;68(3):221e225.
4. Khan SA, Sleem R, Schiralli M, et al. Assessment of Surgical residents' ability to predict appendicitis with computed tomography. Am Surg. 2013;79(12): 1308e1310. 5. Tieng N, Grinberg D, Li SF. Discrepancies in interpretation of ED body computed tomographic scans by radiology residents. Am J Emerg Med. 2007;25:45e48. 6. Eisen LA, Berger JS, Hegde A, Schneider RF. Competency in chest radiography: a comparison of medical students, residents, and fellows. J Gen Intern Med. 2006;21(5):460e465. 7. Kaufman B, Dhar P, O'Neill DK, et al. Chest radiograph interpretation skills of anesthesiologists. J Cardiothorac Vasc Anesth. 2001;15(6):680e683. 8. Gatt ME, Spectre G, Paltiel O, et al. Chest radiographs in the emergency department: is the radiologist really necessary? Postgrad Med J. 2003;79: 214e217. 9. Butler KL, Chang Y, DeMoya M, et al. Needs assessment for a focused radiology curriculum in surgical residency: a multicenter study. Am J Surg. 2016;211: 279e287. 10. Kern SJ, Smith RS, Fry WR, et al. Sonographic examination of abdominal trauma by senior surgical residents. Am Surg. 1997;63(8):669e674. 11. Smith R, Kern SJ, Fry WR, Helmer SD. Institutional learning curve of surgeonperformed trauma ultrasound. Arch Surg. 1998;133(5):530e536. 12. Kotagal M, Quiroga E, Ruffatto BJ, et al. Impact of Point-of-Care ultrasound training on surgical residents' confidence. J Surg Educ. 2015;72(4):82e88.
Please cite this article in press as: Eid JJ, et al., Assessing surgical residents' imaging interpretation skills, The American Journal of Surgery (2016), http://dx.doi.org/10.1016/j.amjsurg.2016.11.029
Contents lists available at ScienceDirect
The American Journal of Surgery journal homepage: www.americanjournalofsurgery.com
Assessing surgical residents' imaging interpretation skills Joseph J. Eid a, *, Francisco Igor B. Macedo a, Edsa Negussie b, Vijay K. Mittal a a b
Department of Surgery, Providence Hospital and Medical Centers, Michigan State University College of Human Medicine, Southfield, MI, USA Department of Radiology, Providence Hospital and Medical Centers, Michigan State University College of Human Medicine, Southfield, MI, USA
a r t i c l e i n f o
a b s t r a c t
Article history: Received 26 July 2016 Received in revised form 10 November 2016 Accepted 18 November 2016 Podium Presentation at the Midwest Surgical Association 2016 Annual Meeting, Mackinanc Island, MI, USA.
Purpose: During general surgery (GS) training, residents are expected to accurately interpret radiologic images. Objective evidence evaluating residents' ability to provide accurate interpretation of imaging studies is currently lacking. Methods: A 30-item web-based test was developed using images from different radiologic modalities. Residents from 6 ACGME accredited GS programs participated. Residents from 1 radiology program served as a control group. Results: 74 GS residents (GSR) enrolled in the online test. The mean score for GSR was 75% (±9) and 83% (±6) for RR (p ¼ 0.006). Residents correctly answered 63% x-rays, 74%, CT(head), 84% CT(body), 69% ultrasound, and 88% tube/line localization questions. Senior residents were more proficient than junior residents at interpreting CT (body) and ultrasound images. Conclusion: GS residents were able to accurately interpret 75% of basic radiology images. In an effort to improve patient care, programs should consider integrating radiological education during surgical training. © 2016 Elsevier Inc. All rights reserved.
Keywords: Radiology interpretation General surgery residency Imaging interpretation Surgical education
1. Introduction As surgeons-in-training, residents are expected to use multiple modalities to accurately diagnose surgical diseases. With technological advancement and the integration of digitally enhanced and reconstructed images, radiology is now an essential tool used to manage various pathologies. In general surgery (GS) specifically, resident and attending physicians have incorporated radiographic analysis as part of patients' perioperative care. Such skills have also become an integral part of critical and trauma care where surgeons need to provide rapid interventions. Concurrently, the American Board of Surgery has emphasized the importance of training surgeons to critically assess, interpret, and integrate clinical imaging in their patient care.1 Despite the lack of formalized curricula during training, GS programs have met the requirements for board certification by providing residents radiology education via conferences, online modules, review of patient encounters under the guidance and supervision of attending physicians. Although there is an ongoing effort by academic programs to
* Corresponding author. Department of Surgery, Providence Hospital and Medical Centers, Michigan State University College of Human Medicine, 16001 W 9 Mile Rd, Southfield, MI, 48075, USA. E-mail address: [email protected] (J.J. Eid).
provide radiology education to GS, and other specialty residents, variable results have been published when investigators assessed trainees' proficiency in diagnostic imaging evaluation. These investigational studies have limited resident assessment to one radiologic modality, focused clinical scenarios such as trauma radiographs, or to a single postgraduate year level.2e4 Similar variability in trainees' imaging interpretation accuracy is seen among different specialties including radiology residents.5e8 Given the discordance in the reported accuracy rates and the ongoing expectation of GSR to provide preliminary interpretation of clinical radiographs, we sought to assess their ability to correctly interpret basic radiology images. We also sought to identify areas of deficiency in surgical education that may ultimately improve patient care. 2. Methods A 30-item web-based test was created on Proprofs.com® that included five different categories of imaging modalities: (1) x-rays (chest/abdomen), (2) computer tomography of the Head (CT head), (3) CT (body), (4) ultrasound (solid organs/vascular), and (5) invasive lines/tube localizing images. After reviewing a previously established radiology education database, de-identified images that were considered common conditions encountered by GSR's
http://dx.doi.org/10.1016/j.amjsurg.2016.11.029 0002-9610/© 2016 Elsevier Inc. All rights reserved.
Please cite this article in press as: Eid JJ, et al., Assessing surgical residents' imaging interpretation skills, The American Journal of Surgery (2016), http://dx.doi.org/10.1016/j.amjsurg.2016.11.029
2
J.J. Eid et al. / The American Journal of Surgery xxx (2016) 1e4
were selected by an expert panel. The panel included clinical radiologists, general surgeons and both GS and radiology program directors. The questions were formatted to include a brief clinical vignette and a corresponding static radiographic image. True/False and multiple choice questions which allowed for single or multiple answers were utilized. Seven allopathic general surgery programs from the Southeast Michigan Center of Medical Education consortium were invited to participate. Radiology residents (RR) from our institution served as the control group. Resident participation was on voluntary basis. An initial invitation letter and 4 additional reminders were sent to improve participation rates. Institutional Review Board approval was obtained prior to the initiation of the study. Results were collected between September and November 2015. GS postgraduate year (PGY)1 and PGY2 trainees were considered junior residents (GS-JR) and PGY3, PGY4, and PGY5 trainees were considered senior residents (GS-SR). PGY2/R1 and PGY3/R2 RR were considered RR-JR), and PGY4/R3 and PGY5/R4 were considered senior residents (RR-SR). GSR's competency in interpreting images was established by the percentage of correct answers. Performance among GSR was compared at different levels of training, and overall performance was compared to RR. 2.1. Statistical analysis Student's t-test and analysis of variance (ANOVA) F-test followed by post-hoc Tukey-Kramer test were used to analyze mean differences in resident accuracy rates between programs, different levels of training, and specialties. Categorical proportions were analyzed using Chi-square Test (c2) or Fisher's exact test as appropriate. Data is presented as mean (±standard deviation) and counts (percent, %). P < 0.05 was considered statistically significant. Statistical analysis was performed using Prism© 7.0 software (GraphPad Inc., CA, USA). 3. Results Of the 173 eligible subjects, 74 (43%) GS residents enrolled from 6 programs of which 3 were community-based with university affiliations, 2 community-based without university affiliations, and 1 university-based. One program was excluded from the analysis due to lack of participation. Examinees were predominantly PGY1 residents (n ¼ 27) compared to PGY2 (n ¼ 14), PGY3 (n ¼ 10), PGY4 (n ¼ 12), and PGY5 (n ¼ 11) residents. Nine (75%) of 12 eligible RR participated. GSR accurately interpreted 75% (±9) of the queried images compared to 83% (±6) for RR (p ¼ 0.006). GS residents from different programs performed similarly (p ¼ 0.9) (Table 1). Table 1 highlights difference in performance among GSR across various PGY levels (p < 0.0001). This discrepancy in scores is seen on post-hoc analysis when comparing GS PGY1 residents to PGY3, 4, and 5 residents (Fig. 1). GS-SR scored higher than their junior colleagues (p ¼ 0.0007). RR-JR (n ¼ 5) scored 81% (±6) compared to their GS-JR counterparts [72% (±9), p ¼ 0.037]. Similar results were seen for SR-RR (n ¼ 4) who interpreted more radiology images correctly than GS-SR [78% (±6) vs. 86% (±4), p ¼ 0.026]. No difference in mean scores was seen between male [74% (±9)] and female residents [74% (±8), p ¼ 0.88]. GSR required 13 min less to complete the test compared to RR (p ¼ 0.019), and no difference in duration was seen between GS-JR and GS-SR (p ¼ 0.12) (Table 1). A correlation between performance and duration of the quiz was not seen (r ¼ 0.11, p ¼ 0.34). Residents who completed the test during working hours answered 76% (±8) of questions correctly, compared to those that completed the quiz after working hours [71% (±9), p ¼ 0.04]. RR were more competent the interpretation of images on CT
(head) and x-rays (chest/abdomen) compared to GSR (Table 1). No difference in performance was seen between GSR and RR in the assessment of CT (body), ultrasound, and invasive line/tube localizing images. Subgroup analysis revealed no difference in accuracy rate between GS-SR and RR in the evaluation of CT (body) [90% (±13) vs. 92% (±10), p ¼ 0.81], however RR were more proficient than GS-JR [79% (±16) vs. 92% (±10), p ¼ 0.019]. The level of training appeared to influence the interpretation of several diagnostic imaging modalities. GS-SR answered 10% more questions correctly involving images on CT (body) compared to their JR colleagues (p ¼ 0.001). Similar results were observed for the ultrasound modality, where GS-SR were more precise than their JR counterparts [65% (±17) vs. 74% (±15), p ¼ 0.03]. Seniority in GS residency did not appear to impact interpretation scores for x-rays, invasive tube/lines localizing, CT (head) images. Level of training in RR did not impact performance on any of the queried modalities. 4. Discussion GSR as well other physicians continue to find themselves providing preliminary reads of clinical images. Such instances occur Table 1 Participant demographics and performance. RR Participants N (rate %) Gender Male Female Duration of exam, mean minutes (±SD) Score, mean %(±SD) GS programs Program A Program B Program C Program D Program E Program F Modalities X-rays (Chest/Abdomen) CT (Head) CT (Body) Ultrasound Invasive Lines/Tubes GS: PGY-Level PGY1 PGY2 PGY3 PGY4 PGY5
Participants, N (%) Duration of exam, mean minutes (±SD) Score, mean %(±SD) X-rays (Chest/Abdomen) CT (Head) CT (Body) Ultrasound Invasive Lines/Tubes
p-value
GSR
p-value
9 (75%)
74 (43%)
3 (6%) 6 (17%) 32 (±31)
44 (94%) 30 (83%) 19 (±13)
83 (±6)
75 (±9)
ns
77 (±12) 100 (±0) 92 (±10) 69 (±16) 90 (±15)
74 75 73 76 70 76
(±11) (±9) (±6) (±5) (±3) (±7)
63 74 84 69 88
(±14) (±22) (±15) (±15) (±13)
70 75 80 77 79
(±10) (±6) (±7) (±3) (±7)
0.019* 0.006* ns
0.019* 0.001* ns ns ns <0.0001
RR-JR
RR-SR pGS-JR value
GS-SR p-value
5 (55%) 37 (±35) 79 (±6) 73 (±15) 100 (±0) 89 (±12) 70 (±7) 84 (±17)
4 (45%) 26 (±27) 86 (±4) 81 (±6) 100 (±0) 96 (±7) 67 (±24) 95 (±10)
33 (45%) 17 (±11) 78 (±6) 67 (±12) 76 (±23) 90 (±13) 74 (±15) 90 (±12)
ns ns ns ns ns ns ns ns
41 (55%) 21 (±14) 72 (±9) 60 (±15) 72 (±14) 79 (±16) 65 (±19) 89 (±14)
ns ns 0.0007* ns ns 0.001* 0.03* ns
*p < 0.05: statistically significant; ns: not statistically significant.
Please cite this article in press as: Eid JJ, et al., Assessing surgical residents' imaging interpretation skills, The American Journal of Surgery (2016), http://dx.doi.org/10.1016/j.amjsurg.2016.11.029
J.J. Eid et al. / The American Journal of Surgery xxx (2016) 1e4
3
Fig. 1. Tukey-Kramer test analyzing difference between group means.
when time is limited and an immediate intervention is required regardless of 24-h availability of in-house radiologists. This skill is generally used by all subspecialty physicians to a variable extent. However, a recent multi-institutional study showed that general surgeons rely greatly upon individual interpretation of clinical images in the operative and trauma setting.9 Inconsistent accuracy rates have been reported when evaluating GSR's ability to independently interpret images,2e4,8 hence, an objective measure to assess resident competency is needed. Anesthesiology, emergency and internal medicine literature have demonstrated a deficiency and lack of certainty in the interpretation of chest radiographs.6e8 In one emergency department with attending physicians, medical and surgical residents, the sensitivity in detecting pathologies on chest radiographs ranged between 20% and 65%.8 In the trauma setting, PGY four and five GSR seem to have similar accuracy rates to radiologists in detecting injuries on both chest/pelvis radiographs; and the majority of missed injuries were clinically insignificant.3 In this study GSR misdiagnosed 37% of chest/abdomen radiographs and were inferior in performance compared to RR. Given the basic nature of the queried clinical images, the rate of misdiagnoses would be expected to be lower especially for senior residents. In centers with resident intensive care unit coverage, GS trainees are expected to immediately assess postplacement radiographs to assure the proper positioning of a central venous catheters, thoracostomy tubes, and nasogastric tubes. In such instances, awaiting a formal radiologist read may not be feasible due to patients' critical condition. Despite the lower scores in the assessment of clinical x-rays (chest/abdomen), both Jr. and Sr surgical trainees were able to localize placed catheters and tubes on plain radiographs. To our knowledge, this objective assessment was the first to display GSR's safety and reliability in providing correct preliminary interpretations of postplacement radiographs. Since the first installed CT scanner in 1971, surgeons have integrated this radiographic modality into their practice. Currently, CT scans have become indispensable to general surgery patient care. Its use in the perioperative period allows surgeons to meticulously plan procedures ahead of time such as in surgical oncology, vascular surgery, and thoracic surgery cases. Due to the short time duration required to scan patients and immediate availability of images, its use has expanded to the acute and trauma surgery setting. Hence, it is of utmost importance to assess GSR's ability to precisely interpret CT images. In a level one trauma center, one study reported that GSR correctly diagnosed 96% of head injuries and 94% of abdominal injuries, but 33% of thoracic injuries were
missed.2 Vorhies et al. concluded that senior GSR were equally proficient as radiologists at interpreting CT Head/Brain.3 This examination presented traumatic and non-traumatic causes of head pathologies that are typically encountered by GSR. Although the training level did not impact GSR performance, there were more interpretation errors than their radiology colleagues. The discrepancy between our results and previously published data may be due to the non-traumatic pathologies on head CT images being included. In the non-trauma setting, second year surgical residents had lower accuracy rates in detecting acute appendicitis on CT compared to radiologists.4 The participating GSR were equally competent to RR in interpreting CT of acute care service pathologies such as small bowel obstruction, pancreatitis, and pneumatosis intestinalis; however, this similarity in proficiency was due to GSSR superior performance on this modality. As expected, training and seniority improved acquisition of this skill. In this era of trauma care, surgeons perform and interpret focused ultrasonography in the emergency department, which has been shown to reliably identify hemoperitoneum and pericardial effusions. Senior surgical residents have been found to be proficient in evaluating traumatic abdominal injuries using focused ultrasonography.10,11 Our examination included both traumatic and nontraumatic pathologies seen on ultrasound. The lack of discrepancy between GSR and RR in correctly diagnosing pathologies on ultrasound may be due to surgical training's integration of dedicated education.11,12 Senior residents may have participated in more ultrasound courses than junior residents, hence demonstrating superiority in ultrasound interpretation. To our knowledge, this is the first assessment of GSR's interpretation skills of multiple modalities with images extrapolated from both trauma and non-trauma settings. Second, the resident enrollment from various types of GS programs and training levels ensured heterogeneity in the sampled population. Third, we have demonstrated that GSR were less competent than RR in the evaluation of basic radiology images that are routinely expected to be read independently. There are several limitations to our study. Despite multiple reminders, the participation rate was 43%. A greater participation rate may have improved the validity of the results. Second, an arbitrary limited sample of images were selected. Although they may represent common conditions encountered by GSR, it does not represent all possible pathologies. The addition of more images to the test may have altered our results, but it also may have induced decreased participation rates. Third, residents from the same program who enrolled at the same time may have shared and
Please cite this article in press as: Eid JJ, et al., Assessing surgical residents' imaging interpretation skills, The American Journal of Surgery (2016), http://dx.doi.org/10.1016/j.amjsurg.2016.11.029
4
J.J. Eid et al. / The American Journal of Surgery xxx (2016) 1e4
discussed answers which may have skewed results. Finally, the images incorporated into the exam were static, making the interpretation of dynamic modalities like CT and ultrasound more difficult. In summary, GSR were able to interpret 75% of basic radiology images, however major deficiencies were identified that may have a negative impact on patient care. In an attempt to provide high quality care, GS program directors should consider identifying gaps in medical education and tailor radiology curricula to both incoming and current general surgery residents. References 1. ABS Booklet of Information e Residency Experience for Initial Certification in General Surgery. 2. Arentz C, Griswold JA, Halldorsson A, et al. Accuracy of Surgery residents' interpretation of computed tomography scan in trauma. Am J Surg. 2008;196: 809e812. 3. Vorhies RW, Harrison PB, Smith RS, Helmer SD. Senior surgical residents can accurately interpret trauma radiographs. Am Surg. 2002;68(3):221e225.
4. Khan SA, Sleem R, Schiralli M, et al. Assessment of Surgical residents' ability to predict appendicitis with computed tomography. Am Surg. 2013;79(12): 1308e1310. 5. Tieng N, Grinberg D, Li SF. Discrepancies in interpretation of ED body computed tomographic scans by radiology residents. Am J Emerg Med. 2007;25:45e48. 6. Eisen LA, Berger JS, Hegde A, Schneider RF. Competency in chest radiography: a comparison of medical students, residents, and fellows. J Gen Intern Med. 2006;21(5):460e465. 7. Kaufman B, Dhar P, O'Neill DK, et al. Chest radiograph interpretation skills of anesthesiologists. J Cardiothorac Vasc Anesth. 2001;15(6):680e683. 8. Gatt ME, Spectre G, Paltiel O, et al. Chest radiographs in the emergency department: is the radiologist really necessary? Postgrad Med J. 2003;79: 214e217. 9. Butler KL, Chang Y, DeMoya M, et al. Needs assessment for a focused radiology curriculum in surgical residency: a multicenter study. Am J Surg. 2016;211: 279e287. 10. Kern SJ, Smith RS, Fry WR, et al. Sonographic examination of abdominal trauma by senior surgical residents. Am Surg. 1997;63(8):669e674. 11. Smith R, Kern SJ, Fry WR, Helmer SD. Institutional learning curve of surgeonperformed trauma ultrasound. Arch Surg. 1998;133(5):530e536. 12. Kotagal M, Quiroga E, Ruffatto BJ, et al. Impact of Point-of-Care ultrasound training on surgical residents' confidence. J Surg Educ. 2015;72(4):82e88.
Please cite this article in press as: Eid JJ, et al., Assessing surgical residents' imaging interpretation skills, The American Journal of Surgery (2016), http://dx.doi.org/10.1016/j.amjsurg.2016.11.029