- Email: [email protected]
Predictive value of general surgery application data for future resident performance
SURGICAL EDUCATION I INTRODUCTION: Attrition in general surgery is a significant problem facing residency programs. On average one out of five residents leaves general surgery for another field. We utilized a randomized, controlled trial to determine whether a brief targeted intervention designed to improve residents’ sense of belonging might decrease attrition.
Non-technical skills of surgery residents: does experiential learning lead to competence? Nicolas J Dedy, MD*, Boris Zevin, MD, Esther M Bonrath, MD, Teodor P Grantcharov, MD, PhD, FACS St. Michael’s Hospital Toronto, Ontario, Canada INTRODUCTION: Non-technical skills (NTS) play a key role in the delivery of safe patient care in the operating room (OR). As a result, regulatory bodies for accreditation and certification have included interpersonal and communication skills in the program requirements for graduate medical education in general surgery. However, controversy exists as to whether NTS are developed through experiential learning during residency. Thus, the purpose of the present study was to evaluate residents’ NTS at different levels of training. METHODS: General surgery residents of postgraduate years (PGY) 1, 3 and 5 without previous structured training in NTS participated in high fidelity hybrid-simulations that required management of standardized intraoperative crisis scenarios. Simulations were video recorded and reviewed by two trained observers who independently rated residents’ NTS using the Non-technical Skills for Surgeons (NOTSS) scale. RESULTS: The observers rated recordings of 36 residents: junior (PGY1, n¼12), intermediate (PGY3, n¼15) and senior (PGY5, n¼9) training levels. High agreement between observers was demonstrated on all four NOTSS categories (Intra-Class Correlation, .766-.899). Marginal or poor ratings (scores 1 or 2) in at least one category were found at all postgraduate levels (PGY1¼67%, n¼8; PGY3¼87%, n¼13; PGY5¼89%, n¼8). Comparison between groups (Kruskal-Wallis test) revealed no significant difference in NOTSS scores (Table). Junior: Intermediate: Senior: Kruskal Wallis PGY1 PGY3 PGY5 Test Situation awareness 3 (1-4) 2 (1-3) 3 (2-3) p¼.346 Decision making 2 (1-4) 2 (1-4) 2 (2-3) p¼.831 Communication/ teamwork 2 (1-3) 2 (1-4) 2 (1-3) p¼.960 Leadership 2.5 (1-4) 2 (1-4) 3 (2-3) p¼.188 Combined 9.5 (5-14) 8 (4-10) 10 (7-11) p¼.522
CONCLUSIONS: The rate of NTS failure was unacceptably high among all residents indicating that conventional surgical training does not lead to proficiency in these skills. Standardized training and assessment of NTS needs to be implemented in competencybased curricula in order to ensure ACGME/CanMEDS competency requirements are met in current surgical training. Belonging: a simple, brief intervention decreases burnout Arghavan Salles, MD, Kiruthiga Nandagopal, PhD, Greg Walton, PhD Stanford Hospital and Clinics, Menlo Park, CA
ª 2013 by the American College of Surgeons Published by Elsevier Inc.
METHODS: Participants were junior residents from seven surgical specialties (n¼35; 15 women). After a survey of attitudes and beliefs, we randomized men and women into either a belonging treatment or control condition. Junior residents read anecdotes from senior residents describing the challenges the seniors had faced early in residency (treatment condition) or describing challenging ethical dilemmas they had encountered early in residency (control condition). Several months later, juniors were asked about their attitudes and beliefs as a proxy for future intentions to leave residency. RESULTS: Residents who felt more like they belonged were more likely to think they would complete residency (r¼0.51, p<0.01) and less likely to see themselves as a different type of physician (r¼0.36, p<0.01). After the intervention, residents in the treatment condition had significantly lower rates of burnout than those in the control condition (p<0.05). CONCLUSIONS: The belonging intervention decreased the rate of burnout for those in the treatment condition. Because burnout positively correlates with absenteeism and high turnover rates, the residents in the treatment may be less likely to quit residency than those in the control. We intend to follow these residents for several years in order to assess the impact of our intervention on attrition. Predictive value of general surgery application data for future resident performance Erynne Azalea Faucett, MD, Albert Amini, MD, Asad E Patanwala, BS, Lucia Arenzana Lopez, BS, Jamie Harris, MD, Diane M Poskus, MS, William J Adamas-Rappaport, MD, Evan S Ong, MD, FACS, Amy L Waer, MD, FACS University of Arizona, Phoenix, AZ INTRODUCTION: USMLE Step 1 and 2 scores have become increasingly important factors in both the screening and selection of general surgery residents. The purpose of this study was to evaluate the association between USMLE scores and clinical and academic performance among surgery residents. METHODS: This was a retrospective cohort study of consecutive categorical resident files from 1998-2008 at our institution. The effect of demographic variables and USMLE Step 1 and 2 scores on clinical and academic performance was evaluated using regression analyses. Clinical performance was assessed using Subjective Clinical Performance Evaluations. Academic performance was assessed using Absite scores and ABS Qualifying Examination (QE) first time pass rates.
S116
ISSN 1072-7515/13/$36.00 http://dx.doi.org/10.1016/j.jamcollsurg.2013.07.266
Vol. 217, No. 3S, September 2013
RESULTS: A total of 69 residents were included in the final cohort. Both USMLE Step 1 and 2 scores were positively associated with Absite scores, with the strongest relationship being with PGY1 scores (R2 ¼0.68 and 0.63, respectively; p<0.001 for each). Step 2 had a stronger association with Clinical Performance Evaluations than Step 1 (R2 ¼0.21 and 0.12, respectively; p<0.05 for each). Step 1 was significantly associated with first time ABS QE passage rates. The mean Step 1 score was 196 in those who did not pass and 220 in those who passed (p<0.001). CONCLUSIONS: Step 1 had a stronger association with ABS QE first time passage rates. However Step 2 had a stronger association with Clinical Performance Evaluations. Step 2 may be a better predictor of clinical performance than Step 1 during surgical residency training. Impact of a third-year surgical apprenticeship model: perceptions and attitudes compared to the traditional medical student clerkship experience Chris M Reid, MD, Dennis Y Kim, MD, Jess Mandel, MD, Alan Smith, Mark A Talamini, MD, FACS, Vishal Bansal, MD, FACS University of California, San Diego Medical Center, San Diego, CA INTRODUCTION: Current literature suggests that medical students may have negative misconceptions of a surgical career partly because of the traditional hierarchical structure of the surgical clerkship. We hypothesized that a novel medical student apprenticeship would result in positive changes in perceptions of both surgeons and surgical careers. METHODS: In the 2011 academic year, third-year medical students were offered a 2 week apprenticeship elective in addition to the standard 8 week surgical clerkship. Unlike a traditional service, students apprenticed directly in a one on one basis with participating faculty. At the clerkships end, a structured questionnaire was distributed assessing perceptions and attitudes towards surgeons and surgery. Subjects responded anonymously using a 5-point Likert scale. A Wilcoxon Rank-Sum was performed comparing students that participated versus those that did not participate in the apprenticeship. RESULTS: There was a 99% survey response (105/106). Of those, 50 (48%) participated in the apprenticeship. Apprenticeship students were more likely to view surgeons as content (p<0.001), well-balanced (p<0.01), respectful (p¼0.01), and as role models (p<0.005). Apprenticeship students were also more likely to participate in the operating room (p<.05) and in patient management (p<0.05). There was no difference in an interest to pursue a surgical career between groups both preceding and upon completion of the clerkship. CONCLUSIONS: Students participating in a surgical apprenticeship had a more positive view of both surgeons and the field of surgery compared to students who did not participate. An
Surgical Forum Abstracts
S117
apprenticeship model may enhance the surgical clerkship experience and improve medical student perceptions of surgery as a career. The influence of dedicated research time during general surgery residency on ABSITE and Board exam performance Jeffrey S Carson, MD, Elizabeth A Johnston, MD, Lynette M Smith, MS, PJ Schenarts, MD, FACS, Sean J Langenfeld, MD, Kenneth S Azarow, MD, FACS, David W Mercer, MD, FACS, Jon S Thompson, MD, FACS, Chandrakanth Are, MD, FRCSI, FACS University of Nebraska Medical Center, Omaha, NE INTRODUCTION: The influence of dedicated research time on American Board of Surgery In-Training Examination (ABSITE) scores and performance on American Board of Surgery (ABS) examinations is unclear. The aims of this study were to analyze the influence of dedicated research time during general surgery residency on a) ABSITE scores and b) first time pass rates on ABS Qualifying and Certifying examinations. METHODS: The ABSITE and ABS results were obtained for residents from a single academic general surgery residency program (1993-2011). Comparative analyses were performed to detect differences in a) ABSITE scores between research and non-research residents and b) first time pass rates on ABS exams for research and non-research residents. The Wilcoxon signed rank test and Fisher’s exact test were utilized for statistical analysis with a p-value of 0.05 considered significant. RESULTS: A total of 59 residents were included of whom 20 residents took dedicated time out of clinical years to pursue research. When compared to non-research residents, there was no difference in ABSITE scores for the research residents after matching for clinical years (p¼0.18-0.95) (Table 1). Similarly, there were no differences in the first time pass rates on the ABS Qualifying and Certifying examinations between the research and non-research residents (p¼0.14 and 0.31 respectively).
Clinical year (CY)
CY CY CY CY CY CY
I II III a III b IV V
Research residents’ median scores
Non-research residents’ median scores
Unadjusted p value
61 44 69 57 54 40
48 52 40 40 46 41
0.95 0.42 0.85 0.18 0.23 0.91
CY III a ¼ Clinical year III scores prior to research years. CY III b ¼ Clinical year III scores after research years.
CONCLUSIONS: The results of our study demonstrate the lack of influence of dedicated research time during general surgery residency on ABSITE and ABS examinations.
Non-technical skills of surgery residents: does experiential learning lead to competence? Nicolas J Dedy, MD*, Boris Zevin, MD, Esther M Bonrath, MD, Teodor P Grantcharov, MD, PhD, FACS St. Michael’s Hospital Toronto, Ontario, Canada INTRODUCTION: Non-technical skills (NTS) play a key role in the delivery of safe patient care in the operating room (OR). As a result, regulatory bodies for accreditation and certification have included interpersonal and communication skills in the program requirements for graduate medical education in general surgery. However, controversy exists as to whether NTS are developed through experiential learning during residency. Thus, the purpose of the present study was to evaluate residents’ NTS at different levels of training. METHODS: General surgery residents of postgraduate years (PGY) 1, 3 and 5 without previous structured training in NTS participated in high fidelity hybrid-simulations that required management of standardized intraoperative crisis scenarios. Simulations were video recorded and reviewed by two trained observers who independently rated residents’ NTS using the Non-technical Skills for Surgeons (NOTSS) scale. RESULTS: The observers rated recordings of 36 residents: junior (PGY1, n¼12), intermediate (PGY3, n¼15) and senior (PGY5, n¼9) training levels. High agreement between observers was demonstrated on all four NOTSS categories (Intra-Class Correlation, .766-.899). Marginal or poor ratings (scores 1 or 2) in at least one category were found at all postgraduate levels (PGY1¼67%, n¼8; PGY3¼87%, n¼13; PGY5¼89%, n¼8). Comparison between groups (Kruskal-Wallis test) revealed no significant difference in NOTSS scores (Table). Junior: Intermediate: Senior: Kruskal Wallis PGY1 PGY3 PGY5 Test Situation awareness 3 (1-4) 2 (1-3) 3 (2-3) p¼.346 Decision making 2 (1-4) 2 (1-4) 2 (2-3) p¼.831 Communication/ teamwork 2 (1-3) 2 (1-4) 2 (1-3) p¼.960 Leadership 2.5 (1-4) 2 (1-4) 3 (2-3) p¼.188 Combined 9.5 (5-14) 8 (4-10) 10 (7-11) p¼.522
CONCLUSIONS: The rate of NTS failure was unacceptably high among all residents indicating that conventional surgical training does not lead to proficiency in these skills. Standardized training and assessment of NTS needs to be implemented in competencybased curricula in order to ensure ACGME/CanMEDS competency requirements are met in current surgical training. Belonging: a simple, brief intervention decreases burnout Arghavan Salles, MD, Kiruthiga Nandagopal, PhD, Greg Walton, PhD Stanford Hospital and Clinics, Menlo Park, CA
ª 2013 by the American College of Surgeons Published by Elsevier Inc.
METHODS: Participants were junior residents from seven surgical specialties (n¼35; 15 women). After a survey of attitudes and beliefs, we randomized men and women into either a belonging treatment or control condition. Junior residents read anecdotes from senior residents describing the challenges the seniors had faced early in residency (treatment condition) or describing challenging ethical dilemmas they had encountered early in residency (control condition). Several months later, juniors were asked about their attitudes and beliefs as a proxy for future intentions to leave residency. RESULTS: Residents who felt more like they belonged were more likely to think they would complete residency (r¼0.51, p<0.01) and less likely to see themselves as a different type of physician (r¼0.36, p<0.01). After the intervention, residents in the treatment condition had significantly lower rates of burnout than those in the control condition (p<0.05). CONCLUSIONS: The belonging intervention decreased the rate of burnout for those in the treatment condition. Because burnout positively correlates with absenteeism and high turnover rates, the residents in the treatment may be less likely to quit residency than those in the control. We intend to follow these residents for several years in order to assess the impact of our intervention on attrition. Predictive value of general surgery application data for future resident performance Erynne Azalea Faucett, MD, Albert Amini, MD, Asad E Patanwala, BS, Lucia Arenzana Lopez, BS, Jamie Harris, MD, Diane M Poskus, MS, William J Adamas-Rappaport, MD, Evan S Ong, MD, FACS, Amy L Waer, MD, FACS University of Arizona, Phoenix, AZ INTRODUCTION: USMLE Step 1 and 2 scores have become increasingly important factors in both the screening and selection of general surgery residents. The purpose of this study was to evaluate the association between USMLE scores and clinical and academic performance among surgery residents. METHODS: This was a retrospective cohort study of consecutive categorical resident files from 1998-2008 at our institution. The effect of demographic variables and USMLE Step 1 and 2 scores on clinical and academic performance was evaluated using regression analyses. Clinical performance was assessed using Subjective Clinical Performance Evaluations. Academic performance was assessed using Absite scores and ABS Qualifying Examination (QE) first time pass rates.
S116
ISSN 1072-7515/13/$36.00 http://dx.doi.org/10.1016/j.jamcollsurg.2013.07.266
Vol. 217, No. 3S, September 2013
RESULTS: A total of 69 residents were included in the final cohort. Both USMLE Step 1 and 2 scores were positively associated with Absite scores, with the strongest relationship being with PGY1 scores (R2 ¼0.68 and 0.63, respectively; p<0.001 for each). Step 2 had a stronger association with Clinical Performance Evaluations than Step 1 (R2 ¼0.21 and 0.12, respectively; p<0.05 for each). Step 1 was significantly associated with first time ABS QE passage rates. The mean Step 1 score was 196 in those who did not pass and 220 in those who passed (p<0.001). CONCLUSIONS: Step 1 had a stronger association with ABS QE first time passage rates. However Step 2 had a stronger association with Clinical Performance Evaluations. Step 2 may be a better predictor of clinical performance than Step 1 during surgical residency training. Impact of a third-year surgical apprenticeship model: perceptions and attitudes compared to the traditional medical student clerkship experience Chris M Reid, MD, Dennis Y Kim, MD, Jess Mandel, MD, Alan Smith, Mark A Talamini, MD, FACS, Vishal Bansal, MD, FACS University of California, San Diego Medical Center, San Diego, CA INTRODUCTION: Current literature suggests that medical students may have negative misconceptions of a surgical career partly because of the traditional hierarchical structure of the surgical clerkship. We hypothesized that a novel medical student apprenticeship would result in positive changes in perceptions of both surgeons and surgical careers. METHODS: In the 2011 academic year, third-year medical students were offered a 2 week apprenticeship elective in addition to the standard 8 week surgical clerkship. Unlike a traditional service, students apprenticed directly in a one on one basis with participating faculty. At the clerkships end, a structured questionnaire was distributed assessing perceptions and attitudes towards surgeons and surgery. Subjects responded anonymously using a 5-point Likert scale. A Wilcoxon Rank-Sum was performed comparing students that participated versus those that did not participate in the apprenticeship. RESULTS: There was a 99% survey response (105/106). Of those, 50 (48%) participated in the apprenticeship. Apprenticeship students were more likely to view surgeons as content (p<0.001), well-balanced (p<0.01), respectful (p¼0.01), and as role models (p<0.005). Apprenticeship students were also more likely to participate in the operating room (p<.05) and in patient management (p<0.05). There was no difference in an interest to pursue a surgical career between groups both preceding and upon completion of the clerkship. CONCLUSIONS: Students participating in a surgical apprenticeship had a more positive view of both surgeons and the field of surgery compared to students who did not participate. An
Surgical Forum Abstracts
S117
apprenticeship model may enhance the surgical clerkship experience and improve medical student perceptions of surgery as a career. The influence of dedicated research time during general surgery residency on ABSITE and Board exam performance Jeffrey S Carson, MD, Elizabeth A Johnston, MD, Lynette M Smith, MS, PJ Schenarts, MD, FACS, Sean J Langenfeld, MD, Kenneth S Azarow, MD, FACS, David W Mercer, MD, FACS, Jon S Thompson, MD, FACS, Chandrakanth Are, MD, FRCSI, FACS University of Nebraska Medical Center, Omaha, NE INTRODUCTION: The influence of dedicated research time on American Board of Surgery In-Training Examination (ABSITE) scores and performance on American Board of Surgery (ABS) examinations is unclear. The aims of this study were to analyze the influence of dedicated research time during general surgery residency on a) ABSITE scores and b) first time pass rates on ABS Qualifying and Certifying examinations. METHODS: The ABSITE and ABS results were obtained for residents from a single academic general surgery residency program (1993-2011). Comparative analyses were performed to detect differences in a) ABSITE scores between research and non-research residents and b) first time pass rates on ABS exams for research and non-research residents. The Wilcoxon signed rank test and Fisher’s exact test were utilized for statistical analysis with a p-value of 0.05 considered significant. RESULTS: A total of 59 residents were included of whom 20 residents took dedicated time out of clinical years to pursue research. When compared to non-research residents, there was no difference in ABSITE scores for the research residents after matching for clinical years (p¼0.18-0.95) (Table 1). Similarly, there were no differences in the first time pass rates on the ABS Qualifying and Certifying examinations between the research and non-research residents (p¼0.14 and 0.31 respectively).
Clinical year (CY)
CY CY CY CY CY CY
I II III a III b IV V
Research residents’ median scores
Non-research residents’ median scores
Unadjusted p value
61 44 69 57 54 40
48 52 40 40 46 41
0.95 0.42 0.85 0.18 0.23 0.91
CY III a ¼ Clinical year III scores prior to research years. CY III b ¼ Clinical year III scores after research years.
CONCLUSIONS: The results of our study demonstrate the lack of influence of dedicated research time during general surgery residency on ABSITE and ABS examinations.