- Email: [email protected]
GYN Residencies and Recommendations for Future Research
ARTICLE IN PRESS ORIGINAL REPORT
ACOG Simulation Working Group: A Needs Assessment of Simulation Training in OB/GYN Residencies and Recommendations for Future Research Christopher C. DeStephano, MD, MPH,*,{ Joshua F. Nitsche, MD, PhD,†,{ Michael G. Heckman, MS,*,‡ Erika Banks, MD,x,{ and Hye-Chun Hur, MD, MPHǁ,{ Mayo Clinic Department of Surgical Gynecology, Jacksonville, Florida; †Wake Forest School of Medicine Department of OB/GYN, Winston-Salem, North Carolina; ‡Mayo Clinic Division of Biomedical Statistics and Informatics, Jacksonville, Florida; §Department of Obstetrics and Gynecology and Women’s Health, Albert Einstein College of Medicine, New York, New York; and ǁDivision of Gynecologic Specialty Surgery, Department of Obstetrics and Gynecology, New York Presbyterian Hospital, Columbia University Medical Center, New York, New York
*
OBJECTIVE: To evaluate current availability and needs of
simulation training among obstetrics/gynecology (OB/ GYN) residency programs. DESIGN: Cross-sectional survey. SETTING: Accreditation Council for Graduate Medical Education accredited OB/GYN residency programs in the United States. PARTICIPANTS: Residency program directors, gynecol-
ogy simulation faculty, obstetrics simulation faculty, and fourth-year residents. RESULTS: Of 673 invited participants, 251 (37.3%) com-
and Gynecology (47.2%) and American College of Obstetrics and Gynecology (27.8%) simulation tools compared to the majority of faculty (84.7% and 72.1%, respectively). More than 80% of trainees and faculty reported they felt the average graduating resident could perform vaginal, laparoscopic, and abdominal hysterectomies independently. CONCLUSIONS: Simulation is now widely available for
both gynecologic and obstetric procedures, but there remains tremendous heterogeneity between programs and the perceptions of residents, program directors, and faculty. The variations in simulation training and readiness for performing different procedures following residency support the need for objective, validated assessments of actual performance to better guide resident learning and faculty teaching efforts. ( J Surg Ed 000:110. Ó 2019 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
pleted the survey. Among the survey responses, OB procedures were more broadly represented compared to the GYN procedures for simulation teaching: 8 (50%) of 16 OB procedures versus 4 (18.2%) of 22 GYN procedures had simulation teaching. Among the simulated procedures, a majority of residents and faculty reported that simulation teaching was available for operative vaginal delivery, postpartum hemorrhage, shoulder dystocia, perineal laceration repair, conventional laparoscopic procedures, and robotic surgery. There were significant differences between residents and faculty perceptions regarding the availability and needs of simulated procedures with a minority of residents having knowledge of Council on Resident Education in Obstetrics
KEY WORDS: gynecology, needs assessment, obstetrics, surgical simulation, surgical performance
Funding source: None. Correspondence: Inquiries to Christopher DeStephano, MD Mayo Clinic, Department of Surgical Gynecology, 4500 San Pablo Road, Jacksonville, FL 32224; fax: (904)-953-0606; e-mail: [email protected]
Simulation teaching has become a part of the training curricula for many obstetrics/gynecology (OB/GYN) residency programs, as simulation has shown promise in teaching and evaluating the performance of trainees and attending physicians. A previous meta-analysis of
{ Members of the ACOG Simulation Working Group.
COMPETENCIES: Medical Knowledge, Practice-Based
Learning and Improvement, Systems-Based Practice
INTRODUCTION
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
1
ARTICLE IN PRESS simulation-based educational assessments demonstrated that these assessments correlate positively with patientrelated outcomes.1 The American College of Obstetrics and Gynecology (ACOG) Simulations Working Group and the Council on Resident Education in Obstetrics and Gynecology (CREOG) have developed and implemented simulationsbased curricula as an adjunct to improve residency education and clinical competence during and after residency.2 The objectives of the ACOG Simulations Working Group is to “develop standardized curricula available to all residency programs, develop standardized teaching and evaluation methods practiced and validated by the Working group to ensure efficacy of simulation education, and to provide validation of simulation-based education for developing and improving surgical, clinical, and behavioral skills.” In order to assess the extent to which simulation is being used at residency programs throughout the United States, and to better determine the state of simulation training in residency, we conducted a needs assessment survey. Our objectives were to evaluate the current state of simulation training resources available to OB/GYN residents and to assess current perceptions regarding when trainees are ready to perform procedures independently.
METHODS Study Design and IRB Approval This was a cross-sectional survey of Accreditation Council for Graduate Medical Education (ACGME)-accredited OB/GYN residency program directors, gynecology simulation faculty, obstetrics simulation faculty, and fourthyear residents. The study was determined to be exempt by institutional review boards at ACOG (IRB # 012) and Beth Israel Deaconess Medical Center (IRB# 2016P000348) under the Code of Federal Regulations, 45 CFR 46.102.
sent to all fourth year residents (i.e., multiple residents from certain institutions would have been more likely to respond at ACOG simulation working group sites). The survey was sent electronically via REDCap from March through June 2017.3 Survey Instrument A multi-item survey was developed and reviewed by representatives of the ACOG Simulations Working Group for accuracy and content validity. The simulation working group consists of 2 representatives from each of 24 ACGME accredited OB/GYN residency programs (i.e., 48 representatives). The first part of the survey assessed demographic information, simulation availability, and additional simulation needs. The second part of the survey evaluated respondents’ perception of the current residency training for common OB/GYN procedures identified from the CREOG 11th Edition of Educational Objectives. The survey questions were framed differently for faculty and residents. Residents were asked whether they “feel confident performing the following procedures independently” by selecting their level of agreement on a 5 level Likert scale ranging from strongly disagree to strongly agree. Faculty and program directors were asked to “specify level of training when the average trainee is ready to perform the procedure independently” by selecting either at the completion of PGY1, PGY2, PGY3, PGY4, or requires additional training after residency. Statistical Analysis The analysis is primarily descriptive, with continuous variables presented as median (interquartile range) and categorical variables presented as frequency (percentage). Categorical variables were compared between groups using a chi-square test or Fisher’s exact test. p values less than 0.05 were considered statistically significant and all tests were 2-sided. All statistical analysis was performed using SAS (version 9.4; SAS Institute, Inc., Cary, North Carolina).
Participant Invitations and Survey Distribution To identify eligible participants, an electronic e-mail form was sent from ACOG to all 256 ACGME-accredited residency program coordinators requesting contact information for the program director, an obstetric faculty member knowledgeable about the obstetric simulation curriculum, a gynecology faculty member knowledgeable about the gynecology simulation curriculum, and a fourth-year resident knowledgeable of the program’s simulation curriculum. The decision to only send the survey to 1 fourth-year resident was due to the deidentified nature of the survey and the possibility of over-representing certain programs if the survey was
2
RESULTS The survey response rate was as follows: 42.3% (99 of 234) program directors (PDs), 64.5% (91 of 141) gynecology simulation faculty (which may have also responded as obstetrics simulation faculty), 42.6% (60 of 141) obstetrics simulation faculty (which may have also responded as gynecology simulation faculty), and 25.5% (41 of 161) fourth-year residents. Although 41 fourthyear residents responded, 5 were incomplete responses, and thus, were excluded from the final analysis to maintain demographic consistency. Of 256 coordinators e-
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
TABLE 1. Demographics of Respondents
Age Gender* Female Male Program type University University-affiliated Community Military health system Total number of residents in the program
PGY4 Residents n = 36
Program Directors (PDs) n = 99
Simulation Faculty n = 116
30.5 (30.0-31.0)
47.0 (42.0-53.0)
43.0 (38.0-50.0)
31 (86.1) 5 (13.9)
54 (55.1) 44 (44.9)
85 (73.3) 31 (26.7)
13 (36.1) 12 (33.3) 11 (30.6) 0 (0.0) 16 (12-25)
54 (54.6) 18 (18.2) 26 (26.3) 1 (1.0) 20 (16-25)
70 (60.3) 26 (23.3) 19 (16.4) 0 (0.0) 23 (16-28)
*Information regarding sex was unavailable for 1 program director. Data presented as median (interquartile range) or n (%).
mailed, 234 (91.4%) responded, however, they did not provide all categories of representatives (e.g., faculty, program director, and residents) requested which resulted in different denominators for each type of representative (i.e., only 161 programs sent an email address for a fourth-year resident representative). Demographic characteristics of respondents are presented in Table 1. There were no significant differences between program director and simulation faculty responses for simulation needs and resident preparedness thus these 2 groups were combined to facilitate comparison of faculty to resident responses in the subsequent tables. The majority of OB/GYN respondents reported availability of simulation curricula and teaching resources for both obstetric and gynecologic procedures (Table 2A). When comparing responses from fourth-year residents and faculty (program directors and simulation faculty), faculty reported more availability of simulation teaching resources compared to residents, which was statistically significant for OB simulation teaching (p = 0.02), gynecology surgical curriculum (p = 0.008), and dedicated teaching time (p = 0.03; Table 2A). Reported availability of additional gynecologic simulation resources are presented in Table 2B. Similar to simulation teaching resources, faculty were more often aware of available online simulation resources such as the ACOG Simulation Toolkit (p < 0.0001) or CREOG surgical curriculum (p < 0.0001) than fourth-year residents. The availability of simulation training for specific gynecologic procedures is presented in Table 3. Both faculty and residents reported simulation teaching to be least available for vaginal surgery and laparotomy, and most available for conventional laparoscopic surgery and robotic surgery. Again, compared to faculty responses, residents consistently reported less availability of simulation teaching for all modes of surgery. The largest difference between faculty and residents was reported in the
Journal of Surgical Education Volume 00 /Number 00 Month 2019
availability of vaginal surgery simulation (8.3% residents versus 37.4% faculty reported vaginal simulation teaching, p = 0.0007). Faculty and trainees both identified total laparoscopic hysterectomy (TLH; 57.9% faculty, 47.2% residents) followed by LEEP (46.3% faculty, 33.3% residents) as the most common GYN procedures in simulation training. Availability and reported additional needs for obstetric simulation are displayed in Table 4. Additional TABLE 2A. Reported Availability of Simulation Curriculum and Teaching Resources Obstetrics
Procedures curriculum Simulation teaching Teaching staff Dedicated teaching time Gynecology Surgical curriculum Simulation teaching Teaching staff Dedicated teaching time
PGY4 Residents n (%)
PDs and Sim faculty n (%)
N = 36
n = 159
p value
22 (61.1)
104 (65.4)
0.83
29 (80.6)
150 (94.3)
0.02
23 (63.9) 23 (63.9)
100 (62.9) 124 (78.0)
0.66 0.10
N = 36 18 (50.0)
N = 190 134 (70.5)
0.008
30 (83.3)
174 (91.6)
0.22
17 (47.2) 24 (66.7)
116 (61.1) 158 (83.2)
0.36 0.03
Sim faculty, simulation faculty. p values result from a chi-square test or Fisher’s exact test. Program directors and simulation faculty were combined as responses were similar between these 2 groups. Response rates for faculty and PDs were different due to different practice patterns of OB/GYN faculty (e.g., some obstetric faculty and PDs no longer practice gynecology and vice versa).
3
ARTICLE IN PRESS
TABLE 2B. Additional Gynecology Simulation Resources Available Resident (n = 36) n (%)
PDs and GYN Sim Faculty (n = 190) n (%)
p value
31 (86.1) 24 (66.7) 31 (86.1) 5 (13.9) 29 (80.6) 30 (83.3) 17 (47.2) 7 (19.4) 9 (25.0) 6 (16.7) 1 (2.8) 17 (47.2) 7 (19.4) 10 (27.8) 3 (8.3)
156 (82.1) 136 (71.6) 168 (88.4) 72 (37.9) 167 (87.9) 142 (74.7) 124 (65.3) 74 (39.0) 83 (43.7) 51 (26.8) 25 (13.2) 161 (84.7) 78 (41.1) 137 (72.1) 60 (31.6)
0.46 0.35 0.87 0.01 0.02 0.33 0.07 0.05 0.07 0.26 0.02 <0.0001 0.40 <0.0001 0.74
Simulation lab access Trainees go to simulation lab Simulation tools available Laparotomy simulation Laparoscopic simulation Robotic simulation Hysteroscopic simulation Vaginal simulation Minimally invasive gynecologic surgery division Fundamentals of Laparoscopic Surgery (FLS) offered Fundamentals of Laparoscopic Surgery (FLS) required Aware of CREOG Surgical Curriculum Use CREOG Surgical Curriculum Aware of ACOG simulation toolkit Use ACOG simulation toolkit
GYN Sim faculty, gynecology simulation faculty. p values result from a chi-square test or Fisher’s exact test. Program directors and GYN simulation faculty were combined as responses were similar between these 2 groups.
simulation teaching for cesarean hysterectomy was the most commonly reported obstetric need (54.1% faculty, 50% of residents, p = 0.66). Among specific OB procedures, shoulder dystocia (89.3% faculty, 72.2% residents, p = 0.01) followed by postpartum hemorrhage (88.1% faculty, 69.4% residents, p = 0.005) were the most commonly reported OB simulation procedures. Faculty generally reported a higher level of availability of simulated obstetric procedures than residents reported. Table 5 presents the perception of trainee preparedness for performing hysterectomies independently. Overall, very few faculty perceived that residents required additional training for total abdominal hysterectomy (TAH 2.2%), TLH (3.3%), and total vaginal hysterectomy (TVH 6.6%). Although the question was asked on a Likert scale of agreement for PGY responses, this trend of perceiving that a procedure could be performed independently was similar for the residents who responded. Of 41 PGY4 residents, 40 (97.6%) stated agreement or strong agreement with the statement that he or she could perform a TAH independently, 37 (90.2%) for TLH, and 80.5% for TVH. Table 6 summarizes a wide range of trainee preparedness for performing procedures independently following residency. PGY 4 responses were based on stating “agree” or “strongly agree” on the statement they could perform the procedure independently while faculty responses were based on the proportion who selected by the completion of PGY4. In general, >80% faculty and PDs reported that residents were prepared to perform the procedures independently except for the following: amniocentesis (PDs 70.1%, 61.7% OB sim
4
faculty), breech vaginal delivery (75.6% PDs, 71.7% OB sim faculty), midurethral sling (74.7% PDs, 79.1% GYN sim faculty), laparoscopic myomectomy (71.7% PDs, 58.0% GYN sim faculty), and robotic surgery (69.7% PDs, 58.2% GYN sim faculty). To limit survey response fatigue, the OB section of the working group did not ask about resident preparedness to complete OB procedures. Less than 80% of PGY4s agreed with the statement that they could perform the following procedures independently: midurethral sling (44.4%), vaginal anterior/posterior repair (66.7%), laparoscopic myomectomy (41.7%), vaginal surgery (77.8%), and robotic surgery (52.8%).
DISCUSSION Responses from this survey suggest the majority of ACGME-accredited OB/GYN residency programs in the United States have simulation resources available to their trainees. Outside of vaginal surgery and laparotomy, less than 50% of respondents reported the need for additional simulation teaching of specific procedures. The majority of respondents reported that graduating residents could perform TLH, TVH, and TAH independently in practice. Our results suggest there has been an increase in the percentage of programs reporting a formal didactic surgical curriculum. In a needs assessment of program directors conducted by Mandel et al. in 2001, only 29% of program directors reported that their programs had formal surgical curricula.4 In our survey, 70.5% of
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
TABLE 3. Availability and Additional Needs for Simulated Gynecologic Procedures Available PGY4 Residents N = 36 Mode of incision Laparotomy Conventional laparoscopic surgery Robotic surgery Hysteroscopy Vaginal surgery Specific GYN procedures Loop electrosurgical excision procedure Cold knife conization Dilation and curettage Dilation and evacuation Cystoscopy Total abdominal hysterectomy Total laparoscopic hysterectomy Total vaginal hysterectomy Abdominal myomectomy Laparoscopic myomectomy Hysteroscopic myomectomy Laparoscopic salpingectomy Laparoscopic bilateral salpingooophorectomy Vaginal anterior repair Vaginal posterior repair Midurethral sling Surgical complications
Additional Needs
PDs and GYN Sim Faculty N = 190
p Value
PGY4 Residents N = 36
PDs and GYN Sim Faculty N = 190
p Value
5 (13.9) 23 (63.9)
78 (41.1) 153 (80.5)
0.002 0.03
15 (41.7) 13 (36.1)
113 (59.5) 52 (27.4)
0.048 0.29
19 (52.8) 17 (47.2) 3 (8.3)
120 (63.2) 114 (60.0) 71 (37.4)
0.24 0.15 0.0007
5 (13.9) 6 (16.7) 22 (61.1)
41 (21.6) 51 (26.8) 127 (66.8)
0.29 0.20 0.51
12 (33.3)
88 (46.3)
0.15
4 (11.1)
40 (21.1)
0.17
2 (5.6)
38 (20.0)
0.04
3 (8.3)
47 (24.7)
0.03
5 (13.9)
53 (27.9)
0.08
2 (5.6)
28 (14.7)
0.18
2 (5.6)
28 (14.7)
0.18
3 (8.3)
52 (27.4)
0.01
4 (11.1) 5 (13.9)
63 (33.2) 49 (25.8)
0.008 0.12
3 (8.3) 9 (25.0)
43 (22.6) 78 (41.1)
0.05 0.07
17 (47.2)
110 (57.9)
0.24
6 (16.7)
56 (29.5)
0.11
15 (41.7)
99 (52.1)
0.25
7 (19.4)
71 (37.4)
0.04
3 (8.3)
75 (39.5)
0.0003
0 (0.0)
4 (2.1)
2 (5.6)
24 (12.6)
0.39
11 (30.6)
98 (51.6)
0.02
8 (22.2)
45 (23.7)
0.85
4 (11.1)
60 (31.6)
0.01
9 (25.0)
69 (36.3)
0.19
2 (5.6)
41 (21.6)
0.02
9 (25.0)
51 (26.8)
0.82
2 (5.6)
43 (22.6)
0.02
0 (0.0)
13 (6.8)
0.23
11 (30.6)
105 (55.3)
0.007
0 (0.0)
13 (6.8)
0.23
12 (33.3)
105 (55.3)
0.02
6 (16.7) 3 (8.3)
45 (23.7) 24 (12.6)
0.36 0.58
15 (41.7) 11 (30.6)
86 (45.3) 86 (45.3)
0.69 0.10
1.0
GYN sim faculty, gynecology simulation faculty. p values result from a chi-square test or Fisher’s exact test. Program directors and GYN simulation faculty were combined as responses were similar between these 2 groups.
program directors and simulation faculty reported their programs had surgical curricula. Mandel and colleagues also found that 64% of program directors reported that laparoscopic hysterectomy was an essential skill for third and fourth-year residents to learn. In contrast, in our survey, 94 (94.9%) of 99 program directors reported that the average graduating resident could complete a TLH
independently at the completion of residency. Compared to laparoscopic simulation training, there was less availability and more perceived need for abdominal and vaginal procedure simulation in our survey; vaginal surgery, laparotomy, and cesarean hysterectomy were most often reported to be desired for additional simulation training. These results are consistent with the
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
5
ARTICLE IN PRESS
TABLE 4. Availability and Additional Needs for Simulated Obstetric Procedures Available PGY4 Residents N = 36 Labor cervical exam Cephalic vaginal delivery Breech vaginal delivery Operative vaginal delivery Postpartum hemorrhage Maternal cardiac arrest Shoulder dystocia Preeclampsia/ eclampsia Routine cesarean section Emergency cesarean section Obstetrical ultrasound Perineal laceration repair Amniocentesis Cervical cerclage Cesarean hysterectomy External cephalic version
Additional Needs
PDs and OB Sim Faculty n = 159
11 (30.6)
86 (54.1)
15 (41.7)
p Value
PGY4 Residents N = 36
0.01
4 (11.1)
115 (72.3)
0.004
11 (30.6)
106 (66.7)
<0.0001
19 (52.8)
126 (79.3)
25 (69.4)
PDs and OB Sim Faculty n = 159
p Value
15 (9.4)
0.76
2 (5.6)
7 (4.4)
0.67
13 (36.1)
28 (17.6)
0.02
0.001
8 (22.2)
31 (19.5)
0.71
140 (88.1)
0.005
1 (2.8)
14 (8.8)
0.31
12 (33.3)
82 (51.6)
0.048
14 (38.9)
46 (28.9)
0.24
26 (72.2) 17 (47.2)
142 (89.3) 98 (61.6)
0.01 0.11
2 (5.6) 5 (13.9)
14 (8.8) 23 (14.5)
0.74 0.93
12 (33.3)
58 (36.5)
0.72
3 (8.3)
32 (20.1)
0.10
8 (22.2)
39 (24.5)
0.77
12 (33.3)
41 (25.8)
0.36
6 (16.7)
43 (27.0)
0.19
8 (22.2)
54 (34.0)
0.17
19 (52.8)
118 (74.2)
0.01
5 (13.9)
20 (12.6)
0.79
4 (11.1) 0 (0.0) 2 (5.6)
22 (13.8) 13 (8.2) 11 (6.9)
0.79 0.13 1.0
17 (47.2) 11 (30.6) 18 (50.0)
66 (41.5) 64 (40.3) 86 (54.1)
0.53 0.28 0.66
1 (2.8)
14 (8.8)
0.31
12 (33.3)
79 (49.7)
0.08
OB sim faculty, obstetrics simulation faculty. p values result from a chi-square test or Fisher’s exact test. Program directors and OB simulation faculty were combined as responses were similar between these 2 groups.
observation that the number of open abdominal and vaginal surgeries are decreasing in residency programs.5 Opportunities for robotic simulation training are also rapidly expanding with numerous studies establishing validity evidence.6-14 Robotic simulation resources were reported as available by 83.3% of residents and 74.7% of PDs and GYN simulation faculty in Table 2B. However, as reported in Table 3, when asked if the respondent’s program “provides simulation training for robotic surgery” 52.8% of residents and 63.2% of PDs and GYN simulation faculty responded in the affirmative and only 13.9% of residents and 21.6% of PDs and GYN simulation faculty reported that there was an additional need for robotic simulation training. Perceived preparedness of graduating residents to perform robotic surgery independently was reported by 52.8% of residents, 69.7% of PDs, and 58.2% of GYN simulation faculty. These results suggest that robotic surgery simulation is less of an
6
emphasis than other procedures in general OB/GYN residency training. Regarding obstetrics simulation, there are numerous simulators in the literature and it appears these simulations have been disseminated widely to obstetrics and gynecology residency programs.15 Documented improvements in patient care have been reported for postpartum hemorrhage and shoulder dystocia simulations.16,17 Performing and teaching forceps deliveries and breech extraction of a second twin were identified needs for additional training in a previous needs assessment of 100 first-year maternal fetal medicine fellows.18 In 2015, a survey of OB/GYN fellowship program directors showed that graduating OB/GYN residents may be underprepared for advanced subspecialty training, implying the possible need to re-evaluate the current structure of resident and fellow curriculum.5,19 Of first year fellows, it was reported by their fellowship
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
ARTICLE IN PRESS
TABLE 5. Perception of Trainee Preparedness for Performing Hysterectomies Independently PDs and Sim Faculty (N = 183) Variable Faculty reports residents can perform: TAH independently by: PGY1 PGY2 PGY3 PGY4 Requires additional training TLH independently by: PGY1 PGY2 PGY3 PGY4 Requires additional training TVH independently by: PGY1 PGY2 PGY3 PGY4 Requires additional training
No. (%) (N = 183)
2 (1.1%) 12 (6.6%) 72 (39.3%) 93 (50.8%) 4 (2.2%) 0 (0.0%) 3 (1.6%) 58 (31.7%) 116 (63.4%) 6 (3.3%) 0 (0.0%) 5 (2.7%) 29 (15.9%) 137 (74.9%) 12 (6.6%)
directors that only 18% could independently perform a total laparoscopic hysterectomy, 20% could independently perform a TVH, and 46% could independently perform a TAH. This assessment was not shared by trainees and faculty in our survey where more than 80% reported graduating residents could perform TVH, TAH, and TLH independently. Another 2019 study using a similar survey of current and former fellows demonstrated that 69% of gynecology oncology fellows reported they could independently perform an abdominal hysterectomy when entering fellowship, 70% of female pelvic reproductive medicine and surgery fellows reported they could independently perform a vaginal hysterectomy, and 70% of reproductive endocrinology fellows reported feeling capable of performing a laparoscopic hysterectomy.20 Clearly, the perceptions of trainee preparedness are subjective, and therefore, it is difficult to assess the true level of resident performance upon completing residency training. This reflects an important area for future research. Medical education (including our working group) must evolve to better assess the surgical training of OB/ GYNs using more objective validated assessment tools to evaluate performance of recent graduates in the simulation and operating room environment. Metacognitive monitoring (identifying “illusions of knowing”) from the cognitive psychology literature may provide a useful path forward in evaluating students’ cognitive activity and how this activity relates to knowledge and skills during surgical procedures.22 At this point, we do not know the level our trainees are performing since surgical
PGY4 (N = 36) Variable PGY4 reports they can perform: TAH independently 1 (strongly disagree) 2 3 4 5 (strongly agree) TLH independently 1 (strongly disagree) 2 3 4 5 (strongly agree) TVH independently 1 (strongly disagree) 2 3 4 5 (strongly agree)
No. (%) (N = 36)
0 (0.0%) 1 (2.8%) 0 (0.0%) 8 (22.2%) 27 (75.0%) (n=35, 1 missing) 0 (0.0%) 0 (0.0%) 2 (5.7%) 10 (28.6%) 23 (65.7%) 1 (2.8%) 4 (11.1%) 3 (8.3%) 12 (33.3%) 16 (44.5%)
learning is a lifelong pursuit with inadequate outcome measures. The high number of simulations reported as available also raises questions about how the simulation curriculum should be structured. When and how frequently should certain simulations be completed during a training program? More simulation may not always be better, since certain mistimed simulations could have unintentional consequences (e.g., overconfidence, perception that less simulation should be required). The disconnect between the perception of learning in active classrooms versus passive learning environments was recently described by Deslauriers et al.: “Attempts to evaluate instruction based on students’ perceptions of learning could inadvertently promote inferior (passive) pedagogical methods.”21 Although most programs had access to simulation resources at their institution, in general, faculty reported more awareness of these resources compared to trainees. This was consistent for both OB and GYN simulation. In addition, faculty and program directors were more likely than residents to report availability of online simulation resources such as the ACOG and CREOG simulation curriculum and tools. Among residents, only 27.8% reported awareness of the ACOG simulation toolkit and 47.2% were aware of the CREOG surgical curriculum, whereas, the majority of faculty, 72.1% and 84.7% were aware of the ACOG and CREOG resources respectively. The reasons for these differences in reporting the availability of simulation resources are unclear, but likely involve a combination of recall bias (residents not
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
7
ARTICLE IN PRESS
TABLE 6. Perception of Trainee Preparedness for Performing Procedures Independently Following Residency
Laparotomy Conventional laparoscopy Robotic surgery Hysteroscopy Vaginal surgery Loop electrosurgical excision procedure Cold knife conization Dilation and curettage Cystoscopy Laparoscopic salpingectomy Laparoscopic bilateral oophorectomy Total abdominal hysterectomy Total laparoscopic hysterectomy Total vaginal hysterectomy Abdominal myomectomy Laparoscopic myomectomy Hysteroscopic myomectomy Vaginal anterior/posterior repair Midurethral sling Labor cervical exam Cephalic vaginal delivery Breech vaginal delivery Operative vaginal delivery Postpartum hemorrhage Maternal cardiac arrest Shoulder dystocia Preeclampsia/eclampsia Routine cesarean section Emergency cesarean section Obstetric ultrasound Perineal laceration repair Amniocentesis Cervical cerclage
Resident N = 36
PDs N = 99
OB Faculty N = 60
GYN Faculty N = 91
36 (100.0) 36 (100.0) 19 (52.8) 36 (100.0) 28 (77.8) 36 (100.0) 31 (86.1) 36 (100.0) 35 (97.0) 36 (100.0) 36 (100.0) 35 (97.2) 33 (91.7) 29 (80.6) 33 (91.7) 15 (41.7) 34 (94.4) 24 (66.7) 16 (44.4) -
94 (94.9) 94 (94.9) 69 (69.7) 94 (94.9) 91 (91.9) 94 (94.9) 92 (92.9) 99 (100.0) 94 (94.9) 95 (96.0) 95 (96.0) 94 (94.9) 94 (94.9) 89 (89.9) 94 (94.9) 71(71.7) 92 (92.9) 85 (85.9) 74 (74.7) 94 (94.9) 95 (96.0) 75 (75.6) 93 (93.9) 95 (96.0) 85 (89.5) 95 (96.0) 95 (96.0) 95 (96.0) 95 (96.0) 94 (94.9) 95 (96.0) 70 (70.1) 88 (88.9)
58 (96.7) 57 (95.0) 43 (71.7) 57 (95.0) 59 (98.3) 53 (88.3) 58 (96.7) 56 (93.3) 59 (98.3) 59 (98.3) 54 (90.0) 56 (93.3) 37 (61.7) 52 (86.7)
84 (92.3) 84 (92.3) 53 (58.2) 85 (93.4) 78 (85.7) 88 (96.7) 88 (96.7) 88 (96.7) 88 (96.7) 88 (96.7) 85 (93.4) 83 (94.3) 82 (93.2) 85 (93.4) 51 (58.0) 81 (94.2) 78 (85.7) 70 (79.1) -
Proportions presented for residents are the proportions of resident respondents who selected “agree” or “strongly agree” when asked: “I feel confident performing the following procedures independently.” Proportions for faculty and PDs are the proportion who selected by the completion of PGY4 (graduation) when asked to “specify level of training when the average trainee is ready to perform the procedure independently.” Faculty only answered questions about resident preparedness for their primary specialty (OB or GYN). OB procedure preparedness was not included by the OB section of the simulation working group to limit survey fatigue for residents.
remembering certain simulations), selection bias (low response rate among residents), and a halo effect (faculty or residents’ perceptions of training being influenced by factors other than training).This highlights the need to being explicit about available training resources and for residents to be taught how to actively seek out these resources. It is also possible that the perceptions of simulation availability and needs would have been different if first, second, or third year residents had been surveyed, but we suspect that there would have been less reported knowledge of simulations as not all simulations are completed annually. We attempted to capture an overall picture of simulation by surveying “fourth-year residents knowledgeable of the program’s simulation curriculum.” A strength of this study is that participants were asked about a full range of obstetrics and gynecology procedures. The responses can be used as a guide for 8
residents, faculty, and program directors to inform curricular decisions about additional simulation and surgical training. Study limitations include those that exist for all cross-sectional surveys with low response rates. The response rate of less than 50%, and specifically resident response rate of 25.5%, may result in poor generalizability. Study coordinators did not provide all categories of representatives (e.g., faculty, program director, and residents) requested. This resulted in a different denominator between the categories which further decreased the generalizability of the study. Another potential limitation was the variability in how residents versus faculty were asked about the level of preparedness for performing hysterectomies independently. Residents were asked on a Likert scale while faculty members were asked the year in which the resident could perform the procedure independently. Comparisons between the responses
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
ARTICLE IN PRESS cannot be made. However, the overall conclusion was that residents and faculty perceived that graduating residents could perform procedures independently after residency. However, this requires further exploration with more objective measures in the future. Simulation is now widely available for conventional, straight-stick laparoscopic procedures and obstetric procedures. This likely reflects the current trend of surgery with an increasing pattern of minimally invasive laparoscopic surgery. However, there seems to be a need for additional vaginal and laparotomy simulation training. This likely reflects the current challenges faced by many OB/GYN residency programs trying to maintain volume in laparotomies and vaginal surgeries, highlighting the need for TAH and TVH simulation teaching. Moving forward, additional study is needed to determine the expectations of future employers and fellowship training programs regarding new technologies and surgical routes to align training experiences with workforce needs. For example, robotic surgery does not appear to be emphasized in OB/GYN resident simulation training based on the results of our survey despite increased robotic surgical practice on the national level.23 To establish a standard of didactic information, surgical knowledge, and manual skills in the performance of basic laparoscopic surgery, the American Board of Obstetrics and Gynecologists (ABOG) recently announced its decision to require Fundamentals of Laparoscopic Surgery (FLS) certification to be eligible for OB/GYN board certification.24 In addition, the ACGME residency review committee updated the minimum surgical numbers required by residents before graduation to emphasize minimally invasive hysterectomies. The implications of these decisions for gynecologic surgery training will be important to evaluate over time. The variations in simulation training and readiness for performing different procedures following residency support the need for objective, validated assessments of actual performance to better guide resident learning and faculty teaching efforts.
ACKNOWLEDGMENTS
and patient-related outcomes: a systematic review and meta-analysis. Acad Med. 2015;90:246–256. 2. ACOG simulations working group. 2017; https://
www.acog.org/About-ACOG/ACOG-Departments/ Simulations-Consortium. Accessed 10/30/2017. 3. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N,
Conde JG. Research electronic data capture (REDCap)a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42: 377–381. 4. Mandel LP, Lentz GM, Goff BA. Teaching and evalu-
ating surgical skills. Obstet Gynecol. 2000;95:783– 785. 5. Guntupalli SR, Doo DW, Guy M, et al. Preparedness
of obstetrics and gynecology residents for fellowship training. Obstet Gynecol. 2015;126:559–568. 6. Dubin AK, Smith R, Julian D, Tanaka A, Mattingly P.
A comparison of robotic simulation performance on basic virtual reality skills: simulator subjective versus objective assessment tools. J Minim Invasive Gynecol. 2017;24:1184–1189. 7. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin
BJ. Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol. 2012;187:247–252. 8. Havemann MC, Dalsgaard T, Sorensen JL, et al.
Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills. J Robot Surg. 2019;13:99–106. 9. Hovgaard LH, Andersen SAW, Konge L, Dalsgaard T,
Larsen CR. Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure. Surg Endosc. 2018;32:4200– 4208. 10. Patel HRH. Simulation training in laparoscopy and
robotic surgery. J Vis Surg. 2017;3:177.
ACOG Simulation Working Group Members (https:// www.acog.org/About-ACOG/ACOG-Departments/Simu lations-Consortium). Miriam J. Haviland, MSPH provided statistical analysis support for this project.
11. Rassweiler J. Robotic simulation: are we ready to go?
REFERENCES
13. Thomaier L, Orlando M, Abernethy M, Paka C, Chen
1. Brydges R, Hatala R, Zendejas B, Erwin PJ, Cook DA.
Linking simulation-based educational assessments
BJU Int. 2016;118:344–345. 12. Sheth SS, Fader AN, Tergas AI, Kushnir CL, Green
IC. Virtual reality robotic surgical simulation: an analysis of gynecology trainees. J Surg Educ. 2014;71:125–132. CCG. Laparoscopic and robotic skills are transferable in a simulation setting: a randomized controlled trial. Surg Endosc. 2017;31:3279–3285.
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
9
ARTICLE IN PRESS 14. Watkinson W, Raison N, Abe T, et al. Establishing
20. Urban RR, Ramzan AA, Doo DW, et al. Fellow per-
objective benchmarks in robotic virtual reality simulation at the level of a competent surgeon using the RobotiX Mentor simulator. Postgrad Med J. 2018;94:270–277.
ceptions of residency training in obstetrics and gynecology. J Surg Educ. 2019;76:93–98.
15. Gavin NR, Satin AJ. Simulation training in obstetrics.
Clin Obstet Gynecol. 2017;60:802–810. 16. Marshall NE, Vanderhoeven J, Eden KB, Segel SY,
Guise JM. Impact of simulation and team training on postpartum hemorrhage management in non-academic centers. J Matern Fetal Neonatal Med. 2015;28:495–499. 17. Shaddeau AK, Deering S. Simulation and shoulder
dystocia. Clin Obstet Gynecol. 2016;59(4):853–858. 18. Eichelberger KY, Bengtson AM, Tolleson-Rinehart S,
Menard MK. Training needs in operative obstetrics for maternal-fetal medicine fellows. J Matern Fetal Neonatal Med. 2015;28:1467–1470. 19. Doo DW, Powell M, Novetsky A, Sheeder J, Guntu-
palli SR. Preparedness of Ob/Gyn residents for fellowship training in gynecologic oncology. Gynecol Oncol Rep. 2015;12:55–60.
10
21. Deslauriers L, McCarty LS, Miller K, Callaghan K,
Kestin G. Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proc Natl Acad Sci U S A. 2019;116:19251–19257. 22. Avhustiuk MM, Pasichnyk ID, Kalamazh RV. The illu-
sion of knowing in metacognitive monitoring: effects of the type of information and of personal, cognitive, metacognitive, and individual psychological characteristics. Eur J Psychol. 2018;14:317–341. 23. Papalekas E, Fisher J. Trends in Route of Hysterec-
tomy after the Implementation of a Comprehensive Robotic Training Program. Minim Invasive Surg. 2018;2018:7362489. 24. American Board of Obstetrics and Gynecology.
ABOG announces new eligibility requirement for board certification. https://www.sages.org/abogannounces-fls-requirement-board-certification/. [Accessed 5/7/2019]. 2018.
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
ACOG Simulation Working Group: A Needs Assessment of Simulation Training in OB/GYN Residencies and Recommendations for Future Research Christopher C. DeStephano, MD, MPH,*,{ Joshua F. Nitsche, MD, PhD,†,{ Michael G. Heckman, MS,*,‡ Erika Banks, MD,x,{ and Hye-Chun Hur, MD, MPHǁ,{ Mayo Clinic Department of Surgical Gynecology, Jacksonville, Florida; †Wake Forest School of Medicine Department of OB/GYN, Winston-Salem, North Carolina; ‡Mayo Clinic Division of Biomedical Statistics and Informatics, Jacksonville, Florida; §Department of Obstetrics and Gynecology and Women’s Health, Albert Einstein College of Medicine, New York, New York; and ǁDivision of Gynecologic Specialty Surgery, Department of Obstetrics and Gynecology, New York Presbyterian Hospital, Columbia University Medical Center, New York, New York
*
OBJECTIVE: To evaluate current availability and needs of
simulation training among obstetrics/gynecology (OB/ GYN) residency programs. DESIGN: Cross-sectional survey. SETTING: Accreditation Council for Graduate Medical Education accredited OB/GYN residency programs in the United States. PARTICIPANTS: Residency program directors, gynecol-
ogy simulation faculty, obstetrics simulation faculty, and fourth-year residents. RESULTS: Of 673 invited participants, 251 (37.3%) com-
and Gynecology (47.2%) and American College of Obstetrics and Gynecology (27.8%) simulation tools compared to the majority of faculty (84.7% and 72.1%, respectively). More than 80% of trainees and faculty reported they felt the average graduating resident could perform vaginal, laparoscopic, and abdominal hysterectomies independently. CONCLUSIONS: Simulation is now widely available for
both gynecologic and obstetric procedures, but there remains tremendous heterogeneity between programs and the perceptions of residents, program directors, and faculty. The variations in simulation training and readiness for performing different procedures following residency support the need for objective, validated assessments of actual performance to better guide resident learning and faculty teaching efforts. ( J Surg Ed 000:110. Ó 2019 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
pleted the survey. Among the survey responses, OB procedures were more broadly represented compared to the GYN procedures for simulation teaching: 8 (50%) of 16 OB procedures versus 4 (18.2%) of 22 GYN procedures had simulation teaching. Among the simulated procedures, a majority of residents and faculty reported that simulation teaching was available for operative vaginal delivery, postpartum hemorrhage, shoulder dystocia, perineal laceration repair, conventional laparoscopic procedures, and robotic surgery. There were significant differences between residents and faculty perceptions regarding the availability and needs of simulated procedures with a minority of residents having knowledge of Council on Resident Education in Obstetrics
KEY WORDS: gynecology, needs assessment, obstetrics, surgical simulation, surgical performance
Funding source: None. Correspondence: Inquiries to Christopher DeStephano, MD Mayo Clinic, Department of Surgical Gynecology, 4500 San Pablo Road, Jacksonville, FL 32224; fax: (904)-953-0606; e-mail: [email protected]
Simulation teaching has become a part of the training curricula for many obstetrics/gynecology (OB/GYN) residency programs, as simulation has shown promise in teaching and evaluating the performance of trainees and attending physicians. A previous meta-analysis of
{ Members of the ACOG Simulation Working Group.
COMPETENCIES: Medical Knowledge, Practice-Based
Learning and Improvement, Systems-Based Practice
INTRODUCTION
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
1
ARTICLE IN PRESS simulation-based educational assessments demonstrated that these assessments correlate positively with patientrelated outcomes.1 The American College of Obstetrics and Gynecology (ACOG) Simulations Working Group and the Council on Resident Education in Obstetrics and Gynecology (CREOG) have developed and implemented simulationsbased curricula as an adjunct to improve residency education and clinical competence during and after residency.2 The objectives of the ACOG Simulations Working Group is to “develop standardized curricula available to all residency programs, develop standardized teaching and evaluation methods practiced and validated by the Working group to ensure efficacy of simulation education, and to provide validation of simulation-based education for developing and improving surgical, clinical, and behavioral skills.” In order to assess the extent to which simulation is being used at residency programs throughout the United States, and to better determine the state of simulation training in residency, we conducted a needs assessment survey. Our objectives were to evaluate the current state of simulation training resources available to OB/GYN residents and to assess current perceptions regarding when trainees are ready to perform procedures independently.
METHODS Study Design and IRB Approval This was a cross-sectional survey of Accreditation Council for Graduate Medical Education (ACGME)-accredited OB/GYN residency program directors, gynecology simulation faculty, obstetrics simulation faculty, and fourthyear residents. The study was determined to be exempt by institutional review boards at ACOG (IRB # 012) and Beth Israel Deaconess Medical Center (IRB# 2016P000348) under the Code of Federal Regulations, 45 CFR 46.102.
sent to all fourth year residents (i.e., multiple residents from certain institutions would have been more likely to respond at ACOG simulation working group sites). The survey was sent electronically via REDCap from March through June 2017.3 Survey Instrument A multi-item survey was developed and reviewed by representatives of the ACOG Simulations Working Group for accuracy and content validity. The simulation working group consists of 2 representatives from each of 24 ACGME accredited OB/GYN residency programs (i.e., 48 representatives). The first part of the survey assessed demographic information, simulation availability, and additional simulation needs. The second part of the survey evaluated respondents’ perception of the current residency training for common OB/GYN procedures identified from the CREOG 11th Edition of Educational Objectives. The survey questions were framed differently for faculty and residents. Residents were asked whether they “feel confident performing the following procedures independently” by selecting their level of agreement on a 5 level Likert scale ranging from strongly disagree to strongly agree. Faculty and program directors were asked to “specify level of training when the average trainee is ready to perform the procedure independently” by selecting either at the completion of PGY1, PGY2, PGY3, PGY4, or requires additional training after residency. Statistical Analysis The analysis is primarily descriptive, with continuous variables presented as median (interquartile range) and categorical variables presented as frequency (percentage). Categorical variables were compared between groups using a chi-square test or Fisher’s exact test. p values less than 0.05 were considered statistically significant and all tests were 2-sided. All statistical analysis was performed using SAS (version 9.4; SAS Institute, Inc., Cary, North Carolina).
Participant Invitations and Survey Distribution To identify eligible participants, an electronic e-mail form was sent from ACOG to all 256 ACGME-accredited residency program coordinators requesting contact information for the program director, an obstetric faculty member knowledgeable about the obstetric simulation curriculum, a gynecology faculty member knowledgeable about the gynecology simulation curriculum, and a fourth-year resident knowledgeable of the program’s simulation curriculum. The decision to only send the survey to 1 fourth-year resident was due to the deidentified nature of the survey and the possibility of over-representing certain programs if the survey was
2
RESULTS The survey response rate was as follows: 42.3% (99 of 234) program directors (PDs), 64.5% (91 of 141) gynecology simulation faculty (which may have also responded as obstetrics simulation faculty), 42.6% (60 of 141) obstetrics simulation faculty (which may have also responded as gynecology simulation faculty), and 25.5% (41 of 161) fourth-year residents. Although 41 fourthyear residents responded, 5 were incomplete responses, and thus, were excluded from the final analysis to maintain demographic consistency. Of 256 coordinators e-
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
TABLE 1. Demographics of Respondents
Age Gender* Female Male Program type University University-affiliated Community Military health system Total number of residents in the program
PGY4 Residents n = 36
Program Directors (PDs) n = 99
Simulation Faculty n = 116
30.5 (30.0-31.0)
47.0 (42.0-53.0)
43.0 (38.0-50.0)
31 (86.1) 5 (13.9)
54 (55.1) 44 (44.9)
85 (73.3) 31 (26.7)
13 (36.1) 12 (33.3) 11 (30.6) 0 (0.0) 16 (12-25)
54 (54.6) 18 (18.2) 26 (26.3) 1 (1.0) 20 (16-25)
70 (60.3) 26 (23.3) 19 (16.4) 0 (0.0) 23 (16-28)
*Information regarding sex was unavailable for 1 program director. Data presented as median (interquartile range) or n (%).
mailed, 234 (91.4%) responded, however, they did not provide all categories of representatives (e.g., faculty, program director, and residents) requested which resulted in different denominators for each type of representative (i.e., only 161 programs sent an email address for a fourth-year resident representative). Demographic characteristics of respondents are presented in Table 1. There were no significant differences between program director and simulation faculty responses for simulation needs and resident preparedness thus these 2 groups were combined to facilitate comparison of faculty to resident responses in the subsequent tables. The majority of OB/GYN respondents reported availability of simulation curricula and teaching resources for both obstetric and gynecologic procedures (Table 2A). When comparing responses from fourth-year residents and faculty (program directors and simulation faculty), faculty reported more availability of simulation teaching resources compared to residents, which was statistically significant for OB simulation teaching (p = 0.02), gynecology surgical curriculum (p = 0.008), and dedicated teaching time (p = 0.03; Table 2A). Reported availability of additional gynecologic simulation resources are presented in Table 2B. Similar to simulation teaching resources, faculty were more often aware of available online simulation resources such as the ACOG Simulation Toolkit (p < 0.0001) or CREOG surgical curriculum (p < 0.0001) than fourth-year residents. The availability of simulation training for specific gynecologic procedures is presented in Table 3. Both faculty and residents reported simulation teaching to be least available for vaginal surgery and laparotomy, and most available for conventional laparoscopic surgery and robotic surgery. Again, compared to faculty responses, residents consistently reported less availability of simulation teaching for all modes of surgery. The largest difference between faculty and residents was reported in the
Journal of Surgical Education Volume 00 /Number 00 Month 2019
availability of vaginal surgery simulation (8.3% residents versus 37.4% faculty reported vaginal simulation teaching, p = 0.0007). Faculty and trainees both identified total laparoscopic hysterectomy (TLH; 57.9% faculty, 47.2% residents) followed by LEEP (46.3% faculty, 33.3% residents) as the most common GYN procedures in simulation training. Availability and reported additional needs for obstetric simulation are displayed in Table 4. Additional TABLE 2A. Reported Availability of Simulation Curriculum and Teaching Resources Obstetrics
Procedures curriculum Simulation teaching Teaching staff Dedicated teaching time Gynecology Surgical curriculum Simulation teaching Teaching staff Dedicated teaching time
PGY4 Residents n (%)
PDs and Sim faculty n (%)
N = 36
n = 159
p value
22 (61.1)
104 (65.4)
0.83
29 (80.6)
150 (94.3)
0.02
23 (63.9) 23 (63.9)
100 (62.9) 124 (78.0)
0.66 0.10
N = 36 18 (50.0)
N = 190 134 (70.5)
0.008
30 (83.3)
174 (91.6)
0.22
17 (47.2) 24 (66.7)
116 (61.1) 158 (83.2)
0.36 0.03
Sim faculty, simulation faculty. p values result from a chi-square test or Fisher’s exact test. Program directors and simulation faculty were combined as responses were similar between these 2 groups. Response rates for faculty and PDs were different due to different practice patterns of OB/GYN faculty (e.g., some obstetric faculty and PDs no longer practice gynecology and vice versa).
3
ARTICLE IN PRESS
TABLE 2B. Additional Gynecology Simulation Resources Available Resident (n = 36) n (%)
PDs and GYN Sim Faculty (n = 190) n (%)
p value
31 (86.1) 24 (66.7) 31 (86.1) 5 (13.9) 29 (80.6) 30 (83.3) 17 (47.2) 7 (19.4) 9 (25.0) 6 (16.7) 1 (2.8) 17 (47.2) 7 (19.4) 10 (27.8) 3 (8.3)
156 (82.1) 136 (71.6) 168 (88.4) 72 (37.9) 167 (87.9) 142 (74.7) 124 (65.3) 74 (39.0) 83 (43.7) 51 (26.8) 25 (13.2) 161 (84.7) 78 (41.1) 137 (72.1) 60 (31.6)
0.46 0.35 0.87 0.01 0.02 0.33 0.07 0.05 0.07 0.26 0.02 <0.0001 0.40 <0.0001 0.74
Simulation lab access Trainees go to simulation lab Simulation tools available Laparotomy simulation Laparoscopic simulation Robotic simulation Hysteroscopic simulation Vaginal simulation Minimally invasive gynecologic surgery division Fundamentals of Laparoscopic Surgery (FLS) offered Fundamentals of Laparoscopic Surgery (FLS) required Aware of CREOG Surgical Curriculum Use CREOG Surgical Curriculum Aware of ACOG simulation toolkit Use ACOG simulation toolkit
GYN Sim faculty, gynecology simulation faculty. p values result from a chi-square test or Fisher’s exact test. Program directors and GYN simulation faculty were combined as responses were similar between these 2 groups.
simulation teaching for cesarean hysterectomy was the most commonly reported obstetric need (54.1% faculty, 50% of residents, p = 0.66). Among specific OB procedures, shoulder dystocia (89.3% faculty, 72.2% residents, p = 0.01) followed by postpartum hemorrhage (88.1% faculty, 69.4% residents, p = 0.005) were the most commonly reported OB simulation procedures. Faculty generally reported a higher level of availability of simulated obstetric procedures than residents reported. Table 5 presents the perception of trainee preparedness for performing hysterectomies independently. Overall, very few faculty perceived that residents required additional training for total abdominal hysterectomy (TAH 2.2%), TLH (3.3%), and total vaginal hysterectomy (TVH 6.6%). Although the question was asked on a Likert scale of agreement for PGY responses, this trend of perceiving that a procedure could be performed independently was similar for the residents who responded. Of 41 PGY4 residents, 40 (97.6%) stated agreement or strong agreement with the statement that he or she could perform a TAH independently, 37 (90.2%) for TLH, and 80.5% for TVH. Table 6 summarizes a wide range of trainee preparedness for performing procedures independently following residency. PGY 4 responses were based on stating “agree” or “strongly agree” on the statement they could perform the procedure independently while faculty responses were based on the proportion who selected by the completion of PGY4. In general, >80% faculty and PDs reported that residents were prepared to perform the procedures independently except for the following: amniocentesis (PDs 70.1%, 61.7% OB sim
4
faculty), breech vaginal delivery (75.6% PDs, 71.7% OB sim faculty), midurethral sling (74.7% PDs, 79.1% GYN sim faculty), laparoscopic myomectomy (71.7% PDs, 58.0% GYN sim faculty), and robotic surgery (69.7% PDs, 58.2% GYN sim faculty). To limit survey response fatigue, the OB section of the working group did not ask about resident preparedness to complete OB procedures. Less than 80% of PGY4s agreed with the statement that they could perform the following procedures independently: midurethral sling (44.4%), vaginal anterior/posterior repair (66.7%), laparoscopic myomectomy (41.7%), vaginal surgery (77.8%), and robotic surgery (52.8%).
DISCUSSION Responses from this survey suggest the majority of ACGME-accredited OB/GYN residency programs in the United States have simulation resources available to their trainees. Outside of vaginal surgery and laparotomy, less than 50% of respondents reported the need for additional simulation teaching of specific procedures. The majority of respondents reported that graduating residents could perform TLH, TVH, and TAH independently in practice. Our results suggest there has been an increase in the percentage of programs reporting a formal didactic surgical curriculum. In a needs assessment of program directors conducted by Mandel et al. in 2001, only 29% of program directors reported that their programs had formal surgical curricula.4 In our survey, 70.5% of
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
TABLE 3. Availability and Additional Needs for Simulated Gynecologic Procedures Available PGY4 Residents N = 36 Mode of incision Laparotomy Conventional laparoscopic surgery Robotic surgery Hysteroscopy Vaginal surgery Specific GYN procedures Loop electrosurgical excision procedure Cold knife conization Dilation and curettage Dilation and evacuation Cystoscopy Total abdominal hysterectomy Total laparoscopic hysterectomy Total vaginal hysterectomy Abdominal myomectomy Laparoscopic myomectomy Hysteroscopic myomectomy Laparoscopic salpingectomy Laparoscopic bilateral salpingooophorectomy Vaginal anterior repair Vaginal posterior repair Midurethral sling Surgical complications
Additional Needs
PDs and GYN Sim Faculty N = 190
p Value
PGY4 Residents N = 36
PDs and GYN Sim Faculty N = 190
p Value
5 (13.9) 23 (63.9)
78 (41.1) 153 (80.5)
0.002 0.03
15 (41.7) 13 (36.1)
113 (59.5) 52 (27.4)
0.048 0.29
19 (52.8) 17 (47.2) 3 (8.3)
120 (63.2) 114 (60.0) 71 (37.4)
0.24 0.15 0.0007
5 (13.9) 6 (16.7) 22 (61.1)
41 (21.6) 51 (26.8) 127 (66.8)
0.29 0.20 0.51
12 (33.3)
88 (46.3)
0.15
4 (11.1)
40 (21.1)
0.17
2 (5.6)
38 (20.0)
0.04
3 (8.3)
47 (24.7)
0.03
5 (13.9)
53 (27.9)
0.08
2 (5.6)
28 (14.7)
0.18
2 (5.6)
28 (14.7)
0.18
3 (8.3)
52 (27.4)
0.01
4 (11.1) 5 (13.9)
63 (33.2) 49 (25.8)
0.008 0.12
3 (8.3) 9 (25.0)
43 (22.6) 78 (41.1)
0.05 0.07
17 (47.2)
110 (57.9)
0.24
6 (16.7)
56 (29.5)
0.11
15 (41.7)
99 (52.1)
0.25
7 (19.4)
71 (37.4)
0.04
3 (8.3)
75 (39.5)
0.0003
0 (0.0)
4 (2.1)
2 (5.6)
24 (12.6)
0.39
11 (30.6)
98 (51.6)
0.02
8 (22.2)
45 (23.7)
0.85
4 (11.1)
60 (31.6)
0.01
9 (25.0)
69 (36.3)
0.19
2 (5.6)
41 (21.6)
0.02
9 (25.0)
51 (26.8)
0.82
2 (5.6)
43 (22.6)
0.02
0 (0.0)
13 (6.8)
0.23
11 (30.6)
105 (55.3)
0.007
0 (0.0)
13 (6.8)
0.23
12 (33.3)
105 (55.3)
0.02
6 (16.7) 3 (8.3)
45 (23.7) 24 (12.6)
0.36 0.58
15 (41.7) 11 (30.6)
86 (45.3) 86 (45.3)
0.69 0.10
1.0
GYN sim faculty, gynecology simulation faculty. p values result from a chi-square test or Fisher’s exact test. Program directors and GYN simulation faculty were combined as responses were similar between these 2 groups.
program directors and simulation faculty reported their programs had surgical curricula. Mandel and colleagues also found that 64% of program directors reported that laparoscopic hysterectomy was an essential skill for third and fourth-year residents to learn. In contrast, in our survey, 94 (94.9%) of 99 program directors reported that the average graduating resident could complete a TLH
independently at the completion of residency. Compared to laparoscopic simulation training, there was less availability and more perceived need for abdominal and vaginal procedure simulation in our survey; vaginal surgery, laparotomy, and cesarean hysterectomy were most often reported to be desired for additional simulation training. These results are consistent with the
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
5
ARTICLE IN PRESS
TABLE 4. Availability and Additional Needs for Simulated Obstetric Procedures Available PGY4 Residents N = 36 Labor cervical exam Cephalic vaginal delivery Breech vaginal delivery Operative vaginal delivery Postpartum hemorrhage Maternal cardiac arrest Shoulder dystocia Preeclampsia/ eclampsia Routine cesarean section Emergency cesarean section Obstetrical ultrasound Perineal laceration repair Amniocentesis Cervical cerclage Cesarean hysterectomy External cephalic version
Additional Needs
PDs and OB Sim Faculty n = 159
11 (30.6)
86 (54.1)
15 (41.7)
p Value
PGY4 Residents N = 36
0.01
4 (11.1)
115 (72.3)
0.004
11 (30.6)
106 (66.7)
<0.0001
19 (52.8)
126 (79.3)
25 (69.4)
PDs and OB Sim Faculty n = 159
p Value
15 (9.4)
0.76
2 (5.6)
7 (4.4)
0.67
13 (36.1)
28 (17.6)
0.02
0.001
8 (22.2)
31 (19.5)
0.71
140 (88.1)
0.005
1 (2.8)
14 (8.8)
0.31
12 (33.3)
82 (51.6)
0.048
14 (38.9)
46 (28.9)
0.24
26 (72.2) 17 (47.2)
142 (89.3) 98 (61.6)
0.01 0.11
2 (5.6) 5 (13.9)
14 (8.8) 23 (14.5)
0.74 0.93
12 (33.3)
58 (36.5)
0.72
3 (8.3)
32 (20.1)
0.10
8 (22.2)
39 (24.5)
0.77
12 (33.3)
41 (25.8)
0.36
6 (16.7)
43 (27.0)
0.19
8 (22.2)
54 (34.0)
0.17
19 (52.8)
118 (74.2)
0.01
5 (13.9)
20 (12.6)
0.79
4 (11.1) 0 (0.0) 2 (5.6)
22 (13.8) 13 (8.2) 11 (6.9)
0.79 0.13 1.0
17 (47.2) 11 (30.6) 18 (50.0)
66 (41.5) 64 (40.3) 86 (54.1)
0.53 0.28 0.66
1 (2.8)
14 (8.8)
0.31
12 (33.3)
79 (49.7)
0.08
OB sim faculty, obstetrics simulation faculty. p values result from a chi-square test or Fisher’s exact test. Program directors and OB simulation faculty were combined as responses were similar between these 2 groups.
observation that the number of open abdominal and vaginal surgeries are decreasing in residency programs.5 Opportunities for robotic simulation training are also rapidly expanding with numerous studies establishing validity evidence.6-14 Robotic simulation resources were reported as available by 83.3% of residents and 74.7% of PDs and GYN simulation faculty in Table 2B. However, as reported in Table 3, when asked if the respondent’s program “provides simulation training for robotic surgery” 52.8% of residents and 63.2% of PDs and GYN simulation faculty responded in the affirmative and only 13.9% of residents and 21.6% of PDs and GYN simulation faculty reported that there was an additional need for robotic simulation training. Perceived preparedness of graduating residents to perform robotic surgery independently was reported by 52.8% of residents, 69.7% of PDs, and 58.2% of GYN simulation faculty. These results suggest that robotic surgery simulation is less of an
6
emphasis than other procedures in general OB/GYN residency training. Regarding obstetrics simulation, there are numerous simulators in the literature and it appears these simulations have been disseminated widely to obstetrics and gynecology residency programs.15 Documented improvements in patient care have been reported for postpartum hemorrhage and shoulder dystocia simulations.16,17 Performing and teaching forceps deliveries and breech extraction of a second twin were identified needs for additional training in a previous needs assessment of 100 first-year maternal fetal medicine fellows.18 In 2015, a survey of OB/GYN fellowship program directors showed that graduating OB/GYN residents may be underprepared for advanced subspecialty training, implying the possible need to re-evaluate the current structure of resident and fellow curriculum.5,19 Of first year fellows, it was reported by their fellowship
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
ARTICLE IN PRESS
TABLE 5. Perception of Trainee Preparedness for Performing Hysterectomies Independently PDs and Sim Faculty (N = 183) Variable Faculty reports residents can perform: TAH independently by: PGY1 PGY2 PGY3 PGY4 Requires additional training TLH independently by: PGY1 PGY2 PGY3 PGY4 Requires additional training TVH independently by: PGY1 PGY2 PGY3 PGY4 Requires additional training
No. (%) (N = 183)
2 (1.1%) 12 (6.6%) 72 (39.3%) 93 (50.8%) 4 (2.2%) 0 (0.0%) 3 (1.6%) 58 (31.7%) 116 (63.4%) 6 (3.3%) 0 (0.0%) 5 (2.7%) 29 (15.9%) 137 (74.9%) 12 (6.6%)
directors that only 18% could independently perform a total laparoscopic hysterectomy, 20% could independently perform a TVH, and 46% could independently perform a TAH. This assessment was not shared by trainees and faculty in our survey where more than 80% reported graduating residents could perform TVH, TAH, and TLH independently. Another 2019 study using a similar survey of current and former fellows demonstrated that 69% of gynecology oncology fellows reported they could independently perform an abdominal hysterectomy when entering fellowship, 70% of female pelvic reproductive medicine and surgery fellows reported they could independently perform a vaginal hysterectomy, and 70% of reproductive endocrinology fellows reported feeling capable of performing a laparoscopic hysterectomy.20 Clearly, the perceptions of trainee preparedness are subjective, and therefore, it is difficult to assess the true level of resident performance upon completing residency training. This reflects an important area for future research. Medical education (including our working group) must evolve to better assess the surgical training of OB/ GYNs using more objective validated assessment tools to evaluate performance of recent graduates in the simulation and operating room environment. Metacognitive monitoring (identifying “illusions of knowing”) from the cognitive psychology literature may provide a useful path forward in evaluating students’ cognitive activity and how this activity relates to knowledge and skills during surgical procedures.22 At this point, we do not know the level our trainees are performing since surgical
PGY4 (N = 36) Variable PGY4 reports they can perform: TAH independently 1 (strongly disagree) 2 3 4 5 (strongly agree) TLH independently 1 (strongly disagree) 2 3 4 5 (strongly agree) TVH independently 1 (strongly disagree) 2 3 4 5 (strongly agree)
No. (%) (N = 36)
0 (0.0%) 1 (2.8%) 0 (0.0%) 8 (22.2%) 27 (75.0%) (n=35, 1 missing) 0 (0.0%) 0 (0.0%) 2 (5.7%) 10 (28.6%) 23 (65.7%) 1 (2.8%) 4 (11.1%) 3 (8.3%) 12 (33.3%) 16 (44.5%)
learning is a lifelong pursuit with inadequate outcome measures. The high number of simulations reported as available also raises questions about how the simulation curriculum should be structured. When and how frequently should certain simulations be completed during a training program? More simulation may not always be better, since certain mistimed simulations could have unintentional consequences (e.g., overconfidence, perception that less simulation should be required). The disconnect between the perception of learning in active classrooms versus passive learning environments was recently described by Deslauriers et al.: “Attempts to evaluate instruction based on students’ perceptions of learning could inadvertently promote inferior (passive) pedagogical methods.”21 Although most programs had access to simulation resources at their institution, in general, faculty reported more awareness of these resources compared to trainees. This was consistent for both OB and GYN simulation. In addition, faculty and program directors were more likely than residents to report availability of online simulation resources such as the ACOG and CREOG simulation curriculum and tools. Among residents, only 27.8% reported awareness of the ACOG simulation toolkit and 47.2% were aware of the CREOG surgical curriculum, whereas, the majority of faculty, 72.1% and 84.7% were aware of the ACOG and CREOG resources respectively. The reasons for these differences in reporting the availability of simulation resources are unclear, but likely involve a combination of recall bias (residents not
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
7
ARTICLE IN PRESS
TABLE 6. Perception of Trainee Preparedness for Performing Procedures Independently Following Residency
Laparotomy Conventional laparoscopy Robotic surgery Hysteroscopy Vaginal surgery Loop electrosurgical excision procedure Cold knife conization Dilation and curettage Cystoscopy Laparoscopic salpingectomy Laparoscopic bilateral oophorectomy Total abdominal hysterectomy Total laparoscopic hysterectomy Total vaginal hysterectomy Abdominal myomectomy Laparoscopic myomectomy Hysteroscopic myomectomy Vaginal anterior/posterior repair Midurethral sling Labor cervical exam Cephalic vaginal delivery Breech vaginal delivery Operative vaginal delivery Postpartum hemorrhage Maternal cardiac arrest Shoulder dystocia Preeclampsia/eclampsia Routine cesarean section Emergency cesarean section Obstetric ultrasound Perineal laceration repair Amniocentesis Cervical cerclage
Resident N = 36
PDs N = 99
OB Faculty N = 60
GYN Faculty N = 91
36 (100.0) 36 (100.0) 19 (52.8) 36 (100.0) 28 (77.8) 36 (100.0) 31 (86.1) 36 (100.0) 35 (97.0) 36 (100.0) 36 (100.0) 35 (97.2) 33 (91.7) 29 (80.6) 33 (91.7) 15 (41.7) 34 (94.4) 24 (66.7) 16 (44.4) -
94 (94.9) 94 (94.9) 69 (69.7) 94 (94.9) 91 (91.9) 94 (94.9) 92 (92.9) 99 (100.0) 94 (94.9) 95 (96.0) 95 (96.0) 94 (94.9) 94 (94.9) 89 (89.9) 94 (94.9) 71(71.7) 92 (92.9) 85 (85.9) 74 (74.7) 94 (94.9) 95 (96.0) 75 (75.6) 93 (93.9) 95 (96.0) 85 (89.5) 95 (96.0) 95 (96.0) 95 (96.0) 95 (96.0) 94 (94.9) 95 (96.0) 70 (70.1) 88 (88.9)
58 (96.7) 57 (95.0) 43 (71.7) 57 (95.0) 59 (98.3) 53 (88.3) 58 (96.7) 56 (93.3) 59 (98.3) 59 (98.3) 54 (90.0) 56 (93.3) 37 (61.7) 52 (86.7)
84 (92.3) 84 (92.3) 53 (58.2) 85 (93.4) 78 (85.7) 88 (96.7) 88 (96.7) 88 (96.7) 88 (96.7) 88 (96.7) 85 (93.4) 83 (94.3) 82 (93.2) 85 (93.4) 51 (58.0) 81 (94.2) 78 (85.7) 70 (79.1) -
Proportions presented for residents are the proportions of resident respondents who selected “agree” or “strongly agree” when asked: “I feel confident performing the following procedures independently.” Proportions for faculty and PDs are the proportion who selected by the completion of PGY4 (graduation) when asked to “specify level of training when the average trainee is ready to perform the procedure independently.” Faculty only answered questions about resident preparedness for their primary specialty (OB or GYN). OB procedure preparedness was not included by the OB section of the simulation working group to limit survey fatigue for residents.
remembering certain simulations), selection bias (low response rate among residents), and a halo effect (faculty or residents’ perceptions of training being influenced by factors other than training).This highlights the need to being explicit about available training resources and for residents to be taught how to actively seek out these resources. It is also possible that the perceptions of simulation availability and needs would have been different if first, second, or third year residents had been surveyed, but we suspect that there would have been less reported knowledge of simulations as not all simulations are completed annually. We attempted to capture an overall picture of simulation by surveying “fourth-year residents knowledgeable of the program’s simulation curriculum.” A strength of this study is that participants were asked about a full range of obstetrics and gynecology procedures. The responses can be used as a guide for 8
residents, faculty, and program directors to inform curricular decisions about additional simulation and surgical training. Study limitations include those that exist for all cross-sectional surveys with low response rates. The response rate of less than 50%, and specifically resident response rate of 25.5%, may result in poor generalizability. Study coordinators did not provide all categories of representatives (e.g., faculty, program director, and residents) requested. This resulted in a different denominator between the categories which further decreased the generalizability of the study. Another potential limitation was the variability in how residents versus faculty were asked about the level of preparedness for performing hysterectomies independently. Residents were asked on a Likert scale while faculty members were asked the year in which the resident could perform the procedure independently. Comparisons between the responses
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
ARTICLE IN PRESS cannot be made. However, the overall conclusion was that residents and faculty perceived that graduating residents could perform procedures independently after residency. However, this requires further exploration with more objective measures in the future. Simulation is now widely available for conventional, straight-stick laparoscopic procedures and obstetric procedures. This likely reflects the current trend of surgery with an increasing pattern of minimally invasive laparoscopic surgery. However, there seems to be a need for additional vaginal and laparotomy simulation training. This likely reflects the current challenges faced by many OB/GYN residency programs trying to maintain volume in laparotomies and vaginal surgeries, highlighting the need for TAH and TVH simulation teaching. Moving forward, additional study is needed to determine the expectations of future employers and fellowship training programs regarding new technologies and surgical routes to align training experiences with workforce needs. For example, robotic surgery does not appear to be emphasized in OB/GYN resident simulation training based on the results of our survey despite increased robotic surgical practice on the national level.23 To establish a standard of didactic information, surgical knowledge, and manual skills in the performance of basic laparoscopic surgery, the American Board of Obstetrics and Gynecologists (ABOG) recently announced its decision to require Fundamentals of Laparoscopic Surgery (FLS) certification to be eligible for OB/GYN board certification.24 In addition, the ACGME residency review committee updated the minimum surgical numbers required by residents before graduation to emphasize minimally invasive hysterectomies. The implications of these decisions for gynecologic surgery training will be important to evaluate over time. The variations in simulation training and readiness for performing different procedures following residency support the need for objective, validated assessments of actual performance to better guide resident learning and faculty teaching efforts.
ACKNOWLEDGMENTS
and patient-related outcomes: a systematic review and meta-analysis. Acad Med. 2015;90:246–256. 2. ACOG simulations working group. 2017; https://
www.acog.org/About-ACOG/ACOG-Departments/ Simulations-Consortium. Accessed 10/30/2017. 3. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N,
Conde JG. Research electronic data capture (REDCap)a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42: 377–381. 4. Mandel LP, Lentz GM, Goff BA. Teaching and evalu-
ating surgical skills. Obstet Gynecol. 2000;95:783– 785. 5. Guntupalli SR, Doo DW, Guy M, et al. Preparedness
of obstetrics and gynecology residents for fellowship training. Obstet Gynecol. 2015;126:559–568. 6. Dubin AK, Smith R, Julian D, Tanaka A, Mattingly P.
A comparison of robotic simulation performance on basic virtual reality skills: simulator subjective versus objective assessment tools. J Minim Invasive Gynecol. 2017;24:1184–1189. 7. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin
BJ. Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol. 2012;187:247–252. 8. Havemann MC, Dalsgaard T, Sorensen JL, et al.
Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills. J Robot Surg. 2019;13:99–106. 9. Hovgaard LH, Andersen SAW, Konge L, Dalsgaard T,
Larsen CR. Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure. Surg Endosc. 2018;32:4200– 4208. 10. Patel HRH. Simulation training in laparoscopy and
robotic surgery. J Vis Surg. 2017;3:177.
ACOG Simulation Working Group Members (https:// www.acog.org/About-ACOG/ACOG-Departments/Simu lations-Consortium). Miriam J. Haviland, MSPH provided statistical analysis support for this project.
11. Rassweiler J. Robotic simulation: are we ready to go?
REFERENCES
13. Thomaier L, Orlando M, Abernethy M, Paka C, Chen
1. Brydges R, Hatala R, Zendejas B, Erwin PJ, Cook DA.
Linking simulation-based educational assessments
BJU Int. 2016;118:344–345. 12. Sheth SS, Fader AN, Tergas AI, Kushnir CL, Green
IC. Virtual reality robotic surgical simulation: an analysis of gynecology trainees. J Surg Educ. 2014;71:125–132. CCG. Laparoscopic and robotic skills are transferable in a simulation setting: a randomized controlled trial. Surg Endosc. 2017;31:3279–3285.
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002
9
ARTICLE IN PRESS 14. Watkinson W, Raison N, Abe T, et al. Establishing
20. Urban RR, Ramzan AA, Doo DW, et al. Fellow per-
objective benchmarks in robotic virtual reality simulation at the level of a competent surgeon using the RobotiX Mentor simulator. Postgrad Med J. 2018;94:270–277.
ceptions of residency training in obstetrics and gynecology. J Surg Educ. 2019;76:93–98.
15. Gavin NR, Satin AJ. Simulation training in obstetrics.
Clin Obstet Gynecol. 2017;60:802–810. 16. Marshall NE, Vanderhoeven J, Eden KB, Segel SY,
Guise JM. Impact of simulation and team training on postpartum hemorrhage management in non-academic centers. J Matern Fetal Neonatal Med. 2015;28:495–499. 17. Shaddeau AK, Deering S. Simulation and shoulder
dystocia. Clin Obstet Gynecol. 2016;59(4):853–858. 18. Eichelberger KY, Bengtson AM, Tolleson-Rinehart S,
Menard MK. Training needs in operative obstetrics for maternal-fetal medicine fellows. J Matern Fetal Neonatal Med. 2015;28:1467–1470. 19. Doo DW, Powell M, Novetsky A, Sheeder J, Guntu-
palli SR. Preparedness of Ob/Gyn residents for fellowship training in gynecologic oncology. Gynecol Oncol Rep. 2015;12:55–60.
10
21. Deslauriers L, McCarty LS, Miller K, Callaghan K,
Kestin G. Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proc Natl Acad Sci U S A. 2019;116:19251–19257. 22. Avhustiuk MM, Pasichnyk ID, Kalamazh RV. The illu-
sion of knowing in metacognitive monitoring: effects of the type of information and of personal, cognitive, metacognitive, and individual psychological characteristics. Eur J Psychol. 2018;14:317–341. 23. Papalekas E, Fisher J. Trends in Route of Hysterec-
tomy after the Implementation of a Comprehensive Robotic Training Program. Minim Invasive Surg. 2018;2018:7362489. 24. American Board of Obstetrics and Gynecology.
ABOG announces new eligibility requirement for board certification. https://www.sages.org/abogannounces-fls-requirement-board-certification/. [Accessed 5/7/2019]. 2018.
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.002