- Email: [email protected]
Obstetric team simulation program challenges
Journal of Clinical Anesthesia (2016) 35, 564–570
Original contribution
Obstetric team simulation program challenges A.S. Bullough MD (Associate Professor)a,e,⁎, S. Wagner MD (Associate Professor)b,e , T. Boland MS, RNC (Clinical Nurse Specialist Women's Health)c,e , T.P. Waters MD (Associate Professor)b,e , K. Kim MD, PhD (Professor)a,e , W. Adams MA (Biostatistician)d,e a
Department of Anesthesiology Department of Obstetrics and Gynecology c Obstetric and Gynecology Nursing Department d Clinical Research Office e Loyola University Health System, Loyola University Stritch School of Medicine b
Received 28 September 2015; revised 16 August 2016; accepted 17 August 2016
Keywords: Team simulation challenges; Obstetric emergency team simulation
Abstract Objective: To describe the challenges associated with the development and assessment of an obstetric emergency team simulation program. Design: The goal was to develop a hybrid, in-situ and high fidelity obstetric emergency team simulation program that incorporated weekly simulation sessions on the labor and delivery unit, and quarterly, education protected sessions in the simulation center. All simulation sessions were video-recorded and reviewed. Setting: Labor and delivery unit and simulation center. Participants: Medical staff covering labor and delivery, anesthesiology and obstetric residents and obstetric nurses. Measurements: Assessments included an on-line knowledge multiple-choice questionnaire about the simulation scenarios. This was completed prior to the initial in-situ simulation session and repeated 3 months later, the Clinical Teamwork Scale with inter-rater reliability, participant confidence surveys and subjective participant satisfaction. A web-based curriculum comprising modules on communication skills, team challenges, and team obstetric emergency scenarios was also developed. Main results: Over 4 months, only 6 labor and delivery unit in-situ sessions out of a possible 14 sessions were carried out. Four high-fidelity sessions were performed in 2 quarterly education protected meetings in the simulation center. Information technology difficulties led to the completion of only 18 pre/post web-based multiple-choice questionnaires. These test results showed no significant improvement in raw score performance from pre-test to post-test (P = .27). During Clinical Teamwork Scale live and video assessment, trained raters and program faculty were in agreement only 31% and 28% of the time, respectively (Kendall's W = .31, P b .001 and W = .28, P b .001). Participant confidence surveys overall revealed confidence significantly increased (P b .05), from pre-scenario briefing to after post-scenario debriefing. Conclusion: Program feedback indicates a high level of participant satisfaction and improved confidence yet further program refinement is required. © 2016 Elsevier Inc. All rights reserved.
⁎ Corresponding author at: Loyola University Health System, 2160 S. First Avenue, Maywood, IL 60153. Tel.: +1 708 216 8866. E-mail address: [email protected] (A.S. Bullough). http://dx.doi.org/10.1016/j.jclinane.2016.08.019 0952-8180/© 2016 Elsevier Inc. All rights reserved.
Teamwork simulation challenges
1. Introduction The American College of Obstetricians and Gynecologists (ACOG) Committee Opinion (No 590) on Preparing for Clinical Emergencies in Obstetrics advocates improvement in response systems for in-patient emergency care [1]. Two ways in which to address this goal include standardization of obstetric emergency management protocols and team simulation training. The initial management and therapeutic interventions are critical when considering patient safety and a positive outcome in an obstetric emergency. Simulation training is able to identify mistakes in management during an obstetric crisis and is endorsed as a use of formal training by the Institute of Medicine, Joint Commission on Accreditation of Healthcare Organizations and ACOG to improve clinical skills, interdisciplinary teamwork and reduce communication-related errors [2-5]. At our tertiary care referral institution, the Obstetric department primarily deals with under-served, high-risk patients with multiple medical issues that can lead to a potential crisis. In the past 2 years, the labor and delivery unit (LDU) has seen an increase in the high-risk parturients. Because of our acuity and in response to the call for increased preparedness, we developed a hybrid, in-situ and high-fidelity obstetric emergency team simulation program comprising OBstetric, ANesthesiology and Nursing professionals (OBANN) to improve emergency clinical management on the LDU. In what follows we present a general description of the program, the team simulation challenges (Fig. 1) and lessons learned from our endeavor.
2. Materials and methods Following institutional review board exemption, we implemented the OBANN program in three phases. Phase 1, the development phase from February 2014 to June 2014, involved scenario and web-based curriculum development as well as determination of assessment and learning instruments. Phase 2 from July 2014 to November 2014 tested the assessment instruments and training for those who were rating the
565 participants in the scenarios. Phase 3 from December 2014 to March 2015 collected pilot data.
2.1. Phase 1 development In Phase 1, an inter-professional group comprising an obstetric clinical nurse specialist, a nominated representative from the maternal fetal medicine group, the obstetric residency director, and two representatives from obstetric anesthesiology met on a monthly basis over a 5-month period to set objectives, create emergency simulation scenarios and develop a curriculum. The program objectives were (1) identification of the roles and responsibilities of the obstetric team members, (2) implementation of clear communication strategies, and (3) creation of an environment that promotes inter-professional mutual support during an obstetric crisis. Four obstetric emergency scenarios with specific clinical objectives were constructed. The cases included eclampsia, shoulder dystocia with basic neonatal resuscitation, cord prolapse with basic neonatal resuscitation, and maternal cardiac arrest. A Web-based tutorial composed of 5 different modules was also created. These modules comprised a program overview, team challenges, team communication skills, team interaction and applied teamwork regarding the simulation scenarios. We determined that all participants working continuously (obstetric nurses) or on rotational assignment (obstetric and anesthetic residents) to the LDU would receive an e-mail invitation to access the OBANN web site. Initiating the program LDU participants were required to complete a 20-question multiple-choice questionnaire (MCQ) based on the simulation scenarios and the on-line web-site team communication modules. The same test was repeated 3 months later after completion of the quarterly high fidelity session. The validated Clinical Teamwork Scale (CTS) Global was determined the best tool to assess scenario non-technical skills such as teamwork and communication. [6] The CTS tool measures 15 points on a 1-10 (unacceptable to perfect) Likert scale. The major CTS assessment headings included overall teamwork, communication, situational awareness, decision making, role responsibility and patient friendliness (in one scenario the patient is awake then has a seizure).
2.2. Phase 2 implementation Developing a simulation team of front-line care leaders Organizational and departmental support Determining objectives and safety aims Measuring training effectiveness Availability of information technology resources Implementing data driven change Creating a non-threatening learning environment Time and personnel limitations for in-situ sessions Fig. 1
Obstetric team simulation program challenges.
During phase 2 two volunteer obstetric nurses became CTS evaluation raters. Rater training involved reviewing the case content and objectives, practice rating, and discussion and revision of the CTS evaluation tool. A faculty member of the designing program team with simulation assessment experience reviewed the live and video practice rating session scoring and clarified any unclear items or missing items during the training period. We also recorded participant self-confidence before and after each scenario. The self-confidence form comprised a short checklist of actions deemed essential for the successful management of the scenario emergency. Each action was followed by a 0-10 Likert confidence scale (Table 1). Participant
566 Table 1
A.S. Bullough et al. Pre-scenario versus post-scenario confidence scores N
Eclampsia Call for TEAM help OB, AN, N Eclampsia: Nominate timer and documenter Eclampsia: Do NOT restrain patient BLS: Place patient in Left lateral position, suction Eclampsia: Place high flow oxygen via face mask Eclampsia: Medications: Mg 6 g IV 5–20 minutes Eclampsia: Stabilize mother: Facemask Oxygen IV access Monitor fetus Maternal cardiac arrest Call the code/TEAM Perform LUD Perform effective chest compressions on a pregnant woman Oxygenate BVM/intubate a pregnant woman Deliver a shock to a pregnant woman Identify cause of cardiac arrest in a parturient Administer correct medications to treat cause Perform a perimortem cesarean delivery within 5 minutes of no ROSC Shoulder dystocia Call for TEAM help OB, AN, N, Neonatology Call out maneuvers Mc Roberts positioning Suprapubic pressure Prevent 3rd/4th degree tear Woods screw maneuver Posterior arm delivery Basic neonatal resuscitation Cord prolapse and neonatal resuscitation Call for TEAM help OB, AN, N, Neonatology Immediately lift presenting part and avoid handling cord Prepare OR stat for C/S Consider mother adopting knee-chest position or Trendelenberg Consider tocolysis while preparing for C/S Determine anesthesia GA vs Neuraxial technique Basic neonatal resuscitation
Median confidence (IQR)
P
Pre-scenario
Post-scenario
10 10 10 10 10 10
9.00 (6.75–10.00) 9.00 (6.00–10.00) 8.00 (6.00–10.00) 8.50 (6.75–10.00) 8.50 (5.75–10.00) 8.00 (6.00–10.00)
10.00 (8.00–10.00) 9.00 (7.75–10.00) 10 .00 (8.00–10.00) 10.00 (8.00–10.00) 9.00 (8.00–10.00) 9.00 (8.00–10.00)
.04 .16 .04 .06 .02 .02
20 20 20 20 20 20 20 20
9.50 (7.00–10.00) 9.50 (5.50–10.00) 8.50 (6.25–9.75) 5.00 (0.00–7.00) 6.00 (3.50–8.00) 6.00 (4.25–7.00) 6.50 (3.50–8.00) 0.00 (0.00–6.50)
10.00 (9.00–10.00) 10.00 (8.25–10.00) 9.00 (8.00–10.00) 6.50 (3.50–8.75) 8.00 (5.25–9.00) 8.00 (6.25–9.00) 8.00 (6.00–9.00) 2.00 (0.00–8.00)
.06 .32 .03 .001 .050 .01 .02 .01
15 15 14 15 15 15 14 15
8.00 (6.00–10.00) 8.00 (1.75–9.00) 8.00 (2.00–10.00) 8.00 (5.00–10.00) 6.00 (0.00–9.00) 4.00 (0.00–7.00) 2.00 (0.00–5.00) 5.00 (3.00–9.00)
9.00 (8.00–10.00) 9.00 (5.00–10.00) 9.00 (5.75–10.00) 9.00 (6.00–10.00) 7.00 (0.00–9.00) 8.00 (2.00–9.00) 6.00 (2.00–9.00) 6.00 (3.00–9.00)
.02 .03 .03 .02 .10 .01 .01 .08
5 5 5 5 5 5 5
8.00 (7.00–9.00) 8.00 (5.50–8.50) 7.00 (2.00–9.50) 8.00 (3.50–9.00) 6.00 (5.00–8.00) 7.00 (0.00–7.50) 5.00 (5.00–8.00)
10.00 (8.00–10.00) 9.00 (8.50–9.50) 9.00 (3.00–10.00) 8.00 (7.00–8.50) 7.00 (6.00–8.00) 7.00 (0.00–9.00) 8.00 (6.00–9.00)
.08 .07 .10 .29 .18 .18 .10
N, number of valid observations to compute the estimate. Significance (P) is based on the nonparametric Wilcoxon signed rank test.
specialty and length of clinical experience on the LDU was also noted on the form. Also in phase 2, the participant simulation team was standardized to include two obstetrics and gynecology residents, one anesthesia resident and two obstetric nurses. A weekly attempt was made to hold an in-situ LDU simulation session. The threemonthly or quarterly high-fidelity sessions at the campus simulation center took place during protected resident teaching time and obstetric nursing provided two nursing participants.
2.3. Simulation session structure The outline structure of both the in-situ and high fidelity simulation sessions comprised the following: a quick selfintroduction of the standardized team, a scenario briefing and review of objectives by the physician-facilitator for a videorecorded scenario of 15–30 minutes duration. Each session ended with a structured oral and video debrief by the
physician-facilitator and clinical nurse specialist i.e. both members of the original team who designed the program. The general debrief included an initial group reflection, discussion of clinical objectives and use of communication strategies.
2.4. LDU materials The in-situ, low fidelity scenarios on the LDU used a NOELLE S550 maternal birthing simulator with the exception of the eclamptic seizure scenario where, to improve realism, a briefed, volunteer female medical student replaced NOELLE as this model did not have an eclamptic seizure function. In each scenario, where indicated, materials were used to improve realism (eg, intravenous fluids and tubing, a vaginal delivery tray, a yellow epidural catheter attached to an epidural pump taped to the back of NOELLE, a cardiac arrest cart, endotracheal tube, functioning laryngoscope, neonatal resuscitaire).
Teamwork simulation challenges Table 2
567
Characteristics of the Participants.
Characteristic
N
Obstetric Experience ⁎ (in months)
Obstetric resident postgraduate year (PGY) PGY 1–2 6 8 PGY 3–4 6 40 Anesthesiology resident PGY 3 4 0.5 Obstetric nursing experience 0–5 years 6 19 N5 years 0 0 N10 years 6 171 (14.25 years) N20 years 2 306 (25.5 years) TOTAL 30 ⁎ Median.
2.5. Simulation center materials A simulation center technician programmed a more complex NOELLE S575 maternal birthing simulator for eclampsia (this model was able to simulate an eclamptic seizure), maternal cardiac arrest, and shoulder dystocia scenarios, and a newborn system, HAL S3010 for neonatal resuscitation. Large monitor screens above NOELLE and HAL displayed pertinent physiological parameters for the chosen scenario. Stage setting with materials tailored to each emergency scenario took place. Material examples included an epidural pump with a bag of saline attached to a yellow epidural catheter taped to NOELLE's back, intravenous fluids attached to an 18G intravenous catheter taped to NOELLE's arm, a bag of whitened saline mimicked lipid emulsion, a cardiac arrest cart with labeled drugs, an emergency surgical cesarean section kit and CPAP airway tubing for oxygen delivery to newborn HAL.
rank test; (2) CTS inter-rater reliability using Kendall's coefficient of concordance (W); and (3) Confidence surveys used a non-parametric Wilcoxon signed rank test to see whether a participant's confidence improved from pre-scenario briefing to post-scenario debriefing. Information technology (IT) difficulties led to the completion and analysis of only 18 pre/post web-based multiplechoice questionnaires by nursing and physician professionals. These results showed there was no significant improvement in raw score performance from pre-test to post-test (P = .27) but this may be due to small sample size. The multiple-choice test had good psychometric properties as supported by the number of questions with good biserial values, but more examinees were needed for advanced analysis. The clinical teamwork scale (CTS) used to rate both live and videoed scenarios showed increased inter-rater variability. During the CTS live assessment, after adjusting for chance agreement alone, the raters were in agreement only 31% of the time across all 15 CTS items evaluated (Kendall's W = .31, P b .001). During the CTS video assessment, after adjusting for chance agreement alone, the raters were in agreement only 28% of the time across all 15 items evaluated (Kendall's = .28, P b .001). This poor inter-rater reliability was explained by regular raters being absent during some of the insitu simulation sessions. Junior participant self-reported experience providing emergent care for an obstetric emergency was limited before participation in the OBANN program. No participant had observed or participated in a maternal cardiac arrest. Confidence surveys overall revealed confidence significantly increased (P b .05), from pre-scenario briefing to after post-scenario debriefing, in 16/29 domains as indicated in Table 3.
4. Discussion 3. Results During Phase 3, over a four-month pilot data collection period, a total of 30 different healthcare professionals (Table 2) participated in both LDU and high fidelity simulation center emergency scenarios; 16 physicians and 14 obstetric nurses. Ten sessions in total were carried out; six labor and delivery unit in-situ sessions out of a possible 14 (if we had been able to run one session a week) were completed. Reasons for this included lack of availability of staff participants due to the clinical workload and work schedules. The latter reason also impacted availability of trained rater staff. Four education-protected, high-fidelity sessions were run over 2 quarterly education meetings at the simulation center. Some physicians and nursing staff completed more than one session and no participant had prior team simulation experience. Pilot data analysis looked at (1) pre and post-test multiplechoice questionnaire using a non-parametric Wilcoxon signed
Current evidence shows that team simulation training augments the educational curriculum to improve patient safety and reduce the likelihood of adverse outcomes [7,8]. Residents usually remember 90% of what they do yet only 10% of what they read [9]. We know that implementation of institutional emergency protocols and development of standardized teaching and evaluation methods promote the efficacy of team simulation training and education, however, the administrative, clinical and time restrictions associated with meeting these ends both as an individual and as a group cannot be understated.
4.1. Challenges encountered in program development and assessment This pilot project reflected numerous challenges associated with developing a team simulation-based obstetric emergency program to promote communication and teamwork on an LDU. An initial challenge was identifying front-line care leaders with a ceaselessly enthusiastic point person to unite
568 Table 3
A.S. Bullough et al. Center-based vs in situ simulation
Center-based
In situ
Technology
Technology
• Highly sophisticated computers • One-way glass • Control station; simulation team communicates freely
Convenience • Convenient for simulation personnel • Challenging for participants (travel, unfamiliar environment, and equipment) • Less likely that all personnel from the clinical settings can simultaneously participate (individuals bring back lesson learned to local group) • May limit combined interdisciplinary team opportunities Scheduling • Need to arrange substantial time away from clinical activities for staff to attend • Reliably available at time when the simulation is scheduled • May offer expanded time for simulation and debriefing
Medication safety and clinical equipment • Medications can appear identical to real medications (little risk for use in real life) • Uses center equipment, which may not match local clinical equipment
Debriefing • High-tech room (with computer/video overlays possible) Team-training implications • Can teach teamwork principles • Able to mix classroom setting and simulated clinical setting • May be more challenging to transfer teamwork skills because of – Limited personnel in attendance – Lack of familiar triggers to imprint into “muscle memory” From Guise et al [18]; reprinted with permission from the Joint Commission Resources.
• Low-tech computers simplify connectivity and technical difficulties • No physical barriers between participants and simulation personnel • No control station; all personnel communicate in role Convenience • Challenging/Inconvenient for simulation personnel (travel, unfamiliar environment/equipment, frequent set ups) • Convenient and familiar to participants • Clinical teams can respond in simulations with their partners in the real clinical setting (promotes and reinforces group learning and adoption)
Scheduling • Staff can participate while working (need to arrange cross-coverage for smaller times) • Less reliable scheduling because clinical volume may result in all rooms being full and need to reschedule • May be more time pressure for simulation and debriefing Medication safety and clinical equipment • Simulated medications intentionally appear fake to prevent administration to real patients. • Can use familiar local medical equipment (i.e., electronic fetal monitors) • Allows for immediate identification for some safety issues in own clinical environment Debriefing • Can use any private room close to simulation • Video must include important visuals (not conducive to overlays) Team-training implications • May be easier to transfer teamwork skills to clinical setting because of – Actual clinical teams practicing and learning together – Familiar surroundings to trigger to imprint into “muscle memory” – Clinical setting may increase meaning and activate learning among participants
Teamwork simulation challenges an inter-professional team. We were fortunate to develop one such team recruited by the point person, who received departmental level support for this initiative. As meetings were rescheduled or missed, we reminded ourselves of the shared goal to improve obstetric patient outcomes through promotion of communication and teamwork among LDU personnel. Another major challenge was development of a curriculum and determination of what evaluation tools provided reliable and valid data. Discussion with education specialists helped guide our choice of assessment tools, but they were readily frustrated by the heterogeneity of the group being assessed and difficulty in acquiring relevant data for in-situ simulation sessions. Determinations of obstetric scenarios were made by looking at national obstetric outcomes, other obstetric emergency simulation-based training programs as well as our institutional clinical climate [10-12]. In addition, a recent revision in management guidelines for maternal cardiac arrest secured its inclusion in the OBANN program [13]. Scenario development also comprised faculty and nursing personal experience. The Web-based communication modules were based on OBANN faculty involvement in Team Strategies and Tools to Enhance Performance and Patient Safety, i.e., Team STEPPS, and Obstetric, Emergency Medicine, Anesthesiology and Neonatology, or OBEMAN, programs and chiefly extorted the values of successful team work such as, leadership, situational awareness, mutual support, and most importantly communication strategies, e.g., Situation, Background Assessment Recommendation (SBAR), and Concerned, Uncomfortable, Safety (CUS) [14]. The aim of the website was to complement the simulation sessions and for the participant to be able to access the site and learn about teamwork and communication strategies at their own pace. Ideally how often the website was accessed and by whom would have been helpful data, however, these data were not available. Other IT challenges relating to test access also existed. The MCQ showed promise as a learning tool to reinforce knowledge, however, only eighteen participants were able to complete both preOBANN and post-OBANN questionnaires. Real-time and video-recorded based evaluations of performance of non-technical skills (NTS's) usually provide equally reliable methods of assessment [15]. Communication and teamwork are fundamental key NTS's [16] that represented our main program measurement outcomes especially as participants were not medical students but represented post-graduate professionals with varying levels of clinical expertise. The poor CTS inter-rater reliability indicated that training was required for ALL CTS raters and that this number of personnel trained should cover unanticipated absences and scheduling conflicts. To ensure in-situ simulation sessions took place on the LDU, OBANN development faculty replaced absent CTS raters. The physician replacements skewed inter-rater reliability by rating more harshly than the regular raters. In summary, from our experience, the challenges or limitations of implementing a team obstetric emergency simulation program included identification of front-line care leaders, the
569 heterogeneity of the participants being evaluated, identification of institutional user-friendly IT that permitted timely completion of the MCQ component of the program, supported a video browser and was able to check OBANN website access. Finally, the availability of required personnel on the labor and delivery unit as determined by clinical workload and work schedules did impact the number of in-situ sessions taking place as well as skewed inter-rater reliability data.
4.2. High vs. low fidelity vs. hybrid simulation training Performance in simulation training varies from other forms of assessment, e.g., viva voce, written examination [17]. Possession of medical knowledge does not translate into being able to apply it correctly especially under stressful circumstances. Any complex medical simulation scenario will require use of several skill sets such as clinical psychomotor, problem solving, communication, and technical skills. Simulation medical education requires regular practice of the aforementioned skills and regular program review in order to develop a program that not only meets the educational needs of the LDU workforce but also makes a difference in obstetric patient outcomes and is ultimately data driven. We know that differences exist between low and high fidelity simulation training (Table 3) [18], with in-situ simulation presenting more logistic and cultural challenges than a high fidelity simulation environment [19]. However, did the hybrid nature of the program detract from the education and training received? Literature shows that there is very little identifiable difference between either low or high fidelity simulation and that equipment is not the only factor that matters. A welldesigned curriculum works well even in low equipment fidelity simulation [20]. In addition, the self-confidence assessment forms revealed significantly improved confidence behaviors among participants in whatever environment the simulation was performed. From a patient safety point of view, the insitu OBANN program also assisted with the identification of previously unrecognized operational patient latent safety threats (LST's) on the LDU, i.e., medication, equipment or system issues [21]. Some of the patient LST's identified on our LDU included, location of emergency drugs such as IV lipid emulsion and a 50mls vial 0.5% lidocaine for possible emergent cesarean section under a local anesthetic field block; location of documenter sheets for use during an obstetric emergency, equipment readiness, ie, suction with no tubing attached and location of a central alarm button for extra anesthesia assistance from main operating room front desk.
4.3. Recommendations for future obstetric team simulation program developers We do recommend revising assessment tools regularly to ensure that any future team simulation program is moving forward based on project data outcomes. This approach will make data relevant as well as help save time in not pursuing poor
570 practices. Leadership, ownership and departmental support from within each profession are also vital to ensure not only initial success but also durability of the program. As well as a motivational group effort, basic materials, extended assessment training and adequate IT support are required to ensure relevant data capture. Through feedback responses and observation, benefits of a hybrid obstetric emergency team simulation program included improved participant confidence, high participant satisfaction and familiarization taking place among the first-line responders to an obstetric emergency on the LDU, promotion of a non-threatening environment, interprofessional communication, identification of patient LST's and possible development of transferable skills and communication techniques [22]. In conclusion, our simulation program was incorporated into both a regular LDU workday and protected educational time. Faculty and nursing professionals of varying levels, some with limited clinical experience and low confidence, were able to participate in in-situ and high fidelity emergency obstetric scenarios before being faced with a real obstetric emergency. Our goal is to continue collecting data, revise evaluation tools and rater training and further refine the OBANN program to future accommodate neonatology and incorporate generic training, milestones so that these changes become a reality in a simulation world.
A.S. Bullough et al.
[6]
[7]
[8]
[9] [10]
[11]
[12]
[13]
[14]
[15]
[16]
Acknowledgements [17]
Alyssa Paratore RN and Krystal Koszola RN for their assistance with CTS rating and Mary Matimore for compiling data. [18]
References [19] [1] American College of Obstetricians and Gynecologists. Preparing for clinical emergencies in obstetrics and gynecology. Committee opinion no. 590. Obstet Gynecol 2014;123:722-5. [2] Maslovitz S, Barkai G, Lessing JB, Ziv A, Many A. Recurrent obstetric management mistakes identified by simulation. Obstet Gynecol 2007; 109:1295-300. [3] Institute of Medicine. Committee on quality of health Care in America. In: Kohn L, Corrigan J, Donaldson M, editors. To err is human: building a safer health system. Washington, DC: National Academy Press; 2000. p. 146. [4] The Joint Commission Sentinel Event Alert. Issue 30: preventing infant death and injury during delivery. 2004 [Available at:] http://www.jointcommission.org/assets/1/18/SEA_30.PDF. [5] American College of Obstetricians and Gynecologists. Professional responsibilities in obstetric-gynecologic medical education and training.
[20]
[21]
[22]
Committee opinion no. 500. Obstet Gynecol 2011;118:400-4 [Reaffirmed 2014]. Guise JM, Deering S, Kanki B, Osterweil P, Li H, Stanford S, et al. STORC OB safety initiative: development and validation of the clinical teamwork scale to evaluate teamwork. Simul Healthc 2008;3:217-23. Draycott T, Sibanda T, Owen L, Akande V, Winter C, Reading S, et al. Does training in obstetric emergencies improve neonatal outcome? BJOG 2006;113:177-82. Merien AE, van de Ven J, Mol BW, Houterman S, Sg O. Multidisciplinary team training in a simulation setting for acute obstetric emergencies: a systematic review. Obstet Gynecol 2010;115:1021-31. Croley WC, Rothenberg DM. Education of trainees in the intensive care unit. Crit Care Med 2007;35:S117-21 [suppl]. Johanson R, Cox C, O'Donnell E, Grady K, Howell C, Jones P. Managing obstetric emergencies and trauma (MOET): structured skills training using models and reality based scenarios. Obstet Gynecol 1999;1:46-52. Beasley JW, Damos JR, Roberts RG, Nesbitt TS. The advanced life support in obstetrics course: a national program to enhance obstetric emergency skill and to support maternity care practice. Arch Fam Med 1994;3:1037-41. Johnson R, Cox C, O'Donnell E, Grady K, Howell C, Jones P. PROMPT education and development: saving mothers' and babies' lives in resource poor settings. BJOG 2009;116:868-9. Beasley JW, Damos JR, Roberts RG, Nesbitt TS. The Society for Obstetric Anesthesia and Perinatology consensus statement on the Management of Cardiac Arrest in pregnancy. Anesth Analg 2014;118:1003-16. Clancy CM, Tornberg DN. Team STEPPS: integrating teamwork principles into healthcare practice. [Available at:] http://www.psqh.com/novdec06/ahrq.html. [Accessed June 2015] 2015. Williams JB, McDonough MA, Hilliard MW, Williams AL, Cuniowski PC, Gonzalez MG. Intermethod reliability of real-time versus delayed videotaped evaluation of a high-fidelity medical simulation septic shock scenario. Acad Emerg Med 2009;16:887-93. Riley W, Davis S, Miller K, Hansen H, Sainfort F, Sweet R. Didactic and simulation nontechnical skills team training to improve perinatal patient outcomes in a community hospital. Jt Comm J Qual Patient Saf 2011; 37:357-64. Savoldelli GL, Main VN, Joo SH, Houston PL, Graham M, Yee B, et al. Evaluation of patient simulator performance as an adjunct to the oral examination for senior anesthesia residents. Anesthesiology 2006;104: 475-81. Guise JM, Lowe NK, Deering S, Lewis PO, O'Haire C, Irwin LK, et al. Mobile in situ obstetric emergency simulation and teamwork training to improve maternal-fetal safety in hospitals. Jt Comm J Qual Patient Saf 2010;36:443-53. Patterson MD, Blike GT, Nadkarni VM. In-situ simulation: challenges and results. In Henriksen K, Battles, JB, Keyes, MA and Grady, ML ed. Advances in patient safety: new directions and alternative approaches (vol 3: performance and tools). Rockville MD: Agency for healthcare Research and Quality. 2008 [Available at: http://www.ahrq.gov/downloads/pub/advances2/vol3/advances-patterson_48.pdf]. Meurling L, Hedman L, Lidefelt KJ, Escher C, Felländer-Tsai L, Wallin CJ. Comparison of high- and low equipment fidelity during paediatric simulation team training: a case control study. BMC Med Educ 2014;14:221. Patterson MD, Geis GL, Falcone RA, LeMaster T, Wears RL. In situ simulation: detection of safety threats and teamwork training in a high risk emergency department. BMJ Qual Saf 2013;22:468-77. van der Nelson HA, Siasskos D, Bennett J, Godfrey M, Spray L, Draycott T, et al. Multiprofessional team simulation training on an obstetric model can improve teamwork in other areas of health care. Am J Med Qual 2014;29:78-82.
Original contribution
Obstetric team simulation program challenges A.S. Bullough MD (Associate Professor)a,e,⁎, S. Wagner MD (Associate Professor)b,e , T. Boland MS, RNC (Clinical Nurse Specialist Women's Health)c,e , T.P. Waters MD (Associate Professor)b,e , K. Kim MD, PhD (Professor)a,e , W. Adams MA (Biostatistician)d,e a
Department of Anesthesiology Department of Obstetrics and Gynecology c Obstetric and Gynecology Nursing Department d Clinical Research Office e Loyola University Health System, Loyola University Stritch School of Medicine b
Received 28 September 2015; revised 16 August 2016; accepted 17 August 2016
Keywords: Team simulation challenges; Obstetric emergency team simulation
Abstract Objective: To describe the challenges associated with the development and assessment of an obstetric emergency team simulation program. Design: The goal was to develop a hybrid, in-situ and high fidelity obstetric emergency team simulation program that incorporated weekly simulation sessions on the labor and delivery unit, and quarterly, education protected sessions in the simulation center. All simulation sessions were video-recorded and reviewed. Setting: Labor and delivery unit and simulation center. Participants: Medical staff covering labor and delivery, anesthesiology and obstetric residents and obstetric nurses. Measurements: Assessments included an on-line knowledge multiple-choice questionnaire about the simulation scenarios. This was completed prior to the initial in-situ simulation session and repeated 3 months later, the Clinical Teamwork Scale with inter-rater reliability, participant confidence surveys and subjective participant satisfaction. A web-based curriculum comprising modules on communication skills, team challenges, and team obstetric emergency scenarios was also developed. Main results: Over 4 months, only 6 labor and delivery unit in-situ sessions out of a possible 14 sessions were carried out. Four high-fidelity sessions were performed in 2 quarterly education protected meetings in the simulation center. Information technology difficulties led to the completion of only 18 pre/post web-based multiple-choice questionnaires. These test results showed no significant improvement in raw score performance from pre-test to post-test (P = .27). During Clinical Teamwork Scale live and video assessment, trained raters and program faculty were in agreement only 31% and 28% of the time, respectively (Kendall's W = .31, P b .001 and W = .28, P b .001). Participant confidence surveys overall revealed confidence significantly increased (P b .05), from pre-scenario briefing to after post-scenario debriefing. Conclusion: Program feedback indicates a high level of participant satisfaction and improved confidence yet further program refinement is required. © 2016 Elsevier Inc. All rights reserved.
⁎ Corresponding author at: Loyola University Health System, 2160 S. First Avenue, Maywood, IL 60153. Tel.: +1 708 216 8866. E-mail address: [email protected] (A.S. Bullough). http://dx.doi.org/10.1016/j.jclinane.2016.08.019 0952-8180/© 2016 Elsevier Inc. All rights reserved.
Teamwork simulation challenges
1. Introduction The American College of Obstetricians and Gynecologists (ACOG) Committee Opinion (No 590) on Preparing for Clinical Emergencies in Obstetrics advocates improvement in response systems for in-patient emergency care [1]. Two ways in which to address this goal include standardization of obstetric emergency management protocols and team simulation training. The initial management and therapeutic interventions are critical when considering patient safety and a positive outcome in an obstetric emergency. Simulation training is able to identify mistakes in management during an obstetric crisis and is endorsed as a use of formal training by the Institute of Medicine, Joint Commission on Accreditation of Healthcare Organizations and ACOG to improve clinical skills, interdisciplinary teamwork and reduce communication-related errors [2-5]. At our tertiary care referral institution, the Obstetric department primarily deals with under-served, high-risk patients with multiple medical issues that can lead to a potential crisis. In the past 2 years, the labor and delivery unit (LDU) has seen an increase in the high-risk parturients. Because of our acuity and in response to the call for increased preparedness, we developed a hybrid, in-situ and high-fidelity obstetric emergency team simulation program comprising OBstetric, ANesthesiology and Nursing professionals (OBANN) to improve emergency clinical management on the LDU. In what follows we present a general description of the program, the team simulation challenges (Fig. 1) and lessons learned from our endeavor.
2. Materials and methods Following institutional review board exemption, we implemented the OBANN program in three phases. Phase 1, the development phase from February 2014 to June 2014, involved scenario and web-based curriculum development as well as determination of assessment and learning instruments. Phase 2 from July 2014 to November 2014 tested the assessment instruments and training for those who were rating the
565 participants in the scenarios. Phase 3 from December 2014 to March 2015 collected pilot data.
2.1. Phase 1 development In Phase 1, an inter-professional group comprising an obstetric clinical nurse specialist, a nominated representative from the maternal fetal medicine group, the obstetric residency director, and two representatives from obstetric anesthesiology met on a monthly basis over a 5-month period to set objectives, create emergency simulation scenarios and develop a curriculum. The program objectives were (1) identification of the roles and responsibilities of the obstetric team members, (2) implementation of clear communication strategies, and (3) creation of an environment that promotes inter-professional mutual support during an obstetric crisis. Four obstetric emergency scenarios with specific clinical objectives were constructed. The cases included eclampsia, shoulder dystocia with basic neonatal resuscitation, cord prolapse with basic neonatal resuscitation, and maternal cardiac arrest. A Web-based tutorial composed of 5 different modules was also created. These modules comprised a program overview, team challenges, team communication skills, team interaction and applied teamwork regarding the simulation scenarios. We determined that all participants working continuously (obstetric nurses) or on rotational assignment (obstetric and anesthetic residents) to the LDU would receive an e-mail invitation to access the OBANN web site. Initiating the program LDU participants were required to complete a 20-question multiple-choice questionnaire (MCQ) based on the simulation scenarios and the on-line web-site team communication modules. The same test was repeated 3 months later after completion of the quarterly high fidelity session. The validated Clinical Teamwork Scale (CTS) Global was determined the best tool to assess scenario non-technical skills such as teamwork and communication. [6] The CTS tool measures 15 points on a 1-10 (unacceptable to perfect) Likert scale. The major CTS assessment headings included overall teamwork, communication, situational awareness, decision making, role responsibility and patient friendliness (in one scenario the patient is awake then has a seizure).
2.2. Phase 2 implementation Developing a simulation team of front-line care leaders Organizational and departmental support Determining objectives and safety aims Measuring training effectiveness Availability of information technology resources Implementing data driven change Creating a non-threatening learning environment Time and personnel limitations for in-situ sessions Fig. 1
Obstetric team simulation program challenges.
During phase 2 two volunteer obstetric nurses became CTS evaluation raters. Rater training involved reviewing the case content and objectives, practice rating, and discussion and revision of the CTS evaluation tool. A faculty member of the designing program team with simulation assessment experience reviewed the live and video practice rating session scoring and clarified any unclear items or missing items during the training period. We also recorded participant self-confidence before and after each scenario. The self-confidence form comprised a short checklist of actions deemed essential for the successful management of the scenario emergency. Each action was followed by a 0-10 Likert confidence scale (Table 1). Participant
566 Table 1
A.S. Bullough et al. Pre-scenario versus post-scenario confidence scores N
Eclampsia Call for TEAM help OB, AN, N Eclampsia: Nominate timer and documenter Eclampsia: Do NOT restrain patient BLS: Place patient in Left lateral position, suction Eclampsia: Place high flow oxygen via face mask Eclampsia: Medications: Mg 6 g IV 5–20 minutes Eclampsia: Stabilize mother: Facemask Oxygen IV access Monitor fetus Maternal cardiac arrest Call the code/TEAM Perform LUD Perform effective chest compressions on a pregnant woman Oxygenate BVM/intubate a pregnant woman Deliver a shock to a pregnant woman Identify cause of cardiac arrest in a parturient Administer correct medications to treat cause Perform a perimortem cesarean delivery within 5 minutes of no ROSC Shoulder dystocia Call for TEAM help OB, AN, N, Neonatology Call out maneuvers Mc Roberts positioning Suprapubic pressure Prevent 3rd/4th degree tear Woods screw maneuver Posterior arm delivery Basic neonatal resuscitation Cord prolapse and neonatal resuscitation Call for TEAM help OB, AN, N, Neonatology Immediately lift presenting part and avoid handling cord Prepare OR stat for C/S Consider mother adopting knee-chest position or Trendelenberg Consider tocolysis while preparing for C/S Determine anesthesia GA vs Neuraxial technique Basic neonatal resuscitation
Median confidence (IQR)
P
Pre-scenario
Post-scenario
10 10 10 10 10 10
9.00 (6.75–10.00) 9.00 (6.00–10.00) 8.00 (6.00–10.00) 8.50 (6.75–10.00) 8.50 (5.75–10.00) 8.00 (6.00–10.00)
10.00 (8.00–10.00) 9.00 (7.75–10.00) 10 .00 (8.00–10.00) 10.00 (8.00–10.00) 9.00 (8.00–10.00) 9.00 (8.00–10.00)
.04 .16 .04 .06 .02 .02
20 20 20 20 20 20 20 20
9.50 (7.00–10.00) 9.50 (5.50–10.00) 8.50 (6.25–9.75) 5.00 (0.00–7.00) 6.00 (3.50–8.00) 6.00 (4.25–7.00) 6.50 (3.50–8.00) 0.00 (0.00–6.50)
10.00 (9.00–10.00) 10.00 (8.25–10.00) 9.00 (8.00–10.00) 6.50 (3.50–8.75) 8.00 (5.25–9.00) 8.00 (6.25–9.00) 8.00 (6.00–9.00) 2.00 (0.00–8.00)
.06 .32 .03 .001 .050 .01 .02 .01
15 15 14 15 15 15 14 15
8.00 (6.00–10.00) 8.00 (1.75–9.00) 8.00 (2.00–10.00) 8.00 (5.00–10.00) 6.00 (0.00–9.00) 4.00 (0.00–7.00) 2.00 (0.00–5.00) 5.00 (3.00–9.00)
9.00 (8.00–10.00) 9.00 (5.00–10.00) 9.00 (5.75–10.00) 9.00 (6.00–10.00) 7.00 (0.00–9.00) 8.00 (2.00–9.00) 6.00 (2.00–9.00) 6.00 (3.00–9.00)
.02 .03 .03 .02 .10 .01 .01 .08
5 5 5 5 5 5 5
8.00 (7.00–9.00) 8.00 (5.50–8.50) 7.00 (2.00–9.50) 8.00 (3.50–9.00) 6.00 (5.00–8.00) 7.00 (0.00–7.50) 5.00 (5.00–8.00)
10.00 (8.00–10.00) 9.00 (8.50–9.50) 9.00 (3.00–10.00) 8.00 (7.00–8.50) 7.00 (6.00–8.00) 7.00 (0.00–9.00) 8.00 (6.00–9.00)
.08 .07 .10 .29 .18 .18 .10
N, number of valid observations to compute the estimate. Significance (P) is based on the nonparametric Wilcoxon signed rank test.
specialty and length of clinical experience on the LDU was also noted on the form. Also in phase 2, the participant simulation team was standardized to include two obstetrics and gynecology residents, one anesthesia resident and two obstetric nurses. A weekly attempt was made to hold an in-situ LDU simulation session. The threemonthly or quarterly high-fidelity sessions at the campus simulation center took place during protected resident teaching time and obstetric nursing provided two nursing participants.
2.3. Simulation session structure The outline structure of both the in-situ and high fidelity simulation sessions comprised the following: a quick selfintroduction of the standardized team, a scenario briefing and review of objectives by the physician-facilitator for a videorecorded scenario of 15–30 minutes duration. Each session ended with a structured oral and video debrief by the
physician-facilitator and clinical nurse specialist i.e. both members of the original team who designed the program. The general debrief included an initial group reflection, discussion of clinical objectives and use of communication strategies.
2.4. LDU materials The in-situ, low fidelity scenarios on the LDU used a NOELLE S550 maternal birthing simulator with the exception of the eclamptic seizure scenario where, to improve realism, a briefed, volunteer female medical student replaced NOELLE as this model did not have an eclamptic seizure function. In each scenario, where indicated, materials were used to improve realism (eg, intravenous fluids and tubing, a vaginal delivery tray, a yellow epidural catheter attached to an epidural pump taped to the back of NOELLE, a cardiac arrest cart, endotracheal tube, functioning laryngoscope, neonatal resuscitaire).
Teamwork simulation challenges Table 2
567
Characteristics of the Participants.
Characteristic
N
Obstetric Experience ⁎ (in months)
Obstetric resident postgraduate year (PGY) PGY 1–2 6 8 PGY 3–4 6 40 Anesthesiology resident PGY 3 4 0.5 Obstetric nursing experience 0–5 years 6 19 N5 years 0 0 N10 years 6 171 (14.25 years) N20 years 2 306 (25.5 years) TOTAL 30 ⁎ Median.
2.5. Simulation center materials A simulation center technician programmed a more complex NOELLE S575 maternal birthing simulator for eclampsia (this model was able to simulate an eclamptic seizure), maternal cardiac arrest, and shoulder dystocia scenarios, and a newborn system, HAL S3010 for neonatal resuscitation. Large monitor screens above NOELLE and HAL displayed pertinent physiological parameters for the chosen scenario. Stage setting with materials tailored to each emergency scenario took place. Material examples included an epidural pump with a bag of saline attached to a yellow epidural catheter taped to NOELLE's back, intravenous fluids attached to an 18G intravenous catheter taped to NOELLE's arm, a bag of whitened saline mimicked lipid emulsion, a cardiac arrest cart with labeled drugs, an emergency surgical cesarean section kit and CPAP airway tubing for oxygen delivery to newborn HAL.
rank test; (2) CTS inter-rater reliability using Kendall's coefficient of concordance (W); and (3) Confidence surveys used a non-parametric Wilcoxon signed rank test to see whether a participant's confidence improved from pre-scenario briefing to post-scenario debriefing. Information technology (IT) difficulties led to the completion and analysis of only 18 pre/post web-based multiplechoice questionnaires by nursing and physician professionals. These results showed there was no significant improvement in raw score performance from pre-test to post-test (P = .27) but this may be due to small sample size. The multiple-choice test had good psychometric properties as supported by the number of questions with good biserial values, but more examinees were needed for advanced analysis. The clinical teamwork scale (CTS) used to rate both live and videoed scenarios showed increased inter-rater variability. During the CTS live assessment, after adjusting for chance agreement alone, the raters were in agreement only 31% of the time across all 15 CTS items evaluated (Kendall's W = .31, P b .001). During the CTS video assessment, after adjusting for chance agreement alone, the raters were in agreement only 28% of the time across all 15 items evaluated (Kendall's = .28, P b .001). This poor inter-rater reliability was explained by regular raters being absent during some of the insitu simulation sessions. Junior participant self-reported experience providing emergent care for an obstetric emergency was limited before participation in the OBANN program. No participant had observed or participated in a maternal cardiac arrest. Confidence surveys overall revealed confidence significantly increased (P b .05), from pre-scenario briefing to after post-scenario debriefing, in 16/29 domains as indicated in Table 3.
4. Discussion 3. Results During Phase 3, over a four-month pilot data collection period, a total of 30 different healthcare professionals (Table 2) participated in both LDU and high fidelity simulation center emergency scenarios; 16 physicians and 14 obstetric nurses. Ten sessions in total were carried out; six labor and delivery unit in-situ sessions out of a possible 14 (if we had been able to run one session a week) were completed. Reasons for this included lack of availability of staff participants due to the clinical workload and work schedules. The latter reason also impacted availability of trained rater staff. Four education-protected, high-fidelity sessions were run over 2 quarterly education meetings at the simulation center. Some physicians and nursing staff completed more than one session and no participant had prior team simulation experience. Pilot data analysis looked at (1) pre and post-test multiplechoice questionnaire using a non-parametric Wilcoxon signed
Current evidence shows that team simulation training augments the educational curriculum to improve patient safety and reduce the likelihood of adverse outcomes [7,8]. Residents usually remember 90% of what they do yet only 10% of what they read [9]. We know that implementation of institutional emergency protocols and development of standardized teaching and evaluation methods promote the efficacy of team simulation training and education, however, the administrative, clinical and time restrictions associated with meeting these ends both as an individual and as a group cannot be understated.
4.1. Challenges encountered in program development and assessment This pilot project reflected numerous challenges associated with developing a team simulation-based obstetric emergency program to promote communication and teamwork on an LDU. An initial challenge was identifying front-line care leaders with a ceaselessly enthusiastic point person to unite
568 Table 3
A.S. Bullough et al. Center-based vs in situ simulation
Center-based
In situ
Technology
Technology
• Highly sophisticated computers • One-way glass • Control station; simulation team communicates freely
Convenience • Convenient for simulation personnel • Challenging for participants (travel, unfamiliar environment, and equipment) • Less likely that all personnel from the clinical settings can simultaneously participate (individuals bring back lesson learned to local group) • May limit combined interdisciplinary team opportunities Scheduling • Need to arrange substantial time away from clinical activities for staff to attend • Reliably available at time when the simulation is scheduled • May offer expanded time for simulation and debriefing
Medication safety and clinical equipment • Medications can appear identical to real medications (little risk for use in real life) • Uses center equipment, which may not match local clinical equipment
Debriefing • High-tech room (with computer/video overlays possible) Team-training implications • Can teach teamwork principles • Able to mix classroom setting and simulated clinical setting • May be more challenging to transfer teamwork skills because of – Limited personnel in attendance – Lack of familiar triggers to imprint into “muscle memory” From Guise et al [18]; reprinted with permission from the Joint Commission Resources.
• Low-tech computers simplify connectivity and technical difficulties • No physical barriers between participants and simulation personnel • No control station; all personnel communicate in role Convenience • Challenging/Inconvenient for simulation personnel (travel, unfamiliar environment/equipment, frequent set ups) • Convenient and familiar to participants • Clinical teams can respond in simulations with their partners in the real clinical setting (promotes and reinforces group learning and adoption)
Scheduling • Staff can participate while working (need to arrange cross-coverage for smaller times) • Less reliable scheduling because clinical volume may result in all rooms being full and need to reschedule • May be more time pressure for simulation and debriefing Medication safety and clinical equipment • Simulated medications intentionally appear fake to prevent administration to real patients. • Can use familiar local medical equipment (i.e., electronic fetal monitors) • Allows for immediate identification for some safety issues in own clinical environment Debriefing • Can use any private room close to simulation • Video must include important visuals (not conducive to overlays) Team-training implications • May be easier to transfer teamwork skills to clinical setting because of – Actual clinical teams practicing and learning together – Familiar surroundings to trigger to imprint into “muscle memory” – Clinical setting may increase meaning and activate learning among participants
Teamwork simulation challenges an inter-professional team. We were fortunate to develop one such team recruited by the point person, who received departmental level support for this initiative. As meetings were rescheduled or missed, we reminded ourselves of the shared goal to improve obstetric patient outcomes through promotion of communication and teamwork among LDU personnel. Another major challenge was development of a curriculum and determination of what evaluation tools provided reliable and valid data. Discussion with education specialists helped guide our choice of assessment tools, but they were readily frustrated by the heterogeneity of the group being assessed and difficulty in acquiring relevant data for in-situ simulation sessions. Determinations of obstetric scenarios were made by looking at national obstetric outcomes, other obstetric emergency simulation-based training programs as well as our institutional clinical climate [10-12]. In addition, a recent revision in management guidelines for maternal cardiac arrest secured its inclusion in the OBANN program [13]. Scenario development also comprised faculty and nursing personal experience. The Web-based communication modules were based on OBANN faculty involvement in Team Strategies and Tools to Enhance Performance and Patient Safety, i.e., Team STEPPS, and Obstetric, Emergency Medicine, Anesthesiology and Neonatology, or OBEMAN, programs and chiefly extorted the values of successful team work such as, leadership, situational awareness, mutual support, and most importantly communication strategies, e.g., Situation, Background Assessment Recommendation (SBAR), and Concerned, Uncomfortable, Safety (CUS) [14]. The aim of the website was to complement the simulation sessions and for the participant to be able to access the site and learn about teamwork and communication strategies at their own pace. Ideally how often the website was accessed and by whom would have been helpful data, however, these data were not available. Other IT challenges relating to test access also existed. The MCQ showed promise as a learning tool to reinforce knowledge, however, only eighteen participants were able to complete both preOBANN and post-OBANN questionnaires. Real-time and video-recorded based evaluations of performance of non-technical skills (NTS's) usually provide equally reliable methods of assessment [15]. Communication and teamwork are fundamental key NTS's [16] that represented our main program measurement outcomes especially as participants were not medical students but represented post-graduate professionals with varying levels of clinical expertise. The poor CTS inter-rater reliability indicated that training was required for ALL CTS raters and that this number of personnel trained should cover unanticipated absences and scheduling conflicts. To ensure in-situ simulation sessions took place on the LDU, OBANN development faculty replaced absent CTS raters. The physician replacements skewed inter-rater reliability by rating more harshly than the regular raters. In summary, from our experience, the challenges or limitations of implementing a team obstetric emergency simulation program included identification of front-line care leaders, the
569 heterogeneity of the participants being evaluated, identification of institutional user-friendly IT that permitted timely completion of the MCQ component of the program, supported a video browser and was able to check OBANN website access. Finally, the availability of required personnel on the labor and delivery unit as determined by clinical workload and work schedules did impact the number of in-situ sessions taking place as well as skewed inter-rater reliability data.
4.2. High vs. low fidelity vs. hybrid simulation training Performance in simulation training varies from other forms of assessment, e.g., viva voce, written examination [17]. Possession of medical knowledge does not translate into being able to apply it correctly especially under stressful circumstances. Any complex medical simulation scenario will require use of several skill sets such as clinical psychomotor, problem solving, communication, and technical skills. Simulation medical education requires regular practice of the aforementioned skills and regular program review in order to develop a program that not only meets the educational needs of the LDU workforce but also makes a difference in obstetric patient outcomes and is ultimately data driven. We know that differences exist between low and high fidelity simulation training (Table 3) [18], with in-situ simulation presenting more logistic and cultural challenges than a high fidelity simulation environment [19]. However, did the hybrid nature of the program detract from the education and training received? Literature shows that there is very little identifiable difference between either low or high fidelity simulation and that equipment is not the only factor that matters. A welldesigned curriculum works well even in low equipment fidelity simulation [20]. In addition, the self-confidence assessment forms revealed significantly improved confidence behaviors among participants in whatever environment the simulation was performed. From a patient safety point of view, the insitu OBANN program also assisted with the identification of previously unrecognized operational patient latent safety threats (LST's) on the LDU, i.e., medication, equipment or system issues [21]. Some of the patient LST's identified on our LDU included, location of emergency drugs such as IV lipid emulsion and a 50mls vial 0.5% lidocaine for possible emergent cesarean section under a local anesthetic field block; location of documenter sheets for use during an obstetric emergency, equipment readiness, ie, suction with no tubing attached and location of a central alarm button for extra anesthesia assistance from main operating room front desk.
4.3. Recommendations for future obstetric team simulation program developers We do recommend revising assessment tools regularly to ensure that any future team simulation program is moving forward based on project data outcomes. This approach will make data relevant as well as help save time in not pursuing poor
570 practices. Leadership, ownership and departmental support from within each profession are also vital to ensure not only initial success but also durability of the program. As well as a motivational group effort, basic materials, extended assessment training and adequate IT support are required to ensure relevant data capture. Through feedback responses and observation, benefits of a hybrid obstetric emergency team simulation program included improved participant confidence, high participant satisfaction and familiarization taking place among the first-line responders to an obstetric emergency on the LDU, promotion of a non-threatening environment, interprofessional communication, identification of patient LST's and possible development of transferable skills and communication techniques [22]. In conclusion, our simulation program was incorporated into both a regular LDU workday and protected educational time. Faculty and nursing professionals of varying levels, some with limited clinical experience and low confidence, were able to participate in in-situ and high fidelity emergency obstetric scenarios before being faced with a real obstetric emergency. Our goal is to continue collecting data, revise evaluation tools and rater training and further refine the OBANN program to future accommodate neonatology and incorporate generic training, milestones so that these changes become a reality in a simulation world.
A.S. Bullough et al.
[6]
[7]
[8]
[9] [10]
[11]
[12]
[13]
[14]
[15]
[16]
Acknowledgements [17]
Alyssa Paratore RN and Krystal Koszola RN for their assistance with CTS rating and Mary Matimore for compiling data. [18]
References [19] [1] American College of Obstetricians and Gynecologists. Preparing for clinical emergencies in obstetrics and gynecology. Committee opinion no. 590. Obstet Gynecol 2014;123:722-5. [2] Maslovitz S, Barkai G, Lessing JB, Ziv A, Many A. Recurrent obstetric management mistakes identified by simulation. Obstet Gynecol 2007; 109:1295-300. [3] Institute of Medicine. Committee on quality of health Care in America. In: Kohn L, Corrigan J, Donaldson M, editors. To err is human: building a safer health system. Washington, DC: National Academy Press; 2000. p. 146. [4] The Joint Commission Sentinel Event Alert. Issue 30: preventing infant death and injury during delivery. 2004 [Available at:] http://www.jointcommission.org/assets/1/18/SEA_30.PDF. [5] American College of Obstetricians and Gynecologists. Professional responsibilities in obstetric-gynecologic medical education and training.
[20]
[21]
[22]
Committee opinion no. 500. Obstet Gynecol 2011;118:400-4 [Reaffirmed 2014]. Guise JM, Deering S, Kanki B, Osterweil P, Li H, Stanford S, et al. STORC OB safety initiative: development and validation of the clinical teamwork scale to evaluate teamwork. Simul Healthc 2008;3:217-23. Draycott T, Sibanda T, Owen L, Akande V, Winter C, Reading S, et al. Does training in obstetric emergencies improve neonatal outcome? BJOG 2006;113:177-82. Merien AE, van de Ven J, Mol BW, Houterman S, Sg O. Multidisciplinary team training in a simulation setting for acute obstetric emergencies: a systematic review. Obstet Gynecol 2010;115:1021-31. Croley WC, Rothenberg DM. Education of trainees in the intensive care unit. Crit Care Med 2007;35:S117-21 [suppl]. Johanson R, Cox C, O'Donnell E, Grady K, Howell C, Jones P. Managing obstetric emergencies and trauma (MOET): structured skills training using models and reality based scenarios. Obstet Gynecol 1999;1:46-52. Beasley JW, Damos JR, Roberts RG, Nesbitt TS. The advanced life support in obstetrics course: a national program to enhance obstetric emergency skill and to support maternity care practice. Arch Fam Med 1994;3:1037-41. Johnson R, Cox C, O'Donnell E, Grady K, Howell C, Jones P. PROMPT education and development: saving mothers' and babies' lives in resource poor settings. BJOG 2009;116:868-9. Beasley JW, Damos JR, Roberts RG, Nesbitt TS. The Society for Obstetric Anesthesia and Perinatology consensus statement on the Management of Cardiac Arrest in pregnancy. Anesth Analg 2014;118:1003-16. Clancy CM, Tornberg DN. Team STEPPS: integrating teamwork principles into healthcare practice. [Available at:] http://www.psqh.com/novdec06/ahrq.html. [Accessed June 2015] 2015. Williams JB, McDonough MA, Hilliard MW, Williams AL, Cuniowski PC, Gonzalez MG. Intermethod reliability of real-time versus delayed videotaped evaluation of a high-fidelity medical simulation septic shock scenario. Acad Emerg Med 2009;16:887-93. Riley W, Davis S, Miller K, Hansen H, Sainfort F, Sweet R. Didactic and simulation nontechnical skills team training to improve perinatal patient outcomes in a community hospital. Jt Comm J Qual Patient Saf 2011; 37:357-64. Savoldelli GL, Main VN, Joo SH, Houston PL, Graham M, Yee B, et al. Evaluation of patient simulator performance as an adjunct to the oral examination for senior anesthesia residents. Anesthesiology 2006;104: 475-81. Guise JM, Lowe NK, Deering S, Lewis PO, O'Haire C, Irwin LK, et al. Mobile in situ obstetric emergency simulation and teamwork training to improve maternal-fetal safety in hospitals. Jt Comm J Qual Patient Saf 2010;36:443-53. Patterson MD, Blike GT, Nadkarni VM. In-situ simulation: challenges and results. In Henriksen K, Battles, JB, Keyes, MA and Grady, ML ed. Advances in patient safety: new directions and alternative approaches (vol 3: performance and tools). Rockville MD: Agency for healthcare Research and Quality. 2008 [Available at: http://www.ahrq.gov/downloads/pub/advances2/vol3/advances-patterson_48.pdf]. Meurling L, Hedman L, Lidefelt KJ, Escher C, Felländer-Tsai L, Wallin CJ. Comparison of high- and low equipment fidelity during paediatric simulation team training: a case control study. BMC Med Educ 2014;14:221. Patterson MD, Geis GL, Falcone RA, LeMaster T, Wears RL. In situ simulation: detection of safety threats and teamwork training in a high risk emergency department. BMJ Qual Saf 2013;22:468-77. van der Nelson HA, Siasskos D, Bennett J, Godfrey M, Spray L, Draycott T, et al. Multiprofessional team simulation training on an obstetric model can improve teamwork in other areas of health care. Am J Med Qual 2014;29:78-82.