- Email: [email protected]
Assessment and reliability of general surgery residents’ intraoperative communication skills
EDUCATION paradigm (blocked [BLK] and random [RND] practice schedules) was used to assess behavioral performance and brain activation during the retention and transfer of simulated laparoscopic surgical tasks.
Assessment and reliability of general surgery residents’ intraoperative communication skills Aimee K Gardner, PhD, Michael A Russo, MD, Ibrahim I Jabbour, MD, MPH, Matthew Kosemund, Deborah Hogg, Daniel J Scott, MD, FACS University of Texas Southwestern Medical Center, Dallas, TX
METHODS: Third-year medical students (n¼11) were randomized to either a BLK or RND practice schedule. Using a LapSim virtual reality (VR) simulator, across three days, students performed 108 acquisition trials of 3 VR laparoscopic tasks (36 trials per task). Seventy-two hours following skill acquisition, 6 retention and 6 transfer trials (using 2 additional VR laparoscopic tasks) were performed. Linear mixed models were used to assess practice schedule and task effects across retention and transfer for the behavioral and hemodynamic oxygenation (right and left PFCs) variables.
INTRODUCTION: A recent needs assessment of ACS-Accredited Education Institutes identified communication with team members as an area of great need among residency training programs. However, there is a paucity of data regarding appropriate evaluation of residents’ communication skills. Accurate assessment of resident communication skills requires valid and reliable instruments, as well as rater training. The purpose of this study was to compare expert ratings to resident self-assessments of intraoperative communication skills and identify specific areas of need to inform curricular efforts.
RESULTS: During retention, RND performed the VR surgical tasks in significantly less time and had lower oxygenation in the right and left PFCs than the BLK group.
METHODS: Surgery faculty and instructors (MD, PhD, Research Fellow, Simulation Technician) underwent a two-hour frame-ofreference training session with the Operating Room Communication (ORCA) instrument. They then independently rated communication skills of 14 PGY1s who participated in a simulated scenario in which residents had to manage an unexpected intraoperative crisis with a confederate team. Residents also rated their own communication skills with the ORCA instrument via videoreview of the scenario. Interrater reliability of trained raters was assessed via intraclass correlation coefficients. Comparisons among trained raters and residents were assessed with Pearson correlation coefficients.
CONCLUSIONS: Our results provide preliminary evidence of fNIR measures of anterior PFC hemodynamic response and its relationship to the performance of surgical laparoscopic tasks in a VR environment. RND practice facilitates memory by better performance and lower neural demands during retention of simulated laparoscopic surgical tasks. Using fNIR as an objective neurophysiological marker in the evaluation of skill acquisition provides important information regarding learning processes that have implications for designing instructional training programs to meet the needs of 21st century educational environments. Examining the feasibility and predictive validity of the SAGAT tool to assess situational awareness among surgical trainees Aimee K Gardner, PhD, Matthew Kosemund, Joseph Martinez, MD University of Texas Southwestern Medical Center, Dallas, TX
RESULTS: Expert interreliability after frame-of-reference training was 0.91. However, the correlation between expert and resident evaluations was non-significant. Further t-tests revealed that residents significantly underestimated their intraoperative communication skills (p<0.05). Use of names, closed loop communication, addressing issues without prompting, and sharing information with team members demonstrated consistently low ratings among all residents.
INTRODUCTION: Situational awareness (SA) describes a team’s ability to perceive environmental elements, comprehend their meaning, and to anticipate future events. Although SA is consistently described as a critical competency among surgical teams, there is a dearth of research identifying efficacious methods to assess and develop SA in such settings. The aim of this study was to investigate the feasibility of implementing an objective tool that has been used to measure SA in other intense and dynamic environments -the Situation Awareness Global Assessment Technique (SAGAT) -and to examine its ability to predict surgical trainee team performance.
CONCLUSIONS: These findings reveal that a number of individuals can be trained to reliably rate resident intraoperative communication performance, and that residents tend to underrate their communication skills. Educational neuroscience with fNIR and simulation training of surgical tasks Patricia A Shewokis, PhD, Faiz U Shariff, MD, Hasan Ayaz, PhD, Lawrence Greenawald, MD, Mashaal Syed, Yichuan Liu, Jorge L Uribe, MD, Lucian Panait, MD, FACS, Andres E Castellanos, MD, FACS, David S Lind, MD, FACS Drexel University, Philadelphia, PA
METHODS: Ten team training sessions were conducted involving two standardized high fidelity trauma simulation scenarios. Teams consisted of four or five participants, and roles were randomly assigned. Team situational awareness was assessed using the SAGAT method, which involves intermittent freezes to probe trainee awareness of the situation. Team performance was assessed using the Mayo High Performance Teamwork Scale. Hierarchical regression was used to examine SA-performance relationships for each scenario.
INTRODUCTION: Using functional near infrared spectroscopy (fNIR), a noninvasive and emergent optical imaging tool, we monitored prefrontal cortex (PFC) activation during the learning of simulated laparoscopic surgical tasks. A contextual interference
ª 2015 by the American College of Surgeons Published by Elsevier Inc.
e9
http://dx.doi.org/10.1016/j.jamcollsurg.2015.08.418 ISSN 1072-7515/15
Assessment and reliability of general surgery residents’ intraoperative communication skills Aimee K Gardner, PhD, Michael A Russo, MD, Ibrahim I Jabbour, MD, MPH, Matthew Kosemund, Deborah Hogg, Daniel J Scott, MD, FACS University of Texas Southwestern Medical Center, Dallas, TX
METHODS: Third-year medical students (n¼11) were randomized to either a BLK or RND practice schedule. Using a LapSim virtual reality (VR) simulator, across three days, students performed 108 acquisition trials of 3 VR laparoscopic tasks (36 trials per task). Seventy-two hours following skill acquisition, 6 retention and 6 transfer trials (using 2 additional VR laparoscopic tasks) were performed. Linear mixed models were used to assess practice schedule and task effects across retention and transfer for the behavioral and hemodynamic oxygenation (right and left PFCs) variables.
INTRODUCTION: A recent needs assessment of ACS-Accredited Education Institutes identified communication with team members as an area of great need among residency training programs. However, there is a paucity of data regarding appropriate evaluation of residents’ communication skills. Accurate assessment of resident communication skills requires valid and reliable instruments, as well as rater training. The purpose of this study was to compare expert ratings to resident self-assessments of intraoperative communication skills and identify specific areas of need to inform curricular efforts.
RESULTS: During retention, RND performed the VR surgical tasks in significantly less time and had lower oxygenation in the right and left PFCs than the BLK group.
METHODS: Surgery faculty and instructors (MD, PhD, Research Fellow, Simulation Technician) underwent a two-hour frame-ofreference training session with the Operating Room Communication (ORCA) instrument. They then independently rated communication skills of 14 PGY1s who participated in a simulated scenario in which residents had to manage an unexpected intraoperative crisis with a confederate team. Residents also rated their own communication skills with the ORCA instrument via videoreview of the scenario. Interrater reliability of trained raters was assessed via intraclass correlation coefficients. Comparisons among trained raters and residents were assessed with Pearson correlation coefficients.
CONCLUSIONS: Our results provide preliminary evidence of fNIR measures of anterior PFC hemodynamic response and its relationship to the performance of surgical laparoscopic tasks in a VR environment. RND practice facilitates memory by better performance and lower neural demands during retention of simulated laparoscopic surgical tasks. Using fNIR as an objective neurophysiological marker in the evaluation of skill acquisition provides important information regarding learning processes that have implications for designing instructional training programs to meet the needs of 21st century educational environments. Examining the feasibility and predictive validity of the SAGAT tool to assess situational awareness among surgical trainees Aimee K Gardner, PhD, Matthew Kosemund, Joseph Martinez, MD University of Texas Southwestern Medical Center, Dallas, TX
RESULTS: Expert interreliability after frame-of-reference training was 0.91. However, the correlation between expert and resident evaluations was non-significant. Further t-tests revealed that residents significantly underestimated their intraoperative communication skills (p<0.05). Use of names, closed loop communication, addressing issues without prompting, and sharing information with team members demonstrated consistently low ratings among all residents.
INTRODUCTION: Situational awareness (SA) describes a team’s ability to perceive environmental elements, comprehend their meaning, and to anticipate future events. Although SA is consistently described as a critical competency among surgical teams, there is a dearth of research identifying efficacious methods to assess and develop SA in such settings. The aim of this study was to investigate the feasibility of implementing an objective tool that has been used to measure SA in other intense and dynamic environments -the Situation Awareness Global Assessment Technique (SAGAT) -and to examine its ability to predict surgical trainee team performance.
CONCLUSIONS: These findings reveal that a number of individuals can be trained to reliably rate resident intraoperative communication performance, and that residents tend to underrate their communication skills. Educational neuroscience with fNIR and simulation training of surgical tasks Patricia A Shewokis, PhD, Faiz U Shariff, MD, Hasan Ayaz, PhD, Lawrence Greenawald, MD, Mashaal Syed, Yichuan Liu, Jorge L Uribe, MD, Lucian Panait, MD, FACS, Andres E Castellanos, MD, FACS, David S Lind, MD, FACS Drexel University, Philadelphia, PA
METHODS: Ten team training sessions were conducted involving two standardized high fidelity trauma simulation scenarios. Teams consisted of four or five participants, and roles were randomly assigned. Team situational awareness was assessed using the SAGAT method, which involves intermittent freezes to probe trainee awareness of the situation. Team performance was assessed using the Mayo High Performance Teamwork Scale. Hierarchical regression was used to examine SA-performance relationships for each scenario.
INTRODUCTION: Using functional near infrared spectroscopy (fNIR), a noninvasive and emergent optical imaging tool, we monitored prefrontal cortex (PFC) activation during the learning of simulated laparoscopic surgical tasks. A contextual interference
ª 2015 by the American College of Surgeons Published by Elsevier Inc.
e9
http://dx.doi.org/10.1016/j.jamcollsurg.2015.08.418 ISSN 1072-7515/15