- Email: [email protected]
Identifying pitfalls in chest tube insertion: improving teaching and performance
S118
Abstracts
Use of an operative competency model with immediate feedback to evaluate resident technical progression William J Curtiss, MD, Vijay K Mittal, MD, FACS, FICS, Richard K Englehardt, MD, Armin Kamyab, MD, Karen McFarlane, MD, Yeon-Jeen Chang, MD Providence Hospital and Medical Centers, Southfield, MI INTRODUCTION: Evaluation of surgical-skills progression is a critical component of surgical education. We developed competency models to assess intra-operative comprehension, surgical-skills, quality of resident-faculty interaction. METHODS: Operation specific competency forms were developed; identifying surgical-milestones based on core-competencies of the ACGME, ABS and ACS. Survey distributed to residents and faculty assessing: case-performance, training percepitions, competency’s influence on case-percentage perfomed by resident. Feedback was analyzed, forms were re-designed and second-iteration of competencyforms were distributed. After six-months, questionnaire distributed comparing efficacy of revised- versus original-forms. RESULTS: Sixteen of 21 residents and 14/16 faculty responded. 75% of faculty, learning to operate was paramount; 79% felt residents learned most when operating. Faculty performed 25– 50%(N⫽4), 50–75%(N⫽4) or ⬎75%(N⫽6) of each operation. 63% of residents regarded technical training paramount, 94% thought they learned most when operating. Residents performed 25–50%(N⫽5), 50-75%(N⫽7) or ⬎75%(N⫽4) of operations. 81% of residents; 75% of faculty did not find OR-competency forms helpful in increasing resident case-percentage; 55% did believe the new forms better evaluated technical-progression. Revised forms were more concise, had defined and measurable goals, placed emphasis on evaluating resident contribution and evaluated resident implementation of the ACGME core-competencies. CONCLUSIONS: Assessment-tools for technical-skills are needed to evaluate resident-surgeons. Operative-competency forms help elucidate progression in mastery of surgical-technique. Operativecompetencies can track progression of resident-skill and preparation for independent-practice. Residents and attendings agree that primary mode of learning and technical competency improved via repetitive-operation with immediate-feedback. In the future, operative-competency forms will help document the resident progression and ultimately may effect the delineation of privileges once they enter practice.
The impact of heat stress on performance and cognitive function during simulated laparoscopic surgical tasks Regan James Berg, MD, Kenji Inaba, MD, FACS, FRCSC, Maura Sullivan, MSN, PhD, Obi Okoye, MD, MRCSI, Michael Minneti, BS, RRT, Pedro, G Teixeira, MD, Peep Talving, MD, PhD, FACS, Demetrios Demetriades, MD, PhD, FACS Los Angeles County and University of Southern California Medical Center, Los Angeles, CA INTRODUCTION: Raising ambient temperature to prevent intraoperative patient hypothermia remains widely advocated despite un-
J Am Coll Surg
convincing evidence of efficacy. Heat stress is associated with decreased cognitive and psychomotor performance across multiple tasks but has not been examined in a surgical context. We assessed the impact of increased ambient temperature on laparoscopic surgical performance and surgeon cognitive stress. METHODS: Twenty-one surgical residents participated in a within subjects, counter-balanced and randomized study protocol. Surgical performance was evaluated with versions of the peg-transfer and intracorporeal knot tying tasks from the MISTELS/FLS program. Participants trained to proficiency before enrollment. Task performance was measured at two ambient temperatures, 19 and 26 celsius. Participants were stratified by experience and randomLy counterbalanced to initial hot or cold exposure before crossing over to the alternate environment. Cognitive stress was measured using the validated Surgical Task Load Index (SURG-TLX). RESULTS: Task workload was assessed using the six bipolar scales of the Surgical-TLX. Significant differences between the hot and cold conditions were found in the areas of physical demands (hot 9.33, cold 6.14, p⫽0.009) and distractions (hot 9.28, cold 4.57, p⫽0.001). Differences trended towards significance in the area of situation stress (hot 9.57, cold 7.85, p⫽0.075). CONCLUSIONS: Increasing ambient temperature, to levels advocated for prevention of intraoperative hypothermia, does not significantly decrease technical performance in short surgical tasks. Surgeons do however, report increased perceptions of physical demand and distraction. The impact of these findings on performance and outcomes during longer operative procedures remains unclear.
Identifying pitfalls in chest tube insertion: improving teaching and performance James S Davis, MD, George D Garcia, MD, Mary M Wyckoff, PhD, Salman Alsafran, MD, Jill M Graygo, MPH, Kelly F Withum, BA, Carl I Schulman, MD, PhD, MSPH, FACS University of Miami Miller School of Medicine, Miami, FL INTRODUCTION: Tube thoracostomies are common surgical procedures, but little is known about how practitioners learn the skill. This study evaluates the easiest and most difficult steps for subjects to master regarding the procedure. METHODS: A mobile learning module was developed, containing stepwise multimedia guidance on chest tube insertion. A 14-item skills checklist, modeled after key steps in the module, was developed and tested. Participants, defined as ’novice’ (fewer than ten chest tubes placed) or ’expert’ (more than 10 placed), were assigned to either the video or control group. A trained clinician used the skills checklist to grade participants inserting a chest tube on the TraumaMan(r) simulator. RESULTS: Sixty-three subjects enrolled, of whom 51% (n⫽32) watched the video. Sixty-eight percent (n⫽43) were novices and 32% (n⫽20) were experts. Novices most frequently connected the tube to a pleurovac (90%), adequately dissected the soft tissue (81%), and scrubbed/anesthetized appropriately (76%). Novices least frequently performed intrapleural finger-sweeps for adhe-
Vol. 215, No. 3S, September 2012
sions (23%), controlled pleural puncture (42%), and avoided the neurovascular bundle (52%). With the video, their greatest improvements were finger-sweeps (49%, p⫽0.001), clamping the chest tube distally (34%, p⫽NS), and dissecting over the rib properly (25%, p⫽NS). Experts were most challenged with finger-sweeps (60%) and avoiding the neurovascular bundle (70%), but also improved most with finger-sweeps and clamping chest tubes distally (20%, p⫽NS). CONCLUSIONS: Avoiding the neurovascular bundle, controlled pleural entry, and finger-sweeps are most often performed incorrectly among novices. This information can help instructors emphasize key didactic steps, possibly easing trainees’ learning curve.
Does intra-operative teaching address the steps delineated in a cognitive task analysis (CTA)? Luise IM Pernar, MD, Katherine Corso, MPH, Elizabeth Breen, MD, FACS Brigham and Women’s Hospital, Boston, MA INTRODUCTION: Much teaching to surgical residents takes placed in the operating room (OR). The explicit content of what is taught in the operating room, however, has not previously been described. This study investigated the content of what is taught in the OR, specifically during laparoscopic cholecystectomies (LCs), for which a CTA, or an expert consensus of processes underlying procedures, was available. METHODS: A convenience sample of LCs was identified from the OR schedule over a 12-month period from February 2011 to February 2012. A checklist of necessary technical and decision-making steps to be executed during performance of LCs, anchored in a previously published CTA, was developed. Using this checklist, a trained observer recorded explicit teaching that occurred regarding these steps during each observed case. All observations were tallied and analyzed. RESULTS: 51 LCs were observed. 14 surgery attendings and 36 residents participated in the observed cases. Of 1042 observable teaching points, only 560 (53.7%) were observed during the study period. As a proportion of all observable steps, technical steps were observed more frequently, 377 (67.3%), than decision-making steps,183 (32.7%). CONCLUSIONS: Only half of all possible observable teaching steps were explicitly taught during LCs in this study. Technical steps were more frequently taught than decision-making steps. These findings may have important implications: a better understanding of the content of intra-operative teaching would allow educators to steer residents’ pre-operative preparation, modulate intra-operative instruction by members of the surgical faculty, and guide residents to the most appropriate teaching venues.
Construct validity of instrument vibrations as a measure of robotic surgical skill Ernest D Gomez, BS, BSE, Karlin Bark, PhD, Charlotte Rivera, William McMahan, MS, Austin Remington, David I Lee, MD,
Abstracts S119
Noel N Williams, MD, FRCS, Kenric M Murayama, MD, FACS, Kristoffel R Dumon, MD, Katherine J Kuchenbecker, PhD University of Pennsylvania, Philadelphia, PA INTRODUCTION: Surgical instrument vibrations physically correspond to how roughly the instruments are being handled, especially during contact with items in the surgical field. We have developed VerroTouch, a low cost system that measures and provides haptic feedback of robotic instrument vibrations. Measurement of these signals may provide an objective, quantitative metric for evaluating technical surgical skill. This study assesses the validity of such a metric. METHODS: Thirteen surgeons (6 experienced, 7 novice) used the da Vinci S surgical system to place a running stitch through holes in a rigid sheet of clear plastic. Each participant performed this task 4 times. Videos of the 52 resultant trials were rated in randomized order by four blinded, experienced minimallyinvasive surgeons using a modified combination of the Objective Structured Assessment of Technical Skill (OSATS) and the Global Evaluative Assessment of Robotic Skills (GEARS). Accelerometers attached to the robotic arms measured instrument vibrations, and a sensor beneath the box-trainer task board measured the applied forces. RESULTS: Compared to novices, experienced surgeons received significantly higher global skill ratings and performed the task with significantly lower instrument vibration magnitudes, lower forces, shorter completion times, and fewer instrument contacts (Table). Regression analysis showed that global skill ratings correlated with root-mean-square (RMS) vibration magnitudes (r2⫽0.29, p⬍ 0.0001), RMS force magnitudes (r2⫽0.23, p⬍0.0005), completion times (r2⫽0.74, p⬍0.0001), and instrument contacts (r2⫽0.34, p⬍0.0001).
Metric
OSATS global skill rating GEARS global skill rating Combined OSATS/ GEARS global skill rating RMS instrument vibrations (m/s2) RMS Forces (N) Completion time (sec) Number of contacts
Novice surgeons (nⴝ7)
Experienced robotic surgeons (nⴝ6)
p Value
0.56 ⫾ 0.13
0.79 ⫾ 0.11
⬍0.00001
0.56 ⫾ 0.13
0.79 ⫾ 0.13
⬍0.00001
0.56 ⫾ 0.13
0.79 ⫾ 0.12
⬍0.00001
1.96 ⫾ 0.42 3.91 ⫾ 1.34
1.60 ⫾ 0.22 2.30 ⫾ 1.37
⬍0.0005 ⬍0.0005
116.1 ⫾ 36.1 83.9 ⫾ 28.2
74.5 ⫾ 27.7 48.9 ⫾ 12.8
⬍0.00001 ⬍0.00001
CONCLUSIONS: Instrument vibrations are a construct-valid measure of technical surgical skill in a robotic surgical training task.
Abstracts
Use of an operative competency model with immediate feedback to evaluate resident technical progression William J Curtiss, MD, Vijay K Mittal, MD, FACS, FICS, Richard K Englehardt, MD, Armin Kamyab, MD, Karen McFarlane, MD, Yeon-Jeen Chang, MD Providence Hospital and Medical Centers, Southfield, MI INTRODUCTION: Evaluation of surgical-skills progression is a critical component of surgical education. We developed competency models to assess intra-operative comprehension, surgical-skills, quality of resident-faculty interaction. METHODS: Operation specific competency forms were developed; identifying surgical-milestones based on core-competencies of the ACGME, ABS and ACS. Survey distributed to residents and faculty assessing: case-performance, training percepitions, competency’s influence on case-percentage perfomed by resident. Feedback was analyzed, forms were re-designed and second-iteration of competencyforms were distributed. After six-months, questionnaire distributed comparing efficacy of revised- versus original-forms. RESULTS: Sixteen of 21 residents and 14/16 faculty responded. 75% of faculty, learning to operate was paramount; 79% felt residents learned most when operating. Faculty performed 25– 50%(N⫽4), 50–75%(N⫽4) or ⬎75%(N⫽6) of each operation. 63% of residents regarded technical training paramount, 94% thought they learned most when operating. Residents performed 25–50%(N⫽5), 50-75%(N⫽7) or ⬎75%(N⫽4) of operations. 81% of residents; 75% of faculty did not find OR-competency forms helpful in increasing resident case-percentage; 55% did believe the new forms better evaluated technical-progression. Revised forms were more concise, had defined and measurable goals, placed emphasis on evaluating resident contribution and evaluated resident implementation of the ACGME core-competencies. CONCLUSIONS: Assessment-tools for technical-skills are needed to evaluate resident-surgeons. Operative-competency forms help elucidate progression in mastery of surgical-technique. Operativecompetencies can track progression of resident-skill and preparation for independent-practice. Residents and attendings agree that primary mode of learning and technical competency improved via repetitive-operation with immediate-feedback. In the future, operative-competency forms will help document the resident progression and ultimately may effect the delineation of privileges once they enter practice.
The impact of heat stress on performance and cognitive function during simulated laparoscopic surgical tasks Regan James Berg, MD, Kenji Inaba, MD, FACS, FRCSC, Maura Sullivan, MSN, PhD, Obi Okoye, MD, MRCSI, Michael Minneti, BS, RRT, Pedro, G Teixeira, MD, Peep Talving, MD, PhD, FACS, Demetrios Demetriades, MD, PhD, FACS Los Angeles County and University of Southern California Medical Center, Los Angeles, CA INTRODUCTION: Raising ambient temperature to prevent intraoperative patient hypothermia remains widely advocated despite un-
J Am Coll Surg
convincing evidence of efficacy. Heat stress is associated with decreased cognitive and psychomotor performance across multiple tasks but has not been examined in a surgical context. We assessed the impact of increased ambient temperature on laparoscopic surgical performance and surgeon cognitive stress. METHODS: Twenty-one surgical residents participated in a within subjects, counter-balanced and randomized study protocol. Surgical performance was evaluated with versions of the peg-transfer and intracorporeal knot tying tasks from the MISTELS/FLS program. Participants trained to proficiency before enrollment. Task performance was measured at two ambient temperatures, 19 and 26 celsius. Participants were stratified by experience and randomLy counterbalanced to initial hot or cold exposure before crossing over to the alternate environment. Cognitive stress was measured using the validated Surgical Task Load Index (SURG-TLX). RESULTS: Task workload was assessed using the six bipolar scales of the Surgical-TLX. Significant differences between the hot and cold conditions were found in the areas of physical demands (hot 9.33, cold 6.14, p⫽0.009) and distractions (hot 9.28, cold 4.57, p⫽0.001). Differences trended towards significance in the area of situation stress (hot 9.57, cold 7.85, p⫽0.075). CONCLUSIONS: Increasing ambient temperature, to levels advocated for prevention of intraoperative hypothermia, does not significantly decrease technical performance in short surgical tasks. Surgeons do however, report increased perceptions of physical demand and distraction. The impact of these findings on performance and outcomes during longer operative procedures remains unclear.
Identifying pitfalls in chest tube insertion: improving teaching and performance James S Davis, MD, George D Garcia, MD, Mary M Wyckoff, PhD, Salman Alsafran, MD, Jill M Graygo, MPH, Kelly F Withum, BA, Carl I Schulman, MD, PhD, MSPH, FACS University of Miami Miller School of Medicine, Miami, FL INTRODUCTION: Tube thoracostomies are common surgical procedures, but little is known about how practitioners learn the skill. This study evaluates the easiest and most difficult steps for subjects to master regarding the procedure. METHODS: A mobile learning module was developed, containing stepwise multimedia guidance on chest tube insertion. A 14-item skills checklist, modeled after key steps in the module, was developed and tested. Participants, defined as ’novice’ (fewer than ten chest tubes placed) or ’expert’ (more than 10 placed), were assigned to either the video or control group. A trained clinician used the skills checklist to grade participants inserting a chest tube on the TraumaMan(r) simulator. RESULTS: Sixty-three subjects enrolled, of whom 51% (n⫽32) watched the video. Sixty-eight percent (n⫽43) were novices and 32% (n⫽20) were experts. Novices most frequently connected the tube to a pleurovac (90%), adequately dissected the soft tissue (81%), and scrubbed/anesthetized appropriately (76%). Novices least frequently performed intrapleural finger-sweeps for adhe-
Vol. 215, No. 3S, September 2012
sions (23%), controlled pleural puncture (42%), and avoided the neurovascular bundle (52%). With the video, their greatest improvements were finger-sweeps (49%, p⫽0.001), clamping the chest tube distally (34%, p⫽NS), and dissecting over the rib properly (25%, p⫽NS). Experts were most challenged with finger-sweeps (60%) and avoiding the neurovascular bundle (70%), but also improved most with finger-sweeps and clamping chest tubes distally (20%, p⫽NS). CONCLUSIONS: Avoiding the neurovascular bundle, controlled pleural entry, and finger-sweeps are most often performed incorrectly among novices. This information can help instructors emphasize key didactic steps, possibly easing trainees’ learning curve.
Does intra-operative teaching address the steps delineated in a cognitive task analysis (CTA)? Luise IM Pernar, MD, Katherine Corso, MPH, Elizabeth Breen, MD, FACS Brigham and Women’s Hospital, Boston, MA INTRODUCTION: Much teaching to surgical residents takes placed in the operating room (OR). The explicit content of what is taught in the operating room, however, has not previously been described. This study investigated the content of what is taught in the OR, specifically during laparoscopic cholecystectomies (LCs), for which a CTA, or an expert consensus of processes underlying procedures, was available. METHODS: A convenience sample of LCs was identified from the OR schedule over a 12-month period from February 2011 to February 2012. A checklist of necessary technical and decision-making steps to be executed during performance of LCs, anchored in a previously published CTA, was developed. Using this checklist, a trained observer recorded explicit teaching that occurred regarding these steps during each observed case. All observations were tallied and analyzed. RESULTS: 51 LCs were observed. 14 surgery attendings and 36 residents participated in the observed cases. Of 1042 observable teaching points, only 560 (53.7%) were observed during the study period. As a proportion of all observable steps, technical steps were observed more frequently, 377 (67.3%), than decision-making steps,183 (32.7%). CONCLUSIONS: Only half of all possible observable teaching steps were explicitly taught during LCs in this study. Technical steps were more frequently taught than decision-making steps. These findings may have important implications: a better understanding of the content of intra-operative teaching would allow educators to steer residents’ pre-operative preparation, modulate intra-operative instruction by members of the surgical faculty, and guide residents to the most appropriate teaching venues.
Construct validity of instrument vibrations as a measure of robotic surgical skill Ernest D Gomez, BS, BSE, Karlin Bark, PhD, Charlotte Rivera, William McMahan, MS, Austin Remington, David I Lee, MD,
Abstracts S119
Noel N Williams, MD, FRCS, Kenric M Murayama, MD, FACS, Kristoffel R Dumon, MD, Katherine J Kuchenbecker, PhD University of Pennsylvania, Philadelphia, PA INTRODUCTION: Surgical instrument vibrations physically correspond to how roughly the instruments are being handled, especially during contact with items in the surgical field. We have developed VerroTouch, a low cost system that measures and provides haptic feedback of robotic instrument vibrations. Measurement of these signals may provide an objective, quantitative metric for evaluating technical surgical skill. This study assesses the validity of such a metric. METHODS: Thirteen surgeons (6 experienced, 7 novice) used the da Vinci S surgical system to place a running stitch through holes in a rigid sheet of clear plastic. Each participant performed this task 4 times. Videos of the 52 resultant trials were rated in randomized order by four blinded, experienced minimallyinvasive surgeons using a modified combination of the Objective Structured Assessment of Technical Skill (OSATS) and the Global Evaluative Assessment of Robotic Skills (GEARS). Accelerometers attached to the robotic arms measured instrument vibrations, and a sensor beneath the box-trainer task board measured the applied forces. RESULTS: Compared to novices, experienced surgeons received significantly higher global skill ratings and performed the task with significantly lower instrument vibration magnitudes, lower forces, shorter completion times, and fewer instrument contacts (Table). Regression analysis showed that global skill ratings correlated with root-mean-square (RMS) vibration magnitudes (r2⫽0.29, p⬍ 0.0001), RMS force magnitudes (r2⫽0.23, p⬍0.0005), completion times (r2⫽0.74, p⬍0.0001), and instrument contacts (r2⫽0.34, p⬍0.0001).
Metric
OSATS global skill rating GEARS global skill rating Combined OSATS/ GEARS global skill rating RMS instrument vibrations (m/s2) RMS Forces (N) Completion time (sec) Number of contacts
Novice surgeons (nⴝ7)
Experienced robotic surgeons (nⴝ6)
p Value
0.56 ⫾ 0.13
0.79 ⫾ 0.11
⬍0.00001
0.56 ⫾ 0.13
0.79 ⫾ 0.13
⬍0.00001
0.56 ⫾ 0.13
0.79 ⫾ 0.12
⬍0.00001
1.96 ⫾ 0.42 3.91 ⫾ 1.34
1.60 ⫾ 0.22 2.30 ⫾ 1.37
⬍0.0005 ⬍0.0005
116.1 ⫾ 36.1 83.9 ⫾ 28.2
74.5 ⫾ 27.7 48.9 ⫾ 12.8
⬍0.00001 ⬍0.00001
CONCLUSIONS: Instrument vibrations are a construct-valid measure of technical surgical skill in a robotic surgical training task.