- Email: [email protected]
2-Year Skill Retention and Certification Exam Performance Following FLS Training and Proficiency Maintenance
316
ASSOCIATION FOR ACADEMIC SURGERY AND SOCIETY OF UNIVERSITY SURGEONS—ABSTRACTS
a retrospective study of 16 R01s from July 2008 through June 2009. These R01s engaged in 16 weekly sessions of surgical skills training (July -October) which included 4 FLS tasks (peg transfer, pattern cut, suturing with extra-corporeal knot-tying, and suturing with intra-corporeal knot-tying), laparoscopic simulator tasks, and open surgical skill tasks. FLS skill tasks were practiced weekly with some feedback but no structured instruction. The R01s were tested PRE (early July), POST (late October) after the training sessions were completed, and DELAY (June, at 7 to 8 months after training). The performance outcome measure was task completion time (TCT). A TCT decrease implies improved performance. Clinical involvement with laparoscopic cases by these R01s was equivalent and minimal during this period. Results: Improvement was seen in POST TCTs compared to PRE TCTs in all R01s for all FLS tasks (p < 0.05). DELAY TCTs were significantly lower than PRE TCTs for all tasks except extra-corporeal knot-tying (p < 0.05). When comparing the DELAY TCTs of individual learners to their POST TCTs, the majority of times increased (9/16 for peg transfer, 12/16 for pattern cut, 16/16 for extra-corporeal knot-tying, and 14/16 for intra-corporeal knottying) implying a fall in skill but the DELAY TCTs were equal to or lower than their POST TCTs in the remaining learners implying a fully retained or increased skill. Conclusions: This is the most extensive report of FLS skill retention available. Our results suggest that FLS training provided effective learning in R01s resulting in considerable skill retention at 7-8 months post training. It is probable that refresher FLS training sessions for R01s would result in even higher performance levels.
38.2. 2-Year Skill Retention and Certification Exam Performance Following FLS Training and Proficiency Maintenance. L. B. Mashaud, L. A. Hollett, A. O. Castellvi, D. C. Hogg, S. T. Tesfay, D. J. Scott; UT Southwestern Medical Center, Dallas, TX Introduction: The Fundamentals of Laparoscopic Surgery (FLS) program has been extensively validated. The purpose of this study was to determine 2-year performance retention and certification exam pass rate after completion of a proficiency-based FLS skills curriculum and subsequent interval training. Methods: Surgery residents (PGY 1-5, n ¼ 91) were enrolled in an IRB-approved protocol. All participants initially underwent proficiency-based training during a 2-month period on all 5 FLS tasks according to a previously published curriculum, which included orientation, pre-test, self-training with video review, and post-test. Subsequently, available residents (PGY3-5, n ¼ 33) were enrolled in a follow-up curriculum focusing on the two most complex tasks (Suturing with Extracorporeal (Task 4) and Intracorporeal (Task 5) Knot-tying). The follow-up curriculum included a retention test on both tasks, followed by mandatory re-training to proficiency if the proficiency levels were not achieved. Retention tests and re-training were scheduled at 6-month intervals for 2 years, and the final retention test (Retention 4) included the actual FLS Certification Examination for PGY4-5 (n ¼ 20) trainees (all 5 tasks þ written exam). All tests included 1 repetition of each task under direct proctor supervision and were scored using standard FLS time and error-based metrics. Comparisons were by ANOVA; mean 6 s.d. reported. Results: For 33 residents, a 96% participation rate was achieved for all curricular components
over the 2-year study period. Retention tests were completed at 6.3 6 1.0, 12.3 6 1.7, 18 6 2.3, and 23.3 6 2.1 months after initial training. For Task 4, re-training was required for 64%, 50%, and 47% of participants at Retention 1, 2 and 3 tests (3.2 6 1.3, 2.9 6 2.2 and 2.8 6 1.8 repetitions required), respectively. For Task 5, re-training was required for 66%, 53% and 44% of participants at Retention 1, 2 and 3 tests (18.9 6 5.7, 15.5 6 7.5, and 14.9 6 2.8 repetitions required), respectively. Performance scores for both tasks are listed below. For both tasks, scores significantly improved after initial training and significantly decreased by Retention 1. However, compared to Post-test, no significant difference was detected for Retention 2-4, as recovery in performance was observed. Compared to Post-test, skill retention at Retention 1-4 was 83%, 94%, 98%, and 91% for Task 4 and 85%, 95%, 96%, and 100% for Task 5, respectively. All PGY4-5 residents passed the FLS Certification Examination, achieving 413 6 28 total score on the skills portion (minimum passing score ¼ 270) and demonstrating 91.8% retention for all 5 tasks. Conclusions: Despite a significant early decrement in performance following training, performance seems to improve and recover in a relatively steady fashion with subsequent ongoing training. This curriculum results in a very high level of skill retention at 2 years and uniformly allows trainees to pass the FLS Certification Examination with a wide comfort margin.
38.3. Non-Operative Management of Solid Organ Injury Diminishes Surgical Resident Operative Experience: Is It Time for Simulation Training?. J. G. Bittner, IV, M. L. Hawkins, R. S. Medeiros, J. S. Beatty, L. R. Atteberry, C. H. Ferdinand, J. D. Mellinger; Department of Surgery, Medical College of Georgia School of Medicine, Augusta, GA Management of traumatic spleen and liver injuries is increasingly non-operative; however, operative experience with these injuries remains an essential skill for general surgery residency graduates. Consequently, training programs are challenged to ensure essential operative trauma experience. Therefore, we examined the impact of non-operative management on resident operative experience with SAOI to determine if curriculum-based simulation training might be necessary to augment clinical experience. Methods: A retrospective cohort analysis was performed on 1,198 consecutive adults admitted to a Level I trauma center over 12 years diagnosed with spleen and/or liver injury (Cohort A: 1996-2001 vs. Cohort B: 2002-2007). Resident data obtained from operative logs (Past: 1996-2001 vs. Recent: 2002-2007) are presented as mean 6 standard error. Cohorts were compared using unpaired Student’s t-test, twotailed Fischer’s exact test, and one-way ANOVA with Bonferroni post-hoc analysis where appropriate (alpha ¼ 0.05). Results: Overall, 24% of patients underwent operation for SAOI (Table 1). Fewer blunt than penetrating injuries required operation (19.9% vs. 49.7%, P < 0.001). Of those managed operatively, 69.5% underwent a spleen procedure (91% total splenectomy) and 42.5% had a liver procedure (95% primary laceration closure). More patients in Cohort A received an operation for SAOI compared to Cohort B (33.7% vs. 16.4%, P < 0.001); however, patient outcomes did not vary between cohorts. Over the study period, 55 residency graduates logged 27 6 1 (range 12-51) operative trauma cases. Graduates performed 3.4 6 0.3 (range 1-10) spleen procedures and 2.4 6 0.2 (range 0-8) liver operations for trauma. The Past graduates recorded more
ASSOCIATION FOR ACADEMIC SURGERY AND SOCIETY OF UNIVERSITY SURGEONS—ABSTRACTS
a retrospective study of 16 R01s from July 2008 through June 2009. These R01s engaged in 16 weekly sessions of surgical skills training (July -October) which included 4 FLS tasks (peg transfer, pattern cut, suturing with extra-corporeal knot-tying, and suturing with intra-corporeal knot-tying), laparoscopic simulator tasks, and open surgical skill tasks. FLS skill tasks were practiced weekly with some feedback but no structured instruction. The R01s were tested PRE (early July), POST (late October) after the training sessions were completed, and DELAY (June, at 7 to 8 months after training). The performance outcome measure was task completion time (TCT). A TCT decrease implies improved performance. Clinical involvement with laparoscopic cases by these R01s was equivalent and minimal during this period. Results: Improvement was seen in POST TCTs compared to PRE TCTs in all R01s for all FLS tasks (p < 0.05). DELAY TCTs were significantly lower than PRE TCTs for all tasks except extra-corporeal knot-tying (p < 0.05). When comparing the DELAY TCTs of individual learners to their POST TCTs, the majority of times increased (9/16 for peg transfer, 12/16 for pattern cut, 16/16 for extra-corporeal knot-tying, and 14/16 for intra-corporeal knottying) implying a fall in skill but the DELAY TCTs were equal to or lower than their POST TCTs in the remaining learners implying a fully retained or increased skill. Conclusions: This is the most extensive report of FLS skill retention available. Our results suggest that FLS training provided effective learning in R01s resulting in considerable skill retention at 7-8 months post training. It is probable that refresher FLS training sessions for R01s would result in even higher performance levels.
38.2. 2-Year Skill Retention and Certification Exam Performance Following FLS Training and Proficiency Maintenance. L. B. Mashaud, L. A. Hollett, A. O. Castellvi, D. C. Hogg, S. T. Tesfay, D. J. Scott; UT Southwestern Medical Center, Dallas, TX Introduction: The Fundamentals of Laparoscopic Surgery (FLS) program has been extensively validated. The purpose of this study was to determine 2-year performance retention and certification exam pass rate after completion of a proficiency-based FLS skills curriculum and subsequent interval training. Methods: Surgery residents (PGY 1-5, n ¼ 91) were enrolled in an IRB-approved protocol. All participants initially underwent proficiency-based training during a 2-month period on all 5 FLS tasks according to a previously published curriculum, which included orientation, pre-test, self-training with video review, and post-test. Subsequently, available residents (PGY3-5, n ¼ 33) were enrolled in a follow-up curriculum focusing on the two most complex tasks (Suturing with Extracorporeal (Task 4) and Intracorporeal (Task 5) Knot-tying). The follow-up curriculum included a retention test on both tasks, followed by mandatory re-training to proficiency if the proficiency levels were not achieved. Retention tests and re-training were scheduled at 6-month intervals for 2 years, and the final retention test (Retention 4) included the actual FLS Certification Examination for PGY4-5 (n ¼ 20) trainees (all 5 tasks þ written exam). All tests included 1 repetition of each task under direct proctor supervision and were scored using standard FLS time and error-based metrics. Comparisons were by ANOVA; mean 6 s.d. reported. Results: For 33 residents, a 96% participation rate was achieved for all curricular components
over the 2-year study period. Retention tests were completed at 6.3 6 1.0, 12.3 6 1.7, 18 6 2.3, and 23.3 6 2.1 months after initial training. For Task 4, re-training was required for 64%, 50%, and 47% of participants at Retention 1, 2 and 3 tests (3.2 6 1.3, 2.9 6 2.2 and 2.8 6 1.8 repetitions required), respectively. For Task 5, re-training was required for 66%, 53% and 44% of participants at Retention 1, 2 and 3 tests (18.9 6 5.7, 15.5 6 7.5, and 14.9 6 2.8 repetitions required), respectively. Performance scores for both tasks are listed below. For both tasks, scores significantly improved after initial training and significantly decreased by Retention 1. However, compared to Post-test, no significant difference was detected for Retention 2-4, as recovery in performance was observed. Compared to Post-test, skill retention at Retention 1-4 was 83%, 94%, 98%, and 91% for Task 4 and 85%, 95%, 96%, and 100% for Task 5, respectively. All PGY4-5 residents passed the FLS Certification Examination, achieving 413 6 28 total score on the skills portion (minimum passing score ¼ 270) and demonstrating 91.8% retention for all 5 tasks. Conclusions: Despite a significant early decrement in performance following training, performance seems to improve and recover in a relatively steady fashion with subsequent ongoing training. This curriculum results in a very high level of skill retention at 2 years and uniformly allows trainees to pass the FLS Certification Examination with a wide comfort margin.
38.3. Non-Operative Management of Solid Organ Injury Diminishes Surgical Resident Operative Experience: Is It Time for Simulation Training?. J. G. Bittner, IV, M. L. Hawkins, R. S. Medeiros, J. S. Beatty, L. R. Atteberry, C. H. Ferdinand, J. D. Mellinger; Department of Surgery, Medical College of Georgia School of Medicine, Augusta, GA Management of traumatic spleen and liver injuries is increasingly non-operative; however, operative experience with these injuries remains an essential skill for general surgery residency graduates. Consequently, training programs are challenged to ensure essential operative trauma experience. Therefore, we examined the impact of non-operative management on resident operative experience with SAOI to determine if curriculum-based simulation training might be necessary to augment clinical experience. Methods: A retrospective cohort analysis was performed on 1,198 consecutive adults admitted to a Level I trauma center over 12 years diagnosed with spleen and/or liver injury (Cohort A: 1996-2001 vs. Cohort B: 2002-2007). Resident data obtained from operative logs (Past: 1996-2001 vs. Recent: 2002-2007) are presented as mean 6 standard error. Cohorts were compared using unpaired Student’s t-test, twotailed Fischer’s exact test, and one-way ANOVA with Bonferroni post-hoc analysis where appropriate (alpha ¼ 0.05). Results: Overall, 24% of patients underwent operation for SAOI (Table 1). Fewer blunt than penetrating injuries required operation (19.9% vs. 49.7%, P < 0.001). Of those managed operatively, 69.5% underwent a spleen procedure (91% total splenectomy) and 42.5% had a liver procedure (95% primary laceration closure). More patients in Cohort A received an operation for SAOI compared to Cohort B (33.7% vs. 16.4%, P < 0.001); however, patient outcomes did not vary between cohorts. Over the study period, 55 residency graduates logged 27 6 1 (range 12-51) operative trauma cases. Graduates performed 3.4 6 0.3 (range 1-10) spleen procedures and 2.4 6 0.2 (range 0-8) liver operations for trauma. The Past graduates recorded more