A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance

For hand-sewn anastomoses, there was a trend toward decreasing completion times as the postgraduate level increased with PGY1 time 64 ⫾ 7.2 minutes versus PGY5 and faculty 38 ⫾ 6 minutes. On the other hand, the bursting pressures for the hand-sewn anastomoses varied from 28 to 117 mm Hg without any difference in the groups to reflect experience. Conclusions: Hand-sewn and stapled inanimate intestinal anastomoses are a valid simulation of intraoperative experience based on the face and content validities noted in this study. Although stapled anastomosis does not have good construct validity, the more challenging hand-sewn anastomosis does differentiate between various skill levels. A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance. Wohaibi EM,* Earle DB,† Ansanitis FE,* Wait RB,* Seymour NE. Purpose: The study aim was to demonstrate that a new database tool for assessment of surgical resident operative skills tracks predictable progression in those skills over successive residency years for specific index case types.

74

Methods: A Web-based interactive database (OpRate) was used to assess selected aspects of resident operative performance in a medium-sized residency (5– 6 residents per PGY). On-line assessments were made at procedure completion by a supervising attending surgeon and consisted of (1) 3 questions pertaining to patient information and technical and disease-specific preparedness; (2) 4 laparoscopic technical skills questions pertaining to tissue handling, dexterity, planning, and ability to function independently; and (3) similar open technical skills questions, with the addition of 2 questions defining knot-tying ability. Two years of assessment data were examined for cholecystectomy (CH), appendectomy (AP), colon resection (CR), ventral hernia repair (VH), and inguinal hernia repair (IH). Mean scores for total, technical, and preparedness responses as well as each individual response area were compared for successive training years for each case type. Mean performance data between postgraduate years were compared by ANOVA. Results: OpRate data for 579 cases (142 CH, 67 AP, 73 CR, 95 IH, 202 VH) were examined. Incremental increase in total technical skills scores by training year were observed for all case types (PGY 1 vs PGY 5: CH, 2.9 ⫾ 0.08 vs 3.9 ⫾ 0.04; AP, 2.6 ⫾ 0.1 vs 3.9 ⫾ 0.1; CR, 3.0 ⫾ 0.5 vs 3.8 ⫾ 0.03; IH,2.7⫾0.08 vs 3.8 ⫾ 0.04; VH, 2.6 ⫾ 0.2 vs 3.7 ⫾ 0.09; p ⬍ 0.001). Individual technical skills as well as technical and disease-specific preparedness response areas also demonstrated significant improvement by successive training year. The training year-dependent improvement was not observed for the patient information preparedness response area. Conclusions: Early results show that the OpRate database is a practical method of tracking expected changes in operative performance across successive training years. As such, the use of this database tool may offer the opportunity to (1) define performance benchmarks for specific levels of training and (2) identify areas where focused training may be required for specific residents.

Journal of Surgical Education • Volume 64/Number 2 • March/April 2007