Effects of a laboratory-based skills curriculum on laparoscopic proficiency: A randomized trial

Education

Effects of a laboratory-based skills curriculum on laparoscopic proficiency: A randomized trial Robert L. Coleman, MD, and Carolyn Y. Muller, MD Dallas, Tex OBJECTIVE: The purpose of this study was to determine the effect and validity of an intensive laboratorybased laparoscopic skills training curriculum on operative proficiency in obstetrics and gynecology residents. STUDY DESIGN: This was a prospective, randomized, block-design trial of postgraduate year 3 and 4 residents. The following study schedule was used: week 1 (prerandomization) included an orientation to study objectives, the administration of a laparoscopic experience questionnaire, timed video-laparoscopic drills (5 total), and the performance of a video-recorded laparoscopic partial salpingectomy; during weeks 2 and 3, the skills group residents repeated the laparoscopic drills 30 minutes daily for 10 days, and the control group residents had no formal practice sessions; during week 4, the week 1 evaluation was repeated. Operative proficiency was quantified by the Global Skills Assessment Tool through blinded, independent scoring of videotapes. RESULTS: Twenty-six residents (skills group, 12; control group, 14) consented to the trial. Patient-related issues excluded 8 residents (30%). At week 1, no significant differences existed in previous laparoscopic experience, timed video skills, or resident operative proficiency (Global Skills Assessment Tool score) between cohorts. At week 4, both groups significantly improved their timed drill test scores. The percent reduction in time from baseline was of greater magnitude in the skills group versus control group (51% vs 18%, P < .0001). Laparoscopic performance also improved in both cohorts (P = .002). However, only the skills group demonstrated significant intracohort improvement from baseline (mean, 4.9 points; P = .015; 95% CI, 1-7.5). CONCLUSION: A core curriculum of intensive video laparoscopic skills training improves not only technical but also operative performance among postgraduate year 3 and 4 residents. (Am J Obstet Gynecol 2002;186:836-42.)

Key words: Video endoscopic, trainer, laparoscopic, proficiency, global surgical assessment tool

In response to a widening disparity between the number of endoscopic gynecologic procedures that are performed and the extent of residency training, the Council on Resident Education in Obstetrics and Gynecology developed a set of training modules to help training programs incorporate an effective laparoscopic educational curriculum.1 Cundiff2 reported the effectiveness of these guidelines in a prospective study of residents who were undergoing didactic and practical instruction. Despite these global programs, however, many residents still report varied laparoscopic experiences and have varied levels of comfort with their own laparoscopic skills on From the Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, and The Southwestern Center for Minimally Invasive Surgery, University of Texas, Southwestern Medical Center. Supported by the Association of Professors of Gynecology and Obstetrics (APGO)/Ortho-McNeil Faculty Development Award. Received for publication August 24, 2001; accepted October 26, 2001. Reprints not available from the authors. Copyright 2002, Mosby, Inc. All rights reserved. 0002-9378/2002 $35.00 + 0 6/1/121254 doi:10.1067/mob.2002.121254

836

completion of their residency. In the experience reported by Cundiff, only 53% of residents reported competency by self-assessment (15% competency by faculty assessment) after incorporation of the guidelines, and subjective improvement in laparoscopic performance by faculty observations showed only upward, nonsignificant trends. Still, many authors advocate animate and inanimate laboratories as methods to improve skill precision and to learn new techniques.3-5 The disparity between skills testing and competency partly results from a lack of reproducible criteria by which to assess surgical skill. Objective measures of surgical performance are difficult to quantify. Good operative skills require not only motorskill finesse, but also knowledge, judgment, and experience. Schueneman et al6 demonstrated that surgical skill (by faculty assessment) correlated, not with speed and precision, but with the residents’ abilities to (1) use landmarks to create a mental 3-dimensional space, (2) interpret sensory cues based on previous experience, and (3) distinguish essential details from nonessential details. Goff et al7 reported their initial experience in validating an objective

Coleman and Muller 837

Volume 186, Number 4 Am J Obstet Gynecol

Fig. Randomization schema and number of participants in each phase of the trial.

measuring tool for surgical technical skills. In this animalbased assessment model, they were able to demonstrate construct validity among junior and senior residents by observing their performance of laparoscopic and open abdominal procedures. A follow-up study from these investigators who used inanimate tools suggests this type of assessment can be adequately accomplished in a more cost-efficient manner.8 Incorporation of concepts and tools (such as these) into a laparoscopic surgical curriculum would be critical to an expectation of improved surgical skill. A recent randomized study by Scott et al9 suggests that intensive laboratory-based endoscopic video training can enhance the skill performance of general surgical residents who perform laparoscopic cholecystectomy. The purpose of this study was to evaluate whether a laboratory-based video laparoscopic skills curriculum could translate into improved surgical skills in the operating room among our senior residents.

Material and methods All postgraduate year (PGY) 3 and 4 obstetrics and gynecology residents at the University of Texas, Southwestern Medical Center, who rotated through the general gynecology rotations at Parkland Health and Hospital Systems were asked to participate in this prospective randomized trial. These residents were recruited during the first gynecology rotation of their respective year. At each rotation block, there were potentially 6 residents (2 residents from each of 3 gynecology services) who were eligible to participate in the trial. The study was approved and monitored by the University of Texas, Southwestern Medical Center Institutional Review Board, and confidentiality of results was ensured. Each study rotation was 26 days in duration and was structured in the following manner: Week 1. A description of the program and its goals and objectives was presented. Informed consent was ob-

838 Coleman and Muller

April 2002 Am J Obstet Gynecol

Table I. Video-endoscopic skills stations used for evaluation Drill maneuver Running string Block move

Checkerboard drill

Bean drop

Suture foam

Task

Test

Participant uses 2 laparoscopic graspers to run a 140-cm string from 1 end to the other by grasping only the colored sections that are arranged at 12-cm intervals. Participant uses needle driver and a curved needle to hook lift and individually move 4 blocks from 1 designated area to another 15 cm away. The nondominant hand controls the endoscope. Participant picks up 16 metallic numbers/letters (1-8; A-H) individually with fine-tipped laparoscopic graspers from a flat surface and transports them to a flat arranged grid. One half of the items were to be moved with the right hand, and one half of the items were to be moved with the left hand. Participant picks up 5 beans individually from a flat dish and drops each of them through a 1-cm hole in an elevated cup that is 15 cm away. The dominant hand moves the beans, with the nondominant hand holding the endoscope. Participant places a single intracorporeal knot that binds 2 pieces of foam using and Endostitch (US Surgical Co, Norwalk, Conn) and fine graspers.

tained, and participating residents were given a questionnaire, the answers for which detailed their previous laparoscopic experiences. They were asked to grade their laparoscopic skills in 5 areas (instrument handling, knowledge of instruments, use of assistants, knowledge of specific procedures, video hand-eye coordination) on a 5-point scale, with “1” representing uncomfortable, “3” representing average, and “5” representing proficiency. A baseline, timed assessment of their performance in the pelvic trainer stations was recorded. We based our curriculum on 5 established laparoscopic drills (Table I): running string, block move, checkerboard, bean drop, and suture foam.10-13 The average time to complete 3 attempts in each skill station was used as the participant’s score. To maintain objectivity, no other assessment of skill was made for baseline testing. Residents were then oriented to a step-by-step “task-tobe-performed” bilateral laparoscopic partial salpingectomy. Identified patients were previously scheduled for elective tubal ligation. The specific skills monitored by videotape in each case were (1) visualized trocar placement, (2) the inspection and restoration of normal anatomy (if altered), (3) the identification of the fallopian tube to the fimbria, (4) the proper use of bipolar electrocautery to create a V across the tube joining in the mesosalpinx, (5) the proper use of endoscopic scissors/ forceps to excise and retrieve a portion of fallopian tube, (6) Endostitch (US Surgical Co, Norwalk, Conn) placement of the distal tube or mesosalpinx for hemostasis, and, (7) the removal of laparoscopic instruments under visualization. Skills 4, 5, and 6 were specifically designed to assess the translation of skills mimicked in the video laboratories. Skills 1, 2, 3, and 7 were skills that are taught customarily to residents as a part of the surgical training during the gynecology rotation. Qualified surgical assis-

Dominant and nondominant hand-eye coordination Dominant hand pronation-supination and 3-dimensional orientation Dominant and nondominant fine-motor grasping and orientation

Dominant hand fine-motor grasping, video orientation, and 3-dimensional training Intracorporeal knot technique, dominant and nondominant hand coordination, and video orientation

tance was provided by the participating faculty (informed of the study objectives) or a PGY-4 resident. Assistants were to respond only to the operating surgeon’s requests and were not to volunteer input unless it was requested. Techniques for port placement, the ability to trouble shoot equipment mishaps, and closure techniques were not assessed. After baseline data collection, residents were randomized to either intensive skills training or no additional skills training (control group). A block design was used to ensure that equal numbers of PGY-3 and -4 residents were in each of the study arms. Weeks 2 and 3. Residents who were randomized to the experimental group underwent skills training in the video laparoscopic training modules, as described, for at least 30 minutes daily for 10 days. A journal was kept of their daily progress. No particular drill rotation was required, but all skills stations were to be practiced until the participant was comfortable performing the task. Both groups underwent standard didactic and operative instruction, within their gynecologic rotation. Residents in the control arm had the opportunity to practice in the skills stations during their subsequent gynecologic rotations. Week 4. The last week of the rotation repeated the events of week 1 (questionnaire, a timed assessment of the 5 skills stations, and a videotape of a partial salpingectomy). The same faculty Attending Physician was present for both pretest and posttest surgical procedures. Residents were asked to report their laparoscopic experience during the rotation and their comfort level with endoscopic procedures. All videos were randomly numbered and blinded to PGY level, randomization cohort, and sequence (first or second procedure). Two faculty members independently reviewed and quantified resident operative proficiency using the global surgical assessment tool (GSAT).14-17

Coleman and Muller 839

Volume 186, Number 4 Am J Obstet Gynecol

Table II. Mean drill completion times (sec) between week 1 (pre) and week 4 (post) testing within and between cohorts

Skills Control P value†

Checkerboard (s)

Bean drop (s)

Pre Post P value*

Pre Post P value*

Pre Post P value* Pre Post P value*

Pre Post P value*

Pre Post P value*

178 95 171 141 NS .006

77 34 .001 75 53 .05 NS .001

68 70 NS

66 52 NS

451 222 < .0001 420 329 .0003 NS .0008

.0002 .0008

String (s)

Block (s)

41 .002 56 .02 .03

62 24 52 33 NS .02

.002 .04

Suture foam (s)

28 46 .04

.001 NS

Totals (s)

NS, Not significant. †Mann-Whitney U test, adjustment made for baseline variation. *Paired t test.

Table III. Percent improvement by task between randomization arms Method used Task Checkerboard Bean drop Running string Block move Suture foam Mean

Skills (Ơ)

Traditional (Ơ)

45.4 52.4 38.9 60.4 56.2 51.0

19.6 21.3 19.5 23.6 6.8 18.0

P value* .0007 .013 .053 .016 .001 <.0001

*Mann-Whitney U test. †Pretest-posttest(s)/pretest(s).

Participants were graded on a 5-point scale for each of the 7 categories (respect for tissue, time and motion, instrument handling, knowledge of instruments, flow of operation, use of assistants, and knowledge of specific procedure). For example, in the time and motion category, a resident who demonstrated many unnecessary moves was graded a “1”; efficient time and motion but with some unnecessary moves was graded a “3”; and a clear economy of movement and maximal efficiency was graded a “5.” A total score of the possible 35 points was recorded from each reviewer. All scoring was performed after the study’s conclusion to reduce bias that may have been related to individuals or problematic surgical procedures, which may have been made known indirectly to the reviewers. In our efforts to reduce bias, a subjective assessment from the attending surgeon who was involved in the surgical procedures was not collected. Statistical considerations. All comparisons were performed with the StatView 5.0 statistical program (SAS Institute, Cary, NC). Binomial data from the prequestionnaire and postquestionnaire were analyzed with the Fisher exact comparison. Ordinal categorical variables (such as comfort levels for individual tasks and global assessment scoring) were evaluated by Friedman’s 2-way analysis of variance. To test the hypothesis that no difference existed in video task performance between the cohorts, a Wilcoxon rank-sum test was calculated. Repeated-measures analysis of variance was used to compare timed skills tasks between cohorts at pretesting and posttesting. Paired intracohort continuous variables

that were not normally distributed were assessed with the Mann-Whitney U test. A paired t test was used to assess the effect of individual normally distributed variables within cohorts. Interrater reliability was assessed with the κ statistic. Results The University of Texas, Southwestern Medical Center Obstetrics and Gynecology Residency program consists of 14 residents at each PGY level. Overall, 26 of 28 potential resident candidates agreed to participate in the trial. The Fig shows the randomization schema and the number of residents that completed the trial. At the trial’s conclusion, 18 (skills cohort, 11; control cohort, 7) of 26 residents (69%) completed all phases of the protocol. Reasons for noncompletion of the trial were a lack of surgical case availability during the weeks intended (n = 6 residents) and a cancellation of a planned surgical case (n = 2 residents). Complete pretest and posttest laparoscopic drill data were collected on 20 residents (77%), 11 residents in the skills cohort and 9 residents in the control group. There was no crossover in this trial. Questionnaire. On the baseline questionnaire, both cohorts reported similar basic laparoscopic experience as a whole, and no significant difference was observed in any category of previous laparoscopic experience or in comfort with laparoscopic procedures. As expected, PGY-3 residents reported significantly fewer total laparoscopic procedures as compared with PGY-4 residents (median, 13 vs 45; P < .0001) and were more likely to grade them-

840 Coleman and Muller

April 2002 Am J Obstet Gynecol

Table IV. GSAT scoring of intraoperative performance by randomization cohort Pretest* Skills group Control group P value‡ (between cohorts)

17.4 16.4 NS

P† Posttest* (within cohort) 21.7 20.3 NS

0.0151 0.0923

NS, Not significant. *Mean of 35 total possible points. †Paired t test. ‡Mann-Whitney U test.

selves as having “low comfort” (on a 5-point scale, with “3” being “average”) with their endoscopic skills (P = .013). However, this effect was balanced by the block randomization design. Previous or concurrent video gaming was reported by 6 of 20 respondents. Two of these residents were in the skills cohort and 4 of these residents were in the control group (P = .2). Interestingly, although gamers reported similar comfort with laparoscopy at pretest as did nongamers, they quantified their skill levels significantly higher than nongamers (P = .043). Other demographic features were balanced between cohorts. Video skills testing. Baseline and posttest drill times from each of the video trainers are given by cohort in Table II. No difference was observed between the cohorts at baseline testing. However, posttest drill times were significantly different in all tasks between the cohorts. In addition, although all residents showed improvement in task completion times at the end of their gynecology rotation, those residents who were randomized to the experimental group showed the greatest improvement (construct validity; Table II). The magnitude of this effect was quantified by an adjustment of the within-group pretest to posttest variance as a measure of the individual baseline completion times. Overall, drill performance was enhanced in the skills group compared with the control group. Task completion times were reduced an average of 51% (range, 39%60%) versus 18% (range, 7%-24%; P < .0001; Table III). As expected, less marked improvement is seen among the most experienced surgeons, and critical minimum values were discernible on each skill. For example, if a resident performed the checkerboard task in ≤150 seconds, they were unlikely to be able to improve further from more practice (data not shown). This performance at baseline was within the 95% CI for the best posttraining times and essentially represented mastery of this skill. There is also much less variance among the performance of individual task trials at the conclusion of the rotation in the experimental group only. Intraoperative skills assessment. Assessment of criterion-related validity was determined by grading intraoperative resident surgical performance by the GSAT scores

both between and within the randomization strata. Collectively, mean GSAT overall scores were similar between the cohorts at baseline and at study conclusion (Table IV). As expected, both cohorts demonstrated significant improvement in laparoscopic skill with the completion of the gynecology rotation. However, only within the skills cohort was there a significant improvement from baseline (mean, 4.3; P = .0151; paired t test). In this manner, PGY3 and PGY-4 residents who underwent brief, but intensive, video endoscopic skills training appeared to enhanced their laparoscopic skills above what might have been obtained through regular rotation-related practice. Although this study was not sufficiently powered to stratify by PGY level, it was observed that the improved intracohort skill assessment was most striking among PGY-3 residents as compared with PGY-4 residents (data not shown). Interrater agreement was good between our independent reviewers on the GSAT (κ, 0.72). Resident skills perception. Both cohorts reported a lower than average pretest comfort in these areas (median, 12 [skills group] and 13.5 [control group] of a possible 25 total points). As expected, self-assessment competency improved in both cohorts after their rotation (median, 16 [skills group] and 17 [control group]). Pretest to posttest self-assessment was improved significantly (P = .001), yet no difference was observed between the skills group and the control group after their rotation. Further, only a weak association (r = .11) was observed between self-perception and objective operative evaluation by the GSAT, even among the skills group. Comment Effective instruction of surgical skill is central to all residency training programs. The manner in which these skills are taught and the assessment of an individual’s skill level are heterogeneous and often ambiguous.7,18 Techniques include lecture, book instruction, animal model dissection, and dry laboratory instruction. However, the most common theater in which to provide surgical instruction is the operating room. A recent survey of residency directors reported that 99% of responders taught surgical skills in the operating room.19 However, only 29% had a formal surgical skills program, and only 17% used testing or standardized assessment. Still, there is mounting evidence that objective measuring tools can be validated and reproduced for both open and laparoendoscopic techniques.7,20-22 Various authors have suggested that educational objectives that reach through laboratory-based skills practice can lead to improved endoscopic proficiency by fine-tuning motor skills such as dexterity and hand-eye coordination. Derossis et al23 demonstrated among general surgical residents that dexterity and visual-spacial skills could be improved through repetition in video drill boxes. Similarly, Cundiff et al2 showed significant im-

Volume 186, Number 4 Am J Obstet Gynecol

provement in endoscopic competency with repetitive practice among a cohort of obstetrics and gynecology residents at various levels of training. Their principal outcome measure was task completion time. Improvement in laboratory-based skills correlated with improvement in endoscopic self-assessment among the residents. No in vivo confirmation was performed. However, faculty assessment of resident proficiency did follow a nonsignificant upward trend. Using laboratory-based proficiency as a surrogate for intraoperative skill is a harder bridge to cross because most measures of success (such as short time to task completion) do not adequately evaluate precision or judgment, which are important intraoperative necessities. Martin et al,15 who used open surgical bench models, compared their results with similar tasks in live anesthesized animals. Highly significant correlation was observed between the 2 models, which validated their bench testing facility. Fried et al20 demonstrated similar findings in a randomized trial of 12 general surgery residents who underwent laparoscopic skills evaluation. These investigators, after baseline assessment in both in vitro (video boxes) and in vivo (porcine model) models, randomized participants to either 5 days of skills training or no training. Repeat testing in both models demonstrated “value-added” improvement of in vivo skills after laboratory-based practice. Scott et al9, in one of the few human in vivo trials, randomized 27 junior general surgery residents to either 10 days of endoscopic video training or no additional training after baseline proficiency evaluation by inanimate task-completion testing and intraoperative GSAT scoring of laparoscopic cholecystectomy. They demonstrated improvement in video-eye-hand skills after training that translated into improved intraoperative skill assessment. “Value-added” improvement was evident in 4 of 8 individual parameters in the GSAT, when compared with control subjects. In the current trial, we suggest a similar extrapolation among PGY3 and PGY4 obstetrics and gynecology residents in the performance of laparoscopic partial salpingectomy. Previous reports that used the same video endoscopic tasks as used in this trial have demonstrated construct validity among varying levels of residents at baseline.2,7,9,12,15 This effect was balanced in the current report by the block randomization technique. However, construct validity was demonstrated after training because significant “value-added” improvement was observed in our skills group compared with control subjects in task completion efficiency. Certain tasks were more discriminating than others, such as intracorporeal knot tying and the checkerboard drill. Modification of the video drills to more closely mimic intraoperative skills may more efficiently teach these desired principles. Intraoperative skill between cohorts as measured by the GSAT was similar at baseline and significantly im-

Coleman and Muller 841

proved at posttest but not statistically between the randomized cohorts. However, operative intracohort improvement from baseline was observed significantly only in the skills cohort. This finding suggests that the endoscopic instruction and experience during a customary gynecologic rotation affects surgical endoscopic skill positively. However, the performance of laboratory-based skills training appears to provide an added benefit through repetitive video-eye-hand practice. Criterionrelated validity is demonstrated. In the current study, a specified task-to-be-completed partial salpingectomy was chosen as the evaluating procedure. This was chosen because candidate patients were abundant and could be enrolled electively at specified times that were relevant to resident training. Although construct validity has not been demonstrated with respect to this procedure, a much greater effect of practice and intracohort improvement was seen in our PGY3 residents. This suggests that the level of complexity for the assessment operation may have discerned critical skills that were gained from laboratory practice among less-experienced surgeons. Formal evaluation should be conducted. There are several important considerations about the conduct of this trial, which require discussion. First, the number of residents who completed all phases of the protocol was approximately two thirds of the initial randomization. Reasons for this (patient availability and case cancellation) were not controllable; however, we are not able to assess whether this excluded resident cohort was representative of the measured cohort. The disproportionate number of residents who completed all tasks (n = 11 [skills cohort] vs 7 [control cohort]) might suggest less motivation in finding a suitable surgical candidate. However, comparison of posttest video scoring that was available for 2 of these residents who did not perform a second surgical case was similar to the control group as a whole. This suggests that the attrition bias may be minimal. Second, adherence to the planned surgical procedure was variable but random. Despite standardization tools, intrinsic patient anatomy, comorbidities and difficulties related to patient factors, entry complications, and trocar and instrument malfunctions could not be controlled. Likewise, the degree to which these deviations were scored by the independent blinded judges was not standardized. However, good overall interobserver agreement was found between our independent reviewers (κ = 0.72). Third, resident proficiency was measured with the GSAT. This objective assessment of surgical skills was pioneered and validated at the University of Toronto in general surgery residents in both animate and inanimate models.14-17 The tool has been validated recently by obstetrics and gynecology residents. Investigators at the University of Washington have used the tool with both animate and inanimate models to assess resident surgical

842 Coleman and Muller

skill effectively and are suggesting the tool be used to identify residents in whom additional surgical training may be beneficial.8 Our study similarly supports its objective use of surgical performance. Last, long-term skill retention was not addressed. Because testing (both laboratory and in vivo) was in close proximity to skills training, the maximal effect of this intervention is measured. Derossis et al24 demonstrated that composite precision/accuracy scores could be followed in general surgical residents over a 2-year period with simulator models. However, whether improved skills by training persist without specific skills training is not known. We intend to incorporate “booster” skills sessions throughout all gynecology rotations in our residency program. In this trial, we demonstrate that a short-term intensive laboratory-based video laparoscopic skills curriculum can translate into better individual operative proficiency. These findings have significant implications not only in the efficiency with which new procedures can be learned but also ultimately in improved patient safety and lowered operative cost. We suggest that formal skills training should be incorporated into obstetric and gynecology resident surgical curriculum to enhance surgical skill. REFERENCES

1. Sanfilippo JS, Laube DW, Sammarco JJ, Thompson RJ, Visscher HC, Youngblood JP. CREOG training guidelines: advanced surgical techniques for residency training programs in obstetrics and gynecology. Washington (DC); 1994. 2. Cundiff GW. Analysis of the effectiveness of an endoscopy education program in improving residents’ laparoscopic skills. Obstet Gynecol 1997;90:854-9. 3. Melvin WS, Johnson JA, Ellison EC. Laparoscopic skills enhancement. Am J Surg 1996;172:377-9. 4. Sammarco MJ, Youngblood JP. A resident teaching program in operative endoscopy. Obstet Gynecol 1993;81:463-6. 5. Wolfe WM, Levine RL, Sanfilippo JS, Eggler S. A teaching model for endoscopic surgery: hysteroscopy and pelviscopic surgery. Fertil Steril 1988;50:662-4. 6. Schueneman AL, Pickleman J, Hesslein R, Freeark RJ. Neuropsychologic predictors of operative skill among general surgery residents. Surgery 1984;96:288-95.

April 2002 Am J Obstet Gynecol

7. Goff BA, Lentz GM, Lee D, Houmard B, Mandel LS. Development of an objective structured assessment of technical skills for obstetric and gynecology residents. Obstet Gynecol 2000;96:146-50. 8. Lentz GM, Mandel LS, Lee D, Gardella C, Melville J, Goff BA. Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: validity and reliability. Am J Obstet Gynecol 2001;184:1462-70. 9. Scott DJ, Bergen PC, Rege RV, et al. Laparoscopic training on bench models: Better and more cost effective than operating room experience? J Am Coll Surg 2000;191:272-83. 10. Jones DB, Brewer JD, Soper NJ. The influence of three-dimensional video systems on laparoscopic task performance. Surg Laparosc Endosc 1996;6:191-7. 11. Nguyen NT, Mayer KL, Bold RJ, et al. Laparoscopic suturing evaluation among surgical residents. J Surg Res 2000;93:133-6. 12. Rosser JC, Rosser LE, Savalgi RS. Skill acquisition and assessment for laparoscopic surgery. Arch Surg 1997;132:200-4. 13. Rosser JC Jr, Rosser LE, Savalgi RS. Objective evaluation of a laparoscopic surgical skill program for residents and senior surgeons. Arch Surg 1998;133:657-61. 14. Faulkner H, Regehr G, Martin J, Reznick R. Validation of an objective structured assessment of technical skill for surgical residents. Acad Med 1996;71:1363-5. 15. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273-8. 16. Reznick R, Regehr G, MacRae H, Martin J, McCulloch W. Testing technical skill via an innovative “bench station” examination. Am J Surg 1997;173:226-30. 17. Winckel CP, Reznick RK, Cohen R, Taylor B. Reliability and construct validity of a structured technical skills assessment form. Am J Surg 1994;167:423-7. 18. Reznick RK. Teaching and testing technical skills. Am J Surg 1993;165:358-61. 19. Mandel LP, Lentz GM, Goff BA. Teaching and evaluating surgical skills. Obstet Gynecol 2000;95:783-5. 20. Fried GM, Derossis AM, Bothwell J, Sigman HH. Comparison of laparoscopic performance in vivo with performance measured in a laparoscopic simulator. Surg Endosc 1999;13:1077-82. 21. Goff BA, Lentz GM, Lee DM, Mandel LS. Formal teaching of surgical skills in an obstetric-gynecologic residency. Obstet Gynecol 1999;93:785-90. 22. Shapiro SJ, Paz-Partlow M, Daykhovsky L, Gordon LA. The use of a modular skills center for the maintenance of laparoscopic skills. Surg Endosc 1996;10:816-9. 23. Derossis AM, Fried GM, Abrahamowicz M, Sigman HH, Barkun JS, Meakins JL. Development of a model for training and evaluation of laparoscopic skills. Am J Surg 1998;175:482-7. 24. Derossis AM, Antoniuk M, Fried GM. Evaluation of laparoscopic skills: a 2-year follow-up during residency training [see comments]. Can J Surg 1999;42:293-6.