- Email: [email protected]
Goal-directed laparoscopic training leads to better laparoscopic skill acquisition
Goal-directed laparoscopic training leads to better laparoscopic skill acquisition Atul K. Madan, MD,a Jason L. Harper, MD,a Raymond J. Taddeucci, MD,b and David S. Tichansky, MD,c Miami, FL, Lincoln, NE, and Memphis, TN
Introduction. Laparoscopic skills training outside the operating room is becoming the standard for educating surgical residents. Because of the restrictions on the work week, it is imperative for this training to be efficient. We hypothesized that goal-directed laparoscopic training (GDLT) would result in better skill acquisition than laparoscopic training without goals (LT). Methods. Second-year general surgery residents participated in this study. Metrics were scores that incorporated time and errors. One group of residents (LT) went through a 10- week laparoscopic training course without goals; one group of residents (GDLT) was given goals to achieve during their course. Each group practiced for the same amount of time. The tasks were peg exercise, run the rope, pattern cutting, clip/cut vessel, extracorporeal knot tying, intracorporeal knot tying, and suturing device. Statistical analysis was performed via 2-tailed Mann-Whitney tests. Results. There were 8 residents in the LT group and 7 residents in the GDLT. The GDLT group had statistically significant higher scores on 7 of the 8 tasks compared the LT group (P < .02 to P < .0001). The GDLT group performed better in the final task, suturing device, than the LT group, but this did not reach statistical significance (451 vs 414; P = .14). Conclusions. GDLT should be used by surgeons instead of LT. Future studies need to examine whether GDLT translates into a better operative technique and outcomes. (Surgery 2008;144:345-50.) From the Division of Laparoendoscopic and Bariatric Surgery, University of Miami, Miami, FL,a the Surgical Associates, P.C., Lincoln, NE,b and the Section of Minimally Invasive Surgery, Department of Surgery, University of Tennessee Health Science Center, Memphis, TN c
IT HAS BECOME well accepted that laparoscopic surgery procedures require a different set of skills compared with open procedures.1-13 Laparoscopic skills differ from open skills because of issues of the altered tactile sensation, the lack of depth perception, the requirement to work in a 3-dimensional environment while visualizing a 2-dimensional environment, the relatively limited degrees of freedom of laparoscopic instruments, the increased length of laparoscopic instrumentation, and the varying eye hand coordination.1-13 The issues of limited resident education time, shortage of manpower, cost of operating room time, decreasing reimbursement for the attending surgeons, and Presented at the 3rd Annual Academic Surgical Congress, Huntington Beach, California, February 2008. Accepted for publication March 14, 2008. Reprint requests: Atul K. Madan, MD, FACS, Department of Surgery, University of Miami, 1475 NW 12th Avenue, Suite 4017, Miami, FL 33136. E-mail: [email protected]. 0039-6060/$ - see front matter Ó 2008 Mosby, Inc. All rights reserved. doi:10.1016/j.surg.2008.03.016
most importantly, patient safety make the operating room a less than ideal situation to teach basic laparoscopic skills. Laparoscopic skills training outside the operating room is becoming the standard for best educating surgical residents.1,5,6,10,11,13-15 The 2 general groups of training devices are virtual reality trainers and inanimate box trainers. Both of these devices have been shown to be useful adjuncts in training basic laparoscopic skills.1,2,13,16,17 Despite the automatic scoring systems, the ability to give feedback on overall and individual hand performance, as well as the potential to be used by trainers as metrics for laparoscopic skills,1,4,5 the major drawbacks of virtual reality trainer include the expense and lack of tactile feedback.1,2 Inanimate box trainers are the most commonly used laparoscopic training devices by surgeons in those residency programs that have laparoscopic skills laboratories.18 In fact, only 46% of laboratories had virtual reality equipment, whereas 99% had an inanimate box trainer. Furthermore, some trainees have a higher opinion of box trainers compared with virtual reality trainers.7 SURGERY 345
346 Madan et al
Surgery August 2008
Table I. Instructions given to each resident for each task Peg transfer
Running the rope Pattern cutting Clip/cut the vessel
Extracorporeal knot
Intracorporeal knot
Suturing device
Lift sleeves from left board with left hand, transfer to right hand in midair, and place on right board. When all sleeves have been transferred, reverse the procedure. Start and stop from home base. ‘‘Run’’ a 1/8 3 60’’ rope with marks every 5.’’ Start with both graspers on the green mark, and end when both are on the red mark. Handle the rope only on the marks. Cut premarked 4-cm circle from the piece of dental dam. Start from resting position, and stop when the cut is complete. Place a clip on each of the outside marks on the foam ‘‘vessel,’’ and cut on the center mark between the clips. Start from resting position, and stop when the cut is complete. Place suture through dots on Penrose drain. Tie an extracorporeal knot (3 throws, carefully squared), and cut the suture. Start from resting position with needle loaded. Stop when suture is cut. Place suture through dots on Penrose drain. Tie an intracorporeal sugeons knot (a double throw followed by 2 single throws, carefully squared), and cut the suture. Start from resting position with needle loaded, and stop when suture is cut. Place suture through dots on Penrose drain using suturing device. Tie a surgeon’s knot (a double throw followed by 2 single throws carefully squared), and cut the suture. Start from resting position, and stop when suture is cut.
Each type of trainer has advantages and disadvantages that have been well summarized elsewhere.1 The issues of training residents in laparoscopic techniques are compounded with the issues of the 80-hour work week.19 Because of the restrictions on the resident work week, it is imperative for any type of laparoscopic skills training to be efficient. The difficulty in obtaining resources and the limited amount of resident time deem it necessary to evaluate any laparoscopic skill training to ensure that it offers appropriate skill acquisition. We hypothesized that goal-directed laparoscopic training (GDLT) would result in better skill acquisition than laparoscopic training without goals (LT). MATERIALS AND METHODS During the time of this investigation, the institution where this study was performed had an intensive laparoscopic skill educational program for the residents. All residents were given access to a laparoscopic skills laboratory that included inanimate box and virtual reality trainers. The residents were assigned specific tasks on the virtual reality trainer depending on their postgraduate year. The upper level residents were taken in pairs to a laparoscopic training facility weekly to perform advanced laparoscopic procedures in human cadavers. The second-year general surgery residents underwent a 10-week laparoscopic training course in inanimate box trainers. Since this study was a retrospective examination of a portion of our laparoscopic curriculum, we received an exemption by our Institutional Review Board. The second-year general surgery resident
course was studied. The course was given to each of our second-year residents for 10 weeks. For the first 8 weeks, the residents were taught and practiced a task each week. The residents worked in pairs and were taught by a surgeon trained in minimally invasive techniques. Immediate feedback was given to the resident after each trial. Scores were recorded by the educator and told to the residents. During week 9, the residents were allowed to practice each task. During week 10, the residents were tested on each task. Every resident was given the training schedule and knew they were going to be tested at the end of the curriculum. Attendance was mandatory and was performed in the morning in order to comply with the 80-hour work-week restriction. Residents who were on vacation made up the session either before or after they left for vacation. All residents had the same amount of time in the skills laboratory (2 hours per week for 10 weeks). Most tasks were modified from tasks taken from the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) and were similar to the Fundamentals of Laparoscopic Surgery.20 The tasks included peg transfer, running the rope, pattern cutting, clip/cut vessel, extracorporeal knot, intracorporeal knot, and suture device. Table I demonstrates the instructions given to the residents for each task. The scoring method is displayed in Table II for each task. The scoring method was given to the residents before each exercise. One group of residents (LT) went through the 10-week laparoscopic training course without
Madan et al 347
Surgery Volume 144, Number 2
Table II. Scoring method for each task Peg transfer Penalty Cutoff Score Running the rope Penalty
Cutoff Score Pattern cutting Penalty
Cutoff Score Clip/cut the vessel Penalty Cutoff Score Extracorporeal knot Penalty
Cutoff Score Intracorporeal knot Penalty
Cutoff Score Suturing device Penalty
Cutoff Score
% of sleeves lost from view 5 seconds for each dropped sleep 300 seconds 300—observed time—penalty 5 seconds for grasping outside the marks 10 seconds for dropping the rope 300 seconds 300—observed time—penalty % of circumference that deviates more than 2 mm from perfect circle 300 seconds 300—observed time—penalty 10 seconds for each mm that a clip or the cut is ‘‘off the mark’’ 120 seconds 120—observed time—penalty 5 seconds for each mm is ‘‘off the mark’’ 5 seconds for each mm of gap 10 seconds for a knot that slips 20 seconds for a knot that disrupts 20 seconds for improperly tied knot 420 seconds 420—observed time—penalty 5 seconds for each mm is ‘‘off the mark’’ 5 seconds for each mm of gap 10 seconds for a knot that slips 20 seconds for a knot that disrupts 20 seconds for improperly tied knot 600 seconds 600—observed time—penalty 5 seconds for each mm is ‘‘off the mark’’ 5 seconds for each mm of gap 10 seconds for a knot that slips 20 seconds for a knot that disrupts 20 seconds for improperly tied knot 600 seconds 600—observed time—penalty
goals. One group of residents (GDLT) was given goals to achieve during their course. The 2 groups were not randomized. The LT group was the first year of this study, and the GDLT was the second
Table III. Laparoscopic goals given to the GDLT group to achieve Exercise
Score
Peg transfer Running the rope Pattern cutting Clip/cut vessel Extracorporeal knot Intracorporeal knot Suture device
204 264 239 113 221 510 521
year of this study. The goals were chosen based on high scores from the LT group. We decided to choose the high scores in order to place high standards in laparoscopic skills for the residents. Choosing lower scores may not have set the bar high enough for the residents. In addition, we felt that this method would be easily transferable to the laparoscopic skill curriculum in other programs. The rest of the curriculum between each group was kept identical. The residents were not told anything different between each year. We compared means for all recorded scores, including the practice and test sessions between the GDLT and LT groups. Statistical analysis was performed with nonparametric, 2-tailed Mann-Whitney tests or the Fisher exact test as appropriate via the GraphPad InStat Version 3.05 (San Diego, CA). A P value less than .05 was considered statistically significant. RESULTS There were 8 residents in the LT group and 7 residents in the GDLT. Table III demonstrates the goals that were given to the GDLT group that were arbitrarily assigned after examining the scores from the LT group. The GDLT group had statistically significant higher scores on 7 of the 8 tasks compared with the LT group (P < .02 to P < .0001; Table IV). The GDLT group performed better in the final task, suturing device, than the LT group, but this did not reach statistical significance (451 vs 414; P = .14). More residents in the GDLT group achieved the set goals than in the LT group as shown in Table V. CONCLUSIONS Numerous studies have demonstrated that laparoscopic training helps laparoscopic skills acquisition.1,13,21 Logic dictates that better skills should ultimately translate into better clinical patient outcomes. Unfortunately, no data are available to support or refute this logic. However, our data does demonstrate that setting goals increases laparoscopic skill acquisition.
348 Madan et al
Surgery August 2008
Table IV. Mean (range) scores for each task between the GDLT and the LT Tasks
GDLT
Peg exercise Run the rope Pattern cutting Clip/cut vessel Extracorporeal knot tying Intracorporeal knot tying Suturing device
185 249 223 109 309 433 451
(40–226) (185–278) (132–264) (101–114) (229–375) (63–537) (303–534)
Table V. Percentage of residents that reached goals by group P value
Tasks
GDLT
LT
Peg exercise Run the rope Pattern cutting Clip/cut vessel Extracorporeal knot tying Intracorporeal knot tying Suturing device
100% 57.1% 57.1% 57.1% 100%
12.5% 12.5% 25% 12.5% 12.5%
< = = = <
57.1%
12.5%
= .12
28.6%
12.5%
= .57
.002 .12 .31 .12 .002
This study does have a limitation in its retrospective nature and lack of randomization. The 2 groups of residents were in different years; thus, it could be that the observed differences were caused by some confounding factor that is not obvious. However, during the time period, there were no major changes to the resident rotations, no major change in faculty, and no major change in cases that second-year residents would normally perform. The sample size was relatively low although it was equivalent to many other studies in laparoscopic skill training for residents. However, the fact that the groups were not randomized, in different time periods, and were relatively small may limit the generalizability of our results. In addition, there was a large variability of scores within each group that also needs to be explored. We did not assess baseline laparoscopic skills, resident demographic data, or nonsurgical skills, which is another limitation as they may have some impact on baseline laparoscopic skills although this is controversial.3,22 We also did not examine skill retention (or skill degradation) that has been found to change over time without continual practice.23 Unlike some of our previous studies,1,3 we did not have formal metrics in an animate model to test translation into actual operative procedures or tasks. In addition, we did not analyze whether the observed differences were caused by time versus errors, which could be
LT 160 225 188 105 212 351 414
(18–212) (127–264) (40–239) (94–113) (198–227) (30–510) (110–521)
P value < < < < < < =
.0001 .0001 .0001 .02 .0001 .0001 .14
important in focusing educational efforts. Despite the limitations of this study, our data demonstrate that with the only change in the curriculum being the addition of goals for each task, we were able to increase the average scores of the residents. Many have suggested that proficiency-based training is the best method to train residents in laparoscopic skills.13,16,24,25 Proficiency-based training is usually defined as training until the trainees reach a predefined set level or goal. We agree that setting goals for any type of training seems to be invaluable. The data for the current study demonstrate that residents do better after goals are set. However, one limitation of the study is that no true method exists to determine what should be the goal. Most studies used aggregate scores of ‘‘experts’’ in laparoscopic surgery.13,26 Successful outcomes in the operating room are not determined by only speed and lack of errors. Decisionmaking ability, anatomical and variant knowledge, team work, crisis management, leadership, experience, surgical judgment, and technical skill are important parts of ensuring excellent patient outcomes.27 It is not known what the effect on our results would have been if our goals were not appropriately set. If our goals were set higher, the residents may have performed even better, although some may have performed worse because of their feeling of unrealistic goals. If our goals were set lower, the residents may have performed worse thinking that they had already achieved the goals. Future studies need to examine whether inappropriate goals have any effect on laparoscopic training. The other issue surrounding many studies in proficiency-based training is that it is usually compared with no training at all.13,16,24 The fact that any medical student, resident, or surgeon does better in a subject after being trained should be no surprise. Although proficiency-based training seems like it is a more appropriate method for training, appropriate research has not been performed to compare it with training for a specific amount of time only. In addition, if goals are set
Madan et al 349
Surgery Volume 144, Number 2
too low or the level of initial skill of the trainee is set too high, proficiency-based training may limit the amount of skill acquisition that can occur outside the operating room and negate the benefits or training outside of the operating room. However, our data lend support to the fact that setting goals for residents increases the average score of residents. Setting goals and making them known to the residents may help decrease the training time required for proficiency-based training. Another issue surrounding pure proficiencybased training is that it is difficult to incorporate into the practical world of general surgery training. Ideally, residents would not be able to perform certain procedures until they have passed a certain level of proficiency in virtual trainers, inanimate box trainers, animate models, and the human cadaver model. The expense and manpower necessary for this type of training is impractical. Although residents can score themselves honestly and accurately without direct observation by an independent evaluator,8 when it may affect their ability to perform cases, self-scoring would be less than ideal. Thus, manpower is an issue not only to educate but also to grade the residents. In addition, a laparoscopic training laboratory is expensive. Many general surgery programs do not have one; and those that do may not be used appropriately.18 Since Graduate Medical Education funds from the Center of Medicare services are not necessarily given to the those who have the true responsibility of training residents (ie, general surgery residencies),28 it is difficult for many financially strapped general surgery residency programs to initiate and maintain a laparoscopic training laboratory. In addition, with the 80-hour work week and the advanced laparoscopic requirements set by the RRC,19,29 denying resident participation in procedures that some residencies have limited supply of may be impractical. Future studies need to examine whether GDLT translates into better operative technique and outcomes. Although much has been written about laparoscopic skills acquisition, future research needs to focus on the specifics on how to best train. In fact, a recent study surveyed members of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES).30 The results of this study demonstrated that the SAGES membership felt that the question ‘‘What methods of simulation are most effective in helping surgeons learn techniques and skills for gastrointestinal and endoscopic surgery?’’ was in the top 12 most important research questions in gastrointestinal and endoscopic surgery. We know training helps, but we
are still far from learning what is the ‘‘most effective’’ method to use for training. The lack of consistent resource, facilities, and abilities make it difficult to determine a standard for training residents in laparoscopic procedures. Formal curricula and minimum standards need to be developed and validated. These formal curricula and minimum standards must be adopted by each general surgery residency program to ensure appropriate laparoscopic skills education. Most residency programs have a limited laparoscopic skill laboratory curriculum. This study helps those programs with limited resources to best maximize resident laparoscopic skill acquisition. Whenever training residents in laparoscopic surgery, goals should be set and scores should be given to the residents. GDLT should be used by trainers instead of LT to ensure better laparoscopic skill acquisition. REFERENCES 1. Madan AK, Frantzides CT. Prospecitve, randomized controlled trial of laparoscopic trainers in laparoscopic skills acquistion. Surg Endosc 2007;21:209-13. 2. Madan AK, Frantzides CT. Substituting virtual reality training for inanimate box trainers does not decrease laparoscopic skill acquisition. JSLS 2007;11:87-9. 3. Madan AK, Frantzides CT, Park WC, Tebbit CL, Kumari NV, O’Leary PJ. Predicting baseline laparoscopic surgery skills. Surg Endosc 2005;19:101-3. 4. Madan AK, Frantzides CT, Sasso L. Laparoscopic baseline ability assessment by virtual reality. J Laparoendosc Adv Surg Tech A 2005;15:13-7. 5. Madan AK, Frantzides CT, Shervin N, Tebbit CL. Assessment of individual hand performance in box trainers compared to virtual reality trainers. Am Surg 2003;69:1112-4. 6. Madan AK, Frantzides CT, Tebbit CL, Park WC, Kumari NV, Shervin N. Evaluation of specialized laparoscopic suturing and tying devices. JSLS 2004;8:191-3. 7. Madan AK, Frantzides CT, Tebbit CL, Quiros RM. Participant’s opinions of laparoscopic trainers during basic laparoscopic training courses. Am J Surg 2005;189:758-61. 8. Madan AK, Frantzides CT, Tebbit CL, Shervin N. Self-reported versus observed scores in laparoscopic skills training. Surg Endosc 2005;19:670-2. 9. Rosser JC, Rosser LE, Savalgi RS. Skill acquisition and assessment for laparoscopic surgery. Arch Surg 1997;132:200-4. 10. Champion JK, Hunter J, Trus T, Laycock W. Teaching basic video skills as an aid in laparoscopic suturing. Surg Endosc 1996;10:23-5. 11. Pearson AM, Gallagher AG, Rosser JC, Satava RM. Evaluation of structured and quantitative training methods for teaching intracorporeal knot tying. Surg Endosc 2002;16:130-7. 12. Ali MR, Mowery Y, Kaplan B, DeMaria EJ. Training the novice in laparoscopy. Surg Endosc 2002;16:1732-6. 13. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458-64. 14. Rosser JC, Rosser LE, Savalgi RS. Objective evaluation of a laparoscopic surgical skill program for residents and senior surgeons. Arch Surg 1998;133:657-61.
350 Madan et al
15. Mori T, Hatano N, Maruyama S, Atomi Y. Significance of ‘‘hands-on training’’ in laparoscopic surgery. Surg Endosc 1998;12:256-60. 16. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky D III. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007;193:797-804. 17. Hasson HM KN, Eekhout J. Training simulator for developing laparoscopic skills. JSLS 2001;5:255-65. 18. Korndorffer JR, Stefanidis D, Scott DJ. Laparoscopic skill laboratories: current assessment and a call for resident training standards. Am J Surg 2006;191:17-22. 19. Duty Hours Language. Available from: URL:http://www. acgme.org/acWebsite/dutyHours/dh_Lang703.pdf. 20. Derossis AM, Fried GM, Abrahamowicz M, Sigman HH, Barkun JS, Meakins JL. Development of a model for training and evaluation of laparoscopic skills. Am J Surg 1998; 175:482-7. 21. Satava RM. Accomplishments and challenges of surgical simulation: dawning of the next generation surgical education. Surg Endosc 2001;15:232-41. 22. Madan AK, Harper JH, Frantzides CT, Tichansky DS. Nonsurgical skills do not predict baseline scores in animate box or virtual reality trainers. Surg Endosc. In press.
Surgery August 2008
23. Torkington J, Smith SG, Rees B, Darzi A. The role of the basic surgical skills course in the acquisition and retneniton of laparoscopic skill. Surg Endosc 2001;15:1071-5. 24. Korndorffer JR, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg 2005;201:23-9. 25. Stefanidis D, Korndorffer JR, Markley S, Sierra R, Scott DJ. Proficiency maintenance: impact of ongoing simulator on laparoscopic skill retention. J Am Coll Surg 2006;202: 599-603. 26. Korndorffer JR Jr, Scott DJ, Sierra R, Brunner WC, Dunne JB, Slakey DP, et al. Developing and testing competency levels for laparoscopic skills training. Arch Surg 2005;140:80-4. 27. Mishra A, Catchpole K, Dale T, McCulloch P. The influence of non-technical performance on technical outcome in laparoscopic cholecystectomy. Surg Endosc 2008;22:68-73. 28. Madan AK, Fabian TC, Tichansky DS. Potential financial impact of first assistant billing by surgical residents. Am Surg 2007;73:652-7. 29. Surgery Policy Information. Available from: URL:http:// www.acgme.org/acWebsite/RRC_440/440_policyArchive.asp. 30. Urbach DR, Horvath KD, Baxter NN, Jobe BA, Madam AK, Pryor AD, et al. A research agenda for gastrointestinal and endoscopic surgery. Surg Endosc 2007;21:1518-25.
Introduction. Laparoscopic skills training outside the operating room is becoming the standard for educating surgical residents. Because of the restrictions on the work week, it is imperative for this training to be efficient. We hypothesized that goal-directed laparoscopic training (GDLT) would result in better skill acquisition than laparoscopic training without goals (LT). Methods. Second-year general surgery residents participated in this study. Metrics were scores that incorporated time and errors. One group of residents (LT) went through a 10- week laparoscopic training course without goals; one group of residents (GDLT) was given goals to achieve during their course. Each group practiced for the same amount of time. The tasks were peg exercise, run the rope, pattern cutting, clip/cut vessel, extracorporeal knot tying, intracorporeal knot tying, and suturing device. Statistical analysis was performed via 2-tailed Mann-Whitney tests. Results. There were 8 residents in the LT group and 7 residents in the GDLT. The GDLT group had statistically significant higher scores on 7 of the 8 tasks compared the LT group (P < .02 to P < .0001). The GDLT group performed better in the final task, suturing device, than the LT group, but this did not reach statistical significance (451 vs 414; P = .14). Conclusions. GDLT should be used by surgeons instead of LT. Future studies need to examine whether GDLT translates into a better operative technique and outcomes. (Surgery 2008;144:345-50.) From the Division of Laparoendoscopic and Bariatric Surgery, University of Miami, Miami, FL,a the Surgical Associates, P.C., Lincoln, NE,b and the Section of Minimally Invasive Surgery, Department of Surgery, University of Tennessee Health Science Center, Memphis, TN c
IT HAS BECOME well accepted that laparoscopic surgery procedures require a different set of skills compared with open procedures.1-13 Laparoscopic skills differ from open skills because of issues of the altered tactile sensation, the lack of depth perception, the requirement to work in a 3-dimensional environment while visualizing a 2-dimensional environment, the relatively limited degrees of freedom of laparoscopic instruments, the increased length of laparoscopic instrumentation, and the varying eye hand coordination.1-13 The issues of limited resident education time, shortage of manpower, cost of operating room time, decreasing reimbursement for the attending surgeons, and Presented at the 3rd Annual Academic Surgical Congress, Huntington Beach, California, February 2008. Accepted for publication March 14, 2008. Reprint requests: Atul K. Madan, MD, FACS, Department of Surgery, University of Miami, 1475 NW 12th Avenue, Suite 4017, Miami, FL 33136. E-mail: [email protected]. 0039-6060/$ - see front matter Ó 2008 Mosby, Inc. All rights reserved. doi:10.1016/j.surg.2008.03.016
most importantly, patient safety make the operating room a less than ideal situation to teach basic laparoscopic skills. Laparoscopic skills training outside the operating room is becoming the standard for best educating surgical residents.1,5,6,10,11,13-15 The 2 general groups of training devices are virtual reality trainers and inanimate box trainers. Both of these devices have been shown to be useful adjuncts in training basic laparoscopic skills.1,2,13,16,17 Despite the automatic scoring systems, the ability to give feedback on overall and individual hand performance, as well as the potential to be used by trainers as metrics for laparoscopic skills,1,4,5 the major drawbacks of virtual reality trainer include the expense and lack of tactile feedback.1,2 Inanimate box trainers are the most commonly used laparoscopic training devices by surgeons in those residency programs that have laparoscopic skills laboratories.18 In fact, only 46% of laboratories had virtual reality equipment, whereas 99% had an inanimate box trainer. Furthermore, some trainees have a higher opinion of box trainers compared with virtual reality trainers.7 SURGERY 345
346 Madan et al
Surgery August 2008
Table I. Instructions given to each resident for each task Peg transfer
Running the rope Pattern cutting Clip/cut the vessel
Extracorporeal knot
Intracorporeal knot
Suturing device
Lift sleeves from left board with left hand, transfer to right hand in midair, and place on right board. When all sleeves have been transferred, reverse the procedure. Start and stop from home base. ‘‘Run’’ a 1/8 3 60’’ rope with marks every 5.’’ Start with both graspers on the green mark, and end when both are on the red mark. Handle the rope only on the marks. Cut premarked 4-cm circle from the piece of dental dam. Start from resting position, and stop when the cut is complete. Place a clip on each of the outside marks on the foam ‘‘vessel,’’ and cut on the center mark between the clips. Start from resting position, and stop when the cut is complete. Place suture through dots on Penrose drain. Tie an extracorporeal knot (3 throws, carefully squared), and cut the suture. Start from resting position with needle loaded. Stop when suture is cut. Place suture through dots on Penrose drain. Tie an intracorporeal sugeons knot (a double throw followed by 2 single throws, carefully squared), and cut the suture. Start from resting position with needle loaded, and stop when suture is cut. Place suture through dots on Penrose drain using suturing device. Tie a surgeon’s knot (a double throw followed by 2 single throws carefully squared), and cut the suture. Start from resting position, and stop when suture is cut.
Each type of trainer has advantages and disadvantages that have been well summarized elsewhere.1 The issues of training residents in laparoscopic techniques are compounded with the issues of the 80-hour work week.19 Because of the restrictions on the resident work week, it is imperative for any type of laparoscopic skills training to be efficient. The difficulty in obtaining resources and the limited amount of resident time deem it necessary to evaluate any laparoscopic skill training to ensure that it offers appropriate skill acquisition. We hypothesized that goal-directed laparoscopic training (GDLT) would result in better skill acquisition than laparoscopic training without goals (LT). MATERIALS AND METHODS During the time of this investigation, the institution where this study was performed had an intensive laparoscopic skill educational program for the residents. All residents were given access to a laparoscopic skills laboratory that included inanimate box and virtual reality trainers. The residents were assigned specific tasks on the virtual reality trainer depending on their postgraduate year. The upper level residents were taken in pairs to a laparoscopic training facility weekly to perform advanced laparoscopic procedures in human cadavers. The second-year general surgery residents underwent a 10-week laparoscopic training course in inanimate box trainers. Since this study was a retrospective examination of a portion of our laparoscopic curriculum, we received an exemption by our Institutional Review Board. The second-year general surgery resident
course was studied. The course was given to each of our second-year residents for 10 weeks. For the first 8 weeks, the residents were taught and practiced a task each week. The residents worked in pairs and were taught by a surgeon trained in minimally invasive techniques. Immediate feedback was given to the resident after each trial. Scores were recorded by the educator and told to the residents. During week 9, the residents were allowed to practice each task. During week 10, the residents were tested on each task. Every resident was given the training schedule and knew they were going to be tested at the end of the curriculum. Attendance was mandatory and was performed in the morning in order to comply with the 80-hour work-week restriction. Residents who were on vacation made up the session either before or after they left for vacation. All residents had the same amount of time in the skills laboratory (2 hours per week for 10 weeks). Most tasks were modified from tasks taken from the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) and were similar to the Fundamentals of Laparoscopic Surgery.20 The tasks included peg transfer, running the rope, pattern cutting, clip/cut vessel, extracorporeal knot, intracorporeal knot, and suture device. Table I demonstrates the instructions given to the residents for each task. The scoring method is displayed in Table II for each task. The scoring method was given to the residents before each exercise. One group of residents (LT) went through the 10-week laparoscopic training course without
Madan et al 347
Surgery Volume 144, Number 2
Table II. Scoring method for each task Peg transfer Penalty Cutoff Score Running the rope Penalty
Cutoff Score Pattern cutting Penalty
Cutoff Score Clip/cut the vessel Penalty Cutoff Score Extracorporeal knot Penalty
Cutoff Score Intracorporeal knot Penalty
Cutoff Score Suturing device Penalty
Cutoff Score
% of sleeves lost from view 5 seconds for each dropped sleep 300 seconds 300—observed time—penalty 5 seconds for grasping outside the marks 10 seconds for dropping the rope 300 seconds 300—observed time—penalty % of circumference that deviates more than 2 mm from perfect circle 300 seconds 300—observed time—penalty 10 seconds for each mm that a clip or the cut is ‘‘off the mark’’ 120 seconds 120—observed time—penalty 5 seconds for each mm is ‘‘off the mark’’ 5 seconds for each mm of gap 10 seconds for a knot that slips 20 seconds for a knot that disrupts 20 seconds for improperly tied knot 420 seconds 420—observed time—penalty 5 seconds for each mm is ‘‘off the mark’’ 5 seconds for each mm of gap 10 seconds for a knot that slips 20 seconds for a knot that disrupts 20 seconds for improperly tied knot 600 seconds 600—observed time—penalty 5 seconds for each mm is ‘‘off the mark’’ 5 seconds for each mm of gap 10 seconds for a knot that slips 20 seconds for a knot that disrupts 20 seconds for improperly tied knot 600 seconds 600—observed time—penalty
goals. One group of residents (GDLT) was given goals to achieve during their course. The 2 groups were not randomized. The LT group was the first year of this study, and the GDLT was the second
Table III. Laparoscopic goals given to the GDLT group to achieve Exercise
Score
Peg transfer Running the rope Pattern cutting Clip/cut vessel Extracorporeal knot Intracorporeal knot Suture device
204 264 239 113 221 510 521
year of this study. The goals were chosen based on high scores from the LT group. We decided to choose the high scores in order to place high standards in laparoscopic skills for the residents. Choosing lower scores may not have set the bar high enough for the residents. In addition, we felt that this method would be easily transferable to the laparoscopic skill curriculum in other programs. The rest of the curriculum between each group was kept identical. The residents were not told anything different between each year. We compared means for all recorded scores, including the practice and test sessions between the GDLT and LT groups. Statistical analysis was performed with nonparametric, 2-tailed Mann-Whitney tests or the Fisher exact test as appropriate via the GraphPad InStat Version 3.05 (San Diego, CA). A P value less than .05 was considered statistically significant. RESULTS There were 8 residents in the LT group and 7 residents in the GDLT. Table III demonstrates the goals that were given to the GDLT group that were arbitrarily assigned after examining the scores from the LT group. The GDLT group had statistically significant higher scores on 7 of the 8 tasks compared with the LT group (P < .02 to P < .0001; Table IV). The GDLT group performed better in the final task, suturing device, than the LT group, but this did not reach statistical significance (451 vs 414; P = .14). More residents in the GDLT group achieved the set goals than in the LT group as shown in Table V. CONCLUSIONS Numerous studies have demonstrated that laparoscopic training helps laparoscopic skills acquisition.1,13,21 Logic dictates that better skills should ultimately translate into better clinical patient outcomes. Unfortunately, no data are available to support or refute this logic. However, our data does demonstrate that setting goals increases laparoscopic skill acquisition.
348 Madan et al
Surgery August 2008
Table IV. Mean (range) scores for each task between the GDLT and the LT Tasks
GDLT
Peg exercise Run the rope Pattern cutting Clip/cut vessel Extracorporeal knot tying Intracorporeal knot tying Suturing device
185 249 223 109 309 433 451
(40–226) (185–278) (132–264) (101–114) (229–375) (63–537) (303–534)
Table V. Percentage of residents that reached goals by group P value
Tasks
GDLT
LT
Peg exercise Run the rope Pattern cutting Clip/cut vessel Extracorporeal knot tying Intracorporeal knot tying Suturing device
100% 57.1% 57.1% 57.1% 100%
12.5% 12.5% 25% 12.5% 12.5%
< = = = <
57.1%
12.5%
= .12
28.6%
12.5%
= .57
.002 .12 .31 .12 .002
This study does have a limitation in its retrospective nature and lack of randomization. The 2 groups of residents were in different years; thus, it could be that the observed differences were caused by some confounding factor that is not obvious. However, during the time period, there were no major changes to the resident rotations, no major change in faculty, and no major change in cases that second-year residents would normally perform. The sample size was relatively low although it was equivalent to many other studies in laparoscopic skill training for residents. However, the fact that the groups were not randomized, in different time periods, and were relatively small may limit the generalizability of our results. In addition, there was a large variability of scores within each group that also needs to be explored. We did not assess baseline laparoscopic skills, resident demographic data, or nonsurgical skills, which is another limitation as they may have some impact on baseline laparoscopic skills although this is controversial.3,22 We also did not examine skill retention (or skill degradation) that has been found to change over time without continual practice.23 Unlike some of our previous studies,1,3 we did not have formal metrics in an animate model to test translation into actual operative procedures or tasks. In addition, we did not analyze whether the observed differences were caused by time versus errors, which could be
LT 160 225 188 105 212 351 414
(18–212) (127–264) (40–239) (94–113) (198–227) (30–510) (110–521)
P value < < < < < < =
.0001 .0001 .0001 .02 .0001 .0001 .14
important in focusing educational efforts. Despite the limitations of this study, our data demonstrate that with the only change in the curriculum being the addition of goals for each task, we were able to increase the average scores of the residents. Many have suggested that proficiency-based training is the best method to train residents in laparoscopic skills.13,16,24,25 Proficiency-based training is usually defined as training until the trainees reach a predefined set level or goal. We agree that setting goals for any type of training seems to be invaluable. The data for the current study demonstrate that residents do better after goals are set. However, one limitation of the study is that no true method exists to determine what should be the goal. Most studies used aggregate scores of ‘‘experts’’ in laparoscopic surgery.13,26 Successful outcomes in the operating room are not determined by only speed and lack of errors. Decisionmaking ability, anatomical and variant knowledge, team work, crisis management, leadership, experience, surgical judgment, and technical skill are important parts of ensuring excellent patient outcomes.27 It is not known what the effect on our results would have been if our goals were not appropriately set. If our goals were set higher, the residents may have performed even better, although some may have performed worse because of their feeling of unrealistic goals. If our goals were set lower, the residents may have performed worse thinking that they had already achieved the goals. Future studies need to examine whether inappropriate goals have any effect on laparoscopic training. The other issue surrounding many studies in proficiency-based training is that it is usually compared with no training at all.13,16,24 The fact that any medical student, resident, or surgeon does better in a subject after being trained should be no surprise. Although proficiency-based training seems like it is a more appropriate method for training, appropriate research has not been performed to compare it with training for a specific amount of time only. In addition, if goals are set
Madan et al 349
Surgery Volume 144, Number 2
too low or the level of initial skill of the trainee is set too high, proficiency-based training may limit the amount of skill acquisition that can occur outside the operating room and negate the benefits or training outside of the operating room. However, our data lend support to the fact that setting goals for residents increases the average score of residents. Setting goals and making them known to the residents may help decrease the training time required for proficiency-based training. Another issue surrounding pure proficiencybased training is that it is difficult to incorporate into the practical world of general surgery training. Ideally, residents would not be able to perform certain procedures until they have passed a certain level of proficiency in virtual trainers, inanimate box trainers, animate models, and the human cadaver model. The expense and manpower necessary for this type of training is impractical. Although residents can score themselves honestly and accurately without direct observation by an independent evaluator,8 when it may affect their ability to perform cases, self-scoring would be less than ideal. Thus, manpower is an issue not only to educate but also to grade the residents. In addition, a laparoscopic training laboratory is expensive. Many general surgery programs do not have one; and those that do may not be used appropriately.18 Since Graduate Medical Education funds from the Center of Medicare services are not necessarily given to the those who have the true responsibility of training residents (ie, general surgery residencies),28 it is difficult for many financially strapped general surgery residency programs to initiate and maintain a laparoscopic training laboratory. In addition, with the 80-hour work week and the advanced laparoscopic requirements set by the RRC,19,29 denying resident participation in procedures that some residencies have limited supply of may be impractical. Future studies need to examine whether GDLT translates into better operative technique and outcomes. Although much has been written about laparoscopic skills acquisition, future research needs to focus on the specifics on how to best train. In fact, a recent study surveyed members of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES).30 The results of this study demonstrated that the SAGES membership felt that the question ‘‘What methods of simulation are most effective in helping surgeons learn techniques and skills for gastrointestinal and endoscopic surgery?’’ was in the top 12 most important research questions in gastrointestinal and endoscopic surgery. We know training helps, but we
are still far from learning what is the ‘‘most effective’’ method to use for training. The lack of consistent resource, facilities, and abilities make it difficult to determine a standard for training residents in laparoscopic procedures. Formal curricula and minimum standards need to be developed and validated. These formal curricula and minimum standards must be adopted by each general surgery residency program to ensure appropriate laparoscopic skills education. Most residency programs have a limited laparoscopic skill laboratory curriculum. This study helps those programs with limited resources to best maximize resident laparoscopic skill acquisition. Whenever training residents in laparoscopic surgery, goals should be set and scores should be given to the residents. GDLT should be used by trainers instead of LT to ensure better laparoscopic skill acquisition. REFERENCES 1. Madan AK, Frantzides CT. Prospecitve, randomized controlled trial of laparoscopic trainers in laparoscopic skills acquistion. Surg Endosc 2007;21:209-13. 2. Madan AK, Frantzides CT. Substituting virtual reality training for inanimate box trainers does not decrease laparoscopic skill acquisition. JSLS 2007;11:87-9. 3. Madan AK, Frantzides CT, Park WC, Tebbit CL, Kumari NV, O’Leary PJ. Predicting baseline laparoscopic surgery skills. Surg Endosc 2005;19:101-3. 4. Madan AK, Frantzides CT, Sasso L. Laparoscopic baseline ability assessment by virtual reality. J Laparoendosc Adv Surg Tech A 2005;15:13-7. 5. Madan AK, Frantzides CT, Shervin N, Tebbit CL. Assessment of individual hand performance in box trainers compared to virtual reality trainers. Am Surg 2003;69:1112-4. 6. Madan AK, Frantzides CT, Tebbit CL, Park WC, Kumari NV, Shervin N. Evaluation of specialized laparoscopic suturing and tying devices. JSLS 2004;8:191-3. 7. Madan AK, Frantzides CT, Tebbit CL, Quiros RM. Participant’s opinions of laparoscopic trainers during basic laparoscopic training courses. Am J Surg 2005;189:758-61. 8. Madan AK, Frantzides CT, Tebbit CL, Shervin N. Self-reported versus observed scores in laparoscopic skills training. Surg Endosc 2005;19:670-2. 9. Rosser JC, Rosser LE, Savalgi RS. Skill acquisition and assessment for laparoscopic surgery. Arch Surg 1997;132:200-4. 10. Champion JK, Hunter J, Trus T, Laycock W. Teaching basic video skills as an aid in laparoscopic suturing. Surg Endosc 1996;10:23-5. 11. Pearson AM, Gallagher AG, Rosser JC, Satava RM. Evaluation of structured and quantitative training methods for teaching intracorporeal knot tying. Surg Endosc 2002;16:130-7. 12. Ali MR, Mowery Y, Kaplan B, DeMaria EJ. Training the novice in laparoscopy. Surg Endosc 2002;16:1732-6. 13. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458-64. 14. Rosser JC, Rosser LE, Savalgi RS. Objective evaluation of a laparoscopic surgical skill program for residents and senior surgeons. Arch Surg 1998;133:657-61.
350 Madan et al
15. Mori T, Hatano N, Maruyama S, Atomi Y. Significance of ‘‘hands-on training’’ in laparoscopic surgery. Surg Endosc 1998;12:256-60. 16. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky D III. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007;193:797-804. 17. Hasson HM KN, Eekhout J. Training simulator for developing laparoscopic skills. JSLS 2001;5:255-65. 18. Korndorffer JR, Stefanidis D, Scott DJ. Laparoscopic skill laboratories: current assessment and a call for resident training standards. Am J Surg 2006;191:17-22. 19. Duty Hours Language. Available from: URL:http://www. acgme.org/acWebsite/dutyHours/dh_Lang703.pdf. 20. Derossis AM, Fried GM, Abrahamowicz M, Sigman HH, Barkun JS, Meakins JL. Development of a model for training and evaluation of laparoscopic skills. Am J Surg 1998; 175:482-7. 21. Satava RM. Accomplishments and challenges of surgical simulation: dawning of the next generation surgical education. Surg Endosc 2001;15:232-41. 22. Madan AK, Harper JH, Frantzides CT, Tichansky DS. Nonsurgical skills do not predict baseline scores in animate box or virtual reality trainers. Surg Endosc. In press.
Surgery August 2008
23. Torkington J, Smith SG, Rees B, Darzi A. The role of the basic surgical skills course in the acquisition and retneniton of laparoscopic skill. Surg Endosc 2001;15:1071-5. 24. Korndorffer JR, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg 2005;201:23-9. 25. Stefanidis D, Korndorffer JR, Markley S, Sierra R, Scott DJ. Proficiency maintenance: impact of ongoing simulator on laparoscopic skill retention. J Am Coll Surg 2006;202: 599-603. 26. Korndorffer JR Jr, Scott DJ, Sierra R, Brunner WC, Dunne JB, Slakey DP, et al. Developing and testing competency levels for laparoscopic skills training. Arch Surg 2005;140:80-4. 27. Mishra A, Catchpole K, Dale T, McCulloch P. The influence of non-technical performance on technical outcome in laparoscopic cholecystectomy. Surg Endosc 2008;22:68-73. 28. Madan AK, Fabian TC, Tichansky DS. Potential financial impact of first assistant billing by surgical residents. Am Surg 2007;73:652-7. 29. Surgery Policy Information. Available from: URL:http:// www.acgme.org/acWebsite/RRC_440/440_policyArchive.asp. 30. Urbach DR, Horvath KD, Baxter NN, Jobe BA, Madam AK, Pryor AD, et al. A research agenda for gastrointestinal and endoscopic surgery. Surg Endosc 2007;21:1518-25.