- Email: [email protected]
Designing and Validating a Customized Virtual Reality-Based Laparoscopic Skills Curriculum
2008 APDS SPRING MEETING
Designing and Validating a Customized Virtual Reality-Based Laparoscopic Skills Curriculum Lucian Panait, MD,* Robert L. Bell, MD,† Kurt E. Roberts, MD,† and Andrew J. Duffy, MD† *Saint Mary’s Hospital, Waterbury, Connecticut and †Section of Gastrointestinal Surgery, Yale School of Medicine, New Haven, Connecticut OBJECTIVE: We developed and instituted a laparoscopic
CONCLUSIONS: Individual performance in our curricu-
skills curriculum based on a virtual reality simulator, LapSim (Surgical Science, Göteborg, Sweden). Our goal was to improve basic skills in our residents. The hypothesis of this study is that performance in our course will differentiate levels of experience in the training program, establishing construct validity for our curriculum.
lum correlates with the level of training for many drills, which establishes construct validity for this curriculum. Noncontributory drills may need to be revised or removed from the curriculum. Successful completion of this curriculum may lead to improved resident technical performance. (J Surg 65: 413-417. © 2008 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
DESIGN: We designed a novel curriculum that consisted of
17 practice modules and a 7-part examination. All residents who completed the curriculum successfully were included in this study. Performance to complete the examination was analyzed. Data were stratified by level of training.
KEY WORDS: proficiency-based training, resident curricu-
lum, surgical simulation, virtual reality COMPETENCIES: Medical Knowledge, Practice-Based Learning and Improvement, Systems-Based Practice
SETTING: University surgical skill training laboratory. PARTICIPANTS: In all, 29 residents of all levels of training and 3 attending surgeons completed the curriculum. RESULTS: The average number of practice repetitions re-
quired was 243. To complete the examination component, junior residents (R1–R3) required more repetitions than senior residents (R4, R5), 28.3 versus 13.9, respectively (p ⬍ 0.002). Tasks on camera and instrument navigation as well as coordination did not reveal significant differences. The complex grasping task demonstrated significant differences in repetitions required for each level of training: 19.5 attempts for R1, 17.2 for R2, 13 for R3, 8.5 for R4, and 3 for R5 (p ⬍ 0.04). The 2 cutting drills, which required precise use of the left hand, required 7.9 repetitions for junior residents versus 2.7 for senior residents (p ⬍ 0.009). A clip application drill differentiated among junior residents with 39.4, 19.8, and 8.5 repetitions required for R1, R2, and R3, respectively (p ⬍ 0.05). Senior residents performed equivalent to attendings on this drill. A lifting and grasping drill differentiates among junior residents, senior residents, and attendings (p ⬍ 0.03). Correspondence: Inquiries to Andrew J. Duffy, MD, Section of Gastrointestinal Surgery, Yale University, School of Medicine, 40 Temple Street, Suite 7B, New Haven, CT 06510; fax: (203) 764-9066; e-mail: [email protected] Presented at an Educational Workshop at the 2007 APDS Meeting in Washington, DC.
INTRODUCTION Much has changed in surgical education since the days of William Stewart Halsted, MD. He established formal graduate surgical training in the United States as a system in which trainees are mentored in a hospital setting as they steadily gain surgical and patient care responsibility. As stated in the landmark paper, “The training of the surgeon,” his system would “produce not only surgeons, but surgeons of the highest type.”1 Currently, the realities of medical and surgical training require a paradigm shift. Graduate surgical training now must conform to resident work-hour limits, the increased pressure for efficiency and volume in clinical practice for those who are teaching, and the development of new technologies and procedures that often have doubled the number of methods of performing a procedure and introduced new skills to be acquired. Minimally invasive surgical techniques clearly fit into the latter. It is now widely accepted that skills training outside of the operating room is an essential part of residency, which is especially true for minimally invasive surgery (MIS). When compared with open surgery, MIS is more difficult to learn, more difficult to teach, and does not adapt well to old models of training. Furthermore, open surgery experience does not transfer to laparoscopy.2 The challenge resides in the fact that MIS
Journal of Surgical Education • © 2008 Association of Program Directors in Surgery Published by Elsevier Inc. All rights reserved.
1931-7204/08/$30.00 doi:10.1016/j.jsurg.2008.08.001
413
requires different skill sets from open surgery, which include ambidextrous coordination, appreciation of a 3-dimensional environment on a 2-dimensional screen, manipulation of delicate structures with limited tactile feedback, and use of instruments with limited degrees of freedom and unique features such as the fulcrum effect. Various training methods and devices have been developed to replicate the laparoscopic environment. The most commonly used methods are “box trainers,” virtual reality surgical simulators, and animal models. Each method has identifiable strengths and limitations with regard to utility, accurate representation of the surgical field and instrumentation, tracking of trainee performance, cost, availability, and maintenance requirements. From a surgical educator’s perspective, the ideal training system develops basic skills that will be transferred to the operating room and provides a means of objective skill assessment. Virtual reality surgical simulators are the tools that come closest to these goals. The concept of simulation in training has been used since the 1920s in pilot training. During the 80 years since then, these tools have been improved, and training curricula have been refined to the point that flight simulator experience is an acceptable substitute for flight time in training and recredential requirements of pilots. Virtual reality simulation in surgery is not yet as developed or integrated. Seymour et al3 were the first to demonstrate that completing a virtual-reality– based curriculum improves operating room performance. They also established the concept that performance, not fixed repetition, is the standard for skill development. Subsequent studies have validated several virtual reality simulators as tools that can teach laparoscopic psychomotor skills. The role of simulators in objective trainee assessment and in a structured curriculum is also becoming accepted.4 Multiple virtual reality surgical simulators are available on the market, each with its own advantages and disadvantages. The cost of acquisition is a universally shared disadvantage for most, but the design of the software for each allows variable degrees of customization by a program director. The LapSim (Surgical Science, Gothemberg, Sweden) simulates tasks required in MIS and has customizable difficulty and metrics. The tasks preinstalled in the device can be adjusted over a wide range of setups to help focus the training experience. Individual tasks can be combined into user-specific curricula to track an individual trainee’s progression on the device and pattern of use. Previous work demonstrated that performance on LapSim drills can distinguish experts from novices.5 Improvements in software and tasks now allow specific curriculum development that can be studied longitudinally. A novel surgical simulation curriculum was designed at Yale University based on the LapSim exercises. We found that the preset simulator metrics and level designations provided by the manufacturer were not aligned with our goals. We also identified many quirks and tricks inherent in the simulations that did not correlate with surgical skills, but they were required to 414
complete simulations on the preset settings. We endeavored to remove these issues from our curriculum. The primary goal of the curriculum was to improve basic laparoscopic skills in our residents through a program that would allow independent practice. The drills were customized with special emphasis on 2-hand coordination, ambidextrous performance, and depth perception. Metrics and pass–fail settings were customized to emphasize task precision and efficiency. The curriculum was designed by, and the metrics were carefully set, based on the performance of an MIS fellowshiptrained surgeon (A.J.D.). The hypothesis of the study is that the customized Yale University basic laparoscopic skills curriculum can differentiate among years of experience, establishing construct validity for the curriculum.
METHODS The Yale University basic laparoscopic skills curriculum consists of 17 practice modules and 7 examination modules. The initial practice drills were designed to familiarize users with the simulator. The drills become increasingly difficult as users advance into the curriculum. This difficulty is achieved by altering camera angles, minimizing the error tolerance, or altering the size of specific objects in some tasks. The practice modules are represented by camera navigation, instrument navigation, coordination, grasping, cutting, clip applying, lifting and grasping, and precision and speed, with most drills repeating at different levels of difficulty. The examination modules include the same drills (except precision and speed) with slightly different parameters. Overall, most skills emphasize ambidextrous movements and hand– eye coordination. Practice modules may be repeated as many times as necessary to pass them, but all modules must be completed before a trainee can start the examination module. The participants had a maximum of 5 attempts at each examination component. Failure of any 1 component requires restarting the Yale University basic laparoscopic curriculum from the beginning and successfully completing all practice components before returning to the examination. Overall, 32 participants at various levels of training completed the Yale University basic laparoscopy curriculum. None of the participants were involved in the design of the curriculum. The parameters analyzed included the individual performance of each trainee until successful completion of the examination. As the performance requirements for most of the parameters that the device tracks were carefully tied to passing or failing a drill, we found the number of repetitions each trainee required to reach this level of proficiency to be the most useful measure of performance relative to other trainees. We compared the total number of repetitions required for completion of the practice drills and examination drills with data stratified by level of training. Data analysis was performed with Student is t-test analysis of variance.
Journal of Surgical Education • Volume 65/Number 6 • November/December 2008
FIGURE 1. Number of attempts required to complete the complex grasping drill. A significant difference is observed among residents at R3, R4, and R5 levels. Data are mean ⫾ standard error. *p ⬍ 0.05.
RESULTS In all, 29 residents of all levels of training and 3 attending surgeons successfully completed the Yale University basic laparoscopic skill curriculum. The average number of practice task repetitions before examination completion was 238 for residents and 65 for attendings. Completion of the examination module required an average of 28.3 repetitions for junior residents (R1–R3), 13.9 repetitions for senior residents (R4 –R5), and 12.7 repetitions for the attending surgeons. The difference between the junior and senior residents reached statistical significance (p ⬍ 0.002). Individual task analysis did not reveal any significant difference among various levels of training for simple drills, such as camera navigation, instrument navigation, coordination, and simple grasping, which includes all the tasks included in the curriculum to familiarize the user with the device. However, more complex tasks demonstrated more noticeable differences. The complex grasping task demonstrated differences in repetitions required for each level of training: 19.5 attempts for R1, 17.2 for R2, 13 for R3, 8.5 for R4, and 3 for R5 (Fig. 1), with statistical significance achieved between R1–R3 and R4, as well as between R4 and R5 (p ⬍ 0.04). The 2-hand grasping and
FIGURE 2. Number of attempts required to complete the grasping and cutting drill. A significant difference is observed between junior and senior residents. Data are mean ⫾ standard error. *p ⬍ 0.05.
FIGURE 3. Number of attempts required to complete the clip application drill. A significant difference is observed among residents R1, R2, and R3 levels. Data are mean ⫾ standard error. *p ⬍ 0.05.
cutting drill (Fig. 2), which entails precise use of the left hand, required 7.9 repetitions for junior residents versus 2.7 repetitions for senior residents (p ⬍ 0.009). An advanced clip application drill differentiated between junior residents with 39.4, 19.8, and 8.5 repetitions required for R1, R2, and R3, respectively (p ⬍ 0.05). Senior residents performed equivalent to attendings on this drill (Fig. 3). The advanced lifting and grasping drill (Fig. 4) differentiates among junior residents, senior residents, and attendings (p ⬍ 0.03).
DISCUSSION The surgical training in the 21st century is transitioning from the old model that is often summarized as “see one, do one, teach one” to proficiency-based learning and skill development. This change is driven by development of new tools for open and endoscopic procedures, as well as by evolving programmatic and practice requirements, which necessitate training outside of the operating room. Proficiency-based simulation training is accepted as a superior method of teaching laparoscopic skills outside of the operating room.6,7 Its advantages over fixed number of task repetitions include self-paced practice and goal-directed
FIGURE 4. Number of attempts required to complete the lifting and grasping drill. A significant difference is observed among junior residents, senior residents, and attendings. Data are mean ⫾ standard error. *p ⬍ 0.05.
Journal of Surgical Education • Volume 65/Number 6 • November/December 2008
415
learning, which lead to a maximal training benefit.8 Although no consensus is unanimously accepted with regard to the degree of proficiency at various training levels, this method is adopted in many basic laparoscopic skills curricula.3,9,10 Most laparoscopy simulators have factory-preset levels, but many of them have not proven to be useful in skill acquisition. The lack of validated training curricula for use on a system that allows independent learning and skill acquisition is a significant limitation to the use of these devices. Much of the recent literature focuses on virtual reality skill training, but not so much on curriculum development. Although various customized laparoscopic curricula have been developed at other institutions, some of them have limited time frames and strict training schedules.11-14 Furthermore, most studies performed to date try to achieve proficiency in 1 or 2 specific tasks.15,16 We are focusing on a complete curriculum and global basic laparoscopic skill acquisition. The Yale University basic laparoscopic skill curriculum was developed in 2006 on the LapSim virtual reality surgical simulator as a means to achieve a minimum level of basic laparoscopic skill before trainee participation in the operating room. The curriculum consists of 17 practice modules and 7 examination modules, which allow achievement of a standardized degree of minimum proficiency when completed. The proficiency standards are set in relation to the performance of our fellowship-trained minimally invasive surgeons. Starting in September 2006, the curriculum became an integral part of the general surgery residency training at Yale University. Its successful completion is mandatory for residents of all levels, before they can participate in laparoscopic cases in the operating room. The LapSim software stores multiple data points generated during each task repetition, which include the time to complete a task, the instrument path length during task execution for each hand, the number of errors, and other measures of excess or inefficient movement. These individually tracked parameters can be used as custom metrics related to the successful completion of individual tasks. Exceeding any individual parameter, as determined by the program director, forces the user to repeat the task until the standard has been met. Performance feedback is provided to the individual trainee by the simulator immediately on completion of the task. As part of our curriculum development, we developed strict minimum standards for all parameters on all our drills. As the minimum performance parameters are the same for all trainees, the most useful method of tracking individual progression among our trainees was the absolute number of repetitions required to meet these standards. An analysis of our results shows that completion of both curriculum components (practice module and examination module) required significantly different numbers of total task repetitions among novices and experts. Not surprisingly, individual tasks analysis demonstrates that simple laparoscopic drills on the surgical simulator offer minimal value in differentiating among different levels of training. This finding was expected as these drills were used to familiarize the user with the platform and program requirements before the more complex 416
tasks. The performance curve on simple drills plateaus quickly, and minimal skill improvement can be subsequently expected. The industry-provided settings tended to fall into this category and are too easy to promote skill development. Our complex laparoscopic drills showed significant superiority. Complex grasping requires ambidextrous coordination and efficient movement in a virtual reality environment in which camera drifts and both instruments are situated to the right side of the telescope. This drill separated performance across all residency levels of training. Grasping and cutting requires timeefficient, ambidextrous coordination of ultrasonic shears and a grasper while avoiding stretch and tissue damage to fixed structures. The task demonstrated significant differences among junior and senior residents, with the latter performing similar to the attendings. We view this result as a favorable reflection on our training program. Two complex clip application drills are in the curriculum. Their completion requires clip application with either hand, vessel division, and performance of salvage maneuvers, whereas the difficulty is enhanced by camera drift and variable camera angles. Both drills differentiated among junior residents at R1, R2, and R3 levels, whereas senior residents and attendings had similar performance levels. The complex lifting and grasping involves fine motor coordination and depth perception while performing object grasping and careful tissue handling. The drill demonstrated statistically significant differences in performance among junior residents, senior residents, and attendings. In conclusion, individual performance in the Yale University basic laparoscopic skills curriculum correlates with the level of training, which establishes the construct validity of the customized curriculum. Revision of noncontributory drills will improve the efficiency of the training experience and the utility of the curriculum. Successful completion of this curriculum may lead to improved resident technical performance. A longitudinal study to assess resident skill development in this simulator is underway. Ongoing reassessment of training tools and implementation of new assessment methods will help tailor and enhance resident training by coordinating trainee operative performance with skills laboratory training.
REFERENCES 1. Halsted W. The training of the surgeon. Bull Johns Hop-
kins Hospital. 1904;15:267-275. 2. Figert PL, Park AE, Witzke DB, et al. Transfer of training
in acquiring laparoscopic skills. J Am Coll Surg. 2001;193: 533-537. 3. Seymour NE, Gallagher AG, Roman SA, et al. Virtual
reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236:458-463, discussion 363-464.
Journal of Surgical Education • Volume 65/Number 6 • November/December 2008
4. Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills
training and assessment. Br J Surg. 2004;91:1549-1558. 5. Duffy AJ, Hogle NJ, McCarthy H, et al. Construct valid-
ity for the LAPSIM laparoscopic surgical simulator. Surg Endosc. 2005;19:401-405. 6. Brydges R, Kurahashi A, Brümmer V, et al. Developing
criteria for proficiency-based training of surgical technical skills using simulation: changes in performances as a function of training year. J Am Coll Surg. 2008;206:205-211. 7. Van Sickle KR, Ritter EM, McClusky DA III, et al. At-
tempted establishment of proficiency levels for laparoscopic performance on a national scale using simulation: the results from the 2004 SAGES minimally invasive surgical trainer-virtual reality (MIST-VR) learning center study. Surg Endosc. 2007;21:5-10. 8. Korndorffer JJR, Dunne JB, Sierra R, et al. Simulator
training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005; 201:23-29. 9. Brunner WC, Korndorffer JR Jr, Sierra R, et al. Deter-
mining standards for laparoscopic proficiency using virtual reality. Am Surg. 2005;71:29-35. 10. Brunner WC, Korndorffer JR Jr, Sierra R, et al. Laparo-
scopic virtual reality training: are 30 repetitions enough? J Surg Res. 2004;122:150-156.
11. Brunt LM, Halpin VJ, Klingensmith ME, et al. Acceler-
ated skills preparation and assessment for senior medical students entering surgical internship. J Am Coll Surg. 2008;206:897-904; discussion 904-907. 12. Kirby TO, Numnum TM, Kilgore LC, et al. A prospec-
tive evaluation of a simulator-based laparoscopic training program for gynecology residents. J Am Coll Surg. 2008; 206:343-348. 13. Stefanidis D, Acker CE, Swiderski D, et al. Challenges
during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program. J Surg Educ. 2008;65:4-7. 14. Van Hove C, Perry KA, Spight DH, et al. Predictors of
technical skill acquisition among resident trainees in a laparoscopic skills education program. World J Surg. 2008;32:1917-1921. 15. Munz Y, Almoudaris AM, Moorthy K, et al. Curriculum-
based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality. Am J Surg. 2007;193: 774-783. 16. Van Sickle KR, Baghai M, Huang IP, et al. Construct
validity of an objective assessment method for laparoscopic intracorporeal suturing and knot tying. Am J Surg. 2008;196:74-80.
Journal of Surgical Education • Volume 65/Number 6 • November/December 2008
417
Designing and Validating a Customized Virtual Reality-Based Laparoscopic Skills Curriculum Lucian Panait, MD,* Robert L. Bell, MD,† Kurt E. Roberts, MD,† and Andrew J. Duffy, MD† *Saint Mary’s Hospital, Waterbury, Connecticut and †Section of Gastrointestinal Surgery, Yale School of Medicine, New Haven, Connecticut OBJECTIVE: We developed and instituted a laparoscopic
CONCLUSIONS: Individual performance in our curricu-
skills curriculum based on a virtual reality simulator, LapSim (Surgical Science, Göteborg, Sweden). Our goal was to improve basic skills in our residents. The hypothesis of this study is that performance in our course will differentiate levels of experience in the training program, establishing construct validity for our curriculum.
lum correlates with the level of training for many drills, which establishes construct validity for this curriculum. Noncontributory drills may need to be revised or removed from the curriculum. Successful completion of this curriculum may lead to improved resident technical performance. (J Surg 65: 413-417. © 2008 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
DESIGN: We designed a novel curriculum that consisted of
17 practice modules and a 7-part examination. All residents who completed the curriculum successfully were included in this study. Performance to complete the examination was analyzed. Data were stratified by level of training.
KEY WORDS: proficiency-based training, resident curricu-
lum, surgical simulation, virtual reality COMPETENCIES: Medical Knowledge, Practice-Based Learning and Improvement, Systems-Based Practice
SETTING: University surgical skill training laboratory. PARTICIPANTS: In all, 29 residents of all levels of training and 3 attending surgeons completed the curriculum. RESULTS: The average number of practice repetitions re-
quired was 243. To complete the examination component, junior residents (R1–R3) required more repetitions than senior residents (R4, R5), 28.3 versus 13.9, respectively (p ⬍ 0.002). Tasks on camera and instrument navigation as well as coordination did not reveal significant differences. The complex grasping task demonstrated significant differences in repetitions required for each level of training: 19.5 attempts for R1, 17.2 for R2, 13 for R3, 8.5 for R4, and 3 for R5 (p ⬍ 0.04). The 2 cutting drills, which required precise use of the left hand, required 7.9 repetitions for junior residents versus 2.7 for senior residents (p ⬍ 0.009). A clip application drill differentiated among junior residents with 39.4, 19.8, and 8.5 repetitions required for R1, R2, and R3, respectively (p ⬍ 0.05). Senior residents performed equivalent to attendings on this drill. A lifting and grasping drill differentiates among junior residents, senior residents, and attendings (p ⬍ 0.03). Correspondence: Inquiries to Andrew J. Duffy, MD, Section of Gastrointestinal Surgery, Yale University, School of Medicine, 40 Temple Street, Suite 7B, New Haven, CT 06510; fax: (203) 764-9066; e-mail: [email protected] Presented at an Educational Workshop at the 2007 APDS Meeting in Washington, DC.
INTRODUCTION Much has changed in surgical education since the days of William Stewart Halsted, MD. He established formal graduate surgical training in the United States as a system in which trainees are mentored in a hospital setting as they steadily gain surgical and patient care responsibility. As stated in the landmark paper, “The training of the surgeon,” his system would “produce not only surgeons, but surgeons of the highest type.”1 Currently, the realities of medical and surgical training require a paradigm shift. Graduate surgical training now must conform to resident work-hour limits, the increased pressure for efficiency and volume in clinical practice for those who are teaching, and the development of new technologies and procedures that often have doubled the number of methods of performing a procedure and introduced new skills to be acquired. Minimally invasive surgical techniques clearly fit into the latter. It is now widely accepted that skills training outside of the operating room is an essential part of residency, which is especially true for minimally invasive surgery (MIS). When compared with open surgery, MIS is more difficult to learn, more difficult to teach, and does not adapt well to old models of training. Furthermore, open surgery experience does not transfer to laparoscopy.2 The challenge resides in the fact that MIS
Journal of Surgical Education • © 2008 Association of Program Directors in Surgery Published by Elsevier Inc. All rights reserved.
1931-7204/08/$30.00 doi:10.1016/j.jsurg.2008.08.001
413
requires different skill sets from open surgery, which include ambidextrous coordination, appreciation of a 3-dimensional environment on a 2-dimensional screen, manipulation of delicate structures with limited tactile feedback, and use of instruments with limited degrees of freedom and unique features such as the fulcrum effect. Various training methods and devices have been developed to replicate the laparoscopic environment. The most commonly used methods are “box trainers,” virtual reality surgical simulators, and animal models. Each method has identifiable strengths and limitations with regard to utility, accurate representation of the surgical field and instrumentation, tracking of trainee performance, cost, availability, and maintenance requirements. From a surgical educator’s perspective, the ideal training system develops basic skills that will be transferred to the operating room and provides a means of objective skill assessment. Virtual reality surgical simulators are the tools that come closest to these goals. The concept of simulation in training has been used since the 1920s in pilot training. During the 80 years since then, these tools have been improved, and training curricula have been refined to the point that flight simulator experience is an acceptable substitute for flight time in training and recredential requirements of pilots. Virtual reality simulation in surgery is not yet as developed or integrated. Seymour et al3 were the first to demonstrate that completing a virtual-reality– based curriculum improves operating room performance. They also established the concept that performance, not fixed repetition, is the standard for skill development. Subsequent studies have validated several virtual reality simulators as tools that can teach laparoscopic psychomotor skills. The role of simulators in objective trainee assessment and in a structured curriculum is also becoming accepted.4 Multiple virtual reality surgical simulators are available on the market, each with its own advantages and disadvantages. The cost of acquisition is a universally shared disadvantage for most, but the design of the software for each allows variable degrees of customization by a program director. The LapSim (Surgical Science, Gothemberg, Sweden) simulates tasks required in MIS and has customizable difficulty and metrics. The tasks preinstalled in the device can be adjusted over a wide range of setups to help focus the training experience. Individual tasks can be combined into user-specific curricula to track an individual trainee’s progression on the device and pattern of use. Previous work demonstrated that performance on LapSim drills can distinguish experts from novices.5 Improvements in software and tasks now allow specific curriculum development that can be studied longitudinally. A novel surgical simulation curriculum was designed at Yale University based on the LapSim exercises. We found that the preset simulator metrics and level designations provided by the manufacturer were not aligned with our goals. We also identified many quirks and tricks inherent in the simulations that did not correlate with surgical skills, but they were required to 414
complete simulations on the preset settings. We endeavored to remove these issues from our curriculum. The primary goal of the curriculum was to improve basic laparoscopic skills in our residents through a program that would allow independent practice. The drills were customized with special emphasis on 2-hand coordination, ambidextrous performance, and depth perception. Metrics and pass–fail settings were customized to emphasize task precision and efficiency. The curriculum was designed by, and the metrics were carefully set, based on the performance of an MIS fellowshiptrained surgeon (A.J.D.). The hypothesis of the study is that the customized Yale University basic laparoscopic skills curriculum can differentiate among years of experience, establishing construct validity for the curriculum.
METHODS The Yale University basic laparoscopic skills curriculum consists of 17 practice modules and 7 examination modules. The initial practice drills were designed to familiarize users with the simulator. The drills become increasingly difficult as users advance into the curriculum. This difficulty is achieved by altering camera angles, minimizing the error tolerance, or altering the size of specific objects in some tasks. The practice modules are represented by camera navigation, instrument navigation, coordination, grasping, cutting, clip applying, lifting and grasping, and precision and speed, with most drills repeating at different levels of difficulty. The examination modules include the same drills (except precision and speed) with slightly different parameters. Overall, most skills emphasize ambidextrous movements and hand– eye coordination. Practice modules may be repeated as many times as necessary to pass them, but all modules must be completed before a trainee can start the examination module. The participants had a maximum of 5 attempts at each examination component. Failure of any 1 component requires restarting the Yale University basic laparoscopic curriculum from the beginning and successfully completing all practice components before returning to the examination. Overall, 32 participants at various levels of training completed the Yale University basic laparoscopy curriculum. None of the participants were involved in the design of the curriculum. The parameters analyzed included the individual performance of each trainee until successful completion of the examination. As the performance requirements for most of the parameters that the device tracks were carefully tied to passing or failing a drill, we found the number of repetitions each trainee required to reach this level of proficiency to be the most useful measure of performance relative to other trainees. We compared the total number of repetitions required for completion of the practice drills and examination drills with data stratified by level of training. Data analysis was performed with Student is t-test analysis of variance.
Journal of Surgical Education • Volume 65/Number 6 • November/December 2008
FIGURE 1. Number of attempts required to complete the complex grasping drill. A significant difference is observed among residents at R3, R4, and R5 levels. Data are mean ⫾ standard error. *p ⬍ 0.05.
RESULTS In all, 29 residents of all levels of training and 3 attending surgeons successfully completed the Yale University basic laparoscopic skill curriculum. The average number of practice task repetitions before examination completion was 238 for residents and 65 for attendings. Completion of the examination module required an average of 28.3 repetitions for junior residents (R1–R3), 13.9 repetitions for senior residents (R4 –R5), and 12.7 repetitions for the attending surgeons. The difference between the junior and senior residents reached statistical significance (p ⬍ 0.002). Individual task analysis did not reveal any significant difference among various levels of training for simple drills, such as camera navigation, instrument navigation, coordination, and simple grasping, which includes all the tasks included in the curriculum to familiarize the user with the device. However, more complex tasks demonstrated more noticeable differences. The complex grasping task demonstrated differences in repetitions required for each level of training: 19.5 attempts for R1, 17.2 for R2, 13 for R3, 8.5 for R4, and 3 for R5 (Fig. 1), with statistical significance achieved between R1–R3 and R4, as well as between R4 and R5 (p ⬍ 0.04). The 2-hand grasping and
FIGURE 2. Number of attempts required to complete the grasping and cutting drill. A significant difference is observed between junior and senior residents. Data are mean ⫾ standard error. *p ⬍ 0.05.
FIGURE 3. Number of attempts required to complete the clip application drill. A significant difference is observed among residents R1, R2, and R3 levels. Data are mean ⫾ standard error. *p ⬍ 0.05.
cutting drill (Fig. 2), which entails precise use of the left hand, required 7.9 repetitions for junior residents versus 2.7 repetitions for senior residents (p ⬍ 0.009). An advanced clip application drill differentiated between junior residents with 39.4, 19.8, and 8.5 repetitions required for R1, R2, and R3, respectively (p ⬍ 0.05). Senior residents performed equivalent to attendings on this drill (Fig. 3). The advanced lifting and grasping drill (Fig. 4) differentiates among junior residents, senior residents, and attendings (p ⬍ 0.03).
DISCUSSION The surgical training in the 21st century is transitioning from the old model that is often summarized as “see one, do one, teach one” to proficiency-based learning and skill development. This change is driven by development of new tools for open and endoscopic procedures, as well as by evolving programmatic and practice requirements, which necessitate training outside of the operating room. Proficiency-based simulation training is accepted as a superior method of teaching laparoscopic skills outside of the operating room.6,7 Its advantages over fixed number of task repetitions include self-paced practice and goal-directed
FIGURE 4. Number of attempts required to complete the lifting and grasping drill. A significant difference is observed among junior residents, senior residents, and attendings. Data are mean ⫾ standard error. *p ⬍ 0.05.
Journal of Surgical Education • Volume 65/Number 6 • November/December 2008
415
learning, which lead to a maximal training benefit.8 Although no consensus is unanimously accepted with regard to the degree of proficiency at various training levels, this method is adopted in many basic laparoscopic skills curricula.3,9,10 Most laparoscopy simulators have factory-preset levels, but many of them have not proven to be useful in skill acquisition. The lack of validated training curricula for use on a system that allows independent learning and skill acquisition is a significant limitation to the use of these devices. Much of the recent literature focuses on virtual reality skill training, but not so much on curriculum development. Although various customized laparoscopic curricula have been developed at other institutions, some of them have limited time frames and strict training schedules.11-14 Furthermore, most studies performed to date try to achieve proficiency in 1 or 2 specific tasks.15,16 We are focusing on a complete curriculum and global basic laparoscopic skill acquisition. The Yale University basic laparoscopic skill curriculum was developed in 2006 on the LapSim virtual reality surgical simulator as a means to achieve a minimum level of basic laparoscopic skill before trainee participation in the operating room. The curriculum consists of 17 practice modules and 7 examination modules, which allow achievement of a standardized degree of minimum proficiency when completed. The proficiency standards are set in relation to the performance of our fellowship-trained minimally invasive surgeons. Starting in September 2006, the curriculum became an integral part of the general surgery residency training at Yale University. Its successful completion is mandatory for residents of all levels, before they can participate in laparoscopic cases in the operating room. The LapSim software stores multiple data points generated during each task repetition, which include the time to complete a task, the instrument path length during task execution for each hand, the number of errors, and other measures of excess or inefficient movement. These individually tracked parameters can be used as custom metrics related to the successful completion of individual tasks. Exceeding any individual parameter, as determined by the program director, forces the user to repeat the task until the standard has been met. Performance feedback is provided to the individual trainee by the simulator immediately on completion of the task. As part of our curriculum development, we developed strict minimum standards for all parameters on all our drills. As the minimum performance parameters are the same for all trainees, the most useful method of tracking individual progression among our trainees was the absolute number of repetitions required to meet these standards. An analysis of our results shows that completion of both curriculum components (practice module and examination module) required significantly different numbers of total task repetitions among novices and experts. Not surprisingly, individual tasks analysis demonstrates that simple laparoscopic drills on the surgical simulator offer minimal value in differentiating among different levels of training. This finding was expected as these drills were used to familiarize the user with the platform and program requirements before the more complex 416
tasks. The performance curve on simple drills plateaus quickly, and minimal skill improvement can be subsequently expected. The industry-provided settings tended to fall into this category and are too easy to promote skill development. Our complex laparoscopic drills showed significant superiority. Complex grasping requires ambidextrous coordination and efficient movement in a virtual reality environment in which camera drifts and both instruments are situated to the right side of the telescope. This drill separated performance across all residency levels of training. Grasping and cutting requires timeefficient, ambidextrous coordination of ultrasonic shears and a grasper while avoiding stretch and tissue damage to fixed structures. The task demonstrated significant differences among junior and senior residents, with the latter performing similar to the attendings. We view this result as a favorable reflection on our training program. Two complex clip application drills are in the curriculum. Their completion requires clip application with either hand, vessel division, and performance of salvage maneuvers, whereas the difficulty is enhanced by camera drift and variable camera angles. Both drills differentiated among junior residents at R1, R2, and R3 levels, whereas senior residents and attendings had similar performance levels. The complex lifting and grasping involves fine motor coordination and depth perception while performing object grasping and careful tissue handling. The drill demonstrated statistically significant differences in performance among junior residents, senior residents, and attendings. In conclusion, individual performance in the Yale University basic laparoscopic skills curriculum correlates with the level of training, which establishes the construct validity of the customized curriculum. Revision of noncontributory drills will improve the efficiency of the training experience and the utility of the curriculum. Successful completion of this curriculum may lead to improved resident technical performance. A longitudinal study to assess resident skill development in this simulator is underway. Ongoing reassessment of training tools and implementation of new assessment methods will help tailor and enhance resident training by coordinating trainee operative performance with skills laboratory training.
REFERENCES 1. Halsted W. The training of the surgeon. Bull Johns Hop-
kins Hospital. 1904;15:267-275. 2. Figert PL, Park AE, Witzke DB, et al. Transfer of training
in acquiring laparoscopic skills. J Am Coll Surg. 2001;193: 533-537. 3. Seymour NE, Gallagher AG, Roman SA, et al. Virtual
reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236:458-463, discussion 363-464.
Journal of Surgical Education • Volume 65/Number 6 • November/December 2008
4. Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills
training and assessment. Br J Surg. 2004;91:1549-1558. 5. Duffy AJ, Hogle NJ, McCarthy H, et al. Construct valid-
ity for the LAPSIM laparoscopic surgical simulator. Surg Endosc. 2005;19:401-405. 6. Brydges R, Kurahashi A, Brümmer V, et al. Developing
criteria for proficiency-based training of surgical technical skills using simulation: changes in performances as a function of training year. J Am Coll Surg. 2008;206:205-211. 7. Van Sickle KR, Ritter EM, McClusky DA III, et al. At-
tempted establishment of proficiency levels for laparoscopic performance on a national scale using simulation: the results from the 2004 SAGES minimally invasive surgical trainer-virtual reality (MIST-VR) learning center study. Surg Endosc. 2007;21:5-10. 8. Korndorffer JJR, Dunne JB, Sierra R, et al. Simulator
training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005; 201:23-29. 9. Brunner WC, Korndorffer JR Jr, Sierra R, et al. Deter-
mining standards for laparoscopic proficiency using virtual reality. Am Surg. 2005;71:29-35. 10. Brunner WC, Korndorffer JR Jr, Sierra R, et al. Laparo-
scopic virtual reality training: are 30 repetitions enough? J Surg Res. 2004;122:150-156.
11. Brunt LM, Halpin VJ, Klingensmith ME, et al. Acceler-
ated skills preparation and assessment for senior medical students entering surgical internship. J Am Coll Surg. 2008;206:897-904; discussion 904-907. 12. Kirby TO, Numnum TM, Kilgore LC, et al. A prospec-
tive evaluation of a simulator-based laparoscopic training program for gynecology residents. J Am Coll Surg. 2008; 206:343-348. 13. Stefanidis D, Acker CE, Swiderski D, et al. Challenges
during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program. J Surg Educ. 2008;65:4-7. 14. Van Hove C, Perry KA, Spight DH, et al. Predictors of
technical skill acquisition among resident trainees in a laparoscopic skills education program. World J Surg. 2008;32:1917-1921. 15. Munz Y, Almoudaris AM, Moorthy K, et al. Curriculum-
based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality. Am J Surg. 2007;193: 774-783. 16. Van Sickle KR, Baghai M, Huang IP, et al. Construct
validity of an objective assessment method for laparoscopic intracorporeal suturing and knot tying. Am J Surg. 2008;196:74-80.
Journal of Surgical Education • Volume 65/Number 6 • November/December 2008
417