- Email: [email protected]
Construction of an evidence-based, graduated training curriculum for D-box, a webcam-based laparoscopic basic skills trainer box
The American Journal of Surgery (2012) 203, 768 –775
Surgical Education
Construction of an evidence-based, graduated training curriculum for D-box, a webcam-based laparoscopic basic skills trainer box Anders J. Debes, M.D.a,*, Rajesh Aggarwal, Ph.D.b, Indran Balasundaram, M.D.b, Morten B.J. Jacobsen, Ph.D.c,d a
Department of Surgery, Oestfold Hospital Trust, N-1603 Fredrikstad, Norway; bDepartment of Biosurgery and Surgical Technology, Imperial College Healthcare NHS Trust, London, UK; cDepartment of Research and Development, Oestfold Hospital Trust, Fredrikstad, Norway; dFaculty of Medicine, University of Oslo, Oslo, Norway KEYWORDS: Laparoscopy; Trainer box; Simulation; Surgical training; Education; Curriculum; Validation
Abstract BACKGROUND: Surgical training programs are now including simulators as training tools for teaching laparoscopic surgery. The aim of this study was to develop a standardized, graduated, and evidence-based curriculum for the newly developed D-box (D-box Medical, Lier, Norway) for training basic laparoscopic skills. METHODS: Eighteen interns with no laparoscopic experience completed a training program on the D-box consisting of 8 sessions of 5 tasks with assessment on a sixth task. Performance was measured by the use of 3-dimensional electromagnetic tracking of hand movements, path length, and time taken. Ten experienced surgeons (⬎100 laparoscopic surgeries, median 250) were recruited for establishing benchmark criteria. RESULTS: Significant learning curves were obtained for all construct valid parameters for tasks 4 (P ⬍ .005) and 5 (P ⬍ .005) and reached plateau levels between the fifth and sixth session. Within the 8 sessions of this study, between 50% and 89% of the interns reached benchmark criteria on tasks 4 and 5. CONCLUSIONS: Benchmark criteria and an evidence-based curriculum have been developed for the D-box. The curriculum is aimed at training and assessing surgical novices in basic laparoscopic skills. © 2012 Elsevier Inc. All rights reserved.
When learning to master a new technical skill, most errors and awkwardness occur in the early phase of the learning curve.1 Thus, this phase is most important to overcome in a safe way. Several high-stakes professions with zero tolerance of errors (eg, airline companies, nuclear power plants, and technical environments with advanced human-machine interfaces) have been training and certify* Corresponding author. Tel.: ⫹47 67-90-00-00; fax: ⫹47 67-96-90-40. E-mail address: [email protected] Manuscript received November 20, 2010; revised manuscript July 18, 2011
0002-9610/$ - see front matter © 2012 Elsevier Inc. All rights reserved. doi:10.1016/j.amjsurg.2011.07.022
ing their professionals by use of simulators for many years. The surgical community has adopted the use of simulators, especially after the introduction of laparoscopic surgery. Laparoscopic simulators have now been a part of surgical education for more than 2 decades, and their implementation as adjuncts to traditional surgical training is gaining acceptance. Over the course of this time, several types of simulators have been developed, from simple video trainers to more advanced, virtual reality (VR)-based simulators with haptic feedback and recently also with added full procedural training. Increasing numbers of surgical training programs are now including
A.J. Debes et al.
Evidence-based curriculum for D-box
simulators as training tools for teaching laparoscopic surgery.2 Although synthetic models (box trainers) and VR simulators have been implemented as modalities to train both basic and advanced skills in laparoscopic surgery, evidence of their efficacy has been accumulating. Studies on novice laparoscopists trained on VR simulators and box trainers have shown a shortening of the learning curve, a reduced time taken to complete tasks, and a reduced number of errors.3–7 There is also increasing evidence that the skills learned in a skills laboratory are transferable to the operating room, resulting in improved performance by the surgical trainees and ultimately leading to improved patient safety.8 –11 However, the best modality for training novice laparoscopists is not yet established. Studies comparing VR simulators and trainer boxes show contradictory results,12,13 which may just be an indication of the 2 types being complementary modalities rather than mutually excluding. Surgical training has traditionally taken place in a very random and unstructured way following the Halstedian principle of master apprenticeship in the operating room. This approach may not be the most efficient or feasible in today’s modern surgical practice. There are several reasons for this, including reduced work hours for surgeons leading to reduced case load and also the society and media’s interest in patient safety and preventable errors. This is where simulation, whether it is a synthetic model and/or VR based, is proposed to replace the early phase of the learning curve.14 However, there is no scientifically based consensus on the best and most effective use of simulators in teaching technical skills today, as shown by a recent systematic review.15 The 11 included studies showed a lack of a uniform study design, and the studies were of variable quality. Often the simulation-based training was in addition to traditional training; thus, the effects of the simulators were not easily deducible. There is a need for developing standardized curricula that include both technical and nontechnical skills in an integrated curriculum embedded in a clinical setting to be able to produce safer surgeons faster. The aim of this study was to develop a standardized, graduated, and evidence-based curriculum for the D-box (D-box Medical, Lier, Norway) (Fig. 1) for the initial acquisition of basic laparoscopic skills. Our objective was first to create benchmark proficiency levels by letting experts perform the different tasks and then to test if novices could acquire basic laparoscopic skills and reach proficiency by assessing learning curves and comparing them with the previously set benchmark levels.
Methods Twenty medical and surgical interns with no previous laparoscopic experience and 10 experienced surgical gastrointestinal and urologic consultants, each with an experience of over 100 laparoscopic surgeries (median 250), were
769
Figure 1 D-box (prototype): the D-box trainer used in this study is composed of an aluminum box measuring 32 ⫻ 45 ⫻ 21 cm (width, depth, and height) and weighing 10.5 kg with 5 prefixed port sites (lined with rubber for realistic port resistance) on the top cover. A webcam (Logitech, Freemont, CA) is roof mounted and connected via a firewire cable to a personal computer and a 19-inch liquid crystal display. A joystick from the top of the trainer box manipulates the webcam. Fourteen light-emitting diode lights mounted inside the box provide sufficient lighting. A slidable tray/drawer in front of the trainer enables insertion of interchangeable task boxes. For the updated version: www.d-box.no.
recruited to participate in this study. Eligible subjects received written information about the study including an offer to participate. By directly contacting the study administrator, the subjects were recruited and accepted to participate. The only exclusion criterion was previous experience with any laparoscopic simulator, whether it was VR based or a trainer box. Before training and assessments, all participants completed a questionnaire on age, sex, dominant hand, if they engaged in computer games on a regular basis, if they played a musical instrument, and graduation year. A signed consent form was obtained from all subjects. The Regional Ethics Committee was informed of the study and had no objections given that the scope of this study fell outside their mandate. The study setup consisted of 8 sessions of 5 different tasks dispersed over a maximum of 4 weeks. Each session
770
The American Journal of Surgery, Vol 203, No 6, June 2012
Table 1 Benchmark criteria (median observation of 10 experienced surgeons)
Task 4 (“running gut”) Task 5 (“rubber plate”) Task 6 (“labyrinth”)*
TIME (s)
THMs
TPL (m)
101 224 183
137 264 198
21.1 42.4 29.2
*Task 6 as used as a pre- and post-assessment task.
lasted about 30 to 60 minutes. Before and after the training program, an assessment run on the sixth task was administered. All training was scheduled during normal working hours at the subject’s convenience, providing a favorable dispersed distribution of training opportunities and a realistic setting for training opportunities in a busy surgical department.16,17 A maximum of 2 sessions per day was allowed, with at least 1 hour between the sessions. All training and assessments were performed in a quiet room under standard ergonomic settings. A study administrator was available at all sessions, providing technical assistance and performance measurements. Before the first session, all tasks were demonstrated by the administrator and repeated when necessary. Practice was not allowed between training sessions, and no participants performed any laparoscopic surgery during the training program. Before the preassessment task, the participants did 2 runs of the proposed easiest task to familiarize themselves with the D-box. Performance measurements on the D-box included the time to completed task (TIME), the total number of hand movements (THMs), and the total path length (TPL); they were recorded using the Isotrak II (Polhemus, Colchester, VT) real-time electromagnetic tracking system and analyzed using validated software (ROVIMAS/ICSAD; Imperial College London, London, UK).9,18,19 The content and face validity for the D-box have previously been established. Also, 3 of the 6 tasks have shown construct validity based on TIME, THMs, and TPL (unpublished data, Debes et al, 2012). Benchmark scores were established by calculating the median score for all performances of the experienced surgeons from session 2 (Table 1). The choice of the second session as the basis for the benchmark criteria was an active choice in an attempt to diminish the effect of the familiarization of the trainer box during the first session. Tasks 4 (“running gut”) and 5 (“rubber plate”) had previously shown construct validity; thus, learning curves were generated based on repeated performance of these tasks (Fig. 2). Task 6 (“labyrinth”) in this study was used as the assessment task, and, therefore, a full learning curve was impossible to obtain. However, construct validity has previously been shown for this task (unpublished data, Debes et al, 2012). The most complex task in a selection of tasks will naturally exhibit the longest learning curve. In this study, the tasks were arranged by the authors in order of the assumed
increasing level of difficulty and were kept identical for all sessions and all participants. To be able to generate a graduated curriculum, the tasks should be administered in a mode of ascending level of difficulty. To examine how the subjects perceived the level of difficulty for each task, after the first and last session the subjects were asked to rank the tasks in order of increasing difficulty from easiest to most challenging. These ranking sequences from the first and last training session were also compared for consistency throughout the training sessions.
D-box trainer The present study used 5 tasks for training purposes and the sixth for pre- and post-training assessment (Fig. 3). Task 1 (“peg transfer”) requires sorting 8 colored pegs into 2 different boxes using an alternating left and right instrument. In task 2 (“sorting pegs”), the subjects are to pick up and pass the 8 colored pegs through an eyebolt using both graspers and then sort the pegs according to color into 2 boxes. Task 3 (“donkey stack”) involves stacking 5 wooden sticks simultaneously on the back of a wooden “donkey.” In task 4 (“running gut”), the object is to run a 170-cm-long, 2-cm-wide silk ribbon from 1 side of the task box to the other using both graspers. Task 5 (“rubber plate”) requires picking up and passing a pin through predefined target holes on a movable rubber plate; both hands have to be used because the pin is alternately transferred from left to right or in reverse order at the same time the rubber plate has to be manipulated from side to side. A total of 5 holes were used. In the pre- and postassessment task (task 6 [“labyrinth”]), the subjects had to pick up a peg and pass it through 7 eyebolts in different prefixed directions and angles in a predefined sequence. Then, they had to lift a rubber plate uncovering the 8 eyebolt to pass through and finally thread the pin through the ninth eyebolt in an opposing fashion. Two identical disposable graspers (Endopath; Ethicon Endo-Surgery, Inc) were used by all subjects for all tasks.
Statistical analysis All data were entered in Microsoft Office Excel 2003 (Microsoft, Seattle, WA) and later analyzed using SPSS (version 17.0; SPSS, Chicago, IL). The choice of 10 consultant surgeons and 20 interns was based on previous studies on VR simulators and box trainers. All numeric variables were considered nonparametric. Descriptive statistics included the median and the range of observations for each variable. Learning curve evaluation was undertaken by use of the Friedman test (nonparametric, repeated measures analysis of variance). Multiple comparisons were then made to identify the point of plateau for no further skills improvement. For all tests, the level of significance was set at 5% (P ⬍ .05).
A.J. Debes et al.
Evidence-based curriculum for D-box
Figure 2
771
The tasks available for the D-box.
Results Subjects Eighteen interns (90%) completed the study. Two interns dropped out because of scheduling problems. The median age at inclusion was 30.0 years (range 25– 43), and the male:female ratio was 9:9. Five participants played a musical instrument, and 2 engaged in computer games weekly. Right-handedness was declared by 16 interns, leaving 2 left-handed in the group.
Learning curves Statistically significant learning curves for the interns were shown for TIME (P ⬍ .005), THMs (P ⬍ .005), and TPL (P ⬍ .005) for task 5 (“rubber plate”) and plateaued at the fifth session for TIME and the sixth session for THMs and TPL. The learning curve for task 4 (“running gut”) was also statistical significant for TIME (P ⬍ .005) and plateaued at the fifth session.
To define whether the subjects had achieved expert levels of psychomotor skills, their performance was compared with the performance of the experienced laparoscopic surgeons. For “running gut,” more than 50% of the interns reached benchmark criteria for all variables (TIME, 50%; TPL, 67%; and THMs, 61%) by 8 sessions. For “rubber plate,” all benchmark criteria were achieved by over 66% of the trainees (TIME, 67%; TPL, 89%; and THMs, 84%).
Task sequence The interns ranked the tasks in the following order (from easiest to most challenging): task 1, task 4, task 3, task 2, task 5, and task 6. This same ranking sequence was reported when all participants were asked again after the last session, adding support by proving consistency throughout the training sessions. Given that only tasks 4, 5, and 6 provided construct validity, these were the only tasks relevant for checking the internal ranks of difficulty. Their ranking by the interns also concurred with the increasing number of attempts necessary to reach the plateau of the learning curve for the same tasks.
772
The American Journal of Surgery, Vol 203, No 6, June 2012
Figure 4 Evidence-based training curriculum for D-box (metrics rounded off for simplicity).
Figure 3 criteria.
The learning curves for tasks 4 and 5 with benchmark
Constructing the curriculum The summarized results given in Table 2 were used as the basis for the development of a graded, proficiency-based curriculum. All construct valid metrics were included, and both tasks showed significant learning curves. Using a methodology previously used for constructing curricula for VR simulators,20 –22 we synthesized an evidence-based curriculum for training basic skills on the D-box (Fig. 4). The
Table 2
curriculum consists of 3 parts: (1) familiarization, (2) training to proficiency, and (3) assessment of skills. For familiarization, the trainees perform all 6 tasks in 2 sessions, each at least 1 hour apart and not more than 2 sessions per day. Then, for training, they perform task 4 (“running gut”) and 5 (“rubber plate”) until they reach proficiency levels on 2 consecutive sessions for all metrics to rule out that the results were achieved just by chance. Then, they move on to the assessment of skills (task six [“labyrinth”]), which they perform until they reach proficiency levels once. The structure of this curriculum adheres to the principles proposed by Fitts and Posner23 for the three-stage model of motor learning (ie, cognition, integration, and automation). For the cognition phase, in our curriculum, the trainees receive a demonstration of the D-box and how to complete the tasks. Then, they commence the program with familiarization, performing 2 sessions of all 5 tasks. Afterward, they start training, aiming at achieving the preset proficiency
Summarized results of metrics for the development of the training curriculum
Task 4 (“running gut”) Task 5 (“rubber plate”) Task 6* (“labyrinth”)
Metric
Construct valid
Learning curve
Plateau session
Benchmark level
TIME (s) TIME (s) TPL (m) THMs TIME (s) TPL (m) THMs
✓ ✓ ✓ ✓ ✓ ✓ ✓
✓ ✓ ✓ ✓ NA NA NA
5th 5th 6th 6th NA NA NA
101 224 42.4 264 183 29.2 198
NA ⫽ not available. *Task 6 was used as a pre- and post-assessment task.
A.J. Debes et al.
Evidence-based curriculum for D-box
levels on tasks 4 and 5. In this integrative phase, their motions will at first be erratic and rough, but by repetitive training their performance gets smoother and with increasing flow of motion. During this practice, the specific psychomotor skills necessary in laparoscopic surgery become autonomous (automation phase). This will later enable the trainee to focus on the other aspects of a procedure rather than carrying out the specific movements, thus overcoming the altered tactile and visual feedback of laparoscopy. Reaching this phase is shown by the learning curves plateauing between sessions 4 and 6 in our study, corresponding with results from similar studies.24 After reaching a plateau, the trainees had no statistically significant improvement in skills (“roofing”) measured by hand movements (THMs and TPL) or TIME. Therefore, it is important to highlight that it is the achievement of preset criteria rather than an arbitrary duration or the number of training sessions that determines the completion of the program. This ensures that the trainees are proficient at basic skills and documents that the acquisition of dexterity is successful without prolonging the training unnecessary for some participants.25–27
Comments This study provided significant learning curves for valid parameters for tasks 4 and 5 and also established benchmark criteria for the same parameters using experienced laparoscopic surgeons. Furthermore, we showed learning curves plateauing for the participants reaching proficiency levels. Within the 8 sessions of this study, between 50% and 89% of the subjects met the benchmark criteria on tasks 4 and 5. These evidence-based results were then used to create a graded training and an assessment curriculum for basic laparoscopic skills on the D-box. Our proposed curriculum represents an individual training program tailored to each trainee’s needs and performance levels. In our study setup, after 8 sessions of all tasks, not all trainees could achieve the preset proficiency levels. However, over 50% of the trainees did reach all the preset proficiency levels before finishing all 8 sessions. This means that for some additional training would be necessary, but some trainees would be proficient with fewer sessions and could move on to other challenges. From the literature, we know that in a group of trainees, a number of alternative learning curves will be represented; thus, completion of the training program will be after completing a different number of training sessions. In an article by Grantcharov and Funch-Jensen,28 4 learning curve patterns were identified. Thirty-seven inexperienced surgical residents performed 10 repetitions of 6 tasks on a VR simulator (Procedicus MIstVR; Mentice, Gothenburg, Sweden). As well as showing that some of the participants (5%) were proficient already from the beginning, they also identified a group (8%) who underperformed from the start and had no significant improvement across the training program, thus showing diffi-
773 culties learning basic laparoscopic technique. These subjects would clearly benefit from an individual training program but might also never reach proficiency levels. The remaining 2 groups (total of 87%) showed improvement across the program, but a subgroup of 16% of these could not meet the preset criteria within the 10 sessions. Other studies have shown similar results and patterns of learning curves.29,30 The D-box is a simple video trainer and has no built-in software or assessment measurement other than the time taken to complete the tasks. In this study, we added electromagnetic tracking of hand movements to increase our options for the objective measurement of technical skills. Adding electromagnetic tracking for assessment during a regular, ongoing training program will demand extra manpower, in the form of a trainer or a dedicated person, for analyzing the data and providing relevant feedback, preferably in real time. The tracking device cost about US$2,800. D-box’s advantages are simplicity, portability, and low cost, and in our experience adding electromagnetic tracking (THMs and TPL) did not further add to the construct validity of the D-box beyond TIME (unpublished data, Debes et al, 2012). The current D-box curriculum proposed by this study is neither an exhaustive curriculum nor a standalone curriculum for achieving laparoscopic proficiency beyond basic skills. However, the D-box is a versatile trainer, and new task boxes are being developed continuously. There is also available a laparoscopic suturing task box, which awaits validation for use in new curricula. For more advanced skills, the versatility of the D-box allows the use of cadaveric or animal tissue and other commercially available synthetic models (eg, laparoscopic cholecystectomy models). It would be beneficial to combine the D-box curriculum with other simulators in a more comprehensive curriculum. Recently, VR simulators have become readily more available and have a growing popularity. VR simulators are sophisticated and advanced, and in addition to training basic and advanced skills, some also include modules for full procedural training and they provide real-time objective feedback of performance. By contrast, they are expensive, less realistic, and lack the tactile feeling of the trainer boxes or real tissue. Several studies have shown that both types provide useful training of basic skills4,11,31,32; however, studies comparing trainer boxes and VR simulators show contradictory results, and the “best” modality has yet to be decided.12,13,33 A study performed by Madan and Frantzides34 divided 65 inexperienced medical students into 4 groups. Two groups received a training program either on a box trainer or a VR trainer, a third group received training on both the box trainer and the VR simulator, and finally the control group received no training at all. The group receiving training by both simulator modalities showed best performance although the results were not impressively different from the other trained groups.34 The authors believe that the 2 modalities are complementary and not mutually ex-
774 cluding, and the use of both within the same curriculum would result in an accumulated learning effect for the trainees. In a previous crossover study by the authors on the transferability of skills between MIST-VR versus D-box, 2 groups were assigned to follow either a training program with a final assessment session on the D-box, finishing with a crossover assessment session on the MIST-VR, or the other way around.31 Both groups significantly improved their performance during the training program on their respective trainer modality, but the skills learned on the MIST-VR proved to be transferable to the D-box; however, this was not evident the other way around. One plausible explanation for this was that the VR environment is more challenging and difficult to familiarize and therefore favors the VR-trained group. When creating a graded curriculum for basic laparoscopic skills, D-box training should be performed before training on a VR simulator. The D-box curriculum would serve as one of the components of a full training program in basic laparoscopic skills, thus contributing to shortening the learning curve on real surgeries in a safe way. We will now seek to spread the D-box curriculum to enable external validation in terms of use and feasibility. By moving the first part of the learning curve out of the operating room and into the skills laboratory, we hope to create pretrained novices that are proficient in basic skills and ready to assist on laparoscopic operations. From there, they can progress onto training on more advanced and procedural skills. Further research will focus on evaluating the implementation of this curriculum and the development of more advanced tasks for further training. This study concludes with an evidence-based curriculum for the D-box, showing significant learning curves, and also establishes a set of benchmark criteria (preset proficiency levels) based on experienced surgeons’ performance. As patients or next of kin, we all want competent surgeons. Yet, surgical competency is a multifactorial variable in which technical skills are just one, yet important, ingredient. Future curricula must consequently also seek to include other parts of surgical competency including theoretic knowledge, communicational skills, team leadership, and surgical decision making. A structured curriculum consisting of knowledge-based learning, stepwise technical skills acquisition with predetermined levels of proficiency, together with ongoing objective feedback will provide confident and skilled trainees.
References 1. Satava RM. Accomplishments and challenges of surgical simulation. Surgical endoscopy and other interventional. Surg Endosc 2001;15: 232– 41. 2. Scott DJ, Ritter EM, Tesfay ST, et al. Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training. Surg Endosc 2008;22:1887–93. 3. Chung SY, Landsittel D, Chon CH, et al. Laparoscopic skills training using a webcam trainer. J Urol 2005;173:180 –3.
The American Journal of Surgery, Vol 203, No 6, June 2012 4. Madan AK, Frantzides CT, Shervin N, et al. Assessment of individual hand performance in box trainers compared to virtual reality trainers. Am Surg 2003;69:1112– 4. 5. Pearson AM, Gallagher AG, Rosser JC, et al. Evaluation of structured and quantitative training methods for teaching intracorporeal knot tying. Surgical endoscopy and other interventional. Surg Endosc 2002;16:130 –7. 6. Ali MR, Mowery Y, Kaplan B, et al. Training the novice in laparoscopy. More challenge is better. Surgical endoscopy and other interventional. Surg Endosc 2002;16:1732– 6. 7. Madan AK, Frantzides CT. Substituting virtual reality trainers for inanimate box trainers does not decrease laparoscopic skills acquisition. JSLS 2007;11:87–9. 8. Korndorffer JR Jr, Clayton JL, Tesfay ST, et al. Multicenter construct validity for southwestern laparoscopic videotrainer stations. J Surg Res 2005;128:114 –9. 9. Torkington J, Smith SG, Rees BI, et al. Skill transfer from virtual reality to a real laparoscopic task. Surgical endoscopy and other interventional. Surg Endosc 2001;15:1076 –9. 10. Grantcharov TP, Kristiansen VB, Bendix J, et al. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004;91:146 –50. 11. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458 – 63. 12. Hamilton EC, Scott DJ, Fleming JB, et al. Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skills. Surgical Endosc 2002;16:406 –11. 13. Munz Y, Kumar BD, Moorthy K, et al. Laparoscopic virtual reality and box trainers: is one superior to the other? Surgical endoscopy and other interventional. Surg Endosc 2004;18:485–94. 14. Aggarwal R, Ward J, Balasundaram I, et al. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg 2007;246:771–9. 15. Sturm LP, Windsor JA, Cosman PH, et al. A systematic review of skills transfer after surgical simulation training. Ann Surg 2008;248:166–79. 16. Mackay S, Morgan P, Datta V, et al. Practice distribution in procedural skills training: a randomized controlled trial. Surg Endosc 2002;16:957–61. 17. Verdaasdonk EG, Stassen LP, van Wijk RP, et al. The influence of different training schedules on the learning of psychomotor skills for endoscopic surgery. Surg Endosc 2007;21:214 –9. 18. Smith SG, Torkington J, Brown TJ, et al. Surgical endoscopy and other interventional. Surg Endosc 2002;16:640 –5. 19. Moorthy K, Munz Y, Dosis A, et al. Motion analysis in the training and assessment of minimally invasive surgery. Minim Invasive Ther Allied Technol 2003;12:137– 42. 20. Aggarwal R, Grantcharov TP, Eriksen JR, et al. An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 2006;244:310 – 4. 21. Aggarwal R, Crochet P, Dias A, et al. Development of a virtual reality training curriculum for laparoscopic cholecystectomy. Br J Surg 2009; 96:1086 –93. 22. Aggarwal R, Grantcharov T, Moorthy K, et al. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg 2006;191:128 –33. 23. Fitts PM, Posner MI. Human performance. Belmont, CA: Brooks and Cole; 1967. 24. Gallagher AG, Satava RM. Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Learning curves and reliability measures. Surg Endosc 2002;16:1746 –52. 25. Korndorffer JR Jr, Scott DJ, Sierra R, et al. Developing and testing competency levels for laparoscopic skills training. Arch Surg 2005;140:80–4. 26. Brunner WC, Korndorffer JR Jr, Sierra R, et al. Determining standards for laparoscopic proficiency using virtual reality. Am J Surg 2005;71:29 –35. 27. Brunner WC, Korndorffer JR Jr, Sierra R, et al. Laparoscopic virtual reality training: are 30 repetitions enough? J Surg Res 2004;122:150 – 6. 28. Grantcharov TP, Funch-Jensen P. Can everyone achieve proficiency with the laparoscopic technique? Learning curve patterns in technical skills acquisition. Am J Surg 2009;197:447–9.
A.J. Debes et al.
Evidence-based curriculum for D-box
29. Schijven MP, Jakimowicz J. The learning curve on the xitact LS 500 laparoscopy simulator: profiles of performance. Surg Endosc 2004;18:121–7. 30. Gallagher AG, Lederman AB, McGlade K, et al. Discriminative validity of the minimally invasive surgical trainer in virtual reality (MIst-VR) using criteria levels based on expert performance. Surg Endosc 2004;18:660 –5. 31. Debes AJ, Aggarwal R, Balasundaram I, et al. A tale of two trainers: virtual reality versus a video trainer for acquisition of basic laparoscopic skills. Am J Surg 2010;199:840 –5.
775 32. Madan AK, Frantzides CT, Tebbit C, et al. Self-reported versus observed scores in laparoscopic skills training. Surgical endoscopy and other interventional. Surg Endosc 2005;19:670 –2. 33. Youngblood PL, Srivastava S, Curet M, et al. Comparison of training on two laparoscopic simulators and assessment of skills transfer to surgical performance. J Am Coll Surg 2005;200:546 –51. 34. Madan AK, Frantzides CT. Prospective randomized controlled trial of laparoscopic trainers for basic laparoscopic skills acquisition. Surg Endosc 2007;21:209 –13.
Surgical Education
Construction of an evidence-based, graduated training curriculum for D-box, a webcam-based laparoscopic basic skills trainer box Anders J. Debes, M.D.a,*, Rajesh Aggarwal, Ph.D.b, Indran Balasundaram, M.D.b, Morten B.J. Jacobsen, Ph.D.c,d a
Department of Surgery, Oestfold Hospital Trust, N-1603 Fredrikstad, Norway; bDepartment of Biosurgery and Surgical Technology, Imperial College Healthcare NHS Trust, London, UK; cDepartment of Research and Development, Oestfold Hospital Trust, Fredrikstad, Norway; dFaculty of Medicine, University of Oslo, Oslo, Norway KEYWORDS: Laparoscopy; Trainer box; Simulation; Surgical training; Education; Curriculum; Validation
Abstract BACKGROUND: Surgical training programs are now including simulators as training tools for teaching laparoscopic surgery. The aim of this study was to develop a standardized, graduated, and evidence-based curriculum for the newly developed D-box (D-box Medical, Lier, Norway) for training basic laparoscopic skills. METHODS: Eighteen interns with no laparoscopic experience completed a training program on the D-box consisting of 8 sessions of 5 tasks with assessment on a sixth task. Performance was measured by the use of 3-dimensional electromagnetic tracking of hand movements, path length, and time taken. Ten experienced surgeons (⬎100 laparoscopic surgeries, median 250) were recruited for establishing benchmark criteria. RESULTS: Significant learning curves were obtained for all construct valid parameters for tasks 4 (P ⬍ .005) and 5 (P ⬍ .005) and reached plateau levels between the fifth and sixth session. Within the 8 sessions of this study, between 50% and 89% of the interns reached benchmark criteria on tasks 4 and 5. CONCLUSIONS: Benchmark criteria and an evidence-based curriculum have been developed for the D-box. The curriculum is aimed at training and assessing surgical novices in basic laparoscopic skills. © 2012 Elsevier Inc. All rights reserved.
When learning to master a new technical skill, most errors and awkwardness occur in the early phase of the learning curve.1 Thus, this phase is most important to overcome in a safe way. Several high-stakes professions with zero tolerance of errors (eg, airline companies, nuclear power plants, and technical environments with advanced human-machine interfaces) have been training and certify* Corresponding author. Tel.: ⫹47 67-90-00-00; fax: ⫹47 67-96-90-40. E-mail address: [email protected] Manuscript received November 20, 2010; revised manuscript July 18, 2011
0002-9610/$ - see front matter © 2012 Elsevier Inc. All rights reserved. doi:10.1016/j.amjsurg.2011.07.022
ing their professionals by use of simulators for many years. The surgical community has adopted the use of simulators, especially after the introduction of laparoscopic surgery. Laparoscopic simulators have now been a part of surgical education for more than 2 decades, and their implementation as adjuncts to traditional surgical training is gaining acceptance. Over the course of this time, several types of simulators have been developed, from simple video trainers to more advanced, virtual reality (VR)-based simulators with haptic feedback and recently also with added full procedural training. Increasing numbers of surgical training programs are now including
A.J. Debes et al.
Evidence-based curriculum for D-box
simulators as training tools for teaching laparoscopic surgery.2 Although synthetic models (box trainers) and VR simulators have been implemented as modalities to train both basic and advanced skills in laparoscopic surgery, evidence of their efficacy has been accumulating. Studies on novice laparoscopists trained on VR simulators and box trainers have shown a shortening of the learning curve, a reduced time taken to complete tasks, and a reduced number of errors.3–7 There is also increasing evidence that the skills learned in a skills laboratory are transferable to the operating room, resulting in improved performance by the surgical trainees and ultimately leading to improved patient safety.8 –11 However, the best modality for training novice laparoscopists is not yet established. Studies comparing VR simulators and trainer boxes show contradictory results,12,13 which may just be an indication of the 2 types being complementary modalities rather than mutually excluding. Surgical training has traditionally taken place in a very random and unstructured way following the Halstedian principle of master apprenticeship in the operating room. This approach may not be the most efficient or feasible in today’s modern surgical practice. There are several reasons for this, including reduced work hours for surgeons leading to reduced case load and also the society and media’s interest in patient safety and preventable errors. This is where simulation, whether it is a synthetic model and/or VR based, is proposed to replace the early phase of the learning curve.14 However, there is no scientifically based consensus on the best and most effective use of simulators in teaching technical skills today, as shown by a recent systematic review.15 The 11 included studies showed a lack of a uniform study design, and the studies were of variable quality. Often the simulation-based training was in addition to traditional training; thus, the effects of the simulators were not easily deducible. There is a need for developing standardized curricula that include both technical and nontechnical skills in an integrated curriculum embedded in a clinical setting to be able to produce safer surgeons faster. The aim of this study was to develop a standardized, graduated, and evidence-based curriculum for the D-box (D-box Medical, Lier, Norway) (Fig. 1) for the initial acquisition of basic laparoscopic skills. Our objective was first to create benchmark proficiency levels by letting experts perform the different tasks and then to test if novices could acquire basic laparoscopic skills and reach proficiency by assessing learning curves and comparing them with the previously set benchmark levels.
Methods Twenty medical and surgical interns with no previous laparoscopic experience and 10 experienced surgical gastrointestinal and urologic consultants, each with an experience of over 100 laparoscopic surgeries (median 250), were
769
Figure 1 D-box (prototype): the D-box trainer used in this study is composed of an aluminum box measuring 32 ⫻ 45 ⫻ 21 cm (width, depth, and height) and weighing 10.5 kg with 5 prefixed port sites (lined with rubber for realistic port resistance) on the top cover. A webcam (Logitech, Freemont, CA) is roof mounted and connected via a firewire cable to a personal computer and a 19-inch liquid crystal display. A joystick from the top of the trainer box manipulates the webcam. Fourteen light-emitting diode lights mounted inside the box provide sufficient lighting. A slidable tray/drawer in front of the trainer enables insertion of interchangeable task boxes. For the updated version: www.d-box.no.
recruited to participate in this study. Eligible subjects received written information about the study including an offer to participate. By directly contacting the study administrator, the subjects were recruited and accepted to participate. The only exclusion criterion was previous experience with any laparoscopic simulator, whether it was VR based or a trainer box. Before training and assessments, all participants completed a questionnaire on age, sex, dominant hand, if they engaged in computer games on a regular basis, if they played a musical instrument, and graduation year. A signed consent form was obtained from all subjects. The Regional Ethics Committee was informed of the study and had no objections given that the scope of this study fell outside their mandate. The study setup consisted of 8 sessions of 5 different tasks dispersed over a maximum of 4 weeks. Each session
770
The American Journal of Surgery, Vol 203, No 6, June 2012
Table 1 Benchmark criteria (median observation of 10 experienced surgeons)
Task 4 (“running gut”) Task 5 (“rubber plate”) Task 6 (“labyrinth”)*
TIME (s)
THMs
TPL (m)
101 224 183
137 264 198
21.1 42.4 29.2
*Task 6 as used as a pre- and post-assessment task.
lasted about 30 to 60 minutes. Before and after the training program, an assessment run on the sixth task was administered. All training was scheduled during normal working hours at the subject’s convenience, providing a favorable dispersed distribution of training opportunities and a realistic setting for training opportunities in a busy surgical department.16,17 A maximum of 2 sessions per day was allowed, with at least 1 hour between the sessions. All training and assessments were performed in a quiet room under standard ergonomic settings. A study administrator was available at all sessions, providing technical assistance and performance measurements. Before the first session, all tasks were demonstrated by the administrator and repeated when necessary. Practice was not allowed between training sessions, and no participants performed any laparoscopic surgery during the training program. Before the preassessment task, the participants did 2 runs of the proposed easiest task to familiarize themselves with the D-box. Performance measurements on the D-box included the time to completed task (TIME), the total number of hand movements (THMs), and the total path length (TPL); they were recorded using the Isotrak II (Polhemus, Colchester, VT) real-time electromagnetic tracking system and analyzed using validated software (ROVIMAS/ICSAD; Imperial College London, London, UK).9,18,19 The content and face validity for the D-box have previously been established. Also, 3 of the 6 tasks have shown construct validity based on TIME, THMs, and TPL (unpublished data, Debes et al, 2012). Benchmark scores were established by calculating the median score for all performances of the experienced surgeons from session 2 (Table 1). The choice of the second session as the basis for the benchmark criteria was an active choice in an attempt to diminish the effect of the familiarization of the trainer box during the first session. Tasks 4 (“running gut”) and 5 (“rubber plate”) had previously shown construct validity; thus, learning curves were generated based on repeated performance of these tasks (Fig. 2). Task 6 (“labyrinth”) in this study was used as the assessment task, and, therefore, a full learning curve was impossible to obtain. However, construct validity has previously been shown for this task (unpublished data, Debes et al, 2012). The most complex task in a selection of tasks will naturally exhibit the longest learning curve. In this study, the tasks were arranged by the authors in order of the assumed
increasing level of difficulty and were kept identical for all sessions and all participants. To be able to generate a graduated curriculum, the tasks should be administered in a mode of ascending level of difficulty. To examine how the subjects perceived the level of difficulty for each task, after the first and last session the subjects were asked to rank the tasks in order of increasing difficulty from easiest to most challenging. These ranking sequences from the first and last training session were also compared for consistency throughout the training sessions.
D-box trainer The present study used 5 tasks for training purposes and the sixth for pre- and post-training assessment (Fig. 3). Task 1 (“peg transfer”) requires sorting 8 colored pegs into 2 different boxes using an alternating left and right instrument. In task 2 (“sorting pegs”), the subjects are to pick up and pass the 8 colored pegs through an eyebolt using both graspers and then sort the pegs according to color into 2 boxes. Task 3 (“donkey stack”) involves stacking 5 wooden sticks simultaneously on the back of a wooden “donkey.” In task 4 (“running gut”), the object is to run a 170-cm-long, 2-cm-wide silk ribbon from 1 side of the task box to the other using both graspers. Task 5 (“rubber plate”) requires picking up and passing a pin through predefined target holes on a movable rubber plate; both hands have to be used because the pin is alternately transferred from left to right or in reverse order at the same time the rubber plate has to be manipulated from side to side. A total of 5 holes were used. In the pre- and postassessment task (task 6 [“labyrinth”]), the subjects had to pick up a peg and pass it through 7 eyebolts in different prefixed directions and angles in a predefined sequence. Then, they had to lift a rubber plate uncovering the 8 eyebolt to pass through and finally thread the pin through the ninth eyebolt in an opposing fashion. Two identical disposable graspers (Endopath; Ethicon Endo-Surgery, Inc) were used by all subjects for all tasks.
Statistical analysis All data were entered in Microsoft Office Excel 2003 (Microsoft, Seattle, WA) and later analyzed using SPSS (version 17.0; SPSS, Chicago, IL). The choice of 10 consultant surgeons and 20 interns was based on previous studies on VR simulators and box trainers. All numeric variables were considered nonparametric. Descriptive statistics included the median and the range of observations for each variable. Learning curve evaluation was undertaken by use of the Friedman test (nonparametric, repeated measures analysis of variance). Multiple comparisons were then made to identify the point of plateau for no further skills improvement. For all tests, the level of significance was set at 5% (P ⬍ .05).
A.J. Debes et al.
Evidence-based curriculum for D-box
Figure 2
771
The tasks available for the D-box.
Results Subjects Eighteen interns (90%) completed the study. Two interns dropped out because of scheduling problems. The median age at inclusion was 30.0 years (range 25– 43), and the male:female ratio was 9:9. Five participants played a musical instrument, and 2 engaged in computer games weekly. Right-handedness was declared by 16 interns, leaving 2 left-handed in the group.
Learning curves Statistically significant learning curves for the interns were shown for TIME (P ⬍ .005), THMs (P ⬍ .005), and TPL (P ⬍ .005) for task 5 (“rubber plate”) and plateaued at the fifth session for TIME and the sixth session for THMs and TPL. The learning curve for task 4 (“running gut”) was also statistical significant for TIME (P ⬍ .005) and plateaued at the fifth session.
To define whether the subjects had achieved expert levels of psychomotor skills, their performance was compared with the performance of the experienced laparoscopic surgeons. For “running gut,” more than 50% of the interns reached benchmark criteria for all variables (TIME, 50%; TPL, 67%; and THMs, 61%) by 8 sessions. For “rubber plate,” all benchmark criteria were achieved by over 66% of the trainees (TIME, 67%; TPL, 89%; and THMs, 84%).
Task sequence The interns ranked the tasks in the following order (from easiest to most challenging): task 1, task 4, task 3, task 2, task 5, and task 6. This same ranking sequence was reported when all participants were asked again after the last session, adding support by proving consistency throughout the training sessions. Given that only tasks 4, 5, and 6 provided construct validity, these were the only tasks relevant for checking the internal ranks of difficulty. Their ranking by the interns also concurred with the increasing number of attempts necessary to reach the plateau of the learning curve for the same tasks.
772
The American Journal of Surgery, Vol 203, No 6, June 2012
Figure 4 Evidence-based training curriculum for D-box (metrics rounded off for simplicity).
Figure 3 criteria.
The learning curves for tasks 4 and 5 with benchmark
Constructing the curriculum The summarized results given in Table 2 were used as the basis for the development of a graded, proficiency-based curriculum. All construct valid metrics were included, and both tasks showed significant learning curves. Using a methodology previously used for constructing curricula for VR simulators,20 –22 we synthesized an evidence-based curriculum for training basic skills on the D-box (Fig. 4). The
Table 2
curriculum consists of 3 parts: (1) familiarization, (2) training to proficiency, and (3) assessment of skills. For familiarization, the trainees perform all 6 tasks in 2 sessions, each at least 1 hour apart and not more than 2 sessions per day. Then, for training, they perform task 4 (“running gut”) and 5 (“rubber plate”) until they reach proficiency levels on 2 consecutive sessions for all metrics to rule out that the results were achieved just by chance. Then, they move on to the assessment of skills (task six [“labyrinth”]), which they perform until they reach proficiency levels once. The structure of this curriculum adheres to the principles proposed by Fitts and Posner23 for the three-stage model of motor learning (ie, cognition, integration, and automation). For the cognition phase, in our curriculum, the trainees receive a demonstration of the D-box and how to complete the tasks. Then, they commence the program with familiarization, performing 2 sessions of all 5 tasks. Afterward, they start training, aiming at achieving the preset proficiency
Summarized results of metrics for the development of the training curriculum
Task 4 (“running gut”) Task 5 (“rubber plate”) Task 6* (“labyrinth”)
Metric
Construct valid
Learning curve
Plateau session
Benchmark level
TIME (s) TIME (s) TPL (m) THMs TIME (s) TPL (m) THMs
✓ ✓ ✓ ✓ ✓ ✓ ✓
✓ ✓ ✓ ✓ NA NA NA
5th 5th 6th 6th NA NA NA
101 224 42.4 264 183 29.2 198
NA ⫽ not available. *Task 6 was used as a pre- and post-assessment task.
A.J. Debes et al.
Evidence-based curriculum for D-box
levels on tasks 4 and 5. In this integrative phase, their motions will at first be erratic and rough, but by repetitive training their performance gets smoother and with increasing flow of motion. During this practice, the specific psychomotor skills necessary in laparoscopic surgery become autonomous (automation phase). This will later enable the trainee to focus on the other aspects of a procedure rather than carrying out the specific movements, thus overcoming the altered tactile and visual feedback of laparoscopy. Reaching this phase is shown by the learning curves plateauing between sessions 4 and 6 in our study, corresponding with results from similar studies.24 After reaching a plateau, the trainees had no statistically significant improvement in skills (“roofing”) measured by hand movements (THMs and TPL) or TIME. Therefore, it is important to highlight that it is the achievement of preset criteria rather than an arbitrary duration or the number of training sessions that determines the completion of the program. This ensures that the trainees are proficient at basic skills and documents that the acquisition of dexterity is successful without prolonging the training unnecessary for some participants.25–27
Comments This study provided significant learning curves for valid parameters for tasks 4 and 5 and also established benchmark criteria for the same parameters using experienced laparoscopic surgeons. Furthermore, we showed learning curves plateauing for the participants reaching proficiency levels. Within the 8 sessions of this study, between 50% and 89% of the subjects met the benchmark criteria on tasks 4 and 5. These evidence-based results were then used to create a graded training and an assessment curriculum for basic laparoscopic skills on the D-box. Our proposed curriculum represents an individual training program tailored to each trainee’s needs and performance levels. In our study setup, after 8 sessions of all tasks, not all trainees could achieve the preset proficiency levels. However, over 50% of the trainees did reach all the preset proficiency levels before finishing all 8 sessions. This means that for some additional training would be necessary, but some trainees would be proficient with fewer sessions and could move on to other challenges. From the literature, we know that in a group of trainees, a number of alternative learning curves will be represented; thus, completion of the training program will be after completing a different number of training sessions. In an article by Grantcharov and Funch-Jensen,28 4 learning curve patterns were identified. Thirty-seven inexperienced surgical residents performed 10 repetitions of 6 tasks on a VR simulator (Procedicus MIstVR; Mentice, Gothenburg, Sweden). As well as showing that some of the participants (5%) were proficient already from the beginning, they also identified a group (8%) who underperformed from the start and had no significant improvement across the training program, thus showing diffi-
773 culties learning basic laparoscopic technique. These subjects would clearly benefit from an individual training program but might also never reach proficiency levels. The remaining 2 groups (total of 87%) showed improvement across the program, but a subgroup of 16% of these could not meet the preset criteria within the 10 sessions. Other studies have shown similar results and patterns of learning curves.29,30 The D-box is a simple video trainer and has no built-in software or assessment measurement other than the time taken to complete the tasks. In this study, we added electromagnetic tracking of hand movements to increase our options for the objective measurement of technical skills. Adding electromagnetic tracking for assessment during a regular, ongoing training program will demand extra manpower, in the form of a trainer or a dedicated person, for analyzing the data and providing relevant feedback, preferably in real time. The tracking device cost about US$2,800. D-box’s advantages are simplicity, portability, and low cost, and in our experience adding electromagnetic tracking (THMs and TPL) did not further add to the construct validity of the D-box beyond TIME (unpublished data, Debes et al, 2012). The current D-box curriculum proposed by this study is neither an exhaustive curriculum nor a standalone curriculum for achieving laparoscopic proficiency beyond basic skills. However, the D-box is a versatile trainer, and new task boxes are being developed continuously. There is also available a laparoscopic suturing task box, which awaits validation for use in new curricula. For more advanced skills, the versatility of the D-box allows the use of cadaveric or animal tissue and other commercially available synthetic models (eg, laparoscopic cholecystectomy models). It would be beneficial to combine the D-box curriculum with other simulators in a more comprehensive curriculum. Recently, VR simulators have become readily more available and have a growing popularity. VR simulators are sophisticated and advanced, and in addition to training basic and advanced skills, some also include modules for full procedural training and they provide real-time objective feedback of performance. By contrast, they are expensive, less realistic, and lack the tactile feeling of the trainer boxes or real tissue. Several studies have shown that both types provide useful training of basic skills4,11,31,32; however, studies comparing trainer boxes and VR simulators show contradictory results, and the “best” modality has yet to be decided.12,13,33 A study performed by Madan and Frantzides34 divided 65 inexperienced medical students into 4 groups. Two groups received a training program either on a box trainer or a VR trainer, a third group received training on both the box trainer and the VR simulator, and finally the control group received no training at all. The group receiving training by both simulator modalities showed best performance although the results were not impressively different from the other trained groups.34 The authors believe that the 2 modalities are complementary and not mutually ex-
774 cluding, and the use of both within the same curriculum would result in an accumulated learning effect for the trainees. In a previous crossover study by the authors on the transferability of skills between MIST-VR versus D-box, 2 groups were assigned to follow either a training program with a final assessment session on the D-box, finishing with a crossover assessment session on the MIST-VR, or the other way around.31 Both groups significantly improved their performance during the training program on their respective trainer modality, but the skills learned on the MIST-VR proved to be transferable to the D-box; however, this was not evident the other way around. One plausible explanation for this was that the VR environment is more challenging and difficult to familiarize and therefore favors the VR-trained group. When creating a graded curriculum for basic laparoscopic skills, D-box training should be performed before training on a VR simulator. The D-box curriculum would serve as one of the components of a full training program in basic laparoscopic skills, thus contributing to shortening the learning curve on real surgeries in a safe way. We will now seek to spread the D-box curriculum to enable external validation in terms of use and feasibility. By moving the first part of the learning curve out of the operating room and into the skills laboratory, we hope to create pretrained novices that are proficient in basic skills and ready to assist on laparoscopic operations. From there, they can progress onto training on more advanced and procedural skills. Further research will focus on evaluating the implementation of this curriculum and the development of more advanced tasks for further training. This study concludes with an evidence-based curriculum for the D-box, showing significant learning curves, and also establishes a set of benchmark criteria (preset proficiency levels) based on experienced surgeons’ performance. As patients or next of kin, we all want competent surgeons. Yet, surgical competency is a multifactorial variable in which technical skills are just one, yet important, ingredient. Future curricula must consequently also seek to include other parts of surgical competency including theoretic knowledge, communicational skills, team leadership, and surgical decision making. A structured curriculum consisting of knowledge-based learning, stepwise technical skills acquisition with predetermined levels of proficiency, together with ongoing objective feedback will provide confident and skilled trainees.
References 1. Satava RM. Accomplishments and challenges of surgical simulation. Surgical endoscopy and other interventional. Surg Endosc 2001;15: 232– 41. 2. Scott DJ, Ritter EM, Tesfay ST, et al. Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training. Surg Endosc 2008;22:1887–93. 3. Chung SY, Landsittel D, Chon CH, et al. Laparoscopic skills training using a webcam trainer. J Urol 2005;173:180 –3.
The American Journal of Surgery, Vol 203, No 6, June 2012 4. Madan AK, Frantzides CT, Shervin N, et al. Assessment of individual hand performance in box trainers compared to virtual reality trainers. Am Surg 2003;69:1112– 4. 5. Pearson AM, Gallagher AG, Rosser JC, et al. Evaluation of structured and quantitative training methods for teaching intracorporeal knot tying. Surgical endoscopy and other interventional. Surg Endosc 2002;16:130 –7. 6. Ali MR, Mowery Y, Kaplan B, et al. Training the novice in laparoscopy. More challenge is better. Surgical endoscopy and other interventional. Surg Endosc 2002;16:1732– 6. 7. Madan AK, Frantzides CT. Substituting virtual reality trainers for inanimate box trainers does not decrease laparoscopic skills acquisition. JSLS 2007;11:87–9. 8. Korndorffer JR Jr, Clayton JL, Tesfay ST, et al. Multicenter construct validity for southwestern laparoscopic videotrainer stations. J Surg Res 2005;128:114 –9. 9. Torkington J, Smith SG, Rees BI, et al. Skill transfer from virtual reality to a real laparoscopic task. Surgical endoscopy and other interventional. Surg Endosc 2001;15:1076 –9. 10. Grantcharov TP, Kristiansen VB, Bendix J, et al. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004;91:146 –50. 11. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458 – 63. 12. Hamilton EC, Scott DJ, Fleming JB, et al. Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skills. Surgical Endosc 2002;16:406 –11. 13. Munz Y, Kumar BD, Moorthy K, et al. Laparoscopic virtual reality and box trainers: is one superior to the other? Surgical endoscopy and other interventional. Surg Endosc 2004;18:485–94. 14. Aggarwal R, Ward J, Balasundaram I, et al. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg 2007;246:771–9. 15. Sturm LP, Windsor JA, Cosman PH, et al. A systematic review of skills transfer after surgical simulation training. Ann Surg 2008;248:166–79. 16. Mackay S, Morgan P, Datta V, et al. Practice distribution in procedural skills training: a randomized controlled trial. Surg Endosc 2002;16:957–61. 17. Verdaasdonk EG, Stassen LP, van Wijk RP, et al. The influence of different training schedules on the learning of psychomotor skills for endoscopic surgery. Surg Endosc 2007;21:214 –9. 18. Smith SG, Torkington J, Brown TJ, et al. Surgical endoscopy and other interventional. Surg Endosc 2002;16:640 –5. 19. Moorthy K, Munz Y, Dosis A, et al. Motion analysis in the training and assessment of minimally invasive surgery. Minim Invasive Ther Allied Technol 2003;12:137– 42. 20. Aggarwal R, Grantcharov TP, Eriksen JR, et al. An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 2006;244:310 – 4. 21. Aggarwal R, Crochet P, Dias A, et al. Development of a virtual reality training curriculum for laparoscopic cholecystectomy. Br J Surg 2009; 96:1086 –93. 22. Aggarwal R, Grantcharov T, Moorthy K, et al. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg 2006;191:128 –33. 23. Fitts PM, Posner MI. Human performance. Belmont, CA: Brooks and Cole; 1967. 24. Gallagher AG, Satava RM. Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Learning curves and reliability measures. Surg Endosc 2002;16:1746 –52. 25. Korndorffer JR Jr, Scott DJ, Sierra R, et al. Developing and testing competency levels for laparoscopic skills training. Arch Surg 2005;140:80–4. 26. Brunner WC, Korndorffer JR Jr, Sierra R, et al. Determining standards for laparoscopic proficiency using virtual reality. Am J Surg 2005;71:29 –35. 27. Brunner WC, Korndorffer JR Jr, Sierra R, et al. Laparoscopic virtual reality training: are 30 repetitions enough? J Surg Res 2004;122:150 – 6. 28. Grantcharov TP, Funch-Jensen P. Can everyone achieve proficiency with the laparoscopic technique? Learning curve patterns in technical skills acquisition. Am J Surg 2009;197:447–9.
A.J. Debes et al.
Evidence-based curriculum for D-box
29. Schijven MP, Jakimowicz J. The learning curve on the xitact LS 500 laparoscopy simulator: profiles of performance. Surg Endosc 2004;18:121–7. 30. Gallagher AG, Lederman AB, McGlade K, et al. Discriminative validity of the minimally invasive surgical trainer in virtual reality (MIst-VR) using criteria levels based on expert performance. Surg Endosc 2004;18:660 –5. 31. Debes AJ, Aggarwal R, Balasundaram I, et al. A tale of two trainers: virtual reality versus a video trainer for acquisition of basic laparoscopic skills. Am J Surg 2010;199:840 –5.
775 32. Madan AK, Frantzides CT, Tebbit C, et al. Self-reported versus observed scores in laparoscopic skills training. Surgical endoscopy and other interventional. Surg Endosc 2005;19:670 –2. 33. Youngblood PL, Srivastava S, Curet M, et al. Comparison of training on two laparoscopic simulators and assessment of skills transfer to surgical performance. J Am Coll Surg 2005;200:546 –51. 34. Madan AK, Frantzides CT. Prospective randomized controlled trial of laparoscopic trainers for basic laparoscopic skills acquisition. Surg Endosc 2007;21:209 –13.