- Email: [email protected]
Completion of a Novel, Virtual-Reality-Based, Advanced Laparoscopic Curriculum Improves Advanced Laparoscopic Skills in Senior Residents
ORIGINAL REPORTS
Completion of a Novel, Virtual-Reality-Based, Advanced Laparoscopic Curriculum Improves Advanced Laparoscopic Skills in Senior Residents Lucian Panait, MD,*† Nancy J. Hogle,‡ Dennis L. Fowler, MD,‡ Robert L. Bell, MD,* Kurt E. Roberts, MD,* and Andrew J. Duffy, MD* *Section of Gastrointestinal Surgery, Yale School of Medicine, New Haven, Connecticut; †Saint Mary’s Hospital, Waterbury, Connecticut; and ‡College of Physicians and Surgeons, Columbia University, New York, New York INTRODUCTION: Virtual reality simulators contribute to
basic laparoscopic skill acquisition. These trainers have not yet been shown to contribute to the acquisition of more advanced laparoscopic skills as measured by the Fundamentals of Laparoscopic Surgery (FLS). We have customized novel basic and advanced curricula for the LapSim trainer (Surgical Science, Göteborg, Sweden). Successful completion of these programs is required of our residents. We hypothesize that the successful completion of our advanced curriculum will result in the significant improvement of our residents’ advanced laparoscopic skills as measured by the FLS skills scores. METHODS: In all, 23 surgical residents (PGY 1-4), who had
already passed our basic skills curriculum, completed our advanced LapSim curriculum. All individuals underwent FLS skills testing before and after completing the training. Laparoscopic case experience during the training period was documented for all trainees. FLS scores were analyzed by t test and controlled for case experience. RESULTS: Posttraining FLS scores demonstrate a significant increase for all residents from a mean of 57-66 (p ⬍ 0.02), especially for seniors (PGY 3-4): 56-68 (p ⬍ 0.01). The operative laparoscopic case volume ranged from 1-90 (mean, 30) for juniors (PGY 1-2) and 12-76 (mean 50) for seniors during the training period. Junior resident FLS improvement was dependent on case volume during the period of training; residents with less than 30 cases had a mean improvement of 0, whereas those with at least 30 cases had a 15 point improvement (p ⬍ 0.01). Senior resident FLS score improvement was independent of case numbers during the training period.
Correspondence: Inquiries to Andrew J. Duffy, MD, Section of Gastrointestinal Surgery, Yale School of Medicine, 40 Temple Street, Suite 7B, New Haven, CT 06510; fax: (203) 764-9066; e-mail: [email protected]
CONCLUSIONS: Completion of our advanced LapSim curriculum results in improved advanced laparoscopic skills in senior residents as measured by FLS scores. This skill improvement is independent of laparoscopic case experience. Continuing to mandate the use of this skills curriculum should improve our residents’ performance in advanced laparoscopic surgical procedures. (J Surg 68:121-125. © 2011 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: virtual reality, laparoscopic curriculum, tech-
nical proficiency COMPETENCIES: Medical Knowledge, Practice Based Learn-
ing and Improvement, Systems Based Practice
INTRODUCTION The need for skill instruction outside the operating room is well recognized in the modern models of general surgery residency training. To achieve this, skills laboratories are now required of residency programs. Furthermore, important surgical organizations like the Association of Program Directors in Surgery, the American College of Surgeons, and the Society of American Gastroenterology and Endoscopic Surgeons work jointly to develop skills curricula, with the ultimate goal to deliver a National Residency Curriculum, as a convergence of theoretical and practical applications.1,2 However, most skills curricula supported by these organizations use inanimate models, and therefore, trainee performance is difficult to objectively evaluate and track over time. Virtual reality simulators can contribute to basic laparoscopic skill acquisition3,4; however they are not part of any skills curriculum developed and supported by major surgical organizations. This may be caused by many factors, including the relatively large number of models on the market (each one with
Journal of Surgical Education • © 2011 Association of Program Directors in Surgery Published by Elsevier Inc. All rights reserved.
1931-7204/$30.00 doi:10.1016/j.jsurg.2010.10.004
121
its own features), the high price of acquisition and maintenance, and the lack of evidence that training on virtual simulators results in better performance than training using less costly methods. The Fundamentals of Laparoscopic Surgery (FLS) includes a validated skills assessment component that measures performance of both basic and advanced laparoscopic skills.5 To date, studies have not confirmed that training on virtual reality simulators results in advanced skill acquisition, as measured by the FLS. We have developed and implemented the use of both basic and advanced curricula for the LapSim trainer (Surgical Science, Göteborg, Sweden). We require successful completion of both of these curricula by our residents. We have demonstrated previously that completion of our basic laparoscopic curriculum improves basic surgical skills in novices.6 For this study, we hypothesized that successful completion of our advanced curriculum would result in significant improvement of our residents’ advanced laparoscopic skills as measured by FLS.
METHODS The Yale Advanced Laparoscopic Skills Curriculum is a proficiency-based program developed on LapSim that consists of 16 practice modules and 5 examination modules. Most practice and examination modules are derived from the ones used in the Yale Basic Laparoscopy Curriculum but with increased levels of difficulty. The increased difficulty is achieved by altering camera angles, minimizing the error tolerance and the time limit to task completion, and/or altering the size of specific objects in some of the tasks. The practice modules include grasping, cutting, clip applying, lifting and grasping, running the bowel, fine dissection, and precision and speed. Some practice modules are used more than once by using different difficulty levels and emphasizing opposite handedness. The examination modules include the same drills with slightly different parameters, except for running the bowel and precision and speed. Overall, most skills emphasize ambidextrous movements, precision, and hand– eye coordination. Practice modules may be repeated as many times as necessary until proficiency is achieved. The successful completion of all practice modules allows users to take the examination. Only 5 attempts on each task on the examination are allowed. The failure of any 1 task on the examination requires restarting the Yale Advanced Laparoscopic Curriculum from the beginning and completing all practice modules successfully before returning to the examination. The methods of the study were approved by the Institutional Review Board. All residents who had completed the Yale Basic Laparoscopic Skills Curriculum successfully were eligible for the current study. None of the participants was involved in the design of the curriculum. Residents underwent baseline FLS testing after completing the basic curriculum. We then recruited residents to participate in the study and complete the Yale Advanced Laparoscopic Skills Curriculum. The residents 122
were allowed to complete the curriculum at their own pace, and they participated in regular clinical duties, including open and laparoscopic cases, throughout the study. After completing the advanced curriculum successfully, residents again underwent FLS testing. Laparoscopic case experience during the training period was documented for all trainees. Pretraining and posttraining FLS scores were compared for the group of residents as a whole and for subgroups based on laparoscopic case volume during the study. This was achieved by dividing the junior and senior residents into 2 groups each, based on the number of laparoscopic cases performed. The FLS examination scores of the residents who performed less than the mean number of cases were compared with the scores of those who performed more than the mean number. A study outline is depicted in Fig. 1.
RESULTS In all, 23 surgical residents (PGY 1-4) participated in the study voluntarily and all completed the Yale Advanced Laparoscopic Skills Curriculum and the post-training FLS examination successfully. There were 14 junior residents (PGY 1-2) and 9 seniors (PGY 3-4). Pretraining FLS scores for junior residents averaged 58 (range, 6-81), whereas the scores for seniors averaged 56 (range, 36-75). The difference between the juniors’ and seniors’ FLS scores was not statistically significant (p ⬎ 0.05). Posttraining FLS testing demonstrated a significant increase in the mean of the scores for all residents when compared with pretraining scores 57 vs 66 (p ⬍ 0.02). The increase in scores was larger for senior residents than for the group as a whole: 56-68 (p ⬍ 0.01) (Figs. 2 and 3). Laparoscopic case volume was recorded for each resident during the study period. The mean number of laparoscopic cases was 30 (range, 1-90) for junior residents and 50 (range, 12-76) for senior residents. Eight junior residents participated in less than 30 laparoscopic cases, whereas 6 of them participated in more than 30 cases (8:6 junior resident distribution in relationship to the mean number of cases). Five senior residents
FIGURE 1. Study outline.
Journal of Surgical Education • Volume 68/Number 2 • March/April 2011
FLS Scores Before and After Completion of the Advanced Laparoscopic Curriculum (n=23) 90 80 70 60 50 40 30 20 10 0
*
40 30 10
66
57
0 -10
After training
FIGURE 2. FLS scores before and after completion of the Yale Advanced Laparoscopic Curriculum (*p ⬍ 0.05).
participated in less than 50 laparoscopic cases, whereas 4 seniors participated in more than 50 cases (therefore, the senior resident distribution in relationship to the mean number of cases was 5:4). In the case of junior residents, the mean FLS score improvement was significantly higher for the group who performed more than 30 laparoscopic cases during the study period than for the group who performed less than 30 laparoscopic cases during the study period (15 vs 0, p ⫽ 0.01) (Fig. 4). On the contrary, senior residents who performed less than 50 laparoscopic cases during the study had a similar improvement of their FLS scores when compared with the group who performed more than 50 laparoscopic cases (12 vs 14, p ⬎ 0.05) (Fig. 4).
DISCUSSION The development of surgical skills curricula is currently the focus of major surgical organizations and university centers, with the goal to provide structured surgical education through training and repetition outside the operating room. The formula for the ideal skills curriculum is not defined completely; however, we believe it should include reproducible tasks similar to operating room maneuvers and a means to quantify trainee performance and progression objectively. Other factors advocated as important include a cognitive component, formative FLS Scores for Junior and Senior Residents Before and After Completion of the Advanced Laparoscopic Curriculum 100
*
80 60 20
*
20
Before training
40
Mean Score Improvement of FLS Scores Correlated with the Number of Laparoscopic Cases
58
64
56
69
0 Juniors Before training
Seniors After training
FIGURE 3. FLS scores of junior and senior residents before and after completion of the Yale Advanced Laparoscopic Curriculum. Juniors: n ⫽ 14, seniors: n ⫽ 9 (*p ⬍ 0.05).
Juniors < Mean cases
Seniors > Mean cases
FIGURE 4. Mean improvement in the FLS scores of the junior and senior residents adjusted for the number of laparoscopic cases. Juniors: n ⫽ 14, seniors: n ⫽ 9 (*p ⬍ 0.05).
assessment as a type of feedback to the trainee, defined performance targets to motivate learners, and the opportunity for deliberate, repetitive practice.7 Although these curricula use animate models, box trainers, and/or virtual reality simulators, only the latter allows immediate performance feedback and documentation of progress over time, even in the absence of a trainer on site. Despite these advantages, the successful implementation of such curricula into residency programs can be challenging and require a lot of resources.8 Other authors have reported use of structured virtual reality simulator curricula.9 LapSim was used in various studies for curriculum development: Sinha et al.10 showed that repeated performance and completion of a proficiency-based curriculum leads to skill retention at 6 months, especially for senior-level residents. In this study, the authors employed a curriculum based on 7 LapSim tasks, with passing criteria based on the performance scores of an expert laparoscopic surgeon. In a different study, the same authors failed to demonstrate translation of simulator skills to the operating room in a group trained on a LapSim curriculum, compared with a nontrained group.11 However, the authors used a small number of trainees and noncustomized laparoscopic curricula, which may justify their findings. Conversely, Aggarwal et al.12 showed that a virtualreality-based laparoscopic curriculum can help novice surgeons achieve similar levels of performance to expert surgeons. The curricula used in this study employed the factory predefined settings for 7 basic LapSim tasks at difficulty levels easy and medium, and had customized settings for 2 basic tasks at difficulty level hard. Additionally, the curriculum included a procedural module (laparoscopic cholecystectomy). In a different study, Ahlberg et al.13 randomized 13 subjects to either training or nontraining on a simulator curriculum comprising 6 basic LapSim modules at three levels of difficulty. The simulatortrained residents performed laparoscopic cholecystectomies in humans more proficiently than the nontrained group. Other virtual reality simulators were used successfully for curriculum development. Aggarwal et al.14 defined a virtual reality training curricula for full procedural simulation, this time using a different simulator called LAP Mentor (Simbionix
Journal of Surgical Education • Volume 68/Number 2 • March/April 2011
123
Corporation, Cleveland, OH). The authors used a stepwise approach, starting with performance of basic tasks, then procedural tasks, before moving to a fully simulated laparoscopic cholecystectomy. The completion of this curriculum allowed novice surgeons to achieve proficiency levels similar to expert surgeons. A similar stepwise approach was used by the same group when developing a curriculum using a different virtual reality simulator called MIST-VR (Mentice, Gothenburg, Sweden).14 Again, trainees achieved expert-level performance after the successful completion of the curriculum that included tasks at increasing difficulty levels. For the virtual reality to operating room study, Seymour et al.15 demonstrated that proficient performance of 1 task on the MIST-VR (Manipulate and Diathermy, at difficulty level “difficult”) leads to decreased operative errors in laparoscopic cholecystectomies in simulatortrained individuals, compared with the nontrained group. Similarly, another 4-day curriculum on LapSim and Xitact LS500 (Xitact SA, Morges, Switzerland) led to superior performance in laparoscopic cholecystectomies in a trained group compared with a nontrained group.16 Stefanidis et al.17 showed that a proficiency-based curriculum based on 12 MIST-VR tasks and 5 box trainer tasks leads to skill retention and similar simulator scores even months after training was completed. At Yale University, we developed a basic laparoscopic curriculum using the LapSim virtual reality simulator. The successful completion of this curriculum is mandated for all general surgery residents before they can participate in laparoscopic cases into the operating room. A positive experience with this program led us to develop an advanced laparoscopic skill curriculum, which is based on the LapSim unit. The latter is directed toward senior surgical residents, with customized drills that emphasize precise complex ambidextrous movements, eye– hand coordination, and accuracy. The study subjects were familiar with the simulator and have achieved at least a minimum competency of basic laparoscopic skills, as determined by previous completion of the Yale basic Laparoscopy Curriculum.6 As a consequence, at the beginning of the current study, the average FLS scores for junior residents were similar to those of the seniors (58 vs 56, p ⬎ 0.05). Because our advanced curriculum uses a proficiency-based approach, the participants completed it at their own paces. During this time, they also participated in laparoscopic cases in the operating room. We recognized that involvement with laparoscopic operations might have influenced the acquisition of surgical skills for our trainees and introduced another variable when assessing their skills after completing the curriculum. Therefore, we analyzed also the scores of residents based on the number of laparoscopic operations they participated in during the study. An overall improvement in the FLS scores of junior residents was noticed after completion of the LapSim curriculum. A subsequent examination of the data showed that residents who performed a larger number of laparoscopic cases had greater improvement in their scores when compared with the junior residents who performed fewer cases. This suggests that the 124
change in scores is related, at least in part, to the number of cases and not exclusively to the impact of training using the advanced virtual reality curriculum. Senior residents also had significantly higher FLS scores after completion of the LapSim curriculum compared with the baseline scores. However, contrary to the junior residents, the seniors who performed fewer laparoscopic cases during the study period had FLS score improvements comparable with the ones with larger numbers of cases. Therefore, the overall improvement of the seniors’ scores on the FLS examination seems to be related to training using advanced curriculum and less so to their volume of laparoscopic cases. We concluded that the improved scores of senior residents were likely a result of training using the advanced curriculum. This suggests that training using the advanced curriculum is more beneficial for senior residents than for junior residents. Even though most junior residents should have more room to improve and, therefore, might benefit the most from the advanced curriculum, these results suggest otherwise. The difference between the 2 groups may be explained by the fact that senior residents have a better-developed basic skill set that is necessary before a trainee can learn more advanced skills effectively. Therefore, the advanced curriculum, which focuses on precise bilateral hand movement and eye– hand coordination, may be more effective for senior residents. In conclusion, completion of the Yale Advanced Laparoscopic Curriculum results in improved advanced laparoscopic skills in senior residents as measured by FLS skills examination scores. This skill improvement is independent of laparoscopic case experience. In contrast, junior resident acquisition of advanced laparoscopic skills seems to be influenced more by case volume than by the advanced curriculum. The use of this virtual-reality-based skills curriculum for senior resident training has the potential to improve their performance in advanced laparoscopic surgical procedures.
REFERENCES 1. Bell RH. Surgical Council on Resident Education: a new
organization devoted to graduate surgical education. J Am Coll Surg. 2007;204:341-346. 2. Sachdeva AK, Bell RH Jr, Britt LD, et al. National efforts
to reform residency education in surgery. Acad Med. 2007; 82:1200-1210. 3. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR.
Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev. 2009:CD006575. 4. Palter VN, Grantcharov TP. Virtual reality in surgical
skills training. Surg Clin North Am. 2010;90:605-617. 5. Vassiliou MC, Dunkin BJ, Marks JM, Fried GM. FLS
and FES: comprehensive models of training and assessment. Surg Clin North Am. 2010;90:535-558.
Journal of Surgical Education • Volume 68/Number 2 • March/April 2011
6. Panait L, Bell RL, Roberts KE, Duffy AJ. Designing and
13. Ahlberg G, Enochsson L, Gallagher AG, et al. Proficiency-
validating a customized virtual reality-based laparoscopic skills curriculum. J Surg Educ. 2008;65:413-417.
based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007;193:797-804.
7. Stefanidis D, Heniford BT. The formula for a successful
laparoscopic skills curriculum. Arch Surg. 2009;144:7782; discussion:82. 8. Stefanidis D, Acker CE, Swiderski D, Heniford BT,
Greene FL. Challenges during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program. J Surg Educ. 2008;65:4-7. 9. Larsen CR, Soerensen JL, Grantcharov TP, et al. Effect of
virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ. 2009;338:b1802.
14. Aggarwal R, Grantcharov T, Moorthy K, Hance J, Darzi
A. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg. 2006;191:128-133. 15. Seymour NE, Gallagher AG, Roman SA, et al. Virtual
reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236:458-463; Discussion:463-454.
10. Sinha P, Hogle NJ, Fowler DL. Do the laparoscopic skills
16. Schijven MP, Jakimowicz JJ, Broeders IA, Tseng LN. The
of trainees deteriorate over time? Surg Endosc. 2008;22: 2018-2025.
Eindhoven laparoscopic cholecystectomy training course— improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc. 2005;19: 1220-1226.
11. Hogle NJ, Chang L, Strong VE, et al. Validation of lapa-
roscopic surgical skills training outside the operating room: a long road. Surg Endosc. 2009;23:1476-1482. 12. Aggarwal R, Grantcharov TP, Eriksen JR, et al. An evidence-
17. Stefanidis D, Korndorffer JR, Jr, Sierra R, et al. Skill re-
based virtual reality training program for novice laparoscopic surgeons. Ann Surg. 2006;244:310-314.
tention following proficiency-based laparoscopic simulator training. Surgery. 2005;138:165-170.
Journal of Surgical Education • Volume 68/Number 2 • March/April 2011
125
Completion of a Novel, Virtual-Reality-Based, Advanced Laparoscopic Curriculum Improves Advanced Laparoscopic Skills in Senior Residents Lucian Panait, MD,*† Nancy J. Hogle,‡ Dennis L. Fowler, MD,‡ Robert L. Bell, MD,* Kurt E. Roberts, MD,* and Andrew J. Duffy, MD* *Section of Gastrointestinal Surgery, Yale School of Medicine, New Haven, Connecticut; †Saint Mary’s Hospital, Waterbury, Connecticut; and ‡College of Physicians and Surgeons, Columbia University, New York, New York INTRODUCTION: Virtual reality simulators contribute to
basic laparoscopic skill acquisition. These trainers have not yet been shown to contribute to the acquisition of more advanced laparoscopic skills as measured by the Fundamentals of Laparoscopic Surgery (FLS). We have customized novel basic and advanced curricula for the LapSim trainer (Surgical Science, Göteborg, Sweden). Successful completion of these programs is required of our residents. We hypothesize that the successful completion of our advanced curriculum will result in the significant improvement of our residents’ advanced laparoscopic skills as measured by the FLS skills scores. METHODS: In all, 23 surgical residents (PGY 1-4), who had
already passed our basic skills curriculum, completed our advanced LapSim curriculum. All individuals underwent FLS skills testing before and after completing the training. Laparoscopic case experience during the training period was documented for all trainees. FLS scores were analyzed by t test and controlled for case experience. RESULTS: Posttraining FLS scores demonstrate a significant increase for all residents from a mean of 57-66 (p ⬍ 0.02), especially for seniors (PGY 3-4): 56-68 (p ⬍ 0.01). The operative laparoscopic case volume ranged from 1-90 (mean, 30) for juniors (PGY 1-2) and 12-76 (mean 50) for seniors during the training period. Junior resident FLS improvement was dependent on case volume during the period of training; residents with less than 30 cases had a mean improvement of 0, whereas those with at least 30 cases had a 15 point improvement (p ⬍ 0.01). Senior resident FLS score improvement was independent of case numbers during the training period.
Correspondence: Inquiries to Andrew J. Duffy, MD, Section of Gastrointestinal Surgery, Yale School of Medicine, 40 Temple Street, Suite 7B, New Haven, CT 06510; fax: (203) 764-9066; e-mail: [email protected]
CONCLUSIONS: Completion of our advanced LapSim curriculum results in improved advanced laparoscopic skills in senior residents as measured by FLS scores. This skill improvement is independent of laparoscopic case experience. Continuing to mandate the use of this skills curriculum should improve our residents’ performance in advanced laparoscopic surgical procedures. (J Surg 68:121-125. © 2011 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: virtual reality, laparoscopic curriculum, tech-
nical proficiency COMPETENCIES: Medical Knowledge, Practice Based Learn-
ing and Improvement, Systems Based Practice
INTRODUCTION The need for skill instruction outside the operating room is well recognized in the modern models of general surgery residency training. To achieve this, skills laboratories are now required of residency programs. Furthermore, important surgical organizations like the Association of Program Directors in Surgery, the American College of Surgeons, and the Society of American Gastroenterology and Endoscopic Surgeons work jointly to develop skills curricula, with the ultimate goal to deliver a National Residency Curriculum, as a convergence of theoretical and practical applications.1,2 However, most skills curricula supported by these organizations use inanimate models, and therefore, trainee performance is difficult to objectively evaluate and track over time. Virtual reality simulators can contribute to basic laparoscopic skill acquisition3,4; however they are not part of any skills curriculum developed and supported by major surgical organizations. This may be caused by many factors, including the relatively large number of models on the market (each one with
Journal of Surgical Education • © 2011 Association of Program Directors in Surgery Published by Elsevier Inc. All rights reserved.
1931-7204/$30.00 doi:10.1016/j.jsurg.2010.10.004
121
its own features), the high price of acquisition and maintenance, and the lack of evidence that training on virtual simulators results in better performance than training using less costly methods. The Fundamentals of Laparoscopic Surgery (FLS) includes a validated skills assessment component that measures performance of both basic and advanced laparoscopic skills.5 To date, studies have not confirmed that training on virtual reality simulators results in advanced skill acquisition, as measured by the FLS. We have developed and implemented the use of both basic and advanced curricula for the LapSim trainer (Surgical Science, Göteborg, Sweden). We require successful completion of both of these curricula by our residents. We have demonstrated previously that completion of our basic laparoscopic curriculum improves basic surgical skills in novices.6 For this study, we hypothesized that successful completion of our advanced curriculum would result in significant improvement of our residents’ advanced laparoscopic skills as measured by FLS.
METHODS The Yale Advanced Laparoscopic Skills Curriculum is a proficiency-based program developed on LapSim that consists of 16 practice modules and 5 examination modules. Most practice and examination modules are derived from the ones used in the Yale Basic Laparoscopy Curriculum but with increased levels of difficulty. The increased difficulty is achieved by altering camera angles, minimizing the error tolerance and the time limit to task completion, and/or altering the size of specific objects in some of the tasks. The practice modules include grasping, cutting, clip applying, lifting and grasping, running the bowel, fine dissection, and precision and speed. Some practice modules are used more than once by using different difficulty levels and emphasizing opposite handedness. The examination modules include the same drills with slightly different parameters, except for running the bowel and precision and speed. Overall, most skills emphasize ambidextrous movements, precision, and hand– eye coordination. Practice modules may be repeated as many times as necessary until proficiency is achieved. The successful completion of all practice modules allows users to take the examination. Only 5 attempts on each task on the examination are allowed. The failure of any 1 task on the examination requires restarting the Yale Advanced Laparoscopic Curriculum from the beginning and completing all practice modules successfully before returning to the examination. The methods of the study were approved by the Institutional Review Board. All residents who had completed the Yale Basic Laparoscopic Skills Curriculum successfully were eligible for the current study. None of the participants was involved in the design of the curriculum. Residents underwent baseline FLS testing after completing the basic curriculum. We then recruited residents to participate in the study and complete the Yale Advanced Laparoscopic Skills Curriculum. The residents 122
were allowed to complete the curriculum at their own pace, and they participated in regular clinical duties, including open and laparoscopic cases, throughout the study. After completing the advanced curriculum successfully, residents again underwent FLS testing. Laparoscopic case experience during the training period was documented for all trainees. Pretraining and posttraining FLS scores were compared for the group of residents as a whole and for subgroups based on laparoscopic case volume during the study. This was achieved by dividing the junior and senior residents into 2 groups each, based on the number of laparoscopic cases performed. The FLS examination scores of the residents who performed less than the mean number of cases were compared with the scores of those who performed more than the mean number. A study outline is depicted in Fig. 1.
RESULTS In all, 23 surgical residents (PGY 1-4) participated in the study voluntarily and all completed the Yale Advanced Laparoscopic Skills Curriculum and the post-training FLS examination successfully. There were 14 junior residents (PGY 1-2) and 9 seniors (PGY 3-4). Pretraining FLS scores for junior residents averaged 58 (range, 6-81), whereas the scores for seniors averaged 56 (range, 36-75). The difference between the juniors’ and seniors’ FLS scores was not statistically significant (p ⬎ 0.05). Posttraining FLS testing demonstrated a significant increase in the mean of the scores for all residents when compared with pretraining scores 57 vs 66 (p ⬍ 0.02). The increase in scores was larger for senior residents than for the group as a whole: 56-68 (p ⬍ 0.01) (Figs. 2 and 3). Laparoscopic case volume was recorded for each resident during the study period. The mean number of laparoscopic cases was 30 (range, 1-90) for junior residents and 50 (range, 12-76) for senior residents. Eight junior residents participated in less than 30 laparoscopic cases, whereas 6 of them participated in more than 30 cases (8:6 junior resident distribution in relationship to the mean number of cases). Five senior residents
FIGURE 1. Study outline.
Journal of Surgical Education • Volume 68/Number 2 • March/April 2011
FLS Scores Before and After Completion of the Advanced Laparoscopic Curriculum (n=23) 90 80 70 60 50 40 30 20 10 0
*
40 30 10
66
57
0 -10
After training
FIGURE 2. FLS scores before and after completion of the Yale Advanced Laparoscopic Curriculum (*p ⬍ 0.05).
participated in less than 50 laparoscopic cases, whereas 4 seniors participated in more than 50 cases (therefore, the senior resident distribution in relationship to the mean number of cases was 5:4). In the case of junior residents, the mean FLS score improvement was significantly higher for the group who performed more than 30 laparoscopic cases during the study period than for the group who performed less than 30 laparoscopic cases during the study period (15 vs 0, p ⫽ 0.01) (Fig. 4). On the contrary, senior residents who performed less than 50 laparoscopic cases during the study had a similar improvement of their FLS scores when compared with the group who performed more than 50 laparoscopic cases (12 vs 14, p ⬎ 0.05) (Fig. 4).
DISCUSSION The development of surgical skills curricula is currently the focus of major surgical organizations and university centers, with the goal to provide structured surgical education through training and repetition outside the operating room. The formula for the ideal skills curriculum is not defined completely; however, we believe it should include reproducible tasks similar to operating room maneuvers and a means to quantify trainee performance and progression objectively. Other factors advocated as important include a cognitive component, formative FLS Scores for Junior and Senior Residents Before and After Completion of the Advanced Laparoscopic Curriculum 100
*
80 60 20
*
20
Before training
40
Mean Score Improvement of FLS Scores Correlated with the Number of Laparoscopic Cases
58
64
56
69
0 Juniors Before training
Seniors After training
FIGURE 3. FLS scores of junior and senior residents before and after completion of the Yale Advanced Laparoscopic Curriculum. Juniors: n ⫽ 14, seniors: n ⫽ 9 (*p ⬍ 0.05).
Juniors < Mean cases
Seniors > Mean cases
FIGURE 4. Mean improvement in the FLS scores of the junior and senior residents adjusted for the number of laparoscopic cases. Juniors: n ⫽ 14, seniors: n ⫽ 9 (*p ⬍ 0.05).
assessment as a type of feedback to the trainee, defined performance targets to motivate learners, and the opportunity for deliberate, repetitive practice.7 Although these curricula use animate models, box trainers, and/or virtual reality simulators, only the latter allows immediate performance feedback and documentation of progress over time, even in the absence of a trainer on site. Despite these advantages, the successful implementation of such curricula into residency programs can be challenging and require a lot of resources.8 Other authors have reported use of structured virtual reality simulator curricula.9 LapSim was used in various studies for curriculum development: Sinha et al.10 showed that repeated performance and completion of a proficiency-based curriculum leads to skill retention at 6 months, especially for senior-level residents. In this study, the authors employed a curriculum based on 7 LapSim tasks, with passing criteria based on the performance scores of an expert laparoscopic surgeon. In a different study, the same authors failed to demonstrate translation of simulator skills to the operating room in a group trained on a LapSim curriculum, compared with a nontrained group.11 However, the authors used a small number of trainees and noncustomized laparoscopic curricula, which may justify their findings. Conversely, Aggarwal et al.12 showed that a virtualreality-based laparoscopic curriculum can help novice surgeons achieve similar levels of performance to expert surgeons. The curricula used in this study employed the factory predefined settings for 7 basic LapSim tasks at difficulty levels easy and medium, and had customized settings for 2 basic tasks at difficulty level hard. Additionally, the curriculum included a procedural module (laparoscopic cholecystectomy). In a different study, Ahlberg et al.13 randomized 13 subjects to either training or nontraining on a simulator curriculum comprising 6 basic LapSim modules at three levels of difficulty. The simulatortrained residents performed laparoscopic cholecystectomies in humans more proficiently than the nontrained group. Other virtual reality simulators were used successfully for curriculum development. Aggarwal et al.14 defined a virtual reality training curricula for full procedural simulation, this time using a different simulator called LAP Mentor (Simbionix
Journal of Surgical Education • Volume 68/Number 2 • March/April 2011
123
Corporation, Cleveland, OH). The authors used a stepwise approach, starting with performance of basic tasks, then procedural tasks, before moving to a fully simulated laparoscopic cholecystectomy. The completion of this curriculum allowed novice surgeons to achieve proficiency levels similar to expert surgeons. A similar stepwise approach was used by the same group when developing a curriculum using a different virtual reality simulator called MIST-VR (Mentice, Gothenburg, Sweden).14 Again, trainees achieved expert-level performance after the successful completion of the curriculum that included tasks at increasing difficulty levels. For the virtual reality to operating room study, Seymour et al.15 demonstrated that proficient performance of 1 task on the MIST-VR (Manipulate and Diathermy, at difficulty level “difficult”) leads to decreased operative errors in laparoscopic cholecystectomies in simulatortrained individuals, compared with the nontrained group. Similarly, another 4-day curriculum on LapSim and Xitact LS500 (Xitact SA, Morges, Switzerland) led to superior performance in laparoscopic cholecystectomies in a trained group compared with a nontrained group.16 Stefanidis et al.17 showed that a proficiency-based curriculum based on 12 MIST-VR tasks and 5 box trainer tasks leads to skill retention and similar simulator scores even months after training was completed. At Yale University, we developed a basic laparoscopic curriculum using the LapSim virtual reality simulator. The successful completion of this curriculum is mandated for all general surgery residents before they can participate in laparoscopic cases into the operating room. A positive experience with this program led us to develop an advanced laparoscopic skill curriculum, which is based on the LapSim unit. The latter is directed toward senior surgical residents, with customized drills that emphasize precise complex ambidextrous movements, eye– hand coordination, and accuracy. The study subjects were familiar with the simulator and have achieved at least a minimum competency of basic laparoscopic skills, as determined by previous completion of the Yale basic Laparoscopy Curriculum.6 As a consequence, at the beginning of the current study, the average FLS scores for junior residents were similar to those of the seniors (58 vs 56, p ⬎ 0.05). Because our advanced curriculum uses a proficiency-based approach, the participants completed it at their own paces. During this time, they also participated in laparoscopic cases in the operating room. We recognized that involvement with laparoscopic operations might have influenced the acquisition of surgical skills for our trainees and introduced another variable when assessing their skills after completing the curriculum. Therefore, we analyzed also the scores of residents based on the number of laparoscopic operations they participated in during the study. An overall improvement in the FLS scores of junior residents was noticed after completion of the LapSim curriculum. A subsequent examination of the data showed that residents who performed a larger number of laparoscopic cases had greater improvement in their scores when compared with the junior residents who performed fewer cases. This suggests that the 124
change in scores is related, at least in part, to the number of cases and not exclusively to the impact of training using the advanced virtual reality curriculum. Senior residents also had significantly higher FLS scores after completion of the LapSim curriculum compared with the baseline scores. However, contrary to the junior residents, the seniors who performed fewer laparoscopic cases during the study period had FLS score improvements comparable with the ones with larger numbers of cases. Therefore, the overall improvement of the seniors’ scores on the FLS examination seems to be related to training using advanced curriculum and less so to their volume of laparoscopic cases. We concluded that the improved scores of senior residents were likely a result of training using the advanced curriculum. This suggests that training using the advanced curriculum is more beneficial for senior residents than for junior residents. Even though most junior residents should have more room to improve and, therefore, might benefit the most from the advanced curriculum, these results suggest otherwise. The difference between the 2 groups may be explained by the fact that senior residents have a better-developed basic skill set that is necessary before a trainee can learn more advanced skills effectively. Therefore, the advanced curriculum, which focuses on precise bilateral hand movement and eye– hand coordination, may be more effective for senior residents. In conclusion, completion of the Yale Advanced Laparoscopic Curriculum results in improved advanced laparoscopic skills in senior residents as measured by FLS skills examination scores. This skill improvement is independent of laparoscopic case experience. In contrast, junior resident acquisition of advanced laparoscopic skills seems to be influenced more by case volume than by the advanced curriculum. The use of this virtual-reality-based skills curriculum for senior resident training has the potential to improve their performance in advanced laparoscopic surgical procedures.
REFERENCES 1. Bell RH. Surgical Council on Resident Education: a new
organization devoted to graduate surgical education. J Am Coll Surg. 2007;204:341-346. 2. Sachdeva AK, Bell RH Jr, Britt LD, et al. National efforts
to reform residency education in surgery. Acad Med. 2007; 82:1200-1210. 3. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR.
Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev. 2009:CD006575. 4. Palter VN, Grantcharov TP. Virtual reality in surgical
skills training. Surg Clin North Am. 2010;90:605-617. 5. Vassiliou MC, Dunkin BJ, Marks JM, Fried GM. FLS
and FES: comprehensive models of training and assessment. Surg Clin North Am. 2010;90:535-558.
Journal of Surgical Education • Volume 68/Number 2 • March/April 2011
6. Panait L, Bell RL, Roberts KE, Duffy AJ. Designing and
13. Ahlberg G, Enochsson L, Gallagher AG, et al. Proficiency-
validating a customized virtual reality-based laparoscopic skills curriculum. J Surg Educ. 2008;65:413-417.
based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007;193:797-804.
7. Stefanidis D, Heniford BT. The formula for a successful
laparoscopic skills curriculum. Arch Surg. 2009;144:7782; discussion:82. 8. Stefanidis D, Acker CE, Swiderski D, Heniford BT,
Greene FL. Challenges during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program. J Surg Educ. 2008;65:4-7. 9. Larsen CR, Soerensen JL, Grantcharov TP, et al. Effect of
virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ. 2009;338:b1802.
14. Aggarwal R, Grantcharov T, Moorthy K, Hance J, Darzi
A. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg. 2006;191:128-133. 15. Seymour NE, Gallagher AG, Roman SA, et al. Virtual
reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236:458-463; Discussion:463-454.
10. Sinha P, Hogle NJ, Fowler DL. Do the laparoscopic skills
16. Schijven MP, Jakimowicz JJ, Broeders IA, Tseng LN. The
of trainees deteriorate over time? Surg Endosc. 2008;22: 2018-2025.
Eindhoven laparoscopic cholecystectomy training course— improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc. 2005;19: 1220-1226.
11. Hogle NJ, Chang L, Strong VE, et al. Validation of lapa-
roscopic surgical skills training outside the operating room: a long road. Surg Endosc. 2009;23:1476-1482. 12. Aggarwal R, Grantcharov TP, Eriksen JR, et al. An evidence-
17. Stefanidis D, Korndorffer JR, Jr, Sierra R, et al. Skill re-
based virtual reality training program for novice laparoscopic surgeons. Ann Surg. 2006;244:310-314.
tention following proficiency-based laparoscopic simulator training. Surgery. 2005;138:165-170.
Journal of Surgical Education • Volume 68/Number 2 • March/April 2011
125