- Email: [email protected]
Training in surgical oncology – The role of VR simulation
Surgical Oncology 20 (2011) 134e139
Contents lists available at ScienceDirect
Surgical Oncology journal homepage: www.elsevier.com/locate/suronc
Training in surgical oncology e The role of VR simulation T.M. Lewis a, *, R. Aggarwal a,1, N. Rajaretnam a, T.P. Grantcharov b, A. Darzi a a b
Department of Cancer and Surgery, Room 1029, 10th Floor, QEQM, St. Marys hospital Imperial College London, UK Division of General Surgery, St Michael’s Hospital, Toronto, ON, Canada
a b s t r a c t Keywords: Laparoscopic Training simulation Virtual reality
There have been dramatic changes in surgical training over the past two decades which have resulted in a number of concerns for the development of future surgeons. Changes in the structure of cancer services, working hour restrictions and a commitment to patient safety has led to a reduction in training opportunities that are available to the surgeon in training. Simulation and in particular virtual reality (VR) simulation has been heralded as an effective adjunct to surgical training. Advances in VR simulation has allowed trainees to practice realistic full length procedures in a safe and controlled environment, where mistakes are permitted and can be used as learning points. There is considerable evidence to demonstrate that the VR simulation can be used to enhance technical skills and improve operating room performance. Future work should focus on the cost effectiveness and predictive validity of VR simulation, which in turn would increase the uptake of simulation and enhance surgical training. Ó 2011 Elsevier Ltd. All rights reserved.
Introduction Over the past decade, there has been a change in surgical training that will impact the development of future surgical oncologists. The essential attributes of a surgical oncologist are proficiency in the specific operations required for the diagnosis and management, either cure or palliation, for solid tumours [1]. Previously, a fully trained general surgeon would have sufficient technical skill, knowledge and experience, to be able to approach the surgical management of a solid tumour in a precise and safe manner, which would result in the best possible management of the patient [2]. However, this level of proficiency may be difficult to reach for current surgical trainees as over the past decade there has been a reduction in the training opportunities in cancer surgery [3]. The reasons for this include the limit on trainees working hours, advances in nonsurgical management of cancer and surgical technology, and in the United Kingdom, the centralisation of cancer services [4,5]. For a trainee surgical oncologist to develop these essential attributes, it is vital that they augment their training using other methods. It is no longer appropriate or acceptable to practice surgical procedures on patients [6]. Therefore, it is widely accepted that surgical training should start in the skills laboratory. This allows a trainee to learn, practice and perfect the technical skills and clinical judgements required to be a surgeon. Simulation allows * Corresponding author. Tel.: þ44 2033121947. E-mail address: [email protected] (T.M. Lewis). 1 Rajesh Aggarwal is funded by a Clinician Scientist Award from the National Institute for Health Research. 0960-7404/$ e see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.suronc.2011.04.005
mistakes, which can be used as useful learning points. This can be done in a safe and controlled environment, in the skills laboratory, without detriment to the patient. Simulation plays an important role in all specialities of surgery, but is increasingly important for surgical oncology, where the use of laparoscopy is increasing for diagnosis, staging and management of many oncological diseases [7,8]. There are considerable challenges for training in laparoscopic surgery, as the technical skills required are vastly different to open surgery [9]. Many laparoscopic operations are technically difficult and have a long and arduous learning curve, with errors and complications more likely to occur with inexperienced surgeons operating for the first time [10]. Virtual reality (VR) simulation has been heralded as an effective adjunct for laparoscopic surgical training. In 1994, Satava et al. first advocated the use of VR simulation for teaching surgeons’ anatomy and technical skills [11]. Since then, VR simulation has been slowly adopted by the surgical community, despite the increase in use of simulation and attendance of skills laboratories [12]. The common criticism of VR simulation is that the simulators are expensive and lack fidelity or realism. However, traditional training is not without cost, both in terms of finance and patient outcome. In 1997, Bridges and Diamond reported that the extra cost to the healthcare system of training surgeons in the operating room was is $47, 970 per trainee, which equates to $54 million per year for all trainees in the USA [13]. This study did not however report the additional costs associated with an increase in complications and medical error that are inherent with a procedural learning curve. There has also been significant improvement in the fidelity of VR simulators,
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139
as computer processing and graphical design have improved in the last 20 years. It is now possible to perform full length basic and advanced procedures in VR, with advanced haptic feedback systems [14]. Types of VR simulator There are numerous VR simulators on the market that provide different training objectives. To enable categorisation of simulators, the term “fidelity” is often used to describe the realism or learning experience that the simulator provides. Fidelity is the extent to which the simulator represents what it is trying to represent [15]. However, this is a broad classification, without defining the key points of simulation training, and where the simulator fits on the spectrum of fidelity. Simulators can be sub-classified in a large number of ways; by the surgical sub-speciality they teach; by the development of graphics, software or hardware; by the tasks that they provide; whether they include procedural or full length operations or the presence of haptic feedback. A simple way of classifying VR simulators is by describing them as high or low fidelity. High fidelity can be defined as a simulator that provides full length operations, with low fidelity providing basic psychomotor tasks. This allows surgeons of varying experiences to easily select a simulator that provides the appropriate training requirements. Low fidelity simulators The first laparoscopic VR simulator developed for teaching technical skills was the Minimally Invasive Surgery Trainer-Virtual Reality (MIST-VR) (Mentice, Gothenburg, Sweden) in 1997 [16]. The MIST-VR is comprised of a laparoscopic interface with motiontracking devices attached to mock laparoscopic instruments. This is attached to a computer that displays the movements of the instruments in real time on a computer monitor [17]. The simulator provides twelve psychomotor tasks that mimic important skills required for laparoscopic surgery in the operating room ranging from the basic “acquire place” to the more complex “stretch diathermy”. The tasks can be performed on easy, intermediate and difficult settings. The computer can measure various parameters such as error scores, hand movements and path length (distance moved) of each hand. This provides the user with immediate feedback of performance on each of the tasks. The MIST-VR was initially developed with assessment and examination of operative skill, not training [16]. However, it soon became evident that it would have a role in training as well. The MIST-VR consists of abstract psychomotor tasks, which allows it to be used for training a wide range of surgical specialities in basic endoscopic techniques, including cardiothoracic, general and gynaecological surgery. However, it does not allow training in specialist techniques that are provided by procedural and full length tasks on other simulators. High fidelity simulators The LapSim (Surgical Science, Gothenburg, Sweden) laparoscopic simulator was developed in 2000. It was developed with more realistic tasks using tissue that is be manipulated and could bleed. Tasks such as “clip and cut” and “suture” introduced tasks that are more relevant to a surgeon. The LapSim was one of the first VR simulators that allowed students to practice parts of operations such as laparoscopic cholecystectomy and gynaecological procedures. The LapSim is used for training and assessment of both surgery and gynaecology trainees [18,19].
135
The LAP Mentor surgery simulator (Simbionix, Chicago, USA) was developed in 2002. It divides modules into abstract tasks, procedural tasks and full length operations. The full length operations range from basic to advanced laparoscopic procedures. They contain modules for laparoscopic cholecystectomy and laparoscopic gastric bypass, but also include oncology modules with laparoscopic colon procedures. The LAP Mentor also combines the extensive range of procedures with advanced haptic feedback hardware that allows the simulator to transmit resistance when tissues or objects are encountered during a simulated task. Haptic feedback is an important part of simulation training as it is obviously present in the operating room. However, it is a feature that is lacking in many other computer-aided simulators [20]. This enhances the realism of the simulator, and as such is used widely for training of surgical trainees [21]. The ProMIS VR simulator (Haptica, Dublin, Ireland) was developed in 2002 and combined VR technology with traditional box trainer simulation. The laparoscopic interface is contained within a traditional torso shaped box trainer covered in neoprene. The box trainer contains three separate cameras that track the motion of the laparoscopic instruments from three different angles [22]. The ProMIS VR simulator has six modules designed to train laparoscopic skills such as: orientation, instrument handling, dissection, suturing, intracorporeal knot tying and diathermy [23]. The ProMIS has both abstract and procedural tasks and analyses performance by tracking instruments in 3-dimensional space. This enables real time measurement of instruments in use. All Tasks use validated metrics, including time taken, path length, economy of movement, hand dominance and task specific errors. On completion of each task, the user is given immediate feedback with summary of their metrics [24]. The trainee may also view a graphical replay or video of their performance, depending on the task selected [22]. Other high and low fidelity VR simulators are currently available that provide a platform for learning and practicing laparoscopic skills, but are not considered in further detail [25] (Table 1). Evidence for VR simulators VR simulators have standardised and reproducible tasks, that allow a subjects performance to be measured using various parameters and metrics. This suggests that they can be effectively used for surgical training and assessment. To confirm this, they must be objectively and vigorously tested to evaluate whether they are both scientifically reliable and valid (Table 2) [26]. Evidence for low fidelity simulation The MIST-VR was the first VR simulator to demonstrate reliability and validity for surgical skills training [27]. The simulator evaluates technical skill through abstract tasks, therefore is not appropriate for face and content validation. Construct validity has been demonstrated in a number of studies that have confirmed significant differences in error scores, path length and hand movements between novice and experienced surgeons [18,27e29]. Concurrent validity has also been demonstrated for all of the abstract tasks implying that it is effective for both training and assessment of technical skills (Table 3) [27,28,30,31]. Evidence for high fidelity simulation The LapSim VR simulator has additional challenges to demonstrate its use as an effective training and assessment tool, as it combines both abstract and procedural tasks. As such both need to demonstrate validity to be used for training and assessment. The abstract tasks have widely demonstrated construct and concurrent validity [23,32,33]. The procedural tasks for both laparoscopic
136
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139
cholecystectomy and laparoscopic gynaecology have been validated for face, construct and concurrent validity (Table 3) [18,34e36]. The LAP Mentor is an advanced laparoscopic simulator that combines abstract and procedural tasks with full length procedures. There is extensive evidence to support its face validity and realism for procedural tasks [21,26,37]. Construct, concurrent and face validity have all been demonstrated for the LAP Mentor in a number of studies indicating its usefulness as a technical skills training and assessment platform (Table 3) [20,21,26]. The ProMIS VR simulator has been validated for construct validity with evidence demonstrating significant differences in performance between novice and expert surgeons [22,38e40]. There is also evidence supporting its concurrent validity (Table 3) [39]. Predictive validity There have been a large number of studies that have demonstrated the validity of VR simulators. This is very important when considering the use of a VR simulator for training technical skills. There is considerable evidence supporting the face and construct validity of VR simulation, with slightly less, but still sufficient evidence demonstrating the concurrent validity of VR simulation. On inspection of Table 3, it is clear that there is a lack of evidence demonstrating the predictive validity of VR simulators. This is due to predictive validity being very difficult to demonstrate. Unlike concurrent validity, which shows a correlation between the simulator and the gold standard for assessment, predictive validity requires a long period of time between baseline testing and future testing. Concurrent validity can be measured on the same day, whereas predictive validity requires a longitudinal study with a gap of a least a year to demonstrate that a VR simulator can accurately predict the future performance in the operating room. It can be argued that predictive validity is not a very important validation type, but as surgical training changes towards improved efficiency, predictive validity would provide important information about the differing needs of individual surgeons. This would allow us to develop personalised training programs which are unique to the needs of the individual surgeon that would enhance their acquisition of technical proficiency. Learning curves and skill acquisition One aim for a laparoscopic VR simulator is that it can improve technical skills of surgeons. A learning curve can be defined as the rate of improvement of performing a task as a function of time. This means that learning curves can be used to demonstrate the acquisition of technical skills by a surgeon as they repeat a task. It is therefore important to analyse the learning curves of surgeons using various VR simulators to demonstrate that there is an improvement in both the time taken and other dexterity parameters as a task is repeated. A learning curve is not a straight line with continued improvement on every repetition. For technical skills training, eventually the curve will reach a plateau of performance where there is a reduction in the rate of improvement. It is useful to know at what point the performance plateau occurs as it will provide information for planning training programs. Gallagher and Satava first reported data on skills acquisition and learning curve analysis for the MIST-VR in 2002 [28]. They reported data on experienced, inexperienced and novice laparoscopic surgeons performing all tasks on the MIST-VR. There was a significant difference in performances between the groups on the first attempt. All groups improved performance with repeated attempts, with a plateau of skills acquisition at the fifth attempt, with most improvement by the novice and intermediate group. Further repeated practice improved the groups to the level of the experience surgeons. This study demonstrates the effectiveness of the MIST-VR simulator for the
acquisition of technical skills in surgeons. Further evidence of the learning effects of VR simulators has been demonstrated with learning curve analysis of novice and experienced surgeons on the LapSim and LAP Mentor VR simulators [20,41].
Transfer to the operating room (OR) Although there is considerable evidence presented on the validity of VR simulation for the acquisition of technical skills, the most important aspect of training should be the ability of a simulator to improve a subject’s performance in the operating room. If a simulator has concurrent validity, it informs us that the performance in the skills lab on VR may correlate with their current performance in the OR. Predictive validity can provide further information regarding how well a surgeon may perform in the future in the operating room. It can be argued that this is the most important form of validity, as it is an assessment of a surgeons potential and predicts how well a surgeon will perform in the OR in the future. Compared to this face validity, the realism of a simulator, and construct validity, a difference between novice and experienced surgeons, seems less important. There have been a number of studies that have investigating the transfer of skills from “VR to OR”, but have not analysed the predictive validity of the simulators. There is evidence supporting an improvement in surgical performance following training on VR simulation. Seymour et al. demonstrated that VR training on the MIST-VR improved OR performance with a decrease in errors and time taken when compared with a non-VR trained group [42]. Grantcharov et al. confirm this finding with MIST-VR training, in addition demonstrating an improvement in economy of movement of the surgeons’ instruments [9]. In 2009, Larsen et al. performed a randomised controlled study investigating the impact on OR performance after training on the LapSim VR simulator. They demonstrated that the VR trained group had a significant improvement in OR performance with reduced operating time and better performance scores using two blinded assessors with validated scoring systems [43]. Further evidence can be found supporting an improvement in OR performance after training on the LapSim VR simulator [37,44]. However, evidence supporting the predictive validity of VR simulation is limited. A number of studies report predictive validity, but in fact demonstrate concurrent validity. Ahlberg et al. demonstrated a weak correlation between performance in the OR and VR for MIST-VR trained surgeons, although the assessment tools that were used were not validated [45]. This finding is supported by further studies that have shown correlation between performance on the MIST-VR with the performance in the OR, for VR trained subjects [9,19,46]. However, these studies do not assess predictive validity. If a correlation is found between VR performance and OR performance, then the simulator has concurrent validity, not predictive validity. For a simulator to show predictive validity, a longitudinal study must take place that assesses a surgeon in VR, and after a considerable period of time, assesses their OR performance. There are also a number of limitations in these studies that should be acknowledged. For example, the assessments that are used are non-validated which consequently decreases the reliability of their findings. Two of the studies also assess the OR performance using a porcine model, which is not validated, and does not give an exact replication of OR performance. These studies also do not train the VR groups in a structured manner; they could be improved using validated training programs for the VR simulators [35,47]. Further work is required to demonstrate predictive validity of VR simulation, with large scale randomised controlled trials assessing surgeons from “VR to OR” over a long period of time.
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139
Cost of VR simulation One of the frequent criticisms of VR simulation is that the simulators are expensive, and not all hospitals can afford them. The MISTVR and LapSim VR simulators are estimated at US$24,000 and US$40,000 respectively, and cost of the LAP Mentor is in excess of US$100,000. The more expensive VR simulators also require an annual subscription for technical support and upgrades. However, between 1993 and 1997, it was estimated that the cost of training surgeons in the operating room in the USA was roughly $12,000 per surgeon per year [13]. This figure was estimated using cost per minute of training surgeons. Therefore, as VR simulation has demonstrated that it can decrease operating time, VR simulation is likely to reduce the costs of training in the operating room by improving efficiency [9,46]. However, this economic evaluation does not take into account the cost of potential litigation that would occur from trainees operating. This cost would also be decreased due to the reduction in medical error that can be demonstrated from prior VR training [46]. It is not feasible to have a simulator in every hospital. There should however, be simulators in strategic venues that are accessible to all. With a high rate of junior surgeons training on the simulators it would dramatically decrease the actual cost by giving a cost per trainee. Although it remains an expensive purchase, when it is compared with the cost of training surgeons in the operating room the initial outlay doesn’t seem too extravagant. The costs to a hospital may also be reduced, as VR simulation has been shown to reduce the operating time of surgeons in training. Therefore, by training surgeons in VR it will improve operating room efficiency and reduce the cost to the operating theatre. Further work should focus on the cost effectiveness of VR simulation in terms of the number needed to train on a VR simulator to make it cost effective. Effectiveness of VR simulation Surgeons in training have a finite amount of time in which they can train. It is therefore vital that they use the time they have as efficiently as possible. VR simulation provides a platform in which surgeons can learn and practice technical skills, and there is a plethora of research demonstrating the realism and the validity of simulators in terms of skills acquisition compared to a control group. However, there is limited data that documenting how effective each VR simulator is compared to other VR simulators. In the aviation industry, the transfer effectiveness ratio (TER) is used to measure how effective a simulator is [48]. It is calculated as follows:
TER ¼
Y0 YX X
Y0 is the median time required by the control group to reach performance criterion and Yx is the corresponding measure for the intervention group after having received a median of X amount of training time on the simulator, which is also the denominator [44,48]. When this is considered for VR simulation, Y0 remains the time for a control group to reach proficiency in the operating room and Yx is the time for a VR trained group to reach proficiency in the operating room. Studies in the aviation industry report a TER of 0.5, which implies every hour spent training in a flight simulator is equivalent to 0.5 h flying in a real plane [48]. The only study investigating the TER of VR simulation was performed by Aggarwal et al. in 2007 [44]. They randomised 20 novice laparoscopic surgeons to a control group and an intervention group trained on the Lapsim VR simulator. After cognitive training, all subjects performed 3 porcine laparoscopic cholecystectomies. Learning curve analysis demonstrated that neither group reached predefined proficiency levels for time. However, by using proficiency for
137
dexterity parameters, the TER was calculated as 2.28. This implies that every hour of training on the Lapsim VR simulator is equivalent to 2.28 h of training on the operating room. Although this figure is not as accurate as it would have been using time as the proficiency parameter, this remains the only study that assesses the transfer effectiveness of a VR simulator. Further work needs to focus on the TER for each VR simulator to discover which VR simulator is the most effective at teaching technical skills. VR procedure rehearsal Virtual reality simulation allows a surgical trainee the opportunity to learn, develop and perfect their technical skills in a safe and controlled environment. Although it is clear that VR simulation can reduce the learning curve in the operating room and improve patient safety, the aim of VR simulation should be to enhance patient outcome [42]. One of the latest advances in VR technology is the ability to load real patient imaging onto the simulator. This has developed the concept of patient specific procedure rehearsal [49]. The technology has been primarily been used for endovascular surgery, where CT scans of patients carotid arteries are loaded on to the simulator. This allows the surgeon or radiologist to not only practice the technical skills required for a procedure, but they can practice on an exact replica of the patient’s anatomy. This would allow enhanced operative planning and anticipate any potential complications and abnormal anatomy prior to entering the operating room. This would enhance technical performance of the procedure and improve patient safety. Initial research into this area has been encouraging. On comparison of OR and VR performance for carotid artery stenting, Willaert et al. demonstrated similar use of endovascular tools and angiography imaging, with an increased feeling of teamwork, communication and patient safety as a result of procedure rehearsal [50]. This concept has important implications for many different oncological procedures including laparoscopic prostate, adrenal or liver resections. It can be foreseen that procedure rehearsal is included in the multi disciplinary management plan for patients that require laparoscopic surgery for cancer treatment. It would be useful for pre-operative planning of solid tumours that have abnormal anatomical variations, or high complexity. This would include full team procedure rehearsal of the operation in a skills laboratory or simulated operating suite which consequently could enhance patient outcome [51]. Discussion The operations performed for the management of oncological diseases are extremely complex, and require the surgeon to have considerable technical skill. Previous generations of surgeons developed their technical skills over a long period of time. Skills were acquired in the operating room by performing a large number of operations using graduated responsibility. It is clear that current training methods are no longer sufficient to provide this time and surgical experience. Simulation, and in particular, VR simulation has been advocated as a solution for this shortfall in training time. It has been widely demonstrated that VR simulation can improve skills acquisition for surgical trainees. The VR simulators currently on the market for laparoscopic surgery have been widely validated for their use in surgical training and assessment. However, despite the fact that VR simulation has been established for well over a decade, it has been slowly adopted, and although there is extensive research validating the VR simulators for use as training and assessment tools, there is limited evidence supporting the impact they will have on future OR performance. It is well documented that surgeons have a finite time to train in, it is therefore vital that they use the time they have as efficiently as possible. Further work on the TER of VR
138
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139
simulation would allow us to accurately state how efficient each simulator is for teaching technical skills, and therefore can advocate the use of the most efficient simulators for surgeons in training. It is also important to analyse the cost to the healthcare system that VR simulation will have. VR simulation requires a considerable financial investment to purchase them. Therefore to encourage hospital trusts and universities to acquire VR simulators, it is vital to be able to demonstrate that each hour of training in a VR simulator will reduce operating room costs by X dollars. It would also be useful to evaluate how many students need to use a VR simulator
and for how long, to make it cost efficient for an institution. This would define the “number needed to train” on each simulator to make them cost efficient. VR simulation has been established as a valid method for teaching technical skills to surgeons. The initial criticism of VR simulation was that it was not realistic, is too costly and has no correlation to future OR performance. There is now considerable evidence demonstrating increased realism of simulators, but more work is needed to motivate and educate the surgical community in the effectiveness of VR simulation in terms of cost and future OR performance.
Appendix Table 1 Summary of other laparoscopic simulators. Simulator
Module
Company
Description
Surgical Educational Platform Surgical Educational Platform Surgical Educational Platform Surgical Educational Platform Virtual Endoscopic Surgery Training system (VEST) Virtual Endoscopic Surgery Training system (VEST) Xitact LS500
SEP Basic SEP Cholecystectomy SEP Ectopic Pregnancy SEP Robot VSOne Cho for VEST
Simsurgery Simsurgery Simsurgery Simsurgery Select-IT VEST Systems AG (Karlsruh, Germany) Select-IT VEST Systems AG (Karlsruh, Germany) Xitact SA, Morges, Switzerland
Simulator for developing an practising basic laparoscopic skills A learning module for laparoscopic cholecystectomy. A learning module for the removal of an ectopic pregnancy. A learning module for robotic surgery For teaching and training laparoscopic cholecystectomy
VSOne Gyn for VEST
For laparoscopic gynaecology training Laparoscopic simulator that provided a modular training environment for basic laparoscopic skills and laparoscopic procedural skills [25].
Table 2 Qualities of the ideal surgical assessment tool defined by Aggarwal et al. [33]. Term
Definition
Feasibility Face Validity Content validity
Measure of whether something is capable of being done or carried out Extent to which the examination resembles real life situations Extent to which the domain that is being measured is measured by the assessment tool e for example, while trying to assess technical skills we may actually be testing knowledge Extent to which a test measures the trait it purports to measure; one inference of construct validity is the extent to which a test discriminates between various levels of expertise Extent to which the results of the assessment tool correlate with the gold standard for that domain Ability of examination to predict future performance Measure of a test to generate similar results when applied at two different points Measure of the extent of agreement between two or more observers when rating the performance of an individual
Construct validity Concurrent validity Predictive validity Test-retest reliability Inter-rater reliability
Table 3 Summary of the validation of VR simulators. VR simulator
Face validity
Content validity
Construct validity
Concurrent validity
Predictive validity
MIST-VR ProMIS LapSim LAP Mentor
e Yes [46] Yes [18,34e36] Yes [21,26,37]
e e Yes [18,34e36] Yes [21,26,37]
Yes Yes Yes Yes
Yes Yes Yes Yes
e e e e
References [1] Reznick RK, MacRae H. Teaching surgical skillsechanges in the wind. N Engl J Med 2006;355(25):2664e9. [2] Kim MJ, Monson JR. The future of innovation and training in surgical oncology. Surg Oncol; 2010. [3] Shah PR, Joseph A, Haray PN. Laparoscopic colorectal surgery: learning curve and training implications. Postgrad Med J 2005;81(958):537e40. [4] Rattner DW, Apelgren KN, Eubanks WS. The need for training opportunities in advanced laparoscopic surgery. Surg Endosc 2001;15(10):1066e70. [5] Cathcart PJ, van der Meulen J, Emberton M, Kelly J. Volume-mortality for cystectomy. Centralisation of cancer services vindicated. BMJ 2010;340: c3336. [6] Aggarwal R, Darzi A. Technical-skills training in the 21st century. N Engl J Med 2006;355(25):2695e6. [7] Rieder E, Swanstrom LL. Advances in cancer surgery: natural orifice surgery (NOTES) for oncological diseases. Surg Oncol; 2010. [8] Monson JR. Advanced techniques in abdominal surgery. BMJ 1993;307(6915): 1346e50.
[24,30,34,35] [22,38e40] [18,34e36] 20,21,26
[30,34,36,37] [39] [18,34e36] 20,21,26
[9] Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, FunchJensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004;91(2):146e50. [10] Reissman P, Cohen S, Weiss EG, Wexner SD. Laparoscopic colorectal surgery: ascending the learning curve. World J Surg 1996;20(3):277e81. discussion 82. [11] Satava RM. Emerging medical applications of virtual reality: a surgeon’s perspective. Artif Intell Med 1994;6(4):281e8. [12] Figert PL, Park AE, Witzke DB, Schwartz RW. Transfer of training in acquiring laparoscopic skills. J Am Coll Surg 2001;193(5):533e7. [13] Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg 1999;177(1):28e32. [14] Gallagher AG, Cates CU. Virtual reality training for the operating room and cardiac catheterisation laboratory. Lancet 2004;364(9444):1538e40. [15] Maran NJ, Glavin RJ. Low- to high-fidelity simulation e a continuum of medical education? Med Educ 2003;37(Suppl. 1):22e8. [16] Wilson MS, Middlebrook A, Sutton C, Stone R, McCloy RF. Mist VR: a virtual reality trainer for laparoscopic surgery assesses performance. Ann R Coll Surg Engl 1997;79(6):403e4. [17] Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills training and assessment. Br J Surg 2004;91(12):1549e58.
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139 [18] Aggarwal R, Grantcharov TP, Eriksen JR, Blirup D, Kristiansen VB, FunchJensen P, et al. An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 2006;244(2):310e4. [19] Hart R, Karthigasu K. The benefits of virtual reality simulator training for laparoscopic surgery. Curr Opin Obstet Gynecol 2007;19(4):297e302. [20] Andreatta PB, Woodrum DT, Birkmeyer JD, Yellamanchilli RK, Doherty GM, Gauger PG, et al. Laparoscopic skills are improved with LapMentor training: results of a randomized, double-blinded study. Ann Surg 2006;243(6): 854e60. discussion 60e3. [21] Aggarwal R, Crochet P, Dias A, Ziprin P, Darzi A. A scientific method for development of a whole procedure virtual reality training curriculum for laparoscopic cholecystectomy. Br J Surg; 2009. [22] Van Sickle KR, McClusky 3rd DA, Gallagher AG, Smith CD. Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc 2005;19(9):1227e31. [23] Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, et al. Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 2005;19(12):1523e32. [24] Haptica. The world-leading ProMIS Surgical Simulator. Ireland. [25] Schijven M, Jakimowicz J. Virtual reality surgical laparoscopic simulators. Surg Endosc 2003;17(12):1943e50. [26] McDougall EM, Corica FA, Boker JR, Sala LG, Stoliar G, Borin JF, et al. Construct validity testing of a laparoscopic surgical simulator. J Am Coll Surg 2006; 202(5):779e87. [27] Taffinder N, Sutton C, Fishwick RJ, McManus IC, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomised controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform 1998;50:124e30. [28] Gallagher AG, Satava RM. Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Learning curves and reliability measures. Surg Endosc 2002;16(12):1746e52. [29] McNatt SS, Smith CD. A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons. Surg Endosc 2001;15(10):1085e9. [30] Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P. Virtual reality computer simulation. Surg Endosc 2001;15(3):242e4. [31] Gallagher HJ, Allan JD, Tolley DA. Spatial awareness in urologists: are they different? BJU Int 2001;88(7):666e70. [32] Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos P, et al. Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc 2005; 19(3):401e5. [33] Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L, Gauger PG, Minter RM. Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 2006;191(1):28e32. [34] van Dongen KW, Tournoij E, van der Zee DC, Schijven MP, Broeders IA. Construct validity of the LapSim: can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc 2007;21(8): 1413e7. [35] Aggarwal R, Grantcharov T, Moorthy K, Hance J, Darzi A. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg 2006;191(1):128e33.
139
[36] Bajka M, Tuchschmid S, Fink D, Szekely G, Harders M. Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system. Surg Endosc;24(1):79e88. [37] Hyltander A, Liljegren E, Rhodin PH, Lonroth H. The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc 2002; 16(9):1324e8. [38] Broe D, Ridgway PF, Johnson S, Tierney S, Conlon KC. Construct validation of a novel hybrid surgical simulator. Surg Endosc 2006;20(6):900e4. [39] Ritter EM, Kindelan TW, Michael C, Pimentel EA, Bowyer MW. Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS). Surg Endosc 2007;21(8):1441e5. [40] Botden SM, Buzink SN, Schijven MP, Jakimowicz JJ. Augmented versus virtual reality laparoscopic simulation: what is the difference? a comparison of the ProMIS augmented reality laparoscopic simulator versus LapSim virtual reality laparoscopic simulator. World J Surg 2007;31(4):764e72. [41] Sherman V, Feldman LS, Stanbridge D, Kazmi R, Fried GM. Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator. Surg Endosc 2005;19(5):678e82. [42] Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236(4):458e63. discussion 63e4. [43] Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 2009;338:b1802. [44] Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg 2007;246(5):771e9. [45] Ahlberg G, Heikkinen T, Iselius L, Leijonmarck CE, Rutqvist J, Arvidsson D. Does training in a virtual reality simulator improve surgical performance? Surg Endosc 2002;16(1):126e9. [46] Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky 3rd DA, et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007;193(6):797e804. [47] Aggarwal R, Hance J, Undre S, Ratnasothy J, Moorthy K, Chang A, et al. Training junior operative residents in laparoscopic suturing skills is feasible and efficacious. Surgery 2006;139(6):729e34. [48] Incremental transfer and cost effectiveness of ground-based flight trainers in a university aviation program. In: Proceedings of the Human Factors and Ergonomics Society 49th annual meeting; 2005. [49] Willaert W, Aggarwal R, Bicknell C, Hamady M, Darzi A, Vermassen F, et al. Patient-specific simulation in carotid artery stenting. J Vasc Surg 2010;52(6): 1700e5. [50] Willaert WI, Aggarwal R, Nestel DF, Gaines PA, Vermassen FE, Darzi AW, et al. Patient-specific simulation for endovascular procedures: qualitative evaluation of the development process. Int J Med Robot 2010;6(2):202e10. [51] Willaert WI, Aggarwal R, Van Herzeele I, O’Donoghue K, Gaines PA, Darzi AW, et al. Patient-specific endovascular simulation Influences Interventionalists performing carotid artery stenting procedures. Eur J Vasc Endovasc Surg; 2011.
Contents lists available at ScienceDirect
Surgical Oncology journal homepage: www.elsevier.com/locate/suronc
Training in surgical oncology e The role of VR simulation T.M. Lewis a, *, R. Aggarwal a,1, N. Rajaretnam a, T.P. Grantcharov b, A. Darzi a a b
Department of Cancer and Surgery, Room 1029, 10th Floor, QEQM, St. Marys hospital Imperial College London, UK Division of General Surgery, St Michael’s Hospital, Toronto, ON, Canada
a b s t r a c t Keywords: Laparoscopic Training simulation Virtual reality
There have been dramatic changes in surgical training over the past two decades which have resulted in a number of concerns for the development of future surgeons. Changes in the structure of cancer services, working hour restrictions and a commitment to patient safety has led to a reduction in training opportunities that are available to the surgeon in training. Simulation and in particular virtual reality (VR) simulation has been heralded as an effective adjunct to surgical training. Advances in VR simulation has allowed trainees to practice realistic full length procedures in a safe and controlled environment, where mistakes are permitted and can be used as learning points. There is considerable evidence to demonstrate that the VR simulation can be used to enhance technical skills and improve operating room performance. Future work should focus on the cost effectiveness and predictive validity of VR simulation, which in turn would increase the uptake of simulation and enhance surgical training. Ó 2011 Elsevier Ltd. All rights reserved.
Introduction Over the past decade, there has been a change in surgical training that will impact the development of future surgical oncologists. The essential attributes of a surgical oncologist are proficiency in the specific operations required for the diagnosis and management, either cure or palliation, for solid tumours [1]. Previously, a fully trained general surgeon would have sufficient technical skill, knowledge and experience, to be able to approach the surgical management of a solid tumour in a precise and safe manner, which would result in the best possible management of the patient [2]. However, this level of proficiency may be difficult to reach for current surgical trainees as over the past decade there has been a reduction in the training opportunities in cancer surgery [3]. The reasons for this include the limit on trainees working hours, advances in nonsurgical management of cancer and surgical technology, and in the United Kingdom, the centralisation of cancer services [4,5]. For a trainee surgical oncologist to develop these essential attributes, it is vital that they augment their training using other methods. It is no longer appropriate or acceptable to practice surgical procedures on patients [6]. Therefore, it is widely accepted that surgical training should start in the skills laboratory. This allows a trainee to learn, practice and perfect the technical skills and clinical judgements required to be a surgeon. Simulation allows * Corresponding author. Tel.: þ44 2033121947. E-mail address: [email protected] (T.M. Lewis). 1 Rajesh Aggarwal is funded by a Clinician Scientist Award from the National Institute for Health Research. 0960-7404/$ e see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.suronc.2011.04.005
mistakes, which can be used as useful learning points. This can be done in a safe and controlled environment, in the skills laboratory, without detriment to the patient. Simulation plays an important role in all specialities of surgery, but is increasingly important for surgical oncology, where the use of laparoscopy is increasing for diagnosis, staging and management of many oncological diseases [7,8]. There are considerable challenges for training in laparoscopic surgery, as the technical skills required are vastly different to open surgery [9]. Many laparoscopic operations are technically difficult and have a long and arduous learning curve, with errors and complications more likely to occur with inexperienced surgeons operating for the first time [10]. Virtual reality (VR) simulation has been heralded as an effective adjunct for laparoscopic surgical training. In 1994, Satava et al. first advocated the use of VR simulation for teaching surgeons’ anatomy and technical skills [11]. Since then, VR simulation has been slowly adopted by the surgical community, despite the increase in use of simulation and attendance of skills laboratories [12]. The common criticism of VR simulation is that the simulators are expensive and lack fidelity or realism. However, traditional training is not without cost, both in terms of finance and patient outcome. In 1997, Bridges and Diamond reported that the extra cost to the healthcare system of training surgeons in the operating room was is $47, 970 per trainee, which equates to $54 million per year for all trainees in the USA [13]. This study did not however report the additional costs associated with an increase in complications and medical error that are inherent with a procedural learning curve. There has also been significant improvement in the fidelity of VR simulators,
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139
as computer processing and graphical design have improved in the last 20 years. It is now possible to perform full length basic and advanced procedures in VR, with advanced haptic feedback systems [14]. Types of VR simulator There are numerous VR simulators on the market that provide different training objectives. To enable categorisation of simulators, the term “fidelity” is often used to describe the realism or learning experience that the simulator provides. Fidelity is the extent to which the simulator represents what it is trying to represent [15]. However, this is a broad classification, without defining the key points of simulation training, and where the simulator fits on the spectrum of fidelity. Simulators can be sub-classified in a large number of ways; by the surgical sub-speciality they teach; by the development of graphics, software or hardware; by the tasks that they provide; whether they include procedural or full length operations or the presence of haptic feedback. A simple way of classifying VR simulators is by describing them as high or low fidelity. High fidelity can be defined as a simulator that provides full length operations, with low fidelity providing basic psychomotor tasks. This allows surgeons of varying experiences to easily select a simulator that provides the appropriate training requirements. Low fidelity simulators The first laparoscopic VR simulator developed for teaching technical skills was the Minimally Invasive Surgery Trainer-Virtual Reality (MIST-VR) (Mentice, Gothenburg, Sweden) in 1997 [16]. The MIST-VR is comprised of a laparoscopic interface with motiontracking devices attached to mock laparoscopic instruments. This is attached to a computer that displays the movements of the instruments in real time on a computer monitor [17]. The simulator provides twelve psychomotor tasks that mimic important skills required for laparoscopic surgery in the operating room ranging from the basic “acquire place” to the more complex “stretch diathermy”. The tasks can be performed on easy, intermediate and difficult settings. The computer can measure various parameters such as error scores, hand movements and path length (distance moved) of each hand. This provides the user with immediate feedback of performance on each of the tasks. The MIST-VR was initially developed with assessment and examination of operative skill, not training [16]. However, it soon became evident that it would have a role in training as well. The MIST-VR consists of abstract psychomotor tasks, which allows it to be used for training a wide range of surgical specialities in basic endoscopic techniques, including cardiothoracic, general and gynaecological surgery. However, it does not allow training in specialist techniques that are provided by procedural and full length tasks on other simulators. High fidelity simulators The LapSim (Surgical Science, Gothenburg, Sweden) laparoscopic simulator was developed in 2000. It was developed with more realistic tasks using tissue that is be manipulated and could bleed. Tasks such as “clip and cut” and “suture” introduced tasks that are more relevant to a surgeon. The LapSim was one of the first VR simulators that allowed students to practice parts of operations such as laparoscopic cholecystectomy and gynaecological procedures. The LapSim is used for training and assessment of both surgery and gynaecology trainees [18,19].
135
The LAP Mentor surgery simulator (Simbionix, Chicago, USA) was developed in 2002. It divides modules into abstract tasks, procedural tasks and full length operations. The full length operations range from basic to advanced laparoscopic procedures. They contain modules for laparoscopic cholecystectomy and laparoscopic gastric bypass, but also include oncology modules with laparoscopic colon procedures. The LAP Mentor also combines the extensive range of procedures with advanced haptic feedback hardware that allows the simulator to transmit resistance when tissues or objects are encountered during a simulated task. Haptic feedback is an important part of simulation training as it is obviously present in the operating room. However, it is a feature that is lacking in many other computer-aided simulators [20]. This enhances the realism of the simulator, and as such is used widely for training of surgical trainees [21]. The ProMIS VR simulator (Haptica, Dublin, Ireland) was developed in 2002 and combined VR technology with traditional box trainer simulation. The laparoscopic interface is contained within a traditional torso shaped box trainer covered in neoprene. The box trainer contains three separate cameras that track the motion of the laparoscopic instruments from three different angles [22]. The ProMIS VR simulator has six modules designed to train laparoscopic skills such as: orientation, instrument handling, dissection, suturing, intracorporeal knot tying and diathermy [23]. The ProMIS has both abstract and procedural tasks and analyses performance by tracking instruments in 3-dimensional space. This enables real time measurement of instruments in use. All Tasks use validated metrics, including time taken, path length, economy of movement, hand dominance and task specific errors. On completion of each task, the user is given immediate feedback with summary of their metrics [24]. The trainee may also view a graphical replay or video of their performance, depending on the task selected [22]. Other high and low fidelity VR simulators are currently available that provide a platform for learning and practicing laparoscopic skills, but are not considered in further detail [25] (Table 1). Evidence for VR simulators VR simulators have standardised and reproducible tasks, that allow a subjects performance to be measured using various parameters and metrics. This suggests that they can be effectively used for surgical training and assessment. To confirm this, they must be objectively and vigorously tested to evaluate whether they are both scientifically reliable and valid (Table 2) [26]. Evidence for low fidelity simulation The MIST-VR was the first VR simulator to demonstrate reliability and validity for surgical skills training [27]. The simulator evaluates technical skill through abstract tasks, therefore is not appropriate for face and content validation. Construct validity has been demonstrated in a number of studies that have confirmed significant differences in error scores, path length and hand movements between novice and experienced surgeons [18,27e29]. Concurrent validity has also been demonstrated for all of the abstract tasks implying that it is effective for both training and assessment of technical skills (Table 3) [27,28,30,31]. Evidence for high fidelity simulation The LapSim VR simulator has additional challenges to demonstrate its use as an effective training and assessment tool, as it combines both abstract and procedural tasks. As such both need to demonstrate validity to be used for training and assessment. The abstract tasks have widely demonstrated construct and concurrent validity [23,32,33]. The procedural tasks for both laparoscopic
136
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139
cholecystectomy and laparoscopic gynaecology have been validated for face, construct and concurrent validity (Table 3) [18,34e36]. The LAP Mentor is an advanced laparoscopic simulator that combines abstract and procedural tasks with full length procedures. There is extensive evidence to support its face validity and realism for procedural tasks [21,26,37]. Construct, concurrent and face validity have all been demonstrated for the LAP Mentor in a number of studies indicating its usefulness as a technical skills training and assessment platform (Table 3) [20,21,26]. The ProMIS VR simulator has been validated for construct validity with evidence demonstrating significant differences in performance between novice and expert surgeons [22,38e40]. There is also evidence supporting its concurrent validity (Table 3) [39]. Predictive validity There have been a large number of studies that have demonstrated the validity of VR simulators. This is very important when considering the use of a VR simulator for training technical skills. There is considerable evidence supporting the face and construct validity of VR simulation, with slightly less, but still sufficient evidence demonstrating the concurrent validity of VR simulation. On inspection of Table 3, it is clear that there is a lack of evidence demonstrating the predictive validity of VR simulators. This is due to predictive validity being very difficult to demonstrate. Unlike concurrent validity, which shows a correlation between the simulator and the gold standard for assessment, predictive validity requires a long period of time between baseline testing and future testing. Concurrent validity can be measured on the same day, whereas predictive validity requires a longitudinal study with a gap of a least a year to demonstrate that a VR simulator can accurately predict the future performance in the operating room. It can be argued that predictive validity is not a very important validation type, but as surgical training changes towards improved efficiency, predictive validity would provide important information about the differing needs of individual surgeons. This would allow us to develop personalised training programs which are unique to the needs of the individual surgeon that would enhance their acquisition of technical proficiency. Learning curves and skill acquisition One aim for a laparoscopic VR simulator is that it can improve technical skills of surgeons. A learning curve can be defined as the rate of improvement of performing a task as a function of time. This means that learning curves can be used to demonstrate the acquisition of technical skills by a surgeon as they repeat a task. It is therefore important to analyse the learning curves of surgeons using various VR simulators to demonstrate that there is an improvement in both the time taken and other dexterity parameters as a task is repeated. A learning curve is not a straight line with continued improvement on every repetition. For technical skills training, eventually the curve will reach a plateau of performance where there is a reduction in the rate of improvement. It is useful to know at what point the performance plateau occurs as it will provide information for planning training programs. Gallagher and Satava first reported data on skills acquisition and learning curve analysis for the MIST-VR in 2002 [28]. They reported data on experienced, inexperienced and novice laparoscopic surgeons performing all tasks on the MIST-VR. There was a significant difference in performances between the groups on the first attempt. All groups improved performance with repeated attempts, with a plateau of skills acquisition at the fifth attempt, with most improvement by the novice and intermediate group. Further repeated practice improved the groups to the level of the experience surgeons. This study demonstrates the effectiveness of the MIST-VR simulator for the
acquisition of technical skills in surgeons. Further evidence of the learning effects of VR simulators has been demonstrated with learning curve analysis of novice and experienced surgeons on the LapSim and LAP Mentor VR simulators [20,41].
Transfer to the operating room (OR) Although there is considerable evidence presented on the validity of VR simulation for the acquisition of technical skills, the most important aspect of training should be the ability of a simulator to improve a subject’s performance in the operating room. If a simulator has concurrent validity, it informs us that the performance in the skills lab on VR may correlate with their current performance in the OR. Predictive validity can provide further information regarding how well a surgeon may perform in the future in the operating room. It can be argued that this is the most important form of validity, as it is an assessment of a surgeons potential and predicts how well a surgeon will perform in the OR in the future. Compared to this face validity, the realism of a simulator, and construct validity, a difference between novice and experienced surgeons, seems less important. There have been a number of studies that have investigating the transfer of skills from “VR to OR”, but have not analysed the predictive validity of the simulators. There is evidence supporting an improvement in surgical performance following training on VR simulation. Seymour et al. demonstrated that VR training on the MIST-VR improved OR performance with a decrease in errors and time taken when compared with a non-VR trained group [42]. Grantcharov et al. confirm this finding with MIST-VR training, in addition demonstrating an improvement in economy of movement of the surgeons’ instruments [9]. In 2009, Larsen et al. performed a randomised controlled study investigating the impact on OR performance after training on the LapSim VR simulator. They demonstrated that the VR trained group had a significant improvement in OR performance with reduced operating time and better performance scores using two blinded assessors with validated scoring systems [43]. Further evidence can be found supporting an improvement in OR performance after training on the LapSim VR simulator [37,44]. However, evidence supporting the predictive validity of VR simulation is limited. A number of studies report predictive validity, but in fact demonstrate concurrent validity. Ahlberg et al. demonstrated a weak correlation between performance in the OR and VR for MIST-VR trained surgeons, although the assessment tools that were used were not validated [45]. This finding is supported by further studies that have shown correlation between performance on the MIST-VR with the performance in the OR, for VR trained subjects [9,19,46]. However, these studies do not assess predictive validity. If a correlation is found between VR performance and OR performance, then the simulator has concurrent validity, not predictive validity. For a simulator to show predictive validity, a longitudinal study must take place that assesses a surgeon in VR, and after a considerable period of time, assesses their OR performance. There are also a number of limitations in these studies that should be acknowledged. For example, the assessments that are used are non-validated which consequently decreases the reliability of their findings. Two of the studies also assess the OR performance using a porcine model, which is not validated, and does not give an exact replication of OR performance. These studies also do not train the VR groups in a structured manner; they could be improved using validated training programs for the VR simulators [35,47]. Further work is required to demonstrate predictive validity of VR simulation, with large scale randomised controlled trials assessing surgeons from “VR to OR” over a long period of time.
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139
Cost of VR simulation One of the frequent criticisms of VR simulation is that the simulators are expensive, and not all hospitals can afford them. The MISTVR and LapSim VR simulators are estimated at US$24,000 and US$40,000 respectively, and cost of the LAP Mentor is in excess of US$100,000. The more expensive VR simulators also require an annual subscription for technical support and upgrades. However, between 1993 and 1997, it was estimated that the cost of training surgeons in the operating room in the USA was roughly $12,000 per surgeon per year [13]. This figure was estimated using cost per minute of training surgeons. Therefore, as VR simulation has demonstrated that it can decrease operating time, VR simulation is likely to reduce the costs of training in the operating room by improving efficiency [9,46]. However, this economic evaluation does not take into account the cost of potential litigation that would occur from trainees operating. This cost would also be decreased due to the reduction in medical error that can be demonstrated from prior VR training [46]. It is not feasible to have a simulator in every hospital. There should however, be simulators in strategic venues that are accessible to all. With a high rate of junior surgeons training on the simulators it would dramatically decrease the actual cost by giving a cost per trainee. Although it remains an expensive purchase, when it is compared with the cost of training surgeons in the operating room the initial outlay doesn’t seem too extravagant. The costs to a hospital may also be reduced, as VR simulation has been shown to reduce the operating time of surgeons in training. Therefore, by training surgeons in VR it will improve operating room efficiency and reduce the cost to the operating theatre. Further work should focus on the cost effectiveness of VR simulation in terms of the number needed to train on a VR simulator to make it cost effective. Effectiveness of VR simulation Surgeons in training have a finite amount of time in which they can train. It is therefore vital that they use the time they have as efficiently as possible. VR simulation provides a platform in which surgeons can learn and practice technical skills, and there is a plethora of research demonstrating the realism and the validity of simulators in terms of skills acquisition compared to a control group. However, there is limited data that documenting how effective each VR simulator is compared to other VR simulators. In the aviation industry, the transfer effectiveness ratio (TER) is used to measure how effective a simulator is [48]. It is calculated as follows:
TER ¼
Y0 YX X
Y0 is the median time required by the control group to reach performance criterion and Yx is the corresponding measure for the intervention group after having received a median of X amount of training time on the simulator, which is also the denominator [44,48]. When this is considered for VR simulation, Y0 remains the time for a control group to reach proficiency in the operating room and Yx is the time for a VR trained group to reach proficiency in the operating room. Studies in the aviation industry report a TER of 0.5, which implies every hour spent training in a flight simulator is equivalent to 0.5 h flying in a real plane [48]. The only study investigating the TER of VR simulation was performed by Aggarwal et al. in 2007 [44]. They randomised 20 novice laparoscopic surgeons to a control group and an intervention group trained on the Lapsim VR simulator. After cognitive training, all subjects performed 3 porcine laparoscopic cholecystectomies. Learning curve analysis demonstrated that neither group reached predefined proficiency levels for time. However, by using proficiency for
137
dexterity parameters, the TER was calculated as 2.28. This implies that every hour of training on the Lapsim VR simulator is equivalent to 2.28 h of training on the operating room. Although this figure is not as accurate as it would have been using time as the proficiency parameter, this remains the only study that assesses the transfer effectiveness of a VR simulator. Further work needs to focus on the TER for each VR simulator to discover which VR simulator is the most effective at teaching technical skills. VR procedure rehearsal Virtual reality simulation allows a surgical trainee the opportunity to learn, develop and perfect their technical skills in a safe and controlled environment. Although it is clear that VR simulation can reduce the learning curve in the operating room and improve patient safety, the aim of VR simulation should be to enhance patient outcome [42]. One of the latest advances in VR technology is the ability to load real patient imaging onto the simulator. This has developed the concept of patient specific procedure rehearsal [49]. The technology has been primarily been used for endovascular surgery, where CT scans of patients carotid arteries are loaded on to the simulator. This allows the surgeon or radiologist to not only practice the technical skills required for a procedure, but they can practice on an exact replica of the patient’s anatomy. This would allow enhanced operative planning and anticipate any potential complications and abnormal anatomy prior to entering the operating room. This would enhance technical performance of the procedure and improve patient safety. Initial research into this area has been encouraging. On comparison of OR and VR performance for carotid artery stenting, Willaert et al. demonstrated similar use of endovascular tools and angiography imaging, with an increased feeling of teamwork, communication and patient safety as a result of procedure rehearsal [50]. This concept has important implications for many different oncological procedures including laparoscopic prostate, adrenal or liver resections. It can be foreseen that procedure rehearsal is included in the multi disciplinary management plan for patients that require laparoscopic surgery for cancer treatment. It would be useful for pre-operative planning of solid tumours that have abnormal anatomical variations, or high complexity. This would include full team procedure rehearsal of the operation in a skills laboratory or simulated operating suite which consequently could enhance patient outcome [51]. Discussion The operations performed for the management of oncological diseases are extremely complex, and require the surgeon to have considerable technical skill. Previous generations of surgeons developed their technical skills over a long period of time. Skills were acquired in the operating room by performing a large number of operations using graduated responsibility. It is clear that current training methods are no longer sufficient to provide this time and surgical experience. Simulation, and in particular, VR simulation has been advocated as a solution for this shortfall in training time. It has been widely demonstrated that VR simulation can improve skills acquisition for surgical trainees. The VR simulators currently on the market for laparoscopic surgery have been widely validated for their use in surgical training and assessment. However, despite the fact that VR simulation has been established for well over a decade, it has been slowly adopted, and although there is extensive research validating the VR simulators for use as training and assessment tools, there is limited evidence supporting the impact they will have on future OR performance. It is well documented that surgeons have a finite time to train in, it is therefore vital that they use the time they have as efficiently as possible. Further work on the TER of VR
138
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139
simulation would allow us to accurately state how efficient each simulator is for teaching technical skills, and therefore can advocate the use of the most efficient simulators for surgeons in training. It is also important to analyse the cost to the healthcare system that VR simulation will have. VR simulation requires a considerable financial investment to purchase them. Therefore to encourage hospital trusts and universities to acquire VR simulators, it is vital to be able to demonstrate that each hour of training in a VR simulator will reduce operating room costs by X dollars. It would also be useful to evaluate how many students need to use a VR simulator
and for how long, to make it cost efficient for an institution. This would define the “number needed to train” on each simulator to make them cost efficient. VR simulation has been established as a valid method for teaching technical skills to surgeons. The initial criticism of VR simulation was that it was not realistic, is too costly and has no correlation to future OR performance. There is now considerable evidence demonstrating increased realism of simulators, but more work is needed to motivate and educate the surgical community in the effectiveness of VR simulation in terms of cost and future OR performance.
Appendix Table 1 Summary of other laparoscopic simulators. Simulator
Module
Company
Description
Surgical Educational Platform Surgical Educational Platform Surgical Educational Platform Surgical Educational Platform Virtual Endoscopic Surgery Training system (VEST) Virtual Endoscopic Surgery Training system (VEST) Xitact LS500
SEP Basic SEP Cholecystectomy SEP Ectopic Pregnancy SEP Robot VSOne Cho for VEST
Simsurgery Simsurgery Simsurgery Simsurgery Select-IT VEST Systems AG (Karlsruh, Germany) Select-IT VEST Systems AG (Karlsruh, Germany) Xitact SA, Morges, Switzerland
Simulator for developing an practising basic laparoscopic skills A learning module for laparoscopic cholecystectomy. A learning module for the removal of an ectopic pregnancy. A learning module for robotic surgery For teaching and training laparoscopic cholecystectomy
VSOne Gyn for VEST
For laparoscopic gynaecology training Laparoscopic simulator that provided a modular training environment for basic laparoscopic skills and laparoscopic procedural skills [25].
Table 2 Qualities of the ideal surgical assessment tool defined by Aggarwal et al. [33]. Term
Definition
Feasibility Face Validity Content validity
Measure of whether something is capable of being done or carried out Extent to which the examination resembles real life situations Extent to which the domain that is being measured is measured by the assessment tool e for example, while trying to assess technical skills we may actually be testing knowledge Extent to which a test measures the trait it purports to measure; one inference of construct validity is the extent to which a test discriminates between various levels of expertise Extent to which the results of the assessment tool correlate with the gold standard for that domain Ability of examination to predict future performance Measure of a test to generate similar results when applied at two different points Measure of the extent of agreement between two or more observers when rating the performance of an individual
Construct validity Concurrent validity Predictive validity Test-retest reliability Inter-rater reliability
Table 3 Summary of the validation of VR simulators. VR simulator
Face validity
Content validity
Construct validity
Concurrent validity
Predictive validity
MIST-VR ProMIS LapSim LAP Mentor
e Yes [46] Yes [18,34e36] Yes [21,26,37]
e e Yes [18,34e36] Yes [21,26,37]
Yes Yes Yes Yes
Yes Yes Yes Yes
e e e e
References [1] Reznick RK, MacRae H. Teaching surgical skillsechanges in the wind. N Engl J Med 2006;355(25):2664e9. [2] Kim MJ, Monson JR. The future of innovation and training in surgical oncology. Surg Oncol; 2010. [3] Shah PR, Joseph A, Haray PN. Laparoscopic colorectal surgery: learning curve and training implications. Postgrad Med J 2005;81(958):537e40. [4] Rattner DW, Apelgren KN, Eubanks WS. The need for training opportunities in advanced laparoscopic surgery. Surg Endosc 2001;15(10):1066e70. [5] Cathcart PJ, van der Meulen J, Emberton M, Kelly J. Volume-mortality for cystectomy. Centralisation of cancer services vindicated. BMJ 2010;340: c3336. [6] Aggarwal R, Darzi A. Technical-skills training in the 21st century. N Engl J Med 2006;355(25):2695e6. [7] Rieder E, Swanstrom LL. Advances in cancer surgery: natural orifice surgery (NOTES) for oncological diseases. Surg Oncol; 2010. [8] Monson JR. Advanced techniques in abdominal surgery. BMJ 1993;307(6915): 1346e50.
[24,30,34,35] [22,38e40] [18,34e36] 20,21,26
[30,34,36,37] [39] [18,34e36] 20,21,26
[9] Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, FunchJensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004;91(2):146e50. [10] Reissman P, Cohen S, Weiss EG, Wexner SD. Laparoscopic colorectal surgery: ascending the learning curve. World J Surg 1996;20(3):277e81. discussion 82. [11] Satava RM. Emerging medical applications of virtual reality: a surgeon’s perspective. Artif Intell Med 1994;6(4):281e8. [12] Figert PL, Park AE, Witzke DB, Schwartz RW. Transfer of training in acquiring laparoscopic skills. J Am Coll Surg 2001;193(5):533e7. [13] Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg 1999;177(1):28e32. [14] Gallagher AG, Cates CU. Virtual reality training for the operating room and cardiac catheterisation laboratory. Lancet 2004;364(9444):1538e40. [15] Maran NJ, Glavin RJ. Low- to high-fidelity simulation e a continuum of medical education? Med Educ 2003;37(Suppl. 1):22e8. [16] Wilson MS, Middlebrook A, Sutton C, Stone R, McCloy RF. Mist VR: a virtual reality trainer for laparoscopic surgery assesses performance. Ann R Coll Surg Engl 1997;79(6):403e4. [17] Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills training and assessment. Br J Surg 2004;91(12):1549e58.
T.M. Lewis et al. / Surgical Oncology 20 (2011) 134e139 [18] Aggarwal R, Grantcharov TP, Eriksen JR, Blirup D, Kristiansen VB, FunchJensen P, et al. An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 2006;244(2):310e4. [19] Hart R, Karthigasu K. The benefits of virtual reality simulator training for laparoscopic surgery. Curr Opin Obstet Gynecol 2007;19(4):297e302. [20] Andreatta PB, Woodrum DT, Birkmeyer JD, Yellamanchilli RK, Doherty GM, Gauger PG, et al. Laparoscopic skills are improved with LapMentor training: results of a randomized, double-blinded study. Ann Surg 2006;243(6): 854e60. discussion 60e3. [21] Aggarwal R, Crochet P, Dias A, Ziprin P, Darzi A. A scientific method for development of a whole procedure virtual reality training curriculum for laparoscopic cholecystectomy. Br J Surg; 2009. [22] Van Sickle KR, McClusky 3rd DA, Gallagher AG, Smith CD. Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surg Endosc 2005;19(9):1227e31. [23] Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, et al. Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 2005;19(12):1523e32. [24] Haptica. The world-leading ProMIS Surgical Simulator. Ireland. [25] Schijven M, Jakimowicz J. Virtual reality surgical laparoscopic simulators. Surg Endosc 2003;17(12):1943e50. [26] McDougall EM, Corica FA, Boker JR, Sala LG, Stoliar G, Borin JF, et al. Construct validity testing of a laparoscopic surgical simulator. J Am Coll Surg 2006; 202(5):779e87. [27] Taffinder N, Sutton C, Fishwick RJ, McManus IC, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomised controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform 1998;50:124e30. [28] Gallagher AG, Satava RM. Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Learning curves and reliability measures. Surg Endosc 2002;16(12):1746e52. [29] McNatt SS, Smith CD. A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons. Surg Endosc 2001;15(10):1085e9. [30] Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P. Virtual reality computer simulation. Surg Endosc 2001;15(3):242e4. [31] Gallagher HJ, Allan JD, Tolley DA. Spatial awareness in urologists: are they different? BJU Int 2001;88(7):666e70. [32] Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos P, et al. Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc 2005; 19(3):401e5. [33] Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L, Gauger PG, Minter RM. Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 2006;191(1):28e32. [34] van Dongen KW, Tournoij E, van der Zee DC, Schijven MP, Broeders IA. Construct validity of the LapSim: can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc 2007;21(8): 1413e7. [35] Aggarwal R, Grantcharov T, Moorthy K, Hance J, Darzi A. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg 2006;191(1):128e33.
139
[36] Bajka M, Tuchschmid S, Fink D, Szekely G, Harders M. Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system. Surg Endosc;24(1):79e88. [37] Hyltander A, Liljegren E, Rhodin PH, Lonroth H. The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc 2002; 16(9):1324e8. [38] Broe D, Ridgway PF, Johnson S, Tierney S, Conlon KC. Construct validation of a novel hybrid surgical simulator. Surg Endosc 2006;20(6):900e4. [39] Ritter EM, Kindelan TW, Michael C, Pimentel EA, Bowyer MW. Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS). Surg Endosc 2007;21(8):1441e5. [40] Botden SM, Buzink SN, Schijven MP, Jakimowicz JJ. Augmented versus virtual reality laparoscopic simulation: what is the difference? a comparison of the ProMIS augmented reality laparoscopic simulator versus LapSim virtual reality laparoscopic simulator. World J Surg 2007;31(4):764e72. [41] Sherman V, Feldman LS, Stanbridge D, Kazmi R, Fried GM. Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator. Surg Endosc 2005;19(5):678e82. [42] Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236(4):458e63. discussion 63e4. [43] Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 2009;338:b1802. [44] Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg 2007;246(5):771e9. [45] Ahlberg G, Heikkinen T, Iselius L, Leijonmarck CE, Rutqvist J, Arvidsson D. Does training in a virtual reality simulator improve surgical performance? Surg Endosc 2002;16(1):126e9. [46] Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky 3rd DA, et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007;193(6):797e804. [47] Aggarwal R, Hance J, Undre S, Ratnasothy J, Moorthy K, Chang A, et al. Training junior operative residents in laparoscopic suturing skills is feasible and efficacious. Surgery 2006;139(6):729e34. [48] Incremental transfer and cost effectiveness of ground-based flight trainers in a university aviation program. In: Proceedings of the Human Factors and Ergonomics Society 49th annual meeting; 2005. [49] Willaert W, Aggarwal R, Bicknell C, Hamady M, Darzi A, Vermassen F, et al. Patient-specific simulation in carotid artery stenting. J Vasc Surg 2010;52(6): 1700e5. [50] Willaert WI, Aggarwal R, Nestel DF, Gaines PA, Vermassen FE, Darzi AW, et al. Patient-specific simulation for endovascular procedures: qualitative evaluation of the development process. Int J Med Robot 2010;6(2):202e10. [51] Willaert WI, Aggarwal R, Van Herzeele I, O’Donoghue K, Gaines PA, Darzi AW, et al. Patient-specific endovascular simulation Influences Interventionalists performing carotid artery stenting procedures. Eur J Vasc Endovasc Surg; 2011.