- Email: [email protected]
Conventional Laparoscopic vs Robotic Training: Which is Better for Naive Users? A Randomized Prospective Crossover Study
ORIGINAL REPORTS
Conventional Laparoscopic vs Robotic Training: Which is Better for Naive Users? A Randomized Prospective Crossover Study Syed Omar Hassan, BA,* Jaimin Dudhia, BA,* Labiq H. Syed, MD,* Kalpesh Patel, MD,† Maham Farshidpour, MD,* Steven C. Cunningham, MD,* and Gopal C. Kowdley, MD, PhD, FACS* Department of Surgery, Saint Agnes Hospital, Baltimore, Maryland; and †Department of Oncology, Johns Hopkins University School of Medicine, Baltimore, Maryland *
OBJECTIVE: Robotic training (RT) using the da Vinci
CONCLUSIONS: Speeds were faster overall with RT than
skills simulator and conventional training (CT) using a laparoscopic “training box” are both used to augment operative skills in minimally invasive surgery. The current study tests the hypothesis that skill acquisition is more rapid using RT than using CT among naive learners.
with CT, but the percentage of speed improvement with trials was similar, suggesting similar learning curves, with C minimal transfer effect appreciated. ( J Surg ]:]]]-]]]. J 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
DESIGN AND PARTICIPANTS: A total of 40 subjects
KEY WORDS: robotic training, conventional laparoscopy,
without laparoscopic or robotic surgical experience were enrolled and randomized to begin with either RT or CT. Then, 2 specific RT tasks were reproduced for CT and repeated 5 times each with RT and CT. Time and quality indicators were measured quantitatively. A crossover technique was used to control for in-study experience bias.
learning curve, naive, transfer effect, efficient
RESULTS: The tasks “pick and place jacks” (PP) and “thread the rings” (TR) were achieved faster with RT than with CT despite crossover (p o 0.0001). An RT-favoring difference was observed in speed for both tasks when changing modality. Percentage improvement with increasing trials was similar for RT and CT: RT completion time averaged 39 seconds and 211 seconds (PP and TR, respectively), compared with 65 seconds and 362 seconds when using CT (p o 0.0001); final improvement averaged 26% and 46% for RT (PP and TR, respectively) vs 31% and 47% for CT (p was 0.76 for PP and 0.20 for TR). Within the PP task, RT times averaged 41 seconds without previous CT experience vs 35 seconds with previous CT experience (p ¼ 0.20); CT times averaged 61 seconds without and 69 seconds with previous RT experience (p ¼ 0.48). Comparable times for the TR task were 212 seconds vs 216 seconds (p ¼ 0.66) and 388 seconds vs 334 seconds (p ¼ 0.17). Both instrument collisions and excessive force occurred more commonly for RT than for CT within the TR task (p o 0.0001).
Correspondence: Inquiries to Gopal C. Kowdley, MD, PhD, FACS, Department of Surgery, St. Agnes Hospital 900 S. Caton Avenue, MB 207, Baltimore, MD 21229; fax: (410)-368 8727; e-mail: [email protected]
COMPETENCIES: Patient Care, Practice Based Learning
and Improvement
INTRODUCTION Though most minimally invasive surgeries are still performed with conventional laparoscopic techniques, robotic surgery (RS) is an increasingly common alternative modality, especially in urology, gynecology, and certain general surgical settings.1,2 The da Vinci Surgical System (Intuitive Surgical, Sunnyvale, CA) is the only US Food and Drug Administration–approved surgical robotic assistant in the United States. It is widely considered to have the potential to compensate for technical drawbacks inherent in conventional laparoscopic surgery (CLS), such as limited degree of freedom, 2-dimensional (2D) vision, and fulcrum and pivoting effects. By contrast, advantages of RS include 3D, high-definition stereoscopic vision, hand tremor–canceling ability, and EndoWrist pivoting technology, allowing for enhanced dexterity, precision, and control.3,4 The benefits of robotics seem more evident where a fine dissection and complex surgical reconstruction are required.5 Even though movement dynamics of the robotic master manipulators are currently still believed to have room for future improvement,6 current robotic systems show superior handling and ergonomics compared with
Journal of Surgical Education & 2015 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.12.008
1
CLS techniques.7,8 A clear advantage of CLS over currentgeneration RS is the presence of haptic feedback in CLS, which is absent in RS. As RS systems, notably the da Vinci system, become increasingly mainstream and standard of care in certain welldefined surgical settings, the number of studies comparing RS and CLS has likewise increased. As a topic of comparative research, learning curves can inform surgical training and the adoption of new clinical procedures and devices; a representative review of learning-curve literature9 has called for improved reporting and statistical evaluation of learning curves. However, most previous studies have included participants with varying degrees of experience, which complicates interpretation of this literature. With this in mind, we wanted to test both learning effectiveness and transference of skill using the 2 training methods most commonly used by general surgical residents at our hospital: a standard laparoscopic “training box” and a da Vinci Surgical Skills Simulator (dVSSS). Each has predictive validity as a reliable bellwether of future clinical performance. We hypothesized that in naive users, the rate of skills acquisition (learning curve) would be faster with robotic training (RT) than with conventional training (CT).
MATERIALS AND METHODS The subjects in the study were recruited from among medical students and junior surgical residents in the general surgery program at St. Agnes Hospital. Overall, 40 subjects were enrolled, of whom 32 were medical students and 8 were junior surgical residents, and 14 were women and 26 were men. Importantly, no subject had prior laparoscopic or robotic instrument experience: the junior residents, recruited and tested at the beginning of residency, were selected from foreign medical graduates who were unexposed to laparoscopic techniques in medical school. The medical students enrolled were third-year medical students
on their surgical clerkships. Participants were enrolled after they signed consent forms and after approval was obtained from the Institutional Review Board. The data were collected independently by 2 researchers. One researcher carried out instruction and camerawork, and the other did task assessment and data recording. The same roles were maintained in a consistent fashion throughout the study. For the CT component of the study, the standardized workspace was reproduced uniformly and in consistent fashion for each participant; each was given a standardized oral explanation of the desired tasks and a manual demonstration of correct technique. In addition, a single explanatory demonstration of each of the tasks was performed by a researcher before commencement of the trials. Camerawork for each participant was carried out in a uniform fashion by the same researcher throughout. The data parameters were recorded on a standardized datasheet (Appendix A). All participants were allowed to ask questions, to which uniform answers were given (Appendix B) and to notify the researchers of technical or equipment-related difficulty, which was addressed by halting the clock and restarting when the issue had been addressed. All participants were prohibited from observing or communicating with each other about their study experience. The robotic simulation space was reproduced to scale and dimensions, as shown in Figure 1. The materials used to carry out the study consisted of the Intuitive Surgical dVSSS, mounted on the da Vinci Si robotic console and scale wood-and-wire models of the pick and place jacks (PP) and thread the rings (TR) tasks from the dVSSS menu; needle holders 26173 KAR/KAL, Hopkins II 301 scope, and Tri-Cam NTSC camera system from KARL STORZ; TASKit laparoscopic training box and XCEL 12-mm trocars from Ethicon; and SL-693 C-14 needles with 2 in of 3-0 black Lactomer suture from Synetur. Two dVSSS tasks, PP and TR were selected from its menu and reproduced to scale for CT. These were repeated
FIGURE 1. Reproductions of robotic tasks to scale for laparoscopic use, left panel: TR, right panel: PP. 2
Journal of Surgical Education Volume ]/Number ] ] 2015
5 times for both RT and CT. Time and quality indicators were measured for 5 parameters, ultimately giving an aggregate of 2000 data points for analysis: (1) time to complete the task, (2) number of drops per trial (needle or jack), (3) number of instrument collisions (ICs), (4) number of instances of the instrument tip being out of view, and (5) number of times when excessive force (EF) was used. EF was measured in RT internally by the da Vinci software, and in CT, EF was measured by visually monitoring movement of anything other than graspers and the object being grasped, such as lifting, sliding, or shifting of the model apparatus, any rotation of the TR rings (free to rotate but always direction indexed before trial), and any slight, even transient, deformity of needle, rings, or jacks. All participants, the 8 junior residents and the 32 medical students, were randomized to begin with either RT or CT; all 40 names were collected together, drawn blindly from a pool, and assigned alternately to “RT-first” or “CT-first” subgroups, which decided the modality sequence by default; 15 subjects began with CT and 15 with RT. The subjects were then asked to perform the other task to create a crossover. This crossover controlled for prior learning by experience with other modality. Statistical analysis was performed using Microsoft Excel 2010 and the Real Statistics Using Excel resource pack (2014 Charles Zaiontz, www.real-statistics.com). An unpaired t test and a Mann-Whitney test of 2 independent samples were used to compare overall RT vs CT mean time to completion and the other 4 qualitative indicators listed above. Within each modality, the overall times to complete tasks were compared for improvement using the Friedman test (trials 1-5) and the Wilcoxon signed rank test (trial 1 vs 5). Both mean and normalized time-to-completion values were plotted to give learning curves comparing RT and CT. A Mann-Whitney test of 2 independent samples was used to calculate transfer effect (expertise within a modality with
vs without exposure to other modality). The significance of p was set at o0.05.
RESULTS Of the 40 subjects, 30 completed tasks with both RT and CT; 6 subjects did RT only and 4 did CT only. Both RT and CT were associated with significant improvement in times to completion with increasing trials (Figs. 2A and 3A). The tasks PP and TR were achieved faster with RT than with CT, despite crossover (p o 0.0001 for both PP and TR), whether for all subjects (Figs. 2A and 3A) or only paired data (Table 1). Though the tasks were completed faster overall with RT, percentage improvement with trial progression was similar for RT and CT across both tasks (Figs. 2B and 3B). Times to completion for RT averaged 39 seconds and 211 seconds (PP and TR, respectively), compared with 65 seconds and 362 seconds, respectively, when using CT (p o 0.0001), and percentage improvement averaged 26% and 46%, respectively, for RT (PP and TR, respectively) compared with 31% and 47% for the same exercises done using CT (nonsignificant), for PP and TR percentages, respectively. To address crossover within the PP task (Table 1), mean times using RT averaged 41 seconds without previous CT experience vs 35 seconds with previous CT experience (p ¼ 0.2); corresponding times using CT averaged 61 seconds without previous RT experience and 69 seconds with previous RT experience (p ¼ 0.48). Comparable times and p values within the TR task (Table 1) were 212 seconds vs 216 seconds (p ¼ 0.66) and 388 seconds vs 334 seconds (p ¼ 0.17), respectively. Incidences of EF were significantly increased (by approximately 5 times) while performing the TR task using RT compared with using CT (p o 0.0001). ICs were also more commonly seen in RT than in CT (p o 0.0001 for TR task, p ¼ 0.13 for PP task). Needle drops in the TR task
FIGURE 2. (A) Mean time(s) to complete “thread the rings” for all tested subjects. Statistics: Friedman p value (5 trials): RT o 0.001. CT o 0.001. Wilcoxon p value (trial 1 vs 5): RT o 0.001. CT o 0.001. Unpaired t test p value (RT vs CT): o0.001. (B) Normalized mean time(s) to complete “thread the rings” for all tested subjects. Statistics: Unpaired t test p value (RT vs CT): 0.338. Journal of Surgical Education Volume ]/Number ] ] 2015
3
FIGURE 3. (A) Mean time(s) to complete “pick and place jacks” for all tested subjects. Statistics: Friedman p value (5 trials): RT o 0.001. CT o 0.001. Wilcoxon p value (trial 1 vs 5): RT o 0.001 and CT o 0.001. Unpaired t test p value (RT vs CT): o 0.001. (B) Normalized mean time(s) to complete “pick and place jacks” for all tested subjects. Statistics: Unpaired t test p value (RT vs CT): 0.965.
and jack drops in the PP task were more common in the CT group (Tables 2 and 3), but not significantly so.
DISCUSSION Though task acquisition and performance has generally been shown to be superior with the robotic platform compared with conventional laparoscopy in the laboratory setting (RT vs CT), conclusive statements regarding the RS vs CLS learning curves in the surgical setting are difficult to make,10 probably because most individuals at the robotic console in an operation are trained surgeons with experience in both RS and CLS, at which point, learning curves may have already reached a working plateau. To date, clinical studies, especially those on naive users, are limited by a lack of standard definitions and objective assessment measurements. Our experimental study included students and residents without any prior laparoscopic instrument or robotic console experience, which prevented experience bias and allowed an adequate comparison of both modalities to assess TABLE 1. Mean Times(s) for Participant Subgroups (n ¼ 15 Each) and Transfer Effect Pick and Place Jacks, Mean ⫾ SD CT first CT last RT first RT last
61 69 41 35
⫾ ⫾ ⫾ ⫾
17 23 10 6
Mann-Whitney U test p value RT first vs RT last 0.199 CT first vs CT last 0.481 SD ¼ standard deviation. Bold values represent significant results (po0.05). 4
Thread the Rings, Mean ⫾ SD 388 334 212 216
⫾ ⫾ ⫾ ⫾
95 87 58 54
0.663 0.165
true learning curves in these naive learners. The RT system using dVSSS was chosen for several reasons, among which are that it is widely used to train residents in urology, gynecology, and general surgery, including general surgical residents at our institution, and also has good face, content, construct, concurrent, and predictive validities,11-13 with these benefits accruing to trainees and making its study relevant. The lowfidelity CT box we chose, with its standard instruments and handheld camera, is widely available for resident usage and has construct validity,14 with a comparable system forming the basis for the technical skills component of the Fundamentals of Laparoscopic Surgery program.14,15 The 2 wellcharacterized tasks chosen were both basic manipulations and are used regularly in surgical training programs. Both the PP and TR tasks were consistently faster with RT than with CT, regardless of which modality was used first. This finding is similar to that seen in other studies.16-18 Learning curves were achieved for both tasks across both RT and CT modalities, and the curves were largely similar for both CT and RT. This finding differs from that of other studies,16,18 where the learning curve has been reported to be steeper for CT, and is consistent with other studies finding equal effectiveness for both CT and RT, regardless of whether participants were entirely naive19 or partially experienced.20 The mean times expressed in the figures do TABLE 2. Quality Indicators for All 40 Participants for “Pick and Place Jacks” Task, Mean Values, and Total Number (n) of Occurrences in RT vs CT, and Comparison RT Mean (n) IC OOV EF Drops
0.5 0.1 0.2 0.2
(47) (10) (25) (24)
CT Mean (n) 0.3 0.1 0.8 0.4
(22) (18) (59) (35)
Unpaired t Test p Value 0.129 0.501 0.154 0.278
OOV ¼ instrument out of view. Journal of Surgical Education Volume ]/Number ] ] 2015
not account for the order in which a task was attempted. It has been suggested21,22 that regardless of experience level and independent of biomechanical advantages, the fact that RS uses 3D vision allows improved performance when using robotics, especially during complex manipulation, and this is consistent with our overall results. We tested for a transfer effect, which when assessing the RT or CT modalities consecutively, would be an increase in speed or efficiency with one modality after experience had previously been gained with the alternate modality. The need to investigate such an effect derives from the fact that surgeons in training may work predominantly with a modality other than that which they were trained in. We did not find a significant transfer effect with either RT or CT. This finding differs from those reported by some others,16,23 where previous CT experience was noted to shorten subsequent RT times, but not vice versa, and from that reported by other authors,24 where a transfer effect was seen in both modalities. A previous study25 comparing RT and CT, using 15 trials of 2 dexterity and depth-perception drills, found that improvements in performance were significantly greater during the first 5 repetitions (the steep portion of the learning curve) with a plateau at trial 8 for the robot and trial 10 for conventional instruments. In comparison, although the number of trials we used was not enough to approach this plateau for either modality, it was not our intention to study plateaus. We chose to examine the initial, more variable portion of the learning curve in our attempt to better characterize modality differences. Of course, in actual clinical practice, 5 or even 10 trials is not adequate training for basic surgical skills, even for trained users. At least for RS, the complexity of the human-machine interface can contribute significantly toward long learning curves that are seen even for laparoscopically trained surgeons.2 A study of actual clinicians undergoing training has found that after 90 robotic cases, there is a plateau of the mean overall score on the dVSSS.26 Our data indicate that the initial learning curves for both RT and CT are similar for basic manipulative tasks in naive users in laboratory conditions. It remains to be seen when a plateau occurs and if one modality eventually yields better efficiency than the other in a way that is relevant to training.
Drops (of the needle in the TR task and of the jack in the PP task) were more common in the CT group and were more frequent while performing the TR task. It is likely that attempts at grasper maneuvering, while hampered by fulcrum or pivot effect, contributed to the increased drops (and slower times) using CT in this group of naive users. Interestingly, incidences of EF were significantly increased while performing the needle-grasping and threading TR task using RT, with EF occurring approximately 5 times more commonly than with CT. ICs were also more commonly seen in RT than in CT, with some users commenting that they were unaware that their (virtual) graspers had touched. A lack of tactile feedback with RT may be the most likely explanation for these phenomena. It is likely that further advances in technology, increased experience with existing robotic technology, or the measuring of a more advanced skill set in a more experienced group would mitigate these perceived shortcomings seen with our group of beginners. Though robotics does command a discrete advantage in precision over standard instruments when fine tasks are performed, such as knot tying and running sutures,27 it has been found that when training practicing laparoscopic surgeons, basic laparoscopic maneuvering and suturing is faster and just as precise when performed manually, as with robotics.3,27 Finally, while it was measured, instruments out of view was the most methodologically problematic aspect of the analysis, as it involved an aspect of error (camera operator in CT) that was extrinsic to the immediate control of the research subject carrying out tasks. In choosing the outcome measures of our study, we tried to use clinically relevant end points from the da Vinci results screen that were amenable to human measurement. In actual surgical learning-curve studies, quantitative measurement of technical skill remains elusive (measured in a minority of studies), while time is the most commonly used proxy for the learning curve, used in 86% of comparable literature.9 Other previously identified variables for proficiency quantification on the da Vinci robot have included bimanual coordination and muscle group activation.28 It is possible that a more clinical comparison of learning curves between robotic and standard laparoscopic instrumentations might have been made had we used, for example, the ProMIS surgical simulator (CAE Healthcare, Montreal, Quebec) (measures total task time, instrument path length, and smoothness). ProMIS has been used successfully as a comparative tool between RT and CT in both naive and experienced users.29,30 Alternatively, the actual da Vinci robotic system does stream all motions and events via Ethernet Application Programming Interface, which provides transparent and immediate access to instrument motion and system configuration information2,31; however, the measurement and collection of such real-time motion data for corresponding laparoscopic tasks remains a technical challenge, with some authors using optoelectric motion/ force sensors32 to quantify and gather process measure data.
Journal of Surgical Education Volume ]/Number ] ] 2015
5
TABLE 3. Quality Indicators for All 40 Participants for “Thread the Rings” Task, Mean Values, and Total Number (n) of Occurrences in RT vs CT and Comparison RT Mean (n) IC OOV EF Drops
8.4 4.2 20.9 1.5
(746) (352) (1795) (125)
CT Mean (n) 2.6 1.8 4.3 1.7
(241) (186) (331) (134)
Unpaired t Test p Value o0.001 0.382 o0.001 0.649
OOV ¼ instrument out of view. Bold values represent significant results (po0.05).
Finally, the Objective Structured Assessment of Technical Skill (OSATS) is a validated and reliable performance evaluation tool33; albeit its global scoring system was difficult to tailor to the multivariate measurement of learning curves. Of course, the observed finding of a quicker completion of tasks using RT does not necessarily equate to a measurable advantage in actual practice, as our sample consisted of naive users and even 10 trials would be insufficient practice for a trainee, let alone a practicing surgeon who would surely have thoroughly practiced many laparoscopic and robotic maneuvers before taking the controls at a robotic console. Regarding its implications for resident training, however, comparative study of learning curves has the potential to change training regimens and channel precious resources towards more efficient practice methods. To that end, especially given the continuing reliability of conventional laparoscopic techniques to achieve successful outcomes in most minimally invasive general surgery, our results support the continued implementation of CT during residency as a “standard,” efficient pathway to achieve and enhance competency, regardless of ultimate modality usage (conventional or robotic) post specialty training. A recent review34 of 58 comparative studies of training models in laparoscopy has found both virtual reality RT and CT to be equally effective with similar consequent improvement in surgical skills; however, for most general surgical programs this alone does not justify the investment in an RT system. Despite the upsurge in use of robotic technology at some hospital centers, CT remains well validated and is known to improve actual surgical skills not only in its real-life counterpart, laparoscopy, but in robotic-assisted surgery as well.35 No longer a novel technology, and increasingly in use, RS has recently sparked debate regarding its cost-effectiveness and appropriate indications. Although these topics are not within the boundaries of our study, it is fair to say based on our results that robotic technology yet has room for improvement; specifically, the improvement of tactile feedback to the user’s hand would very likely improve new users’ tissue handling and haptic finesse, improving morbidity rates and decreasing the likelihood of adverse outcomes.
general surgical residents such as in our study. Inasmuch as its superior handling, better vision, and other ergonomic benefits lead to consistently faster completion of tasks, RS is more intuitive than CLS, though the lack of tactile feedback in the robotic manipulators may cause suboptimal outcomes, especially in naive users.
ACKNOWLEDGMENTS The authors wish to thank Ms. Margaret Firko, PA-C, St. Agnes Hospital, for her valuable assistance with preparation of the templates used for the data collection and the administration and OR staff of St. Agnes Hospital for valuable logistical assistance and support with the da Vinci robot.
REFERENCES 1. Kuo LJ, Lin YK, Chang CC, Tai CJ, Chiou JF, Chang YJ.
Clinical outcomes of robot-assisted intersphincteric resection for low rectal cancer: comparison with conventional laparoscopy and multifactorial analysis of the learning curve for robotic surgery. Int J Colorectal Dis. 2014;29(5):555-562. 2. Kumar R, Jog A, Malpani A, et al. Assessing system
operation skills in robotic surgery trainees. Int J Med Robot. 2012;8(1):118-124. 3. Garcia-Ruiz A, Gagner M, Miller JH, Steiner CP,
Hahn JF. Manual vs robotically assisted laparoscopic surgery in the performance of basic manipulation and suturing tasks. Arch Surg. 1998;133(9):957-961. 4. Guru KA, Kuvshinoff BW, Pavlov-Shapiro S, et al.
Impact of robotics and laparoscopy on surgical skills: a comparative study. J Am Coll Surg. 2007;204(1):96-101. 5. Patriti A, Addeo P, Buchs N, Casciola L, Morel P.
Advanced applications of robotics in digestive surgery. Transl Med UniSa. 2011;1:21-50. 6. Nisky I, Okamura AM, Hsieh MH. Effects of robotic
manipulators on movements of novices and surgeons. Surg Endosc. 2014;28(7):2145-2158. 7. Kenngott HG, Fischer L, Nickel F, Rom J, Rassweiler J,
CONCLUSION We found that speeds were significantly faster overall with RT than with CT, but that the percentage of speed improvement with increasing trials was similar for both RT and CT. We did not find that prior experience with one modality translated into a more efficient usage of the other. Both RT and CT play an important role in augmenting operative skills in minimally invasive surgery, with CT being comparable to RT in efficiency, as a training platform, and even more applicable as a training tool, especially for 6
Müller-Stich BP. Status of robotic assistance—a less traumatic and more accurate minimally invasive surgery? Langenbecks Arch Surg. 2012;397(3):333-341. 8. Lee GI, Lee MR, Clanton T, Sutton E, Park AE,
Marohn MR. Comparative assessment of physical and cognitive ergonomics associated with robotic and traditional laparoscopic surgeries. Surg Endosc. 2014; 28(2):456-465. 9. Harrysson IJ, Cook J, Sirimanna P, Feldman LS, Darzi A,
Aggarwal R. Systematic review of learning curves for Journal of Surgical Education Volume ]/Number ] ] 2015
minimally invasive abdominal surgery: a review of the methodology of data collection, depiction of outcomes, and statistical analysis. Ann Surg. 2014;260(1):37-45. 10. Kho RM. Comparison of robotic-assisted laparoscopy
versus conventional laparoscopy on skill acquisition and performance. Clin Obstet Gynecol. 2011;54 (3):376-381. 11. Alzahrani T, Haddad R, Alkhayal A, et al. Validation
of the da Vinci Surgical Skill Simulator across three surgical disciplines: a pilot study. Can Urol Assoc J. 2013;7(7-8):E520-E529. 12. Schreuder HWR, Persson JEU, Wolswijk RGH, Ihse I,
Schijven MP, Verheijen RHM. Validation of a novel virtual reality simulator for robotic surgery. Sci World J. 2014:507076. 13. Connolly M, Seligman J, Kastenmeier A, Goldblatt M,
Gould JC. Validation of a virtual reality-based robotic surgical skills curriculum. Surg Endosc. 2014;28 (5):1691-1694. 14. de Montbrun SL, MacRae H. Simulation in surgical
education. Clin (3):156-165.
Colon
Rectal
Surg.
2012;25
15. Fried GM. FLS assessment of competency using
simulated laparoscopic tasks. J Gastrointest Surg. 2008;12(2):210-212. 16. Anderberg M, Larsson J, Kockum CC, Arnbjörnsson E.
Robotics versus laparoscopy–an experimental study of the transfer effect in maiden users. Ann Surg Innov Res. 2010;4:3. 17. Sarle R, Tewari A, Shrivastava A, Peabody J, Menon
M. Surgical robotics and laparoscopic training drills. J Endourol. 2004;18(1):63-66. 18. Heemskerk J, van Gemert WG, de Vries J, Greve J,
Bouvy ND. Learning curves of robot-assisted laparoscopic surgery compared with conventional laparoscopic surgery: an experimental study evaluating skill acquisition of robot-assisted laparoscopic tasks compared with conventional laparoscopic tasks in inexperienced users. Surg Laparosc Endosc Percutan Tech. 2007;17(3):171-174.
21. LaGrange CA, Clark CJ, Gerber EW, Strup SE.
Evaluation of three laparoscopic modalities: robotics versus three-dimensional vision laparoscopy versus standard laparoscopy. J Endourol. 2008;22(3):511-516. 22. Byrn JC, Schluender S, Divino CM, et al. Three-
dimensional imaging improves surgical performance for both novice and experienced operators using the da Vinci Robot System. Am J Surg. 2007;193(4): 519-522. 23. Panait L, Shetty S, Shewokis PA, Sanchez JA. Do
laparoscopic skills transfer to robotic surgery? J Surg Res. 2014;187(1):53-58. 24. Obek C, Hubka M, Porter M, Chang L, Porter JR.
Robotic versus conventional laparoscopic skill acquisition: implications for training. J Endourol. 2005;19 (9):1098-1103. 25. Maniar HS, Council ML, Prasad SM, Prasad SM, Chu
C, Damiano RJ Jr. Comparison of skill training with robotic systems and traditional endoscopy: implications on training and adoption. J Surg Res. 2005; 125(1):23-29. 26. Finnegan KT, Meraney AM, Staff I, et al. da Vinci
Skills Simulator construct validation study: correlation of prior robotic experience with overall score and time score simulator performance. Urology. 2012;80(2): 330-335. 27. Dakin GF, Gagner M. Comparison of laparoscopic
skills performance between standard instruments and two surgical robotic systems. Surg Endosc. 2003;17(4): 574-579. 28. Narazaki K, Oleynikov D, Stergiou N. Robotic surgery
training and performance: identifying objective variables for quantifying the extent of proficiency. Surg Endosc. 2006;20(1):96-103. 29. Narula VK, Watson WC, Davis SS, et al. A compu-
terized analysis of robotic versus laparoscopic task performance. Surg Endosc. 2007;21(12):2258-2261. 30. Chandra V, Nehra D, Parent R, et al. A comparison of
laparoscopic and robotic assisted suturing performance by experts and novices. Surgery. 2010;147(6):830-839.
19. Diesen DL, Erhunmwunsee L, Bennett KM, et al.
31. Hernandez JD, Bann SD, Munz Y, et al. Qualitative
Effectiveness of laparoscopic computer simulator versus usage of box trainer for endoscopic surgery training of novices. J Surg Educ. 2011;68(4):282-289.
and quantitative analysis of the learning curve of a simulated surgical task on the da Vinci system. Surg Endosc. 2004;18(3):372-378.
20. Yohannes P, Rotariu P, Pinto P, Smith AD, Lee BR.
32. Dubrowski A, Larmer JC, Leming JK, Brydges R,
Comparison of robotic versus laparoscopic skills: is there a difference in the learning curve? Urology. 2002;60(1):39-45.
Carnahan H, Park J. Quantification of process measures in laparoscopic suturing. Surg Endosc. 2006;20 (12):1862-1866.
Journal of Surgical Education Volume ]/Number ] ] 2015
7
33. Reznick R, Regehr G. Testing technical skills via an
innovative "bench station" examination. Am J Surg. 1997;173(3):226-230. 34. Willaert W, Van De Putte D, Van Renterghem K,
Van Nieuwenhove Y, Ceelen W, Pattyn P. Training models in laparoscopy: a systematic review comparing
their effectiveness in learning surgical skills. Acta Chir Belg. 2013;113(2):77-95. 35. Abaza R. The robotic surgery era and the role of
laparoscopy training. Ther Adv Urol. 2009;1(3): 161-165.
SUPPLEMENTARY MATERIALS Supplementary material cited in this article is available online at doi:10.1016/j.jsurg.2014.12.008.
8
Journal of Surgical Education Volume ]/Number ] ] 2015
Conventional Laparoscopic vs Robotic Training: Which is Better for Naive Users? A Randomized Prospective Crossover Study Syed Omar Hassan, BA,* Jaimin Dudhia, BA,* Labiq H. Syed, MD,* Kalpesh Patel, MD,† Maham Farshidpour, MD,* Steven C. Cunningham, MD,* and Gopal C. Kowdley, MD, PhD, FACS* Department of Surgery, Saint Agnes Hospital, Baltimore, Maryland; and †Department of Oncology, Johns Hopkins University School of Medicine, Baltimore, Maryland *
OBJECTIVE: Robotic training (RT) using the da Vinci
CONCLUSIONS: Speeds were faster overall with RT than
skills simulator and conventional training (CT) using a laparoscopic “training box” are both used to augment operative skills in minimally invasive surgery. The current study tests the hypothesis that skill acquisition is more rapid using RT than using CT among naive learners.
with CT, but the percentage of speed improvement with trials was similar, suggesting similar learning curves, with C minimal transfer effect appreciated. ( J Surg ]:]]]-]]]. J 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
DESIGN AND PARTICIPANTS: A total of 40 subjects
KEY WORDS: robotic training, conventional laparoscopy,
without laparoscopic or robotic surgical experience were enrolled and randomized to begin with either RT or CT. Then, 2 specific RT tasks were reproduced for CT and repeated 5 times each with RT and CT. Time and quality indicators were measured quantitatively. A crossover technique was used to control for in-study experience bias.
learning curve, naive, transfer effect, efficient
RESULTS: The tasks “pick and place jacks” (PP) and “thread the rings” (TR) were achieved faster with RT than with CT despite crossover (p o 0.0001). An RT-favoring difference was observed in speed for both tasks when changing modality. Percentage improvement with increasing trials was similar for RT and CT: RT completion time averaged 39 seconds and 211 seconds (PP and TR, respectively), compared with 65 seconds and 362 seconds when using CT (p o 0.0001); final improvement averaged 26% and 46% for RT (PP and TR, respectively) vs 31% and 47% for CT (p was 0.76 for PP and 0.20 for TR). Within the PP task, RT times averaged 41 seconds without previous CT experience vs 35 seconds with previous CT experience (p ¼ 0.20); CT times averaged 61 seconds without and 69 seconds with previous RT experience (p ¼ 0.48). Comparable times for the TR task were 212 seconds vs 216 seconds (p ¼ 0.66) and 388 seconds vs 334 seconds (p ¼ 0.17). Both instrument collisions and excessive force occurred more commonly for RT than for CT within the TR task (p o 0.0001).
Correspondence: Inquiries to Gopal C. Kowdley, MD, PhD, FACS, Department of Surgery, St. Agnes Hospital 900 S. Caton Avenue, MB 207, Baltimore, MD 21229; fax: (410)-368 8727; e-mail: [email protected]
COMPETENCIES: Patient Care, Practice Based Learning
and Improvement
INTRODUCTION Though most minimally invasive surgeries are still performed with conventional laparoscopic techniques, robotic surgery (RS) is an increasingly common alternative modality, especially in urology, gynecology, and certain general surgical settings.1,2 The da Vinci Surgical System (Intuitive Surgical, Sunnyvale, CA) is the only US Food and Drug Administration–approved surgical robotic assistant in the United States. It is widely considered to have the potential to compensate for technical drawbacks inherent in conventional laparoscopic surgery (CLS), such as limited degree of freedom, 2-dimensional (2D) vision, and fulcrum and pivoting effects. By contrast, advantages of RS include 3D, high-definition stereoscopic vision, hand tremor–canceling ability, and EndoWrist pivoting technology, allowing for enhanced dexterity, precision, and control.3,4 The benefits of robotics seem more evident where a fine dissection and complex surgical reconstruction are required.5 Even though movement dynamics of the robotic master manipulators are currently still believed to have room for future improvement,6 current robotic systems show superior handling and ergonomics compared with
Journal of Surgical Education & 2015 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.12.008
1
CLS techniques.7,8 A clear advantage of CLS over currentgeneration RS is the presence of haptic feedback in CLS, which is absent in RS. As RS systems, notably the da Vinci system, become increasingly mainstream and standard of care in certain welldefined surgical settings, the number of studies comparing RS and CLS has likewise increased. As a topic of comparative research, learning curves can inform surgical training and the adoption of new clinical procedures and devices; a representative review of learning-curve literature9 has called for improved reporting and statistical evaluation of learning curves. However, most previous studies have included participants with varying degrees of experience, which complicates interpretation of this literature. With this in mind, we wanted to test both learning effectiveness and transference of skill using the 2 training methods most commonly used by general surgical residents at our hospital: a standard laparoscopic “training box” and a da Vinci Surgical Skills Simulator (dVSSS). Each has predictive validity as a reliable bellwether of future clinical performance. We hypothesized that in naive users, the rate of skills acquisition (learning curve) would be faster with robotic training (RT) than with conventional training (CT).
MATERIALS AND METHODS The subjects in the study were recruited from among medical students and junior surgical residents in the general surgery program at St. Agnes Hospital. Overall, 40 subjects were enrolled, of whom 32 were medical students and 8 were junior surgical residents, and 14 were women and 26 were men. Importantly, no subject had prior laparoscopic or robotic instrument experience: the junior residents, recruited and tested at the beginning of residency, were selected from foreign medical graduates who were unexposed to laparoscopic techniques in medical school. The medical students enrolled were third-year medical students
on their surgical clerkships. Participants were enrolled after they signed consent forms and after approval was obtained from the Institutional Review Board. The data were collected independently by 2 researchers. One researcher carried out instruction and camerawork, and the other did task assessment and data recording. The same roles were maintained in a consistent fashion throughout the study. For the CT component of the study, the standardized workspace was reproduced uniformly and in consistent fashion for each participant; each was given a standardized oral explanation of the desired tasks and a manual demonstration of correct technique. In addition, a single explanatory demonstration of each of the tasks was performed by a researcher before commencement of the trials. Camerawork for each participant was carried out in a uniform fashion by the same researcher throughout. The data parameters were recorded on a standardized datasheet (Appendix A). All participants were allowed to ask questions, to which uniform answers were given (Appendix B) and to notify the researchers of technical or equipment-related difficulty, which was addressed by halting the clock and restarting when the issue had been addressed. All participants were prohibited from observing or communicating with each other about their study experience. The robotic simulation space was reproduced to scale and dimensions, as shown in Figure 1. The materials used to carry out the study consisted of the Intuitive Surgical dVSSS, mounted on the da Vinci Si robotic console and scale wood-and-wire models of the pick and place jacks (PP) and thread the rings (TR) tasks from the dVSSS menu; needle holders 26173 KAR/KAL, Hopkins II 301 scope, and Tri-Cam NTSC camera system from KARL STORZ; TASKit laparoscopic training box and XCEL 12-mm trocars from Ethicon; and SL-693 C-14 needles with 2 in of 3-0 black Lactomer suture from Synetur. Two dVSSS tasks, PP and TR were selected from its menu and reproduced to scale for CT. These were repeated
FIGURE 1. Reproductions of robotic tasks to scale for laparoscopic use, left panel: TR, right panel: PP. 2
Journal of Surgical Education Volume ]/Number ] ] 2015
5 times for both RT and CT. Time and quality indicators were measured for 5 parameters, ultimately giving an aggregate of 2000 data points for analysis: (1) time to complete the task, (2) number of drops per trial (needle or jack), (3) number of instrument collisions (ICs), (4) number of instances of the instrument tip being out of view, and (5) number of times when excessive force (EF) was used. EF was measured in RT internally by the da Vinci software, and in CT, EF was measured by visually monitoring movement of anything other than graspers and the object being grasped, such as lifting, sliding, or shifting of the model apparatus, any rotation of the TR rings (free to rotate but always direction indexed before trial), and any slight, even transient, deformity of needle, rings, or jacks. All participants, the 8 junior residents and the 32 medical students, were randomized to begin with either RT or CT; all 40 names were collected together, drawn blindly from a pool, and assigned alternately to “RT-first” or “CT-first” subgroups, which decided the modality sequence by default; 15 subjects began with CT and 15 with RT. The subjects were then asked to perform the other task to create a crossover. This crossover controlled for prior learning by experience with other modality. Statistical analysis was performed using Microsoft Excel 2010 and the Real Statistics Using Excel resource pack (2014 Charles Zaiontz, www.real-statistics.com). An unpaired t test and a Mann-Whitney test of 2 independent samples were used to compare overall RT vs CT mean time to completion and the other 4 qualitative indicators listed above. Within each modality, the overall times to complete tasks were compared for improvement using the Friedman test (trials 1-5) and the Wilcoxon signed rank test (trial 1 vs 5). Both mean and normalized time-to-completion values were plotted to give learning curves comparing RT and CT. A Mann-Whitney test of 2 independent samples was used to calculate transfer effect (expertise within a modality with
vs without exposure to other modality). The significance of p was set at o0.05.
RESULTS Of the 40 subjects, 30 completed tasks with both RT and CT; 6 subjects did RT only and 4 did CT only. Both RT and CT were associated with significant improvement in times to completion with increasing trials (Figs. 2A and 3A). The tasks PP and TR were achieved faster with RT than with CT, despite crossover (p o 0.0001 for both PP and TR), whether for all subjects (Figs. 2A and 3A) or only paired data (Table 1). Though the tasks were completed faster overall with RT, percentage improvement with trial progression was similar for RT and CT across both tasks (Figs. 2B and 3B). Times to completion for RT averaged 39 seconds and 211 seconds (PP and TR, respectively), compared with 65 seconds and 362 seconds, respectively, when using CT (p o 0.0001), and percentage improvement averaged 26% and 46%, respectively, for RT (PP and TR, respectively) compared with 31% and 47% for the same exercises done using CT (nonsignificant), for PP and TR percentages, respectively. To address crossover within the PP task (Table 1), mean times using RT averaged 41 seconds without previous CT experience vs 35 seconds with previous CT experience (p ¼ 0.2); corresponding times using CT averaged 61 seconds without previous RT experience and 69 seconds with previous RT experience (p ¼ 0.48). Comparable times and p values within the TR task (Table 1) were 212 seconds vs 216 seconds (p ¼ 0.66) and 388 seconds vs 334 seconds (p ¼ 0.17), respectively. Incidences of EF were significantly increased (by approximately 5 times) while performing the TR task using RT compared with using CT (p o 0.0001). ICs were also more commonly seen in RT than in CT (p o 0.0001 for TR task, p ¼ 0.13 for PP task). Needle drops in the TR task
FIGURE 2. (A) Mean time(s) to complete “thread the rings” for all tested subjects. Statistics: Friedman p value (5 trials): RT o 0.001. CT o 0.001. Wilcoxon p value (trial 1 vs 5): RT o 0.001. CT o 0.001. Unpaired t test p value (RT vs CT): o0.001. (B) Normalized mean time(s) to complete “thread the rings” for all tested subjects. Statistics: Unpaired t test p value (RT vs CT): 0.338. Journal of Surgical Education Volume ]/Number ] ] 2015
3
FIGURE 3. (A) Mean time(s) to complete “pick and place jacks” for all tested subjects. Statistics: Friedman p value (5 trials): RT o 0.001. CT o 0.001. Wilcoxon p value (trial 1 vs 5): RT o 0.001 and CT o 0.001. Unpaired t test p value (RT vs CT): o 0.001. (B) Normalized mean time(s) to complete “pick and place jacks” for all tested subjects. Statistics: Unpaired t test p value (RT vs CT): 0.965.
and jack drops in the PP task were more common in the CT group (Tables 2 and 3), but not significantly so.
DISCUSSION Though task acquisition and performance has generally been shown to be superior with the robotic platform compared with conventional laparoscopy in the laboratory setting (RT vs CT), conclusive statements regarding the RS vs CLS learning curves in the surgical setting are difficult to make,10 probably because most individuals at the robotic console in an operation are trained surgeons with experience in both RS and CLS, at which point, learning curves may have already reached a working plateau. To date, clinical studies, especially those on naive users, are limited by a lack of standard definitions and objective assessment measurements. Our experimental study included students and residents without any prior laparoscopic instrument or robotic console experience, which prevented experience bias and allowed an adequate comparison of both modalities to assess TABLE 1. Mean Times(s) for Participant Subgroups (n ¼ 15 Each) and Transfer Effect Pick and Place Jacks, Mean ⫾ SD CT first CT last RT first RT last
61 69 41 35
⫾ ⫾ ⫾ ⫾
17 23 10 6
Mann-Whitney U test p value RT first vs RT last 0.199 CT first vs CT last 0.481 SD ¼ standard deviation. Bold values represent significant results (po0.05). 4
Thread the Rings, Mean ⫾ SD 388 334 212 216
⫾ ⫾ ⫾ ⫾
95 87 58 54
0.663 0.165
true learning curves in these naive learners. The RT system using dVSSS was chosen for several reasons, among which are that it is widely used to train residents in urology, gynecology, and general surgery, including general surgical residents at our institution, and also has good face, content, construct, concurrent, and predictive validities,11-13 with these benefits accruing to trainees and making its study relevant. The lowfidelity CT box we chose, with its standard instruments and handheld camera, is widely available for resident usage and has construct validity,14 with a comparable system forming the basis for the technical skills component of the Fundamentals of Laparoscopic Surgery program.14,15 The 2 wellcharacterized tasks chosen were both basic manipulations and are used regularly in surgical training programs. Both the PP and TR tasks were consistently faster with RT than with CT, regardless of which modality was used first. This finding is similar to that seen in other studies.16-18 Learning curves were achieved for both tasks across both RT and CT modalities, and the curves were largely similar for both CT and RT. This finding differs from that of other studies,16,18 where the learning curve has been reported to be steeper for CT, and is consistent with other studies finding equal effectiveness for both CT and RT, regardless of whether participants were entirely naive19 or partially experienced.20 The mean times expressed in the figures do TABLE 2. Quality Indicators for All 40 Participants for “Pick and Place Jacks” Task, Mean Values, and Total Number (n) of Occurrences in RT vs CT, and Comparison RT Mean (n) IC OOV EF Drops
0.5 0.1 0.2 0.2
(47) (10) (25) (24)
CT Mean (n) 0.3 0.1 0.8 0.4
(22) (18) (59) (35)
Unpaired t Test p Value 0.129 0.501 0.154 0.278
OOV ¼ instrument out of view. Journal of Surgical Education Volume ]/Number ] ] 2015
not account for the order in which a task was attempted. It has been suggested21,22 that regardless of experience level and independent of biomechanical advantages, the fact that RS uses 3D vision allows improved performance when using robotics, especially during complex manipulation, and this is consistent with our overall results. We tested for a transfer effect, which when assessing the RT or CT modalities consecutively, would be an increase in speed or efficiency with one modality after experience had previously been gained with the alternate modality. The need to investigate such an effect derives from the fact that surgeons in training may work predominantly with a modality other than that which they were trained in. We did not find a significant transfer effect with either RT or CT. This finding differs from those reported by some others,16,23 where previous CT experience was noted to shorten subsequent RT times, but not vice versa, and from that reported by other authors,24 where a transfer effect was seen in both modalities. A previous study25 comparing RT and CT, using 15 trials of 2 dexterity and depth-perception drills, found that improvements in performance were significantly greater during the first 5 repetitions (the steep portion of the learning curve) with a plateau at trial 8 for the robot and trial 10 for conventional instruments. In comparison, although the number of trials we used was not enough to approach this plateau for either modality, it was not our intention to study plateaus. We chose to examine the initial, more variable portion of the learning curve in our attempt to better characterize modality differences. Of course, in actual clinical practice, 5 or even 10 trials is not adequate training for basic surgical skills, even for trained users. At least for RS, the complexity of the human-machine interface can contribute significantly toward long learning curves that are seen even for laparoscopically trained surgeons.2 A study of actual clinicians undergoing training has found that after 90 robotic cases, there is a plateau of the mean overall score on the dVSSS.26 Our data indicate that the initial learning curves for both RT and CT are similar for basic manipulative tasks in naive users in laboratory conditions. It remains to be seen when a plateau occurs and if one modality eventually yields better efficiency than the other in a way that is relevant to training.
Drops (of the needle in the TR task and of the jack in the PP task) were more common in the CT group and were more frequent while performing the TR task. It is likely that attempts at grasper maneuvering, while hampered by fulcrum or pivot effect, contributed to the increased drops (and slower times) using CT in this group of naive users. Interestingly, incidences of EF were significantly increased while performing the needle-grasping and threading TR task using RT, with EF occurring approximately 5 times more commonly than with CT. ICs were also more commonly seen in RT than in CT, with some users commenting that they were unaware that their (virtual) graspers had touched. A lack of tactile feedback with RT may be the most likely explanation for these phenomena. It is likely that further advances in technology, increased experience with existing robotic technology, or the measuring of a more advanced skill set in a more experienced group would mitigate these perceived shortcomings seen with our group of beginners. Though robotics does command a discrete advantage in precision over standard instruments when fine tasks are performed, such as knot tying and running sutures,27 it has been found that when training practicing laparoscopic surgeons, basic laparoscopic maneuvering and suturing is faster and just as precise when performed manually, as with robotics.3,27 Finally, while it was measured, instruments out of view was the most methodologically problematic aspect of the analysis, as it involved an aspect of error (camera operator in CT) that was extrinsic to the immediate control of the research subject carrying out tasks. In choosing the outcome measures of our study, we tried to use clinically relevant end points from the da Vinci results screen that were amenable to human measurement. In actual surgical learning-curve studies, quantitative measurement of technical skill remains elusive (measured in a minority of studies), while time is the most commonly used proxy for the learning curve, used in 86% of comparable literature.9 Other previously identified variables for proficiency quantification on the da Vinci robot have included bimanual coordination and muscle group activation.28 It is possible that a more clinical comparison of learning curves between robotic and standard laparoscopic instrumentations might have been made had we used, for example, the ProMIS surgical simulator (CAE Healthcare, Montreal, Quebec) (measures total task time, instrument path length, and smoothness). ProMIS has been used successfully as a comparative tool between RT and CT in both naive and experienced users.29,30 Alternatively, the actual da Vinci robotic system does stream all motions and events via Ethernet Application Programming Interface, which provides transparent and immediate access to instrument motion and system configuration information2,31; however, the measurement and collection of such real-time motion data for corresponding laparoscopic tasks remains a technical challenge, with some authors using optoelectric motion/ force sensors32 to quantify and gather process measure data.
Journal of Surgical Education Volume ]/Number ] ] 2015
5
TABLE 3. Quality Indicators for All 40 Participants for “Thread the Rings” Task, Mean Values, and Total Number (n) of Occurrences in RT vs CT and Comparison RT Mean (n) IC OOV EF Drops
8.4 4.2 20.9 1.5
(746) (352) (1795) (125)
CT Mean (n) 2.6 1.8 4.3 1.7
(241) (186) (331) (134)
Unpaired t Test p Value o0.001 0.382 o0.001 0.649
OOV ¼ instrument out of view. Bold values represent significant results (po0.05).
Finally, the Objective Structured Assessment of Technical Skill (OSATS) is a validated and reliable performance evaluation tool33; albeit its global scoring system was difficult to tailor to the multivariate measurement of learning curves. Of course, the observed finding of a quicker completion of tasks using RT does not necessarily equate to a measurable advantage in actual practice, as our sample consisted of naive users and even 10 trials would be insufficient practice for a trainee, let alone a practicing surgeon who would surely have thoroughly practiced many laparoscopic and robotic maneuvers before taking the controls at a robotic console. Regarding its implications for resident training, however, comparative study of learning curves has the potential to change training regimens and channel precious resources towards more efficient practice methods. To that end, especially given the continuing reliability of conventional laparoscopic techniques to achieve successful outcomes in most minimally invasive general surgery, our results support the continued implementation of CT during residency as a “standard,” efficient pathway to achieve and enhance competency, regardless of ultimate modality usage (conventional or robotic) post specialty training. A recent review34 of 58 comparative studies of training models in laparoscopy has found both virtual reality RT and CT to be equally effective with similar consequent improvement in surgical skills; however, for most general surgical programs this alone does not justify the investment in an RT system. Despite the upsurge in use of robotic technology at some hospital centers, CT remains well validated and is known to improve actual surgical skills not only in its real-life counterpart, laparoscopy, but in robotic-assisted surgery as well.35 No longer a novel technology, and increasingly in use, RS has recently sparked debate regarding its cost-effectiveness and appropriate indications. Although these topics are not within the boundaries of our study, it is fair to say based on our results that robotic technology yet has room for improvement; specifically, the improvement of tactile feedback to the user’s hand would very likely improve new users’ tissue handling and haptic finesse, improving morbidity rates and decreasing the likelihood of adverse outcomes.
general surgical residents such as in our study. Inasmuch as its superior handling, better vision, and other ergonomic benefits lead to consistently faster completion of tasks, RS is more intuitive than CLS, though the lack of tactile feedback in the robotic manipulators may cause suboptimal outcomes, especially in naive users.
ACKNOWLEDGMENTS The authors wish to thank Ms. Margaret Firko, PA-C, St. Agnes Hospital, for her valuable assistance with preparation of the templates used for the data collection and the administration and OR staff of St. Agnes Hospital for valuable logistical assistance and support with the da Vinci robot.
REFERENCES 1. Kuo LJ, Lin YK, Chang CC, Tai CJ, Chiou JF, Chang YJ.
Clinical outcomes of robot-assisted intersphincteric resection for low rectal cancer: comparison with conventional laparoscopy and multifactorial analysis of the learning curve for robotic surgery. Int J Colorectal Dis. 2014;29(5):555-562. 2. Kumar R, Jog A, Malpani A, et al. Assessing system
operation skills in robotic surgery trainees. Int J Med Robot. 2012;8(1):118-124. 3. Garcia-Ruiz A, Gagner M, Miller JH, Steiner CP,
Hahn JF. Manual vs robotically assisted laparoscopic surgery in the performance of basic manipulation and suturing tasks. Arch Surg. 1998;133(9):957-961. 4. Guru KA, Kuvshinoff BW, Pavlov-Shapiro S, et al.
Impact of robotics and laparoscopy on surgical skills: a comparative study. J Am Coll Surg. 2007;204(1):96-101. 5. Patriti A, Addeo P, Buchs N, Casciola L, Morel P.
Advanced applications of robotics in digestive surgery. Transl Med UniSa. 2011;1:21-50. 6. Nisky I, Okamura AM, Hsieh MH. Effects of robotic
manipulators on movements of novices and surgeons. Surg Endosc. 2014;28(7):2145-2158. 7. Kenngott HG, Fischer L, Nickel F, Rom J, Rassweiler J,
CONCLUSION We found that speeds were significantly faster overall with RT than with CT, but that the percentage of speed improvement with increasing trials was similar for both RT and CT. We did not find that prior experience with one modality translated into a more efficient usage of the other. Both RT and CT play an important role in augmenting operative skills in minimally invasive surgery, with CT being comparable to RT in efficiency, as a training platform, and even more applicable as a training tool, especially for 6
Müller-Stich BP. Status of robotic assistance—a less traumatic and more accurate minimally invasive surgery? Langenbecks Arch Surg. 2012;397(3):333-341. 8. Lee GI, Lee MR, Clanton T, Sutton E, Park AE,
Marohn MR. Comparative assessment of physical and cognitive ergonomics associated with robotic and traditional laparoscopic surgeries. Surg Endosc. 2014; 28(2):456-465. 9. Harrysson IJ, Cook J, Sirimanna P, Feldman LS, Darzi A,
Aggarwal R. Systematic review of learning curves for Journal of Surgical Education Volume ]/Number ] ] 2015
minimally invasive abdominal surgery: a review of the methodology of data collection, depiction of outcomes, and statistical analysis. Ann Surg. 2014;260(1):37-45. 10. Kho RM. Comparison of robotic-assisted laparoscopy
versus conventional laparoscopy on skill acquisition and performance. Clin Obstet Gynecol. 2011;54 (3):376-381. 11. Alzahrani T, Haddad R, Alkhayal A, et al. Validation
of the da Vinci Surgical Skill Simulator across three surgical disciplines: a pilot study. Can Urol Assoc J. 2013;7(7-8):E520-E529. 12. Schreuder HWR, Persson JEU, Wolswijk RGH, Ihse I,
Schijven MP, Verheijen RHM. Validation of a novel virtual reality simulator for robotic surgery. Sci World J. 2014:507076. 13. Connolly M, Seligman J, Kastenmeier A, Goldblatt M,
Gould JC. Validation of a virtual reality-based robotic surgical skills curriculum. Surg Endosc. 2014;28 (5):1691-1694. 14. de Montbrun SL, MacRae H. Simulation in surgical
education. Clin (3):156-165.
Colon
Rectal
Surg.
2012;25
15. Fried GM. FLS assessment of competency using
simulated laparoscopic tasks. J Gastrointest Surg. 2008;12(2):210-212. 16. Anderberg M, Larsson J, Kockum CC, Arnbjörnsson E.
Robotics versus laparoscopy–an experimental study of the transfer effect in maiden users. Ann Surg Innov Res. 2010;4:3. 17. Sarle R, Tewari A, Shrivastava A, Peabody J, Menon
M. Surgical robotics and laparoscopic training drills. J Endourol. 2004;18(1):63-66. 18. Heemskerk J, van Gemert WG, de Vries J, Greve J,
Bouvy ND. Learning curves of robot-assisted laparoscopic surgery compared with conventional laparoscopic surgery: an experimental study evaluating skill acquisition of robot-assisted laparoscopic tasks compared with conventional laparoscopic tasks in inexperienced users. Surg Laparosc Endosc Percutan Tech. 2007;17(3):171-174.
21. LaGrange CA, Clark CJ, Gerber EW, Strup SE.
Evaluation of three laparoscopic modalities: robotics versus three-dimensional vision laparoscopy versus standard laparoscopy. J Endourol. 2008;22(3):511-516. 22. Byrn JC, Schluender S, Divino CM, et al. Three-
dimensional imaging improves surgical performance for both novice and experienced operators using the da Vinci Robot System. Am J Surg. 2007;193(4): 519-522. 23. Panait L, Shetty S, Shewokis PA, Sanchez JA. Do
laparoscopic skills transfer to robotic surgery? J Surg Res. 2014;187(1):53-58. 24. Obek C, Hubka M, Porter M, Chang L, Porter JR.
Robotic versus conventional laparoscopic skill acquisition: implications for training. J Endourol. 2005;19 (9):1098-1103. 25. Maniar HS, Council ML, Prasad SM, Prasad SM, Chu
C, Damiano RJ Jr. Comparison of skill training with robotic systems and traditional endoscopy: implications on training and adoption. J Surg Res. 2005; 125(1):23-29. 26. Finnegan KT, Meraney AM, Staff I, et al. da Vinci
Skills Simulator construct validation study: correlation of prior robotic experience with overall score and time score simulator performance. Urology. 2012;80(2): 330-335. 27. Dakin GF, Gagner M. Comparison of laparoscopic
skills performance between standard instruments and two surgical robotic systems. Surg Endosc. 2003;17(4): 574-579. 28. Narazaki K, Oleynikov D, Stergiou N. Robotic surgery
training and performance: identifying objective variables for quantifying the extent of proficiency. Surg Endosc. 2006;20(1):96-103. 29. Narula VK, Watson WC, Davis SS, et al. A compu-
terized analysis of robotic versus laparoscopic task performance. Surg Endosc. 2007;21(12):2258-2261. 30. Chandra V, Nehra D, Parent R, et al. A comparison of
laparoscopic and robotic assisted suturing performance by experts and novices. Surgery. 2010;147(6):830-839.
19. Diesen DL, Erhunmwunsee L, Bennett KM, et al.
31. Hernandez JD, Bann SD, Munz Y, et al. Qualitative
Effectiveness of laparoscopic computer simulator versus usage of box trainer for endoscopic surgery training of novices. J Surg Educ. 2011;68(4):282-289.
and quantitative analysis of the learning curve of a simulated surgical task on the da Vinci system. Surg Endosc. 2004;18(3):372-378.
20. Yohannes P, Rotariu P, Pinto P, Smith AD, Lee BR.
32. Dubrowski A, Larmer JC, Leming JK, Brydges R,
Comparison of robotic versus laparoscopic skills: is there a difference in the learning curve? Urology. 2002;60(1):39-45.
Carnahan H, Park J. Quantification of process measures in laparoscopic suturing. Surg Endosc. 2006;20 (12):1862-1866.
Journal of Surgical Education Volume ]/Number ] ] 2015
7
33. Reznick R, Regehr G. Testing technical skills via an
innovative "bench station" examination. Am J Surg. 1997;173(3):226-230. 34. Willaert W, Van De Putte D, Van Renterghem K,
Van Nieuwenhove Y, Ceelen W, Pattyn P. Training models in laparoscopy: a systematic review comparing
their effectiveness in learning surgical skills. Acta Chir Belg. 2013;113(2):77-95. 35. Abaza R. The robotic surgery era and the role of
laparoscopy training. Ther Adv Urol. 2009;1(3): 161-165.
SUPPLEMENTARY MATERIALS Supplementary material cited in this article is available online at doi:10.1016/j.jsurg.2014.12.008.
8
Journal of Surgical Education Volume ]/Number ] ] 2015