- Email: [email protected]
A Randomized Controlled Trial of Skills Transfer: From Touch Surgery to Laparoscopic Cholecystectomy
j o u r n a l o f s u r g i c a l r e s e a r c h f e b r u a r y 2 0 1 9 ( 2 3 4 ) 2 1 7 e2 2 3
Available online at www.sciencedirect.com
ScienceDirect journal homepage: www.JournalofSurgicalResearch.com
A Randomized Controlled Trial of Skills Transfer: From Touch Surgery to Laparoscopic Cholecystectomy Swathikan Chidambaram, MBBS, BSc (Hons),1 Simon Erridge, MBBS, BSc (Hons),*,1 Daniel Leff, PhD, MS (Hons), FRCS, and Sanjay Purkayastha, MD, FRCS Department of Surgery and Cancer, Imperial College London, London, UK
article info
abstract
Article history:
Background: Surgical training has traditionally involved teaching trainees in the operating
Received 17 May 2018
room. However, intraoperative training is time-intensive and exposes patients to greater
Received in revised form
risks. Touch Surgery (TS) is an application that uses animation to provide simulation
12 August 2018
training via cognitive task analysis as an adjunct to intraoperative training.
Accepted 12 September 2018
Methods: Forty students were recruited and randomly allocated to either a control or
Available online xxx
intervention group. Each group received the same preparation before intervention, including a 10-min introduction to laparoscopic equipment and a 15-min educational
Keywords:
tutorial on laparoscopic cholecystectomies. The participants then received training via
Surgical training
either TS (intervention) or written information (control). Their performance was compared
Medical education
using a validated scoring tool on a porcine laparoscopic cholecystectomy model. Signifi-
Simulation
cance was defined as P < 0.050.
Touch Surgery
Results: In total, n ¼ 22 and n ¼ 18 participants were randomly assigned to intervention and control groups, respectively. There was no significant difference between age (P ¼ 0.320), year of medical school (P ¼ 0.322), handedness (P ¼ 1.000), or gender (P ¼ 0.360) of the groups.
The
overall
mean
performance
score
was
higher
for
intervention
(mean SD ¼ 41.9 22.5) than control (mean SD ¼ 24.7 19.6; P ¼ 0.016). There was no significant difference between scores for each intraoperative segment between the intervention and control group (P > 0.050). Conclusions: This study demonstrates that TS is effective for providing cognitive training in laparoscopic cholecystectomies to medical students. It is likely that this effect will be seen across modules and other platforms that use cognitive task analysis alongside high-fidelity animation. Further work is necessary to extend this to other surgical procedures for evaluating its longitudinal effectiveness. ª 2018 Elsevier Inc. All rights reserved.
Previously presented at the Society of Academic & Research Surgery (SARS), Nottingham, on January 11th, 2018. * Corresponding author. Division of Surgery, Department of Surgery & Cancer, Imperial College London, Academic Surgical Unit, 10th Floor QEQM, St Mary’s Hospital, South Wharf Road, London, W2 1NY, UK. Tel.: þ44 (0) 203 312 6666; fax: þ44 (0) 203 312 6309. E-mail address: [email protected] (S. Erridge). 1 Each of these authors contributed equally and therefore should be considered co-first author. 0022-4804/$ e see front matter ª 2018 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jss.2018.09.042
218
j o u r n a l o f s u r g i c a l r e s e a r c h f e b r u a r y 2 0 1 9 ( 2 3 4 ) 2 1 7 e2 2 3
Introduction Surgical training is aimed at equipping surgeons with both the technical and nontechnical skills required to perform an operation.1 Spurred by an increasing patient load and longer operating times, traditional methods of training in the operating room have become unstructured and costly.2 They also expose patients to potential errors as trainees progress along the learning curve.3 Furthermore, disruptive technology demands surgeons acquire novel skills rapidly to keep abreast with advances in surgical techniques and procedures.4 Hence, it is necessary to explore other viable options such as simulation training platforms for surgeons to learn and hone their skills. Such simulation platforms need to be evaluated before their introduction and use in surgical practice. Cognitive task analysis (CTA) is the process of creating a comprehensive breakdown of a procedure and is a technique shown to improve trainees’ procedural knowledge and technical skills uptake for central venous catheterization,5 percutaneous tracheostomy placement,6 and knee arthroscopy.7 Cognitive training before virtual reality (VR) endoscopic training has also demonstrated improved operative ability.8 Recent studies have shown simulation training to transfer to improved performance during real cases, with significant reductions in the time taken, fewer errors, and decreased patient discomfort.3,9-11 Touch Surgery (TS) is an interactive mobile CTA-based simulation and rehearsal tool with a touch screen interface for the self-teaching and assessing of various operative procedures based on expert-derived cognitive task analyses. Through the touch screen interface and multiple-choice questions, trainees are encouraged to learn and test their cognitive knowledge of procedural steps and surgical decisions. Trainees can track their achievements, and learners are encouraged to improve their procedural knowledge and achieve higher scores with fewer errors. The TS application is available for free on the Apple App Store (Apple Inc, California) and the Google Play store for Android (Google, California). The aim of this study was to assess the transferability of knowledge of cognitive task simulation and rehearsal app, TS, on the learner’s ability to perform laparoscopic cholecystectomy, and hence evaluate the usefulness of the application for incorporation into surgical training.
Methods TS mobile platform The TS platform (Kinosis Limited, London, UK) is an interactive application available on smartphones and tablets. It uses CTA framework alongside immersive multimedia animations to teach operative steps across a broad range of procedures and specialties. The primary target audience is surgical trainees with modules for different procedures authored by
experienced clinicians, but is accessible for clinicians at all stages of training.
Study design A randomized controlled trial was conducted in accordance with CONSORT guidelines.12 The TS mobile platform was compared against a control educational intervention for teaching the procedural steps of a laparoscopic cholecystectomy. The study was performed under the approval of a local medical education ethics panel.
Setting, participants, and allocation The trial was conducted under controlled conditions in a “dry” laboratory equipped with laparoscopic simulators. Participants were medical students currently studying medicine at Imperial College London. Each participant was provided with a certificate for his or her participation. To motivate performance, they were informed that the participant who performed the highest would be provided with an additional certificate detailing their performance. The inclusion criteria were assessed via declarations made within a consent form and were as follows: 1. Participants had completed a faculty-taught course on gastrointestinal anatomy. 2. Participants had no prior experience with TS. 3. The participants had no prior personal experience with laparoscopic surgery or simulation. Each group received the same preparation before intervention. First, each group was provided with a 10-minute introduction to laparoscopic equipment. This included teaching on how to handle instruments and the function of fundamental laparoscopic equipment including trocars, grasping forceps, Marylands, scissors, clip applicators, and diathermy. This was followed by psychomotor training according to tasks from the Fundamentals of Laparoscopic Surgery (FLS).13 The participants were required to complete the peg transfer and precision cutting tasks in accordance with FLS performance guidelines before instigating the educational intervention. This allowed participants in each group to achieve the same baseline proficiency in basic skills to enable them to complete the assessment. In addition, each subject was given a 15-min educational tutorial on laparoscopic cholecystectomies to place the simulation and previous anatomy knowledge in clinical context using a combination of text, images, and video. This included, for example, making all participants aware of the borders of Calot’s triangle and what the critical view of safety is.14 The participants who met the inclusion criteria were randomly assigned to group 1 (TS) or group 2 (control), using an online random number generator (Google, California). Allocation concealment was maintained by randomizing participants to each intervention after the study preparation. Concealment was broken at time of intervention. Setup of the assessment phase of the study was carried out before
chidambaram et al touch surgery skills transfer
219
this and kept uniform throughout to minimize any potential bias.
teaching content was consistent with the intervention. Time studying the material was limited to 15 min per student.
Intervention
Primary outcomes
Group 1 received all prior preparation alongside the intervention. This consisted of undertaking the laparoscopic cholecystectomy phase 3 module (module version: 110) on the TS application on a mobile tablet device (Fig. 1). This would consist of first completing the “learn phase” of the module. This directs users through dissection and ligation of both the cystic duct and artery, as well as dissection of the gall bladder from the liver bed using “drag and drop” tools. The participants then subsequently would have to repeat the “test phase” of the module. This asks participants to complete multiple-choice questions about the next intraoperative steps until they achieved a single perfect score (100%). Time using TS was limited to 15 min per participant.
The assessment of all participants was conducted using a cadaveric porcine liver and gall bladder, which was rotated at 180 to achieve retraction as previously described.15 A laparoscopic camera was set up before the start of any individual assessment to enable adequate visualization of the liver and gall bladder for the students. The camera was set up in an identical fashion for each assessment and kept in place by a bracket as to prevent confounding from an assistant. Assessment of each group was conducted immediately after they had received either the intervention or control. The procedure was recorded through the laparoscopic camera for blinded independent assessment. Cognitive performance was rated using a previously validated performance scale described by Eubanks et al. (Supplemental Material 2).16 This performance scale was used in preference over other validated scores due to its focus on cognitive performance rather than technical skills.17 This was edited to increase applicability to the study methodology. The scoring focused on the intraoperative steps and as such error points were not counted to prevent confounding from unintentional error in both cohorts who were both previously naı¨ve to laparoscopic skills. The steps were grouped according to initial exposure, initial dissection, cystic duct dissection,
Control Group 2 received all similar prior preparation to group 1. The control intervention however consisted of an itemized breakdown of the intraoperative steps and the order in which they are performed. This information was provided in a table format (Supplemental Material 1) and was compared against the TS module by two authors (S.C. and S.E.) to ensure
Fig. 1 e Snapshots of the Touch Surgery mobile platform showing both (A) learn phase and (B) test phase. (Color version of figure is available online.)
220
j o u r n a l o f s u r g i c a l r e s e a r c h f e b r u a r y 2 0 1 9 ( 2 3 4 ) 2 1 7 e2 2 3
cystic artery dissection, and gall bladder fossa dissection. The maximum score available for performing the intraoperative steps was 80. An independent assessor, who had conducted over 150 laparoscopic cholecystectomies as the primary operator, rated the recordings. The assessor was recruited from a separate institution and carried out all assessment at a remote location. All personal identifying information was removed from the videos, and the videos were assessed in a randomized order to ensure blinding to group allocation.
Statistical analysis The performance scale has previously been evaluated on the assumption of normal distribution.18 This was confirmed using a ShapiroeWilk test on the assessment scores. Analysis of cognitive performance was subsequently carried out using a t-test of the group means, expressed with standard deviations. Comparison of demographic data was carried out using a Fisher exact test. Analysis was performed using SPSS (v.24.0.0.0, IBM, Armonk, NY). Significance was determined at P < 0.050.
Results Forty participants were recruited to take part in the study. Twenty-two and 18 participants were randomly assigned to group 1 and group 2, respectively. One student was excluded from analysis in each group as a result of failure of video recording equipment (Fig. 2). Participant demographic data
are summarized in Table. There was no significant difference between groups in age (P ¼ 0.320), the year of medical school (P ¼ 0.322), handedness (P ¼ 1.000), or gender (P ¼ 0.360). Analysis of cognitive performance by individual intraoperative segments is demonstrated in Figure 3. The overall mean performance score was greater for group 1, 41.9 (22.5), than group 2, 24.7 (19.6; P ¼ 0.016). There was no significant difference found between either groups throughout all intraoperative segments. This included initial exposure (group 1 ¼ 3.4 3.1; group 2 ¼ 2.1 2.1; P ¼ 0.211), initial dissection (group 1 ¼ 9.3 6.0; group 2 ¼ 5.6 5.8; P ¼ 0.063), cystic duct dissection (group 1 ¼ 10.8 7.9; group 2 ¼ 6.5 7.2; P ¼ 0.091), cystic artery dissection (group 1 ¼ 5.5 10.1; group 2 ¼ 1.4 5.8; P ¼ 0.130), and gall bladder fossa dissection (group 1 ¼ 12.7 4.2; group 2 ¼ 9.1 6.9; P ¼ 0.070).
Discussion This study was performed to determine the usefulness of TS in transfer of knowledge to enable a naı¨ve student to perform the intraoperative steps of a laparoscopic cholecystectomy. The results of this study demonstrate that TS significantly contributes to improvements in novices’ cognitive performance during a laparoscopic cholecystectomy. TS is an application that aims to equip trainees with the necessary knowledge and thus improve the acquisition of surgical skills when carrying out the procedure. This has clear benefits in surgical education where simulation is being used with increased frequency.19
Fig. 2 e CONSORT flow diagram. (Color version of figure is available online.)
chidambaram et al touch surgery skills transfer
Table e Demographic data of trial participants. Demographics
Group 1 (n ¼ 22)
Group 2 (n ¼ 18)
Pvalue
Age
21.9 1.3
21.4 1.3
0.320
Female
10 (45%)
11 (61%)
0.360
Male
12 (55%)
7 (39%)
Gender
Handedness Left
2 (9%)
2 (11%)
20 (91%)
16 (89%)
Year 1
0
3 (17%)
Year 2
3 (14%)
2 (11%)
Year 3
8 (36%)
4 (22%)
Year 4
6 (27%)
6 (33%)
Year 5
5 (23%)
3 (17%)
Year 6
0
0
Right
1.000
Year group 0.322
The design of the study stipulated that only medical students with no prior training in performing a laparoscopic cholecystectomy were recruited. Previous studies have recruited surgical trainees as participants,20 and this allows for a variety of confounding effects, including previous experience and knowledge of both the procedure and laparoscopic skills. As such, the statistical difference in the mean performance found in this study can be more highly attributed to TS. While mean overall performance was statistically significant, the individual components did not show statistical significance. The improvements shown by students in this study were after minimal one-off exposure to TS. Surgical training is a longitudinal process and with further investigation, TS may be shown to increase cognitive performance for each intraoperative segment and subsequently shorten the learning curve. Another strength of the study is the use of the laparoscopic cholecystectomy as the procedure assessed, which has been validated as the standard procedure of assessment.21 Similarly, validated tools such as the FLS were used to both teach and assess the students.22
221
The results of this study build on previous work that analyzed the effectiveness of TS to increase knowledge acquisition. Previous evaluation of the intramedullary femoral nailing module demonstrated that the app provides a demonstrable increase in knowledge via both in-app testing and an expert-evaluated testing.23 Similar results were also demonstrated for the cardiopulmonary bypass module.24 TS was also shown to improve performance for simulated tendon repair as assessed by expert reviewers.25 The novelty of the present study, however, is that it is the first to validate knowledge transfer from TS via a previously validated scoring tool. A number of qualitative survey-based assessments have shown that users of TS find that it provides content validity and is enjoyable to use.21,24,26 To demonstrate the applicability of novel technologies, it is important to compare these against existing technologies. Previous work by Aggarwal et al. has demonstrated the validity of a VR competency-based simulation training program for laparoscopic cholecystectomy.27 This study has demonstrated the validity of simulation in training for specific procedures. However, this study used a high-fidelity LapSim VR laparoscopic simulator (Surgical Science, Gothenburg, Sweden). One of the greatest challenges facing simulation is the associated costs that come with such high-fidelity simulators, in addition to finding time away from service provision to accommodate training.19,28 TS is available for free on most smartphones and therefore provides a low-cost adjunct to allied simulation modalities that are currently available. The results of this study have highlighted the potential cognitive benefits of training through the TS platform for a naı¨ve cohort. Any simulation platform is likely to have a greater uptake among surgical trainees rather than in novices. It is more likely that TS will provide the greatest benefit for trainees in the early part of the learning curve for a specific operation. Mental rehearsal has been shown to improve the skills of junior surgeons.29 While the use of TS in specific mental rehearsal requires further research and predictions are speculative, this is where the platform would likely provide the most benefit for users. The study has some notable limitations. First, the baseline performance of participants was not collected before
Fig. 3 e Performance scores compared between groups 1 and 2 in each intraoperative domain and across the procedure as a whole. Group 1dintervention, group 2dcontrol; *P < 0.050.
222
j o u r n a l o f s u r g i c a l r e s e a r c h f e b r u a r y 2 0 1 9 ( 2 3 4 ) 2 1 7 e2 2 3
introducing the intervention. Therefore the apparent improvement in performance as a result TS may reflect preexisting confounders. However, the study design was set up to minimize these as much as possible. The inclusion criteria limited study participants to those with no prior experience of either TS or laparoscopic cholecystectomies. Moreover there were no significant differences in the stage of medical training of either group. No power calculation was conducted for this evaluation being that it is the first evaluation of TS via this approach. A post hoc power analysis found that the power for this study reached 73.3%. This work should be used to influence power calculations for future evaluations of this approach. In addition, it should be noted that the study is underpowered to detect any significant difference between the groups in each intraoperative segment. Another limitation is that the study only investigated one-off use and did not measure long-term retention. Surgical education is also a longitudinal process. Hence, while direct comparisons could be made between the control and intervention groups, the improvement in each participant’s performance over repetitive use could not be calculated to determine learning curves. In addition, a wider variety of procedures pertaining to general surgery could have been evaluated instead of just laparoscopic cholecystectomy to see if the effects of the application carry forward to other procedures as well. Forthcoming work will need to involve a larger population that can execute more tasks at higher repetitions to validate the consistency of the application and explore its incorporation into the training curriculum.
Conclusions Surgical training requires more efficient and cost-effective ways to equip trainees with the necessary skills. This study shows that TS is an effective platform for simulating general surgery procedures in teaching and training medical students. Further work is necessary to extend this conclusion to other procedures.
Acknowledgment The authors would like to give their acknowledgments to the following people: Dr Andre Chow and Dr Ali Bahsoun for their help in coordinating funding from Touch Surgery, United Kingdom (grant number: N/A) to fund this work; and Dr Vivek Bindal for his help in assessing the performance of the participants. Authors’ contributions: S.C, S.E, and S.P were involved in development of the study methodology. S.C. and S.E collected all data and performed data analysis. All authors contributed to article preparation and critical revisions.
Disclosure This work was supported by a grant from TS. The authors had full access to the data. The funders played no role in study design; collection, analysis, and interpretation of data; writing
of the report; or the decision to submit the article for publication. S.P. is a co-founder of TS, but has stepped down and has no financial relationship with the company. No authors have received financial support from TS. The authors have read the journal’s guidelines on finances and have no further conflicts of interest to disclose.
Supplementary data Supplementary data to this article can be found online at https://doi.org/10.1016/j.jss.2018.09.042.
references
1. Moorthy K, Munz Y, Adams S, et al. A human factors analysis of technical and team skills among surgical trainees during procedural simulations in a simulated operating theatre. Ann Surg. 2005;242:631e639. 2. Moorthy K, Munz Y, Forrest D, et al. Surgical crisis management skills training and assessment: a simulation [corrected]-based approach to enhancing operating room performance. Ann Surg J. 2006;244:139e147. 3. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg J. 2002;236:454e458. 4. Aggarwal R, Darzi A. Technical-skills training in the 21st century. N Engl J Med. 2006;355:2695e2696. 5. Velmahos GC, Toutouzas KG, Sillin LF, et al. Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory. Am J Surg. 2004;187:114e119. 6. Sullivan ME, Brown CVR, Peyre SE, et al. The use of cognitive task analysis to improve the learning of percutaneous tracheostomy placement. Am J Surg. 2007;193:96e99. 7. Kohls-Gatzoulis JA, Regehr G, Hutchison C. Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg. 2004;47:277e283. 8. Van Herzeele I, Aggarwal R, Neequaye S, et al. Cognitive training improves clinically relevant outcomes during simulated endovascular procedures. J Vasc Surg. 2008;48:1223e1230.e1. 9. Blum MG, Powers TW, Sundaresan S. Bronchoscopy simulator effectively prepares junior residents to competently perform basic clinical bronchoscopy. Ann Thorac Surg. 2004;78:287e291. 10. Grantcharov TP, Kristiansen VB, Bendix J, et al. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004;91:146e150. 11. Sedlack RE, Kolars JC, Alexander JA. Computer simulation training enhances patient comfort during endoscopy. Clin Gastroenterol Hepatol. 2004;2:348e352. 12. Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. 13. Surgeons S of AG and E. Fundamentals of laparoscopic surgery. Available at: https://www.flsprogram.org/; 2017. 14. Strasberg SM, Brunt LM. Rationale and use of the critical view of safety in laparoscopic cholecystectomy. J Am Coll Surg. 2010;211:132e138. 15. Aggarwal R, Crochet P, Dias A, et al. Development of a virtual reality training curriculum for laparoscopic cholecystectomy. Br J Surg. 2009;96:1086e1093.
chidambaram et al touch surgery skills transfer
16. Eubanks TR, Clements RH, Pohl D, et al. An objective scoring system for laparoscopic cholecystectomy. J Am Coll Surg. 1999;189:566e574. 17. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84:273e278. 18. Hwang H, Lim J, Kinnaird C, et al. Correlating motor performance with surgical error in laparoscopic cholecystectomy. Surg Endosc. 2006;20:651e655. 19. Stefanidis D, Sevdalis N, Paige J, et al. Simulation in surgery: what’s needed next? Ann Surg. 2015;261:846e853. 20. Sutherland LM, Middleton PF, Anthony A, et al. Surgical simulation: a systematic review. Ann Surg. 2006;243:291e300. 21. Kowalewski K-F, Hendrie JD, Schmidt MW, et al. Validation of the mobile serious game application Touch Surgery for cognitive training and assessment of laparoscopic cholecystectomy. Surg Endosc. 2017;31:4058e4066. 22. Marks JM. 1. Fundamentals of laparoscopic surgery (FLS) and of endoscopic surgery (FES). In: Soper NJ, Scott-Conner CEH, eds. The SAGES Manual: Volume 1 Basic Laparoscopy and Endoscopy. New York, NY: Springer New York; 2012:3e13. 23. Sugand K, Mawkin M, Gupte C. Training effect of using Touch Surgery for intramedullary femoral nailing. Injury. 2016;47:448e452.
223
24. Brewer ZE, Ogden WD, Fann JI, et al. Creation and global deployment of a mobile, application-based cognitive simulator for cardiac surgical procedures. Semin Thorac Cardiovasc Surg. 2016;28:1e9. 25. Bunogerane GJ, Taylor K, Lin Y, et al. Using touch surgery to improve surgical education in low- and middle-income settings: a randomized control trial. J Surg Educ. 2018;75:231e237. 26. Sugand K, Mawkin M, Gupte C. Validating Touch SurgeryTM: a cognitive task simulation and rehearsal app for intramedullary femoral nailing. Injury. 2015;46:2212e2216. 27. Aggarwal R, Ward J, Balasundaram I, et al. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg. 2007;246:771e779. 28. Hosny SG, Johnston MJ, Pucher PH, et al. Barriers to the implementation and uptake of simulationbased training programs in general surgery: a multinational qualitative study. J Surg Res. 2017;220:419e426.e2. 29. Arora S, Aggarwal R, Sirimanna P, et al. Mental practice enhances surgical technical skills: a randomized controlled study. Ann Surg. 2011;253:265e270.
Available online at www.sciencedirect.com
ScienceDirect journal homepage: www.JournalofSurgicalResearch.com
A Randomized Controlled Trial of Skills Transfer: From Touch Surgery to Laparoscopic Cholecystectomy Swathikan Chidambaram, MBBS, BSc (Hons),1 Simon Erridge, MBBS, BSc (Hons),*,1 Daniel Leff, PhD, MS (Hons), FRCS, and Sanjay Purkayastha, MD, FRCS Department of Surgery and Cancer, Imperial College London, London, UK
article info
abstract
Article history:
Background: Surgical training has traditionally involved teaching trainees in the operating
Received 17 May 2018
room. However, intraoperative training is time-intensive and exposes patients to greater
Received in revised form
risks. Touch Surgery (TS) is an application that uses animation to provide simulation
12 August 2018
training via cognitive task analysis as an adjunct to intraoperative training.
Accepted 12 September 2018
Methods: Forty students were recruited and randomly allocated to either a control or
Available online xxx
intervention group. Each group received the same preparation before intervention, including a 10-min introduction to laparoscopic equipment and a 15-min educational
Keywords:
tutorial on laparoscopic cholecystectomies. The participants then received training via
Surgical training
either TS (intervention) or written information (control). Their performance was compared
Medical education
using a validated scoring tool on a porcine laparoscopic cholecystectomy model. Signifi-
Simulation
cance was defined as P < 0.050.
Touch Surgery
Results: In total, n ¼ 22 and n ¼ 18 participants were randomly assigned to intervention and control groups, respectively. There was no significant difference between age (P ¼ 0.320), year of medical school (P ¼ 0.322), handedness (P ¼ 1.000), or gender (P ¼ 0.360) of the groups.
The
overall
mean
performance
score
was
higher
for
intervention
(mean SD ¼ 41.9 22.5) than control (mean SD ¼ 24.7 19.6; P ¼ 0.016). There was no significant difference between scores for each intraoperative segment between the intervention and control group (P > 0.050). Conclusions: This study demonstrates that TS is effective for providing cognitive training in laparoscopic cholecystectomies to medical students. It is likely that this effect will be seen across modules and other platforms that use cognitive task analysis alongside high-fidelity animation. Further work is necessary to extend this to other surgical procedures for evaluating its longitudinal effectiveness. ª 2018 Elsevier Inc. All rights reserved.
Previously presented at the Society of Academic & Research Surgery (SARS), Nottingham, on January 11th, 2018. * Corresponding author. Division of Surgery, Department of Surgery & Cancer, Imperial College London, Academic Surgical Unit, 10th Floor QEQM, St Mary’s Hospital, South Wharf Road, London, W2 1NY, UK. Tel.: þ44 (0) 203 312 6666; fax: þ44 (0) 203 312 6309. E-mail address: [email protected] (S. Erridge). 1 Each of these authors contributed equally and therefore should be considered co-first author. 0022-4804/$ e see front matter ª 2018 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jss.2018.09.042
218
j o u r n a l o f s u r g i c a l r e s e a r c h f e b r u a r y 2 0 1 9 ( 2 3 4 ) 2 1 7 e2 2 3
Introduction Surgical training is aimed at equipping surgeons with both the technical and nontechnical skills required to perform an operation.1 Spurred by an increasing patient load and longer operating times, traditional methods of training in the operating room have become unstructured and costly.2 They also expose patients to potential errors as trainees progress along the learning curve.3 Furthermore, disruptive technology demands surgeons acquire novel skills rapidly to keep abreast with advances in surgical techniques and procedures.4 Hence, it is necessary to explore other viable options such as simulation training platforms for surgeons to learn and hone their skills. Such simulation platforms need to be evaluated before their introduction and use in surgical practice. Cognitive task analysis (CTA) is the process of creating a comprehensive breakdown of a procedure and is a technique shown to improve trainees’ procedural knowledge and technical skills uptake for central venous catheterization,5 percutaneous tracheostomy placement,6 and knee arthroscopy.7 Cognitive training before virtual reality (VR) endoscopic training has also demonstrated improved operative ability.8 Recent studies have shown simulation training to transfer to improved performance during real cases, with significant reductions in the time taken, fewer errors, and decreased patient discomfort.3,9-11 Touch Surgery (TS) is an interactive mobile CTA-based simulation and rehearsal tool with a touch screen interface for the self-teaching and assessing of various operative procedures based on expert-derived cognitive task analyses. Through the touch screen interface and multiple-choice questions, trainees are encouraged to learn and test their cognitive knowledge of procedural steps and surgical decisions. Trainees can track their achievements, and learners are encouraged to improve their procedural knowledge and achieve higher scores with fewer errors. The TS application is available for free on the Apple App Store (Apple Inc, California) and the Google Play store for Android (Google, California). The aim of this study was to assess the transferability of knowledge of cognitive task simulation and rehearsal app, TS, on the learner’s ability to perform laparoscopic cholecystectomy, and hence evaluate the usefulness of the application for incorporation into surgical training.
Methods TS mobile platform The TS platform (Kinosis Limited, London, UK) is an interactive application available on smartphones and tablets. It uses CTA framework alongside immersive multimedia animations to teach operative steps across a broad range of procedures and specialties. The primary target audience is surgical trainees with modules for different procedures authored by
experienced clinicians, but is accessible for clinicians at all stages of training.
Study design A randomized controlled trial was conducted in accordance with CONSORT guidelines.12 The TS mobile platform was compared against a control educational intervention for teaching the procedural steps of a laparoscopic cholecystectomy. The study was performed under the approval of a local medical education ethics panel.
Setting, participants, and allocation The trial was conducted under controlled conditions in a “dry” laboratory equipped with laparoscopic simulators. Participants were medical students currently studying medicine at Imperial College London. Each participant was provided with a certificate for his or her participation. To motivate performance, they were informed that the participant who performed the highest would be provided with an additional certificate detailing their performance. The inclusion criteria were assessed via declarations made within a consent form and were as follows: 1. Participants had completed a faculty-taught course on gastrointestinal anatomy. 2. Participants had no prior experience with TS. 3. The participants had no prior personal experience with laparoscopic surgery or simulation. Each group received the same preparation before intervention. First, each group was provided with a 10-minute introduction to laparoscopic equipment. This included teaching on how to handle instruments and the function of fundamental laparoscopic equipment including trocars, grasping forceps, Marylands, scissors, clip applicators, and diathermy. This was followed by psychomotor training according to tasks from the Fundamentals of Laparoscopic Surgery (FLS).13 The participants were required to complete the peg transfer and precision cutting tasks in accordance with FLS performance guidelines before instigating the educational intervention. This allowed participants in each group to achieve the same baseline proficiency in basic skills to enable them to complete the assessment. In addition, each subject was given a 15-min educational tutorial on laparoscopic cholecystectomies to place the simulation and previous anatomy knowledge in clinical context using a combination of text, images, and video. This included, for example, making all participants aware of the borders of Calot’s triangle and what the critical view of safety is.14 The participants who met the inclusion criteria were randomly assigned to group 1 (TS) or group 2 (control), using an online random number generator (Google, California). Allocation concealment was maintained by randomizing participants to each intervention after the study preparation. Concealment was broken at time of intervention. Setup of the assessment phase of the study was carried out before
chidambaram et al touch surgery skills transfer
219
this and kept uniform throughout to minimize any potential bias.
teaching content was consistent with the intervention. Time studying the material was limited to 15 min per student.
Intervention
Primary outcomes
Group 1 received all prior preparation alongside the intervention. This consisted of undertaking the laparoscopic cholecystectomy phase 3 module (module version: 110) on the TS application on a mobile tablet device (Fig. 1). This would consist of first completing the “learn phase” of the module. This directs users through dissection and ligation of both the cystic duct and artery, as well as dissection of the gall bladder from the liver bed using “drag and drop” tools. The participants then subsequently would have to repeat the “test phase” of the module. This asks participants to complete multiple-choice questions about the next intraoperative steps until they achieved a single perfect score (100%). Time using TS was limited to 15 min per participant.
The assessment of all participants was conducted using a cadaveric porcine liver and gall bladder, which was rotated at 180 to achieve retraction as previously described.15 A laparoscopic camera was set up before the start of any individual assessment to enable adequate visualization of the liver and gall bladder for the students. The camera was set up in an identical fashion for each assessment and kept in place by a bracket as to prevent confounding from an assistant. Assessment of each group was conducted immediately after they had received either the intervention or control. The procedure was recorded through the laparoscopic camera for blinded independent assessment. Cognitive performance was rated using a previously validated performance scale described by Eubanks et al. (Supplemental Material 2).16 This performance scale was used in preference over other validated scores due to its focus on cognitive performance rather than technical skills.17 This was edited to increase applicability to the study methodology. The scoring focused on the intraoperative steps and as such error points were not counted to prevent confounding from unintentional error in both cohorts who were both previously naı¨ve to laparoscopic skills. The steps were grouped according to initial exposure, initial dissection, cystic duct dissection,
Control Group 2 received all similar prior preparation to group 1. The control intervention however consisted of an itemized breakdown of the intraoperative steps and the order in which they are performed. This information was provided in a table format (Supplemental Material 1) and was compared against the TS module by two authors (S.C. and S.E.) to ensure
Fig. 1 e Snapshots of the Touch Surgery mobile platform showing both (A) learn phase and (B) test phase. (Color version of figure is available online.)
220
j o u r n a l o f s u r g i c a l r e s e a r c h f e b r u a r y 2 0 1 9 ( 2 3 4 ) 2 1 7 e2 2 3
cystic artery dissection, and gall bladder fossa dissection. The maximum score available for performing the intraoperative steps was 80. An independent assessor, who had conducted over 150 laparoscopic cholecystectomies as the primary operator, rated the recordings. The assessor was recruited from a separate institution and carried out all assessment at a remote location. All personal identifying information was removed from the videos, and the videos were assessed in a randomized order to ensure blinding to group allocation.
Statistical analysis The performance scale has previously been evaluated on the assumption of normal distribution.18 This was confirmed using a ShapiroeWilk test on the assessment scores. Analysis of cognitive performance was subsequently carried out using a t-test of the group means, expressed with standard deviations. Comparison of demographic data was carried out using a Fisher exact test. Analysis was performed using SPSS (v.24.0.0.0, IBM, Armonk, NY). Significance was determined at P < 0.050.
Results Forty participants were recruited to take part in the study. Twenty-two and 18 participants were randomly assigned to group 1 and group 2, respectively. One student was excluded from analysis in each group as a result of failure of video recording equipment (Fig. 2). Participant demographic data
are summarized in Table. There was no significant difference between groups in age (P ¼ 0.320), the year of medical school (P ¼ 0.322), handedness (P ¼ 1.000), or gender (P ¼ 0.360). Analysis of cognitive performance by individual intraoperative segments is demonstrated in Figure 3. The overall mean performance score was greater for group 1, 41.9 (22.5), than group 2, 24.7 (19.6; P ¼ 0.016). There was no significant difference found between either groups throughout all intraoperative segments. This included initial exposure (group 1 ¼ 3.4 3.1; group 2 ¼ 2.1 2.1; P ¼ 0.211), initial dissection (group 1 ¼ 9.3 6.0; group 2 ¼ 5.6 5.8; P ¼ 0.063), cystic duct dissection (group 1 ¼ 10.8 7.9; group 2 ¼ 6.5 7.2; P ¼ 0.091), cystic artery dissection (group 1 ¼ 5.5 10.1; group 2 ¼ 1.4 5.8; P ¼ 0.130), and gall bladder fossa dissection (group 1 ¼ 12.7 4.2; group 2 ¼ 9.1 6.9; P ¼ 0.070).
Discussion This study was performed to determine the usefulness of TS in transfer of knowledge to enable a naı¨ve student to perform the intraoperative steps of a laparoscopic cholecystectomy. The results of this study demonstrate that TS significantly contributes to improvements in novices’ cognitive performance during a laparoscopic cholecystectomy. TS is an application that aims to equip trainees with the necessary knowledge and thus improve the acquisition of surgical skills when carrying out the procedure. This has clear benefits in surgical education where simulation is being used with increased frequency.19
Fig. 2 e CONSORT flow diagram. (Color version of figure is available online.)
chidambaram et al touch surgery skills transfer
Table e Demographic data of trial participants. Demographics
Group 1 (n ¼ 22)
Group 2 (n ¼ 18)
Pvalue
Age
21.9 1.3
21.4 1.3
0.320
Female
10 (45%)
11 (61%)
0.360
Male
12 (55%)
7 (39%)
Gender
Handedness Left
2 (9%)
2 (11%)
20 (91%)
16 (89%)
Year 1
0
3 (17%)
Year 2
3 (14%)
2 (11%)
Year 3
8 (36%)
4 (22%)
Year 4
6 (27%)
6 (33%)
Year 5
5 (23%)
3 (17%)
Year 6
0
0
Right
1.000
Year group 0.322
The design of the study stipulated that only medical students with no prior training in performing a laparoscopic cholecystectomy were recruited. Previous studies have recruited surgical trainees as participants,20 and this allows for a variety of confounding effects, including previous experience and knowledge of both the procedure and laparoscopic skills. As such, the statistical difference in the mean performance found in this study can be more highly attributed to TS. While mean overall performance was statistically significant, the individual components did not show statistical significance. The improvements shown by students in this study were after minimal one-off exposure to TS. Surgical training is a longitudinal process and with further investigation, TS may be shown to increase cognitive performance for each intraoperative segment and subsequently shorten the learning curve. Another strength of the study is the use of the laparoscopic cholecystectomy as the procedure assessed, which has been validated as the standard procedure of assessment.21 Similarly, validated tools such as the FLS were used to both teach and assess the students.22
221
The results of this study build on previous work that analyzed the effectiveness of TS to increase knowledge acquisition. Previous evaluation of the intramedullary femoral nailing module demonstrated that the app provides a demonstrable increase in knowledge via both in-app testing and an expert-evaluated testing.23 Similar results were also demonstrated for the cardiopulmonary bypass module.24 TS was also shown to improve performance for simulated tendon repair as assessed by expert reviewers.25 The novelty of the present study, however, is that it is the first to validate knowledge transfer from TS via a previously validated scoring tool. A number of qualitative survey-based assessments have shown that users of TS find that it provides content validity and is enjoyable to use.21,24,26 To demonstrate the applicability of novel technologies, it is important to compare these against existing technologies. Previous work by Aggarwal et al. has demonstrated the validity of a VR competency-based simulation training program for laparoscopic cholecystectomy.27 This study has demonstrated the validity of simulation in training for specific procedures. However, this study used a high-fidelity LapSim VR laparoscopic simulator (Surgical Science, Gothenburg, Sweden). One of the greatest challenges facing simulation is the associated costs that come with such high-fidelity simulators, in addition to finding time away from service provision to accommodate training.19,28 TS is available for free on most smartphones and therefore provides a low-cost adjunct to allied simulation modalities that are currently available. The results of this study have highlighted the potential cognitive benefits of training through the TS platform for a naı¨ve cohort. Any simulation platform is likely to have a greater uptake among surgical trainees rather than in novices. It is more likely that TS will provide the greatest benefit for trainees in the early part of the learning curve for a specific operation. Mental rehearsal has been shown to improve the skills of junior surgeons.29 While the use of TS in specific mental rehearsal requires further research and predictions are speculative, this is where the platform would likely provide the most benefit for users. The study has some notable limitations. First, the baseline performance of participants was not collected before
Fig. 3 e Performance scores compared between groups 1 and 2 in each intraoperative domain and across the procedure as a whole. Group 1dintervention, group 2dcontrol; *P < 0.050.
222
j o u r n a l o f s u r g i c a l r e s e a r c h f e b r u a r y 2 0 1 9 ( 2 3 4 ) 2 1 7 e2 2 3
introducing the intervention. Therefore the apparent improvement in performance as a result TS may reflect preexisting confounders. However, the study design was set up to minimize these as much as possible. The inclusion criteria limited study participants to those with no prior experience of either TS or laparoscopic cholecystectomies. Moreover there were no significant differences in the stage of medical training of either group. No power calculation was conducted for this evaluation being that it is the first evaluation of TS via this approach. A post hoc power analysis found that the power for this study reached 73.3%. This work should be used to influence power calculations for future evaluations of this approach. In addition, it should be noted that the study is underpowered to detect any significant difference between the groups in each intraoperative segment. Another limitation is that the study only investigated one-off use and did not measure long-term retention. Surgical education is also a longitudinal process. Hence, while direct comparisons could be made between the control and intervention groups, the improvement in each participant’s performance over repetitive use could not be calculated to determine learning curves. In addition, a wider variety of procedures pertaining to general surgery could have been evaluated instead of just laparoscopic cholecystectomy to see if the effects of the application carry forward to other procedures as well. Forthcoming work will need to involve a larger population that can execute more tasks at higher repetitions to validate the consistency of the application and explore its incorporation into the training curriculum.
Conclusions Surgical training requires more efficient and cost-effective ways to equip trainees with the necessary skills. This study shows that TS is an effective platform for simulating general surgery procedures in teaching and training medical students. Further work is necessary to extend this conclusion to other procedures.
Acknowledgment The authors would like to give their acknowledgments to the following people: Dr Andre Chow and Dr Ali Bahsoun for their help in coordinating funding from Touch Surgery, United Kingdom (grant number: N/A) to fund this work; and Dr Vivek Bindal for his help in assessing the performance of the participants. Authors’ contributions: S.C, S.E, and S.P were involved in development of the study methodology. S.C. and S.E collected all data and performed data analysis. All authors contributed to article preparation and critical revisions.
Disclosure This work was supported by a grant from TS. The authors had full access to the data. The funders played no role in study design; collection, analysis, and interpretation of data; writing
of the report; or the decision to submit the article for publication. S.P. is a co-founder of TS, but has stepped down and has no financial relationship with the company. No authors have received financial support from TS. The authors have read the journal’s guidelines on finances and have no further conflicts of interest to disclose.
Supplementary data Supplementary data to this article can be found online at https://doi.org/10.1016/j.jss.2018.09.042.
references
1. Moorthy K, Munz Y, Adams S, et al. A human factors analysis of technical and team skills among surgical trainees during procedural simulations in a simulated operating theatre. Ann Surg. 2005;242:631e639. 2. Moorthy K, Munz Y, Forrest D, et al. Surgical crisis management skills training and assessment: a simulation [corrected]-based approach to enhancing operating room performance. Ann Surg J. 2006;244:139e147. 3. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg J. 2002;236:454e458. 4. Aggarwal R, Darzi A. Technical-skills training in the 21st century. N Engl J Med. 2006;355:2695e2696. 5. Velmahos GC, Toutouzas KG, Sillin LF, et al. Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory. Am J Surg. 2004;187:114e119. 6. Sullivan ME, Brown CVR, Peyre SE, et al. The use of cognitive task analysis to improve the learning of percutaneous tracheostomy placement. Am J Surg. 2007;193:96e99. 7. Kohls-Gatzoulis JA, Regehr G, Hutchison C. Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg. 2004;47:277e283. 8. Van Herzeele I, Aggarwal R, Neequaye S, et al. Cognitive training improves clinically relevant outcomes during simulated endovascular procedures. J Vasc Surg. 2008;48:1223e1230.e1. 9. Blum MG, Powers TW, Sundaresan S. Bronchoscopy simulator effectively prepares junior residents to competently perform basic clinical bronchoscopy. Ann Thorac Surg. 2004;78:287e291. 10. Grantcharov TP, Kristiansen VB, Bendix J, et al. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004;91:146e150. 11. Sedlack RE, Kolars JC, Alexander JA. Computer simulation training enhances patient comfort during endoscopy. Clin Gastroenterol Hepatol. 2004;2:348e352. 12. Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. 13. Surgeons S of AG and E. Fundamentals of laparoscopic surgery. Available at: https://www.flsprogram.org/; 2017. 14. Strasberg SM, Brunt LM. Rationale and use of the critical view of safety in laparoscopic cholecystectomy. J Am Coll Surg. 2010;211:132e138. 15. Aggarwal R, Crochet P, Dias A, et al. Development of a virtual reality training curriculum for laparoscopic cholecystectomy. Br J Surg. 2009;96:1086e1093.
chidambaram et al touch surgery skills transfer
16. Eubanks TR, Clements RH, Pohl D, et al. An objective scoring system for laparoscopic cholecystectomy. J Am Coll Surg. 1999;189:566e574. 17. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84:273e278. 18. Hwang H, Lim J, Kinnaird C, et al. Correlating motor performance with surgical error in laparoscopic cholecystectomy. Surg Endosc. 2006;20:651e655. 19. Stefanidis D, Sevdalis N, Paige J, et al. Simulation in surgery: what’s needed next? Ann Surg. 2015;261:846e853. 20. Sutherland LM, Middleton PF, Anthony A, et al. Surgical simulation: a systematic review. Ann Surg. 2006;243:291e300. 21. Kowalewski K-F, Hendrie JD, Schmidt MW, et al. Validation of the mobile serious game application Touch Surgery for cognitive training and assessment of laparoscopic cholecystectomy. Surg Endosc. 2017;31:4058e4066. 22. Marks JM. 1. Fundamentals of laparoscopic surgery (FLS) and of endoscopic surgery (FES). In: Soper NJ, Scott-Conner CEH, eds. The SAGES Manual: Volume 1 Basic Laparoscopy and Endoscopy. New York, NY: Springer New York; 2012:3e13. 23. Sugand K, Mawkin M, Gupte C. Training effect of using Touch Surgery for intramedullary femoral nailing. Injury. 2016;47:448e452.
223
24. Brewer ZE, Ogden WD, Fann JI, et al. Creation and global deployment of a mobile, application-based cognitive simulator for cardiac surgical procedures. Semin Thorac Cardiovasc Surg. 2016;28:1e9. 25. Bunogerane GJ, Taylor K, Lin Y, et al. Using touch surgery to improve surgical education in low- and middle-income settings: a randomized control trial. J Surg Educ. 2018;75:231e237. 26. Sugand K, Mawkin M, Gupte C. Validating Touch SurgeryTM: a cognitive task simulation and rehearsal app for intramedullary femoral nailing. Injury. 2015;46:2212e2216. 27. Aggarwal R, Ward J, Balasundaram I, et al. Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg. 2007;246:771e779. 28. Hosny SG, Johnston MJ, Pucher PH, et al. Barriers to the implementation and uptake of simulationbased training programs in general surgery: a multinational qualitative study. J Surg Res. 2017;220:419e426.e2. 29. Arora S, Aggarwal R, Sirimanna P, et al. Mental practice enhances surgical technical skills: a randomized controlled study. Ann Surg. 2011;253:265e270.