- Email: [email protected]
Effect of spaced training with a box trainer on the acquisition and retention of basic laparoscopic skills
International Journal of Gynecology and Obstetrics 127 (2014) 309–313
Contents lists available at ScienceDirect
International Journal of Gynecology and Obstetrics journal homepage: www.elsevier.com/locate/ijgo
EDUCATION AND TRAINING
Effect of spaced training with a box trainer on the acquisition and retention of basic laparoscopic skills Ali Akdemir ⁎, Burak Zeybek, Ahmet M. Ergenoglu, Ahmet O. Yeniel, Fatih Sendag Department of Obstetrics and Gynecology, Ege University School of Medicine, Izmir, Turkey
a r t i c l e
i n f o
Article history: Received 2 February 2014 Received in revised form 24 June 2014 Accepted 5 August 2014 Keywords: Box trainer Laparoscopic skill LapSim Skill acquisition Skill retention
a b s t r a c t Objective: To investigate whether basic laparoscopic skills acquired via structured spaced training on a box trainer persist after 6 months. Methods: In a prospective study undertaken at the Ege University School of Medicine (Izmir, Turkey) between January 1, 2012, and June 1, 2013, 22 gynecology residents without previous laparoscopy experience were randomly assigned (1:1) to receive training with a box trainer (1 hour per week for 4 weeks) or to a control group. At baseline and at 5 weeks, residents’ performance was assessed via the salpingectomy module of LapSim. The box trainer group was reassessed for skills retention 6 months later. Results: The box trainer group performed significantly better than the control group in time (P = 0.01) and economy of movement (P = 0.001) at the final test. Error scores did not differ significantly. Deterioration between final and retention tests in the box trainer group were recorded in time (P = 0.041), instrument path length (P = 0.013), and instrument angular path (P = 0.075). However, time and economy of movement scores were better at the retention assessment than at baseline (P = 0.008 and P = 0.003, respectively). Conclusion: Structured training with a box trainer improved laparoscopic skills, but deterioration was evident within 6 months. This deterioration should be considered when planning laparoscopic training programs. © 2014 International Federation of Gynecology and Obstetrics. Published by Elsevier Ireland Ltd. All rights reserved.
1. Introduction Laparoscopic surgery requires unique psychomotor skills [1], and complications are frequently encountered during the early phase of learning this technique [2]. Because patient safety, quality control, and financial constraints—which are all supervised by healthcare systems— have an important place in modern surgical practice, operating room performances have been forced to become more efficient and costeffective [2]. As a result, laboratories for laparoscopic skill transfer have been developed, and surgeons have started to find training opportunities on suitable simulators to acquire laparoscopic skills ranging from basic to advanced. Laparoscopic simulators can be classified into box trainers and virtual reality trainers. Although some regard box trainers as unrealistic and inferior to virtual reality simulators in terms of assessing objective measurements, the skills acquired on either type of simulator are transferable [3–5]. Virtual reality systems, such as LapSim (Surgical Science, Gothenburg, Sweden), have shown great potential for training outside the operating room and facilitate an evaluation of trainee performance with their assessment system software. Furthermore, the validity of LapSim as an evaluation tool has been established [6]. ⁎ Corresponding author at: Department of Obstetrics and Gynecology, Ege University School of Medicine, Bornova TR-35100, Izmir, Turkey. Tel.: +90 505 674 89 04; fax: +90 232 343 07 11. E-mail addresses: [email protected], [email protected] (A. Akdemir).
Motor skill learning is defined as the acquisition of motor skills in which the learned movements are executed more quickly and accurately with practice [7]. It has been demonstrated across different experimental models that two forms of practice sessions are required to acquire a new skill: massed training, in which new skills are leant by practicing all day; and spaced training, in which training is undertaken with breaks. Massed training is usually unsuccessful, and spaced training has been found to be more efficient in behavioral studies [8,9]. Because acquiring skills is important for novice laparoscopy surgeons, and because the time at which the natural course of skill loss commences without regular practice is still unclear, the aim of the present study was to investigate whether the retention of skills acquired on a conventional box trainer persists 6 months after the end of a structured spaced training program. 2. Materials and methods A prospective study was undertaken at the Ege University School of Medicine (Izmir, Turkey) between January 1, 2012, and June 1, 2013. The 22 postgraduate year 1 gynecology residents were enrolled. They had no experience of laparoscopy, box trainers, or virtual reality simulators. The study protocol was approved by the ethics committee of the university and informed consent was obtained from all participants. The residents were randomly assigned (1:1) to receive training with a box trainer or to a control group by being given a sealed envelope with instructions. All residents completed a questionnaire regarding
http://dx.doi.org/10.1016/j.ijgo.2014.07.015 0020-7292/© 2014 International Federation of Gynecology and Obstetrics. Published by Elsevier Ireland Ltd. All rights reserved.
310
A. Akdemir et al. / International Journal of Gynecology and Obstetrics 127 (2014) 309–313
Fig. 1. Demonstration of a simulated salpingectomy with LapSim.
demographics, handedness, and any previous experience with laparoscopic surgery, the box trainer, the LapSim simulator, or video games. During the first week, all residents received lectures about basic laparoscopy, including basic laparoscopic techniques, suturing techniques, laparoscopic salpingectomy, electrosurgical principles, and the purpose of the present study. Each resident then performed a salpingectomy on an ectopic pregnancy module of the LapSim for a baseline test. LapSim was used for assessment because this instrument is feasible and effective for the evaluation of skills [3,10,11]. For the subsequent 4 weeks, the box trainer group had 1-hour practice sessions on a box trainer every week on the basis of a structured training curriculum. Individuals in this group could not use the box trainer or LapSim outside of these training sessions. The control group did not receive any training. Participants in both groups were not allowed to take part in laparoscopic procedures. One week after the training period, participants in both groups performed a salpingectomy on the LapSim as a final test. Six months after the final test, the box trainer group was asked to perform a salpingectomy on the LapSim in a retention test. Again, these residents were not allowed to use the box trainer or the LapSim, or to take part in laparoscopic procedures between the final test and the retention test. To avoid hindering the residency program, the
control group was not subject to a retention test; after the final test, they could receive training and take part in laparoscopic procedures. At the baseline, final, and retention tests, each participant performed four salpingectomy sessions on the LapSim. Because it has been shown that there is a familiarization curve for use of the LapSim [11], the first three sessions were used for practice; the fourth session was used for assessment. Evaluation of performance was based on three variables registered by the computer system: time; economy of movement scores, including the left and right instrument path lengths, and the angular path; and error scores, including blood loss and ovarian thermal damage. The ectopic pregnancy module of the LapSim provides a realistic image of the procedure (Fig. 1), with different difficulty levels that can be altered from level 1 (easy) to level 3 (difficult). For the tests in the present study, difficulty was set to level 1. Once the salpingectomy has been fully completed, a specimen is placed in a virtual bag, and any residual bleeding is controlled. When the individual is satisfied that the task is completed, the simulation ends. The software evaluates the result of each variable according to the minimum and maximum predetermined ranges; thus, if the results are higher than the maximum limit, or lower than the minimum limit, then the individual is deemed to have failed. The results are subsequently recorded by the computer and can be downloaded into a spreadsheet format (Fig. 2). The box trainer used in the present study (Fig. 3) has been described previously [5]. Training took the form of seven different tasks: cutting out a drawn circle; moving pegs; simulating an ovarian cyst enucleation; grasping and throwing beans into a box; peeling an orange; suturing; and introducing a catheter into a tube (Fig. 4) [5]. The data were analyzed via SPSS version 15.0 (SPSS Inc, Chicago, IL, USA). The normality assumption for the continuous responses was not met when checked via the Shapiro–Wilk test; therefore, the nonparametric Mann–Whitney test was used to compare the two groups. Fisher exact tests and Student t tests were used to assess the differences in categorical and noncategorical variables, respectively. The Wilcoxon signed rank test for related data was used to assess differences within groups. P b 0.05 was considered statistically significant. Post-hoc power analysis was performed via Pass 2000 (NCSS, East Kaysville, UT, USA). The group sample size of 11 achieved a 93%, 94%, and 97% power to detect a difference between the two groups for time score, instrument path length, and instrument angular path
Fig. 2. Summary metrics of the LapSim.
A. Akdemir et al. / International Journal of Gynecology and Obstetrics 127 (2014) 309–313
311
Fig. 3. The box trainer. Reproduced from Akdemir et al. [5], by permission of the International Federation of Gynecology and Obstetrics.
scores, respectively, with a significance level (α) of 0.05 via two-sided Mann–Whitney test. 3. Results There were no notable differences in baseline characteristics between the two groups (Table 1). In addition, no significant differences in performance scores for the baseline LapSim test were identified (Table 2). With regard to the final test performance, the box trainer group had significantly lower scores for time (P = 0.01) and economy of movement (P = 0.001) than did the control group (Table 3). However, there were no statistically significant differences between the two groups in terms of error scores (Table 3). When the baseline, final, and retention test performances of the box trainer group were compared, there were significant differences between most tests in terms of performance times and economy of movement scores (Table 4). There was no significant difference in error scores between the sessions (Table 4). 4. Discussion The present study showed that box trainers improve laparoscopic skills as assessed on the LapSim. The greatest improvements were recorded in economy of movement and time, which have been shown
to have the most validity in assessments of laparoscopic technical skills [12]. However, the error scores, which are more likely to be associated with procedural knowledge, did not change after use of a box trainer. This finding might be attributed to the fact that the participants were all novice surgeons and so had little procedural knowledge. Compared with the final test performance in the box trainer group, evaluation of the retention test revealed a slight deterioration in instrument angular path and a significant worsening of scores for time and instrument path length. Nevertheless, the economy of movement and time remained significantly better at the retention test than at baseline. Although the error scores at the retention test were found to be worse than not only those of the final test performance, but also those of the baseline test performance, all error scores were lower than the fail cutoff value, so the participants passed. To interpret the results more accurately, it is helpful to consider both the physiology of motor skill learning and the adage that “practice makes perfect.” Three stages of motor skill learning have been described according to observations during practice sessions [13]. The first stage is the rapid improvement stage, in which rapid skill acquisition begins in the initial practice session [14]. The second stage, consolidation, occurs after the initial practice session and involves transformation of the initial fragile experiences into more stable forms [15,16]. At this stage, periods of rest of more than 4 hours without additional practice, or a night of sleep, have been found to further improve the acquisition of skills,
Fig. 4. The box trainer exercises. Task 1, cutting out a drawn circle of a sponge; task 2, moving pegs; task 3, simulating an ovarian cyst enucleation; task 4, grasping and throwing beans into a box; task 5, peeling a mandarin orange; task 6, suturing; and task 7, introducing a catheter into a tube. Reproduced from Akdemir et al. [5], by permission of the International Federation of Gynecology and Obstetrics.
312
A. Akdemir et al. / International Journal of Gynecology and Obstetrics 127 (2014) 309–313
Table 1 Characteristics of the study groups.a Characteristic
Box trainer group (n = 11)
Control group (n = 11)
Age, y Sex Female Male Dominant hand Right Left Previous laparoscopic experience Previous LapSim experience Previous box trainer experience Video game experience None Occasionally Formerly All the time
28.2 ± 1.7
27.8 ± 2.0
6 5
6 5
10 1 0 0 0
10 1 0 0 0
5 2 1 3
4 2 3 2
a
Values are given as mean ± SD or number.
suggesting “self-rehearsal” during the rest periods [17]. In the third stage, gains are more gradual; there is a plateau in performance that facilitates the fairly permanent retention of learned behavior [13]. These stages lead to a successful processing of two independent but parallel streams [18]. The so-called spatial processing stream encodes the visuospatial coordinates of the novel movement and represents the accuracy, whereas the motor processing stream encodes the motor program and represents the speed. It has been previously noted that accuracy retention is more difficult [18], which is consistent with the present study: although speed decreased between the final and
retention tests, substantial deterioration was observed for instrument path length, which reflected accuracy. Studies investigating the effects of spaced training on laparoscopic skills are limited. Most skill acquisition and retention studies have evaluated the effects of massed training on skill acquisition and retention. Three months after a 1-day course with a virtual reality simulator, Torkington et al. [19] noted that 25% of basic laparoscopic skills had deteriorated. Anastakis et al. [20] did not find a sustained effect of one training session on skill retention after 2 years in a core procedure in the laboratory environment. These results are not surprising given the physiology of learning because it has been clearly shown that, rather than the amount of practice, the distribution of practice over several days is the most important factor affecting learning skills and retention [8,9,16]. In addition, a delay allows for consolidation, possibly making spaced training more effective for skill acquisition [21]. As such, the box trainer group in the present study received 1-hour weekly training sessions over a period of 4 weeks. In a study investigating the effects of spaced training on laparoscopic skills, Gallagher et al. [22] found that a group who received all virtual reality training on 1 day exhibited significantly better performance from the first day and throughout the study than did a control group who received no training. However, the rate of their performance improvement was no better than that of the control group. In contrast, the group who received the same amount of training but on 3 separate days showed a much steeper rate of performance improvement. In a study by Stefanidis et al. [23], participants were trained to proficiency on a previously validated suturing model and then randomly assigned to a control group (no additional training) or an ongoing training group (received additional training at 1 and 3 months). At 6 months, the ongoing training group exhibited better skill retention and showed
Table 2 Baseline test performance.a,b Test variable Time, s d Error score Ovarian diathermy damage, s e Blood loss, mL f Economy of movement Instrument path length, m g Instrument angular path, degrees h a b c d e f g h
Box trainer group (n = 11) 321 (218–597) 0.28 (0.00–0.73) 5.36 (1.56–26.14) 7.35 (5.53–17.57) 1264.34 (845.78–3047.03)
Control group (n = 11)
P valuec
325 (229–383)
0.818
0.55 (0.18–1.14) 14.17 (3.82–67.11)
0.308 0.139
8.52 (6.38–14.52) 1508.93 (911.09–2251.06)
0.412 0.094
Values are given as median (range) unless indicated otherwise. A better performance is represented by a lower score. Mann–Whitney test. 0–360 s considered a pass. 0–10 s considered a pass. 0–500 mL considered a pass. 0–8 m considered a pass. 0–1800 degrees considered a pass.
Table 3 Final test performance.a,b Test variable Time, s d Error score Ovarian diathermy damage, s e Blood loss, mL f Economy of movement Instrument path length, m g Instrument angular path, degrees h a b c d e f g h
Box trainer group (n = 11) 196 (143–283) 0.11 (0.00–1.49) 7.45 (1.08–19.70) 5.03 (2.67–8.40) 693.63 (404.91–1059.03)
Values are given as median (range) unless indicated otherwise. A better performance is represented by a lower score. Mann–Whitney test. 0–360 s considered a pass. 0–10 s considered a pass. 0–500 mL considered a pass. 0–8 m considered a pass. 0–1800 degrees considered a pass.
Control group (n = 11)
P valuec
280 (191–356)
0.01
0.13 (0.00–2.22) 23.78 (6.46–97.84)
0.261 0.317
10.71 (6.51–19.36) 1838.81 (1199.73–3732.62)
0.001 0.001
A. Akdemir et al. / International Journal of Gynecology and Obstetrics 127 (2014) 309–313
313
Table 4 Performance of the box trainer group in the baseline, final, and retention tests.a,b Test variable
Baseline Time, s d Error score Ovarian diathermy damage, s e Blood loss, mL f Economy of movement Instrument path length, m g Instrument angular path, degrees h a b c d e f g h
P valuec
Assessment Final
Retention
Baseline vs final
Baseline vs retention
Final vs retention
321 (218–597)
196 (143–283)
251 (170–465)
0.003
0.008
0.041
0.28 (0.00–0.73) 5.36 (1.56–26.14)
0.11 (0.00–1.49) 7.45 (1.08–19.70)
0.41 (0.00–2.67) 7.99 (1.19–17.28)
0.169 0.248
0.139 0.374
0.120 0.722
0.003 0.003
0.003 0.003
0.013 0.075
7.35 (5.53–17.57) 1264.34 (845.78–3047.03)
5.03 (2.67–8.40) 693.63 (404.91–1059.03)
5.42 (3.17–9.17) 751.79 (600.32–1177.12)
Values are given as median (range), unless indicated otherwise. A better performance is represented by a lower score. Wilcoxon signed rank test. 0–360 s considered a pass. 0–10 s considered a pass. 0–500 mL considered a pass. 0–8 m considered a pass. 0–1800 degrees considered a pass.
a trend toward achieving the proficiency level more than the control group. In another study that investigated retention of laparoscopic skills after spaced training on a box trainer (1 hour per week for 7 weeks) [10], all basic laparoscopic skills except tissue-handling skills were sustained for 1 year. Bonrath et al. [24] revealed that laparoscopic skills gained from a 5-day box trainer curriculum were retained for at least 6 weeks, and skill deterioration was likely to occur 11 weeks after the initial training. In that study, the short duration of skill retention was attributed to the choice of medical students as novice surgeons. The present study has some limitations. First, the number of participants in the study was small; a larger sample size is needed to validate the results. Second, personal differences in skill acquisition abilities might have affected the results and caused a selection bias. Stefanidis et al. [25] suggested that psychomotor testing before surgical training predicts the rapidity of skill acquisition. However, to overcome this potential selection bias, the present participants were randomized into two different groups. Finally, retention skills were not assessed before 6 months. Because there is a familiarization curve with LapSim [11], every assessment before 6 months would have biased the results of the present retention test. In conclusion, 1 hour of structured use of a box trainer each week for a period of 1 month is useful for basic laparoscopic skill acquisition and retention. These skills start to decline within a period of 6 months, which might be particularly relevant to training programs in which residents do not have a chance to use their acquired skills regularly in the operating theatre or to receive continuous education via laboratory training. Conflict of interest The authors have no conflicts of interest. References [1] Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A. Laparoscopic virtual reality and box trainers: is one superior to the other? Surg Endosc 2004;18(3):485–94. [2] Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg 1999;177(1):28–32. [3] Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 2009;338:b1802.
[4] Mohammadi Y, Lerner MA, Sethi AS, Sundaram CP. Comparison of laparoscopy training using the box trainer versus the virtual trainer. JSLS 2010;14(2):205–12. [5] Akdemir A, Sendağ F, Oztekin MK. Laparoscopic virtual reality simulator and box trainer in gynecology. Int J Gynecol Obstet 2014;125(2):181–5. [6] Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sørensen JL, Dalsgaard T, et al. Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 2006;20(9):1460–6. [7] Willingham DB. A neuropsychological theory of motor skill learning. Psychol Rev 1998;105(3):558–84. [8] Lee TD, Genovese ED. Distribution of practice in motor skill acquisition: different effects for discrete and continuous tasks. Res Q Exerc Sport 1989;60(1):59–65. [9] Lee TD, Genovese ED. Some reminiscences on distribution of practice effects. Res Q Exerc Sport 1989;60(3):297–9. [10] Hiemstra E, Kolkman W, van de Put MA, Jansen FW. Retention of basic laparoscopic skills after a structured training program. Gynecol Surg 2009;6(3):229–35. [11] Maagaard M, Sorensen JL, Oestergaard J, Dalsgaard T, Grantcharov TP, Ottesen BS, et al. Retention of laparoscopic procedural skills acquired on a virtual-reality surgical trainer. Surg Endosc 2011;25(3):722–7. [12] Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P. Virtual reality computer simulation. Surg Endosc 2001;15(3):242–4. [13] Korman M, Raz N, Flash T, Karni A. Multiple shifts in the representation of a motor sequence during the acquisition of skilled performance. Proc Natl Acad Sci U S A 2003;100(21):12492–7. [14] Karni A, Sagi D. The time course of learning a visual skill. Nature 1993;365(6443): 250–2. [15] Robertson EM, Pascual-Leone A, Miall RC. Current concepts in procedural consolidation. Nat Rev Neurosci 2004;5(7):576–82. [16] Robertson EM. From creation to consolidation: a novel framework for memory processing. PLoS Biol 2009;7(1):e19. [17] Walker MP, Brakefield T, Seidman J, Morgan A, Hobson JA, Stickgold R. Sleep and the time course of motor skill learning. Learn Mem 2003;10(4):275–84. [18] Hikosaka O, Rand MK, Nakamura K, Miyachi S, Kitaguchi K, Sakai K, et al. Long-term retention of motor skill in macaque monkeys and humans. Exp Brain Res 2002; 147(4):494–504. [19] Torkington J, Smith SG, Rees B, Darzi A. The role of the basic surgical skills course in the acquisition and retention of laparoscopic skill. Surg Endosc 2001;15(10):1071–5. [20] Anastakis DJ, Wanzel KR, Brown MH, McIlroy JH, Hamstra SJ, Ali J, et al. Evaluating the effectiveness of a 2-year curriculum in a surgical skills center. Am J Surg 2003; 185(4):378–85. [21] Savion-Lemieux T, Penhune VB. The effects of practice and delay on motor skill learning and retention. Exp Brain Res 2005;161(4):423–31. [22] Gallagher AG, Jordan-Black JA, O'Sullivan GC. Prospective, randomized assessment of the acquisition, maintenance, and loss of laparoscopic skills. Ann Surg 2012;256(2): 387–93. [23] Stefanidis D, Korndorffer JR, Markley S, Sierra R, Scott DJ. Proficiency maintenance: impact of ongoing simulator training on laparoscopic skill retention. J Am Coll Surg 2006;202(4):599–603. [24] Bonrath EM, Weber BK, Fritz M, Mees ST, Wolters HH, Senninger N, et al. Laparoscopic simulation training: testing for skill acquisition and retention. Surgery 2012;152(1):12–20. [25] Stefanidis D, Korndorffer JR, Black FW, Dunne JB, Sierra R, Touchard CL, et al. Psychomotor testing predicts rate of skill acquisition for proficiency-based laparoscopic skills training. Surgery 2006;140(2):252–62.
Contents lists available at ScienceDirect
International Journal of Gynecology and Obstetrics journal homepage: www.elsevier.com/locate/ijgo
EDUCATION AND TRAINING
Effect of spaced training with a box trainer on the acquisition and retention of basic laparoscopic skills Ali Akdemir ⁎, Burak Zeybek, Ahmet M. Ergenoglu, Ahmet O. Yeniel, Fatih Sendag Department of Obstetrics and Gynecology, Ege University School of Medicine, Izmir, Turkey
a r t i c l e
i n f o
Article history: Received 2 February 2014 Received in revised form 24 June 2014 Accepted 5 August 2014 Keywords: Box trainer Laparoscopic skill LapSim Skill acquisition Skill retention
a b s t r a c t Objective: To investigate whether basic laparoscopic skills acquired via structured spaced training on a box trainer persist after 6 months. Methods: In a prospective study undertaken at the Ege University School of Medicine (Izmir, Turkey) between January 1, 2012, and June 1, 2013, 22 gynecology residents without previous laparoscopy experience were randomly assigned (1:1) to receive training with a box trainer (1 hour per week for 4 weeks) or to a control group. At baseline and at 5 weeks, residents’ performance was assessed via the salpingectomy module of LapSim. The box trainer group was reassessed for skills retention 6 months later. Results: The box trainer group performed significantly better than the control group in time (P = 0.01) and economy of movement (P = 0.001) at the final test. Error scores did not differ significantly. Deterioration between final and retention tests in the box trainer group were recorded in time (P = 0.041), instrument path length (P = 0.013), and instrument angular path (P = 0.075). However, time and economy of movement scores were better at the retention assessment than at baseline (P = 0.008 and P = 0.003, respectively). Conclusion: Structured training with a box trainer improved laparoscopic skills, but deterioration was evident within 6 months. This deterioration should be considered when planning laparoscopic training programs. © 2014 International Federation of Gynecology and Obstetrics. Published by Elsevier Ireland Ltd. All rights reserved.
1. Introduction Laparoscopic surgery requires unique psychomotor skills [1], and complications are frequently encountered during the early phase of learning this technique [2]. Because patient safety, quality control, and financial constraints—which are all supervised by healthcare systems— have an important place in modern surgical practice, operating room performances have been forced to become more efficient and costeffective [2]. As a result, laboratories for laparoscopic skill transfer have been developed, and surgeons have started to find training opportunities on suitable simulators to acquire laparoscopic skills ranging from basic to advanced. Laparoscopic simulators can be classified into box trainers and virtual reality trainers. Although some regard box trainers as unrealistic and inferior to virtual reality simulators in terms of assessing objective measurements, the skills acquired on either type of simulator are transferable [3–5]. Virtual reality systems, such as LapSim (Surgical Science, Gothenburg, Sweden), have shown great potential for training outside the operating room and facilitate an evaluation of trainee performance with their assessment system software. Furthermore, the validity of LapSim as an evaluation tool has been established [6]. ⁎ Corresponding author at: Department of Obstetrics and Gynecology, Ege University School of Medicine, Bornova TR-35100, Izmir, Turkey. Tel.: +90 505 674 89 04; fax: +90 232 343 07 11. E-mail addresses: [email protected], [email protected] (A. Akdemir).
Motor skill learning is defined as the acquisition of motor skills in which the learned movements are executed more quickly and accurately with practice [7]. It has been demonstrated across different experimental models that two forms of practice sessions are required to acquire a new skill: massed training, in which new skills are leant by practicing all day; and spaced training, in which training is undertaken with breaks. Massed training is usually unsuccessful, and spaced training has been found to be more efficient in behavioral studies [8,9]. Because acquiring skills is important for novice laparoscopy surgeons, and because the time at which the natural course of skill loss commences without regular practice is still unclear, the aim of the present study was to investigate whether the retention of skills acquired on a conventional box trainer persists 6 months after the end of a structured spaced training program. 2. Materials and methods A prospective study was undertaken at the Ege University School of Medicine (Izmir, Turkey) between January 1, 2012, and June 1, 2013. The 22 postgraduate year 1 gynecology residents were enrolled. They had no experience of laparoscopy, box trainers, or virtual reality simulators. The study protocol was approved by the ethics committee of the university and informed consent was obtained from all participants. The residents were randomly assigned (1:1) to receive training with a box trainer or to a control group by being given a sealed envelope with instructions. All residents completed a questionnaire regarding
http://dx.doi.org/10.1016/j.ijgo.2014.07.015 0020-7292/© 2014 International Federation of Gynecology and Obstetrics. Published by Elsevier Ireland Ltd. All rights reserved.
310
A. Akdemir et al. / International Journal of Gynecology and Obstetrics 127 (2014) 309–313
Fig. 1. Demonstration of a simulated salpingectomy with LapSim.
demographics, handedness, and any previous experience with laparoscopic surgery, the box trainer, the LapSim simulator, or video games. During the first week, all residents received lectures about basic laparoscopy, including basic laparoscopic techniques, suturing techniques, laparoscopic salpingectomy, electrosurgical principles, and the purpose of the present study. Each resident then performed a salpingectomy on an ectopic pregnancy module of the LapSim for a baseline test. LapSim was used for assessment because this instrument is feasible and effective for the evaluation of skills [3,10,11]. For the subsequent 4 weeks, the box trainer group had 1-hour practice sessions on a box trainer every week on the basis of a structured training curriculum. Individuals in this group could not use the box trainer or LapSim outside of these training sessions. The control group did not receive any training. Participants in both groups were not allowed to take part in laparoscopic procedures. One week after the training period, participants in both groups performed a salpingectomy on the LapSim as a final test. Six months after the final test, the box trainer group was asked to perform a salpingectomy on the LapSim in a retention test. Again, these residents were not allowed to use the box trainer or the LapSim, or to take part in laparoscopic procedures between the final test and the retention test. To avoid hindering the residency program, the
control group was not subject to a retention test; after the final test, they could receive training and take part in laparoscopic procedures. At the baseline, final, and retention tests, each participant performed four salpingectomy sessions on the LapSim. Because it has been shown that there is a familiarization curve for use of the LapSim [11], the first three sessions were used for practice; the fourth session was used for assessment. Evaluation of performance was based on three variables registered by the computer system: time; economy of movement scores, including the left and right instrument path lengths, and the angular path; and error scores, including blood loss and ovarian thermal damage. The ectopic pregnancy module of the LapSim provides a realistic image of the procedure (Fig. 1), with different difficulty levels that can be altered from level 1 (easy) to level 3 (difficult). For the tests in the present study, difficulty was set to level 1. Once the salpingectomy has been fully completed, a specimen is placed in a virtual bag, and any residual bleeding is controlled. When the individual is satisfied that the task is completed, the simulation ends. The software evaluates the result of each variable according to the minimum and maximum predetermined ranges; thus, if the results are higher than the maximum limit, or lower than the minimum limit, then the individual is deemed to have failed. The results are subsequently recorded by the computer and can be downloaded into a spreadsheet format (Fig. 2). The box trainer used in the present study (Fig. 3) has been described previously [5]. Training took the form of seven different tasks: cutting out a drawn circle; moving pegs; simulating an ovarian cyst enucleation; grasping and throwing beans into a box; peeling an orange; suturing; and introducing a catheter into a tube (Fig. 4) [5]. The data were analyzed via SPSS version 15.0 (SPSS Inc, Chicago, IL, USA). The normality assumption for the continuous responses was not met when checked via the Shapiro–Wilk test; therefore, the nonparametric Mann–Whitney test was used to compare the two groups. Fisher exact tests and Student t tests were used to assess the differences in categorical and noncategorical variables, respectively. The Wilcoxon signed rank test for related data was used to assess differences within groups. P b 0.05 was considered statistically significant. Post-hoc power analysis was performed via Pass 2000 (NCSS, East Kaysville, UT, USA). The group sample size of 11 achieved a 93%, 94%, and 97% power to detect a difference between the two groups for time score, instrument path length, and instrument angular path
Fig. 2. Summary metrics of the LapSim.
A. Akdemir et al. / International Journal of Gynecology and Obstetrics 127 (2014) 309–313
311
Fig. 3. The box trainer. Reproduced from Akdemir et al. [5], by permission of the International Federation of Gynecology and Obstetrics.
scores, respectively, with a significance level (α) of 0.05 via two-sided Mann–Whitney test. 3. Results There were no notable differences in baseline characteristics between the two groups (Table 1). In addition, no significant differences in performance scores for the baseline LapSim test were identified (Table 2). With regard to the final test performance, the box trainer group had significantly lower scores for time (P = 0.01) and economy of movement (P = 0.001) than did the control group (Table 3). However, there were no statistically significant differences between the two groups in terms of error scores (Table 3). When the baseline, final, and retention test performances of the box trainer group were compared, there were significant differences between most tests in terms of performance times and economy of movement scores (Table 4). There was no significant difference in error scores between the sessions (Table 4). 4. Discussion The present study showed that box trainers improve laparoscopic skills as assessed on the LapSim. The greatest improvements were recorded in economy of movement and time, which have been shown
to have the most validity in assessments of laparoscopic technical skills [12]. However, the error scores, which are more likely to be associated with procedural knowledge, did not change after use of a box trainer. This finding might be attributed to the fact that the participants were all novice surgeons and so had little procedural knowledge. Compared with the final test performance in the box trainer group, evaluation of the retention test revealed a slight deterioration in instrument angular path and a significant worsening of scores for time and instrument path length. Nevertheless, the economy of movement and time remained significantly better at the retention test than at baseline. Although the error scores at the retention test were found to be worse than not only those of the final test performance, but also those of the baseline test performance, all error scores were lower than the fail cutoff value, so the participants passed. To interpret the results more accurately, it is helpful to consider both the physiology of motor skill learning and the adage that “practice makes perfect.” Three stages of motor skill learning have been described according to observations during practice sessions [13]. The first stage is the rapid improvement stage, in which rapid skill acquisition begins in the initial practice session [14]. The second stage, consolidation, occurs after the initial practice session and involves transformation of the initial fragile experiences into more stable forms [15,16]. At this stage, periods of rest of more than 4 hours without additional practice, or a night of sleep, have been found to further improve the acquisition of skills,
Fig. 4. The box trainer exercises. Task 1, cutting out a drawn circle of a sponge; task 2, moving pegs; task 3, simulating an ovarian cyst enucleation; task 4, grasping and throwing beans into a box; task 5, peeling a mandarin orange; task 6, suturing; and task 7, introducing a catheter into a tube. Reproduced from Akdemir et al. [5], by permission of the International Federation of Gynecology and Obstetrics.
312
A. Akdemir et al. / International Journal of Gynecology and Obstetrics 127 (2014) 309–313
Table 1 Characteristics of the study groups.a Characteristic
Box trainer group (n = 11)
Control group (n = 11)
Age, y Sex Female Male Dominant hand Right Left Previous laparoscopic experience Previous LapSim experience Previous box trainer experience Video game experience None Occasionally Formerly All the time
28.2 ± 1.7
27.8 ± 2.0
6 5
6 5
10 1 0 0 0
10 1 0 0 0
5 2 1 3
4 2 3 2
a
Values are given as mean ± SD or number.
suggesting “self-rehearsal” during the rest periods [17]. In the third stage, gains are more gradual; there is a plateau in performance that facilitates the fairly permanent retention of learned behavior [13]. These stages lead to a successful processing of two independent but parallel streams [18]. The so-called spatial processing stream encodes the visuospatial coordinates of the novel movement and represents the accuracy, whereas the motor processing stream encodes the motor program and represents the speed. It has been previously noted that accuracy retention is more difficult [18], which is consistent with the present study: although speed decreased between the final and
retention tests, substantial deterioration was observed for instrument path length, which reflected accuracy. Studies investigating the effects of spaced training on laparoscopic skills are limited. Most skill acquisition and retention studies have evaluated the effects of massed training on skill acquisition and retention. Three months after a 1-day course with a virtual reality simulator, Torkington et al. [19] noted that 25% of basic laparoscopic skills had deteriorated. Anastakis et al. [20] did not find a sustained effect of one training session on skill retention after 2 years in a core procedure in the laboratory environment. These results are not surprising given the physiology of learning because it has been clearly shown that, rather than the amount of practice, the distribution of practice over several days is the most important factor affecting learning skills and retention [8,9,16]. In addition, a delay allows for consolidation, possibly making spaced training more effective for skill acquisition [21]. As such, the box trainer group in the present study received 1-hour weekly training sessions over a period of 4 weeks. In a study investigating the effects of spaced training on laparoscopic skills, Gallagher et al. [22] found that a group who received all virtual reality training on 1 day exhibited significantly better performance from the first day and throughout the study than did a control group who received no training. However, the rate of their performance improvement was no better than that of the control group. In contrast, the group who received the same amount of training but on 3 separate days showed a much steeper rate of performance improvement. In a study by Stefanidis et al. [23], participants were trained to proficiency on a previously validated suturing model and then randomly assigned to a control group (no additional training) or an ongoing training group (received additional training at 1 and 3 months). At 6 months, the ongoing training group exhibited better skill retention and showed
Table 2 Baseline test performance.a,b Test variable Time, s d Error score Ovarian diathermy damage, s e Blood loss, mL f Economy of movement Instrument path length, m g Instrument angular path, degrees h a b c d e f g h
Box trainer group (n = 11) 321 (218–597) 0.28 (0.00–0.73) 5.36 (1.56–26.14) 7.35 (5.53–17.57) 1264.34 (845.78–3047.03)
Control group (n = 11)
P valuec
325 (229–383)
0.818
0.55 (0.18–1.14) 14.17 (3.82–67.11)
0.308 0.139
8.52 (6.38–14.52) 1508.93 (911.09–2251.06)
0.412 0.094
Values are given as median (range) unless indicated otherwise. A better performance is represented by a lower score. Mann–Whitney test. 0–360 s considered a pass. 0–10 s considered a pass. 0–500 mL considered a pass. 0–8 m considered a pass. 0–1800 degrees considered a pass.
Table 3 Final test performance.a,b Test variable Time, s d Error score Ovarian diathermy damage, s e Blood loss, mL f Economy of movement Instrument path length, m g Instrument angular path, degrees h a b c d e f g h
Box trainer group (n = 11) 196 (143–283) 0.11 (0.00–1.49) 7.45 (1.08–19.70) 5.03 (2.67–8.40) 693.63 (404.91–1059.03)
Values are given as median (range) unless indicated otherwise. A better performance is represented by a lower score. Mann–Whitney test. 0–360 s considered a pass. 0–10 s considered a pass. 0–500 mL considered a pass. 0–8 m considered a pass. 0–1800 degrees considered a pass.
Control group (n = 11)
P valuec
280 (191–356)
0.01
0.13 (0.00–2.22) 23.78 (6.46–97.84)
0.261 0.317
10.71 (6.51–19.36) 1838.81 (1199.73–3732.62)
0.001 0.001
A. Akdemir et al. / International Journal of Gynecology and Obstetrics 127 (2014) 309–313
313
Table 4 Performance of the box trainer group in the baseline, final, and retention tests.a,b Test variable
Baseline Time, s d Error score Ovarian diathermy damage, s e Blood loss, mL f Economy of movement Instrument path length, m g Instrument angular path, degrees h a b c d e f g h
P valuec
Assessment Final
Retention
Baseline vs final
Baseline vs retention
Final vs retention
321 (218–597)
196 (143–283)
251 (170–465)
0.003
0.008
0.041
0.28 (0.00–0.73) 5.36 (1.56–26.14)
0.11 (0.00–1.49) 7.45 (1.08–19.70)
0.41 (0.00–2.67) 7.99 (1.19–17.28)
0.169 0.248
0.139 0.374
0.120 0.722
0.003 0.003
0.003 0.003
0.013 0.075
7.35 (5.53–17.57) 1264.34 (845.78–3047.03)
5.03 (2.67–8.40) 693.63 (404.91–1059.03)
5.42 (3.17–9.17) 751.79 (600.32–1177.12)
Values are given as median (range), unless indicated otherwise. A better performance is represented by a lower score. Wilcoxon signed rank test. 0–360 s considered a pass. 0–10 s considered a pass. 0–500 mL considered a pass. 0–8 m considered a pass. 0–1800 degrees considered a pass.
a trend toward achieving the proficiency level more than the control group. In another study that investigated retention of laparoscopic skills after spaced training on a box trainer (1 hour per week for 7 weeks) [10], all basic laparoscopic skills except tissue-handling skills were sustained for 1 year. Bonrath et al. [24] revealed that laparoscopic skills gained from a 5-day box trainer curriculum were retained for at least 6 weeks, and skill deterioration was likely to occur 11 weeks after the initial training. In that study, the short duration of skill retention was attributed to the choice of medical students as novice surgeons. The present study has some limitations. First, the number of participants in the study was small; a larger sample size is needed to validate the results. Second, personal differences in skill acquisition abilities might have affected the results and caused a selection bias. Stefanidis et al. [25] suggested that psychomotor testing before surgical training predicts the rapidity of skill acquisition. However, to overcome this potential selection bias, the present participants were randomized into two different groups. Finally, retention skills were not assessed before 6 months. Because there is a familiarization curve with LapSim [11], every assessment before 6 months would have biased the results of the present retention test. In conclusion, 1 hour of structured use of a box trainer each week for a period of 1 month is useful for basic laparoscopic skill acquisition and retention. These skills start to decline within a period of 6 months, which might be particularly relevant to training programs in which residents do not have a chance to use their acquired skills regularly in the operating theatre or to receive continuous education via laboratory training. Conflict of interest The authors have no conflicts of interest. References [1] Munz Y, Kumar BD, Moorthy K, Bann S, Darzi A. Laparoscopic virtual reality and box trainers: is one superior to the other? Surg Endosc 2004;18(3):485–94. [2] Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg 1999;177(1):28–32. [3] Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al. Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 2009;338:b1802.
[4] Mohammadi Y, Lerner MA, Sethi AS, Sundaram CP. Comparison of laparoscopy training using the box trainer versus the virtual trainer. JSLS 2010;14(2):205–12. [5] Akdemir A, Sendağ F, Oztekin MK. Laparoscopic virtual reality simulator and box trainer in gynecology. Int J Gynecol Obstet 2014;125(2):181–5. [6] Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sørensen JL, Dalsgaard T, et al. Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 2006;20(9):1460–6. [7] Willingham DB. A neuropsychological theory of motor skill learning. Psychol Rev 1998;105(3):558–84. [8] Lee TD, Genovese ED. Distribution of practice in motor skill acquisition: different effects for discrete and continuous tasks. Res Q Exerc Sport 1989;60(1):59–65. [9] Lee TD, Genovese ED. Some reminiscences on distribution of practice effects. Res Q Exerc Sport 1989;60(3):297–9. [10] Hiemstra E, Kolkman W, van de Put MA, Jansen FW. Retention of basic laparoscopic skills after a structured training program. Gynecol Surg 2009;6(3):229–35. [11] Maagaard M, Sorensen JL, Oestergaard J, Dalsgaard T, Grantcharov TP, Ottesen BS, et al. Retention of laparoscopic procedural skills acquired on a virtual-reality surgical trainer. Surg Endosc 2011;25(3):722–7. [12] Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P. Virtual reality computer simulation. Surg Endosc 2001;15(3):242–4. [13] Korman M, Raz N, Flash T, Karni A. Multiple shifts in the representation of a motor sequence during the acquisition of skilled performance. Proc Natl Acad Sci U S A 2003;100(21):12492–7. [14] Karni A, Sagi D. The time course of learning a visual skill. Nature 1993;365(6443): 250–2. [15] Robertson EM, Pascual-Leone A, Miall RC. Current concepts in procedural consolidation. Nat Rev Neurosci 2004;5(7):576–82. [16] Robertson EM. From creation to consolidation: a novel framework for memory processing. PLoS Biol 2009;7(1):e19. [17] Walker MP, Brakefield T, Seidman J, Morgan A, Hobson JA, Stickgold R. Sleep and the time course of motor skill learning. Learn Mem 2003;10(4):275–84. [18] Hikosaka O, Rand MK, Nakamura K, Miyachi S, Kitaguchi K, Sakai K, et al. Long-term retention of motor skill in macaque monkeys and humans. Exp Brain Res 2002; 147(4):494–504. [19] Torkington J, Smith SG, Rees B, Darzi A. The role of the basic surgical skills course in the acquisition and retention of laparoscopic skill. Surg Endosc 2001;15(10):1071–5. [20] Anastakis DJ, Wanzel KR, Brown MH, McIlroy JH, Hamstra SJ, Ali J, et al. Evaluating the effectiveness of a 2-year curriculum in a surgical skills center. Am J Surg 2003; 185(4):378–85. [21] Savion-Lemieux T, Penhune VB. The effects of practice and delay on motor skill learning and retention. Exp Brain Res 2005;161(4):423–31. [22] Gallagher AG, Jordan-Black JA, O'Sullivan GC. Prospective, randomized assessment of the acquisition, maintenance, and loss of laparoscopic skills. Ann Surg 2012;256(2): 387–93. [23] Stefanidis D, Korndorffer JR, Markley S, Sierra R, Scott DJ. Proficiency maintenance: impact of ongoing simulator training on laparoscopic skill retention. J Am Coll Surg 2006;202(4):599–603. [24] Bonrath EM, Weber BK, Fritz M, Mees ST, Wolters HH, Senninger N, et al. Laparoscopic simulation training: testing for skill acquisition and retention. Surgery 2012;152(1):12–20. [25] Stefanidis D, Korndorffer JR, Black FW, Dunne JB, Sierra R, Touchard CL, et al. Psychomotor testing predicts rate of skill acquisition for proficiency-based laparoscopic skills training. Surgery 2006;140(2):252–62.