- Email: [email protected]
Impact of video game genre on surgical skills development: a feasibility study
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
Available online at www.sciencedirect.com
ScienceDirect journal homepage: www.JournalofSurgicalResearch.com
Impact of video game genre on surgical skills development: a feasibility study Thiago Bozzi de Araujo, BS,a,* Filipe Rodrigues Silveira, BS,a Dante Lucas Santos Souza, BS,a Yuri Thome´ Machado Strey, BS,a Cecilia Dias Flores, MSc, ScD,a,b and Ronaldo Scholze Webster, MD, PhDa,c,d a
School of Medicine, Federal University of Health Sciences of Porto Alegre, Porto Alegre, Brazil Department of Information and Education in Health, Irmandade Santa Casa de Miserico´rida de Porto Alegre, Porto Alegre, Brazil c Department of Surgery, Irmandade Santa Casa de Miserico´rida de Porto Alegre, Porto Alegre, Brazil d Department of Plastic and Reconstructive Microsurgery, Irmandade Santa Casa de Miserico´rida de Porto Alegre, Porto Alegre, Brazil b
article info
abstract
Article history:
Background: The playing of video games (VGs) was previously shown to improve surgical
Received 28 May 2015
skills. This is the first randomized, controlled study to assess the impact of VG genre on the
Received in revised form
development of basic surgical skills.
25 June 2015
Materials and methods: Twenty firsteyear, surgically inexperienced medical students
Accepted 21 July 2015
attended a practical course on surgical knots, suturing, and skin-flap technique. Later, they
Available online 30 July 2015
were randomized into four groups: control and/or nongaming (ContG), first-person-shooter game (ShotG), racing game (RaceG), and surgery game (SurgG). All participants had 3 wk of
Keywords:
Nintendo Wii training. Surgical and VG performances were assessed by two independent,
Video game
blinded surgeons who evaluated basal performance (time 0) and performance after 1 wk
Surgery
(time 1) and 3 wk (time 2) of training.
Basic surgery skills
Results: The training time of RaceG was longer than that of ShotG and SurgG (P ¼ 0.045).
Medical education
Compared to SurgG and RaceG, VG scores for ShotG improved less between times 0 and 1 (P ¼ 0.010) but more between times 1 and 2 (P ¼ 0.004). Improvement in mean surgical performance scores versus time differed in each VG group (P ¼ 0.011). At time 2, surgical performance scores were significantly higher in ShotG (P ¼ 0.002) and SurgG (P ¼ 0.022) than in ContG. The surgical performance scores of RaceG were not significantly different from the score achieved by ContG (P ¼ 0.279). Conclusions: Different VG genres may differentially impact the development of surgical skills by medical students. More complex games seem to improve performance even if played less. Although further studies are needed, surgery-related VGs with sufficient complexity and playability could be a feasible adjuvant to improving surgical skills. ª 2016 Elsevier Inc. All rights reserved.
* Corresponding author. Federal University of Health Sciences of Porto Alegre, School of Medicine, 245 Sarmento Leite St, Porto Alegre, RS 90050-170. Tel.: þ55 51 3303 8832. E-mail address: [email protected] (T.B. de Araujo). 0022-4804/$ e see front matter ª 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jss.2015.07.035
236
1.
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
Introduction
Video games (VGs) have become a permanent part of life in many countries. In 2014, over 150 million Americans played video games, and consumers spent over 22 billion US dollars on the industry (E.S.A 2014 official report on sales, demographic, and usage data). Gaming involves the manipulation of threedimensional objects via a two-dimensional screen and requires, among other skills, manual dexterity. This particular component of the activity has led to intense interest in the relationship between VG playing and surgical skills. There is ample evidence showing that VG playing improves surgeons’ laparoscopic and endoscopic skills [1e12]. Studies that have evaluated the general theoretical basis underlying the cognitive enhancement of demanding motor skills activities resulting from VG usage [3,13] have attributed increases in dexterity, psychomotor abilities, and visual-spatial coordination to fewer errors and less time to perform tasks. Other studies have demonstrated that VG playing correlates with surgical technique in medical school students [8,12,14] and impacts traditional surgery skills [15], such as knot tying, incision-making, dissection, and suturing, which are the first steps in the individual development of surgical techniques. However, whether the particular VG genre plays a role in initial learning development has yet to be determined in a randomized controlled setting. We systematically investigated the impact of VG genre on the learning curve of basic surgery skills by surgically inexperienced medical students.
2.
Materials and methods
2.1.
Background
This study was conducted at the Federal University of Health Sciences of Porto Alegre, Brazil, after approval from the Institutional Ethics Committee. All medical school programs accredited by the Ministry of Education in Brazil are 6 y in length, with students admitted soon after the completion of high school. Therefore, it is highly unlikely that a Brazilian freshmen medical student has any previous surgical experience.
2.2.
Phase 1
An online questionnaire was sent to all first and second-year medical students of our institution through their official academic e-mails. Senior students were not included because they attend the Surgical Techniques Course at this stage of the core curriculum. Demographic data and possible confounders for the study were determined through the questions [5,14,16] (e.g., chopsticks use, musical instrument proficiency, and touch typing). They also assessed past VG experience and specifically queried previous surgical experience, defined as knot tying or suturing knowledge, anatomical dissection, animal research involving surgical techniques, or >4 h of observing surgical procedures. Students were also asked if they would be willing to participate in extensive VG training sessions (at least 3 h per wk [6] for 3 wk duration) and then undergo several performance evaluations.
2.3.
Stratification and sampling
The students were stratified into low and high VG experience based on whether they had played VGs for an average of at least 10 h per wk [5] over the last year. This cutoff was based on the review of Jalink et al. [2], where studies with positive effects on surgical skills have used 3, 7, or 10 h per wk as benchmark for VG experience. The higher amount of hours was chosen to increase sensitivity. After stratification, the participants were randomly divided into four groups: a control nongaming group and three VG groups, each one with a specific game type in which the participant would be trained during the experimental phase of the study. The game genre and titles were chosen based on their applicability to the development of specific skills that are important in the surgical setting (fine motor control, visual attention processing, spatial distribution, reaction time, eyeehand coordination, targeting, non-dominant hand emphasis, and two-dimensional depth perception compensation) [9]. They were also chosen taking into account their acceptance and availability on the market. The Nintendo Wii console (Nintendo Co. Ltd., Tokyo, Japan), and well-known titles within each specific game genre that were playable on it, were chosen over other platforms and games because this system has been used in previous surgical performance studies [6,7,9,11,17] and because its unique controller characteristics (single handheld device, the use of motion-sensing technology via an accelerometer and optical sensor) better simulate the surgical setting [4]. None of the students had previously played the games used in the study. The experimental groups and game titles were as follows: 1. A nongaming, control group (ContG) 2. A surgery game group (SurgG) that played Trauma Center: New Blood (Atlus USA, 2007) 3. A first-person-shooter game group (ShotG) that played Resident Evil 4 (Capcom, 2005) 4. A racing game group (RaceG) that played Need For Speed: Carbon (Electronic Arts, 2006)
2.4.
Phase 2
Before beginning the experiment, all students attended a practical course on surgical knots, suture techniques, and simple skin-flap exercises. A pig’s feet model [18] was used, and training was led by a certified surgeon. Care was taken to ensure that all students were given the same instructions and performed the same amount of training during the course. After the training course (time 0), all students were tested for surgical performance to obtain their basal scores. Students in the VG groups were also tested to obtain basal VG performance data. Students from the VG groups were provided with a brief overview of the Nintendo Wii operational system and control usage. In addition, each group received explanations on game playing and the goals of the specific game. Five minutes of tutoring were granted before the VG performance test.
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
2.5.
Phase 3
A room was equipped with several stations (television þ Wii console) to which students participating in the study had 24 h access. Each student, except those in the control group, was required to play the game title assigned to his or her group for at least 3 h per wk. This amount of time was based on the satisfactory results obtained in previous studies [1,6,12] that used intensive VG training as an intervention. VG training time was self-tracked by the players. The authors were responsible for reminding them how many hours they had played per week to make sure that the minimum required time had been fulfilled. However, students in the VG groups were instructed not to play >7 h per wk, whereas students in the control group were prohibited to play any VG during the course of the study. After the first performance evaluations (time 0), 1 wk of VG training followed. At this point (time 1), all of the students were reevaluated for surgical performance, and the VG group members were reevaluated for VG performance. Two weeks later (time 2), performance was again evaluated. The methodology and timeline of the study are summarized in Figure 1.
2.6.
Surgical performance evaluation
The students were blindly evaluated by two independent surgeons through video footage of each student performing a standardized basic surgical skills exercise using the pigs’ feet model. They had 30 min to perform an advancement skin flap
237
to repair a 1 1 cm defect. The authors designed the flap and the defect in each model previous to the evaluation. The skin flap was 3 cm in length and 1 cm wide and was made using number 10 blades and 4-0 nylon sutures. Care was taken so that the student in each video was unidentifiable; all wore surgical gowns and procedure gloves and were framed from the clavicle down behind a desk. Any hand and/or wrist accessories and nail polish were removed before the evaluation. Editing was performed to remove sound and to mark each video with an ID number. The surgeons used an Objective Structured Assessment of Technical Skills (OSATS), adapted from Chipman and Schmitz [19], to evaluate five criteria: instrument handling, quality of suturing, respect of the tissue, motion and flow, and overall performance. Each criterion was scored from 1 (poor technique, frequent unnecessary movements, and force) to 5 (excellent technique, carefulness, clear, and precise movements). A single surgical performance score was obtained from the sum of the criteria scores.
2.7.
VG performance evaluation
Each VG evaluation consisted of two predefined stages and/or races to be completed by the students. The same stages/races were used throughout the study for each group. The scores of each game were registered at the end of play and included bonuses, specific scores, times, lap positions, and other data. An overall VG score was created by summing the positive scores, multiplying them by the bonus scores, and dividing the
Fig. 1 e Methodology and timeline overview. (Color version of figure is available online.)
238
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
result by the negative scores (e.g., time left in a particular stage). All overall scores were transformed into a percentile for standardization purposes.
2.8.
Statistical analysis
Statistical analyses were performed using SPSS version 21 (IBM Corp, Armonk, NY). The format of the data is specified in each table/figure legend. Demographic and VG experience data were analyzed in all groups using chi-square (c2) and analysis of variance (ANOVA) tests. Mixed ANOVAs with the Greenhouse-Geisser correction were used to analyze VG training time, VG scores, and surgical performance scores between VG groups over the 3 wk of the study. The results were subjected to post-hoc analysis using the Wilcoxon signed-rank test and a Bonferroni correction. Consistency among raters was determined using a weighted kappa interrater reliability analysis. A P value <0.05 was considered to indicate statistical significance.
3.
Results
3.1.
Questionnaire data
Although 178 medical students received the questionnaire, only 38 (21.34%) were willing to participate in the study. Of those, 15 (39.5%) were excluded because of previous surgical experience; the remaining 23 (60.5%) students were surgically inexperienced. However, to obtain a homogenous sample (first-year medical students only), the decision was made to exclude the three (7.89%) second-year medical students. Stratification of the 20 participants into high (12 subjects, 60%) and low (8 subjects, 40%) VG playing experience rendered a 3/2 ratio in each of the study groups. Most of the students were right-handed, undecided as to whether to pursue a surgical career, and had started playing VGs at a young age (before 8 y). There were no statistically significant differences between the groups. Table 1 shows their demographic and VG experience data.
Table 1 e Demographics and VG experience. Variable
N Age Sex (male) Dominant hand (right) Chopstick use Musical instrument proficiency Plan on pursuing a surgical career Yes No Undecided Touch typing VG play start age (y) 0e5 6e8 9e11 12e15 15 N.A. Current VG player VG experience* Low High Most familiar game genre Adventure/RPG FPS Racing Sports None Favorite game genre Adventure/RPG FPS Racing Sports N.A.
General
Group
P value
ContG
SurgG
ShotG
RaceG
20 (100) 19.65 1.90 8 (40) 18 (90) 12 (60) 6 (30)
5 (25) 20.60 1.82 3 (60) 4 (80) 2 (40) 2 (40)
5 (25) 18.8 1.09 1 (20) 5 (100) 3 (60) 1 (20)
5 (25) 19 2.24 1 (20) 5 (100) 4 (80) 1 (20)
5 (25) 20.20 2.17 3 (60) 4 (80) 3 (60) 2 (40)
7 (35) 1 (5) 12 (60) 15 (75)
1 (20) 0 (0) 4 (80) 2 (40)
1 1 3 4
(20) (20) (60) (80)
2 0 3 4
(40) (0) (60) (80)
3 0 2 5
(60) (0) (40) (100)
0.514
5 (25) 11 (55) 2 (10) 0 (0) 1 (5) 1 (5) 11 (55)
0 (0) 3 (60) 1 (20) 0 (0) 1 (20)
2 2 0 0 0
(40) (40) (0) (0) (0)
2 2 1 0 0
(40) (40) (20) (0) (0)
1 4 0 0 0
(20) (80) (0) (0) (0)
0.512
2 (40)
3 (60)
3 (60)
3 (60)
0.895
12 (60) 8 (40)
3 (60) 2 (40)
3 (60) 2 (40)
3 (60) 2 (40)
3 (60) 2 (40)
d
8 (40) 3 (15) 3 (15) 2 (10) 4 (20)
3 (60) 1 (20) 0 (0) 0 (0) 1 (20)
1 1 1 0 2
(20) (20) (20) (0) (40)
2 0 2 0 1
(40) (0) (40) (0) (20)
2 (40) 1 (20) 0 (0) 2 (40) 0 (0)
0.322
10 (50) 4 (20) 2 (10) 3 (15) 1 (5)
3 (60) 2 (40) 0 (0) 0 (0)
1 1 1 1
(20) (20) (20) (20)
4 0 1 0
(80) (0) (20) (0)
2 1 0 2
0.394
N.A. ¼ Non applicable. Data presented as count (%) or mean standard deviation. * Stratification criteria.
(40) (20) (0) (40)
0.377 0.343 0.528 0.644 0.813
0.167
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
239
Table 2 e Mean VG training time by genre (in min). Week
Overall
Group y
SurgG 1 2 3
ShotG
RaceGy
258.47 39.148* 240 25.209 255.4 20.574 280 57.337 220.67 27.958* 201 21.909 225 32.596 236 19.812 204 18.822 206 13.874 187 8.367 219 18.166
Data are presented as the mean standard deviation. * Significant reduction (P ¼ 0.021) between weeks 1 and 2. y Significant difference between groups (P ¼ 0.045).
3.2.
VG training time
Students significantly (P ¼ 0.021) reduced their training time from week 0 to week 1 of training, but not between weeks 1 and 2 (P ¼ 0.221). The lack of a significant interaction effect between VG group and training week [F(4, 24) ¼ 0.981, P ¼ 0.436] suggested that the reduction in VG training minutes over time did not differ between groups. The significant effect of VG group (P ¼ 0.045) in the analysis, however, indicated differences in the average number of minutes played between VG groups: [F(2, 12) ¼ 4.071, r ¼ 1.19]. Mean training time of RaceG was longer compared to SurgG (P ¼ 0.045). The mean training time of ShotG was not significantly different from that of either RaceG (P ¼ 0.133) or SurgG (P ¼ 0.806). Mean VG training times (in min) by genre are shown in Table 2. The results of the evaluation of the VG training time of each VG group over the 3 wk of the study are provided in Figure 2.
Fig. 2 e Mixed ANOVA analysis for training time (min) by VG group per wk. The mean training time was significantly reduced from week 0 to week 1 (P [ 0.021), but not from week 1 to week 2 (P [ 0.221). * RaceG trained, on average, significantly longer than SurgG (P [ 0.45). (Color version of figure is available online.)
Fig. 3 e Mixed ANOVA analysis for overall VG scores (%) by VG group per performance evaluation. Mean scores were significantly increased from evaluation 0 to 1 (P < 0.001) and from evaluation 1 to 2 (P [ 0.010). (Color version of figure is available online.)
3.3.
VG performance
The overall VG scores (as a percentage) of each VG group over the three performance evaluations are shown in Figure 3. Students significantly increased their mean scores from evaluation 0 to evaluation 1 (1.646 versus 5.830, respectively; P < 0.001) and from evaluation 1 to evaluation 2 (5.830 versus 12.792, respectively; P ¼ 0.01). There was a significant interaction effect between VG group and mean performance evaluation scores [F(2.498, 14.986) ¼ 4.711; P ¼ 0.020], suggesting that there were differences between VG groups regarding the way in which the mean scores increased over the course of the evaluations. As seen in Figure 3, there is a less steep increase in ShotG scores than in either SurgG and to RaceG scores from evaluation 0 to 1, whereas the opposite occurred from evaluation 1 to 2, when the increment in mean scores was faster for ShotG than for the other two groups. The rate of increase in RaceG scores was relatively constant across the evaluations. There was no significant effect of VG group (P ¼ 0.983), indicating similar overall VG scores between VG groups [F(2, 12) ¼ 0.017], that is, no genre tended to result in higher or lower scores compared to the others.
3.4.
Surgical performance
3.4.1.
Surgeons’ evaluation
The inter-rater reliability for each surgical performance criterion is shown in Table 3. All criteria yielded a nonsignificant (P > 0.05) kappa coefficient, indicative of concordance between surgeons.
240
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
Table 3 e Surgeons’ inter-reliability for surgical performance evaluations. Performance criterion
Performance evaluation 0 *
K Instrument handling Quality of suturing Respect for tissue Motion and flow Overall performance *
0.154 0.204 0.125 0.092 0.188
P (95% CI) 0.190 (0.527 to 0.170 (0.129 to 0.202 (0.522 to 0.186 (0.457 to 0.063 (0.312 to
0.219) 0.536) 0.272) 0.272) 0.064)
Performance evaluation 1 *
Performance evaluation 2
K
P (95% CI)
k*
P (95% CI)
0.231 0.143 0.421 0.280 0.403
0.198 (0.157 to 0.618) 0.178 (0.207 to 0.492) 0.184 (0.060e0.783) 0.179 (0.071e0.632) 0.163 (0.084e0.722)
0.561 0.612 0.784 0.746 0.740
0.136 (0.295e0.828) 0.154 (0.311e0.914) 0.122 (0.545e1.000) 0.113 (0.524e0.967) 0.123 (0.500e0.981)
Weighted kappa coefficient.
3.4.2.
Control versus gaming groups
The mean surgical performance scores of the nongaming (control) and all gaming groups differed throughout the study. The interaction between evaluation time and surgical performance scores was statistically significant [F(2,36) ¼ 4.344, P ¼ 0.020, partial h2 ¼ 0.194]. Thus, the way in which the scores changed over the 3 wk of the study differed between the groups. At times 0 and 1, the surgical performance scores of the groups did not significantly differ [t(18) ¼ 0.440, P ¼ 0.665, and t(18) ¼ 1.684, P ¼ 0.109, respectively], whereas at time 2, the difference between the VG groups and the control was significant [t(18) ¼ 3.485; P ¼ 0.003, (95% confidence interval ¼ 4.434e1.099)]. The mean scores were greater in the gaming groups (mean ¼ 18.467, SE ¼ 1.55) than in the control group (mean ¼ 15.7, SE ¼ 1.48).
3.4.3.
VG genre analysis
The mean surgical performance scores for each evaluation are listed in Table 4. The scores for each VG group over the 3 wk of the study are shown in Figure 4. The statistically significant interaction between VG group and surgical performance score evaluation time [F(6,32) ¼ 3.980, P ¼ 0.011, partial h2 ¼ 0.427] indicated a difference between VG groups in the way in which the mean scores changed over the study. At times 0 and 1, there were no statistically significant differences in the surgical performance scores of the VG groups [F(3, 16) ¼ 1.127, P ¼ 0.368, partial h2 ¼ 0.174 and F(3, 16) ¼ 1.765, P ¼ 0.194, partial h2 ¼ 0.249, respectively]. A statistically significant difference in the surgical performance scores of the VG groups was, however, determined at time 2 [F(3, 16) ¼ 7.634, P ¼ 0.002, partial h2 ¼ 0.589]. The scores of ShotG (mean ¼ 3.9, SE ¼ 0.86, P ¼ 0.002) and SurgG
(mean ¼ 2.8, SE ¼ 0.86, P ¼ 0.022) were higher than the score achieved by the control group. In contrast, at this evaluation time, there was no statistically significantly difference between the RaceG and ContG scores (mean ¼ 1.6, SE ¼ 0.86, P ¼ 0.279).
3.4.3.1. ContG. There was a statistically significant effect of time on the performance evaluation score of ContG [F(2, 8) ¼ 33.857, P < 0.005, partial h2 ¼ 0.894]. The scores of this group increased between evaluations 0 and 1 (mean ¼ 4.6, SE ¼ 0.48, P ¼ 0.002) but not between evaluations 1 and 2 (mean ¼ 3.1, SE ¼ 0.99, P ¼ 0.106). The difference between evaluations 0 and 2 was also significant (mean ¼ 7.7, SE ¼ 1.2, P ¼ 0.009). 3.4.3.2. ShotG. Time had a statistically significant effect on the performance evaluation score of ShotG [F(2, 8) ¼ 251.782, P < 0.005, partial h2 ¼ 0.984]. ShotG scores increased significantly between evaluations 0 and 1 (mean ¼ 5.2, SE ¼ 0.46, P ¼ 0.001), 1 and 2 (mean ¼ 5.9, SE ¼ 0.58, P ¼ 0.002), and 0 and 2 (mean ¼ 11.1, SE ¼ 0.43, P < 0.005). 3.4.3.3. RaceG. Time also had a statistically significant effect on the performance evaluation score of RaceG [F(2, 8) ¼ 202.486, P < 0.005, partial h2 ¼ 0.981]. RaceG scores significantly increased between evaluations 0 and 1 (mean ¼ 6.6, SE ¼ 0.29, P < 0.005), 1 and 2 (mean ¼ 3.2, SE ¼ 0.46, P ¼ 0.007), and 0 and 2 (mean ¼ 9.8, SE ¼ 0.66, P < 0.005).
3.4.3.4. SurgG. The effect of time on the performance evaluation score of SurgG was also significant [F(2, 8) ¼ 114.115, P < 0.005, partial h2 ¼ 0.966]. The performance evaluation scores of this group increased between evaluations 0 and 1 (mean ¼ 4.1, SE ¼ 0.53, P ¼ 0.005), 1 and 2 (mean ¼ 5.5,
Table 4 e Mean surgical performance scores by performance evaluation. Performance evaluation 0 1 2
Group ContG
SurgG
ShotG
RaceG
8.0 1.27 12.6 1.29 15.7 1.48*,**
8.9 1.67 13.0 1.22 18.5 1.46*
8.5 1.12 13.7 1.25 19.6 0.74**
7.5 0.93 14.1 0.65 17.3 1.56
P value
Data are presented as the mean standard deviation. Values in italics denotes statistical significance in the overall group comparison. *, ** Statistical significant difference between groups.
0.368 0.194 0.002
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
241
randomized study to assess the effects of VGs, and specifically VG genre, on the development of surgical skills. The finding that different types of VGs may have different impacts on skill development has not been reported in other studies.
4.1.
Fig. 4 e Mixed ANOVA analysis of the mean surgical performance scores obtained from the three evaluation times. The interaction between the VG groups and the evaluation time of the surgical performance score was significant (P < 0.001). ⌘ Mean ShotG and SurgG scores were higher than the ContG score (P [ 0.002 and P [ 0.022, respectively). x Mean scores increased significantly from evaluation 0 to evaluation 1 (P [ 0.002), but not from evaluation 1 to evaluation 2 (P [ 0.106). * Means scores increased significantly from evaluation 0 to evaluation 1 and from evaluation 1 to evaluation 2 (P < 0.05). (Color version of figure is available online.) SE ¼ 0.59, P ¼ 0.002), and 0 and 2 (mean ¼ 9.6, SE ¼ 0.76, P < 0.001).
4.
Gaming versus nongaming
Our results are in agreement with data on the impact of VGs on surgical skills, in which playing virtual games was shown to improve surgical skills [1e12]. When analyzed together, the VG groups in our study scored significantly higher in their surgical evaluations than the control group after 3 wk of systematic Wii training. No other study has corroborated a significant positive correlation between VGs and open surgery modalities (i.e., neither laparoscopic nor endoscopic). In the only study that investigated traditional surgical skills [15], VG playing was not a significant predictor of surgical skills scores. However, the participants in that study were veterinary students with previous surgical experience (they had acquired soft-tissue surgical skills during training with various suturing pads). This may explain the nonsignificant improvement in that series, because the impact of VGs is likely to be minor when higher levels of expertise in the task are already present. In addition, the games used in that study were all mini-games (skeet shooting, archery, and target shooting) within an Olympic Games simulation franchise for the Nintendo Wii. Those intuitive, single-goal games differed from the VGs used in our study. Harper et al. [14] showed that VG users acquire the skills for robotic suturing techniques more slowly than those for endoscopic techniques. The same study found significant correlations with a prior history of musical instrument playing and with the practicing of sports for >4 y. Self-reported VG experience (number of hours) was used as the main variable, and the sample size was only 10 students, which might have affected the statistical power.
Discussion 4.2.
Different VG genres were found to differentially impact the basic development of surgical skills of inexperienced medical students. More complex games, with more elements to master, such as first-person-shooter or surgery-simulation games, improved performance even if played less than the more “playable” racing game. Shooter games take longer to master than surgical games and are more appealing to seasoned than to casual gamers. Surgical trainees are busy students, with limited time for VG playing [12]. Surgery games seem to be appropriate adjuvants to surgical skills development. More than one study [2,3,20] have suggested the use of VGs as part of a program to train the basic laparoscopic skills of surgical novices. In fact, VGs have already been used to transfer cognitive-enhanced characteristics to everyday tasks. In a previous study [21], cadets who trained for 10 h on the videogame Space Fortress were rated higher by instructors than a control group. The impact of VGs on surgical skills is an area of current interest, given the reduction of surgical training hours in many medical programs and the need to identify new teaching paradigms in surgical training that will make a difference in clinical practice. Ours is the first controlled and
Playability
In previous studies, a trend toward improvement of laparoscopic skills was found in participants who trained with VGs for 180 min per wk over 1 wk [6]. In our study, the VG training time was roughly 230 min per wk over 3 wk and thus comparable with studies in which subjects trained for 150e300 min per wk over 3 wk [1] or 75e270 min per wk for 2 wk [12]. Our data on VG training time suggest that different VG genres have different levels of acceptance among medical students. A further, important observation was that the students played significantly less after the first week regardless of the genre. This was evidenced by the nonsignificant interaction effect between group and time, which showed that the decrement in training time was essentially the same between groups. However, overall, the racing game was played more than the other games, perhaps because racing games are inherently more “playable” because of the smaller number of elements that must be mastered, the more easily distinguishable objectives, and easier game play. Although not statistically significant, the surgery game was played more often from week 2 to week 3. When asked about
242
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
why they played more during the last week, students stated that, after understanding what each surgical instrument was meant for, how each one worked in terms of the controller, and the purpose of each level, the game became “more fun.” This suggests that surgery games, although more complex than racing games, maintain their playability. Shooter games, such as the one used in the study (which includes a more complex storyline, several projectile types, and the exploration of different scenarios other than just simply “aiming and shooting”), comprise a late mastery and complex game genre, more suitable to experienced VG players.
4.3.
Complexity
Schlickum et al. [1] pointed out the positive transfer effects that only cognitively loaded games (such as a computer virtual chess game) have on laparoscopic simulation, which suggested that virtual cognitive training alone is sufficient to impact surgical performance. More cognitively demanding (i.e., more complex) games may result in even further improvements. In the present study, students improved their VG scores each week. This improvement in scores was different for each VG genre. ShotG showed a poor improvement from evaluation 0 to evaluation 1, especially compared to SurgG and RaceG. In the following period (between evaluations 1 and 2), the pattern of improvement was different: the improvement of SurgG reached a plateau, whereas in ShotG there was a >three-fold increment in scores. The surgery game could be mastered during the 3 wk of this study. RaceG scores improved at a constant rate across evaluations. A longer VG training time and a more detailed and focused analysis would be necessary to suitably explore the complexity patterns of each VG genre. However, some insights can already be gained based on the surgical learning perspective, as discussed in the section.
4.4.
the task and a decreased awareness of irrelevant aspects. Regarding the ShotG and SurgG training times, despite similar patterns of impact on surgical performance, the surgery-simulation games seemed to be generally more appealing to the medical students. Surgery-themed games such as the one used in the study are more easily contextualized and thus perhaps more enjoyable to novice surgeons [20]. Our data suggest that surgery games such as Trauma Center could be more effective for enhancing the development of surgical skills than first-person-shooter games such as Resident Evil. Perhaps the latter would be beneficial to seasoned game players. Nevertheless, our study suggests that VGs can provide harmless and cost-effective improvements in surgical education.
4.5.
Limitations
Although our results point out the potential benefits of exposing medical students to VGs to improve their surgical skills, our study had several limitations. We had a low participation rate (21.34%), which we attribute to having sent the questionnaires close to the end of the semester when the students would have their finals. The small sample limited the amount of data available from the questionnaire as covariates in a multivariate analysis. Raters were not trained in the assessment tool before the study. This may explain why the reliability between the two raters improved with each successive evaluation. Also, the poor inter-rater reliability is a significant limiting factor. Because of these limitations, we refer to our work as a feasibility study. Nonetheless, the solid methodology used in this study and the data it provided are sufficient to support further investigations. Additional studies should focus on different aspects of the relationship between VG playing and surgery skills, because it may offer efficient and cost-effective surgical learning options.
Genre analysis
Supporting our hypothesis, different VG genres were found to differentially impact the potential for surgical performance improvement. The basal surgical scores were the same among the four groups, and after 1 wk of training, there were still no significant differences between them. Thus, a single week of Wii training is not enough to significantly impact the development of surgical skills. In Boyle et al. [6], only a trend toward better performance was found after the same intervention. In the present study, after 3 wk of VG training, surgical performance scores were different between the VG and control groups. Students playing the surgery and firstperson-shooter games scored higher than the control group in the surgical performance evaluation. RaceG scored higher on average than ContG but the difference was not significant, although among the three genres, the racing game was the one that was played most often. Therefore, the “playability” of a game seems to have less of an effect than its complexity with respect to surgical development. This can be attributed to the greater “attentional weighting” [3], a mechanism of perceptual learning, required for more complex games. Thus, complex games more often and more intensively require that players have a heightened awareness of important aspects of
5.
Conclusions
Different VG genres differentially impacted the development of surgical skills by inexperienced medical students. More complex games, such as surgery and first-person-shooter games, improved performance although they were played less. Racing games, although more “playable,” did not significantly improve surgical performance scores over those of a nongaming group. Although further studies are needed, our results suggest that surgery-type VGs, with their complexity and playability, offer an appropriate adjuvant to improve the surgical performance of medical students.
Acknowledgment Funding for this study was obtained from Pro´-Ensino na Sau´de (Project number 39), of the Coordenac¸a˜o de Aperfeic¸oamento de Pessoal de Nı´vel Superior (CAPES). Academic support and funding came from the Fundac¸a˜o de Amparo a` Pesquisa do Estado do Rio Grande do Sul (FAPERGS).
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
Authors’ contributions: T.B.A. conceptualized and designed the study and drafted the initial article. T.B.A., C.D.F., and R.S.W. reviewed and revised the article. T.B.A., D.L.S.S., Y.T.M.S., and R.S.W. carried out the study and collected the data. T.B.A., F.R.S., D.L.S.S., Y.T.M.S., C.D.F., and R.S.W. approved the final version of the article as submitted.
Disclosure The authors have no proprietary or commercial interest in any product mentioned or concept discussed in this article.
references
[1] Schlickum MK, Hedman L, Enochsson L, Kjellin A, FellanderTsai L. Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: a prospective randomized study. World J Surg 2009;33:2360. [2] Jalink MB, Goris J, Heineman E, Pierie JP, ten Cate Hoedemaker HO. The effects of video games on laparoscopic simulator skills. Am J Surg 2014;208:151. [3] Lynch J, Aughwane P, Hammond TM. Video games and surgical ability: a literature review. J Surg Educ 2010;67:184. [4] Adams BJ, Margaron F, Kaplan BJ. Comparing video games and laparoscopic simulators in the development of laparoscopic skills in surgical residents. J Surg Educ 2012;69:714. [5] van Dongen KW, Verleisdonk EJ, Schijven MP, Broeders IA. Will the Playstation generation become better endoscopic surgeons? Surg Endosc 2011;25:2275. [6] Boyle E, Kennedy AM, Traynor O, Hill AD. Training surgical skills using nonsurgical tasksecan Nintendo Wii improve surgical performance? J Surg Educ 2011;68:148. [7] Badurdeen S, Abdul-Samad O, Story G, Wilson C, Down S, et al. Nintendo Wii video-gaming ability predicts laparoscopic skill. Surg Endosc 2010;24:1824. [8] Shane MD, Pettitt BJ, Morgenthal CB, Smith CD. Should surgical novices trade their retractors for joysticks? Videogame experience decreases the time needed to acquire surgical skills. Surg Endosc 2008;22:1294.
243
[9] Rosser JC Jr, Lynch PJ, Cuddihy L, Gentile DA, Klonsky J, et al. The impact of video games on training surgeons in the 21st century. Arch Surg 2007;142:181. discusssion 186. [10] Jalink MB, Heineman E, Pierie JP, Ten Cate Hoedemaker HO. The effect of a preoperative warm-up with a custom-made Nintendo video game on the performance of laparoscopic surgeons. Surg Endosc 2015;29:2284. [11] Ju R, Chang PL, Buckley AP, Wang KC. Comparison of Nintendo Wii and PlayStation2 for enhancing laparoscopic skills. JSLS 2012;16:612. [12] Rosenberg BH, Landsittel D, Averch TD. Can video games be used to predict or improve laparoscopic skills? J Endourol 2005;19:372. [13] Basak C, Voss MW, Erickson KI, Boot WR, Kramer AF. Regional differences in brain volume predict the acquisition of skill in a complex real-time strategy videogame. Brain Cogn 2011;76:407. [14] Harper JD, Kaiser S, Ebrahimi K, et al. Prior video game exposure does not enhance robotic surgical performance. J Endourol 2007;21:1207. [15] Millard HA, Millard RP, Constable PD, Freeman LJ. Relationships among video gaming proficiency and spatial orientation, laparoscopic, and traditional surgical skills of third-year veterinary students. J Am Vet Med Assoc 2014;244:357. [16] Boyd T, Jung I, Van Sickle K, Schwesinger W, Michalek J, Bingener J. Music experience influences laparoscopic skills performance. JSLS 2008;12:292. [17] Willis RE, Gomez PP, Ivatury SJ, Mitra HS, Van Sickle KR. Virtual reality simulators: valuable surgical skills trainers or video games? J Surg Educ 2014;71:426. [18] Denadai R, Saad-Hossne R, Raposo-Amaral CE. Simulationbased Rhomboid flap skills training during medical education: comparing low- and high-fidelity bench models. J Craniofac Surg 2014;25:2134. [19] Chipman JG, Schmitz CC. Using objective structured assessment of technical skills to evaluate a basic skills simulation curriculum for first-year surgical residents. J Am Coll Surg 2009;209:364. [20] Meterissian S, Liberman M, McLeod P. Games as teaching tools in a surgical residency. Med Teach 2007;29:e258. [21] Gopher D. The skill of attention control: acquisition and execution of attention strategies. In: Meyer D, Kornblum S, editors. Attention and Performance XIV (Silver Jubilee Volume): Synergies in Experimental Psychology, Artificial Intelligence, and Cognitive Neuroscience. Cambridge, MA: The MIT Press; 1993. p. 299.
Available online at www.sciencedirect.com
ScienceDirect journal homepage: www.JournalofSurgicalResearch.com
Impact of video game genre on surgical skills development: a feasibility study Thiago Bozzi de Araujo, BS,a,* Filipe Rodrigues Silveira, BS,a Dante Lucas Santos Souza, BS,a Yuri Thome´ Machado Strey, BS,a Cecilia Dias Flores, MSc, ScD,a,b and Ronaldo Scholze Webster, MD, PhDa,c,d a
School of Medicine, Federal University of Health Sciences of Porto Alegre, Porto Alegre, Brazil Department of Information and Education in Health, Irmandade Santa Casa de Miserico´rida de Porto Alegre, Porto Alegre, Brazil c Department of Surgery, Irmandade Santa Casa de Miserico´rida de Porto Alegre, Porto Alegre, Brazil d Department of Plastic and Reconstructive Microsurgery, Irmandade Santa Casa de Miserico´rida de Porto Alegre, Porto Alegre, Brazil b
article info
abstract
Article history:
Background: The playing of video games (VGs) was previously shown to improve surgical
Received 28 May 2015
skills. This is the first randomized, controlled study to assess the impact of VG genre on the
Received in revised form
development of basic surgical skills.
25 June 2015
Materials and methods: Twenty firsteyear, surgically inexperienced medical students
Accepted 21 July 2015
attended a practical course on surgical knots, suturing, and skin-flap technique. Later, they
Available online 30 July 2015
were randomized into four groups: control and/or nongaming (ContG), first-person-shooter game (ShotG), racing game (RaceG), and surgery game (SurgG). All participants had 3 wk of
Keywords:
Nintendo Wii training. Surgical and VG performances were assessed by two independent,
Video game
blinded surgeons who evaluated basal performance (time 0) and performance after 1 wk
Surgery
(time 1) and 3 wk (time 2) of training.
Basic surgery skills
Results: The training time of RaceG was longer than that of ShotG and SurgG (P ¼ 0.045).
Medical education
Compared to SurgG and RaceG, VG scores for ShotG improved less between times 0 and 1 (P ¼ 0.010) but more between times 1 and 2 (P ¼ 0.004). Improvement in mean surgical performance scores versus time differed in each VG group (P ¼ 0.011). At time 2, surgical performance scores were significantly higher in ShotG (P ¼ 0.002) and SurgG (P ¼ 0.022) than in ContG. The surgical performance scores of RaceG were not significantly different from the score achieved by ContG (P ¼ 0.279). Conclusions: Different VG genres may differentially impact the development of surgical skills by medical students. More complex games seem to improve performance even if played less. Although further studies are needed, surgery-related VGs with sufficient complexity and playability could be a feasible adjuvant to improving surgical skills. ª 2016 Elsevier Inc. All rights reserved.
* Corresponding author. Federal University of Health Sciences of Porto Alegre, School of Medicine, 245 Sarmento Leite St, Porto Alegre, RS 90050-170. Tel.: þ55 51 3303 8832. E-mail address: [email protected] (T.B. de Araujo). 0022-4804/$ e see front matter ª 2016 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jss.2015.07.035
236
1.
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
Introduction
Video games (VGs) have become a permanent part of life in many countries. In 2014, over 150 million Americans played video games, and consumers spent over 22 billion US dollars on the industry (E.S.A 2014 official report on sales, demographic, and usage data). Gaming involves the manipulation of threedimensional objects via a two-dimensional screen and requires, among other skills, manual dexterity. This particular component of the activity has led to intense interest in the relationship between VG playing and surgical skills. There is ample evidence showing that VG playing improves surgeons’ laparoscopic and endoscopic skills [1e12]. Studies that have evaluated the general theoretical basis underlying the cognitive enhancement of demanding motor skills activities resulting from VG usage [3,13] have attributed increases in dexterity, psychomotor abilities, and visual-spatial coordination to fewer errors and less time to perform tasks. Other studies have demonstrated that VG playing correlates with surgical technique in medical school students [8,12,14] and impacts traditional surgery skills [15], such as knot tying, incision-making, dissection, and suturing, which are the first steps in the individual development of surgical techniques. However, whether the particular VG genre plays a role in initial learning development has yet to be determined in a randomized controlled setting. We systematically investigated the impact of VG genre on the learning curve of basic surgery skills by surgically inexperienced medical students.
2.
Materials and methods
2.1.
Background
This study was conducted at the Federal University of Health Sciences of Porto Alegre, Brazil, after approval from the Institutional Ethics Committee. All medical school programs accredited by the Ministry of Education in Brazil are 6 y in length, with students admitted soon after the completion of high school. Therefore, it is highly unlikely that a Brazilian freshmen medical student has any previous surgical experience.
2.2.
Phase 1
An online questionnaire was sent to all first and second-year medical students of our institution through their official academic e-mails. Senior students were not included because they attend the Surgical Techniques Course at this stage of the core curriculum. Demographic data and possible confounders for the study were determined through the questions [5,14,16] (e.g., chopsticks use, musical instrument proficiency, and touch typing). They also assessed past VG experience and specifically queried previous surgical experience, defined as knot tying or suturing knowledge, anatomical dissection, animal research involving surgical techniques, or >4 h of observing surgical procedures. Students were also asked if they would be willing to participate in extensive VG training sessions (at least 3 h per wk [6] for 3 wk duration) and then undergo several performance evaluations.
2.3.
Stratification and sampling
The students were stratified into low and high VG experience based on whether they had played VGs for an average of at least 10 h per wk [5] over the last year. This cutoff was based on the review of Jalink et al. [2], where studies with positive effects on surgical skills have used 3, 7, or 10 h per wk as benchmark for VG experience. The higher amount of hours was chosen to increase sensitivity. After stratification, the participants were randomly divided into four groups: a control nongaming group and three VG groups, each one with a specific game type in which the participant would be trained during the experimental phase of the study. The game genre and titles were chosen based on their applicability to the development of specific skills that are important in the surgical setting (fine motor control, visual attention processing, spatial distribution, reaction time, eyeehand coordination, targeting, non-dominant hand emphasis, and two-dimensional depth perception compensation) [9]. They were also chosen taking into account their acceptance and availability on the market. The Nintendo Wii console (Nintendo Co. Ltd., Tokyo, Japan), and well-known titles within each specific game genre that were playable on it, were chosen over other platforms and games because this system has been used in previous surgical performance studies [6,7,9,11,17] and because its unique controller characteristics (single handheld device, the use of motion-sensing technology via an accelerometer and optical sensor) better simulate the surgical setting [4]. None of the students had previously played the games used in the study. The experimental groups and game titles were as follows: 1. A nongaming, control group (ContG) 2. A surgery game group (SurgG) that played Trauma Center: New Blood (Atlus USA, 2007) 3. A first-person-shooter game group (ShotG) that played Resident Evil 4 (Capcom, 2005) 4. A racing game group (RaceG) that played Need For Speed: Carbon (Electronic Arts, 2006)
2.4.
Phase 2
Before beginning the experiment, all students attended a practical course on surgical knots, suture techniques, and simple skin-flap exercises. A pig’s feet model [18] was used, and training was led by a certified surgeon. Care was taken to ensure that all students were given the same instructions and performed the same amount of training during the course. After the training course (time 0), all students were tested for surgical performance to obtain their basal scores. Students in the VG groups were also tested to obtain basal VG performance data. Students from the VG groups were provided with a brief overview of the Nintendo Wii operational system and control usage. In addition, each group received explanations on game playing and the goals of the specific game. Five minutes of tutoring were granted before the VG performance test.
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
2.5.
Phase 3
A room was equipped with several stations (television þ Wii console) to which students participating in the study had 24 h access. Each student, except those in the control group, was required to play the game title assigned to his or her group for at least 3 h per wk. This amount of time was based on the satisfactory results obtained in previous studies [1,6,12] that used intensive VG training as an intervention. VG training time was self-tracked by the players. The authors were responsible for reminding them how many hours they had played per week to make sure that the minimum required time had been fulfilled. However, students in the VG groups were instructed not to play >7 h per wk, whereas students in the control group were prohibited to play any VG during the course of the study. After the first performance evaluations (time 0), 1 wk of VG training followed. At this point (time 1), all of the students were reevaluated for surgical performance, and the VG group members were reevaluated for VG performance. Two weeks later (time 2), performance was again evaluated. The methodology and timeline of the study are summarized in Figure 1.
2.6.
Surgical performance evaluation
The students were blindly evaluated by two independent surgeons through video footage of each student performing a standardized basic surgical skills exercise using the pigs’ feet model. They had 30 min to perform an advancement skin flap
237
to repair a 1 1 cm defect. The authors designed the flap and the defect in each model previous to the evaluation. The skin flap was 3 cm in length and 1 cm wide and was made using number 10 blades and 4-0 nylon sutures. Care was taken so that the student in each video was unidentifiable; all wore surgical gowns and procedure gloves and were framed from the clavicle down behind a desk. Any hand and/or wrist accessories and nail polish were removed before the evaluation. Editing was performed to remove sound and to mark each video with an ID number. The surgeons used an Objective Structured Assessment of Technical Skills (OSATS), adapted from Chipman and Schmitz [19], to evaluate five criteria: instrument handling, quality of suturing, respect of the tissue, motion and flow, and overall performance. Each criterion was scored from 1 (poor technique, frequent unnecessary movements, and force) to 5 (excellent technique, carefulness, clear, and precise movements). A single surgical performance score was obtained from the sum of the criteria scores.
2.7.
VG performance evaluation
Each VG evaluation consisted of two predefined stages and/or races to be completed by the students. The same stages/races were used throughout the study for each group. The scores of each game were registered at the end of play and included bonuses, specific scores, times, lap positions, and other data. An overall VG score was created by summing the positive scores, multiplying them by the bonus scores, and dividing the
Fig. 1 e Methodology and timeline overview. (Color version of figure is available online.)
238
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
result by the negative scores (e.g., time left in a particular stage). All overall scores were transformed into a percentile for standardization purposes.
2.8.
Statistical analysis
Statistical analyses were performed using SPSS version 21 (IBM Corp, Armonk, NY). The format of the data is specified in each table/figure legend. Demographic and VG experience data were analyzed in all groups using chi-square (c2) and analysis of variance (ANOVA) tests. Mixed ANOVAs with the Greenhouse-Geisser correction were used to analyze VG training time, VG scores, and surgical performance scores between VG groups over the 3 wk of the study. The results were subjected to post-hoc analysis using the Wilcoxon signed-rank test and a Bonferroni correction. Consistency among raters was determined using a weighted kappa interrater reliability analysis. A P value <0.05 was considered to indicate statistical significance.
3.
Results
3.1.
Questionnaire data
Although 178 medical students received the questionnaire, only 38 (21.34%) were willing to participate in the study. Of those, 15 (39.5%) were excluded because of previous surgical experience; the remaining 23 (60.5%) students were surgically inexperienced. However, to obtain a homogenous sample (first-year medical students only), the decision was made to exclude the three (7.89%) second-year medical students. Stratification of the 20 participants into high (12 subjects, 60%) and low (8 subjects, 40%) VG playing experience rendered a 3/2 ratio in each of the study groups. Most of the students were right-handed, undecided as to whether to pursue a surgical career, and had started playing VGs at a young age (before 8 y). There were no statistically significant differences between the groups. Table 1 shows their demographic and VG experience data.
Table 1 e Demographics and VG experience. Variable
N Age Sex (male) Dominant hand (right) Chopstick use Musical instrument proficiency Plan on pursuing a surgical career Yes No Undecided Touch typing VG play start age (y) 0e5 6e8 9e11 12e15 15 N.A. Current VG player VG experience* Low High Most familiar game genre Adventure/RPG FPS Racing Sports None Favorite game genre Adventure/RPG FPS Racing Sports N.A.
General
Group
P value
ContG
SurgG
ShotG
RaceG
20 (100) 19.65 1.90 8 (40) 18 (90) 12 (60) 6 (30)
5 (25) 20.60 1.82 3 (60) 4 (80) 2 (40) 2 (40)
5 (25) 18.8 1.09 1 (20) 5 (100) 3 (60) 1 (20)
5 (25) 19 2.24 1 (20) 5 (100) 4 (80) 1 (20)
5 (25) 20.20 2.17 3 (60) 4 (80) 3 (60) 2 (40)
7 (35) 1 (5) 12 (60) 15 (75)
1 (20) 0 (0) 4 (80) 2 (40)
1 1 3 4
(20) (20) (60) (80)
2 0 3 4
(40) (0) (60) (80)
3 0 2 5
(60) (0) (40) (100)
0.514
5 (25) 11 (55) 2 (10) 0 (0) 1 (5) 1 (5) 11 (55)
0 (0) 3 (60) 1 (20) 0 (0) 1 (20)
2 2 0 0 0
(40) (40) (0) (0) (0)
2 2 1 0 0
(40) (40) (20) (0) (0)
1 4 0 0 0
(20) (80) (0) (0) (0)
0.512
2 (40)
3 (60)
3 (60)
3 (60)
0.895
12 (60) 8 (40)
3 (60) 2 (40)
3 (60) 2 (40)
3 (60) 2 (40)
3 (60) 2 (40)
d
8 (40) 3 (15) 3 (15) 2 (10) 4 (20)
3 (60) 1 (20) 0 (0) 0 (0) 1 (20)
1 1 1 0 2
(20) (20) (20) (0) (40)
2 0 2 0 1
(40) (0) (40) (0) (20)
2 (40) 1 (20) 0 (0) 2 (40) 0 (0)
0.322
10 (50) 4 (20) 2 (10) 3 (15) 1 (5)
3 (60) 2 (40) 0 (0) 0 (0)
1 1 1 1
(20) (20) (20) (20)
4 0 1 0
(80) (0) (20) (0)
2 1 0 2
0.394
N.A. ¼ Non applicable. Data presented as count (%) or mean standard deviation. * Stratification criteria.
(40) (20) (0) (40)
0.377 0.343 0.528 0.644 0.813
0.167
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
239
Table 2 e Mean VG training time by genre (in min). Week
Overall
Group y
SurgG 1 2 3
ShotG
RaceGy
258.47 39.148* 240 25.209 255.4 20.574 280 57.337 220.67 27.958* 201 21.909 225 32.596 236 19.812 204 18.822 206 13.874 187 8.367 219 18.166
Data are presented as the mean standard deviation. * Significant reduction (P ¼ 0.021) between weeks 1 and 2. y Significant difference between groups (P ¼ 0.045).
3.2.
VG training time
Students significantly (P ¼ 0.021) reduced their training time from week 0 to week 1 of training, but not between weeks 1 and 2 (P ¼ 0.221). The lack of a significant interaction effect between VG group and training week [F(4, 24) ¼ 0.981, P ¼ 0.436] suggested that the reduction in VG training minutes over time did not differ between groups. The significant effect of VG group (P ¼ 0.045) in the analysis, however, indicated differences in the average number of minutes played between VG groups: [F(2, 12) ¼ 4.071, r ¼ 1.19]. Mean training time of RaceG was longer compared to SurgG (P ¼ 0.045). The mean training time of ShotG was not significantly different from that of either RaceG (P ¼ 0.133) or SurgG (P ¼ 0.806). Mean VG training times (in min) by genre are shown in Table 2. The results of the evaluation of the VG training time of each VG group over the 3 wk of the study are provided in Figure 2.
Fig. 2 e Mixed ANOVA analysis for training time (min) by VG group per wk. The mean training time was significantly reduced from week 0 to week 1 (P [ 0.021), but not from week 1 to week 2 (P [ 0.221). * RaceG trained, on average, significantly longer than SurgG (P [ 0.45). (Color version of figure is available online.)
Fig. 3 e Mixed ANOVA analysis for overall VG scores (%) by VG group per performance evaluation. Mean scores were significantly increased from evaluation 0 to 1 (P < 0.001) and from evaluation 1 to 2 (P [ 0.010). (Color version of figure is available online.)
3.3.
VG performance
The overall VG scores (as a percentage) of each VG group over the three performance evaluations are shown in Figure 3. Students significantly increased their mean scores from evaluation 0 to evaluation 1 (1.646 versus 5.830, respectively; P < 0.001) and from evaluation 1 to evaluation 2 (5.830 versus 12.792, respectively; P ¼ 0.01). There was a significant interaction effect between VG group and mean performance evaluation scores [F(2.498, 14.986) ¼ 4.711; P ¼ 0.020], suggesting that there were differences between VG groups regarding the way in which the mean scores increased over the course of the evaluations. As seen in Figure 3, there is a less steep increase in ShotG scores than in either SurgG and to RaceG scores from evaluation 0 to 1, whereas the opposite occurred from evaluation 1 to 2, when the increment in mean scores was faster for ShotG than for the other two groups. The rate of increase in RaceG scores was relatively constant across the evaluations. There was no significant effect of VG group (P ¼ 0.983), indicating similar overall VG scores between VG groups [F(2, 12) ¼ 0.017], that is, no genre tended to result in higher or lower scores compared to the others.
3.4.
Surgical performance
3.4.1.
Surgeons’ evaluation
The inter-rater reliability for each surgical performance criterion is shown in Table 3. All criteria yielded a nonsignificant (P > 0.05) kappa coefficient, indicative of concordance between surgeons.
240
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
Table 3 e Surgeons’ inter-reliability for surgical performance evaluations. Performance criterion
Performance evaluation 0 *
K Instrument handling Quality of suturing Respect for tissue Motion and flow Overall performance *
0.154 0.204 0.125 0.092 0.188
P (95% CI) 0.190 (0.527 to 0.170 (0.129 to 0.202 (0.522 to 0.186 (0.457 to 0.063 (0.312 to
0.219) 0.536) 0.272) 0.272) 0.064)
Performance evaluation 1 *
Performance evaluation 2
K
P (95% CI)
k*
P (95% CI)
0.231 0.143 0.421 0.280 0.403
0.198 (0.157 to 0.618) 0.178 (0.207 to 0.492) 0.184 (0.060e0.783) 0.179 (0.071e0.632) 0.163 (0.084e0.722)
0.561 0.612 0.784 0.746 0.740
0.136 (0.295e0.828) 0.154 (0.311e0.914) 0.122 (0.545e1.000) 0.113 (0.524e0.967) 0.123 (0.500e0.981)
Weighted kappa coefficient.
3.4.2.
Control versus gaming groups
The mean surgical performance scores of the nongaming (control) and all gaming groups differed throughout the study. The interaction between evaluation time and surgical performance scores was statistically significant [F(2,36) ¼ 4.344, P ¼ 0.020, partial h2 ¼ 0.194]. Thus, the way in which the scores changed over the 3 wk of the study differed between the groups. At times 0 and 1, the surgical performance scores of the groups did not significantly differ [t(18) ¼ 0.440, P ¼ 0.665, and t(18) ¼ 1.684, P ¼ 0.109, respectively], whereas at time 2, the difference between the VG groups and the control was significant [t(18) ¼ 3.485; P ¼ 0.003, (95% confidence interval ¼ 4.434e1.099)]. The mean scores were greater in the gaming groups (mean ¼ 18.467, SE ¼ 1.55) than in the control group (mean ¼ 15.7, SE ¼ 1.48).
3.4.3.
VG genre analysis
The mean surgical performance scores for each evaluation are listed in Table 4. The scores for each VG group over the 3 wk of the study are shown in Figure 4. The statistically significant interaction between VG group and surgical performance score evaluation time [F(6,32) ¼ 3.980, P ¼ 0.011, partial h2 ¼ 0.427] indicated a difference between VG groups in the way in which the mean scores changed over the study. At times 0 and 1, there were no statistically significant differences in the surgical performance scores of the VG groups [F(3, 16) ¼ 1.127, P ¼ 0.368, partial h2 ¼ 0.174 and F(3, 16) ¼ 1.765, P ¼ 0.194, partial h2 ¼ 0.249, respectively]. A statistically significant difference in the surgical performance scores of the VG groups was, however, determined at time 2 [F(3, 16) ¼ 7.634, P ¼ 0.002, partial h2 ¼ 0.589]. The scores of ShotG (mean ¼ 3.9, SE ¼ 0.86, P ¼ 0.002) and SurgG
(mean ¼ 2.8, SE ¼ 0.86, P ¼ 0.022) were higher than the score achieved by the control group. In contrast, at this evaluation time, there was no statistically significantly difference between the RaceG and ContG scores (mean ¼ 1.6, SE ¼ 0.86, P ¼ 0.279).
3.4.3.1. ContG. There was a statistically significant effect of time on the performance evaluation score of ContG [F(2, 8) ¼ 33.857, P < 0.005, partial h2 ¼ 0.894]. The scores of this group increased between evaluations 0 and 1 (mean ¼ 4.6, SE ¼ 0.48, P ¼ 0.002) but not between evaluations 1 and 2 (mean ¼ 3.1, SE ¼ 0.99, P ¼ 0.106). The difference between evaluations 0 and 2 was also significant (mean ¼ 7.7, SE ¼ 1.2, P ¼ 0.009). 3.4.3.2. ShotG. Time had a statistically significant effect on the performance evaluation score of ShotG [F(2, 8) ¼ 251.782, P < 0.005, partial h2 ¼ 0.984]. ShotG scores increased significantly between evaluations 0 and 1 (mean ¼ 5.2, SE ¼ 0.46, P ¼ 0.001), 1 and 2 (mean ¼ 5.9, SE ¼ 0.58, P ¼ 0.002), and 0 and 2 (mean ¼ 11.1, SE ¼ 0.43, P < 0.005). 3.4.3.3. RaceG. Time also had a statistically significant effect on the performance evaluation score of RaceG [F(2, 8) ¼ 202.486, P < 0.005, partial h2 ¼ 0.981]. RaceG scores significantly increased between evaluations 0 and 1 (mean ¼ 6.6, SE ¼ 0.29, P < 0.005), 1 and 2 (mean ¼ 3.2, SE ¼ 0.46, P ¼ 0.007), and 0 and 2 (mean ¼ 9.8, SE ¼ 0.66, P < 0.005).
3.4.3.4. SurgG. The effect of time on the performance evaluation score of SurgG was also significant [F(2, 8) ¼ 114.115, P < 0.005, partial h2 ¼ 0.966]. The performance evaluation scores of this group increased between evaluations 0 and 1 (mean ¼ 4.1, SE ¼ 0.53, P ¼ 0.005), 1 and 2 (mean ¼ 5.5,
Table 4 e Mean surgical performance scores by performance evaluation. Performance evaluation 0 1 2
Group ContG
SurgG
ShotG
RaceG
8.0 1.27 12.6 1.29 15.7 1.48*,**
8.9 1.67 13.0 1.22 18.5 1.46*
8.5 1.12 13.7 1.25 19.6 0.74**
7.5 0.93 14.1 0.65 17.3 1.56
P value
Data are presented as the mean standard deviation. Values in italics denotes statistical significance in the overall group comparison. *, ** Statistical significant difference between groups.
0.368 0.194 0.002
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
241
randomized study to assess the effects of VGs, and specifically VG genre, on the development of surgical skills. The finding that different types of VGs may have different impacts on skill development has not been reported in other studies.
4.1.
Fig. 4 e Mixed ANOVA analysis of the mean surgical performance scores obtained from the three evaluation times. The interaction between the VG groups and the evaluation time of the surgical performance score was significant (P < 0.001). ⌘ Mean ShotG and SurgG scores were higher than the ContG score (P [ 0.002 and P [ 0.022, respectively). x Mean scores increased significantly from evaluation 0 to evaluation 1 (P [ 0.002), but not from evaluation 1 to evaluation 2 (P [ 0.106). * Means scores increased significantly from evaluation 0 to evaluation 1 and from evaluation 1 to evaluation 2 (P < 0.05). (Color version of figure is available online.) SE ¼ 0.59, P ¼ 0.002), and 0 and 2 (mean ¼ 9.6, SE ¼ 0.76, P < 0.001).
4.
Gaming versus nongaming
Our results are in agreement with data on the impact of VGs on surgical skills, in which playing virtual games was shown to improve surgical skills [1e12]. When analyzed together, the VG groups in our study scored significantly higher in their surgical evaluations than the control group after 3 wk of systematic Wii training. No other study has corroborated a significant positive correlation between VGs and open surgery modalities (i.e., neither laparoscopic nor endoscopic). In the only study that investigated traditional surgical skills [15], VG playing was not a significant predictor of surgical skills scores. However, the participants in that study were veterinary students with previous surgical experience (they had acquired soft-tissue surgical skills during training with various suturing pads). This may explain the nonsignificant improvement in that series, because the impact of VGs is likely to be minor when higher levels of expertise in the task are already present. In addition, the games used in that study were all mini-games (skeet shooting, archery, and target shooting) within an Olympic Games simulation franchise for the Nintendo Wii. Those intuitive, single-goal games differed from the VGs used in our study. Harper et al. [14] showed that VG users acquire the skills for robotic suturing techniques more slowly than those for endoscopic techniques. The same study found significant correlations with a prior history of musical instrument playing and with the practicing of sports for >4 y. Self-reported VG experience (number of hours) was used as the main variable, and the sample size was only 10 students, which might have affected the statistical power.
Discussion 4.2.
Different VG genres were found to differentially impact the basic development of surgical skills of inexperienced medical students. More complex games, with more elements to master, such as first-person-shooter or surgery-simulation games, improved performance even if played less than the more “playable” racing game. Shooter games take longer to master than surgical games and are more appealing to seasoned than to casual gamers. Surgical trainees are busy students, with limited time for VG playing [12]. Surgery games seem to be appropriate adjuvants to surgical skills development. More than one study [2,3,20] have suggested the use of VGs as part of a program to train the basic laparoscopic skills of surgical novices. In fact, VGs have already been used to transfer cognitive-enhanced characteristics to everyday tasks. In a previous study [21], cadets who trained for 10 h on the videogame Space Fortress were rated higher by instructors than a control group. The impact of VGs on surgical skills is an area of current interest, given the reduction of surgical training hours in many medical programs and the need to identify new teaching paradigms in surgical training that will make a difference in clinical practice. Ours is the first controlled and
Playability
In previous studies, a trend toward improvement of laparoscopic skills was found in participants who trained with VGs for 180 min per wk over 1 wk [6]. In our study, the VG training time was roughly 230 min per wk over 3 wk and thus comparable with studies in which subjects trained for 150e300 min per wk over 3 wk [1] or 75e270 min per wk for 2 wk [12]. Our data on VG training time suggest that different VG genres have different levels of acceptance among medical students. A further, important observation was that the students played significantly less after the first week regardless of the genre. This was evidenced by the nonsignificant interaction effect between group and time, which showed that the decrement in training time was essentially the same between groups. However, overall, the racing game was played more than the other games, perhaps because racing games are inherently more “playable” because of the smaller number of elements that must be mastered, the more easily distinguishable objectives, and easier game play. Although not statistically significant, the surgery game was played more often from week 2 to week 3. When asked about
242
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
why they played more during the last week, students stated that, after understanding what each surgical instrument was meant for, how each one worked in terms of the controller, and the purpose of each level, the game became “more fun.” This suggests that surgery games, although more complex than racing games, maintain their playability. Shooter games, such as the one used in the study (which includes a more complex storyline, several projectile types, and the exploration of different scenarios other than just simply “aiming and shooting”), comprise a late mastery and complex game genre, more suitable to experienced VG players.
4.3.
Complexity
Schlickum et al. [1] pointed out the positive transfer effects that only cognitively loaded games (such as a computer virtual chess game) have on laparoscopic simulation, which suggested that virtual cognitive training alone is sufficient to impact surgical performance. More cognitively demanding (i.e., more complex) games may result in even further improvements. In the present study, students improved their VG scores each week. This improvement in scores was different for each VG genre. ShotG showed a poor improvement from evaluation 0 to evaluation 1, especially compared to SurgG and RaceG. In the following period (between evaluations 1 and 2), the pattern of improvement was different: the improvement of SurgG reached a plateau, whereas in ShotG there was a >three-fold increment in scores. The surgery game could be mastered during the 3 wk of this study. RaceG scores improved at a constant rate across evaluations. A longer VG training time and a more detailed and focused analysis would be necessary to suitably explore the complexity patterns of each VG genre. However, some insights can already be gained based on the surgical learning perspective, as discussed in the section.
4.4.
the task and a decreased awareness of irrelevant aspects. Regarding the ShotG and SurgG training times, despite similar patterns of impact on surgical performance, the surgery-simulation games seemed to be generally more appealing to the medical students. Surgery-themed games such as the one used in the study are more easily contextualized and thus perhaps more enjoyable to novice surgeons [20]. Our data suggest that surgery games such as Trauma Center could be more effective for enhancing the development of surgical skills than first-person-shooter games such as Resident Evil. Perhaps the latter would be beneficial to seasoned game players. Nevertheless, our study suggests that VGs can provide harmless and cost-effective improvements in surgical education.
4.5.
Limitations
Although our results point out the potential benefits of exposing medical students to VGs to improve their surgical skills, our study had several limitations. We had a low participation rate (21.34%), which we attribute to having sent the questionnaires close to the end of the semester when the students would have their finals. The small sample limited the amount of data available from the questionnaire as covariates in a multivariate analysis. Raters were not trained in the assessment tool before the study. This may explain why the reliability between the two raters improved with each successive evaluation. Also, the poor inter-rater reliability is a significant limiting factor. Because of these limitations, we refer to our work as a feasibility study. Nonetheless, the solid methodology used in this study and the data it provided are sufficient to support further investigations. Additional studies should focus on different aspects of the relationship between VG playing and surgery skills, because it may offer efficient and cost-effective surgical learning options.
Genre analysis
Supporting our hypothesis, different VG genres were found to differentially impact the potential for surgical performance improvement. The basal surgical scores were the same among the four groups, and after 1 wk of training, there were still no significant differences between them. Thus, a single week of Wii training is not enough to significantly impact the development of surgical skills. In Boyle et al. [6], only a trend toward better performance was found after the same intervention. In the present study, after 3 wk of VG training, surgical performance scores were different between the VG and control groups. Students playing the surgery and firstperson-shooter games scored higher than the control group in the surgical performance evaluation. RaceG scored higher on average than ContG but the difference was not significant, although among the three genres, the racing game was the one that was played most often. Therefore, the “playability” of a game seems to have less of an effect than its complexity with respect to surgical development. This can be attributed to the greater “attentional weighting” [3], a mechanism of perceptual learning, required for more complex games. Thus, complex games more often and more intensively require that players have a heightened awareness of important aspects of
5.
Conclusions
Different VG genres differentially impacted the development of surgical skills by inexperienced medical students. More complex games, such as surgery and first-person-shooter games, improved performance although they were played less. Racing games, although more “playable,” did not significantly improve surgical performance scores over those of a nongaming group. Although further studies are needed, our results suggest that surgery-type VGs, with their complexity and playability, offer an appropriate adjuvant to improve the surgical performance of medical students.
Acknowledgment Funding for this study was obtained from Pro´-Ensino na Sau´de (Project number 39), of the Coordenac¸a˜o de Aperfeic¸oamento de Pessoal de Nı´vel Superior (CAPES). Academic support and funding came from the Fundac¸a˜o de Amparo a` Pesquisa do Estado do Rio Grande do Sul (FAPERGS).
j o u r n a l o f s u r g i c a l r e s e a r c h 2 0 1 ( 2 0 1 6 ) 2 3 5 e2 4 3
Authors’ contributions: T.B.A. conceptualized and designed the study and drafted the initial article. T.B.A., C.D.F., and R.S.W. reviewed and revised the article. T.B.A., D.L.S.S., Y.T.M.S., and R.S.W. carried out the study and collected the data. T.B.A., F.R.S., D.L.S.S., Y.T.M.S., C.D.F., and R.S.W. approved the final version of the article as submitted.
Disclosure The authors have no proprietary or commercial interest in any product mentioned or concept discussed in this article.
references
[1] Schlickum MK, Hedman L, Enochsson L, Kjellin A, FellanderTsai L. Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: a prospective randomized study. World J Surg 2009;33:2360. [2] Jalink MB, Goris J, Heineman E, Pierie JP, ten Cate Hoedemaker HO. The effects of video games on laparoscopic simulator skills. Am J Surg 2014;208:151. [3] Lynch J, Aughwane P, Hammond TM. Video games and surgical ability: a literature review. J Surg Educ 2010;67:184. [4] Adams BJ, Margaron F, Kaplan BJ. Comparing video games and laparoscopic simulators in the development of laparoscopic skills in surgical residents. J Surg Educ 2012;69:714. [5] van Dongen KW, Verleisdonk EJ, Schijven MP, Broeders IA. Will the Playstation generation become better endoscopic surgeons? Surg Endosc 2011;25:2275. [6] Boyle E, Kennedy AM, Traynor O, Hill AD. Training surgical skills using nonsurgical tasksecan Nintendo Wii improve surgical performance? J Surg Educ 2011;68:148. [7] Badurdeen S, Abdul-Samad O, Story G, Wilson C, Down S, et al. Nintendo Wii video-gaming ability predicts laparoscopic skill. Surg Endosc 2010;24:1824. [8] Shane MD, Pettitt BJ, Morgenthal CB, Smith CD. Should surgical novices trade their retractors for joysticks? Videogame experience decreases the time needed to acquire surgical skills. Surg Endosc 2008;22:1294.
243
[9] Rosser JC Jr, Lynch PJ, Cuddihy L, Gentile DA, Klonsky J, et al. The impact of video games on training surgeons in the 21st century. Arch Surg 2007;142:181. discusssion 186. [10] Jalink MB, Heineman E, Pierie JP, Ten Cate Hoedemaker HO. The effect of a preoperative warm-up with a custom-made Nintendo video game on the performance of laparoscopic surgeons. Surg Endosc 2015;29:2284. [11] Ju R, Chang PL, Buckley AP, Wang KC. Comparison of Nintendo Wii and PlayStation2 for enhancing laparoscopic skills. JSLS 2012;16:612. [12] Rosenberg BH, Landsittel D, Averch TD. Can video games be used to predict or improve laparoscopic skills? J Endourol 2005;19:372. [13] Basak C, Voss MW, Erickson KI, Boot WR, Kramer AF. Regional differences in brain volume predict the acquisition of skill in a complex real-time strategy videogame. Brain Cogn 2011;76:407. [14] Harper JD, Kaiser S, Ebrahimi K, et al. Prior video game exposure does not enhance robotic surgical performance. J Endourol 2007;21:1207. [15] Millard HA, Millard RP, Constable PD, Freeman LJ. Relationships among video gaming proficiency and spatial orientation, laparoscopic, and traditional surgical skills of third-year veterinary students. J Am Vet Med Assoc 2014;244:357. [16] Boyd T, Jung I, Van Sickle K, Schwesinger W, Michalek J, Bingener J. Music experience influences laparoscopic skills performance. JSLS 2008;12:292. [17] Willis RE, Gomez PP, Ivatury SJ, Mitra HS, Van Sickle KR. Virtual reality simulators: valuable surgical skills trainers or video games? J Surg Educ 2014;71:426. [18] Denadai R, Saad-Hossne R, Raposo-Amaral CE. Simulationbased Rhomboid flap skills training during medical education: comparing low- and high-fidelity bench models. J Craniofac Surg 2014;25:2134. [19] Chipman JG, Schmitz CC. Using objective structured assessment of technical skills to evaluate a basic skills simulation curriculum for first-year surgical residents. J Am Coll Surg 2009;209:364. [20] Meterissian S, Liberman M, McLeod P. Games as teaching tools in a surgical residency. Med Teach 2007;29:e258. [21] Gopher D. The skill of attention control: acquisition and execution of attention strategies. In: Meyer D, Kornblum S, editors. Attention and Performance XIV (Silver Jubilee Volume): Synergies in Experimental Psychology, Artificial Intelligence, and Cognitive Neuroscience. Cambridge, MA: The MIT Press; 1993. p. 299.