- Email: [email protected]
Virtual reality laparoscopic simulator for assessment in gynaecology
BJOG: an International Journal of Obstetrics and Gynaecology February 2003, Vol. 110, pp. 181 –187
MEDICAL EDUCATION
Virtual reality laparoscopic simulator for assessment in gynaecology Mounna Gora, Rory McCloya, Robert Stonea, Anthony Smithb,* A validated virtual reality laparoscopic simulator minimally invasive surgical trainer (MIST) 2 was used to assess the psychomotor skills of 21 gynaecologists (2 consultants, 8 registrars and 11 senior house officers). Nine gynaecologists failed to complete the VR tasks at the first attempt and were excluded for sequential evaluation. Each of the remaining 12 gynaecologists were tested on MIST 2 on four occasions within four weeks. The MIST 2 simulator provided quantitative data on time to complete tasks, errors, economy of movement and economy of diathermy use—for both right and left hand performance. The results show a significant early learning curve for the majority of tasks which plateaued by the third session. This suggests a high quality surgeon –computer interface. MIST 2 provides objective assessment of laparoscopic skills in gynaecologists. Introduction Laparoscopy has been fundamental to gynaecologic diagnosis since the 1970s. Although early use was restricted to diagnostic laparoscopy and laparoscopic sterilisation, the laparoscopic technique is now recommended for the surgical treatment of ectopic pregnancy and is also being applied to traditionally open surgical fields such as hysterectomy, lymphadenectomy and pelvic floor repair. In the absence of simulators, other than simple box trainers, trainee gynaecologists have to acquire the new skills that are particular to laparoscopy (Table 1) by apprenticeship in the operating theatre. A study of medical students showed that as many as 10% of the population may be unable to acquire the skills for laparoscopic surgery1. Unbiased quantitative measurement of performance could give objective feedback to trainees2, help assessors to ensure that minimal standards of ability are met and tailor the length of simulator training to the variable abilities of trainees. During the last few years, virtual reality trainers have been developed to address the issues of assessment and training2 – 4. Studies involving general surgeons have led to implementation of VR training in surgical skills courses in a major international training centre in Germany5. The concept of a part-task VR surgical simulator, and its
implications for surgical practice, has been a subject of a recent review4. This study addresses the features and uses of minimally invasive surgical trainer (MIST) 2, a part-task VR simulator, by presenting data collected from 21 gynaecologists in the Manchester area and comparing the findings with those from studies in general surgery2.
Methods MIST 2 is a second generation of part-task VR laparoscopic simulator (Fig. 1). It is now commercially available as Procedicus MIST from Mentice, Gothenburg, Sweden (www.mentice.com). Like its predecessor, MIST-VR6, MIST 2 was developed in our centre in conjunction with Virtual Presence (London, UK) and consists of an entry level PC and an interface comprising laparoscopic tools and a diathermy pedal with digitised computer output. Standard laparoscopic tool handles are held in gimbals which measure the position of the tools 50 times each second and in this way, generates performance data of time taken, errors made, economy of movement (a ratio of actual instrument path length over the optimal straight path) and economy of diathermy (a ratio of the actual time the diathermy pedal was activated over the required time). Performance can be analysed for both right and left hands separately.
a
Wolfson Centre for Minimally Invasive Therapy, Manchester Royal Infirmary, Manchester, UK b Department of Gynaecology, St Mary’s Hospital, Manchester, UK * Correspondence: Dr A. Smith, St Mary’s Hospital, Oxford Road, Manchester M13 9WL, UK. D RCOG 2003 BJOG: an International Journal of Obstetrics and Gynaecology PII: S 1 4 7 0 - 0 3 2 8 ( 0 2 ) 0 2 5 1 6 - 8
Table 1. Psychomotor problems of laparoscopic surgery. Remote view of operating field 2-D view of 3-D operating field Fulcrum effect Reduced haptic sensory feedback
www.bjog-elsevier.com
182
MEDICAL EDUCATION
Fig. 1. Virtual task being performed on MIST 2. The MIST system consists of an entry level PC connected to a virtual laparoscopic interface via a jig.
Due to the potential enormity of the data files, which could be generated if the simulated task takes an excessive time, the MIST software has a 10-minute cutoff whereby tasks are closed automatically after 10 minutes of use. If the system times out the data for that task is lost. MIST 2 runs the six basic surgical tasks originally made for MIST-VR6. The complex movements that make up a laparoscopic procedure have been deconstructed into abstracted computer graphics, part-tasks, which the user interacts with through the mechanical – virtual instrument interface
(Fig. 1) — details of tasks are presented below. It also runs two new tasks, which test additional skills particularly utilised in gynaecology. These additional skills were developed by an ergonomic analysis of laparoscopic procedures by gynaecologists. The skills in each module can be worked through in a stepwise manner, the final tasks incorporating skills learned in the earlier tasks. Learning a skill has been described as consisting of two parts: initial familiarisation with the task, followed by gradual refinement of the skill2,7. Initial learning rate on a VR simulator is dependent upon the quality of human – computer interaction. A high-quality interaction results in a faster initial familiarisation curve. If familiarisation rate is rapid, a larger proportion of available time can be used for the acquisition of skills. Also, ability is most accurately measured after the process of familiarisation is complete and so must be taken into account during assessment of individuals2. A total of 21 gynaecologists (2 consultants, 8 registrars and 11 senior house officers) from three teaching hospitals in Manchester volunteered to participate in this study. The study involved attendance of four sessions on MIST 2 at similar times of day, over a period of two to four weeks. Each session consisted of the performance of four tasks in a set order (Figs 2– 5). Each task was performed three times with the right hand dominant, and three times with the left hand dominant and the performance data were averaged. Complete sets of performance data could not be obtained for nine participants (one consultant, three registrars and five senior house officers) who failed to complete one or more of the tasks in a maximum time of 10 minutes. Exclusion of these participants from the study analyses allowed for a more appropriate repeated measures statistical test to be applied over the four sessions (see below). We tested traversal and manipulatediathermy tasks (formerly tasks three and six on MIST-VR6), which had been validated by studies with general surgeons2,6. Traversal task (Fig. 2) involves passing along the length
Fig. 2. Traversal task. Two-handed task: 1. locate and grasp a randomly placed tube; 2. walk along the length of the tube by passing it from hand to hand.
D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
MEDICAL EDUCATION
183
Fig. 3. ManipulateDiathermy task. Compound two handed task with diathermy pedal: 1. locate and grasp a randomly placed sphere; 2. when touched with the other tool a box appears. The position of the sphere maintained in the box from until the end of the task; 3. withdraw one tool from the operating field and reinsert as a diathermy hook; 4. apply 3 seconds of diathermy to each of three nodules.
of a cylindrical object by transferring it alternately between right and left handed instruments. ManipulateDiathermy task (Fig. 3) is a compound task which involves
grasping and placing an object in a defined place, changing instruments and the application of diathermy. Subjects were then tested on two new gynaecology tasks. StretchClip
Fig. 4. StretchClip task. Compound two-handed task: 1. locate and grasp the target end of a tube; 2. stretch the tube and hold at a defined length while a clip is applied.
D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
184
MEDICAL EDUCATION
Fig. 5. StretchDiathermy task. Compound two-handed task with diathermy pedal: 1. locate and grasp the target end of a tube; 2. stretch to a predetermined length; 3. maintain this length while diathermy is applied along a target line.
task (Fig. 4) tests the ability to stretch tissue to a predetermined length in order to apply a virtual clip to a predefined target zone, analagous to a laparoscopic sterilisation procedure. StretchDiathermy task (Fig. 5) involves the cutting along a predefined path using a virtual diathermy hook, while holding the tissue in a predetermined stretched position, analogous to the manipulation required on a fallopian tube during removal of an ectopic pregnancy. Previous laparoscopic experience, demographic data and handedness were also recorded. The data for time taken, economy of movement, errors and economy of diathermy were transformed by taking reciprocals, and then the variation among medians were compared using the Friedman test (non-parametric repeated measures ANOVA test.) Multiple comparisons were then made to identify when the most significant changes occurred. Analysis of summary measures8 was also used to test for differences between right and left handed performance.
Results A statistically significant early learning curve, lasting three sessions, was demonstrated for time taken and economy of movement for all of the tasks (Fig. 6). Statistically significant early learning curves were also observed for the number of errors made for Traversal, ManipulateDiathermy and StretchDaithermy tasks. There was a visible nonsignificant improvement in errors made for the StretchClip task. Multiple comparisons (Table 2) showed a significant improvement from the first session to the third session in all tasks. Diathermy was involved in two of the tasks. In StretchDiathermy, where the participant must apply diathermy continuously along a straight line, there was significant familiarisation within three sessions for all parameters except for errors, which took four sessions before a significant improvement was seen. In ManipulateDiathermy,
where diathermy must be applied to three discrete nodules, there was no visible learning curve for economy of diathermy. There was no significant difference in performance of right and left hands.
Discussion During the last five years, there has been increasingly widespread recognition that objective measures of surgical performance are desirable and essential9. It has also been recognised that VR simulators can provide these measures4. The present study of psychomotor performance of gynaecologists on MIST 2 clearly supports existing general surgical evidence2,3,10 that MIST provides objective measurement of relevant laparoscopic skills. As in previous studies on MIST VR, the right and left hands perform with equal dexterity in laparoscopic surgery after initial familiarisation. (Chaudhry et al., Left Handed Performance in Right Handed Surgeons, unpublished observations). This is probably due to a vectored reflex arc that is set up across both shoulders, elbows, wrists and hands by the twohanded surgical technique that is advised and taught for laparoscopy. This study has also demonstrated a high-quality user – computer interface allowing rapid familiarisation with the system and with the tasks. Similar early learning curves were demonstrated for both the gynaecology tasks and the original general tasks, suggesting that the data presented are reliable. Of the nine participants who failed to complete one or more of the tasks on the first session, eight were able to complete the tasks on the second session. This suggests a marked improvement in time taken from the first to the second session and thus supports the existence of early familiarisation. The results for all four sessions (Fig. 6, Table 2) demonstrated that further familiarisation with MIST 2 was complete by the fourth session for most D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
MEDICAL EDUCATION
185
Fig. 6. Mean scores plotted over four sessions for all participants, and for separate groups of gynaecologist. Lower values indicate better performance. All ; senior house officers ; senior gynaecologists .
D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
186
MEDICAL EDUCATION
Table 2. Friedman analysis of learning rate with multiple comparison of consecutive sessions. Task
Time (P)
Economy of movement (P)
Errors (P)
Economy of diathermy (P)
TR Session Session Session Session Session Session
1 1 1 2 2 3
vs vs vs vs vs vs
2 3 4 3 4 4
<0.0001 ns <0.001 <0.001 <0.05 ns ns
<0.0001 ns <0.001 <0.001 ns ns ns
0.0001 ns <0.001 <0.01 ns ns ns
– – – – – – –
MD Session Session Session Session Session Session
1 1 1 2 2 3
vs vs vs vs vs vs
2 3 4 3 4 4
<0.0001 ns <0.001 <0.001 ns <0.01 ns
<0.0001 ns <0.01 <0.001 ns <0.05 ns
<0.001 ns <0.05 <0.001 ns <0.01 ns
ns ns ns ns ns ns ns
SC Session Session Session Session Session Session
1 1 1 2 2 3
vs vs vs vs vs vs
2 3 4 3 4 4
<0.0001 ns <0.001 <0.001 ns ns ns
<0.0001 ns <0.01 <0.001 ns ns ns
ns ns ns ns ns ns ns
– – – – – – –
SD Session Session Session Session Session Session
1 1 1 2 2 3
vs vs vs vs vs vs
2 3 4 3 4 4
<0.0001 ns <0.001 <0.001 <0.05 <0.01 ns
<0.0001 ns <0.001 <0.001 ns <0.01 ns
<0.0011 ns ns <0.01 ns <0.01 ns
<0.0005 ns <0.05 <0.001 ns ns ns
TR ¼ Transversal Task, MD ¼ ManipulateDiathermy Task, SC ¼ StretchClip Task, SD ¼ StretchDiathermy Task.
participants, suggesting that further sessions on MIST 2 would have resulted in refinement of the skills. This fits with the two-part learning model described by Mitta and Packebush7 and is paralleled by findings on general surgeons using MIST2. It was not possible to record performance data for those tasks in which participants ran out of time due to the 10-minute cutoff inherent in the MIST system. Nine of the 21 volunteers timed out on at least one task. This was surprising as the experience of one author (RFM) of testing more than 100 medical students and general surgeons is that less than 5% of subjects studied on MIST are unable to complete tasks within a maximum of 10 minutes. It is possible that these subjects failed to understand what was required to perform on this VR simulator, or were distracted from the tasks by a lack of computer literacy or computer anxiety4,11. However, all participants received a standardised introductory explanation of the simulator and of the tasks. It is more likely that they did not have sufficient psychomotor skills to complete the tasks. Perhaps this highlights the limitations of apprenticeship training. Maybe gynaecologists perform adequate numbers of laparoscopic sterilisation operations from an early stage in their career, but do not have sufficient training
opportunity for training in other operations involving more complex psychomotor skills. It is important, however, to have a range of skills in order that complications be handled effectively. Objective assessment of psychomotor skills might ensure that only those with a basic level of skill enter clinical training in laparoscopic surgery. An assessment tool such as MIST could be used for this type of ‘aptitude’ testing2. This type of summative assessment might also allow more standardisation and structuring of training across the country. A VR trainer allows trainees to repeatedly practice psychomotor skills alone and to obtain structured feedback away from the operating theatre. The Royal College of Obstetricians and Gynaecologists has recommended that surgical training systems such as MIST be evaluated, piloted and introduced into Basic Surgical Skills Courses12.
Conclusion Objective unbiased assessment of laparoscopic performance is a much needed and valuable component of assessment of competence of gynaecologists. Part-task VR D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
MEDICAL EDUCATION
simulators such as MIST assess relevant psychomotor skills consistently and reliably. The rapid initial familiarisation curve consistently demonstrated by individuals in this study indicates a highquality interface and suggests that the most appropriate time to assess performance is on the fourth session when the initial familiarisation process has been overcome.
Acknowledgements The authors would like to thank Christopher Sutton and Jim Gant for continued support for the software development of MIST.
4. 5.
6.
7.
8.
9.
References 1. Cuschieri A. Reflections on surgical training. Surg Endosc 1993;7: 73 – 74. 2. Chaudhry A, Sutton C, Wood J, Stone R, McCloy R. Learning rate for laparoscopic surgical skills on MIST-VR, a virtual reality simulator: quality of human computer interface. Ann R Coll Surg Engl 1999;81:281 – 286. 3. Taffinder N, Sutton C, Fishwick RJ, Mannus IC, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparo-
D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
10.
11. 12.
187
scopic surgery: results from randomised controlled studies using MIST VR laparoscopic simulator. Br J Surg 1998;85(Suppl 1):75. McCloy RF, Stone R. Virtual reality in surgery. BMJ 2001;323: 912 – 915. Buerger T, Erdtmann M. Inevitable developments for VR training systems —the ESI perspective. The Ninth Annual Medicine Meets Virtual Reality Conference. Newport Beach, California, USA, January 2001. Wilson MS, Middlebrook A, Sutton C, Stone R, McCloy RF. MIST VR: a virtual reality trainer for laparoscopic surgery assesses performance. Ann R Coll Surg Engl 1997;79:403 – 404. Mitta DA, Packebush SJ. Improving interface quality: an investigation of human computer interaction and task learning. Ergonomics 1995;38:1307 – 1325. Mathews JNS, Altman Douglas G, Campbell MJ, Royston P. Analysis of serial measurements in medical research. BMJ 1990;300: 230 – 235. Taffinder NJ, McManus IC, Gul Y, Russell RCG, Darzi A. Effect of sleep deprivation on surgeons’ dexterity on laparoscopic simulator. Lancet 1998;352:1191. Gallagher AG. Objective computer-based assessment of perceptual skill that predicts laparoscopic performance in three separate studies. Ninth Annual Medicine Meets Virtual Reality Conference. Newport Beach, California, 2001. Ray NM, Minch RP. Computer anxiety and alienation: towards a definition and parsimonious measure. Hum Factors 1990;32:477 – 491. Report of an RCOG Working Party. Discussion document on further training for doctors in difficulty. R Coll Obstet Gynaecol April 2002.
Accepted 7 August 2002
MEDICAL EDUCATION
Virtual reality laparoscopic simulator for assessment in gynaecology Mounna Gora, Rory McCloya, Robert Stonea, Anthony Smithb,* A validated virtual reality laparoscopic simulator minimally invasive surgical trainer (MIST) 2 was used to assess the psychomotor skills of 21 gynaecologists (2 consultants, 8 registrars and 11 senior house officers). Nine gynaecologists failed to complete the VR tasks at the first attempt and were excluded for sequential evaluation. Each of the remaining 12 gynaecologists were tested on MIST 2 on four occasions within four weeks. The MIST 2 simulator provided quantitative data on time to complete tasks, errors, economy of movement and economy of diathermy use—for both right and left hand performance. The results show a significant early learning curve for the majority of tasks which plateaued by the third session. This suggests a high quality surgeon –computer interface. MIST 2 provides objective assessment of laparoscopic skills in gynaecologists. Introduction Laparoscopy has been fundamental to gynaecologic diagnosis since the 1970s. Although early use was restricted to diagnostic laparoscopy and laparoscopic sterilisation, the laparoscopic technique is now recommended for the surgical treatment of ectopic pregnancy and is also being applied to traditionally open surgical fields such as hysterectomy, lymphadenectomy and pelvic floor repair. In the absence of simulators, other than simple box trainers, trainee gynaecologists have to acquire the new skills that are particular to laparoscopy (Table 1) by apprenticeship in the operating theatre. A study of medical students showed that as many as 10% of the population may be unable to acquire the skills for laparoscopic surgery1. Unbiased quantitative measurement of performance could give objective feedback to trainees2, help assessors to ensure that minimal standards of ability are met and tailor the length of simulator training to the variable abilities of trainees. During the last few years, virtual reality trainers have been developed to address the issues of assessment and training2 – 4. Studies involving general surgeons have led to implementation of VR training in surgical skills courses in a major international training centre in Germany5. The concept of a part-task VR surgical simulator, and its
implications for surgical practice, has been a subject of a recent review4. This study addresses the features and uses of minimally invasive surgical trainer (MIST) 2, a part-task VR simulator, by presenting data collected from 21 gynaecologists in the Manchester area and comparing the findings with those from studies in general surgery2.
Methods MIST 2 is a second generation of part-task VR laparoscopic simulator (Fig. 1). It is now commercially available as Procedicus MIST from Mentice, Gothenburg, Sweden (www.mentice.com). Like its predecessor, MIST-VR6, MIST 2 was developed in our centre in conjunction with Virtual Presence (London, UK) and consists of an entry level PC and an interface comprising laparoscopic tools and a diathermy pedal with digitised computer output. Standard laparoscopic tool handles are held in gimbals which measure the position of the tools 50 times each second and in this way, generates performance data of time taken, errors made, economy of movement (a ratio of actual instrument path length over the optimal straight path) and economy of diathermy (a ratio of the actual time the diathermy pedal was activated over the required time). Performance can be analysed for both right and left hands separately.
a
Wolfson Centre for Minimally Invasive Therapy, Manchester Royal Infirmary, Manchester, UK b Department of Gynaecology, St Mary’s Hospital, Manchester, UK * Correspondence: Dr A. Smith, St Mary’s Hospital, Oxford Road, Manchester M13 9WL, UK. D RCOG 2003 BJOG: an International Journal of Obstetrics and Gynaecology PII: S 1 4 7 0 - 0 3 2 8 ( 0 2 ) 0 2 5 1 6 - 8
Table 1. Psychomotor problems of laparoscopic surgery. Remote view of operating field 2-D view of 3-D operating field Fulcrum effect Reduced haptic sensory feedback
www.bjog-elsevier.com
182
MEDICAL EDUCATION
Fig. 1. Virtual task being performed on MIST 2. The MIST system consists of an entry level PC connected to a virtual laparoscopic interface via a jig.
Due to the potential enormity of the data files, which could be generated if the simulated task takes an excessive time, the MIST software has a 10-minute cutoff whereby tasks are closed automatically after 10 minutes of use. If the system times out the data for that task is lost. MIST 2 runs the six basic surgical tasks originally made for MIST-VR6. The complex movements that make up a laparoscopic procedure have been deconstructed into abstracted computer graphics, part-tasks, which the user interacts with through the mechanical – virtual instrument interface
(Fig. 1) — details of tasks are presented below. It also runs two new tasks, which test additional skills particularly utilised in gynaecology. These additional skills were developed by an ergonomic analysis of laparoscopic procedures by gynaecologists. The skills in each module can be worked through in a stepwise manner, the final tasks incorporating skills learned in the earlier tasks. Learning a skill has been described as consisting of two parts: initial familiarisation with the task, followed by gradual refinement of the skill2,7. Initial learning rate on a VR simulator is dependent upon the quality of human – computer interaction. A high-quality interaction results in a faster initial familiarisation curve. If familiarisation rate is rapid, a larger proportion of available time can be used for the acquisition of skills. Also, ability is most accurately measured after the process of familiarisation is complete and so must be taken into account during assessment of individuals2. A total of 21 gynaecologists (2 consultants, 8 registrars and 11 senior house officers) from three teaching hospitals in Manchester volunteered to participate in this study. The study involved attendance of four sessions on MIST 2 at similar times of day, over a period of two to four weeks. Each session consisted of the performance of four tasks in a set order (Figs 2– 5). Each task was performed three times with the right hand dominant, and three times with the left hand dominant and the performance data were averaged. Complete sets of performance data could not be obtained for nine participants (one consultant, three registrars and five senior house officers) who failed to complete one or more of the tasks in a maximum time of 10 minutes. Exclusion of these participants from the study analyses allowed for a more appropriate repeated measures statistical test to be applied over the four sessions (see below). We tested traversal and manipulatediathermy tasks (formerly tasks three and six on MIST-VR6), which had been validated by studies with general surgeons2,6. Traversal task (Fig. 2) involves passing along the length
Fig. 2. Traversal task. Two-handed task: 1. locate and grasp a randomly placed tube; 2. walk along the length of the tube by passing it from hand to hand.
D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
MEDICAL EDUCATION
183
Fig. 3. ManipulateDiathermy task. Compound two handed task with diathermy pedal: 1. locate and grasp a randomly placed sphere; 2. when touched with the other tool a box appears. The position of the sphere maintained in the box from until the end of the task; 3. withdraw one tool from the operating field and reinsert as a diathermy hook; 4. apply 3 seconds of diathermy to each of three nodules.
of a cylindrical object by transferring it alternately between right and left handed instruments. ManipulateDiathermy task (Fig. 3) is a compound task which involves
grasping and placing an object in a defined place, changing instruments and the application of diathermy. Subjects were then tested on two new gynaecology tasks. StretchClip
Fig. 4. StretchClip task. Compound two-handed task: 1. locate and grasp the target end of a tube; 2. stretch the tube and hold at a defined length while a clip is applied.
D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
184
MEDICAL EDUCATION
Fig. 5. StretchDiathermy task. Compound two-handed task with diathermy pedal: 1. locate and grasp the target end of a tube; 2. stretch to a predetermined length; 3. maintain this length while diathermy is applied along a target line.
task (Fig. 4) tests the ability to stretch tissue to a predetermined length in order to apply a virtual clip to a predefined target zone, analagous to a laparoscopic sterilisation procedure. StretchDiathermy task (Fig. 5) involves the cutting along a predefined path using a virtual diathermy hook, while holding the tissue in a predetermined stretched position, analogous to the manipulation required on a fallopian tube during removal of an ectopic pregnancy. Previous laparoscopic experience, demographic data and handedness were also recorded. The data for time taken, economy of movement, errors and economy of diathermy were transformed by taking reciprocals, and then the variation among medians were compared using the Friedman test (non-parametric repeated measures ANOVA test.) Multiple comparisons were then made to identify when the most significant changes occurred. Analysis of summary measures8 was also used to test for differences between right and left handed performance.
Results A statistically significant early learning curve, lasting three sessions, was demonstrated for time taken and economy of movement for all of the tasks (Fig. 6). Statistically significant early learning curves were also observed for the number of errors made for Traversal, ManipulateDiathermy and StretchDaithermy tasks. There was a visible nonsignificant improvement in errors made for the StretchClip task. Multiple comparisons (Table 2) showed a significant improvement from the first session to the third session in all tasks. Diathermy was involved in two of the tasks. In StretchDiathermy, where the participant must apply diathermy continuously along a straight line, there was significant familiarisation within three sessions for all parameters except for errors, which took four sessions before a significant improvement was seen. In ManipulateDiathermy,
where diathermy must be applied to three discrete nodules, there was no visible learning curve for economy of diathermy. There was no significant difference in performance of right and left hands.
Discussion During the last five years, there has been increasingly widespread recognition that objective measures of surgical performance are desirable and essential9. It has also been recognised that VR simulators can provide these measures4. The present study of psychomotor performance of gynaecologists on MIST 2 clearly supports existing general surgical evidence2,3,10 that MIST provides objective measurement of relevant laparoscopic skills. As in previous studies on MIST VR, the right and left hands perform with equal dexterity in laparoscopic surgery after initial familiarisation. (Chaudhry et al., Left Handed Performance in Right Handed Surgeons, unpublished observations). This is probably due to a vectored reflex arc that is set up across both shoulders, elbows, wrists and hands by the twohanded surgical technique that is advised and taught for laparoscopy. This study has also demonstrated a high-quality user – computer interface allowing rapid familiarisation with the system and with the tasks. Similar early learning curves were demonstrated for both the gynaecology tasks and the original general tasks, suggesting that the data presented are reliable. Of the nine participants who failed to complete one or more of the tasks on the first session, eight were able to complete the tasks on the second session. This suggests a marked improvement in time taken from the first to the second session and thus supports the existence of early familiarisation. The results for all four sessions (Fig. 6, Table 2) demonstrated that further familiarisation with MIST 2 was complete by the fourth session for most D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
MEDICAL EDUCATION
185
Fig. 6. Mean scores plotted over four sessions for all participants, and for separate groups of gynaecologist. Lower values indicate better performance. All ; senior house officers ; senior gynaecologists .
D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
186
MEDICAL EDUCATION
Table 2. Friedman analysis of learning rate with multiple comparison of consecutive sessions. Task
Time (P)
Economy of movement (P)
Errors (P)
Economy of diathermy (P)
TR Session Session Session Session Session Session
1 1 1 2 2 3
vs vs vs vs vs vs
2 3 4 3 4 4
<0.0001 ns <0.001 <0.001 <0.05 ns ns
<0.0001 ns <0.001 <0.001 ns ns ns
0.0001 ns <0.001 <0.01 ns ns ns
– – – – – – –
MD Session Session Session Session Session Session
1 1 1 2 2 3
vs vs vs vs vs vs
2 3 4 3 4 4
<0.0001 ns <0.001 <0.001 ns <0.01 ns
<0.0001 ns <0.01 <0.001 ns <0.05 ns
<0.001 ns <0.05 <0.001 ns <0.01 ns
ns ns ns ns ns ns ns
SC Session Session Session Session Session Session
1 1 1 2 2 3
vs vs vs vs vs vs
2 3 4 3 4 4
<0.0001 ns <0.001 <0.001 ns ns ns
<0.0001 ns <0.01 <0.001 ns ns ns
ns ns ns ns ns ns ns
– – – – – – –
SD Session Session Session Session Session Session
1 1 1 2 2 3
vs vs vs vs vs vs
2 3 4 3 4 4
<0.0001 ns <0.001 <0.001 <0.05 <0.01 ns
<0.0001 ns <0.001 <0.001 ns <0.01 ns
<0.0011 ns ns <0.01 ns <0.01 ns
<0.0005 ns <0.05 <0.001 ns ns ns
TR ¼ Transversal Task, MD ¼ ManipulateDiathermy Task, SC ¼ StretchClip Task, SD ¼ StretchDiathermy Task.
participants, suggesting that further sessions on MIST 2 would have resulted in refinement of the skills. This fits with the two-part learning model described by Mitta and Packebush7 and is paralleled by findings on general surgeons using MIST2. It was not possible to record performance data for those tasks in which participants ran out of time due to the 10-minute cutoff inherent in the MIST system. Nine of the 21 volunteers timed out on at least one task. This was surprising as the experience of one author (RFM) of testing more than 100 medical students and general surgeons is that less than 5% of subjects studied on MIST are unable to complete tasks within a maximum of 10 minutes. It is possible that these subjects failed to understand what was required to perform on this VR simulator, or were distracted from the tasks by a lack of computer literacy or computer anxiety4,11. However, all participants received a standardised introductory explanation of the simulator and of the tasks. It is more likely that they did not have sufficient psychomotor skills to complete the tasks. Perhaps this highlights the limitations of apprenticeship training. Maybe gynaecologists perform adequate numbers of laparoscopic sterilisation operations from an early stage in their career, but do not have sufficient training
opportunity for training in other operations involving more complex psychomotor skills. It is important, however, to have a range of skills in order that complications be handled effectively. Objective assessment of psychomotor skills might ensure that only those with a basic level of skill enter clinical training in laparoscopic surgery. An assessment tool such as MIST could be used for this type of ‘aptitude’ testing2. This type of summative assessment might also allow more standardisation and structuring of training across the country. A VR trainer allows trainees to repeatedly practice psychomotor skills alone and to obtain structured feedback away from the operating theatre. The Royal College of Obstetricians and Gynaecologists has recommended that surgical training systems such as MIST be evaluated, piloted and introduced into Basic Surgical Skills Courses12.
Conclusion Objective unbiased assessment of laparoscopic performance is a much needed and valuable component of assessment of competence of gynaecologists. Part-task VR D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
MEDICAL EDUCATION
simulators such as MIST assess relevant psychomotor skills consistently and reliably. The rapid initial familiarisation curve consistently demonstrated by individuals in this study indicates a highquality interface and suggests that the most appropriate time to assess performance is on the fourth session when the initial familiarisation process has been overcome.
Acknowledgements The authors would like to thank Christopher Sutton and Jim Gant for continued support for the software development of MIST.
4. 5.
6.
7.
8.
9.
References 1. Cuschieri A. Reflections on surgical training. Surg Endosc 1993;7: 73 – 74. 2. Chaudhry A, Sutton C, Wood J, Stone R, McCloy R. Learning rate for laparoscopic surgical skills on MIST-VR, a virtual reality simulator: quality of human computer interface. Ann R Coll Surg Engl 1999;81:281 – 286. 3. Taffinder N, Sutton C, Fishwick RJ, Mannus IC, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparo-
D RCOG 2003 Br J Obstet Gynaecol 110, pp. 181 – 187
10.
11. 12.
187
scopic surgery: results from randomised controlled studies using MIST VR laparoscopic simulator. Br J Surg 1998;85(Suppl 1):75. McCloy RF, Stone R. Virtual reality in surgery. BMJ 2001;323: 912 – 915. Buerger T, Erdtmann M. Inevitable developments for VR training systems —the ESI perspective. The Ninth Annual Medicine Meets Virtual Reality Conference. Newport Beach, California, USA, January 2001. Wilson MS, Middlebrook A, Sutton C, Stone R, McCloy RF. MIST VR: a virtual reality trainer for laparoscopic surgery assesses performance. Ann R Coll Surg Engl 1997;79:403 – 404. Mitta DA, Packebush SJ. Improving interface quality: an investigation of human computer interaction and task learning. Ergonomics 1995;38:1307 – 1325. Mathews JNS, Altman Douglas G, Campbell MJ, Royston P. Analysis of serial measurements in medical research. BMJ 1990;300: 230 – 235. Taffinder NJ, McManus IC, Gul Y, Russell RCG, Darzi A. Effect of sleep deprivation on surgeons’ dexterity on laparoscopic simulator. Lancet 1998;352:1191. Gallagher AG. Objective computer-based assessment of perceptual skill that predicts laparoscopic performance in three separate studies. Ninth Annual Medicine Meets Virtual Reality Conference. Newport Beach, California, 2001. Ray NM, Minch RP. Computer anxiety and alienation: towards a definition and parsimonious measure. Hum Factors 1990;32:477 – 491. Report of an RCOG Working Party. Discussion document on further training for doctors in difficulty. R Coll Obstet Gynaecol April 2002.
Accepted 7 August 2002