- Email: [email protected]
Endoscopic Sinus Surgery Simulator as a teaching tool for anatomy education
The American Journal of Surgery (2008) 196, 120 –124
Surgical Education
Endoscopic Sinus Surgery Simulator as a teaching tool for anatomy education Alla Solyar, M.D.a,*, Hernando Cuellar, M.D.b, Babak Sadoughi, M.D.b, Todd R. Olson, Ph.D.c, Marvin P. Fried, M.D., F.A.C.S.a a
Department of Otolaryngology, Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY, USA; Surgical Simulation Center, Department of Otolaryngology, Montefiore Medical Center, Bronx, NY, USA; cDepartment of Anatomy and Structural biology, Albert Einstein College of Medicine, Bronx, NY, USA b
KEYWORDS: Virtual reality; Anatomy education; Surgical simulation
Abstract BACKGROUND: Virtual reality simulators provide an effective learning environment and are widely used. This study evaluated the Endoscopic Sinus Surgery Simulator (ES3; Lockheed Martin) as a tool for anatomic education. METHODS: Two medical student groups (experimental, n ⫽ 8; control, n ⫽ 7) studied paranasal sinus anatomy using either the simulator or textbooks. Their knowledge was then tested on the identification of anatomic structures on a view of the nasal cavities. RESULTS: The mean scores were 9.4 ⫾ 0.5 and 5.1 ⫾ 3.0 out of 10 for the simulator and textbook groups, respectively (P ⫽ .009). Moreover, the simulator group completed the test in a significantly shorter time, 5.9 ⫾ 1.1 versus 8.3 ⫾ 2.0 minutes (P ⫽ .021). A survey asking the students to rate their respective study modality did not materialize significant differences. CONCLUSION: The ES3 can be an effective tool in teaching sinonasal anatomy. This study may help shape the future of anatomic education and the development of modern educational tools. © 2008 Elsevier Inc. All rights reserved.
Virtual reality is an important tool for medical and surgical training as well as education.1 The Endoscopic Sinus Surgery Simulator (ES3; Lockheed Martin, Bethesda, MD, USA), a virtual reality device developed to train individuals on a virtual patient,2,3 has shown promising results in improving resident skills in sinus surgery.4 It also may be a valuable educational resource in the study of anatomy. This study evaluates the ES3 as an anatomy education tool for medical students. Over the last few decades, the anatomy curriculum in the US medical schools has changed with less time and fewer * Corresponding author. Tel.: ⫹1-917-306-7548; fax: ⫹1-718-4059014. E-mail address: [email protected] Manuscript received April 16, 2007; revised manuscript June 22, 2007
0002-9610/$ - see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.amjsurg.2007.06.026
trained anatomy faculty. Today, because medical schools are cutting back on anatomic education, many training programs believe that incoming residents are not adequately prepared in gross anatomy. According to Cottam,5 who surveyed over 1,000 residency programs in family practice, diagnostic radiology, emergency medicine, and general surgery, 57% of residency programs indicated that incoming residents needed a refresher, whereas 14% reported that they were seriously lacking in the gross anatomic knowledge. Furthermore, 52% of general surgery residency programs surveyed by Cottam felt that residents in 1999 were less prepared than they were in 1989. Thus, the continued devaluing of anatomic education in the curriculum of medical schools since Cottam’s study makes it more important than ever to implement enhanced methods of anatomy in-
A. Solyar et al.
Endoscopic sinus surgery simulator
struction. Virtual reality simulators are tools that can be used to increase the efficiency and effectiveness of anatomic instruction at all levels of education. As novel training and teaching vehicles, they have gained much attention as effective modalities for surgical resident education. However, they can also be used to augment and enrich the teaching of anatomy because the hours devoted to the subject continue to be reduced, the number of trained anatomists decreases, and cadaver costs and scarcity arise. Virtual reality applications provide a 3-dimensional educational experience in which dynamic views of structures and their spatial relationships can be explored. There is a developing belief among educators that virtual reality tools are both engaging and efficacious in medical student education.6 Hoffman and Vu6 comment that new virtual reality systems allow training “without the risks and ethical concerns typically associated with using animal and human subjects.” They add that “those simulated encounters, in combination with existing opportunities to work with real patients, could increase the depth and breadth of learners’ exposure to medical problems, ensure uniformity of training experiences, and enhance the acquisition of clinical skills”.6 To further promote the use of simulators in anatomical education, Hariri et al7 evaluated a surgical simulator as a tool for learning clinical anatomy by testing and comparing shoulder anatomy knowledge in 2 groups of students who used either the simulator or textbooks to study anatomy. Their results indicate that the 2 groups were comparable in the knowledge of shoulder joint anatomy. Furthermore, students rated the simulator a more effective learning tool over the textbook and indicated that they were more likely to use the simulator as a learning tool if it were available to them.7 Although students in the simulator group performed comparably to students in the textbook group, both groups scored poor marks on the shoulder joint anatomy testing. The Endoscopic Sinus Surgery Simulator (ES3), a virtual reality device developed by Lockheed Martin, was evaluated as a teaching and training tool. This computer-based device uses acquired computed tomography scan data to simulate sinus imagery and offers haptic force feedback in a 3-dimensional field.3,8 Endoscopic surgical tools are incorporated, and manipulation and dissection can be performed in a virtual setting. The simulator has 3 levels of difficulty: novice, intermediate, and advanced. The novice setting allows basic skills to be developed, whereas the intermediate setting adds paranasal sinus anatomy. The advanced level setting also uses virtual anatomy but unlike the intermediate level does not label anatomic landmarks. The present study evaluated the efficacy of the ES3 in teaching sinus anatomy to junior medical students.
121 of the 2004 anatomy class. Students in the class of 2008 were randomly assigned to either the experimental (simulator) group or the control (textbook) group. Two of the 9 students in the control group did not complete the study. Enrolled students from both groups were invited to a short introductory lecture where they viewed a video describing the ES3 and were given reading materials on sinus anatomy, consisting of chapters “Sinus Anatomy and Function”9 and “Endoscopic Sinus Surgery”10 from Bailey’s Head and Neck Surgery-Otolaryngology as well as images from the 2nd edition of Netter’s Atlas of Human Anatomy.11 Both groups then took a pretest consisting of identifications of anatomic structures from textbook images to establish a knowledge baseline. The purpose of this pretest was to establish that both groups were at the same level of knowledge about the anatomy of the paranasal sinuses. Both groups viewed an introductory video of the nasal cavity taken during an endoscopic procedure. Students in the control group were asked to study paranasal sinus anatomy from reading materials given previously as well as images from the 4th edition of Rohen’s Color Atlas of Anatomy,12 Stammberger’s Functional Endoscopic Sinus Surgery,13 Weir’s Imaging Atlas of Human Anatomy,14 and Atlas of Endoscopic Sinonasal Surgery.15 No limit to study time was imposed on the control group, and they were allowed to study on their own just up to the time the simulator group completed its training. Students in the simulator group were asked not to use any other study materials and were given supplementary reading material covering step-by-step description of the ES3 modes and description of tools used in the ES3. The simulator group then underwent training on the ES3 with each student having three 1-hour training sessions using the intermediate setting on the simulator (between 6-10 trials total). In this setting, users were asked to navigate through a series of hoops from the nasal vestibule to the nasopharynx and middle meatus, inject a vasoconstrictive agent into the middle turbinate in the first 3 trials, and perform a limited dissection of the proctor-defined structures such as the uncinate process, the ethmoid bulla, and the maxillary ostium. Anatomical structures were clearly labeled. After completion of the intermediate mode, students proceeded to the advanced setting where they performed 3 trials. The advanced setting is similar to the intermediate but lacks labeled anatomic landmarks (Fig. 1).
Materials and Methods After approval of the study by the Albert Einstein College of Medicine Institutional Review Board, 17 first-year medical students were recruited through electronic mail before the start
Figure 1
Study outline of simulator vs. control groups.
122
Figure 2 pre-test.
The American Journal of Surgery, Vol 196, No 1, July 2008
Simulator and control groups’ performance on the
Figure 4 Comparison between simulator and control groups on the time to complete the post-training test.
Results After the study group completed the ES3 training, both groups underwent a second nasal cavity endoscopic view videotape session. In these individual sessions, students were asked to identify the following structures: inferior turbinate, middle turbinate (before and after dissection), nasal septum, maxillary ostium, maxillary sinus, uncinate process, ethmoid bulla, nasopharynx, and Eustachian tube. Each session was scored on the number of structures correctly identified and time to completion. After completion of the test, students were asked to fill out 2 surveys. One survey questioned the extent of their studying. The second assessed students’ satisfaction level with their study modality on a Likert scale. In the latter survey, the students were also asked to grade the effectiveness of the study modality as a tool for learning anatomy, likelihood of future use if modality was available, ease of use, ease of learning, and image realism. Statistical analysis was performed by using a 2-tailed Student t test (SPSS 11.0 software; SPSS Inc, Chicago, IL).
Figure 3 Simulator and control groups’ performance on the post-training test.
The simulator and the control group performed comparably on the pretest, scoring a mean of 6.9 ⫾ 1.2 and 6.7 ⫾ 1.5 correct answers out of 8, respectively (P ⫽ .749, Fig. 2), thus showing that the groups had similar baselines at the beginning of the experiment. During the videotape testing sessions, the simulator group performed significantly better than the control group, scoring a mean of 9.4 ⫾ 0.5 correct identifications out of 10, whereas the control group answered 5.1 ⫾ 3.0 questions out of 10 correctly. This resulted in a statistically significant difference between the 2 groups (P ⫽ .009, Fig. 3). These results also show that students in the simulator group performed more uniformly, whereas students in the control group had more scattered results. In addition to better performance during the videotape sessions, the students in the simulator group completed the test in a significantly shorter period of time, taking a mean of 5.9 ⫾ 1.1 minutes, whereas the control group needed a mean of 8.3 ⫾ 2.0 minutes to complete the test (P ⫽ .021, Fig. 4). On average, the control group spent 78 minutes studying from the given materials before testing and used 68% of the given materials. The results of the Likert scale are shown in Table 1, encompassing mean results for students’ rating of the effectiveness of the study modality assigned to their group. The Likert scale ranges from 1 to 5, with 1 being the most negative response and 5 the most positive response. Students in the subject and control groups rated the effectiveness of their study modality to teach anatomy as 4.3 ⫾ 0.7 and 3.6 ⫾ 1.0 (P ⫽ .143), respectively. Students in the simulator group rated the likelihood of use of modality, if available, as 4.3 ⫾ 0.9, whereas students in the textbook group rated it at 3.9 ⫾ 0.7 (P ⫽ .361). The simulator group ranked ease of use at 2.9 ⫾ 1.5, whereas the textbook-trained group reported ease of use for their study modality as 4.0 ⫾ 0.6 (P ⫽ .079). Ease of learning was rated at 3.9 ⫾ 1.4 by the simulator group, whereas the textbook group rated it at 3.7 ⫾ 0.8 (P ⫽ .786). Finally, the simulator was rated as 3.9 ⫾ 1.0 for image realism, whereas the textbook method was rated at 3.4 ⫾ 1.0 (P ⫽ .397).
A. Solyar et al. Table 1
Endoscopic sinus surgery simulator
123
Mean score on the questionnaire (Likert scale responses: 1 ⫽ extremely negative, 5 ⫽ extremely positive)
Simulator-trained group Textbook-trained group P value
n
Ability to teach anatomy
Likelihood of use of modality if available
8 7
4.3 ⫾ 0.7 3.6 ⫾ 1.0 .143
4.3 ⫾ 0.9 3.9 ⫾ 0.7 .361
Comments The results of this study show that the ES3 is a remarkably effective tool in helping medical students learn nasal and paranasal sinus anatomy. Indeed, although both groups started out with similar levels of knowledge as shown by the comparable results of the pretest, students in the simulator group performed significantly better at the end of the study when they were asked to identify anatomic structures on the videotape. They also had less variability in the number of correct identifications within the group and were able to complete the test in a significantly shorter period of time. The results of the Likert survey, asking students to rate the study modality, were unexpected in that students in the simulator-trained group and the textbook-trained group gave similar ratings to the simulator and textbook methods. The differences between the 2 groups’ responses were not significant. Students in the simulator group found their study modality more difficult to use, giving it a mean score of 2.9 ⫾ 1.5, whereas students in the textbook-trained group rated the ease of use of their modality at a mean score of 4.0 ⫾ 0.6. Even though the P value (.079) for ease of use did not reach statistical significance, it came closer to the cutoff value than in any other category. The reason for a lower than expected rating of the simulator as a study modality may lie in the fact that students in the subject group were asked to learn the anatomy while performing endoscopic sinus surgery on the simulator, a task that entails a frustrating learning curve. The students assigned to the simulator group were actually learning much more than anatomy. Similarly, the advanced level of the ES3 is a very challenging task for novices, adding to further frustration and dislike of the modality, although the acceptance of the simulator does not seem compromised. One of our prior studies had clearly showed that, in general, medical students viewed a device such as the ES3 very positively as an addition to their educational tools.16 One of the limitations of this study was the lack of control over the amount of studying done by the textbook group. It was decided not to control textbook group study time because of methodologic and logistic issues involved in monitoring students during 4 hours of study (approximate amount of time spent by experimental group with the simulator). The textbook group reported having studied for 78 minutes in average, ranging from 20 minutes to 2.5 hours, and used on average 68% of the given materials. The amount of time used to study did not correlate with test
Ease of use
Ease of learning
Image realism
2.9 ⫾ 1.5 4.0 ⫾ 0.6 .079
3.9 ⫾ 1.4 3.7 ⫾ 0.8 .786
3.9 ⫾ 1.0 3.4 ⫾ 1.0 .397
performance. The amount of learning time also posed a dilemma in the Hariri et al7 study in which students were controlled for time, with both simulator and textbook groups being given 10 minutes to study anatomy from their respective modalities. This may have been responsible for low scores in both groups. As Hariri et al7 pointed out, it might have been helpful to increase the duration of the study sessions or to implement multiple teaching sessions in their study. However, the concerns on the reliability of the outcome measurements are quite typical in this type of study design and often impossible to address. As pointed out by Nicholson et al17 who recently published one of the few other randomized controlled trials of a virtual reality tool for anatomy education, “the novelty of the 3-D model may have encouraged the intervention group to spend more time and to concentrate more on the material,” but “one can argue that, either way, the outcome is the same: better understanding of 3-D anatomical relationships.” We also believe that our study could have benefited from time adjustments. Instead of four 1-hour sessions, students in the experimental group could be given one 40-minute session, and, instead of performing actual dissection, they could spend the time exploring the endoscopic view of the nasal cavities while familiarizing themselves with the paranasal anatomy with no or only limited dissection. Nnodim investigated learning of lower limb anatomy using a control group that studied via dissection and an experimental group that studied previously dissected specimens. On the paper and practical examinations that followed, the experimental group performed significantly better, thus suggesting that one does not need to dissect to perform well on this type of tag exam.18 One may hypothesize from Nnodim’s study that the inclusion of simulator-based anatomy learning exercises might be a viable alternative to dissection as a basic anatomic learning tool because student performance on tag examinations did not seem to benefit from actual participation in dissection. Subsequent changes to the experimental design that limits pretest study time from textbooks and eliminates cadaver-based study from the preparation of the experimental group may produce data that could be useful to test these hypotheses. Another advantage of such an adjustment in study design is a better “ease of use” score on the Likert scale questionnaire of the simulator-trained group. The simulator environment can more closely reflect not only cadaveric dissection but also applied clinical procedures.
124
The American Journal of Surgery, Vol 196, No 1, July 2008
Few studies of this nature have been done in the past. The most recent and most similar is that of Hariri et al,7 which showed that a surgical simulator is at least as effective as textbook images for learning anatomy and suggested that student learning was enhanced through increased motivation as shown in their Likert scale results. Our study, which was very similar in structure, showed that the endoscopic sinus surgery simulator was a more effective tool for learning paranasal sinus anatomy than the textbook images. The main difference between the 2 studies, and possibly a reason for the difference in the results, lay with the amount of time allocated for anatomy study. Alternatively, the fact that students in our experimental group performed actual dissection, even though they found it hard, may have helped them score better on the test. This, however, directly contradicts the findings in Nnodim’s study, which showed that one does not need to dissect anatomic structures to score well on laboratory tag examinations.18 In addition, the similarity of the images displayed on the simulator to the actual anatomic views used for the final test may be perceived as an advantage for the experimental group on the final test. However, during the study portion of the protocol, the control group had access to atlas images of live endocopic surgery that replicated the final test even more realistically. Future studies should examine how the amount of time spent studying anatomy on the simulator influences test scores and attempt to better define the role dissection plays in the simulator group performance on the final test. Our observations, nonetheless, strongly support the benefit of simulation technologies for anatomy education, and their potential contribution should further motivate medical educators to rapidly incorporate virtual reality simulation into their curricula.
develop better graphics, evolve to more fully reflect normal human variation and pathologies, and become more costeffective. The value and practical applications of these devices in both surgical dissection instruction and basic anatomic training will almost certainly increase over the coming years. Further investigation of simulator use in conjunction with other learning modalities will help shape the future of both anatomic education and the development of virtual reality simulators.20
Conclusion Our findings suggest that the ES3 is an effective tool for teaching paranasal sinus anatomy, as shown with the students in the simulator group performing significantly better on the final test of identification of anatomic structures and time to completion when compared with the students in the textbook group. Moreover, clinical application of the anatomy is also taught. Students in both groups gave similar marks to both modalities, with no significant differences between the groups’ responses with respect to ability to teach anatomy, likelihood of future use, and overall ease of use. Changes in medical curricula continue to diminish the importance placed on anatomic education as measured by the amount of time being allocated for its teaching. This together with the national shortage of trained anatomists19 is leading many schools to investigate alternatives to the traditional methods of teaching anatomy. We have no doubt that virtual reality simulators will find an important place in teaching anatomy to both surgical residents and medical students. This will be increasingly possible as simulators
References 1. Caversaccio M, Eichenberger A, Hausler R. Virtual simulator as a training tool for endonasal surgery. Am J Rhinol 2003;17:283–90. 2. Uribe JI, Ralph WM Jr, Glaser AY, Fried MP. Learning curves, acquisition, and retention of skills trained with the endoscopic sinus surgery simulator. Am J Rhinol 2004;18:87–92. 3. Edmond CV Jr, Heskamp D, Sluis D, et al. ENT endoscopic surgical training simulator. Stud Health Technol Inform 1997;39:518 –28. 4. Edmond CV Jr. Impact of the endoscopic sinus surgical simulator on operating room performance. Laryngoscope 2002;112:1148 –58. 5. Cottam WW. Adequacy of medical school gross anatomy education as perceived by certain postgraduate residency programs and anatomy course directors. Clin Anat 1999;12:55– 65. 6. Hoffman H, Vu D. Virtual reality: teaching tool of the twenty-first century? Acad Med 1997;72:1076 – 81. 7. Hariri S, Rawn C, Srivastava S, et al. Evaluation of a surgical simulator for learning clinical anatomy. Med Educ 2004;38:896 –902. 8. Ackerman MJ, Spitzer VM, Scherzinger AL, Whitlock DG. The visible human data set: an image resource for anatomical visualization. Medinfo 1995;8:1195– 8. 9. Miller AJ, Amedee R. Sinus anatomy and function. In: Bailey BJ (ed): Head and Neck Surgery–Otolaryngology (ed 2). Philadelphia, PA, Lippincott-Raven Publishers, 1998. 10. Lanza DC, Kennedy DW. Endoscopic sinus surgery. In: Bailey BJ (ed): Head and Neck Surgery–Otolaryngology (ed 2). Philadelphia, PA, Lippincott-Raven Publishers, 1998. 11. Netter FH. Atlas of Human Anatomy (ed 2). Icon Learning Systems, A Division of Havas Medi Media, Teterboro, NJ, 1997. 12. Rohen JW, Yokochi C, Lütjen-Drecoll. Color Atlas of Anatomy (ed 4). Philadelphia, PA: F.K. Schattauer Verlagsgesellschaft GmbH and Lippincott Williams & Wilkins, 1998. 13. Stammberger H. Functional Endoscopic Sinus Surgery: The Messerklinger Technique. Philadelphia, PA: Decker, 1991. 14. Weir J, Abrahams PH. Imaging Atlas of Human Anatomy (ed 2). Mosby, 1998. 15. Mehta D. Atlas of Endoscopic Sinonasal Surgery. Philadelphia, PA: Lea & Febiger, 1993. 16. Glaser AY, Hall CB, Uribe SJ, Fried MP. Medical students’ attitudes toward the use of an endoscopic sinus surgery simulator as a training tool. Am J Rhinol 2006;20:177–9. 17. Nicholson DT, Chalk C, Funnell WR, Daniel SJ. Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Med Educ 2006;40:1081–7. 18. Nnodim JO. Learning human anatomy: by dissection or from prosections? Med Educ 1990;24:389 –95. 19. McCuskey RS, Carmichael SW, Kirch DG. The importance of anatomy in health professions education and the shortage of qualified educators. Acad Med 2005;80:349 –51. 20. Marks SC Jr. Recovering the significance of 3-dimensional data in medical education and clinical practice. Clin Anat 2001;14:90 –1.
Surgical Education
Endoscopic Sinus Surgery Simulator as a teaching tool for anatomy education Alla Solyar, M.D.a,*, Hernando Cuellar, M.D.b, Babak Sadoughi, M.D.b, Todd R. Olson, Ph.D.c, Marvin P. Fried, M.D., F.A.C.S.a a
Department of Otolaryngology, Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY, USA; Surgical Simulation Center, Department of Otolaryngology, Montefiore Medical Center, Bronx, NY, USA; cDepartment of Anatomy and Structural biology, Albert Einstein College of Medicine, Bronx, NY, USA b
KEYWORDS: Virtual reality; Anatomy education; Surgical simulation
Abstract BACKGROUND: Virtual reality simulators provide an effective learning environment and are widely used. This study evaluated the Endoscopic Sinus Surgery Simulator (ES3; Lockheed Martin) as a tool for anatomic education. METHODS: Two medical student groups (experimental, n ⫽ 8; control, n ⫽ 7) studied paranasal sinus anatomy using either the simulator or textbooks. Their knowledge was then tested on the identification of anatomic structures on a view of the nasal cavities. RESULTS: The mean scores were 9.4 ⫾ 0.5 and 5.1 ⫾ 3.0 out of 10 for the simulator and textbook groups, respectively (P ⫽ .009). Moreover, the simulator group completed the test in a significantly shorter time, 5.9 ⫾ 1.1 versus 8.3 ⫾ 2.0 minutes (P ⫽ .021). A survey asking the students to rate their respective study modality did not materialize significant differences. CONCLUSION: The ES3 can be an effective tool in teaching sinonasal anatomy. This study may help shape the future of anatomic education and the development of modern educational tools. © 2008 Elsevier Inc. All rights reserved.
Virtual reality is an important tool for medical and surgical training as well as education.1 The Endoscopic Sinus Surgery Simulator (ES3; Lockheed Martin, Bethesda, MD, USA), a virtual reality device developed to train individuals on a virtual patient,2,3 has shown promising results in improving resident skills in sinus surgery.4 It also may be a valuable educational resource in the study of anatomy. This study evaluates the ES3 as an anatomy education tool for medical students. Over the last few decades, the anatomy curriculum in the US medical schools has changed with less time and fewer * Corresponding author. Tel.: ⫹1-917-306-7548; fax: ⫹1-718-4059014. E-mail address: [email protected] Manuscript received April 16, 2007; revised manuscript June 22, 2007
0002-9610/$ - see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.amjsurg.2007.06.026
trained anatomy faculty. Today, because medical schools are cutting back on anatomic education, many training programs believe that incoming residents are not adequately prepared in gross anatomy. According to Cottam,5 who surveyed over 1,000 residency programs in family practice, diagnostic radiology, emergency medicine, and general surgery, 57% of residency programs indicated that incoming residents needed a refresher, whereas 14% reported that they were seriously lacking in the gross anatomic knowledge. Furthermore, 52% of general surgery residency programs surveyed by Cottam felt that residents in 1999 were less prepared than they were in 1989. Thus, the continued devaluing of anatomic education in the curriculum of medical schools since Cottam’s study makes it more important than ever to implement enhanced methods of anatomy in-
A. Solyar et al.
Endoscopic sinus surgery simulator
struction. Virtual reality simulators are tools that can be used to increase the efficiency and effectiveness of anatomic instruction at all levels of education. As novel training and teaching vehicles, they have gained much attention as effective modalities for surgical resident education. However, they can also be used to augment and enrich the teaching of anatomy because the hours devoted to the subject continue to be reduced, the number of trained anatomists decreases, and cadaver costs and scarcity arise. Virtual reality applications provide a 3-dimensional educational experience in which dynamic views of structures and their spatial relationships can be explored. There is a developing belief among educators that virtual reality tools are both engaging and efficacious in medical student education.6 Hoffman and Vu6 comment that new virtual reality systems allow training “without the risks and ethical concerns typically associated with using animal and human subjects.” They add that “those simulated encounters, in combination with existing opportunities to work with real patients, could increase the depth and breadth of learners’ exposure to medical problems, ensure uniformity of training experiences, and enhance the acquisition of clinical skills”.6 To further promote the use of simulators in anatomical education, Hariri et al7 evaluated a surgical simulator as a tool for learning clinical anatomy by testing and comparing shoulder anatomy knowledge in 2 groups of students who used either the simulator or textbooks to study anatomy. Their results indicate that the 2 groups were comparable in the knowledge of shoulder joint anatomy. Furthermore, students rated the simulator a more effective learning tool over the textbook and indicated that they were more likely to use the simulator as a learning tool if it were available to them.7 Although students in the simulator group performed comparably to students in the textbook group, both groups scored poor marks on the shoulder joint anatomy testing. The Endoscopic Sinus Surgery Simulator (ES3), a virtual reality device developed by Lockheed Martin, was evaluated as a teaching and training tool. This computer-based device uses acquired computed tomography scan data to simulate sinus imagery and offers haptic force feedback in a 3-dimensional field.3,8 Endoscopic surgical tools are incorporated, and manipulation and dissection can be performed in a virtual setting. The simulator has 3 levels of difficulty: novice, intermediate, and advanced. The novice setting allows basic skills to be developed, whereas the intermediate setting adds paranasal sinus anatomy. The advanced level setting also uses virtual anatomy but unlike the intermediate level does not label anatomic landmarks. The present study evaluated the efficacy of the ES3 in teaching sinus anatomy to junior medical students.
121 of the 2004 anatomy class. Students in the class of 2008 were randomly assigned to either the experimental (simulator) group or the control (textbook) group. Two of the 9 students in the control group did not complete the study. Enrolled students from both groups were invited to a short introductory lecture where they viewed a video describing the ES3 and were given reading materials on sinus anatomy, consisting of chapters “Sinus Anatomy and Function”9 and “Endoscopic Sinus Surgery”10 from Bailey’s Head and Neck Surgery-Otolaryngology as well as images from the 2nd edition of Netter’s Atlas of Human Anatomy.11 Both groups then took a pretest consisting of identifications of anatomic structures from textbook images to establish a knowledge baseline. The purpose of this pretest was to establish that both groups were at the same level of knowledge about the anatomy of the paranasal sinuses. Both groups viewed an introductory video of the nasal cavity taken during an endoscopic procedure. Students in the control group were asked to study paranasal sinus anatomy from reading materials given previously as well as images from the 4th edition of Rohen’s Color Atlas of Anatomy,12 Stammberger’s Functional Endoscopic Sinus Surgery,13 Weir’s Imaging Atlas of Human Anatomy,14 and Atlas of Endoscopic Sinonasal Surgery.15 No limit to study time was imposed on the control group, and they were allowed to study on their own just up to the time the simulator group completed its training. Students in the simulator group were asked not to use any other study materials and were given supplementary reading material covering step-by-step description of the ES3 modes and description of tools used in the ES3. The simulator group then underwent training on the ES3 with each student having three 1-hour training sessions using the intermediate setting on the simulator (between 6-10 trials total). In this setting, users were asked to navigate through a series of hoops from the nasal vestibule to the nasopharynx and middle meatus, inject a vasoconstrictive agent into the middle turbinate in the first 3 trials, and perform a limited dissection of the proctor-defined structures such as the uncinate process, the ethmoid bulla, and the maxillary ostium. Anatomical structures were clearly labeled. After completion of the intermediate mode, students proceeded to the advanced setting where they performed 3 trials. The advanced setting is similar to the intermediate but lacks labeled anatomic landmarks (Fig. 1).
Materials and Methods After approval of the study by the Albert Einstein College of Medicine Institutional Review Board, 17 first-year medical students were recruited through electronic mail before the start
Figure 1
Study outline of simulator vs. control groups.
122
Figure 2 pre-test.
The American Journal of Surgery, Vol 196, No 1, July 2008
Simulator and control groups’ performance on the
Figure 4 Comparison between simulator and control groups on the time to complete the post-training test.
Results After the study group completed the ES3 training, both groups underwent a second nasal cavity endoscopic view videotape session. In these individual sessions, students were asked to identify the following structures: inferior turbinate, middle turbinate (before and after dissection), nasal septum, maxillary ostium, maxillary sinus, uncinate process, ethmoid bulla, nasopharynx, and Eustachian tube. Each session was scored on the number of structures correctly identified and time to completion. After completion of the test, students were asked to fill out 2 surveys. One survey questioned the extent of their studying. The second assessed students’ satisfaction level with their study modality on a Likert scale. In the latter survey, the students were also asked to grade the effectiveness of the study modality as a tool for learning anatomy, likelihood of future use if modality was available, ease of use, ease of learning, and image realism. Statistical analysis was performed by using a 2-tailed Student t test (SPSS 11.0 software; SPSS Inc, Chicago, IL).
Figure 3 Simulator and control groups’ performance on the post-training test.
The simulator and the control group performed comparably on the pretest, scoring a mean of 6.9 ⫾ 1.2 and 6.7 ⫾ 1.5 correct answers out of 8, respectively (P ⫽ .749, Fig. 2), thus showing that the groups had similar baselines at the beginning of the experiment. During the videotape testing sessions, the simulator group performed significantly better than the control group, scoring a mean of 9.4 ⫾ 0.5 correct identifications out of 10, whereas the control group answered 5.1 ⫾ 3.0 questions out of 10 correctly. This resulted in a statistically significant difference between the 2 groups (P ⫽ .009, Fig. 3). These results also show that students in the simulator group performed more uniformly, whereas students in the control group had more scattered results. In addition to better performance during the videotape sessions, the students in the simulator group completed the test in a significantly shorter period of time, taking a mean of 5.9 ⫾ 1.1 minutes, whereas the control group needed a mean of 8.3 ⫾ 2.0 minutes to complete the test (P ⫽ .021, Fig. 4). On average, the control group spent 78 minutes studying from the given materials before testing and used 68% of the given materials. The results of the Likert scale are shown in Table 1, encompassing mean results for students’ rating of the effectiveness of the study modality assigned to their group. The Likert scale ranges from 1 to 5, with 1 being the most negative response and 5 the most positive response. Students in the subject and control groups rated the effectiveness of their study modality to teach anatomy as 4.3 ⫾ 0.7 and 3.6 ⫾ 1.0 (P ⫽ .143), respectively. Students in the simulator group rated the likelihood of use of modality, if available, as 4.3 ⫾ 0.9, whereas students in the textbook group rated it at 3.9 ⫾ 0.7 (P ⫽ .361). The simulator group ranked ease of use at 2.9 ⫾ 1.5, whereas the textbook-trained group reported ease of use for their study modality as 4.0 ⫾ 0.6 (P ⫽ .079). Ease of learning was rated at 3.9 ⫾ 1.4 by the simulator group, whereas the textbook group rated it at 3.7 ⫾ 0.8 (P ⫽ .786). Finally, the simulator was rated as 3.9 ⫾ 1.0 for image realism, whereas the textbook method was rated at 3.4 ⫾ 1.0 (P ⫽ .397).
A. Solyar et al. Table 1
Endoscopic sinus surgery simulator
123
Mean score on the questionnaire (Likert scale responses: 1 ⫽ extremely negative, 5 ⫽ extremely positive)
Simulator-trained group Textbook-trained group P value
n
Ability to teach anatomy
Likelihood of use of modality if available
8 7
4.3 ⫾ 0.7 3.6 ⫾ 1.0 .143
4.3 ⫾ 0.9 3.9 ⫾ 0.7 .361
Comments The results of this study show that the ES3 is a remarkably effective tool in helping medical students learn nasal and paranasal sinus anatomy. Indeed, although both groups started out with similar levels of knowledge as shown by the comparable results of the pretest, students in the simulator group performed significantly better at the end of the study when they were asked to identify anatomic structures on the videotape. They also had less variability in the number of correct identifications within the group and were able to complete the test in a significantly shorter period of time. The results of the Likert survey, asking students to rate the study modality, were unexpected in that students in the simulator-trained group and the textbook-trained group gave similar ratings to the simulator and textbook methods. The differences between the 2 groups’ responses were not significant. Students in the simulator group found their study modality more difficult to use, giving it a mean score of 2.9 ⫾ 1.5, whereas students in the textbook-trained group rated the ease of use of their modality at a mean score of 4.0 ⫾ 0.6. Even though the P value (.079) for ease of use did not reach statistical significance, it came closer to the cutoff value than in any other category. The reason for a lower than expected rating of the simulator as a study modality may lie in the fact that students in the subject group were asked to learn the anatomy while performing endoscopic sinus surgery on the simulator, a task that entails a frustrating learning curve. The students assigned to the simulator group were actually learning much more than anatomy. Similarly, the advanced level of the ES3 is a very challenging task for novices, adding to further frustration and dislike of the modality, although the acceptance of the simulator does not seem compromised. One of our prior studies had clearly showed that, in general, medical students viewed a device such as the ES3 very positively as an addition to their educational tools.16 One of the limitations of this study was the lack of control over the amount of studying done by the textbook group. It was decided not to control textbook group study time because of methodologic and logistic issues involved in monitoring students during 4 hours of study (approximate amount of time spent by experimental group with the simulator). The textbook group reported having studied for 78 minutes in average, ranging from 20 minutes to 2.5 hours, and used on average 68% of the given materials. The amount of time used to study did not correlate with test
Ease of use
Ease of learning
Image realism
2.9 ⫾ 1.5 4.0 ⫾ 0.6 .079
3.9 ⫾ 1.4 3.7 ⫾ 0.8 .786
3.9 ⫾ 1.0 3.4 ⫾ 1.0 .397
performance. The amount of learning time also posed a dilemma in the Hariri et al7 study in which students were controlled for time, with both simulator and textbook groups being given 10 minutes to study anatomy from their respective modalities. This may have been responsible for low scores in both groups. As Hariri et al7 pointed out, it might have been helpful to increase the duration of the study sessions or to implement multiple teaching sessions in their study. However, the concerns on the reliability of the outcome measurements are quite typical in this type of study design and often impossible to address. As pointed out by Nicholson et al17 who recently published one of the few other randomized controlled trials of a virtual reality tool for anatomy education, “the novelty of the 3-D model may have encouraged the intervention group to spend more time and to concentrate more on the material,” but “one can argue that, either way, the outcome is the same: better understanding of 3-D anatomical relationships.” We also believe that our study could have benefited from time adjustments. Instead of four 1-hour sessions, students in the experimental group could be given one 40-minute session, and, instead of performing actual dissection, they could spend the time exploring the endoscopic view of the nasal cavities while familiarizing themselves with the paranasal anatomy with no or only limited dissection. Nnodim investigated learning of lower limb anatomy using a control group that studied via dissection and an experimental group that studied previously dissected specimens. On the paper and practical examinations that followed, the experimental group performed significantly better, thus suggesting that one does not need to dissect to perform well on this type of tag exam.18 One may hypothesize from Nnodim’s study that the inclusion of simulator-based anatomy learning exercises might be a viable alternative to dissection as a basic anatomic learning tool because student performance on tag examinations did not seem to benefit from actual participation in dissection. Subsequent changes to the experimental design that limits pretest study time from textbooks and eliminates cadaver-based study from the preparation of the experimental group may produce data that could be useful to test these hypotheses. Another advantage of such an adjustment in study design is a better “ease of use” score on the Likert scale questionnaire of the simulator-trained group. The simulator environment can more closely reflect not only cadaveric dissection but also applied clinical procedures.
124
The American Journal of Surgery, Vol 196, No 1, July 2008
Few studies of this nature have been done in the past. The most recent and most similar is that of Hariri et al,7 which showed that a surgical simulator is at least as effective as textbook images for learning anatomy and suggested that student learning was enhanced through increased motivation as shown in their Likert scale results. Our study, which was very similar in structure, showed that the endoscopic sinus surgery simulator was a more effective tool for learning paranasal sinus anatomy than the textbook images. The main difference between the 2 studies, and possibly a reason for the difference in the results, lay with the amount of time allocated for anatomy study. Alternatively, the fact that students in our experimental group performed actual dissection, even though they found it hard, may have helped them score better on the test. This, however, directly contradicts the findings in Nnodim’s study, which showed that one does not need to dissect anatomic structures to score well on laboratory tag examinations.18 In addition, the similarity of the images displayed on the simulator to the actual anatomic views used for the final test may be perceived as an advantage for the experimental group on the final test. However, during the study portion of the protocol, the control group had access to atlas images of live endocopic surgery that replicated the final test even more realistically. Future studies should examine how the amount of time spent studying anatomy on the simulator influences test scores and attempt to better define the role dissection plays in the simulator group performance on the final test. Our observations, nonetheless, strongly support the benefit of simulation technologies for anatomy education, and their potential contribution should further motivate medical educators to rapidly incorporate virtual reality simulation into their curricula.
develop better graphics, evolve to more fully reflect normal human variation and pathologies, and become more costeffective. The value and practical applications of these devices in both surgical dissection instruction and basic anatomic training will almost certainly increase over the coming years. Further investigation of simulator use in conjunction with other learning modalities will help shape the future of both anatomic education and the development of virtual reality simulators.20
Conclusion Our findings suggest that the ES3 is an effective tool for teaching paranasal sinus anatomy, as shown with the students in the simulator group performing significantly better on the final test of identification of anatomic structures and time to completion when compared with the students in the textbook group. Moreover, clinical application of the anatomy is also taught. Students in both groups gave similar marks to both modalities, with no significant differences between the groups’ responses with respect to ability to teach anatomy, likelihood of future use, and overall ease of use. Changes in medical curricula continue to diminish the importance placed on anatomic education as measured by the amount of time being allocated for its teaching. This together with the national shortage of trained anatomists19 is leading many schools to investigate alternatives to the traditional methods of teaching anatomy. We have no doubt that virtual reality simulators will find an important place in teaching anatomy to both surgical residents and medical students. This will be increasingly possible as simulators
References 1. Caversaccio M, Eichenberger A, Hausler R. Virtual simulator as a training tool for endonasal surgery. Am J Rhinol 2003;17:283–90. 2. Uribe JI, Ralph WM Jr, Glaser AY, Fried MP. Learning curves, acquisition, and retention of skills trained with the endoscopic sinus surgery simulator. Am J Rhinol 2004;18:87–92. 3. Edmond CV Jr, Heskamp D, Sluis D, et al. ENT endoscopic surgical training simulator. Stud Health Technol Inform 1997;39:518 –28. 4. Edmond CV Jr. Impact of the endoscopic sinus surgical simulator on operating room performance. Laryngoscope 2002;112:1148 –58. 5. Cottam WW. Adequacy of medical school gross anatomy education as perceived by certain postgraduate residency programs and anatomy course directors. Clin Anat 1999;12:55– 65. 6. Hoffman H, Vu D. Virtual reality: teaching tool of the twenty-first century? Acad Med 1997;72:1076 – 81. 7. Hariri S, Rawn C, Srivastava S, et al. Evaluation of a surgical simulator for learning clinical anatomy. Med Educ 2004;38:896 –902. 8. Ackerman MJ, Spitzer VM, Scherzinger AL, Whitlock DG. The visible human data set: an image resource for anatomical visualization. Medinfo 1995;8:1195– 8. 9. Miller AJ, Amedee R. Sinus anatomy and function. In: Bailey BJ (ed): Head and Neck Surgery–Otolaryngology (ed 2). Philadelphia, PA, Lippincott-Raven Publishers, 1998. 10. Lanza DC, Kennedy DW. Endoscopic sinus surgery. In: Bailey BJ (ed): Head and Neck Surgery–Otolaryngology (ed 2). Philadelphia, PA, Lippincott-Raven Publishers, 1998. 11. Netter FH. Atlas of Human Anatomy (ed 2). Icon Learning Systems, A Division of Havas Medi Media, Teterboro, NJ, 1997. 12. Rohen JW, Yokochi C, Lütjen-Drecoll. Color Atlas of Anatomy (ed 4). Philadelphia, PA: F.K. Schattauer Verlagsgesellschaft GmbH and Lippincott Williams & Wilkins, 1998. 13. Stammberger H. Functional Endoscopic Sinus Surgery: The Messerklinger Technique. Philadelphia, PA: Decker, 1991. 14. Weir J, Abrahams PH. Imaging Atlas of Human Anatomy (ed 2). Mosby, 1998. 15. Mehta D. Atlas of Endoscopic Sinonasal Surgery. Philadelphia, PA: Lea & Febiger, 1993. 16. Glaser AY, Hall CB, Uribe SJ, Fried MP. Medical students’ attitudes toward the use of an endoscopic sinus surgery simulator as a training tool. Am J Rhinol 2006;20:177–9. 17. Nicholson DT, Chalk C, Funnell WR, Daniel SJ. Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Med Educ 2006;40:1081–7. 18. Nnodim JO. Learning human anatomy: by dissection or from prosections? Med Educ 1990;24:389 –95. 19. McCuskey RS, Carmichael SW, Kirch DG. The importance of anatomy in health professions education and the shortage of qualified educators. Acad Med 2005;80:349 –51. 20. Marks SC Jr. Recovering the significance of 3-dimensional data in medical education and clinical practice. Clin Anat 2001;14:90 –1.