- Email: [email protected]
Immersive Virtual Environments for Medical Training
Immersive Virtual Environments for Medical Training Mark W. Bowyer, MD, FACS, Kevin A. Streete, MD, Gilbert M. Muniz, PhD, and Alan V. Liu, PhD Advances in simulation technology are fueling a paradigmatic shift in how medicine will be taught and practiced in the future. Current simulators range from simplified part task trainers to fully immersive virtual environments. We are on the verge of training platforms that provide realistic representations of medical and surgical scenarios that engage learners in a manner that approximates reality. This article reviews the rationale for developing advanced virtual environments and details the technologies that are currently available. Immersive environments using virtual reality, herein reviewed, include Cave Automated Virtual Environments, Distributive Virtual Environments for collaborative learning over the internet (Project TOUCH), Serious Games for medical education (PULSE and 3DiTeams), and a Wide Area Virtual Environment. The ultimate role of these technologies in surgical education remains to be determined but will undoubtedly play an important part in the future. Semin Colon Rectal Surg 19:90-97 © 2008 Elsevier Inc. All rights reserved.
M
edical and surgical knowledge continue to expand rapidly, and surgeons are faced with increasing numbers of surgical procedures that must be learned and mastered. This revolution is occurring against a backdrop in which practitioners are required to become more efficient in patient care, with fewer hours available for teaching and learning. The added pressure of reduced work hours has led to limited options for responding to new disruptive technologies. When a new procedure such as laparoscopic cholecystectomy is introduced, how can large numbers of practicing surgeons and residents in training be trained to be safe and efficient without compromising patient care? The American College of Surgeons has recognized this problem and has endorsed a program of approved regional skills centers that will offer surgeons, surgical residents, and medical students opportunities to acquire and maintain surgical skills, as well as learn new procedures and the use of emerging technologies. The traditional surgical training method of see one, do one, teach one, in and out of the operating room (OR) has recently undergone reappraisal. Studies have shown that, for a variety of diagnostic and therapeutic procedures, clinicians doing this first few to several dozen cases are more likely to
The National Capital Area Medical Simulation Center, Uniformed Services University of the Health Sciences, Bethesda, Maryland. Address reprint requests to: Mark W. Bowyer, MD, FACS, The National Capital Area Medical Simulation Center of the, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, Maryland 20814. E-mail: [email protected]
90
1043-1489/08/$-see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1053/j.scrs.2008.02.005
make a greater number of errors (the learning curve).1 Some might argue that it has become unreasonable that patients be victims of medical invasive procedural training. On-the-job training with patients can result in prolonged invasive procedures, a potential for erroneous diagnoses, increased patient discomfort, and increased risk for procedure-related morbidity.2 In many ways, the OR is a poor classroom for learning surgical skills. By necessity, there are several distractions, most having nothing to do with education, that take priority (patient issues).3 In general, the opportunity is underused.4 The surgical mentor may not be a good teacher. In the OR, the teaching session cannot always be well designed or predicted. The case at hand may not be well suited for the learner. The progress or sequence of the operation cannot be altered to satisfy educational goals. Dissection and exposure cannot be performed for demonstration only. Steps may not be repeated, and the patient cannot be reassembled to start over if failure occurs.4 In addition, fiscal constraints have resulted in pressure to achieve a high turnover in the OR, allowing less time for the attending staff to teach and trainees to practice skills.5 Bridges and Diamond5 have estimated that the annual cost of training chief residents in the OR amounts to more than $53 million per year and suggest that adjunctive training environments that use traditional and virtual teachings aids may alleviate cost over time. In addition to time constraints, one cannot neglect the ethical issues of teaching and learning using patients.6 There are tremendous advantages to training outside the OR. The learning environment is more easily controlled and
Immersive virtual environments for medical training adjusted. The learning situation can be tailored for each student’s needs and can be altered on a minute-by-minute basis to create the desired effect. Perhaps the most valuable part of this training is granting “permission to fail” in a safe environment where there is no risk to patients. Studies have uncovered significant problems with the current surgical education curriculum. These include lack of continuity from undergraduate to graduate surgical education, and the lack of supervision when acquiring physical examination skill, ultimately resulting in poor performance.7-9 An innovative educational tool, the Objective Structured Clinical Examination, has proven useful in the evaluation of the clinical competence of surgical residents.10 Surgical simulators have, perhaps, the best potential to mitigate surgical risk related to the educational process. A surgeon will be able to practice new procedures repeatedly until he or she is judged proficient without endangering patients. The surgeon can also be presented with cases of increasing complexity as his or her skills progress during training. Computer-based surgical simulators offer the potential for including operative cases representing all known anatomic variations. The training program director can use the simulator and its student tracking software to ensure that each graduating resident has seen and dealt with all the pertinent anatomic variations for that surgical specialty.11 Using simulation, mistakes would lose their consequences and become ways to learn. A master surgeon’s trick of the trade or critical maneuver during an operation could be learned in situ by every simulation user. The opportunity to learn something new this way has never before been available to medicine.12 Another potential justification of virtual reality (VR) training is reducing the length of a surgical residency program. Currently, these training programs require five or more years to permit adequate exposure to a variety of technical procedures and decision-making situations. Training programs are currently time limited and not proficiency based. VR training could potentially reduce 5-year residency programs, because residents would not have to wait for clinical cases to appear. Instead, he or she could call up a variety of cases and perform the procedure in VR several times before doing so on a human.13 One of the added attractions of simulation is that training programs might be able to correct for case-mix inequalities, so that what one learns in residency no longer depends only on what comes through the door when on call.12 Flexibility is important for mastery of skills. Simulation may well offer the additional flexibility required. Though currently costly to implement on a large-scale, simulation offers great promise in future reduction of errors (and malpractice suites), reducing (or eliminating) the use of animals, and helping to establish standards for certain procedures. An additional, and perhaps increasingly crucial, role of simulation may well be the assessment of possible decline in the skills of older surgeons. Measuring technical competence through VR could also be applied to older surgeons. As surgeons age, manual dexterity can decline, but it has always been difficult to objectively assess these skills. There is currently no mechanism to determine when these skill levels have deteriorated to the point where the surgeon should not
91 be allowed to operate.13 This decline in skills and judgment has traditionally been assessed by individual surgeons or chiefs of services. A mature, validated system of simulationbased education could offer for the first time a lifelong log of performance on standardized techniques, allowing measurement of skills independent of age or other arbitrary milestones.12 Thomas Russell, the current executive director of the American College of Surgeons, has stated “The competitive surgeon of the next 10 to 20 years will need to possess a different set of skills than we have needed in the past.”14 Dr. Russell has suggested that the use of simulation will provide early exposure to medical students, piquing their interest in a surgical career. Resident education will involve the use of simulators and experiences outside the OR to enhance the core competencies and move the learning process away from the traditional approach of “see one, do one, teach one” to “see one, practice many, and do one.”14 The surgeon of the future will be required to have periodic cognitive testing every few years as well as testing of their technological skills with the use of simulators as they progress in their careers. The acquisition of new surgical skills in practice will be much more structured in the future. The practice of industry-sponsored short courses with rapid introduction into clinical practice will no longer be acceptable. Surgeons will likely be required to undergo retraining in regional centers in which skills can be learned through validated multimodality curriculum. VR allows computer-based models of the real world to be generated and provides humans with a means to interact with theses models though new human– computer interfaces, and, thus to nearly realistically experience these models. The term “Virtual Reality” was initially coined by Jaron Lanier, and in the early 1980s, he founded VPL Research, the first company to sell VR products. In the late 1980s he led the team that developed the first implementations of multi-person virtual worlds using head-mounted displays, for both local and wide area networks, as well as the first “avatars,” or representations of users within such systems. While at VPL, he and his colleagues developed the first implementations of VR applications in surgical simulation. Today, ‘Virtual Reality’ is used in a variety of ways and often in a confusing and misleading manner. Originally, the term referred to “Immersive Virtual Reality.” In immersive VR, the user becomes fully immersed in an artificial, threedimensional world that is completely generated by a computer. VR-simulated environments allow trainees to repeat procedural experiences at their own leisure. These exercises or procedures would otherwise require numerous real-life encounters and costly hours of supervision.15 A commonly recognized type of VR experience is that of flight simulation. In the aerospace, aviation, and defense industries, flight simulation is mandatory before pilots assume flight responsibilities. In addition, flight simulation is regularly used to help commercial airline pilots maintain their skills, or to become familiar with problems they might one day encounter. Virtual Environments were first used by NASA for mission training in 1993.16
M.W. Bowyer et al.
92 Virtual Environments are now in routine use for extra vehicular activity training. The advantage over other methods of training is that they can be manipulated to replicate natural physical phenomena such as the absence of gravity and friction. Shared virtual environments are being used to train international crews for the Space Station. These environments allow training at a much reduced cost but also allow the concept of just-in-time training, allowing rapid response to emergencies by supporting the training of teams whose members may not be co-located.16 Haluck and coworkers in 200117 noted that virtual environments and computer-based simulators, although well-established training tools in other fields, have not been widely incorporated into surgical education. Concerns over the lack of validation, the cost, and finding time for residents to participate were cited as concerns. There are five major areas in medicine where VR is beginning to emerge: (1) assistance before and during medical surgical procedures; (2) medical education and training; (3) medical database visualization; (4) Medical Team Training; and (5) rehabilitation.18 For the most part, the advantages of flight simulators hold equally true for surgical simulators.18 Surgical simulators can provide a concentrated environment that lends itself to learning complex tactile maneuvers in a relatively quick and proficient manner. Moreover, simulation of infrequent but highly hazardous events provides experience in handling these scenarios that may not be available during a period of routine flight or surgical training.19 The ideal surgical simulator should provide the following: it can be customized to the needs of the student, the variety of cases during training increases significantly, and the student can chose to train only the difficult part of the surgery and repeat it as often as necessary.19 Satava18 has described five components that contribute to the realism of a virtual surgical world: fidelity, organ properties, organ reaction, interactivity, and sensory feedback. He predicts that the future holds promise of a virtual cadaver nearly indistinguishable from a real person.18 This concept is referred to as the Turing test, a standard test that means to determine if a computer could be created that responds the way a human would respond such that a human could not tell the difference between the computer and a human.20,21 The VR Turing test would be met if an interrogating human could not tell the virtual human apart from the real human by sight, hearing, or touch, even dissection.19 Current simulators do not yet meet the criteria of the Turing test. It is conceivable that future improvement in computing power and decreased costs of such technology will allow for development of such realism in a virtual environment. That being said, the level of fidelity required to meet the Turing test is likely not necessary to develop useful simulators that will teach useful skills in a validated fashion. In fact, many simulators are currently being used to teach medicine and range from low tech (inexpensive) to increasingly high tech, with corresponding price tags. The future use and development of simulation will depend in large part on validation of their effectiveness as training tools and to a certain extent the adoption of simulation by the various medical and surgical
boards and societies. As organizations and institutions realize the potential cost savings (in dollars and lives) of training with simulation, investment from both private and public sources should follow. Numerous simulators and VR training (mostly part-task) devices are currently available for training surgeons. Some of these are simple and inexpensive, while others are complex and costly. Simulators encompass everything from simple skills trainers such as knot-tying boards to part-task trainers such as a chest tube trainer, up to full procedural trainers that allow for training a complete laparoscopic or endoscopic procedure. What is lacking in many of current modes of training is the highly realistic environment where the trainee actually perspires, the heart rate goes up, and he reacts in a manner that is consistent with what he would do in a real environment. Effective training must also provide the trainee with a medical problem whose clinical solution requires critical thinking combined with diagnostic and skill competence. In addition, effective training needs to be supported by realistic stressors, eg, time limits, physical and sensory distractions, or even overload. It has been shown in several studies that training in a stressful environment similar to what one will likely encounter in the real world is essential for the effective training of medical intervention teams.22-26 As the field of simulation for medical training has matured, there has been increasing efforts to develop more realistic immersive environments. Although by no means comprehensive, the following represents some of the types of Immersive Environments that are currently available or under development for teaching surgeons with a brief discussion of their utility (where applicable) for training.
Physical Re-Creation of the Operating Room Perhaps the earliest use of an “immersive environment” for medical training can be attributed to the pioneering work of David Gaba and his team from Stanford.27 About 20 years ago this team re-created the operating room and used high-fidelity human patient mannequins to teach crisis management to anesthesiologists Figure 1A and B depicts the simulated operating suite at the National Capital Area Medical Simulation Center. The concepts of fight crew resource management training have been used with great success in this type of immersive environment.25 There are several advantages to this type of simulated environment. It is relatively easy to standardize the learning environment. The use of actual equipment used in the operating room allows for good suspension of disbelief. Such encounters are also videotaped and debriefed, which greatly enhances the training exercise. Some of the disadvantages of this type of immersive environment are that it requires the trainee to come to this fixed site, which may be remote from the actual workplace and training is generally limited to individuals or small teams. This type of simulated experience also tends to be labor intensive for the faculty and supporting cast. Although the
Immersive virtual environments for medical training
93
Figure 1 (A) A simulated Operating Room Environment using a High Fidelity Human Patient Simulator. (B) Students interacting with High Fidelity Human Patient Simulator.
scenarios can be altered to depict a variety of cases, there is difficulty in realistically portraying the scenario at other locations such as the prehospital scene, the emergency room, or the ICU environment.
Cave Automatic Virtual Environment (CAVE) The CAVE is an advanced virtual reality system that consists of a 10= ⫻ 10= room where each wall is a stereoscopic 3D screen similar to an IMAX theater. The CAVE was developed at the University of Illinois at Chicago28 and provides the illusion of immersion by projecting stereo images on the walls and floor of a room-sized cube. Several persons wearing lightweight stereo glasses can enter and walk freely inside the CAVE. A head tracking system continuously adjusts the ste-
reo projection to the current position of the leading viewer. A variety of input devices like data gloves, joysticks, and handheld wands allow the user to navigate through a virtual environment and to interact with virtual objects. Directional sound, tactile and force feedback devices, voice recognition, and other technologies can be employed to enrich the immersive experience. The University of Michigan was perhaps the first to recognize the potential of a CAVE or medical training and coupled the immersive environment (an emergency room bay (Fig. 2A)) with a human patient simulator to create a Hyperrich Fully Immersive Virtual Reality medical environment dubbed the Medical Readiness Trainer (Fig. 2B).29 The prototype Medical Readiness Trainer developed by the University of Michigan included the patient bay of an existing emergency room at the University of Michigan Hospital (Fig.
Figure 2 (A) An emergency room bay depicted in a Cave Automated Virtual Environment (CAVE) (Picture courtesy of Klaus-Peter Beier, PhD). (B) The University of Michigan Medical Readiness Trainer using a human patient simulator in a Cave Automated Virtual Environment (picture courtesy of Klaus-Peter Beier, PhD).
M.W. Bowyer et al.
94
Figure 3 (A) Head-injured patient in distributed immersive environment as seen through head-mounted display with operators hand holding syringe depicted in lower right corner. (B) Two students located in geographically remote locations exchanging an instrument as they work collaboratively on the virtual patient.
2A), a generic operating room whose architecture could be arranged to fit the scenario, and a sick bay of a U.S. Coast Guard Cutter. Additional medical training content was provided by a series of bit map flip books to provide short video clips to present scenario relevant ultrasound clips, medical records, x-rays, etc. The result was a novel Hyper-Rich Action Information environment that integrates and presents a wealth of mixed media medical information. The authors felt that the environment allowed students to master the practical clinical skills while providing simultaneous access to a vast body of supporting knowledge. It allowed for training students to manage very difficult scenarios in which the “reality” of life and death outcomes is a direct consequence of the chosen treatment regimen.29 This immersive environment is very engaging for students but is limited currently by the high cost of building such an environment and the ability to only train individuals and small groups at great expense. This limitation is currently being addressed as work is ongoing to create networked immersive environments for collaborative virtual reality, a task considered to be one of the most challenging areas of research in the field.30
Distributed Interactive Virtual Environment: Project TOUCH The Center for Telehealth research team at the University of New Mexico has developed a virtual reality environment that can be distributed over next-generation Internet 2.31 Project TOUCH (Telehealth Outreach for Unified Community Health) enables participants wearing a head-mounted display and a tracker system to be fully immersed within a Virtual Reality Environment. While in the Virtual Reality Environment students can navigate, locomote, and handle objects with a joy-wand control. This environment contains a
virtual patient with whom the students can collaboratively interact in a dynamic, problem-based, clinical, artificial intelligence rules-based fashion. In initial tests of this project, students in Hawaii and New Mexico were able to work collaboratively in problem-solving and managing of a simulated patient with a closed head injury (Fig. 3A), dividing tasks, handing off objects, and functioning as a team (Fig. 3B). This distributed interaction has also been successfully performed between the University of New Mexico and Western Australia. The ability to bring people together as virtual teams for interactive experiential learning and collaborative training, independent of distance, provides a platform for distributed “just-in-time” training, performance assessment, and credentialing. The promise of this technology is evident and the role for future training remains to be determined. In comparison to the CAVE technology described above, it is much cheaper and has the potential of having multiple (unlimited) individuals observe the training real-time as an eye in the sky.
Serious Games Video games have great potential to immerse users in a virtual environment. Such products as SecondLife® (http:// secondlife.com) and Halo 3® (http://www.halo3.com/) are examples of immersive high-fidelity 3D environments in which users can act independently or in collaboration with others. Serious Games are learning platforms that use gaming technology for educational (serious) purposes. Recently the gaming industry has begun to turn their attention to developing “games” for medicine. As the cost for this technology is relatively high, and the potential market relatively low, efforts to date have been limited and largely as a result of congressional and military funding. Two of the more well developed of these efforts are PULSE, a congressionally funded effort developed by Texas A&M University–Corpus Christi and
Immersive virtual environments for medical training
95
Figure 4 (A) A screen shot from the PULSE learning platform of the Intensive Care Unit of the National Naval Medical Center (courtesy of Claudia Johnston, PhD). (B) A screen shot from the PULSE learning platform showing the interaction with a virtual patient (courtesy of Claudia Johnston, PhD).
game developer Breakaway (http://www.breakawayltd.com/); and 3DiTeams, a military-funded effort developed by Duke University and gaming company Virtual Heroes (http://www. virtualheroes.com/). The PULSE project is described as a “state-of-the-art highfidelity first person learning platform that dynamically responds to real-world contextual inputs”—software with the capability to simulate virtually every aspect of the surgical process (http://seriousgamessource.com/features/feature_ 102506_pulse_1.php). The immersive environment in PULSE is a high-fidelity re-creation of the intensive care unit at the National Naval Medical Center in Bethesda, MD (Fig. 4A). Currently the “game” is a first-person learning platform where the learner is able to examine and care for soldiers who have been injured in Iraq and have just arrived from Ger-
many (Fig. 4B). Ultimately the plan is for the platform to be multiplayer such that teams can collaboratively work on the same patient. The platform is currently undergoing validation testing at Yale School of Medicine, the Johns Hopkins School of Medicine, and the National Naval Medical Center. Duke University’s “game,” 3DiTeams, is set in a military Combat Support Field Hospital emergency room (Fig. 5A and B). The situations that are portrayed using virtual patients have been developed to have relevance to both medical and civilian medical teams. The “game” will connect players via a network that allows them to interact in the virtual environment. The main focus of this learning platform is on team training. The potential for serious games to impact health care training is great. Although currently there is little monetary incen-
Figure 5 (A) A screen shot from 3DiTeams showing team interaction with the virtual patient (courtesy of Jeffrey Taekman, MD). (B) A screen shot from 3DiTeams showing the virtual patient and available adjuncts (courtesy of Jeffrey Taekman, MD).
M.W. Bowyer et al.
96
Figure 6 Concept drawings of a 1000-square-foot 18-screen Wide Area Virtual Environment (A). A medical team works collaboratively to triage an treat victims (B), move them to a helicopter (C), and ultimately perform surgical intervention (D), all within this reconfigurable immersive environment (illustrations by Sophia Del Castillo).
tive for gaming companies to develop such games, as this technology matures and studies of efficacy appear, the demand will increase and the availability and fidelity will continue to improve.
Wide Area Virtual Environment (WAVE) The WAVE is based on the aforementioned CAVE technology but represents an evolution beyond this. Unlike the CAVE where users wear head-mounted displays, the WAVE allows team members to interact with each other in a natural fashion wearing lightweight polarized glasses to view the scene. The National Capital Area Medical Simulation Center of the Uniformed Services University has begun construction on what will ultimately be an 18-screen, 1000-square-foot environment (Fig. 6A).32 Using a space of this size, larger teams can be trained using the equipment and other gear, such as pro-
tective clothing within the environment. The environment can take advantage of blended reality by combining virtual patients with high-fidelity human patient simulators, and live simulated patients. Additionally, part task trainers can be used in the environment to perform various procedures (eg, chest tube placement) as part of an ongoing scenario. The environment can also be enriched with appropriate noise, smells, and elements such as smoke. We envision great utility in using the WAVE to train military medical teams with the ability to create multiple environments within the same training scenario. For example, a team can be asked to triage and treat patients on the scene of a HUMVEE that has been struck by an improvised explosive device in Iraq (Fig. 6B). The environment could include virtual patients, mannequins, and live simulated patients and augmented by the sights, sounds, and smells of an ongoing firefight. Once the team performs initial triage and treatment, they then place the patient on a stretcher and carry him through the corridor to
Immersive virtual environments for medical training what is now the inside of a helicopter (Fig. 6C), where ongoing care is provided. While in the helicopter, the original area of the WAVE is reconfigured to be an operating suite where the patient will be delivered and have appropriate interventions (Fig. 6D). The obvious limitation of this type of environment is the required space and initial and ongoing expense of construction and maintenance. The advantages of a training platform like the WAVE is the ability to create realistic representations of mass-casualty and disaster that could not otherwise be accomplished. As envisioned, this technology will also be distributable, such that multiple geographically remote users (from desktop or small CAVEs) could also participate in training and participate (as avatars) in the ongoing scenario. We have built a three-screen prototype in which we are currently developing and testing educational content to refine the final product.
Summary Over the last decade the availability and sophistication of medical and surgical virtual reality simulations and simulators has increased dramatically. We are on the verge of a paradigmatic shift in how medicine will be taught and practiced in the future. Virtual environments promise highly flexible, well-controlled learning situations that can vary from simple tasks to rare and challenging problems. There are a variety of immersive environments available for medical education and the near future is likely to see an explosion in their development and adoption as an adjunct to medical and surgical education.
References 1. Kaufman DM, Bell W: Teaching and assessing clinical skills using virtual reality. Stud Health Technol Inform 39:467-472, 1997 2. Colt HG, Crawford SW, Galbraith O: Virtual reality bronchoscopy simulation: a revolution in procedural training. Chest 120:1333-1339, 2001 3. Haluck RS, Krummel TM: Computers and virtual reality for surgical education in the 21st century. Arch Surg 135:786-792, 2000 4. Scallon SE, Fairholm DJ, Cochrane DD, et al: Evaluation of the operating room as a teaching venue. Can J Surg 35:173-176, 2001 5. Bridges M, Diamond DL: The financial impact of teaching surgical residents in the operating room. Am J Surg 177:28-32, 2001 6. Gates EA: New surgical procedures: can our patients benefit while we learn? Am J Obstet Gynecol 176:1293-1297, 1997 7. Gorman PJ, Meier AH, Krummel TM: Computer assisted training and learning in surgery. Comp Aid Surg 5:120-130, 2000 8. Chalabian J, Garman K, Wallace P, et al: Clinical breast evaluation skills of house officers and students. Am Surg 62:840-845, 1996 9. Endean ED, Sloan DA, Veldenz HC, et al: Performance of the vascular physical examination by residents and medical students. J Vasc Surg 19:149-156, 1994
97 10. Sloan DA, Donnelly MB, Schwartz RW, et al: The objective structured clinical examination. The new gold standard for evaluating postgraduate clinical performance. Ann Surg 222:735-742, 1995 11. Kaufmann CR: Computers in surgical education and the operating room. Ann Chir Gynaecol 90:141-143, 2001 12. Dawson SL: A critical approach to medical simulation. Bull Am Coll Surg 87:12-18, 2002 13. Ota D, Loftin B, Saito T, Lea R, et al: Virtual reality in surgical education. Comput Biol Med 25:127-137, 1995 14. Russell T: From my perspective. Bull Am Coll Surg 88:3-4, 2003 15. Issenberg SB, McGeghie WC, Hart IR, et al: Simulation technology for health care professional skills training and assessment. JAMA 282:861867, 1999 16. Loftin RB, Kenney P: Training the Hubble Space Telescope Flight team. IEEE Comp Graph Appl 15:31-37, 1995 17. Haluck RS, Marshall RL, Krummel TM, et al: Are surgery training programs ready for virtual reality? A survey of program directors in general surgery. J Am Coll Surg 193:660-665, 2001 18. Satava RM: Medical applications of virtual reality. J Med Syst 19:275280, 1995 19. Lange T, Indelicato DJ, Rosen JM: Virtual reality in surgical training. Surg Oncol Clin North Am 9:61-79, 2000 20. Bleich HL, Turing A: The machine, the enigma, and the test. MD Comput 12:330-334, 1995 21. Heiser JF, Colby KM, Faught WS, et al: Can psychiatrists distinguish a computer simulation of paranoia from the real thing? The limitations of Turing-like test measures of the adequacy of simulation. J Psychiatr Res 15:149-162, 1979 22. Small SD, Wuerz RC, Simon R, et al: Demonstration of high fidelity simulation team training for emergency medicine. Acad Emerg Med 6:312-323, 1999 23. Risser DT, Rice MM, Salisbury ML, et al: The potential for improved teamwork to reduce medical errors in the emergency department. Ann Emerg Med 34:373-383, 1999 24. Helmreich RL: On error management: lessons from aviation. BMJ 320: 781-785, 2000 25. Howard SK, Gaba DM, Fish KJ, et al: Anesthesia crisis resource management training: teaching anesthesiologists to handle critical incidents. Aviat Space Environ Med 63:763-770, 1992 26. Holzman RS, Cooper JB, Gaba DM, et al: Anesthesia crisis resource management: real-life simulation training in operating room crises. J Clin Anesth 7:675-687, 1995 27. Gaba DM, Deanda A: A comprehensive anesthesia simulation environment: Re-creating the operating room for research and training. Anesthesiology 69:740-746, 1988 28. Cruz-Neira C, Sandin DJ, DeFanti TA: Surround-screen projection based virtual reality: the design and implementation of the CAVE. Proc ACM SIGGRAPH 135-142, 1993 29. Von Lubitz DKJE, Beier KP, Freer J, et al: Immersive virtual reality platform for medical training: a “killer-application”. Stud Health Technol Inform 70:207-213, 2000 30. Leigh J, Johnson A, Defanti T: CAVERN: a distributed architecture for supporting scalable persistence and intraoperability in collaborative virtual environments. Virtual Real Res Dev Appl 2:217-237, 1997 31. Alverson DC, Saiki SM, Jacobs J, et al: Distributed interactive virtual environments for collaborative experiential learning and training independent of distance over Internet 2. Stud Health Technol Inform 98:712, 2004 32. Lee C, Liu A, Del Castillo S, et al: Towards an immersive virtual environment for medical team training. Stud Health Technol Inform 127: 274-279, 2007
M
edical and surgical knowledge continue to expand rapidly, and surgeons are faced with increasing numbers of surgical procedures that must be learned and mastered. This revolution is occurring against a backdrop in which practitioners are required to become more efficient in patient care, with fewer hours available for teaching and learning. The added pressure of reduced work hours has led to limited options for responding to new disruptive technologies. When a new procedure such as laparoscopic cholecystectomy is introduced, how can large numbers of practicing surgeons and residents in training be trained to be safe and efficient without compromising patient care? The American College of Surgeons has recognized this problem and has endorsed a program of approved regional skills centers that will offer surgeons, surgical residents, and medical students opportunities to acquire and maintain surgical skills, as well as learn new procedures and the use of emerging technologies. The traditional surgical training method of see one, do one, teach one, in and out of the operating room (OR) has recently undergone reappraisal. Studies have shown that, for a variety of diagnostic and therapeutic procedures, clinicians doing this first few to several dozen cases are more likely to
The National Capital Area Medical Simulation Center, Uniformed Services University of the Health Sciences, Bethesda, Maryland. Address reprint requests to: Mark W. Bowyer, MD, FACS, The National Capital Area Medical Simulation Center of the, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, Maryland 20814. E-mail: [email protected]
90
1043-1489/08/$-see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1053/j.scrs.2008.02.005
make a greater number of errors (the learning curve).1 Some might argue that it has become unreasonable that patients be victims of medical invasive procedural training. On-the-job training with patients can result in prolonged invasive procedures, a potential for erroneous diagnoses, increased patient discomfort, and increased risk for procedure-related morbidity.2 In many ways, the OR is a poor classroom for learning surgical skills. By necessity, there are several distractions, most having nothing to do with education, that take priority (patient issues).3 In general, the opportunity is underused.4 The surgical mentor may not be a good teacher. In the OR, the teaching session cannot always be well designed or predicted. The case at hand may not be well suited for the learner. The progress or sequence of the operation cannot be altered to satisfy educational goals. Dissection and exposure cannot be performed for demonstration only. Steps may not be repeated, and the patient cannot be reassembled to start over if failure occurs.4 In addition, fiscal constraints have resulted in pressure to achieve a high turnover in the OR, allowing less time for the attending staff to teach and trainees to practice skills.5 Bridges and Diamond5 have estimated that the annual cost of training chief residents in the OR amounts to more than $53 million per year and suggest that adjunctive training environments that use traditional and virtual teachings aids may alleviate cost over time. In addition to time constraints, one cannot neglect the ethical issues of teaching and learning using patients.6 There are tremendous advantages to training outside the OR. The learning environment is more easily controlled and
Immersive virtual environments for medical training adjusted. The learning situation can be tailored for each student’s needs and can be altered on a minute-by-minute basis to create the desired effect. Perhaps the most valuable part of this training is granting “permission to fail” in a safe environment where there is no risk to patients. Studies have uncovered significant problems with the current surgical education curriculum. These include lack of continuity from undergraduate to graduate surgical education, and the lack of supervision when acquiring physical examination skill, ultimately resulting in poor performance.7-9 An innovative educational tool, the Objective Structured Clinical Examination, has proven useful in the evaluation of the clinical competence of surgical residents.10 Surgical simulators have, perhaps, the best potential to mitigate surgical risk related to the educational process. A surgeon will be able to practice new procedures repeatedly until he or she is judged proficient without endangering patients. The surgeon can also be presented with cases of increasing complexity as his or her skills progress during training. Computer-based surgical simulators offer the potential for including operative cases representing all known anatomic variations. The training program director can use the simulator and its student tracking software to ensure that each graduating resident has seen and dealt with all the pertinent anatomic variations for that surgical specialty.11 Using simulation, mistakes would lose their consequences and become ways to learn. A master surgeon’s trick of the trade or critical maneuver during an operation could be learned in situ by every simulation user. The opportunity to learn something new this way has never before been available to medicine.12 Another potential justification of virtual reality (VR) training is reducing the length of a surgical residency program. Currently, these training programs require five or more years to permit adequate exposure to a variety of technical procedures and decision-making situations. Training programs are currently time limited and not proficiency based. VR training could potentially reduce 5-year residency programs, because residents would not have to wait for clinical cases to appear. Instead, he or she could call up a variety of cases and perform the procedure in VR several times before doing so on a human.13 One of the added attractions of simulation is that training programs might be able to correct for case-mix inequalities, so that what one learns in residency no longer depends only on what comes through the door when on call.12 Flexibility is important for mastery of skills. Simulation may well offer the additional flexibility required. Though currently costly to implement on a large-scale, simulation offers great promise in future reduction of errors (and malpractice suites), reducing (or eliminating) the use of animals, and helping to establish standards for certain procedures. An additional, and perhaps increasingly crucial, role of simulation may well be the assessment of possible decline in the skills of older surgeons. Measuring technical competence through VR could also be applied to older surgeons. As surgeons age, manual dexterity can decline, but it has always been difficult to objectively assess these skills. There is currently no mechanism to determine when these skill levels have deteriorated to the point where the surgeon should not
91 be allowed to operate.13 This decline in skills and judgment has traditionally been assessed by individual surgeons or chiefs of services. A mature, validated system of simulationbased education could offer for the first time a lifelong log of performance on standardized techniques, allowing measurement of skills independent of age or other arbitrary milestones.12 Thomas Russell, the current executive director of the American College of Surgeons, has stated “The competitive surgeon of the next 10 to 20 years will need to possess a different set of skills than we have needed in the past.”14 Dr. Russell has suggested that the use of simulation will provide early exposure to medical students, piquing their interest in a surgical career. Resident education will involve the use of simulators and experiences outside the OR to enhance the core competencies and move the learning process away from the traditional approach of “see one, do one, teach one” to “see one, practice many, and do one.”14 The surgeon of the future will be required to have periodic cognitive testing every few years as well as testing of their technological skills with the use of simulators as they progress in their careers. The acquisition of new surgical skills in practice will be much more structured in the future. The practice of industry-sponsored short courses with rapid introduction into clinical practice will no longer be acceptable. Surgeons will likely be required to undergo retraining in regional centers in which skills can be learned through validated multimodality curriculum. VR allows computer-based models of the real world to be generated and provides humans with a means to interact with theses models though new human– computer interfaces, and, thus to nearly realistically experience these models. The term “Virtual Reality” was initially coined by Jaron Lanier, and in the early 1980s, he founded VPL Research, the first company to sell VR products. In the late 1980s he led the team that developed the first implementations of multi-person virtual worlds using head-mounted displays, for both local and wide area networks, as well as the first “avatars,” or representations of users within such systems. While at VPL, he and his colleagues developed the first implementations of VR applications in surgical simulation. Today, ‘Virtual Reality’ is used in a variety of ways and often in a confusing and misleading manner. Originally, the term referred to “Immersive Virtual Reality.” In immersive VR, the user becomes fully immersed in an artificial, threedimensional world that is completely generated by a computer. VR-simulated environments allow trainees to repeat procedural experiences at their own leisure. These exercises or procedures would otherwise require numerous real-life encounters and costly hours of supervision.15 A commonly recognized type of VR experience is that of flight simulation. In the aerospace, aviation, and defense industries, flight simulation is mandatory before pilots assume flight responsibilities. In addition, flight simulation is regularly used to help commercial airline pilots maintain their skills, or to become familiar with problems they might one day encounter. Virtual Environments were first used by NASA for mission training in 1993.16
M.W. Bowyer et al.
92 Virtual Environments are now in routine use for extra vehicular activity training. The advantage over other methods of training is that they can be manipulated to replicate natural physical phenomena such as the absence of gravity and friction. Shared virtual environments are being used to train international crews for the Space Station. These environments allow training at a much reduced cost but also allow the concept of just-in-time training, allowing rapid response to emergencies by supporting the training of teams whose members may not be co-located.16 Haluck and coworkers in 200117 noted that virtual environments and computer-based simulators, although well-established training tools in other fields, have not been widely incorporated into surgical education. Concerns over the lack of validation, the cost, and finding time for residents to participate were cited as concerns. There are five major areas in medicine where VR is beginning to emerge: (1) assistance before and during medical surgical procedures; (2) medical education and training; (3) medical database visualization; (4) Medical Team Training; and (5) rehabilitation.18 For the most part, the advantages of flight simulators hold equally true for surgical simulators.18 Surgical simulators can provide a concentrated environment that lends itself to learning complex tactile maneuvers in a relatively quick and proficient manner. Moreover, simulation of infrequent but highly hazardous events provides experience in handling these scenarios that may not be available during a period of routine flight or surgical training.19 The ideal surgical simulator should provide the following: it can be customized to the needs of the student, the variety of cases during training increases significantly, and the student can chose to train only the difficult part of the surgery and repeat it as often as necessary.19 Satava18 has described five components that contribute to the realism of a virtual surgical world: fidelity, organ properties, organ reaction, interactivity, and sensory feedback. He predicts that the future holds promise of a virtual cadaver nearly indistinguishable from a real person.18 This concept is referred to as the Turing test, a standard test that means to determine if a computer could be created that responds the way a human would respond such that a human could not tell the difference between the computer and a human.20,21 The VR Turing test would be met if an interrogating human could not tell the virtual human apart from the real human by sight, hearing, or touch, even dissection.19 Current simulators do not yet meet the criteria of the Turing test. It is conceivable that future improvement in computing power and decreased costs of such technology will allow for development of such realism in a virtual environment. That being said, the level of fidelity required to meet the Turing test is likely not necessary to develop useful simulators that will teach useful skills in a validated fashion. In fact, many simulators are currently being used to teach medicine and range from low tech (inexpensive) to increasingly high tech, with corresponding price tags. The future use and development of simulation will depend in large part on validation of their effectiveness as training tools and to a certain extent the adoption of simulation by the various medical and surgical
boards and societies. As organizations and institutions realize the potential cost savings (in dollars and lives) of training with simulation, investment from both private and public sources should follow. Numerous simulators and VR training (mostly part-task) devices are currently available for training surgeons. Some of these are simple and inexpensive, while others are complex and costly. Simulators encompass everything from simple skills trainers such as knot-tying boards to part-task trainers such as a chest tube trainer, up to full procedural trainers that allow for training a complete laparoscopic or endoscopic procedure. What is lacking in many of current modes of training is the highly realistic environment where the trainee actually perspires, the heart rate goes up, and he reacts in a manner that is consistent with what he would do in a real environment. Effective training must also provide the trainee with a medical problem whose clinical solution requires critical thinking combined with diagnostic and skill competence. In addition, effective training needs to be supported by realistic stressors, eg, time limits, physical and sensory distractions, or even overload. It has been shown in several studies that training in a stressful environment similar to what one will likely encounter in the real world is essential for the effective training of medical intervention teams.22-26 As the field of simulation for medical training has matured, there has been increasing efforts to develop more realistic immersive environments. Although by no means comprehensive, the following represents some of the types of Immersive Environments that are currently available or under development for teaching surgeons with a brief discussion of their utility (where applicable) for training.
Physical Re-Creation of the Operating Room Perhaps the earliest use of an “immersive environment” for medical training can be attributed to the pioneering work of David Gaba and his team from Stanford.27 About 20 years ago this team re-created the operating room and used high-fidelity human patient mannequins to teach crisis management to anesthesiologists Figure 1A and B depicts the simulated operating suite at the National Capital Area Medical Simulation Center. The concepts of fight crew resource management training have been used with great success in this type of immersive environment.25 There are several advantages to this type of simulated environment. It is relatively easy to standardize the learning environment. The use of actual equipment used in the operating room allows for good suspension of disbelief. Such encounters are also videotaped and debriefed, which greatly enhances the training exercise. Some of the disadvantages of this type of immersive environment are that it requires the trainee to come to this fixed site, which may be remote from the actual workplace and training is generally limited to individuals or small teams. This type of simulated experience also tends to be labor intensive for the faculty and supporting cast. Although the
Immersive virtual environments for medical training
93
Figure 1 (A) A simulated Operating Room Environment using a High Fidelity Human Patient Simulator. (B) Students interacting with High Fidelity Human Patient Simulator.
scenarios can be altered to depict a variety of cases, there is difficulty in realistically portraying the scenario at other locations such as the prehospital scene, the emergency room, or the ICU environment.
Cave Automatic Virtual Environment (CAVE) The CAVE is an advanced virtual reality system that consists of a 10= ⫻ 10= room where each wall is a stereoscopic 3D screen similar to an IMAX theater. The CAVE was developed at the University of Illinois at Chicago28 and provides the illusion of immersion by projecting stereo images on the walls and floor of a room-sized cube. Several persons wearing lightweight stereo glasses can enter and walk freely inside the CAVE. A head tracking system continuously adjusts the ste-
reo projection to the current position of the leading viewer. A variety of input devices like data gloves, joysticks, and handheld wands allow the user to navigate through a virtual environment and to interact with virtual objects. Directional sound, tactile and force feedback devices, voice recognition, and other technologies can be employed to enrich the immersive experience. The University of Michigan was perhaps the first to recognize the potential of a CAVE or medical training and coupled the immersive environment (an emergency room bay (Fig. 2A)) with a human patient simulator to create a Hyperrich Fully Immersive Virtual Reality medical environment dubbed the Medical Readiness Trainer (Fig. 2B).29 The prototype Medical Readiness Trainer developed by the University of Michigan included the patient bay of an existing emergency room at the University of Michigan Hospital (Fig.
Figure 2 (A) An emergency room bay depicted in a Cave Automated Virtual Environment (CAVE) (Picture courtesy of Klaus-Peter Beier, PhD). (B) The University of Michigan Medical Readiness Trainer using a human patient simulator in a Cave Automated Virtual Environment (picture courtesy of Klaus-Peter Beier, PhD).
M.W. Bowyer et al.
94
Figure 3 (A) Head-injured patient in distributed immersive environment as seen through head-mounted display with operators hand holding syringe depicted in lower right corner. (B) Two students located in geographically remote locations exchanging an instrument as they work collaboratively on the virtual patient.
2A), a generic operating room whose architecture could be arranged to fit the scenario, and a sick bay of a U.S. Coast Guard Cutter. Additional medical training content was provided by a series of bit map flip books to provide short video clips to present scenario relevant ultrasound clips, medical records, x-rays, etc. The result was a novel Hyper-Rich Action Information environment that integrates and presents a wealth of mixed media medical information. The authors felt that the environment allowed students to master the practical clinical skills while providing simultaneous access to a vast body of supporting knowledge. It allowed for training students to manage very difficult scenarios in which the “reality” of life and death outcomes is a direct consequence of the chosen treatment regimen.29 This immersive environment is very engaging for students but is limited currently by the high cost of building such an environment and the ability to only train individuals and small groups at great expense. This limitation is currently being addressed as work is ongoing to create networked immersive environments for collaborative virtual reality, a task considered to be one of the most challenging areas of research in the field.30
Distributed Interactive Virtual Environment: Project TOUCH The Center for Telehealth research team at the University of New Mexico has developed a virtual reality environment that can be distributed over next-generation Internet 2.31 Project TOUCH (Telehealth Outreach for Unified Community Health) enables participants wearing a head-mounted display and a tracker system to be fully immersed within a Virtual Reality Environment. While in the Virtual Reality Environment students can navigate, locomote, and handle objects with a joy-wand control. This environment contains a
virtual patient with whom the students can collaboratively interact in a dynamic, problem-based, clinical, artificial intelligence rules-based fashion. In initial tests of this project, students in Hawaii and New Mexico were able to work collaboratively in problem-solving and managing of a simulated patient with a closed head injury (Fig. 3A), dividing tasks, handing off objects, and functioning as a team (Fig. 3B). This distributed interaction has also been successfully performed between the University of New Mexico and Western Australia. The ability to bring people together as virtual teams for interactive experiential learning and collaborative training, independent of distance, provides a platform for distributed “just-in-time” training, performance assessment, and credentialing. The promise of this technology is evident and the role for future training remains to be determined. In comparison to the CAVE technology described above, it is much cheaper and has the potential of having multiple (unlimited) individuals observe the training real-time as an eye in the sky.
Serious Games Video games have great potential to immerse users in a virtual environment. Such products as SecondLife® (http:// secondlife.com) and Halo 3® (http://www.halo3.com/) are examples of immersive high-fidelity 3D environments in which users can act independently or in collaboration with others. Serious Games are learning platforms that use gaming technology for educational (serious) purposes. Recently the gaming industry has begun to turn their attention to developing “games” for medicine. As the cost for this technology is relatively high, and the potential market relatively low, efforts to date have been limited and largely as a result of congressional and military funding. Two of the more well developed of these efforts are PULSE, a congressionally funded effort developed by Texas A&M University–Corpus Christi and
Immersive virtual environments for medical training
95
Figure 4 (A) A screen shot from the PULSE learning platform of the Intensive Care Unit of the National Naval Medical Center (courtesy of Claudia Johnston, PhD). (B) A screen shot from the PULSE learning platform showing the interaction with a virtual patient (courtesy of Claudia Johnston, PhD).
game developer Breakaway (http://www.breakawayltd.com/); and 3DiTeams, a military-funded effort developed by Duke University and gaming company Virtual Heroes (http://www. virtualheroes.com/). The PULSE project is described as a “state-of-the-art highfidelity first person learning platform that dynamically responds to real-world contextual inputs”—software with the capability to simulate virtually every aspect of the surgical process (http://seriousgamessource.com/features/feature_ 102506_pulse_1.php). The immersive environment in PULSE is a high-fidelity re-creation of the intensive care unit at the National Naval Medical Center in Bethesda, MD (Fig. 4A). Currently the “game” is a first-person learning platform where the learner is able to examine and care for soldiers who have been injured in Iraq and have just arrived from Ger-
many (Fig. 4B). Ultimately the plan is for the platform to be multiplayer such that teams can collaboratively work on the same patient. The platform is currently undergoing validation testing at Yale School of Medicine, the Johns Hopkins School of Medicine, and the National Naval Medical Center. Duke University’s “game,” 3DiTeams, is set in a military Combat Support Field Hospital emergency room (Fig. 5A and B). The situations that are portrayed using virtual patients have been developed to have relevance to both medical and civilian medical teams. The “game” will connect players via a network that allows them to interact in the virtual environment. The main focus of this learning platform is on team training. The potential for serious games to impact health care training is great. Although currently there is little monetary incen-
Figure 5 (A) A screen shot from 3DiTeams showing team interaction with the virtual patient (courtesy of Jeffrey Taekman, MD). (B) A screen shot from 3DiTeams showing the virtual patient and available adjuncts (courtesy of Jeffrey Taekman, MD).
M.W. Bowyer et al.
96
Figure 6 Concept drawings of a 1000-square-foot 18-screen Wide Area Virtual Environment (A). A medical team works collaboratively to triage an treat victims (B), move them to a helicopter (C), and ultimately perform surgical intervention (D), all within this reconfigurable immersive environment (illustrations by Sophia Del Castillo).
tive for gaming companies to develop such games, as this technology matures and studies of efficacy appear, the demand will increase and the availability and fidelity will continue to improve.
Wide Area Virtual Environment (WAVE) The WAVE is based on the aforementioned CAVE technology but represents an evolution beyond this. Unlike the CAVE where users wear head-mounted displays, the WAVE allows team members to interact with each other in a natural fashion wearing lightweight polarized glasses to view the scene. The National Capital Area Medical Simulation Center of the Uniformed Services University has begun construction on what will ultimately be an 18-screen, 1000-square-foot environment (Fig. 6A).32 Using a space of this size, larger teams can be trained using the equipment and other gear, such as pro-
tective clothing within the environment. The environment can take advantage of blended reality by combining virtual patients with high-fidelity human patient simulators, and live simulated patients. Additionally, part task trainers can be used in the environment to perform various procedures (eg, chest tube placement) as part of an ongoing scenario. The environment can also be enriched with appropriate noise, smells, and elements such as smoke. We envision great utility in using the WAVE to train military medical teams with the ability to create multiple environments within the same training scenario. For example, a team can be asked to triage and treat patients on the scene of a HUMVEE that has been struck by an improvised explosive device in Iraq (Fig. 6B). The environment could include virtual patients, mannequins, and live simulated patients and augmented by the sights, sounds, and smells of an ongoing firefight. Once the team performs initial triage and treatment, they then place the patient on a stretcher and carry him through the corridor to
Immersive virtual environments for medical training what is now the inside of a helicopter (Fig. 6C), where ongoing care is provided. While in the helicopter, the original area of the WAVE is reconfigured to be an operating suite where the patient will be delivered and have appropriate interventions (Fig. 6D). The obvious limitation of this type of environment is the required space and initial and ongoing expense of construction and maintenance. The advantages of a training platform like the WAVE is the ability to create realistic representations of mass-casualty and disaster that could not otherwise be accomplished. As envisioned, this technology will also be distributable, such that multiple geographically remote users (from desktop or small CAVEs) could also participate in training and participate (as avatars) in the ongoing scenario. We have built a three-screen prototype in which we are currently developing and testing educational content to refine the final product.
Summary Over the last decade the availability and sophistication of medical and surgical virtual reality simulations and simulators has increased dramatically. We are on the verge of a paradigmatic shift in how medicine will be taught and practiced in the future. Virtual environments promise highly flexible, well-controlled learning situations that can vary from simple tasks to rare and challenging problems. There are a variety of immersive environments available for medical education and the near future is likely to see an explosion in their development and adoption as an adjunct to medical and surgical education.
References 1. Kaufman DM, Bell W: Teaching and assessing clinical skills using virtual reality. Stud Health Technol Inform 39:467-472, 1997 2. Colt HG, Crawford SW, Galbraith O: Virtual reality bronchoscopy simulation: a revolution in procedural training. Chest 120:1333-1339, 2001 3. Haluck RS, Krummel TM: Computers and virtual reality for surgical education in the 21st century. Arch Surg 135:786-792, 2000 4. Scallon SE, Fairholm DJ, Cochrane DD, et al: Evaluation of the operating room as a teaching venue. Can J Surg 35:173-176, 2001 5. Bridges M, Diamond DL: The financial impact of teaching surgical residents in the operating room. Am J Surg 177:28-32, 2001 6. Gates EA: New surgical procedures: can our patients benefit while we learn? Am J Obstet Gynecol 176:1293-1297, 1997 7. Gorman PJ, Meier AH, Krummel TM: Computer assisted training and learning in surgery. Comp Aid Surg 5:120-130, 2000 8. Chalabian J, Garman K, Wallace P, et al: Clinical breast evaluation skills of house officers and students. Am Surg 62:840-845, 1996 9. Endean ED, Sloan DA, Veldenz HC, et al: Performance of the vascular physical examination by residents and medical students. J Vasc Surg 19:149-156, 1994
97 10. Sloan DA, Donnelly MB, Schwartz RW, et al: The objective structured clinical examination. The new gold standard for evaluating postgraduate clinical performance. Ann Surg 222:735-742, 1995 11. Kaufmann CR: Computers in surgical education and the operating room. Ann Chir Gynaecol 90:141-143, 2001 12. Dawson SL: A critical approach to medical simulation. Bull Am Coll Surg 87:12-18, 2002 13. Ota D, Loftin B, Saito T, Lea R, et al: Virtual reality in surgical education. Comput Biol Med 25:127-137, 1995 14. Russell T: From my perspective. Bull Am Coll Surg 88:3-4, 2003 15. Issenberg SB, McGeghie WC, Hart IR, et al: Simulation technology for health care professional skills training and assessment. JAMA 282:861867, 1999 16. Loftin RB, Kenney P: Training the Hubble Space Telescope Flight team. IEEE Comp Graph Appl 15:31-37, 1995 17. Haluck RS, Marshall RL, Krummel TM, et al: Are surgery training programs ready for virtual reality? A survey of program directors in general surgery. J Am Coll Surg 193:660-665, 2001 18. Satava RM: Medical applications of virtual reality. J Med Syst 19:275280, 1995 19. Lange T, Indelicato DJ, Rosen JM: Virtual reality in surgical training. Surg Oncol Clin North Am 9:61-79, 2000 20. Bleich HL, Turing A: The machine, the enigma, and the test. MD Comput 12:330-334, 1995 21. Heiser JF, Colby KM, Faught WS, et al: Can psychiatrists distinguish a computer simulation of paranoia from the real thing? The limitations of Turing-like test measures of the adequacy of simulation. J Psychiatr Res 15:149-162, 1979 22. Small SD, Wuerz RC, Simon R, et al: Demonstration of high fidelity simulation team training for emergency medicine. Acad Emerg Med 6:312-323, 1999 23. Risser DT, Rice MM, Salisbury ML, et al: The potential for improved teamwork to reduce medical errors in the emergency department. Ann Emerg Med 34:373-383, 1999 24. Helmreich RL: On error management: lessons from aviation. BMJ 320: 781-785, 2000 25. Howard SK, Gaba DM, Fish KJ, et al: Anesthesia crisis resource management training: teaching anesthesiologists to handle critical incidents. Aviat Space Environ Med 63:763-770, 1992 26. Holzman RS, Cooper JB, Gaba DM, et al: Anesthesia crisis resource management: real-life simulation training in operating room crises. J Clin Anesth 7:675-687, 1995 27. Gaba DM, Deanda A: A comprehensive anesthesia simulation environment: Re-creating the operating room for research and training. Anesthesiology 69:740-746, 1988 28. Cruz-Neira C, Sandin DJ, DeFanti TA: Surround-screen projection based virtual reality: the design and implementation of the CAVE. Proc ACM SIGGRAPH 135-142, 1993 29. Von Lubitz DKJE, Beier KP, Freer J, et al: Immersive virtual reality platform for medical training: a “killer-application”. Stud Health Technol Inform 70:207-213, 2000 30. Leigh J, Johnson A, Defanti T: CAVERN: a distributed architecture for supporting scalable persistence and intraoperability in collaborative virtual environments. Virtual Real Res Dev Appl 2:217-237, 1997 31. Alverson DC, Saiki SM, Jacobs J, et al: Distributed interactive virtual environments for collaborative experiential learning and training independent of distance over Internet 2. Stud Health Technol Inform 98:712, 2004 32. Lee C, Liu A, Del Castillo S, et al: Towards an immersive virtual environment for medical team training. Stud Health Technol Inform 127: 274-279, 2007