- Email: [email protected]
Manual Skill Acquisition During Transesophageal Echocardiography Simulator Training of Cardiology Fellows: A Kinematic Assessment
Manual Skill Acquisition During Transesophageal Echocardiography Simulator Training of Cardiology Fellows: A Kinematic Assessment Robina Matyal, MD,* Mario Montealegre-Gallegos, MD,* John D. Mitchell, MD,* Han Kim, MD,† Remco Bergman, MD,‡ Katie M. Hawthorne, MD,§ David O’Halloran, MD,§ Vanessa Wong, BS,* Phillip E. Hess, MD,* and Feroze Mahmood, MD* Objective: To investigate whether a transesophageal echocardiography (TEE) simulator with motion analysis can be used to impart proficiency in TEE in an integrated curriculum-based model. Design: A prospective cohort study. Setting: A tertiary-care university hospital. Participants: TEE-naïve cardiology fellows. Interventions: Participants underwent an 8-session multimodal TEE training program. Manual skills were assessed at the end of sessions 2 and 8 using motion analysis of the TEE simulator’s probe. At the end of the course, participants performed an intraoperative TEE; their examinations were video captured, and a blinded investigator evaluated the total time and image transitions needed for each view. Results are reported as mean ⫾ standard deviation, or median (interquartile range) where appropriate. Measurements and Main Results: Eleven fellows completed the knowledge and kinematic portions of the
study. Five participants were excluded from the evaluation in the clinical setting because of interim exposure to TEE or having participated in a TEE rotation after the training course. An increase of 12.95% in post-test knowledge scores was observed. From the start to the end of the course, there was a significant reduction (p o 0.001 for all) in the number of probe. During clinical performance evaluation, trainees were able to obtain all the required echocardiographic views unassisted but required a longer time and had more probe transitions when compared with an expert. Conclusion: A curriculum-based approach to TEE training for cardiology fellows can be complemented with kinematic analyses to objectify acquisition of manual skills during simulator-based training. & 2015 Elsevier Inc. All rights reserved.
G
of normal echocardiographic anatomy and an expectation that trainees quickly will appreciate abnormal structure and function. As for acquisition of motor skills, trainees also are expected to learn subtle probe manipulations without any objective performance feedback. Assessment of acquisition of manual dexterity during TEE training primarily is endpoint based, with instructor-evaluated quality of the acquired image being used as a metric of success. High-fidelity haptic simulators (ie, simulators with tactile and directional feedback) have been used in multiple specialties to improve psychomotor skills during diverse procedures. Haptic TEE simulators can track the probe motions in 3D space during image acquisition. Using this technology, kinematic analyses were developed by the authors to track the progression of manual skills for TEE in echo-naïve anesthesia residents of various levels of
RADUATE MEDICAL EDUCATION is evolving from a traditional time-based apprenticeship to a competencybased learning model. Simultaneously, graduate medical education-imposed restrictions on the resident physician duty hours have limited trainee participation in clinical activities. These administrative changes have affected training for clinical tasks that require repetitive performance for attaining proficiency. With mandatory time limitations on resident participation, it has become a logistic challenge to demonstrate qualitative and quantitative improvements in training. These constraints also offer opportunities for innovation in medical education and training (eg, simulation). Simulation technology has been used in healthcare to supplement clinical training in multiple specialties.1–8 The role of medical simulation training has evolved recently, and advanced mixed simulators have been introduced with both virtual and real-world elements for complex task training and performance evaluation.9 Trainees can interact physically with simulators, resulting in virtual consequences requiring more physically corrective maneuvers for task completion. The context and difficulty level of the desired virtual task can be varied to suit the skill or experience level of the trainee. Because there are no consequences of failure, these simulators are for learning invasive clinical procedures that require repetitive performance for proficiency.10 TEE is an integral procedure for cardiology practice, and clinical training for it is conferred during accredited cardiology fellowships. Proficiency in TEE requires a solid theoretical background, with significant manual dexterity acquired through repetitive supervised clinical performance. Cognitively, appreciation of echocardiographic display of cardiac anatomy requires spatial orientation. There is an implied understanding
KEY WORDS: transesophageal echocardiography, simulation, kinematic analysis
From the *Department of Anesthesia, Critical Care and Pain Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA; †Department of Anesthesia, St. Michael’s Hospital, Toronto, Canada; ‡Department of Anesthesiology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands; and §Division of Cardiology, Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA. Address reprint requests to: Feroze Mahmood, MD, Department of Anesthesia, Critical Care and Pain Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, 1 Deaconess Road, CC470, Boston, MA 02215. E-mail: [email protected] © 2015 Elsevier Inc. All rights reserved. 1053-0770/2601-0001$36.00/0 http://dx.doi.org/10.1053/j.jvca.2015.05.198
Journal of Cardiothoracic and Vascular Anesthesia, Vol ], No ] (Month), 2015: pp ]]]–]]]
1
2
MATYAL ET AL
seniority.11 The application of these innovative motion metrics was assessed to evaluate manual skill acquisition of TEE-naïve cardiology fellows after a multimodal TEE training course. METHODS
The institutional review board of the authors’ institution approved this study, with a waiver of informed consent. Participants This study was performed over 2 consecutive years (20122013). For each year, a group of TEE-naïve cardiology fellows were invited to participate. Exclusion criteria were performed 45 TEE exams, and having taken the basic or advanced examination of competence in perioperative echocardiography administered by the National Board of Echocardiography. Description of the Training Program Subjects participated in a multimodal 4-week training program, including 8 web-based modules (anesthesiaeducation.net/moodle), as well as 2-hour sessions twice per week consisting of live lectures by a facilitator (30 minutes) and supervised practice on 2 Vimedix TEE simulators (CAE Healthcare, Montreal, Canada). The online modules consisted of video presentations from faculty on specific echocardiography topics. Topics covered in the online modules included TEE probe positioning and manipulation, basic TEE views, left and right ventricular functions, mitral and aortic valves, and hemodynamics. Time of access to the website and attendance of the hands-on simulator sessions were monitored for each participant. The subjects were required to complete specific didactic portions on the website before participation in the hands-on session for that day. Experienced faculty monitored and supervised all hands-on sessions. The hands-on training started with an explanation of the features of the simulator and TEE probe during the first session. On subsequent sessions, image acquisition and interpretation were coordinated with the didactic program. The fellows also were trained on the components of a comprehensive intraoperative TEE examination.12 Knowledge Immediately before the start of the training program, the trainees took an online pretest of basic TEE knowledge. There were 54 multiple-choice questions regarding ultrasound physics, normal echocardiographic anatomy, and image display and optimization. Access to the website was by invitation and password protected. After finalizing the training course, the trainees were asked to complete a similar knowledge test and to evaluate, on a scale of 1 to 5 (1 ¼ very poor, 5 ¼ excellent), the usefulness of the hands-on sessions, online modules, and live lectures.
consensus of the investigators based on their image quality and reproducibility on the simulator. It is important to note that the deep transgastric window was not available on the version of the simulator’s software used in this study. These TCPs corresponded to the midesophageal 4-chamber (ME-4C), ascending aortic short-axis, bicaval, right ventricular inflowoutflow, transcommissural, 2-chamber, and long-axis views, as well as the transgastric basal short-axis, long-axis, and right ventricular inflow views. The TCPs then were reviewed and agreed on by all investigators. Kinematic analysis was performed at the start of the course (at the end of the second training session) and at the end of the course (at the end of the eighth session). During the kinematic analysis, the subject was first asked to acquire the ME-4C view. Subsequently, ME-4C view was used as the starting position for each new image acquisition. From the ME-4C, each subject was asked to acquire an image resembling the requested TCP. During data acquisition, the TCP image was displayed next to the active scan plane (Fig 1). The simulator automatically recorded the real-time data, including the starting point and the change in position in the x, y, and z axes of the tip of the TEE probe over time intervals of o0.2 s. The data were exported from the simulator as a Microsoft Excel (Microsoft Corporation; Redmond, WA) spreadsheet, and the following data were extrapolated: Total time: time from the starting position until the TCP was acquired; path length: the sum of all linear and angular movements of the probe in centimeters; probe accelerations from rest: the number of times the probe acceleration exceeded 0.5 cm/s2; and time-distance integral: area under the curve of a plot of time (x-axis) versus distance travelled (y-axis). Evaluation in the Clinical Setting After the participants completed their TEE training program, they were taken to the operating room to perform a TEE examination on an anesthetized patient. The examinations were performed with a Phillips iE33 machine (Phillips Healthcare; Andover, MA) with an X7-2T transducer. First, an experienced echocardiographer acquired images in each patient, corresponding to the TCPs tested previously on the simulator, with a starting position in the ME-4C view. Afterwards, the trainees were asked to obtain the same images. Expert and novice examinations were not labeled on the screen and were coded for further analysis. The entire examination, as performed by the expert and trainees, was video captured using the screen capture system Epiphan DVI2USB 3 (Epiphan Systems, Inc.; Ottawa, Canada) and a Mac Mini (Apple Inc.; Cupertino, CA) computer. A blinded investigator (M.M.) analyzed the video-captured examinations. For each view, the time necessary to obtain it and the number of probe transitions from the reference image (ie, the ME-4C) to the desired image was noted. Statistical Analysis
Target Cut Planes and Kinematic Testing A single investigator (F.M.) acquired 10 target cut planes (TCP), or simulator reference images corresponding to a standard TEE image. The specific TCPs were selected by
The data were evaluated using SPSS version 21 for Mac OS. Knowledge and course satisfaction data, as well as the clinical setting evaluations and kinematic data for the skills evaluation sessions were assessed for differences using
3
KINEMATIC ANALYSIS AND TRANSESOPHAGEAL ECHO TRAINING
Fig 1. A target cut planes view corresponding to the midesophageal long axis (right) is shown on the screen simultaneously with the image obtained by the trainee (left). Figure reprinted with permission from CAE Healthcare, Montreal, Canada.
Wilcoxon matched-pairs signed ranks test. The alpha level was set at p o 0.05. For the evaluation of kinematic metrics, adjustment of p values for multiple comparisons was performed using the Holm-Bonferroni sequential correction.13 Data are presented as median (interquartile range) or mean ⫾ standard deviation where appropriate. RESULTS
Eleven participants were recruited (n ¼ 5 in year 1 and n ¼ 6 in year 2). All of them completed the knowledge and kinematic testing. Five participants were excluded from the evaluation in the clinical setting due to interim TEE exposure, having completed 45 TEE examinations, or having participated in a TEE rotation before clinical evaluation. The post-test scores were higher than the pretest scores (80.4% ⫾ 7.7 v 67.5% ⫾ 11.2; p o 0.05). There was a trend toward increased self-perceived knowledge of and comfort level with TEE at the end of the course compared with baseline, but it did not reach statistical significance. The participants also found the different Table 1. Knowledge Assessment and Perceived Usefulness of the Training Program Endpoint
Online knowledge test Comfort level with TEE Self-perceived knowledge with TEE Usefulness of the hands-on sessions Usefulness of the live lectures Usefulness of the online modules
modalities of the course useful, particularly the hands-on component (Table 1). The total time, path length, number of probe accelerations from rest and time-distance integral all decreased from the start to the end of the course (Table 2). These observations were consistent across different views (Fig 2). During the clinical evaluation portion of the study, all trainees were able to get all the views solicited without assistance. When compared with the expert, the time for acquiring each view was significantly higher for the trainees (18.3 ⫾ 9.2 s v 8.9 ⫾ 3.3 s; p ¼ 0.028). The same difference was true for the number of transitions per view (4.2 ⫾ 2.8 for the trainees, 2.1 ⫾ 0.8 for the experts, p ¼ 0.027) (Fig 3). DISCUSSION
The present study confirmed the utility of a curriculumbased approach to impart proficiency in performing TEE to cardiology fellows without previous exposure to TEE. A primarily web-based system was used to impart cognitive knowledge, and manual dexterity skills were acquired through training with a TEE simulator. The TEE-naïve trainees were able also to obtain standard views on an intraoperative TEE
Before the Course After the Course
67.48%/100% 2.9%/5% 3.1/5% — — —
80.43%/100% 3.6%/5% 3.6%/5% 4.4%/5% 3.2%/5% 3.6%/5%
Abbreviation: TEE, transesophageal echocardiography. Average score/ Maximum possible score
Table 2. Kinematic Measures Obtained at the Start and End of the Course (n ¼ 11 trainees) Metric
Probe accelerations (n) Time distance integral Total time (s) Path length (cm)
Start of Course
59 1.0 25.3 12.3
(31-130) (0.2-1.6) (16.7-38.4) (2.5-21.0)
End of Course
40 0.4 12.6 6.7
(21-64) (0.1-0.7) (8.9-16.4) (1.2-11.3)
Data are presented as median (interquartile range).
p Value
o0.001 o0.001 o0.001 o0.001
4
MATYAL ET AL
Fig 2. The number of probe accelerations from rest (panel A), total time to acquire a target cut plane (panel B), path length (panel C), and time-distance integral (panel D) are shown for each view at the start and end of the course. Abbreviations: Bicaval, midesophageal bicaval; ME LAX, midesophageal long axis; AV SAX, midesophageal aortic valve short axis; COMM, midesophageal commissural; ME 2CH, midesophageal 2 chamber; TG Basal, transgastric basal short axis; TG MID, transgastric mid short axis; TG RV INF, transgastric right ventricular inflow; TG LAX, transgastric long axis.
examination after this training without instructor assistance, thus demonstrating an element of clinical transferability. This curriculum may help address Accreditation Council Graduate Medical Education (ACGME) requirements, because some of the activities may be time-shifted to comply with duty hour limits. Additionally, the web-based component can be categorized as self-study time, which is not included in the
ACGME definition of duty hour. The authors also demonstrated the reproducibility of kinematic analyses in a group of trainees with a different background and level of experience than this initial report (ie, cardiology fellows as opposed to anesthesiology residents).11 Specifically, there was a significant reduction in total time to acquire the image, path length and the time-distance integral (Figs 4 and 5). Most importantly, there
Fig 3. The number of image transitions (panel A) of the trainees exceeded the expert’s. The total time required to obtain each view was significantly longer for the trainees when compared with the expert (panel B).
KINEMATIC ANALYSIS AND TRANSESOPHAGEAL ECHO TRAINING
5
Fig 4. 3D path course during acquisition of the transgastric basal short-axis view. On the first skill evaluation at the start of the course (panel A), a longer and more tortuous path is observed for the trainee when compared with the evaluation at the end of the course (panel B). The path for an expert is shown for reference (panel C).
was a significant decrease in TEE probe accelerations (“jerking motions”) over time with simulator training. The reductions in these variables describe an increased “economy of motion” after finalizing the training course. This is an important finding because it represents an increased ability to select the most adequate TEE probe motions, which correlate with a faster, more intuitive, and smoother image acquisition. During the clinical evaluation of trainees, the number of image transitions was used as a surrogate for peak motions of the TEE probe. Compared with experts, demonstration of more transitional images by the trainees implied that this was a potential metric to help differentiate novices
from experts. Smoothness of TEE probe motion implied intuitive motion and possibly represented a degree of automaticity. Automaticity is considered a predictor of clinical transferability.14,15 Although it was not specifically evaluated in the present study, TEE-naïve trainees demonstrated elements of clinical transferability. Despite being TEE naïve, they were able to perform an unassisted intraoperative TEE examination on their first attempt, although they had more transitions when compared with an expert. The trainees also required a longer time to perform image acquisition in the clinical setting when compared with an expert, although (approximately 9 seconds) may
Fig 5. Time-velocity plotting shows a greater number of probe jerks during acquisition of a transgastric long axis at the start of the course (panel A) when compared with the end of the course (panel B). The probe acceleration is the slope of the curve at any given time.
6
MATYAL ET AL
not be clinically significant. Because of current technologic limitations (ie, the magnetic properties of the TEE probe), electromagnetic sensors used with the simulator cannot be used to track movements of the TEE probe. Therefore, probe motion metrics could not be used during the intraoperative examination sequence. Kinematic analyses potentially could be used to establish a trainee’s readiness to perform. Acquisition of manual dexterity skills before clinical performance reduces the learning curve and may enhance the quality of educational experience and patient safety during training. Accredited surgical training programs have introduced mandatory laparoscopic simulator-based task training for residents before actual clinical exposure. In the Fundamentals of Laparoscopic Surgery (FLS) program, the trainees have to successfully complete a written examination for cognition along with pre-defined motor tasks on a simulator.16,17 In the Fundamentals of Endoscopic Surgery (FES) program, this simulator-based approach has been extended to endoscopy training.18 Trainee evaluation in the programs is endpointbased (ie, time required for task completion number of errors made by the trainee). In addition to endpoint-based assessment during TEE simulation training, the use of kinematic metrics allows for an objective assessment of the quality of motion. The curriculum-based approach of this study allowed integration of didactics with hands-on training without the unpredictability of clinical case presentation (eg, mitral valve lectures integrated with mitral valve views in cases of mitral valve prolapse on the simulator). Training in perioperative TEE highlights many of the logistic challenges in the medical education paradigm of time-based training. The availability of a virtual reality TEE
simulator and the application of metrics potentially can address these challenges. With the proposed model, the training can be graduated from simple to complex, cognitive learning can be integrated with manual dexterity, and acquisition of motor skills can be objectified. While it does not replace clinical exposure, simulation-based training has the advantages of being focused on the trainee rather than the patient, having no consequences of failure, and posing absolutely no risk to the patient.10 This potentially can reduce the learning curve and improve the quality of TEE education. The present study had some limitations. Firstly, the sample size was small, mainly because a decision was made to include trainees without prior exposure to TEE, which limited the sample. However, the results of the psychomotor skills testing were consistent among the individuals. Another potential limitation was the current lack of an ideal method to assess transferability of simulator-based skills into the clinical setting. Additionally, the study lacked a control group to validate the acquisition of skills during simulation training compared with conventional training. The study was designed to quantify the value of a curriculum-based approach with kinematic metrics, whereas the value of simulator training in improving manual dexterity already is established.19 CONCLUSIONS
It is feasible to implement an integrated curriculum-based approach for TEE training. TEE-naïve cardiology fellows significantly improved their cognitive knowledge via webbased education and their manual dexterity skills as determined with motion metrics. TEE-naïve trainees demonstrated some elements of clinical transferability after finishing the training program.
REFERENCES 1. Neelankavil J, Howard-Quijano K, Hsieh TC, et al: Transthoracic echocardiography simulation is an efficient method to train anesthesiologists in basic transthoracic echocardiography skills. Anesth Analg 115:1042-1051, 2012 2. Platts DG, Humphries J, Burstow DJ, et al: The use of computerised simulators for training of transthoracic and transoesophageal echocardiography. The future of echocardiographic training? Heart Lung Circ 21:267-274, 2012 3. Bose R, Matyal R, Panzica P, et al: Transesophageal echocardiography simulator: A new learning tool. J Cardiothorac Vasc Anesth 23: 544-548, 2009 4. Matyal R, Bose R, Warraich H, et al: Transthoracic echocardiographic simulator: Normal and the abnormal. J Cardiothorac Vasc Anesth 25:177-181, 2011 5. Sutherland C, Hashtrudi-Zaad K, Sellens R, et al: An augmented reality haptic training simulator for spinal needle procedures. IEEE Trans Biomed Eng 60:3009-3018, 2013 6. Fucentese SF, Rahm S, Wieser K, et al: Evaluation of a virtual-reality-based simulator using passive haptic feedback for knee arthroscopy. Knee Surg Sports Traumatol Arthrosc 23: 1077-1085, 2015 7. Yudkowsky R, Luciano C, Banerjee P, et al: Practice on an augmented reality/haptic simulator and library of virtual brains improves residents’ ability to perform a ventriculostomy. Simul Healthc 8:25-31, 2013
8. Barrow A, Akhtar K, Gupte C, et al: Requirements analysis of a 5 degree of freedom haptic simulator for orthopedic trauma surgery. Stud Health Technol Inform 184:43-47, 2013 9. Gaba D.: Simulator training in anesthesia growing rapidly. Available at: http://www.apsf.org/newsletters/html/1995/fall/simulator. html. Accessed February 18, 2015. 10. Shakil O, Mahmood F, Matyal R: Simulation in echocardiography: An ever-expanding frontier. J Cardiothorac Vasc Anesth 26: 476-485, 2012 11. Matyal R, Mitchell JD, Hess PE, et al: Simulator-based transesophageal echocardiographic training with motion analysis: A curriculum-based approach. Anesthesiology 121:389-399, 2014 12. Shanewise JS, Cheung AT, Aronson S, et al: ASE/SCA guidelines for performing a comprehensive intraoperative multiplane transesophageal echocardiography examination: Recommendations of the American Society of Echocardiography Council for Intraoperative Echocardiography and the Society of Cardiovascular Anesthesiologists Task Force for Certification in Perioperative Transesophageal Echocardiography. Anesth Analg 89:870-884, 2014 13. Gaetano J.: Holm-Bonferroni sequential Correction: An EXCEL calculator. Available at: http://www.researchgate.net/publication/242331583_ Holm-Bonferroni_Sequential_Correction_An_EXCEL_Calculator_-_Ver._1.2. Accessed May 12, 2015.
KINEMATIC ANALYSIS AND TRANSESOPHAGEAL ECHO TRAINING
7
14. Stefanidis D, Scerbo MW, Sechrist C, et al: Do novices display automaticity during simulator training? Am J Surg 195:210-213, 2008 15. Stefanidis D, Scerbo MW, Korndorffer JR, et al: Redefining simulator proficiency using automaticity theory. Am J Surg 193: 502-506, 2007 16. Okrainec A, Soper NJ, Swanstrom LL, et al: Trends and results of the first 5 years of Fundamentals of Laparoscopic Surgery (FLS) certification testing. Surg Endosc 25:1192-1198, 2011
17. Fundamentals of Laparoscopic Surgery. Available at: http:// www.flsprogram.org. Accessed December 10, 2014. 18. The Fundamentals of Endoscopic Surgery. Available at: http:// www.fesprogram.org. Accessed December 10, 2014. 19. Stefanidis D, Scerbo MW, Montero PN, et al: Simulator training to automaticity leads to improved skill transfer compared with traditional proficiency-based training: A randomized controlled trial. Ann Surg 255:30-37, 2012
study. Five participants were excluded from the evaluation in the clinical setting because of interim exposure to TEE or having participated in a TEE rotation after the training course. An increase of 12.95% in post-test knowledge scores was observed. From the start to the end of the course, there was a significant reduction (p o 0.001 for all) in the number of probe. During clinical performance evaluation, trainees were able to obtain all the required echocardiographic views unassisted but required a longer time and had more probe transitions when compared with an expert. Conclusion: A curriculum-based approach to TEE training for cardiology fellows can be complemented with kinematic analyses to objectify acquisition of manual skills during simulator-based training. & 2015 Elsevier Inc. All rights reserved.
G
of normal echocardiographic anatomy and an expectation that trainees quickly will appreciate abnormal structure and function. As for acquisition of motor skills, trainees also are expected to learn subtle probe manipulations without any objective performance feedback. Assessment of acquisition of manual dexterity during TEE training primarily is endpoint based, with instructor-evaluated quality of the acquired image being used as a metric of success. High-fidelity haptic simulators (ie, simulators with tactile and directional feedback) have been used in multiple specialties to improve psychomotor skills during diverse procedures. Haptic TEE simulators can track the probe motions in 3D space during image acquisition. Using this technology, kinematic analyses were developed by the authors to track the progression of manual skills for TEE in echo-naïve anesthesia residents of various levels of
RADUATE MEDICAL EDUCATION is evolving from a traditional time-based apprenticeship to a competencybased learning model. Simultaneously, graduate medical education-imposed restrictions on the resident physician duty hours have limited trainee participation in clinical activities. These administrative changes have affected training for clinical tasks that require repetitive performance for attaining proficiency. With mandatory time limitations on resident participation, it has become a logistic challenge to demonstrate qualitative and quantitative improvements in training. These constraints also offer opportunities for innovation in medical education and training (eg, simulation). Simulation technology has been used in healthcare to supplement clinical training in multiple specialties.1–8 The role of medical simulation training has evolved recently, and advanced mixed simulators have been introduced with both virtual and real-world elements for complex task training and performance evaluation.9 Trainees can interact physically with simulators, resulting in virtual consequences requiring more physically corrective maneuvers for task completion. The context and difficulty level of the desired virtual task can be varied to suit the skill or experience level of the trainee. Because there are no consequences of failure, these simulators are for learning invasive clinical procedures that require repetitive performance for proficiency.10 TEE is an integral procedure for cardiology practice, and clinical training for it is conferred during accredited cardiology fellowships. Proficiency in TEE requires a solid theoretical background, with significant manual dexterity acquired through repetitive supervised clinical performance. Cognitively, appreciation of echocardiographic display of cardiac anatomy requires spatial orientation. There is an implied understanding
KEY WORDS: transesophageal echocardiography, simulation, kinematic analysis
From the *Department of Anesthesia, Critical Care and Pain Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA; †Department of Anesthesia, St. Michael’s Hospital, Toronto, Canada; ‡Department of Anesthesiology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands; and §Division of Cardiology, Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA. Address reprint requests to: Feroze Mahmood, MD, Department of Anesthesia, Critical Care and Pain Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, 1 Deaconess Road, CC470, Boston, MA 02215. E-mail: [email protected] © 2015 Elsevier Inc. All rights reserved. 1053-0770/2601-0001$36.00/0 http://dx.doi.org/10.1053/j.jvca.2015.05.198
Journal of Cardiothoracic and Vascular Anesthesia, Vol ], No ] (Month), 2015: pp ]]]–]]]
1
2
MATYAL ET AL
seniority.11 The application of these innovative motion metrics was assessed to evaluate manual skill acquisition of TEE-naïve cardiology fellows after a multimodal TEE training course. METHODS
The institutional review board of the authors’ institution approved this study, with a waiver of informed consent. Participants This study was performed over 2 consecutive years (20122013). For each year, a group of TEE-naïve cardiology fellows were invited to participate. Exclusion criteria were performed 45 TEE exams, and having taken the basic or advanced examination of competence in perioperative echocardiography administered by the National Board of Echocardiography. Description of the Training Program Subjects participated in a multimodal 4-week training program, including 8 web-based modules (anesthesiaeducation.net/moodle), as well as 2-hour sessions twice per week consisting of live lectures by a facilitator (30 minutes) and supervised practice on 2 Vimedix TEE simulators (CAE Healthcare, Montreal, Canada). The online modules consisted of video presentations from faculty on specific echocardiography topics. Topics covered in the online modules included TEE probe positioning and manipulation, basic TEE views, left and right ventricular functions, mitral and aortic valves, and hemodynamics. Time of access to the website and attendance of the hands-on simulator sessions were monitored for each participant. The subjects were required to complete specific didactic portions on the website before participation in the hands-on session for that day. Experienced faculty monitored and supervised all hands-on sessions. The hands-on training started with an explanation of the features of the simulator and TEE probe during the first session. On subsequent sessions, image acquisition and interpretation were coordinated with the didactic program. The fellows also were trained on the components of a comprehensive intraoperative TEE examination.12 Knowledge Immediately before the start of the training program, the trainees took an online pretest of basic TEE knowledge. There were 54 multiple-choice questions regarding ultrasound physics, normal echocardiographic anatomy, and image display and optimization. Access to the website was by invitation and password protected. After finalizing the training course, the trainees were asked to complete a similar knowledge test and to evaluate, on a scale of 1 to 5 (1 ¼ very poor, 5 ¼ excellent), the usefulness of the hands-on sessions, online modules, and live lectures.
consensus of the investigators based on their image quality and reproducibility on the simulator. It is important to note that the deep transgastric window was not available on the version of the simulator’s software used in this study. These TCPs corresponded to the midesophageal 4-chamber (ME-4C), ascending aortic short-axis, bicaval, right ventricular inflowoutflow, transcommissural, 2-chamber, and long-axis views, as well as the transgastric basal short-axis, long-axis, and right ventricular inflow views. The TCPs then were reviewed and agreed on by all investigators. Kinematic analysis was performed at the start of the course (at the end of the second training session) and at the end of the course (at the end of the eighth session). During the kinematic analysis, the subject was first asked to acquire the ME-4C view. Subsequently, ME-4C view was used as the starting position for each new image acquisition. From the ME-4C, each subject was asked to acquire an image resembling the requested TCP. During data acquisition, the TCP image was displayed next to the active scan plane (Fig 1). The simulator automatically recorded the real-time data, including the starting point and the change in position in the x, y, and z axes of the tip of the TEE probe over time intervals of o0.2 s. The data were exported from the simulator as a Microsoft Excel (Microsoft Corporation; Redmond, WA) spreadsheet, and the following data were extrapolated: Total time: time from the starting position until the TCP was acquired; path length: the sum of all linear and angular movements of the probe in centimeters; probe accelerations from rest: the number of times the probe acceleration exceeded 0.5 cm/s2; and time-distance integral: area under the curve of a plot of time (x-axis) versus distance travelled (y-axis). Evaluation in the Clinical Setting After the participants completed their TEE training program, they were taken to the operating room to perform a TEE examination on an anesthetized patient. The examinations were performed with a Phillips iE33 machine (Phillips Healthcare; Andover, MA) with an X7-2T transducer. First, an experienced echocardiographer acquired images in each patient, corresponding to the TCPs tested previously on the simulator, with a starting position in the ME-4C view. Afterwards, the trainees were asked to obtain the same images. Expert and novice examinations were not labeled on the screen and were coded for further analysis. The entire examination, as performed by the expert and trainees, was video captured using the screen capture system Epiphan DVI2USB 3 (Epiphan Systems, Inc.; Ottawa, Canada) and a Mac Mini (Apple Inc.; Cupertino, CA) computer. A blinded investigator (M.M.) analyzed the video-captured examinations. For each view, the time necessary to obtain it and the number of probe transitions from the reference image (ie, the ME-4C) to the desired image was noted. Statistical Analysis
Target Cut Planes and Kinematic Testing A single investigator (F.M.) acquired 10 target cut planes (TCP), or simulator reference images corresponding to a standard TEE image. The specific TCPs were selected by
The data were evaluated using SPSS version 21 for Mac OS. Knowledge and course satisfaction data, as well as the clinical setting evaluations and kinematic data for the skills evaluation sessions were assessed for differences using
3
KINEMATIC ANALYSIS AND TRANSESOPHAGEAL ECHO TRAINING
Fig 1. A target cut planes view corresponding to the midesophageal long axis (right) is shown on the screen simultaneously with the image obtained by the trainee (left). Figure reprinted with permission from CAE Healthcare, Montreal, Canada.
Wilcoxon matched-pairs signed ranks test. The alpha level was set at p o 0.05. For the evaluation of kinematic metrics, adjustment of p values for multiple comparisons was performed using the Holm-Bonferroni sequential correction.13 Data are presented as median (interquartile range) or mean ⫾ standard deviation where appropriate. RESULTS
Eleven participants were recruited (n ¼ 5 in year 1 and n ¼ 6 in year 2). All of them completed the knowledge and kinematic testing. Five participants were excluded from the evaluation in the clinical setting due to interim TEE exposure, having completed 45 TEE examinations, or having participated in a TEE rotation before clinical evaluation. The post-test scores were higher than the pretest scores (80.4% ⫾ 7.7 v 67.5% ⫾ 11.2; p o 0.05). There was a trend toward increased self-perceived knowledge of and comfort level with TEE at the end of the course compared with baseline, but it did not reach statistical significance. The participants also found the different Table 1. Knowledge Assessment and Perceived Usefulness of the Training Program Endpoint
Online knowledge test Comfort level with TEE Self-perceived knowledge with TEE Usefulness of the hands-on sessions Usefulness of the live lectures Usefulness of the online modules
modalities of the course useful, particularly the hands-on component (Table 1). The total time, path length, number of probe accelerations from rest and time-distance integral all decreased from the start to the end of the course (Table 2). These observations were consistent across different views (Fig 2). During the clinical evaluation portion of the study, all trainees were able to get all the views solicited without assistance. When compared with the expert, the time for acquiring each view was significantly higher for the trainees (18.3 ⫾ 9.2 s v 8.9 ⫾ 3.3 s; p ¼ 0.028). The same difference was true for the number of transitions per view (4.2 ⫾ 2.8 for the trainees, 2.1 ⫾ 0.8 for the experts, p ¼ 0.027) (Fig 3). DISCUSSION
The present study confirmed the utility of a curriculumbased approach to impart proficiency in performing TEE to cardiology fellows without previous exposure to TEE. A primarily web-based system was used to impart cognitive knowledge, and manual dexterity skills were acquired through training with a TEE simulator. The TEE-naïve trainees were able also to obtain standard views on an intraoperative TEE
Before the Course After the Course
67.48%/100% 2.9%/5% 3.1/5% — — —
80.43%/100% 3.6%/5% 3.6%/5% 4.4%/5% 3.2%/5% 3.6%/5%
Abbreviation: TEE, transesophageal echocardiography. Average score/ Maximum possible score
Table 2. Kinematic Measures Obtained at the Start and End of the Course (n ¼ 11 trainees) Metric
Probe accelerations (n) Time distance integral Total time (s) Path length (cm)
Start of Course
59 1.0 25.3 12.3
(31-130) (0.2-1.6) (16.7-38.4) (2.5-21.0)
End of Course
40 0.4 12.6 6.7
(21-64) (0.1-0.7) (8.9-16.4) (1.2-11.3)
Data are presented as median (interquartile range).
p Value
o0.001 o0.001 o0.001 o0.001
4
MATYAL ET AL
Fig 2. The number of probe accelerations from rest (panel A), total time to acquire a target cut plane (panel B), path length (panel C), and time-distance integral (panel D) are shown for each view at the start and end of the course. Abbreviations: Bicaval, midesophageal bicaval; ME LAX, midesophageal long axis; AV SAX, midesophageal aortic valve short axis; COMM, midesophageal commissural; ME 2CH, midesophageal 2 chamber; TG Basal, transgastric basal short axis; TG MID, transgastric mid short axis; TG RV INF, transgastric right ventricular inflow; TG LAX, transgastric long axis.
examination after this training without instructor assistance, thus demonstrating an element of clinical transferability. This curriculum may help address Accreditation Council Graduate Medical Education (ACGME) requirements, because some of the activities may be time-shifted to comply with duty hour limits. Additionally, the web-based component can be categorized as self-study time, which is not included in the
ACGME definition of duty hour. The authors also demonstrated the reproducibility of kinematic analyses in a group of trainees with a different background and level of experience than this initial report (ie, cardiology fellows as opposed to anesthesiology residents).11 Specifically, there was a significant reduction in total time to acquire the image, path length and the time-distance integral (Figs 4 and 5). Most importantly, there
Fig 3. The number of image transitions (panel A) of the trainees exceeded the expert’s. The total time required to obtain each view was significantly longer for the trainees when compared with the expert (panel B).
KINEMATIC ANALYSIS AND TRANSESOPHAGEAL ECHO TRAINING
5
Fig 4. 3D path course during acquisition of the transgastric basal short-axis view. On the first skill evaluation at the start of the course (panel A), a longer and more tortuous path is observed for the trainee when compared with the evaluation at the end of the course (panel B). The path for an expert is shown for reference (panel C).
was a significant decrease in TEE probe accelerations (“jerking motions”) over time with simulator training. The reductions in these variables describe an increased “economy of motion” after finalizing the training course. This is an important finding because it represents an increased ability to select the most adequate TEE probe motions, which correlate with a faster, more intuitive, and smoother image acquisition. During the clinical evaluation of trainees, the number of image transitions was used as a surrogate for peak motions of the TEE probe. Compared with experts, demonstration of more transitional images by the trainees implied that this was a potential metric to help differentiate novices
from experts. Smoothness of TEE probe motion implied intuitive motion and possibly represented a degree of automaticity. Automaticity is considered a predictor of clinical transferability.14,15 Although it was not specifically evaluated in the present study, TEE-naïve trainees demonstrated elements of clinical transferability. Despite being TEE naïve, they were able to perform an unassisted intraoperative TEE examination on their first attempt, although they had more transitions when compared with an expert. The trainees also required a longer time to perform image acquisition in the clinical setting when compared with an expert, although (approximately 9 seconds) may
Fig 5. Time-velocity plotting shows a greater number of probe jerks during acquisition of a transgastric long axis at the start of the course (panel A) when compared with the end of the course (panel B). The probe acceleration is the slope of the curve at any given time.
6
MATYAL ET AL
not be clinically significant. Because of current technologic limitations (ie, the magnetic properties of the TEE probe), electromagnetic sensors used with the simulator cannot be used to track movements of the TEE probe. Therefore, probe motion metrics could not be used during the intraoperative examination sequence. Kinematic analyses potentially could be used to establish a trainee’s readiness to perform. Acquisition of manual dexterity skills before clinical performance reduces the learning curve and may enhance the quality of educational experience and patient safety during training. Accredited surgical training programs have introduced mandatory laparoscopic simulator-based task training for residents before actual clinical exposure. In the Fundamentals of Laparoscopic Surgery (FLS) program, the trainees have to successfully complete a written examination for cognition along with pre-defined motor tasks on a simulator.16,17 In the Fundamentals of Endoscopic Surgery (FES) program, this simulator-based approach has been extended to endoscopy training.18 Trainee evaluation in the programs is endpointbased (ie, time required for task completion number of errors made by the trainee). In addition to endpoint-based assessment during TEE simulation training, the use of kinematic metrics allows for an objective assessment of the quality of motion. The curriculum-based approach of this study allowed integration of didactics with hands-on training without the unpredictability of clinical case presentation (eg, mitral valve lectures integrated with mitral valve views in cases of mitral valve prolapse on the simulator). Training in perioperative TEE highlights many of the logistic challenges in the medical education paradigm of time-based training. The availability of a virtual reality TEE
simulator and the application of metrics potentially can address these challenges. With the proposed model, the training can be graduated from simple to complex, cognitive learning can be integrated with manual dexterity, and acquisition of motor skills can be objectified. While it does not replace clinical exposure, simulation-based training has the advantages of being focused on the trainee rather than the patient, having no consequences of failure, and posing absolutely no risk to the patient.10 This potentially can reduce the learning curve and improve the quality of TEE education. The present study had some limitations. Firstly, the sample size was small, mainly because a decision was made to include trainees without prior exposure to TEE, which limited the sample. However, the results of the psychomotor skills testing were consistent among the individuals. Another potential limitation was the current lack of an ideal method to assess transferability of simulator-based skills into the clinical setting. Additionally, the study lacked a control group to validate the acquisition of skills during simulation training compared with conventional training. The study was designed to quantify the value of a curriculum-based approach with kinematic metrics, whereas the value of simulator training in improving manual dexterity already is established.19 CONCLUSIONS
It is feasible to implement an integrated curriculum-based approach for TEE training. TEE-naïve cardiology fellows significantly improved their cognitive knowledge via webbased education and their manual dexterity skills as determined with motion metrics. TEE-naïve trainees demonstrated some elements of clinical transferability after finishing the training program.
REFERENCES 1. Neelankavil J, Howard-Quijano K, Hsieh TC, et al: Transthoracic echocardiography simulation is an efficient method to train anesthesiologists in basic transthoracic echocardiography skills. Anesth Analg 115:1042-1051, 2012 2. Platts DG, Humphries J, Burstow DJ, et al: The use of computerised simulators for training of transthoracic and transoesophageal echocardiography. The future of echocardiographic training? Heart Lung Circ 21:267-274, 2012 3. Bose R, Matyal R, Panzica P, et al: Transesophageal echocardiography simulator: A new learning tool. J Cardiothorac Vasc Anesth 23: 544-548, 2009 4. Matyal R, Bose R, Warraich H, et al: Transthoracic echocardiographic simulator: Normal and the abnormal. J Cardiothorac Vasc Anesth 25:177-181, 2011 5. Sutherland C, Hashtrudi-Zaad K, Sellens R, et al: An augmented reality haptic training simulator for spinal needle procedures. IEEE Trans Biomed Eng 60:3009-3018, 2013 6. Fucentese SF, Rahm S, Wieser K, et al: Evaluation of a virtual-reality-based simulator using passive haptic feedback for knee arthroscopy. Knee Surg Sports Traumatol Arthrosc 23: 1077-1085, 2015 7. Yudkowsky R, Luciano C, Banerjee P, et al: Practice on an augmented reality/haptic simulator and library of virtual brains improves residents’ ability to perform a ventriculostomy. Simul Healthc 8:25-31, 2013
8. Barrow A, Akhtar K, Gupte C, et al: Requirements analysis of a 5 degree of freedom haptic simulator for orthopedic trauma surgery. Stud Health Technol Inform 184:43-47, 2013 9. Gaba D.: Simulator training in anesthesia growing rapidly. Available at: http://www.apsf.org/newsletters/html/1995/fall/simulator. html. Accessed February 18, 2015. 10. Shakil O, Mahmood F, Matyal R: Simulation in echocardiography: An ever-expanding frontier. J Cardiothorac Vasc Anesth 26: 476-485, 2012 11. Matyal R, Mitchell JD, Hess PE, et al: Simulator-based transesophageal echocardiographic training with motion analysis: A curriculum-based approach. Anesthesiology 121:389-399, 2014 12. Shanewise JS, Cheung AT, Aronson S, et al: ASE/SCA guidelines for performing a comprehensive intraoperative multiplane transesophageal echocardiography examination: Recommendations of the American Society of Echocardiography Council for Intraoperative Echocardiography and the Society of Cardiovascular Anesthesiologists Task Force for Certification in Perioperative Transesophageal Echocardiography. Anesth Analg 89:870-884, 2014 13. Gaetano J.: Holm-Bonferroni sequential Correction: An EXCEL calculator. Available at: http://www.researchgate.net/publication/242331583_ Holm-Bonferroni_Sequential_Correction_An_EXCEL_Calculator_-_Ver._1.2. Accessed May 12, 2015.
KINEMATIC ANALYSIS AND TRANSESOPHAGEAL ECHO TRAINING
7
14. Stefanidis D, Scerbo MW, Sechrist C, et al: Do novices display automaticity during simulator training? Am J Surg 195:210-213, 2008 15. Stefanidis D, Scerbo MW, Korndorffer JR, et al: Redefining simulator proficiency using automaticity theory. Am J Surg 193: 502-506, 2007 16. Okrainec A, Soper NJ, Swanstrom LL, et al: Trends and results of the first 5 years of Fundamentals of Laparoscopic Surgery (FLS) certification testing. Surg Endosc 25:1192-1198, 2011
17. Fundamentals of Laparoscopic Surgery. Available at: http:// www.flsprogram.org. Accessed December 10, 2014. 18. The Fundamentals of Endoscopic Surgery. Available at: http:// www.fesprogram.org. Accessed December 10, 2014. 19. Stefanidis D, Scerbo MW, Montero PN, et al: Simulator training to automaticity leads to improved skill transfer compared with traditional proficiency-based training: A randomized controlled trial. Ann Surg 255:30-37, 2012