- Email: [email protected]
A novel method of teaching surgical techniques to residents – Computerized enhanced visual learning (CEVL) with simulation to certify mastery of training: A model using newborn clamp circumcision
Journal of Pediatric Urology (2013) 9, 1210e1213
A novel method of teaching surgical techniques to residents e Computerized enhanced visual learning (CEVL) with simulation to certify mastery of training: A model using newborn clamp circumcision Angela Smith a, Max Maizels b, Ruslan Korets c, John S. Wiener d, Michael Stiener a, Dennis B. Liu e, Richard W. Sutherland a,* a
University of North Carolina School of Medicine, North Carolina, United States Children’s Memorial Hospital, Chicago, United States c Morgan Stanley Children’s Hospital of NY, Columbia Univ. College of Physicians & Surgeons, United States d Duke University School of Medicine, United States e University of Chicago Pritzker School of Medicine, United States b
Received 18 October 2012; accepted 14 May 2013 Available online 9 August 2013
KEYWORDS Residency; Education; Computer-assisted instruction; Circumcision
Abstract Objective: To assess the learning process of combining a web-based video of a simulated surgical procedure with a step-by-step checklist of the same procedure in achieving competency of the simulated technique, in this case a newborn clamp circumcision. Fundamental to this particular learning process is immediate mentor step-by-step feedback which specifically follows the procedure’s step-by-step checklist. Materials and methods: Pediatric residents naı¨ve to newborn circumcision were enrolled (n Z 7). A circumcision simulator, instruments, and web access to the learning module were provided. Residents trained independently and then performed two simulations with the mentor. The first simulation was completed with formative scored feedback. The learner then performed a second scored simulation. Results: All learners showed improvement between the first and second simulation (mean 85.3e 97.4). All residents achieved competency (96/100 or greater) by the second simulation. On post-
Abbreviations: CEVL, computer enhanced visual learning; RTP, Residency Training Program; F/R, feedback/remediation; ROSe, Recording Of Skills evaluation. * Corresponding author. E-mail address: [email protected] (R.W. Sutherland). 1477-5131/$36 ª 2013 Journal of Pediatric Urology Company. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jpurol.2013.05.012
Computerized enhanced visual learning with simulation
1211
procedure surveys, learners demonstrated increased comfort and reduced apprehension in performing the procedure. Conclusion: Combining a web-accessible video of a procedure, a checklist, and a simulator followed by a single mentor session with immediate formative feedback which follows the steps of the checklist is a useful method to teach the simulation technique of circumcision. We plan to study if this paradigm is transferable to clinical circumcision. ª 2013 Journal of Pediatric Urology Company. Published by Elsevier Ltd. All rights reserved.
Introduction There is little research on specific methods to improve efficiency and enjoyment to both the resident and mentor when learning how to perform a surgical procedure. While the Halstedian method of immersion learning has been the standard for over a century, current Residency Training Programs (RTP) are faced with restricted resident work hours as well as financial and legal constraints on the attending surgeon which may limit the resident’s participation in operative cases. RTPs may therefore benefit from the development of new instructional methods that are effective and time efficient for both the learners and mentors. Documentation of the results may provide an excellent tool for feedback and remediation of learners, and additionally provide an efficient method of learning. Furthermore, the American Committee on Graduate Medical Education Common Program Requirements mandates that all RTPs must “provide objective assessments of competence in patient care, medical knowledge, practicebased learning and improvement. and document progressive resident performance improvement.” (http://www. acgme.org/acWebsite/dutyHours/dh_dutyhoursCommonPR0 7012007.pdf). We assessed the paradigm of combining web-accessed materials (computerized enhanced visual learning (CEVL)) along with a step-by-step checklist with instructional simulation and immediate mentor feedback/remediation (F/R) to acquire competency of the index simulation, newborn clamp circumcision. As prior studies have reflected the benefits of the CEVL method (i.e. breaking a surgical procedure into multiple, individual components and developing a tool to teach and evaluate each individual components), we hypothesized that this would be applicable to the instruction of pediatric residents in the competency of newborn clamp circumcision.
Experimental/materials and methods Our objective was to apply this concept to a constructed simulation model which effectively duplicates the experience and technique of newborn clamp circumcision. The CEVL method encompasses repetitive cycles of 1) webbased access of the learning module, 2) preoperative study, 3) operative experience, and 4) postoperative feedback. The simulator created is a reusable silicone mold of an infant male penis 3 cm in length attached to a hard plastic base simulating the lower abdomen, thighs and scrotum. A prepuce is simulated with a specifically designed latex sleeve which is placed over the penis. Instruments used for the simulated circumcisions are the same as the instruments
used in the newborn nursery for clinical newborn circumcisions. The attending surgeons designed the CEVL curriculum for the simulation of newborn clamp circumcision by agreeing on the series of specific steps/skills involved in each component of the procedure. Included in the curriculum is a simulated preoperative evaluation of the patient’s anatomy, anesthetic block, sterilization, draping, and instrument layout as well as the actual 23 steps of the circumcision procedure. Photographs and video of the actual procedure were annotated to demonstrate landmarks. Voiceovers and surgeon specific technical variations were included. The video with accompanying information was available independently to the resident to study the tutorial curriculum prior to the simulation with the mentor surgeon. In the subsequent sessions, individual residents were mentored oneon-one while performing the procedure on the simulator. At the completion of the first simulation, formative feedback with scoring was carried out according to the expected steps to be performed. The mentor and resident convened for feedback on each specific step/skill using a 10-point Likert scale. The scale ranges from: 1dinability to complete the task with assistance to 10dcompetent to perform independently. The mentor rated the resident on the circumcision simulation without regard for the level of training. The final score for each step is made by the mentor. Specific points of critique were recorded to guide the resident toward improved performance during the second iteration of simulation. The cycle of simulation and feedback was then repeated. This cycle of simulationfeedback-simulation-feedback is continued until competency is acquired and documented by score of 96. Appendix A shows an actual ROSe form completed during the feedback session using the CEVL curriculum. The form could be completed electronically or manually during the session depending on Mentor/Learner preference. Pediatric residents (n Z 7) naı¨ve to newborn circumcision at our institution were enrolled in the CEVL curriculum between February and March of 2011. A customized circumcision simulator (CEVL), surgical instruments, and access to the web-based training (CEVL) were provided to the residents. The residents trained independently for up to 1 week and subsequently performed two circumcision simulations with the mentor (RS). The CEVL simulation session included two simulated circumcisions with mentor feedback during and immediately after each circumcision. The first circumcision simulation was completed with feedback and remediation within 30 min. The learner internalized this feedback and remediation, and then in the same setting performed the second simulated circumcision. The skills shown by the learner during both simulation surgeries were scored on a Likert scale as described previously. In
1212
A. Smith et al.
addition, the progress of each resident and their comfort level with the procedure was established by administration of pre-simulation and post-simulation circumcision surveys. Similar surveys were completed by the mentor as well. This study was exempted by the IRB at UNC.
Results Initial scores from the first simulation performed averaged 85.3/100 with a range of 73e93/100. Scores of the second simulated circumcision after the first iteration of the CEVL process increased to 97.4/100 with a range of 96e99/100. All learners showed improvement of the CEVL score between the first and second simulated circumcision. Furthermore, all learners achieved competency with a score of 96/100 or greater by the second simulation. Fig. 1 demonstrates pre and post CEVL ROSe scores in raw form. Appendix A shows a representative score sheet obtained during feedback with the learner. Time commitment for the learner and mentor varied between 45 mine60 min for the total simulation session. All learners were given access to the module and simulators at least 1 week before the mentor session. All sessions were completed within 3 weeks. The amount of time spent by each learner before the session was not tracked. Pre- and post-CEVL surveys highlighted learners’ increased subjective comfort in preparing to perform the procedure as well as reduced apprehension for performing circumcision. All participants believed CEVL to be helpful in learning the individual steps of the procedure. The mentor’s survey highlighted the increased preparation from the learners, improved organization to the learners’ flow of the procedure, and better ability to understand errors in technique.
Discussion As documented in prior research, adult learning has been described according to 5 principles [1,2]: 1) Adults are independent and self-directing. 2) They bring a range of experience to the table. 3) Adults value learning integrated into everyday life. 4) Problem-centered approaches gain more interest. 5) Learners are more motivated by their
interests and desires than external pressure. CEVL has been noted to meet many (if not all) of these guidelines [1]. This web-based circumcision module breaks down the procedure into discrete, easily identified steps and offers an efficient platform for formative feedback. The specific and constructive post-procedure feedback on resident performance provides a basis for further study and improvement. It is the step-by-step, post-procedure feedback that we believe is fundamental to the success of the CEVL technique of learning a procedure. The learner is allowed to focus on a particular portion of the procedure, comprehend and internalize the mentor’s feedback for that portion, and then move onto the next step to learn. This format also creates a learning environment where the mentor gives feedback in an organized step-by-step progression. This reduces the risk of moving forward and backward through the steps of the procedure. We believe a learner will learn more efficiently when given feedback in this manner than moving between steps of a procedure in a non-linear fashion. This is an area of future research, as we compare the CEVL technique to the historical model of “See one, do one, teach one.” This study reveals that this method can be used in multiple disciplines, and does not require specific training on behalf of the attending staff. Furthermore, this simulator has been well received by those residents who completed the post-CEVL survey. Results revealed that pediatric residents who used CEVL for newborn circumcision simulation attained competency in the procedure with a single repeated iteration, as documented objectively by an average increase of 12.1 units. Survey results furthermore suggested that residents have a wide range of opinions on the best way to learn surgery, but all were comfortable with the CEVL method, finding it to be helpful in learning, reducing apprehension, and increasing comfort with the procedure steps. We utilized learner satisfaction from surveys to assess our CEVL learning tool, which has been shown to be a valid outcome in this regard [3,4]. Although this study had a limited number of subjects (n Z 7), we believe that it will be widely applicable to teaching circumcision to residents in multiple RTPs (including family medicine as well as pediatrics, Ob/Gyn, and urology). Additional limitations of the study include the lack of a control group to compare those who used the CEVL simulator and those who did not, as well as a lack of applicability to a true clinical scenario (newborn). These limitations will be addressed in the future as we expand our learner pool and advance the study to its next phase, incorporating CEVL circumcision simulation training with clinical newborn circumcision performance.
Conclusions
Figure 1
Pre- and Post-CEVL ROSe scores.
Combining a web-accessible checklist procedure (CEVL) for simulation training along with simulation practice was shown to be a useful and feasible method to teach circumcision. Residents showed improved technique and increased comfort with the procedure. Based upon these results, we plan to study if this paradigm is transferable to other institutions. Furthermore, we will ultimately examine whether the CEVL paradigm will improve the performance of clinical circumcision.
Computerized enhanced visual learning with simulation
1213
Conflict of interest
References
None.
[1] McQuiston L, Macneily A, Liu D, Mickelson J, Yerkes E, Chaviano A, et al. Computer enhanced visual learning method to train urology residents in pediatric orchiopexy provided a consistent learning experience in a multi-institutional trial. J Urol 2010;184:1748e53. [2] Stagnaro-Green A. Applying adult learning principles to medical education in the United States. Med Teach 2004;26:79e85. [3] Berman NB, Fall LH, Maloney CG, Levine DA. Computer-assisted instruction in clinical education: a roadmap to increasing CAI implementation. Adv Health Sci Educ Theory Pract 2008;13: 373e83. [4] Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med 2006;81:207e12.
Funding Funding for this research was given through a $5000 grant from the Children’s Hospital of North Carolina.
Appendix A. Supplementary data Supplementary data related to this article can be found online at http://dx.doi.org/10.1016/j.jpurol.2013.05.012.
A novel method of teaching surgical techniques to residents e Computerized enhanced visual learning (CEVL) with simulation to certify mastery of training: A model using newborn clamp circumcision Angela Smith a, Max Maizels b, Ruslan Korets c, John S. Wiener d, Michael Stiener a, Dennis B. Liu e, Richard W. Sutherland a,* a
University of North Carolina School of Medicine, North Carolina, United States Children’s Memorial Hospital, Chicago, United States c Morgan Stanley Children’s Hospital of NY, Columbia Univ. College of Physicians & Surgeons, United States d Duke University School of Medicine, United States e University of Chicago Pritzker School of Medicine, United States b
Received 18 October 2012; accepted 14 May 2013 Available online 9 August 2013
KEYWORDS Residency; Education; Computer-assisted instruction; Circumcision
Abstract Objective: To assess the learning process of combining a web-based video of a simulated surgical procedure with a step-by-step checklist of the same procedure in achieving competency of the simulated technique, in this case a newborn clamp circumcision. Fundamental to this particular learning process is immediate mentor step-by-step feedback which specifically follows the procedure’s step-by-step checklist. Materials and methods: Pediatric residents naı¨ve to newborn circumcision were enrolled (n Z 7). A circumcision simulator, instruments, and web access to the learning module were provided. Residents trained independently and then performed two simulations with the mentor. The first simulation was completed with formative scored feedback. The learner then performed a second scored simulation. Results: All learners showed improvement between the first and second simulation (mean 85.3e 97.4). All residents achieved competency (96/100 or greater) by the second simulation. On post-
Abbreviations: CEVL, computer enhanced visual learning; RTP, Residency Training Program; F/R, feedback/remediation; ROSe, Recording Of Skills evaluation. * Corresponding author. E-mail address: [email protected] (R.W. Sutherland). 1477-5131/$36 ª 2013 Journal of Pediatric Urology Company. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jpurol.2013.05.012
Computerized enhanced visual learning with simulation
1211
procedure surveys, learners demonstrated increased comfort and reduced apprehension in performing the procedure. Conclusion: Combining a web-accessible video of a procedure, a checklist, and a simulator followed by a single mentor session with immediate formative feedback which follows the steps of the checklist is a useful method to teach the simulation technique of circumcision. We plan to study if this paradigm is transferable to clinical circumcision. ª 2013 Journal of Pediatric Urology Company. Published by Elsevier Ltd. All rights reserved.
Introduction There is little research on specific methods to improve efficiency and enjoyment to both the resident and mentor when learning how to perform a surgical procedure. While the Halstedian method of immersion learning has been the standard for over a century, current Residency Training Programs (RTP) are faced with restricted resident work hours as well as financial and legal constraints on the attending surgeon which may limit the resident’s participation in operative cases. RTPs may therefore benefit from the development of new instructional methods that are effective and time efficient for both the learners and mentors. Documentation of the results may provide an excellent tool for feedback and remediation of learners, and additionally provide an efficient method of learning. Furthermore, the American Committee on Graduate Medical Education Common Program Requirements mandates that all RTPs must “provide objective assessments of competence in patient care, medical knowledge, practicebased learning and improvement. and document progressive resident performance improvement.” (http://www. acgme.org/acWebsite/dutyHours/dh_dutyhoursCommonPR0 7012007.pdf). We assessed the paradigm of combining web-accessed materials (computerized enhanced visual learning (CEVL)) along with a step-by-step checklist with instructional simulation and immediate mentor feedback/remediation (F/R) to acquire competency of the index simulation, newborn clamp circumcision. As prior studies have reflected the benefits of the CEVL method (i.e. breaking a surgical procedure into multiple, individual components and developing a tool to teach and evaluate each individual components), we hypothesized that this would be applicable to the instruction of pediatric residents in the competency of newborn clamp circumcision.
Experimental/materials and methods Our objective was to apply this concept to a constructed simulation model which effectively duplicates the experience and technique of newborn clamp circumcision. The CEVL method encompasses repetitive cycles of 1) webbased access of the learning module, 2) preoperative study, 3) operative experience, and 4) postoperative feedback. The simulator created is a reusable silicone mold of an infant male penis 3 cm in length attached to a hard plastic base simulating the lower abdomen, thighs and scrotum. A prepuce is simulated with a specifically designed latex sleeve which is placed over the penis. Instruments used for the simulated circumcisions are the same as the instruments
used in the newborn nursery for clinical newborn circumcisions. The attending surgeons designed the CEVL curriculum for the simulation of newborn clamp circumcision by agreeing on the series of specific steps/skills involved in each component of the procedure. Included in the curriculum is a simulated preoperative evaluation of the patient’s anatomy, anesthetic block, sterilization, draping, and instrument layout as well as the actual 23 steps of the circumcision procedure. Photographs and video of the actual procedure were annotated to demonstrate landmarks. Voiceovers and surgeon specific technical variations were included. The video with accompanying information was available independently to the resident to study the tutorial curriculum prior to the simulation with the mentor surgeon. In the subsequent sessions, individual residents were mentored oneon-one while performing the procedure on the simulator. At the completion of the first simulation, formative feedback with scoring was carried out according to the expected steps to be performed. The mentor and resident convened for feedback on each specific step/skill using a 10-point Likert scale. The scale ranges from: 1dinability to complete the task with assistance to 10dcompetent to perform independently. The mentor rated the resident on the circumcision simulation without regard for the level of training. The final score for each step is made by the mentor. Specific points of critique were recorded to guide the resident toward improved performance during the second iteration of simulation. The cycle of simulation and feedback was then repeated. This cycle of simulationfeedback-simulation-feedback is continued until competency is acquired and documented by score of 96. Appendix A shows an actual ROSe form completed during the feedback session using the CEVL curriculum. The form could be completed electronically or manually during the session depending on Mentor/Learner preference. Pediatric residents (n Z 7) naı¨ve to newborn circumcision at our institution were enrolled in the CEVL curriculum between February and March of 2011. A customized circumcision simulator (CEVL), surgical instruments, and access to the web-based training (CEVL) were provided to the residents. The residents trained independently for up to 1 week and subsequently performed two circumcision simulations with the mentor (RS). The CEVL simulation session included two simulated circumcisions with mentor feedback during and immediately after each circumcision. The first circumcision simulation was completed with feedback and remediation within 30 min. The learner internalized this feedback and remediation, and then in the same setting performed the second simulated circumcision. The skills shown by the learner during both simulation surgeries were scored on a Likert scale as described previously. In
1212
A. Smith et al.
addition, the progress of each resident and their comfort level with the procedure was established by administration of pre-simulation and post-simulation circumcision surveys. Similar surveys were completed by the mentor as well. This study was exempted by the IRB at UNC.
Results Initial scores from the first simulation performed averaged 85.3/100 with a range of 73e93/100. Scores of the second simulated circumcision after the first iteration of the CEVL process increased to 97.4/100 with a range of 96e99/100. All learners showed improvement of the CEVL score between the first and second simulated circumcision. Furthermore, all learners achieved competency with a score of 96/100 or greater by the second simulation. Fig. 1 demonstrates pre and post CEVL ROSe scores in raw form. Appendix A shows a representative score sheet obtained during feedback with the learner. Time commitment for the learner and mentor varied between 45 mine60 min for the total simulation session. All learners were given access to the module and simulators at least 1 week before the mentor session. All sessions were completed within 3 weeks. The amount of time spent by each learner before the session was not tracked. Pre- and post-CEVL surveys highlighted learners’ increased subjective comfort in preparing to perform the procedure as well as reduced apprehension for performing circumcision. All participants believed CEVL to be helpful in learning the individual steps of the procedure. The mentor’s survey highlighted the increased preparation from the learners, improved organization to the learners’ flow of the procedure, and better ability to understand errors in technique.
Discussion As documented in prior research, adult learning has been described according to 5 principles [1,2]: 1) Adults are independent and self-directing. 2) They bring a range of experience to the table. 3) Adults value learning integrated into everyday life. 4) Problem-centered approaches gain more interest. 5) Learners are more motivated by their
interests and desires than external pressure. CEVL has been noted to meet many (if not all) of these guidelines [1]. This web-based circumcision module breaks down the procedure into discrete, easily identified steps and offers an efficient platform for formative feedback. The specific and constructive post-procedure feedback on resident performance provides a basis for further study and improvement. It is the step-by-step, post-procedure feedback that we believe is fundamental to the success of the CEVL technique of learning a procedure. The learner is allowed to focus on a particular portion of the procedure, comprehend and internalize the mentor’s feedback for that portion, and then move onto the next step to learn. This format also creates a learning environment where the mentor gives feedback in an organized step-by-step progression. This reduces the risk of moving forward and backward through the steps of the procedure. We believe a learner will learn more efficiently when given feedback in this manner than moving between steps of a procedure in a non-linear fashion. This is an area of future research, as we compare the CEVL technique to the historical model of “See one, do one, teach one.” This study reveals that this method can be used in multiple disciplines, and does not require specific training on behalf of the attending staff. Furthermore, this simulator has been well received by those residents who completed the post-CEVL survey. Results revealed that pediatric residents who used CEVL for newborn circumcision simulation attained competency in the procedure with a single repeated iteration, as documented objectively by an average increase of 12.1 units. Survey results furthermore suggested that residents have a wide range of opinions on the best way to learn surgery, but all were comfortable with the CEVL method, finding it to be helpful in learning, reducing apprehension, and increasing comfort with the procedure steps. We utilized learner satisfaction from surveys to assess our CEVL learning tool, which has been shown to be a valid outcome in this regard [3,4]. Although this study had a limited number of subjects (n Z 7), we believe that it will be widely applicable to teaching circumcision to residents in multiple RTPs (including family medicine as well as pediatrics, Ob/Gyn, and urology). Additional limitations of the study include the lack of a control group to compare those who used the CEVL simulator and those who did not, as well as a lack of applicability to a true clinical scenario (newborn). These limitations will be addressed in the future as we expand our learner pool and advance the study to its next phase, incorporating CEVL circumcision simulation training with clinical newborn circumcision performance.
Conclusions
Figure 1
Pre- and Post-CEVL ROSe scores.
Combining a web-accessible checklist procedure (CEVL) for simulation training along with simulation practice was shown to be a useful and feasible method to teach circumcision. Residents showed improved technique and increased comfort with the procedure. Based upon these results, we plan to study if this paradigm is transferable to other institutions. Furthermore, we will ultimately examine whether the CEVL paradigm will improve the performance of clinical circumcision.
Computerized enhanced visual learning with simulation
1213
Conflict of interest
References
None.
[1] McQuiston L, Macneily A, Liu D, Mickelson J, Yerkes E, Chaviano A, et al. Computer enhanced visual learning method to train urology residents in pediatric orchiopexy provided a consistent learning experience in a multi-institutional trial. J Urol 2010;184:1748e53. [2] Stagnaro-Green A. Applying adult learning principles to medical education in the United States. Med Teach 2004;26:79e85. [3] Berman NB, Fall LH, Maloney CG, Levine DA. Computer-assisted instruction in clinical education: a roadmap to increasing CAI implementation. Adv Health Sci Educ Theory Pract 2008;13: 373e83. [4] Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med 2006;81:207e12.
Funding Funding for this research was given through a $5000 grant from the Children’s Hospital of North Carolina.
Appendix A. Supplementary data Supplementary data related to this article can be found online at http://dx.doi.org/10.1016/j.jpurol.2013.05.012.