- Email: [email protected]
Resident Surgical Competency
Letters to the Editor sessions of 2 hours on the EYESI. At the end of the study, all participants again were evaluated with the CLARA. Changes in participant CLARA scores from the beginning of the study to the end were compared between the simulation and control groups using differences in means and the nonparametric Mann–Whitney test. Despite randomization, baseline performance of the control group was better on average than that of the simulation group in all measured categories of the CLARA. Likewise, overall scores in the control group were higher (control group, 36.00 [standard deviation (SD), 16.2]; simulation group, 25.38 [SD, 12.7]) (Table 1 [available at http:// aaojournal.org]). In the second round of testing, the simulation group performed more similarly to the control group (41.75 [SD, 12.30]; simulation group, 34.30 [SD, 11.40]) (Table 2 [available at http://aaojournal.org]). The simulation group improved more than the control group in 4 of the 6 CLARA categories and had a 60% greater improvement in overall CLARA score (control group, 5.63 [SD, 8.82]; simulation group, 8.92 [SD, 14.8]) (Table 3 [available at http:// aaojournal.org]). Statistical significance was not reached in individual score categories or in overall CLARA scores. This preliminary investigation suggests a trend towards enhanced acquisition of microsurgical skill in students allowed to practice microsurgery on the EYESI ophthalmosurgical simulator. It is possible that this trend was due to the sampling error after randomization that resulted in higher baseline microsurgical performance scores in the control group than in the simulator group. We believe that this trend warrants examination in a larger randomized study powered to establish statistical significance. We have calculated that to show a significant difference in performance we will need a sample size of approximately 74 participants. The George Washington Department of Ophthalmology has begun to recruit participants for this follow-up study. BRAD H. FELDMAN, MD JENNIFER M. AKE, MD CRAIG E. GEIST, MD, MS Washington, DC References 1. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458 – 64. 2. Seymour NE. Integrating simulation into a busy residency program. Minim Invasive Ther Allied Technol 2005;14: 280 – 6. 3. Hart R, Doherty DA, Karthigasu K, Garry R. The value of virtual-reality training in the development of laparoscopic surgical skills. J Minim Invasive Gynecol 2006;13:126 –33. 4. Hassan I, Maschuw K, Rothmund M, et al. Novices in surgery are the target group of a virtual reality training laboratory. Eur Surg Res 2006;38:109 –13.
Resident Surgical Competency Dear Editor: We read with interest Drs Binenbaum and Volpe’s article1 highlighting a common situation faced by most residency
programs, that of how to manage trainees who struggle with surgical competence. As residency programs across the country are attempting to develop assessment methods of their own residents’ surgical performance, the idea of a standardized surgical curriculum (not just a standard minimum caseload) as a means to promote and gauge competence is worth considering. We are proposing several possibilities to standardize cataract surgical training with the goal of improving the educational experience. In the United Kingdom, all beginning residents are required to attend a 3-day microsurgical skills training course. If a standardized cataract surgical training course was developed that incorporated what the programs deemed essential knowledge, then this course could provide all trainees with a backbone of knowledge regardless of the size or financial resources of their home program. After such a course, we believe the programs should institute a mandatory number of practice sessions in a wet laboratory facility that must occur before participation in actual surgery. Most trainees admit they do not fully utilize practice opportunities unless the sessions are deemed required. Also, these wet laboratory sessions are more effective if staffed by an experienced surgical preceptor and include a curriculum with a checklist of required components.2 Programs should implement standardized objective and subjective assessment tools to follow trainees’ surgical performance on a regular basis. These tools can be utilized to follow complication rates and surgical volume and can alert the program director to struggling trainees. In this manner, early intervention can occur to assist in finding solutions to the problems. As Drs Binenbaum and Volpe reported, there is much controversy about the use of applicant screening tests as a predictor of surgical skill. Benjamin described surgical clinical practice as “decision making combined with a flexibility of approach that comes from surgical experience” and said that assessing applicants at the “beginning of their careers may have little bearing on whether they turn out to be competent practitioners.”3 Alternatively, Bann et al proposed the use of a test evaluating the trainee’s ability to detect surgical errors as a valid way of predicting surgical skill and performance.4 From my own experience as a surgical preceptor, I have noticed a good predictor of surgical skill is the ability of the trainee to be a surgical first assistant. The mark of a good first assistant is someone who understands what is happening in the surgery, anticipates every step, pays attention to detail, and is prepared for potential problems. Rather than an applicant entry test, an evaluation of the trainee as a surgical first assistant may be more predictive of surgical aptitude. Another suggestion is for the surgical preceptors to be more properly trained. Currently, no standard or formal courses are offered for surgical preceptors in the United States. The Royal College of Surgeons offers a “training the trainers” course. The creation of a similar course that is funded for all surgical educators at a meeting such as the Educate the Educators session before the Association of University Professors of Ophthalmology annual meeting
829
Ophthalmology
Volume 114, Number 4, April 2007
may improve the quality of teaching for all programs and offer an open forum for discussion of surgical issues. Another approach for managing struggling trainees is to adapt the Horvath system, which was developed at the University of Washington’s department of surgery.5 The Horvath EVAT (emergency coverage, vacation coverage, academic time and Accreditation Council for Graduate Medical Education competency learning, assessment of technical skills training) system is a dedicated education rotation. A similar type of rotation could be implemented as part of the standard curriculum whereby any struggling resident could use this time for more practice wet laboratory sessions or other skills training. Nonstruggling trainees could utilize it to pursue other academic projects. In this manner, the struggling trainee would not suffer the stigma of needing additional time. Finally, the holy grail of standardized surgical assessment would be an effective computerized virtual reality system that could be used both to teach surgical procedures and to assess surgical competence in a meaningful and valid manner. Although these systems are being developed, I believe this final frontier in virtual reality may be several years away. BONNIE AN HENDERSON, MD RASHA ALI, BA JAE YONG KIM, MD, PHD Boston, Massachusetts References 1. Binenbaum G, Volpe NJ. Ophthalmology resident surgical competency. A national survey. Ophthalmology 2006;113: 1237– 44. 2. Smith JH. Teaching phacoemulsification in US ophthalmology residencies: can the quality be maintained? Curr Opin Ophthalmol 2005;16:27–32. 3. Benjamin L. Selection, teaching and training in ophthalmology. Clin Experiment Ophthalmol 2005;33:524 –30. 4. Bann S, Khan M, Datta V, Darzi A. Surgical skill is predicted by the ability to detect errors. Am J Surg 2005;189:412–5. 5. Horvath KD, Mann GN, Pellegrini C. EVATS: a proactive solution to improve surgical education and maintain flexibility in the new training era. Curr Surg 2006;63:151– 4.
830
Author reply Dear Editor: We appreciate the suggestions of Henderson et al for standardizing cataract surgical training. Certainly, mandatory microsurgical laboratory training experiences, which employ an organized curriculum with explicit goals and objectives, validated and reliable assessment tools, meaningful hands-on instruction, and ready access and time for independent practice, will become an increasingly important element of ophthalmology resident surgical training. We do not think, however, that programs must necessarily mandate a specific number of practice sessions. The focus should remain on competency goals achieved rather than experience obtained. Similarly, ongoing formative assessments of resident surgical skills, though essential, should not be centered on complication rates and surgical volume but rather upon surgical competency milestones met, whether they be specific manual skills, knowledge of instruments and equipment, or the development of sound intraoperative judgment, for example. The key point is a change in educational paradigm from tracking experience to measuring and certifying competency. In a true competency-based educational system, residents would be given the resources, opportunities, and guidance to work towards achieving predefined clinical competencies at their own pace rather than in competition with each other or according to a rigid timeline. As Henderson et al suggest, formal faculty training and curricular collaborations across programs would be a great help in working towards such an ideal, as would elective rotations that introduce more flexibility into the 36 months of training currently required by the Accreditation Council for Graduate Medical Education and American Board of Ophthalmology, and as will increasingly more realistic virtual reality surgical simulations, whose widespread use in residency and fellowship training programs may be less far off than we think. GIL BINENBAUM, MD NICHOLAS J. VOLPE, MD Philadelphia, Pennsylvania
Resident Surgical Competency Dear Editor: We read with interest Drs Binenbaum and Volpe’s article1 highlighting a common situation faced by most residency
programs, that of how to manage trainees who struggle with surgical competence. As residency programs across the country are attempting to develop assessment methods of their own residents’ surgical performance, the idea of a standardized surgical curriculum (not just a standard minimum caseload) as a means to promote and gauge competence is worth considering. We are proposing several possibilities to standardize cataract surgical training with the goal of improving the educational experience. In the United Kingdom, all beginning residents are required to attend a 3-day microsurgical skills training course. If a standardized cataract surgical training course was developed that incorporated what the programs deemed essential knowledge, then this course could provide all trainees with a backbone of knowledge regardless of the size or financial resources of their home program. After such a course, we believe the programs should institute a mandatory number of practice sessions in a wet laboratory facility that must occur before participation in actual surgery. Most trainees admit they do not fully utilize practice opportunities unless the sessions are deemed required. Also, these wet laboratory sessions are more effective if staffed by an experienced surgical preceptor and include a curriculum with a checklist of required components.2 Programs should implement standardized objective and subjective assessment tools to follow trainees’ surgical performance on a regular basis. These tools can be utilized to follow complication rates and surgical volume and can alert the program director to struggling trainees. In this manner, early intervention can occur to assist in finding solutions to the problems. As Drs Binenbaum and Volpe reported, there is much controversy about the use of applicant screening tests as a predictor of surgical skill. Benjamin described surgical clinical practice as “decision making combined with a flexibility of approach that comes from surgical experience” and said that assessing applicants at the “beginning of their careers may have little bearing on whether they turn out to be competent practitioners.”3 Alternatively, Bann et al proposed the use of a test evaluating the trainee’s ability to detect surgical errors as a valid way of predicting surgical skill and performance.4 From my own experience as a surgical preceptor, I have noticed a good predictor of surgical skill is the ability of the trainee to be a surgical first assistant. The mark of a good first assistant is someone who understands what is happening in the surgery, anticipates every step, pays attention to detail, and is prepared for potential problems. Rather than an applicant entry test, an evaluation of the trainee as a surgical first assistant may be more predictive of surgical aptitude. Another suggestion is for the surgical preceptors to be more properly trained. Currently, no standard or formal courses are offered for surgical preceptors in the United States. The Royal College of Surgeons offers a “training the trainers” course. The creation of a similar course that is funded for all surgical educators at a meeting such as the Educate the Educators session before the Association of University Professors of Ophthalmology annual meeting
829
Ophthalmology
Volume 114, Number 4, April 2007
may improve the quality of teaching for all programs and offer an open forum for discussion of surgical issues. Another approach for managing struggling trainees is to adapt the Horvath system, which was developed at the University of Washington’s department of surgery.5 The Horvath EVAT (emergency coverage, vacation coverage, academic time and Accreditation Council for Graduate Medical Education competency learning, assessment of technical skills training) system is a dedicated education rotation. A similar type of rotation could be implemented as part of the standard curriculum whereby any struggling resident could use this time for more practice wet laboratory sessions or other skills training. Nonstruggling trainees could utilize it to pursue other academic projects. In this manner, the struggling trainee would not suffer the stigma of needing additional time. Finally, the holy grail of standardized surgical assessment would be an effective computerized virtual reality system that could be used both to teach surgical procedures and to assess surgical competence in a meaningful and valid manner. Although these systems are being developed, I believe this final frontier in virtual reality may be several years away. BONNIE AN HENDERSON, MD RASHA ALI, BA JAE YONG KIM, MD, PHD Boston, Massachusetts References 1. Binenbaum G, Volpe NJ. Ophthalmology resident surgical competency. A national survey. Ophthalmology 2006;113: 1237– 44. 2. Smith JH. Teaching phacoemulsification in US ophthalmology residencies: can the quality be maintained? Curr Opin Ophthalmol 2005;16:27–32. 3. Benjamin L. Selection, teaching and training in ophthalmology. Clin Experiment Ophthalmol 2005;33:524 –30. 4. Bann S, Khan M, Datta V, Darzi A. Surgical skill is predicted by the ability to detect errors. Am J Surg 2005;189:412–5. 5. Horvath KD, Mann GN, Pellegrini C. EVATS: a proactive solution to improve surgical education and maintain flexibility in the new training era. Curr Surg 2006;63:151– 4.
830
Author reply Dear Editor: We appreciate the suggestions of Henderson et al for standardizing cataract surgical training. Certainly, mandatory microsurgical laboratory training experiences, which employ an organized curriculum with explicit goals and objectives, validated and reliable assessment tools, meaningful hands-on instruction, and ready access and time for independent practice, will become an increasingly important element of ophthalmology resident surgical training. We do not think, however, that programs must necessarily mandate a specific number of practice sessions. The focus should remain on competency goals achieved rather than experience obtained. Similarly, ongoing formative assessments of resident surgical skills, though essential, should not be centered on complication rates and surgical volume but rather upon surgical competency milestones met, whether they be specific manual skills, knowledge of instruments and equipment, or the development of sound intraoperative judgment, for example. The key point is a change in educational paradigm from tracking experience to measuring and certifying competency. In a true competency-based educational system, residents would be given the resources, opportunities, and guidance to work towards achieving predefined clinical competencies at their own pace rather than in competition with each other or according to a rigid timeline. As Henderson et al suggest, formal faculty training and curricular collaborations across programs would be a great help in working towards such an ideal, as would elective rotations that introduce more flexibility into the 36 months of training currently required by the Accreditation Council for Graduate Medical Education and American Board of Ophthalmology, and as will increasingly more realistic virtual reality surgical simulations, whose widespread use in residency and fellowship training programs may be less far off than we think. GIL BINENBAUM, MD NICHOLAS J. VOLPE, MD Philadelphia, Pennsylvania