- Email: [email protected]
Development of a valid, cost-effective laparoscopic training program
The American Journal of Surgery 187 (2004) 157–163
Surgical education
Development of a valid, cost-effective laparoscopic training program G.L. Adrales, U.B. Chu, J.D. Hoskins, D.B. Witzke, A.E. Park, M.D.* Department of Surgery, Minimally Invasive Center, University of Kentucky College of Medicine, Lexington, KY, USA Manuscript received December 2, 2002; revised manuscript July 4, 2003
Abstract Background: Practical programs for training and evaluating surgeons in laparoscopy are needed to keep pace with demand for minimally invasive surgery. Methods: At the University of Kentucky five inexpensive simulations have been developed to train and assess surgical residents. Residents are videotaped performing laparoscopic procedures on models. Five surgeons assess the taped performances on 4 global skills. Results: Creating mechanical models reduces training costs. Trainees agreed procedures were well represented by the simulations. Blinded assessment of performances showed high interrater agreement and correlated with the trainees’ level of experience. Nonclinician evaluations on checklists correlated with evaluations by surgeons. Conclusions: Inexpensive simulations of laparoscopic appendectomy, cholecystectomy, inguinal herniorrhaphy, bowel enterotomy, and splenectomy enable surgical residents to practice laparoscopic skills safely. Obtaining masked, objective, and independent evaluations of basic skills in laparoscopic surgery can assist in reliable assessment of surgical trainees. The simulations described can anchor an innovative educational program during residency for training and assessment. © 2004 Excerpta Medica, Inc. All rights reserved. Keywords: Assess; Simulation; Minimally invasive surgery; Medical education; Surgical training; Laparoscopy
The revolutionary development of minimally invasive surgery has fused new technology and the practice of medicine to achieve remarkable benefits in terms of patient recovery and healing. This transformation in healthcare has not been without cost. Incorporation of these advances into surgical practice occurred before important questions about training and credentialing were fully addressed. Widespread incorporation of laparoscopic cholecystectomy into the practices of surgeons occurred without the support of a randomized, clinical trial. By 1992, barely 4 years after the first laparoscopic cholecystectomy was performed in the United States, more than 80% of US surgeons had adopted this technique [1]. One of the consequences of this phenomenon was an early rise in biliary injuries after the adoption of the laparoscopic approach to cholecystectomy. While the absolute necessity of a clinical trial may be questionable, given the apparent benefits of this laparoscopic procedure to patients
* Corresponding author. Tel.: ⫹1-859-323-6346; fax: ⫹1-859-3236840. E-mail address: [email protected]
[2], the need for training and credentialing of surgeons in laparoscopy cannot be ignored. Before the incorporation of laparoscopic skills training into surgical education, the effectiveness of such training must be established. Several investigators have addressed this problem. Derossis and Fried [3,4] demonstrated the concurrent validity of a mechanical laparoscopic simulator. Subjects who had practiced on this inanimate simulator demonstrated superior performance in corresponding laparoscopic procedures in an animal model. Virtual reality systems have also been employed for laparoscopic training to a limited degree [5]. However, the inclusion of virtual reality instructional models is currently impractical and may be cost prohibitive for many training programs. At the University of Kentucky, where we have an active research program in virtual reality and surgical simulation, we have recognized the need for more immediately accessible and practical, cost-effective, inanimate models to target key skills needed in common laparoscopic procedures. Previous studies have demonstrated the validity and reliability of this training and evaluation program [6,7]. In this manuscript, we introduce our models and outline an effec-
0002-9610/04/$ – see front matter © 2004 Excerpta Medica, Inc. All rights reserved. doi:10.1016/j.amjsurg.2003.11.020
158
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
Fig. 1. A. Laparoscopic cholecystectomy simulation. A cholangiocatheter is inserted into the cystic duct, represented by hollow, tan tubing. The ductotomy and clip sites are designated by previously placed markings. B. Laparoscopic appendectomy. The appendix and appendiceal artery are simulated by the small finger of a latex glove and an inserted red rubber band, respectively, in this laparoscopic appendectomy model. C. Laparoscopic inguinal herniorrhaphy simulation. Inexpensive fabric mesh is tacked to an image of the inguinal region. Protruding white tubing simulates the spermatic cord.
tive performance assessment program for training surgical residents.
Model assembly and simulation objectives The University of Kentucky laparoscopic models were developed to train surgical residents in basic and advanced laparoscopic skills. These inanimate models were constructed to resemble closely the corresponding operative procedures both in terms of anatomy and requisite skills. The objective of each simulation varies, yet all focus on the development of fundamental skills that are pertinent to a variety of laparoscopic procedures. Performance evaluation can be completed using an assessment method that has been shown to be valid and reliable (discussed in a later section). The University of Kentucky laparoscopic models are constructed from inexpensive materials and offer the benefits of easy assembly, adaptability, and reusability. A stan-
dard laparoscopic trainer with interchangeable aluminum holders are used to house the models for each procedure. Color prints of high-resolution photographs taken from actual laparoscopic procedures and attached to the platform serve as realistic anatomic backgrounds and provide clinical context. This last simple and inexpensive step has proven to be a remarkable innovation, imparting significant face validity to simulations that have been shown to possess good construct validity [7]. Three University of Kentucky models were described in a previous publication, including laparoscopic cholecystectomy and cholangiography, appendectomy, and inguinal hernia repair [6]. These inanimate models are depicted in Figs. 1 and 2, along with the appropriate model positioning. The position of the models, surgeon, and monitor closely resemble the actual laparoscopic procedural set-up. However, it should be emphasized that the inanimate models are part-task simulations, in that they represent only key portions of the corresponding laparoscopic procedure. The
Fig. 2. A. Laparoscopic cholecystectomy. The model is placed in the upper left trainer box. The trainee stands at the foot of the trainer box, facing the monitor. A three-trocar array is used. B. Laparoscopic appendectomy. A three- trocar array is used to provide triangulation of the working ports (1, 2) and camera port (3). The model (M), camera, and trainee are aligned. C. Laparoscopic inguinal herniorrhaphy. the model is placed in a slightly anterior orientation at the “caudad” end of the trainer. The trainee stands at the “head” of the trainer facing the monitor. Three trocars are used.
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
159
original three simulations as well as two additional models are described as follows. Laparoscopic cholecystectomy and intraoperative cholangiography This simulation focuses on a key element of a laparoscopic cholecystectomy, namely, the challenge of cholangiocatheter insertion. One-handed manipulation of the cholangiocatheter is required, while the surgeon’s left hand retracts the cystic duct. Appropriate lateral retraction on the cystic duct is emphasized as well as avoidance of cystic artery injury during the procedure. Fundamentally, this exercise develops the basic laparoscopic skills of spatial orientation and dexterity, skills that have been defined elsewhere [6,8]. After a clip is placed distally on the cystic duct, a cystic ductotomy is made. The reusable cholangiocatheter is percutaneously introduced into the right upper quadrant and is inserted into the proximal cystic duct. A clip is placed just proximal to the ductotomy to secure the catheter. Proper placement is assessed by gentle external retraction of the catheter to ensure it is securely placed. The proximal clip is then removed, followed by the catheter. The procedure is completed with the placement of two proximal clips on the duct and transection of the duct at the ductotomy site with laparoscopic shears. Laparoscopic appendectomy simulation The objective of this training exercise is to expose, ligate, and divide the appendiceal artery and appendix. This simulation allows the learner to practice gentle handling of tissues and one-handed deployment of previously tied looped ligatures. A degree of two-handed laparoscopic facility is also required to properly expose the appendiceal artery and the appendiceal base. This appendectomy simulation incorporates instrument manipulation, dexterity, spatial orientation, clinical judgment or respect for tissue, operative exposure, and operative planning. The trainee begins by finding the appendix and retracting it anteriorly to identify the appendiceal artery. After ligation with clips placed at the previously marked proximal and distal sites, the artery is divided with laparoscopic scissors. The appendix is divided sharply between looped ligatures at the appendiceal base. The appendiceal stump should be appropriately short and the proximal ligature should be securely tied. Laparoscopic inguinal hernia repair simulation The objective of the procedure is to secure a piece of mesh to the appropriate anatomic landmarks. This simulation aims to develop skills in two-handed laparoscopic navigation, operative exposure, instrumentation and prosthetic mesh manipulation, and recognition of anatomic landmarks.
Fig. 3. A. In this simulation, the trainee must mobilize the bowel, represented by cotton cording, and close an enterotomy. B. Completed model. C. The model is placed along the left side of the trainer, and the box is position parallel to the monitor. The trainee stands in front of the right side of the trainer, facing the monitor. Three trocars are used.
The fundamental skills of dexterity and spatial orientation are targeted [6,8]. To add a degree of complexity to the procedure, the mesh fabric is first cut halfway along a line demarcating the middle and lateral thirds of the material to form a keyhole for the “spermatic cord.” Before inserting the mesh, a loose U-stitch is placed in the mesh securing the two tails around the keyhole to the other side. The mesh is placed anterior to the “spermatic cord” with the tails of the mesh pointed superiorly. The suture is grasped with a fine dissector, cut, and removed. The mesh is positioned and tacked in three places: inferomedially on the “pubis,” superomedially at the “medial rectus muscle,” and superolaterally. The lateral tack should secure both tails of the mesh just lateral to the epigastric vessels. Care is taken to avoid injury to the “spermatic cord” during the procedure. As a cost-saving measure, only three tacks are used in this exercise.
160
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
Fig. 4. A. The laparoscopic splenectomy simulation focuses on fine dissection and vascular control. The hilar vessels are located deep to the gastrosplenic ligament. B. The model is placed in the right upper trainer box. An L-shaped, three-trocar array is used.
Laparoscopic bowel mobilization and enterotomy closure For assembly and preparation; thick cotton cording attached to white fabric backing is used to simulate the colon and its peritoneal attachment. This backing is also glued to a photo reproduction of the lateral abdominal wall. A transverse incision in a latex condom overlying the cording represents an enterotomy (Fig. 3). The ends of the cording are secured to two different aluminum platforms allowing the bowel model to lie flat between the two holders. The model and procedural set-up are shown in Fig. 3. This simulation targets the skills of fine dissection, gentle handling of delicate tissue, and intracorporeal suturing and knot tying. The procedure involves initial mobilization of the “bowel” and closure of an “enterotomy.” The “bowel” is mobilized by sharply dividing the “lateral peritoneal attachment” with laparoscopic shears. The “enterotomy” is then closed with interrupted silk sutures placed intracorporeally. Laparoscopic splenectomy For assembly and preparation; this solid organ model is based on a rice-filled balloon representing the spleen and splenic hilum. The “spleen” is then covered with a hydropolymer that dries to a fascia-like material to represent the splenic attachments (Likelife: Simulated Tissues and Bio-materials; I, J, & J George, Lexington, Kentucky). The splenectomy model is placed in the upper right portion of the trainer box to simulate the splenic position in the left upper quadrant. A three-trocar array is used (Fig. 4). The laparoscopic splenectomy simulation focuses on teaching fine laparoscopic dissection skills as well as the steps involved in establishing and maintaining control of highly vascular structures. It targets the fundamental skills of clinical judgment, dexterity, and the use of laparoscopic stapling devices. The “spleen” is mobilized except for the
“lateral attachments.” The gastrosplenic ligament is divided sharply, and clips are placed on the short gastric vessels. The splenic hilum is then carefully dissected and divided with an endoscopic stapling device. The splenic mobilization is then completed.
Cost The University of Kentucky laparoscopic models could benefit all surgical programs wishing to establish a minimally invasive surgery curriculum. In a cost comparison, these inexpensively constructed models compare favorably with commercially produced simulators. Our models incorporate many reusable and renewable components at a significant cost savings. Fishing line replaces the commercially available pre-tied looped ligatures. Crinoline netting, readily available at any fabric store, serves as a realistic substitute for polypropylene mesh in terms of weight, feel, and handling characteristics. These and many of the items used in the models can be purchased at a nominal cost. With the inclusion of donated and reusable laparoscopic instruments, the University of Kentucky laparoscopic models are remarkably cost effective. Given the differences in model design, a direct cost comparison with commercially available laparoscopic models is difficult. However, in a cost comparison with the list prices of two commercial companies, the Kentucky models represent an estimated savings of approximately $96 to $1,364 per model (Table 1). This cost savings is magnified as the number of simulated procedures increases. For 50 residents to each perform two cholecystectomy simulations, the Kentucky model costs only $40 compared with $6,289 for company A and $4,280 for company B. When combined with a reliable assessment method, these inexpensive simulations can be easily incorporated
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
161
Table 1 Cost comparison of the University of Kentucky laparoscopic models for the basic minimally invasive surgery curriculum with two commercially available simulations Model Laparoscopic cholecystectomy Nonreusable components Reusable components Total Laparoscopic appendectomy Nonreusable components Reusable components Total Laparoscopic inguinal hernia repair Nonreusable components Reusable components Total
University of Kentucky
Company A
Company B
$ 0.13 $26.72 $26.85
$ 62.28 $ 60.55 $122.83
$ 38.00 $ 480.00 $ 518.00
$ 0.25 $26.90 $27.15
Not available
Not available
$ 0.88 $29.64 $30.52
$501.70 $147.05 $648.75 (discounted cost $605.50)
— $1395.00 $1395.00
into general surgery residency training. Several evaluation tools have been examined at our institution.
Evaluation Model evaluation The completed models are evaluated to assess trainee performance. A previous study by Szalay [9] demonstrated the utility of end-product evaluation as an outcome measure in a six-station bench examination of surgical residents and found an interrater reliability of 0.59. In our evaluation, the accuracy of clip and tack placement and the durability of enterotomy closure are just a few examples of these objective product analysis measures. The product analysis for the original three models (laparoscopic cholecystectomy, laparoscopic appendectomy, and inguinal hernia) has been compared with evaluations of trainee performance by both nonclinicians and laparoscopic surgeons [10]. These measurements of the completed models failed to correlate significantly with the other performance evaluations. The majority of the product measurements revealed accuracy across all subjects, suggesting that this was not a distinguishing measure of performance in our simulations. In other words, the trainees generally placed the clips, tacks, and ties in the correct position but did so with varying levels of efficiency and skill, as evidenced by the variability in the ratings by surgeon evaluators. Additionally, construct validity of experience was demonstrated for these three models [7]. Videotaped performance evaluation by nonclinicians In an effort to further assess the proficiency of trainees during the simulations, the procedures have been videotaped and later evaluated. Nonclinicians may readily participate in this evaluation process by applying a checklist assessment scale. The goal of this process is the objective
assessment of trainee performance using quantifiable measures, such as the number of attempts at cholangiocatheter insertion during the laparoscopic cholecystectomy simulation (Table 2). Errors are tabulated according to task-specific checklists. Other investigators have demonstrated the utility of the checklist assessment method. Both global and task-specific checklists have been developed, with global assessment proving to be a more reliable evaluation tool [11,12]. Our study comparing task-specific checklist assessment by nonclinicians and global assessment by laparoscopic faculty members demonstrated a significant correlation between the two [10]. This has important implications for surgical education in that nonclinicians could participate meaningfully in the evaluation of a trainee’s technical skills, partially freeing surgical faculty from this time intensive process. A technical assessment by nonclinicians could then be comTable 2 Task-specific checklist of technical errors for nonclinician evaluators for the three original University of Kentucky laparoscopic models Simulation
Technical error
Appendectomy
Instrument out of control/out of view # Times appendix grasped before first looped ligature # Times appendix grasped before second looped ligature Instrument out of control/out of view Attempts to grasp suture Attempts to cut suture # Times mesh dropped Instrument out of control/out of view # Times cystic artery touched # Attempts at distal clip placement # Attempts to grasp cholangiocatheter # Attempts to insert cholangiocatheter # Attempts at catheter clip placement # Attempts at first proximal clip placement # Attempts at second proximal clip placement
Inguinal herniorrhaphy
Cholecystectomy
162
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
bined with other methods (eg, faculty ward reviews) for a comprehensive evaluation.
subjective faculty evaluations, where personality differences between teacher and trainee may play a role.
Videotaped performance evaluation by laparoscopic surgeons
Comments
Evaluations of surgical residents by attending faculty are notoriously subjective. Although this global assessment tool is widely used, its reliability has been disputed [13,14]. Additionally, the Joint Initiative of the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties asserts that to maintain the reliability and reproducibility of this assessment method evaluators should be trained [15]. Yet faculty evaluators in surgical residency programs are seldom formally trained in such methods. However, faculty evaluators can be trained in the assessment of resident performance from videotapes. This method of assessment provides blinded, objective, and independent evaluations, unlike conventional faculty ward evaluations, which rely largely on recall. Faculty evaluators are taught to rate the simulation performance on a five-point scale (0 to 4) in four global skills categories: clinical judgment or respect for tissue, spatial orientation, serial/simultaneous complexity or flow of the procedure, and dexterity. These were selected as fundamental elements of laparoscopic surgery and integral components of most laparoscopic procedures [8]. These laparoscopic elemental skills are a function of the basic psychomotor skills of visualization, spatial scanning and spatial orientation [16]. Previous investigators have utilized similar scales [11]. The faculty evaluators also are required to assess the overall proficiency of the trainee in each of the University of Kentucky laparoscopic models. This overall competence rating has been shown to correlate highly with the ratings in the four component skills categories, suggesting the importance of each skill in the determination of technical competence [17]. An investigation of this blinded, independent evaluation method has demonstrated the interrater reliability of this assessment tool [6]. Agreement in ratings among five surgeon evaluators was shown by obtaining intraclass correlation coefficients that matched or exceeded the benchmark level of reliability (0.80) [6,18]. Importantly, a comparison of faculty ratings for trainees with varying surgical experience demonstrated the construct validity of our simulations [7]. In this investigation of three models (laparoscopic cholecystectomy, laparoscopic appendectomy, inguinal hernia), Pearson correlational analyses revealed a significant positive correlation between the faculty skills ratings and the trainee’s years of experience. The importance of this objective evaluation method is underscored by the fact that subjective faculty evaluations continue to be the mainstay in the assessment of technical operative skills. The masked assessment of the videotaped performance eliminates the bias associated with traditional,
The need for early instruction in video-assisted surgery is underscored by the widespread adoption of minimally invasive techniques in current surgical practice. In a recent survey of the European Surgical Association representing thirteen countries, more than half of the responders agreed that minimal access surgery training should start early in residency. Yet, only 14% of those responding have incorporated objective methods to evaluate manual dexterity of the trainees [19]. Although the response rate to the survey was limited (62%), the results do emphasize the importance of laparoscopic training and evaluation. Only a valid and reliable method of assessment can determine technical competence. Furthermore, objective assessment facilitates focused instruction and practice to achieve technical mastery. The incorporation of the Kentucky models into a laparoscopic bench training program has provided a safe setting for residents to learn the nonintuitive skills of minimally invasive surgery at our institution. The method is cost effective without sacrificing face validity, and is feasible for a busy general surgeon. Evidence of the construct validity of our faculty evaluation method supports the premise that surgeons can objectively and meaningfully assess technical competence. This assessment by faculty surgeons is further supported by evaluation conducted by nonclinicians. Additionally, this method of assessment has been shown to be reliable. Admittedly, technical mastery constitutes only a part of a surgeon’s overall competence. However, by combining this method of assessment with existing evaluation tools, perhaps the larger issue of determining surgeon competence can be addressed. Clearly, apprenticeship education in the operating room alone is inadequate for the acquisition of the nonintuitive skills required in minimally invasive surgery. The laparoscopic simulations developed at the University of Kentucky can form the foundation of an innovative educational program during residency for early training and assessment.
Acknowledgment Supported in part by an educational grant from Tyco/ U.S. Surgical Corporation.
References [1] Escarce JJ, Bloom BS, Hillman AL, et al. Diffusion of laparoscopic cholecystectomy among general surgeons in the United States. Med Care 1995;33:256 –71.
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163 [2] Hunter JG. Clinical trials and the development of laparoscopic surgery. Surg Endosc 2001;15:1–3. [3] Derossis AM, Fried GM, Abrahamowics M, et al. Development of a model for training and evaluation of laparoscopic skills. Am J Surg 1998;175:482–7. [4] Fried GM, Derossis AM, Bothwell J, Sigman HH. Comparison of laparoscopic performance in vivo with performance measured in a laparoscopic simulator. Surg Endosc 1999;13:1077– 81. [5] Torkington J, Smith SGT, Rees BI, Darzi A. The role of basic surgical skills course in the acquisition and retention of laparoscopic skill. Surg Endosc 2001;15:1071–5. [6] Adrales GL, Chu UB, Witzke DB, et al. Evaluating minimally invasive surgery (MIS) training using low cost mechanical simulations: reliability among surgeon ratings. Surg Endosc 2003;17:580 –5. [7] Adrales GL, Chu UB, Witzke DB, et al. Assessment of competence using laparoscopic simulations: a valid method of training and evaluation. Assoc Acad Surg Proc. 2002. [8] Donnelly M, Witzke D, Mastrangelo M, Park A. The educational implications of the importance of clinical judgment in performing laparoscopic skills. Surg Endosc 2001;15(suppl 1):S119. [9] Szalay D, MacRae H, Regehr G, Reznick R. Using operative outcome to assess technical skill. Am J Surg 2000;180:234 –7. [10] Adrales GL, Chu UB, Witzke DB, et al. Determination of competence as a function of expert surgeon judgment and technical performance [abstract,]. Assoc Surg Educ Proc 2002. [11] Reznick R, Regehr G, MacRae H, et al. Testing technical skill via an
[12]
[13]
[14]
[15]
[16]
[17]
[18] [19]
163
innovative “bench station” examination. Am J Surg 1996;173:226 – 30. Scott DJ, Bergen PC, Rege RV, et al. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg 2000;191:272– 83. Schwartz RW, Donnelly MB, Drake D, Sloan D. Faculty sensitivity in detecting medical students’ clinical competence. Clin Invest Med 1993;16(suppl):B87. Schwartz RW, Donnelly MB, Sloan DA, et al. The relationship between faculty ward evaluations, OSCE, and ABSITE as measures of surgical intern performance. Am J Surg 1995;169:414 –17. Accreditation Council for Graduate Medical Education and American Board of Medical Specialties. Toolbox of assessment methods. ACGME outcomes project, ACGME and ABMS joint initiative, 2000. Available at: http://www.acgme.org/outcome/assess/toolbox. asp. Accessed October 2002. Witzke DB, Adrales GL, Donnelly MB, et al. The relationship of psychomotor ability to minimally invasive surgery (MIS) technical skills performance [abstract]. Assoc Surg Educ Proc 2002:288. Adrales GL, Donnelly MB, Chu UB, et al. Determinants of competency judgments by experienced laparoscopic surgeons. Surg Endosc 2003. In press. Hubbard JP. Measuring medical education. 2nd ed. Philadelphia: Lea and Febiger, 1978. Peracchia A. Presidential address: surgical education in the third millennium. Ann Surg 2001;234:709 –12.
Surgical education
Development of a valid, cost-effective laparoscopic training program G.L. Adrales, U.B. Chu, J.D. Hoskins, D.B. Witzke, A.E. Park, M.D.* Department of Surgery, Minimally Invasive Center, University of Kentucky College of Medicine, Lexington, KY, USA Manuscript received December 2, 2002; revised manuscript July 4, 2003
Abstract Background: Practical programs for training and evaluating surgeons in laparoscopy are needed to keep pace with demand for minimally invasive surgery. Methods: At the University of Kentucky five inexpensive simulations have been developed to train and assess surgical residents. Residents are videotaped performing laparoscopic procedures on models. Five surgeons assess the taped performances on 4 global skills. Results: Creating mechanical models reduces training costs. Trainees agreed procedures were well represented by the simulations. Blinded assessment of performances showed high interrater agreement and correlated with the trainees’ level of experience. Nonclinician evaluations on checklists correlated with evaluations by surgeons. Conclusions: Inexpensive simulations of laparoscopic appendectomy, cholecystectomy, inguinal herniorrhaphy, bowel enterotomy, and splenectomy enable surgical residents to practice laparoscopic skills safely. Obtaining masked, objective, and independent evaluations of basic skills in laparoscopic surgery can assist in reliable assessment of surgical trainees. The simulations described can anchor an innovative educational program during residency for training and assessment. © 2004 Excerpta Medica, Inc. All rights reserved. Keywords: Assess; Simulation; Minimally invasive surgery; Medical education; Surgical training; Laparoscopy
The revolutionary development of minimally invasive surgery has fused new technology and the practice of medicine to achieve remarkable benefits in terms of patient recovery and healing. This transformation in healthcare has not been without cost. Incorporation of these advances into surgical practice occurred before important questions about training and credentialing were fully addressed. Widespread incorporation of laparoscopic cholecystectomy into the practices of surgeons occurred without the support of a randomized, clinical trial. By 1992, barely 4 years after the first laparoscopic cholecystectomy was performed in the United States, more than 80% of US surgeons had adopted this technique [1]. One of the consequences of this phenomenon was an early rise in biliary injuries after the adoption of the laparoscopic approach to cholecystectomy. While the absolute necessity of a clinical trial may be questionable, given the apparent benefits of this laparoscopic procedure to patients
* Corresponding author. Tel.: ⫹1-859-323-6346; fax: ⫹1-859-3236840. E-mail address: [email protected]
[2], the need for training and credentialing of surgeons in laparoscopy cannot be ignored. Before the incorporation of laparoscopic skills training into surgical education, the effectiveness of such training must be established. Several investigators have addressed this problem. Derossis and Fried [3,4] demonstrated the concurrent validity of a mechanical laparoscopic simulator. Subjects who had practiced on this inanimate simulator demonstrated superior performance in corresponding laparoscopic procedures in an animal model. Virtual reality systems have also been employed for laparoscopic training to a limited degree [5]. However, the inclusion of virtual reality instructional models is currently impractical and may be cost prohibitive for many training programs. At the University of Kentucky, where we have an active research program in virtual reality and surgical simulation, we have recognized the need for more immediately accessible and practical, cost-effective, inanimate models to target key skills needed in common laparoscopic procedures. Previous studies have demonstrated the validity and reliability of this training and evaluation program [6,7]. In this manuscript, we introduce our models and outline an effec-
0002-9610/04/$ – see front matter © 2004 Excerpta Medica, Inc. All rights reserved. doi:10.1016/j.amjsurg.2003.11.020
158
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
Fig. 1. A. Laparoscopic cholecystectomy simulation. A cholangiocatheter is inserted into the cystic duct, represented by hollow, tan tubing. The ductotomy and clip sites are designated by previously placed markings. B. Laparoscopic appendectomy. The appendix and appendiceal artery are simulated by the small finger of a latex glove and an inserted red rubber band, respectively, in this laparoscopic appendectomy model. C. Laparoscopic inguinal herniorrhaphy simulation. Inexpensive fabric mesh is tacked to an image of the inguinal region. Protruding white tubing simulates the spermatic cord.
tive performance assessment program for training surgical residents.
Model assembly and simulation objectives The University of Kentucky laparoscopic models were developed to train surgical residents in basic and advanced laparoscopic skills. These inanimate models were constructed to resemble closely the corresponding operative procedures both in terms of anatomy and requisite skills. The objective of each simulation varies, yet all focus on the development of fundamental skills that are pertinent to a variety of laparoscopic procedures. Performance evaluation can be completed using an assessment method that has been shown to be valid and reliable (discussed in a later section). The University of Kentucky laparoscopic models are constructed from inexpensive materials and offer the benefits of easy assembly, adaptability, and reusability. A stan-
dard laparoscopic trainer with interchangeable aluminum holders are used to house the models for each procedure. Color prints of high-resolution photographs taken from actual laparoscopic procedures and attached to the platform serve as realistic anatomic backgrounds and provide clinical context. This last simple and inexpensive step has proven to be a remarkable innovation, imparting significant face validity to simulations that have been shown to possess good construct validity [7]. Three University of Kentucky models were described in a previous publication, including laparoscopic cholecystectomy and cholangiography, appendectomy, and inguinal hernia repair [6]. These inanimate models are depicted in Figs. 1 and 2, along with the appropriate model positioning. The position of the models, surgeon, and monitor closely resemble the actual laparoscopic procedural set-up. However, it should be emphasized that the inanimate models are part-task simulations, in that they represent only key portions of the corresponding laparoscopic procedure. The
Fig. 2. A. Laparoscopic cholecystectomy. The model is placed in the upper left trainer box. The trainee stands at the foot of the trainer box, facing the monitor. A three-trocar array is used. B. Laparoscopic appendectomy. A three- trocar array is used to provide triangulation of the working ports (1, 2) and camera port (3). The model (M), camera, and trainee are aligned. C. Laparoscopic inguinal herniorrhaphy. the model is placed in a slightly anterior orientation at the “caudad” end of the trainer. The trainee stands at the “head” of the trainer facing the monitor. Three trocars are used.
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
159
original three simulations as well as two additional models are described as follows. Laparoscopic cholecystectomy and intraoperative cholangiography This simulation focuses on a key element of a laparoscopic cholecystectomy, namely, the challenge of cholangiocatheter insertion. One-handed manipulation of the cholangiocatheter is required, while the surgeon’s left hand retracts the cystic duct. Appropriate lateral retraction on the cystic duct is emphasized as well as avoidance of cystic artery injury during the procedure. Fundamentally, this exercise develops the basic laparoscopic skills of spatial orientation and dexterity, skills that have been defined elsewhere [6,8]. After a clip is placed distally on the cystic duct, a cystic ductotomy is made. The reusable cholangiocatheter is percutaneously introduced into the right upper quadrant and is inserted into the proximal cystic duct. A clip is placed just proximal to the ductotomy to secure the catheter. Proper placement is assessed by gentle external retraction of the catheter to ensure it is securely placed. The proximal clip is then removed, followed by the catheter. The procedure is completed with the placement of two proximal clips on the duct and transection of the duct at the ductotomy site with laparoscopic shears. Laparoscopic appendectomy simulation The objective of this training exercise is to expose, ligate, and divide the appendiceal artery and appendix. This simulation allows the learner to practice gentle handling of tissues and one-handed deployment of previously tied looped ligatures. A degree of two-handed laparoscopic facility is also required to properly expose the appendiceal artery and the appendiceal base. This appendectomy simulation incorporates instrument manipulation, dexterity, spatial orientation, clinical judgment or respect for tissue, operative exposure, and operative planning. The trainee begins by finding the appendix and retracting it anteriorly to identify the appendiceal artery. After ligation with clips placed at the previously marked proximal and distal sites, the artery is divided with laparoscopic scissors. The appendix is divided sharply between looped ligatures at the appendiceal base. The appendiceal stump should be appropriately short and the proximal ligature should be securely tied. Laparoscopic inguinal hernia repair simulation The objective of the procedure is to secure a piece of mesh to the appropriate anatomic landmarks. This simulation aims to develop skills in two-handed laparoscopic navigation, operative exposure, instrumentation and prosthetic mesh manipulation, and recognition of anatomic landmarks.
Fig. 3. A. In this simulation, the trainee must mobilize the bowel, represented by cotton cording, and close an enterotomy. B. Completed model. C. The model is placed along the left side of the trainer, and the box is position parallel to the monitor. The trainee stands in front of the right side of the trainer, facing the monitor. Three trocars are used.
The fundamental skills of dexterity and spatial orientation are targeted [6,8]. To add a degree of complexity to the procedure, the mesh fabric is first cut halfway along a line demarcating the middle and lateral thirds of the material to form a keyhole for the “spermatic cord.” Before inserting the mesh, a loose U-stitch is placed in the mesh securing the two tails around the keyhole to the other side. The mesh is placed anterior to the “spermatic cord” with the tails of the mesh pointed superiorly. The suture is grasped with a fine dissector, cut, and removed. The mesh is positioned and tacked in three places: inferomedially on the “pubis,” superomedially at the “medial rectus muscle,” and superolaterally. The lateral tack should secure both tails of the mesh just lateral to the epigastric vessels. Care is taken to avoid injury to the “spermatic cord” during the procedure. As a cost-saving measure, only three tacks are used in this exercise.
160
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
Fig. 4. A. The laparoscopic splenectomy simulation focuses on fine dissection and vascular control. The hilar vessels are located deep to the gastrosplenic ligament. B. The model is placed in the right upper trainer box. An L-shaped, three-trocar array is used.
Laparoscopic bowel mobilization and enterotomy closure For assembly and preparation; thick cotton cording attached to white fabric backing is used to simulate the colon and its peritoneal attachment. This backing is also glued to a photo reproduction of the lateral abdominal wall. A transverse incision in a latex condom overlying the cording represents an enterotomy (Fig. 3). The ends of the cording are secured to two different aluminum platforms allowing the bowel model to lie flat between the two holders. The model and procedural set-up are shown in Fig. 3. This simulation targets the skills of fine dissection, gentle handling of delicate tissue, and intracorporeal suturing and knot tying. The procedure involves initial mobilization of the “bowel” and closure of an “enterotomy.” The “bowel” is mobilized by sharply dividing the “lateral peritoneal attachment” with laparoscopic shears. The “enterotomy” is then closed with interrupted silk sutures placed intracorporeally. Laparoscopic splenectomy For assembly and preparation; this solid organ model is based on a rice-filled balloon representing the spleen and splenic hilum. The “spleen” is then covered with a hydropolymer that dries to a fascia-like material to represent the splenic attachments (Likelife: Simulated Tissues and Bio-materials; I, J, & J George, Lexington, Kentucky). The splenectomy model is placed in the upper right portion of the trainer box to simulate the splenic position in the left upper quadrant. A three-trocar array is used (Fig. 4). The laparoscopic splenectomy simulation focuses on teaching fine laparoscopic dissection skills as well as the steps involved in establishing and maintaining control of highly vascular structures. It targets the fundamental skills of clinical judgment, dexterity, and the use of laparoscopic stapling devices. The “spleen” is mobilized except for the
“lateral attachments.” The gastrosplenic ligament is divided sharply, and clips are placed on the short gastric vessels. The splenic hilum is then carefully dissected and divided with an endoscopic stapling device. The splenic mobilization is then completed.
Cost The University of Kentucky laparoscopic models could benefit all surgical programs wishing to establish a minimally invasive surgery curriculum. In a cost comparison, these inexpensively constructed models compare favorably with commercially produced simulators. Our models incorporate many reusable and renewable components at a significant cost savings. Fishing line replaces the commercially available pre-tied looped ligatures. Crinoline netting, readily available at any fabric store, serves as a realistic substitute for polypropylene mesh in terms of weight, feel, and handling characteristics. These and many of the items used in the models can be purchased at a nominal cost. With the inclusion of donated and reusable laparoscopic instruments, the University of Kentucky laparoscopic models are remarkably cost effective. Given the differences in model design, a direct cost comparison with commercially available laparoscopic models is difficult. However, in a cost comparison with the list prices of two commercial companies, the Kentucky models represent an estimated savings of approximately $96 to $1,364 per model (Table 1). This cost savings is magnified as the number of simulated procedures increases. For 50 residents to each perform two cholecystectomy simulations, the Kentucky model costs only $40 compared with $6,289 for company A and $4,280 for company B. When combined with a reliable assessment method, these inexpensive simulations can be easily incorporated
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
161
Table 1 Cost comparison of the University of Kentucky laparoscopic models for the basic minimally invasive surgery curriculum with two commercially available simulations Model Laparoscopic cholecystectomy Nonreusable components Reusable components Total Laparoscopic appendectomy Nonreusable components Reusable components Total Laparoscopic inguinal hernia repair Nonreusable components Reusable components Total
University of Kentucky
Company A
Company B
$ 0.13 $26.72 $26.85
$ 62.28 $ 60.55 $122.83
$ 38.00 $ 480.00 $ 518.00
$ 0.25 $26.90 $27.15
Not available
Not available
$ 0.88 $29.64 $30.52
$501.70 $147.05 $648.75 (discounted cost $605.50)
— $1395.00 $1395.00
into general surgery residency training. Several evaluation tools have been examined at our institution.
Evaluation Model evaluation The completed models are evaluated to assess trainee performance. A previous study by Szalay [9] demonstrated the utility of end-product evaluation as an outcome measure in a six-station bench examination of surgical residents and found an interrater reliability of 0.59. In our evaluation, the accuracy of clip and tack placement and the durability of enterotomy closure are just a few examples of these objective product analysis measures. The product analysis for the original three models (laparoscopic cholecystectomy, laparoscopic appendectomy, and inguinal hernia) has been compared with evaluations of trainee performance by both nonclinicians and laparoscopic surgeons [10]. These measurements of the completed models failed to correlate significantly with the other performance evaluations. The majority of the product measurements revealed accuracy across all subjects, suggesting that this was not a distinguishing measure of performance in our simulations. In other words, the trainees generally placed the clips, tacks, and ties in the correct position but did so with varying levels of efficiency and skill, as evidenced by the variability in the ratings by surgeon evaluators. Additionally, construct validity of experience was demonstrated for these three models [7]. Videotaped performance evaluation by nonclinicians In an effort to further assess the proficiency of trainees during the simulations, the procedures have been videotaped and later evaluated. Nonclinicians may readily participate in this evaluation process by applying a checklist assessment scale. The goal of this process is the objective
assessment of trainee performance using quantifiable measures, such as the number of attempts at cholangiocatheter insertion during the laparoscopic cholecystectomy simulation (Table 2). Errors are tabulated according to task-specific checklists. Other investigators have demonstrated the utility of the checklist assessment method. Both global and task-specific checklists have been developed, with global assessment proving to be a more reliable evaluation tool [11,12]. Our study comparing task-specific checklist assessment by nonclinicians and global assessment by laparoscopic faculty members demonstrated a significant correlation between the two [10]. This has important implications for surgical education in that nonclinicians could participate meaningfully in the evaluation of a trainee’s technical skills, partially freeing surgical faculty from this time intensive process. A technical assessment by nonclinicians could then be comTable 2 Task-specific checklist of technical errors for nonclinician evaluators for the three original University of Kentucky laparoscopic models Simulation
Technical error
Appendectomy
Instrument out of control/out of view # Times appendix grasped before first looped ligature # Times appendix grasped before second looped ligature Instrument out of control/out of view Attempts to grasp suture Attempts to cut suture # Times mesh dropped Instrument out of control/out of view # Times cystic artery touched # Attempts at distal clip placement # Attempts to grasp cholangiocatheter # Attempts to insert cholangiocatheter # Attempts at catheter clip placement # Attempts at first proximal clip placement # Attempts at second proximal clip placement
Inguinal herniorrhaphy
Cholecystectomy
162
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163
bined with other methods (eg, faculty ward reviews) for a comprehensive evaluation.
subjective faculty evaluations, where personality differences between teacher and trainee may play a role.
Videotaped performance evaluation by laparoscopic surgeons
Comments
Evaluations of surgical residents by attending faculty are notoriously subjective. Although this global assessment tool is widely used, its reliability has been disputed [13,14]. Additionally, the Joint Initiative of the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties asserts that to maintain the reliability and reproducibility of this assessment method evaluators should be trained [15]. Yet faculty evaluators in surgical residency programs are seldom formally trained in such methods. However, faculty evaluators can be trained in the assessment of resident performance from videotapes. This method of assessment provides blinded, objective, and independent evaluations, unlike conventional faculty ward evaluations, which rely largely on recall. Faculty evaluators are taught to rate the simulation performance on a five-point scale (0 to 4) in four global skills categories: clinical judgment or respect for tissue, spatial orientation, serial/simultaneous complexity or flow of the procedure, and dexterity. These were selected as fundamental elements of laparoscopic surgery and integral components of most laparoscopic procedures [8]. These laparoscopic elemental skills are a function of the basic psychomotor skills of visualization, spatial scanning and spatial orientation [16]. Previous investigators have utilized similar scales [11]. The faculty evaluators also are required to assess the overall proficiency of the trainee in each of the University of Kentucky laparoscopic models. This overall competence rating has been shown to correlate highly with the ratings in the four component skills categories, suggesting the importance of each skill in the determination of technical competence [17]. An investigation of this blinded, independent evaluation method has demonstrated the interrater reliability of this assessment tool [6]. Agreement in ratings among five surgeon evaluators was shown by obtaining intraclass correlation coefficients that matched or exceeded the benchmark level of reliability (0.80) [6,18]. Importantly, a comparison of faculty ratings for trainees with varying surgical experience demonstrated the construct validity of our simulations [7]. In this investigation of three models (laparoscopic cholecystectomy, laparoscopic appendectomy, inguinal hernia), Pearson correlational analyses revealed a significant positive correlation between the faculty skills ratings and the trainee’s years of experience. The importance of this objective evaluation method is underscored by the fact that subjective faculty evaluations continue to be the mainstay in the assessment of technical operative skills. The masked assessment of the videotaped performance eliminates the bias associated with traditional,
The need for early instruction in video-assisted surgery is underscored by the widespread adoption of minimally invasive techniques in current surgical practice. In a recent survey of the European Surgical Association representing thirteen countries, more than half of the responders agreed that minimal access surgery training should start early in residency. Yet, only 14% of those responding have incorporated objective methods to evaluate manual dexterity of the trainees [19]. Although the response rate to the survey was limited (62%), the results do emphasize the importance of laparoscopic training and evaluation. Only a valid and reliable method of assessment can determine technical competence. Furthermore, objective assessment facilitates focused instruction and practice to achieve technical mastery. The incorporation of the Kentucky models into a laparoscopic bench training program has provided a safe setting for residents to learn the nonintuitive skills of minimally invasive surgery at our institution. The method is cost effective without sacrificing face validity, and is feasible for a busy general surgeon. Evidence of the construct validity of our faculty evaluation method supports the premise that surgeons can objectively and meaningfully assess technical competence. This assessment by faculty surgeons is further supported by evaluation conducted by nonclinicians. Additionally, this method of assessment has been shown to be reliable. Admittedly, technical mastery constitutes only a part of a surgeon’s overall competence. However, by combining this method of assessment with existing evaluation tools, perhaps the larger issue of determining surgeon competence can be addressed. Clearly, apprenticeship education in the operating room alone is inadequate for the acquisition of the nonintuitive skills required in minimally invasive surgery. The laparoscopic simulations developed at the University of Kentucky can form the foundation of an innovative educational program during residency for early training and assessment.
Acknowledgment Supported in part by an educational grant from Tyco/ U.S. Surgical Corporation.
References [1] Escarce JJ, Bloom BS, Hillman AL, et al. Diffusion of laparoscopic cholecystectomy among general surgeons in the United States. Med Care 1995;33:256 –71.
G.L. Adrales et al. / The American Journal of Surgery 187 (2004) 157–163 [2] Hunter JG. Clinical trials and the development of laparoscopic surgery. Surg Endosc 2001;15:1–3. [3] Derossis AM, Fried GM, Abrahamowics M, et al. Development of a model for training and evaluation of laparoscopic skills. Am J Surg 1998;175:482–7. [4] Fried GM, Derossis AM, Bothwell J, Sigman HH. Comparison of laparoscopic performance in vivo with performance measured in a laparoscopic simulator. Surg Endosc 1999;13:1077– 81. [5] Torkington J, Smith SGT, Rees BI, Darzi A. The role of basic surgical skills course in the acquisition and retention of laparoscopic skill. Surg Endosc 2001;15:1071–5. [6] Adrales GL, Chu UB, Witzke DB, et al. Evaluating minimally invasive surgery (MIS) training using low cost mechanical simulations: reliability among surgeon ratings. Surg Endosc 2003;17:580 –5. [7] Adrales GL, Chu UB, Witzke DB, et al. Assessment of competence using laparoscopic simulations: a valid method of training and evaluation. Assoc Acad Surg Proc. 2002. [8] Donnelly M, Witzke D, Mastrangelo M, Park A. The educational implications of the importance of clinical judgment in performing laparoscopic skills. Surg Endosc 2001;15(suppl 1):S119. [9] Szalay D, MacRae H, Regehr G, Reznick R. Using operative outcome to assess technical skill. Am J Surg 2000;180:234 –7. [10] Adrales GL, Chu UB, Witzke DB, et al. Determination of competence as a function of expert surgeon judgment and technical performance [abstract,]. Assoc Surg Educ Proc 2002. [11] Reznick R, Regehr G, MacRae H, et al. Testing technical skill via an
[12]
[13]
[14]
[15]
[16]
[17]
[18] [19]
163
innovative “bench station” examination. Am J Surg 1996;173:226 – 30. Scott DJ, Bergen PC, Rege RV, et al. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg 2000;191:272– 83. Schwartz RW, Donnelly MB, Drake D, Sloan D. Faculty sensitivity in detecting medical students’ clinical competence. Clin Invest Med 1993;16(suppl):B87. Schwartz RW, Donnelly MB, Sloan DA, et al. The relationship between faculty ward evaluations, OSCE, and ABSITE as measures of surgical intern performance. Am J Surg 1995;169:414 –17. Accreditation Council for Graduate Medical Education and American Board of Medical Specialties. Toolbox of assessment methods. ACGME outcomes project, ACGME and ABMS joint initiative, 2000. Available at: http://www.acgme.org/outcome/assess/toolbox. asp. Accessed October 2002. Witzke DB, Adrales GL, Donnelly MB, et al. The relationship of psychomotor ability to minimally invasive surgery (MIS) technical skills performance [abstract]. Assoc Surg Educ Proc 2002:288. Adrales GL, Donnelly MB, Chu UB, et al. Determinants of competency judgments by experienced laparoscopic surgeons. Surg Endosc 2003. In press. Hubbard JP. Measuring medical education. 2nd ed. Philadelphia: Lea and Febiger, 1978. Peracchia A. Presidential address: surgical education in the third millennium. Ann Surg 2001;234:709 –12.