- Email: [email protected]
Efficacy of a Novel Cholangiogram Simulator for Training Laparoscopic Intraoperative Cholangiography
ARTICLE IN PRESS ORIGINAL REPORTS
Efficacy of a Novel Cholangiogram Simulator for Training Laparoscopic Intraoperative Cholangiography Tyler J. Sbrocchi, MD,* William D. Watson, MD, FACS,† Oscar Ruiz, MD, FACS,* and Ngan Nguyen, PhD† *
Department of Medical Education, OhioHealth Learning, Riverside Methodist Hospital, Columbus, Ohio; and OhioHealth Learning, Riverside Methodist Hospital, Columbus, Ohio
†
OBJECTIVE: Determine the efficacy of a novel low-cost, reusable simulator for training fundamental skills associated with safe and effective intraoperative cholangiography (IOC). DESIGN: The simulator uses a balloon and retention
suture tubing (representing the gallbladder, cystic duct, common bile duct, and common hepatic duct) attached to a piece of wood and placed in a laparoscopic trainer (representing the abdominal cavity) covered by a piece of simulated skin to obscure it. Following a tutorial on performing a simulated IOC using this, simulation participants independently completed a video-recorded simulated IOC and a post-training survey about the appearance and perceived usefulness of the IOC simulator as a training tool. Two experienced surgeons assessed participants’ IOC performance using an IOC procedural checklist developed for this purpose. Procedural time (in seconds) was recorded and used as an additional measure of performance. SETTING: The OhioHealth Learning Center simulation
facility in the Department of Surgery at Riverside Methodist Hospital, a large tertiary care independent medical center that is part of the OhioHealth care system. PARTICIPANTS: Eleven attending surgeons and 16 gen-
eral surgery residents of different levels participated in the simulation. Two experienced surgeons assessed participants’ IOC performance. One participated in the simulation along with the other 10 surgeons; the other did not.
Previously presented at the Innovative Techniques session of the 2018 Annual ACS Surgical Simulation Summit: An International Multi-Professional Meeting, March 16-17, 2018 in Chicago, IL. Correspondence: Inquiries to Tyler J. Sbrocchi, MD, Department of Surgery, OhioHealth Riverside Methodist Hospital, 3535 Olentangy River Road, Columbus, OH 43214-3998; fax: 614-566-1073; e-mail: [email protected]
RESULTS: High-experience participants completed more steps and spent less time than low-experience individuals; however, differences were not statistically significant. There was substantial agreement between the 2 observers regarding participants’ performance. Participants scored the simulator as realistic and useful in teaching relevant steps associated with IOC. CONCLUSIONS: Despite differences between high- and
low-experience participants in steps completed and time spent, these results did not prove statistically significant. Additional studies to increase sample size are warranted to determine if significant differences exist. However, participants did generally find the simulator to be an effective training tool. ( J Surg Ed 000:1 7. Ó 2019 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: simulation training technique, resident
surgical education, laparoscopic cholecystectomy, cholangiography COMPETENCIES: Patient Care, Medical Knowledge
INTRODUCTION Laparoscopic cholecystectomy (LC) is one of the most frequently performed abdominal surgery procedures in the United States, with approximately 750,000 Americans undergoing the procedure each year.1 Despite the frequency and safety of LC, common bile duct (CBD) injuries are a dreaded complication related to LC, occurring in approximately 1 in 200 LC cases.1,2 As many as 50% of surgeons who perform LC are expected to cause such an injury during their career,3 making litigation related to CBD injuries among the leading sources of medical malpractice claims against surgeons.4 Serious injuries to the
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.008
1
ARTICLE IN PRESS CBD often require at least 1 surgical repair, and these repairs have variable long-term outcomes.5,6 Despite its potential for early detection of a major CBD injury,7,8 the routine use of intraoperative cholangiography (IOC) continues to be a matter of surgeon preference. IOC can be safely performed by all surgeons as well as surgical residents at every level of training without adding significant time to the operation9 however, there is little reported on teaching IOC to surgical trainees. While simulation is available for training these fundamental skills without sacrificing patient safety,10 evidence for the effectiveness of simulation-based training in facilitating skills acquisition is limited. The purpose of this study was to determine the efficacy of a novel low-cost simulator for training the fundamental steps and skills associated with carrying out safe and effective IOC.
METHODS Participants and Setting General surgeons and surgical residents at Riverside Methodist Hospital were invited to participate. Participants were asked to complete a demographic survey consisting of questions on sex, age, and experience performing laparoscopic IOC. Twelve attending surgeons participated in the overall study; 11 of them participated in the simulation itself. The twelfth surgeon, along with 1 of the 11 simulation participants, assessed participants’ performance (see “Assessments,” below). All surgeons were males between the ages of 32 and 64 years (mean age = 45.70 § 11.06). All surgeons reported performing IOC as part of LC; 3 surgeons (27%) perform IOC on a routine basis and 8 (73%) on a selective basis. Sixteen general surgery residents participated in the study (males = 13; females = 3). Residents’ age ranged from 26 to 38 years (mean age = 28.8 § 2.7). Five (31%) of the residents were in PGY-1, 3 (19%) PGY-2, 2 (12%) PGY-3, 3 (19%) PGY-4, and 3 (19%) PGY-5. We defined high-experience residents as those who had performed 35 to 100 IOCs, 5 (31%). Those who had performed fewer than 35 IOCs were defined as low-experience, 11 (69%). Among the low-experience residents, 1 had no previous experience with IOC, while 10 had various levels of experience, ranging from 2 to 15 IOCs.
because it is close in size to the cystic duct and because of its resemblance, in terms of softness, handling, and movement, to the actual anatomy. This apparatus was placed in a laparoscopic trainer (Fig. 2) (representing the abdominal cavity) which was covered by a piece of simulated skin to prevent participants from looking into the abdominal cavity. In addition to the simulator, a novel IOC procedural checklist (see Table 1) was developed to assess IOC performance. Assessments Following a tutorial on how to perform a simulated IOC using the novel simulator, all simulation participants independently completed a simulated IOC (see e-Component video). Next, participants completed a post-training survey consisting of 9 questions about the appearance of the IOC simulator and its perceived usefulness as an IOC training tool. Completion of the simulated IOC was video-recorded and independently assessed by 2 experienced general surgeons using the IOC checklist. Additionally, the procedural time (in seconds) was recorded and used as an additional measure of IOC performance. Statistical Analysis Content validity of the IOC checklist was established by determining the content validity ratio (CVR) of each item on the checklist, a technique suggested by Lawshe.11 The CVR is based on 2 psychophysical assumptions: (1) any item in which performance is perceived to be “essential”
Simulator Description The simulator developed by members of the research team consists of a balloon (representing the gallbladder) and retention suture tubing (representing the cystic duct, CBD, and common hepatic duct) attached to a piece of wood (Fig. 1). We chose retention suture tubing
2
FIGURE 1. Balloon and tubing model of gallbladder and ducts.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
TABLE 1. The CVR of Each IOC Checklist Item
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
FIGURE 2. Setup for simulated IOC.
by more than half of the raters has some degree of content validity, and (2) the more raters (> 50%) who perceive the item as being “essential” the greater the degree of its content validity. The CVR is calculated using the following formula: CVR = (e N/2) N/2, in which e is the number of experts indicating “essential” and N is the total number of raters. To determine which assessment tool items to retain, a table of minimum CVR scores for item inclusion was developed based on a 1-tailed test with a significant value of p = 0.05. Items that scored below the minimum CVR threshold were removed from the corresponding assessment tool. Next, the mean CVR for the retained items was calculated to determine the content validity index for the overall content validity of the IOC checklist. Face validity of the IOC simulator was established by calculating the mean Likert scale scores and standard deviations for individual items on the post-training survey. High scores (i.e., scores close to 5 or strongly agree) suggest that the simulation is realistic in terms of appearance, and that it is useful as an education and training tool. A scoring rubric developed to facilitate interpretation of post-training survey results was based on the following scores: Disagree (0.0-2.0); Somewhat disagree (2.1-2.9); Neutral: neither agree nor disagree (3.0); Somewhat agree (3.1-3.9); and Agree (4.0-5.0).
Journal of Surgical Education Volume 00 /Number 00 Month 2019
16 17 18 19 20
Checklist Items
CVR
Grasp and retract gall bladder superiorly Adjust camera in order to view insertion of catheter With an 11 blade, make a 1-3 mm right subcostal incision Insert angiocatheter through incision Insert cholangiocatheter through angiocatheter Grasp and retract infundibulum of gallbladder to adjust angle of cystic duct Place a clip on the junction of the gallbladder and cystic duct Make a small transverse incision on the anterior surface of the cystic duct just below the clip Grasp cholangiocatheter and insert it through cystic ductotomy Clip the cholangiocatheter to fix it in place Inject 3-5 cc of saline to ensure that there is no leakage Remove graspers, ports, and camera Stand behind shield with syringe Verify that everyone is ready for x-ray Aspirate and inject diluted radiocontrast (half contrast, half saline) Obtain fluoroscopic images Reinsert camera and ports Grasp infundibulum of gallbladder Remove clip Grasp and retract cholangiocatheter (but not completely)
0.46 0.46 0.27 0.27 0.64* 0.64* 0.46
0.64* 0.64* 0.46 0.27 0.64* 0.64* 0.45 1.00* 0.64* 0.64* 0.64* -0.09
*p < 0.05.
Construct validity of the IOC simulation was established by using an independent t test to compare the performance (i.e., score on the IOC checklist) of high-experience vs. low-experience participants. Construct validity is established if high-experience participants perform significantly better than their low-experience counterparts. High positive correlations between the raters’ evaluations provide evidence for inter-rater reliability. Kappa statistics were used to assess inter-rater reliability and interobserver agreement. The interpretation of the kappa value was based on the commonly cited scale, with kappa < 0 = less than chance agreement; 0.01 to 0.20 = slight agreement; 0.21 to 0.40 = fair agreement; 0.41 to 0.60 = moderate agreement; 0.61 to 0.80 = substantial agreement; and 0.81 to 1.00 = almost perfect agreement.
RESULTS Content Validity of IOC Checklist The CVR of each checklist item is presented in Table 1. Since the content evaluation panel consists of 11
3
ARTICLE IN PRESS members, a minimum CVR of 0.59 was required to satisfy the 5% level.11 Only 10 items (indicated with an asterisk*) out of the 20 items had CVR values meeting the minimum CVR threshold. These items were retained and used in the evaluation of IOC performance (see Construct validity of the IOC simulation below).
TABLE 3. Residents’ (n = 16) Perceived Experience With the IOC Simulation Survey Questions
Resident Response (Scale: 1-5)* Mean § SD
Face Validity of the IOC Simulator
1
4.56 § 0.63
Table 2 shows residents and surgeons’ (n = 22) response to each item of the post-training survey. On average, participants agreed that the model of the gallbladder, cystic duct and CBD appeared realistic. They also agreed that the simulation was useful for teaching the relevant steps associated with IOC and for training application of the clip to the cystic duct and manipulation of both the angiocatheter and cholangiocatheter. The participants somewhat agreed that the model’s handling characteristics mimicked those of a real operative experience. They also somewhat agreed that the simulation was useful for training manipulation of the gallbladder, performance of cystic ductotomy, and injection of contrast agent.
2
TABLE 2. Participants’ (n = 22) Response to Each Item of the PostTraining Survey Survey Questions
Resident and Attending Mean Response (Scale: 1-5)* Mean § SD
1 2 3
4 5 6 7 8 9
The anatomical model (e.g., gallbladder, cystic duct, and common bile duct) appears realistic The model's handling characteristics (haptics) mimic those of a real operative experience The simulation is useful for teaching the relevant procedural steps associated with laparoscopic intraoperative cholangiogram The simulation is useful for training manipulation of the gallbladder The simulation is useful for training performance of cystic ductotomy The simulation is useful for training insertion and manipulation of angiocatheter The simulation is useful for training insertion and manipulation of cholangiocatheter The simulation is useful for training application of the clip to the cystic duct The simulation is useful for training injection of contrast agent
*Scale: 1 (low) to 5 (high). **Meets minimum CVR threshold.
4
4.00 § 0.82** 3.82 § 0.80 4.50 § 0.60**
3.82 § 0.96
3 4
I was engaged in the simulation experience I felt the simulation was challenging The simulation was a valuable use of my time I would like more simulation similar to this setup incorporated in my training
4.19 § 0.98 4.50 § 0.73 4.50 § 0.82
*Scale: 1 (low) to 5 (high).
Table 3 shows residents’ perception of their overall simulation experience. On average, residents felt engaged in the simulation experience. They reported that the simulation was challenging and a valuable use of their time, and recommended that more simulation similar to this setup be incorporated in their training. Construct Validity of the IOC Simulation Figure 3 shows the mean IOC scores of low- (n = 16) and high-experience (n = 11) participants. Low-experience participants included residents who had performed less than 35 IOCs, while high-experience participants included attending surgeons and residents who had performed 35 to 100 IOCs. On average, high-experience participants performed the same number of steps but spent less time (Fig. 4) on the IOC procedure than lowexperience individuals. However, both IOC checklist score, t(20) = 0.36, and time spent performing IOC, p (20) = 1.89, were not statistically different, p > 0.05. Mean kappa value was 0.63 (range = 0.30-1), p < 0.05, indicating that there was substantial agreement between the 2 observers.
3.82 § 1.10 4.23 § 0.92** 4.32 § 0.72** 4.41 § 0.59** 3.91 § 1.11
DISCUSSION Studies have indicated 12,13 that practice with models can further mastery of steps needed for operative procedures, potentially reducing the risk of complications. Considerations for useful and workable models include cost and availability, quality of tactile feedback, ease of use, and creation or enhancement of a skillset. This study was developed in an effort to find a simple, efficient, inexpensive, and reproducible way to teach the steps to perform IOC to lower-level surgery residents (PGY1-3) prior to performance in the operating room. It
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
FIGURE 3. Mean § SD IOC scores of low (n = 16) and high-experience (n = 11) participants.
also creates an accurate representation of the main sequential steps performed in the OR during intraoperative cholangiograms. Historically, animal models, porcine in particular, were sometimes used for training laparoscopic biliary procedures in general 14 and LC in particular 15; however, these are limited by availability, cost and time requirements for preparation plus a limited window for training, as well as clearly not being reusable. Windsor’s model,16 which used a saphenous magna vein and tributaries as a substitute for the biliary tract, is hampered by a similar lack of reusability as well as limited availability and, comparably to our model, more expensive. In contrast, costs for our model are decidedly minimal—5 to 10 dollars—save that of a laparoscopic trainer (which nearly every teaching hospital has) it is therefore readily available. In addition, only portions of the model need to be replaced between participants and/or uses, making it easy to train multiple participants. Our model provides a tactile, hands-on method, offering practice in the necessary motions that are used during IOC. The use of retention suture tubing mimics the cystic duct more closely in terms of softness, handling,
FIGURE 4. Time spent on simulation by participants with low vs. high experience—means § SD.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
and movement, as well as lumen size. As part of development both IV tubing, vesical (Foley catheter) and a red rubber catheter were considered harder to cut than the actual cystic duct. These options were also harder in terms of clip application, requiring more clicks to keep the catheter in place than might be needed or used in an actual operative situation. They move less like the actual anatomical structures and are, in addition, larger than the lumen of the actual duct, which could make the task artificially easier; vesical catheters, moreover, are more expensive. It did prove necessary in our model to use only 1 click in clip application, as more would compress the retention suture tubing too much, and, especially in cases where more clicks were used, the tubing did sometimes stick to the clip. A better alternative than retention suture tubing may yet be found, but in the meantime we wanted to create a device that would train participants to the appropriate “feel.” Indeed, participants generally agreed that the model’s handling characteristics mimicked those of a real operative experience. Though the overall score on this point did not meet the minimum CVR threshold, this model nonetheless offers direct, tactile feedback in a manner that is superior to computer simulations. It also does not incur their expense, and is available to hospitals which would not be likely to have computer simulations as an option for teaching this procedure. Another simulator model, similar in design to ours,17 is also inexpensive and readily available. It does use vesical catheters rather than retention suture tubing, resulting in the problems with a less accurate “feel” as noted above. As these are also larger than retention suture tubing, even at their smallest sizes, they would potentially make the experience easier for novices by creating larger “targets,” which may have affected their results, since an easier task might allow novices to have times that were not sufficiently statistically different from those of experts. This study did use times as the sole criteria measured and in fact, in terms of IOC, it too did not find a statistically relevant difference between participants of differing experience levels, though they did find statistically relevant differences in experience levels regarding other, more complex procedures not evaluated by our study. In addition, their study group was even smaller than ours, consisting of 5 complete novices, 5 novices, and only 4 experts; they also acknowledge that larger study groups are needed. The actual placement of the IOC catheter was relatively easy—which may explain similar performance in both group. This may be attributed to 3 factors: each participant was allowed 1 untimed practice trial, the setting was much less stressful than the OR and we used a consistent model without all the intraoperative variations of real cases.
5
ARTICLE IN PRESS Their preliminary study of their simulator 10 also did not separately evaluate the relevant sequential steps for IOC as elucidated by the simulator or their completion by participants as our study does, creating a validated checklist to evaluate performance as we did. They also did not utilize evaluations of the device by participants in terms of realism, usefulness or perceived contributions. Our use of consistent experts who assessed participants’ performances, with tests for agreement, also distinguishes our study. Limitations of our study do include several procedural steps and evaluation items not meeting the minimum CVR threshold, and both IOC checklist score and time spent performing IOC not proving statistically different, though both did show differences—if not statistically significant ones, still clearly observable—between low- and high-experience participants. It is hard to say valid but it has proved to be a valuable and acceptable method of training. The small number of participants, another limitation, doubtless contributed to this; a larger study may determine if there are statistically significant differences between experience levels. There are several things our study did not set out to assess, which might be explored by other or future studies. We did not assess whether resident performance differed in actual surgeries after training with the simulator. One way to do so would be to construct a study to compare, for instance, skills, efficiency, clinical outcomes, or possibly subjective confidence between 2 different comparable low-experience groups who did and did not train using the simulator. An evaluation of whether this model prepares participants better for the operative experience in terms of handling, confidence or efficiency compared to a computer simulation could be another avenue for future exploration. In addition, the interpretation of IOC results was not a part of the project, but should be considered in a future study. In spite of the small number of participants, this training tool has been validated and used in our surgery program to complement training, with obvious positive, tangible results. Participants found it valuable and requested more studies such as this, stating increased confidence, skill, and efficiency.
CONCLUSIONS Participants definitely agreed that the model of the gallbladder, cystic duct, and CBD appeared realistic and that the simulation was useful for teaching the relevant steps associated with IOC and for training insertion and manipulation of both the angiocatheter and cholangiocatheter and the application of the clip to the cystic duct
6
to secure the catheter. They also generally agreed that it was useful for training manipulation of the gallbladder, performance of ductotomy, and the injection of contrast agent. As noted, the participants also agreed that the model’s handling characteristics mimic those of a real operative experience. Residents also felt engaged in the simulation experience. They reported that the simulation was challenging and a valuable use of their time, and recommended that more simulation similar to this setup be incorporated in their training. This has become a standard tool in our program. Our target groups are PGY2 and PGY3 residents. Additional studies of this type are warranted to increase the sample size in order to enhance the ability to detect differences in IOC performance between highand low-experience individuals.
REFERENCES 1. Flum DR, Dellinger EP, Cheadle A, Chan L, Koepsell
T. Intraoperative cholangiography and risk of common bile duct injury during cholecystectomy. JAMA. 2003;289:1639–1644. 2. Flum DR, Koepsell T, Heagerty P, Sinanan M, Dellin-
ger EP. Common bile duct injury during laparoscopic cholecystectomy and the use of intraoperative cholangiography: adverse outcome or preventable error? Arch Surg. 2001;136:1287– 1292. 3. Francoeur JR, Wiseman K, Buczkowski AK, Chung
SW, Scudamore CH. Surgeons’ anonymous response after bile duct injury during cholecystectomy. Am J Surg. 2003;185:468–475. 4. Kern KA. Malpractice litigation involving laparo-
scopic cholecystectomy. Cost, cause, and consequences. Arch Surg. 1997;132:392–398. 5. Johnson SR, Koehler A, Pennington LK, Hanto DW.
Long-term results of surgical repair of bile duct injuries following laparoscopic cholecystectomy. Surgery. 2000;128:668–677. 6. Walsh RM, Henderson JM, Vogt DP, Brown N. Long-
term outcome of biliary reconstruction for bile duct injuries from laparoscopic cholecystectomies. Surgery. 2007;142:450–457. 7. Photi ES, El-Hadi A, Brown S, et al. The routine use
of cholangiography for laparoscopic cholecystectomy in the modern era. JSLS. 2017;21. e2017.00032.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS 8. Slim K, Martin G. Does routine intra-operative chol-
13. Hyltander A, Liljegren E, Rhodin PH, Lonroth H. The
angiography reduce the risk of biliary injury during laparoscopic cholecystectomy? An evidence-based approach. J Visc Surg. 2013;150:321–324.
transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc. 2002;16: 1324–1328.
9. Hope WW, Bools L, Hooks WB 3, Adams A, Kotwall
14. Cameron BH, O’Regan PJ, Anderson DL. A pig
CA, Clancy TV. Teaching cholangiography in a surgical residency program. J Surg Educ. 2013;70:243– 247.
model for advanced laparoscopic biliary procedures. Surg Endosc. 1994;8:1423–1424.
10. S anchez A, Rodrıguez, Benıtez O, Sanchez G, De la
Fuente R. Development of a training model for laparoscopic common bile duct exploration. JSLS. 2010;14: 41–47. 11. Lawshe CH. A quantitative approach to content
validity. Personnel Psychol. 1975;28:563–575. 12. Figert PL, Park AE, Witzke DB, Schwartz RW. Trans-
fer of training in acquiring laparoscopic skills. J Am Coll Surg. 2001;193:533–537.
15. Kirwan WO, Kaar TK, Waldron R. Starting laparo-
scopic cholecystectomy the pig as a training model. Ir J Med Sci. 1991;160:243–246. 16. Windsor JA. Laparoscopic exploration of the com-
mon bile duct: a training model. J R Coll Surg Edinb. 1993;38:48–49. 17. S anchez A, Ota~ no N, Rodrıguez O, Sanchez R,
Benıtez G, Schweitzer M. Laparoscopic common bile duct exploration four-task training model: construct validity. JSLS. 2012;16:10–15.
SUPPLEMENTARY INFORMATION Supplementary material associated with this article can be found in the online version at doi:10.1016/j. jsurg.2019.12.008.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
7
Efficacy of a Novel Cholangiogram Simulator for Training Laparoscopic Intraoperative Cholangiography Tyler J. Sbrocchi, MD,* William D. Watson, MD, FACS,† Oscar Ruiz, MD, FACS,* and Ngan Nguyen, PhD† *
Department of Medical Education, OhioHealth Learning, Riverside Methodist Hospital, Columbus, Ohio; and OhioHealth Learning, Riverside Methodist Hospital, Columbus, Ohio
†
OBJECTIVE: Determine the efficacy of a novel low-cost, reusable simulator for training fundamental skills associated with safe and effective intraoperative cholangiography (IOC). DESIGN: The simulator uses a balloon and retention
suture tubing (representing the gallbladder, cystic duct, common bile duct, and common hepatic duct) attached to a piece of wood and placed in a laparoscopic trainer (representing the abdominal cavity) covered by a piece of simulated skin to obscure it. Following a tutorial on performing a simulated IOC using this, simulation participants independently completed a video-recorded simulated IOC and a post-training survey about the appearance and perceived usefulness of the IOC simulator as a training tool. Two experienced surgeons assessed participants’ IOC performance using an IOC procedural checklist developed for this purpose. Procedural time (in seconds) was recorded and used as an additional measure of performance. SETTING: The OhioHealth Learning Center simulation
facility in the Department of Surgery at Riverside Methodist Hospital, a large tertiary care independent medical center that is part of the OhioHealth care system. PARTICIPANTS: Eleven attending surgeons and 16 gen-
eral surgery residents of different levels participated in the simulation. Two experienced surgeons assessed participants’ IOC performance. One participated in the simulation along with the other 10 surgeons; the other did not.
Previously presented at the Innovative Techniques session of the 2018 Annual ACS Surgical Simulation Summit: An International Multi-Professional Meeting, March 16-17, 2018 in Chicago, IL. Correspondence: Inquiries to Tyler J. Sbrocchi, MD, Department of Surgery, OhioHealth Riverside Methodist Hospital, 3535 Olentangy River Road, Columbus, OH 43214-3998; fax: 614-566-1073; e-mail: [email protected]
RESULTS: High-experience participants completed more steps and spent less time than low-experience individuals; however, differences were not statistically significant. There was substantial agreement between the 2 observers regarding participants’ performance. Participants scored the simulator as realistic and useful in teaching relevant steps associated with IOC. CONCLUSIONS: Despite differences between high- and
low-experience participants in steps completed and time spent, these results did not prove statistically significant. Additional studies to increase sample size are warranted to determine if significant differences exist. However, participants did generally find the simulator to be an effective training tool. ( J Surg Ed 000:1 7. Ó 2019 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: simulation training technique, resident
surgical education, laparoscopic cholecystectomy, cholangiography COMPETENCIES: Patient Care, Medical Knowledge
INTRODUCTION Laparoscopic cholecystectomy (LC) is one of the most frequently performed abdominal surgery procedures in the United States, with approximately 750,000 Americans undergoing the procedure each year.1 Despite the frequency and safety of LC, common bile duct (CBD) injuries are a dreaded complication related to LC, occurring in approximately 1 in 200 LC cases.1,2 As many as 50% of surgeons who perform LC are expected to cause such an injury during their career,3 making litigation related to CBD injuries among the leading sources of medical malpractice claims against surgeons.4 Serious injuries to the
Journal of Surgical Education © 2019 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jsurg.2019.12.008
1
ARTICLE IN PRESS CBD often require at least 1 surgical repair, and these repairs have variable long-term outcomes.5,6 Despite its potential for early detection of a major CBD injury,7,8 the routine use of intraoperative cholangiography (IOC) continues to be a matter of surgeon preference. IOC can be safely performed by all surgeons as well as surgical residents at every level of training without adding significant time to the operation9 however, there is little reported on teaching IOC to surgical trainees. While simulation is available for training these fundamental skills without sacrificing patient safety,10 evidence for the effectiveness of simulation-based training in facilitating skills acquisition is limited. The purpose of this study was to determine the efficacy of a novel low-cost simulator for training the fundamental steps and skills associated with carrying out safe and effective IOC.
METHODS Participants and Setting General surgeons and surgical residents at Riverside Methodist Hospital were invited to participate. Participants were asked to complete a demographic survey consisting of questions on sex, age, and experience performing laparoscopic IOC. Twelve attending surgeons participated in the overall study; 11 of them participated in the simulation itself. The twelfth surgeon, along with 1 of the 11 simulation participants, assessed participants’ performance (see “Assessments,” below). All surgeons were males between the ages of 32 and 64 years (mean age = 45.70 § 11.06). All surgeons reported performing IOC as part of LC; 3 surgeons (27%) perform IOC on a routine basis and 8 (73%) on a selective basis. Sixteen general surgery residents participated in the study (males = 13; females = 3). Residents’ age ranged from 26 to 38 years (mean age = 28.8 § 2.7). Five (31%) of the residents were in PGY-1, 3 (19%) PGY-2, 2 (12%) PGY-3, 3 (19%) PGY-4, and 3 (19%) PGY-5. We defined high-experience residents as those who had performed 35 to 100 IOCs, 5 (31%). Those who had performed fewer than 35 IOCs were defined as low-experience, 11 (69%). Among the low-experience residents, 1 had no previous experience with IOC, while 10 had various levels of experience, ranging from 2 to 15 IOCs.
because it is close in size to the cystic duct and because of its resemblance, in terms of softness, handling, and movement, to the actual anatomy. This apparatus was placed in a laparoscopic trainer (Fig. 2) (representing the abdominal cavity) which was covered by a piece of simulated skin to prevent participants from looking into the abdominal cavity. In addition to the simulator, a novel IOC procedural checklist (see Table 1) was developed to assess IOC performance. Assessments Following a tutorial on how to perform a simulated IOC using the novel simulator, all simulation participants independently completed a simulated IOC (see e-Component video). Next, participants completed a post-training survey consisting of 9 questions about the appearance of the IOC simulator and its perceived usefulness as an IOC training tool. Completion of the simulated IOC was video-recorded and independently assessed by 2 experienced general surgeons using the IOC checklist. Additionally, the procedural time (in seconds) was recorded and used as an additional measure of IOC performance. Statistical Analysis Content validity of the IOC checklist was established by determining the content validity ratio (CVR) of each item on the checklist, a technique suggested by Lawshe.11 The CVR is based on 2 psychophysical assumptions: (1) any item in which performance is perceived to be “essential”
Simulator Description The simulator developed by members of the research team consists of a balloon (representing the gallbladder) and retention suture tubing (representing the cystic duct, CBD, and common hepatic duct) attached to a piece of wood (Fig. 1). We chose retention suture tubing
2
FIGURE 1. Balloon and tubing model of gallbladder and ducts.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
TABLE 1. The CVR of Each IOC Checklist Item
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
FIGURE 2. Setup for simulated IOC.
by more than half of the raters has some degree of content validity, and (2) the more raters (> 50%) who perceive the item as being “essential” the greater the degree of its content validity. The CVR is calculated using the following formula: CVR = (e N/2) N/2, in which e is the number of experts indicating “essential” and N is the total number of raters. To determine which assessment tool items to retain, a table of minimum CVR scores for item inclusion was developed based on a 1-tailed test with a significant value of p = 0.05. Items that scored below the minimum CVR threshold were removed from the corresponding assessment tool. Next, the mean CVR for the retained items was calculated to determine the content validity index for the overall content validity of the IOC checklist. Face validity of the IOC simulator was established by calculating the mean Likert scale scores and standard deviations for individual items on the post-training survey. High scores (i.e., scores close to 5 or strongly agree) suggest that the simulation is realistic in terms of appearance, and that it is useful as an education and training tool. A scoring rubric developed to facilitate interpretation of post-training survey results was based on the following scores: Disagree (0.0-2.0); Somewhat disagree (2.1-2.9); Neutral: neither agree nor disagree (3.0); Somewhat agree (3.1-3.9); and Agree (4.0-5.0).
Journal of Surgical Education Volume 00 /Number 00 Month 2019
16 17 18 19 20
Checklist Items
CVR
Grasp and retract gall bladder superiorly Adjust camera in order to view insertion of catheter With an 11 blade, make a 1-3 mm right subcostal incision Insert angiocatheter through incision Insert cholangiocatheter through angiocatheter Grasp and retract infundibulum of gallbladder to adjust angle of cystic duct Place a clip on the junction of the gallbladder and cystic duct Make a small transverse incision on the anterior surface of the cystic duct just below the clip Grasp cholangiocatheter and insert it through cystic ductotomy Clip the cholangiocatheter to fix it in place Inject 3-5 cc of saline to ensure that there is no leakage Remove graspers, ports, and camera Stand behind shield with syringe Verify that everyone is ready for x-ray Aspirate and inject diluted radiocontrast (half contrast, half saline) Obtain fluoroscopic images Reinsert camera and ports Grasp infundibulum of gallbladder Remove clip Grasp and retract cholangiocatheter (but not completely)
0.46 0.46 0.27 0.27 0.64* 0.64* 0.46
0.64* 0.64* 0.46 0.27 0.64* 0.64* 0.45 1.00* 0.64* 0.64* 0.64* -0.09
*p < 0.05.
Construct validity of the IOC simulation was established by using an independent t test to compare the performance (i.e., score on the IOC checklist) of high-experience vs. low-experience participants. Construct validity is established if high-experience participants perform significantly better than their low-experience counterparts. High positive correlations between the raters’ evaluations provide evidence for inter-rater reliability. Kappa statistics were used to assess inter-rater reliability and interobserver agreement. The interpretation of the kappa value was based on the commonly cited scale, with kappa < 0 = less than chance agreement; 0.01 to 0.20 = slight agreement; 0.21 to 0.40 = fair agreement; 0.41 to 0.60 = moderate agreement; 0.61 to 0.80 = substantial agreement; and 0.81 to 1.00 = almost perfect agreement.
RESULTS Content Validity of IOC Checklist The CVR of each checklist item is presented in Table 1. Since the content evaluation panel consists of 11
3
ARTICLE IN PRESS members, a minimum CVR of 0.59 was required to satisfy the 5% level.11 Only 10 items (indicated with an asterisk*) out of the 20 items had CVR values meeting the minimum CVR threshold. These items were retained and used in the evaluation of IOC performance (see Construct validity of the IOC simulation below).
TABLE 3. Residents’ (n = 16) Perceived Experience With the IOC Simulation Survey Questions
Resident Response (Scale: 1-5)* Mean § SD
Face Validity of the IOC Simulator
1
4.56 § 0.63
Table 2 shows residents and surgeons’ (n = 22) response to each item of the post-training survey. On average, participants agreed that the model of the gallbladder, cystic duct and CBD appeared realistic. They also agreed that the simulation was useful for teaching the relevant steps associated with IOC and for training application of the clip to the cystic duct and manipulation of both the angiocatheter and cholangiocatheter. The participants somewhat agreed that the model’s handling characteristics mimicked those of a real operative experience. They also somewhat agreed that the simulation was useful for training manipulation of the gallbladder, performance of cystic ductotomy, and injection of contrast agent.
2
TABLE 2. Participants’ (n = 22) Response to Each Item of the PostTraining Survey Survey Questions
Resident and Attending Mean Response (Scale: 1-5)* Mean § SD
1 2 3
4 5 6 7 8 9
The anatomical model (e.g., gallbladder, cystic duct, and common bile duct) appears realistic The model's handling characteristics (haptics) mimic those of a real operative experience The simulation is useful for teaching the relevant procedural steps associated with laparoscopic intraoperative cholangiogram The simulation is useful for training manipulation of the gallbladder The simulation is useful for training performance of cystic ductotomy The simulation is useful for training insertion and manipulation of angiocatheter The simulation is useful for training insertion and manipulation of cholangiocatheter The simulation is useful for training application of the clip to the cystic duct The simulation is useful for training injection of contrast agent
*Scale: 1 (low) to 5 (high). **Meets minimum CVR threshold.
4
4.00 § 0.82** 3.82 § 0.80 4.50 § 0.60**
3.82 § 0.96
3 4
I was engaged in the simulation experience I felt the simulation was challenging The simulation was a valuable use of my time I would like more simulation similar to this setup incorporated in my training
4.19 § 0.98 4.50 § 0.73 4.50 § 0.82
*Scale: 1 (low) to 5 (high).
Table 3 shows residents’ perception of their overall simulation experience. On average, residents felt engaged in the simulation experience. They reported that the simulation was challenging and a valuable use of their time, and recommended that more simulation similar to this setup be incorporated in their training. Construct Validity of the IOC Simulation Figure 3 shows the mean IOC scores of low- (n = 16) and high-experience (n = 11) participants. Low-experience participants included residents who had performed less than 35 IOCs, while high-experience participants included attending surgeons and residents who had performed 35 to 100 IOCs. On average, high-experience participants performed the same number of steps but spent less time (Fig. 4) on the IOC procedure than lowexperience individuals. However, both IOC checklist score, t(20) = 0.36, and time spent performing IOC, p (20) = 1.89, were not statistically different, p > 0.05. Mean kappa value was 0.63 (range = 0.30-1), p < 0.05, indicating that there was substantial agreement between the 2 observers.
3.82 § 1.10 4.23 § 0.92** 4.32 § 0.72** 4.41 § 0.59** 3.91 § 1.11
DISCUSSION Studies have indicated 12,13 that practice with models can further mastery of steps needed for operative procedures, potentially reducing the risk of complications. Considerations for useful and workable models include cost and availability, quality of tactile feedback, ease of use, and creation or enhancement of a skillset. This study was developed in an effort to find a simple, efficient, inexpensive, and reproducible way to teach the steps to perform IOC to lower-level surgery residents (PGY1-3) prior to performance in the operating room. It
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS
FIGURE 3. Mean § SD IOC scores of low (n = 16) and high-experience (n = 11) participants.
also creates an accurate representation of the main sequential steps performed in the OR during intraoperative cholangiograms. Historically, animal models, porcine in particular, were sometimes used for training laparoscopic biliary procedures in general 14 and LC in particular 15; however, these are limited by availability, cost and time requirements for preparation plus a limited window for training, as well as clearly not being reusable. Windsor’s model,16 which used a saphenous magna vein and tributaries as a substitute for the biliary tract, is hampered by a similar lack of reusability as well as limited availability and, comparably to our model, more expensive. In contrast, costs for our model are decidedly minimal—5 to 10 dollars—save that of a laparoscopic trainer (which nearly every teaching hospital has) it is therefore readily available. In addition, only portions of the model need to be replaced between participants and/or uses, making it easy to train multiple participants. Our model provides a tactile, hands-on method, offering practice in the necessary motions that are used during IOC. The use of retention suture tubing mimics the cystic duct more closely in terms of softness, handling,
FIGURE 4. Time spent on simulation by participants with low vs. high experience—means § SD.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
and movement, as well as lumen size. As part of development both IV tubing, vesical (Foley catheter) and a red rubber catheter were considered harder to cut than the actual cystic duct. These options were also harder in terms of clip application, requiring more clicks to keep the catheter in place than might be needed or used in an actual operative situation. They move less like the actual anatomical structures and are, in addition, larger than the lumen of the actual duct, which could make the task artificially easier; vesical catheters, moreover, are more expensive. It did prove necessary in our model to use only 1 click in clip application, as more would compress the retention suture tubing too much, and, especially in cases where more clicks were used, the tubing did sometimes stick to the clip. A better alternative than retention suture tubing may yet be found, but in the meantime we wanted to create a device that would train participants to the appropriate “feel.” Indeed, participants generally agreed that the model’s handling characteristics mimicked those of a real operative experience. Though the overall score on this point did not meet the minimum CVR threshold, this model nonetheless offers direct, tactile feedback in a manner that is superior to computer simulations. It also does not incur their expense, and is available to hospitals which would not be likely to have computer simulations as an option for teaching this procedure. Another simulator model, similar in design to ours,17 is also inexpensive and readily available. It does use vesical catheters rather than retention suture tubing, resulting in the problems with a less accurate “feel” as noted above. As these are also larger than retention suture tubing, even at their smallest sizes, they would potentially make the experience easier for novices by creating larger “targets,” which may have affected their results, since an easier task might allow novices to have times that were not sufficiently statistically different from those of experts. This study did use times as the sole criteria measured and in fact, in terms of IOC, it too did not find a statistically relevant difference between participants of differing experience levels, though they did find statistically relevant differences in experience levels regarding other, more complex procedures not evaluated by our study. In addition, their study group was even smaller than ours, consisting of 5 complete novices, 5 novices, and only 4 experts; they also acknowledge that larger study groups are needed. The actual placement of the IOC catheter was relatively easy—which may explain similar performance in both group. This may be attributed to 3 factors: each participant was allowed 1 untimed practice trial, the setting was much less stressful than the OR and we used a consistent model without all the intraoperative variations of real cases.
5
ARTICLE IN PRESS Their preliminary study of their simulator 10 also did not separately evaluate the relevant sequential steps for IOC as elucidated by the simulator or their completion by participants as our study does, creating a validated checklist to evaluate performance as we did. They also did not utilize evaluations of the device by participants in terms of realism, usefulness or perceived contributions. Our use of consistent experts who assessed participants’ performances, with tests for agreement, also distinguishes our study. Limitations of our study do include several procedural steps and evaluation items not meeting the minimum CVR threshold, and both IOC checklist score and time spent performing IOC not proving statistically different, though both did show differences—if not statistically significant ones, still clearly observable—between low- and high-experience participants. It is hard to say valid but it has proved to be a valuable and acceptable method of training. The small number of participants, another limitation, doubtless contributed to this; a larger study may determine if there are statistically significant differences between experience levels. There are several things our study did not set out to assess, which might be explored by other or future studies. We did not assess whether resident performance differed in actual surgeries after training with the simulator. One way to do so would be to construct a study to compare, for instance, skills, efficiency, clinical outcomes, or possibly subjective confidence between 2 different comparable low-experience groups who did and did not train using the simulator. An evaluation of whether this model prepares participants better for the operative experience in terms of handling, confidence or efficiency compared to a computer simulation could be another avenue for future exploration. In addition, the interpretation of IOC results was not a part of the project, but should be considered in a future study. In spite of the small number of participants, this training tool has been validated and used in our surgery program to complement training, with obvious positive, tangible results. Participants found it valuable and requested more studies such as this, stating increased confidence, skill, and efficiency.
CONCLUSIONS Participants definitely agreed that the model of the gallbladder, cystic duct, and CBD appeared realistic and that the simulation was useful for teaching the relevant steps associated with IOC and for training insertion and manipulation of both the angiocatheter and cholangiocatheter and the application of the clip to the cystic duct
6
to secure the catheter. They also generally agreed that it was useful for training manipulation of the gallbladder, performance of ductotomy, and the injection of contrast agent. As noted, the participants also agreed that the model’s handling characteristics mimic those of a real operative experience. Residents also felt engaged in the simulation experience. They reported that the simulation was challenging and a valuable use of their time, and recommended that more simulation similar to this setup be incorporated in their training. This has become a standard tool in our program. Our target groups are PGY2 and PGY3 residents. Additional studies of this type are warranted to increase the sample size in order to enhance the ability to detect differences in IOC performance between highand low-experience individuals.
REFERENCES 1. Flum DR, Dellinger EP, Cheadle A, Chan L, Koepsell
T. Intraoperative cholangiography and risk of common bile duct injury during cholecystectomy. JAMA. 2003;289:1639–1644. 2. Flum DR, Koepsell T, Heagerty P, Sinanan M, Dellin-
ger EP. Common bile duct injury during laparoscopic cholecystectomy and the use of intraoperative cholangiography: adverse outcome or preventable error? Arch Surg. 2001;136:1287– 1292. 3. Francoeur JR, Wiseman K, Buczkowski AK, Chung
SW, Scudamore CH. Surgeons’ anonymous response after bile duct injury during cholecystectomy. Am J Surg. 2003;185:468–475. 4. Kern KA. Malpractice litigation involving laparo-
scopic cholecystectomy. Cost, cause, and consequences. Arch Surg. 1997;132:392–398. 5. Johnson SR, Koehler A, Pennington LK, Hanto DW.
Long-term results of surgical repair of bile duct injuries following laparoscopic cholecystectomy. Surgery. 2000;128:668–677. 6. Walsh RM, Henderson JM, Vogt DP, Brown N. Long-
term outcome of biliary reconstruction for bile duct injuries from laparoscopic cholecystectomies. Surgery. 2007;142:450–457. 7. Photi ES, El-Hadi A, Brown S, et al. The routine use
of cholangiography for laparoscopic cholecystectomy in the modern era. JSLS. 2017;21. e2017.00032.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
ARTICLE IN PRESS 8. Slim K, Martin G. Does routine intra-operative chol-
13. Hyltander A, Liljegren E, Rhodin PH, Lonroth H. The
angiography reduce the risk of biliary injury during laparoscopic cholecystectomy? An evidence-based approach. J Visc Surg. 2013;150:321–324.
transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc. 2002;16: 1324–1328.
9. Hope WW, Bools L, Hooks WB 3, Adams A, Kotwall
14. Cameron BH, O’Regan PJ, Anderson DL. A pig
CA, Clancy TV. Teaching cholangiography in a surgical residency program. J Surg Educ. 2013;70:243– 247.
model for advanced laparoscopic biliary procedures. Surg Endosc. 1994;8:1423–1424.
10. S anchez A, Rodrıguez, Benıtez O, Sanchez G, De la
Fuente R. Development of a training model for laparoscopic common bile duct exploration. JSLS. 2010;14: 41–47. 11. Lawshe CH. A quantitative approach to content
validity. Personnel Psychol. 1975;28:563–575. 12. Figert PL, Park AE, Witzke DB, Schwartz RW. Trans-
fer of training in acquiring laparoscopic skills. J Am Coll Surg. 2001;193:533–537.
15. Kirwan WO, Kaar TK, Waldron R. Starting laparo-
scopic cholecystectomy the pig as a training model. Ir J Med Sci. 1991;160:243–246. 16. Windsor JA. Laparoscopic exploration of the com-
mon bile duct: a training model. J R Coll Surg Edinb. 1993;38:48–49. 17. S anchez A, Ota~ no N, Rodrıguez O, Sanchez R,
Benıtez G, Schweitzer M. Laparoscopic common bile duct exploration four-task training model: construct validity. JSLS. 2012;16:10–15.
SUPPLEMENTARY INFORMATION Supplementary material associated with this article can be found in the online version at doi:10.1016/j. jsurg.2019.12.008.
Journal of Surgical Education Volume 00 /Number 00 Month 2019
7