- Email: [email protected]
Strategies for New Skill Acquisition by Practicing Surgeons
ORIGINAL REPORTS
Strategies for New Skill Acquisition by Practicing Surgeons Todd A. Jaffe, BBA,* Steven J. Hasday, BS,* Meghan Knol, MS,* Jason Pradarelli, MD, MS,† Sudha R. Pavuluri Quamme, MD, MS,‡ Caprice C. Greenberg, MD, MPH,‡ and Justin B. Dimick, MD, MPH*,§,║ The University of Michigan Medical School, Ann Arbor, Michigan; †Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts; ‡Department of Surgery, Wisconsin Surgical Outcomes Research Program, University of Wisconsin School of Medicine, Madison, Wisconsin; §Center for Healthcare Outcomes and Policy, The University of Michigan, Ann Arbor, Michigan; and ║Department of Surgery, Michigan Medicine, Ann Arbor, Michigan *
OBJECTIVE: To understand how practicing surgeons utilize
CONCLUSIONS: Practicing surgeons use a variety of
available training methods, which methods are perceived as effective, and important barriers to using more effective methods.
training methods when learning new procedures and technologies, and there is disconnect between commonly used training methods and those deemed most effective. Confidence in surgeon’s ability was cited as a reason for this discrepancy; and surgeons found time associated with more effective methods to be prohibitive. ( J Surg Ed ]:]]]-]]]. C J 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
DESIGN: Online survey designed to characterize surgeon
utilization and perception of available training methods. SETTING: Two large Midwestern academic health centers. PARTICIPANTS: 150 faculty surgeons. METHODS: Nominal values were compared using a
McNemar’s Test and Likert-like values were compared using a paired t-test (IBM SPSS Statistics v. 21.0; New York, NY). RESULTS: Survey response rate was 81% (122/150). 98%
of surgeons reported learning a new procedure or technology after formal training. Many surgeons reported scrubbing in expert cases (78%) and self-directed study (66%), while few surgeons (6%) completed a mini-fellowship. The modalities used most commonly were scrubbing in expert cases (34%) and self-directed study (27%). Few surgeons (7%) believed self-directed study would be most effective, whereas 31% and 16% believed operating under supervision and mini-fellowships would be most effective, respectively. Surgeons believed more effective methods “would require too much time” or they had “confidence in their ability to implement safely.” Funding/Support: Dr Dimick receives grant funding from the National Institutes of Health (NIH), the Agency for Healthcare Research and Quality (AHRQ), and BlueCross BlueShield of Michigan Foundation. Conflict of Interest/Disclosures: Dr Dimick is a cofounder of ArborMetrix, Inc, a company that makes software for profiling hospital quality and efficiency. Correspondence: Inquiries to Todd A. Jaffe, BBA, The University of Michigan Medical School, 2800 Plymouth Road, Building 16, 1st Floor, Ann Arbor, MI 48109; e-mail: [email protected]
KEY WORDS: minimally invasive surgery, surgical educa-
tion, surgery, continuing medical education ACGME COMPETENCIES: Patient Care, Medical Knowl-
edge, Practice Based Learning and Improvement, Systems Based Practice
INTRODUCTION Surgical practice is rapidly evolving. New surgical procedures (e.g., sleeve gastrectomy and oncoplastic lumpectomy) and technologies (e.g., laparoscopy and robotics) are developed and introduced regularly.1-4 Practicing surgeons adopt these new techniques to stay current in their field and to improve patient outcomes.5 However, many new procedures and technologies have substantial learning curves which may lead to suboptimal patient outcomes early in the surgeon's experience.6 There is clear evidence of harms associated with the introduction of new techniques in diverse practice settings.7-10 This data suggest that the existing process of the diffusion of new procedures and technologies into practice could be improved. How surgeons choose to learn new procedures may play a role in the suboptimal outcomes achieved as they navigate their learning curve. During residency training, surgeons
Journal of Surgical Education & 2017 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2017.09.016
1
learn and develop their skills under the supervision of senior surgeons and are granted progressive responsibility as they demonstrate improvement.11,12 In contrast, practicing surgeons seldom have access to these structured training methods when learning new procedures or technologies.13 The training methods used by practicing surgeons and surgeons′ perceptions of their relative strengths and weaknesses are poorly characterized.14 In addition, little is known about the practical barriers faced by surgeons when choosing among training modalities. We surveyed practicing surgeons to understand how they learn new procedures and technologies. We first assessed how practicing surgeons use available training methods and the factors that inform their choices. We then examined how those same surgeons believe training could be most effectively used to facilitate improved learning and safer diffusion. Finally, we explored the factors accounting for any discrepancies between current usage and beliefs about most-effective training.
MATERIALS AND METHODS Survey Design and Development We first conducted interviews with 9 practicing surgeons on faculty in the department of surgery at the University of Michigan Health System to understand the most significant questions to ask surgeons about learning new procedures and technologies (T.J., S.H., and M.K.). Interview topics were analyzed and thematically coded. Based on the themes that emerged from the interviews, an initial survey was created using Qualtrics Software (Version 08.2015; Qualtrics, Provo, UT). The preliminary survey was presented to initial interviewees within the department of surgery for feedback. Cognitive interviews were conducted with these surgeons in which they read through the survey instrument with members of the research team. These cognitive interviews provided an opportunity to recognize survey participant comprehension and alignment with question intent.15,16 Feedback from those surgeons was again analyzed and coded, then used to refine the survey. Primary themes generated from interviews fell into 4 general categories: (1) motivations and pressures to innovate; (2) variability in educational modalities available and used; (3) safe introduction and diffusion of new procedures/ technologies in to surgical practice; and (4) ethics of consenting and performing new procedures on patients. From these themes, we designed a 22 question survey to evaluate how practicing surgeons use available training methods when learning new procedures and technologies and the safety and ethical considerations associated with their implementation. This article discusses the variability in educational modalities available and used by practicing surgeons. 2
Participants were asked to select the training methods they used to learn a new procedure or technology after completing their formal training. Answer choices were created to be inclusive of the most common training modalities, and respondents were asked to select all the methods used in a “select all that apply” format. The training modalities were chosen based on the initial interviews with surgeons and were refined from feedback during cognitive interviews. To promote consistency among respondent understanding, the definitions of these learning modalities were reviewed during cognitive interviews.17 One such example is the definition of a “minifellowship,” which was defined as putting one's practice “on hold” to pursue a full time, immersive training program (several weeks to months) that includes handson operative and clinical training as outlined in surgical literature.18 Respondents then selected which one modality they used most commonly to learn a new procedure or technology. The modalities available for surgeons to select for their single most commonly used modality were automatically populated from their answer to the previous “select all that apply” question. Surgeons next indicated which modality they perceived would be most effective. Surgeons who had discordance between what they perceived to be most effective and what they most commonly used were asked to identify specific factors that accounted for this discordance. Answer options for these factors were also chosen based on initial interviews and refined from cognitive interviews. The question prompts and training modality response options are included in Table 1. Survey Administration and Study Population Surgical specialties to be surveyed were those within the department of surgery at these institutions and were reviewed during initial interviews. These specialties included general surgery, thoracic surgery, colorectal surgery, vascular surgery, pediatric surgery, minimally invasive surgery/bariatric surgery, transplant surgery, endocrine surgery, surgical oncology, and trauma/acute care surgery. From August 2015 to July 2016, the survey instrument was distributed via e-mail to 150 surgeons on faculty in the department of surgery at 2 large Midwest academic health centers. With the specific intention of maximizing response rate, these e-mails were drafted with input from a faculty member in each respective department of surgery, and distribution e-mails were sent using that faculty member's name in the e-mail address. The research team sent 2 follow-up e-mails at 2-week intervals from that same e-mail address using the faculty members' names with the goal of further improving response rate. Although e-mails were sent with the faculty members as the “sender,” all distribution and response monitoring was contained within the Qualtrics software. Respondent characteristics are included in Table 2. Journal of Surgical Education Volume ]/Number ] ] 2017
TABLE 1. Survey Questions and Responses for Training Methods Questions
Responses
Which training method have you most commonly used to learn a new surgical procedure or technology? (Please select only 1 answer)
• I have not learned any new surgical procedures since completing my formal training
• Self-directed study using published materials (e.g., textbooks and
online videos) Which training method do you believe would be the most effective way for you to learn a new surgical • Didactic curriculum combined with hands-on training in simulator or animal/cadaver laboratory (e.g., a 1-2 d course) procedure or technology in the future? (Please select only 1 answer) • Observing or scrubbing in on patient cases performed by another surgeon • Performing surgical cases under the supervision of a proctor or mentor • Participating in a mini-fellowship (i.e., 2-12 wk formal course)
Statistical Analysis Survey participants were allowed to complete the survey only once, and responses were aggregated using the Qualtrics software. To assess statistical differences between the number of surgeons who most commonly used a given TABLE 2. Characteristics of Survey Participants Characteristic
N (%)
Sex Male 84 (74%) Female 29 (26%) Years in practice 0-5 38 (33%) 6-10 20 (18%) 11-15 21 (19%) 16þ 34 (30%) New procedures since completing formal training 0 5 (4%) 1-3 43 (38%) 4-6 35 (31%) 7-9 12 (11%) 10þ 18 (16%) Surgical cases performed per week 0 3 (3%) 1-5 48 (42%) 6-10 50 (44%) 11þ 12 (11%) Level of training Residency only 16 (14%) Fellowship 69 (62%) Postfellowship 27 (24%) Surgical specialty General 20 (18.0%) Thoracic 12 (11%) Colorectal 8 (7%) Vascular 15 (14%) Pediatric 11 (10%) MIS and bariatric 8 (7%) Transplant 8 (7%) Endocrine 8 (7%) Surgical oncology 10 (9%) Trauma 11 (10%) MIS, minimally invasive surgery. Journal of Surgical Education Volume ]/Number ] ] 2017
learning modality and those who perceived it to be most effective, nominal values of the responses were compared using McNemar tests. To evaluate surgeon's rationale for the discrepancy between their most commonly used learning modality and those deemed most effective, average values were calculated using a 1 to 5 Likert scale as indicated by surgeon's response (1 ¼ not at all important, 2 ¼ slightly important; 3 ¼ moderately important; 4 ¼ very important; and 5. ¼ extremely important). These Likert-like ordinal values were compared using a paired t-test (IBM SPSS Statistics v. 21.0; New York, NY).
RESULTS The survey response rate was 81% (122/150). A total 119/122 (98%) of surgeons reported they have learned a new procedure or technology since completing their formal training. The most-used learning modalities were scrubbing in on cases performed by another surgeon (78%), self-directed study (66%), performing cases under the supervision of a proctor/ mentor (56%), and didactic curriculum with simulation or cadaver laboratory (50%). Only 6% of surgeons indicated they participated in a mini-fellowship (Fig. 1). Surgeons were also asked to identify the single training method they used most commonly. Surgeons reported that this most commonly used training method was scrubbing in patient cases performed by another surgeon (34%) or selfdirected study (29%). Those considered to be most effective included performing cases under the supervision of a proctor/mentor (31%) and scrubbing in cases (29%). There were significant discrepancies between training methods commonly used and those perceived to be the most effective (Fig. 2). For example, 29% of surgeons most commonly used self-directed study, however only 7% felt this method was the most effective (p o 0.01). Similarly, for performing under proctor supervision, 15% of surgeons most commonly used the modality and 31% believed it to be most effective (p o 0.01). Very few surgeons (2%) most 3
100%
Percent of Respondents
80%
60%
40%
20%
0% Have not learned Self-directed study Didactic with Scrubbing in cases Performing under Participating in a any new procedures simulation/cadaver performed by proctor supervision mini-fellowship lab another surgeon
Learning Modality
FIGURE 1. Training methods used to learn a new surgical procedure or technology. Note: Question asked in "select all the apply" format to characterize usage of training methods.
commonly used mini-fellowships, yet 16% believed it to be most effective (p o 0.01). Respondents were asked to identify the importance of various factors to account for this discrepancy between the most common modality used and those perceived to be most effective. Surgeons indicated that the greatest deterrent from using more effective methods were that the modalities would “require too much time” (3.71) and “confidence that they could implement the new procedure/technology safely without the need for more rigorous/effective training” (3.56). Requiring too much time was a significantly greater factor than “lack of awareness of other opportunities” (3.15; p o 0.01) and “prohibitively expensive” (3.10; p o 0.01). Confidence that the surgeon could implement safely was significantly more important than “prohibitively expensive” (p ¼ 0.04), but not lack of awareness (p ¼ 0.12). Not needing additional training for credentialing/privileging requirements was less important for surgeons compared with the other factors (2.37, p o 0.01) (Fig. 3).
DISCUSSION Our survey results support that surgeons use a wide variety of training modalities when learning new procedures and technologies. The training methods commonly used include self-directed study, didactic learning, simulation laboratories, and scrubbing in cases performed by an expert. Paradoxically we found that practicing surgeons often do not use the learning modalities they deem to be most effective (e.g., performing cases under the supervision of a proctor and completion of a mini-fellowship). The most commonly cited factor contributing to this discrepancy was that more effective training modalities require too much time away from practice. Perhaps most importantly, surgeons' confidence in their own ability to safely implement the new procedure or technology without further training was a critically important finding for understanding why surgeons do not pursue the most-effective learning methods.
40% 35% 30% 25% 20% 15% 10% 5% 0% Self-directed study
Didactic with simulation/cadaver lab
Scrubbing in cases Performing under proctor Participating in a minisupervision fellowship performed by another surgeon
Most Commonly Used
Most Effective
FIGURE 2. Training method most commonly used and perceived to be most effective. Note: Question asked in "select one" format to assess common usage and impression of effectiveness. 4
Journal of Surgical Education Volume ]/Number ] ] 2017
5
Level of Importance
4
3
2
1
0 Program would require too much time
Confidence that I can implement safely
Lack of awareness of Prohibitively expensive available opportunities
Not needed to meet credentialing requirements
FIGURE 3. Rationale for discrepancy between commonly used method and most effective.
Prior work related to this topic has been focused on safety concerns associated with the learning and dissemination of new procedures and technologies. One such concern relates to the surgical learning curve, which suggests that surgical risks may be higher early on in a provider's cumulative experience.19-22 For example, Nallamothu et al.23 found that for patient's undergoing carotid stenting, early operator experience was associated with increased 30-day mortality (2.3%) compared with late experience (1.4%). Another study by Maruthappu et al.24 found a significant reduction in operative time associated with provider experience across multiple surgical specialties. Further research has examined the effectiveness of various learning modalities. There is evidence that more extensive training improves surgeons' perceptions of their capacity and increases future adoption.18,25-27 For example, investigations by Heniford et al.28, 29 found that 93% of surgeons who underwent preceptorship adopted laparoscopic splenectomy compared with only 6% of surgeons who participated in a 1-day training course. However, this body of work does not evaluate the safety of these procedures once surgeons have adopted them. Furthermore, there remains a paucity of literature characterizing which training modalities are most commonly used, and whether surgeons perceive these to be most effective. Our work expands on these previous findings by reporting the training modalities that surgeons use and surgeons′ perceptions of their comparative effectiveness. Although previous studies identified harms associated with inadequate training, our findings identify an important gap between the most commonly used methods for learning a new surgical procedure and the most effective methods. Furthermore, the scope of this study's findings are broad as we have documented surgical perception and training usage across multiple specialties. Our investigation also goes beyond previous work by reporting the barriers to usage of more effective methods and their perceived importance. Although
the benefits of comprehensive training modalities have been documented in recent literature, the constraints responsible for their low usage have not yet been addressed.30,31 This study's limitations deserve consideration. Respondents only included a subset of academic surgeons at 2 large Midwest hospitals. Our findings may vary with a more diverse sample of surgeons, including nonacademic settings and smaller hospitals. Nonetheless, understanding adoption at academic institutions remains important. Our findings are also subject to the limitations commonly associated with survey research including respondents understanding the limited number of available responses.32 As described in our methods, we conducted multiple interviews with surgeons of different experience and specialty while designing our survey to ensure that we captured all potential concerns. One such example is surgeon's definition of a “new procedure” and “new technology.” We found consistency among the responses of our interviewees that the term “new procedure” did not included minor variations, as most felt they make repeated, iterative changes to procedures throughout their careers. Another limitation is that responses may vary based on surgeon's experience with minimally invasive surgery. However, we specifically surveyed practicing surgeons which limits some variation, and furthermore, capturing beliefs of surgeons of different experience is beneficial in itself. Our survey research also does not directly evaluate outcomes, but instead examines surgeon perception. Now that we have cataloged the learning methods commonly used, however, there is an opportunity for further exploration to directly assess the effect that usage of these different methods has on surgical outcomes. It becomes increasingly important to monitor the safe diffusion of new innovations as the rate of technological advancement in surgical practice continues to rise.33,34 Our study suggests that nearly all practicing academic surgeons are learning new procedures or technologies after completing their
Journal of Surgical Education Volume ]/Number ] ] 2017
5
formal training. The efficacy of this training is affected by the complexity of the new innovation as well as surgeon-specific factors, including surgical experience and skill.35,36 Literature suggests that the introduction of novel technologies, including simulation and video review, can improve surgical skill and accelerate the learning curve.37-39 Usage of these strategies may enable more efficient training in practice and limit the time that surgeons are away from practice. Improving how practicing surgeons learn new procedures may also support hospitals in ensuring their safe introduction. The processes of confirming the qualifications of licensed physicians (credentialing) and authorizing them for specific patient care services (privileging) remain central to regulating how and when practicing surgeons may implement something new.40 New skill assessment modalities, including video review and teleproctoring, may help ensure safe diffusion of new technologies as these modes of direct observation have shown to be reliable surgical performance assessment tools.41-43 These platforms can provide surgeons with an objective review of their performance and an additional safeguard against premature adoption without requiring significant time away from surgical practice.
CONCLUSIONS Our study found that surgeons use a wide variety of methods when learning and implementing new procedures and technologies into practice. Surgeons also reported that the learning modalities they most commonly used were not necessarily those deemed the most effective. This discrepancy was largely owing to the perceived time commitment to more intensive training methods and surgeons′ confidence in implementing the new innovation safely on their own. The safety implications of surgeons using less effective training methods are vast, and the responsibility to regulate this process remains in flux. As surgical innovations continue to be introduced, effort must be directed to incentivizing safe and effective uptake in clinical practice.
REFERENCES 1. Ritchie WP, Rhodes RS, Biester TW. Work loads and
practice patterns of general surgeons in the United States, 1995–1997: a report from the American Board of Surgery. Ann Surg. 1999;230:533-542 [discussion 542–543].
6
3. Khosla A, Wagner AA. Robotic surgery of the kidney,
bladder, and prostate. Surg Clin North Am. 2016;96: 615-636. 4. Suda K, Nakauchi M, Inaba K, Ishida Y, Uyama I.
Robotic surgery for upper GI cancer: current status and future perspectives. Dig Endosc. 2016;28(7): 701-713. 5. Cosgrove DM. The innovation imperative. J Thorac
Cardiovasc Surg. 2000;120(5):839-842. 6. Gurgacz SL, Smith JA, Truskett PG, Babidge WJ,
Maddern GJ. Credentialing of surgeons: a systematic review across a number of jurisdictions. ANZ J Surg. 2012;82:492-498. 7. Jung do H, Son SY, Park YS, et al. The learning curve
associated with laparoscopic total gastrectomy. Gastric Cancer. 2016;19:264-272. 8. Arora KS, Khan N, Abboudi H, Khan MS, Dasgupta
P, Ahmed K. Learning curves for cardiothoracic and vascular surgical procedures—a systematic review. Postgrad Med. 2015;127:202-214. 9. Jeong O, Ryu SY, Choi WY, Piao Z, Park YK. Risk
factors and learning curve associated with postoperative morbidity of laparoscopic total gastrectomy for gastric carcinoma. Ann Surg Oncol. 2014;21: 2994-3001. 10. Abboudi H, Khan MS, Guru KA, et al. Learning
curves for urological procedures: a systematic review. BJU Int. 2014;114:617-629. 11. Education ACfCM. ACCME Annual Report. 2014. 12. Education AcfGM. ACGME Program Requirements
for Graduate Medical Education in General Surgery. 2014. 13. Rogers DA, Elstein AS, Bordage G. Improving con-
tinuing medical education for surgical techniques: applying the lessons learned in the first decade of minimal access surgery. Ann Surg. 2001;233:159-166. 14. Cook JA, Ramsay CR, Carr AJ, Rees JL. A question-
naire elicitation of surgeons' belief about learning within a surgical trial. PLoS One. 2012;7:e49178. 15. Willis GB. Cognitive Interviewing: A Tool for
Improving Questionnaire Design. Sage Publications; 2004.
2. Valentine RJ, Jones A, Biester TW, Cogbill TH,
16. DeMaio TJ, Landreth A. Do different cognitive inter-
Borman KR, Rhodes RS. General surgery workloads and practice patterns in the United States, 2007 to 2009: a 10-year update from the American Board of Surgery. Ann Surg. 2011;254:520-525 [discussion 525–526].
view techniques produce different results? In: Presser S, Rothgeb J, Couper M, Lessler J, Martin E, Singer S, editors. Methods for Testing and Evaluating Survey Questionnaires. New York: John Wiley & Sons, Inc, 2004. p. 89-108. Journal of Surgical Education Volume ]/Number ] ] 2017
17. Zorn KC, Gautam G, Shalhav AL, et al. Training,
30. Birch DW, Sample C, Gupta R. The impact of a
credentialing, proctoring and medicolegal risks of robotic urological surgery: recommendations of the society of urologic robotic surgeons. J Urol. 2009; 182:1126-1132.
comprehensive course in advanced minimal access surgery on surgeon practice. Can J Surg. 2007;50:9-12.
18. Cottam D, Holover S, Mattar SG, et al. The mini-
fellowship concept: a six-week focused training program for minimally invasive bariatric surgery. Surg Endosc. 2007;21:2237-2239. 19. Khan N, Abboudi H, Khan MS, Dasgupta P, Ahmed
31. Nakata BN, Cavalini W, Bonin EA, Salvalaggio PR,
Loureiro MP. Impact of continuous training through distributed practice for acquisition of minimally invasive surgical skills. Surg Endosc. 2017. 32. Kelley K, Clark B, Brown V, Sitzia J. Good practice in
the conduct and reporting of survey research. Int J Qual Health Care. 2003;15:261-266.
K. Measuring the surgical 'learning curve′: methods, variables and competency. BJU Int. 2014;113: 504-508.
33. Satava RM. Advanced technologies and the future of
20. Hopper AN, Jamison MH, Lewis WG. Learning
34. Satava R. Innovative technologies. The information
medicine and surgery. Yonsei Med J. 2008;49:873-878.
curves in surgical practice. Postgrad Med J. 2007;83: 777-779.
age and the biointelligence age. Surg Endosc. 2000;14: 417-418.
21. Boone BA, Zenati M, Hogg ME, et al. Assessment of
35. Ergina PL, Cook JA, Blazeby JM, et al. Challenges
quality outcomes for robotic pancreaticoduodenectomy: identification of the learning curve. J Am Med Assoc Surg. 2015;150:416-422.
36. Sheetz KH, Norton EC, Birkmeyer JD, Dimick JB.
22. Kretschmer A, Mandel P, Buchner A, Stief CG, Tilki
D. Surgical learning curve for open radical prostatectomy: is there an end to the learning curve? World J Urol. 2015;33:1721-1727. 23. Nallamothu BK, Gurm HS, Ting HH, et al. Operator
experience and carotid stenting outcomes in Medicare beneficiaries. J Am Med Assoc. 2011;306:1338-1343. 24. Maruthappu M, Duclos A, Lipsitz SR, Orgill D,
Carty MJ. Surgical learning curves and operative efficiency: a cross-specialty observational study. Br Med J Open. 2015;5:e006679. 25. Aggarwal R, Mytton OT, Derbrew M, et al. Training
and simulation for patient safety. Qual Saf Health Care. 2010;19(suppl 2)):i34-i43. 26. Brunckhorst O, et al. Training, simulation, the learn-
ing curve, and how to reduce complications in urology. Eur Urol Focus. 2016;2(1):10-18. 27. See WA, Cooper CS, Fisher RJ. Predictors of laparo-
scopic complications after formal training in laparoscopic surgery. J Am Med Assoc. 1993;270:2689-2692. 28. Heniford BT, Backus CL, Matthews BD, Greene FL,
Teel WB, Sing RF. Optimal teaching environment for laparoscopic splenectomy. Am J Surg. 2001;181: 226-230.
in evaluating surgical innovation. Lancet. 2009;374: 1097-1104. Provider Experience and the Comparative Safety of Laparoscopic and Open Colectomy. Health Serv Res. 2017;52:56-73. 37. Aggarwal R, Moorthy K, Darzi A. Laparoscopic
skills training and assessment. Br J Surg. 2004;91: 1549-1558. 38. Ibrahim AM, Varban OA, Dimick JB. Novel uses
of video to accelerate the surgical learning curve. J Laparoendosc Adv Surg Tech A. 2016;26:240-242. 39. Hasan A, Pozzi M, Hamilton JR. New surgical
procedures: can we minimise the learning curve? Br Med J. 2000;320:171-173. 40. Pradarelli JC, Campbell DA Jr, Dimick JB. Hospital
credentialing and privileging of surgeons: a potential safety blind spot. J Am Med Assoc. 2015;313: 1313-1314. 41. Schlachta CM, Nguyen NT, Ponsky T, Dunkin B.
Project 6 Summit: SAGES telementoring initiative. Surg Endosc. 2016;30:3665-3672. 42. Birkmeyer JD, Finks JF, O’Reilly A, et al. Surgical skill
and complication rates after bariatric surgery. N Engl J Med. 2013;369:1434-1442. 43. Scally CP, Varban OA, Carlin AM, Birkmeyer JD,
teaching environment for laparoscopic ventral herniorrhaphy. Hernia. 2002;6:17-20.
Dimick JB. Michigan Bariatric Surgery C. Video Ratings of Surgical Skill and Late Outcomes of Bariatric Surgery. J Am Med Assoc Surg. 2016;151: e160428.
Journal of Surgical Education Volume ]/Number ] ] 2017
7
29. Heniford BT, Matthews BD, Box EA, et al. Optimal
Strategies for New Skill Acquisition by Practicing Surgeons Todd A. Jaffe, BBA,* Steven J. Hasday, BS,* Meghan Knol, MS,* Jason Pradarelli, MD, MS,† Sudha R. Pavuluri Quamme, MD, MS,‡ Caprice C. Greenberg, MD, MPH,‡ and Justin B. Dimick, MD, MPH*,§,║ The University of Michigan Medical School, Ann Arbor, Michigan; †Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts; ‡Department of Surgery, Wisconsin Surgical Outcomes Research Program, University of Wisconsin School of Medicine, Madison, Wisconsin; §Center for Healthcare Outcomes and Policy, The University of Michigan, Ann Arbor, Michigan; and ║Department of Surgery, Michigan Medicine, Ann Arbor, Michigan *
OBJECTIVE: To understand how practicing surgeons utilize
CONCLUSIONS: Practicing surgeons use a variety of
available training methods, which methods are perceived as effective, and important barriers to using more effective methods.
training methods when learning new procedures and technologies, and there is disconnect between commonly used training methods and those deemed most effective. Confidence in surgeon’s ability was cited as a reason for this discrepancy; and surgeons found time associated with more effective methods to be prohibitive. ( J Surg Ed ]:]]]-]]]. C J 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
DESIGN: Online survey designed to characterize surgeon
utilization and perception of available training methods. SETTING: Two large Midwestern academic health centers. PARTICIPANTS: 150 faculty surgeons. METHODS: Nominal values were compared using a
McNemar’s Test and Likert-like values were compared using a paired t-test (IBM SPSS Statistics v. 21.0; New York, NY). RESULTS: Survey response rate was 81% (122/150). 98%
of surgeons reported learning a new procedure or technology after formal training. Many surgeons reported scrubbing in expert cases (78%) and self-directed study (66%), while few surgeons (6%) completed a mini-fellowship. The modalities used most commonly were scrubbing in expert cases (34%) and self-directed study (27%). Few surgeons (7%) believed self-directed study would be most effective, whereas 31% and 16% believed operating under supervision and mini-fellowships would be most effective, respectively. Surgeons believed more effective methods “would require too much time” or they had “confidence in their ability to implement safely.” Funding/Support: Dr Dimick receives grant funding from the National Institutes of Health (NIH), the Agency for Healthcare Research and Quality (AHRQ), and BlueCross BlueShield of Michigan Foundation. Conflict of Interest/Disclosures: Dr Dimick is a cofounder of ArborMetrix, Inc, a company that makes software for profiling hospital quality and efficiency. Correspondence: Inquiries to Todd A. Jaffe, BBA, The University of Michigan Medical School, 2800 Plymouth Road, Building 16, 1st Floor, Ann Arbor, MI 48109; e-mail: [email protected]
KEY WORDS: minimally invasive surgery, surgical educa-
tion, surgery, continuing medical education ACGME COMPETENCIES: Patient Care, Medical Knowl-
edge, Practice Based Learning and Improvement, Systems Based Practice
INTRODUCTION Surgical practice is rapidly evolving. New surgical procedures (e.g., sleeve gastrectomy and oncoplastic lumpectomy) and technologies (e.g., laparoscopy and robotics) are developed and introduced regularly.1-4 Practicing surgeons adopt these new techniques to stay current in their field and to improve patient outcomes.5 However, many new procedures and technologies have substantial learning curves which may lead to suboptimal patient outcomes early in the surgeon's experience.6 There is clear evidence of harms associated with the introduction of new techniques in diverse practice settings.7-10 This data suggest that the existing process of the diffusion of new procedures and technologies into practice could be improved. How surgeons choose to learn new procedures may play a role in the suboptimal outcomes achieved as they navigate their learning curve. During residency training, surgeons
Journal of Surgical Education & 2017 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2017.09.016
1
learn and develop their skills under the supervision of senior surgeons and are granted progressive responsibility as they demonstrate improvement.11,12 In contrast, practicing surgeons seldom have access to these structured training methods when learning new procedures or technologies.13 The training methods used by practicing surgeons and surgeons′ perceptions of their relative strengths and weaknesses are poorly characterized.14 In addition, little is known about the practical barriers faced by surgeons when choosing among training modalities. We surveyed practicing surgeons to understand how they learn new procedures and technologies. We first assessed how practicing surgeons use available training methods and the factors that inform their choices. We then examined how those same surgeons believe training could be most effectively used to facilitate improved learning and safer diffusion. Finally, we explored the factors accounting for any discrepancies between current usage and beliefs about most-effective training.
MATERIALS AND METHODS Survey Design and Development We first conducted interviews with 9 practicing surgeons on faculty in the department of surgery at the University of Michigan Health System to understand the most significant questions to ask surgeons about learning new procedures and technologies (T.J., S.H., and M.K.). Interview topics were analyzed and thematically coded. Based on the themes that emerged from the interviews, an initial survey was created using Qualtrics Software (Version 08.2015; Qualtrics, Provo, UT). The preliminary survey was presented to initial interviewees within the department of surgery for feedback. Cognitive interviews were conducted with these surgeons in which they read through the survey instrument with members of the research team. These cognitive interviews provided an opportunity to recognize survey participant comprehension and alignment with question intent.15,16 Feedback from those surgeons was again analyzed and coded, then used to refine the survey. Primary themes generated from interviews fell into 4 general categories: (1) motivations and pressures to innovate; (2) variability in educational modalities available and used; (3) safe introduction and diffusion of new procedures/ technologies in to surgical practice; and (4) ethics of consenting and performing new procedures on patients. From these themes, we designed a 22 question survey to evaluate how practicing surgeons use available training methods when learning new procedures and technologies and the safety and ethical considerations associated with their implementation. This article discusses the variability in educational modalities available and used by practicing surgeons. 2
Participants were asked to select the training methods they used to learn a new procedure or technology after completing their formal training. Answer choices were created to be inclusive of the most common training modalities, and respondents were asked to select all the methods used in a “select all that apply” format. The training modalities were chosen based on the initial interviews with surgeons and were refined from feedback during cognitive interviews. To promote consistency among respondent understanding, the definitions of these learning modalities were reviewed during cognitive interviews.17 One such example is the definition of a “minifellowship,” which was defined as putting one's practice “on hold” to pursue a full time, immersive training program (several weeks to months) that includes handson operative and clinical training as outlined in surgical literature.18 Respondents then selected which one modality they used most commonly to learn a new procedure or technology. The modalities available for surgeons to select for their single most commonly used modality were automatically populated from their answer to the previous “select all that apply” question. Surgeons next indicated which modality they perceived would be most effective. Surgeons who had discordance between what they perceived to be most effective and what they most commonly used were asked to identify specific factors that accounted for this discordance. Answer options for these factors were also chosen based on initial interviews and refined from cognitive interviews. The question prompts and training modality response options are included in Table 1. Survey Administration and Study Population Surgical specialties to be surveyed were those within the department of surgery at these institutions and were reviewed during initial interviews. These specialties included general surgery, thoracic surgery, colorectal surgery, vascular surgery, pediatric surgery, minimally invasive surgery/bariatric surgery, transplant surgery, endocrine surgery, surgical oncology, and trauma/acute care surgery. From August 2015 to July 2016, the survey instrument was distributed via e-mail to 150 surgeons on faculty in the department of surgery at 2 large Midwest academic health centers. With the specific intention of maximizing response rate, these e-mails were drafted with input from a faculty member in each respective department of surgery, and distribution e-mails were sent using that faculty member's name in the e-mail address. The research team sent 2 follow-up e-mails at 2-week intervals from that same e-mail address using the faculty members' names with the goal of further improving response rate. Although e-mails were sent with the faculty members as the “sender,” all distribution and response monitoring was contained within the Qualtrics software. Respondent characteristics are included in Table 2. Journal of Surgical Education Volume ]/Number ] ] 2017
TABLE 1. Survey Questions and Responses for Training Methods Questions
Responses
Which training method have you most commonly used to learn a new surgical procedure or technology? (Please select only 1 answer)
• I have not learned any new surgical procedures since completing my formal training
• Self-directed study using published materials (e.g., textbooks and
online videos) Which training method do you believe would be the most effective way for you to learn a new surgical • Didactic curriculum combined with hands-on training in simulator or animal/cadaver laboratory (e.g., a 1-2 d course) procedure or technology in the future? (Please select only 1 answer) • Observing or scrubbing in on patient cases performed by another surgeon • Performing surgical cases under the supervision of a proctor or mentor • Participating in a mini-fellowship (i.e., 2-12 wk formal course)
Statistical Analysis Survey participants were allowed to complete the survey only once, and responses were aggregated using the Qualtrics software. To assess statistical differences between the number of surgeons who most commonly used a given TABLE 2. Characteristics of Survey Participants Characteristic
N (%)
Sex Male 84 (74%) Female 29 (26%) Years in practice 0-5 38 (33%) 6-10 20 (18%) 11-15 21 (19%) 16þ 34 (30%) New procedures since completing formal training 0 5 (4%) 1-3 43 (38%) 4-6 35 (31%) 7-9 12 (11%) 10þ 18 (16%) Surgical cases performed per week 0 3 (3%) 1-5 48 (42%) 6-10 50 (44%) 11þ 12 (11%) Level of training Residency only 16 (14%) Fellowship 69 (62%) Postfellowship 27 (24%) Surgical specialty General 20 (18.0%) Thoracic 12 (11%) Colorectal 8 (7%) Vascular 15 (14%) Pediatric 11 (10%) MIS and bariatric 8 (7%) Transplant 8 (7%) Endocrine 8 (7%) Surgical oncology 10 (9%) Trauma 11 (10%) MIS, minimally invasive surgery. Journal of Surgical Education Volume ]/Number ] ] 2017
learning modality and those who perceived it to be most effective, nominal values of the responses were compared using McNemar tests. To evaluate surgeon's rationale for the discrepancy between their most commonly used learning modality and those deemed most effective, average values were calculated using a 1 to 5 Likert scale as indicated by surgeon's response (1 ¼ not at all important, 2 ¼ slightly important; 3 ¼ moderately important; 4 ¼ very important; and 5. ¼ extremely important). These Likert-like ordinal values were compared using a paired t-test (IBM SPSS Statistics v. 21.0; New York, NY).
RESULTS The survey response rate was 81% (122/150). A total 119/122 (98%) of surgeons reported they have learned a new procedure or technology since completing their formal training. The most-used learning modalities were scrubbing in on cases performed by another surgeon (78%), self-directed study (66%), performing cases under the supervision of a proctor/ mentor (56%), and didactic curriculum with simulation or cadaver laboratory (50%). Only 6% of surgeons indicated they participated in a mini-fellowship (Fig. 1). Surgeons were also asked to identify the single training method they used most commonly. Surgeons reported that this most commonly used training method was scrubbing in patient cases performed by another surgeon (34%) or selfdirected study (29%). Those considered to be most effective included performing cases under the supervision of a proctor/mentor (31%) and scrubbing in cases (29%). There were significant discrepancies between training methods commonly used and those perceived to be the most effective (Fig. 2). For example, 29% of surgeons most commonly used self-directed study, however only 7% felt this method was the most effective (p o 0.01). Similarly, for performing under proctor supervision, 15% of surgeons most commonly used the modality and 31% believed it to be most effective (p o 0.01). Very few surgeons (2%) most 3
100%
Percent of Respondents
80%
60%
40%
20%
0% Have not learned Self-directed study Didactic with Scrubbing in cases Performing under Participating in a any new procedures simulation/cadaver performed by proctor supervision mini-fellowship lab another surgeon
Learning Modality
FIGURE 1. Training methods used to learn a new surgical procedure or technology. Note: Question asked in "select all the apply" format to characterize usage of training methods.
commonly used mini-fellowships, yet 16% believed it to be most effective (p o 0.01). Respondents were asked to identify the importance of various factors to account for this discrepancy between the most common modality used and those perceived to be most effective. Surgeons indicated that the greatest deterrent from using more effective methods were that the modalities would “require too much time” (3.71) and “confidence that they could implement the new procedure/technology safely without the need for more rigorous/effective training” (3.56). Requiring too much time was a significantly greater factor than “lack of awareness of other opportunities” (3.15; p o 0.01) and “prohibitively expensive” (3.10; p o 0.01). Confidence that the surgeon could implement safely was significantly more important than “prohibitively expensive” (p ¼ 0.04), but not lack of awareness (p ¼ 0.12). Not needing additional training for credentialing/privileging requirements was less important for surgeons compared with the other factors (2.37, p o 0.01) (Fig. 3).
DISCUSSION Our survey results support that surgeons use a wide variety of training modalities when learning new procedures and technologies. The training methods commonly used include self-directed study, didactic learning, simulation laboratories, and scrubbing in cases performed by an expert. Paradoxically we found that practicing surgeons often do not use the learning modalities they deem to be most effective (e.g., performing cases under the supervision of a proctor and completion of a mini-fellowship). The most commonly cited factor contributing to this discrepancy was that more effective training modalities require too much time away from practice. Perhaps most importantly, surgeons' confidence in their own ability to safely implement the new procedure or technology without further training was a critically important finding for understanding why surgeons do not pursue the most-effective learning methods.
40% 35% 30% 25% 20% 15% 10% 5% 0% Self-directed study
Didactic with simulation/cadaver lab
Scrubbing in cases Performing under proctor Participating in a minisupervision fellowship performed by another surgeon
Most Commonly Used
Most Effective
FIGURE 2. Training method most commonly used and perceived to be most effective. Note: Question asked in "select one" format to assess common usage and impression of effectiveness. 4
Journal of Surgical Education Volume ]/Number ] ] 2017
5
Level of Importance
4
3
2
1
0 Program would require too much time
Confidence that I can implement safely
Lack of awareness of Prohibitively expensive available opportunities
Not needed to meet credentialing requirements
FIGURE 3. Rationale for discrepancy between commonly used method and most effective.
Prior work related to this topic has been focused on safety concerns associated with the learning and dissemination of new procedures and technologies. One such concern relates to the surgical learning curve, which suggests that surgical risks may be higher early on in a provider's cumulative experience.19-22 For example, Nallamothu et al.23 found that for patient's undergoing carotid stenting, early operator experience was associated with increased 30-day mortality (2.3%) compared with late experience (1.4%). Another study by Maruthappu et al.24 found a significant reduction in operative time associated with provider experience across multiple surgical specialties. Further research has examined the effectiveness of various learning modalities. There is evidence that more extensive training improves surgeons' perceptions of their capacity and increases future adoption.18,25-27 For example, investigations by Heniford et al.28, 29 found that 93% of surgeons who underwent preceptorship adopted laparoscopic splenectomy compared with only 6% of surgeons who participated in a 1-day training course. However, this body of work does not evaluate the safety of these procedures once surgeons have adopted them. Furthermore, there remains a paucity of literature characterizing which training modalities are most commonly used, and whether surgeons perceive these to be most effective. Our work expands on these previous findings by reporting the training modalities that surgeons use and surgeons′ perceptions of their comparative effectiveness. Although previous studies identified harms associated with inadequate training, our findings identify an important gap between the most commonly used methods for learning a new surgical procedure and the most effective methods. Furthermore, the scope of this study's findings are broad as we have documented surgical perception and training usage across multiple specialties. Our investigation also goes beyond previous work by reporting the barriers to usage of more effective methods and their perceived importance. Although
the benefits of comprehensive training modalities have been documented in recent literature, the constraints responsible for their low usage have not yet been addressed.30,31 This study's limitations deserve consideration. Respondents only included a subset of academic surgeons at 2 large Midwest hospitals. Our findings may vary with a more diverse sample of surgeons, including nonacademic settings and smaller hospitals. Nonetheless, understanding adoption at academic institutions remains important. Our findings are also subject to the limitations commonly associated with survey research including respondents understanding the limited number of available responses.32 As described in our methods, we conducted multiple interviews with surgeons of different experience and specialty while designing our survey to ensure that we captured all potential concerns. One such example is surgeon's definition of a “new procedure” and “new technology.” We found consistency among the responses of our interviewees that the term “new procedure” did not included minor variations, as most felt they make repeated, iterative changes to procedures throughout their careers. Another limitation is that responses may vary based on surgeon's experience with minimally invasive surgery. However, we specifically surveyed practicing surgeons which limits some variation, and furthermore, capturing beliefs of surgeons of different experience is beneficial in itself. Our survey research also does not directly evaluate outcomes, but instead examines surgeon perception. Now that we have cataloged the learning methods commonly used, however, there is an opportunity for further exploration to directly assess the effect that usage of these different methods has on surgical outcomes. It becomes increasingly important to monitor the safe diffusion of new innovations as the rate of technological advancement in surgical practice continues to rise.33,34 Our study suggests that nearly all practicing academic surgeons are learning new procedures or technologies after completing their
Journal of Surgical Education Volume ]/Number ] ] 2017
5
formal training. The efficacy of this training is affected by the complexity of the new innovation as well as surgeon-specific factors, including surgical experience and skill.35,36 Literature suggests that the introduction of novel technologies, including simulation and video review, can improve surgical skill and accelerate the learning curve.37-39 Usage of these strategies may enable more efficient training in practice and limit the time that surgeons are away from practice. Improving how practicing surgeons learn new procedures may also support hospitals in ensuring their safe introduction. The processes of confirming the qualifications of licensed physicians (credentialing) and authorizing them for specific patient care services (privileging) remain central to regulating how and when practicing surgeons may implement something new.40 New skill assessment modalities, including video review and teleproctoring, may help ensure safe diffusion of new technologies as these modes of direct observation have shown to be reliable surgical performance assessment tools.41-43 These platforms can provide surgeons with an objective review of their performance and an additional safeguard against premature adoption without requiring significant time away from surgical practice.
CONCLUSIONS Our study found that surgeons use a wide variety of methods when learning and implementing new procedures and technologies into practice. Surgeons also reported that the learning modalities they most commonly used were not necessarily those deemed the most effective. This discrepancy was largely owing to the perceived time commitment to more intensive training methods and surgeons′ confidence in implementing the new innovation safely on their own. The safety implications of surgeons using less effective training methods are vast, and the responsibility to regulate this process remains in flux. As surgical innovations continue to be introduced, effort must be directed to incentivizing safe and effective uptake in clinical practice.
REFERENCES 1. Ritchie WP, Rhodes RS, Biester TW. Work loads and
practice patterns of general surgeons in the United States, 1995–1997: a report from the American Board of Surgery. Ann Surg. 1999;230:533-542 [discussion 542–543].
6
3. Khosla A, Wagner AA. Robotic surgery of the kidney,
bladder, and prostate. Surg Clin North Am. 2016;96: 615-636. 4. Suda K, Nakauchi M, Inaba K, Ishida Y, Uyama I.
Robotic surgery for upper GI cancer: current status and future perspectives. Dig Endosc. 2016;28(7): 701-713. 5. Cosgrove DM. The innovation imperative. J Thorac
Cardiovasc Surg. 2000;120(5):839-842. 6. Gurgacz SL, Smith JA, Truskett PG, Babidge WJ,
Maddern GJ. Credentialing of surgeons: a systematic review across a number of jurisdictions. ANZ J Surg. 2012;82:492-498. 7. Jung do H, Son SY, Park YS, et al. The learning curve
associated with laparoscopic total gastrectomy. Gastric Cancer. 2016;19:264-272. 8. Arora KS, Khan N, Abboudi H, Khan MS, Dasgupta
P, Ahmed K. Learning curves for cardiothoracic and vascular surgical procedures—a systematic review. Postgrad Med. 2015;127:202-214. 9. Jeong O, Ryu SY, Choi WY, Piao Z, Park YK. Risk
factors and learning curve associated with postoperative morbidity of laparoscopic total gastrectomy for gastric carcinoma. Ann Surg Oncol. 2014;21: 2994-3001. 10. Abboudi H, Khan MS, Guru KA, et al. Learning
curves for urological procedures: a systematic review. BJU Int. 2014;114:617-629. 11. Education ACfCM. ACCME Annual Report. 2014. 12. Education AcfGM. ACGME Program Requirements
for Graduate Medical Education in General Surgery. 2014. 13. Rogers DA, Elstein AS, Bordage G. Improving con-
tinuing medical education for surgical techniques: applying the lessons learned in the first decade of minimal access surgery. Ann Surg. 2001;233:159-166. 14. Cook JA, Ramsay CR, Carr AJ, Rees JL. A question-
naire elicitation of surgeons' belief about learning within a surgical trial. PLoS One. 2012;7:e49178. 15. Willis GB. Cognitive Interviewing: A Tool for
Improving Questionnaire Design. Sage Publications; 2004.
2. Valentine RJ, Jones A, Biester TW, Cogbill TH,
16. DeMaio TJ, Landreth A. Do different cognitive inter-
Borman KR, Rhodes RS. General surgery workloads and practice patterns in the United States, 2007 to 2009: a 10-year update from the American Board of Surgery. Ann Surg. 2011;254:520-525 [discussion 525–526].
view techniques produce different results? In: Presser S, Rothgeb J, Couper M, Lessler J, Martin E, Singer S, editors. Methods for Testing and Evaluating Survey Questionnaires. New York: John Wiley & Sons, Inc, 2004. p. 89-108. Journal of Surgical Education Volume ]/Number ] ] 2017
17. Zorn KC, Gautam G, Shalhav AL, et al. Training,
30. Birch DW, Sample C, Gupta R. The impact of a
credentialing, proctoring and medicolegal risks of robotic urological surgery: recommendations of the society of urologic robotic surgeons. J Urol. 2009; 182:1126-1132.
comprehensive course in advanced minimal access surgery on surgeon practice. Can J Surg. 2007;50:9-12.
18. Cottam D, Holover S, Mattar SG, et al. The mini-
fellowship concept: a six-week focused training program for minimally invasive bariatric surgery. Surg Endosc. 2007;21:2237-2239. 19. Khan N, Abboudi H, Khan MS, Dasgupta P, Ahmed
31. Nakata BN, Cavalini W, Bonin EA, Salvalaggio PR,
Loureiro MP. Impact of continuous training through distributed practice for acquisition of minimally invasive surgical skills. Surg Endosc. 2017. 32. Kelley K, Clark B, Brown V, Sitzia J. Good practice in
the conduct and reporting of survey research. Int J Qual Health Care. 2003;15:261-266.
K. Measuring the surgical 'learning curve′: methods, variables and competency. BJU Int. 2014;113: 504-508.
33. Satava RM. Advanced technologies and the future of
20. Hopper AN, Jamison MH, Lewis WG. Learning
34. Satava R. Innovative technologies. The information
medicine and surgery. Yonsei Med J. 2008;49:873-878.
curves in surgical practice. Postgrad Med J. 2007;83: 777-779.
age and the biointelligence age. Surg Endosc. 2000;14: 417-418.
21. Boone BA, Zenati M, Hogg ME, et al. Assessment of
35. Ergina PL, Cook JA, Blazeby JM, et al. Challenges
quality outcomes for robotic pancreaticoduodenectomy: identification of the learning curve. J Am Med Assoc Surg. 2015;150:416-422.
36. Sheetz KH, Norton EC, Birkmeyer JD, Dimick JB.
22. Kretschmer A, Mandel P, Buchner A, Stief CG, Tilki
D. Surgical learning curve for open radical prostatectomy: is there an end to the learning curve? World J Urol. 2015;33:1721-1727. 23. Nallamothu BK, Gurm HS, Ting HH, et al. Operator
experience and carotid stenting outcomes in Medicare beneficiaries. J Am Med Assoc. 2011;306:1338-1343. 24. Maruthappu M, Duclos A, Lipsitz SR, Orgill D,
Carty MJ. Surgical learning curves and operative efficiency: a cross-specialty observational study. Br Med J Open. 2015;5:e006679. 25. Aggarwal R, Mytton OT, Derbrew M, et al. Training
and simulation for patient safety. Qual Saf Health Care. 2010;19(suppl 2)):i34-i43. 26. Brunckhorst O, et al. Training, simulation, the learn-
ing curve, and how to reduce complications in urology. Eur Urol Focus. 2016;2(1):10-18. 27. See WA, Cooper CS, Fisher RJ. Predictors of laparo-
scopic complications after formal training in laparoscopic surgery. J Am Med Assoc. 1993;270:2689-2692. 28. Heniford BT, Backus CL, Matthews BD, Greene FL,
Teel WB, Sing RF. Optimal teaching environment for laparoscopic splenectomy. Am J Surg. 2001;181: 226-230.
in evaluating surgical innovation. Lancet. 2009;374: 1097-1104. Provider Experience and the Comparative Safety of Laparoscopic and Open Colectomy. Health Serv Res. 2017;52:56-73. 37. Aggarwal R, Moorthy K, Darzi A. Laparoscopic
skills training and assessment. Br J Surg. 2004;91: 1549-1558. 38. Ibrahim AM, Varban OA, Dimick JB. Novel uses
of video to accelerate the surgical learning curve. J Laparoendosc Adv Surg Tech A. 2016;26:240-242. 39. Hasan A, Pozzi M, Hamilton JR. New surgical
procedures: can we minimise the learning curve? Br Med J. 2000;320:171-173. 40. Pradarelli JC, Campbell DA Jr, Dimick JB. Hospital
credentialing and privileging of surgeons: a potential safety blind spot. J Am Med Assoc. 2015;313: 1313-1314. 41. Schlachta CM, Nguyen NT, Ponsky T, Dunkin B.
Project 6 Summit: SAGES telementoring initiative. Surg Endosc. 2016;30:3665-3672. 42. Birkmeyer JD, Finks JF, O’Reilly A, et al. Surgical skill
and complication rates after bariatric surgery. N Engl J Med. 2013;369:1434-1442. 43. Scally CP, Varban OA, Carlin AM, Birkmeyer JD,
teaching environment for laparoscopic ventral herniorrhaphy. Hernia. 2002;6:17-20.
Dimick JB. Michigan Bariatric Surgery C. Video Ratings of Surgical Skill and Late Outcomes of Bariatric Surgery. J Am Med Assoc Surg. 2016;151: e160428.
Journal of Surgical Education Volume ]/Number ] ] 2017
7
29. Heniford BT, Matthews BD, Box EA, et al. Optimal