- Email: [email protected]
New Surgeon Outcomes and the Effectiveness of Surgical Training
New Surgeon Outcomes and the Effectiveness of Surgical Training A Population-Based Cohort Study Robert J. Campbell, MD, MSc,1,2,3 Sherif R. El-Defrawy, MD, PhD,4,5 Sudeep S. Gill, MD, MSc,3,6,7 Marlo Whitehead, MSc,3,8 Erica de L.P. Campbell, MD, MHA,1,2 Philip L. Hooper, MD,9,10 Chaim M. Bell, MD, PhD,11,12,13,14 Martin ten Hove, MD, MEng1,2 Purpose: Reports have questioned the technical proficiency of newly graduating surgeons. However, objective data supporting these concerns are limited. Surgical outcomes among recent graduates are an important indicator of residency programs’ ability to graduate surgeons who are ready to meet the needs of their patients. This study aimed to investigate the association between a surgeon’s number of years of independent practice and the risk of surgical adverse events. Design: Population-based cohort study. Participants: All patients 66 years of age or older undergoing isolated cataract operations in Ontario, Canada, between January 1, 1997, and December 31, 2013. Methods: Cataract surgical outcomes for all operations performed by surgeons commencing practice in the study period were evaluated using linked health care databases. Main Outcome Measures: Four serious complications were evaluated: posterior capsule rupture, dropped lens fragments, retinal detachment, and suspected endophthalmitis. Analyses controlled for patient-, surgeon-, and institution-level covariates. Results: The study evaluated 1 431 320 cataract operations. Surgeons in their first year of independent practice were more than 9 times more likely to have high complication rates (2%) than surgeons in their tenth year (odds ratio [OR], 9.3; 95% confidence interval [CI], 2.7e31.9). Each additional year of independent practice was associated with a 10% decrease in the risk of patients experiencing an adverse surgical event (OR, 0.90 per year of surgeon independent practice; 95% CI, 0.87e0.94). Conclusions: In this population-based study, surgical complications were significantly more likely early in surgeons’ careers. Interventions may be needed in postgraduate surgical training and early independent career monitoring and mentoring processes to ensure patient safety while continually renewing the surgical workforce. Ophthalmology 2017;-:1e7 ª 2016 by the American Academy of Ophthalmology Supplemental material is available at www.aaojournal.org.
Recent reports have questioned the technical proficiency of newly graduating surgeons.1e5 Many factors may contribute to gaps in surgical education, including training program financial constraints, pressure to improve hospital efficiency, shortened training periods, and limitations on trainee work hours.2,5e10 The need for trainees to acquire expertise in surgical fields with rapidly evolving technical requirements and increasing levels of subspecialization also may hamper efforts to ensure competency across the spectrum of care.2,11,12 However, although surveys of clinical department leaders, surgical training program directors, and trainees themselves have revealed a widespread perception that graduating surgeons may lack adequate preparation, large-scale, objective data supporting these concerns have been lacking.2,4 Surgical outcomes among early career surgeons are an important indicator of residency programs’ ability to ª 2016 by the American Academy of Ophthalmology Published by Elsevier Inc.
graduate surgeons who are ready to meet the needs of their patients. In particular, cataract surgical outcomes serve as an ideal model for numerous reasons. Cataract surgery is the most frequently performed operation in most developed countries, with more than 3 million procedures carried out in the United States annually.13e15 There is also a welldemonstrated learning curve in cataract surgery, making it a good model for other technical procedures facing parallel issues.16,17 Furthermore, cataract surgical outcomes are independent of the quality of postoperative in-patient hospital care, removing this potential confounding effect. Finally, measurable, nonmortality adverse cataract surgical event indicators have been established previously for use in both quality assurance programs and systems research.18e23 Hence, we conducted a population-based study to investigate the adequacy of surgeon training using cataract surgery
http://dx.doi.org/10.1016/j.ophtha.2016.12.012 ISSN 0161-6420/17
1
Ophthalmology Volume -, Number -, Month 2017 as an exemplar. Specifically, we investigated whether improvements in surgical adverse event rates occur during the initial stages of independent practice.
Methods Overview We conducted a population-based retrospective study to investigate the association between a surgeon’s number of years of independent practice and the risk of cataract surgical adverse events. We analyzed all isolated cataract operations performed by each surgeon who performed cataract surgery in Ontario, Canada, between January 1, 1997, and December 31, 2013. We linked several healthcare databases to study 4 serious complications of cataract surgery: posterior capsule rupture, dropped lens fragments, retinal detachment, and suspected endophthalmitis. Analyses controlled for patient-, surgeon-, and institution-level covariates. The study protocol was approved by the Queen’s University Health Sciences Research Ethics Board and adhered to the tenets of the Declaration of Helsinki. Patient confidentiality was maintained via encrypted health care identification numbers and strict adherence to privacy protocols.
Data Sources In Ontario, universal health care insurance covers the population, which averaged approximately 12 million over the study period. As such, data from the health care databases used in this study are population based and have been used in previous studies.20,21,24e26 Our study linked several health services databases. The Ontario Health Insurance Plan database contains information on inpatient and outpatient physician services and has excellent reliability in recording surgical procedures.27 The Ontario Drug Benefit database contains records of all outpatient prescriptions for formulary drugs dispensed to patients 65 years of age or older and has an error rate of less than 1%.28 The Canadian Institute for Health Information discharge abstract database provides reliable information on all hospital admissions in Ontario.29 The National Ambulatory Care Reporting System database collects comprehensive information on all visits to hospital outpatient surgical units and emergency departments.30,31 The Ontario Registered Persons Database collects demographic information on all insured persons in the province. The validated Ontario Diabetes Database identifies persons diagnosed with diabetes.32 The Institute for Clinical Evaluative Sciences Physician Database contains data on all physicians practicing in Ontario and incorporates information from the Ontario Health Insurance Plan database, the Ontario Health Insurance Plan Corporate Provider database, and the Ontario Physician Human Resources Data Centre database. Telephone interviews with physicians practicing in Ontario are used to validate the database, which has been used in previous studies of physician practice.24,33e35
Patient and Surgeon Inclusion Criteria All patients 66 years of age or older undergoing isolated cataract surgery in Ontario between January 1, 1997, and December 31, 2013, were identified. We limited the study to patients 66 years of age and older because the Ontario Drug Benefit database contains complete data for this age group, allowing for adjustments for potential drugrelated confounders. More than 80% of cataract operations are performed on patients within this age range.20 Cataract operations performed in combination with other procedures were excluded because the additional procedures have been shown to increase the risk of complications.18 Because previous ocular procedures reflect ocular comorbidity, patients who had undergone intraocular
2
surgery or retinal laser therapy in the preceding 5 years and patients who had received an intravitreal injection for retinal disease in the previous year also were excluded. We excluded patients undergoing general anesthetic or bilateral surgery on the same day to avoid confounding because these approaches generally are not used in routine cataract surgery and because patient ocular and systemic comorbidities that influence surgical outcomes may correlate with these approaches. Physicians were identified using their physician specialty code and unique de-identified billing numbers. All ophthalmologists who commenced practice within the study period were included in the analysis. Practice was defined as commencing on the earliest service date for an independently performed procedure, consultation, or office visit. As in previous studies, physicians performing fewer than 25 cataract operations in a calendar year were excluded from the analysis to avoid misclassification bias and because results for very low-volume surgeons may not be generalizable.21
Adverse Cataract Surgical Outcomes Four important complications were evaluated: posterior capsule rupture, dropped lens fragments, retinal detachment, and suspected endophthalmitis. The outcome definitions used were those published by the Ontario Ministry of Health and Long-Term Care as cataract surgery quality indicators in quality assurance processes.22 The definitions have been used in previous studies of cataract surgical outcomes, and details of database codes are provided in the Appendix (available at www.aaojournal.org).20,21 Briefly, posterior capsule rupture was defined as a patient requiring anterior vitrectomy on the same day as the cataract surgery. Dropped lens fragment was defined as a patient undergoing vitrectomy in combination with dislocated lens extraction between 1 and 14 days after cataract surgery. Retinal detachment was defined as a patient undergoing vitrectomy or scleral buckle in combination with airefluid exchange between 1 and 14 days after cataract surgery or a patient undergoing pneumatic retinopexy between 1 and 14 days after cataract surgery. Suspected endophthalmitis was defined as a patient undergoing vitrectomy associated with neither airefluid exchange nor dislocated lens extraction or a patient requiring an intravitreal injection procedure not performed as part of a pneumatic retinopexy between 1 and 14 days after cataract surgery. For patients with multiple complications, only the first was included.
Covariates We adjusted for several potential confounders that have been identified as risk factors for cataract surgical adverse events.16,18e20,36 Patient-level covariates included age on the day of cataract surgery, gender, socioeconomic status as defined by neighborhood income quintile, topical ophthalmic medication use in the preceding 90 days, topical glaucoma medication use in the preceding 90 days, tamsulosin use in the preceding year, antiviral use in the preceding year, and diabetes mellitus. The total number of medications used by each patient in the previous year also was included as a measure of overall comorbidity.21 Surgeon surgical volume, which has been shown to influence surgical outcomes, was included as a surgeonlevel covariate and was defined as the number of cataract operations performed by each surgeon in the calendar year.20 Institutions were classified as academic centers or community centers based on the Ontario Ministry of Health and Long-Term Care classification of hospitals and confirmed via direct communication with all surgical training programs in Ontario.
Statistical Analysis Data were evaluated using descriptive statistics to report and summarize physician-level and patient-level information. We
Campbell et al
Recent Graduate Cataract Surgeon Outcomes
Figure 1. Bar graph showing the effect of surgeon’s number of years of independent practice on overall adverse cataract surgical event rates. Stacked bars represent the distribution of complication rates among surgeons and display the change in this distribution with increasing years of independent practice. Numbers on the x-axis represent year of independent practice (top row) and number of surgeons in the study for each year of experience (bottom row).
performed both patient-level and surgeon-level analyses to address clinical and policy information needs. For the patient-level analyses, random effect logistic models were used to evaluate the association between physician years of independent practice and the risk of adverse events, controlling for patient- and surgeonlevel covariates and institution type. To control for secular trends, all models also adjusted for calendar year of surgery. Multilevel modeling is preferable to conventional regression analysis of health services data because it provides conservative, unbiased estimates that account for clustering of patients.37,38 A parsimonious model was fit for overall complications (a composite of the 4 individual complications examined), with diabetes and surgical institution type (academic vs. nonacademic) included in the model because of prior evidence demonstrating their association with cataract surgical adverse events.16e19,39 This model also was used for the individual complications. For the surgeon-level analyses, logistic regression was used to compare the risk of surgeons having high complication rates (defined in the primary analysis as 2% of cases with complications, and 3% and 4% in sensitivity analyses) at different time points in independent practice. All reported P values were 2-sided. Analyses were performed with SAS software version 9.4 (SAS Institute, Cary, NC).
Results From January 1997 through December 2013, 1 431 302 cataract operations meeting the study inclusion criteria were performed by surgeons in Ontario, Canada. Among these operations, 303 236 (21.2%) were performed in academic surgical centers. Patients included in the study had a median age of 77 years (interquartile
range, 72e81 years) and 26.1% had diabetes. Patients used a median of 9 prescription medications in the year before surgery (interquartile range, 6e13). Over the study period, 9932 (0.7%) cataract operations resulted in one of the adverse events evaluated. Specifically, 6588 (0.5%) patients experienced a posterior capsule rupture, 1648 (0.1%) had a dropped lens fragment, 294 (0.02%) experienced retinal detachment, and 1402 (0.1%) were identified as having suspected endophthalmitis.
Effect of Surgeon’s Number of Years of Independent Practice on Surgical Adverse Events Surgeon-Level Analyses. Figure 1 shows the effect of the number of years of independent practice on the distribution of overall adverse event rates among the 144 surgeons commencing practice during the study period. High complication rates were more common at earlier career stages and steadily became less common with increasing years of independent practice. Surgeons in their first year of independent practice were more than 9 times more likely to have high complication rates (2%) than surgeons in their tenth year (odds ratio [OR], 9.3; 95% confidence interval [CI], 2.7e31.9). Sensitivity analyses defining a high complication rate using higher cutoffs showed analogous results. Surgeon adverse event rates also were more variable at earlier career stages, with many new surgeons achieving very low complication rates and others demonstrating much higher rates. Patient-Level Analyses. Table 1 summarizes results of the random effects logistic regression analysis for overall adverse events. This analysis estimates the effect of years of independent
3
Ophthalmology Volume -, Number -, Month 2017 Table 1. Effect of Surgeon’s Number of Years of Independent Practice and Covariates on Overall Cataract Surgery Complication Risk*,y
Surgeon-level effects Years of independent practicez Surgical volume (cases/yr)x 25e250 251e500 501e1000 Patient-level effects Female gender Age (yrs)k 71e75 76e80 81 No. of unique drugs used{ Glaucoma medication use# Tamsulosin use{ Diabetes Nonacademic surgical center**
Odds Ratio
95% Confidence Interval
P Value
0.90
0.87e0.94
< 0.001
2.45 1.83 1.29
1.90e3.17 1.46e2.29 1.07e1.57
< 0.001 < 0.001 0.008
0.84
0.77e0.92
< 0.001
1.02 1.03 1.53 1.02 1.34 1.24 1.07 0.98
0.90e1.16 0.91e1.17 1.35e1.72 1.01e1.03 1.19e1.51 1.01e1.52 0.98e1.18 0.82e1.17
0.722 0.659 < 0.001 < 0.001 < 0.001 0.041 0.152 0.835
*Overall cataract surgery complications represent a composite of the 4 individual cataract surgical complications (posterior capsule rupture, dropped lens fragments, retinal detachment, and suspected endophthalmitis). y Adjusted for calendar year to control for secular trends. z Per additional year of independent practice. x Reference category: surgical volume >1000 cases/yr. k Reference category: age 66e70 yrs. { Within the year preceding surgery. # Within the 90 days preceding surgery. **Reference category: academic surgical center.
practice on the patient-level risk of cataract complications, after controlling for patient-level covariates, institution type, surgeon case volume, and calendar year. In this analysis, each additional year of surgeon independent practice was associated with a 10% decrease in the risk that patients would experience an adverse surgical event (OR, 0.90 per year of practice; 95% CI, 0.87e0.94; Table 1). Results of the random effects logistic regression models for each of the 4 individual adverse events are summarized in Tables S2eS5 (available at www.aaojournal.org). Among the individual complications, years of independent practice had the greatest effect on posterior capsule rupture, where each additional year of surgeon experience was associated with a 13% decrease in risk (Table S2, available at www.aaojournal.org). Years of independent practice also had a significant effect on the risk of dropped lens fragment, but did not predict the risk of suspected endophthalmitis or retinal detachment (Tables S3eS5, available at www.aaojournal.org).
Effect of Covariates on Surgical Adverse Events Higher surgical volume was associated with a decreased overall risk of complications. Older patient age, male gender, an increasing number of unique drugs used in the year preceding surgery, the use of glaucoma medications, and exposure to tamsulosin significantly
4
increased the overall risk of an adverse event (Table 1). For the individual adverse events, the surgical volume effect was most evident for posterior capsule rupture and dropped lens fragment (Tables S2 and S3, available at www.aaojournal.org). Additionally, posterior capsule rupture was associated with older patient age, male gender, the number of unique drugs used in the previous year, and the use of glaucoma medications (Table S2, available at www.aaojournal.org). Older patient age, male gender, and glaucoma medication use also increased the risk of suspected endophthalmitis (Table S4, available at www.aaojournal.org), whereas older patient age and the number of unique drugs used in the previous year were the only patientlevel variables associated with an increased risk of dropped lens fragment (Table S3, available at www.aaojournal.org). Male gender was the only patient level covariate associated with retinal detachment (Table S5, available at www.aaojournal.org). Of note, undergoing surgery at an academic center was not associated with the risk of complications (Table 1; Tables S2eS5, available at www.aaojournal.org).
Discussion In this population-based study, we found that surgical complications were significantly more likely early in surgeons’ careers. Surgeons in their first year of independent practice were more than 9 times more likely to have high complication rates than surgeons in their tenth year, and the risk of a patient experiencing an adverse event dropped by approximately 10% per year of surgeon independent practice. Although cataract surgery was used as a model, many surgical fields place similarly intense skill acquisition demands on trainees, and the outcomes we observed may reflect issues facing many technically challenging surgical fields.2,7,40 Our study has a number of strengths, including its focus on the initial years of independent practice, as well as its population-based approach, long study period, and large sample size. The evaluation of numerous clinically important outcomes; the inclusion of multiple relevant patient-, surgeon-, and institution-level covariates; and the ability to identify all cases for all surgeons were further strengths. The consistent and logical findings, wherein the outcomes most linked to technical skill demonstrated the greatest effect of years of independent practice, provides additional reassurance. There are also limitations to this study. First, the use of administrative data raises the potential for misclassification. However, the databases used have been validated and the outcomes evaluated have been used in previous research and form the foundation of the Ontario Ministry of Health and Long-Term Care cataract surgery quality evaluation process.20e22 Second, by including only the first adverse event for each patient, we likely have underestimated the absolute risk of some adverse events, particularly those that may occur subsequent to an initial complication. In particular, retinal detachment and endophthalmitis are more likely to occur after posterior capsule rupture. However, this would not be expected to affect our overall conclusions. Third, although many cases of suspected endophthalmitis would be expected to be diagnosed with bacterial endophthalmitis,
Campbell et al
Recent Graduate Cataract Surgeon Outcomes
some likely would not have been proven to have an underlying bacterial infection by culture results. Nevertheless, any severe postoperative inflammation necessitating the interventions defining suspected endophthalmitis would be a serious adverse event regardless of whether proven infectious. Fourth, our study analyzed aggregate data to allow for multilevel modeling. As a result, the study does not show individual surgeon learning curves; instead, the data can be interpreted as the risk faced by patients operated on by surgeons at different career stages. Fifth, although our study controlled for many patient- and surgeon-level covariates, not all potential covariates were included. As a result, our results could be affected by residual confounding if surgeons are able to limit their practice progressively and selectively to less-risky cases over the early years of their practice. Sixth, there is the potential that our findings are not generalizable to other jurisdictions. However, ophthalmologic surgical training in Canada is 1 year longer than that in the United States, and as a result, cataract surgical volumes of Canadian residents significantly exceed those of United States residents.4,41,42 Hence, the issue of recent graduate surgical outcomes is likely applicable in many countries, including the United States. Previous data on surgical outcomes at the outset of independent practice are limited and most previous studies have grouped surgeons into broad career stage intervals, precluding analysis of early career improvements. Among these studies, evaluations of carotid endarterectomy and coronary artery bypass grafting observed an association between poorer outcomes and later surgeon career stage.43,44 In contrast, studies of colorectal and thyroid cancer surgery as well as hernia repair found better outcomes among patients treated by surgeons at later career stages.45e47 A study of 15 cardiac surgeons found decreasing mortality rates over the first 4 years of practice.48 However, the use of mortality outcomes in many studies prevents clear interpretation because mortality reflects many aspects of care beyond the surgical procedure itself, such as postoperative intensive care. Implications for Achieving Surgical Competence and Excellence Surgical outcomes among recent graduates are a measure of residency programs’ ability consistently to graduate surgeons ready for the challenges of independent practice. Our findings are congruent with previous research showing that a substantial proportion of ophthalmology residents have difficulty acquiring surgical skills and that many surgeons commence independent practice despite concerns regarding their technical readiness.2,4,6,7 As a result, opportunities for educational innovation exist at many levels, including clinical teachers, training programs, and accreditation organizations.9,49 In particular, the uptake of competency-based surgical skills curricula could help establish training standards in many specialties.7,50 Furthermore, efforts to link surgical training program accreditation to educational outcomes also will support improvements in the preparation of graduating trainees.50e53 In the effort to ensure all graduating surgeons are
well prepared for independent practice, simulation curricula; expanded use of objective, valid assessment tools, such as Objective Structured Assessments of Technical Skills; and refinements in technology-based assessments also hold promise.1,6,49,54 Despite educational innovations, there may be a residual learning period necessarily faced by surgeons at the outset of independent practice. Hence, accreditation and credentialing bodies have highlighted the need to broaden the use of assessments not only during training, but also during the transition to independent practice.2 Additionally, formal mentorship programs for surgeons early in their careers could help to identify those in need of additional supports and to provide new surgeons with improved opportunities for success. The results of this study suggest that cataract surgical complications are significantly more likely early in a surgeon’s independent practice career. Ongoing innovations may be needed both in surgical training and in early career monitoring and mentoring processes to ensure patient safety while continually renewing the surgical workforce.
References 1. Hampton T. Efforts seek to develop systematic ways to objectively assess surgeons’ skills. JAMA. 2015;313(8): 782-784. 2. Sachdeva AK, Flynn TC, Brigham TP, et al. Interventions to address challenges associated with the transition from residency training to independent surgical practice. Surgery. 2014;155(5):867-882. 3. Bell Jr RH, Biester TW, Tabuenca A, et al. Operative experience of residents in US general surgery programs: a gap between expectation and experience. Ann Surg. 2009;249(5): 719-724. 4. Binenbaum G, Volpe NJ. Ophthalmology resident surgical competency: a national survey. Ophthalmology. 2006;113(7): 1237-1244. 5. Rodrigues IA, Symes RJ, Turner S, et al. Ophthalmic surgical training following modernising medical careers: regional variation in experience across the UK. BMJ Open. 2013;3:e002578. 6. Bell RH. Surgical council on resident education: a new organization devoted to graduate surgical education. J Am Coll Surg. 2007;204(3):341-346. 7. Sachdeva AK, Bell Jr RH, Britt LD, et al. National efforts to reform residency education in surgery. Acad Med. 2007;82(12):1200-1210. 8. Arrighi JA, Hebert JC. Duty hour requirements: time for a new approach? JAMA. 2014;312(22):2342-2344. 9. Tooke J. Postgraduate medical education and training in the UK. BMJ. 2013;347:f7604. 10. Simpson C, Cottam H, Fitzgerald JE, Giddings CE. The European working time directive has a negative impact on surgical training in the UK. Surg J R Coll Surg E. 2011;9(1): 56-57. 11. Au L, Saha K, Fernando B, et al. “Fast-track” cataract services and diagnostic and treatment centre: impact on surgical training. Eye. 2008;22(1):55-59. 12. Campbell RJ, Gill S, ten-Hove M, et al. Strabismus surgical subspecialization: a population-based analysis. JAMA Ophthalmol. 2015;133(5):555-559.
5
Ophthalmology Volume -, Number -, Month 2017 13. Cullen KA, Hall MJ, Golosinskiy A. Ambulatory surgery in the United States, 2006. National Health Statistics Reports, Number 11. Hyattsville, MD: National Center for Health Statistics; 2009. revised September 4, 2009 [cited July 6, 2016]. Available at: https://www.cdc.gov/nchs/data/nhsr/ nhsr011.pdf; Accessed January 4, 2017. 14. Hatch WV, Campbell EDP, Bell CM, et al. Projecting the growth of cataract surgery during the next 25 years. Arch Ophthalmol. 2012;130(11):1479-1481. 15. Lafortune G, Balestat G, Durand A. Comparing activities and performance of the hospital sector in Europe: how many surgical procedures performed as inpatient and day cases? Final Report on Work Package II: OECDdDirectorate for Employment, Labour and Social Affairs, December 2012 [cited July 6, 2016]. Available at: http://www.oecd.org/health/ Comparing-activities-and-performance-of-the-hospital-sectorin-Europe_Inpatient-and-day-cases-surgical-procedures.pdf; Accessed January 4, 2017. 16. Sparrow JM, Taylor H, Qureshi K, et al. The Cataract National Dataset electronic multi-centre audit of 55,567 operations: risk indicators for monocular visual acuity outcomes. Eye. 2012;26(6):821-826. 17. Randleman JB, Wolfe JD, Woodward M, et al. The resident surgeon phacoemulsification learning curve. Arch Ophthalmol. 2007;125(9):1215-1219. 18. Stein JD, Grossman DS, Mundy KM, et al. Severe adverse events after cataract surgery among medicare beneficiaries. Ophthalmology. 2011;118(9):1716-1723. 19. Stein JD. Serious adverse events after cataract surgery. Curr Opin Ophthalmol. 2012;23(3):219-225. 20. Bell CM, Hatch WV, Cernat G, Urbach DR. Surgeon volumes and selected patient outcomes in cataract surgery: a population-based analysis. Ophthalmology. 2007;114(3): 405-410. 21. Bell CM, Hatch WV, Fischer HD, et al. Association between tamsulosin and serious ophthalmic adverse events in older men following cataract surgery. JAMA. 2009;301(19):1991-1996. 22. Ontario Ministry of Health and Long-Term Care Cataract Surgery Quality-Based Procedure Expert Group. Quality indicators for cataract quality-based procedures. Toronto, Canada: October 2014 [cited July 6, 2016]. Available at: https://hsimi.on.ca/hdbportal/node/605; Accessed January 4, 2017. 23. Ontario Ministry of Health and Long-Term Care Cataract Surgery Quality-Based Procedure Expert Group. Qualitybased procedures clinical handbook for cataract day surgery. Toronto, Canada: 2013. updated September 4, 2015 [cited July 6, 2016]. Available at: http://www.health.gov.on.ca/en/pro/ programs/ecfa/docs/qbp_cataract.pdf; Accessed January 4, 2017. 24. Campbell RJ, Bell CM, Gill SS, et al. Subspecialization in glaucoma surgery. Ophthalmology. 2012;119(11):2270-2273. 25. Campbell RJ, Bell CM, Paterson JM, et al. Stroke rates after introduction of vascular endothelial growth factor inhibitors for macular degeneration: a time series analysis. Ophthalmology. 2012;119(8):1604-1608. 26. Campbell RJ, Gill SS, Bronskill SE, et al. Adverse events with intravitreal injection of vascular endothelial growth factor inhibitors: nested case-control study. BMJ. 2012;345: e4203. 27. Williams JI, Young W. A summary of studies on the quality of health care administration databases in Canada. In: Goel V, Williams JI, Anderson GM, et al., eds. Patterns of Health Care in Ontario: The ICES Practice Atlas. 2nd ed. Ottawa: Canadian Medical Association; 1996:339-345.
6
28. Levy AR, O’Brien BJ, Sellors C, et al. Coding accuracy of administrative drug claims in the Ontario Drug Benefit database. Can J Clin Pharmacol. 2003;10(2):67-71. 29. Juurlink DN, Preyra C, Croxford R, et al. Canadian Institute for Health Information Discharge Abstract Database: A Validation Study. ICES Investigative Report. Toronto: Institute for Clinical Evaluative Sciences; 2006 [cited July 6, 2016]. Available at: http://www.ices.on.ca/w/media/Files/AtlasesReports/2006/CIHI-DAD-a-validation-study/Full report.ashx; Accessed January 4, 2017. 30. Gruneir A, Bell CM, Bronskill SE, et al. Frequency and pattern of emergency department visits by long-term care residentsda population-based study. J Am Geriatr Soc. 2010;58(3): 510-517. 31. Gill SS, Anderson GM, Fischer HD, et al. Syncope and its consequences in patients with dementia receiving cholinesterase inhibitors: a population-based cohort study. Arch Intern Med. 2009;169(9):867-873. 32. Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3): 512-516. 33. Jain AK, McLeod I, Huo C, et al. When laboratories report estimated glomerular filtration rates in addition to serum creatinines, nephrology consults increase. Kidney Int. 2009;76(3): 318-323. 34. Grunfeld E, Hodgson DC, Del Giudice ME, Moineddin R. Population-based longitudinal study of follow-up care for breast cancer survivors. J Oncol Pract. 2010;6(4): 174-181. 35. Campbell RJ, Bell CM, Gill SS, et al. Clinic-based glaucoma care in the era of surgical subspecialization. Am J Ophthalmol. 2014;157(3):631-639.e2. 36. Freeman EE, Roy-Gagnon MH, Fortin E, et al. Rate of endophthalmitis after cataract surgery in Quebec, Canada, 1996e2005. Arch Ophthalmol. 2010;128(2):230-234. 37. Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2nd ed. Thousand Oaks, CA: Sage Publications Inc; 2002:3-37. 38. Urbach DR, Austin PC. Conventional models overestimate the statistical significance of volumeeoutcome associations, compared with multilevel models. J Clin Epidemiol. 2005;58(4):391-400. 39. Greenberg PB, Tseng VL, Wu WC, et al. Prevalence and predictors of ocular complications associated with cataract surgery in United States veterans. Ophthalmology. 2011;118(3):507-514. 40. Long DM. Competency-based residency training: the next advance in graduate medical education. Acad Med. 2000;75(12):1178-1183. 41. Zhou AW, Noble J, Lam WC. Canadian ophthalmology residency training: an evaluation of resident satisfaction and comparison with international standards. Can J Ophthalmol. 2009;44(5):540-547. 42. Rowden A, Krishna R. Resident cataract surgical training in United States residency programs. J Cataract Refract Surg. 2002;28(12):2202-2205. 43. O’Neill L, Lanska DJ, Hartz A. Surgeon characteristics associated with mortality and morbidity following carotid endarterectomy. Neurology. 2000;55(6):773-781. 44. Hartz AJ, Kuhn EM, Pulido J. Prestige of training programs and experience of bypass surgeons as factors in adjusted patient mortality rates. Medical Care. 1999;37(1):93-103. 45. Holm T, Johansson H, Cedermark B, et al. Influence of hospital- and surgeon-related factors on outcome after treatment of
Campbell et al
46.
47.
48.
49.
Recent Graduate Cataract Surgeon Outcomes
rectal cancer with or without preoperative radiotherapy. Br J Surg. 1997;84(5):657-663. Prystowsky JB, Bordage G, Feinglass JM. Patient outcomes for segmental colon resection according to surgeon’s training, certification, and experience. Surgery. 2002;132(4):663-670. discussion 70e72. Duclos A, Peix JL, Colin C, et al. Influence of experience on performance of individual surgeons in thyroid surgery: prospective cross sectional multicentre study. BMJ. 2012;344: d8041. Bridgewater B, Grayson AD, Au J, et al. Improving mortality of coronary surgery over first four years of independent practice: retrospective examination of prospectively collected data from 15 surgeons. BMJ. 2004;329(7463): 421. Reznick RK, MacRae H. Teaching surgical skillsdchanges in the wind. N Engl J Med. 2006;355(25):2664-2669.
50. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation systemdrationale and benefits. N Engl J Med. 2012;366(11):1051-1056. 51. Gardner AK, Scott DJ, Hebert JC, et al. Gearing up for milestones in surgery: will simulation play a role? Surgery. 2015;158(5):1421-1427. 52. Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach. 2007;29(7):642-647. 53. Lamont P, Ashworth M, Dalby S, et al. The Curriculum Framework for the Surgical Care Practitioner. London, UK: Royal College of Surgeons of England; 2014 [cited July 6, 2016]. Available at: http://accreditation.rcseng.ac.uk/pdf/SCP %20Curriculum%20Framework%202014.pdf; Accessed January 4, 2017. 54. Voelker R. Medical simulation gets real. JAMA. 2009;302(20): 2190-2192.
Footnotes and Financial Disclosures Originally received: September 7, 2016. Final revision: December 7, 2016. Accepted: December 8, 2016. Available online: ---.
Manuscript no. 2016-266.
1
Department of Ophthalmology, Queen’s University, Kingston, Canada. 2 Department of Ophthalmology, Hotel Dieu and Kingston General Hospitals, Kingston, Canada. 3
Institute for Clinical Evaluative Sciences, Kingston, Canada.
4
Department of Ophthalmology, University of Toronto, Toronto, Canada.
5
Department of Ophthalmology, Kensington Eye Institute, Toronto, Canada. 6 Division of Geriatric Medicine, Queen’s University, Kingston, Canada. 7
Division of Geriatric Medicine, St. Mary’s of the Lake Hospital, Kingston, Canada.
8
Department of Public Health Sciences, Queen’s University, Kingston, Canada. 9 Department of Ophthalmology, Western University, London, Canada. 10
Department of Ophthalmology, St. Joseph’s Hospital, London, Canada.
11
Department of Medicine, University of Toronto, Toronto, Canada.
12
Department of Health Policy Management and Evaluation, University of Toronto, Toronto, Canada.
13
Department of Medicine, Mount Sinai Hospital, Toronto, Canada.
14
the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit for publication. This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long Term Care (MOHLTC). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. In accordance with the Personal Health Information Protection Act (PHIPA) of Ontario, the raw administrative data used for statistical analyses in this manuscript may only be accessed by agents of the Institute for Clinical Evaluative Sciences (ICES), a prescribed entity under Section 45 of the Act, for the purposes of conducting research that contributes to the effectiveness, quality, equity, and efficiency of health care and health services. Dr. R. J. Campbell is an agent of ICES and had full control over the data definitions and analyses used in this manuscript. Dr. R. J. Campbell had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr. Campbell can be contacted regarding supplemental protocol details. Author Contributions: Conception and design: R. Campbell, Gill, Whitehead, E. Campbell, Bell Analysis and interpretation: R. Campbell, El-Defrawy, Gill, Whitehead, E. Campbell, Hooper, Bell, ten Hove Data collection: R. Campbell, Whitehead, E. Campbell
Institute for Clinical Evaluative Sciences, Toronto, Canada. Financial Disclosure(s): The author(s) have no proprietary or commercial interest in any materials discussed in this article.
Obtained funding: R. Campbell
R.J.C.: Supported by a Research Scientist Award Queen’s University and the Southeastern Ontario Academic Medical Organization. S.S.G.: Supported by a Canadian Institutes of Health Research New Investigator Award Institute of Aging.
Abbreviations and Acronyms: CI ¼ confidence interval; OR ¼ odds ratio.
C.M.B.: Supported by a Canadian Institutes of Health Research and Canadian Patient Safety Institute chair in Patient Safety and Continuity of Care. The sponsors of this study had no role in the design and conduct of
Overall responsibility: R. Campbell, El-Defrawy, Gill, Whitehead, E. Campbell, Hooper, Bell, ten Hove
Correspondence: Robert J. Campbell, MD, MSc, Department of Ophthalmology, Hotel Dieu Hospital, 166 Brock Street, Kingston, Ontario K7L 5G2, Canada. E-mail: [email protected].
7
Recent reports have questioned the technical proficiency of newly graduating surgeons.1e5 Many factors may contribute to gaps in surgical education, including training program financial constraints, pressure to improve hospital efficiency, shortened training periods, and limitations on trainee work hours.2,5e10 The need for trainees to acquire expertise in surgical fields with rapidly evolving technical requirements and increasing levels of subspecialization also may hamper efforts to ensure competency across the spectrum of care.2,11,12 However, although surveys of clinical department leaders, surgical training program directors, and trainees themselves have revealed a widespread perception that graduating surgeons may lack adequate preparation, large-scale, objective data supporting these concerns have been lacking.2,4 Surgical outcomes among early career surgeons are an important indicator of residency programs’ ability to ª 2016 by the American Academy of Ophthalmology Published by Elsevier Inc.
graduate surgeons who are ready to meet the needs of their patients. In particular, cataract surgical outcomes serve as an ideal model for numerous reasons. Cataract surgery is the most frequently performed operation in most developed countries, with more than 3 million procedures carried out in the United States annually.13e15 There is also a welldemonstrated learning curve in cataract surgery, making it a good model for other technical procedures facing parallel issues.16,17 Furthermore, cataract surgical outcomes are independent of the quality of postoperative in-patient hospital care, removing this potential confounding effect. Finally, measurable, nonmortality adverse cataract surgical event indicators have been established previously for use in both quality assurance programs and systems research.18e23 Hence, we conducted a population-based study to investigate the adequacy of surgeon training using cataract surgery
http://dx.doi.org/10.1016/j.ophtha.2016.12.012 ISSN 0161-6420/17
1
Ophthalmology Volume -, Number -, Month 2017 as an exemplar. Specifically, we investigated whether improvements in surgical adverse event rates occur during the initial stages of independent practice.
Methods Overview We conducted a population-based retrospective study to investigate the association between a surgeon’s number of years of independent practice and the risk of cataract surgical adverse events. We analyzed all isolated cataract operations performed by each surgeon who performed cataract surgery in Ontario, Canada, between January 1, 1997, and December 31, 2013. We linked several healthcare databases to study 4 serious complications of cataract surgery: posterior capsule rupture, dropped lens fragments, retinal detachment, and suspected endophthalmitis. Analyses controlled for patient-, surgeon-, and institution-level covariates. The study protocol was approved by the Queen’s University Health Sciences Research Ethics Board and adhered to the tenets of the Declaration of Helsinki. Patient confidentiality was maintained via encrypted health care identification numbers and strict adherence to privacy protocols.
Data Sources In Ontario, universal health care insurance covers the population, which averaged approximately 12 million over the study period. As such, data from the health care databases used in this study are population based and have been used in previous studies.20,21,24e26 Our study linked several health services databases. The Ontario Health Insurance Plan database contains information on inpatient and outpatient physician services and has excellent reliability in recording surgical procedures.27 The Ontario Drug Benefit database contains records of all outpatient prescriptions for formulary drugs dispensed to patients 65 years of age or older and has an error rate of less than 1%.28 The Canadian Institute for Health Information discharge abstract database provides reliable information on all hospital admissions in Ontario.29 The National Ambulatory Care Reporting System database collects comprehensive information on all visits to hospital outpatient surgical units and emergency departments.30,31 The Ontario Registered Persons Database collects demographic information on all insured persons in the province. The validated Ontario Diabetes Database identifies persons diagnosed with diabetes.32 The Institute for Clinical Evaluative Sciences Physician Database contains data on all physicians practicing in Ontario and incorporates information from the Ontario Health Insurance Plan database, the Ontario Health Insurance Plan Corporate Provider database, and the Ontario Physician Human Resources Data Centre database. Telephone interviews with physicians practicing in Ontario are used to validate the database, which has been used in previous studies of physician practice.24,33e35
Patient and Surgeon Inclusion Criteria All patients 66 years of age or older undergoing isolated cataract surgery in Ontario between January 1, 1997, and December 31, 2013, were identified. We limited the study to patients 66 years of age and older because the Ontario Drug Benefit database contains complete data for this age group, allowing for adjustments for potential drugrelated confounders. More than 80% of cataract operations are performed on patients within this age range.20 Cataract operations performed in combination with other procedures were excluded because the additional procedures have been shown to increase the risk of complications.18 Because previous ocular procedures reflect ocular comorbidity, patients who had undergone intraocular
2
surgery or retinal laser therapy in the preceding 5 years and patients who had received an intravitreal injection for retinal disease in the previous year also were excluded. We excluded patients undergoing general anesthetic or bilateral surgery on the same day to avoid confounding because these approaches generally are not used in routine cataract surgery and because patient ocular and systemic comorbidities that influence surgical outcomes may correlate with these approaches. Physicians were identified using their physician specialty code and unique de-identified billing numbers. All ophthalmologists who commenced practice within the study period were included in the analysis. Practice was defined as commencing on the earliest service date for an independently performed procedure, consultation, or office visit. As in previous studies, physicians performing fewer than 25 cataract operations in a calendar year were excluded from the analysis to avoid misclassification bias and because results for very low-volume surgeons may not be generalizable.21
Adverse Cataract Surgical Outcomes Four important complications were evaluated: posterior capsule rupture, dropped lens fragments, retinal detachment, and suspected endophthalmitis. The outcome definitions used were those published by the Ontario Ministry of Health and Long-Term Care as cataract surgery quality indicators in quality assurance processes.22 The definitions have been used in previous studies of cataract surgical outcomes, and details of database codes are provided in the Appendix (available at www.aaojournal.org).20,21 Briefly, posterior capsule rupture was defined as a patient requiring anterior vitrectomy on the same day as the cataract surgery. Dropped lens fragment was defined as a patient undergoing vitrectomy in combination with dislocated lens extraction between 1 and 14 days after cataract surgery. Retinal detachment was defined as a patient undergoing vitrectomy or scleral buckle in combination with airefluid exchange between 1 and 14 days after cataract surgery or a patient undergoing pneumatic retinopexy between 1 and 14 days after cataract surgery. Suspected endophthalmitis was defined as a patient undergoing vitrectomy associated with neither airefluid exchange nor dislocated lens extraction or a patient requiring an intravitreal injection procedure not performed as part of a pneumatic retinopexy between 1 and 14 days after cataract surgery. For patients with multiple complications, only the first was included.
Covariates We adjusted for several potential confounders that have been identified as risk factors for cataract surgical adverse events.16,18e20,36 Patient-level covariates included age on the day of cataract surgery, gender, socioeconomic status as defined by neighborhood income quintile, topical ophthalmic medication use in the preceding 90 days, topical glaucoma medication use in the preceding 90 days, tamsulosin use in the preceding year, antiviral use in the preceding year, and diabetes mellitus. The total number of medications used by each patient in the previous year also was included as a measure of overall comorbidity.21 Surgeon surgical volume, which has been shown to influence surgical outcomes, was included as a surgeonlevel covariate and was defined as the number of cataract operations performed by each surgeon in the calendar year.20 Institutions were classified as academic centers or community centers based on the Ontario Ministry of Health and Long-Term Care classification of hospitals and confirmed via direct communication with all surgical training programs in Ontario.
Statistical Analysis Data were evaluated using descriptive statistics to report and summarize physician-level and patient-level information. We
Campbell et al
Recent Graduate Cataract Surgeon Outcomes
Figure 1. Bar graph showing the effect of surgeon’s number of years of independent practice on overall adverse cataract surgical event rates. Stacked bars represent the distribution of complication rates among surgeons and display the change in this distribution with increasing years of independent practice. Numbers on the x-axis represent year of independent practice (top row) and number of surgeons in the study for each year of experience (bottom row).
performed both patient-level and surgeon-level analyses to address clinical and policy information needs. For the patient-level analyses, random effect logistic models were used to evaluate the association between physician years of independent practice and the risk of adverse events, controlling for patient- and surgeonlevel covariates and institution type. To control for secular trends, all models also adjusted for calendar year of surgery. Multilevel modeling is preferable to conventional regression analysis of health services data because it provides conservative, unbiased estimates that account for clustering of patients.37,38 A parsimonious model was fit for overall complications (a composite of the 4 individual complications examined), with diabetes and surgical institution type (academic vs. nonacademic) included in the model because of prior evidence demonstrating their association with cataract surgical adverse events.16e19,39 This model also was used for the individual complications. For the surgeon-level analyses, logistic regression was used to compare the risk of surgeons having high complication rates (defined in the primary analysis as 2% of cases with complications, and 3% and 4% in sensitivity analyses) at different time points in independent practice. All reported P values were 2-sided. Analyses were performed with SAS software version 9.4 (SAS Institute, Cary, NC).
Results From January 1997 through December 2013, 1 431 302 cataract operations meeting the study inclusion criteria were performed by surgeons in Ontario, Canada. Among these operations, 303 236 (21.2%) were performed in academic surgical centers. Patients included in the study had a median age of 77 years (interquartile
range, 72e81 years) and 26.1% had diabetes. Patients used a median of 9 prescription medications in the year before surgery (interquartile range, 6e13). Over the study period, 9932 (0.7%) cataract operations resulted in one of the adverse events evaluated. Specifically, 6588 (0.5%) patients experienced a posterior capsule rupture, 1648 (0.1%) had a dropped lens fragment, 294 (0.02%) experienced retinal detachment, and 1402 (0.1%) were identified as having suspected endophthalmitis.
Effect of Surgeon’s Number of Years of Independent Practice on Surgical Adverse Events Surgeon-Level Analyses. Figure 1 shows the effect of the number of years of independent practice on the distribution of overall adverse event rates among the 144 surgeons commencing practice during the study period. High complication rates were more common at earlier career stages and steadily became less common with increasing years of independent practice. Surgeons in their first year of independent practice were more than 9 times more likely to have high complication rates (2%) than surgeons in their tenth year (odds ratio [OR], 9.3; 95% confidence interval [CI], 2.7e31.9). Sensitivity analyses defining a high complication rate using higher cutoffs showed analogous results. Surgeon adverse event rates also were more variable at earlier career stages, with many new surgeons achieving very low complication rates and others demonstrating much higher rates. Patient-Level Analyses. Table 1 summarizes results of the random effects logistic regression analysis for overall adverse events. This analysis estimates the effect of years of independent
3
Ophthalmology Volume -, Number -, Month 2017 Table 1. Effect of Surgeon’s Number of Years of Independent Practice and Covariates on Overall Cataract Surgery Complication Risk*,y
Surgeon-level effects Years of independent practicez Surgical volume (cases/yr)x 25e250 251e500 501e1000 Patient-level effects Female gender Age (yrs)k 71e75 76e80 81 No. of unique drugs used{ Glaucoma medication use# Tamsulosin use{ Diabetes Nonacademic surgical center**
Odds Ratio
95% Confidence Interval
P Value
0.90
0.87e0.94
< 0.001
2.45 1.83 1.29
1.90e3.17 1.46e2.29 1.07e1.57
< 0.001 < 0.001 0.008
0.84
0.77e0.92
< 0.001
1.02 1.03 1.53 1.02 1.34 1.24 1.07 0.98
0.90e1.16 0.91e1.17 1.35e1.72 1.01e1.03 1.19e1.51 1.01e1.52 0.98e1.18 0.82e1.17
0.722 0.659 < 0.001 < 0.001 < 0.001 0.041 0.152 0.835
*Overall cataract surgery complications represent a composite of the 4 individual cataract surgical complications (posterior capsule rupture, dropped lens fragments, retinal detachment, and suspected endophthalmitis). y Adjusted for calendar year to control for secular trends. z Per additional year of independent practice. x Reference category: surgical volume >1000 cases/yr. k Reference category: age 66e70 yrs. { Within the year preceding surgery. # Within the 90 days preceding surgery. **Reference category: academic surgical center.
practice on the patient-level risk of cataract complications, after controlling for patient-level covariates, institution type, surgeon case volume, and calendar year. In this analysis, each additional year of surgeon independent practice was associated with a 10% decrease in the risk that patients would experience an adverse surgical event (OR, 0.90 per year of practice; 95% CI, 0.87e0.94; Table 1). Results of the random effects logistic regression models for each of the 4 individual adverse events are summarized in Tables S2eS5 (available at www.aaojournal.org). Among the individual complications, years of independent practice had the greatest effect on posterior capsule rupture, where each additional year of surgeon experience was associated with a 13% decrease in risk (Table S2, available at www.aaojournal.org). Years of independent practice also had a significant effect on the risk of dropped lens fragment, but did not predict the risk of suspected endophthalmitis or retinal detachment (Tables S3eS5, available at www.aaojournal.org).
Effect of Covariates on Surgical Adverse Events Higher surgical volume was associated with a decreased overall risk of complications. Older patient age, male gender, an increasing number of unique drugs used in the year preceding surgery, the use of glaucoma medications, and exposure to tamsulosin significantly
4
increased the overall risk of an adverse event (Table 1). For the individual adverse events, the surgical volume effect was most evident for posterior capsule rupture and dropped lens fragment (Tables S2 and S3, available at www.aaojournal.org). Additionally, posterior capsule rupture was associated with older patient age, male gender, the number of unique drugs used in the previous year, and the use of glaucoma medications (Table S2, available at www.aaojournal.org). Older patient age, male gender, and glaucoma medication use also increased the risk of suspected endophthalmitis (Table S4, available at www.aaojournal.org), whereas older patient age and the number of unique drugs used in the previous year were the only patientlevel variables associated with an increased risk of dropped lens fragment (Table S3, available at www.aaojournal.org). Male gender was the only patient level covariate associated with retinal detachment (Table S5, available at www.aaojournal.org). Of note, undergoing surgery at an academic center was not associated with the risk of complications (Table 1; Tables S2eS5, available at www.aaojournal.org).
Discussion In this population-based study, we found that surgical complications were significantly more likely early in surgeons’ careers. Surgeons in their first year of independent practice were more than 9 times more likely to have high complication rates than surgeons in their tenth year, and the risk of a patient experiencing an adverse event dropped by approximately 10% per year of surgeon independent practice. Although cataract surgery was used as a model, many surgical fields place similarly intense skill acquisition demands on trainees, and the outcomes we observed may reflect issues facing many technically challenging surgical fields.2,7,40 Our study has a number of strengths, including its focus on the initial years of independent practice, as well as its population-based approach, long study period, and large sample size. The evaluation of numerous clinically important outcomes; the inclusion of multiple relevant patient-, surgeon-, and institution-level covariates; and the ability to identify all cases for all surgeons were further strengths. The consistent and logical findings, wherein the outcomes most linked to technical skill demonstrated the greatest effect of years of independent practice, provides additional reassurance. There are also limitations to this study. First, the use of administrative data raises the potential for misclassification. However, the databases used have been validated and the outcomes evaluated have been used in previous research and form the foundation of the Ontario Ministry of Health and Long-Term Care cataract surgery quality evaluation process.20e22 Second, by including only the first adverse event for each patient, we likely have underestimated the absolute risk of some adverse events, particularly those that may occur subsequent to an initial complication. In particular, retinal detachment and endophthalmitis are more likely to occur after posterior capsule rupture. However, this would not be expected to affect our overall conclusions. Third, although many cases of suspected endophthalmitis would be expected to be diagnosed with bacterial endophthalmitis,
Campbell et al
Recent Graduate Cataract Surgeon Outcomes
some likely would not have been proven to have an underlying bacterial infection by culture results. Nevertheless, any severe postoperative inflammation necessitating the interventions defining suspected endophthalmitis would be a serious adverse event regardless of whether proven infectious. Fourth, our study analyzed aggregate data to allow for multilevel modeling. As a result, the study does not show individual surgeon learning curves; instead, the data can be interpreted as the risk faced by patients operated on by surgeons at different career stages. Fifth, although our study controlled for many patient- and surgeon-level covariates, not all potential covariates were included. As a result, our results could be affected by residual confounding if surgeons are able to limit their practice progressively and selectively to less-risky cases over the early years of their practice. Sixth, there is the potential that our findings are not generalizable to other jurisdictions. However, ophthalmologic surgical training in Canada is 1 year longer than that in the United States, and as a result, cataract surgical volumes of Canadian residents significantly exceed those of United States residents.4,41,42 Hence, the issue of recent graduate surgical outcomes is likely applicable in many countries, including the United States. Previous data on surgical outcomes at the outset of independent practice are limited and most previous studies have grouped surgeons into broad career stage intervals, precluding analysis of early career improvements. Among these studies, evaluations of carotid endarterectomy and coronary artery bypass grafting observed an association between poorer outcomes and later surgeon career stage.43,44 In contrast, studies of colorectal and thyroid cancer surgery as well as hernia repair found better outcomes among patients treated by surgeons at later career stages.45e47 A study of 15 cardiac surgeons found decreasing mortality rates over the first 4 years of practice.48 However, the use of mortality outcomes in many studies prevents clear interpretation because mortality reflects many aspects of care beyond the surgical procedure itself, such as postoperative intensive care. Implications for Achieving Surgical Competence and Excellence Surgical outcomes among recent graduates are a measure of residency programs’ ability consistently to graduate surgeons ready for the challenges of independent practice. Our findings are congruent with previous research showing that a substantial proportion of ophthalmology residents have difficulty acquiring surgical skills and that many surgeons commence independent practice despite concerns regarding their technical readiness.2,4,6,7 As a result, opportunities for educational innovation exist at many levels, including clinical teachers, training programs, and accreditation organizations.9,49 In particular, the uptake of competency-based surgical skills curricula could help establish training standards in many specialties.7,50 Furthermore, efforts to link surgical training program accreditation to educational outcomes also will support improvements in the preparation of graduating trainees.50e53 In the effort to ensure all graduating surgeons are
well prepared for independent practice, simulation curricula; expanded use of objective, valid assessment tools, such as Objective Structured Assessments of Technical Skills; and refinements in technology-based assessments also hold promise.1,6,49,54 Despite educational innovations, there may be a residual learning period necessarily faced by surgeons at the outset of independent practice. Hence, accreditation and credentialing bodies have highlighted the need to broaden the use of assessments not only during training, but also during the transition to independent practice.2 Additionally, formal mentorship programs for surgeons early in their careers could help to identify those in need of additional supports and to provide new surgeons with improved opportunities for success. The results of this study suggest that cataract surgical complications are significantly more likely early in a surgeon’s independent practice career. Ongoing innovations may be needed both in surgical training and in early career monitoring and mentoring processes to ensure patient safety while continually renewing the surgical workforce.
References 1. Hampton T. Efforts seek to develop systematic ways to objectively assess surgeons’ skills. JAMA. 2015;313(8): 782-784. 2. Sachdeva AK, Flynn TC, Brigham TP, et al. Interventions to address challenges associated with the transition from residency training to independent surgical practice. Surgery. 2014;155(5):867-882. 3. Bell Jr RH, Biester TW, Tabuenca A, et al. Operative experience of residents in US general surgery programs: a gap between expectation and experience. Ann Surg. 2009;249(5): 719-724. 4. Binenbaum G, Volpe NJ. Ophthalmology resident surgical competency: a national survey. Ophthalmology. 2006;113(7): 1237-1244. 5. Rodrigues IA, Symes RJ, Turner S, et al. Ophthalmic surgical training following modernising medical careers: regional variation in experience across the UK. BMJ Open. 2013;3:e002578. 6. Bell RH. Surgical council on resident education: a new organization devoted to graduate surgical education. J Am Coll Surg. 2007;204(3):341-346. 7. Sachdeva AK, Bell Jr RH, Britt LD, et al. National efforts to reform residency education in surgery. Acad Med. 2007;82(12):1200-1210. 8. Arrighi JA, Hebert JC. Duty hour requirements: time for a new approach? JAMA. 2014;312(22):2342-2344. 9. Tooke J. Postgraduate medical education and training in the UK. BMJ. 2013;347:f7604. 10. Simpson C, Cottam H, Fitzgerald JE, Giddings CE. The European working time directive has a negative impact on surgical training in the UK. Surg J R Coll Surg E. 2011;9(1): 56-57. 11. Au L, Saha K, Fernando B, et al. “Fast-track” cataract services and diagnostic and treatment centre: impact on surgical training. Eye. 2008;22(1):55-59. 12. Campbell RJ, Gill S, ten-Hove M, et al. Strabismus surgical subspecialization: a population-based analysis. JAMA Ophthalmol. 2015;133(5):555-559.
5
Ophthalmology Volume -, Number -, Month 2017 13. Cullen KA, Hall MJ, Golosinskiy A. Ambulatory surgery in the United States, 2006. National Health Statistics Reports, Number 11. Hyattsville, MD: National Center for Health Statistics; 2009. revised September 4, 2009 [cited July 6, 2016]. Available at: https://www.cdc.gov/nchs/data/nhsr/ nhsr011.pdf; Accessed January 4, 2017. 14. Hatch WV, Campbell EDP, Bell CM, et al. Projecting the growth of cataract surgery during the next 25 years. Arch Ophthalmol. 2012;130(11):1479-1481. 15. Lafortune G, Balestat G, Durand A. Comparing activities and performance of the hospital sector in Europe: how many surgical procedures performed as inpatient and day cases? Final Report on Work Package II: OECDdDirectorate for Employment, Labour and Social Affairs, December 2012 [cited July 6, 2016]. Available at: http://www.oecd.org/health/ Comparing-activities-and-performance-of-the-hospital-sectorin-Europe_Inpatient-and-day-cases-surgical-procedures.pdf; Accessed January 4, 2017. 16. Sparrow JM, Taylor H, Qureshi K, et al. The Cataract National Dataset electronic multi-centre audit of 55,567 operations: risk indicators for monocular visual acuity outcomes. Eye. 2012;26(6):821-826. 17. Randleman JB, Wolfe JD, Woodward M, et al. The resident surgeon phacoemulsification learning curve. Arch Ophthalmol. 2007;125(9):1215-1219. 18. Stein JD, Grossman DS, Mundy KM, et al. Severe adverse events after cataract surgery among medicare beneficiaries. Ophthalmology. 2011;118(9):1716-1723. 19. Stein JD. Serious adverse events after cataract surgery. Curr Opin Ophthalmol. 2012;23(3):219-225. 20. Bell CM, Hatch WV, Cernat G, Urbach DR. Surgeon volumes and selected patient outcomes in cataract surgery: a population-based analysis. Ophthalmology. 2007;114(3): 405-410. 21. Bell CM, Hatch WV, Fischer HD, et al. Association between tamsulosin and serious ophthalmic adverse events in older men following cataract surgery. JAMA. 2009;301(19):1991-1996. 22. Ontario Ministry of Health and Long-Term Care Cataract Surgery Quality-Based Procedure Expert Group. Quality indicators for cataract quality-based procedures. Toronto, Canada: October 2014 [cited July 6, 2016]. Available at: https://hsimi.on.ca/hdbportal/node/605; Accessed January 4, 2017. 23. Ontario Ministry of Health and Long-Term Care Cataract Surgery Quality-Based Procedure Expert Group. Qualitybased procedures clinical handbook for cataract day surgery. Toronto, Canada: 2013. updated September 4, 2015 [cited July 6, 2016]. Available at: http://www.health.gov.on.ca/en/pro/ programs/ecfa/docs/qbp_cataract.pdf; Accessed January 4, 2017. 24. Campbell RJ, Bell CM, Gill SS, et al. Subspecialization in glaucoma surgery. Ophthalmology. 2012;119(11):2270-2273. 25. Campbell RJ, Bell CM, Paterson JM, et al. Stroke rates after introduction of vascular endothelial growth factor inhibitors for macular degeneration: a time series analysis. Ophthalmology. 2012;119(8):1604-1608. 26. Campbell RJ, Gill SS, Bronskill SE, et al. Adverse events with intravitreal injection of vascular endothelial growth factor inhibitors: nested case-control study. BMJ. 2012;345: e4203. 27. Williams JI, Young W. A summary of studies on the quality of health care administration databases in Canada. In: Goel V, Williams JI, Anderson GM, et al., eds. Patterns of Health Care in Ontario: The ICES Practice Atlas. 2nd ed. Ottawa: Canadian Medical Association; 1996:339-345.
6
28. Levy AR, O’Brien BJ, Sellors C, et al. Coding accuracy of administrative drug claims in the Ontario Drug Benefit database. Can J Clin Pharmacol. 2003;10(2):67-71. 29. Juurlink DN, Preyra C, Croxford R, et al. Canadian Institute for Health Information Discharge Abstract Database: A Validation Study. ICES Investigative Report. Toronto: Institute for Clinical Evaluative Sciences; 2006 [cited July 6, 2016]. Available at: http://www.ices.on.ca/w/media/Files/AtlasesReports/2006/CIHI-DAD-a-validation-study/Full report.ashx; Accessed January 4, 2017. 30. Gruneir A, Bell CM, Bronskill SE, et al. Frequency and pattern of emergency department visits by long-term care residentsda population-based study. J Am Geriatr Soc. 2010;58(3): 510-517. 31. Gill SS, Anderson GM, Fischer HD, et al. Syncope and its consequences in patients with dementia receiving cholinesterase inhibitors: a population-based cohort study. Arch Intern Med. 2009;169(9):867-873. 32. Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3): 512-516. 33. Jain AK, McLeod I, Huo C, et al. When laboratories report estimated glomerular filtration rates in addition to serum creatinines, nephrology consults increase. Kidney Int. 2009;76(3): 318-323. 34. Grunfeld E, Hodgson DC, Del Giudice ME, Moineddin R. Population-based longitudinal study of follow-up care for breast cancer survivors. J Oncol Pract. 2010;6(4): 174-181. 35. Campbell RJ, Bell CM, Gill SS, et al. Clinic-based glaucoma care in the era of surgical subspecialization. Am J Ophthalmol. 2014;157(3):631-639.e2. 36. Freeman EE, Roy-Gagnon MH, Fortin E, et al. Rate of endophthalmitis after cataract surgery in Quebec, Canada, 1996e2005. Arch Ophthalmol. 2010;128(2):230-234. 37. Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2nd ed. Thousand Oaks, CA: Sage Publications Inc; 2002:3-37. 38. Urbach DR, Austin PC. Conventional models overestimate the statistical significance of volumeeoutcome associations, compared with multilevel models. J Clin Epidemiol. 2005;58(4):391-400. 39. Greenberg PB, Tseng VL, Wu WC, et al. Prevalence and predictors of ocular complications associated with cataract surgery in United States veterans. Ophthalmology. 2011;118(3):507-514. 40. Long DM. Competency-based residency training: the next advance in graduate medical education. Acad Med. 2000;75(12):1178-1183. 41. Zhou AW, Noble J, Lam WC. Canadian ophthalmology residency training: an evaluation of resident satisfaction and comparison with international standards. Can J Ophthalmol. 2009;44(5):540-547. 42. Rowden A, Krishna R. Resident cataract surgical training in United States residency programs. J Cataract Refract Surg. 2002;28(12):2202-2205. 43. O’Neill L, Lanska DJ, Hartz A. Surgeon characteristics associated with mortality and morbidity following carotid endarterectomy. Neurology. 2000;55(6):773-781. 44. Hartz AJ, Kuhn EM, Pulido J. Prestige of training programs and experience of bypass surgeons as factors in adjusted patient mortality rates. Medical Care. 1999;37(1):93-103. 45. Holm T, Johansson H, Cedermark B, et al. Influence of hospital- and surgeon-related factors on outcome after treatment of
Campbell et al
46.
47.
48.
49.
Recent Graduate Cataract Surgeon Outcomes
rectal cancer with or without preoperative radiotherapy. Br J Surg. 1997;84(5):657-663. Prystowsky JB, Bordage G, Feinglass JM. Patient outcomes for segmental colon resection according to surgeon’s training, certification, and experience. Surgery. 2002;132(4):663-670. discussion 70e72. Duclos A, Peix JL, Colin C, et al. Influence of experience on performance of individual surgeons in thyroid surgery: prospective cross sectional multicentre study. BMJ. 2012;344: d8041. Bridgewater B, Grayson AD, Au J, et al. Improving mortality of coronary surgery over first four years of independent practice: retrospective examination of prospectively collected data from 15 surgeons. BMJ. 2004;329(7463): 421. Reznick RK, MacRae H. Teaching surgical skillsdchanges in the wind. N Engl J Med. 2006;355(25):2664-2669.
50. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation systemdrationale and benefits. N Engl J Med. 2012;366(11):1051-1056. 51. Gardner AK, Scott DJ, Hebert JC, et al. Gearing up for milestones in surgery: will simulation play a role? Surgery. 2015;158(5):1421-1427. 52. Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach. 2007;29(7):642-647. 53. Lamont P, Ashworth M, Dalby S, et al. The Curriculum Framework for the Surgical Care Practitioner. London, UK: Royal College of Surgeons of England; 2014 [cited July 6, 2016]. Available at: http://accreditation.rcseng.ac.uk/pdf/SCP %20Curriculum%20Framework%202014.pdf; Accessed January 4, 2017. 54. Voelker R. Medical simulation gets real. JAMA. 2009;302(20): 2190-2192.
Footnotes and Financial Disclosures Originally received: September 7, 2016. Final revision: December 7, 2016. Accepted: December 8, 2016. Available online: ---.
Manuscript no. 2016-266.
1
Department of Ophthalmology, Queen’s University, Kingston, Canada. 2 Department of Ophthalmology, Hotel Dieu and Kingston General Hospitals, Kingston, Canada. 3
Institute for Clinical Evaluative Sciences, Kingston, Canada.
4
Department of Ophthalmology, University of Toronto, Toronto, Canada.
5
Department of Ophthalmology, Kensington Eye Institute, Toronto, Canada. 6 Division of Geriatric Medicine, Queen’s University, Kingston, Canada. 7
Division of Geriatric Medicine, St. Mary’s of the Lake Hospital, Kingston, Canada.
8
Department of Public Health Sciences, Queen’s University, Kingston, Canada. 9 Department of Ophthalmology, Western University, London, Canada. 10
Department of Ophthalmology, St. Joseph’s Hospital, London, Canada.
11
Department of Medicine, University of Toronto, Toronto, Canada.
12
Department of Health Policy Management and Evaluation, University of Toronto, Toronto, Canada.
13
Department of Medicine, Mount Sinai Hospital, Toronto, Canada.
14
the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit for publication. This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long Term Care (MOHLTC). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. In accordance with the Personal Health Information Protection Act (PHIPA) of Ontario, the raw administrative data used for statistical analyses in this manuscript may only be accessed by agents of the Institute for Clinical Evaluative Sciences (ICES), a prescribed entity under Section 45 of the Act, for the purposes of conducting research that contributes to the effectiveness, quality, equity, and efficiency of health care and health services. Dr. R. J. Campbell is an agent of ICES and had full control over the data definitions and analyses used in this manuscript. Dr. R. J. Campbell had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr. Campbell can be contacted regarding supplemental protocol details. Author Contributions: Conception and design: R. Campbell, Gill, Whitehead, E. Campbell, Bell Analysis and interpretation: R. Campbell, El-Defrawy, Gill, Whitehead, E. Campbell, Hooper, Bell, ten Hove Data collection: R. Campbell, Whitehead, E. Campbell
Institute for Clinical Evaluative Sciences, Toronto, Canada. Financial Disclosure(s): The author(s) have no proprietary or commercial interest in any materials discussed in this article.
Obtained funding: R. Campbell
R.J.C.: Supported by a Research Scientist Award Queen’s University and the Southeastern Ontario Academic Medical Organization. S.S.G.: Supported by a Canadian Institutes of Health Research New Investigator Award Institute of Aging.
Abbreviations and Acronyms: CI ¼ confidence interval; OR ¼ odds ratio.
C.M.B.: Supported by a Canadian Institutes of Health Research and Canadian Patient Safety Institute chair in Patient Safety and Continuity of Care. The sponsors of this study had no role in the design and conduct of
Overall responsibility: R. Campbell, El-Defrawy, Gill, Whitehead, E. Campbell, Hooper, Bell, ten Hove
Correspondence: Robert J. Campbell, MD, MSc, Department of Ophthalmology, Hotel Dieu Hospital, 166 Brock Street, Kingston, Ontario K7L 5G2, Canada. E-mail: [email protected].
7