- Email: [email protected]
The Relation Between Surgeon Volume and Outcome Following Off-Pump vs On-Pump Coronary Artery Bypass Graft Surgery
The Relation Between Surgeon Volume and Outcome Following Off-Pump vs On-Pump Coronary Artery Bypass Graft Surgery* Laurent G. Glance, MD; Andrew W. Dick, PhD; Turner M. Osler, MD; and Dana B. Mukamel, PhD
Study objective: Off-pump coronary artery bypass graft (CABG) surgery has been recently reintroduced into clinical practice. In light of the relatively low level of experience of most cardiac surgeons with off-pump CABG surgery, and the exceptional technical challenge of working on a “beating heart,” off-pump CABG surgery presents a unique opportunity to explore the effect of surgeon case volume on surgical outcome after controlling for the effects of patient case mix and hospital volume. Design: A retrospective cohort study analyzing the association between surgeon volume and in-hospital mortality rate for off-pump and on-pump CABG surgery using random-effects logistic regression modeling. Setting and patients: The analyses were based on the New York State clinical CABG surgery registry. The study sample consisted of 36,930 patients undergoing isolated CABG surgery between 1998 and 1999 that was performed by 181 surgeons at 33 hospitals. Interventions: None. Results: There is no association between the number of CABG procedures performed off-pump by an individual surgeon and in-hospital mortality rates (p ⴝ 0.93) after controlling for hospital CABG surgery volume and patient-level risk factors. There is also no association between the off-pump CABG surgery mortality rate and the total number of both off-pump and on-pump CABG surgery cases (p ⴝ 0.78). In the on-pump CABG surgery cohort, surgeons performing a high volume of CABG procedures had significantly lower risk-adjusted mortality rates among their patients compared to those performing a very low volume, a low-volume, and a medium volume of CABG procedures (p < 0.006). Conclusion: For off-pump CABG surgery, surgeons performing a high volume of procedures do not have better mortality outcomes than those performing a low volume of procedures. However, higher surgeon case volumes are associated with lower mortality rates for on-pump CABG surgery. The absence of a volume-outcome association for off-pump CABG surgery is especially surprising in light of the more technically demanding nature of off-pump CABG surgery compared to on-pump CABG surgery. (CHEST 2005; 128:829 – 837) Key words: coronary artery bypass graft; off pump; quality; report cards; risk adjustment; volume outcome Abbreviations: CABG ⫽ coronary artery bypass graft; CI ⫽ confidence interval; CPB ⫽ cardiopulmonary bypass; OR ⫽ odds ratio
and hospital volumes are important deP hysician terminants of outcome for many surgical procedures and medical conditions.1– 4 The association between surgical case volume and outcome after *From the University of Rochester School of Medicine and Dentistry (Drs. Glance and Dick), Rochester, NY; the University of Vermont Medical College (Dr. Osler), Burlington, VT; and the University of California, Irvine (Dr. Mukamel), Irvine, CA. This project was supported by a grant from the Agency for Healthcare and Quality Research (RO1 HS 13617). The views presented in this manuscript are those of the authors and may not reflect those of Agency for Healthcare and Quality Research or of the New York State Department of Health or of the Cardiac Advisory Committee. www.chestjournal.org
coronary artery bypass graft (CABG) surgery has been extensively studied.2,4 –13 Many of these studies2,4 – 8,12,13 have demonstrated that hospitals and surgeons with higher case volumes have lower mortality rates. These findings have led to the developManuscript received July 20, 2004; revision accepted January 31, 2005. Reproduction of this article is prohibited without written permission from the American College of Chest Physicians (www.chestjournal. org/misc/reprints.shtml). Correspondence to: Laurent G. Glance, MD, Department of Anesthesiology, University of Rochester Medical Center, 601 Elmwood Ave, Box 604, Rochester, NY 14642; e-mail: [email protected] CHEST / 128 / 2 / AUGUST, 2005
829
ment of guidelines by the American Heart Association/American College of Cardiology14 specifying the minimum number of procedures performed annually by cardiac surgeons. The selective referral of patients undergoing high-risk surgical procedures, such as CABG surgery, has been suggested to have the potential to significantly lower mortality rates by reducing the number of “potentially avoidable deaths.”3 Limited regionalization of CABG care, accomplished through the Certificate of Need Regulation at the state level, is associated with 84% higher hospital case volumes and 22% lower riskadjusted mortality rates.15 Although many studies have examined the association between volume and outcome for conventional CABG surgery, no other study has explored the surgeon volume-outcome association for CABG surgery performed without the use of cardiopulmonary bypass (CPB) [ie, off-pump CABG surgery]. Conventional CABG surgery (ie, “on-pump” surgery) is performed after the heart has been arrested using CPB. CPB allows the cardiac surgeon to perform coronary anastomoses in a bloodless and motionless surgical field. However, CPB initiates a systemic inflammatory response,16 generates microemboli and macroemboli,17 and may be responsible for much of the morbidity associated with conventional CABG surgery. The incidence of morbidity following CABG surgery, based on data from the Society of Thoracic Surgery database18 for 170,895 patients, is 35.6%.Off-pump CABG surgery may avoid many of the adverse effects associated with CPB.19 –24 However, surgery on a beating heart is technically more difficult and may be associated with suboptimal surgical exposure and a greater degree of hemodynamic instability than conventional CABG surgery.25 Despite the fact that off-pump surgery is technically more difficult, the use of the off-pump technique is growing rapidly.25 In New York State, the percentage of CABG procedures performed off-pump increased from 3% in 1997 to 27% in 2000.25 The widespread adoption of the off-pump approach is relatively new, and there are few data on the surgeon volume-outcome association for offpump CABG surgery. Because off-pump CABG surgery is a “new” procedure and is technically more challenging than on-pump CABG surgery for cardiac surgeons, it is anticipated that the outcome benefit associated with higher surgeon case volumes (“practice makes perfect”) is greater for off-pump CABG surgery than for on-pump CABG surgery. Therefore, we conducted a population-based study using the New York State Cardiac Surgery database to determine whether the surgeon “volume effect” for offpump surgery is significantly different than that for 830
on-pump surgery. The existence of a strong volumeoutcome association for off-pump CABG surgery may have important policy implications for surgeons performing a low volume of procedures and for those currently not performing off-pump CABG surgery. Materials and Methods We obtained data from the Cardiac Surgery Reporting System on all patients (n ⫽ 36,930) undergoing isolated CABG surgery in New York State between 1998 and 1999. This database includes information on patient demographics, hospital and physician identifiers (encrypted), preoperative risk factors, and outcomes. Operative mortality was the outcome of interest and was defined as “in-hospital death.” The data collection tool was developed by a Cardiac Advisory Committee, which included cardiothoracic surgeons and cardiologists. The quality of the data was monitored using audits conducted by the New York State Department of Health at the hospital level.7 For the purposes of our analysis, cases were identified as being off-pump if either (1) the CPB time was zero or (2) the case was converted from an off-pump CABG procedure to an on-pump CABG procedure. Separate multivariate logistic regression models were created for on-pump and off-pump CABG using a statistical software program (STATA SE/8.2; STATA Corp; College Station, TX). These models included only patient-level data. Stepwise selection was used to identify significant predictors of in-hospital mortality (p ⬍ 0.05). The following list of risk factors was considered for inclusion in the models: demographics (ie, age, gender, and body surface area [BSA]); measures of disease severity (ie, clinical status, ejection fraction, history of previous myocardial infarction, history of congestive heart failure, previous open heart surgery, and coronary anatomy); and comorbidities (ie, COPD, diabetes, renal failure, peripheral vascular disease, and hypertension). The method of fractional polynomials was used to determine the optimal transformation of continuous covariates.26 Model discrimination was assessed using the C statistic.27 Random-effects models were then constructed in order to model the effect of surgeon volume on outcome, after adjusting for the effect of hospital volume and patient case mix, using the macro PROC GLIMMIX (SAS; SAS Institute; Cary, NC).28 Random-effects models were used because patients are clustered by surgeons, and surgeons are clustered within hospitals. This statistical macro (PROC GLIMMIX) can be used to analyze “cross-classified” binary outcome data when the data structure is not purely hierarchical. Surgeons are not completely nested by hospital since some surgeons operate at more than one hospital. The random-effects models included the same patient-level explanatory variables used to construct the nonhierarchical models, in addition to surgeon volume and hospital volume. Hospital volume was based on the total number of off-pump and on-pump CABG procedures performed at a particular hospital. The effect of surgeon volume on mortality was examined by treating surgeon volume either as a categoric or a continuous variable; hospital volume was modeled as a continuous variable (Table 1, models 1a and 1b). Surgeon case volumes in the off-pump models were based only on procedures performed off-pump, and surgeon case volumes in the on-pump models were based only on procedures performed on-pump. Surgeons were divided into volume quartiles with equal numbers of surgeons in each quartile. For off-pump surgery, surgeons with very low case volumes performed ⬍ 5 cases over the 2-year period, surgeons with low case volumes performed between 5 and 10 cases, surgeons with medium case volumes performed between 11 and 31 cases, and surgeons with high case volumes Clinical Investigations
Table 1—Description of Models* Model Type Off-pump Death ⵑ patient risk factors ⫹ hospital volume vlv ⫹ lv ⫹ mv Death ⵑ patient risk factors ⫹ hospital volume surgeon volume (off-pump cases only) Death ⵑ patient risk factors ⫹ hospital volume surgeon volume (all CABG cases) Death ⵑ patient risk factors ⫹ hospital volume very-high volume surgeons (off-pump cases only) On-pump Death ⵑ patient risk factors ⫹ hospital volume vlv ⫹ lv ⫹ mv Death ⵑ patient risk factors ⫹ hospital volume surgeon volume (on-pump cases only
Model No. ⫹
1a
⫹
2a
⫹
3
⫹
4
patients undergoing procedures performed by surgeons with a very high case volume, based on off-pump CABG surgeon case volume (top tenth percentile), after adjusting for patient risk factors and hospital volume, to that among patients undergoing procedures performed by all of the other surgeons (model 4). C-statistics were calculated for each of the models to assess model discrimination.27 These models are shown below. Random-effects models were used to calculate risk-adjusted mortality rates for each of the volume ranges for the off-pump CABG surgery cohort (model 1a) and for the on-pump CABG surgery cohort (model 1b).
Results ⫹
1b
⫹
2b
*vlv ⫽ very low surgeon case volume; lv ⫽ low surgeon case volume; mv ⫽ medium surgeon case volume.
performed ⬎ 31 cases. For on-pump surgery, surgeons with very low case volumes performed ⬍ 52 procedures over the 2-year period, surgeons with low case volumes performed between 52 and 155 procedures, surgeons with medium case volumes performed between 156 and 273 procedures, and surgeons with high case volumes performed ⬎ 273 procedures over the 2-year period. Surgeon volume was also modeled as a continuous variable in separate models for off-pump and on-pump CABGs (Table 1, models 2a and 2b). A model was also created to evaluate the association between surgeon case volume based on all CABG cases (ie, on-pump and off-pump) and mortality after off-pump CABG (model 3). A final model was created for the off-pump CABG surgery cohort, which compared the mortality rate among
The study population included all patients undergoing isolated CABG surgery in New York state who were discharged from the hospital between 1998 and 1999. These patients were treated by 181 surgeons working at 33 hospitals. Overall, 14.1% of the CABG procedures were performed off-pump. Of the 181 surgeons in the study cohort, 154 performed one or more off-pump CABG procedures over the 2-year period. Figure 1 displays the distributions of offpump and on-pump procedures by hospital. Figure 2 displays the distribution of off-pump and on-pump procedures by surgeon. Figures 1 and 2 show wide ranges in procedure volumes for both hospitals and surgeons performing on-pump and off-pump surgery. Separate random-effects logistic regression models were created for off-pump and on-pump CABG procedures. The regression coefficients for models
Figure 1. Off-pump and on-pump CABG procedure volume by hospital, giving the number of off-pump and on-pump CABG procedures performed at each hospital over a 2-year period. www.chestjournal.org
CHEST / 128 / 2 / AUGUST, 2005
831
Figure 2. Off-pump and on-pump CABG procedure volume by surgeon, giving the number of off-pump and on-pump CABG procedures performed by each surgeon over a 2-year period. Note that encrypted surgeon identifiers have been deleted from the x-axis due to the large number of surgeons.
1a and 1b are shown in Tables 2 and 3. All models exhibited excellent discrimination (C-statistics, ⬎ 0.84). Significant independent predictors of mortality for both models included age, BSA, shock, hemodynamic instability, ejection fraction, previous open heart surgery, and comorbidities (eg, diabetes, end-stage renal disease receiving dialysis, hepatic failure, carotid disease, and the presence of a calcified aorta). These regression models were used to examine the impact of surgeon case volume (based on on-pump cases) on the on-pump CABG surgery mortality rate, and the impact of surgeon case volume (based on off-pump cases) on the off-pump CABG surgery mortality rate. Figure 3 displays the effect of surgeon volume on the adjusted mortality rate of patients undergoing off-pump and on-pump CABG surgery. There was no evidence of an association between the number of cases performed off-pump by an individual surgeon and the in-hospital mortality rate after controlling for hospital volume and patient-level risk factors. Compared to surgeons with high case volumes, the adjusted odds ratio (OR) for surgeons with very low case volumes was 0.65 (95% confidence interval [CI], 0.18 to 2.38), the adjusted OR for surgeons with low case volumes surgeons was 0.97 (95% CI, 0.48 to 2.00), and the adjusted OR for surgeons with medium case volumes was 0.78 (95% CI, 0.45 to 1.35) [Table 4]. There was also no 832
association between the off-pump CABG mortality rate and the total number of both off-pump and on-pump cases performed by surgeons (p ⫽ 0.78). Finally, the adjusted mortality rate for patients in the off-pump CABG cohort (90th percentile) undergoing procedures performed by surgeons with the highest case volumes was not significantly different from that of patients undergoing procedures by the other surgeons (p ⫽ 0.37). The median number of off-pump cases performed by this group of surgeons with very high case volumes was 127 over a 2-year period. After controlling for the effects of surgeon case volume, the effect of hospital case volume (based on the total number of on-pump and offpump CABG procedures) was not significant (p ⫽ 0.15). Another model was constructed to rule out the possibility that the effect of surgeon off-pump case volume was being obscured by the effect of the hospital case volume secondary to multicollinearity problems. This model, which did not include hospital volume, did not demonstrate a significant surgeon volume effect (p ⫽ 0.95). Since we used an intention-to-treat analysis (we defined off-pump cases to include those that started off-pump and were converted to on-pump), we were concerned that a bias could be introduced if surgeons with a lower case volume (ie, less experienced surgeons) were more likely to convert off-pump Clinical Investigations
Table 2—Model for Off-Pump CABG Surgery Mortality* Patient Risk Factors Demographics Age BSA BSA2 Disease severity Unstable Shock Stent thrombosis Ejection fraction 20–29% 30–39% 40–49% IV NTG within 24 h before operation Malignant ventricular arrhythmia More than one previous MI 70–100% stenosis in LAD (mid or distal) or major diagonal artery Prior hospital admission for previous PTCA Previous open-heart surgery Comorbidities Diabetes Dialysis Hepatic failure Calcified aorta Carotid disease Surgeon case volume Very low Low Medium Hospital volume (increments of 100 cases) Model performance patient number C-statistic
OR
p Value
1.05 0.44 8.39
⬍ 0.0001 0.013 0.005
3.50 29.19 2.18 2.62
0.001 ⬍ 0.0001 0.114 ⬍ 0.0001
2.11 1.66 2.07 2.46 1.51 0.71
⬍ 0.001 0.007 ⬍ 0.0001 0.006 0.022 0.025
0.55 1.87
0.014 0.002
1.48 6.22 5.88 2.51 1.86
0.012 ⬍ 0.0001 0.006 ⬍ 0.0001 ⬍ 0.0001
0.65 0.97 0.78 0.97
0.513 0.945 0.374 0.149
5,298 0.87
NA NA
*Surgeon case volumes (based on off-pump CABG procedures) are compared to surgeons with high case volumes. NTG ⫽ nitroglycerin; MI ⫽ myocardial infarction; LAD ⫽ left anterior descending coronary artery; PTCA ⫽ percutaneous coronary angioplasty; NA ⫽ not applicable.
cases to on-pump cases compared to surgeons with higher case volumes. We found that the rates of conversion were 8.2% for surgeons with very low case volumes vs 1.6% for the surgeons with high case volumes surgeons (p ⬍ 0.001). However, omitting the patients undergoing procedures that were converted from off-pump to on-pump did not change the results of our analyses; there was still no evidence of a volume-outcome association for off-pump CABG surgery (p ⫽ 0.81). In the on-pump CABG surgery cohort, surgeons with high case volumes had significantly lower riskadjusted mortality rates compared to surgeons with very low case volumes, low case volumes, and medium case volumes. Compared to surgeons with high case volumes, the adjusted OR for surgeons with very low case volumes was 2.13 (95% CI, 1.38 to 3.29), the adjusted OR for surgeons with low case www.chestjournal.org
Table 3—Model for On-Pump CABG Surgery Mortality* Patient Risk Factor Demographic Age Female BSA2 Disease severity Unstable Shock Stent thrombosis CPR Ejection fraction ⬍ 20% 20–29% 30–39% ⱖ 70% CHF This hospital admission History of IV NTG within 24 h before operation Previous MI ⬍6 h 6–23 h 1–7 d ⬎7 d Left main artery disease 70–89% ⱖ 90% Previous open-heart surgery Previous PTCA during this hospital admission Malignant ventricular arrhythmia Comorbidities COPD Diabetes Dialysis Renal failure, patient not receiving dialysis Hepatic failure Aortoiliac disease Calcified aorta Stroke Carotid disease Hypertension Surgeon case volume Very low Low Medium Hospital volume (increments of 100 cases) Model performance Patient No. C-statistic
OR
p Value
1.06 1.50 6.24
⬍ 0.0001 ⬍ 0.0001 ⬍ 0.0001
2.16 4.59 1.73 3.97 2.61
⬍ 0.0001 ⬍ 0.0001 0.041 ⬍ 0.0001 ⬍ 0.0001
1.36 1.48 0.57 1.58
0.014 ⬍ 0.0001 0.046 ⬍ 0.0001
1.40 1.28 2.22
0.002 0.004 0.002
2.29 1.61 1.31 1.38
0.001 ⬍ 0.0001 0.005 0.004
1.46 2.74 1.82
0.002 ⬍ 0.0001 0.023
2.09
⬍ 0.0001
1.62 1.44 4.42 1.45 12.18 1.55 2.02 1.33 1.22 1.22
⬍ 0.0001 ⬍ 0.0001 ⬍ 0.0001 0.033 ⬍ 0.0001 ⬍ 0.0004 ⬍ 0.0001 0.015 0.035 0.044
2.13 1.69 1.48 0.99
⬍ 0.001 0.002 0.005 0.511
31,632 0.840
NA NA
*Surgeon case volumes (based on the performance of on-pump CABG procedures) are compared to surgeons with high case volumes. CHF ⫽ congestive heart failure; CPR ⫽ cardiopulmonary resuscitation. See Table 2 for other abbreviations not used in the text.
volumes was 1.69 (95% CI, 1.24 to 2.29), and the adjusted OR for surgeons with medium case volumes was 1.48 (95% CI, 1.12 to 1.95) [Table 5]. Controlling for the effects of surgeon case volume, the effect CHEST / 128 / 2 / AUGUST, 2005
833
Figure 3. Impact of surgeon case volume on mortality rate after adjusting for patient risk factors and hospital volume. The following volume ranges are for the on-pump surgery cohort based on the number of on-pump CABG procedures performed between 1998 and 1999: very low case volumes, ⬍ 52 procedures; low case volumes, 52 to 155 procedures; medium case volume, 156 to 273 procedures; and high case volume, ⬎ 273 procedures. The following case volume ranges for the off-pump procedure cohort are based on the number of off-pump CABG procedures performed between 1998 and 1999: very low case volume, ⬍ 5 procedures; low case volumes, 5 to 10 procedures; medium case volumes, 11 to 31 procedures; and high case volume, ⬎ 31 procedures. * ⫽ p ⬍ 0.006; † ⫽ none of the comparisons between the off-pump CABG surgery groups and the reference group (performed by surgeons with high case volumes) were significant (all p values ⬎ 0.35).
of hospital volume (based on the total number of on-pump and off-pump CABG procedures) was not significant (p ⫽ 0.51).
This study shows that surgeons with high case volumes performing off-pump CABG surgery do not
have better mortality outcomes among their patients than surgeons with low case volumes. The absence of a volume-outcome association for off-pump CABG surgery persisted when the procedure volume was based on the total number of off-pump and on-pump cases. However, surgeons with high case volumes have better outcomes than those with low case volumes performing on-pump CABG surgery. The
Table 4 —Effect of Surgeon Case Volume on InHospital Mortality Rate for Off-Pump CABG Surgery After Adjusting for Patient-Level Risk Factors and Hospital Case Volume*
Table 5—Effect of Surgeon Case Volume on InHospital Mortality Rate for On-Pump CABG Surgery After Adjusting for Patient-Level Risk Factors and Hospital Volume*
Discussion
Surgeon Case Volume
Adjusted OR (95% CI)
p Value
Surgeon Case Volume
Adjusted OR (95% CI)
p Value
Very low (⬍ 5 procedures) Low (5–10 procedures) Medium (11–31 procedures)
0.65 (0.18–2.38) 0.97 (0.48–2.00) 0.78 (0.45–1.35)
0.51 0.95 0.37
Very low (⬍ 52 procedures) Low (52–155 procedures) Medium (156–273 procedures)
2.13 (1.38–3.29) 1.69 (1.24–2.29) 1.48 (1.12–1.95)
⬍ 0.001 0.002 0.005
*Surgeon case volume is based only on off-pump CABG procedures. The reference population is the group undergoing procedures performed by surgeons with a high case volume (ie, ⬎ 31 cases per 2 yr). 834
*Surgeon case volume is based only on on-pump CABG procedures. The reference population is the group undergoing procedures performed by surgeons with a high case volume (ie, ⬎ 31 cases per 2 yr). Clinical Investigations
absence of a volume-outcome relation for off-pump CABG surgery is especially surprising in light of the more technically demanding nature of off-pump CABG surgery compared to on-pump CABG surgery. The existence of a direct relationship between surgical mortality and procedure volume has been shown for many high-risk surgical procedures. However, the mechanism for the volume-outcome relationship is not known. The following two mechanisms have been postulated: practice makes perfect; and selective referral.29 Practice makes perfect is based on the assumption that higher procedure volumes result in greater levels of expertise. Selective referral assumes that patients are more likely to seek treatment from or be referred to a higher quality provider. To our knowledge, no other study has compared the volume-outcome association for two closely related procedures that are characterized by significant differences in technical difficulty and that are performed by the same group of surgeons. The unexpected findings of this study may serve to elucidate the mechanism underlying the volumeoutcome association seen in many high-risk surgical procedures. Since off-pump CABG surgery is a relatively new procedure, it is unlikely that selective referral would play any role in the volume-outcome association for off-pump CABG surgery because information on off-pump CABG surgery outcomes would not have been available to patients or to their cardiologists (there is a 3-year time lag between data collection and public release of surgeon and hospital CABG performance in New York state). Given the inherent logic of practice makes perfect and the technically challenging nature of off-pump CABG surgery, we expected to find that surgeons with high case volumes performing off-pump CABG surgery would have better outcomes compared to surgeons with low case volumes. One possible explanation for the absence of a volume-outcome association for offpump CABG surgery is that the mechanism for the volume-outcome association for CABG surgery is selective referral, as opposed to practice makes perfect. If we assume that patients were more likely to go to surgeons with better reputations, they would be drawn to providers whose reputations were based only on performing on-pump CABG surgery, since very few of the CABG procedures performed in New York state prior to 1998 were performed off-pump. Thus, if selective referral is the basis for the volumeoutcome association for on-pump CABG, and if patients chose their surgeon based on outcome measures for on-pump CABG surgery, then the lack of an association between volume and outcome for off-pump CABG surgery is reasonable. www.chestjournal.org
If selective referral does not apply to this cohort of off-pump CABG surgeons, then the absence of an association between the number of off-pump cases performed by individual surgeons and mortality suggests that practice makes perfect is not an important determinant of surgical outcome for this high-risk surgical procedure. Even among surgeons with the highest case volumes (ie, 90th percentile), who had performed a median number of 126 off-pump CABG procedures over the 2-year study period, there was no improvement in outcomes. Nonetheless, it is possible that a critical off-pump procedure volume threshold, above which outcomes would have improved, was not achieved in this study population. Only one other study30 has investigated the volume-outcome association for off-pump CABG surgery. The study by Brown et al,30 which was based on a private voluntary administrative database, did not demonstrate a significant association between hospital volume and mortality. However, they demonstrated that hospitals with high case volumes (defined as hospitals handling ⬎ 100 cases annually) had lower rates of complications compared to hospitals with low case volumes. This study did not examine the relation between surgeon case volume and outcome. Our analysis has several strengths. First, it is based on a comprehensive clinical data set that contains ⬎ 50 clinical predictors, as opposed to an administrative data set. Some of the potential problems associated with the use of administrative data include the inability to effectively distinguish complications from comorbidities, the lack of completeness of administrative records due to the limited number of coding slots, and the absence of precise definitions for International Classification of Diseases, ninth revision-clinical modification, codes.31 According to a systematic review32 of the volume-outcome literature, only one in four volume-outcome studies are based on clinical data. Second, since the New York state clinical CABG surgery registry includes data on all isolated CABG procedures performed in New York, this study avoids the problem of selection bias that is potentially present in voluntary data sets. It should be noted, however, that the use of data from a single state may limit the generalizability of our findings to other regions in the United States. Certificate of Need regulations in New York state have resulted in a degree of regionalization of CABG surgery not seen in many areas around the country.13 However, these regulations do not distinguish between on-pump and off-pump CABG surgery. Thus, a large proportion of the patients in this study had off-pump CABG surgery performed by surgeons with very low case volumes. Finally, the use of CHEST / 128 / 2 / AUGUST, 2005
835
random-effects modeling allowed us to simultaneously control for the effects of clustering of patients by surgeon, clustering of surgeons by hospital, and hospital volume. Our analysis also has several potential limitations. First, although clinical data are generally more accurate than administrative data, clinical data may also be subject to “gaming” (ie, some surgeons may deliberately miscode patient risk factors in order to increase the “apparent” disease severity of their patients and to decrease their risk-adjusted mortality rates).33 However, the New York State Department of Health conducts yearly audits comparing registry data to data abstracted from medical records in order to verify the accuracy of the data in the registry.34 Second, the number of CABG procedures performed off-pump may not have been large enough to detect a more significant volume-outcome association for off-pump CABG surgery. Because the 95% CI for the adjusted OR for surgeon off-pump case volume (in increments of 10 cases) was sufficiently small (0.98 to 1.02), inadequate power is unlikely to be important. Moreover, the only other large clinical databases for CABG surgery are maintained by the Society of Thoracic Surgery and by the Veterans Affairs hospital system. However, these databases cannot be used to investigate the volume-outcome association for surgeons since they do not contain surgeon identifiers.10,19 Third, it could be argued that the number of cases performed by surgeons with high case volumes (ie, ⬎ 31 cases over a 2-year period) may not have been sufficient to detect a significant volume-outcome effect. However, a separate analysis comparing the surgeons with the highest case volumes (ie, 90th percentile), who performed ⬎ 126 procedures over a 2-year period, also did not reveal a volume-outcome effect. Fourth, this cross-sectional study represents a 2-year “snapshot” during which some surgeons were first attempting off-pump CABG surgery while others may have been abandoning this technique. But the likely impact of using a time window encompassing rapid change, with respect to the adoption of a new technique, would be to bias the results toward showing a greater “volume” effect (assuming that “practice makes perfect”). Surgeons who are just starting to perform off-pump procedures and those who are abandoning the off-pump approach would be expected to have the worst outcomes. Fifth, our use of an intention-to-treat analysis could have created a bias if surgeons with low case volumes were more likely than those with high case volumes to convert off-pump CABG procedures to on-pump CABG procedures (assuming that patients undergoing CABG surgery performed by surgeons with low case volumes have better outcomes with 836
on-pump CABG surgery than with off-pump CABG surgery). In fact, surgeons with very low case volumes were five times as likely to convert off-pump CABG procedures to on-pump procedures compared to surgeons with high case volumes (p ⬍ 0.001). However, omitting these cases did not change the results of our analysis. But, it is possible that surgeons are underreporting conversions to on-pump procedures and that the lack of a volume effect may be in due in part to those cases that “went bad” not being included in the analysis of off-pump surgery outcomes. Since this data field cannot be audited, it is impossible for us to rule out this source of bias. Finally, the in-hospital mortality rate was used instead of the 30-day mortality rate. Although data on the 30-day mortality rate were unavailable to us, the use of the 30-day mortality rate is preferred over in-hospital death, because many of the “early deaths” that occur during the 30-day window are related to complications of surgery.13 The findings of this study are especially interesting following the publication of the study by Birkmeyer and colleagues4 demonstrating that the observed association between hospital volume and surgical mortality after major cardiovascular procedures is largely mediated by surgeon case volume. We expected to find that surgeon case volume would be a more important determinant of outcome for patients undergoing off-pump CABG surgery compared to those undergoing on-pump CABG surgery. Instead, we were unable to detect a volume-outcome association for the recently reintroduced, technically more difficult off-pump CABG procedure, while confirming the findings of others of a strong volumeoutcome association for the closely related, wellestablished, and technically less difficult on-pump CABG procedure. Caution should be used in interpreting the lack of a volume-outcome association for off-pump CABG surgery given the limited scope of this study. However, if our findings are confirmed in follow-up studies with larger sample sizes, then the significance of “practice makes perfect” as a determinant of surgical outcome may be shown to be less important than currently believed for this procedure. Our findings should prompt others to carefully reexamine the validity of initiatives aiming to institute evidence-based referrals using surgeon or hospital volume criteria alone. If selective referral is the primary mechanism underlying the volume-outcome association for on-pump CABG surgery and other well-established high-risk surgical procedures, then measures aiming to increase the case volumes of surgeons with low and medium case volumes are unlikely to result in improved population outcomes. Clinical Investigations
References 1 Hewitt M. Interpreting the volume-outcome relationship in the context of health care quality: 1. Washington, DC: Institute of Medicine, 2000 2 Birkmeyer JD, Siewers AE, Finlayson EV, et al. Hospital volume and surgical mortality in the United States. N Engl J Med 2002; 346:1128 –1144 3 Dudley RA, Johansen KL, Brand R, et al. Selective referral to high-volume hospitals: estimating potentially avoidable deaths. JAMA 2000; 283:1159 –1166 4 Birkmeyer JD, Stukel TA, Siewers AE, et al. Surgeon volume and operative mortality in the United States. N Engl J Med 2003; 349:2117–2127 5 Showstack JA, Rosenfeld KE, Garnick DW, et al. Association of volume with outcome of coronary artery bypass graft surgery: scheduled vs nonscheduled operations. JAMA 1987; 257:785–789 6 Hannan EL, O‘Donnell JF, Kilburn H Jr, et al. Investigation of the relationship between volume and mortality for surgical procedures performed in New York State hospitals. JAMA 1989; 262:503–510 7 Hannan EL, Kilburn H Jr, Bernard H, et al. Coronary artery bypass surgery: the relationship between inhospital mortality rate and surgical volume after controlling for clinical risk factors. Med Care 1991; 29:1094 –1107 8 Hannan EL, Siu AL, Kumar D, et al. The decline in coronary artery bypass graft surgery mortality in New York State: the role of surgeon volume. JAMA 1995; 273:209 –213 9 Clark RE, Crawford FA, Anderson RP, et al. Outcome as a function of annual coronary artery bypass graft volume. Ann Thorac Surg 1996; 61:21–26 10 Shroyer AL, Marshall G, Warner BA, et al. No continuous relationship between Veterans Affairs hospital coronary artery bypass grafting surgical volume and operative mortality. Ann Thorac Surg 1996; 61:17–20 11 Sollano JA, Gelijns AC, Moskowitz AJ, et al. Volume-outcome relationships in cardiovascular operations: New York State, 1990 –1995. J Thorac Cardiovasc Surg 1999; 117:419 – 428 12 Nallamothu BK, Saint S, Ramsey SD, et al. The role of hospital volume in coronary artery bypass grafting: is more always better? J Am Coll Cardiol 2001; 38:1923–1930 13 Hannan EL, Wu C, Ryan TJ, et al. Do hospitals and surgeons with higher coronary artery bypass graft surgery volumes still have lower risk-adjusted mortality rates? Circulation 2003; 108:795– 801 14 ACC/AHA guidelines and indications for coronary artery bypass graft surgery: a report of the American College of Cardiology/American Heart Association Task Force on Assessment of Diagnostic and Therapeutic Cardiovascular Procedures (Subcommittee on Coronary Artery Bypass Graft Surgery). Circulation 1991; 83:1125–1173 15 Vaughan-Sarrazin MS, Hannan EL, Gormley CJ, et al. Mortality in Medicare beneficiaries following coronary artery bypass graft surgery in states with and without certificate of need regulation. JAMA 2002; 288:1859 –1866 16 Bruins P, te Velthuis H, Yazdanbakhsh AP, et al. Activation of the complement system during and after cardiopulmonary bypass surgery: postsurgery activation involves C-reactive
www.chestjournal.org
17
18
19
20 21 22
23
24
25 26 27 28 29 30
31 32 33 34
protein and is associated with postoperative arrhythmia. Circulation 1997; 96:3542–3548 Hartman GS, Yao FS, Bruefach M III, et al. Severity of aortic atheromatous disease diagnosed by transesophageal echocardiography predicts stroke and other outcomes associated with coronary artery surgery: a prospective study. Anesth Analg 1996; 83:701–708 Borst C, Grundeman PF. Minimally invasive coronary artery bypass grafting: an experimental perspective. Circulation 1999; 99:1400 –1403 Cleveland JC Jr., Shroyer AL, Chen AY, et al. Off-pump coronary artery bypass grafting decreases risk-adjusted mortality and morbidity. Ann Thorac Surg 2001; 72:1282–1288 Plomondon ME, Cleveland JC Jr, Ludwig ST, et al. Off-pump coronary artery bypass is associated with improved riskadjusted outcomes. Ann Thorac Surg 2001; 72:114 –119 Sabik JF, Gillinov AM, Blackstone EH, et al. Does off-pump coronary surgery reduce morbidity and mortality? J Thorac Cardiovasc Surg 2002; 124:698 –707 Angelini GD, Taylor FC, Reeves BC, et al. Early and midterm outcome after off-pump and on-pump surgery in Beating Heart Against Cardioplegic Arrest Studies (BHACAS 1 and 2): a pooled analysis of two randomised controlled trials. Lancet 2002; 359:1194 –1199 Magee MJ, Jablonski KA, Stamou SC, et al. Elimination of cardiopulmonary bypass improves early survival for multivessel coronary artery bypass patients. Ann Thorac Surg 2002; 73:1196 –1202 van Dijk D, Nierich AP, Jansen EW, et al. Early outcome after off-pump versus on-pump coronary bypass surgery: results from a randomized study. Circulation 2001; 104:1761– 1766 Rose EA. Off-pump coronary-artery bypass surgery. N Engl J Med 2003; 348:379 –380 Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: parsimonious parameteric modeling. Appl Stat 1994; 43:429 – 467 Hosmer DW, Lemeshow S. Applied logistic regression. 2nd ed. New York, NY: Wiley-Interscience Publications, 2000; 10 Littel R, Millikan GA, Stroup WW, et al. SAS system for mixed models. Cary, NC: SAS Institute, 1996 Luft HS, Hunt SS, Maerki SC. The volume-outcome relationship: practice-makes-perfect or selective- referral patterns? Health Serv Res 1987; 22:157–182 Brown PP, Mack MJ, Simon AW, et al. Comparing clinical outcomes in high-volume and low-volume off-pump coronary bypass operation programs. Ann Thorac Surg 2001; 72: S1009 –S1015 Iezzoni LI. Assessing quality using administrative data. Ann Intern Med 127:666 – 674, 1997 Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med 2002; 137:511–520 Jollis JG, Romano PS. Pennsylvania’s focus on heart attack: grading the scorecard. N Engl J Med 1998; 338:983–987 Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly. N Engl J Med 1996; 334:394 –398
CHEST / 128 / 2 / AUGUST, 2005
837
Study objective: Off-pump coronary artery bypass graft (CABG) surgery has been recently reintroduced into clinical practice. In light of the relatively low level of experience of most cardiac surgeons with off-pump CABG surgery, and the exceptional technical challenge of working on a “beating heart,” off-pump CABG surgery presents a unique opportunity to explore the effect of surgeon case volume on surgical outcome after controlling for the effects of patient case mix and hospital volume. Design: A retrospective cohort study analyzing the association between surgeon volume and in-hospital mortality rate for off-pump and on-pump CABG surgery using random-effects logistic regression modeling. Setting and patients: The analyses were based on the New York State clinical CABG surgery registry. The study sample consisted of 36,930 patients undergoing isolated CABG surgery between 1998 and 1999 that was performed by 181 surgeons at 33 hospitals. Interventions: None. Results: There is no association between the number of CABG procedures performed off-pump by an individual surgeon and in-hospital mortality rates (p ⴝ 0.93) after controlling for hospital CABG surgery volume and patient-level risk factors. There is also no association between the off-pump CABG surgery mortality rate and the total number of both off-pump and on-pump CABG surgery cases (p ⴝ 0.78). In the on-pump CABG surgery cohort, surgeons performing a high volume of CABG procedures had significantly lower risk-adjusted mortality rates among their patients compared to those performing a very low volume, a low-volume, and a medium volume of CABG procedures (p < 0.006). Conclusion: For off-pump CABG surgery, surgeons performing a high volume of procedures do not have better mortality outcomes than those performing a low volume of procedures. However, higher surgeon case volumes are associated with lower mortality rates for on-pump CABG surgery. The absence of a volume-outcome association for off-pump CABG surgery is especially surprising in light of the more technically demanding nature of off-pump CABG surgery compared to on-pump CABG surgery. (CHEST 2005; 128:829 – 837) Key words: coronary artery bypass graft; off pump; quality; report cards; risk adjustment; volume outcome Abbreviations: CABG ⫽ coronary artery bypass graft; CI ⫽ confidence interval; CPB ⫽ cardiopulmonary bypass; OR ⫽ odds ratio
and hospital volumes are important deP hysician terminants of outcome for many surgical procedures and medical conditions.1– 4 The association between surgical case volume and outcome after *From the University of Rochester School of Medicine and Dentistry (Drs. Glance and Dick), Rochester, NY; the University of Vermont Medical College (Dr. Osler), Burlington, VT; and the University of California, Irvine (Dr. Mukamel), Irvine, CA. This project was supported by a grant from the Agency for Healthcare and Quality Research (RO1 HS 13617). The views presented in this manuscript are those of the authors and may not reflect those of Agency for Healthcare and Quality Research or of the New York State Department of Health or of the Cardiac Advisory Committee. www.chestjournal.org
coronary artery bypass graft (CABG) surgery has been extensively studied.2,4 –13 Many of these studies2,4 – 8,12,13 have demonstrated that hospitals and surgeons with higher case volumes have lower mortality rates. These findings have led to the developManuscript received July 20, 2004; revision accepted January 31, 2005. Reproduction of this article is prohibited without written permission from the American College of Chest Physicians (www.chestjournal. org/misc/reprints.shtml). Correspondence to: Laurent G. Glance, MD, Department of Anesthesiology, University of Rochester Medical Center, 601 Elmwood Ave, Box 604, Rochester, NY 14642; e-mail: [email protected] CHEST / 128 / 2 / AUGUST, 2005
829
ment of guidelines by the American Heart Association/American College of Cardiology14 specifying the minimum number of procedures performed annually by cardiac surgeons. The selective referral of patients undergoing high-risk surgical procedures, such as CABG surgery, has been suggested to have the potential to significantly lower mortality rates by reducing the number of “potentially avoidable deaths.”3 Limited regionalization of CABG care, accomplished through the Certificate of Need Regulation at the state level, is associated with 84% higher hospital case volumes and 22% lower riskadjusted mortality rates.15 Although many studies have examined the association between volume and outcome for conventional CABG surgery, no other study has explored the surgeon volume-outcome association for CABG surgery performed without the use of cardiopulmonary bypass (CPB) [ie, off-pump CABG surgery]. Conventional CABG surgery (ie, “on-pump” surgery) is performed after the heart has been arrested using CPB. CPB allows the cardiac surgeon to perform coronary anastomoses in a bloodless and motionless surgical field. However, CPB initiates a systemic inflammatory response,16 generates microemboli and macroemboli,17 and may be responsible for much of the morbidity associated with conventional CABG surgery. The incidence of morbidity following CABG surgery, based on data from the Society of Thoracic Surgery database18 for 170,895 patients, is 35.6%.Off-pump CABG surgery may avoid many of the adverse effects associated with CPB.19 –24 However, surgery on a beating heart is technically more difficult and may be associated with suboptimal surgical exposure and a greater degree of hemodynamic instability than conventional CABG surgery.25 Despite the fact that off-pump surgery is technically more difficult, the use of the off-pump technique is growing rapidly.25 In New York State, the percentage of CABG procedures performed off-pump increased from 3% in 1997 to 27% in 2000.25 The widespread adoption of the off-pump approach is relatively new, and there are few data on the surgeon volume-outcome association for offpump CABG surgery. Because off-pump CABG surgery is a “new” procedure and is technically more challenging than on-pump CABG surgery for cardiac surgeons, it is anticipated that the outcome benefit associated with higher surgeon case volumes (“practice makes perfect”) is greater for off-pump CABG surgery than for on-pump CABG surgery. Therefore, we conducted a population-based study using the New York State Cardiac Surgery database to determine whether the surgeon “volume effect” for offpump surgery is significantly different than that for 830
on-pump surgery. The existence of a strong volumeoutcome association for off-pump CABG surgery may have important policy implications for surgeons performing a low volume of procedures and for those currently not performing off-pump CABG surgery. Materials and Methods We obtained data from the Cardiac Surgery Reporting System on all patients (n ⫽ 36,930) undergoing isolated CABG surgery in New York State between 1998 and 1999. This database includes information on patient demographics, hospital and physician identifiers (encrypted), preoperative risk factors, and outcomes. Operative mortality was the outcome of interest and was defined as “in-hospital death.” The data collection tool was developed by a Cardiac Advisory Committee, which included cardiothoracic surgeons and cardiologists. The quality of the data was monitored using audits conducted by the New York State Department of Health at the hospital level.7 For the purposes of our analysis, cases were identified as being off-pump if either (1) the CPB time was zero or (2) the case was converted from an off-pump CABG procedure to an on-pump CABG procedure. Separate multivariate logistic regression models were created for on-pump and off-pump CABG using a statistical software program (STATA SE/8.2; STATA Corp; College Station, TX). These models included only patient-level data. Stepwise selection was used to identify significant predictors of in-hospital mortality (p ⬍ 0.05). The following list of risk factors was considered for inclusion in the models: demographics (ie, age, gender, and body surface area [BSA]); measures of disease severity (ie, clinical status, ejection fraction, history of previous myocardial infarction, history of congestive heart failure, previous open heart surgery, and coronary anatomy); and comorbidities (ie, COPD, diabetes, renal failure, peripheral vascular disease, and hypertension). The method of fractional polynomials was used to determine the optimal transformation of continuous covariates.26 Model discrimination was assessed using the C statistic.27 Random-effects models were then constructed in order to model the effect of surgeon volume on outcome, after adjusting for the effect of hospital volume and patient case mix, using the macro PROC GLIMMIX (SAS; SAS Institute; Cary, NC).28 Random-effects models were used because patients are clustered by surgeons, and surgeons are clustered within hospitals. This statistical macro (PROC GLIMMIX) can be used to analyze “cross-classified” binary outcome data when the data structure is not purely hierarchical. Surgeons are not completely nested by hospital since some surgeons operate at more than one hospital. The random-effects models included the same patient-level explanatory variables used to construct the nonhierarchical models, in addition to surgeon volume and hospital volume. Hospital volume was based on the total number of off-pump and on-pump CABG procedures performed at a particular hospital. The effect of surgeon volume on mortality was examined by treating surgeon volume either as a categoric or a continuous variable; hospital volume was modeled as a continuous variable (Table 1, models 1a and 1b). Surgeon case volumes in the off-pump models were based only on procedures performed off-pump, and surgeon case volumes in the on-pump models were based only on procedures performed on-pump. Surgeons were divided into volume quartiles with equal numbers of surgeons in each quartile. For off-pump surgery, surgeons with very low case volumes performed ⬍ 5 cases over the 2-year period, surgeons with low case volumes performed between 5 and 10 cases, surgeons with medium case volumes performed between 11 and 31 cases, and surgeons with high case volumes Clinical Investigations
Table 1—Description of Models* Model Type Off-pump Death ⵑ patient risk factors ⫹ hospital volume vlv ⫹ lv ⫹ mv Death ⵑ patient risk factors ⫹ hospital volume surgeon volume (off-pump cases only) Death ⵑ patient risk factors ⫹ hospital volume surgeon volume (all CABG cases) Death ⵑ patient risk factors ⫹ hospital volume very-high volume surgeons (off-pump cases only) On-pump Death ⵑ patient risk factors ⫹ hospital volume vlv ⫹ lv ⫹ mv Death ⵑ patient risk factors ⫹ hospital volume surgeon volume (on-pump cases only
Model No. ⫹
1a
⫹
2a
⫹
3
⫹
4
patients undergoing procedures performed by surgeons with a very high case volume, based on off-pump CABG surgeon case volume (top tenth percentile), after adjusting for patient risk factors and hospital volume, to that among patients undergoing procedures performed by all of the other surgeons (model 4). C-statistics were calculated for each of the models to assess model discrimination.27 These models are shown below. Random-effects models were used to calculate risk-adjusted mortality rates for each of the volume ranges for the off-pump CABG surgery cohort (model 1a) and for the on-pump CABG surgery cohort (model 1b).
Results ⫹
1b
⫹
2b
*vlv ⫽ very low surgeon case volume; lv ⫽ low surgeon case volume; mv ⫽ medium surgeon case volume.
performed ⬎ 31 cases. For on-pump surgery, surgeons with very low case volumes performed ⬍ 52 procedures over the 2-year period, surgeons with low case volumes performed between 52 and 155 procedures, surgeons with medium case volumes performed between 156 and 273 procedures, and surgeons with high case volumes performed ⬎ 273 procedures over the 2-year period. Surgeon volume was also modeled as a continuous variable in separate models for off-pump and on-pump CABGs (Table 1, models 2a and 2b). A model was also created to evaluate the association between surgeon case volume based on all CABG cases (ie, on-pump and off-pump) and mortality after off-pump CABG (model 3). A final model was created for the off-pump CABG surgery cohort, which compared the mortality rate among
The study population included all patients undergoing isolated CABG surgery in New York state who were discharged from the hospital between 1998 and 1999. These patients were treated by 181 surgeons working at 33 hospitals. Overall, 14.1% of the CABG procedures were performed off-pump. Of the 181 surgeons in the study cohort, 154 performed one or more off-pump CABG procedures over the 2-year period. Figure 1 displays the distributions of offpump and on-pump procedures by hospital. Figure 2 displays the distribution of off-pump and on-pump procedures by surgeon. Figures 1 and 2 show wide ranges in procedure volumes for both hospitals and surgeons performing on-pump and off-pump surgery. Separate random-effects logistic regression models were created for off-pump and on-pump CABG procedures. The regression coefficients for models
Figure 1. Off-pump and on-pump CABG procedure volume by hospital, giving the number of off-pump and on-pump CABG procedures performed at each hospital over a 2-year period. www.chestjournal.org
CHEST / 128 / 2 / AUGUST, 2005
831
Figure 2. Off-pump and on-pump CABG procedure volume by surgeon, giving the number of off-pump and on-pump CABG procedures performed by each surgeon over a 2-year period. Note that encrypted surgeon identifiers have been deleted from the x-axis due to the large number of surgeons.
1a and 1b are shown in Tables 2 and 3. All models exhibited excellent discrimination (C-statistics, ⬎ 0.84). Significant independent predictors of mortality for both models included age, BSA, shock, hemodynamic instability, ejection fraction, previous open heart surgery, and comorbidities (eg, diabetes, end-stage renal disease receiving dialysis, hepatic failure, carotid disease, and the presence of a calcified aorta). These regression models were used to examine the impact of surgeon case volume (based on on-pump cases) on the on-pump CABG surgery mortality rate, and the impact of surgeon case volume (based on off-pump cases) on the off-pump CABG surgery mortality rate. Figure 3 displays the effect of surgeon volume on the adjusted mortality rate of patients undergoing off-pump and on-pump CABG surgery. There was no evidence of an association between the number of cases performed off-pump by an individual surgeon and the in-hospital mortality rate after controlling for hospital volume and patient-level risk factors. Compared to surgeons with high case volumes, the adjusted odds ratio (OR) for surgeons with very low case volumes was 0.65 (95% confidence interval [CI], 0.18 to 2.38), the adjusted OR for surgeons with low case volumes surgeons was 0.97 (95% CI, 0.48 to 2.00), and the adjusted OR for surgeons with medium case volumes was 0.78 (95% CI, 0.45 to 1.35) [Table 4]. There was also no 832
association between the off-pump CABG mortality rate and the total number of both off-pump and on-pump cases performed by surgeons (p ⫽ 0.78). Finally, the adjusted mortality rate for patients in the off-pump CABG cohort (90th percentile) undergoing procedures performed by surgeons with the highest case volumes was not significantly different from that of patients undergoing procedures by the other surgeons (p ⫽ 0.37). The median number of off-pump cases performed by this group of surgeons with very high case volumes was 127 over a 2-year period. After controlling for the effects of surgeon case volume, the effect of hospital case volume (based on the total number of on-pump and offpump CABG procedures) was not significant (p ⫽ 0.15). Another model was constructed to rule out the possibility that the effect of surgeon off-pump case volume was being obscured by the effect of the hospital case volume secondary to multicollinearity problems. This model, which did not include hospital volume, did not demonstrate a significant surgeon volume effect (p ⫽ 0.95). Since we used an intention-to-treat analysis (we defined off-pump cases to include those that started off-pump and were converted to on-pump), we were concerned that a bias could be introduced if surgeons with a lower case volume (ie, less experienced surgeons) were more likely to convert off-pump Clinical Investigations
Table 2—Model for Off-Pump CABG Surgery Mortality* Patient Risk Factors Demographics Age BSA BSA2 Disease severity Unstable Shock Stent thrombosis Ejection fraction 20–29% 30–39% 40–49% IV NTG within 24 h before operation Malignant ventricular arrhythmia More than one previous MI 70–100% stenosis in LAD (mid or distal) or major diagonal artery Prior hospital admission for previous PTCA Previous open-heart surgery Comorbidities Diabetes Dialysis Hepatic failure Calcified aorta Carotid disease Surgeon case volume Very low Low Medium Hospital volume (increments of 100 cases) Model performance patient number C-statistic
OR
p Value
1.05 0.44 8.39
⬍ 0.0001 0.013 0.005
3.50 29.19 2.18 2.62
0.001 ⬍ 0.0001 0.114 ⬍ 0.0001
2.11 1.66 2.07 2.46 1.51 0.71
⬍ 0.001 0.007 ⬍ 0.0001 0.006 0.022 0.025
0.55 1.87
0.014 0.002
1.48 6.22 5.88 2.51 1.86
0.012 ⬍ 0.0001 0.006 ⬍ 0.0001 ⬍ 0.0001
0.65 0.97 0.78 0.97
0.513 0.945 0.374 0.149
5,298 0.87
NA NA
*Surgeon case volumes (based on off-pump CABG procedures) are compared to surgeons with high case volumes. NTG ⫽ nitroglycerin; MI ⫽ myocardial infarction; LAD ⫽ left anterior descending coronary artery; PTCA ⫽ percutaneous coronary angioplasty; NA ⫽ not applicable.
cases to on-pump cases compared to surgeons with higher case volumes. We found that the rates of conversion were 8.2% for surgeons with very low case volumes vs 1.6% for the surgeons with high case volumes surgeons (p ⬍ 0.001). However, omitting the patients undergoing procedures that were converted from off-pump to on-pump did not change the results of our analyses; there was still no evidence of a volume-outcome association for off-pump CABG surgery (p ⫽ 0.81). In the on-pump CABG surgery cohort, surgeons with high case volumes had significantly lower riskadjusted mortality rates compared to surgeons with very low case volumes, low case volumes, and medium case volumes. Compared to surgeons with high case volumes, the adjusted OR for surgeons with very low case volumes was 2.13 (95% CI, 1.38 to 3.29), the adjusted OR for surgeons with low case www.chestjournal.org
Table 3—Model for On-Pump CABG Surgery Mortality* Patient Risk Factor Demographic Age Female BSA2 Disease severity Unstable Shock Stent thrombosis CPR Ejection fraction ⬍ 20% 20–29% 30–39% ⱖ 70% CHF This hospital admission History of IV NTG within 24 h before operation Previous MI ⬍6 h 6–23 h 1–7 d ⬎7 d Left main artery disease 70–89% ⱖ 90% Previous open-heart surgery Previous PTCA during this hospital admission Malignant ventricular arrhythmia Comorbidities COPD Diabetes Dialysis Renal failure, patient not receiving dialysis Hepatic failure Aortoiliac disease Calcified aorta Stroke Carotid disease Hypertension Surgeon case volume Very low Low Medium Hospital volume (increments of 100 cases) Model performance Patient No. C-statistic
OR
p Value
1.06 1.50 6.24
⬍ 0.0001 ⬍ 0.0001 ⬍ 0.0001
2.16 4.59 1.73 3.97 2.61
⬍ 0.0001 ⬍ 0.0001 0.041 ⬍ 0.0001 ⬍ 0.0001
1.36 1.48 0.57 1.58
0.014 ⬍ 0.0001 0.046 ⬍ 0.0001
1.40 1.28 2.22
0.002 0.004 0.002
2.29 1.61 1.31 1.38
0.001 ⬍ 0.0001 0.005 0.004
1.46 2.74 1.82
0.002 ⬍ 0.0001 0.023
2.09
⬍ 0.0001
1.62 1.44 4.42 1.45 12.18 1.55 2.02 1.33 1.22 1.22
⬍ 0.0001 ⬍ 0.0001 ⬍ 0.0001 0.033 ⬍ 0.0001 ⬍ 0.0004 ⬍ 0.0001 0.015 0.035 0.044
2.13 1.69 1.48 0.99
⬍ 0.001 0.002 0.005 0.511
31,632 0.840
NA NA
*Surgeon case volumes (based on the performance of on-pump CABG procedures) are compared to surgeons with high case volumes. CHF ⫽ congestive heart failure; CPR ⫽ cardiopulmonary resuscitation. See Table 2 for other abbreviations not used in the text.
volumes was 1.69 (95% CI, 1.24 to 2.29), and the adjusted OR for surgeons with medium case volumes was 1.48 (95% CI, 1.12 to 1.95) [Table 5]. Controlling for the effects of surgeon case volume, the effect CHEST / 128 / 2 / AUGUST, 2005
833
Figure 3. Impact of surgeon case volume on mortality rate after adjusting for patient risk factors and hospital volume. The following volume ranges are for the on-pump surgery cohort based on the number of on-pump CABG procedures performed between 1998 and 1999: very low case volumes, ⬍ 52 procedures; low case volumes, 52 to 155 procedures; medium case volume, 156 to 273 procedures; and high case volume, ⬎ 273 procedures. The following case volume ranges for the off-pump procedure cohort are based on the number of off-pump CABG procedures performed between 1998 and 1999: very low case volume, ⬍ 5 procedures; low case volumes, 5 to 10 procedures; medium case volumes, 11 to 31 procedures; and high case volume, ⬎ 31 procedures. * ⫽ p ⬍ 0.006; † ⫽ none of the comparisons between the off-pump CABG surgery groups and the reference group (performed by surgeons with high case volumes) were significant (all p values ⬎ 0.35).
of hospital volume (based on the total number of on-pump and off-pump CABG procedures) was not significant (p ⫽ 0.51).
This study shows that surgeons with high case volumes performing off-pump CABG surgery do not
have better mortality outcomes among their patients than surgeons with low case volumes. The absence of a volume-outcome association for off-pump CABG surgery persisted when the procedure volume was based on the total number of off-pump and on-pump cases. However, surgeons with high case volumes have better outcomes than those with low case volumes performing on-pump CABG surgery. The
Table 4 —Effect of Surgeon Case Volume on InHospital Mortality Rate for Off-Pump CABG Surgery After Adjusting for Patient-Level Risk Factors and Hospital Case Volume*
Table 5—Effect of Surgeon Case Volume on InHospital Mortality Rate for On-Pump CABG Surgery After Adjusting for Patient-Level Risk Factors and Hospital Volume*
Discussion
Surgeon Case Volume
Adjusted OR (95% CI)
p Value
Surgeon Case Volume
Adjusted OR (95% CI)
p Value
Very low (⬍ 5 procedures) Low (5–10 procedures) Medium (11–31 procedures)
0.65 (0.18–2.38) 0.97 (0.48–2.00) 0.78 (0.45–1.35)
0.51 0.95 0.37
Very low (⬍ 52 procedures) Low (52–155 procedures) Medium (156–273 procedures)
2.13 (1.38–3.29) 1.69 (1.24–2.29) 1.48 (1.12–1.95)
⬍ 0.001 0.002 0.005
*Surgeon case volume is based only on off-pump CABG procedures. The reference population is the group undergoing procedures performed by surgeons with a high case volume (ie, ⬎ 31 cases per 2 yr). 834
*Surgeon case volume is based only on on-pump CABG procedures. The reference population is the group undergoing procedures performed by surgeons with a high case volume (ie, ⬎ 31 cases per 2 yr). Clinical Investigations
absence of a volume-outcome relation for off-pump CABG surgery is especially surprising in light of the more technically demanding nature of off-pump CABG surgery compared to on-pump CABG surgery. The existence of a direct relationship between surgical mortality and procedure volume has been shown for many high-risk surgical procedures. However, the mechanism for the volume-outcome relationship is not known. The following two mechanisms have been postulated: practice makes perfect; and selective referral.29 Practice makes perfect is based on the assumption that higher procedure volumes result in greater levels of expertise. Selective referral assumes that patients are more likely to seek treatment from or be referred to a higher quality provider. To our knowledge, no other study has compared the volume-outcome association for two closely related procedures that are characterized by significant differences in technical difficulty and that are performed by the same group of surgeons. The unexpected findings of this study may serve to elucidate the mechanism underlying the volumeoutcome association seen in many high-risk surgical procedures. Since off-pump CABG surgery is a relatively new procedure, it is unlikely that selective referral would play any role in the volume-outcome association for off-pump CABG surgery because information on off-pump CABG surgery outcomes would not have been available to patients or to their cardiologists (there is a 3-year time lag between data collection and public release of surgeon and hospital CABG performance in New York state). Given the inherent logic of practice makes perfect and the technically challenging nature of off-pump CABG surgery, we expected to find that surgeons with high case volumes performing off-pump CABG surgery would have better outcomes compared to surgeons with low case volumes. One possible explanation for the absence of a volume-outcome association for offpump CABG surgery is that the mechanism for the volume-outcome association for CABG surgery is selective referral, as opposed to practice makes perfect. If we assume that patients were more likely to go to surgeons with better reputations, they would be drawn to providers whose reputations were based only on performing on-pump CABG surgery, since very few of the CABG procedures performed in New York state prior to 1998 were performed off-pump. Thus, if selective referral is the basis for the volumeoutcome association for on-pump CABG, and if patients chose their surgeon based on outcome measures for on-pump CABG surgery, then the lack of an association between volume and outcome for off-pump CABG surgery is reasonable. www.chestjournal.org
If selective referral does not apply to this cohort of off-pump CABG surgeons, then the absence of an association between the number of off-pump cases performed by individual surgeons and mortality suggests that practice makes perfect is not an important determinant of surgical outcome for this high-risk surgical procedure. Even among surgeons with the highest case volumes (ie, 90th percentile), who had performed a median number of 126 off-pump CABG procedures over the 2-year study period, there was no improvement in outcomes. Nonetheless, it is possible that a critical off-pump procedure volume threshold, above which outcomes would have improved, was not achieved in this study population. Only one other study30 has investigated the volume-outcome association for off-pump CABG surgery. The study by Brown et al,30 which was based on a private voluntary administrative database, did not demonstrate a significant association between hospital volume and mortality. However, they demonstrated that hospitals with high case volumes (defined as hospitals handling ⬎ 100 cases annually) had lower rates of complications compared to hospitals with low case volumes. This study did not examine the relation between surgeon case volume and outcome. Our analysis has several strengths. First, it is based on a comprehensive clinical data set that contains ⬎ 50 clinical predictors, as opposed to an administrative data set. Some of the potential problems associated with the use of administrative data include the inability to effectively distinguish complications from comorbidities, the lack of completeness of administrative records due to the limited number of coding slots, and the absence of precise definitions for International Classification of Diseases, ninth revision-clinical modification, codes.31 According to a systematic review32 of the volume-outcome literature, only one in four volume-outcome studies are based on clinical data. Second, since the New York state clinical CABG surgery registry includes data on all isolated CABG procedures performed in New York, this study avoids the problem of selection bias that is potentially present in voluntary data sets. It should be noted, however, that the use of data from a single state may limit the generalizability of our findings to other regions in the United States. Certificate of Need regulations in New York state have resulted in a degree of regionalization of CABG surgery not seen in many areas around the country.13 However, these regulations do not distinguish between on-pump and off-pump CABG surgery. Thus, a large proportion of the patients in this study had off-pump CABG surgery performed by surgeons with very low case volumes. Finally, the use of CHEST / 128 / 2 / AUGUST, 2005
835
random-effects modeling allowed us to simultaneously control for the effects of clustering of patients by surgeon, clustering of surgeons by hospital, and hospital volume. Our analysis also has several potential limitations. First, although clinical data are generally more accurate than administrative data, clinical data may also be subject to “gaming” (ie, some surgeons may deliberately miscode patient risk factors in order to increase the “apparent” disease severity of their patients and to decrease their risk-adjusted mortality rates).33 However, the New York State Department of Health conducts yearly audits comparing registry data to data abstracted from medical records in order to verify the accuracy of the data in the registry.34 Second, the number of CABG procedures performed off-pump may not have been large enough to detect a more significant volume-outcome association for off-pump CABG surgery. Because the 95% CI for the adjusted OR for surgeon off-pump case volume (in increments of 10 cases) was sufficiently small (0.98 to 1.02), inadequate power is unlikely to be important. Moreover, the only other large clinical databases for CABG surgery are maintained by the Society of Thoracic Surgery and by the Veterans Affairs hospital system. However, these databases cannot be used to investigate the volume-outcome association for surgeons since they do not contain surgeon identifiers.10,19 Third, it could be argued that the number of cases performed by surgeons with high case volumes (ie, ⬎ 31 cases over a 2-year period) may not have been sufficient to detect a significant volume-outcome effect. However, a separate analysis comparing the surgeons with the highest case volumes (ie, 90th percentile), who performed ⬎ 126 procedures over a 2-year period, also did not reveal a volume-outcome effect. Fourth, this cross-sectional study represents a 2-year “snapshot” during which some surgeons were first attempting off-pump CABG surgery while others may have been abandoning this technique. But the likely impact of using a time window encompassing rapid change, with respect to the adoption of a new technique, would be to bias the results toward showing a greater “volume” effect (assuming that “practice makes perfect”). Surgeons who are just starting to perform off-pump procedures and those who are abandoning the off-pump approach would be expected to have the worst outcomes. Fifth, our use of an intention-to-treat analysis could have created a bias if surgeons with low case volumes were more likely than those with high case volumes to convert off-pump CABG procedures to on-pump CABG procedures (assuming that patients undergoing CABG surgery performed by surgeons with low case volumes have better outcomes with 836
on-pump CABG surgery than with off-pump CABG surgery). In fact, surgeons with very low case volumes were five times as likely to convert off-pump CABG procedures to on-pump procedures compared to surgeons with high case volumes (p ⬍ 0.001). However, omitting these cases did not change the results of our analysis. But, it is possible that surgeons are underreporting conversions to on-pump procedures and that the lack of a volume effect may be in due in part to those cases that “went bad” not being included in the analysis of off-pump surgery outcomes. Since this data field cannot be audited, it is impossible for us to rule out this source of bias. Finally, the in-hospital mortality rate was used instead of the 30-day mortality rate. Although data on the 30-day mortality rate were unavailable to us, the use of the 30-day mortality rate is preferred over in-hospital death, because many of the “early deaths” that occur during the 30-day window are related to complications of surgery.13 The findings of this study are especially interesting following the publication of the study by Birkmeyer and colleagues4 demonstrating that the observed association between hospital volume and surgical mortality after major cardiovascular procedures is largely mediated by surgeon case volume. We expected to find that surgeon case volume would be a more important determinant of outcome for patients undergoing off-pump CABG surgery compared to those undergoing on-pump CABG surgery. Instead, we were unable to detect a volume-outcome association for the recently reintroduced, technically more difficult off-pump CABG procedure, while confirming the findings of others of a strong volumeoutcome association for the closely related, wellestablished, and technically less difficult on-pump CABG procedure. Caution should be used in interpreting the lack of a volume-outcome association for off-pump CABG surgery given the limited scope of this study. However, if our findings are confirmed in follow-up studies with larger sample sizes, then the significance of “practice makes perfect” as a determinant of surgical outcome may be shown to be less important than currently believed for this procedure. Our findings should prompt others to carefully reexamine the validity of initiatives aiming to institute evidence-based referrals using surgeon or hospital volume criteria alone. If selective referral is the primary mechanism underlying the volume-outcome association for on-pump CABG surgery and other well-established high-risk surgical procedures, then measures aiming to increase the case volumes of surgeons with low and medium case volumes are unlikely to result in improved population outcomes. Clinical Investigations
References 1 Hewitt M. Interpreting the volume-outcome relationship in the context of health care quality: 1. Washington, DC: Institute of Medicine, 2000 2 Birkmeyer JD, Siewers AE, Finlayson EV, et al. Hospital volume and surgical mortality in the United States. N Engl J Med 2002; 346:1128 –1144 3 Dudley RA, Johansen KL, Brand R, et al. Selective referral to high-volume hospitals: estimating potentially avoidable deaths. JAMA 2000; 283:1159 –1166 4 Birkmeyer JD, Stukel TA, Siewers AE, et al. Surgeon volume and operative mortality in the United States. N Engl J Med 2003; 349:2117–2127 5 Showstack JA, Rosenfeld KE, Garnick DW, et al. Association of volume with outcome of coronary artery bypass graft surgery: scheduled vs nonscheduled operations. JAMA 1987; 257:785–789 6 Hannan EL, O‘Donnell JF, Kilburn H Jr, et al. Investigation of the relationship between volume and mortality for surgical procedures performed in New York State hospitals. JAMA 1989; 262:503–510 7 Hannan EL, Kilburn H Jr, Bernard H, et al. Coronary artery bypass surgery: the relationship between inhospital mortality rate and surgical volume after controlling for clinical risk factors. Med Care 1991; 29:1094 –1107 8 Hannan EL, Siu AL, Kumar D, et al. The decline in coronary artery bypass graft surgery mortality in New York State: the role of surgeon volume. JAMA 1995; 273:209 –213 9 Clark RE, Crawford FA, Anderson RP, et al. Outcome as a function of annual coronary artery bypass graft volume. Ann Thorac Surg 1996; 61:21–26 10 Shroyer AL, Marshall G, Warner BA, et al. No continuous relationship between Veterans Affairs hospital coronary artery bypass grafting surgical volume and operative mortality. Ann Thorac Surg 1996; 61:17–20 11 Sollano JA, Gelijns AC, Moskowitz AJ, et al. Volume-outcome relationships in cardiovascular operations: New York State, 1990 –1995. J Thorac Cardiovasc Surg 1999; 117:419 – 428 12 Nallamothu BK, Saint S, Ramsey SD, et al. The role of hospital volume in coronary artery bypass grafting: is more always better? J Am Coll Cardiol 2001; 38:1923–1930 13 Hannan EL, Wu C, Ryan TJ, et al. Do hospitals and surgeons with higher coronary artery bypass graft surgery volumes still have lower risk-adjusted mortality rates? Circulation 2003; 108:795– 801 14 ACC/AHA guidelines and indications for coronary artery bypass graft surgery: a report of the American College of Cardiology/American Heart Association Task Force on Assessment of Diagnostic and Therapeutic Cardiovascular Procedures (Subcommittee on Coronary Artery Bypass Graft Surgery). Circulation 1991; 83:1125–1173 15 Vaughan-Sarrazin MS, Hannan EL, Gormley CJ, et al. Mortality in Medicare beneficiaries following coronary artery bypass graft surgery in states with and without certificate of need regulation. JAMA 2002; 288:1859 –1866 16 Bruins P, te Velthuis H, Yazdanbakhsh AP, et al. Activation of the complement system during and after cardiopulmonary bypass surgery: postsurgery activation involves C-reactive
www.chestjournal.org
17
18
19
20 21 22
23
24
25 26 27 28 29 30
31 32 33 34
protein and is associated with postoperative arrhythmia. Circulation 1997; 96:3542–3548 Hartman GS, Yao FS, Bruefach M III, et al. Severity of aortic atheromatous disease diagnosed by transesophageal echocardiography predicts stroke and other outcomes associated with coronary artery surgery: a prospective study. Anesth Analg 1996; 83:701–708 Borst C, Grundeman PF. Minimally invasive coronary artery bypass grafting: an experimental perspective. Circulation 1999; 99:1400 –1403 Cleveland JC Jr., Shroyer AL, Chen AY, et al. Off-pump coronary artery bypass grafting decreases risk-adjusted mortality and morbidity. Ann Thorac Surg 2001; 72:1282–1288 Plomondon ME, Cleveland JC Jr, Ludwig ST, et al. Off-pump coronary artery bypass is associated with improved riskadjusted outcomes. Ann Thorac Surg 2001; 72:114 –119 Sabik JF, Gillinov AM, Blackstone EH, et al. Does off-pump coronary surgery reduce morbidity and mortality? J Thorac Cardiovasc Surg 2002; 124:698 –707 Angelini GD, Taylor FC, Reeves BC, et al. Early and midterm outcome after off-pump and on-pump surgery in Beating Heart Against Cardioplegic Arrest Studies (BHACAS 1 and 2): a pooled analysis of two randomised controlled trials. Lancet 2002; 359:1194 –1199 Magee MJ, Jablonski KA, Stamou SC, et al. Elimination of cardiopulmonary bypass improves early survival for multivessel coronary artery bypass patients. Ann Thorac Surg 2002; 73:1196 –1202 van Dijk D, Nierich AP, Jansen EW, et al. Early outcome after off-pump versus on-pump coronary bypass surgery: results from a randomized study. Circulation 2001; 104:1761– 1766 Rose EA. Off-pump coronary-artery bypass surgery. N Engl J Med 2003; 348:379 –380 Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: parsimonious parameteric modeling. Appl Stat 1994; 43:429 – 467 Hosmer DW, Lemeshow S. Applied logistic regression. 2nd ed. New York, NY: Wiley-Interscience Publications, 2000; 10 Littel R, Millikan GA, Stroup WW, et al. SAS system for mixed models. Cary, NC: SAS Institute, 1996 Luft HS, Hunt SS, Maerki SC. The volume-outcome relationship: practice-makes-perfect or selective- referral patterns? Health Serv Res 1987; 22:157–182 Brown PP, Mack MJ, Simon AW, et al. Comparing clinical outcomes in high-volume and low-volume off-pump coronary bypass operation programs. Ann Thorac Surg 2001; 72: S1009 –S1015 Iezzoni LI. Assessing quality using administrative data. Ann Intern Med 127:666 – 674, 1997 Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med 2002; 137:511–520 Jollis JG, Romano PS. Pennsylvania’s focus on heart attack: grading the scorecard. N Engl J Med 1998; 338:983–987 Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly. N Engl J Med 1996; 334:394 –398
CHEST / 128 / 2 / AUGUST, 2005
837