- Email: [email protected]
Reliability of hospital readmission rates in vascular surgery
From the Society for Vascular Surgery
Reliability of hospital readmission rates in vascular surgery Andrew A. Gonzalez, MD, JD, MPH,a,c Micah E. Girotti, MD, MSc,a Terry Shih, MD,a Thomas W. Wakefield, MD,b and Justin B. Dimick, MD, MPH,a Ann Arbor, Mich; and Chicago, Ill Objective: The Center for Medicare and Medicaid Services recently began assessing financial penalties to hospitals with high readmission rates for a narrow set of medical conditions. Because these penalties will be extended to surgical conditions in the near future, we sought to determine whether readmissions are a reliable predictor of hospital performance with vascular surgery. Methods: We examined 4 years of national Medicare claims data from 1576 hospitals on beneficiaries undergoing three common vascular procedures: open or endovascular abdominal aortic aneurysm repair (n [ 81,520) or lower extremity arterial bypass (n [ 57,190). First, we divided our population into two groups on the basis of operative date (2005-2006 and 2007-2008) and generated hospital risk- and reliability-adjusted readmission rates for each time period. We evaluated reliability through the use of the “test-retest” method; highly reliable measures will show little variation in rates over time. Specifically, we evaluated the year-to-year reliability of readmissions by calculating Spearman rank correlation and weighted k tests for readmission rates between the two time periods. Results: The Spearman coefficient between 2005-2006 readmissions rankings and 2007-2008 readmissions rankings was 0.57 (P < .001) and weighted k was 0.42 (P < .001), indicating a moderate correlation. However, only 32% of the variation in hospital readmission rates in 2007-2008 was explained by readmissions during the 2 prior years. There were major reclassifications of hospital rankings between years, with 63% of hospitals migrating among performance quintiles between 2005-2006 and 2007-2008. Conclusions: Risk-adjusted readmission rates for vascular surgery vary substantially year to year; this implies that much of the observed variation in readmission rates is either random or caused by unmeasured factors and not caused by changes in hospital quality that may be captured by administrative data. (J Vasc Surg 2014;59:1638-43.)
Because readmissions are common and costly, many policymakers view reducing readmissions as the rare opportunity to improve quality while decreasing cost. For example, new provisions under the Affordable Care Act target readmissions in an effort to reduce their $12 billion contribution to annual Medicare spending.1 In addition to setting reimbursement, the Act mandates public reporting of readmissions performance on the Hospital Compare website.2 Although these changes have initially affected From the Department of Surgery, Center for Healthcare Outcomes and Policy (CHOP),a and the Section of Vascular Surgery,b University of Michigan, Ann Arbor; and the Department of Surgery, University of Illinois Hospital and Health Sciences System, Chicago.c This study was supported by grant R01 AG039434-03 (J.B.D.) from the National Institute on Aging; K08 HS017765-04 (J.B.D.) from the Agency for Healthcare Research and Quality; and Ruth L. Kirschstein National Research Service Awards T32 HL076123-09 and T32 HL076123-10 from the National Heart, Lung, and Blood Institute (A.A.G.; T.S.; T.W.W.). Author conflict of interest: none. Presented as a poster under the categories of abdominal aortic, outcomes analysis, and peripheral arterial occlusive disease at the 2013 Vascular Annual Meeting of the Society for Vascular Surgery, San Francisco, Calif, May 30-June 1, 2013. Reprint requests: Andrew A. Gonzalez, MD, JD, MPH, Center for Healthcare Outcomes and Policy (CHOP), North Campus Research Complex, 2800 Plymouth Rd, Bldg 16, Rm 100-14, Ann Arbor, MI 48109 (e-mail: [email protected]). The editors and reviewers of this article have no relevant financial relationships to disclose per the JVS policy that requires reviewers to decline review of any manuscript for which they may have a conflict of interest. 0741-5214/$36.00 Copyright Ó 2014 by the Society for Vascular Surgery. http://dx.doi.org/10.1016/j.jvs.2013.12.040
1638
select medical conditions and hip or knee surgery, they will be extended to include vascular surgery by 2015.1,3 Whether risk-adjusted readmission rates are a reliable indicator of quality remains unknown. To be a reliable indicator of quality, a measure should demonstrate stability over time. Thus, hospital rankings that are based on readmissions should vary little from year to year unless there were significant improvements in coordination of care. Prior studies have examined the temporal stability of various outcome measures. For instance, historical rates of surgical mortality can reliably predict future rates for some operations but not others.4,5 Beyond the theoretical implications, the concept of stability over time is of practical significance because the Affordable Care Act’s Hospital Readmissions Reduction Program (HRRP) will use data from 2008 through 2011 to determine pay-forperformance readmission penalties for fiscal year (FY) 2013.6 This study seeks to examine whether hospital readmission rates are persistent year to year or reflect random yearly variation. To investigate this question, we used data on Medicare beneficiaries undergoing abdominal aortic aneurysm repair or lower extremity arterial bypass. With the use of Spearman correlation, we ranked hospitals on the basis of 2005-2006 readmission rates and determined how well they predicted 2007-2008 ranks. Furthermore, we quantified the extent to which 2007-2008 rankings remained similar to 2005-2006 rankings. METHODS Databases, subjects. We used Medicare Provider Analysis and Review (MedPAR) files for beneficiaries
JOURNAL OF VASCULAR SURGERY Volume 59, Number 6
between the ages of 65 and 99 years. MedPAR files contain administrative data from discharge abstracts for all Medicare beneficiaries hospitalized on a fee-for-service basis in acute care facilities. We used International Classification of Diseases, Ninth Revision (ICD-9) procedure codes to identify patients who underwent lower extremity arterial bypass, open abdominal aortic aneurysm repair (OAR), or endovascular aortic repair (EVAR) between 2005 and 2008. For OAR and EVAR, we excluded cases with an admitting diagnosis indicating aneurysm rupture. We selected these three operations, a priori because each has a high incidence of readmission (in excess of 10%). To harmonize our study population with the methodology used by the Centers for Medicare and Medicaid Services, we excluded patients who were discharged against medical advice (n ¼ 157 patients), patients admitted as transfers from other hospitals or facilities (n ¼ 5092), and patients who were nonelective admissions (n ¼ 46,816). Finally, we excluded hospitals that did not perform at least one operation in each year of our study period (n ¼ 745 hospitals). Statistical analysis. We used the hospital as our unit of analysis. Our primary outcome was a composite hospital risk- and reliability-adjusted 30-day readmission rate. We calculated point estimates and confidence intervals for this outcome by a three-step process. First, we used standard logistic regression to calculate an empirically derived risk score. This score was calculated through the use of a continuous variable for age, a dichotomous variable for sex, and “dummy” variables for preexisting medical comorbidities on the basis of Elixhauser’s methods.7 For each hospital, a separate risk score was created for each operation and each year (12 in all). Beyond simplifying the model, a single risk score reduces the likelihood of model nonconvergence. In contrast, the use of individual variables often results in failure of the statistical program to fit an adaptive model to the covariate matrix. This approach has previously been applied to similar data in surgical patients and to the reporting of vascular outcomes.8,9 Specifically, this risk score was generated by means of the logistic postestimation command that predicts the log(odds) of readmission for each patient (-xb option in STATA). We used modeled readmissions in units of log(odds) rather than predicted probability because log(odds) is linear with respect to binary outcomes (eg, readmission). Second, we used a hierarchical mixed-effects logistic regression model that was based on the patient risk score (described above) with a hospital variable as the random effect. The Centers for Medicare and Medicaid Services uses hierarchical logistic regression to calculate hospital readmission rates for the HRRP.10 Hierarchical regression is an advanced statistical technique that leverages empirical Bayes theorem to directly model variation at the hospital level. As a result, hierarchical modeling yields more stable estimates of both coefficients and standard errors.8,11 Prior studies of outcomes after vascular surgery have validated this technique and shown it to reduce statistical noise and provide more robust estimates of hospital quality compared with risk adjustment alone.9,12
Gonzalez et al 1639
Table. Patient characteristics by cohort Variable
2005-2006
2007-2008
Abdominal aortic aneurysm repair Median age (IQR), years Sex (% women)a % Black race % 3þ Comorbiditiesa % Endovasculara Lower extremity arterial bypass Median age (IQR), years Sex (% women)a % Black racea % 3þ Comorbiditiesa % Claudicationa,b % Tissue lossc
33,973 76 (71-80) 24.7 3.8 32 59 24,606 75 (70-81) 45.4 12.6 40 27 43
47,547 75 (70-81) 23.9 3.7 39 69 32,584 75 (70-81) 42.8 11.5 45 30 43
ICD-9, International Classification of Diseases, Ninth Revision; IQR, interquartile range. a Statistically significant at P < .05. b ICD-9 code 440.21 (claudication) in any diagnostic code field. c ICD-9 codes 440.22 (rest pain); 440.23 (ulceration); 440.24 (gangrene).
Third, we generated a composite readmission rate for 2005-2006 and for 2007-2008. For each hospital, we multiplied its procedure-specific readmission rate by its volume for that procedure; summated the resulting products; and divided the result by the hospital’s total volume for all three procedures. This adjusted for differences in volume over time and for differences in the proportion of each operation performed at any individual hospitals. Thus, a hospital that performed many bypasses but few EVARs would have a composite readmission rate closer to its bypass readmission rate. Finally, we ranked hospitals according to risk- and reliability-adjusted 30-day readmission rates during the historical time period (2005-2006) and during the outcome time period (2007-2008). These ranks were used to stratify hospitals into quintiles of readmission for each time period. We used a reclassification matrix to analyze hospital migration among quintiles between the two time periods.13 Correlation was assessed by means of Spearman rank coefficients and weighted k tests. In addition to composite rankings that were based on all operations, we evaluated each operation separately. Sensitivity analysis. To ensure that our results were robust to any anomalies unique to the combination of years forming the historical and outcome time periods, we analyzed reliability and reclassification across three 1-year periods: 2005-2006, 2006-2007, and 2007-2008. RESULTS We evaluated 1576 hospitals performing OAR, EVAR, and lower extremity arterial bypass between 2005 and 2008. The mean adjusted readmission rate was 19.3% (standard deviation, 2.8%; range, 15.5%-23.4%) in 20052006 and 15.8% (standard deviation, 1.9%; range, 13.3%18.6%) in 2007-2008. Overall, 129 hospitals (8.2%) saw a decrease in their readmission rate by at least 5 percentage points from 2005-2006 to 2007-2008. In contrast, only 45 hospitals (2.9%) saw any increase in their readmission rate.
JOURNAL OF VASCULAR SURGERY June 2014
1640 Gonzalez et al
Fig 1. Hospital rankings based on risk- and reliability-adjusted readmission rates in 2005-2006 vs 2007-2008, stratified by operation type. Spearman correlation coefficients and associate P values are listed for each operation. AAA, Abdominal aortic aneurysm; EVAR, endovascular aortic repair.
In evaluating patient characteristics for both year groups, there were statistically significant differences for all demographic factors with the exception of age (Table). However, the use of EVAR was the only clinically meaningful difference between 2005-2006 and 2007-2008 (30% vs 37%; P < .001). The Spearman coefficient for the correlation between 2005-2006 ranks and 2007-2008 ranks was 0.57 (P value for independence of ranks <.001); weighted k ¼ 0.42 (P value for agreement <.001; Fig 1). This indicates a moderate correlation between hospital rankings that were based on 2005-2006 readmission rates and rankings that were based on 2007-2008 rates. Only 32% of the variation in 2007-2008 readmission rates could be explained by hospital performance in 2005-2006 (historical time period). Restated, 68% of the variation was explained by factors other than historical performance. Thus, we observed significant fluctuation in quintile rankings between the two time periods (Fig 2). Overall, 61% of hospitals were reclassified into different quintiles of performance between 2005-2006 and 2007-2008. With the exception of hospitals in the worst quintile (highest readmission rates) for 2005-2007, there are no clinically meaningful differences between the other 80% of hospitals (Fig 3). Our results were similar when examining operations individually (data not included).
Sensitivity analysis. When analyzing changes for each year individually, we found that R2 varied from 0.25 (2006 vs 2007) to 0.41 (2007 vs 2008). DISCUSSION In this study, we assessed the year-to-year variability of hospital readmission rates after major vascular surgery. To draw conclusions about groups (eg, hospitals), it is generally accepted that a measure should have reliability of at least .70.14 We found that readmission rates have suboptimal predictive reliability (.32). This implies that readmissions following major vascular surgery do not reflect a fixed set of attributes capturing a hospital’s ability to provide coordinated transitions of care (ie, optimal discharge practices). From a practical standpoint, this study has important policy implications, challenging whether Medicare’s recent implementation of penalties under the HRRP is going to incentivize hospitals in a productive way. If the penalized hospitals do not have control over the outcome (ie, readmission is due to chance alone), then assessing such penalties will not have the desired effect of reducing readmission rates. Our main finding, that readmission rates have suboptimal reliability, implies that factors other than quality drive readmissions performance. Aside from actual quality improvement (or degradation), the changes in readmission
JOURNAL OF VASCULAR SURGERY Volume 59, Number 6
Gonzalez et al 1641
Fig 2. Comparison of quintile ranking between 2005-2006 and 2007-2008 for 1576 hospitals. Shading intensity represents the magnitude reclassification; (n) represents the number of hospitals in each quintile combination. Overall, 63% of hospitals were reclassified between the 2-year groups.
Fig 3. Risk- and reliability-adjusted hospital readmission rates in 2007-2008, according to quintiles of historical readmission performance in 2005-2006. Both historical and subsequent readmission rates are adjusted for patient characteristics.
rankings that we observed may be explained by chance year-to-year variation or by unmeasured patient factors. Although the problem of chance variation in outcome measures was recognized in the 1950s,15 prior literature around reliability remains sparse. The few existing studies evaluating the reliability of surgical outcomes have focused exclusively on postoperative mortality.4,11 For instance, in the case of coronary artery bypass grafting, elective repair of abdominal aortic aneurysm, and pancreatectomy, historical mortality predicts only one-third to one-half of the variation in subsequent mortality.4 The remaining variation (the majority) is due to chance or statistical “noise.” Akin to our readmission findings, chance variation in mortality rates have led to inconsistent hospital performance ratings across years. For example, Luft et al studied coronary artery bypass grafting in California and found that only a minority of hospitals were consistent low-mortality outliers. Furthermore, not a single hospital was a consistent high-mortality outlier.5 Even among the most extreme outliers (ie, zero-mortality hospitals), there is poor reliability over time. One study examined hospitals with no postoperative deaths for 3 consecutive years. When analyzing data for the subsequent year, the authors found that these zero-mortality hospitals had no difference in
mortality rates compared with other hospitals.16 Yet, as a cross-sectional measure, mortality rate may be fairly reliable, depending on the clinical context. For example, in postcolectomy patients, the reliability of mortality ranges from 0.24 to 0.45 (5th to 95th percentile of volume), but, in trauma patients, it ranges from 0.69 to 0.97 (depending on volume).8,17 Independent of clinical context, readmissions simply may be a less reliable measure of hospital quality. Although no prior studies have examined the reliability of postoperative readmission rates, there is previous evidence from Medicare patients hospitalized for pneumonia, myocardial infarction, or heart failure.18 Press et al found that hospital readmission rankings fluctuated significantly during a 3-year period. In fact, hospitals with the lowest readmission rates in 2009 tended to have the highest increases in readmissions by 2011. The authors attributed their findings primarily to regression to the mean.18 In other words, extreme outliers observed in any one time period will migrate toward average performance as time goes on. Our study adds to the literature by examining the reliability of readmissions in the context of vascular surgery. It is especially timely because the Centers for Medicare and Medicaid Services (CMS) has specifically targeted vascular procedures for expansion of the HRRP.1 After the FY 2013 evaluation, more than 2200 hospitals were penalized, with a total of $500 million. In FY 2013, only medical procedures were targeted and penalties were capped at 1% of the hospital’s operating revenue. However, by FY 2015, the HRRP will expand to include vascular procedures and the maximum cap will triple to 3%, raising aggregate potential penalties to $1 billion. Policy implications. Our findings suggest that readmissions are a suboptimal proxy for hospital quality. In fact, penalizing hospitals for higher-than-expected readmission rates carries the potential for significant harm. First, it inappropriately labels some hospitals as poor performers. In turn, these institutions may divert resources toward addressing a misperceived readmissions problem that is in fact an artifact of random yearly variation. Alternatively, other hospitals with true coordination of care problems
JOURNAL OF VASCULAR SURGERY June 2014
1642 Gonzalez et al
(not captured by readmission data) may be lulled into a false sense of security. One solution for the issues arising from unreliable readmission rates may be to directly measure coordination of care.19 Although coordination of care is frequently cited as essential to improving postoperative outcomes,20,21 hospitals are infrequently assessed along this domain of quality.22 However, there are validated instruments specifically designed to measure coordination of care. For instance, the National Quality Forum, a leading patient safety organization, endorses the Three-Item Care Transition Measure (CTM-3). The CTM-3 evaluates three domains of care coordination: patient understanding of postdischarge self-care, medication management, and use of a patient-centered discharge care plan.23 Whereas other instruments focus on questions asked of patients during the discharge process, the CTM-3 focuses on actions taken by providers in preparation for discharge. The CTM-3 has been shown to discriminate between patients who did versus did not have a subsequent readmission.24,25 Moreover, CMS’s current methodology for quantifying performance under HRRP creates several unintended but perverse economic consequences. Chiefly, hospitals caring for poor and minority patients are most heavily penalized. In one study examining the impact of HRRP on safety-net hospitals, the authors found that safety-net hospitals were more than twice as likely to be penalized as other hospitals.26 In fact, only 20% of safety-net hospitals avoided penalties under HRRP. This situation is particularly problematic in surgery, given that Medicaid and selfinsured patients often generate net revenue losses for hospitals (on a per-operation basis).27 Thus, HRRP may inadvertently reduce scarce resources available for quality improvement at institutions already under the most financial strain. Lack of reliability makes the use of readmissions for setting monetary fines problematic. Yet, this does not mean that longitudinally tracking readmissions is without merit. Readmissions may nonetheless play other roles in improving the quality of care. Though not suitable as a penalty measure, readmissions may be useful as an improvement measure. Limitations. Our study is not without limitations. First, the findings may not be widely generalizable, given that our study population included only Medicare patients ages 65 to 99 years. However, the significant overlap in age range between patients with vascular disease and the feefor-service Medicare population somewhat mitigates this limitation. Second, our risk-adjustment model may be biased by unobserved differences in patient factors. However, the focus of the present study was to assess reliability, an attribute relatively unaffected by an imperfect ability to adjust for patient severity. Furthermore, it is important to consider that we found 30-day all-cause postoperative readmissions an unreliable measure of quality at the hospital level. However, at the patient level, readmissions may be predictable (or even preventable), depending on the clinical context.
CONCLUSIONS In this study, we examined the predictive reliability of readmission rates in vascular surgery. Our major finding, hospital rankings that are based on readmission rates vary from year to year due to chance (or unmeasured variables), suggests that measuring readmissions with the use of administrative data is an unreliable indicator of hospital quality. Although reducing unnecessary rehospitalization is a laudable goal, the lack of reliability raises concerns about the use of this measure for public reporting of hospital quality. From a practical standpoint, the use of an unreliable measure to financially penalize hospitals (eg, under CMS’s HRRP) has at least three unintended consequences. First, hospitals misclassified as low performers may allocate scarce resources toward correcting misperceived readmission problems. Second, the problems with coordination of care problems may remain undetected at hospitals misclassified as good performers. Finally, the policy creates perverse economic incentives that penalize the nation’s most vulnerable hospitals. AUTHOR CONTRIBUTIONS Conception and design: AG, MG, TS, JD Analysis and interpretation: AG, MG, TS, JD Data collection: AG, JD Writing the article: AG, TS, JD Critical revision of the article: AG, TW, JD Final approval of the article: AG, TW, JD Statistical analysis: AG, JD Obtained funding: TW, JD Overall responsibility: AG REFERENCES 1. Medicare Payment Advisory Commission (MedPAC). Report to Congress: Promoting Greater Efficiency in Medicare. Chapter 5. Washington D.C.: MedPAC; 2007. Available at: http://1.usa.gov/ 1lp2gJ9. Accessed February 12, 2014. 2. Hospital readmissions reduction program. 1395ww(q)(1) United States: 42 USC 1395ww(q)(1); 2010, Washington. D.C. Available at: http://1.usa.gov/1m9jwYg. Accessed: February 12, 2014. 3. Expansion of Applicable Conditions. 42 U.S.C. xx1395ww(q)(5)(B) 42 U.S.C. xx1395ww(q)(5)(B); 2010, Washington. D.C. Available at: http://1.usa.gov/1m9jwYg. Accessed: February 12, 2014. 4. Birkmeyer JD, Dimick JB, Staiger DO. Operative mortality and procedure volume as predictors of subsequent hospital performance. Ann Surg 2006;243:411-7. 5. Luft HS, Romano PS. Chance, continuity, and change in hospital mortality rates: coronary artery bypass graft patients in California hospitals, 1983 to 1989. JAMA 1993;270:331-7. 6. Centers for Medicare & Medicaid Services. Readmissions Reduction Program; 2012. Available at: http://go.cms.gov/1eUTfaO. Accessed February 12, 2014. 7. Elixhauser A, Steiner C, Harris D, Coffey RM. Comorbidity measures for use with administrative data. Med Care 1998;36:8-27. 8. Dimick JB, Ghaferi AA, Osborne NH, Ko CY, Hall BL. Reliability adjustment for reporting hospital outcomes. Ann Surg 2012;255:703-7. 9. Osborne NH, Ko CY, Upchurch GR, Dimick JB. The impact of adjusting for reliability on hospital quality rankings in vascular surgery. J Vasc Surg 2011;53:1-5. 10. Quality Net. Measure Methodology Reports: Readmission Measures; 2013. Available at: http://bit.ly/1hrSC9s. Accessed February 12, 2014.
JOURNAL OF VASCULAR SURGERY Volume 59, Number 6
11. Dimick JB, Staiger DO, Birkmeyer JD. Ranking hospitals on surgical mortality: the importance of reliability adjustment. Health Serv Res 2010;45(6 Part 1):1614-29. 12. Kuhan G, Marshall EC, Abidia AF, Chetter IC, McCollum PTA. Bayesian hierarchical approach to comparative audit for carotid surgery. Eur J Vasc Endovasc Surg 2002;24:505-10. 13. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128-38. 14. Adams JL. The Reliability of Provider Profiling: A Tutorial. Rand Corporation; 2009. Available at: http://bit.ly/18FzwJP. Accessed February 12, 2014. 15. Schnur S. Mortality rates in acute myocardial infarction, I: the “normal” yearly variation, and the effect of hospital admission policy. Ann Intern Med 1953;39:1014-7. 16. Dimick JB, Welch HG. The zero mortality paradox in surgery. J Am Coll Surg 2008;206:13-6. 17. Hashmi ZG, Dimick JB, Efron DT, Haut ER, Schneider EB, Zafar SN, et al. Reliability adjustment: a necessity for trauma center ranking and benchmarking. J Trauma Acute Care Surg 2013;75:166-72. 18. Press MJ, Scanlon DP, Ryan AM, Zhu J, Navathe AS, Mittler JN, et al. Limits of readmission rates in measuring hospital quality suggest the need for added metrics. Health Aff 2013;32:1083-91. 19. Brock J, Mitchell J, Irby K, Stevens B, Archibald T, Goroski A, et al. Association between quality improvement for care transitions in communities and rehospitalizations among Medicare beneficiaries. JAMA 2013;309:381-91.
Gonzalez et al 1643
20. Main DS, Cavender TA, Nowels CT, Henderson WG, Fink AS, Khuri SF. Relationship of processes and structures of care in general surgery to postoperative outcomes: a qualitative analysis. J Am Coll Surg 2007;204:1147-56. 21. Feigenbaum P, Neuwirth E, Trowbridge L, Teplitsky S, Barnes CA, Fireman E, et al. Factors contributing to all-cause 30-day readmissions: a structured case series across 18 hospitals. Med Care 2012;50: 599-605. 22. Pham HH, Grossman JM, Cohen G, Bodenheimer T. Hospitalists and care transitions: the divorce of inpatient and outpatient care. Health Aff (Millwood) 2008;27:1315-27. 23. Coleman EA. Care transitions measure frequently asked questions; 2013. p. 1-10. Available at: http://bit.ly/16kI4n7. Accessed February 12, 2014. 24. Shadmi E, Zisberg A, Coleman EA. Translation and validation of the care transition measure into Hebrew and Arabic. Int J Qual Health Care 2009;21:97-102. 25. Coleman EA, Parry C, Chalmers S, Min S. The care transitions intervention: results of a randomized controlled trial. Arch Intern Med 2006;166:1822-8. 26. Joynt KE, Jha AK. Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA 2013;309: 342-3. 27. Eappen S, Lane BH, Rosenberg B, Lipsitz SA, Sadoff D, Matheson D, et al. Relationship between occurrence of surgical complications and hospital finances. JAMA 2013;309:1599-606. Submitted Oct 24, 2013; accepted Dec 19, 2013.
Reliability of hospital readmission rates in vascular surgery Andrew A. Gonzalez, MD, JD, MPH,a,c Micah E. Girotti, MD, MSc,a Terry Shih, MD,a Thomas W. Wakefield, MD,b and Justin B. Dimick, MD, MPH,a Ann Arbor, Mich; and Chicago, Ill Objective: The Center for Medicare and Medicaid Services recently began assessing financial penalties to hospitals with high readmission rates for a narrow set of medical conditions. Because these penalties will be extended to surgical conditions in the near future, we sought to determine whether readmissions are a reliable predictor of hospital performance with vascular surgery. Methods: We examined 4 years of national Medicare claims data from 1576 hospitals on beneficiaries undergoing three common vascular procedures: open or endovascular abdominal aortic aneurysm repair (n [ 81,520) or lower extremity arterial bypass (n [ 57,190). First, we divided our population into two groups on the basis of operative date (2005-2006 and 2007-2008) and generated hospital risk- and reliability-adjusted readmission rates for each time period. We evaluated reliability through the use of the “test-retest” method; highly reliable measures will show little variation in rates over time. Specifically, we evaluated the year-to-year reliability of readmissions by calculating Spearman rank correlation and weighted k tests for readmission rates between the two time periods. Results: The Spearman coefficient between 2005-2006 readmissions rankings and 2007-2008 readmissions rankings was 0.57 (P < .001) and weighted k was 0.42 (P < .001), indicating a moderate correlation. However, only 32% of the variation in hospital readmission rates in 2007-2008 was explained by readmissions during the 2 prior years. There were major reclassifications of hospital rankings between years, with 63% of hospitals migrating among performance quintiles between 2005-2006 and 2007-2008. Conclusions: Risk-adjusted readmission rates for vascular surgery vary substantially year to year; this implies that much of the observed variation in readmission rates is either random or caused by unmeasured factors and not caused by changes in hospital quality that may be captured by administrative data. (J Vasc Surg 2014;59:1638-43.)
Because readmissions are common and costly, many policymakers view reducing readmissions as the rare opportunity to improve quality while decreasing cost. For example, new provisions under the Affordable Care Act target readmissions in an effort to reduce their $12 billion contribution to annual Medicare spending.1 In addition to setting reimbursement, the Act mandates public reporting of readmissions performance on the Hospital Compare website.2 Although these changes have initially affected From the Department of Surgery, Center for Healthcare Outcomes and Policy (CHOP),a and the Section of Vascular Surgery,b University of Michigan, Ann Arbor; and the Department of Surgery, University of Illinois Hospital and Health Sciences System, Chicago.c This study was supported by grant R01 AG039434-03 (J.B.D.) from the National Institute on Aging; K08 HS017765-04 (J.B.D.) from the Agency for Healthcare Research and Quality; and Ruth L. Kirschstein National Research Service Awards T32 HL076123-09 and T32 HL076123-10 from the National Heart, Lung, and Blood Institute (A.A.G.; T.S.; T.W.W.). Author conflict of interest: none. Presented as a poster under the categories of abdominal aortic, outcomes analysis, and peripheral arterial occlusive disease at the 2013 Vascular Annual Meeting of the Society for Vascular Surgery, San Francisco, Calif, May 30-June 1, 2013. Reprint requests: Andrew A. Gonzalez, MD, JD, MPH, Center for Healthcare Outcomes and Policy (CHOP), North Campus Research Complex, 2800 Plymouth Rd, Bldg 16, Rm 100-14, Ann Arbor, MI 48109 (e-mail: [email protected]). The editors and reviewers of this article have no relevant financial relationships to disclose per the JVS policy that requires reviewers to decline review of any manuscript for which they may have a conflict of interest. 0741-5214/$36.00 Copyright Ó 2014 by the Society for Vascular Surgery. http://dx.doi.org/10.1016/j.jvs.2013.12.040
1638
select medical conditions and hip or knee surgery, they will be extended to include vascular surgery by 2015.1,3 Whether risk-adjusted readmission rates are a reliable indicator of quality remains unknown. To be a reliable indicator of quality, a measure should demonstrate stability over time. Thus, hospital rankings that are based on readmissions should vary little from year to year unless there were significant improvements in coordination of care. Prior studies have examined the temporal stability of various outcome measures. For instance, historical rates of surgical mortality can reliably predict future rates for some operations but not others.4,5 Beyond the theoretical implications, the concept of stability over time is of practical significance because the Affordable Care Act’s Hospital Readmissions Reduction Program (HRRP) will use data from 2008 through 2011 to determine pay-forperformance readmission penalties for fiscal year (FY) 2013.6 This study seeks to examine whether hospital readmission rates are persistent year to year or reflect random yearly variation. To investigate this question, we used data on Medicare beneficiaries undergoing abdominal aortic aneurysm repair or lower extremity arterial bypass. With the use of Spearman correlation, we ranked hospitals on the basis of 2005-2006 readmission rates and determined how well they predicted 2007-2008 ranks. Furthermore, we quantified the extent to which 2007-2008 rankings remained similar to 2005-2006 rankings. METHODS Databases, subjects. We used Medicare Provider Analysis and Review (MedPAR) files for beneficiaries
JOURNAL OF VASCULAR SURGERY Volume 59, Number 6
between the ages of 65 and 99 years. MedPAR files contain administrative data from discharge abstracts for all Medicare beneficiaries hospitalized on a fee-for-service basis in acute care facilities. We used International Classification of Diseases, Ninth Revision (ICD-9) procedure codes to identify patients who underwent lower extremity arterial bypass, open abdominal aortic aneurysm repair (OAR), or endovascular aortic repair (EVAR) between 2005 and 2008. For OAR and EVAR, we excluded cases with an admitting diagnosis indicating aneurysm rupture. We selected these three operations, a priori because each has a high incidence of readmission (in excess of 10%). To harmonize our study population with the methodology used by the Centers for Medicare and Medicaid Services, we excluded patients who were discharged against medical advice (n ¼ 157 patients), patients admitted as transfers from other hospitals or facilities (n ¼ 5092), and patients who were nonelective admissions (n ¼ 46,816). Finally, we excluded hospitals that did not perform at least one operation in each year of our study period (n ¼ 745 hospitals). Statistical analysis. We used the hospital as our unit of analysis. Our primary outcome was a composite hospital risk- and reliability-adjusted 30-day readmission rate. We calculated point estimates and confidence intervals for this outcome by a three-step process. First, we used standard logistic regression to calculate an empirically derived risk score. This score was calculated through the use of a continuous variable for age, a dichotomous variable for sex, and “dummy” variables for preexisting medical comorbidities on the basis of Elixhauser’s methods.7 For each hospital, a separate risk score was created for each operation and each year (12 in all). Beyond simplifying the model, a single risk score reduces the likelihood of model nonconvergence. In contrast, the use of individual variables often results in failure of the statistical program to fit an adaptive model to the covariate matrix. This approach has previously been applied to similar data in surgical patients and to the reporting of vascular outcomes.8,9 Specifically, this risk score was generated by means of the logistic postestimation command that predicts the log(odds) of readmission for each patient (-xb option in STATA). We used modeled readmissions in units of log(odds) rather than predicted probability because log(odds) is linear with respect to binary outcomes (eg, readmission). Second, we used a hierarchical mixed-effects logistic regression model that was based on the patient risk score (described above) with a hospital variable as the random effect. The Centers for Medicare and Medicaid Services uses hierarchical logistic regression to calculate hospital readmission rates for the HRRP.10 Hierarchical regression is an advanced statistical technique that leverages empirical Bayes theorem to directly model variation at the hospital level. As a result, hierarchical modeling yields more stable estimates of both coefficients and standard errors.8,11 Prior studies of outcomes after vascular surgery have validated this technique and shown it to reduce statistical noise and provide more robust estimates of hospital quality compared with risk adjustment alone.9,12
Gonzalez et al 1639
Table. Patient characteristics by cohort Variable
2005-2006
2007-2008
Abdominal aortic aneurysm repair Median age (IQR), years Sex (% women)a % Black race % 3þ Comorbiditiesa % Endovasculara Lower extremity arterial bypass Median age (IQR), years Sex (% women)a % Black racea % 3þ Comorbiditiesa % Claudicationa,b % Tissue lossc
33,973 76 (71-80) 24.7 3.8 32 59 24,606 75 (70-81) 45.4 12.6 40 27 43
47,547 75 (70-81) 23.9 3.7 39 69 32,584 75 (70-81) 42.8 11.5 45 30 43
ICD-9, International Classification of Diseases, Ninth Revision; IQR, interquartile range. a Statistically significant at P < .05. b ICD-9 code 440.21 (claudication) in any diagnostic code field. c ICD-9 codes 440.22 (rest pain); 440.23 (ulceration); 440.24 (gangrene).
Third, we generated a composite readmission rate for 2005-2006 and for 2007-2008. For each hospital, we multiplied its procedure-specific readmission rate by its volume for that procedure; summated the resulting products; and divided the result by the hospital’s total volume for all three procedures. This adjusted for differences in volume over time and for differences in the proportion of each operation performed at any individual hospitals. Thus, a hospital that performed many bypasses but few EVARs would have a composite readmission rate closer to its bypass readmission rate. Finally, we ranked hospitals according to risk- and reliability-adjusted 30-day readmission rates during the historical time period (2005-2006) and during the outcome time period (2007-2008). These ranks were used to stratify hospitals into quintiles of readmission for each time period. We used a reclassification matrix to analyze hospital migration among quintiles between the two time periods.13 Correlation was assessed by means of Spearman rank coefficients and weighted k tests. In addition to composite rankings that were based on all operations, we evaluated each operation separately. Sensitivity analysis. To ensure that our results were robust to any anomalies unique to the combination of years forming the historical and outcome time periods, we analyzed reliability and reclassification across three 1-year periods: 2005-2006, 2006-2007, and 2007-2008. RESULTS We evaluated 1576 hospitals performing OAR, EVAR, and lower extremity arterial bypass between 2005 and 2008. The mean adjusted readmission rate was 19.3% (standard deviation, 2.8%; range, 15.5%-23.4%) in 20052006 and 15.8% (standard deviation, 1.9%; range, 13.3%18.6%) in 2007-2008. Overall, 129 hospitals (8.2%) saw a decrease in their readmission rate by at least 5 percentage points from 2005-2006 to 2007-2008. In contrast, only 45 hospitals (2.9%) saw any increase in their readmission rate.
JOURNAL OF VASCULAR SURGERY June 2014
1640 Gonzalez et al
Fig 1. Hospital rankings based on risk- and reliability-adjusted readmission rates in 2005-2006 vs 2007-2008, stratified by operation type. Spearman correlation coefficients and associate P values are listed for each operation. AAA, Abdominal aortic aneurysm; EVAR, endovascular aortic repair.
In evaluating patient characteristics for both year groups, there were statistically significant differences for all demographic factors with the exception of age (Table). However, the use of EVAR was the only clinically meaningful difference between 2005-2006 and 2007-2008 (30% vs 37%; P < .001). The Spearman coefficient for the correlation between 2005-2006 ranks and 2007-2008 ranks was 0.57 (P value for independence of ranks <.001); weighted k ¼ 0.42 (P value for agreement <.001; Fig 1). This indicates a moderate correlation between hospital rankings that were based on 2005-2006 readmission rates and rankings that were based on 2007-2008 rates. Only 32% of the variation in 2007-2008 readmission rates could be explained by hospital performance in 2005-2006 (historical time period). Restated, 68% of the variation was explained by factors other than historical performance. Thus, we observed significant fluctuation in quintile rankings between the two time periods (Fig 2). Overall, 61% of hospitals were reclassified into different quintiles of performance between 2005-2006 and 2007-2008. With the exception of hospitals in the worst quintile (highest readmission rates) for 2005-2007, there are no clinically meaningful differences between the other 80% of hospitals (Fig 3). Our results were similar when examining operations individually (data not included).
Sensitivity analysis. When analyzing changes for each year individually, we found that R2 varied from 0.25 (2006 vs 2007) to 0.41 (2007 vs 2008). DISCUSSION In this study, we assessed the year-to-year variability of hospital readmission rates after major vascular surgery. To draw conclusions about groups (eg, hospitals), it is generally accepted that a measure should have reliability of at least .70.14 We found that readmission rates have suboptimal predictive reliability (.32). This implies that readmissions following major vascular surgery do not reflect a fixed set of attributes capturing a hospital’s ability to provide coordinated transitions of care (ie, optimal discharge practices). From a practical standpoint, this study has important policy implications, challenging whether Medicare’s recent implementation of penalties under the HRRP is going to incentivize hospitals in a productive way. If the penalized hospitals do not have control over the outcome (ie, readmission is due to chance alone), then assessing such penalties will not have the desired effect of reducing readmission rates. Our main finding, that readmission rates have suboptimal reliability, implies that factors other than quality drive readmissions performance. Aside from actual quality improvement (or degradation), the changes in readmission
JOURNAL OF VASCULAR SURGERY Volume 59, Number 6
Gonzalez et al 1641
Fig 2. Comparison of quintile ranking between 2005-2006 and 2007-2008 for 1576 hospitals. Shading intensity represents the magnitude reclassification; (n) represents the number of hospitals in each quintile combination. Overall, 63% of hospitals were reclassified between the 2-year groups.
Fig 3. Risk- and reliability-adjusted hospital readmission rates in 2007-2008, according to quintiles of historical readmission performance in 2005-2006. Both historical and subsequent readmission rates are adjusted for patient characteristics.
rankings that we observed may be explained by chance year-to-year variation or by unmeasured patient factors. Although the problem of chance variation in outcome measures was recognized in the 1950s,15 prior literature around reliability remains sparse. The few existing studies evaluating the reliability of surgical outcomes have focused exclusively on postoperative mortality.4,11 For instance, in the case of coronary artery bypass grafting, elective repair of abdominal aortic aneurysm, and pancreatectomy, historical mortality predicts only one-third to one-half of the variation in subsequent mortality.4 The remaining variation (the majority) is due to chance or statistical “noise.” Akin to our readmission findings, chance variation in mortality rates have led to inconsistent hospital performance ratings across years. For example, Luft et al studied coronary artery bypass grafting in California and found that only a minority of hospitals were consistent low-mortality outliers. Furthermore, not a single hospital was a consistent high-mortality outlier.5 Even among the most extreme outliers (ie, zero-mortality hospitals), there is poor reliability over time. One study examined hospitals with no postoperative deaths for 3 consecutive years. When analyzing data for the subsequent year, the authors found that these zero-mortality hospitals had no difference in
mortality rates compared with other hospitals.16 Yet, as a cross-sectional measure, mortality rate may be fairly reliable, depending on the clinical context. For example, in postcolectomy patients, the reliability of mortality ranges from 0.24 to 0.45 (5th to 95th percentile of volume), but, in trauma patients, it ranges from 0.69 to 0.97 (depending on volume).8,17 Independent of clinical context, readmissions simply may be a less reliable measure of hospital quality. Although no prior studies have examined the reliability of postoperative readmission rates, there is previous evidence from Medicare patients hospitalized for pneumonia, myocardial infarction, or heart failure.18 Press et al found that hospital readmission rankings fluctuated significantly during a 3-year period. In fact, hospitals with the lowest readmission rates in 2009 tended to have the highest increases in readmissions by 2011. The authors attributed their findings primarily to regression to the mean.18 In other words, extreme outliers observed in any one time period will migrate toward average performance as time goes on. Our study adds to the literature by examining the reliability of readmissions in the context of vascular surgery. It is especially timely because the Centers for Medicare and Medicaid Services (CMS) has specifically targeted vascular procedures for expansion of the HRRP.1 After the FY 2013 evaluation, more than 2200 hospitals were penalized, with a total of $500 million. In FY 2013, only medical procedures were targeted and penalties were capped at 1% of the hospital’s operating revenue. However, by FY 2015, the HRRP will expand to include vascular procedures and the maximum cap will triple to 3%, raising aggregate potential penalties to $1 billion. Policy implications. Our findings suggest that readmissions are a suboptimal proxy for hospital quality. In fact, penalizing hospitals for higher-than-expected readmission rates carries the potential for significant harm. First, it inappropriately labels some hospitals as poor performers. In turn, these institutions may divert resources toward addressing a misperceived readmissions problem that is in fact an artifact of random yearly variation. Alternatively, other hospitals with true coordination of care problems
JOURNAL OF VASCULAR SURGERY June 2014
1642 Gonzalez et al
(not captured by readmission data) may be lulled into a false sense of security. One solution for the issues arising from unreliable readmission rates may be to directly measure coordination of care.19 Although coordination of care is frequently cited as essential to improving postoperative outcomes,20,21 hospitals are infrequently assessed along this domain of quality.22 However, there are validated instruments specifically designed to measure coordination of care. For instance, the National Quality Forum, a leading patient safety organization, endorses the Three-Item Care Transition Measure (CTM-3). The CTM-3 evaluates three domains of care coordination: patient understanding of postdischarge self-care, medication management, and use of a patient-centered discharge care plan.23 Whereas other instruments focus on questions asked of patients during the discharge process, the CTM-3 focuses on actions taken by providers in preparation for discharge. The CTM-3 has been shown to discriminate between patients who did versus did not have a subsequent readmission.24,25 Moreover, CMS’s current methodology for quantifying performance under HRRP creates several unintended but perverse economic consequences. Chiefly, hospitals caring for poor and minority patients are most heavily penalized. In one study examining the impact of HRRP on safety-net hospitals, the authors found that safety-net hospitals were more than twice as likely to be penalized as other hospitals.26 In fact, only 20% of safety-net hospitals avoided penalties under HRRP. This situation is particularly problematic in surgery, given that Medicaid and selfinsured patients often generate net revenue losses for hospitals (on a per-operation basis).27 Thus, HRRP may inadvertently reduce scarce resources available for quality improvement at institutions already under the most financial strain. Lack of reliability makes the use of readmissions for setting monetary fines problematic. Yet, this does not mean that longitudinally tracking readmissions is without merit. Readmissions may nonetheless play other roles in improving the quality of care. Though not suitable as a penalty measure, readmissions may be useful as an improvement measure. Limitations. Our study is not without limitations. First, the findings may not be widely generalizable, given that our study population included only Medicare patients ages 65 to 99 years. However, the significant overlap in age range between patients with vascular disease and the feefor-service Medicare population somewhat mitigates this limitation. Second, our risk-adjustment model may be biased by unobserved differences in patient factors. However, the focus of the present study was to assess reliability, an attribute relatively unaffected by an imperfect ability to adjust for patient severity. Furthermore, it is important to consider that we found 30-day all-cause postoperative readmissions an unreliable measure of quality at the hospital level. However, at the patient level, readmissions may be predictable (or even preventable), depending on the clinical context.
CONCLUSIONS In this study, we examined the predictive reliability of readmission rates in vascular surgery. Our major finding, hospital rankings that are based on readmission rates vary from year to year due to chance (or unmeasured variables), suggests that measuring readmissions with the use of administrative data is an unreliable indicator of hospital quality. Although reducing unnecessary rehospitalization is a laudable goal, the lack of reliability raises concerns about the use of this measure for public reporting of hospital quality. From a practical standpoint, the use of an unreliable measure to financially penalize hospitals (eg, under CMS’s HRRP) has at least three unintended consequences. First, hospitals misclassified as low performers may allocate scarce resources toward correcting misperceived readmission problems. Second, the problems with coordination of care problems may remain undetected at hospitals misclassified as good performers. Finally, the policy creates perverse economic incentives that penalize the nation’s most vulnerable hospitals. AUTHOR CONTRIBUTIONS Conception and design: AG, MG, TS, JD Analysis and interpretation: AG, MG, TS, JD Data collection: AG, JD Writing the article: AG, TS, JD Critical revision of the article: AG, TW, JD Final approval of the article: AG, TW, JD Statistical analysis: AG, JD Obtained funding: TW, JD Overall responsibility: AG REFERENCES 1. Medicare Payment Advisory Commission (MedPAC). Report to Congress: Promoting Greater Efficiency in Medicare. Chapter 5. Washington D.C.: MedPAC; 2007. Available at: http://1.usa.gov/ 1lp2gJ9. Accessed February 12, 2014. 2. Hospital readmissions reduction program. 1395ww(q)(1) United States: 42 USC 1395ww(q)(1); 2010, Washington. D.C. Available at: http://1.usa.gov/1m9jwYg. Accessed: February 12, 2014. 3. Expansion of Applicable Conditions. 42 U.S.C. xx1395ww(q)(5)(B) 42 U.S.C. xx1395ww(q)(5)(B); 2010, Washington. D.C. Available at: http://1.usa.gov/1m9jwYg. Accessed: February 12, 2014. 4. Birkmeyer JD, Dimick JB, Staiger DO. Operative mortality and procedure volume as predictors of subsequent hospital performance. Ann Surg 2006;243:411-7. 5. Luft HS, Romano PS. Chance, continuity, and change in hospital mortality rates: coronary artery bypass graft patients in California hospitals, 1983 to 1989. JAMA 1993;270:331-7. 6. Centers for Medicare & Medicaid Services. Readmissions Reduction Program; 2012. Available at: http://go.cms.gov/1eUTfaO. Accessed February 12, 2014. 7. Elixhauser A, Steiner C, Harris D, Coffey RM. Comorbidity measures for use with administrative data. Med Care 1998;36:8-27. 8. Dimick JB, Ghaferi AA, Osborne NH, Ko CY, Hall BL. Reliability adjustment for reporting hospital outcomes. Ann Surg 2012;255:703-7. 9. Osborne NH, Ko CY, Upchurch GR, Dimick JB. The impact of adjusting for reliability on hospital quality rankings in vascular surgery. J Vasc Surg 2011;53:1-5. 10. Quality Net. Measure Methodology Reports: Readmission Measures; 2013. Available at: http://bit.ly/1hrSC9s. Accessed February 12, 2014.
JOURNAL OF VASCULAR SURGERY Volume 59, Number 6
11. Dimick JB, Staiger DO, Birkmeyer JD. Ranking hospitals on surgical mortality: the importance of reliability adjustment. Health Serv Res 2010;45(6 Part 1):1614-29. 12. Kuhan G, Marshall EC, Abidia AF, Chetter IC, McCollum PTA. Bayesian hierarchical approach to comparative audit for carotid surgery. Eur J Vasc Endovasc Surg 2002;24:505-10. 13. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128-38. 14. Adams JL. The Reliability of Provider Profiling: A Tutorial. Rand Corporation; 2009. Available at: http://bit.ly/18FzwJP. Accessed February 12, 2014. 15. Schnur S. Mortality rates in acute myocardial infarction, I: the “normal” yearly variation, and the effect of hospital admission policy. Ann Intern Med 1953;39:1014-7. 16. Dimick JB, Welch HG. The zero mortality paradox in surgery. J Am Coll Surg 2008;206:13-6. 17. Hashmi ZG, Dimick JB, Efron DT, Haut ER, Schneider EB, Zafar SN, et al. Reliability adjustment: a necessity for trauma center ranking and benchmarking. J Trauma Acute Care Surg 2013;75:166-72. 18. Press MJ, Scanlon DP, Ryan AM, Zhu J, Navathe AS, Mittler JN, et al. Limits of readmission rates in measuring hospital quality suggest the need for added metrics. Health Aff 2013;32:1083-91. 19. Brock J, Mitchell J, Irby K, Stevens B, Archibald T, Goroski A, et al. Association between quality improvement for care transitions in communities and rehospitalizations among Medicare beneficiaries. JAMA 2013;309:381-91.
Gonzalez et al 1643
20. Main DS, Cavender TA, Nowels CT, Henderson WG, Fink AS, Khuri SF. Relationship of processes and structures of care in general surgery to postoperative outcomes: a qualitative analysis. J Am Coll Surg 2007;204:1147-56. 21. Feigenbaum P, Neuwirth E, Trowbridge L, Teplitsky S, Barnes CA, Fireman E, et al. Factors contributing to all-cause 30-day readmissions: a structured case series across 18 hospitals. Med Care 2012;50: 599-605. 22. Pham HH, Grossman JM, Cohen G, Bodenheimer T. Hospitalists and care transitions: the divorce of inpatient and outpatient care. Health Aff (Millwood) 2008;27:1315-27. 23. Coleman EA. Care transitions measure frequently asked questions; 2013. p. 1-10. Available at: http://bit.ly/16kI4n7. Accessed February 12, 2014. 24. Shadmi E, Zisberg A, Coleman EA. Translation and validation of the care transition measure into Hebrew and Arabic. Int J Qual Health Care 2009;21:97-102. 25. Coleman EA, Parry C, Chalmers S, Min S. The care transitions intervention: results of a randomized controlled trial. Arch Intern Med 2006;166:1822-8. 26. Joynt KE, Jha AK. Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA 2013;309: 342-3. 27. Eappen S, Lane BH, Rosenberg B, Lipsitz SA, Sadoff D, Matheson D, et al. Relationship between occurrence of surgical complications and hospital finances. JAMA 2013;309:1599-606. Submitted Oct 24, 2013; accepted Dec 19, 2013.