- Email: [email protected]
Measuring heart failure care by 30-day readmission: Rethinking the quality of outcome measures
Editorials
Measuring heart failure care by 30-day readmission: Rethinking the quality of outcome measures Marco D. Huesch, MBBS, PhD, a,b,c Michael K. Ong, MD, PhD, d and Gregg C. Fonarow, MD e,f Los Angeles, CA; and Durham, NC
The Centers for Medicare and Medicaid Services (CMS) has operationalized hospital quality of care outcomes through publicly reported 30-day risk-standardized mortality and readmission rates and associated penalties for higher-than-expected rates. The objective of the measure is to motivate hospitals to improve the quality of care they provide patients hospitalized for heart failure (HF) and other conditions such as acute myocardial infarction, pneumonia, with expected future expansion to other conditions. Many studies have looked into these measures and believe them to be well constructed. 1,2 Yet, others have questioned or raised concerns with the readmission measure. 3–7 If these measures misrepresent quality of care, then incentive and disincentives will be misapplied, clinicians and their patients may suffer, and the ultimate objective of improved and more homogenous high quality of care will not be met. In this editorial both quality measures are assessed for HF, and the validity of the readmission measure as a valid measure of quality of care is investigated. No extramural funding was used to support this work. The authors are solely responsible for the design and conduct of this study, all study analyses, the drafting and editing of the manuscript, and its final contents.
Statistical estimation of the measures CMS continues to refine the 2 risk-standardized measures, but a number of technical issues are known.
From the aUSC Sol Price School of Public Policy, Schaeffer Center for Health Policy and Economics, Los Angeles, CA, bDepartment of Community & Family Medicine, Duke University School of Medicine, Durham, NC, cDuke Fuqua School of Business, Health Sector Management Area, Durham, NC, dDivision of General Internal Medicine and Health Services Research, David Geffen School of Medicine at the University of California, Los Angeles, CA, eDivision of Cardiology, David Geffen School of Medicine at the University of California, Los Angeles, Los Angeles, CA, and fAhmanson-UCLA Cardiomyopathy Center, Ronald Reagan UCLA Medical Center, Los Angeles, CA. Louise Pilote, MD, MPH, PhD, served as guest editor for this article. Submitted June 1, 2013; accepted July 23, 2013. Reprint requests: Marco D. Huesch, MBBS, PhD, 3335 S. Figueroa St, USC Gateway Unit A, Los Angeles, CA 90089–7273. E-mail: [email protected] 0002-8703/$ - see front matter © 2013, Mosby, Inc. All rights reserved. http://dx.doi.org/10.1016/j.ahj.2013.07.026
Hospital Compare's mortality estimator has already been found to be less accurate than alternative measures. 8 The Bayesian mortality estimation methodology may also unjustifiably 'shrink' the estimates of mortality for smaller hospitals towards the grand mean, thereby ignoring the well-known hospital volume-mortality relationship. 9 Given that there are many smaller hospitals, and fewer very large hospitals, this shrinkage has the effect of significantly underestimating the “true” mortality of smaller hospitals, and slightly overestimating that of the larger hospitals. This tends to reduce the variation between hospitals. Another smaller bias towards the null of finding no differences in mortality between hospitals is the exclusion criteria Hospital Compare uses. For all the discharges in a calendar year, CMS randomly chooses only one hospitalization per year per patient, discarding a little less than 5% of all discharges. 10 This must bias mortality downwards. Consider a patient with 2 hospitalizations per year. If the first is randomly chosen, then it is clear that the patient cannot have died in that admission.
Challenge in HF quality measurement: why is there little association between short-term mortality and readmission rates? A recent analysis linking Medicare fee for service claims with patients in the American Heart Association's Get With The Guidelines–Heart Failure registry examined the association between short-term mortality and rehospitalization rates. There was no statistically significant difference in the odds of 30-day post discharge mortality in rehospitalization quartiles, compared to the quartile with the highest readmission rate. 11 The largest national study of fee for service Medicare patients similarly found only a weak negative association between RSRR and RSMR for HF (Pearson correlation −0.17, P b .05). 1 The absence of a positive association is puzzling. Good HF care should improve both measures, while the competing risks of mortality and readmission should cause them to vary inversely.
The impact of high quality HF care The face validity of these two hospital measures requires them to both be associated with high quality
American Heart Journal October 2013
606 Huesch, Ong, and Fonarow
care. This suggests that they should be positively correlated, after correcting for differences in patient risk. Hospitals providing care that results in better 30 day risk standardized mortality (RSMR) should also perform very well on 30 day risk standardized rehospitalization (RSRR) and conversely. It is possible that the type of HF care that lowers readmission risk has little impact on mortality or vice versa. 1 But HF care process and disease management approaches have in general shown favorable impact on both mortality and rehospitalization. 12–15 Such improvements appear indeed to have led to a convergence to the “no different than the US national rate” for risk-standardized readmission as well as mortality according to CMS measures. 16 It is not clear why a positive relationship between these quality measures is not seen.
The impact of competing risks On the other hand the 2 risk-standardized measures should be negatively correlated due to their competing risks nature. 6,7,17,18 A patient who dies can no longer be readmitted (biasing readmission down). In HF, about 7% of admissions lead to death between discharge and the end of the readmission window. 19 In over half of those cases, the patient died without being readmitted, 19 potentially producing a competing risks bias. Even when deaths are no longer counted, after 30 days postadmission, they may still censor readmission at the tail end of the 30 day post-discharge interval (Fig. 1). Conversely, a patient who is readmitted may have death prevented through targeted procedures in hospital (biasing mortality down). Around one sixth of all HF 30day readmissions occur just in the final 5 days of the readmission window, 20 hence still within the mortality measure's window, suggesting a potential for competing risk bias.
Simulating the impact of high quality care and competing risks We examined this potential for competing risks to affect the two hospital-level measures in a simple simulation model calibrated to typical HF readmission and mortality values. For each condition, a set of 4,000 hospitals was created, each with 100 patients. These models assumed a stylized 5 day admission during which death can occur, followed by 30 days during which death and/or readmission can occur. The models specified different data generating processes for the realization of a readmission event or a mortality event. In one scenario, hospital efforts to improve readmission do not impact mortality and vice versa. Accordingly these two events were modeled as occurring independently. In a different scenario, any hospital efforts improve both readmission
Figure 1 Admission
Discharge
30 day mortality measure Days
30 day readmission measure 3 3 3 3 3 3 0 1 2 3 4 5
1 2 3 4 5 6
No competing risk
Competing risk of mortality and readmission
No competing risk
Quality measures—schematic view of mortality and readmission measures over their intervals. Presence of competing risks is illustrated for typical cardiovascular disease hospitalizations.
and mortality, and the two events were modeled as occurring with positive covariance (see online Appendix Supplemental Material for Review). Patients eligible for inclusion in the mortality measure comprised a different cohort than patients eligible for inclusion in readmission measure, in line with CMS algorithms. Patients who experienced a simulated mortality event during the index admission were not eligible to experience a readmission event subsequently. The model accounted for subsequent competing risks between death and rehospitalization as follows. Patients who died in the period 5–30 days after admission (ie, in the last 25 days of the 30 day mortality measure interval) were not able to be readmitted in the later period 30–35 days after admission (ie, in the last 5 days of the 30-day readmission measure interval). Conservatively, we did not insist that death within the period 5–30 days after admission censored rehospitalization within that period, although a more complex model could have allowed such additional competing risks. Similarly conservatively, we did not insist that rehospitalization sometimes censored death, although the main purpose of rehospitalization is clearly to avoid or delay death. We hypothesized that at the hospital level there would be an inverse relationship between readmission and mortality in the absence of hospital initiatives to improve on both measures of care quality. The presence of such hypothesized initiatives could contribute to a net effect of no observed association between RSRR and RSMR in the simulations. Results of this simple simulation were consistent with these hypotheses (Fig. 2). A modest but strongly statistically significant negative association was found between the readmission and mortality measures (Pearson correlation: −0.11, P b .001). However, when hospital initiatives were simulated that improved both the underlying readmission and mortality processes, the
American Heart Journal Volume 166, Number 4
Huesch, Ong, and Fonarow 607
Figure 2
A
.1
.2
RSRR
.3
.4
Lowess smoother
0
.05
.1
.15
.2
RSMR bandwidth = .8
B
.3 .1
.2
RSRR
.4
Lowess smoother
0
.05
.1
RSMR
.15
.2
bandwidth = .8
A, When rehospitalization and mortality are actually independent, competing risk bias yields an inverse relationship between hospitallevel RSRR and RSMR in a simulation of HF. B, When rehospitalization and mortality are both positively affected by simulated hospital quality initiatives, the effect of competing risks is masked, resulting in a lack of apparent association between hospital-level RSRR and RSMR in a simulation of HF.
prior negative association was masked and essentially disappeared (Pearson correlation: +0.04, P = .02). It is important to point out that the competing risks remain, but are outweighed by the simulated hospital-level initiatives. These results are consistent with the recent findings of little correlation between readmission and mortality for HF. 1,11
Unobserved index admission severity Currently quality is measured only at the hospital level, conditional on at least an index admission. 21 This event is potentially endogenous and thus under the control of the hospital. Admission criteria for HF are poorly standard-
ized. To the extent that the decision to admit patients varies systematically between hospitals, hospital performance under these measures will be biased. Those hospitals with a lower threshold for admission may admit healthier patients merely as a function of spare capacity. 22,23 Such patients may do better on mortality or rehospitalization measures, pulling up the average performance of their hospital. Other hospitals with a higher threshold for admission would experience the opposite effect. Such potential unobserved confounding is consistent with evidence that hospitals that do well on a measure for one condition do well on that measure for the other conditions as well. 2 We replicated this using Hospital Compare data (Fig. 3), and as previous research has shown, 2 a strong association was observed between a hospital's readmission rate for acute myocardial infarction and for HF (Pearson correlation: 0.40, P b .001). Prior research also suggests that variation in hospital capacity is associated with hospitalization rates, although not with mortality. 24 Similarly, variation in rehospitalization rates is strongly associated with variation in the original index hospitalization rates. Regionally such correlations have been recently estimated at more than 50% for HF. 25 If readmission was a function of regional variations in capacity and admission thresholds, then readmission does not accurately or directly reflect hospital quality of care. These are all threats to the construct validity of the readmission measure. Selection biases introduced by hospital-level admission thresholds could potentially be addressed by consideration of the full set of presenting patients, including those presenting to the Emergency Department but not admitted; those boarded, observed, and discharged; or those “admitted to observation” but not formally admitted. Given the clear and perverse incentives with current rehospitalization measures excluding out-ofhospital deaths makes understanding such selection biases better even more important.
Unobserved risks that influence readmission Ideally, hospitals should be able to intervene on select patients identified at highest risk of readmission. Unfortunately, the models that estimate readmission risk are poor in this regard, and it seems very difficult to stratify patients based on risk. 26 Models that seek to predict readmission in HF tend to have only modest (c-indices of 0.55-0.61) discrimination for this event. 27 The CMS model for HF readmission is at the top end of this range at 0.61. 19 To put this in perspective, a c-index of 0.5 represents no better ability than a coin toss to distinguish a case from a control based on estimated risk alone. Clearly, other factors such as social support, site
American Heart Journal October 2013
608 Huesch, Ong, and Fonarow
Figure 3
15
20
HF 30 day RSRR
25
30
Readmission rates for AMI vs HF
20
25
30
35
HF 30 day RSRR Readmission rates for AMI and HF—Hospital Compare data showing hospital-level association between 30 day risk-adjusted rehospitalization rates for AMI and HF. Pearson correlation: 0.40 (P b .001).
and socio-economic status help to determine readmission risk apart from patient and hospital covariates. 4 For example, state-level income inequality was found to be significantly associated with 30 day rehospitalization for HF, although not with mortality. 5 If these readmission models do not incorporate necessary content beyond demographic and comorbidity data, then both their content and predictive validity may be threatened. 28 The lack of correlation between hospital's 30-day readmission and 30-day mortality rates could therefore also indicate that the 30-day risk-standardized readmission rates are substantially misclassifying hospital performance. Those hospitals providing higher care quality resulting in lower risk-standardized mortality may also have had lower 30-day rehospitalization rates had models with better discrimination been applied. While the impact of such reclassification has previously been noted at the individual hospital level, 29 their possible effect on the relationship between readmission and mortality has not yet been investigated in HF. We note that empirical analyses of such classification biases among hospitals treating stroke patients in the Get With The Guidelines program found substantial effects on hospital performance rankings. 29
Conclusions Measuring, reporting, profiling and incenting hospitals on meaningful and valid quality metrics is an uncontro-
versial objective. Yet issues with the CMS readmission measure and its interaction with the mortality measure raise concerns about the validity of current approaches. Given the importance, use for accountability, and basis for substantial financial penalties of 30 day risk standardized outcome measures, it is vital that they continue to be scrutinized. 30 Valid measures will allow payors to reward objective excellence, and will incent providers to improve performance with effective clinical strategies. Patients and their referring physicians would have greater confidence in Hospital Compare and rely on the results to seek better categorized hospitals. Our scrutiny of the risk-standardized measures that CMS uses reveals some concerns as to their validity, especially that of the rehospitalization measure. As a result, it is possible that hospitals may fail to undertake appropriate actions or misjudge the likely success of recommended actions. Worse, the current structural focus on rehospitalization implies that out-of-hospital death is an acceptable outcome. 7 The resulting financial disincentive to readmit a deteriorating patient before the 30-day post discharge window is very troublesome for hospitals and clinicians determined to provide patient-centered excellent care. Other approaches to measuring and incenting quality in cardiovascular care may be needed to better operationalize the measurement and achievement of hospital quality of care. Our view is that healthcare reform has set into place a number of important changes in payment policy that may allow such better operationalization. Chief among these are the bundled payments for an episode of care (EOC) comprising care delivered in-hospital during the index admission and in or out of the hospital during the ensuing 30 days. Economically, the total number of inpatient days in this window will be the largest driver of hospital costs. Conceptually, rather than dichotomizing readmission, aggregating readmitted days may better reflect total resource use and hospital management objectives vis-àvis bundled payments. Facing a fixed reimbursement for an initial HF EOC, hospitals and their Accountable Care Organization delivery partners will have a clear incentive to better manage and reduce EOC inpatient days. 11 Partly, they will see opportunities for better, more efficient resource use during the index admission. Partly, too, they will respond to opportunities to better coordinate care with community-based partners who deliver high-quality, efficient outpatient care after discharge in a way that minimizes readmissions. We thus see the policy decision of payment reform effectively decentralizing the problem to the hospital level. Strategically, aggregated inpatient days in a 30-day EOC may thus better serve as a national hospital-level quality improvement objective. The hospital, in this future view, would have responsibility for creating the quality initiatives that provide more efficient care. This
American Heart Journal Volume 166, Number 4
EOC measure would be operationalized as follows. Rather than reporting, rating and either penalizing or rewarding hospitals based on risk standardized rehospitalization rates, risk standardized EOC days would be reported and used for the penalties or rewards. In a recent study, better EOC performance (ie, lower total EOC inpatient days) was associated with better 30day survival in Get With The Guidelines-HF patients. 11 However, to ensure that perverse incentives do not arise towards the rationing of care and towards out of hospital death, other risk-adjusted metrics focusing on mortality are likely to continue to be needed to balance efficiency and outcome objectives. 11 For example, consideration of a composite of early mortality and/or readmission events, or focusing on the complement of mortality (eg, such as total days alive, or composite metrics such as total days alive out of hospital) may be more fruitful. Combinations of such new and alternative measures may make more sense, reflect the underlying construct of quality in a more accurate way, and incorporate the necessary content in order to be able to predict quality better. In the meantime, incenting hospitals on riskstandardized readmission may be sub-optimal and bears re-examination.
Disclosures Dr Huesch reports support from grants by AHRQ and Lockheed Martin and is consulting to IOM and Precision Health Economics. Dr Ong reports support from grants by AHRQ and NIH and is consulting to IOM and the Public Health Institute. Dr Fonarow reports support from grants by GlaxoSmithKline, NIH, and AHRQ; is serving on speakers' bureaus for Boston Scientific/Guidant, GlaxoSmithKline, Medtronic, Merck, Novartis, Pfizer, and St Jude Medical; and is consulting to Amgen, Gambro, GlaxoSmithKline, Medtronic, Merck, Novartis, Pfizer, Relypsa, Scios, St Jude Medical, Medicines Company, Johnson and Johnson, and Takeda. He holds the Eliot Corday Chair of Cardiovascular Medicine at UCLA and is also supported by the Ahmanson Foundation.
References 1. Krumholz HM, Lin Z, Keenan PS, et al. Relationship between hospital readmission and mortality rates for patients hospitalized with acute myocardial infarction, heart failure, or pneumonia. JAMA 2013; 309:587-93. 2. Horwitz LI, Wang Y, Desai MM, et al. Correlations among riskstandardized mortality rates and among risk-standardized readmission rates within hospitals. J Hosp Med 2012;7:690-6. 3. Joynt KE, Jha AK. A path forward on Medicare readmissions. N Eng J Med 2013, http://dx.doi.org/10.1056/NEJMp1300122. 4. Joynt JE, Orav EJ, Jha AK. Thirty-day readmission rates for Medicare beneficiaries by race and site of care. JAMA 2011;305: 675-81.
Huesch, Ong, and Fonarow 609
5. Lindenauer PK, Lagu T, Rothberg MB, et al. Income inequality and 30 day outcomes after acute myocardial infarction, heart failure, and pneumonia: retrospective cohort study. BMJ 2013;346:f521. 6. Gheorghiade M, Vaduganathan M, Fonarow GC, et al. Rehospitalization for heart failure: problems and perspectives. J Am Coll Cardiol 2013;61:391-403. 7. Konstam MA. Heart failure in the lifetime of Musca domestica (the common housefly). JACC: Heart Failure 2013, http: //dx.doi.org/10.1016/j.jchf.2013.02.003. 8. Ryan A, Burgess J, Strawderman R, et al. What is the best way to estimate hospital quality outcomes? A simulation approach. Health Serv Res 2012, http://dx.doi.org/10.1111/j.1475-6773.2012. 01382.x. 9. Silber JH, Rosenbaum PR, Brachet TJ, et al. The Hospital Compare mortality model and the volume-outcome relationship. Health Serv Res 2010;45:1148-67. 10. QualityNet. 2012 Measures maintenance technical report: acute myocardial infarction, heart failure, and pneumonia 30-day risk-standardized mortality measures. http://qualitynet.org/dcs/ ContentServer?c=Page&pagename=QnetPublic% 2FPage%2FQnetTier4&cid=1163010421830. 11. Kociol RD, Liang L, Hernandez AF, et al. Are we targeting the right metric for heart failure? Comparison of hospital 30-day readmission rates and total episode of care inpatient days. Am Heart J 2013;165: 987-994.e1. 12. Naylor MD, Brooten DA, Campbell RL, et al. Transitional care of older adults hospitalized with heart failure: a randomized, controlled trial. J Am Geriatr Soc 2004;52:675-84. 13. Rich MW, Beckham V, Wittenberg C, et al. A multidisciplinary intervention to prevent the readmission of elderly patients with congestive heart failure. N Engl J Med 1995;333:1190-5. 14. Lappé JM, Muhlestein JB, Lappé DL, et al. Improvements in 1-year cardiovascular clinical outcomes associated with a hospital-based discharge medication program. Ann Intern Med 2004;141:446-53. 15. Phillips CO, Wright SM, Kern DE, et al. Comprehensive discharge planning with postdischarge support for older patients with congestive heart failure: a meta-analysis. JAMA 2004;291:1358-67. 16. Bernheim SM, Grady JN, Lin Z, et al. National patterns of riskstandardized mortality and readmission for acute myocardial infarction and heart failure: update on publicly reported outcomes measures based on the 2010 release. Circ Cardiovasc Qual Outcomes 2010;3:459-67. 17. Rothman KJ, Greenland S. Cohort studies. In: Rothman KJ, Greenland S, eds. Modern epidemiology. 2nd ed. Lippincott-Raven; 1998. p. 79-91. 18. Gorodeski EZ, Starling RC, Blackstone EH. Are all readmissions bad readmissions? N Engl J Med 2010;363:297-8. 19. Keenan PS, Normand S-LT, Lin Z, et al. An administrative claims measure suitable for profiling hospital performance on the basis of 30-day all-cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes 2008;1:29-37. 20. Dharmarajan K, Hsieh AF, Lin Z, et al. Diagnoses and timing of 30-day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA 2013; 309:355-63. 21. Huesch MD. Payment policy by measurement of health care spending and outcomes. JAMA 2010;303:2405-6. 22. Wennberg JE, Freeman JL, Shelton RM, et al. Hospital use and mortality among Medicare beneficiaries in Boston and New Haven. N Engl J Med 1989;321:1168-73. 23. Wennberg JE, Fisher ES, Goodman DC, et al. Dartmouth atlas of health care: tracking the care of patients with severe chronic illness.
610 Huesch, Ong, and Fonarow
Lebanon, NH: Dartmouth Institute for Health Policy and Clinical Practice. 2008. 24. Fisher ES, Wennberg JE, Stukel TA, et al. Associations among hospital capacity, utilization, and mortality of US Medicare beneficiaries, controlling for sociodemographic factors. Health Serv Res 2000;34:1351-62. 25. Epstein AM, Jha AK, Orav EJ. The relationship between hospital admission rates and rehospitalizations. N Engl J Med 2011;365:2287-95. 26. Ross JS, Mulvey GK, Stauffer B, et al. Statistical models and patient predictors of readmission for heart failure: a systematic review. Arch Intern Med 2008;168:1371-86.
American Heart Journal October 2013
27. Au AG, McAlister FA, Bakal JA, et al. Predicting the risk of unplanned readmission or death within 30 days of discharge after a heart failure hospitalization. Am Heart J 2012;164:365-72. 28. Fonarow GC, Peterson ED. Heart failure performance measures and outcomes: real or illusory gains. JAMA 2009;302:792-4. 29. Fonarow GC, Pan W, Saver JL, et al. Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity. JAMA 2012;308: 257-64. 30. Krumholz HM, Lin Z, Normand ST. Measuring hospital clinical outcomes. BMJ 2013;346:f620.
American Heart Journal Volume 166, Number 4
Appendix Methods We report additional detail on the approaches and simulation code underlying the figures in the text. Re Figure 2 We performed a competing risk simulation. We simplify the data generating processes (DGP) for readmission and mortality in the following crude but reasonably realistic way: • Days 1-5: the only event is death, which happens or doesn't in this period regardless of which day. Death is modeled as having occurred in this period through realization of a uniformly distributed random variate. Death in this period censors subsequent readmission. • Days 5-30: events are death, death and readmission, readmission only, or neither. These events only happen if there is no death in the first period above. These events happen or don't in this period regardless of which day. Death and readmission are modeled as a DGP in which they are jointly determined by a bivariate standard normal distribution with usermodified covariance. Our figures show 0 or +0.5 correlation respectively. The former implies that hospital initiatives do not affect both measures, the latter implies hospital-level initiatives can affect both events. If a random normal variate is above the z value corresponding to the estimated event rate for that measure, then the event is modeled as occurring. ○ Even if death occurs in this period, readmission is not barred from happening in this period. ○ Similarly if readmission happens, there is no bar on death happening. ○ Conservatively, the competing risk bias arises only because death in a period censors readmission in a subsequent period. • Days 30-35: analogous to above. Events here happen only if there is no death in either of the above periods. There is no positive or negative covariance allowed in this separate bivariate standard normal distribution which generates death and or readmission. We did not include dependence in this DGP because the temporal distance from discharge seemed too large to induce dependence (ie, hospital-level initiatives that affect both events). • Risk-standardized mortality is aggregated as death events in the first period, days 1-5, and the second period, days 5-30. Risk-standardized readmission is aggregated as readmission events in the second period, days 5-30, and the third period, days 30-35. • Limitations of this approach include the oversimplification of dividing the relevant time into just three periods instead of individual days. A conservative bias arises from restricting the effect of death to only be on subsequent periods. This tends to underestimate
Huesch, Ong, and Fonarow 610.e1
competing risk bias. For example, a death on day 11 would censor any readmissions on days 12-35 post admission (in the real world), not just readmission between days 30 and 35 (as in our model). Another conservative bias is that we did not model the fact that readmission is likely to censor some deaths. That is, being readmitted is likely to prolong life for at least some patients, and avoid a death within the period of interest. This again underestimates the competing risk bias. A related countervailing bias is that we did not model dependence in the final period, reducing the potential impact of hospital-level initiatives to continue to reduce both events. It is not clear to what extent these biases affect the simulation results; however, we believe the simulation provides a useful and conservative illustration of the impact of competing risk bias on observed correlations between measures, and a similarly useful illustration of the impact of positively correlated events due to hospital-level initiatives that affect readmission and mortality simultaneously. Our STATA code is given below to facilitate reproduction of our approach: program readmitmortHF, rclass version 12 drop _all set obs 100 gen x = runiform() gen Mort_OnAdmission = cond(xb= 0.04,1,0) * now code for 5 to 30 day post admission interval * now drawing jointly normal distributions with specified covariance matrix matrix C = (1, 0 \ 0, 1) * initially specifying zero covariance between readmission and mortality, will flex later drawnorm a b, corr (C) * jointly normal readmission and mortality gen p2 = 1- 0.07 * what z value should the first normal variate take so that between days 5 and 30, mortality is 7%… gen z2 = invnormal(p2) * to force mortality realization within the ~7% probability required gen Mort_Between5and30 = cond(Mort_OnAdmission==0 & a N= z2,1,0) * can't die between days 5 and 30 if already died during admission * now for the readmission event gen p3 = 1- 0.15 * since between days 5, 30, readmission is 15%… gen z3 = invnormal(p3) gen Admit_Between5and30 = cond(Mort_OnAdmission==0 & b N= z3,1,0) * to force readmission realization within the ~15% probability required * if we specify a positive value for the covariance in C, higher chance both events occur
610.e2 Huesch, Ong, and Fonarow
* but can't get admitted between days 5 and 30 if already died during admission * Can obviously get admitted and still die here. Published heart failure data (Keenan et al) has * about 3% of admissions both die after discharge and get readmitted * now analogous code for the 30-35 day interval matrix D = (1, 0 \ 0, 1) * Will want to use a lower value for covariance here than before, seeing as these * are so far temporally from discharge drawnorm c d, corr (D) gen p4 = 1- 0.01 * between days 30 and 35, mortality is 1% (my estimate), but no longer counted in 30 day mortality. Sstill need to account for it given the competing risks set up gen z4 = invnormal(p4) gen Mort_Between30and35 = cond(Mort_OnAdmission==0 & Mort_Between5and30==0 & c N= z4,1,0) * can't die between days 30 and 35 if already died earlier * now for the readmission event gen p5 = 1- 0.10 * between days 30 and 35, readmission is 10% … gen z5 = invnormal(p5) gen Admit_Between30and35 = cond(Mort_OnAdmission==0 & Mort_Between5and30==0 & d N= z5,1,0) * can't get readmitted later if already died earlier
American Heart Journal October 2013
* now generate hospital-level summary of the outcomes of the 100 patients in this run * first generate 30 day mortality between 0 and 30 days summarize Mort_OnAdmission return scalar Mort_1to5 = r(sum)/100 summarize Mort_Between5and30 return scalar Mort_5to30 = r(sum)/100 * next generate 30 day readmission between 5 and 35 days summarize Admit_Between5and30 return scalar Admit_5to30 = r(sum)/100 summarize Admit_Between30and35 return scalar Admit_30to35 = r(sum)/100 end set seed 1234 * Assume 4,000 hospitals simulate, reps(4000) nodots saving(results.dta, replace): readmitmortHF gen RSMR = Mort_1to5 + Mort_5to30 gen RSRR = Admit_5to30 + Admit_30to35 pwcorr RSMR RSRR, sig lowess RSRR RSMR Re Figure 3 We used risk-standardized mortality and risk-standardized readmission rates for the 4776 US hospitals reported by Hospital Compare. Data source is the official Centers for Medicare and Medicaid Services website https://data. medicare.gov/. We added a lowess tricubic spline smoothed local regression line and computed Pearson correlations.
Measuring heart failure care by 30-day readmission: Rethinking the quality of outcome measures Marco D. Huesch, MBBS, PhD, a,b,c Michael K. Ong, MD, PhD, d and Gregg C. Fonarow, MD e,f Los Angeles, CA; and Durham, NC
The Centers for Medicare and Medicaid Services (CMS) has operationalized hospital quality of care outcomes through publicly reported 30-day risk-standardized mortality and readmission rates and associated penalties for higher-than-expected rates. The objective of the measure is to motivate hospitals to improve the quality of care they provide patients hospitalized for heart failure (HF) and other conditions such as acute myocardial infarction, pneumonia, with expected future expansion to other conditions. Many studies have looked into these measures and believe them to be well constructed. 1,2 Yet, others have questioned or raised concerns with the readmission measure. 3–7 If these measures misrepresent quality of care, then incentive and disincentives will be misapplied, clinicians and their patients may suffer, and the ultimate objective of improved and more homogenous high quality of care will not be met. In this editorial both quality measures are assessed for HF, and the validity of the readmission measure as a valid measure of quality of care is investigated. No extramural funding was used to support this work. The authors are solely responsible for the design and conduct of this study, all study analyses, the drafting and editing of the manuscript, and its final contents.
Statistical estimation of the measures CMS continues to refine the 2 risk-standardized measures, but a number of technical issues are known.
From the aUSC Sol Price School of Public Policy, Schaeffer Center for Health Policy and Economics, Los Angeles, CA, bDepartment of Community & Family Medicine, Duke University School of Medicine, Durham, NC, cDuke Fuqua School of Business, Health Sector Management Area, Durham, NC, dDivision of General Internal Medicine and Health Services Research, David Geffen School of Medicine at the University of California, Los Angeles, CA, eDivision of Cardiology, David Geffen School of Medicine at the University of California, Los Angeles, Los Angeles, CA, and fAhmanson-UCLA Cardiomyopathy Center, Ronald Reagan UCLA Medical Center, Los Angeles, CA. Louise Pilote, MD, MPH, PhD, served as guest editor for this article. Submitted June 1, 2013; accepted July 23, 2013. Reprint requests: Marco D. Huesch, MBBS, PhD, 3335 S. Figueroa St, USC Gateway Unit A, Los Angeles, CA 90089–7273. E-mail: [email protected] 0002-8703/$ - see front matter © 2013, Mosby, Inc. All rights reserved. http://dx.doi.org/10.1016/j.ahj.2013.07.026
Hospital Compare's mortality estimator has already been found to be less accurate than alternative measures. 8 The Bayesian mortality estimation methodology may also unjustifiably 'shrink' the estimates of mortality for smaller hospitals towards the grand mean, thereby ignoring the well-known hospital volume-mortality relationship. 9 Given that there are many smaller hospitals, and fewer very large hospitals, this shrinkage has the effect of significantly underestimating the “true” mortality of smaller hospitals, and slightly overestimating that of the larger hospitals. This tends to reduce the variation between hospitals. Another smaller bias towards the null of finding no differences in mortality between hospitals is the exclusion criteria Hospital Compare uses. For all the discharges in a calendar year, CMS randomly chooses only one hospitalization per year per patient, discarding a little less than 5% of all discharges. 10 This must bias mortality downwards. Consider a patient with 2 hospitalizations per year. If the first is randomly chosen, then it is clear that the patient cannot have died in that admission.
Challenge in HF quality measurement: why is there little association between short-term mortality and readmission rates? A recent analysis linking Medicare fee for service claims with patients in the American Heart Association's Get With The Guidelines–Heart Failure registry examined the association between short-term mortality and rehospitalization rates. There was no statistically significant difference in the odds of 30-day post discharge mortality in rehospitalization quartiles, compared to the quartile with the highest readmission rate. 11 The largest national study of fee for service Medicare patients similarly found only a weak negative association between RSRR and RSMR for HF (Pearson correlation −0.17, P b .05). 1 The absence of a positive association is puzzling. Good HF care should improve both measures, while the competing risks of mortality and readmission should cause them to vary inversely.
The impact of high quality HF care The face validity of these two hospital measures requires them to both be associated with high quality
American Heart Journal October 2013
606 Huesch, Ong, and Fonarow
care. This suggests that they should be positively correlated, after correcting for differences in patient risk. Hospitals providing care that results in better 30 day risk standardized mortality (RSMR) should also perform very well on 30 day risk standardized rehospitalization (RSRR) and conversely. It is possible that the type of HF care that lowers readmission risk has little impact on mortality or vice versa. 1 But HF care process and disease management approaches have in general shown favorable impact on both mortality and rehospitalization. 12–15 Such improvements appear indeed to have led to a convergence to the “no different than the US national rate” for risk-standardized readmission as well as mortality according to CMS measures. 16 It is not clear why a positive relationship between these quality measures is not seen.
The impact of competing risks On the other hand the 2 risk-standardized measures should be negatively correlated due to their competing risks nature. 6,7,17,18 A patient who dies can no longer be readmitted (biasing readmission down). In HF, about 7% of admissions lead to death between discharge and the end of the readmission window. 19 In over half of those cases, the patient died without being readmitted, 19 potentially producing a competing risks bias. Even when deaths are no longer counted, after 30 days postadmission, they may still censor readmission at the tail end of the 30 day post-discharge interval (Fig. 1). Conversely, a patient who is readmitted may have death prevented through targeted procedures in hospital (biasing mortality down). Around one sixth of all HF 30day readmissions occur just in the final 5 days of the readmission window, 20 hence still within the mortality measure's window, suggesting a potential for competing risk bias.
Simulating the impact of high quality care and competing risks We examined this potential for competing risks to affect the two hospital-level measures in a simple simulation model calibrated to typical HF readmission and mortality values. For each condition, a set of 4,000 hospitals was created, each with 100 patients. These models assumed a stylized 5 day admission during which death can occur, followed by 30 days during which death and/or readmission can occur. The models specified different data generating processes for the realization of a readmission event or a mortality event. In one scenario, hospital efforts to improve readmission do not impact mortality and vice versa. Accordingly these two events were modeled as occurring independently. In a different scenario, any hospital efforts improve both readmission
Figure 1 Admission
Discharge
30 day mortality measure Days
30 day readmission measure 3 3 3 3 3 3 0 1 2 3 4 5
1 2 3 4 5 6
No competing risk
Competing risk of mortality and readmission
No competing risk
Quality measures—schematic view of mortality and readmission measures over their intervals. Presence of competing risks is illustrated for typical cardiovascular disease hospitalizations.
and mortality, and the two events were modeled as occurring with positive covariance (see online Appendix Supplemental Material for Review). Patients eligible for inclusion in the mortality measure comprised a different cohort than patients eligible for inclusion in readmission measure, in line with CMS algorithms. Patients who experienced a simulated mortality event during the index admission were not eligible to experience a readmission event subsequently. The model accounted for subsequent competing risks between death and rehospitalization as follows. Patients who died in the period 5–30 days after admission (ie, in the last 25 days of the 30 day mortality measure interval) were not able to be readmitted in the later period 30–35 days after admission (ie, in the last 5 days of the 30-day readmission measure interval). Conservatively, we did not insist that death within the period 5–30 days after admission censored rehospitalization within that period, although a more complex model could have allowed such additional competing risks. Similarly conservatively, we did not insist that rehospitalization sometimes censored death, although the main purpose of rehospitalization is clearly to avoid or delay death. We hypothesized that at the hospital level there would be an inverse relationship between readmission and mortality in the absence of hospital initiatives to improve on both measures of care quality. The presence of such hypothesized initiatives could contribute to a net effect of no observed association between RSRR and RSMR in the simulations. Results of this simple simulation were consistent with these hypotheses (Fig. 2). A modest but strongly statistically significant negative association was found between the readmission and mortality measures (Pearson correlation: −0.11, P b .001). However, when hospital initiatives were simulated that improved both the underlying readmission and mortality processes, the
American Heart Journal Volume 166, Number 4
Huesch, Ong, and Fonarow 607
Figure 2
A
.1
.2
RSRR
.3
.4
Lowess smoother
0
.05
.1
.15
.2
RSMR bandwidth = .8
B
.3 .1
.2
RSRR
.4
Lowess smoother
0
.05
.1
RSMR
.15
.2
bandwidth = .8
A, When rehospitalization and mortality are actually independent, competing risk bias yields an inverse relationship between hospitallevel RSRR and RSMR in a simulation of HF. B, When rehospitalization and mortality are both positively affected by simulated hospital quality initiatives, the effect of competing risks is masked, resulting in a lack of apparent association between hospital-level RSRR and RSMR in a simulation of HF.
prior negative association was masked and essentially disappeared (Pearson correlation: +0.04, P = .02). It is important to point out that the competing risks remain, but are outweighed by the simulated hospital-level initiatives. These results are consistent with the recent findings of little correlation between readmission and mortality for HF. 1,11
Unobserved index admission severity Currently quality is measured only at the hospital level, conditional on at least an index admission. 21 This event is potentially endogenous and thus under the control of the hospital. Admission criteria for HF are poorly standard-
ized. To the extent that the decision to admit patients varies systematically between hospitals, hospital performance under these measures will be biased. Those hospitals with a lower threshold for admission may admit healthier patients merely as a function of spare capacity. 22,23 Such patients may do better on mortality or rehospitalization measures, pulling up the average performance of their hospital. Other hospitals with a higher threshold for admission would experience the opposite effect. Such potential unobserved confounding is consistent with evidence that hospitals that do well on a measure for one condition do well on that measure for the other conditions as well. 2 We replicated this using Hospital Compare data (Fig. 3), and as previous research has shown, 2 a strong association was observed between a hospital's readmission rate for acute myocardial infarction and for HF (Pearson correlation: 0.40, P b .001). Prior research also suggests that variation in hospital capacity is associated with hospitalization rates, although not with mortality. 24 Similarly, variation in rehospitalization rates is strongly associated with variation in the original index hospitalization rates. Regionally such correlations have been recently estimated at more than 50% for HF. 25 If readmission was a function of regional variations in capacity and admission thresholds, then readmission does not accurately or directly reflect hospital quality of care. These are all threats to the construct validity of the readmission measure. Selection biases introduced by hospital-level admission thresholds could potentially be addressed by consideration of the full set of presenting patients, including those presenting to the Emergency Department but not admitted; those boarded, observed, and discharged; or those “admitted to observation” but not formally admitted. Given the clear and perverse incentives with current rehospitalization measures excluding out-ofhospital deaths makes understanding such selection biases better even more important.
Unobserved risks that influence readmission Ideally, hospitals should be able to intervene on select patients identified at highest risk of readmission. Unfortunately, the models that estimate readmission risk are poor in this regard, and it seems very difficult to stratify patients based on risk. 26 Models that seek to predict readmission in HF tend to have only modest (c-indices of 0.55-0.61) discrimination for this event. 27 The CMS model for HF readmission is at the top end of this range at 0.61. 19 To put this in perspective, a c-index of 0.5 represents no better ability than a coin toss to distinguish a case from a control based on estimated risk alone. Clearly, other factors such as social support, site
American Heart Journal October 2013
608 Huesch, Ong, and Fonarow
Figure 3
15
20
HF 30 day RSRR
25
30
Readmission rates for AMI vs HF
20
25
30
35
HF 30 day RSRR Readmission rates for AMI and HF—Hospital Compare data showing hospital-level association between 30 day risk-adjusted rehospitalization rates for AMI and HF. Pearson correlation: 0.40 (P b .001).
and socio-economic status help to determine readmission risk apart from patient and hospital covariates. 4 For example, state-level income inequality was found to be significantly associated with 30 day rehospitalization for HF, although not with mortality. 5 If these readmission models do not incorporate necessary content beyond demographic and comorbidity data, then both their content and predictive validity may be threatened. 28 The lack of correlation between hospital's 30-day readmission and 30-day mortality rates could therefore also indicate that the 30-day risk-standardized readmission rates are substantially misclassifying hospital performance. Those hospitals providing higher care quality resulting in lower risk-standardized mortality may also have had lower 30-day rehospitalization rates had models with better discrimination been applied. While the impact of such reclassification has previously been noted at the individual hospital level, 29 their possible effect on the relationship between readmission and mortality has not yet been investigated in HF. We note that empirical analyses of such classification biases among hospitals treating stroke patients in the Get With The Guidelines program found substantial effects on hospital performance rankings. 29
Conclusions Measuring, reporting, profiling and incenting hospitals on meaningful and valid quality metrics is an uncontro-
versial objective. Yet issues with the CMS readmission measure and its interaction with the mortality measure raise concerns about the validity of current approaches. Given the importance, use for accountability, and basis for substantial financial penalties of 30 day risk standardized outcome measures, it is vital that they continue to be scrutinized. 30 Valid measures will allow payors to reward objective excellence, and will incent providers to improve performance with effective clinical strategies. Patients and their referring physicians would have greater confidence in Hospital Compare and rely on the results to seek better categorized hospitals. Our scrutiny of the risk-standardized measures that CMS uses reveals some concerns as to their validity, especially that of the rehospitalization measure. As a result, it is possible that hospitals may fail to undertake appropriate actions or misjudge the likely success of recommended actions. Worse, the current structural focus on rehospitalization implies that out-of-hospital death is an acceptable outcome. 7 The resulting financial disincentive to readmit a deteriorating patient before the 30-day post discharge window is very troublesome for hospitals and clinicians determined to provide patient-centered excellent care. Other approaches to measuring and incenting quality in cardiovascular care may be needed to better operationalize the measurement and achievement of hospital quality of care. Our view is that healthcare reform has set into place a number of important changes in payment policy that may allow such better operationalization. Chief among these are the bundled payments for an episode of care (EOC) comprising care delivered in-hospital during the index admission and in or out of the hospital during the ensuing 30 days. Economically, the total number of inpatient days in this window will be the largest driver of hospital costs. Conceptually, rather than dichotomizing readmission, aggregating readmitted days may better reflect total resource use and hospital management objectives vis-àvis bundled payments. Facing a fixed reimbursement for an initial HF EOC, hospitals and their Accountable Care Organization delivery partners will have a clear incentive to better manage and reduce EOC inpatient days. 11 Partly, they will see opportunities for better, more efficient resource use during the index admission. Partly, too, they will respond to opportunities to better coordinate care with community-based partners who deliver high-quality, efficient outpatient care after discharge in a way that minimizes readmissions. We thus see the policy decision of payment reform effectively decentralizing the problem to the hospital level. Strategically, aggregated inpatient days in a 30-day EOC may thus better serve as a national hospital-level quality improvement objective. The hospital, in this future view, would have responsibility for creating the quality initiatives that provide more efficient care. This
American Heart Journal Volume 166, Number 4
EOC measure would be operationalized as follows. Rather than reporting, rating and either penalizing or rewarding hospitals based on risk standardized rehospitalization rates, risk standardized EOC days would be reported and used for the penalties or rewards. In a recent study, better EOC performance (ie, lower total EOC inpatient days) was associated with better 30day survival in Get With The Guidelines-HF patients. 11 However, to ensure that perverse incentives do not arise towards the rationing of care and towards out of hospital death, other risk-adjusted metrics focusing on mortality are likely to continue to be needed to balance efficiency and outcome objectives. 11 For example, consideration of a composite of early mortality and/or readmission events, or focusing on the complement of mortality (eg, such as total days alive, or composite metrics such as total days alive out of hospital) may be more fruitful. Combinations of such new and alternative measures may make more sense, reflect the underlying construct of quality in a more accurate way, and incorporate the necessary content in order to be able to predict quality better. In the meantime, incenting hospitals on riskstandardized readmission may be sub-optimal and bears re-examination.
Disclosures Dr Huesch reports support from grants by AHRQ and Lockheed Martin and is consulting to IOM and Precision Health Economics. Dr Ong reports support from grants by AHRQ and NIH and is consulting to IOM and the Public Health Institute. Dr Fonarow reports support from grants by GlaxoSmithKline, NIH, and AHRQ; is serving on speakers' bureaus for Boston Scientific/Guidant, GlaxoSmithKline, Medtronic, Merck, Novartis, Pfizer, and St Jude Medical; and is consulting to Amgen, Gambro, GlaxoSmithKline, Medtronic, Merck, Novartis, Pfizer, Relypsa, Scios, St Jude Medical, Medicines Company, Johnson and Johnson, and Takeda. He holds the Eliot Corday Chair of Cardiovascular Medicine at UCLA and is also supported by the Ahmanson Foundation.
References 1. Krumholz HM, Lin Z, Keenan PS, et al. Relationship between hospital readmission and mortality rates for patients hospitalized with acute myocardial infarction, heart failure, or pneumonia. JAMA 2013; 309:587-93. 2. Horwitz LI, Wang Y, Desai MM, et al. Correlations among riskstandardized mortality rates and among risk-standardized readmission rates within hospitals. J Hosp Med 2012;7:690-6. 3. Joynt KE, Jha AK. A path forward on Medicare readmissions. N Eng J Med 2013, http://dx.doi.org/10.1056/NEJMp1300122. 4. Joynt JE, Orav EJ, Jha AK. Thirty-day readmission rates for Medicare beneficiaries by race and site of care. JAMA 2011;305: 675-81.
Huesch, Ong, and Fonarow 609
5. Lindenauer PK, Lagu T, Rothberg MB, et al. Income inequality and 30 day outcomes after acute myocardial infarction, heart failure, and pneumonia: retrospective cohort study. BMJ 2013;346:f521. 6. Gheorghiade M, Vaduganathan M, Fonarow GC, et al. Rehospitalization for heart failure: problems and perspectives. J Am Coll Cardiol 2013;61:391-403. 7. Konstam MA. Heart failure in the lifetime of Musca domestica (the common housefly). JACC: Heart Failure 2013, http: //dx.doi.org/10.1016/j.jchf.2013.02.003. 8. Ryan A, Burgess J, Strawderman R, et al. What is the best way to estimate hospital quality outcomes? A simulation approach. Health Serv Res 2012, http://dx.doi.org/10.1111/j.1475-6773.2012. 01382.x. 9. Silber JH, Rosenbaum PR, Brachet TJ, et al. The Hospital Compare mortality model and the volume-outcome relationship. Health Serv Res 2010;45:1148-67. 10. QualityNet. 2012 Measures maintenance technical report: acute myocardial infarction, heart failure, and pneumonia 30-day risk-standardized mortality measures. http://qualitynet.org/dcs/ ContentServer?c=Page&pagename=QnetPublic% 2FPage%2FQnetTier4&cid=1163010421830. 11. Kociol RD, Liang L, Hernandez AF, et al. Are we targeting the right metric for heart failure? Comparison of hospital 30-day readmission rates and total episode of care inpatient days. Am Heart J 2013;165: 987-994.e1. 12. Naylor MD, Brooten DA, Campbell RL, et al. Transitional care of older adults hospitalized with heart failure: a randomized, controlled trial. J Am Geriatr Soc 2004;52:675-84. 13. Rich MW, Beckham V, Wittenberg C, et al. A multidisciplinary intervention to prevent the readmission of elderly patients with congestive heart failure. N Engl J Med 1995;333:1190-5. 14. Lappé JM, Muhlestein JB, Lappé DL, et al. Improvements in 1-year cardiovascular clinical outcomes associated with a hospital-based discharge medication program. Ann Intern Med 2004;141:446-53. 15. Phillips CO, Wright SM, Kern DE, et al. Comprehensive discharge planning with postdischarge support for older patients with congestive heart failure: a meta-analysis. JAMA 2004;291:1358-67. 16. Bernheim SM, Grady JN, Lin Z, et al. National patterns of riskstandardized mortality and readmission for acute myocardial infarction and heart failure: update on publicly reported outcomes measures based on the 2010 release. Circ Cardiovasc Qual Outcomes 2010;3:459-67. 17. Rothman KJ, Greenland S. Cohort studies. In: Rothman KJ, Greenland S, eds. Modern epidemiology. 2nd ed. Lippincott-Raven; 1998. p. 79-91. 18. Gorodeski EZ, Starling RC, Blackstone EH. Are all readmissions bad readmissions? N Engl J Med 2010;363:297-8. 19. Keenan PS, Normand S-LT, Lin Z, et al. An administrative claims measure suitable for profiling hospital performance on the basis of 30-day all-cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes 2008;1:29-37. 20. Dharmarajan K, Hsieh AF, Lin Z, et al. Diagnoses and timing of 30-day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA 2013; 309:355-63. 21. Huesch MD. Payment policy by measurement of health care spending and outcomes. JAMA 2010;303:2405-6. 22. Wennberg JE, Freeman JL, Shelton RM, et al. Hospital use and mortality among Medicare beneficiaries in Boston and New Haven. N Engl J Med 1989;321:1168-73. 23. Wennberg JE, Fisher ES, Goodman DC, et al. Dartmouth atlas of health care: tracking the care of patients with severe chronic illness.
610 Huesch, Ong, and Fonarow
Lebanon, NH: Dartmouth Institute for Health Policy and Clinical Practice. 2008. 24. Fisher ES, Wennberg JE, Stukel TA, et al. Associations among hospital capacity, utilization, and mortality of US Medicare beneficiaries, controlling for sociodemographic factors. Health Serv Res 2000;34:1351-62. 25. Epstein AM, Jha AK, Orav EJ. The relationship between hospital admission rates and rehospitalizations. N Engl J Med 2011;365:2287-95. 26. Ross JS, Mulvey GK, Stauffer B, et al. Statistical models and patient predictors of readmission for heart failure: a systematic review. Arch Intern Med 2008;168:1371-86.
American Heart Journal October 2013
27. Au AG, McAlister FA, Bakal JA, et al. Predicting the risk of unplanned readmission or death within 30 days of discharge after a heart failure hospitalization. Am Heart J 2012;164:365-72. 28. Fonarow GC, Peterson ED. Heart failure performance measures and outcomes: real or illusory gains. JAMA 2009;302:792-4. 29. Fonarow GC, Pan W, Saver JL, et al. Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity. JAMA 2012;308: 257-64. 30. Krumholz HM, Lin Z, Normand ST. Measuring hospital clinical outcomes. BMJ 2013;346:f620.
American Heart Journal Volume 166, Number 4
Appendix Methods We report additional detail on the approaches and simulation code underlying the figures in the text. Re Figure 2 We performed a competing risk simulation. We simplify the data generating processes (DGP) for readmission and mortality in the following crude but reasonably realistic way: • Days 1-5: the only event is death, which happens or doesn't in this period regardless of which day. Death is modeled as having occurred in this period through realization of a uniformly distributed random variate. Death in this period censors subsequent readmission. • Days 5-30: events are death, death and readmission, readmission only, or neither. These events only happen if there is no death in the first period above. These events happen or don't in this period regardless of which day. Death and readmission are modeled as a DGP in which they are jointly determined by a bivariate standard normal distribution with usermodified covariance. Our figures show 0 or +0.5 correlation respectively. The former implies that hospital initiatives do not affect both measures, the latter implies hospital-level initiatives can affect both events. If a random normal variate is above the z value corresponding to the estimated event rate for that measure, then the event is modeled as occurring. ○ Even if death occurs in this period, readmission is not barred from happening in this period. ○ Similarly if readmission happens, there is no bar on death happening. ○ Conservatively, the competing risk bias arises only because death in a period censors readmission in a subsequent period. • Days 30-35: analogous to above. Events here happen only if there is no death in either of the above periods. There is no positive or negative covariance allowed in this separate bivariate standard normal distribution which generates death and or readmission. We did not include dependence in this DGP because the temporal distance from discharge seemed too large to induce dependence (ie, hospital-level initiatives that affect both events). • Risk-standardized mortality is aggregated as death events in the first period, days 1-5, and the second period, days 5-30. Risk-standardized readmission is aggregated as readmission events in the second period, days 5-30, and the third period, days 30-35. • Limitations of this approach include the oversimplification of dividing the relevant time into just three periods instead of individual days. A conservative bias arises from restricting the effect of death to only be on subsequent periods. This tends to underestimate
Huesch, Ong, and Fonarow 610.e1
competing risk bias. For example, a death on day 11 would censor any readmissions on days 12-35 post admission (in the real world), not just readmission between days 30 and 35 (as in our model). Another conservative bias is that we did not model the fact that readmission is likely to censor some deaths. That is, being readmitted is likely to prolong life for at least some patients, and avoid a death within the period of interest. This again underestimates the competing risk bias. A related countervailing bias is that we did not model dependence in the final period, reducing the potential impact of hospital-level initiatives to continue to reduce both events. It is not clear to what extent these biases affect the simulation results; however, we believe the simulation provides a useful and conservative illustration of the impact of competing risk bias on observed correlations between measures, and a similarly useful illustration of the impact of positively correlated events due to hospital-level initiatives that affect readmission and mortality simultaneously. Our STATA code is given below to facilitate reproduction of our approach: program readmitmortHF, rclass version 12 drop _all set obs 100 gen x = runiform() gen Mort_OnAdmission = cond(xb= 0.04,1,0) * now code for 5 to 30 day post admission interval * now drawing jointly normal distributions with specified covariance matrix matrix C = (1, 0 \ 0, 1) * initially specifying zero covariance between readmission and mortality, will flex later drawnorm a b, corr (C) * jointly normal readmission and mortality gen p2 = 1- 0.07 * what z value should the first normal variate take so that between days 5 and 30, mortality is 7%… gen z2 = invnormal(p2) * to force mortality realization within the ~7% probability required gen Mort_Between5and30 = cond(Mort_OnAdmission==0 & a N= z2,1,0) * can't die between days 5 and 30 if already died during admission * now for the readmission event gen p3 = 1- 0.15 * since between days 5, 30, readmission is 15%… gen z3 = invnormal(p3) gen Admit_Between5and30 = cond(Mort_OnAdmission==0 & b N= z3,1,0) * to force readmission realization within the ~15% probability required * if we specify a positive value for the covariance in C, higher chance both events occur
610.e2 Huesch, Ong, and Fonarow
* but can't get admitted between days 5 and 30 if already died during admission * Can obviously get admitted and still die here. Published heart failure data (Keenan et al) has * about 3% of admissions both die after discharge and get readmitted * now analogous code for the 30-35 day interval matrix D = (1, 0 \ 0, 1) * Will want to use a lower value for covariance here than before, seeing as these * are so far temporally from discharge drawnorm c d, corr (D) gen p4 = 1- 0.01 * between days 30 and 35, mortality is 1% (my estimate), but no longer counted in 30 day mortality. Sstill need to account for it given the competing risks set up gen z4 = invnormal(p4) gen Mort_Between30and35 = cond(Mort_OnAdmission==0 & Mort_Between5and30==0 & c N= z4,1,0) * can't die between days 30 and 35 if already died earlier * now for the readmission event gen p5 = 1- 0.10 * between days 30 and 35, readmission is 10% … gen z5 = invnormal(p5) gen Admit_Between30and35 = cond(Mort_OnAdmission==0 & Mort_Between5and30==0 & d N= z5,1,0) * can't get readmitted later if already died earlier
American Heart Journal October 2013
* now generate hospital-level summary of the outcomes of the 100 patients in this run * first generate 30 day mortality between 0 and 30 days summarize Mort_OnAdmission return scalar Mort_1to5 = r(sum)/100 summarize Mort_Between5and30 return scalar Mort_5to30 = r(sum)/100 * next generate 30 day readmission between 5 and 35 days summarize Admit_Between5and30 return scalar Admit_5to30 = r(sum)/100 summarize Admit_Between30and35 return scalar Admit_30to35 = r(sum)/100 end set seed 1234 * Assume 4,000 hospitals simulate, reps(4000) nodots saving(results.dta, replace): readmitmortHF gen RSMR = Mort_1to5 + Mort_5to30 gen RSRR = Admit_5to30 + Admit_30to35 pwcorr RSMR RSRR, sig lowess RSRR RSMR Re Figure 3 We used risk-standardized mortality and risk-standardized readmission rates for the 4776 US hospitals reported by Hospital Compare. Data source is the official Centers for Medicare and Medicaid Services website https://data. medicare.gov/. We added a lowess tricubic spline smoothed local regression line and computed Pearson correlations.