- Email: [email protected]
Identification of risk factors for increased cost, charges, and length of stay for cardiac patients
Identification of Risk Factors for Increased Cost, Charges, and Length of Stay for Cardiac Patients Samantha MaWhinney, ScD, Elizabeth R. Brown, MS, Janet Malcolm, MBA, Catherine VillaNueva, MBA, Bertron M. Groves, MD, Robert A. Quaife, MD, JoAnn Lindenfeld, MD, Bradley A. Warner, PhD, Karl E. Hammermeister, MD, Frederick L. Grover, MD, and A. Laurie W. Shroyer, PhD Departments of Preventive Medicine and Biometrics, Medicine, and Surgery, University of Colorado Health Sciences Center, Denver, Department of Mathematics, United States Air Force Academy, Colorado Springs, and Department of Veterans Affairs Medical Center, Denver, Colorado
Background. In this study we explored different risk model options to provide clinicians with predictions for resource utilization. The hypotheses were that predictors of mortality are not predictive of resource consumption, and that there is a correlation between cost estimates derived using a cost-to-charge ratio or a product-line costing approach. Methods. From March 1992 to June 1995, 2,481 University of Colorado Hospital patients admitted for ischemic heart disease were classified by diagnosis-related group code as having undergone or experienced coronary bypass procedures (CBP), percutaneous cardiovascular procedures (PCVP), acute myocardial infarction (AMI), and other cardiac-related discharges (Other). For each diag-
nosis-related group, Cox proportional hazards models were developed to determine predictors of cost, charges, and length of stay. Results. The diagnosis groups differed in the clinical factors that predicted resource use. As the two costing methods were highly correlated, either approach may be used to assess relative resource consumption provided costs are reconciled to audited financial statements. Conclusions. To develop valid prediction models for costs of care, the clinical risk factors that are traditionally used to predict risk-adjusted mortality may need to be expanded. (Ann Thorac Surg 2000;70:702–10) © 2000 by The Society of Thoracic Surgeons
T
product-line costing approach. Thus, either cost estimation approach may be used in analyses and reports for clinical decision-making purposes.
his study expands upon previous research efforts to provide clinicians with predicted outcomes and costs of care based on statistical models to enhance patient care decisions. A variety of predictive models are available to estimate a patient’s clinical outcomes following hospital admission for ischemic heart disease. Ideally, the clinical care team would select and implement the strategy of care that maximizes patient outcomes without wasting scarce resources. Therefore, information regarding both the patient’s medical outcomes and the estimated relative resource use associated with a set of alternative therapies become important considerations in the clinical team’s decision-making process. The study’s primary hypothesis was that there is no difference between the patient risk factors and procedural details that predict mortality and those that predict resource consumption. If this hypothesis is true, then the risk factors and procedural details included in current risk-adjusted mortality may be used to predict resource consumption measures such as costs, charges, and length of stay. The secondary study hypothesis was that there is a correlation between costs estimated by using a cost-tocharge ratio approach and costs estimated by using a Accepted for publication Mar 31, 2000. Address reprint requests to Dr Shroyer, Division of Cardiac Research, Denver VA Medical Center, 1055 Clermont St (151R), Denver, CO 80220; e-mail: [email protected].
© 2000 by The Society of Thoracic Surgeons Published by Elsevier Science Inc
Background Risk Model Predictions for Cost and Death “An ideal physician is defined as one who selects and implements the strategy of care that maximizes health status improvement without wasted resources” [1]. Both the probability of an outcome and the relative costs associated with a set of alternative therapies are important considerations in the decision to recommend a given therapy for a specific patient. Statistical model predictions are gaining popularity as clinical decision-making tools for patient management [2–5]. To date, these statistical models have focused primarily on the ability to predict the probability of adverse outcomes given a patient’s risk characteristics. Recent research has demonstrated that the introduction of a prediction risk model in clinical settings can alter patient management decisions [6, 7]. Statistical prediction models have not been used widely to estimate patient-specific costs, although the conceptual basis is similar to that for predicting outcomes. Given relatively comparable predicted clinical outcomes associated with a set of treatment alternatives, patient cost estimates should be helpful information to 0003-4975/00/$20.00 PII S0003-4975(00)01510-1
Ann Thorac Surg 2000;70:702–10
the clinician. This cost information may be particularly important in clinical decisions regarding high cost therapies (such as cardiac surgery). Greater precision in predicting outcome and costs following therapy may provide clinicians with reference points to augment decision making by increasing the level of information and communication [8]. No statistical risk model can replace clinical judgment, but good decision making must begin with good information. Hence, risk stratification represents a technological advancement that may be used to enhance the clinical decision-making process. Iezzoni and colleagues [9] explored the patient-specific risk factors that predict costs and quality of care. Their results (which included only four cardiac diagnosisrelated groups [DRGs] related to acute myocardial infarction/heart failure and shock) indicated that the MedisGroups risk-adjustment methodology, a method for assessing hospital admission severity of illness [10], performed well in predicting mortality. However, this risk adjustment method generally did poorly in predicting the cost of hospital admissions. Iezzoni and colleagues [9] also found that there was a positive relationship between risk score and cost. However, there was an inverse relationship between risk score and cost for patients dying in the hospital. The cost per day for the patients who died was higher than the cost per day for patients who lived. In summary, the MedisGroups riskassessment method could not be generically applied to both outcomes— cost and mortality rates. Iezzoni and colleagues [11] then built empirical models to predict cost and mortality rates from the patient risk characteristic data available. The most powerful risk factor predictors for costs differed from those that predicted the probability of in-hospital death. The risk factors selected for cost prediction tended to be less physiologic and more condition specific (eg, presence of congestive heart failure). In contrast, the most powerful predictors of death were more physiologic indicators of general function and less condition specific (eg, presence of renal dysfunction). Similarly, in the Colorado Health Data Commission’s 1992 Outcomes Report [12], the MedisGroups model used was a poor predictor of the observed variation in either charges or length of stay. Across the range of cardiac diagnoses reported, the prediction model explained only 7% of the variation in length of stay not previously explained by DRG adjustment alone. For charges, the risk adjustment at best explained up to 16% of the variation not previously explained by DRG adjustment alone. No estimation of costs was performed as part of this study.
Relationship Between Cost and Quality Conflicting empirical research findings on the relationship between cost and quality of medical care currently exists— demonstrating both positive and negative relationships [13–15]. Most of these studies hypothesized simple linear relationships between cost and quality. However, Donabedian and colleagues [1] suggested in 1982 that marginal cost may vary over the range of quality depending on the preexisting level of quality of
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
703
care provided. The cost– quality relationship may be nonlinear in nature.
Material and Methods Patient Population For this study, the population consisted of 2,481 patients from the University Hospital in Denver, Colorado, who had had admissions for ischemic heart disease-related diagnoses during the period from March 1, 1992 to June 30, 1995. Data for this project were obtained on 2,420 patient records (97.5% of the original 2,481) from four different databases including a medical records system, a clinical severity classification system, a patient billing system, and a product-line costing system. Records were dropped from the study for the following reasons: patients were under the age of 18 years; records did not meet study inclusion criteria for the DRG set examined related to a reported diagnosis of ischemic heart disease; or the data on the patient were unavailable from one or more of the required databases. Moreover, patient records with extreme charge or cost data were reviewed. After administrative review, these very extreme outliers (above the 99% threshold of charges or cost) were dropped due to concerns about potential errors and inconsistencies found in the billing or costing data reported. In total, 61 records were removed from the analysis.
Data Sources Data from the four databases were merged. Before proceeding with the analyses, all data quality or completeness discrepancies identified (such as mismatching dates of service) were resolved and updated. To address these issues, the patient charts were pulled as needed. For costing analysis purposes, the audited Medicare Hospital Cost Report was used to generate five revenue categoryspecific cost-to-charge ratios; the categories were routine, pharmacy, laboratory, radiology, and all other costs (eg, intensive care unit costs). The reformat routine used to reclassify the detailed billing charge codes into these five summary charge categories was the standard conversion used by the Colorado Hospital Association. The costs for all records were initially estimated by multiplying the hospital-specific charges in these five revenue categories by the weighted average ratio of cost to charges (RCC) for the corresponding hospital cost centers. For the period from March 1, 1992 to June 30, 1993, the University Hospital did not estimate costs using the product-line costing system. For discharges from July 1, 1993 to June 30, 1995, the inpatient hospital costs were derived from the fully costed patient level databases in the University Hospital Decision Support System (DSS). This system applies an industrial cost accounting approach to the healthcare processes to arrive at a detailed intermediate product costing at the patient level. All charges and costs were adjusted appropriately for inflation (based on the Denver regional consumer price index for medical care goods and services) to the initial
704
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
Ann Thorac Surg 2000;70:702–10
Table 1. Patient Baseline Characteristics and In-Hospital Clinical Outcomes for Patient Discharges Variable Number/percent of discharges Age (y) Median, (1st, 3rd quartile) Mean, SD Gender Female (%) Race White (%) Black (%) Hispanic (%) Other/unknown (%) Admit source ER (%) Admission office (%) Outpatient (%) Other hospital, acute (%) Other (%) Admission type Urgent/emergent (%) Prior failed PCVP (%) CAD left main ⬎ 49% (%) CHF (%) Age unspecified MI (%) CPK ⬎ 150 u/l (%) Ejection fraction ⬍ 41% (%) MedisGroups/Atlas risk estimate Median Risk Estimate 1st, 3rd quartile Mean risk estimate In-hospital death Number Percent MedisGroups Mortality Model C-Index 95% confidence interval
CBP
PCVP
AMI
All Other
Total
221/9%
1,046/43%
183/8%
970/40%
2,420
61 (54, 68) 60.7, 10.7
55 (45, 66) 54.7, 15.1
61 (53, 72) 61.7, 13.8
56 (47, 65) 56.2, 13.2
57 (47, 66) 56.4, 14.0
23.1
35.1
34.1
42.1
36.7
72.4 6.8 15.8 5.0
80.2 6.5 8.7 4.6
71.0 13.1 12.0 3.9
71.2 12.1 11.8 4.9
75.2 9.3 10.8 4.7
24.0 17.7 30.3 26.2 1.8
27.0 21.0 29.2 21.7 1.0
82.5 2.2 3.3 9.8 2.2
60.7 11.7 20.2 6.4 1.0
15.5 44.4 23.8 15.1 1.2
49.8 3.6 15.4 20.8 31.7 34.4 6.3
49.1 9.3 1.5 6.9 17.4 16.1 3.3
90.7a 1.6 1.6 19.1 49.7 77.6 7.1
69.2 1.6 2.6 10.6 12.2 15.8 8.1
39.6a 5.1 3.2 10.6 19.1 22.3 5.8
0.012 0.007, 0.022 0.024
0.004 0.004, 0.013 0.018
0.033 0.011, 0.124 0.118
0.008 0.004, 0.018 0.025
0.008 0.004, 0.018 0.029
10 4.52 0.71 (0.42, 0.87)
15 1.43 0.90 (0.69, 0.97)
19 10.38 0.89 (0.72, 0.95)
5 0.52 0.88 (0.73, 0.99)
49 2.02 0.89 (0.82, 0.93)
Four categories include: coronary bypass procedures (CBP) (DRGs 106 and 107), percutaneous cardiovascular procedures (PCVP) (DRG 112), acute myocardial infarction (AMI) (DRGs 121, 122, 123), and other coronary artery disease-related discharges (Other) (DRGs 124, 125, 140). a
AMI N ⫽ 182;
Total n ⫽ 2,419.
CAD ⫽ coronary artery disease; infarction.
CHF ⫽ congestive heart failure;
CPK ⫽ creatine phosphokinase;
study periods of March 1992. Both the Medicare Hospital Cost Report data and the product-line costing data were confirmed to reconcile to the hospital’s period-specific financial statements. In general, the DSS information was more detailed based on the actual activities performed (including direct labor expenses) and supplies used (including direct supply expenses) in comparison with the RCC approach. Given the greater accounting detail used to derive DSS costs, it was thought that perhaps the DSS costing methodology would provide greater precision for the purposes of this project. Thus, the RCC costs for the period from July 1, 1993 to June 30, 1995 were subsequently compared with the DSS product-line costing estimates obtained using linear regression techniques.
ER ⫽ emergency room;
MI ⫽ myocardial
Classification of Patients for Study Purposes Based on DRG coding assignment, patients were classified into four categories: coronary bypass procedures (CBP) (DRGs 106 and 107), percutaneous cardiovascular procedures (PCVP) (DRG 112, which includes procedures other than percutaneous transluminal coronary angioplasty [PTCA]), acute myocardial infarction (AMI) (DRGs 121, 122, 123), and other coronary artery disease-related discharges (Other) (DRGs 124, 125, 140). The clinical severity classification system (MedisGroups/Atlas) automatically assigned a severity estimate to each patient record using patient-specific risk variables. This severity assignment was used unaltered for purposes of this project.
Ann Thorac Surg 2000;70:702–10
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
Statistical Analysis
Table 2. Resource Consumption Summary Data
There are several features of Cox proportional hazards that make it superior for analyzing cost, charge, and length of stay data. First, an important feature of the Cox model is its lack of distributional assumptions. This feature is especially important when looking at highly skewed data such as cost and length of stay. Because the Cox model can accommodate right censoring, patients with incomplete or censored observations can be included in the analysis. For the prediction of resource consumption, it is important to include patients who have incomplete data due to an in-hospital death. These patients may be assumed to be high risk; therefore, leaving them out of the analysis could lead to an underestimation of the impact of high-risk patient characteristics. Also, as the quality of patient care improves, patients who would have died in the past can be expected to survive. Therefore, the contributions of these high-risk patients to cost, length of stay, and charge may be expected to increase in the future. Although there may be a bias due to informative censoring [16], Cox models have been shown to be superior for cost and charge data over traditional methods such as a linear regression that treats censored costs as if they were uncensored [17]. Cox proportional hazards models were developed for length of stay, costs, and charges using S-Plus statistical software (StatSci Software, Seattle, WA) and Harrell’s Design Library [18]. Ideally, the model would be built by first postulating the important predictors and then fitting the model. To reduce the number of risk factors considered in the final model, variables were restricted to those judged to be clinically relevant. These risk factors were further restricted to those present in at least 10% of the patients. A univariate Cox model was then used to further screen the predictors and only those that were significant at p values less than or equal to 0.1 were considered for inclusion in the multivariate model. The predictor, “admission type,” was forced into all models and an indicator of current admission catheterization was forced into all CBP models. Then a backward selection process was used to obtain the final model assuming a significance level of p values less than or equal to 0.1. The Cox model assumes linearity of covariates and proportional hazards [19]. To test the linearity assumption, each covariate was plotted against the martingale residuals from a model built without the covariate. A smoothed line was added, to provide graphical information about the functional form of the variable. Proportional hazards were assessed both graphically and statistically. Our graph of the Schoenfeld residuals showed no significant slope over time. This was supported by a statistical test of this slope. Therefore, there is no violation of proportional hazards in our models. The models were then validated using resampling validation techniques. All models performed well. For logistic regression analyses, the c-index is often used as a measure of risk model performance. The c-index represents the area under the receiver operating characteristic curve (that reflects the relative sensitivity
Variable
CBP
PCVP
AMI
705
Other Total
Total length of stay (days) Mean 11.40 3.68 6.16 3.70 Median 9 3 5 2 Charges (dollars) Mean 61,585 17,091 17,873 11,025 Median 52,932 13,608 14,701 7,761 Percent charges Routine (%) 17.2 15.6 27.1 23.0 Laboratory (%) 14.3 9.1 18.1 17.1 Pharmacy (%) 14.4 7.9 13.7 13.1 Radiology (%) 2.0 4.5 5.6 6.6 Other (%) 52.1 62.9 35.5 40.2 DSS cost estimates (dollars) Mean 27,091 8,982 9,421 5,894 Median 24,577 7,418 8,208 4,254
4.59 3 18,818 12,170 18.7 13.2 11.5 4.3 52.4
9,445 6,793
AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; DSS ⫽ University Hospital Decision Support System; Other ⫽ other cardiac-related discharges; PCVP ⫽ percutaneous cardiovascular procedures.
and specificity of the model in predicting risk of operative death). Theoretically, the c-index may range from 0 to 1. In general, a c-index of 0.5 is useless for prediction purposes. Confidence intervals for the c-indices were calculated using BCa bootstrap confidence intervals [20].
Results General Findings The base line inpatient risk characteristics and inhospital mortality for the four different DRG categories and overall are presented in Table 1. An analysis of variance determined that the MedisGroups risk model did not predict the risk of in-hospital death equivalently across the four diagnosis groups (p ⬍ 0.001). Likewise the in-hospital death rates were not the same for the four diagnosis groups when compared using Fisher’s exact test (p ⬍ 0.001). The clinical classification system for CBP patients does not discriminate well, as the c-index confidence intervals contain the value of 0.5. The summary descriptive statistics for the resource utilization findings for the different DRG categories are listed in Table 2. Given the inherent differences in the prevalence of patient risk characteristics and calculated severity of illness among categories, as well as the inherent variations in resource utilization, Cox proportional hazards models were developed for each DRG category separately.
Comparison of Different Costing Approaches To answer the question, “Can RCC costs be used to impute missing values of DSS cost in cost analysis?,” the data set was split randomly into model building and model validating data sets. We first performed a simple linear regression with RCC cost as the predictor variable
706
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
Ann Thorac Surg 2000;70:702–10
Table 3. Risk Factors in Final Models for Cost, Charge, and Length of Stay (Compared to MedisGroups/Atlas Mortality Model) CBP Risk Factors Acute Admission Type WBC ⬍ 5.0 or ⬎ 17.0 Age unspecified MI Dilated chambers Pulmonary edema Failed PCVP Decreased contractility CAD left main ⬎ 49% S3 gallop Pleural effusion CPK ⬎ 150 U/L Chronic Pulse ⬍ 40, ⬍ 65 or ⬎ 129 Sys BP ⬍ 60, ⬍ 90, or ⬎ 119 Pulmonary vascular disease Regurgitation CAD LAD ⬎ 49% Murmur (protective) Acute and/or chronic Atrial fibrillation CHF BUN ⬎ 30 mg/100 mL Ejection fraction ⬍ 41% Creatinine ⬎ 1.7 mg/dl O2 Saturation ⬍ 86% Other Catheterization (CBP only) Age (Older)
Mort.
x x
PCVP
AMI
Other
LOS
Cost
Chg
LOS
Cost
Chg
LOS
Cost
Chg
LOS
Cost
Chg
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x x x x x x
x
x
x
x x
x x
x x
x
x
x
x x x
x
x
x
x x
x x x
x x
x x x x
x x x x
x x x x
x x x x
x
x
x
x
x x
x x
x x
x
x
x
x
x
x
x x
x
x
x
x x x x
x x
x x
x
x x
x x
x x
x
x
x
x
x
x
x x
x x
AMI ⫽ acute myocardial infarction; BP ⫽ blood pressure; BUN ⫽ blood urea nitrogen; bypass procedures; CHF ⫽ congestive heart failure; CPK ⫽ creatine phosphokinase; myocardial infarction; LOS ⫽ length of stay; Other ⫽ other cardiac-related discharges; WBC ⫽ white blood cells.
and DSS cost as the response variable. This model had an R2adj (adjusted R2) value of 0.955, but the residuals were not normally distributed and their variance increased as the fitted value of DSS increased. In an attempt to normalize and account for the pattern in the residuals, we used a log10 transformation for both DSS and RCC costs. This transformation resulted in a random scatter of the residuals and the model fit the validating data set well. The residuals from this model were close to normally distributed, but with heavy tails. The R2adj for this model is 0.958, a slight increase above the model with no transformations. To bring in the tails of the distribution of the residuals, a squared term, (log10(RCC))2, was then added to the prediction model. Although this model sacrifices interpretability, the primary use is for prediction purposes. For the extreme values, this log-transformed, quadratic model was superior to the simpler models. The R2adj for this model is 0.962. The formula for the fitted line is
CAD ⫽ coronary artery disease; CBP ⫽ coronary LAD ⫽ left anterior descending artery; MI ⫽ PCVP ⫽ percutaneous cardiovascular procedures;
log 10(DSS) ⫽ ⫺1.5579⫹1.8559 ⴱ log 10(RCC) ⫺0.1161 ⴱ (log 10(RCC))2 In summary, the two different costing methodologies were highly correlated. Thus, either of the two different costing methodologies may be used for purposes of estimating costs for relative cost comparisons of these alternative therapies in clinical decisions.
Comparison of Models to Predict Cost and Death For each model developed, the patient risk characteristics that were found in the different models to be statistically important predictors by DRG category are listed in Table 3. The risk factors that were found to be predictive of resource consumption were generally not the same as those used to predict mortality. Table 3 compares the variables that were statistically significant in the various models reported. The risk factors were divided into the categories of acute, chronic, acute and/or chronic, and other. For reference, the first
Ann Thorac Surg 2000;70:702–10
Fig 1. Hazard ratios for total cost for coronary bypass patients. The hazard ratios for each risk variable are plotted for the cost model for the diagnosis-related group (DRG) category coronary bypass. A bar that denotes the 95% confidence interval surrounds each hazard ratio. The outcome event is the cost of care from admission to discharge. The graph indicates that patients at higher risk have a lower hazard for discharge and cost termination and therefore a higher expected cost. (BP ⫽ blood pressure; CHF ⫽ congestive heart failure; MI ⫽ myocardial infarction; PTCA ⫽ percutaneous transluminal coronary angioplasty.)
column indicates which of these variables were also significant in the MedisGroups/Atlas mortality model for the ischemic heart disease diagnosis. Admission type was categorized into two groups, elective and urgent/emergent. This variable was included in all the models. For each group an urgent/emergent
Fig 2. Hazard ratios for total charges for coronary bypass patients. The hazard ratios for each risk variable are plotted for the charge model for the diagnosis-related group (DRG) category coronary bypass. A bar that denotes the 95% confidence interval surrounds each hazard ratio. The outcome event is the charges for care from admission to discharge. The graph indicates that patients at higher risk have a lower hazard for discharge and charge termination and therefore higher expected charges. (BP ⫽ blood pressure; CHF ⫽ congestive heart failure; MI ⫽ myocardial infarction; PTCA ⫽ percutaneous transluminal coronary angioplasty.)
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
707
Fig 3. Hazard ratios for length of stay for coronary bypass patients. The hazard ratios for each risk variable are plotted for the length of stay model for the diagnosis-related group (DRG) category coronary bypass. A bar that denotes the 95% confidence interval surrounds each hazard ratio. The outcome event is time to discharge. The graph indicates that patients at higher risk have a lower hazard for discharge and therefore a longer expected length of stay. (BUN ⫽ blood urea nitrogen; MI ⫽ myocardial infarction.)
admission resulted in a longer length of stay, increased cost, and increased charges. The graphs of the hazard ratios for each risk variable in each of the three models for the DRG category CBP are presented in Figures 1 through 3. In Figures 1 through 3, the outcome events for cost, charges, and length of stay are the cost of care from admission to discharge, the total charges from admission to discharge, and time to discharge, respectively. The length of stay graph (Fig 3) can be interpreted such that patients at higher risk have a lower hazard of discharge and therefore a longer length of stay. The hazard ratio of emergent to non-emergent classification at admission is less than 1 because, as expected, the hazard of discharge is larger for the nonemergent patients. Similarly, patients with a catheterization at the current admission will experience a longer length of stay. As the hazard of discharge is higher for the patients who were not catheterized during the current admission, this result is demonstrated by a hazard ratio of discharge that is greater than 1 for the patients with no catheterization (DRG 107, coronary bypass without cardiac catheterization) compared with catheterization (DRG 106, coronary bypass with cardiac catheterization). For reference, the Kaplan-Meier survival curves for each DRG group for total cost, total charges, and length of stay are presented in Figures 4 through 6.
Relationship of Severity of Illness Estimates to Predicted Resource Utilization Cox models were fit to determine the contribution of the preoperative risk of death (MedisGroups mortality) as a predictor variable. The risk variable was first linearized by taking the logit. The logit of the risk was then allowed to be nonlinear in the model by using the restricted cubic spline components as predictors. All models satisfied the linearity and proportional hazards assumptions. We use
708
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
Ann Thorac Surg 2000;70:702–10
Fig 4. Kaplan-Meier curves for total cost by diagnosis-related group categories, truncated at $50,000. The symbols indicate an in-hospital death or censored observation. (AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; Other ⫽ other cardiac-related discharges; PCVP ⫽ percutaneous cardiovascular procedures.)
Nagelkerke’s R2 (R2N) as a measure of predictive ability for the Cox model. An increase in the R2N between models with the same outcome indicates an improved model fit [19]. The R2N value was obtained for these models then compared to the models built with risk factors. The results are reported in Table 4. These results indicate that for CBP, PCVP, and Other patients using risk factors to predict cost gives more information than using only the preoperative risk of death. For these data, building a model specifically to predict resource utilization is preferable to utilizing existing mortality model predictions to estimate resource use. For the AMI group, the R2N actually improved slightly in the model based solely on the MedisGroups mortality. Fig 5. Kaplan-Meier curves for total charges by diagnosis-related group categories, truncated at $100,000. The symbols indicate an in-hospital death or censored observation. (AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; Other ⫽ other cardiac-related discharges; PCVP ⫽ percutaneous cardiovascular procedures.)
Comment This single-center study focused exclusively on cardiac care patient discharges and was conducted in a university-based setting that serves as a tertiary referral care center for the state of Colorado. Therefore, generalizations to other patient populations may not be appropriate. MedisGroups severity adjustment approach is developed on national data set, but in this study had been applied to local data. In this context, model performance metrics may be compromised. For interventions, the MedisGroups adjustment was based on an assessment at time of admission rather than time of intervention. Thus, the timeliness of the risk factor assessment may be
Ann Thorac Surg 2000;70:702–10
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
709
Fig 6. Kaplan-Meier curves for length of stay by diagnosis-related group categories, truncated at 40 days. Due to the discreteness of the data, censoring symbols are not included. (AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; Other ⫽ other cardiac-related discharges; PCVP ⫽ percutaneous cardiovascular procedures.)
questioned. Additionally, the MedisGroup data entry default was a negative assessment. If a given risk factor was not coded, then this result was interpreted as a negative clinical finding. Ideally, a risk model to predict in-hospital mortality would be developed from this data set. As noted, the four diagnoses groups have different incidence of death. However, sample size limitations precluded the development of a model to predict death for this study; risk factors evaluated were limited predominantly to the MedisGroups key clinical findings abstracted and DRG data. With a larger data set, moreover, risk models may be able to detect rare but potentially potent risk factor predictions. Additionally, some of the important resource use predictors may not have been eligible in this proprietary system’s model development process. More extensive information on demographic or socioeconomic information would be useful to evaluate—as well as a more extensive set of clinical patient risk characteristics. The unidimensional view of quality using severity of illness assessment may limit the overall applicability of Table 4. R2N Values Independent Variables Dependent Variables CBP PCVP AMI Other procedure
Risk Factors
MedisGroups Mortality Modela
0.306 0.362 0.233 0.350
0.115 0.166 0.268 0.120
A comparison of the R2N values for the cost model built with risk factors and the cost model built with preoperative risk of death. a
Model covariates are a restricted cubic spline of the logit of preoperative risk of death. AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; PCVP ⫽ percutaneous cardiovascular procedures.
the cost– quality comparison analysis. Care providerbased variations were not accounted for in the analysis. Risk factors with low prevalence were not included in the analysis; therefore, we were not able to detect important, but rare risk factors that predict resource consumption. Either the RCC or DSS costing methodology can be used for internal resource utilization assessment purposes, as both costing approaches are highly correlated. However, the DSS costing approach provides more detailed information based on activity level accounting data. A catheterization procedure performed during coronary artery bypass graft procedure (DRG 106) increases costs, charges, and length of stay for CBP patients; however, the catheterization procedure does not enter the MedisGroups disease severity. Similarly, a failed PCVP procedure impacts costs and charges, but does not impact MedisGroups disease severity or length of stay. Moreover, an extremely high or low white blood cell count does not impact any resource use measures (cost, charge, or length of stay) but does impact MedisGroups disease severity assignment. Finally, admission type directly impacts all resource consumption measures; however, admission type is not incorporated into the MedisGroups model. Generally, different risk factors have different impacts on resource consumption and inhospital mortality within a DRG. The relationship between risk factor subsets and specific outcomes of care need to be evaluated separately. For the disease groups studied, the risk factors that predict in-hospital death are not the same as those that predict resource use. Moreover, the risk factors that predict length of stay are not the same as those that predict costs or charges. Preoperative risk of death did not predict resource utilization as well as the risk factors selected shown in Table 3 for each group except AMI. This finding is clearly indicated by the R2N in Table 4. Using a patient’s predicted preoperative risk of death will not predict his or her
710
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
resource utilization as well as the Cox regression models presented in this report. Based on this study’s findings, clinicians cannot confidently use risk models that predict mortality for the purposes of predicting resource consumption. Clinical decision support tools in the future will likely require a more extensive set of clinical and socioeconomic risk factors. This more extensive set of risk factors, therefore, can be used in prediction algorithms appropriately to project both population-based resource utilization and clinical outcomes for comparison purposes. To appropriately capture risk data to identify severity of illness and projected resource consumption for clinical decision-making purposes, it is likely that the risk factors commonly used to predict the risk of adverse outcomes might need to be supplemented. If appropriately reconciled to financial statements, then different costing methodologies may likely be used interchangeably in the internal decision-making process. This project was funded in part by the University Hospital Research Grants Program. Additionally, Dr Laurie Shroyer’s participation in this project was funded in part by the Department of Veterans Affairs Cooperative Studies Program. Doctor Samantha MaWhinney thanks the Alschuler, Grossman, and Pines Charitable Fund for contributing to her research.
References 1. Donabedian A, Wheeler JRC, Wyszewianski L. Quality, cost, and health: an integrative model. Med Care 1982;20:975–92. 2. Higgins T, Estafanous GF, Loop FD, Beck GJ, Blum JM, Paranandi L. Stratification of morbidity and mortality outcome by preoperative risk factors in coronary artery bypass patients: a clinical severity score. JAMA 1992;267:2344– 8. 3. Tremblay NA, Hardy JF, Perrault J, Carrier M. A simple classification of the risk in cardiac surgery: the first decade. Can J Anaesth 1993;40:103–11. 4. Lee TH, Juarez G, Cook EF, et al. Ruling out acute myocardial infarction: a prospective multicenter validation of a 12-hour strategy for patients at low risk. N Engl J Med 1991; 324:1239– 46. 5. Eagle KA. Medical decision-making in patients with chest pain. N Engl J Med 1991;324:1282– 6.
Ann Thorac Surg 2000;70:702–10
6. Hubbard BL, Gibbons RJ, Lapeyre AC, Zinsmeister AR, Clements IP. Identification of severe coronary artery disease using simple clinical parameters. Arch Intern Med 1992;152: 309–12. 7. Murray LS, Teasdale GM, Murray GD, et al. Does prediction of outcome alter patient management? Lancet 1993;341: 1487–91. 8. Knaus W. Ethical implications of risk stratification in the acute care setting. Camb Q Healthc Ethics 1993;2:193– 6. 9. Iezzoni LI, Shwartz M, Moskowitz MA, Ash AS, Sawitz E, Burnside S. Illness severity and costs of admission at teaching and nonteaching hospitals. JAMA 1990;264:1426–31. 10. Steen PM, Brewster AC, Bradbury RC, Estabrook E, Young JA. Predicted probabilities of hospital death as a measure of admission severity of illness. Inquiry 1993;30:128– 41. 11. Iezzoni LI, Moskowitz MA, Ash AS. The ability of MedisGroups and its clinical variables to predict cost and inhospital death. Technical Report 18-C-98526/1-04. Washington, DC: Health Care Financing Administration; 1988 July. 12. Colorado Health Data Commission. Colorado hospital outcomes: mortality, length of stay, and charges for cardiovascular and other diseases, 1992. Technical report. Denver: Colorado Health Data Commission; 1994 August. 13. Bradbury RC, Golec JH, Steen PM. Relating hospital health outcomes and resource expenditures. Inquiry 1994;31:56– 65. 14. Fleming ST. The relationship between quality and cost: pure and simple? Inquiry 1991;28:29–38. 15. Morey RC, Fine DJ, Loree SW, Retzlaff-Roberts DK, Tsubakitani S. The trade-off between hospital cost and quality of care: an exploratory empirical analysis. Med Care 1992;30: 677–98. 16. Obenchain RL, Johnstone BM. Mixed-model imputation of cost data for early discontinuers from a randomized clinical trial. Drug Information Journal 1999;33:191–209. 17. Dudley RA, Harrell FE, Smith LR, et al. Comparison of analytic models for estimating the effect of clinical factors on the cost of coronary artery bypass graft surgery. J Clin Epidemiol 1993;46:261–71. 18. Harrell FE. Design. S-Plus functions for biostatistical/ epidemiologic modeling, testing, estimation, validation, graphics, prediction, and typesetting by storing enhanced model design attributes in the fit. UNIX, Linux, and Microsoft Windows/NT versions available from lib.stat. cmu.edu, 1999. 19. Harrell FE. Predicting outcomes: applied survival analysis and logistic regression. Charlottesville, VA: University of Virginia Book Store, pp 6.1– 6.48. 20. Efron B, Tibshirani RJ. An introduction to the bootstrap. New York: Chapman & Hall, 1993:184 – 8.
Background. In this study we explored different risk model options to provide clinicians with predictions for resource utilization. The hypotheses were that predictors of mortality are not predictive of resource consumption, and that there is a correlation between cost estimates derived using a cost-to-charge ratio or a product-line costing approach. Methods. From March 1992 to June 1995, 2,481 University of Colorado Hospital patients admitted for ischemic heart disease were classified by diagnosis-related group code as having undergone or experienced coronary bypass procedures (CBP), percutaneous cardiovascular procedures (PCVP), acute myocardial infarction (AMI), and other cardiac-related discharges (Other). For each diag-
nosis-related group, Cox proportional hazards models were developed to determine predictors of cost, charges, and length of stay. Results. The diagnosis groups differed in the clinical factors that predicted resource use. As the two costing methods were highly correlated, either approach may be used to assess relative resource consumption provided costs are reconciled to audited financial statements. Conclusions. To develop valid prediction models for costs of care, the clinical risk factors that are traditionally used to predict risk-adjusted mortality may need to be expanded. (Ann Thorac Surg 2000;70:702–10) © 2000 by The Society of Thoracic Surgeons
T
product-line costing approach. Thus, either cost estimation approach may be used in analyses and reports for clinical decision-making purposes.
his study expands upon previous research efforts to provide clinicians with predicted outcomes and costs of care based on statistical models to enhance patient care decisions. A variety of predictive models are available to estimate a patient’s clinical outcomes following hospital admission for ischemic heart disease. Ideally, the clinical care team would select and implement the strategy of care that maximizes patient outcomes without wasting scarce resources. Therefore, information regarding both the patient’s medical outcomes and the estimated relative resource use associated with a set of alternative therapies become important considerations in the clinical team’s decision-making process. The study’s primary hypothesis was that there is no difference between the patient risk factors and procedural details that predict mortality and those that predict resource consumption. If this hypothesis is true, then the risk factors and procedural details included in current risk-adjusted mortality may be used to predict resource consumption measures such as costs, charges, and length of stay. The secondary study hypothesis was that there is a correlation between costs estimated by using a cost-tocharge ratio approach and costs estimated by using a Accepted for publication Mar 31, 2000. Address reprint requests to Dr Shroyer, Division of Cardiac Research, Denver VA Medical Center, 1055 Clermont St (151R), Denver, CO 80220; e-mail: [email protected].
© 2000 by The Society of Thoracic Surgeons Published by Elsevier Science Inc
Background Risk Model Predictions for Cost and Death “An ideal physician is defined as one who selects and implements the strategy of care that maximizes health status improvement without wasted resources” [1]. Both the probability of an outcome and the relative costs associated with a set of alternative therapies are important considerations in the decision to recommend a given therapy for a specific patient. Statistical model predictions are gaining popularity as clinical decision-making tools for patient management [2–5]. To date, these statistical models have focused primarily on the ability to predict the probability of adverse outcomes given a patient’s risk characteristics. Recent research has demonstrated that the introduction of a prediction risk model in clinical settings can alter patient management decisions [6, 7]. Statistical prediction models have not been used widely to estimate patient-specific costs, although the conceptual basis is similar to that for predicting outcomes. Given relatively comparable predicted clinical outcomes associated with a set of treatment alternatives, patient cost estimates should be helpful information to 0003-4975/00/$20.00 PII S0003-4975(00)01510-1
Ann Thorac Surg 2000;70:702–10
the clinician. This cost information may be particularly important in clinical decisions regarding high cost therapies (such as cardiac surgery). Greater precision in predicting outcome and costs following therapy may provide clinicians with reference points to augment decision making by increasing the level of information and communication [8]. No statistical risk model can replace clinical judgment, but good decision making must begin with good information. Hence, risk stratification represents a technological advancement that may be used to enhance the clinical decision-making process. Iezzoni and colleagues [9] explored the patient-specific risk factors that predict costs and quality of care. Their results (which included only four cardiac diagnosisrelated groups [DRGs] related to acute myocardial infarction/heart failure and shock) indicated that the MedisGroups risk-adjustment methodology, a method for assessing hospital admission severity of illness [10], performed well in predicting mortality. However, this risk adjustment method generally did poorly in predicting the cost of hospital admissions. Iezzoni and colleagues [9] also found that there was a positive relationship between risk score and cost. However, there was an inverse relationship between risk score and cost for patients dying in the hospital. The cost per day for the patients who died was higher than the cost per day for patients who lived. In summary, the MedisGroups riskassessment method could not be generically applied to both outcomes— cost and mortality rates. Iezzoni and colleagues [11] then built empirical models to predict cost and mortality rates from the patient risk characteristic data available. The most powerful risk factor predictors for costs differed from those that predicted the probability of in-hospital death. The risk factors selected for cost prediction tended to be less physiologic and more condition specific (eg, presence of congestive heart failure). In contrast, the most powerful predictors of death were more physiologic indicators of general function and less condition specific (eg, presence of renal dysfunction). Similarly, in the Colorado Health Data Commission’s 1992 Outcomes Report [12], the MedisGroups model used was a poor predictor of the observed variation in either charges or length of stay. Across the range of cardiac diagnoses reported, the prediction model explained only 7% of the variation in length of stay not previously explained by DRG adjustment alone. For charges, the risk adjustment at best explained up to 16% of the variation not previously explained by DRG adjustment alone. No estimation of costs was performed as part of this study.
Relationship Between Cost and Quality Conflicting empirical research findings on the relationship between cost and quality of medical care currently exists— demonstrating both positive and negative relationships [13–15]. Most of these studies hypothesized simple linear relationships between cost and quality. However, Donabedian and colleagues [1] suggested in 1982 that marginal cost may vary over the range of quality depending on the preexisting level of quality of
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
703
care provided. The cost– quality relationship may be nonlinear in nature.
Material and Methods Patient Population For this study, the population consisted of 2,481 patients from the University Hospital in Denver, Colorado, who had had admissions for ischemic heart disease-related diagnoses during the period from March 1, 1992 to June 30, 1995. Data for this project were obtained on 2,420 patient records (97.5% of the original 2,481) from four different databases including a medical records system, a clinical severity classification system, a patient billing system, and a product-line costing system. Records were dropped from the study for the following reasons: patients were under the age of 18 years; records did not meet study inclusion criteria for the DRG set examined related to a reported diagnosis of ischemic heart disease; or the data on the patient were unavailable from one or more of the required databases. Moreover, patient records with extreme charge or cost data were reviewed. After administrative review, these very extreme outliers (above the 99% threshold of charges or cost) were dropped due to concerns about potential errors and inconsistencies found in the billing or costing data reported. In total, 61 records were removed from the analysis.
Data Sources Data from the four databases were merged. Before proceeding with the analyses, all data quality or completeness discrepancies identified (such as mismatching dates of service) were resolved and updated. To address these issues, the patient charts were pulled as needed. For costing analysis purposes, the audited Medicare Hospital Cost Report was used to generate five revenue categoryspecific cost-to-charge ratios; the categories were routine, pharmacy, laboratory, radiology, and all other costs (eg, intensive care unit costs). The reformat routine used to reclassify the detailed billing charge codes into these five summary charge categories was the standard conversion used by the Colorado Hospital Association. The costs for all records were initially estimated by multiplying the hospital-specific charges in these five revenue categories by the weighted average ratio of cost to charges (RCC) for the corresponding hospital cost centers. For the period from March 1, 1992 to June 30, 1993, the University Hospital did not estimate costs using the product-line costing system. For discharges from July 1, 1993 to June 30, 1995, the inpatient hospital costs were derived from the fully costed patient level databases in the University Hospital Decision Support System (DSS). This system applies an industrial cost accounting approach to the healthcare processes to arrive at a detailed intermediate product costing at the patient level. All charges and costs were adjusted appropriately for inflation (based on the Denver regional consumer price index for medical care goods and services) to the initial
704
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
Ann Thorac Surg 2000;70:702–10
Table 1. Patient Baseline Characteristics and In-Hospital Clinical Outcomes for Patient Discharges Variable Number/percent of discharges Age (y) Median, (1st, 3rd quartile) Mean, SD Gender Female (%) Race White (%) Black (%) Hispanic (%) Other/unknown (%) Admit source ER (%) Admission office (%) Outpatient (%) Other hospital, acute (%) Other (%) Admission type Urgent/emergent (%) Prior failed PCVP (%) CAD left main ⬎ 49% (%) CHF (%) Age unspecified MI (%) CPK ⬎ 150 u/l (%) Ejection fraction ⬍ 41% (%) MedisGroups/Atlas risk estimate Median Risk Estimate 1st, 3rd quartile Mean risk estimate In-hospital death Number Percent MedisGroups Mortality Model C-Index 95% confidence interval
CBP
PCVP
AMI
All Other
Total
221/9%
1,046/43%
183/8%
970/40%
2,420
61 (54, 68) 60.7, 10.7
55 (45, 66) 54.7, 15.1
61 (53, 72) 61.7, 13.8
56 (47, 65) 56.2, 13.2
57 (47, 66) 56.4, 14.0
23.1
35.1
34.1
42.1
36.7
72.4 6.8 15.8 5.0
80.2 6.5 8.7 4.6
71.0 13.1 12.0 3.9
71.2 12.1 11.8 4.9
75.2 9.3 10.8 4.7
24.0 17.7 30.3 26.2 1.8
27.0 21.0 29.2 21.7 1.0
82.5 2.2 3.3 9.8 2.2
60.7 11.7 20.2 6.4 1.0
15.5 44.4 23.8 15.1 1.2
49.8 3.6 15.4 20.8 31.7 34.4 6.3
49.1 9.3 1.5 6.9 17.4 16.1 3.3
90.7a 1.6 1.6 19.1 49.7 77.6 7.1
69.2 1.6 2.6 10.6 12.2 15.8 8.1
39.6a 5.1 3.2 10.6 19.1 22.3 5.8
0.012 0.007, 0.022 0.024
0.004 0.004, 0.013 0.018
0.033 0.011, 0.124 0.118
0.008 0.004, 0.018 0.025
0.008 0.004, 0.018 0.029
10 4.52 0.71 (0.42, 0.87)
15 1.43 0.90 (0.69, 0.97)
19 10.38 0.89 (0.72, 0.95)
5 0.52 0.88 (0.73, 0.99)
49 2.02 0.89 (0.82, 0.93)
Four categories include: coronary bypass procedures (CBP) (DRGs 106 and 107), percutaneous cardiovascular procedures (PCVP) (DRG 112), acute myocardial infarction (AMI) (DRGs 121, 122, 123), and other coronary artery disease-related discharges (Other) (DRGs 124, 125, 140). a
AMI N ⫽ 182;
Total n ⫽ 2,419.
CAD ⫽ coronary artery disease; infarction.
CHF ⫽ congestive heart failure;
CPK ⫽ creatine phosphokinase;
study periods of March 1992. Both the Medicare Hospital Cost Report data and the product-line costing data were confirmed to reconcile to the hospital’s period-specific financial statements. In general, the DSS information was more detailed based on the actual activities performed (including direct labor expenses) and supplies used (including direct supply expenses) in comparison with the RCC approach. Given the greater accounting detail used to derive DSS costs, it was thought that perhaps the DSS costing methodology would provide greater precision for the purposes of this project. Thus, the RCC costs for the period from July 1, 1993 to June 30, 1995 were subsequently compared with the DSS product-line costing estimates obtained using linear regression techniques.
ER ⫽ emergency room;
MI ⫽ myocardial
Classification of Patients for Study Purposes Based on DRG coding assignment, patients were classified into four categories: coronary bypass procedures (CBP) (DRGs 106 and 107), percutaneous cardiovascular procedures (PCVP) (DRG 112, which includes procedures other than percutaneous transluminal coronary angioplasty [PTCA]), acute myocardial infarction (AMI) (DRGs 121, 122, 123), and other coronary artery disease-related discharges (Other) (DRGs 124, 125, 140). The clinical severity classification system (MedisGroups/Atlas) automatically assigned a severity estimate to each patient record using patient-specific risk variables. This severity assignment was used unaltered for purposes of this project.
Ann Thorac Surg 2000;70:702–10
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
Statistical Analysis
Table 2. Resource Consumption Summary Data
There are several features of Cox proportional hazards that make it superior for analyzing cost, charge, and length of stay data. First, an important feature of the Cox model is its lack of distributional assumptions. This feature is especially important when looking at highly skewed data such as cost and length of stay. Because the Cox model can accommodate right censoring, patients with incomplete or censored observations can be included in the analysis. For the prediction of resource consumption, it is important to include patients who have incomplete data due to an in-hospital death. These patients may be assumed to be high risk; therefore, leaving them out of the analysis could lead to an underestimation of the impact of high-risk patient characteristics. Also, as the quality of patient care improves, patients who would have died in the past can be expected to survive. Therefore, the contributions of these high-risk patients to cost, length of stay, and charge may be expected to increase in the future. Although there may be a bias due to informative censoring [16], Cox models have been shown to be superior for cost and charge data over traditional methods such as a linear regression that treats censored costs as if they were uncensored [17]. Cox proportional hazards models were developed for length of stay, costs, and charges using S-Plus statistical software (StatSci Software, Seattle, WA) and Harrell’s Design Library [18]. Ideally, the model would be built by first postulating the important predictors and then fitting the model. To reduce the number of risk factors considered in the final model, variables were restricted to those judged to be clinically relevant. These risk factors were further restricted to those present in at least 10% of the patients. A univariate Cox model was then used to further screen the predictors and only those that were significant at p values less than or equal to 0.1 were considered for inclusion in the multivariate model. The predictor, “admission type,” was forced into all models and an indicator of current admission catheterization was forced into all CBP models. Then a backward selection process was used to obtain the final model assuming a significance level of p values less than or equal to 0.1. The Cox model assumes linearity of covariates and proportional hazards [19]. To test the linearity assumption, each covariate was plotted against the martingale residuals from a model built without the covariate. A smoothed line was added, to provide graphical information about the functional form of the variable. Proportional hazards were assessed both graphically and statistically. Our graph of the Schoenfeld residuals showed no significant slope over time. This was supported by a statistical test of this slope. Therefore, there is no violation of proportional hazards in our models. The models were then validated using resampling validation techniques. All models performed well. For logistic regression analyses, the c-index is often used as a measure of risk model performance. The c-index represents the area under the receiver operating characteristic curve (that reflects the relative sensitivity
Variable
CBP
PCVP
AMI
705
Other Total
Total length of stay (days) Mean 11.40 3.68 6.16 3.70 Median 9 3 5 2 Charges (dollars) Mean 61,585 17,091 17,873 11,025 Median 52,932 13,608 14,701 7,761 Percent charges Routine (%) 17.2 15.6 27.1 23.0 Laboratory (%) 14.3 9.1 18.1 17.1 Pharmacy (%) 14.4 7.9 13.7 13.1 Radiology (%) 2.0 4.5 5.6 6.6 Other (%) 52.1 62.9 35.5 40.2 DSS cost estimates (dollars) Mean 27,091 8,982 9,421 5,894 Median 24,577 7,418 8,208 4,254
4.59 3 18,818 12,170 18.7 13.2 11.5 4.3 52.4
9,445 6,793
AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; DSS ⫽ University Hospital Decision Support System; Other ⫽ other cardiac-related discharges; PCVP ⫽ percutaneous cardiovascular procedures.
and specificity of the model in predicting risk of operative death). Theoretically, the c-index may range from 0 to 1. In general, a c-index of 0.5 is useless for prediction purposes. Confidence intervals for the c-indices were calculated using BCa bootstrap confidence intervals [20].
Results General Findings The base line inpatient risk characteristics and inhospital mortality for the four different DRG categories and overall are presented in Table 1. An analysis of variance determined that the MedisGroups risk model did not predict the risk of in-hospital death equivalently across the four diagnosis groups (p ⬍ 0.001). Likewise the in-hospital death rates were not the same for the four diagnosis groups when compared using Fisher’s exact test (p ⬍ 0.001). The clinical classification system for CBP patients does not discriminate well, as the c-index confidence intervals contain the value of 0.5. The summary descriptive statistics for the resource utilization findings for the different DRG categories are listed in Table 2. Given the inherent differences in the prevalence of patient risk characteristics and calculated severity of illness among categories, as well as the inherent variations in resource utilization, Cox proportional hazards models were developed for each DRG category separately.
Comparison of Different Costing Approaches To answer the question, “Can RCC costs be used to impute missing values of DSS cost in cost analysis?,” the data set was split randomly into model building and model validating data sets. We first performed a simple linear regression with RCC cost as the predictor variable
706
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
Ann Thorac Surg 2000;70:702–10
Table 3. Risk Factors in Final Models for Cost, Charge, and Length of Stay (Compared to MedisGroups/Atlas Mortality Model) CBP Risk Factors Acute Admission Type WBC ⬍ 5.0 or ⬎ 17.0 Age unspecified MI Dilated chambers Pulmonary edema Failed PCVP Decreased contractility CAD left main ⬎ 49% S3 gallop Pleural effusion CPK ⬎ 150 U/L Chronic Pulse ⬍ 40, ⬍ 65 or ⬎ 129 Sys BP ⬍ 60, ⬍ 90, or ⬎ 119 Pulmonary vascular disease Regurgitation CAD LAD ⬎ 49% Murmur (protective) Acute and/or chronic Atrial fibrillation CHF BUN ⬎ 30 mg/100 mL Ejection fraction ⬍ 41% Creatinine ⬎ 1.7 mg/dl O2 Saturation ⬍ 86% Other Catheterization (CBP only) Age (Older)
Mort.
x x
PCVP
AMI
Other
LOS
Cost
Chg
LOS
Cost
Chg
LOS
Cost
Chg
LOS
Cost
Chg
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x x x x x x
x
x
x
x x
x x
x x
x
x
x
x x x
x
x
x
x x
x x x
x x
x x x x
x x x x
x x x x
x x x x
x
x
x
x
x x
x x
x x
x
x
x
x
x
x
x x
x
x
x
x x x x
x x
x x
x
x x
x x
x x
x
x
x
x
x
x
x x
x x
AMI ⫽ acute myocardial infarction; BP ⫽ blood pressure; BUN ⫽ blood urea nitrogen; bypass procedures; CHF ⫽ congestive heart failure; CPK ⫽ creatine phosphokinase; myocardial infarction; LOS ⫽ length of stay; Other ⫽ other cardiac-related discharges; WBC ⫽ white blood cells.
and DSS cost as the response variable. This model had an R2adj (adjusted R2) value of 0.955, but the residuals were not normally distributed and their variance increased as the fitted value of DSS increased. In an attempt to normalize and account for the pattern in the residuals, we used a log10 transformation for both DSS and RCC costs. This transformation resulted in a random scatter of the residuals and the model fit the validating data set well. The residuals from this model were close to normally distributed, but with heavy tails. The R2adj for this model is 0.958, a slight increase above the model with no transformations. To bring in the tails of the distribution of the residuals, a squared term, (log10(RCC))2, was then added to the prediction model. Although this model sacrifices interpretability, the primary use is for prediction purposes. For the extreme values, this log-transformed, quadratic model was superior to the simpler models. The R2adj for this model is 0.962. The formula for the fitted line is
CAD ⫽ coronary artery disease; CBP ⫽ coronary LAD ⫽ left anterior descending artery; MI ⫽ PCVP ⫽ percutaneous cardiovascular procedures;
log 10(DSS) ⫽ ⫺1.5579⫹1.8559 ⴱ log 10(RCC) ⫺0.1161 ⴱ (log 10(RCC))2 In summary, the two different costing methodologies were highly correlated. Thus, either of the two different costing methodologies may be used for purposes of estimating costs for relative cost comparisons of these alternative therapies in clinical decisions.
Comparison of Models to Predict Cost and Death For each model developed, the patient risk characteristics that were found in the different models to be statistically important predictors by DRG category are listed in Table 3. The risk factors that were found to be predictive of resource consumption were generally not the same as those used to predict mortality. Table 3 compares the variables that were statistically significant in the various models reported. The risk factors were divided into the categories of acute, chronic, acute and/or chronic, and other. For reference, the first
Ann Thorac Surg 2000;70:702–10
Fig 1. Hazard ratios for total cost for coronary bypass patients. The hazard ratios for each risk variable are plotted for the cost model for the diagnosis-related group (DRG) category coronary bypass. A bar that denotes the 95% confidence interval surrounds each hazard ratio. The outcome event is the cost of care from admission to discharge. The graph indicates that patients at higher risk have a lower hazard for discharge and cost termination and therefore a higher expected cost. (BP ⫽ blood pressure; CHF ⫽ congestive heart failure; MI ⫽ myocardial infarction; PTCA ⫽ percutaneous transluminal coronary angioplasty.)
column indicates which of these variables were also significant in the MedisGroups/Atlas mortality model for the ischemic heart disease diagnosis. Admission type was categorized into two groups, elective and urgent/emergent. This variable was included in all the models. For each group an urgent/emergent
Fig 2. Hazard ratios for total charges for coronary bypass patients. The hazard ratios for each risk variable are plotted for the charge model for the diagnosis-related group (DRG) category coronary bypass. A bar that denotes the 95% confidence interval surrounds each hazard ratio. The outcome event is the charges for care from admission to discharge. The graph indicates that patients at higher risk have a lower hazard for discharge and charge termination and therefore higher expected charges. (BP ⫽ blood pressure; CHF ⫽ congestive heart failure; MI ⫽ myocardial infarction; PTCA ⫽ percutaneous transluminal coronary angioplasty.)
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
707
Fig 3. Hazard ratios for length of stay for coronary bypass patients. The hazard ratios for each risk variable are plotted for the length of stay model for the diagnosis-related group (DRG) category coronary bypass. A bar that denotes the 95% confidence interval surrounds each hazard ratio. The outcome event is time to discharge. The graph indicates that patients at higher risk have a lower hazard for discharge and therefore a longer expected length of stay. (BUN ⫽ blood urea nitrogen; MI ⫽ myocardial infarction.)
admission resulted in a longer length of stay, increased cost, and increased charges. The graphs of the hazard ratios for each risk variable in each of the three models for the DRG category CBP are presented in Figures 1 through 3. In Figures 1 through 3, the outcome events for cost, charges, and length of stay are the cost of care from admission to discharge, the total charges from admission to discharge, and time to discharge, respectively. The length of stay graph (Fig 3) can be interpreted such that patients at higher risk have a lower hazard of discharge and therefore a longer length of stay. The hazard ratio of emergent to non-emergent classification at admission is less than 1 because, as expected, the hazard of discharge is larger for the nonemergent patients. Similarly, patients with a catheterization at the current admission will experience a longer length of stay. As the hazard of discharge is higher for the patients who were not catheterized during the current admission, this result is demonstrated by a hazard ratio of discharge that is greater than 1 for the patients with no catheterization (DRG 107, coronary bypass without cardiac catheterization) compared with catheterization (DRG 106, coronary bypass with cardiac catheterization). For reference, the Kaplan-Meier survival curves for each DRG group for total cost, total charges, and length of stay are presented in Figures 4 through 6.
Relationship of Severity of Illness Estimates to Predicted Resource Utilization Cox models were fit to determine the contribution of the preoperative risk of death (MedisGroups mortality) as a predictor variable. The risk variable was first linearized by taking the logit. The logit of the risk was then allowed to be nonlinear in the model by using the restricted cubic spline components as predictors. All models satisfied the linearity and proportional hazards assumptions. We use
708
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
Ann Thorac Surg 2000;70:702–10
Fig 4. Kaplan-Meier curves for total cost by diagnosis-related group categories, truncated at $50,000. The symbols indicate an in-hospital death or censored observation. (AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; Other ⫽ other cardiac-related discharges; PCVP ⫽ percutaneous cardiovascular procedures.)
Nagelkerke’s R2 (R2N) as a measure of predictive ability for the Cox model. An increase in the R2N between models with the same outcome indicates an improved model fit [19]. The R2N value was obtained for these models then compared to the models built with risk factors. The results are reported in Table 4. These results indicate that for CBP, PCVP, and Other patients using risk factors to predict cost gives more information than using only the preoperative risk of death. For these data, building a model specifically to predict resource utilization is preferable to utilizing existing mortality model predictions to estimate resource use. For the AMI group, the R2N actually improved slightly in the model based solely on the MedisGroups mortality. Fig 5. Kaplan-Meier curves for total charges by diagnosis-related group categories, truncated at $100,000. The symbols indicate an in-hospital death or censored observation. (AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; Other ⫽ other cardiac-related discharges; PCVP ⫽ percutaneous cardiovascular procedures.)
Comment This single-center study focused exclusively on cardiac care patient discharges and was conducted in a university-based setting that serves as a tertiary referral care center for the state of Colorado. Therefore, generalizations to other patient populations may not be appropriate. MedisGroups severity adjustment approach is developed on national data set, but in this study had been applied to local data. In this context, model performance metrics may be compromised. For interventions, the MedisGroups adjustment was based on an assessment at time of admission rather than time of intervention. Thus, the timeliness of the risk factor assessment may be
Ann Thorac Surg 2000;70:702–10
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
709
Fig 6. Kaplan-Meier curves for length of stay by diagnosis-related group categories, truncated at 40 days. Due to the discreteness of the data, censoring symbols are not included. (AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; Other ⫽ other cardiac-related discharges; PCVP ⫽ percutaneous cardiovascular procedures.)
questioned. Additionally, the MedisGroup data entry default was a negative assessment. If a given risk factor was not coded, then this result was interpreted as a negative clinical finding. Ideally, a risk model to predict in-hospital mortality would be developed from this data set. As noted, the four diagnoses groups have different incidence of death. However, sample size limitations precluded the development of a model to predict death for this study; risk factors evaluated were limited predominantly to the MedisGroups key clinical findings abstracted and DRG data. With a larger data set, moreover, risk models may be able to detect rare but potentially potent risk factor predictions. Additionally, some of the important resource use predictors may not have been eligible in this proprietary system’s model development process. More extensive information on demographic or socioeconomic information would be useful to evaluate—as well as a more extensive set of clinical patient risk characteristics. The unidimensional view of quality using severity of illness assessment may limit the overall applicability of Table 4. R2N Values Independent Variables Dependent Variables CBP PCVP AMI Other procedure
Risk Factors
MedisGroups Mortality Modela
0.306 0.362 0.233 0.350
0.115 0.166 0.268 0.120
A comparison of the R2N values for the cost model built with risk factors and the cost model built with preoperative risk of death. a
Model covariates are a restricted cubic spline of the logit of preoperative risk of death. AMI ⫽ acute myocardial infarction; CBP ⫽ coronary bypass procedures; PCVP ⫽ percutaneous cardiovascular procedures.
the cost– quality comparison analysis. Care providerbased variations were not accounted for in the analysis. Risk factors with low prevalence were not included in the analysis; therefore, we were not able to detect important, but rare risk factors that predict resource consumption. Either the RCC or DSS costing methodology can be used for internal resource utilization assessment purposes, as both costing approaches are highly correlated. However, the DSS costing approach provides more detailed information based on activity level accounting data. A catheterization procedure performed during coronary artery bypass graft procedure (DRG 106) increases costs, charges, and length of stay for CBP patients; however, the catheterization procedure does not enter the MedisGroups disease severity. Similarly, a failed PCVP procedure impacts costs and charges, but does not impact MedisGroups disease severity or length of stay. Moreover, an extremely high or low white blood cell count does not impact any resource use measures (cost, charge, or length of stay) but does impact MedisGroups disease severity assignment. Finally, admission type directly impacts all resource consumption measures; however, admission type is not incorporated into the MedisGroups model. Generally, different risk factors have different impacts on resource consumption and inhospital mortality within a DRG. The relationship between risk factor subsets and specific outcomes of care need to be evaluated separately. For the disease groups studied, the risk factors that predict in-hospital death are not the same as those that predict resource use. Moreover, the risk factors that predict length of stay are not the same as those that predict costs or charges. Preoperative risk of death did not predict resource utilization as well as the risk factors selected shown in Table 3 for each group except AMI. This finding is clearly indicated by the R2N in Table 4. Using a patient’s predicted preoperative risk of death will not predict his or her
710
MaWHINNEY ET AL IDENTIFICATION OF RISK FACTORS
resource utilization as well as the Cox regression models presented in this report. Based on this study’s findings, clinicians cannot confidently use risk models that predict mortality for the purposes of predicting resource consumption. Clinical decision support tools in the future will likely require a more extensive set of clinical and socioeconomic risk factors. This more extensive set of risk factors, therefore, can be used in prediction algorithms appropriately to project both population-based resource utilization and clinical outcomes for comparison purposes. To appropriately capture risk data to identify severity of illness and projected resource consumption for clinical decision-making purposes, it is likely that the risk factors commonly used to predict the risk of adverse outcomes might need to be supplemented. If appropriately reconciled to financial statements, then different costing methodologies may likely be used interchangeably in the internal decision-making process. This project was funded in part by the University Hospital Research Grants Program. Additionally, Dr Laurie Shroyer’s participation in this project was funded in part by the Department of Veterans Affairs Cooperative Studies Program. Doctor Samantha MaWhinney thanks the Alschuler, Grossman, and Pines Charitable Fund for contributing to her research.
References 1. Donabedian A, Wheeler JRC, Wyszewianski L. Quality, cost, and health: an integrative model. Med Care 1982;20:975–92. 2. Higgins T, Estafanous GF, Loop FD, Beck GJ, Blum JM, Paranandi L. Stratification of morbidity and mortality outcome by preoperative risk factors in coronary artery bypass patients: a clinical severity score. JAMA 1992;267:2344– 8. 3. Tremblay NA, Hardy JF, Perrault J, Carrier M. A simple classification of the risk in cardiac surgery: the first decade. Can J Anaesth 1993;40:103–11. 4. Lee TH, Juarez G, Cook EF, et al. Ruling out acute myocardial infarction: a prospective multicenter validation of a 12-hour strategy for patients at low risk. N Engl J Med 1991; 324:1239– 46. 5. Eagle KA. Medical decision-making in patients with chest pain. N Engl J Med 1991;324:1282– 6.
Ann Thorac Surg 2000;70:702–10
6. Hubbard BL, Gibbons RJ, Lapeyre AC, Zinsmeister AR, Clements IP. Identification of severe coronary artery disease using simple clinical parameters. Arch Intern Med 1992;152: 309–12. 7. Murray LS, Teasdale GM, Murray GD, et al. Does prediction of outcome alter patient management? Lancet 1993;341: 1487–91. 8. Knaus W. Ethical implications of risk stratification in the acute care setting. Camb Q Healthc Ethics 1993;2:193– 6. 9. Iezzoni LI, Shwartz M, Moskowitz MA, Ash AS, Sawitz E, Burnside S. Illness severity and costs of admission at teaching and nonteaching hospitals. JAMA 1990;264:1426–31. 10. Steen PM, Brewster AC, Bradbury RC, Estabrook E, Young JA. Predicted probabilities of hospital death as a measure of admission severity of illness. Inquiry 1993;30:128– 41. 11. Iezzoni LI, Moskowitz MA, Ash AS. The ability of MedisGroups and its clinical variables to predict cost and inhospital death. Technical Report 18-C-98526/1-04. Washington, DC: Health Care Financing Administration; 1988 July. 12. Colorado Health Data Commission. Colorado hospital outcomes: mortality, length of stay, and charges for cardiovascular and other diseases, 1992. Technical report. Denver: Colorado Health Data Commission; 1994 August. 13. Bradbury RC, Golec JH, Steen PM. Relating hospital health outcomes and resource expenditures. Inquiry 1994;31:56– 65. 14. Fleming ST. The relationship between quality and cost: pure and simple? Inquiry 1991;28:29–38. 15. Morey RC, Fine DJ, Loree SW, Retzlaff-Roberts DK, Tsubakitani S. The trade-off between hospital cost and quality of care: an exploratory empirical analysis. Med Care 1992;30: 677–98. 16. Obenchain RL, Johnstone BM. Mixed-model imputation of cost data for early discontinuers from a randomized clinical trial. Drug Information Journal 1999;33:191–209. 17. Dudley RA, Harrell FE, Smith LR, et al. Comparison of analytic models for estimating the effect of clinical factors on the cost of coronary artery bypass graft surgery. J Clin Epidemiol 1993;46:261–71. 18. Harrell FE. Design. S-Plus functions for biostatistical/ epidemiologic modeling, testing, estimation, validation, graphics, prediction, and typesetting by storing enhanced model design attributes in the fit. UNIX, Linux, and Microsoft Windows/NT versions available from lib.stat. cmu.edu, 1999. 19. Harrell FE. Predicting outcomes: applied survival analysis and logistic regression. Charlottesville, VA: University of Virginia Book Store, pp 6.1– 6.48. 20. Efron B, Tibshirani RJ. An introduction to the bootstrap. New York: Chapman & Hall, 1993:184 – 8.