- Email: [email protected]
A new comorbidity index: the health-related quality of life comorbidity index
Journal of Clinical Epidemiology 64 (2011) 309e319
A new comorbidity index: the health-related quality of life comorbidity index Bhramar Mukherjeea, Huang-Tz Oub, Fei Wanga, Steven R. Ericksonb,* b
a Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, MI 48109, USA Department of Clinical, Social, and Administrative Sciences, College of Pharmacy, University of Michigan, Ann Arbor, MI 48109, USA
Accepted 31 January 2010
Abstract Objective: To derive and validate the health-related quality of life comorbidity index (HRQL-CI). Study Design and Setting: Of 261 clinical classification codes (CCCs) in the 2003 Medical Expenditure Panel Survey (MEPS), 44 were identified as adult, gender-neutral, chronic conditions. The least absolute shrinkage and selection operator (LASSO) procedure identified CCCs significantly associated with the Short Form-12 physical component summary (PCS) and mental component summary (MCS) scores. Regression models were fitted with the selected CCCs, resulting in two subsets corresponding to PCS and MCS, collectively called the HRQL-CI. Internal validation was assessed using 10-fold cross-validation, whereas external validation in terms of prediction accuracy was assessed in the 2005 MEPS database. Prediction errors and model R2 were compared between HRQL-CI models and models using the Charlson-CI. Results: LASSO identified 20 CCCs significantly associated with PCS and 15 with MCS. The R2 for the models, including the HRQLCI (0.28 for PCS and 0.16 for MCS) were greater than those using the Charlson-CI (0.13 for PCS and 0.01 for MCS). The same pattern of higher R2 for models using the HRQL-CI was observed in the validation tests. Conclusion: The HRQL-CI is a valid risk adjustment index, outperforming the Charlson-CI. Further work is needed to test its performance in other patient populations and measures of HRQL. Ó 2011 Elsevier Inc. All rights reserved. Keywords: Health-related quality of life; Comorbidity; Charlson comorbidity index; Risk prediction models; LASSO; SF-12; Health status
1. Introduction Comorbidity is the existence or occurrence of any distinct additional disease or diseases during the clinical course of a patient who has an index disease under study. A comorbidity index (CI) is a weighted measure that, when conducting statistical analyses, will control for the potential influence of those illness on an outcome of interest [1,2]. A relatively uncomplicated method of controlling for comorbidity is to use the simple count of illnesses [3]. Diagnosis methods often
This project was funded by the Agency for Health Care Research and Quality, R03 grant 1 R03 HS017461-01A1 ‘‘Developing a Comorbidity Index for Health-Related Quality of Life Studies,’’ Steven R. Erickson PI, from September 1, 2008 to August 31, 2009. Abstract of this article was accepted for poster presentation for the October 2009 meeting of the International Society for Quality of Life Research, New Orleans, LA. * Corresponding author. College of Pharmacy, University of Michigan, 428 Church Street, Ann Arbor, MI 48109-1065, USA. Tel.: þ734-7634989; fax: þ734-763-2022. E-mail address: [email protected] (S.R. Erickson). 0895-4356/$ - see front matter Ó 2011 Elsevier Inc. All rights reserved. doi: 10.1016/j.jclinepi.2010.01.025
use data available from a medical record derived from hospital stays. These are known as discharge-based CIs [4e10]. A common diagnosis-based method was derived by Charlson et al. [11]. This method was initially developed as a chartbased weighted index that provides a simple and valid method for estimating the risk of death associated with comorbid illness, taking into account both the number and seriousness of comorbid illnesses. Other versions of the Charlson-CI use data derived from electronic claims data, electronic hospital records, and patient self-report [4,12]. CIs are used in health services research to control for confounding in observational studies and for risk adjustment in studies of health care quality [13e16]. Comorbidity affects mortality [17e21], health resource utilization [22e24], admission and readmission to hospital [20,22,25], and healthrelated quality of life (HRQL) or functional status [26e28]. Without adequate measures to adjust for intervening comorbidity differences, valid comparisons of health status and HRQL outcomes in population studies cannot be made [15]. Large nationally representative data sets, such as the National Health and Nutrition Examination Survey and the
310
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
2. Methods What is new? Key findings A new risk adjustment index was derived for use in health-related quality of life (HRQL) studies. What this adds to what is known? The new HRQL comorbidity index (HRQL-CI) provides better predictive ability for HRQL measures than the commonly used Charlson-CI. What is the implication, what should change now? Researchers have a comorbidity risk adjustment index to control for the potential influence of other illnesses when studying the effect of specific diseases or other risk factors associated with HRQL measures, which outperforms commonly used Charlson-CI.
Medical Expenditure Panel Survey (MEPS) include data derived from health status or HRQL instruments. These measures have and will continue to be used as primary or secondary outcome variables in analyses. It is therefore of interest to control for the presence of comorbidity when using these measures to study HRQL attributed to specific illnesses. HRQL refers to the physical, emotional, and social impact of disease and related treatments and is distinct from physiologic measures of disease [29,30]. Generally, HRQL decreases with increasing comorbidity [31e35]. Two types of questionnaires are used to measure HRQL, general and disease- or intervention-specific. General measures assess concepts that are relevant to a wide range of people, including ability to function in everyday life and emotional wellbeing [36]. They are not specific to any age, disease, or treatment group and are designed to be broadly applied across different populations to allow for comparisons across many conditions [37e40]. Researchers have validated measures of comorbidity by how well they predict mortality, health resource use, and expenditures, either as a predictor themselves or to adjust for the contributing effects of other diseases when studying the association of a specific illness. Comparatively little research has been conducted validating existing CIs with HRQL or health status as the primary outcome variable. Moreover, few studies have been conducted to develop and validate HRQL- or health status-specific CI. The purpose of this study was to derive and validate a CI using diseases that have the greatest association with HRQL. The secondary goal was to compare the results of explanatory models that use the new index derived specifically for HRQL with a CI originally derived to predict mortality and health care resource use, the Charlson-CI using MEPS database.
2.1. General description of the data sets used for the study The MEPS is a nationally representative, public domain data set maintained by the Agency for Healthcare Research and Quality. The MEPS data set was considered an appropriate data source because of the reliability and validity of the data, in particular the disease, treatment, and HRQL data. Further specific information may be found at http:// www.meps.ahrq.gov/mepsweb/. The 2003 Full-Year Consolidated Data File (HC-079) and the 2003 Medical Conditions File (HC-078) were used to derive the HRQL-CI. The HC-079 and HC-078 data files contain person-level data from the respondent and others in the home for two panels during the full year of 2003. All medical conditions related to a reported encounter with a health care professional or institution for the year of 2003 are documented for each respondent and proxy. To validate the HRQL-CI, the 2005 Full-Year Consolidated Data File (HC-097) and the 2005 Medical Conditions File (HC-096) were used. Both HC-079 and HC-097 Full-Year Consolidated Data Files include the Short Form-12 (SF12) HRQL survey, version 2.0, the source of HRQL data for this study. The variable DUPERSID is common to the Full-Year Consolidated data files and the Medical Conditions data files. It serves to link person-specific data between all MEPS data files. There is no overlap of people between the 2003 and 2005 databases ensuring that the validation set is independent of the model development or training set. 2.2. Diagnoses or conditions data Diagnoses data were derived from the Medical Conditions Files HC-078 (2003) for index development and HC-096 (2005) for index validation. Included in these files are the ICD-9 codes that correspond to health resource use data for each respondent and proxy. ICD-9 codes are reduced to the first three digits for confidentiality reasons. Similar ICD-9 codes are grouped into clinical classification codes (CCCs), which represent groupings of clinically similar medical conditions. Most CCCs in the Medical Conditions files are related to actual medical conditions, although a few are reserved for administrative uses. The CCCs were used for this study. There are 261 unique CCCs in the 2003 Medical Conditions File (HC-078). Each CCC is labeled by a three-digit number and the frequency of occurrence within the data set is provided by MEPS. Not all CCCs were appropriate to use for this study. Figure 1 provides the sequential approach used to exclude and condense CCCs from the original 261. A total of 164 CCCs were initially excluded and are listed in Appendix A (see appendix on the journal’s Web site at www. elsevier.com) with the reason for exclusion. An additional
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
Health-Related Quality of Life Comorbidity Index Development Step MEPS 2003 HC-078 Medical Conditions File 261 Clinical Classification Codes (CCC) Excluded CCCs – Refer to Appendix A Administrative Codes (n = 4) Acute Infections, Poisonings, Injury, Trauma (n = 84) Gender-specific (n = 32) Pediatric/Maternity/Intrauterine (n = 27) Cognitive Impairment (n = 5) Rare conditions, less than 10 people Or mixed acute/few chronic (n = 10) Result: 164 CCCs excluded
97 CCCs remained. Of these, 7 CCCs did not have persons documented with the condition. These CCCs were removed, leaving 90 CCCs. Remaining 90 CCCs submitted to further condensing because of similar organ-system involvement/pathophysiology. Refer to Appendix B. Of these, 64 CCCs were condensed to 18 CCCs. Thus, we have 90– 64+18=44 CCCs Final Working CCC set consisted of 44 CCCs derived from the 2003 HC-078 Medical Conditions file Fig. 1. Flow chart for selection of CCCs for analysis. Abbreviations: MEPS, medical expenditure panel survey; CCS, clinical classification codes.
seven CCCs were considered appropriate to be included but were not present in the 2003 Medical Conditions File because of no person having this condition. The 90 remaining CCCs were examined for similarity in clinical attributes in an effort to combine CCCs that may have similar effects on HRQL. This left a total of 44 CCCs from which further analysis was conducted. Appendix B (see appendix on the journal’s Web site at www.elsevier.com) provides a list of the final 44 CCCs.
311
2.3. Self-administered questionnaire and HRQL measures in MEPS The self-administered questionnaire (SAQ) is a paperand-pencil questionnaire designed to collect a variety of health status and health care quality measures of adults in MEPS panels. All adults aged 18 and older were asked to complete an SAQ. The questionnaires were administered late in each year. For this analysis, only those SAQs completed by the actual respondent, not a proxy, were used. Self-reported health status or HRQL is documented within the MEPS data set using two sets of health status or HRQL measures: (1) the SF-12 Health Survey Version 2.0, with the physical component summary (PCS-12) and mental component summary (MCS-12) scores [41], and (2) two single-item core health status questions measuring perceived general and mental health. Both the SF-12 and the two core measures are found in both the Consolidated Full-Year data files HC-079 (2003) and HC-098 (2005). The SF-12 was used to derive the HRQL index. Responses to the SF-12 PCS and MCS items are coded, summed, transformed to a scale ranging from 0 (worst health status) to 100 (best health status), and then adjusted to norm based scaling for a final score for each health concept. The questionnaire uses a 4-week recall period. The two single-item health status items measure perceived health status and mental health status. Respondents are asked to rate their perception using a 5-point Likert scale ranging from excellent to poor. These two single-item core health status measures were used to further validate the HRQL index.
2.4. Charlson-CI Charlson’s method consists of a weighted index based on up to 19 comorbid conditions that predict 1-year survival [4,11,42,43]. The index was derived from an initial population of 559 medical inpatients at New York hospital and validated on a testing population of 685 breast cancer patients. Weights were assigned based on relative risk of death, ranging from one to six. The total score is the sum of the weights, with a score that can range from 0 to 33. The Katz version (3 digit ICD-9-CM codes based on selfreported conditions) was used for this study as a comparator index for the new HRQL-CI [42,44]. We used an exact regression based version and a more commonly used point based version of the Charlson-CI.
2.5. Statistical methods 2.5.1. Characteristics of the study population Descriptive summaries of the variables used in the study were examined by using means (standard deviation) for quantitative measures and frequency (percentage) for categorical variables both in 2003 and 2005 MEPS data sets.
312
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
2.5.2. Model selection To select a set of important predictors for HRQL-CIs from the 44 candidate variables, we used the least absolute shrinkage and selection operator (LASSO) [45], which is a shrinkage and selection method for linear regression by minimizing the usual sum of squared errors with a bound on the sum of the absolute values of coefficients. The LASSO is particularly designed for a large set of predictors, where some predictors could have negligibly small effect. To account for the different factors affecting PCS and MCS, we carried out the model selection procedure separately for the two outcomes. The criterion for selecting the dimension of the model was based on plotting several model selection criteria (Like adjusted R2, Akaike Information Criterion, and Bayes Information Criterion) for the LASSO-selected model corresponding to each given dimension against the model size. Beyond a certain threshold, there were only small increments in the model fit, and thus we decided to stop at the corresponding model size. 2.5.3. Model validation Once the newly developed models were constructed and fitted on the 2003 database, we considered two versions of the model, one based on the exactly fitted equation with given regression coefficients and the other by assigning a certain point-based weight to the CCCs based on strength of regression coefficients corresponding to CCC compounded with clinical judgment. Although the former is more accurate in terms of predictive power, the latter is more readily interpretable to a clinician. For internal validation, we calculated the 10-fold cross-validation prediction error of each model within the 2003 data set. For external validation, we predicted the PCS and MCS of the subjects in the 2005 database using the equation we derived from the 2003 database. Average prediction error was calculated along with correlation of observed and predicted values. Graphical diagnostic plots for the difference of observed and predicted values were calculated for the 2005 test data set. We also examined how the model predicted the core single-item HRQL measures, general health, and mental health, in both the 2003 and again in the 2005 MEPS Consolidate Full-Year data sets. The performance of the PCS and MCS indices when used as controlling factor in associating health care utilization with HRQL measures within asthmatic patients was also studied to reflect a scenario where such indices will typically be used in practice. While constructing the indices for the asthmatic population, asthma CCC was excluded from construction of the new indices as well as from Charlson-CI. We used the CharlsonCI in all scenarios as a comparative benchmark.
3. Results Descriptive statistics for subject characteristics and HRQL measures are provided in Table 1 for both 2003
Table 1 Characteristics of respondents in year 2003 and year 2005 data sets
Variable Sex Male Female Race White Black American Indian/Alaskan Native/Asian/Native Hawaiian/Pacific Islander/multiple races reported
2003 Frequency 2005 Frequency (percent) or mean (percent) or mean (standard deviation) (standard deviation) 4,879 (38.4) 7,834 (61.6)
5,055 (39.5) 7,757 (60.5)
10,196 (80.2) 1,778 (14.0) 739 (5.8)
1,0147 (79.2) 1,912 (14.9) 753 (5.9)
Educationdhighest grade achieved No degree 2,777 General Equivalency 683 Diploma High school diploma 5,691 Bachelor’s degree 1,763 Graduate degree 900 Other degree 868 Poverty category Low income Middle income High income
(21.9) (5.4)
2,625 (20.5) 634 (5.0)
(44.9) (13.9) (7.1) (6.8)
5,812 1,830 927 946
(45.5) (14.3) (7.3) (7.4)
4,674 (36.8) 3,720 (29.3) 4,319 (34.0)
4,781 (37.3) 3,573 (27.9) 4,458 (34.8)
PCS
47.3 (11.5)
46.8 (11.6)
MCS General health Excellent Very good Good Fair Poor
49.3 (10.5)
49.4 (10.5)
2,110 4,030 3,966 1,945 661
(16.6) (31.7) (31.2) (15.3) (5.2)
2,088 4,061 4,049 1,034 379
(16.3) (31.7) (31.6) (15.1) (5.3)
Mental health Excellent Very good Good Fair Poor
3,801 3,903 3,674 1,081 254
(29.9) (30.7) (28.9) (8.5) (2.0)
3,818 3,972 3,664 1,102 256
(29.8) (31.0) (28.6) (8.6) (2.0)
Abbreviations: PCS, physical component summary; MCS, mental component summary.
and 2005 data sets. The table shows that the characteristics of the study population are very consistent across the 2 years and there has not been any temporal shift in the population characteristics. The model selection and the model building procedure were entirely carried out on 2003 data set using 12,713 respondents. Figure 2 presents a summary of the model selection procedures where several model selection criteria are plotted against the best model of each dimension, the best model being suggested by the LASSO procedure for each given model size. One can notice that after a certain number of predictors, the improvement in model fit is incremental. We chose a model with 20 variables for PCS and a model with 15 variables for MCS based on the diagnostic assessment. Refer to Table 2 for a listing of CCCs for PCS
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
313
Fig. 2. Plot of R2, Adjusted R2, AIC, and BIC as a function of the model size or the number of variables included in the model. For each given model size, the criteria are plotted for the LASSO selected model. Abbreviations: AIC, Akaike information criterion; BIC, Bayes information criterion; LASSO, least absolute shrinkage and selection operator; PCS, physical component summary; MCS, mental component summary.
and MCS models. Table 3 presents the results of linear regression adjusted by the cluster-specific weights used in MEPS on PCS and MCS with the predictors selected by our model selection approach. Description of each chosen predictor for PCS and MCS is contained in Table 2. Table 3 also includes the points assigned to each predictor in terms of their strength of association with the outcome measures. One can notice that the R2 for the PCS model is 0.28, whereas the R2 for the MCS model is 0.16 with our exact regression analysis. The model R2 for the pointbased system provides very similar numbers. Table 4 presents the corresponding regression results using the point-based and exactly fitted regression equation for the Charlson-CI. The exact method provides an R2 for the PCS model of 0.13, whereas the R2 for the MCS model is 0.01. Thus, the improvement in the model fit is remarkable for both the measures by using the new HRQL-CI index as compared with Charlson-CI. The next step was to conduct a validation process for the proposed model, using the Charlson-CI for comparison. This was accomplished first by performing 10-fold crossvalidation within the 2003 database, each time leaving
out approximately 1/10th of the observations and using the fitted equation in Tables 3 and 4 to predict PCS or MCS values. The average Prediction error defined as 1/nS(observedpredicted)2, where n is the number of observations in the set of observations that were predicted, was calculated. This process was repeated 100 times. The average 10-fold cross-validation prediction errors across 100 replicates for the 2003 data set are reported in the first two rows of Table 5. One can notice the improvement over Charlson-CI in terms of prediction. We then considered 12,812 observations in the 2005 database and predicted the PCS and MCS measures by our models in Tables 3 and 4, adjusted by the 2005 cluster specific weights. The results presented in the last two rows of Table 5 again exhibit the same pattern as the initial analyses, with smaller prediction error by the newly proposed index, when compared with Charlson-CI. Figure 3 presents the residuals corresponding to the 2005-observed PCS and MCS after predicting with the new HRQL-CI and the Charlson-CI. One can notice substantial reduction in variance around zero with the new HRQL-CI suggesting enhanced accuracy than the Charlson-CI. The correlation
314
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
Table 2 HRQL indices (CCC) model selection Selected CCCs for physical health statusdPCS
Selected CCCs for mental health statusdMCS
Osteoarthritis/joint pain Spondylosis, intervertebral disc/ back problems Atherosclerosis and old myocardial infarction Diabetes Essential hypertension Heart failure and pulmonary hypertension Neurologic diseases Chronic obstructive pulmonary disease Gastrointestinal ulcer Rheumatoid arthritis and related diseases Paralysis Asthma
Anxiety and other mental illness Mental illnessdsevere
Systemic lupus erythematosus and connective tissue Esophageal disease Anxiety and other mental illness
Headache, including migraine Diabetes Gastrointestinal ulcer Spondylosis, intervertebral disc/ back problems Neurologic diseases Epilepsy, convulsions Asthma Anemia Liver disease Heart failure and pulmonary hypertension HIV infection Hepatitis Systemic lupus erythematosus and connective tissue
Peripheral artery disease Thyroid disorders Ophthalmic diseases Hepatitis Arrhythmias Abbreviations: HRQL-CI, health-related quality of life comorbidity index; CCC, clinical classification code; PCS, physical component summary; MCS, mental component summary; HIV, human immunodeficiency virus. In addition to these 15 CCCs, the initial model selection of MCS HRQL index also included the following nine CCCs: disorders of lipid metabolism, osteoporosis, skin conditions, ophthalmic diseases, cancer, rheumatoid arthritis and related conditions, other upper respiratory disease, chronic renal failure, and atherosclerosis and old myocardial infarction. These were excluded from the MCS model because they exhibited a counter-intuitive positive but insignificant association with the outcome MCS.
between observed and predicted PCS is 0.57 for HRQL-CI as opposed to 0.38 with Charlson-CI. The corresponding correlation values are 0.37 and 0.11 for MCS, respectively. We then evaluated our methods by conducting to two exercises. In the first exercise, we used the newly constructed indices to predict the scores of the single-item general health status and mental health status measures using both the 2003 and 2005 data sets. We included both PCS HRQLCI and MCS HRQL-CI as predictors in each regression model where the single-item general health status was the dependent variable in the first model and the single-item mental health status variable score was the dependent variable in the second model. The results in Table 6 show substantial improvement in terms of R2 for both outcomes and the expected relative strength of the regression coefficients corresponding to the two indices, PCS HRQL-CI being stronger in predicting general health status and MCS HRQL-CI being stronger in predicting mental health status. The improvement over Charlson-CI is again significant and
impressive. The relative boost to R2 by using the newly developed PCS HRQL-CI and MCS HRQL-CI by using Charlson-CI as the comparative benchmark is more pronounced for mental health status outcome, although the gain is profound under every model. The CharlsonCI-based models in Table 6 included only one predictor, namely the Charlson CI (point- or beta weight-based) and the health status items as response. In the second exercise, we selected a subset of 859 patients with asthma based on a set of questions in MEPS from the 2005 data set, which identify patients who had an asthma diagnosis at the time of the survey and predicted the HRQL measures in a regression model, while controlling for other comorbidities as captured through our two new HRQL-CI. We excluded the asthma CCC while constructing these indices. For PCS as response, the HRQLCI-adjusted model had a higher R2 of 0.42 compared with the Charlson-CI-adjusted model, where the R2 was 0.33. For the SF-12 MCS models, the HRQL-CI model had a higher R2 of 0.37, whereas the Charlson-CI model had an R2 of 0.11. Appendix D (see appendix on the journal’s Web site at www.elsevier.com) contains descriptive summaries for this subpopulation, whereas Appendix E (see appendix on the journal’s Web site at www.elsevier.com) contains the detailed regression results.
4. Discussion Controlling for comorbidity is essential in studies using general HRQL instruments. We have proven enormous gain in explaining the variation in HRQL measures by these new HRQL-CIs (from 2- to 16-fold increase in R2 across the models we studied) over the Charlson-CI. A researcher has superior power to characterize true associations between other predictors and such HRQL outcome measures by using this index as a controlling covariate in a regression model. One strength of this study was the use of a rigorous statistical approach to construct and validate the new HRQL-CI. Comorbidity, summarized as a single value representing overall burden of all illnesses present or as a single value that is controlled for when studying a specific illness, can influence HRQL [46]. In an analysis of results from randomized controlled clinical trials, Xuan et al. [47] demonstrated that comorbidity had an extensive effect on patients’ scores on generic HRQL instruments. Furthermore, in a cross-sectional sample of an Australian population, statistically and clinically significant decreases in usual activity levels and in SF-12 PCS-12 scores were identified when another chronic condition was present in additional to the index disease of interest [48]. A review article recently published by Fortin et al. [49,50] summarized the medical literature assessing multimorbidity and its influence on HRQL. They found a consistent and significant negative relationship between the number of medical conditions reported and the physical and mental domains of HRQL.
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
315
Table 3 HRQL-CIs beta coefficient- and point system-based indices Physical HRQL score (SF-12 PCS) as response CCC
Mental HRQL score (SF-12 MCS) as response Beta Point coefficientc assignmentd
Point Beta coefficienta assignmentb CCC
Paralysis 13.91 Rheumatoid arthritis and rheumatic disorders 10.55 Heart failure 10.36 Systemic lupus erythematosus 6.91 Ischemic heart disease 6.28 Osteoarthritis/nontraumatic joint disorders 6.07 Hepatitis 5.46 Diabetes 4.89 Degenerative neurologic disorders 4.78 Peripheral and central vascular diseases 4.70 Spinal column disorders 4.38 Obstructive pulmonary disease 3.79 Gastric and duodenal ulcer 3.33 Hypertension 3.21 Asthma 2.69 Arrhythmias 2.03 Esophageal disorders 1.63 Thyroid disorders 1.60 Vision disorders 1.53 Anxiety, depression 1.06
3 3 3 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1
Affective disorders, schizophrenia, other psychoses 8.70 Anxiety, depression 8.24 HIV infection 5.05 Epilepsy, convulsions 3.12 Hepatitis 2.97 Systemic lupus erythematosus 2.73 Heart failure 2.19 Headaches 1.87 Biliary and liver disorders 1.74 Anemia 1.53 Gastric and duodenal ulcer 1.44 Degenerative neurologic disorders 1.23 Diabetes 1.20 Asthma 1.15 Spinal column disorders 0.74
3 3 3 2 2 2 2 1 1 1 1 1 1 1 1
Abbreviations: HRQL-CI, health-related quality of life comorbidity index; SF-12, Short Form 12; PCS, physical component summary; MCS, mental component summary; CCC, clinical classification code; HIV, human immunodeficiency virus; MEPS, medical expenditure panel survey. a Intercept: 53.93; R2: 0.28. b R2: 0.27. c Intercept: 52.84; R2: 0.16. d R2: 0.15; All estimates are based on the analysis adjusted by MEPS survey weights.
In patients with chronic obstructive pulmonary disease (COPD), a CI improved predictive models of HRQL scores, as measured by the SF-36, when compared with models containing COPD only.[51] A study reported by Byles
et al. [52] found that the number of conditions noted by patients was associated with reduced HRQL as measured by the SF-36. The importance of this study is that symptomatic but not life-threatening conditions were included.
Table 4 Charlson-CIs [43] Beta coefficient-based index Diagnoses
Point-based indexa
PCS as responseb
MCS as responsec
Metastatic solid tumor Moderate or severe liver disease Diabetes Hemiplegia Leukemia Lymphoma Moderate or severe renal disease Any tumor Heart failure Chronic pulmonary disease Connective tissue disease Cerebral vascular disease Dementia Myocardial infarction Mild liver disease Peripheral vascular disease Ulcer disease
6 3 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1
4.38 4.76 7.20 9.36 14.17 0.86 8.60 3.32 13.58 4.43 13.09 3.03 11.5571 7.60 6.37 6.73 6.46
2.71 5.03 1.21 3.20 4.60 0.78 2.12 0.52 3.05 1.72 4.12 0.16 3.04 0.88 5.95 1.75 1.57
Abbreviations: CIs, comorbidity indices; PCS, physical component summary; MCS, mental component summary; MEPS, medical expenditure panel survey. a R2: 0.095 for PCS; R2: 0.0049 for MCS. b Intercept: 50.28; R2: 0.14. c Intercept: 50.49; R2: 0.013; All estimates are based on the analysis adjusted by MEPS survey weights.
316
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
Table 5 Average prediction errors for the SF-12 PCS and MCS in year 2003a and 2005 HRQL-CIs
Charlson-CIs
Dependent variable
Beta coefficient-based
Point system-based
Beta coefficient-based
Point system-based
PCS (2003) MCS (2003) PCS (2005) MCS (2005)
95.85 94.39 96.92 94.88
97.53 94.21 98.10 93.78
110.90 108.09 115.03 109.41
114.67 111.13 118.73 110.15
Abbreviations: SF-12, Short Form 12; PCS, physical component summary; MCS, mental component summary; HRQL, health-related quality of life; CIs, comorbidity indices. a Prediction errors in 2003 were calculated based on 10-fold cross-validation approach by selecting 10% of observations in the 2003 database randomly and predicting with the fitted models provided in Table 3 and 4. Prediction error is calculated by computing 1/nS(observedpredicted)2 for each random split of the data and then averaging over 100 such random samples.
The HRQL-CI is a compilation of conditions that are associated with both physical and mental-related functional status. For example, the CCCs with the greatest association with PCS or physical-related health status included musculoskeletal, cardiovascular, and neurologic conditions. The same is true with the MCS-related CI, with anxiety, depression, and conditions associated with more limiting mental illnesses have the greatest association with MCS score. It is interesting that a number of physical-related conditions appear in the MCS index, such as asthma, heart failure, neurologic conditions, and pain-related conditions. Research in the area of HRQL has demonstrated that these conditions are often associated with depression and/or anxiety. The new HRQL-CI outperformed a commonly used version of the Charlson-CI. Prediction errors for the SF-12
PCS and MCS in the validation year analysis demonstrated consistently better prediction and lower error terms for the HRQL-CI as compared with the Charlson-CI models. The regression coefficients for both point-based and beta weight-based scored HRQL-CI models obtained in the 2005 validation data set exhibited much stronger association with the outcome than that of the Charlson-CI models with two additional HRQL indicators, the two core health status questions asked in the MEPS survey. Similar gains in power were noted in the regression analysis on the asthmatic subpopulation with lower P-values noted in the HRQL-CI adjusted model. Thus the HRQL-CI outperformed the Charlson-CI in the 2003 development analysis, in the 2005 validation analysis, and in the disease-specific example using asthma patients.
Fig. 3. Histogram of the (observedpredicted) values for the 12,813 observations in the 2005 MEPS database by the CCC-based new HRQL-CI and the Charlson-CI. Abbreviations: MEPS, Medical Expenditure Panel Survey; CCC, clinical classification code; HRQL, health-related quality of life; CI, comorbidity index; Charlson-CI, Charlson comorbidity index; PCS, physical component summary; MCS, mental component summary.
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
317
Table 6 Comparisons of R2 when the outcome of interest is MEPS core single-item health status scores, namely general health status and mental health status in year 2003 and 2005 Outcome Health status variablea (y)
HRQL-CI Beta coefficientscored modelsb
HRQL-CI Point-scored modelsb
Charlson-CI Beta-coefficientscored modelsb
Charlson-CI Point-scored modelsb
General health status (2003) Model R2 PCS-index specific P-value MCS-index specific b-coefficient, P-value Charlson-CI variable b-coefficient, P-value
0.16 0.06, P!0.001 0.04, P!0.001 d
0.15 0.14, P!0.001 0.09, P!0.001 d
0.098 0.07, P!0.001 0.03, P50.53 d
0.079 d d 0.29, P!0.001
Mental health status (2003) Model R2 PCS-HRQL-CI variable b-coefficient, P-value MCS-HRQL-CI variable b-coefficient, P-value Charlson-CI variable b-coefficient, P-value
0.14 0.02, P!0.001 0.08, P!0.001 d
0.13 0.02, P!0.001 0.20, P!0.001 d
0.027 0.03, P!0.001 0.05, P50.0025 d
0.019 d d 0.13, P!0.001
General health status (2005) Model R2 PCS-HRQL-CI variable b-coefficient, P-value MCS-HRQL-CI variable b-coefficient, P-value Charlson-CI variable b-coefficient, P-value
0.16 0.06, P!0.001 0.04, P!0.001 d
0.16 0.15, P!0.001 0.08, P!0.001 d
0.097 0.07, P!0.001 0.05, P50.0025 d
0.077 d d 0.27, P!0.001
Mental health status (2005) Model R2 PCS-HRQL-CI variable b-coefficient, P-value MCS-HRQL-CI variable b-coefficient, P-value Charlson-CI variable b-coefficient, P-value
0.1323 0.02, P!0.001 0.08, P!0.001 d
0.1268 0.03, P!0.001 0.19, P!0.001 d
0.02627 0.10, P!0.001 0.04, P50.0198 d
0.02086 d d 0.14, P!0.001
Abbreviations: MEPS, medical expenditure panel survey; HRQL, health-related quality of life; CI, comorbidity index; PCS, physical component summary; MCS, mental component summary. a Both the general and mental health status scores range from 1 to 5 indicating excellent to poor general and mental health perceptions, respectively. b The PCS and MCS beta coefficient-based and point-based indices were computed by summing up the magnitude of the beta weights (without the negative sign) or point weights for the diseases that are presented in Table 3. In both cases, high values of indices indicate poor health condition. Similarly, the Charlson indices are defined by beta weights and point weights as indicated in Table 4. In each of the above situations, we fitted a cluster-adjusted regression model of the following form with two covariates: (a) Outcome 5 b0 þ b1 HRQL-PCS-CI (beta- or point-based) þ b1 HRQL-MCS-CI (beta- or point-based) þ error; For Charlson-CI we fitted model with single covariate: (b) Outcome 5 b0 þ b1 Charlson-CI (Beta- or point-based) þ error.
Two uses of CIs are to predict a health outcome and to control for the potential influence of other illnesses when analyzing a specific outcome. In the latter case, researchers analyzing HRQL data of a specific illness will want to control for the potential influence of other illnesses on the dependent variable, especially when a general measure of HRQL is used, such as the SF-36. This study chose the illness of asthma to test whether the new HRQL-CI provided greater predictive ability than the Charlson-CI when focusing only on patients with asthma using the SF-12 as the measure of HRQL. Previous studies have demonstrated that comorbid conditions do indeed have an effect on predictive modeling when asthma is the disease of interest [48,53,54]. Presently, the HRQL-CI uses beta weights or an empirically derived point system to score. Users of the index would be able to use the points or beta weights as they are listed here or derive weights from their own patient population using the list of conditions in the index. Which one to use is a fair question at this point. Based strictly on significance values, the beta weight-derived score performed marginally better than the point system in most analyses. However, the point system is easier to use and implement, and for most practical purposes has the same relative significance as the beta weight-derived scores.
4.1. Limitations Not all medical conditions were included in the initial set from which the final indices were created. One excluded group were the relatively rare conditions, such as cystic fibrosis, a chronic condition that involves the lungs and exocrine glands. HRQL is impaired significantly in advanced cases. However, the number of adult persons in the MEPS data set in 2003 that reported this condition was so low that the group was excluded. There were other conditions that are more common, which were not included because of the fact that no adult person in the MEPS data set in 2003 reported the condition. An example would be pancreatic cancer. Researchers electing to evaluate the HRQL-CI may choose to explore the influence of these conditions when present in the data sets they work with. Other excluded conditions included gender-specific conditions. The index was intended to be used in both genders, so that excluding gender-specific conditions allowed the resulting indices to be generalizable. Future research should focus on the relative contribution of gender-specific conditions. Another limitation is that the general HRQL instrument used for the development of the HRQL-CI was the SF-12. Other general measures exist, which provide a single score
318
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
or which have multiple dimensions. Our study demonstrated excellent performance when the general health and mental health single-item health status measures were analyzed. Future studies using other general HRQL instruments as the dependent variable are warranted to provide future validation that the new index does indeed function well. This study compared the new indices with the CharlsonCI, a commonly used index. Other indices exist, which require assessment to determine whether the new HRQL-CI will outperform them as well. 5. Conclusion The new HRQL-CI is a valid risk adjustment index. It outperforms the Charlson-CI when predicting health status for the SF-12 PCS and MCS, the two core single-item health status measures as well as in an asthma specific population. Further work is necessary to test its performance in other patient populations. Appendix Supplementary material Supplementary material can be found, in the online version, at 10.1016/j.jclinepi.2010.01.025. References [1] Hall SF. A user’s guide to selecting a comorbidity index for clinical research. J Clin Epidemiol 2006;59:849e55. [2] Greenfield S, Nelson EC. Recent developments and future issues in the use of health status assessment measures in clinical setting. Med Care 1992;30:MS23e41. [3] Hitchcock Noel P, Williams JW, Unutzer J, Worchel J, Lee S, Cornell J, et al. Depression and comorbid illness in elderly primary care patients: impact on multiple domains of health status and well-being. Ann Fam Med 2004;2:555e62. [4] Deyo DA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with IOCD-9-CM administrative databases. J Clin Epidemiol 1992;45:613e9. [5] Elixhauser AC, Steiner C, Harris DR, Coffey RM. Comorbidity measures to use with administrative data. Med Care 1998;36:8e27. [6] Greenfield S, Apolone G, McNeil BJ, Cleary PD. The importance of co-existent disease in the occurrence of postoperative complications and one-year recovery in patients undergoing total hip replacement. Comorbidity and outcomes after hip replacement. Med Care 1993;31:141e54. [7] Linn BS, Linn MW, Gurel L. Cumulative illness rating scale. J Am Geriatr Soc 1968;16:622e7. [8] Miller MD, Paradis CF, Houck PR. Rating chronic medical illness burden in geropsychiatric practice and research: application of the Cumulative Illness Rating scale. Psychiatry Res 1992;41:237e48. [9] Parkerson GR, Broadhead WE, Tse CK. The Duke Severity of Illness Checklist (DUSOI) for measurement of severity and comorbidity. J Clin Epidemiol 1993;46:379e93. [10] Roe CJ, Dodich N, Kulinskaya E, Adam WR. Comorbidities and prediction of length of hospital stay. Aust N Z J Med 1998;28:811e5. [11] Charlson ME, Szatrowski TP, Peterson J, Gold J. Validation of a combined comorbidity index. J Clin Epidemiol 1994;47:1245e51.
[12] Voaklander DC, Kelly KD, Jones KCA, Suarez-Almazor ME. Self report co-morbidity and health related quality of lifeda comparison with record based co-morbidity measures. Soc Indic Res 2004;66:213e28. [13] Wang PS, Walker A, Tsuang M, Orav EJ, Levin R, Avorn J. Strategies for improving comorbidity measures based on Medicare and Medicaid claims data. J Clin Epidemiol 2000;53:571e8. [14] Guralnik JM. Assessing the impact of comorbidity in the older population. Ann Epidemiol 1996;6:376e80. [15] Iezzoni LI. Dimensions of risk. In: Iezzoni LI, editor. Risk adjustment for measuring healthcare outcomes. Chicago, IL: Health Administration Press; 1997. p. 43e168. [16] Ghali WA, Quan H, Brandt R. Risk adjustment using administrative data. Impact of a diagnosis-type indicator. J Gen Intern Med 2001;16: 519e24. [17] Incalzi RA, Capparella O, Gemma A, Landi F, Bruno E, Meo F, et al. The interaction between age and comorbidity contributes to predicting the mortality of geriatric patients in the acute-care hospital. J Intern Med 1997;242:291e8. [18] Menotti A, Mulder I, Nissinen A, Giampaoli S, Feskens EJM, Kromhout D. Prevalence of morbidity and multimorbidity in elderly male populations and their impact on 10-year all-cause mortality: the FINE study (Finland, Italy, Netherlands, Elderly). J Clin Epidemiol 2001;54:680e6. [19] Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care 2004;42:355e60. [20] Holman CDJ, Preen DB, Baynham NJ, Finn JC, Semmens JB. A multipurpose comorbidity scoring system performed better than the Charlson index. J Clin Epidemiol 2005;58:1006e14. [21] Bravo G, Dubois MF, Herert R, DeWals P, Messier L. A prospective evaluation of the Charlson comorbidity index for use in long-term care patients. J Am Geriatr Soc 2002;50:740e5. [22] Librero J, Peiro S, Ordinana R. Chronic comorbidity and outcomes of hospital care: length of stay, mortality, and readmission at 30 and 365 days. J Clin Epidemiol 1999;52:171e9. [23] Rochon PA, Katz JN, Morrow LA, McGlinchey-Berroth R, Ahlquist MM, Sakkarati M, et al. Comorbid illness is associated with survival and length of hospital stay in patients with chronic disability: a prospective comparison of three comorbidity indices. Med Care 1996;34:1093e101. [24] Fishman PA, Goodman MJ, Hornbrook MC, Meenan RT, Bachman DJ, O’Keefe Rosetti MC. Risk adjustment using automated ambulatory pharmacy data. The RxRisk model. Med Care 2003;41: 84e99. [25] Hamilton SM, Johnston DWC, Voaklander DC. An analysis of surgical outcomes following regionalization in the Canadian health care system. Can J Surg 2001;44:51e8. [26] Jones CA, Voaklander DC, Johnston DWC, Suarez-Almazor ME. Health-related quality of life outcomes after total hip and knee arthroplasties in a community-based sample. J Rheumatol 2000;27: 1745e52. [27] Jones CA, Voaklander DC, Johnston DWC, Suarez-Almazor ME. The effect of age on pain, function, and quality of life. Arch Intern Med 2001;161:454e60. [28] Amin SP, Mullins SC, Duncan BS, Blanford L. Direct health care costs for treatment of diabetes mellitus and hypertension in an IPA-group-model HMO. Am J Health Syst Pharm 1999;56: 1515e20. [29] Guyatt GH, Feeny DH, Patrick DKL. Measuring health-related quality of life. Ann Intern Med 1993;118:622e9. [30] Fitzpatrick R, Fletcher A, Gore S. Quality of life measures in health care: applications and issues in assessment. Br Med J 1992;305:1074e7. [31] Michelson H, Bolund C, Brandberg Y. Multiple chronic health problems are negatively associated with health related quality of life (HRQoL) irrespective of age. Qual Life Res 2001; 9:1093e104.
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319 [32] Rijken M, van Kerkhof M, Dekker J, Schellevis FG. Comorbidity of chronic diseases: effects of disease pairs on physical and mental functioning. Qual Life Res 2005;14:45e55. [33] Selim AJ, Fincke G, Ren XS, Lee A, Roger WH, Miller DR, et al. Comorbidity assessments based on patient report: results from the Veterans health Study. J Ambul Care Manage 2004;27: 281e95. [34] Kazis LE, Miller DR, Clark J, Skinner K, Lee A, Rogers W, et al. Health-related quality of life in patients served by the department of Veterans Affairs. Results from the Veterans Health Study. Arch Intern Med 1998;158:626e32. [35] Alonsao J, Ferrer M, Gandek B, Ware JE, Aaronson NK, Mosconi P, et al., the IQOLA Project Group. Health-related quality of life associated with chronic conditions in eight countries: results from the International Quality of Life Assessment (IQOLA) project. Qual Life Res 2004;13:283e98. [36] Stewart AL, Ware JE, editors. Measuring functioning and well-being: the medical outcomes study approach. Durham, NC: Duke University Press; 1992. [37] Temkin NR, Dikmen S, Machamer J, McLean A. General versus disease-specific measures: further work on the Sickness Impact Profile for head injury. Med Care 1989;27:S44e53. [38] Kantz ME, Harris WJ, Levitsky K. Methods for assessing condition-specific and generic functional status outcomes after total knee replacement. Med Care 1992;30:MS240e52. [39] Patrick DL, Erickson P. Health status and health policy: allocating resources to health care. New York, NY: Oxford University Press; 1993. [40] Mauskopf JA, Austin R, Dix LP, Berzon RA. Estimating the value of a generic quality of life measure. Med Care 1995;33:AS195e202. [41] Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996;34:220. [42] D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson index with administrative data bases. J Clin Epidemiol 1996;49:1429e33.
319
[43] D’Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson comorbidity index. Methods Inf Med 1993;32: 382e7. [44] Katz JN, Chang LC, Sangha O, Fossel AH, Bates DW. Can comorbidity be measured by questionnaire rather than medical record review? Med Care 1996;34:73e84. [45] Tibshirani R. Regression Shrinkage and Selection via the Lasso. J R Stat Soc Series B Stat Methodol 1996;58:267e88. [46] Gijsen R, Hoeymans N, Schellevis FG, Ruwaard D, Satariano WA, van den Bos GAM. Causes and consequences of comorbidity: a review. J Clin Epidemiol 2001;54:661e7. [47] Xuan J, Kirchdoerfer LJ, Boyer JG, Norwood GJ. Effects of comorbidity on health-related quality of life scores: an analysis of clinical trial data. Clin Ther 1999;21:383e403. [48] Adams RJ, Wilson DH, Taylor AW, Daly A, d’Espaignet ET, DalGrande E, et al. Coexistent chronic conditions and asthma quality of life: a population-based study. Chest 2006;129:285e91. [49] Fortin M, Lapointe L, Hudon C, Vanasse A, Ntetu AL, Maltais D. Multimorbidity and quality of life in primary care: a systematic review. Health Quality of Life Outcomes 2004;2:51. [50] Fortin M, Bravo G, Hudon C, Lapointe L, Almirall J, Dubois MF, et al. Relationship between multimorbidity and health-related quality of life of patients in primary care. Qual Life Res 2006;15:83e91. [51] VanManen JG, Bindels PJE, Dekker FW, Izermans CJ, Bottema BJAM, VanDerZee JS, et al. Added value of co-morbidity in predicting health-related quality of life in COPD patients. Respir Med 2001;95:496e504. [52] Byles JE, D’Este C, Parkinson L, O’Connell R, Treloar C. Single index of multimorbidity did not predict multiple outcomes. J Clin Epidemiol 2005;58:997e1005. [53] Schneider A, Lowe B, Meyer FJ, Biessecker K, Joos S, Szecsenyi J. Depression and panic disorder as predictors of health outcomes for patients with asthma in primary care. Respir Med 2008;102:359e66. [54] Wijnhoven HA, Kriegsman DM, Hesselink AE, de Haan M, Schellevis FG. The influence of co-morbidity on health-related quality of life in asthma and COPD patients. Respir Med 2003;97:468e75.
A new comorbidity index: the health-related quality of life comorbidity index Bhramar Mukherjeea, Huang-Tz Oub, Fei Wanga, Steven R. Ericksonb,* b
a Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, MI 48109, USA Department of Clinical, Social, and Administrative Sciences, College of Pharmacy, University of Michigan, Ann Arbor, MI 48109, USA
Accepted 31 January 2010
Abstract Objective: To derive and validate the health-related quality of life comorbidity index (HRQL-CI). Study Design and Setting: Of 261 clinical classification codes (CCCs) in the 2003 Medical Expenditure Panel Survey (MEPS), 44 were identified as adult, gender-neutral, chronic conditions. The least absolute shrinkage and selection operator (LASSO) procedure identified CCCs significantly associated with the Short Form-12 physical component summary (PCS) and mental component summary (MCS) scores. Regression models were fitted with the selected CCCs, resulting in two subsets corresponding to PCS and MCS, collectively called the HRQL-CI. Internal validation was assessed using 10-fold cross-validation, whereas external validation in terms of prediction accuracy was assessed in the 2005 MEPS database. Prediction errors and model R2 were compared between HRQL-CI models and models using the Charlson-CI. Results: LASSO identified 20 CCCs significantly associated with PCS and 15 with MCS. The R2 for the models, including the HRQLCI (0.28 for PCS and 0.16 for MCS) were greater than those using the Charlson-CI (0.13 for PCS and 0.01 for MCS). The same pattern of higher R2 for models using the HRQL-CI was observed in the validation tests. Conclusion: The HRQL-CI is a valid risk adjustment index, outperforming the Charlson-CI. Further work is needed to test its performance in other patient populations and measures of HRQL. Ó 2011 Elsevier Inc. All rights reserved. Keywords: Health-related quality of life; Comorbidity; Charlson comorbidity index; Risk prediction models; LASSO; SF-12; Health status
1. Introduction Comorbidity is the existence or occurrence of any distinct additional disease or diseases during the clinical course of a patient who has an index disease under study. A comorbidity index (CI) is a weighted measure that, when conducting statistical analyses, will control for the potential influence of those illness on an outcome of interest [1,2]. A relatively uncomplicated method of controlling for comorbidity is to use the simple count of illnesses [3]. Diagnosis methods often
This project was funded by the Agency for Health Care Research and Quality, R03 grant 1 R03 HS017461-01A1 ‘‘Developing a Comorbidity Index for Health-Related Quality of Life Studies,’’ Steven R. Erickson PI, from September 1, 2008 to August 31, 2009. Abstract of this article was accepted for poster presentation for the October 2009 meeting of the International Society for Quality of Life Research, New Orleans, LA. * Corresponding author. College of Pharmacy, University of Michigan, 428 Church Street, Ann Arbor, MI 48109-1065, USA. Tel.: þ734-7634989; fax: þ734-763-2022. E-mail address: [email protected] (S.R. Erickson). 0895-4356/$ - see front matter Ó 2011 Elsevier Inc. All rights reserved. doi: 10.1016/j.jclinepi.2010.01.025
use data available from a medical record derived from hospital stays. These are known as discharge-based CIs [4e10]. A common diagnosis-based method was derived by Charlson et al. [11]. This method was initially developed as a chartbased weighted index that provides a simple and valid method for estimating the risk of death associated with comorbid illness, taking into account both the number and seriousness of comorbid illnesses. Other versions of the Charlson-CI use data derived from electronic claims data, electronic hospital records, and patient self-report [4,12]. CIs are used in health services research to control for confounding in observational studies and for risk adjustment in studies of health care quality [13e16]. Comorbidity affects mortality [17e21], health resource utilization [22e24], admission and readmission to hospital [20,22,25], and healthrelated quality of life (HRQL) or functional status [26e28]. Without adequate measures to adjust for intervening comorbidity differences, valid comparisons of health status and HRQL outcomes in population studies cannot be made [15]. Large nationally representative data sets, such as the National Health and Nutrition Examination Survey and the
310
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
2. Methods What is new? Key findings A new risk adjustment index was derived for use in health-related quality of life (HRQL) studies. What this adds to what is known? The new HRQL comorbidity index (HRQL-CI) provides better predictive ability for HRQL measures than the commonly used Charlson-CI. What is the implication, what should change now? Researchers have a comorbidity risk adjustment index to control for the potential influence of other illnesses when studying the effect of specific diseases or other risk factors associated with HRQL measures, which outperforms commonly used Charlson-CI.
Medical Expenditure Panel Survey (MEPS) include data derived from health status or HRQL instruments. These measures have and will continue to be used as primary or secondary outcome variables in analyses. It is therefore of interest to control for the presence of comorbidity when using these measures to study HRQL attributed to specific illnesses. HRQL refers to the physical, emotional, and social impact of disease and related treatments and is distinct from physiologic measures of disease [29,30]. Generally, HRQL decreases with increasing comorbidity [31e35]. Two types of questionnaires are used to measure HRQL, general and disease- or intervention-specific. General measures assess concepts that are relevant to a wide range of people, including ability to function in everyday life and emotional wellbeing [36]. They are not specific to any age, disease, or treatment group and are designed to be broadly applied across different populations to allow for comparisons across many conditions [37e40]. Researchers have validated measures of comorbidity by how well they predict mortality, health resource use, and expenditures, either as a predictor themselves or to adjust for the contributing effects of other diseases when studying the association of a specific illness. Comparatively little research has been conducted validating existing CIs with HRQL or health status as the primary outcome variable. Moreover, few studies have been conducted to develop and validate HRQL- or health status-specific CI. The purpose of this study was to derive and validate a CI using diseases that have the greatest association with HRQL. The secondary goal was to compare the results of explanatory models that use the new index derived specifically for HRQL with a CI originally derived to predict mortality and health care resource use, the Charlson-CI using MEPS database.
2.1. General description of the data sets used for the study The MEPS is a nationally representative, public domain data set maintained by the Agency for Healthcare Research and Quality. The MEPS data set was considered an appropriate data source because of the reliability and validity of the data, in particular the disease, treatment, and HRQL data. Further specific information may be found at http:// www.meps.ahrq.gov/mepsweb/. The 2003 Full-Year Consolidated Data File (HC-079) and the 2003 Medical Conditions File (HC-078) were used to derive the HRQL-CI. The HC-079 and HC-078 data files contain person-level data from the respondent and others in the home for two panels during the full year of 2003. All medical conditions related to a reported encounter with a health care professional or institution for the year of 2003 are documented for each respondent and proxy. To validate the HRQL-CI, the 2005 Full-Year Consolidated Data File (HC-097) and the 2005 Medical Conditions File (HC-096) were used. Both HC-079 and HC-097 Full-Year Consolidated Data Files include the Short Form-12 (SF12) HRQL survey, version 2.0, the source of HRQL data for this study. The variable DUPERSID is common to the Full-Year Consolidated data files and the Medical Conditions data files. It serves to link person-specific data between all MEPS data files. There is no overlap of people between the 2003 and 2005 databases ensuring that the validation set is independent of the model development or training set. 2.2. Diagnoses or conditions data Diagnoses data were derived from the Medical Conditions Files HC-078 (2003) for index development and HC-096 (2005) for index validation. Included in these files are the ICD-9 codes that correspond to health resource use data for each respondent and proxy. ICD-9 codes are reduced to the first three digits for confidentiality reasons. Similar ICD-9 codes are grouped into clinical classification codes (CCCs), which represent groupings of clinically similar medical conditions. Most CCCs in the Medical Conditions files are related to actual medical conditions, although a few are reserved for administrative uses. The CCCs were used for this study. There are 261 unique CCCs in the 2003 Medical Conditions File (HC-078). Each CCC is labeled by a three-digit number and the frequency of occurrence within the data set is provided by MEPS. Not all CCCs were appropriate to use for this study. Figure 1 provides the sequential approach used to exclude and condense CCCs from the original 261. A total of 164 CCCs were initially excluded and are listed in Appendix A (see appendix on the journal’s Web site at www. elsevier.com) with the reason for exclusion. An additional
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
Health-Related Quality of Life Comorbidity Index Development Step MEPS 2003 HC-078 Medical Conditions File 261 Clinical Classification Codes (CCC) Excluded CCCs – Refer to Appendix A Administrative Codes (n = 4) Acute Infections, Poisonings, Injury, Trauma (n = 84) Gender-specific (n = 32) Pediatric/Maternity/Intrauterine (n = 27) Cognitive Impairment (n = 5) Rare conditions, less than 10 people Or mixed acute/few chronic (n = 10) Result: 164 CCCs excluded
97 CCCs remained. Of these, 7 CCCs did not have persons documented with the condition. These CCCs were removed, leaving 90 CCCs. Remaining 90 CCCs submitted to further condensing because of similar organ-system involvement/pathophysiology. Refer to Appendix B. Of these, 64 CCCs were condensed to 18 CCCs. Thus, we have 90– 64+18=44 CCCs Final Working CCC set consisted of 44 CCCs derived from the 2003 HC-078 Medical Conditions file Fig. 1. Flow chart for selection of CCCs for analysis. Abbreviations: MEPS, medical expenditure panel survey; CCS, clinical classification codes.
seven CCCs were considered appropriate to be included but were not present in the 2003 Medical Conditions File because of no person having this condition. The 90 remaining CCCs were examined for similarity in clinical attributes in an effort to combine CCCs that may have similar effects on HRQL. This left a total of 44 CCCs from which further analysis was conducted. Appendix B (see appendix on the journal’s Web site at www.elsevier.com) provides a list of the final 44 CCCs.
311
2.3. Self-administered questionnaire and HRQL measures in MEPS The self-administered questionnaire (SAQ) is a paperand-pencil questionnaire designed to collect a variety of health status and health care quality measures of adults in MEPS panels. All adults aged 18 and older were asked to complete an SAQ. The questionnaires were administered late in each year. For this analysis, only those SAQs completed by the actual respondent, not a proxy, were used. Self-reported health status or HRQL is documented within the MEPS data set using two sets of health status or HRQL measures: (1) the SF-12 Health Survey Version 2.0, with the physical component summary (PCS-12) and mental component summary (MCS-12) scores [41], and (2) two single-item core health status questions measuring perceived general and mental health. Both the SF-12 and the two core measures are found in both the Consolidated Full-Year data files HC-079 (2003) and HC-098 (2005). The SF-12 was used to derive the HRQL index. Responses to the SF-12 PCS and MCS items are coded, summed, transformed to a scale ranging from 0 (worst health status) to 100 (best health status), and then adjusted to norm based scaling for a final score for each health concept. The questionnaire uses a 4-week recall period. The two single-item health status items measure perceived health status and mental health status. Respondents are asked to rate their perception using a 5-point Likert scale ranging from excellent to poor. These two single-item core health status measures were used to further validate the HRQL index.
2.4. Charlson-CI Charlson’s method consists of a weighted index based on up to 19 comorbid conditions that predict 1-year survival [4,11,42,43]. The index was derived from an initial population of 559 medical inpatients at New York hospital and validated on a testing population of 685 breast cancer patients. Weights were assigned based on relative risk of death, ranging from one to six. The total score is the sum of the weights, with a score that can range from 0 to 33. The Katz version (3 digit ICD-9-CM codes based on selfreported conditions) was used for this study as a comparator index for the new HRQL-CI [42,44]. We used an exact regression based version and a more commonly used point based version of the Charlson-CI.
2.5. Statistical methods 2.5.1. Characteristics of the study population Descriptive summaries of the variables used in the study were examined by using means (standard deviation) for quantitative measures and frequency (percentage) for categorical variables both in 2003 and 2005 MEPS data sets.
312
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
2.5.2. Model selection To select a set of important predictors for HRQL-CIs from the 44 candidate variables, we used the least absolute shrinkage and selection operator (LASSO) [45], which is a shrinkage and selection method for linear regression by minimizing the usual sum of squared errors with a bound on the sum of the absolute values of coefficients. The LASSO is particularly designed for a large set of predictors, where some predictors could have negligibly small effect. To account for the different factors affecting PCS and MCS, we carried out the model selection procedure separately for the two outcomes. The criterion for selecting the dimension of the model was based on plotting several model selection criteria (Like adjusted R2, Akaike Information Criterion, and Bayes Information Criterion) for the LASSO-selected model corresponding to each given dimension against the model size. Beyond a certain threshold, there were only small increments in the model fit, and thus we decided to stop at the corresponding model size. 2.5.3. Model validation Once the newly developed models were constructed and fitted on the 2003 database, we considered two versions of the model, one based on the exactly fitted equation with given regression coefficients and the other by assigning a certain point-based weight to the CCCs based on strength of regression coefficients corresponding to CCC compounded with clinical judgment. Although the former is more accurate in terms of predictive power, the latter is more readily interpretable to a clinician. For internal validation, we calculated the 10-fold cross-validation prediction error of each model within the 2003 data set. For external validation, we predicted the PCS and MCS of the subjects in the 2005 database using the equation we derived from the 2003 database. Average prediction error was calculated along with correlation of observed and predicted values. Graphical diagnostic plots for the difference of observed and predicted values were calculated for the 2005 test data set. We also examined how the model predicted the core single-item HRQL measures, general health, and mental health, in both the 2003 and again in the 2005 MEPS Consolidate Full-Year data sets. The performance of the PCS and MCS indices when used as controlling factor in associating health care utilization with HRQL measures within asthmatic patients was also studied to reflect a scenario where such indices will typically be used in practice. While constructing the indices for the asthmatic population, asthma CCC was excluded from construction of the new indices as well as from Charlson-CI. We used the CharlsonCI in all scenarios as a comparative benchmark.
3. Results Descriptive statistics for subject characteristics and HRQL measures are provided in Table 1 for both 2003
Table 1 Characteristics of respondents in year 2003 and year 2005 data sets
Variable Sex Male Female Race White Black American Indian/Alaskan Native/Asian/Native Hawaiian/Pacific Islander/multiple races reported
2003 Frequency 2005 Frequency (percent) or mean (percent) or mean (standard deviation) (standard deviation) 4,879 (38.4) 7,834 (61.6)
5,055 (39.5) 7,757 (60.5)
10,196 (80.2) 1,778 (14.0) 739 (5.8)
1,0147 (79.2) 1,912 (14.9) 753 (5.9)
Educationdhighest grade achieved No degree 2,777 General Equivalency 683 Diploma High school diploma 5,691 Bachelor’s degree 1,763 Graduate degree 900 Other degree 868 Poverty category Low income Middle income High income
(21.9) (5.4)
2,625 (20.5) 634 (5.0)
(44.9) (13.9) (7.1) (6.8)
5,812 1,830 927 946
(45.5) (14.3) (7.3) (7.4)
4,674 (36.8) 3,720 (29.3) 4,319 (34.0)
4,781 (37.3) 3,573 (27.9) 4,458 (34.8)
PCS
47.3 (11.5)
46.8 (11.6)
MCS General health Excellent Very good Good Fair Poor
49.3 (10.5)
49.4 (10.5)
2,110 4,030 3,966 1,945 661
(16.6) (31.7) (31.2) (15.3) (5.2)
2,088 4,061 4,049 1,034 379
(16.3) (31.7) (31.6) (15.1) (5.3)
Mental health Excellent Very good Good Fair Poor
3,801 3,903 3,674 1,081 254
(29.9) (30.7) (28.9) (8.5) (2.0)
3,818 3,972 3,664 1,102 256
(29.8) (31.0) (28.6) (8.6) (2.0)
Abbreviations: PCS, physical component summary; MCS, mental component summary.
and 2005 data sets. The table shows that the characteristics of the study population are very consistent across the 2 years and there has not been any temporal shift in the population characteristics. The model selection and the model building procedure were entirely carried out on 2003 data set using 12,713 respondents. Figure 2 presents a summary of the model selection procedures where several model selection criteria are plotted against the best model of each dimension, the best model being suggested by the LASSO procedure for each given model size. One can notice that after a certain number of predictors, the improvement in model fit is incremental. We chose a model with 20 variables for PCS and a model with 15 variables for MCS based on the diagnostic assessment. Refer to Table 2 for a listing of CCCs for PCS
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
313
Fig. 2. Plot of R2, Adjusted R2, AIC, and BIC as a function of the model size or the number of variables included in the model. For each given model size, the criteria are plotted for the LASSO selected model. Abbreviations: AIC, Akaike information criterion; BIC, Bayes information criterion; LASSO, least absolute shrinkage and selection operator; PCS, physical component summary; MCS, mental component summary.
and MCS models. Table 3 presents the results of linear regression adjusted by the cluster-specific weights used in MEPS on PCS and MCS with the predictors selected by our model selection approach. Description of each chosen predictor for PCS and MCS is contained in Table 2. Table 3 also includes the points assigned to each predictor in terms of their strength of association with the outcome measures. One can notice that the R2 for the PCS model is 0.28, whereas the R2 for the MCS model is 0.16 with our exact regression analysis. The model R2 for the pointbased system provides very similar numbers. Table 4 presents the corresponding regression results using the point-based and exactly fitted regression equation for the Charlson-CI. The exact method provides an R2 for the PCS model of 0.13, whereas the R2 for the MCS model is 0.01. Thus, the improvement in the model fit is remarkable for both the measures by using the new HRQL-CI index as compared with Charlson-CI. The next step was to conduct a validation process for the proposed model, using the Charlson-CI for comparison. This was accomplished first by performing 10-fold crossvalidation within the 2003 database, each time leaving
out approximately 1/10th of the observations and using the fitted equation in Tables 3 and 4 to predict PCS or MCS values. The average Prediction error defined as 1/nS(observedpredicted)2, where n is the number of observations in the set of observations that were predicted, was calculated. This process was repeated 100 times. The average 10-fold cross-validation prediction errors across 100 replicates for the 2003 data set are reported in the first two rows of Table 5. One can notice the improvement over Charlson-CI in terms of prediction. We then considered 12,812 observations in the 2005 database and predicted the PCS and MCS measures by our models in Tables 3 and 4, adjusted by the 2005 cluster specific weights. The results presented in the last two rows of Table 5 again exhibit the same pattern as the initial analyses, with smaller prediction error by the newly proposed index, when compared with Charlson-CI. Figure 3 presents the residuals corresponding to the 2005-observed PCS and MCS after predicting with the new HRQL-CI and the Charlson-CI. One can notice substantial reduction in variance around zero with the new HRQL-CI suggesting enhanced accuracy than the Charlson-CI. The correlation
314
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
Table 2 HRQL indices (CCC) model selection Selected CCCs for physical health statusdPCS
Selected CCCs for mental health statusdMCS
Osteoarthritis/joint pain Spondylosis, intervertebral disc/ back problems Atherosclerosis and old myocardial infarction Diabetes Essential hypertension Heart failure and pulmonary hypertension Neurologic diseases Chronic obstructive pulmonary disease Gastrointestinal ulcer Rheumatoid arthritis and related diseases Paralysis Asthma
Anxiety and other mental illness Mental illnessdsevere
Systemic lupus erythematosus and connective tissue Esophageal disease Anxiety and other mental illness
Headache, including migraine Diabetes Gastrointestinal ulcer Spondylosis, intervertebral disc/ back problems Neurologic diseases Epilepsy, convulsions Asthma Anemia Liver disease Heart failure and pulmonary hypertension HIV infection Hepatitis Systemic lupus erythematosus and connective tissue
Peripheral artery disease Thyroid disorders Ophthalmic diseases Hepatitis Arrhythmias Abbreviations: HRQL-CI, health-related quality of life comorbidity index; CCC, clinical classification code; PCS, physical component summary; MCS, mental component summary; HIV, human immunodeficiency virus. In addition to these 15 CCCs, the initial model selection of MCS HRQL index also included the following nine CCCs: disorders of lipid metabolism, osteoporosis, skin conditions, ophthalmic diseases, cancer, rheumatoid arthritis and related conditions, other upper respiratory disease, chronic renal failure, and atherosclerosis and old myocardial infarction. These were excluded from the MCS model because they exhibited a counter-intuitive positive but insignificant association with the outcome MCS.
between observed and predicted PCS is 0.57 for HRQL-CI as opposed to 0.38 with Charlson-CI. The corresponding correlation values are 0.37 and 0.11 for MCS, respectively. We then evaluated our methods by conducting to two exercises. In the first exercise, we used the newly constructed indices to predict the scores of the single-item general health status and mental health status measures using both the 2003 and 2005 data sets. We included both PCS HRQLCI and MCS HRQL-CI as predictors in each regression model where the single-item general health status was the dependent variable in the first model and the single-item mental health status variable score was the dependent variable in the second model. The results in Table 6 show substantial improvement in terms of R2 for both outcomes and the expected relative strength of the regression coefficients corresponding to the two indices, PCS HRQL-CI being stronger in predicting general health status and MCS HRQL-CI being stronger in predicting mental health status. The improvement over Charlson-CI is again significant and
impressive. The relative boost to R2 by using the newly developed PCS HRQL-CI and MCS HRQL-CI by using Charlson-CI as the comparative benchmark is more pronounced for mental health status outcome, although the gain is profound under every model. The CharlsonCI-based models in Table 6 included only one predictor, namely the Charlson CI (point- or beta weight-based) and the health status items as response. In the second exercise, we selected a subset of 859 patients with asthma based on a set of questions in MEPS from the 2005 data set, which identify patients who had an asthma diagnosis at the time of the survey and predicted the HRQL measures in a regression model, while controlling for other comorbidities as captured through our two new HRQL-CI. We excluded the asthma CCC while constructing these indices. For PCS as response, the HRQLCI-adjusted model had a higher R2 of 0.42 compared with the Charlson-CI-adjusted model, where the R2 was 0.33. For the SF-12 MCS models, the HRQL-CI model had a higher R2 of 0.37, whereas the Charlson-CI model had an R2 of 0.11. Appendix D (see appendix on the journal’s Web site at www.elsevier.com) contains descriptive summaries for this subpopulation, whereas Appendix E (see appendix on the journal’s Web site at www.elsevier.com) contains the detailed regression results.
4. Discussion Controlling for comorbidity is essential in studies using general HRQL instruments. We have proven enormous gain in explaining the variation in HRQL measures by these new HRQL-CIs (from 2- to 16-fold increase in R2 across the models we studied) over the Charlson-CI. A researcher has superior power to characterize true associations between other predictors and such HRQL outcome measures by using this index as a controlling covariate in a regression model. One strength of this study was the use of a rigorous statistical approach to construct and validate the new HRQL-CI. Comorbidity, summarized as a single value representing overall burden of all illnesses present or as a single value that is controlled for when studying a specific illness, can influence HRQL [46]. In an analysis of results from randomized controlled clinical trials, Xuan et al. [47] demonstrated that comorbidity had an extensive effect on patients’ scores on generic HRQL instruments. Furthermore, in a cross-sectional sample of an Australian population, statistically and clinically significant decreases in usual activity levels and in SF-12 PCS-12 scores were identified when another chronic condition was present in additional to the index disease of interest [48]. A review article recently published by Fortin et al. [49,50] summarized the medical literature assessing multimorbidity and its influence on HRQL. They found a consistent and significant negative relationship between the number of medical conditions reported and the physical and mental domains of HRQL.
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
315
Table 3 HRQL-CIs beta coefficient- and point system-based indices Physical HRQL score (SF-12 PCS) as response CCC
Mental HRQL score (SF-12 MCS) as response Beta Point coefficientc assignmentd
Point Beta coefficienta assignmentb CCC
Paralysis 13.91 Rheumatoid arthritis and rheumatic disorders 10.55 Heart failure 10.36 Systemic lupus erythematosus 6.91 Ischemic heart disease 6.28 Osteoarthritis/nontraumatic joint disorders 6.07 Hepatitis 5.46 Diabetes 4.89 Degenerative neurologic disorders 4.78 Peripheral and central vascular diseases 4.70 Spinal column disorders 4.38 Obstructive pulmonary disease 3.79 Gastric and duodenal ulcer 3.33 Hypertension 3.21 Asthma 2.69 Arrhythmias 2.03 Esophageal disorders 1.63 Thyroid disorders 1.60 Vision disorders 1.53 Anxiety, depression 1.06
3 3 3 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1
Affective disorders, schizophrenia, other psychoses 8.70 Anxiety, depression 8.24 HIV infection 5.05 Epilepsy, convulsions 3.12 Hepatitis 2.97 Systemic lupus erythematosus 2.73 Heart failure 2.19 Headaches 1.87 Biliary and liver disorders 1.74 Anemia 1.53 Gastric and duodenal ulcer 1.44 Degenerative neurologic disorders 1.23 Diabetes 1.20 Asthma 1.15 Spinal column disorders 0.74
3 3 3 2 2 2 2 1 1 1 1 1 1 1 1
Abbreviations: HRQL-CI, health-related quality of life comorbidity index; SF-12, Short Form 12; PCS, physical component summary; MCS, mental component summary; CCC, clinical classification code; HIV, human immunodeficiency virus; MEPS, medical expenditure panel survey. a Intercept: 53.93; R2: 0.28. b R2: 0.27. c Intercept: 52.84; R2: 0.16. d R2: 0.15; All estimates are based on the analysis adjusted by MEPS survey weights.
In patients with chronic obstructive pulmonary disease (COPD), a CI improved predictive models of HRQL scores, as measured by the SF-36, when compared with models containing COPD only.[51] A study reported by Byles
et al. [52] found that the number of conditions noted by patients was associated with reduced HRQL as measured by the SF-36. The importance of this study is that symptomatic but not life-threatening conditions were included.
Table 4 Charlson-CIs [43] Beta coefficient-based index Diagnoses
Point-based indexa
PCS as responseb
MCS as responsec
Metastatic solid tumor Moderate or severe liver disease Diabetes Hemiplegia Leukemia Lymphoma Moderate or severe renal disease Any tumor Heart failure Chronic pulmonary disease Connective tissue disease Cerebral vascular disease Dementia Myocardial infarction Mild liver disease Peripheral vascular disease Ulcer disease
6 3 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1
4.38 4.76 7.20 9.36 14.17 0.86 8.60 3.32 13.58 4.43 13.09 3.03 11.5571 7.60 6.37 6.73 6.46
2.71 5.03 1.21 3.20 4.60 0.78 2.12 0.52 3.05 1.72 4.12 0.16 3.04 0.88 5.95 1.75 1.57
Abbreviations: CIs, comorbidity indices; PCS, physical component summary; MCS, mental component summary; MEPS, medical expenditure panel survey. a R2: 0.095 for PCS; R2: 0.0049 for MCS. b Intercept: 50.28; R2: 0.14. c Intercept: 50.49; R2: 0.013; All estimates are based on the analysis adjusted by MEPS survey weights.
316
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
Table 5 Average prediction errors for the SF-12 PCS and MCS in year 2003a and 2005 HRQL-CIs
Charlson-CIs
Dependent variable
Beta coefficient-based
Point system-based
Beta coefficient-based
Point system-based
PCS (2003) MCS (2003) PCS (2005) MCS (2005)
95.85 94.39 96.92 94.88
97.53 94.21 98.10 93.78
110.90 108.09 115.03 109.41
114.67 111.13 118.73 110.15
Abbreviations: SF-12, Short Form 12; PCS, physical component summary; MCS, mental component summary; HRQL, health-related quality of life; CIs, comorbidity indices. a Prediction errors in 2003 were calculated based on 10-fold cross-validation approach by selecting 10% of observations in the 2003 database randomly and predicting with the fitted models provided in Table 3 and 4. Prediction error is calculated by computing 1/nS(observedpredicted)2 for each random split of the data and then averaging over 100 such random samples.
The HRQL-CI is a compilation of conditions that are associated with both physical and mental-related functional status. For example, the CCCs with the greatest association with PCS or physical-related health status included musculoskeletal, cardiovascular, and neurologic conditions. The same is true with the MCS-related CI, with anxiety, depression, and conditions associated with more limiting mental illnesses have the greatest association with MCS score. It is interesting that a number of physical-related conditions appear in the MCS index, such as asthma, heart failure, neurologic conditions, and pain-related conditions. Research in the area of HRQL has demonstrated that these conditions are often associated with depression and/or anxiety. The new HRQL-CI outperformed a commonly used version of the Charlson-CI. Prediction errors for the SF-12
PCS and MCS in the validation year analysis demonstrated consistently better prediction and lower error terms for the HRQL-CI as compared with the Charlson-CI models. The regression coefficients for both point-based and beta weight-based scored HRQL-CI models obtained in the 2005 validation data set exhibited much stronger association with the outcome than that of the Charlson-CI models with two additional HRQL indicators, the two core health status questions asked in the MEPS survey. Similar gains in power were noted in the regression analysis on the asthmatic subpopulation with lower P-values noted in the HRQL-CI adjusted model. Thus the HRQL-CI outperformed the Charlson-CI in the 2003 development analysis, in the 2005 validation analysis, and in the disease-specific example using asthma patients.
Fig. 3. Histogram of the (observedpredicted) values for the 12,813 observations in the 2005 MEPS database by the CCC-based new HRQL-CI and the Charlson-CI. Abbreviations: MEPS, Medical Expenditure Panel Survey; CCC, clinical classification code; HRQL, health-related quality of life; CI, comorbidity index; Charlson-CI, Charlson comorbidity index; PCS, physical component summary; MCS, mental component summary.
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
317
Table 6 Comparisons of R2 when the outcome of interest is MEPS core single-item health status scores, namely general health status and mental health status in year 2003 and 2005 Outcome Health status variablea (y)
HRQL-CI Beta coefficientscored modelsb
HRQL-CI Point-scored modelsb
Charlson-CI Beta-coefficientscored modelsb
Charlson-CI Point-scored modelsb
General health status (2003) Model R2 PCS-index specific P-value MCS-index specific b-coefficient, P-value Charlson-CI variable b-coefficient, P-value
0.16 0.06, P!0.001 0.04, P!0.001 d
0.15 0.14, P!0.001 0.09, P!0.001 d
0.098 0.07, P!0.001 0.03, P50.53 d
0.079 d d 0.29, P!0.001
Mental health status (2003) Model R2 PCS-HRQL-CI variable b-coefficient, P-value MCS-HRQL-CI variable b-coefficient, P-value Charlson-CI variable b-coefficient, P-value
0.14 0.02, P!0.001 0.08, P!0.001 d
0.13 0.02, P!0.001 0.20, P!0.001 d
0.027 0.03, P!0.001 0.05, P50.0025 d
0.019 d d 0.13, P!0.001
General health status (2005) Model R2 PCS-HRQL-CI variable b-coefficient, P-value MCS-HRQL-CI variable b-coefficient, P-value Charlson-CI variable b-coefficient, P-value
0.16 0.06, P!0.001 0.04, P!0.001 d
0.16 0.15, P!0.001 0.08, P!0.001 d
0.097 0.07, P!0.001 0.05, P50.0025 d
0.077 d d 0.27, P!0.001
Mental health status (2005) Model R2 PCS-HRQL-CI variable b-coefficient, P-value MCS-HRQL-CI variable b-coefficient, P-value Charlson-CI variable b-coefficient, P-value
0.1323 0.02, P!0.001 0.08, P!0.001 d
0.1268 0.03, P!0.001 0.19, P!0.001 d
0.02627 0.10, P!0.001 0.04, P50.0198 d
0.02086 d d 0.14, P!0.001
Abbreviations: MEPS, medical expenditure panel survey; HRQL, health-related quality of life; CI, comorbidity index; PCS, physical component summary; MCS, mental component summary. a Both the general and mental health status scores range from 1 to 5 indicating excellent to poor general and mental health perceptions, respectively. b The PCS and MCS beta coefficient-based and point-based indices were computed by summing up the magnitude of the beta weights (without the negative sign) or point weights for the diseases that are presented in Table 3. In both cases, high values of indices indicate poor health condition. Similarly, the Charlson indices are defined by beta weights and point weights as indicated in Table 4. In each of the above situations, we fitted a cluster-adjusted regression model of the following form with two covariates: (a) Outcome 5 b0 þ b1 HRQL-PCS-CI (beta- or point-based) þ b1 HRQL-MCS-CI (beta- or point-based) þ error; For Charlson-CI we fitted model with single covariate: (b) Outcome 5 b0 þ b1 Charlson-CI (Beta- or point-based) þ error.
Two uses of CIs are to predict a health outcome and to control for the potential influence of other illnesses when analyzing a specific outcome. In the latter case, researchers analyzing HRQL data of a specific illness will want to control for the potential influence of other illnesses on the dependent variable, especially when a general measure of HRQL is used, such as the SF-36. This study chose the illness of asthma to test whether the new HRQL-CI provided greater predictive ability than the Charlson-CI when focusing only on patients with asthma using the SF-12 as the measure of HRQL. Previous studies have demonstrated that comorbid conditions do indeed have an effect on predictive modeling when asthma is the disease of interest [48,53,54]. Presently, the HRQL-CI uses beta weights or an empirically derived point system to score. Users of the index would be able to use the points or beta weights as they are listed here or derive weights from their own patient population using the list of conditions in the index. Which one to use is a fair question at this point. Based strictly on significance values, the beta weight-derived score performed marginally better than the point system in most analyses. However, the point system is easier to use and implement, and for most practical purposes has the same relative significance as the beta weight-derived scores.
4.1. Limitations Not all medical conditions were included in the initial set from which the final indices were created. One excluded group were the relatively rare conditions, such as cystic fibrosis, a chronic condition that involves the lungs and exocrine glands. HRQL is impaired significantly in advanced cases. However, the number of adult persons in the MEPS data set in 2003 that reported this condition was so low that the group was excluded. There were other conditions that are more common, which were not included because of the fact that no adult person in the MEPS data set in 2003 reported the condition. An example would be pancreatic cancer. Researchers electing to evaluate the HRQL-CI may choose to explore the influence of these conditions when present in the data sets they work with. Other excluded conditions included gender-specific conditions. The index was intended to be used in both genders, so that excluding gender-specific conditions allowed the resulting indices to be generalizable. Future research should focus on the relative contribution of gender-specific conditions. Another limitation is that the general HRQL instrument used for the development of the HRQL-CI was the SF-12. Other general measures exist, which provide a single score
318
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319
or which have multiple dimensions. Our study demonstrated excellent performance when the general health and mental health single-item health status measures were analyzed. Future studies using other general HRQL instruments as the dependent variable are warranted to provide future validation that the new index does indeed function well. This study compared the new indices with the CharlsonCI, a commonly used index. Other indices exist, which require assessment to determine whether the new HRQL-CI will outperform them as well. 5. Conclusion The new HRQL-CI is a valid risk adjustment index. It outperforms the Charlson-CI when predicting health status for the SF-12 PCS and MCS, the two core single-item health status measures as well as in an asthma specific population. Further work is necessary to test its performance in other patient populations. Appendix Supplementary material Supplementary material can be found, in the online version, at 10.1016/j.jclinepi.2010.01.025. References [1] Hall SF. A user’s guide to selecting a comorbidity index for clinical research. J Clin Epidemiol 2006;59:849e55. [2] Greenfield S, Nelson EC. Recent developments and future issues in the use of health status assessment measures in clinical setting. Med Care 1992;30:MS23e41. [3] Hitchcock Noel P, Williams JW, Unutzer J, Worchel J, Lee S, Cornell J, et al. Depression and comorbid illness in elderly primary care patients: impact on multiple domains of health status and well-being. Ann Fam Med 2004;2:555e62. [4] Deyo DA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with IOCD-9-CM administrative databases. J Clin Epidemiol 1992;45:613e9. [5] Elixhauser AC, Steiner C, Harris DR, Coffey RM. Comorbidity measures to use with administrative data. Med Care 1998;36:8e27. [6] Greenfield S, Apolone G, McNeil BJ, Cleary PD. The importance of co-existent disease in the occurrence of postoperative complications and one-year recovery in patients undergoing total hip replacement. Comorbidity and outcomes after hip replacement. Med Care 1993;31:141e54. [7] Linn BS, Linn MW, Gurel L. Cumulative illness rating scale. J Am Geriatr Soc 1968;16:622e7. [8] Miller MD, Paradis CF, Houck PR. Rating chronic medical illness burden in geropsychiatric practice and research: application of the Cumulative Illness Rating scale. Psychiatry Res 1992;41:237e48. [9] Parkerson GR, Broadhead WE, Tse CK. The Duke Severity of Illness Checklist (DUSOI) for measurement of severity and comorbidity. J Clin Epidemiol 1993;46:379e93. [10] Roe CJ, Dodich N, Kulinskaya E, Adam WR. Comorbidities and prediction of length of hospital stay. Aust N Z J Med 1998;28:811e5. [11] Charlson ME, Szatrowski TP, Peterson J, Gold J. Validation of a combined comorbidity index. J Clin Epidemiol 1994;47:1245e51.
[12] Voaklander DC, Kelly KD, Jones KCA, Suarez-Almazor ME. Self report co-morbidity and health related quality of lifeda comparison with record based co-morbidity measures. Soc Indic Res 2004;66:213e28. [13] Wang PS, Walker A, Tsuang M, Orav EJ, Levin R, Avorn J. Strategies for improving comorbidity measures based on Medicare and Medicaid claims data. J Clin Epidemiol 2000;53:571e8. [14] Guralnik JM. Assessing the impact of comorbidity in the older population. Ann Epidemiol 1996;6:376e80. [15] Iezzoni LI. Dimensions of risk. In: Iezzoni LI, editor. Risk adjustment for measuring healthcare outcomes. Chicago, IL: Health Administration Press; 1997. p. 43e168. [16] Ghali WA, Quan H, Brandt R. Risk adjustment using administrative data. Impact of a diagnosis-type indicator. J Gen Intern Med 2001;16: 519e24. [17] Incalzi RA, Capparella O, Gemma A, Landi F, Bruno E, Meo F, et al. The interaction between age and comorbidity contributes to predicting the mortality of geriatric patients in the acute-care hospital. J Intern Med 1997;242:291e8. [18] Menotti A, Mulder I, Nissinen A, Giampaoli S, Feskens EJM, Kromhout D. Prevalence of morbidity and multimorbidity in elderly male populations and their impact on 10-year all-cause mortality: the FINE study (Finland, Italy, Netherlands, Elderly). J Clin Epidemiol 2001;54:680e6. [19] Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care 2004;42:355e60. [20] Holman CDJ, Preen DB, Baynham NJ, Finn JC, Semmens JB. A multipurpose comorbidity scoring system performed better than the Charlson index. J Clin Epidemiol 2005;58:1006e14. [21] Bravo G, Dubois MF, Herert R, DeWals P, Messier L. A prospective evaluation of the Charlson comorbidity index for use in long-term care patients. J Am Geriatr Soc 2002;50:740e5. [22] Librero J, Peiro S, Ordinana R. Chronic comorbidity and outcomes of hospital care: length of stay, mortality, and readmission at 30 and 365 days. J Clin Epidemiol 1999;52:171e9. [23] Rochon PA, Katz JN, Morrow LA, McGlinchey-Berroth R, Ahlquist MM, Sakkarati M, et al. Comorbid illness is associated with survival and length of hospital stay in patients with chronic disability: a prospective comparison of three comorbidity indices. Med Care 1996;34:1093e101. [24] Fishman PA, Goodman MJ, Hornbrook MC, Meenan RT, Bachman DJ, O’Keefe Rosetti MC. Risk adjustment using automated ambulatory pharmacy data. The RxRisk model. Med Care 2003;41: 84e99. [25] Hamilton SM, Johnston DWC, Voaklander DC. An analysis of surgical outcomes following regionalization in the Canadian health care system. Can J Surg 2001;44:51e8. [26] Jones CA, Voaklander DC, Johnston DWC, Suarez-Almazor ME. Health-related quality of life outcomes after total hip and knee arthroplasties in a community-based sample. J Rheumatol 2000;27: 1745e52. [27] Jones CA, Voaklander DC, Johnston DWC, Suarez-Almazor ME. The effect of age on pain, function, and quality of life. Arch Intern Med 2001;161:454e60. [28] Amin SP, Mullins SC, Duncan BS, Blanford L. Direct health care costs for treatment of diabetes mellitus and hypertension in an IPA-group-model HMO. Am J Health Syst Pharm 1999;56: 1515e20. [29] Guyatt GH, Feeny DH, Patrick DKL. Measuring health-related quality of life. Ann Intern Med 1993;118:622e9. [30] Fitzpatrick R, Fletcher A, Gore S. Quality of life measures in health care: applications and issues in assessment. Br Med J 1992;305:1074e7. [31] Michelson H, Bolund C, Brandberg Y. Multiple chronic health problems are negatively associated with health related quality of life (HRQoL) irrespective of age. Qual Life Res 2001; 9:1093e104.
B. Mukherjee et al. / Journal of Clinical Epidemiology 64 (2011) 309e319 [32] Rijken M, van Kerkhof M, Dekker J, Schellevis FG. Comorbidity of chronic diseases: effects of disease pairs on physical and mental functioning. Qual Life Res 2005;14:45e55. [33] Selim AJ, Fincke G, Ren XS, Lee A, Roger WH, Miller DR, et al. Comorbidity assessments based on patient report: results from the Veterans health Study. J Ambul Care Manage 2004;27: 281e95. [34] Kazis LE, Miller DR, Clark J, Skinner K, Lee A, Rogers W, et al. Health-related quality of life in patients served by the department of Veterans Affairs. Results from the Veterans Health Study. Arch Intern Med 1998;158:626e32. [35] Alonsao J, Ferrer M, Gandek B, Ware JE, Aaronson NK, Mosconi P, et al., the IQOLA Project Group. Health-related quality of life associated with chronic conditions in eight countries: results from the International Quality of Life Assessment (IQOLA) project. Qual Life Res 2004;13:283e98. [36] Stewart AL, Ware JE, editors. Measuring functioning and well-being: the medical outcomes study approach. Durham, NC: Duke University Press; 1992. [37] Temkin NR, Dikmen S, Machamer J, McLean A. General versus disease-specific measures: further work on the Sickness Impact Profile for head injury. Med Care 1989;27:S44e53. [38] Kantz ME, Harris WJ, Levitsky K. Methods for assessing condition-specific and generic functional status outcomes after total knee replacement. Med Care 1992;30:MS240e52. [39] Patrick DL, Erickson P. Health status and health policy: allocating resources to health care. New York, NY: Oxford University Press; 1993. [40] Mauskopf JA, Austin R, Dix LP, Berzon RA. Estimating the value of a generic quality of life measure. Med Care 1995;33:AS195e202. [41] Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996;34:220. [42] D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson index with administrative data bases. J Clin Epidemiol 1996;49:1429e33.
319
[43] D’Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson comorbidity index. Methods Inf Med 1993;32: 382e7. [44] Katz JN, Chang LC, Sangha O, Fossel AH, Bates DW. Can comorbidity be measured by questionnaire rather than medical record review? Med Care 1996;34:73e84. [45] Tibshirani R. Regression Shrinkage and Selection via the Lasso. J R Stat Soc Series B Stat Methodol 1996;58:267e88. [46] Gijsen R, Hoeymans N, Schellevis FG, Ruwaard D, Satariano WA, van den Bos GAM. Causes and consequences of comorbidity: a review. J Clin Epidemiol 2001;54:661e7. [47] Xuan J, Kirchdoerfer LJ, Boyer JG, Norwood GJ. Effects of comorbidity on health-related quality of life scores: an analysis of clinical trial data. Clin Ther 1999;21:383e403. [48] Adams RJ, Wilson DH, Taylor AW, Daly A, d’Espaignet ET, DalGrande E, et al. Coexistent chronic conditions and asthma quality of life: a population-based study. Chest 2006;129:285e91. [49] Fortin M, Lapointe L, Hudon C, Vanasse A, Ntetu AL, Maltais D. Multimorbidity and quality of life in primary care: a systematic review. Health Quality of Life Outcomes 2004;2:51. [50] Fortin M, Bravo G, Hudon C, Lapointe L, Almirall J, Dubois MF, et al. Relationship between multimorbidity and health-related quality of life of patients in primary care. Qual Life Res 2006;15:83e91. [51] VanManen JG, Bindels PJE, Dekker FW, Izermans CJ, Bottema BJAM, VanDerZee JS, et al. Added value of co-morbidity in predicting health-related quality of life in COPD patients. Respir Med 2001;95:496e504. [52] Byles JE, D’Este C, Parkinson L, O’Connell R, Treloar C. Single index of multimorbidity did not predict multiple outcomes. J Clin Epidemiol 2005;58:997e1005. [53] Schneider A, Lowe B, Meyer FJ, Biessecker K, Joos S, Szecsenyi J. Depression and panic disorder as predictors of health outcomes for patients with asthma in primary care. Respir Med 2008;102:359e66. [54] Wijnhoven HA, Kriegsman DM, Hesselink AE, de Haan M, Schellevis FG. The influence of co-morbidity on health-related quality of life in asthma and COPD patients. Respir Med 2003;97:468e75.