Cancer-specific administrative data–based comorbidity indices provided valid alternative to Charlson and National Cancer Institute Indices

Journal of Clinical Epidemiology 67 (2014) 586e595

Cancer-specific administrative dataebased comorbidity indices provided valid alternative to Charlson and National Cancer Institute Indices Diana Sarfatia,*, Jason Gurneya, James Stanleya, Clare Salmondb, Peter Cramptonc, Elizabeth Dennettd, Jonathan Koeae, Neil Pearcef,g a

Department of Public Health, School of Medicine and Health Sciences, University of Otago, PO Box 7343, Wellington South, Wellington 6022, New Zealand b Retired c Faculty of Health Sciences, University of Otago, PO Box 56, Dunedin 9054, New Zealand d Department of Surgery and Anaesthesia, School of Medicine and Health Sciences, University of Otago, PO Box 7343, Wellington South, Wellington 6022, New Zealand e Department of Surgery, North Shore Hospital, Waitemata District Health Board, Private Bag 93-503 Takapuna, Auckland 0740, New Zealand f Centre for Public Health Research, Massey University, PO Box 756, Wellington 6022, New Zealand g Department of Medical Statistics, London School of Hygiene and Tropical Medicine, Keppell Street, Bloomsbury, London WC1E7HT, UK Accepted 29 November 2013; Published online 25 February 2014

Abstract Objective: We aimed to develop and validate administrative dataebased comorbidity indices for a range of cancer types that included all relevant concomitant conditions. Study Design and Settings: Patients diagnosed with colorectal, breast, gynecological, upper gastrointestinal, or urological cancers identified from the National Cancer Registry between July 1, 2006 and June 30, 2008 for the development cohort (n 5 14,096) and July 1, 2008 to December 31, 2009 for the validation cohort (n 5 11,014) were identified. A total of 50 conditions were identified using hospital discharge data before cancer diagnosis. Five site-specific indices and a combined site index were developed, with conditions weighted according to their log hazard ratios from age- and stage-adjusted Cox regression models with noncancer death as the outcome. We compared the performance of these indices (the C3 indices) with the Charlson and National Cancer Institute (NCI) comorbidity indices. Results: The correlation between the Charlson and C3 index scores ranged between 0.61 and 0.78. The C3 index outperformed the Charlson and NCI indices for all sites combined, colorectal, and upper gastrointestinal cancer, performing similarly for urological, breast, and gynecological cancers. Conclusion: The C3 indices provide a valid alternative to measuring comorbidity in cancer populations, in some cases providing a modest improvement over other indices. Ó 2014 Elsevier Inc. All rights reserved. Keywords: Comorbidity; Multimorbidity; Cancer; Measurement; Validity; Indices

1. Introduction Patients diagnosed with cancer frequently have other chronic medical conditions. These concomitant conditions, or comorbidity, can affect how or when a patient is diagnosed with cancer, the treatment options available or offered, and a patients’ ultimate prognosis [1e13]. At an individual level, a clinician can assess the presence and impact of comorbidity in a patient diagnosed with cancer.

Funding sources: This work was funded by a grant from the Health Research Council of New Zealand (HRC 10/404). Conflicts of interest: None. * Corresponding author. Tel.: þ64-27-480-5660; fax: þ64 4 389 5319. E-mail address: [email protected] (D. Sarfati). 0895-4356/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2013.11.012

However, at the population level, assessing comorbidity is much more difficult. The severity of a patient’s comorbidity depends on the number, pattern, and severity of conditions present, and the likely impact may vary depending on the specific cancer diagnosed [4,11,14e17]. Despite these difficulties, measuring comorbidity at the population level is important, as it provides researchers, policy makers, and health service planners with the necessary tools to allow them to stratify patients into groups according to risk in the same way they do for demographic and disease factors such as age and tumor stage [16,18]. There have been many attempts to measure comorbidity in cancer patient populations [4]. The most commonly cited approach is that of Charlson et al. [19]. These investigators identified all comorbid conditions from the medical records

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595

What is new? Key findings  This study provides validated comorbidity indices for cancer populations using administrative hospitalization data. Site-specific indices did not outperform a more general cancer index. The new indices included a more extensive list of conditions, and more up-to-date and site-specific weights. They outperformed the Charlson and National Cancer Institute (NCI) indices for all sites combined and colorectal cancer, and to a lesser extent for upper gastrointestinal cancers. For other sites, the new indices performed similarly to the Charlson index. What this adds to what was known?  The C3 indices provide a valid alterative to the Charlson or NCI indices in cancer populations, although for many purposes any of these three measures of comorbidity will give similar results. What is the implications and what should change now?  Consideration of comorbidity in studies of cancer population is more important than the measure used to describe it.

of a relatively small cohort of 559 general medical patients admitted to a single hospital. They assessed the impact of each condition on 1-year mortality, and excluded any with a relative risk of less than 1.2. They developed a weighted index, with the weights being equivalent to the (rounded) adjusted relative risks for mortality for each condition, with a maximum weight of six. Subsequently, the Charlson index has been validated on data from administrative records [20e27]. The Charlson index has been used as the basis for other comorbidity indices, most notably the National Cancer Institute (NCI) comorbidity index, which uses the same conditions, but uses the regression coefficients (ie, the log of the relative risk rather than the relative risk itself) of the association of each condition with 1-year mortality to assign weights [28]. These latter investigators also argued for the importance of site-specific weights in the development of comorbidity indices for use with cancer patients. Despite this extensive work, there has been little consideration of whether the conditions that Charlson and colleagues identified in their general medical cohort nearly 30 years ago are those that are most important for cancer patients today. The Charlson index includes some conditions that may not have an impact on survival among patients with cancer because of substantial improvements in management (eg, peptic ulcer disease), and it excludes some that do have such an impact (eg, noncerebrovascular

587

neurological conditions and major psychiatric conditions) [19]. Comorbidity is a composite construct defined by the presence or absence of concomitant conditions. As such, theoretically, comorbidity will be best measured when as many relevant items are included as possible, and it is likely that the weighting of individual conditions will be less important than their inclusion [29]. However, a ‘‘reduced’’ comorbidity index involving a smaller number of conditions may be desirable for practical reasons. We aimed to develop administrative dataebased indices, which address some of the issues identified in previous work, to assess whether they performed better than other wellestablished comorbidity indices, particularly the Charlson and NCI indices. We wanted to ensure that all conditions that may be important in defining comorbidity in cancer patient populations were included. We also wanted to account for the possibility that the importance of these conditions may vary depending on the primary site of cancer, either because of different underlying prevalence rates of specific conditions or because of differential impact of individual conditions on particular cancer sites. We used data from more than 14,000 patients diagnosed with one of the nine cancers (colon, rectal, breast, ovarian, endometrial, stomach, liver, bladder, or kidney) identified from the New Zealand cancer registry, with data identified from hospitalizations for the 5 years before diagnosis on 50 potentially concomitant conditions. We combined these into site-specific and non-siteespecific indices and evaluated the performance of these against the Charlson and NCI (site-specific) indices.

2. Methods 2.1. Study population and data The development cohort consisted of patients who had been diagnosed with colon (ICD-10 C18-19), rectal (C20), uterine (C54), ovarian (C56), liver (C22), stomach (C16), female breast (C50), kidney (C64), or bladder (C67) cancers between July 1, 2006 and June 30, 2008 (‘‘development cohort’’). To validate the indices, we obtained data from patients diagnosed with these same cancers diagnosed between July 1, 2008 and December 31, 2009 (‘‘validation cohort’’). Patients were identified from the New Zealand Cancer Registry, which is a populationbased register of all primary cancers diagnosed in New Zealand excluding nonmelanoma skin cancers. Patients were excluded if they were diagnosed with carcinoma in situ, aged younger than 25 years at diagnosis, normally resident outside New Zealand, had a previous diagnosis with the same cancer, or diagnosed at postmortem. We collected data on the extent of disease from the Cancer Registry, using the Surveillance Epidemiology and End Results Summary Staging System and categorized the extent of disease into local, regional, distant, and unknown [30].

588

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595

We also collected patients’ demographic details, such as age at diagnosis, sex, and ethnicity (Maori or non-Maori). The Cancer Registry data were linked using a unique identification number (National Health Index number) to routine hospital discharge data (National Minimum Data set), and national mortality data held by the New Zealand Ministry of Health. Details on primary and secondary diagnoses for all public hospital admissions for at least 5 years before the cancer diagnosis were provided. Trained coders based within District Health Boards use all available data in the clinical record to identify relevant diagnoses and procedures. Codes are assigned using appropriate coding standards and conventions according to the ICD-10-AM procedures. Outcome data (vital status and cause of death) were obtained by linking study patients to the national mortality database, with follow-up to the end of 2009 for the development cohort and 2010 for the validation cohort (ie, at least 12 months follow-up for each patient). Patients whose deaths were not recorded in the mortality database were assumed to be still alive at the end of follow-up. 2.2. Identification of important conditions The process for identifying important comorbid conditions has been previously described [31]. The aim was to identify all important concurrent chronic conditions that were likely to have an impact on function or length of life, among individuals with the specified cancers. For this reason, conditions that were likely to be acute, selflimiting, and/or highly localized were not included. To ensure a standard set of conditions, gender-specific conditions other than comorbid malignancies were also excluded. We identified categories of conditions used in other comorbidity indices and/or those identified by clinicians as potentially important to cancer patients. Conditions that might be closely related to the primary cancer of interest or its treatment were excluded; for example, codes related to upper gastrointestinal disease were excluded for patients with stomach cancer. For each cancer site, any codes relating to malignancy at that site were omitted. For all patients, malignancies of brain, liver, bone, or lung were excluded because of the possibility that these were in fact metastases caused by the primary disease. We then carefully coded the resulting 50 condition categories, again with input from clinicians to ensure that only the specific conditions that they considered relevant to function or length of life were included in these categories. We defined a period of 5 years before diagnosis up until an index hospitalization occurring at time of diagnosis (or date of diagnosis in lieu of an index hospitalization) as our time frame of interest to identify comorbid conditions. The index hospitalization was the first admission that occurred at or within 4 weeks of the date of diagnosis. Where no such admission was identifiable, we took the date of diagnosis of cancer as the index date. Some conditions were only included if they were identified before the

diagnosis of cancer or index admission to exclude complications of the primary disease or its treatment; specifically, we excluded myocardial infarction, congestive heart failure, pulmonary embolism, anxiety and behavioral disorders, anemia, hypertension, and cardiac arrhythmias. 2.3. Index development To maximize the numbers available for index development, we combined cancers into five site groups, namely colorectal, breast, gynecological (ovary and uterine), upper gastrointestinal (liver and stomach), and urological (kidney and bladder). Within these site groups, we identified all conditions with a prevalence of 0.5% or greater. These are shown in Table 2. We fitted Cox proportional hazards regression models with noncancer death as our outcome, and estimated crude and age-/stage-adjusted hazard ratios for each comorbid condition (using separate models for each condition). Patients who died of cancer and those who were alive at the end of follow up were censored at this time. Noncancer death was selected as the outcome because both cancer and all-cause mortality are heavily influenced by the prognosis of the cancer itself and therefore will be less sensitive to the impacts of other conditions when the cancer prognosis is poor. We used the parameter estimates of the age- and stageadjusted models for each condition as weights in the sitespecific comorbidity indices. Where there were fewer than five noncancer deaths within a particular site among patients with a given comorbid condition, we substituted the parameter estimates from the ‘‘all-site’’ model for that condition, adjusted for the mean parameter estimates of the conditions that had been included for the given site (for more information, see Appendix at www.jclinepi.com). We also calculated an ‘‘all-sites’’ version of the C3 index in which all 42 conditions used in the C3 indices were included but the weights were now standardized across sites, that is, parameter estimates from Cox regression models of noncancer death for all included sites combined, adjusted for age, site, and stage. As for site-specific indices, conditions that may have been caused by a particular primary cancer were excluded from patients with that cancer. The index scores were calculated for each patient by adding together all parameter estimates (ie, the log hazard ratios) for all comorbid conditions recorded for that patient. We refer to these scores as either site-specific or all-site C3 index scores (because they were developed as part of the Cancer, Care, and Comorbidity or C3 studies). Scores were treated as continuous except where specified otherwise. We calculated Charlson scores using the coding by Quan et al. [25] and the weights specified by Charlson et al. [19] excluding cancer-related codes for all sites. For colorectal and breast cancers, we also calculated NCI (site-specific) indices using the weights specified by Klabunde et al. [28]. Site-specific weights were not provided for our other cancer sites of interest. For the Charlson and NCI indices,

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595

we only included congestive heart failure and myocardial infarction if they were recorded before index admission. For each of these indices, the score was calculated by summing the weights of conditions that were present for a given individual. As with the C3 Index, the Charlson and NCI scores were treated as continuous except for some descriptive analyses where they were categorized into scores of 0, 1, 2, and 3þ. 2.4. Validation To test for concurrent validity, we calculated Spearman’s rank correlation coefficient (SAS v 9.2; Proc Corr; SAS Institute, Inc., Cary, NC) to compare the C3 indices with the Charlson index. Because the Charlson index is itself not a perfect measure of comorbidity, we were aiming for a correlation in the range of 0.4e0.8 [29]. Next, we compared the ability of each comorbidity index to discriminate between those who died and those who did not using a rank correlation measure of goodness of fit, c, which is the proportion of pairs of observations that are concordant, allowing for tied observations. This is the equivalent of calculating a receiver operating curve for the outcome based on the predicted probabilities from the logistic regression models. We fitted two sets of logistic regression models for each site; one for 1-year all-cause mortality and the other for 1-year noncancer mortality. Within each of these sets, we calculated the c-statistic for baseline models, which included age, stage, and sex (for relevant cancer sites), then recalculated the c-statistics for each model after the addition of the site-specific C3, NCI, or Charlson indices. We compared the c-statistics for models that included each measure of comorbidity to the baseline models and to each other. Confidence intervals for the c-index were calculated using bootstrap estimation processes: for each cancer site, the c-index was calculated in each of 10,000 bootstrap samples (sampled with replacement from the original site-specific data set, using PROC SURVEYSELECT) for both outcomes and for each of the four models. The same 10,000 bootstrap samples were used for all models for each cancer site, to allow estimation of differences in model performance between the comorbidity indices. Differences between the models were calculated within each bootstrap sample, and the empirical distribution of differences was used to calculate the median difference and 95% confidence interval. The reported c-indices in the results represent the median of the bootstrapped c-indices, with an empirical 95% confidence interval (ie, the 2.5th and 97.5th percentile of the bootstrapped c-indices). The differences in c-indices by comorbidity measure were also calculated in this manner. Medians and confidence intervals were calculated based only on bootstrap iterations that successfully converged. Finally, to assess for goodness of fit, we compared baseline models of 1-year (noncancer and all-cause) mortality,

589

which included age, sex, and stage with models that included each of the measures of comorbidity, using the Akaike information criterion (AIC). The AIC uses the log likelihood and number of parameters in each model to assess model fit. The absolute value of the AIC is not relevant but a lower value suggests a better model, with an absolute difference of 10 being considered important [32]. All analyses were carried out in SAS v9.2. Approval for this study was granted by the New Zealand Multi-Region Ethics Committee.

3. Results 3.1. Description of cohorts Table 1 shows the sex, age, and ethnicity of patients in both the development and validation cohorts, and the number and crude proportions of all-cause, and noncancer deaths by cancer site. Just more than half the patients with colorectal cancer, and about two-thirds of those with upper gastrointestinal or urological cancers were male. Women with breast and gynecological cancers tended to be younger than patients diagnosed with other cancers, and Maori were overrepresented among those with cancers of the liver and stomach compared with other sites (21% of patients with liver or stomach cancer were Maori compared with 5e12% for other cancers). The proportion of patients dying from any cause over the entire follow-up period was highest for those with liver or stomach cancers (66e72%) and lowest for breast cancer (7.8e10.4%); these differences were largely driven by cancer-specific deaths. The proportion of patients dying from noncancer causes was low for all groups (1.3e6.1%), particularly among women with breast or gynecological cancers. 3.2. Development of index Table 2 shows the list of conditions included in each of the cancer site-specific indices, along with the parameter estimates and resulting hazard ratios from the Cox proportional hazard models from the development cohorts using noncancer mortality as outcome. The results show that, almost without exception, comorbid conditions were associated with an increased risk of noncancer death in these cancer site cohorts. The relative magnitude of the hazard ratio point estimates varied somewhat across the sites, but with the exception of diabetes without complications, all were greater than 1.0. Table 3 shows the descriptive statistics for the resultant comorbidity indices in the development cohorts. The ranges of C3 scores across individuals are similar by site, with the lowest scores at or just below 0 and the highest scores around 13e14. The C3 scores tended to be lower among patients with breast cancer, and highest among those with upper gastrointestinal cancers, a pattern that is entirely consistent with the age and likely comorbidity profiles of

590

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595

Table 1. Characteristics of five cancer cohorts in development and validation cohorts Colorectal

Breast

Developmenta Validationb Development Characteristics Sex Male Female Age, yr 25e49 50e64 65e74 75þ Ethnicity M aori Non-M aori Charlson score 0 1 2 3þ Deathsc All-cause Noncancer

n (%)

n (%)

n (%)

Gynecological

Liver/stomach

Validation

Development

Validation

n (%)

n (%)

n (%)

Urological

Development Validation Development n (%)

n (%)

2,812 (49) 2,113 (49) 0 (0) 0 (0) 0 (0) 0 (0) 2,564 (62) 1,934 (62) 5,076 (61) 4,059 (60) 1,223 (64) 1,041 (64)

652 (60) 309 (44)

508 (62) 225 (43)

990 (62) 470 (47)

752 (60) 382 (47)

351 1,196 1,672 2,157

127 264 263 307

85 218 185 245

138 393 385 544

122 301 307 404

(7) 267 (7) 1,423 (28) 1,120 (28) (22) 939 (23) 1,854 (37) 1,555 (38) (31) 1,222 (30) 901 (18) 713 (18) (40) 1,619 (40) 898 (18) 671 (17)

181 495 280 267

(15) (40) (23) (22)

176 418 243 204

(17) (40) (23) (20)

(13) (27) (27) (32)

(12) (30) (25) (33)

n (%)

Validation

(9) (27) (26) (37)

n (%)

(11) (27) (27) (36)

248 (5) 217 (5) 586 (12) 484 (12) 130 (11) 5,128 (95) 3,830 (95) 4,490 (88) 3,575 (88) 1,093 (89)

129 (12) 912 (88)

210 (22) 751 (78)

152 (21) 113 (8) 80 (7) 581 (79) 1,347 (92) 1,054 (93)

3,821 834 287 434

836 103 49 53

547 206 63 145

456 156 55 66

(71) 3,039 (75) 4,431 (87) 3,590 (88) (16) 520 (13) 380 (7) 252 (6) (5) 219 (5) 106 (2) 115 (3) (8) 269 (7) 159 (3) 102 (3)

1,957 (36) 1,199 (30) 255 (5) 152 (4)

527 (10) 130 (3)

317 (8) 80 (2)

960 174 33 56

(78) (14) (3) (5)

397 (32) 30 (2)

(80) (10) (5) (5)

274 (26) 14 (1)

(57) (21) (7) (15)

696 (72) 36 (4)

(62) 1,030 (71) (21) 226 (15) (8) 89 (6) (9) 115 (8)

485 (66) 36 (5)

618 (42) 77 (5)

813 177 64 80

(72) (16) (6) (7)

365 (32) 69 (6)

Colorectal: colon and rectal cancers; gynecological: uterine and ovarian cancers; urological: bladder and renal cancers. a Diagnosed between July 1, 2006 and June 30, 2008. b Diagnosed between July 1, 2008 and December 31, 2009. c During entire follow up period.

those cohorts. Similar associations are seen with the Charlson comorbidity scores, although patients were more likely to be identified as having comorbidity with the C3 indices than the Charlson index. 3.3. Performance and validation of C3 indices The Spearman’s rank correlation coefficients, which measured the correlation between ranked Charlson scores and C3 index scores, ranged from 0.61 for colorectal cancer to 0.78 for upper gastrointestinal cancers in the development cohorts (Table 3), with the correlation coefficient for all sites combined being 0.69, supporting concurrent validity of the C3 indices. The range of correlation coefficients between Charlson and the C3 indices were very similar in the validation cohorts (data not shown). Table 4 shows the results of analyses using the validation cohorts. These analyses compared the ability of C3 indices (both site-specific and all-sites versions) to discriminate between those who died (from any cause or from a noncancer cause) within a year of diagnosis compared with the NCI and the Charlson indices. For the all-cause death outcome, nonconvergence (owing to quasi-complete separation) of the logistic regression models was rare (O98% of bootstrap samples converged across all cancer sites), with a similarly high convergence rate for noncancer deaths for colorectal and urological cancer. However, for noncancer deaths, model convergence was only 90.9% for breast cancer patients; 62.3% for the upper gastrointestinal cancer group; and for the gynecological cancers, no bootstrapped model

converged for non-cancer deaths because of relatively small numbers of deaths in these categories. The performance of the Charlson and NCI indices was almost identical for both breast and colorectal cancer (the NCI index was not available for other sites). Similarly, there was almost no difference in the performance of the site-specific and all-sites versions of the C3 indices. For all sites combined and for colorectal cancer specifically, there were small but statistically significant improvements in c-statistics, as well as reductions in AIC in models that included C3 indices compared with those that included the Charlson index. Similarly, for upper gastrointestinal cancers, the C3 indices (both site-specific and all-sites versions) outperformed the Charlson index for noncancer death only. In contrast, there was little difference in c-statistics or AICs between models that included Charlson compared with those that included C3 indices for gynecological, breast, or urological cancers. Table 5 provides a list of the conditions included in the C3 (all-sites) index, and their weights.

4. Discussion This study has demonstrated the validity of new comorbidity indices specifically designed for cancer populations that outperformed the Charlson index for all-sites combined, colorectal cancer, and for upper gastrointestinal cancers (for noncancer death). There was little or no difference in the performance of the new indices and the Charlson

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595

591

Table 2. List of conditions in the site-specific C3 indices, coefficient estimates, and hazard ratios from age- and stage-adjusted Cox proportional hazards models with noncancer death as outcome in development cohorts Colorectal Conditions Alcohol abuse Anemia Angina Anxiety and behavioral disorders Cardiac arrhythmia Cardiac valve disorder Cerebrovascular disease Congestive heart failure Coagulopathy/blood disorders Connective tissue disease COPD and asthma Dementia Diabetes with no complications Diabetes with complications Endocrine disorders Epilepsy Eye problems GI disease Hepatitis: chronic viral Hypertension Inflammatory bowel disorder Inner ear disorder Intestinal disorders Joint or spinal disorders Liverdmoderate/severe disease Major psychiatric condition Malnutrition Metabolic disorder Myocardial infarction Neurological conditions excluding epilepsy Obesity Osteoporosis and bone disorders Other cardiac conditions Other malignancy Paralysis Peripheral nerve or muscular disorder Pulmonary circulation disorder Peripheral vascular disease Renal disease Sleep disorder Urinary tract disorder Venous insufficiency

Breast

Gynecological

Liver/stomach

Urological

Coefficient

HR

Coefficient

HR

Coefficient

HR

Coefficient

HR

Coefficient

HR

1.03a NI 0.58 0.99 0.84 0.98 1.21 1.16 0.56 0.59 1.00 1.14 0.13 0.64 0.68 NI 0.43 0.46 NI 0.78 0.55 0.43 NI 0.63 1.21 0.75 1.11a 0.51 0.94 1.14 0.43 0.40 0.53 0.20 1.28 1.05 1.14 0.47 1.18 1.35a 0.12 NI

2.8 NI 1.8 2.7 2.3 2.7 3.4 3.2 1.8 1.8 2.7 3.1 0.9 1.9 2.0 NI 1.5 1.6 NI 2.2 1.7 1.5 NI 1.9 3.4 2.1 3.0 1.7 2.6 3.1 1.5 1.5 1.7 1.2 3.6 2.9 3.1 1.6 3.3 3.9 1.1 NI

NI 0.64 0.40 0.59a 0.94 1.32 0.99 0.90 1.14 0.53a 0.96 0.94 0.18 1.01 0.61 NI 0.86 0.11a NI 0.53 0.70 0.80 0.11a 0.82 NI 0.82a NI 0.53 0.91 1.12 1.03 0.55 0.62 0.18a 0.80 NI NI 1.29 1.74 NI NI NI

NI 1.9 1.5 1.8 2.6 3.7 3.1 2.5 3.1 1.7 2.6 2.6 0.8 2.8 1.8 NI 2.4 1.1 NI 1.7 2.0 2.2 1.1 2.3 NI 2.3 NI 1.7 2.5 3.1 2.8 1.7 1.9 1.2 2.2 NI NI 3.7 5.7 NI NI NI

NI 0.73a 0.64a 0.70a 0.58 1.36a 0.87 1.22 1.19 0.63a 1.36a NI 1.05 1.18 0.95a NI 0.78a 0.14a NI 0.63 0.65a 0.67a 0.13a 0.86a 1.15a 0.98a 1.44a 0.77a 1.37 1.32a 1.03a 0.61a 0.77a 0.21a 1.28a NI NI 1.22a 1.53 1.75a NI NI

NI 2.1 1.9 2.0 1.8 3.9 2.4 3.4 3.3 1.9 3.9 NI 2.9 3.2 2.6 NI 2.2 1.2 NI 1.9 1.9 2.0 1.1 2.4 3.2 2.7 4.2 2.2 3.9 3.8 2.8 1.9 2.2 1.2 3.6 NI NI 3.4 4.6 5.7 NI NI

1.17a NI 0.87 0.61a 0.33 1.19a 1.18a 1.44 NI 0.55a 0.86 1.46a 0.33 1.02 0.83a 1.13a 0.68a NI 0.42a 0.70 0.57a 0.59a 0.12a 0.75a NI 0.85a 1.26a 0.72 0.84 1.15a 0.90a 0.54a 0.48 0.18a 1.12a 1.30a 1.03a 1.63 1.34 1.53a 0.13a 0.76a

3.2 NI 2.4 1.9 1.4 3.3 3.3 4.2 NI 1.7 2.4 4.3 1.4 2.8 2.3 3.1 2.0 NI 1.5 2.0 1.8 1.8 1.1 2.1 NI 2.4 3.5 2.1 2.3 3.2 2.5 1.7 1.6 1.2 3.1 3.7 2.8 5.1 3.8 4.6 1.1 2.2

1.32a 0.72a 0.50 0.69a 0.51 0.89 1.25 1.82 1.25 0.62a 1.41 2.24 0.03a 1.22 0.94a NI 0.77a 0.13a 0.47a 0.76 0.64a 0.66a 0.13a 0.84a 1.13a 0.96a 1.42a 0.80 0.83 1.30a 1.02a 0.61a 0.76 0.21a 0.99 1.47a 1.16a 1.54 NI 1.72a NI NI

3.8 2.1 1.7 2.0 1.7 2.4 3.5 6.2 3.5 1.9 4.1 9.4 1.0 3.4 2.6 NI 2.2 1.1 1.6 2.2 1.9 1.9 1.1 2.3 3.1 2.6 4.1 2.2 2.3 3.7 2.8 1.8 2.1 1.2 2.7 4.3 3.2 4.7 NI 5.6 NI NI

Abbreviations: HR, hazard ratio; COPD, chronic obstructive pulmonary disease; GI, gastrointestinal. Colorectal: colon and rectal cancers; gynecological: uterine and ovarian cancers; urological: bladder and renal cancers. a Substituted weight; NI, Not included because prevalence less than 0.5%, or condition closely related to primary disease.

index for breast, gynecological, or urological cancers. We found little or no difference in the performance between indices created using cancer site-specific weights and those based on more general cancer weights. 4.1. Strengths and weakness of the C3 indices The C3 indices were designed specifically to measure the impact of comorbidity among patients with cancer. As such they are likely to have a high level of content and face validity in this context; the process of selecting conditions was based both on empirical analysis and expert input, and

all (or nearly all) relevant items were likely to have been included. The likely different impact between conditions was accounted for by weighting according to their individual associations with noncancer death, and both the conditions included and the weighting of conditions were allowed to vary between sites. However, we did not include a measure of the severity within particular comorbid condition categories, with the exception of diabetes where we included two categories (diabetes with and without complications). The reasons for this were two-fold, first where this has been attempted using administrative data, there has been a low correlation between the severity identified in

592

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595

Table 3. Score distributions of C3 indices and comparison with Charlson index scores in development cohorts Parameters Distributions of C3 index scores Range Mean (SD) Median (IQR) Correlation with Charlson indexa Charlson score categories (%) 0 1 2 3þ Total C3 index score categories (%) 0 1 2 3 4 Total

Colorectal (n [ 5,376)

Breast (n [ 5,076)

Gynecological (n [ 1,223)

Liver/stomach (n [ 961)

Urological (n [ 1,460)

0.13, 14.01 0.98 (1.68) 0.00 (1.24) 0.61

0.18, 12.58 0.43 (1.22) 0.00 (0.00) 0.71

0, 13.99 0.85 (1.68) 0.00 (1.05) 0.76

0, 12.83 1.45 (2.14) 0.41 (2.19) 0.78

0.03, 14.17 1.27 (2.17) 0.00 (1.69) 0.71

71.1 15.5 5.3 8.1 100

87.3 7.5 2.1 3.1 100

78.5 14.2 2.7 4.6 100

56.9 21.4 6.6 15.1 100

70.5 15.5 6.1 7.9 100

53.9 15.7 12.3 6.4 11.7 100

78.4 8.7 5.5 2.7 4.7 100

61.7 10.1 12.3 6.8 9.2 100

40.9 19.6 12.9 7.9 18.7 100

54.8 12.3 9.7 7.9 15.4 100

Abbreviations: SD, standard deviation; IQR, interquartile range. Colorectal: colon and rectal cancers, gynecological: uterine and ovarian cancers, urological: bladder and renal cancers. a Spearman’s rank coefficient.

clinical notes and that from administrative data [33]. Second, further subdividing conditions by severity would increase the complexity of the C3 indices, and would be unlikely to substantially improve their performance. Interestingly, diabetes without complications was associated with a slight survival advantage in some cancer sites. This may be a form of selection bias where those with uncomplicated diabetes diagnosed tend to be those with better access to health care services, and generally have better health than other cancer patients. It is also possible that this finding reflects the use of metformin, a medication used to treat type II diabetes, which has been found to inhibit cancer growth [34e36]. We did not investigate the effects of possible interactions between conditions, simply because of the impractically large number of potential interactions between the conditions. The weighting procedure we used was based on the association of each condition with noncancer death. We used this outcome because both cancer-specific and all-cause mortality are largely driven by cancer-related factors particularly stage of disease, and therefore are less affected by comorbidity. Furthermore, other authors have used noncancer mortality as their primary outcome in the development of comorbidity indices [19,28]. However, relatively few patients died of noncancer causes in our cohorts, which meant that the weights we calculated may be somewhat imprecise, and additionally meant that we had to impute weights for some conditions. Reassuringly, there are both good theoretical reasons and empirical evidence to suggest that the weight given to conditions is relatively unimportant compared with the inclusion of all relevant conditions within an index [28,29,37]. In the

study presented here, for example, there was almost no difference in the two sets of indices that included the same conditions (Charlson vs. NCI, and C3 site-specific vs. C3 all-site indices). Klabunde et al. [28] found that their NCI indices performed better than Charlson, but acknowledged that they had excluded a number of conditions in their calculation of Charlson, which were included in the NCI indices. Similarly, Baldwin et al. [37] found that unweighted and weighted indices had similar predictive ability in a cohort of patients with colorectal cancer. We also tested other weighting approaches (including parameter estimates from models using all-cause mortality as outcome, and hazard ratios rather than parameter estimates). We found that our results were similar regardless of the weights we used. Furthermore, we tested simplified versions of the C3 indices that included only the 20 conditions that had a hazard ratio for noncancer death greater than 1.2 compared with those who do not have the condition, and a prevalence of 2%. The performance of these indices was not as good as the full indices that included all conditions, and only very slightly better than the Charlson index. Another key strength of the C3 indices is that they are designed to be used with administrative hospitalization data. These data are relatively quick and easy to access and usually easily available at population level. However, there are inherent weaknesses with administrative data. Data may be missing or inaccurate, it can be difficult to differentiate complications of disease from preexisting conditions, and there may be biases inherent in coding practices, for example in some jurisdictions there may be an overemphasis on recording those conditions that attract higher funding [27,38,39].

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595

593

Table 4. Comparison of concordance statistics and AICs from logistic regression models predicting deaths from all- and noncancer causes within 1 year of diagnosis using the validation cohorts by cancer site and for all sites combined All-cause mortality Cancer sites All sites (n 5 11,014)

Difference in c-statistics Colorectal (n 5 4,047)

Difference in c-statistics Breast (n 5 4,059)

Difference in c-statistics Gynecological (n 5 1,041)

Difference in c-statistics Liver/stomach (n 5 733)

Difference in c-statistics Urological (n 5 1,134)

Difference in c-statistics

Model Baselinea Charlson C3 (site specific) C3 (all-site) Charlson/C3 (all-site) Baselinea Charlson NCI index C3 (site specific) C3 (all-site) Charlson/C3 (all-site) Baselinea Charlson NCI Index C3 (site specific) C3 (all-site) Charlson/C3 (all-site) Baselinea Charlson C3 (site specific) C3 (all-site) Charlson/C3 (all-site) Baselinea Charlson C3 (site specific) C3 (all-site) Charlson/C3 (all-site) Baselinea Charlson C3 (site specific) C3 (all-site) Charlson/C3 (all-site)

c-Statistic (95% CI) 0.89 0.90 0.90 0.90 0.0012 0.82 0.83 0.83 0.84 0.84 0.0034 0.90 0.91 0.91 0.92 0.92 0.0036 0.90 0.91 0.91 0.91 0.0006 0.78 0.78 0.78 0.78 0.0003 0.85 0.86 0.86 0.86 0.0026

(0.89, 0.90) (0.89, 0.91) (0.89, 0.91) (0.89, 0.91) (0.0002, 0.0023) (0.81, 0.84) (0.82, 0.85) (0.82, 0.85) (0.82, 0.85) (0.82, 0.85) (0.0005, 0.0066) (0.88, 0.93) (0.89, 0.93) (0.89, 0.93) (0.89, 0.94) (0.89, 0.94) (0.0009, 0.0092) (0.88, 0.92) (0.89, 0.93) (0.89, 0.93) (0.89, 0.93) (0.0037, 0.0052) (0.74, 0.81) (0.75, 0.81) (0.75, 0.81) (0.75, 0.81) (0.0061, 0.0063) (0.83, 0.88) (0.83, 0.88) (0.84, 0.88) (0.84, 0.88) (0.0005, 0.0072)

Noncancer mortality AIC 6,699.5 6,598.6 6,572.3 6,564.7 3,296.5 3,242.6 3,244.3 3,216.9 3,214.9 1,011.5 981.2 978.1 984.3 982.7

c-Statistic (95% CI) 0.82 0.85 0.86 0.86 0.0048 0.75 0.79 0.79 0.80 0.80 0.0174 0.91 0.93 0.94 0.94 0.94 0.0096

644.2 619.1 626.2 623.1 824.3 824.3 824.1 823.9 939.8 937.7 928.9 931.3

0.67 0.71 0.76 0.76 0.0536 0.80 0.82 0.82 0.82 0.0005

(0.80, 0.84) (0.82, 0.87) (0.84, 0.88) (0.84, 0.88) (0.0009, 0.0086) (0.70, 0.79) (0.74, 0.83) (0.75, 0.83) (0.76, 0.84) (0.76, 0.84) (0.0019, 0.0381) (0.87, 0.94) (0.89, 0.96) (0.90, 0.96) (0.91, 0.96) (0.91, 0.96) (0.0028, 0.0309) d d d d d (0.59, 0.75) (0.62, 0.79) (0.67, 0.83) (0.68, 0.84) (0.0065, 0.1133) (0.73, 0.85) (0.76, 0.87) (0.76, 0.87) (0.76, 0.87) (0.0131, 0.0168)

AIC 1,990.3 1,920.8 1,894.8 1,886.5 911.6 884.9 884.6 869.8 868.0 363.1 335.9 332.0 337.4 334.7 d d d d d 244.3 242.2 233.4 232.9 371.1 363.1 359.7 360.7

Abbreviations: CI, confidence interval; AIC, Akaike information criterion. ‘‘d’’ Indicates not estimable because of lack of convergence in bootstrapping. AICs are from ‘‘canonical’’ application of the model (ie, the model run on the original cohort for each cancer). Colorectal: colon and rectal cancers, gynecological: uterine and ovarian cancers, urological: bladder and renal cancers. Bolded text indicates statistically significant difference between c-statistics. a Includes age, stage, and sex (where relevant).

4.1.1. Performance of the C3 indices Concurrent criterion validity is the extent to which a measure correlates with some other measure of the construct under study taken at the same time. If the other measure is itself not perfect, then the ideal correlation is in the 0.4e0.8 range, indicating a correlation between the two measures but not a perfect one [29]. The assumption is that if the new measure has a higher predictive validity, then it is likely to be more strongly correlated with the underlying construct than the original measure. If the correlation between a new and an original measure is very high (approaching 1.0), then both are measuring the same construct in the same way, and thus the new measure is unlikely to be an improvement on the original one. We found that the correlation coefficients between the C3 indices and the Charlson index were between 0.61 and 0.78. Given that the Charlson index is a validated but not perfect measure of comorbidity for the reasons discussed previously, this is

within the desired range and consistent with the C3 indices having concurrent validity. Predictive criterion validity is the extent to which the measure is able to predict future outcomes. The C3 indices were generally better at predicting both all-cause and noncancer mortality than either the Charlson or the NCI index, but the improvement over baseline models was generally modest. Why is this? The improvement is likely to be owing to the fact that the C3 indices included more relevant conditions. However, we found that the baseline models, which included only age, sex (where relevant), and stage, were generally very good at predicting both outcomes with most c-statistics around or greater than 0.80. Any measure of comorbidity improved these models’ predictive abilities, but there was little room for one to perform substantially better than another. The cancers for which the improvement was minimal (breast and gynecological cancers) are those where patients tend to be younger, and where comorbidity is less common. The

594

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595

Table 5. Weights used for calculation of C3 (all-sites) index Conditions Angina Inflammatory bowel disease Anemia Metabolic conditions Other cardiac conditions (Major) Eye conditions affecting vision Hypertension Coagulopathies and other blood disorders Cardiac arrhythmia Obesity Diabetes with complications Liver disease Previous MI Peripheral vascular disease Cerebrovascular disease COPD and asthma Cardiac valve disorders Congestive heart failure Renal disease Alcohol abuse Anxiety/behavioral disorders Connective tissue disorders Dementia Diabetes without complications Endocrine disorders Upper GI disorders Inner ear disorders Intestinal disorders Joint and spinal disorders Major psychiatric disorders Nutritional disorders Neurological disorders excluding epilepsy Osteoporosis and bone disorders Hemi/para/quadriplegia Peripheral nerve/muscular disorder Pulmonary circulation disorder Sleep disorders (Chronic) Urinary tract disorders Venous insufficiency Other malignancya Chronic hepatitis Epilepsy

Weights for C3 (all-sites) 0.51 0.52 0.59 0.61 0.62 0.63 0.72 0.75 0.77 0.83 0.88 0.92 0.93 0.98 1.09 1.09 1.10 1.26 1.38 1.08 0.57 0.51 1.35 0.03 0.77 0.11 0.54 0.11 0.69 0.79 1.16 1.06 0.49 1.03 1.20 0.95 1.41 0.12 0.70 0.17 0.39 1.04

Abbreviations: MI, myocardial infarction; COPD, chronic obstructive pulmonary disease; GI, gastrointestinal. a Excludes cancers related to primary, secondary cancers and lung, bone, liver or brain cancers.

finding that site-specific weights for the comorbidity indices did not improve the performance is a useful one. Some authors have suggested that site-specific indices are likely to perform better than more general ones [28,40], but the findings here suggest that this is unlikely to be the case, supporting the use of a single index for cancer in general. Of course, if all cancers (beyond those investigated here) were used to develop the all-sites weights, it is likely that these weights would be somewhat different. This is unlikely to be of substantial concern for two main reasons. First, the impact of specific conditions was reasonably consistent (in relative terms) across cancer sites, so their relative impact in any combined index is also likely to be similar. Second, the exact

magnitude of the weights used in the index was considerably less important than the fact that the conditions were included. Conditions were selected on the basis of their importance to cancer in general, not to these sites specifically, so it is likely that all (or most) important conditions will have been included. For these reasons, it seems reasonable to generalize the use of the indices to cancers beyond those specifically investigated here. In conclusion, the C3 indices provide a valid and useful approach to measure comorbidity using administrative data in populations of patients with cancer. Indices based on non-siteespecific weights performed almost identically to those based on site-specific weights. Although the C3 indices tended to outperform the Charlson or NCI index for all (included) sites combined, for colorectal and upper gastrointestinal cancers, there was no improvement in performance compared with the Charlson or NCI indices for breast, gynecological, or urological cancers. Our findings suggest that any of these measures of comorbidity are likely to be adequate in many situations. Acknowledgments C3 (Cancer Care and Comorbidity study) is funded by the Health Research Council of New Zealand. The authors would like to acknowledge the contribution of the C3 study team and advisory groups.

Appendix Supplementary material Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.jclinepi.2013.11.012. References [1] Coebergh JW, Janssen-Heijnen ML, Post PN, Razenberg PP. Serious co-morbidity among unselected cancer patients newly diagnosed in the southeastern part of The Netherlands in 1993-1996. J Clin Epidemiol 1999;52:1131e6. [2] Cronin-Fenton DP, Norgaard M, Jacobsen J, Garne JP, Ewertz M, Lash TL, et al. Comorbidity and survival of Danish breast cancer patients from 1995 to 2005. Br J Cancer 2007;96:1462e8. [3] Daskivich TJ, van de Poll-Franse LV, Kwan L, Sadetsky N, Stein DM, Litwin MS. From bad to worse: comorbidity severity and quality of life after treatment for early-stage prostate cancer. Prostate Cancer Prostatic Dis 2010;13:320e7. [4] Extermann M. Measurement and impact of comorbidity in older cancer patients. Crit Rev Oncol Hematol 2000;35(3):181e200. [5] Fleming ST, Pursley HG, Newman B, Pavlov D, Chen K. Comorbidity as a predictor of stage of illness for patients with breast cancer. Med Care 2005;43:132e40. [6] Gijsen R, Hoeymans N, Schellevis FG, Ruwaard D, Satariano WA, van den Bos GA. Causes and consequences of comorbidity: a review. J Clin Epidemiol 2001;54:661e74. [7] Gonzalez EC, Ferrante JM, Van Durme DJ, Pal N, Roetzheim RG. Comorbid illness and the early detection of cancer. South Med J 2001;94:913e20.

D. Sarfati et al. / Journal of Clinical Epidemiology 67 (2014) 586e595 [8] Mandelblatt JS, Bierman AS, Gold K, Zhang Y, Ng JH, Maserejian N, et al. Constructs of burden of illness in older patients with breast cancer: a comparison of measurement methods. Health Serv Res 2001;36(6 Pt 1):1085e107. [9] Sarfati D, Hill S, Blakely T, Robson B, Purdie G, Dennett E, et al. The effect of comorbidity on the use of adjuvant chemotherapy and survival from colon cancer: a retrospective cohort study. BMC Cancer 2009;9:116. [10] Satariano WA. Aging, comorbidity, and breast cancer survival: an epidemiologic view. Adv Exp Med Biol 1993;330:1e11. [11] Satariano WA, Silliman RA. Comorbidity: implications for research and practice in geriatric oncology. Crit Rev Oncol Hematol 2003; 48(2):239e48. [12] Tammemagi CM, Neslund-Dudas C, Simoff M, Kvale P. Impact of comorbidity on lung cancer survival. Int J Cancer 2003;103: 792e802. [13] Yancik R, Ershler W, Satariano W, Hazzard W, Cohen HJ, Ferrucci L. Report of the national institute on aging task force on comorbidity. J Gerontol A Biol Med Sci 2007;62(3):275e80. [14] Extermann M. Interaction between comorbidity and cancer. Cancer Control 2007;14(1):13e22. [15] Kendal WS. Dying with cancer: the influence of age, comorbidity, and cancer site. Cancer 2008;112(6):1354e62. [16] Piccirillo JF, Tierney RM, Costas I, Grove L, Spitznagel EL Jr. Prognostic importance of comorbidity in a hospital-based cancer registry. JAMA 2004;291:2441e7. [17] Read WL, Tierney RM, Page NC, Costas I, Govindan R, Spitznagel EL, et al. Differential prognostic impact of comorbidity. J Clin Oncol 2004;22:3099e103. [18] Sarfati D. Review of methods to measure comorbidity in cancer populations: no gold standard exists. J Clin Epidemiol 2012;65:924e33. [19] Charlson M, Pompei P, Ales K, Mackenzie C. A new method for classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373e83. [20] Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992;45:613e9. [21] D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson comorbidity index with administrative data bases. J Clin Epidemiol 1996;49:1429e33. [22] Kieszak SM, Flanders WD, Kosinski AS, Shipp CC, Karp H. A comparison of the Charlson comorbidity index derived from medical record data and administrative billing data. J Clin Epidemiol 1999;52: 137e42. [23] Malenka DJ, McLerran D, Roos N, Fisher ES, Wennberg JE. Using administrative data to describe case mix: a comparison with the medical record. J Clin Epidemiol 1994;47:1027e32. [24] Quan H, Li B, Couris C, Fushimi K, Graham P, Hider P, et al. Updating and validating Charlson Comorbidity Index and score for risk adjustment in hospital discharge abstracts using data from 6

[25]

[26]

[27]

[28]

[29]

[30] [31]

[32] [33]

[34]

[35]

[36]

[37]

[38] [39]

[40]

595

countries. Am J Epidemiol 2011;173:676e82.http://dx.doi.org/10. 1093/aje/kwq433. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi J-C, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005;43:1130e9. Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol 1993;46:1075e9. discussion 1081e1090. Sarfati D, Hill S, Purdie G, Dennett E, Blakely T. How well does routine hospitalisation data capture information on comorbidity in New Zealand? N Z Med J 2010;123(1310):50e61. Klabunde CN, Legler JM, Warren JL, Baldwin L-M, Schrag D. A refined comorbidity measurement algorithm for claims-based studies of breast, prostate, colorectal, and lung cancer patients. Ann Epidemiol 2007;17:584e90. Streiner D, Norman G. Health measurement scales. A practical guide to their development and use. 4th ed. Oxford, UK: Oxford University Press; 2008. American Joint Committee on Cancer. AJCC Cancer Staging Manual. 7th ed. New York, NY: Springer Verlag; 2009. Sarfati D, Gurney J, Lim B, Bagheri N, Simpson A, Koea J, et al. Identifying important comorbidity among cancer populations using administrative data: prevalence and impact on survival. Asia Pac J Clin Oncol; 2014: http://dx.doi.org/10.1111/ajco.12130. In press. Hosmer D, Lemeshow S. Applied logistic regression. 2nd ed. New York, NY: Wiley; 2000. Fleming ST, Sabatino SA, Kimmick G, Cress R, Wu X-C, TrenthamDietz A, et al. Developing a claim-based version of the ACE-27 comorbidity index: a comparison with medical record review. Med Care 2011; 49:752e60. Evans JMM, Donnelly LA, Emslie-Smith AM, Alessi DR, Morris AD. Metformin and reduced risk of cancer in diabetic patients. BMJ 2005;330:1304e5. Landman GWD, Kleefstra N, van Hateren KJJ, Groenier KH, Gans ROB, Bilo HJG. Metformin associated with lower cancer mortality in type 2 diabetes: ZODIAC-16. Diabetes Care 2010;33:322e6. Libby G, Donnelly LA, Donnan PT, Alessi DR, Morris AD, Evans JMM. New users of metformin are at low risk of incident cancer: a cohort study among people with type 2 diabetes. Diabetes Care 2009;32:1620e5. Baldwin L-M, Klabunde CN, Green P, Barlow W, Wright G. In search of the perfect comorbidity measure for use with administrative claims data: does it exist? Med Care 2006;44:745e53. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care 1998;36:8e27. Mnatzaganian G, Ryan P, Norman PE, Hiller JE. Accuracy of hospital morbidity data and the performance of comorbidity scores as predictors of mortality. J Clin Epidemiol 2012;65:107e15. Fleming ST, Pearce KA, McDavid K, Pavlov D. The development and validation of a comorbidity index for prostate cancer among Black men. J Clin Epidemiol 2003;56:1064e75.