- Email: [email protected]
WHO Performance Status and ASA Score as a Measure of Functional Status
258
Journal of Pain and Symptom Management
Vol. 49 No. 2 February 2015
Brief Report
Comparison of ECOG/WHO Performance Status and ASA Score as a Measure of Functional Status Jane Young, MBBS, MPH, PhD, FAFPHM, Tim Badgery-Parker, MBiostat, Timothy Dobbins, BMath, PhD, Mikaela Jorgensen, BAppSc (Sp Path) Hons, Peter Gibbs, MBBS, MD, FRACP, Ian Faragher, MBBS, FRACS, Ian Jones, MBBS, FRACS, and David Currow, BMed, MPH, FRACP Cancer Epidemiology and Services Research (J.Y., T.B.-P., T.D., M.J.), Sydney School of Public Health, University of Sydney, Sydney; Cancer Institute NSW (J.Y., D.C.), Sydney; Surgical Outcomes Research Centre (SOuRCe) (J.Y., T.B.-P.), Sydney Local Health District, Sydney, New South Wales; Walter and Eliza Hall Institute of Medical Research (P.G.), Parkville; Department of Colorectal Surgery (I.F.), Western Hospital, Footscray; and Department of Colorectal Surgery (I.J.), Royal Melbourne Hospital, Melbourne, Victoria, Australia
Abstract Context. The Eastern Cooperative Oncology Group/World Health Organization Performance Status (ECOG/WHO PS) is a prognostic factor. It should be used in analyzing health outcomes such as risk-adjusted hospital performance models in cancer populations. Performance status is rarely recorded in surgery, often the place where cancer is first diagnosed. Could a universally collected preoperative measure be substituted for ECOG/WHO PS? Objectives. The aim of this study was to assess whether the American Society of Anesthesiologists (ASA) score could be used as a proxy for ECOG/WHO PS in risk adjustment models predicting extended length of stay (LOS) after cancer surgery. Methods. Data were obtained from the BioGrid Colorectal Cancer Database for 2540 treatment episodes (2528 patients) at five hospitals in Victoria and Tasmania, Australia, from 2003 to 2012. Using extended LOS as the index outcome measure, a risk adjustment model was developed using patient demographic and clinical variables. The ECOG/WHO PS and ASA score were added to this model, and the relative percentage change in hospital coefficients were examined. Model fit was compared using Akaike’s information criterion (AIC) and concordance statistic (c). Results. Adding ECOG/WHO PS or ASA score to the model resulted in relative changes in the hospital coefficients of up to 27%. The ECOG/WHO PS and ASA score performed similarly, with addition of either improving the AIC from 988.2 to 976.3. Inclusion of both measures further improved AIC to 972.4.
Address correspondence to: Jane Young, MBBS, MPH, PhD, FAFPHM, Cancer Epidemiology and Services Research (CESR), Sydney School of Public Health, Level 6 North, Lifehouse (C39Z), Ó 2015 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved.
University of Sydney, Sydney, NSW 2006, Australia. E-mail: [email protected] Accepted for publication: June 11, 2014. 0885-3924/$ - see front matter http://dx.doi.org/10.1016/j.jpainsymman.2014.06.006
Vol. 49 No. 2 February 2015
Comparison of ECOG/WHO PS and ASA Scores
259
Conclusion. The ASA score can be used as a proxy for ECOG/WHO PS in risk adjustment models predicting cancer surgery. Further studies should assess its broader application for other outcomes and in other settings. J Pain Symptom Manage 2015;49:258e264. Ó 2015 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved. Key Words Risk adjustment, ECOG/WHO performance status, statistical models, ASA
Introduction Investigation of variability in patient outcomes at the hospital level is a useful tool to identify where there may be potential to improve surgical care.1e4 However, such a comparative analysis must account for differences in the characteristics of patients treated in different institutions.5e7 Risk adjustment modeling is a method that accounts for such differences by adjusting for all relevant patient factors that are associated with the outcomes of interest. Analysis of routinely collected administrative health data provides a means to conduct population-based investigations of variations in surgical outcomes. However, a major limitation of studies based on administrative health data is that important pieces of information, such as preoperative functional status, may not be included in such data collections. This is particularly relevant with cancer, as surgery is often done when cancer is diagnosed and hence the first real contact with health services, so preoperative measures are unlikely to be available. Factors that are commonly included in risk adjustment models for cancer outcomes include patient age, sex, cancer site, and stage of disease, and these are generally available in routinely collected data sets such as cancer registry records and hospital episode statistics. Functional status is an important patient factor that is likely to be associated with the types of treatment that are appropriate, as well as prognosis.8 Functional status may be recorded in clinical records, but it is generally not included in routinely collected administrative data sets. One measure of functional status is the Eastern Cooperative Oncology Group Performance Status (ECOG PS), which is a score ranging from zero (‘‘fully active’’) through three (’’capable of only limited self-care’’) to five (‘‘dead’’) and has been adopted by the World Health Organization
(WHO).9 In recognition of the potential value of information about functional status for research in populations with cancer, there is international interest in future inclusion of measures of performance, such as ECOG/WHO PS, in routinely collected datasets.10,11 Another measure of a patient’s well-being is the American Society of Anesthesiologists (ASA) score. This is a measure of physical status ranging from one for a normal healthy patient, through various levels of systemic disease, to six for a person who is brain-dead.12 As it is required to be recorded routinely by anesthetists before any surgical procedure, it is available in many data collections. Previously, Mayr et al13 demonstrated that, for individual patients who had radical cystectomy for bladder cancer, ASA score was superior to ECOG/WHO PS to predict those who would die within 90 days of surgery. However, the relative performance of these two measures for risk adjustment of hospital-level surgical outcomes, rather than individual patient outcomes, requires further investigation. The aim of this study was to assess whether ASA score could substitute for, or improve on, ECOG/WHO PS in a risk adjustment model of surgical performance for people with colorectal cancer to further develop methods for comparative assessment of outcomes based on routinely collected data. The hypothesis was that ASA score may be a useful proxy for ECOG/WHO PS, and that this could be tested in risk adjustment models.
Methods Ethics Statement Data were obtained from the BioGrid Australia Colorectal Cancer Database under ethics approval granted by the Melbourne Health Human Research Ethics Committee and the
260
Young et al.
Vol. 49 No. 2 February 2015
Table 1 Patient Characteristics and Univariate Associations With Extended Length of Stay (LOS) Extended LOS Characteristics Hospital A B C D E Mean age (yrs) Sex Male Female Past medical history Diabetes mellitus Inflammatory bowel disease Other cancer Smoking status Nonsmoker Ex-smoker Current Site Colon Rectum Both Tumor stage A B C D Admission type Elective Emergency Insurance status Public Private Location of residence Metropolitan Non-metropolitan ECOG/WHO PS 0 1 2 3 4 ASA score 1 2 3 4 5
N (%)
No
Yes
P-value
(30.9) (25.2) (9.1) (25.9) (8.9) (12.6)
704 (31.9) 551 (25.0) 182 (8.3) 590 (26.7) 179 (8.1) 66.6
40 (27.4) 44 (30.1) 18 (12.3) 33 (22.6) 11 (7.5) 71.0
0.2
1417 (55.8) 1122 (44.2)
1215 (55.1) 990 (44.9)
89 (61.0) 57 (39.0)
0.17
453 (17.8) 30 (4.1) 387 (15.2)
391 (18.2) 25 (1.2) 326 (15.2)
35 (24.5) 3 (2.1) 36 (25.0)
0.08 0.4 0.003
1380 (54.3) 713 (28.1) 300 (11.8)
1206 (57.3) 635 (30.2) 262 (12.5)
77 (54.6) 49 (34.8) 15 (10.6)
0.5
1726 (68.0) 787 (31.0) 22 (0.9)
1504 (68.3) 680 (30.9) 18 (0.8)
100 (68.5) 44 (30.1) 2 (1.4)
0.6
786 641 230 657 226 66.9
456 838 682 369
(18.0) (33.0) (26.9) (14.5)
413 737 586 330
(19.4) (34.6) (27.5) (15.5)
20 52 43 24
(14.1) (36.6) (30.3) (16.9)
<0.001
0.6
2153 (84.8) 328 (12.9)
1883 (87.3) 273 (12.7)
111 (77.6) 32 (22.4)
0.002
1938 (76.3) 597 (23.5)
1691 (76.8) 510 (23.2)
127 (87.0) 19 (13.0)
0.004
2084 (82.1) 431 (17.0)
1829 (83.6) 359 (16.4)
121 (84.0) 23 (16.0)
1.0
1518 653 264 92 12
(59.8) (25.7) (10.4) (3.6) (0.5)
1346 564 219 68 8
(61.0) (25.6) (9.9) (3.1) (0.4)
62 41 25 16 2
(42.5) (28.1) (17.1) (11.0) (1.4)
<0.001
399 1143 854 143 1
(15.7) (45.0) (33.6) (5.6) (0.04)
328 1038 729 110 1
(14.9) (47.1) (33.0) (5.0) (0.05)
14 38 69 24
(9.7) (26.2) (47.6) (16.6) 0
<0.001
ECOG/WHO PS ¼ Eastern Cooperative Oncology Group/World Health Organization Performance Status; ASA ¼ American Society of Anesthesiologists. P-values from Fisher’s exact test, except for age (t-test). Where data are missing, categories do not sum to 100%.
ethics committees for all other hospitals that contributed data.14 The ethics committees waived the requirement for patients to provide written consent for their information to be stored in the database and used for research on the condition that all data were de-identified before use for research, given that these were routinely collected data.
Data Set For colorectal cancer, five hospitals in Victoria and Tasmania, Australia, contributed information on presentation, treatment, outcomes, pathology, and follow-up of patients to the BioGrid Australia Colorectal Cancer Database. Both ECOG/WHO PS and ASA score are routinely collected in this data set.
Vol. 49 No. 2 February 2015
Comparison of ECOG/WHO PS and ASA Scores
history of inflammatory bowel disease, history of other cancer, tumor site (colon or rectum), tumor stage (Australian ClinicoPathological Stage),16 emergency or elective presentation, private health insurance status, rural or metropolitan residence, year, ECOG/WHO PS, and ASA score.
Table 2 Agreement Between ASA Score and ECOG/ WHO PS ASA Score ECOG/WHO PS 0 1 2 3 4 Total
1
2
3
4
5
Total
313 80 5 1 0 399
816 246 74 7 0 1143
352 282 152 62 5 853
37 45 33 21 7 143
0 0 0 1 0 1
1518 653 264 92 12 2539
261
Statistical Analysis The ASA score and ECOG/WHO PS were cross-tabulated and agreement between the two measures was investigated using a weighted Kappa statistic. The proportion of patients who had an extended LOS was calculated for each of the five hospitals in the data set. Univariate associations between patients’ clinical and demographic characteristics and extended LOS were examined using Fisher’s exact tests and t-tests. A base logistic regression model (without ECOG/WHO status or ASA score) was developed that included variables representing the five hospitals and patient characteristics identified using backward elimination, starting with variables that had P-value lower than 0.2 in the univariate analysis. Age, sex, and tumor stage were included, regardless of statistical significance, as these are known predictors of cancer surgical outcome. The decision to keep other variables was made based on Pvalues and on whether including them noticeably changed the coefficients for hospital performance. The ECOG/WHO PS and ASA score (as categorical variables) were added to the base model, first individually and then in combination, and the effect on hospital performance coefficients was assessed as percentage change from base model coefficients. Models were
ECOG/WHO PS ¼ Eastern Cooperative Oncology Group/World Health Organization Performance Status; ASA ¼ American Society of Anesthesiologists.
The study sample comprised all patients with colorectal cancer who were treated at one of the five participating hospitals between January 1, 1994 and April 27, 2012. Extended length of stay (LOS), defined as hospital LOS greater than 21 days (the standard for reporting in New South Wales), was chosen as the primary outcome measure as it was based on the most accurate outcome data available in the data set and was recorded for all patients. Extended LOS may be considered a proxy for postoperative complications15 and is a commonly reported surgical outcome in hospital benchmarking studies based on routinely collected administrative health data. Although postoperative mortality would have provided the criterion standard for postsurgical complications, this could not be used because of concerns about completeness of the data for out-of-hospital deaths. Furthermore, the number of deaths was too small for stable statistical modeling. Variables available for risk adjustment were patients’ age, sex, history of diabetes, history of smoking (nonsmoker, ex-smoker, or current smoker),
Table 3 Comparison of Risk Adjustment Models Parameters Model c
Base Base þ ECOG/WHO PS Base þ ASA Base þ ECOG/WHO PS þ ASA
Hospital Aa b b 0 0 0 0
Hospital B b b (SE) 0.37 0.39 0.42 0.42
(0.25) (0.25) (0.25) (0.25)
Hospital C bb
%D b 6 14 14
b b (SE) 0.61 0.55 0.66 0.60
(0.34) (0.35) (0.35) (0.35)
%D b b 11 7 2
Hospital D b b (SE) 0.29 0.21 0.36 0.27
(0.27) (0.27) (0.27) (0.27)
%D b b 27 26 6
Hospital E b b (SE) 0.32 0.38 0.39 0.41
(0.47) (0.48) (0.48) (0.49)
%D b b 20 23 28
SE ¼ standard error; ECOG/WHO PS ¼ Eastern Cooperative Oncology Group/World Health Organization Performance Status; ASA ¼ American Society of Anesthesiologists. a Hospital A is the reference (all coefficients 0). b %D b b ¼ relative percentage change in coefficient compared with base model coefficient. c Base model includes age, sex, tumor stage, tumor site, year, history of other cancer, private or public patient, elective or emergency admission, and rural or metropolitan residence.
262
Young et al.
Vol. 49 No. 2 February 2015
Table 4 Measures of Model Fit Model Base ECOG/WHO PS ASA ECOG/WHO PS þ ASA
AIC 988.2 976.3 976.3 972.4
c (95% CI)a 0.70 0.73 0.72 0.74
(0.68e0.76) (0.70e0.79) (0.70e0.78) (0.72e0.80)
Difference From Base Model c 0.025 (0.006e0.049) 0.020 (0.006e0.043) 0.037 (0.017e0.065)
AIC ¼ Akaike’s Information Criterion; c ¼ concordance statistic (area under receiver operating characteristic curve); ECOG/WHO PS ¼ Eastern Cooperative Oncology Group/World Health Organization Performance Status; ASA ¼ American Society of Anesthesiologists. a Bootstrap percentile confidence interval from 1000 replications.
compared using the Akaike’s Information Criterion (AIC) with lower values indicating better model fit. For each model, discriminant ability was assessed with concordance (c) statistics. Confidence intervals for c statistics and differences in c statistics between models were estimated using the percentile bootstrap method with 1000 replications. Calibration curves were used to assess how each model performed across the range of risk of an event (extended LOS). In these, the observed and expected numbers of events were plotted against the level of risk.
Results Of 2630 treatment episodes in the BioGrid colorectal data set, 62 that occurred before 2003 and 28 where year of admission was not recorded were excluded, leaving 2540 episodes for 2528 patients for analysis. Demographic and clinical characteristics of patients are summarized in Table 1. Across the five hospitals, the proportion of patients who experienced an extended LOS ranged from 5.3% to 9.0%. Older patients, those with a history of another cancer, those with an emergency admission, or treated in a public hospital were more likely to have extended LOS in univariate analysis. Both ASA score and ECOG/WHO PS were significantly associated with extended LOS (Table 1). There was poor agreement between ASA score and ECOG/ WHO PS (weighted Kappa ¼ 0.17, 95% confidence interval ¼ 0.15e0.19; Table 2).17 When ASA score was added to the base model, the coefficients for hospitals changed by 7% to 26% (Table 3). Addition of ECOG/ WHO PS to the base model changed the hospital coefficients by a similar amount. Measures of model fit showed that ASA score and ECOG/WHO PS made similar improvements to the base model (Table 4).
Adding both measures simultaneously to the base model led to a further increase in model fit, as shown by a reduction in the AIC (Table 3). The c statistic for the model improved significantly with the inclusion of ASA score or ECOG/WHO PS (Table 4). The improvement in adding either one or both of ASA score and ECOG/WHO PS to the base model is evident from the receiver operating characteristic curves (Fig. 1). There was no evidence of multicollinearity (variance inflation factors for all <2). The Hosmer-Lemeshow goodness-of-fit Chi-square statistic was not significant, indicating no significant lack of fit. Calibration curves of observed and predicted numbers of events by decile of predicted risk (Fig. 2) showed that the ECOG/WHO PS model had somewhat poorer prediction than the ASA model at the higher risk levels.
Fig. 1. Receiver operating characteristic (ROC) curves for the four models. ECOG ¼ Eastern Cooperative Oncology Group; ASA ¼ American Society of Anesthesiologists.
Vol. 49 No. 2 February 2015
Comparison of ECOG/WHO PS and ASA Scores
Fig. 2. Calibration curves for the four models. ECOG ¼ Eastern Cooperative Oncology Group; ASA ¼ American Society of Anesthesiologists.
Discussion Both ASA score and ECOG/WHO PS changed the estimated hospital coefficients and improved model fit when added to risk adjustment models for extended LOS. Both performed similarly, suggesting that ASA score can be used as a proxy for functional status in risk adjustment models for extended LOS after cancer surgery. As model fit was further improved by inclusion of both factors, efforts to increase the routine recording of ECOG/ WHO PS will enhance the validity of comparative assessment of hospital performance. Another widely used variable for risk adjustment is the Charlson comorbidity score, which is predictive of mortality outcomes in a wide range of populations and diseases.18,19 Recently, Dekker et al15 reported that, although the ASA and Charlson scores had similarly predictive value for postoperative morbidity and mortality in surgical patients with colorectal cancer, neither made a substantial improvement to a multivariate risk adjustment model, as the influence of comorbidity may already have been captured in other variables, such as age, in the model. Unfortunately, the Charlson score could not be calculated for this study, as the full range of comorbidities was not recorded in the BioGrid data set, so the performance of risk adjustment models that included ASA score, ECOG/WHO PS, and Charlson score
263
separately or in combination could not be compared. In the study by Mayr et al13 comparing ASA score, ECOG/WHO PS, and Charlson score for risk adjustment of 90-day mortality for men with urothelial cancer, both the ASA score and ECOG/WHO PS performed better than the Charlson score. Furthermore, this previous study found ASA score to be superior to ECOG/WHO PS, whereas there was little difference between the two in the present analysis, although it appears from the calibration curve that the ASA score performs somewhat better than ECOG/WHO PS when the event rate is higher. It is possible that the relative performance of these measures could vary between cancer types, health systems, or the specific surgical outcomes under investigation, highlighting the need to replicate this work in a wider sample of hospitals, and across different types of cancer and various outcomes. Our study focused on the outcome of extended LOS, as a proxy for postoperative complications. Mortality and unplanned readmission rates are other important surgical outcomes. Given the low event rates, these parameters could not be assessed. It is possible that the contribution of functional or health status to risk adjustment modeling may depend on the specific outcome, so this work should be repeated for other outcomes of interest or in other settings such as outpatient settings.
Conclusion This study suggests that inclusion of ASA score and ECOG/WHO PS, individually or in combination, improves risk adjustment models for extended LOS after cancer surgery. Further studies should assess its broader application for other cancer surgical outcomes and in other settings.
Disclosures and Acknowledgments Dr. Gibbs is employed by BioGrid Australia. The other authors declare that they have no competing interests. Dr. Young is supported by an Academic Leader in Cancer Epidemiology award from the Cancer Institute NSW. The authors thank BioGrid Australia for providing the data.
264
Young et al.
References 1. Australian Council on Healthcare Standards. Australasian clinical indicator report: 2001e2008. Determining the potential to improve quality of care. Sydney: Australian Council on Healthcare Standards, 2008.
Vol. 49 No. 2 February 2015
11. Young JM, Jorgensen M, Dobbins T, Solomon MJ. The quality and usefulness of the NSW Clinical Cancer Registry Minimum Dataset and Colorectal Dataset Extension for colorectal cancer services research. CESR Technical Report 1. Sydney: University of Sydney, 2012.
2. Center MM, Jemal A, Smith RA, Ward E. Worldwide variations in colorectal cancer. CA Cancer J Clin 2009;59:366e378.
12. Owens WD, Felts JA, Spitznagel EL. ASA physical status classificationsdstudy of consistency of ratings. Anesthesiology 1978;49:239e243.
3. Habib MR, Solomon MJ, Young JM, et al. Evidence-based and clinical outcome scores to facilitate audit and feedback for colorectal cancer care. Dis Colon Rectum 2009;52:616e622.
13. Mayr R, May M, Martini T, et al. Predictive capacity of four comorbidity indices estimating perioperative mortality after radical cystectomy for urothelial carcinoma of the bladder. BJU Int 2012; 110:E222eE227.
4. Malin JL, Schneider AM, Epstein JA, Ezekiel JE, Kahn KL. Results of the national initiative for cancer care quality: how can we improve the quality of cancer care in the United States? J Clin Oncol 2006;24:626e634. 5. Schrag D. Enhancing cancer registry data to promote rational health system design. J Natl Cancer Inst 2008;100:378e379. 6. Daley J, Khuri SF, Henderson W, et al. Risk adjustment of the postoperative morbidity rate for the comparative assessment of the quality of surgical care: results of the National Veterans Affairs Surgical Risk Study. J Am Coll Surg 1997;185:328e340.
14. Merriel RB, Gibbs P, O’Brien TJ, Hibbert M. BioGrid Australia facilitates collaborative medical and bioinformatics research across hospitals and medical research institutes by linking data from diverse disease and data types. Hum Mutat 2011; 32:517e525. 15. Dekker JW, Gooiker GA, van der Geest LG, et al. Use of different comorbidity scores for riskadjustment in the evaluation of quality of colorectal cancer surgery: does it matter? Eur J Surg Oncol 2012;38:1071e1078.
7. Iezzoni LI. Using risk-adjusted outcomes to assess clinical-practicedan overview of issues pertaining to risk adjustment. Ann Thorac Surg 1994; 58:1822e1826.
16. Davis NC, Evans EB, Cohen JR, Theile DE. Staging of colorectal cancer. The Australian clinicopathological staging (ACPS) system compared with Dukes’ system. Dis Colon Rectum 1984;27:707e713.
8. Extermann M, Overcash J, Lyman GH, Parr J, Balducci L. Comorbidity and functional status are independent in older cancer patients. J Clin Oncol 1998;16:1582e1587.
17. Altman D. Practical statistics for medical research. London, UK: Chapman & Hall, 1991.
9. Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol 1982;5: 649e655.
18. Charlson ME, Pompei P, Ales KL, Mackenzie CR. A new method of classifying prognostic co-morbidity in longitudinal-studiesddevelopment and validation. J Chronic Dis 1987;40: 373e383.
10. National Committee on Vital Health and Statistics. Classifying and reporting vital statistics. Available at: http://ncvhs.hhs.gov/010617rp.pdf. Accessed 21 July 2014.
19. Sundararajan V, Henderson T, Perry C, et al. New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J Clin Epidemiol 2004;57:1288e1294.
Journal of Pain and Symptom Management
Vol. 49 No. 2 February 2015
Brief Report
Comparison of ECOG/WHO Performance Status and ASA Score as a Measure of Functional Status Jane Young, MBBS, MPH, PhD, FAFPHM, Tim Badgery-Parker, MBiostat, Timothy Dobbins, BMath, PhD, Mikaela Jorgensen, BAppSc (Sp Path) Hons, Peter Gibbs, MBBS, MD, FRACP, Ian Faragher, MBBS, FRACS, Ian Jones, MBBS, FRACS, and David Currow, BMed, MPH, FRACP Cancer Epidemiology and Services Research (J.Y., T.B.-P., T.D., M.J.), Sydney School of Public Health, University of Sydney, Sydney; Cancer Institute NSW (J.Y., D.C.), Sydney; Surgical Outcomes Research Centre (SOuRCe) (J.Y., T.B.-P.), Sydney Local Health District, Sydney, New South Wales; Walter and Eliza Hall Institute of Medical Research (P.G.), Parkville; Department of Colorectal Surgery (I.F.), Western Hospital, Footscray; and Department of Colorectal Surgery (I.J.), Royal Melbourne Hospital, Melbourne, Victoria, Australia
Abstract Context. The Eastern Cooperative Oncology Group/World Health Organization Performance Status (ECOG/WHO PS) is a prognostic factor. It should be used in analyzing health outcomes such as risk-adjusted hospital performance models in cancer populations. Performance status is rarely recorded in surgery, often the place where cancer is first diagnosed. Could a universally collected preoperative measure be substituted for ECOG/WHO PS? Objectives. The aim of this study was to assess whether the American Society of Anesthesiologists (ASA) score could be used as a proxy for ECOG/WHO PS in risk adjustment models predicting extended length of stay (LOS) after cancer surgery. Methods. Data were obtained from the BioGrid Colorectal Cancer Database for 2540 treatment episodes (2528 patients) at five hospitals in Victoria and Tasmania, Australia, from 2003 to 2012. Using extended LOS as the index outcome measure, a risk adjustment model was developed using patient demographic and clinical variables. The ECOG/WHO PS and ASA score were added to this model, and the relative percentage change in hospital coefficients were examined. Model fit was compared using Akaike’s information criterion (AIC) and concordance statistic (c). Results. Adding ECOG/WHO PS or ASA score to the model resulted in relative changes in the hospital coefficients of up to 27%. The ECOG/WHO PS and ASA score performed similarly, with addition of either improving the AIC from 988.2 to 976.3. Inclusion of both measures further improved AIC to 972.4.
Address correspondence to: Jane Young, MBBS, MPH, PhD, FAFPHM, Cancer Epidemiology and Services Research (CESR), Sydney School of Public Health, Level 6 North, Lifehouse (C39Z), Ó 2015 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved.
University of Sydney, Sydney, NSW 2006, Australia. E-mail: [email protected] Accepted for publication: June 11, 2014. 0885-3924/$ - see front matter http://dx.doi.org/10.1016/j.jpainsymman.2014.06.006
Vol. 49 No. 2 February 2015
Comparison of ECOG/WHO PS and ASA Scores
259
Conclusion. The ASA score can be used as a proxy for ECOG/WHO PS in risk adjustment models predicting cancer surgery. Further studies should assess its broader application for other outcomes and in other settings. J Pain Symptom Manage 2015;49:258e264. Ó 2015 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved. Key Words Risk adjustment, ECOG/WHO performance status, statistical models, ASA
Introduction Investigation of variability in patient outcomes at the hospital level is a useful tool to identify where there may be potential to improve surgical care.1e4 However, such a comparative analysis must account for differences in the characteristics of patients treated in different institutions.5e7 Risk adjustment modeling is a method that accounts for such differences by adjusting for all relevant patient factors that are associated with the outcomes of interest. Analysis of routinely collected administrative health data provides a means to conduct population-based investigations of variations in surgical outcomes. However, a major limitation of studies based on administrative health data is that important pieces of information, such as preoperative functional status, may not be included in such data collections. This is particularly relevant with cancer, as surgery is often done when cancer is diagnosed and hence the first real contact with health services, so preoperative measures are unlikely to be available. Factors that are commonly included in risk adjustment models for cancer outcomes include patient age, sex, cancer site, and stage of disease, and these are generally available in routinely collected data sets such as cancer registry records and hospital episode statistics. Functional status is an important patient factor that is likely to be associated with the types of treatment that are appropriate, as well as prognosis.8 Functional status may be recorded in clinical records, but it is generally not included in routinely collected administrative data sets. One measure of functional status is the Eastern Cooperative Oncology Group Performance Status (ECOG PS), which is a score ranging from zero (‘‘fully active’’) through three (’’capable of only limited self-care’’) to five (‘‘dead’’) and has been adopted by the World Health Organization
(WHO).9 In recognition of the potential value of information about functional status for research in populations with cancer, there is international interest in future inclusion of measures of performance, such as ECOG/WHO PS, in routinely collected datasets.10,11 Another measure of a patient’s well-being is the American Society of Anesthesiologists (ASA) score. This is a measure of physical status ranging from one for a normal healthy patient, through various levels of systemic disease, to six for a person who is brain-dead.12 As it is required to be recorded routinely by anesthetists before any surgical procedure, it is available in many data collections. Previously, Mayr et al13 demonstrated that, for individual patients who had radical cystectomy for bladder cancer, ASA score was superior to ECOG/WHO PS to predict those who would die within 90 days of surgery. However, the relative performance of these two measures for risk adjustment of hospital-level surgical outcomes, rather than individual patient outcomes, requires further investigation. The aim of this study was to assess whether ASA score could substitute for, or improve on, ECOG/WHO PS in a risk adjustment model of surgical performance for people with colorectal cancer to further develop methods for comparative assessment of outcomes based on routinely collected data. The hypothesis was that ASA score may be a useful proxy for ECOG/WHO PS, and that this could be tested in risk adjustment models.
Methods Ethics Statement Data were obtained from the BioGrid Australia Colorectal Cancer Database under ethics approval granted by the Melbourne Health Human Research Ethics Committee and the
260
Young et al.
Vol. 49 No. 2 February 2015
Table 1 Patient Characteristics and Univariate Associations With Extended Length of Stay (LOS) Extended LOS Characteristics Hospital A B C D E Mean age (yrs) Sex Male Female Past medical history Diabetes mellitus Inflammatory bowel disease Other cancer Smoking status Nonsmoker Ex-smoker Current Site Colon Rectum Both Tumor stage A B C D Admission type Elective Emergency Insurance status Public Private Location of residence Metropolitan Non-metropolitan ECOG/WHO PS 0 1 2 3 4 ASA score 1 2 3 4 5
N (%)
No
Yes
P-value
(30.9) (25.2) (9.1) (25.9) (8.9) (12.6)
704 (31.9) 551 (25.0) 182 (8.3) 590 (26.7) 179 (8.1) 66.6
40 (27.4) 44 (30.1) 18 (12.3) 33 (22.6) 11 (7.5) 71.0
0.2
1417 (55.8) 1122 (44.2)
1215 (55.1) 990 (44.9)
89 (61.0) 57 (39.0)
0.17
453 (17.8) 30 (4.1) 387 (15.2)
391 (18.2) 25 (1.2) 326 (15.2)
35 (24.5) 3 (2.1) 36 (25.0)
0.08 0.4 0.003
1380 (54.3) 713 (28.1) 300 (11.8)
1206 (57.3) 635 (30.2) 262 (12.5)
77 (54.6) 49 (34.8) 15 (10.6)
0.5
1726 (68.0) 787 (31.0) 22 (0.9)
1504 (68.3) 680 (30.9) 18 (0.8)
100 (68.5) 44 (30.1) 2 (1.4)
0.6
786 641 230 657 226 66.9
456 838 682 369
(18.0) (33.0) (26.9) (14.5)
413 737 586 330
(19.4) (34.6) (27.5) (15.5)
20 52 43 24
(14.1) (36.6) (30.3) (16.9)
<0.001
0.6
2153 (84.8) 328 (12.9)
1883 (87.3) 273 (12.7)
111 (77.6) 32 (22.4)
0.002
1938 (76.3) 597 (23.5)
1691 (76.8) 510 (23.2)
127 (87.0) 19 (13.0)
0.004
2084 (82.1) 431 (17.0)
1829 (83.6) 359 (16.4)
121 (84.0) 23 (16.0)
1.0
1518 653 264 92 12
(59.8) (25.7) (10.4) (3.6) (0.5)
1346 564 219 68 8
(61.0) (25.6) (9.9) (3.1) (0.4)
62 41 25 16 2
(42.5) (28.1) (17.1) (11.0) (1.4)
<0.001
399 1143 854 143 1
(15.7) (45.0) (33.6) (5.6) (0.04)
328 1038 729 110 1
(14.9) (47.1) (33.0) (5.0) (0.05)
14 38 69 24
(9.7) (26.2) (47.6) (16.6) 0
<0.001
ECOG/WHO PS ¼ Eastern Cooperative Oncology Group/World Health Organization Performance Status; ASA ¼ American Society of Anesthesiologists. P-values from Fisher’s exact test, except for age (t-test). Where data are missing, categories do not sum to 100%.
ethics committees for all other hospitals that contributed data.14 The ethics committees waived the requirement for patients to provide written consent for their information to be stored in the database and used for research on the condition that all data were de-identified before use for research, given that these were routinely collected data.
Data Set For colorectal cancer, five hospitals in Victoria and Tasmania, Australia, contributed information on presentation, treatment, outcomes, pathology, and follow-up of patients to the BioGrid Australia Colorectal Cancer Database. Both ECOG/WHO PS and ASA score are routinely collected in this data set.
Vol. 49 No. 2 February 2015
Comparison of ECOG/WHO PS and ASA Scores
history of inflammatory bowel disease, history of other cancer, tumor site (colon or rectum), tumor stage (Australian ClinicoPathological Stage),16 emergency or elective presentation, private health insurance status, rural or metropolitan residence, year, ECOG/WHO PS, and ASA score.
Table 2 Agreement Between ASA Score and ECOG/ WHO PS ASA Score ECOG/WHO PS 0 1 2 3 4 Total
1
2
3
4
5
Total
313 80 5 1 0 399
816 246 74 7 0 1143
352 282 152 62 5 853
37 45 33 21 7 143
0 0 0 1 0 1
1518 653 264 92 12 2539
261
Statistical Analysis The ASA score and ECOG/WHO PS were cross-tabulated and agreement between the two measures was investigated using a weighted Kappa statistic. The proportion of patients who had an extended LOS was calculated for each of the five hospitals in the data set. Univariate associations between patients’ clinical and demographic characteristics and extended LOS were examined using Fisher’s exact tests and t-tests. A base logistic regression model (without ECOG/WHO status or ASA score) was developed that included variables representing the five hospitals and patient characteristics identified using backward elimination, starting with variables that had P-value lower than 0.2 in the univariate analysis. Age, sex, and tumor stage were included, regardless of statistical significance, as these are known predictors of cancer surgical outcome. The decision to keep other variables was made based on Pvalues and on whether including them noticeably changed the coefficients for hospital performance. The ECOG/WHO PS and ASA score (as categorical variables) were added to the base model, first individually and then in combination, and the effect on hospital performance coefficients was assessed as percentage change from base model coefficients. Models were
ECOG/WHO PS ¼ Eastern Cooperative Oncology Group/World Health Organization Performance Status; ASA ¼ American Society of Anesthesiologists.
The study sample comprised all patients with colorectal cancer who were treated at one of the five participating hospitals between January 1, 1994 and April 27, 2012. Extended length of stay (LOS), defined as hospital LOS greater than 21 days (the standard for reporting in New South Wales), was chosen as the primary outcome measure as it was based on the most accurate outcome data available in the data set and was recorded for all patients. Extended LOS may be considered a proxy for postoperative complications15 and is a commonly reported surgical outcome in hospital benchmarking studies based on routinely collected administrative health data. Although postoperative mortality would have provided the criterion standard for postsurgical complications, this could not be used because of concerns about completeness of the data for out-of-hospital deaths. Furthermore, the number of deaths was too small for stable statistical modeling. Variables available for risk adjustment were patients’ age, sex, history of diabetes, history of smoking (nonsmoker, ex-smoker, or current smoker),
Table 3 Comparison of Risk Adjustment Models Parameters Model c
Base Base þ ECOG/WHO PS Base þ ASA Base þ ECOG/WHO PS þ ASA
Hospital Aa b b 0 0 0 0
Hospital B b b (SE) 0.37 0.39 0.42 0.42
(0.25) (0.25) (0.25) (0.25)
Hospital C bb
%D b 6 14 14
b b (SE) 0.61 0.55 0.66 0.60
(0.34) (0.35) (0.35) (0.35)
%D b b 11 7 2
Hospital D b b (SE) 0.29 0.21 0.36 0.27
(0.27) (0.27) (0.27) (0.27)
%D b b 27 26 6
Hospital E b b (SE) 0.32 0.38 0.39 0.41
(0.47) (0.48) (0.48) (0.49)
%D b b 20 23 28
SE ¼ standard error; ECOG/WHO PS ¼ Eastern Cooperative Oncology Group/World Health Organization Performance Status; ASA ¼ American Society of Anesthesiologists. a Hospital A is the reference (all coefficients 0). b %D b b ¼ relative percentage change in coefficient compared with base model coefficient. c Base model includes age, sex, tumor stage, tumor site, year, history of other cancer, private or public patient, elective or emergency admission, and rural or metropolitan residence.
262
Young et al.
Vol. 49 No. 2 February 2015
Table 4 Measures of Model Fit Model Base ECOG/WHO PS ASA ECOG/WHO PS þ ASA
AIC 988.2 976.3 976.3 972.4
c (95% CI)a 0.70 0.73 0.72 0.74
(0.68e0.76) (0.70e0.79) (0.70e0.78) (0.72e0.80)
Difference From Base Model c 0.025 (0.006e0.049) 0.020 (0.006e0.043) 0.037 (0.017e0.065)
AIC ¼ Akaike’s Information Criterion; c ¼ concordance statistic (area under receiver operating characteristic curve); ECOG/WHO PS ¼ Eastern Cooperative Oncology Group/World Health Organization Performance Status; ASA ¼ American Society of Anesthesiologists. a Bootstrap percentile confidence interval from 1000 replications.
compared using the Akaike’s Information Criterion (AIC) with lower values indicating better model fit. For each model, discriminant ability was assessed with concordance (c) statistics. Confidence intervals for c statistics and differences in c statistics between models were estimated using the percentile bootstrap method with 1000 replications. Calibration curves were used to assess how each model performed across the range of risk of an event (extended LOS). In these, the observed and expected numbers of events were plotted against the level of risk.
Results Of 2630 treatment episodes in the BioGrid colorectal data set, 62 that occurred before 2003 and 28 where year of admission was not recorded were excluded, leaving 2540 episodes for 2528 patients for analysis. Demographic and clinical characteristics of patients are summarized in Table 1. Across the five hospitals, the proportion of patients who experienced an extended LOS ranged from 5.3% to 9.0%. Older patients, those with a history of another cancer, those with an emergency admission, or treated in a public hospital were more likely to have extended LOS in univariate analysis. Both ASA score and ECOG/WHO PS were significantly associated with extended LOS (Table 1). There was poor agreement between ASA score and ECOG/ WHO PS (weighted Kappa ¼ 0.17, 95% confidence interval ¼ 0.15e0.19; Table 2).17 When ASA score was added to the base model, the coefficients for hospitals changed by 7% to 26% (Table 3). Addition of ECOG/ WHO PS to the base model changed the hospital coefficients by a similar amount. Measures of model fit showed that ASA score and ECOG/WHO PS made similar improvements to the base model (Table 4).
Adding both measures simultaneously to the base model led to a further increase in model fit, as shown by a reduction in the AIC (Table 3). The c statistic for the model improved significantly with the inclusion of ASA score or ECOG/WHO PS (Table 4). The improvement in adding either one or both of ASA score and ECOG/WHO PS to the base model is evident from the receiver operating characteristic curves (Fig. 1). There was no evidence of multicollinearity (variance inflation factors for all <2). The Hosmer-Lemeshow goodness-of-fit Chi-square statistic was not significant, indicating no significant lack of fit. Calibration curves of observed and predicted numbers of events by decile of predicted risk (Fig. 2) showed that the ECOG/WHO PS model had somewhat poorer prediction than the ASA model at the higher risk levels.
Fig. 1. Receiver operating characteristic (ROC) curves for the four models. ECOG ¼ Eastern Cooperative Oncology Group; ASA ¼ American Society of Anesthesiologists.
Vol. 49 No. 2 February 2015
Comparison of ECOG/WHO PS and ASA Scores
Fig. 2. Calibration curves for the four models. ECOG ¼ Eastern Cooperative Oncology Group; ASA ¼ American Society of Anesthesiologists.
Discussion Both ASA score and ECOG/WHO PS changed the estimated hospital coefficients and improved model fit when added to risk adjustment models for extended LOS. Both performed similarly, suggesting that ASA score can be used as a proxy for functional status in risk adjustment models for extended LOS after cancer surgery. As model fit was further improved by inclusion of both factors, efforts to increase the routine recording of ECOG/ WHO PS will enhance the validity of comparative assessment of hospital performance. Another widely used variable for risk adjustment is the Charlson comorbidity score, which is predictive of mortality outcomes in a wide range of populations and diseases.18,19 Recently, Dekker et al15 reported that, although the ASA and Charlson scores had similarly predictive value for postoperative morbidity and mortality in surgical patients with colorectal cancer, neither made a substantial improvement to a multivariate risk adjustment model, as the influence of comorbidity may already have been captured in other variables, such as age, in the model. Unfortunately, the Charlson score could not be calculated for this study, as the full range of comorbidities was not recorded in the BioGrid data set, so the performance of risk adjustment models that included ASA score, ECOG/WHO PS, and Charlson score
263
separately or in combination could not be compared. In the study by Mayr et al13 comparing ASA score, ECOG/WHO PS, and Charlson score for risk adjustment of 90-day mortality for men with urothelial cancer, both the ASA score and ECOG/WHO PS performed better than the Charlson score. Furthermore, this previous study found ASA score to be superior to ECOG/WHO PS, whereas there was little difference between the two in the present analysis, although it appears from the calibration curve that the ASA score performs somewhat better than ECOG/WHO PS when the event rate is higher. It is possible that the relative performance of these measures could vary between cancer types, health systems, or the specific surgical outcomes under investigation, highlighting the need to replicate this work in a wider sample of hospitals, and across different types of cancer and various outcomes. Our study focused on the outcome of extended LOS, as a proxy for postoperative complications. Mortality and unplanned readmission rates are other important surgical outcomes. Given the low event rates, these parameters could not be assessed. It is possible that the contribution of functional or health status to risk adjustment modeling may depend on the specific outcome, so this work should be repeated for other outcomes of interest or in other settings such as outpatient settings.
Conclusion This study suggests that inclusion of ASA score and ECOG/WHO PS, individually or in combination, improves risk adjustment models for extended LOS after cancer surgery. Further studies should assess its broader application for other cancer surgical outcomes and in other settings.
Disclosures and Acknowledgments Dr. Gibbs is employed by BioGrid Australia. The other authors declare that they have no competing interests. Dr. Young is supported by an Academic Leader in Cancer Epidemiology award from the Cancer Institute NSW. The authors thank BioGrid Australia for providing the data.
264
Young et al.
References 1. Australian Council on Healthcare Standards. Australasian clinical indicator report: 2001e2008. Determining the potential to improve quality of care. Sydney: Australian Council on Healthcare Standards, 2008.
Vol. 49 No. 2 February 2015
11. Young JM, Jorgensen M, Dobbins T, Solomon MJ. The quality and usefulness of the NSW Clinical Cancer Registry Minimum Dataset and Colorectal Dataset Extension for colorectal cancer services research. CESR Technical Report 1. Sydney: University of Sydney, 2012.
2. Center MM, Jemal A, Smith RA, Ward E. Worldwide variations in colorectal cancer. CA Cancer J Clin 2009;59:366e378.
12. Owens WD, Felts JA, Spitznagel EL. ASA physical status classificationsdstudy of consistency of ratings. Anesthesiology 1978;49:239e243.
3. Habib MR, Solomon MJ, Young JM, et al. Evidence-based and clinical outcome scores to facilitate audit and feedback for colorectal cancer care. Dis Colon Rectum 2009;52:616e622.
13. Mayr R, May M, Martini T, et al. Predictive capacity of four comorbidity indices estimating perioperative mortality after radical cystectomy for urothelial carcinoma of the bladder. BJU Int 2012; 110:E222eE227.
4. Malin JL, Schneider AM, Epstein JA, Ezekiel JE, Kahn KL. Results of the national initiative for cancer care quality: how can we improve the quality of cancer care in the United States? J Clin Oncol 2006;24:626e634. 5. Schrag D. Enhancing cancer registry data to promote rational health system design. J Natl Cancer Inst 2008;100:378e379. 6. Daley J, Khuri SF, Henderson W, et al. Risk adjustment of the postoperative morbidity rate for the comparative assessment of the quality of surgical care: results of the National Veterans Affairs Surgical Risk Study. J Am Coll Surg 1997;185:328e340.
14. Merriel RB, Gibbs P, O’Brien TJ, Hibbert M. BioGrid Australia facilitates collaborative medical and bioinformatics research across hospitals and medical research institutes by linking data from diverse disease and data types. Hum Mutat 2011; 32:517e525. 15. Dekker JW, Gooiker GA, van der Geest LG, et al. Use of different comorbidity scores for riskadjustment in the evaluation of quality of colorectal cancer surgery: does it matter? Eur J Surg Oncol 2012;38:1071e1078.
7. Iezzoni LI. Using risk-adjusted outcomes to assess clinical-practicedan overview of issues pertaining to risk adjustment. Ann Thorac Surg 1994; 58:1822e1826.
16. Davis NC, Evans EB, Cohen JR, Theile DE. Staging of colorectal cancer. The Australian clinicopathological staging (ACPS) system compared with Dukes’ system. Dis Colon Rectum 1984;27:707e713.
8. Extermann M, Overcash J, Lyman GH, Parr J, Balducci L. Comorbidity and functional status are independent in older cancer patients. J Clin Oncol 1998;16:1582e1587.
17. Altman D. Practical statistics for medical research. London, UK: Chapman & Hall, 1991.
9. Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol 1982;5: 649e655.
18. Charlson ME, Pompei P, Ales KL, Mackenzie CR. A new method of classifying prognostic co-morbidity in longitudinal-studiesddevelopment and validation. J Chronic Dis 1987;40: 373e383.
10. National Committee on Vital Health and Statistics. Classifying and reporting vital statistics. Available at: http://ncvhs.hhs.gov/010617rp.pdf. Accessed 21 July 2014.
19. Sundararajan V, Henderson T, Perry C, et al. New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J Clin Epidemiol 2004;57:1288e1294.