- Email: [email protected]
Predicting Survival for Patients With Metastatic Disease
Journal Pre-proof Predicting Survival for Patients with Metastatic Disease Kathryn RK. Benson, BA, Sonya Aggarwal, BA, Justin N. Carter, BA, Rie von Eyben, MS, Pooja Pradhan, BS, Nicolas D. Prionas, MD PhD, Justin L. Bui, BS, Scott G. Soltys, MD, Steven Hancock, MD, Michael F. Gensheimer, MD, Albert C. Koong, MD PhD, Daniel T. Chang, MD PII:
S0360-3016(19)33949-5
DOI:
https://doi.org/10.1016/j.ijrobp.2019.10.032
Reference:
ROB 26007
To appear in:
International Journal of Radiation Oncology • Biology • Physics
Received Date: 2 July 2019 Revised Date:
10 September 2019
Accepted Date: 12 October 2019
Please cite this article as: Benson KR, Aggarwal S, Carter JN, von Eyben R, Pradhan P, Prionas ND, Bui JL, Soltys SG, Hancock S, Gensheimer MF, Koong AC, Chang DT, Predicting Survival for Patients with Metastatic Disease, International Journal of Radiation Oncology • Biology • Physics (2019), doi: https://doi.org/10.1016/j.ijrobp.2019.10.032. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Elsevier Inc. All rights reserved.
Predicting Survival for Patients with Metastatic Disease Running title: Predicting Patient Survival Authors: Kathryn RK Benson BA1, Sonya Aggarwal BA1, Justin N. Carter BA1, Rie von Eyben MS1, Pooja Pradhan BS1, Nicolas D. Prionas MD PhD1, Justin L. Bui BS1, Scott G. Soltys MD1, Steven Hancock MD1, Michael F. Gensheimer MD1, Albert C. Koong MD PhD2, Daniel T. Chang MD1 1 2
Radiation Oncology Department, Stanford Cancer Institute Radiation Oncology Department, MD Anderson Cancer Center
Corresponding Author information: Daniel T Chang, Department of Radiation Oncology Stanford University, 875 Blake Wilbur Dr., Stanford, CA 94305-5847. Email: [email protected]; Phone: 650-724-3547; Fax: 650-725-8231
Author responsible for statistical analyses: Rie von Eyben, Department of Radiation Oncology Stanford University, 875 Blake Wilbur Dr., Stanford, CA 94305-5847. Email: [email protected]; Phone: 650-723-0914; Fax: 650-725-8231
Conflict of interests: The authors have no conflicts of interest to disclose.
Funding sources: None.
Acknowledgements: Billy Loo MD/PhD, Clement Ho MD, Carol Marquez MD, David Shultz MD/PhD, Elizabeth Kidd MD, Iris Gibbs MD, Jennifer Shah MD, Lynn Million MD, Mark Buyyounouski MD/MS, Maximilian Diehn MD/PhD, Patrick Swift MD, Quynh-Thu Le MD, Richard Hoppe MD, Sarah Donaldson MD, Susan Knox MD/PhD, Steven Hancock MD, Wendy Hara MD
Predicting Survival for Patients with Metastatic Disease Running title: Predicting Patient Survival Abstract
Purpose This prospective study aimed to determine the accuracy of radiation oncologists in predicting the survival of patients with metastatic disease receiving radiotherapy and to understand factors associated with their accuracy. Methods and Materials This single-institution study surveyed 22 attending radiation oncologists to estimate patient survival. Survival predictions were defined as accurate if the observed survival (OS) was within the correct survival prediction category (0-6 months, >6-12 months, >12-24 months, and >24 months). The physicians made survival estimates for each course of radiation, yielding 877 analyzable predictions for 689 unique patients. Data analysis included Stuart’s Tau C, logistic regression models, ordinal logistic regression models, and stepwise selection to examine variable interactions. Results Of the 877 radiation oncologists’ predictions, 39.7% were accurate, 26.5% underestimations, and 33.9% overestimations. Stuart’s Tau C showed low correlation between OS and survival estimates (0.3499), consistent with the inaccuracy reported in literature. However, results showed less systematic over-prediction than reported in the literature. Karnofsky performance status (KPS) was the most significant predictor of accuracy with greater accuracy for patients with shorter OS. Estimates were also more accurate for patients with lower KPS. Accuracy by
1
patient age varied by primary site and race. Physician years of experience did not correlate with accuracy. Conclusions The sampled radiation oncologists have relatively low accuracy in predicting patient survival. Future investigation should explore how survival estimates influence treatment decisions and how to improve survival prediction accuracy.
2
Introduction End of life care is estimated to comprise 10%-12% of the United States healthcare budget and 27%-30% of the Medicare budget.1,2 Of the total expenditure on end of life care, approximately 13% was spent on care in the final year of life.3 Estimates of the percentage of healthcare costs within the last year of life that occur in the final month range from 33%-78%.4,5 Care for cancer patients represents a significant portion of these costs. Using Medicaid administrative data, a 2015 study estimated that patients with cancer spend about $10,000 more in the final 4 months of life than non-cancer patients.6
Part of end-of-life cancer care involves palliative radiation therapy (PRT). SEER-Medicare database analysis of patients with stage IV cancer showed that 41% received PRT, generally ranging from 1-15 fractions.7 Of the SEER-Medicare database of patients receiving PRT, 78% received a single course, 17% two courses, and 5% three or greater courses.7 According to a 2017 systematic review, 5-10% of patients who died from cancer received PRT in their final month of life.8 For patients who received PRT for metastatic bony disease, treatment with single fraction regimens are associated with higher retreatment rates of 23-25% compared with that of multi-fraction regimens of 7-10%.9,10 However, growing evidence suggests that for most histologies, there is no clinical advantage to greater fractions.9–15 Despite the higher retreatment rates, single fraction regimens are still more cost-effective and provide equal palliation for patients with metastatic cancer to the bone .9,13
The decision to offer shorter versus longer fractionation schemes is often based empirically on the perceived life expectancy of the patient. Thus, patients with poor estimated prognosis may
3
preferentially receive shorter courses of radiation because of their lower likelihood of needing retreatment.13 Physicians’ ability to accurately predict survival can therefore impact the type of treatment given to terminally ill patients. Thus, there may be an opportunity to optimize the utilization of palliative radiotherapy based on patient prognosis.
The literature has shown that physicians generally overestimate survival among patients with terminal cancers.16–18 The limited literature on radiation oncologists reports that they are accurate between 10-60% of the time, the large range resulting from the heterogeneity in defining accuracy, and generally overestimate patient survival.19–23 The aim of this study is to determine the accuracy of radiation oncologists at XXX in predicting survival of patients with metastatic disease treated with radiotherapy and to understand the factors that correlate with accuracy. With a larger sample size of radiation oncologists and an analysis of factors influencing under, accurate, and overestimates, this study has important implications for how radiation oncologists make treatment decisions for patients with incurable disease.
Methods and Materials In this IRB-approved prospective study, we identified patients who had metastatic cancer and were receiving conventionally fractionated radiotherapy (CFRT) or stereotactic ablative radiotherapy (SABR) from March 2015 to November 2016 in the Department of Radiation Oncology at XXX. For each palliative treatment course, within the week of starting radiation treatment, the treating attending physicians and each member of that physician’s care team (the resident physician, nurse coordinator, and radiation therapy technologist) were given a survey that asked them to 1) predict the life expectancy for this patient, 2) provide the main reasons for
4
this prediction, and 3) identify the primary reason for the chosen fractionation schedule (Appendix A). Surveys were administered for each course of radiation. Therefore, patients being re-treated with radiotherapy would have more than one round of surveys completed about their predicted survival. The clinical treatment characteristics and details were recorded, and the patients were followed for survival using electronic medical records (EMR) and public record obituaries.
Survival was defined as the time from the date of the consultation until date of death. If death was not observed then the patient was censored at the date of last follow up. Only patients who died or lived longer than 24 months were included in the analyses. Stuart’s Tau C was used to assess the correlation between the predicted and observed survival times. Survival predictions were defined as accurate if the patient survival was within the survival prediction category (0-6 months, >6-12 months, >12-24 months, and >24 months). The binary outcome of accurate or inaccurate predictions were analyzed using logistic regression. The ordinal outcome of underestimated prediction, accurate prediction, or overestimated prediction was analyzed using ordinal logistic regression. A stepwise selections algorithm was used to select the predictors to be included in the models with a criterion of 0.1 for entry into the model and 0.05 for exiting the model. To assess if the observations were more highly correlated within physician, an ordinal logistic regression model was fit with stratification by physician. All analyses were two-sided tests with an alpha level of 0.05; and all analyses were performed using SAS v 9.4 (SAS Institute Inc, Cary, NC). Predictors included in the model the following: 1) Primary site, 2) Age at time of receiving radiation, 3) Race – White, Black, Asian, Hispanic, other. 4) Insurance Status – private, public, VA, none, 5) Marital status – married, single, widowed, divorced, missing, 6)
5
Gender, 7) KPS - ≤50, 60-70, ≥80, 8) Fractions prescribed – SABR (1-5 fractions), single fraction, 2-5 fractions, >5 fractions, 9) Experience of attending physician defined as the number or years since graduating residency - <10 years, 10-20 years, >20 years.
Results Prediction Accuracy Participants in the study included 22 attending physicians out of a total of 23 faculty (96%) within the Department of Radiation Oncology at XXX during the years of data collection for this study. Of the physicians sampled, 10 were women, 12 were men, 9 had less than 10 years of experience, 6 had 10 to 20 years of experience, and 7 had greater than 20 years of experience, with experience defined as the years in practice from residency graduation to study initiation time point.
In total, 1190 patient survival estimates were collected in this study. Out of these, 877 predictions were included in the analysis with 52 excluded because of lack of follow-up data (4.4%), 70 excluded because they were alive but not yet past 24 months (5.8%), and 191 (16.0%) excluded because of a missing physician prediction. The 877 physician survival estimates represented 689 unique patients, with patient characteristics shown in Table 1. The overall physician accuracy was 39.7% with 26.5% underestimates and 33.9% overestimates. This data yielded a Stuart’s Tau C value of 0.3499, indicating low correlation between the predictions and actual patient survival. Table 2 shows physician underestimation, accuracy, and overestimation by disease and patient characteristics.
6
Physician accuracy decreased with increasing KPS, with accuracy rates of 68% for KPS of 4050, 45% for KPS of 60-70, and 34% for KPS 80-100 (p<0.0001). By physician experience, accuracy was lowest in those who had 10 to 20 years of experience (33%) compared to those who had <10 years (45%) and >20 years (41%) experience. Physicians were more accurate in survival predictions of patients with shorter OS, 50% accurate for OS 0-6 months, 44% for >6-12 months OS, 33% for >12-24 months OS, and 25% for >24 months OS (p<0.0001). Physician accuracy was lowest for patients prescribed SABR (31%) compared to CFRT >5 fractions (42%), CFRT 2-5 fractions (57%), and CFRT 1 fraction (62%), which was consistent with the inverse relationship between OS and accuracy. SABR had the longest survival median (13 months), followed by CFRT >5 (9 months), 2-5 (5 months), and 1 (2.5 months). Although the accuracy for SABR patients was only 31%, physicians underestimated OS 42% of the time and overestimated only 27% for this cohort.
Predictors of Accuracy On univariable analysis, only KPS correlated with physician accuracy (p<0.0001), while age, gender, race, insurance coverage, primary disease site, marital status, fractions prescribed, and experience of attending did not. On multivariable analysis including all predictors as a main effect, KPS (p<0.0001) and marital status (p=0.0428) were correlated with prediction accuracy (table 3).
Physician Prediction Outcomes – Underestimate, Accurate and Overestimate Physician prediction outcomes were categorized as underestimates, accurate, and overestimates. Table 4 shows the results of the single predictor analysis, with KPS (p=0.0003), fractions
7
prescribed (p=0.0350), and experience of attending (p=0.0296) correlating with prediction outcome. On multivariable analysis including main effects only, KPS (p=0.0003) was the only factor that correlated with prediction outcome.
A stepwise selection was used to choose a final model and the algorithm was allowed to select among all variables as main effects and the two-way interactions between potentially clinically relevant factors: race, KPS, gender, age, fractions prescribed, experience of attending, and primary site. The model selected (Table 4) includes the interactions between age and primary site (0.0200), age and race (0.0005) as well as the main effects of age (0.1071), primary site (0.0063), race (0.0010) and KPS (0.0005).
For most primary sites, accuracy remained similar regardless of patient age, with the exception being lung and genitourinary (GU) where accuracy exhibited roughly a bell curve, peaking around 60-70 years of age for GU and peaking around 80 for lung. GU and lung exhibited highest overestimate rate among older patients and highest underestimate rate among younger patients. For patients with breast, GYN, lymphoma, and sarcoma primary sites, accuracy slightly increased with age, overestimate rate increased with age, and underestimate rate decreased with age. GI and H&N cancers exhibited no age-associated prediction trends (Figure 1A). Regarding race/ethnicity and age, White, Asian, Hispanic, and Other patients, underestimate rate decreased with age while accuracy and overestimate rates increased with age. However, for Black patients, the opposite trend was observed (Figure 1B).
8
Finally, a sensitivity analysis was performed in order to test if the observations within each physician were correlated. The final model using the stepwise selection algorithm was fit with stratification by physician. There were only very minor differences in the p-values between the model with and without stratification indicating that the observations within physician are correlated enough to warrant a model without stratification (data not shown).
Physician-Reported Factors Influencing Prediction On each survey, the physicians ranked the top three factors informing their survival prediction. The four most common primary factors were type of cancer 289 (33%), KPS 265 (30%), tumor burden 173 (20%), pace of disease 130 (15%). The group of physicians who reported type of cancer as the primary factor influencing their prediction had the lowest prediction accuracy (34%), followed by KPS (42%), tumor burden (42%), and pace of disease (45%). There was no statistically significant correlation with these factors and accuracy (p=0.0914).
Discussion The results showed that physician accuracy was only 40%. In addition, in 27%, physicians underestimated survival while in 34%, physicians overestimated survival. Directly comparing physician accuracy across studies in the literature is difficult because of the heterogeneity in how estimates are made and how accuracy is defined. In our study, physicians were asked to predict survival within 3-6 month time bins, and accuracy was defined as predicting the time bin (0-6 months, >6-12 months, >12-24 months, and >24 months) containing the actual survival. Whereas some studies have similarly had physicians estimate the correct survival time bin, other studies
9
have asked physicians to predict the precise survival. The wide range in accuracy reported in the literature, 10-60%, is likely because of the heterogeneity in defining accuracy.19–23 Gripp et al had radiation oncologists, including residents, make estimates in the binned categories of <1 month, 1-6 months, and >6 months and found 61% accuracy.23 Hartsell et al and Fairchild et al both reported accuracy as the radiation oncologists correctly predicting survival within 1 month of actual survival, and found 10% and 28.7% accuracy respectively.20,21
One comparable measure between many notable studies is the ratio of the percentage of overestimates and underestimates. Hartsell et al found a 2.10 ratio of overestimation to underestimation percentages among radiation oncologists in the palliative care setting20, while Christakis et al found a 3.71 ratio among physicians referring patients to hospice18, and Glare et al found an average of a 2.25 ratio in a systematic review of physician predictions for patients with terminal cancers.16 Our study found 33.9% of physicians overestimated and 26.5% underestimated, which was a much lower 1.28 ratio.
Our analysis showed that physician accuracy was higher for patients with shorter survival (Table 2), a finding called the “horizon effect,” a meteorology concept describing the observation that predicting short term events is more accurate than predicting long term events24. The horizon effect has been observed in some studies though not universally corraborated.17,25,26 Similarly, the physicians were more accurate at predicting survival of patients with low KPS (40-50), consistent with Lambden et al.27 The analysis further showed accuracy by patient age varied by primary site and race/ethnicity. Physicians had less accuracy and greater underestimates for older, Black patients than younger, Black patients. These trends were opposite of those seen with
10
patients in the White, Asian, Hispanic, and Other categories. Given the known disparities in patterns of care and access to care for underserved populations, further investigation is needed to better understand the reasons for these differences in survival predictions.28-30 Contrary to our initial hypothesis, there was no association between physician experience and prediction accuracy, confirming findings in the literature which similarly showed the lack of correlation of experience and accuracy.19,23,31–33
Given the high cost of end of life care, improving accuracy to allow for more tailored therapy based on prognosis is increasingly important1-2. Given the wide variation of available fractionation schedules, customization of palliative radiation is a major area of opportunity to maximize value of care. Multiple prospective trials have shown that single-fraction radiation treatments are equally effective at relieving pain in patients with painful bony metastases for most histologies but are associated with higher rates of re-treatment.9–12 Practitioners may therefore empirically select their fractionation schedules based on the estimated survival to avoid the need to re-irradiate, using longer courses for those expected to live longer and shorter courses for those with a shorter life expectancy. Therefore, improving upon prediction accuracy by understanding the factors that influence this accuracy is crucial.
SABR has been steadily increasing in use with improvements in systemic chemotherapy, recognition of the oligometastatic disease state, and data supporting aggressive local therapy in improving overall survival in patients with limited metastatic disease34-35. In addition, a recent trial showed single fraction SABR has improved pain relief and local control compared to
11
conventional 30 Gy in 10 fractions36. This trend is reflected in the fact that 49% of the predictions in this study were on patients receiving SABR for metastatic disease. While it is reassuring that the SABR group of patients had the longest survival at 13 months, the accuracy for these patients was lower than in the overall cohort (31% vs 40%). Interestingly, these patients had a 42% underestimate rate compared to a 27% rate of overestimate rate. This result is opposite of our initial hypothesis that physicians largely overestimate survival for this group and would thus, be more willing to offer aggressive treatment. These results again highlight the need to better understand how predicted life expectancy impacts clinical decision-making.
There are several limitations of this analysis that should be acknowledged. Potential bias may have been introduced by excluding patients who lacked follow-up data and who remained alive but not past 24 months, although this only represented 4.4% and 5.8% of the original dataset respectively. Additionally, survival predictions were collected from a single academic institution and thus subject to institutional and regional practice patterns, biases, and economic influences that may not apply universally to other Radiation Oncology departments or practices. We do believe, however, that the relatively large sample size of physicians, well-balanced gender representation, and wide range of years of experience, offer a degree of generalizability. The sample size of physicians and patients is also larger than many published studies. Due to the specialized nature of an academic department where physicians subspecialize in specific disease sites, the number of physicians providing survival predictions for each primary site is further limited. As mentioned previously, the method of collecting survival predictions in pre-defined time bins of 3-6 months, a decision made to facilitate greater survey compliance, limited the analyses that could be performed on the dataset. In addition, our findings are based on correctly 12
survival within bin sizes of ≥6 months, larger than those used in other studies, making it difficult to compare our accuracy rates with those of other reported studies that used different bin sizes. Finally, the bin sizes chosen may not be optimal for clinical utility. For instance, assessing the accuracy of predicting survival of <1 month might have important implications on how these patients are managed clinically. However, survival predictions <3 months were recorded, and 64% of these predictions were correct. These results would indicate that the physicians were better at identifying those with very poor prognosis, which may impact how they managed these patients.
With these results, an area of further study is to determine how physician estimates impact their choice of fractionation schedule for radiation, which is the subject of a separate study. Another important area of investigation is how to improve prediction accuracy. There are a number of validated prognostic models (e.g. TEACHH37, Glasgow Prognostic Score38, the Palliative Prognostic model39) proposed for patients with advanced cancer which have shown good ability to identify those with limited survival. Such models may play a major role in helping aide physicians in clinical decision-making, though it is unclear how much they are in clinical use. In this study, we did not ask physicians to use any specific models when making their survival predictions which is also an important limitation, and a potential future study would be determining whether accuracy can be improved by incorporating one of these prognostic models into the survey. More recently, machine learning algorithms using complex computer-based models have begun to be explored for these purposes. For instance, a recent study showed that a model using natural language processing using the clinic notes and medical records from the
13
electronic health record was able to stratify patients distinctly into different risk groups.33 Further investigation would be to test whether these tools can successfully improve survival accuracy.
Conclusions The accuracy of radiation oncologists at a single academic center in predicting survival of patients with metastatic cancer was 40% with a 34% overestimate rate and a 27% underestimate rate. The factor that most correlated with accuracy was KPS. Further investigation is needed to determine how predictions influence treatment decisions and how to improve prediction accuracy.
14
References 1.
Emanuel EJ. Cost savings at the end of life. What do the data show? JAMA.
1996;275(24):1907-1914. 2.
Barnato AE, McClellan MB, Kagay CR, Garber AM. Trends in inpatient treatment
intensity among Medicare beneficiaries at the end of life. Health Serv Res. 2004;39(2):363-375. doi:10.1111/j.1475-6773.2004.00232.x 3.
Aldridge MD, Kelley AS. The Myth Regarding the High Cost of End-of-Life Care. Am J
Public Health. 2015;105(12):2411-2415. doi:10.2105/AJPH.2015.302889 4.
Emanuel EJ, Ash A, Yu W, et al. Managed care, hospice use, site of death, and medical
expenditures in the last year of life. Arch Intern Med. 2002;162(15):1722-1728. 5.
Yu W. End of Life Care: Medical Treatments and Costs by Age, Race, and Region.
HSRD Study IIR 02-189. https://www.hsrd.research.va.gov/research/abstracts/IIR_02-189.htm. Accessed November 12, 2018. 6.
Tangka FKL, Subramanian S, Sabatino SA, et al. End-of-Life Medical Costs of Medicaid
Cancer Patients. Health Serv Res. 2015;50(3):690-709. doi:10.1111/1475-6773.12259 7.
Murphy JD, Nelson LM, Chang DT, Mell LK, Le Q-T. Patterns of Care in Palliative
Radiotherapy: A Population-Based Study. J Oncol Pract. 2013;9(5):e220-e227. doi:10.1200/JOP.2012.000835 8.
Park KR, Lee CG, Tseng YD, et al. Palliative radiation therapy in the last 30 days of life:
A systematic review. Radiother Oncol. 2017;125(2):193-199. doi:10.1016/j.radonc.2017.09.016 9.
van den Hout WB, van der Linden YM, Steenland E, et al. Single- versus multiple-
fraction radiotherapy in patients with painful bone metastases: cost-utility analysis based on a randomized trial. J Natl Cancer Inst. 2003;95(3):222-229.
15
10.
Yarnold JR. 8 Gy single fraction radiotherapy for the treatment of metastatic skeletal
pain: randomised comparison with a multifraction schedule over 12 months of patient followupOn behalf of the Bone Pain Trial Working Party. Radiother Oncol. 1999;52(2):111-121. doi:10.1016/S0167-8140(99)00097-3 11.
Roos DE, Turner SL, O’Brien PC, et al. Randomized trial of 8 Gy in 1 versus 20 Gy in 5
fractions of radiotherapy for neuropathic pain due to bone metastases (Trans-Tasman Radiation Oncology Group, TROG 96.05). Radiother Oncol J Eur Soc Ther Radiol Oncol. 2005;75(1):5463. doi:10.1016/j.radonc.2004.09.017 12.
Hartsell WF, Scott CB, Bruner DW, et al. Randomized trial of short- versus long-course
radiotherapy for palliation of painful bone metastases. J Natl Cancer Inst. 2005;97(11):798-804. doi:10.1093/jnci/dji139 13.
Chow E, Harris K, Fan G, Tsao M, Sze WM. Palliative radiotherapy trials for bone
metastases: a systematic review. J Clin Oncol Off J Am Soc Clin Oncol. 2007;25(11):1423-1436. doi:10.1200/JCO.2006.09.5281 14.
Bezjak A, Dixon P, Brundage M, et al. Randomized phase III trial of single versus
fractionated thoracic radiation in the palliation of patients with lung cancer (NCIC CTG SC.15). Int J Radiat Oncol Biol Phys. 2002;54(3):719-728. 15.
Lutz ST, Jones J, Chow E. Role of Radiation Therapy in Palliative Care of the Patient
With Cancer. J Clin Oncol. 2014;32(26):2913-2919. doi:10.1200/JCO.2014.55.1143 16.
Glare P, Virik K, Jones M, et al. A systematic review of physicians’ survival predictions
in terminally ill cancer patients. BMJ. 2003;327(7408):195.
16
17.
How Accurate are Physicians’ Clinical Predictions of Survival and the Available
Prognostic Tools in Estimating Survival Times in Terminally III Cancer Patients? A Systematic Review. Clin Oncol. 2001;13(3):209-218. doi:10.1053/clon.2001.9256 18.
Christakis NA, Lamont EB. Extent and determinants of error in doctors’ prognoses in
terminally ill patients: prospective cohort study. BMJ. 2000;320(7233):469-472. 19.
Chow E, Davis L, Panzarella T, et al. Accuracy of survival prediction by palliative
radiation oncologists. Int J Radiat Oncol • Biol • Phys. 2005;61(3):870-873. doi:10.1016/j.ijrobp.2004.07.697 20.
Hartsell WF, Desilvio M, Bruner DW, et al. Can Physicians Accurately Predict Survival
Time in Patients with Metastatic Cancer? Analysis of RTOG 97-14. J Palliat Med. 2008;11(5):723-728. doi:10.1089/jpm.2007.0259 21.
Fairchild A, Debenham B, Danielson B, Huang F, Ghosh S. Comparative
multidisciplinary prediction of survival in patients with advanced cancer. Support Care Cancer Off J Multinatl Assoc Support Care Cancer. 2014;22(3):611-617. doi:10.1007/s00520-013-20132 22.
Kondziolka D, Parry PV, Lunsford LD, et al. The accuracy of predicting survival in
individual patients with cancer. J Neurosurg. 2014;120(1):24-30. doi:10.3171/2013.9.JNS13788 23.
Gripp S, Moeller S, Bölke E, et al. Survival prediction in terminally ill cancer patients by
clinical estimates, laboratory tests, and self-rated anxiety and depression. J Clin Oncol Off J Am Soc Clin Oncol. 2007;25(22):3313-3320. doi:10.1200/JCO.2006.10.5411 24.
Mackillop WJ, Quirt CF: Measuring the accuracy of prognostic judgments in oncology. J
Clin Epidemiol 50:21-9, 1997
17
25.
Oxenham D, Cornbleet MA. Accuracy of prediction of survival by different professional
groups in a hospice. Palliat Med. 1998;12(2):117-118. doi:10.1191/026921698672034203 26.
Mackillop WJ, Quirt CF. Measuring the accuracy of prognostic judgments in oncology. J
Clin Epidemiol. 1997;50(1):21-29. 27.
Lambden J, Zhang B, Friedlander R, Prigerson HG. Accuracy of Oncologists’ Life-
Expectancy Estimates Recalled by Their Advanced Cancer Patients: Correlates and Outcomes. J Palliat Med. 2016;19(12):1296-1303. doi:10.1089/jpm.2016.0121 28.
O’Keefe EB, Meltzer JP, Bethea TN. Health Disparities and Cancer: Racial Disparities in
Cancer Mortality in the United States, 2000–2010. Front Public Health. 2015;3. doi:10.3389/fpubh.2015.00051 29.
Wong SL. Medically Underserved Populations: Disparities in Quality and Outcomes. J
Oncol Pract. 2015;11(3):193-194. doi:10.1200/JOP.2015.004259 30.
Disparities in Cancer Care. J Oncol Pract. 2006;2(5):234-239.
31.
Tseng YD, Krishnan MS, Sullivan AJ, Jones JA, Chow E, Balboni TA. How Radiation
Oncologists Evaluate and Incorporate Life Expectancy Estimates Into the Treatment of Palliative Cancer Patients: A Survey-Based Study. Int J Radiat Oncol Biol Phys. 2013;87(3):471-478. doi:10.1016/j.ijrobp.2013.06.2046 32.
Gwilliam B, Keeley V, Todd C, et al. Prognosticating in patients with advanced cancer--
observational study comparing the accuracy of clinicians’ and patients’ estimates of survival. Ann Oncol Off J Eur Soc Med Oncol. 2013;24(2):482-488. doi:10.1093/annonc/mds341 33.
Bruera E, Miller MJ, Kuehn N, MacEachern T, Hanson J. Estimate of survival of patients
admitted to a palliative care unit: a prospective study. J Pain Symptom Manage. 1992;7(2):82-86.
18
33.
Banerjee I, Gensheimer MF, Wood DJ, et al. Probabilistic Prognostic Estimates of
Survival in Metastatic Cancer Patients (PPES-Met) Utilizing Free-Text Clinical Narratives. Sci Rep. 2018;8. doi:10.1038/s41598-018-27946-5 34.
Palma DA, Olson R, Harrow S, et al: Stereotactic ablative radiotherapy versus standard
of care palliative treatment in patients with oligometastatic cancers (SABR-COMET): a randomised, phase 2, open-label trial. Lancet 393:2051-2058, 2019 35.
Gomez DR, Blumenschein GR, Jr., Lee JJ, et al: Local consolidative therapy versus
maintenance therapy or observation for patients with oligometastatic non-small-cell lung cancer without progression after first-line systemic therapy: a multicentre, randomised, controlled, phase 2 study. Lancet Oncol 17:1672-1682, 2016 36.
Nguyen QN, Chun SG, Chow E, et al: Single-Fraction Stereotactic vs Conventional
Multifraction Radiotherapy for Pain Relief in Patients With Predominantly Nonspine Bone Metastases: A Randomized Phase 2 Trial. JAMA Oncol 5:872-878, 2019 37.
Krishnan MS, Epstein-Peterson Z, Chen YH, et al: Predicting life expectancy in patients
with metastatic cancer receiving palliative radiotherapy: the TEACHH model. Cancer 120:13441, 2014 38.
McMillan DC: The systemic inflammation-based Glasgow Prognostic Score: a decade of
experience in patients with cancer. Cancer Treat Rev 39:534-40, 2013 39.
Glare PA, Eychmueller S, McMahon P: Diagnostic accuracy of the palliative prognostic
score in hospitalized patients with advanced cancer. J Clin Oncol 22:4823-8, 2004
19
Figure Legends Figure 1. Significant Variable Interactions for Attending Predictions for (A) Two-way interactions of age and primary site and (B) Two-way interactions of age and race.
20
Tables Table 1. Patient Characteristics Variable
Count
Primary
CNS
Site
Insurance
Marital
Number of
Number of
surveys
patients
(N= 877)
(N=689)* 3 (0.3%)
2 (0.3%)
GI
140 (16%)
120 (17%)
GU
92 (10%)
78 (11%)
H&N
49 (6%)
44 (6%)
Lymphoma
26 (3%)
23 (3%)
Sarcoma
38 (4%)
29 (4%)
GYN
54 (6%)
42 (6%)
Lung
275 (31%)
202 (29%)
Breast
128 (15%)
92 (13%)
Skin
39 (4%)
35 (5%)
Other
33 (4%)
22 (3%)
Private
396 (45%)
295 (43%)
Public
387 (44%)
316 (46%)
VA
88 (10%)
74 (11%)
None†
6 (0.7%)
6 (0.9%)
589 (67%)
455 (66%)
Married
162 (18.5%)
129 (19%)
Widowed
54 (6%)
42 (6%)
Divorced
56 (6%)
50 (7%)
Missing†
16 (2%)
14 (2%)
<=50
73 (8%)
69 (10%)
60-70
256 (29%)
235 (34%)
548 (62.5%)
456 (66%)
White
464 (53%)
374 (54%)
Black
34 (4%)
28 (4%)
Asian
204 (23%)
153 (22%)
96 (11%)
77 (11%)
79 (9%)
59 (9%)
Female
460 (52.5%)
347 (50%)
Male
417 (47.5%)
342 (50%)
Experience <10 years
359 (41%)
287 (42%)
of
10-20 years
302 (34%)
243 (35%)
Attending
>20 years
216 (25%)
191 (28%)
Fractions
SABR (1-5
431 (49%)
344 (50%)
Prescribed
fractions) 50 (6%)
49 (7%)
CFRT 2-5 fractions
115 (13%)
108 (16%)
CFRT >5 fractions
281 (32%)
266 (39%)
Status
KPS
Single
80-100 Race
Hispanic Other Gender
(years)
CFRT 1 fraction
Age
Patient OS
<65
447 (51%)
335 (49%)
65 +
430 (49%)
354 (51%)
0-6 months
336 (38%)
289 (42%)
>6-12 months
168 (19%)
154 (22%)
>12-24 months
153 (17%)
131 (19%)
>24 months
220 (25%)
187 (27%)
Abbreviations: KPS, Karnofsky performance status; OS, observed survival; CNS, central nervous system; GI, gastrointestinal; GU, genitourinary; H&N, head and neck; GYN, gynecological; SABR, stereotactic ablative radiotherapy; CFRT, conventionally fractionated radiotherapy. *Many of the number of patients (N) for each variable will not sum to the 689 unique patients analyzed because a patient could be counted in multiple categories per variable †Excluded from Table 2 and analyses because limited number
Table 2. Physician Accuracy and Survival Statistics by Patient and Disease Characteristics Variable
N under
N accurate
N over
(%)
(%)
(%)
Median Follow up
Range Follow up
Primary Site
CNS
1 (33%)
1 (33%)
1 (33%)
3.0
[2.0, 10.0]
(p=0.1644)*
GI
42 (30%)
67 (48%)
31 (22%)
9.0
[0.0, 38.0]
GU
29 (32%)
34 (37%)
29 (32%)
11.5
[0.0, 38.0]
H&N
14 (30%)
18 (38%)
15 (32%)
9.0
[0.0, 38.0]
Lymphoma
6 (23%)
10 (38%)
10 (38%)
11.0
[0.0, 39.0]
Sarcoma
8 (21%)
17 (45%)
13 (34%)
6.5
[0.0, 38.0]
GYN
19 (35%)
22 (41%)
13 (24%)
7.0
[0.0, 38.0]
Lung
110 (40%)
102 (37%)
63 (23%)
11.0
[0.0, 39.0]
Breast
45 (35%)
47 (37%)
36 (28%)
12.0
[0.0, 38.0]
Unknown
6 (18%)
19 (58%)
8 (24%)
6.0
[1.0, 35.0]
Skin
15 (38%)
11 (28%)
13 (33%)
9.0
[0.0, 34.0]
Insurance
Private
141 (36%)
165 (42%)
90 (23%)
12.0
[0.0, 39.0]
(p=0.2946)*
Public
123 (32%)
150 (39%)
114 (30%)
9.0
[0.0, 39.0]
VA
31 (35%)
29 (33%)
28 (32%)
7.0
[0.0, 38.0]
Marital
Married
196 (33%)
241 (41%)
152 (26%)
10.0
[0.0, 39.0]
Status
Single
53 (33%)
70 (43%)
39 (24%)
11.0
[0.0, 39.0]
(p=0.0786)*
Widowed
22 (41%)
14 (26%)
18 (33%)
11.0
[0.0, 38.0]
Divorced
20 (36%)
18 (32%)
18 (32%)
9.0
[0.0, 35.0]
KPS
<=50
11 (15%)
50 (68%)
12 (16%)
2.0
[0.0, 38.0]
(p<0.0001)*
60-70
62 (24%)
114 (45%)
80 (31%)
5.0
[0.0, 38.0]
80-100
224 (41%)
184 (34%)
140 (26%)
15.0
[0.0, 39.0]
Race
White
156 (34%)
173 (37%)
135 (29%)
10.0
[0.0, 39.0]
(p=0.3114)*
Black
13 (38%)
13 (38%)
8 (24%)
8.5
[0.0, 39.0]
Asian
70 (34%)
91 (45%)
43 (21%)
10.0
[0.0, 39.0]
Hispanic
34 (35%)
35 (36%)
27 (28%)
13.5
[0.0, 39.0]
Other
24 (30%)
36 (46%)
19 (24%)
10.0
[1.0, 36.0]
Gender
Female
154 (33%)
188 (41%)
118 (26%)
11.0
[0.0, 38.0]
(p=0.4498)*
Male
143 (34%)
160 (38%)
114 (27%)
9.0
[0.0, 39.0]
Experience
< 10 years
98 (27%)
160 (45%)
101 (28%)
9.0
[0.0, 39.0]
of Attending
10-20 years
140 (46%)
99 (33%)
63 (21%)
10.0
[0.0, 39.0]
(years)
>20 years
59 (27%)
89 (41%)
68 (31%) 12.0
[0.0, 38.0]
13.0
[0.0, 39.0]
2.5
[0.0, 38.0]
5.0
[0.0, 38.0]
9.0
[0.0, 39.0]
(p<0.0001)* Fractions
SABR (1-5
Prescribed
fractions)
(p<0.0001)*
CFRT 1
181 (42%)
11 (22%)
132 (31%)
31 (62%)
118 (27%)
8 (16%)
fraction CFRT 2-5
25 (22%)
66 (57%)
24 (21%)
fractions CFRT >5
80 (28%)
119 (42%)
82 (29%)
fractions Age
<65
161 (36%)
187 (42%)
99 (22%)
12.0
[0.0, 39.0]
(p=0.1838)*
65 +
136 (32%)
161 (37%)
133 (31%)
9.0
[0.0, 39.0]
Patient OS
0-6 months
0 (0%)
169 (50%)
167 (50%)
3.0
[0.0, 6.0]
(p<0.0001)*
>6-12
42 (25%)
74 (44%)
52 (31%) 9.0
[7.0, 12.0]
17.0
[13.0, 24.0]
months >12-24
90 (59%)
50 (33%)
13 (9%)
months >24
165 (75%)
55 (25%)
0 (0%)
† [25.0, 39.0] months
Abbreviations: KPS, Karnofsky performance status; OS, observed survival; CNS, central nervous system; GI, gastrointestinal; GU, genitourinary; H&N, head and neck; GYN,
gynecological; SABR, stereotactic ablative radiotherapy; CFRT, conventionally fractionated radiotherapy. *p-values assessing difference in percentages of accurate estimates within variable subcategories †No median available because many patients with observed survival >24 months are still living
Table 3. Univariable and Multivariable Models for Accuracy of Physician Survival Predictions Accuracy Effect
Univariable
Multivariable
Models (p-values)
Model (p-values)
Insurance
0.2329
0.7604
Race
0.3114
0.0648
Marital Status
0.0786
0.0428
< 0.0001
<.0001
Gender
0.4498
0.3058
Age
0.1824
0.6016
Fractions Prescribed
0.4359
0.3302
Experience Attending
0.5988
0.7240
Primary Site
0.1806
0.3170
KPS
Abbreviations: KPS, Karnofsky performance status
Table 4. Univariable, Multivariable, and Parsimonious Multivariable Models for Physician Survival Prediction Outcomes, Including Variable Interactions Univariable and Multivariable Models Effect
Univariable
Multivariable
Models (p-values)
Model (p-values)
Insurance
0.1595
0.5251
Race
0.7519
0.7971
Marital Status
0.9842
0.9267
KPS
0.0003
0.0003
Gender
0.8879
0.6759
Age
0.1292
0.2183
Fractions (Fx) Prescribed
0.0350
0.2130
Experience Attending
0.0296
0.1929
Primary Site
0.2454
0.3335
Multivariable Parsimonious Model with Variable Interactions Effect
Parsimonious Multivariable Model (p-values)
KPS
0.0005
Primary Site
0.0063
Interaction of Age and Primary Site
0.0200
Age
0.1071
Race
0.0010
Interaction of Age and Race
0.0005
Abbreviations: KPS, Karnofsky performance status
Genitourinary (n=92)
Lung (n=275)
Underestimate
0.8 Accurate
Accurate 0.8
0.6
Accurate
Overestimate
Overestimate
Overestimate
0.4 0.2
Predicted Probabilities
Underestimate
Predicted Probabilities
Predicted Probabilities
0.8
1.0
Underestimate
0.6 0.4 0.2 0.0
0.0 10
20
30
40
50
60
70
80
20
30
Breast (n=128)
40
60
70
80
90 100
10
0.6 0.4 0.2
60
70
80
Accurate
Overestimate
Overestimate
Overestimate
0.6 0.4 0.2
20
30
40
50
60
70
80
40
50
60
Age
0.2
90 100
10
20
30
40
70
80
90 100
50
60
70
80
90 100
Age
Sarcoma (n=38) 1.0
Underestimate
Underestimate
0.8 Accurate
0.8 Accurate
Accurate
Overestimate
Overestimate
Overestimate
Predicted Probabilities
Underestimate
0.6 0.4 0.2 0.0
30
0.4
0.0 10
Predicted Probabilities
0.0 20
0.6
Gastrointestinal (n=140)
0.2
90 100
Underestimate
1.0
0.4
80
0.8 Accurate
Head and Neck (n=49)
0.6
70
Gynecologic (n=54)
Age
0.8
60
Underestimate
90 100
1.0
50
0.8 Accurate
Age
10
40
1.0
0.0 50
30
Underestimate
0.0 40
20
Age
Predicted Probabilities
0.8
30
0.2
Skin (n=39)
Predicted Probabilities
Predicted Probabilities
50
1.0
20
0.4
Age
1.0
10
0.6
0.0 10
90 100
Age
Predicted Probabilities
Lymphoma (n=26)
1.0
1.0
0.6 0.4 0.2 0.0
10
20
30
40
50
60
Age
70
80
90 100
10
20
30
40
50
60
70
80
90 100
Age
Figure 1. Significant Variable Interactions for Attending Predictions (continued) Figure 1B. Two-way interactions of age and race
S0360-3016(19)33949-5
DOI:
https://doi.org/10.1016/j.ijrobp.2019.10.032
Reference:
ROB 26007
To appear in:
International Journal of Radiation Oncology • Biology • Physics
Received Date: 2 July 2019 Revised Date:
10 September 2019
Accepted Date: 12 October 2019
Please cite this article as: Benson KR, Aggarwal S, Carter JN, von Eyben R, Pradhan P, Prionas ND, Bui JL, Soltys SG, Hancock S, Gensheimer MF, Koong AC, Chang DT, Predicting Survival for Patients with Metastatic Disease, International Journal of Radiation Oncology • Biology • Physics (2019), doi: https://doi.org/10.1016/j.ijrobp.2019.10.032. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Elsevier Inc. All rights reserved.
Predicting Survival for Patients with Metastatic Disease Running title: Predicting Patient Survival Authors: Kathryn RK Benson BA1, Sonya Aggarwal BA1, Justin N. Carter BA1, Rie von Eyben MS1, Pooja Pradhan BS1, Nicolas D. Prionas MD PhD1, Justin L. Bui BS1, Scott G. Soltys MD1, Steven Hancock MD1, Michael F. Gensheimer MD1, Albert C. Koong MD PhD2, Daniel T. Chang MD1 1 2
Radiation Oncology Department, Stanford Cancer Institute Radiation Oncology Department, MD Anderson Cancer Center
Corresponding Author information: Daniel T Chang, Department of Radiation Oncology Stanford University, 875 Blake Wilbur Dr., Stanford, CA 94305-5847. Email: [email protected]; Phone: 650-724-3547; Fax: 650-725-8231
Author responsible for statistical analyses: Rie von Eyben, Department of Radiation Oncology Stanford University, 875 Blake Wilbur Dr., Stanford, CA 94305-5847. Email: [email protected]; Phone: 650-723-0914; Fax: 650-725-8231
Conflict of interests: The authors have no conflicts of interest to disclose.
Funding sources: None.
Acknowledgements: Billy Loo MD/PhD, Clement Ho MD, Carol Marquez MD, David Shultz MD/PhD, Elizabeth Kidd MD, Iris Gibbs MD, Jennifer Shah MD, Lynn Million MD, Mark Buyyounouski MD/MS, Maximilian Diehn MD/PhD, Patrick Swift MD, Quynh-Thu Le MD, Richard Hoppe MD, Sarah Donaldson MD, Susan Knox MD/PhD, Steven Hancock MD, Wendy Hara MD
Predicting Survival for Patients with Metastatic Disease Running title: Predicting Patient Survival Abstract
Purpose This prospective study aimed to determine the accuracy of radiation oncologists in predicting the survival of patients with metastatic disease receiving radiotherapy and to understand factors associated with their accuracy. Methods and Materials This single-institution study surveyed 22 attending radiation oncologists to estimate patient survival. Survival predictions were defined as accurate if the observed survival (OS) was within the correct survival prediction category (0-6 months, >6-12 months, >12-24 months, and >24 months). The physicians made survival estimates for each course of radiation, yielding 877 analyzable predictions for 689 unique patients. Data analysis included Stuart’s Tau C, logistic regression models, ordinal logistic regression models, and stepwise selection to examine variable interactions. Results Of the 877 radiation oncologists’ predictions, 39.7% were accurate, 26.5% underestimations, and 33.9% overestimations. Stuart’s Tau C showed low correlation between OS and survival estimates (0.3499), consistent with the inaccuracy reported in literature. However, results showed less systematic over-prediction than reported in the literature. Karnofsky performance status (KPS) was the most significant predictor of accuracy with greater accuracy for patients with shorter OS. Estimates were also more accurate for patients with lower KPS. Accuracy by
1
patient age varied by primary site and race. Physician years of experience did not correlate with accuracy. Conclusions The sampled radiation oncologists have relatively low accuracy in predicting patient survival. Future investigation should explore how survival estimates influence treatment decisions and how to improve survival prediction accuracy.
2
Introduction End of life care is estimated to comprise 10%-12% of the United States healthcare budget and 27%-30% of the Medicare budget.1,2 Of the total expenditure on end of life care, approximately 13% was spent on care in the final year of life.3 Estimates of the percentage of healthcare costs within the last year of life that occur in the final month range from 33%-78%.4,5 Care for cancer patients represents a significant portion of these costs. Using Medicaid administrative data, a 2015 study estimated that patients with cancer spend about $10,000 more in the final 4 months of life than non-cancer patients.6
Part of end-of-life cancer care involves palliative radiation therapy (PRT). SEER-Medicare database analysis of patients with stage IV cancer showed that 41% received PRT, generally ranging from 1-15 fractions.7 Of the SEER-Medicare database of patients receiving PRT, 78% received a single course, 17% two courses, and 5% three or greater courses.7 According to a 2017 systematic review, 5-10% of patients who died from cancer received PRT in their final month of life.8 For patients who received PRT for metastatic bony disease, treatment with single fraction regimens are associated with higher retreatment rates of 23-25% compared with that of multi-fraction regimens of 7-10%.9,10 However, growing evidence suggests that for most histologies, there is no clinical advantage to greater fractions.9–15 Despite the higher retreatment rates, single fraction regimens are still more cost-effective and provide equal palliation for patients with metastatic cancer to the bone .9,13
The decision to offer shorter versus longer fractionation schemes is often based empirically on the perceived life expectancy of the patient. Thus, patients with poor estimated prognosis may
3
preferentially receive shorter courses of radiation because of their lower likelihood of needing retreatment.13 Physicians’ ability to accurately predict survival can therefore impact the type of treatment given to terminally ill patients. Thus, there may be an opportunity to optimize the utilization of palliative radiotherapy based on patient prognosis.
The literature has shown that physicians generally overestimate survival among patients with terminal cancers.16–18 The limited literature on radiation oncologists reports that they are accurate between 10-60% of the time, the large range resulting from the heterogeneity in defining accuracy, and generally overestimate patient survival.19–23 The aim of this study is to determine the accuracy of radiation oncologists at XXX in predicting survival of patients with metastatic disease treated with radiotherapy and to understand the factors that correlate with accuracy. With a larger sample size of radiation oncologists and an analysis of factors influencing under, accurate, and overestimates, this study has important implications for how radiation oncologists make treatment decisions for patients with incurable disease.
Methods and Materials In this IRB-approved prospective study, we identified patients who had metastatic cancer and were receiving conventionally fractionated radiotherapy (CFRT) or stereotactic ablative radiotherapy (SABR) from March 2015 to November 2016 in the Department of Radiation Oncology at XXX. For each palliative treatment course, within the week of starting radiation treatment, the treating attending physicians and each member of that physician’s care team (the resident physician, nurse coordinator, and radiation therapy technologist) were given a survey that asked them to 1) predict the life expectancy for this patient, 2) provide the main reasons for
4
this prediction, and 3) identify the primary reason for the chosen fractionation schedule (Appendix A). Surveys were administered for each course of radiation. Therefore, patients being re-treated with radiotherapy would have more than one round of surveys completed about their predicted survival. The clinical treatment characteristics and details were recorded, and the patients were followed for survival using electronic medical records (EMR) and public record obituaries.
Survival was defined as the time from the date of the consultation until date of death. If death was not observed then the patient was censored at the date of last follow up. Only patients who died or lived longer than 24 months were included in the analyses. Stuart’s Tau C was used to assess the correlation between the predicted and observed survival times. Survival predictions were defined as accurate if the patient survival was within the survival prediction category (0-6 months, >6-12 months, >12-24 months, and >24 months). The binary outcome of accurate or inaccurate predictions were analyzed using logistic regression. The ordinal outcome of underestimated prediction, accurate prediction, or overestimated prediction was analyzed using ordinal logistic regression. A stepwise selections algorithm was used to select the predictors to be included in the models with a criterion of 0.1 for entry into the model and 0.05 for exiting the model. To assess if the observations were more highly correlated within physician, an ordinal logistic regression model was fit with stratification by physician. All analyses were two-sided tests with an alpha level of 0.05; and all analyses were performed using SAS v 9.4 (SAS Institute Inc, Cary, NC). Predictors included in the model the following: 1) Primary site, 2) Age at time of receiving radiation, 3) Race – White, Black, Asian, Hispanic, other. 4) Insurance Status – private, public, VA, none, 5) Marital status – married, single, widowed, divorced, missing, 6)
5
Gender, 7) KPS - ≤50, 60-70, ≥80, 8) Fractions prescribed – SABR (1-5 fractions), single fraction, 2-5 fractions, >5 fractions, 9) Experience of attending physician defined as the number or years since graduating residency - <10 years, 10-20 years, >20 years.
Results Prediction Accuracy Participants in the study included 22 attending physicians out of a total of 23 faculty (96%) within the Department of Radiation Oncology at XXX during the years of data collection for this study. Of the physicians sampled, 10 were women, 12 were men, 9 had less than 10 years of experience, 6 had 10 to 20 years of experience, and 7 had greater than 20 years of experience, with experience defined as the years in practice from residency graduation to study initiation time point.
In total, 1190 patient survival estimates were collected in this study. Out of these, 877 predictions were included in the analysis with 52 excluded because of lack of follow-up data (4.4%), 70 excluded because they were alive but not yet past 24 months (5.8%), and 191 (16.0%) excluded because of a missing physician prediction. The 877 physician survival estimates represented 689 unique patients, with patient characteristics shown in Table 1. The overall physician accuracy was 39.7% with 26.5% underestimates and 33.9% overestimates. This data yielded a Stuart’s Tau C value of 0.3499, indicating low correlation between the predictions and actual patient survival. Table 2 shows physician underestimation, accuracy, and overestimation by disease and patient characteristics.
6
Physician accuracy decreased with increasing KPS, with accuracy rates of 68% for KPS of 4050, 45% for KPS of 60-70, and 34% for KPS 80-100 (p<0.0001). By physician experience, accuracy was lowest in those who had 10 to 20 years of experience (33%) compared to those who had <10 years (45%) and >20 years (41%) experience. Physicians were more accurate in survival predictions of patients with shorter OS, 50% accurate for OS 0-6 months, 44% for >6-12 months OS, 33% for >12-24 months OS, and 25% for >24 months OS (p<0.0001). Physician accuracy was lowest for patients prescribed SABR (31%) compared to CFRT >5 fractions (42%), CFRT 2-5 fractions (57%), and CFRT 1 fraction (62%), which was consistent with the inverse relationship between OS and accuracy. SABR had the longest survival median (13 months), followed by CFRT >5 (9 months), 2-5 (5 months), and 1 (2.5 months). Although the accuracy for SABR patients was only 31%, physicians underestimated OS 42% of the time and overestimated only 27% for this cohort.
Predictors of Accuracy On univariable analysis, only KPS correlated with physician accuracy (p<0.0001), while age, gender, race, insurance coverage, primary disease site, marital status, fractions prescribed, and experience of attending did not. On multivariable analysis including all predictors as a main effect, KPS (p<0.0001) and marital status (p=0.0428) were correlated with prediction accuracy (table 3).
Physician Prediction Outcomes – Underestimate, Accurate and Overestimate Physician prediction outcomes were categorized as underestimates, accurate, and overestimates. Table 4 shows the results of the single predictor analysis, with KPS (p=0.0003), fractions
7
prescribed (p=0.0350), and experience of attending (p=0.0296) correlating with prediction outcome. On multivariable analysis including main effects only, KPS (p=0.0003) was the only factor that correlated with prediction outcome.
A stepwise selection was used to choose a final model and the algorithm was allowed to select among all variables as main effects and the two-way interactions between potentially clinically relevant factors: race, KPS, gender, age, fractions prescribed, experience of attending, and primary site. The model selected (Table 4) includes the interactions between age and primary site (0.0200), age and race (0.0005) as well as the main effects of age (0.1071), primary site (0.0063), race (0.0010) and KPS (0.0005).
For most primary sites, accuracy remained similar regardless of patient age, with the exception being lung and genitourinary (GU) where accuracy exhibited roughly a bell curve, peaking around 60-70 years of age for GU and peaking around 80 for lung. GU and lung exhibited highest overestimate rate among older patients and highest underestimate rate among younger patients. For patients with breast, GYN, lymphoma, and sarcoma primary sites, accuracy slightly increased with age, overestimate rate increased with age, and underestimate rate decreased with age. GI and H&N cancers exhibited no age-associated prediction trends (Figure 1A). Regarding race/ethnicity and age, White, Asian, Hispanic, and Other patients, underestimate rate decreased with age while accuracy and overestimate rates increased with age. However, for Black patients, the opposite trend was observed (Figure 1B).
8
Finally, a sensitivity analysis was performed in order to test if the observations within each physician were correlated. The final model using the stepwise selection algorithm was fit with stratification by physician. There were only very minor differences in the p-values between the model with and without stratification indicating that the observations within physician are correlated enough to warrant a model without stratification (data not shown).
Physician-Reported Factors Influencing Prediction On each survey, the physicians ranked the top three factors informing their survival prediction. The four most common primary factors were type of cancer 289 (33%), KPS 265 (30%), tumor burden 173 (20%), pace of disease 130 (15%). The group of physicians who reported type of cancer as the primary factor influencing their prediction had the lowest prediction accuracy (34%), followed by KPS (42%), tumor burden (42%), and pace of disease (45%). There was no statistically significant correlation with these factors and accuracy (p=0.0914).
Discussion The results showed that physician accuracy was only 40%. In addition, in 27%, physicians underestimated survival while in 34%, physicians overestimated survival. Directly comparing physician accuracy across studies in the literature is difficult because of the heterogeneity in how estimates are made and how accuracy is defined. In our study, physicians were asked to predict survival within 3-6 month time bins, and accuracy was defined as predicting the time bin (0-6 months, >6-12 months, >12-24 months, and >24 months) containing the actual survival. Whereas some studies have similarly had physicians estimate the correct survival time bin, other studies
9
have asked physicians to predict the precise survival. The wide range in accuracy reported in the literature, 10-60%, is likely because of the heterogeneity in defining accuracy.19–23 Gripp et al had radiation oncologists, including residents, make estimates in the binned categories of <1 month, 1-6 months, and >6 months and found 61% accuracy.23 Hartsell et al and Fairchild et al both reported accuracy as the radiation oncologists correctly predicting survival within 1 month of actual survival, and found 10% and 28.7% accuracy respectively.20,21
One comparable measure between many notable studies is the ratio of the percentage of overestimates and underestimates. Hartsell et al found a 2.10 ratio of overestimation to underestimation percentages among radiation oncologists in the palliative care setting20, while Christakis et al found a 3.71 ratio among physicians referring patients to hospice18, and Glare et al found an average of a 2.25 ratio in a systematic review of physician predictions for patients with terminal cancers.16 Our study found 33.9% of physicians overestimated and 26.5% underestimated, which was a much lower 1.28 ratio.
Our analysis showed that physician accuracy was higher for patients with shorter survival (Table 2), a finding called the “horizon effect,” a meteorology concept describing the observation that predicting short term events is more accurate than predicting long term events24. The horizon effect has been observed in some studies though not universally corraborated.17,25,26 Similarly, the physicians were more accurate at predicting survival of patients with low KPS (40-50), consistent with Lambden et al.27 The analysis further showed accuracy by patient age varied by primary site and race/ethnicity. Physicians had less accuracy and greater underestimates for older, Black patients than younger, Black patients. These trends were opposite of those seen with
10
patients in the White, Asian, Hispanic, and Other categories. Given the known disparities in patterns of care and access to care for underserved populations, further investigation is needed to better understand the reasons for these differences in survival predictions.28-30 Contrary to our initial hypothesis, there was no association between physician experience and prediction accuracy, confirming findings in the literature which similarly showed the lack of correlation of experience and accuracy.19,23,31–33
Given the high cost of end of life care, improving accuracy to allow for more tailored therapy based on prognosis is increasingly important1-2. Given the wide variation of available fractionation schedules, customization of palliative radiation is a major area of opportunity to maximize value of care. Multiple prospective trials have shown that single-fraction radiation treatments are equally effective at relieving pain in patients with painful bony metastases for most histologies but are associated with higher rates of re-treatment.9–12 Practitioners may therefore empirically select their fractionation schedules based on the estimated survival to avoid the need to re-irradiate, using longer courses for those expected to live longer and shorter courses for those with a shorter life expectancy. Therefore, improving upon prediction accuracy by understanding the factors that influence this accuracy is crucial.
SABR has been steadily increasing in use with improvements in systemic chemotherapy, recognition of the oligometastatic disease state, and data supporting aggressive local therapy in improving overall survival in patients with limited metastatic disease34-35. In addition, a recent trial showed single fraction SABR has improved pain relief and local control compared to
11
conventional 30 Gy in 10 fractions36. This trend is reflected in the fact that 49% of the predictions in this study were on patients receiving SABR for metastatic disease. While it is reassuring that the SABR group of patients had the longest survival at 13 months, the accuracy for these patients was lower than in the overall cohort (31% vs 40%). Interestingly, these patients had a 42% underestimate rate compared to a 27% rate of overestimate rate. This result is opposite of our initial hypothesis that physicians largely overestimate survival for this group and would thus, be more willing to offer aggressive treatment. These results again highlight the need to better understand how predicted life expectancy impacts clinical decision-making.
There are several limitations of this analysis that should be acknowledged. Potential bias may have been introduced by excluding patients who lacked follow-up data and who remained alive but not past 24 months, although this only represented 4.4% and 5.8% of the original dataset respectively. Additionally, survival predictions were collected from a single academic institution and thus subject to institutional and regional practice patterns, biases, and economic influences that may not apply universally to other Radiation Oncology departments or practices. We do believe, however, that the relatively large sample size of physicians, well-balanced gender representation, and wide range of years of experience, offer a degree of generalizability. The sample size of physicians and patients is also larger than many published studies. Due to the specialized nature of an academic department where physicians subspecialize in specific disease sites, the number of physicians providing survival predictions for each primary site is further limited. As mentioned previously, the method of collecting survival predictions in pre-defined time bins of 3-6 months, a decision made to facilitate greater survey compliance, limited the analyses that could be performed on the dataset. In addition, our findings are based on correctly 12
survival within bin sizes of ≥6 months, larger than those used in other studies, making it difficult to compare our accuracy rates with those of other reported studies that used different bin sizes. Finally, the bin sizes chosen may not be optimal for clinical utility. For instance, assessing the accuracy of predicting survival of <1 month might have important implications on how these patients are managed clinically. However, survival predictions <3 months were recorded, and 64% of these predictions were correct. These results would indicate that the physicians were better at identifying those with very poor prognosis, which may impact how they managed these patients.
With these results, an area of further study is to determine how physician estimates impact their choice of fractionation schedule for radiation, which is the subject of a separate study. Another important area of investigation is how to improve prediction accuracy. There are a number of validated prognostic models (e.g. TEACHH37, Glasgow Prognostic Score38, the Palliative Prognostic model39) proposed for patients with advanced cancer which have shown good ability to identify those with limited survival. Such models may play a major role in helping aide physicians in clinical decision-making, though it is unclear how much they are in clinical use. In this study, we did not ask physicians to use any specific models when making their survival predictions which is also an important limitation, and a potential future study would be determining whether accuracy can be improved by incorporating one of these prognostic models into the survey. More recently, machine learning algorithms using complex computer-based models have begun to be explored for these purposes. For instance, a recent study showed that a model using natural language processing using the clinic notes and medical records from the
13
electronic health record was able to stratify patients distinctly into different risk groups.33 Further investigation would be to test whether these tools can successfully improve survival accuracy.
Conclusions The accuracy of radiation oncologists at a single academic center in predicting survival of patients with metastatic cancer was 40% with a 34% overestimate rate and a 27% underestimate rate. The factor that most correlated with accuracy was KPS. Further investigation is needed to determine how predictions influence treatment decisions and how to improve prediction accuracy.
14
References 1.
Emanuel EJ. Cost savings at the end of life. What do the data show? JAMA.
1996;275(24):1907-1914. 2.
Barnato AE, McClellan MB, Kagay CR, Garber AM. Trends in inpatient treatment
intensity among Medicare beneficiaries at the end of life. Health Serv Res. 2004;39(2):363-375. doi:10.1111/j.1475-6773.2004.00232.x 3.
Aldridge MD, Kelley AS. The Myth Regarding the High Cost of End-of-Life Care. Am J
Public Health. 2015;105(12):2411-2415. doi:10.2105/AJPH.2015.302889 4.
Emanuel EJ, Ash A, Yu W, et al. Managed care, hospice use, site of death, and medical
expenditures in the last year of life. Arch Intern Med. 2002;162(15):1722-1728. 5.
Yu W. End of Life Care: Medical Treatments and Costs by Age, Race, and Region.
HSRD Study IIR 02-189. https://www.hsrd.research.va.gov/research/abstracts/IIR_02-189.htm. Accessed November 12, 2018. 6.
Tangka FKL, Subramanian S, Sabatino SA, et al. End-of-Life Medical Costs of Medicaid
Cancer Patients. Health Serv Res. 2015;50(3):690-709. doi:10.1111/1475-6773.12259 7.
Murphy JD, Nelson LM, Chang DT, Mell LK, Le Q-T. Patterns of Care in Palliative
Radiotherapy: A Population-Based Study. J Oncol Pract. 2013;9(5):e220-e227. doi:10.1200/JOP.2012.000835 8.
Park KR, Lee CG, Tseng YD, et al. Palliative radiation therapy in the last 30 days of life:
A systematic review. Radiother Oncol. 2017;125(2):193-199. doi:10.1016/j.radonc.2017.09.016 9.
van den Hout WB, van der Linden YM, Steenland E, et al. Single- versus multiple-
fraction radiotherapy in patients with painful bone metastases: cost-utility analysis based on a randomized trial. J Natl Cancer Inst. 2003;95(3):222-229.
15
10.
Yarnold JR. 8 Gy single fraction radiotherapy for the treatment of metastatic skeletal
pain: randomised comparison with a multifraction schedule over 12 months of patient followupOn behalf of the Bone Pain Trial Working Party. Radiother Oncol. 1999;52(2):111-121. doi:10.1016/S0167-8140(99)00097-3 11.
Roos DE, Turner SL, O’Brien PC, et al. Randomized trial of 8 Gy in 1 versus 20 Gy in 5
fractions of radiotherapy for neuropathic pain due to bone metastases (Trans-Tasman Radiation Oncology Group, TROG 96.05). Radiother Oncol J Eur Soc Ther Radiol Oncol. 2005;75(1):5463. doi:10.1016/j.radonc.2004.09.017 12.
Hartsell WF, Scott CB, Bruner DW, et al. Randomized trial of short- versus long-course
radiotherapy for palliation of painful bone metastases. J Natl Cancer Inst. 2005;97(11):798-804. doi:10.1093/jnci/dji139 13.
Chow E, Harris K, Fan G, Tsao M, Sze WM. Palliative radiotherapy trials for bone
metastases: a systematic review. J Clin Oncol Off J Am Soc Clin Oncol. 2007;25(11):1423-1436. doi:10.1200/JCO.2006.09.5281 14.
Bezjak A, Dixon P, Brundage M, et al. Randomized phase III trial of single versus
fractionated thoracic radiation in the palliation of patients with lung cancer (NCIC CTG SC.15). Int J Radiat Oncol Biol Phys. 2002;54(3):719-728. 15.
Lutz ST, Jones J, Chow E. Role of Radiation Therapy in Palliative Care of the Patient
With Cancer. J Clin Oncol. 2014;32(26):2913-2919. doi:10.1200/JCO.2014.55.1143 16.
Glare P, Virik K, Jones M, et al. A systematic review of physicians’ survival predictions
in terminally ill cancer patients. BMJ. 2003;327(7408):195.
16
17.
How Accurate are Physicians’ Clinical Predictions of Survival and the Available
Prognostic Tools in Estimating Survival Times in Terminally III Cancer Patients? A Systematic Review. Clin Oncol. 2001;13(3):209-218. doi:10.1053/clon.2001.9256 18.
Christakis NA, Lamont EB. Extent and determinants of error in doctors’ prognoses in
terminally ill patients: prospective cohort study. BMJ. 2000;320(7233):469-472. 19.
Chow E, Davis L, Panzarella T, et al. Accuracy of survival prediction by palliative
radiation oncologists. Int J Radiat Oncol • Biol • Phys. 2005;61(3):870-873. doi:10.1016/j.ijrobp.2004.07.697 20.
Hartsell WF, Desilvio M, Bruner DW, et al. Can Physicians Accurately Predict Survival
Time in Patients with Metastatic Cancer? Analysis of RTOG 97-14. J Palliat Med. 2008;11(5):723-728. doi:10.1089/jpm.2007.0259 21.
Fairchild A, Debenham B, Danielson B, Huang F, Ghosh S. Comparative
multidisciplinary prediction of survival in patients with advanced cancer. Support Care Cancer Off J Multinatl Assoc Support Care Cancer. 2014;22(3):611-617. doi:10.1007/s00520-013-20132 22.
Kondziolka D, Parry PV, Lunsford LD, et al. The accuracy of predicting survival in
individual patients with cancer. J Neurosurg. 2014;120(1):24-30. doi:10.3171/2013.9.JNS13788 23.
Gripp S, Moeller S, Bölke E, et al. Survival prediction in terminally ill cancer patients by
clinical estimates, laboratory tests, and self-rated anxiety and depression. J Clin Oncol Off J Am Soc Clin Oncol. 2007;25(22):3313-3320. doi:10.1200/JCO.2006.10.5411 24.
Mackillop WJ, Quirt CF: Measuring the accuracy of prognostic judgments in oncology. J
Clin Epidemiol 50:21-9, 1997
17
25.
Oxenham D, Cornbleet MA. Accuracy of prediction of survival by different professional
groups in a hospice. Palliat Med. 1998;12(2):117-118. doi:10.1191/026921698672034203 26.
Mackillop WJ, Quirt CF. Measuring the accuracy of prognostic judgments in oncology. J
Clin Epidemiol. 1997;50(1):21-29. 27.
Lambden J, Zhang B, Friedlander R, Prigerson HG. Accuracy of Oncologists’ Life-
Expectancy Estimates Recalled by Their Advanced Cancer Patients: Correlates and Outcomes. J Palliat Med. 2016;19(12):1296-1303. doi:10.1089/jpm.2016.0121 28.
O’Keefe EB, Meltzer JP, Bethea TN. Health Disparities and Cancer: Racial Disparities in
Cancer Mortality in the United States, 2000–2010. Front Public Health. 2015;3. doi:10.3389/fpubh.2015.00051 29.
Wong SL. Medically Underserved Populations: Disparities in Quality and Outcomes. J
Oncol Pract. 2015;11(3):193-194. doi:10.1200/JOP.2015.004259 30.
Disparities in Cancer Care. J Oncol Pract. 2006;2(5):234-239.
31.
Tseng YD, Krishnan MS, Sullivan AJ, Jones JA, Chow E, Balboni TA. How Radiation
Oncologists Evaluate and Incorporate Life Expectancy Estimates Into the Treatment of Palliative Cancer Patients: A Survey-Based Study. Int J Radiat Oncol Biol Phys. 2013;87(3):471-478. doi:10.1016/j.ijrobp.2013.06.2046 32.
Gwilliam B, Keeley V, Todd C, et al. Prognosticating in patients with advanced cancer--
observational study comparing the accuracy of clinicians’ and patients’ estimates of survival. Ann Oncol Off J Eur Soc Med Oncol. 2013;24(2):482-488. doi:10.1093/annonc/mds341 33.
Bruera E, Miller MJ, Kuehn N, MacEachern T, Hanson J. Estimate of survival of patients
admitted to a palliative care unit: a prospective study. J Pain Symptom Manage. 1992;7(2):82-86.
18
33.
Banerjee I, Gensheimer MF, Wood DJ, et al. Probabilistic Prognostic Estimates of
Survival in Metastatic Cancer Patients (PPES-Met) Utilizing Free-Text Clinical Narratives. Sci Rep. 2018;8. doi:10.1038/s41598-018-27946-5 34.
Palma DA, Olson R, Harrow S, et al: Stereotactic ablative radiotherapy versus standard
of care palliative treatment in patients with oligometastatic cancers (SABR-COMET): a randomised, phase 2, open-label trial. Lancet 393:2051-2058, 2019 35.
Gomez DR, Blumenschein GR, Jr., Lee JJ, et al: Local consolidative therapy versus
maintenance therapy or observation for patients with oligometastatic non-small-cell lung cancer without progression after first-line systemic therapy: a multicentre, randomised, controlled, phase 2 study. Lancet Oncol 17:1672-1682, 2016 36.
Nguyen QN, Chun SG, Chow E, et al: Single-Fraction Stereotactic vs Conventional
Multifraction Radiotherapy for Pain Relief in Patients With Predominantly Nonspine Bone Metastases: A Randomized Phase 2 Trial. JAMA Oncol 5:872-878, 2019 37.
Krishnan MS, Epstein-Peterson Z, Chen YH, et al: Predicting life expectancy in patients
with metastatic cancer receiving palliative radiotherapy: the TEACHH model. Cancer 120:13441, 2014 38.
McMillan DC: The systemic inflammation-based Glasgow Prognostic Score: a decade of
experience in patients with cancer. Cancer Treat Rev 39:534-40, 2013 39.
Glare PA, Eychmueller S, McMahon P: Diagnostic accuracy of the palliative prognostic
score in hospitalized patients with advanced cancer. J Clin Oncol 22:4823-8, 2004
19
Figure Legends Figure 1. Significant Variable Interactions for Attending Predictions for (A) Two-way interactions of age and primary site and (B) Two-way interactions of age and race.
20
Tables Table 1. Patient Characteristics Variable
Count
Primary
CNS
Site
Insurance
Marital
Number of
Number of
surveys
patients
(N= 877)
(N=689)* 3 (0.3%)
2 (0.3%)
GI
140 (16%)
120 (17%)
GU
92 (10%)
78 (11%)
H&N
49 (6%)
44 (6%)
Lymphoma
26 (3%)
23 (3%)
Sarcoma
38 (4%)
29 (4%)
GYN
54 (6%)
42 (6%)
Lung
275 (31%)
202 (29%)
Breast
128 (15%)
92 (13%)
Skin
39 (4%)
35 (5%)
Other
33 (4%)
22 (3%)
Private
396 (45%)
295 (43%)
Public
387 (44%)
316 (46%)
VA
88 (10%)
74 (11%)
None†
6 (0.7%)
6 (0.9%)
589 (67%)
455 (66%)
Married
162 (18.5%)
129 (19%)
Widowed
54 (6%)
42 (6%)
Divorced
56 (6%)
50 (7%)
Missing†
16 (2%)
14 (2%)
<=50
73 (8%)
69 (10%)
60-70
256 (29%)
235 (34%)
548 (62.5%)
456 (66%)
White
464 (53%)
374 (54%)
Black
34 (4%)
28 (4%)
Asian
204 (23%)
153 (22%)
96 (11%)
77 (11%)
79 (9%)
59 (9%)
Female
460 (52.5%)
347 (50%)
Male
417 (47.5%)
342 (50%)
Experience <10 years
359 (41%)
287 (42%)
of
10-20 years
302 (34%)
243 (35%)
Attending
>20 years
216 (25%)
191 (28%)
Fractions
SABR (1-5
431 (49%)
344 (50%)
Prescribed
fractions) 50 (6%)
49 (7%)
CFRT 2-5 fractions
115 (13%)
108 (16%)
CFRT >5 fractions
281 (32%)
266 (39%)
Status
KPS
Single
80-100 Race
Hispanic Other Gender
(years)
CFRT 1 fraction
Age
Patient OS
<65
447 (51%)
335 (49%)
65 +
430 (49%)
354 (51%)
0-6 months
336 (38%)
289 (42%)
>6-12 months
168 (19%)
154 (22%)
>12-24 months
153 (17%)
131 (19%)
>24 months
220 (25%)
187 (27%)
Abbreviations: KPS, Karnofsky performance status; OS, observed survival; CNS, central nervous system; GI, gastrointestinal; GU, genitourinary; H&N, head and neck; GYN, gynecological; SABR, stereotactic ablative radiotherapy; CFRT, conventionally fractionated radiotherapy. *Many of the number of patients (N) for each variable will not sum to the 689 unique patients analyzed because a patient could be counted in multiple categories per variable †Excluded from Table 2 and analyses because limited number
Table 2. Physician Accuracy and Survival Statistics by Patient and Disease Characteristics Variable
N under
N accurate
N over
(%)
(%)
(%)
Median Follow up
Range Follow up
Primary Site
CNS
1 (33%)
1 (33%)
1 (33%)
3.0
[2.0, 10.0]
(p=0.1644)*
GI
42 (30%)
67 (48%)
31 (22%)
9.0
[0.0, 38.0]
GU
29 (32%)
34 (37%)
29 (32%)
11.5
[0.0, 38.0]
H&N
14 (30%)
18 (38%)
15 (32%)
9.0
[0.0, 38.0]
Lymphoma
6 (23%)
10 (38%)
10 (38%)
11.0
[0.0, 39.0]
Sarcoma
8 (21%)
17 (45%)
13 (34%)
6.5
[0.0, 38.0]
GYN
19 (35%)
22 (41%)
13 (24%)
7.0
[0.0, 38.0]
Lung
110 (40%)
102 (37%)
63 (23%)
11.0
[0.0, 39.0]
Breast
45 (35%)
47 (37%)
36 (28%)
12.0
[0.0, 38.0]
Unknown
6 (18%)
19 (58%)
8 (24%)
6.0
[1.0, 35.0]
Skin
15 (38%)
11 (28%)
13 (33%)
9.0
[0.0, 34.0]
Insurance
Private
141 (36%)
165 (42%)
90 (23%)
12.0
[0.0, 39.0]
(p=0.2946)*
Public
123 (32%)
150 (39%)
114 (30%)
9.0
[0.0, 39.0]
VA
31 (35%)
29 (33%)
28 (32%)
7.0
[0.0, 38.0]
Marital
Married
196 (33%)
241 (41%)
152 (26%)
10.0
[0.0, 39.0]
Status
Single
53 (33%)
70 (43%)
39 (24%)
11.0
[0.0, 39.0]
(p=0.0786)*
Widowed
22 (41%)
14 (26%)
18 (33%)
11.0
[0.0, 38.0]
Divorced
20 (36%)
18 (32%)
18 (32%)
9.0
[0.0, 35.0]
KPS
<=50
11 (15%)
50 (68%)
12 (16%)
2.0
[0.0, 38.0]
(p<0.0001)*
60-70
62 (24%)
114 (45%)
80 (31%)
5.0
[0.0, 38.0]
80-100
224 (41%)
184 (34%)
140 (26%)
15.0
[0.0, 39.0]
Race
White
156 (34%)
173 (37%)
135 (29%)
10.0
[0.0, 39.0]
(p=0.3114)*
Black
13 (38%)
13 (38%)
8 (24%)
8.5
[0.0, 39.0]
Asian
70 (34%)
91 (45%)
43 (21%)
10.0
[0.0, 39.0]
Hispanic
34 (35%)
35 (36%)
27 (28%)
13.5
[0.0, 39.0]
Other
24 (30%)
36 (46%)
19 (24%)
10.0
[1.0, 36.0]
Gender
Female
154 (33%)
188 (41%)
118 (26%)
11.0
[0.0, 38.0]
(p=0.4498)*
Male
143 (34%)
160 (38%)
114 (27%)
9.0
[0.0, 39.0]
Experience
< 10 years
98 (27%)
160 (45%)
101 (28%)
9.0
[0.0, 39.0]
of Attending
10-20 years
140 (46%)
99 (33%)
63 (21%)
10.0
[0.0, 39.0]
(years)
>20 years
59 (27%)
89 (41%)
68 (31%) 12.0
[0.0, 38.0]
13.0
[0.0, 39.0]
2.5
[0.0, 38.0]
5.0
[0.0, 38.0]
9.0
[0.0, 39.0]
(p<0.0001)* Fractions
SABR (1-5
Prescribed
fractions)
(p<0.0001)*
CFRT 1
181 (42%)
11 (22%)
132 (31%)
31 (62%)
118 (27%)
8 (16%)
fraction CFRT 2-5
25 (22%)
66 (57%)
24 (21%)
fractions CFRT >5
80 (28%)
119 (42%)
82 (29%)
fractions Age
<65
161 (36%)
187 (42%)
99 (22%)
12.0
[0.0, 39.0]
(p=0.1838)*
65 +
136 (32%)
161 (37%)
133 (31%)
9.0
[0.0, 39.0]
Patient OS
0-6 months
0 (0%)
169 (50%)
167 (50%)
3.0
[0.0, 6.0]
(p<0.0001)*
>6-12
42 (25%)
74 (44%)
52 (31%) 9.0
[7.0, 12.0]
17.0
[13.0, 24.0]
months >12-24
90 (59%)
50 (33%)
13 (9%)
months >24
165 (75%)
55 (25%)
0 (0%)
† [25.0, 39.0] months
Abbreviations: KPS, Karnofsky performance status; OS, observed survival; CNS, central nervous system; GI, gastrointestinal; GU, genitourinary; H&N, head and neck; GYN,
gynecological; SABR, stereotactic ablative radiotherapy; CFRT, conventionally fractionated radiotherapy. *p-values assessing difference in percentages of accurate estimates within variable subcategories †No median available because many patients with observed survival >24 months are still living
Table 3. Univariable and Multivariable Models for Accuracy of Physician Survival Predictions Accuracy Effect
Univariable
Multivariable
Models (p-values)
Model (p-values)
Insurance
0.2329
0.7604
Race
0.3114
0.0648
Marital Status
0.0786
0.0428
< 0.0001
<.0001
Gender
0.4498
0.3058
Age
0.1824
0.6016
Fractions Prescribed
0.4359
0.3302
Experience Attending
0.5988
0.7240
Primary Site
0.1806
0.3170
KPS
Abbreviations: KPS, Karnofsky performance status
Table 4. Univariable, Multivariable, and Parsimonious Multivariable Models for Physician Survival Prediction Outcomes, Including Variable Interactions Univariable and Multivariable Models Effect
Univariable
Multivariable
Models (p-values)
Model (p-values)
Insurance
0.1595
0.5251
Race
0.7519
0.7971
Marital Status
0.9842
0.9267
KPS
0.0003
0.0003
Gender
0.8879
0.6759
Age
0.1292
0.2183
Fractions (Fx) Prescribed
0.0350
0.2130
Experience Attending
0.0296
0.1929
Primary Site
0.2454
0.3335
Multivariable Parsimonious Model with Variable Interactions Effect
Parsimonious Multivariable Model (p-values)
KPS
0.0005
Primary Site
0.0063
Interaction of Age and Primary Site
0.0200
Age
0.1071
Race
0.0010
Interaction of Age and Race
0.0005
Abbreviations: KPS, Karnofsky performance status
Genitourinary (n=92)
Lung (n=275)
Underestimate
0.8 Accurate
Accurate 0.8
0.6
Accurate
Overestimate
Overestimate
Overestimate
0.4 0.2
Predicted Probabilities
Underestimate
Predicted Probabilities
Predicted Probabilities
0.8
1.0
Underestimate
0.6 0.4 0.2 0.0
0.0 10
20
30
40
50
60
70
80
20
30
Breast (n=128)
40
60
70
80
90 100
10
0.6 0.4 0.2
60
70
80
Accurate
Overestimate
Overestimate
Overestimate
0.6 0.4 0.2
20
30
40
50
60
70
80
40
50
60
Age
0.2
90 100
10
20
30
40
70
80
90 100
50
60
70
80
90 100
Age
Sarcoma (n=38) 1.0
Underestimate
Underestimate
0.8 Accurate
0.8 Accurate
Accurate
Overestimate
Overestimate
Overestimate
Predicted Probabilities
Underestimate
0.6 0.4 0.2 0.0
30
0.4
0.0 10
Predicted Probabilities
0.0 20
0.6
Gastrointestinal (n=140)
0.2
90 100
Underestimate
1.0
0.4
80
0.8 Accurate
Head and Neck (n=49)
0.6
70
Gynecologic (n=54)
Age
0.8
60
Underestimate
90 100
1.0
50
0.8 Accurate
Age
10
40
1.0
0.0 50
30
Underestimate
0.0 40
20
Age
Predicted Probabilities
0.8
30
0.2
Skin (n=39)
Predicted Probabilities
Predicted Probabilities
50
1.0
20
0.4
Age
1.0
10
0.6
0.0 10
90 100
Age
Predicted Probabilities
Lymphoma (n=26)
1.0
1.0
0.6 0.4 0.2 0.0
10
20
30
40
50
60
Age
70
80
90 100
10
20
30
40
50
60
70
80
90 100
Age
Figure 1. Significant Variable Interactions for Attending Predictions (continued) Figure 1B. Two-way interactions of age and race