Predicting Nonmuscle Invasive Bladder Cancer Recurrence and Progression in a United States Population

Author's Accepted Manuscript Predicting non-muscle invasive bladder cancer recurrence and progression in a US population Kourosh Ravvaz , Marcus E. Walz , John A. Weissert , Tracy M. Downs

PII: DOI: Reference:

S0022-5347(17)54785-0 10.1016/j.juro.2017.04.077 JURO 14708

To appear in: The Journal of Urology Accepted Date: 12 April 2017 Please cite this article as: Ravvaz K, Walz ME, Weissert JA, Downs TM, Predicting non-muscle invasive bladder cancer recurrence and progression in a US population, The Journal of Urology® (2017), doi: 10.1016/j.juro.2017.04.077. DISCLAIMER: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our subscribers we are providing this early version of the article. The paper will be copy edited and typeset, and proof will be reviewed before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to The Journal pertain.

Embargo Policy All article content is under embargo until uncorrected proof of the article becomes available online. We will provide journalists and editors with full-text copies of the articles in question prior to the embargo date so that stories can be adequately researched and written. The standard embargo time is 12:01 AM ET on that date. Questions regarding embargo should be directed to [email protected].

ACCEPTED MANUSCRIPT

Predicting non-muscle invasive bladder cancer recurrence and

RI PT

progression in a US population Running head:

M AN U

Authors Kourosh Ravvaza, MD, PhD, MPH Marcus E Walza, BS John A Weisserta, BA Tracy M Downsb,c, MD

SC

Comparing EORTC, CUETO models with NCCN guidelines

Affiliations aAurora Research Institute, Aurora Health Care, Milwaukee, WI, USA; bDepartment of Urology, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA; cUniversity of Wisconsin Carbone Comprehensive Cancer Center, Madison, WI, USA

TE D

Emails for authors: Kourosh Ravvaz: [email protected] Marcus E Walz: [email protected] John A Weissert: [email protected] Tracy M Downs: [email protected]

EP

Keywords: Bladder cancer, CUETO, EORTC, NCCN, risk prediction

AC C

Word count of Abstract: 249

Word count of text (including abstract but not references): 2494

1

ACCEPTED MANUSCRIPT

Abstract Purpose: To assess performance of The European Organization for Research and Treatment of Cancer (EORTC) and Club Urologico Español de Tratamiento Oncologico (CUETO) non-muscle

RI PT

invasive bladder cancer (NMIBC) predictive models compared to current US National Comprehensive Cancer Network (NCCN) treatment guidelines in a US population.

SC

Materials and Methods: Retrospective analysis using electronic medical records of patients with NMIBC within a multicenter US patient population. We evaluated recurrence-free

M AN U

survival and progression-free survival according to EORTC and CUETO, and assessed discriminative performance with concordance index (c-index) at one and five years. We then compared the discrimination of EORTC and CUETO to the discrimination of the four NMIBC treatment groups described in NCCN guidelines.

TE D

Results: We identified 1333 patients with NMIBC and a median follow-up of 37months. At five years, the c-indices of EORTC and CUETO for recurrence were 0.59 and 0.56, respectively, while for progression they were higher at 0.74 and 0.72, respectively. NCCN guidelines

EP

demonstrated similar c-indices of 0.56 and 0.75, respectively. The discrimination of all three risk models diminished in patients receiving bacillus Calmette-Guerin (BCG). EORTC was

AC C

better able to identify patients at low risk of recurrence or progression but overestimated five-year risk of progression in high-risk patients. This study is limited by its retrospective design.

Conclusion: This work illustrates the need for improved predicative tools for clinicians treating NMIBC patients. However, until new tools are developed, NCCN guidelines are a 2

ACCEPTED MANUSCRIPT

simple option for clinicians treating NMIBC patients that provide comparable predictive

AC C

EP

TE D

M AN U

SC

RI PT

power to the EORTC and CUETO models.

3

ACCEPTED MANUSCRIPT

1.

Introduction

Non-muscle invasive bladder cancer (NMIBC) accounts for ~70% of new bladder cancer cases.1 NMIBC is highly recurrent, and patients require frequent follow-up with lifetime

RI PT

monitoring and care, causing bladder cancer to be one of the most costly malignancies.2 In 2006, the European Organization for Research and Treatment of Cancer (EORTC) developed two risk models to predict one- and five-year probability of NMIBC recurrence and

SC

progression after transurethral resection of bladder tumor (TURBT).3 Club Urologico Español

M AN U

de Tratamiento Oncologico (CUETO) later published two similar models specifically for patients receiving bacillus Calmette-Guerin (BCG) immunotherapy.4 EORTC and CUETO risk models stratify patients into four risk groups (low, intermediate-low, intermediate-high, high) based on retrospective analysis of clinical trial data. EORTC has since been incorporated into

TE D

European Association of Urology bladder cancer guidelines.3,5

Although EORTC and CUETO models were externally validated in international populations, neither model has been validated on an entirely U.S. population. In contrast to EORTC and

EP

CUETO, the National Comprehensive Cancer Network (NCCN) guidelines for patients with NMIBC use a simple combination of tumor stage (Ta, T1, or Tis) and histologic grade (high or

AC C

low) to determine treatment strategy and follow-up approaches. Both tumor stage and grade are correlated to risk of cancer progression and recurrence, respectively.3,6 Accordingly, NCCN guidelines offer four treatment pathways: Low-grade Ta tumor (LGTa), high-grade Ta tumor (HGTa), T1 tumor of either high or low-grade (T1), or any carcinoma in-situ (Tis).1 However, the NCCN NMIBC guidelines have not provided both one- and five-year risk estimates of 4

ACCEPTED MANUSCRIPT

recurrence and progression associated with the four treatment pathways.7 One and five-year risk estimates for both recurrence and progression associated with NCCN treatment pathways are important for patient counseling, determining follow-up schedules, and guiding

RI PT

treatments.

In this study we examined risk estimates of one- and five-year NMIBC recurrence and

SC

progression associated with the four NCCN treatment pathways. We then compared the discriminative ability of the four treatment groups delineated by NCCN guidelines to the four

M AN U

prognostic groups specified by each of the more complex EORTC and CUETO recurrence and progression models. We analyzed EORTC and CUETO’s ability to discriminate the population into prognostic risk groups and accurately predict one- and five-year risk of bladder cancer recurrence and progression in this study’s heterogeneous NMIBC population. In doing so, we

TE D

assessed the performance of the EORTC and CUETO predictive models compared to current NCCN treatment guidelines.

Materials and Methods

2.1

Study population

EP

2.

AC C

This study population received care for bladder cancer between 2006 and 2016 within Aurora Health Care (Aurora)—a network of 15 hospitals and ~150 clinics serving 1.2 million patients/year in Eastern Wisconsin and Northern Illinois. This study was approved by Aurora’s Institutional Review Board.

5

ACCEPTED MANUSCRIPT

We included patients with NMIBC and a primary or recurrent diagnosis of Ta or T1 transitional cell carcinoma. Patients were excluded if sex was unknown. Patients without a record of index TURBT and associated initial pathology report (as defined in supplemental

RI PT

materials) within 90 days following the first record of bladder cancer diagnosis were excluded (Figure 1). Data sources

SC

2.2

We extracted patient demographics, cancer diagnosis, recurrence, and treatment from the

M AN U

Aurora cancer registry. Cancer registry data was complemented with electronic medical records (EMR) including surgery and pathology reports, medication orders, lab tests, procedure codes, and clinical and billing diagnoses (Supplemental materials). 2.3

Outcome measures

TE D

In accordance with EORTC and CUETO, we measured time from index TURBT to first cancer recurrence or progression to muscle invasive bladder cancer. Bladder cancer recurrence and progression were identified using pathology reports and dates of recurrence or progression

EP

listed within the cancer registry. Patients without record of recurrence or progression were censored upon cystectomy or last known bladder-cancer–directed treatment—a

2.4

AC C

cystourethroscopy, TURBT, urologist visit, BCG, chemo instillation, or urine cytology test. Statistical analysis

EORTC and CUETO risk scores for recurrence and progression were calculated for each patient and then stratified into one of four risk groups by their score. Additionally, we 6

ACCEPTED MANUSCRIPT

stratified the population into four groups based on NCCN treatment pathways: Low Grade Ta (low), High Grade Ta (intermediate-low), T1 (intermediate-high), or Any Tis (high). Any patient with concomitant carcinoma in-situ was placed in the Any Tis group. Patients with

RI PT

missing tumor sizes were included in the EORTC analysis only if their risk group would not change by tumor size.

SC

We measured discrimination for each model using the concordance measurement (c-index) described in Wolbers et al in order to account for competing risks.8 The c-indices were

M AN U

calculated at one and five years using the entire population and the subpopulation who received BCG within 90 days of study recruitment. For each risk group, the one- and five-year cumulative incident recurrence and progression were calculated by accounting for death as a competing risk. These incidents were compared to their corresponding predictions made by

TE D

the EORTC and CUETO through the use of calibration plots. NCCN has not published one- and five-year risk estimates for recurrence or progression with standard errors; thus, we could not assess its calibration. The prognostic separation index (PSEP) was measured at one and

EP

five years by subtracting the probability of recurrence or progression for the high-risk group from that of the low-risk group.9 A higher PSEP indicates greater prognostic stratification

AC C

between high- and low-risk patients We have also conducted decision curve analysis (details in supplemental materials).10,11 All analyses were conducted in R 3.2.3.12

7

ACCEPTED MANUSCRIPT

3.

Results

We identified 2086 patients with bladder cancer from 2006 -2016. After applying inclusion and exclusion criteria, the study population included 1333 patients with NMIBC (Figure 1).

RI PT

Median follow-up was 37 months, with a maximum follow-up of 10.9 years. Median age in the study population was 72 years old (Table 1). Treatments stratified by tumor stage and tumor grade are presented in the supplemental materials. Within our population 17.7% underwent

SC

restaging TURBT, with 33.4% of HGT1 patients receiving restaging TURBT. Twenty-one

M AN U

percent of the population received BCG induction therapy within 90 days of TURBT and 30% received perioperative chemotherapy (Table 1). 3.1

Recurrence

In this study population, 573 patients experienced a recurrence. The cumulative probability of

TE D

recurrence is 28% (95% CI: 26–31) at one year and 47% (95% CI: 44–50) at five years. EORTC had marginally higher discrimination for predicting recurrence, with a c-index of 0.63 at one year and 0.59 at five years for the whole study population (Table 2). However, NCCN

EP

guidelines provided similar discrimination for recurrence. In patients treated with BCG, NCCN

years.

AC C

had the greatest discrimination, with a c-index of 0.60 at one year and a c-index of 0.59 at five

EORTC had the greatest prognostic separation between low- and high-risk groups (Figure 2). EORTC’s lowest risk group had a 10% (95% CI: 5.9–14) probability of recurrence in our cohort at one year, which is lower than the low-risk group in CUETO of 22% (95% CI: 19–25) 8

ACCEPTED MANUSCRIPT

or NCCN of 20% (95% CI: 17–23) (Tables 3, 4). The PSEP was 0.36, 0.20, and 0.20 at one year and decreased to 0.26, 0.10, and 0.14 at five years for EORTC, CUETO, and NCCN recurrence,

RI PT

respectively (Table 5). The calibration for EORTC shows acceptable risk stratification at one year, but at five years, EORTC overestimates recurrence in our patients in the intermediate-high and high-risk

SC

groups (Table 3, Figure 3). CUETO underestimates the risk of recurrence at one year and five years for the three lower risk study groups. NCCN risk estimates are presented in Table 4. Progression

M AN U

3.2

Sixty-two patients in the study population progressed to muscle invasive disease. Based on cumulative incidence estimates, the cumulative probability of progression is 2.4% (95% CI:

TE D

1.5–3.2) at one year and 5.3% (95% CI: 4.0–6.7) at five years.

NCCN demonstrated a slightly higher c-index of 0.82 at one year and 0.75 at five years across the whole study population (Table 2) compared to EORTC with c-indices of 0.79 and 0.74 and

EP

compared with CUETO with c-indices of 0.79 and 0.72 at one year and five year respectively. The c-index was lower in patients treated with BCG for all three models; NCCN maintained the

AC C

highest c-index of 0.73 at one year and 0.71 at five years. EORTC has the greatest prognostic separation between patients at low and high risk for progression at five years (Figure 2). PSEP is 0.09, 0.07, and 0.09 at one year and 0.16, 0.11, and 0.15 at five years for EORTC, CUETO, and NCCN progression models, respectively (Table 5). The one-year probability of progression reported by EORTC and CUETO are similar to the 9

ACCEPTED MANUSCRIPT

probabilities observed in Aurora’s population (Table 3, Figure 3). Both EORTC and CUETO overestimate the five-year risk of progression. The probability of progression based on NCCN

4.

RI PT

risk groups are presented in Table 4. Discussion

Risk stratifying patients at the time of diagnosis is critical for guiding treatments and assisting

SC

in patient counseling.3 To date, the EORTC and CUETO risk models have not been evaluated in a US patient population. Furthermore, the NCCN guidelines have not published risk estimates

M AN U

to assist in guiding treatments and aid in the counsel of patients. Our study provides the first external validation of the EORTC and CUETO models for a US patient population from multiple centers of care and compares their discriminative ability to tumor grade and stage alone per NCCN treatment guidelines.

TE D

Like prior external validations in European and Asian populations, we found that EORTC and CUETO demonstrated poor discriminative ability to predict NMIBC recurrence. 13-15 Five-year c-indices were 0.59 and 0.56 for EORTC and CUTEO recurrence models, respectively. NCCN

EP

demonstrated similar discriminative ability for predicting recurrence, with a five-year c-index of 0.57. These results are consistent with a similar multicenter validation with c-indices of

AC C

0.60 and 0.52 for EORTC and CUETO recurrence models, respectively.14 For patients treated with BCG, discrimination diminished in all three recurrence models. CUETO had no discriminatory advantage on the BCG-treated subpopulation compared to EORTC and NCCN.

10

ACCEPTED MANUSCRIPT

EORTC’s recurrence model was more accurate during calibration testing (Table 3, Figure 3) compared to CUETO, particularly at one year. We assessed calibration on the entire population rather than an exclusively BCG-treated population, which could partly explain

RI PT

CUETO’s underestimation of recurrence in the three lower risk groups. One and five-year risk estimates provided by EORTC provide accurate estimation of risk for recurrence in this population and these estimates could be consulted when considering patient counseling. Of

SC

note, EORTC’s lowest risk group, those with solitary Ta tumors <3cm in size, had the lowest

M AN U

absolute risk for recurrence compared to NCCN or CUETO.

All three models demonstrated good ability to discriminate progression. Five-year c-indices were 0.74, 0.72, and 0.75 for EORTC, CUETO, and NCCN, respectively. The c-indices of all three models are similar to the five year c-indices of 0.74 and 0.70 reported in the original

TE D

EORTC and CUETO publications.3,4 However, all three progression models—including CUETO, which was designed for BCG patients—demonstrated worse predictive power in BCG patients. Neither EORTC nor CUETO demonstrated accurate risk estimation during progression model

EP

calibration. EORTC predicted 45% of high-risk patients would experience a progression by 5 years, while CUETO predicted 34% for the high-risk group. We observed 15.8% of patients in

AC C

the highest risk EORTC group progressed in five years and 12.7% of patient in the highest risk CUETO group progressed in five years. Our results are consistent with more recent studies of high-grade T1 patients where restaging TURBT is common practice.16,17 Several external validations have also noted that EORTC overestimates risk for this group, but this study demonstrates the greatest overestimation.13,18,19 Given this evidence, neither EORTC nor 11

ACCEPTED MANUSCRIPT

CUETO risk estimates accurately reflect the probability of progression for high-risk patients in our cohort and therefore should not be consulted for patient counseling.

RI PT

Advances in NMIBC management, particularly restaging TURBT and postoperative intravesical treatment, likely reduced risk of progression since 1989 when the most recent EORTC trial stopped enrolling patients.20 In this study, restaging TURBT identified muscle

SC

invasion in twenty-seven patients and were excluded. Under the EORTC protocol, these patients would have been considered an early progression. Conversely, low progression rates

M AN U

could be attributed to stage migration with the WHO 2004 grading scheme, which broadens the definition of high grade.21 Furthermore, patients in our retrospective cohort are older and therefore more likely to experience mortality before they progress. Poor predictive performance of all three NMIBC models is likely multifactorial. Many

TE D

predictive variables are imprecise and have poor interobserver agreement. A previous study found tumor stage can vary by up to 38% between pathologists, and tumor grade can vary by almost 39%.22 Additionally, the rate of recurrence at first follow-up varies both by surgery

EP

center and quality of initial TURBT as measured by the presence of detrusor muscle in biopsy specimens.23,24 Three single-center validations of EORTC and CUETO have reported higher c-

AC C

indices than multicenter validations like this study.13,15,25,26 A recent predictive model for BCGtreated patients with NMIBC attempts to mitigate early variability in recurrence by creating two separate models: one model to predict recurrence within 4.5 months and another model for recurrence thereafter.27 However, internal model validation failed to demonstrate higher c-indices than the three models compared in this study for BCG patients. 12

ACCEPTED MANUSCRIPT

Compared to EORTC and CUETO populations, patients in this study were older, more likely to have a primary NMIBC diagnosis with a single Ta tumor, to experience mortality, and to have a shorter follow-up. Additionally, tumors were graded under the WHO 2004 criteria in this

RI PT

study population, which when mapped to the WHO 1973 system used in the EORTC and CUETO clinical trials, resulted in no G2 tumors. Fewer patients were evaluated under the EORTC model than CUETO and NCCN because 19% of the study population was missing

SC

tumor size. However, the results were unaffected because we excluded patients when missing values created uncertainty in their respective risk group. This study is limited by its

M AN U

retrospective design and the use of multiple centers of care; this means that the follow-up and adjuvant treatment was not controlled for and could vary based on physician and patient preferences. Few progressions were observed, thus validation of the EORTC and CUETO

4.

Conclusion

TE D

progression risk models is not definitive.

These results suggest that NCCN guidelines provide equivalent risk prediction of recurrence

EP

and progression compared to the more complicated EORTC or CUETO models and thus represent a simple option for clinicians. However, patients with a solitary low-grade Ta tumor

AC C

<3 cm, identified by EORTC’s low risk group, have the lowest absolute risk of recurrence and progression. Patients with these characteristics, as well as the clinicians treating them, should be aware of their reduced risk of recurrence and progression. Critical gaps in NMIBC risk stratification remain. All three models demonstrated poor performance for predicting recurrence. Until NCCN risk estimates presented in this study are externally validated, no 13

ACCEPTED MANUSCRIPT

model provides accurate risk estimates for progression and only the EORTC model provides fair calibration of one- and five-year risk estimates for recurrence. Furthermore, discrimination of all three models diminished in patients treated with BCG. Thus, this work

5.

RI PT

illustrates an immediate need for improved predictive tools for NMIBC.

Acknowledgments

SC

This research was supported by an Aurora Health Care Cancer Care Research Award from the Aurora Research Institute, funded by the Vince Lombardi Cancer Foundation. We thank

M AN U

Aurora Health Care’s Cancer Registry team and Aurora Research Institute IT Analytics for data extraction and support; especially Mary Kissinger, Lisa Robinson, Chris Blumberg, and Douglas Carlson. Special thanks to Maharaj Singh, PhD, biostatistician, for data consultation

AC C

EP

TE D

and Sandra Kear for editing.

14

ACCEPTED MANUSCRIPT

References

RI PT

[1] NCCN Clinical Practice Guidelines in Oncology—Bladder Cancer. Version 2.2016. Cancer Network. https://www.nccn.org/professionals/physician_gls/pdf/bladder.pdf. Accessed March 6, 2016. [2] Yeung C, Dinh T, Lee J. The health economics of bladder cancer: an updated review of the published literature. Pharmacoeconomics. 2014;32(11):1093-104.

SC

[3] Sylvester RJ, van der Meijden AP, Oosterlinck W, et al. Predicting recurrence and progression in individual patients with stage Ta T1 bladder cancer using EORTC risk tables: a combined analysis of 2596 patients from seven EORTC trials. Eur Urol. 2006;49(3):466-5; discussion 475-7.

M AN U

[4] Fernandez-Gomez J, Madero R, Solsona E, et al. Predicting nonmuscle invasive bladder cancer recurrence and progression in patients treated with bacillus calmette-guerin: the CUETO scoring model. J Urol. 2009;182(5):2195-203. [5] Babjuk M, Böhle A, Burger M, et al. EAU guidelines on non–muscle-invasive urothelial carcinoma of the bladder: update 2016. European urology. 2016 Jun 17.

TE D

[6] Fernandez-Gomez J, Solsona E, Unda M, et al. Prognostic factors in patients with non– muscle-invasive bladder cancer treated with bacillus Calmette-Guérin: multivariate analysis of data from four randomized CUETO trials. Eur Urol. 2008;53(5):992-1001. [7] Clark PE, Agarwal N, Biagioli MC, et al. Bladder cancer. J Natl Compr Canc Netw. 2013 Apr 01;11(4):446-75.

EP

[8] Wolbers M, Blanche P, Koller MT, et al. Concordance for prognostic models with competing risks. Biostatistics. 2014 Jul 1;15(3):526-39.

AC C

[9] Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19(4):453-73. [10] Vickers AJ, Elkin EB. Decision curve analysis: A novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565-74. [11] Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak. 2008;8:53. [12] R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2016. https://www.R-project.org/. 15

ACCEPTED MANUSCRIPT

[13] Xylinas E, Kent M, Kluth L, et al. Accuracy of the EORTC risk tables and of the CUETO scoring model to predict outcomes in non-muscle-invasive urothelial carcinoma of the bladder. Br J Cancer. 2013;109(6):1460-6.

RI PT

[14] Vedder MM, Márquez M, Bekker-Grob EW de, et al. Risk prediction scores for recurrence and progression of non-muscle invasive bladder cancer: an international validation in primary tumours. PLoS One. 2014;9(6):e96849. [15] Xu T, Zhu Z, Zhang X, et al. Predicting recurrence and progression in Chinese patients with nonmuscle-invasive bladder cancer using EORTC and CUETO scoring models. Urology. 2013;82(2):387-93.

SC

[16] Herr HW. Restaging transurethral resection of high risk superficial bladder cancer improves the initial response to bacillus calmette-guerin therapy. J Urol. 2005;174(6):2134-7.

M AN U

[17] Orsola A, Werner L, de Torres I, et al. Reexamining treatment of high-grade T1 bladder cancer according to depth of lamina propria invasion: A prospective trial of 200 patients. Br J Cancer. 2015;112(3):468-74. [18] Busato JWF, Almeida GL, Ribas CA, et al. EORTC risk model to predict progression in patients with non–muscle-invasive bladder cancer: Is it safe to use in clinical practice? Clin Genitourin Cancer. 2016;14(2):176-82.

TE D

[19] Fernandez-Gomez J, Madero R, Solsona E, et al. The EORTC tables overestimate the risk of recurrence and progression in patients with non–muscle-invasive bladder cancer treated with bacillus Calmette-Guérin: external validation of the EORTC risk tables. Eur Urol. 2011;60(3):423-30

EP

[20] Sylvester RJ. How well can you actually predict which non–muscle-invasive bladder cancer patients will progress? Eur Urol. 2011;60(3):431-3; discussion 433-4. [21] Grignon DJ. The current classification of urothelial neoplasms. Mod Pathol. 2009;22 Suppl 2:S60-9.

AC C

[22] Tosoni I, Wagner U, Sauter G, et al. Clinical significance of interobserver differences in the staging and grading of superficial bladder cancer. BJU Int. 2000;85(1):48-53. [23] Brausi M, Collette L, Kurth K, et al. Variability in the recurrence rate at first follow-up cystoscopy after TUR in stage Ta T1 transitional cell carcinoma of the bladder: A combined analysis of seven EORTC studies. Eur Urol. 2002;41(5):523-31. [24] Mariappan P, Finney SM, Head E, et al. Good quality white-light transurethral resection of bladder tumours (GQ-WLTURBT) with experienced surgeons performing complete resections 16

ACCEPTED MANUSCRIPT

and obtaining detrusor muscle reduces early recurrence in new non-muscle-invasive bladder cancer: validation across time and place and recommendation for benchmarking. BJU international. 2012;109(11):1666-73.

RI PT

[25] Miyake M, Gotoh D, Shimada K, et al. Exploration of risk factors predicting outcomes for primary T1 high-grade bladder cancer and validation of the Spanish Urological Club for Oncological Treatment scoring model: Long-term follow-up experience at a single institute: Risk factors of T1HG-BC. Int J Urol. 2015;22(6):541-7.

SC

[26] Choi SY, Ryu JH, Chang IH, et al. Predicting recurrence and progression of non-muscle-invasive bladder cancer in Korean patients: A comparison of the EORTC and CUETO models. Korean J Urol. 2014;55(10):643-9.

AC C

EP

TE D

M AN U

[27] Cambier S, Sylvester RJ, Collette L, et al. EORTC nomograms and risk groups for predicting recurrence, progression, and disease-specific and overall survival in non–muscle-invasive stage Ta–T1 urothelial bladder cancer patients treated with 1–3 years of maintenance bacillus Calmette-Guérin. European urology. 2016;69(1):60-9.

17

ACCEPTED MANUSCRIPT

Table Legends: Table 1- The characteristics of Aurora’s NMIBC population compared to the original EORTC and CUETO studies.

RI PT

a Prior recurrence is nonprimary if patient had a history of bladder cancer before the diagnosis date used to randomize patients. b When we translated tumor grade from the 2004 WHO system used within our study population to the 1973 WHO tumor grading system used in EORTC and CUETO models, no patients were mapped to G2 because it corresponds to both high and low grade in the 2004 system.

SC

c Prior bladder cancer history is reported by CUETO as either Primary or Recurrent—a Recurrent patient could be either (≤1 rec/y) or (>1 rec/y) in EORTC.

M AN U

Table 2- C-indices at one year and five years for EORTC, CUETO, and NCCN recurrence and progression groups on two populations: entire study population and subpopulation who received BCG within 90 days. Table 3- Cumulative incidence estimates (and 95% CI) of the probability of recurrence or progression at one and five years as predicted by EORTC and CUETO compared to the incidents observed in Aurora population for each risk group. Table 4- Cumulative incidence estimates (and 95% CI) of the probability of recurrence or progression at one and five years for NCCN groups observed in Aurora population.

Figure Legends:

EP

TE D

Table 5- The prognostic separation index (PSEP) for EORTC, CUETO, and NCCN based on cumulative incidence estimates of the probability of recurrence and progression. Greatest PSEP is bolded.

Figure 1- Study population identification process.

AC C

Figure 2- Cumulative-risk curves showing the probability of EORTC, CUETO, and NCCN recurrence and progression by risk group over time. n = number of patients assigned to each risk group. Obs. = number of recurrences or progressions observed. At Risk = number of uncensored patients that have not experienced recurrence, progression, or death at months 24, 48, 72, and 96. Figure 3 - Calibration of one- and five-year probabilities for recurrence and progression of EORTC and CUETO models. Vertical bars indicate 95% confidence interval (CI) in the study population (Aurora) and the 95% CI reported by EORTC and CUETO. 18

ACCEPTED MANUSCRIPT

Table 1:

Aurora Population N=1333

CUETO Population N=1296

Number (%)

Number (%)

221 (17) 377 (28) 735 (55)

859 (33) 890 (34) 808 (31)

404 (31) 487 (38) 405 (31)

1003 (75) 330 (25)

2044 (79) 515 (20)

951 (90) 111 (10)

1405 (55) 505 (20) 645 (25)

864 (67)

SC

RI PT

Number (%)

M AN U

1185 (89) 148 (11) 0 ( 0.0)

432 (33)c

921 (69) 311 (23) 83 (6.2) 18 (1.4)

1465 (56)

255 (10)

639 (48) 348 (27) 194 (15) 115 (8.9)

530 (40) 544 (41) 259 (19)

1087 (80) 464 (18) -

390 (40) 593 (60) -

951 (71) 382 (29)

1451 (56) 1108 (43)

251 (20) 1001 (80)

1251 (94) 82 (6.2)

2440 (90) 113 (4)

982 (93) 80 (7.5)

758 (57) 0 (0) 575 (43)

1121 (43) 1139 (44) 271 (10)

197 (16) 750 (60.0) 305 (24)

401 (30) 116 (9) 282 (21) 573 (57) 62 (4.7) 440 (33)

2025 (78) 155 (6) 1240 (52) 279 (11) 853 (33)

193 (15) 1062 (100) 346 (33) 142 (13) -

TE D

836 (32)

EP

AC C

Age <60 60-70 >70 Gender Male Female Prior recurrence ratea Primary ≤1 rec/y >1 rec/y No. of tumors 1 2-3 4-7 ≥8 Tumor Size (cm) <3 ≥3 Unknown Tumor Category Ta T1 Carcinoma in-situ No Yes Tumor Grade G1 G2b G3 Chemo Perioperative Postoperative BCG Recurrence Progression Death

EORTC Population N=2596

1

ACCEPTED MANUSCRIPT

Table 2: EORTC

CUETO

NCCN

1-yr

5-yrs

1-yr

5-yrs

1-yr

5-yr

All Pts

0.63

0.59

0.59

0.56

0.61

0.56

BCG Pts

0.57

0.53

0.56

0.57

0.60

0.59

All Pts

0.79

0.74

0.79

0.72

0.82

0.75

BCG Pts

0.71

0.69

0.64

0.61

0.73

0.71

AC C

EP

TE D

SC

M AN U

Progression

RI PT

Recurrence

1

ACCEPTED MANUSCRIPT

Table 3:

Observed

10% (5.9-14) 28% (23-32) 36% (31-40) 46% (34-56)

31% (24–37) 46% (42–49) 62% (58–65) 78% (73–84)

33% (25-39) 49% (43-54) 52% (47-57) 59% (46-69)

22% (18-25) 32% (27-37) 43% (35-49) 41% (24-55)

21% (17-24.5) 36% (29–42) 48% (40–55) 68% (54–82)

43% (39-47) 46% (41-51) 58% (50-65) 53% (34-67)

0.0% (0.0-0.0) 0.5% (0.0-1.2) 3.8% (1.8-5.7) 9.0% (3.8-14)

0.8% (0.0–1.7) 6.0% (5.0–8.0) 17% (14–20) 45% (35–55)

SC

RI PT

Predicted

0.4% (0.0-0.9) 0.0% (0.0-0.0) 4.4% (1.8-6.9) 7.2% (3.7-11)

AC C

EP

TE D

EORTC Recurrence 0 15% (10–19) 1–4 24% (21–26) 5–9 38% (35–41) 10–17 61% (55–67) CUETO Recurrence 0–4 8.2% (5.9–11) 5–6 12% (8.0–16) 7–9 25% (20–31) 10–16 42% (28–56) EORTC Progression 0 0.2% (0.0–0.7) 2–6 1.0% (0.4–1.6) 7–13 5.0% (4.0–7.0) 14–23 17% (10–24) CUETO Progression 0–4 1.2% (0.2–2.2) 5–6 3.0% (0.8–5.2) 7–9 5.6% (2.7–8.4) 10–14 14% (6.6–21)

5 Years Observed

M AN U

1 Year Predicted

1

3.8% (1.9–5.6) 12% (7.6–16) 21% (16–27) 34% (23–44)

0.0% (0.0-0.0) 2.8% (0.9-4.6) 8.6% (5.4-12) 16% (8.8-22) 1.8% (0.7-2.9) 5.8% (0.1-11) 8.5% (4.7-12) 13% (7.9-17)

-

ACCEPTED MANUSCRIPT

Table 4: Progression 1 Year 5 Years 0.2% (0.0-0.5) 1.3% (0.3-2.3) 0.9% (0.0-2.2) 4.7% (1.6-7.7) 6.3% (3.6-9.0) 11% (7.5-15) 8.8% (2.4-15) 16% (7.2-24)

AC C

EP

TE D

M AN U

SC

5 Years 42% (37-46) 52% (45-59) 51% (45-56) 56% (43-66)

RI PT

Recurrence 1 Year 20% (17-23) 33% (26-39) 40% (34-45) 39% (28-49)

NCCN Group LGTa HgTa T1 Any Tis

1

ACCEPTED MANUSCRIPT

Table 5: Recurrence 1 Year 5 Years 0.36 0.26 0.20 0.10 0.20 0.14

PSEP

AC C

EP

TE D

M AN U

SC

RI PT

EORTC CUETO NCCN

Progression 1 Year 5 Years 0.09 0.16 0.07 0.11 0.09 0.15

1

ACCEPTED MANUSCRIPT

Table Legends: Table 1- The characteristics of Aurora’s NMIBC population compared to the original EORTC and CUETO studies. a Prior

SC

RI PT

recurrence is nonprimary if patient had a history of bladder cancer before the diagnosis date used to randomize patients. b When we translated tumor grade from the 2004 WHO system used within our study population to the 1973 WHO tumor grading system used in EORTC and CUETO models, no patients were mapped to G2 because it corresponds to both high and low grade in the 2004 system. c Prior bladder cancer history is reported by CUETO as either Primary or Recurrent—a Recurrent patient could be either (≤1 rec/y) or (>1 rec/y) in EORTC.

M AN U

Table 2- C-indices at one year and five years for EORTC, CUETO, and NCCN recurrence and progression groups on two populations: entire study population and subpopulation who received BCG within 90 days. Table 3- Cumulative incidence estimates (and 95% CI) of the probability of recurrence or progression at one and five years as predicted by EORTC and CUETO compared to the incidents observed in Aurora population for each risk group. Table 4- Cumulative incidence estimates (and 95% CI) of the probability of recurrence or progression at one and five years for NCCN groups observed in Aurora population.

AC C

EP

TE D

Table 5- The prognostic separation index (PSEP) for EORTC, CUETO, and NCCN based on cumulative incidence estimates of the probability of recurrence and progression. Greatest PSEP is bolded.

1

ACCEPTED MANUSCRIPT

Figure Legends: Figure 1- Study population identification process.

RI PT

Figure 2- Cumulative-risk curves showing the probability of EORTC, CUETO, and NCCN recurrence and progression by risk group over time. n = number of patients assigned to each risk group. Obs. = number of recurrences or progressions observed. At Risk = number of uncensored patients that have not experienced recurrence, progression, or death at months 24, 48, 72, and 96.

AC C

EP

TE D

M AN U

SC

Figure 3 - Calibration of one- and five-year probabilities for recurrence and progression of EORTC and CUETO models. Vertical bars indicate 95% confidence interval (CI) in the study population (Aurora) and the 95% CI reported by EORTC and CUETO.

1

Electronic Medical Records

2,086 Pa)ents

ACCEPTED MANUSCRIPT

From Cancer Registry

SC

RI PT

Patients without transitional cell carcinoma excluded (n=66)

M AN U

Patients with unknown sex excluded (n=2)

AC C

EP

TE D

Patients without initial pathology excluded (n=378)

Tumor stage is Tis-only or muscle invasive excluded (n=265)

Patients without record of TURBT excluded (n=42)

1,333 Pa)ents

Met Inclusion Criteria

ACCEPTED MANUSCRIPT

Figure 2: Recurrence EORTC

CUETO

NCCN

100%

100%

75%

75%

75%

50%

50%

50%

25%

25%

25%

0% 24 0

0

48 1−4

n

Obs.

234

72

5−9

0%

96

0

24

10−17

48

0−4

At Risk

n

72

5−6

7−9

Obs.

96

0

10−16

At Risk

24

48

LGTa

M AN U

0

SC

0%

RI PT

100%

HGTa

n Obs.

72 T1

96 Any Tis

At Risk

64

138

61

28

8

0–4 699

268

360 163 76

28

LGTa 676

253

362 165 73

1–4

387 169

184

75

39

13

5–6 390

171

172

82

33

5

HGTa 229

115

97

50

19

5

5–9

398 194

145

71

30

9

7–9 200

112

61

33

16

8

T1 346

162

112

50

28

11

31

13

7

3

22

12

5

2

0

43

34

18

7

2

10–17

88

48

10–16

44

Any Tis

82

23

Progression CUETO

TE D

EORTC 50% 40% 30% 20%

0%

50%

40%

40%

30%

30%

20%

20%

10%

10%

EP

10%

50%

0%

0

24 2−6

72

7−13

96

n

0–1 234

Obs.

0% 0

14−23

AC C

0

48

At Risk 92

NCCN

24

48

0−4

n

0

167

43

16

0–4 748

5−6

72 7−9

96

0

10−16

24

48

LGTa

T1

96 Any Tis

Obs.

At Risk

Obs.

At Risk

11

549 307 139 57

LGTa 676

7

498 280 128 55

2–6 404

9

307 165 81

34

5–6

82

5

51

16

4

HGTa 229

10

162

90

46

12

7–13 398

28

242 118 53

16

7–9 269

19

184 100 53

20

T1 346

33

197

89

44

16

14–23 133

19

78

8

10–16 234

27

124

7

12

51

30

13

5

39

19

1

31

n

HGTa

72

51

23

Any Tis

82

ACCEPTED MANUSCRIPT

Figure 3: EORTC Recurrence

CUETO Recurrence

EORTC Progression

CUETO Progression

100% 75%

100%

SC

75% 50%

0% 0

1−4 5−9 10−17

M AN U

25%

0−4 5−6 7−9 10−16

0

2−6 7−1314−23

Risk Group CUETO

EP

TE D

Aurora

1

EORTC

0−4 5−6 7−9 10−16

5 Years

RI PT

0%

AC C

Probabilty

25%

1 Year

50%

ACCEPTED MANUSCRIPT

Key of Definitions for Abbreviations: - NCCN: US National Comprehensive Cancer Network - EORTC: The European Organization for Research and Treatment of Cancer

- TURBT: Transurethral Resection of Bladder Tumor - NMIBC: non-muscle invasive bladder cancer

SC

- BCG: bacillus Calmette-Guerin

RI PT

- CUETO: Club Urologico Español de Tratamiento Oncologico

- LGTa: low-grade Ta tumor

M AN U

- HGTa: high-grade Ta tumor - T1: T1 stage tumor of either high or low-grade - Tis: carcinoma in-situ - Aurora: Aurora Health Care

AC C

EP

TE D

- EMR: electronic medical records

ACCEPTED MANUSCRIPT

Supplemental Materials [Ravvaz et al, 2017]

2. Treatments stratified by stage and grade (Table 1A) 3. EORTC and CUETO scoring systems (Tables 2A, 3A)

RI PT

1. Data Sources

SC

4. Probability of death before recurrence and progression (Table 4A)

AC C

EP

TE D

6. Supplemental Materials References

M AN U

5. Decision Curve Analysis (DCA)

ACCEPTED MANUSCRIPT

Data Sources Patients’ demographic information was extracted from the cancer registry Aurora Health

RI PT

Care (Aurora). Additionally, data on tumor stage, including the presence of concomitant insitu tumors, and tumor grade were drawn from the pathology reports demonstrating the first evidence of bladder cancer within patients’ EMR records. We called this the “initial

SC

pathology report” and used this report to align patient EMR data with cancer registry data. Patients' initial pathology reports were generated primarily through TURBT. When a

M AN U

patient’s initial pathology report detected bladder cancer, but could not definitely rule out muscle invasion, we considered the subsequent pathology report with definitive tumor stage as the patient’s initial pathology report. If two or more pathology reports demonstrated conflicting stage and grading, we defaulted to the report with a more severe

TE D

prognosis. The TURBT occurring on the same day as the patient’s initial pathology report was considered the “Index TURBT.” Patients without an associated initial pathology report or Index TURBT were excluded. To ensure alignment between EMR and cancer registry

EP

data, we excluded patients without an initial pathology report or Index TURBT within 90 days of the bladder cancer diagnosis date reported by the cancer registry. Number of

AC C

tumors, size of tumor, prior bladder cancer history, BCG and chemotherapy, and relevant surgical treatments were extracted using a combination of cancer registry and EMR data. All patient characteristics were identified on or prior to Index TURBT. Additional treatments other than Index TURBT included restaging TURBT, cystectomy, perioperative chemotherapy, and induction BCG and chemotherapy. Restaging TURBT was defined as any TURBT within 8 weeks of the first recorded TURBT. Induction BCG and

ACCEPTED MANUSCRIPT

chemotherapy must have been within 3 months of Index TURBT. Only patients with record of at least two follow-up cystourethroscopy within one year were included in the analysis

receiving BCG or chemotherapy outside the Aurora network.

RI PT

of induction BCG and chemotherapy. This ensures that we did not include patients

Tumors in our dataset were primarily graded using the 2004 WHO system and translated

SC

to the 1973 WHO tumor grading system used in EORTC and CUETO models. We assumed tumors considered either “low grade” or “papillary urothelial neoplasm of low malignant

M AN U

potential” under the 2004 WHO system were G1 in the 1973 system, while patients with tumors considered high grade under the 2004 system were considered G3 in the 1973 system. No patients were mapped to G2 because it corresponds to both high and low grade in the 2004 system.1 To ensure data accuracy, tumor stage and grade from pathology

TE D

reports were checked against the tumor stage and grade recorded in the cancer registry. Whenever a patient’s tumor stage or grade as reported in the cancer registry differed from the initial pathology report, a second manual review of the patient’s initial pathology report

EP

was undertaken.

Number of tumors was obtained from the cancer registry. When number of tumors was

AC C

missing from a cancer registry, a manual review of the surgical notes was conducted. If the tumor number remained unknown, we conducted another manual review of the initial pathology report.

Tumor size was extrapolated from Current Procedure Terminology (CPT) code or surgical note associated with the patient’s Index TURBT. CPT codes for the excision of minor and/or small tumors, (52224, 52234), were considered as <3 cm, and the CPT code for excisions of

ACCEPTED MANUSCRIPT

large tumors (52240), were considered to have tumors >3 cm. When the CPT code from Index TURBT did not provide information on tumor size, it was derived from either the

RI PT

cancer registry or surgical notes. History of bladder cancer and frequency of bladder cancer were established through

review of ICD-9 diagnosis codes, surgical notes, and the cancer registry. A patient was

SC

classified as recurrent if there was evidence of bladder cancer prior to his or her initial pathology report. If a patient had no evidence of prior NMIBC from these three sources, it

M AN U

was assumed that the patient received care for a primary NMIBC.

In accordance with EORTC and CUETO definitions,2-3 we measured the time from Index TURBT to first cancer recurrence or progression to muscle invasive bladder cancer. Bladder cancer recurrence and progression were identified using pathology reports and

TE D

the dates of recurrence or progression listed within the cancer registry. Patients who did not experience a recurrence or progression were censored upon cystectomy or the last known bladder-cancer–directed treatment—a cystoscopy, TURBT, urologist visit, BCG or

AC C

EP

chemotherapy instillation, or urine cytology test.

ACCEPTED MANUSCRIPT

Treatments stratified by stage and grade LGTa 12.3% (85/690)

HGTa 20.7% (54/261)

LGT1 22.1% (15/ 68)

HGT1 39.5% (124/314)

Any Tis 31.7% (26/82)

Cystectomy

1.2% (8/690)

6.5% (17/261)

4.4% (3/68)

12.1% (38/314)

13.4% (11/82)

32.2% (84/261)

26.5% (18/68)

29.3% (92/314)

Perioperative 30.0% -chemo (207/690) 6.8% (23/336)

13.0% (13/100)

10.7% (3/28)

22.0% (31/141)

InductionBCG*

11.0% (37/336)

56.0% (56/100)

35.7% (0/28)

66.7% (94/141)

32.9% (27/82) 10.3% (4/39)

M AN U

SC

Inductionchemo*

RI PT

Group Restaging TURBT

92.3% (36/ 39)

Table 1A: Percent of Aurora patients receiving treatments (absolute fraction of patients are in parenthesis).*Induction chemotherapy and induction BCG includes only patients who

AC C

EP

TE D

have received multiple cystourethroscopies within one year.

ACCEPTED MANUSCRIPT

EORTC and CUETO scoring systems

– – –

– –

– –

0 2 4

0 2 2

RI PT

– – –

0 1 2

0 0 2

0 3

0 0

0 2 2

0 0 2 2

0 0 1 1

0 3

– –

– –

0 4

– –

0 2

0 1

0 6

0 2

0 1

0 1 2

0 0 5

0 1 3

0 2 6

M AN U

0 4 4

EP

0 3

0 3 3 3

TE D

0 3 3 6

0 1

AC C

Age <60 60–70 >70 Gender Male Female Prior Recurrence Rate Primary ≤1 rec/y >1 rec/y No. of Tumors 1 2–3 4–7 ≥8 Tumor Size (cm) <3 ≥3 Tumor Category Ta T1 Carcinoma in-situ No Yes Tumor Grade G1 G2 G3

CUETO Score Recurrence Progression Score Score

SC

EORTC Score Recurrence Progression Score Score

Table 2A: EORTC and CUETO scoring system for recurrence and progression.

ACCEPTED MANUSCRIPT

1-Yr Probability 5-Yr Probability 31% (24–37) 46% (42–49) 62% (58–65) 78% (73–84)

8.2% (5.9–11) 12% (8.0–16) 25% (20–31) 42% (28–56)

21% (17–24.5) 36% (29–42) 48% (40–55) 68% (54–82)

SC

RI PT

15% (10–19) 24% (21–26) 38% (35–41) 61% (55–67)

0.8% (0.0–1.7) 6.0% (5.0–8.0) 17% (14–20) 45% (35–55)

1.2% (0.2–2.2) 3.0% (0.8–5.2) 5.6% (2.7–8.4) 14% (6.6–21)

3.8% (1.9–5.6) 12% (7.6–16) 21% (16–27) 34% (23–44)

M AN U

0.2% (0.0–0.7) 1.0% (0.4–1.6) 5.0% (4.0–7.0) 17% (10–24)

TE D

EORTC Recurrence 0 1–4 5–9 10–17 CUETO Recurrence 0–4 5–6 7–9 10–16 EORTC Progression 0 2–6 7–13 14–23 CUETO Progression 0–4 5–6 7–9 10–14

AC C

EP

Table 3A: EORTC and CUETO’s probability of recurrence and progression according to total score.

ACCEPTED MANUSCRIPT

Probability of death before recurrence and progression

5 yrs (95% CI)

1

5.9% (2.7-8.9)

21.6% (14.6-27.9)

2

5.1% (2.9-7.3)

19.2% (14.5-23.6)

3

6.8% (4.2-9.3)

22.3% (17.5-26.8)

4

6.0% (0.8-11.0)

15.9% (5.7-24.9)

1

4.2% (2.7-5.8)

2

7.1% (4.5-9.7)

3

6.7% (3.1-10.2)

4

14.7% (3.1-24.9)

23.7% (8.7-36.2)

1

5.1% (3.4-6.7)

21.1% (17.3-24.7)

2

4.1% (1.5-6.7)

18.4% (12.6-23.8)

3

8.7% (5.5-11.7)

23.4% (18.1-28.4)

4

4.9% (0.1-9.4)

12.3% (4.3-19.7)

1

5.9% (2.7-9.0)

28.1% (20.0-35.4)

2

5.2% (2.9-7.3)

25.2% (20.0-30.0)

3

8.5% (5.7-11.3)

38.7% (32.5-44.3)

4

9.6% (4.3-14.7)

34.1% (23.3-43.4)

1

5.5% (3.8-7.1)

28.0% (23.9-31.9)

2

2.7% (0.0-6.3)

20.5% (8.8-30.7)

3

4.0% (1.5-6.3)

30.6% (23.6-37.0)

4

16.0% (11.1-20.7)

48.1% (39.7-55.3)

1

5.6% (3.8-7.3)

27.8% (23.5-31.9)

2

5.2% (2.1-8.1)

31.9% (24.2-38.8)

3

10.9% (7.4-14.2)

40.0% (33.2-46.0)

4

6.1% (0.8-11.2)

29.4% (16.9-40.0)

CUETO

NCCN

EORTC

AC C

EP

CUETO

TE D

Progression

NCCN

SC

EORTC

17.9% (14.5-21.1) 27.8% (22.5-32.8) 14.5% (9.0-19.7)

M AN U

Recurrence

RI PT

Risk 1 yr (95% CI) Group

ACCEPTED MANUSCRIPT

Table 4A: Cumulative incidence of death before recurrence and death before progression at one and five years stratified by each model’s risk group. Cause of death was not available

AC C

EP

TE D

M AN U

SC

RI PT

for >50% of this cohort and was therefore not reported.

ACCEPTED MANUSCRIPT

Decision Curve Analysis (DCA) Decision curve analysis (DCA) was conducted to compare the net benefit of EORTC, CUETO, and NCCN risk models. We conducted DCA in two ways. First, DCA presented in Fig. 1A

RI PT

compares the net benefit of EORTC and CUETO as measured by the probabilities reported in the original publications against a treat-all or treat-none treatment regime. In the second DCA presented in Fig. 2A, we used EORTC, CUETO, and NCCN models’ observed recurrence

SC

and progression probabilities in our study population. DCA was evaluated at one and five

M AN U

years for both recurrence and progression.

0.6

0.3

A

Method

CUETO

0.2

EORTC

0.1

Treat All

0.0

Net Benefit

Net Benefit

0.4

Treat None

-0.1 0%

20%

40%

60%

TE D

0.00 0%

5%

10%

Treat All Treat None

20%

40%

60%

80%

Threshold Probability 0.08

Treat All

Net Benefit

EP

Net Benefit

EORTC

0.01

EORTC

Method

CUETO

CUETO

0.0

0%

C

0.02

Method

0.2

Threshold Probability 0.03

B

0.4

D

0.06

Method

0.04

EORTC

0.02

Treat All

Treat None

15%

AC C

Threshold Probability

CUETO

Treat None 0.00 0%

10%

20%

30%

Threshold Probability

Fig. 1A – Decision curve analysis using reported probabilities from EORTC and CUETO. Panel A: One-year recurrence. Panel B: Five-year recurrence. Panel C: One-year progression. Panel D: Five-year progression. For the first DCA (Fig. 1A), NCCN could not be included because NCCN guidelines do not publish the expected probability for one- and five-year recurrence and progression for

ACCEPTED MANUSCRIPT

specific treatment groups. We calculated net benefit according to the methodology described in Vickers et al, where net benefit = (true-positives/n) – false-positives/n * (pt/(1-pt)).4 When the EORTC and CUETO recurrence risk models are analyzed at one and

RI PT

five years using the reported probabilities in the original publication, both models

demonstrated low net benefit compared to a treat-all or treat-none strategy, particularly for recurrence. Neither recurrence risk model shows substantial benefit over a treat-all or a

SC

treat-none strategy. CUETO, in particular, has reduced net benefit at one and five year risk of recurrence (Fig. 1A, Panel A, B) and shows less net benefit than a treat-all strategy for

M AN U

the majority of threshold probabilities. The low net benefit compared to a treat-all or a treat-none strategy can be linked to the significant underestimation of recurrence by the CUETO model. As noted in the paper’s Discussion, this underestimation of recurrence at one and five years may be linked to our analysis being conducted on the entire Aurora

TE D

study population in contrast to an entirely BCG-treated population, which was used to develop the CUETO model. In looking at DCA of EORTC and CUETO progression models (Fig. 1A, Panels C and D), both models produced net benefit higher than a treat-all or a

EP

treat-none strategy. For one-year risk of progression, the EORTC progression model has

AC C

higher net benefit than CUETO, but only marginally higher net benefit compared to CUETO at five years.

ACCEPTED MANUSCRIPT

CUETO

0.2

EORTC

0.1

NCCN Treat All

0.0

0%

20%

40%

60%

EORTC NCCN

0.2

Treat All Treat None

0%

Threshold Probability

CUETO

0.02

EORTC NCCN

0.01

Treat All 0.00

Treat None 5%

10%

0.08

Method

15%

Threshold Probability

Net Benefit

C

0%

20%

40%

60%

80%

Threshold Probability

D

0.06

Method

CUETO EORTC

0.04 0.02 0.00 0%

10%

M AN U

Net Benefit

0.03

Method CUETO

0.4

0.0

Treat None

-0.1

B

SC

0.3

0.6

Method

RI PT

A

Net Benefit

Net Benefit

0.4

20%

NCCN Treat All Treat None

30%

Threshold Probability

Fig. 2A – Decision curve analysis with observed data in the Aurora study population. Panel A: One-year recurrence. Panel B: Five-year recurrence. Panel C: One-year progression.

TE D

Panel D: Five-year progression.

For the second method (Fig. 2A), DCA was conducted across EORTC, CUETO, and NCCN as measured by the cumulative incident calculated in the Aurora study population compared

EP

to a treat-all or treat-none treatment regime. For recurrence, EORTC, CUETO, and NCCN have a similar net benefit at one year (Fig. 2A, Panel A) with EORTC demonstrating slightly

AC C

higher net benefit at low-threshold probabilities between 15–30%. At five years (Fig. 2A, Panel B) although EORTC demonstrates a marginally higher net benefit than CUETO and NCCN, none of the three models provides a significant net benefit compared to a treat-all or treat-none approach. DCA of recurrence models demonstrates no net benefit for any of the recurrence models at five years.

ACCEPTED MANUSCRIPT

DCA for progression at one year (Fig. 2A, Panel C) and five years (Figure 2A, Panel D) show EORTC and CUETO have a similar net benefit, and both progression models have a higher net benefit than the treat-none and treat-all approaches for a range of thresholds.

RI PT

Interestingly, DCA of one-year risk of progression shows NCCN has the highest net benefit at low-threshold probabilities between 1–7%. At five years, all three models demonstrate

SC

comparable net benefit compared to a treat-all or treat-none approach.

DCA supports the paper’s conclusions that NCCN offers comparable clinical utility to either

M AN U

CUETO or EORTC, particularly for predicting progression. All three progression models offer much greater net benefit when compared to recurrence models. We conclude DCA of recurrence models demonstrates the need for improved NMIBC recurrence prediction models to help better guide patients and clinicians on treatment

TE D

strategies. Using EORTC, CUETO, and NCCN recurrence models have marginal net benefit assuming all patients will recur or no patients will recur, particularly within five-years. DCA of progression models is more promising and indicates that that there is some greater

EP

clinical utility from using EORTC progression model to using the CUETO or NCCN progression model at five years. However, NCCN progression model does offer some

AC C

greater utility between the threshold probabilities of 1–7% for one-year risk of progression.

It is important to note several limitations of DCA within this study. In method one, we used the risk of recurrence and progression as reported by EORTC and CUETO to calculate net benefit. However, as demonstrated in this paper and others, EORTC and CUETO risk tables do not accurately predict the risk of five-year recurrence or progression. In the second

ACCEPTED MANUSCRIPT

method of DCA, we used the risk of recurrence and progression observed within the Aurora study population. DCA curves appear to offer greater net benefit than DCA constructed using the one- and five-year predictions reported by EORTC and CUETO. It is unclear how

RI PT

well our study population translates to a generalized NMIBC cohort. Furthermore, clinical treatments such as chemotherapy, BCG, and monitoring regime vary substantially between risk groups within the Aurora study population. Therefore, given the above limitations, we

AC C

EP

TE D

M AN U

SC

suggest caution in the interpretation of DCA for this study.

ACCEPTED MANUSCRIPT

Supplementary Materials References [1]

Grignon DJ. The current classification of urothelial neoplasms. Mod Pathol. 2009

[2]

RI PT

Jun;22 Suppl 2:S60-9. Sylvester RJ, van der Meijden AP, Oosterlinck W, et al. Predicting recurrence and

progression in individual patients with stage Ta T1 bladder cancer using EORTC risk tables:

SC

A combined analysis of 2596 patients from seven EORTC trials. Eur Urol. 2006;49(3):4665; discussion 475-7.

Fernandez-Gomez J, Madero R, Solsona E, et al. Predicting nonmuscle invasive

M AN U

[3]

bladder cancer recurrence and progression in patients treated with bacillus CalmetteGuerin: The CUETO scoring model. J Urol. 2009;182(5):2195-203. [4]

Vickers, A J and Elkin EB. Decision curve analysis: a novel method for evaluating

AC C

EP

TE D

prediction models. Med Decis Making. 2006;26(6):565-74.