- Email: [email protected]
External validation of the pretreatment nomogram to predict acute urinary retention after 125I prostate brachytherapy
Brachytherapy 11 (2012) 256e264
External validation of the pretreatment nomogram to predict acute urinary retention after 125I prostate brachytherapy Ellen M. Roeloffzen1,*, Juanita Crook2, Evelyn M. Monninkhof3, Michael McLean4, Marco van Vulpen1, Elantholi P. Saibishkumar4 1 Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, The Netherlands Department of Radiation Oncology, BCCA Center for the Southern Interior/University of British Columbia, Kelowna, BC, Canada 3 Department of Epidemiology, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands 4 Department of Radiation Oncology, Princess Margaret Hospital/Univeristy of Toronto, Toronto, ON, Canada 2
ABSTRACT
PURPOSE: Acute urinary retention (AUR) after 125I prostate brachytherapy has a negative impact on quality of life. Recently, the authors developed a nomogram to predict the risk of AUR preoperatively. The aim of this study was to assess the external validity of the nomogram. METHODS AND MATERIALS: The nomogram was initially developed on 714 patients treated with 125I prostate brachytherapy at the University Medical Center Utrecht, the Netherlands. Predictive factors included in the nomogram were prostate volume, international prostate symptom score, neoadjuvant hormonal treatment, and prostate protrusion. For external validation, the data of 715 consecutive patients treated between January 2003 and July 2008 at the Princess Margaret Hospital, Toronto, were used. The performance of the nomogram was evaluated by discrimination (ability to distinguish between patients who develop AUR yes or no) and calibration (agreement between observed and predicted numbers of AUR). RESULTS: Of the 715 patients treated at the Princess Margaret Hospital, 67 patients (9.4%) developed AUR compared with 8.0% in the University Medical Center Utrecht cohort. In the validation data set, the discriminatory ability of the nomogram was good (receive operating characteristic area: 0.86; 95% confidence interval: 0.82e0.91), and comparable to the derivation data set (receive operating characteristic area: 0.82; 95% confidence interval: 0.77e0.88). Comparison between the predicted risks and the observed frequencies of AUR showed underestimation of the nomogram in the validation data set for high AUR risks values. Still, the negative predictive value for the risk of AUR, using a cutoff value of 5%, was high (98.1%). CONCLUSION: External validation of the nomogram shows adequate discrimination of patients with and without AUR. Therefore, the nomogram can aid in individualized treatment decision making. Ó 2012 American Brachytherapy Society. Published by Elsevier Inc. All rights reserved.
Keywords:
Prostate cancer; Acute urinary retention; Prostate brachytherapy; Predictive nomogram; External validation
Introduction The most predominant severe acute toxicity after prostate brachytherapy is acute urinary retention (AUR). Received 10 August 2011; received in revised form 19 December 2011; accepted 28 December 2011. The research was conducted at the Princess Margaret Hospital, Toronto, ON, Canada and at University Medical Center Utrecht, Utrecht, The Netherlands. The authors declare no conflicts of interest. * Corresponding author. Department of Radiation Oncology, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands. Tel.: þ31-887558800; fax: þ31-887555850. E-mail address: [email protected] (E.M. Roeloffzen).
Published AUR rates vary from 6% to 34% (1e5). It is known that AUR negatively influences quality of life (6, 7). Preoperative prediction of AUR is useful for patient counseling and for clinical decision making in patients with localized prostate cancer. In a previous study, the authors developed a clinical nomogram to predict the risk of AUR after 125I prostate brachytherapy preoperatively, using the data of 714 consecutive patients treated at their center (Appendix) (8). The nomogram was based on the most important pretreatment risk factors for AUR, that is, prostate volume, international prostate symptom score (IPSS), neoadjuvant hormonal treatment (HT), and the extent of prostate protrusion into
1538-4721/$ - see front matter Ó 2012 American Brachytherapy Society. Published by Elsevier Inc. All rights reserved. doi:10.1016/j.brachy.2011.12.011
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
the bladder. Both calibration and discrimination were adequate (receiver operating characteristic [ROC] area: 0.82). The nomogram showed that among patients with a very low sum score (!18 points), the risk of AUR was only 0e5%, and that in patients with a high sum score (O35 points), the risk of AUR was more than 20%. However, as the nomogram was based on single-center data and patient selection and treatment techniques may differ between centers, its predictive value in other patient populations is unknown (9e11). The aim of this study was to perform external validation of the nomogram, by using data of patients treated at the Princess Margaret Hospital (PMH) in Toronto, Canada.
Methods and materials Patients The derivation study population consisted of 714 consecutive patients with localized prostate cancer treated with 125I seed implantation between January 2005 and December 2008 at the University Medical Center Utrecht (UMCU), the Netherlands (8). The validation population consisted of 715 consecutive patients with localized prostate cancer treated with 125I seed implantation between January 2003 and July 2008 at the PMH, Toronto, Canada. The PMH was chosen for external validation for two main reasons: (1) MR imaging for postimplant dose evaluation is performed, which is required for adequate determination of prostate protrusion (8, 12, 13), (2) it is a high-volume center with meticulous followup and documentation of toxicity. The implantation techniques and dosimetric analyses were similar and according to the guidelines of Groupe-Europeen de Curietherapie-European Society for Therapeutic Radiology and Oncology and American Brachytherapy Society (14, 15). Table 1 summarizes the similarities and differences in 125I prostate brachytherapy procedures between the centers. The UMCU and the PMH brachytherapy procedures have both been extensively described previously (4, 6, 16, 17). All patients were treated in lithotomy position. The radioactive seeds were inserted transperineally according to the preplan in a modified peripherally loaded Seattle technique (4). All implants were evaluated at 1 month, by using CT and 1.5 or 3.0 T MRI fusion. Implant quality was defined in terms of the standard dosimetric parameters D90, V100, V150, and V200 (14, 15). Urinary function was assessed using IPSS questionnaires, which were completed at baseline and at each followup visit. AUR was defined as any need for urinary catheterization within 3 months after implantation (18). Research Ethics Board approval was obtained to access the data from the PMH prospective database. A consecutive cohort of patients was selected between January 2003 and July 2008, ensuring adequate patient numbers and a substantial followup. Baseline characteristics and post-treatment sequelae were retrieved, including the dates and duration of retention and catheterization.
257
Determination of prostate protrusion The extent of prostate protrusion into the bladder was recently shown to be a strong independent predictor of AUR (8, 13). It relates to the large median lobes and was defined as the maximum distance from bladder base to prostate base (13). The extent of prostate protrusion was determined retrospectively on sagital MR images at 1 month after implantation for all patients at PMH (Fig. 1). All delineations were performed by the same physician (EMR), who was blinded to patient’s AUR status. The authors previous study (8) showed that the inter- and intraobserver variability of prostate protrusion measurements were good (i.e., 0.7 mm [standard deviation 0.9] and 0.4 mm [standard deviation 0.7], respectively). Pearson correlation coefficients (r) were calculated and showed that both inter- and intraobserver repeatability of prostate protrusion measurements were high (r 5 0.97 and r 5 0.94, respectively). Statistical analysis Patient characteristics of the UMCU and PMH were compared using independent sample t tests (continuous variables) or c2 tests (categorical or dichotomous variables). Multivariate logistic regression analysis was performed to explore the predictive values of the predefined predictors (i.e., prostate volume, IPSS, neoadjuvant HT, and the extent of prostate protrusion) of AUR in the validation data set (8). Proper validation requires the use of the fully specified existing prognostic model to predict outcomes for the patients in the validation data set and then compare these predictions with the patients’ actual outcomes. Therefore, the risk of AUR was calculated for each individual patient in the validation data set using the following equation (Appendix) (8): Risk of AUR 5 1=½1 þ expðlinear predictorÞ 100% The predictive accuracy of the nomogram was quantified using discrimination and calibration measures. Differences in discriminative ability (i.e., the ability of the model to distinguish patients who develop AUR: yes or no) between the derivation and validation model were quantified by the area under the ROC curve (ROC area). The ROC area may theoretically range from 0.5 (discrimination equivalent to that of chance) to 1.0 (perfect discrimination). Calibration of the model (i.e., agreement between observed and predicted numbers of AUR) was determined by comparing the predicted and the observed numbers of AUR among five risk groups. In addition, calibration was statistically tested across deciles of predicted risks with the HosmereLemeshow test, where an insignificant test indicates good model fit (11). Furthermore, the negative predictive value using a cutoff value of 5% was computed. The 5% cutoff value was chosen because the AUR rate in our population was 8.0% and a reduction in AUR rate is aimed for.
258 Table 1 Summery of the
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264 125
I prostate brachytherapy implantation protocols at the UMCU and the PMH
Characteristics
UMCU derivation data set
PMH validation data set
Eligibility criteria Age at implantation Clinical tumor stage Initial PSA (ng/mL) Gleason sum score Prostate volume (cc) Neoadjuvant HT
Any T1eT2 !20 #7 #50 Prostate volume O50 cc
Any T1eT2 !10; (10e20 under RTOG protocol) #6; (7 under RTOG protocol) #60 Prostate volume O50 cca
Dosimetry goals Prescribed dose (Gy) Prostate D90 (Gy) Prostate V100 (%) Prostate V150 (%) Prostate V200 (%) Urethra dose Rectal dose
145 O160 O95 66 33 #150% of prescribed dose #100% of prescribed dose
145 170e180 O99 54e60 12e20 #150% of prescribed dose No preplan goals
Imaging 3D-TRUS Fluoroscopy MRI CT
Before and after needle insertion (real time) After implantation Preimplant and 4 weeks postimplant (1.5 or 3 T) 4 weeks postimplant
Before and after needle insertion (real time) After each row of needles 4 weeks post-implant (1.5 T) 4 weeks postimplant
MRI based Spinal 125 I 0.51 U
TRUS based General 125 I 0.4 U
Stranded (50%) or loose (50%) No Taken out immediately after procedure No For 1 month; and as long as symptoms persisted
Stranded (17%) or loose (82%) No Taken out immediately after procedure No Start 1 week before to 3 months after implant; and as long as symptoms persisted
At 1, 3, 6, 9, and 12 months and twice annually thereafter Alternately by radiation oncologist and urologist AUA, toxicity scores, intervention data 6 months
At 1, 3, 6, 9, and 12 months, and twice annually thereafter Radiation oncologist AUA, toxicity scores, intervention data 6 months
Procedure Preplanning Anesthesia Source type Average seed activity (1U 5 1 mGym2 h1) Type of seeds Supplemental EBRT Postimplant catheter Routine steroids a-blocker use Followup Frequency Physician Documentation Minimum followup
PSA 5 prostate-specific antigen; HT 5 hormonal treatment; TRUS 5 transrectal ultrasound. a Initially, neoadjuvant HT for downsizing of the prostate was given to patients with prostate volumes O50 cc; however, the use of HT diminished over time after reports on increased urinary retention with this approach and also generalized toxicity of HT.
A commercial statistical package of social sciences (SPSS version 16.0; SPSS, Chicago, IL) and R was used for statistical analysis of the data.
Results The AUR rates at the UMCU and the PMH were 8.0% and 9.4%, respectively. The median time to AUR was 10 days (range: 0e90) at the UMCU and 1 day (range: 0e90) at the PMH. The median duration of catheter dependency was 37 days (range: 2e140 days) and 10 days (range: 1e360), respectively. Table 2 summarizes the clinical and treatment characteristics of patients treated at the UMCU (derivation data set) and the PMH (validation data set). The data sets were
largely comparable, except for neoadjuvant HT, Gleason score, and initial prostate-specific antigen values, which were all lower in the validation data set (owing to local treatment policy and provincial eligibility criteria). The mean number of implanted needles and seeds were higher in patients treated at the PMH, probably owing to the lower average seed strength. Like at the UMCU (8), patients who developed AUR at the PMH had significantly higher pretreatment prostate volumes ( p! 0.001), higher IPSS ( p 5 0.001), larger extents of prostate protrusion ( p! 0.001), and were more frequently treated with neoadjuvant HT ( p 5 0.05) compared with patients without AUR (Table 3). Although statistically significant, the differences in age, prostate D90, V150 and V200 between AUR and no-AUR patients are not considered clinically relevant. Multivariate logistic
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
259
Table 2 Clinical and treatment characteristics for patients treated at the UMCU and patients treated at the PMH
Fig. 1. Illustration of determination of the extent of prostate protrusion into the bladder (defined as the maximum distance from bladder base to prostate base). Delineation: red 5 prostate (target volume); yellow 5 urethra; blue 5 bladder; green 5 construction lines.
regression analyses of the derivation data set and the validation data set are shown in Table 4. The odds ratios between the derivation and validation data set are largely comparable, except for prostate protrusion (higher predictive value in the validation dataset). The discriminative value of the nomogram in the validation data set was high (ROC area: 0.86; 95% confidence interval: 0.82e0.91), which is comparable with the discrimination in the derivation data set (ROC area: 0.82; 95% confidence interval: 0.77e0.88). In Fig. 2, the ROC curve for the validation data set is shown. Figure 3 shows the calibration plot. The dotted line shows the ideal and the solid line shows the association between the predicted risk and the observed frequency of AUR. The nomogram underestimated the risk of AUR in the validation data set for high-AUR risk values. The Hosmere Lemeshow test was statistically significant ( p !0.001; c2 5 47.5), which also indicates poor calibration. However, 70% of the patients had an AUR risk of !5%, and for risks !5% the calibration was sufficient. Moreover, the negative predictive value for AUR (using a clinically useful cutoff value of 5%) was high (98.1%), indicating that for a predicted AUR risk of less than 5%, indeed 98.1% of patients did not develop AUR.
Discussion External validation of clinical prediction tools is important. Accurate predictions in patients that were used to
Characteristic
UMCU (n 5 714)
PMH (n 5 715)
Pretreatment Age at implantation (y)
65.1 6.4
62.2 6.9
Clinical tumor stage, n (%) T1 T2
498 (70) 216 (30)
481 (67) 234 (33)
Gleason sum score !7, n (%) 7, n (%) iPSA (ng/mL)
548 (77) 166 (23) 9.8 5.3
668 (93) 47 (7) 5.5 2.4
Pretreatment IPSS, n (%) 0 6 11 O20
257 199 243 15
393 195 117 10
Pretreatment TURP, n (%) Yes No
11 (1.5) 703 (98.5)
1 (0.1) 714 (99.9)
Neoadjuvant HT Yes, n (%) No, n (%) Pretreatment prostate volume (cm3) Prostate length (cm) Prostate protrusion (mm)
137 (19) 577 (81) 35.4 9.0 4.2 0.6 1.1 1.9
24 (3) 691 (97) 35.1 10.6 3.8 0.4 1.0 1.3
Treatment Needles (n) Seeds (n) Prostate D90 (Gy) Prostate V100 (%) Prostate V150 (%) Prostate V200 (%)
24.1 3.8 73.3 13.1 164.1 28.5 93.1 6.7 71.3 11.6 39.1 11.7
30.5 4.3 105.3 16.6 167.0 18.4 95.3 4.2 60.7 10.5 29.3 8.1
(36) (28) (34) (2)
(55) (27) (16) (1)
UMCU 5 university medical center Utrecht; PMH 5Princess Margaret Hospital; n.s. 5 not statistically significant; iPSA 5 initial prostate-specific antigen level; IPSS 5 international prostate symptom score; TURP 5 transurethral resection of the prostate; HT 5 hormonal treatment; D90 5 minimal dose received by 90% of prostate; V100, V150, V200 5 percentage of prostate/ urethra volume receiving 100%, 150% and 200% of prescribed dose, respectively; SD 5 standard deviation. Values are means (SD) or n (%)
develop a nomogram are no guarantee for good predictions in other patient populations (9e11). Only after external validation, a model is considered generally applicable. In this study, the authors found accurate discrimination for AUR of our nomogram when tested in the validation population (ROC area: 0.86). Calibration was sufficient for low AUR risks, but poor for high AUR risks. Because the negative predictive value for AUR at a cutoff value of 5% was high (98.1%), the nomogram is able to correctly identify low-risk patients, but for a reliable estimation of risks, higher than 5% calibration of the nomogram should be improved. Theoretically, differences in patient and treatment techniques might influence the risk of AUR. The authors did not find a major difference in AUR rate between the UMCU
260
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
Table 3 Clinical and treatment characteristics for patients who developed AUR and patients who did not (no AUR), according to clinical center UMCU
PMH
Characteristics
AUR (n 5 57)
No AUR (n 5 657)
p-Value
AUR (n 5 67)
No AUR (n 5 648)
p-Value
Pretreatment Age at implantation (y)
65.8 6.6
65.0 6.3
NS
63.9 6.7
62.0 6.9
0.031
Clinical tumor stage, n (%) T1 T2
41 (72) 16 (28)
457 (70) 200 (30)
NS
45 (67) 22 (33)
436 (67) 212 (33)
NS
Gleason sum score !7, n (%) 7, n (%)
44 (77) 13 (23)
504 (77) 153 (23)
NS
64 (96) 3 (4)
604 (93) 44 (7)
NS
iPSA (ng/mL)
9.2 4.2
9.9 5.3
NS
5.6 2.3
5.5 2.4
NS
Pretreatment IPSS, n (%) 0e5 6e10 11e20 O20
8 (14) 14 (25) 32 (56) 3 (5)
249 (38) 185 (28) 211 (32) 12 (2)
!0.001
24 (36) 24 (36) 17 (25) 2 (3)
369 (57) 171 (26) 100 (15) 8 (1)
0.001
Pretreatment TURP, n (%) Yes No
1 (2) 56 (98)
10 (2) 647 (98)
NS
0 (0) 67 (100)
1 (0.2) 647 (99.8)
NS
Neoadjuvant HT, n (%) Yes No
16 (28) 41 (72)
121 (18) 536 (82)
NS 0.076
5 (7.5) 62 (92.5)
19 (3) 629 (97)
0.05
Prostate volume, TRUS (cm3)
38.9 9.1
35.1 8.9
0.002
42.2 11.5
34.4 10.3
!0.001
Prostate length (cm)
4.2 0.6
4.2 0.6
NS
4.01 0.45
3.77 0.42
NS
Prostate protrusion (mm)
3.5 2.9
0.9 1.6
!0.001
3.1 1.6
0.8 1.1
!0.001
Treatment Needles (n) Seeds (n) Prostate D90 (Gy) Prostate V100 (%) Prostate V150 (%) Prostate V200 (%)
25.3 3.5 78.0 11.7 161.6 27.4 91.5 12.3 69.5 13.1 37.4 9.7
24.0 3.8 72.9 13.2 164.3 28.6 93.2 6.0 71.5 11.5 39.3 11.9
0.013 0.005 NS NS NS NS
32.6 4.4 116.3 16.4 161.4 16.9 94.5 4.2 57.7 9.7 27.0 7.7
30.3 4.3 104.1 16.2 167.6 18.5 95.4 4.2 61.0 10.6 29.5 8.2
!0.001 !0.001 0.009 NS 0.016 0.015
UMCU 5 university medical center Utrecht; PMH 5 Princess Margaret Hospital; AUR 5 acute urinary retention; NS 5 not statistically significant; iPSA 5 initial prostate-specific antigen level; IPSS 5 international prostate symptom score; TURP 5 transurethral resection of the prostate; HT 5 hormonal treatment; TRUS 5 transrectal ultrasound; D90 5 minimal dose received by 90% of prostate; V100, V150, V200 5 percentage of prostate/urethra volume receiving 100%, 150% and 200% of prescribed dose, respectively; SD 5 standard deviation. Values are means (SD) or n (%).
and PMH (8.0% vs. 9.4%). Patients at the PMH had lower mean initial prostate-specific antigen levels, lower Gleason scores, and were less frequently treated with neoadjuvant HT (Table 2), which was because of the local treatment policy and provincial eligibility criteria for 125I prostate brachytherapy (Table 1). The higher number of implanted needles and seeds in patients treated at the PMH (Table 2) might be explained by the lower average seed strength used because the U per cc of prostate volume was approximately the same for both centers (0.015 UMCU vs. 0.011 PMH). Although some minor differences in patient and treatment characteristics existed, these differences did not influence the discrimination of the nomogram. This implies that the nomogram might also be applicable to patients treated at other centers. Furthermore, the high ROC value indicates that the nomogram indeed
contained the most important predictors of AUR. An extensive evaluation of these risk factors in the context of recent literature was performed in the authors’ previous article (8). The slightly higher ROC value in the validation population compared with that of the derivation population (0.86 vs. 0.82) might be explained by the more heterogeneous validation population. A more heterogeneous sample according to the included predictors has been shown to be related to a higher discriminative ability because the model can distinguish more subjects with very low or very high predictions (19). When comparing the predicted risks with the observed frequencies of AUR (calibration), the nomogram underestimated the risk of AUR in the validation population for high-AUR risk values (Fig. 3). Several explanations are available for this finding: (1) The higher incidence of
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
261
Table 4 Multivariate logistic regression analysis for the prediction of AUR after prostate brachytherapy in the UMCU and PMH data set UMCU (derivation data set)
PMH (validation data set)
Factor
OR
95% CI
p
Prostate volume (cm3)
1.02
0.99e1.06
0.243
Pretreatment IPSS 0e5 6e10 11e20 O20
1.78 4.18 6.05
0.69e4.64 1.78e9.81 1.18e30.9
Neoadjuvant HT (yes/no)
1.04
0.52e2.07
0.914
Prostate protrusion (mm)
1.58
1.40e1.79
0.000
b coefficient
a
OR
95% CI
p
0.021
1.00
0.97e1.03
0.916
0.538 1.332 1.677
2.03 1.86 8.62
1.01e4.08 0.81e4.27 1.45e51.4
0.035
0.71
0.18e2.76
0.617
0.427
3.13
2.42e4.06
0.000
0.003
Intercept 2Log likelihood ROC area
b coefficient 0.002
0.043
4.84 284.4 0.82
0.707 0.619 2.154 0.348 1.141 4.69 293.5 0.86
AUR 5 acute urinary retention; UMCU 5 university medical center Utrecht; PMH 5 Princess Margaret Hospital; OR 5 odds ratio; CI 5 confidence interval; IPSS 5 international prostate symptom score; HT 5 hormonal treatment; ROC 5 receiver operating characteristic. a b coefficients were multiplied by a shrinkage factor of 0.93, to adjust for optimism that might be expected when the model is applied to new, but similar patients. The intercept was also adjusted to the new situation.
patients with AUR in the validation data set. Although the absolute difference was only 1.4%, the relative difference was 18%. (2) The slight differences in patient and treatment characteristics might influence calibration measures. At the PMH, acceptance for brachytherapy and prediction of retention was based mostly on urinary voiding study results, specifically peak flow rate. There is evidence in literature that peak flow rate is a powerful predictor of AUR (20, 21). Because patients with low-peak flow rates (!10 mL/s) were not necessarily excluded for brachytherapy when they were aware and willing to accept the high-AUR risk, this might be an explanation for the underestimation for high-AUR risk values in the validation population. In ongoing research, the authors will further
explore the relation between peak flow rate and the risk of AUR. (3) The low number of patients with an AUR risk of O5% (70% of the patients had an AUR risk of !5%). In Fig. 3, it can be seen that predictions up to 5% are adequate. Therefore, the nomogram can be safely used for risk scores up to 5%. However, caution is warranted for interpretation of risk scores greater than 5%. Because the AUR rate in our population was 8.0% and a reduction in AUR rate is aimed for, a cutoff value of 5% is reasonable. They showed that the negative predictive value for AUR, using a cutoff value of 5%, was high (98.1%). This indicates that for a predicted AUR risk of !5%, 98.1% of patients indeed did not develop AUR. In future, by expanding the number of patients included in the analysis, the authors hope to
Fig. 2. ROC curve for the predictability of acute urinary retention in the derivation data set, based on pretreatment prostate volume, International Prostate Symptom Score, neoadjuvant hormonal treatment, and prostate protrusion. ROC 5 Receiver operating characteristic.
Fig. 3. Calibration plot. The continuous line shows the relation between observed frequencies and predicted probabilities. The dotted line indicates perfect calibration, that is, observed frequencies and predicted probabilities are in complete agreement.
262
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
improve the calibration and predictive value for high-risk patients. The fundamental question is ‘‘how much does this nomogram really add?’’ Based on clinical experience of a radiation oncologist, the additive value of the nomogram for very low-risk patients (small volume, low IPSS, and no HT) might be modest. However, this nomogram provides a more accurate and validated risk assessment to individual patients when discussing brachytherapy as one of their treatment options. Furthermore, the model also includes prostate protrusion as an independent risk factor and the nomogram can easily be used by less-experienced radiation oncologists as well. Although the authors would have a better predictive value for high-risk patients, the decision on where to place the cutoff value for an ‘‘unacceptable’’ high risk of AUR remains unclear and is dependent on the expert opinion of the physician and the patient’s individual preference. A ‘‘high’’ risk of AUR does not necessarily imply to withhold patients from brachytherapy. However, in patients with a very high risk of retention, alternative treatment options might be considered. The risk of AUR still has to be weighed against the toxicity profile of other treatment options. Conversely, men with a favorable risk score may be more confident in their decision to proceed with seed implants and ultimately the overall rate of retention could decrease in time with appropriate patient selection. Furthermore, in the last few years, strategies of active surveillance have emerged, which is considered as a reasonable option for highly selected patients with low-risk prostate cancer (22, 23). Recent studies suggest that outcomes after direct treatment or active surveillance combined with delayed treatment are similar; however, long-term followup is still needed (22). A disadvantage of active surveillance is the possible anxiety and distress for patients of withholding radical treatment (24). There are some limitations of this study, which should be mentioned. First, the model requires MRI scans to determine the extent of prostate protrusion; however, unfortunately, MRI is not available at all brachytherapy centers. Although MRI is preferable for delineation because of its high soft tissue contrast (12); in the ongoing research, the authors will further explore the predictive value of prostate protrusion measured on TRUS to allow wide use of the nomogram. Second, at the PMH, prostate protrusion was measured at 1 month after implantation instead of before implantation like at the UMCU. In one of our previous publications, the authors showed that the edema factor had no predictive value for the development of AUR (25). This might be explained by the fact that in most patients, prostate swelling has resolved at 4 weeks after implantation (26, 27). Therefore, they do not expect that the difference in time of prostate protrusion measurement has any influence on the present study outcome. Third, Vergouwe et al. (9) recommended at least 100 events for external validation to obtain enough power
(80%). However, this is the ideal situation, but may not be feasible clinically, especially when the event rate is low. In literature, sample sizes of validation sets differ over a wide range. In a review of Altman and Royston (28), validation samples with sizes varying between 52 and 479 patients were described, and the number of events ranged from 24 to 115. Therefore, by including 715 patients and 67 events, our validation population can be considered adequate compared with other published validation studies. Fourth, according to the state of the art methodology guidelines (11), our nomogram is based on a limited set of predictors at multivariate analyses from reviewed literature. However, more potential predictors have been described in single studies or on univariate analysis only. Further research might lead to an extension and improvement of the performance of the model by addition of other relevant risk factors. Updating of the nomogram might lead to improvement of calibration measures (10). Continued efforts to refine and establish validity of the nomogram across worldwide patient populations are thus needed to realize the goal of assisting men and their care providers in appraising risks and making individualized treatment choices. Nevertheless, the authors showed that in two individual patient cohorts of more than 700 patients each, the model performs reasonably well.
Conclusions External validation of the nomogram shows adequate discrimination for the risk of AUR. The nomogram is able to correctly identify low risk patients with a negative predictive value of 98%. Therefore, the nomogram can be widely used to predict the risk of AUR after 125I prostate brachytherapy. Because the balance between treatment outcome and quality of life is considered very important nowadays, the nomogram might be a useful tool for physicians and patients in individualized treatment decision making in low-risk prostate cancer. References [1] Terk MD, Stock RG, Stone NN. Identification of patients at increased risk for prolonged urinary retention following radioactive seed implantation of the prostate. J Urol 1998;160:1379e1382. [2] Lee N, Wuu CS, Brody R, et al. Factors predicting for postimplantation urinary retention after permanent prostate brachytherapy. Int J Radiat Oncol Biol Phys 2000;48:1457e1460. [3] Bucci J, Morris WJ, Keyes M, et al. Predictive factors of urinary retention following prostate brachytherapy. Int J Radiat Oncol Biol Phys 2002;53:91e98. [4] Crook J, McLean M, Catton C, et al. Factors influencing risk of acute urinary retention after TRUS-guided permanent prostate seed implantation. Int J Radiat Oncol Biol Phys 2002;52:453e460. [5] Locke J, Ellis W, Wallner K, et al. Risk factors for acute urinary retention requiring temporary intermittent catheterization after prostate brachytherapy: a prospective study. Int J Radiat Oncol Biol Phys 2002;52:712e719.
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264 [6] Roeloffzen EM, Hinnen KA, Battermann JJ, et al. The impact of acute urinary retention after iodine-125 prostate brachytherapy on healthrelated quality of life. Int J Radiat Oncol Biol Phys 2010;77:1322e1328. [7] Crook J, Fleshner N, Roberts C, et al. Long-term urinary sequelae following 125iodine prostate brachytherapy. J Urol 2008;179:141e145. [8] Roeloffzen EM, Vulpen van M, Battermann JJ, et al. Pretreatment nomogram to predict the risk of acute urinary retention after I-125 prostate brachytherapy. Int J Radiat Oncol Biol Phys 2011;81:737e744. [9] Vergouwe Y, Steyerberg EW, Eijkemans MJ, et al. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol 2005;58:475e483. [10] Toll DB, Janssen KJ, Vergouwe Y, et al. Validation, updating and impact of clinical prediction rules: A review. J Clin Epidemiol 2008;61:1085e1094. [11] Royston P, Moons KG, Altman DG, et al. Prognosis and prognostic research: Developing a prognostic model. BMJ 2009;338:b604. [12] Villeirs GM, De Meerleer GO. Magnetic resonance imaging (MRI) anatomy of the prostate and application of MRI in radiotherapy planning. Eur J Radiol 2007;63:361e368. [13] Roeloffzen EM, Monninkhof EM, Battermann JJ, et al. Acute urinary retention after I-125 prostate brachytherapy in relation to dose in different regions of the prostate. Int J Radiat Oncol Biol Phys 2010;80:76e84. [14] Ash D, Flynn A, Battermann J, et al. ESTRO/EAU/EORTC recommendations on permanent seed implantation for localized prostate cancer. Radiother Oncol 2000;57:315e321. [15] Nag S, Beyer D, Friedland J, et al. American Brachytherapy Society (ABS) recommendations for transperineal permanent brachytherapy of prostate cancer. Int J Radiat Oncol Biol Phys 1999;44:789e799. [16] Battermann JJ, Boon TA, Moerland MA. Results of permanent prostate brachytherapy, 13 years of experience at a single institution. Radiother Oncol 2004;71:23e28. [17] Crook J, Toi A, McLean M, Pond G. The utility of transition zone index in predicting acute urinary morbidity after 125I prostate brachytherapy. Brachytherapy 2002;1:131e137.
263
[18] Trotti A. The evolution and application of toxicity criteria. Semin Radiat Oncol 2002;12:1e3. [19] Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: Use of benchmark values to disentangle a casemix effect from incorrect coefficients. Am J Epidemiol 2010; 172:971e980. [20] Martens C, Pond G, Webster D, et al. Relationship of the International Prostate Symptom score with urinary flow studies, and catheterization rates following 125I prostate brachytherapy. Brachytherapy 2006;5: 9e13. [21] Ikeda T, Shinohara K. Peak flow rate is the best predictor of acute urinary retention following prostate brachytherapy: our experience and literature review. Int J Urol 2009;16:558e560. [22] Van den Bergh RC, Vasarainen H, van der Poel HG, et al. Short-term outcomes of the prospective multicentre ‘‘Prostate Cancer Research International: Active Surveillance’ study. BJU Int 2010;105: 956e962. [23] Klotz L, Zhang L, Lam A, et al. Clinical results of long-term followup of a large, active surveillance cohort with localized prostate cancer. J Clin Oncol 2009;28:126e131. [24] Van den Bergh RC, Essink-Bot L, Roobol MJ, et al. Do anxiety and distress increase during active surveillance for low risk prostate cancer? J Urol 2010;183:1786e1791. [25] Roeloffzen EM, Battermann JJ, Van Deursen MJ, et al. The influence of dose on the risk of acute urinary retention after I-125 prostate brachytherapy. Int J Radiat Oncol Biol Phys 2010;80:76e84. [26] Taussky D, Austen L, Toi A, et al. Sequential evaluation of prostate edema after permanent seed prostate brachytherapy using CT-MRI fusion. Int J Radiat Oncol Biol Phys 2005;62:974e980. [27] Waterman FM, Yue N, Corn BW, et al. Edema associated with I-125 or Pd-103 prostate brachytherapy and its impact on post-implant dosimetry: an analysis based on serial CT acquisition. Int J Radiat Oncol Biol Phys 1998;41:1069e1077. [28] Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med 2000;19:453e473.
264
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
Appendix Nomogram to predict the risk of acute urinary retention after
125
I prostate brachytherapy
Predictor Prostate volume (cm3) Value #16 Score 3
17e21 4
22e25 5
26e30 6
IPSS Value Score
0e5 0
6e10 5
11e20 13
O20 17
Neoadjuvant HT Value Score
No 0
Yes 1
Prostate protrusion (mm) Value 0 Score 0
31e35 7
36e40 8
41e45 9
46e50 10
$51 11
.
.
1 4
2 8
3 12
4 17
5 21
6 25
7 29
O8 36
Total sum score Total sum score Risk of AUR (%)
#18 0e5
.
19e26 O5e10
27e30 O10e15
31e34 O15e20
35e39 O20e30
40e43 O30e40
44e48 O40e50
. ___D . $49 O50
IPSS 5 international prostate symptom score; HT 5 hormonal treatment; AUR 5 acute urinary retention. For each individual patient, the risk of AUR can also be calculated by applying the following formulas: Linear predictor 5 4.84 þ (0.021 prostate volume) þ (0.538 IPSS_1a) þ (1.332 IPSS_2b) þ (1.677 IPSS_3c) þ (0.035 HT) þ (0.427 prostate protrusion). Risk of AUR 5 1/(1 þ exp(linear predictor)) 100%. For example, the linear predictor value of a patient with a prostate volume of 47 cm3, an IPSS score of 15, no neoadjuvant HT, and 3 mm prostate protrusion is 1.24 [4.84 þ (0.021 47) þ (1.332 1) þ (0 HT) þ (0.427 3) 5 1.24]. The corresponding calculated risk of AUR is then 22% [1/1 þ exp(1.24)]. a IPSS_1 5 1, if IPSS 6e10. b IPSS_2 5 1, if IPSS 11e20. c IPSS_3 5 1, if IPSS O20.
External validation of the pretreatment nomogram to predict acute urinary retention after 125I prostate brachytherapy Ellen M. Roeloffzen1,*, Juanita Crook2, Evelyn M. Monninkhof3, Michael McLean4, Marco van Vulpen1, Elantholi P. Saibishkumar4 1 Department of Radiation Oncology, University Medical Center Utrecht, Utrecht, The Netherlands Department of Radiation Oncology, BCCA Center for the Southern Interior/University of British Columbia, Kelowna, BC, Canada 3 Department of Epidemiology, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands 4 Department of Radiation Oncology, Princess Margaret Hospital/Univeristy of Toronto, Toronto, ON, Canada 2
ABSTRACT
PURPOSE: Acute urinary retention (AUR) after 125I prostate brachytherapy has a negative impact on quality of life. Recently, the authors developed a nomogram to predict the risk of AUR preoperatively. The aim of this study was to assess the external validity of the nomogram. METHODS AND MATERIALS: The nomogram was initially developed on 714 patients treated with 125I prostate brachytherapy at the University Medical Center Utrecht, the Netherlands. Predictive factors included in the nomogram were prostate volume, international prostate symptom score, neoadjuvant hormonal treatment, and prostate protrusion. For external validation, the data of 715 consecutive patients treated between January 2003 and July 2008 at the Princess Margaret Hospital, Toronto, were used. The performance of the nomogram was evaluated by discrimination (ability to distinguish between patients who develop AUR yes or no) and calibration (agreement between observed and predicted numbers of AUR). RESULTS: Of the 715 patients treated at the Princess Margaret Hospital, 67 patients (9.4%) developed AUR compared with 8.0% in the University Medical Center Utrecht cohort. In the validation data set, the discriminatory ability of the nomogram was good (receive operating characteristic area: 0.86; 95% confidence interval: 0.82e0.91), and comparable to the derivation data set (receive operating characteristic area: 0.82; 95% confidence interval: 0.77e0.88). Comparison between the predicted risks and the observed frequencies of AUR showed underestimation of the nomogram in the validation data set for high AUR risks values. Still, the negative predictive value for the risk of AUR, using a cutoff value of 5%, was high (98.1%). CONCLUSION: External validation of the nomogram shows adequate discrimination of patients with and without AUR. Therefore, the nomogram can aid in individualized treatment decision making. Ó 2012 American Brachytherapy Society. Published by Elsevier Inc. All rights reserved.
Keywords:
Prostate cancer; Acute urinary retention; Prostate brachytherapy; Predictive nomogram; External validation
Introduction The most predominant severe acute toxicity after prostate brachytherapy is acute urinary retention (AUR). Received 10 August 2011; received in revised form 19 December 2011; accepted 28 December 2011. The research was conducted at the Princess Margaret Hospital, Toronto, ON, Canada and at University Medical Center Utrecht, Utrecht, The Netherlands. The authors declare no conflicts of interest. * Corresponding author. Department of Radiation Oncology, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands. Tel.: þ31-887558800; fax: þ31-887555850. E-mail address: [email protected] (E.M. Roeloffzen).
Published AUR rates vary from 6% to 34% (1e5). It is known that AUR negatively influences quality of life (6, 7). Preoperative prediction of AUR is useful for patient counseling and for clinical decision making in patients with localized prostate cancer. In a previous study, the authors developed a clinical nomogram to predict the risk of AUR after 125I prostate brachytherapy preoperatively, using the data of 714 consecutive patients treated at their center (Appendix) (8). The nomogram was based on the most important pretreatment risk factors for AUR, that is, prostate volume, international prostate symptom score (IPSS), neoadjuvant hormonal treatment (HT), and the extent of prostate protrusion into
1538-4721/$ - see front matter Ó 2012 American Brachytherapy Society. Published by Elsevier Inc. All rights reserved. doi:10.1016/j.brachy.2011.12.011
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
the bladder. Both calibration and discrimination were adequate (receiver operating characteristic [ROC] area: 0.82). The nomogram showed that among patients with a very low sum score (!18 points), the risk of AUR was only 0e5%, and that in patients with a high sum score (O35 points), the risk of AUR was more than 20%. However, as the nomogram was based on single-center data and patient selection and treatment techniques may differ between centers, its predictive value in other patient populations is unknown (9e11). The aim of this study was to perform external validation of the nomogram, by using data of patients treated at the Princess Margaret Hospital (PMH) in Toronto, Canada.
Methods and materials Patients The derivation study population consisted of 714 consecutive patients with localized prostate cancer treated with 125I seed implantation between January 2005 and December 2008 at the University Medical Center Utrecht (UMCU), the Netherlands (8). The validation population consisted of 715 consecutive patients with localized prostate cancer treated with 125I seed implantation between January 2003 and July 2008 at the PMH, Toronto, Canada. The PMH was chosen for external validation for two main reasons: (1) MR imaging for postimplant dose evaluation is performed, which is required for adequate determination of prostate protrusion (8, 12, 13), (2) it is a high-volume center with meticulous followup and documentation of toxicity. The implantation techniques and dosimetric analyses were similar and according to the guidelines of Groupe-Europeen de Curietherapie-European Society for Therapeutic Radiology and Oncology and American Brachytherapy Society (14, 15). Table 1 summarizes the similarities and differences in 125I prostate brachytherapy procedures between the centers. The UMCU and the PMH brachytherapy procedures have both been extensively described previously (4, 6, 16, 17). All patients were treated in lithotomy position. The radioactive seeds were inserted transperineally according to the preplan in a modified peripherally loaded Seattle technique (4). All implants were evaluated at 1 month, by using CT and 1.5 or 3.0 T MRI fusion. Implant quality was defined in terms of the standard dosimetric parameters D90, V100, V150, and V200 (14, 15). Urinary function was assessed using IPSS questionnaires, which were completed at baseline and at each followup visit. AUR was defined as any need for urinary catheterization within 3 months after implantation (18). Research Ethics Board approval was obtained to access the data from the PMH prospective database. A consecutive cohort of patients was selected between January 2003 and July 2008, ensuring adequate patient numbers and a substantial followup. Baseline characteristics and post-treatment sequelae were retrieved, including the dates and duration of retention and catheterization.
257
Determination of prostate protrusion The extent of prostate protrusion into the bladder was recently shown to be a strong independent predictor of AUR (8, 13). It relates to the large median lobes and was defined as the maximum distance from bladder base to prostate base (13). The extent of prostate protrusion was determined retrospectively on sagital MR images at 1 month after implantation for all patients at PMH (Fig. 1). All delineations were performed by the same physician (EMR), who was blinded to patient’s AUR status. The authors previous study (8) showed that the inter- and intraobserver variability of prostate protrusion measurements were good (i.e., 0.7 mm [standard deviation 0.9] and 0.4 mm [standard deviation 0.7], respectively). Pearson correlation coefficients (r) were calculated and showed that both inter- and intraobserver repeatability of prostate protrusion measurements were high (r 5 0.97 and r 5 0.94, respectively). Statistical analysis Patient characteristics of the UMCU and PMH were compared using independent sample t tests (continuous variables) or c2 tests (categorical or dichotomous variables). Multivariate logistic regression analysis was performed to explore the predictive values of the predefined predictors (i.e., prostate volume, IPSS, neoadjuvant HT, and the extent of prostate protrusion) of AUR in the validation data set (8). Proper validation requires the use of the fully specified existing prognostic model to predict outcomes for the patients in the validation data set and then compare these predictions with the patients’ actual outcomes. Therefore, the risk of AUR was calculated for each individual patient in the validation data set using the following equation (Appendix) (8): Risk of AUR 5 1=½1 þ expðlinear predictorÞ 100% The predictive accuracy of the nomogram was quantified using discrimination and calibration measures. Differences in discriminative ability (i.e., the ability of the model to distinguish patients who develop AUR: yes or no) between the derivation and validation model were quantified by the area under the ROC curve (ROC area). The ROC area may theoretically range from 0.5 (discrimination equivalent to that of chance) to 1.0 (perfect discrimination). Calibration of the model (i.e., agreement between observed and predicted numbers of AUR) was determined by comparing the predicted and the observed numbers of AUR among five risk groups. In addition, calibration was statistically tested across deciles of predicted risks with the HosmereLemeshow test, where an insignificant test indicates good model fit (11). Furthermore, the negative predictive value using a cutoff value of 5% was computed. The 5% cutoff value was chosen because the AUR rate in our population was 8.0% and a reduction in AUR rate is aimed for.
258 Table 1 Summery of the
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264 125
I prostate brachytherapy implantation protocols at the UMCU and the PMH
Characteristics
UMCU derivation data set
PMH validation data set
Eligibility criteria Age at implantation Clinical tumor stage Initial PSA (ng/mL) Gleason sum score Prostate volume (cc) Neoadjuvant HT
Any T1eT2 !20 #7 #50 Prostate volume O50 cc
Any T1eT2 !10; (10e20 under RTOG protocol) #6; (7 under RTOG protocol) #60 Prostate volume O50 cca
Dosimetry goals Prescribed dose (Gy) Prostate D90 (Gy) Prostate V100 (%) Prostate V150 (%) Prostate V200 (%) Urethra dose Rectal dose
145 O160 O95 66 33 #150% of prescribed dose #100% of prescribed dose
145 170e180 O99 54e60 12e20 #150% of prescribed dose No preplan goals
Imaging 3D-TRUS Fluoroscopy MRI CT
Before and after needle insertion (real time) After implantation Preimplant and 4 weeks postimplant (1.5 or 3 T) 4 weeks postimplant
Before and after needle insertion (real time) After each row of needles 4 weeks post-implant (1.5 T) 4 weeks postimplant
MRI based Spinal 125 I 0.51 U
TRUS based General 125 I 0.4 U
Stranded (50%) or loose (50%) No Taken out immediately after procedure No For 1 month; and as long as symptoms persisted
Stranded (17%) or loose (82%) No Taken out immediately after procedure No Start 1 week before to 3 months after implant; and as long as symptoms persisted
At 1, 3, 6, 9, and 12 months and twice annually thereafter Alternately by radiation oncologist and urologist AUA, toxicity scores, intervention data 6 months
At 1, 3, 6, 9, and 12 months, and twice annually thereafter Radiation oncologist AUA, toxicity scores, intervention data 6 months
Procedure Preplanning Anesthesia Source type Average seed activity (1U 5 1 mGym2 h1) Type of seeds Supplemental EBRT Postimplant catheter Routine steroids a-blocker use Followup Frequency Physician Documentation Minimum followup
PSA 5 prostate-specific antigen; HT 5 hormonal treatment; TRUS 5 transrectal ultrasound. a Initially, neoadjuvant HT for downsizing of the prostate was given to patients with prostate volumes O50 cc; however, the use of HT diminished over time after reports on increased urinary retention with this approach and also generalized toxicity of HT.
A commercial statistical package of social sciences (SPSS version 16.0; SPSS, Chicago, IL) and R was used for statistical analysis of the data.
Results The AUR rates at the UMCU and the PMH were 8.0% and 9.4%, respectively. The median time to AUR was 10 days (range: 0e90) at the UMCU and 1 day (range: 0e90) at the PMH. The median duration of catheter dependency was 37 days (range: 2e140 days) and 10 days (range: 1e360), respectively. Table 2 summarizes the clinical and treatment characteristics of patients treated at the UMCU (derivation data set) and the PMH (validation data set). The data sets were
largely comparable, except for neoadjuvant HT, Gleason score, and initial prostate-specific antigen values, which were all lower in the validation data set (owing to local treatment policy and provincial eligibility criteria). The mean number of implanted needles and seeds were higher in patients treated at the PMH, probably owing to the lower average seed strength. Like at the UMCU (8), patients who developed AUR at the PMH had significantly higher pretreatment prostate volumes ( p! 0.001), higher IPSS ( p 5 0.001), larger extents of prostate protrusion ( p! 0.001), and were more frequently treated with neoadjuvant HT ( p 5 0.05) compared with patients without AUR (Table 3). Although statistically significant, the differences in age, prostate D90, V150 and V200 between AUR and no-AUR patients are not considered clinically relevant. Multivariate logistic
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
259
Table 2 Clinical and treatment characteristics for patients treated at the UMCU and patients treated at the PMH
Fig. 1. Illustration of determination of the extent of prostate protrusion into the bladder (defined as the maximum distance from bladder base to prostate base). Delineation: red 5 prostate (target volume); yellow 5 urethra; blue 5 bladder; green 5 construction lines.
regression analyses of the derivation data set and the validation data set are shown in Table 4. The odds ratios between the derivation and validation data set are largely comparable, except for prostate protrusion (higher predictive value in the validation dataset). The discriminative value of the nomogram in the validation data set was high (ROC area: 0.86; 95% confidence interval: 0.82e0.91), which is comparable with the discrimination in the derivation data set (ROC area: 0.82; 95% confidence interval: 0.77e0.88). In Fig. 2, the ROC curve for the validation data set is shown. Figure 3 shows the calibration plot. The dotted line shows the ideal and the solid line shows the association between the predicted risk and the observed frequency of AUR. The nomogram underestimated the risk of AUR in the validation data set for high-AUR risk values. The Hosmere Lemeshow test was statistically significant ( p !0.001; c2 5 47.5), which also indicates poor calibration. However, 70% of the patients had an AUR risk of !5%, and for risks !5% the calibration was sufficient. Moreover, the negative predictive value for AUR (using a clinically useful cutoff value of 5%) was high (98.1%), indicating that for a predicted AUR risk of less than 5%, indeed 98.1% of patients did not develop AUR.
Discussion External validation of clinical prediction tools is important. Accurate predictions in patients that were used to
Characteristic
UMCU (n 5 714)
PMH (n 5 715)
Pretreatment Age at implantation (y)
65.1 6.4
62.2 6.9
Clinical tumor stage, n (%) T1 T2
498 (70) 216 (30)
481 (67) 234 (33)
Gleason sum score !7, n (%) 7, n (%) iPSA (ng/mL)
548 (77) 166 (23) 9.8 5.3
668 (93) 47 (7) 5.5 2.4
Pretreatment IPSS, n (%) 0 6 11 O20
257 199 243 15
393 195 117 10
Pretreatment TURP, n (%) Yes No
11 (1.5) 703 (98.5)
1 (0.1) 714 (99.9)
Neoadjuvant HT Yes, n (%) No, n (%) Pretreatment prostate volume (cm3) Prostate length (cm) Prostate protrusion (mm)
137 (19) 577 (81) 35.4 9.0 4.2 0.6 1.1 1.9
24 (3) 691 (97) 35.1 10.6 3.8 0.4 1.0 1.3
Treatment Needles (n) Seeds (n) Prostate D90 (Gy) Prostate V100 (%) Prostate V150 (%) Prostate V200 (%)
24.1 3.8 73.3 13.1 164.1 28.5 93.1 6.7 71.3 11.6 39.1 11.7
30.5 4.3 105.3 16.6 167.0 18.4 95.3 4.2 60.7 10.5 29.3 8.1
(36) (28) (34) (2)
(55) (27) (16) (1)
UMCU 5 university medical center Utrecht; PMH 5Princess Margaret Hospital; n.s. 5 not statistically significant; iPSA 5 initial prostate-specific antigen level; IPSS 5 international prostate symptom score; TURP 5 transurethral resection of the prostate; HT 5 hormonal treatment; D90 5 minimal dose received by 90% of prostate; V100, V150, V200 5 percentage of prostate/ urethra volume receiving 100%, 150% and 200% of prescribed dose, respectively; SD 5 standard deviation. Values are means (SD) or n (%)
develop a nomogram are no guarantee for good predictions in other patient populations (9e11). Only after external validation, a model is considered generally applicable. In this study, the authors found accurate discrimination for AUR of our nomogram when tested in the validation population (ROC area: 0.86). Calibration was sufficient for low AUR risks, but poor for high AUR risks. Because the negative predictive value for AUR at a cutoff value of 5% was high (98.1%), the nomogram is able to correctly identify low-risk patients, but for a reliable estimation of risks, higher than 5% calibration of the nomogram should be improved. Theoretically, differences in patient and treatment techniques might influence the risk of AUR. The authors did not find a major difference in AUR rate between the UMCU
260
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
Table 3 Clinical and treatment characteristics for patients who developed AUR and patients who did not (no AUR), according to clinical center UMCU
PMH
Characteristics
AUR (n 5 57)
No AUR (n 5 657)
p-Value
AUR (n 5 67)
No AUR (n 5 648)
p-Value
Pretreatment Age at implantation (y)
65.8 6.6
65.0 6.3
NS
63.9 6.7
62.0 6.9
0.031
Clinical tumor stage, n (%) T1 T2
41 (72) 16 (28)
457 (70) 200 (30)
NS
45 (67) 22 (33)
436 (67) 212 (33)
NS
Gleason sum score !7, n (%) 7, n (%)
44 (77) 13 (23)
504 (77) 153 (23)
NS
64 (96) 3 (4)
604 (93) 44 (7)
NS
iPSA (ng/mL)
9.2 4.2
9.9 5.3
NS
5.6 2.3
5.5 2.4
NS
Pretreatment IPSS, n (%) 0e5 6e10 11e20 O20
8 (14) 14 (25) 32 (56) 3 (5)
249 (38) 185 (28) 211 (32) 12 (2)
!0.001
24 (36) 24 (36) 17 (25) 2 (3)
369 (57) 171 (26) 100 (15) 8 (1)
0.001
Pretreatment TURP, n (%) Yes No
1 (2) 56 (98)
10 (2) 647 (98)
NS
0 (0) 67 (100)
1 (0.2) 647 (99.8)
NS
Neoadjuvant HT, n (%) Yes No
16 (28) 41 (72)
121 (18) 536 (82)
NS 0.076
5 (7.5) 62 (92.5)
19 (3) 629 (97)
0.05
Prostate volume, TRUS (cm3)
38.9 9.1
35.1 8.9
0.002
42.2 11.5
34.4 10.3
!0.001
Prostate length (cm)
4.2 0.6
4.2 0.6
NS
4.01 0.45
3.77 0.42
NS
Prostate protrusion (mm)
3.5 2.9
0.9 1.6
!0.001
3.1 1.6
0.8 1.1
!0.001
Treatment Needles (n) Seeds (n) Prostate D90 (Gy) Prostate V100 (%) Prostate V150 (%) Prostate V200 (%)
25.3 3.5 78.0 11.7 161.6 27.4 91.5 12.3 69.5 13.1 37.4 9.7
24.0 3.8 72.9 13.2 164.3 28.6 93.2 6.0 71.5 11.5 39.3 11.9
0.013 0.005 NS NS NS NS
32.6 4.4 116.3 16.4 161.4 16.9 94.5 4.2 57.7 9.7 27.0 7.7
30.3 4.3 104.1 16.2 167.6 18.5 95.4 4.2 61.0 10.6 29.5 8.2
!0.001 !0.001 0.009 NS 0.016 0.015
UMCU 5 university medical center Utrecht; PMH 5 Princess Margaret Hospital; AUR 5 acute urinary retention; NS 5 not statistically significant; iPSA 5 initial prostate-specific antigen level; IPSS 5 international prostate symptom score; TURP 5 transurethral resection of the prostate; HT 5 hormonal treatment; TRUS 5 transrectal ultrasound; D90 5 minimal dose received by 90% of prostate; V100, V150, V200 5 percentage of prostate/urethra volume receiving 100%, 150% and 200% of prescribed dose, respectively; SD 5 standard deviation. Values are means (SD) or n (%).
and PMH (8.0% vs. 9.4%). Patients at the PMH had lower mean initial prostate-specific antigen levels, lower Gleason scores, and were less frequently treated with neoadjuvant HT (Table 2), which was because of the local treatment policy and provincial eligibility criteria for 125I prostate brachytherapy (Table 1). The higher number of implanted needles and seeds in patients treated at the PMH (Table 2) might be explained by the lower average seed strength used because the U per cc of prostate volume was approximately the same for both centers (0.015 UMCU vs. 0.011 PMH). Although some minor differences in patient and treatment characteristics existed, these differences did not influence the discrimination of the nomogram. This implies that the nomogram might also be applicable to patients treated at other centers. Furthermore, the high ROC value indicates that the nomogram indeed
contained the most important predictors of AUR. An extensive evaluation of these risk factors in the context of recent literature was performed in the authors’ previous article (8). The slightly higher ROC value in the validation population compared with that of the derivation population (0.86 vs. 0.82) might be explained by the more heterogeneous validation population. A more heterogeneous sample according to the included predictors has been shown to be related to a higher discriminative ability because the model can distinguish more subjects with very low or very high predictions (19). When comparing the predicted risks with the observed frequencies of AUR (calibration), the nomogram underestimated the risk of AUR in the validation population for high-AUR risk values (Fig. 3). Several explanations are available for this finding: (1) The higher incidence of
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
261
Table 4 Multivariate logistic regression analysis for the prediction of AUR after prostate brachytherapy in the UMCU and PMH data set UMCU (derivation data set)
PMH (validation data set)
Factor
OR
95% CI
p
Prostate volume (cm3)
1.02
0.99e1.06
0.243
Pretreatment IPSS 0e5 6e10 11e20 O20
1.78 4.18 6.05
0.69e4.64 1.78e9.81 1.18e30.9
Neoadjuvant HT (yes/no)
1.04
0.52e2.07
0.914
Prostate protrusion (mm)
1.58
1.40e1.79
0.000
b coefficient
a
OR
95% CI
p
0.021
1.00
0.97e1.03
0.916
0.538 1.332 1.677
2.03 1.86 8.62
1.01e4.08 0.81e4.27 1.45e51.4
0.035
0.71
0.18e2.76
0.617
0.427
3.13
2.42e4.06
0.000
0.003
Intercept 2Log likelihood ROC area
b coefficient 0.002
0.043
4.84 284.4 0.82
0.707 0.619 2.154 0.348 1.141 4.69 293.5 0.86
AUR 5 acute urinary retention; UMCU 5 university medical center Utrecht; PMH 5 Princess Margaret Hospital; OR 5 odds ratio; CI 5 confidence interval; IPSS 5 international prostate symptom score; HT 5 hormonal treatment; ROC 5 receiver operating characteristic. a b coefficients were multiplied by a shrinkage factor of 0.93, to adjust for optimism that might be expected when the model is applied to new, but similar patients. The intercept was also adjusted to the new situation.
patients with AUR in the validation data set. Although the absolute difference was only 1.4%, the relative difference was 18%. (2) The slight differences in patient and treatment characteristics might influence calibration measures. At the PMH, acceptance for brachytherapy and prediction of retention was based mostly on urinary voiding study results, specifically peak flow rate. There is evidence in literature that peak flow rate is a powerful predictor of AUR (20, 21). Because patients with low-peak flow rates (!10 mL/s) were not necessarily excluded for brachytherapy when they were aware and willing to accept the high-AUR risk, this might be an explanation for the underestimation for high-AUR risk values in the validation population. In ongoing research, the authors will further
explore the relation between peak flow rate and the risk of AUR. (3) The low number of patients with an AUR risk of O5% (70% of the patients had an AUR risk of !5%). In Fig. 3, it can be seen that predictions up to 5% are adequate. Therefore, the nomogram can be safely used for risk scores up to 5%. However, caution is warranted for interpretation of risk scores greater than 5%. Because the AUR rate in our population was 8.0% and a reduction in AUR rate is aimed for, a cutoff value of 5% is reasonable. They showed that the negative predictive value for AUR, using a cutoff value of 5%, was high (98.1%). This indicates that for a predicted AUR risk of !5%, 98.1% of patients indeed did not develop AUR. In future, by expanding the number of patients included in the analysis, the authors hope to
Fig. 2. ROC curve for the predictability of acute urinary retention in the derivation data set, based on pretreatment prostate volume, International Prostate Symptom Score, neoadjuvant hormonal treatment, and prostate protrusion. ROC 5 Receiver operating characteristic.
Fig. 3. Calibration plot. The continuous line shows the relation between observed frequencies and predicted probabilities. The dotted line indicates perfect calibration, that is, observed frequencies and predicted probabilities are in complete agreement.
262
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
improve the calibration and predictive value for high-risk patients. The fundamental question is ‘‘how much does this nomogram really add?’’ Based on clinical experience of a radiation oncologist, the additive value of the nomogram for very low-risk patients (small volume, low IPSS, and no HT) might be modest. However, this nomogram provides a more accurate and validated risk assessment to individual patients when discussing brachytherapy as one of their treatment options. Furthermore, the model also includes prostate protrusion as an independent risk factor and the nomogram can easily be used by less-experienced radiation oncologists as well. Although the authors would have a better predictive value for high-risk patients, the decision on where to place the cutoff value for an ‘‘unacceptable’’ high risk of AUR remains unclear and is dependent on the expert opinion of the physician and the patient’s individual preference. A ‘‘high’’ risk of AUR does not necessarily imply to withhold patients from brachytherapy. However, in patients with a very high risk of retention, alternative treatment options might be considered. The risk of AUR still has to be weighed against the toxicity profile of other treatment options. Conversely, men with a favorable risk score may be more confident in their decision to proceed with seed implants and ultimately the overall rate of retention could decrease in time with appropriate patient selection. Furthermore, in the last few years, strategies of active surveillance have emerged, which is considered as a reasonable option for highly selected patients with low-risk prostate cancer (22, 23). Recent studies suggest that outcomes after direct treatment or active surveillance combined with delayed treatment are similar; however, long-term followup is still needed (22). A disadvantage of active surveillance is the possible anxiety and distress for patients of withholding radical treatment (24). There are some limitations of this study, which should be mentioned. First, the model requires MRI scans to determine the extent of prostate protrusion; however, unfortunately, MRI is not available at all brachytherapy centers. Although MRI is preferable for delineation because of its high soft tissue contrast (12); in the ongoing research, the authors will further explore the predictive value of prostate protrusion measured on TRUS to allow wide use of the nomogram. Second, at the PMH, prostate protrusion was measured at 1 month after implantation instead of before implantation like at the UMCU. In one of our previous publications, the authors showed that the edema factor had no predictive value for the development of AUR (25). This might be explained by the fact that in most patients, prostate swelling has resolved at 4 weeks after implantation (26, 27). Therefore, they do not expect that the difference in time of prostate protrusion measurement has any influence on the present study outcome. Third, Vergouwe et al. (9) recommended at least 100 events for external validation to obtain enough power
(80%). However, this is the ideal situation, but may not be feasible clinically, especially when the event rate is low. In literature, sample sizes of validation sets differ over a wide range. In a review of Altman and Royston (28), validation samples with sizes varying between 52 and 479 patients were described, and the number of events ranged from 24 to 115. Therefore, by including 715 patients and 67 events, our validation population can be considered adequate compared with other published validation studies. Fourth, according to the state of the art methodology guidelines (11), our nomogram is based on a limited set of predictors at multivariate analyses from reviewed literature. However, more potential predictors have been described in single studies or on univariate analysis only. Further research might lead to an extension and improvement of the performance of the model by addition of other relevant risk factors. Updating of the nomogram might lead to improvement of calibration measures (10). Continued efforts to refine and establish validity of the nomogram across worldwide patient populations are thus needed to realize the goal of assisting men and their care providers in appraising risks and making individualized treatment choices. Nevertheless, the authors showed that in two individual patient cohorts of more than 700 patients each, the model performs reasonably well.
Conclusions External validation of the nomogram shows adequate discrimination for the risk of AUR. The nomogram is able to correctly identify low risk patients with a negative predictive value of 98%. Therefore, the nomogram can be widely used to predict the risk of AUR after 125I prostate brachytherapy. Because the balance between treatment outcome and quality of life is considered very important nowadays, the nomogram might be a useful tool for physicians and patients in individualized treatment decision making in low-risk prostate cancer. References [1] Terk MD, Stock RG, Stone NN. Identification of patients at increased risk for prolonged urinary retention following radioactive seed implantation of the prostate. J Urol 1998;160:1379e1382. [2] Lee N, Wuu CS, Brody R, et al. Factors predicting for postimplantation urinary retention after permanent prostate brachytherapy. Int J Radiat Oncol Biol Phys 2000;48:1457e1460. [3] Bucci J, Morris WJ, Keyes M, et al. Predictive factors of urinary retention following prostate brachytherapy. Int J Radiat Oncol Biol Phys 2002;53:91e98. [4] Crook J, McLean M, Catton C, et al. Factors influencing risk of acute urinary retention after TRUS-guided permanent prostate seed implantation. Int J Radiat Oncol Biol Phys 2002;52:453e460. [5] Locke J, Ellis W, Wallner K, et al. Risk factors for acute urinary retention requiring temporary intermittent catheterization after prostate brachytherapy: a prospective study. Int J Radiat Oncol Biol Phys 2002;52:712e719.
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264 [6] Roeloffzen EM, Hinnen KA, Battermann JJ, et al. The impact of acute urinary retention after iodine-125 prostate brachytherapy on healthrelated quality of life. Int J Radiat Oncol Biol Phys 2010;77:1322e1328. [7] Crook J, Fleshner N, Roberts C, et al. Long-term urinary sequelae following 125iodine prostate brachytherapy. J Urol 2008;179:141e145. [8] Roeloffzen EM, Vulpen van M, Battermann JJ, et al. Pretreatment nomogram to predict the risk of acute urinary retention after I-125 prostate brachytherapy. Int J Radiat Oncol Biol Phys 2011;81:737e744. [9] Vergouwe Y, Steyerberg EW, Eijkemans MJ, et al. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol 2005;58:475e483. [10] Toll DB, Janssen KJ, Vergouwe Y, et al. Validation, updating and impact of clinical prediction rules: A review. J Clin Epidemiol 2008;61:1085e1094. [11] Royston P, Moons KG, Altman DG, et al. Prognosis and prognostic research: Developing a prognostic model. BMJ 2009;338:b604. [12] Villeirs GM, De Meerleer GO. Magnetic resonance imaging (MRI) anatomy of the prostate and application of MRI in radiotherapy planning. Eur J Radiol 2007;63:361e368. [13] Roeloffzen EM, Monninkhof EM, Battermann JJ, et al. Acute urinary retention after I-125 prostate brachytherapy in relation to dose in different regions of the prostate. Int J Radiat Oncol Biol Phys 2010;80:76e84. [14] Ash D, Flynn A, Battermann J, et al. ESTRO/EAU/EORTC recommendations on permanent seed implantation for localized prostate cancer. Radiother Oncol 2000;57:315e321. [15] Nag S, Beyer D, Friedland J, et al. American Brachytherapy Society (ABS) recommendations for transperineal permanent brachytherapy of prostate cancer. Int J Radiat Oncol Biol Phys 1999;44:789e799. [16] Battermann JJ, Boon TA, Moerland MA. Results of permanent prostate brachytherapy, 13 years of experience at a single institution. Radiother Oncol 2004;71:23e28. [17] Crook J, Toi A, McLean M, Pond G. The utility of transition zone index in predicting acute urinary morbidity after 125I prostate brachytherapy. Brachytherapy 2002;1:131e137.
263
[18] Trotti A. The evolution and application of toxicity criteria. Semin Radiat Oncol 2002;12:1e3. [19] Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: Use of benchmark values to disentangle a casemix effect from incorrect coefficients. Am J Epidemiol 2010; 172:971e980. [20] Martens C, Pond G, Webster D, et al. Relationship of the International Prostate Symptom score with urinary flow studies, and catheterization rates following 125I prostate brachytherapy. Brachytherapy 2006;5: 9e13. [21] Ikeda T, Shinohara K. Peak flow rate is the best predictor of acute urinary retention following prostate brachytherapy: our experience and literature review. Int J Urol 2009;16:558e560. [22] Van den Bergh RC, Vasarainen H, van der Poel HG, et al. Short-term outcomes of the prospective multicentre ‘‘Prostate Cancer Research International: Active Surveillance’ study. BJU Int 2010;105: 956e962. [23] Klotz L, Zhang L, Lam A, et al. Clinical results of long-term followup of a large, active surveillance cohort with localized prostate cancer. J Clin Oncol 2009;28:126e131. [24] Van den Bergh RC, Essink-Bot L, Roobol MJ, et al. Do anxiety and distress increase during active surveillance for low risk prostate cancer? J Urol 2010;183:1786e1791. [25] Roeloffzen EM, Battermann JJ, Van Deursen MJ, et al. The influence of dose on the risk of acute urinary retention after I-125 prostate brachytherapy. Int J Radiat Oncol Biol Phys 2010;80:76e84. [26] Taussky D, Austen L, Toi A, et al. Sequential evaluation of prostate edema after permanent seed prostate brachytherapy using CT-MRI fusion. Int J Radiat Oncol Biol Phys 2005;62:974e980. [27] Waterman FM, Yue N, Corn BW, et al. Edema associated with I-125 or Pd-103 prostate brachytherapy and its impact on post-implant dosimetry: an analysis based on serial CT acquisition. Int J Radiat Oncol Biol Phys 1998;41:1069e1077. [28] Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med 2000;19:453e473.
264
E.M. Roeloffzen et al. / Brachytherapy 11 (2012) 256e264
Appendix Nomogram to predict the risk of acute urinary retention after
125
I prostate brachytherapy
Predictor Prostate volume (cm3) Value #16 Score 3
17e21 4
22e25 5
26e30 6
IPSS Value Score
0e5 0
6e10 5
11e20 13
O20 17
Neoadjuvant HT Value Score
No 0
Yes 1
Prostate protrusion (mm) Value 0 Score 0
31e35 7
36e40 8
41e45 9
46e50 10
$51 11
.
.
1 4
2 8
3 12
4 17
5 21
6 25
7 29
O8 36
Total sum score Total sum score Risk of AUR (%)
#18 0e5
.
19e26 O5e10
27e30 O10e15
31e34 O15e20
35e39 O20e30
40e43 O30e40
44e48 O40e50
. ___D . $49 O50
IPSS 5 international prostate symptom score; HT 5 hormonal treatment; AUR 5 acute urinary retention. For each individual patient, the risk of AUR can also be calculated by applying the following formulas: Linear predictor 5 4.84 þ (0.021 prostate volume) þ (0.538 IPSS_1a) þ (1.332 IPSS_2b) þ (1.677 IPSS_3c) þ (0.035 HT) þ (0.427 prostate protrusion). Risk of AUR 5 1/(1 þ exp(linear predictor)) 100%. For example, the linear predictor value of a patient with a prostate volume of 47 cm3, an IPSS score of 15, no neoadjuvant HT, and 3 mm prostate protrusion is 1.24 [4.84 þ (0.021 47) þ (1.332 1) þ (0 HT) þ (0.427 3) 5 1.24]. The corresponding calculated risk of AUR is then 22% [1/1 þ exp(1.24)]. a IPSS_1 5 1, if IPSS 6e10. b IPSS_2 5 1, if IPSS 11e20. c IPSS_3 5 1, if IPSS O20.