- Email: [email protected]
Application of Prostate Imaging Reporting and Data System Version 2 (PI-RADS v2)
ARTICLE IN PRESS
Original Investigation
Application of Prostate Imaging Reporting and Data System Version 2 (PI-RADS v2): Interobserver Agreement and Positive Predictive Value for Localization of Intermediate- and High-Grade Prostate Cancers on Multiparametric Magnetic Resonance Imaging Frank Chen, MD, Steven Cen, PhD, Suzanne Palmer, MD Rationale and Objectives: To evaluate interobserver agreement with the use of and the positive predictive value (PPV) of Prostate Imaging Reporting and Data System version 2 (PI-RADS v2) for the localization of intermediate- and high-grade prostate cancers on multiparametric magnetic resonance imaging (mpMRI). Materials and Methods: In this retrospective, institutional review board-approved study, 131 consecutive patients who had mpMRI followed by transrectal ultrasound-MR imaging fusion-guided biopsy of the prostate were included. Two readers who were blinded to initial mpMRI reports, clinical data, and pathologic outcomes reviewed the MR images, identified all prostate lesions, and scored each lesion based on the PI-RADS v2. Interobserver agreement was assessed by intraclass correlation coefficient (ICC), and PPV was calculated for each PI-RADS category. Results: PI-RADS v2 was found to have a moderate level of interobserver agreement between two readers of varying experience, with ICC of 0.74, 0.72, and 0.67 for all lesions, peripheral zone lesions, and transitional zone lesions, respectively. Despite only moderate interobserver agreement, the calculated PPV in the detection of intermediate- and high-grade prostate cancers for each PI-RADS category was very similar between the two readers, with approximate PPV of 0%, 12%, 64%, and 87% for PI-RADS categories 2, 3, 4, and 5, respectively. Conclusions: In our study, PI-RADS v2 has only moderate interobserver agreement, a similar finding in studies of the original PIRADS and in initial studies of PI-RADS v2. Despite this, PI-RADS v2 appears to be a useful system to predict significant prostate cancer, with PI-RADS scores correlating well with the likelihood of intermediate- and high-grade cancers. Key Words: Prostate cancer; multiparametric magnetic resonance imaging; PI-RADS; interobserver agreement; TRUS-MR imaging fusion biopsy. © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.
INTRODUCTION Acad Radiol 2017; ■:■■–■■ From the Keck School of Medicine of USC, 1500 San Pablo St, 2nd Floor Imaging, Los Angeles, CA 90033. Received November 12, 2016; revised January 13, 2017; accepted March 22, 2017. Address correspondence to: F.C. e-mail: [email protected] © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.acra.2017.03.019
I
n an attempt to standardize image interpretation and reporting of multiparametric magnetic resonance imaging (mpMRI) of the prostate, the European Society of Urogenital Radiology published the first Prostate Imaging Reporting and Data System (PI-RADS 1.0) in 2012. PI-RADS 1.0 is based on assessment of T2-weighted MRI, diffusion weighted imaging (DWI), and dynamic contrast enhanced-MRI (DCE-MRI) with 1
CHEN ET AL
or without MR spectroscopy for each lesion according to a 5-point scale and assigning a sum score ranging from 3 to 15 without MR spectroscopy (MRS) and from 4 to 20 with MRS (1). Several studies have validated the original PI-RADS in terms of accuracy; however, interobserver agreement was only moderate (2–6). Recent studies also suggest that assigning a score for T2-weighted imaging (T2WI) to transitional zone lesions and for DWI to peripheral zone lesion is adequate for stratification of patients for further diagnostic workup (6). On the other hand, MRS was rarely used in recent studies, and DCEMRI curve-type analysis does not seem to add significant value in the characterization of prostate lesions (6–8). Based on these new data, the PI-RADS steering committee of the American College of Radiology and the European Society of Urogenital Radiology prostate MRI working group revised PI-RADS and published PI-RADS version 2.0 in early 2015 (9). In this new version of PI-RADS, a primary determinant MRI sequence is used to evaluate each prostate lesion based on location. For peripheral zone lesions, DWI is the dominant sequence, and for transitional zone lesions, T2weighted sequence is the dominant sequence. On DCEMRI, results are scored as positive or negative based on the presence or absence of focal early enhancement, and the previously used curve-type analysis was abandoned. DCEMRI is used strictly for characterization of peripheral zone lesions and is applied only if it makes a clinically relevant difference in cases where a lesion is upgraded from PI-RADS 3 to 4. Finally, a final score of 1–5 is assigned to each prostate lesion based on the revised rules. In order to establish it as the standard for the evaluation of the prostate on mpMRI, studies are needed to evaluate interobserver agreement and test performance of PI-RADS v2. Thus, the purpose of our study is twofold. First, we want to evaluate interobserver agreement with the use of PIRADS v2. Second, we want to evaluate the positive predictive value (PPV) of each PI-RADS category in the detection of intermediate- and high-grade prostate cancers (tumors with Gleason score ≥7). MATERIALS AND METHODS Study Design and Patient Population
This retrospective study was institutional review board approved and Health Insurance Portability and Accountability Act compliant. Informed consent was waived by the review board. Between March 2013 and July 2015, 136 consecutive patients (mean age = 65.6) with abnormal prostate-specific antigen (PSA) (mean PSA = 7.6 µg/mL [1.3–47.4 µg/mL]), abnormal results on digital rectal examination, and/or on active surveillance underwent mpMRI followed by transrectal ultrasound-magnetic resonance (TRUS-MR) imaging fusionguided biopsy of the prostate within 2 months of prebiopsy MRI. Five patients were excluded from the study secondary to nondiagnostic mpMRI studies related to marked susceptibility artifact from hip prostheses. 2
Academic Radiology, Vol ■, No ■■, ■■ 2017
MR Imaging Protocol
All mpMRI examinations were performed on a 3-Tesla MRI system (GE Healthcare, WI, Milwaukee) using a multichannel phased-array coil. The MRI acquisition protocol included T2WI in axial, coronal, and sagittal planes; diffusionweighted MRI with b-values of 0, 600, and 1000 s/mm2; and T1-weighted dynamic contrast-enhanced images. Parametric apparent diffusion coefficient maps were calculated on the MRI console from the diffusion-weighted images (utilizing b-values of 0 and 600 s/mm2) on a voxelwise basis using a monoexponential model. For contrast media, gadobenate dimeglumine (MultiHance; Bracco Diagnostics, Cranbury, NJ) in a weight-adapted standard dose (0.1 mmol/kg) was injected intravenously at 2.5 cc/s. MR imaging sequence parameters are listed in Table 1. Prostate Biopsy
All prostate biopsies were performed by an experienced urologist who has been in practice for 20 years in the outpatient setting. All patients underwent TRUS-MR imaging fusionguided biopsy of MRI suspicious prostate lesions and 10–12 systematic core prostate biopsies. TRUS-MRI fusionguided biopsy was achieved with elastic image fusion and realtime 3D tracking technology (Urostation; Koelis, La Tranche, France) (10,11). Biopsy core specimens were immediately fixed in formalin and stained with hematoxylin and eosin, followed by routine histopathologic evaluation. MR Analysis
MR images were retrospectively and independently reviewed by two radiologists with 17 years (S.P., reader 1) and 4 years (F.C., reader 2) of experience in prostate imaging. The readers were blinded to initial mpMRI reports, clinical data, and pathologic outcomes. Each reader reviewed the MR images, identifying all lesions in each patient suspicious for clinically significant cancer and assigning a score for each lesion based on PI-RADS v2 classification. All assessment was made on a commercial PACS workstation (Synapse PACS; Fujifilm Medical Systems, Stamford, CT). During reader review of images, ovoid regions of interests were placed on the identified lesions on apparent diffusion coefficient (ADC) and T2WI, and MR images were saved in jpeg format in our research database. The saved MR images served as the basis for localization and pathologic correlation. Radiologic-Pathologic Correlation
Lesions identified on mpMRI by each reader were correlated with prostate biopsy results. Not all lesions identified by the readers were described in the initial radiology interpretation prior to biopsy. As a result, not all lesions underwent targeted TRUS-MR imaging fusion-guided biopsy. The MR lesions that did not undergo fusion-guided biopsy were matched
APPLICATION OF PI-RADS V2
7
– – –
to biopsy results based on location of systematic core biopsies and location of the lesion on MRI.
53 Zoom 20 6:07 160 × 160 4 15 Minimum full 3 DCE
DCE, dynamic contrast-enhanced; DWI, diffusion weighted imaging; FOV, field of view; T1WI, T1-weighted imaging; T2WI, T2-weighted imaging.
– – – 5:11 4:56 2:24 320 × 320 224 × 256 128 × 128 4 4 4 3427 500 3800 T2WI T1WI DWI
102 Minimum full Minimum full
180 180 180
Imaging Time Acquisition Matrix
20 20 20
4 1 1 (b value of 0) 12 (b value of 600) 1 (b value of 1000) 1
Zoom Zoom Zoom
Statistical Analysis
Slice Thickness (mm) Flip Angle (°) Echo Time (ms) Repetition Time (ms) Parameter
TABLE 1. Magnetic Resonance Imaging Sequence Parameters
FOV (cm)
NEX
Scan Mode
No. of Repetitions
Repetition Time (s)
Academic Radiology, Vol ■, No ■■, ■■ 2017
Lesions that were identified by both readers were used to evaluate interobserver agreement. The agreement for the PIRADS score between readers was assessed by intraclass correlation coefficient (ICC) with absolute agreement. Using all lesions identified by the readers, PPV with binomial 95% confidence interval (CI) for each PI-RADS score level in classifying intermediate- and high-grade prostate cancers was calculated for each reader. Statistical analysis was performed by a PhD-trained biostatistician (S.C.) who used SAS 9.4 software (SAS Institute, Cary, NC) for all data analysis. RESULTS Lesion Characteristics
Readers 1 and 2 identified PI-RADS 2 lesions related to benign prostatic hypertrophy in all study patients. Besides PIRADS 2 lesions, reader 1 described an additional 115 lesions (90 peripheral zone lesions and 25 transitional zone lesions). Of these 115 lesions, 70 were found to be prostate cancers on histopathology (29 lesions, Gleason score ≤6; 41 lesions, Gleason score ≥7) and the remainder to be benign. Reader 2 described an additional 109 lesions (90 peripheral zone lesions and 19 transitional zone lesions). Of these 109 lesions, 77 were found to be prostate cancers on histopathology (36 lesions, Gleason score ≤6; 41 lesions, Gleason score ≥7) and the remainder to be benign. A total of 8 biopsy-proven intermediateand high-grade prostate cancers (seven Gleason 7 lesions and one Gleason 9 lesion) had no MRI correlate (mean PSA = 8.5 µg/mL [4.2–14.8 µg/mL]). Prostate lesion characteristics are shown in Figure 1 flowchart. Interobserver Agreement
Besides PI-RADS 2 lesions, 73 lesions were described by both readers 1 and 2, 60 lesions in the peripheral zone (49 malignant lesions by histopathology), and 13 lesions in the transitional zone (six malignant lesions by histopathology). Of the 55 cancers, 18 lesions were assigned Gleason scores equal to or less than 6, and 37 lesions were assigned Gleason scores equal to or greater than 7. Among lesions identified by both readers, there was only a moderate level of interobserver agreement between readers 1 and 2 when classifying these lesions using the new PIRADS classification system with ICC of 0.74 (95% CI: 0.60, 0.83). Classification of peripheral zone lesions had a higher degree of interobserver agreement than transitional zone lesions. Figure 2 shows a representative peripheral zone lesion with interobserver variability, and Figure 3 shows a representative transitional zone lesion with interobserver variability. When intermediate- and high-grade prostate cancers were isolated, 3
Academic Radiology, Vol ■, No ■■, ■■ 2017
CHEN ET AL
Figure 1.
Flowchart of lesion characteristics.
(a)
(b)
(c)
Figure 2. Images in a 71-year-old man with prostate-specific antigen of 4.69 ng/mL. Readers identified a right peripheral zone lesion on (a) transverse T2-weighted image, (b) ADC map, and (c) DCE image. For this lesion, reader 1 assigned a PI-RADS score of 4, and reader 2 assigned a PI-RADS score of 3. On TRUS-MR imaging fusion-guided biopsy, the lesion was found to be a Gleason 7 tumor. ADC, apparent diffusion coefficient; DCE, dynamic contrast-enhanced; PI-RADS, Prostate Imaging Reporting and Data System; TRUS-MR, transrectal ultrasound-magnetic resonance.
TABLE 2. Interobserver Agreement—Intraclass Correlation Coefficient Based on Lesion Type Lesion Type All All PZ lesions All TZ lesions All intermediate- and high-grade lesions
Intraclass Correlation Coefficient 0.74 (95% CI: 0.60, 0.83) 0.72 (95% CI: 0.56, 0.83) 0.67 (95% CI: 0.22, 0.89) 0.60 (95% CI: 0.34, 0.78)
CI, confidence interval; PZ, peripheral zone; TZ, transitional zone.
the interobserver agreement decreased, with ICC of 0.60 (95% CI: 0.34, 0.78). Overall ICC values are listed in Table 2. Positive Predictive Value
The calculated PPV in the detection of intermediate- and highprostate cancers for each PI-RADS category was very similar between readers 1 and 2. For both readers, the PPV for PI4
TABLE 3. Positive Predictive Value for Each PI-RADS Score
Reader 1
Reader 2
PI-RADS Score
PPV (%)
2 3 4 5 2 3 4 5
0 13.0 (95% CI: 6.5, 23.8) 64.9 (95% CI: 47.4, 79.3) 88.9 (95% CI: 50.7, 99.4) 0 11.9 (95% CI: 5.3, 23.5) 62.2 (95% CI: 44.8, 77.1) 84.6 (95% CI: 53.7, 97.3)
CI, confidence interval; PI-RADS, Prostate Imaging Reporting and Data System; PPV, positive predictive value.
RADS 2 lesions is 0%, the PPV for PI-RADS 3 lesions is approximately 12% (95% CI: 5%, 24%), for PI-RADS 4 lesions is approximately 64% (95% CI: 45%, 80%), and for PIRADS 5 lesions is approximately 87% (95% CI: 50%, 100%). PPV for each PI-RADS category is listed in Table 3.
Academic Radiology, Vol ■, No ■■, ■■ 2017
(a)
APPLICATION OF PI-RADS V2
(b)
(c)
Figure 3. Images in a 64-year-old man with prostate-specific antigen of 5.9 ng/mL. Readers identified a right transitional zone lesion on (a) transverse T2-weighted image, (b) ADC map, and (c) DCE image. For this lesion, reader 1 assigned a PI-RADS score of 3, and reader 2 assigned a PI-RADS score of 4. On TRUS-MR imaging fusion-guided biopsy, the lesion was found to be a Gleason 7 tumor. ADC, apparent diffusion coefficient; DCE, dynamic contrast-enhanced; PI-RADS, Prostate Imaging Reporting and Data System; TRUS-MR, transrectal ultrasound-magnetic resonance.
DISCUSSION In our study, PI-RADS v2 was found to have only a moderate level of interobserver agreement for two readers of varying experience. The level of interobserver agreement is similar to levels reported for the original PI-RADS and initial studies of PI-RADS v2. Vaché et al., using whole-mount sections of prostatectomy specimens as the reference standard, showed that the original PI-RADS had poor-to-moderate interobserver agreement among three readers of varying experience with weight κ values of 0.378–0.441 (12). Schimmöller et al., using in-bore MRI-guided biopsy as the reference standard, reported a moderate-to-good level of interobserver agreement for the original PI-RADS among three readers with similar experience (T2WI, κ = 0.55; DWI, κ = 0.64; DCE-MRI, κ = 0.65) (13). Renard-Penna et al., using TRUS-MRI fusionguided biopsy as the reference standard, showed a good level of interobserver agreement for the original PI-RADS among two readers with κ = 0.73 (14). In an initial study of PIRADS v2, Muller et al. reported a moderate level of interobserver agreement among five readers of varying experience with a κ = 0.46. Authors of that study used TRUSMRI imaging fusion-guided biopsy as the reference standard (15). In a recently published study, Rosenkrantz et al. reported a moderate level of interobserver agreement among six readers at six different institutions with κ coefficients of 0.593 and 0.509 in the peripheral zone and transitional zone, respectively, for PI-RADS scores of 4 and greater (16). In a recent study by Vargas et al., PI-RADS v2 correctly diagnosed 94%–95% of clinically significant prostate cancers, detecting 118/125 cancers in the peripheral zone and 42/44 cancers in the transition zone (17). In that study, the authors used whole-mount pathology as the reference standard. In our study, PI-RADS v2 detected 48 of 56, or approximately 86%, of intermediate- and high-grade prostate cancers. The remaining eight cancers were not MRI evident. Despite only moderate interobserver agreement between the two readers in our study, the calculated PPV in the
detection of intermediate- and high-grade cancers for each PI-RADS category was very similar between readers 1 and 2. As PI-RADS score increased from 3 to 5, the PPV for clinically significant prostate cancer increased from 12% to 87%. These findings suggest that PI-RADS v2 is a useful system in predicting the likelihood of intermediate- and high-grade malignancy. In a recent study by NiMhurchu et al., the authors found similar PPV for overall PI-RADS scores of 3, 4, and 5 using the original PI-RADS scoring system (PI-RADS 3, 10.6%; PI-RADS 4, 44%; PI-RADS 5, 100%) (18). Our study has several limitations. First, the reference standards were systematic biopsy and TRUS-MRI fusiontargeted biopsy. In this retrospective study, some of the lesions detected on mpMRI by our readers were not identified by the original radiologist interpreting the study. As a result, targeted biopsy was not performed on all lesions identified by our readers, thus necessitating the need to correlate with systematic biopsy. Therefore, for these lesions, we cannot confidently exclude sampling error during biopsy. Also, though TRUS-MRI fusion-targeted biopsy has been shown to accurately sample lesions in the prostate, prostatectomy specimens are still considered the gold standard. The second limitation of our study is that we evaluated intermediate- and highgrade prostate cancers as these are often deemed clinically significant. However, this assumption is not a universally accepted consensus. The third limitation of this study was that high b-value DWI was not obtained as it was not a standard part of our imaging protocol until recently. High b-value DWI can sometimes improve detection of clinically significant cancers compared to ADC maps alone. The last limitation is that the two readers were radiologists at the same institution. Reproducibility of PI-RADS v2 was not assessed among radiologists from different institutions. In conclusion, our study shows that PI-RADS v2 has only moderate interobserver agreement, a similar finding in studies of the original PI-RADS and in initial studies of PI-RADS v2. Despite this, PI-RADS v2 appears to be a useful system 5
CHEN ET AL
to predict significant prostate cancer, with PI-RADS scores correlating well with the likelihood of intermediate- and highgrade cancers. REFERENCES 1. Barentsz JO, Richenberg J, Clements R, et al. ESUR prostate MR guidelines 2012. Eur Radiol 2012; 22:746–757. 2. Hamoen EH, de Rooij M, Witjes JA, et al. Use of the Prostate Imaging Reporting and Data System (PI-RADS) for prostate cancer detection with multiparametric magnetic resonance imaging: a diagnostic metaanalysis. Eur Urol 2015; 67:1112–1121. 3. Renard-Penna R, Mozer P, Cornud F, et al. Prostate imaging reporting and data system and Likert scoring system: multiparametric MR imaging validation study to screen patients for initial biopsy. Radiology 2015; 275:458–468. 4. Vache T, Bratan F, Mege-Lechevallier F, et al. Characterization of prostate lesions as benign or malignant at multiparametric MR imaging: comparison of three scoring systems in patients treated with radical prostatectomy. Radiology 2014; 272:446–455. 5. Thompson JE, van Leeuwen PJ, Moses D, et al. The diagnostic performance of multiparametric magnetic resonance imaging to detect significant prostate cancer. J Urol 2015; 195:1428–1435. 6. Baur AD, Maxeiner A, Franiel T, et al. Evaluation of the prostate imaging reporting and data system for the detection of prostate cancer by the results of targeted biopsy of the prostate. Invest Radiol 2014; 49:411– 420. 7. Junker D, Quentin M, Nagele U, et al. Evaluation of the PI-RADS scoring system for mpMRI of the prostate: a whole-mount step-section analysis. World J Urol 2015; 33:1023–1030. 8. Hansford BG, Peng Y, Jiang Y, et al. Dynamic contrast-enhanced MR imaging curve-type analysis: is it helpful in the differentiation of prostate cancer from healthy peripheral zone? Radiology 2015; 275:448– 457.
6
Academic Radiology, Vol ■, No ■■, ■■ 2017
9. American College of Radiology. PIRADS v2. Reston, VA: American College of Radiology, 2014. 10. Ukimura O, Desai MM, Palmer S, et al. 3-Dimensional elastic registration system of prostate biopsy location by real-time 3-dimensional transrectal ultrasound guidance with magnetic resonance/transrectal ultrasound image fusion. J Urol 2012; 187:1080–1086. 11. Baco E, Ukimura O, Rud E, et al. Magnetic resonance imaging-transrectal ultrasound image-fusion biopsies accurately characterize the index tumor: correlation with step-sectioned radical prostatectomy specimens in 135 patients. Eur Urol 2015; 67:787–794. 12. Vaché T, Bratan F, Mège-Lechevallier F, et al. Characterization of prostate lesions as benign or malignant at multiparametric MR imaging: comparison of three scoring systems in patients treated with radical prostatectomy. Radiology 2014; 272:446–455. 13. Schimmöller L, Quentin M, Arsov C, et al. Inter-reader agreement of the ESUR score for prostate MRI using in-bore MRI-guided biopsies as the reference standard. Eur Radiol 2013; 23:3185–3190. 14. Renard-Penna R, Mozer P, Cornud F, et al. Prostate imaging reporting and data system and Likert scoring system: multiparametric MR imaging validating study to screen patients for initial biopsy. Radiology 2015; 275:458–468. 15. Muller BG, Shih JH, Sankineni S, et al. Prostate cancer: interobserver agreement and accuracy with the revised prostate imaging reporting and data system at multiparametric MR imaging. Radiology 2015; 277:741– 750. 16. Rosenkrantz AB, Ginocchio LA, Cornfeld D, et al. Interobserver reproducibility of the PI-RADS version 2 lexicon: a multicenter study of six experienced prostate radiologists. Radiology 2016; 280:793–804. 17. Vargas HA, Hotker AM, Goldman DA, et al. Updated prostate imaging reporting and data system (PIRADS v2) recommendations for the detection of clinically significant prostate cancer using multiparametric MRI: critical evaluation using whole-mount pathology as standard of reference. Eur Radiol 2016; 26:1606–1612. 18. NiMhurchu E, O’Kelly F, Murphy IG, et al. Predictive value of PI-RADS classification in MRI-directed transrectal ultrasound guided prostate biopsy. Clin Radiol 2016; 71:375–380.
Original Investigation
Application of Prostate Imaging Reporting and Data System Version 2 (PI-RADS v2): Interobserver Agreement and Positive Predictive Value for Localization of Intermediate- and High-Grade Prostate Cancers on Multiparametric Magnetic Resonance Imaging Frank Chen, MD, Steven Cen, PhD, Suzanne Palmer, MD Rationale and Objectives: To evaluate interobserver agreement with the use of and the positive predictive value (PPV) of Prostate Imaging Reporting and Data System version 2 (PI-RADS v2) for the localization of intermediate- and high-grade prostate cancers on multiparametric magnetic resonance imaging (mpMRI). Materials and Methods: In this retrospective, institutional review board-approved study, 131 consecutive patients who had mpMRI followed by transrectal ultrasound-MR imaging fusion-guided biopsy of the prostate were included. Two readers who were blinded to initial mpMRI reports, clinical data, and pathologic outcomes reviewed the MR images, identified all prostate lesions, and scored each lesion based on the PI-RADS v2. Interobserver agreement was assessed by intraclass correlation coefficient (ICC), and PPV was calculated for each PI-RADS category. Results: PI-RADS v2 was found to have a moderate level of interobserver agreement between two readers of varying experience, with ICC of 0.74, 0.72, and 0.67 for all lesions, peripheral zone lesions, and transitional zone lesions, respectively. Despite only moderate interobserver agreement, the calculated PPV in the detection of intermediate- and high-grade prostate cancers for each PI-RADS category was very similar between the two readers, with approximate PPV of 0%, 12%, 64%, and 87% for PI-RADS categories 2, 3, 4, and 5, respectively. Conclusions: In our study, PI-RADS v2 has only moderate interobserver agreement, a similar finding in studies of the original PIRADS and in initial studies of PI-RADS v2. Despite this, PI-RADS v2 appears to be a useful system to predict significant prostate cancer, with PI-RADS scores correlating well with the likelihood of intermediate- and high-grade cancers. Key Words: Prostate cancer; multiparametric magnetic resonance imaging; PI-RADS; interobserver agreement; TRUS-MR imaging fusion biopsy. © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.
INTRODUCTION Acad Radiol 2017; ■:■■–■■ From the Keck School of Medicine of USC, 1500 San Pablo St, 2nd Floor Imaging, Los Angeles, CA 90033. Received November 12, 2016; revised January 13, 2017; accepted March 22, 2017. Address correspondence to: F.C. e-mail: [email protected] © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.acra.2017.03.019
I
n an attempt to standardize image interpretation and reporting of multiparametric magnetic resonance imaging (mpMRI) of the prostate, the European Society of Urogenital Radiology published the first Prostate Imaging Reporting and Data System (PI-RADS 1.0) in 2012. PI-RADS 1.0 is based on assessment of T2-weighted MRI, diffusion weighted imaging (DWI), and dynamic contrast enhanced-MRI (DCE-MRI) with 1
CHEN ET AL
or without MR spectroscopy for each lesion according to a 5-point scale and assigning a sum score ranging from 3 to 15 without MR spectroscopy (MRS) and from 4 to 20 with MRS (1). Several studies have validated the original PI-RADS in terms of accuracy; however, interobserver agreement was only moderate (2–6). Recent studies also suggest that assigning a score for T2-weighted imaging (T2WI) to transitional zone lesions and for DWI to peripheral zone lesion is adequate for stratification of patients for further diagnostic workup (6). On the other hand, MRS was rarely used in recent studies, and DCEMRI curve-type analysis does not seem to add significant value in the characterization of prostate lesions (6–8). Based on these new data, the PI-RADS steering committee of the American College of Radiology and the European Society of Urogenital Radiology prostate MRI working group revised PI-RADS and published PI-RADS version 2.0 in early 2015 (9). In this new version of PI-RADS, a primary determinant MRI sequence is used to evaluate each prostate lesion based on location. For peripheral zone lesions, DWI is the dominant sequence, and for transitional zone lesions, T2weighted sequence is the dominant sequence. On DCEMRI, results are scored as positive or negative based on the presence or absence of focal early enhancement, and the previously used curve-type analysis was abandoned. DCEMRI is used strictly for characterization of peripheral zone lesions and is applied only if it makes a clinically relevant difference in cases where a lesion is upgraded from PI-RADS 3 to 4. Finally, a final score of 1–5 is assigned to each prostate lesion based on the revised rules. In order to establish it as the standard for the evaluation of the prostate on mpMRI, studies are needed to evaluate interobserver agreement and test performance of PI-RADS v2. Thus, the purpose of our study is twofold. First, we want to evaluate interobserver agreement with the use of PIRADS v2. Second, we want to evaluate the positive predictive value (PPV) of each PI-RADS category in the detection of intermediate- and high-grade prostate cancers (tumors with Gleason score ≥7). MATERIALS AND METHODS Study Design and Patient Population
This retrospective study was institutional review board approved and Health Insurance Portability and Accountability Act compliant. Informed consent was waived by the review board. Between March 2013 and July 2015, 136 consecutive patients (mean age = 65.6) with abnormal prostate-specific antigen (PSA) (mean PSA = 7.6 µg/mL [1.3–47.4 µg/mL]), abnormal results on digital rectal examination, and/or on active surveillance underwent mpMRI followed by transrectal ultrasound-magnetic resonance (TRUS-MR) imaging fusionguided biopsy of the prostate within 2 months of prebiopsy MRI. Five patients were excluded from the study secondary to nondiagnostic mpMRI studies related to marked susceptibility artifact from hip prostheses. 2
Academic Radiology, Vol ■, No ■■, ■■ 2017
MR Imaging Protocol
All mpMRI examinations were performed on a 3-Tesla MRI system (GE Healthcare, WI, Milwaukee) using a multichannel phased-array coil. The MRI acquisition protocol included T2WI in axial, coronal, and sagittal planes; diffusionweighted MRI with b-values of 0, 600, and 1000 s/mm2; and T1-weighted dynamic contrast-enhanced images. Parametric apparent diffusion coefficient maps were calculated on the MRI console from the diffusion-weighted images (utilizing b-values of 0 and 600 s/mm2) on a voxelwise basis using a monoexponential model. For contrast media, gadobenate dimeglumine (MultiHance; Bracco Diagnostics, Cranbury, NJ) in a weight-adapted standard dose (0.1 mmol/kg) was injected intravenously at 2.5 cc/s. MR imaging sequence parameters are listed in Table 1. Prostate Biopsy
All prostate biopsies were performed by an experienced urologist who has been in practice for 20 years in the outpatient setting. All patients underwent TRUS-MR imaging fusionguided biopsy of MRI suspicious prostate lesions and 10–12 systematic core prostate biopsies. TRUS-MRI fusionguided biopsy was achieved with elastic image fusion and realtime 3D tracking technology (Urostation; Koelis, La Tranche, France) (10,11). Biopsy core specimens were immediately fixed in formalin and stained with hematoxylin and eosin, followed by routine histopathologic evaluation. MR Analysis
MR images were retrospectively and independently reviewed by two radiologists with 17 years (S.P., reader 1) and 4 years (F.C., reader 2) of experience in prostate imaging. The readers were blinded to initial mpMRI reports, clinical data, and pathologic outcomes. Each reader reviewed the MR images, identifying all lesions in each patient suspicious for clinically significant cancer and assigning a score for each lesion based on PI-RADS v2 classification. All assessment was made on a commercial PACS workstation (Synapse PACS; Fujifilm Medical Systems, Stamford, CT). During reader review of images, ovoid regions of interests were placed on the identified lesions on apparent diffusion coefficient (ADC) and T2WI, and MR images were saved in jpeg format in our research database. The saved MR images served as the basis for localization and pathologic correlation. Radiologic-Pathologic Correlation
Lesions identified on mpMRI by each reader were correlated with prostate biopsy results. Not all lesions identified by the readers were described in the initial radiology interpretation prior to biopsy. As a result, not all lesions underwent targeted TRUS-MR imaging fusion-guided biopsy. The MR lesions that did not undergo fusion-guided biopsy were matched
APPLICATION OF PI-RADS V2
7
– – –
to biopsy results based on location of systematic core biopsies and location of the lesion on MRI.
53 Zoom 20 6:07 160 × 160 4 15 Minimum full 3 DCE
DCE, dynamic contrast-enhanced; DWI, diffusion weighted imaging; FOV, field of view; T1WI, T1-weighted imaging; T2WI, T2-weighted imaging.
– – – 5:11 4:56 2:24 320 × 320 224 × 256 128 × 128 4 4 4 3427 500 3800 T2WI T1WI DWI
102 Minimum full Minimum full
180 180 180
Imaging Time Acquisition Matrix
20 20 20
4 1 1 (b value of 0) 12 (b value of 600) 1 (b value of 1000) 1
Zoom Zoom Zoom
Statistical Analysis
Slice Thickness (mm) Flip Angle (°) Echo Time (ms) Repetition Time (ms) Parameter
TABLE 1. Magnetic Resonance Imaging Sequence Parameters
FOV (cm)
NEX
Scan Mode
No. of Repetitions
Repetition Time (s)
Academic Radiology, Vol ■, No ■■, ■■ 2017
Lesions that were identified by both readers were used to evaluate interobserver agreement. The agreement for the PIRADS score between readers was assessed by intraclass correlation coefficient (ICC) with absolute agreement. Using all lesions identified by the readers, PPV with binomial 95% confidence interval (CI) for each PI-RADS score level in classifying intermediate- and high-grade prostate cancers was calculated for each reader. Statistical analysis was performed by a PhD-trained biostatistician (S.C.) who used SAS 9.4 software (SAS Institute, Cary, NC) for all data analysis. RESULTS Lesion Characteristics
Readers 1 and 2 identified PI-RADS 2 lesions related to benign prostatic hypertrophy in all study patients. Besides PIRADS 2 lesions, reader 1 described an additional 115 lesions (90 peripheral zone lesions and 25 transitional zone lesions). Of these 115 lesions, 70 were found to be prostate cancers on histopathology (29 lesions, Gleason score ≤6; 41 lesions, Gleason score ≥7) and the remainder to be benign. Reader 2 described an additional 109 lesions (90 peripheral zone lesions and 19 transitional zone lesions). Of these 109 lesions, 77 were found to be prostate cancers on histopathology (36 lesions, Gleason score ≤6; 41 lesions, Gleason score ≥7) and the remainder to be benign. A total of 8 biopsy-proven intermediateand high-grade prostate cancers (seven Gleason 7 lesions and one Gleason 9 lesion) had no MRI correlate (mean PSA = 8.5 µg/mL [4.2–14.8 µg/mL]). Prostate lesion characteristics are shown in Figure 1 flowchart. Interobserver Agreement
Besides PI-RADS 2 lesions, 73 lesions were described by both readers 1 and 2, 60 lesions in the peripheral zone (49 malignant lesions by histopathology), and 13 lesions in the transitional zone (six malignant lesions by histopathology). Of the 55 cancers, 18 lesions were assigned Gleason scores equal to or less than 6, and 37 lesions were assigned Gleason scores equal to or greater than 7. Among lesions identified by both readers, there was only a moderate level of interobserver agreement between readers 1 and 2 when classifying these lesions using the new PIRADS classification system with ICC of 0.74 (95% CI: 0.60, 0.83). Classification of peripheral zone lesions had a higher degree of interobserver agreement than transitional zone lesions. Figure 2 shows a representative peripheral zone lesion with interobserver variability, and Figure 3 shows a representative transitional zone lesion with interobserver variability. When intermediate- and high-grade prostate cancers were isolated, 3
Academic Radiology, Vol ■, No ■■, ■■ 2017
CHEN ET AL
Figure 1.
Flowchart of lesion characteristics.
(a)
(b)
(c)
Figure 2. Images in a 71-year-old man with prostate-specific antigen of 4.69 ng/mL. Readers identified a right peripheral zone lesion on (a) transverse T2-weighted image, (b) ADC map, and (c) DCE image. For this lesion, reader 1 assigned a PI-RADS score of 4, and reader 2 assigned a PI-RADS score of 3. On TRUS-MR imaging fusion-guided biopsy, the lesion was found to be a Gleason 7 tumor. ADC, apparent diffusion coefficient; DCE, dynamic contrast-enhanced; PI-RADS, Prostate Imaging Reporting and Data System; TRUS-MR, transrectal ultrasound-magnetic resonance.
TABLE 2. Interobserver Agreement—Intraclass Correlation Coefficient Based on Lesion Type Lesion Type All All PZ lesions All TZ lesions All intermediate- and high-grade lesions
Intraclass Correlation Coefficient 0.74 (95% CI: 0.60, 0.83) 0.72 (95% CI: 0.56, 0.83) 0.67 (95% CI: 0.22, 0.89) 0.60 (95% CI: 0.34, 0.78)
CI, confidence interval; PZ, peripheral zone; TZ, transitional zone.
the interobserver agreement decreased, with ICC of 0.60 (95% CI: 0.34, 0.78). Overall ICC values are listed in Table 2. Positive Predictive Value
The calculated PPV in the detection of intermediate- and highprostate cancers for each PI-RADS category was very similar between readers 1 and 2. For both readers, the PPV for PI4
TABLE 3. Positive Predictive Value for Each PI-RADS Score
Reader 1
Reader 2
PI-RADS Score
PPV (%)
2 3 4 5 2 3 4 5
0 13.0 (95% CI: 6.5, 23.8) 64.9 (95% CI: 47.4, 79.3) 88.9 (95% CI: 50.7, 99.4) 0 11.9 (95% CI: 5.3, 23.5) 62.2 (95% CI: 44.8, 77.1) 84.6 (95% CI: 53.7, 97.3)
CI, confidence interval; PI-RADS, Prostate Imaging Reporting and Data System; PPV, positive predictive value.
RADS 2 lesions is 0%, the PPV for PI-RADS 3 lesions is approximately 12% (95% CI: 5%, 24%), for PI-RADS 4 lesions is approximately 64% (95% CI: 45%, 80%), and for PIRADS 5 lesions is approximately 87% (95% CI: 50%, 100%). PPV for each PI-RADS category is listed in Table 3.
Academic Radiology, Vol ■, No ■■, ■■ 2017
(a)
APPLICATION OF PI-RADS V2
(b)
(c)
Figure 3. Images in a 64-year-old man with prostate-specific antigen of 5.9 ng/mL. Readers identified a right transitional zone lesion on (a) transverse T2-weighted image, (b) ADC map, and (c) DCE image. For this lesion, reader 1 assigned a PI-RADS score of 3, and reader 2 assigned a PI-RADS score of 4. On TRUS-MR imaging fusion-guided biopsy, the lesion was found to be a Gleason 7 tumor. ADC, apparent diffusion coefficient; DCE, dynamic contrast-enhanced; PI-RADS, Prostate Imaging Reporting and Data System; TRUS-MR, transrectal ultrasound-magnetic resonance.
DISCUSSION In our study, PI-RADS v2 was found to have only a moderate level of interobserver agreement for two readers of varying experience. The level of interobserver agreement is similar to levels reported for the original PI-RADS and initial studies of PI-RADS v2. Vaché et al., using whole-mount sections of prostatectomy specimens as the reference standard, showed that the original PI-RADS had poor-to-moderate interobserver agreement among three readers of varying experience with weight κ values of 0.378–0.441 (12). Schimmöller et al., using in-bore MRI-guided biopsy as the reference standard, reported a moderate-to-good level of interobserver agreement for the original PI-RADS among three readers with similar experience (T2WI, κ = 0.55; DWI, κ = 0.64; DCE-MRI, κ = 0.65) (13). Renard-Penna et al., using TRUS-MRI fusionguided biopsy as the reference standard, showed a good level of interobserver agreement for the original PI-RADS among two readers with κ = 0.73 (14). In an initial study of PIRADS v2, Muller et al. reported a moderate level of interobserver agreement among five readers of varying experience with a κ = 0.46. Authors of that study used TRUSMRI imaging fusion-guided biopsy as the reference standard (15). In a recently published study, Rosenkrantz et al. reported a moderate level of interobserver agreement among six readers at six different institutions with κ coefficients of 0.593 and 0.509 in the peripheral zone and transitional zone, respectively, for PI-RADS scores of 4 and greater (16). In a recent study by Vargas et al., PI-RADS v2 correctly diagnosed 94%–95% of clinically significant prostate cancers, detecting 118/125 cancers in the peripheral zone and 42/44 cancers in the transition zone (17). In that study, the authors used whole-mount pathology as the reference standard. In our study, PI-RADS v2 detected 48 of 56, or approximately 86%, of intermediate- and high-grade prostate cancers. The remaining eight cancers were not MRI evident. Despite only moderate interobserver agreement between the two readers in our study, the calculated PPV in the
detection of intermediate- and high-grade cancers for each PI-RADS category was very similar between readers 1 and 2. As PI-RADS score increased from 3 to 5, the PPV for clinically significant prostate cancer increased from 12% to 87%. These findings suggest that PI-RADS v2 is a useful system in predicting the likelihood of intermediate- and high-grade malignancy. In a recent study by NiMhurchu et al., the authors found similar PPV for overall PI-RADS scores of 3, 4, and 5 using the original PI-RADS scoring system (PI-RADS 3, 10.6%; PI-RADS 4, 44%; PI-RADS 5, 100%) (18). Our study has several limitations. First, the reference standards were systematic biopsy and TRUS-MRI fusiontargeted biopsy. In this retrospective study, some of the lesions detected on mpMRI by our readers were not identified by the original radiologist interpreting the study. As a result, targeted biopsy was not performed on all lesions identified by our readers, thus necessitating the need to correlate with systematic biopsy. Therefore, for these lesions, we cannot confidently exclude sampling error during biopsy. Also, though TRUS-MRI fusion-targeted biopsy has been shown to accurately sample lesions in the prostate, prostatectomy specimens are still considered the gold standard. The second limitation of our study is that we evaluated intermediate- and highgrade prostate cancers as these are often deemed clinically significant. However, this assumption is not a universally accepted consensus. The third limitation of this study was that high b-value DWI was not obtained as it was not a standard part of our imaging protocol until recently. High b-value DWI can sometimes improve detection of clinically significant cancers compared to ADC maps alone. The last limitation is that the two readers were radiologists at the same institution. Reproducibility of PI-RADS v2 was not assessed among radiologists from different institutions. In conclusion, our study shows that PI-RADS v2 has only moderate interobserver agreement, a similar finding in studies of the original PI-RADS and in initial studies of PI-RADS v2. Despite this, PI-RADS v2 appears to be a useful system 5
CHEN ET AL
to predict significant prostate cancer, with PI-RADS scores correlating well with the likelihood of intermediate- and highgrade cancers. REFERENCES 1. Barentsz JO, Richenberg J, Clements R, et al. ESUR prostate MR guidelines 2012. Eur Radiol 2012; 22:746–757. 2. Hamoen EH, de Rooij M, Witjes JA, et al. Use of the Prostate Imaging Reporting and Data System (PI-RADS) for prostate cancer detection with multiparametric magnetic resonance imaging: a diagnostic metaanalysis. Eur Urol 2015; 67:1112–1121. 3. Renard-Penna R, Mozer P, Cornud F, et al. Prostate imaging reporting and data system and Likert scoring system: multiparametric MR imaging validation study to screen patients for initial biopsy. Radiology 2015; 275:458–468. 4. Vache T, Bratan F, Mege-Lechevallier F, et al. Characterization of prostate lesions as benign or malignant at multiparametric MR imaging: comparison of three scoring systems in patients treated with radical prostatectomy. Radiology 2014; 272:446–455. 5. Thompson JE, van Leeuwen PJ, Moses D, et al. The diagnostic performance of multiparametric magnetic resonance imaging to detect significant prostate cancer. J Urol 2015; 195:1428–1435. 6. Baur AD, Maxeiner A, Franiel T, et al. Evaluation of the prostate imaging reporting and data system for the detection of prostate cancer by the results of targeted biopsy of the prostate. Invest Radiol 2014; 49:411– 420. 7. Junker D, Quentin M, Nagele U, et al. Evaluation of the PI-RADS scoring system for mpMRI of the prostate: a whole-mount step-section analysis. World J Urol 2015; 33:1023–1030. 8. Hansford BG, Peng Y, Jiang Y, et al. Dynamic contrast-enhanced MR imaging curve-type analysis: is it helpful in the differentiation of prostate cancer from healthy peripheral zone? Radiology 2015; 275:448– 457.
6
Academic Radiology, Vol ■, No ■■, ■■ 2017
9. American College of Radiology. PIRADS v2. Reston, VA: American College of Radiology, 2014. 10. Ukimura O, Desai MM, Palmer S, et al. 3-Dimensional elastic registration system of prostate biopsy location by real-time 3-dimensional transrectal ultrasound guidance with magnetic resonance/transrectal ultrasound image fusion. J Urol 2012; 187:1080–1086. 11. Baco E, Ukimura O, Rud E, et al. Magnetic resonance imaging-transrectal ultrasound image-fusion biopsies accurately characterize the index tumor: correlation with step-sectioned radical prostatectomy specimens in 135 patients. Eur Urol 2015; 67:787–794. 12. Vaché T, Bratan F, Mège-Lechevallier F, et al. Characterization of prostate lesions as benign or malignant at multiparametric MR imaging: comparison of three scoring systems in patients treated with radical prostatectomy. Radiology 2014; 272:446–455. 13. Schimmöller L, Quentin M, Arsov C, et al. Inter-reader agreement of the ESUR score for prostate MRI using in-bore MRI-guided biopsies as the reference standard. Eur Radiol 2013; 23:3185–3190. 14. Renard-Penna R, Mozer P, Cornud F, et al. Prostate imaging reporting and data system and Likert scoring system: multiparametric MR imaging validating study to screen patients for initial biopsy. Radiology 2015; 275:458–468. 15. Muller BG, Shih JH, Sankineni S, et al. Prostate cancer: interobserver agreement and accuracy with the revised prostate imaging reporting and data system at multiparametric MR imaging. Radiology 2015; 277:741– 750. 16. Rosenkrantz AB, Ginocchio LA, Cornfeld D, et al. Interobserver reproducibility of the PI-RADS version 2 lexicon: a multicenter study of six experienced prostate radiologists. Radiology 2016; 280:793–804. 17. Vargas HA, Hotker AM, Goldman DA, et al. Updated prostate imaging reporting and data system (PIRADS v2) recommendations for the detection of clinically significant prostate cancer using multiparametric MRI: critical evaluation using whole-mount pathology as standard of reference. Eur Radiol 2016; 26:1606–1612. 18. NiMhurchu E, O’Kelly F, Murphy IG, et al. Predictive value of PI-RADS classification in MRI-directed transrectal ultrasound guided prostate biopsy. Clin Radiol 2016; 71:375–380.